2023-07-14T15:37:53.005 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T15:37:53.006 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --verbose --name rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} -- /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/orig.config.yaml 2023-07-14T15:37:53.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T15:37:53.146 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606 branch: wip-rishabh-2023Jul13 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7337606' kernel: flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: aa6a80e6cc655fff48452a7eca02070e61d074fc timestamp: 2023-07-14_10:26:42 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-14T15:37:53.147 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa; will attempt to use it 2023-07-14T15:37:53.147 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks 2023-07-14T15:37:53.148 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-14T15:37:53.150 INFO:teuthology.task.internal:Checking packages... 2023-07-14T15:37:53.178 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '3fc0089cc1082313a98853d48b892acdc1385b8b' 2023-07-14T15:37:53.178 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-14T15:37:53.179 INFO:teuthology.packaging:ref: None 2023-07-14T15:37:53.179 INFO:teuthology.packaging:tag: None 2023-07-14T15:37:53.179 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13 2023-07-14T15:37:53.179 INFO:teuthology.packaging:sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T15:37:53.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13 2023-07-14T15:37:53.506 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4757.g3fc0089c 2023-07-14T15:37:53.507 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-14T15:37:53.513 INFO:teuthology.task.internal:no buildpackages task found 2023-07-14T15:37:53.513 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-14T15:37:53.534 INFO:teuthology.task.internal:Saving configuration 2023-07-14T15:37:53.552 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-14T15:37:53.561 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T15:37:53.590 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 15:31:12.468236', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T15:37:53.616 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 15:31:12.469664', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T15:37:53.640 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 15:31:12.470699', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T15:37:53.641 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-14T15:37:53.646 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-14T15:37:53.647 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-14T15:37:53.647 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-14T15:37:53.647 INFO:teuthology.run_tasks:Running task console_log... 2023-07-14T15:37:53.763 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdbb7fec4c0>, signals=[15]) 2023-07-14T15:37:53.764 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-14T15:37:53.770 INFO:teuthology.task.internal:Opening connections... 2023-07-14T15:37:53.771 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2023-07-14T15:37:53.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:37:53.841 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2023-07-14T15:37:53.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:37:53.908 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2023-07-14T15:37:53.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:38:08.985 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-14T15:38:08.993 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-07-14T15:38:09.015 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-07-14T15:38:09.015 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-07-14T15:38:09.074 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T15:38:09.075 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T15:38:09.075 INFO:teuthology.orchestra.run.smithi062.stdout:ID="rhel" 2023-07-14T15:38:09.075 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="fedora" 2023-07-14T15:38:09.075 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8.6" 2023-07-14T15:38:09.075 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2023-07-14T15:38:09.075 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T15:38:09.076 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-07-14T15:38:09.076 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T15:38:09.076 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T15:38:09.076 INFO:teuthology.orchestra.run.smithi062.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T15:38:09.076 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T15:38:09.076 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T15:38:09.077 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T15:38:09.077 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T15:38:09.077 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T15:38:09.077 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T15:38:09.078 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-07-14T15:38:14.111 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2023-07-14T15:38:14.132 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2023-07-14T15:38:14.132 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2023-07-14T15:38:14.191 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T15:38:14.191 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T15:38:14.192 INFO:teuthology.orchestra.run.smithi081.stdout:ID="rhel" 2023-07-14T15:38:14.192 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="fedora" 2023-07-14T15:38:14.192 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="8.6" 2023-07-14T15:38:14.192 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el8" 2023-07-14T15:38:14.192 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T15:38:14.192 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2023-07-14T15:38:14.193 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T15:38:14.193 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T15:38:14.193 INFO:teuthology.orchestra.run.smithi081.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T15:38:14.193 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T15:38:14.193 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T15:38:14.193 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T15:38:14.193 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T15:38:14.193 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T15:38:14.194 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T15:38:14.194 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2023-07-14T15:38:16.267 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2023-07-14T15:38:16.288 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2023-07-14T15:38:16.289 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2023-07-14T15:38:16.348 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T15:38:16.348 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T15:38:16.349 INFO:teuthology.orchestra.run.smithi164.stdout:ID="rhel" 2023-07-14T15:38:16.349 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="fedora" 2023-07-14T15:38:16.349 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8.6" 2023-07-14T15:38:16.349 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2023-07-14T15:38:16.349 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T15:38:16.349 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2023-07-14T15:38:16.349 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T15:38:16.350 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T15:38:16.350 INFO:teuthology.orchestra.run.smithi164.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T15:38:16.350 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T15:38:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T15:38:16.350 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T15:38:16.350 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T15:38:16.350 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T15:38:16.350 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T15:38:16.351 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2023-07-14T15:38:16.376 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-14T15:38:16.388 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-14T15:38:16.402 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-14T15:38:16.402 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2023-07-14T15:38:16.405 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2023-07-14T15:38:16.407 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2023-07-14T15:38:16.425 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-14T15:38:16.431 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-14T15:38:16.431 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2023-07-14T15:38:16.464 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2023-07-14T15:38:16.466 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2023-07-14T15:38:16.497 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-14T15:38:21.600 INFO:teuthology.run_tasks:Running task kernel... 2023-07-14T15:38:21.622 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-14T15:38:21.622 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'}}, timeout 300 2023-07-14T15:38:21.623 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T15:38:21.624 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T15:38:21.624 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T15:38:21.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T15:38:21.644 DEBUG:teuthology.task.kernel:sha1 for mon.b is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.644 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-14T15:38:21.644 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2023-07-14T15:38:21.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T15:38:21.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T15:38:21.646 DEBUG:teuthology.task.kernel:sha1 for mon.a is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.646 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-14T15:38:21.646 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-07-14T15:38:21.647 DEBUG:teuthology.task.kernel:sha1 for mon.c is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.647 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-14T15:38:21.647 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2023-07-14T15:38:21.703 INFO:teuthology.orchestra.run.smithi081.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-14T15:38:21.703 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.704 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-14T15:38:21.704 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.710 INFO:teuthology.orchestra.run.smithi164.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-14T15:38:21.710 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-14T15:38:21.710 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.710 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-14T15:38:21.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.712 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.712 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-14T15:38:21.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:21.977 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum upgrade -y linux-firmware 2023-07-14T15:38:22.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum upgrade -y linux-firmware 2023-07-14T15:38:22.019 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum upgrade -y linux-firmware 2023-07-14T15:38:23.243 INFO:teuthology.orchestra.run.smithi081.stdout:Updating Subscription Management repositories. 2023-07-14T15:38:23.243 INFO:teuthology.orchestra.run.smithi081.stdout:Unable to read consumer identity 2023-07-14T15:38:23.288 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2023-07-14T15:38:23.289 INFO:teuthology.orchestra.run.smithi164.stdout:Unable to read consumer identity 2023-07-14T15:38:23.299 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-07-14T15:38:23.299 INFO:teuthology.orchestra.run.smithi062.stdout:Unable to read consumer identity 2023-07-14T15:38:24.759 INFO:teuthology.orchestra.run.smithi081.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-07-14T15:38:24.818 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-07-14T15:38:24.850 INFO:teuthology.orchestra.run.smithi164.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-07-14T15:38:32.759 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 2.0 MB/s | 16 MB 00:07 2023-07-14T15:38:36.392 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 426 kB/s | 24 kB 00:00 2023-07-14T15:38:37.111 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-07-14T15:38:37.112 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-07-14T15:38:37.112 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-07-14T15:38:37.240 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.a... 2023-07-14T15:38:37.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:37.628 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-14T15:38:37.628 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-14T15:38:37.631 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi062.front.sepia.ceph.com to complete... 2023-07-14T15:38:39.990 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:15 2023-07-14T15:38:41.922 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 937 kB/s | 16 MB 00:17 2023-07-14T15:38:43.456 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 513 kB/s | 24 kB 00:00 2023-07-14T15:38:44.171 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2023-07-14T15:38:44.172 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2023-07-14T15:38:44.172 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2023-07-14T15:38:44.293 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.b... 2023-07-14T15:38:44.294 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:44.596 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-14T15:38:44.597 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-14T15:38:44.599 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi081.front.sepia.ceph.com to complete... 2023-07-14T15:38:45.364 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-07-14T15:38:46.119 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2023-07-14T15:38:46.120 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2023-07-14T15:38:46.121 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2023-07-14T15:38:46.240 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.c... 2023-07-14T15:38:46.240 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-14T15:38:46.559 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-14T15:38:46.559 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-14T15:38:46.561 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi164.front.sepia.ceph.com to complete... 2023-07-14T15:41:37.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14 15:41:37 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-14T15:41:37.591 INFO:teuthology.orchestra.run.smithi081.stderr:FINISHED --2023-07-14 15:41:37-- 2023-07-14T15:41:37.591 INFO:teuthology.orchestra.run.smithi081.stderr:Total wall clock time: 2m 53s 2023-07-14T15:41:37.591 INFO:teuthology.orchestra.run.smithi081.stderr:Downloaded: 1 files, 363M in 2m 53s (2.10 MB/s) 2023-07-14T15:41:37.593 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.b... 2023-07-14T15:41:37.593 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-14T15:41:39.196 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying... ######################################## 2023-07-14T15:41:39.275 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing... ######################################## 2023-07-14T15:41:39.371 INFO:teuthology.orchestra.run.smithi081.stdout:Updating / installing... 2023-07-14T15:42:17.959 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-14T15:42:32.660 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14 15:42:32 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-14T15:42:32.661 INFO:teuthology.orchestra.run.smithi164.stderr:FINISHED --2023-07-14 15:42:32-- 2023-07-14T15:42:32.661 INFO:teuthology.orchestra.run.smithi164.stderr:Total wall clock time: 3m 46s 2023-07-14T15:42:32.661 INFO:teuthology.orchestra.run.smithi164.stderr:Downloaded: 1 files, 363M in 3m 46s (1.61 MB/s) 2023-07-14T15:42:32.662 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.c... 2023-07-14T15:42:32.663 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-14T15:42:34.272 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying... ######################################## 2023-07-14T15:42:34.350 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing... ######################################## 2023-07-14T15:42:34.444 INFO:teuthology.orchestra.run.smithi164.stdout:Updating / installing... 2023-07-14T15:42:42.999 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-14T15:42:42.999 DEBUG:teuthology.orchestra.run.smithi081:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-14T15:42:43.056 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.056 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.056 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.056 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.056 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-14T15:42:43.057 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-14T15:42:43.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-14T15:42:43.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-14T15:42:43.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-14T15:42:43.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-14T15:42:43.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-14T15:42:43.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-14T15:42:43.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-14T15:42:43.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-14T15:42:43.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-14T15:42:43.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-14T15:42:43.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-14T15:42:43.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-14T15:42:43.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-14T15:42:43.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-14T15:42:43.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-14T15:42:43.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-14T15:42:43.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-14T15:42:43.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-14T15:42:43.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-14T15:42:43.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-14T15:42:43.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-14T15:42:43.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-14T15:42:43.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-14T15:42:43.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-14T15:42:43.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-14T15:42:43.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-14T15:42:43.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-14T15:42:43.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-14T15:42:43.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-14T15:42:43.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-14T15:42:43.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-14T15:42:43.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-14T15:42:43.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-14T15:42:43.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-14T15:42:43.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-14T15:42:43.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-14T15:42:43.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-14T15:42:43.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-14T15:42:43.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-14T15:42:43.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-14T15:42:43.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-14T15:42:43.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-14T15:42:43.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-14T15:42:43.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-14T15:42:43.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-14T15:42:43.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-14T15:42:43.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-14T15:42:43.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-14T15:42:43.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-14T15:42:43.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-14T15:42:43.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-14T15:42:43.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-14T15:42:43.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-14T15:42:43.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-14T15:42:43.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-14T15:42:43.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-14T15:42:43.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-14T15:42:43.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-14T15:42:43.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-14T15:42:43.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-14T15:42:43.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-14T15:42:43.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-14T15:42:43.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-14T15:42:43.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-14T15:42:43.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-14T15:42:43.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-14T15:42:43.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-14T15:42:43.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-14T15:42:43.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-14T15:42:43.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-14T15:42:43.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-14T15:42:43.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-14T15:42:43.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-14T15:42:43.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-14T15:42:43.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-14T15:42:43.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-14T15:42:43.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-14T15:42:43.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-14T15:42:43.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-14T15:42:43.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-14T15:42:43.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-14T15:42:43.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-14T15:42:43.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-14T15:42:43.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-14T15:42:43.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-14T15:42:43.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-14T15:42:43.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-14T15:42:43.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-14T15:42:43.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-14T15:42:43.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-14T15:42:43.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-14T15:42:43.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-14T15:42:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-14T15:42:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-14T15:42:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-14T15:42:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-14T15:42:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-14T15:42:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-14T15:42:43.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-14T15:42:43.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-14T15:42:43.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-14T15:42:43.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-14T15:42:43.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-14T15:42:43.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-14T15:42:43.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-14T15:42:43.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-14T15:42:43.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-14T15:42:43.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-14T15:42:43.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-14T15:42:43.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-14T15:42:43.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-14T15:42:43.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-14T15:42:43.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-14T15:42:43.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-14T15:42:43.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-14T15:42:43.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-14T15:42:43.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-14T15:42:43.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-14T15:42:43.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-14T15:42:43.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-14T15:42:43.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-14T15:42:43.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-14T15:42:43.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-14T15:42:43.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-14T15:42:43.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-14T15:42:43.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-14T15:42:43.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-14T15:42:43.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-14T15:42:43.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-14T15:42:43.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-14T15:42:43.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-14T15:42:43.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-14T15:42:43.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-14T15:42:43.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-14T15:42:43.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-14T15:42:43.086 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-14T15:42:43.086 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-14T15:42:43.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-14T15:42:43.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-14T15:42:43.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-14T15:42:43.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-14T15:42:43.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-14T15:42:43.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-14T15:42:43.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-14T15:42:43.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-14T15:42:43.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-14T15:42:43.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-14T15:42:43.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-14T15:42:43.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-14T15:42:43.090 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-14T15:42:43.090 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-14T15:42:43.090 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-14T15:42:43.091 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-14T15:42:43.091 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-14T15:42:43.091 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-14T15:42:43.092 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-14T15:42:43.092 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-14T15:42:43.092 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-14T15:42:43.093 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-14T15:42:43.093 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-14T15:42:43.093 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-14T15:42:43.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-14T15:42:43.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-14T15:42:43.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-14T15:42:43.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-14T15:42:43.095 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-14T15:42:43.095 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-14T15:42:43.095 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-14T15:42:43.096 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-14T15:42:43.096 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-14T15:42:43.096 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-14T15:42:43.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-14T15:42:43.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-14T15:42:43.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-14T15:42:43.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-14T15:42:43.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-14T15:42:43.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-14T15:42:43.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-14T15:42:43.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-14T15:42:43.099 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-14T15:42:43.099 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-14T15:42:43.099 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-14T15:42:43.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-14T15:42:43.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-14T15:42:43.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-14T15:42:43.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-14T15:42:43.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-14T15:42:43.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-14T15:42:43.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-14T15:42:43.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-14T15:42:43.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-14T15:42:43.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-14T15:42:43.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-14T15:42:43.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-14T15:42:43.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-14T15:42:43.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-14T15:42:43.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-14T15:42:43.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-14T15:42:43.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-14T15:42:43.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-14T15:42:43.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-14T15:42:43.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-14T15:42:43.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-14T15:42:43.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-14T15:42:43.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-14T15:42:43.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-14T15:42:43.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-14T15:42:43.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-14T15:42:43.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-14T15:42:43.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-14T15:42:43.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-14T15:42:43.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-14T15:42:43.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-14T15:42:43.108 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-14T15:42:43.108 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-14T15:42:43.108 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-14T15:42:43.109 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-14T15:42:43.109 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-14T15:42:43.109 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-14T15:42:43.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-14T15:42:43.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-14T15:42:43.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-14T15:42:43.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-14T15:42:43.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-14T15:42:43.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-14T15:42:43.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-14T15:42:43.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-14T15:42:43.112 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-14T15:42:43.112 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-14T15:42:43.112 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-14T15:42:43.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-14T15:42:43.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-14T15:42:43.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-14T15:42:43.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-14T15:42:43.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-14T15:42:43.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-14T15:42:43.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-14T15:42:43.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-14T15:42:43.115 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-14T15:42:43.115 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-14T15:42:43.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-14T15:42:43.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-14T15:42:43.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-14T15:42:43.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-14T15:42:43.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-14T15:42:43.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-14T15:42:43.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-14T15:42:43.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-14T15:42:43.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-14T15:42:43.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-14T15:42:43.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-14T15:42:43.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-14T15:42:43.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-14T15:42:43.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-14T15:42:43.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-14T15:42:43.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-14T15:42:43.120 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-14T15:42:43.120 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-14T15:42:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-14T15:42:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-14T15:42:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-14T15:42:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-14T15:42:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-14T15:42:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-14T15:42:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-14T15:42:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-14T15:42:43.123 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-14T15:42:43.123 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-14T15:42:43.123 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-14T15:42:43.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-14T15:42:43.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-14T15:42:43.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-14T15:42:43.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-14T15:42:43.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-14T15:42:43.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-14T15:42:43.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-14T15:42:43.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-14T15:42:43.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-14T15:42:43.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-14T15:42:43.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-14T15:42:43.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-14T15:42:43.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-14T15:42:43.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-14T15:42:43.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-14T15:42:43.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-14T15:42:43.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-14T15:42:43.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-14T15:42:43.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-14T15:42:43.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-14T15:42:43.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-14T15:42:43.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-14T15:42:43.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-14T15:42:43.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-14T15:42:43.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-14T15:42:43.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-14T15:42:43.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-14T15:42:43.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-14T15:42:43.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-14T15:42:43.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-14T15:42:43.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-14T15:42:43.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-14T15:42:43.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-14T15:42:43.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-14T15:42:43.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-14T15:42:43.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-14T15:42:43.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-14T15:42:43.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-14T15:42:43.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-14T15:42:43.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-14T15:42:43.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-14T15:42:43.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-14T15:42:43.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-14T15:42:43.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-14T15:42:43.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-14T15:42:43.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-14T15:42:43.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-14T15:42:43.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-14T15:42:43.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-14T15:42:43.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-14T15:42:43.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-14T15:42:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-14T15:42:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-14T15:42:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-14T15:42:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-14T15:42:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-14T15:42:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-14T15:42:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-14T15:42:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-14T15:42:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-14T15:42:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-14T15:42:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-14T15:42:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-14T15:42:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-14T15:42:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-14T15:42:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-14T15:42:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-14T15:42:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-14T15:42:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-14T15:42:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-14T15:42:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-14T15:42:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-14T15:42:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-14T15:42:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-14T15:42:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-14T15:42:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-14T15:42:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-14T15:42:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-14T15:42:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-14T15:42:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-14T15:42:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-14T15:42:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-14T15:42:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-14T15:42:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-14T15:42:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-14T15:42:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-14T15:42:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-14T15:42:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-14T15:42:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-14T15:42:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-14T15:42:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-14T15:42:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-14T15:42:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-14T15:42:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-14T15:42:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-14T15:42:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-14T15:42:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-14T15:42:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-14T15:42:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-14T15:42:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-14T15:42:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-14T15:42:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-14T15:42:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-14T15:42:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-14T15:42:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-14T15:42:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-14T15:42:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-14T15:42:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-14T15:42:43.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-14T15:42:43.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-14T15:42:43.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-14T15:42:43.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-14T15:42:43.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-14T15:42:43.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-14T15:42:43.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-14T15:42:43.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-14T15:42:43.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-14T15:42:43.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-14T15:42:43.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-14T15:42:43.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-14T15:42:43.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-14T15:42:43.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-14T15:42:43.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-14T15:42:43.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-14T15:42:43.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-14T15:42:43.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-14T15:42:43.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-14T15:42:43.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-14T15:42:43.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-14T15:42:43.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-14T15:42:43.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-14T15:42:43.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-14T15:42:43.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-14T15:42:43.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-14T15:42:43.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-14T15:42:43.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-14T15:42:43.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-14T15:42:43.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-14T15:42:43.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-14T15:42:43.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-14T15:42:43.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-14T15:42:43.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-14T15:42:43.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-14T15:42:43.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-14T15:42:43.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-14T15:42:43.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-14T15:42:43.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-14T15:42:43.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-14T15:42:43.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-14T15:42:43.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-14T15:42:43.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-14T15:42:43.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-14T15:42:43.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-14T15:42:43.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-14T15:42:43.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-14T15:42:43.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-14T15:42:43.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-14T15:42:43.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-14T15:42:43.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-14T15:42:43.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-14T15:42:43.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-14T15:42:43.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-14T15:42:43.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-14T15:42:43.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-14T15:42:43.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-14T15:42:43.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-14T15:42:43.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-14T15:42:43.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-14T15:42:43.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-14T15:42:43.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-14T15:42:43.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-14T15:42:43.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-14T15:42:43.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-14T15:42:43.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-14T15:42:43.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-14T15:42:43.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-14T15:42:43.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-14T15:42:43.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-14T15:42:43.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-14T15:42:43.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-14T15:42:43.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-14T15:42:43.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-14T15:42:43.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-14T15:42:43.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-14T15:42:43.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-14T15:42:43.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-14T15:42:43.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-14T15:42:43.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-14T15:42:43.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-14T15:42:43.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-14T15:42:43.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-14T15:42:43.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-14T15:42:43.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-14T15:42:43.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-14T15:42:43.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-14T15:42:43.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-14T15:42:43.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-14T15:42:43.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-14T15:42:43.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-14T15:42:43.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-14T15:42:43.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-14T15:42:43.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-14T15:42:43.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-14T15:42:43.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-14T15:42:43.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-14T15:42:43.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-14T15:42:43.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-14T15:42:43.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-14T15:42:43.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-14T15:42:43.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-14T15:42:43.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-14T15:42:43.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-14T15:42:43.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-14T15:42:43.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-14T15:42:43.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-14T15:42:43.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-14T15:42:43.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-14T15:42:43.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-14T15:42:43.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-14T15:42:43.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-14T15:42:43.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-14T15:42:43.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-14T15:42:43.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-14T15:42:43.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-14T15:42:43.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-14T15:42:43.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-14T15:42:43.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-14T15:42:43.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-14T15:42:43.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-14T15:42:43.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-14T15:42:43.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-14T15:42:43.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-14T15:42:43.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-14T15:42:43.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-14T15:42:43.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-14T15:42:43.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-14T15:42:43.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-14T15:42:43.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-14T15:42:43.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-14T15:42:43.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-14T15:42:43.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-14T15:42:43.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-14T15:42:43.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-14T15:42:43.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-14T15:42:43.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-14T15:42:43.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-14T15:42:43.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-14T15:42:43.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-14T15:42:43.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-14T15:42:43.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-14T15:42:43.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-14T15:42:43.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-14T15:42:43.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-14T15:42:43.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-14T15:42:43.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-14T15:42:43.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-14T15:42:43.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-14T15:42:43.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-14T15:42:43.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-14T15:42:43.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-14T15:42:43.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-14T15:42:43.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-14T15:42:43.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-14T15:42:43.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-14T15:42:43.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-14T15:42:43.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-14T15:42:43.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-14T15:42:43.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-14T15:42:43.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-14T15:42:43.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-14T15:42:43.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-14T15:42:43.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-14T15:42:43.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-14T15:42:43.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-14T15:42:43.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-14T15:42:43.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-14T15:42:43.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-14T15:42:43.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-14T15:42:43.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-14T15:42:43.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-14T15:42:43.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-14T15:42:43.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-14T15:42:43.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-14T15:42:43.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-14T15:42:43.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-14T15:42:43.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-14T15:42:43.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-14T15:42:43.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-14T15:42:43.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-14T15:42:43.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-14T15:42:43.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-14T15:42:43.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-14T15:42:43.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-14T15:42:43.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-14T15:42:43.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-14T15:42:43.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-14T15:42:43.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-14T15:42:43.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-14T15:42:43.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-14T15:42:43.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-14T15:42:43.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-14T15:42:43.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-14T15:42:43.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-14T15:42:43.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-14T15:42:43.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-14T15:42:43.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-14T15:42:43.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-14T15:42:43.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-14T15:42:43.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-14T15:42:43.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-14T15:42:43.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-14T15:42:43.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-14T15:42:43.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-14T15:42:43.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-14T15:42:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-14T15:42:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-14T15:42:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-14T15:42:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-14T15:42:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-14T15:42:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-14T15:42:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-14T15:42:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-14T15:42:43.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-14T15:42:43.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-14T15:42:43.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-14T15:42:43.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-14T15:42:43.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-14T15:42:43.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-14T15:42:43.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-14T15:42:43.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-14T15:42:43.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-14T15:42:43.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-14T15:42:43.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-14T15:42:43.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-14T15:42:43.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-14T15:42:43.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-14T15:42:43.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-14T15:42:43.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-14T15:42:43.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-14T15:42:43.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-14T15:42:43.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-14T15:42:43.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-14T15:42:43.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-14T15:42:43.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-14T15:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-14T15:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-14T15:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-14T15:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-14T15:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-14T15:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-14T15:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-14T15:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-14T15:42:43.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-14T15:42:43.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-14T15:42:43.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-14T15:42:43.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-14T15:42:43.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-14T15:42:43.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-14T15:42:43.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-14T15:42:43.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-14T15:42:43.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-14T15:42:43.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-14T15:42:43.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-14T15:42:43.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-14T15:42:43.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-14T15:42:43.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-14T15:42:43.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-14T15:42:43.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-14T15:42:43.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-14T15:42:43.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-14T15:42:43.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-14T15:42:43.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-14T15:42:43.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-14T15:42:43.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-14T15:42:43.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-14T15:42:43.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-14T15:42:43.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-14T15:42:43.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-14T15:42:43.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-14T15:42:43.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-14T15:42:43.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-14T15:42:43.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-14T15:42:43.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-14T15:42:43.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-14T15:42:43.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-14T15:42:43.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-14T15:42:43.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-14T15:42:43.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-14T15:42:43.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-14T15:42:43.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-14T15:42:43.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-14T15:42:43.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-14T15:42:43.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-14T15:42:43.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-14T15:42:43.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-14T15:42:43.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-14T15:42:43.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-14T15:42:43.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-14T15:42:43.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-14T15:42:43.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-14T15:42:43.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-14T15:42:43.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-14T15:42:43.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-14T15:42:43.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-14T15:42:43.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-14T15:42:43.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-14T15:42:43.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-14T15:42:43.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-14T15:42:43.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-14T15:42:43.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-14T15:42:43.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-14T15:42:43.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-14T15:42:43.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-14T15:42:43.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-14T15:42:43.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-14T15:42:43.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-14T15:42:43.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-14T15:42:43.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-14T15:42:43.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-14T15:42:43.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-14T15:42:43.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-14T15:42:43.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-14T15:42:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-14T15:42:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-14T15:42:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-14T15:42:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-14T15:42:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-14T15:42:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-14T15:42:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-14T15:42:43.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-14T15:42:43.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-14T15:42:43.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-14T15:42:43.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-14T15:42:43.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-14T15:42:43.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-14T15:42:43.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-14T15:42:43.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-14T15:42:43.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-14T15:42:43.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-14T15:42:43.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-14T15:42:43.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-14T15:42:43.186 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-14T15:42:43.186 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-14T15:42:43.186 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-14T15:42:43.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-14T15:42:43.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-14T15:42:43.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-14T15:42:43.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-14T15:42:43.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-14T15:42:43.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-14T15:42:43.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-14T15:42:43.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-14T15:42:43.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-14T15:42:43.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-14T15:42:43.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-14T15:42:43.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-14T15:42:43.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-14T15:42:43.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-14T15:42:43.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-14T15:42:43.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-14T15:42:43.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-14T15:42:43.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-14T15:42:43.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-14T15:42:43.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-14T15:42:43.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-14T15:42:43.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-14T15:42:43.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-14T15:42:43.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-14T15:42:43.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-14T15:42:43.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-14T15:42:43.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-14T15:42:43.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-14T15:42:43.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-14T15:42:43.192 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-14T15:42:43.192 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-14T15:42:43.192 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-14T15:42:43.193 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-14T15:42:43.193 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-14T15:42:43.193 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-14T15:42:43.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-14T15:42:43.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-14T15:42:43.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-14T15:42:43.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-14T15:42:43.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-14T15:42:43.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-14T15:42:43.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-14T15:42:43.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-14T15:42:43.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-14T15:42:43.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-14T15:42:43.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-14T15:42:43.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-14T15:42:43.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-14T15:42:43.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-14T15:42:43.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-14T15:42:43.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-14T15:42:43.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-14T15:42:43.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-14T15:42:43.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-14T15:42:43.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-14T15:42:43.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-14T15:42:43.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-14T15:42:43.200 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-14T15:42:43.200 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-14T15:42:43.200 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-14T15:42:43.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-14T15:42:43.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-14T15:42:43.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-14T15:42:43.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-14T15:42:43.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-14T15:42:43.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-14T15:42:43.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-14T15:42:43.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-14T15:42:43.203 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-14T15:42:43.203 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-14T15:42:43.203 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-14T15:42:43.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-14T15:42:43.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-14T15:42:43.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-14T15:42:43.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-14T15:42:43.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-14T15:42:43.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-14T15:42:43.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-14T15:42:43.206 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-14T15:42:43.206 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-14T15:42:43.206 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-14T15:42:43.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-14T15:42:43.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-14T15:42:43.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-14T15:42:43.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-14T15:42:43.208 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-14T15:42:43.208 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-14T15:42:43.208 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-14T15:42:43.208 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-14T15:42:43.208 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-14T15:42:43.209 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-14T15:42:43.209 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-14T15:42:43.209 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-14T15:42:43.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-14T15:42:43.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-14T15:42:43.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-14T15:42:43.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-14T15:42:43.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-14T15:42:43.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-14T15:42:43.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-14T15:42:43.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-14T15:42:43.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-14T15:42:43.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-14T15:42:43.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-14T15:42:43.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-14T15:42:43.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-14T15:42:43.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-14T15:42:43.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-14T15:42:43.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-14T15:42:43.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-14T15:42:43.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-14T15:42:43.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-14T15:42:43.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-14T15:42:43.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-14T15:42:43.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-14T15:42:43.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-14T15:42:43.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-14T15:42:43.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-14T15:42:43.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-14T15:42:43.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-14T15:42:43.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-14T15:42:43.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-14T15:42:43.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-14T15:42:43.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-14T15:42:43.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-14T15:42:43.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-14T15:42:43.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-14T15:42:43.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-14T15:42:43.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-14T15:42:43.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-14T15:42:43.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-14T15:42:43.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-14T15:42:43.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-14T15:42:43.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-14T15:42:43.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-14T15:42:43.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-14T15:42:43.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-14T15:42:43.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-14T15:42:43.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-14T15:42:43.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-14T15:42:43.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-14T15:42:43.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-14T15:42:43.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-14T15:42:43.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-14T15:42:43.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-14T15:42:43.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-14T15:42:43.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-14T15:42:43.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-14T15:42:43.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-14T15:42:43.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-14T15:42:43.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-14T15:42:43.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-14T15:42:43.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-14T15:42:43.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-14T15:42:43.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-14T15:42:43.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-14T15:42:43.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-14T15:42:43.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-14T15:42:43.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-14T15:42:43.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-14T15:42:43.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-14T15:42:43.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-14T15:42:43.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-14T15:42:43.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-14T15:42:43.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-14T15:42:43.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-14T15:42:43.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-14T15:42:43.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-14T15:42:43.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-14T15:42:43.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-14T15:42:43.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-14T15:42:43.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-14T15:42:43.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-14T15:42:43.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-14T15:42:43.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-14T15:42:43.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-14T15:42:43.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-14T15:42:43.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-14T15:42:43.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-14T15:42:43.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-14T15:42:43.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-14T15:42:43.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-14T15:42:43.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-14T15:42:43.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-14T15:42:43.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-14T15:42:43.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-14T15:42:43.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-14T15:42:43.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-14T15:42:43.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-14T15:42:43.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-14T15:42:43.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-14T15:42:43.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-14T15:42:43.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-14T15:42:43.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-14T15:42:43.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-14T15:42:43.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-14T15:42:43.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-14T15:42:43.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-14T15:42:43.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-14T15:42:43.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-14T15:42:43.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-14T15:42:43.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-14T15:42:43.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-14T15:42:43.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-14T15:42:43.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-14T15:42:43.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-14T15:42:43.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-14T15:42:43.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-14T15:42:43.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-14T15:42:43.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-14T15:42:43.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-14T15:42:43.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-14T15:42:43.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-14T15:42:43.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-14T15:42:43.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-14T15:42:43.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-14T15:42:43.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-14T15:42:43.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-14T15:42:43.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-14T15:42:43.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-14T15:42:43.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-14T15:42:43.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-14T15:42:43.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-14T15:42:43.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-14T15:42:43.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-14T15:42:43.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-14T15:42:43.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-14T15:42:43.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-14T15:42:43.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-14T15:42:43.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-14T15:42:43.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-14T15:42:43.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-14T15:42:43.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-14T15:42:43.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-14T15:42:43.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-14T15:42:43.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-14T15:42:43.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-14T15:42:43.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-14T15:42:43.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-14T15:42:43.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-14T15:42:43.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-14T15:42:43.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-14T15:42:43.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-14T15:42:43.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-14T15:42:43.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-14T15:42:43.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-14T15:42:43.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-14T15:42:43.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-14T15:42:43.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-14T15:42:43.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-14T15:42:43.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-14T15:42:43.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-14T15:42:43.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-14T15:42:43.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-14T15:42:43.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-14T15:42:43.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-14T15:42:43.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-14T15:42:43.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-14T15:42:43.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-14T15:42:43.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-14T15:42:43.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-14T15:42:43.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-14T15:42:43.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-14T15:42:43.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-14T15:42:43.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-14T15:42:43.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-14T15:42:43.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-14T15:42:43.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-14T15:42:43.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-14T15:42:43.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-14T15:42:43.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-14T15:42:43.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-14T15:42:43.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-14T15:42:43.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-14T15:42:43.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-14T15:42:43.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-14T15:42:43.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-14T15:42:43.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-14T15:42:43.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-14T15:42:43.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-14T15:42:43.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-14T15:42:43.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-14T15:42:43.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-14T15:42:43.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-14T15:42:43.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-14T15:42:43.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-14T15:42:43.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-14T15:42:43.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-14T15:42:43.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-14T15:42:43.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-14T15:42:43.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-14T15:42:43.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-14T15:42:43.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-14T15:42:43.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-14T15:42:43.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-14T15:42:43.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-14T15:42:43.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-14T15:42:43.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-14T15:42:43.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-14T15:42:43.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-14T15:42:43.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-14T15:42:43.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-14T15:42:43.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-14T15:42:43.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-14T15:42:43.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-14T15:42:43.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-14T15:42:43.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-14T15:42:43.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-14T15:42:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-14T15:42:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-14T15:42:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-14T15:42:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-14T15:42:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-14T15:42:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-14T15:42:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-14T15:42:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-14T15:42:43.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-14T15:42:43.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-14T15:42:43.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-14T15:42:43.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-14T15:42:43.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-14T15:42:43.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-14T15:42:43.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-14T15:42:43.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-14T15:42:43.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-14T15:42:43.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-14T15:42:43.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-14T15:42:43.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-14T15:42:43.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-14T15:42:43.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-14T15:42:43.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-14T15:42:43.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-14T15:42:43.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-14T15:42:43.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-14T15:42:43.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-14T15:42:43.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-14T15:42:43.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-14T15:42:43.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-14T15:42:43.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-14T15:42:43.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-14T15:42:43.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-14T15:42:43.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-14T15:42:43.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-14T15:42:43.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-14T15:42:43.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-14T15:42:43.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-14T15:42:43.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-14T15:42:43.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-14T15:42:43.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-14T15:42:43.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-14T15:42:43.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-14T15:42:43.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-14T15:42:43.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-14T15:42:43.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-14T15:42:43.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-14T15:42:43.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-14T15:42:43.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-14T15:42:43.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-14T15:42:43.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-14T15:42:43.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-14T15:42:43.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-14T15:42:43.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-14T15:42:43.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-14T15:42:43.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-14T15:42:43.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-14T15:42:43.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-14T15:42:43.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-14T15:42:43.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-14T15:42:43.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-14T15:42:43.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-14T15:42:43.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-14T15:42:43.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-14T15:42:43.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-14T15:42:43.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-14T15:42:43.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-14T15:42:43.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-14T15:42:43.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-14T15:42:43.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-14T15:42:43.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-14T15:42:43.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-14T15:42:43.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-14T15:42:43.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-14T15:42:43.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-14T15:42:43.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-14T15:42:43.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-14T15:42:43.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-14T15:42:43.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-14T15:42:43.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-14T15:42:43.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-14T15:42:43.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-14T15:42:43.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-14T15:42:43.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-14T15:42:43.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-14T15:42:43.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-14T15:42:43.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-14T15:42:43.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-14T15:42:43.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-14T15:42:43.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-14T15:42:43.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-14T15:42:43.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-14T15:42:43.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-14T15:42:43.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-14T15:42:43.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-14T15:42:43.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-14T15:42:43.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-14T15:42:43.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-14T15:42:43.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-14T15:42:43.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-14T15:42:43.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-14T15:42:43.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-14T15:42:43.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-14T15:42:43.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-14T15:42:43.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-14T15:42:43.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-14T15:42:43.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-14T15:42:43.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-14T15:42:43.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-14T15:42:43.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-14T15:42:43.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-14T15:42:43.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-14T15:42:43.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-14T15:42:43.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-14T15:42:43.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-14T15:42:43.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-14T15:42:43.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-14T15:42:43.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-14T15:42:43.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-14T15:42:43.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-14T15:42:43.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-14T15:42:43.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-14T15:42:43.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-14T15:42:43.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-14T15:42:43.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-14T15:42:43.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-14T15:42:43.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-14T15:42:43.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-14T15:42:43.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-14T15:42:43.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-14T15:42:43.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-14T15:42:43.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-14T15:42:43.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-14T15:42:43.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-14T15:42:43.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-14T15:42:43.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-14T15:42:43.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-14T15:42:43.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-14T15:42:43.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-14T15:42:43.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-14T15:42:43.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-14T15:42:43.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-14T15:42:43.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-14T15:42:43.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-14T15:42:43.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-14T15:42:43.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-14T15:42:43.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-14T15:42:43.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-14T15:42:43.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-14T15:42:43.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-14T15:42:43.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-14T15:42:43.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-14T15:42:43.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-14T15:42:43.264 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-14T15:42:43.264 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-14T15:42:43.264 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-14T15:42:43.264 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-14T15:42:43.264 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-14T15:42:43.264 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-14T15:42:43.264 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-14T15:42:43.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-14T15:42:43.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-14T15:42:43.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-14T15:42:43.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-14T15:42:43.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-14T15:42:43.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-14T15:42:43.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-14T15:42:43.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-14T15:42:43.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-14T15:42:43.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-14T15:42:43.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-14T15:42:43.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-14T15:42:43.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-14T15:42:43.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-14T15:42:43.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-14T15:42:43.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-14T15:42:43.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-14T15:42:43.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-14T15:42:43.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-14T15:42:43.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-14T15:42:43.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-14T15:42:43.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-14T15:42:43.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-14T15:42:43.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-14T15:42:43.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-14T15:42:43.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-14T15:42:43.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-14T15:42:43.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-14T15:42:43.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-14T15:42:43.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-14T15:42:43.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-14T15:42:43.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-14T15:42:43.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-14T15:42:43.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-14T15:42:43.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-14T15:42:43.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-14T15:42:43.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-14T15:42:43.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-14T15:42:43.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-14T15:42:43.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-14T15:42:43.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-14T15:42:43.272 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-14T15:42:43.272 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-14T15:42:43.272 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-14T15:42:43.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-14T15:42:43.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-14T15:42:43.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-14T15:42:43.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-14T15:42:43.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-14T15:42:43.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-14T15:42:43.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-14T15:42:43.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-14T15:42:43.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-14T15:42:43.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-14T15:42:43.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-14T15:42:43.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-14T15:42:43.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-14T15:42:43.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-14T15:42:43.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-14T15:42:43.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-14T15:42:43.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-14T15:42:43.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-14T15:42:43.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-14T15:42:43.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-14T15:42:43.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-14T15:42:43.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-14T15:42:43.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-14T15:42:43.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-14T15:42:43.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-14T15:42:43.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-14T15:42:43.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-14T15:42:43.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-14T15:42:43.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-14T15:42:43.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-14T15:42:43.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-14T15:42:43.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-14T15:42:43.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-14T15:42:43.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-14T15:42:43.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-14T15:42:43.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-14T15:42:43.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-14T15:42:43.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-14T15:42:43.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-14T15:42:43.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-14T15:42:43.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-14T15:42:43.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-14T15:42:43.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-14T15:42:43.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-14T15:42:43.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-14T15:42:43.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-14T15:42:43.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-14T15:42:43.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-14T15:42:43.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-14T15:42:43.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-14T15:42:43.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-14T15:42:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-14T15:42:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-14T15:42:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-14T15:42:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-14T15:42:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-14T15:42:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-14T15:42:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-14T15:42:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-14T15:42:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-14T15:42:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-14T15:42:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-14T15:42:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-14T15:42:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-14T15:42:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-14T15:42:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-14T15:42:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-14T15:42:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-14T15:42:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-14T15:42:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-14T15:42:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-14T15:42:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-14T15:42:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-14T15:42:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-14T15:42:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-14T15:42:43.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-14T15:42:43.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-14T15:42:43.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-14T15:42:43.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-14T15:42:43.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-14T15:42:43.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-14T15:42:43.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-14T15:42:43.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-14T15:42:43.286 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-14T15:42:43.286 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-14T15:42:43.286 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-14T15:42:43.286 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-14T15:42:43.286 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-14T15:42:43.286 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-14T15:42:43.286 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-14T15:42:43.286 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-14T15:42:43.287 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-14T15:42:43.287 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-14T15:42:43.287 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-14T15:42:43.287 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-14T15:42:43.287 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-14T15:42:43.287 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-14T15:42:43.287 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-14T15:42:43.287 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-14T15:42:43.288 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-14T15:42:43.288 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-14T15:42:43.288 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-14T15:42:43.288 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-14T15:42:43.289 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-14T15:42:43.289 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-14T15:42:43.289 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-14T15:42:43.289 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-14T15:42:43.289 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-14T15:42:43.290 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-14T15:42:43.290 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-14T15:42:43.290 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-14T15:42:43.290 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-14T15:42:43.290 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-14T15:42:43.290 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-14T15:42:43.290 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-14T15:42:43.291 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-14T15:42:43.291 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-14T15:42:43.291 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-14T15:42:43.291 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-14T15:42:43.292 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-14T15:42:43.292 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-14T15:42:43.292 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-14T15:42:43.293 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-14T15:42:43.293 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-14T15:42:43.293 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-14T15:42:43.293 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-14T15:42:43.293 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-14T15:42:43.294 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-14T15:42:43.294 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-14T15:42:43.294 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-14T15:42:43.294 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-14T15:42:43.294 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-14T15:42:43.294 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-14T15:42:43.295 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-14T15:42:43.295 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-14T15:42:43.295 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-14T15:42:43.295 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-14T15:42:43.295 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-14T15:42:43.295 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-14T15:42:43.295 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-14T15:42:43.295 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-14T15:42:43.296 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-14T15:42:43.296 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-14T15:42:43.296 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-14T15:42:43.296 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-14T15:42:43.296 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-14T15:42:43.296 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-14T15:42:43.296 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-14T15:42:43.296 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-14T15:42:43.297 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-14T15:42:43.297 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-14T15:42:43.297 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-14T15:42:43.297 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-14T15:42:43.298 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-14T15:42:43.298 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-14T15:42:43.298 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-14T15:42:43.298 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-14T15:42:43.299 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-14T15:42:43.299 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-14T15:42:43.299 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-14T15:42:43.299 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-14T15:42:43.299 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-14T15:42:43.299 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-14T15:42:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-14T15:42:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-14T15:42:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-14T15:42:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-14T15:42:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-14T15:42:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-14T15:42:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-14T15:42:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-14T15:42:43.301 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-14T15:42:43.301 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-14T15:42:43.301 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-14T15:42:43.301 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-14T15:42:43.301 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-14T15:42:43.301 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-14T15:42:43.301 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-14T15:42:43.301 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-14T15:42:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-14T15:42:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-14T15:42:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-14T15:42:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-14T15:42:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-14T15:42:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-14T15:42:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-14T15:42:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-14T15:42:43.303 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-14T15:42:43.303 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-14T15:42:43.303 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-14T15:42:43.303 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-14T15:42:43.303 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-14T15:42:43.303 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-14T15:42:43.303 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-14T15:42:43.303 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-14T15:42:43.304 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-14T15:42:43.304 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-14T15:42:43.304 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-14T15:42:43.304 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-14T15:42:43.304 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-14T15:42:43.304 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-14T15:42:43.304 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-14T15:42:43.304 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-14T15:42:43.305 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-14T15:42:43.305 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-14T15:42:43.305 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-14T15:42:43.305 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-14T15:42:43.305 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-14T15:42:43.305 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-14T15:42:43.305 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-14T15:42:43.305 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-14T15:42:43.306 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-14T15:42:43.306 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-14T15:42:43.306 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-14T15:42:43.306 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-14T15:42:43.306 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-14T15:42:43.306 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-14T15:42:43.306 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-14T15:42:43.306 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-14T15:42:43.307 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-14T15:42:43.307 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-14T15:42:43.307 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-14T15:42:43.307 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-14T15:42:43.307 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-14T15:42:43.307 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-14T15:42:43.307 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-14T15:42:43.308 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-14T15:42:43.308 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-14T15:42:43.308 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-14T15:42:43.308 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-14T15:42:43.308 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-14T15:42:43.309 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-14T15:42:43.309 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-14T15:42:43.309 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-14T15:42:43.309 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-14T15:42:43.310 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-14T15:42:43.310 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-14T15:42:43.310 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-14T15:42:43.310 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-14T15:42:43.310 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-14T15:42:43.310 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-14T15:42:43.310 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-14T15:42:43.311 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-14T15:42:43.311 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-14T15:42:43.311 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-14T15:42:43.311 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-14T15:42:43.312 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-14T15:42:43.312 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-14T15:42:43.312 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-14T15:42:43.312 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-14T15:42:43.313 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-14T15:42:43.313 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-14T15:42:43.313 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-14T15:42:43.314 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-14T15:42:43.314 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-14T15:42:43.314 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-14T15:42:43.314 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-14T15:42:43.314 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-14T15:42:43.315 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-14T15:42:43.315 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-14T15:42:43.315 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-14T15:42:43.315 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-14T15:42:43.316 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-14T15:42:43.316 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-14T15:42:43.316 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-14T15:42:43.316 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-14T15:42:43.316 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-14T15:42:43.316 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-14T15:42:43.317 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-14T15:42:43.317 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-14T15:42:43.317 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-14T15:42:43.317 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-14T15:42:43.317 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-14T15:42:43.317 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-14T15:42:43.317 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-14T15:42:43.317 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-14T15:42:43.318 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-14T15:42:43.318 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-14T15:42:43.318 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-14T15:42:43.318 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-14T15:42:43.318 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-14T15:42:43.318 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-14T15:42:43.319 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-14T15:42:43.319 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-14T15:42:43.319 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-14T15:42:43.319 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-14T15:42:43.320 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-14T15:42:43.320 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-14T15:42:43.320 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-14T15:42:43.320 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-14T15:42:43.320 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-14T15:42:43.320 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-14T15:42:43.321 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-14T15:42:43.321 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-14T15:42:43.321 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-14T15:42:43.321 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-14T15:42:43.321 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-14T15:42:43.321 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-14T15:42:43.321 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-14T15:42:43.322 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-14T15:42:43.322 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-14T15:42:43.322 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-14T15:42:43.322 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-14T15:42:43.322 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-14T15:42:43.322 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-14T15:42:43.322 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-14T15:42:43.322 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-14T15:42:43.323 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-14T15:42:43.323 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-14T15:42:43.323 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-14T15:42:43.323 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-14T15:42:43.323 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-14T15:42:43.324 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-14T15:42:43.324 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-14T15:42:43.324 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-14T15:42:43.324 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-14T15:42:43.325 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-14T15:42:43.325 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-14T15:42:43.325 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-14T15:42:43.325 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-14T15:42:43.326 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-14T15:42:43.326 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-14T15:42:43.326 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-14T15:42:43.326 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-14T15:42:43.327 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-14T15:42:43.327 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-14T15:42:43.327 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-14T15:42:43.327 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-14T15:42:43.327 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-14T15:42:43.327 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-14T15:42:43.328 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-14T15:42:43.328 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-14T15:42:43.328 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-14T15:42:43.328 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-14T15:42:43.329 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-14T15:42:43.329 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-14T15:42:43.329 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-14T15:42:43.329 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-14T15:42:43.329 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-14T15:42:43.330 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-14T15:42:43.330 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-14T15:42:43.330 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-14T15:42:43.330 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-14T15:42:43.330 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-14T15:42:43.330 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-14T15:42:43.331 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-14T15:42:43.331 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-14T15:42:43.331 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-14T15:42:43.331 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-14T15:42:43.332 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-14T15:42:43.332 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-14T15:42:43.332 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-14T15:42:43.333 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-14T15:42:43.333 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-14T15:42:43.333 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-14T15:42:43.333 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-14T15:42:43.334 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-14T15:42:43.334 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-14T15:42:43.334 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-14T15:42:43.334 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-14T15:42:43.334 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-14T15:42:43.335 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-14T15:42:43.335 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-14T15:42:43.335 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-14T15:42:43.335 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-14T15:42:43.335 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-14T15:42:43.335 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-14T15:42:43.335 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-14T15:42:43.336 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-14T15:42:43.336 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-14T15:42:43.336 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-14T15:42:43.336 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-14T15:42:43.336 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-14T15:42:43.336 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-14T15:42:43.336 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-14T15:42:43.337 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-14T15:42:43.337 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-14T15:42:43.337 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-14T15:42:43.337 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-14T15:42:43.337 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-14T15:42:43.338 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-14T15:42:43.338 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-14T15:42:43.338 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-14T15:42:43.338 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-14T15:42:43.339 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-14T15:42:43.339 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-14T15:42:43.339 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-14T15:42:43.339 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-14T15:42:43.339 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-14T15:42:43.340 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-14T15:42:43.340 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-14T15:42:43.340 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-14T15:42:43.340 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-14T15:42:43.340 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-14T15:42:43.340 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-14T15:42:43.340 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-14T15:42:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-14T15:42:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-14T15:42:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-14T15:42:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-14T15:42:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-14T15:42:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-14T15:42:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-14T15:42:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-14T15:42:43.342 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-14T15:42:43.342 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-14T15:42:43.342 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-14T15:42:43.342 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-14T15:42:43.342 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-14T15:42:43.343 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-14T15:42:43.343 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-14T15:42:43.343 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-14T15:42:43.343 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-14T15:42:43.343 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-14T15:42:43.343 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-14T15:42:43.343 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-14T15:42:43.344 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-14T15:42:43.344 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-14T15:42:43.344 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-14T15:42:43.344 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-14T15:42:43.344 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-14T15:42:43.344 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-14T15:42:43.344 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-14T15:42:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-14T15:42:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-14T15:42:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-14T15:42:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-14T15:42:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-14T15:42:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-14T15:42:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-14T15:42:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-14T15:42:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-14T15:42:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-14T15:42:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-14T15:42:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-14T15:42:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-14T15:42:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-14T15:42:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-14T15:42:43.347 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-14T15:42:43.347 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-14T15:42:43.347 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-14T15:42:43.347 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-14T15:42:43.347 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-14T15:42:43.347 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-14T15:42:43.348 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-14T15:42:43.348 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-14T15:42:43.348 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-14T15:42:43.348 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-14T15:42:43.349 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-14T15:42:43.349 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-14T15:42:43.349 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-14T15:42:43.349 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-14T15:42:43.349 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-14T15:42:43.350 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-14T15:42:43.350 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-14T15:42:43.350 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-14T15:42:43.350 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-14T15:42:43.350 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-14T15:42:43.350 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-14T15:42:43.351 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-14T15:42:43.351 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-14T15:42:43.351 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-14T15:42:43.351 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-14T15:42:43.352 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-14T15:42:43.352 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-14T15:42:43.352 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-14T15:42:43.352 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-14T15:42:43.353 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-14T15:42:43.353 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-14T15:42:43.353 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-14T15:42:43.354 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-14T15:42:43.354 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-14T15:42:43.354 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-14T15:42:43.354 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-14T15:42:43.354 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-14T15:42:43.354 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-14T15:42:43.354 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-14T15:42:43.355 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-14T15:42:43.355 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-14T15:42:43.355 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-14T15:42:43.355 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-14T15:42:43.355 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-14T15:42:43.355 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-14T15:42:43.355 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-14T15:42:43.356 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-14T15:42:43.356 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-14T15:42:43.356 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-14T15:42:43.356 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-14T15:42:43.356 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-14T15:42:43.356 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-14T15:42:43.357 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-14T15:42:43.357 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-14T15:42:43.357 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-14T15:42:43.357 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-14T15:42:43.358 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-14T15:42:43.358 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-14T15:42:43.358 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-14T15:42:43.358 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-14T15:42:43.359 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-14T15:42:43.359 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-14T15:42:43.359 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-14T15:42:43.359 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-14T15:42:43.360 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-14T15:42:43.360 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-14T15:42:43.360 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-14T15:42:43.361 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-14T15:42:43.361 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-14T15:42:43.362 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-14T15:42:43.362 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-14T15:42:43.362 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-14T15:42:43.363 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-14T15:42:43.363 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-14T15:42:43.363 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-14T15:42:43.364 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-14T15:42:43.364 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-14T15:42:43.364 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-14T15:42:43.365 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-14T15:42:43.365 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-14T15:42:43.365 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-14T15:42:43.365 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-14T15:42:43.366 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-14T15:42:43.366 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-14T15:42:43.366 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-14T15:42:43.366 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-14T15:42:43.367 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-14T15:42:43.367 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-14T15:42:43.367 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-14T15:42:43.367 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-14T15:42:43.368 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-14T15:42:43.368 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-14T15:42:43.368 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-14T15:42:43.368 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-14T15:42:43.369 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-14T15:42:43.369 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-14T15:42:43.369 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-14T15:42:43.370 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-14T15:42:43.370 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-14T15:42:43.370 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-14T15:42:43.370 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-14T15:42:43.371 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-14T15:42:43.371 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-14T15:42:43.371 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-14T15:42:43.371 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-14T15:42:43.372 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-14T15:42:43.372 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-14T15:42:43.372 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-14T15:42:43.372 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-14T15:42:43.373 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-14T15:42:43.373 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-14T15:42:43.373 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-14T15:42:43.373 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-14T15:42:43.374 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-14T15:42:43.374 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-14T15:42:43.374 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-14T15:42:43.374 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-14T15:42:43.375 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-14T15:42:43.375 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-14T15:42:43.375 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-14T15:42:43.375 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-14T15:42:43.376 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-14T15:42:43.376 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-14T15:42:43.376 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-14T15:42:43.379 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-14T15:42:43.380 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-14T15:42:43.380 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-14T15:42:43.380 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-14T15:42:43.380 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-14T15:42:43.381 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-14T15:42:43.381 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-14T15:42:43.381 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-14T15:42:43.381 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-14T15:42:43.382 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-14T15:42:43.382 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-14T15:42:43.382 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-14T15:42:43.383 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-14T15:42:43.383 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-14T15:42:43.383 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-14T15:42:43.383 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-14T15:42:43.383 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-14T15:42:43.384 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-14T15:42:43.384 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-14T15:42:43.384 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-14T15:42:43.384 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-14T15:42:43.385 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-14T15:42:43.385 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-14T15:42:43.385 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-14T15:42:43.385 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-14T15:42:43.386 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-14T15:42:43.386 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-14T15:42:43.386 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-14T15:42:43.386 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-14T15:42:43.387 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-14T15:42:43.387 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-14T15:42:43.387 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-14T15:42:43.388 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-14T15:42:43.388 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-14T15:42:43.388 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-14T15:42:43.388 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-14T15:42:43.389 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-14T15:42:43.389 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-14T15:42:43.389 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-14T15:42:43.389 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-14T15:42:43.389 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-14T15:42:43.390 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-14T15:42:43.390 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-14T15:42:43.390 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-14T15:42:43.390 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-14T15:42:43.391 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-14T15:42:43.391 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-14T15:42:43.391 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-14T15:42:43.392 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-14T15:42:43.392 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-14T15:42:43.392 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-14T15:42:43.392 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-14T15:42:43.393 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-14T15:42:43.393 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-14T15:42:43.393 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-14T15:42:43.393 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-14T15:42:43.394 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-14T15:42:43.394 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-14T15:42:43.394 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-14T15:42:43.394 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-14T15:42:43.395 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-14T15:42:43.395 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-14T15:42:43.395 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-14T15:42:43.395 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-14T15:42:43.396 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-14T15:42:43.396 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-14T15:42:43.396 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-14T15:42:43.396 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-14T15:42:43.396 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-14T15:42:43.396 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-14T15:42:43.396 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-14T15:42:43.397 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-14T15:42:43.397 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-14T15:42:43.397 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-14T15:42:43.397 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-14T15:42:43.397 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-14T15:42:43.397 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-14T15:42:43.398 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-14T15:42:43.398 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-14T15:42:43.398 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-14T15:42:43.398 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-14T15:42:43.399 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-14T15:42:43.399 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-14T15:42:43.399 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-14T15:42:43.399 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-14T15:42:43.400 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-14T15:42:43.400 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-14T15:42:43.400 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-14T15:42:43.400 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-14T15:42:43.400 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-14T15:42:43.400 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-14T15:42:43.401 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-14T15:42:43.401 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-14T15:42:43.401 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-14T15:42:43.401 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-14T15:42:43.401 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-14T15:42:43.401 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-14T15:42:43.402 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-14T15:42:43.402 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-14T15:42:43.402 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-14T15:42:43.402 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-14T15:42:43.402 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-14T15:42:43.402 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-14T15:42:43.403 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-14T15:42:43.403 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-14T15:42:43.403 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-14T15:42:43.403 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-14T15:42:43.403 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-14T15:42:43.404 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-14T15:42:43.404 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-14T15:42:43.404 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-14T15:42:43.404 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-14T15:42:43.404 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-14T15:42:43.404 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-14T15:42:43.404 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-14T15:42:43.405 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-14T15:42:43.405 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-14T15:42:43.405 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-14T15:42:43.405 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-14T15:42:43.405 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-14T15:42:43.405 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-14T15:42:43.405 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-14T15:42:43.405 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-14T15:42:43.406 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-14T15:42:43.406 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-14T15:42:43.406 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-14T15:42:43.406 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-14T15:42:43.406 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-14T15:42:43.406 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-14T15:42:43.406 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-14T15:42:43.407 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-14T15:42:43.407 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-14T15:42:43.407 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-14T15:42:43.407 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-14T15:42:43.407 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-14T15:42:43.407 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-14T15:42:43.408 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-14T15:42:43.408 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-14T15:42:43.408 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-14T15:42:43.409 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-14T15:42:43.409 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-14T15:42:43.409 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-14T15:42:43.409 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-14T15:42:43.409 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-14T15:42:43.409 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-14T15:42:43.410 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-14T15:42:43.410 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-14T15:42:43.410 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-14T15:42:43.410 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-14T15:42:43.410 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-14T15:42:43.410 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-14T15:42:43.410 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-14T15:42:43.410 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-14T15:42:43.411 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-14T15:42:43.411 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-14T15:42:43.411 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-14T15:42:43.411 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-14T15:42:43.411 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-14T15:42:43.411 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-14T15:42:43.413 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.413 DEBUG:teuthology.orchestra.run.smithi081:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-14T15:42:43.440 INFO:teuthology.orchestra.run.smithi081.stdout:postinstall scriptlet (using /bin/sh): 2023-07-14T15:42:43.441 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-14T15:42:43.441 INFO:teuthology.orchestra.run.smithi081.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:42:43.441 INFO:teuthology.orchestra.run.smithi081.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:42:43.441 INFO:teuthology.orchestra.run.smithi081.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.441 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:42:43.441 INFO:teuthology.orchestra.run.smithi081.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:42:43.441 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2023-07-14T15:42:43.442 INFO:teuthology.orchestra.run.smithi081.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-14T15:42:43.442 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-14T15:42:43.442 INFO:teuthology.orchestra.run.smithi081.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-14T15:42:43.442 INFO:teuthology.orchestra.run.smithi081.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-14T15:42:43.442 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.442 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2023-07-14T15:42:43.442 INFO:teuthology.orchestra.run.smithi081.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-14T15:42:43.442 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-14T15:42:43.443 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.443 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2023-07-14T15:42:43.444 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2023-07-14T15:42:43.522 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2023-07-14T15:42:43.522 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2023-07-14T15:42:43.522 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.02 2023-07-14T15:42:43.522 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 123.el8 2023-07-14T15:42:43.523 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2023-07-14T15:42:43.523 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-14T15:42:43.523 INFO:teuthology.orchestra.run.smithi081.stdout:Group : System Environment/Base 2023-07-14T15:42:43.523 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 9265450 2023-07-14T15:42:43.523 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2023-07-14T15:42:43.523 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-14T15:42:43.523 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-14T15:42:43.523 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-14T15:42:43.524 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-14T15:42:43.524 INFO:teuthology.orchestra.run.smithi081.stdout:Relocations : (not relocatable) 2023-07-14T15:42:43.524 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : Red Hat, Inc. 2023-07-14T15:42:43.524 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : Red Hat, Inc. 2023-07-14T15:42:43.524 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-14T15:42:43.524 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2023-07-14T15:42:43.524 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2023-07-14T15:42:43.525 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T15:42:43.525 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-14T15:42:43.525 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-14T15:42:43.525 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-14T15:42:43.525 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2023-07-14T15:42:43.525 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T15:42:43.526 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2023-07-14T15:42:43.527 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-14T15:42:43.527 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:43.527 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-14T15:42:45.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14 15:42:45 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-14T15:42:45.959 INFO:teuthology.orchestra.run.smithi062.stderr:FINISHED --2023-07-14 15:42:45-- 2023-07-14T15:42:45.959 INFO:teuthology.orchestra.run.smithi062.stderr:Total wall clock time: 4m 8s 2023-07-14T15:42:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:Downloaded: 1 files, 363M in 4m 8s (1.46 MB/s) 2023-07-14T15:42:45.961 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on mon.a... 2023-07-14T15:42:45.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-14T15:42:46.583 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2023-07-14T15:42:47.644 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying... ######################################## 2023-07-14T15:42:47.722 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing... ######################################## 2023-07-14T15:42:47.812 INFO:teuthology.orchestra.run.smithi062.stdout:Updating / installing... 2023-07-14T15:42:48.443 INFO:teuthology.orchestra.run.smithi081.stderr:done 2023-07-14T15:42:48.532 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2023-07-14T15:42:48.550 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.h9FfBy0Ekf 2023-07-14T15:42:48.551 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.h9FfBy0Ekf 2023-07-14T15:42:48.624 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.h9FfBy0Ekf 2023-07-14T15:42:48.774 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.h9FfBy0Ekf is 6KB 2023-07-14T15:42:48.821 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.h9FfBy0Ekf 2023-07-14T15:42:48.837 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2023-07-14T15:42:48.918 INFO:teuthology.orchestra.run.smithi081.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-14T15:42:48.918 INFO:teuthology.orchestra.run.smithi081.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-14T15:42:48.918 INFO:teuthology.orchestra.run.smithi081.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-14T15:42:48.919 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-14T15:42:49.059 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2023-07-14T15:43:12.343 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-14T15:43:19.062 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-07-14T15:43:19.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-07-14T15:43:19.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:43:28.516 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-14T15:43:42.160 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-14T15:43:42.161 DEBUG:teuthology.orchestra.run.smithi164:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-14T15:43:42.213 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.214 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.214 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-14T15:43:42.214 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-14T15:43:42.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-14T15:43:42.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-14T15:43:42.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-14T15:43:42.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-14T15:43:42.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-14T15:43:42.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-14T15:43:42.215 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-14T15:43:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-14T15:43:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-14T15:43:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-14T15:43:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-14T15:43:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-14T15:43:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-14T15:43:42.216 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-14T15:43:42.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-14T15:43:42.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-14T15:43:42.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-14T15:43:42.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-14T15:43:42.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-14T15:43:42.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-14T15:43:42.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-14T15:43:42.217 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-14T15:43:42.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-14T15:43:42.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-14T15:43:42.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-14T15:43:42.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-14T15:43:42.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-14T15:43:42.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-14T15:43:42.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-14T15:43:42.218 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-14T15:43:42.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-14T15:43:42.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-14T15:43:42.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-14T15:43:42.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-14T15:43:42.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-14T15:43:42.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-14T15:43:42.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-14T15:43:42.219 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-14T15:43:42.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-14T15:43:42.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-14T15:43:42.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-14T15:43:42.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-14T15:43:42.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-14T15:43:42.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-14T15:43:42.220 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-14T15:43:42.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-14T15:43:42.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-14T15:43:42.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-14T15:43:42.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-14T15:43:42.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-14T15:43:42.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-14T15:43:42.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-14T15:43:42.221 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-14T15:43:42.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-14T15:43:42.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-14T15:43:42.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-14T15:43:42.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-14T15:43:42.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-14T15:43:42.222 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-14T15:43:42.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-14T15:43:42.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-14T15:43:42.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-14T15:43:42.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-14T15:43:42.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-14T15:43:42.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-14T15:43:42.223 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-14T15:43:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-14T15:43:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-14T15:43:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-14T15:43:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-14T15:43:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-14T15:43:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-14T15:43:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-14T15:43:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-14T15:43:42.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-14T15:43:42.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-14T15:43:42.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-14T15:43:42.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-14T15:43:42.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-14T15:43:42.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-14T15:43:42.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-14T15:43:42.225 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-14T15:43:42.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-14T15:43:42.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-14T15:43:42.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-14T15:43:42.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-14T15:43:42.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-14T15:43:42.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-14T15:43:42.226 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-14T15:43:42.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-14T15:43:42.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-14T15:43:42.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-14T15:43:42.227 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-14T15:43:42.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-14T15:43:42.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-14T15:43:42.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-14T15:43:42.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-14T15:43:42.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-14T15:43:42.228 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-14T15:43:42.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-14T15:43:42.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-14T15:43:42.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-14T15:43:42.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-14T15:43:42.229 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-14T15:43:42.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-14T15:43:42.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-14T15:43:42.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-14T15:43:42.230 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-14T15:43:42.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-14T15:43:42.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-14T15:43:42.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-14T15:43:42.231 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-14T15:43:42.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-14T15:43:42.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-14T15:43:42.232 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-14T15:43:42.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-14T15:43:42.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-14T15:43:42.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-14T15:43:42.233 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-14T15:43:42.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-14T15:43:42.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-14T15:43:42.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-14T15:43:42.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-14T15:43:42.234 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-14T15:43:42.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-14T15:43:42.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-14T15:43:42.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-14T15:43:42.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-14T15:43:42.235 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-14T15:43:42.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-14T15:43:42.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-14T15:43:42.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-14T15:43:42.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-14T15:43:42.236 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-14T15:43:42.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-14T15:43:42.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-14T15:43:42.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-14T15:43:42.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-14T15:43:42.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-14T15:43:42.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-14T15:43:42.237 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-14T15:43:42.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-14T15:43:42.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-14T15:43:42.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-14T15:43:42.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-14T15:43:42.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-14T15:43:42.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-14T15:43:42.238 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-14T15:43:42.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-14T15:43:42.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-14T15:43:42.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-14T15:43:42.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-14T15:43:42.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-14T15:43:42.239 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-14T15:43:42.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-14T15:43:42.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-14T15:43:42.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-14T15:43:42.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-14T15:43:42.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-14T15:43:42.240 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-14T15:43:42.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-14T15:43:42.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-14T15:43:42.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-14T15:43:42.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-14T15:43:42.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-14T15:43:42.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-14T15:43:42.241 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-14T15:43:42.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-14T15:43:42.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-14T15:43:42.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-14T15:43:42.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-14T15:43:42.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-14T15:43:42.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-14T15:43:42.242 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-14T15:43:42.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-14T15:43:42.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-14T15:43:42.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-14T15:43:42.243 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-14T15:43:42.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-14T15:43:42.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-14T15:43:42.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-14T15:43:42.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-14T15:43:42.244 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-14T15:43:42.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-14T15:43:42.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-14T15:43:42.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-14T15:43:42.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-14T15:43:42.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-14T15:43:42.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-14T15:43:42.245 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-14T15:43:42.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-14T15:43:42.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-14T15:43:42.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-14T15:43:42.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-14T15:43:42.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-14T15:43:42.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-14T15:43:42.246 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-14T15:43:42.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-14T15:43:42.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-14T15:43:42.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-14T15:43:42.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-14T15:43:42.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-14T15:43:42.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-14T15:43:42.247 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-14T15:43:42.248 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-14T15:43:42.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-14T15:43:42.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-14T15:43:42.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-14T15:43:42.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-14T15:43:42.249 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-14T15:43:42.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-14T15:43:42.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-14T15:43:42.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-14T15:43:42.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-14T15:43:42.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-14T15:43:42.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-14T15:43:42.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-14T15:43:42.250 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-14T15:43:42.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-14T15:43:42.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-14T15:43:42.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-14T15:43:42.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-14T15:43:42.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-14T15:43:42.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-14T15:43:42.251 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-14T15:43:42.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-14T15:43:42.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-14T15:43:42.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-14T15:43:42.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-14T15:43:42.252 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-14T15:43:42.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-14T15:43:42.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-14T15:43:42.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-14T15:43:42.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-14T15:43:42.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-14T15:43:42.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-14T15:43:42.253 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-14T15:43:42.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-14T15:43:42.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-14T15:43:42.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-14T15:43:42.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-14T15:43:42.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-14T15:43:42.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-14T15:43:42.254 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-14T15:43:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-14T15:43:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-14T15:43:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-14T15:43:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-14T15:43:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-14T15:43:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-14T15:43:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-14T15:43:42.255 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-14T15:43:42.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-14T15:43:42.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-14T15:43:42.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-14T15:43:42.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-14T15:43:42.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-14T15:43:42.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-14T15:43:42.256 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-14T15:43:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-14T15:43:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-14T15:43:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-14T15:43:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-14T15:43:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-14T15:43:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-14T15:43:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-14T15:43:42.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-14T15:43:42.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-14T15:43:42.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-14T15:43:42.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-14T15:43:42.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-14T15:43:42.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-14T15:43:42.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-14T15:43:42.258 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-14T15:43:42.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-14T15:43:42.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-14T15:43:42.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-14T15:43:42.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-14T15:43:42.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-14T15:43:42.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-14T15:43:42.259 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-14T15:43:42.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-14T15:43:42.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-14T15:43:42.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-14T15:43:42.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-14T15:43:42.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-14T15:43:42.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-14T15:43:42.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-14T15:43:42.260 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-14T15:43:42.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-14T15:43:42.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-14T15:43:42.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-14T15:43:42.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-14T15:43:42.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-14T15:43:42.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-14T15:43:42.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-14T15:43:42.261 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-14T15:43:42.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-14T15:43:42.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-14T15:43:42.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-14T15:43:42.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-14T15:43:42.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-14T15:43:42.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-14T15:43:42.262 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-14T15:43:42.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-14T15:43:42.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-14T15:43:42.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-14T15:43:42.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-14T15:43:42.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-14T15:43:42.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-14T15:43:42.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-14T15:43:42.263 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-14T15:43:42.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-14T15:43:42.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-14T15:43:42.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-14T15:43:42.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-14T15:43:42.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-14T15:43:42.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-14T15:43:42.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-14T15:43:42.264 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-14T15:43:42.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-14T15:43:42.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-14T15:43:42.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-14T15:43:42.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-14T15:43:42.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-14T15:43:42.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-14T15:43:42.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-14T15:43:42.265 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-14T15:43:42.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-14T15:43:42.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-14T15:43:42.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-14T15:43:42.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-14T15:43:42.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-14T15:43:42.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-14T15:43:42.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-14T15:43:42.266 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-14T15:43:42.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-14T15:43:42.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-14T15:43:42.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-14T15:43:42.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-14T15:43:42.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-14T15:43:42.267 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-14T15:43:42.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-14T15:43:42.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-14T15:43:42.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-14T15:43:42.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-14T15:43:42.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-14T15:43:42.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-14T15:43:42.268 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-14T15:43:42.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-14T15:43:42.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-14T15:43:42.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-14T15:43:42.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-14T15:43:42.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-14T15:43:42.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-14T15:43:42.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-14T15:43:42.269 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-14T15:43:42.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-14T15:43:42.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-14T15:43:42.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-14T15:43:42.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-14T15:43:42.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-14T15:43:42.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-14T15:43:42.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-14T15:43:42.270 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-14T15:43:42.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-14T15:43:42.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-14T15:43:42.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-14T15:43:42.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-14T15:43:42.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-14T15:43:42.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-14T15:43:42.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-14T15:43:42.271 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-14T15:43:42.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-14T15:43:42.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-14T15:43:42.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-14T15:43:42.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-14T15:43:42.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-14T15:43:42.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-14T15:43:42.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-14T15:43:42.272 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-14T15:43:42.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-14T15:43:42.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-14T15:43:42.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-14T15:43:42.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-14T15:43:42.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-14T15:43:42.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-14T15:43:42.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-14T15:43:42.273 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-14T15:43:42.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-14T15:43:42.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-14T15:43:42.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-14T15:43:42.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-14T15:43:42.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-14T15:43:42.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-14T15:43:42.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-14T15:43:42.274 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-14T15:43:42.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-14T15:43:42.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-14T15:43:42.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-14T15:43:42.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-14T15:43:42.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-14T15:43:42.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-14T15:43:42.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-14T15:43:42.275 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-14T15:43:42.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-14T15:43:42.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-14T15:43:42.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-14T15:43:42.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-14T15:43:42.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-14T15:43:42.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-14T15:43:42.276 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-14T15:43:42.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-14T15:43:42.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-14T15:43:42.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-14T15:43:42.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-14T15:43:42.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-14T15:43:42.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-14T15:43:42.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-14T15:43:42.277 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-14T15:43:42.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-14T15:43:42.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-14T15:43:42.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-14T15:43:42.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-14T15:43:42.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-14T15:43:42.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-14T15:43:42.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-14T15:43:42.278 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-14T15:43:42.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-14T15:43:42.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-14T15:43:42.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-14T15:43:42.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-14T15:43:42.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-14T15:43:42.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-14T15:43:42.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-14T15:43:42.279 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-14T15:43:42.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-14T15:43:42.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-14T15:43:42.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-14T15:43:42.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-14T15:43:42.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-14T15:43:42.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-14T15:43:42.280 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-14T15:43:42.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-14T15:43:42.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-14T15:43:42.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-14T15:43:42.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-14T15:43:42.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-14T15:43:42.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-14T15:43:42.281 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-14T15:43:42.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-14T15:43:42.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-14T15:43:42.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-14T15:43:42.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-14T15:43:42.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-14T15:43:42.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-14T15:43:42.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-14T15:43:42.282 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-14T15:43:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-14T15:43:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-14T15:43:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-14T15:43:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-14T15:43:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-14T15:43:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-14T15:43:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-14T15:43:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-14T15:43:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-14T15:43:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-14T15:43:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-14T15:43:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-14T15:43:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-14T15:43:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-14T15:43:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-14T15:43:42.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-14T15:43:42.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-14T15:43:42.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-14T15:43:42.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-14T15:43:42.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-14T15:43:42.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-14T15:43:42.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-14T15:43:42.285 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-14T15:43:42.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-14T15:43:42.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-14T15:43:42.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-14T15:43:42.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-14T15:43:42.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-14T15:43:42.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-14T15:43:42.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-14T15:43:42.286 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-14T15:43:42.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-14T15:43:42.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-14T15:43:42.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-14T15:43:42.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-14T15:43:42.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-14T15:43:42.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-14T15:43:42.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-14T15:43:42.287 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-14T15:43:42.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-14T15:43:42.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-14T15:43:42.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-14T15:43:42.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-14T15:43:42.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-14T15:43:42.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-14T15:43:42.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-14T15:43:42.288 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-14T15:43:42.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-14T15:43:42.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-14T15:43:42.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-14T15:43:42.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-14T15:43:42.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-14T15:43:42.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-14T15:43:42.289 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-14T15:43:42.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-14T15:43:42.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-14T15:43:42.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-14T15:43:42.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-14T15:43:42.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-14T15:43:42.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-14T15:43:42.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-14T15:43:42.290 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-14T15:43:42.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-14T15:43:42.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-14T15:43:42.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-14T15:43:42.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-14T15:43:42.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-14T15:43:42.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-14T15:43:42.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-14T15:43:42.291 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-14T15:43:42.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-14T15:43:42.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-14T15:43:42.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-14T15:43:42.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-14T15:43:42.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-14T15:43:42.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-14T15:43:42.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-14T15:43:42.292 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-14T15:43:42.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-14T15:43:42.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-14T15:43:42.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-14T15:43:42.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-14T15:43:42.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-14T15:43:42.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-14T15:43:42.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-14T15:43:42.293 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-14T15:43:42.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-14T15:43:42.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-14T15:43:42.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-14T15:43:42.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-14T15:43:42.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-14T15:43:42.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-14T15:43:42.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-14T15:43:42.294 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-14T15:43:42.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-14T15:43:42.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-14T15:43:42.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-14T15:43:42.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-14T15:43:42.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-14T15:43:42.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-14T15:43:42.295 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-14T15:43:42.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-14T15:43:42.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-14T15:43:42.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-14T15:43:42.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-14T15:43:42.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-14T15:43:42.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-14T15:43:42.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-14T15:43:42.296 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-14T15:43:42.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-14T15:43:42.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-14T15:43:42.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-14T15:43:42.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-14T15:43:42.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-14T15:43:42.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-14T15:43:42.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-14T15:43:42.297 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-14T15:43:42.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-14T15:43:42.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-14T15:43:42.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-14T15:43:42.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-14T15:43:42.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-14T15:43:42.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-14T15:43:42.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-14T15:43:42.298 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-14T15:43:42.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-14T15:43:42.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-14T15:43:42.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-14T15:43:42.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-14T15:43:42.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-14T15:43:42.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-14T15:43:42.299 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-14T15:43:42.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-14T15:43:42.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-14T15:43:42.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-14T15:43:42.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-14T15:43:42.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-14T15:43:42.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-14T15:43:42.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-14T15:43:42.300 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-14T15:43:42.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-14T15:43:42.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-14T15:43:42.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-14T15:43:42.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-14T15:43:42.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-14T15:43:42.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-14T15:43:42.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-14T15:43:42.301 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-14T15:43:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-14T15:43:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-14T15:43:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-14T15:43:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-14T15:43:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-14T15:43:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-14T15:43:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-14T15:43:42.302 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-14T15:43:42.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-14T15:43:42.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-14T15:43:42.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-14T15:43:42.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-14T15:43:42.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-14T15:43:42.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-14T15:43:42.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-14T15:43:42.303 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-14T15:43:42.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-14T15:43:42.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-14T15:43:42.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-14T15:43:42.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-14T15:43:42.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-14T15:43:42.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-14T15:43:42.304 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-14T15:43:42.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-14T15:43:42.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-14T15:43:42.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-14T15:43:42.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-14T15:43:42.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-14T15:43:42.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-14T15:43:42.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-14T15:43:42.305 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-14T15:43:42.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-14T15:43:42.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-14T15:43:42.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-14T15:43:42.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-14T15:43:42.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-14T15:43:42.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-14T15:43:42.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-14T15:43:42.306 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-14T15:43:42.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-14T15:43:42.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-14T15:43:42.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-14T15:43:42.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-14T15:43:42.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-14T15:43:42.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-14T15:43:42.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-14T15:43:42.307 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-14T15:43:42.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-14T15:43:42.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-14T15:43:42.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-14T15:43:42.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-14T15:43:42.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-14T15:43:42.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-14T15:43:42.308 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-14T15:43:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-14T15:43:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-14T15:43:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-14T15:43:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-14T15:43:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-14T15:43:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-14T15:43:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-14T15:43:42.309 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-14T15:43:42.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-14T15:43:42.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-14T15:43:42.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-14T15:43:42.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-14T15:43:42.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-14T15:43:42.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-14T15:43:42.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-14T15:43:42.310 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-14T15:43:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-14T15:43:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-14T15:43:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-14T15:43:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-14T15:43:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-14T15:43:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-14T15:43:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-14T15:43:42.311 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-14T15:43:42.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-14T15:43:42.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-14T15:43:42.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-14T15:43:42.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-14T15:43:42.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-14T15:43:42.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-14T15:43:42.312 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-14T15:43:42.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-14T15:43:42.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-14T15:43:42.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-14T15:43:42.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-14T15:43:42.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-14T15:43:42.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-14T15:43:42.313 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-14T15:43:42.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-14T15:43:42.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-14T15:43:42.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-14T15:43:42.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-14T15:43:42.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-14T15:43:42.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-14T15:43:42.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-14T15:43:42.314 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-14T15:43:42.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-14T15:43:42.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-14T15:43:42.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-14T15:43:42.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-14T15:43:42.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-14T15:43:42.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-14T15:43:42.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-14T15:43:42.315 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-14T15:43:42.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-14T15:43:42.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-14T15:43:42.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-14T15:43:42.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-14T15:43:42.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-14T15:43:42.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-14T15:43:42.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-14T15:43:42.316 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-14T15:43:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-14T15:43:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-14T15:43:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-14T15:43:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-14T15:43:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-14T15:43:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-14T15:43:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-14T15:43:42.317 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-14T15:43:42.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-14T15:43:42.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-14T15:43:42.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-14T15:43:42.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-14T15:43:42.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-14T15:43:42.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-14T15:43:42.318 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-14T15:43:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-14T15:43:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-14T15:43:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-14T15:43:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-14T15:43:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-14T15:43:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-14T15:43:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-14T15:43:42.319 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-14T15:43:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-14T15:43:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-14T15:43:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-14T15:43:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-14T15:43:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-14T15:43:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-14T15:43:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-14T15:43:42.320 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-14T15:43:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-14T15:43:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-14T15:43:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-14T15:43:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-14T15:43:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-14T15:43:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-14T15:43:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-14T15:43:42.321 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-14T15:43:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-14T15:43:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-14T15:43:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-14T15:43:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-14T15:43:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-14T15:43:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-14T15:43:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-14T15:43:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-14T15:43:42.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-14T15:43:42.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-14T15:43:42.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-14T15:43:42.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-14T15:43:42.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-14T15:43:42.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-14T15:43:42.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-14T15:43:42.323 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-14T15:43:42.324 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-14T15:43:42.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-14T15:43:42.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-14T15:43:42.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-14T15:43:42.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-14T15:43:42.325 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-14T15:43:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-14T15:43:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-14T15:43:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-14T15:43:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-14T15:43:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-14T15:43:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-14T15:43:42.326 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-14T15:43:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-14T15:43:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-14T15:43:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-14T15:43:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-14T15:43:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-14T15:43:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-14T15:43:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-14T15:43:42.327 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-14T15:43:42.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-14T15:43:42.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-14T15:43:42.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-14T15:43:42.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-14T15:43:42.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-14T15:43:42.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-14T15:43:42.328 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-14T15:43:42.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-14T15:43:42.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-14T15:43:42.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-14T15:43:42.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-14T15:43:42.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-14T15:43:42.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-14T15:43:42.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-14T15:43:42.329 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-14T15:43:42.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-14T15:43:42.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-14T15:43:42.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-14T15:43:42.330 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-14T15:43:42.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-14T15:43:42.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-14T15:43:42.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-14T15:43:42.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-14T15:43:42.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-14T15:43:42.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-14T15:43:42.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-14T15:43:42.331 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-14T15:43:42.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-14T15:43:42.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-14T15:43:42.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-14T15:43:42.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-14T15:43:42.332 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-14T15:43:42.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-14T15:43:42.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-14T15:43:42.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-14T15:43:42.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-14T15:43:42.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-14T15:43:42.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-14T15:43:42.333 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-14T15:43:42.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-14T15:43:42.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-14T15:43:42.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-14T15:43:42.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-14T15:43:42.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-14T15:43:42.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-14T15:43:42.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-14T15:43:42.334 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-14T15:43:42.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-14T15:43:42.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-14T15:43:42.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-14T15:43:42.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-14T15:43:42.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-14T15:43:42.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-14T15:43:42.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-14T15:43:42.335 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-14T15:43:42.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-14T15:43:42.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-14T15:43:42.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-14T15:43:42.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-14T15:43:42.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-14T15:43:42.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-14T15:43:42.336 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-14T15:43:42.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-14T15:43:42.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-14T15:43:42.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-14T15:43:42.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-14T15:43:42.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-14T15:43:42.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-14T15:43:42.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-14T15:43:42.337 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-14T15:43:42.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-14T15:43:42.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-14T15:43:42.338 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-14T15:43:42.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-14T15:43:42.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-14T15:43:42.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-14T15:43:42.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-14T15:43:42.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-14T15:43:42.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-14T15:43:42.339 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-14T15:43:42.340 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-14T15:43:42.340 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-14T15:43:42.340 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-14T15:43:42.340 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-14T15:43:42.340 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-14T15:43:42.340 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-14T15:43:42.340 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-14T15:43:42.341 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-14T15:43:42.341 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-14T15:43:42.341 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-14T15:43:42.341 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-14T15:43:42.341 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-14T15:43:42.342 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-14T15:43:42.342 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-14T15:43:42.342 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-14T15:43:42.342 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-14T15:43:42.342 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-14T15:43:42.342 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-14T15:43:42.342 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-14T15:43:42.342 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-14T15:43:42.343 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-14T15:43:42.343 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-14T15:43:42.343 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-14T15:43:42.343 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-14T15:43:42.343 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-14T15:43:42.343 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-14T15:43:42.343 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-14T15:43:42.344 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-14T15:43:42.344 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-14T15:43:42.344 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-14T15:43:42.344 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-14T15:43:42.344 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-14T15:43:42.344 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-14T15:43:42.344 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-14T15:43:42.344 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-14T15:43:42.345 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-14T15:43:42.345 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-14T15:43:42.345 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-14T15:43:42.345 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-14T15:43:42.345 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-14T15:43:42.345 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-14T15:43:42.345 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-14T15:43:42.345 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-14T15:43:42.346 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-14T15:43:42.346 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-14T15:43:42.346 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-14T15:43:42.346 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-14T15:43:42.346 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-14T15:43:42.346 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-14T15:43:42.346 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-14T15:43:42.346 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-14T15:43:42.347 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-14T15:43:42.347 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-14T15:43:42.347 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-14T15:43:42.347 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-14T15:43:42.347 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-14T15:43:42.347 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-14T15:43:42.348 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-14T15:43:42.348 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-14T15:43:42.348 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-14T15:43:42.348 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-14T15:43:42.348 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-14T15:43:42.348 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-14T15:43:42.348 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-14T15:43:42.349 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-14T15:43:42.349 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-14T15:43:42.349 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-14T15:43:42.349 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-14T15:43:42.349 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-14T15:43:42.349 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-14T15:43:42.350 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-14T15:43:42.350 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-14T15:43:42.350 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-14T15:43:42.350 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-14T15:43:42.350 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-14T15:43:42.350 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-14T15:43:42.351 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-14T15:43:42.351 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-14T15:43:42.351 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-14T15:43:42.351 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-14T15:43:42.351 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-14T15:43:42.351 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-14T15:43:42.351 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-14T15:43:42.351 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-14T15:43:42.352 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-14T15:43:42.352 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-14T15:43:42.352 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-14T15:43:42.352 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-14T15:43:42.352 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-14T15:43:42.352 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-14T15:43:42.353 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-14T15:43:42.353 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-14T15:43:42.353 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-14T15:43:42.353 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-14T15:43:42.353 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-14T15:43:42.353 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-14T15:43:42.353 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-14T15:43:42.354 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-14T15:43:42.354 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-14T15:43:42.354 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-14T15:43:42.354 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-14T15:43:42.354 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-14T15:43:42.354 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-14T15:43:42.354 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-14T15:43:42.354 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-14T15:43:42.355 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-14T15:43:42.355 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-14T15:43:42.355 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-14T15:43:42.355 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-14T15:43:42.355 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-14T15:43:42.355 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-14T15:43:42.355 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-14T15:43:42.355 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-14T15:43:42.356 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-14T15:43:42.356 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-14T15:43:42.356 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-14T15:43:42.356 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-14T15:43:42.356 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-14T15:43:42.356 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-14T15:43:42.356 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-14T15:43:42.356 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-14T15:43:42.357 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-14T15:43:42.357 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-14T15:43:42.357 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-14T15:43:42.357 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-14T15:43:42.357 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-14T15:43:42.357 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-14T15:43:42.357 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-14T15:43:42.357 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-14T15:43:42.358 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-14T15:43:42.358 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-14T15:43:42.358 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-14T15:43:42.358 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-14T15:43:42.358 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-14T15:43:42.358 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-14T15:43:42.358 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-14T15:43:42.358 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-14T15:43:42.359 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-14T15:43:42.359 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-14T15:43:42.359 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-14T15:43:42.359 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-14T15:43:42.359 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-14T15:43:42.359 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-14T15:43:42.359 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-14T15:43:42.360 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-14T15:43:42.360 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-14T15:43:42.360 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-14T15:43:42.360 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-14T15:43:42.360 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-14T15:43:42.360 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-14T15:43:42.360 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-14T15:43:42.360 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-14T15:43:42.361 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-14T15:43:42.361 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-14T15:43:42.361 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-14T15:43:42.361 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-14T15:43:42.361 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-14T15:43:42.361 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-14T15:43:42.361 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-14T15:43:42.361 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-14T15:43:42.362 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-14T15:43:42.362 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-14T15:43:42.362 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-14T15:43:42.362 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-14T15:43:42.362 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-14T15:43:42.362 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-14T15:43:42.362 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-14T15:43:42.362 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-14T15:43:42.363 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-14T15:43:42.363 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-14T15:43:42.363 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-14T15:43:42.363 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-14T15:43:42.363 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-14T15:43:42.363 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-14T15:43:42.363 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-14T15:43:42.363 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-14T15:43:42.364 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-14T15:43:42.364 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-14T15:43:42.364 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-14T15:43:42.364 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-14T15:43:42.364 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-14T15:43:42.364 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-14T15:43:42.364 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-14T15:43:42.364 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-14T15:43:42.365 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-14T15:43:42.365 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-14T15:43:42.365 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-14T15:43:42.365 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-14T15:43:42.365 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-14T15:43:42.365 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-14T15:43:42.365 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-14T15:43:42.366 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-14T15:43:42.366 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-14T15:43:42.366 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-14T15:43:42.366 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-14T15:43:42.366 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-14T15:43:42.366 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-14T15:43:42.366 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-14T15:43:42.366 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-14T15:43:42.367 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-14T15:43:42.367 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-14T15:43:42.367 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-14T15:43:42.367 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-14T15:43:42.367 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-14T15:43:42.367 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-14T15:43:42.367 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-14T15:43:42.367 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-14T15:43:42.368 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-14T15:43:42.368 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-14T15:43:42.368 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-14T15:43:42.368 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-14T15:43:42.368 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-14T15:43:42.368 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-14T15:43:42.368 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-14T15:43:42.368 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-14T15:43:42.369 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-14T15:43:42.369 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-14T15:43:42.369 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-14T15:43:42.369 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-14T15:43:42.369 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-14T15:43:42.369 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-14T15:43:42.369 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-14T15:43:42.369 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-14T15:43:42.370 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-14T15:43:42.370 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-14T15:43:42.370 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-14T15:43:42.370 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-14T15:43:42.370 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-14T15:43:42.370 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-14T15:43:42.370 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-14T15:43:42.370 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-14T15:43:42.371 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-14T15:43:42.371 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-14T15:43:42.371 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-14T15:43:42.371 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-14T15:43:42.371 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-14T15:43:42.371 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-14T15:43:42.371 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-14T15:43:42.371 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-14T15:43:42.372 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-14T15:43:42.372 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-14T15:43:42.372 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-14T15:43:42.372 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-14T15:43:42.372 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-14T15:43:42.372 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-14T15:43:42.372 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-14T15:43:42.373 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-14T15:43:42.373 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-14T15:43:42.373 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-14T15:43:42.373 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-14T15:43:42.373 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-14T15:43:42.373 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-14T15:43:42.373 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-14T15:43:42.373 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-14T15:43:42.374 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-14T15:43:42.374 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-14T15:43:42.374 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-14T15:43:42.374 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-14T15:43:42.374 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-14T15:43:42.374 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-14T15:43:42.374 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-14T15:43:42.374 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-14T15:43:42.375 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-14T15:43:42.375 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-14T15:43:42.375 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-14T15:43:42.375 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-14T15:43:42.375 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-14T15:43:42.375 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-14T15:43:42.375 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-14T15:43:42.375 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-14T15:43:42.376 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-14T15:43:42.376 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-14T15:43:42.376 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-14T15:43:42.376 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-14T15:43:42.376 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-14T15:43:42.376 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-14T15:43:42.376 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-14T15:43:42.377 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-14T15:43:42.377 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-14T15:43:42.377 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-14T15:43:42.377 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-14T15:43:42.377 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-14T15:43:42.377 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-14T15:43:42.377 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-14T15:43:42.377 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-14T15:43:42.378 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-14T15:43:42.378 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-14T15:43:42.378 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-14T15:43:42.378 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-14T15:43:42.378 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-14T15:43:42.378 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-14T15:43:42.378 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-14T15:43:42.378 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-14T15:43:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-14T15:43:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-14T15:43:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-14T15:43:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-14T15:43:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-14T15:43:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-14T15:43:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-14T15:43:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-14T15:43:42.380 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-14T15:43:42.380 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-14T15:43:42.380 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-14T15:43:42.380 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-14T15:43:42.380 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-14T15:43:42.380 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-14T15:43:42.380 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-14T15:43:42.380 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-14T15:43:42.381 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-14T15:43:42.381 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-14T15:43:42.381 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-14T15:43:42.381 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-14T15:43:42.381 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-14T15:43:42.381 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-14T15:43:42.381 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-14T15:43:42.381 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-14T15:43:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-14T15:43:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-14T15:43:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-14T15:43:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-14T15:43:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-14T15:43:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-14T15:43:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-14T15:43:42.382 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-14T15:43:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-14T15:43:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-14T15:43:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-14T15:43:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-14T15:43:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-14T15:43:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-14T15:43:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-14T15:43:42.383 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-14T15:43:42.384 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-14T15:43:42.384 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-14T15:43:42.384 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-14T15:43:42.384 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-14T15:43:42.384 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-14T15:43:42.384 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-14T15:43:42.384 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-14T15:43:42.384 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-14T15:43:42.385 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-14T15:43:42.385 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-14T15:43:42.385 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-14T15:43:42.385 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-14T15:43:42.385 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-14T15:43:42.385 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-14T15:43:42.385 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-14T15:43:42.385 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-14T15:43:42.386 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-14T15:43:42.386 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-14T15:43:42.386 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-14T15:43:42.386 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-14T15:43:42.386 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-14T15:43:42.386 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-14T15:43:42.386 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-14T15:43:42.386 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-14T15:43:42.387 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-14T15:43:42.387 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-14T15:43:42.387 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-14T15:43:42.387 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-14T15:43:42.387 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-14T15:43:42.387 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-14T15:43:42.387 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-14T15:43:42.388 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-14T15:43:42.388 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-14T15:43:42.388 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-14T15:43:42.388 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-14T15:43:42.388 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-14T15:43:42.388 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-14T15:43:42.388 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-14T15:43:42.388 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-14T15:43:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-14T15:43:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-14T15:43:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-14T15:43:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-14T15:43:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-14T15:43:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-14T15:43:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-14T15:43:42.389 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-14T15:43:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-14T15:43:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-14T15:43:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-14T15:43:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-14T15:43:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-14T15:43:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-14T15:43:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-14T15:43:42.390 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-14T15:43:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-14T15:43:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-14T15:43:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-14T15:43:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-14T15:43:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-14T15:43:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-14T15:43:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-14T15:43:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-14T15:43:42.392 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-14T15:43:42.392 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-14T15:43:42.392 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-14T15:43:42.392 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-14T15:43:42.392 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-14T15:43:42.392 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-14T15:43:42.392 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-14T15:43:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-14T15:43:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-14T15:43:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-14T15:43:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-14T15:43:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-14T15:43:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-14T15:43:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-14T15:43:42.393 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-14T15:43:42.394 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-14T15:43:42.394 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-14T15:43:42.394 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-14T15:43:42.394 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-14T15:43:42.394 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-14T15:43:42.394 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-14T15:43:42.394 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-14T15:43:42.394 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-14T15:43:42.395 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-14T15:43:42.395 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-14T15:43:42.395 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-14T15:43:42.395 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-14T15:43:42.395 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-14T15:43:42.395 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-14T15:43:42.395 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-14T15:43:42.395 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-14T15:43:42.396 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-14T15:43:42.396 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-14T15:43:42.396 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-14T15:43:42.396 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-14T15:43:42.396 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-14T15:43:42.396 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-14T15:43:42.396 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-14T15:43:42.396 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-14T15:43:42.397 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-14T15:43:42.397 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-14T15:43:42.397 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-14T15:43:42.397 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-14T15:43:42.397 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-14T15:43:42.397 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-14T15:43:42.397 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-14T15:43:42.397 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-14T15:43:42.398 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-14T15:43:42.398 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-14T15:43:42.398 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-14T15:43:42.398 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-14T15:43:42.398 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-14T15:43:42.398 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-14T15:43:42.398 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-14T15:43:42.398 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-14T15:43:42.399 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-14T15:43:42.399 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-14T15:43:42.399 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-14T15:43:42.399 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-14T15:43:42.399 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-14T15:43:42.399 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-14T15:43:42.399 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-14T15:43:42.399 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-14T15:43:42.400 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-14T15:43:42.400 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-14T15:43:42.400 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-14T15:43:42.400 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-14T15:43:42.400 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-14T15:43:42.400 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-14T15:43:42.400 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-14T15:43:42.400 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-14T15:43:42.401 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-14T15:43:42.401 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-14T15:43:42.401 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-14T15:43:42.401 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-14T15:43:42.401 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-14T15:43:42.401 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-14T15:43:42.401 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-14T15:43:42.401 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-14T15:43:42.402 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-14T15:43:42.402 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-14T15:43:42.402 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-14T15:43:42.402 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-14T15:43:42.402 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-14T15:43:42.402 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-14T15:43:42.402 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-14T15:43:42.403 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-14T15:43:42.403 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-14T15:43:42.403 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-14T15:43:42.403 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-14T15:43:42.403 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-14T15:43:42.403 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-14T15:43:42.403 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-14T15:43:42.403 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-14T15:43:42.404 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-14T15:43:42.404 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-14T15:43:42.404 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-14T15:43:42.404 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-14T15:43:42.404 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-14T15:43:42.404 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-14T15:43:42.404 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-14T15:43:42.404 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-14T15:43:42.405 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-14T15:43:42.405 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-14T15:43:42.405 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-14T15:43:42.405 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-14T15:43:42.405 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-14T15:43:42.405 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-14T15:43:42.405 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-14T15:43:42.405 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-14T15:43:42.406 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-14T15:43:42.406 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-14T15:43:42.406 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-14T15:43:42.406 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-14T15:43:42.406 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-14T15:43:42.406 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-14T15:43:42.406 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-14T15:43:42.406 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-14T15:43:42.407 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-14T15:43:42.407 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-14T15:43:42.407 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-14T15:43:42.407 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-14T15:43:42.407 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-14T15:43:42.407 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-14T15:43:42.407 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-14T15:43:42.407 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-14T15:43:42.408 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-14T15:43:42.408 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-14T15:43:42.408 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-14T15:43:42.408 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-14T15:43:42.408 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-14T15:43:42.408 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-14T15:43:42.408 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-14T15:43:42.409 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-14T15:43:42.409 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-14T15:43:42.409 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-14T15:43:42.409 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-14T15:43:42.409 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-14T15:43:42.409 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-14T15:43:42.409 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-14T15:43:42.409 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-14T15:43:42.410 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-14T15:43:42.410 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-14T15:43:42.410 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-14T15:43:42.410 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-14T15:43:42.410 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-14T15:43:42.410 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-14T15:43:42.410 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-14T15:43:42.410 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-14T15:43:42.411 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-14T15:43:42.411 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-14T15:43:42.411 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-14T15:43:42.411 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-14T15:43:42.411 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-14T15:43:42.411 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-14T15:43:42.411 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-14T15:43:42.411 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-14T15:43:42.412 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-14T15:43:42.412 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-14T15:43:42.412 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-14T15:43:42.412 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-14T15:43:42.412 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-14T15:43:42.412 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-14T15:43:42.412 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-14T15:43:42.412 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-14T15:43:42.413 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-14T15:43:42.413 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-14T15:43:42.413 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-14T15:43:42.413 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-14T15:43:42.413 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-14T15:43:42.413 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-14T15:43:42.413 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-14T15:43:42.413 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-14T15:43:42.414 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-14T15:43:42.414 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-14T15:43:42.414 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-14T15:43:42.414 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-14T15:43:42.414 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-14T15:43:42.414 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-14T15:43:42.414 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-14T15:43:42.414 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-14T15:43:42.415 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-14T15:43:42.415 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-14T15:43:42.415 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-14T15:43:42.415 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-14T15:43:42.415 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-14T15:43:42.415 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-14T15:43:42.415 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-14T15:43:42.415 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-14T15:43:42.416 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-14T15:43:42.416 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-14T15:43:42.416 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-14T15:43:42.416 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-14T15:43:42.416 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-14T15:43:42.416 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-14T15:43:42.416 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-14T15:43:42.416 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-14T15:43:42.417 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-14T15:43:42.417 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-14T15:43:42.417 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-14T15:43:42.417 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-14T15:43:42.417 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-14T15:43:42.417 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-14T15:43:42.417 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-14T15:43:42.417 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-14T15:43:42.418 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-14T15:43:42.418 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-14T15:43:42.418 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-14T15:43:42.418 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-14T15:43:42.418 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-14T15:43:42.418 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-14T15:43:42.418 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-14T15:43:42.418 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-14T15:43:42.419 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-14T15:43:42.419 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-14T15:43:42.419 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-14T15:43:42.419 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-14T15:43:42.419 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-14T15:43:42.419 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-14T15:43:42.419 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-14T15:43:42.420 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-14T15:43:42.420 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-14T15:43:42.420 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-14T15:43:42.420 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-14T15:43:42.420 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-14T15:43:42.420 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-14T15:43:42.420 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-14T15:43:42.420 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-14T15:43:42.421 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-14T15:43:42.421 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-14T15:43:42.421 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-14T15:43:42.421 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-14T15:43:42.421 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-14T15:43:42.421 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-14T15:43:42.421 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-14T15:43:42.421 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-14T15:43:42.422 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-14T15:43:42.422 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-14T15:43:42.422 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-14T15:43:42.422 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-14T15:43:42.422 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-14T15:43:42.422 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-14T15:43:42.422 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-14T15:43:42.422 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-14T15:43:42.423 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-14T15:43:42.423 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-14T15:43:42.423 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-14T15:43:42.423 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-14T15:43:42.423 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-14T15:43:42.423 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-14T15:43:42.423 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-14T15:43:42.423 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-14T15:43:42.424 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-14T15:43:42.424 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-14T15:43:42.424 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-14T15:43:42.424 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-14T15:43:42.424 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-14T15:43:42.424 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-14T15:43:42.424 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-14T15:43:42.424 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-14T15:43:42.425 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-14T15:43:42.425 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-14T15:43:42.425 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-14T15:43:42.425 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-14T15:43:42.425 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-14T15:43:42.425 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-14T15:43:42.425 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-14T15:43:42.426 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-14T15:43:42.426 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-14T15:43:42.426 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-14T15:43:42.426 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-14T15:43:42.426 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-14T15:43:42.426 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-14T15:43:42.426 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-14T15:43:42.426 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-14T15:43:42.427 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-14T15:43:42.427 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-14T15:43:42.427 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-14T15:43:42.427 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-14T15:43:42.427 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-14T15:43:42.427 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-14T15:43:42.427 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-14T15:43:42.427 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-14T15:43:42.428 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-14T15:43:42.428 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-14T15:43:42.428 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-14T15:43:42.428 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-14T15:43:42.428 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-14T15:43:42.428 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-14T15:43:42.428 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-14T15:43:42.428 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-14T15:43:42.429 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-14T15:43:42.429 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-14T15:43:42.429 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-14T15:43:42.429 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-14T15:43:42.429 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-14T15:43:42.429 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-14T15:43:42.429 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-14T15:43:42.429 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-14T15:43:42.430 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-14T15:43:42.430 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-14T15:43:42.430 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-14T15:43:42.430 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-14T15:43:42.430 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-14T15:43:42.430 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-14T15:43:42.430 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-14T15:43:42.430 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-14T15:43:42.431 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-14T15:43:42.431 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-14T15:43:42.431 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-14T15:43:42.431 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-14T15:43:42.431 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-14T15:43:42.431 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-14T15:43:42.431 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-14T15:43:42.431 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-14T15:43:42.432 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-14T15:43:42.432 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-14T15:43:42.432 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-14T15:43:42.432 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-14T15:43:42.432 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-14T15:43:42.432 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-14T15:43:42.432 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-14T15:43:42.432 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-14T15:43:42.433 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-14T15:43:42.433 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-14T15:43:42.433 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-14T15:43:42.433 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-14T15:43:42.433 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-14T15:43:42.433 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-14T15:43:42.433 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-14T15:43:42.433 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-14T15:43:42.434 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-14T15:43:42.434 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-14T15:43:42.434 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-14T15:43:42.434 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-14T15:43:42.434 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-14T15:43:42.434 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-14T15:43:42.434 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-14T15:43:42.434 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-14T15:43:42.435 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-14T15:43:42.435 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-14T15:43:42.435 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-14T15:43:42.435 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-14T15:43:42.435 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-14T15:43:42.435 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-14T15:43:42.435 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-14T15:43:42.436 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-14T15:43:42.436 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-14T15:43:42.436 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-14T15:43:42.436 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-14T15:43:42.436 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-14T15:43:42.436 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-14T15:43:42.436 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-14T15:43:42.436 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-14T15:43:42.437 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-14T15:43:42.437 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-14T15:43:42.437 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-14T15:43:42.437 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-14T15:43:42.437 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-14T15:43:42.437 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-14T15:43:42.437 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-14T15:43:42.437 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-14T15:43:42.438 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-14T15:43:42.438 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-14T15:43:42.438 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-14T15:43:42.438 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-14T15:43:42.438 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-14T15:43:42.438 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-14T15:43:42.438 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-14T15:43:42.438 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-14T15:43:42.439 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-14T15:43:42.439 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-14T15:43:42.439 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-14T15:43:42.439 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-14T15:43:42.439 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-14T15:43:42.439 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-14T15:43:42.440 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-14T15:43:42.440 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-14T15:43:42.440 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-14T15:43:42.440 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-14T15:43:42.440 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-14T15:43:42.440 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-14T15:43:42.440 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-14T15:43:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-14T15:43:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-14T15:43:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-14T15:43:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-14T15:43:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-14T15:43:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-14T15:43:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-14T15:43:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-14T15:43:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-14T15:43:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-14T15:43:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-14T15:43:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-14T15:43:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-14T15:43:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-14T15:43:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-14T15:43:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-14T15:43:42.443 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-14T15:43:42.443 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-14T15:43:42.443 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-14T15:43:42.443 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-14T15:43:42.443 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-14T15:43:42.443 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-14T15:43:42.443 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-14T15:43:42.443 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-14T15:43:42.444 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-14T15:43:42.444 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-14T15:43:42.444 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-14T15:43:42.444 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-14T15:43:42.444 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-14T15:43:42.444 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-14T15:43:42.444 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-14T15:43:42.444 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-14T15:43:42.445 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-14T15:43:42.445 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-14T15:43:42.445 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-14T15:43:42.445 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-14T15:43:42.445 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-14T15:43:42.445 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-14T15:43:42.445 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-14T15:43:42.445 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-14T15:43:42.446 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-14T15:43:42.446 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-14T15:43:42.446 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-14T15:43:42.446 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-14T15:43:42.446 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-14T15:43:42.446 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-14T15:43:42.446 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-14T15:43:42.446 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-14T15:43:42.447 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-14T15:43:42.447 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-14T15:43:42.447 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-14T15:43:42.447 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-14T15:43:42.447 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-14T15:43:42.447 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-14T15:43:42.447 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-14T15:43:42.447 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-14T15:43:42.448 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-14T15:43:42.448 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-14T15:43:42.448 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-14T15:43:42.448 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-14T15:43:42.448 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-14T15:43:42.448 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-14T15:43:42.448 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-14T15:43:42.448 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-14T15:43:42.449 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-14T15:43:42.449 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-14T15:43:42.449 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-14T15:43:42.449 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-14T15:43:42.449 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-14T15:43:42.449 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-14T15:43:42.449 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-14T15:43:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-14T15:43:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-14T15:43:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-14T15:43:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-14T15:43:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-14T15:43:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-14T15:43:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-14T15:43:42.450 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-14T15:43:42.451 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-14T15:43:42.451 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-14T15:43:42.451 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-14T15:43:42.451 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-14T15:43:42.451 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-14T15:43:42.451 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-14T15:43:42.451 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-14T15:43:42.451 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-14T15:43:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-14T15:43:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-14T15:43:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-14T15:43:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-14T15:43:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-14T15:43:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-14T15:43:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-14T15:43:42.452 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-14T15:43:42.453 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-14T15:43:42.453 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-14T15:43:42.453 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-14T15:43:42.453 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-14T15:43:42.453 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-14T15:43:42.453 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-14T15:43:42.454 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-14T15:43:42.454 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-14T15:43:42.454 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-14T15:43:42.454 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-14T15:43:42.454 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-14T15:43:42.454 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-14T15:43:42.454 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-14T15:43:42.455 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-14T15:43:42.455 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-14T15:43:42.455 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-14T15:43:42.455 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-14T15:43:42.455 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-14T15:43:42.455 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-14T15:43:42.455 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-14T15:43:42.455 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-14T15:43:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-14T15:43:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-14T15:43:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-14T15:43:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-14T15:43:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-14T15:43:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-14T15:43:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-14T15:43:42.456 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-14T15:43:42.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-14T15:43:42.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-14T15:43:42.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-14T15:43:42.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-14T15:43:42.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-14T15:43:42.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-14T15:43:42.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-14T15:43:42.457 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-14T15:43:42.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-14T15:43:42.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-14T15:43:42.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-14T15:43:42.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-14T15:43:42.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-14T15:43:42.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-14T15:43:42.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-14T15:43:42.458 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-14T15:43:42.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-14T15:43:42.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-14T15:43:42.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-14T15:43:42.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-14T15:43:42.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-14T15:43:42.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-14T15:43:42.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-14T15:43:42.459 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-14T15:43:42.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-14T15:43:42.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-14T15:43:42.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-14T15:43:42.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-14T15:43:42.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-14T15:43:42.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-14T15:43:42.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-14T15:43:42.460 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-14T15:43:42.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-14T15:43:42.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-14T15:43:42.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-14T15:43:42.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-14T15:43:42.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-14T15:43:42.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-14T15:43:42.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-14T15:43:42.461 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-14T15:43:42.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-14T15:43:42.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-14T15:43:42.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-14T15:43:42.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-14T15:43:42.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-14T15:43:42.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-14T15:43:42.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-14T15:43:42.462 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-14T15:43:42.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-14T15:43:42.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-14T15:43:42.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-14T15:43:42.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-14T15:43:42.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-14T15:43:42.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-14T15:43:42.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-14T15:43:42.463 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-14T15:43:42.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-14T15:43:42.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-14T15:43:42.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-14T15:43:42.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-14T15:43:42.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-14T15:43:42.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-14T15:43:42.464 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-14T15:43:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-14T15:43:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-14T15:43:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-14T15:43:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-14T15:43:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-14T15:43:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-14T15:43:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-14T15:43:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-14T15:43:42.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-14T15:43:42.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-14T15:43:42.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-14T15:43:42.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-14T15:43:42.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-14T15:43:42.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-14T15:43:42.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-14T15:43:42.466 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-14T15:43:42.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-14T15:43:42.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-14T15:43:42.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-14T15:43:42.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-14T15:43:42.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-14T15:43:42.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-14T15:43:42.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-14T15:43:42.467 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-14T15:43:42.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-14T15:43:42.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-14T15:43:42.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-14T15:43:42.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-14T15:43:42.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-14T15:43:42.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-14T15:43:42.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-14T15:43:42.468 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-14T15:43:42.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-14T15:43:42.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-14T15:43:42.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-14T15:43:42.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-14T15:43:42.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-14T15:43:42.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-14T15:43:42.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-14T15:43:42.469 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-14T15:43:42.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-14T15:43:42.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-14T15:43:42.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-14T15:43:42.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-14T15:43:42.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-14T15:43:42.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-14T15:43:42.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-14T15:43:42.470 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-14T15:43:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-14T15:43:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-14T15:43:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-14T15:43:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-14T15:43:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-14T15:43:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-14T15:43:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-14T15:43:42.471 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-14T15:43:42.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-14T15:43:42.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-14T15:43:42.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-14T15:43:42.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-14T15:43:42.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-14T15:43:42.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-14T15:43:42.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-14T15:43:42.472 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-14T15:43:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-14T15:43:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-14T15:43:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-14T15:43:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-14T15:43:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-14T15:43:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-14T15:43:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-14T15:43:42.473 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-14T15:43:42.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-14T15:43:42.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-14T15:43:42.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-14T15:43:42.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-14T15:43:42.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-14T15:43:42.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-14T15:43:42.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-14T15:43:42.474 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-14T15:43:42.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-14T15:43:42.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-14T15:43:42.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-14T15:43:42.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-14T15:43:42.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-14T15:43:42.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-14T15:43:42.475 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-14T15:43:42.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-14T15:43:42.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-14T15:43:42.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-14T15:43:42.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-14T15:43:42.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-14T15:43:42.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-14T15:43:42.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-14T15:43:42.476 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-14T15:43:42.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-14T15:43:42.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-14T15:43:42.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-14T15:43:42.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-14T15:43:42.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-14T15:43:42.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-14T15:43:42.477 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-14T15:43:42.478 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.478 DEBUG:teuthology.orchestra.run.smithi164:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-14T15:43:42.505 INFO:teuthology.orchestra.run.smithi164.stdout:postinstall scriptlet (using /bin/sh): 2023-07-14T15:43:42.506 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-14T15:43:42.506 INFO:teuthology.orchestra.run.smithi164.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:43:42.506 INFO:teuthology.orchestra.run.smithi164.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:43:42.506 INFO:teuthology.orchestra.run.smithi164.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.506 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:43:42.506 INFO:teuthology.orchestra.run.smithi164.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:43:42.507 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2023-07-14T15:43:42.507 INFO:teuthology.orchestra.run.smithi164.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-14T15:43:42.507 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-14T15:43:42.507 INFO:teuthology.orchestra.run.smithi164.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-14T15:43:42.507 INFO:teuthology.orchestra.run.smithi164.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-14T15:43:42.507 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.507 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2023-07-14T15:43:42.507 INFO:teuthology.orchestra.run.smithi164.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-14T15:43:42.508 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-14T15:43:42.508 INFO:teuthology.orchestra.run.smithi164.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.508 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2023-07-14T15:43:42.509 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2023-07-14T15:43:42.590 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2023-07-14T15:43:42.590 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2023-07-14T15:43:42.590 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.02 2023-07-14T15:43:42.591 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 123.el8 2023-07-14T15:43:42.591 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2023-07-14T15:43:42.591 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-14T15:43:42.591 INFO:teuthology.orchestra.run.smithi164.stdout:Group : System Environment/Base 2023-07-14T15:43:42.591 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 9265450 2023-07-14T15:43:42.592 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2023-07-14T15:43:42.592 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-14T15:43:42.592 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-14T15:43:42.592 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-14T15:43:42.592 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-14T15:43:42.592 INFO:teuthology.orchestra.run.smithi164.stdout:Relocations : (not relocatable) 2023-07-14T15:43:42.593 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : Red Hat, Inc. 2023-07-14T15:43:42.593 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : Red Hat, Inc. 2023-07-14T15:43:42.593 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-14T15:43:42.594 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2023-07-14T15:43:42.594 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2023-07-14T15:43:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T15:43:42.594 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-14T15:43:42.595 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-14T15:43:42.595 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-14T15:43:42.595 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2023-07-14T15:43:42.595 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T15:43:42.596 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2023-07-14T15:43:42.597 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-14T15:43:42.597 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:42.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-14T15:43:43.652 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2023-07-14T15:43:45.692 INFO:teuthology.orchestra.run.smithi164.stderr:done 2023-07-14T15:43:45.693 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2023-07-14T15:43:45.752 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.6t2obFk9T8 2023-07-14T15:43:45.756 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6t2obFk9T8 2023-07-14T15:43:45.846 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.6t2obFk9T8 2023-07-14T15:43:45.977 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.6t2obFk9T8 is 6KB 2023-07-14T15:43:46.031 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.6t2obFk9T8 2023-07-14T15:43:46.048 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2023-07-14T15:43:46.119 INFO:teuthology.orchestra.run.smithi164.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-14T15:43:46.120 INFO:teuthology.orchestra.run.smithi164.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-14T15:43:46.120 INFO:teuthology.orchestra.run.smithi164.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-14T15:43:46.121 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:46.271 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2023-07-14T15:43:52.710 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-14T15:43:52.710 DEBUG:teuthology.orchestra.run.smithi062:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-14T15:43:52.757 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:52.758 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:52.758 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:52.758 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:52.758 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-14T15:43:52.758 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-14T15:43:52.758 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-14T15:43:52.759 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-14T15:43:52.759 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-14T15:43:52.759 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-14T15:43:52.759 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-14T15:43:52.759 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-14T15:43:52.759 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-14T15:43:52.759 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-14T15:43:52.759 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-14T15:43:52.760 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-14T15:43:52.760 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-14T15:43:52.760 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-14T15:43:52.760 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-14T15:43:52.760 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-14T15:43:52.760 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-14T15:43:52.760 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-14T15:43:52.761 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-14T15:43:52.761 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-14T15:43:52.761 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-14T15:43:52.761 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-14T15:43:52.761 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-14T15:43:52.761 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-14T15:43:52.761 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-14T15:43:52.761 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-14T15:43:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-14T15:43:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-14T15:43:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-14T15:43:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-14T15:43:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-14T15:43:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-14T15:43:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-14T15:43:52.762 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-14T15:43:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-14T15:43:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-14T15:43:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-14T15:43:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-14T15:43:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-14T15:43:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-14T15:43:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-14T15:43:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-14T15:43:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-14T15:43:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-14T15:43:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-14T15:43:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-14T15:43:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-14T15:43:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-14T15:43:52.764 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-14T15:43:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-14T15:43:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-14T15:43:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-14T15:43:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-14T15:43:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-14T15:43:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-14T15:43:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-14T15:43:52.765 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-14T15:43:52.766 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-14T15:43:52.766 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-14T15:43:52.766 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-14T15:43:52.766 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-14T15:43:52.766 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-14T15:43:52.766 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-14T15:43:52.766 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-14T15:43:52.766 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-14T15:43:52.767 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-14T15:43:52.767 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-14T15:43:52.767 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-14T15:43:52.767 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-14T15:43:52.767 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-14T15:43:52.767 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-14T15:43:52.767 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-14T15:43:52.767 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-14T15:43:52.768 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-14T15:43:52.768 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-14T15:43:52.768 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-14T15:43:52.768 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-14T15:43:52.768 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-14T15:43:52.768 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-14T15:43:52.768 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-14T15:43:52.769 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-14T15:43:52.769 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-14T15:43:52.769 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-14T15:43:52.769 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-14T15:43:52.769 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-14T15:43:52.769 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-14T15:43:52.769 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-14T15:43:52.769 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-14T15:43:52.770 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-14T15:43:52.770 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-14T15:43:52.770 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-14T15:43:52.770 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-14T15:43:52.770 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-14T15:43:52.770 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-14T15:43:52.770 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-14T15:43:52.770 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-14T15:43:52.771 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-14T15:43:52.771 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-14T15:43:52.771 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-14T15:43:52.771 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-14T15:43:52.771 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-14T15:43:52.771 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-14T15:43:52.771 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-14T15:43:52.771 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-14T15:43:52.772 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-14T15:43:52.772 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-14T15:43:52.772 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-14T15:43:52.772 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-14T15:43:52.772 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-14T15:43:52.772 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-14T15:43:52.772 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-14T15:43:52.772 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-14T15:43:52.773 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-14T15:43:52.773 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-14T15:43:52.773 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-14T15:43:52.773 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-14T15:43:52.773 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-14T15:43:52.773 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-14T15:43:52.773 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-14T15:43:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-14T15:43:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-14T15:43:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-14T15:43:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-14T15:43:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-14T15:43:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-14T15:43:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-14T15:43:52.774 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-14T15:43:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-14T15:43:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-14T15:43:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-14T15:43:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-14T15:43:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-14T15:43:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-14T15:43:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-14T15:43:52.775 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-14T15:43:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-14T15:43:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-14T15:43:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-14T15:43:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-14T15:43:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-14T15:43:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-14T15:43:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-14T15:43:52.776 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-14T15:43:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-14T15:43:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-14T15:43:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-14T15:43:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-14T15:43:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-14T15:43:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-14T15:43:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-14T15:43:52.777 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-14T15:43:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-14T15:43:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-14T15:43:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-14T15:43:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-14T15:43:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-14T15:43:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-14T15:43:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-14T15:43:52.778 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-14T15:43:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-14T15:43:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-14T15:43:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-14T15:43:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-14T15:43:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-14T15:43:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-14T15:43:52.779 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-14T15:43:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-14T15:43:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-14T15:43:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-14T15:43:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-14T15:43:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-14T15:43:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-14T15:43:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-14T15:43:52.780 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-14T15:43:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-14T15:43:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-14T15:43:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-14T15:43:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-14T15:43:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-14T15:43:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-14T15:43:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-14T15:43:52.781 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-14T15:43:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-14T15:43:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-14T15:43:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-14T15:43:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-14T15:43:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-14T15:43:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-14T15:43:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-14T15:43:52.782 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-14T15:43:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-14T15:43:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-14T15:43:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-14T15:43:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-14T15:43:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-14T15:43:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-14T15:43:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-14T15:43:52.783 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-14T15:43:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-14T15:43:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-14T15:43:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-14T15:43:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-14T15:43:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-14T15:43:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-14T15:43:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-14T15:43:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-14T15:43:52.785 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-14T15:43:52.785 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-14T15:43:52.785 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-14T15:43:52.785 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-14T15:43:52.785 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-14T15:43:52.785 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-14T15:43:52.785 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-14T15:43:52.785 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-14T15:43:52.786 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-14T15:43:52.786 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-14T15:43:52.786 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-14T15:43:52.786 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-14T15:43:52.786 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-14T15:43:52.786 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-14T15:43:52.786 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-14T15:43:52.786 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-14T15:43:52.787 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-14T15:43:52.787 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-14T15:43:52.787 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-14T15:43:52.787 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-14T15:43:52.787 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-14T15:43:52.787 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-14T15:43:52.787 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-14T15:43:52.788 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-14T15:43:52.788 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-14T15:43:52.788 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-14T15:43:52.788 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-14T15:43:52.788 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-14T15:43:52.788 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-14T15:43:52.788 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-14T15:43:52.788 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-14T15:43:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-14T15:43:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-14T15:43:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-14T15:43:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-14T15:43:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-14T15:43:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-14T15:43:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-14T15:43:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-14T15:43:52.790 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-14T15:43:52.790 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-14T15:43:52.790 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-14T15:43:52.790 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-14T15:43:52.790 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-14T15:43:52.790 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-14T15:43:52.790 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-14T15:43:52.790 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-14T15:43:52.791 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-14T15:43:52.791 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-14T15:43:52.791 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-14T15:43:52.791 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-14T15:43:52.791 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-14T15:43:52.791 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-14T15:43:52.791 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-14T15:43:52.791 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-14T15:43:52.792 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-14T15:43:52.792 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-14T15:43:52.792 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-14T15:43:52.792 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-14T15:43:52.792 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-14T15:43:52.792 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-14T15:43:52.792 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-14T15:43:52.792 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-14T15:43:52.793 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-14T15:43:52.793 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-14T15:43:52.793 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-14T15:43:52.793 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-14T15:43:52.793 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-14T15:43:52.793 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-14T15:43:52.793 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-14T15:43:52.794 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-14T15:43:52.794 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-14T15:43:52.794 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-14T15:43:52.794 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-14T15:43:52.794 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-14T15:43:52.794 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-14T15:43:52.794 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-14T15:43:52.794 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-14T15:43:52.795 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-14T15:43:52.795 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-14T15:43:52.795 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-14T15:43:52.795 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-14T15:43:52.795 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-14T15:43:52.795 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-14T15:43:52.795 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-14T15:43:52.795 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-14T15:43:52.796 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-14T15:43:52.796 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-14T15:43:52.796 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-14T15:43:52.796 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-14T15:43:52.796 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-14T15:43:52.796 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-14T15:43:52.796 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-14T15:43:52.796 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-14T15:43:52.797 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-14T15:43:52.797 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-14T15:43:52.797 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-14T15:43:52.797 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-14T15:43:52.797 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-14T15:43:52.797 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-14T15:43:52.797 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-14T15:43:52.797 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-14T15:43:52.798 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-14T15:43:52.798 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-14T15:43:52.798 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-14T15:43:52.798 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-14T15:43:52.798 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-14T15:43:52.798 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-14T15:43:52.798 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-14T15:43:52.798 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-14T15:43:52.799 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-14T15:43:52.799 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-14T15:43:52.799 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-14T15:43:52.799 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-14T15:43:52.799 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-14T15:43:52.799 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-14T15:43:52.799 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-14T15:43:52.799 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-14T15:43:52.800 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-14T15:43:52.800 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-14T15:43:52.800 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-14T15:43:52.800 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-14T15:43:52.800 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-14T15:43:52.800 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-14T15:43:52.800 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-14T15:43:52.800 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-14T15:43:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-14T15:43:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-14T15:43:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-14T15:43:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-14T15:43:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-14T15:43:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-14T15:43:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-14T15:43:52.801 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-14T15:43:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-14T15:43:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-14T15:43:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-14T15:43:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-14T15:43:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-14T15:43:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-14T15:43:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-14T15:43:52.802 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-14T15:43:52.803 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-14T15:43:52.803 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-14T15:43:52.803 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-14T15:43:52.803 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-14T15:43:52.803 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-14T15:43:52.803 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-14T15:43:52.803 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-14T15:43:52.804 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-14T15:43:52.804 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-14T15:43:52.804 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-14T15:43:52.804 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-14T15:43:52.804 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-14T15:43:52.804 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-14T15:43:52.804 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-14T15:43:52.804 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-14T15:43:52.805 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-14T15:43:52.805 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-14T15:43:52.805 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-14T15:43:52.805 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-14T15:43:52.805 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-14T15:43:52.805 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-14T15:43:52.805 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-14T15:43:52.805 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-14T15:43:52.806 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-14T15:43:52.806 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-14T15:43:52.806 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-14T15:43:52.806 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-14T15:43:52.806 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-14T15:43:52.806 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-14T15:43:52.806 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-14T15:43:52.806 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-14T15:43:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-14T15:43:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-14T15:43:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-14T15:43:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-14T15:43:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-14T15:43:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-14T15:43:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-14T15:43:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-14T15:43:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-14T15:43:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-14T15:43:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-14T15:43:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-14T15:43:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-14T15:43:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-14T15:43:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-14T15:43:52.808 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-14T15:43:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-14T15:43:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-14T15:43:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-14T15:43:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-14T15:43:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-14T15:43:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-14T15:43:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-14T15:43:52.809 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-14T15:43:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-14T15:43:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-14T15:43:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-14T15:43:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-14T15:43:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-14T15:43:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-14T15:43:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-14T15:43:52.810 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-14T15:43:52.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-14T15:43:52.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-14T15:43:52.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-14T15:43:52.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-14T15:43:52.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-14T15:43:52.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-14T15:43:52.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-14T15:43:52.811 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-14T15:43:52.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-14T15:43:52.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-14T15:43:52.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-14T15:43:52.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-14T15:43:52.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-14T15:43:52.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-14T15:43:52.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-14T15:43:52.812 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-14T15:43:52.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-14T15:43:52.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-14T15:43:52.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-14T15:43:52.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-14T15:43:52.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-14T15:43:52.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-14T15:43:52.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-14T15:43:52.813 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-14T15:43:52.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-14T15:43:52.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-14T15:43:52.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-14T15:43:52.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-14T15:43:52.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-14T15:43:52.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-14T15:43:52.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-14T15:43:52.814 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-14T15:43:52.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-14T15:43:52.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-14T15:43:52.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-14T15:43:52.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-14T15:43:52.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-14T15:43:52.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-14T15:43:52.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-14T15:43:52.815 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-14T15:43:52.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-14T15:43:52.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-14T15:43:52.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-14T15:43:52.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-14T15:43:52.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-14T15:43:52.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-14T15:43:52.816 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-14T15:43:52.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-14T15:43:52.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-14T15:43:52.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-14T15:43:52.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-14T15:43:52.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-14T15:43:52.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-14T15:43:52.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-14T15:43:52.817 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-14T15:43:52.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-14T15:43:52.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-14T15:43:52.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-14T15:43:52.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-14T15:43:52.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-14T15:43:52.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-14T15:43:52.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-14T15:43:52.818 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-14T15:43:52.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-14T15:43:52.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-14T15:43:52.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-14T15:43:52.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-14T15:43:52.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-14T15:43:52.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-14T15:43:52.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-14T15:43:52.819 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-14T15:43:52.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-14T15:43:52.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-14T15:43:52.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-14T15:43:52.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-14T15:43:52.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-14T15:43:52.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-14T15:43:52.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-14T15:43:52.820 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-14T15:43:52.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-14T15:43:52.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-14T15:43:52.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-14T15:43:52.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-14T15:43:52.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-14T15:43:52.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-14T15:43:52.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-14T15:43:52.821 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-14T15:43:52.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-14T15:43:52.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-14T15:43:52.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-14T15:43:52.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-14T15:43:52.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-14T15:43:52.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-14T15:43:52.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-14T15:43:52.822 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-14T15:43:52.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-14T15:43:52.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-14T15:43:52.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-14T15:43:52.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-14T15:43:52.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-14T15:43:52.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-14T15:43:52.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-14T15:43:52.823 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-14T15:43:52.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-14T15:43:52.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-14T15:43:52.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-14T15:43:52.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-14T15:43:52.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-14T15:43:52.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-14T15:43:52.824 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-14T15:43:52.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-14T15:43:52.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-14T15:43:52.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-14T15:43:52.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-14T15:43:52.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-14T15:43:52.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-14T15:43:52.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-14T15:43:52.825 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-14T15:43:52.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-14T15:43:52.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-14T15:43:52.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-14T15:43:52.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-14T15:43:52.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-14T15:43:52.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-14T15:43:52.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-14T15:43:52.826 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-14T15:43:52.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-14T15:43:52.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-14T15:43:52.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-14T15:43:52.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-14T15:43:52.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-14T15:43:52.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-14T15:43:52.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-14T15:43:52.827 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-14T15:43:52.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-14T15:43:52.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-14T15:43:52.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-14T15:43:52.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-14T15:43:52.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-14T15:43:52.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-14T15:43:52.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-14T15:43:52.828 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-14T15:43:52.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-14T15:43:52.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-14T15:43:52.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-14T15:43:52.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-14T15:43:52.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-14T15:43:52.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-14T15:43:52.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-14T15:43:52.829 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-14T15:43:52.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-14T15:43:52.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-14T15:43:52.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-14T15:43:52.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-14T15:43:52.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-14T15:43:52.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-14T15:43:52.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-14T15:43:52.830 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-14T15:43:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-14T15:43:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-14T15:43:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-14T15:43:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-14T15:43:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-14T15:43:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-14T15:43:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-14T15:43:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-14T15:43:52.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-14T15:43:52.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-14T15:43:52.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-14T15:43:52.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-14T15:43:52.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-14T15:43:52.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-14T15:43:52.832 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-14T15:43:52.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-14T15:43:52.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-14T15:43:52.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-14T15:43:52.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-14T15:43:52.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-14T15:43:52.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-14T15:43:52.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-14T15:43:52.833 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-14T15:43:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-14T15:43:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-14T15:43:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-14T15:43:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-14T15:43:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-14T15:43:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-14T15:43:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-14T15:43:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-14T15:43:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-14T15:43:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-14T15:43:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-14T15:43:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-14T15:43:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-14T15:43:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-14T15:43:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-14T15:43:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-14T15:43:52.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-14T15:43:52.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-14T15:43:52.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-14T15:43:52.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-14T15:43:52.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-14T15:43:52.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-14T15:43:52.836 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-14T15:43:52.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-14T15:43:52.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-14T15:43:52.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-14T15:43:52.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-14T15:43:52.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-14T15:43:52.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-14T15:43:52.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-14T15:43:52.837 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-14T15:43:52.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-14T15:43:52.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-14T15:43:52.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-14T15:43:52.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-14T15:43:52.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-14T15:43:52.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-14T15:43:52.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-14T15:43:52.838 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-14T15:43:52.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-14T15:43:52.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-14T15:43:52.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-14T15:43:52.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-14T15:43:52.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-14T15:43:52.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-14T15:43:52.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-14T15:43:52.839 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-14T15:43:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-14T15:43:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-14T15:43:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-14T15:43:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-14T15:43:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-14T15:43:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-14T15:43:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-14T15:43:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-14T15:43:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-14T15:43:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-14T15:43:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-14T15:43:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-14T15:43:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-14T15:43:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-14T15:43:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-14T15:43:52.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-14T15:43:52.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-14T15:43:52.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-14T15:43:52.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-14T15:43:52.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-14T15:43:52.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-14T15:43:52.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-14T15:43:52.842 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-14T15:43:52.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-14T15:43:52.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-14T15:43:52.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-14T15:43:52.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-14T15:43:52.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-14T15:43:52.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-14T15:43:52.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-14T15:43:52.843 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-14T15:43:52.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-14T15:43:52.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-14T15:43:52.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-14T15:43:52.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-14T15:43:52.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-14T15:43:52.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-14T15:43:52.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-14T15:43:52.844 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-14T15:43:52.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-14T15:43:52.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-14T15:43:52.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-14T15:43:52.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-14T15:43:52.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-14T15:43:52.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-14T15:43:52.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-14T15:43:52.845 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-14T15:43:52.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-14T15:43:52.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-14T15:43:52.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-14T15:43:52.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-14T15:43:52.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-14T15:43:52.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-14T15:43:52.846 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-14T15:43:52.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-14T15:43:52.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-14T15:43:52.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-14T15:43:52.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-14T15:43:52.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-14T15:43:52.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-14T15:43:52.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-14T15:43:52.847 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-14T15:43:52.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-14T15:43:52.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-14T15:43:52.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-14T15:43:52.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-14T15:43:52.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-14T15:43:52.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-14T15:43:52.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-14T15:43:52.848 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-14T15:43:52.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-14T15:43:52.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-14T15:43:52.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-14T15:43:52.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-14T15:43:52.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-14T15:43:52.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-14T15:43:52.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-14T15:43:52.849 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-14T15:43:52.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-14T15:43:52.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-14T15:43:52.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-14T15:43:52.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-14T15:43:52.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-14T15:43:52.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-14T15:43:52.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-14T15:43:52.850 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-14T15:43:52.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-14T15:43:52.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-14T15:43:52.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-14T15:43:52.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-14T15:43:52.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-14T15:43:52.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-14T15:43:52.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-14T15:43:52.851 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-14T15:43:52.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-14T15:43:52.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-14T15:43:52.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-14T15:43:52.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-14T15:43:52.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-14T15:43:52.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-14T15:43:52.852 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-14T15:43:52.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-14T15:43:52.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-14T15:43:52.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-14T15:43:52.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-14T15:43:52.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-14T15:43:52.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-14T15:43:52.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-14T15:43:52.853 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-14T15:43:52.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-14T15:43:52.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-14T15:43:52.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-14T15:43:52.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-14T15:43:52.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-14T15:43:52.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-14T15:43:52.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-14T15:43:52.854 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-14T15:43:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-14T15:43:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-14T15:43:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-14T15:43:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-14T15:43:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-14T15:43:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-14T15:43:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-14T15:43:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-14T15:43:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-14T15:43:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-14T15:43:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-14T15:43:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-14T15:43:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-14T15:43:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-14T15:43:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-14T15:43:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-14T15:43:52.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-14T15:43:52.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-14T15:43:52.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-14T15:43:52.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-14T15:43:52.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-14T15:43:52.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-14T15:43:52.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-14T15:43:52.857 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-14T15:43:52.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-14T15:43:52.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-14T15:43:52.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-14T15:43:52.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-14T15:43:52.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-14T15:43:52.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-14T15:43:52.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-14T15:43:52.858 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-14T15:43:52.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-14T15:43:52.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-14T15:43:52.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-14T15:43:52.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-14T15:43:52.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-14T15:43:52.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-14T15:43:52.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-14T15:43:52.859 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-14T15:43:52.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-14T15:43:52.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-14T15:43:52.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-14T15:43:52.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-14T15:43:52.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-14T15:43:52.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-14T15:43:52.860 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-14T15:43:52.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-14T15:43:52.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-14T15:43:52.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-14T15:43:52.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-14T15:43:52.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-14T15:43:52.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-14T15:43:52.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-14T15:43:52.861 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-14T15:43:52.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-14T15:43:52.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-14T15:43:52.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-14T15:43:52.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-14T15:43:52.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-14T15:43:52.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-14T15:43:52.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-14T15:43:52.862 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-14T15:43:52.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-14T15:43:52.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-14T15:43:52.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-14T15:43:52.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-14T15:43:52.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-14T15:43:52.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-14T15:43:52.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-14T15:43:52.863 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-14T15:43:52.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-14T15:43:52.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-14T15:43:52.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-14T15:43:52.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-14T15:43:52.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-14T15:43:52.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-14T15:43:52.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-14T15:43:52.864 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-14T15:43:52.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-14T15:43:52.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-14T15:43:52.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-14T15:43:52.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-14T15:43:52.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-14T15:43:52.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-14T15:43:52.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-14T15:43:52.865 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-14T15:43:52.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-14T15:43:52.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-14T15:43:52.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-14T15:43:52.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-14T15:43:52.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-14T15:43:52.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-14T15:43:52.866 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-14T15:43:52.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-14T15:43:52.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-14T15:43:52.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-14T15:43:52.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-14T15:43:52.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-14T15:43:52.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-14T15:43:52.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-14T15:43:52.867 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-14T15:43:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-14T15:43:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-14T15:43:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-14T15:43:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-14T15:43:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-14T15:43:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-14T15:43:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-14T15:43:52.868 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-14T15:43:52.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-14T15:43:52.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-14T15:43:52.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-14T15:43:52.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-14T15:43:52.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-14T15:43:52.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-14T15:43:52.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-14T15:43:52.869 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-14T15:43:52.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-14T15:43:52.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-14T15:43:52.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-14T15:43:52.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-14T15:43:52.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-14T15:43:52.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-14T15:43:52.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-14T15:43:52.870 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-14T15:43:52.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-14T15:43:52.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-14T15:43:52.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-14T15:43:52.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-14T15:43:52.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-14T15:43:52.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-14T15:43:52.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-14T15:43:52.871 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-14T15:43:52.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-14T15:43:52.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-14T15:43:52.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-14T15:43:52.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-14T15:43:52.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-14T15:43:52.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-14T15:43:52.872 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-14T15:43:52.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-14T15:43:52.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-14T15:43:52.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-14T15:43:52.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-14T15:43:52.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-14T15:43:52.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-14T15:43:52.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-14T15:43:52.873 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-14T15:43:52.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-14T15:43:52.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-14T15:43:52.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-14T15:43:52.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-14T15:43:52.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-14T15:43:52.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-14T15:43:52.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-14T15:43:52.874 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-14T15:43:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-14T15:43:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-14T15:43:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-14T15:43:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-14T15:43:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-14T15:43:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-14T15:43:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-14T15:43:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-14T15:43:52.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-14T15:43:52.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-14T15:43:52.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-14T15:43:52.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-14T15:43:52.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-14T15:43:52.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-14T15:43:52.876 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-14T15:43:52.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-14T15:43:52.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-14T15:43:52.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-14T15:43:52.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-14T15:43:52.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-14T15:43:52.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-14T15:43:52.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-14T15:43:52.877 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-14T15:43:52.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-14T15:43:52.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-14T15:43:52.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-14T15:43:52.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-14T15:43:52.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-14T15:43:52.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-14T15:43:52.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-14T15:43:52.878 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-14T15:43:52.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-14T15:43:52.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-14T15:43:52.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-14T15:43:52.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-14T15:43:52.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-14T15:43:52.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-14T15:43:52.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-14T15:43:52.879 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-14T15:43:52.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-14T15:43:52.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-14T15:43:52.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-14T15:43:52.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-14T15:43:52.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-14T15:43:52.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-14T15:43:52.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-14T15:43:52.880 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-14T15:43:52.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-14T15:43:52.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-14T15:43:52.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-14T15:43:52.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-14T15:43:52.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-14T15:43:52.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-14T15:43:52.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-14T15:43:52.881 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-14T15:43:52.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-14T15:43:52.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-14T15:43:52.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-14T15:43:52.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-14T15:43:52.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-14T15:43:52.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-14T15:43:52.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-14T15:43:52.882 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-14T15:43:52.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-14T15:43:52.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-14T15:43:52.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-14T15:43:52.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-14T15:43:52.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-14T15:43:52.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-14T15:43:52.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-14T15:43:52.883 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-14T15:43:52.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-14T15:43:52.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-14T15:43:52.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-14T15:43:52.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-14T15:43:52.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-14T15:43:52.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-14T15:43:52.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-14T15:43:52.884 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-14T15:43:52.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-14T15:43:52.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-14T15:43:52.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-14T15:43:52.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-14T15:43:52.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-14T15:43:52.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-14T15:43:52.885 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-14T15:43:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-14T15:43:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-14T15:43:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-14T15:43:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-14T15:43:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-14T15:43:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-14T15:43:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-14T15:43:52.886 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-14T15:43:52.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-14T15:43:52.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-14T15:43:52.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-14T15:43:52.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-14T15:43:52.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-14T15:43:52.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-14T15:43:52.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-14T15:43:52.887 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-14T15:43:52.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-14T15:43:52.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-14T15:43:52.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-14T15:43:52.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-14T15:43:52.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-14T15:43:52.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-14T15:43:52.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-14T15:43:52.888 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-14T15:43:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-14T15:43:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-14T15:43:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-14T15:43:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-14T15:43:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-14T15:43:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-14T15:43:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-14T15:43:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-14T15:43:52.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-14T15:43:52.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-14T15:43:52.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-14T15:43:52.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-14T15:43:52.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-14T15:43:52.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-14T15:43:52.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-14T15:43:52.890 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-14T15:43:52.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-14T15:43:52.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-14T15:43:52.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-14T15:43:52.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-14T15:43:52.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-14T15:43:52.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-14T15:43:52.891 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-14T15:43:52.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-14T15:43:52.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-14T15:43:52.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-14T15:43:52.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-14T15:43:52.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-14T15:43:52.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-14T15:43:52.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-14T15:43:52.892 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-14T15:43:52.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-14T15:43:52.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-14T15:43:52.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-14T15:43:52.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-14T15:43:52.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-14T15:43:52.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-14T15:43:52.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-14T15:43:52.893 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-14T15:43:52.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-14T15:43:52.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-14T15:43:52.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-14T15:43:52.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-14T15:43:52.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-14T15:43:52.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-14T15:43:52.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-14T15:43:52.894 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-14T15:43:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-14T15:43:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-14T15:43:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-14T15:43:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-14T15:43:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-14T15:43:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-14T15:43:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-14T15:43:52.895 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-14T15:43:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-14T15:43:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-14T15:43:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-14T15:43:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-14T15:43:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-14T15:43:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-14T15:43:52.896 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-14T15:43:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-14T15:43:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-14T15:43:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-14T15:43:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-14T15:43:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-14T15:43:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-14T15:43:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-14T15:43:52.897 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-14T15:43:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-14T15:43:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-14T15:43:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-14T15:43:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-14T15:43:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-14T15:43:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-14T15:43:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-14T15:43:52.898 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-14T15:43:52.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-14T15:43:52.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-14T15:43:52.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-14T15:43:52.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-14T15:43:52.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-14T15:43:52.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-14T15:43:52.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-14T15:43:52.899 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-14T15:43:52.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-14T15:43:52.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-14T15:43:52.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-14T15:43:52.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-14T15:43:52.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-14T15:43:52.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-14T15:43:52.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-14T15:43:52.900 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-14T15:43:52.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-14T15:43:52.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-14T15:43:52.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-14T15:43:52.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-14T15:43:52.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-14T15:43:52.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-14T15:43:52.901 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-14T15:43:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-14T15:43:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-14T15:43:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-14T15:43:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-14T15:43:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-14T15:43:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-14T15:43:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-14T15:43:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-14T15:43:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-14T15:43:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-14T15:43:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-14T15:43:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-14T15:43:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-14T15:43:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-14T15:43:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-14T15:43:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-14T15:43:52.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-14T15:43:52.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-14T15:43:52.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-14T15:43:52.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-14T15:43:52.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-14T15:43:52.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-14T15:43:52.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-14T15:43:52.904 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-14T15:43:52.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-14T15:43:52.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-14T15:43:52.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-14T15:43:52.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-14T15:43:52.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-14T15:43:52.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-14T15:43:52.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-14T15:43:52.905 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-14T15:43:52.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-14T15:43:52.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-14T15:43:52.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-14T15:43:52.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-14T15:43:52.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-14T15:43:52.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-14T15:43:52.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-14T15:43:52.906 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-14T15:43:52.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-14T15:43:52.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-14T15:43:52.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-14T15:43:52.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-14T15:43:52.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-14T15:43:52.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-14T15:43:52.907 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-14T15:43:52.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-14T15:43:52.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-14T15:43:52.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-14T15:43:52.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-14T15:43:52.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-14T15:43:52.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-14T15:43:52.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-14T15:43:52.908 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-14T15:43:52.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-14T15:43:52.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-14T15:43:52.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-14T15:43:52.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-14T15:43:52.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-14T15:43:52.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-14T15:43:52.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-14T15:43:52.909 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-14T15:43:52.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-14T15:43:52.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-14T15:43:52.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-14T15:43:52.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-14T15:43:52.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-14T15:43:52.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-14T15:43:52.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-14T15:43:52.910 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-14T15:43:52.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-14T15:43:52.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-14T15:43:52.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-14T15:43:52.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-14T15:43:52.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-14T15:43:52.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-14T15:43:52.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-14T15:43:52.911 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-14T15:43:52.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-14T15:43:52.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-14T15:43:52.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-14T15:43:52.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-14T15:43:52.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-14T15:43:52.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-14T15:43:52.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-14T15:43:52.912 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-14T15:43:52.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-14T15:43:52.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-14T15:43:52.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-14T15:43:52.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-14T15:43:52.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-14T15:43:52.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-14T15:43:52.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-14T15:43:52.913 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-14T15:43:52.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-14T15:43:52.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-14T15:43:52.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-14T15:43:52.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-14T15:43:52.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-14T15:43:52.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-14T15:43:52.914 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-14T15:43:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-14T15:43:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-14T15:43:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-14T15:43:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-14T15:43:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-14T15:43:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-14T15:43:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-14T15:43:52.915 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-14T15:43:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-14T15:43:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-14T15:43:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-14T15:43:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-14T15:43:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-14T15:43:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-14T15:43:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-14T15:43:52.916 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-14T15:43:52.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-14T15:43:52.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-14T15:43:52.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-14T15:43:52.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-14T15:43:52.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-14T15:43:52.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-14T15:43:52.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-14T15:43:52.917 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-14T15:43:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-14T15:43:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-14T15:43:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-14T15:43:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-14T15:43:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-14T15:43:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-14T15:43:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-14T15:43:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-14T15:43:52.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-14T15:43:52.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-14T15:43:52.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-14T15:43:52.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-14T15:43:52.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-14T15:43:52.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-14T15:43:52.919 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-14T15:43:52.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-14T15:43:52.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-14T15:43:52.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-14T15:43:52.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-14T15:43:52.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-14T15:43:52.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-14T15:43:52.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-14T15:43:52.920 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-14T15:43:52.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-14T15:43:52.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-14T15:43:52.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-14T15:43:52.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-14T15:43:52.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-14T15:43:52.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-14T15:43:52.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-14T15:43:52.921 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-14T15:43:52.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-14T15:43:52.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-14T15:43:52.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-14T15:43:52.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-14T15:43:52.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-14T15:43:52.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-14T15:43:52.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-14T15:43:52.922 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-14T15:43:52.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-14T15:43:52.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-14T15:43:52.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-14T15:43:52.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-14T15:43:52.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-14T15:43:52.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-14T15:43:52.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-14T15:43:52.923 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-14T15:43:52.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-14T15:43:52.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-14T15:43:52.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-14T15:43:52.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-14T15:43:52.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-14T15:43:52.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-14T15:43:52.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-14T15:43:52.924 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-14T15:43:52.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-14T15:43:52.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-14T15:43:52.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-14T15:43:52.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-14T15:43:52.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-14T15:43:52.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-14T15:43:52.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-14T15:43:52.925 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-14T15:43:52.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-14T15:43:52.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-14T15:43:52.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-14T15:43:52.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-14T15:43:52.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-14T15:43:52.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-14T15:43:52.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-14T15:43:52.926 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-14T15:43:52.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-14T15:43:52.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-14T15:43:52.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-14T15:43:52.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-14T15:43:52.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-14T15:43:52.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-14T15:43:52.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-14T15:43:52.927 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-14T15:43:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-14T15:43:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-14T15:43:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-14T15:43:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-14T15:43:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-14T15:43:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-14T15:43:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-14T15:43:52.928 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-14T15:43:52.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-14T15:43:52.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-14T15:43:52.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-14T15:43:52.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-14T15:43:52.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-14T15:43:52.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-14T15:43:52.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-14T15:43:52.929 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-14T15:43:52.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-14T15:43:52.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-14T15:43:52.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-14T15:43:52.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-14T15:43:52.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-14T15:43:52.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-14T15:43:52.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-14T15:43:52.930 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-14T15:43:52.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-14T15:43:52.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-14T15:43:52.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-14T15:43:52.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-14T15:43:52.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-14T15:43:52.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-14T15:43:52.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-14T15:43:52.931 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-14T15:43:52.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-14T15:43:52.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-14T15:43:52.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-14T15:43:52.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-14T15:43:52.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-14T15:43:52.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-14T15:43:52.932 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-14T15:43:52.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-14T15:43:52.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-14T15:43:52.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-14T15:43:52.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-14T15:43:52.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-14T15:43:52.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-14T15:43:52.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-14T15:43:52.933 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-14T15:43:52.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-14T15:43:52.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-14T15:43:52.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-14T15:43:52.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-14T15:43:52.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-14T15:43:52.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-14T15:43:52.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-14T15:43:52.934 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-14T15:43:52.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-14T15:43:52.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-14T15:43:52.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-14T15:43:52.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-14T15:43:52.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-14T15:43:52.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-14T15:43:52.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-14T15:43:52.935 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-14T15:43:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-14T15:43:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-14T15:43:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-14T15:43:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-14T15:43:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-14T15:43:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-14T15:43:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-14T15:43:52.936 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-14T15:43:52.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-14T15:43:52.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-14T15:43:52.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-14T15:43:52.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-14T15:43:52.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-14T15:43:52.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-14T15:43:52.937 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-14T15:43:52.938 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-14T15:43:52.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-14T15:43:52.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-14T15:43:52.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-14T15:43:52.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-14T15:43:52.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-14T15:43:52.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-14T15:43:52.939 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-14T15:43:52.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-14T15:43:52.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-14T15:43:52.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-14T15:43:52.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-14T15:43:52.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-14T15:43:52.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-14T15:43:52.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-14T15:43:52.940 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-14T15:43:52.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-14T15:43:52.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-14T15:43:52.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-14T15:43:52.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-14T15:43:52.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-14T15:43:52.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-14T15:43:52.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-14T15:43:52.941 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-14T15:43:52.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-14T15:43:52.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-14T15:43:52.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-14T15:43:52.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-14T15:43:52.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-14T15:43:52.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-14T15:43:52.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-14T15:43:52.942 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-14T15:43:52.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-14T15:43:52.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-14T15:43:52.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-14T15:43:52.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-14T15:43:52.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-14T15:43:52.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-14T15:43:52.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-14T15:43:52.943 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-14T15:43:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-14T15:43:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-14T15:43:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-14T15:43:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-14T15:43:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-14T15:43:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-14T15:43:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-14T15:43:52.944 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-14T15:43:52.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-14T15:43:52.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-14T15:43:52.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-14T15:43:52.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-14T15:43:52.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-14T15:43:52.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-14T15:43:52.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-14T15:43:52.945 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-14T15:43:52.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-14T15:43:52.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-14T15:43:52.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-14T15:43:52.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-14T15:43:52.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-14T15:43:52.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-14T15:43:52.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-14T15:43:52.946 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-14T15:43:52.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-14T15:43:52.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-14T15:43:52.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-14T15:43:52.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-14T15:43:52.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-14T15:43:52.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-14T15:43:52.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-14T15:43:52.947 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-14T15:43:52.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-14T15:43:52.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-14T15:43:52.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-14T15:43:52.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-14T15:43:52.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-14T15:43:52.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-14T15:43:52.948 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-14T15:43:52.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-14T15:43:52.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-14T15:43:52.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-14T15:43:52.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-14T15:43:52.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-14T15:43:52.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-14T15:43:52.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-14T15:43:52.949 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-14T15:43:52.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-14T15:43:52.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-14T15:43:52.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-14T15:43:52.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-14T15:43:52.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-14T15:43:52.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-14T15:43:52.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-14T15:43:52.950 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-14T15:43:52.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-14T15:43:52.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-14T15:43:52.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-14T15:43:52.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-14T15:43:52.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-14T15:43:52.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-14T15:43:52.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-14T15:43:52.951 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-14T15:43:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-14T15:43:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-14T15:43:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-14T15:43:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-14T15:43:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-14T15:43:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-14T15:43:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-14T15:43:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-14T15:43:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-14T15:43:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-14T15:43:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-14T15:43:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-14T15:43:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-14T15:43:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-14T15:43:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-14T15:43:52.953 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-14T15:43:52.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-14T15:43:52.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-14T15:43:52.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-14T15:43:52.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-14T15:43:52.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-14T15:43:52.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-14T15:43:52.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-14T15:43:52.954 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-14T15:43:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-14T15:43:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-14T15:43:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-14T15:43:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-14T15:43:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-14T15:43:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-14T15:43:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-14T15:43:52.955 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-14T15:43:52.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-14T15:43:52.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-14T15:43:52.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-14T15:43:52.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-14T15:43:52.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-14T15:43:52.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-14T15:43:52.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-14T15:43:52.956 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-14T15:43:52.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-14T15:43:52.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-14T15:43:52.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-14T15:43:52.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-14T15:43:52.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-14T15:43:52.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-14T15:43:52.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-14T15:43:52.957 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-14T15:43:52.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-14T15:43:52.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-14T15:43:52.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-14T15:43:52.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-14T15:43:52.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-14T15:43:52.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-14T15:43:52.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-14T15:43:52.958 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-14T15:43:52.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-14T15:43:52.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-14T15:43:52.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-14T15:43:52.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-14T15:43:52.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-14T15:43:52.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-14T15:43:52.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-14T15:43:52.959 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-14T15:43:52.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-14T15:43:52.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-14T15:43:52.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-14T15:43:52.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-14T15:43:52.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-14T15:43:52.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-14T15:43:52.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-14T15:43:52.960 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-14T15:43:52.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-14T15:43:52.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-14T15:43:52.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-14T15:43:52.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-14T15:43:52.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-14T15:43:52.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-14T15:43:52.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-14T15:43:52.961 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-14T15:43:52.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-14T15:43:52.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-14T15:43:52.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-14T15:43:52.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-14T15:43:52.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-14T15:43:52.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-14T15:43:52.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-14T15:43:52.962 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-14T15:43:52.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-14T15:43:52.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-14T15:43:52.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-14T15:43:52.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-14T15:43:52.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-14T15:43:52.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-14T15:43:52.963 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-14T15:43:52.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-14T15:43:52.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-14T15:43:52.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-14T15:43:52.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-14T15:43:52.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-14T15:43:52.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-14T15:43:52.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-14T15:43:52.964 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-14T15:43:52.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-14T15:43:52.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-14T15:43:52.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-14T15:43:52.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-14T15:43:52.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-14T15:43:52.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-14T15:43:52.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-14T15:43:52.965 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-14T15:43:52.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-14T15:43:52.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-14T15:43:52.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-14T15:43:52.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-14T15:43:52.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-14T15:43:52.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-14T15:43:52.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-14T15:43:52.966 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-14T15:43:52.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-14T15:43:52.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-14T15:43:52.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-14T15:43:52.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-14T15:43:52.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-14T15:43:52.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-14T15:43:52.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-14T15:43:52.967 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-14T15:43:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-14T15:43:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-14T15:43:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-14T15:43:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-14T15:43:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-14T15:43:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-14T15:43:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-14T15:43:52.968 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-14T15:43:52.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-14T15:43:52.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-14T15:43:52.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-14T15:43:52.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-14T15:43:52.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-14T15:43:52.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-14T15:43:52.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-14T15:43:52.969 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-14T15:43:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-14T15:43:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-14T15:43:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-14T15:43:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-14T15:43:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-14T15:43:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-14T15:43:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-14T15:43:52.970 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-14T15:43:52.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-14T15:43:52.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-14T15:43:52.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-14T15:43:52.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-14T15:43:52.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-14T15:43:52.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-14T15:43:52.971 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-14T15:43:52.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-14T15:43:52.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-14T15:43:52.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-14T15:43:52.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-14T15:43:52.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-14T15:43:52.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-14T15:43:52.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-14T15:43:52.972 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-14T15:43:52.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-14T15:43:52.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-14T15:43:52.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-14T15:43:52.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-14T15:43:52.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-14T15:43:52.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-14T15:43:52.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-14T15:43:52.973 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-14T15:43:52.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-14T15:43:52.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-14T15:43:52.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-14T15:43:52.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-14T15:43:52.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-14T15:43:52.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-14T15:43:52.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-14T15:43:52.974 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-14T15:43:52.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-14T15:43:52.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-14T15:43:52.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-14T15:43:52.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-14T15:43:52.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-14T15:43:52.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-14T15:43:52.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-14T15:43:52.975 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-14T15:43:52.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-14T15:43:52.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-14T15:43:52.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-14T15:43:52.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-14T15:43:52.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-14T15:43:52.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-14T15:43:52.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-14T15:43:52.976 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-14T15:43:52.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-14T15:43:52.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-14T15:43:52.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-14T15:43:52.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-14T15:43:52.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-14T15:43:52.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-14T15:43:52.977 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-14T15:43:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-14T15:43:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-14T15:43:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-14T15:43:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-14T15:43:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-14T15:43:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-14T15:43:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-14T15:43:52.978 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-14T15:43:52.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-14T15:43:52.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-14T15:43:52.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-14T15:43:52.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-14T15:43:52.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-14T15:43:52.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-14T15:43:52.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-14T15:43:52.979 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-14T15:43:52.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-14T15:43:52.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-14T15:43:52.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-14T15:43:52.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-14T15:43:52.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-14T15:43:52.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-14T15:43:52.980 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-14T15:43:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-14T15:43:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-14T15:43:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-14T15:43:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-14T15:43:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-14T15:43:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-14T15:43:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-14T15:43:52.981 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-14T15:43:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-14T15:43:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-14T15:43:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-14T15:43:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-14T15:43:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-14T15:43:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-14T15:43:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-14T15:43:52.982 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-14T15:43:52.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-14T15:43:52.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-14T15:43:52.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-14T15:43:52.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-14T15:43:52.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-14T15:43:52.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-14T15:43:52.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-14T15:43:52.983 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-14T15:43:52.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-14T15:43:52.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-14T15:43:52.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-14T15:43:52.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-14T15:43:52.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-14T15:43:52.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-14T15:43:52.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-14T15:43:52.984 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-14T15:43:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-14T15:43:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-14T15:43:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-14T15:43:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-14T15:43:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-14T15:43:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-14T15:43:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-14T15:43:52.985 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-14T15:43:52.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-14T15:43:52.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-14T15:43:52.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-14T15:43:52.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-14T15:43:52.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-14T15:43:52.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-14T15:43:52.986 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-14T15:43:52.987 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-14T15:43:52.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-14T15:43:52.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-14T15:43:52.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-14T15:43:52.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-14T15:43:52.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-14T15:43:52.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-14T15:43:52.988 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-14T15:43:52.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-14T15:43:52.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-14T15:43:52.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-14T15:43:52.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-14T15:43:52.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-14T15:43:52.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-14T15:43:52.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-14T15:43:52.989 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-14T15:43:52.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-14T15:43:52.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-14T15:43:52.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-14T15:43:52.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-14T15:43:52.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-14T15:43:52.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-14T15:43:52.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-14T15:43:52.990 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-14T15:43:52.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-14T15:43:52.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-14T15:43:52.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-14T15:43:52.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-14T15:43:52.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-14T15:43:52.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-14T15:43:52.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-14T15:43:52.991 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-14T15:43:52.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-14T15:43:52.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-14T15:43:52.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-14T15:43:52.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-14T15:43:52.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-14T15:43:52.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-14T15:43:52.992 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-14T15:43:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-14T15:43:52.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-14T15:43:52.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-14T15:43:52.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-14T15:43:52.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-14T15:43:52.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-14T15:43:52.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-14T15:43:52.994 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-14T15:43:52.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-14T15:43:52.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-14T15:43:52.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-14T15:43:52.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-14T15:43:52.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-14T15:43:52.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-14T15:43:52.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-14T15:43:52.995 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-14T15:43:52.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-14T15:43:52.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-14T15:43:52.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-14T15:43:52.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-14T15:43:52.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-14T15:43:52.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-14T15:43:52.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-14T15:43:52.996 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-14T15:43:52.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-14T15:43:52.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-14T15:43:52.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-14T15:43:52.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-14T15:43:52.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-14T15:43:52.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-14T15:43:52.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-14T15:43:52.997 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-14T15:43:52.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-14T15:43:52.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-14T15:43:52.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-14T15:43:52.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-14T15:43:52.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-14T15:43:52.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-14T15:43:52.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-14T15:43:52.998 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-14T15:43:52.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-14T15:43:52.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-14T15:43:52.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-14T15:43:52.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-14T15:43:52.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-14T15:43:52.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-14T15:43:52.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-14T15:43:52.999 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-14T15:43:53.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-14T15:43:53.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-14T15:43:53.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-14T15:43:53.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-14T15:43:53.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-14T15:43:53.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-14T15:43:53.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-14T15:43:53.000 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-14T15:43:53.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-14T15:43:53.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-14T15:43:53.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-14T15:43:53.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-14T15:43:53.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-14T15:43:53.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-14T15:43:53.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-14T15:43:53.001 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-14T15:43:53.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-14T15:43:53.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-14T15:43:53.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-14T15:43:53.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-14T15:43:53.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-14T15:43:53.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-14T15:43:53.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-14T15:43:53.002 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-14T15:43:53.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-14T15:43:53.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-14T15:43:53.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-14T15:43:53.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-14T15:43:53.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-14T15:43:53.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-14T15:43:53.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-14T15:43:53.003 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-14T15:43:53.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-14T15:43:53.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-14T15:43:53.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-14T15:43:53.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-14T15:43:53.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-14T15:43:53.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-14T15:43:53.004 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-14T15:43:53.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-14T15:43:53.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-14T15:43:53.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-14T15:43:53.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-14T15:43:53.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-14T15:43:53.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-14T15:43:53.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-14T15:43:53.005 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-14T15:43:53.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-14T15:43:53.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-14T15:43:53.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-14T15:43:53.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-14T15:43:53.006 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-14T15:43:53.007 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:53.007 DEBUG:teuthology.orchestra.run.smithi062:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-14T15:43:53.034 INFO:teuthology.orchestra.run.smithi062.stdout:postinstall scriptlet (using /bin/sh): 2023-07-14T15:43:53.034 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-14T15:43:53.034 INFO:teuthology.orchestra.run.smithi062.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:43:53.035 INFO:teuthology.orchestra.run.smithi062.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:43:53.035 INFO:teuthology.orchestra.run.smithi062.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:53.035 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:43:53.035 INFO:teuthology.orchestra.run.smithi062.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-14T15:43:53.035 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-07-14T15:43:53.035 INFO:teuthology.orchestra.run.smithi062.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-14T15:43:53.035 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-14T15:43:53.035 INFO:teuthology.orchestra.run.smithi062.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-14T15:43:53.036 INFO:teuthology.orchestra.run.smithi062.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-14T15:43:53.036 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:53.036 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-07-14T15:43:53.036 INFO:teuthology.orchestra.run.smithi062.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-14T15:43:53.036 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-14T15:43:53.036 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:53.036 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2023-07-14T15:43:53.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2023-07-14T15:43:53.117 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2023-07-14T15:43:53.117 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2023-07-14T15:43:53.117 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.02 2023-07-14T15:43:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 123.el8 2023-07-14T15:43:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2023-07-14T15:43:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-14T15:43:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Group : System Environment/Base 2023-07-14T15:43:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 9265450 2023-07-14T15:43:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2023-07-14T15:43:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-14T15:43:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-14T15:43:53.119 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-14T15:43:53.119 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-14T15:43:53.119 INFO:teuthology.orchestra.run.smithi062.stdout:Relocations : (not relocatable) 2023-07-14T15:43:53.119 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : Red Hat, Inc. 2023-07-14T15:43:53.119 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : Red Hat, Inc. 2023-07-14T15:43:53.119 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-14T15:43:53.119 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2023-07-14T15:43:53.120 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2023-07-14T15:43:53.120 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T15:43:53.120 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-14T15:43:53.120 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-14T15:43:53.120 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-14T15:43:53.120 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2023-07-14T15:43:53.120 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T15:43:53.120 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2023-07-14T15:43:53.121 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-14T15:43:53.122 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 6.5.0-rc1-ga440c79eee02 2023-07-14T15:43:53.122 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-14T15:43:57.361 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2023-07-14T15:43:59.835 INFO:teuthology.orchestra.run.smithi062.stderr:done 2023-07-14T15:43:59.837 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2023-07-14T15:43:59.862 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.wS1VNiYJ2y 2023-07-14T15:43:59.863 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wS1VNiYJ2y 2023-07-14T15:43:59.935 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.wS1VNiYJ2y 2023-07-14T15:44:00.082 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.wS1VNiYJ2y is 6KB 2023-07-14T15:44:00.132 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.wS1VNiYJ2y 2023-07-14T15:44:00.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2023-07-14T15:44:00.213 INFO:teuthology.orchestra.run.smithi062.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-14T15:44:00.214 INFO:teuthology.orchestra.run.smithi062.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-14T15:44:00.214 INFO:teuthology.orchestra.run.smithi062.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-14T15:44:00.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-14T15:44:00.373 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2023-07-14T15:44:16.288 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2023-07-14T15:44:16.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-07-14T15:44:16.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:44:19.068 DEBUG:teuthology.orchestra.remote:timed out 2023-07-14T15:44:28.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-07-14T15:44:28.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:44:33.081 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-07-14T15:44:33.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-14T15:44:33.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:44:38.426 DEBUG:teuthology.orchestra.run.smithi081:> true 2023-07-14T15:44:38.831 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-07-14T15:44:38.831 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc1-ga440c79eee02"... 2023-07-14T15:44:38.831 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2023-07-14T15:44:38.874 INFO:teuthology.orchestra.run.smithi081.stdout:6.5.0-rc1-ga440c79eee02 2023-07-14T15:44:38.874 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-14T15:44:38.875 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-14T15:44:38.875 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-14T15:44:39.876 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-07-14T15:44:39.877 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-14T15:44:40.015 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2023-07-14T15:44:40.038 DEBUG:teuthology.parallel:result is None 2023-07-14T15:44:50.800 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-07-14T15:44:56.437 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-14T15:44:59.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-07-14T15:44:59.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:45:05.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-14T15:45:05.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:45:13.520 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-14T15:45:25.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-14T15:45:25.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:45:28.592 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-14T15:45:43.594 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-14T15:45:43.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:45:43.946 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-07-14T15:45:44.399 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-14T15:45:44.400 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc1-ga440c79eee02"... 2023-07-14T15:45:44.400 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-07-14T15:45:44.444 INFO:teuthology.orchestra.run.smithi062.stdout:6.5.0-rc1-ga440c79eee02 2023-07-14T15:45:44.444 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-14T15:45:44.445 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-14T15:45:44.445 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-14T15:45:45.446 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-07-14T15:45:45.446 DEBUG:teuthology.orchestra.run.smithi062:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-14T15:45:45.576 INFO:teuthology.orchestra.run.smithi062.stdout:ttyS1 2023-07-14T15:45:45.601 DEBUG:teuthology.parallel:result is None 2023-07-14T15:45:59.805 DEBUG:teuthology.orchestra.remote:timed out 2023-07-14T15:46:11.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-07-14T15:46:11.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:46:17.167 DEBUG:teuthology.orchestra.run.smithi164:> true 2023-07-14T15:46:17.594 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-07-14T15:46:17.594 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.5.0-rc1-ga440c79eee02"... 2023-07-14T15:46:17.594 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2023-07-14T15:46:17.637 INFO:teuthology.orchestra.run.smithi164.stdout:6.5.0-rc1-ga440c79eee02 2023-07-14T15:46:17.638 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-14T15:46:17.638 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-14T15:46:17.638 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-14T15:46:18.639 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2023-07-14T15:46:18.640 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-14T15:46:18.768 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2023-07-14T15:46:18.790 DEBUG:teuthology.parallel:result is None 2023-07-14T15:46:18.791 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-14T15:46:18.798 INFO:teuthology.task.internal:Creating test directory... 2023-07-14T15:46:18.799 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T15:46:18.805 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T15:46:18.811 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T15:46:18.855 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-14T15:46:18.863 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-14T15:46:18.915 INFO:teuthology.task.internal:Creating archive directory... 2023-07-14T15:46:18.916 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T15:46:18.921 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T15:46:18.926 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T15:46:18.989 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-14T15:46:18.997 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-14T15:46:18.998 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T15:46:19.002 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T15:46:19.007 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T15:46:19.080 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T15:46:19.084 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T15:46:19.094 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T15:46:19.108 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T15:46:19.113 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T15:46:19.122 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T15:46:19.125 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-14T15:46:19.137 INFO:teuthology.task.internal:Configuring sudo... 2023-07-14T15:46:19.138 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T15:46:19.143 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T15:46:19.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T15:46:19.217 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-14T15:46:19.227 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-14T15:46:19.228 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T15:46:19.232 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T15:46:19.236 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T15:46:19.279 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T15:46:24.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T15:46:24.471 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T15:46:29.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T15:46:29.628 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T15:46:29.629 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T15:46:29.699 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T15:46:32.459 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T15:46:32.547 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T15:46:32.620 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T15:46:32.688 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T15:46:32.689 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T15:46:32.757 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T15:46:37.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T15:46:37.935 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T15:46:43.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T15:46:43.096 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T15:46:43.096 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T15:46:43.167 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2023-07-14T15:46:43.174 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2023-07-14T15:46:43.179 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2023-07-14T15:46:43.296 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T15:46:43.305 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T15:46:43.333 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T15:46:43.710 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-14T15:46:43.719 INFO:teuthology.task.internal:Starting timer... 2023-07-14T15:46:43.720 INFO:teuthology.run_tasks:Running task pcp... 2023-07-14T15:46:43.751 INFO:teuthology.run_tasks:Running task selinux... 2023-07-14T15:46:43.769 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-14T15:46:43.769 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2023-07-14T15:46:43.896 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2023-07-14T15:46:43.899 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2023-07-14T15:46:44.015 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2023-07-14T15:46:44.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2023-07-14T15:46:44.136 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2023-07-14T15:46:44.140 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-14T15:46:44.141 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2023-07-14T15:46:44.193 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2023-07-14T15:46:44.194 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2023-07-14T15:46:44.246 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2023-07-14T15:46:44.246 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2023-07-14T15:46:44.309 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2023-07-14T15:46:44.309 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2023-07-14T15:46:44.310 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T15:46:44.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T15:46:44.384 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T15:46:44.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T15:46:44.456 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T15:46:44.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T15:46:44.527 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-14T15:46:44.527 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2023-07-14T15:46:44.598 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2023-07-14T15:46:44.668 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2023-07-14T15:46:44.755 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-14T15:46:44.765 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-14T15:46:44.771 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-14T15:46:49.934 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-14T15:46:49.954 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-14T15:46:49.955 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi081.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-14T15:59:48.784 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2023-07-14T15:59:48.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-14T15:59:48.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:59:48.880 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-07-14T15:59:48.991 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-14T15:59:48.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-07-14T15:59:48.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:59:49.124 DEBUG:teuthology.orchestra.run.smithi081:> true 2023-07-14T15:59:49.238 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-07-14T15:59:49.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-07-14T15:59:49.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T15:59:49.333 DEBUG:teuthology.orchestra.run.smithi164:> true 2023-07-14T15:59:49.452 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-07-14T15:59:49.452 INFO:teuthology.run_tasks:Running task clock... 2023-07-14T15:59:49.463 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-14T15:59:49.463 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T15:59:49.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T15:59:49.469 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T15:59:49.469 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T15:59:49.473 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T15:59:49.474 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T15:59:49.543 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T15:59:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T15:59:49.568 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T15:59:49.577 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T15:59:49.579 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T15:59:49.602 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T15:59:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2023-07-14T15:59:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2023-07-14T15:59:49.695 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2023-07-14T15:59:49.695 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2023-07-14T15:59:49.707 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2023-07-14T15:59:49.724 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2023-07-14T15:59:49.729 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T15:59:49.742 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T15:59:49.759 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T15:59:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T15:59:49.777 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T15:59:49.793 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T15:59:49.853 INFO:teuthology.orchestra.run.smithi081.stderr:bash: ntpq: command not found 2023-07-14T15:59:49.861 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T15:59:49.861 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2023-07-14T15:59:49.861 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.861 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.861 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.862 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.868 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-07-14T15:59:49.876 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T15:59:49.877 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-07-14T15:59:49.877 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.877 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.877 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.877 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.880 INFO:teuthology.orchestra.run.smithi164.stderr:bash: ntpq: command not found 2023-07-14T15:59:49.887 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T15:59:49.888 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2023-07-14T15:59:49.888 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.888 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.888 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.888 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T15:59:49.889 INFO:teuthology.run_tasks:Running task pexec... 2023-07-14T15:59:49.900 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-14T15:59:49.901 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi062.front.sepia.ceph.com 2023-07-14T15:59:49.901 DEBUG:teuthology.orchestra.run.smithi062:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T15:59:49.902 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi081.front.sepia.ceph.com 2023-07-14T15:59:49.902 DEBUG:teuthology.orchestra.run.smithi081:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T15:59:49.903 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi164.front.sepia.ceph.com 2023-07-14T15:59:49.903 DEBUG:teuthology.orchestra.run.smithi164:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T15:59:50.379 INFO:teuthology.orchestra.run.smithi081.stdout:Updating Subscription Management repositories. 2023-07-14T15:59:50.389 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2023-07-14T15:59:50.415 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-07-14T15:59:51.430 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:58 ago on Fri 14 Jul 2023 03:57:52 PM UTC. 2023-07-14T15:59:51.442 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:55 ago on Fri 14 Jul 2023 03:57:55 PM UTC. 2023-07-14T15:59:51.536 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:17 ago on Fri 14 Jul 2023 03:57:34 PM UTC. 2023-07-14T15:59:53.231 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2023-07-14T15:59:53.232 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2023-07-14T15:59:53.233 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2023-07-14T15:59:53.233 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2023-07-14T15:59:53.233 INFO:teuthology.orchestra.run.smithi081.stdout:Resetting modules: 2023-07-14T15:59:53.233 INFO:teuthology.orchestra.run.smithi081.stdout: container-tools 2023-07-14T15:59:53.233 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T15:59:53.233 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2023-07-14T15:59:53.234 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2023-07-14T15:59:53.234 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T15:59:53.286 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2023-07-14T15:59:53.287 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2023-07-14T15:59:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2023-07-14T15:59:53.287 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2023-07-14T15:59:53.287 INFO:teuthology.orchestra.run.smithi164.stdout:Resetting modules: 2023-07-14T15:59:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: container-tools 2023-07-14T15:59:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T15:59:53.288 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2023-07-14T15:59:53.288 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2023-07-14T15:59:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T15:59:53.614 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-07-14T15:59:53.615 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-07-14T15:59:53.615 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2023-07-14T15:59:53.615 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-07-14T15:59:53.616 INFO:teuthology.orchestra.run.smithi062.stdout:Resetting modules: 2023-07-14T15:59:53.616 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools 2023-07-14T15:59:53.616 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T15:59:53.616 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-07-14T15:59:53.616 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-07-14T15:59:53.616 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T15:59:54.456 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:01 ago on Fri 14 Jul 2023 03:57:52 PM UTC. 2023-07-14T15:59:54.548 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:59 ago on Fri 14 Jul 2023 03:57:55 PM UTC. 2023-07-14T15:59:54.879 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:20 ago on Fri 14 Jul 2023 03:57:34 PM UTC. 2023-07-14T15:59:57.698 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2023-07-14T15:59:57.747 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2023-07-14T15:59:58.242 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-07-14T15:59:58.280 INFO:teuthology.orchestra.run.smithi081.stdout:Updating Subscription Management repositories. 2023-07-14T15:59:58.331 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2023-07-14T15:59:58.866 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-07-14T15:59:59.258 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:06 ago on Fri 14 Jul 2023 03:57:52 PM UTC. 2023-07-14T15:59:59.334 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:02:03 ago on Fri 14 Jul 2023 03:57:55 PM UTC. 2023-07-14T15:59:59.896 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:25 ago on Fri 14 Jul 2023 03:57:34 PM UTC. 2023-07-14T16:00:01.041 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2023-07-14T16:00:01.139 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================================ 2023-07-14T16:00:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2023-07-14T16:00:01.140 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================================ 2023-07-14T16:00:01.140 INFO:teuthology.orchestra.run.smithi081.stdout:Installing group/module packages: 2023-07-14T16:00:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T16:00:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T16:00:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T16:00:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T16:00:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T16:00:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T16:00:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T16:00:01.142 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2023-07-14T16:00:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T16:00:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T16:00:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T16:00:01.142 INFO:teuthology.orchestra.run.smithi081.stdout:Installing module profiles: 2023-07-14T16:00:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: container-tools/common 2023-07-14T16:00:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:00:01.143 INFO:teuthology.orchestra.run.smithi081.stdout:Enabling module streams: 2023-07-14T16:00:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: container-tools rhel8 2023-07-14T16:00:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:00:01.143 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2023-07-14T16:00:01.143 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================================ 2023-07-14T16:00:01.143 INFO:teuthology.orchestra.run.smithi081.stdout:Install 10 Packages 2023-07-14T16:00:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:00:01.144 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 18 M 2023-07-14T16:00:01.144 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 65 M 2023-07-14T16:00:01.144 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2023-07-14T16:00:01.168 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2023-07-14T16:00:01.202 INFO:teuthology.orchestra.run.smithi081.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.5 MB/s | 94 kB 00:00 2023-07-14T16:00:01.219 INFO:teuthology.orchestra.run.smithi081.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 325 kB/s | 25 kB 00:00 2023-07-14T16:00:01.236 INFO:teuthology.orchestra.run.smithi081.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 437 kB/s | 41 kB 00:00 2023-07-14T16:00:01.253 INFO:teuthology.orchestra.run.smithi081.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 978 kB/s | 49 kB 00:00 2023-07-14T16:00:01.267 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================================ 2023-07-14T16:00:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2023-07-14T16:00:01.268 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================================ 2023-07-14T16:00:01.268 INFO:teuthology.orchestra.run.smithi164.stdout:Installing group/module packages: 2023-07-14T16:00:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T16:00:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T16:00:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T16:00:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T16:00:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T16:00:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T16:00:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T16:00:01.269 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2023-07-14T16:00:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T16:00:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T16:00:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T16:00:01.269 INFO:teuthology.orchestra.run.smithi164.stdout:Installing module profiles: 2023-07-14T16:00:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: container-tools/common 2023-07-14T16:00:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:00:01.270 INFO:teuthology.orchestra.run.smithi164.stdout:Enabling module streams: 2023-07-14T16:00:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: container-tools rhel8 2023-07-14T16:00:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:00:01.270 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2023-07-14T16:00:01.270 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================================================ 2023-07-14T16:00:01.270 INFO:teuthology.orchestra.run.smithi164.stdout:Install 10 Packages 2023-07-14T16:00:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:00:01.271 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 18 M 2023-07-14T16:00:01.271 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 65 M 2023-07-14T16:00:01.272 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2023-07-14T16:00:01.296 INFO:teuthology.orchestra.run.smithi081.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.8 MB/s | 524 kB 00:00 2023-07-14T16:00:01.320 INFO:teuthology.orchestra.run.smithi081.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-07-14T16:00:01.343 INFO:teuthology.orchestra.run.smithi164.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-07-14T16:00:01.354 INFO:teuthology.orchestra.run.smithi081.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-07-14T16:00:01.368 INFO:teuthology.orchestra.run.smithi164.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 255 kB/s | 25 kB 00:00 2023-07-14T16:00:01.393 INFO:teuthology.orchestra.run.smithi164.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 333 kB/s | 41 kB 00:00 2023-07-14T16:00:01.444 INFO:teuthology.orchestra.run.smithi164.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 491 kB/s | 49 kB 00:00 2023-07-14T16:00:01.463 INFO:teuthology.orchestra.run.smithi081.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-07-14T16:00:01.477 INFO:teuthology.orchestra.run.smithi164.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.7 MB/s | 524 kB 00:00 2023-07-14T16:00:01.502 INFO:teuthology.orchestra.run.smithi164.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-14T16:00:01.536 INFO:teuthology.orchestra.run.smithi164.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-07-14T16:00:01.605 INFO:teuthology.orchestra.run.smithi081.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 25 MB/s | 7.0 MB 00:00 2023-07-14T16:00:01.661 INFO:teuthology.orchestra.run.smithi164.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-07-14T16:00:01.730 INFO:teuthology.orchestra.run.smithi081.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-07-14T16:00:01.731 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2023-07-14T16:00:01.731 INFO:teuthology.orchestra.run.smithi081.stdout:Total 31 MB/s | 18 MB 00:00 2023-07-14T16:00:01.845 INFO:teuthology.orchestra.run.smithi164.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 21 MB/s | 7.0 MB 00:00 2023-07-14T16:00:01.901 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-07-14T16:00:01.945 INFO:teuthology.orchestra.run.smithi164.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 14 MB/s | 8.0 MB 00:00 2023-07-14T16:00:01.950 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2023-07-14T16:00:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:Total 27 MB/s | 18 MB 00:00 2023-07-14T16:00:02.006 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================================ 2023-07-14T16:00:02.007 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-07-14T16:00:02.007 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================================ 2023-07-14T16:00:02.007 INFO:teuthology.orchestra.run.smithi062.stdout:Installing group/module packages: 2023-07-14T16:00:02.007 INFO:teuthology.orchestra.run.smithi062.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T16:00:02.007 INFO:teuthology.orchestra.run.smithi062.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T16:00:02.008 INFO:teuthology.orchestra.run.smithi062.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T16:00:02.008 INFO:teuthology.orchestra.run.smithi062.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T16:00:02.008 INFO:teuthology.orchestra.run.smithi062.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T16:00:02.008 INFO:teuthology.orchestra.run.smithi062.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T16:00:02.008 INFO:teuthology.orchestra.run.smithi062.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T16:00:02.009 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-07-14T16:00:02.009 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T16:00:02.009 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T16:00:02.009 INFO:teuthology.orchestra.run.smithi062.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T16:00:02.009 INFO:teuthology.orchestra.run.smithi062.stdout:Installing module profiles: 2023-07-14T16:00:02.009 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools/common 2023-07-14T16:00:02.009 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:00:02.010 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling module streams: 2023-07-14T16:00:02.010 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools rhel8 2023-07-14T16:00:02.010 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:00:02.010 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-07-14T16:00:02.010 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================================ 2023-07-14T16:00:02.010 INFO:teuthology.orchestra.run.smithi062.stdout:Install 10 Packages 2023-07-14T16:00:02.011 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:00:02.011 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 18 M 2023-07-14T16:00:02.011 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 65 M 2023-07-14T16:00:02.011 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-07-14T16:00:02.062 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2023-07-14T16:00:02.085 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2023-07-14T16:00:02.085 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2023-07-14T16:00:02.100 INFO:teuthology.orchestra.run.smithi062.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 275 kB/s | 25 kB 00:00 2023-07-14T16:00:02.126 INFO:teuthology.orchestra.run.smithi062.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 812 kB/s | 94 kB 00:00 2023-07-14T16:00:02.151 INFO:teuthology.orchestra.run.smithi062.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 290 kB/s | 41 kB 00:00 2023-07-14T16:00:02.176 INFO:teuthology.orchestra.run.smithi062.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-07-14T16:00:02.210 INFO:teuthology.orchestra.run.smithi062.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-14T16:00:02.235 INFO:teuthology.orchestra.run.smithi062.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-14T16:00:02.273 INFO:teuthology.orchestra.run.smithi062.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-07-14T16:00:02.281 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2023-07-14T16:00:02.304 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2023-07-14T16:00:02.305 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2023-07-14T16:00:02.394 INFO:teuthology.orchestra.run.smithi062.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-07-14T16:00:02.553 INFO:teuthology.orchestra.run.smithi062.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 22 MB/s | 7.0 MB 00:00 2023-07-14T16:00:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-07-14T16:00:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-07-14T16:00:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:Total 28 MB/s | 18 MB 00:00 2023-07-14T16:00:02.665 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2023-07-14T16:00:02.667 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2023-07-14T16:00:02.807 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2023-07-14T16:00:02.812 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2023-07-14T16:00:02.995 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-07-14T16:00:03.019 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-07-14T16:00:03.019 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-07-14T16:00:03.204 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2023-07-14T16:00:03.392 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T16:00:03.393 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2023-07-14T16:00:03.539 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-07-14T16:00:03.544 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-07-14T16:00:03.565 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T16:00:03.580 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T16:00:03.784 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T16:00:03.807 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T16:00:03.994 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T16:00:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T16:00:04.100 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-07-14T16:00:04.244 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T16:00:04.305 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T16:00:04.435 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T16:00:04.483 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T16:00:04.603 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T16:00:04.721 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T16:00:04.970 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T16:00:05.238 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T16:00:05.347 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T16:00:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T16:00:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T16:00:06.208 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T16:00:06.407 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T16:00:06.458 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T16:00:06.637 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T16:00:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T16:00:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T16:00:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T16:00:07.080 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T16:00:07.162 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T16:00:07.236 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T16:00:07.237 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T16:00:07.237 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T16:00:07.237 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T16:00:07.237 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T16:00:07.237 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T16:00:07.237 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T16:00:07.237 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T16:00:07.238 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T16:00:07.238 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T16:00:07.420 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T16:00:07.485 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T16:00:07.485 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T16:00:07.485 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T16:00:07.486 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T16:00:07.486 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T16:00:07.486 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T16:00:07.486 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T16:00:07.487 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T16:00:07.487 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T16:00:07.487 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T16:00:07.596 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T16:00:07.937 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T16:00:08.362 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T16:00:08.362 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T16:00:08.363 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T16:00:08.363 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T16:00:08.363 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T16:00:08.363 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T16:00:08.363 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T16:00:08.363 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T16:00:08.363 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T16:00:08.363 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T16:00:08.776 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T16:00:08.776 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:16 ago on Fri 14 Jul 2023 03:57:52 PM UTC. 2023-07-14T16:00:08.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T16:00:08.944 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:02:13 ago on Fri 14 Jul 2023 03:57:55 PM UTC. 2023-07-14T16:00:09.847 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T16:00:09.847 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:35 ago on Fri 14 Jul 2023 03:57:34 PM UTC. 2023-07-14T16:00:14.296 INFO:teuthology.orchestra.run.smithi164.stdout:Installed products updated. 2023-07-14T16:00:14.507 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:00:14.507 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2023-07-14T16:00:14.507 INFO:teuthology.orchestra.run.smithi164.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:14.507 INFO:teuthology.orchestra.run.smithi164.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T16:00:14.507 INFO:teuthology.orchestra.run.smithi164.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:14.508 INFO:teuthology.orchestra.run.smithi164.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T16:00:14.508 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T16:00:14.508 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T16:00:14.508 INFO:teuthology.orchestra.run.smithi164.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:14.508 INFO:teuthology.orchestra.run.smithi164.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:14.508 INFO:teuthology.orchestra.run.smithi164.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T16:00:14.509 INFO:teuthology.orchestra.run.smithi164.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T16:00:14.509 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:00:14.509 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2023-07-14T16:00:14.711 DEBUG:teuthology.parallel:result is None 2023-07-14T16:00:14.764 INFO:teuthology.orchestra.run.smithi081.stdout:Installed products updated. 2023-07-14T16:00:15.013 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-07-14T16:00:15.016 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:00:15.017 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2023-07-14T16:00:15.017 INFO:teuthology.orchestra.run.smithi081.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:15.017 INFO:teuthology.orchestra.run.smithi081.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T16:00:15.017 INFO:teuthology.orchestra.run.smithi081.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:15.017 INFO:teuthology.orchestra.run.smithi081.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T16:00:15.017 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T16:00:15.017 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T16:00:15.018 INFO:teuthology.orchestra.run.smithi081.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:15.018 INFO:teuthology.orchestra.run.smithi081.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:15.018 INFO:teuthology.orchestra.run.smithi081.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T16:00:15.018 INFO:teuthology.orchestra.run.smithi081.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T16:00:15.018 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:00:15.018 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2023-07-14T16:00:15.235 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:00:15.235 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-07-14T16:00:15.235 INFO:teuthology.orchestra.run.smithi062.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:15.235 INFO:teuthology.orchestra.run.smithi062.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T16:00:15.236 INFO:teuthology.orchestra.run.smithi062.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:15.236 INFO:teuthology.orchestra.run.smithi062.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T16:00:15.236 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T16:00:15.236 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T16:00:15.236 INFO:teuthology.orchestra.run.smithi062.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:15.236 INFO:teuthology.orchestra.run.smithi062.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T16:00:15.236 INFO:teuthology.orchestra.run.smithi062.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T16:00:15.237 INFO:teuthology.orchestra.run.smithi062.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T16:00:15.237 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:00:15.237 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-07-14T16:00:15.252 DEBUG:teuthology.parallel:result is None 2023-07-14T16:00:15.452 DEBUG:teuthology.parallel:result is None 2023-07-14T16:00:15.453 INFO:teuthology.run_tasks:Running task install... 2023-07-14T16:00:15.469 DEBUG:teuthology.task.install:project ceph 2023-07-14T16:00:15.470 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}} 2023-07-14T16:00:15.470 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'} 2023-07-14T16:00:15.470 INFO:teuthology.task.install:Using flavor: default 2023-07-14T16:00:15.476 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-14T16:00:15.476 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-14T16:00:15.477 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T16:00:15.477 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:00:15.479 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T16:00:15.480 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:00:15.481 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-14T16:00:15.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:00:30.737 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T16:00:30.737 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T16:00:35.748 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T16:00:35.749 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T16:00:35.750 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-14T16:00:35.750 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-14T16:00:45.910 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T16:00:45.911 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:00:45.911 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T16:00:45.913 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T16:00:45.914 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:00:45.914 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T16:00:45.946 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T16:00:45.947 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:00:45.947 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T16:00:45.984 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T16:00:45.984 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T16:00:45.991 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T16:00:45.992 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T16:00:46.026 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T16:00:46.026 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T16:00:46.093 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T16:00:46.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T16:00:46.139 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T16:00:46.189 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T16:00:46.200 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T16:00:46.237 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T16:00:46.256 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2023-07-14T16:00:46.260 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2023-07-14T16:00:46.268 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2023-07-14T16:00:46.271 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2023-07-14T16:00:46.306 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2023-07-14T16:00:46.309 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2023-07-14T16:00:46.703 INFO:teuthology.orchestra.run.smithi081.stdout:Updating Subscription Management repositories. 2023-07-14T16:00:46.723 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2023-07-14T16:00:46.793 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-07-14T16:00:47.018 INFO:teuthology.orchestra.run.smithi081.stdout:44 files removed 2023-07-14T16:00:47.027 INFO:teuthology.orchestra.run.smithi164.stdout:44 files removed 2023-07-14T16:00:47.072 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T16:00:47.083 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T16:00:47.142 INFO:teuthology.orchestra.run.smithi062.stdout:44 files removed 2023-07-14T16:00:47.204 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T16:00:47.516 INFO:teuthology.orchestra.run.smithi081.stdout:Updating Subscription Management repositories. 2023-07-14T16:00:47.534 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2023-07-14T16:00:47.685 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-07-14T16:00:48.373 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 180 kB/s | 81 kB 00:00 2023-07-14T16:00:48.411 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 176 kB/s | 81 kB 00:00 2023-07-14T16:00:48.646 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 159 kB/s | 81 kB 00:00 2023-07-14T16:00:48.949 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-07-14T16:00:48.978 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-07-14T16:00:49.180 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2023-07-14T16:00:49.400 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 3.5 kB/s | 1.5 kB 00:00 2023-07-14T16:00:49.438 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 3.4 kB/s | 1.5 kB 00:00 2023-07-14T16:00:49.572 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2023-07-14T16:00:49.601 INFO:teuthology.orchestra.run.smithi081.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-07-14T16:00:49.638 INFO:teuthology.orchestra.run.smithi164.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-07-14T16:00:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-07-14T16:00:50.627 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2023-07-14T16:00:54.760 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 523 kB/s | 24 kB 00:00 2023-07-14T16:00:55.912 INFO:teuthology.orchestra.run.smithi081.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-07-14T16:00:56.113 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 2.4 MB/s | 16 MB 00:06 2023-07-14T16:00:59.847 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 16 MB 00:10 2023-07-14T16:01:00.180 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 268 kB/s | 24 kB 00:00 2023-07-14T16:01:01.224 INFO:teuthology.orchestra.run.smithi164.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 MB/s | 47 MB 00:00 2023-07-14T16:01:04.129 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 274 kB/s | 24 kB 00:00 2023-07-14T16:01:04.963 INFO:teuthology.orchestra.run.smithi081.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 47 MB 00:01 2023-07-14T16:01:05.274 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-07-14T16:01:09.824 INFO:teuthology.orchestra.run.smithi164.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-07-14T16:01:14.538 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-07-14T16:01:16.780 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T16:01:16.781 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T16:01:16.796 INFO:teuthology.orchestra.run.smithi081.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T16:01:16.797 INFO:teuthology.orchestra.run.smithi081.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T16:01:16.798 INFO:teuthology.orchestra.run.smithi081.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T16:01:16.800 INFO:teuthology.orchestra.run.smithi081.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T16:01:16.802 INFO:teuthology.orchestra.run.smithi081.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T16:01:16.805 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T16:01:16.807 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T16:01:16.946 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2023-07-14T16:01:16.958 INFO:teuthology.orchestra.run.smithi081.stdout:=========================================================================================================================== 2023-07-14T16:01:16.958 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2023-07-14T16:01:16.958 INFO:teuthology.orchestra.run.smithi081.stdout:=========================================================================================================================== 2023-07-14T16:01:16.959 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2023-07-14T16:01:16.959 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T16:01:16.959 INFO:teuthology.orchestra.run.smithi081.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T16:01:16.959 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T16:01:16.959 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T16:01:16.959 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T16:01:16.959 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T16:01:16.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T16:01:16.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T16:01:16.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T16:01:16.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T16:01:16.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T16:01:16.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T16:01:16.961 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T16:01:16.961 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T16:01:16.961 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T16:01:16.961 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T16:01:16.961 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T16:01:16.961 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T16:01:16.961 INFO:teuthology.orchestra.run.smithi081.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T16:01:16.961 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T16:01:16.962 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T16:01:16.962 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T16:01:16.962 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T16:01:16.962 INFO:teuthology.orchestra.run.smithi081.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T16:01:16.962 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T16:01:16.962 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T16:01:16.962 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T16:01:16.962 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T16:01:16.963 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T16:01:16.963 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T16:01:16.963 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T16:01:16.963 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T16:01:16.963 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T16:01:16.964 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T16:01:16.964 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T16:01:16.964 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T16:01:16.964 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T16:01:16.964 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2023-07-14T16:01:16.964 INFO:teuthology.orchestra.run.smithi081.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T16:01:16.964 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T16:01:16.965 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T16:01:16.965 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2023-07-14T16:01:16.965 INFO:teuthology.orchestra.run.smithi081.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T16:01:16.965 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T16:01:16.965 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T16:01:16.965 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T16:01:16.965 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T16:01:16.965 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T16:01:16.966 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T16:01:16.966 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T16:01:16.966 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T16:01:16.966 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T16:01:16.966 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T16:01:16.966 INFO:teuthology.orchestra.run.smithi081.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T16:01:16.966 INFO:teuthology.orchestra.run.smithi081.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T16:01:16.967 INFO:teuthology.orchestra.run.smithi081.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T16:01:16.967 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T16:01:16.967 INFO:teuthology.orchestra.run.smithi081.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T16:01:16.967 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T16:01:16.968 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T16:01:16.968 INFO:teuthology.orchestra.run.smithi081.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T16:01:16.968 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T16:01:16.968 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T16:01:16.968 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T16:01:16.969 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T16:01:16.969 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T16:01:16.969 INFO:teuthology.orchestra.run.smithi081.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T16:01:16.969 INFO:teuthology.orchestra.run.smithi081.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T16:01:16.969 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T16:01:16.969 INFO:teuthology.orchestra.run.smithi081.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T16:01:16.969 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T16:01:16.970 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T16:01:16.970 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T16:01:16.970 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T16:01:16.970 INFO:teuthology.orchestra.run.smithi081.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T16:01:16.970 INFO:teuthology.orchestra.run.smithi081.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T16:01:16.970 INFO:teuthology.orchestra.run.smithi081.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T16:01:16.970 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T16:01:16.970 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T16:01:16.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T16:01:16.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T16:01:16.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T16:01:16.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T16:01:16.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T16:01:16.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T16:01:16.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T16:01:16.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T16:01:16.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T16:01:16.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T16:01:16.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T16:01:16.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T16:01:16.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T16:01:16.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T16:01:16.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T16:01:16.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T16:01:16.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T16:01:16.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T16:01:16.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T16:01:16.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T16:01:16.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T16:01:16.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T16:01:16.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T16:01:16.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T16:01:16.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T16:01:16.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T16:01:16.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T16:01:16.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T16:01:16.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T16:01:16.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T16:01:16.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T16:01:16.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T16:01:16.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T16:01:16.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T16:01:16.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T16:01:16.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T16:01:16.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T16:01:16.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T16:01:16.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T16:01:16.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T16:01:16.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T16:01:16.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T16:01:16.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T16:01:16.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T16:01:16.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T16:01:16.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T16:01:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T16:01:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T16:01:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T16:01:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T16:01:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T16:01:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T16:01:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T16:01:16.978 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T16:01:16.979 INFO:teuthology.orchestra.run.smithi081.stdout:Enabling module streams: 2023-07-14T16:01:16.979 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql 10 2023-07-14T16:01:16.979 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:01:16.979 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2023-07-14T16:01:16.979 INFO:teuthology.orchestra.run.smithi081.stdout:=========================================================================================================================== 2023-07-14T16:01:16.979 INFO:teuthology.orchestra.run.smithi081.stdout:Install 128 Packages 2023-07-14T16:01:16.979 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 3 Packages 2023-07-14T16:01:16.980 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:01:16.980 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 205 M 2023-07-14T16:01:16.980 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2023-07-14T16:01:17.430 INFO:teuthology.orchestra.run.smithi081.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 26 kB/s | 6.4 kB 00:00 2023-07-14T16:01:18.049 INFO:teuthology.orchestra.run.smithi081.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.3 MB/s | 848 kB 00:00 2023-07-14T16:01:18.214 INFO:teuthology.orchestra.run.smithi081.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 1.0 MB/s | 165 kB 00:00 2023-07-14T16:01:20.173 INFO:teuthology.orchestra.run.smithi081.stdout:(4/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-07-14T16:01:20.298 INFO:teuthology.orchestra.run.smithi081.stdout:(5/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 1.7 MB/s | 5.4 MB 00:03 2023-07-14T16:01:21.305 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T16:01:21.307 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T16:01:21.322 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T16:01:21.323 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T16:01:21.324 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T16:01:21.325 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T16:01:21.327 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T16:01:21.331 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T16:01:21.332 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T16:01:21.456 INFO:teuthology.orchestra.run.smithi081.stdout:(6/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 1.2 MB/s | 1.5 MB 00:01 2023-07-14T16:01:21.472 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2023-07-14T16:01:21.485 INFO:teuthology.orchestra.run.smithi164.stdout:=========================================================================================================================== 2023-07-14T16:01:21.485 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2023-07-14T16:01:21.485 INFO:teuthology.orchestra.run.smithi164.stdout:=========================================================================================================================== 2023-07-14T16:01:21.486 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2023-07-14T16:01:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T16:01:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T16:01:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T16:01:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T16:01:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T16:01:21.486 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T16:01:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T16:01:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T16:01:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T16:01:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T16:01:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T16:01:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T16:01:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T16:01:21.487 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T16:01:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T16:01:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T16:01:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T16:01:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T16:01:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T16:01:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T16:01:21.488 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T16:01:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T16:01:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T16:01:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T16:01:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T16:01:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T16:01:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T16:01:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T16:01:21.489 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T16:01:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T16:01:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T16:01:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T16:01:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T16:01:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T16:01:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T16:01:21.490 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T16:01:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T16:01:21.491 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2023-07-14T16:01:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T16:01:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T16:01:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T16:01:21.491 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2023-07-14T16:01:21.491 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T16:01:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T16:01:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T16:01:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T16:01:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T16:01:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T16:01:21.492 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T16:01:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T16:01:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T16:01:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T16:01:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T16:01:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T16:01:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T16:01:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T16:01:21.493 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T16:01:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T16:01:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T16:01:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T16:01:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T16:01:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T16:01:21.494 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T16:01:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T16:01:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T16:01:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T16:01:21.495 INFO:teuthology.orchestra.run.smithi164.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T16:01:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T16:01:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T16:01:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T16:01:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T16:01:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T16:01:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T16:01:21.496 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T16:01:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T16:01:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T16:01:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T16:01:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T16:01:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T16:01:21.497 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T16:01:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T16:01:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T16:01:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T16:01:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T16:01:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T16:01:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T16:01:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T16:01:21.498 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T16:01:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T16:01:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T16:01:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T16:01:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T16:01:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T16:01:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T16:01:21.499 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T16:01:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T16:01:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T16:01:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T16:01:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T16:01:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T16:01:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T16:01:21.500 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T16:01:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T16:01:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T16:01:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T16:01:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T16:01:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T16:01:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T16:01:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T16:01:21.501 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T16:01:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T16:01:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T16:01:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T16:01:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T16:01:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T16:01:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T16:01:21.502 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T16:01:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T16:01:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T16:01:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T16:01:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T16:01:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T16:01:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T16:01:21.503 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T16:01:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T16:01:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T16:01:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T16:01:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T16:01:21.504 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T16:01:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T16:01:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T16:01:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T16:01:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T16:01:21.505 INFO:teuthology.orchestra.run.smithi164.stdout:Enabling module streams: 2023-07-14T16:01:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql 10 2023-07-14T16:01:21.505 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:01:21.506 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2023-07-14T16:01:21.506 INFO:teuthology.orchestra.run.smithi164.stdout:=========================================================================================================================== 2023-07-14T16:01:21.506 INFO:teuthology.orchestra.run.smithi164.stdout:Install 128 Packages 2023-07-14T16:01:21.506 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 3 Packages 2023-07-14T16:01:21.506 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:01:21.507 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 205 M 2023-07-14T16:01:21.507 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2023-07-14T16:01:21.907 INFO:teuthology.orchestra.run.smithi164.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-07-14T16:01:23.141 INFO:teuthology.orchestra.run.smithi164.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 688 kB/s | 848 kB 00:01 2023-07-14T16:01:23.424 INFO:teuthology.orchestra.run.smithi164.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 584 kB/s | 165 kB 00:00 2023-07-14T16:01:24.173 INFO:teuthology.orchestra.run.smithi081.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 1.2 MB/s | 4.6 MB 00:03 2023-07-14T16:01:25.388 INFO:teuthology.orchestra.run.smithi164.stdout:(4/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-07-14T16:01:25.691 INFO:teuthology.orchestra.run.smithi164.stdout:(5/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 1.3 MB/s | 5.4 MB 00:04 2023-07-14T16:01:26.475 INFO:teuthology.orchestra.run.smithi164.stdout:(6/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 1.4 MB/s | 1.5 MB 00:01 2023-07-14T16:01:26.882 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T16:01:26.883 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T16:01:26.898 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T16:01:26.900 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T16:01:26.901 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T16:01:26.902 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T16:01:26.904 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T16:01:26.908 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T16:01:26.909 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T16:01:27.063 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-07-14T16:01:27.077 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================================================== 2023-07-14T16:01:27.077 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-07-14T16:01:27.078 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================================================== 2023-07-14T16:01:27.078 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-07-14T16:01:27.078 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T16:01:27.078 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T16:01:27.078 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-14T16:01:27.078 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-14T16:01:27.078 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-14T16:01:27.079 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-14T16:01:27.079 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-14T16:01:27.079 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-14T16:01:27.079 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-14T16:01:27.079 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-14T16:01:27.079 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-14T16:01:27.079 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-14T16:01:27.080 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-14T16:01:27.080 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-14T16:01:27.080 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-14T16:01:27.080 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-14T16:01:27.080 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-14T16:01:27.080 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T16:01:27.080 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T16:01:27.081 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T16:01:27.081 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-14T16:01:27.081 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-14T16:01:27.081 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-14T16:01:27.081 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T16:01:27.081 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T16:01:27.081 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T16:01:27.081 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T16:01:27.082 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T16:01:27.082 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T16:01:27.082 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-14T16:01:27.082 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-14T16:01:27.082 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-14T16:01:27.082 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-14T16:01:27.084 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-14T16:01:27.084 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-14T16:01:27.084 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-14T16:01:27.085 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T16:01:27.085 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2023-07-14T16:01:27.085 INFO:teuthology.orchestra.run.smithi062.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T16:01:27.086 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-14T16:01:27.086 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-14T16:01:27.086 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-07-14T16:01:27.087 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T16:01:27.087 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-14T16:01:27.087 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-14T16:01:27.088 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-14T16:01:27.088 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-14T16:01:27.088 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-14T16:01:27.089 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-14T16:01:27.089 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-14T16:01:27.089 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-14T16:01:27.089 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T16:01:27.089 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T16:01:27.090 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T16:01:27.090 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T16:01:27.090 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T16:01:27.090 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-14T16:01:27.091 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T16:01:27.091 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T16:01:27.091 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T16:01:27.091 INFO:teuthology.orchestra.run.smithi062.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T16:01:27.092 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T16:01:27.092 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T16:01:27.092 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-14T16:01:27.092 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T16:01:27.092 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-14T16:01:27.093 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T16:01:27.093 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T16:01:27.093 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T16:01:27.093 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T16:01:27.093 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T16:01:27.094 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T16:01:27.094 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T16:01:27.094 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T16:01:27.094 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T16:01:27.094 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T16:01:27.094 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T16:01:27.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T16:01:27.094 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T16:01:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T16:01:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T16:01:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T16:01:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-14T16:01:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-14T16:01:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T16:01:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T16:01:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T16:01:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T16:01:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T16:01:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T16:01:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T16:01:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T16:01:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T16:01:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T16:01:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T16:01:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T16:01:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T16:01:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T16:01:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T16:01:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T16:01:27.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T16:01:27.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T16:01:27.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T16:01:27.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T16:01:27.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T16:01:27.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T16:01:27.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T16:01:27.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T16:01:27.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T16:01:27.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T16:01:27.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T16:01:27.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T16:01:27.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T16:01:27.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T16:01:27.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T16:01:27.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T16:01:27.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T16:01:27.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T16:01:27.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T16:01:27.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T16:01:27.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T16:01:27.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T16:01:27.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T16:01:27.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T16:01:27.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T16:01:27.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T16:01:27.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T16:01:27.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T16:01:27.101 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T16:01:27.101 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T16:01:27.101 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T16:01:27.102 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T16:01:27.102 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T16:01:27.102 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling module streams: 2023-07-14T16:01:27.102 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql 10 2023-07-14T16:01:27.102 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:01:27.102 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-07-14T16:01:27.102 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================================================== 2023-07-14T16:01:27.102 INFO:teuthology.orchestra.run.smithi062.stdout:Install 128 Packages 2023-07-14T16:01:27.103 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 3 Packages 2023-07-14T16:01:27.103 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:01:27.103 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 205 M 2023-07-14T16:01:27.104 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-07-14T16:01:27.434 INFO:teuthology.orchestra.run.smithi062.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-07-14T16:01:28.392 INFO:teuthology.orchestra.run.smithi062.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 886 kB/s | 848 kB 00:00 2023-07-14T16:01:28.559 INFO:teuthology.orchestra.run.smithi062.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 993 kB/s | 165 kB 00:00 2023-07-14T16:01:28.750 INFO:teuthology.orchestra.run.smithi164.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 1.5 MB/s | 4.6 MB 00:03 2023-07-14T16:01:29.951 INFO:teuthology.orchestra.run.smithi062.stdout:(4/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 1.9 MB/s | 5.4 MB 00:02 2023-07-14T16:01:30.134 INFO:teuthology.orchestra.run.smithi062.stdout:(5/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 1.5 MB/s | 2.3 MB 00:01 2023-07-14T16:01:30.468 INFO:teuthology.orchestra.run.smithi062.stdout:(6/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 3.0 MB/s | 1.5 MB 00:00 2023-07-14T16:01:31.493 INFO:teuthology.orchestra.run.smithi062.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 3.4 MB/s | 4.6 MB 00:01 2023-07-14T16:01:32.235 INFO:teuthology.orchestra.run.smithi062.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 4.3 MB/s | 22 MB 00:05 2023-07-14T16:01:32.360 INFO:teuthology.orchestra.run.smithi062.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 195 kB/s | 24 kB 00:00 2023-07-14T16:01:32.919 INFO:teuthology.orchestra.run.smithi062.stdout:(10/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 7.2 MB/s | 18 MB 00:02 2023-07-14T16:01:33.077 INFO:teuthology.orchestra.run.smithi062.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 1.6 MB/s | 258 kB 00:00 2023-07-14T16:01:33.178 INFO:teuthology.orchestra.run.smithi062.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 310 kB/s | 31 kB 00:00 2023-07-14T16:01:33.320 INFO:teuthology.orchestra.run.smithi062.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 5.3 MB/s | 771 kB 00:00 2023-07-14T16:01:33.503 INFO:teuthology.orchestra.run.smithi062.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 749 kB/s | 137 kB 00:00 2023-07-14T16:01:33.603 INFO:teuthology.orchestra.run.smithi062.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 1.4 MB/s | 139 kB 00:00 2023-07-14T16:01:33.720 INFO:teuthology.orchestra.run.smithi062.stdout:(16/131): libradosstriper1-18.0.0-4757.g3fc0089 4.0 MB/s | 473 kB 00:00 2023-07-14T16:01:33.789 INFO:teuthology.orchestra.run.smithi081.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 1.3 MB/s | 22 MB 00:16 2023-07-14T16:01:33.997 INFO:teuthology.orchestra.run.smithi081.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 117 kB/s | 24 kB 00:00 2023-07-14T16:01:34.279 INFO:teuthology.orchestra.run.smithi062.stdout:(17/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 4.8 MB/s | 13 MB 00:02 2023-07-14T16:01:34.487 INFO:teuthology.orchestra.run.smithi062.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 215 kB/s | 45 kB 00:00 2023-07-14T16:01:34.571 INFO:teuthology.orchestra.run.smithi062.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 1.4 MB/s | 119 kB 00:00 2023-07-14T16:01:35.355 INFO:teuthology.orchestra.run.smithi062.stdout:(20/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 4.1 MB/s | 6.7 MB 00:01 2023-07-14T16:01:35.380 INFO:teuthology.orchestra.run.smithi062.stdout:(21/131): python3-cephfs-18.0.0-4757.g3fc0089c. 243 kB/s | 197 kB 00:00 2023-07-14T16:01:35.522 INFO:teuthology.orchestra.run.smithi062.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 2.5 MB/s | 364 kB 00:00 2023-07-14T16:01:35.547 INFO:teuthology.orchestra.run.smithi062.stdout:(23/131): python3-rados-18.0.0-4757.g3fc0089c.e 1.9 MB/s | 371 kB 00:00 2023-07-14T16:01:35.598 INFO:teuthology.orchestra.run.smithi062.stdout:(24/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 1.4 MB/s | 107 kB 00:00 2023-07-14T16:01:35.623 INFO:teuthology.orchestra.run.smithi062.stdout:(25/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.2 MB/s | 94 kB 00:00 2023-07-14T16:01:35.756 INFO:teuthology.orchestra.run.smithi062.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.3 MB/s | 179 kB 00:00 2023-07-14T16:01:35.856 INFO:teuthology.orchestra.run.smithi062.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 233 kB/s | 23 kB 00:00 2023-07-14T16:01:35.999 INFO:teuthology.orchestra.run.smithi062.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 906 kB/s | 129 kB 00:00 2023-07-14T16:01:36.464 INFO:teuthology.orchestra.run.smithi081.stdout:(10/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 1.2 MB/s | 18 MB 00:15 2023-07-14T16:01:36.482 INFO:teuthology.orchestra.run.smithi062.stdout:(29/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 3.9 MB/s | 3.4 MB 00:00 2023-07-14T16:01:36.524 INFO:teuthology.orchestra.run.smithi062.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 3.3 MB/s | 1.7 MB 00:00 2023-07-14T16:01:36.606 INFO:teuthology.orchestra.run.smithi081.stdout:(11/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 1.1 MB/s | 13 MB 00:12 2023-07-14T16:01:36.633 INFO:teuthology.orchestra.run.smithi062.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 2.2 MB/s | 240 kB 00:00 2023-07-14T16:01:36.700 INFO:teuthology.orchestra.run.smithi062.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 715 kB/s | 47 kB 00:00 2023-07-14T16:01:36.714 INFO:teuthology.orchestra.run.smithi081.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 288 kB/s | 31 kB 00:00 2023-07-14T16:01:36.767 INFO:teuthology.orchestra.run.smithi062.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 218 kB/s | 15 kB 00:00 2023-07-14T16:01:36.774 INFO:teuthology.orchestra.run.smithi081.stdout:(13/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 836 kB/s | 258 kB 00:00 2023-07-14T16:01:36.875 INFO:teuthology.orchestra.run.smithi062.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 2.3 MB/s | 258 kB 00:00 2023-07-14T16:01:36.915 INFO:teuthology.orchestra.run.smithi081.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 968 kB/s | 137 kB 00:00 2023-07-14T16:01:36.951 INFO:teuthology.orchestra.run.smithi062.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 2.2 MB/s | 166 kB 00:00 2023-07-14T16:01:37.034 INFO:teuthology.orchestra.run.smithi062.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 217 kB/s | 18 kB 00:00 2023-07-14T16:01:37.074 INFO:teuthology.orchestra.run.smithi081.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 880 kB/s | 139 kB 00:00 2023-07-14T16:01:37.241 INFO:teuthology.orchestra.run.smithi164.stdout:(8/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x86 1.6 MB/s | 18 MB 00:10 2023-07-14T16:01:37.333 INFO:teuthology.orchestra.run.smithi164.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 267 kB/s | 24 kB 00:00 2023-07-14T16:01:37.374 INFO:teuthology.orchestra.run.smithi081.stdout:(16/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 1.1 MB/s | 771 kB 00:00 2023-07-14T16:01:37.551 INFO:teuthology.orchestra.run.smithi062.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 593 kB/s | 306 kB 00:00 2023-07-14T16:01:37.660 INFO:teuthology.orchestra.run.smithi062.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 543 kB/s | 59 kB 00:00 2023-07-14T16:01:37.666 INFO:teuthology.orchestra.run.smithi081.stdout:(17/131): libradosstriper1-18.0.0-4757.g3fc0089 800 kB/s | 473 kB 00:00 2023-07-14T16:01:37.749 INFO:teuthology.orchestra.run.smithi081.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 538 kB/s | 45 kB 00:00 2023-07-14T16:01:37.760 INFO:teuthology.orchestra.run.smithi062.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 754 kB/s | 75 kB 00:00 2023-07-14T16:01:37.983 INFO:teuthology.orchestra.run.smithi081.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 509 kB/s | 119 kB 00:00 2023-07-14T16:01:38.325 INFO:teuthology.orchestra.run.smithi081.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 576 kB/s | 197 kB 00:00 2023-07-14T16:01:38.427 INFO:teuthology.orchestra.run.smithi062.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 714 kB/s | 476 kB 00:00 2023-07-14T16:01:38.586 INFO:teuthology.orchestra.run.smithi062.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 276 kB/s | 44 kB 00:00 2023-07-14T16:01:38.803 INFO:teuthology.orchestra.run.smithi062.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 856 kB/s | 185 kB 00:00 2023-07-14T16:01:38.903 INFO:teuthology.orchestra.run.smithi062.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 331 kB/s | 33 kB 00:00 2023-07-14T16:01:38.917 INFO:teuthology.orchestra.run.smithi081.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 627 kB/s | 371 kB 00:00 2023-07-14T16:01:38.987 INFO:teuthology.orchestra.run.smithi062.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 196 kB/s | 16 kB 00:00 2023-07-14T16:01:39.070 INFO:teuthology.orchestra.run.smithi062.stdout:(45/131): ceph-mgr-diskprediction-local-18.0.0- 2.9 MB/s | 7.4 MB 00:02 2023-07-14T16:01:39.220 INFO:teuthology.orchestra.run.smithi062.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 740 kB/s | 173 kB 00:00 2023-07-14T16:01:39.404 INFO:teuthology.orchestra.run.smithi062.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 443 kB/s | 81 kB 00:00 2023-07-14T16:01:39.418 INFO:teuthology.orchestra.run.smithi081.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 729 kB/s | 364 kB 00:00 2023-07-14T16:01:39.479 INFO:teuthology.orchestra.run.smithi062.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 149 kB/s | 11 kB 00:00 2023-07-14T16:01:39.563 INFO:teuthology.orchestra.run.smithi062.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-07-14T16:01:39.576 INFO:teuthology.orchestra.run.smithi081.stdout:(23/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 675 kB/s | 107 kB 00:00 2023-07-14T16:01:39.588 INFO:teuthology.orchestra.run.smithi062.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 743 kB/s | 384 kB 00:00 2023-07-14T16:01:39.672 INFO:teuthology.orchestra.run.smithi062.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 585 kB/s | 49 kB 00:00 2023-07-14T16:01:39.726 INFO:teuthology.orchestra.run.smithi081.stdout:(24/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 630 kB/s | 94 kB 00:00 2023-07-14T16:01:39.747 INFO:teuthology.orchestra.run.smithi062.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 782 kB/s | 59 kB 00:00 2023-07-14T16:01:39.823 INFO:teuthology.orchestra.run.smithi062.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 808 kB/s | 60 kB 00:00 2023-07-14T16:01:40.015 INFO:teuthology.orchestra.run.smithi062.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-07-14T16:01:40.107 INFO:teuthology.orchestra.run.smithi062.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-07-14T16:01:40.190 INFO:teuthology.orchestra.run.smithi062.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 405 kB/s | 34 kB 00:00 2023-07-14T16:01:40.357 INFO:teuthology.orchestra.run.smithi062.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-07-14T16:01:40.475 INFO:teuthology.orchestra.run.smithi062.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 525 kB/s | 61 kB 00:00 2023-07-14T16:01:40.575 INFO:teuthology.orchestra.run.smithi062.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 189 kB/s | 19 kB 00:00 2023-07-14T16:01:40.658 INFO:teuthology.orchestra.run.smithi062.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 288 kB/s | 24 kB 00:00 2023-07-14T16:01:40.733 INFO:teuthology.orchestra.run.smithi062.stdout:(61/131): python3-kubernetes-11.0.0-6.el8.noarc 1.0 MB/s | 1.2 MB 00:01 2023-07-14T16:01:40.750 INFO:teuthology.orchestra.run.smithi164.stdout:(10/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 1.1 MB/s | 13 MB 00:11 2023-07-14T16:01:40.759 INFO:teuthology.orchestra.run.smithi062.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 292 kB/s | 29 kB 00:00 2023-07-14T16:01:40.826 INFO:teuthology.orchestra.run.smithi062.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 300 kB/s | 27 kB 00:00 2023-07-14T16:01:40.959 INFO:teuthology.orchestra.run.smithi062.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-07-14T16:01:41.025 INFO:teuthology.orchestra.run.smithi164.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 940 kB/s | 258 kB 00:00 2023-07-14T16:01:41.059 INFO:teuthology.orchestra.run.smithi062.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 614 kB/s | 61 kB 00:00 2023-07-14T16:01:41.085 INFO:teuthology.orchestra.run.smithi062.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 971 kB/s | 251 kB 00:00 2023-07-14T16:01:41.108 INFO:teuthology.orchestra.run.smithi164.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 373 kB/s | 31 kB 00:00 2023-07-14T16:01:41.135 INFO:teuthology.orchestra.run.smithi062.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-07-14T16:01:41.152 INFO:teuthology.orchestra.run.smithi062.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 343 kB/s | 23 kB 00:00 2023-07-14T16:01:41.177 INFO:teuthology.orchestra.run.smithi062.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T16:01:41.606 INFO:teuthology.orchestra.run.smithi062.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 32 MB/s | 14 MB 00:00 2023-07-14T16:01:41.670 INFO:teuthology.orchestra.run.smithi062.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.7 MB/s | 110 kB 00:00 2023-07-14T16:01:41.709 INFO:teuthology.orchestra.run.smithi164.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 1.3 MB/s | 771 kB 00:00 2023-07-14T16:01:41.711 INFO:teuthology.orchestra.run.smithi062.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-14T16:01:41.770 INFO:teuthology.orchestra.run.smithi062.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-07-14T16:01:41.826 INFO:teuthology.orchestra.run.smithi164.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 1.2 MB/s | 137 kB 00:00 2023-07-14T16:01:41.828 INFO:teuthology.orchestra.run.smithi062.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-14T16:01:41.879 INFO:teuthology.orchestra.run.smithi062.stdout:(75/131): thrift-0.13.0-2.el8.x86_64.rpm 2.3 MB/s | 1.7 MB 00:00 2023-07-14T16:01:41.904 INFO:teuthology.orchestra.run.smithi062.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 515 kB/s | 39 kB 00:00 2023-07-14T16:01:41.967 INFO:teuthology.orchestra.run.smithi164.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 982 kB/s | 139 kB 00:00 2023-07-14T16:01:41.980 INFO:teuthology.orchestra.run.smithi062.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-07-14T16:01:42.016 INFO:teuthology.orchestra.run.smithi062.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2023-07-14T16:01:42.031 INFO:teuthology.orchestra.run.smithi062.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.0 MB/s | 157 kB 00:00 2023-07-14T16:01:42.072 INFO:teuthology.orchestra.run.smithi062.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-07-14T16:01:42.101 INFO:teuthology.orchestra.run.smithi081.stdout:(25/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 1.4 MB/s | 6.7 MB 00:04 2023-07-14T16:01:42.122 INFO:teuthology.orchestra.run.smithi062.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-07-14T16:01:42.206 INFO:teuthology.orchestra.run.smithi062.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 318 kB/s | 27 kB 00:00 2023-07-14T16:01:42.243 INFO:teuthology.orchestra.run.smithi081.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.2 MB/s | 179 kB 00:00 2023-07-14T16:01:42.265 INFO:teuthology.orchestra.run.smithi062.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 5.4 MB/s | 320 kB 00:00 2023-07-14T16:01:42.342 INFO:teuthology.orchestra.run.smithi081.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 234 kB/s | 23 kB 00:00 2023-07-14T16:01:42.343 INFO:teuthology.orchestra.run.smithi164.stdout:(16/131): libradosstriper1-18.0.0-4757.g3fc0089 1.2 MB/s | 473 kB 00:00 2023-07-14T16:01:42.357 INFO:teuthology.orchestra.run.smithi062.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-07-14T16:01:42.382 INFO:teuthology.orchestra.run.smithi062.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 376 kB/s | 44 kB 00:00 2023-07-14T16:01:42.441 INFO:teuthology.orchestra.run.smithi062.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-07-14T16:01:42.460 INFO:teuthology.orchestra.run.smithi081.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 1.1 MB/s | 129 kB 00:00 2023-07-14T16:01:42.524 INFO:teuthology.orchestra.run.smithi062.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 22 MB/s | 3.7 MB 00:00 2023-07-14T16:01:42.550 INFO:teuthology.orchestra.run.smithi062.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.0 MB/s | 225 kB 00:00 2023-07-14T16:01:42.575 INFO:teuthology.orchestra.run.smithi062.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2023-07-14T16:01:42.600 INFO:teuthology.orchestra.run.smithi062.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-07-14T16:01:42.625 INFO:teuthology.orchestra.run.smithi062.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-07-14T16:01:42.651 INFO:teuthology.orchestra.run.smithi062.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-14T16:01:42.693 INFO:teuthology.orchestra.run.smithi062.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-14T16:01:42.776 INFO:teuthology.orchestra.run.smithi062.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-07-14T16:01:42.901 INFO:teuthology.orchestra.run.smithi062.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 19 MB/s | 4.9 MB 00:00 2023-07-14T16:01:42.952 INFO:teuthology.orchestra.run.smithi062.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-07-14T16:01:42.969 INFO:teuthology.orchestra.run.smithi081.stdout:(29/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 1.1 MB/s | 3.4 MB 00:03 2023-07-14T16:01:43.044 INFO:teuthology.orchestra.run.smithi062.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-07-14T16:01:43.077 INFO:teuthology.orchestra.run.smithi062.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2023-07-14T16:01:43.146 INFO:teuthology.orchestra.run.smithi062.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2023-07-14T16:01:43.261 INFO:teuthology.orchestra.run.smithi062.stdout:(100/131): postgresql-server-10.21-2.module+el8 23 MB/s | 5.1 MB 00:00 2023-07-14T16:01:43.303 INFO:teuthology.orchestra.run.smithi062.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-07-14T16:01:43.362 INFO:teuthology.orchestra.run.smithi062.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 466 kB/s | 47 kB 00:00 2023-07-14T16:01:43.387 INFO:teuthology.orchestra.run.smithi062.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.3 MB/s | 200 kB 00:00 2023-07-14T16:01:43.404 INFO:teuthology.orchestra.run.smithi062.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-14T16:01:43.429 INFO:teuthology.orchestra.run.smithi062.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-14T16:01:43.454 INFO:teuthology.orchestra.run.smithi062.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 960 kB/s | 48 kB 00:00 2023-07-14T16:01:43.478 INFO:teuthology.orchestra.run.smithi081.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 1.7 MB/s | 1.7 MB 00:01 2023-07-14T16:01:43.480 INFO:teuthology.orchestra.run.smithi062.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-14T16:01:43.505 INFO:teuthology.orchestra.run.smithi062.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 847 kB/s | 43 kB 00:00 2023-07-14T16:01:43.531 INFO:teuthology.orchestra.run.smithi062.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-07-14T16:01:43.556 INFO:teuthology.orchestra.run.smithi062.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-07-14T16:01:43.703 INFO:teuthology.orchestra.run.smithi081.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 1.0 MB/s | 240 kB 00:00 2023-07-14T16:01:43.787 INFO:teuthology.orchestra.run.smithi081.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 568 kB/s | 47 kB 00:00 2023-07-14T16:01:43.865 INFO:teuthology.orchestra.run.smithi062.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 1.6 MB/s | 560 kB 00:00 2023-07-14T16:01:43.879 INFO:teuthology.orchestra.run.smithi081.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 159 kB/s | 15 kB 00:00 2023-07-14T16:01:43.919 INFO:teuthology.orchestra.run.smithi062.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 52 kB/s | 19 kB 00:00 2023-07-14T16:01:43.949 INFO:teuthology.orchestra.run.smithi062.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.8 MB/s | 155 kB 00:00 2023-07-14T16:01:43.982 INFO:teuthology.orchestra.run.smithi062.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.6 MB/s | 559 kB 00:00 2023-07-14T16:01:44.008 INFO:teuthology.orchestra.run.smithi062.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-07-14T16:01:44.050 INFO:teuthology.orchestra.run.smithi062.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-07-14T16:01:44.083 INFO:teuthology.orchestra.run.smithi062.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-07-14T16:01:44.125 INFO:teuthology.orchestra.run.smithi062.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-07-14T16:01:44.154 INFO:teuthology.orchestra.run.smithi081.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 940 kB/s | 258 kB 00:00 2023-07-14T16:01:44.167 INFO:teuthology.orchestra.run.smithi062.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-14T16:01:44.226 INFO:teuthology.orchestra.run.smithi062.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 13 MB/s | 2.3 MB 00:00 2023-07-14T16:01:44.251 INFO:teuthology.orchestra.run.smithi062.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 718 kB/s | 60 kB 00:00 2023-07-14T16:01:44.276 INFO:teuthology.orchestra.run.smithi062.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-14T16:01:44.301 INFO:teuthology.orchestra.run.smithi062.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-14T16:01:44.327 INFO:teuthology.orchestra.run.smithi062.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-07-14T16:01:44.377 INFO:teuthology.orchestra.run.smithi062.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.4 MB/s | 644 kB 00:00 2023-07-14T16:01:44.388 INFO:teuthology.orchestra.run.smithi081.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 713 kB/s | 166 kB 00:00 2023-07-14T16:01:44.402 INFO:teuthology.orchestra.run.smithi062.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-07-14T16:01:44.427 INFO:teuthology.orchestra.run.smithi062.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-14T16:01:44.488 INFO:teuthology.orchestra.run.smithi081.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 180 kB/s | 18 kB 00:00 2023-07-14T16:01:44.493 INFO:teuthology.orchestra.run.smithi164.stdout:(17/131): ceph-common-18.0.0-4757.g3fc0089c.el8 989 kB/s | 22 MB 00:22 2023-07-14T16:01:44.601 INFO:teuthology.orchestra.run.smithi164.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 414 kB/s | 45 kB 00:00 2023-07-14T16:01:44.785 INFO:teuthology.orchestra.run.smithi164.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 648 kB/s | 119 kB 00:00 2023-07-14T16:01:45.043 INFO:teuthology.orchestra.run.smithi164.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 762 kB/s | 197 kB 00:00 2023-07-14T16:01:45.044 INFO:teuthology.orchestra.run.smithi081.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 557 kB/s | 306 kB 00:00 2023-07-14T16:01:45.138 INFO:teuthology.orchestra.run.smithi081.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 589 kB/s | 59 kB 00:00 2023-07-14T16:01:45.230 INFO:teuthology.orchestra.run.smithi081.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 819 kB/s | 75 kB 00:00 2023-07-14T16:01:45.369 INFO:teuthology.orchestra.run.smithi164.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 1.1 MB/s | 371 kB 00:00 2023-07-14T16:01:45.397 INFO:teuthology.orchestra.run.smithi081.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.8 MB/s | 476 kB 00:00 2023-07-14T16:01:45.481 INFO:teuthology.orchestra.run.smithi081.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 525 kB/s | 44 kB 00:00 2023-07-14T16:01:45.565 INFO:teuthology.orchestra.run.smithi081.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-07-14T16:01:45.594 INFO:teuthology.orchestra.run.smithi164.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 1.6 MB/s | 364 kB 00:00 2023-07-14T16:01:45.640 INFO:teuthology.orchestra.run.smithi081.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 440 kB/s | 33 kB 00:00 2023-07-14T16:01:45.699 INFO:teuthology.orchestra.run.smithi081.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 280 kB/s | 16 kB 00:00 2023-07-14T16:01:45.711 INFO:teuthology.orchestra.run.smithi164.stdout:(23/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 916 kB/s | 107 kB 00:00 2023-07-14T16:01:45.766 INFO:teuthology.orchestra.run.smithi081.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-07-14T16:01:45.794 INFO:teuthology.orchestra.run.smithi164.stdout:(24/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.1 MB/s | 94 kB 00:00 2023-07-14T16:01:45.866 INFO:teuthology.orchestra.run.smithi081.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.8 MB/s | 384 kB 00:00 2023-07-14T16:01:46.050 INFO:teuthology.orchestra.run.smithi081.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 444 kB/s | 81 kB 00:00 2023-07-14T16:01:46.094 INFO:teuthology.orchestra.run.smithi062.stdout:(128/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 2.1 MB/s | 3.6 MB 00:01 2023-07-14T16:01:46.117 INFO:teuthology.orchestra.run.smithi081.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-07-14T16:01:46.153 INFO:teuthology.orchestra.run.smithi062.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-14T16:01:46.192 INFO:teuthology.orchestra.run.smithi081.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-07-14T16:01:46.401 INFO:teuthology.orchestra.run.smithi081.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2023-07-14T16:01:46.493 INFO:teuthology.orchestra.run.smithi081.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 533 kB/s | 49 kB 00:00 2023-07-14T16:01:46.560 INFO:teuthology.orchestra.run.smithi081.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 881 kB/s | 59 kB 00:00 2023-07-14T16:01:46.635 INFO:teuthology.orchestra.run.smithi081.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 804 kB/s | 60 kB 00:00 2023-07-14T16:01:46.736 INFO:teuthology.orchestra.run.smithi081.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-07-14T16:01:46.836 INFO:teuthology.orchestra.run.smithi081.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-07-14T16:01:46.919 INFO:teuthology.orchestra.run.smithi081.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 405 kB/s | 34 kB 00:00 2023-07-14T16:01:47.012 INFO:teuthology.orchestra.run.smithi081.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-07-14T16:01:47.103 INFO:teuthology.orchestra.run.smithi081.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 672 kB/s | 61 kB 00:00 2023-07-14T16:01:47.203 INFO:teuthology.orchestra.run.smithi164.stdout:(25/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 1.4 MB/s | 6.7 MB 00:04 2023-07-14T16:01:47.204 INFO:teuthology.orchestra.run.smithi081.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-07-14T16:01:47.263 INFO:teuthology.orchestra.run.smithi081.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 412 kB/s | 24 kB 00:00 2023-07-14T16:01:47.338 INFO:teuthology.orchestra.run.smithi081.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 390 kB/s | 29 kB 00:00 2023-07-14T16:01:47.361 INFO:teuthology.orchestra.run.smithi164.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.1 MB/s | 179 kB 00:00 2023-07-14T16:01:47.438 INFO:teuthology.orchestra.run.smithi081.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 275 kB/s | 27 kB 00:00 2023-07-14T16:01:47.453 INFO:teuthology.orchestra.run.smithi164.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 252 kB/s | 23 kB 00:00 2023-07-14T16:01:47.522 INFO:teuthology.orchestra.run.smithi081.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-07-14T16:01:47.595 INFO:teuthology.orchestra.run.smithi164.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 909 kB/s | 129 kB 00:00 2023-07-14T16:01:47.622 INFO:teuthology.orchestra.run.smithi081.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.5 MB/s | 251 kB 00:00 2023-07-14T16:01:47.673 INFO:teuthology.orchestra.run.smithi081.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-07-14T16:01:47.740 INFO:teuthology.orchestra.run.smithi081.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 1.3 MB/s | 85 kB 00:00 2023-07-14T16:01:47.790 INFO:teuthology.orchestra.run.smithi081.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 461 kB/s | 23 kB 00:00 2023-07-14T16:01:47.941 INFO:teuthology.orchestra.run.smithi081.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-07-14T16:01:47.962 INFO:teuthology.orchestra.run.smithi164.stdout:(29/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 1.6 MB/s | 3.4 MB 00:02 2023-07-14T16:01:47.991 INFO:teuthology.orchestra.run.smithi081.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-07-14T16:01:48.050 INFO:teuthology.orchestra.run.smithi081.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 803 kB/s | 47 kB 00:00 2023-07-14T16:01:48.091 INFO:teuthology.orchestra.run.smithi081.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-07-14T16:01:48.134 INFO:teuthology.orchestra.run.smithi081.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-14T16:01:48.175 INFO:teuthology.orchestra.run.smithi081.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-14T16:01:48.218 INFO:teuthology.orchestra.run.smithi081.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-07-14T16:01:48.268 INFO:teuthology.orchestra.run.smithi081.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-14T16:01:48.310 INFO:teuthology.orchestra.run.smithi081.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-14T16:01:48.352 INFO:teuthology.orchestra.run.smithi081.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-07-14T16:01:48.394 INFO:teuthology.orchestra.run.smithi081.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-07-14T16:01:48.420 INFO:teuthology.orchestra.run.smithi062.stdout:(130/131): librados2-18.0.0-4757.g3fc0089c.el8. 934 kB/s | 3.7 MB 00:04 2023-07-14T16:01:48.429 INFO:teuthology.orchestra.run.smithi164.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 2.1 MB/s | 1.7 MB 00:00 2023-07-14T16:01:48.453 INFO:teuthology.orchestra.run.smithi081.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-07-14T16:01:48.511 INFO:teuthology.orchestra.run.smithi081.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 322 kB/s | 19 kB 00:00 2023-07-14T16:01:48.553 INFO:teuthology.orchestra.run.smithi081.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.7 MB/s | 155 kB 00:00 2023-07-14T16:01:48.604 INFO:teuthology.orchestra.run.smithi081.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 11 MB/s | 559 kB 00:00 2023-07-14T16:01:48.645 INFO:teuthology.orchestra.run.smithi081.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 434 kB/s | 18 kB 00:00 2023-07-14T16:01:48.696 INFO:teuthology.orchestra.run.smithi081.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-07-14T16:01:48.738 INFO:teuthology.orchestra.run.smithi081.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-07-14T16:01:48.829 INFO:teuthology.orchestra.run.smithi081.stdout:(86/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 2.3 MB 00:00 2023-07-14T16:01:48.863 INFO:teuthology.orchestra.run.smithi164.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 555 kB/s | 240 kB 00:00 2023-07-14T16:01:48.872 INFO:teuthology.orchestra.run.smithi081.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-14T16:01:48.914 INFO:teuthology.orchestra.run.smithi081.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2023-07-14T16:01:48.938 INFO:teuthology.orchestra.run.smithi164.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 634 kB/s | 47 kB 00:00 2023-07-14T16:01:49.013 INFO:teuthology.orchestra.run.smithi164.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 193 kB/s | 15 kB 00:00 2023-07-14T16:01:49.040 INFO:teuthology.orchestra.run.smithi081.stdout:(89/131): ceph-mgr-diskprediction-local-18.0.0- 1.2 MB/s | 7.4 MB 00:06 2023-07-14T16:01:49.064 INFO:teuthology.orchestra.run.smithi081.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 399 kB/s | 60 kB 00:00 2023-07-14T16:01:49.115 INFO:teuthology.orchestra.run.smithi081.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-07-14T16:01:49.140 INFO:teuthology.orchestra.run.smithi081.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 982 kB/s | 98 kB 00:00 2023-07-14T16:01:49.165 INFO:teuthology.orchestra.run.smithi081.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 772 kB/s | 39 kB 00:00 2023-07-14T16:01:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-14T16:01:49.224 INFO:teuthology.orchestra.run.smithi081.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-14T16:01:49.250 INFO:teuthology.orchestra.run.smithi081.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-14T16:01:49.289 INFO:teuthology.orchestra.run.smithi164.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 940 kB/s | 258 kB 00:00 2023-07-14T16:01:49.291 INFO:teuthology.orchestra.run.smithi081.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-14T16:01:49.325 INFO:teuthology.orchestra.run.smithi081.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-14T16:01:49.367 INFO:teuthology.orchestra.run.smithi081.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-14T16:01:49.417 INFO:teuthology.orchestra.run.smithi081.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.4 MB/s | 323 kB 00:00 2023-07-14T16:01:49.459 INFO:teuthology.orchestra.run.smithi081.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 934 kB/s | 39 kB 00:00 2023-07-14T16:01:49.481 INFO:teuthology.orchestra.run.smithi164.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 870 kB/s | 166 kB 00:00 2023-07-14T16:01:49.543 INFO:teuthology.orchestra.run.smithi081.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 8.1 MB/s | 688 kB 00:00 2023-07-14T16:01:49.593 INFO:teuthology.orchestra.run.smithi081.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-14T16:01:49.594 INFO:teuthology.orchestra.run.smithi164.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 159 kB/s | 18 kB 00:00 2023-07-14T16:01:49.635 INFO:teuthology.orchestra.run.smithi081.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-14T16:01:49.677 INFO:teuthology.orchestra.run.smithi081.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-14T16:01:49.736 INFO:teuthology.orchestra.run.smithi081.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-14T16:01:49.920 INFO:teuthology.orchestra.run.smithi081.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 20 MB/s | 14 MB 00:00 2023-07-14T16:01:49.953 INFO:teuthology.orchestra.run.smithi081.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 805 kB/s | 27 kB 00:00 2023-07-14T16:01:50.029 INFO:teuthology.orchestra.run.smithi081.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 4.2 MB/s | 320 kB 00:00 2023-07-14T16:01:50.070 INFO:teuthology.orchestra.run.smithi081.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-07-14T16:01:50.154 INFO:teuthology.orchestra.run.smithi081.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-07-14T16:01:50.204 INFO:teuthology.orchestra.run.smithi081.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-07-14T16:01:50.296 INFO:teuthology.orchestra.run.smithi081.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-07-14T16:01:50.322 INFO:teuthology.orchestra.run.smithi081.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.9 MB/s | 225 kB 00:00 2023-07-14T16:01:50.347 INFO:teuthology.orchestra.run.smithi081.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-14T16:01:50.372 INFO:teuthology.orchestra.run.smithi081.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-14T16:01:50.397 INFO:teuthology.orchestra.run.smithi081.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-07-14T16:01:50.423 INFO:teuthology.orchestra.run.smithi081.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-14T16:01:50.457 INFO:teuthology.orchestra.run.smithi081.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-14T16:01:50.533 INFO:teuthology.orchestra.run.smithi081.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 2.5 MB/s | 198 kB 00:00 2023-07-14T16:01:50.667 INFO:teuthology.orchestra.run.smithi081.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 20 MB/s | 4.9 MB 00:00 2023-07-14T16:01:50.708 INFO:teuthology.orchestra.run.smithi081.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-14T16:01:50.783 INFO:teuthology.orchestra.run.smithi081.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-07-14T16:01:50.825 INFO:teuthology.orchestra.run.smithi081.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.9 MB/s | 354 kB 00:00 2023-07-14T16:01:50.883 INFO:teuthology.orchestra.run.smithi081.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-07-14T16:01:50.959 INFO:teuthology.orchestra.run.smithi081.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 20 MB/s | 1.5 MB 00:00 2023-07-14T16:01:51.047 INFO:teuthology.orchestra.run.smithi164.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 211 kB/s | 306 kB 00:01 2023-07-14T16:01:51.067 INFO:teuthology.orchestra.run.smithi081.stdout:(127/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-07-14T16:01:51.356 INFO:teuthology.orchestra.run.smithi164.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 191 kB/s | 59 kB 00:00 2023-07-14T16:01:51.656 INFO:teuthology.orchestra.run.smithi164.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 250 kB/s | 75 kB 00:00 2023-07-14T16:01:53.823 INFO:teuthology.orchestra.run.smithi164.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 220 kB/s | 476 kB 00:02 2023-07-14T16:01:53.948 INFO:teuthology.orchestra.run.smithi164.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 349 kB/s | 44 kB 00:00 2023-07-14T16:01:54.284 INFO:teuthology.orchestra.run.smithi081.stdout:(128/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 1.1 MB/s | 3.6 MB 00:03 2023-07-14T16:01:54.324 INFO:teuthology.orchestra.run.smithi164.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 494 kB/s | 185 kB 00:00 2023-07-14T16:01:54.343 INFO:teuthology.orchestra.run.smithi081.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-14T16:01:54.466 INFO:teuthology.orchestra.run.smithi164.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 233 kB/s | 33 kB 00:00 2023-07-14T16:01:54.549 INFO:teuthology.orchestra.run.smithi164.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 196 kB/s | 16 kB 00:00 2023-07-14T16:01:55.016 INFO:teuthology.orchestra.run.smithi164.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 370 kB/s | 173 kB 00:00 2023-07-14T16:01:55.849 INFO:teuthology.orchestra.run.smithi164.stdout:(46/131): ceph-mgr-diskprediction-local-18.0.0- 962 kB/s | 7.4 MB 00:07 2023-07-14T16:01:56.051 INFO:teuthology.orchestra.run.smithi081.stdout:(130/131): librados2-18.0.0-4757.g3fc0089c.el8. 737 kB/s | 3.7 MB 00:05 2023-07-14T16:01:56.216 INFO:teuthology.orchestra.run.smithi164.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 222 kB/s | 81 kB 00:00 2023-07-14T16:01:56.542 INFO:teuthology.orchestra.run.smithi164.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 34 kB/s | 11 kB 00:00 2023-07-14T16:01:56.642 INFO:teuthology.orchestra.run.smithi164.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 176 kB/s | 18 kB 00:00 2023-07-14T16:01:56.751 INFO:teuthology.orchestra.run.smithi164.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 222 kB/s | 384 kB 00:01 2023-07-14T16:01:56.927 INFO:teuthology.orchestra.run.smithi062.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 1.9 MB/s | 48 MB 00:24 2023-07-14T16:01:56.930 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-07-14T16:01:56.931 INFO:teuthology.orchestra.run.smithi062.stdout:Total 6.9 MB/s | 205 MB 00:29 2023-07-14T16:01:57.084 INFO:teuthology.orchestra.run.smithi164.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 146 kB/s | 49 kB 00:00 2023-07-14T16:01:57.818 INFO:teuthology.orchestra.run.smithi164.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 80 kB/s | 59 kB 00:00 2023-07-14T16:01:58.393 INFO:teuthology.orchestra.run.smithi164.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 105 kB/s | 60 kB 00:00 2023-07-14T16:01:58.620 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-07-14T16:01:58.766 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-07-14T16:01:58.766 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-07-14T16:01:59.151 INFO:teuthology.orchestra.run.smithi164.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 476 kB/s | 1.2 MB 00:02 2023-07-14T16:01:59.360 INFO:teuthology.orchestra.run.smithi164.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 293 kB/s | 283 kB 00:00 2023-07-14T16:01:59.394 INFO:teuthology.orchestra.run.smithi164.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 64 kB/s | 16 kB 00:00 2023-07-14T16:01:59.553 INFO:teuthology.orchestra.run.smithi164.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 175 kB/s | 34 kB 00:00 2023-07-14T16:01:59.836 INFO:teuthology.orchestra.run.smithi164.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 217 kB/s | 61 kB 00:00 2023-07-14T16:01:59.945 INFO:teuthology.orchestra.run.smithi164.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-07-14T16:02:00.054 INFO:teuthology.orchestra.run.smithi164.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 221 kB/s | 24 kB 00:00 2023-07-14T16:02:00.202 INFO:teuthology.orchestra.run.smithi164.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 197 kB/s | 29 kB 00:00 2023-07-14T16:02:00.313 INFO:teuthology.orchestra.run.smithi164.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 250 kB/s | 27 kB 00:00 2023-07-14T16:02:00.338 INFO:teuthology.orchestra.run.smithi164.stdout:(63/131): python3-routes-2.4.1-12.el8.noarch.rp 208 kB/s | 196 kB 00:00 2023-07-14T16:02:00.380 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-07-14T16:02:00.385 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-07-14T16:02:00.913 INFO:teuthology.orchestra.run.smithi164.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 425 kB/s | 255 kB 00:00 2023-07-14T16:02:01.072 INFO:teuthology.orchestra.run.smithi164.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 388 kB/s | 61 kB 00:00 2023-07-14T16:02:01.222 INFO:teuthology.orchestra.run.smithi164.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 284 kB/s | 251 kB 00:00 2023-07-14T16:02:01.247 INFO:teuthology.orchestra.run.smithi164.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 487 kB/s | 85 kB 00:00 2023-07-14T16:02:01.364 INFO:teuthology.orchestra.run.smithi164.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 162 kB/s | 23 kB 00:00 2023-07-14T16:02:01.389 INFO:teuthology.orchestra.run.smithi164.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T16:02:01.781 INFO:teuthology.orchestra.run.smithi164.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 35 MB/s | 14 MB 00:00 2023-07-14T16:02:01.823 INFO:teuthology.orchestra.run.smithi164.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-14T16:02:01.865 INFO:teuthology.orchestra.run.smithi164.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-14T16:02:01.907 INFO:teuthology.orchestra.run.smithi164.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-14T16:02:01.957 INFO:teuthology.orchestra.run.smithi164.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-14T16:02:01.999 INFO:teuthology.orchestra.run.smithi164.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 926 kB/s | 39 kB 00:00 2023-07-14T16:02:02.049 INFO:teuthology.orchestra.run.smithi164.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 14 MB/s | 688 kB 00:00 2023-07-14T16:02:02.108 INFO:teuthology.orchestra.run.smithi164.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 9.0 MB/s | 538 kB 00:00 2023-07-14T16:02:02.167 INFO:teuthology.orchestra.run.smithi164.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-14T16:02:02.217 INFO:teuthology.orchestra.run.smithi164.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-14T16:02:02.267 INFO:teuthology.orchestra.run.smithi164.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-14T16:02:02.409 INFO:teuthology.orchestra.run.smithi164.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 34 MB/s | 4.8 MB 00:00 2023-07-14T16:02:02.451 INFO:teuthology.orchestra.run.smithi164.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 639 kB/s | 27 kB 00:00 2023-07-14T16:02:02.493 INFO:teuthology.orchestra.run.smithi164.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-14T16:02:02.560 INFO:teuthology.orchestra.run.smithi164.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 661 kB/s | 44 kB 00:00 2023-07-14T16:02:02.677 INFO:teuthology.orchestra.run.smithi164.stdout:(85/131): python3-numpy-1.14.3-10.el8.x86_64.rp 32 MB/s | 3.7 MB 00:00 2023-07-14T16:02:02.719 INFO:teuthology.orchestra.run.smithi164.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-07-14T16:02:02.761 INFO:teuthology.orchestra.run.smithi164.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-07-14T16:02:02.795 INFO:teuthology.orchestra.run.smithi164.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-07-14T16:02:02.837 INFO:teuthology.orchestra.run.smithi164.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2023-07-14T16:02:02.879 INFO:teuthology.orchestra.run.smithi164.stdout:(90/131): python3-jsonpointer-1.10-11.el8.noarc 481 kB/s | 20 kB 00:00 2023-07-14T16:02:02.946 INFO:teuthology.orchestra.run.smithi164.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2023-07-14T16:02:02.996 INFO:teuthology.orchestra.run.smithi164.stdout:(92/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-07-14T16:02:03.138 INFO:teuthology.orchestra.run.smithi164.stdout:(93/131): openblas-threads-0.3.15-3.el8.x86_64. 34 MB/s | 4.9 MB 00:00 2023-07-14T16:02:03.180 INFO:teuthology.orchestra.run.smithi164.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-07-14T16:02:03.339 INFO:teuthology.orchestra.run.smithi164.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 30 MB/s | 4.7 MB 00:00 2023-07-14T16:02:03.389 INFO:teuthology.orchestra.run.smithi164.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-07-14T16:02:03.431 INFO:teuthology.orchestra.run.smithi164.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.4 MB/s | 354 kB 00:00 2023-07-14T16:02:03.573 INFO:teuthology.orchestra.run.smithi164.stdout:(98/131): postgresql-server-10.21-2.module+el8. 36 MB/s | 5.1 MB 00:00 2023-07-14T16:02:03.632 INFO:teuthology.orchestra.run.smithi164.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-07-14T16:02:03.707 INFO:teuthology.orchestra.run.smithi164.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 20 MB/s | 1.5 MB 00:00 2023-07-14T16:02:03.740 INFO:teuthology.orchestra.run.smithi164.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-07-14T16:02:03.782 INFO:teuthology.orchestra.run.smithi164.stdout:(102/131): libselinux-devel-2.9-5.el8.x86_64.rp 4.7 MB/s | 200 kB 00:00 2023-07-14T16:02:03.816 INFO:teuthology.orchestra.run.smithi164.stdout:(103/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-07-14T16:02:03.858 INFO:teuthology.orchestra.run.smithi164.stdout:(104/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-14T16:02:03.891 INFO:teuthology.orchestra.run.smithi164.stdout:(105/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.4 MB/s | 48 kB 00:00 2023-07-14T16:02:03.933 INFO:teuthology.orchestra.run.smithi164.stdout:(106/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-14T16:02:03.967 INFO:teuthology.orchestra.run.smithi164.stdout:(107/131): python3-requests-oauthlib-1.0.0-1.el 1.2 MB/s | 43 kB 00:00 2023-07-14T16:02:04.009 INFO:teuthology.orchestra.run.smithi164.stdout:(108/131): python3-pycparser-2.14-14.el8.noarch 2.6 MB/s | 109 kB 00:00 2023-07-14T16:02:04.067 INFO:teuthology.orchestra.run.smithi164.stdout:(109/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-07-14T16:02:04.118 INFO:teuthology.orchestra.run.smithi164.stdout:(110/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 11 MB/s | 560 kB 00:00 2023-07-14T16:02:04.152 INFO:teuthology.orchestra.run.smithi164.stdout:(111/131): libaio-devel-0.3.112-1.el8.x86_64.rp 560 kB/s | 19 kB 00:00 2023-07-14T16:02:04.193 INFO:teuthology.orchestra.run.smithi164.stdout:(112/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-07-14T16:02:04.244 INFO:teuthology.orchestra.run.smithi164.stdout:(113/131): python3-cryptography-3.2.1-5.el8.x86 11 MB/s | 559 kB 00:00 2023-07-14T16:02:04.278 INFO:teuthology.orchestra.run.smithi164.stdout:(114/131): libverto-devel-0.3.0-5.el8.x86_64.rp 535 kB/s | 18 kB 00:00 2023-07-14T16:02:04.336 INFO:teuthology.orchestra.run.smithi164.stdout:(115/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-14T16:02:04.378 INFO:teuthology.orchestra.run.smithi164.stdout:(116/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-07-14T16:02:04.470 INFO:teuthology.orchestra.run.smithi164.stdout:(117/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 25 MB/s | 2.3 MB 00:00 2023-07-14T16:02:04.512 INFO:teuthology.orchestra.run.smithi164.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-07-14T16:02:04.554 INFO:teuthology.orchestra.run.smithi164.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-14T16:02:04.588 INFO:teuthology.orchestra.run.smithi164.stdout:(120/131): elfutils-libelf-devel-0.186-1.el8.x8 1.8 MB/s | 60 kB 00:00 2023-07-14T16:02:04.621 INFO:teuthology.orchestra.run.smithi164.stdout:(121/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.9 MB/s | 98 kB 00:00 2023-07-14T16:02:04.655 INFO:teuthology.orchestra.run.smithi164.stdout:(122/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.7 MB/s | 58 kB 00:00 2023-07-14T16:02:04.688 INFO:teuthology.orchestra.run.smithi164.stdout:(123/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-07-14T16:02:04.739 INFO:teuthology.orchestra.run.smithi164.stdout:(124/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 13 MB/s | 644 kB 00:00 2023-07-14T16:02:04.811 INFO:teuthology.orchestra.run.smithi164.stdout:(125/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.3 MB/s | 171 kB 00:00 2023-07-14T16:02:04.856 INFO:teuthology.orchestra.run.smithi164.stdout:(126/131): NetworkManager-initscripts-updown-1. 3.0 MB/s | 138 kB 00:00 2023-07-14T16:02:05.489 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-07-14T16:02:05.637 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T16:02:05.820 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T16:02:06.024 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T16:02:06.212 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T16:02:06.340 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T16:02:06.587 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T16:02:07.084 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T16:02:07.312 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:02:07.514 INFO:teuthology.orchestra.run.smithi164.stdout:(127/131): thrift-0.13.0-2.el8.x86_64.rpm 283 kB/s | 1.7 MB 00:06 2023-07-14T16:02:07.796 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:02:07.961 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:02:08.113 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:02:08.287 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T16:02:08.389 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T16:02:08.414 INFO:teuthology.orchestra.run.smithi164.stdout:(128/131): librados2-18.0.0-4757.g3fc0089c.el8. 1.0 MB/s | 3.7 MB 00:03 2023-07-14T16:02:08.473 INFO:teuthology.orchestra.run.smithi164.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-14T16:02:08.567 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T16:02:08.856 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T16:02:09.085 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T16:02:09.512 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T16:02:09.734 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T16:02:09.928 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T16:02:10.045 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T16:02:10.186 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T16:02:10.291 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T16:02:10.436 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T16:02:10.612 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T16:02:10.789 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T16:02:11.532 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T16:02:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T16:02:12.462 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T16:02:12.728 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T16:02:13.620 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T16:02:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T16:02:14.177 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T16:02:14.189 INFO:teuthology.orchestra.run.smithi164.stdout:(130/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 546 kB/s | 3.6 MB 00:06 2023-07-14T16:02:14.404 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T16:02:14.629 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T16:02:14.884 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T16:02:15.201 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T16:02:15.385 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T16:02:15.525 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T16:02:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T16:02:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T16:02:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T16:02:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T16:02:17.827 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T16:02:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T16:02:20.422 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T16:02:20.650 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T16:02:20.902 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T16:02:21.102 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T16:02:21.390 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T16:02:21.544 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T16:02:21.714 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T16:02:21.946 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T16:02:22.109 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T16:02:22.328 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T16:02:22.596 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T16:02:23.029 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T16:02:23.368 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T16:02:23.576 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T16:02:23.853 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T16:02:24.087 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T16:02:24.445 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T16:02:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T16:02:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T16:02:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T16:02:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T16:02:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T16:02:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T16:02:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T16:02:26.396 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T16:02:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T16:02:26.747 INFO:teuthology.orchestra.run.smithi081.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 928 kB/s | 48 MB 00:52 2023-07-14T16:02:26.750 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2023-07-14T16:02:26.751 INFO:teuthology.orchestra.run.smithi081.stdout:Total 2.9 MB/s | 205 MB 01:09 2023-07-14T16:02:26.802 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T16:02:26.972 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T16:02:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T16:02:27.296 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T16:02:27.522 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T16:02:27.749 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T16:02:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T16:02:28.000 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T16:02:28.012 INFO:teuthology.orchestra.run.smithi164.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 966 kB/s | 48 MB 00:50 2023-07-14T16:02:28.015 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2023-07-14T16:02:28.016 INFO:teuthology.orchestra.run.smithi164.stdout:Total 3.1 MB/s | 205 MB 01:06 2023-07-14T16:02:28.233 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T16:02:28.377 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2023-07-14T16:02:28.445 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T16:02:28.512 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2023-07-14T16:02:28.512 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2023-07-14T16:02:28.627 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T16:02:28.826 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T16:02:29.140 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T16:02:29.367 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T16:02:29.555 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T16:02:29.666 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2023-07-14T16:02:29.764 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T16:02:29.801 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2023-07-14T16:02:29.802 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2023-07-14T16:02:29.960 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T16:02:30.104 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2023-07-14T16:02:30.108 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2023-07-14T16:02:30.185 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T16:02:30.367 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T16:02:30.604 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T16:02:30.767 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T16:02:30.998 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T16:02:31.257 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T16:02:31.387 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2023-07-14T16:02:31.392 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2023-07-14T16:02:31.449 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T16:02:31.758 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T16:02:31.993 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T16:02:32.171 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T16:02:32.449 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T16:02:33.827 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T16:02:34.157 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T16:02:34.424 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T16:02:34.836 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T16:02:35.085 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2023-07-14T16:02:35.273 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T16:02:35.372 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T16:02:35.467 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T16:02:35.573 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T16:02:35.657 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T16:02:35.809 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T16:02:36.050 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T16:02:36.491 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T16:02:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2023-07-14T16:02:36.519 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T16:02:36.674 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T16:02:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T16:02:36.749 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-14T16:02:36.768 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T16:02:36.932 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T16:02:36.992 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T16:02:37.133 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T16:02:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-14T16:02:37.331 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T16:02:37.382 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:02:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T16:02:37.529 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:02:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:02:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T16:02:37.824 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:02:38.022 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T16:02:38.213 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-14T16:02:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T16:02:38.456 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T16:02:38.475 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:02:38.704 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T16:02:38.939 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:02:39.106 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:02:39.116 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T16:02:39.261 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:02:39.353 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T16:02:39.447 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T16:02:39.535 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T16:02:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T16:02:39.672 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T16:02:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T16:02:39.819 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T16:02:39.993 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T16:02:40.021 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T16:02:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T16:02:40.255 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-14T16:02:40.424 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-14T16:02:40.622 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-14T16:02:40.700 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T16:02:40.757 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T16:02:40.920 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T16:02:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T16:02:41.113 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T16:02:41.132 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T16:02:41.257 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-14T16:02:41.389 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T16:02:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-14T16:02:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T16:02:41.833 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T16:02:41.849 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T16:02:42.038 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T16:02:42.040 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T16:02:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:02:42.291 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:02:42.736 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T16:02:42.741 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T16:02:42.932 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T16:02:42.938 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T16:02:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:02:43.327 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T16:02:43.460 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T16:02:43.614 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T16:02:43.618 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T16:02:43.748 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T16:02:43.846 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T16:02:44.022 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-14T16:02:44.275 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-14T16:02:44.550 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T16:02:44.690 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T16:02:44.788 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T16:02:44.936 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T16:02:44.979 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T16:02:45.216 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T16:02:45.224 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T16:02:45.439 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T16:02:45.487 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T16:02:45.672 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T16:02:45.941 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T16:02:46.242 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T16:02:46.445 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-14T16:02:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-14T16:02:47.496 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T16:02:47.667 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-14T16:02:48.011 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T16:02:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T16:02:48.600 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T16:02:48.674 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-14T16:02:48.840 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T16:02:49.060 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-14T16:02:49.379 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-14T16:02:49.637 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-14T16:02:49.885 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T16:02:50.078 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T16:02:50.297 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T16:02:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T16:02:50.656 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-14T16:02:50.834 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-14T16:02:50.943 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T16:02:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T16:02:51.115 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T16:02:51.339 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T16:02:51.393 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-14T16:02:51.594 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-14T16:02:51.600 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T16:02:51.801 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T16:02:51.805 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T16:02:51.902 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T16:02:52.087 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-14T16:02:52.131 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-14T16:02:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-14T16:02:52.335 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-14T16:02:52.488 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T16:02:52.488 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-14T16:02:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-14T16:02:52.716 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T16:02:52.875 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-14T16:02:52.923 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T16:02:53.018 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-14T16:02:53.118 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T16:02:53.233 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-14T16:02:53.343 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T16:02:53.483 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-14T16:02:53.603 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T16:02:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-14T16:02:54.059 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T16:02:54.212 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-14T16:02:54.403 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T16:02:54.453 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-14T16:02:54.624 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T16:02:54.682 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-14T16:02:54.849 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T16:02:54.931 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-14T16:02:55.155 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T16:02:55.263 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-14T16:02:55.498 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-14T16:02:55.539 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T16:02:55.718 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-14T16:02:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T16:02:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T16:02:56.000 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-14T16:02:56.197 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-14T16:02:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T16:02:56.361 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-14T16:02:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T16:02:56.589 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T16:02:56.719 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-14T16:02:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T16:02:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-14T16:02:57.224 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T16:02:57.320 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-14T16:02:57.534 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T16:02:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-14T16:02:57.790 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T16:02:57.827 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T16:02:58.012 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T16:02:58.013 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T16:02:58.229 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T16:02:58.315 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T16:02:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T16:02:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T16:02:58.563 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T16:02:58.709 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T16:02:58.798 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T16:02:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T16:02:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T16:02:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T16:02:59.186 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T16:02:59.283 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T16:02:59.288 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T16:02:59.494 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T16:02:59.529 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T16:02:59.696 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T16:02:59.717 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T16:02:59.870 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T16:02:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T16:03:00.113 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T16:03:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T16:03:00.360 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T16:03:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T16:03:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T16:03:00.644 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T16:03:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T16:03:00.870 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T16:03:01.008 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T16:03:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T16:03:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T16:03:01.291 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T16:03:01.478 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T16:03:01.534 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T16:03:01.694 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T16:03:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T16:03:02.051 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T16:03:02.269 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T16:03:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T16:03:02.767 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T16:03:02.915 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T16:03:03.042 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T16:03:03.248 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T16:03:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T16:03:03.518 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T16:03:03.538 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T16:03:03.779 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T16:03:03.925 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T16:03:04.263 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T16:03:04.468 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T16:03:05.130 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T16:03:05.336 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T16:03:05.336 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2023-07-14T16:03:05.337 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2023-07-14T16:03:05.337 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-07-14T16:03:05.337 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /run 2023-07-14T16:03:05.337 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2023-07-14T16:03:05.337 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2023-07-14T16:03:05.337 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2023-07-14T16:03:05.337 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2023-07-14T16:03:05.337 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-07-14T16:03:05.338 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:03:05.367 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T16:03:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T16:03:05.528 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T16:03:05.651 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T16:03:05.738 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T16:03:05.757 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T16:03:05.821 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T16:03:06.166 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T16:03:06.446 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T16:03:06.695 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T16:03:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T16:03:07.581 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T16:03:07.798 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T16:03:07.809 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T16:03:07.909 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-14T16:03:08.076 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-14T16:03:08.083 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T16:03:08.261 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T16:03:08.823 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T16:03:09.028 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T16:03:09.071 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T16:03:10.905 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:03:11.120 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:03:11.819 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:03:11.885 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T16:03:12.068 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T16:03:12.186 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T16:03:12.417 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T16:03:12.610 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T16:03:12.625 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T16:03:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T16:03:12.884 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T16:03:12.969 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T16:03:13.044 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:03:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T16:03:13.692 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:03:14.320 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T16:03:14.396 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-14T16:03:14.485 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T16:03:14.620 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T16:03:14.676 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T16:03:14.839 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-14T16:03:14.999 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T16:03:15.040 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T16:03:15.284 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T16:03:15.352 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T16:03:15.527 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T16:03:15.556 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T16:03:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T16:03:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T16:03:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T16:03:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T16:03:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T16:03:17.760 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T16:03:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T16:03:18.368 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T16:03:23.749 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T16:03:24.063 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T16:03:24.208 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T16:03:24.477 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T16:03:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T16:03:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T16:03:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/134 2023-07-14T16:03:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T16:03:26.822 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T16:03:26.976 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T16:03:30.466 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:30.819 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:30.857 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:31.018 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T16:03:31.244 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T16:03:31.611 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T16:03:31.893 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T16:03:32.116 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T16:03:32.140 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T16:03:32.384 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T16:03:32.599 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T16:03:32.715 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T16:03:32.941 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T16:03:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T16:03:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:03:33.214 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:03:33.215 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T16:03:33.253 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T16:03:33.253 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2023-07-14T16:03:33.253 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2023-07-14T16:03:33.254 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /dev 2023-07-14T16:03:33.254 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /run 2023-07-14T16:03:33.254 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2023-07-14T16:03:33.254 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2023-07-14T16:03:33.254 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2023-07-14T16:03:33.255 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2023-07-14T16:03:33.255 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /dev 2023-07-14T16:03:33.255 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:03:33.464 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T16:03:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T16:03:33.649 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:03:33.947 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:03:33.948 INFO:teuthology.orchestra.run.smithi062.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-14T16:03:33.948 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:03:34.592 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T16:03:34.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T16:03:34.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T16:03:34.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T16:03:34.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T16:03:34.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:03:34.593 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:03:34.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T16:03:34.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T16:03:34.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T16:03:34.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T16:03:34.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T16:03:34.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T16:03:34.594 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T16:03:34.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T16:03:34.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T16:03:34.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T16:03:34.596 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T16:03:34.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T16:03:34.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T16:03:34.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T16:03:34.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T16:03:34.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T16:03:34.597 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T16:03:34.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T16:03:34.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T16:03:34.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T16:03:34.598 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T16:03:34.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T16:03:34.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T16:03:34.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T16:03:34.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T16:03:34.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T16:03:34.605 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T16:03:34.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T16:03:34.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T16:03:34.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T16:03:34.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T16:03:34.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T16:03:34.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T16:03:34.606 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T16:03:34.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T16:03:34.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T16:03:34.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T16:03:34.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T16:03:34.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T16:03:34.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T16:03:34.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T16:03:34.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T16:03:34.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T16:03:34.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T16:03:34.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T16:03:34.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T16:03:34.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T16:03:34.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T16:03:34.608 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T16:03:34.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T16:03:34.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T16:03:34.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T16:03:34.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T16:03:34.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T16:03:34.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T16:03:34.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T16:03:34.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T16:03:34.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T16:03:34.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T16:03:34.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T16:03:34.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T16:03:34.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T16:03:34.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T16:03:34.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T16:03:34.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T16:03:34.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T16:03:34.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T16:03:34.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T16:03:34.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T16:03:34.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T16:03:34.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T16:03:34.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T16:03:34.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T16:03:34.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T16:03:34.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T16:03:34.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T16:03:34.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T16:03:34.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T16:03:34.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T16:03:34.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T16:03:34.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T16:03:34.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T16:03:34.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T16:03:34.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T16:03:34.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T16:03:34.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T16:03:34.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T16:03:34.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T16:03:34.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T16:03:34.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T16:03:34.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T16:03:34.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T16:03:34.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T16:03:34.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T16:03:34.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T16:03:34.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T16:03:34.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T16:03:34.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T16:03:34.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T16:03:34.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T16:03:34.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T16:03:34.616 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T16:03:34.616 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T16:03:34.616 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T16:03:34.616 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T16:03:34.616 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T16:03:34.616 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T16:03:34.616 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T16:03:34.617 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T16:03:34.617 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T16:03:34.617 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T16:03:34.617 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T16:03:34.617 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T16:03:34.617 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T16:03:34.617 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T16:03:34.617 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T16:03:34.618 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T16:03:34.618 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T16:03:34.618 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T16:03:34.618 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T16:03:34.618 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T16:03:34.618 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T16:03:34.618 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T16:03:34.618 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T16:03:34.619 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:03:34.619 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T16:03:35.256 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-14T16:03:35.256 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2023-07-14T16:03:35.257 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2023-07-14T16:03:35.257 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /dev 2023-07-14T16:03:35.257 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /run 2023-07-14T16:03:35.257 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2023-07-14T16:03:35.257 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2023-07-14T16:03:35.257 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2023-07-14T16:03:35.257 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2023-07-14T16:03:35.258 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /dev 2023-07-14T16:03:35.258 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:03:35.352 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T16:03:35.521 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T16:03:35.671 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T16:03:35.790 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T16:03:35.974 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T16:03:36.501 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T16:03:36.674 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T16:03:36.716 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T16:03:37.296 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:03:37.296 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:23 ago on Fri 14 Jul 2023 04:01:14 PM UTC. 2023-07-14T16:03:37.396 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-14T16:03:37.564 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T16:03:37.833 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-14T16:03:38.012 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T16:03:38.539 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-14T16:03:38.721 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T16:03:38.762 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-14T16:03:39.390 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T16:03:39.582 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T16:03:39.715 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T16:03:39.961 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T16:03:40.148 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T16:03:40.353 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-07-14T16:03:40.409 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T16:03:40.595 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:03:40.595 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2023-07-14T16:03:40.595 INFO:teuthology.orchestra.run.smithi062.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T16:03:40.595 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.595 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.596 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-07-14T16:03:40.596 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T16:03:40.596 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T16:03:40.596 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T16:03:40.596 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.596 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.596 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.596 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.597 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.597 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.597 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.597 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.598 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.598 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.598 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.598 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.598 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.598 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.598 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.598 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.599 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.599 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.599 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.599 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.599 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.599 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.599 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:03:40.600 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T16:03:40.600 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T16:03:40.600 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T16:03:40.600 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T16:03:40.600 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-3.el8.x86_64 2023-07-14T16:03:40.600 INFO:teuthology.orchestra.run.smithi062.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T16:03:40.601 INFO:teuthology.orchestra.run.smithi062.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T16:03:40.601 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T16:03:40.601 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.601 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.601 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.601 INFO:teuthology.orchestra.run.smithi062.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T16:03:40.601 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T16:03:40.601 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T16:03:40.602 INFO:teuthology.orchestra.run.smithi062.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T16:03:40.602 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T16:03:40.602 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T16:03:40.602 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.602 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.602 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T16:03:40.602 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.602 INFO:teuthology.orchestra.run.smithi062.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T16:03:40.603 INFO:teuthology.orchestra.run.smithi062.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T16:03:40.603 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T16:03:40.603 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T16:03:40.603 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T16:03:40.603 INFO:teuthology.orchestra.run.smithi062.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T16:03:40.603 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T16:03:40.603 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T16:03:40.604 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T16:03:40.604 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T16:03:40.604 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T16:03:40.604 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T16:03:40.604 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T16:03:40.604 INFO:teuthology.orchestra.run.smithi062.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T16:03:40.604 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:03:40.604 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:03:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:03:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T16:03:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T16:03:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T16:03:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T16:03:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T16:03:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.606 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.606 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T16:03:40.606 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T16:03:40.606 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T16:03:40.606 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T16:03:40.606 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T16:03:40.606 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T16:03:40.606 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T16:03:40.607 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T16:03:40.607 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T16:03:40.607 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T16:03:40.607 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T16:03:40.607 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T16:03:40.607 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T16:03:40.607 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T16:03:40.608 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T16:03:40.608 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T16:03:40.608 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T16:03:40.608 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T16:03:40.608 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T16:03:40.608 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T16:03:40.608 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T16:03:40.608 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T16:03:40.609 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T16:03:40.609 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T16:03:40.609 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T16:03:40.609 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T16:03:40.609 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T16:03:40.609 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T16:03:40.610 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T16:03:40.610 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.610 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.610 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T16:03:40.610 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T16:03:40.611 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.611 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T16:03:40.611 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T16:03:40.611 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T16:03:40.611 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T16:03:40.611 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T16:03:40.611 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T16:03:40.611 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T16:03:40.612 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T16:03:40.612 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T16:03:40.612 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T16:03:40.612 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T16:03:40.612 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T16:03:40.612 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T16:03:40.613 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.613 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.613 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:03:40.613 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T16:03:40.613 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T16:03:40.613 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T16:03:40.613 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T16:03:40.614 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T16:03:40.614 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T16:03:40.614 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T16:03:40.614 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:03:40.615 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-07-14T16:03:40.903 DEBUG:teuthology.parallel:result is None 2023-07-14T16:03:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T16:03:41.708 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-14T16:03:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T16:03:41.828 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-14T16:03:41.940 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T16:03:42.066 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T16:03:42.067 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-14T16:03:42.261 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T16:03:42.444 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T16:03:42.526 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-14T16:03:42.663 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T16:03:42.712 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T16:03:42.728 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T16:03:42.956 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T16:03:42.984 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T16:03:43.830 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-14T16:03:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T16:03:44.148 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-14T16:03:44.375 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T16:03:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-14T16:03:44.601 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T16:03:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T16:03:44.831 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T16:03:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T16:03:44.912 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-14T16:03:44.940 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T16:03:45.122 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T16:03:45.127 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T16:03:45.155 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-14T16:03:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T16:03:45.681 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T16:03:46.559 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-14T16:03:46.779 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T16:03:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-14T16:03:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T16:03:47.138 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-14T16:03:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T16:03:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-14T16:03:47.953 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T16:03:50.719 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T16:03:51.973 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T16:03:52.292 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-14T16:03:52.519 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 117/134 2023-07-14T16:03:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-14T16:03:52.938 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T16:03:52.997 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-14T16:03:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T16:03:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-14T16:03:53.435 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-14T16:03:53.550 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T16:03:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T16:03:53.806 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T16:03:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T16:03:55.050 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T16:03:55.381 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T16:03:55.548 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/134 2023-07-14T16:03:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T16:03:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-14T16:03:56.143 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T16:03:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:57.321 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:57.358 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:57.621 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T16:03:57.821 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T16:03:58.150 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T16:03:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T16:03:58.712 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T16:03:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T16:03:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T16:03:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T16:03:59.262 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T16:03:59.284 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T16:03:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:59.449 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-14T16:03:59.449 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:03:59.701 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:03:59.702 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T16:03:59.742 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T16:03:59.960 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T16:04:00.026 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-14T16:04:00.212 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:04:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-14T16:04:00.473 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:04:00.474 INFO:teuthology.orchestra.run.smithi081.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-14T16:04:00.474 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:04:00.578 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-14T16:04:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T16:04:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T16:04:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T16:04:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T16:04:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T16:04:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T16:04:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:04:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:04:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T16:04:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T16:04:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T16:04:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T16:04:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T16:04:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T16:04:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T16:04:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T16:04:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T16:04:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T16:04:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T16:04:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T16:04:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T16:04:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T16:04:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T16:04:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T16:04:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T16:04:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T16:04:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T16:04:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T16:04:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T16:04:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T16:04:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T16:04:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T16:04:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T16:04:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T16:04:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T16:04:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T16:04:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T16:04:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T16:04:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T16:04:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T16:04:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T16:04:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T16:04:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T16:04:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T16:04:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T16:04:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T16:04:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T16:04:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T16:04:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T16:04:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T16:04:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T16:04:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T16:04:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T16:04:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T16:04:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T16:04:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T16:04:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T16:04:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T16:04:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T16:04:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T16:04:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T16:04:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T16:04:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T16:04:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T16:04:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T16:04:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T16:04:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T16:04:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T16:04:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T16:04:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T16:04:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T16:04:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T16:04:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-14T16:04:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-14T16:04:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-14T16:04:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-14T16:04:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-14T16:04:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-14T16:04:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-14T16:04:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-14T16:04:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-14T16:04:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-14T16:04:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-14T16:04:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-14T16:04:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-14T16:04:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-14T16:04:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-14T16:04:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-14T16:04:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-14T16:04:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-14T16:04:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-14T16:04:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-14T16:04:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-14T16:04:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-14T16:04:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-14T16:04:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-14T16:04:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-14T16:04:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-14T16:04:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-14T16:04:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-14T16:04:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-14T16:04:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-14T16:04:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-14T16:04:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-14T16:04:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-14T16:04:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-14T16:04:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-14T16:04:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-14T16:04:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-14T16:04:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-14T16:04:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-14T16:04:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-14T16:04:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-14T16:04:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-14T16:04:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-14T16:04:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-14T16:04:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-14T16:04:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-14T16:04:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-14T16:04:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-14T16:04:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-14T16:04:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-14T16:04:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-14T16:04:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-14T16:04:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-14T16:04:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-14T16:04:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-14T16:04:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-14T16:04:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-14T16:04:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T16:04:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T16:04:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T16:04:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:04:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T16:04:01.124 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T16:04:01.148 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T16:04:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T16:04:01.582 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T16:04:01.694 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T16:04:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T16:04:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T16:04:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:04:02.173 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:04:02.173 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T16:04:02.439 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T16:04:02.666 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:04:02.929 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:04:02.929 INFO:teuthology.orchestra.run.smithi164.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-14T16:04:02.930 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:04:03.548 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-14T16:04:03.548 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-14T16:04:03.548 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-14T16:04:03.548 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-14T16:04:03.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-14T16:04:03.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-14T16:04:03.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-14T16:04:03.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-14T16:04:03.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-14T16:04:03.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-14T16:04:03.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-14T16:04:03.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-14T16:04:03.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-14T16:04:03.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-14T16:04:03.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-14T16:04:03.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-14T16:04:03.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-14T16:04:03.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-14T16:04:03.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-14T16:04:03.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-14T16:04:03.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-14T16:04:03.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-14T16:04:03.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-14T16:04:03.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-14T16:04:03.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-14T16:04:03.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-14T16:04:03.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-14T16:04:03.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-14T16:04:03.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-14T16:04:03.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-14T16:04:03.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-14T16:04:03.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-14T16:04:03.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-14T16:04:03.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-14T16:04:03.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-14T16:04:03.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-14T16:04:03.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-14T16:04:03.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-14T16:04:03.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T16:04:03.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T16:04:03.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T16:04:03.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T16:04:03.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T16:04:03.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T16:04:03.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T16:04:03.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T16:04:03.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T16:04:03.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T16:04:03.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T16:04:03.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T16:04:03.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T16:04:03.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T16:04:03.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T16:04:03.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T16:04:03.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T16:04:03.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T16:04:03.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T16:04:03.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T16:04:03.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T16:04:03.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T16:04:03.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T16:04:03.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T16:04:03.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T16:04:03.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T16:04:03.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T16:04:03.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T16:04:03.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T16:04:03.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T16:04:03.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T16:04:03.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T16:04:03.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T16:04:03.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T16:04:03.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T16:04:03.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T16:04:03.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T16:04:03.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T16:04:03.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T16:04:03.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T16:04:03.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T16:04:03.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T16:04:03.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T16:04:03.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T16:04:03.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T16:04:03.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T16:04:03.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T16:04:03.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T16:04:03.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T16:04:03.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T16:04:03.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T16:04:03.559 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T16:04:03.560 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T16:04:03.560 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T16:04:03.560 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T16:04:03.560 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T16:04:03.560 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T16:04:03.560 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T16:04:03.560 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T16:04:03.561 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T16:04:03.561 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T16:04:03.561 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T16:04:03.561 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T16:04:03.561 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T16:04:03.561 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T16:04:03.561 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T16:04:03.561 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T16:04:03.562 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T16:04:03.562 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T16:04:03.562 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T16:04:03.562 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T16:04:03.562 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T16:04:03.562 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T16:04:03.562 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T16:04:03.562 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T16:04:03.564 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T16:04:03.564 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T16:04:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T16:04:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T16:04:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T16:04:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T16:04:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T16:04:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T16:04:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T16:04:03.565 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T16:04:03.566 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T16:04:03.566 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T16:04:03.566 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T16:04:03.566 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T16:04:03.566 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T16:04:03.566 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-14T16:04:03.566 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T16:04:03.566 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-14T16:04:03.567 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T16:04:03.567 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T16:04:03.831 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:04:03.832 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:59 ago on Fri 14 Jul 2023 04:01:04 PM UTC. 2023-07-14T16:04:06.369 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T16:04:06.369 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:02:57 ago on Fri 14 Jul 2023 04:01:09 PM UTC. 2023-07-14T16:04:06.588 INFO:teuthology.orchestra.run.smithi081.stdout:Installed products updated. 2023-07-14T16:04:06.824 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:04:06.824 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2023-07-14T16:04:06.824 INFO:teuthology.orchestra.run.smithi081.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T16:04:06.825 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.825 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.825 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2023-07-14T16:04:06.825 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T16:04:06.825 INFO:teuthology.orchestra.run.smithi081.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T16:04:06.825 INFO:teuthology.orchestra.run.smithi081.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T16:04:06.825 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.825 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.826 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.826 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.826 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.826 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.826 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.826 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.826 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.827 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.827 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.827 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.827 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.827 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.827 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.827 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.827 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.828 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.828 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.828 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.828 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.828 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.828 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:06.828 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T16:04:06.829 INFO:teuthology.orchestra.run.smithi081.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T16:04:06.829 INFO:teuthology.orchestra.run.smithi081.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T16:04:06.829 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T16:04:06.829 INFO:teuthology.orchestra.run.smithi081.stdout: jq-1.6-3.el8.x86_64 2023-07-14T16:04:06.829 INFO:teuthology.orchestra.run.smithi081.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T16:04:06.829 INFO:teuthology.orchestra.run.smithi081.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T16:04:06.829 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T16:04:06.830 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.830 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.830 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.830 INFO:teuthology.orchestra.run.smithi081.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T16:04:06.830 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T16:04:06.830 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T16:04:06.830 INFO:teuthology.orchestra.run.smithi081.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T16:04:06.830 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T16:04:06.831 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T16:04:06.831 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.831 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.831 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T16:04:06.831 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.831 INFO:teuthology.orchestra.run.smithi081.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T16:04:06.831 INFO:teuthology.orchestra.run.smithi081.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T16:04:06.831 INFO:teuthology.orchestra.run.smithi081.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T16:04:06.832 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T16:04:06.832 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T16:04:06.832 INFO:teuthology.orchestra.run.smithi081.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T16:04:06.832 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T16:04:06.832 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T16:04:06.832 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T16:04:06.832 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T16:04:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T16:04:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T16:04:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T16:04:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T16:04:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:04:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:04:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:04:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T16:04:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T16:04:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T16:04:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T16:04:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T16:04:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T16:04:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T16:04:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T16:04:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T16:04:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T16:04:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T16:04:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T16:04:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T16:04:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T16:04:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T16:04:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T16:04:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T16:04:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T16:04:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T16:04:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T16:04:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T16:04:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T16:04:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T16:04:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T16:04:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T16:04:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T16:04:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T16:04:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T16:04:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T16:04:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T16:04:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T16:04:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T16:04:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T16:04:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T16:04:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T16:04:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T16:04:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T16:04:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T16:04:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T16:04:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T16:04:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T16:04:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T16:04:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T16:04:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T16:04:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T16:04:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T16:04:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T16:04:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T16:04:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T16:04:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T16:04:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T16:04:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T16:04:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T16:04:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T16:04:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T16:04:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-07-14T16:04:06.842 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2023-07-14T16:04:07.223 DEBUG:teuthology.parallel:result is None 2023-07-14T16:04:09.142 INFO:teuthology.orchestra.run.smithi164.stdout:Installed products updated. 2023-07-14T16:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2023-07-14T16:04:09.380 INFO:teuthology.orchestra.run.smithi164.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T16:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2023-07-14T16:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T16:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T16:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T16:04:09.381 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.382 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.383 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.384 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-14T16:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T16:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T16:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T16:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T16:04:09.385 INFO:teuthology.orchestra.run.smithi164.stdout: jq-1.6-3.el8.x86_64 2023-07-14T16:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T16:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T16:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T16:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T16:04:09.386 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T16:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T16:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T16:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T16:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T16:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T16:04:09.387 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T16:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T16:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T16:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T16:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T16:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T16:04:09.388 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T16:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T16:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T16:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T16:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T16:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T16:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T16:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T16:04:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T16:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T16:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T16:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T16:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T16:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T16:04:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T16:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T16:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T16:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T16:04:09.391 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T16:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T16:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T16:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T16:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T16:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T16:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T16:04:09.392 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T16:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T16:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T16:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T16:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T16:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T16:04:09.393 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T16:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T16:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T16:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T16:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T16:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T16:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T16:04:09.394 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T16:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T16:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T16:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T16:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T16:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.395 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T16:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T16:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T16:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T16:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T16:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T16:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T16:04:09.396 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T16:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T16:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T16:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T16:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T16:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T16:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T16:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T16:04:09.397 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-14T16:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T16:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T16:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T16:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T16:04:09.398 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T16:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T16:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:04:09.399 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2023-07-14T16:04:09.828 DEBUG:teuthology.parallel:result is None 2023-07-14T16:04:09.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:04:10.241 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T16:04:10.303 INFO:teuthology.orchestra.run.smithi062.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T16:04:10.303 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T16:04:10.304 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T16:04:10.304 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:04:10.691 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T16:04:10.750 INFO:teuthology.orchestra.run.smithi081.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T16:04:10.751 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T16:04:10.751 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T16:04:10.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:04:11.121 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T16:04:11.175 INFO:teuthology.orchestra.run.smithi164.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-14T16:04:11.175 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-14T16:04:11.176 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-14T16:04:11.177 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-14T16:04:11.178 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:04:11.178 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T16:04:11.276 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:04:11.276 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T16:04:11.370 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:04:11.371 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T16:04:11.460 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-14T16:04:11.460 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:04:11.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T16:04:11.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T16:04:11.642 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:04:11.642 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T16:04:11.719 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T16:04:11.827 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:04:11.828 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T16:04:11.896 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T16:04:12.003 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-14T16:04:12.004 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:04:12.004 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T16:04:12.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T16:04:12.180 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:04:12.181 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T16:04:12.254 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T16:04:12.363 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:04:12.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T16:04:12.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T16:04:12.541 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-14T16:04:12.541 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:04:12.541 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T16:04:12.612 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T16:04:12.720 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:04:12.721 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T16:04:12.788 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T16:04:12.896 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:04:12.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T16:04:12.966 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T16:04:13.073 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-14T16:04:13.160 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-07-14T16:04:13.161 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:04:13.173 INFO:tasks.cephadm:Cluster fsid is 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:04:13.174 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-14T16:04:13.174 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.62', 'mon.b': '172.21.15.81', 'mon.c': '172.21.15.164'} 2023-07-14T16:04:13.174 INFO:tasks.cephadm:First mon is mon.a on smithi062 2023-07-14T16:04:13.175 INFO:tasks.cephadm:First mgr is x 2023-07-14T16:04:13.175 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-14T16:04:13.175 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2023-07-14T16:04:13.248 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname $(hostname -s) 2023-07-14T16:04:13.318 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2023-07-14T16:04:13.388 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-14T16:04:13.390 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:04:13.662 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71399/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4757-g3fc0089c', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4757.g3fc0089c'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-13 19:47:46.444052', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-14T16:04:13.932 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023Jul13, sha1 3fc0089cc1082313a98853d48b892acdc1385b8b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:04:13.933 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm 2023-07-14T16:04:13.933 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:14.770 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 16:04 /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:14.770 DEBUG:teuthology.orchestra.run.smithi081:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:15.689 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 16:04 /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:15.690 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:16.574 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 14 16:04 /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:16.575 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:16.625 DEBUG:teuthology.orchestra.run.smithi081:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:16.676 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T16:04:16.740 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b on all hosts... 2023-07-14T16:04:16.741 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T16:04:16.745 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T16:04:16.750 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-14T16:04:18.227 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T16:04:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T16:04:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T16:04:45.991 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:04:45.991 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T16:04:45.991 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T16:04:45.992 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2023-07-14T16:04:45.992 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T16:04:45.992 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:04:45.992 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:05:05.018 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2023-07-14T16:05:05.018 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T16:05:05.018 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T16:05:05.018 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2023-07-14T16:05:05.018 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T16:05:05.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2023-07-14T16:05:05.019 INFO:teuthology.orchestra.run.smithi164.stdout:} 2023-07-14T16:05:05.389 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-07-14T16:05:05.389 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T16:05:05.390 INFO:teuthology.orchestra.run.smithi081.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-14T16:05:05.390 INFO:teuthology.orchestra.run.smithi081.stdout: "repo_digests": [ 2023-07-14T16:05:05.390 INFO:teuthology.orchestra.run.smithi081.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-14T16:05:05.390 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2023-07-14T16:05:05.390 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-07-14T16:05:05.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2023-07-14T16:05:05.497 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph 2023-07-14T16:05:05.569 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2023-07-14T16:05:05.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2023-07-14T16:05:05.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /etc/ceph 2023-07-14T16:05:05.781 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2023-07-14T16:05:05.850 INFO:tasks.cephadm:Writing seed config... 2023-07-14T16:05:05.851 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-14T16:05:05.851 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-14T16:05:05.851 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-14T16:05:05.852 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-14T16:05:05.852 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-14T16:05:05.852 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-14T16:05:05.852 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-14T16:05:05.853 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-14T16:05:05.853 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-14T16:05:05.853 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-14T16:05:05.853 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-14T16:05:05.853 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-14T16:05:05.854 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-14T16:05:05.854 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-14T16:05:05.854 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-14T16:05:05.854 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-14T16:05:05.854 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-14T16:05:05.855 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-14T16:05:05.855 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-14T16:05:05.855 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-14T16:05:05.855 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-14T16:05:05.855 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-14T16:05:05.855 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-14T16:05:05.856 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-14T16:05:05.856 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-14T16:05:05.856 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-14T16:05:05.856 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-14T16:05:05.857 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-07-14T16:05:05.857 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-14T16:05:05.857 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-07-14T16:05:05.857 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-07-14T16:05:05.857 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-14T16:05:05.858 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-14T16:05:05.858 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-07-14T16:05:05.858 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2023-07-14T16:05:05.858 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-14T16:05:05.858 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-14T16:05:05.859 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-14T16:05:05.859 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-14T16:05:05.859 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2023-07-14T16:05:05.859 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-14T16:05:05.860 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-14T16:05:05.860 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-14T16:05:05.860 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-14T16:05:05.861 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:05:05.862 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-14T16:05:05.905 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1356e48a-2260-11ee-9b33-001a4aab830c [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-14T16:05:05.906 DEBUG:teuthology.orchestra.run.smithi062:mon.a> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.a.service 2023-07-14T16:05:05.951 DEBUG:teuthology.orchestra.run.smithi062:mgr.x> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.x.service 2023-07-14T16:05:05.996 INFO:tasks.cephadm:Bootstrapping... 2023-07-14T16:05:05.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b -v bootstrap --fsid 1356e48a-2260-11ee-9b33-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.62 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:05:06.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:05:06.067 INFO:journalctl@ceph.mgr.x.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:05:06.227 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-07-14T16:05:06.227 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b', '-v', 'bootstrap', '--fsid', '1356e48a-2260-11ee-9b33-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.62', '--skip-admin-label'] 2023-07-14T16:05:06.401 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 4.1.1 2023-07-14T16:05:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-14T16:05:06.402 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying podman|docker is present... 2023-07-14T16:05:06.574 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 4.1.1 2023-07-14T16:05:06.575 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying lvm2 is present... 2023-07-14T16:05:06.575 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying time synchronization is in place... 2023-07-14T16:05:06.589 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T16:05:06.589 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T16:05:06.602 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T16:05:06.602 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2023-07-14T16:05:06.616 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2023-07-14T16:05:06.630 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2023-07-14T16:05:06.630 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2023-07-14T16:05:06.630 INFO:teuthology.orchestra.run.smithi062.stdout:Repeating the final host check... 2023-07-14T16:05:06.801 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 4.1.1 2023-07-14T16:05:06.802 INFO:teuthology.orchestra.run.smithi062.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-14T16:05:06.802 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl is present 2023-07-14T16:05:06.802 INFO:teuthology.orchestra.run.smithi062.stdout:lvcreate is present 2023-07-14T16:05:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T16:05:06.815 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T16:05:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T16:05:06.828 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2023-07-14T16:05:06.841 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2023-07-14T16:05:06.854 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2023-07-14T16:05:06.855 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2023-07-14T16:05:06.855 INFO:teuthology.orchestra.run.smithi062.stdout:Host looks OK 2023-07-14T16:05:06.855 INFO:teuthology.orchestra.run.smithi062.stdout:Cluster fsid: 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:06.855 INFO:teuthology.orchestra.run.smithi062.stdout:Acquiring lock 140304137464296 on /run/cephadm/1356e48a-2260-11ee-9b33-001a4aab830c.lock 2023-07-14T16:05:06.856 INFO:teuthology.orchestra.run.smithi062.stdout:Lock 140304137464296 acquired on /run/cephadm/1356e48a-2260-11ee-9b33-001a4aab830c.lock 2023-07-14T16:05:06.856 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 3300 ... 2023-07-14T16:05:06.857 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 6789 ... 2023-07-14T16:05:06.857 INFO:teuthology.orchestra.run.smithi062.stdout:Base mon IP(s) is [172.21.15.62:3300, 172.21.15.62:6789], mon addrv is [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2023-07-14T16:05:06.865 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-14T16:05:06.865 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.62 metric 100 2023-07-14T16:05:06.871 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-14T16:05:06.872 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-14T16:05:06.872 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-14T16:05:06.878 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-14T16:05:06.879 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-14T16:05:06.879 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T16:05:06.879 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-14T16:05:06.880 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 fe80::f0a7:c988:251c:603/64 scope link noprefixroute 2023-07-14T16:05:06.880 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T16:05:06.881 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2023-07-14T16:05:06.881 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2023-07-14T16:05:06.881 INFO:teuthology.orchestra.run.smithi062.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-14T16:05:06.881 INFO:teuthology.orchestra.run.smithi062.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-14T16:05:06.882 INFO:teuthology.orchestra.run.smithi062.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T16:05:07.455 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295 2023-07-14T16:05:07.455 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-14T16:05:07.455 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Getting image source signatures 2023-07-14T16:05:07.456 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:d8eac6e0ff3c58bd5607bffacb1c2214d20cdd3ff8cfaf23cfeb0cc60b2d0d6b 2023-07-14T16:05:07.456 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-14T16:05:07.456 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying config sha256:afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295 2023-07-14T16:05:07.456 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-14T16:05:07.456 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Storing signatures 2023-07-14T16:05:08.432 INFO:teuthology.orchestra.run.smithi062.stdout:ceph: stdout ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev) 2023-07-14T16:05:08.433 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph version: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev) 2023-07-14T16:05:08.434 INFO:teuthology.orchestra.run.smithi062.stdout:Extracting ceph user uid/gid from container image... 2023-07-14T16:05:09.307 INFO:teuthology.orchestra.run.smithi062.stdout:stat: stdout 167 167 2023-07-14T16:05:09.307 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial keys... 2023-07-14T16:05:10.083 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQA1crFkCbryKxAAMMS4N06RC4MRwmdO9MiLkw== 2023-07-14T16:05:10.965 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQA2crFkI0uHHhAAEPfj+maQQpHokKVo3AQ7MQ== 2023-07-14T16:05:18.957 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQA+crFkGWSaJBAAs+OOoA97YujHoGlY7YodlQ== 2023-07-14T16:05:18.958 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial monmap... 2023-07-14T16:05:19.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T16:05:19.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-14T16:05:19.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:19.835 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T16:05:19.836 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool for a [v2:172.21.15.62:3300,v1:172.21.15.62:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T16:05:19.836 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = pacific 2023-07-14T16:05:19.836 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: set fsid to 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:19.836 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T16:05:19.836 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:05:19.836 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mon... 2023-07-14T16:05:20.712 INFO:teuthology.orchestra.run.smithi062.stdout:create mon.a on 2023-07-14T16:05:20.952 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-14T16:05:21.160 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-14T16:05:21.407 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1356e48a-2260-11ee-9b33-001a4aab830c.target → /etc/systemd/system/ceph-1356e48a-2260-11ee-9b33-001a4aab830c.target. 2023-07-14T16:05:21.407 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1356e48a-2260-11ee-9b33-001a4aab830c.target → /etc/systemd/system/ceph-1356e48a-2260-11ee-9b33-001a4aab830c.target. 2023-07-14T16:05:21.758 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.a 2023-07-14T16:05:21.758 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.a.service: Unit ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.a.service not loaded. 2023-07-14T16:05:21.970 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1356e48a-2260-11ee-9b33-001a4aab830c.target.wants/ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-1356e48a-2260-11ee-9b33-001a4aab830c@.service. 2023-07-14T16:05:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:22 smithi062 systemd[1]: Starting Ceph mon.a for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:05:22.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:22 smithi062 podman[94996]: 2023-07-14T16:05:22.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:22 smithi062 bash[94996]: c5d4c6eebcc691df8023ee25f25c337d7620632383b0e970391c917d38e6d7a4 2023-07-14T16:05:22.785 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T16:05:22.786 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2023-07-14T16:05:22.801 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T16:05:22.802 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2023-07-14T16:05:22.802 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2023-07-14T16:05:22.802 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T16:05:22.802 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon to start... 2023-07-14T16:05:22.803 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon... 2023-07-14T16:05:23.077 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:22 smithi062 ceph-mon[95036]: mkfs 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:23.078 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:22 smithi062 ceph-mon[95036]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T16:05:23.078 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:22 smithi062 systemd[1]: Started Ceph mon.a for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:05:23.887 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout cluster: 2023-07-14T16:05:23.888 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout id: 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:23.889 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-14T16:05:23.889 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:23.889 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout services: 2023-07-14T16:05:23.889 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.674504s) 2023-07-14T16:05:23.889 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-14T16:05:23.889 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-14T16:05:23.890 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:23.890 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout data: 2023-07-14T16:05:23.890 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-14T16:05:23.894 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-14T16:05:23.894 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-14T16:05:23.894 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pgs: 2023-07-14T16:05:23.894 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:23.894 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.354+0000 7fb629a6e700 1 Processor -- start 2023-07-14T16:05:23.894 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.354+0000 7fb629a6e700 1 -- start start 2023-07-14T16:05:23.894 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.354+0000 7fb629a6e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb6240791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:23.895 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.354+0000 7fb629a6e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624079720 con 0x7fb62407ace0 2023-07-14T16:05:23.895 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.355+0000 7fb628a6c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb6240791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:23.895 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.355+0000 7fb628a6c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb6240791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42374/0 (socket says 172.21.15.62:42374) 2023-07-14T16:05:23.895 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.355+0000 7fb628a6c700 1 -- 172.21.15.62:0/2793898780 learned_addr learned my addr 172.21.15.62:0/2793898780 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:23.895 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.355+0000 7fb628a6c700 1 -- 172.21.15.62:0/2793898780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624079860 con 0x7fb62407ace0 2023-07-14T16:05:23.895 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.355+0000 7fb628a6c700 1 --2- 172.21.15.62:0/2793898780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb6240791e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6180098a0 tx=0x7fb618009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86ac55c31ded0b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:23.895 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb6237fe700 1 -- 172.21.15.62:0/2793898780 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb618004160 con 0x7fb62407ace0 2023-07-14T16:05:23.896 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb6237fe700 1 -- 172.21.15.62:0/2793898780 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6180042c0 con 0x7fb62407ace0 2023-07-14T16:05:23.896 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb6237fe700 1 -- 172.21.15.62:0/2793898780 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6180045a0 con 0x7fb62407ace0 2023-07-14T16:05:23.896 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb629a6e700 1 -- 172.21.15.62:0/2793898780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 msgr2=0x7fb6240791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:23.896 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb629a6e700 1 --2- 172.21.15.62:0/2793898780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb6240791e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6180098a0 tx=0x7fb618009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:23.896 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb629a6e700 1 -- 172.21.15.62:0/2793898780 shutdown_connections 2023-07-14T16:05:23.896 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb629a6e700 1 --2- 172.21.15.62:0/2793898780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb6240791e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:23.896 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb629a6e700 1 -- 172.21.15.62:0/2793898780 >> 172.21.15.62:0/2793898780 conn(0x7fb624100710 msgr2=0x7fb624102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:23.897 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb629a6e700 1 -- 172.21.15.62:0/2793898780 shutdown_connections 2023-07-14T16:05:23.897 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.356+0000 7fb629a6e700 1 -- 172.21.15.62:0/2793898780 wait complete. 2023-07-14T16:05:23.897 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.358+0000 7fb629a6e700 1 Processor -- start 2023-07-14T16:05:23.897 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.358+0000 7fb629a6e700 1 -- start start 2023-07-14T16:05:23.897 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.359+0000 7fb629a6e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb62419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:23.897 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.359+0000 7fb629a6e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62419ac70 con 0x7fb62407ace0 2023-07-14T16:05:23.897 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.359+0000 7fb628a6c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb62419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:23.897 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.359+0000 7fb628a6c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb62419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42378/0 (socket says 172.21.15.62:42378) 2023-07-14T16:05:23.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.359+0000 7fb628a6c700 1 -- 172.21.15.62:0/1139351134 learned_addr learned my addr 172.21.15.62:0/1139351134 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:23.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.359+0000 7fb628a6c700 1 -- 172.21.15.62:0/1139351134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb618009550 con 0x7fb62407ace0 2023-07-14T16:05:23.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.360+0000 7fb628a6c700 1 --2- 172.21.15.62:0/1139351134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb62419a730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb618004000 tx=0x7fb618004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:23.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.360+0000 7fb621ffb700 1 -- 172.21.15.62:0/1139351134 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb618011630 con 0x7fb62407ace0 2023-07-14T16:05:23.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.360+0000 7fb621ffb700 1 -- 172.21.15.62:0/1139351134 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618011790 con 0x7fb62407ace0 2023-07-14T16:05:23.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.360+0000 7fb621ffb700 1 -- 172.21.15.62:0/1139351134 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb618011a30 con 0x7fb62407ace0 2023-07-14T16:05:23.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.360+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62419ae70 con 0x7fb62407ace0 2023-07-14T16:05:23.898 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.360+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62419b290 con 0x7fb62407ace0 2023-07-14T16:05:23.899 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.360+0000 7fb621ffb700 1 -- 172.21.15.62:0/1139351134 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb618022070 con 0x7fb62407ace0 2023-07-14T16:05:23.899 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.360+0000 7fb621ffb700 1 -- 172.21.15.62:0/1139351134 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb61801c450 con 0x7fb62407ace0 2023-07-14T16:05:23.899 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.361+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb624194570 con 0x7fb62407ace0 2023-07-14T16:05:23.899 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.364+0000 7fb621ffb700 1 -- 172.21.15.62:0/1139351134 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb61801b450 con 0x7fb62407ace0 2023-07-14T16:05:23.899 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.440+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb62402c9b0 con 0x7fb62407ace0 2023-07-14T16:05:23.899 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.440+0000 7fb621ffb700 1 -- 172.21.15.62:0/1139351134 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fb61801b5f0 con 0x7fb62407ace0 2023-07-14T16:05:23.899 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.441+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 msgr2=0x7fb62419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:23.900 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.441+0000 7fb629a6e700 1 --2- 172.21.15.62:0/1139351134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb62419a730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb618004000 tx=0x7fb618004750 comp rx=0 tx=0).stop 2023-07-14T16:05:23.900 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.441+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 shutdown_connections 2023-07-14T16:05:23.900 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.441+0000 7fb629a6e700 1 --2- 172.21.15.62:0/1139351134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb62407ace0 0x7fb62419a730 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:23.900 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.441+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 >> 172.21.15.62:0/1139351134 conn(0x7fb624100710 msgr2=0x7fb624101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:23.900 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.441+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 shutdown_connections 2023-07-14T16:05:23.900 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:23.441+0000 7fb629a6e700 1 -- 172.21.15.62:0/1139351134 wait complete. 2023-07-14T16:05:23.900 INFO:teuthology.orchestra.run.smithi062.stdout:mon is available 2023-07-14T16:05:23.900 INFO:teuthology.orchestra.run.smithi062.stdout:Assimilating anything we can from ceph.conf... 2023-07-14T16:05:24.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:23 smithi062 ceph-mon[95036]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T16:05:24.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:23 smithi062 ceph-mon[95036]: monmap e1: 1 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} 2023-07-14T16:05:24.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:23 smithi062 ceph-mon[95036]: fsmap 2023-07-14T16:05:24.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:23 smithi062 ceph-mon[95036]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T16:05:24.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:23 smithi062 ceph-mon[95036]: mgrmap e1: no daemons active 2023-07-14T16:05:24.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:23 smithi062 ceph-mon[95036]: from='client.? 172.21.15.62:0/1139351134' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-14T16:05:24.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:24 smithi062 ceph-mon[95036]: from='client.? 172.21.15.62:0/1028528845' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T16:05:24.860 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:24 smithi062 ceph-mon[95036]: from='client.? 172.21.15.62:0/1028528845' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-14T16:05:24.877 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:24.878 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T16:05:24.878 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:24.878 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2023-07-14T16:05:24.878 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T16:05:24.878 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T16:05:24.879 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T16:05:24.879 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T16:05:24.879 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:24.879 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T16:05:24.879 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T16:05:24.879 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:24.879 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T16:05:24.879 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2023-07-14T16:05:24.880 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T16:05:24.880 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T16:05:24.880 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T16:05:24.880 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T16:05:24.880 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T16:05:24.880 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.436+0000 7f2a1c5b9700 1 Processor -- start 2023-07-14T16:05:24.881 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.436+0000 7f2a1c5b9700 1 -- start start 2023-07-14T16:05:24.881 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.436+0000 7f2a1c5b9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a14079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:24.881 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.436+0000 7f2a1c5b9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a14079740 con 0x7f2a1407ad00 2023-07-14T16:05:24.881 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.437+0000 7f2a1a355700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a14079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:24.881 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.437+0000 7f2a1a355700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a14079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42382/0 (socket says 172.21.15.62:42382) 2023-07-14T16:05:24.881 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.437+0000 7f2a1a355700 1 -- 172.21.15.62:0/1057996811 learned_addr learned my addr 172.21.15.62:0/1057996811 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:24.881 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.437+0000 7f2a1a355700 1 -- 172.21.15.62:0/1057996811 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14079880 con 0x7f2a1407ad00 2023-07-14T16:05:24.881 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.437+0000 7f2a1a355700 1 --2- 172.21.15.62:0/1057996811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a14079200 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2a100098a0 tx=0x7f2a10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313d484d17b70310 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:24.882 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a19353700 1 -- 172.21.15.62:0/1057996811 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a10004160 con 0x7f2a1407ad00 2023-07-14T16:05:24.882 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a19353700 1 -- 172.21.15.62:0/1057996811 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a100042c0 con 0x7f2a1407ad00 2023-07-14T16:05:24.882 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a19353700 1 -- 172.21.15.62:0/1057996811 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a100045a0 con 0x7f2a1407ad00 2023-07-14T16:05:24.882 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1057996811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 msgr2=0x7f2a14079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:24.882 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a1c5b9700 1 --2- 172.21.15.62:0/1057996811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a14079200 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2a100098a0 tx=0x7f2a10009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:24.882 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1057996811 shutdown_connections 2023-07-14T16:05:24.882 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a1c5b9700 1 --2- 172.21.15.62:0/1057996811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a14079200 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:24.883 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1057996811 >> 172.21.15.62:0/1057996811 conn(0x7f2a14100870 msgr2=0x7f2a14102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:24.883 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.438+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1057996811 shutdown_connections 2023-07-14T16:05:24.883 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.439+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1057996811 wait complete. 2023-07-14T16:05:24.883 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.440+0000 7f2a1c5b9700 1 Processor -- start 2023-07-14T16:05:24.883 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.441+0000 7f2a1c5b9700 1 -- start start 2023-07-14T16:05:24.883 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.441+0000 7f2a1c5b9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a1419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:24.883 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.441+0000 7f2a1c5b9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1419ae90 con 0x7f2a1407ad00 2023-07-14T16:05:24.883 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.441+0000 7f2a1a355700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a1419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:24.884 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.441+0000 7f2a1a355700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a1419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42390/0 (socket says 172.21.15.62:42390) 2023-07-14T16:05:24.884 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.441+0000 7f2a1a355700 1 -- 172.21.15.62:0/1028528845 learned_addr learned my addr 172.21.15.62:0/1028528845 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:24.884 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.442+0000 7f2a1a355700 1 -- 172.21.15.62:0/1028528845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a10009550 con 0x7f2a1407ad00 2023-07-14T16:05:24.884 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.442+0000 7f2a1a355700 1 --2- 172.21.15.62:0/1028528845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a1419a950 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2a10004000 tx=0x7f2a10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:24.884 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.442+0000 7f2a0b7fe700 1 -- 172.21.15.62:0/1028528845 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a10011630 con 0x7f2a1407ad00 2023-07-14T16:05:24.884 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.442+0000 7f2a0b7fe700 1 -- 172.21.15.62:0/1028528845 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a10011790 con 0x7f2a1407ad00 2023-07-14T16:05:24.884 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.442+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1419b090 con 0x7f2a1407ad00 2023-07-14T16:05:24.885 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.442+0000 7f2a0b7fe700 1 -- 172.21.15.62:0/1028528845 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a10011a30 con 0x7f2a1407ad00 2023-07-14T16:05:24.885 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.442+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1419b4b0 con 0x7f2a1407ad00 2023-07-14T16:05:24.885 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.443+0000 7f2a0b7fe700 1 -- 172.21.15.62:0/1028528845 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2a10022070 con 0x7f2a1407ad00 2023-07-14T16:05:24.885 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.443+0000 7f2a0b7fe700 1 -- 172.21.15.62:0/1028528845 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2a1001c450 con 0x7f2a1407ad00 2023-07-14T16:05:24.885 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.444+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a14194750 con 0x7f2a1407ad00 2023-07-14T16:05:24.885 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.446+0000 7f2a0b7fe700 1 -- 172.21.15.62:0/1028528845 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f2a1001b450 con 0x7f2a1407ad00 2023-07-14T16:05:24.885 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.520+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2a1402c9e0 con 0x7f2a1407ad00 2023-07-14T16:05:24.885 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.524+0000 7f2a0b7fe700 1 -- 172.21.15.62:0/1028528845 <== mon.0 v2:172.21.15.62:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2a1001b5f0 con 0x7f2a1407ad00 2023-07-14T16:05:24.886 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.524+0000 7f2a0b7fe700 1 -- 172.21.15.62:0/1028528845 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+454 (secure 0 0 0) 0x7f2a10037540 con 0x7f2a1407ad00 2023-07-14T16:05:24.886 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.525+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 msgr2=0x7f2a1419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:24.886 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.525+0000 7f2a1c5b9700 1 --2- 172.21.15.62:0/1028528845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a1419a950 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2a10004000 tx=0x7f2a10004750 comp rx=0 tx=0).stop 2023-07-14T16:05:24.886 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.525+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 shutdown_connections 2023-07-14T16:05:24.886 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.525+0000 7f2a1c5b9700 1 --2- 172.21.15.62:0/1028528845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a1407ad00 0x7f2a1419a950 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:24.886 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.525+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 >> 172.21.15.62:0/1028528845 conn(0x7f2a14100870 msgr2=0x7f2a141021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:24.887 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.525+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 shutdown_connections 2023-07-14T16:05:24.887 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:24.526+0000 7f2a1c5b9700 1 -- 172.21.15.62:0/1028528845 wait complete. 2023-07-14T16:05:24.887 INFO:teuthology.orchestra.run.smithi062.stdout:Generating new minimal ceph.conf... 2023-07-14T16:05:25.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.442+0000 7f03429e2700 1 Processor -- start 2023-07-14T16:05:25.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.443+0000 7f03429e2700 1 -- start start 2023-07-14T16:05:25.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.443+0000 7f03429e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:25.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.443+0000 7f03429e2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c079700 con 0x7f033c07acc0 2023-07-14T16:05:25.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.444+0000 7f033bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:25.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.444+0000 7f033bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42396/0 (socket says 172.21.15.62:42396) 2023-07-14T16:05:25.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.444+0000 7f033bfff700 1 -- 172.21.15.62:0/1514456076 learned_addr learned my addr 172.21.15.62:0/1514456076 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:25.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.444+0000 7f033bfff700 1 -- 172.21.15.62:0/1514456076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c079840 con 0x7f033c07acc0 2023-07-14T16:05:25.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.444+0000 7f033bfff700 1 --2- 172.21.15.62:0/1514456076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c0791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098a0 tx=0x7f032c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c67e2f35f5ee40 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:25.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.444+0000 7f033affd700 1 -- 172.21.15.62:0/1514456076 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032c004160 con 0x7f033c07acc0 2023-07-14T16:05:25.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.444+0000 7f033affd700 1 -- 172.21.15.62:0/1514456076 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f032c0042c0 con 0x7f033c07acc0 2023-07-14T16:05:25.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.445+0000 7f033affd700 1 -- 172.21.15.62:0/1514456076 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032c011940 con 0x7f033c07acc0 2023-07-14T16:05:25.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.445+0000 7f03429e2700 1 -- 172.21.15.62:0/1514456076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 msgr2=0x7f033c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:25.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.445+0000 7f03429e2700 1 --2- 172.21.15.62:0/1514456076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c0791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098a0 tx=0x7f032c009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:25.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.445+0000 7f03429e2700 1 -- 172.21.15.62:0/1514456076 shutdown_connections 2023-07-14T16:05:25.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.445+0000 7f03429e2700 1 --2- 172.21.15.62:0/1514456076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c0791c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:25.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.445+0000 7f03429e2700 1 -- 172.21.15.62:0/1514456076 >> 172.21.15.62:0/1514456076 conn(0x7f033c100870 msgr2=0x7f033c102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:25.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.445+0000 7f03429e2700 1 -- 172.21.15.62:0/1514456076 shutdown_connections 2023-07-14T16:05:25.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.445+0000 7f03429e2700 1 -- 172.21.15.62:0/1514456076 wait complete. 2023-07-14T16:05:25.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.446+0000 7f03429e2700 1 Processor -- start 2023-07-14T16:05:25.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.446+0000 7f03429e2700 1 -- start start 2023-07-14T16:05:25.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.447+0000 7f03429e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:25.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.447+0000 7f03429e2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c19b630 con 0x7f033c07acc0 2023-07-14T16:05:25.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.447+0000 7f033bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:25.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.447+0000 7f033bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c19b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42400/0 (socket says 172.21.15.62:42400) 2023-07-14T16:05:25.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.447+0000 7f033bfff700 1 -- 172.21.15.62:0/3078066791 learned_addr learned my addr 172.21.15.62:0/3078066791 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:25.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.447+0000 7f033bfff700 1 -- 172.21.15.62:0/3078066791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009550 con 0x7f033c07acc0 2023-07-14T16:05:25.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.448+0000 7f033bfff700 1 --2- 172.21.15.62:0/3078066791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c19b0f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f032c000c00 tx=0x7f032c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:25.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.448+0000 7f03397fa700 1 -- 172.21.15.62:0/3078066791 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032c0036a0 con 0x7f033c07acc0 2023-07-14T16:05:25.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.448+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c19b830 con 0x7f033c07acc0 2023-07-14T16:05:25.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.448+0000 7f03397fa700 1 -- 172.21.15.62:0/3078066791 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f032c003c70 con 0x7f033c07acc0 2023-07-14T16:05:25.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.448+0000 7f03397fa700 1 -- 172.21.15.62:0/3078066791 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032c0193f0 con 0x7f033c07acc0 2023-07-14T16:05:25.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.448+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c19bc50 con 0x7f033c07acc0 2023-07-14T16:05:25.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.448+0000 7f03397fa700 1 -- 172.21.15.62:0/3078066791 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f032c019970 con 0x7f033c07acc0 2023-07-14T16:05:25.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.448+0000 7f03397fa700 1 -- 172.21.15.62:0/3078066791 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f032c01a3d0 con 0x7f033c07acc0 2023-07-14T16:05:25.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.449+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c061b40 con 0x7f033c07acc0 2023-07-14T16:05:25.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.452+0000 7f03397fa700 1 -- 172.21.15.62:0/3078066791 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f032c029060 con 0x7f033c07acc0 2023-07-14T16:05:25.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.525+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f033c19e780 con 0x7f033c07acc0 2023-07-14T16:05:25.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.525+0000 7f03397fa700 1 -- 172.21.15.62:0/3078066791 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f032c033090 con 0x7f033c07acc0 2023-07-14T16:05:25.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.526+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 msgr2=0x7f033c19b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:25.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.526+0000 7f03429e2700 1 --2- 172.21.15.62:0/3078066791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c19b0f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f032c000c00 tx=0x7f032c011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:25.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.526+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 shutdown_connections 2023-07-14T16:05:25.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.526+0000 7f03429e2700 1 --2- 172.21.15.62:0/3078066791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f033c07acc0 0x7f033c19b0f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:25.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.526+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 >> 172.21.15.62:0/3078066791 conn(0x7f033c100870 msgr2=0x7f033c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:25.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.526+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 shutdown_connections 2023-07-14T16:05:25.865 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:25.527+0000 7f03429e2700 1 -- 172.21.15.62:0/3078066791 wait complete. 2023-07-14T16:05:25.865 INFO:teuthology.orchestra.run.smithi062.stdout:Restarting the monitor... 2023-07-14T16:05:25.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:25 smithi062 ceph-mon[95036]: from='client.? 172.21.15.62:0/3078066791' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:05:26.321 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:25 smithi062 systemd[1]: Stopping Ceph mon.a for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:05:26.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:26 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-a[95012]: 2023-07-14T16:05:26.343+0000 7f2c5c852700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:05:26.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:26 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-a[95012]: 2023-07-14T16:05:26.343+0000 7f2c5c852700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-14T16:05:26.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:26 smithi062 bash[95373]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-a 2023-07-14T16:05:27.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 systemd[1]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.a.service: Succeeded. 2023-07-14T16:05:27.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 systemd[1]: Stopped Ceph mon.a for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:05:27.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 systemd[1]: Starting Ceph mon.a for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:05:27.817 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 podman[95503]: 2023-07-14T16:05:27.817 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 bash[95503]: cdb498e2ac4253e7596236c2955c7bf15e1694a32fba599995a66e07e428faa7 2023-07-14T16:05:27.817 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T16:05:27.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-14T16:05:27.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: pidfile_write: ignore empty --pid-file 2023-07-14T16:05:27.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: load: jerasure load: lrc 2023-07-14T16:05:27.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: RocksDB version: 7.9.2 2023-07-14T16:05:27.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Git sha 0 2023-07-14T16:05:27.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-14T16:05:27.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: DB SUMMARY 2023-07-14T16:05:27.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: DB Session ID: OZ9LMFFSX0UB1ADPYVWV 2023-07-14T16:05:27.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: CURRENT file: CURRENT 2023-07-14T16:05:27.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T16:05:27.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-14T16:05:27.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-14T16:05:27.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98172 ; 2023-07-14T16:05:27.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.error_if_exists: 0 2023-07-14T16:05:27.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.create_if_missing: 0 2023-07-14T16:05:27.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.paranoid_checks: 1 2023-07-14T16:05:27.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T16:05:27.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T16:05:27.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T16:05:27.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.env: 0x5612d98a2720 2023-07-14T16:05:27.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T16:05:27.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.info_log: 0x5612da93d280 2023-07-14T16:05:27.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T16:05:27.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.statistics: (nil) 2023-07-14T16:05:27.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.use_fsync: 0 2023-07-14T16:05:27.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_log_file_size: 0 2023-07-14T16:05:27.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T16:05:27.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T16:05:27.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T16:05:27.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T16:05:27.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.allow_fallocate: 1 2023-07-14T16:05:27.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T16:05:27.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T16:05:27.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.use_direct_reads: 0 2023-07-14T16:05:27.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T16:05:27.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T16:05:27.879 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.db_log_dir: 2023-07-14T16:05:27.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.wal_dir: 2023-07-14T16:05:27.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T16:05:27.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T16:05:27.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T16:05:27.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T16:05:27.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T16:05:27.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T16:05:27.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T16:05:27.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T16:05:27.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.write_buffer_manager: 0x5612da85e3c0 2023-07-14T16:05:27.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T16:05:27.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T16:05:27.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T16:05:27.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T16:05:27.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T16:05:27.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T16:05:27.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T16:05:27.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T16:05:27.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.unordered_write: 0 2023-07-14T16:05:27.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T16:05:27.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T16:05:27.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T16:05:27.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T16:05:27.883 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.row_cache: None 2023-07-14T16:05:27.883 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.wal_filter: None 2023-07-14T16:05:27.883 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T16:05:27.883 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T16:05:27.883 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.two_write_queues: 0 2023-07-14T16:05:27.883 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T16:05:27.883 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.wal_compression: 0 2023-07-14T16:05:27.884 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.atomic_flush: 0 2023-07-14T16:05:27.884 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T16:05:27.884 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T16:05:27.884 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T16:05:27.884 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.log_readahead_size: 0 2023-07-14T16:05:27.884 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T16:05:27.884 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T16:05:27.884 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T16:05:27.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T16:05:27.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T16:05:27.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T16:05:27.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T16:05:27.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_background_jobs: 2 2023-07-14T16:05:27.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_background_compactions: -1 2023-07-14T16:05:27.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_subcompactions: 1 2023-07-14T16:05:27.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T16:05:27.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T16:05:27.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T16:05:27.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T16:05:27.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T16:05:27.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T16:05:27.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T16:05:27.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T16:05:27.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_open_files: -1 2023-07-14T16:05:27.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T16:05:27.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T16:05:27.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T16:05:27.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T16:05:27.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_background_flushes: -1 2023-07-14T16:05:27.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Compression algorithms supported: 2023-07-14T16:05:27.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T16:05:27.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: kZSTD supported: 0 2023-07-14T16:05:27.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: kXpressCompression supported: 0 2023-07-14T16:05:27.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T16:05:27.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: kZlibCompression supported: 1 2023-07-14T16:05:27.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: kSnappyCompression supported: 1 2023-07-14T16:05:27.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: kLZ4Compression supported: 1 2023-07-14T16:05:27.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: kBZip2Compression supported: 0 2023-07-14T16:05:27.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T16:05:27.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T16:05:27.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-14T16:05:27.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T16:05:27.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T16:05:27.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.merge_operator: 2023-07-14T16:05:27.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_filter: None 2023-07-14T16:05:27.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T16:05:27.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T16:05:27.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T16:05:27.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T16:05:27.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5612da93d380) 2023-07-14T16:05:27.891 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2023-07-14T16:05:27.891 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T16:05:27.891 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T16:05:27.892 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2023-07-14T16:05:27.892 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2023-07-14T16:05:27.892 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2023-07-14T16:05:27.892 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2023-07-14T16:05:27.892 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T16:05:27.892 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 4 2023-07-14T16:05:27.892 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2023-07-14T16:05:27.892 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x5612da8fd350 2023-07-14T16:05:27.893 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2023-07-14T16:05:27.893 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2023-07-14T16:05:27.893 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2023-07-14T16:05:27.893 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2023-07-14T16:05:27.893 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2023-07-14T16:05:27.893 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2023-07-14T16:05:27.893 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2023-07-14T16:05:27.893 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2023-07-14T16:05:27.894 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2023-07-14T16:05:27.894 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2023-07-14T16:05:27.894 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2023-07-14T16:05:27.894 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2023-07-14T16:05:27.894 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2023-07-14T16:05:27.894 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2023-07-14T16:05:27.894 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2023-07-14T16:05:27.894 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: bloomfilter 2023-07-14T16:05:27.895 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2023-07-14T16:05:27.895 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2023-07-14T16:05:27.895 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2023-07-14T16:05:27.895 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 5 2023-07-14T16:05:27.895 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2023-07-14T16:05:27.895 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2023-07-14T16:05:27.895 INFO:journalctl@ceph.mon.a.smithi062.stdout: max_auto_readahead_size: 262144 2023-07-14T16:05:27.895 INFO:journalctl@ceph.mon.a.smithi062.stdout: prepopulate_block_cache: 0 2023-07-14T16:05:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout: initial_auto_readahead_size: 8192 2023-07-14T16:05:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T16:05:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T16:05:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T16:05:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression: NoCompression 2023-07-14T16:05:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T16:05:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T16:05:27.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T16:05:27.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.num_levels: 7 2023-07-14T16:05:27.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T16:05:27.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T16:05:27.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T16:05:27.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T16:05:27.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T16:05:27.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T16:05:27.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T16:05:27.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T16:05:27.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T16:05:27.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T16:05:27.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T16:05:27.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T16:05:27.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T16:05:27.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T16:05:27.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T16:05:27.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T16:05:27.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T16:05:27.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T16:05:27.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T16:05:27.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T16:05:27.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T16:05:27.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T16:05:27.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T16:05:27.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T16:05:27.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T16:05:27.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T16:05:27.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T16:05:27.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T16:05:27.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T16:05:27.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T16:05:27.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T16:05:27.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T16:05:27.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T16:05:27.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T16:05:27.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T16:05:27.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T16:05:27.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T16:05:27.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T16:05:27.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T16:05:27.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T16:05:27.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T16:05:27.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T16:05:27.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T16:05:27.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T16:05:27.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T16:05:27.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T16:05:27.903 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T16:05:27.903 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T16:05:27.903 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T16:05:27.903 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T16:05:27.903 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T16:05:27.903 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T16:05:27.903 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T16:05:27.904 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.table_properties_collectors: 2023-07-14T16:05:27.904 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.inplace_update_support: 0 2023-07-14T16:05:27.904 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T16:05:27.904 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T16:05:27.904 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T16:05:27.904 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T16:05:27.904 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.bloom_locality: 0 2023-07-14T16:05:27.904 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.max_successive_merges: 0 2023-07-14T16:05:27.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T16:05:27.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T16:05:27.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T16:05:27.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T16:05:27.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.ttl: 2592000 2023-07-14T16:05:27.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T16:05:27.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T16:05:27.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T16:05:27.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.enable_blob_files: false 2023-07-14T16:05:27.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.min_blob_size: 0 2023-07-14T16:05:27.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T16:05:27.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T16:05:27.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T16:05:27.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T16:05:27.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T16:05:27.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T16:05:27.907 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T16:05:27.907 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T16:05:27.907 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-14T16:05:27.907 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-14T16:05:27.907 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 66774197-9746-45ff-9b7b-4b390356b996 2023-07-14T16:05:27.907 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350727849436, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-14T16:05:27.907 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-14T16:05:27.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350727851298, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 91517, "index_size": 287, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13511, "raw_average_key_size": 49, "raw_value_size": 84109, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689350727, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "66774197-9746-45ff-9b7b-4b390356b996", "db_session_id": "OZ9LMFFSX0UB1ADPYVWV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-14T16:05:27.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350727851430, "job": 1, "event": "recovery_finished"} 2023-07-14T16:05:27.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-14T16:05:27.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T16:05:27.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5612da996000 2023-07-14T16:05:27.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: DB pointer 0x5612da982000 2023-07-14T16:05:27.908 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T16:05:27.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T16:05:27.909 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2023-07-14T16:05:27.909 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T16:05:27.909 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T16:05:27.909 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T16:05:27.909 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T16:05:27.909 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T16:05:27.909 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T16:05:27.910 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T16:05:27.910 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2023-07-14T16:05:27.910 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2023-07-14T16:05:27.910 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T16:05:27.910 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T16:05:27.910 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 93.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:05:27.910 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 93.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:05:27.910 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:05:27.911 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2023-07-14T16:05:27.911 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2023-07-14T16:05:27.911 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T16:05:27.911 INFO:journalctl@ceph.mon.a.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T16:05:27.911 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 74.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:05:27.911 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2023-07-14T16:05:27.911 INFO:journalctl@ceph.mon.a.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T16:05:27.911 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2023-07-14T16:05:27.912 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T16:05:27.912 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T16:05:27.912 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T16:05:27.912 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T16:05:27.912 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T16:05:27.912 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T16:05:27.912 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 13.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T16:05:27.913 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 13.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T16:05:27.913 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T16:05:27.913 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache BinnedLRUCache@0x5612da8fd350#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-14T16:05:27.913 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-14T16:05:27.913 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2023-07-14T16:05:27.913 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T16:05:27.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:27.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a@-1(???) e1 preinit fsid 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:27.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a@-1(???).mds e1 new map 2023-07-14T16:05:27.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a@-1(???).mds e1 print_map 2023-07-14T16:05:27.914 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2023-07-14T16:05:27.914 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T16:05:27.914 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T16:05:27.914 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2023-07-14T16:05:27.914 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2023-07-14T16:05:27.914 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2023-07-14T16:05:27.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T16:05:27.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:05:27.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:05:27.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:05:27.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-14T16:05:27.916 INFO:teuthology.orchestra.run.smithi062.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-14T16:05:28.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T16:05:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: monmap e1: 1 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} 2023-07-14T16:05:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: fsmap 2023-07-14T16:05:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T16:05:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 ceph-mon[95543]: mgrmap e1: no daemons active 2023-07-14T16:05:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:27 smithi062 systemd[1]: Started Ceph mon.a for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:05:28.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.439+0000 7f07c9c64700 1 Processor -- start 2023-07-14T16:05:28.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.439+0000 7f07c9c64700 1 -- start start 2023-07-14T16:05:28.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.439+0000 7f07c9c64700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c40791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:29.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.439+0000 7f07c9c64700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c4079720 con 0x7f07c407ace0 2023-07-14T16:05:29.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.440+0000 7f07c8c62700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c40791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:29.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.440+0000 7f07c8c62700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c40791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42410/0 (socket says 172.21.15.62:42410) 2023-07-14T16:05:29.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.440+0000 7f07c8c62700 1 -- 172.21.15.62:0/4197470751 learned_addr learned my addr 172.21.15.62:0/4197470751 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:29.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.441+0000 7f07c8c62700 1 -- 172.21.15.62:0/4197470751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c4079860 con 0x7f07c407ace0 2023-07-14T16:05:29.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.441+0000 7f07c8c62700 1 --2- 172.21.15.62:0/4197470751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c40791e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f07b80098a0 tx=0x7f07b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ee65648ccf13f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:29.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.441+0000 7f07c37fe700 1 -- 172.21.15.62:0/4197470751 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07b8004160 con 0x7f07c407ace0 2023-07-14T16:05:29.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.441+0000 7f07c37fe700 1 -- 172.21.15.62:0/4197470751 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f07b80042c0 con 0x7f07c407ace0 2023-07-14T16:05:29.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.441+0000 7f07c37fe700 1 -- 172.21.15.62:0/4197470751 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07b8011940 con 0x7f07c407ace0 2023-07-14T16:05:29.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.442+0000 7f07c9c64700 1 -- 172.21.15.62:0/4197470751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 msgr2=0x7f07c40791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:29.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.442+0000 7f07c9c64700 1 --2- 172.21.15.62:0/4197470751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c40791e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f07b80098a0 tx=0x7f07b8009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:29.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.442+0000 7f07c9c64700 1 -- 172.21.15.62:0/4197470751 shutdown_connections 2023-07-14T16:05:29.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.442+0000 7f07c9c64700 1 --2- 172.21.15.62:0/4197470751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c40791e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:29.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.442+0000 7f07c9c64700 1 -- 172.21.15.62:0/4197470751 >> 172.21.15.62:0/4197470751 conn(0x7f07c4100710 msgr2=0x7f07c4102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:29.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.442+0000 7f07c9c64700 1 -- 172.21.15.62:0/4197470751 shutdown_connections 2023-07-14T16:05:29.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.442+0000 7f07c9c64700 1 -- 172.21.15.62:0/4197470751 wait complete. 2023-07-14T16:05:29.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.443+0000 7f07c9c64700 1 Processor -- start 2023-07-14T16:05:29.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.443+0000 7f07c9c64700 1 -- start start 2023-07-14T16:05:29.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.443+0000 7f07c9c64700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:29.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.443+0000 7f07c9c64700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c419b3f0 con 0x7f07c407ace0 2023-07-14T16:05:29.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.444+0000 7f07c8c62700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:29.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.444+0000 7f07c8c62700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42422/0 (socket says 172.21.15.62:42422) 2023-07-14T16:05:29.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.444+0000 7f07c8c62700 1 -- 172.21.15.62:0/1936868224 learned_addr learned my addr 172.21.15.62:0/1936868224 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:29.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.444+0000 7f07c8c62700 1 -- 172.21.15.62:0/1936868224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b8009550 con 0x7f07c407ace0 2023-07-14T16:05:29.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.444+0000 7f07c8c62700 1 --2- 172.21.15.62:0/1936868224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c419aeb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f07b8000c00 tx=0x7f07b8004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:29.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.444+0000 7f07c1ffb700 1 -- 172.21.15.62:0/1936868224 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07b80036a0 con 0x7f07c407ace0 2023-07-14T16:05:29.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.444+0000 7f07c1ffb700 1 -- 172.21.15.62:0/1936868224 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f07b8003c70 con 0x7f07c407ace0 2023-07-14T16:05:29.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.444+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c419b5f0 con 0x7f07c407ace0 2023-07-14T16:05:29.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.445+0000 7f07c1ffb700 1 -- 172.21.15.62:0/1936868224 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07b8024d70 con 0x7f07c407ace0 2023-07-14T16:05:29.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.445+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c419ba10 con 0x7f07c407ace0 2023-07-14T16:05:29.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.445+0000 7f07c1ffb700 1 -- 172.21.15.62:0/1936868224 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f07b801a730 con 0x7f07c407ace0 2023-07-14T16:05:29.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.445+0000 7f07c1ffb700 1 -- 172.21.15.62:0/1936868224 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f07b802dd00 con 0x7f07c407ace0 2023-07-14T16:05:29.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.446+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c410b300 con 0x7f07c407ace0 2023-07-14T16:05:29.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.449+0000 7f07c1ffb700 1 -- 172.21.15.62:0/1936868224 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f07b802d3f0 con 0x7f07c407ace0 2023-07-14T16:05:29.005 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.521+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f07c410b4c0 con 0x7f07c407ace0 2023-07-14T16:05:29.005 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.524+0000 7f07c1ffb700 1 -- 172.21.15.62:0/1936868224 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f07b80245d0 con 0x7f07c407ace0 2023-07-14T16:05:29.005 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.524+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 msgr2=0x7f07c419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:29.005 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.524+0000 7f07c9c64700 1 --2- 172.21.15.62:0/1936868224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c419aeb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f07b8000c00 tx=0x7f07b8004250 comp rx=0 tx=0).stop 2023-07-14T16:05:29.005 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.524+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 shutdown_connections 2023-07-14T16:05:29.005 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.524+0000 7f07c9c64700 1 --2- 172.21.15.62:0/1936868224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07c407ace0 0x7f07c419aeb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:29.005 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.524+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 >> 172.21.15.62:0/1936868224 conn(0x7f07c4100710 msgr2=0x7f07c4101480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:29.005 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.525+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 shutdown_connections 2023-07-14T16:05:29.006 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:28.525+0000 7f07c9c64700 1 -- 172.21.15.62:0/1936868224 wait complete. 2023-07-14T16:05:29.006 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-14T16:05:29.025 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:05:29.026 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mgr... 2023-07-14T16:05:29.026 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 9283 ... 2023-07-14T16:05:29.026 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 8765 ... 2023-07-14T16:05:29.387 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.x 2023-07-14T16:05:29.387 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.x.service: Unit ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.x.service not loaded. 2023-07-14T16:05:29.596 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1356e48a-2260-11ee-9b33-001a4aab830c.target.wants/ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-1356e48a-2260-11ee-9b33-001a4aab830c@.service. 2023-07-14T16:05:29.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:29 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1936868224' entity='client.admin' 2023-07-14T16:05:29.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:29 smithi062 systemd[1]: Starting Ceph mgr.x for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:05:30.348 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:30 smithi062 podman[95774]: 2023-07-14T16:05:30.422 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T16:05:30.422 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2023-07-14T16:05:30.438 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T16:05:30.438 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2023-07-14T16:05:30.439 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2023-07-14T16:05:30.439 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T16:05:30.457 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T16:05:30.457 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2023-07-14T16:05:30.474 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T16:05:30.474 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2023-07-14T16:05:30.474 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2023-07-14T16:05:30.474 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-14T16:05:30.475 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr to start... 2023-07-14T16:05:30.475 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr... 2023-07-14T16:05:30.624 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:30 smithi062 bash[95774]: e1ef15f63302489c0a085226a008acf73d2aad89d2cc6c61a0a639b0ac55b591 2023-07-14T16:05:30.624 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:30 smithi062 systemd[1]: Started Ceph mgr.x for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:05:30.624 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:30 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:30.561+0000 7f706f565200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T16:05:30.907 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:30 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:30.669+0000 7f706f565200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T16:05:30.908 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:30 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:30.792+0000 7f706f565200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T16:05:31.172 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:30 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:30.929+0000 7f706f565200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T16:05:31.172 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:31 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:31.166+0000 7f706f565200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T16:05:31.542 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:31 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2343909595' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T16:05:31.561 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:31.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2023-07-14T16:05:31.575 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:05:31.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T16:05:31.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T16:05:31.576 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T16:05:31.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T16:05:31.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:31.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T16:05:31.577 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T16:05:31.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2023-07-14T16:05:31.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:31.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T16:05:31.578 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T16:05:31.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:31.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-07-14T16:05:31.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T16:05:31.579 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:31.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T16:05:31.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T16:05:31.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:31.580 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T16:05:31.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:31.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T16:05:31.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T16:05:31.581 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T16:05:31.582 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T16:05:31.582 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T16:05:31.582 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T16:05:31.583 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:31.583 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T16:05:31.583 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T16:05:31.583 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T16:05:31.584 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T16:05:31.584 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T16:05:31.584 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T16:05:31.584 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T16:05:31.584 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T16:05:31.585 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T16:05:31.585 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:31.585 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T16:05:31.585 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:31.586 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T16:05:31.586 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T16:05:31.586 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:31.586 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T16:05:31.587 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T16:05:31.587 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T16:05:31.587 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T16:05:31.587 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T16:05:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T16:05:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T16:05:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T16:05:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T16:05:31.589 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:31.589 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T16:05:22.768042+0000", 2023-07-14T16:05:31.589 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T16:05:31.589 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:31.589 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T16:05:31.589 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2023-07-14T16:05:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.057+0000 7f34fe7a4700 1 Processor -- start 2023-07-14T16:05:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.057+0000 7f34fe7a4700 1 -- start start 2023-07-14T16:05:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.057+0000 7f34fe7a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.057+0000 7f34fe7a4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8079700 con 0x7f34f807acc0 2023-07-14T16:05:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.058+0000 7f34f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.058+0000 7f34f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f80791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42448/0 (socket says 172.21.15.62:42448) 2023-07-14T16:05:31.590 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.058+0000 7f34f7fff700 1 -- 172.21.15.62:0/545347973 learned_addr learned my addr 172.21.15.62:0/545347973 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:31.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.058+0000 7f34f7fff700 1 -- 172.21.15.62:0/545347973 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8079840 con 0x7f34f807acc0 2023-07-14T16:05:31.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.058+0000 7f34f7fff700 1 --2- 172.21.15.62:0/545347973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f80791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f34e80098a0 tx=0x7f34e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c633aefcbd7f2648 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:31.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.059+0000 7f34f6ffd700 1 -- 172.21.15.62:0/545347973 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34e8004160 con 0x7f34f807acc0 2023-07-14T16:05:31.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.059+0000 7f34f6ffd700 1 -- 172.21.15.62:0/545347973 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f34e80042c0 con 0x7f34f807acc0 2023-07-14T16:05:31.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.059+0000 7f34f6ffd700 1 -- 172.21.15.62:0/545347973 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34e8011940 con 0x7f34f807acc0 2023-07-14T16:05:31.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.059+0000 7f34fe7a4700 1 -- 172.21.15.62:0/545347973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 msgr2=0x7f34f80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:31.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.059+0000 7f34fe7a4700 1 --2- 172.21.15.62:0/545347973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f80791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f34e80098a0 tx=0x7f34e8009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:31.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.059+0000 7f34fe7a4700 1 -- 172.21.15.62:0/545347973 shutdown_connections 2023-07-14T16:05:31.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.059+0000 7f34fe7a4700 1 --2- 172.21.15.62:0/545347973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f80791c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:31.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.059+0000 7f34fe7a4700 1 -- 172.21.15.62:0/545347973 >> 172.21.15.62:0/545347973 conn(0x7f34f8100870 msgr2=0x7f34f8102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:31.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.060+0000 7f34fe7a4700 1 -- 172.21.15.62:0/545347973 shutdown_connections 2023-07-14T16:05:31.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.060+0000 7f34fe7a4700 1 -- 172.21.15.62:0/545347973 wait complete. 2023-07-14T16:05:31.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.061+0000 7f34fe7a4700 1 Processor -- start 2023-07-14T16:05:31.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.061+0000 7f34fe7a4700 1 -- start start 2023-07-14T16:05:31.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.061+0000 7f34fe7a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:31.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.061+0000 7f34fe7a4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f819b630 con 0x7f34f807acc0 2023-07-14T16:05:31.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:31.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42464/0 (socket says 172.21.15.62:42464) 2023-07-14T16:05:31.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34f7fff700 1 -- 172.21.15.62:0/2343909595 learned_addr learned my addr 172.21.15.62:0/2343909595 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:31.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34f7fff700 1 -- 172.21.15.62:0/2343909595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e8009550 con 0x7f34f807acc0 2023-07-14T16:05:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34f7fff700 1 --2- 172.21.15.62:0/2343909595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f819b0f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f34e8000c00 tx=0x7f34e8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34f57fa700 1 -- 172.21.15.62:0/2343909595 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34e80036a0 con 0x7f34f807acc0 2023-07-14T16:05:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34f57fa700 1 -- 172.21.15.62:0/2343909595 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f34e8003c70 con 0x7f34f807acc0 2023-07-14T16:05:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f819b830 con 0x7f34f807acc0 2023-07-14T16:05:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.062+0000 7f34f57fa700 1 -- 172.21.15.62:0/2343909595 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34e80193f0 con 0x7f34f807acc0 2023-07-14T16:05:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.063+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f819bc50 con 0x7f34f807acc0 2023-07-14T16:05:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.063+0000 7f34f57fa700 1 -- 172.21.15.62:0/2343909595 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f34e8019970 con 0x7f34f807acc0 2023-07-14T16:05:31.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.063+0000 7f34f57fa700 1 -- 172.21.15.62:0/2343909595 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f34e801a3d0 con 0x7f34f807acc0 2023-07-14T16:05:31.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.064+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f810b500 con 0x7f34f807acc0 2023-07-14T16:05:31.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.066+0000 7f34f57fa700 1 -- 172.21.15.62:0/2343909595 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f34e8029060 con 0x7f34f807acc0 2023-07-14T16:05:31.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.143+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f34f810b6f0 con 0x7f34f807acc0 2023-07-14T16:05:31.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.143+0000 7f34f57fa700 1 -- 172.21.15.62:0/2343909595 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f34e8033090 con 0x7f34f807acc0 2023-07-14T16:05:31.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.144+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 msgr2=0x7f34f819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:31.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.144+0000 7f34fe7a4700 1 --2- 172.21.15.62:0/2343909595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f819b0f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f34e8000c00 tx=0x7f34e8011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:31.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.144+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 shutdown_connections 2023-07-14T16:05:31.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.144+0000 7f34fe7a4700 1 --2- 172.21.15.62:0/2343909595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34f807acc0 0x7f34f819b0f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:31.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.144+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 >> 172.21.15.62:0/2343909595 conn(0x7f34f8100870 msgr2=0x7f34f8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:31.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.145+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 shutdown_connections 2023-07-14T16:05:31.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:31.145+0000 7f34fe7a4700 1 -- 172.21.15.62:0/2343909595 wait complete. 2023-07-14T16:05:31.596 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (1/15)... 2023-07-14T16:05:31.857 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:31 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:31.595+0000 7f706f565200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T16:05:31.857 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:31 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:31.713+0000 7f706f565200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T16:05:31.857 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:31 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:31.879+0000 7f706f565200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T16:05:32.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:32 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:32.005+0000 7f706f565200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T16:05:32.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:32 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:32.568+0000 7f706f565200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T16:05:33.624 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:33 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:33.376+0000 7f706f565200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T16:05:33.625 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:33 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:33.588+0000 7f706f565200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T16:05:34.583 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:34 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3654057076' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T16:05:34.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:34.663 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2023-07-14T16:05:34.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:05:34.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T16:05:34.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T16:05:34.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T16:05:34.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T16:05:34.664 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:34.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T16:05:34.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T16:05:34.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2023-07-14T16:05:34.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:34.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T16:05:34.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T16:05:34.665 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:34.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-07-14T16:05:34.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T16:05:34.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:34.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T16:05:34.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T16:05:34.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:34.666 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T16:05:34.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:34.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T16:05:34.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T16:05:34.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T16:05:34.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T16:05:34.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T16:05:34.667 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T16:05:34.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:34.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T16:05:34.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T16:05:34.668 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T16:05:34.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T16:05:34.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T16:05:34.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T16:05:34.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T16:05:34.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T16:05:34.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T16:05:34.669 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:34.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T16:05:34.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:34.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T16:05:34.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T16:05:34.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:34.670 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T16:05:34.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T16:05:34.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T16:05:34.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T16:05:34.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T16:05:34.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T16:05:34.671 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T16:05:34.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:34.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T16:05:34.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:34.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T16:05:34.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:34.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T16:05:22.768042+0000", 2023-07-14T16:05:34.672 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T16:05:34.673 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:34.673 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T16:05:34.673 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2023-07-14T16:05:34.673 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.155+0000 7f161af7f700 1 Processor -- start 2023-07-14T16:05:34.673 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.156+0000 7f161af7f700 1 -- start start 2023-07-14T16:05:34.674 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.156+0000 7f161af7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f1614079220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:34.674 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.156+0000 7f161af7f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1614079760 con 0x7f161407ad20 2023-07-14T16:05:34.674 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.156+0000 7f1618d1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f1614079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:34.674 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.156+0000 7f1618d1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f1614079220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54312/0 (socket says 172.21.15.62:54312) 2023-07-14T16:05:34.674 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.156+0000 7f1618d1b700 1 -- 172.21.15.62:0/613022252 learned_addr learned my addr 172.21.15.62:0/613022252 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:34.674 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.157+0000 7f1618d1b700 1 -- 172.21.15.62:0/613022252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140798a0 con 0x7f161407ad20 2023-07-14T16:05:34.674 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.157+0000 7f1618d1b700 1 --2- 172.21.15.62:0/613022252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f1614079220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16080098a0 tx=0x7f1608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0ae204c2cffcf7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:34.675 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.157+0000 7f16137fe700 1 -- 172.21.15.62:0/613022252 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1608004160 con 0x7f161407ad20 2023-07-14T16:05:34.675 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.157+0000 7f16137fe700 1 -- 172.21.15.62:0/613022252 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f16080042c0 con 0x7f161407ad20 2023-07-14T16:05:34.675 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.157+0000 7f16137fe700 1 -- 172.21.15.62:0/613022252 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1608011940 con 0x7f161407ad20 2023-07-14T16:05:34.675 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.158+0000 7f161af7f700 1 -- 172.21.15.62:0/613022252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 msgr2=0x7f1614079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:34.675 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.158+0000 7f161af7f700 1 --2- 172.21.15.62:0/613022252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f1614079220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16080098a0 tx=0x7f1608009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:34.675 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.158+0000 7f161af7f700 1 -- 172.21.15.62:0/613022252 shutdown_connections 2023-07-14T16:05:34.676 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.158+0000 7f161af7f700 1 --2- 172.21.15.62:0/613022252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f1614079220 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:34.676 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.158+0000 7f161af7f700 1 -- 172.21.15.62:0/613022252 >> 172.21.15.62:0/613022252 conn(0x7f1614100870 msgr2=0x7f1614102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:34.676 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.158+0000 7f161af7f700 1 -- 172.21.15.62:0/613022252 shutdown_connections 2023-07-14T16:05:34.676 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.158+0000 7f161af7f700 1 -- 172.21.15.62:0/613022252 wait complete. 2023-07-14T16:05:34.676 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.159+0000 7f161af7f700 1 Processor -- start 2023-07-14T16:05:34.676 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.160+0000 7f161af7f700 1 -- start start 2023-07-14T16:05:34.677 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.160+0000 7f161af7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f161419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:34.677 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.160+0000 7f161af7f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161419f9f0 con 0x7f161407ad20 2023-07-14T16:05:34.677 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.160+0000 7f1618d1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f161419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:34.677 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.160+0000 7f1618d1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f161419f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54324/0 (socket says 172.21.15.62:54324) 2023-07-14T16:05:34.677 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.160+0000 7f1618d1b700 1 -- 172.21.15.62:0/3654057076 learned_addr learned my addr 172.21.15.62:0/3654057076 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:34.677 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.160+0000 7f1618d1b700 1 -- 172.21.15.62:0/3654057076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608009550 con 0x7f161407ad20 2023-07-14T16:05:34.678 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.161+0000 7f1618d1b700 1 --2- 172.21.15.62:0/3654057076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f161419f4b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1608000c00 tx=0x7f1608011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:34.678 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.161+0000 7f1611ffb700 1 -- 172.21.15.62:0/3654057076 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16080036a0 con 0x7f161407ad20 2023-07-14T16:05:34.678 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.161+0000 7f1611ffb700 1 -- 172.21.15.62:0/3654057076 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f1608003c70 con 0x7f161407ad20 2023-07-14T16:05:34.678 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.161+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161419fbf0 con 0x7f161407ad20 2023-07-14T16:05:34.678 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.161+0000 7f1611ffb700 1 -- 172.21.15.62:0/3654057076 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16080193f0 con 0x7f161407ad20 2023-07-14T16:05:34.678 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.161+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16141a0010 con 0x7f161407ad20 2023-07-14T16:05:34.679 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.162+0000 7f1611ffb700 1 -- 172.21.15.62:0/3654057076 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1608019970 con 0x7f161407ad20 2023-07-14T16:05:34.679 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.162+0000 7f1611ffb700 1 -- 172.21.15.62:0/3654057076 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f160801a3d0 con 0x7f161407ad20 2023-07-14T16:05:34.679 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.162+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1614198d40 con 0x7f161407ad20 2023-07-14T16:05:34.679 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.165+0000 7f1611ffb700 1 -- 172.21.15.62:0/3654057076 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f1608029060 con 0x7f161407ad20 2023-07-14T16:05:34.680 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.242+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f161402c850 con 0x7f161407ad20 2023-07-14T16:05:34.680 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.243+0000 7f1611ffb700 1 -- 172.21.15.62:0/3654057076 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f1608033090 con 0x7f161407ad20 2023-07-14T16:05:34.680 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.244+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 msgr2=0x7f161419f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:34.680 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.244+0000 7f161af7f700 1 --2- 172.21.15.62:0/3654057076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f161419f4b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1608000c00 tx=0x7f1608011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:34.680 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.244+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 shutdown_connections 2023-07-14T16:05:34.680 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.244+0000 7f161af7f700 1 --2- 172.21.15.62:0/3654057076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f161407ad20 0x7f161419f4b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:34.681 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.244+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 >> 172.21.15.62:0/3654057076 conn(0x7f1614100870 msgr2=0x7f1614102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:34.681 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.244+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 shutdown_connections 2023-07-14T16:05:34.681 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:34.244+0000 7f161af7f700 1 -- 172.21.15.62:0/3654057076 wait complete. 2023-07-14T16:05:34.681 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (2/15)... 2023-07-14T16:05:34.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:34 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:34.605+0000 7f706f565200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:05:34.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:34 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:34.731+0000 7f706f565200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T16:05:35.307 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:34 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:34.958+0000 7f706f565200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:05:35.587 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:35 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:35.329+0000 7f706f565200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T16:05:35.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:35 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:35.609+0000 7f706f565200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T16:05:35.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:35 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:35.806+0000 7f706f565200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T16:05:36.624 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:36 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:36.156+0000 7f706f565200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T16:05:37.292 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:37 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:37.131+0000 7f706f565200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T16:05:37.292 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:37 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:37.314+0000 7f706f565200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T16:05:37.550 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:37 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3256415420' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T16:05:37.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:37.743 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2023-07-14T16:05:37.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:05:37.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T16:05:37.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T16:05:37.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T16:05:37.744 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T16:05:37.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:37.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T16:05:37.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T16:05:37.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2023-07-14T16:05:37.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:37.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T16:05:37.746 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T16:05:37.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:37.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-07-14T16:05:37.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T16:05:37.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:37.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T16:05:37.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T16:05:37.747 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:37.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T16:05:37.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:37.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T16:05:37.748 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T16:05:37.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T16:05:37.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T16:05:37.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T16:05:37.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T16:05:37.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:37.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T16:05:37.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T16:05:37.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T16:05:37.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T16:05:37.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T16:05:37.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T16:05:37.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T16:05:37.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T16:05:37.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T16:05:37.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:37.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T16:05:37.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:37.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T16:05:37.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T16:05:37.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:37.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T16:05:37.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T16:05:37.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T16:05:37.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T16:05:37.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T16:05:37.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T16:05:37.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T16:05:37.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:37.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T16:05:37.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:37.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T16:05:37.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:37.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T16:05:22.768042+0000", 2023-07-14T16:05:37.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T16:05:37.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:37.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T16:05:37.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2023-07-14T16:05:37.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.206+0000 7f4fc3b30700 1 Processor -- start 2023-07-14T16:05:37.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.207+0000 7f4fc3b30700 1 -- start start 2023-07-14T16:05:37.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.207+0000 7f4fc3b30700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:37.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.207+0000 7f4fc3b30700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc079700 con 0x7f4fbc07acc0 2023-07-14T16:05:37.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.207+0000 7f4fc18cc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:37.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.208+0000 7f4fc18cc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54328/0 (socket says 172.21.15.62:54328) 2023-07-14T16:05:37.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.208+0000 7f4fc18cc700 1 -- 172.21.15.62:0/756156806 learned_addr learned my addr 172.21.15.62:0/756156806 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:37.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.208+0000 7f4fc18cc700 1 -- 172.21.15.62:0/756156806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc079840 con 0x7f4fbc07acc0 2023-07-14T16:05:37.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.208+0000 7f4fc18cc700 1 --2- 172.21.15.62:0/756156806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc0791c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098a0 tx=0x7f4fb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0467d8386dc6b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:37.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.208+0000 7f4fc08ca700 1 -- 172.21.15.62:0/756156806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fb8004160 con 0x7f4fbc07acc0 2023-07-14T16:05:37.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.208+0000 7f4fc08ca700 1 -- 172.21.15.62:0/756156806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4fb80042c0 con 0x7f4fbc07acc0 2023-07-14T16:05:37.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.209+0000 7f4fc08ca700 1 -- 172.21.15.62:0/756156806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fb8011940 con 0x7f4fbc07acc0 2023-07-14T16:05:37.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.209+0000 7f4fc3b30700 1 -- 172.21.15.62:0/756156806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 msgr2=0x7f4fbc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:37.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.209+0000 7f4fc3b30700 1 --2- 172.21.15.62:0/756156806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc0791c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098a0 tx=0x7f4fb8009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:37.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.209+0000 7f4fc3b30700 1 -- 172.21.15.62:0/756156806 shutdown_connections 2023-07-14T16:05:37.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.209+0000 7f4fc3b30700 1 --2- 172.21.15.62:0/756156806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc0791c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:37.760 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.209+0000 7f4fc3b30700 1 -- 172.21.15.62:0/756156806 >> 172.21.15.62:0/756156806 conn(0x7f4fbc100870 msgr2=0x7f4fbc102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:37.760 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.209+0000 7f4fc3b30700 1 -- 172.21.15.62:0/756156806 shutdown_connections 2023-07-14T16:05:37.760 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.209+0000 7f4fc3b30700 1 -- 172.21.15.62:0/756156806 wait complete. 2023-07-14T16:05:37.760 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.210+0000 7f4fc3b30700 1 Processor -- start 2023-07-14T16:05:37.760 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.211+0000 7f4fc3b30700 1 -- start start 2023-07-14T16:05:37.760 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.211+0000 7f4fc3b30700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:37.760 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.211+0000 7f4fc3b30700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc19b630 con 0x7f4fbc07acc0 2023-07-14T16:05:37.761 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.211+0000 7f4fc18cc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:37.761 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.211+0000 7f4fc18cc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc19b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54334/0 (socket says 172.21.15.62:54334) 2023-07-14T16:05:37.761 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.211+0000 7f4fc18cc700 1 -- 172.21.15.62:0/3256415420 learned_addr learned my addr 172.21.15.62:0/3256415420 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:37.761 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.212+0000 7f4fc18cc700 1 -- 172.21.15.62:0/3256415420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8009550 con 0x7f4fbc07acc0 2023-07-14T16:05:37.761 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.212+0000 7f4fc18cc700 1 --2- 172.21.15.62:0/3256415420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc19b0f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000c00 tx=0x7f4fb8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:37.761 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.212+0000 7f4fb2ffd700 1 -- 172.21.15.62:0/3256415420 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fb80036a0 con 0x7f4fbc07acc0 2023-07-14T16:05:37.761 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.212+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc19b830 con 0x7f4fbc07acc0 2023-07-14T16:05:37.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.212+0000 7f4fb2ffd700 1 -- 172.21.15.62:0/3256415420 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4fb8003c70 con 0x7f4fbc07acc0 2023-07-14T16:05:37.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.212+0000 7f4fb2ffd700 1 -- 172.21.15.62:0/3256415420 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fb80193f0 con 0x7f4fbc07acc0 2023-07-14T16:05:37.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.212+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc19bc50 con 0x7f4fbc07acc0 2023-07-14T16:05:37.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.213+0000 7f4fb2ffd700 1 -- 172.21.15.62:0/3256415420 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4fb8019970 con 0x7f4fbc07acc0 2023-07-14T16:05:37.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.213+0000 7f4fb2ffd700 1 -- 172.21.15.62:0/3256415420 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4fb801a3d0 con 0x7f4fbc07acc0 2023-07-14T16:05:37.763 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.213+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc061b40 con 0x7f4fbc07acc0 2023-07-14T16:05:37.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.216+0000 7f4fb2ffd700 1 -- 172.21.15.62:0/3256415420 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f4fb8029060 con 0x7f4fbc07acc0 2023-07-14T16:05:37.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.293+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4fbc19e690 con 0x7f4fbc07acc0 2023-07-14T16:05:37.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.293+0000 7f4fb2ffd700 1 -- 172.21.15.62:0/3256415420 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f4fb8033090 con 0x7f4fbc07acc0 2023-07-14T16:05:37.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.294+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 msgr2=0x7f4fbc19b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:37.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.294+0000 7f4fc3b30700 1 --2- 172.21.15.62:0/3256415420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc19b0f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000c00 tx=0x7f4fb8011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:37.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.294+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 shutdown_connections 2023-07-14T16:05:37.764 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.294+0000 7f4fc3b30700 1 --2- 172.21.15.62:0/3256415420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc07acc0 0x7f4fbc19b0f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:37.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.294+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 >> 172.21.15.62:0/3256415420 conn(0x7f4fbc100870 msgr2=0x7f4fbc101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:37.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.294+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 shutdown_connections 2023-07-14T16:05:37.765 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:37.294+0000 7f4fc3b30700 1 -- 172.21.15.62:0/3256415420 wait complete. 2023-07-14T16:05:37.765 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (3/15)... 2023-07-14T16:05:37.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:37 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:37.572+0000 7f706f565200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T16:05:37.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:37 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:37.725+0000 7f706f565200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T16:05:38.374 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:38 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:38.015+0000 7f706f565200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T16:05:39.374 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:39.112+0000 7f706f565200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T16:05:39.378 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: Activating manager daemon x 2023-07-14T16:05:39.378 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: mgrmap e2: x(active, starting, since 0.00422625s) 2023-07-14T16:05:39.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T16:05:39.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T16:05:39.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T16:05:39.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:05:39.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T16:05:39.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: Manager daemon x is now available 2023-07-14T16:05:39.380 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T16:05:39.380 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' 2023-07-14T16:05:39.380 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T16:05:39.380 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' 2023-07-14T16:05:39.380 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:39 smithi062 ceph-mon[95543]: from='mgr.14100 172.21.15.62:0/269081064' entity='mgr.x' 2023-07-14T16:05:41.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:41.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2023-07-14T16:05:41.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:05:41.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T16:05:41.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T16:05:41.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T16:05:41.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T16:05:41.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:41.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T16:05:41.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T16:05:41.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2023-07-14T16:05:41.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:41.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T16:05:41.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T16:05:41.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:41.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-07-14T16:05:41.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T16:05:41.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:41.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T16:05:41.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T16:05:41.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:41.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T16:05:41.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:41.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T16:05:41.052 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T16:05:41.053 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T16:05:41.053 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T16:05:41.053 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T16:05:41.053 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T16:05:41.053 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:41.053 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T16:05:41.053 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T16:05:41.054 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T16:05:41.054 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T16:05:41.054 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T16:05:41.054 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T16:05:41.054 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T16:05:41.054 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T16:05:41.054 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T16:05:41.055 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:41.055 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T16:05:41.055 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:41.055 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T16:05:41.055 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T16:05:41.055 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:41.055 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T16:05:41.055 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T16:05:41.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T16:05:41.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T16:05:41.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T16:05:41.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T16:05:41.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T16:05:41.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2023-07-14T16:05:41.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T16:05:41.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:41.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T16:05:41.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T16:05:41.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T16:05:22.768042+0000", 2023-07-14T16:05:41.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T16:05:41.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2023-07-14T16:05:41.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T16:05:41.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2023-07-14T16:05:41.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.304+0000 7f9130594700 1 Processor -- start 2023-07-14T16:05:41.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.305+0000 7f9130594700 1 -- start start 2023-07-14T16:05:41.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.305+0000 7f9130594700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:41.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.305+0000 7f9130594700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9128105b50 con 0x7f9128105200 2023-07-14T16:05:41.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.305+0000 7f912e330700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:41.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.305+0000 7f912e330700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54404/0 (socket says 172.21.15.62:54404) 2023-07-14T16:05:41.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.305+0000 7f912e330700 1 -- 172.21.15.62:0/2628417220 learned_addr learned my addr 172.21.15.62:0/2628417220 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:41.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.306+0000 7f912e330700 1 -- 172.21.15.62:0/2628417220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128105c90 con 0x7f9128105200 2023-07-14T16:05:41.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.306+0000 7f912e330700 1 --2- 172.21.15.62:0/2628417220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128105610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f91240098a0 tx=0x7f9124009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1b17bbb2fc7afd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:41.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.306+0000 7f912d32e700 1 -- 172.21.15.62:0/2628417220 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9124004160 con 0x7f9128105200 2023-07-14T16:05:41.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.306+0000 7f912d32e700 1 -- 172.21.15.62:0/2628417220 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f91240042c0 con 0x7f9128105200 2023-07-14T16:05:41.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.306+0000 7f912d32e700 1 -- 172.21.15.62:0/2628417220 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9124011940 con 0x7f9128105200 2023-07-14T16:05:41.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.307+0000 7f9130594700 1 -- 172.21.15.62:0/2628417220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 msgr2=0x7f9128105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:41.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.307+0000 7f9130594700 1 --2- 172.21.15.62:0/2628417220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128105610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f91240098a0 tx=0x7f9124009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:41.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.307+0000 7f9130594700 1 -- 172.21.15.62:0/2628417220 shutdown_connections 2023-07-14T16:05:41.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.307+0000 7f9130594700 1 --2- 172.21.15.62:0/2628417220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128105610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:41.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.307+0000 7f9130594700 1 -- 172.21.15.62:0/2628417220 >> 172.21.15.62:0/2628417220 conn(0x7f9128100870 msgr2=0x7f9128102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:41.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.307+0000 7f9130594700 1 -- 172.21.15.62:0/2628417220 shutdown_connections 2023-07-14T16:05:41.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.307+0000 7f9130594700 1 -- 172.21.15.62:0/2628417220 wait complete. 2023-07-14T16:05:41.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.308+0000 7f9130594700 1 Processor -- start 2023-07-14T16:05:41.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.309+0000 7f9130594700 1 -- start start 2023-07-14T16:05:41.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.309+0000 7f9130594700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:41.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.309+0000 7f9130594700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91281993b0 con 0x7f9128105200 2023-07-14T16:05:41.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.309+0000 7f912e330700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:41.061 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.309+0000 7f912e330700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54414/0 (socket says 172.21.15.62:54414) 2023-07-14T16:05:41.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.309+0000 7f912e330700 1 -- 172.21.15.62:0/159912925 learned_addr learned my addr 172.21.15.62:0/159912925 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:41.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.310+0000 7f912e330700 1 -- 172.21.15.62:0/159912925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124009550 con 0x7f9128105200 2023-07-14T16:05:41.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.310+0000 7f912e330700 1 --2- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128198e70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9124006af0 tx=0x7f9124011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:41.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.310+0000 7f911f7fe700 1 -- 172.21.15.62:0/159912925 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9124003bf0 con 0x7f9128105200 2023-07-14T16:05:41.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.310+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91281995b0 con 0x7f9128105200 2023-07-14T16:05:41.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.310+0000 7f911f7fe700 1 -- 172.21.15.62:0/159912925 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f912401b8b0 con 0x7f9128105200 2023-07-14T16:05:41.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.310+0000 7f911f7fe700 1 -- 172.21.15.62:0/159912925 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91240197b0 con 0x7f9128105200 2023-07-14T16:05:41.062 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.310+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91281999d0 con 0x7f9128105200 2023-07-14T16:05:41.063 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.311+0000 7f911f7fe700 1 -- 172.21.15.62:0/159912925 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f912401b440 con 0x7f9128105200 2023-07-14T16:05:41.063 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.311+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9128061b40 con 0x7f9128105200 2023-07-14T16:05:41.063 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.312+0000 7f911f7fe700 1 --2- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f9114038550 0x7f911403aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:41.063 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.312+0000 7f911f7fe700 1 -- 172.21.15.62:0/159912925 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f912404c7a0 con 0x7f9128105200 2023-07-14T16:05:41.063 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.312+0000 7f912db2f700 1 --2- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f9114038550 0x7f911403aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:41.063 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.313+0000 7f912db2f700 1 --2- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f9114038550 0x7f911403aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9118006fd0 tx=0x7f9118006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:41.063 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.318+0000 7f911f7fe700 1 -- 172.21.15.62:0/159912925 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f912401ba20 con 0x7f9128105200 2023-07-14T16:05:41.063 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.605+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9128107060 con 0x7f9128105200 2023-07-14T16:05:41.064 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.606+0000 7f911f7fe700 1 -- 172.21.15.62:0/159912925 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f912402bb40 con 0x7f9128105200 2023-07-14T16:05:41.064 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.608+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f9114038550 msgr2=0x7f911403aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:41.064 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.608+0000 7f9130594700 1 --2- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f9114038550 0x7f911403aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9118006fd0 tx=0x7f9118006e40 comp rx=0 tx=0).stop 2023-07-14T16:05:41.064 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.608+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 msgr2=0x7f9128198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:41.064 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.608+0000 7f9130594700 1 --2- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128198e70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9124006af0 tx=0x7f9124011a30 comp rx=0 tx=0).stop 2023-07-14T16:05:41.064 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.608+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 shutdown_connections 2023-07-14T16:05:41.064 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.608+0000 7f9130594700 1 --2- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f9114038550 0x7f911403aa00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:41.065 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.608+0000 7f9130594700 1 --2- 172.21.15.62:0/159912925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9128105200 0x7f9128198e70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:41.065 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.608+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 >> 172.21.15.62:0/159912925 conn(0x7f9128100870 msgr2=0x7f91281015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:41.065 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.609+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 shutdown_connections 2023-07-14T16:05:41.065 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:40.609+0000 7f9130594700 1 -- 172.21.15.62:0/159912925 wait complete. 2023-07-14T16:05:41.065 INFO:teuthology.orchestra.run.smithi062.stdout:mgr is available 2023-07-14T16:05:41.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:41 smithi062 ceph-mon[95543]: mgrmap e3: x(active, since 1.00994s) 2023-07-14T16:05:41.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:41 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/159912925' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T16:05:42.226 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:42.226 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T16:05:42.226 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:42.226 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T16:05:42.226 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T16:05:42.227 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T16:05:42.227 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T16:05:42.227 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:42.227 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T16:05:42.227 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T16:05:42.227 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2023-07-14T16:05:42.227 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T16:05:42.227 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2023-07-14T16:05:42.228 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T16:05:42.228 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T16:05:42.232 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T16:05:42.232 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T16:05:42.232 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T16:05:42.232 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.631+0000 7fd257a82700 1 Processor -- start 2023-07-14T16:05:42.232 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.631+0000 7fd257a82700 1 -- start start 2023-07-14T16:05:42.232 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.631+0000 7fd257a82700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd2500791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:42.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.631+0000 7fd257a82700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250079700 con 0x7fd25007acc0 2023-07-14T16:05:42.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.631+0000 7fd25581e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd2500791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:42.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.632+0000 7fd25581e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd2500791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54420/0 (socket says 172.21.15.62:54420) 2023-07-14T16:05:42.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.632+0000 7fd25581e700 1 -- 172.21.15.62:0/475254870 learned_addr learned my addr 172.21.15.62:0/475254870 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:42.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.632+0000 7fd25581e700 1 -- 172.21.15.62:0/475254870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250079840 con 0x7fd25007acc0 2023-07-14T16:05:42.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.632+0000 7fd25581e700 1 --2- 172.21.15.62:0/475254870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd2500791c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0098a0 tx=0x7fd24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fa5d29ce0de0e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:42.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.632+0000 7fd25481c700 1 -- 172.21.15.62:0/475254870 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd24c004160 con 0x7fd25007acc0 2023-07-14T16:05:42.233 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.632+0000 7fd25481c700 1 -- 172.21.15.62:0/475254870 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd24c0042c0 con 0x7fd25007acc0 2023-07-14T16:05:42.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.632+0000 7fd25481c700 1 -- 172.21.15.62:0/475254870 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd24c011940 con 0x7fd25007acc0 2023-07-14T16:05:42.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.633+0000 7fd257a82700 1 -- 172.21.15.62:0/475254870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 msgr2=0x7fd2500791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:42.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.633+0000 7fd257a82700 1 --2- 172.21.15.62:0/475254870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd2500791c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0098a0 tx=0x7fd24c009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:42.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.633+0000 7fd257a82700 1 -- 172.21.15.62:0/475254870 shutdown_connections 2023-07-14T16:05:42.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.633+0000 7fd257a82700 1 --2- 172.21.15.62:0/475254870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd2500791c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:42.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.633+0000 7fd257a82700 1 -- 172.21.15.62:0/475254870 >> 172.21.15.62:0/475254870 conn(0x7fd250100870 msgr2=0x7fd250102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:42.234 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.633+0000 7fd257a82700 1 -- 172.21.15.62:0/475254870 shutdown_connections 2023-07-14T16:05:42.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.633+0000 7fd257a82700 1 -- 172.21.15.62:0/475254870 wait complete. 2023-07-14T16:05:42.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.634+0000 7fd257a82700 1 Processor -- start 2023-07-14T16:05:42.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.634+0000 7fd257a82700 1 -- start start 2023-07-14T16:05:42.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.635+0000 7fd257a82700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd25019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:42.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.635+0000 7fd257a82700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019b630 con 0x7fd25007acc0 2023-07-14T16:05:42.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.635+0000 7fd25581e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd25019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:42.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.635+0000 7fd25581e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd25019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54430/0 (socket says 172.21.15.62:54430) 2023-07-14T16:05:42.235 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.635+0000 7fd25581e700 1 -- 172.21.15.62:0/3944989319 learned_addr learned my addr 172.21.15.62:0/3944989319 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:42.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.635+0000 7fd25581e700 1 -- 172.21.15.62:0/3944989319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c009550 con 0x7fd25007acc0 2023-07-14T16:05:42.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.636+0000 7fd25581e700 1 --2- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd25019b0f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd24c000c00 tx=0x7fd24c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:42.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.636+0000 7fd246ffd700 1 -- 172.21.15.62:0/3944989319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd24c0036a0 con 0x7fd25007acc0 2023-07-14T16:05:42.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.636+0000 7fd246ffd700 1 -- 172.21.15.62:0/3944989319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd24c003c70 con 0x7fd25007acc0 2023-07-14T16:05:42.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.636+0000 7fd246ffd700 1 -- 172.21.15.62:0/3944989319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd24c0193f0 con 0x7fd25007acc0 2023-07-14T16:05:42.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.636+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25019b830 con 0x7fd25007acc0 2023-07-14T16:05:42.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.636+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25019bc50 con 0x7fd25007acc0 2023-07-14T16:05:42.236 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.637+0000 7fd246ffd700 1 -- 172.21.15.62:0/3944989319 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fd24c01b440 con 0x7fd25007acc0 2023-07-14T16:05:42.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.637+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25010b500 con 0x7fd25007acc0 2023-07-14T16:05:42.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.638+0000 7fd246ffd700 1 --2- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fd23c038710 0x7fd23c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:42.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.638+0000 7fd246ffd700 1 -- 172.21.15.62:0/3944989319 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd24c04c470 con 0x7fd25007acc0 2023-07-14T16:05:42.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.638+0000 7fd25501d700 1 --2- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fd23c038710 0x7fd23c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:42.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.639+0000 7fd25501d700 1 --2- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fd23c038710 0x7fd23c03abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd240006fd0 tx=0x7fd240006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:42.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.643+0000 7fd246ffd700 1 -- 172.21.15.62:0/3944989319 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd24c022930 con 0x7fd25007acc0 2023-07-14T16:05:42.237 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.846+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd25010b6f0 con 0x7fd25007acc0 2023-07-14T16:05:42.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.847+0000 7fd246ffd700 1 -- 172.21.15.62:0/3944989319 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+398 (secure 0 0 0) 0x7fd24c01f070 con 0x7fd25007acc0 2023-07-14T16:05:42.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.849+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fd23c038710 msgr2=0x7fd23c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:42.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.849+0000 7fd257a82700 1 --2- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fd23c038710 0x7fd23c03abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd240006fd0 tx=0x7fd240006e40 comp rx=0 tx=0).stop 2023-07-14T16:05:42.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.849+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 msgr2=0x7fd25019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:42.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.849+0000 7fd257a82700 1 --2- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd25019b0f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd24c000c00 tx=0x7fd24c011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:42.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.849+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 shutdown_connections 2023-07-14T16:05:42.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.849+0000 7fd257a82700 1 --2- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fd23c038710 0x7fd23c03abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:42.238 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.849+0000 7fd257a82700 1 --2- 172.21.15.62:0/3944989319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25007acc0 0x7fd25019b0f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:42.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.849+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 >> 172.21.15.62:0/3944989319 conn(0x7fd250100870 msgr2=0x7fd250101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:42.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.850+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 shutdown_connections 2023-07-14T16:05:42.239 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:41.850+0000 7fd257a82700 1 -- 172.21.15.62:0/3944989319 wait complete. 2023-07-14T16:05:42.239 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling cephadm module... 2023-07-14T16:05:42.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:42 smithi062 ceph-mon[95543]: mgrmap e4: x(active, since 2s) 2023-07-14T16:05:42.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:42 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3944989319' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T16:05:44.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:43 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3836931647' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-14T16:05:44.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:43 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: ignoring --setuser ceph since I am not root 2023-07-14T16:05:44.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:43 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: ignoring --setgroup ceph since I am not root 2023-07-14T16:05:44.316 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.816+0000 7fa868d1d700 1 Processor -- start 2023-07-14T16:05:44.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.816+0000 7fa868d1d700 1 -- start start 2023-07-14T16:05:44.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.816+0000 7fa868d1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa8640791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:44.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.816+0000 7fa868d1d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa864079720 con 0x7fa86407ace0 2023-07-14T16:05:44.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.816+0000 7fa86259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa8640791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:44.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.816+0000 7fa86259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa8640791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56670/0 (socket says 172.21.15.62:56670) 2023-07-14T16:05:44.317 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.816+0000 7fa86259c700 1 -- 172.21.15.62:0/3890156066 learned_addr learned my addr 172.21.15.62:0/3890156066 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:44.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.817+0000 7fa86259c700 1 -- 172.21.15.62:0/3890156066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864079860 con 0x7fa86407ace0 2023-07-14T16:05:44.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.817+0000 7fa86259c700 1 --2- 172.21.15.62:0/3890156066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa8640791e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098a0 tx=0x7fa854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd8ed1b760b67d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:44.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.817+0000 7fa86159a700 1 -- 172.21.15.62:0/3890156066 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa854004160 con 0x7fa86407ace0 2023-07-14T16:05:44.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.817+0000 7fa86159a700 1 -- 172.21.15.62:0/3890156066 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa8540042c0 con 0x7fa86407ace0 2023-07-14T16:05:44.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.817+0000 7fa86159a700 1 -- 172.21.15.62:0/3890156066 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa854011940 con 0x7fa86407ace0 2023-07-14T16:05:44.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.818+0000 7fa868d1d700 1 -- 172.21.15.62:0/3890156066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 msgr2=0x7fa8640791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:44.318 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.818+0000 7fa868d1d700 1 --2- 172.21.15.62:0/3890156066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa8640791e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098a0 tx=0x7fa854009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:44.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.818+0000 7fa868d1d700 1 -- 172.21.15.62:0/3890156066 shutdown_connections 2023-07-14T16:05:44.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.818+0000 7fa868d1d700 1 --2- 172.21.15.62:0/3890156066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa8640791e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:44.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.818+0000 7fa868d1d700 1 -- 172.21.15.62:0/3890156066 >> 172.21.15.62:0/3890156066 conn(0x7fa864100830 msgr2=0x7fa864102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:44.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.818+0000 7fa868d1d700 1 -- 172.21.15.62:0/3890156066 shutdown_connections 2023-07-14T16:05:44.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.818+0000 7fa868d1d700 1 -- 172.21.15.62:0/3890156066 wait complete. 2023-07-14T16:05:44.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.819+0000 7fa868d1d700 1 Processor -- start 2023-07-14T16:05:44.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.819+0000 7fa868d1d700 1 -- start start 2023-07-14T16:05:44.319 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.819+0000 7fa868d1d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa86419b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:44.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.819+0000 7fa868d1d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86419b580 con 0x7fa86407ace0 2023-07-14T16:05:44.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.820+0000 7fa86259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa86419b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:44.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.820+0000 7fa86259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa86419b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56680/0 (socket says 172.21.15.62:56680) 2023-07-14T16:05:44.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.820+0000 7fa86259c700 1 -- 172.21.15.62:0/3836931647 learned_addr learned my addr 172.21.15.62:0/3836931647 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:44.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.820+0000 7fa86259c700 1 -- 172.21.15.62:0/3836931647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa854009550 con 0x7fa86407ace0 2023-07-14T16:05:44.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.820+0000 7fa86259c700 1 --2- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa86419b040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa854000c00 tx=0x7fa854004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:44.320 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.820+0000 7fa8537fe700 1 -- 172.21.15.62:0/3836931647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8540036a0 con 0x7fa86407ace0 2023-07-14T16:05:44.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.820+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86419b780 con 0x7fa86407ace0 2023-07-14T16:05:44.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.820+0000 7fa8537fe700 1 -- 172.21.15.62:0/3836931647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa854003c70 con 0x7fa86407ace0 2023-07-14T16:05:44.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.821+0000 7fa8537fe700 1 -- 172.21.15.62:0/3836931647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa854024d70 con 0x7fa86407ace0 2023-07-14T16:05:44.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.821+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86419bc20 con 0x7fa86407ace0 2023-07-14T16:05:44.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.822+0000 7fa8537fe700 1 -- 172.21.15.62:0/3836931647 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fa8540245d0 con 0x7fa86407ace0 2023-07-14T16:05:44.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.822+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86410b4a0 con 0x7fa86407ace0 2023-07-14T16:05:44.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.822+0000 7fa8537fe700 1 --2- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fa84c0386c0 0x7fa84c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:44.321 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.822+0000 7fa8537fe700 1 -- 172.21.15.62:0/3836931647 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa85404c3d0 con 0x7fa86407ace0 2023-07-14T16:05:44.322 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.822+0000 7fa861d9b700 1 --2- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fa84c0386c0 0x7fa84c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:44.322 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.823+0000 7fa861d9b700 1 --2- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fa84c0386c0 0x7fa84c03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa858006fd0 tx=0x7fa858006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:44.322 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:42.827+0000 7fa8537fe700 1 -- 172.21.15.62:0/3836931647 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa854018b80 con 0x7fa86407ace0 2023-07-14T16:05:44.322 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.092+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa86410b6c0 con 0x7fa86407ace0 2023-07-14T16:05:44.322 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.871+0000 7fa8537fe700 1 -- 172.21.15.62:0/3836931647 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fa85402d910 con 0x7fa86407ace0 2023-07-14T16:05:44.322 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.873+0000 7fa8537fe700 1 -- 172.21.15.62:0/3836931647 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa85404bdf0 con 0x7fa86407ace0 2023-07-14T16:05:44.322 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fa84c0386c0 msgr2=0x7fa84c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:44.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 --2- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fa84c0386c0 0x7fa84c03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa858006fd0 tx=0x7fa858006e40 comp rx=0 tx=0).stop 2023-07-14T16:05:44.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 msgr2=0x7fa86419b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:44.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 --2- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa86419b040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa854000c00 tx=0x7fa854004250 comp rx=0 tx=0).stop 2023-07-14T16:05:44.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 shutdown_connections 2023-07-14T16:05:44.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 --2- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fa84c0386c0 0x7fa84c03ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:44.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 --2- 172.21.15.62:0/3836931647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa86407ace0 0x7fa86419b040 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:44.323 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 >> 172.21.15.62:0/3836931647 conn(0x7fa864100830 msgr2=0x7fa864101590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:44.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.876+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 shutdown_connections 2023-07-14T16:05:44.324 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:43.877+0000 7fa868d1d700 1 -- 172.21.15.62:0/3836931647 wait complete. 2023-07-14T16:05:44.488 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:44 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:44.152+0000 7ff327ba9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T16:05:44.489 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:44 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:44.261+0000 7ff327ba9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T16:05:44.489 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:44 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:44.384+0000 7ff327ba9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T16:05:44.740 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:44 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:44.511+0000 7ff327ba9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T16:05:45.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:44 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3836931647' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-14T16:05:45.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:44 smithi062 ceph-mon[95543]: mgrmap e5: x(active, since 4s) 2023-07-14T16:05:45.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:44 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:44.762+0000 7ff327ba9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T16:05:45.475 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:45 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:45.178+0000 7ff327ba9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T16:05:45.476 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:45 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:45.331+0000 7ff327ba9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T16:05:45.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2023-07-14T16:05:45.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-14T16:05:45.601 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T16:05:45.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T16:05:45.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T16:05:45.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2023-07-14T16:05:45.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.883+0000 7fe6010eb700 1 Processor -- start 2023-07-14T16:05:45.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.883+0000 7fe6010eb700 1 -- start start 2023-07-14T16:05:45.602 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.883+0000 7fe6010eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:45.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.883+0000 7fe6010eb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc079740 con 0x7fe5fc07ad00 2023-07-14T16:05:45.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.883+0000 7fe5fad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:45.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.884+0000 7fe5fad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56708/0 (socket says 172.21.15.62:56708) 2023-07-14T16:05:45.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.884+0000 7fe5fad9d700 1 -- 172.21.15.62:0/2557955495 learned_addr learned my addr 172.21.15.62:0/2557955495 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:45.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.884+0000 7fe5fad9d700 1 -- 172.21.15.62:0/2557955495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc079880 con 0x7fe5fc07ad00 2023-07-14T16:05:45.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.884+0000 7fe5fad9d700 1 --2- 172.21.15.62:0/2557955495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc079200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0098a0 tx=0x7fe5ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f828c9d89c3209 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:45.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.884+0000 7fe5f9d9b700 1 -- 172.21.15.62:0/2557955495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5ec004160 con 0x7fe5fc07ad00 2023-07-14T16:05:45.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.884+0000 7fe5f9d9b700 1 -- 172.21.15.62:0/2557955495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe5ec0042c0 con 0x7fe5fc07ad00 2023-07-14T16:05:45.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.884+0000 7fe5f9d9b700 1 -- 172.21.15.62:0/2557955495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5ec011940 con 0x7fe5fc07ad00 2023-07-14T16:05:45.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.885+0000 7fe6010eb700 1 -- 172.21.15.62:0/2557955495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 msgr2=0x7fe5fc079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:45.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.885+0000 7fe6010eb700 1 --2- 172.21.15.62:0/2557955495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc079200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0098a0 tx=0x7fe5ec009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:45.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.885+0000 7fe6010eb700 1 -- 172.21.15.62:0/2557955495 shutdown_connections 2023-07-14T16:05:45.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.885+0000 7fe6010eb700 1 --2- 172.21.15.62:0/2557955495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc079200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:45.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.885+0000 7fe6010eb700 1 -- 172.21.15.62:0/2557955495 >> 172.21.15.62:0/2557955495 conn(0x7fe5fc100870 msgr2=0x7fe5fc102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:45.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.885+0000 7fe6010eb700 1 -- 172.21.15.62:0/2557955495 shutdown_connections 2023-07-14T16:05:45.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.885+0000 7fe6010eb700 1 -- 172.21.15.62:0/2557955495 wait complete. 2023-07-14T16:05:45.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.887+0000 7fe6010eb700 1 Processor -- start 2023-07-14T16:05:45.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.887+0000 7fe6010eb700 1 -- start start 2023-07-14T16:05:45.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.887+0000 7fe6010eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:45.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.887+0000 7fe6010eb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc19b620 con 0x7fe5fc07ad00 2023-07-14T16:05:45.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.888+0000 7fe5fad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:45.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.888+0000 7fe5fad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc19b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56714/0 (socket says 172.21.15.62:56714) 2023-07-14T16:05:45.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.888+0000 7fe5fad9d700 1 -- 172.21.15.62:0/1488667318 learned_addr learned my addr 172.21.15.62:0/1488667318 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:45.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.888+0000 7fe5fad9d700 1 -- 172.21.15.62:0/1488667318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec009550 con 0x7fe5fc07ad00 2023-07-14T16:05:45.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.888+0000 7fe5fad9d700 1 --2- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc19b0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec000c00 tx=0x7fe5ec011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:45.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.888+0000 7fe5e3fff700 1 -- 172.21.15.62:0/1488667318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5ec0036a0 con 0x7fe5fc07ad00 2023-07-14T16:05:45.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.889+0000 7fe5e3fff700 1 -- 172.21.15.62:0/1488667318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe5ec003c70 con 0x7fe5fc07ad00 2023-07-14T16:05:45.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.889+0000 7fe5e3fff700 1 -- 172.21.15.62:0/1488667318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5ec0193f0 con 0x7fe5fc07ad00 2023-07-14T16:05:45.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.889+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc19b820 con 0x7fe5fc07ad00 2023-07-14T16:05:45.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.889+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc19bc40 con 0x7fe5fc07ad00 2023-07-14T16:05:45.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.890+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc061b40 con 0x7fe5fc07ad00 2023-07-14T16:05:45.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.890+0000 7fe5e3fff700 1 -- 172.21.15.62:0/1488667318 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe5ec01b440 con 0x7fe5fc07ad00 2023-07-14T16:05:45.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.890+0000 7fe5e3fff700 1 --2- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fe5e40387b0 0x7fe5e403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:45.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.891+0000 7fe5e3fff700 1 -- 172.21.15.62:0/1488667318 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe5ec04c500 con 0x7fe5fc07ad00 2023-07-14T16:05:45.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.891+0000 7fe5fa59c700 1 -- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fe5e40387b0 msgr2=0x7fe5e403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1177875314 2023-07-14T16:05:45.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.891+0000 7fe5fa59c700 1 --2- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fe5e40387b0 0x7fe5e403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T16:05:45.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:44.895+0000 7fe5e3fff700 1 -- 172.21.15.62:0/1488667318 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe5ec022950 con 0x7fe5fc07ad00 2023-07-14T16:05:45.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.092+0000 7fe5fa59c700 1 -- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fe5e40387b0 msgr2=0x7fe5e403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1177875314 2023-07-14T16:05:45.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.092+0000 7fe5fa59c700 1 --2- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fe5e40387b0 0x7fe5e403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T16:05:45.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.156+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe5fc19edc0 con 0x7fe5fc07ad00 2023-07-14T16:05:45.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.157+0000 7fe5e3fff700 1 -- 172.21.15.62:0/1488667318 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe5ec01f020 con 0x7fe5fc07ad00 2023-07-14T16:05:45.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fe5e40387b0 msgr2=0x7fe5e403ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:05:45.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 --2- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fe5e40387b0 0x7fe5e403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:45.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 msgr2=0x7fe5fc19b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:45.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 --2- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc19b0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec000c00 tx=0x7fe5ec011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:45.609 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 shutdown_connections 2023-07-14T16:05:45.609 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 --2- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7fe5e40387b0 0x7fe5e403ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:45.609 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 --2- 172.21.15.62:0/1488667318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5fc07ad00 0x7fe5fc19b0e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:45.609 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 >> 172.21.15.62:0/1488667318 conn(0x7fe5fc100870 msgr2=0x7fe5fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:45.609 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 shutdown_connections 2023-07-14T16:05:45.609 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:45.159+0000 7fe6010eb700 1 -- 172.21.15.62:0/1488667318 wait complete. 2023-07-14T16:05:45.609 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2023-07-14T16:05:45.610 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 5... 2023-07-14T16:05:45.829 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:45 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:45.498+0000 7ff327ba9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T16:05:45.829 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:45 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:45.630+0000 7ff327ba9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T16:05:46.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:45 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1488667318' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T16:05:46.624 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:46 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:46.219+0000 7ff327ba9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T16:05:47.373 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:46 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:46.989+0000 7ff327ba9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T16:05:47.374 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:47 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:47.201+0000 7ff327ba9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T16:05:48.505 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:48 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:48.177+0000 7ff327ba9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:05:48.505 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:48 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:48.300+0000 7ff327ba9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T16:05:48.505 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:48 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:48.527+0000 7ff327ba9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:05:49.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:48 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:48.896+0000 7ff327ba9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T16:05:49.624 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:49 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:49.174+0000 7ff327ba9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T16:05:49.624 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:49 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:49.374+0000 7ff327ba9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T16:05:50.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:49 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:49.725+0000 7ff327ba9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T16:05:51.112 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:50 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:50.705+0000 7ff327ba9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T16:05:51.113 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:50 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:50.885+0000 7ff327ba9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T16:05:51.113 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:51 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:51.135+0000 7ff327ba9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T16:05:51.373 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:51 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:51.286+0000 7ff327ba9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T16:05:51.873 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:51 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:51.567+0000 7ff327ba9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T16:05:53.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:05:52.660+0000 7ff327ba9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T16:05:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: Active manager daemon x restarted 2023-07-14T16:05:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: Activating manager daemon x 2023-07-14T16:05:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: osdmap e2: 0 total, 0 up, 0 in 2023-07-14T16:05:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: mgrmap e6: x(active, starting, since 0.0040733s) 2023-07-14T16:05:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:05:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T16:05:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T16:05:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T16:05:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T16:05:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: Manager daemon x is now available 2023-07-14T16:05:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:05:53.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:05:53.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:05:53.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:05:53.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:52 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:05:54.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:53 smithi062 ceph-mon[95543]: Found migration_current of "None". Setting to last migration. 2023-07-14T16:05:54.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:53 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T16:05:54.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:53 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T16:05:54.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:53 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:05:54.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:53 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:05:54.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:53 smithi062 ceph-mon[95543]: mgrmap e7: x(active, since 1.00651s) 2023-07-14T16:05:54.198 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2023-07-14T16:05:54.198 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-14T16:05:54.198 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T16:05:54.199 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2023-07-14T16:05:54.199 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.121+0000 7f51970ea700 1 Processor -- start 2023-07-14T16:05:54.199 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.121+0000 7f51970ea700 1 -- start start 2023-07-14T16:05:54.199 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.121+0000 7f51970ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f51900791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:54.199 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.121+0000 7f51970ea700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5190079700 con 0x7f519007acc0 2023-07-14T16:05:54.199 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.122+0000 7f5194e86700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f51900791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:54.199 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.122+0000 7f5194e86700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f51900791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56720/0 (socket says 172.21.15.62:56720) 2023-07-14T16:05:54.199 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.122+0000 7f5194e86700 1 -- 172.21.15.62:0/991634428 learned_addr learned my addr 172.21.15.62:0/991634428 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:54.200 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.122+0000 7f5194e86700 1 -- 172.21.15.62:0/991634428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5190079840 con 0x7f519007acc0 2023-07-14T16:05:54.200 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.122+0000 7f5194e86700 1 --2- 172.21.15.62:0/991634428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f51900791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51840098a0 tx=0x7f5184009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dd6a37935d7173 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:54.200 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.123+0000 7f518f7fe700 1 -- 172.21.15.62:0/991634428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5184004160 con 0x7f519007acc0 2023-07-14T16:05:54.200 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.123+0000 7f518f7fe700 1 -- 172.21.15.62:0/991634428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f51840042c0 con 0x7f519007acc0 2023-07-14T16:05:54.200 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.123+0000 7f518f7fe700 1 -- 172.21.15.62:0/991634428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5184011940 con 0x7f519007acc0 2023-07-14T16:05:54.200 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.123+0000 7f51970ea700 1 -- 172.21.15.62:0/991634428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 msgr2=0x7f51900791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:54.200 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.123+0000 7f51970ea700 1 --2- 172.21.15.62:0/991634428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f51900791c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51840098a0 tx=0x7f5184009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.123+0000 7f51970ea700 1 -- 172.21.15.62:0/991634428 shutdown_connections 2023-07-14T16:05:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.123+0000 7f51970ea700 1 --2- 172.21.15.62:0/991634428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f51900791c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.123+0000 7f51970ea700 1 -- 172.21.15.62:0/991634428 >> 172.21.15.62:0/991634428 conn(0x7f5190100870 msgr2=0x7f5190102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.124+0000 7f51970ea700 1 -- 172.21.15.62:0/991634428 shutdown_connections 2023-07-14T16:05:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.124+0000 7f51970ea700 1 -- 172.21.15.62:0/991634428 wait complete. 2023-07-14T16:05:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.125+0000 7f51970ea700 1 Processor -- start 2023-07-14T16:05:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.125+0000 7f51970ea700 1 -- start start 2023-07-14T16:05:54.202 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.125+0000 7f51970ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f519019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:54.202 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.125+0000 7f51970ea700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519019b630 con 0x7f519007acc0 2023-07-14T16:05:54.202 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.125+0000 7f5194e86700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f519019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:54.202 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.125+0000 7f5194e86700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f519019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56724/0 (socket says 172.21.15.62:56724) 2023-07-14T16:05:54.202 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.125+0000 7f5194e86700 1 -- 172.21.15.62:0/4107221955 learned_addr learned my addr 172.21.15.62:0/4107221955 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:54.202 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.126+0000 7f5194e86700 1 -- 172.21.15.62:0/4107221955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184009550 con 0x7f519007acc0 2023-07-14T16:05:54.202 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.126+0000 7f5194e86700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f519019b0f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5184006af0 tx=0x7f5184011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:54.202 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.126+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5184003bf0 con 0x7f519007acc0 2023-07-14T16:05:54.203 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.126+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f518401b8b0 con 0x7f519007acc0 2023-07-14T16:05:54.203 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.126+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519019b830 con 0x7f519007acc0 2023-07-14T16:05:54.203 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.126+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51840197e0 con 0x7f519007acc0 2023-07-14T16:05:54.203 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.126+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519019bc50 con 0x7f519007acc0 2023-07-14T16:05:54.203 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.128+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5184019aa0 con 0x7f519007acc0 2023-07-14T16:05:54.203 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.128+0000 7f518dffb700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:54.203 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.128+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 --> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f518003b280 con 0x7f51800386c0 2023-07-14T16:05:54.203 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.128+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f518404cb80 con 0x7f519007acc0 2023-07-14T16:05:54.204 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.128+0000 7f518ffff700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 msgr2=0x7f518003ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1177875314 2023-07-14T16:05:54.204 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.128+0000 7f518ffff700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T16:05:54.204 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.329+0000 7f518ffff700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 msgr2=0x7f518003ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1177875314 2023-07-14T16:05:54.204 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.329+0000 7f518ffff700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T16:05:54.204 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.730+0000 7f518ffff700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 msgr2=0x7f518003ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1177875314 2023-07-14T16:05:54.204 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:46.730+0000 7f518ffff700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T16:05:54.204 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:47.531+0000 7f518ffff700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 msgr2=0x7f518003ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1177875314 2023-07-14T16:05:54.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:47.531+0000 7f518ffff700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T16:05:54.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:49.132+0000 7f518ffff700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 msgr2=0x7f518003ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1177875314 2023-07-14T16:05:54.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:49.132+0000 7f518ffff700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T16:05:54.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:52.335+0000 7f518ffff700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 msgr2=0x7f518003ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/1177875314 2023-07-14T16:05:54.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:52.335+0000 7f518ffff700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-14T16:05:54.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:52.664+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f518401a8f0 con 0x7f519007acc0 2023-07-14T16:05:54.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:52.665+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 msgr2=0x7f518003ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:05:54.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:52.665+0000 7f518dffb700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:54.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.666+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f518404e050 con 0x7f519007acc0 2023-07-14T16:05:54.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.666+0000 7f518dffb700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f518003bfe0 0x7f518003e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:54.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.666+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f518003b280 con 0x7f518003bfe0 2023-07-14T16:05:54.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.667+0000 7f518ffff700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f518003bfe0 0x7f518003e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:54.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.668+0000 7f518ffff700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f518003bfe0 0x7f518003e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f517c003a10 tx=0x7f517c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:54.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.669+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mgr.14120 v2:172.21.15.62:6800/105803689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f518003b280 con 0x7f518003bfe0 2023-07-14T16:05:54.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.676+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f519019c430 con 0x7f518003bfe0 2023-07-14T16:05:54.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.676+0000 7f518dffb700 1 -- 172.21.15.62:0/4107221955 <== mgr.14120 v2:172.21.15.62:6800/105803689 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f519019c430 con 0x7f518003bfe0 2023-07-14T16:05:54.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.676+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f518003bfe0 msgr2=0x7f518003e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:54.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.676+0000 7f51970ea700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f518003bfe0 0x7f518003e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f517c003a10 tx=0x7f517c009290 comp rx=0 tx=0).stop 2023-07-14T16:05:54.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.676+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 msgr2=0x7f519019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:54.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.676+0000 7f51970ea700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f519019b0f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5184006af0 tx=0x7f5184011a30 comp rx=0 tx=0).stop 2023-07-14T16:05:54.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.677+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 shutdown_connections 2023-07-14T16:05:54.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.677+0000 7f51970ea700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f518003bfe0 0x7f518003e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:54.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.677+0000 7f51970ea700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519007acc0 0x7f519019b0f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:54.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.677+0000 7f51970ea700 1 --2- 172.21.15.62:0/4107221955 >> [v2:172.21.15.62:6800/1177875314,v1:172.21.15.62:6801/1177875314] conn(0x7f51800386c0 0x7f518003ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:54.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.677+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 >> 172.21.15.62:0/4107221955 conn(0x7f5190100870 msgr2=0x7f5190101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:54.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.677+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 shutdown_connections 2023-07-14T16:05:54.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:53.677+0000 7f51970ea700 1 -- 172.21.15.62:0/4107221955 wait complete. 2023-07-14T16:05:54.208 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 5 is available 2023-07-14T16:05:54.208 INFO:teuthology.orchestra.run.smithi062.stdout:Setting orchestrator backend to cephadm... 2023-07-14T16:05:54.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:54 smithi062 ceph-mon[95543]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-14T16:05:54.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:54 smithi062 ceph-mon[95543]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-14T16:05:54.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:54 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:05:55.376 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.741+0000 7f72eddd8700 1 Processor -- start 2023-07-14T16:05:55.376 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.741+0000 7f72eddd8700 1 -- start start 2023-07-14T16:05:55.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.741+0000 7f72eddd8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:55.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.742+0000 7f72eddd8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e8079700 con 0x7f72e807acc0 2023-07-14T16:05:55.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.742+0000 7f72e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:55.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.742+0000 7f72e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e80791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35936/0 (socket says 172.21.15.62:35936) 2023-07-14T16:05:55.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.742+0000 7f72e77fe700 1 -- 172.21.15.62:0/2054322755 learned_addr learned my addr 172.21.15.62:0/2054322755 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:55.377 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.742+0000 7f72e77fe700 1 -- 172.21.15.62:0/2054322755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8079840 con 0x7f72e807acc0 2023-07-14T16:05:55.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.742+0000 7f72e77fe700 1 --2- 172.21.15.62:0/2054322755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e80791c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f72d80098a0 tx=0x7f72d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b8b0935dbedf1f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:55.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72e67fc700 1 -- 172.21.15.62:0/2054322755 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d8004160 con 0x7f72e807acc0 2023-07-14T16:05:55.378 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72e67fc700 1 -- 172.21.15.62:0/2054322755 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f72d80042c0 con 0x7f72e807acc0 2023-07-14T16:05:55.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72e67fc700 1 -- 172.21.15.62:0/2054322755 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d8011940 con 0x7f72e807acc0 2023-07-14T16:05:55.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72eddd8700 1 -- 172.21.15.62:0/2054322755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 msgr2=0x7f72e80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:55.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72eddd8700 1 --2- 172.21.15.62:0/2054322755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e80791c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f72d80098a0 tx=0x7f72d8009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:55.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72eddd8700 1 -- 172.21.15.62:0/2054322755 shutdown_connections 2023-07-14T16:05:55.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72eddd8700 1 --2- 172.21.15.62:0/2054322755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e80791c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:55.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72eddd8700 1 -- 172.21.15.62:0/2054322755 >> 172.21.15.62:0/2054322755 conn(0x7f72e81007f0 msgr2=0x7f72e8102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:55.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72eddd8700 1 -- 172.21.15.62:0/2054322755 shutdown_connections 2023-07-14T16:05:55.379 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.743+0000 7f72eddd8700 1 -- 172.21.15.62:0/2054322755 wait complete. 2023-07-14T16:05:55.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.744+0000 7f72eddd8700 1 Processor -- start 2023-07-14T16:05:55.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.745+0000 7f72eddd8700 1 -- start start 2023-07-14T16:05:55.380 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.745+0000 7f72eddd8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:55.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.745+0000 7f72eddd8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e819b4e0 con 0x7f72e807acc0 2023-07-14T16:05:55.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.745+0000 7f72e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:55.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.745+0000 7f72e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35946/0 (socket says 172.21.15.62:35946) 2023-07-14T16:05:55.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.745+0000 7f72e77fe700 1 -- 172.21.15.62:0/1445584340 learned_addr learned my addr 172.21.15.62:0/1445584340 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:55.381 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.745+0000 7f72e77fe700 1 -- 172.21.15.62:0/1445584340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8009550 con 0x7f72e807acc0 2023-07-14T16:05:55.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.746+0000 7f72e77fe700 1 --2- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e819afa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000c00 tx=0x7f72d8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:55.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.746+0000 7f72e4ff9700 1 -- 172.21.15.62:0/1445584340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d80036a0 con 0x7f72e807acc0 2023-07-14T16:05:55.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.746+0000 7f72e4ff9700 1 -- 172.21.15.62:0/1445584340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f72d8003c70 con 0x7f72e807acc0 2023-07-14T16:05:55.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.746+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e819b6e0 con 0x7f72e807acc0 2023-07-14T16:05:55.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.746+0000 7f72e4ff9700 1 -- 172.21.15.62:0/1445584340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d80193f0 con 0x7f72e807acc0 2023-07-14T16:05:55.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.746+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e819bb80 con 0x7f72e807acc0 2023-07-14T16:05:55.382 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.747+0000 7f72e4ff9700 1 -- 172.21.15.62:0/1445584340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f72d801b440 con 0x7f72e807acc0 2023-07-14T16:05:55.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.747+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e810b440 con 0x7f72e807acc0 2023-07-14T16:05:55.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.748+0000 7f72e4ff9700 1 --2- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f72d0038760 0x7f72d003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:55.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.748+0000 7f72e4ff9700 1 -- 172.21.15.62:0/1445584340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f72d804c510 con 0x7f72e807acc0 2023-07-14T16:05:55.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.748+0000 7f72e6ffd700 1 --2- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f72d0038760 0x7f72d003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:55.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.749+0000 7f72e6ffd700 1 --2- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f72d0038760 0x7f72d003ac10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72dc006fd0 tx=0x7f72dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:55.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.754+0000 7f72e4ff9700 1 -- 172.21.15.62:0/1445584340 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f72d8022950 con 0x7f72e807acc0 2023-07-14T16:05:55.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.976+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f72e8102aa0 con 0x7f72d0038760 2023-07-14T16:05:55.383 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.983+0000 7f72e4ff9700 1 -- 172.21.15.62:0/1445584340 <== mgr.14120 v2:172.21.15.62:6800/105803689 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f72e8102aa0 con 0x7f72d0038760 2023-07-14T16:05:55.384 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f72d0038760 msgr2=0x7f72d003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:55.384 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 --2- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f72d0038760 0x7f72d003ac10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72dc006fd0 tx=0x7f72dc006e40 comp rx=0 tx=0).stop 2023-07-14T16:05:55.384 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 msgr2=0x7f72e819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:55.384 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 --2- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e819afa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000c00 tx=0x7f72d8011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:55.384 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 shutdown_connections 2023-07-14T16:05:55.384 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 --2- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f72d0038760 0x7f72d003ac10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:55.384 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 --2- 172.21.15.62:0/1445584340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72e807acc0 0x7f72e819afa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:55.385 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 >> 172.21.15.62:0/1445584340 conn(0x7f72e81007f0 msgr2=0x7f72e8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:55.385 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 shutdown_connections 2023-07-14T16:05:55.385 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:54.986+0000 7f72eddd8700 1 -- 172.21.15.62:0/1445584340 wait complete. 2023-07-14T16:05:56.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:55 smithi062 ceph-mon[95543]: [14/Jul/2023:16:05:54] ENGINE Bus STARTING 2023-07-14T16:05:56.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:55 smithi062 ceph-mon[95543]: [14/Jul/2023:16:05:54] ENGINE Serving on https://172.21.15.62:7150 2023-07-14T16:05:56.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:55 smithi062 ceph-mon[95543]: [14/Jul/2023:16:05:54] ENGINE Serving on http://172.21.15.62:8765 2023-07-14T16:05:56.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:55 smithi062 ceph-mon[95543]: [14/Jul/2023:16:05:54] ENGINE Bus STARTED 2023-07-14T16:05:56.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:55 smithi062 ceph-mon[95543]: mgrmap e8: x(active, since 2s) 2023-07-14T16:05:56.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:55 smithi062 ceph-mon[95543]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:05:56.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:55 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:05:56.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:55 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:05:56.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-14T16:05:56.558 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.938+0000 7f466759e700 1 Processor -- start 2023-07-14T16:05:56.558 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.939+0000 7f466759e700 1 -- start start 2023-07-14T16:05:56.558 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.939+0000 7f466759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f4668105490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:56.559 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.939+0000 7f466759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46681059d0 con 0x7f4668105080 2023-07-14T16:05:56.559 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.939+0000 7f466659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f4668105490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:56.559 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.939+0000 7f466659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f4668105490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35954/0 (socket says 172.21.15.62:35954) 2023-07-14T16:05:56.559 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.939+0000 7f466659c700 1 -- 172.21.15.62:0/3379303858 learned_addr learned my addr 172.21.15.62:0/3379303858 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:56.560 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.940+0000 7f466659c700 1 -- 172.21.15.62:0/3379303858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4668105b10 con 0x7f4668105080 2023-07-14T16:05:56.560 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.940+0000 7f466659c700 1 --2- 172.21.15.62:0/3379303858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f4668105490 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46580098a0 tx=0x7f4658009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2117b75cb5142e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:56.560 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.940+0000 7f466559a700 1 -- 172.21.15.62:0/3379303858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4658004160 con 0x7f4668105080 2023-07-14T16:05:56.561 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.940+0000 7f466559a700 1 -- 172.21.15.62:0/3379303858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f46580042c0 con 0x7f4668105080 2023-07-14T16:05:56.561 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.940+0000 7f466559a700 1 -- 172.21.15.62:0/3379303858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4658011940 con 0x7f4668105080 2023-07-14T16:05:56.561 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.941+0000 7f466759e700 1 -- 172.21.15.62:0/3379303858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 msgr2=0x7f4668105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:56.561 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.941+0000 7f466759e700 1 --2- 172.21.15.62:0/3379303858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f4668105490 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46580098a0 tx=0x7f4658009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:56.562 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.941+0000 7f466759e700 1 -- 172.21.15.62:0/3379303858 shutdown_connections 2023-07-14T16:05:56.562 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.941+0000 7f466759e700 1 --2- 172.21.15.62:0/3379303858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f4668105490 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:56.562 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.941+0000 7f466759e700 1 -- 172.21.15.62:0/3379303858 >> 172.21.15.62:0/3379303858 conn(0x7f4668100710 msgr2=0x7f4668102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:56.562 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.941+0000 7f466759e700 1 -- 172.21.15.62:0/3379303858 shutdown_connections 2023-07-14T16:05:56.563 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.941+0000 7f466759e700 1 -- 172.21.15.62:0/3379303858 wait complete. 2023-07-14T16:05:56.563 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.942+0000 7f466759e700 1 Processor -- start 2023-07-14T16:05:56.563 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.943+0000 7f466759e700 1 -- start start 2023-07-14T16:05:56.564 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.943+0000 7f466759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f466807ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:56.564 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.943+0000 7f466759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466807b070 con 0x7f4668105080 2023-07-14T16:05:56.564 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.943+0000 7f466659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f466807ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:56.564 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.943+0000 7f466659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f466807ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35970/0 (socket says 172.21.15.62:35970) 2023-07-14T16:05:56.565 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.943+0000 7f466659c700 1 -- 172.21.15.62:0/3449593104 learned_addr learned my addr 172.21.15.62:0/3449593104 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:56.565 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.944+0000 7f466659c700 1 -- 172.21.15.62:0/3449593104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4658009550 con 0x7f4668105080 2023-07-14T16:05:56.565 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.944+0000 7f466659c700 1 --2- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f466807ab30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4658011dc0 tx=0x7f4658011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:56.565 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.944+0000 7f46577fe700 1 -- 172.21.15.62:0/3449593104 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46580037d0 con 0x7f4668105080 2023-07-14T16:05:56.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.944+0000 7f46577fe700 1 -- 172.21.15.62:0/3449593104 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f465801b440 con 0x7f4668105080 2023-07-14T16:05:56.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.944+0000 7f46577fe700 1 -- 172.21.15.62:0/3449593104 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f465801a560 con 0x7f4668105080 2023-07-14T16:05:56.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.944+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4668079180 con 0x7f4668105080 2023-07-14T16:05:56.566 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.944+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46680795a0 con 0x7f4668105080 2023-07-14T16:05:56.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.945+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4668061bb0 con 0x7f4668105080 2023-07-14T16:05:56.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.945+0000 7f46577fe700 1 -- 172.21.15.62:0/3449593104 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f465801b5b0 con 0x7f4668105080 2023-07-14T16:05:56.567 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.946+0000 7f46577fe700 1 --2- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f4650038760 0x7f465003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:56.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.946+0000 7f46577fe700 1 -- 172.21.15.62:0/3449593104 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f465804c5f0 con 0x7f4668105080 2023-07-14T16:05:56.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.946+0000 7f4665d9b700 1 --2- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f4650038760 0x7f465003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:56.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.947+0000 7f4665d9b700 1 --2- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f4650038760 0x7f465003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f465c006fd0 tx=0x7f465c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:56.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:55.951+0000 7f46577fe700 1 -- 172.21.15.62:0/3449593104 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4668061bb0 con 0x7f4668105080 2023-07-14T16:05:56.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.159+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f4668061b40 con 0x7f4650038760 2023-07-14T16:05:56.568 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.160+0000 7f46577fe700 1 -- 172.21.15.62:0/3449593104 <== mgr.14120 v2:172.21.15.62:6800/105803689 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f4668061b40 con 0x7f4650038760 2023-07-14T16:05:56.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f4650038760 msgr2=0x7f465003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:56.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 --2- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f4650038760 0x7f465003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f465c006fd0 tx=0x7f465c006e40 comp rx=0 tx=0).stop 2023-07-14T16:05:56.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 msgr2=0x7f466807ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:56.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 --2- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f466807ab30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4658011dc0 tx=0x7f4658011ea0 comp rx=0 tx=0).stop 2023-07-14T16:05:56.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 shutdown_connections 2023-07-14T16:05:56.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 --2- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f4650038760 0x7f465003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:56.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 --2- 172.21.15.62:0/3449593104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4668105080 0x7f466807ab30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:56.569 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 >> 172.21.15.62:0/3449593104 conn(0x7f4668100710 msgr2=0x7f4668101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:56.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 shutdown_connections 2023-07-14T16:05:56.570 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:56.162+0000 7f466759e700 1 -- 172.21.15.62:0/3449593104 wait complete. 2023-07-14T16:05:56.570 INFO:teuthology.orchestra.run.smithi062.stdout:Generating ssh key... 2023-07-14T16:05:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-mon[95543]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:05:57.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: Generating public/private rsa key pair. 2023-07-14T16:05:57.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: Your identification has been saved in /tmp/tmp21glc1pe/key. 2023-07-14T16:05:57.874 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: Your public key has been saved in /tmp/tmp21glc1pe/key.pub. 2023-07-14T16:05:57.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: The key fingerprint is: 2023-07-14T16:05:57.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: SHA256:DwfnIi/IXX1eXRqLvQ4TfLN4kGfPpjBIkKkfB/bs2DE ceph-1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:57.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: The key's randomart image is: 2023-07-14T16:05:57.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: +---[RSA 3072]----+ 2023-07-14T16:05:57.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | | 2023-07-14T16:05:57.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | o | 2023-07-14T16:05:57.875 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | B . . .| 2023-07-14T16:05:57.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | o X . + =.| 2023-07-14T16:05:57.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | o S E B X .| 2023-07-14T16:05:57.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | . o = % * O * | 2023-07-14T16:05:57.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | o o + = B + +| 2023-07-14T16:05:57.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | . B o | 2023-07-14T16:05:57.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: | o | 2023-07-14T16:05:57.876 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:05:57 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: +----[SHA256]-----+ 2023-07-14T16:05:57.934 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.154+0000 7f7a375c7700 1 Processor -- start 2023-07-14T16:05:57.934 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.154+0000 7f7a375c7700 1 -- start start 2023-07-14T16:05:57.934 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.155+0000 7f7a375c7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a300791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:57.935 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.155+0000 7f7a375c7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30079700 con 0x7f7a3007acc0 2023-07-14T16:05:57.935 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.155+0000 7f7a35363700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a300791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:57.935 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.155+0000 7f7a35363700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a300791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35974/0 (socket says 172.21.15.62:35974) 2023-07-14T16:05:57.935 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.155+0000 7f7a35363700 1 -- 172.21.15.62:0/636897620 learned_addr learned my addr 172.21.15.62:0/636897620 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:57.935 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.155+0000 7f7a35363700 1 -- 172.21.15.62:0/636897620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30079840 con 0x7f7a3007acc0 2023-07-14T16:05:57.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.156+0000 7f7a35363700 1 --2- 172.21.15.62:0/636897620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a300791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0098a0 tx=0x7f7a2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9590fe4b6e0c8949 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:57.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.156+0000 7f7a27fff700 1 -- 172.21.15.62:0/636897620 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c004160 con 0x7f7a3007acc0 2023-07-14T16:05:57.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.156+0000 7f7a27fff700 1 -- 172.21.15.62:0/636897620 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f7a2c0042c0 con 0x7f7a3007acc0 2023-07-14T16:05:57.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.156+0000 7f7a27fff700 1 -- 172.21.15.62:0/636897620 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c011940 con 0x7f7a3007acc0 2023-07-14T16:05:57.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.156+0000 7f7a375c7700 1 -- 172.21.15.62:0/636897620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 msgr2=0x7f7a300791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:57.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.156+0000 7f7a375c7700 1 --2- 172.21.15.62:0/636897620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a300791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0098a0 tx=0x7f7a2c009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:57.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.157+0000 7f7a375c7700 1 -- 172.21.15.62:0/636897620 shutdown_connections 2023-07-14T16:05:57.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.157+0000 7f7a375c7700 1 --2- 172.21.15.62:0/636897620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a300791c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:57.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.157+0000 7f7a375c7700 1 -- 172.21.15.62:0/636897620 >> 172.21.15.62:0/636897620 conn(0x7f7a30100870 msgr2=0x7f7a30102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:57.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.157+0000 7f7a375c7700 1 -- 172.21.15.62:0/636897620 shutdown_connections 2023-07-14T16:05:57.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.157+0000 7f7a375c7700 1 -- 172.21.15.62:0/636897620 wait complete. 2023-07-14T16:05:57.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.158+0000 7f7a375c7700 1 Processor -- start 2023-07-14T16:05:57.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.158+0000 7f7a375c7700 1 -- start start 2023-07-14T16:05:57.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.158+0000 7f7a375c7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a3019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.158+0000 7f7a375c7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019b630 con 0x7f7a3007acc0 2023-07-14T16:05:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.158+0000 7f7a35363700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a3019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.158+0000 7f7a35363700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a3019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35978/0 (socket says 172.21.15.62:35978) 2023-07-14T16:05:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.158+0000 7f7a35363700 1 -- 172.21.15.62:0/3933584263 learned_addr learned my addr 172.21.15.62:0/3933584263 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.159+0000 7f7a35363700 1 -- 172.21.15.62:0/3933584263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009550 con 0x7f7a3007acc0 2023-07-14T16:05:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.159+0000 7f7a35363700 1 --2- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a3019b0f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c000c00 tx=0x7f7a2c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.159+0000 7f7a267fc700 1 -- 172.21.15.62:0/3933584263 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c0036a0 con 0x7f7a3007acc0 2023-07-14T16:05:57.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.159+0000 7f7a267fc700 1 -- 172.21.15.62:0/3933584263 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f7a2c003c70 con 0x7f7a3007acc0 2023-07-14T16:05:57.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.159+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3019b830 con 0x7f7a3007acc0 2023-07-14T16:05:57.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.159+0000 7f7a267fc700 1 -- 172.21.15.62:0/3933584263 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c0193f0 con 0x7f7a3007acc0 2023-07-14T16:05:57.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.159+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3019bc50 con 0x7f7a3007acc0 2023-07-14T16:05:57.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.160+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3010b500 con 0x7f7a3007acc0 2023-07-14T16:05:57.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.160+0000 7f7a267fc700 1 -- 172.21.15.62:0/3933584263 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7a2c01b440 con 0x7f7a3007acc0 2023-07-14T16:05:57.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.161+0000 7f7a267fc700 1 --2- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7a1c038760 0x7f7a1c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:57.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.161+0000 7f7a267fc700 1 -- 172.21.15.62:0/3933584263 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7a2c04c5d0 con 0x7f7a3007acc0 2023-07-14T16:05:57.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.161+0000 7f7a34b62700 1 --2- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7a1c038760 0x7f7a1c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:57.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.162+0000 7f7a34b62700 1 --2- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7a1c038760 0x7f7a1c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a20006fd0 tx=0x7f7a20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:57.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.166+0000 7f7a267fc700 1 -- 172.21.15.62:0/3933584263 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7a2c022950 con 0x7f7a3007acc0 2023-07-14T16:05:57.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.373+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a3019eab0 con 0x7f7a1c038760 2023-07-14T16:05:57.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.532+0000 7f7a267fc700 1 -- 172.21.15.62:0/3933584263 <== mgr.14120 v2:172.21.15.62:6800/105803689 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7a3019eab0 con 0x7f7a1c038760 2023-07-14T16:05:57.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7a1c038760 msgr2=0x7f7a1c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:57.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 --2- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7a1c038760 0x7f7a1c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a20006fd0 tx=0x7f7a20006e40 comp rx=0 tx=0).stop 2023-07-14T16:05:57.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 msgr2=0x7f7a3019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 --2- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a3019b0f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c000c00 tx=0x7f7a2c011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 shutdown_connections 2023-07-14T16:05:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 --2- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7a1c038760 0x7f7a1c03ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 --2- 172.21.15.62:0/3933584263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3007acc0 0x7f7a3019b0f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 >> 172.21.15.62:0/3933584263 conn(0x7f7a30100870 msgr2=0x7f7a30101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 shutdown_connections 2023-07-14T16:05:57.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:57.534+0000 7f7a375c7700 1 -- 172.21.15.62:0/3933584263 wait complete. 2023-07-14T16:05:58.814 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:58 smithi062 ceph-mon[95543]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:05:58.814 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:58 smithi062 ceph-mon[95543]: Generating ssh key... 2023-07-14T16:05:58.814 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:58 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:05:58.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:58 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:05:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7lsLbyjedCWGjthQtNo+U03AkS5b+ezfdaRGTZs7P2L3Mk1KHLb8BPRg33iJIJ+63iWi7gGQnPXIdpG98qK+d5osB+3SVuqX3nFAkWhKrr8Tx06zKWxKxAQtPvIPB8fTfrIIwszb2cX7mAUEpwn+gqOH4ucSArZfHxUR0Ktn8JzfNZiSJWBKAS9ituoVjHNjEDWKSwnqFf4PjhMDbEO1NzU2wyZ0WHzzhZaqTYfpqAL2k1Ivg0ai2IHZWFJQq8rV/akvYpIOhmNbMNrdmB5zVVuXoJTqUpL5XAkJKlAJyI+KXnm1fSXo0z6m0IXAtXEFw1R+/wnuw5Mt6FK1Y8NodQA8iq5e+8eS4OOt/NejGusu9nJUKEZkPzcM4t6QjcSVAbxLg3we9mTUgV8OKN3mgZvldgMUqV5KpAaVtXucyZopkMnwVFOeMXbAucTHw3mEmBNzamA0hizf9FsX/slDA7V7S/xTbnrmVVNvcnUA9rKQAg59AC+FDNAL53rbE4FE= ceph-1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:05:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.455+0000 7f118cdb5700 1 Processor -- start 2023-07-14T16:05:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.455+0000 7f118cdb5700 1 -- start start 2023-07-14T16:05:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.455+0000 7f118cdb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f11880791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.455+0000 7f118cdb5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880796e0 con 0x7f118807aca0 2023-07-14T16:05:59.119 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.456+0000 7f118659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f11880791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.456+0000 7f118659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f11880791a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35984/0 (socket says 172.21.15.62:35984) 2023-07-14T16:05:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.456+0000 7f118659c700 1 -- 172.21.15.62:0/1726388495 learned_addr learned my addr 172.21.15.62:0/1726388495 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.456+0000 7f118659c700 1 -- 172.21.15.62:0/1726388495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188079820 con 0x7f118807aca0 2023-07-14T16:05:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.456+0000 7f118659c700 1 --2- 172.21.15.62:0/1726388495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f11880791a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f11780098a0 tx=0x7f1178009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee637655ce51fd86 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118559a700 1 -- 172.21.15.62:0/1726388495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1178004160 con 0x7f118807aca0 2023-07-14T16:05:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118559a700 1 -- 172.21.15.62:0/1726388495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f11780042c0 con 0x7f118807aca0 2023-07-14T16:05:59.120 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118559a700 1 -- 172.21.15.62:0/1726388495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1178011940 con 0x7f118807aca0 2023-07-14T16:05:59.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118cdb5700 1 -- 172.21.15.62:0/1726388495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 msgr2=0x7f11880791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:59.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118cdb5700 1 --2- 172.21.15.62:0/1726388495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f11880791a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f11780098a0 tx=0x7f1178009bb0 comp rx=0 tx=0).stop 2023-07-14T16:05:59.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118cdb5700 1 -- 172.21.15.62:0/1726388495 shutdown_connections 2023-07-14T16:05:59.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118cdb5700 1 --2- 172.21.15.62:0/1726388495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f11880791a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:59.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118cdb5700 1 -- 172.21.15.62:0/1726388495 >> 172.21.15.62:0/1726388495 conn(0x7f1188100870 msgr2=0x7f1188102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:59.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.457+0000 7f118cdb5700 1 -- 172.21.15.62:0/1726388495 shutdown_connections 2023-07-14T16:05:59.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.458+0000 7f118cdb5700 1 -- 172.21.15.62:0/1726388495 wait complete. 2023-07-14T16:05:59.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.458+0000 7f118cdb5700 1 Processor -- start 2023-07-14T16:05:59.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.459+0000 7f118cdb5700 1 -- start start 2023-07-14T16:05:59.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.459+0000 7f118cdb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f118819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:59.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.459+0000 7f118cdb5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118819b610 con 0x7f118807aca0 2023-07-14T16:05:59.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.459+0000 7f118659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f118819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:59.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.459+0000 7f118659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f118819b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35996/0 (socket says 172.21.15.62:35996) 2023-07-14T16:05:59.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.459+0000 7f118659c700 1 -- 172.21.15.62:0/1045959539 learned_addr learned my addr 172.21.15.62:0/1045959539 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:05:59.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.460+0000 7f118659c700 1 -- 172.21.15.62:0/1045959539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178009550 con 0x7f118807aca0 2023-07-14T16:05:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.460+0000 7f118659c700 1 --2- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f118819b0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1178000c00 tx=0x7f1178011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.460+0000 7f11777fe700 1 -- 172.21.15.62:0/1045959539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11780036a0 con 0x7f118807aca0 2023-07-14T16:05:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.460+0000 7f11777fe700 1 -- 172.21.15.62:0/1045959539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f1178003c70 con 0x7f118807aca0 2023-07-14T16:05:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.460+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118819b810 con 0x7f118807aca0 2023-07-14T16:05:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.460+0000 7f11777fe700 1 -- 172.21.15.62:0/1045959539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11780193f0 con 0x7f118807aca0 2023-07-14T16:05:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.460+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118819bc30 con 0x7f118807aca0 2023-07-14T16:05:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.461+0000 7f11777fe700 1 -- 172.21.15.62:0/1045959539 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f117801b440 con 0x7f118807aca0 2023-07-14T16:05:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.461+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1188061b40 con 0x7f118807aca0 2023-07-14T16:05:59.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.462+0000 7f11777fe700 1 --2- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1170038760 0x7f117003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:05:59.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.462+0000 7f11777fe700 1 -- 172.21.15.62:0/1045959539 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f117804c5d0 con 0x7f118807aca0 2023-07-14T16:05:59.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.462+0000 7f1185d9b700 1 --2- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1170038760 0x7f117003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:05:59.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.463+0000 7f1185d9b700 1 --2- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1170038760 0x7f117003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f117c006fd0 tx=0x7f117c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:05:59.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.467+0000 7f11777fe700 1 -- 172.21.15.62:0/1045959539 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1178022950 con 0x7f118807aca0 2023-07-14T16:05:59.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.674+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1188102b50 con 0x7f1170038760 2023-07-14T16:05:59.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.675+0000 7f11777fe700 1 -- 172.21.15.62:0/1045959539 <== mgr.14120 v2:172.21.15.62:6800/105803689 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1188102b50 con 0x7f1170038760 2023-07-14T16:05:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1170038760 msgr2=0x7f117003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 --2- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1170038760 0x7f117003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f117c006fd0 tx=0x7f117c006e40 comp rx=0 tx=0).stop 2023-07-14T16:05:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 msgr2=0x7f118819b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:05:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 --2- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f118819b0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1178000c00 tx=0x7f1178011e70 comp rx=0 tx=0).stop 2023-07-14T16:05:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 shutdown_connections 2023-07-14T16:05:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 --2- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1170038760 0x7f117003ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 --2- 172.21.15.62:0/1045959539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f118807aca0 0x7f118819b0d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:05:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 >> 172.21.15.62:0/1045959539 conn(0x7f1188100870 msgr2=0x7f1188101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:05:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 shutdown_connections 2023-07-14T16:05:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:58.677+0000 7f118cdb5700 1 -- 172.21.15.62:0/1045959539 wait complete. 2023-07-14T16:05:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-14T16:05:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:Adding key to root@localhost authorized_keys... 2023-07-14T16:05:59.127 INFO:teuthology.orchestra.run.smithi062.stdout:Adding host smithi062... 2023-07-14T16:05:59.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:05:59 smithi062 ceph-mon[95543]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:00.569 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:00 smithi062 ceph-mon[95543]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:01.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:01 smithi062 ceph-mon[95543]: Deploying cephadm binary to smithi062 2023-07-14T16:06:03.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Added host 'smithi062' with addr '172.21.15.62' 2023-07-14T16:06:03.508 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.674+0000 7f817640e700 1 Processor -- start 2023-07-14T16:06:03.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.674+0000 7f817640e700 1 -- start start 2023-07-14T16:06:03.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.675+0000 7f817640e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f81700791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:03.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.675+0000 7f817640e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170079700 con 0x7f817007acc0 2023-07-14T16:06:03.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.675+0000 7f816ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f81700791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:03.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.675+0000 7f816ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f81700791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36000/0 (socket says 172.21.15.62:36000) 2023-07-14T16:06:03.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.675+0000 7f816ffff700 1 -- 172.21.15.62:0/684070715 learned_addr learned my addr 172.21.15.62:0/684070715 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:03.509 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.676+0000 7f816ffff700 1 -- 172.21.15.62:0/684070715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170079840 con 0x7f817007acc0 2023-07-14T16:06:03.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.676+0000 7f816ffff700 1 --2- 172.21.15.62:0/684070715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f81700791c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81600098a0 tx=0x7f8160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb41d30cb0353f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:03.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.676+0000 7f816effd700 1 -- 172.21.15.62:0/684070715 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8160004160 con 0x7f817007acc0 2023-07-14T16:06:03.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.676+0000 7f816effd700 1 -- 172.21.15.62:0/684070715 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f81600042c0 con 0x7f817007acc0 2023-07-14T16:06:03.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.676+0000 7f816effd700 1 -- 172.21.15.62:0/684070715 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8160011940 con 0x7f817007acc0 2023-07-14T16:06:03.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.676+0000 7f817640e700 1 -- 172.21.15.62:0/684070715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 msgr2=0x7f81700791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:03.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.676+0000 7f817640e700 1 --2- 172.21.15.62:0/684070715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f81700791c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81600098a0 tx=0x7f8160009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:03.510 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.677+0000 7f817640e700 1 -- 172.21.15.62:0/684070715 shutdown_connections 2023-07-14T16:06:03.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.677+0000 7f817640e700 1 --2- 172.21.15.62:0/684070715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f81700791c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:03.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.677+0000 7f817640e700 1 -- 172.21.15.62:0/684070715 >> 172.21.15.62:0/684070715 conn(0x7f8170100830 msgr2=0x7f8170102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:03.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.677+0000 7f817640e700 1 -- 172.21.15.62:0/684070715 shutdown_connections 2023-07-14T16:06:03.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.677+0000 7f817640e700 1 -- 172.21.15.62:0/684070715 wait complete. 2023-07-14T16:06:03.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.678+0000 7f817640e700 1 Processor -- start 2023-07-14T16:06:03.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.678+0000 7f817640e700 1 -- start start 2023-07-14T16:06:03.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.678+0000 7f817640e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f817019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:03.511 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.678+0000 7f817640e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817019b560 con 0x7f817007acc0 2023-07-14T16:06:03.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f816ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f817019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:03.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f816ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f817019b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36004/0 (socket says 172.21.15.62:36004) 2023-07-14T16:06:03.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f816ffff700 1 -- 172.21.15.62:0/2060688536 learned_addr learned my addr 172.21.15.62:0/2060688536 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:03.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f816ffff700 1 -- 172.21.15.62:0/2060688536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160009550 con 0x7f817007acc0 2023-07-14T16:06:03.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f816ffff700 1 --2- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f817019b020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8160000c00 tx=0x7f8160011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:03.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f816d7fa700 1 -- 172.21.15.62:0/2060688536 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81600036a0 con 0x7f817007acc0 2023-07-14T16:06:03.512 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f816d7fa700 1 -- 172.21.15.62:0/2060688536 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8160003c70 con 0x7f817007acc0 2023-07-14T16:06:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f816d7fa700 1 -- 172.21.15.62:0/2060688536 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81600193f0 con 0x7f817007acc0 2023-07-14T16:06:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.679+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817019b760 con 0x7f817007acc0 2023-07-14T16:06:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.680+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817019bc00 con 0x7f817007acc0 2023-07-14T16:06:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.681+0000 7f816d7fa700 1 -- 172.21.15.62:0/2060688536 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f816001b440 con 0x7f817007acc0 2023-07-14T16:06:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.681+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817010b4c0 con 0x7f817007acc0 2023-07-14T16:06:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.681+0000 7f816d7fa700 1 --2- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8158038760 0x7f815803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.681+0000 7f816d7fa700 1 -- 172.21.15.62:0/2060688536 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f816004c5d0 con 0x7f817007acc0 2023-07-14T16:06:03.513 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.681+0000 7f816f7fe700 1 --2- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8158038760 0x7f815803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:03.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.682+0000 7f816f7fe700 1 --2- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8158038760 0x7f815803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8164006fd0 tx=0x7f8164006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:03.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.686+0000 7f816d7fa700 1 -- 172.21.15.62:0/2060688536 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8160022950 con 0x7f817007acc0 2023-07-14T16:06:03.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:05:59.896+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}) v1 -- 0x7f8170102ad0 con 0x7f8158038760 2023-07-14T16:06:03.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.102+0000 7f816d7fa700 1 -- 172.21.15.62:0/2060688536 <== mgr.14120 v2:172.21.15.62:6800/105803689 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f8170102ad0 con 0x7f8158038760 2023-07-14T16:06:03.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8158038760 msgr2=0x7f815803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:03.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 --2- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8158038760 0x7f815803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8164006fd0 tx=0x7f8164006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:03.514 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 msgr2=0x7f817019b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:03.515 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 --2- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f817019b020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8160000c00 tx=0x7f8160011e70 comp rx=0 tx=0).stop 2023-07-14T16:06:03.515 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 shutdown_connections 2023-07-14T16:06:03.515 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 --2- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8158038760 0x7f815803ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:03.515 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 --2- 172.21.15.62:0/2060688536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f817007acc0 0x7f817019b020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:03.515 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 >> 172.21.15.62:0/2060688536 conn(0x7f8170100830 msgr2=0x7f8170101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:03.515 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 shutdown_connections 2023-07-14T16:06:03.515 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:03.106+0000 7f817640e700 1 -- 172.21.15.62:0/2060688536 wait complete. 2023-07-14T16:06:03.516 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mon service... 2023-07-14T16:06:04.363 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:04 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:06:04.363 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:04 smithi062 ceph-mon[95543]: Added host smithi062 2023-07-14T16:06:04.363 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:04 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:04.987 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-14T16:06:04.995 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.120+0000 7f5d43d58700 1 Processor -- start 2023-07-14T16:06:04.996 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.120+0000 7f5d43d58700 1 -- start start 2023-07-14T16:06:04.996 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.120+0000 7f5d43d58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:04.996 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.120+0000 7f5d43d58700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c079700 con 0x7f5d3c07acc0 2023-07-14T16:06:04.996 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.121+0000 7f5d41af4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:04.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.121+0000 7f5d41af4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44282/0 (socket says 172.21.15.62:44282) 2023-07-14T16:06:04.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.121+0000 7f5d41af4700 1 -- 172.21.15.62:0/2999186642 learned_addr learned my addr 172.21.15.62:0/2999186642 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:04.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.121+0000 7f5d41af4700 1 -- 172.21.15.62:0/2999186642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c079840 con 0x7f5d3c07acc0 2023-07-14T16:06:04.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.121+0000 7f5d41af4700 1 --2- 172.21.15.62:0/2999186642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c0791c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d380098a0 tx=0x7f5d38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87689e99ad6a454 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:04.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d40af2700 1 -- 172.21.15.62:0/2999186642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d38004160 con 0x7f5d3c07acc0 2023-07-14T16:06:04.997 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d40af2700 1 -- 172.21.15.62:0/2999186642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5d380042c0 con 0x7f5d3c07acc0 2023-07-14T16:06:04.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d40af2700 1 -- 172.21.15.62:0/2999186642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d38011940 con 0x7f5d3c07acc0 2023-07-14T16:06:04.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d43d58700 1 -- 172.21.15.62:0/2999186642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 msgr2=0x7f5d3c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:04.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d43d58700 1 --2- 172.21.15.62:0/2999186642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c0791c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d380098a0 tx=0x7f5d38009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:04.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d43d58700 1 -- 172.21.15.62:0/2999186642 shutdown_connections 2023-07-14T16:06:04.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d43d58700 1 --2- 172.21.15.62:0/2999186642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c0791c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:04.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d43d58700 1 -- 172.21.15.62:0/2999186642 >> 172.21.15.62:0/2999186642 conn(0x7f5d3c100830 msgr2=0x7f5d3c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:04.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.122+0000 7f5d43d58700 1 -- 172.21.15.62:0/2999186642 shutdown_connections 2023-07-14T16:06:04.998 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.123+0000 7f5d43d58700 1 -- 172.21.15.62:0/2999186642 wait complete. 2023-07-14T16:06:04.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.124+0000 7f5d43d58700 1 Processor -- start 2023-07-14T16:06:04.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.124+0000 7f5d43d58700 1 -- start start 2023-07-14T16:06:04.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.124+0000 7f5d43d58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:04.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.124+0000 7f5d43d58700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c19b560 con 0x7f5d3c07acc0 2023-07-14T16:06:04.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.124+0000 7f5d41af4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:04.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.124+0000 7f5d41af4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c19b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44298/0 (socket says 172.21.15.62:44298) 2023-07-14T16:06:04.999 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.124+0000 7f5d41af4700 1 -- 172.21.15.62:0/3521424857 learned_addr learned my addr 172.21.15.62:0/3521424857 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:05.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.125+0000 7f5d41af4700 1 -- 172.21.15.62:0/3521424857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d38009550 con 0x7f5d3c07acc0 2023-07-14T16:06:05.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.125+0000 7f5d41af4700 1 --2- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c19b020 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000c00 tx=0x7f5d38011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:05.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.125+0000 7f5d32ffd700 1 -- 172.21.15.62:0/3521424857 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d380036a0 con 0x7f5d3c07acc0 2023-07-14T16:06:05.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.125+0000 7f5d32ffd700 1 -- 172.21.15.62:0/3521424857 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5d38003c70 con 0x7f5d3c07acc0 2023-07-14T16:06:05.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.125+0000 7f5d32ffd700 1 -- 172.21.15.62:0/3521424857 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d380193f0 con 0x7f5d3c07acc0 2023-07-14T16:06:05.000 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.125+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c19b760 con 0x7f5d3c07acc0 2023-07-14T16:06:05.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.125+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c19bc00 con 0x7f5d3c07acc0 2023-07-14T16:06:05.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.126+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c10b4c0 con 0x7f5d3c07acc0 2023-07-14T16:06:05.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.126+0000 7f5d32ffd700 1 -- 172.21.15.62:0/3521424857 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5d3801b440 con 0x7f5d3c07acc0 2023-07-14T16:06:05.001 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.127+0000 7f5d32ffd700 1 --2- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f5d28038760 0x7f5d2803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.127+0000 7f5d32ffd700 1 -- 172.21.15.62:0/3521424857 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5d3804c5d0 con 0x7f5d3c07acc0 2023-07-14T16:06:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.127+0000 7f5d412f3700 1 --2- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f5d28038760 0x7f5d2803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.128+0000 7f5d412f3700 1 --2- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f5d28038760 0x7f5d2803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c006fd0 tx=0x7f5d2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.132+0000 7f5d32ffd700 1 -- 172.21.15.62:0/3521424857 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5d38022950 con 0x7f5d3c07acc0 2023-07-14T16:06:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.355+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5d3c102ad0 con 0x7f5d28038760 2023-07-14T16:06:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.362+0000 7f5d32ffd700 1 -- 172.21.15.62:0/3521424857 <== mgr.14120 v2:172.21.15.62:6800/105803689 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5d3c102ad0 con 0x7f5d28038760 2023-07-14T16:06:05.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f5d28038760 msgr2=0x7f5d2803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:05.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 --2- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f5d28038760 0x7f5d2803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c006fd0 tx=0x7f5d2c006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:05.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 msgr2=0x7f5d3c19b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:05.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 --2- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c19b020 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000c00 tx=0x7f5d38011e70 comp rx=0 tx=0).stop 2023-07-14T16:06:05.003 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 shutdown_connections 2023-07-14T16:06:05.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 --2- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f5d28038760 0x7f5d2803ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:05.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 --2- 172.21.15.62:0/3521424857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d3c07acc0 0x7f5d3c19b020 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:05.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 >> 172.21.15.62:0/3521424857 conn(0x7f5d3c100830 msgr2=0x7f5d3c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:05.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 shutdown_connections 2023-07-14T16:06:05.004 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:04.365+0000 7f5d43d58700 1 -- 172.21.15.62:0/3521424857 wait complete. 2023-07-14T16:06:05.004 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mgr service... 2023-07-14T16:06:05.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:05 smithi062 ceph-mon[95543]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:05.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:05 smithi062 ceph-mon[95543]: Saving service mon spec with placement count:5 2023-07-14T16:06:05.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:05 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:06:06.476 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-14T16:06:06.477 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.801+0000 7fb562a0a700 1 Processor -- start 2023-07-14T16:06:06.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.802+0000 7fb562a0a700 1 -- start start 2023-07-14T16:06:06.478 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.802+0000 7fb562a0a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:06.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.802+0000 7fb562a0a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c079700 con 0x7fb55c07acc0 2023-07-14T16:06:06.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.802+0000 7fb55bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:06.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.803+0000 7fb55bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44302/0 (socket says 172.21.15.62:44302) 2023-07-14T16:06:06.479 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.803+0000 7fb55bfff700 1 -- 172.21.15.62:0/3148372896 learned_addr learned my addr 172.21.15.62:0/3148372896 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:06.480 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.803+0000 7fb55bfff700 1 -- 172.21.15.62:0/3148372896 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c079840 con 0x7fb55c07acc0 2023-07-14T16:06:06.480 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.803+0000 7fb55bfff700 1 --2- 172.21.15.62:0/3148372896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c0791c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0098a0 tx=0x7fb54c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9348ebf2e3d3efdd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:06.480 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.803+0000 7fb55affd700 1 -- 172.21.15.62:0/3148372896 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54c004160 con 0x7fb55c07acc0 2023-07-14T16:06:06.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.803+0000 7fb55affd700 1 -- 172.21.15.62:0/3148372896 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb54c0042c0 con 0x7fb55c07acc0 2023-07-14T16:06:06.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.803+0000 7fb55affd700 1 -- 172.21.15.62:0/3148372896 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54c011940 con 0x7fb55c07acc0 2023-07-14T16:06:06.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.804+0000 7fb562a0a700 1 -- 172.21.15.62:0/3148372896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 msgr2=0x7fb55c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:06.481 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.804+0000 7fb562a0a700 1 --2- 172.21.15.62:0/3148372896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c0791c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0098a0 tx=0x7fb54c009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:06.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.804+0000 7fb562a0a700 1 -- 172.21.15.62:0/3148372896 shutdown_connections 2023-07-14T16:06:06.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.804+0000 7fb562a0a700 1 --2- 172.21.15.62:0/3148372896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c0791c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:06.482 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.804+0000 7fb562a0a700 1 -- 172.21.15.62:0/3148372896 >> 172.21.15.62:0/3148372896 conn(0x7fb55c100830 msgr2=0x7fb55c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:06.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.805+0000 7fb562a0a700 1 -- 172.21.15.62:0/3148372896 shutdown_connections 2023-07-14T16:06:06.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.805+0000 7fb562a0a700 1 -- 172.21.15.62:0/3148372896 wait complete. 2023-07-14T16:06:06.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.806+0000 7fb562a0a700 1 Processor -- start 2023-07-14T16:06:06.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.806+0000 7fb562a0a700 1 -- start start 2023-07-14T16:06:06.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.806+0000 7fb562a0a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:06.483 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.806+0000 7fb562a0a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c19b560 con 0x7fb55c07acc0 2023-07-14T16:06:06.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.806+0000 7fb55bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:06.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.806+0000 7fb55bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c19b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44318/0 (socket says 172.21.15.62:44318) 2023-07-14T16:06:06.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.806+0000 7fb55bfff700 1 -- 172.21.15.62:0/2540371337 learned_addr learned my addr 172.21.15.62:0/2540371337 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:06.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.807+0000 7fb55bfff700 1 -- 172.21.15.62:0/2540371337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c009550 con 0x7fb55c07acc0 2023-07-14T16:06:06.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.807+0000 7fb55bfff700 1 --2- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c19b020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb54c000c00 tx=0x7fb54c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:06.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.807+0000 7fb5597fa700 1 -- 172.21.15.62:0/2540371337 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54c0036a0 con 0x7fb55c07acc0 2023-07-14T16:06:06.484 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.807+0000 7fb5597fa700 1 -- 172.21.15.62:0/2540371337 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb54c003c70 con 0x7fb55c07acc0 2023-07-14T16:06:06.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.807+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55c19b760 con 0x7fb55c07acc0 2023-07-14T16:06:06.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.807+0000 7fb5597fa700 1 -- 172.21.15.62:0/2540371337 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54c0193f0 con 0x7fb55c07acc0 2023-07-14T16:06:06.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.808+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c19bc00 con 0x7fb55c07acc0 2023-07-14T16:06:06.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.809+0000 7fb5597fa700 1 -- 172.21.15.62:0/2540371337 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fb54c019620 con 0x7fb55c07acc0 2023-07-14T16:06:06.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.809+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55c10b4c0 con 0x7fb55c07acc0 2023-07-14T16:06:06.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.809+0000 7fb5597fa700 1 --2- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7fb544038760 0x7fb54403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:06.485 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.809+0000 7fb5597fa700 1 -- 172.21.15.62:0/2540371337 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb54c04c5e0 con 0x7fb55c07acc0 2023-07-14T16:06:06.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.810+0000 7fb55b7fe700 1 --2- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7fb544038760 0x7fb54403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:06.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.810+0000 7fb55b7fe700 1 --2- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7fb544038760 0x7fb54403ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb550006fd0 tx=0x7fb550006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:06.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.814+0000 7fb5597fa700 1 -- 172.21.15.62:0/2540371337 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb54c022720 con 0x7fb55c07acc0 2023-07-14T16:06:06.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:05.837+0000 7fb5597fa700 1 -- 172.21.15.62:0/2540371337 <== mon.0 v2:172.21.15.62:3300/0 7 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb54c019e90 con 0x7fb55c07acc0 2023-07-14T16:06:06.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.049+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb55c102ad0 con 0x7fb544038760 2023-07-14T16:06:06.486 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.057+0000 7fb5597fa700 1 -- 172.21.15.62:0/2540371337 <== mgr.14120 v2:172.21.15.62:6800/105803689 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb55c102ad0 con 0x7fb544038760 2023-07-14T16:06:06.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.059+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7fb544038760 msgr2=0x7fb54403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:06.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.059+0000 7fb562a0a700 1 --2- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7fb544038760 0x7fb54403ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb550006fd0 tx=0x7fb550006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:06.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.059+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 msgr2=0x7fb55c19b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:06.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.059+0000 7fb562a0a700 1 --2- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c19b020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb54c000c00 tx=0x7fb54c011e70 comp rx=0 tx=0).stop 2023-07-14T16:06:06.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.060+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 shutdown_connections 2023-07-14T16:06:06.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.060+0000 7fb562a0a700 1 --2- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7fb544038760 0x7fb54403ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:06.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.060+0000 7fb562a0a700 1 --2- 172.21.15.62:0/2540371337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb55c07acc0 0x7fb55c19b020 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:06.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.060+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 >> 172.21.15.62:0/2540371337 conn(0x7fb55c100830 msgr2=0x7fb55c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:06.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.060+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 shutdown_connections 2023-07-14T16:06:06.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:06.060+0000 7fb562a0a700 1 -- 172.21.15.62:0/2540371337 wait complete. 2023-07-14T16:06:06.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:06 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:06:06.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:06 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:06:06.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:06 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:06:07.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.030+0000 7f8ea62fc700 1 Processor -- start 2023-07-14T16:06:07.656 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.031+0000 7f8ea62fc700 1 -- start start 2023-07-14T16:06:07.656 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.031+0000 7f8ea62fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea0105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:07.656 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.031+0000 7f8ea62fc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0105b50 con 0x7f8ea0105200 2023-07-14T16:06:07.656 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.031+0000 7f8e9ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea0105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:07.656 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.032+0000 7f8e9ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea0105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44332/0 (socket says 172.21.15.62:44332) 2023-07-14T16:06:07.656 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.032+0000 7f8e9ffff700 1 -- 172.21.15.62:0/1702214492 learned_addr learned my addr 172.21.15.62:0/1702214492 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:07.657 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.032+0000 7f8e9ffff700 1 -- 172.21.15.62:0/1702214492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0105c90 con 0x7f8ea0105200 2023-07-14T16:06:07.657 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.032+0000 7f8e9ffff700 1 --2- 172.21.15.62:0/1702214492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea0105610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8e900098a0 tx=0x7f8e90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9239a967c692239 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:07.657 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.032+0000 7f8e9effd700 1 -- 172.21.15.62:0/1702214492 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e90004160 con 0x7f8ea0105200 2023-07-14T16:06:07.657 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.032+0000 7f8e9effd700 1 -- 172.21.15.62:0/1702214492 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e900042c0 con 0x7f8ea0105200 2023-07-14T16:06:07.657 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.032+0000 7f8e9effd700 1 -- 172.21.15.62:0/1702214492 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e900118e0 con 0x7f8ea0105200 2023-07-14T16:06:07.657 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.033+0000 7f8ea62fc700 1 -- 172.21.15.62:0/1702214492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 msgr2=0x7f8ea0105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:07.657 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.033+0000 7f8ea62fc700 1 --2- 172.21.15.62:0/1702214492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea0105610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8e900098a0 tx=0x7f8e90009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:07.658 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.033+0000 7f8ea62fc700 1 -- 172.21.15.62:0/1702214492 shutdown_connections 2023-07-14T16:06:07.658 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.033+0000 7f8ea62fc700 1 --2- 172.21.15.62:0/1702214492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea0105610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:07.658 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.033+0000 7f8ea62fc700 1 -- 172.21.15.62:0/1702214492 >> 172.21.15.62:0/1702214492 conn(0x7f8ea0100870 msgr2=0x7f8ea0102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:07.658 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.033+0000 7f8ea62fc700 1 -- 172.21.15.62:0/1702214492 shutdown_connections 2023-07-14T16:06:07.658 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.033+0000 7f8ea62fc700 1 -- 172.21.15.62:0/1702214492 wait complete. 2023-07-14T16:06:07.658 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.034+0000 7f8ea62fc700 1 Processor -- start 2023-07-14T16:06:07.658 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.034+0000 7f8ea62fc700 1 -- start start 2023-07-14T16:06:07.658 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.035+0000 7f8ea62fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:07.659 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.035+0000 7f8ea62fc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea019b660 con 0x7f8ea0105200 2023-07-14T16:06:07.659 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.035+0000 7f8e9ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:07.659 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.035+0000 7f8e9ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44346/0 (socket says 172.21.15.62:44346) 2023-07-14T16:06:07.659 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.035+0000 7f8e9ffff700 1 -- 172.21.15.62:0/727213077 learned_addr learned my addr 172.21.15.62:0/727213077 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:07.659 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.035+0000 7f8e9ffff700 1 -- 172.21.15.62:0/727213077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e90009550 con 0x7f8ea0105200 2023-07-14T16:06:07.659 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.036+0000 7f8e9ffff700 1 --2- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea019b120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8e90003e30 tx=0x7f8e900119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:07.659 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.036+0000 7f8e9d7fa700 1 -- 172.21.15.62:0/727213077 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e900036a0 con 0x7f8ea0105200 2023-07-14T16:06:07.660 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.036+0000 7f8e9d7fa700 1 -- 172.21.15.62:0/727213077 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e9001b440 con 0x7f8ea0105200 2023-07-14T16:06:07.660 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.036+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea019b860 con 0x7f8ea0105200 2023-07-14T16:06:07.660 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.036+0000 7f8e9d7fa700 1 -- 172.21.15.62:0/727213077 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e90023400 con 0x7f8ea0105200 2023-07-14T16:06:07.660 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.036+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea019bc80 con 0x7f8ea0105200 2023-07-14T16:06:07.660 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.037+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea0061b40 con 0x7f8ea0105200 2023-07-14T16:06:07.660 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.037+0000 7f8e9d7fa700 1 -- 172.21.15.62:0/727213077 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8e9001b5b0 con 0x7f8ea0105200 2023-07-14T16:06:07.660 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.038+0000 7f8e9d7fa700 1 --2- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8e88038760 0x7f8e8803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:07.660 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.038+0000 7f8e9d7fa700 1 -- 172.21.15.62:0/727213077 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8e9004c710 con 0x7f8ea0105200 2023-07-14T16:06:07.661 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.038+0000 7f8e9f7fe700 1 --2- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8e88038760 0x7f8e8803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:07.661 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.039+0000 7f8e9f7fe700 1 --2- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8e88038760 0x7f8e8803ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:07.661 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.042+0000 7f8e9d7fa700 1 -- 172.21.15.62:0/727213077 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8e90019840 con 0x7f8ea0105200 2023-07-14T16:06:07.661 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.254+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f8ea01096a0 con 0x7f8ea0105200 2023-07-14T16:06:07.661 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.257+0000 7f8e9d7fa700 1 -- 172.21.15.62:0/727213077 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8e90019840 con 0x7f8ea0105200 2023-07-14T16:06:07.661 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.260+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8e88038760 msgr2=0x7f8e8803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:07.661 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.260+0000 7f8ea62fc700 1 --2- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8e88038760 0x7f8e8803ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:07.662 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.260+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 msgr2=0x7f8ea019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:07.662 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.260+0000 7f8ea62fc700 1 --2- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea019b120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8e90003e30 tx=0x7f8e900119a0 comp rx=0 tx=0).stop 2023-07-14T16:06:07.662 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.260+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 shutdown_connections 2023-07-14T16:06:07.662 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.260+0000 7f8ea62fc700 1 --2- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f8e88038760 0x7f8e8803ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:07.662 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.260+0000 7f8ea62fc700 1 --2- 172.21.15.62:0/727213077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ea0105200 0x7f8ea019b120 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:07.662 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.260+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 >> 172.21.15.62:0/727213077 conn(0x7f8ea0100870 msgr2=0x7f8ea0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:07.662 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.261+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 shutdown_connections 2023-07-14T16:06:07.663 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:07.261+0000 7f8ea62fc700 1 -- 172.21.15.62:0/727213077 wait complete. 2023-07-14T16:06:07.928 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:07 smithi062 ceph-mon[95543]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:07.929 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:07 smithi062 ceph-mon[95543]: Saving service mgr spec with placement count:2 2023-07-14T16:06:07.929 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:07 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/727213077' entity='client.admin' 2023-07-14T16:06:08.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.285+0000 7faccd90c700 1 Processor -- start 2023-07-14T16:06:08.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.285+0000 7faccd90c700 1 -- start start 2023-07-14T16:06:08.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.285+0000 7faccd90c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc81055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:08.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.285+0000 7faccd90c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc8105b00 con 0x7facc81051b0 2023-07-14T16:06:08.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.286+0000 7facc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc81055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:08.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.286+0000 7facc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc81055c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44358/0 (socket says 172.21.15.62:44358) 2023-07-14T16:06:08.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.286+0000 7facc6ffd700 1 -- 172.21.15.62:0/2415718902 learned_addr learned my addr 172.21.15.62:0/2415718902 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:08.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.286+0000 7facc6ffd700 1 -- 172.21.15.62:0/2415718902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8105c40 con 0x7facc81051b0 2023-07-14T16:06:08.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.287+0000 7facc6ffd700 1 --2- 172.21.15.62:0/2415718902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc81055c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facc0009e80 tx=0x7facc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafff02e65e5f589 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:08.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.287+0000 7facc5ffb700 1 -- 172.21.15.62:0/2415718902 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc000e070 con 0x7facc81051b0 2023-07-14T16:06:08.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.287+0000 7facc5ffb700 1 -- 172.21.15.62:0/2415718902 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facc000b8c0 con 0x7facc81051b0 2023-07-14T16:06:08.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.287+0000 7facc5ffb700 1 -- 172.21.15.62:0/2415718902 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc0004360 con 0x7facc81051b0 2023-07-14T16:06:08.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.288+0000 7faccd90c700 1 -- 172.21.15.62:0/2415718902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 msgr2=0x7facc81055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:08.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.288+0000 7faccd90c700 1 --2- 172.21.15.62:0/2415718902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc81055c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facc0009e80 tx=0x7facc000b330 comp rx=0 tx=0).stop 2023-07-14T16:06:08.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.288+0000 7faccd90c700 1 -- 172.21.15.62:0/2415718902 shutdown_connections 2023-07-14T16:06:08.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.288+0000 7faccd90c700 1 --2- 172.21.15.62:0/2415718902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc81055c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:08.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.288+0000 7faccd90c700 1 -- 172.21.15.62:0/2415718902 >> 172.21.15.62:0/2415718902 conn(0x7facc8100840 msgr2=0x7facc8102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:08.946 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.288+0000 7faccd90c700 1 -- 172.21.15.62:0/2415718902 shutdown_connections 2023-07-14T16:06:08.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.288+0000 7faccd90c700 1 -- 172.21.15.62:0/2415718902 wait complete. 2023-07-14T16:06:08.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.289+0000 7faccd90c700 1 Processor -- start 2023-07-14T16:06:08.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.289+0000 7faccd90c700 1 -- start start 2023-07-14T16:06:08.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.289+0000 7faccd90c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:08.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.290+0000 7faccd90c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc819b570 con 0x7facc81051b0 2023-07-14T16:06:08.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.290+0000 7facc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc819b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:08.947 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.290+0000 7facc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc819b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44374/0 (socket says 172.21.15.62:44374) 2023-07-14T16:06:08.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.290+0000 7facc6ffd700 1 -- 172.21.15.62:0/3794885471 learned_addr learned my addr 172.21.15.62:0/3794885471 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:08.948 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.290+0000 7facc6ffd700 1 -- 172.21.15.62:0/3794885471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0009b30 con 0x7facc81051b0 2023-07-14T16:06:08.950 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.290+0000 7facc6ffd700 1 --2- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc819b030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facc0009e50 tx=0x7facc0003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:08.950 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.291+0000 7faccc90a700 1 -- 172.21.15.62:0/3794885471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc000e040 con 0x7facc81051b0 2023-07-14T16:06:08.950 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.291+0000 7faccc90a700 1 -- 172.21.15.62:0/3794885471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facc000bd70 con 0x7facc81051b0 2023-07-14T16:06:08.950 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.291+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc819b770 con 0x7facc81051b0 2023-07-14T16:06:08.950 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.291+0000 7faccc90a700 1 -- 172.21.15.62:0/3794885471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc001c590 con 0x7facc81051b0 2023-07-14T16:06:08.951 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.291+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc819bc70 con 0x7facc81051b0 2023-07-14T16:06:08.951 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.292+0000 7faccc90a700 1 -- 172.21.15.62:0/3794885471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7facc0012430 con 0x7facc81051b0 2023-07-14T16:06:08.951 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.292+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc8061b40 con 0x7facc81051b0 2023-07-14T16:06:08.951 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.293+0000 7faccc90a700 1 --2- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7facb0038710 0x7facb003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:08.951 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.293+0000 7faccc90a700 1 -- 172.21.15.62:0/3794885471 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7facc004c720 con 0x7facc81051b0 2023-07-14T16:06:08.951 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.293+0000 7facc67fc700 1 --2- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7facb0038710 0x7facb003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:08.951 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.294+0000 7facc67fc700 1 --2- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7facb0038710 0x7facb003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7facb8006fd0 tx=0x7facb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:08.952 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.298+0000 7faccc90a700 1 -- 172.21.15.62:0/3794885471 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7facc0011390 con 0x7facc81051b0 2023-07-14T16:06:08.952 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.513+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7facc819e9f0 con 0x7facc81051b0 2023-07-14T16:06:08.952 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.516+0000 7faccc90a700 1 -- 172.21.15.62:0/3794885471 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7facc0020020 con 0x7facc81051b0 2023-07-14T16:06:08.952 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7facb0038710 msgr2=0x7facb003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:08.952 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 --2- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7facb0038710 0x7facb003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7facb8006fd0 tx=0x7facb8006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:08.952 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 msgr2=0x7facc819b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:08.952 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 --2- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc819b030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facc0009e50 tx=0x7facc0003c50 comp rx=0 tx=0).stop 2023-07-14T16:06:08.953 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 shutdown_connections 2023-07-14T16:06:08.953 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 --2- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7facb0038710 0x7facb003abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:08.953 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 --2- 172.21.15.62:0/3794885471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facc81051b0 0x7facc819b030 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:08.953 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 >> 172.21.15.62:0/3794885471 conn(0x7facc8100840 msgr2=0x7facc81015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:08.953 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 shutdown_connections 2023-07-14T16:06:08.953 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:08.519+0000 7faccd90c700 1 -- 172.21.15.62:0/3794885471 wait complete. 2023-07-14T16:06:08.953 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling the dashboard module... 2023-07-14T16:06:09.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:09 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3794885471' entity='client.admin' 2023-07-14T16:06:09.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:09 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:06:10.814 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:10 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1917667191' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-14T16:06:10.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:10 smithi062 ceph-mon[95543]: from='mgr.14120 172.21.15.62:0/1477051639' entity='mgr.x' 2023-07-14T16:06:10.815 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:10 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: ignoring --setuser ceph since I am not root 2023-07-14T16:06:10.815 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:10 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: ignoring --setgroup ceph since I am not root 2023-07-14T16:06:10.936 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.518+0000 7f577a0d6700 1 Processor -- start 2023-07-14T16:06:10.936 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.518+0000 7f577a0d6700 1 -- start start 2023-07-14T16:06:10.936 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.519+0000 7f577a0d6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5774073b10 0x7f5774107d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:10.936 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.519+0000 7f577a0d6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774108290 con 0x7f5774073b10 2023-07-14T16:06:10.937 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.519+0000 7f57737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5774073b10 0x7f5774107d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:10.937 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.519+0000 7f57737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5774073b10 0x7f5774107d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44384/0 (socket says 172.21.15.62:44384) 2023-07-14T16:06:10.937 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.519+0000 7f57737fe700 1 -- 172.21.15.62:0/655895959 learned_addr learned my addr 172.21.15.62:0/655895959 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:10.937 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.520+0000 7f57737fe700 1 -- 172.21.15.62:0/655895959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57741083d0 con 0x7f5774073b10 2023-07-14T16:06:10.937 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.520+0000 7f57737fe700 1 --2- 172.21.15.62:0/655895959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5774073b10 0x7f5774107d50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f57640098a0 tx=0x7f5764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd25d0bd86f7f8b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:10.937 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.520+0000 7f57727fc700 1 -- 172.21.15.62:0/655895959 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5764004160 con 0x7f5774073b10 2023-07-14T16:06:10.937 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.520+0000 7f57727fc700 1 -- 172.21.15.62:0/655895959 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57640042c0 con 0x7f5774073b10 2023-07-14T16:06:10.937 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.520+0000 7f57727fc700 1 -- 172.21.15.62:0/655895959 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57640118e0 con 0x7f5774073b10 2023-07-14T16:06:10.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.521+0000 7f577a0d6700 1 -- 172.21.15.62:0/655895959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5774073b10 msgr2=0x7f5774107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:10.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.521+0000 7f577a0d6700 1 --2- 172.21.15.62:0/655895959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5774073b10 0x7f5774107d50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f57640098a0 tx=0x7f5764009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:10.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.521+0000 7f577a0d6700 1 -- 172.21.15.62:0/655895959 shutdown_connections 2023-07-14T16:06:10.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.521+0000 7f577a0d6700 1 --2- 172.21.15.62:0/655895959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5774073b10 0x7f5774107d50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:10.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.521+0000 7f577a0d6700 1 -- 172.21.15.62:0/655895959 >> 172.21.15.62:0/655895959 conn(0x7f57740ff720 msgr2=0x7f5774101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:10.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.521+0000 7f577a0d6700 1 -- 172.21.15.62:0/655895959 shutdown_connections 2023-07-14T16:06:10.938 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.521+0000 7f577a0d6700 1 -- 172.21.15.62:0/655895959 wait complete. 2023-07-14T16:06:10.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.522+0000 7f577a0d6700 1 Processor -- start 2023-07-14T16:06:10.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.522+0000 7f577a0d6700 1 -- start start 2023-07-14T16:06:10.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.523+0000 7f577a0d6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577419b080 0x7f577419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:10.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.523+0000 7f577a0d6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577419b9d0 con 0x7f577419b080 2023-07-14T16:06:10.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.523+0000 7f57737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577419b080 0x7f577419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:10.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.523+0000 7f57737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577419b080 0x7f577419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44394/0 (socket says 172.21.15.62:44394) 2023-07-14T16:06:10.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.523+0000 7f57737fe700 1 -- 172.21.15.62:0/1917667191 learned_addr learned my addr 172.21.15.62:0/1917667191 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:10.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.523+0000 7f57737fe700 1 -- 172.21.15.62:0/1917667191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5764009550 con 0x7f577419b080 2023-07-14T16:06:10.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.524+0000 7f57737fe700 1 --2- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577419b080 0x7f577419b490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5764003e30 tx=0x7f57640119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:10.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.524+0000 7f5770ff9700 1 -- 172.21.15.62:0/1917667191 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57640036a0 con 0x7f577419b080 2023-07-14T16:06:10.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.524+0000 7f5770ff9700 1 -- 172.21.15.62:0/1917667191 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f576401b440 con 0x7f577419b080 2023-07-14T16:06:10.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.524+0000 7f5770ff9700 1 -- 172.21.15.62:0/1917667191 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5764023400 con 0x7f577419b080 2023-07-14T16:06:10.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.524+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577419bbd0 con 0x7f577419b080 2023-07-14T16:06:10.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.524+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577419e830 con 0x7f577419b080 2023-07-14T16:06:10.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.525+0000 7f5770ff9700 1 -- 172.21.15.62:0/1917667191 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5764003800 con 0x7f577419b080 2023-07-14T16:06:10.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.525+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5774073b10 con 0x7f577419b080 2023-07-14T16:06:10.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.526+0000 7f5770ff9700 1 --2- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f575c038760 0x7f575c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:10.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.526+0000 7f5770ff9700 1 -- 172.21.15.62:0/1917667191 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f576404c6d0 con 0x7f577419b080 2023-07-14T16:06:10.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.526+0000 7f5772ffd700 1 --2- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f575c038760 0x7f575c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:10.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.527+0000 7f5772ffd700 1 --2- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f575c038760 0x7f575c03ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5768006fd0 tx=0x7f5768006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:10.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.531+0000 7f5770ff9700 1 -- 172.21.15.62:0/1917667191 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f576402b350 con 0x7f577419b080 2023-07-14T16:06:10.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:09.819+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f5774073d30 con 0x7f577419b080 2023-07-14T16:06:10.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.522+0000 7f5770ff9700 1 -- 172.21.15.62:0/1917667191 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f576401b960 con 0x7f577419b080 2023-07-14T16:06:10.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.524+0000 7f5770ff9700 1 -- 172.21.15.62:0/1917667191 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f576404c020 con 0x7f577419b080 2023-07-14T16:06:10.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f575c038760 msgr2=0x7f575c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:10.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 --2- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f575c038760 0x7f575c03ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5768006fd0 tx=0x7f5768006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:10.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577419b080 msgr2=0x7f577419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:10.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 --2- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577419b080 0x7f577419b490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5764003e30 tx=0x7f57640119a0 comp rx=0 tx=0).stop 2023-07-14T16:06:10.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 shutdown_connections 2023-07-14T16:06:10.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 --2- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f575c038760 0x7f575c03ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:10.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 --2- 172.21.15.62:0/1917667191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f577419b080 0x7f577419b490 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:10.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 >> 172.21.15.62:0/1917667191 conn(0x7f57740ff720 msgr2=0x7f5774100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:10.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.526+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 shutdown_connections 2023-07-14T16:06:10.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:10.527+0000 7f577a0d6700 1 -- 172.21.15.62:0/1917667191 wait complete. 2023-07-14T16:06:11.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:10 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:10.838+0000 7fdbc23e9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T16:06:11.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:10 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:10.945+0000 7fdbc23e9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T16:06:11.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:11 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:11.083+0000 7fdbc23e9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T16:06:11.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:11 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:11.216+0000 7fdbc23e9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T16:06:11.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:11 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:11.465+0000 7fdbc23e9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T16:06:11.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:11 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1917667191' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-14T16:06:11.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:11 smithi062 ceph-mon[95543]: mgrmap e9: x(active, since 17s) 2023-07-14T16:06:12.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:11 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:11.925+0000 7fdbc23e9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T16:06:12.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:12 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:12.063+0000 7fdbc23e9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T16:06:12.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:12 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:12.231+0000 7fdbc23e9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T16:06:12.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:12 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:12.360+0000 7fdbc23e9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T16:06:12.526 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2023-07-14T16:06:12.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-14T16:06:12.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T16:06:12.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T16:06:12.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T16:06:12.527 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2023-07-14T16:06:12.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.565+0000 7f1344077700 1 Processor -- start 2023-07-14T16:06:12.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.565+0000 7f1344077700 1 -- start start 2023-07-14T16:06:12.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.566+0000 7f1344077700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c108140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:12.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.566+0000 7f1344077700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c073b30 con 0x7f133c105d60 2023-07-14T16:06:12.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.566+0000 7f1341e13700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:12.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.566+0000 7f1341e13700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44430/0 (socket says 172.21.15.62:44430) 2023-07-14T16:06:12.528 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.566+0000 7f1341e13700 1 -- 172.21.15.62:0/2917386437 learned_addr learned my addr 172.21.15.62:0/2917386437 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:12.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.567+0000 7f1341e13700 1 -- 172.21.15.62:0/2917386437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c073c70 con 0x7f133c105d60 2023-07-14T16:06:12.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.567+0000 7f1341e13700 1 --2- 172.21.15.62:0/2917386437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c108140 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13380098a0 tx=0x7f1338009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd1f3d9e479bc4a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:12.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.567+0000 7f1340e11700 1 -- 172.21.15.62:0/2917386437 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1338004160 con 0x7f133c105d60 2023-07-14T16:06:12.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.567+0000 7f1340e11700 1 -- 172.21.15.62:0/2917386437 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13380042c0 con 0x7f133c105d60 2023-07-14T16:06:12.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.567+0000 7f1340e11700 1 -- 172.21.15.62:0/2917386437 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13380118e0 con 0x7f133c105d60 2023-07-14T16:06:12.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.568+0000 7f1344077700 1 -- 172.21.15.62:0/2917386437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 msgr2=0x7f133c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:12.529 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.568+0000 7f1344077700 1 --2- 172.21.15.62:0/2917386437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c108140 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13380098a0 tx=0x7f1338009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:12.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.568+0000 7f1344077700 1 -- 172.21.15.62:0/2917386437 shutdown_connections 2023-07-14T16:06:12.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.568+0000 7f1344077700 1 --2- 172.21.15.62:0/2917386437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c108140 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:12.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.568+0000 7f1344077700 1 -- 172.21.15.62:0/2917386437 >> 172.21.15.62:0/2917386437 conn(0x7f133c0ff760 msgr2=0x7f133c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:12.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.568+0000 7f1344077700 1 -- 172.21.15.62:0/2917386437 shutdown_connections 2023-07-14T16:06:12.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.569+0000 7f1344077700 1 -- 172.21.15.62:0/2917386437 wait complete. 2023-07-14T16:06:12.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.570+0000 7f1344077700 1 Processor -- start 2023-07-14T16:06:12.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.570+0000 7f1344077700 1 -- start start 2023-07-14T16:06:12.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.570+0000 7f1344077700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:12.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.570+0000 7f1344077700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c19b6a0 con 0x7f133c105d60 2023-07-14T16:06:12.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.570+0000 7f1341e13700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:12.531 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.570+0000 7f1341e13700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c19b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44444/0 (socket says 172.21.15.62:44444) 2023-07-14T16:06:12.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.570+0000 7f1341e13700 1 -- 172.21.15.62:0/2974579391 learned_addr learned my addr 172.21.15.62:0/2974579391 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:12.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.571+0000 7f1341e13700 1 -- 172.21.15.62:0/2974579391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1338009550 con 0x7f133c105d60 2023-07-14T16:06:12.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.571+0000 7f1341e13700 1 --2- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c19b160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1338003e30 tx=0x7f13380119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:12.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.571+0000 7f1332ffd700 1 -- 172.21.15.62:0/2974579391 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13380036a0 con 0x7f133c105d60 2023-07-14T16:06:12.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.572+0000 7f1332ffd700 1 -- 172.21.15.62:0/2974579391 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f133801b440 con 0x7f133c105d60 2023-07-14T16:06:12.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.572+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c19b8a0 con 0x7f133c105d60 2023-07-14T16:06:12.532 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.572+0000 7f1332ffd700 1 -- 172.21.15.62:0/2974579391 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1338023400 con 0x7f133c105d60 2023-07-14T16:06:12.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.572+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c19bcc0 con 0x7f133c105d60 2023-07-14T16:06:12.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.573+0000 7f1332ffd700 1 -- 172.21.15.62:0/2974579391 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1338003800 con 0x7f133c105d60 2023-07-14T16:06:12.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.573+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c1092d0 con 0x7f133c105d60 2023-07-14T16:06:12.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.573+0000 7f1332ffd700 1 --2- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1328038760 0x7f132803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:12.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.574+0000 7f1332ffd700 1 -- 172.21.15.62:0/2974579391 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f133804c770 con 0x7f133c105d60 2023-07-14T16:06:12.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.574+0000 7f1341612700 1 -- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1328038760 msgr2=0x7f132803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/105803689 2023-07-14T16:06:12.533 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.574+0000 7f1341612700 1 --2- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1328038760 0x7f132803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T16:06:12.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.582+0000 7f1332ffd700 1 -- 172.21.15.62:0/2974579391 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f133802b350 con 0x7f133c105d60 2023-07-14T16:06:12.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.775+0000 7f1341612700 1 -- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1328038760 msgr2=0x7f132803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/105803689 2023-07-14T16:06:12.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.775+0000 7f1341612700 1 --2- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1328038760 0x7f132803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T16:06:12.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.857+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f133c1094c0 con 0x7f133c105d60 2023-07-14T16:06:12.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.858+0000 7f1332ffd700 1 -- 172.21.15.62:0/2974579391 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f133802b350 con 0x7f133c105d60 2023-07-14T16:06:12.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.862+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1328038760 msgr2=0x7f132803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:06:12.534 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.862+0000 7f1344077700 1 --2- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1328038760 0x7f132803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:12.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.862+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 msgr2=0x7f133c19b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:12.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.862+0000 7f1344077700 1 --2- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c19b160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1338003e30 tx=0x7f13380119a0 comp rx=0 tx=0).stop 2023-07-14T16:06:12.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.862+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 shutdown_connections 2023-07-14T16:06:12.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.862+0000 7f1344077700 1 --2- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f1328038760 0x7f132803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:12.535 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.862+0000 7f1344077700 1 --2- 172.21.15.62:0/2974579391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133c105d60 0x7f133c19b160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:12.536 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.862+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 >> 172.21.15.62:0/2974579391 conn(0x7f133c0ff760 msgr2=0x7f133c100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:12.536 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.863+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 shutdown_connections 2023-07-14T16:06:12.537 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:11.863+0000 7f1344077700 1 -- 172.21.15.62:0/2974579391 wait complete. 2023-07-14T16:06:12.537 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2023-07-14T16:06:12.537 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 9... 2023-07-14T16:06:12.782 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:12 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2974579391' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T16:06:13.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:12 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:12.938+0000 7fdbc23e9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T16:06:14.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:13 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:13.708+0000 7fdbc23e9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T16:06:14.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:13 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:13.921+0000 7fdbc23e9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T16:06:15.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:14 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:14.895+0000 7fdbc23e9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:06:15.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:15 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:15.018+0000 7fdbc23e9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T16:06:15.590 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:15 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:15.246+0000 7fdbc23e9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:06:16.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:15 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:15.614+0000 7fdbc23e9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T16:06:16.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:15 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:15.896+0000 7fdbc23e9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T16:06:16.124 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:16 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:16.093+0000 7fdbc23e9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T16:06:16.873 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:16 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:16.447+0000 7fdbc23e9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T16:06:17.812 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:17 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:17.404+0000 7fdbc23e9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T16:06:17.812 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:17 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:17.583+0000 7fdbc23e9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T16:06:17.813 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:17 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:17.836+0000 7fdbc23e9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T16:06:18.123 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:17 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:17.988+0000 7fdbc23e9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T16:06:18.623 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:18 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:18.270+0000 7fdbc23e9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T16:06:19.623 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:06:19.367+0000 7fdbc23e9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T16:06:19.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: Active manager daemon x restarted 2023-07-14T16:06:19.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: Activating manager daemon x 2023-07-14T16:06:19.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: osdmap e3: 0 total, 0 up, 0 in 2023-07-14T16:06:19.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: mgrmap e10: x(active, starting, since 0.006013s) 2023-07-14T16:06:19.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:06:19.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T16:06:19.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T16:06:19.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T16:06:19.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T16:06:19.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: Manager daemon x is now available 2023-07-14T16:06:19.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:19.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:20.728 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2023-07-14T16:06:20.728 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-14T16:06:20.729 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T16:06:20.729 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2023-07-14T16:06:20.729 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.101+0000 7f765bd4e700 1 Processor -- start 2023-07-14T16:06:20.729 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.101+0000 7f765bd4e700 1 -- start start 2023-07-14T16:06:20.729 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.102+0000 7f765bd4e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f76540791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:20.729 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.102+0000 7f765bd4e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654079720 con 0x7f765407ace0 2023-07-14T16:06:20.730 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.102+0000 7f7659aea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f76540791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:20.731 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.102+0000 7f7659aea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f76540791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46968/0 (socket says 172.21.15.62:46968) 2023-07-14T16:06:20.731 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.102+0000 7f7659aea700 1 -- 172.21.15.62:0/3505412090 learned_addr learned my addr 172.21.15.62:0/3505412090 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:20.731 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.103+0000 7f7659aea700 1 -- 172.21.15.62:0/3505412090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7654079860 con 0x7f765407ace0 2023-07-14T16:06:20.731 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.103+0000 7f7659aea700 1 --2- 172.21.15.62:0/3505412090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f76540791e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76500098a0 tx=0x7f7650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60f4fefb6585770 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:20.731 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.103+0000 7f7658ae8700 1 -- 172.21.15.62:0/3505412090 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7650004160 con 0x7f765407ace0 2023-07-14T16:06:20.732 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.103+0000 7f7658ae8700 1 -- 172.21.15.62:0/3505412090 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76500042c0 con 0x7f765407ace0 2023-07-14T16:06:20.732 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.103+0000 7f7658ae8700 1 -- 172.21.15.62:0/3505412090 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76500118e0 con 0x7f765407ace0 2023-07-14T16:06:20.732 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.104+0000 7f765bd4e700 1 -- 172.21.15.62:0/3505412090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 msgr2=0x7f76540791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:20.732 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.104+0000 7f765bd4e700 1 --2- 172.21.15.62:0/3505412090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f76540791e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76500098a0 tx=0x7f7650009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:20.732 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.104+0000 7f765bd4e700 1 -- 172.21.15.62:0/3505412090 shutdown_connections 2023-07-14T16:06:20.732 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.104+0000 7f765bd4e700 1 --2- 172.21.15.62:0/3505412090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f76540791e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:20.732 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.104+0000 7f765bd4e700 1 -- 172.21.15.62:0/3505412090 >> 172.21.15.62:0/3505412090 conn(0x7f7654100870 msgr2=0x7f7654102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:20.733 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.104+0000 7f765bd4e700 1 -- 172.21.15.62:0/3505412090 shutdown_connections 2023-07-14T16:06:20.733 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.104+0000 7f765bd4e700 1 -- 172.21.15.62:0/3505412090 wait complete. 2023-07-14T16:06:20.733 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.105+0000 7f765bd4e700 1 Processor -- start 2023-07-14T16:06:20.733 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.105+0000 7f765bd4e700 1 -- start start 2023-07-14T16:06:20.733 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.105+0000 7f765bd4e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f765419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:20.733 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.105+0000 7f765bd4e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765419b6e0 con 0x7f765407ace0 2023-07-14T16:06:20.733 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.106+0000 7f7659aea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f765419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:20.733 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.106+0000 7f7659aea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f765419b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46982/0 (socket says 172.21.15.62:46982) 2023-07-14T16:06:20.734 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.106+0000 7f7659aea700 1 -- 172.21.15.62:0/598794972 learned_addr learned my addr 172.21.15.62:0/598794972 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:20.734 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.106+0000 7f7659aea700 1 -- 172.21.15.62:0/598794972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7650009550 con 0x7f765407ace0 2023-07-14T16:06:20.734 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.106+0000 7f7659aea700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f765419b1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7650000c00 tx=0x7f76500119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:20.734 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.107+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76500036a0 con 0x7f765407ace0 2023-07-14T16:06:20.734 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.107+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765419b8e0 con 0x7f765407ace0 2023-07-14T16:06:20.734 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.107+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7650024440 con 0x7f765407ace0 2023-07-14T16:06:20.735 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.107+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f765001a430 con 0x7f765407ace0 2023-07-14T16:06:20.735 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.107+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765419bd00 con 0x7f765407ace0 2023-07-14T16:06:20.735 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.108+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f765001a6b0 con 0x7f765407ace0 2023-07-14T16:06:20.735 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.108+0000 7f764affd700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:20.735 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.108+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 --> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f764003b320 con 0x7f7640038760 2023-07-14T16:06:20.735 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.108+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f765004c550 con 0x7f765407ace0 2023-07-14T16:06:20.736 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.109+0000 7f76592e9700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 msgr2=0x7f764003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/105803689 2023-07-14T16:06:20.736 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.109+0000 7f76592e9700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T16:06:20.736 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.310+0000 7f76592e9700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 msgr2=0x7f764003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/105803689 2023-07-14T16:06:20.736 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.310+0000 7f76592e9700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T16:06:20.736 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.711+0000 7f76592e9700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 msgr2=0x7f764003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/105803689 2023-07-14T16:06:20.736 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:13.711+0000 7f76592e9700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T16:06:20.736 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:14.512+0000 7f76592e9700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 msgr2=0x7f764003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/105803689 2023-07-14T16:06:20.737 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:14.512+0000 7f76592e9700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T16:06:20.737 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:16.113+0000 7f76592e9700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 msgr2=0x7f764003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/105803689 2023-07-14T16:06:20.737 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:16.114+0000 7f76592e9700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T16:06:20.737 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:19.317+0000 7f76592e9700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 msgr2=0x7f764003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/105803689 2023-07-14T16:06:20.737 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:19.317+0000 7f76592e9700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-14T16:06:20.737 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:19.374+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f765002d910 con 0x7f765407ace0 2023-07-14T16:06:20.737 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:19.374+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 msgr2=0x7f764003ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:06:20.738 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:19.374+0000 7f764affd700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:20.738 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.375+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f765001ab70 con 0x7f765407ace0 2023-07-14T16:06:20.738 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.375+0000 7f764affd700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f764003c220 0x7f764003e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:20.738 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.375+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f764003b320 con 0x7f764003c220 2023-07-14T16:06:20.738 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.375+0000 7f76592e9700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f764003c220 0x7f764003e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:20.738 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.376+0000 7f76592e9700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f764003c220 0x7f764003e600 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7644003a10 tx=0x7f76440092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:20.738 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.377+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f764003b320 con 0x7f764003c220 2023-07-14T16:06:20.738 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.384+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f765419c4e0 con 0x7f764003c220 2023-07-14T16:06:20.739 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f764affd700 1 -- 172.21.15.62:0/598794972 <== mgr.14152 v2:172.21.15.62:6800/4048727837 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f765419c4e0 con 0x7f764003c220 2023-07-14T16:06:20.739 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f764003c220 msgr2=0x7f764003e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:20.739 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f764003c220 0x7f764003e600 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7644003a10 tx=0x7f76440092b0 comp rx=0 tx=0).stop 2023-07-14T16:06:20.739 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 msgr2=0x7f765419b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:20.739 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f765419b1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7650000c00 tx=0x7f76500119a0 comp rx=0 tx=0).stop 2023-07-14T16:06:20.739 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 shutdown_connections 2023-07-14T16:06:20.739 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f764003c220 0x7f764003e600 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:20.740 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f765407ace0 0x7f765419b1a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:20.740 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 --2- 172.21.15.62:0/598794972 >> [v2:172.21.15.62:6800/105803689,v1:172.21.15.62:6801/105803689] conn(0x7f7640038760 0x7f764003ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:20.740 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 >> 172.21.15.62:0/598794972 conn(0x7f7654100870 msgr2=0x7f7654101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:20.740 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 shutdown_connections 2023-07-14T16:06:20.740 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:20.385+0000 7f765bd4e700 1 -- 172.21.15.62:0/598794972 wait complete. 2023-07-14T16:06:20.740 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 9 is available 2023-07-14T16:06:20.740 INFO:teuthology.orchestra.run.smithi062.stdout:Generating a dashboard self-signed certificate... 2023-07-14T16:06:20.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T16:06:20.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T16:06:20.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:20 smithi062 ceph-mon[95543]: mgrmap e11: x(active, since 1.0084s) 2023-07-14T16:06:21.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:21 smithi062 ceph-mon[95543]: [14/Jul/2023:16:06:20] ENGINE Bus STARTING 2023-07-14T16:06:21.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:21 smithi062 ceph-mon[95543]: [14/Jul/2023:16:06:20] ENGINE Serving on http://172.21.15.62:8765 2023-07-14T16:06:21.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:21 smithi062 ceph-mon[95543]: [14/Jul/2023:16:06:20] ENGINE Serving on https://172.21.15.62:7150 2023-07-14T16:06:21.718 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:21 smithi062 ceph-mon[95543]: [14/Jul/2023:16:06:20] ENGINE Bus STARTED 2023-07-14T16:06:22.109 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-14T16:06:22.109 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.374+0000 7f1715618700 1 Processor -- start 2023-07-14T16:06:22.109 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.374+0000 7f1715618700 1 -- start start 2023-07-14T16:06:22.110 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.375+0000 7f1715618700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f17100791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:22.110 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.375+0000 7f1715618700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1710079700 con 0x7f171007acc0 2023-07-14T16:06:22.110 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.375+0000 7f170effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f17100791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:22.110 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.375+0000 7f170effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f17100791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47048/0 (socket says 172.21.15.62:47048) 2023-07-14T16:06:22.110 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.375+0000 7f170effd700 1 -- 172.21.15.62:0/4277896693 learned_addr learned my addr 172.21.15.62:0/4277896693 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:22.110 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.376+0000 7f170effd700 1 -- 172.21.15.62:0/4277896693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1710079840 con 0x7f171007acc0 2023-07-14T16:06:22.111 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.376+0000 7f170effd700 1 --2- 172.21.15.62:0/4277896693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f17100791c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17000098a0 tx=0x7f1700009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8ac3be3848d789 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:22.111 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.376+0000 7f170dffb700 1 -- 172.21.15.62:0/4277896693 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1700004160 con 0x7f171007acc0 2023-07-14T16:06:22.111 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.376+0000 7f170dffb700 1 -- 172.21.15.62:0/4277896693 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17000042c0 con 0x7f171007acc0 2023-07-14T16:06:22.111 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.376+0000 7f170dffb700 1 -- 172.21.15.62:0/4277896693 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17000118e0 con 0x7f171007acc0 2023-07-14T16:06:22.111 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.377+0000 7f1715618700 1 -- 172.21.15.62:0/4277896693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 msgr2=0x7f17100791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:22.111 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.377+0000 7f1715618700 1 --2- 172.21.15.62:0/4277896693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f17100791c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17000098a0 tx=0x7f1700009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:22.111 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.377+0000 7f1715618700 1 -- 172.21.15.62:0/4277896693 shutdown_connections 2023-07-14T16:06:22.112 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.377+0000 7f1715618700 1 --2- 172.21.15.62:0/4277896693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f17100791c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:22.112 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.377+0000 7f1715618700 1 -- 172.21.15.62:0/4277896693 >> 172.21.15.62:0/4277896693 conn(0x7f1710100870 msgr2=0x7f1710102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:22.112 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.377+0000 7f1715618700 1 -- 172.21.15.62:0/4277896693 shutdown_connections 2023-07-14T16:06:22.112 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.377+0000 7f1715618700 1 -- 172.21.15.62:0/4277896693 wait complete. 2023-07-14T16:06:22.112 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.378+0000 7f1715618700 1 Processor -- start 2023-07-14T16:06:22.112 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.378+0000 7f1715618700 1 -- start start 2023-07-14T16:06:22.112 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.379+0000 7f1715618700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f171019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:22.113 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.379+0000 7f1715618700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171019b6d0 con 0x7f171007acc0 2023-07-14T16:06:22.113 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.379+0000 7f170effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f171019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:22.113 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.379+0000 7f170effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f171019b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47060/0 (socket says 172.21.15.62:47060) 2023-07-14T16:06:22.113 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.379+0000 7f170effd700 1 -- 172.21.15.62:0/2070320862 learned_addr learned my addr 172.21.15.62:0/2070320862 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:22.113 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.379+0000 7f170effd700 1 -- 172.21.15.62:0/2070320862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1700009550 con 0x7f171007acc0 2023-07-14T16:06:22.113 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.380+0000 7f170effd700 1 --2- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f171019b190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1700003e30 tx=0x7f17000119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:22.113 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.380+0000 7f16f7fff700 1 -- 172.21.15.62:0/2070320862 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17000036a0 con 0x7f171007acc0 2023-07-14T16:06:22.114 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.380+0000 7f16f7fff700 1 -- 172.21.15.62:0/2070320862 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f170001b440 con 0x7f171007acc0 2023-07-14T16:06:22.114 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.380+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171019b8d0 con 0x7f171007acc0 2023-07-14T16:06:22.114 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.380+0000 7f16f7fff700 1 -- 172.21.15.62:0/2070320862 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1700023400 con 0x7f171007acc0 2023-07-14T16:06:22.114 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.380+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171019bcf0 con 0x7f171007acc0 2023-07-14T16:06:22.114 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.381+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171010b550 con 0x7f171007acc0 2023-07-14T16:06:22.114 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.382+0000 7f16f7fff700 1 -- 172.21.15.62:0/2070320862 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f170001b5b0 con 0x7f171007acc0 2023-07-14T16:06:22.114 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.383+0000 7f16f7fff700 1 --2- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16f8038680 0x7f16f803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:22.114 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.383+0000 7f16f7fff700 1 -- 172.21.15.62:0/2070320862 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f170004c620 con 0x7f171007acc0 2023-07-14T16:06:22.115 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.383+0000 7f170e7fc700 1 --2- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16f8038680 0x7f16f803ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:22.115 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.384+0000 7f170e7fc700 1 --2- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16f8038680 0x7f16f803ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1704006fd0 tx=0x7f1704006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:22.115 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.388+0000 7f16f7fff700 1 -- 172.21.15.62:0/2070320862 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f170000bc10 con 0x7f171007acc0 2023-07-14T16:06:22.115 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.430+0000 7f16f7fff700 1 -- 172.21.15.62:0/2070320862 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1700023680 con 0x7f171007acc0 2023-07-14T16:06:22.115 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.617+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f171019eb50 con 0x7f16f8038680 2023-07-14T16:06:22.115 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.718+0000 7f16f7fff700 1 -- 172.21.15.62:0/2070320862 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f171019eb50 con 0x7f16f8038680 2023-07-14T16:06:22.116 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16f8038680 msgr2=0x7f16f803ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:22.116 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 --2- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16f8038680 0x7f16f803ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1704006fd0 tx=0x7f1704006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:22.116 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 msgr2=0x7f171019b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:22.116 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 --2- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f171019b190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1700003e30 tx=0x7f17000119a0 comp rx=0 tx=0).stop 2023-07-14T16:06:22.116 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 shutdown_connections 2023-07-14T16:06:22.116 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 --2- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16f8038680 0x7f16f803ab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:22.116 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 --2- 172.21.15.62:0/2070320862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f171007acc0 0x7f171019b190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:22.117 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 >> 172.21.15.62:0/2070320862 conn(0x7f1710100870 msgr2=0x7f1710101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:22.117 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 shutdown_connections 2023-07-14T16:06:22.117 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:21.721+0000 7f1715618700 1 -- 172.21.15.62:0/2070320862 wait complete. 2023-07-14T16:06:22.117 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial admin user... 2023-07-14T16:06:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:22 smithi062 ceph-mon[95543]: mgrmap e12: x(active, since 2s) 2023-07-14T16:06:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:22 smithi062 ceph-mon[95543]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:23.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$v0sKAVq6yOYeo.IcqRgumeL75R9P8qcfQmw6WyaojV8rwirXBUZkS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689350783, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-14T16:06:23.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.674+0000 7f38597ab700 1 Processor -- start 2023-07-14T16:06:23.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.674+0000 7f38597ab700 1 -- start start 2023-07-14T16:06:23.591 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.674+0000 7f38597ab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:23.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.674+0000 7f38597ab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854105b50 con 0x7f3854105200 2023-07-14T16:06:23.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.675+0000 7f3852ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:23.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.675+0000 7f3852ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47074/0 (socket says 172.21.15.62:47074) 2023-07-14T16:06:23.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.675+0000 7f3852ffd700 1 -- 172.21.15.62:0/1505942473 learned_addr learned my addr 172.21.15.62:0/1505942473 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:23.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.675+0000 7f3852ffd700 1 -- 172.21.15.62:0/1505942473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854105c90 con 0x7f3854105200 2023-07-14T16:06:23.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f3852ffd700 1 --2- 172.21.15.62:0/1505942473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854105610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38440098a0 tx=0x7f3844009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae566cc49ecd774 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:23.592 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f3851ffb700 1 -- 172.21.15.62:0/1505942473 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3844004160 con 0x7f3854105200 2023-07-14T16:06:23.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f3851ffb700 1 -- 172.21.15.62:0/1505942473 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38440042c0 con 0x7f3854105200 2023-07-14T16:06:23.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f3851ffb700 1 -- 172.21.15.62:0/1505942473 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38440118e0 con 0x7f3854105200 2023-07-14T16:06:23.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f38597ab700 1 -- 172.21.15.62:0/1505942473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 msgr2=0x7f3854105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:23.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f38597ab700 1 --2- 172.21.15.62:0/1505942473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854105610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38440098a0 tx=0x7f3844009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:23.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f38597ab700 1 -- 172.21.15.62:0/1505942473 shutdown_connections 2023-07-14T16:06:23.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f38597ab700 1 --2- 172.21.15.62:0/1505942473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854105610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:23.593 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.676+0000 7f38597ab700 1 -- 172.21.15.62:0/1505942473 >> 172.21.15.62:0/1505942473 conn(0x7f3854100870 msgr2=0x7f3854102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:23.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.677+0000 7f38597ab700 1 -- 172.21.15.62:0/1505942473 shutdown_connections 2023-07-14T16:06:23.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.677+0000 7f38597ab700 1 -- 172.21.15.62:0/1505942473 wait complete. 2023-07-14T16:06:23.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.678+0000 7f38597ab700 1 Processor -- start 2023-07-14T16:06:23.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.678+0000 7f38597ab700 1 -- start start 2023-07-14T16:06:23.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.679+0000 7f38597ab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:23.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.679+0000 7f38597ab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854199470 con 0x7f3854105200 2023-07-14T16:06:23.594 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.679+0000 7f3852ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:23.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.679+0000 7f3852ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47086/0 (socket says 172.21.15.62:47086) 2023-07-14T16:06:23.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.679+0000 7f3852ffd700 1 -- 172.21.15.62:0/4262149875 learned_addr learned my addr 172.21.15.62:0/4262149875 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:23.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.679+0000 7f3852ffd700 1 -- 172.21.15.62:0/4262149875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844009550 con 0x7f3854105200 2023-07-14T16:06:23.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.680+0000 7f3852ffd700 1 --2- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854198f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3844003e30 tx=0x7f38440119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:23.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.680+0000 7f383bfff700 1 -- 172.21.15.62:0/4262149875 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38440036a0 con 0x7f3854105200 2023-07-14T16:06:23.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.680+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3854199670 con 0x7f3854105200 2023-07-14T16:06:23.595 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.680+0000 7f383bfff700 1 -- 172.21.15.62:0/4262149875 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f384401b440 con 0x7f3854105200 2023-07-14T16:06:23.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.680+0000 7f383bfff700 1 -- 172.21.15.62:0/4262149875 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3844023400 con 0x7f3854105200 2023-07-14T16:06:23.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.680+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854199a90 con 0x7f3854105200 2023-07-14T16:06:23.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.681+0000 7f383bfff700 1 -- 172.21.15.62:0/4262149875 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f3844003800 con 0x7f3854105200 2023-07-14T16:06:23.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.682+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854061b40 con 0x7f3854105200 2023-07-14T16:06:23.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.682+0000 7f383bfff700 1 --2- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f383c038760 0x7f383c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:23.596 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.682+0000 7f383bfff700 1 -- 172.21.15.62:0/4262149875 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f384404c880 con 0x7f3854105200 2023-07-14T16:06:23.597 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.682+0000 7f38527fc700 1 --2- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f383c038760 0x7f383c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:23.597 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.683+0000 7f38527fc700 1 --2- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f383c038760 0x7f383c03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3848006fd0 tx=0x7f3848006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:23.597 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.687+0000 7f383bfff700 1 -- 172.21.15.62:0/4262149875 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3844019bc0 con 0x7f3854105200 2023-07-14T16:06:23.597 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:22.927+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3854061960 con 0x7f383c038760 2023-07-14T16:06:23.597 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.178+0000 7f383bfff700 1 -- 172.21.15.62:0/4262149875 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f3854061960 con 0x7f383c038760 2023-07-14T16:06:23.597 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f383c038760 msgr2=0x7f383c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:23.597 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 --2- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f383c038760 0x7f383c03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3848006fd0 tx=0x7f3848006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:23.598 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 msgr2=0x7f3854198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:23.598 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 --2- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854198f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3844003e30 tx=0x7f38440119a0 comp rx=0 tx=0).stop 2023-07-14T16:06:23.598 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 shutdown_connections 2023-07-14T16:06:23.598 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 --2- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f383c038760 0x7f383c03ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:23.598 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 --2- 172.21.15.62:0/4262149875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3854105200 0x7f3854198f30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:23.598 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 >> 172.21.15.62:0/4262149875 conn(0x7f3854100870 msgr2=0x7f3854101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:23.598 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 shutdown_connections 2023-07-14T16:06:23.599 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:23.181+0000 7f38597ab700 1 -- 172.21.15.62:0/4262149875 wait complete. 2023-07-14T16:06:23.599 INFO:teuthology.orchestra.run.smithi062.stdout:Fetching dashboard port number... 2023-07-14T16:06:24.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:24 smithi062 ceph-mon[95543]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:24.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:24.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:24.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 8443 2023-07-14T16:06:24.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.142+0000 7f73bce45700 1 Processor -- start 2023-07-14T16:06:24.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.142+0000 7f73bce45700 1 -- start start 2023-07-14T16:06:24.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.142+0000 7f73bce45700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b80791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:24.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.142+0000 7f73bce45700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8079700 con 0x7f73b807acc0 2023-07-14T16:06:24.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.143+0000 7f73b659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b80791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:24.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.143+0000 7f73b659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b80791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41516/0 (socket says 172.21.15.62:41516) 2023-07-14T16:06:24.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.143+0000 7f73b659c700 1 -- 172.21.15.62:0/1465315834 learned_addr learned my addr 172.21.15.62:0/1465315834 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:24.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.143+0000 7f73b659c700 1 -- 172.21.15.62:0/1465315834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b8079840 con 0x7f73b807acc0 2023-07-14T16:06:24.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.143+0000 7f73b659c700 1 --2- 172.21.15.62:0/1465315834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b80791c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73a80098a0 tx=0x7f73a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bf1d5109403c9b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:24.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.143+0000 7f73b559a700 1 -- 172.21.15.62:0/1465315834 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73a8004160 con 0x7f73b807acc0 2023-07-14T16:06:24.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73b559a700 1 -- 172.21.15.62:0/1465315834 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73a80042c0 con 0x7f73b807acc0 2023-07-14T16:06:24.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73b559a700 1 -- 172.21.15.62:0/1465315834 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73a80118e0 con 0x7f73b807acc0 2023-07-14T16:06:24.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73bce45700 1 -- 172.21.15.62:0/1465315834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 msgr2=0x7f73b80791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:24.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73bce45700 1 --2- 172.21.15.62:0/1465315834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b80791c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73a80098a0 tx=0x7f73a8009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:24.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73bce45700 1 -- 172.21.15.62:0/1465315834 shutdown_connections 2023-07-14T16:06:24.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73bce45700 1 --2- 172.21.15.62:0/1465315834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b80791c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:24.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73bce45700 1 -- 172.21.15.62:0/1465315834 >> 172.21.15.62:0/1465315834 conn(0x7f73b8100830 msgr2=0x7f73b8102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:24.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73bce45700 1 -- 172.21.15.62:0/1465315834 shutdown_connections 2023-07-14T16:06:24.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.144+0000 7f73bce45700 1 -- 172.21.15.62:0/1465315834 wait complete. 2023-07-14T16:06:24.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.145+0000 7f73bce45700 1 Processor -- start 2023-07-14T16:06:24.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.146+0000 7f73bce45700 1 -- start start 2023-07-14T16:06:24.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.146+0000 7f73bce45700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:24.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.146+0000 7f73bce45700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b819b600 con 0x7f73b807acc0 2023-07-14T16:06:24.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.146+0000 7f73b659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:24.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.146+0000 7f73b659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b819b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41524/0 (socket says 172.21.15.62:41524) 2023-07-14T16:06:24.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.146+0000 7f73b659c700 1 -- 172.21.15.62:0/2334568312 learned_addr learned my addr 172.21.15.62:0/2334568312 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:24.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.147+0000 7f73b659c700 1 -- 172.21.15.62:0/2334568312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a8009550 con 0x7f73b807acc0 2023-07-14T16:06:24.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.147+0000 7f73b659c700 1 --2- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b819b0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73a8003e30 tx=0x7f73a80119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:24.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.147+0000 7f73a77fe700 1 -- 172.21.15.62:0/2334568312 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73a80036a0 con 0x7f73b807acc0 2023-07-14T16:06:24.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.147+0000 7f73a77fe700 1 -- 172.21.15.62:0/2334568312 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73a801b440 con 0x7f73b807acc0 2023-07-14T16:06:24.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.147+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b819b800 con 0x7f73b807acc0 2023-07-14T16:06:24.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.147+0000 7f73a77fe700 1 -- 172.21.15.62:0/2334568312 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73a8023400 con 0x7f73b807acc0 2023-07-14T16:06:24.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.147+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b819bca0 con 0x7f73b807acc0 2023-07-14T16:06:24.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.148+0000 7f73a77fe700 1 -- 172.21.15.62:0/2334568312 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f73a8003800 con 0x7f73b807acc0 2023-07-14T16:06:24.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.149+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b810b510 con 0x7f73b807acc0 2023-07-14T16:06:24.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.149+0000 7f73a77fe700 1 --2- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73a00387b0 0x7f73a003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:24.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.149+0000 7f73a77fe700 1 -- 172.21.15.62:0/2334568312 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f73a804c7d0 con 0x7f73b807acc0 2023-07-14T16:06:24.874 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.149+0000 7f73b5d9b700 1 --2- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73a00387b0 0x7f73a003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:24.874 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.150+0000 7f73b5d9b700 1 --2- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73a00387b0 0x7f73a003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73ac006fd0 tx=0x7f73ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:24.874 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.155+0000 7f73a77fe700 1 -- 172.21.15.62:0/2334568312 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f73a800bbd0 con 0x7f73b807acc0 2023-07-14T16:06:24.874 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.368+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f73b819eae0 con 0x7f73b807acc0 2023-07-14T16:06:24.874 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.368+0000 7f73a77fe700 1 -- 172.21.15.62:0/2334568312 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f73a800bbd0 con 0x7f73b807acc0 2023-07-14T16:06:24.874 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73a00387b0 msgr2=0x7f73a003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:24.874 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 --2- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73a00387b0 0x7f73a003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73ac006fd0 tx=0x7f73ac006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:24.875 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 msgr2=0x7f73b819b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:24.875 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 --2- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b819b0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73a8003e30 tx=0x7f73a80119a0 comp rx=0 tx=0).stop 2023-07-14T16:06:24.875 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 shutdown_connections 2023-07-14T16:06:24.875 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 --2- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73a00387b0 0x7f73a003ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:24.875 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 --2- 172.21.15.62:0/2334568312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73b807acc0 0x7f73b819b0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:24.875 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 >> 172.21.15.62:0/2334568312 conn(0x7f73b8100830 msgr2=0x7f73b8101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:24.875 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 shutdown_connections 2023-07-14T16:06:24.875 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:24.371+0000 7f73bce45700 1 -- 172.21.15.62:0/2334568312 wait complete. 2023-07-14T16:06:24.888 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T16:06:24.888 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2023-07-14T16:06:24.925 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T16:06:24.926 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2023-07-14T16:06:24.926 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2023-07-14T16:06:24.926 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-14T16:06:24.927 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph Dashboard is now available at: 2023-07-14T16:06:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: URL: https://smithi062.front.sepia.ceph.com:8443/ 2023-07-14T16:06:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: User: admin 2023-07-14T16:06:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: Password: sl0a7adkkm 2023-07-14T16:06:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:24.929 INFO:teuthology.orchestra.run.smithi062.stdout:Saving cluster configuration to /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config directory 2023-07-14T16:06:24.929 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling autotune for osd_memory_target 2023-07-14T16:06:25.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:25 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2334568312' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-14T16:06:26.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.820+0000 7f9aa5a14700 1 Processor -- start 2023-07-14T16:06:26.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.821+0000 7f9aa5a14700 1 -- start start 2023-07-14T16:06:26.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.821+0000 7f9aa5a14700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa0105490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:26.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.821+0000 7f9aa5a14700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa01059d0 con 0x7f9aa0105080 2023-07-14T16:06:26.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.821+0000 7f9aa4a12700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa0105490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:26.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.822+0000 7f9aa4a12700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa0105490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41530/0 (socket says 172.21.15.62:41530) 2023-07-14T16:06:26.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.822+0000 7f9aa4a12700 1 -- 172.21.15.62:0/2492226240 learned_addr learned my addr 172.21.15.62:0/2492226240 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:26.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.822+0000 7f9aa4a12700 1 -- 172.21.15.62:0/2492226240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa0105b10 con 0x7f9aa0105080 2023-07-14T16:06:26.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.822+0000 7f9aa4a12700 1 --2- 172.21.15.62:0/2492226240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa0105490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a940098a0 tx=0x7f9a94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19e96bddefe44d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:26.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.822+0000 7f9a9f7fe700 1 -- 172.21.15.62:0/2492226240 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a94004160 con 0x7f9aa0105080 2023-07-14T16:06:26.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.822+0000 7f9a9f7fe700 1 -- 172.21.15.62:0/2492226240 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a940042c0 con 0x7f9aa0105080 2023-07-14T16:06:26.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.822+0000 7f9a9f7fe700 1 -- 172.21.15.62:0/2492226240 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a940118e0 con 0x7f9aa0105080 2023-07-14T16:06:26.417 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.823+0000 7f9aa5a14700 1 -- 172.21.15.62:0/2492226240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 msgr2=0x7f9aa0105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:26.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.823+0000 7f9aa5a14700 1 --2- 172.21.15.62:0/2492226240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa0105490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a940098a0 tx=0x7f9a94009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:26.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.823+0000 7f9aa5a14700 1 -- 172.21.15.62:0/2492226240 shutdown_connections 2023-07-14T16:06:26.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.823+0000 7f9aa5a14700 1 --2- 172.21.15.62:0/2492226240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa0105490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:26.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.823+0000 7f9aa5a14700 1 -- 172.21.15.62:0/2492226240 >> 172.21.15.62:0/2492226240 conn(0x7f9aa0100710 msgr2=0x7f9aa0102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:26.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.823+0000 7f9aa5a14700 1 -- 172.21.15.62:0/2492226240 shutdown_connections 2023-07-14T16:06:26.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.823+0000 7f9aa5a14700 1 -- 172.21.15.62:0/2492226240 wait complete. 2023-07-14T16:06:26.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.825+0000 7f9aa5a14700 1 Processor -- start 2023-07-14T16:06:26.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.825+0000 7f9aa5a14700 1 -- start start 2023-07-14T16:06:26.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.825+0000 7f9aa5a14700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa007ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:26.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.825+0000 7f9aa5a14700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa007b070 con 0x7f9aa0105080 2023-07-14T16:06:26.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9aa4a12700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa007ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:26.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9aa4a12700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa007ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41536/0 (socket says 172.21.15.62:41536) 2023-07-14T16:06:26.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9aa4a12700 1 -- 172.21.15.62:0/3846787987 learned_addr learned my addr 172.21.15.62:0/3846787987 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:26.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9aa4a12700 1 -- 172.21.15.62:0/3846787987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94009550 con 0x7f9aa0105080 2023-07-14T16:06:26.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9aa4a12700 1 --2- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa007ab30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a94003e30 tx=0x7f9a940119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:26.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9a9dffb700 1 -- 172.21.15.62:0/3846787987 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a940036a0 con 0x7f9aa0105080 2023-07-14T16:06:26.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9a9dffb700 1 -- 172.21.15.62:0/3846787987 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a9401b440 con 0x7f9aa0105080 2023-07-14T16:06:26.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9a9dffb700 1 -- 172.21.15.62:0/3846787987 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a94023400 con 0x7f9aa0105080 2023-07-14T16:06:26.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.826+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa0079180 con 0x7f9aa0105080 2023-07-14T16:06:26.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.827+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa00795a0 con 0x7f9aa0105080 2023-07-14T16:06:26.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.828+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa0061b40 con 0x7f9aa0105080 2023-07-14T16:06:26.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.828+0000 7f9a9dffb700 1 -- 172.21.15.62:0/3846787987 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f9a9401b5b0 con 0x7f9aa0105080 2023-07-14T16:06:26.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.828+0000 7f9a9dffb700 1 --2- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9a900387b0 0x7f9a9003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:26.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.828+0000 7f9a9dffb700 1 -- 172.21.15.62:0/3846787987 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a9404c860 con 0x7f9aa0105080 2023-07-14T16:06:26.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.828+0000 7f9a9ffff700 1 --2- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9a900387b0 0x7f9a9003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:26.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.829+0000 7f9a9ffff700 1 --2- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9a900387b0 0x7f9a9003ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c006fd0 tx=0x7f9a8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:26.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:25.833+0000 7f9a9dffb700 1 -- 172.21.15.62:0/3846787987 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9a94019c30 con 0x7f9aa0105080 2023-07-14T16:06:26.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.038+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f9aa0106f20 con 0x7f9aa0105080 2023-07-14T16:06:26.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.039+0000 7f9a9dffb700 1 -- 172.21.15.62:0/3846787987 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f9a94019840 con 0x7f9aa0105080 2023-07-14T16:06:26.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.041+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9a900387b0 msgr2=0x7f9a9003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:26.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.041+0000 7f9aa5a14700 1 --2- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9a900387b0 0x7f9a9003ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c006fd0 tx=0x7f9a8c006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:26.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.041+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 msgr2=0x7f9aa007ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:26.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.041+0000 7f9aa5a14700 1 --2- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa007ab30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a94003e30 tx=0x7f9a940119a0 comp rx=0 tx=0).stop 2023-07-14T16:06:26.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.041+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 shutdown_connections 2023-07-14T16:06:26.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.041+0000 7f9aa5a14700 1 --2- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9a900387b0 0x7f9a9003ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:26.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.041+0000 7f9aa5a14700 1 --2- 172.21.15.62:0/3846787987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aa0105080 0x7f9aa007ab30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:26.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.041+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 >> 172.21.15.62:0/3846787987 conn(0x7f9aa0100710 msgr2=0x7f9aa0101490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:26.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.042+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 shutdown_connections 2023-07-14T16:06:26.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.042+0000 7f9aa5a14700 1 -- 172.21.15.62:0/3846787987 wait complete. 2023-07-14T16:06:27.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:27 smithi062 ceph-mon[95543]: mgrmap e13: x(active, since 6s) 2023-07-14T16:06:27.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.970+0000 7f46319af700 1 Processor -- start 2023-07-14T16:06:27.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.970+0000 7f46319af700 1 -- start start 2023-07-14T16:06:27.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.970+0000 7f46319af700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:27.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.970+0000 7f46319af700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c079740 con 0x7f462c07ad00 2023-07-14T16:06:27.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.971+0000 7f462affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:27.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.971+0000 7f462affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41542/0 (socket says 172.21.15.62:41542) 2023-07-14T16:06:27.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.971+0000 7f462affd700 1 -- 172.21.15.62:0/812449554 learned_addr learned my addr 172.21.15.62:0/812449554 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:27.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.971+0000 7f462affd700 1 -- 172.21.15.62:0/812449554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c079880 con 0x7f462c07ad00 2023-07-14T16:06:27.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.971+0000 7f462affd700 1 --2- 172.21.15.62:0/812449554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c079200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f461c0098a0 tx=0x7f461c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0003396dab31cc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:27.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f4629ffb700 1 -- 172.21.15.62:0/812449554 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461c004160 con 0x7f462c07ad00 2023-07-14T16:06:27.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f4629ffb700 1 -- 172.21.15.62:0/812449554 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f461c0042c0 con 0x7f462c07ad00 2023-07-14T16:06:27.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f4629ffb700 1 -- 172.21.15.62:0/812449554 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461c0118e0 con 0x7f462c07ad00 2023-07-14T16:06:27.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f46319af700 1 -- 172.21.15.62:0/812449554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 msgr2=0x7f462c079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:27.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f46319af700 1 --2- 172.21.15.62:0/812449554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c079200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f461c0098a0 tx=0x7f461c009bb0 comp rx=0 tx=0).stop 2023-07-14T16:06:27.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f46319af700 1 -- 172.21.15.62:0/812449554 shutdown_connections 2023-07-14T16:06:27.698 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f46319af700 1 --2- 172.21.15.62:0/812449554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c079200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:27.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f46319af700 1 -- 172.21.15.62:0/812449554 >> 172.21.15.62:0/812449554 conn(0x7f462c100830 msgr2=0x7f462c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:27.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f46319af700 1 -- 172.21.15.62:0/812449554 shutdown_connections 2023-07-14T16:06:27.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.972+0000 7f46319af700 1 -- 172.21.15.62:0/812449554 wait complete. 2023-07-14T16:06:27.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.974+0000 7f46319af700 1 Processor -- start 2023-07-14T16:06:27.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.974+0000 7f46319af700 1 -- start start 2023-07-14T16:06:27.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.974+0000 7f46319af700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c19b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:27.699 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.974+0000 7f46319af700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c19b610 con 0x7f462c07ad00 2023-07-14T16:06:27.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.974+0000 7f462affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c19b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:27.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.974+0000 7f462affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c19b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41554/0 (socket says 172.21.15.62:41554) 2023-07-14T16:06:27.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.974+0000 7f462affd700 1 -- 172.21.15.62:0/942544618 learned_addr learned my addr 172.21.15.62:0/942544618 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:27.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.975+0000 7f462affd700 1 -- 172.21.15.62:0/942544618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c009550 con 0x7f462c07ad00 2023-07-14T16:06:27.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.975+0000 7f462affd700 1 --2- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c19b0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f461c003e30 tx=0x7f461c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:27.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.975+0000 7f46309ad700 1 -- 172.21.15.62:0/942544618 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461c0037d0 con 0x7f462c07ad00 2023-07-14T16:06:27.700 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.975+0000 7f46309ad700 1 -- 172.21.15.62:0/942544618 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f461c01b440 con 0x7f462c07ad00 2023-07-14T16:06:27.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.975+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c19b810 con 0x7f462c07ad00 2023-07-14T16:06:27.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.975+0000 7f46309ad700 1 -- 172.21.15.62:0/942544618 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461c023400 con 0x7f462c07ad00 2023-07-14T16:06:27.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.975+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c19bcb0 con 0x7f462c07ad00 2023-07-14T16:06:27.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.976+0000 7f46309ad700 1 -- 172.21.15.62:0/942544618 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f461c003930 con 0x7f462c07ad00 2023-07-14T16:06:27.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.976+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c10b4d0 con 0x7f462c07ad00 2023-07-14T16:06:27.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.977+0000 7f46309ad700 1 --2- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46140387e0 0x7f461403ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:27.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.977+0000 7f46309ad700 1 -- 172.21.15.62:0/942544618 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f461c04c970 con 0x7f462c07ad00 2023-07-14T16:06:27.701 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.977+0000 7f462a7fc700 1 --2- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46140387e0 0x7f461403ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:27.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.978+0000 7f462a7fc700 1 --2- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46140387e0 0x7f461403ac90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4620006fd0 tx=0x7f4620006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:27.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:26.982+0000 7f46309ad700 1 -- 172.21.15.62:0/942544618 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f461c00bdb0 con 0x7f462c07ad00 2023-07-14T16:06:27.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.304+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f462c10b6c0 con 0x7f462c07ad00 2023-07-14T16:06:27.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.309+0000 7f46309ad700 1 -- 172.21.15.62:0/942544618 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f461c00bbd0 con 0x7f462c07ad00 2023-07-14T16:06:27.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.311+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46140387e0 msgr2=0x7f461403ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:27.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.311+0000 7f46319af700 1 --2- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46140387e0 0x7f461403ac90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4620006fd0 tx=0x7f4620006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:27.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.311+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 msgr2=0x7f462c19b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:27.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.311+0000 7f46319af700 1 --2- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c19b0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f461c003e30 tx=0x7f461c011b50 comp rx=0 tx=0).stop 2023-07-14T16:06:27.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.312+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 shutdown_connections 2023-07-14T16:06:27.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.312+0000 7f46319af700 1 --2- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46140387e0 0x7f461403ac90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:27.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.312+0000 7f46319af700 1 --2- 172.21.15.62:0/942544618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c07ad00 0x7f462c19b0d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:27.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.312+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 >> 172.21.15.62:0/942544618 conn(0x7f462c100830 msgr2=0x7f462c101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:27.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.312+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 shutdown_connections 2023-07-14T16:06:27.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2023-07-14T16:06:27.312+0000 7f46319af700 1 -- 172.21.15.62:0/942544618 wait complete. 2023-07-14T16:06:27.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-14T16:06:27.704 INFO:teuthology.orchestra.run.smithi062.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-14T16:06:27.704 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:27.704 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:06:27.704 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:27.704 INFO:teuthology.orchestra.run.smithi062.stdout:Or, if you are only running a single cluster on this host: 2023-07-14T16:06:27.704 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:27.704 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-14T16:06:27.704 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:27.705 INFO:teuthology.orchestra.run.smithi062.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-14T16:06:27.705 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:27.705 INFO:teuthology.orchestra.run.smithi062.stdout: ceph telemetry on 2023-07-14T16:06:27.705 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:27.705 INFO:teuthology.orchestra.run.smithi062.stdout:For more information see: 2023-07-14T16:06:27.705 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:27.705 INFO:teuthology.orchestra.run.smithi062.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-14T16:06:27.705 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:27.706 INFO:teuthology.orchestra.run.smithi062.stdout:Bootstrap complete. 2023-07-14T16:06:27.757 INFO:tasks.cephadm:Fetching config... 2023-07-14T16:06:27.758 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:06:27.758 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-14T16:06:27.801 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-14T16:06:27.801 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:06:27.801 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-14T16:06:27.888 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-14T16:06:27.889 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:06:27.889 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-14T16:06:28.000 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-14T16:06:28.001 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:06:28.001 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-14T16:06:28.086 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-14T16:06:28.087 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7lsLbyjedCWGjthQtNo+U03AkS5b+ezfdaRGTZs7P2L3Mk1KHLb8BPRg33iJIJ+63iWi7gGQnPXIdpG98qK+d5osB+3SVuqX3nFAkWhKrr8Tx06zKWxKxAQtPvIPB8fTfrIIwszb2cX7mAUEpwn+gqOH4ucSArZfHxUR0Ktn8JzfNZiSJWBKAS9ituoVjHNjEDWKSwnqFf4PjhMDbEO1NzU2wyZ0WHzzhZaqTYfpqAL2k1Ivg0ai2IHZWFJQq8rV/akvYpIOhmNbMNrdmB5zVVuXoJTqUpL5XAkJKlAJyI+KXnm1fSXo0z6m0IXAtXEFw1R+/wnuw5Mt6FK1Y8NodQA8iq5e+8eS4OOt/NejGusu9nJUKEZkPzcM4t6QjcSVAbxLg3we9mTUgV8OKN3mgZvldgMUqV5KpAaVtXucyZopkMnwVFOeMXbAucTHw3mEmBNzamA0hizf9FsX/slDA7V7S/xTbnrmVVNvcnUA9rKQAg59AC+FDNAL53rbE4FE= ceph-1356e48a-2260-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T16:06:28.229 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7lsLbyjedCWGjthQtNo+U03AkS5b+ezfdaRGTZs7P2L3Mk1KHLb8BPRg33iJIJ+63iWi7gGQnPXIdpG98qK+d5osB+3SVuqX3nFAkWhKrr8Tx06zKWxKxAQtPvIPB8fTfrIIwszb2cX7mAUEpwn+gqOH4ucSArZfHxUR0Ktn8JzfNZiSJWBKAS9ituoVjHNjEDWKSwnqFf4PjhMDbEO1NzU2wyZ0WHzzhZaqTYfpqAL2k1Ivg0ai2IHZWFJQq8rV/akvYpIOhmNbMNrdmB5zVVuXoJTqUpL5XAkJKlAJyI+KXnm1fSXo0z6m0IXAtXEFw1R+/wnuw5Mt6FK1Y8NodQA8iq5e+8eS4OOt/NejGusu9nJUKEZkPzcM4t6QjcSVAbxLg3we9mTUgV8OKN3mgZvldgMUqV5KpAaVtXucyZopkMnwVFOeMXbAucTHw3mEmBNzamA0hizf9FsX/slDA7V7S/xTbnrmVVNvcnUA9rKQAg59AC+FDNAL53rbE4FE= ceph-1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:06:28.261 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7lsLbyjedCWGjthQtNo+U03AkS5b+ezfdaRGTZs7P2L3Mk1KHLb8BPRg33iJIJ+63iWi7gGQnPXIdpG98qK+d5osB+3SVuqX3nFAkWhKrr8Tx06zKWxKxAQtPvIPB8fTfrIIwszb2cX7mAUEpwn+gqOH4ucSArZfHxUR0Ktn8JzfNZiSJWBKAS9ituoVjHNjEDWKSwnqFf4PjhMDbEO1NzU2wyZ0WHzzhZaqTYfpqAL2k1Ivg0ai2IHZWFJQq8rV/akvYpIOhmNbMNrdmB5zVVuXoJTqUpL5XAkJKlAJyI+KXnm1fSXo0z6m0IXAtXEFw1R+/wnuw5Mt6FK1Y8NodQA8iq5e+8eS4OOt/NejGusu9nJUKEZkPzcM4t6QjcSVAbxLg3we9mTUgV8OKN3mgZvldgMUqV5KpAaVtXucyZopkMnwVFOeMXbAucTHw3mEmBNzamA0hizf9FsX/slDA7V7S/xTbnrmVVNvcnUA9rKQAg59AC+FDNAL53rbE4FE= ceph-1356e48a-2260-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T16:06:28.366 INFO:teuthology.orchestra.run.smithi081.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7lsLbyjedCWGjthQtNo+U03AkS5b+ezfdaRGTZs7P2L3Mk1KHLb8BPRg33iJIJ+63iWi7gGQnPXIdpG98qK+d5osB+3SVuqX3nFAkWhKrr8Tx06zKWxKxAQtPvIPB8fTfrIIwszb2cX7mAUEpwn+gqOH4ucSArZfHxUR0Ktn8JzfNZiSJWBKAS9ituoVjHNjEDWKSwnqFf4PjhMDbEO1NzU2wyZ0WHzzhZaqTYfpqAL2k1Ivg0ai2IHZWFJQq8rV/akvYpIOhmNbMNrdmB5zVVuXoJTqUpL5XAkJKlAJyI+KXnm1fSXo0z6m0IXAtXEFw1R+/wnuw5Mt6FK1Y8NodQA8iq5e+8eS4OOt/NejGusu9nJUKEZkPzcM4t6QjcSVAbxLg3we9mTUgV8OKN3mgZvldgMUqV5KpAaVtXucyZopkMnwVFOeMXbAucTHw3mEmBNzamA0hizf9FsX/slDA7V7S/xTbnrmVVNvcnUA9rKQAg59AC+FDNAL53rbE4FE= ceph-1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:06:28.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:28 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/942544618' entity='client.admin' 2023-07-14T16:06:28.396 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7lsLbyjedCWGjthQtNo+U03AkS5b+ezfdaRGTZs7P2L3Mk1KHLb8BPRg33iJIJ+63iWi7gGQnPXIdpG98qK+d5osB+3SVuqX3nFAkWhKrr8Tx06zKWxKxAQtPvIPB8fTfrIIwszb2cX7mAUEpwn+gqOH4ucSArZfHxUR0Ktn8JzfNZiSJWBKAS9ituoVjHNjEDWKSwnqFf4PjhMDbEO1NzU2wyZ0WHzzhZaqTYfpqAL2k1Ivg0ai2IHZWFJQq8rV/akvYpIOhmNbMNrdmB5zVVuXoJTqUpL5XAkJKlAJyI+KXnm1fSXo0z6m0IXAtXEFw1R+/wnuw5Mt6FK1Y8NodQA8iq5e+8eS4OOt/NejGusu9nJUKEZkPzcM4t6QjcSVAbxLg3we9mTUgV8OKN3mgZvldgMUqV5KpAaVtXucyZopkMnwVFOeMXbAucTHw3mEmBNzamA0hizf9FsX/slDA7V7S/xTbnrmVVNvcnUA9rKQAg59AC+FDNAL53rbE4FE= ceph-1356e48a-2260-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T16:06:28.502 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7lsLbyjedCWGjthQtNo+U03AkS5b+ezfdaRGTZs7P2L3Mk1KHLb8BPRg33iJIJ+63iWi7gGQnPXIdpG98qK+d5osB+3SVuqX3nFAkWhKrr8Tx06zKWxKxAQtPvIPB8fTfrIIwszb2cX7mAUEpwn+gqOH4ucSArZfHxUR0Ktn8JzfNZiSJWBKAS9ituoVjHNjEDWKSwnqFf4PjhMDbEO1NzU2wyZ0WHzzhZaqTYfpqAL2k1Ivg0ai2IHZWFJQq8rV/akvYpIOhmNbMNrdmB5zVVuXoJTqUpL5XAkJKlAJyI+KXnm1fSXo0z6m0IXAtXEFw1R+/wnuw5Mt6FK1Y8NodQA8iq5e+8eS4OOt/NejGusu9nJUKEZkPzcM4t6QjcSVAbxLg3we9mTUgV8OKN3mgZvldgMUqV5KpAaVtXucyZopkMnwVFOeMXbAucTHw3mEmBNzamA0hizf9FsX/slDA7V7S/xTbnrmVVNvcnUA9rKQAg59AC+FDNAL53rbE4FE= ceph-1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:06:28.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-14T16:06:29.578 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:06:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.573+0000 7f0b6d699700 1 -- 172.21.15.62:0/2306799157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 msgr2=0x7f0b680ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.573+0000 7f0b6d699700 1 --2- 172.21.15.62:0/2306799157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 0x7f0b680ff9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009910 tx=0x7f0b58009c20 comp rx=0 tx=0).stop 2023-07-14T16:06:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.573+0000 7f0b6d699700 1 -- 172.21.15.62:0/2306799157 shutdown_connections 2023-07-14T16:06:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.573+0000 7f0b6d699700 1 --2- 172.21.15.62:0/2306799157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 0x7f0b680ff9f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.573+0000 7f0b6d699700 1 -- 172.21.15.62:0/2306799157 >> 172.21.15.62:0/2306799157 conn(0x7f0b680fab90 msgr2=0x7f0b680fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.573+0000 7f0b6d699700 1 -- 172.21.15.62:0/2306799157 shutdown_connections 2023-07-14T16:06:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.573+0000 7f0b6d699700 1 -- 172.21.15.62:0/2306799157 wait complete. 2023-07-14T16:06:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.574+0000 7f0b6d699700 1 Processor -- start 2023-07-14T16:06:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.574+0000 7f0b6d699700 1 -- start start 2023-07-14T16:06:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.574+0000 7f0b6d699700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 0x7f0b68194660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.574+0000 7f0b6d699700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68194ba0 con 0x7f0b680ff5e0 2023-07-14T16:06:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.574+0000 7f0b66ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 0x7f0b68194660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b66ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 0x7f0b68194660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41586/0 (socket says 172.21.15.62:41586) 2023-07-14T16:06:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b66ffd700 1 -- 172.21.15.62:0/4241234651 learned_addr learned my addr 172.21.15.62:0/4241234651 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b66ffd700 1 -- 172.21.15.62:0/4241234651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b580095f0 con 0x7f0b680ff5e0 2023-07-14T16:06:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b66ffd700 1 --2- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 0x7f0b68194660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b58005950 tx=0x7f0b58005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b4ffff700 1 -- 172.21.15.62:0/4241234651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b5800ba90 con 0x7f0b680ff5e0 2023-07-14T16:06:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b4ffff700 1 -- 172.21.15.62:0/4241234651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b5800bbf0 con 0x7f0b680ff5e0 2023-07-14T16:06:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b4ffff700 1 -- 172.21.15.62:0/4241234651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b58017440 con 0x7f0b680ff5e0 2023-07-14T16:06:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b68194da0 con 0x7f0b680ff5e0 2023-07-14T16:06:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.575+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b68195240 con 0x7f0b680ff5e0 2023-07-14T16:06:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.577+0000 7f0b4ffff700 1 -- 172.21.15.62:0/4241234651 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0b5800f460 con 0x7f0b680ff5e0 2023-07-14T16:06:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.577+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6818e060 con 0x7f0b680ff5e0 2023-07-14T16:06:31.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.577+0000 7f0b4ffff700 1 --2- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0b50038800 0x7f0b5003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:31.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.577+0000 7f0b4ffff700 1 -- 172.21.15.62:0/4241234651 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b5804c8a0 con 0x7f0b680ff5e0 2023-07-14T16:06:31.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.577+0000 7f0b667fc700 1 --2- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0b50038800 0x7f0b5003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:31.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.578+0000 7f0b667fc700 1 --2- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0b50038800 0x7f0b5003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006fd0 tx=0x7f0b5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:31.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.582+0000 7f0b4ffff700 1 -- 172.21.15.62:0/4241234651 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0b5802ad30 con 0x7f0b680ff5e0 2023-07-14T16:06:31.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.787+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0b6802c850 con 0x7f0b680ff5e0 2023-07-14T16:06:31.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.790+0000 7f0b4ffff700 1 -- 172.21.15.62:0/4241234651 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0b58027070 con 0x7f0b680ff5e0 2023-07-14T16:06:31.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0b50038800 msgr2=0x7f0b5003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:31.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 --2- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0b50038800 0x7f0b5003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006fd0 tx=0x7f0b5c006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:31.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 msgr2=0x7f0b68194660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:31.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 --2- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 0x7f0b68194660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b58005950 tx=0x7f0b58005810 comp rx=0 tx=0).stop 2023-07-14T16:06:31.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 shutdown_connections 2023-07-14T16:06:31.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 --2- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0b50038800 0x7f0b5003acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:31.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 --2- 172.21.15.62:0/4241234651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b680ff5e0 0x7f0b68194660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:31.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 >> 172.21.15.62:0/4241234651 conn(0x7f0b680fab90 msgr2=0x7f0b680fb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:31.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 shutdown_connections 2023-07-14T16:06:31.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:31.793+0000 7f0b6d699700 1 -- 172.21.15.62:0/4241234651 wait complete. 2023-07-14T16:06:32.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:32.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:32.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:06:32.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:32.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:32.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:32.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/4241234651' entity='client.admin' 2023-07-14T16:06:32.362 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:32.362 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:32.363 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-14T16:06:32.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-14T16:06:33.091 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:06:34.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.490+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1106724693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 msgr2=0x7fe5180729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.490+0000 7fe51cdc5700 1 --2- 172.21.15.62:0/1106724693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 0x7fe5180729a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe508009910 tx=0x7fe508009c20 comp rx=0 tx=0).stop 2023-07-14T16:06:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.490+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1106724693 shutdown_connections 2023-07-14T16:06:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.490+0000 7fe51cdc5700 1 --2- 172.21.15.62:0/1106724693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 0x7fe5180729a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.490+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1106724693 >> 172.21.15.62:0/1106724693 conn(0x7fe5180fab90 msgr2=0x7fe5180fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.490+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1106724693 shutdown_connections 2023-07-14T16:06:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.490+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1106724693 wait complete. 2023-07-14T16:06:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.491+0000 7fe51cdc5700 1 Processor -- start 2023-07-14T16:06:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.491+0000 7fe51cdc5700 1 -- start start 2023-07-14T16:06:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.492+0000 7fe51cdc5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 0x7fe518196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.492+0000 7fe51cdc5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe518196da0 con 0x7fe518074540 2023-07-14T16:06:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.492+0000 7fe51659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 0x7fe518196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.492+0000 7fe51659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 0x7fe518196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53148/0 (socket says 172.21.15.62:53148) 2023-07-14T16:06:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.492+0000 7fe51659c700 1 -- 172.21.15.62:0/1555322755 learned_addr learned my addr 172.21.15.62:0/1555322755 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.492+0000 7fe51659c700 1 -- 172.21.15.62:0/1555322755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080095f0 con 0x7fe518074540 2023-07-14T16:06:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.493+0000 7fe51659c700 1 --2- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 0x7fe518196860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe508005760 tx=0x7fe508004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.493+0000 7fe5077fe700 1 -- 172.21.15.62:0/1555322755 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe50800ba90 con 0x7fe518074540 2023-07-14T16:06:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.493+0000 7fe5077fe700 1 -- 172.21.15.62:0/1555322755 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe50800bbf0 con 0x7fe518074540 2023-07-14T16:06:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.493+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe518196fa0 con 0x7fe518074540 2023-07-14T16:06:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.493+0000 7fe5077fe700 1 -- 172.21.15.62:0/1555322755 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe508017440 con 0x7fe518074540 2023-07-14T16:06:34.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.493+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe518197440 con 0x7fe518074540 2023-07-14T16:06:34.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.494+0000 7fe5077fe700 1 -- 172.21.15.62:0/1555322755 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fe50800f460 con 0x7fe518074540 2023-07-14T16:06:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.494+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe518190260 con 0x7fe518074540 2023-07-14T16:06:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.494+0000 7fe5077fe700 1 --2- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe500038800 0x7fe50003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.494+0000 7fe5077fe700 1 -- 172.21.15.62:0/1555322755 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe50804c9a0 con 0x7fe518074540 2023-07-14T16:06:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.495+0000 7fe515d9b700 1 --2- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe500038800 0x7fe50003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.495+0000 7fe515d9b700 1 --2- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe500038800 0x7fe50003acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe50c006fd0 tx=0x7fe50c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:34.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.500+0000 7fe5077fe700 1 -- 172.21.15.62:0/1555322755 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe50802ad30 con 0x7fe518074540 2023-07-14T16:06:34.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.714+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5180609e0 con 0x7fe500038800 2023-07-14T16:06:34.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.718+0000 7fe5077fe700 1 -- 172.21.15.62:0/1555322755 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe5180609e0 con 0x7fe500038800 2023-07-14T16:06:34.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.720+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe500038800 msgr2=0x7fe50003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:34.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.720+0000 7fe51cdc5700 1 --2- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe500038800 0x7fe50003acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe50c006fd0 tx=0x7fe50c006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.720+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 msgr2=0x7fe518196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.720+0000 7fe51cdc5700 1 --2- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 0x7fe518196860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe508005760 tx=0x7fe508004f20 comp rx=0 tx=0).stop 2023-07-14T16:06:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.720+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 shutdown_connections 2023-07-14T16:06:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.721+0000 7fe51cdc5700 1 --2- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe500038800 0x7fe50003acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.721+0000 7fe51cdc5700 1 --2- 172.21.15.62:0/1555322755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe518074540 0x7fe518196860 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.721+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 >> 172.21.15.62:0/1555322755 conn(0x7fe5180fab90 msgr2=0x7fe5180fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.721+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 shutdown_connections 2023-07-14T16:06:34.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:34.721+0000 7fe51cdc5700 1 -- 172.21.15.62:0/1555322755 wait complete. 2023-07-14T16:06:35.178 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi081 2023-07-14T16:06:35.179 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:06:35.180 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.conf 2023-07-14T16:06:35.228 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:06:35.228 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:06:35.313 INFO:tasks.cephadm:Adding host smithi081 to orchestrator... 2023-07-14T16:06:35.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch host add smithi081 2023-07-14T16:06:36.065 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:06:36.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:35 smithi062 ceph-mon[95543]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:36.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:35 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:36.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:35 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:36.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:35 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:06:36.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:35 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:06:36.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:35 smithi062 ceph-mon[95543]: Updating smithi062:/etc/ceph/ceph.conf 2023-07-14T16:06:36.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:35 smithi062 ceph-mon[95543]: Updating smithi062:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:06:37.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:36 smithi062 ceph-mon[95543]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:06:37.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:36 smithi062 ceph-mon[95543]: Updating smithi062:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T16:06:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.595+0000 7f200a34b700 1 -- 172.21.15.62:0/3672048077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 msgr2=0x7f2004105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.595+0000 7f200a34b700 1 --2- 172.21.15.62:0/3672048077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 0x7f2004105a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009910 tx=0x7f1ff4009c20 comp rx=0 tx=0).stop 2023-07-14T16:06:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.595+0000 7f200a34b700 1 -- 172.21.15.62:0/3672048077 shutdown_connections 2023-07-14T16:06:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.595+0000 7f200a34b700 1 --2- 172.21.15.62:0/3672048077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 0x7f2004105a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.595+0000 7f200a34b700 1 -- 172.21.15.62:0/3672048077 >> 172.21.15.62:0/3672048077 conn(0x7f20040fb130 msgr2=0x7f20040fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.595+0000 7f200a34b700 1 -- 172.21.15.62:0/3672048077 shutdown_connections 2023-07-14T16:06:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.595+0000 7f200a34b700 1 -- 172.21.15.62:0/3672048077 wait complete. 2023-07-14T16:06:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.596+0000 7f200a34b700 1 Processor -- start 2023-07-14T16:06:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.596+0000 7f200a34b700 1 -- start start 2023-07-14T16:06:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.597+0000 7f200a34b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 0x7f2004072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.597+0000 7f200a34b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2004074790 con 0x7f2004105630 2023-07-14T16:06:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.597+0000 7f2003fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 0x7f2004072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.597+0000 7f2003fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 0x7f2004072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53166/0 (socket says 172.21.15.62:53166) 2023-07-14T16:06:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.597+0000 7f2003fff700 1 -- 172.21.15.62:0/374294570 learned_addr learned my addr 172.21.15.62:0/374294570 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.597+0000 7f2003fff700 1 -- 172.21.15.62:0/374294570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff40095f0 con 0x7f2004105630 2023-07-14T16:06:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.598+0000 7f2003fff700 1 --2- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 0x7f2004072810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4004f40 tx=0x7f1ff4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.598+0000 7f20017fa700 1 -- 172.21.15.62:0/374294570 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ff401d070 con 0x7f2004105630 2023-07-14T16:06:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.598+0000 7f20017fa700 1 -- 172.21.15.62:0/374294570 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ff40044b0 con 0x7f2004105630 2023-07-14T16:06:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.598+0000 7f20017fa700 1 -- 172.21.15.62:0/374294570 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ff4003860 con 0x7f2004105630 2023-07-14T16:06:37.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.598+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2004072d50 con 0x7f2004105630 2023-07-14T16:06:37.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.598+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2004073250 con 0x7f2004105630 2023-07-14T16:06:37.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.599+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20041048d0 con 0x7f2004105630 2023-07-14T16:06:37.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.599+0000 7f20017fa700 1 -- 172.21.15.62:0/374294570 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f1ff400b960 con 0x7f2004105630 2023-07-14T16:06:37.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.600+0000 7f20017fa700 1 --2- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec038850 0x7f1fec03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:37.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.600+0000 7f20017fa700 1 -- 172.21.15.62:0/374294570 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1ff404c6f0 con 0x7f2004105630 2023-07-14T16:06:37.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.600+0000 7f20037fe700 1 --2- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec038850 0x7f1fec03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:37.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.601+0000 7f20037fe700 1 --2- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec038850 0x7f1fec03ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8006fd0 tx=0x7f1ff8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:37.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.605+0000 7f20017fa700 1 -- 172.21.15.62:0/374294570 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1ff402a540 con 0x7f2004105630 2023-07-14T16:06:37.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:37.813+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}) v1 -- 0x7f2004073d40 con 0x7f1fec038850 2023-07-14T16:06:38.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:38.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:38.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:39.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:38 smithi062 ceph-mon[95543]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:39.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:38 smithi062 ceph-mon[95543]: Deploying cephadm binary to smithi081 2023-07-14T16:06:40.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:40 smithi062 ceph-mon[95543]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:41.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.028+0000 7f20017fa700 1 -- 172.21.15.62:0/374294570 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f2004073d40 con 0x7f1fec038850 2023-07-14T16:06:41.005 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi081' with addr '172.21.15.81' 2023-07-14T16:06:41.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec038850 msgr2=0x7f1fec03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 --2- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec038850 0x7f1fec03ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8006fd0 tx=0x7f1ff8006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 msgr2=0x7f2004072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 --2- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 0x7f2004072810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4004f40 tx=0x7f1ff4005020 comp rx=0 tx=0).stop 2023-07-14T16:06:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 shutdown_connections 2023-07-14T16:06:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 --2- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec038850 0x7f1fec03ad00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:41.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 --2- 172.21.15.62:0/374294570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2004105630 0x7f2004072810 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:41.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 >> 172.21.15.62:0/374294570 conn(0x7f20040fb130 msgr2=0x7f20040fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:41.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 shutdown_connections 2023-07-14T16:06:41.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:41.031+0000 7f200a34b700 1 -- 172.21.15.62:0/374294570 wait complete. 2023-07-14T16:06:41.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T16:06:42.183 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:06:42.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:42.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:42 smithi062 ceph-mon[95543]: Added host smithi081 2023-07-14T16:06:42.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:42.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:42 smithi062 ceph-mon[95543]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:42.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.745+0000 7f8f6f6df700 1 -- 172.21.15.62:0/217621762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 msgr2=0x7f8f681059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.745+0000 7f8f6f6df700 1 --2- 172.21.15.62:0/217621762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 0x7f8f681059a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009910 tx=0x7f8f64009c20 comp rx=0 tx=0).stop 2023-07-14T16:06:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.745+0000 7f8f6f6df700 1 -- 172.21.15.62:0/217621762 shutdown_connections 2023-07-14T16:06:43.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.745+0000 7f8f6f6df700 1 --2- 172.21.15.62:0/217621762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 0x7f8f681059a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:43.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.745+0000 7f8f6f6df700 1 -- 172.21.15.62:0/217621762 >> 172.21.15.62:0/217621762 conn(0x7f8f680742b0 msgr2=0x7f8f680746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:43.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.745+0000 7f8f6f6df700 1 -- 172.21.15.62:0/217621762 shutdown_connections 2023-07-14T16:06:43.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.745+0000 7f8f6f6df700 1 -- 172.21.15.62:0/217621762 wait complete. 2023-07-14T16:06:43.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.746+0000 7f8f6f6df700 1 Processor -- start 2023-07-14T16:06:43.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.746+0000 7f8f6f6df700 1 -- start start 2023-07-14T16:06:43.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.747+0000 7f8f6f6df700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 0x7f8f68194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:43.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.747+0000 7f8f6f6df700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f68194eb0 con 0x7f8f681055d0 2023-07-14T16:06:43.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.747+0000 7f8f6d47b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 0x7f8f68194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:43.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.747+0000 7f8f6d47b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 0x7f8f68194970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40348/0 (socket says 172.21.15.62:40348) 2023-07-14T16:06:43.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.747+0000 7f8f6d47b700 1 -- 172.21.15.62:0/2322396614 learned_addr learned my addr 172.21.15.62:0/2322396614 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:43.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.747+0000 7f8f6d47b700 1 -- 172.21.15.62:0/2322396614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f640095f0 con 0x7f8f681055d0 2023-07-14T16:06:43.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.747+0000 7f8f6d47b700 1 --2- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 0x7f8f68194970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f64006010 tx=0x7f8f64004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.748+0000 7f8f5e7fc700 1 -- 172.21.15.62:0/2322396614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f6401d070 con 0x7f8f681055d0 2023-07-14T16:06:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.748+0000 7f8f5e7fc700 1 -- 172.21.15.62:0/2322396614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f6400bdf0 con 0x7f8f681055d0 2023-07-14T16:06:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.748+0000 7f8f5e7fc700 1 -- 172.21.15.62:0/2322396614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f64022d80 con 0x7f8f681055d0 2023-07-14T16:06:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.748+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f681950b0 con 0x7f8f681055d0 2023-07-14T16:06:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.748+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f68195550 con 0x7f8f681055d0 2023-07-14T16:06:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.749+0000 7f8f5e7fc700 1 -- 172.21.15.62:0/2322396614 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8f64022570 con 0x7f8f681055d0 2023-07-14T16:06:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.749+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f68061b40 con 0x7f8f681055d0 2023-07-14T16:06:43.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.749+0000 7f8f5e7fc700 1 --2- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8f540387b0 0x7f8f5403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:43.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.749+0000 7f8f5e7fc700 1 -- 172.21.15.62:0/2322396614 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8f6404c960 con 0x7f8f681055d0 2023-07-14T16:06:43.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.750+0000 7f8f6cc7a700 1 --2- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8f540387b0 0x7f8f5403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:43.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.750+0000 7f8f6cc7a700 1 --2- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8f540387b0 0x7f8f5403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8f58006fd0 tx=0x7f8f58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:43.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.755+0000 7f8f5e7fc700 1 -- 172.21.15.62:0/2322396614 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8f6402bba0 con 0x7f8f681055d0 2023-07-14T16:06:43.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:43 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:43.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.968+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f68104430 con 0x7f8f540387b0 2023-07-14T16:06:43.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.969+0000 7f8f5e7fc700 1 -- 172.21.15.62:0/2322396614 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f8f68104430 con 0x7f8f540387b0 2023-07-14T16:06:43.946 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:43.946 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}] 2023-07-14T16:06:43.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8f540387b0 msgr2=0x7f8f5403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:43.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 --2- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8f540387b0 0x7f8f5403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8f58006fd0 tx=0x7f8f58006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:43.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 msgr2=0x7f8f68194970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:43.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 --2- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 0x7f8f68194970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f64006010 tx=0x7f8f64004ff0 comp rx=0 tx=0).stop 2023-07-14T16:06:43.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 shutdown_connections 2023-07-14T16:06:43.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 --2- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8f540387b0 0x7f8f5403ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:43.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 --2- 172.21.15.62:0/2322396614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f681055d0 0x7f8f68194970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:43.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 >> 172.21.15.62:0/2322396614 conn(0x7f8f680742b0 msgr2=0x7f8f68106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:43.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 shutdown_connections 2023-07-14T16:06:43.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:43.972+0000 7f8f6f6df700 1 -- 172.21.15.62:0/2322396614 wait complete. 2023-07-14T16:06:44.461 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2023-07-14T16:06:44.461 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:06:44.461 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2023-07-14T16:06:44.510 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:06:44.511 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:06:44.595 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2023-07-14T16:06:44.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch host add smithi164 2023-07-14T16:06:44.658 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:44 smithi062 ceph-mon[95543]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:44.658 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:45.321 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:06:45.722 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:45 smithi062 ceph-mon[95543]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:06:46.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.760+0000 7fd315453700 1 -- 172.21.15.62:0/3449857256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 msgr2=0x7fd310072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:46.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.760+0000 7fd315453700 1 --2- 172.21.15.62:0/3449857256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 0x7fd310072c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd300009910 tx=0x7fd300009c20 comp rx=0 tx=0).stop 2023-07-14T16:06:46.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.760+0000 7fd315453700 1 -- 172.21.15.62:0/3449857256 shutdown_connections 2023-07-14T16:06:46.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.760+0000 7fd315453700 1 --2- 172.21.15.62:0/3449857256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 0x7fd310072c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:46.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.760+0000 7fd315453700 1 -- 172.21.15.62:0/3449857256 >> 172.21.15.62:0/3449857256 conn(0x7fd3100fb130 msgr2=0x7fd3100fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:46.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.760+0000 7fd315453700 1 -- 172.21.15.62:0/3449857256 shutdown_connections 2023-07-14T16:06:46.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.760+0000 7fd315453700 1 -- 172.21.15.62:0/3449857256 wait complete. 2023-07-14T16:06:46.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.761+0000 7fd315453700 1 Processor -- start 2023-07-14T16:06:46.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.761+0000 7fd315453700 1 -- start start 2023-07-14T16:06:46.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.762+0000 7fd315453700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 0x7fd31019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:46.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.762+0000 7fd315453700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31019ce90 con 0x7fd3100728b0 2023-07-14T16:06:46.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.762+0000 7fd30effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 0x7fd31019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:46.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.762+0000 7fd30effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 0x7fd31019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40364/0 (socket says 172.21.15.62:40364) 2023-07-14T16:06:46.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.762+0000 7fd30effd700 1 -- 172.21.15.62:0/184231420 learned_addr learned my addr 172.21.15.62:0/184231420 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:46.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.762+0000 7fd30effd700 1 -- 172.21.15.62:0/184231420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000095f0 con 0x7fd3100728b0 2023-07-14T16:06:46.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.763+0000 7fd30effd700 1 --2- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 0x7fd31019c950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd300004e90 tx=0x7fd300004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:46.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.763+0000 7fd2f7fff700 1 -- 172.21.15.62:0/184231420 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd30001d070 con 0x7fd3100728b0 2023-07-14T16:06:46.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.763+0000 7fd2f7fff700 1 -- 172.21.15.62:0/184231420 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3000044b0 con 0x7fd3100728b0 2023-07-14T16:06:46.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.763+0000 7fd2f7fff700 1 -- 172.21.15.62:0/184231420 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3000038f0 con 0x7fd3100728b0 2023-07-14T16:06:46.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.763+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31019d090 con 0x7fd3100728b0 2023-07-14T16:06:46.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.763+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31019d450 con 0x7fd3100728b0 2023-07-14T16:06:46.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.764+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd310061b40 con 0x7fd3100728b0 2023-07-14T16:06:46.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.764+0000 7fd2f7fff700 1 -- 172.21.15.62:0/184231420 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd30000b960 con 0x7fd3100728b0 2023-07-14T16:06:46.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.765+0000 7fd2f7fff700 1 --2- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2f8038800 0x7fd2f803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:46.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.765+0000 7fd2f7fff700 1 -- 172.21.15.62:0/184231420 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd30004c7e0 con 0x7fd3100728b0 2023-07-14T16:06:46.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.765+0000 7fd30e7fc700 1 --2- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2f8038800 0x7fd2f803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:46.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.765+0000 7fd30e7fc700 1 --2- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2f8038800 0x7fd2f803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:46.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.770+0000 7fd2f7fff700 1 -- 172.21.15.62:0/184231420 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd30002a930 con 0x7fd3100728b0 2023-07-14T16:06:46.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:46 smithi062 ceph-mon[95543]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:46.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:46.980+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}) v1 -- 0x7fd310108c80 con 0x7fd2f8038800 2023-07-14T16:06:48.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:06:48.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:48.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:48.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:48.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:48.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:06:48.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:06:48.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:06:48.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: Updating smithi081:/etc/ceph/ceph.conf 2023-07-14T16:06:48.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:48.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:48 smithi062 ceph-mon[95543]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:06:49.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:49 smithi062 ceph-mon[95543]: Deploying cephadm binary to smithi164 2023-07-14T16:06:49.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:49 smithi062 ceph-mon[95543]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:06:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.293+0000 7fd2f7fff700 1 -- 172.21.15.62:0/184231420 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd310108c80 con 0x7fd2f8038800 2023-07-14T16:06:50.270 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi164' with addr '172.21.15.164' 2023-07-14T16:06:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.295+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2f8038800 msgr2=0x7fd2f803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.295+0000 7fd315453700 1 --2- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2f8038800 0x7fd2f803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.295+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 msgr2=0x7fd31019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.295+0000 7fd315453700 1 --2- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 0x7fd31019c950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd300004e90 tx=0x7fd300004f70 comp rx=0 tx=0).stop 2023-07-14T16:06:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.296+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 shutdown_connections 2023-07-14T16:06:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.296+0000 7fd315453700 1 --2- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2f8038800 0x7fd2f803acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.296+0000 7fd315453700 1 --2- 172.21.15.62:0/184231420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3100728b0 0x7fd31019c950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.296+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 >> 172.21.15.62:0/184231420 conn(0x7fd3100fb130 msgr2=0x7fd310106130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.296+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 shutdown_connections 2023-07-14T16:06:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:50.296+0000 7fd315453700 1 -- 172.21.15.62:0/184231420 wait complete. 2023-07-14T16:06:50.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:50 smithi062 ceph-mon[95543]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T16:06:50.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:50 smithi062 ceph-mon[95543]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:50.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:50.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:50.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:50.710 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T16:06:51.445 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:06:51.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:51 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:51.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:51 smithi062 ceph-mon[95543]: Added host smithi164 2023-07-14T16:06:51.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:51 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:51.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:51 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:52.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.959+0000 7f30feec9700 1 -- 172.21.15.62:0/1941127841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 msgr2=0x7f30f8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:52.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.959+0000 7f30feec9700 1 --2- 172.21.15.62:0/1941127841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 0x7f30f8072c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009910 tx=0x7f30ec009c20 comp rx=0 tx=0).stop 2023-07-14T16:06:52.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.959+0000 7f30feec9700 1 -- 172.21.15.62:0/1941127841 shutdown_connections 2023-07-14T16:06:52.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.959+0000 7f30feec9700 1 --2- 172.21.15.62:0/1941127841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 0x7f30f8072c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:52.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.959+0000 7f30feec9700 1 -- 172.21.15.62:0/1941127841 >> 172.21.15.62:0/1941127841 conn(0x7f30f80fb130 msgr2=0x7f30f80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:52.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.960+0000 7f30feec9700 1 -- 172.21.15.62:0/1941127841 shutdown_connections 2023-07-14T16:06:52.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.960+0000 7f30feec9700 1 -- 172.21.15.62:0/1941127841 wait complete. 2023-07-14T16:06:52.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.961+0000 7f30feec9700 1 Processor -- start 2023-07-14T16:06:52.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.961+0000 7f30feec9700 1 -- start start 2023-07-14T16:06:52.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.961+0000 7f30feec9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 0x7f30f819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:52.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.961+0000 7f30feec9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f819ce90 con 0x7f30f80728b0 2023-07-14T16:06:52.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.961+0000 7f30fcc65700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 0x7f30f819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:52.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.961+0000 7f30fcc65700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 0x7f30f819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54088/0 (socket says 172.21.15.62:54088) 2023-07-14T16:06:52.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.961+0000 7f30fcc65700 1 -- 172.21.15.62:0/926246161 learned_addr learned my addr 172.21.15.62:0/926246161 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:52.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.962+0000 7f30fcc65700 1 -- 172.21.15.62:0/926246161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec0095f0 con 0x7f30f80728b0 2023-07-14T16:06:52.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.962+0000 7f30fcc65700 1 --2- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 0x7f30f819c950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f30ec004e90 tx=0x7f30ec004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:52.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.962+0000 7f30f5ffb700 1 -- 172.21.15.62:0/926246161 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30ec01d070 con 0x7f30f80728b0 2023-07-14T16:06:52.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.962+0000 7f30f5ffb700 1 -- 172.21.15.62:0/926246161 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30ec0044b0 con 0x7f30f80728b0 2023-07-14T16:06:52.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.962+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f819d090 con 0x7f30f80728b0 2023-07-14T16:06:52.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.962+0000 7f30f5ffb700 1 -- 172.21.15.62:0/926246161 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30ec0038f0 con 0x7f30f80728b0 2023-07-14T16:06:52.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.962+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f819d450 con 0x7f30f80728b0 2023-07-14T16:06:52.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.964+0000 7f30f5ffb700 1 -- 172.21.15.62:0/926246161 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f30ec00b960 con 0x7f30f80728b0 2023-07-14T16:06:52.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.964+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f8061b40 con 0x7f30f80728b0 2023-07-14T16:06:52.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.964+0000 7f30f5ffb700 1 --2- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30e80387b0 0x7f30e803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:52.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.964+0000 7f30f5ffb700 1 -- 172.21.15.62:0/926246161 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f30ec04c760 con 0x7f30f80728b0 2023-07-14T16:06:52.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.964+0000 7f30f7fff700 1 --2- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30e80387b0 0x7f30e803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:52.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.965+0000 7f30f7fff700 1 --2- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30e80387b0 0x7f30e803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f30e4006fd0 tx=0x7f30e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:52.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:52.969+0000 7f30f5ffb700 1 -- 172.21.15.62:0/926246161 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f30ec02a930 con 0x7f30f80728b0 2023-07-14T16:06:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:52 smithi062 ceph-mon[95543]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:52 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:53.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.186+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30f8108ca0 con 0x7f30e80387b0 2023-07-14T16:06:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.187+0000 7f30f5ffb700 1 -- 172.21.15.62:0/926246161 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f30f8108ca0 con 0x7f30e80387b0 2023-07-14T16:06:53.164 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:06:53.164 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2023-07-14T16:06:53.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.189+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30e80387b0 msgr2=0x7f30e803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:53.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.189+0000 7f30feec9700 1 --2- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30e80387b0 0x7f30e803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f30e4006fd0 tx=0x7f30e4006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:53.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.189+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 msgr2=0x7f30f819c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:53.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.189+0000 7f30feec9700 1 --2- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 0x7f30f819c950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f30ec004e90 tx=0x7f30ec004f70 comp rx=0 tx=0).stop 2023-07-14T16:06:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.189+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 shutdown_connections 2023-07-14T16:06:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.190+0000 7f30feec9700 1 --2- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30e80387b0 0x7f30e803ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.190+0000 7f30feec9700 1 --2- 172.21.15.62:0/926246161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30f80728b0 0x7f30f819c950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.190+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 >> 172.21.15.62:0/926246161 conn(0x7f30f80fb130 msgr2=0x7f30f8106130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.190+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 shutdown_connections 2023-07-14T16:06:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:53.190+0000 7f30feec9700 1 -- 172.21.15.62:0/926246161 wait complete. 2023-07-14T16:06:53.622 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-14T16:06:53.622 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd crush tunables default 2023-07-14T16:06:54.349 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:06:54.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:54.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:54 smithi062 ceph-mon[95543]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:06:54.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:54 smithi062 ceph-mon[95543]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.893+0000 7f3daea96700 1 -- 172.21.15.62:0/2454853340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 msgr2=0x7f3da8105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.893+0000 7f3daea96700 1 --2- 172.21.15.62:0/2454853340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 0x7f3da8105a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009910 tx=0x7f3d9c009c20 comp rx=0 tx=0).stop 2023-07-14T16:06:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.893+0000 7f3daea96700 1 -- 172.21.15.62:0/2454853340 shutdown_connections 2023-07-14T16:06:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.893+0000 7f3daea96700 1 --2- 172.21.15.62:0/2454853340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 0x7f3da8105a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:55.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.893+0000 7f3daea96700 1 -- 172.21.15.62:0/2454853340 >> 172.21.15.62:0/2454853340 conn(0x7f3da80fb130 msgr2=0x7f3da80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:55.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.893+0000 7f3daea96700 1 -- 172.21.15.62:0/2454853340 shutdown_connections 2023-07-14T16:06:55.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.893+0000 7f3daea96700 1 -- 172.21.15.62:0/2454853340 wait complete. 2023-07-14T16:06:55.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.894+0000 7f3daea96700 1 Processor -- start 2023-07-14T16:06:55.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.894+0000 7f3daea96700 1 -- start start 2023-07-14T16:06:55.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.895+0000 7f3daea96700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 0x7f3da819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:55.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.895+0000 7f3daea96700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da819cd80 con 0x7f3da8105630 2023-07-14T16:06:55.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.895+0000 7f3dac832700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 0x7f3da819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:55.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.895+0000 7f3dac832700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 0x7f3da819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54108/0 (socket says 172.21.15.62:54108) 2023-07-14T16:06:55.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.895+0000 7f3dac832700 1 -- 172.21.15.62:0/3437251395 learned_addr learned my addr 172.21.15.62:0/3437251395 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:06:55.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.895+0000 7f3dac832700 1 -- 172.21.15.62:0/3437251395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0095f0 con 0x7f3da8105630 2023-07-14T16:06:55.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.895+0000 7f3dac832700 1 --2- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 0x7f3da819c840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c006010 tx=0x7f3d9c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:55.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.896+0000 7f3da5ffb700 1 -- 172.21.15.62:0/3437251395 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d9c01d070 con 0x7f3da8105630 2023-07-14T16:06:55.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.896+0000 7f3da5ffb700 1 -- 172.21.15.62:0/3437251395 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d9c00bdf0 con 0x7f3da8105630 2023-07-14T16:06:55.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.896+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da819cfe0 con 0x7f3da8105630 2023-07-14T16:06:55.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.896+0000 7f3da5ffb700 1 -- 172.21.15.62:0/3437251395 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d9c022d80 con 0x7f3da8105630 2023-07-14T16:06:55.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.896+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da81968c0 con 0x7f3da8105630 2023-07-14T16:06:55.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.897+0000 7f3da5ffb700 1 -- 172.21.15.62:0/3437251395 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f3d9c022570 con 0x7f3da8105630 2023-07-14T16:06:55.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.897+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da8061b40 con 0x7f3da8105630 2023-07-14T16:06:55.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.897+0000 7f3da5ffb700 1 --2- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3d98038800 0x7f3d9803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:55.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.897+0000 7f3da5ffb700 1 -- 172.21.15.62:0/3437251395 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3d9c04c960 con 0x7f3da8105630 2023-07-14T16:06:55.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.898+0000 7f3da7fff700 1 --2- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3d98038800 0x7f3d9803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:55.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.898+0000 7f3da7fff700 1 --2- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3d98038800 0x7f3d9803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d94006fd0 tx=0x7f3d94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:55.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:55.902+0000 7f3da5ffb700 1 -- 172.21.15.62:0/3437251395 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3d9c02b590 con 0x7f3da8105630 2023-07-14T16:06:56.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.114+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3da8197dd0 con 0x7f3da8105630 2023-07-14T16:06:56.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.442+0000 7f3da5ffb700 1 -- 172.21.15.62:0/3437251395 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3d9c027020 con 0x7f3da8105630 2023-07-14T16:06:56.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3d98038800 msgr2=0x7f3d9803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:56.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 --2- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3d98038800 0x7f3d9803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d94006fd0 tx=0x7f3d94006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 msgr2=0x7f3da819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 --2- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 0x7f3da819c840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c006010 tx=0x7f3d9c004ff0 comp rx=0 tx=0).stop 2023-07-14T16:06:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 shutdown_connections 2023-07-14T16:06:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 --2- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3d98038800 0x7f3d9803acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 --2- 172.21.15.62:0/3437251395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3da8105630 0x7f3da819c840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 >> 172.21.15.62:0/3437251395 conn(0x7f3da80fb130 msgr2=0x7f3da80fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 shutdown_connections 2023-07-14T16:06:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:06:56.445+0000 7f3daea96700 1 -- 172.21.15.62:0/3437251395 wait complete. 2023-07-14T16:06:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2023-07-14T16:06:56.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:56 smithi062 ceph-mon[95543]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:56.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:56 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3437251395' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T16:06:56.924 INFO:tasks.cephadm:Adding mon.a on smithi062 2023-07-14T16:06:56.924 INFO:tasks.cephadm:Adding mon.b on smithi081 2023-07-14T16:06:56.924 INFO:tasks.cephadm:Adding mon.c on smithi164 2023-07-14T16:06:56.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch apply mon '3;smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c' 2023-07-14T16:06:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:06:57.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3437251395' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T16:06:57.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:06:57.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:57.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:57.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:57.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:57.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:06:57.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:06:57.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:06:57.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:57 smithi062 ceph-mon[95543]: Updating smithi164:/etc/ceph/ceph.conf 2023-07-14T16:06:58.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:58 smithi062 ceph-mon[95543]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:06:58.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:58 smithi062 ceph-mon[95543]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:06:58.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:58 smithi062 ceph-mon[95543]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:06:59.310 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.334+0000 7f0714a71700 1 -- 172.21.15.164:0/455795122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 msgr2=0x7f0710105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:59.310 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.334+0000 7f0714a71700 1 --2- 172.21.15.164:0/455795122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 0x7f0710105a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0700009910 tx=0x7f0700009c20 comp rx=0 tx=0).stop 2023-07-14T16:06:59.311 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.334+0000 7f0714a71700 1 -- 172.21.15.164:0/455795122 shutdown_connections 2023-07-14T16:06:59.311 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.334+0000 7f0714a71700 1 --2- 172.21.15.164:0/455795122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 0x7f0710105a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:59.311 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.334+0000 7f0714a71700 1 -- 172.21.15.164:0/455795122 >> 172.21.15.164:0/455795122 conn(0x7f07100fb130 msgr2=0x7f07100fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:59.311 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.334+0000 7f0714a71700 1 -- 172.21.15.164:0/455795122 shutdown_connections 2023-07-14T16:06:59.311 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.334+0000 7f0714a71700 1 -- 172.21.15.164:0/455795122 wait complete. 2023-07-14T16:06:59.312 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.335+0000 7f0714a71700 1 Processor -- start 2023-07-14T16:06:59.312 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.335+0000 7f0714a71700 1 -- start start 2023-07-14T16:06:59.312 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.335+0000 7f0714a71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 0x7f0710072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:59.312 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.335+0000 7f0714a71700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0710074790 con 0x7f0710105630 2023-07-14T16:06:59.312 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.336+0000 7f070e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 0x7f0710072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:59.313 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.336+0000 7f070e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 0x7f0710072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.164:38716/0 (socket says 172.21.15.164:38716) 2023-07-14T16:06:59.313 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.336+0000 7f070e59c700 1 -- 172.21.15.164:0/3130346666 learned_addr learned my addr 172.21.15.164:0/3130346666 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:06:59.313 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.336+0000 7f070e59c700 1 -- 172.21.15.164:0/3130346666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000095f0 con 0x7f0710105630 2023-07-14T16:06:59.314 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.337+0000 7f070e59c700 1 --2- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 0x7f0710072810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0700006010 tx=0x7f0700004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:59.314 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.337+0000 7f06ff7fe700 1 -- 172.21.15.164:0/3130346666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f070001d070 con 0x7f0710105630 2023-07-14T16:06:59.314 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.337+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0710072d50 con 0x7f0710105630 2023-07-14T16:06:59.314 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.337+0000 7f06ff7fe700 1 -- 172.21.15.164:0/3130346666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f070000bdf0 con 0x7f0710105630 2023-07-14T16:06:59.314 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.337+0000 7f06ff7fe700 1 -- 172.21.15.164:0/3130346666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0700022d80 con 0x7f0710105630 2023-07-14T16:06:59.314 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.337+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0710073250 con 0x7f0710105630 2023-07-14T16:06:59.315 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.338+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07101048d0 con 0x7f0710105630 2023-07-14T16:06:59.315 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.339+0000 7f06ff7fe700 1 -- 172.21.15.164:0/3130346666 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0700022570 con 0x7f0710105630 2023-07-14T16:06:59.315 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.339+0000 7f06ff7fe700 1 --2- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f06f80387b0 0x7f06f803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:06:59.315 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.339+0000 7f06ff7fe700 1 -- 172.21.15.164:0/3130346666 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f070004c960 con 0x7f0710105630 2023-07-14T16:06:59.316 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.339+0000 7f070dd9b700 1 --2- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f06f80387b0 0x7f06f803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:06:59.316 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.340+0000 7f070dd9b700 1 --2- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f06f80387b0 0x7f06f803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0704006fd0 tx=0x7f0704006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:06:59.321 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.345+0000 7f06ff7fe700 1 -- 172.21.15.164:0/3130346666 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f070002ab10 con 0x7f0710105630 2023-07-14T16:06:59.528 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.552+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f0710000990 con 0x7f06f80387b0 2023-07-14T16:06:59.535 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.559+0000 7f06ff7fe700 1 -- 172.21.15.164:0/3130346666 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0710000990 con 0x7f06f80387b0 2023-07-14T16:06:59.536 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mon update... 2023-07-14T16:06:59.538 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f06f80387b0 msgr2=0x7f06f803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:59.538 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 --2- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f06f80387b0 0x7f06f803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0704006fd0 tx=0x7f0704006e40 comp rx=0 tx=0).stop 2023-07-14T16:06:59.538 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 msgr2=0x7f0710072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:06:59.538 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 --2- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 0x7f0710072810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0700006010 tx=0x7f0700004ff0 comp rx=0 tx=0).stop 2023-07-14T16:06:59.539 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 shutdown_connections 2023-07-14T16:06:59.539 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 --2- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f06f80387b0 0x7f06f803ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:59.539 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 --2- 172.21.15.164:0/3130346666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0710105630 0x7f0710072810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:06:59.539 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 >> 172.21.15.164:0/3130346666 conn(0x7f07100fb130 msgr2=0x7f07100fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:06:59.539 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 shutdown_connections 2023-07-14T16:06:59.539 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:06:59.562+0000 7f0714a71700 1 -- 172.21.15.164:0/3130346666 wait complete. 2023-07-14T16:06:59.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T16:06:59.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:59.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:59.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:59.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:59.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:06:59.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:06:59.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:06:59.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:06:59.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:06:59.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:06:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:00.046 DEBUG:teuthology.orchestra.run.smithi081:mon.b> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.b.service 2023-07-14T16:07:00.050 DEBUG:teuthology.orchestra.run.smithi164:mon.c> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.c.service 2023-07-14T16:07:00.055 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T16:07:00.055 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T16:07:00.128 INFO:journalctl@ceph.mon.b.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:07:00.131 INFO:journalctl@ceph.mon.c.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:07:00.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:00 smithi062 ceph-mon[95543]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:00.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:00 smithi062 ceph-mon[95543]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:00.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:00 smithi062 ceph-mon[95543]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c;count:3 2023-07-14T16:07:00.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:00 smithi062 ceph-mon[95543]: Deploying daemon mon.c on smithi164 2023-07-14T16:07:01.248 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:02.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:02 smithi062 ceph-mon[95543]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:03.462 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.485+0000 7ff9413eb700 1 -- 172.21.15.164:0/3455020033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 msgr2=0x7ff93c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:03.462 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.486+0000 7ff9413eb700 1 --2- 172.21.15.164:0/3455020033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 0x7ff93c072c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009910 tx=0x7ff92c009c20 comp rx=0 tx=0).stop 2023-07-14T16:07:03.463 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.486+0000 7ff9413eb700 1 -- 172.21.15.164:0/3455020033 shutdown_connections 2023-07-14T16:07:03.463 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.486+0000 7ff9413eb700 1 --2- 172.21.15.164:0/3455020033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 0x7ff93c072c80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:03.463 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.486+0000 7ff9413eb700 1 -- 172.21.15.164:0/3455020033 >> 172.21.15.164:0/3455020033 conn(0x7ff93c0fb130 msgr2=0x7ff93c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:03.464 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.486+0000 7ff9413eb700 1 -- 172.21.15.164:0/3455020033 shutdown_connections 2023-07-14T16:07:03.464 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.486+0000 7ff9413eb700 1 -- 172.21.15.164:0/3455020033 wait complete. 2023-07-14T16:07:03.466 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.487+0000 7ff9413eb700 1 Processor -- start 2023-07-14T16:07:03.466 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.487+0000 7ff9413eb700 1 -- start start 2023-07-14T16:07:03.466 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.487+0000 7ff9413eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 0x7ff93c19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:03.467 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.487+0000 7ff9413eb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c19b5b0 con 0x7ff93c0728b0 2023-07-14T16:07:03.467 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.488+0000 7ff93affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 0x7ff93c19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:03.467 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.488+0000 7ff93affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 0x7ff93c19b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.164:48716/0 (socket says 172.21.15.164:48716) 2023-07-14T16:07:03.467 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.488+0000 7ff93affd700 1 -- 172.21.15.164:0/4109426413 learned_addr learned my addr 172.21.15.164:0/4109426413 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:07:03.468 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.488+0000 7ff93affd700 1 -- 172.21.15.164:0/4109426413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0095f0 con 0x7ff93c0728b0 2023-07-14T16:07:03.468 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.488+0000 7ff93affd700 1 --2- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 0x7ff93c19b070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff92c004e90 tx=0x7ff92c004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:03.468 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.489+0000 7ff923fff700 1 -- 172.21.15.164:0/4109426413 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff92c01d070 con 0x7ff93c0728b0 2023-07-14T16:07:03.468 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.489+0000 7ff923fff700 1 -- 172.21.15.164:0/4109426413 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff92c0044b0 con 0x7ff93c0728b0 2023-07-14T16:07:03.468 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.489+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c19b7b0 con 0x7ff93c0728b0 2023-07-14T16:07:03.468 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.489+0000 7ff923fff700 1 -- 172.21.15.164:0/4109426413 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff92c0038f0 con 0x7ff93c0728b0 2023-07-14T16:07:03.469 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.489+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c19bb70 con 0x7ff93c0728b0 2023-07-14T16:07:03.469 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.490+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c061b40 con 0x7ff93c0728b0 2023-07-14T16:07:03.470 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.490+0000 7ff923fff700 1 -- 172.21.15.164:0/4109426413 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7ff92c00b960 con 0x7ff93c0728b0 2023-07-14T16:07:03.470 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.491+0000 7ff923fff700 1 --2- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9240387b0 0x7ff92403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:03.470 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.491+0000 7ff923fff700 1 -- 172.21.15.164:0/4109426413 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff92c04c820 con 0x7ff93c0728b0 2023-07-14T16:07:03.470 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.491+0000 7ff93a7fc700 1 --2- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9240387b0 0x7ff92403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:03.470 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.492+0000 7ff93a7fc700 1 --2- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9240387b0 0x7ff92403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff930006fd0 tx=0x7ff930006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:03.473 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.497+0000 7ff923fff700 1 -- 172.21.15.164:0/4109426413 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff92c02a930 con 0x7ff93c0728b0 2023-07-14T16:07:03.786 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.810+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff93c102510 con 0x7ff93c0728b0 2023-07-14T16:07:03.787 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.811+0000 7ff923fff700 1 -- 172.21.15.164:0/4109426413 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7ff92c02a360 con 0x7ff93c0728b0 2023-07-14T16:07:03.787 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:07:03.787 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":1,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","modified":"2023-07-14T16:05:19.375259Z","created":"2023-07-14T16:05:19.375259Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-14T16:07:03.790 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9240387b0 msgr2=0x7ff92403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:03.790 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 --2- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9240387b0 0x7ff92403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff930006fd0 tx=0x7ff930006e40 comp rx=0 tx=0).stop 2023-07-14T16:07:03.790 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 msgr2=0x7ff93c19b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:03.790 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 --2- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 0x7ff93c19b070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff92c004e90 tx=0x7ff92c004f70 comp rx=0 tx=0).stop 2023-07-14T16:07:03.790 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 shutdown_connections 2023-07-14T16:07:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 --2- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9240387b0 0x7ff92403ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 --2- 172.21.15.164:0/4109426413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff93c0728b0 0x7ff93c19b070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 >> 172.21.15.164:0/4109426413 conn(0x7ff93c0fb130 msgr2=0x7ff93c0fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 shutdown_connections 2023-07-14T16:07:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:03.813+0000 7ff9413eb700 1 -- 172.21.15.164:0/4109426413 wait complete. 2023-07-14T16:07:03.791 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 1 2023-07-14T16:07:04.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:03 smithi062 ceph-mon[95543]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:04.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:03 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/4109426413' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:04.975 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:04 smithi164 podman[97387]: 2023-07-14T16:07:05.241 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T16:07:05.242 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T16:07:05.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 bash[97387]: 833b2587957d9aa0be8562eb6095d614a2a9f0d4d7616b0f6e61f28be62ee7fa 2023-07-14T16:07:05.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T16:07:05.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-14T16:07:05.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pidfile_write: ignore empty --pid-file 2023-07-14T16:07:05.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: load: jerasure load: lrc 2023-07-14T16:07:05.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: RocksDB version: 7.9.2 2023-07-14T16:07:05.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Git sha 0 2023-07-14T16:07:05.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-14T16:07:05.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: DB SUMMARY 2023-07-14T16:07:05.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: DB Session ID: 5DYBUML9VPY0PIBGXZWU 2023-07-14T16:07:05.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: CURRENT file: CURRENT 2023-07-14T16:07:05.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T16:07:05.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-14T16:07:05.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-14T16:07:05.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-14T16:07:05.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.error_if_exists: 0 2023-07-14T16:07:05.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.create_if_missing: 0 2023-07-14T16:07:05.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.paranoid_checks: 1 2023-07-14T16:07:05.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T16:07:05.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T16:07:05.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T16:07:05.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.env: 0x557641c32720 2023-07-14T16:07:05.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T16:07:05.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.info_log: 0x55764379d280 2023-07-14T16:07:05.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T16:07:05.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.statistics: (nil) 2023-07-14T16:07:05.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.use_fsync: 0 2023-07-14T16:07:05.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_log_file_size: 0 2023-07-14T16:07:05.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T16:07:05.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T16:07:05.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T16:07:05.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T16:07:05.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.allow_fallocate: 1 2023-07-14T16:07:05.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T16:07:05.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T16:07:05.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.use_direct_reads: 0 2023-07-14T16:07:05.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T16:07:05.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T16:07:05.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.db_log_dir: 2023-07-14T16:07:05.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.wal_dir: 2023-07-14T16:07:05.281 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T16:07:05.281 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T16:07:05.281 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T16:07:05.281 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T16:07:05.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T16:07:05.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T16:07:05.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T16:07:05.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T16:07:05.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.write_buffer_manager: 0x5576436be3c0 2023-07-14T16:07:05.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T16:07:05.282 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T16:07:05.283 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T16:07:05.283 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T16:07:05.283 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T16:07:05.283 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T16:07:05.283 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T16:07:05.283 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T16:07:05.283 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.unordered_write: 0 2023-07-14T16:07:05.283 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T16:07:05.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T16:07:05.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T16:07:05.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T16:07:05.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.row_cache: None 2023-07-14T16:07:05.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.wal_filter: None 2023-07-14T16:07:05.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T16:07:05.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T16:07:05.284 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.two_write_queues: 0 2023-07-14T16:07:05.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T16:07:05.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.wal_compression: 0 2023-07-14T16:07:05.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.atomic_flush: 0 2023-07-14T16:07:05.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T16:07:05.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T16:07:05.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T16:07:05.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.log_readahead_size: 0 2023-07-14T16:07:05.285 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T16:07:05.286 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T16:07:05.286 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T16:07:05.286 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T16:07:05.286 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T16:07:05.286 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T16:07:05.286 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T16:07:05.286 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_background_jobs: 2 2023-07-14T16:07:05.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_background_compactions: -1 2023-07-14T16:07:05.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_subcompactions: 1 2023-07-14T16:07:05.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T16:07:05.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T16:07:05.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T16:07:05.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T16:07:05.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T16:07:05.287 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T16:07:05.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T16:07:05.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T16:07:05.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_open_files: -1 2023-07-14T16:07:05.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T16:07:05.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T16:07:05.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T16:07:05.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T16:07:05.288 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_background_flushes: -1 2023-07-14T16:07:05.289 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Compression algorithms supported: 2023-07-14T16:07:05.289 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T16:07:05.289 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: kZSTD supported: 0 2023-07-14T16:07:05.289 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: kXpressCompression supported: 0 2023-07-14T16:07:05.289 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T16:07:05.289 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: kZlibCompression supported: 1 2023-07-14T16:07:05.289 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: kSnappyCompression supported: 1 2023-07-14T16:07:05.289 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: kLZ4Compression supported: 1 2023-07-14T16:07:05.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: kBZip2Compression supported: 0 2023-07-14T16:07:05.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T16:07:05.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T16:07:05.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-14T16:07:05.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T16:07:05.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T16:07:05.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.merge_operator: 2023-07-14T16:07:05.290 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_filter: None 2023-07-14T16:07:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T16:07:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T16:07:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T16:07:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T16:07:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55764379d380) 2023-07-14T16:07:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks: 1 2023-07-14T16:07:05.291 INFO:journalctl@ceph.mon.c.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T16:07:05.292 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T16:07:05.292 INFO:journalctl@ceph.mon.c.smithi164.stdout: pin_top_level_index_and_filter: 1 2023-07-14T16:07:05.292 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_type: 0 2023-07-14T16:07:05.292 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_index_type: 0 2023-07-14T16:07:05.292 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_shortening: 1 2023-07-14T16:07:05.292 INFO:journalctl@ceph.mon.c.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T16:07:05.292 INFO:journalctl@ceph.mon.c.smithi164.stdout: checksum: 4 2023-07-14T16:07:05.293 INFO:journalctl@ceph.mon.c.smithi164.stdout: no_block_cache: 0 2023-07-14T16:07:05.293 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache: 0x55764375d1f0 2023-07-14T16:07:05.293 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_name: BinnedLRUCache 2023-07-14T16:07:05.293 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_options: 2023-07-14T16:07:05.293 INFO:journalctl@ceph.mon.c.smithi164.stdout: capacity : 536870912 2023-07-14T16:07:05.293 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_shard_bits : 4 2023-07-14T16:07:05.293 INFO:journalctl@ceph.mon.c.smithi164.stdout: strict_capacity_limit : 0 2023-07-14T16:07:05.293 INFO:journalctl@ceph.mon.c.smithi164.stdout: high_pri_pool_ratio: 0.000 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_cache_compressed: (nil) 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: persistent_cache: (nil) 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size: 4096 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_size_deviation: 10 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_restart_interval: 16 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: index_block_restart_interval: 1 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: metadata_block_size: 4096 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: partition_filters: 0 2023-07-14T16:07:05.294 INFO:journalctl@ceph.mon.c.smithi164.stdout: use_delta_encoding: 1 2023-07-14T16:07:05.295 INFO:journalctl@ceph.mon.c.smithi164.stdout: filter_policy: bloomfilter 2023-07-14T16:07:05.295 INFO:journalctl@ceph.mon.c.smithi164.stdout: whole_key_filtering: 1 2023-07-14T16:07:05.295 INFO:journalctl@ceph.mon.c.smithi164.stdout: verify_compression: 0 2023-07-14T16:07:05.295 INFO:journalctl@ceph.mon.c.smithi164.stdout: read_amp_bytes_per_bit: 0 2023-07-14T16:07:05.295 INFO:journalctl@ceph.mon.c.smithi164.stdout: format_version: 5 2023-07-14T16:07:05.295 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_index_compression: 1 2023-07-14T16:07:05.295 INFO:journalctl@ceph.mon.c.smithi164.stdout: block_align: 0 2023-07-14T16:07:05.295 INFO:journalctl@ceph.mon.c.smithi164.stdout: max_auto_readahead_size: 262144 2023-07-14T16:07:05.296 INFO:journalctl@ceph.mon.c.smithi164.stdout: prepopulate_block_cache: 0 2023-07-14T16:07:05.296 INFO:journalctl@ceph.mon.c.smithi164.stdout: initial_auto_readahead_size: 8192 2023-07-14T16:07:05.296 INFO:journalctl@ceph.mon.c.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T16:07:05.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T16:07:05.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T16:07:05.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression: NoCompression 2023-07-14T16:07:05.296 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T16:07:05.297 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T16:07:05.297 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T16:07:05.297 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.num_levels: 7 2023-07-14T16:07:05.297 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T16:07:05.297 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T16:07:05.297 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T16:07:05.297 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T16:07:05.297 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T16:07:05.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T16:07:05.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T16:07:05.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T16:07:05.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T16:07:05.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T16:07:05.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T16:07:05.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T16:07:05.298 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T16:07:05.299 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T16:07:05.299 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T16:07:05.299 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T16:07:05.299 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T16:07:05.299 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T16:07:05.299 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T16:07:05.299 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T16:07:05.300 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T16:07:05.300 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T16:07:05.300 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T16:07:05.300 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T16:07:05.300 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T16:07:05.300 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T16:07:05.300 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T16:07:05.300 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T16:07:05.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T16:07:05.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T16:07:05.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T16:07:05.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T16:07:05.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T16:07:05.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T16:07:05.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T16:07:05.301 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T16:07:05.302 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T16:07:05.302 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T16:07:05.302 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T16:07:05.302 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T16:07:05.302 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T16:07:05.303 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T16:07:05.303 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T16:07:05.303 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T16:07:05.303 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T16:07:05.303 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T16:07:05.303 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T16:07:05.303 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T16:07:05.304 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T16:07:05.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T16:07:05.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T16:07:05.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T16:07:05.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T16:07:05.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.table_properties_collectors: 2023-07-14T16:07:05.305 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.inplace_update_support: 0 2023-07-14T16:07:05.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T16:07:05.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T16:07:05.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T16:07:05.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T16:07:05.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.bloom_locality: 0 2023-07-14T16:07:05.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.max_successive_merges: 0 2023-07-14T16:07:05.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T16:07:05.306 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T16:07:05.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T16:07:05.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T16:07:05.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.ttl: 2592000 2023-07-14T16:07:05.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T16:07:05.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T16:07:05.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T16:07:05.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.enable_blob_files: false 2023-07-14T16:07:05.307 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.min_blob_size: 0 2023-07-14T16:07:05.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T16:07:05.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T16:07:05.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T16:07:05.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T16:07:05.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T16:07:05.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T16:07:05.308 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T16:07:05.309 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T16:07:05.309 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-14T16:07:05.309 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-14T16:07:05.309 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ddd5c215-ad21-44ce-9ced-da5f88140cb2 2023-07-14T16:07:05.309 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350825013371, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-14T16:07:05.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-14T16:07:05.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350825014195, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689350825, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ddd5c215-ad21-44ce-9ced-da5f88140cb2", "db_session_id": "5DYBUML9VPY0PIBGXZWU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-14T16:07:05.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350825014443, "job": 1, "event": "recovery_finished"} 2023-07-14T16:07:05.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-14T16:07:05.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T16:07:05.310 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5576437f6000 2023-07-14T16:07:05.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: DB pointer 0x5576437e2000 2023-07-14T16:07:05.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T16:07:05.311 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T16:07:05.311 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** DB Stats ** 2023-07-14T16:07:05.311 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T16:07:05.311 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T16:07:05.311 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T16:07:05.312 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T16:07:05.312 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T16:07:05.312 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T16:07:05.312 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T16:07:05.312 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2023-07-14T16:07:05.313 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2023-07-14T16:07:05.313 INFO:journalctl@ceph.mon.c.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T16:07:05.313 INFO:journalctl@ceph.mon.c.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T16:07:05.313 INFO:journalctl@ceph.mon.c.smithi164.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:07:05.313 INFO:journalctl@ceph.mon.c.smithi164.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:07:05.313 INFO:journalctl@ceph.mon.c.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:07:05.314 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2023-07-14T16:07:05.314 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** Compaction Stats [default] ** 2023-07-14T16:07:05.314 INFO:journalctl@ceph.mon.c.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T16:07:05.314 INFO:journalctl@ceph.mon.c.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T16:07:05.314 INFO:journalctl@ceph.mon.c.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:07:05.314 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2023-07-14T16:07:05.314 INFO:journalctl@ceph.mon.c.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T16:07:05.315 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2023-07-14T16:07:05.315 INFO:journalctl@ceph.mon.c.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T16:07:05.315 INFO:journalctl@ceph.mon.c.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T16:07:05.315 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T16:07:05.315 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T16:07:05.315 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T16:07:05.315 INFO:journalctl@ceph.mon.c.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T16:07:05.315 INFO:journalctl@ceph.mon.c.smithi164.stdout: Cumulative compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T16:07:05.316 INFO:journalctl@ceph.mon.c.smithi164.stdout: Interval compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T16:07:05.316 INFO:journalctl@ceph.mon.c.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T16:07:05.316 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache BinnedLRUCache@0x55764375d1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-14T16:07:05.316 INFO:journalctl@ceph.mon.c.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-14T16:07:05.316 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2023-07-14T16:07:05.316 INFO:journalctl@ceph.mon.c.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T16:07:05.316 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-14T16:07:05.316 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: using public_addr v2:172.21.15.164:0/0 -> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2023-07-14T16:07:05.317 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: starting mon.c rank -1 at public addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] at bind addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:07:05.317 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(???) e0 preinit fsid 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:07:05.317 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).mds e1 new map 2023-07-14T16:07:05.317 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-14T16:07:05.317 INFO:journalctl@ceph.mon.c.smithi164.stdout: e1 2023-07-14T16:07:05.317 INFO:journalctl@ceph.mon.c.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T16:07:05.317 INFO:journalctl@ceph.mon.c.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T16:07:05.318 INFO:journalctl@ceph.mon.c.smithi164.stdout: legacy client fscid: -1 2023-07-14T16:07:05.318 INFO:journalctl@ceph.mon.c.smithi164.stdout: 2023-07-14T16:07:05.318 INFO:journalctl@ceph.mon.c.smithi164.stdout: No filesystems configured 2023-07-14T16:07:05.318 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T16:07:05.318 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T16:07:05.318 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T16:07:05.318 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T16:07:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T16:07:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T16:07:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T16:07:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:07:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:07:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:07:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.319 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:05.320 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.320 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.320 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.320 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.320 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:07:05.320 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:05.320 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:05.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Updating smithi081:/etc/ceph/ceph.conf 2023-07-14T16:07:05.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:05.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Deploying cephadm binary to smithi164 2023-07-14T16:07:05.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:07:05.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T16:07:05.321 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Added host smithi164 2023-07-14T16:07:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.322 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:07:05.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3437251395' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T16:07:05.323 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3437251395' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T16:07:05.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:07:05.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:07:05.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:05.324 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:05.325 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Updating smithi164:/etc/ceph/ceph.conf 2023-07-14T16:07:05.325 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:05.325 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.325 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:07:05.325 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T16:07:05.325 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.325 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.325 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:05.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:05.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:05.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:05.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:05.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:05.326 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.327 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:05.327 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c;count:3 2023-07-14T16:07:05.327 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: Deploying daemon mon.c on smithi164 2023-07-14T16:07:05.327 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.327 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:05.327 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/4109426413' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:05.327 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T16:07:05.328 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta expand map: {default=false} 2023-07-14T16:07:05.328 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta from 'false' to 'false' 2023-07-14T16:07:05.328 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta expanded map: {default=false} 2023-07-14T16:07:05.328 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta expand map: {default=info} 2023-07-14T16:07:05.328 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta from 'info' to 'info' 2023-07-14T16:07:05.328 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta expanded map: {default=info} 2023-07-14T16:07:05.329 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta expand map: {default=daemon} 2023-07-14T16:07:05.329 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-14T16:07:05.329 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta expanded map: {default=daemon} 2023-07-14T16:07:05.329 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta expand map: {default=debug} 2023-07-14T16:07:05.329 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta from 'debug' to 'debug' 2023-07-14T16:07:05.329 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: expand_channel_meta expanded map: {default=debug} 2023-07-14T16:07:05.329 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 ceph-mon[97427]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-14T16:07:05.329 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:05 smithi164 systemd[1]: Started Ceph mon.c for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:07:05.965 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:07:10.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:07:10.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: mon.a calling monitor election 2023-07-14T16:07:10.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:10.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: mon.c calling monitor election 2023-07-14T16:07:10.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:10.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:10.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T16:07:10.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: monmap e2: 2 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} 2023-07-14T16:07:10.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: fsmap 2023-07-14T16:07:10.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:07:10.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: mgrmap e13: x(active, since 50s) 2023-07-14T16:07:10.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: overall HEALTH_OK 2023-07-14T16:07:10.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:10.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:10.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:07:10.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: mon.a calling monitor election 2023-07-14T16:07:10.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:10.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: mon.c calling monitor election 2023-07-14T16:07:10.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:10.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:10.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:10.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T16:07:10.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: monmap e2: 2 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} 2023-07-14T16:07:10.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: fsmap 2023-07-14T16:07:10.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:07:10.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: mgrmap e13: x(active, since 50s) 2023-07-14T16:07:10.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: overall HEALTH_OK 2023-07-14T16:07:10.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:10.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:10.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.602+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1612292086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 msgr2=0x7f8390005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:10.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.602+0000 7f83ad4e6700 1 --2- 172.21.15.164:0/1612292086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 0x7f8390005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83a8100bc0 tx=0x7f839800fe60 comp rx=0 tx=0).stop 2023-07-14T16:07:10.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.603+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1612292086 shutdown_connections 2023-07-14T16:07:10.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.603+0000 7f83ad4e6700 1 --2- 172.21.15.164:0/1612292086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 0x7f8390005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:10.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.603+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1612292086 >> 172.21.15.164:0/1612292086 conn(0x7f83a80fb210 msgr2=0x7f83a80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:10.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.603+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1612292086 shutdown_connections 2023-07-14T16:07:10.580 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.603+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1612292086 wait complete. 2023-07-14T16:07:10.580 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.604+0000 7f83ad4e6700 1 Processor -- start 2023-07-14T16:07:10.580 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.604+0000 7f83ad4e6700 1 -- start start 2023-07-14T16:07:10.581 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.605+0000 7f83ad4e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 0x7f83a8196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:10.581 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.605+0000 7f83ad4e6700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 0x7f83a819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:10.581 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.605+0000 7f83ad4e6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a8197770 con 0x7f83900035c0 2023-07-14T16:07:10.581 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.605+0000 7f83ad4e6700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a81978b0 con 0x7f83a81971f0 2023-07-14T16:07:10.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.605+0000 7f83a6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 0x7f83a8196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:10.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.605+0000 7f83a6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 0x7f83a8196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.164:48796/0 (socket says 172.21.15.164:48796) 2023-07-14T16:07:10.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.605+0000 7f83a6ffd700 1 -- 172.21.15.164:0/1923903477 learned_addr learned my addr 172.21.15.164:0/1923903477 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:07:10.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.605+0000 7f83a67fc700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 0x7f83a819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:10.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f83a67fc700 1 -- 172.21.15.164:0/1923903477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 msgr2=0x7f83a819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-14T16:07:10.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f83a67fc700 1 -- 172.21.15.164:0/1923903477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 msgr2=0x7f83a819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T16:07:10.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f83a67fc700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 0x7f83a819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T16:07:10.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f83a6ffd700 1 -- 172.21.15.164:0/1923903477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 msgr2=0x7f83a819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:10.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f83a67fc700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 0x7f83a819a5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T16:07:10.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f83a6ffd700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 0x7f83a819a5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:10.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f83a6ffd700 1 -- 172.21.15.164:0/1923903477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839800f8b0 con 0x7f83900035c0 2023-07-14T16:07:10.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f83a6ffd700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 0x7f83a8196cb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8398000c00 tx=0x7f8398013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:10.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.606+0000 7f838ffff700 1 -- 172.21.15.164:0/1923903477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8398023070 con 0x7f83900035c0 2023-07-14T16:07:10.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.607+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a819aaf0 con 0x7f83900035c0 2023-07-14T16:07:10.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.607+0000 7f838ffff700 1 -- 172.21.15.164:0/1923903477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f83980041d0 con 0x7f83900035c0 2023-07-14T16:07:10.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.607+0000 7f838ffff700 1 -- 172.21.15.164:0/1923903477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f839801cb60 con 0x7f83900035c0 2023-07-14T16:07:10.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.607+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a819b0c0 con 0x7f83900035c0 2023-07-14T16:07:10.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.608+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a8061b40 con 0x7f83900035c0 2023-07-14T16:07:10.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.608+0000 7f838ffff700 1 -- 172.21.15.164:0/1923903477 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8398003d40 con 0x7f83900035c0 2023-07-14T16:07:10.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.609+0000 7f838ffff700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f83940385e0 0x7f839403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:10.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.609+0000 7f838ffff700 1 -- 172.21.15.164:0/1923903477 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f839804d420 con 0x7f83900035c0 2023-07-14T16:07:10.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.609+0000 7f83a67fc700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f83940385e0 0x7f839403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:10.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.610+0000 7f83a67fc700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f83940385e0 0x7f839403aa90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f839c009850 tx=0x7f839c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:10.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.614+0000 7f838ffff700 1 -- 172.21.15.164:0/1923903477 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f839802f770 con 0x7f83900035c0 2023-07-14T16:07:10.860 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.885+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f83a81a1930 con 0x7f83900035c0 2023-07-14T16:07:10.861 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.886+0000 7f838ffff700 1 -- 172.21.15.164:0/1923903477 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f839802f360 con 0x7f83900035c0 2023-07-14T16:07:10.862 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:07:10.862 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":2,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","modified":"2023-07-14T16:07:05.052070Z","created":"2023-07-14T16:05:19.375259Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-14T16:07:10.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f83940385e0 msgr2=0x7f839403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:10.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f83940385e0 0x7f839403aa90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f839c009850 tx=0x7f839c008040 comp rx=0 tx=0).stop 2023-07-14T16:07:10.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 msgr2=0x7f83a8196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 0x7f83a8196cb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8398000c00 tx=0x7f8398013c70 comp rx=0 tx=0).stop 2023-07-14T16:07:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 shutdown_connections 2023-07-14T16:07:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f83940385e0 0x7f839403aa90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83900035c0 0x7f83a8196cb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 --2- 172.21.15.164:0/1923903477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a81971f0 0x7f83a819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 >> 172.21.15.164:0/1923903477 conn(0x7f83a80fb210 msgr2=0x7f83a8073070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 shutdown_connections 2023-07-14T16:07:10.866 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:10.888+0000 7f83ad4e6700 1 -- 172.21.15.164:0/1923903477 wait complete. 2023-07-14T16:07:10.866 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 2 2023-07-14T16:07:11.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:11 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:11.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:11 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:11.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:11 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:11.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:11 smithi164 ceph-mon[97427]: Deploying daemon mon.b on smithi081 2023-07-14T16:07:11.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:11 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/1923903477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:11.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:11 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:11.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:11 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:11.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:11 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:11.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:11 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:11.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:11 smithi062 ceph-mon[95543]: Deploying daemon mon.b on smithi081 2023-07-14T16:07:11.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:11 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/1923903477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:11.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:11 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:12.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:12 smithi062 ceph-mon[95543]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:12.447 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T16:07:12.448 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T16:07:12.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:12 smithi164 ceph-mon[97427]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:13.175 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:07:13.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:13 smithi081 systemd[1]: Starting Ceph mon.b for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:07:14.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:13 smithi164 ceph-mon[97427]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:13 smithi062 ceph-mon[95543]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 podman[127755]: 2023-07-14T16:07:14.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T16:07:14.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-14T16:07:14.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pidfile_write: ignore empty --pid-file 2023-07-14T16:07:14.539 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: load: jerasure load: lrc 2023-07-14T16:07:14.540 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: RocksDB version: 7.9.2 2023-07-14T16:07:14.540 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Git sha 0 2023-07-14T16:07:14.540 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-14T16:07:14.540 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: DB SUMMARY 2023-07-14T16:07:14.540 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: DB Session ID: 1WG835KRJ6EQ8MWCRJXT 2023-07-14T16:07:14.540 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: CURRENT file: CURRENT 2023-07-14T16:07:14.541 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T16:07:14.541 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-14T16:07:14.541 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-14T16:07:14.541 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-14T16:07:14.541 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.error_if_exists: 0 2023-07-14T16:07:14.541 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.create_if_missing: 0 2023-07-14T16:07:14.541 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.paranoid_checks: 1 2023-07-14T16:07:14.542 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T16:07:14.542 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T16:07:14.542 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T16:07:14.542 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.env: 0x556e0882a720 2023-07-14T16:07:14.542 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T16:07:14.542 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.info_log: 0x556e099e1280 2023-07-14T16:07:14.543 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T16:07:14.543 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.statistics: (nil) 2023-07-14T16:07:14.543 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.use_fsync: 0 2023-07-14T16:07:14.543 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_log_file_size: 0 2023-07-14T16:07:14.543 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T16:07:14.543 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T16:07:14.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T16:07:14.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T16:07:14.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.allow_fallocate: 1 2023-07-14T16:07:14.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T16:07:14.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T16:07:14.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.use_direct_reads: 0 2023-07-14T16:07:14.544 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T16:07:14.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T16:07:14.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.db_log_dir: 2023-07-14T16:07:14.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.wal_dir: 2023-07-14T16:07:14.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T16:07:14.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T16:07:14.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T16:07:14.545 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T16:07:14.546 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T16:07:14.546 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T16:07:14.546 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T16:07:14.546 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T16:07:14.546 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.write_buffer_manager: 0x556e099023c0 2023-07-14T16:07:14.547 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T16:07:14.547 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T16:07:14.547 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T16:07:14.547 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T16:07:14.547 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T16:07:14.548 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T16:07:14.548 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T16:07:14.548 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T16:07:14.548 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.unordered_write: 0 2023-07-14T16:07:14.548 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T16:07:14.548 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T16:07:14.548 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T16:07:14.549 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T16:07:14.549 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.row_cache: None 2023-07-14T16:07:14.549 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.wal_filter: None 2023-07-14T16:07:14.549 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T16:07:14.549 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T16:07:14.549 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.two_write_queues: 0 2023-07-14T16:07:14.549 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T16:07:14.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.wal_compression: 0 2023-07-14T16:07:14.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.atomic_flush: 0 2023-07-14T16:07:14.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T16:07:14.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T16:07:14.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T16:07:14.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.log_readahead_size: 0 2023-07-14T16:07:14.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T16:07:14.551 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T16:07:14.551 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T16:07:14.551 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T16:07:14.551 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T16:07:14.551 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T16:07:14.551 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T16:07:14.552 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_background_jobs: 2 2023-07-14T16:07:14.552 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_background_compactions: -1 2023-07-14T16:07:14.552 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_subcompactions: 1 2023-07-14T16:07:14.552 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T16:07:14.552 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T16:07:14.552 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T16:07:14.552 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T16:07:14.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T16:07:14.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T16:07:14.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T16:07:14.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T16:07:14.553 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_open_files: -1 2023-07-14T16:07:14.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T16:07:14.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T16:07:14.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T16:07:14.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T16:07:14.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_background_flushes: -1 2023-07-14T16:07:14.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Compression algorithms supported: 2023-07-14T16:07:14.554 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T16:07:14.555 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: kZSTD supported: 0 2023-07-14T16:07:14.555 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: kXpressCompression supported: 0 2023-07-14T16:07:14.555 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T16:07:14.555 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: kZlibCompression supported: 1 2023-07-14T16:07:14.555 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: kSnappyCompression supported: 1 2023-07-14T16:07:14.555 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: kLZ4Compression supported: 1 2023-07-14T16:07:14.555 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: kBZip2Compression supported: 0 2023-07-14T16:07:14.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T16:07:14.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T16:07:14.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-14T16:07:14.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T16:07:14.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T16:07:14.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.merge_operator: 2023-07-14T16:07:14.556 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_filter: None 2023-07-14T16:07:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T16:07:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T16:07:14.557 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T16:07:14.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T16:07:14.558 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e099e1380) 2023-07-14T16:07:14.558 INFO:journalctl@ceph.mon.b.smithi081.stdout: cache_index_and_filter_blocks: 1 2023-07-14T16:07:14.558 INFO:journalctl@ceph.mon.b.smithi081.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T16:07:14.558 INFO:journalctl@ceph.mon.b.smithi081.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T16:07:14.558 INFO:journalctl@ceph.mon.b.smithi081.stdout: pin_top_level_index_and_filter: 1 2023-07-14T16:07:14.558 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_type: 0 2023-07-14T16:07:14.559 INFO:journalctl@ceph.mon.b.smithi081.stdout: data_block_index_type: 0 2023-07-14T16:07:14.559 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_shortening: 1 2023-07-14T16:07:14.559 INFO:journalctl@ceph.mon.b.smithi081.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T16:07:14.559 INFO:journalctl@ceph.mon.b.smithi081.stdout: checksum: 4 2023-07-14T16:07:14.559 INFO:journalctl@ceph.mon.b.smithi081.stdout: no_block_cache: 0 2023-07-14T16:07:14.559 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache: 0x556e099a11f0 2023-07-14T16:07:14.559 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_name: BinnedLRUCache 2023-07-14T16:07:14.560 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_options: 2023-07-14T16:07:14.560 INFO:journalctl@ceph.mon.b.smithi081.stdout: capacity : 536870912 2023-07-14T16:07:14.560 INFO:journalctl@ceph.mon.b.smithi081.stdout: num_shard_bits : 4 2023-07-14T16:07:14.560 INFO:journalctl@ceph.mon.b.smithi081.stdout: strict_capacity_limit : 0 2023-07-14T16:07:14.560 INFO:journalctl@ceph.mon.b.smithi081.stdout: high_pri_pool_ratio: 0.000 2023-07-14T16:07:14.560 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_cache_compressed: (nil) 2023-07-14T16:07:14.560 INFO:journalctl@ceph.mon.b.smithi081.stdout: persistent_cache: (nil) 2023-07-14T16:07:14.561 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_size: 4096 2023-07-14T16:07:14.561 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_size_deviation: 10 2023-07-14T16:07:14.561 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_restart_interval: 16 2023-07-14T16:07:14.561 INFO:journalctl@ceph.mon.b.smithi081.stdout: index_block_restart_interval: 1 2023-07-14T16:07:14.561 INFO:journalctl@ceph.mon.b.smithi081.stdout: metadata_block_size: 4096 2023-07-14T16:07:14.561 INFO:journalctl@ceph.mon.b.smithi081.stdout: partition_filters: 0 2023-07-14T16:07:14.561 INFO:journalctl@ceph.mon.b.smithi081.stdout: use_delta_encoding: 1 2023-07-14T16:07:14.562 INFO:journalctl@ceph.mon.b.smithi081.stdout: filter_policy: bloomfilter 2023-07-14T16:07:14.562 INFO:journalctl@ceph.mon.b.smithi081.stdout: whole_key_filtering: 1 2023-07-14T16:07:14.562 INFO:journalctl@ceph.mon.b.smithi081.stdout: verify_compression: 0 2023-07-14T16:07:14.562 INFO:journalctl@ceph.mon.b.smithi081.stdout: read_amp_bytes_per_bit: 0 2023-07-14T16:07:14.562 INFO:journalctl@ceph.mon.b.smithi081.stdout: format_version: 5 2023-07-14T16:07:14.562 INFO:journalctl@ceph.mon.b.smithi081.stdout: enable_index_compression: 1 2023-07-14T16:07:14.562 INFO:journalctl@ceph.mon.b.smithi081.stdout: block_align: 0 2023-07-14T16:07:14.563 INFO:journalctl@ceph.mon.b.smithi081.stdout: max_auto_readahead_size: 262144 2023-07-14T16:07:14.563 INFO:journalctl@ceph.mon.b.smithi081.stdout: prepopulate_block_cache: 0 2023-07-14T16:07:14.563 INFO:journalctl@ceph.mon.b.smithi081.stdout: initial_auto_readahead_size: 8192 2023-07-14T16:07:14.563 INFO:journalctl@ceph.mon.b.smithi081.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T16:07:14.563 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T16:07:14.563 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T16:07:14.564 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression: NoCompression 2023-07-14T16:07:14.564 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T16:07:14.564 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T16:07:14.564 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T16:07:14.564 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.num_levels: 7 2023-07-14T16:07:14.565 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T16:07:14.565 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T16:07:14.565 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T16:07:14.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T16:07:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T16:07:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T16:07:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T16:07:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T16:07:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T16:07:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T16:07:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T16:07:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T16:07:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T16:07:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T16:07:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T16:07:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T16:07:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T16:07:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T16:07:14.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T16:07:14.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T16:07:14.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T16:07:14.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T16:07:14.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T16:07:14.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T16:07:14.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T16:07:14.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T16:07:14.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T16:07:14.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T16:07:14.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T16:07:14.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T16:07:14.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T16:07:14.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T16:07:14.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T16:07:14.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T16:07:14.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T16:07:14.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T16:07:14.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T16:07:14.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T16:07:14.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T16:07:14.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T16:07:14.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T16:07:14.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T16:07:14.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T16:07:14.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T16:07:14.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T16:07:14.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T16:07:14.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T16:07:14.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T16:07:14.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T16:07:14.908 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T16:07:14.908 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T16:07:14.908 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T16:07:14.908 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T16:07:14.908 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.table_properties_collectors: 2023-07-14T16:07:14.908 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.inplace_update_support: 0 2023-07-14T16:07:14.908 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T16:07:14.909 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T16:07:14.909 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T16:07:14.909 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T16:07:14.909 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.bloom_locality: 0 2023-07-14T16:07:14.909 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.max_successive_merges: 0 2023-07-14T16:07:14.910 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T16:07:14.910 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T16:07:14.910 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T16:07:14.910 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T16:07:14.910 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.ttl: 2592000 2023-07-14T16:07:14.910 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T16:07:14.910 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T16:07:14.910 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T16:07:14.911 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.enable_blob_files: false 2023-07-14T16:07:14.911 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.min_blob_size: 0 2023-07-14T16:07:14.911 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T16:07:14.911 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T16:07:14.911 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T16:07:14.912 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T16:07:14.912 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T16:07:14.912 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T16:07:14.912 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T16:07:14.912 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T16:07:14.912 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-14T16:07:14.913 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-14T16:07:14.913 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3d36c6b9-2199-4939-ad3e-a067b36a8bcf 2023-07-14T16:07:14.913 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350834562056, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-14T16:07:14.913 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-14T16:07:14.913 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350834562964, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689350834, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3d36c6b9-2199-4939-ad3e-a067b36a8bcf", "db_session_id": "1WG835KRJ6EQ8MWCRJXT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-14T16:07:14.913 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689350834563191, "job": 1, "event": "recovery_finished"} 2023-07-14T16:07:14.913 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-14T16:07:14.913 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T16:07:14.914 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556e09a3a000 2023-07-14T16:07:14.914 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: DB pointer 0x556e09a26000 2023-07-14T16:07:14.914 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T16:07:14.914 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T16:07:14.914 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** DB Stats ** 2023-07-14T16:07:14.914 INFO:journalctl@ceph.mon.b.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T16:07:14.914 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T16:07:14.915 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T16:07:14.915 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T16:07:14.915 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T16:07:14.915 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T16:07:14.915 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T16:07:14.915 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2023-07-14T16:07:14.915 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** Compaction Stats [default] ** 2023-07-14T16:07:14.915 INFO:journalctl@ceph.mon.b.smithi081.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T16:07:14.916 INFO:journalctl@ceph.mon.b.smithi081.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T16:07:14.916 INFO:journalctl@ceph.mon.b.smithi081.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:07:14.916 INFO:journalctl@ceph.mon.b.smithi081.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:07:14.916 INFO:journalctl@ceph.mon.b.smithi081.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:07:14.916 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2023-07-14T16:07:14.916 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** Compaction Stats [default] ** 2023-07-14T16:07:14.916 INFO:journalctl@ceph.mon.b.smithi081.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T16:07:14.916 INFO:journalctl@ceph.mon.b.smithi081.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T16:07:14.917 INFO:journalctl@ceph.mon.b.smithi081.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T16:07:14.917 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2023-07-14T16:07:14.917 INFO:journalctl@ceph.mon.b.smithi081.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T16:07:14.917 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2023-07-14T16:07:14.917 INFO:journalctl@ceph.mon.b.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T16:07:14.917 INFO:journalctl@ceph.mon.b.smithi081.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T16:07:14.917 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T16:07:14.917 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T16:07:14.918 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T16:07:14.918 INFO:journalctl@ceph.mon.b.smithi081.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T16:07:14.918 INFO:journalctl@ceph.mon.b.smithi081.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T16:07:14.918 INFO:journalctl@ceph.mon.b.smithi081.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T16:07:14.918 INFO:journalctl@ceph.mon.b.smithi081.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T16:07:14.918 INFO:journalctl@ceph.mon.b.smithi081.stdout: Block cache BinnedLRUCache@0x556e099a11f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-14T16:07:14.918 INFO:journalctl@ceph.mon.b.smithi081.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-14T16:07:14.919 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2023-07-14T16:07:14.919 INFO:journalctl@ceph.mon.b.smithi081.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T16:07:14.919 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-14T16:07:14.919 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: using public_addr v2:172.21.15.81:0/0 -> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] 2023-07-14T16:07:14.919 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: starting mon.b rank -1 at public addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] at bind addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:07:14.920 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(???) e0 preinit fsid 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:07:14.920 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 bash[127755]: 5ac89713accbbf9d569c3cb8c2b3e058c1ca14fa200973ef86bbe5bdfeb60e37 2023-07-14T16:07:14.920 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 systemd[1]: Started Ceph mon.b for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:07:14.920 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).mds e1 new map 2023-07-14T16:07:14.920 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-14T16:07:14.920 INFO:journalctl@ceph.mon.b.smithi081.stdout: e1 2023-07-14T16:07:14.920 INFO:journalctl@ceph.mon.b.smithi081.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T16:07:14.921 INFO:journalctl@ceph.mon.b.smithi081.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T16:07:14.921 INFO:journalctl@ceph.mon.b.smithi081.stdout: legacy client fscid: -1 2023-07-14T16:07:14.921 INFO:journalctl@ceph.mon.b.smithi081.stdout: 2023-07-14T16:07:14.921 INFO:journalctl@ceph.mon.b.smithi081.stdout: No filesystems configured 2023-07-14T16:07:14.921 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T16:07:14.921 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T16:07:14.921 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T16:07:14.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T16:07:14.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T16:07:14.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T16:07:14.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T16:07:14.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:07:14.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:07:14.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T16:07:14.922 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:14.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:07:14.923 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:14.924 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:14.924 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Updating smithi081:/etc/ceph/ceph.conf 2023-07-14T16:07:14.924 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.925 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:14.925 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Deploying cephadm binary to smithi164 2023-07-14T16:07:14.925 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:07:14.925 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T16:07:14.925 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.925 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.925 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.925 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.926 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.926 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Added host smithi164 2023-07-14T16:07:14.926 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:14.926 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.926 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.926 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.926 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.927 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:07:14.927 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.927 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.927 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3437251395' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T16:07:14.927 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3437251395' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T16:07:14.927 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:07:14.927 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.927 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.928 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.928 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.928 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:07:14.928 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:14.928 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:14.928 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Updating smithi164:/etc/ceph/ceph.conf 2023-07-14T16:07:14.928 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:14.928 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.929 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:07:14.929 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T16:07:14.929 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.929 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.929 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.930 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.930 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:14.930 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:14.930 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:14.931 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.931 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:14.931 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:14.931 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.931 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:14.931 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c;count:3 2023-07-14T16:07:14.931 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Deploying daemon mon.c on smithi164 2023-07-14T16:07:14.931 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.932 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.932 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/4109426413' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:14.932 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:07:14.932 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:14.932 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.a calling monitor election 2023-07-14T16:07:14.932 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.932 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:14.933 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:14.933 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.c calling monitor election 2023-07-14T16:07:14.933 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.933 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:14.933 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:14.933 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.933 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:14.933 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T16:07:14.934 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: monmap e2: 2 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} 2023-07-14T16:07:14.934 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: fsmap 2023-07-14T16:07:14.934 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:07:14.934 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mgrmap e13: x(active, since 50s) 2023-07-14T16:07:14.934 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: overall HEALTH_OK 2023-07-14T16:07:14.934 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.934 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.934 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:14.935 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:14.935 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:14.935 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: Deploying daemon mon.b on smithi081 2023-07-14T16:07:14.935 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/1923903477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:14.935 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:14.935 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.935 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:14.935 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T16:07:14.936 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:14 smithi081 ceph-mon[127795]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-14T16:07:20.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:07:20.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: mon.a calling monitor election 2023-07-14T16:07:20.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: mon.c calling monitor election 2023-07-14T16:07:20.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:20.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:20.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: mon.b calling monitor election 2023-07-14T16:07:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T16:07:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0],c=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} 2023-07-14T16:07:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: fsmap 2023-07-14T16:07:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:07:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: mgrmap e13: x(active, since 60s) 2023-07-14T16:07:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: overall HEALTH_OK 2023-07-14T16:07:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:07:20.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: mon.a calling monitor election 2023-07-14T16:07:20.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: mon.c calling monitor election 2023-07-14T16:07:20.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:20.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: mon.b calling monitor election 2023-07-14T16:07:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T16:07:20.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0],c=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} 2023-07-14T16:07:20.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: fsmap 2023-07-14T16:07:20.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:07:20.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: mgrmap e13: x(active, since 60s) 2023-07-14T16:07:20.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: overall HEALTH_OK 2023-07-14T16:07:20.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:07:20.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: mon.a calling monitor election 2023-07-14T16:07:20.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: mon.c calling monitor election 2023-07-14T16:07:20.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:07:20.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:20.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: mon.b calling monitor election 2023-07-14T16:07:20.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:20.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:20.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T16:07:20.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0],c=[v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0]} removed_ranks: {} 2023-07-14T16:07:20.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: fsmap 2023-07-14T16:07:20.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T16:07:20.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: mgrmap e13: x(active, since 60s) 2023-07-14T16:07:20.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: overall HEALTH_OK 2023-07-14T16:07:20.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:20.663 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.687+0000 7fd21f46e700 1 -- 172.21.15.164:0/3569273457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 msgr2=0x7fd204008690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:20.663 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.687+0000 7fd21f46e700 1 --2- 172.21.15.164:0/3569273457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 0x7fd204008690 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd2181007f0 tx=0x7fd21400fe20 comp rx=0 tx=0).stop 2023-07-14T16:07:20.663 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.687+0000 7fd21f46e700 1 -- 172.21.15.164:0/3569273457 shutdown_connections 2023-07-14T16:07:20.664 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.687+0000 7fd21f46e700 1 --2- 172.21.15.164:0/3569273457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 0x7fd204008690 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:20.664 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.687+0000 7fd21f46e700 1 -- 172.21.15.164:0/3569273457 >> 172.21.15.164:0/3569273457 conn(0x7fd2180fb210 msgr2=0x7fd2180fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:20.664 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.687+0000 7fd21f46e700 1 -- 172.21.15.164:0/3569273457 shutdown_connections 2023-07-14T16:07:20.664 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.687+0000 7fd21f46e700 1 -- 172.21.15.164:0/3569273457 wait complete. 2023-07-14T16:07:20.666 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.688+0000 7fd21f46e700 1 Processor -- start 2023-07-14T16:07:20.666 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.688+0000 7fd21f46e700 1 -- start start 2023-07-14T16:07:20.667 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21f46e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 0x7fd218072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:20.667 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21f46e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd218072d50 0x7fd2180731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:20.667 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21f46e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2181a0e10 0x7fd2181a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:20.667 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21f46e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181a3840 con 0x7fd204006230 2023-07-14T16:07:20.668 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21f46e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181a39b0 con 0x7fd2181a0e10 2023-07-14T16:07:20.668 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21f46e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181a3b70 con 0x7fd218072d50 2023-07-14T16:07:20.668 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21ca09700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd218072d50 0x7fd2180731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:20.669 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21ca09700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd218072d50 0x7fd2180731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:46466/0 (socket says 172.21.15.164:46466) 2023-07-14T16:07:20.669 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21ca09700 1 -- 172.21.15.164:0/406793150 learned_addr learned my addr 172.21.15.164:0/406793150 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:07:20.669 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.689+0000 7fd21d20a700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 0x7fd218072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:20.669 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.690+0000 7fd21ca09700 1 -- 172.21.15.164:0/406793150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2181a0e10 msgr2=0x7fd2181a31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:20.670 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.690+0000 7fd21ca09700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2181a0e10 0x7fd2181a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:20.670 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.690+0000 7fd21ca09700 1 -- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 msgr2=0x7fd218072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:20.670 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.690+0000 7fd21ca09700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 0x7fd218072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:20.671 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.690+0000 7fd21ca09700 1 -- 172.21.15.164:0/406793150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21400f870 con 0x7fd218072d50 2023-07-14T16:07:20.671 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.690+0000 7fd21d20a700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 0x7fd218072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:07:20.671 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.690+0000 7fd21ca09700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd218072d50 0x7fd2180731c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd20800b780 tx=0x7fd20800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:20.673 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.692+0000 7fd20e7fc700 1 -- 172.21.15.164:0/406793150 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208004230 con 0x7fd218072d50 2023-07-14T16:07:20.673 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.693+0000 7fd20e7fc700 1 -- 172.21.15.164:0/406793150 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd208004390 con 0x7fd218072d50 2023-07-14T16:07:20.674 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.693+0000 7fd20e7fc700 1 -- 172.21.15.164:0/406793150 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208011a50 con 0x7fd218072d50 2023-07-14T16:07:20.674 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.693+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2181a3e50 con 0x7fd218072d50 2023-07-14T16:07:20.674 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.693+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2181a4350 con 0x7fd218072d50 2023-07-14T16:07:20.674 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.694+0000 7fd20e7fc700 1 -- 172.21.15.164:0/406793150 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd208011bb0 con 0x7fd218072d50 2023-07-14T16:07:20.674 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.694+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd218061b40 con 0x7fd218072d50 2023-07-14T16:07:20.674 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.695+0000 7fd20e7fc700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd1fc038c80 0x7fd1fc03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:20.674 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.695+0000 7fd20e7fc700 1 -- 172.21.15.164:0/406793150 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd20804ac10 con 0x7fd218072d50 2023-07-14T16:07:20.675 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.695+0000 7fd21d20a700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd1fc038c80 0x7fd1fc03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:20.675 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.696+0000 7fd21d20a700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd1fc038c80 0x7fd1fc03b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd2181007f0 tx=0x7fd214001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:20.677 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.701+0000 7fd20e7fc700 1 -- 172.21.15.164:0/406793150 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd208021760 con 0x7fd218072d50 2023-07-14T16:07:20.959 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.984+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd218073d40 con 0x7fd218072d50 2023-07-14T16:07:20.960 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.984+0000 7fd20e7fc700 1 -- 172.21.15.164:0/406793150 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1272 (secure 0 0 0) 0x7fd208011e60 con 0x7fd218072d50 2023-07-14T16:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: 2023-07-14T16:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","modified":"2023-07-14T16:07:14.617209Z","created":"2023-07-14T16:05:19.375259Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-07-14T16:07:20.963 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd1fc038c80 msgr2=0x7fd1fc03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:20.963 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd1fc038c80 0x7fd1fc03b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd2181007f0 tx=0x7fd214001e50 comp rx=0 tx=0).stop 2023-07-14T16:07:20.963 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd218072d50 msgr2=0x7fd2180731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:20.963 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd218072d50 0x7fd2180731c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd20800b780 tx=0x7fd20800ba90 comp rx=0 tx=0).stop 2023-07-14T16:07:20.963 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 shutdown_connections 2023-07-14T16:07:20.963 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd1fc038c80 0x7fd1fc03b130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:20.963 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2181a0e10 0x7fd2181a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:20.964 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd204006230 0x7fd218072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:20.964 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 --2- 172.21.15.164:0/406793150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd218072d50 0x7fd2180731c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:20.964 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 >> 172.21.15.164:0/406793150 conn(0x7fd2180fb210 msgr2=0x7fd218101370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:20.964 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 shutdown_connections 2023-07-14T16:07:20.964 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:20.987+0000 7fd21f46e700 1 -- 172.21.15.164:0/406793150 wait complete. 2023-07-14T16:07:20.964 INFO:teuthology.orchestra.run.smithi164.stderr:dumped monmap epoch 3 2023-07-14T16:07:20.984 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:20 smithi164 ceph-mon[97427]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:20.984 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:20.985 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:21.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:20 smithi062 ceph-mon[95543]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:21.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:21.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:21.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:20 smithi081 ceph-mon[127795]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:21.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:21.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:07:21.561 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-14T16:07:21.562 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph config generate-minimal-conf 2023-07-14T16:07:21.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:21 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/406793150' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:21.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:21 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/406793150' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:21.873 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:07:21 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:07:21.617+0000 7fdb7bd72700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-14T16:07:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:21 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/406793150' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T16:07:22.315 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:07:22.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: Updating smithi062:/etc/ceph/ceph.conf 2023-07-14T16:07:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: Updating smithi081:/etc/ceph/ceph.conf 2023-07-14T16:07:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: Updating smithi164:/etc/ceph/ceph.conf 2023-07-14T16:07:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:22 smithi062 ceph-mon[95543]: Updating smithi062:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:23.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:23.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:23.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: Updating smithi062:/etc/ceph/ceph.conf 2023-07-14T16:07:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: Updating smithi081:/etc/ceph/ceph.conf 2023-07-14T16:07:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: Updating smithi164:/etc/ceph/ceph.conf 2023-07-14T16:07:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:22 smithi164 ceph-mon[97427]: Updating smithi062:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:23.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:23.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:23.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: Updating smithi062:/etc/ceph/ceph.conf 2023-07-14T16:07:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: Updating smithi081:/etc/ceph/ceph.conf 2023-07-14T16:07:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: Updating smithi164:/etc/ceph/ceph.conf 2023-07-14T16:07:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: Updating smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: Updating smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:22 smithi081 ceph-mon[127795]: Updating smithi062:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T16:07:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.824+0000 7f8b14974700 1 -- 172.21.15.62:0/28729185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b100728b0 msgr2=0x7f8b10072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.824+0000 7f8b14974700 1 --2- 172.21.15.62:0/28729185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b100728b0 0x7f8b10072c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009910 tx=0x7f8b00009c20 comp rx=0 tx=0).stop 2023-07-14T16:07:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.825+0000 7f8b14974700 1 -- 172.21.15.62:0/28729185 shutdown_connections 2023-07-14T16:07:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.825+0000 7f8b14974700 1 --2- 172.21.15.62:0/28729185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b100728b0 0x7f8b10072c80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.825+0000 7f8b14974700 1 -- 172.21.15.62:0/28729185 >> 172.21.15.62:0/28729185 conn(0x7f8b100fb130 msgr2=0x7f8b100fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.825+0000 7f8b14974700 1 -- 172.21.15.62:0/28729185 shutdown_connections 2023-07-14T16:07:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.825+0000 7f8b14974700 1 -- 172.21.15.62:0/28729185 wait complete. 2023-07-14T16:07:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.826+0000 7f8b14974700 1 Processor -- start 2023-07-14T16:07:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.826+0000 7f8b14974700 1 -- start start 2023-07-14T16:07:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.826+0000 7f8b14974700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b100728b0 0x7f8b1019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.826+0000 7f8b14974700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1019b760 0x7f8b1019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.826+0000 7f8b14974700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b101a0110 0x7f8b101a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.826+0000 7f8b14974700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1019be30 con 0x7f8b1019b760 2023-07-14T16:07:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.826+0000 7f8b14974700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b101a7e80 con 0x7f8b101a0110 2023-07-14T16:07:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.826+0000 7f8b14974700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b101a7fc0 con 0x7f8b100728b0 2023-07-14T16:07:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1019b760 0x7f8b1019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1019b760 0x7f8b1019fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58716/0 (socket says 172.21.15.62:58716) 2023-07-14T16:07:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0dd9b700 1 -- 172.21.15.62:0/2765696564 learned_addr learned my addr 172.21.15.62:0/2765696564 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:07:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0ed9d700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b101a0110 0x7f8b101a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0e59c700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b100728b0 0x7f8b1019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0dd9b700 1 -- 172.21.15.62:0/2765696564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b100728b0 msgr2=0x7f8b1019b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0dd9b700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b100728b0 0x7f8b1019b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0dd9b700 1 -- 172.21.15.62:0/2765696564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b101a0110 msgr2=0x7f8b101a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0dd9b700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b101a0110 0x7f8b101a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.827+0000 7f8b0dd9b700 1 -- 172.21.15.62:0/2765696564 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b000095f0 con 0x7f8b1019b760 2023-07-14T16:07:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.828+0000 7f8b0dd9b700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1019b760 0x7f8b1019fbd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b7b0 tx=0x7f8b0400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.828+0000 7f8aff7fe700 1 -- 172.21.15.62:0/2765696564 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04004230 con 0x7f8b1019b760 2023-07-14T16:07:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.828+0000 7f8aff7fe700 1 -- 172.21.15.62:0/2765696564 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b04004390 con 0x7f8b1019b760 2023-07-14T16:07:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.828+0000 7f8aff7fe700 1 -- 172.21.15.62:0/2765696564 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04011a50 con 0x7f8b1019b760 2023-07-14T16:07:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.828+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b101a8270 con 0x7f8b1019b760 2023-07-14T16:07:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.828+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b101a8720 con 0x7f8b1019b760 2023-07-14T16:07:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.829+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b10061b40 con 0x7f8b1019b760 2023-07-14T16:07:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.830+0000 7f8aff7fe700 1 -- 172.21.15.62:0/2765696564 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8b04011bb0 con 0x7f8b1019b760 2023-07-14T16:07:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.830+0000 7f8aff7fe700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8af8038c30 0x7f8af803b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.830+0000 7f8aff7fe700 1 -- 172.21.15.62:0/2765696564 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8b0404a720 con 0x7f8b1019b760 2023-07-14T16:07:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.831+0000 7f8b0e59c700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8af8038c30 0x7f8af803b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.831+0000 7f8b0e59c700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8af8038c30 0x7f8af803b0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b00006010 tx=0x7f8b0000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:23.836+0000 7f8aff7fe700 1 -- 172.21.15.62:0/2765696564 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8b0400f8d0 con 0x7f8b1019b760 2023-07-14T16:07:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.055+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8b101a8b90 con 0x7f8b1019b760 2023-07-14T16:07:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.055+0000 7f8aff7fe700 1 -- 172.21.15.62:0/2765696564 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f8b04011e60 con 0x7f8b1019b760 2023-07-14T16:07:24.030 INFO:teuthology.orchestra.run.smithi062.stdout:# minimal ceph.conf for 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:07:24.031 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2023-07-14T16:07:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:07:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: mon_host = [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2023-07-14T16:07:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8af8038c30 msgr2=0x7f8af803b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8af8038c30 0x7f8af803b0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b00006010 tx=0x7f8b0000b540 comp rx=0 tx=0).stop 2023-07-14T16:07:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1019b760 msgr2=0x7f8b1019fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1019b760 0x7f8b1019fbd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b7b0 tx=0x7f8b0400bac0 comp rx=0 tx=0).stop 2023-07-14T16:07:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 shutdown_connections 2023-07-14T16:07:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8af8038c30 0x7f8af803b0e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b101a0110 0x7f8b101a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b100728b0 0x7f8b1019b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 --2- 172.21.15.62:0/2765696564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b1019b760 0x7f8b1019fbd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 >> 172.21.15.62:0/2765696564 conn(0x7f8b100fb130 msgr2=0x7f8b100fb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 shutdown_connections 2023-07-14T16:07:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:24.058+0000 7f8b14974700 1 -- 172.21.15.62:0/2765696564 wait complete. 2023-07-14T16:07:24.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:24.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T16:07:24.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:24.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:24.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:24.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: Reconfiguring daemon mon.a on smithi062 2023-07-14T16:07:24.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:24 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2765696564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:24.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:24.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T16:07:24.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:24.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:24.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:24.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: Reconfiguring daemon mon.a on smithi062 2023-07-14T16:07:24.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:24 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2765696564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:24.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:24.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:24.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T16:07:24.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:24.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:24.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:24.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: Reconfiguring daemon mon.a on smithi062 2023-07-14T16:07:24.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:24 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2765696564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:24.703 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-14T16:07:24.703 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:07:24.705 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T16:07:24.777 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:07:24.777 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:07:24.886 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:07:24.887 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T16:07:24.965 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:07:24.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:07:25.075 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:07:25.076 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T16:07:25.154 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:07:25.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T16:07:25.265 INFO:tasks.cephadm:Adding mgr.x on smithi062 2023-07-14T16:07:25.266 INFO:tasks.cephadm:Adding mgr.y on smithi081 2023-07-14T16:07:25.266 INFO:tasks.cephadm:Adding mgr.z on smithi164 2023-07-14T16:07:25.266 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch apply mgr '3;smithi062=x;smithi081=y;smithi164=z' 2023-07-14T16:07:25.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:25.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:25.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:25.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:25.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:25.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:25.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:25.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:25.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:25.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:26.021 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:07:26.616 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:26 smithi081 ceph-mon[127795]: Reconfiguring mon.b (monmap changed)... 2023-07-14T16:07:26.617 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:26 smithi081 ceph-mon[127795]: Reconfiguring daemon mon.b on smithi081 2023-07-14T16:07:26.617 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:26 smithi081 ceph-mon[127795]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:26.623 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:26 smithi164 ceph-mon[97427]: Reconfiguring mon.b (monmap changed)... 2023-07-14T16:07:26.623 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:26 smithi164 ceph-mon[97427]: Reconfiguring daemon mon.b on smithi081 2023-07-14T16:07:26.624 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:26 smithi164 ceph-mon[97427]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:26 smithi062 ceph-mon[95543]: Reconfiguring mon.b (monmap changed)... 2023-07-14T16:07:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:26 smithi062 ceph-mon[95543]: Reconfiguring daemon mon.b on smithi081 2023-07-14T16:07:26.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:26 smithi062 ceph-mon[95543]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:27.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.613+0000 7f110a526700 1 -- 172.21.15.164:0/1210356823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11040ff700 msgr2=0x7f11040ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:27.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.613+0000 7f110a526700 1 --2- 172.21.15.164:0/1210356823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11040ff700 0x7f11040ffb10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009910 tx=0x7f10f4009c20 comp rx=0 tx=0).stop 2023-07-14T16:07:27.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.613+0000 7f110a526700 1 -- 172.21.15.164:0/1210356823 shutdown_connections 2023-07-14T16:07:27.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.613+0000 7f110a526700 1 --2- 172.21.15.164:0/1210356823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11040ff700 0x7f11040ffb10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:27.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.613+0000 7f110a526700 1 -- 172.21.15.164:0/1210356823 >> 172.21.15.164:0/1210356823 conn(0x7f11040facd0 msgr2=0x7f11040fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:27.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.613+0000 7f110a526700 1 -- 172.21.15.164:0/1210356823 shutdown_connections 2023-07-14T16:07:27.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.613+0000 7f110a526700 1 -- 172.21.15.164:0/1210356823 wait complete. 2023-07-14T16:07:27.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.614+0000 7f110a526700 1 Processor -- start 2023-07-14T16:07:27.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f110a526700 1 -- start start 2023-07-14T16:07:27.591 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f110a526700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11040ff700 0x7f11041949c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f110a526700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1104194f00 0x7f1104199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f110a526700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1104195370 0x7f110419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f110a526700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110419cc80 con 0x7f1104194f00 2023-07-14T16:07:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f110a526700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110419cdf0 con 0x7f11040ff700 2023-07-14T16:07:27.592 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f110a526700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110419cfb0 con 0x7f1104195370 2023-07-14T16:07:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f11037fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1104194f00 0x7f1104199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.615+0000 7f1108ac3700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1104195370 0x7f110419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f11037fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1104194f00 0x7f1104199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.164:33592/0 (socket says 172.21.15.164:33592) 2023-07-14T16:07:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f11037fe700 1 -- 172.21.15.164:0/2033786936 learned_addr learned my addr 172.21.15.164:0/2033786936 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:07:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1108ac3700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1104195370 0x7f110419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36314/0 (socket says 172.21.15.164:36314) 2023-07-14T16:07:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1103fff700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11040ff700 0x7f11041949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:27.593 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1108ac3700 1 -- 172.21.15.164:0/2033786936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11040ff700 msgr2=0x7f11041949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:27.594 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1108ac3700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11040ff700 0x7f11041949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:27.594 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1108ac3700 1 -- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1104194f00 msgr2=0x7f1104199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:27.594 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1108ac3700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1104194f00 0x7f1104199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:27.594 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1108ac3700 1 -- 172.21.15.164:0/2033786936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f40095f0 con 0x7f1104195370 2023-07-14T16:07:27.594 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f11037fe700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1104194f00 0x7f1104199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:07:27.594 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1103fff700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11040ff700 0x7f11041949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:07:27.594 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f1108ac3700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1104195370 0x7f110419c4e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00bd50 tx=0x7f10fc00bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:27.595 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f11017fa700 1 -- 172.21.15.164:0/2033786936 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00cc90 con 0x7f1104195370 2023-07-14T16:07:27.595 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.617+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110419d290 con 0x7f1104195370 2023-07-14T16:07:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.616+0000 7f11017fa700 1 -- 172.21.15.164:0/2033786936 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10fc00cdf0 con 0x7f1104195370 2023-07-14T16:07:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.617+0000 7f11017fa700 1 -- 172.21.15.164:0/2033786936 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc012670 con 0x7f1104195370 2023-07-14T16:07:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.617+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110419d7c0 con 0x7f1104195370 2023-07-14T16:07:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.618+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110418e180 con 0x7f1104195370 2023-07-14T16:07:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.618+0000 7f11017fa700 1 -- 172.21.15.164:0/2033786936 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f10fc014440 con 0x7f1104195370 2023-07-14T16:07:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.618+0000 7f11017fa700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10ec038c80 0x7f10ec03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:27.596 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.618+0000 7f11017fa700 1 -- 172.21.15.164:0/2033786936 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f10fc04ac40 con 0x7f1104195370 2023-07-14T16:07:27.597 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.619+0000 7f1103fff700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10ec038c80 0x7f10ec03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:27.597 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.619+0000 7f1103fff700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10ec038c80 0x7f10ec03b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009fd0 tx=0x7f10f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:27.599 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.624+0000 7f11017fa700 1 -- 172.21.15.164:0/2033786936 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f10fc014a60 con 0x7f1104195370 2023-07-14T16:07:27.848 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.873+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi081=y;smithi164=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f11040609e0 con 0x7f10ec038c80 2023-07-14T16:07:27.860 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.885+0000 7f11017fa700 1 -- 172.21.15.164:0/2033786936 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f11040609e0 con 0x7f10ec038c80 2023-07-14T16:07:27.861 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mgr update... 2023-07-14T16:07:27.863 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10ec038c80 msgr2=0x7f10ec03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10ec038c80 0x7f10ec03b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009fd0 tx=0x7f10f40058e0 comp rx=0 tx=0).stop 2023-07-14T16:07:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1104195370 msgr2=0x7f110419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1104195370 0x7f110419c4e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00bd50 tx=0x7f10fc00bd80 comp rx=0 tx=0).stop 2023-07-14T16:07:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 shutdown_connections 2023-07-14T16:07:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1104195370 0x7f110419c4e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10ec038c80 0x7f10ec03b130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11040ff700 0x7f11041949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:27.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 --2- 172.21.15.164:0/2033786936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1104194f00 0x7f1104199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:27.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 >> 172.21.15.164:0/2033786936 conn(0x7f11040facd0 msgr2=0x7f11040fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:27.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 shutdown_connections 2023-07-14T16:07:27.865 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:07:27.888+0000 7f110a526700 1 -- 172.21.15.164:0/2033786936 wait complete. 2023-07-14T16:07:28.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:28.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:28.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:27 smithi164 ceph-mon[97427]: Reconfiguring mon.c (monmap changed)... 2023-07-14T16:07:28.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:28.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:28.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:28.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:27 smithi164 ceph-mon[97427]: Reconfiguring daemon mon.c on smithi164 2023-07-14T16:07:28.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:27 smithi164 ceph-mon[97427]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:28.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:28.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:28.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:27 smithi062 ceph-mon[95543]: Reconfiguring mon.c (monmap changed)... 2023-07-14T16:07:28.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:28.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:28.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:28.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:27 smithi062 ceph-mon[95543]: Reconfiguring daemon mon.c on smithi164 2023-07-14T16:07:28.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:27 smithi062 ceph-mon[95543]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:28.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:28.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:28.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:27 smithi081 ceph-mon[127795]: Reconfiguring mon.c (monmap changed)... 2023-07-14T16:07:28.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T16:07:28.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T16:07:28.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:28.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:27 smithi081 ceph-mon[127795]: Reconfiguring daemon mon.c on smithi164 2023-07-14T16:07:28.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:27 smithi081 ceph-mon[127795]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:28.708 DEBUG:teuthology.orchestra.run.smithi081:mgr.y> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.y.service 2023-07-14T16:07:28.713 DEBUG:teuthology.orchestra.run.smithi164:mgr.z> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.z.service 2023-07-14T16:07:28.717 INFO:tasks.cephadm:Deploying OSDs... 2023-07-14T16:07:28.717 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:07:28.718 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T16:07:28.766 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T16:07:28.767 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2023-07-14T16:07:28.790 INFO:journalctl@ceph.mgr.y.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:07:28.792 INFO:journalctl@ceph.mgr.z.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:07:28.855 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T16:07:28.856 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:28.856 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 450 Links: 1 2023-07-14T16:07:28.856 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:28.856 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:28.856 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-07-14 16:06:30.489973523 +0000 2023-07-14T16:07:28.856 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-07-14 15:58:53.222011042 +0000 2023-07-14T16:07:28.857 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-07-14 15:58:53.222011042 +0000 2023-07-14T16:07:28.857 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2023-07-14 15:58:53.195011484 +0000 2023-07-14T16:07:28.857 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T16:07:28.930 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-07-14T16:07:28.930 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-07-14T16:07:28.931 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000406182 s, 1.3 MB/s 2023-07-14T16:07:28.933 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T16:07:28.979 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2023-07-14T16:07:29.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:28 smithi164 ceph-mon[97427]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi081=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:29.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:28 smithi164 ceph-mon[97427]: Saving service mgr spec with placement smithi062=x;smithi081=y;smithi164=z;count:3 2023-07-14T16:07:29.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:28 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:28 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:28 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.026 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T16:07:29.026 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:29.026 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 456 Links: 1 2023-07-14T16:07:29.026 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:29.027 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:29.027 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-07-14 16:06:30.489973523 +0000 2023-07-14T16:07:29.027 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-07-14 15:58:54.068997180 +0000 2023-07-14T16:07:29.027 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-07-14 15:58:54.068997180 +0000 2023-07-14T16:07:29.028 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2023-07-14 15:58:54.041997622 +0000 2023-07-14T16:07:29.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T16:07:29.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:28 smithi062 ceph-mon[95543]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi081=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:29.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:28 smithi062 ceph-mon[95543]: Saving service mgr spec with placement smithi062=x;smithi081=y;smithi164=z;count:3 2023-07-14T16:07:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:28 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:28 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:28 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-07-14T16:07:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-07-14T16:07:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000287503 s, 1.8 MB/s 2023-07-14T16:07:29.135 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T16:07:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:28 smithi081 ceph-mon[127795]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi081=y;smithi164=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:28 smithi081 ceph-mon[127795]: Saving service mgr spec with placement smithi062=x;smithi081=y;smithi164=z;count:3 2023-07-14T16:07:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:28 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:28 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:28 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:29.181 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2023-07-14T16:07:29.265 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T16:07:29.266 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:29.266 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 462 Links: 1 2023-07-14T16:07:29.266 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:29.266 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:29.266 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-07-14 16:06:30.489973523 +0000 2023-07-14T16:07:29.267 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-07-14 15:58:54.916983302 +0000 2023-07-14T16:07:29.267 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-07-14 15:58:54.916983302 +0000 2023-07-14T16:07:29.267 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2023-07-14 15:58:54.888983760 +0000 2023-07-14T16:07:29.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T16:07:29.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-07-14T16:07:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-07-14T16:07:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000291055 s, 1.8 MB/s 2023-07-14T16:07:29.376 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T16:07:29.423 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2023-07-14T16:07:29.508 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T16:07:29.508 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:29.508 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 468 Links: 1 2023-07-14T16:07:29.509 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:29.509 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:29.509 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-07-14 16:06:30.489973523 +0000 2023-07-14T16:07:29.509 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-07-14 15:58:55.774969260 +0000 2023-07-14T16:07:29.509 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-07-14 15:58:55.774969260 +0000 2023-07-14T16:07:29.509 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2023-07-14 15:58:55.746969718 +0000 2023-07-14T16:07:29.510 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T16:07:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-07-14T16:07:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-07-14T16:07:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000313321 s, 1.6 MB/s 2023-07-14T16:07:29.625 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T16:07:29.671 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-07-14T16:07:29.671 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T16:07:29.719 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T16:07:29.719 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2023-07-14T16:07:29.806 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T16:07:29.806 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:29.806 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 446 Links: 1 2023-07-14T16:07:29.806 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:29.807 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:29.807 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2023-07-14 16:07:13.717807924 +0000 2023-07-14T16:07:29.807 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2023-07-14 15:59:05.353330995 +0000 2023-07-14T16:07:29.807 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2023-07-14 15:59:05.353330995 +0000 2023-07-14T16:07:29.807 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2023-07-14 15:59:05.326331518 +0000 2023-07-14T16:07:29.807 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T16:07:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2023-07-14T16:07:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2023-07-14T16:07:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000468382 s, 1.1 MB/s 2023-07-14T16:07:29.916 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T16:07:29.962 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2023-07-14T16:07:30.046 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T16:07:30.046 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:30.046 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-07-14T16:07:30.046 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:30.047 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:30.047 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2023-07-14 16:07:13.717807924 +0000 2023-07-14T16:07:30.047 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2023-07-14 15:59:06.193314719 +0000 2023-07-14T16:07:30.047 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2023-07-14 15:59:06.193314719 +0000 2023-07-14T16:07:30.047 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2023-07-14 15:59:06.165315262 +0000 2023-07-14T16:07:30.047 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T16:07:30.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:30.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:30.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:30.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:30.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:30.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:30.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T16:07:30.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:30.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:30.144 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:30.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:30.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:30.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:30.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:30.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:30.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T16:07:30.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:30.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:30.150 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2023-07-14T16:07:30.151 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2023-07-14T16:07:30.151 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000306584 s, 1.7 MB/s 2023-07-14T16:07:30.154 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T16:07:30.200 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2023-07-14T16:07:30.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:30.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:30.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:30.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:30.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:30.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:30.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T16:07:30.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:30.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:30.283 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T16:07:30.283 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:30.283 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-07-14T16:07:30.284 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:30.284 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:30.284 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2023-07-14 16:07:13.717807924 +0000 2023-07-14T16:07:30.284 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2023-07-14 15:59:07.037298366 +0000 2023-07-14T16:07:30.284 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2023-07-14 15:59:07.037298366 +0000 2023-07-14T16:07:30.284 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2023-07-14 15:59:07.011298870 +0000 2023-07-14T16:07:30.285 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T16:07:30.388 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2023-07-14T16:07:30.388 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2023-07-14T16:07:30.389 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000280353 s, 1.8 MB/s 2023-07-14T16:07:30.391 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T16:07:30.435 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2023-07-14T16:07:30.519 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T16:07:30.519 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:30.519 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-07-14T16:07:30.519 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:30.519 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:30.519 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2023-07-14 16:07:13.717807924 +0000 2023-07-14T16:07:30.520 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2023-07-14 15:59:07.851282593 +0000 2023-07-14T16:07:30.520 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2023-07-14 15:59:07.851282593 +0000 2023-07-14T16:07:30.520 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2023-07-14 15:59:07.825283097 +0000 2023-07-14T16:07:30.520 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T16:07:30.623 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2023-07-14T16:07:30.624 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2023-07-14T16:07:30.624 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000334568 s, 1.5 MB/s 2023-07-14T16:07:30.626 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T16:07:30.670 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2023-07-14T16:07:30.671 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T16:07:30.717 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T16:07:30.718 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2023-07-14T16:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T16:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 446 Links: 1 2023-07-14T16:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2023-07-14 16:07:04.087815677 +0000 2023-07-14T16:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2023-07-14 15:59:06.769276554 +0000 2023-07-14T16:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2023-07-14 15:59:06.769276554 +0000 2023-07-14T16:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2023-07-14 15:59:06.741277032 +0000 2023-07-14T16:07:30.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T16:07:30.911 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2023-07-14T16:07:30.911 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2023-07-14T16:07:30.911 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000272076 s, 1.9 MB/s 2023-07-14T16:07:30.913 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T16:07:30.960 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2023-07-14T16:07:31.009 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T16:07:31.009 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:31.010 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-07-14T16:07:31.010 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:31.010 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:31.010 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2023-07-14 16:07:04.087815677 +0000 2023-07-14T16:07:31.011 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2023-07-14 15:59:07.607262258 +0000 2023-07-14T16:07:31.011 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2023-07-14 15:59:07.607262258 +0000 2023-07-14T16:07:31.011 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2023-07-14 15:59:07.580262718 +0000 2023-07-14T16:07:31.011 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T16:07:31.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:30 smithi164 ceph-mon[97427]: Deploying daemon mgr.z on smithi164 2023-07-14T16:07:31.089 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2023-07-14T16:07:31.089 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2023-07-14T16:07:31.090 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000293664 s, 1.7 MB/s 2023-07-14T16:07:31.092 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T16:07:31.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:30 smithi062 ceph-mon[95543]: Deploying daemon mgr.z on smithi164 2023-07-14T16:07:31.137 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2023-07-14T16:07:31.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:30 smithi081 ceph-mon[127795]: Deploying daemon mgr.z on smithi164 2023-07-14T16:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T16:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-07-14T16:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2023-07-14 16:07:04.087815677 +0000 2023-07-14T16:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2023-07-14 15:59:08.611245128 +0000 2023-07-14T16:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2023-07-14 15:59:08.611245128 +0000 2023-07-14T16:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2023-07-14 15:59:08.585245571 +0000 2023-07-14T16:07:31.222 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T16:07:31.326 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2023-07-14T16:07:31.326 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2023-07-14T16:07:31.326 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000323252 s, 1.6 MB/s 2023-07-14T16:07:31.328 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T16:07:31.374 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2023-07-14T16:07:31.458 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T16:07:31.459 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T16:07:31.459 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-07-14T16:07:31.459 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:07:31.459 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T16:07:31.459 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2023-07-14 16:07:04.088815659 +0000 2023-07-14T16:07:31.459 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2023-07-14 15:59:09.449230832 +0000 2023-07-14T16:07:31.460 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2023-07-14 15:59:09.449230832 +0000 2023-07-14T16:07:31.460 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2023-07-14 15:59:09.422231293 +0000 2023-07-14T16:07:31.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T16:07:31.766 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2023-07-14T16:07:31.766 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2023-07-14T16:07:31.767 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000297032 s, 1.7 MB/s 2023-07-14T16:07:31.770 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T16:07:31.819 INFO:tasks.cephadm:Deploying osd.0 on smithi062 with /dev/vg_nvme/lv_4... 2023-07-14T16:07:31.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T16:07:32.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:31 smithi164 ceph-mon[97427]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:32.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:31 smithi081 ceph-mon[127795]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:32.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:31 smithi062 ceph-mon[95543]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:32.523 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:32 smithi164 systemd[1]: Starting Ceph mgr.z for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:07:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:07:32.820 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:32 smithi164 podman[99531]: 2023-07-14T16:07:32.820 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:32 smithi164 ceph-mgr[99579]: -- 172.21.15.164:0/415908930 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c5677ef1e0 con 0x55c5677c4800 2023-07-14T16:07:33.070 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:32 smithi164 bash[99531]: a6854e882f80e9c77745d50d1bacc01681d8fe8f4d261bb90ce27296e0ebdc68 2023-07-14T16:07:33.070 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:32 smithi164 systemd[1]: Started Ceph mgr.z for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:07:33.071 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:32 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:32.989+0000 7ff226f5d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T16:07:33.520 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:33.094+0000 7ff226f5d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T16:07:33.520 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:33.209+0000 7ff226f5d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T16:07:33.520 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:33.327+0000 7ff226f5d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T16:07:33.520 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:33.544+0000 7ff226f5d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T16:07:34.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:34.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T16:07:34.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:34.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:34.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: Deploying daemon mgr.y on smithi081 2023-07-14T16:07:34.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:33 smithi062 ceph-mon[95543]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:34.196 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:33.941+0000 7ff226f5d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T16:07:34.196 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:34 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:34.059+0000 7ff226f5d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T16:07:34.197 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.197 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.197 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.198 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:34.198 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T16:07:34.198 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:34.198 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:34.198 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: Deploying daemon mgr.y on smithi081 2023-07-14T16:07:34.198 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:33 smithi164 ceph-mon[97427]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:34.239 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.240 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.240 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:34.240 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:34.240 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T16:07:34.240 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:34.241 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:34.241 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: Deploying daemon mgr.y on smithi081 2023-07-14T16:07:34.241 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:33 smithi081 ceph-mon[127795]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:34.452 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T16:07:34.453 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T16:07:34.453 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2023-07-14T16:07:34.453 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2023-07-14T16:07:34.454 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0306243 s, 342 MB/s 2023-07-14T16:07:34.454 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2023-07-14T16:07:34.523 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:34 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:34.220+0000 7ff226f5d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T16:07:34.523 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:34 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:34.343+0000 7ff226f5d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T16:07:34.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2023-07-14T16:07:35.273 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:34 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:34.887+0000 7ff226f5d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T16:07:35.401 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:35 smithi081 systemd[1]: Starting Ceph mgr.y for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:07:35.563 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:07:35.957 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:35 smithi081 podman[129237]: 2023-07-14T16:07:36.023 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:35 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:35.664+0000 7ff226f5d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T16:07:36.024 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:35 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:35.874+0000 7ff226f5d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T16:07:36.237 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:35 smithi081 bash[129237]: 5ad013c04f616f3baa0d2c01b4f4c64037a357b2f3240ddc20945f1ae24151be 2023-07-14T16:07:36.237 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:35 smithi081 ceph-mgr[129277]: -- 172.21.15.81:0/475742105 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e0743e91e0 con 0x55e0743bec00 2023-07-14T16:07:36.237 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:36 smithi081 systemd[1]: Started Ceph mgr.y for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:07:36.237 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:36.159+0000 7f5507172200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T16:07:36.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-mon[127795]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:36.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.652 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:36.261+0000 7f5507172200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T16:07:36.652 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:36.380+0000 7f5507172200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T16:07:36.652 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:36.498+0000 7f5507172200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T16:07:36.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:36 smithi164 ceph-mon[97427]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:36.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.809 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:36 smithi062 ceph-mon[95543]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:36.809 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.809 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.809 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:36.809 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:37.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 -- 172.21.15.62:0/3201056081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4107a60 msgr2=0x7fedf4109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:37.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/3201056081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4107a60 0x7fedf4109ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b0a0 tx=0x7fedec00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:07:37.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 -- 172.21.15.62:0/3201056081 shutdown_connections 2023-07-14T16:07:37.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/3201056081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4107a60 0x7fedf4109ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:37.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/3201056081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf4074540 0x7fedf40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:37.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/3201056081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf4072f70 0x7fedf40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:37.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 -- 172.21.15.62:0/3201056081 >> 172.21.15.62:0/3201056081 conn(0x7fedf40fb1a0 msgr2=0x7fedf40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:37.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 -- 172.21.15.62:0/3201056081 shutdown_connections 2023-07-14T16:07:37.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.081+0000 7fedfa5f4700 1 -- 172.21.15.62:0/3201056081 wait complete. 2023-07-14T16:07:37.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.082+0000 7fedfa5f4700 1 Processor -- start 2023-07-14T16:07:37.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.082+0000 7fedfa5f4700 1 -- start start 2023-07-14T16:07:37.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedfa5f4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf4072f70 0x7fedf4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:37.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedfa5f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4074540 0x7fedf4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedfa5f4700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf4107a60 0x7fedf419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedfa5f4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf419f620 con 0x7fedf4074540 2023-07-14T16:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedfa5f4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf419f760 con 0x7fedf4072f70 2023-07-14T16:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedfa5f4700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf419f8f0 con 0x7fedf4107a60 2023-07-14T16:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf4072f70 0x7fedf4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4074540 0x7fedf4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf4072f70 0x7fedf4197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:60650/0 (socket says 172.21.15.62:60650) 2023-07-14T16:07:37.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf3fff700 1 -- 172.21.15.62:0/1964396607 learned_addr learned my addr 172.21.15.62:0/1964396607 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:07:37.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf8b91700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf4107a60 0x7fedf419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:37.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf37fe700 1 -- 172.21.15.62:0/1964396607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf4107a60 msgr2=0x7fedf419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:37.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf37fe700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf4107a60 0x7fedf419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:37.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf37fe700 1 -- 172.21.15.62:0/1964396607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf4072f70 msgr2=0x7fedf4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:37.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf37fe700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf4072f70 0x7fedf4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:37.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf37fe700 1 -- 172.21.15.62:0/1964396607 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec009d00 con 0x7fedf4074540 2023-07-14T16:07:37.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.083+0000 7fedf3fff700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf4072f70 0x7fedf4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:07:37.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.084+0000 7fedf37fe700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4074540 0x7fedf4197fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fede800ea20 tx=0x7fede800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:37.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.084+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede800cbd0 con 0x7fedf4074540 2023-07-14T16:07:37.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.084+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf419fbd0 con 0x7fedf4074540 2023-07-14T16:07:37.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.084+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fede800ee60 con 0x7fedf4074540 2023-07-14T16:07:37.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.084+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8018420 con 0x7fedf4074540 2023-07-14T16:07:37.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.084+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf41a0120 con 0x7fedf4074540 2023-07-14T16:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.085+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf4106110 con 0x7fedf4074540 2023-07-14T16:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.085+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fede8010430 con 0x7fedf4074540 2023-07-14T16:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.086+0000 7fedf17fa700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feddc0389e0 0x7feddc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:07:37.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.086+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fede8014070 con 0x7fedf4074540 2023-07-14T16:07:37.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.086+0000 7fedf3fff700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feddc0389e0 0x7feddc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:07:37.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.087+0000 7fedf3fff700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feddc0389e0 0x7feddc03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fede4005950 tx=0x7fede400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:07:37.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.091+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fede8016330 con 0x7fedf4074540 2023-07-14T16:07:37.130 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:36 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:36.729+0000 7f5507172200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T16:07:37.130 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:37 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:37.153+0000 7f5507172200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T16:07:37.180 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:36 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:36.849+0000 7ff226f5d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:07:37.180 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:36 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:36.973+0000 7ff226f5d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T16:07:37.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:37.310+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fedf4060950 con 0x7feddc0389e0 2023-07-14T16:07:37.401 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:37 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:37.274+0000 7f5507172200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T16:07:37.523 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:37 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:37.205+0000 7ff226f5d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:07:37.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:37.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:07:37.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:07:37.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:37.848 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:37 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:37.583+0000 7ff226f5d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T16:07:37.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:37.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:07:37.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:07:37.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:37.900 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:37 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:37.444+0000 7f5507172200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T16:07:37.901 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:37 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:37.585+0000 7f5507172200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T16:07:37.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:37.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:07:37.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:07:37.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:38.273 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:37 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:37.873+0000 7ff226f5d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T16:07:38.273 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:38 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:38.075+0000 7ff226f5d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T16:07:38.400 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:38 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:38.160+0000 7f5507172200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T16:07:38.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:38 smithi062 ceph-mon[95543]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:38.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:38 smithi062 ceph-mon[95543]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:38.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:38 smithi164 ceph-mon[97427]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:38.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:38 smithi164 ceph-mon[97427]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:38.773 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:38 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:38.451+0000 7ff226f5d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T16:07:38.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:38 smithi081 ceph-mon[127795]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:07:38.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:38 smithi081 ceph-mon[127795]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:39.400 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:38 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:38.953+0000 7f5507172200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T16:07:39.401 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:39.156+0000 7f5507172200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T16:07:39.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:39.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:39.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:39.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:39.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:39.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.773 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:39.385+0000 7ff226f5d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T16:07:39.773 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:39.557+0000 7ff226f5d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T16:07:39.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:39.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:39.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:39.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:39.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:39.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:39.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:39.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:39.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T16:07:39.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T16:07:39.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:39.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:40.196 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:39.800+0000 7ff226f5d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T16:07:40.196 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:39 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:39.948+0000 7ff226f5d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T16:07:40.196 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:40 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:40.221+0000 7ff226f5d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T16:07:40.398 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:40.087+0000 7f5507172200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:07:40.398 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:40.206+0000 7f5507172200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T16:07:40.650 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:40.422+0000 7f5507172200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T16:07:40.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-mon[127795]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T16:07:40.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-mon[127795]: Reconfiguring daemon mgr.x on smithi062 2023-07-14T16:07:40.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-mon[127795]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:40.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2701731108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae1fdd9-0e93-4148-af93-87c47080d59d"}]: dispatch 2023-07-14T16:07:40.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2701731108' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ae1fdd9-0e93-4148-af93-87c47080d59d"}]': finished 2023-07-14T16:07:40.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-mon[127795]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T16:07:40.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:07:40.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:40 smithi062 ceph-mon[95543]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T16:07:40.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:40 smithi062 ceph-mon[95543]: Reconfiguring daemon mgr.x on smithi062 2023-07-14T16:07:40.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:40 smithi062 ceph-mon[95543]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:40.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:40 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2701731108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae1fdd9-0e93-4148-af93-87c47080d59d"}]: dispatch 2023-07-14T16:07:40.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:40 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2701731108' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ae1fdd9-0e93-4148-af93-87c47080d59d"}]': finished 2023-07-14T16:07:40.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:40 smithi062 ceph-mon[95543]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T16:07:40.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:07:41.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:40 smithi164 ceph-mon[97427]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T16:07:41.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:40 smithi164 ceph-mon[97427]: Reconfiguring daemon mgr.x on smithi062 2023-07-14T16:07:41.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:40 smithi164 ceph-mon[97427]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:41.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:40 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2701731108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae1fdd9-0e93-4148-af93-87c47080d59d"}]: dispatch 2023-07-14T16:07:41.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:40 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2701731108' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ae1fdd9-0e93-4148-af93-87c47080d59d"}]': finished 2023-07-14T16:07:41.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:40 smithi164 ceph-mon[97427]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T16:07:41.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:07:41.024 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:40 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:40.776+0000 7f5507172200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T16:07:41.400 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:41.048+0000 7f5507172200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T16:07:41.401 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:41.236+0000 7f5507172200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T16:07:41.523 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:07:41 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z[99555]: 2023-07-14T16:07:41.298+0000 7ff226f5d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T16:07:41.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:41.599+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7fede8010830 con 0x7fedf4074540 2023-07-14T16:07:41.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:41 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/669392921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:07:41.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:41 smithi062 ceph-mon[95543]: Standby manager daemon z started 2023-07-14T16:07:41.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:41 smithi062 ceph-mon[95543]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T16:07:41.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:41 smithi062 ceph-mon[95543]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T16:07:41.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:41 smithi062 ceph-mon[95543]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T16:07:41.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:41 smithi062 ceph-mon[95543]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T16:07:41.900 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:41.571+0000 7f5507172200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T16:07:41.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/669392921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:07:41.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-mon[127795]: Standby manager daemon z started 2023-07-14T16:07:41.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-mon[127795]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T16:07:41.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-mon[127795]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T16:07:41.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-mon[127795]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T16:07:41.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:41 smithi081 ceph-mon[127795]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T16:07:42.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:41 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/669392921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:07:42.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:41 smithi164 ceph-mon[97427]: Standby manager daemon z started 2023-07-14T16:07:42.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:41 smithi164 ceph-mon[97427]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T16:07:42.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:41 smithi164 ceph-mon[97427]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T16:07:42.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:41 smithi164 ceph-mon[97427]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T16:07:42.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:41 smithi164 ceph-mon[97427]: from='mgr.? 172.21.15.164:0/501592770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T16:07:42.863 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:42 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:42.480+0000 7f5507172200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T16:07:42.863 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:42 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:42.649+0000 7f5507172200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T16:07:42.864 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:42 smithi081 ceph-mon[127795]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:42.864 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:42 smithi081 ceph-mon[127795]: mgrmap e14: x(active, since 82s), standbys: z 2023-07-14T16:07:42.864 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T16:07:42.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:42 smithi062 ceph-mon[95543]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:42.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:42 smithi062 ceph-mon[95543]: mgrmap e14: x(active, since 82s), standbys: z 2023-07-14T16:07:42.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T16:07:43.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:42 smithi164 ceph-mon[97427]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:43.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:42 smithi164 ceph-mon[97427]: mgrmap e14: x(active, since 82s), standbys: z 2023-07-14T16:07:43.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T16:07:43.150 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:42 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:42.888+0000 7f5507172200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T16:07:43.151 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:43 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:43.036+0000 7f5507172200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T16:07:43.650 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:43 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:43.308+0000 7f5507172200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T16:07:44.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:43 smithi062 ceph-mon[95543]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:44.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:43 smithi081 ceph-mon[127795]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:44.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:43 smithi164 ceph-mon[97427]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:44.650 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:07:44 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y[129253]: 2023-07-14T16:07:44.356+0000 7f5507172200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T16:07:44.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:44.903+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fede807d4e0 con 0x7fedf4074540 2023-07-14T16:07:45.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:44 smithi062 ceph-mon[95543]: Standby manager daemon y started 2023-07-14T16:07:45.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:44 smithi062 ceph-mon[95543]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T16:07:45.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:44 smithi062 ceph-mon[95543]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T16:07:45.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:44 smithi062 ceph-mon[95543]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T16:07:45.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:44 smithi062 ceph-mon[95543]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T16:07:45.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:44 smithi081 ceph-mon[127795]: Standby manager daemon y started 2023-07-14T16:07:45.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:44 smithi081 ceph-mon[127795]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T16:07:45.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:44 smithi081 ceph-mon[127795]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T16:07:45.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:44 smithi081 ceph-mon[127795]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T16:07:45.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:44 smithi081 ceph-mon[127795]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T16:07:45.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:44 smithi164 ceph-mon[97427]: Standby manager daemon y started 2023-07-14T16:07:45.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:44 smithi164 ceph-mon[97427]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T16:07:45.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:44 smithi164 ceph-mon[97427]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T16:07:45.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:44 smithi164 ceph-mon[97427]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T16:07:45.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:44 smithi164 ceph-mon[97427]: from='mgr.? 172.21.15.81:0/1663576139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T16:07:46.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:45 smithi062 ceph-mon[95543]: mgrmap e15: x(active, since 85s), standbys: z, y 2023-07-14T16:07:46.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T16:07:46.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:45 smithi062 ceph-mon[95543]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:46.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:46.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:46.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:45 smithi081 ceph-mon[127795]: mgrmap e15: x(active, since 85s), standbys: z, y 2023-07-14T16:07:46.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T16:07:46.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:45 smithi081 ceph-mon[127795]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:46.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:46.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:46.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:45 smithi164 ceph-mon[97427]: mgrmap e15: x(active, since 85s), standbys: z, y 2023-07-14T16:07:46.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T16:07:46.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:45 smithi164 ceph-mon[97427]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:46.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:46.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:47.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:47.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:47.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:47.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:47.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:47.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:47.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:47.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:48.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:47 smithi062 ceph-mon[95543]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:48.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:47 smithi081 ceph-mon[127795]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:48.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:47 smithi164 ceph-mon[97427]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:49.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T16:07:49.148 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:49.148 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:48 smithi062 ceph-mon[95543]: Deploying daemon osd.0 on smithi062 2023-07-14T16:07:49.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T16:07:49.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:49.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:48 smithi081 ceph-mon[127795]: Deploying daemon osd.0 on smithi062 2023-07-14T16:07:49.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T16:07:49.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:49.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:48 smithi164 ceph-mon[97427]: Deploying daemon osd.0 on smithi062 2023-07-14T16:07:50.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:49 smithi062 ceph-mon[95543]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:50.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:49 smithi081 ceph-mon[127795]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:50.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:49 smithi164 ceph-mon[97427]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:52.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:52 smithi062 ceph-mon[95543]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:52.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:52 smithi164 ceph-mon[97427]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:52.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:52 smithi081 ceph-mon[127795]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:54.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:53 smithi062 ceph-mon[95543]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:54.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:53 smithi081 ceph-mon[127795]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:54.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:53 smithi164 ceph-mon[97427]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:55.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:55.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:55.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:55.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:55.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:55.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:55.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:55.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:55.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:56.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:56 smithi062 ceph-mon[95543]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:56.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:56 smithi081 ceph-mon[127795]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:56.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:56 smithi164 ceph-mon[97427]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:57.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:57.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:57.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:57.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:57.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:57.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:57 smithi062 ceph-mon[95543]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:58.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:58.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:58.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:58.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:58.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:58.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:57 smithi164 ceph-mon[97427]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.137+0000 7fedf17fa700 1 -- 172.21.15.62:0/1964396607 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fedf4060950 con 0x7feddc0389e0 2023-07-14T16:07:58.113 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 0 on host 'smithi062' 2023-07-14T16:07:58.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feddc0389e0 msgr2=0x7feddc03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feddc0389e0 0x7feddc03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fede4005950 tx=0x7fede400b500 comp rx=0 tx=0).stop 2023-07-14T16:07:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4074540 msgr2=0x7fedf4197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:07:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4074540 0x7fedf4197fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fede800ea20 tx=0x7fede800ed30 comp rx=0 tx=0).stop 2023-07-14T16:07:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 shutdown_connections 2023-07-14T16:07:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf4107a60 0x7fedf419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feddc0389e0 0x7feddc03ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf4072f70 0x7fedf4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 --2- 172.21.15.62:0/1964396607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf4074540 0x7fedf4197fd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:07:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 >> 172.21.15.62:0/1964396607 conn(0x7fedf40fb1a0 msgr2=0x7fedf40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:07:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.140+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 shutdown_connections 2023-07-14T16:07:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:07:58.141+0000 7fedfa5f4700 1 -- 172.21.15.62:0/1964396607 wait complete. 2023-07-14T16:07:58.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:58.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:58.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:07:58.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:07:58.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:58.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:57 smithi081 ceph-mon[127795]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:07:58.583 DEBUG:teuthology.orchestra.run.smithi062:osd.0> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.0.service 2023-07-14T16:07:58.587 INFO:tasks.cephadm:Deploying osd.1 on smithi081 with /dev/vg_nvme/lv_4... 2023-07-14T16:07:58.588 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T16:07:58.662 INFO:journalctl@ceph.osd.0.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:07:59.308 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:07:59.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:59.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:59.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:59.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:07:59 smithi062 ceph-mon[95543]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T16:07:59.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:59 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:59.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:59 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:59.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:59 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:59.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:07:59 smithi081 ceph-mon[127795]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T16:07:59.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:59 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:59.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:59 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:07:59.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:59 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:07:59.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:07:59 smithi164 ceph-mon[97427]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T16:08:00.251 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:00 smithi081 ceph-mon[127795]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T16:08:00.252 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:00 smithi081 ceph-mon[127795]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T16:08:00.252 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:00.252 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:00 smithi081 ceph-mon[127795]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:08:00.252 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:00 smithi081 ceph-mon[127795]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:08:00.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:00 smithi062 ceph-mon[95543]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T16:08:00.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:00 smithi062 ceph-mon[95543]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T16:08:00.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:00.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:00 smithi062 ceph-mon[95543]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:08:00.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:00 smithi062 ceph-mon[95543]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:08:00.373 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:08:00 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-0[108585]: 2023-07-14T16:08:00.107+0000 7efde2757700 -1 osd.0 0 waiting for initial osdmap 2023-07-14T16:08:00.373 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:08:00 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-0[108585]: 2023-07-14T16:08:00.114+0000 7efddcd49700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:08:00.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:00 smithi164 ceph-mon[97427]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T16:08:00.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:00 smithi164 ceph-mon[97427]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T16:08:00.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:00.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:00 smithi164 ceph-mon[97427]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:08:00.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:00 smithi164 ceph-mon[97427]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:08:01.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:01 smithi062 ceph-mon[95543]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:08:01.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:01 smithi062 ceph-mon[95543]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T16:08:01.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:01.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:01.380 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T16:08:01.380 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T16:08:01.381 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2023-07-14T16:08:01.381 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2023-07-14T16:08:01.381 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0392532 s, 267 MB/s 2023-07-14T16:08:01.381 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2023-07-14T16:08:01.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:01 smithi081 ceph-mon[127795]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:08:01.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:01 smithi081 ceph-mon[127795]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T16:08:01.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:01.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:01.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:01 smithi164 ceph-mon[97427]: from='osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:08:01.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:01 smithi164 ceph-mon[97427]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T16:08:01.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:01.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:01.974 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi081:vg_nvme/lv_4 2023-07-14T16:08:02.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:08:02.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:08:02.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] boot 2023-07-14T16:08:02.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T16:08:02.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:02.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi062 2023-07-14T16:08:02.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:08:02.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:02.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi062 to 17267M 2023-07-14T16:08:02.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:02.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:02.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:08:02.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:08:02.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] boot 2023-07-14T16:08:02.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T16:08:02.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:02.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi062 2023-07-14T16:08:02.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:08:02.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:02.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi062 to 17267M 2023-07-14T16:08:02.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:02.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:02.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:08:02.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:08:02.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: osd.0 [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] boot 2023-07-14T16:08:02.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T16:08:02.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T16:08:02.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi062 2023-07-14T16:08:02.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T16:08:02.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:02.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi062 to 17267M 2023-07-14T16:08:02.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:02.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:02.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:02.692 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:08:03.560 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:03 smithi081 ceph-mon[127795]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T16:08:03.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:03 smithi164 ceph-mon[97427]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T16:08:03.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:03 smithi062 ceph-mon[95543]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T16:08:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 -- 172.21.15.81:0/1862021427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 msgr2=0x7ff604107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 --2- 172.21.15.81:0/1862021427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 0x7ff604107fe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b0a0 tx=0x7ff5fc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:08:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 -- 172.21.15.81:0/1862021427 shutdown_connections 2023-07-14T16:08:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 --2- 172.21.15.81:0/1862021427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 0x7ff604107fe0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 --2- 172.21.15.81:0/1862021427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6040ffb90 0x7ff6040fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 --2- 172.21.15.81:0/1862021427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff604100d90 0x7ff6041011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 -- 172.21.15.81:0/1862021427 >> 172.21.15.81:0/1862021427 conn(0x7ff6040fb1a0 msgr2=0x7ff6040fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:08:04.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 -- 172.21.15.81:0/1862021427 shutdown_connections 2023-07-14T16:08:04.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.240+0000 7ff60c555700 1 -- 172.21.15.81:0/1862021427 wait complete. 2023-07-14T16:08:04.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.241+0000 7ff60c555700 1 Processor -- start 2023-07-14T16:08:04.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.241+0000 7ff60c555700 1 -- start start 2023-07-14T16:08:04.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60c555700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6040ffb90 0x7ff604197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:04.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60c555700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff604100d90 0x7ff604197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:04.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60c555700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 0x7ff60419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:04.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60c555700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419f630 con 0x7ff604100d90 2023-07-14T16:08:04.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60c555700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419f770 con 0x7ff604101720 2023-07-14T16:08:04.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60c555700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60419f900 con 0x7ff6040ffb90 2023-07-14T16:08:04.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60aaf2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 0x7ff60419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:04.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff609af0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff604100d90 0x7ff604197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:04.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60a2f1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6040ffb90 0x7ff604197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:04.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60a2f1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6040ffb90 0x7ff604197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.81:53860/0 (socket says 172.21.15.81:53860) 2023-07-14T16:08:04.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff609af0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff604100d90 0x7ff604197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.81:36210/0 (socket says 172.21.15.81:36210) 2023-07-14T16:08:04.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60a2f1700 1 -- 172.21.15.81:0/2180929796 learned_addr learned my addr 172.21.15.81:0/2180929796 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-07-14T16:08:04.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60aaf2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 0x7ff60419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33524/0 (socket says 172.21.15.81:33524) 2023-07-14T16:08:04.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60aaf2700 1 -- 172.21.15.81:0/2180929796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6040ffb90 msgr2=0x7ff604197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60aaf2700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6040ffb90 0x7ff604197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60aaf2700 1 -- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff604100d90 msgr2=0x7ff604197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60aaf2700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff604100d90 0x7ff604197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.242+0000 7ff60aaf2700 1 -- 172.21.15.81:0/2180929796 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc009d00 con 0x7ff604101720 2023-07-14T16:08:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.243+0000 7ff60aaf2700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 0x7ff60419f060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b070 tx=0x7ff5fc00bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:08:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.243+0000 7ff5fb7fe700 1 -- 172.21.15.81:0/2180929796 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00e040 con 0x7ff604101720 2023-07-14T16:08:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.243+0000 7ff5fb7fe700 1 -- 172.21.15.81:0/2180929796 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5fc009480 con 0x7ff604101720 2023-07-14T16:08:04.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.243+0000 7ff5fb7fe700 1 -- 172.21.15.81:0/2180929796 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0052a0 con 0x7ff604101720 2023-07-14T16:08:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.243+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60419fb80 con 0x7ff604101720 2023-07-14T16:08:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.243+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60419ff90 con 0x7ff604101720 2023-07-14T16:08:04.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.244+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60410b210 con 0x7ff604101720 2023-07-14T16:08:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.246+0000 7ff5fb7fe700 1 -- 172.21.15.81:0/2180929796 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7ff5fc005400 con 0x7ff604101720 2023-07-14T16:08:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.247+0000 7ff5fb7fe700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5f00a1140 0x7ff5f00a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.248+0000 7ff5fb7fe700 1 -- 172.21.15.81:0/2180929796 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7ff5fc0cc260 con 0x7ff604101720 2023-07-14T16:08:04.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.248+0000 7ff60a2f1700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5f00a1140 0x7ff5f00a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:04.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.249+0000 7ff60a2f1700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5f00a1140 0x7ff5f00a35f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff600005950 tx=0x7ff600009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:08:04.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.251+0000 7ff5fb7fe700 1 -- 172.21.15.81:0/2180929796 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff5fc097410 con 0x7ff604101720 2023-07-14T16:08:04.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:04.457+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6041076f0 con 0x7ff5f00a1140 2023-07-14T16:08:04.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:04 smithi081 ceph-mon[127795]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:04.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:04 smithi164 ceph-mon[97427]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:04.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:04 smithi062 ceph-mon[95543]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:05.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:05 smithi081 ceph-mon[127795]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:08:05.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:08:05.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:08:05.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:05.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:05 smithi164 ceph-mon[97427]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:08:05.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:08:05.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:08:05.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:05 smithi062 ceph-mon[95543]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:08:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:05 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:08:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:05 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:08:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:05 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:06.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:06 smithi081 ceph-mon[127795]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:06.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:06 smithi164 ceph-mon[97427]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:06.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:06 smithi062 ceph-mon[95543]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:08.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:08 smithi081 ceph-mon[127795]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:08.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:08 smithi081 ceph-mon[127795]: from='client.? 172.21.15.81:0/4229828731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]: dispatch 2023-07-14T16:08:08.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:08 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]: dispatch 2023-07-14T16:08:08.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:08 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]': finished 2023-07-14T16:08:08.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:08 smithi081 ceph-mon[127795]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T16:08:08.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:08.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:08 smithi081 ceph-mon[127795]: from='client.? 172.21.15.81:0/2391474220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:08:08.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:08 smithi164 ceph-mon[97427]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:08.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:08 smithi164 ceph-mon[97427]: from='client.? 172.21.15.81:0/4229828731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]: dispatch 2023-07-14T16:08:08.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:08 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]: dispatch 2023-07-14T16:08:08.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:08 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]': finished 2023-07-14T16:08:08.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:08 smithi164 ceph-mon[97427]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T16:08:08.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:08.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:08 smithi164 ceph-mon[97427]: from='client.? 172.21.15.81:0/2391474220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:08:08.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:08 smithi062 ceph-mon[95543]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:08.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:08 smithi062 ceph-mon[95543]: from='client.? 172.21.15.81:0/4229828731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]: dispatch 2023-07-14T16:08:08.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:08 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]: dispatch 2023-07-14T16:08:08.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:08 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e"}]': finished 2023-07-14T16:08:08.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:08 smithi062 ceph-mon[95543]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T16:08:08.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:08.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:08 smithi062 ceph-mon[95543]: from='client.? 172.21.15.81:0/2391474220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:08:10.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:10 smithi081 ceph-mon[127795]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:10.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:10 smithi164 ceph-mon[97427]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:10.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:10 smithi062 ceph-mon[95543]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:12.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:12 smithi081 ceph-mon[127795]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:12.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:12 smithi164 ceph-mon[97427]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:12.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:12 smithi062 ceph-mon[95543]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:14.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:13 smithi062 ceph-mon[95543]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:14.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:13 smithi081 ceph-mon[127795]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:14.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:13 smithi164 ceph-mon[97427]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:15.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T16:08:15.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:15.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T16:08:15.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:15.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T16:08:15.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:16.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:16 smithi081 ceph-mon[127795]: Deploying daemon osd.1 on smithi081 2023-07-14T16:08:16.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:16 smithi081 ceph-mon[127795]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:16.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:16 smithi062 ceph-mon[95543]: Deploying daemon osd.1 on smithi081 2023-07-14T16:08:16.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:16 smithi062 ceph-mon[95543]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:16.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:16 smithi164 ceph-mon[97427]: Deploying daemon osd.1 on smithi081 2023-07-14T16:08:16.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:16 smithi164 ceph-mon[97427]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:18.694 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:18 smithi081 ceph-mon[127795]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:18.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:18 smithi164 ceph-mon[97427]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:18.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:18 smithi062 ceph-mon[95543]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:20.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:20 smithi164 ceph-mon[97427]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:20.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:20 smithi062 ceph-mon[95543]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:20.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:20 smithi081 ceph-mon[127795]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:22.717 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:22 smithi081 ceph-mon[127795]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:22.717 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:22.718 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:22.718 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:22.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:22 smithi164 ceph-mon[97427]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:22.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:22.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:22.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:22.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:22 smithi062 ceph-mon[95543]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:22.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:22.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:24.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:23 smithi081 ceph-mon[127795]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:24.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:23 smithi164 ceph-mon[97427]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:24.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:23 smithi062 ceph-mon[95543]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:25.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.025 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:25.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:25.026 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:25.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:25.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:25.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.434+0000 7ff5fb7fe700 1 -- 172.21.15.81:0/2180929796 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff6041076f0 con 0x7ff5f00a1140 2023-07-14T16:08:25.410 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 1 on host 'smithi081' 2023-07-14T16:08:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5f00a1140 msgr2=0x7ff5f00a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5f00a1140 0x7ff5f00a35f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff600005950 tx=0x7ff600009260 comp rx=0 tx=0).stop 2023-07-14T16:08:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 msgr2=0x7ff60419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 0x7ff60419f060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b070 tx=0x7ff5fc00bcc0 comp rx=0 tx=0).stop 2023-07-14T16:08:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 shutdown_connections 2023-07-14T16:08:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5f00a1140 0x7ff5f00a35f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff604101720 0x7ff60419f060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6040ffb90 0x7ff604197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 --2- 172.21.15.81:0/2180929796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff604100d90 0x7ff604197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.437+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 >> 172.21.15.81:0/2180929796 conn(0x7ff6040fb1a0 msgr2=0x7ff6040fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:08:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.438+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 shutdown_connections 2023-07-14T16:08:25.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:08:25.438+0000 7ff60c555700 1 -- 172.21.15.81:0/2180929796 wait complete. 2023-07-14T16:08:25.912 DEBUG:teuthology.orchestra.run.smithi081:osd.1> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.1.service 2023-07-14T16:08:25.918 INFO:tasks.cephadm:Deploying osd.2 on smithi164 with /dev/vg_nvme/lv_4... 2023-07-14T16:08:25.918 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T16:08:25.996 INFO:journalctl@ceph.osd.1.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:08:26.644 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:08:26.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:26.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:26 smithi081 ceph-mon[127795]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:26.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:26.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:26 smithi081 ceph-mon[127795]: from='osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T16:08:26.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:26 smithi081 ceph-mon[127795]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T16:08:26.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:26.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:26 smithi164 ceph-mon[97427]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:26 smithi164 ceph-mon[97427]: from='osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T16:08:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:26 smithi164 ceph-mon[97427]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T16:08:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:26 smithi062 ceph-mon[95543]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:26 smithi062 ceph-mon[95543]: from='osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T16:08:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:26 smithi062 ceph-mon[95543]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T16:08:26.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:27.393 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:27 smithi081 ceph-mon[127795]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T16:08:27.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:27 smithi081 ceph-mon[127795]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T16:08:27.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:27.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:27 smithi081 ceph-mon[127795]: from='osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:08:27.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:27 smithi081 ceph-mon[127795]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:08:27.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:27 smithi164 ceph-mon[97427]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T16:08:27.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:27 smithi164 ceph-mon[97427]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T16:08:27.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:27.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:27 smithi164 ceph-mon[97427]: from='osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:08:27.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:27 smithi164 ceph-mon[97427]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:08:27.650 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:08:27 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-1[132879]: 2023-07-14T16:08:27.417+0000 7f61f5d3c700 -1 osd.1 0 waiting for initial osdmap 2023-07-14T16:08:27.651 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:08:27 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-1[132879]: 2023-07-14T16:08:27.421+0000 7f61edb29700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:08:27.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:27 smithi062 ceph-mon[95543]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T16:08:27.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:27 smithi062 ceph-mon[95543]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T16:08:27.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:27.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:27 smithi062 ceph-mon[95543]: from='osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:08:27.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:27 smithi062 ceph-mon[95543]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:08:28.600 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T16:08:28.601 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T16:08:28.601 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2023-07-14T16:08:28.601 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2023-07-14T16:08:28.601 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0298192 s, 352 MB/s 2023-07-14T16:08:28.601 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2023-07-14T16:08:28.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:28 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:08:28.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:28 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:08:28.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:28 smithi081 ceph-mon[127795]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:28.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:28 smithi081 ceph-mon[127795]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:08:28.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:28 smithi081 ceph-mon[127795]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T16:08:28.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:28 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:28.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:28 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:28.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:28 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:08:28.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:28 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:08:28.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:28 smithi164 ceph-mon[97427]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:28.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:28 smithi164 ceph-mon[97427]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:08:28.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:28 smithi164 ceph-mon[97427]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T16:08:28.747 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:28 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:28.747 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:28 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:28.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:28 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:08:28.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:28 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:08:28.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:28 smithi062 ceph-mon[95543]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T16:08:28.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:28 smithi062 ceph-mon[95543]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:08:28.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:28 smithi062 ceph-mon[95543]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T16:08:28.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:28 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:28.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:28 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:29.220 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2023-07-14T16:08:29.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] boot 2023-07-14T16:08:29.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T16:08:29.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:29.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:29.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:29.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:29.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] boot 2023-07-14T16:08:29.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T16:08:29.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:29.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:29.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:29.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:29.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: osd.1 [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] boot 2023-07-14T16:08:29.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T16:08:29.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T16:08:29.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:29.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:29.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:29.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:29.940 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:08:30.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:30 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi081 2023-07-14T16:08:30.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:30 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi081 to 17267M 2023-07-14T16:08:30.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:30 smithi164 ceph-mon[97427]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:30.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:30 smithi164 ceph-mon[97427]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T16:08:30.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:30 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi081 2023-07-14T16:08:30.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:30 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi081 to 17267M 2023-07-14T16:08:30.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:30 smithi062 ceph-mon[95543]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:30.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:30 smithi062 ceph-mon[95543]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T16:08:30.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:30 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi081 2023-07-14T16:08:30.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:30 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi081 to 17267M 2023-07-14T16:08:30.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:30 smithi081 ceph-mon[127795]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:30.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:30 smithi081 ceph-mon[127795]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T16:08:31.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 -- 172.21.15.164:0/1037743725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f8101720 msgr2=0x7f80f8107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:31.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 --2- 172.21.15.164:0/1037743725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f8101720 0x7f80f8107fe0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b0a0 tx=0x7f80f000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:08:31.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 -- 172.21.15.164:0/1037743725 shutdown_connections 2023-07-14T16:08:31.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 --2- 172.21.15.164:0/1037743725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f8101720 0x7f80f8107fe0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:31.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 --2- 172.21.15.164:0/1037743725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80f80ffb90 0x7f80f80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:31.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 --2- 172.21.15.164:0/1037743725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f8100d90 0x7f80f81011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:31.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 -- 172.21.15.164:0/1037743725 >> 172.21.15.164:0/1037743725 conn(0x7f80f80fb1a0 msgr2=0x7f80f80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:08:31.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 -- 172.21.15.164:0/1037743725 shutdown_connections 2023-07-14T16:08:31.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.607+0000 7f80febcf700 1 -- 172.21.15.164:0/1037743725 wait complete. 2023-07-14T16:08:31.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.608+0000 7f80febcf700 1 Processor -- start 2023-07-14T16:08:31.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.608+0000 7f80febcf700 1 -- start start 2023-07-14T16:08:31.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80febcf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80f80ffb90 0x7f80f8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:31.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80febcf700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f8100d90 0x7f80f8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:31.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80febcf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f8101720 0x7f80f819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:31.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80febcf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f819f600 con 0x7f80f8101720 2023-07-14T16:08:31.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80febcf700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f819f740 con 0x7f80f80ffb90 2023-07-14T16:08:31.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80febcf700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f819f8d0 con 0x7f80f8100d90 2023-07-14T16:08:31.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80fc96b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80f80ffb90 0x7f80f8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:31.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80fd16c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f8101720 0x7f80f819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:31.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80fc96b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80f80ffb90 0x7f80f8197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.164:51566/0 (socket says 172.21.15.164:51566) 2023-07-14T16:08:31.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80fc96b700 1 -- 172.21.15.164:0/372133820 learned_addr learned my addr 172.21.15.164:0/372133820 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:08:31.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.609+0000 7f80f7fff700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f8100d90 0x7f80f8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:31.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80fd16c700 1 -- 172.21.15.164:0/372133820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f8100d90 msgr2=0x7f80f8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:31.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80fd16c700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f8100d90 0x7f80f8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:31.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80fd16c700 1 -- 172.21.15.164:0/372133820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80f80ffb90 msgr2=0x7f80f8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:31.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80fd16c700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80f80ffb90 0x7f80f8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:31.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80fd16c700 1 -- 172.21.15.164:0/372133820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f0009d00 con 0x7f80f8101720 2023-07-14T16:08:31.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80fd16c700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f8101720 0x7f80f819f030 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b070 tx=0x7f80f0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:08:31.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80f5ffb700 1 -- 172.21.15.164:0/372133820 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f000e040 con 0x7f80f8101720 2023-07-14T16:08:31.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80f5ffb700 1 -- 172.21.15.164:0/372133820 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80f0004e70 con 0x7f80f8101720 2023-07-14T16:08:31.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f819fb50 con 0x7f80f8101720 2023-07-14T16:08:31.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80f5ffb700 1 -- 172.21.15.164:0/372133820 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f001da20 con 0x7f80f8101720 2023-07-14T16:08:31.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.610+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f819ff60 con 0x7f80f8101720 2023-07-14T16:08:31.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.611+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f810b2d0 con 0x7f80f8101720 2023-07-14T16:08:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.614+0000 7f80f5ffb700 1 -- 172.21.15.164:0/372133820 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f80f00058c0 con 0x7f80f8101720 2023-07-14T16:08:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.615+0000 7f80f5ffb700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80e80a10c0 0x7f80e80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.615+0000 7f80f5ffb700 1 -- 172.21.15.164:0/372133820 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f80f00ccbb0 con 0x7f80f8101720 2023-07-14T16:08:31.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.615+0000 7f80fc96b700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80e80a10c0 0x7f80e80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:31.591 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.616+0000 7f80fc96b700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80e80a10c0 0x7f80e80a3570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00ace0 tx=0x7f80ec00a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:08:31.592 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.618+0000 7f80f5ffb700 1 -- 172.21.15.164:0/372133820 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f80f0097b40 con 0x7f80f8101720 2023-07-14T16:08:31.793 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:31.818+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f80f8107d60 con 0x7f80e80a10c0 2023-07-14T16:08:32.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:32 smithi164 ceph-mon[97427]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:32.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:08:32.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:08:32.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:32.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:32 smithi062 ceph-mon[95543]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:32.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:08:32.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:08:32.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:32.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:32 smithi081 ceph-mon[127795]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:32.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:08:32.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:08:32.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:33.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:33 smithi164 ceph-mon[97427]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:08:33.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:33 smithi062 ceph-mon[95543]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:08:33.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:33 smithi081 ceph-mon[127795]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:08:34.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:34 smithi164 ceph-mon[97427]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:34.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:34 smithi062 ceph-mon[95543]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:34.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:34 smithi081 ceph-mon[127795]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:35.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:35 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/2437937114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]: dispatch 2023-07-14T16:08:35.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:35 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]: dispatch 2023-07-14T16:08:35.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:35 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]': finished 2023-07-14T16:08:35.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:35 smithi164 ceph-mon[97427]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T16:08:35.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:35 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:35.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:35 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/2437937114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]: dispatch 2023-07-14T16:08:35.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:35 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]: dispatch 2023-07-14T16:08:35.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:35 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]': finished 2023-07-14T16:08:35.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:35 smithi062 ceph-mon[95543]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T16:08:35.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:35 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:35.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:35 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/2437937114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]: dispatch 2023-07-14T16:08:35.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:35 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]: dispatch 2023-07-14T16:08:35.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:35 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "831ec85f-142a-4e71-bd2c-9f2c1e8d8d43"}]': finished 2023-07-14T16:08:35.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:35 smithi081 ceph-mon[127795]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T16:08:35.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:35 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:36.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:36 smithi164 ceph-mon[97427]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:36.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:36 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/2408275952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:08:36.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:36 smithi062 ceph-mon[95543]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:36.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:36 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/2408275952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:08:36.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:36 smithi081 ceph-mon[127795]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:36.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:36 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/2408275952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:08:38.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:38 smithi164 ceph-mon[97427]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:38.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:38 smithi062 ceph-mon[95543]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:38.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:38 smithi081 ceph-mon[127795]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:40.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:40 smithi164 ceph-mon[97427]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:40.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:40 smithi062 ceph-mon[95543]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:40.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:40 smithi081 ceph-mon[127795]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:42.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:42 smithi164 ceph-mon[97427]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:42.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:42 smithi062 ceph-mon[95543]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:42.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:42 smithi081 ceph-mon[127795]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:43.733 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:43 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T16:08:43.734 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:43 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:43.734 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:43 smithi164 ceph-mon[97427]: Deploying daemon osd.2 on smithi164 2023-07-14T16:08:43.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:43 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T16:08:43.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:43 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:43.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:43 smithi062 ceph-mon[95543]: Deploying daemon osd.2 on smithi164 2023-07-14T16:08:43.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:43 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T16:08:43.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:43 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:43.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:43 smithi081 ceph-mon[127795]: Deploying daemon osd.2 on smithi164 2023-07-14T16:08:44.750 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:44 smithi164 ceph-mon[97427]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:44.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:44 smithi062 ceph-mon[95543]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:44.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:44 smithi081 ceph-mon[127795]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:46.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:46 smithi164 ceph-mon[97427]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:46.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:46 smithi062 ceph-mon[95543]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:46.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:46 smithi081 ceph-mon[127795]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:48.772 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:48 smithi164 ceph-mon[97427]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:48.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:48 smithi062 ceph-mon[95543]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:48.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:48 smithi081 ceph-mon[127795]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:49.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:49 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:49.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:49 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:49.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:49 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:49.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:49 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:49.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:49 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:49.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:49 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:49.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:49 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:49.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:49 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:49.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:49 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:50.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:50 smithi164 ceph-mon[97427]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:50.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:50 smithi062 ceph-mon[95543]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:50.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:50 smithi081 ceph-mon[127795]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:52.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:52 smithi081 ceph-mon[127795]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:52.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:52 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:52.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:52 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:52.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:52 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:52.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:52 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:52.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:52 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:53.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:52 smithi164 ceph-mon[97427]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:53.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:52 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:53.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:52 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:53.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:52 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:53.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:52 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:53.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:52 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:53.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:52 smithi062 ceph-mon[95543]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:53.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:52 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:53.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:52 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:53.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:52 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:52 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:52 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:53.174 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 2 on host 'smithi164' 2023-07-14T16:08:53.174 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.198+0000 7f80f5ffb700 1 -- 172.21.15.164:0/372133820 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f80f8107d60 con 0x7f80e80a10c0 2023-07-14T16:08:53.176 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80e80a10c0 msgr2=0x7f80e80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:53.176 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80e80a10c0 0x7f80e80a3570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00ace0 tx=0x7f80ec00a430 comp rx=0 tx=0).stop 2023-07-14T16:08:53.176 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f8101720 msgr2=0x7f80f819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:53.177 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f8101720 0x7f80f819f030 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b070 tx=0x7f80f0004c80 comp rx=0 tx=0).stop 2023-07-14T16:08:53.177 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 shutdown_connections 2023-07-14T16:08:53.177 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f8101720 0x7f80f819f030 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:53.177 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80e80a10c0 0x7f80e80a3570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:53.177 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80f80ffb90 0x7f80f8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:53.177 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 --2- 172.21.15.164:0/372133820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80f8100d90 0x7f80f8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:53.177 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 >> 172.21.15.164:0/372133820 conn(0x7f80f80fb1a0 msgr2=0x7f80f8106640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:08:53.178 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 shutdown_connections 2023-07-14T16:08:53.178 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:08:53.201+0000 7f80febcf700 1 -- 172.21.15.164:0/372133820 wait complete. 2023-07-14T16:08:53.660 DEBUG:teuthology.orchestra.run.smithi164:osd.2> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.2.service 2023-07-14T16:08:53.665 INFO:tasks.cephadm:Deploying osd.3 on smithi062 with /dev/vg_nvme/lv_3... 2023-07-14T16:08:53.665 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T16:08:53.740 INFO:journalctl@ceph.osd.2.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:08:54.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:53 smithi164 ceph-mon[97427]: from='osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T16:08:54.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:53 smithi164 ceph-mon[97427]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T16:08:54.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:54.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:54.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:54.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:53 smithi164 ceph-mon[97427]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:54.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:53 smithi062 ceph-mon[95543]: from='osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T16:08:54.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:53 smithi062 ceph-mon[95543]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T16:08:54.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:54.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:54.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:54.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:53 smithi062 ceph-mon[95543]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:54.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:53 smithi081 ceph-mon[127795]: from='osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T16:08:54.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:53 smithi081 ceph-mon[127795]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T16:08:54.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:54.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:54.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:08:54.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:53 smithi081 ceph-mon[127795]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:08:54.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:54 smithi062 ceph-mon[95543]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T16:08:54.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:54 smithi062 ceph-mon[95543]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T16:08:54.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:54.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:54 smithi062 ceph-mon[95543]: from='osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:08:54.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:54 smithi062 ceph-mon[95543]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:08:55.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:54 smithi081 ceph-mon[127795]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T16:08:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:54 smithi081 ceph-mon[127795]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T16:08:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:54 smithi081 ceph-mon[127795]: from='osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:08:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:54 smithi081 ceph-mon[127795]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:08:55.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:54 smithi164 ceph-mon[97427]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T16:08:55.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:54 smithi164 ceph-mon[97427]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T16:08:55.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:55.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:54 smithi164 ceph-mon[97427]: from='osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:08:55.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:54 smithi164 ceph-mon[97427]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:08:55.274 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:08:54 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-2[103230]: 2023-07-14T16:08:54.922+0000 7ff7d7e47700 -1 osd.2 0 waiting for initial osdmap 2023-07-14T16:08:55.274 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:08:54 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-2[103230]: 2023-07-14T16:08:54.927+0000 7ff7d343b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:08:56.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:55 smithi062 ceph-mon[95543]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:08:56.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:55 smithi062 ceph-mon[95543]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T16:08:56.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:56.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:56.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:55 smithi062 ceph-mon[95543]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:56.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:55 smithi164 ceph-mon[97427]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:08:56.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:55 smithi164 ceph-mon[97427]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T16:08:56.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:56.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:56.150 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:55 smithi164 ceph-mon[97427]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:56.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:55 smithi081 ceph-mon[127795]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:08:56.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:55 smithi081 ceph-mon[127795]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T16:08:56.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:56.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:56.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:55 smithi081 ceph-mon[127795]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T16:08:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T16:08:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T16:08:56.379 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2023-07-14T16:08:56.379 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2023-07-14T16:08:56.379 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0304471 s, 344 MB/s 2023-07-14T16:08:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2023-07-14T16:08:56.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2023-07-14T16:08:57.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:08:57.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:08:57.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] boot 2023-07-14T16:08:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T16:08:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi164 2023-07-14T16:08:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi164 to 17267M 2023-07-14T16:08:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:57.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:57.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:08:57.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:08:57.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] boot 2023-07-14T16:08:57.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T16:08:57.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:57.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi164 2023-07-14T16:08:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi164 to 17267M 2023-07-14T16:08:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:08:57.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:08:57.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: osd.2 [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] boot 2023-07-14T16:08:57.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T16:08:57.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T16:08:57.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi164 2023-07-14T16:08:57.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:08:57.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi164 to 17267M 2023-07-14T16:08:57.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:08:57.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:08:57.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:08:57.626 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:08:58.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:57 smithi062 ceph-mon[95543]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T16:08:58.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:57 smithi164 ceph-mon[97427]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T16:08:58.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:57 smithi081 ceph-mon[127795]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T16:08:59.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:58 smithi062 ceph-mon[95543]: pgmap v88: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:08:59.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T16:08:59.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T16:08:59.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:58 smithi062 ceph-mon[95543]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T16:08:59.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:08:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T16:08:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.155+0000 7f720885b700 1 -- 172.21.15.62:0/2659858074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7204101720 msgr2=0x7f7204107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.155+0000 7f720885b700 1 --2- 172.21.15.62:0/2659858074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7204101720 0x7f7204107fe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009f90 tx=0x7f71fc00b330 comp rx=0 tx=0).stop 2023-07-14T16:08:59.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.155+0000 7f720885b700 1 -- 172.21.15.62:0/2659858074 shutdown_connections 2023-07-14T16:08:59.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.155+0000 7f720885b700 1 --2- 172.21.15.62:0/2659858074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7204101720 0x7f7204107fe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:59.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.155+0000 7f720885b700 1 --2- 172.21.15.62:0/2659858074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72040ffb90 0x7f72040fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:59.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.155+0000 7f720885b700 1 --2- 172.21.15.62:0/2659858074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7204100d90 0x7f72041011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:59.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.155+0000 7f720885b700 1 -- 172.21.15.62:0/2659858074 >> 172.21.15.62:0/2659858074 conn(0x7f72040fb1a0 msgr2=0x7f72040fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:08:59.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.156+0000 7f720885b700 1 -- 172.21.15.62:0/2659858074 shutdown_connections 2023-07-14T16:08:59.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.156+0000 7f720885b700 1 -- 172.21.15.62:0/2659858074 wait complete. 2023-07-14T16:08:59.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720885b700 1 Processor -- start 2023-07-14T16:08:59.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720885b700 1 -- start start 2023-07-14T16:08:59.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720885b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72040ffb90 0x7f72040742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:59.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720885b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7204100d90 0x7f7204072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:59.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720885b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7204101720 0x7f7204072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:59.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720885b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040735c0 con 0x7f7204101720 2023-07-14T16:08:59.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720885b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204073730 con 0x7f72040ffb90 2023-07-14T16:08:59.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720885b700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720419f910 con 0x7f7204100d90 2023-07-14T16:08:59.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72040ffb90 0x7f72040742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:59.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72040ffb90 0x7f72040742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35832/0 (socket says 172.21.15.62:35832) 2023-07-14T16:08:59.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f720259c700 1 -- 172.21.15.62:0/3669176136 learned_addr learned my addr 172.21.15.62:0/3669176136 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:08:59.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.157+0000 7f7202d9d700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7204101720 0x7f7204072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:59.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f7201d9b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7204100d90 0x7f7204072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:59.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f720259c700 1 -- 172.21.15.62:0/3669176136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7204100d90 msgr2=0x7f7204072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:59.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f720259c700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7204100d90 0x7f7204072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:59.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f720259c700 1 -- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7204101720 msgr2=0x7f7204072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:08:59.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f720259c700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7204101720 0x7f7204072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:08:59.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f720259c700 1 -- 172.21.15.62:0/3669176136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40095f0 con 0x7f72040ffb90 2023-07-14T16:08:59.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f720259c700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72040ffb90 0x7f72040742b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f71f400ebf0 tx=0x7f71f400ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:08:59.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f71f37fe700 1 -- 172.21.15.62:0/3669176136 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4009a10 con 0x7f72040ffb90 2023-07-14T16:08:59.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f71f37fe700 1 -- 172.21.15.62:0/3669176136 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71f400cd40 con 0x7f72040ffb90 2023-07-14T16:08:59.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc009c40 con 0x7f72040ffb90 2023-07-14T16:08:59.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.158+0000 7f71f37fe700 1 -- 172.21.15.62:0/3669176136 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4018700 con 0x7f72040ffb90 2023-07-14T16:08:59.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.159+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720419ff60 con 0x7f72040ffb90 2023-07-14T16:08:59.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.160+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7204107190 con 0x7f72040ffb90 2023-07-14T16:08:59.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.162+0000 7f71f37fe700 1 -- 172.21.15.62:0/3669176136 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f71f4018960 con 0x7f72040ffb90 2023-07-14T16:08:59.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.163+0000 7f71f37fe700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71ec0a1140 0x7f71ec0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:08:59.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.163+0000 7f71f37fe700 1 -- 172.21.15.62:0/3669176136 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f71f4014070 con 0x7f72040ffb90 2023-07-14T16:08:59.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.163+0000 7f7201d9b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71ec0a1140 0x7f71ec0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:08:59.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.164+0000 7f7201d9b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71ec0a1140 0x7f71ec0a35f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005950 tx=0x7f71f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:08:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.166+0000 7f71f37fe700 1 -- 172.21.15.62:0/3669176136 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f71f40968d0 con 0x7f72040ffb90 2023-07-14T16:08:59.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:58 smithi164 ceph-mon[97427]: pgmap v88: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:08:59.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T16:08:59.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T16:08:59.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:58 smithi164 ceph-mon[97427]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T16:08:59.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:08:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T16:08:59.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:08:59.381+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7204060950 con 0x7f71ec0a1140 2023-07-14T16:08:59.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:58 smithi081 ceph-mon[127795]: pgmap v88: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:08:59.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T16:08:59.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T16:08:59.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:58 smithi081 ceph-mon[127795]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T16:08:59.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:08:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T16:09:00.012 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:08:59 smithi062 sudo[111648]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T16:09:00.012 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:08:59 smithi062 sudo[111648]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:00.012 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:08:59 smithi062 sudo[111648]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:00.012 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:08:59 smithi062 sudo[111648]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:00.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:00.046+0000 7f71f37fe700 1 -- 172.21.15.62:0/3669176136 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f71f4096ca0 con 0x7f72040ffb90 2023-07-14T16:09:00.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T16:09:00.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:00 smithi164 ceph-mon[97427]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T16:09:00.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:00 smithi164 ceph-mon[97427]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:00.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:00.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:00.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:00.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:00 smithi164 ceph-mon[97427]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:00.372 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:09:00 smithi062 sudo[111696]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T16:09:00.372 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:09:00 smithi062 sudo[111696]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:00.372 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:09:00 smithi062 sudo[111696]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:00.372 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:09:00 smithi062 sudo[111696]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:00.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T16:09:00.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:00 smithi062 ceph-mon[95543]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T16:09:00.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:00 smithi062 ceph-mon[95543]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:00.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:00.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:00.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:00.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:00 smithi062 ceph-mon[95543]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:00.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T16:09:00.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:00 smithi081 ceph-mon[127795]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T16:09:00.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:00 smithi081 ceph-mon[127795]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:00.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:00.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:00.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:00.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:00 smithi081 ceph-mon[127795]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:01.013 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:09:00 smithi081 sudo[135297]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T16:09:01.013 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:09:00 smithi081 sudo[135297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:01.013 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:09:00 smithi081 sudo[135297]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:01.014 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:09:00 smithi081 sudo[135297]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:01.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:01 smithi164 ceph-mon[97427]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T16:09:01.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:01 smithi164 ceph-mon[97427]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T16:09:01.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:01.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:01 smithi062 ceph-mon[95543]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T16:09:01.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:01 smithi062 ceph-mon[95543]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T16:09:01.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:01.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:01 smithi081 ceph-mon[127795]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T16:09:01.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:01 smithi081 ceph-mon[127795]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T16:09:01.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:01.401 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:09:01 smithi081 sudo[135300]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T16:09:01.401 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:09:01 smithi081 sudo[135300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:01.402 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:09:01 smithi081 sudo[135300]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:01.402 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:09:01 smithi081 sudo[135300]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:02.273 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:09:01 smithi164 sudo[105485]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T16:09:02.273 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:09:01 smithi164 sudo[105485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:02.274 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:09:01 smithi164 sudo[105485]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:02.274 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:09:01 smithi164 sudo[105485]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:02.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:02.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:02 smithi062 ceph-mon[95543]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:02.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:02.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:02 smithi081 ceph-mon[127795]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:02.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:02.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:02 smithi164 ceph-mon[97427]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:02.773 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:09:02 smithi164 sudo[105504]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T16:09:02.774 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:09:02 smithi164 sudo[105504]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:02.774 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:09:02 smithi164 sudo[105504]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:02.774 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:09:02 smithi164 sudo[105504]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:02 smithi062 sudo[111879]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T16:09:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:02 smithi062 sudo[111879]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:02 smithi062 sudo[111879]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:02 smithi062 sudo[111879]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:03.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:03 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3518361453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b2e9feb-b1c8-471d-8ded-98528b1b5890"}]: dispatch 2023-07-14T16:09:03.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:03 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3518361453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b2e9feb-b1c8-471d-8ded-98528b1b5890"}]': finished 2023-07-14T16:09:03.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:03 smithi164 ceph-mon[97427]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T16:09:03.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:03.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:03.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:03 smithi164 ceph-mon[97427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:03.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 sudo[111882]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T16:09:03.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 sudo[111882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:03.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 sudo[111882]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:03.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 sudo[111882]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:03.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3518361453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b2e9feb-b1c8-471d-8ded-98528b1b5890"}]: dispatch 2023-07-14T16:09:03.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3518361453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b2e9feb-b1c8-471d-8ded-98528b1b5890"}]': finished 2023-07-14T16:09:03.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 ceph-mon[95543]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T16:09:03.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:03.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:03.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:03 smithi062 ceph-mon[95543]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:03.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:03 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3518361453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b2e9feb-b1c8-471d-8ded-98528b1b5890"}]: dispatch 2023-07-14T16:09:03.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:03 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3518361453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b2e9feb-b1c8-471d-8ded-98528b1b5890"}]': finished 2023-07-14T16:09:03.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:03 smithi081 ceph-mon[127795]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T16:09:03.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:03.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:03.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:03 smithi081 ceph-mon[127795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:04.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 sudo[105595]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T16:09:04.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 sudo[105595]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:04.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 sudo[105595]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:04.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 sudo[105595]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:04.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:04.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:04.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:04 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/337549576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:05.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 sudo[105611]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T16:09:05.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 sudo[105611]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:05.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 sudo[105611]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:05.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 sudo[105611]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:04 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/337549576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:05.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 sudo[135303]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T16:09:05.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 sudo[135303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:05.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 sudo[135303]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:05.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 sudo[135303]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:05 smithi062 ceph-mon[95543]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:05 smithi062 ceph-mon[95543]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:05 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:05 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:05.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:05 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:05.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 sudo[135306]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T16:09:05.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 sudo[135306]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T16:09:05.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 sudo[135306]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T16:09:05.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 sudo[135306]: pam_unix(sudo:session): session closed for user root 2023-07-14T16:09:05.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:05.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:05.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:05.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:05.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:05.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:05.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:05.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:05.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/337549576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:05.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:05.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:05.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:05.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:05.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:05 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:06.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:05 smithi164 ceph-mon[97427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:06.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:05 smithi164 ceph-mon[97427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:06.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T16:09:06.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T16:09:06.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:05 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T16:09:06.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:06 smithi062 ceph-mon[95543]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:06.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:06 smithi062 ceph-mon[95543]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:06.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:06 smithi062 ceph-mon[95543]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:06.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:06 smithi081 ceph-mon[127795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:06.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:06 smithi081 ceph-mon[127795]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:06.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:06 smithi081 ceph-mon[127795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:07.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:06 smithi164 ceph-mon[97427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T16:09:07.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:06 smithi164 ceph-mon[97427]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:07.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:06 smithi164 ceph-mon[97427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T16:09:08.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:08 smithi062 ceph-mon[95543]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:08.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:08 smithi081 ceph-mon[127795]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:09.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:08 smithi164 ceph-mon[97427]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:10.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:10 smithi062 ceph-mon[95543]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-07-14T16:09:10.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T16:09:10.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:10.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:10 smithi081 ceph-mon[127795]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-07-14T16:09:10.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:10 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T16:09:10.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:10 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:11.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:10 smithi164 ceph-mon[97427]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-07-14T16:09:11.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T16:09:11.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:11.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:11 smithi062 ceph-mon[95543]: Deploying daemon osd.3 on smithi062 2023-07-14T16:09:11.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:11 smithi081 ceph-mon[127795]: Deploying daemon osd.3 on smithi062 2023-07-14T16:09:12.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:11 smithi164 ceph-mon[97427]: Deploying daemon osd.3 on smithi062 2023-07-14T16:09:12.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:12 smithi062 ceph-mon[95543]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-07-14T16:09:12.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:12 smithi081 ceph-mon[127795]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-07-14T16:09:13.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:12 smithi164 ceph-mon[97427]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-07-14T16:09:14.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:14 smithi062 ceph-mon[95543]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-07-14T16:09:14.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:14 smithi081 ceph-mon[127795]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-07-14T16:09:15.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:14 smithi164 ceph-mon[97427]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-07-14T16:09:16.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:16 smithi062 ceph-mon[95543]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-14T16:09:16.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:16 smithi081 ceph-mon[127795]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-14T16:09:17.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:16 smithi164 ceph-mon[97427]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-14T16:09:17.990 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:17.990 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:17.990 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:17.990 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:17 smithi062 ceph-mon[95543]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-14T16:09:18.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:18.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:18.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:18.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:17 smithi164 ceph-mon[97427]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-14T16:09:18.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:18.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:18.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:18.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:17 smithi081 ceph-mon[127795]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-14T16:09:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.193+0000 7f71f37fe700 1 -- 172.21.15.62:0/3669176136 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7204060950 con 0x7f71ec0a1140 2023-07-14T16:09:20.169 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 3 on host 'smithi062' 2023-07-14T16:09:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71ec0a1140 msgr2=0x7f71ec0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71ec0a1140 0x7f71ec0a35f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005950 tx=0x7f71f80058e0 comp rx=0 tx=0).stop 2023-07-14T16:09:20.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72040ffb90 msgr2=0x7f72040742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:20.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72040ffb90 0x7f72040742b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f71f400ebf0 tx=0x7f71f400ef00 comp rx=0 tx=0).stop 2023-07-14T16:09:20.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 shutdown_connections 2023-07-14T16:09:20.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7204101720 0x7f7204072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71ec0a1140 0x7f71ec0a35f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72040ffb90 0x7f72040742b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 --2- 172.21.15.62:0/3669176136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7204100d90 0x7f7204072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 >> 172.21.15.62:0/3669176136 conn(0x7f72040fb1a0 msgr2=0x7f72041042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:09:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 shutdown_connections 2023-07-14T16:09:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:09:20.196+0000 7f720885b700 1 -- 172.21.15.62:0/3669176136 wait complete. 2023-07-14T16:09:20.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-14T16:09:20.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:20.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:20.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.645 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:20.645 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.645 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.646 DEBUG:teuthology.orchestra.run.smithi062:osd.3> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.3.service 2023-07-14T16:09:20.650 INFO:tasks.cephadm:Deploying osd.4 on smithi081 with /dev/vg_nvme/lv_3... 2023-07-14T16:09:20.651 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T16:09:20.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-14T16:09:20.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:20.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:20.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:20.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.755 INFO:journalctl@ceph.osd.3.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:09:20.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-14T16:09:20.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:20.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:20.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:20.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:20.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:21.394 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:09:21.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:21 smithi062 ceph-mon[95543]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T16:09:21.645 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:21 smithi081 ceph-mon[127795]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T16:09:21.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:21 smithi164 ceph-mon[97427]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T16:09:22.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:22 smithi062 ceph-mon[95543]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:22.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:22 smithi062 ceph-mon[95543]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T16:09:22.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:22 smithi062 ceph-mon[95543]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T16:09:22.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:22.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:22 smithi062 ceph-mon[95543]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:09:22.623 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jul 14 16:09:22 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-3[114098]: 2023-07-14T16:09:22.430+0000 7f9e5e6b8700 -1 osd.3 0 waiting for initial osdmap 2023-07-14T16:09:22.623 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jul 14 16:09:22 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-3[114098]: 2023-07-14T16:09:22.436+0000 7f9e5a4ad700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:09:22.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:22 smithi081 ceph-mon[127795]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:22.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:22 smithi081 ceph-mon[127795]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T16:09:22.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:22 smithi081 ceph-mon[127795]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T16:09:22.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:22.638 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:22 smithi081 ceph-mon[127795]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:09:22.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:22 smithi164 ceph-mon[97427]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:22.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:22 smithi164 ceph-mon[97427]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T16:09:22.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:22 smithi164 ceph-mon[97427]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T16:09:22.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:22.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:22 smithi164 ceph-mon[97427]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:09:23.337 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T16:09:23.337 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T16:09:23.337 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2023-07-14T16:09:23.337 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2023-07-14T16:09:23.337 INFO:teuthology.orchestra.run.smithi081.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.02925 s, 358 MB/s 2023-07-14T16:09:23.337 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2023-07-14T16:09:23.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:23 smithi081 ceph-mon[127795]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:09:23.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:23 smithi081 ceph-mon[127795]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T16:09:23.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:23 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:23.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:23 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:23.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:23 smithi164 ceph-mon[97427]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:09:23.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:23 smithi164 ceph-mon[97427]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T16:09:23.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:23 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:23.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:23 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:23.862 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi081:vg_nvme/lv_3 2023-07-14T16:09:23.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:23 smithi062 ceph-mon[95543]: from='osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:09:23.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:23 smithi062 ceph-mon[95543]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T16:09:23.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:23 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:23.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:23 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:24.581 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:09:24.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:09:24.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:09:24.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] boot 2023-07-14T16:09:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T16:09:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T16:09:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi062 2023-07-14T16:09:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:24.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:24.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi062 to 8633M 2023-07-14T16:09:24.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:24.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:24.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:09:24.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:09:24.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:24.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] boot 2023-07-14T16:09:24.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T16:09:24.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:24.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T16:09:24.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi062 2023-07-14T16:09:24.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:24.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:24.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi062 to 8633M 2023-07-14T16:09:24.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:24.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:24.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:09:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:09:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T16:09:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: osd.3 [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] boot 2023-07-14T16:09:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T16:09:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T16:09:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T16:09:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi062 2023-07-14T16:09:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi062 to 8633M 2023-07-14T16:09:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:24.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:25.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:25 smithi164 ceph-mon[97427]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T16:09:25.826 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:25 smithi081 ceph-mon[127795]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T16:09:25.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:25 smithi062 ceph-mon[95543]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T16:09:26.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 -- 172.21.15.81:0/829919323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e4100d90 msgr2=0x7f64e41011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:26.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 --2- 172.21.15.81:0/829919323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e4100d90 0x7f64e41011e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009990 tx=0x7f64d8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:09:26.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 -- 172.21.15.81:0/829919323 shutdown_connections 2023-07-14T16:09:26.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 --2- 172.21.15.81:0/829919323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4101720 0x7f64e4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:26.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 --2- 172.21.15.81:0/829919323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64e40ffb90 0x7f64e40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:26.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 --2- 172.21.15.81:0/829919323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e4100d90 0x7f64e41011e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:26.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 -- 172.21.15.81:0/829919323 >> 172.21.15.81:0/829919323 conn(0x7f64e40fb1a0 msgr2=0x7f64e40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:09:26.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 -- 172.21.15.81:0/829919323 shutdown_connections 2023-07-14T16:09:26.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.140+0000 7f64e93fd700 1 -- 172.21.15.81:0/829919323 wait complete. 2023-07-14T16:09:26.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.141+0000 7f64e93fd700 1 Processor -- start 2023-07-14T16:09:26.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.141+0000 7f64e93fd700 1 -- start start 2023-07-14T16:09:26.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e93fd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e40ffb90 0x7f64e41957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:09:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e93fd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64e4100d90 0x7f64e4195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:09:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e93fd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4101720 0x7f64e419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:09:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e93fd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e419d370 con 0x7f64e4101720 2023-07-14T16:09:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e93fd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e419d4b0 con 0x7f64e40ffb90 2023-07-14T16:09:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e93fd700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e419d640 con 0x7f64e4100d90 2023-07-14T16:09:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e40ffb90 0x7f64e41957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:09:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e40ffb90 0x7f64e41957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49798/0 (socket says 172.21.15.81:49798) 2023-07-14T16:09:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 -- 172.21.15.81:0/3545175426 learned_addr learned my addr 172.21.15.81:0/3545175426 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-07-14T16:09:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 -- 172.21.15.81:0/3545175426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64e4100d90 msgr2=0x7f64e4195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64e4100d90 0x7f64e4195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 -- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4101720 msgr2=0x7f64e419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4101720 0x7f64e419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 -- 172.21.15.81:0/3545175426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80095f0 con 0x7f64e40ffb90 2023-07-14T16:09:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.142+0000 7f64e2ffd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e40ffb90 0x7f64e41957e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f64d400d980 tx=0x7f64d400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:09:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.143+0000 7f64cbfff700 1 -- 172.21.15.81:0/3545175426 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d400de60 con 0x7f64e40ffb90 2023-07-14T16:09:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.143+0000 7f64cbfff700 1 -- 172.21.15.81:0/3545175426 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64d40044b0 con 0x7f64e40ffb90 2023-07-14T16:09:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.143+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e419d920 con 0x7f64e40ffb90 2023-07-14T16:09:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.143+0000 7f64cbfff700 1 -- 172.21.15.81:0/3545175426 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4003b20 con 0x7f64e40ffb90 2023-07-14T16:09:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.143+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e419de20 con 0x7f64e40ffb90 2023-07-14T16:09:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.144+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e4106b20 con 0x7f64e40ffb90 2023-07-14T16:09:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.146+0000 7f64cbfff700 1 -- 172.21.15.81:0/3545175426 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f64d4003c80 con 0x7f64e40ffb90 2023-07-14T16:09:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.147+0000 7f64cbfff700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64cc0a11d0 0x7f64cc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:09:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.147+0000 7f64cbfff700 1 -- 172.21.15.81:0/3545175426 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f64d40cb470 con 0x7f64e40ffb90 2023-07-14T16:09:26.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.148+0000 7f64e27fc700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64cc0a11d0 0x7f64cc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:09:26.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.148+0000 7f64e27fc700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64cc0a11d0 0x7f64cc0a3680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64d80098e0 tx=0x7f64d8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:09:26.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.151+0000 7f64cbfff700 1 -- 172.21.15.81:0/3545175426 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f64d4095d40 con 0x7f64e40ffb90 2023-07-14T16:09:26.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:26.351+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f64e4060950 con 0x7f64cc0a11d0 2023-07-14T16:09:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:26 smithi164 ceph-mon[97427]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:26 smithi164 ceph-mon[97427]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T16:09:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:26.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:26.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:26 smithi062 ceph-mon[95543]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:26.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:26 smithi062 ceph-mon[95543]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T16:09:26.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:26.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:26.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:26.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:26 smithi081 ceph-mon[127795]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:26.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:26 smithi081 ceph-mon[127795]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T16:09:26.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:26.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:26.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:27.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:27 smithi081 ceph-mon[127795]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:27.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:27 smithi081 ceph-mon[127795]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T16:09:27.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:27 smithi164 ceph-mon[97427]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:27.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:27 smithi164 ceph-mon[97427]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T16:09:27.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:27 smithi062 ceph-mon[95543]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:27.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:27 smithi062 ceph-mon[95543]: osdmap e30: 4 total, 4 up, 4 in 2023-07-14T16:09:28.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:28 smithi081 ceph-mon[127795]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:28.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:28 smithi164 ceph-mon[97427]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:28.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:28 smithi062 ceph-mon[95543]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:29.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:29 smithi164 ceph-mon[97427]: from='client.? 172.21.15.81:0/1639580332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]: dispatch 2023-07-14T16:09:29.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:29 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]: dispatch 2023-07-14T16:09:29.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:29 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]': finished 2023-07-14T16:09:29.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:29 smithi164 ceph-mon[97427]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T16:09:29.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:29 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:29.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:29 smithi062 ceph-mon[95543]: from='client.? 172.21.15.81:0/1639580332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]: dispatch 2023-07-14T16:09:29.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:29 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]: dispatch 2023-07-14T16:09:29.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:29 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]': finished 2023-07-14T16:09:29.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:29 smithi062 ceph-mon[95543]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T16:09:29.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:29 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:29.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:29 smithi081 ceph-mon[127795]: from='client.? 172.21.15.81:0/1639580332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]: dispatch 2023-07-14T16:09:29.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:29 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]: dispatch 2023-07-14T16:09:29.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:29 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6647685-0173-4443-b612-13675c942486"}]': finished 2023-07-14T16:09:29.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:29 smithi081 ceph-mon[127795]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T16:09:29.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:29 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:30.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:30 smithi164 ceph-mon[97427]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-14T16:09:30.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:30 smithi164 ceph-mon[97427]: from='client.? 172.21.15.81:0/1866844510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:30.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:30 smithi062 ceph-mon[95543]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-14T16:09:30.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:30 smithi062 ceph-mon[95543]: from='client.? 172.21.15.81:0/1866844510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:30.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:30 smithi081 ceph-mon[127795]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-14T16:09:30.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:30 smithi081 ceph-mon[127795]: from='client.? 172.21.15.81:0/1866844510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:32.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:32 smithi164 ceph-mon[97427]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:09:32.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:32 smithi062 ceph-mon[95543]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:09:32.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:32 smithi081 ceph-mon[127795]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:09:34.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:34 smithi164 ceph-mon[97427]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T16:09:34.844 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:34 smithi081 ceph-mon[127795]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T16:09:34.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:34 smithi062 ceph-mon[95543]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T16:09:36.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:36 smithi164 ceph-mon[97427]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-14T16:09:36.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:36 smithi062 ceph-mon[95543]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-14T16:09:36.876 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:36 smithi081 ceph-mon[127795]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-14T16:09:37.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T16:09:37.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:37.798 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T16:09:37.799 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:37.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T16:09:37.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:38.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:38 smithi164 ceph-mon[97427]: Deploying daemon osd.4 on smithi081 2023-07-14T16:09:38.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:38 smithi164 ceph-mon[97427]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:09:38.858 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:38 smithi081 ceph-mon[127795]: Deploying daemon osd.4 on smithi081 2023-07-14T16:09:38.858 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:38 smithi081 ceph-mon[127795]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:09:38.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:38 smithi062 ceph-mon[95543]: Deploying daemon osd.4 on smithi081 2023-07-14T16:09:38.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:38 smithi062 ceph-mon[95543]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:09:40.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:40 smithi164 ceph-mon[97427]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:40.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:40 smithi062 ceph-mon[95543]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:40.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:40 smithi081 ceph-mon[127795]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:42.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:42 smithi164 ceph-mon[97427]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:42.819 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:42 smithi081 ceph-mon[127795]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:42.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:42 smithi062 ceph-mon[95543]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:44.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:44.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:44.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:44.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:44 smithi081 ceph-mon[127795]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:44.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:44.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:44.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:44.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:44 smithi164 ceph-mon[97427]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:44.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:44.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:44.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:44.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:44 smithi062 ceph-mon[95543]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:46.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:46 smithi081 ceph-mon[127795]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:46.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:46.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:46.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:46.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:46.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:46.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:46 smithi164 ceph-mon[97427]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:46.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:46.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:46.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:46.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:46.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:46.854 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 4 on host 'smithi081' 2023-07-14T16:09:46.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.878+0000 7f64cbfff700 1 -- 172.21.15.81:0/3545175426 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f64e4060950 con 0x7f64cc0a11d0 2023-07-14T16:09:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64cc0a11d0 msgr2=0x7f64cc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64cc0a11d0 0x7f64cc0a3680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64d80098e0 tx=0x7f64d8000bc0 comp rx=0 tx=0).stop 2023-07-14T16:09:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e40ffb90 msgr2=0x7f64e41957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e40ffb90 0x7f64e41957e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f64d400d980 tx=0x7f64d400dd40 comp rx=0 tx=0).stop 2023-07-14T16:09:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 shutdown_connections 2023-07-14T16:09:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e4101720 0x7f64e419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:46.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64cc0a11d0 0x7f64cc0a3680 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:46.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64e40ffb90 0x7f64e41957e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:46.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 --2- 172.21.15.81:0/3545175426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64e4100d90 0x7f64e4195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:46.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 >> 172.21.15.81:0/3545175426 conn(0x7f64e40fb1a0 msgr2=0x7f64e40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:09:46.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 shutdown_connections 2023-07-14T16:09:46.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:09:46.881+0000 7f64e93fd700 1 -- 172.21.15.81:0/3545175426 wait complete. 2023-07-14T16:09:46.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:46 smithi062 ceph-mon[95543]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:46.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:46.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:46.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:46.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:46.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:47.305 DEBUG:teuthology.orchestra.run.smithi081:osd.4> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.4.service 2023-07-14T16:09:47.311 INFO:tasks.cephadm:Deploying osd.5 on smithi164 with /dev/vg_nvme/lv_3... 2023-07-14T16:09:47.312 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T16:09:47.385 INFO:journalctl@ceph.osd.4.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:09:48.059 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:09:48.116 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:48.116 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:48.116 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:48.116 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:47 smithi081 ceph-mon[127795]: from='osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T16:09:48.116 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:47 smithi081 ceph-mon[127795]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T16:09:48.117 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:47 smithi081 ceph-mon[127795]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:48.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:48.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:48.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:48.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:47 smithi062 ceph-mon[95543]: from='osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T16:09:48.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:47 smithi062 ceph-mon[95543]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T16:09:48.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:47 smithi062 ceph-mon[95543]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:48.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:48.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:48.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:09:48.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:47 smithi164 ceph-mon[97427]: from='osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T16:09:48.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:47 smithi164 ceph-mon[97427]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T16:09:48.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:47 smithi164 ceph-mon[97427]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:48.841 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:48 smithi081 ceph-mon[127795]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T16:09:48.841 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:48 smithi081 ceph-mon[127795]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T16:09:48.841 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:48.841 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:48 smithi081 ceph-mon[127795]: from='osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:09:48.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:48 smithi081 ceph-mon[127795]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:09:48.842 INFO:journalctl@ceph.osd.4.smithi081.stdout:Jul 14 16:09:48 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-4[138437]: 2023-07-14T16:09:48.853+0000 7fb95e90c700 -1 osd.4 0 waiting for initial osdmap 2023-07-14T16:09:48.842 INFO:journalctl@ceph.osd.4.smithi081.stdout:Jul 14 16:09:48 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-4[138437]: 2023-07-14T16:09:48.865+0000 7fb9586fd700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:09:48.932 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:48 smithi164 ceph-mon[97427]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T16:09:48.933 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:48 smithi164 ceph-mon[97427]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T16:09:48.933 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:48.933 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:48 smithi164 ceph-mon[97427]: from='osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:09:48.933 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:48 smithi164 ceph-mon[97427]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:09:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:48 smithi062 ceph-mon[95543]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T16:09:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:48 smithi062 ceph-mon[95543]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T16:09:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:48 smithi062 ceph-mon[95543]: from='osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:09:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:48 smithi062 ceph-mon[95543]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:09:50.024 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T16:09:50.024 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T16:09:50.024 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2023-07-14T16:09:50.024 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2023-07-14T16:09:50.025 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0316716 s, 331 MB/s 2023-07-14T16:09:50.025 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2023-07-14T16:09:50.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:49 smithi062 ceph-mon[95543]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:09:50.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:49 smithi062 ceph-mon[95543]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T16:09:50.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:49 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:50.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:49 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:50.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:49 smithi062 ceph-mon[95543]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:50.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:49 smithi081 ceph-mon[127795]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:09:50.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:49 smithi081 ceph-mon[127795]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T16:09:50.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:49 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:50.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:49 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:50.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:49 smithi081 ceph-mon[127795]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:50.242 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:49 smithi164 ceph-mon[97427]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:09:50.242 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:49 smithi164 ceph-mon[97427]: osdmap e33: 5 total, 4 up, 5 in 2023-07-14T16:09:50.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:49 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:50.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:49 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:50.243 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:49 smithi164 ceph-mon[97427]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-14T16:09:50.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2023-07-14T16:09:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:09:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:09:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] boot 2023-07-14T16:09:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T16:09:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi081 2023-07-14T16:09:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi081 to 8633M 2023-07-14T16:09:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:09:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:09:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] boot 2023-07-14T16:09:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T16:09:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi081 2023-07-14T16:09:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi081 to 8633M 2023-07-14T16:09:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:09:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:09:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: osd.4 [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] boot 2023-07-14T16:09:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T16:09:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T16:09:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi081 2023-07-14T16:09:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:51.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:09:51.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi081 to 8633M 2023-07-14T16:09:51.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:51.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:09:51.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:09:51.355 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:09:52.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:52 smithi164 ceph-mon[97427]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T16:09:52.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:52 smithi164 ceph-mon[97427]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:52.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:52 smithi062 ceph-mon[95543]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T16:09:52.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:52 smithi062 ceph-mon[95543]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:52.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:52 smithi081 ceph-mon[127795]: osdmap e35: 5 total, 5 up, 5 in 2023-07-14T16:09:52.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:52 smithi081 ceph-mon[127795]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:52.894 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 -- 172.21.15.164:0/2193963888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 msgr2=0x7f2c54108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:52.895 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2193963888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 0x7f2c54108070 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00b0a0 tx=0x7f2c4c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:09:52.895 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 -- 172.21.15.164:0/2193963888 shutdown_connections 2023-07-14T16:09:52.895 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2193963888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 0x7f2c54108070 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:52.895 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2193963888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c540ffc20 0x7f2c54100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:52.895 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2193963888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c54100e20 0x7f2c54101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:52.895 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 -- 172.21.15.164:0/2193963888 >> 172.21.15.164:0/2193963888 conn(0x7f2c540fb250 msgr2=0x7f2c540fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:09:52.896 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 -- 172.21.15.164:0/2193963888 shutdown_connections 2023-07-14T16:09:52.896 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.918+0000 7f2c590a7700 1 -- 172.21.15.164:0/2193963888 wait complete. 2023-07-14T16:09:52.897 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.919+0000 7f2c590a7700 1 Processor -- start 2023-07-14T16:09:52.897 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c590a7700 1 -- start start 2023-07-14T16:09:52.897 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c590a7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c540ffc20 0x7f2c540742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:09:52.897 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c590a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c54100e20 0x7f2c54072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:09:52.898 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c590a7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 0x7f2c54072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:09:52.898 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c590a7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c54199760 con 0x7f2c54100e20 2023-07-14T16:09:52.898 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c590a7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c54073380 con 0x7f2c540ffc20 2023-07-14T16:09:52.898 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c590a7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c54073540 con 0x7f2c541017b0 2023-07-14T16:09:52.898 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c5359e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 0x7f2c54072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:09:52.898 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c5259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c54100e20 0x7f2c54072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:09:52.898 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c5359e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 0x7f2c54072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:34108/0 (socket says 172.21.15.164:34108) 2023-07-14T16:09:52.898 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c5359e700 1 -- 172.21.15.164:0/2549644075 learned_addr learned my addr 172.21.15.164:0/2549644075 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:09:52.899 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.920+0000 7f2c52d9d700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c540ffc20 0x7f2c540742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:09:52.899 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c5359e700 1 -- 172.21.15.164:0/2549644075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c540ffc20 msgr2=0x7f2c540742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:52.899 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c5359e700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c540ffc20 0x7f2c540742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:52.899 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c5359e700 1 -- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c54100e20 msgr2=0x7f2c54072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:09:52.899 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c5359e700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c54100e20 0x7f2c54072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:09:52.899 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c5359e700 1 -- 172.21.15.164:0/2549644075 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c009d00 con 0x7f2c541017b0 2023-07-14T16:09:52.899 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c52d9d700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c540ffc20 0x7f2c540742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:09:52.899 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c5359e700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 0x7f2c54072e40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00d010 tx=0x7f2c4c00bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:09:52.900 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c3bfff700 1 -- 172.21.15.164:0/2549644075 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00e040 con 0x7f2c541017b0 2023-07-14T16:09:52.900 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c541a4130 con 0x7f2c541017b0 2023-07-14T16:09:52.900 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c3bfff700 1 -- 172.21.15.164:0/2549644075 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c4c003e10 con 0x7f2c541017b0 2023-07-14T16:09:52.900 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c3bfff700 1 -- 172.21.15.164:0/2549644075 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c005260 con 0x7f2c541017b0 2023-07-14T16:09:52.900 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.921+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c541a4490 con 0x7f2c541017b0 2023-07-14T16:09:52.901 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.922+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5410b2a0 con 0x7f2c541017b0 2023-07-14T16:09:52.901 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.925+0000 7f2c3bfff700 1 -- 172.21.15.164:0/2549644075 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c4c01da10 con 0x7f2c541017b0 2023-07-14T16:09:52.902 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.927+0000 7f2c3bfff700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c3c0a1150 0x7f2c3c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:09:52.903 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.927+0000 7f2c3bfff700 1 -- 172.21.15.164:0/2549644075 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f2c4c0cd150 con 0x7f2c541017b0 2023-07-14T16:09:52.903 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.927+0000 7f2c52d9d700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c3c0a1150 0x7f2c3c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:09:52.904 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.928+0000 7f2c52d9d700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c3c0a1150 0x7f2c3c0a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c54100c80 tx=0x7f2c44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:09:52.905 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:52.930+0000 7f2c3bfff700 1 -- 172.21.15.164:0/2549644075 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c4c097900 con 0x7f2c541017b0 2023-07-14T16:09:53.107 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:09:53.131+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c54060950 con 0x7f2c3c0a1150 2023-07-14T16:09:53.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:53.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:53.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:53.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:53.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:53.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:53.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:09:53.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:09:53.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:09:54.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:54 smithi164 ceph-mon[97427]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:54.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:54 smithi164 ceph-mon[97427]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:54.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:54 smithi062 ceph-mon[95543]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:54.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:54 smithi062 ceph-mon[95543]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:54.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:54 smithi081 ceph-mon[127795]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:09:54.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:54 smithi081 ceph-mon[127795]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:56.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:56 smithi164 ceph-mon[97427]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:56.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:56 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/1652843691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]: dispatch 2023-07-14T16:09:56.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:56 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]: dispatch 2023-07-14T16:09:56.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:56 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]': finished 2023-07-14T16:09:56.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:56 smithi164 ceph-mon[97427]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T16:09:56.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:09:56.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:56 smithi062 ceph-mon[95543]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:56.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:56 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/1652843691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]: dispatch 2023-07-14T16:09:56.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:56 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]: dispatch 2023-07-14T16:09:56.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:56 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]': finished 2023-07-14T16:09:56.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:56 smithi062 ceph-mon[95543]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T16:09:56.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:09:56.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:56 smithi081 ceph-mon[127795]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:56.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:56 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/1652843691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]: dispatch 2023-07-14T16:09:56.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:56 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]: dispatch 2023-07-14T16:09:56.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:56 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee43949b-be2c-40e7-8efa-b8768a6d0165"}]': finished 2023-07-14T16:09:56.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:56 smithi081 ceph-mon[127795]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T16:09:56.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:09:57.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:57 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/2577446047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:57.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:57 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/2577446047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:57.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:57 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/2577446047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:09:58.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:09:58 smithi164 ceph-mon[97427]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:58.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:09:58 smithi062 ceph-mon[95543]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:09:58.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:09:58 smithi081 ceph-mon[127795]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:00.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:00 smithi164 ceph-mon[97427]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:00.690 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:00 smithi164 ceph-mon[97427]: overall HEALTH_OK 2023-07-14T16:10:00.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:00 smithi062 ceph-mon[95543]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:00.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:00 smithi062 ceph-mon[95543]: overall HEALTH_OK 2023-07-14T16:10:00.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:00 smithi081 ceph-mon[127795]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:00.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:00 smithi081 ceph-mon[127795]: overall HEALTH_OK 2023-07-14T16:10:02.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:02 smithi164 ceph-mon[97427]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:02.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:02 smithi062 ceph-mon[95543]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:02.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:02 smithi081 ceph-mon[127795]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:04.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:04 smithi164 ceph-mon[97427]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:04.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T16:10:04.746 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:04 smithi062 ceph-mon[95543]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T16:10:04.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:04.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:04 smithi081 ceph-mon[127795]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:04.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:04 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T16:10:04.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:04 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:05.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:05 smithi164 ceph-mon[97427]: Deploying daemon osd.5 on smithi164 2023-07-14T16:10:05.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:05 smithi062 ceph-mon[95543]: Deploying daemon osd.5 on smithi164 2023-07-14T16:10:05.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:05 smithi081 ceph-mon[127795]: Deploying daemon osd.5 on smithi164 2023-07-14T16:10:06.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:06 smithi164 ceph-mon[97427]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:06.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:06 smithi062 ceph-mon[95543]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:06.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:06 smithi081 ceph-mon[127795]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:08.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:08 smithi164 ceph-mon[97427]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:08.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:08 smithi062 ceph-mon[95543]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:08.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:08 smithi081 ceph-mon[127795]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:10.773 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:10 smithi164 ceph-mon[97427]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:10.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:10.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:10.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:10.834 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:10 smithi081 ceph-mon[127795]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:10.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:10 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:10.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:10 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:10.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:10 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:10.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:10 smithi062 ceph-mon[95543]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:10.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:10.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:10.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:12.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:12 smithi164 ceph-mon[97427]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:12.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:12 smithi062 ceph-mon[95543]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:12.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:12 smithi081 ceph-mon[127795]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:14.344 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.368+0000 7f2c3bfff700 1 -- 172.21.15.164:0/2549644075 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2c54060950 con 0x7f2c3c0a1150 2023-07-14T16:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 5 on host 'smithi164' 2023-07-14T16:10:14.347 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c3c0a1150 msgr2=0x7f2c3c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:14.347 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c3c0a1150 0x7f2c3c0a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c54100c80 tx=0x7f2c44009260 comp rx=0 tx=0).stop 2023-07-14T16:10:14.347 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 msgr2=0x7f2c54072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:14.347 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 0x7f2c54072e40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00d010 tx=0x7f2c4c00bcc0 comp rx=0 tx=0).stop 2023-07-14T16:10:14.349 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 shutdown_connections 2023-07-14T16:10:14.349 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c541017b0 0x7f2c54072e40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:14.349 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c3c0a1150 0x7f2c3c0a3600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:14.349 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c540ffc20 0x7f2c540742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:14.350 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.370+0000 7f2c590a7700 1 --2- 172.21.15.164:0/2549644075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c54100e20 0x7f2c54072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:14.350 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.371+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 >> 172.21.15.164:0/2549644075 conn(0x7f2c540fb250 msgr2=0x7f2c540fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:10:14.350 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.371+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 shutdown_connections 2023-07-14T16:10:14.350 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:10:14.371+0000 7f2c590a7700 1 -- 172.21.15.164:0/2549644075 wait complete. 2023-07-14T16:10:14.489 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.489 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.489 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:14.490 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:14.490 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.490 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:14.490 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.490 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:14.490 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:14.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:14.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:14.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:14.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:14.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:14.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:14.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:14.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:14.879 DEBUG:teuthology.orchestra.run.smithi164:osd.5> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.5.service 2023-07-14T16:10:14.883 INFO:tasks.cephadm:Deploying osd.6 on smithi062 with /dev/vg_nvme/lv_2... 2023-07-14T16:10:14.884 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T16:10:14.957 INFO:journalctl@ceph.osd.5.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:10:15.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:15 smithi062 ceph-mon[95543]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T16:10:15.625 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:10:15.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:15 smithi081 ceph-mon[127795]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T16:10:15.654 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:15 smithi164 ceph-mon[97427]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T16:10:16.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:16 smithi164 ceph-mon[97427]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T16:10:16.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:16 smithi164 ceph-mon[97427]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T16:10:16.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:16 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:16.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:16 smithi164 ceph-mon[97427]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:10:16.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:16 smithi164 ceph-mon[97427]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:16.525 INFO:journalctl@ceph.osd.5.smithi164.stdout:Jul 14 16:10:16 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-5[108925]: 2023-07-14T16:10:16.370+0000 7f5392db9700 -1 osd.5 0 waiting for initial osdmap 2023-07-14T16:10:16.525 INFO:journalctl@ceph.osd.5.smithi164.stdout:Jul 14 16:10:16 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-5[108925]: 2023-07-14T16:10:16.379+0000 7f538c3a9700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:10:16.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:16 smithi062 ceph-mon[95543]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T16:10:16.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:16 smithi062 ceph-mon[95543]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T16:10:16.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:16 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:16.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:16 smithi062 ceph-mon[95543]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:10:16.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:16 smithi062 ceph-mon[95543]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:16.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:16 smithi081 ceph-mon[127795]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T16:10:16.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:16 smithi081 ceph-mon[127795]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T16:10:16.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:16 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:16.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:16 smithi081 ceph-mon[127795]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:10:16.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:16 smithi081 ceph-mon[127795]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:17.550 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T16:10:17.551 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T16:10:17.551 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2023-07-14T16:10:17.551 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2023-07-14T16:10:17.551 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0401221 s, 261 MB/s 2023-07-14T16:10:17.551 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2023-07-14T16:10:17.612 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:17 smithi164 ceph-mon[97427]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:10:17.613 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:17 smithi164 ceph-mon[97427]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T16:10:17.613 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:17.613 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:17.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:17 smithi062 ceph-mon[95543]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:10:17.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:17 smithi062 ceph-mon[95543]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T16:10:17.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:17.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:17.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:17 smithi081 ceph-mon[127795]: from='osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:10:17.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:17 smithi081 ceph-mon[127795]: osdmap e38: 6 total, 5 up, 6 in 2023-07-14T16:10:17.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:17.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:18.100 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2023-07-14T16:10:18.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:10:18.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:10:18.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:18.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] boot 2023-07-14T16:10:18.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T16:10:18.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:18.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:18.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:18.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:18.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:18.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:18.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:10:18.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:10:18.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:18.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] boot 2023-07-14T16:10:18.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T16:10:18.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:18.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:18.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:18.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:18.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:18.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:18.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:10:18.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:10:18.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:18.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: osd.5 [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] boot 2023-07-14T16:10:18.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T16:10:18.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T16:10:18.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T16:10:18.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:18.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:18.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:18.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:18.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:18.839 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:10:19.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:19 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi164 2023-07-14T16:10:19.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:19 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi164 to 8633M 2023-07-14T16:10:19.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:19 smithi062 ceph-mon[95543]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T16:10:19.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:19 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi164 2023-07-14T16:10:19.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:19 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi164 to 8633M 2023-07-14T16:10:19.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:19 smithi081 ceph-mon[127795]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T16:10:19.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:19 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi164 2023-07-14T16:10:19.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:19 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi164 to 8633M 2023-07-14T16:10:19.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:19 smithi164 ceph-mon[97427]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T16:10:20.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.360+0000 7f53df059700 1 -- 172.21.15.62:0/147119776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d8100610 msgr2=0x7f53d8106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:20.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.360+0000 7f53df059700 1 --2- 172.21.15.62:0/147119776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d8100610 0x7f53d8106ed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b0a0 tx=0x7f53d000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:10:20.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.360+0000 7f53df059700 1 -- 172.21.15.62:0/147119776 shutdown_connections 2023-07-14T16:10:20.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.360+0000 7f53df059700 1 --2- 172.21.15.62:0/147119776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d8100610 0x7f53d8106ed0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:20.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.360+0000 7f53df059700 1 --2- 172.21.15.62:0/147119776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d80fea80 0x7f53d80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:20.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.360+0000 7f53df059700 1 --2- 172.21.15.62:0/147119776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53d80ffc80 0x7f53d81000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:20.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.360+0000 7f53df059700 1 -- 172.21.15.62:0/147119776 >> 172.21.15.62:0/147119776 conn(0x7f53d80fa0b0 msgr2=0x7f53d80fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:10:20.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.361+0000 7f53df059700 1 -- 172.21.15.62:0/147119776 shutdown_connections 2023-07-14T16:10:20.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.361+0000 7f53df059700 1 -- 172.21.15.62:0/147119776 wait complete. 2023-07-14T16:10:20.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.361+0000 7f53df059700 1 Processor -- start 2023-07-14T16:10:20.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53df059700 1 -- start start 2023-07-14T16:10:20.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53df059700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d80fea80 0x7f53d81958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:10:20.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53df059700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d80ffc80 0x7f53d8195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:10:20.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53df059700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53d8100610 0x7f53d819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:10:20.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53df059700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d819d450 con 0x7f53d80fea80 2023-07-14T16:10:20.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53df059700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d819d590 con 0x7f53d8100610 2023-07-14T16:10:20.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53df059700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d819d720 con 0x7f53d80ffc80 2023-07-14T16:10:20.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53dcdf5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d80fea80 0x7f53d81958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:10:20.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53dcdf5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d80fea80 0x7f53d81958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38206/0 (socket says 172.21.15.62:38206) 2023-07-14T16:10:20.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53dcdf5700 1 -- 172.21.15.62:0/2602984660 learned_addr learned my addr 172.21.15.62:0/2602984660 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:10:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53dd5f6700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53d8100610 0x7f53d819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:10:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.362+0000 7f53d7fff700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d80ffc80 0x7f53d8195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:10:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53dcdf5700 1 -- 172.21.15.62:0/2602984660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d80ffc80 msgr2=0x7f53d8195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53dcdf5700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d80ffc80 0x7f53d8195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53dcdf5700 1 -- 172.21.15.62:0/2602984660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53d8100610 msgr2=0x7f53d819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53dcdf5700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53d8100610 0x7f53d819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53dcdf5700 1 -- 172.21.15.62:0/2602984660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009d00 con 0x7f53d80fea80 2023-07-14T16:10:20.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53dd5f6700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53d8100610 0x7f53d819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:10:20.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53dcdf5700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d80fea80 0x7f53d81958c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00d9b0 tx=0x7f53cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:10:20.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53d5ffb700 1 -- 172.21.15.62:0/2602984660 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004020 con 0x7f53d80fea80 2023-07-14T16:10:20.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53d5ffb700 1 -- 172.21.15.62:0/2602984660 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53cc009730 con 0x7f53d80fea80 2023-07-14T16:10:20.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d819da00 con 0x7f53d80fea80 2023-07-14T16:10:20.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.363+0000 7f53d5ffb700 1 -- 172.21.15.62:0/2602984660 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0038e0 con 0x7f53d80fea80 2023-07-14T16:10:20.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.364+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d819df50 con 0x7f53d80fea80 2023-07-14T16:10:20.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.365+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d810a120 con 0x7f53d80fea80 2023-07-14T16:10:20.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.366+0000 7f53d5ffb700 1 -- 172.21.15.62:0/2602984660 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f53cc0098a0 con 0x7f53d80fea80 2023-07-14T16:10:20.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.368+0000 7f53d5ffb700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53c80a11d0 0x7f53c80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:10:20.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.368+0000 7f53d5ffb700 1 -- 172.21.15.62:0/2602984660 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f53cc0cb6d0 con 0x7f53d80fea80 2023-07-14T16:10:20.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.368+0000 7f53d7fff700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53c80a11d0 0x7f53c80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:10:20.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.369+0000 7f53d7fff700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53c80a11d0 0x7f53c80a3680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009c60 tx=0x7f53c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:10:20.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.372+0000 7f53d5ffb700 1 -- 172.21.15.62:0/2602984660 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f53cc095c20 con 0x7f53d80fea80 2023-07-14T16:10:20.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:20.593+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f53d8068700 con 0x7f53c80a11d0 2023-07-14T16:10:20.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:20 smithi081 ceph-mon[127795]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T16:10:20.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:20 smithi081 ceph-mon[127795]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:20.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:20 smithi164 ceph-mon[97427]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T16:10:20.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:20 smithi164 ceph-mon[97427]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:20.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:20 smithi062 ceph-mon[95543]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T16:10:20.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:20 smithi062 ceph-mon[95543]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:21.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:21 smithi062 ceph-mon[95543]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T16:10:21.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:21 smithi062 ceph-mon[95543]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:10:21.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:10:21.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:10:21.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:21.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:21 smithi081 ceph-mon[127795]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T16:10:21.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:21 smithi081 ceph-mon[127795]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:10:21.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:10:21.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:10:21.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:21.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:21 smithi164 ceph-mon[97427]: osdmap e42: 6 total, 6 up, 6 in 2023-07-14T16:10:21.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:21 smithi164 ceph-mon[97427]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:10:21.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:10:21.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:10:21.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:22.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:22 smithi081 ceph-mon[127795]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T16:10:22.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:22 smithi081 ceph-mon[127795]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:22.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:22 smithi164 ceph-mon[97427]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T16:10:22.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:22 smithi164 ceph-mon[97427]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:22 smithi062 ceph-mon[95543]: osdmap e43: 6 total, 6 up, 6 in 2023-07-14T16:10:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:22 smithi062 ceph-mon[95543]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:24 smithi164 ceph-mon[97427]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:24 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3421725636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47c5b778-8a5b-4a01-9976-c0ddf8635724"}]: dispatch 2023-07-14T16:10:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:24 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3421725636' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47c5b778-8a5b-4a01-9976-c0ddf8635724"}]': finished 2023-07-14T16:10:24.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:24 smithi164 ceph-mon[97427]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T16:10:24.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:24.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:24 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2281008426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:10:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:24 smithi062 ceph-mon[95543]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:24 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3421725636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47c5b778-8a5b-4a01-9976-c0ddf8635724"}]: dispatch 2023-07-14T16:10:24.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:24 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3421725636' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47c5b778-8a5b-4a01-9976-c0ddf8635724"}]': finished 2023-07-14T16:10:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:24 smithi062 ceph-mon[95543]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T16:10:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:24 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2281008426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:10:24.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:24 smithi081 ceph-mon[127795]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:24.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:24 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3421725636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47c5b778-8a5b-4a01-9976-c0ddf8635724"}]: dispatch 2023-07-14T16:10:24.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:24 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3421725636' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47c5b778-8a5b-4a01-9976-c0ddf8635724"}]': finished 2023-07-14T16:10:24.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:24 smithi081 ceph-mon[127795]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T16:10:24.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:24.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:24 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2281008426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:10:26.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:26 smithi164 ceph-mon[97427]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:10:26.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:26 smithi062 ceph-mon[95543]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:10:26.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:26 smithi081 ceph-mon[127795]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:10:28.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:28 smithi164 ceph-mon[97427]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-14T16:10:28.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:28 smithi062 ceph-mon[95543]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-14T16:10:28.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:28 smithi081 ceph-mon[127795]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-14T16:10:30.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:30 smithi164 ceph-mon[97427]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T16:10:30.785 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:30 smithi062 ceph-mon[95543]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T16:10:30.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:30 smithi081 ceph-mon[127795]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T16:10:31.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T16:10:31.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:31.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T16:10:31.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:31.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T16:10:31.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:32.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:32 smithi062 ceph-mon[95543]: Deploying daemon osd.6 on smithi062 2023-07-14T16:10:32.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:32 smithi062 ceph-mon[95543]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:10:32.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:32 smithi164 ceph-mon[97427]: Deploying daemon osd.6 on smithi062 2023-07-14T16:10:32.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:32 smithi164 ceph-mon[97427]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:10:32.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:32 smithi081 ceph-mon[127795]: Deploying daemon osd.6 on smithi062 2023-07-14T16:10:32.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:32 smithi081 ceph-mon[127795]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:10:34.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:34 smithi062 ceph-mon[95543]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:10:34.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:34 smithi164 ceph-mon[97427]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:10:34.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:34 smithi081 ceph-mon[127795]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-14T16:10:36.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:36 smithi164 ceph-mon[97427]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:36.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:36 smithi062 ceph-mon[95543]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:36.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:36 smithi081 ceph-mon[127795]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:38.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:38 smithi062 ceph-mon[95543]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:38.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:38.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:38.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:38.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:38 smithi164 ceph-mon[97427]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:38.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:38.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:38.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:38.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:38 smithi081 ceph-mon[127795]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:38.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:38.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:38.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:40.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:40 smithi164 ceph-mon[97427]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:40.783 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:40 smithi062 ceph-mon[95543]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:40.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:40 smithi081 ceph-mon[127795]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:41.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.503+0000 7f53d5ffb700 1 -- 172.21.15.62:0/2602984660 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f53d8068700 con 0x7f53c80a11d0 2023-07-14T16:10:41.481 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 6 on host 'smithi062' 2023-07-14T16:10:41.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53c80a11d0 msgr2=0x7f53c80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:41.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53c80a11d0 0x7f53c80a3680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009c60 tx=0x7f53c4009260 comp rx=0 tx=0).stop 2023-07-14T16:10:41.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d80fea80 msgr2=0x7f53d81958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:41.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d80fea80 0x7f53d81958c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00d9b0 tx=0x7f53cc00dcc0 comp rx=0 tx=0).stop 2023-07-14T16:10:41.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 shutdown_connections 2023-07-14T16:10:41.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53c80a11d0 0x7f53c80a3680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:41.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53d8100610 0x7f53d819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:41.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53d80fea80 0x7f53d81958c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:41.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 --2- 172.21.15.62:0/2602984660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d80ffc80 0x7f53d8195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:41.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 >> 172.21.15.62:0/2602984660 conn(0x7f53d80fa0b0 msgr2=0x7f53d8067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:10:41.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 shutdown_connections 2023-07-14T16:10:41.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:10:41.506+0000 7f53df059700 1 -- 172.21.15.62:0/2602984660 wait complete. 2023-07-14T16:10:41.676 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.676 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.676 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:41.676 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:41.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T16:10:41.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:41.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:41.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:41.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T16:10:41.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:41.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:41.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:41.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T16:10:41.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:10:41.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:41.959 DEBUG:teuthology.orchestra.run.smithi062:osd.6> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.6.service 2023-07-14T16:10:41.963 INFO:tasks.cephadm:Deploying osd.7 on smithi081 with /dev/vg_nvme/lv_2... 2023-07-14T16:10:41.964 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T16:10:42.040 INFO:journalctl@ceph.osd.6.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:10:42.692 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:10:42.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:42 smithi062 ceph-mon[95543]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:42.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:42 smithi062 ceph-mon[95543]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T16:10:42.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:42 smithi062 ceph-mon[95543]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T16:10:42.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:42.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:42 smithi062 ceph-mon[95543]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:10:42.809 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jul 14 16:10:42 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-6[119574]: 2023-07-14T16:10:42.572+0000 7ffa42f39700 -1 osd.6 0 waiting for initial osdmap 2023-07-14T16:10:42.809 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jul 14 16:10:42 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-6[119574]: 2023-07-14T16:10:42.587+0000 7ffa3c529700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:10:42.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:42 smithi081 ceph-mon[127795]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:42.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:42 smithi081 ceph-mon[127795]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T16:10:42.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:42 smithi081 ceph-mon[127795]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T16:10:42.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:42.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:42 smithi081 ceph-mon[127795]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:10:43.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:42 smithi164 ceph-mon[97427]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:43.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:42 smithi164 ceph-mon[97427]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T16:10:43.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:42 smithi164 ceph-mon[97427]: osdmap e45: 7 total, 6 up, 7 in 2023-07-14T16:10:43.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:42 smithi164 ceph-mon[97427]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:10:43.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:43 smithi062 ceph-mon[95543]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:10:43.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:43 smithi062 ceph-mon[95543]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T16:10:43.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:43 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:43.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:43 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:43.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:43 smithi081 ceph-mon[127795]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:10:43.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:43 smithi081 ceph-mon[127795]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T16:10:43.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:43 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:43.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:43 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:44.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:43 smithi164 ceph-mon[97427]: from='osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:10:44.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:43 smithi164 ceph-mon[97427]: osdmap e46: 7 total, 6 up, 7 in 2023-07-14T16:10:44.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:43 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:44.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:43 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:44.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:44 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:10:44.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:44 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:10:44.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:44 smithi062 ceph-mon[95543]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:44.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:44.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:44 smithi062 ceph-mon[95543]: osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] boot 2023-07-14T16:10:44.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:44 smithi062 ceph-mon[95543]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T16:10:44.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:44.709 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T16:10:44.709 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T16:10:44.709 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2023-07-14T16:10:44.710 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2023-07-14T16:10:44.710 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.054953 s, 191 MB/s 2023-07-14T16:10:44.710 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2023-07-14T16:10:44.886 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:44 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:10:44.886 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:44 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:10:44.886 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:44 smithi081 ceph-mon[127795]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:44.886 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:44.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:44 smithi081 ceph-mon[127795]: osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] boot 2023-07-14T16:10:44.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:44 smithi081 ceph-mon[127795]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T16:10:44.887 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:45.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:44 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:10:45.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:44 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:10:45.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:44 smithi164 ceph-mon[97427]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T16:10:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:44 smithi164 ceph-mon[97427]: osd.6 [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] boot 2023-07-14T16:10:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:44 smithi164 ceph-mon[97427]: osdmap e47: 7 total, 7 up, 7 in 2023-07-14T16:10:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T16:10:45.171 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi081:vg_nvme/lv_2 2023-07-14T16:10:45.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T16:10:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:45.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:45.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:45.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:45.892 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:10:45.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T16:10:45.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:45.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:45.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:45.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:45.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:45.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:45.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:45.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:45.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:46.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: osdmap e48: 7 total, 7 up, 7 in 2023-07-14T16:10:46.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:46.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:10:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:10:46.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:10:46.808 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:46 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi062 2023-07-14T16:10:46.809 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:46 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi062 to 5755M 2023-07-14T16:10:46.809 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:46 smithi081 ceph-mon[127795]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:46.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:46 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi062 2023-07-14T16:10:46.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:46 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi062 to 5755M 2023-07-14T16:10:46.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:46 smithi062 ceph-mon[95543]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:47.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:46 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi062 2023-07-14T16:10:47.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:46 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi062 to 5755M 2023-07-14T16:10:47.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:46 smithi164 ceph-mon[97427]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:47.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 -- 172.21.15.81:0/3709178150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4101720 msgr2=0x7f48c4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:47.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 --2- 172.21.15.81:0/3709178150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4101720 0x7f48c4107fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00b0a0 tx=0x7f48bc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:10:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 -- 172.21.15.81:0/3709178150 shutdown_connections 2023-07-14T16:10:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 --2- 172.21.15.81:0/3709178150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4101720 0x7f48c4107fe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 --2- 172.21.15.81:0/3709178150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c40ffb90 0x7f48c40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 --2- 172.21.15.81:0/3709178150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48c4100d90 0x7f48c41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:47.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 -- 172.21.15.81:0/3709178150 >> 172.21.15.81:0/3709178150 conn(0x7f48c40fb1a0 msgr2=0x7f48c40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:10:47.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 -- 172.21.15.81:0/3709178150 shutdown_connections 2023-07-14T16:10:47.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.403+0000 7f48c977e700 1 -- 172.21.15.81:0/3709178150 wait complete. 2023-07-14T16:10:47.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.404+0000 7f48c977e700 1 Processor -- start 2023-07-14T16:10:47.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.404+0000 7f48c977e700 1 -- start start 2023-07-14T16:10:47.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c977e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48c40ffb90 0x7f48c4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:10:47.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c977e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4100d90 0x7f48c4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:10:47.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c977e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4101720 0x7f48c419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:10:47.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c977e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c419f630 con 0x7f48c4101720 2023-07-14T16:10:47.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c977e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c419f770 con 0x7f48c4100d90 2023-07-14T16:10:47.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c977e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c419f900 con 0x7f48c40ffb90 2023-07-14T16:10:47.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4100d90 0x7f48c4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:10:47.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4100d90 0x7f48c4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50576/0 (socket says 172.21.15.81:50576) 2023-07-14T16:10:47.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c27fc700 1 -- 172.21.15.81:0/138909352 learned_addr learned my addr 172.21.15.81:0/138909352 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-07-14T16:10:47.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c2ffd700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48c40ffb90 0x7f48c4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:10:47.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c37fe700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4101720 0x7f48c419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:10:47.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c27fc700 1 -- 172.21.15.81:0/138909352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48c40ffb90 msgr2=0x7f48c4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:47.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c27fc700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48c40ffb90 0x7f48c4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:47.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c27fc700 1 -- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4101720 msgr2=0x7f48c419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:10:47.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c27fc700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4101720 0x7f48c419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:10:47.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.405+0000 7f48c27fc700 1 -- 172.21.15.81:0/138909352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc009d00 con 0x7f48c4100d90 2023-07-14T16:10:47.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.406+0000 7f48c27fc700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4100d90 0x7f48c4197fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48b800e9a0 tx=0x7f48b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:10:47.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.406+0000 7f48abfff700 1 -- 172.21.15.81:0/138909352 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8009790 con 0x7f48c4100d90 2023-07-14T16:10:47.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.406+0000 7f48abfff700 1 -- 172.21.15.81:0/138909352 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48b8010430 con 0x7f48c4100d90 2023-07-14T16:10:47.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.406+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c419fbe0 con 0x7f48c4100d90 2023-07-14T16:10:47.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.406+0000 7f48abfff700 1 -- 172.21.15.81:0/138909352 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b800cea0 con 0x7f48c4100d90 2023-07-14T16:10:47.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.406+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c41a0130 con 0x7f48c4100d90 2023-07-14T16:10:47.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.407+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c41920d0 con 0x7f48c4100d90 2023-07-14T16:10:47.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.409+0000 7f48abfff700 1 -- 172.21.15.81:0/138909352 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f48b80105a0 con 0x7f48c4100d90 2023-07-14T16:10:47.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.412+0000 7f48abfff700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48ac0a11d0 0x7f48ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:10:47.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.412+0000 7f48abfff700 1 -- 172.21.15.81:0/138909352 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f48b8014070 con 0x7f48c4100d90 2023-07-14T16:10:47.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.412+0000 7f48c2ffd700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48ac0a11d0 0x7f48ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:10:47.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.413+0000 7f48c2ffd700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48ac0a11d0 0x7f48ac0a3680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48c4100bf0 tx=0x7f48b4007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:10:47.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.416+0000 7f48abfff700 1 -- 172.21.15.81:0/138909352 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f48b8051790 con 0x7f48c4100d90 2023-07-14T16:10:47.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:10:47.619+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f48c4060950 con 0x7f48ac0a11d0 2023-07-14T16:10:48.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:48 smithi062 ceph-mon[95543]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:48.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:48 smithi062 ceph-mon[95543]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:10:48.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:10:48.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:10:48.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:48.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:48 smithi081 ceph-mon[127795]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:48.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:48 smithi081 ceph-mon[127795]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:10:48.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:10:48.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:10:48.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:49.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:48 smithi164 ceph-mon[97427]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:49.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:48 smithi164 ceph-mon[97427]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:10:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:10:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:10:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:50.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:50 smithi062 ceph-mon[95543]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:50.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:50 smithi081 ceph-mon[127795]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:51.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:50 smithi164 ceph-mon[97427]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:51.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:51 smithi062 ceph-mon[95543]: from='client.? 172.21.15.81:0/4026560592' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]: dispatch 2023-07-14T16:10:51.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:51 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]: dispatch 2023-07-14T16:10:51.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:51 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]': finished 2023-07-14T16:10:51.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:51 smithi062 ceph-mon[95543]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T16:10:51.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:51 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:10:51.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:51 smithi062 ceph-mon[95543]: from='client.? 172.21.15.81:0/3110721628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:10:51.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:51 smithi081 ceph-mon[127795]: from='client.? 172.21.15.81:0/4026560592' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]: dispatch 2023-07-14T16:10:51.979 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:51 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]: dispatch 2023-07-14T16:10:51.979 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:51 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]': finished 2023-07-14T16:10:51.979 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:51 smithi081 ceph-mon[127795]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T16:10:51.979 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:51 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:10:51.979 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:51 smithi081 ceph-mon[127795]: from='client.? 172.21.15.81:0/3110721628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:10:52.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:51 smithi164 ceph-mon[97427]: from='client.? 172.21.15.81:0/4026560592' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]: dispatch 2023-07-14T16:10:52.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:51 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]: dispatch 2023-07-14T16:10:52.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:51 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02e215e4-f68b-44d5-8172-836fed42ce7c"}]': finished 2023-07-14T16:10:52.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:51 smithi164 ceph-mon[97427]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T16:10:52.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:51 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:10:52.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:51 smithi164 ceph-mon[97427]: from='client.? 172.21.15.81:0/3110721628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:10:52.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:52 smithi062 ceph-mon[95543]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:52.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:52 smithi081 ceph-mon[127795]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:53.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:52 smithi164 ceph-mon[97427]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:54.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:54 smithi062 ceph-mon[95543]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:54.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:54 smithi081 ceph-mon[127795]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:55.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:54 smithi164 ceph-mon[97427]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:56.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:56 smithi062 ceph-mon[95543]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:56.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:56 smithi081 ceph-mon[127795]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:57.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:56 smithi164 ceph-mon[97427]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:58.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:58 smithi081 ceph-mon[127795]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:58.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T16:10:58.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:59.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:58 smithi164 ceph-mon[97427]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:59.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T16:10:59.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:59.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:58 smithi062 ceph-mon[95543]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:10:59.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T16:10:59.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:10:59.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:10:59 smithi081 ceph-mon[127795]: Deploying daemon osd.7 on smithi081 2023-07-14T16:11:00.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:10:59 smithi164 ceph-mon[97427]: Deploying daemon osd.7 on smithi081 2023-07-14T16:11:00.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:10:59 smithi062 ceph-mon[95543]: Deploying daemon osd.7 on smithi081 2023-07-14T16:11:00.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:00 smithi081 ceph-mon[127795]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:01.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:00 smithi164 ceph-mon[97427]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:01.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:00 smithi062 ceph-mon[95543]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:02 smithi081 ceph-mon[127795]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:03.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:02 smithi164 ceph-mon[97427]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:03.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:02 smithi062 ceph-mon[95543]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:04.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:04 smithi081 ceph-mon[127795]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:04.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:04 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:04.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:04 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:04.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:04 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:04 smithi164 ceph-mon[97427]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:05.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:04 smithi062 ceph-mon[95543]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:06 smithi081 ceph-mon[127795]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:07.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:06 smithi164 ceph-mon[97427]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:07.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:06 smithi062 ceph-mon[95543]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:08.375 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 7 on host 'smithi081' 2023-07-14T16:11:08.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.399+0000 7f48abfff700 1 -- 172.21.15.81:0/138909352 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f48c4060950 con 0x7f48ac0a11d0 2023-07-14T16:11:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48ac0a11d0 msgr2=0x7f48ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48ac0a11d0 0x7f48ac0a3680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48c4100bf0 tx=0x7f48b4007400 comp rx=0 tx=0).stop 2023-07-14T16:11:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4100d90 msgr2=0x7f48c4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4100d90 0x7f48c4197fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48b800e9a0 tx=0x7f48b800ecb0 comp rx=0 tx=0).stop 2023-07-14T16:11:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 shutdown_connections 2023-07-14T16:11:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4101720 0x7f48c419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48c40ffb90 0x7f48c4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48ac0a11d0 0x7f48ac0a3680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 --2- 172.21.15.81:0/138909352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48c4100d90 0x7f48c4197fe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 >> 172.21.15.81:0/138909352 conn(0x7f48c40fb1a0 msgr2=0x7f48c4106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:11:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 shutdown_connections 2023-07-14T16:11:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:11:08.401+0000 7f48c977e700 1 -- 172.21.15.81:0/138909352 wait complete. 2023-07-14T16:11:08.549 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:08.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:08.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:08 smithi081 ceph-mon[127795]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:08.550 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:08 smithi062 ceph-mon[95543]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:08.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:08.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:08.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:08 smithi164 ceph-mon[97427]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:08.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:08.909 DEBUG:teuthology.orchestra.run.smithi081:osd.7> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.7.service 2023-07-14T16:11:08.919 INFO:tasks.cephadm:Deploying osd.8 on smithi164 with /dev/vg_nvme/lv_2... 2023-07-14T16:11:08.919 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T16:11:08.989 INFO:journalctl@ceph.osd.7.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:11:09.588 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:09 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:09.588 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:09 smithi081 ceph-mon[127795]: from='osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T16:11:09.589 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:09 smithi081 ceph-mon[127795]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T16:11:09.589 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:09 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:09.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:09 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:09.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:09 smithi062 ceph-mon[95543]: from='osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T16:11:09.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:09 smithi062 ceph-mon[95543]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T16:11:09.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:09 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:09.649 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:11:09.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:09 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:09.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:09 smithi164 ceph-mon[97427]: from='osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T16:11:09.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:09 smithi164 ceph-mon[97427]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T16:11:09.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:09 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:10.506 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:10 smithi081 ceph-mon[127795]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T16:11:10.506 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:10 smithi081 ceph-mon[127795]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T16:11:10.507 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:10 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:10.507 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:10 smithi081 ceph-mon[127795]: from='osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:11:10.507 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:10 smithi081 ceph-mon[127795]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:11:10.507 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:10 smithi081 ceph-mon[127795]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:10.508 INFO:journalctl@ceph.osd.7.smithi081.stdout:Jul 14 16:11:10 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-7[143993]: 2023-07-14T16:11:10.373+0000 7fbcf9690700 -1 osd.7 0 waiting for initial osdmap 2023-07-14T16:11:10.508 INFO:journalctl@ceph.osd.7.smithi081.stdout:Jul 14 16:11:10 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-7[143993]: 2023-07-14T16:11:10.384+0000 7fbcf3c82700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:11:10.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:10 smithi062 ceph-mon[95543]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T16:11:10.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:10 smithi062 ceph-mon[95543]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T16:11:10.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:10 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:10.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:10 smithi062 ceph-mon[95543]: from='osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:11:10.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:10 smithi062 ceph-mon[95543]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:11:10.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:10 smithi062 ceph-mon[95543]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:10.625 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:10 smithi164 ceph-mon[97427]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T16:11:10.625 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:10 smithi164 ceph-mon[97427]: osdmap e50: 8 total, 7 up, 8 in 2023-07-14T16:11:10.625 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:10 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:10.625 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:10 smithi164 ceph-mon[97427]: from='osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:11:10.625 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:10 smithi164 ceph-mon[97427]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:11:10.626 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:10 smithi164 ceph-mon[97427]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T16:11:11.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:11 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:11:11.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:11 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:11:11.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:11 smithi062 ceph-mon[95543]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:11:11.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:11 smithi062 ceph-mon[95543]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T16:11:11.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:11 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:11.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:11 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:11.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:11 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:11:11.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:11 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:11:11.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:11 smithi081 ceph-mon[127795]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:11:11.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:11 smithi081 ceph-mon[127795]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T16:11:11.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:11 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:11.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:11 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T16:11:11.767 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T16:11:11.768 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2023-07-14T16:11:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2023-07-14T16:11:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0412569 s, 254 MB/s 2023-07-14T16:11:11.768 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 2023-07-14T16:11:11.768 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2023-07-14T16:11:11.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:11 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:11:11.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:11 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:11:11.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:11 smithi164 ceph-mon[97427]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:11:11.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:11 smithi164 ceph-mon[97427]: osdmap e51: 8 total, 7 up, 8 in 2023-07-14T16:11:11.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:11 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:11.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:11 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:12.228 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2023-07-14T16:11:12.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] boot 2023-07-14T16:11:12.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T16:11:12.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:12.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:12.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:12 smithi164 ceph-mon[97427]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T16:11:12.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] boot 2023-07-14T16:11:12.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T16:11:12.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:12.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:12.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:12.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:12.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:12 smithi062 ceph-mon[95543]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T16:11:12.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: osd.7 [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] boot 2023-07-14T16:11:12.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T16:11:12.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T16:11:12.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:12.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:12.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:12.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:12.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:12.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:12 smithi081 ceph-mon[127795]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T16:11:12.954 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:11:13.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:13 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi081 2023-07-14T16:11:13.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:13 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi081 to 5755M 2023-07-14T16:11:13.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:13 smithi164 ceph-mon[97427]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T16:11:13.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:13 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi081 2023-07-14T16:11:13.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:13 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi081 to 5755M 2023-07-14T16:11:13.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:13 smithi062 ceph-mon[95543]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T16:11:13.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:13 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi081 2023-07-14T16:11:13.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:13 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi081 to 5755M 2023-07-14T16:11:13.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:13 smithi081 ceph-mon[127795]: osdmap e54: 8 total, 8 up, 8 in 2023-07-14T16:11:14.578 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 -- 172.21.15.164:0/3861378178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b70100d90 msgr2=0x7f7b701011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:14.578 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 --2- 172.21.15.164:0/3861378178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b70100d90 0x7f7b701011e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009990 tx=0x7f7b64009ca0 comp rx=0 tx=0).stop 2023-07-14T16:11:14.578 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 -- 172.21.15.164:0/3861378178 shutdown_connections 2023-07-14T16:11:14.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 --2- 172.21.15.164:0/3861378178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b70101720 0x7f7b70107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:14.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 --2- 172.21.15.164:0/3861378178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b700ffb90 0x7f7b700fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:14.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 --2- 172.21.15.164:0/3861378178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b70100d90 0x7f7b701011e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:14.579 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 -- 172.21.15.164:0/3861378178 >> 172.21.15.164:0/3861378178 conn(0x7f7b700fb1a0 msgr2=0x7f7b700fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:11:14.580 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 -- 172.21.15.164:0/3861378178 shutdown_connections 2023-07-14T16:11:14.580 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.601+0000 7f7b7648f700 1 -- 172.21.15.164:0/3861378178 wait complete. 2023-07-14T16:11:14.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.602+0000 7f7b7648f700 1 Processor -- start 2023-07-14T16:11:14.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.602+0000 7f7b7648f700 1 -- start start 2023-07-14T16:11:14.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.602+0000 7f7b7648f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b700ffb90 0x7f7b70197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:11:14.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.602+0000 7f7b7648f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b70100d90 0x7f7b70197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:11:14.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.602+0000 7f7b7648f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b70101720 0x7f7b7019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:11:14.582 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b7648f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7019f590 con 0x7f7b70100d90 2023-07-14T16:11:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b7648f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7019f6d0 con 0x7f7b70101720 2023-07-14T16:11:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b7648f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7019f860 con 0x7f7b700ffb90 2023-07-14T16:11:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b700ffb90 0x7f7b70197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:11:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b700ffb90 0x7f7b70197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37352/0 (socket says 172.21.15.164:37352) 2023-07-14T16:11:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 -- 172.21.15.164:0/4033975663 learned_addr learned my addr 172.21.15.164:0/4033975663 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:11:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6f7fe700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b70100d90 0x7f7b70197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:11:14.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b74a2c700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b70101720 0x7f7b7019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:11:14.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 -- 172.21.15.164:0/4033975663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b70101720 msgr2=0x7f7b7019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:14.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b70101720 0x7f7b7019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:14.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 -- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b70100d90 msgr2=0x7f7b70197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:14.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b70100d90 0x7f7b70197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:14.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 -- 172.21.15.164:0/4033975663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b640095f0 con 0x7f7b700ffb90 2023-07-14T16:11:14.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.603+0000 7f7b6ffff700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b700ffb90 0x7f7b70197a00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000d9b0 tx=0x7f7b6000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:11:14.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.604+0000 7f7b6d7fa700 1 -- 172.21.15.164:0/4033975663 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60004020 con 0x7f7b700ffb90 2023-07-14T16:11:14.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.604+0000 7f7b6d7fa700 1 -- 172.21.15.164:0/4033975663 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b60009730 con 0x7f7b700ffb90 2023-07-14T16:11:14.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.604+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7019fb40 con 0x7f7b700ffb90 2023-07-14T16:11:14.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.604+0000 7f7b6d7fa700 1 -- 172.21.15.164:0/4033975663 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b600038e0 con 0x7f7b700ffb90 2023-07-14T16:11:14.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.604+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b701a0040 con 0x7f7b700ffb90 2023-07-14T16:11:14.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.605+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7010b250 con 0x7f7b700ffb90 2023-07-14T16:11:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.607+0000 7f7b6d7fa700 1 -- 172.21.15.164:0/4033975663 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7b600098a0 con 0x7f7b700ffb90 2023-07-14T16:11:14.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.608+0000 7f7b6d7fa700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b580a1150 0x7f7b580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:11:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.609+0000 7f7b6d7fa700 1 -- 172.21.15.164:0/4033975663 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f7b600cba70 con 0x7f7b700ffb90 2023-07-14T16:11:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.609+0000 7f7b6f7fe700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b580a1150 0x7f7b580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:11:14.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.609+0000 7f7b6f7fe700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b580a1150 0x7f7b580a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b5c0 tx=0x7f7b64009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:11:14.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.612+0000 7f7b6d7fa700 1 -- 172.21.15.164:0/4033975663 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7b60095c80 con 0x7f7b700ffb90 2023-07-14T16:11:14.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:14 smithi164 ceph-mon[97427]: pgmap v191: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T16:11:14.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:14 smithi164 ceph-mon[97427]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-14T16:11:14.795 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:14.819+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b70107780 con 0x7f7b580a1150 2023-07-14T16:11:14.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:14 smithi062 ceph-mon[95543]: pgmap v191: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T16:11:14.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:14 smithi062 ceph-mon[95543]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-14T16:11:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:14 smithi081 ceph-mon[127795]: pgmap v191: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T16:11:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:14 smithi081 ceph-mon[127795]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-07-14T16:11:15.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:15 smithi164 ceph-mon[97427]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T16:11:15.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:11:15.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:11:15.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:15.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:15 smithi062 ceph-mon[95543]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T16:11:15.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:11:15.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:11:15.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:15.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:15 smithi081 ceph-mon[127795]: osdmap e55: 8 total, 8 up, 8 in 2023-07-14T16:11:15.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:11:15.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:11:15.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:16.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:16 smithi164 ceph-mon[97427]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:11:16.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:16 smithi164 ceph-mon[97427]: pgmap v193: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T16:11:16.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:16 smithi164 ceph-mon[97427]: osdmap e56: 8 total, 8 up, 8 in 2023-07-14T16:11:16.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:16 smithi062 ceph-mon[95543]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:11:16.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:16 smithi062 ceph-mon[95543]: pgmap v193: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T16:11:16.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:16 smithi062 ceph-mon[95543]: osdmap e56: 8 total, 8 up, 8 in 2023-07-14T16:11:16.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:16 smithi081 ceph-mon[127795]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:11:16.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:16 smithi081 ceph-mon[127795]: pgmap v193: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 2/6 objects degraded (33.333%) 2023-07-14T16:11:16.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:16 smithi081 ceph-mon[127795]: osdmap e56: 8 total, 8 up, 8 in 2023-07-14T16:11:17.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:17 smithi062 ceph-mon[95543]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-14T16:11:17.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:17 smithi062 ceph-mon[95543]: Cluster is now healthy 2023-07-14T16:11:17.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:17 smithi081 ceph-mon[127795]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-14T16:11:17.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:17 smithi081 ceph-mon[127795]: Cluster is now healthy 2023-07-14T16:11:18.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:17 smithi164 ceph-mon[97427]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-07-14T16:11:18.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:17 smithi164 ceph-mon[97427]: Cluster is now healthy 2023-07-14T16:11:18.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:18 smithi062 ceph-mon[95543]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:18.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:18 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/2870738199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]: dispatch 2023-07-14T16:11:18.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:18 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]: dispatch 2023-07-14T16:11:18.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:18 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]': finished 2023-07-14T16:11:18.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:18 smithi062 ceph-mon[95543]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T16:11:18.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:18.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:18 smithi081 ceph-mon[127795]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:18.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:18 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/2870738199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]: dispatch 2023-07-14T16:11:18.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:18 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]: dispatch 2023-07-14T16:11:18.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:18 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]': finished 2023-07-14T16:11:18.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:18 smithi081 ceph-mon[127795]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T16:11:18.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:19.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:18 smithi164 ceph-mon[97427]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:18 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/2870738199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]: dispatch 2023-07-14T16:11:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:18 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]: dispatch 2023-07-14T16:11:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:18 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30aea523-6f39-4ad1-8852-ec62c90e4c7a"}]': finished 2023-07-14T16:11:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:18 smithi164 ceph-mon[97427]: osdmap e57: 9 total, 8 up, 9 in 2023-07-14T16:11:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:19.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:19 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/3675307065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:11:19.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:19 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/3675307065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:11:20.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:19 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/3675307065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:11:20.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:20 smithi062 ceph-mon[95543]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:20.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:20 smithi081 ceph-mon[127795]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:21.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:20 smithi164 ceph-mon[97427]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:22.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:22 smithi164 ceph-mon[97427]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:22.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:22 smithi062 ceph-mon[95543]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:22.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:22 smithi081 ceph-mon[127795]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:24 smithi062 ceph-mon[95543]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:24.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:24 smithi081 ceph-mon[127795]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:25.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:24 smithi164 ceph-mon[97427]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:26.859 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:26 smithi164 ceph-mon[97427]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:26.859 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T16:11:26.859 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:26.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:26 smithi062 ceph-mon[95543]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:26.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T16:11:26.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:26.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:26 smithi081 ceph-mon[127795]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T16:11:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:27.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:27 smithi164 ceph-mon[97427]: Deploying daemon osd.8 on smithi164 2023-07-14T16:11:27.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:27 smithi062 ceph-mon[95543]: Deploying daemon osd.8 on smithi164 2023-07-14T16:11:27.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:27 smithi081 ceph-mon[127795]: Deploying daemon osd.8 on smithi164 2023-07-14T16:11:28.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:28 smithi062 ceph-mon[95543]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:28.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:28 smithi081 ceph-mon[127795]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:29.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:28 smithi164 ceph-mon[97427]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:30.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:30 smithi062 ceph-mon[95543]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:30.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:30 smithi081 ceph-mon[127795]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:31.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:30 smithi164 ceph-mon[97427]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:32.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:32 smithi164 ceph-mon[97427]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:32.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:32.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:32.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:32.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:32 smithi062 ceph-mon[95543]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:32.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:32.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:32.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:32.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:32 smithi081 ceph-mon[127795]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:32.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:32.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:32.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:34.751 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:34 smithi164 ceph-mon[97427]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:34.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:34 smithi062 ceph-mon[95543]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:34.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:34 smithi081 ceph-mon[127795]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:36.127 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 8 on host 'smithi164' 2023-07-14T16:11:36.127 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.151+0000 7f7b6d7fa700 1 -- 172.21.15.164:0/4033975663 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7b70107780 con 0x7f7b580a1150 2023-07-14T16:11:36.129 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.153+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b580a1150 msgr2=0x7f7b580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:36.129 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.153+0000 7f7b7648f700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b580a1150 0x7f7b580a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b5c0 tx=0x7f7b64009f00 comp rx=0 tx=0).stop 2023-07-14T16:11:36.130 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b700ffb90 msgr2=0x7f7b70197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:36.130 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b700ffb90 0x7f7b70197a00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000d9b0 tx=0x7f7b6000dcc0 comp rx=0 tx=0).stop 2023-07-14T16:11:36.130 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 shutdown_connections 2023-07-14T16:11:36.130 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b580a1150 0x7f7b580a3600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:36.130 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b70101720 0x7f7b7019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:36.130 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b700ffb90 0x7f7b70197a00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:36.130 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 --2- 172.21.15.164:0/4033975663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b70100d90 0x7f7b70197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:36.131 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 >> 172.21.15.164:0/4033975663 conn(0x7f7b700fb1a0 msgr2=0x7f7b700fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:11:36.131 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 shutdown_connections 2023-07-14T16:11:36.131 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:11:36.154+0000 7f7b7648f700 1 -- 172.21.15.164:0/4033975663 wait complete. 2023-07-14T16:11:36.674 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:36.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.675 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:36.676 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:36.676 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.676 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.676 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.677 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:36.677 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:36 smithi164 ceph-mon[97427]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T16:11:36.679 DEBUG:teuthology.orchestra.run.smithi164:osd.8> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.8.service 2023-07-14T16:11:36.684 INFO:tasks.cephadm:Deploying osd.9 on smithi062 with /dev/vg_nvme/lv_1... 2023-07-14T16:11:36.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T16:11:36.757 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:36.757 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.757 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.757 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:36.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:36.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:36.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:36 smithi062 ceph-mon[95543]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T16:11:36.763 INFO:journalctl@ceph.osd.8.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:11:36.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:36.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:11:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:36 smithi081 ceph-mon[127795]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T16:11:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:11:38.305 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:38 smithi062 ceph-mon[95543]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T16:11:38.305 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:38 smithi062 ceph-mon[95543]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T16:11:38.305 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:38.305 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:38 smithi062 ceph-mon[95543]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:11:38.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:38 smithi062 ceph-mon[95543]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:38.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:38 smithi081 ceph-mon[127795]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T16:11:38.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:38 smithi081 ceph-mon[127795]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T16:11:38.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:38.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:38 smithi081 ceph-mon[127795]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:11:38.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:38 smithi081 ceph-mon[127795]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:38.501 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:38 smithi164 ceph-mon[97427]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T16:11:38.501 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:38 smithi164 ceph-mon[97427]: osdmap e58: 9 total, 8 up, 9 in 2023-07-14T16:11:38.502 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:38.502 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:38 smithi164 ceph-mon[97427]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:11:38.502 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:38 smithi164 ceph-mon[97427]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T16:11:38.502 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:11:38 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:11:38.138+0000 7f88cba5a700 -1 osd.8 0 waiting for initial osdmap 2023-07-14T16:11:38.502 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:11:38 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:11:38.151+0000 7f88c784f700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:11:39.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:39 smithi062 ceph-mon[95543]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:11:39.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:39 smithi062 ceph-mon[95543]: osdmap e59: 9 total, 8 up, 9 in 2023-07-14T16:11:39.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:39 smithi062 ceph-mon[95543]: osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] boot 2023-07-14T16:11:39.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:39 smithi062 ceph-mon[95543]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T16:11:39.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.392 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:39 smithi164 ceph-mon[97427]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:11:39.392 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:39 smithi164 ceph-mon[97427]: osdmap e59: 9 total, 8 up, 9 in 2023-07-14T16:11:39.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.393 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:39 smithi164 ceph-mon[97427]: osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] boot 2023-07-14T16:11:39.394 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:39 smithi164 ceph-mon[97427]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T16:11:39.394 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:39 smithi081 ceph-mon[127795]: from='osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:11:39.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:39 smithi081 ceph-mon[127795]: osdmap e59: 9 total, 8 up, 9 in 2023-07-14T16:11:39.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:39 smithi081 ceph-mon[127795]: osd.8 [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] boot 2023-07-14T16:11:39.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:39 smithi081 ceph-mon[127795]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T16:11:39.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T16:11:39.410 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T16:11:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T16:11:39.411 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2023-07-14T16:11:39.411 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2023-07-14T16:11:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0301718 s, 348 MB/s 2023-07-14T16:11:39.412 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2023-07-14T16:11:39.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2023-07-14T16:11:40.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:11:40.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:11:40.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:40.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T16:11:40.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:40.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:40.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:11:40.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:11:40.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:40.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T16:11:40.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:40.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:40.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:11:40.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:11:40.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:40.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: osdmap e61: 9 total, 9 up, 9 in 2023-07-14T16:11:40.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:11:40.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:40.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:11:40.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:11:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:11:41.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:41 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi164 2023-07-14T16:11:41.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:41 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi164 to 5755M 2023-07-14T16:11:41.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:41 smithi062 ceph-mon[95543]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T16:11:41.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:41 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi164 2023-07-14T16:11:41.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:41 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi164 to 5755M 2023-07-14T16:11:41.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:41 smithi164 ceph-mon[97427]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T16:11:41.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:41 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi164 2023-07-14T16:11:41.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:41 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi164 to 5755M 2023-07-14T16:11:41.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:41 smithi081 ceph-mon[127795]: osdmap e62: 9 total, 9 up, 9 in 2023-07-14T16:11:42.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.256+0000 7f2ac45d6700 1 -- 172.21.15.62:0/856078630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc101720 msgr2=0x7f2abc107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:42.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.256+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/856078630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc101720 0x7f2abc107fe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b0a0 tx=0x7f2ab400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:11:42.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.257+0000 7f2ac45d6700 1 -- 172.21.15.62:0/856078630 shutdown_connections 2023-07-14T16:11:42.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.257+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/856078630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc101720 0x7f2abc107fe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:42.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.257+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/856078630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2abc0ffb90 0x7f2abc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:42.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.257+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/856078630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2abc100d90 0x7f2abc1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:42.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.257+0000 7f2ac45d6700 1 -- 172.21.15.62:0/856078630 >> 172.21.15.62:0/856078630 conn(0x7f2abc0fb1a0 msgr2=0x7f2abc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:11:42.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.257+0000 7f2ac45d6700 1 -- 172.21.15.62:0/856078630 shutdown_connections 2023-07-14T16:11:42.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.257+0000 7f2ac45d6700 1 -- 172.21.15.62:0/856078630 wait complete. 2023-07-14T16:11:42.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac45d6700 1 Processor -- start 2023-07-14T16:11:42.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac45d6700 1 -- start start 2023-07-14T16:11:42.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac45d6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2abc0ffb90 0x7f2abc1958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:11:42.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac45d6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc100d90 0x7f2abc195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:11:42.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac45d6700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2abc101720 0x7f2abc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:11:42.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac45d6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d430 con 0x7f2abc100d90 2023-07-14T16:11:42.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac45d6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d570 con 0x7f2abc0ffb90 2023-07-14T16:11:42.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac45d6700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d700 con 0x7f2abc101720 2023-07-14T16:11:42.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac1b71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc100d90 0x7f2abc195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:11:42.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac1b71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc100d90 0x7f2abc195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41380/0 (socket says 172.21.15.62:41380) 2023-07-14T16:11:42.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.258+0000 7f2ac1b71700 1 -- 172.21.15.62:0/1147233009 learned_addr learned my addr 172.21.15.62:0/1147233009 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:11:42.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac2b73700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2abc101720 0x7f2abc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:11:42.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac2372700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2abc0ffb90 0x7f2abc1958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:11:42.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac1b71700 1 -- 172.21.15.62:0/1147233009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2abc101720 msgr2=0x7f2abc19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:42.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac1b71700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2abc101720 0x7f2abc19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:42.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac1b71700 1 -- 172.21.15.62:0/1147233009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2abc0ffb90 msgr2=0x7f2abc1958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:11:42.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac1b71700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2abc0ffb90 0x7f2abc1958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:11:42.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac1b71700 1 -- 172.21.15.62:0/1147233009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4009d00 con 0x7f2abc100d90 2023-07-14T16:11:42.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac2372700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2abc0ffb90 0x7f2abc1958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:11:42.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac1b71700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc100d90 0x7f2abc195de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00e9d0 tx=0x7f2aac00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:11:42.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1147233009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac009790 con 0x7f2abc100d90 2023-07-14T16:11:42.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc19d9e0 con 0x7f2abc100d90 2023-07-14T16:11:42.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1147233009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2aac00cbd0 con 0x7f2abc100d90 2023-07-14T16:11:42.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.259+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1147233009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac018680 con 0x7f2abc100d90 2023-07-14T16:11:42.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.260+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc19dee0 con 0x7f2abc100d90 2023-07-14T16:11:42.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.261+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc107190 con 0x7f2abc100d90 2023-07-14T16:11:42.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.263+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1147233009 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2aac00cd40 con 0x7f2abc100d90 2023-07-14T16:11:42.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.264+0000 7f2ab37fe700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2aa80a1150 0x7f2aa80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:11:42.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.264+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1147233009 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f2aac014070 con 0x7f2abc100d90 2023-07-14T16:11:42.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.264+0000 7f2ac2372700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2aa80a1150 0x7f2aa80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:11:42.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.265+0000 7f2ac2372700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2aa80a1150 0x7f2aa80a3600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8005950 tx=0x7f2ab80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:11:42.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.268+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1147233009 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2aac051790 con 0x7f2abc100d90 2023-07-14T16:11:42.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:11:42.486+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2abc060950 con 0x7f2aa80a1150 2023-07-14T16:11:42.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:42 smithi164 ceph-mon[97427]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:11:42.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:42 smithi164 ceph-mon[97427]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T16:11:42.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:11:42.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:11:42.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:42.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:42 smithi062 ceph-mon[95543]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:11:42.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:42 smithi062 ceph-mon[95543]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T16:11:42.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:11:42.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:11:42.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:42.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:42 smithi081 ceph-mon[127795]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:11:42.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:42 smithi081 ceph-mon[127795]: osdmap e63: 9 total, 9 up, 9 in 2023-07-14T16:11:42.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:11:42.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:11:42.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:43.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:43 smithi164 ceph-mon[97427]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:11:43.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:43 smithi164 ceph-mon[97427]: osdmap e64: 9 total, 9 up, 9 in 2023-07-14T16:11:43.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:43 smithi062 ceph-mon[95543]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:11:43.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:43 smithi062 ceph-mon[95543]: osdmap e64: 9 total, 9 up, 9 in 2023-07-14T16:11:43.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:43 smithi081 ceph-mon[127795]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:11:43.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:43 smithi081 ceph-mon[127795]: osdmap e64: 9 total, 9 up, 9 in 2023-07-14T16:11:44.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:44 smithi164 ceph-mon[97427]: pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:11:44.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:44 smithi062 ceph-mon[95543]: pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:11:44.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:44 smithi081 ceph-mon[127795]: pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:11:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:45 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2306653137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52d65846-0e8d-4cef-b984-1e4a5e97f5b3"}]: dispatch 2023-07-14T16:11:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:45 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2306653137' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52d65846-0e8d-4cef-b984-1e4a5e97f5b3"}]': finished 2023-07-14T16:11:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:45 smithi062 ceph-mon[95543]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T16:11:45.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:45 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:11:45.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:45 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2306653137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52d65846-0e8d-4cef-b984-1e4a5e97f5b3"}]: dispatch 2023-07-14T16:11:45.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:45 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2306653137' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52d65846-0e8d-4cef-b984-1e4a5e97f5b3"}]': finished 2023-07-14T16:11:45.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:45 smithi081 ceph-mon[127795]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T16:11:45.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:45 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:11:46.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:45 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2306653137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52d65846-0e8d-4cef-b984-1e4a5e97f5b3"}]: dispatch 2023-07-14T16:11:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:45 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2306653137' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52d65846-0e8d-4cef-b984-1e4a5e97f5b3"}]': finished 2023-07-14T16:11:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:45 smithi164 ceph-mon[97427]: osdmap e65: 10 total, 9 up, 10 in 2023-07-14T16:11:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:45 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:11:46.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:46 smithi062 ceph-mon[95543]: pgmap v217: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T16:11:46.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:46 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3194341144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:11:46.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:46 smithi081 ceph-mon[127795]: pgmap v217: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T16:11:46.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:46 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3194341144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:11:47.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:46 smithi164 ceph-mon[97427]: pgmap v217: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T16:11:47.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:46 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3194341144' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:11:48.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:48 smithi062 ceph-mon[95543]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:48.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:48 smithi081 ceph-mon[127795]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:48 smithi164 ceph-mon[97427]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:50.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:50 smithi062 ceph-mon[95543]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:50.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:50 smithi081 ceph-mon[127795]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:51.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:50 smithi164 ceph-mon[97427]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:52.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:52 smithi062 ceph-mon[95543]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:52.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:52 smithi081 ceph-mon[127795]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:53.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:52 smithi164 ceph-mon[97427]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:53.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T16:11:53.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:53.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T16:11:53.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:54.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T16:11:54.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:11:54.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:54 smithi062 ceph-mon[95543]: Deploying daemon osd.9 on smithi062 2023-07-14T16:11:54.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:54 smithi062 ceph-mon[95543]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:54.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:54 smithi081 ceph-mon[127795]: Deploying daemon osd.9 on smithi062 2023-07-14T16:11:54.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:54 smithi081 ceph-mon[127795]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:55.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:54 smithi164 ceph-mon[97427]: Deploying daemon osd.9 on smithi062 2023-07-14T16:11:55.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:54 smithi164 ceph-mon[97427]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:56.809 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:56 smithi062 ceph-mon[95543]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:56.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:56 smithi081 ceph-mon[127795]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:57.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:56 smithi164 ceph-mon[97427]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:58.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:11:58 smithi062 ceph-mon[95543]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:58.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:11:58 smithi081 ceph-mon[127795]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:11:59.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:11:58 smithi164 ceph-mon[97427]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:00.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:00.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:00.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:00.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:00 smithi062 ceph-mon[95543]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:00.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:00.652 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:00.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:00.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:00 smithi081 ceph-mon[127795]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:00.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:00.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:00.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:00.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:00 smithi164 ceph-mon[97427]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:02.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:02 smithi164 ceph-mon[97427]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:02.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:02.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:02.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:02.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:02.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:02.832 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:02 smithi062 ceph-mon[95543]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:02.832 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:02.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:02.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:02.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:02.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:02.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:02 smithi081 ceph-mon[127795]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:02.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:02.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:03.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.195+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1147233009 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2abc060950 con 0x7f2aa80a1150 2023-07-14T16:12:03.173 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 9 on host 'smithi062' 2023-07-14T16:12:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2aa80a1150 msgr2=0x7f2aa80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2aa80a1150 0x7f2aa80a3600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8005950 tx=0x7f2ab80058e0 comp rx=0 tx=0).stop 2023-07-14T16:12:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc100d90 msgr2=0x7f2abc195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc100d90 0x7f2abc195de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00e9d0 tx=0x7f2aac00ece0 comp rx=0 tx=0).stop 2023-07-14T16:12:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 shutdown_connections 2023-07-14T16:12:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2abc101720 0x7f2abc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2aa80a1150 0x7f2aa80a3600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2abc0ffb90 0x7f2abc1958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 --2- 172.21.15.62:0/1147233009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2abc100d90 0x7f2abc195de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 >> 172.21.15.62:0/1147233009 conn(0x7f2abc0fb1a0 msgr2=0x7f2abc1042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:12:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 shutdown_connections 2023-07-14T16:12:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:12:03.198+0000 7f2ac45d6700 1 -- 172.21.15.62:0/1147233009 wait complete. 2023-07-14T16:12:03.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:03.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:03.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:03.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:03 smithi062 ceph-mon[95543]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T16:12:03.662 DEBUG:teuthology.orchestra.run.smithi062:osd.9> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.9.service 2023-07-14T16:12:03.668 INFO:tasks.cephadm:Deploying osd.10 on smithi081 with /dev/vg_nvme/lv_1... 2023-07-14T16:12:03.668 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T16:12:03.738 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:03.738 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:03.738 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:03.739 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:03 smithi081 ceph-mon[127795]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T16:12:03.750 INFO:journalctl@ceph.osd.9.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:12:03.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:03.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:03.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:03.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:03 smithi164 ceph-mon[97427]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T16:12:04.397 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:12:04.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:04 smithi062 ceph-mon[95543]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:04.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:04 smithi062 ceph-mon[95543]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T16:12:04.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:04 smithi062 ceph-mon[95543]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T16:12:04.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:04 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:04.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:04 smithi062 ceph-mon[95543]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:12:04.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:04 smithi164 ceph-mon[97427]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:04.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:04 smithi164 ceph-mon[97427]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T16:12:04.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:04 smithi164 ceph-mon[97427]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T16:12:04.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:04 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:04.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:04 smithi164 ceph-mon[97427]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:12:04.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:04 smithi081 ceph-mon[127795]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:04.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:04 smithi081 ceph-mon[127795]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T16:12:04.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:04 smithi081 ceph-mon[127795]: osdmap e66: 10 total, 9 up, 10 in 2023-07-14T16:12:04.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:04 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:04.843 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:04 smithi081 ceph-mon[127795]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2023-07-14T16:12:05.624 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:12:05 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:12:05.164+0000 7f61989a1700 -1 osd.9 0 waiting for initial osdmap 2023-07-14T16:12:05.624 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:12:05 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:12:05.183+0000 7f6192792700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:12:06.329 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T16:12:06.329 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T16:12:06.329 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2023-07-14T16:12:06.330 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2023-07-14T16:12:06.330 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0313976 s, 334 MB/s 2023-07-14T16:12:06.330 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2023-07-14T16:12:06.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:06 smithi062 ceph-mon[95543]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:12:06.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:06 smithi062 ceph-mon[95543]: osdmap e67: 10 total, 9 up, 10 in 2023-07-14T16:12:06.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:06 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:06.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:06 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:06.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:06 smithi062 ceph-mon[95543]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:06.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:06 smithi081 ceph-mon[127795]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:12:06.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:06 smithi081 ceph-mon[127795]: osdmap e67: 10 total, 9 up, 10 in 2023-07-14T16:12:06.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:06 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:06.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:06 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:06.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:06 smithi081 ceph-mon[127795]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:06.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:06 smithi164 ceph-mon[97427]: from='osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2023-07-14T16:12:06.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:06 smithi164 ceph-mon[97427]: osdmap e67: 10 total, 9 up, 10 in 2023-07-14T16:12:06.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:06 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:06.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:06 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:06.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:06 smithi164 ceph-mon[97427]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T16:12:06.767 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi081:vg_nvme/lv_1 2023-07-14T16:12:07.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:12:07.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:12:07.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] boot 2023-07-14T16:12:07.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T16:12:07.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:07.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi062 2023-07-14T16:12:07.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi062 to 4316M 2023-07-14T16:12:07.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:07.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:07.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.486 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:12:07.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:12:07.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:12:07.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] boot 2023-07-14T16:12:07.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T16:12:07.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:07.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi062 2023-07-14T16:12:07.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi062 to 4316M 2023-07-14T16:12:07.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:07.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:07.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:12:07.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:12:07.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: osd.9 [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] boot 2023-07-14T16:12:07.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: osdmap e68: 10 total, 10 up, 10 in 2023-07-14T16:12:07.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T16:12:07.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi062 2023-07-14T16:12:07.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:07.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi062 to 4316M 2023-07-14T16:12:07.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:07.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:07.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:07.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:08.322 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:08 smithi081 ceph-mon[127795]: osdmap e69: 10 total, 10 up, 10 in 2023-07-14T16:12:08.322 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:08 smithi081 ceph-mon[127795]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:08.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:08 smithi164 ceph-mon[97427]: osdmap e69: 10 total, 10 up, 10 in 2023-07-14T16:12:08.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:08 smithi164 ceph-mon[97427]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:08 smithi062 ceph-mon[95543]: osdmap e69: 10 total, 10 up, 10 in 2023-07-14T16:12:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:08 smithi062 ceph-mon[95543]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:09.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 -- 172.21.15.81:0/2999760745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4780100d90 msgr2=0x7f47801011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:09.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 --2- 172.21.15.81:0/2999760745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4780100d90 0x7f47801011e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f476c009990 tx=0x7f476c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:12:09.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 -- 172.21.15.81:0/2999760745 shutdown_connections 2023-07-14T16:12:09.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 --2- 172.21.15.81:0/2999760745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780101720 0x7f4780107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:09.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 --2- 172.21.15.81:0/2999760745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47800ffb90 0x7f47800fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:09.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 --2- 172.21.15.81:0/2999760745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4780100d90 0x7f47801011e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:09.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 -- 172.21.15.81:0/2999760745 >> 172.21.15.81:0/2999760745 conn(0x7f47800fb1a0 msgr2=0x7f47800fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:12:09.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 -- 172.21.15.81:0/2999760745 shutdown_connections 2023-07-14T16:12:09.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.046+0000 7f4786b12700 1 -- 172.21.15.81:0/2999760745 wait complete. 2023-07-14T16:12:09.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.047+0000 7f4786b12700 1 Processor -- start 2023-07-14T16:12:09.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.047+0000 7f4786b12700 1 -- start start 2023-07-14T16:12:09.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.047+0000 7f4786b12700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47800ffb90 0x7f4780197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:12:09.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f4786b12700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4780100d90 0x7f4780197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:12:09.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f4786b12700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780101720 0x7f478019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:12:09.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f4786b12700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f5c0 con 0x7f4780100d90 2023-07-14T16:12:09.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f4786b12700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f700 con 0x7f47800ffb90 2023-07-14T16:12:09.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f4786b12700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f890 con 0x7f4780101720 2023-07-14T16:12:09.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47848ae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47800ffb90 0x7f4780197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:12:09.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47848ae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47800ffb90 0x7f4780197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34162/0 (socket says 172.21.15.81:34162) 2023-07-14T16:12:09.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47848ae700 1 -- 172.21.15.81:0/4223932713 learned_addr learned my addr 172.21.15.81:0/4223932713 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-07-14T16:12:09.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47850af700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780101720 0x7f478019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:12:09.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f477ffff700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4780100d90 0x7f4780197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:12:09.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47848ae700 1 -- 172.21.15.81:0/4223932713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780101720 msgr2=0x7f478019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:09.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47848ae700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780101720 0x7f478019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:09.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47848ae700 1 -- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4780100d90 msgr2=0x7f4780197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:09.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47848ae700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4780100d90 0x7f4780197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:09.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.048+0000 7f47848ae700 1 -- 172.21.15.81:0/4223932713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0095f0 con 0x7f47800ffb90 2023-07-14T16:12:09.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.049+0000 7f47848ae700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47800ffb90 0x7f4780197a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f477400e970 tx=0x7f477400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:12:09.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.049+0000 7f477dffb700 1 -- 172.21.15.81:0/4223932713 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774009790 con 0x7f47800ffb90 2023-07-14T16:12:09.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.049+0000 7f477dffb700 1 -- 172.21.15.81:0/4223932713 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f477400eec0 con 0x7f47800ffb90 2023-07-14T16:12:09.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.049+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478019fb70 con 0x7f47800ffb90 2023-07-14T16:12:09.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.049+0000 7f477dffb700 1 -- 172.21.15.81:0/4223932713 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774003c40 con 0x7f47800ffb90 2023-07-14T16:12:09.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.049+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47801a0070 con 0x7f47800ffb90 2023-07-14T16:12:09.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.050+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478010b250 con 0x7f47800ffb90 2023-07-14T16:12:09.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.052+0000 7f477dffb700 1 -- 172.21.15.81:0/4223932713 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4774003da0 con 0x7f47800ffb90 2023-07-14T16:12:09.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.053+0000 7f477dffb700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f47700a1150 0x7f47700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:12:09.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.053+0000 7f477dffb700 1 -- 172.21.15.81:0/4223932713 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f4774014070 con 0x7f47800ffb90 2023-07-14T16:12:09.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.053+0000 7f477ffff700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f47700a1150 0x7f47700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:12:09.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.054+0000 7f477ffff700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f47700a1150 0x7f47700a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f476c006010 tx=0x7f476c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:12:09.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.057+0000 7f477dffb700 1 -- 172.21.15.81:0/4223932713 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4774096260 con 0x7f47800ffb90 2023-07-14T16:12:09.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:09.261+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4780107790 con 0x7f47700a1150 2023-07-14T16:12:09.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:09 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:12:09.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:09 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:12:09.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:09 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:09.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:09 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:12:09.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:09 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:12:09.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:09 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:09.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:09 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:12:09.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:09 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:12:09.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:09 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:10.599 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:10 smithi081 ceph-mon[127795]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:12:10.600 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:10 smithi081 ceph-mon[127795]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:10.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:10 smithi164 ceph-mon[97427]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:12:10.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:10 smithi164 ceph-mon[97427]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:10.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:10 smithi062 ceph-mon[95543]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:12:10.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:10 smithi062 ceph-mon[95543]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:12.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:12 smithi164 ceph-mon[97427]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:12.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:12 smithi164 ceph-mon[97427]: from='client.? 172.21.15.81:0/3665910693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]: dispatch 2023-07-14T16:12:12.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:12 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]: dispatch 2023-07-14T16:12:12.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:12 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]': finished 2023-07-14T16:12:12.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:12 smithi164 ceph-mon[97427]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T16:12:12.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:12.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:12 smithi062 ceph-mon[95543]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:12.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:12 smithi062 ceph-mon[95543]: from='client.? 172.21.15.81:0/3665910693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]: dispatch 2023-07-14T16:12:12.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:12 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]: dispatch 2023-07-14T16:12:12.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:12 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]': finished 2023-07-14T16:12:12.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:12 smithi062 ceph-mon[95543]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T16:12:12.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:12.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:12 smithi081 ceph-mon[127795]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:12 smithi081 ceph-mon[127795]: from='client.? 172.21.15.81:0/3665910693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]: dispatch 2023-07-14T16:12:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:12 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]: dispatch 2023-07-14T16:12:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:12 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1"}]': finished 2023-07-14T16:12:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:12 smithi081 ceph-mon[127795]: osdmap e70: 11 total, 10 up, 11 in 2023-07-14T16:12:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:13.774 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:13 smithi164 ceph-mon[97427]: from='client.? 172.21.15.81:0/649081038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:12:13.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:13 smithi062 ceph-mon[95543]: from='client.? 172.21.15.81:0/649081038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:12:13.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:13 smithi081 ceph-mon[127795]: from='client.? 172.21.15.81:0/649081038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:12:14.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:14 smithi164 ceph-mon[97427]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:14.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:14 smithi062 ceph-mon[95543]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:14.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:14 smithi081 ceph-mon[127795]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:16.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:16 smithi062 ceph-mon[95543]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:16.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:16 smithi081 ceph-mon[127795]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:17.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:16 smithi164 ceph-mon[97427]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:18.787 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:18 smithi081 ceph-mon[127795]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:18.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:18 smithi062 ceph-mon[95543]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:18 smithi164 ceph-mon[97427]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:20.804 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:20 smithi081 ceph-mon[127795]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:20.805 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T16:12:20.805 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:20.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:20 smithi062 ceph-mon[95543]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:20.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T16:12:20.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:20 smithi164 ceph-mon[97427]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T16:12:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:21.842 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:21 smithi081 ceph-mon[127795]: Deploying daemon osd.10 on smithi081 2023-07-14T16:12:21.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:21 smithi062 ceph-mon[95543]: Deploying daemon osd.10 on smithi081 2023-07-14T16:12:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:21 smithi164 ceph-mon[97427]: Deploying daemon osd.10 on smithi081 2023-07-14T16:12:22.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:22 smithi062 ceph-mon[95543]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:22.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:22 smithi081 ceph-mon[127795]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:22 smithi164 ceph-mon[97427]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:24 smithi062 ceph-mon[95543]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:24.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:24 smithi081 ceph-mon[127795]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:24 smithi164 ceph-mon[97427]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:26.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:26 smithi062 ceph-mon[95543]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:26.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:26.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:26.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:26.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:26 smithi081 ceph-mon[127795]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:27.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:26 smithi164 ceph-mon[97427]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:27.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:28.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:28 smithi062 ceph-mon[95543]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:28.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:28 smithi081 ceph-mon[127795]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:28 smithi164 ceph-mon[97427]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:30.264 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 10 on host 'smithi081' 2023-07-14T16:12:30.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.286+0000 7f477dffb700 1 -- 172.21.15.81:0/4223932713 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4780107790 con 0x7f47700a1150 2023-07-14T16:12:30.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f47700a1150 msgr2=0x7f47700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:30.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f47700a1150 0x7f47700a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f476c006010 tx=0x7f476c00b260 comp rx=0 tx=0).stop 2023-07-14T16:12:30.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47800ffb90 msgr2=0x7f4780197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:30.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47800ffb90 0x7f4780197a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f477400e970 tx=0x7f477400ec80 comp rx=0 tx=0).stop 2023-07-14T16:12:30.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 shutdown_connections 2023-07-14T16:12:30.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4780101720 0x7f478019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:30.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f47700a1150 0x7f47700a3600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:30.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47800ffb90 0x7f4780197a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:30.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 --2- 172.21.15.81:0/4223932713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4780100d90 0x7f4780197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:30.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 >> 172.21.15.81:0/4223932713 conn(0x7f47800fb1a0 msgr2=0x7f47800fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:12:30.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 shutdown_connections 2023-07-14T16:12:30.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-07-14T16:12:30.289+0000 7f4786b12700 1 -- 172.21.15.81:0/4223932713 wait complete. 2023-07-14T16:12:30.783 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:30.783 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.784 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.784 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:30.784 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:30.784 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.784 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.784 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.784 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:30.785 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T16:12:30.785 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:30 smithi081 ceph-mon[127795]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T16:12:30.822 DEBUG:teuthology.orchestra.run.smithi081:osd.10> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.10.service 2023-07-14T16:12:30.827 INFO:tasks.cephadm:Deploying osd.11 on smithi164 with /dev/vg_nvme/lv_1... 2023-07-14T16:12:30.827 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T16:12:30.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:30.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:30.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T16:12:30.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:30 smithi062 ceph-mon[95543]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T16:12:30.896 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:30.897 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.897 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.897 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:30.897 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:30.897 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.897 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.898 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:30.898 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:30.898 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T16:12:30.898 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:30 smithi164 ceph-mon[97427]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T16:12:30.899 INFO:journalctl@ceph.osd.10.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:12:31.561 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:12:32.517 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:32 smithi164 ceph-mon[97427]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T16:12:32.517 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:32 smithi164 ceph-mon[97427]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T16:12:32.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:32.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:32 smithi164 ceph-mon[97427]: from='osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:12:32.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:32 smithi164 ceph-mon[97427]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:12:32.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:32 smithi164 ceph-mon[97427]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:32.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:32 smithi062 ceph-mon[95543]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T16:12:32.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:32 smithi062 ceph-mon[95543]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T16:12:32.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:32.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:32 smithi062 ceph-mon[95543]: from='osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:12:32.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:32 smithi062 ceph-mon[95543]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:12:32.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:32 smithi062 ceph-mon[95543]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:32.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:32 smithi081 ceph-mon[127795]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T16:12:32.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:32 smithi081 ceph-mon[127795]: osdmap e71: 11 total, 10 up, 11 in 2023-07-14T16:12:32.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:32.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:32 smithi081 ceph-mon[127795]: from='osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:12:32.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:32 smithi081 ceph-mon[127795]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2023-07-14T16:12:32.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:32 smithi081 ceph-mon[127795]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T16:12:32.655 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:12:32 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:12:32.291+0000 7f32b3370700 -1 osd.10 0 waiting for initial osdmap 2023-07-14T16:12:32.655 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:12:32 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:12:32.306+0000 7f32af165700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:12:33.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:33 smithi081 ceph-mon[127795]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:12:33.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:33 smithi081 ceph-mon[127795]: osdmap e72: 11 total, 10 up, 11 in 2023-07-14T16:12:33.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:33.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:33.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:33 smithi164 ceph-mon[97427]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:12:33.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:33 smithi164 ceph-mon[97427]: osdmap e72: 11 total, 10 up, 11 in 2023-07-14T16:12:33.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:33.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:33.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:33 smithi062 ceph-mon[95543]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2023-07-14T16:12:33.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:33 smithi062 ceph-mon[95543]: osdmap e72: 11 total, 10 up, 11 in 2023-07-14T16:12:33.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:33.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:33.680 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T16:12:33.681 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T16:12:33.681 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2023-07-14T16:12:33.681 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2023-07-14T16:12:33.681 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0385666 s, 272 MB/s 2023-07-14T16:12:33.681 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2023-07-14T16:12:34.237 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2023-07-14T16:12:34.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:12:34.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:12:34.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] boot 2023-07-14T16:12:34.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T16:12:34.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:34.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:34.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T16:12:34.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi081 2023-07-14T16:12:34.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.528 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.528 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.528 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi081 to 4316M 2023-07-14T16:12:34.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:34.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:34.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:12:34.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:12:34.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] boot 2023-07-14T16:12:34.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T16:12:34.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:34.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:34.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T16:12:34.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi081 2023-07-14T16:12:34.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi081 to 4316M 2023-07-14T16:12:34.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:34.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:34.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:12:34.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:12:34.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: osd.10 [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] boot 2023-07-14T16:12:34.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: osdmap e73: 11 total, 11 up, 11 in 2023-07-14T16:12:34.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T16:12:34.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:34.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: osdmap e74: 11 total, 11 up, 11 in 2023-07-14T16:12:34.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi081 2023-07-14T16:12:34.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:12:34.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi081 to 4316M 2023-07-14T16:12:34.656 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.656 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:34.656 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:34.656 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:34.962 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:12:35.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:35 smithi164 ceph-mon[97427]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T16:12:35.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:35 smithi062 ceph-mon[95543]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T16:12:35.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:35 smithi081 ceph-mon[127795]: osdmap e75: 11 total, 11 up, 11 in 2023-07-14T16:12:36.583 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 -- 172.21.15.164:0/244446499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258101720 msgr2=0x7fb258107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 --2- 172.21.15.164:0/244446499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258101720 0x7fb258107fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb25000b0a0 tx=0x7fb25000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:12:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 -- 172.21.15.164:0/244446499 shutdown_connections 2023-07-14T16:12:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 --2- 172.21.15.164:0/244446499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258101720 0x7fb258107fe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 --2- 172.21.15.164:0/244446499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2580ffb90 0x7fb2580fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:36.584 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 --2- 172.21.15.164:0/244446499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb258100d90 0x7fb2581011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:36.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 -- 172.21.15.164:0/244446499 >> 172.21.15.164:0/244446499 conn(0x7fb2580fb1a0 msgr2=0x7fb2580fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:12:36.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 -- 172.21.15.164:0/244446499 shutdown_connections 2023-07-14T16:12:36.585 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.607+0000 7fb25e53f700 1 -- 172.21.15.164:0/244446499 wait complete. 2023-07-14T16:12:36.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.608+0000 7fb25e53f700 1 Processor -- start 2023-07-14T16:12:36.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.608+0000 7fb25e53f700 1 -- start start 2023-07-14T16:12:36.586 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.608+0000 7fb25e53f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2580ffb90 0x7fb258197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:12:36.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.608+0000 7fb25e53f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258100d90 0x7fb2581980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:12:36.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.608+0000 7fb25e53f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb258101720 0x7fb25819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:12:36.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.608+0000 7fb25e53f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25819f740 con 0x7fb258101720 2023-07-14T16:12:36.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.608+0000 7fb25e53f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25819f880 con 0x7fb2580ffb90 2023-07-14T16:12:36.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.608+0000 7fb25e53f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25819fa10 con 0x7fb258100d90 2023-07-14T16:12:36.587 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258100d90 0x7fb2581980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:12:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258100d90 0x7fb2581980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:38240/0 (socket says 172.21.15.164:38240) 2023-07-14T16:12:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 -- 172.21.15.164:0/1891369486 learned_addr learned my addr 172.21.15.164:0/1891369486 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:12:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb25cadc700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb258101720 0x7fb25819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:12:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb257fff700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2580ffb90 0x7fb258197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:12:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 -- 172.21.15.164:0/1891369486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2580ffb90 msgr2=0x7fb258197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2580ffb90 0x7fb258197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 -- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb258101720 msgr2=0x7fb25819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:36.588 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb258101720 0x7fb25819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 -- 172.21.15.164:0/1891369486 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb250009d00 con 0x7fb258100d90 2023-07-14T16:12:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb257fff700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2580ffb90 0x7fb258197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:12:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.609+0000 7fb2577fe700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258100d90 0x7fb2581980f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d980 tx=0x7fb24c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:12:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.610+0000 7fb2557fa700 1 -- 172.21.15.164:0/1891369486 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c009b40 con 0x7fb258100d90 2023-07-14T16:12:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.610+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25819fcf0 con 0x7fb258100d90 2023-07-14T16:12:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.610+0000 7fb2557fa700 1 -- 172.21.15.164:0/1891369486 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb24c00b6e0 con 0x7fb258100d90 2023-07-14T16:12:36.589 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.610+0000 7fb2557fa700 1 -- 172.21.15.164:0/1891369486 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00f3c0 con 0x7fb258100d90 2023-07-14T16:12:36.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.610+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2581a0260 con 0x7fb258100d90 2023-07-14T16:12:36.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.611+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25810b2f0 con 0x7fb258100d90 2023-07-14T16:12:36.590 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.613+0000 7fb2557fa700 1 -- 172.21.15.164:0/1891369486 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb24c010460 con 0x7fb258100d90 2023-07-14T16:12:36.591 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.614+0000 7fb2557fa700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2400a11d0 0x7fb2400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:12:36.591 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.614+0000 7fb2557fa700 1 -- 172.21.15.164:0/1891369486 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6694+0+0 (secure 0 0 0) 0x7fb24c0cc1c0 con 0x7fb258100d90 2023-07-14T16:12:36.591 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.615+0000 7fb257fff700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2400a11d0 0x7fb2400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:12:36.591 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.615+0000 7fb257fff700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2400a11d0 0x7fb2400a3680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb258100bf0 tx=0x7fb248008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:12:36.592 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.616+0000 7fb2557fa700 1 -- 172.21.15.164:0/1891369486 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb24c095ec0 con 0x7fb258100d90 2023-07-14T16:12:36.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:36 smithi164 ceph-mon[97427]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:36.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:36 smithi164 ceph-mon[97427]: osdmap e76: 11 total, 11 up, 11 in 2023-07-14T16:12:36.796 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:36.820+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2581076f0 con 0x7fb2400a11d0 2023-07-14T16:12:36.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:36 smithi062 ceph-mon[95543]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:36.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:36 smithi062 ceph-mon[95543]: osdmap e76: 11 total, 11 up, 11 in 2023-07-14T16:12:36.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:36 smithi081 ceph-mon[127795]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:36 smithi081 ceph-mon[127795]: osdmap e76: 11 total, 11 up, 11 in 2023-07-14T16:12:37.848 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:37 smithi164 ceph-mon[97427]: osdmap e77: 11 total, 11 up, 11 in 2023-07-14T16:12:37.848 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:12:37.848 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:12:37.848 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:37.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:37 smithi062 ceph-mon[95543]: osdmap e77: 11 total, 11 up, 11 in 2023-07-14T16:12:37.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:12:37.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:12:37.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:37.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:37 smithi081 ceph-mon[127795]: osdmap e77: 11 total, 11 up, 11 in 2023-07-14T16:12:37.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T16:12:37.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T16:12:37.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:38.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:38 smithi164 ceph-mon[97427]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:12:38.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:38 smithi164 ceph-mon[97427]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:12:38.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:38 smithi062 ceph-mon[95543]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:12:38.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:38 smithi062 ceph-mon[95543]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:12:38.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:38 smithi081 ceph-mon[127795]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:12:38.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:38 smithi081 ceph-mon[127795]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-14T16:12:40.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:40 smithi062 ceph-mon[95543]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T16:12:40.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:40 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/1806537019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]: dispatch 2023-07-14T16:12:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:40 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]: dispatch 2023-07-14T16:12:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:40 smithi062 ceph-mon[95543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]': finished 2023-07-14T16:12:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:40 smithi062 ceph-mon[95543]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T16:12:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:12:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:40 smithi062 ceph-mon[95543]: from='client.? 172.21.15.164:0/355247824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:12:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:40 smithi081 ceph-mon[127795]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T16:12:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:40 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/1806537019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]: dispatch 2023-07-14T16:12:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:40 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]: dispatch 2023-07-14T16:12:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:40 smithi081 ceph-mon[127795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]': finished 2023-07-14T16:12:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:40 smithi081 ceph-mon[127795]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T16:12:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:12:40.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:40 smithi081 ceph-mon[127795]: from='client.? 172.21.15.164:0/355247824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:12:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:40 smithi164 ceph-mon[97427]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-07-14T16:12:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:40 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/1806537019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]: dispatch 2023-07-14T16:12:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:40 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]: dispatch 2023-07-14T16:12:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:40 smithi164 ceph-mon[97427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fe4fc0d-eb60-4df7-869d-1925ffddd634"}]': finished 2023-07-14T16:12:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:40 smithi164 ceph-mon[97427]: osdmap e78: 12 total, 11 up, 12 in 2023-07-14T16:12:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:12:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:40 smithi164 ceph-mon[97427]: from='client.? 172.21.15.164:0/355247824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T16:12:42.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:42 smithi062 ceph-mon[95543]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:12:42.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:42 smithi081 ceph-mon[127795]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:12:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:42 smithi164 ceph-mon[97427]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T16:12:44.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:44 smithi062 ceph-mon[95543]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T16:12:44.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:44 smithi081 ceph-mon[127795]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T16:12:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:44 smithi164 ceph-mon[97427]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-14T16:12:46.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:46 smithi164 ceph-mon[97427]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:46.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:46 smithi062 ceph-mon[95543]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:46.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:46 smithi081 ceph-mon[127795]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:47.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T16:12:47.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:47.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T16:12:47.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:47.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T16:12:47.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:48.631 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:48 smithi164 ceph-mon[97427]: Deploying daemon osd.11 on smithi164 2023-07-14T16:12:48.632 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:48 smithi164 ceph-mon[97427]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:48.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:48 smithi062 ceph-mon[95543]: Deploying daemon osd.11 on smithi164 2023-07-14T16:12:48.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:48 smithi062 ceph-mon[95543]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:48.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:48 smithi081 ceph-mon[127795]: Deploying daemon osd.11 on smithi164 2023-07-14T16:12:48.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:48 smithi081 ceph-mon[127795]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:50.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:50 smithi164 ceph-mon[97427]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:50.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:50 smithi062 ceph-mon[95543]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:50.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:50 smithi081 ceph-mon[127795]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:52.821 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:52 smithi164 ceph-mon[97427]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:52.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:52 smithi062 ceph-mon[95543]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:52.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:52 smithi081 ceph-mon[127795]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:54.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:54 smithi164 ceph-mon[97427]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:54.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:54.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:54.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:54.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:54 smithi062 ceph-mon[95543]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:54.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:54.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:54.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:54.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:54 smithi081 ceph-mon[127795]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:54.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:54.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:54.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:56.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:56 smithi164 ceph-mon[97427]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:56.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:56 smithi062 ceph-mon[95543]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:56.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:56 smithi081 ceph-mon[127795]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:57.774 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.796+0000 7fb2557fa700 1 -- 172.21.15.164:0/1891369486 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb2581076f0 con 0x7fb2400a11d0 2023-07-14T16:12:57.774 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 11 on host 'smithi164' 2023-07-14T16:12:57.777 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2400a11d0 msgr2=0x7fb2400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:57.777 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2400a11d0 0x7fb2400a3680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb258100bf0 tx=0x7fb248008040 comp rx=0 tx=0).stop 2023-07-14T16:12:57.779 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258100d90 msgr2=0x7fb2581980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:12:57.779 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258100d90 0x7fb2581980f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d980 tx=0x7fb24c00dc90 comp rx=0 tx=0).stop 2023-07-14T16:12:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 shutdown_connections 2023-07-14T16:12:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb258101720 0x7fb25819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2400a11d0 0x7fb2400a3680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2580ffb90 0x7fb258197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 --2- 172.21.15.164:0/1891369486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb258100d90 0x7fb2581980f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:12:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.799+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 >> 172.21.15.164:0/1891369486 conn(0x7fb2580fb1a0 msgr2=0x7fb2580fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:12:57.780 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.800+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 shutdown_connections 2023-07-14T16:12:57.781 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:12:57.800+0000 7fb25e53f700 1 -- 172.21.15.164:0/1891369486 wait complete. 2023-07-14T16:12:58.280 DEBUG:teuthology.orchestra.run.smithi164:osd.11> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.11.service 2023-07-14T16:12:58.284 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-14T16:12:58.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd stat -f json 2023-07-14T16:12:58.362 INFO:journalctl@ceph.osd.11.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:12:58.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:58.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:58.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:58.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.629 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:58.629 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T16:12:58.629 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:58 smithi062 ceph-mon[95543]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T16:12:58.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:58.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:58.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:58.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.777 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.777 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:58.777 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T16:12:58.777 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:58 smithi164 ceph-mon[97427]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T16:12:58.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:12:58.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:12:58.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:12:58.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:12:58.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:12:58.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T16:12:58.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:58 smithi081 ceph-mon[127795]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T16:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:12:59.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:59 smithi062 ceph-mon[95543]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T16:12:59.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:59 smithi062 ceph-mon[95543]: osdmap e79: 12 total, 11 up, 12 in 2023-07-14T16:12:59.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:12:59.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:59 smithi062 ceph-mon[95543]: from='osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:12:59.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:59 smithi062 ceph-mon[95543]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:12:59.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:12:59 smithi062 ceph-mon[95543]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:13:00.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:59 smithi164 ceph-mon[97427]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T16:13:00.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:59 smithi164 ceph-mon[97427]: osdmap e79: 12 total, 11 up, 12 in 2023-07-14T16:13:00.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:59 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:00.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:59 smithi164 ceph-mon[97427]: from='osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:13:00.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:59 smithi164 ceph-mon[97427]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:13:00.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:12:59 smithi164 ceph-mon[97427]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:13:00.028 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:12:59 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:12:59.801+0000 7ff7e0748700 -1 osd.11 0 waiting for initial osdmap 2023-07-14T16:13:00.028 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:12:59 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:12:59.819+0000 7ff7dc53d700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T16:13:00.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:59 smithi081 ceph-mon[127795]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T16:13:00.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:59 smithi081 ceph-mon[127795]: osdmap e79: 12 total, 11 up, 12 in 2023-07-14T16:13:00.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:59 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:00.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:59 smithi081 ceph-mon[127795]: from='osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:13:00.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:59 smithi081 ceph-mon[127795]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2023-07-14T16:13:00.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:12:59 smithi081 ceph-mon[127795]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T16:13:00.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 -- 172.21.15.62:0/519965318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 msgr2=0x7f9368107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:00.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 --2- 172.21.15.62:0/519965318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 0x7f9368107fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f936000b0a0 tx=0x7f936000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 -- 172.21.15.62:0/519965318 shutdown_connections 2023-07-14T16:13:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 --2- 172.21.15.62:0/519965318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 0x7f9368107fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 --2- 172.21.15.62:0/519965318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93680ffb90 0x7f93680fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 --2- 172.21.15.62:0/519965318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9368100d90 0x7f93681011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 -- 172.21.15.62:0/519965318 >> 172.21.15.62:0/519965318 conn(0x7f93680fb1a0 msgr2=0x7f93680fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:00.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 -- 172.21.15.62:0/519965318 shutdown_connections 2023-07-14T16:13:00.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.561+0000 7f936e809700 1 -- 172.21.15.62:0/519965318 wait complete. 2023-07-14T16:13:00.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.562+0000 7f936e809700 1 Processor -- start 2023-07-14T16:13:00.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936e809700 1 -- start start 2023-07-14T16:13:00.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936e809700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93680ffb90 0x7f93681958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:00.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936e809700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9368100d90 0x7f9368195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:00.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936e809700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 0x7f936819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:00.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936e809700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936819d480 con 0x7f9368101720 2023-07-14T16:13:00.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936e809700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936819d5c0 con 0x7f93680ffb90 2023-07-14T16:13:00.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936e809700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936819d750 con 0x7f9368100d90 2023-07-14T16:13:00.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936cda6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 0x7f936819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:00.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936cda6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 0x7f936819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47224/0 (socket says 172.21.15.62:47224) 2023-07-14T16:13:00.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f936cda6700 1 -- 172.21.15.62:0/767003599 learned_addr learned my addr 172.21.15.62:0/767003599 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:00.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.563+0000 7f93677fe700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9368100d90 0x7f9368195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:00.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f9367fff700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93680ffb90 0x7f93681958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:00.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f936cda6700 1 -- 172.21.15.62:0/767003599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9368100d90 msgr2=0x7f9368195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:00.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f936cda6700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9368100d90 0x7f9368195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f936cda6700 1 -- 172.21.15.62:0/767003599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93680ffb90 msgr2=0x7f93681958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:00.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f936cda6700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93680ffb90 0x7f93681958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f936cda6700 1 -- 172.21.15.62:0/767003599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360009d00 con 0x7f9368101720 2023-07-14T16:13:00.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f936cda6700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 0x7f936819ceb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9360006070 tx=0x7f9360004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:00.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f93657fa700 1 -- 172.21.15.62:0/767003599 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936000e040 con 0x7f9368101720 2023-07-14T16:13:00.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.564+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936819d9d0 con 0x7f9368101720 2023-07-14T16:13:00.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.565+0000 7f93657fa700 1 -- 172.21.15.62:0/767003599 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9360003e90 con 0x7f9368101720 2023-07-14T16:13:00.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.565+0000 7f93657fa700 1 -- 172.21.15.62:0/767003599 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360004020 con 0x7f9368101720 2023-07-14T16:13:00.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.565+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936819de90 con 0x7f9368101720 2023-07-14T16:13:00.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.566+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9368107190 con 0x7f9368101720 2023-07-14T16:13:00.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.568+0000 7f93657fa700 1 -- 172.21.15.62:0/767003599 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9360003a00 con 0x7f9368101720 2023-07-14T16:13:00.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.569+0000 7f93657fa700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93500a1150 0x7f93500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:00.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.570+0000 7f93657fa700 1 -- 172.21.15.62:0/767003599 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f93600ce000 con 0x7f9368101720 2023-07-14T16:13:00.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.570+0000 7f9367fff700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93500a1150 0x7f93500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:00.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.570+0000 7f9367fff700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93500a1150 0x7f93500a3600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9368100bf0 tx=0x7f935800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:00.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.571+0000 7f93657fa700 1 -- 172.21.15.62:0/767003599 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9360097d00 con 0x7f9368101720 2023-07-14T16:13:00.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.779+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f936802c850 con 0x7f9368101720 2023-07-14T16:13:00.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.779+0000 7f93657fa700 1 -- 172.21.15.62:0/767003599 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f9360023360 con 0x7f9368101720 2023-07-14T16:13:00.758 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93500a1150 msgr2=0x7f93500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93500a1150 0x7f93500a3600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9368100bf0 tx=0x7f935800b410 comp rx=0 tx=0).stop 2023-07-14T16:13:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 msgr2=0x7f936819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 0x7f936819ceb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9360006070 tx=0x7f9360004c80 comp rx=0 tx=0).stop 2023-07-14T16:13:00.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 shutdown_connections 2023-07-14T16:13:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368101720 0x7f936819ceb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93500a1150 0x7f93500a3600 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93680ffb90 0x7f93681958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 --2- 172.21.15.62:0/767003599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9368100d90 0x7f9368195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 >> 172.21.15.62:0/767003599 conn(0x7f93680fb1a0 msgr2=0x7f93681042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:00.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 shutdown_connections 2023-07-14T16:13:00.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:00.782+0000 7f936e809700 1 -- 172.21.15.62:0/767003599 wait complete. 2023-07-14T16:13:01.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:00 smithi062 ceph-mon[95543]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:13:01.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:00 smithi062 ceph-mon[95543]: osdmap e80: 12 total, 11 up, 12 in 2023-07-14T16:13:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:00 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:00 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/767003599' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T16:13:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:00 smithi081 ceph-mon[127795]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:13:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:00 smithi081 ceph-mon[127795]: osdmap e80: 12 total, 11 up, 12 in 2023-07-14T16:13:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:00 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:00 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/767003599' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T16:13:01.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:00 smithi164 ceph-mon[97427]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2023-07-14T16:13:01.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:00 smithi164 ceph-mon[97427]: osdmap e80: 12 total, 11 up, 12 in 2023-07-14T16:13:01.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:01.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:00 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:01.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:00 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/767003599' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T16:13:01.308 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1689351153,"num_in_osds":12,"osd_in_since":1689351159,"num_remapped_pgs":0} 2023-07-14T16:13:02.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: purged_snaps scrub starts 2023-07-14T16:13:02.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: purged_snaps scrub ok 2023-07-14T16:13:02.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] boot 2023-07-14T16:13:02.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T16:13:02.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:02.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: Detected new or changed devices on smithi164 2023-07-14T16:13:02.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi164 to 4316M 2023-07-14T16:13:02.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:02.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:02.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:01 smithi062 ceph-mon[95543]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:02.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: purged_snaps scrub starts 2023-07-14T16:13:02.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: purged_snaps scrub ok 2023-07-14T16:13:02.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] boot 2023-07-14T16:13:02.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T16:13:02.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:02.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: Detected new or changed devices on smithi164 2023-07-14T16:13:02.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi164 to 4316M 2023-07-14T16:13:02.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:02.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:02.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:01 smithi081 ceph-mon[127795]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:02.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: purged_snaps scrub starts 2023-07-14T16:13:02.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: purged_snaps scrub ok 2023-07-14T16:13:02.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: osd.11 [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] boot 2023-07-14T16:13:02.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T16:13:02.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T16:13:02.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: Detected new or changed devices on smithi164 2023-07-14T16:13:02.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:13:02.277 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi164 to 4316M 2023-07-14T16:13:02.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:02.278 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:02.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:02.279 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:01 smithi164 ceph-mon[97427]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:02.309 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd stat -f json 2023-07-14T16:13:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:13:03.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:02 smithi062 ceph-mon[95543]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T16:13:03.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:02 smithi081 ceph-mon[127795]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T16:13:03.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:02 smithi164 ceph-mon[97427]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T16:13:04.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 -- 172.21.15.62:0/344351653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec101720 msgr2=0x7f5bec107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:04.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/344351653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec101720 0x7f5bec107fe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5be400b0a0 tx=0x7f5be400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 -- 172.21.15.62:0/344351653 shutdown_connections 2023-07-14T16:13:04.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/344351653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec101720 0x7f5bec107fe0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/344351653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bec0ffb90 0x7f5bec0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/344351653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bec100d90 0x7f5bec1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 -- 172.21.15.62:0/344351653 >> 172.21.15.62:0/344351653 conn(0x7f5bec0fb1a0 msgr2=0x7f5bec0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:04.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 -- 172.21.15.62:0/344351653 shutdown_connections 2023-07-14T16:13:04.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.560+0000 7f5bf0c94700 1 -- 172.21.15.62:0/344351653 wait complete. 2023-07-14T16:13:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.561+0000 7f5bf0c94700 1 Processor -- start 2023-07-14T16:13:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.561+0000 7f5bf0c94700 1 -- start start 2023-07-14T16:13:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bf0c94700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec0ffb90 0x7f5bec197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bf0c94700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bec100d90 0x7f5bec197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bf0c94700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bec101720 0x7f5bec19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bf0c94700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19f620 con 0x7f5bec0ffb90 2023-07-14T16:13:04.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bf0c94700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19f760 con 0x7f5bec101720 2023-07-14T16:13:04.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bf0c94700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19f8f0 con 0x7f5bec100d90 2023-07-14T16:13:04.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bea59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec0ffb90 0x7f5bec197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:04.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bea59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec0ffb90 0x7f5bec197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56474/0 (socket says 172.21.15.62:56474) 2023-07-14T16:13:04.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bea59c700 1 -- 172.21.15.62:0/324308303 learned_addr learned my addr 172.21.15.62:0/324308303 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:04.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bead9d700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bec101720 0x7f5bec19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:04.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5be9d9b700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bec100d90 0x7f5bec197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:04.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bea59c700 1 -- 172.21.15.62:0/324308303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bec100d90 msgr2=0x7f5bec197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:04.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bea59c700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bec100d90 0x7f5bec197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bea59c700 1 -- 172.21.15.62:0/324308303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bec101720 msgr2=0x7f5bec19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bea59c700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bec101720 0x7f5bec19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bea59c700 1 -- 172.21.15.62:0/324308303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be4009d00 con 0x7f5bec0ffb90 2023-07-14T16:13:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5bead9d700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bec101720 0x7f5bec19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:13:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.562+0000 7f5be9d9b700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bec100d90 0x7f5bec197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:13:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.563+0000 7f5bea59c700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec0ffb90 0x7f5bec197a90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e9a0 tx=0x7f5bdc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.563+0000 7f5bdb7fe700 1 -- 172.21.15.62:0/324308303 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc009790 con 0x7f5bec0ffb90 2023-07-14T16:13:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.563+0000 7f5bdb7fe700 1 -- 172.21.15.62:0/324308303 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bdc0044b0 con 0x7f5bec0ffb90 2023-07-14T16:13:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.563+0000 7f5bdb7fe700 1 -- 172.21.15.62:0/324308303 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc003d00 con 0x7f5bec0ffb90 2023-07-14T16:13:04.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.563+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec19fbd0 con 0x7f5bec0ffb90 2023-07-14T16:13:04.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.563+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec1a0080 con 0x7f5bec0ffb90 2023-07-14T16:13:04.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.564+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec10b210 con 0x7f5bec0ffb90 2023-07-14T16:13:04.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.566+0000 7f5bdb7fe700 1 -- 172.21.15.62:0/324308303 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5bdc003e60 con 0x7f5bec0ffb90 2023-07-14T16:13:04.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.567+0000 7f5bdb7fe700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bd40a1150 0x7f5bd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:04.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.567+0000 7f5bdb7fe700 1 -- 172.21.15.62:0/324308303 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5bdc014070 con 0x7f5bec0ffb90 2023-07-14T16:13:04.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.568+0000 7f5be9d9b700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bd40a1150 0x7f5bd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:04.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.568+0000 7f5be9d9b700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bd40a1150 0x7f5bd40a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5be0005fd0 tx=0x7f5be0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:04.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.571+0000 7f5bdb7fe700 1 -- 172.21.15.62:0/324308303 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5bdc096370 con 0x7f5bec0ffb90 2023-07-14T16:13:04.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:04 smithi062 ceph-mon[95543]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:04.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.776+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5bec10b430 con 0x7f5bec0ffb90 2023-07-14T16:13:04.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.777+0000 7f5bdb7fe700 1 -- 172.21.15.62:0/324308303 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7f5bdc016330 con 0x7f5bec0ffb90 2023-07-14T16:13:04.756 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:04.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bd40a1150 msgr2=0x7f5bd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:04.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bd40a1150 0x7f5bd40a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5be0005fd0 tx=0x7f5be0005ee0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec0ffb90 msgr2=0x7f5bec197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:04.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec0ffb90 0x7f5bec197a90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e9a0 tx=0x7f5bdc00ed60 comp rx=0 tx=0).stop 2023-07-14T16:13:04.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 shutdown_connections 2023-07-14T16:13:04.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bd40a1150 0x7f5bd40a3600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bec101720 0x7f5bec19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bec0ffb90 0x7f5bec197a90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 --2- 172.21.15.62:0/324308303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bec100d90 0x7f5bec197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:04.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 >> 172.21.15.62:0/324308303 conn(0x7f5bec0fb1a0 msgr2=0x7f5bec0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:04.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 shutdown_connections 2023-07-14T16:13:04.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:04.779+0000 7f5bf0c94700 1 -- 172.21.15.62:0/324308303 wait complete. 2023-07-14T16:13:04.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:04 smithi164 ceph-mon[97427]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:04 smithi081 ceph-mon[127795]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:05.224 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1689351180,"num_in_osds":12,"osd_in_since":1689351159,"num_remapped_pgs":0} 2023-07-14T16:13:05.225 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T16:13:05.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:05 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/324308303' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T16:13:05.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:05 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/324308303' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T16:13:05.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:05 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/324308303' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T16:13:05.954 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:13:06.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:06 smithi062 ceph-mon[95543]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:06.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:06 smithi164 ceph-mon[97427]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:06 smithi081 ceph-mon[127795]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:07.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.677+0000 7fe660961700 1 -- 172.21.15.62:0/866436725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1007a0 msgr2=0x7fe65c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:07.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.677+0000 7fe660961700 1 --2- 172.21.15.62:0/866436725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1007a0 0x7fe65c104770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6500099c0 tx=0x7fe650009cd0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.678+0000 7fe660961700 1 -- 172.21.15.62:0/866436725 shutdown_connections 2023-07-14T16:13:07.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.678+0000 7fe660961700 1 --2- 172.21.15.62:0/866436725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c104ea0 0x7fe65c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.678+0000 7fe660961700 1 --2- 172.21.15.62:0/866436725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe65c0ffe00 0x7fe65c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.678+0000 7fe660961700 1 --2- 172.21.15.62:0/866436725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1007a0 0x7fe65c104770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.678+0000 7fe660961700 1 -- 172.21.15.62:0/866436725 >> 172.21.15.62:0/866436725 conn(0x7fe65c0fb660 msgr2=0x7fe65c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:07.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.678+0000 7fe660961700 1 -- 172.21.15.62:0/866436725 shutdown_connections 2023-07-14T16:13:07.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.678+0000 7fe660961700 1 -- 172.21.15.62:0/866436725 wait complete. 2023-07-14T16:13:07.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.679+0000 7fe660961700 1 Processor -- start 2023-07-14T16:13:07.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.679+0000 7fe660961700 1 -- start start 2023-07-14T16:13:07.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.679+0000 7fe660961700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe65c0ffe00 0x7fe65c19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:07.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.679+0000 7fe660961700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c1007a0 0x7fe65c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:07.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.679+0000 7fe660961700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c104ea0 0x7fe65c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:07.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.679+0000 7fe660961700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c195f50 con 0x7fe65c104ea0 2023-07-14T16:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.679+0000 7fe660961700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c1960c0 con 0x7fe65c0ffe00 2023-07-14T16:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.679+0000 7fe660961700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c196280 con 0x7fe65c1007a0 2023-07-14T16:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe65c0ffe00 0x7fe65c19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe659d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c1007a0 0x7fe65c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe65c0ffe00 0x7fe65c19b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:44096/0 (socket says 172.21.15.62:44096) 2023-07-14T16:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c104ea0 0x7fe65c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe659d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c1007a0 0x7fe65c19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37744/0 (socket says 172.21.15.62:37744) 2023-07-14T16:13:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 -- 172.21.15.62:0/475887680 learned_addr learned my addr 172.21.15.62:0/475887680 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 -- 172.21.15.62:0/475887680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c1007a0 msgr2=0x7fe65c19bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c1007a0 0x7fe65c19bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 -- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c104ea0 msgr2=0x7fe65c1958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c104ea0 0x7fe65c1958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 -- 172.21.15.62:0/475887680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6500095f0 con 0x7fe65c0ffe00 2023-07-14T16:13:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65ad9d700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c104ea0 0x7fe65c1958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.680+0000 7fe65a59c700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe65c0ffe00 0x7fe65c19b860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00c9b0 tx=0x7fe64c00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.681+0000 7fe64b7fe700 1 -- 172.21.15.62:0/475887680 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c007b00 con 0x7fe65c0ffe00 2023-07-14T16:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.681+0000 7fe64b7fe700 1 -- 172.21.15.62:0/475887680 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe64c00ce50 con 0x7fe65c0ffe00 2023-07-14T16:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.681+0000 7fe64b7fe700 1 -- 172.21.15.62:0/475887680 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c018420 con 0x7fe65c0ffe00 2023-07-14T16:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.681+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c196560 con 0x7fe65c0ffe00 2023-07-14T16:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.681+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c1a2850 con 0x7fe65c0ffe00 2023-07-14T16:13:07.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.682+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c04e210 con 0x7fe65c0ffe00 2023-07-14T16:13:07.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.684+0000 7fe64b7fe700 1 -- 172.21.15.62:0/475887680 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe64c00f450 con 0x7fe65c0ffe00 2023-07-14T16:13:07.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.685+0000 7fe64b7fe700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe6440a11d0 0x7fe6440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:07.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.686+0000 7fe64b7fe700 1 -- 172.21.15.62:0/475887680 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe64c0cc6a0 con 0x7fe65c0ffe00 2023-07-14T16:13:07.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.686+0000 7fe659d9b700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe6440a11d0 0x7fe6440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:07.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.686+0000 7fe659d9b700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe6440a11d0 0x7fe6440a3680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe650006010 tx=0x7fe650009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.688+0000 7fe64b7fe700 1 -- 172.21.15.62:0/475887680 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe64c096300 con 0x7fe65c0ffe00 2023-07-14T16:13:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.895+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe65c196df0 con 0x7fe65c0ffe00 2023-07-14T16:13:07.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.897+0000 7fe64b7fe700 1 -- 172.21.15.62:0/475887680 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18371 (secure 0 0 0) 0x7fe64c00f700 con 0x7fe65c0ffe00 2023-07-14T16:13:07.874 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:07.875 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":82,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:13:01.805737+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:13:07.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe6440a11d0 msgr2=0x7fe6440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe6440a11d0 0x7fe6440a3680 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe650006010 tx=0x7fe650009f10 comp rx=0 tx=0).stop 2023-07-14T16:13:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe65c0ffe00 msgr2=0x7fe65c19b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe65c0ffe00 0x7fe65c19b860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00c9b0 tx=0x7fe64c00ccc0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 shutdown_connections 2023-07-14T16:13:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c104ea0 0x7fe65c1958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe6440a11d0 0x7fe6440a3680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe65c0ffe00 0x7fe65c19b860 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 --2- 172.21.15.62:0/475887680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe65c1007a0 0x7fe65c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:07.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 >> 172.21.15.62:0/475887680 conn(0x7fe65c0fb660 msgr2=0x7fe65c0fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:07.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 shutdown_connections 2023-07-14T16:13:07.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:07.899+0000 7fe660961700 1 -- 172.21.15.62:0/475887680 wait complete. 2023-07-14T16:13:08.338 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:13:08.339 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-14T16:13:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:08 smithi062 ceph-mon[95543]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:08 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/475887680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:08.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:08 smithi081 ceph-mon[127795]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:08.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:08 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/475887680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:08 smithi164 ceph-mon[97427]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:08 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/475887680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:09.070 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:13:10.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 -- 172.21.15.62:0/2387878135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 msgr2=0x7fd30010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:10.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 --2- 172.21.15.62:0/2387878135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 0x7fd30010ca20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800b0d0 tx=0x7fd2f800b3e0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 -- 172.21.15.62:0/2387878135 shutdown_connections 2023-07-14T16:13:10.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 --2- 172.21.15.62:0/2387878135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 0x7fd30010ca20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 --2- 172.21.15.62:0/2387878135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3000ffe00 0x7fd3001001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 --2- 172.21.15.62:0/2387878135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3001007a0 0x7fd300104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 -- 172.21.15.62:0/2387878135 >> 172.21.15.62:0/2387878135 conn(0x7fd3000fb660 msgr2=0x7fd3000fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:10.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 -- 172.21.15.62:0/2387878135 shutdown_connections 2023-07-14T16:13:10.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.580+0000 7fd307b10700 1 -- 172.21.15.62:0/2387878135 wait complete. 2023-07-14T16:13:10.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.581+0000 7fd307b10700 1 Processor -- start 2023-07-14T16:13:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd307b10700 1 -- start start 2023-07-14T16:13:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd307b10700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3000ffe00 0x7fd300195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd307b10700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3001007a0 0x7fd3001961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd307b10700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 0x7fd3001995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd307b10700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300199bb0 con 0x7fd300104ea0 2023-07-14T16:13:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd307b10700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300199d20 con 0x7fd3000ffe00 2023-07-14T16:13:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd307b10700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300199ee0 con 0x7fd3001007a0 2023-07-14T16:13:10.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd3060ad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 0x7fd3001995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:10.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd3060ad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 0x7fd3001995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56512/0 (socket says 172.21.15.62:56512) 2023-07-14T16:13:10.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd3060ad700 1 -- 172.21.15.62:0/627782664 learned_addr learned my addr 172.21.15.62:0/627782664 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:10.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.582+0000 7fd3058ac700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3000ffe00 0x7fd300195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd3050ab700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3001007a0 0x7fd3001961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd3060ad700 1 -- 172.21.15.62:0/627782664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3001007a0 msgr2=0x7fd3001961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd3060ad700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3001007a0 0x7fd3001961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd3060ad700 1 -- 172.21.15.62:0/627782664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3000ffe00 msgr2=0x7fd300195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd3060ad700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3000ffe00 0x7fd300195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd3060ad700 1 -- 172.21.15.62:0/627782664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f8009d00 con 0x7fd300104ea0 2023-07-14T16:13:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd3058ac700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3000ffe00 0x7fd300195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:13:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd3060ad700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 0x7fd3001995b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009300 tx=0x7fd2f80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:10.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd2f6ffd700 1 -- 172.21.15.62:0/627782664 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f800e040 con 0x7fd300104ea0 2023-07-14T16:13:10.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd2f6ffd700 1 -- 172.21.15.62:0/627782664 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2f8003e10 con 0x7fd300104ea0 2023-07-14T16:13:10.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30019a160 con 0x7fd300104ea0 2023-07-14T16:13:10.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd2f6ffd700 1 -- 172.21.15.62:0/627782664 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f801b8e0 con 0x7fd300104ea0 2023-07-14T16:13:10.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.583+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30019a430 con 0x7fd300104ea0 2023-07-14T16:13:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.584+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30004e210 con 0x7fd300104ea0 2023-07-14T16:13:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.587+0000 7fd2f6ffd700 1 -- 172.21.15.62:0/627782664 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd2f80050d0 con 0x7fd300104ea0 2023-07-14T16:13:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.589+0000 7fd2f6ffd700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2ec0a11d0 0x7fd2ec0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.589+0000 7fd2f6ffd700 1 -- 172.21.15.62:0/627782664 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2f80ce0b0 con 0x7fd300104ea0 2023-07-14T16:13:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.589+0000 7fd3058ac700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2ec0a11d0 0x7fd2ec0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.590+0000 7fd3058ac700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2ec0a11d0 0x7fd2ec0a3680 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005950 tx=0x7fd2fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.591+0000 7fd2f6ffd700 1 -- 172.21.15.62:0/627782664 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd2f8097c90 con 0x7fd300104ea0 2023-07-14T16:13:10.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:10 smithi062 ceph-mon[95543]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:10.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.796+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3001a2ce0 con 0x7fd300104ea0 2023-07-14T16:13:10.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.797+0000 7fd2f6ffd700 1 -- 172.21.15.62:0/627782664 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2f809b2b0 con 0x7fd300104ea0 2023-07-14T16:13:10.774 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:13:10.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2ec0a11d0 msgr2=0x7fd2ec0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:10.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2ec0a11d0 0x7fd2ec0a3680 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005950 tx=0x7fd2fc0058e0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 msgr2=0x7fd3001995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:10.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 0x7fd3001995b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009300 tx=0x7fd2f80093e0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 shutdown_connections 2023-07-14T16:13:10.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd300104ea0 0x7fd3001995b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2ec0a11d0 0x7fd2ec0a3680 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3000ffe00 0x7fd300195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 --2- 172.21.15.62:0/627782664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3001007a0 0x7fd3001961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:10.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.799+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 >> 172.21.15.62:0/627782664 conn(0x7fd3000fb660 msgr2=0x7fd3000fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:10.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.800+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 shutdown_connections 2023-07-14T16:13:10.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:10.800+0000 7fd307b10700 1 -- 172.21.15.62:0/627782664 wait complete. 2023-07-14T16:13:10.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:10 smithi081 ceph-mon[127795]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:11.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:10 smithi164 ceph-mon[97427]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:11.250 INFO:tasks.cephadm:Adding mds.a on smithi062 2023-07-14T16:13:11.251 INFO:tasks.cephadm:Adding mds.d on smithi062 2023-07-14T16:13:11.251 INFO:tasks.cephadm:Adding mds.g on smithi062 2023-07-14T16:13:11.251 INFO:tasks.cephadm:Adding mds.j on smithi062 2023-07-14T16:13:11.251 INFO:tasks.cephadm:Adding mds.b on smithi081 2023-07-14T16:13:11.252 INFO:tasks.cephadm:Adding mds.e on smithi081 2023-07-14T16:13:11.252 INFO:tasks.cephadm:Adding mds.h on smithi081 2023-07-14T16:13:11.252 INFO:tasks.cephadm:Adding mds.k on smithi081 2023-07-14T16:13:11.252 INFO:tasks.cephadm:Adding mds.c on smithi164 2023-07-14T16:13:11.252 INFO:tasks.cephadm:Adding mds.f on smithi164 2023-07-14T16:13:11.253 INFO:tasks.cephadm:Adding mds.i on smithi164 2023-07-14T16:13:11.254 INFO:tasks.cephadm:Adding mds.l on smithi164 2023-07-14T16:13:11.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph orch apply mds all '12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l' 2023-07-14T16:13:11.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:11 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/627782664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:11.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:11 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/627782664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:11.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:11 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/627782664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:11.991 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:13:12.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:12 smithi062 ceph-mon[95543]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:12 smithi081 ceph-mon[127795]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:12.946 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:12 smithi164 ceph-mon[97427]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:13.631 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.653+0000 7f7fa91fe700 1 -- 172.21.15.164:0/1350570389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa40ffb90 msgr2=0x7f7fa40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:13.632 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.653+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/1350570389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa40ffb90 0x7f7fa40fffa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009990 tx=0x7f7f94009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.632 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.654+0000 7f7fa91fe700 1 -- 172.21.15.164:0/1350570389 shutdown_connections 2023-07-14T16:13:13.632 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.654+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/1350570389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa4101720 0x7f7fa4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.632 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.654+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/1350570389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa40ffb90 0x7f7fa40fffa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.633 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.654+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/1350570389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa4100d90 0x7f7fa41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.633 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.654+0000 7f7fa91fe700 1 -- 172.21.15.164:0/1350570389 >> 172.21.15.164:0/1350570389 conn(0x7f7fa40fb1a0 msgr2=0x7f7fa40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:13.633 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.654+0000 7f7fa91fe700 1 -- 172.21.15.164:0/1350570389 shutdown_connections 2023-07-14T16:13:13.633 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.654+0000 7f7fa91fe700 1 -- 172.21.15.164:0/1350570389 wait complete. 2023-07-14T16:13:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa91fe700 1 Processor -- start 2023-07-14T16:13:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa91fe700 1 -- start start 2023-07-14T16:13:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa91fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa40ffb90 0x7f7fa4195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa91fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa4100d90 0x7f7fa4195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa91fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa4101720 0x7f7fa419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:13.635 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa91fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa419d3c0 con 0x7f7fa40ffb90 2023-07-14T16:13:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa91fe700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa419d500 con 0x7f7fa4101720 2023-07-14T16:13:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa91fe700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa419d690 con 0x7f7fa4100d90 2023-07-14T16:13:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa4100d90 0x7f7fa4195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa4100d90 0x7f7fa4195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42006/0 (socket says 172.21.15.164:42006) 2023-07-14T16:13:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa259c700 1 -- 172.21.15.164:0/2420146349 learned_addr learned my addr 172.21.15.164:0/2420146349 (peer_addr_for_me v2:172.21.15.164:0/0) 2023-07-14T16:13:13.636 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa2d9d700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa40ffb90 0x7f7fa4195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:13.637 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.655+0000 7f7fa359e700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa4101720 0x7f7fa419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:13.637 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa259c700 1 -- 172.21.15.164:0/2420146349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa4101720 msgr2=0x7f7fa419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:13.637 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa259c700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa4101720 0x7f7fa419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.637 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa259c700 1 -- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa40ffb90 msgr2=0x7f7fa4195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:13.637 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa259c700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa40ffb90 0x7f7fa4195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.638 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa259c700 1 -- 172.21.15.164:0/2420146349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f940095f0 con 0x7f7fa4100d90 2023-07-14T16:13:13.638 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa359e700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa4101720 0x7f7fa419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:13:13.638 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa259c700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa4100d90 0x7f7fa4195d70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800d980 tx=0x7f7f9800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:13.638 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7f8bfff700 1 -- 172.21.15.164:0/2420146349 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98009b40 con 0x7f7fa4100d90 2023-07-14T16:13:13.638 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa419d970 con 0x7f7fa4100d90 2023-07-14T16:13:13.638 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7f8bfff700 1 -- 172.21.15.164:0/2420146349 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f9800b6e0 con 0x7f7fa4100d90 2023-07-14T16:13:13.638 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7f8bfff700 1 -- 172.21.15.164:0/2420146349 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800f3c0 con 0x7f7fa4100d90 2023-07-14T16:13:13.639 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.656+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa419de70 con 0x7f7fa4100d90 2023-07-14T16:13:13.639 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.657+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa4106b60 con 0x7f7fa4100d90 2023-07-14T16:13:13.639 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.659+0000 7f7f8bfff700 1 -- 172.21.15.164:0/2420146349 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7f98010460 con 0x7f7fa4100d90 2023-07-14T16:13:13.640 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.660+0000 7f7f8bfff700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f8c0a1150 0x7f7f8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:13.640 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.661+0000 7f7f8bfff700 1 -- 172.21.15.164:0/2420146349 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7f980cc3c0 con 0x7f7fa4100d90 2023-07-14T16:13:13.640 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.661+0000 7f7fa2d9d700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f8c0a1150 0x7f7f8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:13.640 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.662+0000 7f7fa2d9d700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f8c0a1150 0x7f7f8c0a3600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400b5c0 tx=0x7f7f94005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:13.642 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.665+0000 7f7f8bfff700 1 -- 172.21.15.164:0/2420146349 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7f98095f10 con 0x7f7fa4100d90 2023-07-14T16:13:13.848 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.870+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fa4060a50 con 0x7f7f8c0a1150 2023-07-14T16:13:13.857 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.879+0000 7f7f8bfff700 1 -- 172.21.15.164:0/2420146349 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f7fa4060a50 con 0x7f7f8c0a1150 2023-07-14T16:13:13.857 INFO:teuthology.orchestra.run.smithi164.stdout:Scheduled mds.all update... 2023-07-14T16:13:13.860 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f8c0a1150 msgr2=0x7f7f8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:13.860 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f8c0a1150 0x7f7f8c0a3600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400b5c0 tx=0x7f7f94005fb0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.860 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa4100d90 msgr2=0x7f7fa4195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:13.861 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa4100d90 0x7f7fa4195d70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800d980 tx=0x7f7f9800dc90 comp rx=0 tx=0).stop 2023-07-14T16:13:13.861 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 shutdown_connections 2023-07-14T16:13:13.861 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f8c0a1150 0x7f7f8c0a3600 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.861 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa4101720 0x7f7fa419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.861 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa40ffb90 0x7f7fa4195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.861 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 --2- 172.21.15.164:0/2420146349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa4100d90 0x7f7fa4195d70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:13.861 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 >> 172.21.15.164:0/2420146349 conn(0x7f7fa40fb1a0 msgr2=0x7f7fa40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:13.862 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 shutdown_connections 2023-07-14T16:13:13.862 INFO:teuthology.orchestra.run.smithi164.stderr:2023-07-14T16:13:13.882+0000 7f7fa91fe700 1 -- 172.21.15.164:0/2420146349 wait complete. 2023-07-14T16:13:14.470 DEBUG:teuthology.orchestra.run.smithi062:mds.a> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.a.service 2023-07-14T16:13:14.476 DEBUG:teuthology.orchestra.run.smithi062:mds.d> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.d.service 2023-07-14T16:13:14.521 DEBUG:teuthology.orchestra.run.smithi062:mds.g> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.g.service 2023-07-14T16:13:14.560 INFO:journalctl@ceph.mds.a.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:13:14.563 DEBUG:teuthology.orchestra.run.smithi062:mds.j> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.j.service 2023-07-14T16:13:14.598 INFO:journalctl@ceph.mds.d.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:13:14.601 DEBUG:teuthology.orchestra.run.smithi081:mds.b> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.b.service 2023-07-14T16:13:14.607 DEBUG:teuthology.orchestra.run.smithi081:mds.e> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.e.service 2023-07-14T16:13:14.636 INFO:journalctl@ceph.mds.g.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:13:14.653 DEBUG:teuthology.orchestra.run.smithi081:mds.h> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.h.service 2023-07-14T16:13:14.670 INFO:journalctl@ceph.mds.j.smithi062.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:13:14.688 INFO:journalctl@ceph.mds.b.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:13:14.691 DEBUG:teuthology.orchestra.run.smithi081:mds.k> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.k.service 2023-07-14T16:13:14.728 INFO:journalctl@ceph.mds.e.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:13:14.730 DEBUG:teuthology.orchestra.run.smithi164:mds.c> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.c.service 2023-07-14T16:13:14.735 DEBUG:teuthology.orchestra.run.smithi164:mds.f> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.f.service 2023-07-14T16:13:14.764 INFO:journalctl@ceph.mds.h.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:13:14.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:14 smithi164 ceph-mon[97427]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:14.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:14.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:14.781 DEBUG:teuthology.orchestra.run.smithi164:mds.i> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.i.service 2023-07-14T16:13:14.800 INFO:journalctl@ceph.mds.k.smithi081.stdout:-- Logs begin at Fri 2023-07-14 15:44:14 UTC. -- 2023-07-14T16:13:14.813 INFO:journalctl@ceph.mds.c.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:13:14.816 DEBUG:teuthology.orchestra.run.smithi164:mds.l> sudo journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.l.service 2023-07-14T16:13:14.858 INFO:journalctl@ceph.mds.f.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:13:14.861 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-14T16:13:14.862 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:13:14.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:14 smithi062 ceph-mon[95543]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:14.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:14.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:14.892 INFO:journalctl@ceph.mds.i.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:13:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:14 smithi081 ceph-mon[127795]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:14.933 INFO:journalctl@ceph.mds.l.smithi164.stdout:-- Logs begin at Fri 2023-07-14 15:45:23 UTC. -- 2023-07-14T16:13:15.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 -- 172.21.15.62:0/692737065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c0744f0 msgr2=0x7f887c072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 --2- 172.21.15.62:0/692737065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c0744f0 0x7f887c072940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f886c009990 tx=0x7f886c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 -- 172.21.15.62:0/692737065 shutdown_connections 2023-07-14T16:13:15.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 --2- 172.21.15.62:0/692737065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f887c1079b0 0x7f887c109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 --2- 172.21.15.62:0/692737065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c0744f0 0x7f887c072940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 --2- 172.21.15.62:0/692737065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c072f10 0x7f887c073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 -- 172.21.15.62:0/692737065 >> 172.21.15.62:0/692737065 conn(0x7f887c0fb160 msgr2=0x7f887c0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:15.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 -- 172.21.15.62:0/692737065 shutdown_connections 2023-07-14T16:13:15.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.110+0000 7f8881fff700 1 -- 172.21.15.62:0/692737065 wait complete. 2023-07-14T16:13:15.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.111+0000 7f8881fff700 1 Processor -- start 2023-07-14T16:13:15.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f8881fff700 1 -- start start 2023-07-14T16:13:15.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f8881fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c072f10 0x7f887c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:15.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f8881fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c0744f0 0x7f887c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:15.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f8881fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f887c1079b0 0x7f887c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:15.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f8881fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c19f4d0 con 0x7f887c072f10 2023-07-14T16:13:15.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f8881fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c19f610 con 0x7f887c0744f0 2023-07-14T16:13:15.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f8881fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c19f7a0 con 0x7f887c1079b0 2023-07-14T16:13:15.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f887b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c072f10 0x7f887c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:15.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f887b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c072f10 0x7f887c197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58944/0 (socket says 172.21.15.62:58944) 2023-07-14T16:13:15.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f887b7fe700 1 -- 172.21.15.62:0/3521257834 learned_addr learned my addr 172.21.15.62:0/3521257834 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:15.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f8873fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c0744f0 0x7f887c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.112+0000 7f887bfff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f887c1079b0 0x7f887c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f887b7fe700 1 -- 172.21.15.62:0/3521257834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f887c1079b0 msgr2=0x7f887c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f887b7fe700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f887c1079b0 0x7f887c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f887b7fe700 1 -- 172.21.15.62:0/3521257834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c0744f0 msgr2=0x7f887c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f887b7fe700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c0744f0 0x7f887c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f887b7fe700 1 -- 172.21.15.62:0/3521257834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c0095f0 con 0x7f887c072f10 2023-07-14T16:13:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f887b7fe700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c072f10 0x7f887c197950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f886c005910 tx=0x7f886c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f88797fa700 1 -- 172.21.15.62:0/3521257834 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c01d070 con 0x7f887c072f10 2023-07-14T16:13:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f88797fa700 1 -- 172.21.15.62:0/3521257834 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f886c00be70 con 0x7f887c072f10 2023-07-14T16:13:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c19fa20 con 0x7f887c072f10 2023-07-14T16:13:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f88797fa700 1 -- 172.21.15.62:0/3521257834 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c022e60 con 0x7f887c072f10 2023-07-14T16:13:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.113+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c19fe40 con 0x7f887c072f10 2023-07-14T16:13:15.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.114+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c1060f0 con 0x7f887c072f10 2023-07-14T16:13:15.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.116+0000 7f88797fa700 1 -- 172.21.15.62:0/3521257834 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f886c022600 con 0x7f887c072f10 2023-07-14T16:13:15.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.118+0000 7f88797fa700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88680a11a0 0x7f88680a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.118+0000 7f88797fa700 1 -- 172.21.15.62:0/3521257834 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f886c0cd8f0 con 0x7f887c072f10 2023-07-14T16:13:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.118+0000 7f8873fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88680a11a0 0x7f88680a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.118+0000 7f8873fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88680a11a0 0x7f88680a3650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8864009c60 tx=0x7f8864009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.122+0000 7f88797fa700 1 -- 172.21.15.62:0/3521257834 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f886c05c790 con 0x7f887c072f10 2023-07-14T16:13:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.332+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f887c02c850 con 0x7f887c072f10 2023-07-14T16:13:15.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.333+0000 7f88797fa700 1 -- 172.21.15.62:0/3521257834 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18371 (secure 0 0 0) 0x7f886c0974d0 con 0x7f887c072f10 2023-07-14T16:13:15.515 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:15.515 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":82,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:13:01.805737+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:13:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.335+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88680a11a0 msgr2=0x7f88680a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.335+0000 7f8881fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88680a11a0 0x7f88680a3650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8864009c60 tx=0x7f8864009260 comp rx=0 tx=0).stop 2023-07-14T16:13:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.335+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c072f10 msgr2=0x7f887c197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.335+0000 7f8881fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c072f10 0x7f887c197950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f886c005910 tx=0x7f886c005740 comp rx=0 tx=0).stop 2023-07-14T16:13:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.335+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 shutdown_connections 2023-07-14T16:13:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.335+0000 7f8881fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f887c1079b0 0x7f887c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.335+0000 7f8881fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f887c072f10 0x7f887c197950 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.336+0000 7f8881fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88680a11a0 0x7f88680a3650 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.336+0000 7f8881fff700 1 --2- 172.21.15.62:0/3521257834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f887c0744f0 0x7f887c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.336+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 >> 172.21.15.62:0/3521257834 conn(0x7f887c0fb160 msgr2=0x7f887c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.336+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 shutdown_connections 2023-07-14T16:13:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.336+0000 7f8881fff700 1 -- 172.21.15.62:0/3521257834 wait complete. 2023-07-14T16:13:15.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:13:15.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:13:15.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:13:15.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2023-07-14T16:13:15.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:15.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:15.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:15.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:15.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:15.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:15.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:15 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3521257834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:15.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 -- 172.21.15.62:0/775344363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 msgr2=0x7fa96c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 --2- 172.21.15.62:0/775344363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 0x7fa96c101130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa960009990 tx=0x7fa960009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 -- 172.21.15.62:0/775344363 shutdown_connections 2023-07-14T16:13:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 --2- 172.21.15.62:0/775344363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c101670 0x7fa96c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 --2- 172.21.15.62:0/775344363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa96c0ffaf0 0x7fa96c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 --2- 172.21.15.62:0/775344363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 0x7fa96c101130 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 -- 172.21.15.62:0/775344363 >> 172.21.15.62:0/775344363 conn(0x7fa96c0fb130 msgr2=0x7fa96c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:15.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 -- 172.21.15.62:0/775344363 shutdown_connections 2023-07-14T16:13:15.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.790+0000 7fa970dff700 1 -- 172.21.15.62:0/775344363 wait complete. 2023-07-14T16:13:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.791+0000 7fa970dff700 1 Processor -- start 2023-07-14T16:13:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.791+0000 7fa970dff700 1 -- start start 2023-07-14T16:13:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.791+0000 7fa970dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa96c0ffaf0 0x7fa96c195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa970dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 0x7fa96c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:15.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa970dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c101670 0x7fa96c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:15.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa970dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c19d300 con 0x7fa96c100ce0 2023-07-14T16:13:15.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa970dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c19d440 con 0x7fa96c0ffaf0 2023-07-14T16:13:15.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa970dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c19d5d0 con 0x7fa96c101670 2023-07-14T16:13:15.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 0x7fa96c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:15.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 0x7fa96c195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58960/0 (socket says 172.21.15.62:58960) 2023-07-14T16:13:15.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 -- 172.21.15.62:0/3566250038 learned_addr learned my addr 172.21.15.62:0/3566250038 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:15.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa96ad9d700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c101670 0x7fa96c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa96a59c700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa96c0ffaf0 0x7fa96c195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 -- 172.21.15.62:0/3566250038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c101670 msgr2=0x7fa96c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c101670 0x7fa96c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 -- 172.21.15.62:0/3566250038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa96c0ffaf0 msgr2=0x7fa96c195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa96c0ffaf0 0x7fa96c195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 -- 172.21.15.62:0/3566250038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9600095f0 con 0x7fa96c100ce0 2023-07-14T16:13:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.792+0000 7fa969d9b700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 0x7fa96c195cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa960005910 tx=0x7fa960005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:15.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.793+0000 7fa95b7fe700 1 -- 172.21.15.62:0/3566250038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96001d070 con 0x7fa96c100ce0 2023-07-14T16:13:15.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.793+0000 7fa95b7fe700 1 -- 172.21.15.62:0/3566250038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa96000be90 con 0x7fa96c100ce0 2023-07-14T16:13:15.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.793+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c19d850 con 0x7fa96c100ce0 2023-07-14T16:13:15.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.793+0000 7fa95b7fe700 1 -- 172.21.15.62:0/3566250038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960022e60 con 0x7fa96c100ce0 2023-07-14T16:13:15.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.793+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c19dc70 con 0x7fa96c100ce0 2023-07-14T16:13:15.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.794+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c10b1c0 con 0x7fa96c100ce0 2023-07-14T16:13:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.796+0000 7fa95b7fe700 1 -- 172.21.15.62:0/3566250038 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa960022600 con 0x7fa96c100ce0 2023-07-14T16:13:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.797+0000 7fa95b7fe700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9540a1150 0x7fa9540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:15.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.797+0000 7fa95b7fe700 1 -- 172.21.15.62:0/3566250038 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9600cd910 con 0x7fa96c100ce0 2023-07-14T16:13:15.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.798+0000 7fa96a59c700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9540a1150 0x7fa9540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:15.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.798+0000 7fa96a59c700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9540a1150 0x7fa9540a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa95c005950 tx=0x7fa95c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:15.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:15.801+0000 7fa95b7fe700 1 -- 172.21.15.62:0/3566250038 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa96005c790 con 0x7fa96c100ce0 2023-07-14T16:13:15.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:13:15.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2023-07-14T16:13:15.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:15.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:15.834 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:15.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:15.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:15.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:15.835 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:15 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3521257834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:15.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:13:15.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2023-07-14T16:13:15.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:15.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:15.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:15.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:15.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:15.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:15.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:15 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3521257834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:15.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.010+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa96c106a70 con 0x7fa96c100ce0 2023-07-14T16:13:15.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.011+0000 7fa95b7fe700 1 -- 172.21.15.62:0/3566250038 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9600974f0 con 0x7fa96c100ce0 2023-07-14T16:13:15.991 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:13:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9540a1150 msgr2=0x7fa9540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9540a1150 0x7fa9540a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa95c005950 tx=0x7fa95c0058e0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 msgr2=0x7fa96c195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 0x7fa96c195cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa960005910 tx=0x7fa960005720 comp rx=0 tx=0).stop 2023-07-14T16:13:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 shutdown_connections 2023-07-14T16:13:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa96c101670 0x7fa96c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9540a1150 0x7fa9540a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa96c0ffaf0 0x7fa96c195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 --2- 172.21.15.62:0/3566250038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96c100ce0 0x7fa96c195cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:15.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 >> 172.21.15.62:0/3566250038 conn(0x7fa96c0fb130 msgr2=0x7fa96c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:15.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.013+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 shutdown_connections 2023-07-14T16:13:15.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.014+0000 7fa970dff700 1 -- 172.21.15.62:0/3566250038 wait complete. 2023-07-14T16:13:16.009 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:16.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 -- 172.21.15.62:0/3494687765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c0ffbc0 msgr2=0x7f101c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:16.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 --2- 172.21.15.62:0/3494687765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c0ffbc0 0x7f101c100010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1010009990 tx=0x7f1010009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 -- 172.21.15.62:0/3494687765 shutdown_connections 2023-07-14T16:13:16.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 --2- 172.21.15.62:0/3494687765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f101c100550 0x7f101c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 --2- 172.21.15.62:0/3494687765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f101c0fe9d0 0x7f101c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 --2- 172.21.15.62:0/3494687765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c0ffbc0 0x7f101c100010 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 -- 172.21.15.62:0/3494687765 >> 172.21.15.62:0/3494687765 conn(0x7f101c0f9fd0 msgr2=0x7f101c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:16.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 -- 172.21.15.62:0/3494687765 shutdown_connections 2023-07-14T16:13:16.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.238+0000 7f1020fff700 1 -- 172.21.15.62:0/3494687765 wait complete. 2023-07-14T16:13:16.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.239+0000 7f1020fff700 1 Processor -- start 2023-07-14T16:13:16.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f1020fff700 1 -- start start 2023-07-14T16:13:16.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f1020fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f101c0fe9d0 0x7f101c102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:16.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f1020fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f101c0ffbc0 0x7f101c100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:16.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f1020fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c100550 0x7f101c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:16.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f1020fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c101c80 con 0x7f101c100550 2023-07-14T16:13:16.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f1020fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c101df0 con 0x7f101c0ffbc0 2023-07-14T16:13:16.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f1020fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c19f7a0 con 0x7f101c0fe9d0 2023-07-14T16:13:16.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f101ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c100550 0x7f101c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:16.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f101ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c100550 0x7f101c1014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58974/0 (socket says 172.21.15.62:58974) 2023-07-14T16:13:16.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.240+0000 7f101ad9d700 1 -- 172.21.15.62:0/2957191497 learned_addr learned my addr 172.21.15.62:0/2957191497 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:16.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f101a59c700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f101c0fe9d0 0x7f101c102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:16.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f1019d9b700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f101c0ffbc0 0x7f101c100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:16.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f101ad9d700 1 -- 172.21.15.62:0/2957191497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f101c0fe9d0 msgr2=0x7f101c102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:16.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f101ad9d700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f101c0fe9d0 0x7f101c102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f101ad9d700 1 -- 172.21.15.62:0/2957191497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f101c0ffbc0 msgr2=0x7f101c100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:16.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f101ad9d700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f101c0ffbc0 0x7f101c100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f101ad9d700 1 -- 172.21.15.62:0/2957191497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10100095f0 con 0x7f101c100550 2023-07-14T16:13:16.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f1019d9b700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f101c0ffbc0 0x7f101c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:13:16.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f101ad9d700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c100550 0x7f101c1014e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1014007ee0 tx=0x7f101400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:16.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f100b7fe700 1 -- 172.21.15.62:0/2957191497 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014017070 con 0x7f101c100550 2023-07-14T16:13:16.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c19fa80 con 0x7f101c100550 2023-07-14T16:13:16.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f100b7fe700 1 -- 172.21.15.62:0/2957191497 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f101400f040 con 0x7f101c100550 2023-07-14T16:13:16.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f100b7fe700 1 -- 172.21.15.62:0/2957191497 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014005710 con 0x7f101c100550 2023-07-14T16:13:16.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.241+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c19ffe0 con 0x7f101c100550 2023-07-14T16:13:16.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.242+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c10a0a0 con 0x7f101c100550 2023-07-14T16:13:16.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.244+0000 7f100b7fe700 1 -- 172.21.15.62:0/2957191497 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1014005870 con 0x7f101c100550 2023-07-14T16:13:16.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.246+0000 7f100b7fe700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10040a11a0 0x7f10040a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:16.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.247+0000 7f100b7fe700 1 -- 172.21.15.62:0/2957191497 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f10140cbd90 con 0x7f101c100550 2023-07-14T16:13:16.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.247+0000 7f101a59c700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10040a11a0 0x7f10040a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:16.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.247+0000 7f101a59c700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10040a11a0 0x7f10040a3650 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f101c0ffa20 tx=0x7f100c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:16.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.251+0000 7f100b7fe700 1 -- 172.21.15.62:0/2957191497 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f10140958e0 con 0x7f101c100550 2023-07-14T16:13:16.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.506+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f101c10a290 con 0x7f101c100550 2023-07-14T16:13:16.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.506+0000 7f100b7fe700 1 -- 172.21.15.62:0/2957191497 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f1014098f00 con 0x7f101c100550 2023-07-14T16:13:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:16.484 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-14T16:13:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10040a11a0 msgr2=0x7f10040a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10040a11a0 0x7f10040a3650 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f101c0ffa20 tx=0x7f100c009260 comp rx=0 tx=0).stop 2023-07-14T16:13:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c100550 msgr2=0x7f101c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c100550 0x7f101c1014e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1014007ee0 tx=0x7f101400d3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 shutdown_connections 2023-07-14T16:13:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f101c100550 0x7f101c1014e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f101c0fe9d0 0x7f101c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10040a11a0 0x7f10040a3650 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 --2- 172.21.15.62:0/2957191497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f101c0ffbc0 0x7f101c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.508+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 >> 172.21.15.62:0/2957191497 conn(0x7f101c0f9fd0 msgr2=0x7f101c0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.509+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 shutdown_connections 2023-07-14T16:13:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:16.509+0000 7f1020fff700 1 -- 172.21.15.62:0/2957191497 wait complete. 2023-07-14T16:13:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 1 2023-07-14T16:13:16.730 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:16 smithi164 ceph-mon[97427]: Deploying daemon mds.l on smithi164 2023-07-14T16:13:16.731 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:16 smithi164 ceph-mon[97427]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:16.731 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:16 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3566250038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:16.731 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:16 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2957191497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:16.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:16 smithi062 ceph-mon[95543]: Deploying daemon mds.l on smithi164 2023-07-14T16:13:16.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:16 smithi062 ceph-mon[95543]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:16.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:16 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3566250038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:16.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:16 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2957191497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:16.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:16 smithi081 ceph-mon[127795]: Deploying daemon mds.l on smithi164 2023-07-14T16:13:16.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:16 smithi081 ceph-mon[127795]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:16.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:16 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3566250038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:16.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:16 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2957191497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:16.982 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:13:17 smithi164 systemd[1]: Starting Ceph mds.l for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:17.733 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:13:17 smithi164 podman[122845]: 2023-07-14T16:13:18.025 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:13:17 smithi164 bash[122845]: 6d81e6da785a9a1ab5efc25739d76e4ee0ba0b2cbca31e9f1c0e4593b6b4f1ea 2023-07-14T16:13:18.025 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:13:17 smithi164 ceph-mds[122885]: -- 172.21.15.164:0/885148931 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5578c37fd1e0 con 0x5578c37d2800 2023-07-14T16:13:18.026 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:13:17 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-l[122861]: starting mds.l at 2023-07-14T16:13:18.026 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:13:17 smithi164 systemd[1]: Started Ceph mds.l for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:18.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:18.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 -- 172.21.15.62:0/3491192297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 msgr2=0x7f0938107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 --2- 172.21.15.62:0/3491192297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 0x7f0938107f40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f093000b0a0 tx=0x7f093000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 -- 172.21.15.62:0/3491192297 shutdown_connections 2023-07-14T16:13:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 --2- 172.21.15.62:0/3491192297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 0x7f0938107f40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 --2- 172.21.15.62:0/3491192297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09380ffb00 0x7f09380fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 --2- 172.21.15.62:0/3491192297 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0938100cf0 0x7f0938101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 -- 172.21.15.62:0/3491192297 >> 172.21.15.62:0/3491192297 conn(0x7f09380fb0e0 msgr2=0x7f09380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 -- 172.21.15.62:0/3491192297 shutdown_connections 2023-07-14T16:13:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.747+0000 7f09401ff700 1 -- 172.21.15.62:0/3491192297 wait complete. 2023-07-14T16:13:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.748+0000 7f09401ff700 1 Processor -- start 2023-07-14T16:13:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.748+0000 7f09401ff700 1 -- start start 2023-07-14T16:13:18.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f09401ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09380ffb00 0x7f09380742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:18.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f09401ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0938100cf0 0x7f0938072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:18.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f09401ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 0x7f0938072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:18.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f09401ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09380735e0 con 0x7f0938101680 2023-07-14T16:13:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f09401ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0938073750 con 0x7f0938100cf0 2023-07-14T16:13:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f09401ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093819f880 con 0x7f09380ffb00 2023-07-14T16:13:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 0x7f0938072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093df9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09380ffb00 0x7f09380742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 0x7f0938072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58996/0 (socket says 172.21.15.62:58996) 2023-07-14T16:13:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093df9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09380ffb00 0x7f09380742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:48764/0 (socket says 172.21.15.62:48764) 2023-07-14T16:13:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093e79c700 1 -- 172.21.15.62:0/4049877572 learned_addr learned my addr 172.21.15.62:0/4049877572 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:18.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093d79a700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0938100cf0 0x7f0938072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:18.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093e79c700 1 -- 172.21.15.62:0/4049877572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09380ffb00 msgr2=0x7f09380742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:18.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093e79c700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09380ffb00 0x7f09380742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093e79c700 1 -- 172.21.15.62:0/4049877572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0938100cf0 msgr2=0x7f0938072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:18.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093e79c700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0938100cf0 0x7f0938072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.749+0000 7f093e79c700 1 -- 172.21.15.62:0/4049877572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930009d00 con 0x7f0938101680 2023-07-14T16:13:18.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.750+0000 7f093e79c700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 0x7f0938072e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0930009fd0 tx=0x7f093000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:18.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.750+0000 7f092effd700 1 -- 172.21.15.62:0/4049877572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000e040 con 0x7f0938101680 2023-07-14T16:13:18.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.750+0000 7f092effd700 1 -- 172.21.15.62:0/4049877572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f093000be90 con 0x7f0938101680 2023-07-14T16:13:18.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.750+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093819fb00 con 0x7f0938101680 2023-07-14T16:13:18.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.750+0000 7f092effd700 1 -- 172.21.15.62:0/4049877572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093001b460 con 0x7f0938101680 2023-07-14T16:13:18.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.750+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093819ff20 con 0x7f0938101680 2023-07-14T16:13:18.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.751+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0938106940 con 0x7f0938101680 2023-07-14T16:13:18.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.753+0000 7f092effd700 1 -- 172.21.15.62:0/4049877572 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f093001d470 con 0x7f0938101680 2023-07-14T16:13:18.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.755+0000 7f092effd700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f09240a1220 0x7f09240a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:18.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.755+0000 7f092effd700 1 -- 172.21.15.62:0/4049877572 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09300cd610 con 0x7f0938101680 2023-07-14T16:13:18.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.755+0000 7f093df9b700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f09240a1220 0x7f09240a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:18.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.755+0000 7f093df9b700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f09240a1220 0x7f09240a36d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0938100b50 tx=0x7f0934009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:18.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:18.758+0000 7f092effd700 1 -- 172.21.15.62:0/4049877572 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f09300971f0 con 0x7f0938101680 2023-07-14T16:13:18.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:18.835 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.164:6832/903698978,v1:172.21.15.164:6833/903698978] up:boot 2023-07-14T16:13:18.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: fsmap 1 up:standby 2023-07-14T16:13:18.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T16:13:18.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:18.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:18.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:18.836 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:18.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:18.837 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:18.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:18.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.164:6832/903698978,v1:172.21.15.164:6833/903698978] up:boot 2023-07-14T16:13:18.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: fsmap 1 up:standby 2023-07-14T16:13:18.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T16:13:18.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:18.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:18.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:18.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:18.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:18.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:18.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.011+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f093802c850 con 0x7f0938101680 2023-07-14T16:13:18.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.012+0000 7f092effd700 1 -- 172.21.15.62:0/4049877572 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7f0930003c00 con 0x7f0938101680 2023-07-14T16:13:18.989 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:18.989 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T16:13:18.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f09240a1220 msgr2=0x7f09240a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:18.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f09240a1220 0x7f09240a36d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0938100b50 tx=0x7f0934009260 comp rx=0 tx=0).stop 2023-07-14T16:13:18.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 msgr2=0x7f0938072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 0x7f0938072e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0930009fd0 tx=0x7f093000ba50 comp rx=0 tx=0).stop 2023-07-14T16:13:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 shutdown_connections 2023-07-14T16:13:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0938101680 0x7f0938072e40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f09380ffb00 0x7f09380742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f09240a1220 0x7f09240a36d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 --2- 172.21.15.62:0/4049877572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0938100cf0 0x7f0938072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.014+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 >> 172.21.15.62:0/4049877572 conn(0x7f09380fb0e0 msgr2=0x7f09380fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:18.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.015+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 shutdown_connections 2023-07-14T16:13:18.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:19.015+0000 7f09401ff700 1 -- 172.21.15.62:0/4049877572 wait complete. 2023-07-14T16:13:18.993 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 2 2023-07-14T16:13:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.164:6832/903698978,v1:172.21.15.164:6833/903698978] up:boot 2023-07-14T16:13:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: fsmap 1 up:standby 2023-07-14T16:13:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T16:13:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:19.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:19.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:19.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:19.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:19 smithi081 ceph-mon[127795]: Deploying daemon mds.k on smithi081 2023-07-14T16:13:19.828 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:19 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/4049877572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:19.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:19 smithi062 ceph-mon[95543]: Deploying daemon mds.k on smithi081 2023-07-14T16:13:19.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:19 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/4049877572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:19 smithi164 ceph-mon[97427]: Deploying daemon mds.k on smithi081 2023-07-14T16:13:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:19 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/4049877572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:20.083 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:13:20 smithi081 systemd[1]: Starting Ceph mds.k for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:20.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:20 smithi062 ceph-mon[95543]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:20.897 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:20 smithi081 ceph-mon[127795]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:20.897 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:13:20 smithi081 podman[152169]: 2023-07-14T16:13:21.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:20 smithi164 ceph-mon[97427]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.258+0000 7f0ce03ff700 1 -- 172.21.15.62:0/977145744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd80744d0 msgr2=0x7f0cd8072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.258+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/977145744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd80744d0 0x7f0cd8072920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009990 tx=0x7f0cd4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.259+0000 7f0ce03ff700 1 -- 172.21.15.62:0/977145744 shutdown_connections 2023-07-14T16:13:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.259+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/977145744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd81079b0 0x7f0cd8109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.259+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/977145744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd80744d0 0x7f0cd8072920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.259+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/977145744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cd8072ef0 0x7f0cd8073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.259+0000 7f0ce03ff700 1 -- 172.21.15.62:0/977145744 >> 172.21.15.62:0/977145744 conn(0x7f0cd80fb0e0 msgr2=0x7f0cd80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:21.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.259+0000 7f0ce03ff700 1 -- 172.21.15.62:0/977145744 shutdown_connections 2023-07-14T16:13:21.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.259+0000 7f0ce03ff700 1 -- 172.21.15.62:0/977145744 wait complete. 2023-07-14T16:13:21.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.260+0000 7f0ce03ff700 1 Processor -- start 2023-07-14T16:13:21.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.260+0000 7f0ce03ff700 1 -- start start 2023-07-14T16:13:21.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.260+0000 7f0ce03ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8072ef0 0x7f0cd8197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:21.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.260+0000 7f0ce03ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cd80744d0 0x7f0cd8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:21.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.260+0000 7f0ce03ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd81079b0 0x7f0cd819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:21.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.260+0000 7f0ce03ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd819f480 con 0x7f0cd8072ef0 2023-07-14T16:13:21.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.260+0000 7f0ce03ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd819f5c0 con 0x7f0cd80744d0 2023-07-14T16:13:21.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.260+0000 7f0ce03ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd819f750 con 0x7f0cd81079b0 2023-07-14T16:13:21.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cde19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8072ef0 0x7f0cd8197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:21.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cdd99a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cd80744d0 0x7f0cd8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:21.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cde19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8072ef0 0x7f0cd8197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59010/0 (socket says 172.21.15.62:59010) 2023-07-14T16:13:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cde19b700 1 -- 172.21.15.62:0/2037100509 learned_addr learned my addr 172.21.15.62:0/2037100509 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cde99c700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd81079b0 0x7f0cd819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cdd99a700 1 -- 172.21.15.62:0/2037100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd81079b0 msgr2=0x7f0cd819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cdd99a700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd81079b0 0x7f0cd819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cdd99a700 1 -- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8072ef0 msgr2=0x7f0cd8197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cdd99a700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8072ef0 0x7f0cd8197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cdd99a700 1 -- 172.21.15.62:0/2037100509 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40095f0 con 0x7f0cd80744d0 2023-07-14T16:13:21.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cde19b700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8072ef0 0x7f0cd8197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cde99c700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd81079b0 0x7f0cd819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:13:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.261+0000 7f0cdd99a700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cd80744d0 0x7f0cd8197e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800e9f0 tx=0x7f0cc800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.262+0000 7f0ccf7fe700 1 -- 172.21.15.62:0/2037100509 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc80096d0 con 0x7f0cd80744d0 2023-07-14T16:13:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.262+0000 7f0ccf7fe700 1 -- 172.21.15.62:0/2037100509 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cc800cbd0 con 0x7f0cd80744d0 2023-07-14T16:13:21.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.262+0000 7f0ccf7fe700 1 -- 172.21.15.62:0/2037100509 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8018680 con 0x7f0cd80744d0 2023-07-14T16:13:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.262+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd819fa30 con 0x7f0cd80744d0 2023-07-14T16:13:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.262+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd819fef0 con 0x7f0cd80744d0 2023-07-14T16:13:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.263+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd81060f0 con 0x7f0cd80744d0 2023-07-14T16:13:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.265+0000 7f0ccf7fe700 1 -- 172.21.15.62:0/2037100509 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0cc800cd40 con 0x7f0cd80744d0 2023-07-14T16:13:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.266+0000 7f0ccf7fe700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0cc40a11a0 0x7f0cc40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.266+0000 7f0ccf7fe700 1 -- 172.21.15.62:0/2037100509 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0cc8014070 con 0x7f0cd80744d0 2023-07-14T16:13:21.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.266+0000 7f0cde19b700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0cc40a11a0 0x7f0cc40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:21.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.267+0000 7f0cde19b700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0cc40a11a0 0x7f0cc40a3650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006010 tx=0x7f0cd400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:21.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.270+0000 7f0ccf7fe700 1 -- 172.21.15.62:0/2037100509 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0cc8051790 con 0x7f0cd80744d0 2023-07-14T16:13:21.408 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:13:21 smithi081 bash[152169]: c760b13c1d11ca68fdeafb4c7bee78cb4c8ecc267b8f7b982ce51237a1067c66 2023-07-14T16:13:21.409 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:13:21 smithi081 ceph-mds[152209]: -- 172.21.15.81:0/1866790710 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e6b3ffd1e0 con 0x55e6b3fd2c00 2023-07-14T16:13:21.409 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:13:21 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-k[152185]: starting mds.k at 2023-07-14T16:13:21.409 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:13:21 smithi081 systemd[1]: Started Ceph mds.k for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:21.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.530+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cd802c850 con 0x7f0cd80744d0 2023-07-14T16:13:21.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.531+0000 7f0ccf7fe700 1 -- 172.21.15.62:0/2037100509 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7f0cc8096280 con 0x7f0cd80744d0 2023-07-14T16:13:21.508 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:21.508 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T16:13:21.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0cc40a11a0 msgr2=0x7f0cc40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:21.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0cc40a11a0 0x7f0cc40a3650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006010 tx=0x7f0cd400b260 comp rx=0 tx=0).stop 2023-07-14T16:13:21.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cd80744d0 msgr2=0x7f0cd8197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:21.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cd80744d0 0x7f0cd8197e40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800e9f0 tx=0x7f0cc800ed00 comp rx=0 tx=0).stop 2023-07-14T16:13:21.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 shutdown_connections 2023-07-14T16:13:21.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cd81079b0 0x7f0cd819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd8072ef0 0x7f0cd8197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0cc40a11a0 0x7f0cc40a3650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 --2- 172.21.15.62:0/2037100509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cd80744d0 0x7f0cd8197e40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:21.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 >> 172.21.15.62:0/2037100509 conn(0x7f0cd80fb0e0 msgr2=0x7f0cd80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:21.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 shutdown_connections 2023-07-14T16:13:21.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:21.534+0000 7f0ce03ff700 1 -- 172.21.15.62:0/2037100509 wait complete. 2023-07-14T16:13:21.513 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 2 2023-07-14T16:13:22.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:22.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:22.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:22.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: Deploying daemon mds.j on smithi062 2023-07-14T16:13:22.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:22.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2037100509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:22.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.81:6832/2570105256,v1:172.21.15.81:6833/2570105256] up:boot 2023-07-14T16:13:22.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: fsmap 2 up:standby 2023-07-14T16:13:22.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T16:13:22.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:22.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:22.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:22.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: Deploying daemon mds.j on smithi062 2023-07-14T16:13:22.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:22.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2037100509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:22.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.81:6832/2570105256,v1:172.21.15.81:6833/2570105256] up:boot 2023-07-14T16:13:22.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: fsmap 2 up:standby 2023-07-14T16:13:22.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T16:13:22.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: Deploying daemon mds.j on smithi062 2023-07-14T16:13:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2037100509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:22.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.81:6832/2570105256,v1:172.21.15.81:6833/2570105256] up:boot 2023-07-14T16:13:22.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: fsmap 2 up:standby 2023-07-14T16:13:22.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T16:13:23.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:23.603 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:13:23 smithi062 systemd[1]: Starting Ceph mds.j for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:23.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.774+0000 7f79abbff700 1 -- 172.21.15.62:0/4087365663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 msgr2=0x7f79a40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.774+0000 7f79abbff700 1 --2- 172.21.15.62:0/4087365663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 0x7f79a40fff10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009990 tx=0x7f79a0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.775+0000 7f79abbff700 1 -- 172.21.15.62:0/4087365663 shutdown_connections 2023-07-14T16:13:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.775+0000 7f79abbff700 1 --2- 172.21.15.62:0/4087365663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79a4101680 0x7f79a4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.775+0000 7f79abbff700 1 --2- 172.21.15.62:0/4087365663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 0x7f79a40fff10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.775+0000 7f79abbff700 1 --2- 172.21.15.62:0/4087365663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a4100cf0 0x7f79a4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.775+0000 7f79abbff700 1 -- 172.21.15.62:0/4087365663 >> 172.21.15.62:0/4087365663 conn(0x7f79a40fb0e0 msgr2=0x7f79a40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.775+0000 7f79abbff700 1 -- 172.21.15.62:0/4087365663 shutdown_connections 2023-07-14T16:13:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.775+0000 7f79abbff700 1 -- 172.21.15.62:0/4087365663 wait complete. 2023-07-14T16:13:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.776+0000 7f79abbff700 1 Processor -- start 2023-07-14T16:13:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.776+0000 7f79abbff700 1 -- start start 2023-07-14T16:13:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.776+0000 7f79abbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 0x7f79a4197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.776+0000 7f79abbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a4100cf0 0x7f79a4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79abbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79a4101680 0x7f79a419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79abbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419f500 con 0x7f79a40ffb00 2023-07-14T16:13:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79abbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419f640 con 0x7f79a4100cf0 2023-07-14T16:13:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79abbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419f7d0 con 0x7f79a4101680 2023-07-14T16:13:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 0x7f79a4197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 0x7f79a4197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52586/0 (socket says 172.21.15.62:52586) 2023-07-14T16:13:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a999b700 1 -- 172.21.15.62:0/1724364498 learned_addr learned my addr 172.21.15.62:0/1724364498 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a919a700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a4100cf0 0x7f79a4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a999b700 1 -- 172.21.15.62:0/1724364498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79a4101680 msgr2=0x7f79a419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79aa19c700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79a4101680 0x7f79a419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a999b700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79a4101680 0x7f79a419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a999b700 1 -- 172.21.15.62:0/1724364498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a4100cf0 msgr2=0x7f79a4197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a999b700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a4100cf0 0x7f79a4197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a999b700 1 -- 172.21.15.62:0/1724364498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a00095f0 con 0x7f79a40ffb00 2023-07-14T16:13:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79aa19c700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79a4101680 0x7f79a419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:13:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.777+0000 7f79a919a700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a4100cf0 0x7f79a4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.778+0000 7f79a999b700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 0x7f79a4197980 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f79a0004730 tx=0x7f79a0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:23.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.778+0000 7f799affd700 1 -- 172.21.15.62:0/1724364498 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a001d070 con 0x7f79a40ffb00 2023-07-14T16:13:23.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.778+0000 7f799affd700 1 -- 172.21.15.62:0/1724364498 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f79a0004ca0 con 0x7f79a40ffb00 2023-07-14T16:13:23.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.778+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a419fa50 con 0x7f79a40ffb00 2023-07-14T16:13:23.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.778+0000 7f799affd700 1 -- 172.21.15.62:0/1724364498 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0003af0 con 0x7f79a40ffb00 2023-07-14T16:13:23.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.778+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a419fed0 con 0x7f79a40ffb00 2023-07-14T16:13:23.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.779+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a410b1d0 con 0x7f79a40ffb00 2023-07-14T16:13:23.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.781+0000 7f799affd700 1 -- 172.21.15.62:0/1724364498 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f79a0004810 con 0x7f79a40ffb00 2023-07-14T16:13:23.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.783+0000 7f799affd700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f79900a1220 0x7f79900a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:23.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.783+0000 7f799affd700 1 -- 172.21.15.62:0/1724364498 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f79a00cd440 con 0x7f79a40ffb00 2023-07-14T16:13:23.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.783+0000 7f79a919a700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f79900a1220 0x7f79900a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:23.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.784+0000 7f79a919a700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f79900a1220 0x7f79900a36d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7994005950 tx=0x7f799400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:23.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:23.787+0000 7f799affd700 1 -- 172.21.15.62:0/1724364498 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f79a0097020 con 0x7f79a40ffb00 2023-07-14T16:13:23.855 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:13:23 smithi062 podman[129175]: 2023-07-14T16:13:24.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.063+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79a41a0480 con 0x7f79a40ffb00 2023-07-14T16:13:24.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.063+0000 7f799affd700 1 -- 172.21.15.62:0/1724364498 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2014 (secure 0 0 0) 0x7f79a009a640 con 0x7f79a40ffb00 2023-07-14T16:13:24.040 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:24.041 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T16:13:24.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.066+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f79900a1220 msgr2=0x7f79900a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:24.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.066+0000 7f79abbff700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f79900a1220 0x7f79900a36d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7994005950 tx=0x7f799400b410 comp rx=0 tx=0).stop 2023-07-14T16:13:24.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.066+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 msgr2=0x7f79a4197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:24.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.066+0000 7f79abbff700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 0x7f79a4197980 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f79a0004730 tx=0x7f79a0009580 comp rx=0 tx=0).stop 2023-07-14T16:13:24.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.067+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 shutdown_connections 2023-07-14T16:13:24.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.067+0000 7f79abbff700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79a4101680 0x7f79a419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:24.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.067+0000 7f79abbff700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79a40ffb00 0x7f79a4197980 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:24.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.067+0000 7f79abbff700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f79900a1220 0x7f79900a36d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:24.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.067+0000 7f79abbff700 1 --2- 172.21.15.62:0/1724364498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a4100cf0 0x7f79a4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:24.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.067+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 >> 172.21.15.62:0/1724364498 conn(0x7f79a40fb0e0 msgr2=0x7f79a40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:24.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.067+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 shutdown_connections 2023-07-14T16:13:24.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:24.067+0000 7f79abbff700 1 -- 172.21.15.62:0/1724364498 wait complete. 2023-07-14T16:13:24.046 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 3 2023-07-14T16:13:24.374 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:13:23 smithi062 ceph-mds[129245]: -- 172.21.15.62:0/580449393 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562b1cd431e0 con 0x562b1cd18c00 2023-07-14T16:13:24.374 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:13:23 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-j[129221]: starting mds.j at 2023-07-14T16:13:24.374 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:13:23 smithi062 bash[129175]: 3c59f72942cf5f3fb55e13369ef6e32a319d9589388a1fa072defac7587d196b 2023-07-14T16:13:24.375 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:13:24 smithi062 systemd[1]: Started Ceph mds.j for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:24.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:24 smithi062 ceph-mon[95543]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:24.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:24 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1724364498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:24.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:24.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:24.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:24.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:24.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:24.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:24.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:24 smithi081 ceph-mon[127795]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:24.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:24 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1724364498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:24.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:24.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:24.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:24.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:24.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:24.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:24 smithi164 ceph-mon[97427]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:24 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1724364498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:25.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:25.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:25.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:25.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:25.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:25.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:25.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:25 smithi062 ceph-mon[95543]: Deploying daemon mds.d on smithi062 2023-07-14T16:13:25.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:25 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.62:6834/3850024319,v1:172.21.15.62:6835/3850024319] up:boot 2023-07-14T16:13:25.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:25 smithi062 ceph-mon[95543]: fsmap 3 up:standby 2023-07-14T16:13:25.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T16:13:25.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:25 smithi081 ceph-mon[127795]: Deploying daemon mds.d on smithi062 2023-07-14T16:13:25.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:25 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.62:6834/3850024319,v1:172.21.15.62:6835/3850024319] up:boot 2023-07-14T16:13:25.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:25 smithi081 ceph-mon[127795]: fsmap 3 up:standby 2023-07-14T16:13:25.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T16:13:26.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:25 smithi164 ceph-mon[97427]: Deploying daemon mds.d on smithi062 2023-07-14T16:13:26.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:25 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.62:6834/3850024319,v1:172.21.15.62:6835/3850024319] up:boot 2023-07-14T16:13:26.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:25 smithi164 ceph-mon[97427]: fsmap 3 up:standby 2023-07-14T16:13:26.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T16:13:26.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:26.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.334+0000 7fbd385ff700 1 -- 172.21.15.62:0/2785715270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd30074550 msgr2=0x7fbd300729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:26.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.334+0000 7fbd385ff700 1 --2- 172.21.15.62:0/2785715270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd30074550 0x7fbd300729a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0098f0 tx=0x7fbd2c009c00 comp rx=0 tx=0).stop 2023-07-14T16:13:26.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.334+0000 7fbd385ff700 1 -- 172.21.15.62:0/2785715270 shutdown_connections 2023-07-14T16:13:26.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.334+0000 7fbd385ff700 1 --2- 172.21.15.62:0/2785715270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd301079c0 0x7fbd30109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.334+0000 7fbd385ff700 1 --2- 172.21.15.62:0/2785715270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd30074550 0x7fbd300729a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.334+0000 7fbd385ff700 1 --2- 172.21.15.62:0/2785715270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd30072f70 0x7fbd300733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.334+0000 7fbd385ff700 1 -- 172.21.15.62:0/2785715270 >> 172.21.15.62:0/2785715270 conn(0x7fbd300fb0e0 msgr2=0x7fbd300fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:26.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.334+0000 7fbd385ff700 1 -- 172.21.15.62:0/2785715270 shutdown_connections 2023-07-14T16:13:26.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.335+0000 7fbd385ff700 1 -- 172.21.15.62:0/2785715270 wait complete. 2023-07-14T16:13:26.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.335+0000 7fbd385ff700 1 Processor -- start 2023-07-14T16:13:26.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd385ff700 1 -- start start 2023-07-14T16:13:26.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd385ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd30072f70 0x7fbd301979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:26.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd385ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd30074550 0x7fbd30197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:26.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd385ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd301079c0 0x7fbd3019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:26.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd385ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3019f560 con 0x7fbd30072f70 2023-07-14T16:13:26.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd385ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3019f6a0 con 0x7fbd301079c0 2023-07-14T16:13:26.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd385ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3019f830 con 0x7fbd30074550 2023-07-14T16:13:26.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd3639b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd30072f70 0x7fbd301979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:26.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd3639b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd30072f70 0x7fbd301979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52616/0 (socket says 172.21.15.62:52616) 2023-07-14T16:13:26.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.336+0000 7fbd3639b700 1 -- 172.21.15.62:0/1333079124 learned_addr learned my addr 172.21.15.62:0/1333079124 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:26.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd36b9c700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd301079c0 0x7fbd3019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:26.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd35b9a700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd30074550 0x7fbd30197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:26.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd3639b700 1 -- 172.21.15.62:0/1333079124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd30074550 msgr2=0x7fbd30197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:26.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd3639b700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd30074550 0x7fbd30197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd3639b700 1 -- 172.21.15.62:0/1333079124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd301079c0 msgr2=0x7fbd3019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:26.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd3639b700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd301079c0 0x7fbd3019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd3639b700 1 -- 172.21.15.62:0/1333079124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd28009d00 con 0x7fbd30072f70 2023-07-14T16:13:26.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd3639b700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd30072f70 0x7fbd301979e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009fd0 tx=0x7fbd2c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:26.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd277fe700 1 -- 172.21.15.62:0/1333079124 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c01c070 con 0x7fbd30072f70 2023-07-14T16:13:26.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd277fe700 1 -- 172.21.15.62:0/1333079124 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd2c00f7b0 con 0x7fbd30072f70 2023-07-14T16:13:26.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2c009520 con 0x7fbd30072f70 2023-07-14T16:13:26.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.337+0000 7fbd277fe700 1 -- 172.21.15.62:0/1333079124 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c003970 con 0x7fbd30072f70 2023-07-14T16:13:26.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.338+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3019fda0 con 0x7fbd30072f70 2023-07-14T16:13:26.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.339+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd30106070 con 0x7fbd30072f70 2023-07-14T16:13:26.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.340+0000 7fbd277fe700 1 -- 172.21.15.62:0/1333079124 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbd2c020030 con 0x7fbd30072f70 2023-07-14T16:13:26.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.342+0000 7fbd277fe700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd1c0a1220 0x7fbd1c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:26.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.342+0000 7fbd277fe700 1 -- 172.21.15.62:0/1333079124 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd2c0cd3f0 con 0x7fbd30072f70 2023-07-14T16:13:26.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.342+0000 7fbd35b9a700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd1c0a1220 0x7fbd1c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:26.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.343+0000 7fbd35b9a700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd1c0a1220 0x7fbd1c0a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd30074270 tx=0x7fbd20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:26.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.345+0000 7fbd277fe700 1 -- 172.21.15.62:0/1333079124 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbd2c096fd0 con 0x7fbd30072f70 2023-07-14T16:13:26.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.622+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd3002c850 con 0x7fbd30072f70 2023-07-14T16:13:26.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.622+0000 7fbd277fe700 1 -- 172.21.15.62:0/1333079124 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7fbd2c09a5f0 con 0x7fbd30072f70 2023-07-14T16:13:26.599 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:26.600 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T16:13:26.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd1c0a1220 msgr2=0x7fbd1c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd1c0a1220 0x7fbd1c0a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd30074270 tx=0x7fbd20009260 comp rx=0 tx=0).stop 2023-07-14T16:13:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd30072f70 msgr2=0x7fbd301979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd30072f70 0x7fbd301979e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009fd0 tx=0x7fbd2c00b790 comp rx=0 tx=0).stop 2023-07-14T16:13:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 shutdown_connections 2023-07-14T16:13:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd1c0a1220 0x7fbd1c0a36d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd301079c0 0x7fbd3019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd30072f70 0x7fbd301979e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 --2- 172.21.15.62:0/1333079124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd30074550 0x7fbd30197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:26.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 >> 172.21.15.62:0/1333079124 conn(0x7fbd300fb0e0 msgr2=0x7fbd300fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:26.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 shutdown_connections 2023-07-14T16:13:26.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:26.625+0000 7fbd385ff700 1 -- 172.21.15.62:0/1333079124 wait complete. 2023-07-14T16:13:26.603 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 4 2023-07-14T16:13:26.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:26 smithi062 ceph-mon[95543]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:26.875 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:13:26 smithi062 systemd[1]: Starting Ceph mds.d for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:26 smithi081 ceph-mon[127795]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:27.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:26 smithi164 ceph-mon[97427]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:27.305 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:13:27 smithi062 podman[129592]: 2023-07-14T16:13:27.305 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:13:27 smithi062 bash[129592]: 13729483a565e52cdb4f01d2f4cfb8e2d9817dd0b8640e5746d72fb438028f5e 2023-07-14T16:13:27.306 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-mds[129632]: -- 172.21.15.62:0/2471668950 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ffbf7f1e0 con 0x563ffbf54c00 2023-07-14T16:13:27.306 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-d[129608]: starting mds.d at 2023-07-14T16:13:27.624 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:13:27 smithi062 systemd[1]: Started Ceph mds.d for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:27.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1333079124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:27.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:27.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:27.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:27.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:27.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:27.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:27 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:27.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:27 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1333079124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:27.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:27.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:27.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:27.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:27.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:27.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:27 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:28.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:27 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1333079124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:28.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:27 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:28.624 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:28.748 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:28 smithi081 ceph-mon[127795]: Deploying daemon mds.e on smithi081 2023-07-14T16:13:28.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:28 smithi081 ceph-mon[127795]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:28.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:28 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.62:6836/1601453648,v1:172.21.15.62:6837/1601453648] up:boot 2023-07-14T16:13:28.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:28 smithi081 ceph-mon[127795]: fsmap 4 up:standby 2023-07-14T16:13:28.749 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:28 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T16:13:28.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 -- 172.21.15.62:0/4211539853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b98100cf0 msgr2=0x7f9b98101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:28.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/4211539853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b98100cf0 0x7f9b98101140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009990 tx=0x7f9b88009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:28.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 -- 172.21.15.62:0/4211539853 shutdown_connections 2023-07-14T16:13:28.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/4211539853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b98101680 0x7f9b98107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:28.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/4211539853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b980ffb00 0x7f9b980fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:28.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/4211539853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b98100cf0 0x7f9b98101140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:28.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 -- 172.21.15.62:0/4211539853 >> 172.21.15.62:0/4211539853 conn(0x7f9b980fb0e0 msgr2=0x7f9b980fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:28.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 -- 172.21.15.62:0/4211539853 shutdown_connections 2023-07-14T16:13:28.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.866+0000 7f9b9ffff700 1 -- 172.21.15.62:0/4211539853 wait complete. 2023-07-14T16:13:28.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.867+0000 7f9b9ffff700 1 Processor -- start 2023-07-14T16:13:28.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.867+0000 7f9b9ffff700 1 -- start start 2023-07-14T16:13:28.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.867+0000 7f9b9ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b980ffb00 0x7f9b980742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:28.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.867+0000 7f9b9ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b98100cf0 0x7f9b98072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:28.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.867+0000 7f9b9ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b98101680 0x7f9b98072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:28.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.867+0000 7f9b9ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980735e0 con 0x7f9b980ffb00 2023-07-14T16:13:28.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.867+0000 7f9b9ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98073750 con 0x7f9b98100cf0 2023-07-14T16:13:28.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9ffff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9819f7e0 con 0x7f9b98101680 2023-07-14T16:13:28.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b980ffb00 0x7f9b980742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:28.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b980ffb00 0x7f9b980742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52668/0 (socket says 172.21.15.62:52668) 2023-07-14T16:13:28.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 -- 172.21.15.62:0/588519237 learned_addr learned my addr 172.21.15.62:0/588519237 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:28.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9d59a700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b98100cf0 0x7f9b98072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:28.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9e59c700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b98101680 0x7f9b98072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:28.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 -- 172.21.15.62:0/588519237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b98101680 msgr2=0x7f9b98072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:28.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b98101680 0x7f9b98072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:28.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 -- 172.21.15.62:0/588519237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b98100cf0 msgr2=0x7f9b98072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:28.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b98100cf0 0x7f9b98072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:28.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 -- 172.21.15.62:0/588519237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880095f0 con 0x7f9b980ffb00 2023-07-14T16:13:28.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9d59a700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b98100cf0 0x7f9b98072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:13:28.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b9dd9b700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b980ffb00 0x7f9b980742c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400e970 tx=0x7f9b9400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:28.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b8effd700 1 -- 172.21.15.62:0/588519237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94009790 con 0x7f9b980ffb00 2023-07-14T16:13:28.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b8effd700 1 -- 172.21.15.62:0/588519237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b940044b0 con 0x7f9b980ffb00 2023-07-14T16:13:28.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.868+0000 7f9b8effd700 1 -- 172.21.15.62:0/588519237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94003dd0 con 0x7f9b980ffb00 2023-07-14T16:13:28.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.869+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9819fac0 con 0x7f9b980ffb00 2023-07-14T16:13:28.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.869+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9819ff80 con 0x7f9b980ffb00 2023-07-14T16:13:28.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.870+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9810b1d0 con 0x7f9b980ffb00 2023-07-14T16:13:28.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.872+0000 7f9b8effd700 1 -- 172.21.15.62:0/588519237 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9b94003f30 con 0x7f9b980ffb00 2023-07-14T16:13:28.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.873+0000 7f9b8effd700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b840a11a0 0x7f9b840a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:28.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.873+0000 7f9b8effd700 1 -- 172.21.15.62:0/588519237 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9b94014070 con 0x7f9b980ffb00 2023-07-14T16:13:28.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.873+0000 7f9b9d59a700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b840a11a0 0x7f9b840a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:28.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.874+0000 7f9b9d59a700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b840a11a0 0x7f9b840a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9b88006010 tx=0x7f9b8800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:28.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:28.877+0000 7f9b8effd700 1 -- 172.21.15.62:0/588519237 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9b940963a0 con 0x7f9b980ffb00 2023-07-14T16:13:28.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:28 smithi062 ceph-mon[95543]: Deploying daemon mds.e on smithi081 2023-07-14T16:13:28.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:28 smithi062 ceph-mon[95543]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:28.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:28 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.62:6836/1601453648,v1:172.21.15.62:6837/1601453648] up:boot 2023-07-14T16:13:28.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:28 smithi062 ceph-mon[95543]: fsmap 4 up:standby 2023-07-14T16:13:28.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:28 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T16:13:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:28 smithi164 ceph-mon[97427]: Deploying daemon mds.e on smithi081 2023-07-14T16:13:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:28 smithi164 ceph-mon[97427]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:28 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.62:6836/1601453648,v1:172.21.15.62:6837/1601453648] up:boot 2023-07-14T16:13:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:28 smithi164 ceph-mon[97427]: fsmap 4 up:standby 2023-07-14T16:13:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:28 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T16:13:29.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.134+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b98106a30 con 0x7f9b980ffb00 2023-07-14T16:13:29.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.134+0000 7f9b8effd700 1 -- 172.21.15.62:0/588519237 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7f9b940999c0 con 0x7f9b980ffb00 2023-07-14T16:13:29.111 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:29.112 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T16:13:29.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.136+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b840a11a0 msgr2=0x7f9b840a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:29.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.136+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b840a11a0 0x7f9b840a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9b88006010 tx=0x7f9b8800b260 comp rx=0 tx=0).stop 2023-07-14T16:13:29.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.136+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b980ffb00 msgr2=0x7f9b980742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:29.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.136+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b980ffb00 0x7f9b980742c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400e970 tx=0x7f9b9400ed30 comp rx=0 tx=0).stop 2023-07-14T16:13:29.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.137+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 shutdown_connections 2023-07-14T16:13:29.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.137+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b98101680 0x7f9b98072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.137+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b980ffb00 0x7f9b980742c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.137+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b840a11a0 0x7f9b840a3650 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.137+0000 7f9b9ffff700 1 --2- 172.21.15.62:0/588519237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b98100cf0 0x7f9b98072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.137+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 >> 172.21.15.62:0/588519237 conn(0x7f9b980fb0e0 msgr2=0x7f9b980fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.137+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 shutdown_connections 2023-07-14T16:13:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:29.137+0000 7f9b9ffff700 1 -- 172.21.15.62:0/588519237 wait complete. 2023-07-14T16:13:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 5 2023-07-14T16:13:29.750 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:29 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/588519237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:29.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:29 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/588519237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:30.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:29 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/588519237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:30.153 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:13:29 smithi081 systemd[1]: Starting Ceph mds.e for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:30.470 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:13:30 smithi081 podman[152523]: 2023-07-14T16:13:30.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:30 smithi062 ceph-mon[95543]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:30.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:30 smithi081 ceph-mon[127795]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:30.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:13:30 smithi081 bash[152523]: a6ec5512115b4bd4de6ffe41d500ad2f67075e74132ce6aeaa03e36c255d78ac 2023-07-14T16:13:30.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:13:30 smithi081 ceph-mds[152563]: -- 172.21.15.81:0/3422713232 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c1dae0f1e0 con 0x55c1dade4800 2023-07-14T16:13:30.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:13:30 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: starting mds.e at 2023-07-14T16:13:30.904 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:13:30 smithi081 systemd[1]: Started Ceph mds.e for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:31.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:30 smithi164 ceph-mon[97427]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:31.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:31.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 -- 172.21.15.62:0/3566306375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2468101680 msgr2=0x7f2468107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:31.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 --2- 172.21.15.62:0/3566306375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2468101680 0x7f2468107f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2460009f90 tx=0x7f246000b330 comp rx=0 tx=0).stop 2023-07-14T16:13:31.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 -- 172.21.15.62:0/3566306375 shutdown_connections 2023-07-14T16:13:31.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 --2- 172.21.15.62:0/3566306375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2468101680 0x7f2468107f40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 --2- 172.21.15.62:0/3566306375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24680ffb00 0x7f24680fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 --2- 172.21.15.62:0/3566306375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2468100cf0 0x7f2468101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 -- 172.21.15.62:0/3566306375 >> 172.21.15.62:0/3566306375 conn(0x7f24680fb0e0 msgr2=0x7f24680fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:31.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 -- 172.21.15.62:0/3566306375 shutdown_connections 2023-07-14T16:13:31.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.372+0000 7f246f7ff700 1 -- 172.21.15.62:0/3566306375 wait complete. 2023-07-14T16:13:31.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.373+0000 7f246f7ff700 1 Processor -- start 2023-07-14T16:13:31.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246f7ff700 1 -- start start 2023-07-14T16:13:31.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246f7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24680ffb00 0x7f2468195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:31.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246f7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2468100cf0 0x7f2468195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:31.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246f7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2468101680 0x7f246819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:31.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246f7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246819d3d0 con 0x7f24680ffb00 2023-07-14T16:13:31.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246f7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246819d510 con 0x7f2468100cf0 2023-07-14T16:13:31.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246f7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246819d6a0 con 0x7f2468101680 2023-07-14T16:13:31.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24680ffb00 0x7f2468195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:31.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24680ffb00 0x7f2468195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52694/0 (socket says 172.21.15.62:52694) 2023-07-14T16:13:31.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.374+0000 7f246d59b700 1 -- 172.21.15.62:0/512902287 learned_addr learned my addr 172.21.15.62:0/512902287 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:31.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246cd9a700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2468100cf0 0x7f2468195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:31.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246d59b700 1 -- 172.21.15.62:0/512902287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2468101680 msgr2=0x7f246819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246d59b700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2468101680 0x7f246819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246d59b700 1 -- 172.21.15.62:0/512902287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2468100cf0 msgr2=0x7f2468195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246d59b700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2468100cf0 0x7f2468195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246d59b700 1 -- 172.21.15.62:0/512902287 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24640095f0 con 0x7f24680ffb00 2023-07-14T16:13:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246d59b700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24680ffb00 0x7f2468195850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f246400ebf0 tx=0x7f246400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f245e7fc700 1 -- 172.21.15.62:0/512902287 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464009a10 con 0x7f24680ffb00 2023-07-14T16:13:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f245e7fc700 1 -- 172.21.15.62:0/512902287 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f246400ce10 con 0x7f24680ffb00 2023-07-14T16:13:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2460009c40 con 0x7f24680ffb00 2023-07-14T16:13:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f245e7fc700 1 -- 172.21.15.62:0/512902287 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24640187c0 con 0x7f24680ffb00 2023-07-14T16:13:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.375+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246819dd00 con 0x7f24680ffb00 2023-07-14T16:13:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.376+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2468106960 con 0x7f24680ffb00 2023-07-14T16:13:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.379+0000 7f245e7fc700 1 -- 172.21.15.62:0/512902287 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2464010430 con 0x7f24680ffb00 2023-07-14T16:13:31.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.380+0000 7f245e7fc700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24540a11a0 0x7f24540a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:31.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.380+0000 7f245e7fc700 1 -- 172.21.15.62:0/512902287 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2464014070 con 0x7f24680ffb00 2023-07-14T16:13:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.380+0000 7f246cd9a700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24540a11a0 0x7f24540a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.381+0000 7f246cd9a700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24540a11a0 0x7f24540a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2458009bf0 tx=0x7f2458009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:31.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.382+0000 7f245e7fc700 1 -- 172.21.15.62:0/512902287 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2464096d10 con 0x7f24680ffb00 2023-07-14T16:13:31.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.641+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f246802c850 con 0x7f24680ffb00 2023-07-14T16:13:31.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.642+0000 7f245e7fc700 1 -- 172.21.15.62:0/512902287 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4282 (secure 0 0 0) 0x7f246409a330 con 0x7f24680ffb00 2023-07-14T16:13:31.619 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:31.619 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-14T16:13:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24540a11a0 msgr2=0x7f24540a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24540a11a0 0x7f24540a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2458009bf0 tx=0x7f2458009260 comp rx=0 tx=0).stop 2023-07-14T16:13:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24680ffb00 msgr2=0x7f2468195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24680ffb00 0x7f2468195850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f246400ebf0 tx=0x7f246400c2d0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 shutdown_connections 2023-07-14T16:13:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2468101680 0x7f246819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24680ffb00 0x7f2468195850 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24540a11a0 0x7f24540a3650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 --2- 172.21.15.62:0/512902287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2468100cf0 0x7f2468195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 >> 172.21.15.62:0/512902287 conn(0x7f24680fb0e0 msgr2=0x7f24680fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 shutdown_connections 2023-07-14T16:13:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:31.645+0000 7f246f7ff700 1 -- 172.21.15.62:0/512902287 wait complete. 2023-07-14T16:13:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 6 2023-07-14T16:13:31.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:boot 2023-07-14T16:13:31.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: fsmap 5 up:standby 2023-07-14T16:13:31.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T16:13:31.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:31.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:31.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:31.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:31 smithi062 ceph-mon[95543]: Deploying daemon mds.f on smithi164 2023-07-14T16:13:31.900 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:boot 2023-07-14T16:13:31.900 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: fsmap 5 up:standby 2023-07-14T16:13:31.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T16:13:31.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:31.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:31.901 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:31.902 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:31 smithi164 ceph-mon[97427]: Deploying daemon mds.f on smithi164 2023-07-14T16:13:31.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:boot 2023-07-14T16:13:31.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: fsmap 5 up:standby 2023-07-14T16:13:31.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T16:13:31.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:31.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:31.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:31.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:31.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:31 smithi081 ceph-mon[127795]: Deploying daemon mds.f on smithi164 2023-07-14T16:13:32.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:32 smithi062 ceph-mon[95543]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:32.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:32 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/512902287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:32.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:32 smithi081 ceph-mon[127795]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:32.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:32 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/512902287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:32.981 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:32 smithi164 ceph-mon[97427]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:32.981 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:32 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/512902287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:33.276 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:13:33 smithi164 systemd[1]: Starting Ceph mds.f for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:33.641 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:33.756 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:13:33 smithi164 podman[123195]: 2023-07-14T16:13:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.880+0000 7f6482bff700 1 -- 172.21.15.62:0/2558740545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 msgr2=0x7f647c0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.880+0000 7f6482bff700 1 --2- 172.21.15.62:0/2558740545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 0x7f647c0729a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6470009990 tx=0x7f6470009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.880+0000 7f6482bff700 1 -- 172.21.15.62:0/2558740545 shutdown_connections 2023-07-14T16:13:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.880+0000 7f6482bff700 1 --2- 172.21.15.62:0/2558740545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f647c1079c0 0x7f647c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.880+0000 7f6482bff700 1 --2- 172.21.15.62:0/2558740545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 0x7f647c0729a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.880+0000 7f6482bff700 1 --2- 172.21.15.62:0/2558740545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f647c072f70 0x7f647c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.880+0000 7f6482bff700 1 -- 172.21.15.62:0/2558740545 >> 172.21.15.62:0/2558740545 conn(0x7f647c0fb0e0 msgr2=0x7f647c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.881+0000 7f6482bff700 1 -- 172.21.15.62:0/2558740545 shutdown_connections 2023-07-14T16:13:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.881+0000 7f6482bff700 1 -- 172.21.15.62:0/2558740545 wait complete. 2023-07-14T16:13:33.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f6482bff700 1 Processor -- start 2023-07-14T16:13:33.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f6482bff700 1 -- start start 2023-07-14T16:13:33.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f6482bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f647c072f70 0x7f647c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:33.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f6482bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 0x7f647c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:33.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f6482bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f647c1079c0 0x7f647c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:33.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f6482bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c19f530 con 0x7f647c074550 2023-07-14T16:13:33.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f6482bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c19f670 con 0x7f647c072f70 2023-07-14T16:13:33.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f6482bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c19f800 con 0x7f647c1079c0 2023-07-14T16:13:33.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f647bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 0x7f647c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:33.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f647bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 0x7f647c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45488/0 (socket says 172.21.15.62:45488) 2023-07-14T16:13:33.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.882+0000 7f647bfff700 1 -- 172.21.15.62:0/3401914935 learned_addr learned my addr 172.21.15.62:0/3401914935 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:33.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f648119c700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f647c1079c0 0x7f647c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:33.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f648099b700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f647c072f70 0x7f647c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:33.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f647bfff700 1 -- 172.21.15.62:0/3401914935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f647c1079c0 msgr2=0x7f647c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:33.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f647bfff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f647c1079c0 0x7f647c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:33.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f647bfff700 1 -- 172.21.15.62:0/3401914935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f647c072f70 msgr2=0x7f647c1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:33.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f647bfff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f647c072f70 0x7f647c1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:33.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f647bfff700 1 -- 172.21.15.62:0/3401914935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64700095f0 con 0x7f647c074550 2023-07-14T16:13:33.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f648099b700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f647c072f70 0x7f647c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:13:33.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f647bfff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 0x7f647c197ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f646800d980 tx=0x7f646800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:33.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f6479ffb700 1 -- 172.21.15.62:0/3401914935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646800b950 con 0x7f647c074550 2023-07-14T16:13:33.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f6479ffb700 1 -- 172.21.15.62:0/3401914935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6468007620 con 0x7f647c074550 2023-07-14T16:13:33.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f6479ffb700 1 -- 172.21.15.62:0/3401914935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646800f3c0 con 0x7f647c074550 2023-07-14T16:13:33.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.883+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c19fae0 con 0x7f647c074550 2023-07-14T16:13:33.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.884+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c19ffa0 con 0x7f647c074550 2023-07-14T16:13:33.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.885+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c106070 con 0x7f647c074550 2023-07-14T16:13:33.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.886+0000 7f6479ffb700 1 -- 172.21.15.62:0/3401914935 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f646800bab0 con 0x7f647c074550 2023-07-14T16:13:33.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.888+0000 7f6479ffb700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f646c0a1220 0x7f646c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:33.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.888+0000 7f6479ffb700 1 -- 172.21.15.62:0/3401914935 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f64680cc6e0 con 0x7f647c074550 2023-07-14T16:13:33.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.888+0000 7f648099b700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f646c0a1220 0x7f646c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:33.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.889+0000 7f648099b700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f646c0a1220 0x7f646c0a36d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f64700098e0 tx=0x7f6470005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:33.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:33.892+0000 7f6479ffb700 1 -- 172.21.15.62:0/3401914935 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6468096230 con 0x7f647c074550 2023-07-14T16:13:34.025 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:13:33 smithi164 bash[123195]: ff33534edad2a6f4d3ec89c4ebd75a07bbf9baa675c46e2b676051a99a9a7e8a 2023-07-14T16:13:34.026 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:13:33 smithi164 ceph-mds[123235]: -- 172.21.15.164:0/3479613725 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56415fd251e0 con 0x56415fcfa800 2023-07-14T16:13:34.026 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:13:33 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-f[123211]: starting mds.f at 2023-07-14T16:13:34.026 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:13:33 smithi164 systemd[1]: Started Ceph mds.f for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:34.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.145+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f647c02cce0 con 0x7f647c074550 2023-07-14T16:13:34.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.145+0000 7f6479ffb700 1 -- 172.21.15.62:0/3401914935 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4282 (secure 0 0 0) 0x7f6468099850 con 0x7f647c074550 2023-07-14T16:13:34.122 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:34.122 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-14T16:13:34.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.147+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f646c0a1220 msgr2=0x7f646c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:34.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.147+0000 7f6482bff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f646c0a1220 0x7f646c0a36d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f64700098e0 tx=0x7f6470005fd0 comp rx=0 tx=0).stop 2023-07-14T16:13:34.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.147+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 msgr2=0x7f647c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:34.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.147+0000 7f6482bff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 0x7f647c197ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f646800d980 tx=0x7f646800dc90 comp rx=0 tx=0).stop 2023-07-14T16:13:34.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.148+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 shutdown_connections 2023-07-14T16:13:34.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.148+0000 7f6482bff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f647c1079c0 0x7f647c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:34.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.148+0000 7f6482bff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f646c0a1220 0x7f646c0a36d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:34.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.148+0000 7f6482bff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f647c072f70 0x7f647c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:34.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.148+0000 7f6482bff700 1 --2- 172.21.15.62:0/3401914935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c074550 0x7f647c197ef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:34.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.148+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 >> 172.21.15.62:0/3401914935 conn(0x7f647c0fb0e0 msgr2=0x7f647c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:34.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.148+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 shutdown_connections 2023-07-14T16:13:34.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:34.148+0000 7f6482bff700 1 -- 172.21.15.62:0/3401914935 wait complete. 2023-07-14T16:13:34.126 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 6 2023-07-14T16:13:34.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:34 smithi062 ceph-mon[95543]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:34.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:34.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:34.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:34.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:34 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3401914935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:34.892 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:34 smithi164 ceph-mon[97427]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:34.893 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.893 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.893 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.893 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:34.893 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:34.893 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:34.893 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:34 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3401914935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:34.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:34 smithi081 ceph-mon[127795]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:34.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:34.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:34.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:34.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:34.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:34 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3401914935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:35.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:35 smithi081 ceph-mon[127795]: Deploying daemon mds.c on smithi164 2023-07-14T16:13:35.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:35 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.164:6834/1558258971,v1:172.21.15.164:6835/1558258971] up:boot 2023-07-14T16:13:35.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:35 smithi081 ceph-mon[127795]: fsmap 6 up:standby 2023-07-14T16:13:35.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:35 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T16:13:35.993 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:35 smithi164 ceph-mon[97427]: Deploying daemon mds.c on smithi164 2023-07-14T16:13:35.993 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:35 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.164:6834/1558258971,v1:172.21.15.164:6835/1558258971] up:boot 2023-07-14T16:13:35.993 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:35 smithi164 ceph-mon[97427]: fsmap 6 up:standby 2023-07-14T16:13:35.993 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:35 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T16:13:36.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:35 smithi062 ceph-mon[95543]: Deploying daemon mds.c on smithi164 2023-07-14T16:13:36.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:35 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.164:6834/1558258971,v1:172.21.15.164:6835/1558258971] up:boot 2023-07-14T16:13:36.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:35 smithi062 ceph-mon[95543]: fsmap 6 up:standby 2023-07-14T16:13:36.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:35 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T16:13:36.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:36.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 -- 172.21.15.62:0/2390046414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a900ffb00 msgr2=0x7f5a900fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:36.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2390046414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a900ffb00 0x7f5a900fff10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0098f0 tx=0x7f5a8c009c00 comp rx=0 tx=0).stop 2023-07-14T16:13:36.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 -- 172.21.15.62:0/2390046414 shutdown_connections 2023-07-14T16:13:36.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2390046414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a90101680 0x7f5a90107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2390046414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a900ffb00 0x7f5a900fff10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2390046414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a90100cf0 0x7f5a90101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 -- 172.21.15.62:0/2390046414 >> 172.21.15.62:0/2390046414 conn(0x7f5a900fb0e0 msgr2=0x7f5a900fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:36.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 -- 172.21.15.62:0/2390046414 shutdown_connections 2023-07-14T16:13:36.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.381+0000 7f5a979ff700 1 -- 172.21.15.62:0/2390046414 wait complete. 2023-07-14T16:13:36.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.383+0000 7f5a979ff700 1 Processor -- start 2023-07-14T16:13:36.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.383+0000 7f5a979ff700 1 -- start start 2023-07-14T16:13:36.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a979ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a900ffb00 0x7f5a900742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:36.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a979ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a90100cf0 0x7f5a90072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:36.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a979ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a90101680 0x7f5a90072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:36.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a979ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a900735e0 con 0x7f5a90100cf0 2023-07-14T16:13:36.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a979ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90073750 con 0x7f5a900ffb00 2023-07-14T16:13:36.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a979ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9019f7e0 con 0x7f5a90101680 2023-07-14T16:13:36.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a94f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a90100cf0 0x7f5a90072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:36.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a94f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a90100cf0 0x7f5a90072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45510/0 (socket says 172.21.15.62:45510) 2023-07-14T16:13:36.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a94f9a700 1 -- 172.21.15.62:0/2529567891 learned_addr learned my addr 172.21.15.62:0/2529567891 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:36.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a9579b700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a900ffb00 0x7f5a900742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:36.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a95f9c700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a90101680 0x7f5a90072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:36.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a94f9a700 1 -- 172.21.15.62:0/2529567891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a90101680 msgr2=0x7f5a90072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:36.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a94f9a700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a90101680 0x7f5a90072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a94f9a700 1 -- 172.21.15.62:0/2529567891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a900ffb00 msgr2=0x7f5a900742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:36.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a94f9a700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a900ffb00 0x7f5a900742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.384+0000 7f5a94f9a700 1 -- 172.21.15.62:0/2529567891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c009520 con 0x7f5a90100cf0 2023-07-14T16:13:36.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.385+0000 7f5a94f9a700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a90100cf0 0x7f5a90072900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000e9a0 tx=0x7f5a8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:36.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.385+0000 7f5a867fc700 1 -- 172.21.15.62:0/2529567891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80009790 con 0x7f5a90100cf0 2023-07-14T16:13:36.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.385+0000 7f5a867fc700 1 -- 172.21.15.62:0/2529567891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a80010430 con 0x7f5a90100cf0 2023-07-14T16:13:36.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.385+0000 7f5a867fc700 1 -- 172.21.15.62:0/2529567891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000cea0 con 0x7f5a90100cf0 2023-07-14T16:13:36.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.385+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9019fac0 con 0x7f5a90100cf0 2023-07-14T16:13:36.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.385+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9019ff80 con 0x7f5a90100cf0 2023-07-14T16:13:36.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.386+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a90107120 con 0x7f5a90100cf0 2023-07-14T16:13:36.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.388+0000 7f5a867fc700 1 -- 172.21.15.62:0/2529567891 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5a800105a0 con 0x7f5a90100cf0 2023-07-14T16:13:36.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.389+0000 7f5a867fc700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a7c0a11a0 0x7f5a7c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:36.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.389+0000 7f5a867fc700 1 -- 172.21.15.62:0/2529567891 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5a80014070 con 0x7f5a90100cf0 2023-07-14T16:13:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.389+0000 7f5a9579b700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a7c0a11a0 0x7f5a7c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.390+0000 7f5a9579b700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a7c0a11a0 0x7f5a7c0a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009fd0 tx=0x7f5a8c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.393+0000 7f5a867fc700 1 -- 172.21.15.62:0/2529567891 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5a80051790 con 0x7f5a90100cf0 2023-07-14T16:13:36.527 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:13:36 smithi164 systemd[1]: Starting Ceph mds.c for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:36.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.651+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a9002c850 con 0x7f5a90100cf0 2023-07-14T16:13:36.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.652+0000 7f5a867fc700 1 -- 172.21.15.62:0/2529567891 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5042 (secure 0 0 0) 0x7f5a800962b0 con 0x7f5a90100cf0 2023-07-14T16:13:36.630 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:36.630 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-07-14T16:13:36.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.654+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a7c0a11a0 msgr2=0x7f5a7c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:36.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.654+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a7c0a11a0 0x7f5a7c0a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009fd0 tx=0x7f5a8c019040 comp rx=0 tx=0).stop 2023-07-14T16:13:36.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a90100cf0 msgr2=0x7f5a90072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:36.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a90100cf0 0x7f5a90072900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000e9a0 tx=0x7f5a8000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 shutdown_connections 2023-07-14T16:13:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a90101680 0x7f5a90072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a7c0a11a0 0x7f5a7c0a3650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a900ffb00 0x7f5a900742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 --2- 172.21.15.62:0/2529567891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a90100cf0 0x7f5a90072900 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 >> 172.21.15.62:0/2529567891 conn(0x7f5a900fb0e0 msgr2=0x7f5a90104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 shutdown_connections 2023-07-14T16:13:36.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:36.655+0000 7f5a979ff700 1 -- 172.21.15.62:0/2529567891 wait complete. 2023-07-14T16:13:36.634 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2023-07-14T16:13:36.898 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:13:36 smithi164 podman[123540]: 2023-07-14T16:13:36.900 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:36 smithi164 ceph-mon[97427]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:36.900 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:36 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2529567891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:36.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:36 smithi081 ceph-mon[127795]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:36.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:36 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2529567891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:37.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:36 smithi062 ceph-mon[95543]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:37.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:36 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2529567891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:37.275 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:13:37 smithi164 ceph-mds[123580]: -- 172.21.15.164:0/637564134 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e9c44411e0 con 0x55e9c4416800 2023-07-14T16:13:37.275 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:13:37 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-c[123556]: starting mds.c at 2023-07-14T16:13:37.276 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:13:37 smithi164 bash[123540]: 0f64132aaa49117244d837cbef919e112742a5f37c6758f297201721e2470309 2023-07-14T16:13:37.276 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:13:37 smithi164 systemd[1]: Started Ceph mds.c for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:38.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:38.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:38.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:38.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: Deploying daemon mds.h on smithi081 2023-07-14T16:13:38.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:38.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.164:6836/194603776,v1:172.21.15.164:6837/194603776] up:boot 2023-07-14T16:13:38.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: fsmap 7 up:standby 2023-07-14T16:13:38.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T16:13:38.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:38.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:38.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:38.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: Deploying daemon mds.h on smithi081 2023-07-14T16:13:38.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:38.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.164:6836/194603776,v1:172.21.15.164:6837/194603776] up:boot 2023-07-14T16:13:38.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: fsmap 7 up:standby 2023-07-14T16:13:38.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T16:13:38.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:38.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:38.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:38.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:38.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: Deploying daemon mds.h on smithi081 2023-07-14T16:13:38.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:38.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.164:6836/194603776,v1:172.21.15.164:6837/194603776] up:boot 2023-07-14T16:13:38.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: fsmap 7 up:standby 2023-07-14T16:13:38.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T16:13:38.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:38.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.891+0000 7fe46f9ff700 1 -- 172.21.15.62:0/4144829199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe468102e70 msgr2=0x7fe468105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.891+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/4144829199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe468102e70 0x7fe468105250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe458009990 tx=0x7fe458009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.891+0000 7fe46f9ff700 1 -- 172.21.15.62:0/4144829199 shutdown_connections 2023-07-14T16:13:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.891+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/4144829199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe468105790 0x7fe468107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.891+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/4144829199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe468100550 0x7fe468102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.891+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/4144829199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe468102e70 0x7fe468105250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.891+0000 7fe46f9ff700 1 -- 172.21.15.62:0/4144829199 >> 172.21.15.62:0/4144829199 conn(0x7fe4680f9fd0 msgr2=0x7fe4680fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:38.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.891+0000 7fe46f9ff700 1 -- 172.21.15.62:0/4144829199 shutdown_connections 2023-07-14T16:13:38.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.892+0000 7fe46f9ff700 1 -- 172.21.15.62:0/4144829199 wait complete. 2023-07-14T16:13:38.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.892+0000 7fe46f9ff700 1 Processor -- start 2023-07-14T16:13:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46f9ff700 1 -- start start 2023-07-14T16:13:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46f9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe468100550 0x7fe468197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46f9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe468102e70 0x7fe468197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46f9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe468105790 0x7fe46819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46f9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46819f4d0 con 0x7fe468102e70 2023-07-14T16:13:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46f9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46819f610 con 0x7fe468105790 2023-07-14T16:13:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46f9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46819f7a0 con 0x7fe468100550 2023-07-14T16:13:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe468102e70 0x7fe468197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe468102e70 0x7fe468197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45518/0 (socket says 172.21.15.62:45518) 2023-07-14T16:13:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46cf9a700 1 -- 172.21.15.62:0/567131835 learned_addr learned my addr 172.21.15.62:0/567131835 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46df9c700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe468105790 0x7fe46819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.893+0000 7fe46d79b700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe468100550 0x7fe468197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:38.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe46cf9a700 1 -- 172.21.15.62:0/567131835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe468100550 msgr2=0x7fe468197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe46cf9a700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe468100550 0x7fe468197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe46cf9a700 1 -- 172.21.15.62:0/567131835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe468105790 msgr2=0x7fe46819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe46cf9a700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe468105790 0x7fe46819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe46cf9a700 1 -- 172.21.15.62:0/567131835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe460009c40 con 0x7fe468102e70 2023-07-14T16:13:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe46cf9a700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe468102e70 0x7fe468197e90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4580098e0 tx=0x7fe458004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe45e7fc700 1 -- 172.21.15.62:0/567131835 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45801d070 con 0x7fe468102e70 2023-07-14T16:13:38.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4580095f0 con 0x7fe468102e70 2023-07-14T16:13:38.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe45e7fc700 1 -- 172.21.15.62:0/567131835 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe45800be30 con 0x7fe468102e70 2023-07-14T16:13:38.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe45e7fc700 1 -- 172.21.15.62:0/567131835 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458022d90 con 0x7fe468102e70 2023-07-14T16:13:38.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.894+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46819fd10 con 0x7fe468102e70 2023-07-14T16:13:38.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.895+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe468107350 con 0x7fe468102e70 2023-07-14T16:13:38.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.897+0000 7fe45e7fc700 1 -- 172.21.15.62:0/567131835 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe45800b9a0 con 0x7fe468102e70 2023-07-14T16:13:38.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.899+0000 7fe45e7fc700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4540a11a0 0x7fe4540a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:38.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.899+0000 7fe45e7fc700 1 -- 172.21.15.62:0/567131835 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4580ce1e0 con 0x7fe468102e70 2023-07-14T16:13:38.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.899+0000 7fe46d79b700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4540a11a0 0x7fe4540a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:38.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.899+0000 7fe46d79b700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4540a11a0 0x7fe4540a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe464005950 tx=0x7fe46400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:38.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:38.901+0000 7fe45e7fc700 1 -- 172.21.15.62:0/567131835 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe458097dc0 con 0x7fe468102e70 2023-07-14T16:13:39.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.153+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe46802c850 con 0x7fe468102e70 2023-07-14T16:13:39.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.154+0000 7fe45e7fc700 1 -- 172.21.15.62:0/567131835 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5799 (secure 0 0 0) 0x7fe45809b3e0 con 0x7fe468102e70 2023-07-14T16:13:39.131 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:39.131 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-07-14T16:13:39.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4540a11a0 msgr2=0x7fe4540a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:39.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4540a11a0 0x7fe4540a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe464005950 tx=0x7fe46400b410 comp rx=0 tx=0).stop 2023-07-14T16:13:39.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe468102e70 msgr2=0x7fe468197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:39.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe468102e70 0x7fe468197e90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4580098e0 tx=0x7fe458004730 comp rx=0 tx=0).stop 2023-07-14T16:13:39.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 shutdown_connections 2023-07-14T16:13:39.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4540a11a0 0x7fe4540a3650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:39.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe468105790 0x7fe46819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:39.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe468100550 0x7fe468197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:39.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 --2- 172.21.15.62:0/567131835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe468102e70 0x7fe468197e90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:39.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 >> 172.21.15.62:0/567131835 conn(0x7fe4680f9fd0 msgr2=0x7fe4681011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:39.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 shutdown_connections 2023-07-14T16:13:39.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:39.156+0000 7fe46f9ff700 1 -- 172.21.15.62:0/567131835 wait complete. 2023-07-14T16:13:39.135 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2023-07-14T16:13:39.440 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:13:39 smithi081 systemd[1]: Starting Ceph mds.h for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:39.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:39 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/567131835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:39.892 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:39 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/567131835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:40.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:39 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/567131835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:40.153 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:13:39 smithi081 podman[152866]: 2023-07-14T16:13:40.537 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:13:40 smithi081 bash[152866]: f35b473a3f5402cde95b175d8ae184b67151d3b3376403be6dc1d65e5fa80c50 2023-07-14T16:13:40.538 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-mds[152907]: -- 172.21.15.81:0/249077189 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e434e71e0 con 0x560e434bd000 2023-07-14T16:13:40.538 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-h[152882]: starting mds.h at 2023-07-14T16:13:40.538 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:13:40 smithi081 systemd[1]: Started Ceph mds.h for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:40 smithi062 ceph-mon[95543]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:40.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:40.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-mon[127795]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:40.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:40.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:40.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:40 smithi164 ceph-mon[97427]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:41.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:41.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 -- 172.21.15.62:0/1616246617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 msgr2=0x7fd97c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:41.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 --2- 172.21.15.62:0/1616246617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 0x7fd97c107f40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd97400b0a0 tx=0x7fd97400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 -- 172.21.15.62:0/1616246617 shutdown_connections 2023-07-14T16:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 --2- 172.21.15.62:0/1616246617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 0x7fd97c107f40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 --2- 172.21.15.62:0/1616246617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0ffb00 0x7fd97c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 --2- 172.21.15.62:0/1616246617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c100cf0 0x7fd97c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 -- 172.21.15.62:0/1616246617 >> 172.21.15.62:0/1616246617 conn(0x7fd97c0fb0e0 msgr2=0x7fd97c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 -- 172.21.15.62:0/1616246617 shutdown_connections 2023-07-14T16:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.390+0000 7fd9825ff700 1 -- 172.21.15.62:0/1616246617 wait complete. 2023-07-14T16:13:41.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.391+0000 7fd9825ff700 1 Processor -- start 2023-07-14T16:13:41.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.391+0000 7fd9825ff700 1 -- start start 2023-07-14T16:13:41.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.391+0000 7fd9825ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0ffb00 0x7fd97c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:41.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd9825ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c100cf0 0x7fd97c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:41.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd9825ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 0x7fd97c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:41.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd9825ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0735e0 con 0x7fd97c101680 2023-07-14T16:13:41.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd9825ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c073750 con 0x7fd97c100cf0 2023-07-14T16:13:41.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd9825ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c19f880 con 0x7fd97c0ffb00 2023-07-14T16:13:41.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 0x7fd97c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:41.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 0x7fd97c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45528/0 (socket says 172.21.15.62:45528) 2023-07-14T16:13:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 -- 172.21.15.62:0/3260098159 learned_addr learned my addr 172.21.15.62:0/3260098159 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd97b7fe700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c100cf0 0x7fd97c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 -- 172.21.15.62:0/3260098159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0ffb00 msgr2=0x7fd97c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0ffb00 0x7fd97c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 -- 172.21.15.62:0/3260098159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c100cf0 msgr2=0x7fd97c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c100cf0 0x7fd97c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 -- 172.21.15.62:0/3260098159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd974009d00 con 0x7fd97c101680 2023-07-14T16:13:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.392+0000 7fd980b9c700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 0x7fd97c072e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd974009fd0 tx=0x7fd97400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.393+0000 7fd9797fa700 1 -- 172.21.15.62:0/3260098159 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400e040 con 0x7fd97c101680 2023-07-14T16:13:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.393+0000 7fd9797fa700 1 -- 172.21.15.62:0/3260098159 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd974003e10 con 0x7fd97c101680 2023-07-14T16:13:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.393+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c19fb00 con 0x7fd97c101680 2023-07-14T16:13:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.393+0000 7fd9797fa700 1 -- 172.21.15.62:0/3260098159 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97401b6c0 con 0x7fd97c101680 2023-07-14T16:13:41.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.393+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c19fed0 con 0x7fd97c101680 2023-07-14T16:13:41.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.394+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c106940 con 0x7fd97c101680 2023-07-14T16:13:41.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.396+0000 7fd9797fa700 1 -- 172.21.15.62:0/3260098159 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd97401d990 con 0x7fd97c101680 2023-07-14T16:13:41.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.397+0000 7fd9797fa700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9640a11a0 0x7fd9640a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:41.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.398+0000 7fd9797fa700 1 -- 172.21.15.62:0/3260098159 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9740cdf40 con 0x7fd97c101680 2023-07-14T16:13:41.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.398+0000 7fd97bfff700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9640a11a0 0x7fd9640a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:41.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.398+0000 7fd97bfff700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9640a11a0 0x7fd9640a3650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd97c100b50 tx=0x7fd96c0096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:41.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.399+0000 7fd9797fa700 1 -- 172.21.15.62:0/3260098159 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd974097bc0 con 0x7fd97c101680 2023-07-14T16:13:41.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.654+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd97c02cbf0 con 0x7fd97c101680 2023-07-14T16:13:41.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.655+0000 7fd9797fa700 1 -- 172.21.15.62:0/3260098159 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6556 (secure 0 0 0) 0x7fd97409b1e0 con 0x7fd97c101680 2023-07-14T16:13:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:41.633 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-14T16:13:41.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9640a11a0 msgr2=0x7fd9640a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:41.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9640a11a0 0x7fd9640a3650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd97c100b50 tx=0x7fd96c0096c0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 msgr2=0x7fd97c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 0x7fd97c072e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd974009fd0 tx=0x7fd97400bd30 comp rx=0 tx=0).stop 2023-07-14T16:13:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 shutdown_connections 2023-07-14T16:13:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd97c101680 0x7fd97c072e40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd97c0ffb00 0x7fd97c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9640a11a0 0x7fd9640a3650 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 --2- 172.21.15.62:0/3260098159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd97c100cf0 0x7fd97c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 >> 172.21.15.62:0/3260098159 conn(0x7fd97c0fb0e0 msgr2=0x7fd97c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.657+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 shutdown_connections 2023-07-14T16:13:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:41.658+0000 7fd9825ff700 1 -- 172.21.15.62:0/3260098159 wait complete. 2023-07-14T16:13:41.637 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 9 2023-07-14T16:13:41.657 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:41 smithi164 ceph-mon[97427]: Deploying daemon mds.i on smithi164 2023-07-14T16:13:41.658 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:41 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.81:6836/3627777030,v1:172.21.15.81:6837/3627777030] up:boot 2023-07-14T16:13:41.658 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:41 smithi164 ceph-mon[97427]: fsmap 8 up:standby 2023-07-14T16:13:41.658 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T16:13:41.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:41 smithi062 ceph-mon[95543]: Deploying daemon mds.i on smithi164 2023-07-14T16:13:41.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:41 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.81:6836/3627777030,v1:172.21.15.81:6837/3627777030] up:boot 2023-07-14T16:13:41.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:41 smithi062 ceph-mon[95543]: fsmap 8 up:standby 2023-07-14T16:13:41.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T16:13:41.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:41 smithi081 ceph-mon[127795]: Deploying daemon mds.i on smithi164 2023-07-14T16:13:41.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:41 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.81:6836/3627777030,v1:172.21.15.81:6837/3627777030] up:boot 2023-07-14T16:13:41.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:41 smithi081 ceph-mon[127795]: fsmap 8 up:standby 2023-07-14T16:13:41.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T16:13:42.634 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:13:42 smithi164 systemd[1]: Starting Ceph mds.i for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:42.634 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:42 smithi164 ceph-mon[97427]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:42.634 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:42 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3260098159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:42.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:42 smithi062 ceph-mon[95543]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:42.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:42 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3260098159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:42.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:42 smithi081 ceph-mon[127795]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:42.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:42 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3260098159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:43.379 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:13:43 smithi164 podman[123888]: 2023-07-14T16:13:43.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:43.775 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:13:43 smithi164 bash[123888]: 9b9ef37605d27a26ac26738c42d7a9b0d869d271ef9fe9a81926e1e433153096 2023-07-14T16:13:43.775 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:13:43 smithi164 ceph-mds[123929]: -- 172.21.15.164:0/3623204775 <== mon.1 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0757631e0 con 0x55a075738800 2023-07-14T16:13:43.776 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:13:43 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-i[123904]: starting mds.i at 2023-07-14T16:13:43.776 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:13:43 smithi164 systemd[1]: Started Ceph mds.i for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:43.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 -- 172.21.15.62:0/2380813978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0072f10 msgr2=0x7f46e0073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:43.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 --2- 172.21.15.62:0/2380813978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0072f10 0x7f46e0073380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009990 tx=0x7f46d4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 -- 172.21.15.62:0/2380813978 shutdown_connections 2023-07-14T16:13:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 --2- 172.21.15.62:0/2380813978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46e0107990 0x7f46e0109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 --2- 172.21.15.62:0/2380813978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e00744f0 0x7f46e0072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 --2- 172.21.15.62:0/2380813978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0072f10 0x7f46e0073380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 -- 172.21.15.62:0/2380813978 >> 172.21.15.62:0/2380813978 conn(0x7f46e00fb0e0 msgr2=0x7f46e00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 -- 172.21.15.62:0/2380813978 shutdown_connections 2023-07-14T16:13:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.886+0000 7f46e4fff700 1 -- 172.21.15.62:0/2380813978 wait complete. 2023-07-14T16:13:43.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.887+0000 7f46e4fff700 1 Processor -- start 2023-07-14T16:13:43.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.887+0000 7f46e4fff700 1 -- start start 2023-07-14T16:13:43.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46e4fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46e0072f10 0x7f46e0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46e4fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e00744f0 0x7f46e0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46e4fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0107990 0x7f46e019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46e4fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e019f410 con 0x7f46e00744f0 2023-07-14T16:13:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46e4fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e019f550 con 0x7f46e0072f10 2023-07-14T16:13:43.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46e4fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e019f6e0 con 0x7f46e0107990 2023-07-14T16:13:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46ddd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e00744f0 0x7f46e0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46ddd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e00744f0 0x7f46e0197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54114/0 (socket says 172.21.15.62:54114) 2023-07-14T16:13:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46ded9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0107990 0x7f46e019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46ddd9b700 1 -- 172.21.15.62:0/3416273510 learned_addr learned my addr 172.21.15.62:0/3416273510 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.888+0000 7f46de59c700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46e0072f10 0x7f46e0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46ded9d700 1 -- 172.21.15.62:0/3416273510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46e0072f10 msgr2=0x7f46e0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46ded9d700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46e0072f10 0x7f46e0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46ded9d700 1 -- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e00744f0 msgr2=0x7f46e0197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46ded9d700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e00744f0 0x7f46e0197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46ded9d700 1 -- 172.21.15.62:0/3416273510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d40095f0 con 0x7f46e0107990 2023-07-14T16:13:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46de59c700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46e0072f10 0x7f46e0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:13:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46ddd9b700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e00744f0 0x7f46e0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46ded9d700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0107990 0x7f46e019ee40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46d800cc50 tx=0x7f46d800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46cf7fe700 1 -- 172.21.15.62:0/3416273510 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d800ab40 con 0x7f46e0107990 2023-07-14T16:13:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46cf7fe700 1 -- 172.21.15.62:0/3416273510 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46d800aca0 con 0x7f46e0107990 2023-07-14T16:13:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e019f9c0 con 0x7f46e0107990 2023-07-14T16:13:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46cf7fe700 1 -- 172.21.15.62:0/3416273510 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80115e0 con 0x7f46e0107990 2023-07-14T16:13:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.889+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e019ff20 con 0x7f46e0107990 2023-07-14T16:13:43.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.890+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e01060d0 con 0x7f46e0107990 2023-07-14T16:13:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.892+0000 7f46cf7fe700 1 -- 172.21.15.62:0/3416273510 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f46d8004ad0 con 0x7f46e0107990 2023-07-14T16:13:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.894+0000 7f46cf7fe700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46c80a1220 0x7f46c80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.894+0000 7f46cf7fe700 1 -- 172.21.15.62:0/3416273510 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f46d80cc5d0 con 0x7f46e0107990 2023-07-14T16:13:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.894+0000 7f46de59c700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46c80a1220 0x7f46c80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.894+0000 7f46de59c700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46c80a1220 0x7f46c80a36d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009cb0 tx=0x7f46d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:43.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:43.896+0000 7f46cf7fe700 1 -- 172.21.15.62:0/3416273510 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f46d8096120 con 0x7f46e0107990 2023-07-14T16:13:44.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.160+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46e002c850 con 0x7f46e0107990 2023-07-14T16:13:44.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.161+0000 7f46cf7fe700 1 -- 172.21.15.62:0/3416273510 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7315 (secure 0 0 0) 0x7f46d8016090 con 0x7f46e0107990 2023-07-14T16:13:44.139 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:44.139 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T16:13:44.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.163+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46c80a1220 msgr2=0x7f46c80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.163+0000 7f46e4fff700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46c80a1220 0x7f46c80a36d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009cb0 tx=0x7f46d0009260 comp rx=0 tx=0).stop 2023-07-14T16:13:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.163+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0107990 msgr2=0x7f46e019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.163+0000 7f46e4fff700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0107990 0x7f46e019ee40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f46d800cc50 tx=0x7f46d800cf60 comp rx=0 tx=0).stop 2023-07-14T16:13:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.164+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 shutdown_connections 2023-07-14T16:13:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.164+0000 7f46e4fff700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46e0107990 0x7f46e019ee40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.164+0000 7f46e4fff700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46c80a1220 0x7f46c80a36d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.164+0000 7f46e4fff700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46e0072f10 0x7f46e0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.164+0000 7f46e4fff700 1 --2- 172.21.15.62:0/3416273510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46e00744f0 0x7f46e0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.164+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 >> 172.21.15.62:0/3416273510 conn(0x7f46e00fb0e0 msgr2=0x7f46e00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.164+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 shutdown_connections 2023-07-14T16:13:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:44.164+0000 7f46e4fff700 1 -- 172.21.15.62:0/3416273510 wait complete. 2023-07-14T16:13:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 10 2023-07-14T16:13:44.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:44.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:44.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:44.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:44.776 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: Deploying daemon mds.g on smithi062 2023-07-14T16:13:44.777 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] up:boot 2023-07-14T16:13:44.777 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: fsmap 9 up:standby 2023-07-14T16:13:44.777 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T16:13:44.777 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:44 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3416273510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:44.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:44.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:44.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:44.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:44.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: Deploying daemon mds.g on smithi062 2023-07-14T16:13:44.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] up:boot 2023-07-14T16:13:44.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: fsmap 9 up:standby 2023-07-14T16:13:44.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T16:13:44.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:44 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3416273510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:44.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:44.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:44.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:44.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:44.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:44.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: Deploying daemon mds.g on smithi062 2023-07-14T16:13:44.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] up:boot 2023-07-14T16:13:44.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: fsmap 9 up:standby 2023-07-14T16:13:44.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T16:13:44.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:44 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3416273510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:45.735 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:13:45 smithi062 systemd[1]: Starting Ceph mds.g for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:46.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:46.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 -- 172.21.15.62:0/2536382264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 msgr2=0x7f40d8102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:46.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 --2- 172.21.15.62:0/2536382264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 0x7f40d8102930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f40d400b0a0 tx=0x7f40d400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 -- 172.21.15.62:0/2536382264 shutdown_connections 2023-07-14T16:13:46.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 --2- 172.21.15.62:0/2536382264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40d8105790 0x7f40d8107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 --2- 172.21.15.62:0/2536382264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 0x7f40d8102930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 --2- 172.21.15.62:0/2536382264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40d8102e70 0x7f40d8105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 -- 172.21.15.62:0/2536382264 >> 172.21.15.62:0/2536382264 conn(0x7f40d80f9fd0 msgr2=0x7f40d80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:46.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 -- 172.21.15.62:0/2536382264 shutdown_connections 2023-07-14T16:13:46.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.428+0000 7f40dfbff700 1 -- 172.21.15.62:0/2536382264 wait complete. 2023-07-14T16:13:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.429+0000 7f40dfbff700 1 Processor -- start 2023-07-14T16:13:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.429+0000 7f40dfbff700 1 -- start start 2023-07-14T16:13:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.429+0000 7f40dfbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 0x7f40d8195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.429+0000 7f40dfbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40d8102e70 0x7f40d8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.429+0000 7f40dfbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40d8105790 0x7f40d819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.429+0000 7f40dfbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819d3f0 con 0x7f40d8100550 2023-07-14T16:13:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.429+0000 7f40dfbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819d530 con 0x7f40d8102e70 2023-07-14T16:13:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.429+0000 7f40dfbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d819d6c0 con 0x7f40d8105790 2023-07-14T16:13:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 0x7f40d8195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 0x7f40d8195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54134/0 (socket says 172.21.15.62:54134) 2023-07-14T16:13:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 -- 172.21.15.62:0/1143904118 learned_addr learned my addr 172.21.15.62:0/1143904118 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd19a700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40d8102e70 0x7f40d8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40de19c700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40d8105790 0x7f40d819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 -- 172.21.15.62:0/1143904118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40d8105790 msgr2=0x7f40d819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40d8105790 0x7f40d819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 -- 172.21.15.62:0/1143904118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40d8102e70 msgr2=0x7f40d8195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40d8102e70 0x7f40d8195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 -- 172.21.15.62:0/1143904118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4009d00 con 0x7f40d8100550 2023-07-14T16:13:46.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40de19c700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40d8105790 0x7f40d819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:13:46.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.430+0000 7f40dd99b700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 0x7f40d8195870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f40d400b780 tx=0x7f40d400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:46.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.431+0000 7f40ceffd700 1 -- 172.21.15.62:0/1143904118 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400e040 con 0x7f40d8100550 2023-07-14T16:13:46.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.431+0000 7f40ceffd700 1 -- 172.21.15.62:0/1143904118 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40d4003e10 con 0x7f40d8100550 2023-07-14T16:13:46.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.431+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d819d940 con 0x7f40d8100550 2023-07-14T16:13:46.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.431+0000 7f40ceffd700 1 -- 172.21.15.62:0/1143904118 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40052a0 con 0x7f40d8100550 2023-07-14T16:13:46.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.431+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d819ddc0 con 0x7f40d8100550 2023-07-14T16:13:46.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.432+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d80fbb40 con 0x7f40d8100550 2023-07-14T16:13:46.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.434+0000 7f40ceffd700 1 -- 172.21.15.62:0/1143904118 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f40d401d9a0 con 0x7f40d8100550 2023-07-14T16:13:46.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.435+0000 7f40ceffd700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40c40a1220 0x7f40c40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:46.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.435+0000 7f40ceffd700 1 -- 172.21.15.62:0/1143904118 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40d40cd720 con 0x7f40d8100550 2023-07-14T16:13:46.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.436+0000 7f40dd19a700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40c40a1220 0x7f40c40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:46.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.436+0000 7f40dd19a700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40c40a1220 0x7f40c40a36d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009c60 tx=0x7f40c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:46.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.439+0000 7f40ceffd700 1 -- 172.21.15.62:0/1143904118 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f40d4097300 con 0x7f40d8100550 2023-07-14T16:13:46.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:46 smithi062 ceph-mon[95543]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:46.554 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:13:46 smithi062 podman[130263]: 2023-07-14T16:13:46.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.727+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40d802c850 con 0x7f40d8100550 2023-07-14T16:13:46.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.728+0000 7f40ceffd700 1 -- 172.21.15.62:0/1143904118 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7315 (secure 0 0 0) 0x7f40d409a920 con 0x7f40d8100550 2023-07-14T16:13:46.707 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:46.707 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T16:13:46.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40c40a1220 msgr2=0x7f40c40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:46.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40c40a1220 0x7f40c40a36d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009c60 tx=0x7f40c8009260 comp rx=0 tx=0).stop 2023-07-14T16:13:46.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 msgr2=0x7f40d8195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:46.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 0x7f40d8195870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f40d400b780 tx=0x7f40d400bd40 comp rx=0 tx=0).stop 2023-07-14T16:13:46.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 shutdown_connections 2023-07-14T16:13:46.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40d8105790 0x7f40d819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40d8100550 0x7f40d8195870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40c40a1220 0x7f40c40a36d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 --2- 172.21.15.62:0/1143904118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40d8102e70 0x7f40d8195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:46.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.730+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 >> 172.21.15.62:0/1143904118 conn(0x7f40d80f9fd0 msgr2=0x7f40d8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:46.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.731+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 shutdown_connections 2023-07-14T16:13:46.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:46.731+0000 7f40dfbff700 1 -- 172.21.15.62:0/1143904118 wait complete. 2023-07-14T16:13:46.711 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 10 2023-07-14T16:13:46.775 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:46 smithi164 ceph-mon[97427]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:46.874 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:13:46 smithi062 ceph-mds[130346]: -- 172.21.15.62:0/3611044986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56550bc471e0 con 0x56550bc1d000 2023-07-14T16:13:46.874 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:13:46 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-g[130302]: starting mds.g at 2023-07-14T16:13:46.875 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:13:46 smithi062 bash[130263]: 94ae63251fb05065837c1af24fd50a2f865ae84c30d1a9445f8a9bb182f85e71 2023-07-14T16:13:46.875 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:13:46 smithi062 systemd[1]: Started Ceph mds.g for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:46.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:46 smithi081 ceph-mon[127795]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:47.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:47.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:47.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1143904118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:47.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:47.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:47.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:47.971 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:47.971 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: Deploying daemon mds.a on smithi062 2023-07-14T16:13:47.971 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.62:6838/3506657450,v1:172.21.15.62:6839/3506657450] up:boot 2023-07-14T16:13:47.971 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: fsmap 10 up:standby 2023-07-14T16:13:47.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:47 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T16:13:48.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1143904118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:48.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:48.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: Deploying daemon mds.a on smithi062 2023-07-14T16:13:48.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.62:6838/3506657450,v1:172.21.15.62:6839/3506657450] up:boot 2023-07-14T16:13:48.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: fsmap 10 up:standby 2023-07-14T16:13:48.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:47 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T16:13:48.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:48.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:48.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1143904118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:48.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:48.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:48.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:48.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:48.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: Deploying daemon mds.a on smithi062 2023-07-14T16:13:48.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.62:6838/3506657450,v1:172.21.15.62:6839/3506657450] up:boot 2023-07-14T16:13:48.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: fsmap 10 up:standby 2023-07-14T16:13:48.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:47 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T16:13:48.729 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:48.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:48 smithi062 ceph-mon[95543]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:48 smithi164 ceph-mon[97427]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:49.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 -- 172.21.15.62:0/1515525652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 msgr2=0x7f49800fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:49.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 --2- 172.21.15.62:0/1515525652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 0x7f49800fff30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f497c009990 tx=0x7f497c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 -- 172.21.15.62:0/1515525652 shutdown_connections 2023-07-14T16:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 --2- 172.21.15.62:0/1515525652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49801016a0 0x7f4980107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 --2- 172.21.15.62:0/1515525652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 0x7f49800fff30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 --2- 172.21.15.62:0/1515525652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4980100d10 0x7f4980101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 -- 172.21.15.62:0/1515525652 >> 172.21.15.62:0/1515525652 conn(0x7f49800fb160 msgr2=0x7f49800fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 -- 172.21.15.62:0/1515525652 shutdown_connections 2023-07-14T16:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.124+0000 7f49883ff700 1 -- 172.21.15.62:0/1515525652 wait complete. 2023-07-14T16:13:49.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.126+0000 7f49883ff700 1 Processor -- start 2023-07-14T16:13:49.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.126+0000 7f49883ff700 1 -- start start 2023-07-14T16:13:49.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.126+0000 7f49883ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 0x7f49800742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:49.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.126+0000 7f49883ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4980100d10 0x7f4980072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:49.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.126+0000 7f49883ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49801016a0 0x7f4980072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:49.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.126+0000 7f49883ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800735e0 con 0x7f49800ffb20 2023-07-14T16:13:49.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.126+0000 7f49883ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4980073750 con 0x7f4980100d10 2023-07-14T16:13:49.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.126+0000 7f49883ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498019f7c0 con 0x7f49801016a0 2023-07-14T16:13:49.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 0x7f49800742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:49.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 0x7f49800742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54182/0 (socket says 172.21.15.62:54182) 2023-07-14T16:13:49.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 -- 172.21.15.62:0/39179944 learned_addr learned my addr 172.21.15.62:0/39179944 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:49.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498599a700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4980100d10 0x7f4980072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:49.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498699c700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49801016a0 0x7f4980072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:49.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 -- 172.21.15.62:0/39179944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49801016a0 msgr2=0x7f4980072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:49.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49801016a0 0x7f4980072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 -- 172.21.15.62:0/39179944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4980100d10 msgr2=0x7f4980072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:49.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4980100d10 0x7f4980072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 -- 172.21.15.62:0/39179944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c0095f0 con 0x7f49800ffb20 2023-07-14T16:13:49.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.127+0000 7f498619b700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 0x7f49800742c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f497c0098e0 tx=0x7f497c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:49.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.128+0000 7f49777fe700 1 -- 172.21.15.62:0/39179944 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c01d070 con 0x7f49800ffb20 2023-07-14T16:13:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.128+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498019fa40 con 0x7f49800ffb20 2023-07-14T16:13:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.128+0000 7f49777fe700 1 -- 172.21.15.62:0/39179944 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f497c00bdd0 con 0x7f49800ffb20 2023-07-14T16:13:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.128+0000 7f49777fe700 1 -- 172.21.15.62:0/39179944 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c022d20 con 0x7f49800ffb20 2023-07-14T16:13:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.128+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498019fec0 con 0x7f49800ffb20 2023-07-14T16:13:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.129+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498010b1f0 con 0x7f49800ffb20 2023-07-14T16:13:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.131+0000 7f49777fe700 1 -- 172.21.15.62:0/39179944 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f497c00b940 con 0x7f49800ffb20 2023-07-14T16:13:49.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.133+0000 7f49777fe700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f496c0a1220 0x7f496c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:49.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.133+0000 7f49777fe700 1 -- 172.21.15.62:0/39179944 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f497c0cdea0 con 0x7f49800ffb20 2023-07-14T16:13:49.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.133+0000 7f498599a700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f496c0a1220 0x7f496c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:49.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.134+0000 7f498599a700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f496c0a1220 0x7f496c0a36d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4970005950 tx=0x7f497000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:49.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.135+0000 7f49777fe700 1 -- 172.21.15.62:0/39179944 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f497c097a80 con 0x7f49800ffb20 2023-07-14T16:13:49.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:48 smithi081 ceph-mon[127795]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:49.374 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:13:48 smithi062 systemd[1]: Starting Ceph mds.a for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.408+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4980106a00 con 0x7f49800ffb20 2023-07-14T16:13:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.409+0000 7f49777fe700 1 -- 172.21.15.62:0/39179944 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8073 (secure 0 0 0) 0x7f497c09b0a0 con 0x7f49800ffb20 2023-07-14T16:13:49.386 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:49.387 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T16:13:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f496c0a1220 msgr2=0x7f496c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f496c0a1220 0x7f496c0a36d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4970005950 tx=0x7f497000b410 comp rx=0 tx=0).stop 2023-07-14T16:13:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 msgr2=0x7f49800742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 0x7f49800742c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f497c0098e0 tx=0x7f497c004730 comp rx=0 tx=0).stop 2023-07-14T16:13:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 shutdown_connections 2023-07-14T16:13:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49801016a0 0x7f4980072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49800ffb20 0x7f49800742c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f496c0a1220 0x7f496c0a36d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 --2- 172.21.15.62:0/39179944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4980100d10 0x7f4980072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 >> 172.21.15.62:0/39179944 conn(0x7f49800fb160 msgr2=0x7f49800fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 shutdown_connections 2023-07-14T16:13:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:49.411+0000 7f49883ff700 1 -- 172.21.15.62:0/39179944 wait complete. 2023-07-14T16:13:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 11 2023-07-14T16:13:49.707 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:13:49 smithi062 podman[130690]: 2023-07-14T16:13:50.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:49 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/39179944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:50.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:49 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/39179944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:50.125 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:13:49 smithi062 bash[130690]: 94df584da400efa18708f2ba71ba6be323dde1600be712137e6b7838c80609a6 2023-07-14T16:13:50.125 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:13:49 smithi062 ceph-mds[130732]: -- 172.21.15.62:0/1881757535 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56004c59d1e0 con 0x56004c572800 2023-07-14T16:13:50.125 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:13:49 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-a[130708]: starting mds.a at 2023-07-14T16:13:50.125 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:13:49 smithi062 systemd[1]: Started Ceph mds.a for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:50.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:49 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/39179944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:51.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:51.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.62:6840/33062382,v1:172.21.15.62:6841/33062382] up:boot 2023-07-14T16:13:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: fsmap 11 up:standby 2023-07-14T16:13:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T16:13:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:51.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:50 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.62:6840/33062382,v1:172.21.15.62:6841/33062382] up:boot 2023-07-14T16:13:51.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: fsmap 11 up:standby 2023-07-14T16:13:51.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T16:13:51.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.129 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:51.129 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:51.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:50 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.62:6840/33062382,v1:172.21.15.62:6841/33062382] up:boot 2023-07-14T16:13:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: fsmap 11 up:standby 2023-07-14T16:13:51.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T16:13:51.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:51.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T16:13:51.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T16:13:51.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:50 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:51.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:51.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.650+0000 7f807efff700 1 -- 172.21.15.62:0/724096700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80780ffb00 msgr2=0x7f80780fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:51.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.650+0000 7f807efff700 1 --2- 172.21.15.62:0/724096700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80780ffb00 0x7f80780fff10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f806c009990 tx=0x7f806c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.650+0000 7f807efff700 1 -- 172.21.15.62:0/724096700 shutdown_connections 2023-07-14T16:13:51.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.650+0000 7f807efff700 1 --2- 172.21.15.62:0/724096700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8078101680 0x7f8078107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.650+0000 7f807efff700 1 --2- 172.21.15.62:0/724096700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80780ffb00 0x7f80780fff10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.650+0000 7f807efff700 1 --2- 172.21.15.62:0/724096700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8078100cf0 0x7f8078101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.650+0000 7f807efff700 1 -- 172.21.15.62:0/724096700 >> 172.21.15.62:0/724096700 conn(0x7f80780fb0e0 msgr2=0x7f80780fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:51.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.651+0000 7f807efff700 1 -- 172.21.15.62:0/724096700 shutdown_connections 2023-07-14T16:13:51.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.651+0000 7f807efff700 1 -- 172.21.15.62:0/724096700 wait complete. 2023-07-14T16:13:51.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.652+0000 7f807efff700 1 Processor -- start 2023-07-14T16:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.652+0000 7f807efff700 1 -- start start 2023-07-14T16:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.652+0000 7f807efff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80780ffb00 0x7f80781957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.652+0000 7f807efff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8078100cf0 0x7f8078195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.652+0000 7f807efff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8078101680 0x7f807819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.652+0000 7f807efff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807819d370 con 0x7f8078101680 2023-07-14T16:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.652+0000 7f807efff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807819d4b0 con 0x7f8078100cf0 2023-07-14T16:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.652+0000 7f807efff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807819d640 con 0x7f80780ffb00 2023-07-14T16:13:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8078101680 0x7f807819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f8077fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8078100cf0 0x7f8078195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8078101680 0x7f807819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54224/0 (socket says 172.21.15.62:54224) 2023-07-14T16:13:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f8077fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8078100cf0 0x7f8078195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59582/0 (socket says 172.21.15.62:59582) 2023-07-14T16:13:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 -- 172.21.15.62:0/1605903520 learned_addr learned my addr 172.21.15.62:0/1605903520 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807cd9b700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80780ffb00 0x7f80781957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 -- 172.21.15.62:0/1605903520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80780ffb00 msgr2=0x7f80781957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:51.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80780ffb00 0x7f80781957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 -- 172.21.15.62:0/1605903520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8078100cf0 msgr2=0x7f8078195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:51.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8078100cf0 0x7f8078195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 -- 172.21.15.62:0/1605903520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c0095f0 con 0x7f8078101680 2023-07-14T16:13:51.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f8077fff700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8078100cf0 0x7f8078195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:51.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.653+0000 7f807d59c700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8078101680 0x7f807819cda0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8070007ee0 tx=0x7f807000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:51.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.654+0000 7f8075ffb700 1 -- 172.21.15.62:0/1605903520 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070017070 con 0x7f8078101680 2023-07-14T16:13:51.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.654+0000 7f8075ffb700 1 -- 172.21.15.62:0/1605903520 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f807000f040 con 0x7f8078101680 2023-07-14T16:13:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.654+0000 7f8075ffb700 1 -- 172.21.15.62:0/1605903520 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070005570 con 0x7f8078101680 2023-07-14T16:13:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.654+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807819d920 con 0x7f8078101680 2023-07-14T16:13:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.654+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807819de30 con 0x7f8078101680 2023-07-14T16:13:51.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.655+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807810b1d0 con 0x7f8078101680 2023-07-14T16:13:51.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.657+0000 7f8075ffb700 1 -- 172.21.15.62:0/1605903520 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f807000db20 con 0x7f8078101680 2023-07-14T16:13:51.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.659+0000 7f8075ffb700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80680a1220 0x7f80680a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:51.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.659+0000 7f8075ffb700 1 -- 172.21.15.62:0/1605903520 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f80700cbb40 con 0x7f8078101680 2023-07-14T16:13:51.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.660+0000 7f807cd9b700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80680a1220 0x7f80680a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:51.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.660+0000 7f807cd9b700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80680a1220 0x7f80680a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f806c005910 tx=0x7f806c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:51.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.663+0000 7f8075ffb700 1 -- 172.21.15.62:0/1605903520 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8070095690 con 0x7f8078101680 2023-07-14T16:13:51.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.924+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f807810b3c0 con 0x7f8078101680 2023-07-14T16:13:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.924+0000 7f8075ffb700 1 -- 172.21.15.62:0/1605903520 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8825 (secure 0 0 0) 0x7f8070098cb0 con 0x7f8078101680 2023-07-14T16:13:51.909 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:51.910 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-14T16:13:51.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80680a1220 msgr2=0x7f80680a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:51.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80680a1220 0x7f80680a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f806c005910 tx=0x7f806c00b260 comp rx=0 tx=0).stop 2023-07-14T16:13:51.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8078101680 msgr2=0x7f807819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:51.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8078101680 0x7f807819cda0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8070007ee0 tx=0x7f807000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 shutdown_connections 2023-07-14T16:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8078101680 0x7f807819cda0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80780ffb00 0x7f80781957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80680a1220 0x7f80680a36d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 --2- 172.21.15.62:0/1605903520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8078100cf0 0x7f8078195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 >> 172.21.15.62:0/1605903520 conn(0x7f80780fb0e0 msgr2=0x7f80780fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 shutdown_connections 2023-07-14T16:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:51.927+0000 7f807efff700 1 -- 172.21.15.62:0/1605903520 wait complete. 2023-07-14T16:13:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 12 2023-07-14T16:13:52.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:51 smithi164 ceph-mon[97427]: Deploying daemon mds.b on smithi081 2023-07-14T16:13:52.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:51 smithi062 ceph-mon[95543]: Deploying daemon mds.b on smithi081 2023-07-14T16:13:52.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:51 smithi081 ceph-mon[127795]: Deploying daemon mds.b on smithi081 2023-07-14T16:13:52.653 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:13:52 smithi081 systemd[1]: Starting Ceph mds.b for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:13:52.968 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:52 smithi081 ceph-mon[127795]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:52.968 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:52 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1605903520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:52.968 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:13:52 smithi081 podman[153210]: 2023-07-14T16:13:53.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:52 smithi164 ceph-mon[97427]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:53.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:52 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1605903520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:52 smithi062 ceph-mon[95543]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:53.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:52 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1605903520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:53.403 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:13:52 smithi081 bash[153210]: 3060dcfd8d17f9596a1bf9e24e01d0872dda1a3e47b6a3237eeea851026ae2f6 2023-07-14T16:13:53.403 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:13:52 smithi081 ceph-mds[153250]: -- 172.21.15.81:0/3333979193 <== mon.2 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5565ddcc51e0 con 0x5565ddc9a800 2023-07-14T16:13:53.404 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:13:52 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: starting mds.b at 2023-07-14T16:13:53.404 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:13:53 smithi081 systemd[1]: Started Ceph mds.b for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:13:53.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:13:54.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.157+0000 7f0c4bfff700 1 -- 172.21.15.62:0/1822277405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c44100560 msgr2=0x7f0c44102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.157+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/1822277405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c44100560 0x7f0c44102940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009990 tx=0x7f0c40009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.158+0000 7f0c4bfff700 1 -- 172.21.15.62:0/1822277405 shutdown_connections 2023-07-14T16:13:54.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.158+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/1822277405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c441057a0 0x7f0c44107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.158+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/1822277405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c44100560 0x7f0c44102940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.158+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/1822277405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c44102e80 0x7f0c44105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.158+0000 7f0c4bfff700 1 -- 172.21.15.62:0/1822277405 >> 172.21.15.62:0/1822277405 conn(0x7f0c440fa000 msgr2=0x7f0c440fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:54.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.158+0000 7f0c4bfff700 1 -- 172.21.15.62:0/1822277405 shutdown_connections 2023-07-14T16:13:54.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.158+0000 7f0c4bfff700 1 -- 172.21.15.62:0/1822277405 wait complete. 2023-07-14T16:13:54.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4bfff700 1 Processor -- start 2023-07-14T16:13:54.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4bfff700 1 -- start start 2023-07-14T16:13:54.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c44100560 0x7f0c441956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:54.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c44102e80 0x7f0c44195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:54.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c441057a0 0x7f0c4419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:54.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4419d230 con 0x7f0c441057a0 2023-07-14T16:13:54.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4bfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4419d370 con 0x7f0c44100560 2023-07-14T16:13:54.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4bfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4419d500 con 0x7f0c44102e80 2023-07-14T16:13:54.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c441057a0 0x7f0c4419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:54.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c49d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c44100560 0x7f0c441956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:54.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c441057a0 0x7f0c4419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60350/0 (socket says 172.21.15.62:60350) 2023-07-14T16:13:54.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c49d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c44100560 0x7f0c441956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:45252/0 (socket says 172.21.15.62:45252) 2023-07-14T16:13:54.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4a59c700 1 -- 172.21.15.62:0/2480943933 learned_addr learned my addr 172.21.15.62:0/2480943933 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:54.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.159+0000 7f0c4959a700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c44102e80 0x7f0c44195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:54.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c49d9b700 1 -- 172.21.15.62:0/2480943933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c44102e80 msgr2=0x7f0c44195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c49d9b700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c44102e80 0x7f0c44195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c49d9b700 1 -- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c441057a0 msgr2=0x7f0c4419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c49d9b700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c441057a0 0x7f0c4419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c49d9b700 1 -- 172.21.15.62:0/2480943933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c400095f0 con 0x7f0c44100560 2023-07-14T16:13:54.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c4959a700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c44102e80 0x7f0c44195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:54.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c4a59c700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c441057a0 0x7f0c4419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:54.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c49d9b700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c44100560 0x7f0c441956b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0c40003f60 tx=0x7f0c40005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:54.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c3affd700 1 -- 172.21.15.62:0/2480943933 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4001d070 con 0x7f0c44100560 2023-07-14T16:13:54.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c3affd700 1 -- 172.21.15.62:0/2480943933 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c4000bdb0 con 0x7f0c44100560 2023-07-14T16:13:54.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.160+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4419d780 con 0x7f0c44100560 2023-07-14T16:13:54.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.161+0000 7f0c3affd700 1 -- 172.21.15.62:0/2480943933 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40022cb0 con 0x7f0c44100560 2023-07-14T16:13:54.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.161+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4419dc00 con 0x7f0c44100560 2023-07-14T16:13:54.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.162+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4410b140 con 0x7f0c44100560 2023-07-14T16:13:54.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.163+0000 7f0c3affd700 1 -- 172.21.15.62:0/2480943933 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0c40022470 con 0x7f0c44100560 2023-07-14T16:13:54.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.165+0000 7f0c3affd700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0c300a1220 0x7f0c300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:54.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.165+0000 7f0c3affd700 1 -- 172.21.15.62:0/2480943933 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0c400cda40 con 0x7f0c44100560 2023-07-14T16:13:54.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.165+0000 7f0c4959a700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0c300a1220 0x7f0c300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:54.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.165+0000 7f0c4959a700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0c300a1220 0x7f0c300a36d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c34011490 tx=0x7f0c34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:54.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.169+0000 7f0c3affd700 1 -- 172.21.15.62:0/2480943933 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0c40097620 con 0x7f0c44100560 2023-07-14T16:13:54.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:54.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:54.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] up:boot 2023-07-14T16:13:54.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: fsmap 12 up:standby 2023-07-14T16:13:54.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:54 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T16:13:54.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:54.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:54.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] up:boot 2023-07-14T16:13:54.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: fsmap 12 up:standby 2023-07-14T16:13:54.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:54 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T16:13:54.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.457+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c4410b330 con 0x7f0c44100560 2023-07-14T16:13:54.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.458+0000 7f0c3affd700 1 -- 172.21.15.62:0/2480943933 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9583 (secure 0 0 0) 0x7f0c4009ac40 con 0x7f0c44100560 2023-07-14T16:13:54.436 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:54.436 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24551,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-14T16:13:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0c300a1220 msgr2=0x7f0c300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0c300a1220 0x7f0c300a36d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c34011490 tx=0x7f0c34009260 comp rx=0 tx=0).stop 2023-07-14T16:13:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c44100560 msgr2=0x7f0c441956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c44100560 0x7f0c441956b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0c40003f60 tx=0x7f0c40005740 comp rx=0 tx=0).stop 2023-07-14T16:13:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 shutdown_connections 2023-07-14T16:13:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c441057a0 0x7f0c4419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0c300a1220 0x7f0c300a36d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c44100560 0x7f0c441956b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 --2- 172.21.15.62:0/2480943933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0c44102e80 0x7f0c44195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.461+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 >> 172.21.15.62:0/2480943933 conn(0x7f0c440fa000 msgr2=0x7f0c44101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:54.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.462+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 shutdown_connections 2023-07-14T16:13:54.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.462+0000 7f0c4bfff700 1 -- 172.21.15.62:0/2480943933 wait complete. 2023-07-14T16:13:54.442 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 13 2023-07-14T16:13:54.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:13:54.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:54.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:54.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:54.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] up:boot 2023-07-14T16:13:54.528 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: fsmap 12 up:standby 2023-07-14T16:13:54.528 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:54 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T16:13:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 -- 172.21.15.62:0/3791381307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04100550 msgr2=0x7f0b04102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 --2- 172.21.15.62:0/3791381307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04100550 0x7f0b04102930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009990 tx=0x7f0af4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 -- 172.21.15.62:0/3791381307 shutdown_connections 2023-07-14T16:13:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 --2- 172.21.15.62:0/3791381307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b04105790 0x7f0b04107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 --2- 172.21.15.62:0/3791381307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04100550 0x7f0b04102930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 --2- 172.21.15.62:0/3791381307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b04102e70 0x7f0b04105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 -- 172.21.15.62:0/3791381307 >> 172.21.15.62:0/3791381307 conn(0x7f0b040f9fd0 msgr2=0x7f0b040fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:54.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 -- 172.21.15.62:0/3791381307 shutdown_connections 2023-07-14T16:13:54.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.694+0000 7f0b091ff700 1 -- 172.21.15.62:0/3791381307 wait complete. 2023-07-14T16:13:54.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b091ff700 1 Processor -- start 2023-07-14T16:13:54.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b091ff700 1 -- start start 2023-07-14T16:13:54.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b091ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b04100550 0x7f0b040ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b091ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b04102e70 0x7f0b040fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b091ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04105790 0x7f0b040fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b091ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b040ff1e0 con 0x7f0b04105790 2023-07-14T16:13:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b091ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b040ff350 con 0x7f0b04100550 2023-07-14T16:13:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b091ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0419f7c0 con 0x7f0b04102e70 2023-07-14T16:13:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.695+0000 7f0b0359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04105790 0x7f0b040fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04105790 0x7f0b040fea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60362/0 (socket says 172.21.15.62:60362) 2023-07-14T16:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0359e700 1 -- 172.21.15.62:0/1660316726 learned_addr learned my addr 172.21.15.62:0/1660316726 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0259c700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b04102e70 0x7f0b040fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b02d9d700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b04100550 0x7f0b040ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0359e700 1 -- 172.21.15.62:0/1660316726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b04102e70 msgr2=0x7f0b040fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0359e700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b04102e70 0x7f0b040fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0359e700 1 -- 172.21.15.62:0/1660316726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b04100550 msgr2=0x7f0b040ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0359e700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b04100550 0x7f0b040ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0359e700 1 -- 172.21.15.62:0/1660316726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af40095f0 con 0x7f0b04105790 2023-07-14T16:13:54.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b0359e700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04105790 0x7f0b040fea40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00b970 tx=0x7f0afc00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:54.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0aebfff700 1 -- 172.21.15.62:0/1660316726 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc00c9b0 con 0x7f0b04105790 2023-07-14T16:13:54.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0419faa0 con 0x7f0b04105790 2023-07-14T16:13:54.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0aebfff700 1 -- 172.21.15.62:0/1660316726 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0afc014440 con 0x7f0b04105790 2023-07-14T16:13:54.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.696+0000 7f0aebfff700 1 -- 172.21.15.62:0/1660316726 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0075d0 con 0x7f0b04105790 2023-07-14T16:13:54.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.697+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0419ffb0 con 0x7f0b04105790 2023-07-14T16:13:54.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.698+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b040fbb40 con 0x7f0b04105790 2023-07-14T16:13:54.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.700+0000 7f0aebfff700 1 -- 172.21.15.62:0/1660316726 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0afc0145b0 con 0x7f0b04105790 2023-07-14T16:13:54.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.701+0000 7f0aebfff700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0aec0a11a0 0x7f0aec0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.701+0000 7f0aebfff700 1 -- 172.21.15.62:0/1660316726 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0afc006e40 con 0x7f0b04105790 2023-07-14T16:13:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.701+0000 7f0b02d9d700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0aec0a11a0 0x7f0aec0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.702+0000 7f0b02d9d700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0aec0a11a0 0x7f0aec0a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006010 tx=0x7f0af400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.705+0000 7f0aebfff700 1 -- 172.21.15.62:0/1660316726 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0afc095970 con 0x7f0b04105790 2023-07-14T16:13:54.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.940+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b0402c850 con 0x7f0b04105790 2023-07-14T16:13:54.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.941+0000 7f0aebfff700 1 -- 172.21.15.62:0/1660316726 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18371 (secure 0 0 0) 0x7f0afc098f90 con 0x7f0b04105790 2023-07-14T16:13:54.920 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:54.920 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":82,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:13:01.805737+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:13:54.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0aec0a11a0 msgr2=0x7f0aec0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0aec0a11a0 0x7f0aec0a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006010 tx=0x7f0af400b260 comp rx=0 tx=0).stop 2023-07-14T16:13:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04105790 msgr2=0x7f0b040fea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04105790 0x7f0b040fea40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00b970 tx=0x7f0afc00bd30 comp rx=0 tx=0).stop 2023-07-14T16:13:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 shutdown_connections 2023-07-14T16:13:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b04105790 0x7f0b040fea40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0aec0a11a0 0x7f0aec0a3650 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b04100550 0x7f0b040ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 --2- 172.21.15.62:0/1660316726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b04102e70 0x7f0b040fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:54.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 >> 172.21.15.62:0/1660316726 conn(0x7f0b040f9fd0 msgr2=0x7f0b041011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:54.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 shutdown_connections 2023-07-14T16:13:54.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:54.944+0000 7f0b091ff700 1 -- 172.21.15.62:0/1660316726 wait complete. 2023-07-14T16:13:54.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:13:54.942 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:13:55.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.188+0000 7f14de9ff700 1 -- 172.21.15.62:0/1670576275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d8100560 msgr2=0x7f14d8102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.188+0000 7f14de9ff700 1 --2- 172.21.15.62:0/1670576275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d8100560 0x7f14d8102940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009990 tx=0x7f14c8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.189+0000 7f14de9ff700 1 -- 172.21.15.62:0/1670576275 shutdown_connections 2023-07-14T16:13:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.189+0000 7f14de9ff700 1 --2- 172.21.15.62:0/1670576275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14d81057a0 0x7f14d8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.189+0000 7f14de9ff700 1 --2- 172.21.15.62:0/1670576275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d8100560 0x7f14d8102940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.189+0000 7f14de9ff700 1 --2- 172.21.15.62:0/1670576275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14d8102e80 0x7f14d8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.189+0000 7f14de9ff700 1 -- 172.21.15.62:0/1670576275 >> 172.21.15.62:0/1670576275 conn(0x7f14d80fa000 msgr2=0x7f14d80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.189+0000 7f14de9ff700 1 -- 172.21.15.62:0/1670576275 shutdown_connections 2023-07-14T16:13:55.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.189+0000 7f14de9ff700 1 -- 172.21.15.62:0/1670576275 wait complete. 2023-07-14T16:13:55.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.190+0000 7f14de9ff700 1 Processor -- start 2023-07-14T16:13:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.190+0000 7f14de9ff700 1 -- start start 2023-07-14T16:13:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.190+0000 7f14de9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14d8100560 0x7f14d8197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.190+0000 7f14de9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14d8102e80 0x7f14d8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.190+0000 7f14de9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d81057a0 0x7f14d819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.190+0000 7f14de9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d819f500 con 0x7f14d81057a0 2023-07-14T16:13:55.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.190+0000 7f14de9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d819f640 con 0x7f14d8102e80 2023-07-14T16:13:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.190+0000 7f14de9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d819f7d0 con 0x7f14d8100560 2023-07-14T16:13:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14dcf9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d81057a0 0x7f14d819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14d8100560 0x7f14d8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14d8102e80 0x7f14d8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14d8100560 0x7f14d8197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55028/0 (socket says 172.21.15.62:55028) 2023-07-14T16:13:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d7fff700 1 -- 172.21.15.62:0/927998858 learned_addr learned my addr 172.21.15.62:0/927998858 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:55.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d77fe700 1 -- 172.21.15.62:0/927998858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14d8100560 msgr2=0x7f14d8197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d77fe700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14d8100560 0x7f14d8197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d77fe700 1 -- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d81057a0 msgr2=0x7f14d819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d77fe700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d81057a0 0x7f14d819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d77fe700 1 -- 172.21.15.62:0/927998858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c80095f0 con 0x7f14d8102e80 2023-07-14T16:13:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14dcf9c700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d81057a0 0x7f14d819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.191+0000 7f14d77fe700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14d8102e80 0x7f14d8197ec0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d950 tx=0x7f14c000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.192+0000 7f14d57fa700 1 -- 172.21.15.62:0/927998858 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c000f890 con 0x7f14d8102e80 2023-07-14T16:13:55.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.192+0000 7f14d57fa700 1 -- 172.21.15.62:0/927998858 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14c000fe80 con 0x7f14d8102e80 2023-07-14T16:13:55.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.192+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d819fab0 con 0x7f14d8102e80 2023-07-14T16:13:55.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.192+0000 7f14d57fa700 1 -- 172.21.15.62:0/927998858 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c000e3b0 con 0x7f14d8102e80 2023-07-14T16:13:55.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.192+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d81a0010 con 0x7f14d8102e80 2023-07-14T16:13:55.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.193+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d8107360 con 0x7f14d8102e80 2023-07-14T16:13:55.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.196+0000 7f14d57fa700 1 -- 172.21.15.62:0/927998858 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f14c000f9f0 con 0x7f14d8102e80 2023-07-14T16:13:55.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.197+0000 7f14d57fa700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14b80a11d0 0x7f14b80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:55.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.197+0000 7f14d57fa700 1 -- 172.21.15.62:0/927998858 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14c00cc130 con 0x7f14d8102e80 2023-07-14T16:13:55.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.197+0000 7f14d7fff700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14b80a11d0 0x7f14b80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:55.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.198+0000 7f14d7fff700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14b80a11d0 0x7f14b80a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14c800b5c0 tx=0x7f14c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:55.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.201+0000 7f14d57fa700 1 -- 172.21.15.62:0/927998858 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f14c0095cc0 con 0x7f14d8102e80 2023-07-14T16:13:55.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.414+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14d80611c0 con 0x7f14d8102e80 2023-07-14T16:13:55.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.415+0000 7f14d57fa700 1 -- 172.21.15.62:0/927998858 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14c00992e0 con 0x7f14d8102e80 2023-07-14T16:13:55.393 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:13:55.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14b80a11d0 msgr2=0x7f14b80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14b80a11d0 0x7f14b80a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14c800b5c0 tx=0x7f14c8005fb0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14d8102e80 msgr2=0x7f14d8197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14d8102e80 0x7f14d8197ec0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14c000d950 tx=0x7f14c000dc60 comp rx=0 tx=0).stop 2023-07-14T16:13:55.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 shutdown_connections 2023-07-14T16:13:55.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d81057a0 0x7f14d819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14d8100560 0x7f14d8197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14b80a11d0 0x7f14b80a3680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 --2- 172.21.15.62:0/927998858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14d8102e80 0x7f14d8197ec0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 >> 172.21.15.62:0/927998858 conn(0x7f14d80fa000 msgr2=0x7f14d80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:55.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 shutdown_connections 2023-07-14T16:13:55.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.417+0000 7f14de9ff700 1 -- 172.21.15.62:0/927998858 wait complete. 2023-07-14T16:13:55.416 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-14T16:13:55.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:55 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2480943933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:55.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:55.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:55 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1660316726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:55.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:55 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2480943933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:55.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:55.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:55 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1660316726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:55.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 -- 172.21.15.62:0/1402531933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc100cf0 msgr2=0x7f32cc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 --2- 172.21.15.62:0/1402531933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc100cf0 0x7f32cc101140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009990 tx=0x7f32c0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 -- 172.21.15.62:0/1402531933 shutdown_connections 2023-07-14T16:13:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 --2- 172.21.15.62:0/1402531933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32cc101680 0x7f32cc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 --2- 172.21.15.62:0/1402531933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32cc0ffb00 0x7f32cc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 --2- 172.21.15.62:0/1402531933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc100cf0 0x7f32cc101140 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 -- 172.21.15.62:0/1402531933 >> 172.21.15.62:0/1402531933 conn(0x7f32cc0fb0e0 msgr2=0x7f32cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:55.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 -- 172.21.15.62:0/1402531933 shutdown_connections 2023-07-14T16:13:55.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.651+0000 7f32d17ff700 1 -- 172.21.15.62:0/1402531933 wait complete. 2023-07-14T16:13:55.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.652+0000 7f32d17ff700 1 Processor -- start 2023-07-14T16:13:55.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32d17ff700 1 -- start start 2023-07-14T16:13:55.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32d17ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32cc0ffb00 0x7f32cc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32d17ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32cc100cf0 0x7f32cc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32d17ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc101680 0x7f32cc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32d17ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc199650 con 0x7f32cc101680 2023-07-14T16:13:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32d17ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc073380 con 0x7f32cc0ffb00 2023-07-14T16:13:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32d17ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc073540 con 0x7f32cc100cf0 2023-07-14T16:13:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32cb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc101680 0x7f32cc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32cb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc101680 0x7f32cc072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60404/0 (socket says 172.21.15.62:60404) 2023-07-14T16:13:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32ca7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32cc100cf0 0x7f32cc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32cb7fe700 1 -- 172.21.15.62:0/3986742822 learned_addr learned my addr 172.21.15.62:0/3986742822 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.653+0000 7f32caffd700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32cc0ffb00 0x7f32cc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32cb7fe700 1 -- 172.21.15.62:0/3986742822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32cc100cf0 msgr2=0x7f32cc072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32cb7fe700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32cc100cf0 0x7f32cc072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32cb7fe700 1 -- 172.21.15.62:0/3986742822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32cc0ffb00 msgr2=0x7f32cc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32cb7fe700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32cc0ffb00 0x7f32cc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32cb7fe700 1 -- 172.21.15.62:0/3986742822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00095f0 con 0x7f32cc101680 2023-07-14T16:13:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32caffd700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32cc0ffb00 0x7f32cc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:13:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32cb7fe700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc101680 0x7f32cc072e40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32c4007ee0 tx=0x7f32c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32b3fff700 1 -- 172.21.15.62:0/3986742822 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c400ddd0 con 0x7f32cc101680 2023-07-14T16:13:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32b3fff700 1 -- 172.21.15.62:0/3986742822 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32c400f040 con 0x7f32cc101680 2023-07-14T16:13:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc1a4000 con 0x7f32cc101680 2023-07-14T16:13:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32b3fff700 1 -- 172.21.15.62:0/3986742822 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c40047d0 con 0x7f32cc101680 2023-07-14T16:13:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.654+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc1a4420 con 0x7f32cc101680 2023-07-14T16:13:55.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.656+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc10b1d0 con 0x7f32cc101680 2023-07-14T16:13:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.657+0000 7f32b3fff700 1 -- 172.21.15.62:0/3986742822 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f32c4003a00 con 0x7f32cc101680 2023-07-14T16:13:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.659+0000 7f32b3fff700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32b40a11a0 0x7f32b40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.659+0000 7f32b3fff700 1 -- 172.21.15.62:0/3986742822 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f32c40cbdf0 con 0x7f32cc101680 2023-07-14T16:13:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.659+0000 7f32caffd700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32b40a11a0 0x7f32b40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.660+0000 7f32caffd700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32b40a11a0 0x7f32b40a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f32cc100b50 tx=0x7f32bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:55.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.663+0000 7f32b3fff700 1 -- 172.21.15.62:0/3986742822 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f32c404e790 con 0x7f32cc101680 2023-07-14T16:13:55.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:55 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2480943933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:13:55.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:55.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:55 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1660316726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:13:55.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.879+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f32cc10b390 con 0x7f32cc101680 2023-07-14T16:13:55.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.880+0000 7f32b3fff700 1 -- 172.21.15.62:0/3986742822 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f32c4095940 con 0x7f32cc101680 2023-07-14T16:13:55.858 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:13:55.858 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2023-07-14T16:13:55.858 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:13:55.858 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2023-07-14T16:13:55.858 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2023-07-14T16:13:55.858 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:13:55.859 INFO:teuthology.orchestra.run.smithi062.stdout:] 2023-07-14T16:13:55.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32b40a11a0 msgr2=0x7f32b40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32b40a11a0 0x7f32b40a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f32cc100b50 tx=0x7f32bc009260 comp rx=0 tx=0).stop 2023-07-14T16:13:55.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc101680 msgr2=0x7f32cc072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:55.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc101680 0x7f32cc072e40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32c4007ee0 tx=0x7f32c400d3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 shutdown_connections 2023-07-14T16:13:55.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc101680 0x7f32cc072e40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32b40a11a0 0x7f32b40a3650 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32cc0ffb00 0x7f32cc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 --2- 172.21.15.62:0/3986742822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32cc100cf0 0x7f32cc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 >> 172.21.15.62:0/3986742822 conn(0x7f32cc0fb0e0 msgr2=0x7f32cc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:55.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 shutdown_connections 2023-07-14T16:13:55.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:55.882+0000 7f32d17ff700 1 -- 172.21.15.62:0/3986742822 wait complete. 2023-07-14T16:13:55.881 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-14T16:13:55.881 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-14T16:13:56.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.116+0000 7f72a7dff700 1 -- 172.21.15.62:0/4065762477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0072f70 msgr2=0x7f72a00733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:56.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.116+0000 7f72a7dff700 1 --2- 172.21.15.62:0/4065762477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0072f70 0x7f72a00733e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f72900098f0 tx=0x7f7290009c00 comp rx=0 tx=0).stop 2023-07-14T16:13:56.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.116+0000 7f72a7dff700 1 -- 172.21.15.62:0/4065762477 shutdown_connections 2023-07-14T16:13:56.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.116+0000 7f72a7dff700 1 --2- 172.21.15.62:0/4065762477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72a01079a0 0x7f72a0109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:56.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.116+0000 7f72a7dff700 1 --2- 172.21.15.62:0/4065762477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72a0074550 0x7f72a00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:56.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.116+0000 7f72a7dff700 1 --2- 172.21.15.62:0/4065762477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a0072f70 0x7f72a00733e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:56.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.116+0000 7f72a7dff700 1 -- 172.21.15.62:0/4065762477 >> 172.21.15.62:0/4065762477 conn(0x7f72a00fb0e0 msgr2=0x7f72a00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:56.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.116+0000 7f72a7dff700 1 -- 172.21.15.62:0/4065762477 shutdown_connections 2023-07-14T16:13:56.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.117+0000 7f72a7dff700 1 -- 172.21.15.62:0/4065762477 wait complete. 2023-07-14T16:13:56.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.117+0000 7f72a7dff700 1 Processor -- start 2023-07-14T16:13:56.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a7dff700 1 -- start start 2023-07-14T16:13:56.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a7dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72a0072f70 0x7f72a0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:56.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a7dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72a0074550 0x7f72a0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:56.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a7dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a01079a0 0x7f72a019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:56.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a7dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a019f640 con 0x7f72a01079a0 2023-07-14T16:13:56.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a7dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a019f780 con 0x7f72a0072f70 2023-07-14T16:13:56.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a7dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a019f910 con 0x7f72a0074550 2023-07-14T16:13:56.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a639c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a01079a0 0x7f72a019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:56.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a539a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72a0074550 0x7f72a0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:56.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a5b9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72a0072f70 0x7f72a0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:56.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a639c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a01079a0 0x7f72a019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60418/0 (socket says 172.21.15.62:60418) 2023-07-14T16:13:56.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a639c700 1 -- 172.21.15.62:0/2914605171 learned_addr learned my addr 172.21.15.62:0/2914605171 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:56.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.118+0000 7f72a539a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72a0074550 0x7f72a0198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55056/0 (socket says 172.21.15.62:55056) 2023-07-14T16:13:56.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f72a639c700 1 -- 172.21.15.62:0/2914605171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72a0074550 msgr2=0x7f72a0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:56.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f72a639c700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72a0074550 0x7f72a0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:56.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f72a639c700 1 -- 172.21.15.62:0/2914605171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72a0072f70 msgr2=0x7f72a0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:56.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f72a639c700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72a0072f70 0x7f72a0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:56.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f72a639c700 1 -- 172.21.15.62:0/2914605171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c0095f0 con 0x7f72a01079a0 2023-07-14T16:13:56.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f72a639c700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a01079a0 0x7f72a019f070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f729800ef30 tx=0x7f729800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:56.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f7296ffd700 1 -- 172.21.15.62:0/2914605171 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298018070 con 0x7f72a01079a0 2023-07-14T16:13:56.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f7296ffd700 1 -- 172.21.15.62:0/2914605171 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f729800f040 con 0x7f72a01079a0 2023-07-14T16:13:56.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7290009520 con 0x7f72a01079a0 2023-07-14T16:13:56.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f7296ffd700 1 -- 172.21.15.62:0/2914605171 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729800cbc0 con 0x7f72a01079a0 2023-07-14T16:13:56.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.119+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a019fec0 con 0x7f72a01079a0 2023-07-14T16:13:56.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.120+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a0106050 con 0x7f72a01079a0 2023-07-14T16:13:56.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.122+0000 7f7296ffd700 1 -- 172.21.15.62:0/2914605171 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7298007500 con 0x7f72a01079a0 2023-07-14T16:13:56.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.124+0000 7f7296ffd700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f728c0a1150 0x7f728c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:56.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.124+0000 7f7296ffd700 1 -- 172.21.15.62:0/2914605171 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f72980cbf60 con 0x7f72a01079a0 2023-07-14T16:13:56.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.124+0000 7f72a5b9b700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f728c0a1150 0x7f728c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:56.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.124+0000 7f72a5b9b700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f728c0a1150 0x7f728c0a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f72a0074920 tx=0x7f729c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:56.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.127+0000 7f7296ffd700 1 -- 172.21.15.62:0/2914605171 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7298095b40 con 0x7f72a01079a0 2023-07-14T16:13:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:56.354+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f72a002c850 con 0x7f72a01079a0 2023-07-14T16:13:56.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:56 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/927998858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:56.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:56 smithi164 ceph-mon[97427]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:56.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:56 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3986742822' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T16:13:56.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:56 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/927998858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:56.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:56 smithi062 ceph-mon[95543]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:56.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:56 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3986742822' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T16:13:56.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:56 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/927998858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:13:56.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:56 smithi081 ceph-mon[127795]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:56.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:56 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3986742822' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T16:13:57.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.240+0000 7f7296ffd700 1 -- 172.21.15.62:0/2914605171 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7298099160 con 0x7f72a01079a0 2023-07-14T16:13:57.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.245+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f728c0a1150 msgr2=0x7f728c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:57.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.245+0000 7f72a7dff700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f728c0a1150 0x7f728c0a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f72a0074920 tx=0x7f729c009310 comp rx=0 tx=0).stop 2023-07-14T16:13:57.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.245+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a01079a0 msgr2=0x7f72a019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:57.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.245+0000 7f72a7dff700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a01079a0 0x7f72a019f070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f729800ef30 tx=0x7f729800c3a0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.246+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 shutdown_connections 2023-07-14T16:13:57.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.246+0000 7f72a7dff700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72a01079a0 0x7f72a019f070 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.246+0000 7f72a7dff700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f728c0a1150 0x7f728c0a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.246+0000 7f72a7dff700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72a0072f70 0x7f72a0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.246+0000 7f72a7dff700 1 --2- 172.21.15.62:0/2914605171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72a0074550 0x7f72a0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.246+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 >> 172.21.15.62:0/2914605171 conn(0x7f72a00fb0e0 msgr2=0x7f72a00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:57.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.246+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 shutdown_connections 2023-07-14T16:13:57.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.246+0000 7f72a7dff700 1 -- 172.21.15.62:0/2914605171 wait complete. 2023-07-14T16:13:57.227 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2023-07-14T16:13:57.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-14T16:13:57.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.489+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/4230057893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc88072f70 msgr2=0x7fcc880733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:57.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.489+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/4230057893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc88072f70 0x7fcc880733e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009990 tx=0x7fcc7c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.489+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/4230057893 shutdown_connections 2023-07-14T16:13:57.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.489+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/4230057893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc8810c0c0 0x7fcc8810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.489+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/4230057893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc88074550 0x7fcc880729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.489+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/4230057893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc88072f70 0x7fcc880733e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.489+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/4230057893 >> 172.21.15.62:0/4230057893 conn(0x7fcc880fb130 msgr2=0x7fcc880fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:57.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.489+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/4230057893 shutdown_connections 2023-07-14T16:13:57.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.490+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/4230057893 wait complete. 2023-07-14T16:13:57.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.491+0000 7fcc8c9ff700 1 Processor -- start 2023-07-14T16:13:57.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.491+0000 7fcc8c9ff700 1 -- start start 2023-07-14T16:13:57.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc8c9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc88072f70 0x7fcc8819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:57.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc8c9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc88074550 0x7fcc8819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:57.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc8c9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc8810c0c0 0x7fcc881a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:57.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc8c9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc881a3960 con 0x7fcc88074550 2023-07-14T16:13:57.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc8c9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc881a3aa0 con 0x7fcc8810c0c0 2023-07-14T16:13:57.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc8c9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc881a3c30 con 0x7fcc88072f70 2023-07-14T16:13:57.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc85d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc88074550 0x7fcc8819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:57.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc85d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc88074550 0x7fcc8819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60438/0 (socket says 172.21.15.62:60438) 2023-07-14T16:13:57.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc85d9b700 1 -- 172.21.15.62:0/2408375212 learned_addr learned my addr 172.21.15.62:0/2408375212 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:57.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc86d9d700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc8810c0c0 0x7fcc881a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:57.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc8659c700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc88072f70 0x7fcc8819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:57.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc85d9b700 1 -- 172.21.15.62:0/2408375212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc88072f70 msgr2=0x7fcc8819bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:57.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc85d9b700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc88072f70 0x7fcc8819bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc85d9b700 1 -- 172.21.15.62:0/2408375212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc8810c0c0 msgr2=0x7fcc881a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:57.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc85d9b700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc8810c0c0 0x7fcc881a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:57.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc85d9b700 1 -- 172.21.15.62:0/2408375212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0095f0 con 0x7fcc88074550 2023-07-14T16:13:57.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.492+0000 7fcc86d9d700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc8810c0c0 0x7fcc881a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:13:57.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.493+0000 7fcc85d9b700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc88074550 0x7fcc8819c320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0098e0 tx=0x7fcc7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:57.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.493+0000 7fcc777fe700 1 -- 172.21.15.62:0/2408375212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c01d070 con 0x7fcc88074550 2023-07-14T16:13:57.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.493+0000 7fcc777fe700 1 -- 172.21.15.62:0/2408375212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc7c00bdb0 con 0x7fcc88074550 2023-07-14T16:13:57.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.493+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc881a3eb0 con 0x7fcc88074550 2023-07-14T16:13:57.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.493+0000 7fcc777fe700 1 -- 172.21.15.62:0/2408375212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c022cf0 con 0x7fcc88074550 2023-07-14T16:13:57.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.493+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc881a4220 con 0x7fcc88074550 2023-07-14T16:13:57.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.494+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc88101c10 con 0x7fcc88074550 2023-07-14T16:13:57.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.497+0000 7fcc777fe700 1 -- 172.21.15.62:0/2408375212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcc7c022470 con 0x7fcc88074550 2023-07-14T16:13:57.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.498+0000 7fcc777fe700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcc700a1150 0x7fcc700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:57.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.499+0000 7fcc777fe700 1 -- 172.21.15.62:0/2408375212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fcc7c0ce0c0 con 0x7fcc88074550 2023-07-14T16:13:57.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.499+0000 7fcc8659c700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcc700a1150 0x7fcc700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:57.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.499+0000 7fcc8659c700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcc700a1150 0x7fcc700a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcc88074920 tx=0x7fcc78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.505+0000 7fcc777fe700 1 -- 172.21.15.62:0/2408375212 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcc7c097c50 con 0x7fcc88074550 2023-07-14T16:13:57.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:57 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2914605171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T16:13:57.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:57 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:57 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2914605171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T16:13:57.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:57 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:57 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2914605171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T16:13:57.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:57 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:57.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:57.721+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcc8802cc70 con 0x7fcc88074550 2023-07-14T16:13:58.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.263+0000 7fcc777fe700 1 -- 172.21.15.62:0/2408375212 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcc7c09b270 con 0x7fcc88074550 2023-07-14T16:13:58.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.279+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcc700a1150 msgr2=0x7fcc700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:58.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.279+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcc700a1150 0x7fcc700a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcc88074920 tx=0x7fcc78009260 comp rx=0 tx=0).stop 2023-07-14T16:13:58.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.279+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc88074550 msgr2=0x7fcc8819c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:58.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.279+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc88074550 0x7fcc8819c320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0098e0 tx=0x7fcc7c004730 comp rx=0 tx=0).stop 2023-07-14T16:13:58.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.280+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 shutdown_connections 2023-07-14T16:13:58.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.280+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcc700a1150 0x7fcc700a3600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.280+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc8810c0c0 0x7fcc881a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.280+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc88072f70 0x7fcc8819bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.280+0000 7fcc8c9ff700 1 --2- 172.21.15.62:0/2408375212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc88074550 0x7fcc8819c320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.280+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 >> 172.21.15.62:0/2408375212 conn(0x7fcc880fb130 msgr2=0x7fcc880fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:58.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.282+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 shutdown_connections 2023-07-14T16:13:58.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.283+0000 7fcc8c9ff700 1 -- 172.21.15.62:0/2408375212 wait complete. 2023-07-14T16:13:58.262 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2023-07-14T16:13:58.282 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-14T16:13:58.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.515+0000 7f9e341ff700 1 -- 172.21.15.62:0/3364445610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c1079a0 msgr2=0x7f9e2c109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:58.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.515+0000 7f9e341ff700 1 --2- 172.21.15.62:0/3364445610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c1079a0 0x7f9e2c109e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b0a0 tx=0x7f9e2400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.515+0000 7f9e341ff700 1 -- 172.21.15.62:0/3364445610 shutdown_connections 2023-07-14T16:13:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.515+0000 7f9e341ff700 1 --2- 172.21.15.62:0/3364445610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c1079a0 0x7f9e2c109e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.515+0000 7f9e341ff700 1 --2- 172.21.15.62:0/3364445610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e2c074550 0x7f9e2c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.515+0000 7f9e341ff700 1 --2- 172.21.15.62:0/3364445610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e2c072f70 0x7f9e2c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.515+0000 7f9e341ff700 1 -- 172.21.15.62:0/3364445610 >> 172.21.15.62:0/3364445610 conn(0x7f9e2c0fb0e0 msgr2=0x7f9e2c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.515+0000 7f9e341ff700 1 -- 172.21.15.62:0/3364445610 shutdown_connections 2023-07-14T16:13:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.516+0000 7f9e341ff700 1 -- 172.21.15.62:0/3364445610 wait complete. 2023-07-14T16:13:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.516+0000 7f9e341ff700 1 Processor -- start 2023-07-14T16:13:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e341ff700 1 -- start start 2023-07-14T16:13:58.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e341ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c072f70 0x7f9e2c197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e341ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e2c074550 0x7f9e2c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e341ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e2c1079a0 0x7f9e2c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e341ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c19f5f0 con 0x7f9e2c072f70 2023-07-14T16:13:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e341ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c19f730 con 0x7f9e2c074550 2023-07-14T16:13:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e341ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c19f8c0 con 0x7f9e2c1079a0 2023-07-14T16:13:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e31f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c072f70 0x7f9e2c197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:58.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e31f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c072f70 0x7f9e2c197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60460/0 (socket says 172.21.15.62:60460) 2023-07-14T16:13:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e31f9b700 1 -- 172.21.15.62:0/2425140010 learned_addr learned my addr 172.21.15.62:0/2425140010 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e3279c700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e2c1079a0 0x7f9e2c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e31f9b700 1 -- 172.21.15.62:0/2425140010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e2c1079a0 msgr2=0x7f9e2c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e31f9b700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e2c1079a0 0x7f9e2c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e3179a700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e2c074550 0x7f9e2c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:58.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e31f9b700 1 -- 172.21.15.62:0/2425140010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e2c074550 msgr2=0x7f9e2c197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:58.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.517+0000 7f9e31f9b700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e2c074550 0x7f9e2c197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:58.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.518+0000 7f9e31f9b700 1 -- 172.21.15.62:0/2425140010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e24009d00 con 0x7f9e2c072f70 2023-07-14T16:13:58.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.518+0000 7f9e3179a700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e2c074550 0x7f9e2c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:13:58.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.518+0000 7f9e31f9b700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c072f70 0x7f9e2c197a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800e970 tx=0x7f9e2800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:58.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.518+0000 7f9e22ffd700 1 -- 172.21.15.62:0/2425140010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28009790 con 0x7f9e2c072f70 2023-07-14T16:13:58.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.518+0000 7f9e22ffd700 1 -- 172.21.15.62:0/2425140010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e28004cc0 con 0x7f9e2c072f70 2023-07-14T16:13:58.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.518+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2c19fba0 con 0x7f9e2c072f70 2023-07-14T16:13:58.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.518+0000 7f9e22ffd700 1 -- 172.21.15.62:0/2425140010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280055b0 con 0x7f9e2c072f70 2023-07-14T16:13:58.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.518+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2c1a00b0 con 0x7f9e2c072f70 2023-07-14T16:13:58.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.519+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2c106050 con 0x7f9e2c072f70 2023-07-14T16:13:58.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.522+0000 7f9e22ffd700 1 -- 172.21.15.62:0/2425140010 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9e28004e30 con 0x7f9e2c072f70 2023-07-14T16:13:58.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.523+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e180a1150 0x7f9e180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:58.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.523+0000 7f9e22ffd700 1 -- 172.21.15.62:0/2425140010 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f9e28014070 con 0x7f9e2c072f70 2023-07-14T16:13:58.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.523+0000 7f9e3179a700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e180a1150 0x7f9e180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:58.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.524+0000 7f9e3179a700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e180a1150 0x7f9e180a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c074270 tx=0x7f9e1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:58.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.526+0000 7f9e22ffd700 1 -- 172.21.15.62:0/2425140010 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9e28096530 con 0x7f9e2c072f70 2023-07-14T16:13:58.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:58 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2914605171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T16:13:58.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:58 smithi164 ceph-mon[97427]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T16:13:58.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:58.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:58 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:58.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:58 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2914605171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T16:13:58.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:58 smithi062 ceph-mon[95543]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T16:13:58.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:58.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:58 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:58.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:58 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2914605171' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T16:13:58.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:58 smithi081 ceph-mon[127795]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T16:13:58.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:58.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:58 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:58.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:58.802+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9e2c02cb00 con 0x7f9e2c072f70 2023-07-14T16:13:59.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: pgmap v302: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:59.528 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2408375212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T16:13:59.528 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:59.528 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2408375212' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T16:13:59.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T16:13:59.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:59.529 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:59.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:59.531 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2425140010' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T16:13:59.531 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:13:59 smithi164 ceph-mon[97427]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T16:13:59.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.558+0000 7f9e22ffd700 1 -- 172.21.15.62:0/2425140010 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9e28099b50 con 0x7f9e2c072f70 2023-07-14T16:13:59.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e180a1150 msgr2=0x7f9e180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:59.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e180a1150 0x7f9e180a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c074270 tx=0x7f9e1c009260 comp rx=0 tx=0).stop 2023-07-14T16:13:59.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c072f70 msgr2=0x7f9e2c197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:59.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c072f70 0x7f9e2c197a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800e970 tx=0x7f9e2800ed30 comp rx=0 tx=0).stop 2023-07-14T16:13:59.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 shutdown_connections 2023-07-14T16:13:59.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e2c1079a0 0x7f9e2c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e2c072f70 0x7f9e2c197a70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e180a1150 0x7f9e180a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 --2- 172.21.15.62:0/2425140010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e2c074550 0x7f9e2c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.560+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 >> 172.21.15.62:0/2425140010 conn(0x7f9e2c0fb0e0 msgr2=0x7f9e2c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:59.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.561+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 shutdown_connections 2023-07-14T16:13:59.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.561+0000 7f9e341ff700 1 -- 172.21.15.62:0/2425140010 wait complete. 2023-07-14T16:13:59.541 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2023-07-14T16:13:59.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: pgmap v302: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:59.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2408375212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T16:13:59.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:59.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2408375212' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T16:13:59.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T16:13:59.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:59.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:59.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:59.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2425140010' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T16:13:59.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:13:59 smithi062 ceph-mon[95543]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T16:13:59.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:13:59.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: pgmap v302: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:13:59.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2408375212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T16:13:59.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:13:59.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2408375212' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T16:13:59.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T16:13:59.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:13:59.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:13:59.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:13:59.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2425140010' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T16:13:59.655 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:13:59 smithi081 ceph-mon[127795]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T16:13:59.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.810+0000 7fb8459ff700 1 -- 172.21.15.62:0/1556206116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8400ffb00 msgr2=0x7fb8400fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:59.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.810+0000 7fb8459ff700 1 --2- 172.21.15.62:0/1556206116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8400ffb00 0x7fb8400fff10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb830009990 tx=0x7fb830009ca0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.811+0000 7fb8459ff700 1 -- 172.21.15.62:0/1556206116 shutdown_connections 2023-07-14T16:13:59.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.811+0000 7fb8459ff700 1 --2- 172.21.15.62:0/1556206116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840101680 0x7fb840107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.811+0000 7fb8459ff700 1 --2- 172.21.15.62:0/1556206116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8400ffb00 0x7fb8400fff10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.811+0000 7fb8459ff700 1 --2- 172.21.15.62:0/1556206116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb840100cf0 0x7fb840101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.811+0000 7fb8459ff700 1 -- 172.21.15.62:0/1556206116 >> 172.21.15.62:0/1556206116 conn(0x7fb8400fb0e0 msgr2=0x7fb8400fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:13:59.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.811+0000 7fb8459ff700 1 -- 172.21.15.62:0/1556206116 shutdown_connections 2023-07-14T16:13:59.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.811+0000 7fb8459ff700 1 -- 172.21.15.62:0/1556206116 wait complete. 2023-07-14T16:13:59.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb8459ff700 1 Processor -- start 2023-07-14T16:13:59.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb8459ff700 1 -- start start 2023-07-14T16:13:59.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb8459ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400ffb00 0x7fb840197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:59.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb8459ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840100cf0 0x7fb840197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:59.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb8459ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840101680 0x7fb84019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:59.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb8459ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84019f500 con 0x7fb840100cf0 2023-07-14T16:13:59.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb8459ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84019f640 con 0x7fb840101680 2023-07-14T16:13:59.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb8459ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84019f7d0 con 0x7fb8400ffb00 2023-07-14T16:13:59.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb83e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840100cf0 0x7fb840197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:59.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb83e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840100cf0 0x7fb840197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60486/0 (socket says 172.21.15.62:60486) 2023-07-14T16:13:59.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb83e7fc700 1 -- 172.21.15.62:0/2418083847 learned_addr learned my addr 172.21.15.62:0/2418083847 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:13:59.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.812+0000 7fb83effd700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400ffb00 0x7fb840197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:59.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840101680 0x7fb84019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:59.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb83e7fc700 1 -- 172.21.15.62:0/2418083847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400ffb00 msgr2=0x7fb840197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:59.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb83e7fc700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400ffb00 0x7fb840197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb83e7fc700 1 -- 172.21.15.62:0/2418083847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840101680 msgr2=0x7fb84019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:13:59.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb83e7fc700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840101680 0x7fb84019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:13:59.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb83e7fc700 1 -- 172.21.15.62:0/2418083847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300095f0 con 0x7fb840100cf0 2023-07-14T16:13:59.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb83e7fc700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840100cf0 0x7fb840197ec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb83400e9a0 tx=0x7fb83400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:59.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb8449fd700 1 -- 172.21.15.62:0/2418083847 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834009790 con 0x7fb840100cf0 2023-07-14T16:13:59.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb8449fd700 1 -- 172.21.15.62:0/2418083847 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb834010430 con 0x7fb840100cf0 2023-07-14T16:13:59.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb8449fd700 1 -- 172.21.15.62:0/2418083847 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400cea0 con 0x7fb840100cf0 2023-07-14T16:13:59.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.813+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84019fab0 con 0x7fb840100cf0 2023-07-14T16:13:59.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.814+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84019ffc0 con 0x7fb840100cf0 2023-07-14T16:13:59.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.815+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb840192090 con 0x7fb840100cf0 2023-07-14T16:13:59.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.816+0000 7fb8449fd700 1 -- 172.21.15.62:0/2418083847 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb8340105a0 con 0x7fb840100cf0 2023-07-14T16:13:59.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.818+0000 7fb8449fd700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1220 0x7fb8280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:13:59.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.818+0000 7fb8449fd700 1 -- 172.21.15.62:0/2418083847 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb834014070 con 0x7fb840100cf0 2023-07-14T16:13:59.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.818+0000 7fb83effd700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1220 0x7fb8280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:13:59.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.819+0000 7fb83effd700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1220 0x7fb8280a36d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb830005910 tx=0x7fb83000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:13:59.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:13:59.821+0000 7fb8449fd700 1 -- 172.21.15.62:0/2418083847 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb834051790 con 0x7fb840100cf0 2023-07-14T16:14:00.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.045+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb84002c850 con 0x7fb840100cf0 2023-07-14T16:14:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.047+0000 7fb8449fd700 1 -- 172.21.15.62:0/2418083847 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22081 (secure 0 0 0) 0x7fb8340961c0 con 0x7fb840100cf0 2023-07-14T16:14:00.026 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:00.026 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":85,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:13:58.804818+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:14:00.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1220 msgr2=0x7fb8280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1220 0x7fb8280a36d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb830005910 tx=0x7fb83000b260 comp rx=0 tx=0).stop 2023-07-14T16:14:00.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840100cf0 msgr2=0x7fb840197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840100cf0 0x7fb840197ec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb83400e9a0 tx=0x7fb83400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 shutdown_connections 2023-07-14T16:14:00.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1220 0x7fb8280a36d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840101680 0x7fb84019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8400ffb00 0x7fb840197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 --2- 172.21.15.62:0/2418083847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840100cf0 0x7fb840197ec0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 >> 172.21.15.62:0/2418083847 conn(0x7fb8400fb0e0 msgr2=0x7fb8401065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 shutdown_connections 2023-07-14T16:14:00.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.049+0000 7fb8459ff700 1 -- 172.21.15.62:0/2418083847 wait complete. 2023-07-14T16:14:00.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:14:00.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 -- 172.21.15.62:0/2562012987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54100ffb00 msgr2=0x7f54100fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 --2- 172.21.15.62:0/2562012987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54100ffb00 0x7f54100fff10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5400009990 tx=0x7f5400009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 -- 172.21.15.62:0/2562012987 shutdown_connections 2023-07-14T16:14:00.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 --2- 172.21.15.62:0/2562012987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5410101680 0x7f5410107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 --2- 172.21.15.62:0/2562012987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54100ffb00 0x7f54100fff10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 --2- 172.21.15.62:0/2562012987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410100cf0 0x7f5410101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 -- 172.21.15.62:0/2562012987 >> 172.21.15.62:0/2562012987 conn(0x7f54100fb0e0 msgr2=0x7f54100fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:00.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 -- 172.21.15.62:0/2562012987 shutdown_connections 2023-07-14T16:14:00.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.292+0000 7f5415dff700 1 -- 172.21.15.62:0/2562012987 wait complete. 2023-07-14T16:14:00.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.294+0000 7f5415dff700 1 Processor -- start 2023-07-14T16:14:00.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.294+0000 7f5415dff700 1 -- start start 2023-07-14T16:14:00.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.294+0000 7f5415dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54100ffb00 0x7f5410195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:00.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f5415dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5410100cf0 0x7f5410195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f5415dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410101680 0x7f541019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f5415dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541019d2b0 con 0x7f54100ffb00 2023-07-14T16:14:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f5415dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541019d3f0 con 0x7f5410101680 2023-07-14T16:14:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f5415dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541019d580 con 0x7f5410100cf0 2023-07-14T16:14:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54100ffb00 0x7f5410195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54100ffb00 0x7f5410195730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60504/0 (socket says 172.21.15.62:60504) 2023-07-14T16:14:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540f7fe700 1 -- 172.21.15.62:0/1802487165 learned_addr learned my addr 172.21.15.62:0/1802487165 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540effd700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5410100cf0 0x7f5410195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540ffff700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410101680 0x7f541019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540f7fe700 1 -- 172.21.15.62:0/1802487165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5410100cf0 msgr2=0x7f5410195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540f7fe700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5410100cf0 0x7f5410195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540f7fe700 1 -- 172.21.15.62:0/1802487165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410101680 msgr2=0x7f541019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540f7fe700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410101680 0x7f541019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540f7fe700 1 -- 172.21.15.62:0/1802487165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000095f0 con 0x7f54100ffb00 2023-07-14T16:14:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.295+0000 7f540ffff700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410101680 0x7f541019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:14:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.296+0000 7f540f7fe700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54100ffb00 0x7f5410195730 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f54000098e0 tx=0x7f5400004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.296+0000 7f540cff9700 1 -- 172.21.15.62:0/1802487165 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540001d070 con 0x7f54100ffb00 2023-07-14T16:14:00.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.296+0000 7f540cff9700 1 -- 172.21.15.62:0/1802487165 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f540000be30 con 0x7f54100ffb00 2023-07-14T16:14:00.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.296+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541019d800 con 0x7f54100ffb00 2023-07-14T16:14:00.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.296+0000 7f540cff9700 1 -- 172.21.15.62:0/1802487165 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400022df0 con 0x7f54100ffb00 2023-07-14T16:14:00.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.296+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541019dc80 con 0x7f54100ffb00 2023-07-14T16:14:00.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.297+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5410106ab0 con 0x7f54100ffb00 2023-07-14T16:14:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.299+0000 7f540cff9700 1 -- 172.21.15.62:0/1802487165 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5400022470 con 0x7f54100ffb00 2023-07-14T16:14:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.301+0000 7f540cff9700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53f80a11a0 0x7f53f80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.301+0000 7f540cff9700 1 -- 172.21.15.62:0/1802487165 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54000ce080 con 0x7f54100ffb00 2023-07-14T16:14:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.301+0000 7f540effd700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53f80a11a0 0x7f53f80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.302+0000 7f540effd700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53f80a11a0 0x7f53f80a3650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5404009c60 tx=0x7f5404009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.303+0000 7f540cff9700 1 -- 172.21.15.62:0/1802487165 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f54000979b0 con 0x7f54100ffb00 2023-07-14T16:14:00.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:00 smithi062 ceph-mon[95543]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T16:14:00.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:00 smithi062 ceph-mon[95543]: pgmap v305: 97 pgs: 27 creating+peering, 18 active+clean, 52 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:14:00.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:00 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2425140010' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T16:14:00.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:00 smithi062 ceph-mon[95543]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T16:14:00.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:00 smithi062 ceph-mon[95543]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T16:14:00.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:00 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2418083847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:00.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:00 smithi164 ceph-mon[97427]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T16:14:00.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:00 smithi164 ceph-mon[97427]: pgmap v305: 97 pgs: 27 creating+peering, 18 active+clean, 52 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:14:00.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:00 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2425140010' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T16:14:00.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:00 smithi164 ceph-mon[97427]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T16:14:00.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:00 smithi164 ceph-mon[97427]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T16:14:00.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:00 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2418083847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:00.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.547+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f541002c850 con 0x7f54100ffb00 2023-07-14T16:14:00.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.549+0000 7f540cff9700 1 -- 172.21.15.62:0/1802487165 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22068 (secure 0 0 0) 0x7f540009afd0 con 0x7f54100ffb00 2023-07-14T16:14:00.528 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:00.528 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:00.277438+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:14:00.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.551+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53f80a11a0 msgr2=0x7f53f80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.551+0000 7f5415dff700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53f80a11a0 0x7f53f80a3650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5404009c60 tx=0x7f5404009260 comp rx=0 tx=0).stop 2023-07-14T16:14:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.551+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54100ffb00 msgr2=0x7f5410195730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.551+0000 7f5415dff700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54100ffb00 0x7f5410195730 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f54000098e0 tx=0x7f5400004730 comp rx=0 tx=0).stop 2023-07-14T16:14:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.552+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 shutdown_connections 2023-07-14T16:14:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.552+0000 7f5415dff700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53f80a11a0 0x7f53f80a3650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.552+0000 7f5415dff700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5410101680 0x7f541019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.552+0000 7f5415dff700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54100ffb00 0x7f5410195730 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.552+0000 7f5415dff700 1 --2- 172.21.15.62:0/1802487165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5410100cf0 0x7f5410195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.552+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 >> 172.21.15.62:0/1802487165 conn(0x7f54100fb0e0 msgr2=0x7f54100fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.552+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 shutdown_connections 2023-07-14T16:14:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.552+0000 7f5415dff700 1 -- 172.21.15.62:0/1802487165 wait complete. 2023-07-14T16:14:00.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-14T16:14:00.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:00 smithi081 ceph-mon[127795]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T16:14:00.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:00 smithi081 ceph-mon[127795]: pgmap v305: 97 pgs: 27 creating+peering, 18 active+clean, 52 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T16:14:00.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:00 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2425140010' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T16:14:00.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:00 smithi081 ceph-mon[127795]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T16:14:00.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:00 smithi081 ceph-mon[127795]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T16:14:00.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:00 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2418083847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:00.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.793+0000 7f7bd29ff700 1 -- 172.21.15.62:0/2362925958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc10c0b0 msgr2=0x7f7bcc10e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.793+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/2362925958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc10c0b0 0x7f7bcc10e490 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400b0a0 tx=0x7f7bc400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.793+0000 7f7bd29ff700 1 -- 172.21.15.62:0/2362925958 shutdown_connections 2023-07-14T16:14:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.793+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/2362925958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc10c0b0 0x7f7bcc10e490 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.793+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/2362925958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bcc074550 0x7f7bcc0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.793+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/2362925958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc072f70 0x7f7bcc0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.793+0000 7f7bd29ff700 1 -- 172.21.15.62:0/2362925958 >> 172.21.15.62:0/2362925958 conn(0x7f7bcc0fb0e0 msgr2=0x7f7bcc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.794+0000 7f7bd29ff700 1 -- 172.21.15.62:0/2362925958 shutdown_connections 2023-07-14T16:14:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.794+0000 7f7bd29ff700 1 -- 172.21.15.62:0/2362925958 wait complete. 2023-07-14T16:14:00.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.795+0000 7f7bd29ff700 1 Processor -- start 2023-07-14T16:14:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.795+0000 7f7bd29ff700 1 -- start start 2023-07-14T16:14:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bd29ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc072f70 0x7f7bcc19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:00.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bd29ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc074550 0x7f7bcc19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bd29ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bcc10c0b0 0x7f7bcc1a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bd29ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1a39f0 con 0x7f7bcc074550 2023-07-14T16:14:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bd29ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1a3b30 con 0x7f7bcc10c0b0 2023-07-14T16:14:00.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bd29ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1a3cc0 con 0x7f7bcc072f70 2023-07-14T16:14:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bcb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc074550 0x7f7bcc19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc072f70 0x7f7bcc19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bd0f9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bcc10c0b0 0x7f7bcc1a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bcbfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc072f70 0x7f7bcc19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55148/0 (socket says 172.21.15.62:55148) 2023-07-14T16:14:00.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.796+0000 7f7bcbfff700 1 -- 172.21.15.62:0/3997873318 learned_addr learned my addr 172.21.15.62:0/3997873318 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bcb7fe700 1 -- 172.21.15.62:0/3997873318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc072f70 msgr2=0x7f7bcc19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bcb7fe700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc072f70 0x7f7bcc19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bcb7fe700 1 -- 172.21.15.62:0/3997873318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bcc10c0b0 msgr2=0x7f7bcc1a3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bcb7fe700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bcc10c0b0 0x7f7bcc1a3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bcb7fe700 1 -- 172.21.15.62:0/3997873318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc0009520 con 0x7f7bcc074550 2023-07-14T16:14:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bcb7fe700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc074550 0x7f7bcc19c3b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000e9c0 tx=0x7f7bc000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bc97fa700 1 -- 172.21.15.62:0/3997873318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000cb70 con 0x7f7bcc074550 2023-07-14T16:14:00.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bc97fa700 1 -- 172.21.15.62:0/3997873318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bc000ee60 con 0x7f7bcc074550 2023-07-14T16:14:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc4009d00 con 0x7f7bcc074550 2023-07-14T16:14:00.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bc97fa700 1 -- 172.21.15.62:0/3997873318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0018680 con 0x7f7bcc074550 2023-07-14T16:14:00.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.797+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1a42e0 con 0x7f7bcc074550 2023-07-14T16:14:00.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.799+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc101c00 con 0x7f7bcc074550 2023-07-14T16:14:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.800+0000 7f7bc97fa700 1 -- 172.21.15.62:0/3997873318 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7bc0010430 con 0x7f7bcc074550 2023-07-14T16:14:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.803+0000 7f7bc97fa700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7bb40a11d0 0x7f7bb40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.803+0000 7f7bc97fa700 1 -- 172.21.15.62:0/3997873318 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7bc0014070 con 0x7f7bcc074550 2023-07-14T16:14:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.803+0000 7f7bcbfff700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7bb40a11d0 0x7f7bb40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.804+0000 7f7bcbfff700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7bb40a11d0 0x7f7bb40a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00bd10 tx=0x7f7bbc00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:00.807+0000 7f7bc97fa700 1 -- 172.21.15.62:0/3997873318 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7bc0096320 con 0x7f7bcc074550 2023-07-14T16:14:01.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.071+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7bcc02cc80 con 0x7f7bcc074550 2023-07-14T16:14:01.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:01 smithi164 ceph-mon[97427]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T16:14:01.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:01 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1802487165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:01.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:01 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] up:active 2023-07-14T16:14:01.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:01 smithi164 ceph-mon[97427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:01.530 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:01 smithi164 ceph-mon[97427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:01.531 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:01 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3997873318' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T16:14:01.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.570+0000 7f7bc97fa700 1 -- 172.21.15.62:0/3997873318 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7bc000ce60 con 0x7f7bcc074550 2023-07-14T16:14:01.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.572+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7bb40a11d0 msgr2=0x7f7bb40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:01.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7bb40a11d0 0x7f7bb40a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00bd10 tx=0x7f7bbc00b480 comp rx=0 tx=0).stop 2023-07-14T16:14:01.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc074550 msgr2=0x7f7bcc19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:01.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc074550 0x7f7bcc19c3b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000e9c0 tx=0x7f7bc000ecd0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 shutdown_connections 2023-07-14T16:14:01.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7bb40a11d0 0x7f7bb40a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bcc10c0b0 0x7f7bcc1a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bcc072f70 0x7f7bcc19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 --2- 172.21.15.62:0/3997873318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bcc074550 0x7f7bcc19c3b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 >> 172.21.15.62:0/3997873318 conn(0x7f7bcc0fb0e0 msgr2=0x7f7bcc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:01.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 shutdown_connections 2023-07-14T16:14:01.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.573+0000 7f7bd29ff700 1 -- 172.21.15.62:0/3997873318 wait complete. 2023-07-14T16:14:01.568 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:01 smithi062 ceph-mon[95543]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T16:14:01.568 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:01 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1802487165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:01.568 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:01 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] up:active 2023-07-14T16:14:01.569 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:01 smithi062 ceph-mon[95543]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:01.569 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:01 smithi062 ceph-mon[95543]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:01.569 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:01 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3997873318' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T16:14:01.570 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-07-14T16:14:01.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-07-14T16:14:01.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:01 smithi081 ceph-mon[127795]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T16:14:01.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:01 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1802487165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:01.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:01 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] up:active 2023-07-14T16:14:01.653 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:01 smithi081 ceph-mon[127795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:01.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:01 smithi081 ceph-mon[127795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:01.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:01 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3997873318' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T16:14:01.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 -- 172.21.15.62:0/123671969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f4100560 msgr2=0x7f88f4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/123671969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f4100560 0x7f88f4102940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009990 tx=0x7f88f0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 -- 172.21.15.62:0/123671969 shutdown_connections 2023-07-14T16:14:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/123671969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f41057a0 0x7f88f4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/123671969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f4100560 0x7f88f4102940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/123671969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88f4102e80 0x7f88f4105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 -- 172.21.15.62:0/123671969 >> 172.21.15.62:0/123671969 conn(0x7f88f40fa000 msgr2=0x7f88f40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 -- 172.21.15.62:0/123671969 shutdown_connections 2023-07-14T16:14:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.809+0000 7f88fb3ff700 1 -- 172.21.15.62:0/123671969 wait complete. 2023-07-14T16:14:01.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.810+0000 7f88fb3ff700 1 Processor -- start 2023-07-14T16:14:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88fb3ff700 1 -- start start 2023-07-14T16:14:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88fb3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f4100560 0x7f88f4195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88fb3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88f4102e80 0x7f88f4195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88fb3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f41057a0 0x7f88f419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88fb3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f419d2c0 con 0x7f88f41057a0 2023-07-14T16:14:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88fb3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f419d400 con 0x7f88f4102e80 2023-07-14T16:14:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88fb3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f419d590 con 0x7f88f4100560 2023-07-14T16:14:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88f999c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f41057a0 0x7f88f419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88f899a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88f4102e80 0x7f88f4195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88f999c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f41057a0 0x7f88f419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60542/0 (socket says 172.21.15.62:60542) 2023-07-14T16:14:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88f999c700 1 -- 172.21.15.62:0/2325449374 learned_addr learned my addr 172.21.15.62:0/2325449374 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.811+0000 7f88f919b700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f4100560 0x7f88f4195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88f999c700 1 -- 172.21.15.62:0/2325449374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f4100560 msgr2=0x7f88f4195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88f999c700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f4100560 0x7f88f4195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88f999c700 1 -- 172.21.15.62:0/2325449374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88f4102e80 msgr2=0x7f88f4195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88f999c700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88f4102e80 0x7f88f4195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88f999c700 1 -- 172.21.15.62:0/2325449374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e4009520 con 0x7f88f41057a0 2023-07-14T16:14:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88f999c700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f41057a0 0x7f88f419ccf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00ef30 tx=0x7f88ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88ea7fc700 1 -- 172.21.15.62:0/2325449374 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec018070 con 0x7f88f41057a0 2023-07-14T16:14:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88ea7fc700 1 -- 172.21.15.62:0/2325449374 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88ec00f040 con 0x7f88f41057a0 2023-07-14T16:14:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88ea7fc700 1 -- 172.21.15.62:0/2325449374 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00cbc0 con 0x7f88f41057a0 2023-07-14T16:14:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f00095f0 con 0x7f88f41057a0 2023-07-14T16:14:01.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.812+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f419db40 con 0x7f88f41057a0 2023-07-14T16:14:01.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.814+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f40fbb70 con 0x7f88f41057a0 2023-07-14T16:14:01.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.817+0000 7f88ea7fc700 1 -- 172.21.15.62:0/2325449374 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f88ec00cd80 con 0x7f88f41057a0 2023-07-14T16:14:01.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.818+0000 7f88ea7fc700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88e00a11d0 0x7f88e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:01.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.819+0000 7f88ea7fc700 1 -- 172.21.15.62:0/2325449374 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f88ec0cc280 con 0x7f88f41057a0 2023-07-14T16:14:01.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.819+0000 7f88f919b700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88e00a11d0 0x7f88e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:01.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.819+0000 7f88f919b700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88e00a11d0 0x7f88e00a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f88f00059d0 tx=0x7f88f0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:01.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:01.824+0000 7f88ea7fc700 1 -- 172.21.15.62:0/2325449374 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f88ec095b10 con 0x7f88f41057a0 2023-07-14T16:14:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.085+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f88f40611c0 con 0x7f88f41057a0 2023-07-14T16:14:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.579+0000 7f88ea7fc700 1 -- 172.21.15.62:0/2325449374 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f88ec099130 con 0x7f88f41057a0 2023-07-14T16:14:02.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.581+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88e00a11d0 msgr2=0x7f88e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:02.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.581+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88e00a11d0 0x7f88e00a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f88f00059d0 tx=0x7f88f0005960 comp rx=0 tx=0).stop 2023-07-14T16:14:02.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f41057a0 msgr2=0x7f88f419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:02.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f41057a0 0x7f88f419ccf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00ef30 tx=0x7f88ec00c3a0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 shutdown_connections 2023-07-14T16:14:02.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88f41057a0 0x7f88f419ccf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88f4100560 0x7f88f4195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88e00a11d0 0x7f88e00a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 --2- 172.21.15.62:0/2325449374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88f4102e80 0x7f88f4195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 >> 172.21.15.62:0/2325449374 conn(0x7f88f40fa000 msgr2=0x7f88f41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:02.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 shutdown_connections 2023-07-14T16:14:02.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.582+0000 7f88fb3ff700 1 -- 172.21.15.62:0/2325449374 wait complete. 2023-07-14T16:14:02.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-14T16:14:02.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.811+0000 7f80343ff700 1 -- 172.21.15.62:0/2114947473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c0ffaf0 msgr2=0x7f802c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:02.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.811+0000 7f80343ff700 1 --2- 172.21.15.62:0/2114947473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c0ffaf0 0x7f802c0fff00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8028009990 tx=0x7f8028009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.811+0000 7f80343ff700 1 -- 172.21.15.62:0/2114947473 shutdown_connections 2023-07-14T16:14:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.811+0000 7f80343ff700 1 --2- 172.21.15.62:0/2114947473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f802c101670 0x7f802c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.811+0000 7f80343ff700 1 --2- 172.21.15.62:0/2114947473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c0ffaf0 0x7f802c0fff00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.811+0000 7f80343ff700 1 --2- 172.21.15.62:0/2114947473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f802c100ce0 0x7f802c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.811+0000 7f80343ff700 1 -- 172.21.15.62:0/2114947473 >> 172.21.15.62:0/2114947473 conn(0x7f802c0fb130 msgr2=0x7f802c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.812+0000 7f80343ff700 1 -- 172.21.15.62:0/2114947473 shutdown_connections 2023-07-14T16:14:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.812+0000 7f80343ff700 1 -- 172.21.15.62:0/2114947473 wait complete. 2023-07-14T16:14:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.812+0000 7f80343ff700 1 Processor -- start 2023-07-14T16:14:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f80343ff700 1 -- start start 2023-07-14T16:14:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f80343ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f802c0ffaf0 0x7f802c195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f80343ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c100ce0 0x7f802c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f80343ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f802c101670 0x7f802c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f80343ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19d380 con 0x7f802c100ce0 2023-07-14T16:14:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f80343ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19d4c0 con 0x7f802c0ffaf0 2023-07-14T16:14:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f80343ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19d650 con 0x7f802c101670 2023-07-14T16:14:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f803199a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c100ce0 0x7f802c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:02.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f803199a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c100ce0 0x7f802c195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46062/0 (socket says 172.21.15.62:46062) 2023-07-14T16:14:02.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f803199a700 1 -- 172.21.15.62:0/441701141 learned_addr learned my addr 172.21.15.62:0/441701141 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:02.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.813+0000 7f803219b700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f802c0ffaf0 0x7f802c195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:02.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803299c700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f802c101670 0x7f802c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:02.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803199a700 1 -- 172.21.15.62:0/441701141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f802c101670 msgr2=0x7f802c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:02.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803199a700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f802c101670 0x7f802c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803199a700 1 -- 172.21.15.62:0/441701141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f802c0ffaf0 msgr2=0x7f802c195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803199a700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f802c0ffaf0 0x7f802c195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803199a700 1 -- 172.21.15.62:0/441701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024009c40 con 0x7f802c100ce0 2023-07-14T16:14:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803219b700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f802c0ffaf0 0x7f802c195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803299c700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f802c101670 0x7f802c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:14:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f803199a700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c100ce0 0x7f802c195d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f801c00e9a0 tx=0x7f801c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f80237fe700 1 -- 172.21.15.62:0/441701141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c009790 con 0x7f802c100ce0 2023-07-14T16:14:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f80237fe700 1 -- 172.21.15.62:0/441701141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f801c00cbd0 con 0x7f802c100ce0 2023-07-14T16:14:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80280095f0 con 0x7f802c100ce0 2023-07-14T16:14:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f80237fe700 1 -- 172.21.15.62:0/441701141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c018680 con 0x7f802c100ce0 2023-07-14T16:14:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.814+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c19dcb0 con 0x7f802c100ce0 2023-07-14T16:14:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.815+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c10b1c0 con 0x7f802c100ce0 2023-07-14T16:14:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.818+0000 7f80237fe700 1 -- 172.21.15.62:0/441701141 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f801c00cd40 con 0x7f802c100ce0 2023-07-14T16:14:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.819+0000 7f80237fe700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80180a1150 0x7f80180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:02.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.819+0000 7f80237fe700 1 -- 172.21.15.62:0/441701141 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f801c014070 con 0x7f802c100ce0 2023-07-14T16:14:02.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.819+0000 7f803219b700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80180a1150 0x7f80180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:02.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.820+0000 7f803219b700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80180a1150 0x7f80180a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8028005910 tx=0x7f802800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:02.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:02.823+0000 7f80237fe700 1 -- 172.21.15.62:0/441701141 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f801c051790 con 0x7f802c100ce0 2023-07-14T16:14:02.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: pgmap v307: 97 pgs: 27 creating+peering, 62 active+clean, 8 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 3 op/s 2023-07-14T16:14:02.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3997873318' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T16:14:02.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:02.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2325449374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T16:14:02.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:14:02.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:14:02.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: pgmap v307: 97 pgs: 27 creating+peering, 62 active+clean, 8 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 3 op/s 2023-07-14T16:14:02.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3997873318' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T16:14:02.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:02.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2325449374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T16:14:02.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:02.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:14:02.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:14:02.907 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:03.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: pgmap v307: 97 pgs: 27 creating+peering, 62 active+clean, 8 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 3 op/s 2023-07-14T16:14:03.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3997873318' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T16:14:03.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:03.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2325449374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T16:14:03.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:03.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:03.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:03.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:03.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:03.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:03.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:14:03.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:14:03.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:14:03.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.082+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f802c106aa0 con 0x7f802c100ce0 2023-07-14T16:14:03.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.557+0000 7f80237fe700 1 -- 172.21.15.62:0/441701141 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f801c0963a0 con 0x7f802c100ce0 2023-07-14T16:14:03.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80180a1150 msgr2=0x7f80180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:03.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80180a1150 0x7f80180a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8028005910 tx=0x7f802800b260 comp rx=0 tx=0).stop 2023-07-14T16:14:03.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c100ce0 msgr2=0x7f802c195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:03.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c100ce0 0x7f802c195d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f801c00e9a0 tx=0x7f801c00ed60 comp rx=0 tx=0).stop 2023-07-14T16:14:03.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 shutdown_connections 2023-07-14T16:14:03.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f802c101670 0x7f802c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80180a1150 0x7f80180a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f802c0ffaf0 0x7f802c195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 --2- 172.21.15.62:0/441701141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f802c100ce0 0x7f802c195d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 >> 172.21.15.62:0/441701141 conn(0x7f802c0fb130 msgr2=0x7f802c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:03.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 shutdown_connections 2023-07-14T16:14:03.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.560+0000 7f80343ff700 1 -- 172.21.15.62:0/441701141 wait complete. 2023-07-14T16:14:03.564 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-14T16:14:03.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 -- 172.21.15.62:0/1665549380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c101660 msgr2=0x7fc31c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:03.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 --2- 172.21.15.62:0/1665549380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c101660 0x7fc31c107f20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc31400b0a0 tx=0x7fc31400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 -- 172.21.15.62:0/1665549380 shutdown_connections 2023-07-14T16:14:03.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 --2- 172.21.15.62:0/1665549380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c101660 0x7fc31c107f20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 --2- 172.21.15.62:0/1665549380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc31c0ffae0 0x7fc31c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 --2- 172.21.15.62:0/1665549380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c100cd0 0x7fc31c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 -- 172.21.15.62:0/1665549380 >> 172.21.15.62:0/1665549380 conn(0x7fc31c0fb0e0 msgr2=0x7fc31c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:03.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 -- 172.21.15.62:0/1665549380 shutdown_connections 2023-07-14T16:14:03.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.794+0000 7fc3221ff700 1 -- 172.21.15.62:0/1665549380 wait complete. 2023-07-14T16:14:03.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.795+0000 7fc3221ff700 1 Processor -- start 2023-07-14T16:14:03.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.795+0000 7fc3221ff700 1 -- start start 2023-07-14T16:14:03.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.795+0000 7fc3221ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc31c0ffae0 0x7fc31c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:03.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.795+0000 7fc3221ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c100cd0 0x7fc31c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:03.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc3221ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101660 0x7fc31c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:03.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc3221ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c0735e0 con 0x7fc31c100cd0 2023-07-14T16:14:03.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc3221ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c073750 con 0x7fc31c0ffae0 2023-07-14T16:14:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc3221ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c19f900 con 0x7fc31c101660 2023-07-14T16:14:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c100cd0 0x7fc31c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c100cd0 0x7fc31c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46078/0 (socket says 172.21.15.62:46078) 2023-07-14T16:14:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 -- 172.21.15.62:0/2852209013 learned_addr learned my addr 172.21.15.62:0/2852209013 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31b7fe700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc31c0ffae0 0x7fc31c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31bfff700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101660 0x7fc31c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 -- 172.21.15.62:0/2852209013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101660 msgr2=0x7fc31c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101660 0x7fc31c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 -- 172.21.15.62:0/2852209013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc31c0ffae0 msgr2=0x7fc31c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc31c0ffae0 0x7fc31c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 -- 172.21.15.62:0/2852209013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc314009d00 con 0x7fc31c100cd0 2023-07-14T16:14:03.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.796+0000 7fc31affd700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c100cd0 0x7fc31c072900 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc31000d950 tx=0x7fc31000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:03.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.797+0000 7fc318ff9700 1 -- 172.21.15.62:0/2852209013 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310009940 con 0x7fc31c100cd0 2023-07-14T16:14:03.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.797+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31c19fbe0 con 0x7fc31c100cd0 2023-07-14T16:14:03.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.797+0000 7fc318ff9700 1 -- 172.21.15.62:0/2852209013 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc310010460 con 0x7fc31c100cd0 2023-07-14T16:14:03.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.797+0000 7fc318ff9700 1 -- 172.21.15.62:0/2852209013 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000f620 con 0x7fc31c100cd0 2023-07-14T16:14:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.797+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31c1a0140 con 0x7fc31c100cd0 2023-07-14T16:14:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.798+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31c10b1c0 con 0x7fc31c100cd0 2023-07-14T16:14:03.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.800+0000 7fc318ff9700 1 -- 172.21.15.62:0/2852209013 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc3100105d0 con 0x7fc31c100cd0 2023-07-14T16:14:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.801+0000 7fc318ff9700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3040a1150 0x7fc3040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.802+0000 7fc318ff9700 1 -- 172.21.15.62:0/2852209013 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3100cc6d0 con 0x7fc31c100cd0 2023-07-14T16:14:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.802+0000 7fc31b7fe700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3040a1150 0x7fc3040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.802+0000 7fc31b7fe700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3040a1150 0x7fc3040a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc31c100b30 tx=0x7fc30c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:03.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:03.806+0000 7fc318ff9700 1 -- 172.21.15.62:0/2852209013 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc310095ed0 con 0x7fc31c100cd0 2023-07-14T16:14:03.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T16:14:03.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2325449374' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T16:14:03.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T16:14:03.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:03.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T16:14:03.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T16:14:03.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/441701141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T16:14:03.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] up:active 2023-07-14T16:14:03.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/441701141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T16:14:03.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T16:14:03.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T16:14:03.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: Cluster is now healthy 2023-07-14T16:14:03.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-14T16:14:03.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:03 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-07-14T16:14:03.903 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:14:03.776+0000 7fdd03f85700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T16:14:03.903 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:14:03.776+0000 7fdd03f85700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-07-14T16:14:03.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T16:14:03.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2325449374' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T16:14:03.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T16:14:03.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:03.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T16:14:03.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T16:14:03.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/441701141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T16:14:03.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] up:active 2023-07-14T16:14:03.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/441701141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T16:14:03.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T16:14:03.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T16:14:03.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: Cluster is now healthy 2023-07-14T16:14:03.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-14T16:14:03.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:03 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-07-14T16:14:04.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T16:14:04.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2325449374' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T16:14:04.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T16:14:04.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T16:14:04.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-14T16:14:04.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-14T16:14:04.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/441701141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T16:14:04.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] up:active 2023-07-14T16:14:04.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/441701141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T16:14:04.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T16:14:04.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T16:14:04.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: Cluster is now healthy 2023-07-14T16:14:04.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-14T16:14:04.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:03 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-07-14T16:14:04.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.066+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc31c10b3b0 con 0x7fc31c100cd0 2023-07-14T16:14:04.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.589+0000 7fc318ff9700 1 -- 172.21.15.62:0/2852209013 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc3100994f0 con 0x7fc31c100cd0 2023-07-14T16:14:04.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3040a1150 msgr2=0x7fc3040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:04.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3040a1150 0x7fc3040a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc31c100b30 tx=0x7fc30c00b410 comp rx=0 tx=0).stop 2023-07-14T16:14:04.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c100cd0 msgr2=0x7fc31c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:04.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c100cd0 0x7fc31c072900 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc31000d950 tx=0x7fc31000dc60 comp rx=0 tx=0).stop 2023-07-14T16:14:04.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 shutdown_connections 2023-07-14T16:14:04.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc31c101660 0x7fc31c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3040a1150 0x7fc3040a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc31c0ffae0 0x7fc31c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 --2- 172.21.15.62:0/2852209013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc31c100cd0 0x7fc31c072900 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 >> 172.21.15.62:0/2852209013 conn(0x7fc31c0fb0e0 msgr2=0x7fc31c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:04.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 shutdown_connections 2023-07-14T16:14:04.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.592+0000 7fc3221ff700 1 -- 172.21.15.62:0/2852209013 wait complete. 2023-07-14T16:14:04.590 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-07-14T16:14:04.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-14T16:14:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 -- 172.21.15.62:0/2697028046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 msgr2=0x7f740c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 --2- 172.21.15.62:0/2697028046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 0x7f740c0ffef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7408009990 tx=0x7f7408009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 -- 172.21.15.62:0/2697028046 shutdown_connections 2023-07-14T16:14:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 --2- 172.21.15.62:0/2697028046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740c101660 0x7f740c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 --2- 172.21.15.62:0/2697028046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 0x7f740c0ffef0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 --2- 172.21.15.62:0/2697028046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f740c100cd0 0x7f740c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 -- 172.21.15.62:0/2697028046 >> 172.21.15.62:0/2697028046 conn(0x7f740c0fb0e0 msgr2=0x7f740c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 -- 172.21.15.62:0/2697028046 shutdown_connections 2023-07-14T16:14:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.823+0000 7f74133ff700 1 -- 172.21.15.62:0/2697028046 wait complete. 2023-07-14T16:14:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.824+0000 7f74133ff700 1 Processor -- start 2023-07-14T16:14:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.824+0000 7f74133ff700 1 -- start start 2023-07-14T16:14:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.824+0000 7f74133ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 0x7f740c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f74133ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f740c100cd0 0x7f740c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f74133ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740c101660 0x7f740c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f74133ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c19f510 con 0x7f740c0ffae0 2023-07-14T16:14:04.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f74133ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c19f650 con 0x7f740c100cd0 2023-07-14T16:14:04.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f74133ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c19f7e0 con 0x7f740c101660 2023-07-14T16:14:04.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 0x7f740c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:04.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 0x7f740c197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46096/0 (socket says 172.21.15.62:46096) 2023-07-14T16:14:04.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 -- 172.21.15.62:0/2910296366 learned_addr learned my addr 172.21.15.62:0/2910296366 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:04.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741199c700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740c101660 0x7f740c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:04.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 -- 172.21.15.62:0/2910296366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740c101660 msgr2=0x7f740c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:04.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740c101660 0x7f740c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 -- 172.21.15.62:0/2910296366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f740c100cd0 msgr2=0x7f740c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:04.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f740c100cd0 0x7f740c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:04.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 -- 172.21.15.62:0/2910296366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080095f0 con 0x7f740c0ffae0 2023-07-14T16:14:04.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741199c700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740c101660 0x7f740c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:04.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.825+0000 7f741119b700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 0x7f740c197990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f74080098e0 tx=0x7f7408005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:04.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.826+0000 7f74027fc700 1 -- 172.21.15.62:0/2910296366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740801d070 con 0x7f740c0ffae0 2023-07-14T16:14:04.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.826+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c19fa60 con 0x7f740c0ffae0 2023-07-14T16:14:04.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.826+0000 7f74027fc700 1 -- 172.21.15.62:0/2910296366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f740800bed0 con 0x7f740c0ffae0 2023-07-14T16:14:04.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.826+0000 7f74027fc700 1 -- 172.21.15.62:0/2910296366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740800f460 con 0x7f740c0ffae0 2023-07-14T16:14:04.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.826+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c19fe90 con 0x7f740c0ffae0 2023-07-14T16:14:04.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.827+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c10b1b0 con 0x7f740c0ffae0 2023-07-14T16:14:04.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.829+0000 7f74027fc700 1 -- 172.21.15.62:0/2910296366 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f740800ba40 con 0x7f740c0ffae0 2023-07-14T16:14:04.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.831+0000 7f74027fc700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73f80a1150 0x7f73f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:04.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.831+0000 7f74027fc700 1 -- 172.21.15.62:0/2910296366 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f74080cdcc0 con 0x7f740c0ffae0 2023-07-14T16:14:04.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.831+0000 7f741099a700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73f80a1150 0x7f73f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:04.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.832+0000 7f741099a700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73f80a1150 0x7f73f80a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73fc005950 tx=0x7f73fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:04.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:04.835+0000 7f74027fc700 1 -- 172.21.15.62:0/2910296366 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f740805c790 con 0x7f740c0ffae0 2023-07-14T16:14:04.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:04 smithi062 ceph-mon[95543]: pgmap v308: 97 pgs: 10 creating+peering, 87 active+clean; 580 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 6 op/s 2023-07-14T16:14:04.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:04 smithi062 ceph-mon[95543]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-07-14T16:14:04.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:04 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2852209013' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T16:14:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:04 smithi081 ceph-mon[127795]: pgmap v308: 97 pgs: 10 creating+peering, 87 active+clean; 580 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 6 op/s 2023-07-14T16:14:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:04 smithi081 ceph-mon[127795]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-07-14T16:14:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:04 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2852209013' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T16:14:05.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:04 smithi164 ceph-mon[97427]: pgmap v308: 97 pgs: 10 creating+peering, 87 active+clean; 580 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 6 op/s 2023-07-14T16:14:05.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:04 smithi164 ceph-mon[97427]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-07-14T16:14:05.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:04 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2852209013' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T16:14:05.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.075+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f740c107720 con 0x7f73f80a1150 2023-07-14T16:14:05.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.121+0000 7f74027fc700 1 -- 172.21.15.62:0/2910296366 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f740c107720 con 0x7f73f80a1150 2023-07-14T16:14:05.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.123+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73f80a1150 msgr2=0x7f73f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.123+0000 7f74133ff700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73f80a1150 0x7f73f80a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73fc005950 tx=0x7f73fc0058e0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.123+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 msgr2=0x7f740c197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.123+0000 7f74133ff700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 0x7f740c197990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f74080098e0 tx=0x7f7408005ef0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.123+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 shutdown_connections 2023-07-14T16:14:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.124+0000 7f74133ff700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740c101660 0x7f740c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.124+0000 7f74133ff700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f740c0ffae0 0x7f740c197990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.124+0000 7f74133ff700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f73f80a1150 0x7f73f80a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.124+0000 7f74133ff700 1 --2- 172.21.15.62:0/2910296366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f740c100cd0 0x7f740c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.124+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 >> 172.21.15.62:0/2910296366 conn(0x7f740c0fb0e0 msgr2=0x7f740c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.124+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 shutdown_connections 2023-07-14T16:14:05.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.124+0000 7f74133ff700 1 -- 172.21.15.62:0/2910296366 wait complete. 2023-07-14T16:14:05.120 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-14T16:14:05.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 -- 172.21.15.62:0/4075191111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 msgr2=0x7f2c840729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 --2- 172.21.15.62:0/4075191111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 0x7f2c840729a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009990 tx=0x7f2c74009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 -- 172.21.15.62:0/4075191111 shutdown_connections 2023-07-14T16:14:05.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 --2- 172.21.15.62:0/4075191111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c841079b0 0x7f2c84109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 --2- 172.21.15.62:0/4075191111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 0x7f2c840729a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 --2- 172.21.15.62:0/4075191111 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c84072f70 0x7f2c840733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 -- 172.21.15.62:0/4075191111 >> 172.21.15.62:0/4075191111 conn(0x7f2c840fb130 msgr2=0x7f2c840fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 -- 172.21.15.62:0/4075191111 shutdown_connections 2023-07-14T16:14:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.349+0000 7f2c88fff700 1 -- 172.21.15.62:0/4075191111 wait complete. 2023-07-14T16:14:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.350+0000 7f2c88fff700 1 Processor -- start 2023-07-14T16:14:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c88fff700 1 -- start start 2023-07-14T16:14:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c88fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c84072f70 0x7f2c841979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:05.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c88fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 0x7f2c84197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:05.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c88fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c841079b0 0x7f2c8419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:05.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c88fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8419f530 con 0x7f2c84074550 2023-07-14T16:14:05.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c88fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8419f670 con 0x7f2c841079b0 2023-07-14T16:14:05.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c88fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8419f800 con 0x7f2c84072f70 2023-07-14T16:14:05.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c81d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 0x7f2c84197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:05.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c81d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 0x7f2c84197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46130/0 (socket says 172.21.15.62:46130) 2023-07-14T16:14:05.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c81d9b700 1 -- 172.21.15.62:0/3738536679 learned_addr learned my addr 172.21.15.62:0/3738536679 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:05.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c82d9d700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c841079b0 0x7f2c8419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:05.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.351+0000 7f2c8259c700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c84072f70 0x7f2c841979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:05.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c81d9b700 1 -- 172.21.15.62:0/3738536679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c84072f70 msgr2=0x7f2c841979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c81d9b700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c84072f70 0x7f2c841979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c81d9b700 1 -- 172.21.15.62:0/3738536679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c841079b0 msgr2=0x7f2c8419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c81d9b700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c841079b0 0x7f2c8419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c81d9b700 1 -- 172.21.15.62:0/3738536679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740095f0 con 0x7f2c84074550 2023-07-14T16:14:05.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c81d9b700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 0x7f2c84197ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800d980 tx=0x7f2c7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:05.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c737fe700 1 -- 172.21.15.62:0/3738536679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800b950 con 0x7f2c84074550 2023-07-14T16:14:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c737fe700 1 -- 172.21.15.62:0/3738536679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c78007620 con 0x7f2c84074550 2023-07-14T16:14:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8419fae0 con 0x7f2c84074550 2023-07-14T16:14:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c737fe700 1 -- 172.21.15.62:0/3738536679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800f3c0 con 0x7f2c84074550 2023-07-14T16:14:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.352+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c841a0040 con 0x7f2c84074550 2023-07-14T16:14:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.353+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c84106060 con 0x7f2c84074550 2023-07-14T16:14:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.355+0000 7f2c737fe700 1 -- 172.21.15.62:0/3738536679 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c7800bab0 con 0x7f2c84074550 2023-07-14T16:14:05.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.358+0000 7f2c737fe700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c6c0a11d0 0x7f2c6c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.358+0000 7f2c737fe700 1 -- 172.21.15.62:0/3738536679 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c780cc8d0 con 0x7f2c84074550 2023-07-14T16:14:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.358+0000 7f2c8259c700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c6c0a11d0 0x7f2c6c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.358+0000 7f2c8259c700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c6c0a11d0 0x7f2c6c0a3680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c74005910 tx=0x7f2c7400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.362+0000 7f2c737fe700 1 -- 172.21.15.62:0/3738536679 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c78096170 con 0x7f2c84074550 2023-07-14T16:14:05.374 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:14:05.082+0000 7fdb2534f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T16:14:05.375 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:14:05.082+0000 7fdb2534f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T16:14:05.375 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:14:05.082+0000 7fdb2534f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T16:14:05.375 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:14:05.082+0000 7fdb2534f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T16:14:05.376 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:14:05.082+0000 7fdb2534f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T16:14:05.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.610+0000 7f2c737fe700 1 -- 172.21.15.62:0/3738536679 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2c78096540 con 0x7f2c84074550 2023-07-14T16:14:05.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.610+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c84060950 con 0x7f2c6c0a11d0 2023-07-14T16:14:05.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.615+0000 7f2c737fe700 1 -- 172.21.15.62:0/3738536679 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2c84060950 con 0x7f2c6c0a11d0 2023-07-14T16:14:05.593 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 2023-07-14T16:14:05.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c6c0a11d0 msgr2=0x7f2c6c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c6c0a11d0 0x7f2c6c0a3680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c74005910 tx=0x7f2c7400b260 comp rx=0 tx=0).stop 2023-07-14T16:14:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 msgr2=0x7f2c84197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 0x7f2c84197ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800d980 tx=0x7f2c7800dc90 comp rx=0 tx=0).stop 2023-07-14T16:14:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 shutdown_connections 2023-07-14T16:14:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c6c0a11d0 0x7f2c6c0a3680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c841079b0 0x7f2c8419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c84072f70 0x7f2c841979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 --2- 172.21.15.62:0/3738536679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c84074550 0x7f2c84197ef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.617+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 >> 172.21.15.62:0/3738536679 conn(0x7f2c840fb130 msgr2=0x7f2c840fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:05.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.618+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 shutdown_connections 2023-07-14T16:14:05.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.618+0000 7f2c88fff700 1 -- 172.21.15.62:0/3738536679 wait complete. 2023-07-14T16:14:05.614 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-14T16:14:05.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 -- 172.21.15.62:0/1655714323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c1079a0 msgr2=0x7fbd0c109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 --2- 172.21.15.62:0/1655714323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c1079a0 0x7fbd0c109e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400b0a0 tx=0x7fbd0400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 -- 172.21.15.62:0/1655714323 shutdown_connections 2023-07-14T16:14:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 --2- 172.21.15.62:0/1655714323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c1079a0 0x7fbd0c109e20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 --2- 172.21.15.62:0/1655714323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd0c074550 0x7fbd0c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 --2- 172.21.15.62:0/1655714323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd0c072f70 0x7fbd0c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 -- 172.21.15.62:0/1655714323 >> 172.21.15.62:0/1655714323 conn(0x7fbd0c0fb0e0 msgr2=0x7fbd0c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 -- 172.21.15.62:0/1655714323 shutdown_connections 2023-07-14T16:14:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.844+0000 7fbd121ff700 1 -- 172.21.15.62:0/1655714323 wait complete. 2023-07-14T16:14:05.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.845+0000 7fbd121ff700 1 Processor -- start 2023-07-14T16:14:05.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd121ff700 1 -- start start 2023-07-14T16:14:05.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd121ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd0c072f70 0x7fbd0c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:05.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd121ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c074550 0x7fbd0c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:05.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd121ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd0c1079a0 0x7fbd0c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:05.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd121ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c19f640 con 0x7fbd0c074550 2023-07-14T16:14:05.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd121ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c19f780 con 0x7fbd0c072f70 2023-07-14T16:14:05.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd121ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c19f910 con 0x7fbd0c1079a0 2023-07-14T16:14:05.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c074550 0x7fbd0c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:05.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c074550 0x7fbd0c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46160/0 (socket says 172.21.15.62:46160) 2023-07-14T16:14:05.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.846+0000 7fbd0affd700 1 -- 172.21.15.62:0/4256603943 learned_addr learned my addr 172.21.15.62:0/4256603943 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:05.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0b7fe700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd0c072f70 0x7fbd0c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:05.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0bfff700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd0c1079a0 0x7fbd0c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:05.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0affd700 1 -- 172.21.15.62:0/4256603943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd0c1079a0 msgr2=0x7fbd0c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0affd700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd0c1079a0 0x7fbd0c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0affd700 1 -- 172.21.15.62:0/4256603943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd0c072f70 msgr2=0x7fbd0c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:05.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0affd700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd0c072f70 0x7fbd0c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:05.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0affd700 1 -- 172.21.15.62:0/4256603943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04009d00 con 0x7fbd0c074550 2023-07-14T16:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0bfff700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd0c1079a0 0x7fbd0c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd0affd700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c074550 0x7fbd0c198050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d980 tx=0x7fbd0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd08ff9700 1 -- 172.21.15.62:0/4256603943 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000b950 con 0x7fbd0c074550 2023-07-14T16:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd08ff9700 1 -- 172.21.15.62:0/4256603943 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd00007620 con 0x7fbd0c074550 2023-07-14T16:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c19fbf0 con 0x7fbd0c074550 2023-07-14T16:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd08ff9700 1 -- 172.21.15.62:0/4256603943 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000f3c0 con 0x7fbd0c074550 2023-07-14T16:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.847+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c1a0100 con 0x7fbd0c074550 2023-07-14T16:14:05.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.848+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c106050 con 0x7fbd0c074550 2023-07-14T16:14:05.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.850+0000 7fbd08ff9700 1 -- 172.21.15.62:0/4256603943 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbd0000bab0 con 0x7fbd0c074550 2023-07-14T16:14:05.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.852+0000 7fbd08ff9700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbcf40a11e0 0x7fbcf40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:05.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.852+0000 7fbd08ff9700 1 -- 172.21.15.62:0/4256603943 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbd000ccad0 con 0x7fbd0c074550 2023-07-14T16:14:05.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.852+0000 7fbd0b7fe700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbcf40a11e0 0x7fbcf40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:05.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.852+0000 7fbd0b7fe700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbcf40a11e0 0x7fbcf40a3690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc005950 tx=0x7fbcfc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:05.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:05.856+0000 7fbd08ff9700 1 -- 172.21.15.62:0/4256603943 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbd00096270 con 0x7fbd0c074550 2023-07-14T16:14:05.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:active 2023-07-14T16:14:05.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2852209013' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T16:14:05.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T16:14:05.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:05 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:14:05.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:05 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:active 2023-07-14T16:14:05.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:05 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2852209013' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T16:14:05.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:05 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T16:14:05.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:05 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:14:06.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:05 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:active 2023-07-14T16:14:06.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:05 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2852209013' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T16:14:06.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:05 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T16:14:06.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:05 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:14:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.095+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd0c0609e0 con 0x7fbcf40a11e0 2023-07-14T16:14:06.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.126+0000 7fbd08ff9700 1 -- 172.21.15.62:0/4256603943 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbd0c0609e0 con 0x7fbcf40a11e0 2023-07-14T16:14:06.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbcf40a11e0 msgr2=0x7fbcf40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbcf40a11e0 0x7fbcf40a3690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc005950 tx=0x7fbcfc00b410 comp rx=0 tx=0).stop 2023-07-14T16:14:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c074550 msgr2=0x7fbd0c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c074550 0x7fbd0c198050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d980 tx=0x7fbd0000dc90 comp rx=0 tx=0).stop 2023-07-14T16:14:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 shutdown_connections 2023-07-14T16:14:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd0c1079a0 0x7fbd0c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbcf40a11e0 0x7fbcf40a3690 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd0c072f70 0x7fbd0c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 --2- 172.21.15.62:0/4256603943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd0c074550 0x7fbd0c198050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 >> 172.21.15.62:0/4256603943 conn(0x7fbd0c0fb0e0 msgr2=0x7fbd0c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:06.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 shutdown_connections 2023-07-14T16:14:06.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.128+0000 7fbd121ff700 1 -- 172.21.15.62:0/4256603943 wait complete. 2023-07-14T16:14:06.124 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-14T16:14:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.354+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4154361496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 msgr2=0x7fc9dc101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.354+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4154361496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 0x7fc9dc101120 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009990 tx=0x7fc9cc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.354+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4154361496 shutdown_connections 2023-07-14T16:14:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.354+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4154361496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9dc101660 0x7fc9dc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.354+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4154361496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9dc0ffae0 0x7fc9dc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.354+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4154361496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 0x7fc9dc101120 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.354+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4154361496 >> 172.21.15.62:0/4154361496 conn(0x7fc9dc0fb0e0 msgr2=0x7fc9dc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.355+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4154361496 shutdown_connections 2023-07-14T16:14:06.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.355+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4154361496 wait complete. 2023-07-14T16:14:06.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e3dff700 1 Processor -- start 2023-07-14T16:14:06.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e3dff700 1 -- start start 2023-07-14T16:14:06.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e3dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9dc0ffae0 0x7fc9dc1957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:06.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e3dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 0x7fc9dc195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:06.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e3dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9dc101660 0x7fc9dc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:06.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e3dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc19d360 con 0x7fc9dc100cd0 2023-07-14T16:14:06.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e3dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc19d4a0 con 0x7fc9dc101660 2023-07-14T16:14:06.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e3dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc19d630 con 0x7fc9dc0ffae0 2023-07-14T16:14:06.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e139a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 0x7fc9dc195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:06.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e139a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 0x7fc9dc195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46180/0 (socket says 172.21.15.62:46180) 2023-07-14T16:14:06.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e139a700 1 -- 172.21.15.62:0/4007733042 learned_addr learned my addr 172.21.15.62:0/4007733042 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:06.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.356+0000 7fc9e1b9b700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9dc0ffae0 0x7fc9dc1957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:06.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9e239c700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9dc101660 0x7fc9dc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:06.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9e139a700 1 -- 172.21.15.62:0/4007733042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9dc0ffae0 msgr2=0x7fc9dc1957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9e139a700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9dc0ffae0 0x7fc9dc1957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9e139a700 1 -- 172.21.15.62:0/4007733042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9dc101660 msgr2=0x7fc9dc19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9e139a700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9dc101660 0x7fc9dc19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9e139a700 1 -- 172.21.15.62:0/4007733042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc0095f0 con 0x7fc9dc100cd0 2023-07-14T16:14:06.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9e139a700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 0x7fc9dc195d20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc000c00 tx=0x7fc9cc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:06.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9d2ffd700 1 -- 172.21.15.62:0/4007733042 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc01d070 con 0x7fc9dc100cd0 2023-07-14T16:14:06.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc19d8b0 con 0x7fc9dc100cd0 2023-07-14T16:14:06.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9d2ffd700 1 -- 172.21.15.62:0/4007733042 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9cc003ec0 con 0x7fc9dc100cd0 2023-07-14T16:14:06.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.357+0000 7fc9d2ffd700 1 -- 172.21.15.62:0/4007733042 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc022ac0 con 0x7fc9dc100cd0 2023-07-14T16:14:06.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.358+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc19dd60 con 0x7fc9dc100cd0 2023-07-14T16:14:06.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.358+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc10b230 con 0x7fc9dc100cd0 2023-07-14T16:14:06.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.360+0000 7fc9d2ffd700 1 -- 172.21.15.62:0/4007733042 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc9cc004030 con 0x7fc9dc100cd0 2023-07-14T16:14:06.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.362+0000 7fc9d2ffd700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9c80a11e0 0x7fc9c80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:06.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.362+0000 7fc9d2ffd700 1 -- 172.21.15.62:0/4007733042 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc9cc0cdf80 con 0x7fc9dc100cd0 2023-07-14T16:14:06.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.362+0000 7fc9e1b9b700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9c80a11e0 0x7fc9c80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:06.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.362+0000 7fc9e1b9b700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9c80a11e0 0x7fc9c80a3690 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc100b30 tx=0x7fc9d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:06.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.366+0000 7fc9d2ffd700 1 -- 172.21.15.62:0/4007733042 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc9cc0977b0 con 0x7fc9dc100cd0 2023-07-14T16:14:06.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.612+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9dc060950 con 0x7fc9c80a11e0 2023-07-14T16:14:06.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.616+0000 7fc9d2ffd700 1 -- 172.21.15.62:0/4007733042 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fc9dc060950 con 0x7fc9c80a11e0 2023-07-14T16:14:06.594 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/_nogroup/sv_1/97c402a8-6468-4e08-a480-fd81b2e91a6d 2023-07-14T16:14:06.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9c80a11e0 msgr2=0x7fc9c80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9c80a11e0 0x7fc9c80a3690 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc100b30 tx=0x7fc9d800b410 comp rx=0 tx=0).stop 2023-07-14T16:14:06.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 msgr2=0x7fc9dc195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 0x7fc9dc195d20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc000c00 tx=0x7fc9cc005740 comp rx=0 tx=0).stop 2023-07-14T16:14:06.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 shutdown_connections 2023-07-14T16:14:06.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9c80a11e0 0x7fc9c80a3690 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9dc101660 0x7fc9dc19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9dc0ffae0 0x7fc9dc1957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 --2- 172.21.15.62:0/4007733042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9dc100cd0 0x7fc9dc195d20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 >> 172.21.15.62:0/4007733042 conn(0x7fc9dc0fb0e0 msgr2=0x7fc9dc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:06.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 shutdown_connections 2023-07-14T16:14:06.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.618+0000 7fc9e3dff700 1 -- 172.21.15.62:0/4007733042 wait complete. 2023-07-14T16:14:06.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:14:06.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 -- 172.21.15.62:0/1318147420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19740ffb00 msgr2=0x7f19740fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 --2- 172.21.15.62:0/1318147420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19740ffb00 0x7f19740fff10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1964009990 tx=0x7f1964009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 -- 172.21.15.62:0/1318147420 shutdown_connections 2023-07-14T16:14:06.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 --2- 172.21.15.62:0/1318147420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1974101680 0x7f1974107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 --2- 172.21.15.62:0/1318147420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19740ffb00 0x7f19740fff10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 --2- 172.21.15.62:0/1318147420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1974100cf0 0x7f1974101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 -- 172.21.15.62:0/1318147420 >> 172.21.15.62:0/1318147420 conn(0x7f19740fb0e0 msgr2=0x7f19740fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:06.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 -- 172.21.15.62:0/1318147420 shutdown_connections 2023-07-14T16:14:06.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.842+0000 7f19799ff700 1 -- 172.21.15.62:0/1318147420 wait complete. 2023-07-14T16:14:06.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.843+0000 7f19799ff700 1 Processor -- start 2023-07-14T16:14:06.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.843+0000 7f19799ff700 1 -- start start 2023-07-14T16:14:06.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.843+0000 7f19799ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19740ffb00 0x7f19740742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:06.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19799ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1974100cf0 0x7f1974072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19799ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1974101680 0x7f1974072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19799ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19740735e0 con 0x7f1974101680 2023-07-14T16:14:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19799ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974073750 con 0x7f1974100cf0 2023-07-14T16:14:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19799ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197419f830 con 0x7f19740ffb00 2023-07-14T16:14:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1974101680 0x7f1974072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1974101680 0x7f1974072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46194/0 (socket says 172.21.15.62:46194) 2023-07-14T16:14:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 -- 172.21.15.62:0/1027444529 learned_addr learned my addr 172.21.15.62:0/1027444529 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:06.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19727fc700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1974100cf0 0x7f1974072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:06.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f1972ffd700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19740ffb00 0x7f19740742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:06.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 -- 172.21.15.62:0/1027444529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19740ffb00 msgr2=0x7f19740742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19740ffb00 0x7f19740742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 -- 172.21.15.62:0/1027444529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1974100cf0 msgr2=0x7f1974072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:06.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1974100cf0 0x7f1974072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:06.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 -- 172.21.15.62:0/1027444529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640095f0 con 0x7f1974101680 2023-07-14T16:14:06.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.844+0000 7f19737fe700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1974101680 0x7f1974072e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f196c00ef30 tx=0x7f196c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:06.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.845+0000 7f19789fd700 1 -- 172.21.15.62:0/1027444529 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c018070 con 0x7f1974101680 2023-07-14T16:14:06.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.845+0000 7f19789fd700 1 -- 172.21.15.62:0/1027444529 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f196c00f040 con 0x7f1974101680 2023-07-14T16:14:06.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.845+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197419fab0 con 0x7f1974101680 2023-07-14T16:14:06.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.845+0000 7f19789fd700 1 -- 172.21.15.62:0/1027444529 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c00cc70 con 0x7f1974101680 2023-07-14T16:14:06.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.845+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19741a0010 con 0x7f1974101680 2023-07-14T16:14:06.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.846+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19741070e0 con 0x7f1974101680 2023-07-14T16:14:06.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.848+0000 7f19789fd700 1 -- 172.21.15.62:0/1027444529 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f196c007500 con 0x7f1974101680 2023-07-14T16:14:06.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.849+0000 7f19789fd700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f195c0a12a0 0x7f195c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:06.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.850+0000 7f19789fd700 1 -- 172.21.15.62:0/1027444529 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f196c0cc400 con 0x7f1974101680 2023-07-14T16:14:06.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.850+0000 7f1972ffd700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f195c0a12a0 0x7f195c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:06.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.850+0000 7f1972ffd700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f195c0a12a0 0x7f195c0a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f19640098e0 tx=0x7f196400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:06.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:06.853+0000 7f19789fd700 1 -- 172.21.15.62:0/1027444529 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f196c095b30 con 0x7f1974101680 2023-07-14T16:14:06.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:06 smithi062 ceph-mon[95543]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:06.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:06 smithi062 ceph-mon[95543]: pgmap v309: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2023-07-14T16:14:06.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:06 smithi062 ceph-mon[95543]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T16:14:06.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:06 smithi062 ceph-mon[95543]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:06 smithi081 ceph-mon[127795]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:06 smithi081 ceph-mon[127795]: pgmap v309: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2023-07-14T16:14:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:06 smithi081 ceph-mon[127795]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T16:14:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:06 smithi081 ceph-mon[127795]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:06 smithi164 ceph-mon[97427]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:06 smithi164 ceph-mon[97427]: pgmap v309: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2023-07-14T16:14:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:06 smithi164 ceph-mon[97427]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-14T16:14:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:06 smithi164 ceph-mon[97427]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:07.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.106+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f197402c850 con 0x7f1974101680 2023-07-14T16:14:07.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.106+0000 7f19789fd700 1 -- 172.21.15.62:0/1027444529 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f196c099150 con 0x7f1974101680 2023-07-14T16:14:07.085 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:07.085 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:14:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f195c0a12a0 msgr2=0x7f195c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f195c0a12a0 0x7f195c0a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f19640098e0 tx=0x7f196400b260 comp rx=0 tx=0).stop 2023-07-14T16:14:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1974101680 msgr2=0x7f1974072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1974101680 0x7f1974072e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f196c00ef30 tx=0x7f196c00c3a0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 shutdown_connections 2023-07-14T16:14:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1974101680 0x7f1974072e40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19740ffb00 0x7f19740742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f195c0a12a0 0x7f195c0a3750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 --2- 172.21.15.62:0/1027444529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1974100cf0 0x7f1974072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 >> 172.21.15.62:0/1027444529 conn(0x7f19740fb0e0 msgr2=0x7f1974104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 shutdown_connections 2023-07-14T16:14:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.109+0000 7f19799ff700 1 -- 172.21.15.62:0/1027444529 wait complete. 2023-07-14T16:14:07.089 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:14:07.106 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:14:07.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 -- 172.21.15.62:0/3593314897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c0ffb00 msgr2=0x7f050c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 --2- 172.21.15.62:0/3593314897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c0ffb00 0x7f050c0fff10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009990 tx=0x7f04fc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 -- 172.21.15.62:0/3593314897 shutdown_connections 2023-07-14T16:14:07.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 --2- 172.21.15.62:0/3593314897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f050c101680 0x7f050c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 --2- 172.21.15.62:0/3593314897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c0ffb00 0x7f050c0fff10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 --2- 172.21.15.62:0/3593314897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f050c100cf0 0x7f050c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 -- 172.21.15.62:0/3593314897 >> 172.21.15.62:0/3593314897 conn(0x7f050c0fb0e0 msgr2=0x7f050c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:07.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 -- 172.21.15.62:0/3593314897 shutdown_connections 2023-07-14T16:14:07.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.332+0000 7f05117ff700 1 -- 172.21.15.62:0/3593314897 wait complete. 2023-07-14T16:14:07.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.333+0000 7f05117ff700 1 Processor -- start 2023-07-14T16:14:07.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.333+0000 7f05117ff700 1 -- start start 2023-07-14T16:14:07.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f05117ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f050c0ffb00 0x7f050c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:07.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f05117ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f050c100cf0 0x7f050c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:07.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f05117ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c101680 0x7f050c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:07.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f05117ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c19f500 con 0x7f050c101680 2023-07-14T16:14:07.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f05117ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c19f640 con 0x7f050c0ffb00 2023-07-14T16:14:07.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f05117ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c19f7d0 con 0x7f050c100cf0 2023-07-14T16:14:07.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c101680 0x7f050c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:07.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c101680 0x7f050c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46206/0 (socket says 172.21.15.62:46206) 2023-07-14T16:14:07.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050b7fe700 1 -- 172.21.15.62:0/347501096 learned_addr learned my addr 172.21.15.62:0/347501096 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:07.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050affd700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f050c0ffb00 0x7f050c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050a7fc700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f050c100cf0 0x7f050c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050b7fe700 1 -- 172.21.15.62:0/347501096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f050c100cf0 msgr2=0x7f050c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050b7fe700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f050c100cf0 0x7f050c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050b7fe700 1 -- 172.21.15.62:0/347501096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f050c0ffb00 msgr2=0x7f050c197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050b7fe700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f050c0ffb00 0x7f050c197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050b7fe700 1 -- 172.21.15.62:0/347501096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc0095f0 con 0x7f050c101680 2023-07-14T16:14:07.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050affd700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f050c0ffb00 0x7f050c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.334+0000 7f050a7fc700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f050c100cf0 0x7f050c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:14:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.335+0000 7f050b7fe700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c101680 0x7f050c19ef30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0504007ee0 tx=0x7f050400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.335+0000 7f04f3fff700 1 -- 172.21.15.62:0/347501096 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504017070 con 0x7f050c101680 2023-07-14T16:14:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.335+0000 7f04f3fff700 1 -- 172.21.15.62:0/347501096 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f050400f040 con 0x7f050c101680 2023-07-14T16:14:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.335+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c19fab0 con 0x7f050c101680 2023-07-14T16:14:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.335+0000 7f04f3fff700 1 -- 172.21.15.62:0/347501096 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504005710 con 0x7f050c101680 2023-07-14T16:14:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.335+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c1a0010 con 0x7f050c101680 2023-07-14T16:14:07.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.336+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c192090 con 0x7f050c101680 2023-07-14T16:14:07.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.338+0000 7f04f3fff700 1 -- 172.21.15.62:0/347501096 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0504005870 con 0x7f050c101680 2023-07-14T16:14:07.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.340+0000 7f04f3fff700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04f40a12a0 0x7f04f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:07.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.340+0000 7f04f3fff700 1 -- 172.21.15.62:0/347501096 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f05040cc1d0 con 0x7f050c101680 2023-07-14T16:14:07.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.340+0000 7f050affd700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04f40a12a0 0x7f04f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:07.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.340+0000 7f050affd700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04f40a12a0 0x7f04f40a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f04fc005910 tx=0x7f04fc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.343+0000 7f04f3fff700 1 -- 172.21.15.62:0/347501096 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0504095870 con 0x7f050c101680 2023-07-14T16:14:07.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.554+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f050c02c850 con 0x7f050c101680 2023-07-14T16:14:07.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.556+0000 7f04f3fff700 1 -- 172.21.15.62:0/347501096 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22068 (secure 0 0 0) 0x7f0504098e90 con 0x7f050c101680 2023-07-14T16:14:07.535 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:07.535 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:00.277438+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:14:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04f40a12a0 msgr2=0x7f04f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04f40a12a0 0x7f04f40a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f04fc005910 tx=0x7f04fc00b260 comp rx=0 tx=0).stop 2023-07-14T16:14:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c101680 msgr2=0x7f050c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c101680 0x7f050c19ef30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0504007ee0 tx=0x7f050400d3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 shutdown_connections 2023-07-14T16:14:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f050c101680 0x7f050c19ef30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04f40a12a0 0x7f04f40a3750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f050c0ffb00 0x7f050c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 --2- 172.21.15.62:0/347501096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f050c100cf0 0x7f050c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 >> 172.21.15.62:0/347501096 conn(0x7f050c0fb0e0 msgr2=0x7f050c1065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 shutdown_connections 2023-07-14T16:14:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.558+0000 7f05117ff700 1 -- 172.21.15.62:0/347501096 wait complete. 2023-07-14T16:14:07.556 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-14T16:14:07.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T16:14:07.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 -- 172.21.15.62:0/1567159176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2440107990 msgr2=0x7f2440109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 --2- 172.21.15.62:0/1567159176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2440107990 0x7f2440109e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f243800b0a0 tx=0x7f243800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 -- 172.21.15.62:0/1567159176 shutdown_connections 2023-07-14T16:14:07.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 --2- 172.21.15.62:0/1567159176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2440107990 0x7f2440109e10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 --2- 172.21.15.62:0/1567159176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24400744f0 0x7f2440072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 --2- 172.21.15.62:0/1567159176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2440072f10 0x7f2440073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 -- 172.21.15.62:0/1567159176 >> 172.21.15.62:0/1567159176 conn(0x7f24400fb0e0 msgr2=0x7f24400fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:07.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 -- 172.21.15.62:0/1567159176 shutdown_connections 2023-07-14T16:14:07.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.783+0000 7f2447fff700 1 -- 172.21.15.62:0/1567159176 wait complete. 2023-07-14T16:14:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.784+0000 7f2447fff700 1 Processor -- start 2023-07-14T16:14:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.784+0000 7f2447fff700 1 -- start start 2023-07-14T16:14:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.784+0000 7f2447fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2440072f10 0x7f24401979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.784+0000 7f2447fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24400744f0 0x7f2440197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.784+0000 7f2447fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2440107990 0x7f244019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.784+0000 7f2447fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019f530 con 0x7f24400744f0 2023-07-14T16:14:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.784+0000 7f2447fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019f670 con 0x7f2440107990 2023-07-14T16:14:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.784+0000 7f2447fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019f800 con 0x7f2440072f10 2023-07-14T16:14:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24400744f0 0x7f2440197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24400744f0 0x7f2440197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46230/0 (socket says 172.21.15.62:46230) 2023-07-14T16:14:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 -- 172.21.15.62:0/1671392870 learned_addr learned my addr 172.21.15.62:0/1671392870 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f2445d9b700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2440072f10 0x7f24401979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244659c700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2440107990 0x7f244019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 -- 172.21.15.62:0/1671392870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2440072f10 msgr2=0x7f24401979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2440072f10 0x7f24401979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 -- 172.21.15.62:0/1671392870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2440107990 msgr2=0x7f244019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2440107990 0x7f244019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 -- 172.21.15.62:0/1671392870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2438009d00 con 0x7f24400744f0 2023-07-14T16:14:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.785+0000 7f244559a700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24400744f0 0x7f2440197ef0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f243000e9a0 tx=0x7f243000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.786+0000 7f2436ffd700 1 -- 172.21.15.62:0/1671392870 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430009790 con 0x7f24400744f0 2023-07-14T16:14:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.786+0000 7f2436ffd700 1 -- 172.21.15.62:0/1671392870 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f243000cbd0 con 0x7f24400744f0 2023-07-14T16:14:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.786+0000 7f2436ffd700 1 -- 172.21.15.62:0/1671392870 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430018680 con 0x7f24400744f0 2023-07-14T16:14:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.786+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244019fae0 con 0x7f24400744f0 2023-07-14T16:14:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.786+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244019ffa0 con 0x7f24400744f0 2023-07-14T16:14:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.787+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24401060d0 con 0x7f24400744f0 2023-07-14T16:14:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.789+0000 7f2436ffd700 1 -- 172.21.15.62:0/1671392870 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f243000cd40 con 0x7f24400744f0 2023-07-14T16:14:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.790+0000 7f2436ffd700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f242c0a1220 0x7f242c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.790+0000 7f2436ffd700 1 -- 172.21.15.62:0/1671392870 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2430014070 con 0x7f24400744f0 2023-07-14T16:14:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.790+0000 7f2445d9b700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f242c0a1220 0x7f242c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.791+0000 7f2445d9b700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f242c0a1220 0x7f242c0a36d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f243c009c60 tx=0x7f243c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.794+0000 7f2436ffd700 1 -- 172.21.15.62:0/1671392870 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2430051790 con 0x7f24400744f0 2023-07-14T16:14:07.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:07 smithi062 ceph-mon[95543]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:07.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:07 smithi062 ceph-mon[95543]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:07.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:07 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1027444529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:14:07.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:07 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/347501096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:07 smithi081 ceph-mon[127795]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:07 smithi081 ceph-mon[127795]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:07 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1027444529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:14:07.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:07 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/347501096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:07.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:07.997+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2440060950 con 0x7f242c0a1220 2023-07-14T16:14:07.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.011+0000 7f2436ffd700 1 -- 172.21.15.62:0/1671392870 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+338939 (secure 0 0 0) 0x7f2440060950 con 0x7f242c0a1220 2023-07-14T16:14:07.994 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:07.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f242c0a1220 msgr2=0x7f242c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f242c0a1220 0x7f242c0a36d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f243c009c60 tx=0x7f243c009260 comp rx=0 tx=0).stop 2023-07-14T16:14:07.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24400744f0 msgr2=0x7f2440197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:07.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24400744f0 0x7f2440197ef0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f243000e9a0 tx=0x7f243000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 shutdown_connections 2023-07-14T16:14:07.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f242c0a1220 0x7f242c0a36d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2440107990 0x7f244019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2440072f10 0x7f24401979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.015+0000 7f2447fff700 1 --2- 172.21.15.62:0/1671392870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24400744f0 0x7f2440197ef0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:07.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.016+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 >> 172.21.15.62:0/1671392870 conn(0x7f24400fb0e0 msgr2=0x7f24400fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:07.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.016+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 shutdown_connections 2023-07-14T16:14:07.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.016+0000 7f2447fff700 1 -- 172.21.15.62:0/1671392870 wait complete. 2023-07-14T16:14:07.999 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-07-14T16:14:08.013 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2023-07-14T16:14:07.467601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595569,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":140,"num_read_kb":122,"num_write":302,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334584,"kb_used_data":11160,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607240,"statfs":{"total":1151940427776,"available":1151597813760,"internally_reserved":0,"allocated":11427840,"data_stored":7097643,"data_compressed":107328,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5201,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":121,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.190004"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290756+0000","last_change":"2023-07-14T16:13:59.301892+0000","last_active":"2023-07-14T16:14:00.290756+0000","last_peered":"2023-07-14T16:14:00.290756+0000","last_clean":"2023-07-14T16:14:00.290756+0000","last_became_active":"2023-07-14T16:13:59.301379+0000","last_became_peered":"2023-07-14T16:13:59.301379+0000","last_unstale":"2023-07-14T16:14:00.290756+0000","last_undegraded":"2023-07-14T16:14:00.290756+0000","last_fullsized":"2023-07-14T16:14:00.290756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:49:42.914040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173960+0000","last_change":"2023-07-14T16:13:59.309322+0000","last_active":"2023-07-14T16:14:01.173960+0000","last_peered":"2023-07-14T16:14:01.173960+0000","last_clean":"2023-07-14T16:14:01.173960+0000","last_became_active":"2023-07-14T16:13:59.309035+0000","last_became_peered":"2023-07-14T16:13:59.309035+0000","last_unstale":"2023-07-14T16:14:01.173960+0000","last_undegraded":"2023-07-14T16:14:01.173960+0000","last_fullsized":"2023-07-14T16:14:01.173960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:12.977618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174298+0000","last_change":"2023-07-14T16:13:59.304371+0000","last_active":"2023-07-14T16:14:01.174298+0000","last_peered":"2023-07-14T16:14:01.174298+0000","last_clean":"2023-07-14T16:14:01.174298+0000","last_became_active":"2023-07-14T16:13:59.303965+0000","last_became_peered":"2023-07-14T16:13:59.303965+0000","last_unstale":"2023-07-14T16:14:01.174298+0000","last_undegraded":"2023-07-14T16:14:01.174298+0000","last_fullsized":"2023-07-14T16:14:01.174298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:54.688081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173597+0000","last_change":"2023-07-14T16:13:59.303434+0000","last_active":"2023-07-14T16:14:01.173597+0000","last_peered":"2023-07-14T16:14:01.173597+0000","last_clean":"2023-07-14T16:14:01.173597+0000","last_became_active":"2023-07-14T16:13:59.302960+0000","last_became_peered":"2023-07-14T16:13:59.302960+0000","last_unstale":"2023-07-14T16:14:01.173597+0000","last_undegraded":"2023-07-14T16:14:01.173597+0000","last_fullsized":"2023-07-14T16:14:01.173597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:16.564005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175087+0000","last_change":"2023-07-14T16:13:59.298094+0000","last_active":"2023-07-14T16:14:01.175087+0000","last_peered":"2023-07-14T16:14:01.175087+0000","last_clean":"2023-07-14T16:14:01.175087+0000","last_became_active":"2023-07-14T16:13:59.297764+0000","last_became_peered":"2023-07-14T16:13:59.297764+0000","last_unstale":"2023-07-14T16:14:01.175087+0000","last_undegraded":"2023-07-14T16:14:01.175087+0000","last_fullsized":"2023-07-14T16:14:01.175087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:54:57.248080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577317+0000","last_change":"2023-07-14T16:13:59.299735+0000","last_active":"2023-07-14T16:14:00.577317+0000","last_peered":"2023-07-14T16:14:00.577317+0000","last_clean":"2023-07-14T16:14:00.577317+0000","last_became_active":"2023-07-14T16:13:59.299413+0000","last_became_peered":"2023-07-14T16:13:59.299413+0000","last_unstale":"2023-07-14T16:14:00.577317+0000","last_undegraded":"2023-07-14T16:14:00.577317+0000","last_fullsized":"2023-07-14T16:14:00.577317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:43.699831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.991748+0000","last_change":"2023-07-14T16:13:59.299682+0000","last_active":"2023-07-14T16:14:00.991748+0000","last_peered":"2023-07-14T16:14:00.991748+0000","last_clean":"2023-07-14T16:14:00.991748+0000","last_became_active":"2023-07-14T16:13:59.299327+0000","last_became_peered":"2023-07-14T16:13:59.299327+0000","last_unstale":"2023-07-14T16:14:00.991748+0000","last_undegraded":"2023-07-14T16:14:00.991748+0000","last_fullsized":"2023-07-14T16:14:00.991748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:44.359819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290178+0000","last_change":"2023-07-14T16:13:59.311818+0000","last_active":"2023-07-14T16:14:00.290178+0000","last_peered":"2023-07-14T16:14:00.290178+0000","last_clean":"2023-07-14T16:14:00.290178+0000","last_became_active":"2023-07-14T16:13:59.311478+0000","last_became_peered":"2023-07-14T16:13:59.311478+0000","last_unstale":"2023-07-14T16:14:00.290178+0000","last_undegraded":"2023-07-14T16:14:00.290178+0000","last_fullsized":"2023-07-14T16:14:00.290178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:20.336283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286736+0000","last_change":"2023-07-14T16:13:59.309665+0000","last_active":"2023-07-14T16:14:00.286736+0000","last_peered":"2023-07-14T16:14:00.286736+0000","last_clean":"2023-07-14T16:14:00.286736+0000","last_became_active":"2023-07-14T16:13:59.309361+0000","last_became_peered":"2023-07-14T16:13:59.309361+0000","last_unstale":"2023-07-14T16:14:00.286736+0000","last_undegraded":"2023-07-14T16:14:00.286736+0000","last_fullsized":"2023-07-14T16:14:00.286736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:35:51.164062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577723+0000","last_change":"2023-07-14T16:13:59.296397+0000","last_active":"2023-07-14T16:14:00.577723+0000","last_peered":"2023-07-14T16:14:00.577723+0000","last_clean":"2023-07-14T16:14:00.577723+0000","last_became_active":"2023-07-14T16:13:59.295965+0000","last_became_peered":"2023-07-14T16:13:59.295965+0000","last_unstale":"2023-07-14T16:14:00.577723+0000","last_undegraded":"2023-07-14T16:14:00.577723+0000","last_fullsized":"2023-07-14T16:14:00.577723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:07.343379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992110+0000","last_change":"2023-07-14T16:13:59.302892+0000","last_active":"2023-07-14T16:14:00.992110+0000","last_peered":"2023-07-14T16:14:00.992110+0000","last_clean":"2023-07-14T16:14:00.992110+0000","last_became_active":"2023-07-14T16:13:59.302295+0000","last_became_peered":"2023-07-14T16:13:59.302295+0000","last_unstale":"2023-07-14T16:14:00.992110+0000","last_undegraded":"2023-07-14T16:14:00.992110+0000","last_fullsized":"2023-07-14T16:14:00.992110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:39:08.544022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173544+0000","last_change":"2023-07-14T16:13:59.301913+0000","last_active":"2023-07-14T16:14:01.173544+0000","last_peered":"2023-07-14T16:14:01.173544+0000","last_clean":"2023-07-14T16:14:01.173544+0000","last_became_active":"2023-07-14T16:13:59.301590+0000","last_became_peered":"2023-07-14T16:13:59.301590+0000","last_unstale":"2023-07-14T16:14:01.173544+0000","last_undegraded":"2023-07-14T16:14:01.173544+0000","last_fullsized":"2023-07-14T16:14:01.173544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:59:58.070130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658997+0000","last_change":"2023-07-14T16:13:59.302006+0000","last_active":"2023-07-14T16:14:00.658997+0000","last_peered":"2023-07-14T16:14:00.658997+0000","last_clean":"2023-07-14T16:14:00.658997+0000","last_became_active":"2023-07-14T16:13:59.301752+0000","last_became_peered":"2023-07-14T16:13:59.301752+0000","last_unstale":"2023-07-14T16:14:00.658997+0000","last_undegraded":"2023-07-14T16:14:00.658997+0000","last_fullsized":"2023-07-14T16:14:00.658997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:16.314105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174059+0000","last_change":"2023-07-14T16:13:59.309318+0000","last_active":"2023-07-14T16:14:01.174059+0000","last_peered":"2023-07-14T16:14:01.174059+0000","last_clean":"2023-07-14T16:14:01.174059+0000","last_became_active":"2023-07-14T16:13:59.308931+0000","last_became_peered":"2023-07-14T16:13:59.308931+0000","last_unstale":"2023-07-14T16:14:01.174059+0000","last_undegraded":"2023-07-14T16:14:01.174059+0000","last_fullsized":"2023-07-14T16:14:01.174059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:00:49.873512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173842+0000","last_change":"2023-07-14T16:13:59.304410+0000","last_active":"2023-07-14T16:14:01.173842+0000","last_peered":"2023-07-14T16:14:01.173842+0000","last_clean":"2023-07-14T16:14:01.173842+0000","last_became_active":"2023-07-14T16:13:59.304022+0000","last_became_peered":"2023-07-14T16:13:59.304022+0000","last_unstale":"2023-07-14T16:14:01.173842+0000","last_undegraded":"2023-07-14T16:14:01.173842+0000","last_fullsized":"2023-07-14T16:14:01.173842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:02:41.890599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286140+0000","last_change":"2023-07-14T16:13:59.307986+0000","last_active":"2023-07-14T16:14:00.286140+0000","last_peered":"2023-07-14T16:14:00.286140+0000","last_clean":"2023-07-14T16:14:00.286140+0000","last_became_active":"2023-07-14T16:13:59.307701+0000","last_became_peered":"2023-07-14T16:13:59.307701+0000","last_unstale":"2023-07-14T16:14:00.286140+0000","last_undegraded":"2023-07-14T16:14:00.286140+0000","last_fullsized":"2023-07-14T16:14:00.286140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:36:37.853083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290724+0000","last_change":"2023-07-14T16:13:59.301685+0000","last_active":"2023-07-14T16:14:00.290724+0000","last_peered":"2023-07-14T16:14:00.290724+0000","last_clean":"2023-07-14T16:14:00.290724+0000","last_became_active":"2023-07-14T16:13:59.300905+0000","last_became_peered":"2023-07-14T16:13:59.300905+0000","last_unstale":"2023-07-14T16:14:00.290724+0000","last_undegraded":"2023-07-14T16:14:00.290724+0000","last_fullsized":"2023-07-14T16:14:00.290724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:27:22.529080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174068+0000","last_change":"2023-07-14T16:13:59.306027+0000","last_active":"2023-07-14T16:14:01.174068+0000","last_peered":"2023-07-14T16:14:01.174068+0000","last_clean":"2023-07-14T16:14:01.174068+0000","last_became_active":"2023-07-14T16:13:59.305718+0000","last_became_peered":"2023-07-14T16:13:59.305718+0000","last_unstale":"2023-07-14T16:14:01.174068+0000","last_undegraded":"2023-07-14T16:14:01.174068+0000","last_fullsized":"2023-07-14T16:14:01.174068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:14:59.099925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.605145+0000","last_change":"2023-07-14T16:13:59.294140+0000","last_active":"2023-07-14T16:14:00.605145+0000","last_peered":"2023-07-14T16:14:00.605145+0000","last_clean":"2023-07-14T16:14:00.605145+0000","last_became_active":"2023-07-14T16:13:59.293825+0000","last_became_peered":"2023-07-14T16:13:59.293825+0000","last_unstale":"2023-07-14T16:14:00.605145+0000","last_undegraded":"2023-07-14T16:14:00.605145+0000","last_fullsized":"2023-07-14T16:14:00.605145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:14:48.934852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578067+0000","last_change":"2023-07-14T16:13:58.277154+0000","last_active":"2023-07-14T16:14:03.578067+0000","last_peered":"2023-07-14T16:14:03.578067+0000","last_clean":"2023-07-14T16:14:03.578067+0000","last_became_active":"2023-07-14T16:13:58.276800+0000","last_became_peered":"2023-07-14T16:13:58.276800+0000","last_unstale":"2023-07-14T16:14:03.578067+0000","last_undegraded":"2023-07-14T16:14:03.578067+0000","last_fullsized":"2023-07-14T16:14:03.578067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:04.499469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290353+0000","last_change":"2023-07-14T16:13:59.299819+0000","last_active":"2023-07-14T16:14:00.290353+0000","last_peered":"2023-07-14T16:14:00.290353+0000","last_clean":"2023-07-14T16:14:00.290353+0000","last_became_active":"2023-07-14T16:13:59.299294+0000","last_became_peered":"2023-07-14T16:13:59.299294+0000","last_unstale":"2023-07-14T16:14:00.290353+0000","last_undegraded":"2023-07-14T16:14:00.290353+0000","last_fullsized":"2023-07-14T16:14:00.290353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:07:58.187097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290582+0000","last_change":"2023-07-14T16:13:58.278954+0000","last_active":"2023-07-14T16:14:00.290582+0000","last_peered":"2023-07-14T16:14:00.290582+0000","last_clean":"2023-07-14T16:14:00.290582+0000","last_became_active":"2023-07-14T16:13:58.278454+0000","last_became_peered":"2023-07-14T16:13:58.278454+0000","last_unstale":"2023-07-14T16:14:00.290582+0000","last_undegraded":"2023-07-14T16:14:00.290582+0000","last_fullsized":"2023-07-14T16:14:00.290582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:50:30.360986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174298+0000","last_change":"2023-07-14T16:13:59.299877+0000","last_active":"2023-07-14T16:14:01.174298+0000","last_peered":"2023-07-14T16:14:01.174298+0000","last_clean":"2023-07-14T16:14:01.174298+0000","last_became_active":"2023-07-14T16:13:59.299557+0000","last_became_peered":"2023-07-14T16:13:59.299557+0000","last_unstale":"2023-07-14T16:14:01.174298+0000","last_undegraded":"2023-07-14T16:14:01.174298+0000","last_fullsized":"2023-07-14T16:14:01.174298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:22:28.534140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.575576+0000","last_change":"2023-07-14T16:13:58.281711+0000","last_active":"2023-07-14T16:14:03.575576+0000","last_peered":"2023-07-14T16:14:03.575576+0000","last_clean":"2023-07-14T16:14:03.575576+0000","last_became_active":"2023-07-14T16:13:58.281185+0000","last_became_peered":"2023-07-14T16:13:58.281185+0000","last_unstale":"2023-07-14T16:14:03.575576+0000","last_undegraded":"2023-07-14T16:14:03.575576+0000","last_fullsized":"2023-07-14T16:14:03.575576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:12:59.648173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286382+0000","last_change":"2023-07-14T16:13:59.307987+0000","last_active":"2023-07-14T16:14:00.286382+0000","last_peered":"2023-07-14T16:14:00.286382+0000","last_clean":"2023-07-14T16:14:00.286382+0000","last_became_active":"2023-07-14T16:13:59.307457+0000","last_became_peered":"2023-07-14T16:13:59.307457+0000","last_unstale":"2023-07-14T16:14:00.286382+0000","last_undegraded":"2023-07-14T16:14:00.286382+0000","last_fullsized":"2023-07-14T16:14:00.286382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:38.798631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175418+0000","last_change":"2023-07-14T16:13:58.282332+0000","last_active":"2023-07-14T16:14:01.175418+0000","last_peered":"2023-07-14T16:14:01.175418+0000","last_clean":"2023-07-14T16:14:01.175418+0000","last_became_active":"2023-07-14T16:13:58.278103+0000","last_became_peered":"2023-07-14T16:13:58.278103+0000","last_unstale":"2023-07-14T16:14:01.175418+0000","last_undegraded":"2023-07-14T16:14:01.175418+0000","last_fullsized":"2023-07-14T16:14:01.175418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:23:14.369512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.493190+0000","last_change":"2023-07-14T16:13:59.301036+0000","last_active":"2023-07-14T16:14:00.493190+0000","last_peered":"2023-07-14T16:14:00.493190+0000","last_clean":"2023-07-14T16:14:00.493190+0000","last_became_active":"2023-07-14T16:13:59.300787+0000","last_became_peered":"2023-07-14T16:13:59.300787+0000","last_unstale":"2023-07-14T16:14:00.493190+0000","last_undegraded":"2023-07-14T16:14:00.493190+0000","last_fullsized":"2023-07-14T16:14:00.493190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:49:27.481208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.595824+0000","last_change":"2023-07-14T16:13:58.282392+0000","last_active":"2023-07-14T16:14:02.595824+0000","last_peered":"2023-07-14T16:14:02.595824+0000","last_clean":"2023-07-14T16:14:02.595824+0000","last_became_active":"2023-07-14T16:13:58.282030+0000","last_became_peered":"2023-07-14T16:13:58.282030+0000","last_unstale":"2023-07-14T16:14:02.595824+0000","last_undegraded":"2023-07-14T16:14:02.595824+0000","last_fullsized":"2023-07-14T16:14:02.595824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:43.582004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175117+0000","last_change":"2023-07-14T16:13:59.292681+0000","last_active":"2023-07-14T16:14:01.175117+0000","last_peered":"2023-07-14T16:14:01.175117+0000","last_clean":"2023-07-14T16:14:01.175117+0000","last_became_active":"2023-07-14T16:13:59.292289+0000","last_became_peered":"2023-07-14T16:13:59.292289+0000","last_unstale":"2023-07-14T16:14:01.175117+0000","last_undegraded":"2023-07-14T16:14:01.175117+0000","last_fullsized":"2023-07-14T16:14:01.175117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:43:34.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577227+0000","last_change":"2023-07-14T16:13:58.282111+0000","last_active":"2023-07-14T16:14:00.577227+0000","last_peered":"2023-07-14T16:14:00.577227+0000","last_clean":"2023-07-14T16:14:00.577227+0000","last_became_active":"2023-07-14T16:13:58.281783+0000","last_became_peered":"2023-07-14T16:13:58.281783+0000","last_unstale":"2023-07-14T16:14:00.577227+0000","last_undegraded":"2023-07-14T16:14:00.577227+0000","last_fullsized":"2023-07-14T16:14:00.577227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:59:52.556309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.493025+0000","last_change":"2023-07-14T16:13:59.297439+0000","last_active":"2023-07-14T16:14:00.493025+0000","last_peered":"2023-07-14T16:14:00.493025+0000","last_clean":"2023-07-14T16:14:00.493025+0000","last_became_active":"2023-07-14T16:13:59.297117+0000","last_became_peered":"2023-07-14T16:13:59.297117+0000","last_unstale":"2023-07-14T16:14:00.493025+0000","last_undegraded":"2023-07-14T16:14:00.493025+0000","last_fullsized":"2023-07-14T16:14:00.493025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:04:47.449697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578028+0000","last_change":"2023-07-14T16:13:58.274826+0000","last_active":"2023-07-14T16:14:03.578028+0000","last_peered":"2023-07-14T16:14:03.578028+0000","last_clean":"2023-07-14T16:14:03.578028+0000","last_became_active":"2023-07-14T16:13:58.274406+0000","last_became_peered":"2023-07-14T16:13:58.274406+0000","last_unstale":"2023-07-14T16:14:03.578028+0000","last_undegraded":"2023-07-14T16:14:03.578028+0000","last_fullsized":"2023-07-14T16:14:03.578028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:51:38.339653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290081+0000","last_change":"2023-07-14T16:13:59.296730+0000","last_active":"2023-07-14T16:14:00.290081+0000","last_peered":"2023-07-14T16:14:00.290081+0000","last_clean":"2023-07-14T16:14:00.290081+0000","last_became_active":"2023-07-14T16:13:59.296372+0000","last_became_peered":"2023-07-14T16:13:59.296372+0000","last_unstale":"2023-07-14T16:14:00.290081+0000","last_undegraded":"2023-07-14T16:14:00.290081+0000","last_fullsized":"2023-07-14T16:14:00.290081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:59.457423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.492695+0000","last_change":"2023-07-14T16:13:58.269122+0000","last_active":"2023-07-14T16:14:00.492695+0000","last_peered":"2023-07-14T16:14:00.492695+0000","last_clean":"2023-07-14T16:14:00.492695+0000","last_became_active":"2023-07-14T16:13:58.268770+0000","last_became_peered":"2023-07-14T16:13:58.268770+0000","last_unstale":"2023-07-14T16:14:00.492695+0000","last_undegraded":"2023-07-14T16:14:00.492695+0000","last_fullsized":"2023-07-14T16:14:00.492695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:15.228768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173621+0000","last_change":"2023-07-14T16:13:59.306987+0000","last_active":"2023-07-14T16:14:01.173621+0000","last_peered":"2023-07-14T16:14:01.173621+0000","last_clean":"2023-07-14T16:14:01.173621+0000","last_became_active":"2023-07-14T16:13:59.306649+0000","last_became_peered":"2023-07-14T16:13:59.306649+0000","last_unstale":"2023-07-14T16:14:01.173621+0000","last_undegraded":"2023-07-14T16:14:01.173621+0000","last_fullsized":"2023-07-14T16:14:01.173621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:25:37.926244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.597656+0000","last_change":"2023-07-14T16:13:58.281319+0000","last_active":"2023-07-14T16:14:02.597656+0000","last_peered":"2023-07-14T16:14:02.597656+0000","last_clean":"2023-07-14T16:14:02.597656+0000","last_became_active":"2023-07-14T16:13:58.280928+0000","last_became_peered":"2023-07-14T16:13:58.280928+0000","last_unstale":"2023-07-14T16:14:02.597656+0000","last_undegraded":"2023-07-14T16:14:02.597656+0000","last_fullsized":"2023-07-14T16:14:02.597656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:11:11.509173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286496+0000","last_change":"2023-07-14T16:13:59.309586+0000","last_active":"2023-07-14T16:14:00.286496+0000","last_peered":"2023-07-14T16:14:00.286496+0000","last_clean":"2023-07-14T16:14:00.286496+0000","last_became_active":"2023-07-14T16:13:59.309243+0000","last_became_peered":"2023-07-14T16:13:59.309243+0000","last_unstale":"2023-07-14T16:14:00.286496+0000","last_undegraded":"2023-07-14T16:14:00.286496+0000","last_fullsized":"2023-07-14T16:14:00.286496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:38.787965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.596724+0000","last_change":"2023-07-14T16:13:58.279769+0000","last_active":"2023-07-14T16:14:02.596724+0000","last_peered":"2023-07-14T16:14:02.596724+0000","last_clean":"2023-07-14T16:14:02.596724+0000","last_became_active":"2023-07-14T16:13:58.279409+0000","last_became_peered":"2023-07-14T16:13:58.279409+0000","last_unstale":"2023-07-14T16:14:02.596724+0000","last_undegraded":"2023-07-14T16:14:02.596724+0000","last_fullsized":"2023-07-14T16:14:02.596724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:51:40.918469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174342+0000","last_change":"2023-07-14T16:13:59.300382+0000","last_active":"2023-07-14T16:14:01.174342+0000","last_peered":"2023-07-14T16:14:01.174342+0000","last_clean":"2023-07-14T16:14:01.174342+0000","last_became_active":"2023-07-14T16:13:59.299981+0000","last_became_peered":"2023-07-14T16:13:59.299981+0000","last_unstale":"2023-07-14T16:14:01.174342+0000","last_undegraded":"2023-07-14T16:14:01.174342+0000","last_fullsized":"2023-07-14T16:14:01.174342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:23.017109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.493269+0000","last_change":"2023-07-14T16:13:58.274128+0000","last_active":"2023-07-14T16:14:00.493269+0000","last_peered":"2023-07-14T16:14:00.493269+0000","last_clean":"2023-07-14T16:14:00.493269+0000","last_became_active":"2023-07-14T16:13:58.273446+0000","last_became_peered":"2023-07-14T16:13:58.273446+0000","last_unstale":"2023-07-14T16:14:00.493269+0000","last_undegraded":"2023-07-14T16:14:00.493269+0000","last_fullsized":"2023-07-14T16:14:00.493269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:16:16.599740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286535+0000","last_change":"2023-07-14T16:13:59.301551+0000","last_active":"2023-07-14T16:14:00.286535+0000","last_peered":"2023-07-14T16:14:00.286535+0000","last_clean":"2023-07-14T16:14:00.286535+0000","last_became_active":"2023-07-14T16:13:59.301206+0000","last_became_peered":"2023-07-14T16:13:59.301206+0000","last_unstale":"2023-07-14T16:14:00.286535+0000","last_undegraded":"2023-07-14T16:14:00.286535+0000","last_fullsized":"2023-07-14T16:14:00.286535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:07:40.084145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.597342+0000","last_change":"2023-07-14T16:13:58.283304+0000","last_active":"2023-07-14T16:14:02.597342+0000","last_peered":"2023-07-14T16:14:02.597342+0000","last_clean":"2023-07-14T16:14:02.597342+0000","last_became_active":"2023-07-14T16:13:58.282930+0000","last_became_peered":"2023-07-14T16:13:58.282930+0000","last_unstale":"2023-07-14T16:14:02.597342+0000","last_undegraded":"2023-07-14T16:14:02.597342+0000","last_fullsized":"2023-07-14T16:14:02.597342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:00.947603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175257+0000","last_change":"2023-07-14T16:13:59.304144+0000","last_active":"2023-07-14T16:14:01.175257+0000","last_peered":"2023-07-14T16:14:01.175257+0000","last_clean":"2023-07-14T16:14:01.175257+0000","last_became_active":"2023-07-14T16:13:59.303745+0000","last_became_peered":"2023-07-14T16:13:59.303745+0000","last_unstale":"2023-07-14T16:14:01.175257+0000","last_undegraded":"2023-07-14T16:14:01.175257+0000","last_fullsized":"2023-07-14T16:14:01.175257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:55.949263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.577159+0000","last_change":"2023-07-14T16:13:58.283734+0000","last_active":"2023-07-14T16:14:03.577159+0000","last_peered":"2023-07-14T16:14:03.577159+0000","last_clean":"2023-07-14T16:14:03.577159+0000","last_became_active":"2023-07-14T16:13:58.282765+0000","last_became_peered":"2023-07-14T16:13:58.282765+0000","last_unstale":"2023-07-14T16:14:03.577159+0000","last_undegraded":"2023-07-14T16:14:03.577159+0000","last_fullsized":"2023-07-14T16:14:03.577159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:35:33.247905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658906+0000","last_change":"2023-07-14T16:13:59.308445+0000","last_active":"2023-07-14T16:14:00.658906+0000","last_peered":"2023-07-14T16:14:00.658906+0000","last_clean":"2023-07-14T16:14:00.658906+0000","last_became_active":"2023-07-14T16:13:59.308130+0000","last_became_peered":"2023-07-14T16:13:59.308130+0000","last_unstale":"2023-07-14T16:14:00.658906+0000","last_undegraded":"2023-07-14T16:14:00.658906+0000","last_fullsized":"2023-07-14T16:14:00.658906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:40:57.345220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.596174+0000","last_change":"2023-07-14T16:13:58.273973+0000","last_active":"2023-07-14T16:14:02.596174+0000","last_peered":"2023-07-14T16:14:02.596174+0000","last_clean":"2023-07-14T16:14:02.596174+0000","last_became_active":"2023-07-14T16:13:58.273606+0000","last_became_peered":"2023-07-14T16:13:58.273606+0000","last_unstale":"2023-07-14T16:14:02.596174+0000","last_undegraded":"2023-07-14T16:14:02.596174+0000","last_fullsized":"2023-07-14T16:14:02.596174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:13.396401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992012+0000","last_change":"2023-07-14T16:13:59.306428+0000","last_active":"2023-07-14T16:14:00.992012+0000","last_peered":"2023-07-14T16:14:00.992012+0000","last_clean":"2023-07-14T16:14:00.992012+0000","last_became_active":"2023-07-14T16:13:59.306090+0000","last_became_peered":"2023-07-14T16:13:59.306090+0000","last_unstale":"2023-07-14T16:14:00.992012+0000","last_undegraded":"2023-07-14T16:14:00.992012+0000","last_fullsized":"2023-07-14T16:14:00.992012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:02:50.168112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:04.600599+0000","last_change":"2023-07-14T16:13:58.277123+0000","last_active":"2023-07-14T16:14:04.600599+0000","last_peered":"2023-07-14T16:14:04.600599+0000","last_clean":"2023-07-14T16:14:04.600599+0000","last_became_active":"2023-07-14T16:13:58.276815+0000","last_became_peered":"2023-07-14T16:13:58.276815+0000","last_unstale":"2023-07-14T16:14:04.600599+0000","last_undegraded":"2023-07-14T16:14:04.600599+0000","last_fullsized":"2023-07-14T16:14:04.600599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:23.871523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.991732+0000","last_change":"2023-07-14T16:13:59.310238+0000","last_active":"2023-07-14T16:14:00.991732+0000","last_peered":"2023-07-14T16:14:00.991732+0000","last_clean":"2023-07-14T16:14:00.991732+0000","last_became_active":"2023-07-14T16:13:59.309918+0000","last_became_peered":"2023-07-14T16:13:59.309918+0000","last_unstale":"2023-07-14T16:14:00.991732+0000","last_undegraded":"2023-07-14T16:14:00.991732+0000","last_fullsized":"2023-07-14T16:14:00.991732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:12.761924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578594+0000","last_change":"2023-07-14T16:13:58.280834+0000","last_active":"2023-07-14T16:14:03.578594+0000","last_peered":"2023-07-14T16:14:03.578594+0000","last_clean":"2023-07-14T16:14:03.578594+0000","last_became_active":"2023-07-14T16:13:58.280497+0000","last_became_peered":"2023-07-14T16:13:58.280497+0000","last_unstale":"2023-07-14T16:14:03.578594+0000","last_undegraded":"2023-07-14T16:14:03.578594+0000","last_fullsized":"2023-07-14T16:14:03.578594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:08:09.982085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.492856+0000","last_change":"2023-07-14T16:13:59.301464+0000","last_active":"2023-07-14T16:14:00.492856+0000","last_peered":"2023-07-14T16:14:00.492856+0000","last_clean":"2023-07-14T16:14:00.492856+0000","last_became_active":"2023-07-14T16:13:59.301246+0000","last_became_peered":"2023-07-14T16:13:59.301246+0000","last_unstale":"2023-07-14T16:14:00.492856+0000","last_undegraded":"2023-07-14T16:14:00.492856+0000","last_fullsized":"2023-07-14T16:14:00.492856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:03:47.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175347+0000","last_change":"2023-07-14T16:13:58.278480+0000","last_active":"2023-07-14T16:14:01.175347+0000","last_peered":"2023-07-14T16:14:01.175347+0000","last_clean":"2023-07-14T16:14:01.175347+0000","last_became_active":"2023-07-14T16:13:58.277906+0000","last_became_peered":"2023-07-14T16:13:58.277906+0000","last_unstale":"2023-07-14T16:14:01.175347+0000","last_undegraded":"2023-07-14T16:14:01.175347+0000","last_fullsized":"2023-07-14T16:14:01.175347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:27:51.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290556+0000","last_change":"2023-07-14T16:13:59.297644+0000","last_active":"2023-07-14T16:14:00.290556+0000","last_peered":"2023-07-14T16:14:00.290556+0000","last_clean":"2023-07-14T16:14:00.290556+0000","last_became_active":"2023-07-14T16:13:59.297332+0000","last_became_peered":"2023-07-14T16:13:59.297332+0000","last_unstale":"2023-07-14T16:14:00.290556+0000","last_undegraded":"2023-07-14T16:14:00.290556+0000","last_fullsized":"2023-07-14T16:14:00.290556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:22.525437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173843+0000","last_change":"2023-07-14T16:13:58.280632+0000","last_active":"2023-07-14T16:14:01.173843+0000","last_peered":"2023-07-14T16:14:01.173843+0000","last_clean":"2023-07-14T16:14:01.173843+0000","last_became_active":"2023-07-14T16:13:58.280296+0000","last_became_peered":"2023-07-14T16:13:58.280296+0000","last_unstale":"2023-07-14T16:14:01.173843+0000","last_undegraded":"2023-07-14T16:14:01.173843+0000","last_fullsized":"2023-07-14T16:14:01.173843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:28.463404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577583+0000","last_change":"2023-07-14T16:13:59.297092+0000","last_active":"2023-07-14T16:14:00.577583+0000","last_peered":"2023-07-14T16:14:00.577583+0000","last_clean":"2023-07-14T16:14:00.577583+0000","last_became_active":"2023-07-14T16:13:59.296750+0000","last_became_peered":"2023-07-14T16:13:59.296750+0000","last_unstale":"2023-07-14T16:14:00.577583+0000","last_undegraded":"2023-07-14T16:14:00.577583+0000","last_fullsized":"2023-07-14T16:14:00.577583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:16:36.897671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.395958+0000","last_change":"2023-07-14T16:13:58.281786+0000","last_active":"2023-07-14T16:14:01.395958+0000","last_peered":"2023-07-14T16:14:01.395958+0000","last_clean":"2023-07-14T16:14:01.395958+0000","last_became_active":"2023-07-14T16:13:58.281471+0000","last_became_peered":"2023-07-14T16:13:58.281471+0000","last_unstale":"2023-07-14T16:14:01.395958+0000","last_undegraded":"2023-07-14T16:14:01.395958+0000","last_fullsized":"2023-07-14T16:14:01.395958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:23:49.206754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577341+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:14:00.577341+0000","last_peered":"2023-07-14T16:14:00.577341+0000","last_clean":"2023-07-14T16:14:00.577341+0000","last_became_active":"2023-07-14T16:13:59.299157+0000","last_became_peered":"2023-07-14T16:13:59.299157+0000","last_unstale":"2023-07-14T16:14:00.577341+0000","last_undegraded":"2023-07-14T16:14:00.577341+0000","last_fullsized":"2023-07-14T16:14:00.577341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:07:10.431621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.492945+0000","last_change":"2023-07-14T16:13:59.288836+0000","last_active":"2023-07-14T16:14:00.492945+0000","last_peered":"2023-07-14T16:14:00.492945+0000","last_clean":"2023-07-14T16:14:00.492945+0000","last_became_active":"2023-07-14T16:13:59.288517+0000","last_became_peered":"2023-07-14T16:13:59.288517+0000","last_unstale":"2023-07-14T16:14:00.492945+0000","last_undegraded":"2023-07-14T16:14:00.492945+0000","last_fullsized":"2023-07-14T16:14:00.492945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:49:17.986192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.577752+0000","last_change":"2023-07-14T16:13:58.282415+0000","last_active":"2023-07-14T16:14:03.577752+0000","last_peered":"2023-07-14T16:14:03.577752+0000","last_clean":"2023-07-14T16:14:03.577752+0000","last_became_active":"2023-07-14T16:13:58.282118+0000","last_became_peered":"2023-07-14T16:13:58.282118+0000","last_unstale":"2023-07-14T16:14:03.577752+0000","last_undegraded":"2023-07-14T16:14:03.577752+0000","last_fullsized":"2023-07-14T16:14:03.577752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:14:30.064921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290603+0000","last_change":"2023-07-14T16:13:59.302860+0000","last_active":"2023-07-14T16:14:00.290603+0000","last_peered":"2023-07-14T16:14:00.290603+0000","last_clean":"2023-07-14T16:14:00.290603+0000","last_became_active":"2023-07-14T16:13:59.302454+0000","last_became_peered":"2023-07-14T16:13:59.302454+0000","last_unstale":"2023-07-14T16:14:00.290603+0000","last_undegraded":"2023-07-14T16:14:00.290603+0000","last_fullsized":"2023-07-14T16:14:00.290603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:28:49.992295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.576848+0000","last_change":"2023-07-14T16:13:58.273908+0000","last_active":"2023-07-14T16:14:03.576848+0000","last_peered":"2023-07-14T16:14:03.576848+0000","last_clean":"2023-07-14T16:14:03.576848+0000","last_became_active":"2023-07-14T16:13:58.273497+0000","last_became_peered":"2023-07-14T16:13:58.273497+0000","last_unstale":"2023-07-14T16:14:03.576848+0000","last_undegraded":"2023-07-14T16:14:03.576848+0000","last_fullsized":"2023-07-14T16:14:03.576848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:12:07.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.991727+0000","last_change":"2023-07-14T16:12:36.546475+0000","last_active":"2023-07-14T16:14:00.991727+0000","last_peered":"2023-07-14T16:14:00.991727+0000","last_clean":"2023-07-14T16:14:00.991727+0000","last_became_active":"2023-07-14T16:12:36.545943+0000","last_became_peered":"2023-07-14T16:12:36.545943+0000","last_unstale":"2023-07-14T16:14:00.991727+0000","last_undegraded":"2023-07-14T16:14:00.991727+0000","last_fullsized":"2023-07-14T16:14:00.991727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_clean_scrub_stamp":"2023-07-14T16:08:58.024408+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:41.410101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658527+0000","last_change":"2023-07-14T16:13:59.297669+0000","last_active":"2023-07-14T16:14:00.658527+0000","last_peered":"2023-07-14T16:14:00.658527+0000","last_clean":"2023-07-14T16:14:00.658527+0000","last_became_active":"2023-07-14T16:13:59.297364+0000","last_became_peered":"2023-07-14T16:13:59.297364+0000","last_unstale":"2023-07-14T16:14:00.658527+0000","last_undegraded":"2023-07-14T16:14:00.658527+0000","last_fullsized":"2023-07-14T16:14:00.658527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:08:05.988095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.576274+0000","last_change":"2023-07-14T16:13:58.276850+0000","last_active":"2023-07-14T16:14:03.576274+0000","last_peered":"2023-07-14T16:14:03.576274+0000","last_clean":"2023-07-14T16:14:03.576274+0000","last_became_active":"2023-07-14T16:13:58.276504+0000","last_became_peered":"2023-07-14T16:13:58.276504+0000","last_unstale":"2023-07-14T16:14:03.576274+0000","last_undegraded":"2023-07-14T16:14:03.576274+0000","last_fullsized":"2023-07-14T16:14:03.576274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:39.490295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.594373+0000","last_change":"2023-07-14T16:13:58.276257+0000","last_active":"2023-07-14T16:14:02.594373+0000","last_peered":"2023-07-14T16:14:02.594373+0000","last_clean":"2023-07-14T16:14:02.594373+0000","last_became_active":"2023-07-14T16:13:58.275910+0000","last_became_peered":"2023-07-14T16:13:58.275910+0000","last_unstale":"2023-07-14T16:14:02.594373+0000","last_undegraded":"2023-07-14T16:14:02.594373+0000","last_fullsized":"2023-07-14T16:14:02.594373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:51:40.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658945+0000","last_change":"2023-07-14T16:13:59.299135+0000","last_active":"2023-07-14T16:14:00.658945+0000","last_peered":"2023-07-14T16:14:00.658945+0000","last_clean":"2023-07-14T16:14:00.658945+0000","last_became_active":"2023-07-14T16:13:59.298031+0000","last_became_peered":"2023-07-14T16:13:59.298031+0000","last_unstale":"2023-07-14T16:14:00.658945+0000","last_undegraded":"2023-07-14T16:14:00.658945+0000","last_fullsized":"2023-07-14T16:14:00.658945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:53:54.675199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578745+0000","last_change":"2023-07-14T16:13:58.279664+0000","last_active":"2023-07-14T16:14:03.578745+0000","last_peered":"2023-07-14T16:14:03.578745+0000","last_clean":"2023-07-14T16:14:03.578745+0000","last_became_active":"2023-07-14T16:13:58.279137+0000","last_became_peered":"2023-07-14T16:13:58.279137+0000","last_unstale":"2023-07-14T16:14:03.578745+0000","last_undegraded":"2023-07-14T16:14:03.578745+0000","last_fullsized":"2023-07-14T16:14:03.578745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:12:14.121097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290612+0000","last_change":"2023-07-14T16:13:59.305536+0000","last_active":"2023-07-14T16:14:00.290612+0000","last_peered":"2023-07-14T16:14:00.290612+0000","last_clean":"2023-07-14T16:14:00.290612+0000","last_became_active":"2023-07-14T16:13:59.305176+0000","last_became_peered":"2023-07-14T16:13:59.305176+0000","last_unstale":"2023-07-14T16:14:00.290612+0000","last_undegraded":"2023-07-14T16:14:00.290612+0000","last_fullsized":"2023-07-14T16:14:00.290612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:20.875570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.596570+0000","last_change":"2023-07-14T16:13:58.269652+0000","last_active":"2023-07-14T16:14:02.596570+0000","last_peered":"2023-07-14T16:14:02.596570+0000","last_clean":"2023-07-14T16:14:02.596570+0000","last_became_active":"2023-07-14T16:13:58.269338+0000","last_became_peered":"2023-07-14T16:13:58.269338+0000","last_unstale":"2023-07-14T16:14:02.596570+0000","last_undegraded":"2023-07-14T16:14:02.596570+0000","last_fullsized":"2023-07-14T16:14:02.596570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:17:54.070686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174080+0000","last_change":"2023-07-14T16:13:59.297574+0000","last_active":"2023-07-14T16:14:01.174080+0000","last_peered":"2023-07-14T16:14:01.174080+0000","last_clean":"2023-07-14T16:14:01.174080+0000","last_became_active":"2023-07-14T16:13:59.297124+0000","last_became_peered":"2023-07-14T16:13:59.297124+0000","last_unstale":"2023-07-14T16:14:01.174080+0000","last_undegraded":"2023-07-14T16:14:01.174080+0000","last_fullsized":"2023-07-14T16:14:01.174080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:42:49.037176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.595928+0000","last_change":"2023-07-14T16:13:58.278011+0000","last_active":"2023-07-14T16:14:02.595928+0000","last_peered":"2023-07-14T16:14:02.595928+0000","last_clean":"2023-07-14T16:14:02.595928+0000","last_became_active":"2023-07-14T16:13:58.277563+0000","last_became_peered":"2023-07-14T16:13:58.277563+0000","last_unstale":"2023-07-14T16:14:02.595928+0000","last_undegraded":"2023-07-14T16:14:02.595928+0000","last_fullsized":"2023-07-14T16:14:02.595928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:50.032636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174973+0000","last_change":"2023-07-14T16:13:59.308836+0000","last_active":"2023-07-14T16:14:01.174973+0000","last_peered":"2023-07-14T16:14:01.174973+0000","last_clean":"2023-07-14T16:14:01.174973+0000","last_became_active":"2023-07-14T16:13:59.308510+0000","last_became_peered":"2023-07-14T16:13:59.308510+0000","last_unstale":"2023-07-14T16:14:01.174973+0000","last_undegraded":"2023-07-14T16:14:01.174973+0000","last_fullsized":"2023-07-14T16:14:01.174973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:54:55.671765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.597865+0000","last_change":"2023-07-14T16:13:58.277438+0000","last_active":"2023-07-14T16:14:02.597865+0000","last_peered":"2023-07-14T16:14:02.597865+0000","last_clean":"2023-07-14T16:14:02.597865+0000","last_became_active":"2023-07-14T16:13:58.277114+0000","last_became_peered":"2023-07-14T16:13:58.277114+0000","last_unstale":"2023-07-14T16:14:02.597865+0000","last_undegraded":"2023-07-14T16:14:02.597865+0000","last_fullsized":"2023-07-14T16:14:02.597865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:01:02.972663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290717+0000","last_change":"2023-07-14T16:13:59.304491+0000","last_active":"2023-07-14T16:14:00.290717+0000","last_peered":"2023-07-14T16:14:00.290717+0000","last_clean":"2023-07-14T16:14:00.290717+0000","last_became_active":"2023-07-14T16:13:59.303925+0000","last_became_peered":"2023-07-14T16:13:59.303925+0000","last_unstale":"2023-07-14T16:14:00.290717+0000","last_undegraded":"2023-07-14T16:14:00.290717+0000","last_fullsized":"2023-07-14T16:14:00.290717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:16:43.617439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.604769+0000","last_change":"2023-07-14T16:13:58.286276+0000","last_active":"2023-07-14T16:14:00.604769+0000","last_peered":"2023-07-14T16:14:00.604769+0000","last_clean":"2023-07-14T16:14:00.604769+0000","last_became_active":"2023-07-14T16:13:58.285855+0000","last_became_peered":"2023-07-14T16:13:58.285855+0000","last_unstale":"2023-07-14T16:14:00.604769+0000","last_undegraded":"2023-07-14T16:14:00.604769+0000","last_fullsized":"2023-07-14T16:14:00.604769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:31:05.621307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290474+0000","last_change":"2023-07-14T16:13:59.310983+0000","last_active":"2023-07-14T16:14:00.290474+0000","last_peered":"2023-07-14T16:14:00.290474+0000","last_clean":"2023-07-14T16:14:00.290474+0000","last_became_active":"2023-07-14T16:13:59.310636+0000","last_became_peered":"2023-07-14T16:13:59.310636+0000","last_unstale":"2023-07-14T16:14:00.290474+0000","last_undegraded":"2023-07-14T16:14:00.290474+0000","last_fullsized":"2023-07-14T16:14:00.290474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:39.709513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.597150+0000","last_change":"2023-07-14T16:13:58.280153+0000","last_active":"2023-07-14T16:14:02.597150+0000","last_peered":"2023-07-14T16:14:02.597150+0000","last_clean":"2023-07-14T16:14:02.597150+0000","last_became_active":"2023-07-14T16:13:58.279771+0000","last_became_peered":"2023-07-14T16:13:58.279771+0000","last_unstale":"2023-07-14T16:14:02.597150+0000","last_undegraded":"2023-07-14T16:14:02.597150+0000","last_fullsized":"2023-07-14T16:14:02.597150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:48:48.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992111+0000","last_change":"2023-07-14T16:13:59.304854+0000","last_active":"2023-07-14T16:14:00.992111+0000","last_peered":"2023-07-14T16:14:00.992111+0000","last_clean":"2023-07-14T16:14:00.992111+0000","last_became_active":"2023-07-14T16:13:59.304374+0000","last_became_peered":"2023-07-14T16:13:59.304374+0000","last_unstale":"2023-07-14T16:14:00.992111+0000","last_undegraded":"2023-07-14T16:14:00.992111+0000","last_fullsized":"2023-07-14T16:14:00.992111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:16.308607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578934+0000","last_change":"2023-07-14T16:13:58.281885+0000","last_active":"2023-07-14T16:14:03.578934+0000","last_peered":"2023-07-14T16:14:03.578934+0000","last_clean":"2023-07-14T16:14:03.578934+0000","last_became_active":"2023-07-14T16:13:58.278759+0000","last_became_peered":"2023-07-14T16:13:58.278759+0000","last_unstale":"2023-07-14T16:14:03.578934+0000","last_undegraded":"2023-07-14T16:14:03.578934+0000","last_fullsized":"2023-07-14T16:14:03.578934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:05.139803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992167+0000","last_change":"2023-07-14T16:13:59.302680+0000","last_active":"2023-07-14T16:14:00.992167+0000","last_peered":"2023-07-14T16:14:00.992167+0000","last_clean":"2023-07-14T16:14:00.992167+0000","last_became_active":"2023-07-14T16:13:59.301854+0000","last_became_peered":"2023-07-14T16:13:59.301854+0000","last_unstale":"2023-07-14T16:14:00.992167+0000","last_undegraded":"2023-07-14T16:14:00.992167+0000","last_fullsized":"2023-07-14T16:14:00.992167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:35.155529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.493130+0000","last_change":"2023-07-14T16:13:58.274254+0000","last_active":"2023-07-14T16:14:00.493130+0000","last_peered":"2023-07-14T16:14:00.493130+0000","last_clean":"2023-07-14T16:14:00.493130+0000","last_became_active":"2023-07-14T16:13:58.273933+0000","last_became_peered":"2023-07-14T16:13:58.273933+0000","last_unstale":"2023-07-14T16:14:00.493130+0000","last_undegraded":"2023-07-14T16:14:00.493130+0000","last_fullsized":"2023-07-14T16:14:00.493130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:55:43.492876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.395985+0000","last_change":"2023-07-14T16:13:59.305615+0000","last_active":"2023-07-14T16:14:01.395985+0000","last_peered":"2023-07-14T16:14:01.395985+0000","last_clean":"2023-07-14T16:14:01.395985+0000","last_became_active":"2023-07-14T16:13:59.305206+0000","last_became_peered":"2023-07-14T16:13:59.305206+0000","last_unstale":"2023-07-14T16:14:01.395985+0000","last_undegraded":"2023-07-14T16:14:01.395985+0000","last_fullsized":"2023-07-14T16:14:01.395985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:16:21.502671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.659055+0000","last_change":"2023-07-14T16:13:58.277976+0000","last_active":"2023-07-14T16:14:00.659055+0000","last_peered":"2023-07-14T16:14:00.659055+0000","last_clean":"2023-07-14T16:14:00.659055+0000","last_became_active":"2023-07-14T16:13:58.277618+0000","last_became_peered":"2023-07-14T16:13:58.277618+0000","last_unstale":"2023-07-14T16:14:00.659055+0000","last_undegraded":"2023-07-14T16:14:00.659055+0000","last_fullsized":"2023-07-14T16:14:00.659055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:48.596257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992090+0000","last_change":"2023-07-14T16:13:59.302041+0000","last_active":"2023-07-14T16:14:00.992090+0000","last_peered":"2023-07-14T16:14:00.992090+0000","last_clean":"2023-07-14T16:14:00.992090+0000","last_became_active":"2023-07-14T16:13:59.301620+0000","last_became_peered":"2023-07-14T16:13:59.301620+0000","last_unstale":"2023-07-14T16:14:00.992090+0000","last_undegraded":"2023-07-14T16:14:00.992090+0000","last_fullsized":"2023-07-14T16:14:00.992090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:29:12.896095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290670+0000","last_change":"2023-07-14T16:13:59.304222+0000","last_active":"2023-07-14T16:14:00.290670+0000","last_peered":"2023-07-14T16:14:00.290670+0000","last_clean":"2023-07-14T16:14:00.290670+0000","last_became_active":"2023-07-14T16:13:59.303404+0000","last_became_peered":"2023-07-14T16:13:59.303404+0000","last_unstale":"2023-07-14T16:14:00.290670+0000","last_undegraded":"2023-07-14T16:14:00.290670+0000","last_fullsized":"2023-07-14T16:14:00.290670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:43:48.365055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.604893+0000","last_change":"2023-07-14T16:13:59.296433+0000","last_active":"2023-07-14T16:14:00.604893+0000","last_peered":"2023-07-14T16:14:00.604893+0000","last_clean":"2023-07-14T16:14:00.604893+0000","last_became_active":"2023-07-14T16:13:59.295604+0000","last_became_peered":"2023-07-14T16:13:59.295604+0000","last_unstale":"2023-07-14T16:14:00.604893+0000","last_undegraded":"2023-07-14T16:14:00.604893+0000","last_fullsized":"2023-07-14T16:14:00.604893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:00.910666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.604768+0000","last_change":"2023-07-14T16:13:59.294198+0000","last_active":"2023-07-14T16:14:00.604768+0000","last_peered":"2023-07-14T16:14:00.604768+0000","last_clean":"2023-07-14T16:14:00.604768+0000","last_became_active":"2023-07-14T16:13:59.293891+0000","last_became_peered":"2023-07-14T16:13:59.293891+0000","last_unstale":"2023-07-14T16:14:00.604768+0000","last_undegraded":"2023-07-14T16:14:00.604768+0000","last_fullsized":"2023-07-14T16:14:00.604768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:37:04.786333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174107+0000","last_change":"2023-07-14T16:13:59.298724+0000","last_active":"2023-07-14T16:14:01.174107+0000","last_peered":"2023-07-14T16:14:01.174107+0000","last_clean":"2023-07-14T16:14:01.174107+0000","last_became_active":"2023-07-14T16:13:59.298318+0000","last_became_peered":"2023-07-14T16:13:59.298318+0000","last_unstale":"2023-07-14T16:14:01.174107+0000","last_undegraded":"2023-07-14T16:14:01.174107+0000","last_fullsized":"2023-07-14T16:14:01.174107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:25:27.131220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658791+0000","last_change":"2023-07-14T16:13:59.294652+0000","last_active":"2023-07-14T16:14:00.658791+0000","last_peered":"2023-07-14T16:14:00.658791+0000","last_clean":"2023-07-14T16:14:00.658791+0000","last_became_active":"2023-07-14T16:13:59.294278+0000","last_became_peered":"2023-07-14T16:13:59.294278+0000","last_unstale":"2023-07-14T16:14:00.658791+0000","last_undegraded":"2023-07-14T16:14:00.658791+0000","last_fullsized":"2023-07-14T16:14:00.658791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:38.225606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290756+0000","last_change":"2023-07-14T16:13:59.299228+0000","last_active":"2023-07-14T16:14:00.290756+0000","last_peered":"2023-07-14T16:14:00.290756+0000","last_clean":"2023-07-14T16:14:00.290756+0000","last_became_active":"2023-07-14T16:13:59.297982+0000","last_became_peered":"2023-07-14T16:13:59.297982+0000","last_unstale":"2023-07-14T16:14:00.290756+0000","last_undegraded":"2023-07-14T16:14:00.290756+0000","last_fullsized":"2023-07-14T16:14:00.290756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:30:52.523034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.605179+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:14:00.605179+0000","last_peered":"2023-07-14T16:14:00.605179+0000","last_clean":"2023-07-14T16:14:00.605179+0000","last_became_active":"2023-07-14T16:13:59.299181+0000","last_became_peered":"2023-07-14T16:13:59.299181+0000","last_unstale":"2023-07-14T16:14:00.605179+0000","last_undegraded":"2023-07-14T16:14:00.605179+0000","last_fullsized":"2023-07-14T16:14:00.605179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:12:50.267320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290291+0000","last_change":"2023-07-14T16:13:59.303282+0000","last_active":"2023-07-14T16:14:00.290291+0000","last_peered":"2023-07-14T16:14:00.290291+0000","last_clean":"2023-07-14T16:14:00.290291+0000","last_became_active":"2023-07-14T16:13:59.302922+0000","last_became_peered":"2023-07-14T16:13:59.302922+0000","last_unstale":"2023-07-14T16:14:00.290291+0000","last_undegraded":"2023-07-14T16:14:00.290291+0000","last_fullsized":"2023-07-14T16:14:00.290291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:06.464416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.492756+0000","last_change":"2023-07-14T16:13:59.296389+0000","last_active":"2023-07-14T16:14:00.492756+0000","last_peered":"2023-07-14T16:14:00.492756+0000","last_clean":"2023-07-14T16:14:00.492756+0000","last_became_active":"2023-07-14T16:13:59.296080+0000","last_became_peered":"2023-07-14T16:13:59.296080+0000","last_unstale":"2023-07-14T16:14:00.492756+0000","last_undegraded":"2023-07-14T16:14:00.492756+0000","last_fullsized":"2023-07-14T16:14:00.492756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:26:27.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290083+0000","last_change":"2023-07-14T16:13:59.297774+0000","last_active":"2023-07-14T16:14:00.290083+0000","last_peered":"2023-07-14T16:14:00.290083+0000","last_clean":"2023-07-14T16:14:00.290083+0000","last_became_active":"2023-07-14T16:13:59.297463+0000","last_became_peered":"2023-07-14T16:13:59.297463+0000","last_unstale":"2023-07-14T16:14:00.290083+0000","last_undegraded":"2023-07-14T16:14:00.290083+0000","last_fullsized":"2023-07-14T16:14:00.290083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:16.595689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286560+0000","last_change":"2023-07-14T16:13:59.306795+0000","last_active":"2023-07-14T16:14:00.286560+0000","last_peered":"2023-07-14T16:14:00.286560+0000","last_clean":"2023-07-14T16:14:00.286560+0000","last_became_active":"2023-07-14T16:13:59.306457+0000","last_became_peered":"2023-07-14T16:13:59.306457+0000","last_unstale":"2023-07-14T16:14:00.286560+0000","last_undegraded":"2023-07-14T16:14:00.286560+0000","last_fullsized":"2023-07-14T16:14:00.286560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:37:11.860651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658903+0000","last_change":"2023-07-14T16:13:59.305338+0000","last_active":"2023-07-14T16:14:00.658903+0000","last_peered":"2023-07-14T16:14:00.658903+0000","last_clean":"2023-07-14T16:14:00.658903+0000","last_became_active":"2023-07-14T16:13:59.305047+0000","last_became_peered":"2023-07-14T16:13:59.305047+0000","last_unstale":"2023-07-14T16:14:00.658903+0000","last_undegraded":"2023-07-14T16:14:00.658903+0000","last_fullsized":"2023-07-14T16:14:00.658903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:18:53.732011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290522+0000","last_change":"2023-07-14T16:13:59.299005+0000","last_active":"2023-07-14T16:14:00.290522+0000","last_peered":"2023-07-14T16:14:00.290522+0000","last_clean":"2023-07-14T16:14:00.290522+0000","last_became_active":"2023-07-14T16:13:59.298705+0000","last_became_peered":"2023-07-14T16:13:59.298705+0000","last_unstale":"2023-07-14T16:14:00.290522+0000","last_undegraded":"2023-07-14T16:14:00.290522+0000","last_fullsized":"2023-07-14T16:14:00.290522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:44:45.399081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247144,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5078,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":120,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":32325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":441942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.88400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.94999999999999996}]},{"osd":6,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.98299999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":9,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.016},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.1830000000000001}]},{"osd":10,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.081}]}]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":1033967,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.38}]},{"osd":2,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.501}]},{"osd":3,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.748},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.61699999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.65600000000000003}]},{"osd":9,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.71099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":442178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.746},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.94199999999999995}]},{"osd":2,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.68000000000000005}]},{"osd":3,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.879}]},{"osd":7,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90200000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.85699999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.002}]}]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":442098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.745,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":6.3410000000000002,"5min":6.3410000000000002,"15min":6.3410000000000002},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93600000000000005,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.53800000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.92000000000000004,"5min":1.216,"15min":1.216},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66000000000000003,"5min":1.081,"15min":1.081},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.747,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0720000000000001,"5min":1.175,"15min":1.175},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.48899999999999999}]},{"osd":4,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":1.149,"15min":1.149},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.996,"5min":1.218,"15min":1.218},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83099999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":1.21,"15min":1.21},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1160000000000001,"5min":1.29,"15min":1.29},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0309999999999999,"5min":1.256,"15min":1.256},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.92100000000000004,"15min":0.88700000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.3209999999999997,"5min":6.3209999999999997,"15min":6.3209999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.88300000000000001,"15min":0.89500000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.046,"5min":1.373,"15min":1.373},"last":0.77500000000000002}]},{"osd":9,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.621}]},{"osd":10,"last update":"Fri Jul 14 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.751}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":1033937,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.181,"15min":1.181},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.52000000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.88600000000000001,"5min":1.008,"15min":1.008},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.628}]},{"osd":3,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.36799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.111,"5min":1.137,"15min":1.137},"last":0.64200000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.84899999999999998,"15min":0.83699999999999997},"min":{"1min":0.61199999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.876}]},{"osd":9,"last update":"Fri Jul 14 16:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.84799999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":441450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.46000000000000002},"min":{"1min":0.34000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.69899999999999995,"15min":0.68600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.69599999999999995,"15min":0.67500000000000004},"min":{"1min":0.51000000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0640000000000001,"5min":1.111,"15min":1.111},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.72099999999999997},"min":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76200000000000001,"15min":0.73399999999999999},"min":{"1min":0.59799999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.0940000000000001,"5min":1.105,"15min":1.105},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.497,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.89300000000000002}]},{"osd":5,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79300000000000004,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.82599999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.65500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.748},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.82299999999999995,"15min":0.79700000000000004},"min":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.84299999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.94499999999999995,"15min":0.94199999999999995},"min":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.873},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.872,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92000000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.77600000000000002}]},{"osd":10,"last update":"Fri Jul 14 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.98499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":442924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97799999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.57799999999999996,"15min":0.52800000000000002},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Fri Jul 14 16:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.62,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.627,"15min":0.60899999999999999},"min":{"1min":0.51600000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.81000000000000005}]},{"osd":3,"last update":"Fri Jul 14 16:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.63,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.61399999999999999,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88800000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.53600000000000003,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95899999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.49199999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.50700000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.68899999999999995,"15min":0.66100000000000003},"min":{"1min":0.57899999999999996,"5min":0.37,"15min":0.37},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.89300000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84699999999999998,"15min":0.84099999999999997},"min":{"1min":0.69699999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.137,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.70299999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.878}]},{"osd":9,"last update":"Fri Jul 14 16:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.92700000000000005}]},{"osd":10,"last update":"Fri Jul 14 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.99299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":442838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69899999999999995,"15min":0.626},"min":{"1min":0.60099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.71199999999999997,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.871}]},{"osd":3,"last update":"Fri Jul 14 16:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74399999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.505}]},{"osd":4,"last update":"Fri Jul 14 16:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Jul 14 16:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.49099999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.04,"5min":1.127,"15min":1.127},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.748,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83799999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.91700000000000004,"15min":0.92500000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.1279999999999999,"5min":1.208,"15min":1.208},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.86299999999999999,"15min":0.86699999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0820000000000001,"5min":1.099,"15min":1.099},"last":0.71399999999999997}]},{"osd":9,"last update":"Fri Jul 14 16:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.64600000000000002}]},{"osd":10,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.80900000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":461591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.54900000000000004},"min":{"1min":0.379,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.56299999999999994},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90500000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0549999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.64200000000000002},"min":{"1min":0.48099999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.624},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64500000000000002,"15min":0.62},"min":{"1min":0.51500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Jul 14 16:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.99099999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.67000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.022,"5min":1.117,"15min":1.117},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.016},"last":0.52800000000000002}]},{"osd":9,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.754}]},{"osd":10,"last update":"Fri Jul 14 16:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":441183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.41899999999999998,"15min":0.39700000000000002},"min":{"1min":0.35699999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.44600000000000001,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.503,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64100000000000001,"15min":0.61699999999999999},"min":{"1min":0.45700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0269999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.505}]},{"osd":2,"last update":"Fri Jul 14 16:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.70099999999999996,"15min":0.67500000000000004},"min":{"1min":0.52400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72999999999999998,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.448,"15min":3.448},"last":0.623}]},{"osd":4,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.66000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":3.359,"15min":3.359},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71099999999999997,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96499999999999997,"5min":1.222,"15min":1.222},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Jul 14 16:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.64300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Fri Jul 14 16:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.042,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0329999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.81100000000000005,"15min":0.78700000000000003},"min":{"1min":0.746,"5min":0.499,"15min":0.499},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.754},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.624,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.60399999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.876,"15min":0.88500000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1259999999999999,"5min":1.546,"15min":1.546},"last":0.62},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91800000000000004,"15min":0.92100000000000004},"min":{"1min":0.61899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.163,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.58099999999999996}]},{"osd":10,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":441956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.95599999999999996,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46800000000000003,"15min":0.432},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.501,"15min":0.45700000000000002},"min":{"1min":0.35499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68000000000000005,"15min":0.64200000000000002},"min":{"1min":0.505,"5min":0.439,"15min":0.439},"max":{"1min":1.048,"5min":3.698,"15min":3.698},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.70199999999999996,"15min":0.67800000000000005},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1160000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.54900000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.995,"5min":1.03,"15min":1.03},"last":1.014}]},{"osd":5,"last update":"Fri Jul 14 16:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.52400000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2949999999999999,"5min":1.506,"15min":1.506},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.53800000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.89700000000000002}]},{"osd":6,"last update":"Fri Jul 14 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.72499999999999998,"15min":0.71099999999999997},"min":{"1min":0.55500000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0129999999999999,"5min":1.321,"15min":1.321},"last":0.879},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.56699999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Jul 14 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56699999999999995}]},{"osd":8,"last update":"Fri Jul 14 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.85099999999999998,"15min":0.83999999999999997},"min":{"1min":0.71699999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1040000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.66400000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.93899999999999995}]},{"osd":9,"last update":"Fri Jul 14 16:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.99299999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.81399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":1031579,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86499999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79000000000000004,"5min":1.131,"15min":1.131},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48599999999999999,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70299999999999996,"15min":0.65900000000000003},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93400000000000005,"5min":7.056,"15min":7.056},"last":0.62}]},{"osd":4,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70999999999999996,"15min":0.68999999999999995},"min":{"1min":0.58599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.997,"5min":1.052,"15min":1.052},"last":0.503},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.53500000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Jul 14 16:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.2569999999999999,"5min":7.109,"15min":7.109},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.754,"15min":0.746},"min":{"1min":0.629,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.79500000000000004,"15min":0.77900000000000003},"min":{"1min":0.65700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.503,"5min":0.437,"15min":0.437},"max":{"1min":1.0489999999999999,"5min":1.286,"15min":1.286},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78900000000000003,"15min":0.79300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.55500000000000005}]},{"osd":9,"last update":"Fri Jul 14 16:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.68100000000000005}]},{"osd":10,"last update":"Fri Jul 14 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T16:14:08.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T16:14:08.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:07 smithi164 ceph-mon[97427]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:08.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:07 smithi164 ceph-mon[97427]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:14:08.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:07 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1027444529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:14:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:07 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/347501096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:08.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 -- 172.21.15.62:0/227508461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc100cf0 msgr2=0x7fb0fc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:08.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 --2- 172.21.15.62:0/227508461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc100cf0 0x7fb0fc101140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009990 tx=0x7fb0e8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 -- 172.21.15.62:0/227508461 shutdown_connections 2023-07-14T16:14:08.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 --2- 172.21.15.62:0/227508461 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0fc101680 0x7fb0fc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 --2- 172.21.15.62:0/227508461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0fc0ffb00 0x7fb0fc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 --2- 172.21.15.62:0/227508461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc100cf0 0x7fb0fc101140 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 -- 172.21.15.62:0/227508461 >> 172.21.15.62:0/227508461 conn(0x7fb0fc0fb0e0 msgr2=0x7fb0fc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:08.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 -- 172.21.15.62:0/227508461 shutdown_connections 2023-07-14T16:14:08.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.278+0000 7fb1031ff700 1 -- 172.21.15.62:0/227508461 wait complete. 2023-07-14T16:14:08.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.279+0000 7fb1031ff700 1 Processor -- start 2023-07-14T16:14:08.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.279+0000 7fb1031ff700 1 -- start start 2023-07-14T16:14:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.279+0000 7fb1031ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc0ffb00 0x7fb0fc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.279+0000 7fb1031ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0fc100cf0 0x7fb0fc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.279+0000 7fb1031ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0fc101680 0x7fb0fc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.279+0000 7fb1031ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19f520 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.279+0000 7fb1031ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19f660 con 0x7fb0fc101680 2023-07-14T16:14:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.279+0000 7fb1031ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19f7f0 con 0x7fb0fc100cf0 2023-07-14T16:14:08.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc0ffb00 0x7fb0fc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:08.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0fc100cf0 0x7fb0fc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:08.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc0ffb00 0x7fb0fc1979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46246/0 (socket says 172.21.15.62:46246) 2023-07-14T16:14:08.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 -- 172.21.15.62:0/3145613671 learned_addr learned my addr 172.21.15.62:0/3145613671 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:08.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0fc100cf0 0x7fb0fc197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:48542/0 (socket says 172.21.15.62:48542) 2023-07-14T16:14:08.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb10179c700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0fc101680 0x7fb0fc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:08.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 -- 172.21.15.62:0/3145613671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0fc100cf0 msgr2=0x7fb0fc197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0fc100cf0 0x7fb0fc197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 -- 172.21.15.62:0/3145613671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0fc101680 msgr2=0x7fb0fc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0fc101680 0x7fb0fc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 -- 172.21.15.62:0/3145613671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80095f0 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.280+0000 7fb100f9b700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc0ffb00 0x7fb0fc1979a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000c970 tx=0x7fb0f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:08.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.281+0000 7fb0f9ffb700 1 -- 172.21.15.62:0/3145613671 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0004220 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.281+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc19fad0 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.281+0000 7fb0f9ffb700 1 -- 172.21.15.62:0/3145613671 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0f0007d30 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.281+0000 7fb0f9ffb700 1 -- 172.21.15.62:0/3145613671 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0003b30 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.281+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc1a0030 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.282+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc192110 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.284+0000 7fb0f9ffb700 1 -- 172.21.15.62:0/3145613671 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb0f0003cb0 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.285+0000 7fb0f9ffb700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ec0a1220 0x7fb0ec0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.285+0000 7fb0f9ffb700 1 -- 172.21.15.62:0/3145613671 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0f00cc600 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.286+0000 7fb0fbfff700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ec0a1220 0x7fb0ec0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.286+0000 7fb0fbfff700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ec0a1220 0x7fb0ec0a36d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c010 tx=0x7fb0e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.289+0000 7fb0f9ffb700 1 -- 172.21.15.62:0/3145613671 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb0f0095ca0 con 0x7fb0fc0ffb00 2023-07-14T16:14:08.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.497+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0fc060950 con 0x7fb0ec0a1220 2023-07-14T16:14:08.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.507+0000 7fb0f9ffb700 1 -- 172.21.15.62:0/3145613671 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+338939 (secure 0 0 0) 0x7fb0fc060950 con 0x7fb0ec0a1220 2023-07-14T16:14:08.486 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ec0a1220 msgr2=0x7fb0ec0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ec0a1220 0x7fb0ec0a36d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c010 tx=0x7fb0e8005fb0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc0ffb00 msgr2=0x7fb0fc1979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc0ffb00 0x7fb0fc1979a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000c970 tx=0x7fb0f000cc80 comp rx=0 tx=0).stop 2023-07-14T16:14:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 shutdown_connections 2023-07-14T16:14:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ec0a1220 0x7fb0ec0a36d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0fc101680 0x7fb0fc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0fc0ffb00 0x7fb0fc1979a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 --2- 172.21.15.62:0/3145613671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0fc100cf0 0x7fb0fc197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:08.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 >> 172.21.15.62:0/3145613671 conn(0x7fb0fc0fb0e0 msgr2=0x7fb0fc1065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:08.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 shutdown_connections 2023-07-14T16:14:08.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:08.512+0000 7fb1031ff700 1 -- 172.21.15.62:0/3145613671 wait complete. 2023-07-14T16:14:08.496 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-07-14T16:14:08.510 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2023-07-14T16:14:07.467601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595569,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":140,"num_read_kb":122,"num_write":302,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334584,"kb_used_data":11160,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607240,"statfs":{"total":1151940427776,"available":1151597813760,"internally_reserved":0,"allocated":11427840,"data_stored":7097643,"data_compressed":107328,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5201,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":121,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.190004"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290756+0000","last_change":"2023-07-14T16:13:59.301892+0000","last_active":"2023-07-14T16:14:00.290756+0000","last_peered":"2023-07-14T16:14:00.290756+0000","last_clean":"2023-07-14T16:14:00.290756+0000","last_became_active":"2023-07-14T16:13:59.301379+0000","last_became_peered":"2023-07-14T16:13:59.301379+0000","last_unstale":"2023-07-14T16:14:00.290756+0000","last_undegraded":"2023-07-14T16:14:00.290756+0000","last_fullsized":"2023-07-14T16:14:00.290756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:49:42.914040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173960+0000","last_change":"2023-07-14T16:13:59.309322+0000","last_active":"2023-07-14T16:14:01.173960+0000","last_peered":"2023-07-14T16:14:01.173960+0000","last_clean":"2023-07-14T16:14:01.173960+0000","last_became_active":"2023-07-14T16:13:59.309035+0000","last_became_peered":"2023-07-14T16:13:59.309035+0000","last_unstale":"2023-07-14T16:14:01.173960+0000","last_undegraded":"2023-07-14T16:14:01.173960+0000","last_fullsized":"2023-07-14T16:14:01.173960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:12.977618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174298+0000","last_change":"2023-07-14T16:13:59.304371+0000","last_active":"2023-07-14T16:14:01.174298+0000","last_peered":"2023-07-14T16:14:01.174298+0000","last_clean":"2023-07-14T16:14:01.174298+0000","last_became_active":"2023-07-14T16:13:59.303965+0000","last_became_peered":"2023-07-14T16:13:59.303965+0000","last_unstale":"2023-07-14T16:14:01.174298+0000","last_undegraded":"2023-07-14T16:14:01.174298+0000","last_fullsized":"2023-07-14T16:14:01.174298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:54.688081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173597+0000","last_change":"2023-07-14T16:13:59.303434+0000","last_active":"2023-07-14T16:14:01.173597+0000","last_peered":"2023-07-14T16:14:01.173597+0000","last_clean":"2023-07-14T16:14:01.173597+0000","last_became_active":"2023-07-14T16:13:59.302960+0000","last_became_peered":"2023-07-14T16:13:59.302960+0000","last_unstale":"2023-07-14T16:14:01.173597+0000","last_undegraded":"2023-07-14T16:14:01.173597+0000","last_fullsized":"2023-07-14T16:14:01.173597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:16.564005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175087+0000","last_change":"2023-07-14T16:13:59.298094+0000","last_active":"2023-07-14T16:14:01.175087+0000","last_peered":"2023-07-14T16:14:01.175087+0000","last_clean":"2023-07-14T16:14:01.175087+0000","last_became_active":"2023-07-14T16:13:59.297764+0000","last_became_peered":"2023-07-14T16:13:59.297764+0000","last_unstale":"2023-07-14T16:14:01.175087+0000","last_undegraded":"2023-07-14T16:14:01.175087+0000","last_fullsized":"2023-07-14T16:14:01.175087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:54:57.248080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577317+0000","last_change":"2023-07-14T16:13:59.299735+0000","last_active":"2023-07-14T16:14:00.577317+0000","last_peered":"2023-07-14T16:14:00.577317+0000","last_clean":"2023-07-14T16:14:00.577317+0000","last_became_active":"2023-07-14T16:13:59.299413+0000","last_became_peered":"2023-07-14T16:13:59.299413+0000","last_unstale":"2023-07-14T16:14:00.577317+0000","last_undegraded":"2023-07-14T16:14:00.577317+0000","last_fullsized":"2023-07-14T16:14:00.577317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:11:43.699831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.991748+0000","last_change":"2023-07-14T16:13:59.299682+0000","last_active":"2023-07-14T16:14:00.991748+0000","last_peered":"2023-07-14T16:14:00.991748+0000","last_clean":"2023-07-14T16:14:00.991748+0000","last_became_active":"2023-07-14T16:13:59.299327+0000","last_became_peered":"2023-07-14T16:13:59.299327+0000","last_unstale":"2023-07-14T16:14:00.991748+0000","last_undegraded":"2023-07-14T16:14:00.991748+0000","last_fullsized":"2023-07-14T16:14:00.991748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:44.359819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290178+0000","last_change":"2023-07-14T16:13:59.311818+0000","last_active":"2023-07-14T16:14:00.290178+0000","last_peered":"2023-07-14T16:14:00.290178+0000","last_clean":"2023-07-14T16:14:00.290178+0000","last_became_active":"2023-07-14T16:13:59.311478+0000","last_became_peered":"2023-07-14T16:13:59.311478+0000","last_unstale":"2023-07-14T16:14:00.290178+0000","last_undegraded":"2023-07-14T16:14:00.290178+0000","last_fullsized":"2023-07-14T16:14:00.290178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:20.336283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286736+0000","last_change":"2023-07-14T16:13:59.309665+0000","last_active":"2023-07-14T16:14:00.286736+0000","last_peered":"2023-07-14T16:14:00.286736+0000","last_clean":"2023-07-14T16:14:00.286736+0000","last_became_active":"2023-07-14T16:13:59.309361+0000","last_became_peered":"2023-07-14T16:13:59.309361+0000","last_unstale":"2023-07-14T16:14:00.286736+0000","last_undegraded":"2023-07-14T16:14:00.286736+0000","last_fullsized":"2023-07-14T16:14:00.286736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:35:51.164062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577723+0000","last_change":"2023-07-14T16:13:59.296397+0000","last_active":"2023-07-14T16:14:00.577723+0000","last_peered":"2023-07-14T16:14:00.577723+0000","last_clean":"2023-07-14T16:14:00.577723+0000","last_became_active":"2023-07-14T16:13:59.295965+0000","last_became_peered":"2023-07-14T16:13:59.295965+0000","last_unstale":"2023-07-14T16:14:00.577723+0000","last_undegraded":"2023-07-14T16:14:00.577723+0000","last_fullsized":"2023-07-14T16:14:00.577723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:24:07.343379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992110+0000","last_change":"2023-07-14T16:13:59.302892+0000","last_active":"2023-07-14T16:14:00.992110+0000","last_peered":"2023-07-14T16:14:00.992110+0000","last_clean":"2023-07-14T16:14:00.992110+0000","last_became_active":"2023-07-14T16:13:59.302295+0000","last_became_peered":"2023-07-14T16:13:59.302295+0000","last_unstale":"2023-07-14T16:14:00.992110+0000","last_undegraded":"2023-07-14T16:14:00.992110+0000","last_fullsized":"2023-07-14T16:14:00.992110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:39:08.544022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173544+0000","last_change":"2023-07-14T16:13:59.301913+0000","last_active":"2023-07-14T16:14:01.173544+0000","last_peered":"2023-07-14T16:14:01.173544+0000","last_clean":"2023-07-14T16:14:01.173544+0000","last_became_active":"2023-07-14T16:13:59.301590+0000","last_became_peered":"2023-07-14T16:13:59.301590+0000","last_unstale":"2023-07-14T16:14:01.173544+0000","last_undegraded":"2023-07-14T16:14:01.173544+0000","last_fullsized":"2023-07-14T16:14:01.173544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:59:58.070130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658997+0000","last_change":"2023-07-14T16:13:59.302006+0000","last_active":"2023-07-14T16:14:00.658997+0000","last_peered":"2023-07-14T16:14:00.658997+0000","last_clean":"2023-07-14T16:14:00.658997+0000","last_became_active":"2023-07-14T16:13:59.301752+0000","last_became_peered":"2023-07-14T16:13:59.301752+0000","last_unstale":"2023-07-14T16:14:00.658997+0000","last_undegraded":"2023-07-14T16:14:00.658997+0000","last_fullsized":"2023-07-14T16:14:00.658997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:16.314105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174059+0000","last_change":"2023-07-14T16:13:59.309318+0000","last_active":"2023-07-14T16:14:01.174059+0000","last_peered":"2023-07-14T16:14:01.174059+0000","last_clean":"2023-07-14T16:14:01.174059+0000","last_became_active":"2023-07-14T16:13:59.308931+0000","last_became_peered":"2023-07-14T16:13:59.308931+0000","last_unstale":"2023-07-14T16:14:01.174059+0000","last_undegraded":"2023-07-14T16:14:01.174059+0000","last_fullsized":"2023-07-14T16:14:01.174059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:00:49.873512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173842+0000","last_change":"2023-07-14T16:13:59.304410+0000","last_active":"2023-07-14T16:14:01.173842+0000","last_peered":"2023-07-14T16:14:01.173842+0000","last_clean":"2023-07-14T16:14:01.173842+0000","last_became_active":"2023-07-14T16:13:59.304022+0000","last_became_peered":"2023-07-14T16:13:59.304022+0000","last_unstale":"2023-07-14T16:14:01.173842+0000","last_undegraded":"2023-07-14T16:14:01.173842+0000","last_fullsized":"2023-07-14T16:14:01.173842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:02:41.890599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286140+0000","last_change":"2023-07-14T16:13:59.307986+0000","last_active":"2023-07-14T16:14:00.286140+0000","last_peered":"2023-07-14T16:14:00.286140+0000","last_clean":"2023-07-14T16:14:00.286140+0000","last_became_active":"2023-07-14T16:13:59.307701+0000","last_became_peered":"2023-07-14T16:13:59.307701+0000","last_unstale":"2023-07-14T16:14:00.286140+0000","last_undegraded":"2023-07-14T16:14:00.286140+0000","last_fullsized":"2023-07-14T16:14:00.286140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:36:37.853083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290724+0000","last_change":"2023-07-14T16:13:59.301685+0000","last_active":"2023-07-14T16:14:00.290724+0000","last_peered":"2023-07-14T16:14:00.290724+0000","last_clean":"2023-07-14T16:14:00.290724+0000","last_became_active":"2023-07-14T16:13:59.300905+0000","last_became_peered":"2023-07-14T16:13:59.300905+0000","last_unstale":"2023-07-14T16:14:00.290724+0000","last_undegraded":"2023-07-14T16:14:00.290724+0000","last_fullsized":"2023-07-14T16:14:00.290724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:27:22.529080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174068+0000","last_change":"2023-07-14T16:13:59.306027+0000","last_active":"2023-07-14T16:14:01.174068+0000","last_peered":"2023-07-14T16:14:01.174068+0000","last_clean":"2023-07-14T16:14:01.174068+0000","last_became_active":"2023-07-14T16:13:59.305718+0000","last_became_peered":"2023-07-14T16:13:59.305718+0000","last_unstale":"2023-07-14T16:14:01.174068+0000","last_undegraded":"2023-07-14T16:14:01.174068+0000","last_fullsized":"2023-07-14T16:14:01.174068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:14:59.099925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.605145+0000","last_change":"2023-07-14T16:13:59.294140+0000","last_active":"2023-07-14T16:14:00.605145+0000","last_peered":"2023-07-14T16:14:00.605145+0000","last_clean":"2023-07-14T16:14:00.605145+0000","last_became_active":"2023-07-14T16:13:59.293825+0000","last_became_peered":"2023-07-14T16:13:59.293825+0000","last_unstale":"2023-07-14T16:14:00.605145+0000","last_undegraded":"2023-07-14T16:14:00.605145+0000","last_fullsized":"2023-07-14T16:14:00.605145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:14:48.934852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578067+0000","last_change":"2023-07-14T16:13:58.277154+0000","last_active":"2023-07-14T16:14:03.578067+0000","last_peered":"2023-07-14T16:14:03.578067+0000","last_clean":"2023-07-14T16:14:03.578067+0000","last_became_active":"2023-07-14T16:13:58.276800+0000","last_became_peered":"2023-07-14T16:13:58.276800+0000","last_unstale":"2023-07-14T16:14:03.578067+0000","last_undegraded":"2023-07-14T16:14:03.578067+0000","last_fullsized":"2023-07-14T16:14:03.578067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:04.499469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290353+0000","last_change":"2023-07-14T16:13:59.299819+0000","last_active":"2023-07-14T16:14:00.290353+0000","last_peered":"2023-07-14T16:14:00.290353+0000","last_clean":"2023-07-14T16:14:00.290353+0000","last_became_active":"2023-07-14T16:13:59.299294+0000","last_became_peered":"2023-07-14T16:13:59.299294+0000","last_unstale":"2023-07-14T16:14:00.290353+0000","last_undegraded":"2023-07-14T16:14:00.290353+0000","last_fullsized":"2023-07-14T16:14:00.290353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:07:58.187097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290582+0000","last_change":"2023-07-14T16:13:58.278954+0000","last_active":"2023-07-14T16:14:00.290582+0000","last_peered":"2023-07-14T16:14:00.290582+0000","last_clean":"2023-07-14T16:14:00.290582+0000","last_became_active":"2023-07-14T16:13:58.278454+0000","last_became_peered":"2023-07-14T16:13:58.278454+0000","last_unstale":"2023-07-14T16:14:00.290582+0000","last_undegraded":"2023-07-14T16:14:00.290582+0000","last_fullsized":"2023-07-14T16:14:00.290582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:50:30.360986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174298+0000","last_change":"2023-07-14T16:13:59.299877+0000","last_active":"2023-07-14T16:14:01.174298+0000","last_peered":"2023-07-14T16:14:01.174298+0000","last_clean":"2023-07-14T16:14:01.174298+0000","last_became_active":"2023-07-14T16:13:59.299557+0000","last_became_peered":"2023-07-14T16:13:59.299557+0000","last_unstale":"2023-07-14T16:14:01.174298+0000","last_undegraded":"2023-07-14T16:14:01.174298+0000","last_fullsized":"2023-07-14T16:14:01.174298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:22:28.534140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.575576+0000","last_change":"2023-07-14T16:13:58.281711+0000","last_active":"2023-07-14T16:14:03.575576+0000","last_peered":"2023-07-14T16:14:03.575576+0000","last_clean":"2023-07-14T16:14:03.575576+0000","last_became_active":"2023-07-14T16:13:58.281185+0000","last_became_peered":"2023-07-14T16:13:58.281185+0000","last_unstale":"2023-07-14T16:14:03.575576+0000","last_undegraded":"2023-07-14T16:14:03.575576+0000","last_fullsized":"2023-07-14T16:14:03.575576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:12:59.648173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286382+0000","last_change":"2023-07-14T16:13:59.307987+0000","last_active":"2023-07-14T16:14:00.286382+0000","last_peered":"2023-07-14T16:14:00.286382+0000","last_clean":"2023-07-14T16:14:00.286382+0000","last_became_active":"2023-07-14T16:13:59.307457+0000","last_became_peered":"2023-07-14T16:13:59.307457+0000","last_unstale":"2023-07-14T16:14:00.286382+0000","last_undegraded":"2023-07-14T16:14:00.286382+0000","last_fullsized":"2023-07-14T16:14:00.286382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:26:38.798631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175418+0000","last_change":"2023-07-14T16:13:58.282332+0000","last_active":"2023-07-14T16:14:01.175418+0000","last_peered":"2023-07-14T16:14:01.175418+0000","last_clean":"2023-07-14T16:14:01.175418+0000","last_became_active":"2023-07-14T16:13:58.278103+0000","last_became_peered":"2023-07-14T16:13:58.278103+0000","last_unstale":"2023-07-14T16:14:01.175418+0000","last_undegraded":"2023-07-14T16:14:01.175418+0000","last_fullsized":"2023-07-14T16:14:01.175418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:23:14.369512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.493190+0000","last_change":"2023-07-14T16:13:59.301036+0000","last_active":"2023-07-14T16:14:00.493190+0000","last_peered":"2023-07-14T16:14:00.493190+0000","last_clean":"2023-07-14T16:14:00.493190+0000","last_became_active":"2023-07-14T16:13:59.300787+0000","last_became_peered":"2023-07-14T16:13:59.300787+0000","last_unstale":"2023-07-14T16:14:00.493190+0000","last_undegraded":"2023-07-14T16:14:00.493190+0000","last_fullsized":"2023-07-14T16:14:00.493190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:49:27.481208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.595824+0000","last_change":"2023-07-14T16:13:58.282392+0000","last_active":"2023-07-14T16:14:02.595824+0000","last_peered":"2023-07-14T16:14:02.595824+0000","last_clean":"2023-07-14T16:14:02.595824+0000","last_became_active":"2023-07-14T16:13:58.282030+0000","last_became_peered":"2023-07-14T16:13:58.282030+0000","last_unstale":"2023-07-14T16:14:02.595824+0000","last_undegraded":"2023-07-14T16:14:02.595824+0000","last_fullsized":"2023-07-14T16:14:02.595824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:43.582004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175117+0000","last_change":"2023-07-14T16:13:59.292681+0000","last_active":"2023-07-14T16:14:01.175117+0000","last_peered":"2023-07-14T16:14:01.175117+0000","last_clean":"2023-07-14T16:14:01.175117+0000","last_became_active":"2023-07-14T16:13:59.292289+0000","last_became_peered":"2023-07-14T16:13:59.292289+0000","last_unstale":"2023-07-14T16:14:01.175117+0000","last_undegraded":"2023-07-14T16:14:01.175117+0000","last_fullsized":"2023-07-14T16:14:01.175117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:43:34.481094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577227+0000","last_change":"2023-07-14T16:13:58.282111+0000","last_active":"2023-07-14T16:14:00.577227+0000","last_peered":"2023-07-14T16:14:00.577227+0000","last_clean":"2023-07-14T16:14:00.577227+0000","last_became_active":"2023-07-14T16:13:58.281783+0000","last_became_peered":"2023-07-14T16:13:58.281783+0000","last_unstale":"2023-07-14T16:14:00.577227+0000","last_undegraded":"2023-07-14T16:14:00.577227+0000","last_fullsized":"2023-07-14T16:14:00.577227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:59:52.556309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.493025+0000","last_change":"2023-07-14T16:13:59.297439+0000","last_active":"2023-07-14T16:14:00.493025+0000","last_peered":"2023-07-14T16:14:00.493025+0000","last_clean":"2023-07-14T16:14:00.493025+0000","last_became_active":"2023-07-14T16:13:59.297117+0000","last_became_peered":"2023-07-14T16:13:59.297117+0000","last_unstale":"2023-07-14T16:14:00.493025+0000","last_undegraded":"2023-07-14T16:14:00.493025+0000","last_fullsized":"2023-07-14T16:14:00.493025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:04:47.449697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578028+0000","last_change":"2023-07-14T16:13:58.274826+0000","last_active":"2023-07-14T16:14:03.578028+0000","last_peered":"2023-07-14T16:14:03.578028+0000","last_clean":"2023-07-14T16:14:03.578028+0000","last_became_active":"2023-07-14T16:13:58.274406+0000","last_became_peered":"2023-07-14T16:13:58.274406+0000","last_unstale":"2023-07-14T16:14:03.578028+0000","last_undegraded":"2023-07-14T16:14:03.578028+0000","last_fullsized":"2023-07-14T16:14:03.578028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:51:38.339653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290081+0000","last_change":"2023-07-14T16:13:59.296730+0000","last_active":"2023-07-14T16:14:00.290081+0000","last_peered":"2023-07-14T16:14:00.290081+0000","last_clean":"2023-07-14T16:14:00.290081+0000","last_became_active":"2023-07-14T16:13:59.296372+0000","last_became_peered":"2023-07-14T16:13:59.296372+0000","last_unstale":"2023-07-14T16:14:00.290081+0000","last_undegraded":"2023-07-14T16:14:00.290081+0000","last_fullsized":"2023-07-14T16:14:00.290081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:59.457423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.492695+0000","last_change":"2023-07-14T16:13:58.269122+0000","last_active":"2023-07-14T16:14:00.492695+0000","last_peered":"2023-07-14T16:14:00.492695+0000","last_clean":"2023-07-14T16:14:00.492695+0000","last_became_active":"2023-07-14T16:13:58.268770+0000","last_became_peered":"2023-07-14T16:13:58.268770+0000","last_unstale":"2023-07-14T16:14:00.492695+0000","last_undegraded":"2023-07-14T16:14:00.492695+0000","last_fullsized":"2023-07-14T16:14:00.492695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:15.228768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173621+0000","last_change":"2023-07-14T16:13:59.306987+0000","last_active":"2023-07-14T16:14:01.173621+0000","last_peered":"2023-07-14T16:14:01.173621+0000","last_clean":"2023-07-14T16:14:01.173621+0000","last_became_active":"2023-07-14T16:13:59.306649+0000","last_became_peered":"2023-07-14T16:13:59.306649+0000","last_unstale":"2023-07-14T16:14:01.173621+0000","last_undegraded":"2023-07-14T16:14:01.173621+0000","last_fullsized":"2023-07-14T16:14:01.173621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:25:37.926244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.597656+0000","last_change":"2023-07-14T16:13:58.281319+0000","last_active":"2023-07-14T16:14:02.597656+0000","last_peered":"2023-07-14T16:14:02.597656+0000","last_clean":"2023-07-14T16:14:02.597656+0000","last_became_active":"2023-07-14T16:13:58.280928+0000","last_became_peered":"2023-07-14T16:13:58.280928+0000","last_unstale":"2023-07-14T16:14:02.597656+0000","last_undegraded":"2023-07-14T16:14:02.597656+0000","last_fullsized":"2023-07-14T16:14:02.597656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:11:11.509173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286496+0000","last_change":"2023-07-14T16:13:59.309586+0000","last_active":"2023-07-14T16:14:00.286496+0000","last_peered":"2023-07-14T16:14:00.286496+0000","last_clean":"2023-07-14T16:14:00.286496+0000","last_became_active":"2023-07-14T16:13:59.309243+0000","last_became_peered":"2023-07-14T16:13:59.309243+0000","last_unstale":"2023-07-14T16:14:00.286496+0000","last_undegraded":"2023-07-14T16:14:00.286496+0000","last_fullsized":"2023-07-14T16:14:00.286496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:38.787965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.596724+0000","last_change":"2023-07-14T16:13:58.279769+0000","last_active":"2023-07-14T16:14:02.596724+0000","last_peered":"2023-07-14T16:14:02.596724+0000","last_clean":"2023-07-14T16:14:02.596724+0000","last_became_active":"2023-07-14T16:13:58.279409+0000","last_became_peered":"2023-07-14T16:13:58.279409+0000","last_unstale":"2023-07-14T16:14:02.596724+0000","last_undegraded":"2023-07-14T16:14:02.596724+0000","last_fullsized":"2023-07-14T16:14:02.596724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:51:40.918469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174342+0000","last_change":"2023-07-14T16:13:59.300382+0000","last_active":"2023-07-14T16:14:01.174342+0000","last_peered":"2023-07-14T16:14:01.174342+0000","last_clean":"2023-07-14T16:14:01.174342+0000","last_became_active":"2023-07-14T16:13:59.299981+0000","last_became_peered":"2023-07-14T16:13:59.299981+0000","last_unstale":"2023-07-14T16:14:01.174342+0000","last_undegraded":"2023-07-14T16:14:01.174342+0000","last_fullsized":"2023-07-14T16:14:01.174342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:23.017109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.493269+0000","last_change":"2023-07-14T16:13:58.274128+0000","last_active":"2023-07-14T16:14:00.493269+0000","last_peered":"2023-07-14T16:14:00.493269+0000","last_clean":"2023-07-14T16:14:00.493269+0000","last_became_active":"2023-07-14T16:13:58.273446+0000","last_became_peered":"2023-07-14T16:13:58.273446+0000","last_unstale":"2023-07-14T16:14:00.493269+0000","last_undegraded":"2023-07-14T16:14:00.493269+0000","last_fullsized":"2023-07-14T16:14:00.493269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:16:16.599740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286535+0000","last_change":"2023-07-14T16:13:59.301551+0000","last_active":"2023-07-14T16:14:00.286535+0000","last_peered":"2023-07-14T16:14:00.286535+0000","last_clean":"2023-07-14T16:14:00.286535+0000","last_became_active":"2023-07-14T16:13:59.301206+0000","last_became_peered":"2023-07-14T16:13:59.301206+0000","last_unstale":"2023-07-14T16:14:00.286535+0000","last_undegraded":"2023-07-14T16:14:00.286535+0000","last_fullsized":"2023-07-14T16:14:00.286535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:07:40.084145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.597342+0000","last_change":"2023-07-14T16:13:58.283304+0000","last_active":"2023-07-14T16:14:02.597342+0000","last_peered":"2023-07-14T16:14:02.597342+0000","last_clean":"2023-07-14T16:14:02.597342+0000","last_became_active":"2023-07-14T16:13:58.282930+0000","last_became_peered":"2023-07-14T16:13:58.282930+0000","last_unstale":"2023-07-14T16:14:02.597342+0000","last_undegraded":"2023-07-14T16:14:02.597342+0000","last_fullsized":"2023-07-14T16:14:02.597342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:00.947603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175257+0000","last_change":"2023-07-14T16:13:59.304144+0000","last_active":"2023-07-14T16:14:01.175257+0000","last_peered":"2023-07-14T16:14:01.175257+0000","last_clean":"2023-07-14T16:14:01.175257+0000","last_became_active":"2023-07-14T16:13:59.303745+0000","last_became_peered":"2023-07-14T16:13:59.303745+0000","last_unstale":"2023-07-14T16:14:01.175257+0000","last_undegraded":"2023-07-14T16:14:01.175257+0000","last_fullsized":"2023-07-14T16:14:01.175257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:58:55.949263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.577159+0000","last_change":"2023-07-14T16:13:58.283734+0000","last_active":"2023-07-14T16:14:03.577159+0000","last_peered":"2023-07-14T16:14:03.577159+0000","last_clean":"2023-07-14T16:14:03.577159+0000","last_became_active":"2023-07-14T16:13:58.282765+0000","last_became_peered":"2023-07-14T16:13:58.282765+0000","last_unstale":"2023-07-14T16:14:03.577159+0000","last_undegraded":"2023-07-14T16:14:03.577159+0000","last_fullsized":"2023-07-14T16:14:03.577159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:35:33.247905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658906+0000","last_change":"2023-07-14T16:13:59.308445+0000","last_active":"2023-07-14T16:14:00.658906+0000","last_peered":"2023-07-14T16:14:00.658906+0000","last_clean":"2023-07-14T16:14:00.658906+0000","last_became_active":"2023-07-14T16:13:59.308130+0000","last_became_peered":"2023-07-14T16:13:59.308130+0000","last_unstale":"2023-07-14T16:14:00.658906+0000","last_undegraded":"2023-07-14T16:14:00.658906+0000","last_fullsized":"2023-07-14T16:14:00.658906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:40:57.345220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.596174+0000","last_change":"2023-07-14T16:13:58.273973+0000","last_active":"2023-07-14T16:14:02.596174+0000","last_peered":"2023-07-14T16:14:02.596174+0000","last_clean":"2023-07-14T16:14:02.596174+0000","last_became_active":"2023-07-14T16:13:58.273606+0000","last_became_peered":"2023-07-14T16:13:58.273606+0000","last_unstale":"2023-07-14T16:14:02.596174+0000","last_undegraded":"2023-07-14T16:14:02.596174+0000","last_fullsized":"2023-07-14T16:14:02.596174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:52:13.396401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992012+0000","last_change":"2023-07-14T16:13:59.306428+0000","last_active":"2023-07-14T16:14:00.992012+0000","last_peered":"2023-07-14T16:14:00.992012+0000","last_clean":"2023-07-14T16:14:00.992012+0000","last_became_active":"2023-07-14T16:13:59.306090+0000","last_became_peered":"2023-07-14T16:13:59.306090+0000","last_unstale":"2023-07-14T16:14:00.992012+0000","last_undegraded":"2023-07-14T16:14:00.992012+0000","last_fullsized":"2023-07-14T16:14:00.992012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:02:50.168112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:04.600599+0000","last_change":"2023-07-14T16:13:58.277123+0000","last_active":"2023-07-14T16:14:04.600599+0000","last_peered":"2023-07-14T16:14:04.600599+0000","last_clean":"2023-07-14T16:14:04.600599+0000","last_became_active":"2023-07-14T16:13:58.276815+0000","last_became_peered":"2023-07-14T16:13:58.276815+0000","last_unstale":"2023-07-14T16:14:04.600599+0000","last_undegraded":"2023-07-14T16:14:04.600599+0000","last_fullsized":"2023-07-14T16:14:04.600599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:23.871523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.991732+0000","last_change":"2023-07-14T16:13:59.310238+0000","last_active":"2023-07-14T16:14:00.991732+0000","last_peered":"2023-07-14T16:14:00.991732+0000","last_clean":"2023-07-14T16:14:00.991732+0000","last_became_active":"2023-07-14T16:13:59.309918+0000","last_became_peered":"2023-07-14T16:13:59.309918+0000","last_unstale":"2023-07-14T16:14:00.991732+0000","last_undegraded":"2023-07-14T16:14:00.991732+0000","last_fullsized":"2023-07-14T16:14:00.991732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:12.761924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578594+0000","last_change":"2023-07-14T16:13:58.280834+0000","last_active":"2023-07-14T16:14:03.578594+0000","last_peered":"2023-07-14T16:14:03.578594+0000","last_clean":"2023-07-14T16:14:03.578594+0000","last_became_active":"2023-07-14T16:13:58.280497+0000","last_became_peered":"2023-07-14T16:13:58.280497+0000","last_unstale":"2023-07-14T16:14:03.578594+0000","last_undegraded":"2023-07-14T16:14:03.578594+0000","last_fullsized":"2023-07-14T16:14:03.578594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:08:09.982085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.492856+0000","last_change":"2023-07-14T16:13:59.301464+0000","last_active":"2023-07-14T16:14:00.492856+0000","last_peered":"2023-07-14T16:14:00.492856+0000","last_clean":"2023-07-14T16:14:00.492856+0000","last_became_active":"2023-07-14T16:13:59.301246+0000","last_became_peered":"2023-07-14T16:13:59.301246+0000","last_unstale":"2023-07-14T16:14:00.492856+0000","last_undegraded":"2023-07-14T16:14:00.492856+0000","last_fullsized":"2023-07-14T16:14:00.492856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:03:47.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.175347+0000","last_change":"2023-07-14T16:13:58.278480+0000","last_active":"2023-07-14T16:14:01.175347+0000","last_peered":"2023-07-14T16:14:01.175347+0000","last_clean":"2023-07-14T16:14:01.175347+0000","last_became_active":"2023-07-14T16:13:58.277906+0000","last_became_peered":"2023-07-14T16:13:58.277906+0000","last_unstale":"2023-07-14T16:14:01.175347+0000","last_undegraded":"2023-07-14T16:14:01.175347+0000","last_fullsized":"2023-07-14T16:14:01.175347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:27:51.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290556+0000","last_change":"2023-07-14T16:13:59.297644+0000","last_active":"2023-07-14T16:14:00.290556+0000","last_peered":"2023-07-14T16:14:00.290556+0000","last_clean":"2023-07-14T16:14:00.290556+0000","last_became_active":"2023-07-14T16:13:59.297332+0000","last_became_peered":"2023-07-14T16:13:59.297332+0000","last_unstale":"2023-07-14T16:14:00.290556+0000","last_undegraded":"2023-07-14T16:14:00.290556+0000","last_fullsized":"2023-07-14T16:14:00.290556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:22.525437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.173843+0000","last_change":"2023-07-14T16:13:58.280632+0000","last_active":"2023-07-14T16:14:01.173843+0000","last_peered":"2023-07-14T16:14:01.173843+0000","last_clean":"2023-07-14T16:14:01.173843+0000","last_became_active":"2023-07-14T16:13:58.280296+0000","last_became_peered":"2023-07-14T16:13:58.280296+0000","last_unstale":"2023-07-14T16:14:01.173843+0000","last_undegraded":"2023-07-14T16:14:01.173843+0000","last_fullsized":"2023-07-14T16:14:01.173843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:28.463404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577583+0000","last_change":"2023-07-14T16:13:59.297092+0000","last_active":"2023-07-14T16:14:00.577583+0000","last_peered":"2023-07-14T16:14:00.577583+0000","last_clean":"2023-07-14T16:14:00.577583+0000","last_became_active":"2023-07-14T16:13:59.296750+0000","last_became_peered":"2023-07-14T16:13:59.296750+0000","last_unstale":"2023-07-14T16:14:00.577583+0000","last_undegraded":"2023-07-14T16:14:00.577583+0000","last_fullsized":"2023-07-14T16:14:00.577583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:16:36.897671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.395958+0000","last_change":"2023-07-14T16:13:58.281786+0000","last_active":"2023-07-14T16:14:01.395958+0000","last_peered":"2023-07-14T16:14:01.395958+0000","last_clean":"2023-07-14T16:14:01.395958+0000","last_became_active":"2023-07-14T16:13:58.281471+0000","last_became_peered":"2023-07-14T16:13:58.281471+0000","last_unstale":"2023-07-14T16:14:01.395958+0000","last_undegraded":"2023-07-14T16:14:01.395958+0000","last_fullsized":"2023-07-14T16:14:01.395958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:23:49.206754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.577341+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:14:00.577341+0000","last_peered":"2023-07-14T16:14:00.577341+0000","last_clean":"2023-07-14T16:14:00.577341+0000","last_became_active":"2023-07-14T16:13:59.299157+0000","last_became_peered":"2023-07-14T16:13:59.299157+0000","last_unstale":"2023-07-14T16:14:00.577341+0000","last_undegraded":"2023-07-14T16:14:00.577341+0000","last_fullsized":"2023-07-14T16:14:00.577341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:07:10.431621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.492945+0000","last_change":"2023-07-14T16:13:59.288836+0000","last_active":"2023-07-14T16:14:00.492945+0000","last_peered":"2023-07-14T16:14:00.492945+0000","last_clean":"2023-07-14T16:14:00.492945+0000","last_became_active":"2023-07-14T16:13:59.288517+0000","last_became_peered":"2023-07-14T16:13:59.288517+0000","last_unstale":"2023-07-14T16:14:00.492945+0000","last_undegraded":"2023-07-14T16:14:00.492945+0000","last_fullsized":"2023-07-14T16:14:00.492945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:49:17.986192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.577752+0000","last_change":"2023-07-14T16:13:58.282415+0000","last_active":"2023-07-14T16:14:03.577752+0000","last_peered":"2023-07-14T16:14:03.577752+0000","last_clean":"2023-07-14T16:14:03.577752+0000","last_became_active":"2023-07-14T16:13:58.282118+0000","last_became_peered":"2023-07-14T16:13:58.282118+0000","last_unstale":"2023-07-14T16:14:03.577752+0000","last_undegraded":"2023-07-14T16:14:03.577752+0000","last_fullsized":"2023-07-14T16:14:03.577752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:14:30.064921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290603+0000","last_change":"2023-07-14T16:13:59.302860+0000","last_active":"2023-07-14T16:14:00.290603+0000","last_peered":"2023-07-14T16:14:00.290603+0000","last_clean":"2023-07-14T16:14:00.290603+0000","last_became_active":"2023-07-14T16:13:59.302454+0000","last_became_peered":"2023-07-14T16:13:59.302454+0000","last_unstale":"2023-07-14T16:14:00.290603+0000","last_undegraded":"2023-07-14T16:14:00.290603+0000","last_fullsized":"2023-07-14T16:14:00.290603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:28:49.992295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.576848+0000","last_change":"2023-07-14T16:13:58.273908+0000","last_active":"2023-07-14T16:14:03.576848+0000","last_peered":"2023-07-14T16:14:03.576848+0000","last_clean":"2023-07-14T16:14:03.576848+0000","last_became_active":"2023-07-14T16:13:58.273497+0000","last_became_peered":"2023-07-14T16:13:58.273497+0000","last_unstale":"2023-07-14T16:14:03.576848+0000","last_undegraded":"2023-07-14T16:14:03.576848+0000","last_fullsized":"2023-07-14T16:14:03.576848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:12:07.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.991727+0000","last_change":"2023-07-14T16:12:36.546475+0000","last_active":"2023-07-14T16:14:00.991727+0000","last_peered":"2023-07-14T16:14:00.991727+0000","last_clean":"2023-07-14T16:14:00.991727+0000","last_became_active":"2023-07-14T16:12:36.545943+0000","last_became_peered":"2023-07-14T16:12:36.545943+0000","last_unstale":"2023-07-14T16:14:00.991727+0000","last_undegraded":"2023-07-14T16:14:00.991727+0000","last_fullsized":"2023-07-14T16:14:00.991727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_clean_scrub_stamp":"2023-07-14T16:08:58.024408+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:41.410101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658527+0000","last_change":"2023-07-14T16:13:59.297669+0000","last_active":"2023-07-14T16:14:00.658527+0000","last_peered":"2023-07-14T16:14:00.658527+0000","last_clean":"2023-07-14T16:14:00.658527+0000","last_became_active":"2023-07-14T16:13:59.297364+0000","last_became_peered":"2023-07-14T16:13:59.297364+0000","last_unstale":"2023-07-14T16:14:00.658527+0000","last_undegraded":"2023-07-14T16:14:00.658527+0000","last_fullsized":"2023-07-14T16:14:00.658527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:08:05.988095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.576274+0000","last_change":"2023-07-14T16:13:58.276850+0000","last_active":"2023-07-14T16:14:03.576274+0000","last_peered":"2023-07-14T16:14:03.576274+0000","last_clean":"2023-07-14T16:14:03.576274+0000","last_became_active":"2023-07-14T16:13:58.276504+0000","last_became_peered":"2023-07-14T16:13:58.276504+0000","last_unstale":"2023-07-14T16:14:03.576274+0000","last_undegraded":"2023-07-14T16:14:03.576274+0000","last_fullsized":"2023-07-14T16:14:03.576274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:42:39.490295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.594373+0000","last_change":"2023-07-14T16:13:58.276257+0000","last_active":"2023-07-14T16:14:02.594373+0000","last_peered":"2023-07-14T16:14:02.594373+0000","last_clean":"2023-07-14T16:14:02.594373+0000","last_became_active":"2023-07-14T16:13:58.275910+0000","last_became_peered":"2023-07-14T16:13:58.275910+0000","last_unstale":"2023-07-14T16:14:02.594373+0000","last_undegraded":"2023-07-14T16:14:02.594373+0000","last_fullsized":"2023-07-14T16:14:02.594373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:51:40.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658945+0000","last_change":"2023-07-14T16:13:59.299135+0000","last_active":"2023-07-14T16:14:00.658945+0000","last_peered":"2023-07-14T16:14:00.658945+0000","last_clean":"2023-07-14T16:14:00.658945+0000","last_became_active":"2023-07-14T16:13:59.298031+0000","last_became_peered":"2023-07-14T16:13:59.298031+0000","last_unstale":"2023-07-14T16:14:00.658945+0000","last_undegraded":"2023-07-14T16:14:00.658945+0000","last_fullsized":"2023-07-14T16:14:00.658945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:53:54.675199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578745+0000","last_change":"2023-07-14T16:13:58.279664+0000","last_active":"2023-07-14T16:14:03.578745+0000","last_peered":"2023-07-14T16:14:03.578745+0000","last_clean":"2023-07-14T16:14:03.578745+0000","last_became_active":"2023-07-14T16:13:58.279137+0000","last_became_peered":"2023-07-14T16:13:58.279137+0000","last_unstale":"2023-07-14T16:14:03.578745+0000","last_undegraded":"2023-07-14T16:14:03.578745+0000","last_fullsized":"2023-07-14T16:14:03.578745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:12:14.121097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290612+0000","last_change":"2023-07-14T16:13:59.305536+0000","last_active":"2023-07-14T16:14:00.290612+0000","last_peered":"2023-07-14T16:14:00.290612+0000","last_clean":"2023-07-14T16:14:00.290612+0000","last_became_active":"2023-07-14T16:13:59.305176+0000","last_became_peered":"2023-07-14T16:13:59.305176+0000","last_unstale":"2023-07-14T16:14:00.290612+0000","last_undegraded":"2023-07-14T16:14:00.290612+0000","last_fullsized":"2023-07-14T16:14:00.290612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:46:20.875570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.596570+0000","last_change":"2023-07-14T16:13:58.269652+0000","last_active":"2023-07-14T16:14:02.596570+0000","last_peered":"2023-07-14T16:14:02.596570+0000","last_clean":"2023-07-14T16:14:02.596570+0000","last_became_active":"2023-07-14T16:13:58.269338+0000","last_became_peered":"2023-07-14T16:13:58.269338+0000","last_unstale":"2023-07-14T16:14:02.596570+0000","last_undegraded":"2023-07-14T16:14:02.596570+0000","last_fullsized":"2023-07-14T16:14:02.596570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:17:54.070686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174080+0000","last_change":"2023-07-14T16:13:59.297574+0000","last_active":"2023-07-14T16:14:01.174080+0000","last_peered":"2023-07-14T16:14:01.174080+0000","last_clean":"2023-07-14T16:14:01.174080+0000","last_became_active":"2023-07-14T16:13:59.297124+0000","last_became_peered":"2023-07-14T16:13:59.297124+0000","last_unstale":"2023-07-14T16:14:01.174080+0000","last_undegraded":"2023-07-14T16:14:01.174080+0000","last_fullsized":"2023-07-14T16:14:01.174080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:42:49.037176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.595928+0000","last_change":"2023-07-14T16:13:58.278011+0000","last_active":"2023-07-14T16:14:02.595928+0000","last_peered":"2023-07-14T16:14:02.595928+0000","last_clean":"2023-07-14T16:14:02.595928+0000","last_became_active":"2023-07-14T16:13:58.277563+0000","last_became_peered":"2023-07-14T16:13:58.277563+0000","last_unstale":"2023-07-14T16:14:02.595928+0000","last_undegraded":"2023-07-14T16:14:02.595928+0000","last_fullsized":"2023-07-14T16:14:02.595928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:50.032636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174973+0000","last_change":"2023-07-14T16:13:59.308836+0000","last_active":"2023-07-14T16:14:01.174973+0000","last_peered":"2023-07-14T16:14:01.174973+0000","last_clean":"2023-07-14T16:14:01.174973+0000","last_became_active":"2023-07-14T16:13:59.308510+0000","last_became_peered":"2023-07-14T16:13:59.308510+0000","last_unstale":"2023-07-14T16:14:01.174973+0000","last_undegraded":"2023-07-14T16:14:01.174973+0000","last_fullsized":"2023-07-14T16:14:01.174973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:54:55.671765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.597865+0000","last_change":"2023-07-14T16:13:58.277438+0000","last_active":"2023-07-14T16:14:02.597865+0000","last_peered":"2023-07-14T16:14:02.597865+0000","last_clean":"2023-07-14T16:14:02.597865+0000","last_became_active":"2023-07-14T16:13:58.277114+0000","last_became_peered":"2023-07-14T16:13:58.277114+0000","last_unstale":"2023-07-14T16:14:02.597865+0000","last_undegraded":"2023-07-14T16:14:02.597865+0000","last_fullsized":"2023-07-14T16:14:02.597865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:01:02.972663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290717+0000","last_change":"2023-07-14T16:13:59.304491+0000","last_active":"2023-07-14T16:14:00.290717+0000","last_peered":"2023-07-14T16:14:00.290717+0000","last_clean":"2023-07-14T16:14:00.290717+0000","last_became_active":"2023-07-14T16:13:59.303925+0000","last_became_peered":"2023-07-14T16:13:59.303925+0000","last_unstale":"2023-07-14T16:14:00.290717+0000","last_undegraded":"2023-07-14T16:14:00.290717+0000","last_fullsized":"2023-07-14T16:14:00.290717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:16:43.617439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.604769+0000","last_change":"2023-07-14T16:13:58.286276+0000","last_active":"2023-07-14T16:14:00.604769+0000","last_peered":"2023-07-14T16:14:00.604769+0000","last_clean":"2023-07-14T16:14:00.604769+0000","last_became_active":"2023-07-14T16:13:58.285855+0000","last_became_peered":"2023-07-14T16:13:58.285855+0000","last_unstale":"2023-07-14T16:14:00.604769+0000","last_undegraded":"2023-07-14T16:14:00.604769+0000","last_fullsized":"2023-07-14T16:14:00.604769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:31:05.621307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290474+0000","last_change":"2023-07-14T16:13:59.310983+0000","last_active":"2023-07-14T16:14:00.290474+0000","last_peered":"2023-07-14T16:14:00.290474+0000","last_clean":"2023-07-14T16:14:00.290474+0000","last_became_active":"2023-07-14T16:13:59.310636+0000","last_became_peered":"2023-07-14T16:13:59.310636+0000","last_unstale":"2023-07-14T16:14:00.290474+0000","last_undegraded":"2023-07-14T16:14:00.290474+0000","last_fullsized":"2023-07-14T16:14:00.290474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:39.709513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:02.597150+0000","last_change":"2023-07-14T16:13:58.280153+0000","last_active":"2023-07-14T16:14:02.597150+0000","last_peered":"2023-07-14T16:14:02.597150+0000","last_clean":"2023-07-14T16:14:02.597150+0000","last_became_active":"2023-07-14T16:13:58.279771+0000","last_became_peered":"2023-07-14T16:13:58.279771+0000","last_unstale":"2023-07-14T16:14:02.597150+0000","last_undegraded":"2023-07-14T16:14:02.597150+0000","last_fullsized":"2023-07-14T16:14:02.597150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:48:48.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992111+0000","last_change":"2023-07-14T16:13:59.304854+0000","last_active":"2023-07-14T16:14:00.992111+0000","last_peered":"2023-07-14T16:14:00.992111+0000","last_clean":"2023-07-14T16:14:00.992111+0000","last_became_active":"2023-07-14T16:13:59.304374+0000","last_became_peered":"2023-07-14T16:13:59.304374+0000","last_unstale":"2023-07-14T16:14:00.992111+0000","last_undegraded":"2023-07-14T16:14:00.992111+0000","last_fullsized":"2023-07-14T16:14:00.992111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:16.308607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:03.578934+0000","last_change":"2023-07-14T16:13:58.281885+0000","last_active":"2023-07-14T16:14:03.578934+0000","last_peered":"2023-07-14T16:14:03.578934+0000","last_clean":"2023-07-14T16:14:03.578934+0000","last_became_active":"2023-07-14T16:13:58.278759+0000","last_became_peered":"2023-07-14T16:13:58.278759+0000","last_unstale":"2023-07-14T16:14:03.578934+0000","last_undegraded":"2023-07-14T16:14:03.578934+0000","last_fullsized":"2023-07-14T16:14:03.578934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:41:05.139803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992167+0000","last_change":"2023-07-14T16:13:59.302680+0000","last_active":"2023-07-14T16:14:00.992167+0000","last_peered":"2023-07-14T16:14:00.992167+0000","last_clean":"2023-07-14T16:14:00.992167+0000","last_became_active":"2023-07-14T16:13:59.301854+0000","last_became_peered":"2023-07-14T16:13:59.301854+0000","last_unstale":"2023-07-14T16:14:00.992167+0000","last_undegraded":"2023-07-14T16:14:00.992167+0000","last_fullsized":"2023-07-14T16:14:00.992167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:09:35.155529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.493130+0000","last_change":"2023-07-14T16:13:58.274254+0000","last_active":"2023-07-14T16:14:00.493130+0000","last_peered":"2023-07-14T16:14:00.493130+0000","last_clean":"2023-07-14T16:14:00.493130+0000","last_became_active":"2023-07-14T16:13:58.273933+0000","last_became_peered":"2023-07-14T16:13:58.273933+0000","last_unstale":"2023-07-14T16:14:00.493130+0000","last_undegraded":"2023-07-14T16:14:00.493130+0000","last_fullsized":"2023-07-14T16:14:00.493130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:55:43.492876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.395985+0000","last_change":"2023-07-14T16:13:59.305615+0000","last_active":"2023-07-14T16:14:01.395985+0000","last_peered":"2023-07-14T16:14:01.395985+0000","last_clean":"2023-07-14T16:14:01.395985+0000","last_became_active":"2023-07-14T16:13:59.305206+0000","last_became_peered":"2023-07-14T16:13:59.305206+0000","last_unstale":"2023-07-14T16:14:01.395985+0000","last_undegraded":"2023-07-14T16:14:01.395985+0000","last_fullsized":"2023-07-14T16:14:01.395985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:16:21.502671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.659055+0000","last_change":"2023-07-14T16:13:58.277976+0000","last_active":"2023-07-14T16:14:00.659055+0000","last_peered":"2023-07-14T16:14:00.659055+0000","last_clean":"2023-07-14T16:14:00.659055+0000","last_became_active":"2023-07-14T16:13:58.277618+0000","last_became_peered":"2023-07-14T16:13:58.277618+0000","last_unstale":"2023-07-14T16:14:00.659055+0000","last_undegraded":"2023-07-14T16:14:00.659055+0000","last_fullsized":"2023-07-14T16:14:00.659055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:48.596257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.992090+0000","last_change":"2023-07-14T16:13:59.302041+0000","last_active":"2023-07-14T16:14:00.992090+0000","last_peered":"2023-07-14T16:14:00.992090+0000","last_clean":"2023-07-14T16:14:00.992090+0000","last_became_active":"2023-07-14T16:13:59.301620+0000","last_became_peered":"2023-07-14T16:13:59.301620+0000","last_unstale":"2023-07-14T16:14:00.992090+0000","last_undegraded":"2023-07-14T16:14:00.992090+0000","last_fullsized":"2023-07-14T16:14:00.992090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:29:12.896095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290670+0000","last_change":"2023-07-14T16:13:59.304222+0000","last_active":"2023-07-14T16:14:00.290670+0000","last_peered":"2023-07-14T16:14:00.290670+0000","last_clean":"2023-07-14T16:14:00.290670+0000","last_became_active":"2023-07-14T16:13:59.303404+0000","last_became_peered":"2023-07-14T16:13:59.303404+0000","last_unstale":"2023-07-14T16:14:00.290670+0000","last_undegraded":"2023-07-14T16:14:00.290670+0000","last_fullsized":"2023-07-14T16:14:00.290670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:43:48.365055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.604893+0000","last_change":"2023-07-14T16:13:59.296433+0000","last_active":"2023-07-14T16:14:00.604893+0000","last_peered":"2023-07-14T16:14:00.604893+0000","last_clean":"2023-07-14T16:14:00.604893+0000","last_became_active":"2023-07-14T16:13:59.295604+0000","last_became_peered":"2023-07-14T16:13:59.295604+0000","last_unstale":"2023-07-14T16:14:00.604893+0000","last_undegraded":"2023-07-14T16:14:00.604893+0000","last_fullsized":"2023-07-14T16:14:00.604893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:00.910666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.604768+0000","last_change":"2023-07-14T16:13:59.294198+0000","last_active":"2023-07-14T16:14:00.604768+0000","last_peered":"2023-07-14T16:14:00.604768+0000","last_clean":"2023-07-14T16:14:00.604768+0000","last_became_active":"2023-07-14T16:13:59.293891+0000","last_became_peered":"2023-07-14T16:13:59.293891+0000","last_unstale":"2023-07-14T16:14:00.604768+0000","last_undegraded":"2023-07-14T16:14:00.604768+0000","last_fullsized":"2023-07-14T16:14:00.604768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:37:04.786333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:01.174107+0000","last_change":"2023-07-14T16:13:59.298724+0000","last_active":"2023-07-14T16:14:01.174107+0000","last_peered":"2023-07-14T16:14:01.174107+0000","last_clean":"2023-07-14T16:14:01.174107+0000","last_became_active":"2023-07-14T16:13:59.298318+0000","last_became_peered":"2023-07-14T16:13:59.298318+0000","last_unstale":"2023-07-14T16:14:01.174107+0000","last_undegraded":"2023-07-14T16:14:01.174107+0000","last_fullsized":"2023-07-14T16:14:01.174107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:25:27.131220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658791+0000","last_change":"2023-07-14T16:13:59.294652+0000","last_active":"2023-07-14T16:14:00.658791+0000","last_peered":"2023-07-14T16:14:00.658791+0000","last_clean":"2023-07-14T16:14:00.658791+0000","last_became_active":"2023-07-14T16:13:59.294278+0000","last_became_peered":"2023-07-14T16:13:59.294278+0000","last_unstale":"2023-07-14T16:14:00.658791+0000","last_undegraded":"2023-07-14T16:14:00.658791+0000","last_fullsized":"2023-07-14T16:14:00.658791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:43:38.225606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290756+0000","last_change":"2023-07-14T16:13:59.299228+0000","last_active":"2023-07-14T16:14:00.290756+0000","last_peered":"2023-07-14T16:14:00.290756+0000","last_clean":"2023-07-14T16:14:00.290756+0000","last_became_active":"2023-07-14T16:13:59.297982+0000","last_became_peered":"2023-07-14T16:13:59.297982+0000","last_unstale":"2023-07-14T16:14:00.290756+0000","last_undegraded":"2023-07-14T16:14:00.290756+0000","last_fullsized":"2023-07-14T16:14:00.290756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:30:52.523034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.605179+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:14:00.605179+0000","last_peered":"2023-07-14T16:14:00.605179+0000","last_clean":"2023-07-14T16:14:00.605179+0000","last_became_active":"2023-07-14T16:13:59.299181+0000","last_became_peered":"2023-07-14T16:13:59.299181+0000","last_unstale":"2023-07-14T16:14:00.605179+0000","last_undegraded":"2023-07-14T16:14:00.605179+0000","last_fullsized":"2023-07-14T16:14:00.605179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:12:50.267320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290291+0000","last_change":"2023-07-14T16:13:59.303282+0000","last_active":"2023-07-14T16:14:00.290291+0000","last_peered":"2023-07-14T16:14:00.290291+0000","last_clean":"2023-07-14T16:14:00.290291+0000","last_became_active":"2023-07-14T16:13:59.302922+0000","last_became_peered":"2023-07-14T16:13:59.302922+0000","last_unstale":"2023-07-14T16:14:00.290291+0000","last_undegraded":"2023-07-14T16:14:00.290291+0000","last_fullsized":"2023-07-14T16:14:00.290291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:06.464416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.492756+0000","last_change":"2023-07-14T16:13:59.296389+0000","last_active":"2023-07-14T16:14:00.492756+0000","last_peered":"2023-07-14T16:14:00.492756+0000","last_clean":"2023-07-14T16:14:00.492756+0000","last_became_active":"2023-07-14T16:13:59.296080+0000","last_became_peered":"2023-07-14T16:13:59.296080+0000","last_unstale":"2023-07-14T16:14:00.492756+0000","last_undegraded":"2023-07-14T16:14:00.492756+0000","last_fullsized":"2023-07-14T16:14:00.492756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:26:27.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290083+0000","last_change":"2023-07-14T16:13:59.297774+0000","last_active":"2023-07-14T16:14:00.290083+0000","last_peered":"2023-07-14T16:14:00.290083+0000","last_clean":"2023-07-14T16:14:00.290083+0000","last_became_active":"2023-07-14T16:13:59.297463+0000","last_became_peered":"2023-07-14T16:13:59.297463+0000","last_unstale":"2023-07-14T16:14:00.290083+0000","last_undegraded":"2023-07-14T16:14:00.290083+0000","last_fullsized":"2023-07-14T16:14:00.290083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:32:16.595689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.286560+0000","last_change":"2023-07-14T16:13:59.306795+0000","last_active":"2023-07-14T16:14:00.286560+0000","last_peered":"2023-07-14T16:14:00.286560+0000","last_clean":"2023-07-14T16:14:00.286560+0000","last_became_active":"2023-07-14T16:13:59.306457+0000","last_became_peered":"2023-07-14T16:13:59.306457+0000","last_unstale":"2023-07-14T16:14:00.286560+0000","last_undegraded":"2023-07-14T16:14:00.286560+0000","last_fullsized":"2023-07-14T16:14:00.286560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:37:11.860651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.658903+0000","last_change":"2023-07-14T16:13:59.305338+0000","last_active":"2023-07-14T16:14:00.658903+0000","last_peered":"2023-07-14T16:14:00.658903+0000","last_clean":"2023-07-14T16:14:00.658903+0000","last_became_active":"2023-07-14T16:13:59.305047+0000","last_became_peered":"2023-07-14T16:13:59.305047+0000","last_unstale":"2023-07-14T16:14:00.658903+0000","last_undegraded":"2023-07-14T16:14:00.658903+0000","last_fullsized":"2023-07-14T16:14:00.658903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:18:53.732011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-07-14T16:14:00.290522+0000","last_change":"2023-07-14T16:13:59.299005+0000","last_active":"2023-07-14T16:14:00.290522+0000","last_peered":"2023-07-14T16:14:00.290522+0000","last_clean":"2023-07-14T16:14:00.290522+0000","last_became_active":"2023-07-14T16:13:59.298705+0000","last_became_peered":"2023-07-14T16:13:59.298705+0000","last_unstale":"2023-07-14T16:14:00.290522+0000","last_undegraded":"2023-07-14T16:14:00.290522+0000","last_fullsized":"2023-07-14T16:14:00.290522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:44:45.399081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247144,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5078,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":120,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":32325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":441942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.88400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84199999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.94999999999999996}]},{"osd":6,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.98299999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":9,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.016},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.1830000000000001}]},{"osd":10,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.081}]}]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":1033967,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.38}]},{"osd":2,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.501}]},{"osd":3,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.748},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.61699999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.65600000000000003}]},{"osd":9,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.71099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":442178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.746},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.94199999999999995}]},{"osd":2,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.68000000000000005}]},{"osd":3,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.879}]},{"osd":7,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.90200000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.85699999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.002}]}]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":442098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.745,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":6.3410000000000002,"5min":6.3410000000000002,"15min":6.3410000000000002},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93600000000000005,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.53800000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.92000000000000004,"5min":1.216,"15min":1.216},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66000000000000003,"5min":1.081,"15min":1.081},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.747,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0720000000000001,"5min":1.175,"15min":1.175},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.48899999999999999}]},{"osd":4,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":1.149,"15min":1.149},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.996,"5min":1.218,"15min":1.218},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83099999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":1.21,"15min":1.21},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1160000000000001,"5min":1.29,"15min":1.29},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0309999999999999,"5min":1.256,"15min":1.256},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.92100000000000004,"15min":0.88700000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.3209999999999997,"5min":6.3209999999999997,"15min":6.3209999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.88300000000000001,"15min":0.89500000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.046,"5min":1.373,"15min":1.373},"last":0.77500000000000002}]},{"osd":9,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.621}]},{"osd":10,"last update":"Fri Jul 14 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.751}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":1033937,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.181,"15min":1.181},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.52000000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.88600000000000001,"5min":1.008,"15min":1.008},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.628}]},{"osd":3,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.36799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77600000000000002,"15min":0.77400000000000002},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.111,"5min":1.137,"15min":1.137},"last":0.64200000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.84899999999999998,"15min":0.83699999999999997},"min":{"1min":0.61199999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.876}]},{"osd":9,"last update":"Fri Jul 14 16:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.84799999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":441450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.46000000000000002},"min":{"1min":0.34000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.69899999999999995,"15min":0.68600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.69599999999999995,"15min":0.67500000000000004},"min":{"1min":0.51000000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0640000000000001,"5min":1.111,"15min":1.111},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.72099999999999997},"min":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76200000000000001,"15min":0.73399999999999999},"min":{"1min":0.59799999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.0940000000000001,"5min":1.105,"15min":1.105},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59299999999999997,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.497,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.89300000000000002}]},{"osd":5,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79300000000000004,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.82599999999999996,"15min":0.81200000000000006},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.80000000000000004,"15min":0.77800000000000002},"min":{"1min":0.65500000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.748},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.82299999999999995,"15min":0.79700000000000004},"min":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.84299999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.94499999999999995,"15min":0.94199999999999995},"min":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.873},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.872,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92000000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.77600000000000002}]},{"osd":10,"last update":"Fri Jul 14 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.98499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":442924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97799999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.57799999999999996,"15min":0.52800000000000002},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Fri Jul 14 16:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.62,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.627,"15min":0.60899999999999999},"min":{"1min":0.51600000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.81000000000000005}]},{"osd":3,"last update":"Fri Jul 14 16:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.63,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.61399999999999999,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88800000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.53600000000000003,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95899999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.66200000000000003,"15min":0.64800000000000002},"min":{"1min":0.49199999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.50700000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.68899999999999995,"15min":0.66100000000000003},"min":{"1min":0.57899999999999996,"5min":0.37,"15min":0.37},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.89300000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84699999999999998,"15min":0.84099999999999997},"min":{"1min":0.69699999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.137,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.70299999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.878}]},{"osd":9,"last update":"Fri Jul 14 16:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.92700000000000005}]},{"osd":10,"last update":"Fri Jul 14 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.99299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":442838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69899999999999995,"15min":0.626},"min":{"1min":0.60099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.71199999999999997,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.871}]},{"osd":3,"last update":"Fri Jul 14 16:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74399999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.505}]},{"osd":4,"last update":"Fri Jul 14 16:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Jul 14 16:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.49099999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.04,"5min":1.127,"15min":1.127},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.748,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83799999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.91700000000000004,"15min":0.92500000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.1279999999999999,"5min":1.208,"15min":1.208},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.86299999999999999,"15min":0.86699999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0820000000000001,"5min":1.099,"15min":1.099},"last":0.71399999999999997}]},{"osd":9,"last update":"Fri Jul 14 16:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.64600000000000002}]},{"osd":10,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.80900000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":461591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.54900000000000004},"min":{"1min":0.379,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.56299999999999994},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90500000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0549999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66100000000000003,"15min":0.64200000000000002},"min":{"1min":0.48099999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.624},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64500000000000002,"15min":0.62},"min":{"1min":0.51500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93300000000000005,"5min":0.999,"15min":0.999},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Jul 14 16:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.56399999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94699999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.99099999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.67000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.022,"5min":1.117,"15min":1.117},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.016},"last":0.52800000000000002}]},{"osd":9,"last update":"Fri Jul 14 16:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.754}]},{"osd":10,"last update":"Fri Jul 14 16:13:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":441183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.41899999999999998,"15min":0.39700000000000002},"min":{"1min":0.35699999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.44600000000000001,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.503,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64100000000000001,"15min":0.61699999999999999},"min":{"1min":0.45700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0269999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.505}]},{"osd":2,"last update":"Fri Jul 14 16:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.70099999999999996,"15min":0.67500000000000004},"min":{"1min":0.52400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72999999999999998,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.448,"15min":3.448},"last":0.623}]},{"osd":4,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.66000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":3.359,"15min":3.359},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71099999999999997,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96499999999999997,"5min":1.222,"15min":1.222},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Jul 14 16:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.64300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Fri Jul 14 16:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.042,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0329999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.81100000000000005,"15min":0.78700000000000003},"min":{"1min":0.746,"5min":0.499,"15min":0.499},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.754},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.624,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.60399999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.876,"15min":0.88500000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1259999999999999,"5min":1.546,"15min":1.546},"last":0.62},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91800000000000004,"15min":0.92100000000000004},"min":{"1min":0.61899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.163,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.58099999999999996}]},{"osd":10,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":441956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.95599999999999996,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46800000000000003,"15min":0.432},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.501,"15min":0.45700000000000002},"min":{"1min":0.35499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68000000000000005,"15min":0.64200000000000002},"min":{"1min":0.505,"5min":0.439,"15min":0.439},"max":{"1min":1.048,"5min":3.698,"15min":3.698},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.70199999999999996,"15min":0.67800000000000005},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1160000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.54900000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.995,"5min":1.03,"15min":1.03},"last":1.014}]},{"osd":5,"last update":"Fri Jul 14 16:13:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.52400000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2949999999999999,"5min":1.506,"15min":1.506},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.53800000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.89700000000000002}]},{"osd":6,"last update":"Fri Jul 14 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.72499999999999998,"15min":0.71099999999999997},"min":{"1min":0.55500000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0129999999999999,"5min":1.321,"15min":1.321},"last":0.879},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.56699999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Jul 14 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56699999999999995}]},{"osd":8,"last update":"Fri Jul 14 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.85099999999999998,"15min":0.83999999999999997},"min":{"1min":0.71699999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1040000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.76900000000000002,"15min":0.754},"min":{"1min":0.66400000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.93899999999999995}]},{"osd":9,"last update":"Fri Jul 14 16:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.99299999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.81399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":1031579,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86499999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79000000000000004,"5min":1.131,"15min":1.131},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48599999999999999,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70299999999999996,"15min":0.65900000000000003},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93400000000000005,"5min":7.056,"15min":7.056},"last":0.62}]},{"osd":4,"last update":"Fri Jul 14 16:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70999999999999996,"15min":0.68999999999999995},"min":{"1min":0.58599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.997,"5min":1.052,"15min":1.052},"last":0.503},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.53500000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Jul 14 16:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.2569999999999999,"5min":7.109,"15min":7.109},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.754,"15min":0.746},"min":{"1min":0.629,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.79500000000000004,"15min":0.77900000000000003},"min":{"1min":0.65700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.503,"5min":0.437,"15min":0.437},"max":{"1min":1.0489999999999999,"5min":1.286,"15min":1.286},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78900000000000003,"15min":0.79300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.55500000000000005}]},{"osd":9,"last update":"Fri Jul 14 16:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.68100000000000005}]},{"osd":10,"last update":"Fri Jul 14 16:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T16:14:08.522 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-14T16:14:08.523 INFO:tasks.cephadm:Setting up client nodes... 2023-07-14T16:14:08.524 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-14T16:14:08.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:08 smithi062 ceph-mon[95543]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.8 KiB/s wr, 16 op/s 2023-07-14T16:14:08.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:08 smithi081 ceph-mon[127795]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.8 KiB/s wr, 16 op/s 2023-07-14T16:14:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:08 smithi164 ceph-mon[97427]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.8 KiB/s wr, 16 op/s 2023-07-14T16:14:09.276 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:09.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:09 smithi062 ceph-mon[95543]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:09.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:09 smithi062 ceph-mon[95543]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:09.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:09 smithi081 ceph-mon[127795]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:09.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:09 smithi081 ceph-mon[127795]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:10.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:09 smithi164 ceph-mon[97427]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:10.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:09 smithi164 ceph-mon[97427]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:10.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 -- 172.21.15.62:0/901323074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 msgr2=0x7fbc080fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:10.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/901323074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 0x7fbc080fffc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009990 tx=0x7fbbf8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:10.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 -- 172.21.15.62:0/901323074 shutdown_connections 2023-07-14T16:14:10.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/901323074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc08101740 0x7fbc08108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:10.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/901323074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 0x7fbc080fffc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:10.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/901323074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc08100db0 0x7fbc08101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:10.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 -- 172.21.15.62:0/901323074 >> 172.21.15.62:0/901323074 conn(0x7fbc080fb200 msgr2=0x7fbc080fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:10.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 -- 172.21.15.62:0/901323074 shutdown_connections 2023-07-14T16:14:10.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.894+0000 7fbc0dcff700 1 -- 172.21.15.62:0/901323074 wait complete. 2023-07-14T16:14:10.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.895+0000 7fbc0dcff700 1 Processor -- start 2023-07-14T16:14:10.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.895+0000 7fbc0dcff700 1 -- start start 2023-07-14T16:14:10.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.895+0000 7fbc0dcff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 0x7fbc080742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:10.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.895+0000 7fbc0dcff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc08100db0 0x7fbc08072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:10.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.895+0000 7fbc0dcff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc08101740 0x7fbc08072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:10.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.895+0000 7fbc0dcff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080735c0 con 0x7fbc080ffbb0 2023-07-14T16:14:10.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.895+0000 7fbc0dcff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc08073730 con 0x7fbc08101740 2023-07-14T16:14:10.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.895+0000 7fbc0dcff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0819f8f0 con 0x7fbc08100db0 2023-07-14T16:14:10.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 0x7fbc080742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:10.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 0x7fbc080742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46262/0 (socket says 172.21.15.62:46262) 2023-07-14T16:14:10.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 -- 172.21.15.62:0/473256300 learned_addr learned my addr 172.21.15.62:0/473256300 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:10.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc06ffd700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc08100db0 0x7fbc08072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:10.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc07fff700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc08101740 0x7fbc08072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:10.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 -- 172.21.15.62:0/473256300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc08100db0 msgr2=0x7fbc08072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:10.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc08100db0 0x7fbc08072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:10.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 -- 172.21.15.62:0/473256300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc08101740 msgr2=0x7fbc08072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:10.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc08101740 0x7fbc08072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:10.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 -- 172.21.15.62:0/473256300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf80095f0 con 0x7fbc080ffbb0 2023-07-14T16:14:10.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc06ffd700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc08100db0 0x7fbc08072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:10.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:10 smithi062 ceph-mon[95543]: pgmap v311: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.1 KiB/s wr, 17 op/s 2023-07-14T16:14:10.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.896+0000 7fbc077fe700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 0x7fbc080742b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80095c0 tx=0x7fbbf8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:10.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.897+0000 7fbc04ff9700 1 -- 172.21.15.62:0/473256300 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf801d070 con 0x7fbc080ffbb0 2023-07-14T16:14:10.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.897+0000 7fbc04ff9700 1 -- 172.21.15.62:0/473256300 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbf8004080 con 0x7fbc080ffbb0 2023-07-14T16:14:10.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.897+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0819fb70 con 0x7fbc080ffbb0 2023-07-14T16:14:10.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.897+0000 7fbc04ff9700 1 -- 172.21.15.62:0/473256300 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8022cb0 con 0x7fbc080ffbb0 2023-07-14T16:14:10.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.897+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0819ffe0 con 0x7fbc080ffbb0 2023-07-14T16:14:10.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.898+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc08107210 con 0x7fbc080ffbb0 2023-07-14T16:14:10.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.900+0000 7fbc04ff9700 1 -- 172.21.15.62:0/473256300 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbbf8003bf0 con 0x7fbc080ffbb0 2023-07-14T16:14:10.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.901+0000 7fbc04ff9700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbbf00a1260 0x7fbbf00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:10.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.901+0000 7fbc04ff9700 1 -- 172.21.15.62:0/473256300 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbbf80ce080 con 0x7fbc080ffbb0 2023-07-14T16:14:10.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.902+0000 7fbc06ffd700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbbf00a1260 0x7fbbf00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:10.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.902+0000 7fbc06ffd700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbbf00a1260 0x7fbbf00a3710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc005950 tx=0x7fbbfc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:10.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:10.906+0000 7fbc04ff9700 1 -- 172.21.15.62:0/473256300 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbbf80977b0 con 0x7fbc080ffbb0 2023-07-14T16:14:10.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:10 smithi081 ceph-mon[127795]: pgmap v311: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.1 KiB/s wr, 17 op/s 2023-07-14T16:14:11.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:10 smithi164 ceph-mon[97427]: pgmap v311: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.1 KiB/s wr, 17 op/s 2023-07-14T16:14:11.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.198+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fbc0802cce0 con 0x7fbc080ffbb0 2023-07-14T16:14:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.203+0000 7fbc04ff9700 1 -- 172.21.15.62:0/473256300 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fbbf80041f0 con 0x7fbc080ffbb0 2023-07-14T16:14:11.182 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2023-07-14T16:14:11.182 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBTdLFkJzvzCxAA4ed09iJ6qbHwtyGRRghI7A== 2023-07-14T16:14:11.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbbf00a1260 msgr2=0x7fbbf00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:11.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbbf00a1260 0x7fbbf00a3710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc005950 tx=0x7fbbfc0058e0 comp rx=0 tx=0).stop 2023-07-14T16:14:11.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 msgr2=0x7fbc080742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:11.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 0x7fbc080742b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80095c0 tx=0x7fbbf8005ef0 comp rx=0 tx=0).stop 2023-07-14T16:14:11.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 shutdown_connections 2023-07-14T16:14:11.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbbf00a1260 0x7fbbf00a3710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:11.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc08101740 0x7fbc08072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:11.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc080ffbb0 0x7fbc080742b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:11.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 --2- 172.21.15.62:0/473256300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc08100db0 0x7fbc08072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:11.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 >> 172.21.15.62:0/473256300 conn(0x7fbc080fb200 msgr2=0x7fbc08104350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:11.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 shutdown_connections 2023-07-14T16:14:11.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:11.206+0000 7fbc0dcff700 1 -- 172.21.15.62:0/473256300 wait complete. 2023-07-14T16:14:11.656 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:14:11.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-14T16:14:11.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-14T16:14:11.766 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-14T16:14:11.767 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-14T16:14:11.767 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph mgr dump --format=json 2023-07-14T16:14:11.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:11 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/473256300' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T16:14:11.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:11 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/473256300' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T16:14:11.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:11 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/473256300' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T16:14:11.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:11 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/473256300' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T16:14:12.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:11 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/473256300' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T16:14:12.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:11 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/473256300' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T16:14:12.536 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:12.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:12 smithi062 ceph-mon[95543]: pgmap v312: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2023-07-14T16:14:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:12 smithi081 ceph-mon[127795]: pgmap v312: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2023-07-14T16:14:13.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:12 smithi164 ceph-mon[97427]: pgmap v312: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2023-07-14T16:14:14.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2795849136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a28104ea0 msgr2=0x7f3a2810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:14.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2795849136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a28104ea0 0x7f3a2810ca20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000b0d0 tx=0x7f3a2000b3e0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2795849136 shutdown_connections 2023-07-14T16:14:14.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2795849136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a28104ea0 0x7f3a2810ca20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2795849136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a280ffe00 0x7f3a281001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2795849136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a281007a0 0x7f3a28104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2795849136 >> 172.21.15.62:0/2795849136 conn(0x7f3a280fb660 msgr2=0x7f3a280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:14.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2795849136 shutdown_connections 2023-07-14T16:14:14.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.060+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2795849136 wait complete. 2023-07-14T16:14:14.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.061+0000 7f3a2ed56700 1 Processor -- start 2023-07-14T16:14:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.061+0000 7f3a2ed56700 1 -- start start 2023-07-14T16:14:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2ed56700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a280ffe00 0x7f3a28197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:14.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2ed56700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a281007a0 0x7f3a281983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2ed56700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a28104ea0 0x7f3a2819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2ed56700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2819ce20 con 0x7f3a280ffe00 2023-07-14T16:14:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2ed56700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2819cf90 con 0x7f3a28104ea0 2023-07-14T16:14:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2ed56700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2819d150 con 0x7f3a281007a0 2023-07-14T16:14:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2caf2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a280ffe00 0x7f3a28197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2caf2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a280ffe00 0x7f3a28197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46918/0 (socket says 172.21.15.62:46918) 2023-07-14T16:14:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2caf2700 1 -- 172.21.15.62:0/2515009795 learned_addr learned my addr 172.21.15.62:0/2515009795 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:14.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2d2f3700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a28104ea0 0x7f3a2819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a27fff700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a281007a0 0x7f3a281983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:14.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2caf2700 1 -- 172.21.15.62:0/2515009795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a281007a0 msgr2=0x7f3a281983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2caf2700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a281007a0 0x7f3a281983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2caf2700 1 -- 172.21.15.62:0/2515009795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a28104ea0 msgr2=0x7f3a2819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2caf2700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a28104ea0 0x7f3a2819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a2caf2700 1 -- 172.21.15.62:0/2515009795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20009d00 con 0x7f3a280ffe00 2023-07-14T16:14:14.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.062+0000 7f3a27fff700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a281007a0 0x7f3a281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.063+0000 7f3a2caf2700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a280ffe00 0x7f3a28197e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00c9b0 tx=0x7f3a1c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.063+0000 7f3a25ffb700 1 -- 172.21.15.62:0/2515009795 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c007b00 con 0x7f3a280ffe00 2023-07-14T16:14:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.063+0000 7f3a25ffb700 1 -- 172.21.15.62:0/2515009795 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a1c007c60 con 0x7f3a280ffe00 2023-07-14T16:14:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.063+0000 7f3a25ffb700 1 -- 172.21.15.62:0/2515009795 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c018660 con 0x7f3a280ffe00 2023-07-14T16:14:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.063+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2819d430 con 0x7f3a280ffe00 2023-07-14T16:14:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.063+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a281a4c90 con 0x7f3a280ffe00 2023-07-14T16:14:14.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.064+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2804e210 con 0x7f3a280ffe00 2023-07-14T16:14:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.066+0000 7f3a25ffb700 1 -- 172.21.15.62:0/2515009795 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3a1c00f450 con 0x7f3a280ffe00 2023-07-14T16:14:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.068+0000 7f3a25ffb700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3a180a1260 0x7f3a180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.068+0000 7f3a25ffb700 1 -- 172.21.15.62:0/2515009795 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3a1c0cc970 con 0x7f3a280ffe00 2023-07-14T16:14:14.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.068+0000 7f3a27fff700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3a180a1260 0x7f3a180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:14.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.068+0000 7f3a27fff700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3a180a1260 0x7f3a180a3710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a14005e10 tx=0x7f3a14005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:14.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.072+0000 7f3a25ffb700 1 -- 172.21.15.62:0/2515009795 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3a1c096010 con 0x7f3a280ffe00 2023-07-14T16:14:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.337+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3a281a4fe0 con 0x7f3a280ffe00 2023-07-14T16:14:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.342+0000 7f3a25ffb700 1 -- 172.21.15.62:0/2515009795 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260125 (secure 0 0 0) 0x7f3a1c00fe60 con 0x7f3a280ffe00 2023-07-14T16:14:14.322 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:14.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3a180a1260 msgr2=0x7f3a180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:14.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3a180a1260 0x7f3a180a3710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a14005e10 tx=0x7f3a14005d60 comp rx=0 tx=0).stop 2023-07-14T16:14:14.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a280ffe00 msgr2=0x7f3a28197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:14.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a280ffe00 0x7f3a28197e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00c9b0 tx=0x7f3a1c00ccc0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 shutdown_connections 2023-07-14T16:14:14.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3a180a1260 0x7f3a180a3710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a28104ea0 0x7f3a2819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a280ffe00 0x7f3a28197e60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 --2- 172.21.15.62:0/2515009795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3a281007a0 0x7f3a281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:14.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.346+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 >> 172.21.15.62:0/2515009795 conn(0x7f3a280fb660 msgr2=0x7f3a280fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:14.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.347+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 shutdown_connections 2023-07-14T16:14:14.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:14.347+0000 7f3a2ed56700 1 -- 172.21.15.62:0/2515009795 wait complete. 2023-07-14T16:14:14.812 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":4048727837},{"type":"v1","addr":"172.21.15.62:6801","nonce":4048727837}]},"active_addr":"172.21.15.62:6801/4048727837","active_change":"2023-07-14T16:06:19.370143+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.62:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":2265576573}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":955325623}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1924318640}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":2133524705}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":3048859547}]}]} 2023-07-14T16:14:14.818 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-14T16:14:14.819 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-14T16:14:14.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T16:14:14.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:14 smithi062 ceph-mon[95543]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-07-14T16:14:14.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:14 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2515009795' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T16:14:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:14 smithi081 ceph-mon[127795]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-07-14T16:14:14.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:14 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2515009795' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T16:14:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:14 smithi164 ceph-mon[97427]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-07-14T16:14:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:14 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2515009795' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T16:14:15.553 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:16.800 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:16 smithi062 ceph-mon[95543]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2023-07-14T16:14:16.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:16 smithi081 ceph-mon[127795]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2023-07-14T16:14:17.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:16 smithi164 ceph-mon[97427]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2023-07-14T16:14:17.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.076+0000 7f1efc219700 1 -- 172.21.15.62:0/3517703955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef410c3e0 msgr2=0x7f1ef410e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:17.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.076+0000 7f1efc219700 1 --2- 172.21.15.62:0/3517703955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef410c3e0 0x7f1ef410e7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00b0a0 tx=0x7f1eec00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.077+0000 7f1efc219700 1 -- 172.21.15.62:0/3517703955 shutdown_connections 2023-07-14T16:14:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.077+0000 7f1efc219700 1 --2- 172.21.15.62:0/3517703955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef410c3e0 0x7f1ef410e7c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.077+0000 7f1efc219700 1 --2- 172.21.15.62:0/3517703955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef40728b0 0x7f1ef4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.077+0000 7f1efc219700 1 --2- 172.21.15.62:0/3517703955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ef40731c0 0x7f1ef410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.077+0000 7f1efc219700 1 -- 172.21.15.62:0/3517703955 >> 172.21.15.62:0/3517703955 conn(0x7f1ef40fb660 msgr2=0x7f1ef40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.077+0000 7f1efc219700 1 -- 172.21.15.62:0/3517703955 shutdown_connections 2023-07-14T16:14:17.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.077+0000 7f1efc219700 1 -- 172.21.15.62:0/3517703955 wait complete. 2023-07-14T16:14:17.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.078+0000 7f1efc219700 1 Processor -- start 2023-07-14T16:14:17.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.078+0000 7f1efc219700 1 -- start start 2023-07-14T16:14:17.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.078+0000 7f1efc219700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef40728b0 0x7f1ef4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:17.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.078+0000 7f1efc219700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ef40731c0 0x7f1ef4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:17.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.078+0000 7f1efc219700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef410c3e0 0x7f1ef419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:17.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.078+0000 7f1efc219700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef419ce60 con 0x7f1ef40728b0 2023-07-14T16:14:17.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.078+0000 7f1efc219700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef419cfd0 con 0x7f1ef410c3e0 2023-07-14T16:14:17.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.078+0000 7f1efc219700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef419d190 con 0x7f1ef40731c0 2023-07-14T16:14:17.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef40728b0 0x7f1ef4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:17.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef40728b0 0x7f1ef4197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46934/0 (socket says 172.21.15.62:46934) 2023-07-14T16:14:17.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 -- 172.21.15.62:0/2705131653 learned_addr learned my addr 172.21.15.62:0/2705131653 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:17.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef97b4700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ef40731c0 0x7f1ef4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:17.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1efa7b6700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef410c3e0 0x7f1ef419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:17.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 -- 172.21.15.62:0/2705131653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ef40731c0 msgr2=0x7f1ef4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:17.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ef40731c0 0x7f1ef4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 -- 172.21.15.62:0/2705131653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef410c3e0 msgr2=0x7f1ef419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:17.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef410c3e0 0x7f1ef419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 -- 172.21.15.62:0/2705131653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec009d00 con 0x7f1ef40728b0 2023-07-14T16:14:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1efa7b6700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef410c3e0 0x7f1ef419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.079+0000 7f1ef9fb5700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef40728b0 0x7f1ef4197e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000d950 tx=0x7f1ef000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.080+0000 7f1eeaffd700 1 -- 172.21.15.62:0/2705131653 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0009940 con 0x7f1ef40728b0 2023-07-14T16:14:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.080+0000 7f1eeaffd700 1 -- 172.21.15.62:0/2705131653 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ef0010460 con 0x7f1ef40728b0 2023-07-14T16:14:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.080+0000 7f1eeaffd700 1 -- 172.21.15.62:0/2705131653 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000f620 con 0x7f1ef40728b0 2023-07-14T16:14:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.080+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef419d470 con 0x7f1ef40728b0 2023-07-14T16:14:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.080+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef41a4c40 con 0x7f1ef40728b0 2023-07-14T16:14:17.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.081+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef404e210 con 0x7f1ef40728b0 2023-07-14T16:14:17.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.083+0000 7f1eeaffd700 1 -- 172.21.15.62:0/2705131653 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1ef00105d0 con 0x7f1ef40728b0 2023-07-14T16:14:17.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.084+0000 7f1eeaffd700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ee00a1260 0x7f1ee00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:17.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.084+0000 7f1eeaffd700 1 -- 172.21.15.62:0/2705131653 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ef00cc7c0 con 0x7f1ef40728b0 2023-07-14T16:14:17.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.085+0000 7f1ef97b4700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ee00a1260 0x7f1ee00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:17.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.085+0000 7f1ef97b4700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ee00a1260 0x7f1ee00a3710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4199430 tx=0x7f1ee400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:17.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.088+0000 7f1eeaffd700 1 -- 172.21.15.62:0/2705131653 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1ef0095e60 con 0x7f1ef40728b0 2023-07-14T16:14:17.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.294+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ef4199170 con 0x7f1ef40728b0 2023-07-14T16:14:17.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.295+0000 7f1eeaffd700 1 -- 172.21.15.62:0/2705131653 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22068 (secure 0 0 0) 0x7f1ef0099480 con 0x7f1ef40728b0 2023-07-14T16:14:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:17.273 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:00.277438+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:14:17.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ee00a1260 msgr2=0x7f1ee00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:17.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ee00a1260 0x7f1ee00a3710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4199430 tx=0x7f1ee400b410 comp rx=0 tx=0).stop 2023-07-14T16:14:17.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef40728b0 msgr2=0x7f1ef4197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:17.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef40728b0 0x7f1ef4197e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000d950 tx=0x7f1ef000dc60 comp rx=0 tx=0).stop 2023-07-14T16:14:17.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 shutdown_connections 2023-07-14T16:14:17.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ee00a1260 0x7f1ee00a3710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef410c3e0 0x7f1ef419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ef40728b0 0x7f1ef4197e10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 --2- 172.21.15.62:0/2705131653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ef40731c0 0x7f1ef4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:17.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 >> 172.21.15.62:0/2705131653 conn(0x7f1ef40fb660 msgr2=0x7f1ef410e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:17.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 shutdown_connections 2023-07-14T16:14:17.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:17.298+0000 7f1efc219700 1 -- 172.21.15.62:0/2705131653 wait complete. 2023-07-14T16:14:17.830 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-14T16:14:17.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T16:14:17.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:17 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2705131653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:17.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:17 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2705131653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:18.030 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:17 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2705131653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:18.558 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:18.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:18 smithi062 ceph-mon[95543]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2023-07-14T16:14:18.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:18 smithi081 ceph-mon[127795]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2023-07-14T16:14:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:18 smithi164 ceph-mon[97427]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2023-07-14T16:14:19.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-07-14T16:14:19.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T16:14:19.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-14T16:14:19.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-07-14T16:14:19.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-14T16:14:19.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T16:14:19.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-07-14T16:14:19.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-07-14T16:14:19.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-07-14T16:14:19.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T16:14:19.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-14T16:14:19.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-07-14T16:14:19.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-14T16:14:19.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T16:14:19.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-07-14T16:14:19.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-07-14T16:14:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2023-07-14T16:14:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T16:14:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-14T16:14:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-07-14T16:14:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-14T16:14:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T16:14:20.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-07-14T16:14:20.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-07-14T16:14:20.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 -- 172.21.15.62:0/1876970123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed01007a0 msgr2=0x7f4ed0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:20.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 --2- 172.21.15.62:0/1876970123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed01007a0 0x7f4ed0104770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0099c0 tx=0x7f4ebc009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 -- 172.21.15.62:0/1876970123 shutdown_connections 2023-07-14T16:14:20.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 --2- 172.21.15.62:0/1876970123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ed0104ea0 0x7f4ed010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 --2- 172.21.15.62:0/1876970123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed00ffe00 0x7f4ed01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 --2- 172.21.15.62:0/1876970123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed01007a0 0x7f4ed0104770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 -- 172.21.15.62:0/1876970123 >> 172.21.15.62:0/1876970123 conn(0x7f4ed00fb660 msgr2=0x7f4ed00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:20.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 -- 172.21.15.62:0/1876970123 shutdown_connections 2023-07-14T16:14:20.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.226+0000 7f4ed7085700 1 -- 172.21.15.62:0/1876970123 wait complete. 2023-07-14T16:14:20.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.227+0000 7f4ed7085700 1 Processor -- start 2023-07-14T16:14:20.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.227+0000 7f4ed7085700 1 -- start start 2023-07-14T16:14:20.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.227+0000 7f4ed7085700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ed00ffe00 0x7f4ed019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:20.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.227+0000 7f4ed7085700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed01007a0 0x7f4ed019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:20.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.227+0000 7f4ed7085700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed0104ea0 0x7f4ed0195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:20.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.227+0000 7f4ed7085700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0196090 con 0x7f4ed0104ea0 2023-07-14T16:14:20.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.227+0000 7f4ed7085700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0196200 con 0x7f4ed01007a0 2023-07-14T16:14:20.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.227+0000 7f4ed7085700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed01963c0 con 0x7f4ed00ffe00 2023-07-14T16:14:20.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed5622700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed0104ea0 0x7f4ed0195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:20.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed4e21700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ed00ffe00 0x7f4ed019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:20.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed4e21700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ed00ffe00 0x7f4ed019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54910/0 (socket says 172.21.15.62:54910) 2023-07-14T16:14:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed4e21700 1 -- 172.21.15.62:0/2637555748 learned_addr learned my addr 172.21.15.62:0/2637555748 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed5622700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed0104ea0 0x7f4ed0195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46958/0 (socket says 172.21.15.62:46958) 2023-07-14T16:14:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ecffff700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed01007a0 0x7f4ed019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed5622700 1 -- 172.21.15.62:0/2637555748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ed00ffe00 msgr2=0x7f4ed019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed5622700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ed00ffe00 0x7f4ed019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed5622700 1 -- 172.21.15.62:0/2637555748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed01007a0 msgr2=0x7f4ed019bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed5622700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed01007a0 0x7f4ed019bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed5622700 1 -- 172.21.15.62:0/2637555748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0095f0 con 0x7f4ed0104ea0 2023-07-14T16:14:20.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed4e21700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ed00ffe00 0x7f4ed019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:14:20.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.228+0000 7f4ed5622700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed0104ea0 0x7f4ed0195a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800ab40 tx=0x7f4ec800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.229+0000 7f4ecdffb700 1 -- 172.21.15.62:0/2637555748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800f070 con 0x7f4ed0104ea0 2023-07-14T16:14:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.229+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed01966a0 con 0x7f4ed0104ea0 2023-07-14T16:14:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.229+0000 7f4ecdffb700 1 -- 172.21.15.62:0/2637555748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec8007a70 con 0x7f4ed0104ea0 2023-07-14T16:14:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.229+0000 7f4ecdffb700 1 -- 172.21.15.62:0/2637555748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8005280 con 0x7f4ed0104ea0 2023-07-14T16:14:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.229+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed01a2990 con 0x7f4ed0104ea0 2023-07-14T16:14:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.230+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed004e210 con 0x7f4ed0104ea0 2023-07-14T16:14:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.232+0000 7f4ecdffb700 1 -- 172.21.15.62:0/2637555748 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4ec8004ad0 con 0x7f4ed0104ea0 2023-07-14T16:14:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.233+0000 7f4ecdffb700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4ec00a11e0 0x7f4ec00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.234+0000 7f4ecdffb700 1 -- 172.21.15.62:0/2637555748 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ec80cbe70 con 0x7f4ed0104ea0 2023-07-14T16:14:20.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.234+0000 7f4ed4e21700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4ec00a11e0 0x7f4ec00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:20.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.234+0000 7f4ed4e21700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4ec00a11e0 0x7f4ec00a3690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005950 tx=0x7f4ec400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:20.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.237+0000 7f4ecdffb700 1 -- 172.21.15.62:0/2637555748 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4ec80954a0 con 0x7f4ed0104ea0 2023-07-14T16:14:20.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.445+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ed01a2c20 con 0x7f4ed0104ea0 2023-07-14T16:14:20.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.447+0000 7f4ecdffb700 1 -- 172.21.15.62:0/2637555748 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f4ec8098ac0 con 0x7f4ed0104ea0 2023-07-14T16:14:20.424 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:20.425 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":87,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:19.649121+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:14:20.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4ec00a11e0 msgr2=0x7f4ec00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:20.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4ec00a11e0 0x7f4ec00a3690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005950 tx=0x7f4ec400b500 comp rx=0 tx=0).stop 2023-07-14T16:14:20.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed0104ea0 msgr2=0x7f4ed0195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:20.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed0104ea0 0x7f4ed0195a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800ab40 tx=0x7f4ec800ae50 comp rx=0 tx=0).stop 2023-07-14T16:14:20.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 shutdown_connections 2023-07-14T16:14:20.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ed0104ea0 0x7f4ed0195a20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ed00ffe00 0x7f4ed019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4ec00a11e0 0x7f4ec00a3690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 --2- 172.21.15.62:0/2637555748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ed01007a0 0x7f4ed019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:20.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 >> 172.21.15.62:0/2637555748 conn(0x7f4ed00fb660 msgr2=0x7f4ed00fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:20.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 shutdown_connections 2023-07-14T16:14:20.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:20.449+0000 7f4ed7085700 1 -- 172.21.15.62:0/2637555748 wait complete. 2023-07-14T16:14:20.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2023-07-14T16:14:20.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-07-14T16:14:20.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T16:14:20.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-14T16:14:20.836 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-07-14T16:14:20.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-14T16:14:20.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T16:14:20.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-07-14T16:14:20.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-07-14T16:14:20.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T16:14:20.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:20 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2637555748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:20.839 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-14T16:14:20.839 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-14T16:14:20.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-14T16:14:20.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-14T16:14:20.841 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-14T16:14:20.841 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-14T16:14:20.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-14T16:14:20.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-14T16:14:20.843 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-14T16:14:20.843 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-14T16:14:20.844 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-14T16:14:20.844 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-14T16:14:20.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2023-07-14T16:14:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-07-14T16:14:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T16:14:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-14T16:14:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-07-14T16:14:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-14T16:14:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T16:14:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-07-14T16:14:20.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-07-14T16:14:20.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T16:14:20.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:20 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2637555748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:21.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2023-07-14T16:14:21.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2023-07-14T16:14:21.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T16:14:21.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-14T16:14:21.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-07-14T16:14:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-14T16:14:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T16:14:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-07-14T16:14:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-07-14T16:14:21.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T16:14:21.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:20 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2637555748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:14:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:21 smithi164 ceph-mon[97427]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T16:14:22.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:21 smithi062 ceph-mon[95543]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T16:14:22.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:21 smithi081 ceph-mon[127795]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T16:14:22.789 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:22.865 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:22.968 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:22.969 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:22.972 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:22 smithi164 ceph-mon[97427]: pgmap v319: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2023-07-14T16:14:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:23.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:22 smithi062 ceph-mon[95543]: pgmap v319: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2023-07-14T16:14:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:22 smithi081 ceph-mon[127795]: pgmap v319: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2023-07-14T16:14:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:24.772 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:24 smithi062 ceph-mon[95543]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 383 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-14T16:14:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:24 smithi164 ceph-mon[97427]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 383 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-14T16:14:25.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:24 smithi081 ceph-mon[127795]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 383 B/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2023-07-14T16:14:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:26 smithi164 ceph-mon[97427]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 383 B/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:27.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:26 smithi062 ceph-mon[95543]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 383 B/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:27.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:26 smithi081 ceph-mon[127795]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 383 B/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:28.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2616294055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 msgr2=0x7f9cd810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:28.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 --2- 172.21.15.62:0/2616294055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 0x7f9cd810ca20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000b0d0 tx=0x7f9cd000b3e0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2616294055 shutdown_connections 2023-07-14T16:14:28.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 --2- 172.21.15.62:0/2616294055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 0x7f9cd810ca20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 --2- 172.21.15.62:0/2616294055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cd80ffe00 0x7f9cd81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 --2- 172.21.15.62:0/2616294055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd81007a0 0x7f9cd8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2616294055 >> 172.21.15.62:0/2616294055 conn(0x7f9cd80fb660 msgr2=0x7f9cd80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:28.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2616294055 shutdown_connections 2023-07-14T16:14:28.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.818+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2616294055 wait complete. 2023-07-14T16:14:28.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.819+0000 7f9ce00e6700 1 Processor -- start 2023-07-14T16:14:28.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.819+0000 7f9ce00e6700 1 -- start start 2023-07-14T16:14:28.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9ce00e6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd80ffe00 0x7f9cd8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:28.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9ce00e6700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cd81007a0 0x7f9cd8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:28.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9ce00e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 0x7f9cd819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:28.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9ce00e6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd819cdf0 con 0x7f9cd8104ea0 2023-07-14T16:14:28.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9ce00e6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd819cf60 con 0x7f9cd80ffe00 2023-07-14T16:14:28.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9ce00e6700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd819d120 con 0x7f9cd81007a0 2023-07-14T16:14:28.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cde683700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 0x7f9cd819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:28.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cde683700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 0x7f9cd819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46042/0 (socket says 172.21.15.62:46042) 2023-07-14T16:14:28.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cde683700 1 -- 172.21.15.62:0/2401765084 learned_addr learned my addr 172.21.15.62:0/2401765084 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:28.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cdde82700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd80ffe00 0x7f9cd8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:28.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cdd681700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cd81007a0 0x7f9cd8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:28.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cde683700 1 -- 172.21.15.62:0/2401765084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cd81007a0 msgr2=0x7f9cd8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:28.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cde683700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cd81007a0 0x7f9cd8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cde683700 1 -- 172.21.15.62:0/2401765084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd80ffe00 msgr2=0x7f9cd8197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:28.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cde683700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd80ffe00 0x7f9cd8197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cde683700 1 -- 172.21.15.62:0/2401765084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd0009d00 con 0x7f9cd8104ea0 2023-07-14T16:14:28.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.820+0000 7f9cdde82700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd80ffe00 0x7f9cd8197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:28.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.821+0000 7f9cde683700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 0x7f9cd819c7e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009300 tx=0x7f9cd00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:28.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.821+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000e040 con 0x7f9cd8104ea0 2023-07-14T16:14:28.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.821+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cd0003e10 con 0x7f9cd8104ea0 2023-07-14T16:14:28.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.821+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd819d3a0 con 0x7f9cd8104ea0 2023-07-14T16:14:28.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.821+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd001b8e0 con 0x7f9cd8104ea0 2023-07-14T16:14:28.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.821+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd81a4c60 con 0x7f9cd8104ea0 2023-07-14T16:14:28.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.822+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9cd819d670 con 0x7f9cd8104ea0 2023-07-14T16:14:28.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.824+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9cd0005090 con 0x7f9cd8104ea0 2023-07-14T16:14:28.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.826+0000 7f9cceffd700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9cc40a11e0 0x7f9cc40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:28.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.826+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9cd00ce610 con 0x7f9cd8104ea0 2023-07-14T16:14:28.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.826+0000 7f9cdde82700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9cc40a11e0 0x7f9cc40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:28.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.826+0000 7f9cceffd700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] conn(0x7f9cc40aad10 0x7f9cc40ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:28.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.826+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 --> [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9cc40ad870 con 0x7f9cc40aad10 2023-07-14T16:14:28.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.826+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cd00ce980 con 0x7f9cd8104ea0 2023-07-14T16:14:28.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.826+0000 7f9cdde82700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9cc40a11e0 0x7f9cc40a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4005950 tx=0x7f9cd40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:28.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.827+0000 7f9cdd681700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] conn(0x7f9cc40aad10 0x7f9cc40ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:28.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.827+0000 7f9cdd681700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] conn(0x7f9cc40aad10 0x7f9cc40ad1a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:28.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.828+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 <== osd.11 v2:172.21.15.164:6824/1185509832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9cc40ad870 con 0x7f9cc40aad10 2023-07-14T16:14:28.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.845+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 --> [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9cd810fc40 con 0x7f9cc40aad10 2023-07-14T16:14:28.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9cceffd700 1 -- 172.21.15.62:0/2401765084 <== osd.11 v2:172.21.15.164:6824/1185509832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9cc800a700 con 0x7f9cc40aad10 2023-07-14T16:14:28.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 >> [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] conn(0x7f9cc40aad10 msgr2=0x7f9cc40ad1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:28.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] conn(0x7f9cc40aad10 0x7f9cc40ad1a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9cc40a11e0 msgr2=0x7f9cc40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:28.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9cc40a11e0 0x7f9cc40a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4005950 tx=0x7f9cd40058e0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 msgr2=0x7f9cd819c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:28.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 --2- 172.21.15.62:0/2401765084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cd8104ea0 0x7f9cd819c7e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009300 tx=0x7f9cd00093e0 comp rx=0 tx=0).stop 2023-07-14T16:14:28.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9cde683700 1 -- 172.21.15.62:0/2401765084 reap_dead start 2023-07-14T16:14:28.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 shutdown_connections 2023-07-14T16:14:28.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 >> 172.21.15.62:0/2401765084 conn(0x7f9cd80fb660 msgr2=0x7f9cd80fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:28.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 shutdown_connections 2023-07-14T16:14:28.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:28.846+0000 7f9ce00e6700 1 -- 172.21.15.62:0/2401765084 wait complete. 2023-07-14T16:14:28.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:28 smithi062 ceph-mon[95543]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 383 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:28 smithi164 ceph-mon[97427]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 383 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:29.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:28 smithi081 ceph-mon[127795]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 383 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:29.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 -- 172.21.15.62:0/3164283903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8584104ea0 msgr2=0x7f858410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 --2- 172.21.15.62:0/3164283903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8584104ea0 0x7f858410ca20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f857c00b0d0 tx=0x7f857c00b3e0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 -- 172.21.15.62:0/3164283903 shutdown_connections 2023-07-14T16:14:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 --2- 172.21.15.62:0/3164283903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8584104ea0 0x7f858410ca20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 --2- 172.21.15.62:0/3164283903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85840ffe00 0x7f85841001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 --2- 172.21.15.62:0/3164283903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85841007a0 0x7f8584104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 -- 172.21.15.62:0/3164283903 >> 172.21.15.62:0/3164283903 conn(0x7f85840fb660 msgr2=0x7f85840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 -- 172.21.15.62:0/3164283903 shutdown_connections 2023-07-14T16:14:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.487+0000 7f8589fb1700 1 -- 172.21.15.62:0/3164283903 wait complete. 2023-07-14T16:14:29.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.489+0000 7f8589fb1700 1 Processor -- start 2023-07-14T16:14:29.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.489+0000 7f8589fb1700 1 -- start start 2023-07-14T16:14:29.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.489+0000 7f8589fb1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85840ffe00 0x7f8584072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.489+0000 7f8589fb1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85841007a0 0x7f8584072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.489+0000 7f8589fb1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8584104ea0 0x7f8584073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8589fb1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a4110 con 0x7f85841007a0 2023-07-14T16:14:29.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8589fb1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a4280 con 0x7f8584104ea0 2023-07-14T16:14:29.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8589fb1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a4440 con 0x7f85840ffe00 2023-07-14T16:14:29.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8582ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85841007a0 0x7f8584072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f85837fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85840ffe00 0x7f8584072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8582ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85841007a0 0x7f8584072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46070/0 (socket says 172.21.15.62:46070) 2023-07-14T16:14:29.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8582ffd700 1 -- 172.21.15.62:0/2095103094 learned_addr learned my addr 172.21.15.62:0/2095103094 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:29.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8583fff700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8584104ea0 0x7f8584073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8582ffd700 1 -- 172.21.15.62:0/2095103094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85840ffe00 msgr2=0x7f8584072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8582ffd700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85840ffe00 0x7f8584072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8582ffd700 1 -- 172.21.15.62:0/2095103094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8584104ea0 msgr2=0x7f8584073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8582ffd700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8584104ea0 0x7f8584073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f8582ffd700 1 -- 172.21.15.62:0/2095103094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c009d00 con 0x7f85841007a0 2023-07-14T16:14:29.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.490+0000 7f85837fe700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85840ffe00 0x7f8584072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:14:29.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.491+0000 7f8582ffd700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85841007a0 0x7f8584072d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f857800e970 tx=0x7f857800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:29.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.491+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578009790 con 0x7f85841007a0 2023-07-14T16:14:29.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.491+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f857800c7e0 con 0x7f85841007a0 2023-07-14T16:14:29.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.491+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85841a4720 con 0x7f85841007a0 2023-07-14T16:14:29.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.491+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578005440 con 0x7f85841007a0 2023-07-14T16:14:29.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.491+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85841a4c70 con 0x7f85841007a0 2023-07-14T16:14:29.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.492+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f858410fc10 con 0x7f85841007a0 2023-07-14T16:14:29.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.494+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f857800c950 con 0x7f85841007a0 2023-07-14T16:14:29.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.496+0000 7f8580ff9700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f856c0a1260 0x7f856c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.496+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8578014070 con 0x7f85841007a0 2023-07-14T16:14:29.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.496+0000 7f85837fe700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f856c0a1260 0x7f856c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.496+0000 7f8580ff9700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] conn(0x7f856c0aad90 0x7f856c0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.496+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 --> [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f856c0ad8f0 con 0x7f856c0aad90 2023-07-14T16:14:29.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.496+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85780cd1e0 con 0x7f85841007a0 2023-07-14T16:14:29.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.496+0000 7f8583fff700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] conn(0x7f856c0aad90 0x7f856c0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.497+0000 7f85837fe700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f856c0a1260 0x7f856c0a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8574011490 tx=0x7f8574009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:29.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.497+0000 7f8583fff700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] conn(0x7f856c0aad90 0x7f856c0ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:29.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.498+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 <== osd.4 v2:172.21.15.81:6808/550163990 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f856c0ad8f0 con 0x7f856c0aad90 2023-07-14T16:14:29.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.515+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 --> [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f858404e210 con 0x7f856c0aad90 2023-07-14T16:14:29.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.516+0000 7f8580ff9700 1 -- 172.21.15.62:0/2095103094 <== osd.4 v2:172.21.15.81:6808/550163990 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f858404e210 con 0x7f856c0aad90 2023-07-14T16:14:29.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.516+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 >> [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] conn(0x7f856c0aad90 msgr2=0x7f856c0ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.516+0000 7f8589fb1700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] conn(0x7f856c0aad90 0x7f856c0ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.516+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f856c0a1260 msgr2=0x7f856c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.516+0000 7f8589fb1700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f856c0a1260 0x7f856c0a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8574011490 tx=0x7f8574009260 comp rx=0 tx=0).stop 2023-07-14T16:14:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.516+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85841007a0 msgr2=0x7f8584072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.516+0000 7f8589fb1700 1 --2- 172.21.15.62:0/2095103094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85841007a0 0x7f8584072d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f857800e970 tx=0x7f857800ec80 comp rx=0 tx=0).stop 2023-07-14T16:14:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.516+0000 7f8583fff700 1 -- 172.21.15.62:0/2095103094 reap_dead start 2023-07-14T16:14:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.517+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 shutdown_connections 2023-07-14T16:14:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.517+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 >> 172.21.15.62:0/2095103094 conn(0x7f85840fb660 msgr2=0x7f85840fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.517+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 shutdown_connections 2023-07-14T16:14:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.517+0000 7f8589fb1700 1 -- 172.21.15.62:0/2095103094 wait complete. 2023-07-14T16:14:29.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.539+0000 7f7431d6f700 1 -- 172.21.15.62:0/2803366406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c0728b0 msgr2=0x7f742c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.539+0000 7f7431d6f700 1 --2- 172.21.15.62:0/2803366406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c0728b0 0x7f742c072c80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f741c009990 tx=0x7f741c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.539+0000 7f7431d6f700 1 -- 172.21.15.62:0/2803366406 shutdown_connections 2023-07-14T16:14:29.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.539+0000 7f7431d6f700 1 --2- 172.21.15.62:0/2803366406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f742c10c3e0 0x7f742c10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.539+0000 7f7431d6f700 1 --2- 172.21.15.62:0/2803366406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c0728b0 0x7f742c072c80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.539+0000 7f7431d6f700 1 --2- 172.21.15.62:0/2803366406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f742c0731c0 0x7f742c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.539+0000 7f7431d6f700 1 -- 172.21.15.62:0/2803366406 >> 172.21.15.62:0/2803366406 conn(0x7f742c0fb660 msgr2=0x7f742c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:29.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.540+0000 7f7431d6f700 1 -- 172.21.15.62:0/2803366406 shutdown_connections 2023-07-14T16:14:29.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.540+0000 7f7431d6f700 1 -- 172.21.15.62:0/2803366406 wait complete. 2023-07-14T16:14:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.541+0000 7f7431d6f700 1 Processor -- start 2023-07-14T16:14:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.541+0000 7f7431d6f700 1 -- start start 2023-07-14T16:14:29.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.541+0000 7f7431d6f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f742c0728b0 0x7f742c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.541+0000 7f7431d6f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f742c0731c0 0x7f742c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.541+0000 7f7431d6f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c10c3e0 0x7f742c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.541+0000 7f7431d6f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c19cda0 con 0x7f742c10c3e0 2023-07-14T16:14:29.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.541+0000 7f7431d6f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c19cf10 con 0x7f742c0731c0 2023-07-14T16:14:29.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.541+0000 7f7431d6f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c19d0d0 con 0x7f742c0728b0 2023-07-14T16:14:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f742c0731c0 0x7f742c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f742c0731c0 0x7f742c198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:45316/0 (socket says 172.21.15.62:45316) 2023-07-14T16:14:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742affd700 1 -- 172.21.15.62:0/4134231769 learned_addr learned my addr 172.21.15.62:0/4134231769 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742bfff700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c10c3e0 0x7f742c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742b7fe700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f742c0728b0 0x7f742c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742bfff700 1 -- 172.21.15.62:0/4134231769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f742c0728b0 msgr2=0x7f742c197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742bfff700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f742c0728b0 0x7f742c197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742bfff700 1 -- 172.21.15.62:0/4134231769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f742c0731c0 msgr2=0x7f742c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742bfff700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f742c0731c0 0x7f742c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742bfff700 1 -- 172.21.15.62:0/4134231769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0095f0 con 0x7f742c10c3e0 2023-07-14T16:14:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742affd700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f742c0731c0 0x7f742c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:14:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.542+0000 7f742bfff700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c10c3e0 0x7f742c19c700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7424007f10 tx=0x7f742400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.543+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424009ea0 con 0x7f742c10c3e0 2023-07-14T16:14:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.543+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f742400f040 con 0x7f742c10c3e0 2023-07-14T16:14:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.543+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c19d3b0 con 0x7f742c10c3e0 2023-07-14T16:14:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.543+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424004280 con 0x7f742c10c3e0 2023-07-14T16:14:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.543+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c1a4b80 con 0x7f742c10c3e0 2023-07-14T16:14:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.544+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f742c04e210 con 0x7f742c10c3e0 2023-07-14T16:14:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.546+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f742400db20 con 0x7f742c10c3e0 2023-07-14T16:14:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.547+0000 7f7428ff9700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f74140a1260 0x7f74140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.548+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f74240ccfe0 con 0x7f742c10c3e0 2023-07-14T16:14:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.548+0000 7f742b7fe700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f74140a1260 0x7f74140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.548+0000 7f7428ff9700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] conn(0x7f74140aad90 0x7f74140ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.548+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 --> [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74140ad8f0 con 0x7f74140aad90 2023-07-14T16:14:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.548+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74240cd3d0 con 0x7f742c10c3e0 2023-07-14T16:14:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.548+0000 7f742b7fe700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f74140a1260 0x7f74140a3710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f741c00b5c0 tx=0x7f741c009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.548+0000 7f742affd700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] conn(0x7f74140aad90 0x7f74140ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.549+0000 7f742affd700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] conn(0x7f74140aad90 0x7f74140ad220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.550+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 <== osd.1 v2:172.21.15.81:6800/3123830761 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f74140ad8f0 con 0x7f74140aad90 2023-07-14T16:14:29.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.567+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 --> [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f742c111e20 con 0x7f74140aad90 2023-07-14T16:14:29.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.568+0000 7f7428ff9700 1 -- 172.21.15.62:0/4134231769 <== osd.1 v2:172.21.15.81:6800/3123830761 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f742000be10 con 0x7f74140aad90 2023-07-14T16:14:29.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 >> [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] conn(0x7f74140aad90 msgr2=0x7f74140ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] conn(0x7f74140aad90 0x7f74140ad220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f74140a1260 msgr2=0x7f74140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f74140a1260 0x7f74140a3710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f741c00b5c0 tx=0x7f741c009f00 comp rx=0 tx=0).stop 2023-07-14T16:14:29.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c10c3e0 msgr2=0x7f742c19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:29.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 --2- 172.21.15.62:0/4134231769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f742c10c3e0 0x7f742c19c700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7424007f10 tx=0x7f742400d3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:29.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f742bfff700 1 -- 172.21.15.62:0/4134231769 reap_dead start 2023-07-14T16:14:29.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 shutdown_connections 2023-07-14T16:14:29.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 >> 172.21.15.62:0/4134231769 conn(0x7f742c0fb660 msgr2=0x7f742c10e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:29.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 shutdown_connections 2023-07-14T16:14:29.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:29.569+0000 7f7431d6f700 1 -- 172.21.15.62:0/4134231769 wait complete. 2023-07-14T16:14:30.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 -- 172.21.15.62:0/346621927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbdc1007a0 msgr2=0x7fdbdc104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:30.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 --2- 172.21.15.62:0/346621927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbdc1007a0 0x7fdbdc104770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00099c0 tx=0x7fdbd0009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:30.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 -- 172.21.15.62:0/346621927 shutdown_connections 2023-07-14T16:14:30.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 --2- 172.21.15.62:0/346621927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc104ea0 0x7fdbdc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:30.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 --2- 172.21.15.62:0/346621927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbdc0ffe00 0x7fdbdc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:30.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 --2- 172.21.15.62:0/346621927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbdc1007a0 0x7fdbdc104770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:30.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 -- 172.21.15.62:0/346621927 >> 172.21.15.62:0/346621927 conn(0x7fdbdc0fb660 msgr2=0x7fdbdc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:30.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 -- 172.21.15.62:0/346621927 shutdown_connections 2023-07-14T16:14:30.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.658+0000 7fdbe12f4700 1 -- 172.21.15.62:0/346621927 wait complete. 2023-07-14T16:14:30.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.659+0000 7fdbe12f4700 1 Processor -- start 2023-07-14T16:14:30.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.660+0000 7fdbe12f4700 1 -- start start 2023-07-14T16:14:30.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.660+0000 7fdbe12f4700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc0ffe00 0x7fdbdc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:30.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.660+0000 7fdbe12f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbdc1007a0 0x7fdbdc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:30.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.660+0000 7fdbe12f4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbdc104ea0 0x7fdbdc073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:30.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.660+0000 7fdbe12f4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc1a8530 con 0x7fdbdc1007a0 2023-07-14T16:14:30.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.660+0000 7fdbe12f4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc1a8670 con 0x7fdbdc104ea0 2023-07-14T16:14:30.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.660+0000 7fdbe12f4700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc1a87b0 con 0x7fdbdc0ffe00 2023-07-14T16:14:30.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbdc104ea0 0x7fdbdc073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:30.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbdc104ea0 0x7fdbdc073290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:45326/0 (socket says 172.21.15.62:45326) 2023-07-14T16:14:30.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdb7fe700 1 -- 172.21.15.62:0/637945150 learned_addr learned my addr 172.21.15.62:0/637945150 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:30.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbda7fc700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbdc1007a0 0x7fdbdc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:30.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdaffd700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc0ffe00 0x7fdbdc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:30.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdb7fe700 1 -- 172.21.15.62:0/637945150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc0ffe00 msgr2=0x7fdbdc072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:30.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdb7fe700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc0ffe00 0x7fdbdc072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:30.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdb7fe700 1 -- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbdc1007a0 msgr2=0x7fdbdc072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:30.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdb7fe700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbdc1007a0 0x7fdbdc072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:30.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbdb7fe700 1 -- 172.21.15.62:0/637945150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00095f0 con 0x7fdbdc104ea0 2023-07-14T16:14:30.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.661+0000 7fdbda7fc700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbdc1007a0 0x7fdbdc072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:14:30.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.662+0000 7fdbdb7fe700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbdc104ea0 0x7fdbdc073290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4007d70 tx=0x7fdbd400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:30.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.662+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400ac90 con 0x7fdbdc104ea0 2023-07-14T16:14:30.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.662+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd400adf0 con 0x7fdbdc104ea0 2023-07-14T16:14:30.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.662+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc1a8a90 con 0x7fdbdc104ea0 2023-07-14T16:14:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.662+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40115e0 con 0x7fdbdc104ea0 2023-07-14T16:14:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.662+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc1a8f10 con 0x7fdbdc104ea0 2023-07-14T16:14:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.663+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdbdc10fb50 con 0x7fdbdc104ea0 2023-07-14T16:14:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.665+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdbd4004ad0 con 0x7fdbdc104ea0 2023-07-14T16:14:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.667+0000 7fdbc3fff700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdbc40a11e0 0x7fdbc40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.667+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdbd40ccee0 con 0x7fdbdc104ea0 2023-07-14T16:14:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.667+0000 7fdbdaffd700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdbc40a11e0 0x7fdbc40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.667+0000 7fdbc3fff700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] conn(0x7fdbc40aad10 0x7fdbc40ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.667+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 --> [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbc40ad870 con 0x7fdbc40aad10 2023-07-14T16:14:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.667+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd40cd250 con 0x7fdbdc104ea0 2023-07-14T16:14:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.668+0000 7fdbdaffd700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdbc40a11e0 0x7fdbc40a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005950 tx=0x7fdbcc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:30.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.668+0000 7fdbda7fc700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] conn(0x7fdbc40aad10 0x7fdbc40ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:30.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.668+0000 7fdbda7fc700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] conn(0x7fdbc40aad10 0x7fdbc40ad1a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:30.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.669+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 <== osd.5 v2:172.21.15.164:6808/1499523865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdbc40ad870 con 0x7fdbc40aad10 2023-07-14T16:14:30.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.686+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 --> [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdbdc04e210 con 0x7fdbc40aad10 2023-07-14T16:14:30.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbc3fff700 1 -- 172.21.15.62:0/637945150 <== osd.5 v2:172.21.15.164:6808/1499523865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdbdc04e210 con 0x7fdbc40aad10 2023-07-14T16:14:30.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 >> [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] conn(0x7fdbc40aad10 msgr2=0x7fdbc40ad1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:30.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbe12f4700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] conn(0x7fdbc40aad10 0x7fdbc40ad1a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:30.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdbc40a11e0 msgr2=0x7fdbc40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:30.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbe12f4700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdbc40a11e0 0x7fdbc40a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005950 tx=0x7fdbcc00b410 comp rx=0 tx=0).stop 2023-07-14T16:14:30.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbdc104ea0 msgr2=0x7fdbdc073290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:30.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbe12f4700 1 --2- 172.21.15.62:0/637945150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbdc104ea0 0x7fdbdc073290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4007d70 tx=0x7fdbd400a340 comp rx=0 tx=0).stop 2023-07-14T16:14:30.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbdb7fe700 1 -- 172.21.15.62:0/637945150 reap_dead start 2023-07-14T16:14:30.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 shutdown_connections 2023-07-14T16:14:30.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.687+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 >> 172.21.15.62:0/637945150 conn(0x7fdbdc0fb660 msgr2=0x7fdbdc0fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:30.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.688+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 shutdown_connections 2023-07-14T16:14:30.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:30.688+0000 7fdbe12f4700 1 -- 172.21.15.62:0/637945150 wait complete. 2023-07-14T16:14:31.017 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:30 smithi062 ceph-mon[95543]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 312 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:31.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:30 smithi164 ceph-mon[97427]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 312 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:31.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:30 smithi081 ceph-mon[127795]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 312 B/s wr, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T16:14:31.172 INFO:teuthology.orchestra.run.smithi062.stdout:347892350996 2023-07-14T16:14:31.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-14T16:14:31.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 -- 172.21.15.62:0/2477575504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0ffe00 msgr2=0x7f639c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 --2- 172.21.15.62:0/2477575504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0ffe00 0x7f639c1001d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f63980099c0 tx=0x7f6398009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 -- 172.21.15.62:0/2477575504 shutdown_connections 2023-07-14T16:14:31.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 --2- 172.21.15.62:0/2477575504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c104ea0 0x7f639c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 --2- 172.21.15.62:0/2477575504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c0ffe00 0x7f639c1001d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 --2- 172.21.15.62:0/2477575504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f639c1007a0 0x7f639c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 -- 172.21.15.62:0/2477575504 >> 172.21.15.62:0/2477575504 conn(0x7f639c0fb660 msgr2=0x7f639c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:31.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 -- 172.21.15.62:0/2477575504 shutdown_connections 2023-07-14T16:14:31.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.701+0000 7f63a4181700 1 -- 172.21.15.62:0/2477575504 wait complete. 2023-07-14T16:14:31.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.703+0000 7f63a4181700 1 Processor -- start 2023-07-14T16:14:31.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.703+0000 7f63a4181700 1 -- start start 2023-07-14T16:14:31.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a4181700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c0ffe00 0x7f639c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a4181700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f639c1007a0 0x7f639c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a4181700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c104ea0 0x7f639c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a4181700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19cd70 con 0x7f639c104ea0 2023-07-14T16:14:31.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a4181700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19cee0 con 0x7f639c1007a0 2023-07-14T16:14:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a4181700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19d0a0 con 0x7f639c0ffe00 2023-07-14T16:14:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a271e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c104ea0 0x7f639c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a271e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c104ea0 0x7f639c19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46148/0 (socket says 172.21.15.62:46148) 2023-07-14T16:14:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a271e700 1 -- 172.21.15.62:0/3496880776 learned_addr learned my addr 172.21.15.62:0/3496880776 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a1f1d700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c0ffe00 0x7f639c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a171c700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f639c1007a0 0x7f639c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a271e700 1 -- 172.21.15.62:0/3496880776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c0ffe00 msgr2=0x7f639c197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a271e700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f639c0ffe00 0x7f639c197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a271e700 1 -- 172.21.15.62:0/3496880776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f639c1007a0 msgr2=0x7f639c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a271e700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f639c1007a0 0x7f639c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.704+0000 7f63a271e700 1 -- 172.21.15.62:0/3496880776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63980095f0 con 0x7f639c104ea0 2023-07-14T16:14:31.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.705+0000 7f63a271e700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c104ea0 0x7f639c19c760 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6394007ee0 tx=0x7f639400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:31.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.705+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394017070 con 0x7f639c104ea0 2023-07-14T16:14:31.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.705+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f639400f040 con 0x7f639c104ea0 2023-07-14T16:14:31.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.705+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c19d380 con 0x7f639c104ea0 2023-07-14T16:14:31.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.705+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394005570 con 0x7f639c104ea0 2023-07-14T16:14:31.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.705+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c1a4be0 con 0x7f639c104ea0 2023-07-14T16:14:31.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.706+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f639c10fc50 con 0x7f639c104ea0 2023-07-14T16:14:31.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.708+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f639400db20 con 0x7f639c104ea0 2023-07-14T16:14:31.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.709+0000 7f6392ffd700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63880a1260 0x7f63880a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.709+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f63940cc120 con 0x7f639c104ea0 2023-07-14T16:14:31.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.709+0000 7f63a1f1d700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63880a1260 0x7f63880a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.710+0000 7f6392ffd700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] conn(0x7f63880aad90 0x7f63880ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.710+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 --> [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63880ad8f0 con 0x7f63880aad90 2023-07-14T16:14:31.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.710+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63940ce700 con 0x7f639c104ea0 2023-07-14T16:14:31.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.710+0000 7f63a1f1d700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63880a1260 0x7f63880a3710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6398000c00 tx=0x7f6398009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:31.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.710+0000 7f63a171c700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] conn(0x7f63880aad90 0x7f63880ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.711+0000 7f63a171c700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] conn(0x7f63880aad90 0x7f63880ad220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:31.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.712+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 <== osd.8 v2:172.21.15.164:6816/1299247730 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f63880ad8f0 con 0x7f63880aad90 2023-07-14T16:14:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.728+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 --> [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f639c04e210 con 0x7f63880aad90 2023-07-14T16:14:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f6392ffd700 1 -- 172.21.15.62:0/3496880776 <== osd.8 v2:172.21.15.164:6816/1299247730 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f639c04e210 con 0x7f63880aad90 2023-07-14T16:14:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 >> [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] conn(0x7f63880aad90 msgr2=0x7f63880ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] conn(0x7f63880aad90 0x7f63880ad220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63880a1260 msgr2=0x7f63880a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63880a1260 0x7f63880a3710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6398000c00 tx=0x7f6398009f10 comp rx=0 tx=0).stop 2023-07-14T16:14:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c104ea0 msgr2=0x7f639c19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 --2- 172.21.15.62:0/3496880776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c104ea0 0x7f639c19c760 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6394007ee0 tx=0x7f639400d3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a271e700 1 -- 172.21.15.62:0/3496880776 reap_dead start 2023-07-14T16:14:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 shutdown_connections 2023-07-14T16:14:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 >> 172.21.15.62:0/3496880776 conn(0x7f639c0fb660 msgr2=0x7f639c0fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 shutdown_connections 2023-07-14T16:14:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.729+0000 7f63a4181700 1 -- 172.21.15.62:0/3496880776 wait complete. 2023-07-14T16:14:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.509+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3811865735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1d0107ae0 msgr2=0x7fc1d0107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.509+0000 7fc1d6fc0700 1 --2- 172.21.15.62:0/3811865735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1d0107ae0 0x7fc1d0107f50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009990 tx=0x7fc1bc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.509+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3811865735 shutdown_connections 2023-07-14T16:14:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.509+0000 7fc1d6fc0700 1 --2- 172.21.15.62:0/3811865735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1d0112930 0x7fc1d0114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.509+0000 7fc1d6fc0700 1 --2- 172.21.15.62:0/3811865735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d006da70 0x7fc1d0107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.509+0000 7fc1d6fc0700 1 --2- 172.21.15.62:0/3811865735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1d0107ae0 0x7fc1d0107f50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.510+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3811865735 >> 172.21.15.62:0/3811865735 conn(0x7fc1d006be40 msgr2=0x7fc1d0071020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.510+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3811865735 shutdown_connections 2023-07-14T16:14:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.510+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3811865735 wait complete. 2023-07-14T16:14:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.511+0000 7fc1d6fc0700 1 Processor -- start 2023-07-14T16:14:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.511+0000 7fc1d6fc0700 1 -- start start 2023-07-14T16:14:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1d6fc0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1d006da70 0x7fc1d0108710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1d6fc0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1d0107ae0 0x7fc1d0108c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1d6fc0700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d0112930 0x7fc1d0109190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1d6fc0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01a2550 con 0x7fc1d006da70 2023-07-14T16:14:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1d6fc0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01a26c0 con 0x7fc1d0107ae0 2023-07-14T16:14:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1d6fc0700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01a2880 con 0x7fc1d0112930 2023-07-14T16:14:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1cffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1d0107ae0 0x7fc1d0108c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1cffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1d0107ae0 0x7fc1d0108c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:45350/0 (socket says 172.21.15.62:45350) 2023-07-14T16:14:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1cffff700 1 -- 172.21.15.62:0/3322637395 learned_addr learned my addr 172.21.15.62:0/3322637395 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.512+0000 7fc1d555d700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d0112930 0x7fc1d0109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1d4d5c700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1d006da70 0x7fc1d0108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1cffff700 1 -- 172.21.15.62:0/3322637395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d0112930 msgr2=0x7fc1d0109190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1cffff700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d0112930 0x7fc1d0109190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1cffff700 1 -- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1d006da70 msgr2=0x7fc1d0108710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1cffff700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1d006da70 0x7fc1d0108710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1cffff700 1 -- 172.21.15.62:0/3322637395 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0095f0 con 0x7fc1d0107ae0 2023-07-14T16:14:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1d555d700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d0112930 0x7fc1d0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:14:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1d4d5c700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1d006da70 0x7fc1d0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:31.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.513+0000 7fc1cffff700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1d0107ae0 0x7fc1d0108c50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc006010 tx=0x7fc1bc00be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:31.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.514+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc00fb00 con 0x7fc1d0107ae0 2023-07-14T16:14:31.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.514+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1bc03f070 con 0x7fc1d0107ae0 2023-07-14T16:14:31.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.514+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0173f0 con 0x7fc1d0107ae0 2023-07-14T16:14:31.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.514+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d01a2b00 con 0x7fc1d0107ae0 2023-07-14T16:14:31.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.514+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d01a2f50 con 0x7fc1d0107ae0 2023-07-14T16:14:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.515+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1d01183e0 con 0x7fc1d0107ae0 2023-07-14T16:14:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.519+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc1bc044480 con 0x7fc1d0107ae0 2023-07-14T16:14:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.520+0000 7fc1cdffb700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1c00a1260 0x7fc1c00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.521+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc1bc0f0410 con 0x7fc1d0107ae0 2023-07-14T16:14:31.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.521+0000 7fc1cdffb700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] conn(0x7fc1c00aad90 0x7fc1c00ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:31.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.521+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 --> [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1c00ad8f0 con 0x7fc1c00aad90 2023-07-14T16:14:31.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.521+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1bc0f0780 con 0x7fc1d0107ae0 2023-07-14T16:14:31.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.521+0000 7fc1d4d5c700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1c00a1260 0x7fc1c00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.522+0000 7fc1d555d700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] conn(0x7fc1c00aad90 0x7fc1c00ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:31.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.522+0000 7fc1d4d5c700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1c00a1260 0x7fc1c00a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009cb0 tx=0x7fc1c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.522+0000 7fc1d555d700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] conn(0x7fc1c00aad90 0x7fc1c00ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.523+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 <== osd.7 v2:172.21.15.81:6816/1584123297 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc1c00ad8f0 con 0x7fc1c00aad90 2023-07-14T16:14:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.541+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 --> [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1d004e210 con 0x7fc1c00aad90 2023-07-14T16:14:31.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.542+0000 7fc1cdffb700 1 -- 172.21.15.62:0/3322637395 <== osd.7 v2:172.21.15.81:6816/1584123297 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc1d004e210 con 0x7fc1c00aad90 2023-07-14T16:14:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.542+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 >> [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] conn(0x7fc1c00aad90 msgr2=0x7fc1c00ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.542+0000 7fc1d6fc0700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] conn(0x7fc1c00aad90 0x7fc1c00ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.542+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1c00a1260 msgr2=0x7fc1c00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.542+0000 7fc1d6fc0700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1c00a1260 0x7fc1c00a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009cb0 tx=0x7fc1c4009260 comp rx=0 tx=0).stop 2023-07-14T16:14:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.542+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1d0107ae0 msgr2=0x7fc1d0108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.542+0000 7fc1d6fc0700 1 --2- 172.21.15.62:0/3322637395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1d0107ae0 0x7fc1d0108c50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc006010 tx=0x7fc1bc00be50 comp rx=0 tx=0).stop 2023-07-14T16:14:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.543+0000 7fc1d555d700 1 -- 172.21.15.62:0/3322637395 reap_dead start 2023-07-14T16:14:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.543+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 shutdown_connections 2023-07-14T16:14:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.543+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 >> 172.21.15.62:0/3322637395 conn(0x7fc1d006be40 msgr2=0x7fc1d010a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.543+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 shutdown_connections 2023-07-14T16:14:31.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:31.543+0000 7fc1d6fc0700 1 -- 172.21.15.62:0/3322637395 wait complete. 2023-07-14T16:14:32.110 INFO:teuthology.orchestra.run.smithi062.stdout:55834574922 2023-07-14T16:14:32.110 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-14T16:14:32.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.243+0000 7f64fe539700 1 -- 172.21.15.62:0/3552020347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8100df0 msgr2=0x7f64f81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:32.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.243+0000 7f64fe539700 1 --2- 172.21.15.62:0/3552020347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8100df0 0x7f64f81011c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009990 tx=0x7f64e8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:32.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.244+0000 7f64fe539700 1 -- 172.21.15.62:0/3552020347 shutdown_connections 2023-07-14T16:14:32.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.244+0000 7f64fe539700 1 --2- 172.21.15.62:0/3552020347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f810a140 0x7f64f810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:32.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.244+0000 7f64fe539700 1 --2- 172.21.15.62:0/3552020347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8100df0 0x7f64f81011c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:32.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.244+0000 7f64fe539700 1 --2- 172.21.15.62:0/3552020347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64f8101700 0x7f64f8109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:32.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.244+0000 7f64fe539700 1 -- 172.21.15.62:0/3552020347 >> 172.21.15.62:0/3552020347 conn(0x7f64f8075b70 msgr2=0x7f64f8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.244+0000 7f64fe539700 1 -- 172.21.15.62:0/3552020347 shutdown_connections 2023-07-14T16:14:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.244+0000 7f64fe539700 1 -- 172.21.15.62:0/3552020347 wait complete. 2023-07-14T16:14:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.245+0000 7f64fe539700 1 Processor -- start 2023-07-14T16:14:32.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.245+0000 7f64fe539700 1 -- start start 2023-07-14T16:14:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64fe539700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f8100df0 0x7f64f8071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64fe539700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8101700 0x7f64f8072140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64fe539700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64f810a140 0x7f64f80fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64fe539700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80ff220 con 0x7f64f8101700 2023-07-14T16:14:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64fe539700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80ff390 con 0x7f64f8100df0 2023-07-14T16:14:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64fe539700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80ff550 con 0x7f64f810a140 2023-07-14T16:14:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8101700 0x7f64f8072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8101700 0x7f64f8072140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46174/0 (socket says 172.21.15.62:46174) 2023-07-14T16:14:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64f77fe700 1 -- 172.21.15.62:0/2802448116 learned_addr learned my addr 172.21.15.62:0/2802448116 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.246+0000 7f64f7fff700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f8100df0 0x7f64f8071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64fcad6700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64f810a140 0x7f64f80fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f77fe700 1 -- 172.21.15.62:0/2802448116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64f810a140 msgr2=0x7f64f80fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f77fe700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64f810a140 0x7f64f80fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f77fe700 1 -- 172.21.15.62:0/2802448116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f8100df0 msgr2=0x7f64f8071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f77fe700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64f8100df0 0x7f64f8071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f77fe700 1 -- 172.21.15.62:0/2802448116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80095f0 con 0x7f64f8101700 2023-07-14T16:14:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f77fe700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8101700 0x7f64f8072140 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00d950 tx=0x7f64ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec009940 con 0x7f64f8101700 2023-07-14T16:14:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64ec010460 con 0x7f64f8101700 2023-07-14T16:14:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec00f620 con 0x7f64f8101700 2023-07-14T16:14:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f80ff830 con 0x7f64f8101700 2023-07-14T16:14:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.247+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f81ad2c0 con 0x7f64f8101700 2023-07-14T16:14:32.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.248+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64f804e210 con 0x7f64f8101700 2023-07-14T16:14:32.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.251+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f64ec0105d0 con 0x7f64f8101700 2023-07-14T16:14:32.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.252+0000 7f64f57fa700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64e00a1260 0x7f64e00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:32.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.253+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f64ec0ccdc0 con 0x7f64f8101700 2023-07-14T16:14:32.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.253+0000 7f64f57fa700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] conn(0x7f64e00aad90 0x7f64e00ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:32.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.253+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 --> [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64e00ad8f0 con 0x7f64e00aad90 2023-07-14T16:14:32.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.253+0000 7f64f7fff700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64e00a1260 0x7f64e00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:32.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.253+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64ec0cd1d0 con 0x7f64f8101700 2023-07-14T16:14:32.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.253+0000 7f64fcad6700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] conn(0x7f64e00aad90 0x7f64e00ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:32.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.254+0000 7f64f7fff700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64e00a1260 0x7f64e00a3710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64e800b5c0 tx=0x7f64e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:32.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.254+0000 7f64fcad6700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] conn(0x7f64e00aad90 0x7f64e00ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:32.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.255+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 <== osd.9 v2:172.21.15.62:6826/1938724436 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f64e00ad8f0 con 0x7f64e00aad90 2023-07-14T16:14:32.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.274+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 --> [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64f810fc20 con 0x7f64e00aad90 2023-07-14T16:14:32.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.275+0000 7f64f57fa700 1 -- 172.21.15.62:0/2802448116 <== osd.9 v2:172.21.15.62:6826/1938724436 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f64f00098c0 con 0x7f64e00aad90 2023-07-14T16:14:32.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.276+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] conn(0x7f64e00aad90 msgr2=0x7f64e00ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:32.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.276+0000 7f64fe539700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] conn(0x7f64e00aad90 0x7f64e00ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:32.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.276+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64e00a1260 msgr2=0x7f64e00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:32.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.276+0000 7f64fe539700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64e00a1260 0x7f64e00a3710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64e800b5c0 tx=0x7f64e8005fb0 comp rx=0 tx=0).stop 2023-07-14T16:14:32.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.276+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8101700 msgr2=0x7f64f8072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:32.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.276+0000 7f64fe539700 1 --2- 172.21.15.62:0/2802448116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64f8101700 0x7f64f8072140 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00d950 tx=0x7f64ec00dc60 comp rx=0 tx=0).stop 2023-07-14T16:14:32.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.276+0000 7f64fcad6700 1 -- 172.21.15.62:0/2802448116 reap_dead start 2023-07-14T16:14:32.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.277+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 shutdown_connections 2023-07-14T16:14:32.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.277+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 >> 172.21.15.62:0/2802448116 conn(0x7f64f8075b70 msgr2=0x7f64f8101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:32.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.278+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 shutdown_connections 2023-07-14T16:14:32.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.278+0000 7f64fe539700 1 -- 172.21.15.62:0/2802448116 wait complete. 2023-07-14T16:14:32.935 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:32 smithi062 ceph-mon[95543]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 284 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:33.010 INFO:teuthology.orchestra.run.smithi062.stdout:257698037796 2023-07-14T16:14:33.010 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-14T16:14:33.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:32 smithi164 ceph-mon[97427]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 284 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:33.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:32 smithi081 ceph-mon[127795]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 284 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:33.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.933+0000 7f69ddf26700 1 -- 172.21.15.62:0/2783701722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 msgr2=0x7f69d80fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.933+0000 7f69ddf26700 1 --2- 172.21.15.62:0/2783701722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 0x7f69d80fef20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009990 tx=0x7f69cc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.933+0000 7f69ddf26700 1 -- 172.21.15.62:0/2783701722 shutdown_connections 2023-07-14T16:14:33.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.933+0000 7f69ddf26700 1 --2- 172.21.15.62:0/2783701722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69d80ff460 0x7f69d810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.933+0000 7f69ddf26700 1 --2- 172.21.15.62:0/2783701722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d8104ab0 0x7f69d8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.933+0000 7f69ddf26700 1 --2- 172.21.15.62:0/2783701722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 0x7f69d80fef20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.933+0000 7f69ddf26700 1 -- 172.21.15.62:0/2783701722 >> 172.21.15.62:0/2783701722 conn(0x7f69d8075b70 msgr2=0x7f69d8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:33.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.933+0000 7f69ddf26700 1 -- 172.21.15.62:0/2783701722 shutdown_connections 2023-07-14T16:14:33.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.934+0000 7f69ddf26700 1 -- 172.21.15.62:0/2783701722 wait complete. 2023-07-14T16:14:33.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.935+0000 7f69ddf26700 1 Processor -- start 2023-07-14T16:14:33.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.935+0000 7f69ddf26700 1 -- start start 2023-07-14T16:14:33.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69ddf26700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 0x7f69d8196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69ddf26700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d80ff460 0x7f69d8196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69ddf26700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69d8104ab0 0x7f69d81a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69ddf26700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a1f20 con 0x7f69d80feab0 2023-07-14T16:14:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69ddf26700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a2060 con 0x7f69d8104ab0 2023-07-14T16:14:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69ddf26700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81a21a0 con 0x7f69d80ff460 2023-07-14T16:14:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 0x7f69d8196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 0x7f69d8196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48360/0 (socket says 172.21.15.62:48360) 2023-07-14T16:14:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69d77fe700 1 -- 172.21.15.62:0/2700335188 learned_addr learned my addr 172.21.15.62:0/2700335188 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.936+0000 7f69d6ffd700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d80ff460 0x7f69d8196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d7fff700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69d8104ab0 0x7f69d81a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d77fe700 1 -- 172.21.15.62:0/2700335188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d80ff460 msgr2=0x7f69d8196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d77fe700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69d80ff460 0x7f69d8196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d77fe700 1 -- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69d8104ab0 msgr2=0x7f69d81a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d77fe700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69d8104ab0 0x7f69d81a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d77fe700 1 -- 172.21.15.62:0/2700335188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc0095f0 con 0x7f69d80feab0 2023-07-14T16:14:33.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d7fff700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69d8104ab0 0x7f69d81a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:33.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d77fe700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 0x7f69d8196960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f69c800d980 tx=0x7f69c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8004020 con 0x7f69d80feab0 2023-07-14T16:14:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69c8009730 con 0x7f69d80feab0 2023-07-14T16:14:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d81a2400 con 0x7f69d80feab0 2023-07-14T16:14:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8003a00 con 0x7f69d80feab0 2023-07-14T16:14:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.937+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d81a28d0 con 0x7f69d80feab0 2023-07-14T16:14:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.939+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f69d810eb70 con 0x7f69d80feab0 2023-07-14T16:14:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.942+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f69c80098a0 con 0x7f69d80feab0 2023-07-14T16:14:33.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.944+0000 7f69d4ff9700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69c00a11e0 0x7f69c00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.944+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69c80cced0 con 0x7f69d80feab0 2023-07-14T16:14:33.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.944+0000 7f69d6ffd700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69c00a11e0 0x7f69c00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.944+0000 7f69d4ff9700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] conn(0x7f69c00aad10 0x7f69c00ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.944+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 --> [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69c00ad870 con 0x7f69c00aad10 2023-07-14T16:14:33.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.944+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69c80cd240 con 0x7f69d80feab0 2023-07-14T16:14:33.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.945+0000 7f69d6ffd700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69c00a11e0 0x7f69c00a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69d80690b0 tx=0x7f69cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.945+0000 7f69d7fff700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] conn(0x7f69c00aad10 0x7f69c00ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.945+0000 7f69d7fff700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] conn(0x7f69c00aad10 0x7f69c00ad1a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.946+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 <== osd.10 v2:172.21.15.81:6824/737826799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f69c00ad870 con 0x7f69c00aad10 2023-07-14T16:14:33.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.964+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 --> [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f69d804e210 con 0x7f69c00aad10 2023-07-14T16:14:33.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.965+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2700335188 <== osd.10 v2:172.21.15.81:6824/737826799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f69d804e210 con 0x7f69c00aad10 2023-07-14T16:14:33.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.965+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] conn(0x7f69c00aad10 msgr2=0x7f69c00ad1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.965+0000 7f69ddf26700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] conn(0x7f69c00aad10 0x7f69c00ad1a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.965+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69c00a11e0 msgr2=0x7f69c00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.965+0000 7f69ddf26700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69c00a11e0 0x7f69c00a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69d80690b0 tx=0x7f69cc005fb0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.965+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 msgr2=0x7f69d8196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.965+0000 7f69ddf26700 1 --2- 172.21.15.62:0/2700335188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d80feab0 0x7f69d8196960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f69c800d980 tx=0x7f69c800dc90 comp rx=0 tx=0).stop 2023-07-14T16:14:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.965+0000 7f69d7fff700 1 -- 172.21.15.62:0/2700335188 reap_dead start 2023-07-14T16:14:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.966+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 shutdown_connections 2023-07-14T16:14:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.966+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 >> 172.21.15.62:0/2700335188 conn(0x7f69d8075b70 msgr2=0x7f69d80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.966+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 shutdown_connections 2023-07-14T16:14:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.966+0000 7f69ddf26700 1 -- 172.21.15.62:0/2700335188 wait complete. 2023-07-14T16:14:33.659 INFO:teuthology.orchestra.run.smithi062.stdout:167503724596 2023-07-14T16:14:33.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-14T16:14:33.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 -- 172.21.15.62:0/3647890813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc104ea0 msgr2=0x7fb9fc10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 --2- 172.21.15.62:0/3647890813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc104ea0 0x7fb9fc10ca20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400b0d0 tx=0x7fb9f400b3e0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 -- 172.21.15.62:0/3647890813 shutdown_connections 2023-07-14T16:14:33.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 --2- 172.21.15.62:0/3647890813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc104ea0 0x7fb9fc10ca20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 --2- 172.21.15.62:0/3647890813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9fc0ffe00 0x7fb9fc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 --2- 172.21.15.62:0/3647890813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9fc1007a0 0x7fb9fc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 -- 172.21.15.62:0/3647890813 >> 172.21.15.62:0/3647890813 conn(0x7fb9fc0fb660 msgr2=0x7fb9fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:33.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 -- 172.21.15.62:0/3647890813 shutdown_connections 2023-07-14T16:14:33.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.270+0000 7fba02237700 1 -- 172.21.15.62:0/3647890813 wait complete. 2023-07-14T16:14:33.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.271+0000 7fba02237700 1 Processor -- start 2023-07-14T16:14:33.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fba02237700 1 -- start start 2023-07-14T16:14:33.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fba02237700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc0ffe00 0x7fb9fc195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fba02237700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9fc1007a0 0x7fb9fc196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fba02237700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9fc104ea0 0x7fb9fc19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fba02237700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc19acf0 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fba02237700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc19ae60 con 0x7fb9fc1007a0 2023-07-14T16:14:33.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fba02237700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc19b020 con 0x7fb9fc104ea0 2023-07-14T16:14:33.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fb9fb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc0ffe00 0x7fb9fc195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fb9fb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc0ffe00 0x7fb9fc195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48378/0 (socket says 172.21.15.62:48378) 2023-07-14T16:14:33.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.272+0000 7fb9fb7fe700 1 -- 172.21.15.62:0/3070977099 learned_addr learned my addr 172.21.15.62:0/3070977099 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:33.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9fbfff700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9fc104ea0 0x7fb9fc19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9faffd700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9fc1007a0 0x7fb9fc196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9fb7fe700 1 -- 172.21.15.62:0/3070977099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9fc104ea0 msgr2=0x7fb9fc19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9fb7fe700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9fc104ea0 0x7fb9fc19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9fb7fe700 1 -- 172.21.15.62:0/3070977099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9fc1007a0 msgr2=0x7fb9fc196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9fb7fe700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9fc1007a0 0x7fb9fc196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9fb7fe700 1 -- 172.21.15.62:0/3070977099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f4009d00 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9fb7fe700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc0ffe00 0x7fb9fc195d30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00e9a0 tx=0x7fb9ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec009790 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9ec010430 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00cea0 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.273+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc19b300 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.274+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc1a2b60 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.275+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9fc10fb30 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.277+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb9ec0105a0 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.278+0000 7fb9f8ff9700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e40a1260 0x7fb9e40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.279+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9ec014070 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.279+0000 7fb9faffd700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e40a1260 0x7fb9e40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.279+0000 7fb9f8ff9700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] conn(0x7fb9e40aad90 0x7fb9e40ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.279+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 --> [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9e40ad8f0 con 0x7fb9e40aad90 2023-07-14T16:14:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.279+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9ec0cd280 con 0x7fb9fc0ffe00 2023-07-14T16:14:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.279+0000 7fb9fbfff700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] conn(0x7fb9e40aad90 0x7fb9e40ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.280+0000 7fb9faffd700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e40a1260 0x7fb9e40a3710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc197350 tx=0x7fb9f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.280+0000 7fb9fbfff700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] conn(0x7fb9e40aad90 0x7fb9e40ad220 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.281+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 <== osd.0 v2:172.21.15.62:6802/2960730798 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb9e40ad8f0 con 0x7fb9e40aad90 2023-07-14T16:14:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.313+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 --> [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9fc04e210 con 0x7fb9e40aad90 2023-07-14T16:14:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.314+0000 7fb9f8ff9700 1 -- 172.21.15.62:0/3070977099 <== osd.0 v2:172.21.15.62:6802/2960730798 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb9fc04e210 con 0x7fb9e40aad90 2023-07-14T16:14:33.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.314+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] conn(0x7fb9e40aad90 msgr2=0x7fb9e40ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.314+0000 7fba02237700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] conn(0x7fb9e40aad90 0x7fb9e40ad220 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e40a1260 msgr2=0x7fb9e40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fba02237700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e40a1260 0x7fb9e40a3710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc197350 tx=0x7fb9f0009260 comp rx=0 tx=0).stop 2023-07-14T16:14:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc0ffe00 msgr2=0x7fb9fc195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fba02237700 1 --2- 172.21.15.62:0/3070977099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9fc0ffe00 0x7fb9fc195d30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00e9a0 tx=0x7fb9ec00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fb9fbfff700 1 -- 172.21.15.62:0/3070977099 reap_dead start 2023-07-14T16:14:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 shutdown_connections 2023-07-14T16:14:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 >> 172.21.15.62:0/3070977099 conn(0x7fb9fc0fb660 msgr2=0x7fb9fc0fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 shutdown_connections 2023-07-14T16:14:33.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.315+0000 7fba02237700 1 -- 172.21.15.62:0/3070977099 wait complete. 2023-07-14T16:14:33.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.358+0000 7f2ca4d20700 1 -- 172.21.15.62:0/180373065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca01007a0 msgr2=0x7f2ca0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.358+0000 7f2ca4d20700 1 --2- 172.21.15.62:0/180373065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca01007a0 0x7f2ca0104770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2c940099c0 tx=0x7f2c94009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.359+0000 7f2ca4d20700 1 -- 172.21.15.62:0/180373065 shutdown_connections 2023-07-14T16:14:33.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.359+0000 7f2ca4d20700 1 --2- 172.21.15.62:0/180373065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ca0104ea0 0x7f2ca010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.359+0000 7f2ca4d20700 1 --2- 172.21.15.62:0/180373065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca00ffe00 0x7f2ca01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.359+0000 7f2ca4d20700 1 --2- 172.21.15.62:0/180373065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca01007a0 0x7f2ca0104770 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.359+0000 7f2ca4d20700 1 -- 172.21.15.62:0/180373065 >> 172.21.15.62:0/180373065 conn(0x7f2ca00fb660 msgr2=0x7f2ca00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:33.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.359+0000 7f2ca4d20700 1 -- 172.21.15.62:0/180373065 shutdown_connections 2023-07-14T16:14:33.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.359+0000 7f2ca4d20700 1 -- 172.21.15.62:0/180373065 wait complete. 2023-07-14T16:14:33.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.360+0000 7f2ca4d20700 1 Processor -- start 2023-07-14T16:14:33.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.360+0000 7f2ca4d20700 1 -- start start 2023-07-14T16:14:33.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2ca4d20700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ca00ffe00 0x7f2ca0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2ca4d20700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca01007a0 0x7f2ca0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2ca4d20700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca0104ea0 0x7f2ca00732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2ca4d20700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca01a40c0 con 0x7f2ca0104ea0 2023-07-14T16:14:33.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2ca4d20700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca01a4230 con 0x7f2ca01007a0 2023-07-14T16:14:33.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2ca4d20700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca01a43f0 con 0x7f2ca00ffe00 2023-07-14T16:14:33.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2c9ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca0104ea0 0x7f2ca00732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2c9e59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ca00ffe00 0x7f2ca0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2c9ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca0104ea0 0x7f2ca00732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48408/0 (socket says 172.21.15.62:48408) 2023-07-14T16:14:33.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2c9ed9d700 1 -- 172.21.15.62:0/2170780400 learned_addr learned my addr 172.21.15.62:0/2170780400 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:33.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.361+0000 7f2c9dd9b700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca01007a0 0x7f2ca0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c9ed9d700 1 -- 172.21.15.62:0/2170780400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ca00ffe00 msgr2=0x7f2ca0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c9ed9d700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ca00ffe00 0x7f2ca0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c9ed9d700 1 -- 172.21.15.62:0/2170780400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca01007a0 msgr2=0x7f2ca0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c9ed9d700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca01007a0 0x7f2ca0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c9ed9d700 1 -- 172.21.15.62:0/2170780400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940095f0 con 0x7f2ca0104ea0 2023-07-14T16:14:33.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c9dd9b700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ca01007a0 0x7f2ca0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:14:33.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c9ed9d700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca0104ea0 0x7f2ca00732d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b970 tx=0x7f2c9800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800d860 con 0x7f2ca0104ea0 2023-07-14T16:14:33.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca01a46d0 con 0x7f2ca0104ea0 2023-07-14T16:14:33.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c9800de50 con 0x7f2ca0104ea0 2023-07-14T16:14:33.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98017400 con 0x7f2ca0104ea0 2023-07-14T16:14:33.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.362+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca01a4bf0 con 0x7f2ca0104ea0 2023-07-14T16:14:33.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.363+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ca010fc50 con 0x7f2ca0104ea0 2023-07-14T16:14:33.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.365+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2c9800d9c0 con 0x7f2ca0104ea0 2023-07-14T16:14:33.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.367+0000 7f2c8f7fe700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c880a11e0 0x7f2c880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.367+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c98006e40 con 0x7f2ca0104ea0 2023-07-14T16:14:33.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.367+0000 7f2c9e59c700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c880a11e0 0x7f2c880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.367+0000 7f2c8f7fe700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] conn(0x7f2c880aad10 0x7f2c880ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:33.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.367+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 --> [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c880ad870 con 0x7f2c880aad10 2023-07-14T16:14:33.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.367+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c980ccc80 con 0x7f2ca0104ea0 2023-07-14T16:14:33.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.368+0000 7f2c9e59c700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c880a11e0 0x7f2c880a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009c60 tx=0x7f2c90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.368+0000 7f2c9dd9b700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] conn(0x7f2c880aad10 0x7f2c880ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:33.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.369+0000 7f2c9dd9b700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] conn(0x7f2c880aad10 0x7f2c880ad1a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:33.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.370+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 <== osd.2 v2:172.21.15.164:6800/3167529694 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2c880ad870 con 0x7f2c880aad10 2023-07-14T16:14:33.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.386+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 --> [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2ca004e210 con 0x7f2c880aad10 2023-07-14T16:14:33.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.387+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/2170780400 <== osd.2 v2:172.21.15.164:6800/3167529694 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2ca004e210 con 0x7f2c880aad10 2023-07-14T16:14:33.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.387+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 >> [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] conn(0x7f2c880aad10 msgr2=0x7f2c880ad1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.387+0000 7f2ca4d20700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] conn(0x7f2c880aad10 0x7f2c880ad1a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:33.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c880a11e0 msgr2=0x7f2c880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2ca4d20700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c880a11e0 0x7f2c880a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009c60 tx=0x7f2c90009260 comp rx=0 tx=0).stop 2023-07-14T16:14:33.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca0104ea0 msgr2=0x7f2ca00732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:33.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2ca4d20700 1 --2- 172.21.15.62:0/2170780400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ca0104ea0 0x7f2ca00732d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b970 tx=0x7f2c9800bc80 comp rx=0 tx=0).stop 2023-07-14T16:14:33.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2c9ed9d700 1 -- 172.21.15.62:0/2170780400 reap_dead start 2023-07-14T16:14:33.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 shutdown_connections 2023-07-14T16:14:33.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 >> 172.21.15.62:0/2170780400 conn(0x7f2ca00fb660 msgr2=0x7f2ca00fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:33.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 shutdown_connections 2023-07-14T16:14:33.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:33.388+0000 7f2ca4d20700 1 -- 172.21.15.62:0/2170780400 wait complete. 2023-07-14T16:14:33.919 INFO:teuthology.orchestra.run.smithi062.stdout:146028888122 2023-07-14T16:14:33.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-14T16:14:34.006 INFO:teuthology.orchestra.run.smithi062.stdout:223338299434 2023-07-14T16:14:34.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-14T16:14:34.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.286+0000 7f652c9a7700 1 -- 172.21.15.62:0/1829402282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65281007a0 msgr2=0x7f6528104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.286+0000 7f652c9a7700 1 --2- 172.21.15.62:0/1829402282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65281007a0 0x7f6528104770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f651c0099c0 tx=0x7f651c009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.287+0000 7f652c9a7700 1 -- 172.21.15.62:0/1829402282 shutdown_connections 2023-07-14T16:14:34.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.287+0000 7f652c9a7700 1 --2- 172.21.15.62:0/1829402282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6528104ea0 0x7f652810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.287+0000 7f652c9a7700 1 --2- 172.21.15.62:0/1829402282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65280ffe00 0x7f65281001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.287+0000 7f652c9a7700 1 --2- 172.21.15.62:0/1829402282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65281007a0 0x7f6528104770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.287+0000 7f652c9a7700 1 -- 172.21.15.62:0/1829402282 >> 172.21.15.62:0/1829402282 conn(0x7f65280fb660 msgr2=0x7f65280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.287+0000 7f652c9a7700 1 -- 172.21.15.62:0/1829402282 shutdown_connections 2023-07-14T16:14:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.287+0000 7f652c9a7700 1 -- 172.21.15.62:0/1829402282 wait complete. 2023-07-14T16:14:34.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.288+0000 7f652c9a7700 1 Processor -- start 2023-07-14T16:14:34.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.288+0000 7f652c9a7700 1 -- start start 2023-07-14T16:14:34.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f652c9a7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65280ffe00 0x7f652806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f652c9a7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65281007a0 0x7f652806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f652c9a7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528104ea0 0x7f652806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f652c9a7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652806af00 con 0x7f6528104ea0 2023-07-14T16:14:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f652c9a7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652806b040 con 0x7f65280ffe00 2023-07-14T16:14:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f652c9a7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652806b200 con 0x7f65281007a0 2023-07-14T16:14:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f6525d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65281007a0 0x7f652806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f6525d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65281007a0 0x7f652806fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:36478/0 (socket says 172.21.15.62:36478) 2023-07-14T16:14:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f6525d9b700 1 -- 172.21.15.62:0/962336976 learned_addr learned my addr 172.21.15.62:0/962336976 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.289+0000 7f6526d9d700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528104ea0 0x7f652806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.290+0000 7f652659c700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65280ffe00 0x7f652806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.290+0000 7f6526d9d700 1 -- 172.21.15.62:0/962336976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65281007a0 msgr2=0x7f652806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.290+0000 7f6526d9d700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65281007a0 0x7f652806fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.290+0000 7f6526d9d700 1 -- 172.21.15.62:0/962336976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65280ffe00 msgr2=0x7f652806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.290+0000 7f6526d9d700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65280ffe00 0x7f652806f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.290+0000 7f6526d9d700 1 -- 172.21.15.62:0/962336976 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0095f0 con 0x7f6528104ea0 2023-07-14T16:14:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.290+0000 7f6526d9d700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528104ea0 0x7f652806a8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6520009460 tx=0x7f652000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.291+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520004db0 con 0x7f6528104ea0 2023-07-14T16:14:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.291+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6520016070 con 0x7f6528104ea0 2023-07-14T16:14:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.291+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652806b4e0 con 0x7f6528104ea0 2023-07-14T16:14:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.291+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520012e80 con 0x7f6528104ea0 2023-07-14T16:14:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.291+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652806e630 con 0x7f6528104ea0 2023-07-14T16:14:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.292+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f652804e210 con 0x7f6528104ea0 2023-07-14T16:14:34.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.295+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f652000db20 con 0x7f6528104ea0 2023-07-14T16:14:34.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.298+0000 7f65177fe700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f65100a1260 0x7f65100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.298+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6520018070 con 0x7f6528104ea0 2023-07-14T16:14:34.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.298+0000 7f652659c700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f65100a1260 0x7f65100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.299+0000 7f65177fe700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] conn(0x7f65100aad90 0x7f65100ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.299+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 --> [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65100ad8f0 con 0x7f65100aad90 2023-07-14T16:14:34.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.299+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65200d07c0 con 0x7f6528104ea0 2023-07-14T16:14:34.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.299+0000 7f6525d9b700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] conn(0x7f65100aad90 0x7f65100ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.299+0000 7f6525d9b700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] conn(0x7f65100aad90 0x7f65100ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:34.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.299+0000 7f652659c700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f65100a1260 0x7f65100a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6518009bf0 tx=0x7f6518009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:34.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.302+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 <== osd.3 v2:172.21.15.62:6810/1003536176 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f65100ad8f0 con 0x7f65100aad90 2023-07-14T16:14:34.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.324+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 --> [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f652810fc40 con 0x7f65100aad90 2023-07-14T16:14:34.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.325+0000 7f65177fe700 1 -- 172.21.15.62:0/962336976 <== osd.3 v2:172.21.15.62:6810/1003536176 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f651c01e040 con 0x7f65100aad90 2023-07-14T16:14:34.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.325+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] conn(0x7f65100aad90 msgr2=0x7f65100ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.325+0000 7f652c9a7700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] conn(0x7f65100aad90 0x7f65100ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f65100a1260 msgr2=0x7f65100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f652c9a7700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f65100a1260 0x7f65100a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6518009bf0 tx=0x7f6518009260 comp rx=0 tx=0).stop 2023-07-14T16:14:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528104ea0 msgr2=0x7f652806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f652c9a7700 1 --2- 172.21.15.62:0/962336976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528104ea0 0x7f652806a8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6520009460 tx=0x7f652000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f6526d9d700 1 -- 172.21.15.62:0/962336976 reap_dead start 2023-07-14T16:14:34.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 shutdown_connections 2023-07-14T16:14:34.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 >> 172.21.15.62:0/962336976 conn(0x7f65280fb660 msgr2=0x7f65280fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:34.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 shutdown_connections 2023-07-14T16:14:34.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:34.326+0000 7f652c9a7700 1 -- 172.21.15.62:0/962336976 wait complete. 2023-07-14T16:14:34.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/2999115557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e410c3e0 msgr2=0x7fd9e410e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 --2- 172.21.15.62:0/2999115557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e410c3e0 0x7fd9e410e7c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b0a0 tx=0x7fd9dc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/2999115557 shutdown_connections 2023-07-14T16:14:34.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 --2- 172.21.15.62:0/2999115557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e410c3e0 0x7fd9e410e7c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 --2- 172.21.15.62:0/2999115557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9e40728b0 0x7fd9e4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 --2- 172.21.15.62:0/2999115557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e40731c0 0x7fd9e410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/2999115557 >> 172.21.15.62:0/2999115557 conn(0x7fd9e40fb660 msgr2=0x7fd9e40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:34.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/2999115557 shutdown_connections 2023-07-14T16:14:34.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.722+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/2999115557 wait complete. 2023-07-14T16:14:34.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9ea7dc700 1 Processor -- start 2023-07-14T16:14:34.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9ea7dc700 1 -- start start 2023-07-14T16:14:34.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9ea7dc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e40728b0 0x7fd9e4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9ea7dc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e40731c0 0x7fd9e41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9ea7dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9e410c3e0 0x7fd9e419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9ea7dc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e419cee0 con 0x7fd9e40731c0 2023-07-14T16:14:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9ea7dc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e419d050 con 0x7fd9e410c3e0 2023-07-14T16:14:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9ea7dc700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e419d210 con 0x7fd9e40728b0 2023-07-14T16:14:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9e37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e40731c0 0x7fd9e41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9e37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e40731c0 0x7fd9e41983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46186/0 (socket says 172.21.15.62:46186) 2023-07-14T16:14:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.724+0000 7fd9e37fe700 1 -- 172.21.15.62:0/935149669 learned_addr learned my addr 172.21.15.62:0/935149669 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e3fff700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e40728b0 0x7fd9e4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e8d79700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9e410c3e0 0x7fd9e419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e37fe700 1 -- 172.21.15.62:0/935149669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e40728b0 msgr2=0x7fd9e4197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e37fe700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e40728b0 0x7fd9e4197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e37fe700 1 -- 172.21.15.62:0/935149669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9e410c3e0 msgr2=0x7fd9e419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e37fe700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9e410c3e0 0x7fd9e419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e37fe700 1 -- 172.21.15.62:0/935149669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc009d00 con 0x7fd9e40731c0 2023-07-14T16:14:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e37fe700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e40731c0 0x7fd9e41983d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800b530 tx=0x7fd9d800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011830 con 0x7fd9e40731c0 2023-07-14T16:14:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9d8011e20 con 0x7fd9e40731c0 2023-07-14T16:14:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.725+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e419d4f0 con 0x7fd9e40731c0 2023-07-14T16:14:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.726+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8010db0 con 0x7fd9e40731c0 2023-07-14T16:14:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.726+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e41a4d10 con 0x7fd9e40731c0 2023-07-14T16:14:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.727+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9e404e210 con 0x7fd9e40731c0 2023-07-14T16:14:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.728+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd9d8011990 con 0x7fd9e40731c0 2023-07-14T16:14:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.731+0000 7fd9e17fa700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9cc0a1260 0x7fd9cc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.731+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9d80cc350 con 0x7fd9e40731c0 2023-07-14T16:14:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.731+0000 7fd9e3fff700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9cc0a1260 0x7fd9cc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.731+0000 7fd9e17fa700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] conn(0x7fd9cc0aad90 0x7fd9cc0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.731+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 --> [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9cc0ad8f0 con 0x7fd9cc0aad90 2023-07-14T16:14:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.731+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d80ce930 con 0x7fd9e40731c0 2023-07-14T16:14:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.731+0000 7fd9e8d79700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] conn(0x7fd9cc0aad90 0x7fd9cc0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.732+0000 7fd9e3fff700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9cc0a1260 0x7fd9cc0a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005950 tx=0x7fd9d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.732+0000 7fd9e8d79700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] conn(0x7fd9cc0aad90 0x7fd9cc0ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.733+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 <== osd.6 v2:172.21.15.62:6818/348099216 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd9cc0ad8f0 con 0x7fd9cc0aad90 2023-07-14T16:14:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.753+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 --> [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9e4111ea0 con 0x7fd9cc0aad90 2023-07-14T16:14:34.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.753+0000 7fd9e17fa700 1 -- 172.21.15.62:0/935149669 <== osd.6 v2:172.21.15.62:6818/348099216 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd9dc009b80 con 0x7fd9cc0aad90 2023-07-14T16:14:34.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] conn(0x7fd9cc0aad90 msgr2=0x7fd9cc0ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] conn(0x7fd9cc0aad90 0x7fd9cc0ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:34.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9cc0a1260 msgr2=0x7fd9cc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9cc0a1260 0x7fd9cc0a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4005950 tx=0x7fd9d400b410 comp rx=0 tx=0).stop 2023-07-14T16:14:34.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e40731c0 msgr2=0x7fd9e41983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:34.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 --2- 172.21.15.62:0/935149669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e40731c0 0x7fd9e41983d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800b530 tx=0x7fd9d800b840 comp rx=0 tx=0).stop 2023-07-14T16:14:34.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9e8d79700 1 -- 172.21.15.62:0/935149669 reap_dead start 2023-07-14T16:14:34.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 shutdown_connections 2023-07-14T16:14:34.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 >> 172.21.15.62:0/935149669 conn(0x7fd9e40fb660 msgr2=0x7fd9e410e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:34.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 shutdown_connections 2023-07-14T16:14:34.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:32.754+0000 7fd9ea7dc700 1 -- 172.21.15.62:0/935149669 wait complete. 2023-07-14T16:14:34.844 INFO:teuthology.orchestra.run.smithi062.stdout:313532612633 2023-07-14T16:14:34.844 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-14T16:14:34.845 INFO:teuthology.orchestra.run.smithi062.stdout:34359738450 2023-07-14T16:14:34.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-14T16:14:34.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:34 smithi062 ceph-mon[95543]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:35.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:34 smithi164 ceph-mon[97427]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:34 smithi081 ceph-mon[127795]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:35.227 INFO:teuthology.orchestra.run.smithi062.stdout:292057776160 2023-07-14T16:14:35.228 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-14T16:14:35.608 INFO:teuthology.orchestra.run.smithi062.stdout:77309411397 2023-07-14T16:14:35.609 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-14T16:14:35.707 INFO:teuthology.orchestra.run.smithi062.stdout:111669149761 2023-07-14T16:14:35.707 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-14T16:14:36.200 INFO:teuthology.orchestra.run.smithi062.stdout:201863462961 2023-07-14T16:14:36.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-14T16:14:36.274 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:36.306 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:36.350 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:36.817 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:36.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:36 smithi062 ceph-mon[95543]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:37.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:36 smithi164 ceph-mon[97427]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:37.118 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:37.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:36 smithi081 ceph-mon[127795]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T16:14:37.189 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:37.189 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:38.469 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:38.508 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:39.017 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:38 smithi062 ceph-mon[95543]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T16:14:39.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:38 smithi164 ceph-mon[97427]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T16:14:39.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:38 smithi081 ceph-mon[127795]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-14T16:14:40.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.883+0000 7fad0ddab700 1 -- 172.21.15.62:0/3768298311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080731c0 msgr2=0x7fad0810bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:40.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.883+0000 7fad0ddab700 1 --2- 172.21.15.62:0/3768298311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080731c0 0x7fad0810bea0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7facfc009990 tx=0x7facfc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:40.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.883+0000 7fad0ddab700 1 -- 172.21.15.62:0/3768298311 shutdown_connections 2023-07-14T16:14:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.883+0000 7fad0ddab700 1 --2- 172.21.15.62:0/3768298311 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad0810c3e0 0x7fad0810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.883+0000 7fad0ddab700 1 --2- 172.21.15.62:0/3768298311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad080728b0 0x7fad08072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.883+0000 7fad0ddab700 1 --2- 172.21.15.62:0/3768298311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080731c0 0x7fad0810bea0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.883+0000 7fad0ddab700 1 -- 172.21.15.62:0/3768298311 >> 172.21.15.62:0/3768298311 conn(0x7fad080fb660 msgr2=0x7fad080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.884+0000 7fad0ddab700 1 -- 172.21.15.62:0/3768298311 shutdown_connections 2023-07-14T16:14:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.884+0000 7fad0ddab700 1 -- 172.21.15.62:0/3768298311 wait complete. 2023-07-14T16:14:40.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.885+0000 7fad0ddab700 1 Processor -- start 2023-07-14T16:14:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.885+0000 7fad0ddab700 1 -- start start 2023-07-14T16:14:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.885+0000 7fad0ddab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080728b0 0x7fad0819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.885+0000 7fad0ddab700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad080731c0 0x7fad0819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.885+0000 7fad0ddab700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad0810c3e0 0x7fad08197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.885+0000 7fad0ddab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad081982a0 con 0x7fad080728b0 2023-07-14T16:14:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.885+0000 7fad0ddab700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08198410 con 0x7fad080731c0 2023-07-14T16:14:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.885+0000 7fad0ddab700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad081985d0 con 0x7fad0810c3e0 2023-07-14T16:14:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080728b0 0x7fad0819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080728b0 0x7fad0819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48438/0 (socket says 172.21.15.62:48438) 2023-07-14T16:14:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad06ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad080731c0 0x7fad0819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad077fe700 1 -- 172.21.15.62:0/2931985089 learned_addr learned my addr 172.21.15.62:0/2931985089 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad07fff700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad0810c3e0 0x7fad08197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad06ffd700 1 -- 172.21.15.62:0/2931985089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad0810c3e0 msgr2=0x7fad08197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad06ffd700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad0810c3e0 0x7fad08197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad06ffd700 1 -- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080728b0 msgr2=0x7fad0819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad06ffd700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080728b0 0x7fad0819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad06ffd700 1 -- 172.21.15.62:0/2931985089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0095f0 con 0x7fad080731c0 2023-07-14T16:14:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.886+0000 7fad06ffd700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad080731c0 0x7fad0819e010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7facfc000c00 tx=0x7facfc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.887+0000 7fad04ff9700 1 -- 172.21.15.62:0/2931985089 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc01d070 con 0x7fad080731c0 2023-07-14T16:14:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.887+0000 7fad04ff9700 1 -- 172.21.15.62:0/2931985089 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facfc003ec0 con 0x7fad080731c0 2023-07-14T16:14:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.887+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad08198850 con 0x7fad080731c0 2023-07-14T16:14:40.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.887+0000 7fad04ff9700 1 -- 172.21.15.62:0/2931985089 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc022b20 con 0x7fad080731c0 2023-07-14T16:14:40.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.887+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad081a4ac0 con 0x7fad080731c0 2023-07-14T16:14:40.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.888+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0804e210 con 0x7fad080731c0 2023-07-14T16:14:40.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.890+0000 7fad04ff9700 1 -- 172.21.15.62:0/2931985089 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7facfc004030 con 0x7fad080731c0 2023-07-14T16:14:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.891+0000 7fad04ff9700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7facf00a1260 0x7facf00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.892+0000 7fad04ff9700 1 -- 172.21.15.62:0/2931985089 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7facfc0cdfd0 con 0x7fad080731c0 2023-07-14T16:14:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.892+0000 7fad077fe700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7facf00a1260 0x7facf00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.893+0000 7fad077fe700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7facf00a1260 0x7facf00a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7facf8005950 tx=0x7facf800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:40.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:40.896+0000 7fad04ff9700 1 -- 172.21.15.62:0/2931985089 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7facfc097600 con 0x7fad080731c0 2023-07-14T16:14:40.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:40 smithi062 ceph-mon[95543]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-07-14T16:14:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:40 smithi164 ceph-mon[97427]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-07-14T16:14:41.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.164+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fad081a4d50 con 0x7fad080731c0 2023-07-14T16:14:41.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.165+0000 7fad04ff9700 1 -- 172.21.15.62:0/2931985089 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7facfc003a30 con 0x7fad080731c0 2023-07-14T16:14:41.145 INFO:teuthology.orchestra.run.smithi062.stdout:347892350998 2023-07-14T16:14:41.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7facf00a1260 msgr2=0x7facf00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7facf00a1260 0x7facf00a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7facf8005950 tx=0x7facf800b500 comp rx=0 tx=0).stop 2023-07-14T16:14:41.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad080731c0 msgr2=0x7fad0819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad080731c0 0x7fad0819e010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7facfc000c00 tx=0x7facfc009580 comp rx=0 tx=0).stop 2023-07-14T16:14:41.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 shutdown_connections 2023-07-14T16:14:41.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad0810c3e0 0x7fad08197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad080728b0 0x7fad0819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7facf00a1260 0x7facf00a3710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 --2- 172.21.15.62:0/2931985089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad080731c0 0x7fad0819e010 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 >> 172.21.15.62:0/2931985089 conn(0x7fad080fb660 msgr2=0x7fad0810e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:41.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 shutdown_connections 2023-07-14T16:14:41.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.170+0000 7fad0ddab700 1 -- 172.21.15.62:0/2931985089 wait complete. 2023-07-14T16:14:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:40 smithi081 ceph-mon[127795]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-07-14T16:14:41.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:41 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2931985089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T16:14:41.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:41 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1530646939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T16:14:41.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:41 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1510935649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T16:14:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 -- 172.21.15.62:0/2891870632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f70100df0 msgr2=0x7f4f701011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 --2- 172.21.15.62:0/2891870632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f70100df0 0x7f4f701011c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009990 tx=0x7f4f60009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 -- 172.21.15.62:0/2891870632 shutdown_connections 2023-07-14T16:14:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 --2- 172.21.15.62:0/2891870632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f7010a140 0x7f4f7010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 --2- 172.21.15.62:0/2891870632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f70100df0 0x7f4f701011c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 --2- 172.21.15.62:0/2891870632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f70101700 0x7f4f70109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 -- 172.21.15.62:0/2891870632 >> 172.21.15.62:0/2891870632 conn(0x7f4f70075b70 msgr2=0x7f4f70075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 -- 172.21.15.62:0/2891870632 shutdown_connections 2023-07-14T16:14:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.436+0000 7f4f75439700 1 -- 172.21.15.62:0/2891870632 wait complete. 2023-07-14T16:14:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.437+0000 7f4f75439700 1 Processor -- start 2023-07-14T16:14:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.437+0000 7f4f75439700 1 -- start start 2023-07-14T16:14:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f75439700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f70100df0 0x7f4f70197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f75439700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f70101700 0x7f4f70198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f75439700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7010a140 0x7f4f7019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f75439700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7019cda0 con 0x7f4f70100df0 2023-07-14T16:14:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f75439700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7019cf10 con 0x7f4f7010a140 2023-07-14T16:14:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f75439700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7019d0d0 con 0x7f4f70101700 2023-07-14T16:14:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f6effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f70100df0 0x7f4f70197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f6effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f70100df0 0x7f4f70197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48464/0 (socket says 172.21.15.62:48464) 2023-07-14T16:14:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f6effd700 1 -- 172.21.15.62:0/1510935649 learned_addr learned my addr 172.21.15.62:0/1510935649 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f6e7fc700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f70101700 0x7f4f70198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.438+0000 7f4f6f7fe700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7010a140 0x7f4f7019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f6effd700 1 -- 172.21.15.62:0/1510935649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f70101700 msgr2=0x7f4f70198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f6effd700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f70101700 0x7f4f70198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f6effd700 1 -- 172.21.15.62:0/1510935649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7010a140 msgr2=0x7f4f7019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f6effd700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7010a140 0x7f4f7019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f6effd700 1 -- 172.21.15.62:0/1510935649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68009c40 con 0x7f4f70100df0 2023-07-14T16:14:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f6effd700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f70100df0 0x7f4f70197de0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f60005770 tx=0x7f4f600057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f57fff700 1 -- 172.21.15.62:0/1510935649 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6001d070 con 0x7f4f70100df0 2023-07-14T16:14:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f600095f0 con 0x7f4f70100df0 2023-07-14T16:14:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f57fff700 1 -- 172.21.15.62:0/1510935649 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f60021410 con 0x7f4f70100df0 2023-07-14T16:14:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f57fff700 1 -- 172.21.15.62:0/1510935649 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f600173f0 con 0x7f4f70100df0 2023-07-14T16:14:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.439+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f701a4a40 con 0x7f4f70100df0 2023-07-14T16:14:41.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.441+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7004e210 con 0x7f4f70100df0 2023-07-14T16:14:41.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.444+0000 7f4f57fff700 1 -- 172.21.15.62:0/1510935649 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4f6000fac0 con 0x7f4f70100df0 2023-07-14T16:14:41.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.446+0000 7f4f57fff700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f580a11e0 0x7f4f580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:41.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.446+0000 7f4f57fff700 1 -- 172.21.15.62:0/1510935649 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f600cdbc0 con 0x7f4f70100df0 2023-07-14T16:14:41.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.447+0000 7f4f6e7fc700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f580a11e0 0x7f4f580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:41.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.447+0000 7f4f6e7fc700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f580a11e0 0x7f4f580a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f70199400 tx=0x7f4f64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:41.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.452+0000 7f4f57fff700 1 -- 172.21.15.62:0/1510935649 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4f6005b790 con 0x7f4f70100df0 2023-07-14T16:14:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.705+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4f701a4c30 con 0x7f4f70100df0 2023-07-14T16:14:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.705+0000 7f4f57fff700 1 -- 172.21.15.62:0/1510935649 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f60026090 con 0x7f4f70100df0 2023-07-14T16:14:41.907 INFO:teuthology.orchestra.run.smithi062.stdout:167503724598 2023-07-14T16:14:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f580a11e0 msgr2=0x7f4f580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f580a11e0 0x7f4f580a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f70199400 tx=0x7f4f64009260 comp rx=0 tx=0).stop 2023-07-14T16:14:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f70100df0 msgr2=0x7f4f70197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f70100df0 0x7f4f70197de0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f60005770 tx=0x7f4f600057a0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 shutdown_connections 2023-07-14T16:14:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f580a11e0 0x7f4f580a3690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f7010a140 0x7f4f7019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f70100df0 0x7f4f70197de0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 --2- 172.21.15.62:0/1510935649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f70101700 0x7f4f70198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 >> 172.21.15.62:0/1510935649 conn(0x7f4f70075b70 msgr2=0x7f4f700fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:41.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 shutdown_connections 2023-07-14T16:14:41.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.708+0000 7f4f75439700 1 -- 172.21.15.62:0/1510935649 wait complete. 2023-07-14T16:14:41.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.337+0000 7f9e788a1700 1 -- 172.21.15.62:0/2490212165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e74104ea0 msgr2=0x7f9e7410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.337+0000 7f9e788a1700 1 --2- 172.21.15.62:0/2490212165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e74104ea0 0x7f9e7410ca20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009990 tx=0x7f9e6c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.338+0000 7f9e788a1700 1 -- 172.21.15.62:0/2490212165 shutdown_connections 2023-07-14T16:14:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.338+0000 7f9e788a1700 1 --2- 172.21.15.62:0/2490212165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e74104ea0 0x7f9e7410ca20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.338+0000 7f9e788a1700 1 --2- 172.21.15.62:0/2490212165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e740ffe00 0x7f9e741001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.338+0000 7f9e788a1700 1 --2- 172.21.15.62:0/2490212165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e741007a0 0x7f9e74104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.338+0000 7f9e788a1700 1 -- 172.21.15.62:0/2490212165 >> 172.21.15.62:0/2490212165 conn(0x7f9e740fb660 msgr2=0x7f9e740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.338+0000 7f9e788a1700 1 -- 172.21.15.62:0/2490212165 shutdown_connections 2023-07-14T16:14:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.338+0000 7f9e788a1700 1 -- 172.21.15.62:0/2490212165 wait complete. 2023-07-14T16:14:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.340+0000 7f9e788a1700 1 Processor -- start 2023-07-14T16:14:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.340+0000 7f9e788a1700 1 -- start start 2023-07-14T16:14:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.340+0000 7f9e788a1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e740ffe00 0x7f9e741997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.340+0000 7f9e788a1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e741007a0 0x7f9e74199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.341+0000 7f9e788a1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e74104ea0 0x7f9e74193820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.341+0000 7f9e788a1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e74193e90 con 0x7f9e740ffe00 2023-07-14T16:14:41.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.341+0000 7f9e788a1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e74194000 con 0x7f9e741007a0 2023-07-14T16:14:41.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.341+0000 7f9e788a1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e741941c0 con 0x7f9e74104ea0 2023-07-14T16:14:41.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.341+0000 7f9e72d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e74104ea0 0x7f9e74193820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:41.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.341+0000 7f9e72d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e74104ea0 0x7f9e74193820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:36532/0 (socket says 172.21.15.62:36532) 2023-07-14T16:14:41.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.341+0000 7f9e72d9d700 1 -- 172.21.15.62:0/1530646939 learned_addr learned my addr 172.21.15.62:0/1530646939 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:41.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.342+0000 7f9e7259c700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e740ffe00 0x7f9e741997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:41.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.342+0000 7f9e69d9b700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e741007a0 0x7f9e74199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:41.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.343+0000 7f9e69d9b700 1 -- 172.21.15.62:0/1530646939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e74104ea0 msgr2=0x7f9e74193820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.343+0000 7f9e69d9b700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e74104ea0 0x7f9e74193820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.343+0000 7f9e69d9b700 1 -- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e740ffe00 msgr2=0x7f9e741997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.343+0000 7f9e69d9b700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e740ffe00 0x7f9e741997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.343+0000 7f9e69d9b700 1 -- 172.21.15.62:0/1530646939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0095f0 con 0x7f9e741007a0 2023-07-14T16:14:41.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.347+0000 7f9e69d9b700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e741007a0 0x7f9e74199ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c01c530 tx=0x7f9e5c01c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:41.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.347+0000 7f9e6bfff700 1 -- 172.21.15.62:0/1530646939 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c022640 con 0x7f9e741007a0 2023-07-14T16:14:41.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.347+0000 7f9e6bfff700 1 -- 172.21.15.62:0/1530646939 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e5c022c30 con 0x7f9e741007a0 2023-07-14T16:14:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.347+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e741944a0 con 0x7f9e741007a0 2023-07-14T16:14:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.347+0000 7f9e6bfff700 1 -- 172.21.15.62:0/1530646939 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c021b70 con 0x7f9e741007a0 2023-07-14T16:14:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.348+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e741a0790 con 0x7f9e741007a0 2023-07-14T16:14:41.938 INFO:teuthology.orchestra.run.smithi062.stdout:223338299435 2023-07-14T16:14:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.350+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7404e210 con 0x7f9e741007a0 2023-07-14T16:14:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.354+0000 7f9e6bfff700 1 -- 172.21.15.62:0/1530646939 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9e5c021cd0 con 0x7f9e741007a0 2023-07-14T16:14:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.357+0000 7f9e6bfff700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1390 0x7f9e600a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.357+0000 7f9e6bfff700 1 -- 172.21.15.62:0/1530646939 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e5c0dd090 con 0x7f9e741007a0 2023-07-14T16:14:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.358+0000 7f9e7259c700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1390 0x7f9e600a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.359+0000 7f9e6bfff700 1 -- 172.21.15.62:0/1530646939 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9e5c0a66b0 con 0x7f9e741007a0 2023-07-14T16:14:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.370+0000 7f9e7259c700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1390 0x7f9e600a3840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009550 tx=0x7f9e640091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.615+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9e74194d30 con 0x7f9e741007a0 2023-07-14T16:14:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.619+0000 7f9e6bfff700 1 -- 172.21.15.62:0/1530646939 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e5c026020 con 0x7f9e741007a0 2023-07-14T16:14:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.622+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1390 msgr2=0x7f9e600a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.622+0000 7f9e788a1700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1390 0x7f9e600a3840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009550 tx=0x7f9e640091b0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.622+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e741007a0 msgr2=0x7f9e74199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.622+0000 7f9e788a1700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e741007a0 0x7f9e74199ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c01c530 tx=0x7f9e5c01c840 comp rx=0 tx=0).stop 2023-07-14T16:14:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.622+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 shutdown_connections 2023-07-14T16:14:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.622+0000 7f9e788a1700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e74104ea0 0x7f9e74193820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.622+0000 7f9e788a1700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e740ffe00 0x7f9e741997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.623+0000 7f9e788a1700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1390 0x7f9e600a3840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.623+0000 7f9e788a1700 1 --2- 172.21.15.62:0/1530646939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e741007a0 0x7f9e74199ce0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.623+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 >> 172.21.15.62:0/1530646939 conn(0x7f9e740fb660 msgr2=0x7f9e740fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.623+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 shutdown_connections 2023-07-14T16:14:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:41.623+0000 7f9e788a1700 1 -- 172.21.15.62:0/1530646939 wait complete. 2023-07-14T16:14:42.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:41 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2931985089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T16:14:42.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:41 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1530646939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T16:14:42.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:41 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1510935649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T16:14:42.035 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350996 got 347892350998 for osd.11 2023-07-14T16:14:42.036 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:42.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:41 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2931985089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T16:14:42.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:41 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1530646939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T16:14:42.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:41 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1510935649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T16:14:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 -- 172.21.15.62:0/1727866714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900fe980 msgr2=0x7fa6900fed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 --2- 172.21.15.62:0/1727866714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900fe980 0x7fa6900fed50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa680009990 tx=0x7fa680009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 -- 172.21.15.62:0/1727866714 shutdown_connections 2023-07-14T16:14:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 --2- 172.21.15.62:0/1727866714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa69010c530 0x7fa69010e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 --2- 172.21.15.62:0/1727866714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900fe980 0x7fa6900fed50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 --2- 172.21.15.62:0/1727866714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6900ff290 0x7fa69010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 -- 172.21.15.62:0/1727866714 >> 172.21.15.62:0/1727866714 conn(0x7fa690075b70 msgr2=0x7fa690075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 -- 172.21.15.62:0/1727866714 shutdown_connections 2023-07-14T16:14:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.827+0000 7fa695c58700 1 -- 172.21.15.62:0/1727866714 wait complete. 2023-07-14T16:14:42.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.828+0000 7fa695c58700 1 Processor -- start 2023-07-14T16:14:42.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.829+0000 7fa695c58700 1 -- start start 2023-07-14T16:14:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.829+0000 7fa695c58700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6900fe980 0x7fa690197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.829+0000 7fa695c58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900ff290 0x7fa6901982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.829+0000 7fa695c58700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa69010c530 0x7fa69019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.829+0000 7fa695c58700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69019cd50 con 0x7fa6900ff290 2023-07-14T16:14:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.829+0000 7fa695c58700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69019cec0 con 0x7fa6900fe980 2023-07-14T16:14:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.829+0000 7fa695c58700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69019d080 con 0x7fa69010c530 2023-07-14T16:14:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.829+0000 7fa68effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900ff290 0x7fa6901982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900ff290 0x7fa6901982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57762/0 (socket says 172.21.15.62:57762) 2023-07-14T16:14:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68effd700 1 -- 172.21.15.62:0/1354702882 learned_addr learned my addr 172.21.15.62:0/1354702882 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68f7fe700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6900fe980 0x7fa690197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68ffff700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa69010c530 0x7fa69019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68effd700 1 -- 172.21.15.62:0/1354702882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa69010c530 msgr2=0x7fa69019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68effd700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa69010c530 0x7fa69019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68effd700 1 -- 172.21.15.62:0/1354702882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6900fe980 msgr2=0x7fa690197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68effd700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6900fe980 0x7fa690197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68effd700 1 -- 172.21.15.62:0/1354702882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6800095f0 con 0x7fa6900ff290 2023-07-14T16:14:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68effd700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900ff290 0x7fa6901982d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c9b0 tx=0x7fa68400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.830+0000 7fa68cff9700 1 -- 172.21.15.62:0/1354702882 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684007b00 con 0x7fa6900ff290 2023-07-14T16:14:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.831+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69019d360 con 0x7fa6900ff290 2023-07-14T16:14:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.831+0000 7fa68cff9700 1 -- 172.21.15.62:0/1354702882 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa684007c60 con 0x7fa6900ff290 2023-07-14T16:14:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.831+0000 7fa68cff9700 1 -- 172.21.15.62:0/1354702882 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684018660 con 0x7fa6900ff290 2023-07-14T16:14:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.831+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6901a4bc0 con 0x7fa6900ff290 2023-07-14T16:14:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.832+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69004e210 con 0x7fa6900ff290 2023-07-14T16:14:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.834+0000 7fa68cff9700 1 -- 172.21.15.62:0/1354702882 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa68400f450 con 0x7fa6900ff290 2023-07-14T16:14:42.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.836+0000 7fa68cff9700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa6780a1260 0x7fa6780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:42.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.836+0000 7fa68cff9700 1 -- 172.21.15.62:0/1354702882 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa6840ccdb0 con 0x7fa6900ff290 2023-07-14T16:14:42.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.836+0000 7fa68f7fe700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa6780a1260 0x7fa6780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:42.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.837+0000 7fa68f7fe700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa6780a1260 0x7fa6780a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b5c0 tx=0x7fa680009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:42.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:42.838+0000 7fa68cff9700 1 -- 172.21.15.62:0/1354702882 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa6840963d0 con 0x7fa6900ff290 2023-07-14T16:14:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:42 smithi164 ceph-mon[97427]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:43.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.053+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa6901990f0 con 0x7fa6900ff290 2023-07-14T16:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.053+0000 7fa68cff9700 1 -- 172.21.15.62:0/1354702882 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa68400fe60 con 0x7fa6900ff290 2023-07-14T16:14:43.032 INFO:teuthology.orchestra.run.smithi062.stdout:77309411399 2023-07-14T16:14:43.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa6780a1260 msgr2=0x7fa6780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:43.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa6780a1260 0x7fa6780a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b5c0 tx=0x7fa680009f00 comp rx=0 tx=0).stop 2023-07-14T16:14:43.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900ff290 msgr2=0x7fa6901982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:43.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900ff290 0x7fa6901982d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c9b0 tx=0x7fa68400ccc0 comp rx=0 tx=0).stop 2023-07-14T16:14:43.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 shutdown_connections 2023-07-14T16:14:43.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa69010c530 0x7fa69019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:43.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa6780a1260 0x7fa6780a3710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:43.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6900fe980 0x7fa690197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:43.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 --2- 172.21.15.62:0/1354702882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6900ff290 0x7fa6901982d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:43.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 >> 172.21.15.62:0/1354702882 conn(0x7fa690075b70 msgr2=0x7fa6900fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:43.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 shutdown_connections 2023-07-14T16:14:43.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:43.056+0000 7fa695c58700 1 -- 172.21.15.62:0/1354702882 wait complete. 2023-07-14T16:14:43.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:42 smithi062 ceph-mon[95543]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:43.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:42 smithi081 ceph-mon[127795]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:43.659 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299435 for osd.7 2023-07-14T16:14:43.660 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:43.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:43 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1354702882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T16:14:44.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:43 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1354702882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T16:14:44.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724598 for osd.5 2023-07-14T16:14:44.143 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:44.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:43 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1354702882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T16:14:44.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 -- 172.21.15.62:0/837751052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 msgr2=0x7f615c10e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:44.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 --2- 172.21.15.62:0/837751052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 0x7f615c10e7c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f615400b0a0 tx=0x7f615400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 -- 172.21.15.62:0/837751052 shutdown_connections 2023-07-14T16:14:44.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 --2- 172.21.15.62:0/837751052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 0x7f615c10e7c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 --2- 172.21.15.62:0/837751052 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615c0728b0 0x7f615c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 --2- 172.21.15.62:0/837751052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f615c0731c0 0x7f615c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 -- 172.21.15.62:0/837751052 >> 172.21.15.62:0/837751052 conn(0x7f615c0fb660 msgr2=0x7f615c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:44.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 -- 172.21.15.62:0/837751052 shutdown_connections 2023-07-14T16:14:44.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.425+0000 7f61629f6700 1 -- 172.21.15.62:0/837751052 wait complete. 2023-07-14T16:14:44.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.426+0000 7f61629f6700 1 Processor -- start 2023-07-14T16:14:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.426+0000 7f61629f6700 1 -- start start 2023-07-14T16:14:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f61629f6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615c0728b0 0x7f615c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f61629f6700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f615c0731c0 0x7f615c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f61629f6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 0x7f615c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f61629f6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c19cdd0 con 0x7f615c10c3e0 2023-07-14T16:14:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f61629f6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c19cf40 con 0x7f615c0728b0 2023-07-14T16:14:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f61629f6700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c19d100 con 0x7f615c0731c0 2023-07-14T16:14:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f6160f93700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 0x7f615c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f6160f93700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 0x7f615c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57774/0 (socket says 172.21.15.62:57774) 2023-07-14T16:14:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f6160f93700 1 -- 172.21.15.62:0/3790824339 learned_addr learned my addr 172.21.15.62:0/3790824339 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f615b7fe700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f615c0731c0 0x7f615c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.427+0000 7f615bfff700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615c0728b0 0x7f615c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f6160f93700 1 -- 172.21.15.62:0/3790824339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f615c0731c0 msgr2=0x7f615c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f6160f93700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f615c0731c0 0x7f615c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f6160f93700 1 -- 172.21.15.62:0/3790824339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615c0728b0 msgr2=0x7f615c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f6160f93700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615c0728b0 0x7f615c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f6160f93700 1 -- 172.21.15.62:0/3790824339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6154009d00 con 0x7f615c10c3e0 2023-07-14T16:14:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f615bfff700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615c0728b0 0x7f615c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f6160f93700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 0x7f615c19c7c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6154009fd0 tx=0x7f6154004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f61597fa700 1 -- 172.21.15.62:0/3790824339 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615400e040 con 0x7f615c10c3e0 2023-07-14T16:14:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f61597fa700 1 -- 172.21.15.62:0/3790824339 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f615400ceb0 con 0x7f615c10c3e0 2023-07-14T16:14:44.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f61597fa700 1 -- 172.21.15.62:0/3790824339 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154005680 con 0x7f615c10c3e0 2023-07-14T16:14:44.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c19d380 con 0x7f615c10c3e0 2023-07-14T16:14:44.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.428+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c1a4c40 con 0x7f615c10c3e0 2023-07-14T16:14:44.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.430+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c04e210 con 0x7f615c10c3e0 2023-07-14T16:14:44.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.431+0000 7f61597fa700 1 -- 172.21.15.62:0/3790824339 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f615401bd60 con 0x7f615c10c3e0 2023-07-14T16:14:44.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.433+0000 7f61597fa700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f61440a1260 0x7f61440a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:44.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.433+0000 7f61597fa700 1 -- 172.21.15.62:0/3790824339 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f61540ce320 con 0x7f615c10c3e0 2023-07-14T16:14:44.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.433+0000 7f615bfff700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f61440a1260 0x7f61440a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:44.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.434+0000 7f615bfff700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f61440a1260 0x7f61440a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f614c005950 tx=0x7f614c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.437+0000 7f61597fa700 1 -- 172.21.15.62:0/3790824339 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6154097950 con 0x7f615c10c3e0 2023-07-14T16:14:44.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.693+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f615c199170 con 0x7f615c10c3e0 2023-07-14T16:14:44.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.694+0000 7f61597fa700 1 -- 172.21.15.62:0/3790824339 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f615409af70 con 0x7f615c10c3e0 2023-07-14T16:14:44.673 INFO:teuthology.orchestra.run.smithi062.stdout:292057776162 2023-07-14T16:14:44.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f61440a1260 msgr2=0x7f61440a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:44.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f61440a1260 0x7f61440a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f614c005950 tx=0x7f614c0058e0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 msgr2=0x7f615c19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:44.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 0x7f615c19c7c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6154009fd0 tx=0x7f6154004c80 comp rx=0 tx=0).stop 2023-07-14T16:14:44.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 shutdown_connections 2023-07-14T16:14:44.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f615c10c3e0 0x7f615c19c7c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f61440a1260 0x7f61440a3710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f615c0728b0 0x7f615c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 --2- 172.21.15.62:0/3790824339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f615c0731c0 0x7f615c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:44.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 >> 172.21.15.62:0/3790824339 conn(0x7f615c0fb660 msgr2=0x7f615c10e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:44.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 shutdown_connections 2023-07-14T16:14:44.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:44.697+0000 7f61629f6700 1 -- 172.21.15.62:0/3790824339 wait complete. 2023-07-14T16:14:44.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:44 smithi062 ceph-mon[95543]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:44.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:44 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3790824339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T16:14:45.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:44 smithi164 ceph-mon[97427]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:45.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:44 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3790824339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T16:14:45.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:44 smithi081 ceph-mon[127795]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:45.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:44 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3790824339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T16:14:45.205 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411399 for osd.2 2023-07-14T16:14:45.205 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:45.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 -- 172.21.15.62:0/292728347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0fe980 msgr2=0x7f929c0fed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 --2- 172.21.15.62:0/292728347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0fe980 0x7f929c0fed50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f928c009990 tx=0x7f928c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 -- 172.21.15.62:0/292728347 shutdown_connections 2023-07-14T16:14:45.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 --2- 172.21.15.62:0/292728347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f929c10c530 0x7f929c10e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 --2- 172.21.15.62:0/292728347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0fe980 0x7f929c0fed50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 --2- 172.21.15.62:0/292728347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f929c0ff290 0x7f929c10bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 -- 172.21.15.62:0/292728347 >> 172.21.15.62:0/292728347 conn(0x7f929c075b70 msgr2=0x7f929c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:45.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 -- 172.21.15.62:0/292728347 shutdown_connections 2023-07-14T16:14:45.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.265+0000 7f92a296c700 1 -- 172.21.15.62:0/292728347 wait complete. 2023-07-14T16:14:45.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.266+0000 7f92a296c700 1 Processor -- start 2023-07-14T16:14:45.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.266+0000 7f92a296c700 1 -- start start 2023-07-14T16:14:45.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f92a296c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f929c0fe980 0x7f929c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f92a296c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0ff290 0x7f929c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f92a296c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f929c10c530 0x7f929c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f92a296c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c19cd80 con 0x7f929c0ff290 2023-07-14T16:14:45.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f92a296c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c19cef0 con 0x7f929c10c530 2023-07-14T16:14:45.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f92a296c700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c19d0b0 con 0x7f929c0fe980 2023-07-14T16:14:45.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0ff290 0x7f929c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0ff290 0x7f929c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57822/0 (socket says 172.21.15.62:57822) 2023-07-14T16:14:45.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929b7fe700 1 -- 172.21.15.62:0/322351453 learned_addr learned my addr 172.21.15.62:0/322351453 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:45.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929bfff700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f929c0fe980 0x7f929c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929b7fe700 1 -- 172.21.15.62:0/322351453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f929c0fe980 msgr2=0x7f929c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929b7fe700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f929c0fe980 0x7f929c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929b7fe700 1 -- 172.21.15.62:0/322351453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f929c10c530 msgr2=0x7f929c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929b7fe700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f929c10c530 0x7f929c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.267+0000 7f929b7fe700 1 -- 172.21.15.62:0/322351453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0095f0 con 0x7f929c0ff290 2023-07-14T16:14:45.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.268+0000 7f929b7fe700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0ff290 0x7f929c198300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f929000c9b0 tx=0x7f929000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:45.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.268+0000 7f92997fa700 1 -- 172.21.15.62:0/322351453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290007b00 con 0x7f929c0ff290 2023-07-14T16:14:45.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.268+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c19d390 con 0x7f929c0ff290 2023-07-14T16:14:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.268+0000 7f92997fa700 1 -- 172.21.15.62:0/322351453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9290007c60 con 0x7f929c0ff290 2023-07-14T16:14:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.268+0000 7f92997fa700 1 -- 172.21.15.62:0/322351453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290018660 con 0x7f929c0ff290 2023-07-14T16:14:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.268+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c1a4bf0 con 0x7f929c0ff290 2023-07-14T16:14:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.269+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c04e210 con 0x7f929c0ff290 2023-07-14T16:14:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.271+0000 7f92997fa700 1 -- 172.21.15.62:0/322351453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f929000f450 con 0x7f929c0ff290 2023-07-14T16:14:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.272+0000 7f92997fa700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f92840a1260 0x7f92840a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.273+0000 7f92997fa700 1 -- 172.21.15.62:0/322351453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f92900cca70 con 0x7f929c0ff290 2023-07-14T16:14:45.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.273+0000 7f929bfff700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f92840a1260 0x7f92840a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.273+0000 7f929bfff700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f92840a1260 0x7f92840a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f928c00b5c0 tx=0x7f928c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:45.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.276+0000 7f92997fa700 1 -- 172.21.15.62:0/322351453 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9290096010 con 0x7f929c0ff290 2023-07-14T16:14:45.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.114+0000 7f18d8e82700 1 -- 172.21.15.62:0/2728575208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d410c3e0 msgr2=0x7f18d410e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.114+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2728575208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d410c3e0 0x7f18d410e7c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00b0a0 tx=0x7f18cc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.114+0000 7f18d8e82700 1 -- 172.21.15.62:0/2728575208 shutdown_connections 2023-07-14T16:14:45.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.114+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2728575208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d410c3e0 0x7f18d410e7c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.114+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2728575208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d40728b0 0x7f18d4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.114+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2728575208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18d40731c0 0x7f18d410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.114+0000 7f18d8e82700 1 -- 172.21.15.62:0/2728575208 >> 172.21.15.62:0/2728575208 conn(0x7f18d40fb660 msgr2=0x7f18d40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.115+0000 7f18d8e82700 1 -- 172.21.15.62:0/2728575208 shutdown_connections 2023-07-14T16:14:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.115+0000 7f18d8e82700 1 -- 172.21.15.62:0/2728575208 wait complete. 2023-07-14T16:14:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.116+0000 7f18d8e82700 1 Processor -- start 2023-07-14T16:14:45.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.116+0000 7f18d8e82700 1 -- start start 2023-07-14T16:14:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d8e82700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18d40728b0 0x7f18d4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d8e82700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d40731c0 0x7f18d4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d8e82700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d410c3e0 0x7f18d419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d8e82700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419ce50 con 0x7f18d40731c0 2023-07-14T16:14:45.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d8e82700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419cfc0 con 0x7f18d410c3e0 2023-07-14T16:14:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d8e82700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d419d180 con 0x7f18d40728b0 2023-07-14T16:14:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18d40728b0 0x7f18d4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18d40728b0 0x7f18d4197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34864/0 (socket says 172.21.15.62:34864) 2023-07-14T16:14:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d259c700 1 -- 172.21.15.62:0/2812822711 learned_addr learned my addr 172.21.15.62:0/2812822711 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:45.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d1d9b700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d40731c0 0x7f18d4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.117+0000 7f18d2d9d700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d410c3e0 0x7f18d419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18d259c700 1 -- 172.21.15.62:0/2812822711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d410c3e0 msgr2=0x7f18d419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18d259c700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d410c3e0 0x7f18d419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18d259c700 1 -- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d40731c0 msgr2=0x7f18d4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18d259c700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d40731c0 0x7f18d4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18d259c700 1 -- 172.21.15.62:0/2812822711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc009d00 con 0x7f18d40728b0 2023-07-14T16:14:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18d259c700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18d40728b0 0x7f18d4197e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18c400e9a0 tx=0x7f18c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18c37fe700 1 -- 172.21.15.62:0/2812822711 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c400ca10 con 0x7f18d40728b0 2023-07-14T16:14:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18c37fe700 1 -- 172.21.15.62:0/2812822711 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18c4010430 con 0x7f18d40728b0 2023-07-14T16:14:45.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d419d460 con 0x7f18d40728b0 2023-07-14T16:14:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.118+0000 7f18c37fe700 1 -- 172.21.15.62:0/2812822711 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c4018680 con 0x7f18d40728b0 2023-07-14T16:14:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.119+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d41a4c80 con 0x7f18d40728b0 2023-07-14T16:14:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.120+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d404e210 con 0x7f18d40728b0 2023-07-14T16:14:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.122+0000 7f18c37fe700 1 -- 172.21.15.62:0/2812822711 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f18c400cb70 con 0x7f18d40728b0 2023-07-14T16:14:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.125+0000 7f18c37fe700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18bc0a1260 0x7f18bc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.125+0000 7f18c37fe700 1 -- 172.21.15.62:0/2812822711 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18c4014070 con 0x7f18d40728b0 2023-07-14T16:14:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.125+0000 7f18d1d9b700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18bc0a1260 0x7f18bc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.126+0000 7f18d1d9b700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18bc0a1260 0x7f18bc0a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18d4199420 tx=0x7f18c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.126+0000 7f18c37fe700 1 -- 172.21.15.62:0/2812822711 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f18c4096640 con 0x7f18d40728b0 2023-07-14T16:14:45.328 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.370+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f18d419d5f0 con 0x7f18d40728b0 2023-07-14T16:14:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.370+0000 7f18c37fe700 1 -- 172.21.15.62:0/2812822711 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18c4099c60 con 0x7f18d40728b0 2023-07-14T16:14:45.349 INFO:teuthology.orchestra.run.smithi062.stdout:146028888125 2023-07-14T16:14:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18bc0a1260 msgr2=0x7f18bc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18bc0a1260 0x7f18bc0a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18d4199420 tx=0x7f18c8009260 comp rx=0 tx=0).stop 2023-07-14T16:14:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18d40728b0 msgr2=0x7f18d4197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18d40728b0 0x7f18d4197e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18c400e9a0 tx=0x7f18c400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 shutdown_connections 2023-07-14T16:14:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18bc0a1260 0x7f18bc0a3710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d410c3e0 0x7f18d419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18d40728b0 0x7f18d4197e00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 --2- 172.21.15.62:0/2812822711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d40731c0 0x7f18d4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 >> 172.21.15.62:0/2812822711 conn(0x7f18d40fb660 msgr2=0x7f18d410e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:45.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.373+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 shutdown_connections 2023-07-14T16:14:45.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.374+0000 7f18d8e82700 1 -- 172.21.15.62:0/2812822711 wait complete. 2023-07-14T16:14:45.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.538+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f929c199120 con 0x7f929c0ff290 2023-07-14T16:14:45.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.538+0000 7f92997fa700 1 -- 172.21.15.62:0/322351453 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f929000fe60 con 0x7f929c0ff290 2023-07-14T16:14:45.517 INFO:teuthology.orchestra.run.smithi062.stdout:313532612636 2023-07-14T16:14:45.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f92840a1260 msgr2=0x7f92840a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f92840a1260 0x7f92840a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f928c00b5c0 tx=0x7f928c005fb0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0ff290 msgr2=0x7f929c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0ff290 0x7f929c198300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f929000c9b0 tx=0x7f929000ccc0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 shutdown_connections 2023-07-14T16:14:45.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f92840a1260 0x7f92840a3710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f929c10c530 0x7f929c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f929c0fe980 0x7f929c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 --2- 172.21.15.62:0/322351453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f929c0ff290 0x7f929c198300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 >> 172.21.15.62:0/322351453 conn(0x7f929c075b70 msgr2=0x7f929c0fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:45.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 shutdown_connections 2023-07-14T16:14:45.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.541+0000 7f92a296c700 1 -- 172.21.15.62:0/322351453 wait complete. 2023-07-14T16:14:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 -- 172.21.15.62:0/2813498241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 msgr2=0x7fe54c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 --2- 172.21.15.62:0/2813498241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 0x7fe54c1001d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5480099c0 tx=0x7fe548009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 -- 172.21.15.62:0/2813498241 shutdown_connections 2023-07-14T16:14:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 --2- 172.21.15.62:0/2813498241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c104ea0 0x7fe54c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 --2- 172.21.15.62:0/2813498241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 0x7fe54c1001d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 --2- 172.21.15.62:0/2813498241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe54c1007a0 0x7fe54c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 -- 172.21.15.62:0/2813498241 >> 172.21.15.62:0/2813498241 conn(0x7fe54c0fb660 msgr2=0x7fe54c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 -- 172.21.15.62:0/2813498241 shutdown_connections 2023-07-14T16:14:45.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.782+0000 7fe5532c2700 1 -- 172.21.15.62:0/2813498241 wait complete. 2023-07-14T16:14:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.783+0000 7fe5532c2700 1 Processor -- start 2023-07-14T16:14:45.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe5532c2700 1 -- start start 2023-07-14T16:14:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe5532c2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 0x7fe54c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe5532c2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c1007a0 0x7fe54c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe5532c2700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe54c104ea0 0x7fe54c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe5532c2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c19cdb0 con 0x7fe54c0ffe00 2023-07-14T16:14:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe5532c2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c19cf20 con 0x7fe54c1007a0 2023-07-14T16:14:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe5532c2700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c19d0e0 con 0x7fe54c104ea0 2023-07-14T16:14:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe55105e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 0x7fe54c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe55105e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 0x7fe54c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57840/0 (socket says 172.21.15.62:57840) 2023-07-14T16:14:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe55105e700 1 -- 172.21.15.62:0/1185162750 learned_addr learned my addr 172.21.15.62:0/1185162750 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.784+0000 7fe55085d700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c1007a0 0x7fe54c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe55185f700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe54c104ea0 0x7fe54c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe55105e700 1 -- 172.21.15.62:0/1185162750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe54c104ea0 msgr2=0x7fe54c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe55105e700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe54c104ea0 0x7fe54c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe55105e700 1 -- 172.21.15.62:0/1185162750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c1007a0 msgr2=0x7fe54c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe55105e700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c1007a0 0x7fe54c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe55105e700 1 -- 172.21.15.62:0/1185162750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480095f0 con 0x7fe54c0ffe00 2023-07-14T16:14:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe55105e700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 0x7fe54c197df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe548005910 tx=0x7fe54800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe5427fc700 1 -- 172.21.15.62:0/1185162750 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54801d070 con 0x7fe54c0ffe00 2023-07-14T16:14:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c19d360 con 0x7fe54c0ffe00 2023-07-14T16:14:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe5427fc700 1 -- 172.21.15.62:0/1185162750 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5480054e0 con 0x7fe54c0ffe00 2023-07-14T16:14:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.785+0000 7fe5427fc700 1 -- 172.21.15.62:0/1185162750 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548004a70 con 0x7fe54c0ffe00 2023-07-14T16:14:45.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.786+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c1a4c20 con 0x7fe54c0ffe00 2023-07-14T16:14:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.787+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c04e210 con 0x7fe54c0ffe00 2023-07-14T16:14:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.789+0000 7fe5427fc700 1 -- 172.21.15.62:0/1185162750 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe54800fb80 con 0x7fe54c0ffe00 2023-07-14T16:14:45.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.790+0000 7fe5427fc700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5380a11e0 0x7fe5380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.790+0000 7fe5427fc700 1 -- 172.21.15.62:0/1185162750 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe5480cdcd0 con 0x7fe54c0ffe00 2023-07-14T16:14:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.791+0000 7fe55085d700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5380a11e0 0x7fe5380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:45.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.791+0000 7fe55085d700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5380a11e0 0x7fe5380a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe54c199410 tx=0x7fe53c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:45.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:45.792+0000 7fe5427fc700 1 -- 172.21.15.62:0/1185162750 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe548097360 con 0x7fe54c0ffe00 2023-07-14T16:14:45.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:45 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2812822711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T16:14:45.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:45 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/322351453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T16:14:46.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.031+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe54c199150 con 0x7fe54c0ffe00 2023-07-14T16:14:46.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.031+0000 7fe5427fc700 1 -- 172.21.15.62:0/1185162750 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe54809a980 con 0x7fe54c0ffe00 2023-07-14T16:14:46.010 INFO:teuthology.orchestra.run.smithi062.stdout:34359738452 2023-07-14T16:14:46.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5380a11e0 msgr2=0x7fe5380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:46.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5380a11e0 0x7fe5380a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe54c199410 tx=0x7fe53c00b410 comp rx=0 tx=0).stop 2023-07-14T16:14:46.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 msgr2=0x7fe54c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:46.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 0x7fe54c197df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe548005910 tx=0x7fe54800f690 comp rx=0 tx=0).stop 2023-07-14T16:14:46.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 shutdown_connections 2023-07-14T16:14:46.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe54c104ea0 0x7fe54c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe54c0ffe00 0x7fe54c197df0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5380a11e0 0x7fe5380a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 --2- 172.21.15.62:0/1185162750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c1007a0 0x7fe54c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 >> 172.21.15.62:0/1185162750 conn(0x7fe54c0fb660 msgr2=0x7fe54c0fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:46.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 shutdown_connections 2023-07-14T16:14:46.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.034+0000 7fe5532c2700 1 -- 172.21.15.62:0/1185162750 wait complete. 2023-07-14T16:14:46.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:45 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2812822711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T16:14:46.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:45 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/322351453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T16:14:46.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:45 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2812822711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T16:14:46.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:45 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/322351453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T16:14:46.782 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776162 for osd.9 2023-07-14T16:14:46.782 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:46.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 -- 172.21.15.62:0/1710224188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 msgr2=0x7f2418104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:46.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 --2- 172.21.15.62:0/1710224188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 0x7f2418104770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f240c0099c0 tx=0x7f240c009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 -- 172.21.15.62:0/1710224188 shutdown_connections 2023-07-14T16:14:46.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 --2- 172.21.15.62:0/1710224188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2418104ea0 0x7f241810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 --2- 172.21.15.62:0/1710224188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24180ffe00 0x7f24181001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 --2- 172.21.15.62:0/1710224188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 0x7f2418104770 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 -- 172.21.15.62:0/1710224188 >> 172.21.15.62:0/1710224188 conn(0x7f24180fb660 msgr2=0x7f24180fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:46.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 -- 172.21.15.62:0/1710224188 shutdown_connections 2023-07-14T16:14:46.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.826+0000 7f241e889700 1 -- 172.21.15.62:0/1710224188 wait complete. 2023-07-14T16:14:46.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.827+0000 7f241e889700 1 Processor -- start 2023-07-14T16:14:46.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.828+0000 7f241e889700 1 -- start start 2023-07-14T16:14:46.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.828+0000 7f241e889700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24180ffe00 0x7f241819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:46.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.828+0000 7f241e889700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 0x7f241819be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:46.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.828+0000 7f241e889700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2418104ea0 0x7f2418195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:46.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.828+0000 7f241e889700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2418196080 con 0x7f24181007a0 2023-07-14T16:14:46.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.828+0000 7f241e889700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24181961f0 con 0x7f2418104ea0 2023-07-14T16:14:46.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.828+0000 7f241e889700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24181963b0 con 0x7f24180ffe00 2023-07-14T16:14:46.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f24177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 0x7f241819be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:46.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f24177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 0x7f241819be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57872/0 (socket says 172.21.15.62:57872) 2023-07-14T16:14:46.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f24177fe700 1 -- 172.21.15.62:0/2861872243 learned_addr learned my addr 172.21.15.62:0/2861872243 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:46.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f2417fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24180ffe00 0x7f241819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:46.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f241ce26700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2418104ea0 0x7f2418195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:46.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f241ce26700 1 -- 172.21.15.62:0/2861872243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24180ffe00 msgr2=0x7f241819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:46.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f241ce26700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24180ffe00 0x7f241819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f241ce26700 1 -- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 msgr2=0x7f241819be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:46.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f241ce26700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 0x7f241819be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:46.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f241ce26700 1 -- 172.21.15.62:0/2861872243 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2408009520 con 0x7f2418104ea0 2023-07-14T16:14:46.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f2417fff700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24180ffe00 0x7f241819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:14:46.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.829+0000 7f24177fe700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 0x7f241819be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:14:46.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.830+0000 7f241ce26700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2418104ea0 0x7f2418195a20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2410007f10 tx=0x7f241000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:46.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.830+0000 7f24157fa700 1 -- 172.21.15.62:0/2861872243 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410004db0 con 0x7f2418104ea0 2023-07-14T16:14:46.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.830+0000 7f24157fa700 1 -- 172.21.15.62:0/2861872243 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f241000f040 con 0x7f2418104ea0 2023-07-14T16:14:46.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.830+0000 7f24157fa700 1 -- 172.21.15.62:0/2861872243 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410005960 con 0x7f2418104ea0 2023-07-14T16:14:46.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.830+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c0095f0 con 0x7f2418104ea0 2023-07-14T16:14:46.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.830+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24181a2750 con 0x7f2418104ea0 2023-07-14T16:14:46.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.832+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241804e210 con 0x7f2418104ea0 2023-07-14T16:14:46.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.837+0000 7f24157fa700 1 -- 172.21.15.62:0/2861872243 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f241000db20 con 0x7f2418104ea0 2023-07-14T16:14:46.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.838+0000 7f24157fa700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24000a12e0 0x7f24000a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:46.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.839+0000 7f24157fa700 1 -- 172.21.15.62:0/2861872243 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f24100cc300 con 0x7f2418104ea0 2023-07-14T16:14:46.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.839+0000 7f2417fff700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24000a12e0 0x7f24000a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:46.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.840+0000 7f2417fff700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24000a12e0 0x7f24000a3790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2408009fd0 tx=0x7f2408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:46.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:46.841+0000 7f24157fa700 1 -- 172.21.15.62:0/2861872243 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f241004e790 con 0x7f2418104ea0 2023-07-14T16:14:46.950 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888122 got 146028888125 for osd.4 2023-07-14T16:14:46.951 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:47.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:46 smithi164 ceph-mon[97427]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-14T16:14:47.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:46 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1185162750' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T16:14:47.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.057+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f24181a2aa0 con 0x7f2418104ea0 2023-07-14T16:14:47.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.058+0000 7f24157fa700 1 -- 172.21.15.62:0/2861872243 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2410095030 con 0x7f2418104ea0 2023-07-14T16:14:47.037 INFO:teuthology.orchestra.run.smithi062.stdout:111669149763 2023-07-14T16:14:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24000a12e0 msgr2=0x7f24000a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24000a12e0 0x7f24000a3790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2408009fd0 tx=0x7f2408009260 comp rx=0 tx=0).stop 2023-07-14T16:14:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2418104ea0 msgr2=0x7f2418195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2418104ea0 0x7f2418195a20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2410007f10 tx=0x7f241000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 shutdown_connections 2023-07-14T16:14:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24000a12e0 0x7f24000a3790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2418104ea0 0x7f2418195a20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24180ffe00 0x7f241819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 --2- 172.21.15.62:0/2861872243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24181007a0 0x7f241819be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 >> 172.21.15.62:0/2861872243 conn(0x7f24180fb660 msgr2=0x7f24180fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 shutdown_connections 2023-07-14T16:14:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.061+0000 7f241e889700 1 -- 172.21.15.62:0/2861872243 wait complete. 2023-07-14T16:14:47.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:46 smithi062 ceph-mon[95543]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-14T16:14:47.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:46 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1185162750' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T16:14:47.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:46 smithi081 ceph-mon[127795]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-07-14T16:14:47.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:46 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1185162750' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T16:14:47.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738452 for osd.0 2023-07-14T16:14:47.508 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:47.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 -- 172.21.15.62:0/1472132550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 msgr2=0x7f42ac110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 --2- 172.21.15.62:0/1472132550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 0x7f42ac110670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009990 tx=0x7f42a0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 -- 172.21.15.62:0/1472132550 shutdown_connections 2023-07-14T16:14:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 --2- 172.21.15.62:0/1472132550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42ac110bb0 0x7f42ac112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 --2- 172.21.15.62:0/1472132550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42ac0728b0 0x7f42ac072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 --2- 172.21.15.62:0/1472132550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 0x7f42ac110670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 -- 172.21.15.62:0/1472132550 >> 172.21.15.62:0/1472132550 conn(0x7f42ac0fb660 msgr2=0x7f42ac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:47.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 -- 172.21.15.62:0/1472132550 shutdown_connections 2023-07-14T16:14:47.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.728+0000 7f42b2861700 1 -- 172.21.15.62:0/1472132550 wait complete. 2023-07-14T16:14:47.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.729+0000 7f42b2861700 1 Processor -- start 2023-07-14T16:14:47.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.730+0000 7f42b2861700 1 -- start start 2023-07-14T16:14:47.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.730+0000 7f42b2861700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42ac0728b0 0x7f42ac19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:47.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.730+0000 7f42b2861700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 0x7f42ac19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:47.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.730+0000 7f42b2861700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42ac110bb0 0x7f42ac1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:47.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.730+0000 7f42b2861700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1a1150 con 0x7f42ac0731c0 2023-07-14T16:14:47.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.730+0000 7f42b2861700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1a12c0 con 0x7f42ac110bb0 2023-07-14T16:14:47.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.730+0000 7f42b2861700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1a1480 con 0x7f42ac0728b0 2023-07-14T16:14:47.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.730+0000 7f42ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 0x7f42ac19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:47.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 0x7f42ac19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57888/0 (socket says 172.21.15.62:57888) 2023-07-14T16:14:47.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42ab7fe700 1 -- 172.21.15.62:0/430554675 learned_addr learned my addr 172.21.15.62:0/430554675 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:47.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42abfff700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42ac0728b0 0x7f42ac19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:47.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42b0dfe700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42ac110bb0 0x7f42ac1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:47.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42ab7fe700 1 -- 172.21.15.62:0/430554675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42ac0728b0 msgr2=0x7f42ac19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42ab7fe700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42ac0728b0 0x7f42ac19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42ab7fe700 1 -- 172.21.15.62:0/430554675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42ac110bb0 msgr2=0x7f42ac1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42ab7fe700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42ac110bb0 0x7f42ac1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42ab7fe700 1 -- 172.21.15.62:0/430554675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00095f0 con 0x7f42ac0731c0 2023-07-14T16:14:47.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42ab7fe700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 0x7f42ac19c6d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42a0005910 tx=0x7f42a0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:47.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.731+0000 7f42a97fa700 1 -- 172.21.15.62:0/430554675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a001d070 con 0x7f42ac0731c0 2023-07-14T16:14:47.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.732+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac1a1700 con 0x7f42ac0731c0 2023-07-14T16:14:47.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.732+0000 7f42a97fa700 1 -- 172.21.15.62:0/430554675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42a0004c80 con 0x7f42ac0731c0 2023-07-14T16:14:47.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.732+0000 7f42a97fa700 1 -- 172.21.15.62:0/430554675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0003a30 con 0x7f42ac0731c0 2023-07-14T16:14:47.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.732+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac1a8fc0 con 0x7f42ac0731c0 2023-07-14T16:14:47.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.733+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac103090 con 0x7f42ac0731c0 2023-07-14T16:14:47.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.735+0000 7f42a97fa700 1 -- 172.21.15.62:0/430554675 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f42a0004fd0 con 0x7f42ac0731c0 2023-07-14T16:14:47.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.737+0000 7f42a97fa700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f42940a1260 0x7f42940a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.737+0000 7f42a97fa700 1 -- 172.21.15.62:0/430554675 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f42a00cd8b0 con 0x7f42ac0731c0 2023-07-14T16:14:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.737+0000 7f42abfff700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f42940a1260 0x7f42940a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.738+0000 7f42abfff700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f42940a1260 0x7f42940a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f429c007be0 tx=0x7f429c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:47.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.741+0000 7f42a97fa700 1 -- 172.21.15.62:0/430554675 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f42a0097000 con 0x7f42ac0731c0 2023-07-14T16:14:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 -- 172.21.15.62:0/832650624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa030104e60 msgr2=0x7fa03010c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 --2- 172.21.15.62:0/832650624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa030104e60 0x7fa03010c9e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa02800b0d0 tx=0x7fa02800b3e0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 -- 172.21.15.62:0/832650624 shutdown_connections 2023-07-14T16:14:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 --2- 172.21.15.62:0/832650624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa030104e60 0x7fa03010c9e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 --2- 172.21.15.62:0/832650624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0300ffdc0 0x7fa030100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 --2- 172.21.15.62:0/832650624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa030100760 0x7fa030104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 -- 172.21.15.62:0/832650624 >> 172.21.15.62:0/832650624 conn(0x7fa0300fb680 msgr2=0x7fa0300fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 -- 172.21.15.62:0/832650624 shutdown_connections 2023-07-14T16:14:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.748+0000 7fa036de5700 1 -- 172.21.15.62:0/832650624 wait complete. 2023-07-14T16:14:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.749+0000 7fa036de5700 1 Processor -- start 2023-07-14T16:14:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.749+0000 7fa036de5700 1 -- start start 2023-07-14T16:14:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa036de5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0300ffdc0 0x7fa030195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa036de5700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030100760 0x7fa030196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa036de5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa030104e60 0x7fa03019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa036de5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03019ad90 con 0x7fa0300ffdc0 2023-07-14T16:14:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa036de5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03019af00 con 0x7fa030104e60 2023-07-14T16:14:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa036de5700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03019b0c0 con 0x7fa030100760 2023-07-14T16:14:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa034b81700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0300ffdc0 0x7fa030195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa034b81700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0300ffdc0 0x7fa030195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57906/0 (socket says 172.21.15.62:57906) 2023-07-14T16:14:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa034b81700 1 -- 172.21.15.62:0/3247435136 learned_addr learned my addr 172.21.15.62:0/3247435136 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:47.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa035382700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa030104e60 0x7fa03019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:47.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa02ffff700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030100760 0x7fa030196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:47.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa034b81700 1 -- 172.21.15.62:0/3247435136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030100760 msgr2=0x7fa030196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa034b81700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030100760 0x7fa030196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa034b81700 1 -- 172.21.15.62:0/3247435136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa030104e60 msgr2=0x7fa03019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa034b81700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa030104e60 0x7fa03019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa034b81700 1 -- 172.21.15.62:0/3247435136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028009d00 con 0x7fa0300ffdc0 2023-07-14T16:14:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.750+0000 7fa02ffff700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030100760 0x7fa030196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:14:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.751+0000 7fa034b81700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0300ffdc0 0x7fa030195dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa02400e9a0 tx=0x7fa02400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.751+0000 7fa02dffb700 1 -- 172.21.15.62:0/3247435136 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024009790 con 0x7fa0300ffdc0 2023-07-14T16:14:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.751+0000 7fa02dffb700 1 -- 172.21.15.62:0/3247435136 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa02400cbd0 con 0x7fa0300ffdc0 2023-07-14T16:14:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.751+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03019b3a0 con 0x7fa0300ffdc0 2023-07-14T16:14:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.751+0000 7fa02dffb700 1 -- 172.21.15.62:0/3247435136 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024018680 con 0x7fa0300ffdc0 2023-07-14T16:14:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.751+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0301a2c00 con 0x7fa0300ffdc0 2023-07-14T16:14:47.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.752+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03004e210 con 0x7fa0300ffdc0 2023-07-14T16:14:47.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.754+0000 7fa02dffb700 1 -- 172.21.15.62:0/3247435136 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa02400cd40 con 0x7fa0300ffdc0 2023-07-14T16:14:47.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.757+0000 7fa02dffb700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa0200a1260 0x7fa0200a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:47.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.757+0000 7fa02dffb700 1 -- 172.21.15.62:0/3247435136 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa024014070 con 0x7fa0300ffdc0 2023-07-14T16:14:47.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.757+0000 7fa02ffff700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa0200a1260 0x7fa0200a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:47.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.758+0000 7fa02ffff700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa0200a1260 0x7fa0200a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa0301973f0 tx=0x7fa01c009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:47.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.760+0000 7fa02dffb700 1 -- 172.21.15.62:0/3247435136 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa024051790 con 0x7fa0300ffdc0 2023-07-14T16:14:47.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.970+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f42ac04e210 con 0x7f42ac0731c0 2023-07-14T16:14:47.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.970+0000 7f42a97fa700 1 -- 172.21.15.62:0/430554675 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f42a0026090 con 0x7f42ac0731c0 2023-07-14T16:14:47.949 INFO:teuthology.orchestra.run.smithi062.stdout:55834574925 2023-07-14T16:14:47.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f42940a1260 msgr2=0x7f42940a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f42940a1260 0x7f42940a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f429c007be0 tx=0x7f429c0073d0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 msgr2=0x7f42ac19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 0x7f42ac19c6d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42a0005910 tx=0x7f42a0009580 comp rx=0 tx=0).stop 2023-07-14T16:14:47.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 shutdown_connections 2023-07-14T16:14:47.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f42940a1260 0x7f42940a3710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42ac110bb0 0x7f42ac1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42ac0728b0 0x7f42ac19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 --2- 172.21.15.62:0/430554675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42ac0731c0 0x7f42ac19c6d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 >> 172.21.15.62:0/430554675 conn(0x7f42ac0fb660 msgr2=0x7f42ac0fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:47.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 shutdown_connections 2023-07-14T16:14:47.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.973+0000 7f42b2861700 1 -- 172.21.15.62:0/430554675 wait complete. 2023-07-14T16:14:47.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.976+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fa030197130 con 0x7fa0300ffdc0 2023-07-14T16:14:47.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.976+0000 7fa02dffb700 1 -- 172.21.15.62:0/3247435136 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa0240962b0 con 0x7fa0300ffdc0 2023-07-14T16:14:47.955 INFO:teuthology.orchestra.run.smithi062.stdout:257698037799 2023-07-14T16:14:47.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.980+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa0200a1260 msgr2=0x7fa0200a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.980+0000 7fa036de5700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa0200a1260 0x7fa0200a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa0301973f0 tx=0x7fa01c009550 comp rx=0 tx=0).stop 2023-07-14T16:14:47.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.980+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0300ffdc0 msgr2=0x7fa030195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:47.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.980+0000 7fa036de5700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0300ffdc0 0x7fa030195dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa02400e9a0 tx=0x7fa02400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.981+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 shutdown_connections 2023-07-14T16:14:47.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.981+0000 7fa036de5700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa0200a1260 0x7fa0200a3710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.981+0000 7fa036de5700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa030104e60 0x7fa03019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.981+0000 7fa036de5700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0300ffdc0 0x7fa030195dd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.981+0000 7fa036de5700 1 --2- 172.21.15.62:0/3247435136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030100760 0x7fa030196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:47.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.981+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 >> 172.21.15.62:0/3247435136 conn(0x7fa0300fb680 msgr2=0x7fa0300fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:47.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.981+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 shutdown_connections 2023-07-14T16:14:47.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:47.981+0000 7fa036de5700 1 -- 172.21.15.62:0/3247435136 wait complete. 2023-07-14T16:14:48.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:47 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2861872243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T16:14:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:47 smithi164 ceph-mon[97427]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:14:48.027 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:47 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2861872243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T16:14:48.027 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:47 smithi062 ceph-mon[95543]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:14:48.145 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612636 for osd.10 2023-07-14T16:14:48.146 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:48.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:47 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2861872243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T16:14:48.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:47 smithi081 ceph-mon[127795]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:14:48.879 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149761 got 111669149763 for osd.3 2023-07-14T16:14:48.880 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:48.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037799 for osd.8 2023-07-14T16:14:48.915 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:48 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/430554675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T16:14:49.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:48 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3247435136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T16:14:49.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:48 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/430554675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T16:14:49.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:48 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3247435136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T16:14:49.104 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574925 for osd.1 2023-07-14T16:14:49.105 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:49.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:48 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/430554675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T16:14:49.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:48 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3247435136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T16:14:49.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 -- 172.21.15.62:0/2127695054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c381007a0 msgr2=0x7f5c38104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/2127695054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c381007a0 0x7f5c38104770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0099c0 tx=0x7f5c2c009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 -- 172.21.15.62:0/2127695054 shutdown_connections 2023-07-14T16:14:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/2127695054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c38104ea0 0x7f5c3810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/2127695054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c380ffe00 0x7f5c381001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/2127695054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c381007a0 0x7f5c38104770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 -- 172.21.15.62:0/2127695054 >> 172.21.15.62:0/2127695054 conn(0x7f5c380fb660 msgr2=0x7f5c380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 -- 172.21.15.62:0/2127695054 shutdown_connections 2023-07-14T16:14:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.612+0000 7f5c3cf12700 1 -- 172.21.15.62:0/2127695054 wait complete. 2023-07-14T16:14:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.613+0000 7f5c3cf12700 1 Processor -- start 2023-07-14T16:14:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.613+0000 7f5c3cf12700 1 -- start start 2023-07-14T16:14:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.613+0000 7f5c3cf12700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c380ffe00 0x7f5c38072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.613+0000 7f5c3cf12700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c381007a0 0x7f5c38072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.613+0000 7f5c3cf12700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c38104ea0 0x7f5c380732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c3cf12700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381a4040 con 0x7f5c38104ea0 2023-07-14T16:14:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c3cf12700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381a41b0 con 0x7f5c380ffe00 2023-07-14T16:14:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c3cf12700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381a4370 con 0x7f5c381007a0 2023-07-14T16:14:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c36d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c38104ea0 0x7f5c380732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c36d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c38104ea0 0x7f5c380732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57916/0 (socket says 172.21.15.62:57916) 2023-07-14T16:14:49.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c36d9d700 1 -- 172.21.15.62:0/3218627187 learned_addr learned my addr 172.21.15.62:0/3218627187 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:49.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c3659c700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c380ffe00 0x7f5c38072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:49.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c35d9b700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c381007a0 0x7f5c38072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:49.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c36d9d700 1 -- 172.21.15.62:0/3218627187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c381007a0 msgr2=0x7f5c38072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:49.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c36d9d700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c381007a0 0x7f5c38072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c36d9d700 1 -- 172.21.15.62:0/3218627187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c380ffe00 msgr2=0x7f5c38072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:49.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c36d9d700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c380ffe00 0x7f5c38072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c36d9d700 1 -- 172.21.15.62:0/3218627187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c0095f0 con 0x7f5c38104ea0 2023-07-14T16:14:49.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.614+0000 7f5c3659c700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c380ffe00 0x7f5c38072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:49.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.615+0000 7f5c36d9d700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c38104ea0 0x7f5c380732d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5c30007ee0 tx=0x7f5c3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:49.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.615+0000 7f5c277fe700 1 -- 172.21.15.62:0/3218627187 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30017070 con 0x7f5c38104ea0 2023-07-14T16:14:49.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.615+0000 7f5c277fe700 1 -- 172.21.15.62:0/3218627187 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c3000f040 con 0x7f5c38104ea0 2023-07-14T16:14:49.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.615+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c381a4650 con 0x7f5c38104ea0 2023-07-14T16:14:49.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.615+0000 7f5c277fe700 1 -- 172.21.15.62:0/3218627187 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30005710 con 0x7f5c38104ea0 2023-07-14T16:14:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.615+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c381a4b20 con 0x7f5c38104ea0 2023-07-14T16:14:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.616+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3804e210 con 0x7f5c38104ea0 2023-07-14T16:14:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.618+0000 7f5c277fe700 1 -- 172.21.15.62:0/3218627187 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5c30005870 con 0x7f5c38104ea0 2023-07-14T16:14:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.619+0000 7f5c277fe700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c200a11e0 0x7f5c200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.620+0000 7f5c277fe700 1 -- 172.21.15.62:0/3218627187 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5c300cc2e0 con 0x7f5c38104ea0 2023-07-14T16:14:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.620+0000 7f5c3659c700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c200a11e0 0x7f5c200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.620+0000 7f5c3659c700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c200a11e0 0x7f5c200a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005950 tx=0x7f5c280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.624+0000 7f5c277fe700 1 -- 172.21.15.62:0/3218627187 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c30095880 con 0x7f5c38104ea0 2023-07-14T16:14:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.831+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5c381a4e70 con 0x7f5c38104ea0 2023-07-14T16:14:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.831+0000 7f5c277fe700 1 -- 172.21.15.62:0/3218627187 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c30098ea0 con 0x7f5c38104ea0 2023-07-14T16:14:49.810 INFO:teuthology.orchestra.run.smithi062.stdout:201863462964 2023-07-14T16:14:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c200a11e0 msgr2=0x7f5c200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c200a11e0 0x7f5c200a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005950 tx=0x7f5c280058e0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c38104ea0 msgr2=0x7f5c380732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c38104ea0 0x7f5c380732d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5c30007ee0 tx=0x7f5c3000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 shutdown_connections 2023-07-14T16:14:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c38104ea0 0x7f5c380732d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c200a11e0 0x7f5c200a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c380ffe00 0x7f5c38072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 --2- 172.21.15.62:0/3218627187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c381007a0 0x7f5c38072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 >> 172.21.15.62:0/3218627187 conn(0x7f5c380fb660 msgr2=0x7f5c380fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.833+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 shutdown_connections 2023-07-14T16:14:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:49.834+0000 7f5c3cf12700 1 -- 172.21.15.62:0/3218627187 wait complete. 2023-07-14T16:14:50.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:49 smithi164 ceph-mon[97427]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:50.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:49 smithi062 ceph-mon[95543]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:50.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:49 smithi081 ceph-mon[127795]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:50.270 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462964 for osd.6 2023-07-14T16:14:50.270 DEBUG:teuthology.parallel:result is None 2023-07-14T16:14:50.271 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-14T16:14:50.271 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph pg dump --format=json 2023-07-14T16:14:51.001 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:50 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3218627187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T16:14:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:50 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3218627187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T16:14:51.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:50 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3218627187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T16:14:51.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:51 smithi062 ceph-mon[95543]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:52.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:51 smithi081 ceph-mon[127795]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:52.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:51 smithi164 ceph-mon[97427]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:52.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.561+0000 7f401b2d9700 1 -- 172.21.15.62:0/1384297292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 msgr2=0x7f401410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:52.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.561+0000 7f401b2d9700 1 --2- 172.21.15.62:0/1384297292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 0x7f401410ca20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f400c00b0d0 tx=0x7f400c00b3e0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.561+0000 7f401b2d9700 1 -- 172.21.15.62:0/1384297292 shutdown_connections 2023-07-14T16:14:52.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.561+0000 7f401b2d9700 1 --2- 172.21.15.62:0/1384297292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 0x7f401410ca20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.561+0000 7f401b2d9700 1 --2- 172.21.15.62:0/1384297292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140ffe00 0x7f40141001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.561+0000 7f401b2d9700 1 --2- 172.21.15.62:0/1384297292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40141007a0 0x7f4014104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.561+0000 7f401b2d9700 1 -- 172.21.15.62:0/1384297292 >> 172.21.15.62:0/1384297292 conn(0x7f40140fb660 msgr2=0x7f40140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:52.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.561+0000 7f401b2d9700 1 -- 172.21.15.62:0/1384297292 shutdown_connections 2023-07-14T16:14:52.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.562+0000 7f401b2d9700 1 -- 172.21.15.62:0/1384297292 wait complete. 2023-07-14T16:14:52.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.562+0000 7f401b2d9700 1 Processor -- start 2023-07-14T16:14:52.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f401b2d9700 1 -- start start 2023-07-14T16:14:52.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f401b2d9700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140ffe00 0x7f4014197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:52.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f401b2d9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40141007a0 0x7f4014198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:52.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f401b2d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 0x7f401419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:52.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f401b2d9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401419cdf0 con 0x7f4014104ea0 2023-07-14T16:14:52.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f401b2d9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401419cf60 con 0x7f40141007a0 2023-07-14T16:14:52.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f401b2d9700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401419d120 con 0x7f40140ffe00 2023-07-14T16:14:52.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f4019876700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 0x7f401419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:52.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f4019876700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 0x7f401419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57938/0 (socket says 172.21.15.62:57938) 2023-07-14T16:14:52.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.563+0000 7f4019876700 1 -- 172.21.15.62:0/2650213777 learned_addr learned my addr 172.21.15.62:0/2650213777 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4019075700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140ffe00 0x7f4014197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4018874700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40141007a0 0x7f4014198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4019876700 1 -- 172.21.15.62:0/2650213777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140ffe00 msgr2=0x7f4014197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4019876700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140ffe00 0x7f4014197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4019876700 1 -- 172.21.15.62:0/2650213777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40141007a0 msgr2=0x7f4014198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4019876700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40141007a0 0x7f4014198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4019876700 1 -- 172.21.15.62:0/2650213777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c009d00 con 0x7f4014104ea0 2023-07-14T16:14:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4018874700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40141007a0 0x7f4014198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:14:52.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f4019876700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 0x7f401419c7e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f400c009300 tx=0x7f400c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:52.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f400a7fc700 1 -- 172.21.15.62:0/2650213777 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c00e040 con 0x7f4014104ea0 2023-07-14T16:14:52.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f400a7fc700 1 -- 172.21.15.62:0/2650213777 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f400c003e10 con 0x7f4014104ea0 2023-07-14T16:14:52.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f400a7fc700 1 -- 172.21.15.62:0/2650213777 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c01b8e0 con 0x7f4014104ea0 2023-07-14T16:14:52.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.564+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401419d3a0 con 0x7f4014104ea0 2023-07-14T16:14:52.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.565+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40141a4a90 con 0x7f4014104ea0 2023-07-14T16:14:52.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.566+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401404e210 con 0x7f4014104ea0 2023-07-14T16:14:52.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.568+0000 7f400a7fc700 1 -- 172.21.15.62:0/2650213777 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f400c012470 con 0x7f4014104ea0 2023-07-14T16:14:52.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.569+0000 7f400a7fc700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40000a1260 0x7f40000a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:52.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.569+0000 7f400a7fc700 1 -- 172.21.15.62:0/2650213777 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f400c0ce190 con 0x7f4014104ea0 2023-07-14T16:14:52.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.569+0000 7f4019075700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40000a1260 0x7f40000a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:52.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.570+0000 7f4019075700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40000a1260 0x7f40000a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4010005950 tx=0x7f40100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:52.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.573+0000 7f400a7fc700 1 -- 172.21.15.62:0/2650213777 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f400c097820 con 0x7f4014104ea0 2023-07-14T16:14:52.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.775+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40140fc840 con 0x7f40000a1260 2023-07-14T16:14:52.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.787+0000 7f400a7fc700 1 -- 172.21.15.62:0/2650213777 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339190 (secure 0 0 0) 0x7f40140fc840 con 0x7f40000a1260 2023-07-14T16:14:52.769 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:52.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40000a1260 msgr2=0x7f40000a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40000a1260 0x7f40000a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4010005950 tx=0x7f40100058e0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 msgr2=0x7f401419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 0x7f401419c7e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f400c009300 tx=0x7f400c0093e0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 shutdown_connections 2023-07-14T16:14:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4014104ea0 0x7f401419c7e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140ffe00 0x7f4014197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40000a1260 0x7f40000a3710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 --2- 172.21.15.62:0/2650213777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40141007a0 0x7f4014198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.791+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 >> 172.21.15.62:0/2650213777 conn(0x7f40140fb660 msgr2=0x7f40140fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.792+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 shutdown_connections 2023-07-14T16:14:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:52.792+0000 7f401b2d9700 1 -- 172.21.15.62:0/2650213777 wait complete. 2023-07-14T16:14:52.772 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-07-14T16:14:53.276 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2023-07-14T16:14:51.482231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652049,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":422,"num_read_kb":321,"num_write":331,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335052,"kb_used_data":11628,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606772,"statfs":{"total":1151940427776,"available":1151597334528,"internally_reserved":0,"allocated":11907072,"data_stored":7459582,"data_compressed":112137,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003816"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661040+0000","last_change":"2023-07-14T16:13:59.301892+0000","last_active":"2023-07-14T16:14:20.661040+0000","last_peered":"2023-07-14T16:14:20.661040+0000","last_clean":"2023-07-14T16:14:20.661040+0000","last_became_active":"2023-07-14T16:13:59.301379+0000","last_became_peered":"2023-07-14T16:13:59.301379+0000","last_unstale":"2023-07-14T16:14:20.661040+0000","last_undegraded":"2023-07-14T16:14:20.661040+0000","last_fullsized":"2023-07-14T16:14:20.661040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:49:12.243022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665853+0000","last_change":"2023-07-14T16:13:59.309322+0000","last_active":"2023-07-14T16:14:20.665853+0000","last_peered":"2023-07-14T16:14:20.665853+0000","last_clean":"2023-07-14T16:14:20.665853+0000","last_became_active":"2023-07-14T16:13:59.309035+0000","last_became_peered":"2023-07-14T16:13:59.309035+0000","last_unstale":"2023-07-14T16:14:20.665853+0000","last_undegraded":"2023-07-14T16:14:20.665853+0000","last_fullsized":"2023-07-14T16:14:20.665853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:46.013552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665545+0000","last_change":"2023-07-14T16:13:59.304371+0000","last_active":"2023-07-14T16:14:20.665545+0000","last_peered":"2023-07-14T16:14:20.665545+0000","last_clean":"2023-07-14T16:14:20.665545+0000","last_became_active":"2023-07-14T16:13:59.303965+0000","last_became_peered":"2023-07-14T16:13:59.303965+0000","last_unstale":"2023-07-14T16:14:20.665545+0000","last_undegraded":"2023-07-14T16:14:20.665545+0000","last_fullsized":"2023-07-14T16:14:20.665545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:19.938384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.674685+0000","last_change":"2023-07-14T16:14:20.670094+0000","last_active":"2023-07-14T16:14:20.674685+0000","last_peered":"2023-07-14T16:14:20.674685+0000","last_clean":"2023-07-14T16:14:20.674685+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:14:20.674685+0000","last_undegraded":"2023-07-14T16:14:20.674685+0000","last_fullsized":"2023-07-14T16:14:20.674685+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:32.039209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.662901+0000","last_change":"2023-07-14T16:13:59.298094+0000","last_active":"2023-07-14T16:14:20.662901+0000","last_peered":"2023-07-14T16:14:20.662901+0000","last_clean":"2023-07-14T16:14:20.662901+0000","last_became_active":"2023-07-14T16:13:59.297764+0000","last_became_peered":"2023-07-14T16:13:59.297764+0000","last_unstale":"2023-07-14T16:14:20.662901+0000","last_undegraded":"2023-07-14T16:14:20.662901+0000","last_fullsized":"2023-07-14T16:14:20.662901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:40:48.501912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666802+0000","last_change":"2023-07-14T16:13:59.299735+0000","last_active":"2023-07-14T16:14:20.666802+0000","last_peered":"2023-07-14T16:14:20.666802+0000","last_clean":"2023-07-14T16:14:20.666802+0000","last_became_active":"2023-07-14T16:13:59.299413+0000","last_became_peered":"2023-07-14T16:13:59.299413+0000","last_unstale":"2023-07-14T16:14:20.666802+0000","last_undegraded":"2023-07-14T16:14:20.666802+0000","last_fullsized":"2023-07-14T16:14:20.666802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:09:10.174873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660827+0000","last_change":"2023-07-14T16:13:59.299682+0000","last_active":"2023-07-14T16:14:20.660827+0000","last_peered":"2023-07-14T16:14:20.660827+0000","last_clean":"2023-07-14T16:14:20.660827+0000","last_became_active":"2023-07-14T16:13:59.299327+0000","last_became_peered":"2023-07-14T16:13:59.299327+0000","last_unstale":"2023-07-14T16:14:20.660827+0000","last_undegraded":"2023-07-14T16:14:20.660827+0000","last_fullsized":"2023-07-14T16:14:20.660827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:30.735405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663232+0000","last_change":"2023-07-14T16:13:59.311818+0000","last_active":"2023-07-14T16:14:20.663232+0000","last_peered":"2023-07-14T16:14:20.663232+0000","last_clean":"2023-07-14T16:14:20.663232+0000","last_became_active":"2023-07-14T16:13:59.311478+0000","last_became_peered":"2023-07-14T16:13:59.311478+0000","last_unstale":"2023-07-14T16:14:20.663232+0000","last_undegraded":"2023-07-14T16:14:20.663232+0000","last_fullsized":"2023-07-14T16:14:20.663232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:51.107741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763755+0000","last_change":"2023-07-14T16:13:59.309665+0000","last_active":"2023-07-14T16:14:20.763755+0000","last_peered":"2023-07-14T16:14:20.763755+0000","last_clean":"2023-07-14T16:14:20.763755+0000","last_became_active":"2023-07-14T16:13:59.309361+0000","last_became_peered":"2023-07-14T16:13:59.309361+0000","last_unstale":"2023-07-14T16:14:20.763755+0000","last_undegraded":"2023-07-14T16:14:20.763755+0000","last_fullsized":"2023-07-14T16:14:20.763755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:14.297747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666802+0000","last_change":"2023-07-14T16:13:59.296397+0000","last_active":"2023-07-14T16:14:20.666802+0000","last_peered":"2023-07-14T16:14:20.666802+0000","last_clean":"2023-07-14T16:14:20.666802+0000","last_became_active":"2023-07-14T16:13:59.295965+0000","last_became_peered":"2023-07-14T16:13:59.295965+0000","last_unstale":"2023-07-14T16:14:20.666802+0000","last_undegraded":"2023-07-14T16:14:20.666802+0000","last_fullsized":"2023-07-14T16:14:20.666802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:51.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661239+0000","last_change":"2023-07-14T16:13:59.302892+0000","last_active":"2023-07-14T16:14:20.661239+0000","last_peered":"2023-07-14T16:14:20.661239+0000","last_clean":"2023-07-14T16:14:20.661239+0000","last_became_active":"2023-07-14T16:13:59.302295+0000","last_became_peered":"2023-07-14T16:13:59.302295+0000","last_unstale":"2023-07-14T16:14:20.661239+0000","last_undegraded":"2023-07-14T16:14:20.661239+0000","last_fullsized":"2023-07-14T16:14:20.661239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:11.009590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664108+0000","last_change":"2023-07-14T16:13:59.301913+0000","last_active":"2023-07-14T16:14:20.664108+0000","last_peered":"2023-07-14T16:14:20.664108+0000","last_clean":"2023-07-14T16:14:20.664108+0000","last_became_active":"2023-07-14T16:13:59.301590+0000","last_became_peered":"2023-07-14T16:13:59.301590+0000","last_unstale":"2023-07-14T16:14:20.664108+0000","last_undegraded":"2023-07-14T16:14:20.664108+0000","last_fullsized":"2023-07-14T16:14:20.664108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:09:55.206983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763393+0000","last_change":"2023-07-14T16:13:59.302006+0000","last_active":"2023-07-14T16:14:20.763393+0000","last_peered":"2023-07-14T16:14:20.763393+0000","last_clean":"2023-07-14T16:14:20.763393+0000","last_became_active":"2023-07-14T16:13:59.301752+0000","last_became_peered":"2023-07-14T16:13:59.301752+0000","last_unstale":"2023-07-14T16:14:20.763393+0000","last_undegraded":"2023-07-14T16:14:20.763393+0000","last_fullsized":"2023-07-14T16:14:20.763393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:08.923165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665808+0000","last_change":"2023-07-14T16:13:59.309318+0000","last_active":"2023-07-14T16:14:20.665808+0000","last_peered":"2023-07-14T16:14:20.665808+0000","last_clean":"2023-07-14T16:14:20.665808+0000","last_became_active":"2023-07-14T16:13:59.308931+0000","last_became_peered":"2023-07-14T16:13:59.308931+0000","last_unstale":"2023-07-14T16:14:20.665808+0000","last_undegraded":"2023-07-14T16:14:20.665808+0000","last_fullsized":"2023-07-14T16:14:20.665808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:21:02.747919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.670208+0000","last_change":"2023-07-14T16:14:20.670208+0000","last_active":"2023-07-14T16:14:20.670208+0000","last_peered":"2023-07-14T16:14:20.670208+0000","last_clean":"2023-07-14T16:14:20.670208+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:14:20.670208+0000","last_undegraded":"2023-07-14T16:14:20.670208+0000","last_fullsized":"2023-07-14T16:14:20.670208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:53.265928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763182+0000","last_change":"2023-07-14T16:13:59.307986+0000","last_active":"2023-07-14T16:14:20.763182+0000","last_peered":"2023-07-14T16:14:20.763182+0000","last_clean":"2023-07-14T16:14:20.763182+0000","last_became_active":"2023-07-14T16:13:59.307701+0000","last_became_peered":"2023-07-14T16:13:59.307701+0000","last_unstale":"2023-07-14T16:14:20.763182+0000","last_undegraded":"2023-07-14T16:14:20.763182+0000","last_fullsized":"2023-07-14T16:14:20.763182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:56:03.104096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661208+0000","last_change":"2023-07-14T16:13:59.301685+0000","last_active":"2023-07-14T16:14:20.661208+0000","last_peered":"2023-07-14T16:14:20.661208+0000","last_clean":"2023-07-14T16:14:20.661208+0000","last_became_active":"2023-07-14T16:13:59.300905+0000","last_became_peered":"2023-07-14T16:13:59.300905+0000","last_unstale":"2023-07-14T16:14:20.661208+0000","last_undegraded":"2023-07-14T16:14:20.661208+0000","last_fullsized":"2023-07-14T16:14:20.661208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:48:24.721908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665233+0000","last_change":"2023-07-14T16:13:59.306027+0000","last_active":"2023-07-14T16:14:20.665233+0000","last_peered":"2023-07-14T16:14:20.665233+0000","last_clean":"2023-07-14T16:14:20.665233+0000","last_became_active":"2023-07-14T16:13:59.305718+0000","last_became_peered":"2023-07-14T16:13:59.305718+0000","last_unstale":"2023-07-14T16:14:20.665233+0000","last_undegraded":"2023-07-14T16:14:20.665233+0000","last_fullsized":"2023-07-14T16:14:20.665233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:20:47.951543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.668632+0000","last_change":"2023-07-14T16:13:59.294140+0000","last_active":"2023-07-14T16:14:20.668632+0000","last_peered":"2023-07-14T16:14:20.668632+0000","last_clean":"2023-07-14T16:14:20.668632+0000","last_became_active":"2023-07-14T16:13:59.293825+0000","last_became_peered":"2023-07-14T16:13:59.293825+0000","last_unstale":"2023-07-14T16:14:20.668632+0000","last_undegraded":"2023-07-14T16:14:20.668632+0000","last_fullsized":"2023-07-14T16:14:20.668632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:59.348563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665523+0000","last_change":"2023-07-14T16:13:58.277154+0000","last_active":"2023-07-14T16:14:20.665523+0000","last_peered":"2023-07-14T16:14:20.665523+0000","last_clean":"2023-07-14T16:14:20.665523+0000","last_became_active":"2023-07-14T16:13:58.276800+0000","last_became_peered":"2023-07-14T16:13:58.276800+0000","last_unstale":"2023-07-14T16:14:20.665523+0000","last_undegraded":"2023-07-14T16:14:20.665523+0000","last_fullsized":"2023-07-14T16:14:20.665523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:45.631137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663871+0000","last_change":"2023-07-14T16:13:59.299819+0000","last_active":"2023-07-14T16:14:20.663871+0000","last_peered":"2023-07-14T16:14:20.663871+0000","last_clean":"2023-07-14T16:14:20.663871+0000","last_became_active":"2023-07-14T16:13:59.299294+0000","last_became_peered":"2023-07-14T16:13:59.299294+0000","last_unstale":"2023-07-14T16:14:20.663871+0000","last_undegraded":"2023-07-14T16:14:20.663871+0000","last_fullsized":"2023-07-14T16:14:20.663871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:43.461827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664040+0000","last_change":"2023-07-14T16:13:58.278954+0000","last_active":"2023-07-14T16:14:20.664040+0000","last_peered":"2023-07-14T16:14:20.664040+0000","last_clean":"2023-07-14T16:14:20.664040+0000","last_became_active":"2023-07-14T16:13:58.278454+0000","last_became_peered":"2023-07-14T16:13:58.278454+0000","last_unstale":"2023-07-14T16:14:20.664040+0000","last_undegraded":"2023-07-14T16:14:20.664040+0000","last_fullsized":"2023-07-14T16:14:20.664040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:33:34.015342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665978+0000","last_change":"2023-07-14T16:13:59.299877+0000","last_active":"2023-07-14T16:14:20.665978+0000","last_peered":"2023-07-14T16:14:20.665978+0000","last_clean":"2023-07-14T16:14:20.665978+0000","last_became_active":"2023-07-14T16:13:59.299557+0000","last_became_peered":"2023-07-14T16:13:59.299557+0000","last_unstale":"2023-07-14T16:14:20.665978+0000","last_undegraded":"2023-07-14T16:14:20.665978+0000","last_fullsized":"2023-07-14T16:14:20.665978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:45.331826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.677912+0000","last_change":"2023-07-14T16:14:20.677912+0000","last_active":"2023-07-14T16:14:20.677912+0000","last_peered":"2023-07-14T16:14:20.677912+0000","last_clean":"2023-07-14T16:14:20.677912+0000","last_became_active":"2023-07-14T16:14:20.668156+0000","last_became_peered":"2023-07-14T16:14:20.668156+0000","last_unstale":"2023-07-14T16:14:20.677912+0000","last_undegraded":"2023-07-14T16:14:20.677912+0000","last_fullsized":"2023-07-14T16:14:20.677912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:51:50.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763275+0000","last_change":"2023-07-14T16:13:59.307987+0000","last_active":"2023-07-14T16:14:20.763275+0000","last_peered":"2023-07-14T16:14:20.763275+0000","last_clean":"2023-07-14T16:14:20.763275+0000","last_became_active":"2023-07-14T16:13:59.307457+0000","last_became_peered":"2023-07-14T16:13:59.307457+0000","last_unstale":"2023-07-14T16:14:20.763275+0000","last_undegraded":"2023-07-14T16:14:20.763275+0000","last_fullsized":"2023-07-14T16:14:20.763275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:32:28.569384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663585+0000","last_change":"2023-07-14T16:13:58.282332+0000","last_active":"2023-07-14T16:14:20.663585+0000","last_peered":"2023-07-14T16:14:20.663585+0000","last_clean":"2023-07-14T16:14:20.663585+0000","last_became_active":"2023-07-14T16:13:58.278103+0000","last_became_peered":"2023-07-14T16:13:58.278103+0000","last_unstale":"2023-07-14T16:14:20.663585+0000","last_undegraded":"2023-07-14T16:14:20.663585+0000","last_fullsized":"2023-07-14T16:14:20.663585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:00.081710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665002+0000","last_change":"2023-07-14T16:13:59.301036+0000","last_active":"2023-07-14T16:14:20.665002+0000","last_peered":"2023-07-14T16:14:20.665002+0000","last_clean":"2023-07-14T16:14:20.665002+0000","last_became_active":"2023-07-14T16:13:59.300787+0000","last_became_peered":"2023-07-14T16:13:59.300787+0000","last_unstale":"2023-07-14T16:14:20.665002+0000","last_undegraded":"2023-07-14T16:14:20.665002+0000","last_fullsized":"2023-07-14T16:14:20.665002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:40:56.654793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:46.635519+0000","last_change":"2023-07-14T16:13:58.282392+0000","last_active":"2023-07-14T16:14:46.635519+0000","last_peered":"2023-07-14T16:14:46.635519+0000","last_clean":"2023-07-14T16:14:46.635519+0000","last_became_active":"2023-07-14T16:13:58.282030+0000","last_became_peered":"2023-07-14T16:13:58.282030+0000","last_unstale":"2023-07-14T16:14:46.635519+0000","last_undegraded":"2023-07-14T16:14:46.635519+0000","last_fullsized":"2023-07-14T16:14:46.635519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:17.795386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663473+0000","last_change":"2023-07-14T16:13:59.292681+0000","last_active":"2023-07-14T16:14:20.663473+0000","last_peered":"2023-07-14T16:14:20.663473+0000","last_clean":"2023-07-14T16:14:20.663473+0000","last_became_active":"2023-07-14T16:13:59.292289+0000","last_became_peered":"2023-07-14T16:13:59.292289+0000","last_unstale":"2023-07-14T16:14:20.663473+0000","last_undegraded":"2023-07-14T16:14:20.663473+0000","last_fullsized":"2023-07-14T16:14:20.663473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:33.971255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666548+0000","last_change":"2023-07-14T16:13:58.282111+0000","last_active":"2023-07-14T16:14:20.666548+0000","last_peered":"2023-07-14T16:14:20.666548+0000","last_clean":"2023-07-14T16:14:20.666548+0000","last_became_active":"2023-07-14T16:13:58.281783+0000","last_became_peered":"2023-07-14T16:13:58.281783+0000","last_unstale":"2023-07-14T16:14:20.666548+0000","last_undegraded":"2023-07-14T16:14:20.666548+0000","last_fullsized":"2023-07-14T16:14:20.666548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:51:40.718143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664212+0000","last_change":"2023-07-14T16:13:59.297439+0000","last_active":"2023-07-14T16:14:20.664212+0000","last_peered":"2023-07-14T16:14:20.664212+0000","last_clean":"2023-07-14T16:14:20.664212+0000","last_became_active":"2023-07-14T16:13:59.297117+0000","last_became_peered":"2023-07-14T16:13:59.297117+0000","last_unstale":"2023-07-14T16:14:20.664212+0000","last_undegraded":"2023-07-14T16:14:20.664212+0000","last_fullsized":"2023-07-14T16:14:20.664212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:08.122607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763285+0000","last_change":"2023-07-14T16:13:58.274826+0000","last_active":"2023-07-14T16:14:20.763285+0000","last_peered":"2023-07-14T16:14:20.763285+0000","last_clean":"2023-07-14T16:14:20.763285+0000","last_became_active":"2023-07-14T16:13:58.274406+0000","last_became_peered":"2023-07-14T16:13:58.274406+0000","last_unstale":"2023-07-14T16:14:20.763285+0000","last_undegraded":"2023-07-14T16:14:20.763285+0000","last_fullsized":"2023-07-14T16:14:20.763285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:09.422218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663188+0000","last_change":"2023-07-14T16:13:59.296730+0000","last_active":"2023-07-14T16:14:20.663188+0000","last_peered":"2023-07-14T16:14:20.663188+0000","last_clean":"2023-07-14T16:14:20.663188+0000","last_became_active":"2023-07-14T16:13:59.296372+0000","last_became_peered":"2023-07-14T16:13:59.296372+0000","last_unstale":"2023-07-14T16:14:20.663188+0000","last_undegraded":"2023-07-14T16:14:20.663188+0000","last_fullsized":"2023-07-14T16:14:20.663188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:07.809329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664772+0000","last_change":"2023-07-14T16:13:58.269122+0000","last_active":"2023-07-14T16:14:20.664772+0000","last_peered":"2023-07-14T16:14:20.664772+0000","last_clean":"2023-07-14T16:14:20.664772+0000","last_became_active":"2023-07-14T16:13:58.268770+0000","last_became_peered":"2023-07-14T16:13:58.268770+0000","last_unstale":"2023-07-14T16:14:20.664772+0000","last_undegraded":"2023-07-14T16:14:20.664772+0000","last_fullsized":"2023-07-14T16:14:20.664772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:43.584997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666674+0000","last_change":"2023-07-14T16:13:59.306987+0000","last_active":"2023-07-14T16:14:20.666674+0000","last_peered":"2023-07-14T16:14:20.666674+0000","last_clean":"2023-07-14T16:14:20.666674+0000","last_became_active":"2023-07-14T16:13:59.306649+0000","last_became_peered":"2023-07-14T16:13:59.306649+0000","last_unstale":"2023-07-14T16:14:20.666674+0000","last_undegraded":"2023-07-14T16:14:20.666674+0000","last_fullsized":"2023-07-14T16:14:20.666674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:35:23.579814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:46.634803+0000","last_change":"2023-07-14T16:13:58.281319+0000","last_active":"2023-07-14T16:14:46.634803+0000","last_peered":"2023-07-14T16:14:46.634803+0000","last_clean":"2023-07-14T16:14:46.634803+0000","last_became_active":"2023-07-14T16:13:58.280928+0000","last_became_peered":"2023-07-14T16:13:58.280928+0000","last_unstale":"2023-07-14T16:14:46.634803+0000","last_undegraded":"2023-07-14T16:14:46.634803+0000","last_fullsized":"2023-07-14T16:14:46.634803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:41.337954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763717+0000","last_change":"2023-07-14T16:13:59.309586+0000","last_active":"2023-07-14T16:14:20.763717+0000","last_peered":"2023-07-14T16:14:20.763717+0000","last_clean":"2023-07-14T16:14:20.763717+0000","last_became_active":"2023-07-14T16:13:59.309243+0000","last_became_peered":"2023-07-14T16:13:59.309243+0000","last_unstale":"2023-07-14T16:14:20.763717+0000","last_undegraded":"2023-07-14T16:14:20.763717+0000","last_fullsized":"2023-07-14T16:14:20.763717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:30.700680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661328+0000","last_change":"2023-07-14T16:13:58.279769+0000","last_active":"2023-07-14T16:14:20.661328+0000","last_peered":"2023-07-14T16:14:20.661328+0000","last_clean":"2023-07-14T16:14:20.661328+0000","last_became_active":"2023-07-14T16:13:58.279409+0000","last_became_peered":"2023-07-14T16:13:58.279409+0000","last_unstale":"2023-07-14T16:14:20.661328+0000","last_undegraded":"2023-07-14T16:14:20.661328+0000","last_fullsized":"2023-07-14T16:14:20.661328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:32.751188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666121+0000","last_change":"2023-07-14T16:13:59.300382+0000","last_active":"2023-07-14T16:14:20.666121+0000","last_peered":"2023-07-14T16:14:20.666121+0000","last_clean":"2023-07-14T16:14:20.666121+0000","last_became_active":"2023-07-14T16:13:59.299981+0000","last_became_peered":"2023-07-14T16:13:59.299981+0000","last_unstale":"2023-07-14T16:14:20.666121+0000","last_undegraded":"2023-07-14T16:14:20.666121+0000","last_fullsized":"2023-07-14T16:14:20.666121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:46:04.769609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665031+0000","last_change":"2023-07-14T16:13:58.274128+0000","last_active":"2023-07-14T16:14:20.665031+0000","last_peered":"2023-07-14T16:14:20.665031+0000","last_clean":"2023-07-14T16:14:20.665031+0000","last_became_active":"2023-07-14T16:13:58.273446+0000","last_became_peered":"2023-07-14T16:13:58.273446+0000","last_unstale":"2023-07-14T16:14:20.665031+0000","last_undegraded":"2023-07-14T16:14:20.665031+0000","last_fullsized":"2023-07-14T16:14:20.665031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:12.510793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763503+0000","last_change":"2023-07-14T16:13:59.301551+0000","last_active":"2023-07-14T16:14:20.763503+0000","last_peered":"2023-07-14T16:14:20.763503+0000","last_clean":"2023-07-14T16:14:20.763503+0000","last_became_active":"2023-07-14T16:13:59.301206+0000","last_became_peered":"2023-07-14T16:13:59.301206+0000","last_unstale":"2023-07-14T16:14:20.763503+0000","last_undegraded":"2023-07-14T16:14:20.763503+0000","last_fullsized":"2023-07-14T16:14:20.763503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:16.227849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":141,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:47.636086+0000","last_change":"2023-07-14T16:13:58.283304+0000","last_active":"2023-07-14T16:14:47.636086+0000","last_peered":"2023-07-14T16:14:47.636086+0000","last_clean":"2023-07-14T16:14:47.636086+0000","last_became_active":"2023-07-14T16:13:58.282930+0000","last_became_peered":"2023-07-14T16:13:58.282930+0000","last_unstale":"2023-07-14T16:14:47.636086+0000","last_undegraded":"2023-07-14T16:14:47.636086+0000","last_fullsized":"2023-07-14T16:14:47.636086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:02.538305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57531,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":97,"num_read_kb":104,"num_write":27,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664117+0000","last_change":"2023-07-14T16:13:59.304144+0000","last_active":"2023-07-14T16:14:20.664117+0000","last_peered":"2023-07-14T16:14:20.664117+0000","last_clean":"2023-07-14T16:14:20.664117+0000","last_became_active":"2023-07-14T16:13:59.303745+0000","last_became_peered":"2023-07-14T16:13:59.303745+0000","last_unstale":"2023-07-14T16:14:20.664117+0000","last_undegraded":"2023-07-14T16:14:20.664117+0000","last_fullsized":"2023-07-14T16:14:20.664117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:20.828221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667830+0000","last_change":"2023-07-14T16:13:58.283734+0000","last_active":"2023-07-14T16:14:20.667830+0000","last_peered":"2023-07-14T16:14:20.667830+0000","last_clean":"2023-07-14T16:14:20.667830+0000","last_became_active":"2023-07-14T16:13:58.282765+0000","last_became_peered":"2023-07-14T16:13:58.282765+0000","last_unstale":"2023-07-14T16:14:20.667830+0000","last_undegraded":"2023-07-14T16:14:20.667830+0000","last_fullsized":"2023-07-14T16:14:20.667830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:13.217664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763262+0000","last_change":"2023-07-14T16:13:59.308445+0000","last_active":"2023-07-14T16:14:20.763262+0000","last_peered":"2023-07-14T16:14:20.763262+0000","last_clean":"2023-07-14T16:14:20.763262+0000","last_became_active":"2023-07-14T16:13:59.308130+0000","last_became_peered":"2023-07-14T16:13:59.308130+0000","last_unstale":"2023-07-14T16:14:20.763262+0000","last_undegraded":"2023-07-14T16:14:20.763262+0000","last_fullsized":"2023-07-14T16:14:20.763262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:45.004512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664202+0000","last_change":"2023-07-14T16:13:58.273973+0000","last_active":"2023-07-14T16:14:20.664202+0000","last_peered":"2023-07-14T16:14:20.664202+0000","last_clean":"2023-07-14T16:14:20.664202+0000","last_became_active":"2023-07-14T16:13:58.273606+0000","last_became_peered":"2023-07-14T16:13:58.273606+0000","last_unstale":"2023-07-14T16:14:20.664202+0000","last_undegraded":"2023-07-14T16:14:20.664202+0000","last_fullsized":"2023-07-14T16:14:20.664202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:12.304491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661141+0000","last_change":"2023-07-14T16:13:59.306428+0000","last_active":"2023-07-14T16:14:20.661141+0000","last_peered":"2023-07-14T16:14:20.661141+0000","last_clean":"2023-07-14T16:14:20.661141+0000","last_became_active":"2023-07-14T16:13:59.306090+0000","last_became_peered":"2023-07-14T16:13:59.306090+0000","last_unstale":"2023-07-14T16:14:20.661141+0000","last_undegraded":"2023-07-14T16:14:20.661141+0000","last_fullsized":"2023-07-14T16:14:20.661141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:16.033202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:48.692781+0000","last_change":"2023-07-14T16:13:58.277123+0000","last_active":"2023-07-14T16:14:48.692781+0000","last_peered":"2023-07-14T16:14:48.692781+0000","last_clean":"2023-07-14T16:14:48.692781+0000","last_became_active":"2023-07-14T16:13:58.276815+0000","last_became_peered":"2023-07-14T16:13:58.276815+0000","last_unstale":"2023-07-14T16:14:48.692781+0000","last_undegraded":"2023-07-14T16:14:48.692781+0000","last_fullsized":"2023-07-14T16:14:48.692781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:41.019233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660748+0000","last_change":"2023-07-14T16:13:59.310238+0000","last_active":"2023-07-14T16:14:20.660748+0000","last_peered":"2023-07-14T16:14:20.660748+0000","last_clean":"2023-07-14T16:14:20.660748+0000","last_became_active":"2023-07-14T16:13:59.309918+0000","last_became_peered":"2023-07-14T16:13:59.309918+0000","last_unstale":"2023-07-14T16:14:20.660748+0000","last_undegraded":"2023-07-14T16:14:20.660748+0000","last_fullsized":"2023-07-14T16:14:20.660748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:25.020575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663660+0000","last_change":"2023-07-14T16:13:58.280834+0000","last_active":"2023-07-14T16:14:20.663660+0000","last_peered":"2023-07-14T16:14:20.663660+0000","last_clean":"2023-07-14T16:14:20.663660+0000","last_became_active":"2023-07-14T16:13:58.280497+0000","last_became_peered":"2023-07-14T16:13:58.280497+0000","last_unstale":"2023-07-14T16:14:20.663660+0000","last_undegraded":"2023-07-14T16:14:20.663660+0000","last_fullsized":"2023-07-14T16:14:20.663660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:07:02.308388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.765253+0000","last_change":"2023-07-14T16:14:20.764846+0000","last_active":"2023-07-14T16:14:20.765253+0000","last_peered":"2023-07-14T16:14:20.765253+0000","last_clean":"2023-07-14T16:14:20.765253+0000","last_became_active":"2023-07-14T16:14:20.764485+0000","last_became_peered":"2023-07-14T16:14:20.764485+0000","last_unstale":"2023-07-14T16:14:20.765253+0000","last_undegraded":"2023-07-14T16:14:20.765253+0000","last_fullsized":"2023-07-14T16:14:20.765253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:29.245874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.682843+0000","last_change":"2023-07-14T16:14:20.682331+0000","last_active":"2023-07-14T16:14:20.682843+0000","last_peered":"2023-07-14T16:14:20.682843+0000","last_clean":"2023-07-14T16:14:20.682843+0000","last_became_active":"2023-07-14T16:14:20.669179+0000","last_became_peered":"2023-07-14T16:14:20.669179+0000","last_unstale":"2023-07-14T16:14:20.682843+0000","last_undegraded":"2023-07-14T16:14:20.682843+0000","last_fullsized":"2023-07-14T16:14:20.682843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:55:10.629011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660894+0000","last_change":"2023-07-14T16:13:59.297644+0000","last_active":"2023-07-14T16:14:20.660894+0000","last_peered":"2023-07-14T16:14:20.660894+0000","last_clean":"2023-07-14T16:14:20.660894+0000","last_became_active":"2023-07-14T16:13:59.297332+0000","last_became_peered":"2023-07-14T16:13:59.297332+0000","last_unstale":"2023-07-14T16:14:20.660894+0000","last_undegraded":"2023-07-14T16:14:20.660894+0000","last_fullsized":"2023-07-14T16:14:20.660894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:36.421075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665695+0000","last_change":"2023-07-14T16:13:58.280632+0000","last_active":"2023-07-14T16:14:20.665695+0000","last_peered":"2023-07-14T16:14:20.665695+0000","last_clean":"2023-07-14T16:14:20.665695+0000","last_became_active":"2023-07-14T16:13:58.280296+0000","last_became_peered":"2023-07-14T16:13:58.280296+0000","last_unstale":"2023-07-14T16:14:20.665695+0000","last_undegraded":"2023-07-14T16:14:20.665695+0000","last_fullsized":"2023-07-14T16:14:20.665695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:26.897678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666919+0000","last_change":"2023-07-14T16:13:59.297092+0000","last_active":"2023-07-14T16:14:20.666919+0000","last_peered":"2023-07-14T16:14:20.666919+0000","last_clean":"2023-07-14T16:14:20.666919+0000","last_became_active":"2023-07-14T16:13:59.296750+0000","last_became_peered":"2023-07-14T16:13:59.296750+0000","last_unstale":"2023-07-14T16:14:20.666919+0000","last_undegraded":"2023-07-14T16:14:20.666919+0000","last_fullsized":"2023-07-14T16:14:20.666919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:32.797912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660888+0000","last_change":"2023-07-14T16:13:58.281786+0000","last_active":"2023-07-14T16:14:20.660888+0000","last_peered":"2023-07-14T16:14:20.660888+0000","last_clean":"2023-07-14T16:14:20.660888+0000","last_became_active":"2023-07-14T16:13:58.281471+0000","last_became_peered":"2023-07-14T16:13:58.281471+0000","last_unstale":"2023-07-14T16:14:20.660888+0000","last_undegraded":"2023-07-14T16:14:20.660888+0000","last_fullsized":"2023-07-14T16:14:20.660888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:09.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.672813+0000","last_change":"2023-07-14T16:14:20.672813+0000","last_active":"2023-07-14T16:14:20.672813+0000","last_peered":"2023-07-14T16:14:20.672813+0000","last_clean":"2023-07-14T16:14:20.672813+0000","last_became_active":"2023-07-14T16:14:20.672447+0000","last_became_peered":"2023-07-14T16:14:20.672447+0000","last_unstale":"2023-07-14T16:14:20.672813+0000","last_undegraded":"2023-07-14T16:14:20.672813+0000","last_fullsized":"2023-07-14T16:14:20.672813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:33:07.582055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665347+0000","last_change":"2023-07-14T16:13:59.288836+0000","last_active":"2023-07-14T16:14:20.665347+0000","last_peered":"2023-07-14T16:14:20.665347+0000","last_clean":"2023-07-14T16:14:20.665347+0000","last_became_active":"2023-07-14T16:13:59.288517+0000","last_became_peered":"2023-07-14T16:13:59.288517+0000","last_unstale":"2023-07-14T16:14:20.665347+0000","last_undegraded":"2023-07-14T16:14:20.665347+0000","last_fullsized":"2023-07-14T16:14:20.665347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:48.293313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":67,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:47.690051+0000","last_change":"2023-07-14T16:13:58.282415+0000","last_active":"2023-07-14T16:14:47.690051+0000","last_peered":"2023-07-14T16:14:47.690051+0000","last_clean":"2023-07-14T16:14:47.690051+0000","last_became_active":"2023-07-14T16:13:58.282118+0000","last_became_peered":"2023-07-14T16:13:58.282118+0000","last_unstale":"2023-07-14T16:14:47.690051+0000","last_undegraded":"2023-07-14T16:14:47.690051+0000","last_fullsized":"2023-07-14T16:14:47.690051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:41.532341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663573+0000","last_change":"2023-07-14T16:13:59.302860+0000","last_active":"2023-07-14T16:14:20.663573+0000","last_peered":"2023-07-14T16:14:20.663573+0000","last_clean":"2023-07-14T16:14:20.663573+0000","last_became_active":"2023-07-14T16:13:59.302454+0000","last_became_peered":"2023-07-14T16:13:59.302454+0000","last_unstale":"2023-07-14T16:14:20.663573+0000","last_undegraded":"2023-07-14T16:14:20.663573+0000","last_fullsized":"2023-07-14T16:14:20.663573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:34.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664397+0000","last_change":"2023-07-14T16:13:58.273908+0000","last_active":"2023-07-14T16:14:20.664397+0000","last_peered":"2023-07-14T16:14:20.664397+0000","last_clean":"2023-07-14T16:14:20.664397+0000","last_became_active":"2023-07-14T16:13:58.273497+0000","last_became_peered":"2023-07-14T16:13:58.273497+0000","last_unstale":"2023-07-14T16:14:20.664397+0000","last_undegraded":"2023-07-14T16:14:20.664397+0000","last_fullsized":"2023-07-14T16:14:20.664397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:55:44.978027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660941+0000","last_change":"2023-07-14T16:12:36.546475+0000","last_active":"2023-07-14T16:14:20.660941+0000","last_peered":"2023-07-14T16:14:20.660941+0000","last_clean":"2023-07-14T16:14:20.660941+0000","last_became_active":"2023-07-14T16:12:36.545943+0000","last_became_peered":"2023-07-14T16:12:36.545943+0000","last_unstale":"2023-07-14T16:14:20.660941+0000","last_undegraded":"2023-07-14T16:14:20.660941+0000","last_fullsized":"2023-07-14T16:14:20.660941+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_clean_scrub_stamp":"2023-07-14T16:08:58.024408+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:27.166990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763112+0000","last_change":"2023-07-14T16:13:59.297669+0000","last_active":"2023-07-14T16:14:20.763112+0000","last_peered":"2023-07-14T16:14:20.763112+0000","last_clean":"2023-07-14T16:14:20.763112+0000","last_became_active":"2023-07-14T16:13:59.297364+0000","last_became_peered":"2023-07-14T16:13:59.297364+0000","last_unstale":"2023-07-14T16:14:20.763112+0000","last_undegraded":"2023-07-14T16:14:20.763112+0000","last_fullsized":"2023-07-14T16:14:20.763112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:50:00.972727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.673166+0000","last_change":"2023-07-14T16:14:20.673166+0000","last_active":"2023-07-14T16:14:20.673166+0000","last_peered":"2023-07-14T16:14:20.673166+0000","last_clean":"2023-07-14T16:14:20.673166+0000","last_became_active":"2023-07-14T16:14:20.667664+0000","last_became_peered":"2023-07-14T16:14:20.667664+0000","last_unstale":"2023-07-14T16:14:20.673166+0000","last_undegraded":"2023-07-14T16:14:20.673166+0000","last_fullsized":"2023-07-14T16:14:20.673166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:24.328768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664285+0000","last_change":"2023-07-14T16:13:58.276257+0000","last_active":"2023-07-14T16:14:20.664285+0000","last_peered":"2023-07-14T16:14:20.664285+0000","last_clean":"2023-07-14T16:14:20.664285+0000","last_became_active":"2023-07-14T16:13:58.275910+0000","last_became_peered":"2023-07-14T16:13:58.275910+0000","last_unstale":"2023-07-14T16:14:20.664285+0000","last_undegraded":"2023-07-14T16:14:20.664285+0000","last_fullsized":"2023-07-14T16:14:20.664285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:47.700225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763397+0000","last_change":"2023-07-14T16:13:59.299135+0000","last_active":"2023-07-14T16:14:20.763397+0000","last_peered":"2023-07-14T16:14:20.763397+0000","last_clean":"2023-07-14T16:14:20.763397+0000","last_became_active":"2023-07-14T16:13:59.298031+0000","last_became_peered":"2023-07-14T16:13:59.298031+0000","last_unstale":"2023-07-14T16:14:20.763397+0000","last_undegraded":"2023-07-14T16:14:20.763397+0000","last_fullsized":"2023-07-14T16:14:20.763397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:21.882960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665374+0000","last_change":"2023-07-14T16:13:58.279664+0000","last_active":"2023-07-14T16:14:20.665374+0000","last_peered":"2023-07-14T16:14:20.665374+0000","last_clean":"2023-07-14T16:14:20.665374+0000","last_became_active":"2023-07-14T16:13:58.279137+0000","last_became_peered":"2023-07-14T16:13:58.279137+0000","last_unstale":"2023-07-14T16:14:20.665374+0000","last_undegraded":"2023-07-14T16:14:20.665374+0000","last_fullsized":"2023-07-14T16:14:20.665374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:07.657781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663754+0000","last_change":"2023-07-14T16:13:59.305536+0000","last_active":"2023-07-14T16:14:20.663754+0000","last_peered":"2023-07-14T16:14:20.663754+0000","last_clean":"2023-07-14T16:14:20.663754+0000","last_became_active":"2023-07-14T16:13:59.305176+0000","last_became_peered":"2023-07-14T16:13:59.305176+0000","last_unstale":"2023-07-14T16:14:20.663754+0000","last_undegraded":"2023-07-14T16:14:20.663754+0000","last_fullsized":"2023-07-14T16:14:20.663754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:18:18.141957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664829+0000","last_change":"2023-07-14T16:13:58.269652+0000","last_active":"2023-07-14T16:14:20.664829+0000","last_peered":"2023-07-14T16:14:20.664829+0000","last_clean":"2023-07-14T16:14:20.664829+0000","last_became_active":"2023-07-14T16:13:58.269338+0000","last_became_peered":"2023-07-14T16:13:58.269338+0000","last_unstale":"2023-07-14T16:14:20.664829+0000","last_undegraded":"2023-07-14T16:14:20.664829+0000","last_fullsized":"2023-07-14T16:14:20.664829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:55:21.910971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665994+0000","last_change":"2023-07-14T16:13:59.297574+0000","last_active":"2023-07-14T16:14:20.665994+0000","last_peered":"2023-07-14T16:14:20.665994+0000","last_clean":"2023-07-14T16:14:20.665994+0000","last_became_active":"2023-07-14T16:13:59.297124+0000","last_became_peered":"2023-07-14T16:13:59.297124+0000","last_unstale":"2023-07-14T16:14:20.665994+0000","last_undegraded":"2023-07-14T16:14:20.665994+0000","last_fullsized":"2023-07-14T16:14:20.665994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:20.051395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763343+0000","last_change":"2023-07-14T16:13:58.278011+0000","last_active":"2023-07-14T16:14:20.763343+0000","last_peered":"2023-07-14T16:14:20.763343+0000","last_clean":"2023-07-14T16:14:20.763343+0000","last_became_active":"2023-07-14T16:13:58.277563+0000","last_became_peered":"2023-07-14T16:13:58.277563+0000","last_unstale":"2023-07-14T16:14:20.763343+0000","last_undegraded":"2023-07-14T16:14:20.763343+0000","last_fullsized":"2023-07-14T16:14:20.763343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:06.354928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.662905+0000","last_change":"2023-07-14T16:13:59.308836+0000","last_active":"2023-07-14T16:14:20.662905+0000","last_peered":"2023-07-14T16:14:20.662905+0000","last_clean":"2023-07-14T16:14:20.662905+0000","last_became_active":"2023-07-14T16:13:59.308510+0000","last_became_peered":"2023-07-14T16:13:59.308510+0000","last_unstale":"2023-07-14T16:14:20.662905+0000","last_undegraded":"2023-07-14T16:14:20.662905+0000","last_fullsized":"2023-07-14T16:14:20.662905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:48:07.973849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663352+0000","last_change":"2023-07-14T16:13:58.277438+0000","last_active":"2023-07-14T16:14:20.663352+0000","last_peered":"2023-07-14T16:14:20.663352+0000","last_clean":"2023-07-14T16:14:20.663352+0000","last_became_active":"2023-07-14T16:13:58.277114+0000","last_became_peered":"2023-07-14T16:13:58.277114+0000","last_unstale":"2023-07-14T16:14:20.663352+0000","last_undegraded":"2023-07-14T16:14:20.663352+0000","last_fullsized":"2023-07-14T16:14:20.663352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:40.636429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660728+0000","last_change":"2023-07-14T16:13:59.304491+0000","last_active":"2023-07-14T16:14:20.660728+0000","last_peered":"2023-07-14T16:14:20.660728+0000","last_clean":"2023-07-14T16:14:20.660728+0000","last_became_active":"2023-07-14T16:13:59.303925+0000","last_became_peered":"2023-07-14T16:13:59.303925+0000","last_unstale":"2023-07-14T16:14:20.660728+0000","last_undegraded":"2023-07-14T16:14:20.660728+0000","last_fullsized":"2023-07-14T16:14:20.660728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.410847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.685314+0000","last_change":"2023-07-14T16:14:20.684842+0000","last_active":"2023-07-14T16:14:20.685314+0000","last_peered":"2023-07-14T16:14:20.685314+0000","last_clean":"2023-07-14T16:14:20.685314+0000","last_became_active":"2023-07-14T16:14:20.670098+0000","last_became_peered":"2023-07-14T16:14:20.670098+0000","last_unstale":"2023-07-14T16:14:20.685314+0000","last_undegraded":"2023-07-14T16:14:20.685314+0000","last_fullsized":"2023-07-14T16:14:20.685314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:54:58.021152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663708+0000","last_change":"2023-07-14T16:13:59.310983+0000","last_active":"2023-07-14T16:14:20.663708+0000","last_peered":"2023-07-14T16:14:20.663708+0000","last_clean":"2023-07-14T16:14:20.663708+0000","last_became_active":"2023-07-14T16:13:59.310636+0000","last_became_peered":"2023-07-14T16:13:59.310636+0000","last_unstale":"2023-07-14T16:14:20.663708+0000","last_undegraded":"2023-07-14T16:14:20.663708+0000","last_fullsized":"2023-07-14T16:14:20.663708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:31.003517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663068+0000","last_change":"2023-07-14T16:13:58.280153+0000","last_active":"2023-07-14T16:14:20.663068+0000","last_peered":"2023-07-14T16:14:20.663068+0000","last_clean":"2023-07-14T16:14:20.663068+0000","last_became_active":"2023-07-14T16:13:58.279771+0000","last_became_peered":"2023-07-14T16:13:58.279771+0000","last_unstale":"2023-07-14T16:14:20.663068+0000","last_undegraded":"2023-07-14T16:14:20.663068+0000","last_fullsized":"2023-07-14T16:14:20.663068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:46.290429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660909+0000","last_change":"2023-07-14T16:13:59.304854+0000","last_active":"2023-07-14T16:14:20.660909+0000","last_peered":"2023-07-14T16:14:20.660909+0000","last_clean":"2023-07-14T16:14:20.660909+0000","last_became_active":"2023-07-14T16:13:59.304374+0000","last_became_peered":"2023-07-14T16:13:59.304374+0000","last_unstale":"2023-07-14T16:14:20.660909+0000","last_undegraded":"2023-07-14T16:14:20.660909+0000","last_fullsized":"2023-07-14T16:14:20.660909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:52.213737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667834+0000","last_change":"2023-07-14T16:13:58.281885+0000","last_active":"2023-07-14T16:14:20.667834+0000","last_peered":"2023-07-14T16:14:20.667834+0000","last_clean":"2023-07-14T16:14:20.667834+0000","last_became_active":"2023-07-14T16:13:58.278759+0000","last_became_peered":"2023-07-14T16:13:58.278759+0000","last_unstale":"2023-07-14T16:14:20.667834+0000","last_undegraded":"2023-07-14T16:14:20.667834+0000","last_fullsized":"2023-07-14T16:14:20.667834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:11.816765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661411+0000","last_change":"2023-07-14T16:13:59.302680+0000","last_active":"2023-07-14T16:14:20.661411+0000","last_peered":"2023-07-14T16:14:20.661411+0000","last_clean":"2023-07-14T16:14:20.661411+0000","last_became_active":"2023-07-14T16:13:59.301854+0000","last_became_peered":"2023-07-14T16:13:59.301854+0000","last_unstale":"2023-07-14T16:14:20.661411+0000","last_undegraded":"2023-07-14T16:14:20.661411+0000","last_fullsized":"2023-07-14T16:14:20.661411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:29:42.861394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664860+0000","last_change":"2023-07-14T16:13:58.274254+0000","last_active":"2023-07-14T16:14:20.664860+0000","last_peered":"2023-07-14T16:14:20.664860+0000","last_clean":"2023-07-14T16:14:20.664860+0000","last_became_active":"2023-07-14T16:13:58.273933+0000","last_became_peered":"2023-07-14T16:13:58.273933+0000","last_unstale":"2023-07-14T16:14:20.664860+0000","last_undegraded":"2023-07-14T16:14:20.664860+0000","last_fullsized":"2023-07-14T16:14:20.664860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:53.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661253+0000","last_change":"2023-07-14T16:13:59.305615+0000","last_active":"2023-07-14T16:14:20.661253+0000","last_peered":"2023-07-14T16:14:20.661253+0000","last_clean":"2023-07-14T16:14:20.661253+0000","last_became_active":"2023-07-14T16:13:59.305206+0000","last_became_peered":"2023-07-14T16:13:59.305206+0000","last_unstale":"2023-07-14T16:14:20.661253+0000","last_undegraded":"2023-07-14T16:14:20.661253+0000","last_fullsized":"2023-07-14T16:14:20.661253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:41.771061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763463+0000","last_change":"2023-07-14T16:13:58.277976+0000","last_active":"2023-07-14T16:14:20.763463+0000","last_peered":"2023-07-14T16:14:20.763463+0000","last_clean":"2023-07-14T16:14:20.763463+0000","last_became_active":"2023-07-14T16:13:58.277618+0000","last_became_peered":"2023-07-14T16:13:58.277618+0000","last_unstale":"2023-07-14T16:14:20.763463+0000","last_undegraded":"2023-07-14T16:14:20.763463+0000","last_fullsized":"2023-07-14T16:14:20.763463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:25.780591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661191+0000","last_change":"2023-07-14T16:13:59.302041+0000","last_active":"2023-07-14T16:14:20.661191+0000","last_peered":"2023-07-14T16:14:20.661191+0000","last_clean":"2023-07-14T16:14:20.661191+0000","last_became_active":"2023-07-14T16:13:59.301620+0000","last_became_peered":"2023-07-14T16:13:59.301620+0000","last_unstale":"2023-07-14T16:14:20.661191+0000","last_undegraded":"2023-07-14T16:14:20.661191+0000","last_fullsized":"2023-07-14T16:14:20.661191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:18.969064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660855+0000","last_change":"2023-07-14T16:13:59.304222+0000","last_active":"2023-07-14T16:14:20.660855+0000","last_peered":"2023-07-14T16:14:20.660855+0000","last_clean":"2023-07-14T16:14:20.660855+0000","last_became_active":"2023-07-14T16:13:59.303404+0000","last_became_peered":"2023-07-14T16:13:59.303404+0000","last_unstale":"2023-07-14T16:14:20.660855+0000","last_undegraded":"2023-07-14T16:14:20.660855+0000","last_fullsized":"2023-07-14T16:14:20.660855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:26.218664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666713+0000","last_change":"2023-07-14T16:13:59.296433+0000","last_active":"2023-07-14T16:14:20.666713+0000","last_peered":"2023-07-14T16:14:20.666713+0000","last_clean":"2023-07-14T16:14:20.666713+0000","last_became_active":"2023-07-14T16:13:59.295604+0000","last_became_peered":"2023-07-14T16:13:59.295604+0000","last_unstale":"2023-07-14T16:14:20.666713+0000","last_undegraded":"2023-07-14T16:14:20.666713+0000","last_fullsized":"2023-07-14T16:14:20.666713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:17:57.269484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667209+0000","last_change":"2023-07-14T16:13:59.294198+0000","last_active":"2023-07-14T16:14:20.667209+0000","last_peered":"2023-07-14T16:14:20.667209+0000","last_clean":"2023-07-14T16:14:20.667209+0000","last_became_active":"2023-07-14T16:13:59.293891+0000","last_became_peered":"2023-07-14T16:13:59.293891+0000","last_unstale":"2023-07-14T16:14:20.667209+0000","last_undegraded":"2023-07-14T16:14:20.667209+0000","last_fullsized":"2023-07-14T16:14:20.667209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:41.192604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665734+0000","last_change":"2023-07-14T16:13:59.298724+0000","last_active":"2023-07-14T16:14:20.665734+0000","last_peered":"2023-07-14T16:14:20.665734+0000","last_clean":"2023-07-14T16:14:20.665734+0000","last_became_active":"2023-07-14T16:13:59.298318+0000","last_became_peered":"2023-07-14T16:13:59.298318+0000","last_unstale":"2023-07-14T16:14:20.665734+0000","last_undegraded":"2023-07-14T16:14:20.665734+0000","last_fullsized":"2023-07-14T16:14:20.665734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:30:13.650949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763466+0000","last_change":"2023-07-14T16:13:59.294652+0000","last_active":"2023-07-14T16:14:20.763466+0000","last_peered":"2023-07-14T16:14:20.763466+0000","last_clean":"2023-07-14T16:14:20.763466+0000","last_became_active":"2023-07-14T16:13:59.294278+0000","last_became_peered":"2023-07-14T16:13:59.294278+0000","last_unstale":"2023-07-14T16:14:20.763466+0000","last_undegraded":"2023-07-14T16:14:20.763466+0000","last_fullsized":"2023-07-14T16:14:20.763466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:45:58.125276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660903+0000","last_change":"2023-07-14T16:13:59.299228+0000","last_active":"2023-07-14T16:14:20.660903+0000","last_peered":"2023-07-14T16:14:20.660903+0000","last_clean":"2023-07-14T16:14:20.660903+0000","last_became_active":"2023-07-14T16:13:59.297982+0000","last_became_peered":"2023-07-14T16:13:59.297982+0000","last_unstale":"2023-07-14T16:14:20.660903+0000","last_undegraded":"2023-07-14T16:14:20.660903+0000","last_fullsized":"2023-07-14T16:14:20.660903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:24.432565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.668368+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:14:20.668368+0000","last_peered":"2023-07-14T16:14:20.668368+0000","last_clean":"2023-07-14T16:14:20.668368+0000","last_became_active":"2023-07-14T16:13:59.299181+0000","last_became_peered":"2023-07-14T16:13:59.299181+0000","last_unstale":"2023-07-14T16:14:20.668368+0000","last_undegraded":"2023-07-14T16:14:20.668368+0000","last_fullsized":"2023-07-14T16:14:20.668368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:42.604278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664195+0000","last_change":"2023-07-14T16:13:59.303282+0000","last_active":"2023-07-14T16:14:20.664195+0000","last_peered":"2023-07-14T16:14:20.664195+0000","last_clean":"2023-07-14T16:14:20.664195+0000","last_became_active":"2023-07-14T16:13:59.302922+0000","last_became_peered":"2023-07-14T16:13:59.302922+0000","last_unstale":"2023-07-14T16:14:20.664195+0000","last_undegraded":"2023-07-14T16:14:20.664195+0000","last_fullsized":"2023-07-14T16:14:20.664195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.750328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664788+0000","last_change":"2023-07-14T16:13:59.296389+0000","last_active":"2023-07-14T16:14:20.664788+0000","last_peered":"2023-07-14T16:14:20.664788+0000","last_clean":"2023-07-14T16:14:20.664788+0000","last_became_active":"2023-07-14T16:13:59.296080+0000","last_became_peered":"2023-07-14T16:13:59.296080+0000","last_unstale":"2023-07-14T16:14:20.664788+0000","last_undegraded":"2023-07-14T16:14:20.664788+0000","last_fullsized":"2023-07-14T16:14:20.664788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:02.060808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663247+0000","last_change":"2023-07-14T16:13:59.297774+0000","last_active":"2023-07-14T16:14:20.663247+0000","last_peered":"2023-07-14T16:14:20.663247+0000","last_clean":"2023-07-14T16:14:20.663247+0000","last_became_active":"2023-07-14T16:13:59.297463+0000","last_became_peered":"2023-07-14T16:13:59.297463+0000","last_unstale":"2023-07-14T16:14:20.663247+0000","last_undegraded":"2023-07-14T16:14:20.663247+0000","last_fullsized":"2023-07-14T16:14:20.663247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:49.304022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763413+0000","last_change":"2023-07-14T16:13:59.306795+0000","last_active":"2023-07-14T16:14:20.763413+0000","last_peered":"2023-07-14T16:14:20.763413+0000","last_clean":"2023-07-14T16:14:20.763413+0000","last_became_active":"2023-07-14T16:13:59.306457+0000","last_became_peered":"2023-07-14T16:13:59.306457+0000","last_unstale":"2023-07-14T16:14:20.763413+0000","last_undegraded":"2023-07-14T16:14:20.763413+0000","last_fullsized":"2023-07-14T16:14:20.763413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:47.873750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763333+0000","last_change":"2023-07-14T16:13:59.305338+0000","last_active":"2023-07-14T16:14:20.763333+0000","last_peered":"2023-07-14T16:14:20.763333+0000","last_clean":"2023-07-14T16:14:20.763333+0000","last_became_active":"2023-07-14T16:13:59.305047+0000","last_became_peered":"2023-07-14T16:13:59.305047+0000","last_unstale":"2023-07-14T16:14:20.763333+0000","last_undegraded":"2023-07-14T16:14:20.763333+0000","last_fullsized":"2023-07-14T16:14:20.763333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:07:02.694956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660864+0000","last_change":"2023-07-14T16:13:59.299005+0000","last_active":"2023-07-14T16:14:20.660864+0000","last_peered":"2023-07-14T16:14:20.660864+0000","last_clean":"2023-07-14T16:14:20.660864+0000","last_became_active":"2023-07-14T16:13:59.298705+0000","last_became_peered":"2023-07-14T16:13:59.298705+0000","last_unstale":"2023-07-14T16:14:20.660864+0000","last_undegraded":"2023-07-14T16:14:20.660864+0000","last_fullsized":"2023-07-14T16:14:20.660864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:59:41.986408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247144,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61435,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":296,"num_read_kb":212,"num_write":148,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":184305,"data_compressed":4809,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":459298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.5},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.86699999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.036}]},{"osd":6,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.95699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.161}]},{"osd":10,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.91600000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1089536,"data_stored":1108355,"data_compressed":37379,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":1,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75800000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1080000000000001,"5min":1.173,"15min":1.173},"last":0.72399999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.85099999999999998,"15min":0.84399999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.82599999999999996,"15min":0.81599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.4429999999999996,"5min":4.4429999999999996,"15min":4.4429999999999996},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92400000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.755,"15min":0.76600000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":1.613,"15min":1.613},"last":0.626}]},{"osd":5,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.94299999999999995,"15min":0.92200000000000004},"min":{"1min":0.62,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":4.9749999999999996,"5min":4.9749999999999996,"15min":4.9749999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.91500000000000004,"15min":0.92100000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5289999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.026,"15min":0.95599999999999996},"min":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":6.609,"5min":6.609,"15min":6.609},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.228,"5min":0.97699999999999998,"15min":0.93500000000000005},"min":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":6.5759999999999996,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.61699999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.98499999999999999,"15min":0.98599999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.996,"15min":1.012},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.159,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.90100000000000002}]},{"osd":9,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":0.99299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.1359999999999999,"15min":1.0660000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":7.3579999999999997,"5min":7.3579999999999997,"15min":7.3579999999999997},"last":0.95499999999999996}]},{"osd":11,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.018}]}]},{"osd":9,"up_from":68,"seq":292057776164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":459512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":1.492},{"interface":"front","average":{"1min":0.501,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70599999999999996,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82099999999999995,"15min":0.83299999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.032,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94199999999999995,"5min":1.129,"15min":1.129},"last":0.995}]},{"osd":2,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.86099999999999999,"15min":0.875},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.98899999999999999,"5min":1.151,"15min":1.151},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.64600000000000002,"15min":0.66400000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":1.6859999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.47399999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.228,"5min":1.268,"15min":1.268},"last":1.361}]},{"osd":4,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.92600000000000005,"15min":0.94799999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96199999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":2.2480000000000002}]},{"osd":5,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.099,"5min":1.125,"15min":1.125},"last":2.1259999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.97699999999999998,"15min":0.99399999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.3220000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.996,"15min":1.022},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.448},{"interface":"front","average":{"1min":0.79700000000000004,"5min":1,"15min":1.034},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1879999999999999,"5min":1.288,"15min":1.288},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.157,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":2.21},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0680000000000001,"15min":1.087},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.3979999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0740000000000001,"15min":1.101},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2529999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.0269999999999999,"15min":1.0549999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":2.169}]},{"osd":10,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.004,"15min":1.0129999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.296,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.95699999999999996,"15min":0.96499999999999997},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":2.2879999999999998}]}]},{"osd":8,"up_from":60,"seq":257698037800,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":459410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.745,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.434,"15min":0.434},"max":{"1min":6.3410000000000002,"5min":6.3410000000000002,"15min":6.3410000000000002},"last":0.56200000000000006}]},{"osd":1,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93600000000000005,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.53800000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.92000000000000004,"5min":1.216,"15min":1.216},"last":0.872}]},{"osd":2,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66000000000000003,"5min":1.081,"15min":1.081},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.433,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.747,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0720000000000001,"5min":1.175,"15min":1.175},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78200000000000003,"15min":0.79600000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0820000000000001,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":1.149,"15min":1.149},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.52700000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.996,"5min":1.218,"15min":1.218},"last":0.88800000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83099999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.629},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":1.21,"15min":1.21},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":0.85999999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1160000000000001,"5min":1.29,"15min":1.29},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0309999999999999,"5min":1.256,"15min":1.256},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Jul 14 16:13:46 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.92100000000000004,"15min":0.88700000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.3209999999999997,"5min":6.3209999999999997,"15min":6.3209999999999997},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.88300000000000001,"15min":0.89500000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.046,"5min":1.373,"15min":1.373},"last":0.95099999999999996}]},{"osd":9,"last update":"Fri Jul 14 16:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":0.90200000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.02,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0409999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.61099999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.873,"15min":0.85599999999999998},"min":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.89000000000000001,"15min":0.877},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1619999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.84199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]}]},{"osd":7,"up_from":52,"seq":223338299438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":1051271,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.181,"15min":1.181},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76400000000000001,"5min":1.046,"15min":1.046},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95899999999999996,"5min":1.008,"15min":1.008},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93799999999999994,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70599999999999996}]},{"osd":3,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66600000000000004,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.85399999999999998,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96299999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.46500000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90900000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.89100000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.58899999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0589999999999999,"5min":1.246,"15min":1.246},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.77700000000000002},"min":{"1min":0.624,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.94399999999999995}]},{"osd":6,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0629999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78900000000000003,"15min":0.78100000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.997,"5min":1.137,"15min":1.137},"last":0.84999999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.67400000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0369999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.874,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.69999999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.079,"5min":1.331,"15min":1.331},"last":0.876}]},{"osd":9,"last update":"Fri Jul 14 16:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89400000000000002,"15min":0.89600000000000002},"min":{"1min":0.69799999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1910000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.86699999999999999,"15min":0.86899999999999999},"min":{"1min":0.61899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0660000000000001,"5min":1.262,"15min":1.262},"last":0.93000000000000005}]},{"osd":10,"last update":"Fri Jul 14 16:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.879,"15min":0.88500000000000001},"min":{"1min":0.628,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.036,"5min":1.238,"15min":1.238},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.90100000000000002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.042,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.96699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":458907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.68999999999999995},"min":{"1min":0.5,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.127},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.67800000000000005},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98599999999999999,"5min":1.111,"15min":1.111},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0009999999999999,"5min":1.121,"15min":1.121},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94799999999999995,"5min":1.105,"15min":1.105},"last":0.94799999999999995}]},{"osd":3,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.202},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.64000000000000001}]},{"osd":4,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.79100000000000004,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.073}]},{"osd":5,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82099999999999995,"15min":0.81000000000000005},"min":{"1min":0.53800000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0720000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.60399999999999998}]},{"osd":7,"last update":"Fri Jul 14 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.78300000000000003},"min":{"1min":0.56999999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0469999999999999,"5min":1.28,"15min":1.28},"last":1.165},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83899999999999997,"15min":0.80200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.145,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.238}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93000000000000005,"15min":0.93700000000000006},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2929999999999999,"5min":1.353,"15min":1.353},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.873,"15min":0.86499999999999999},"min":{"1min":0.72299999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2470000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.58199999999999996}]},{"osd":9,"last update":"Fri Jul 14 16:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2170000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.97899999999999998,"15min":0.997},"min":{"1min":0.56499999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2330000000000001,"5min":1.331,"15min":1.331},"last":1.0349999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.88},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.97799999999999998,"15min":0.99099999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.179,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":516225,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61399999999999999,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60699999999999998,"15min":0.53700000000000003},"min":{"1min":0.42099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.46100000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65800000000000003,"15min":0.59899999999999998},"min":{"1min":0.54100000000000004,"5min":0.377,"15min":0.377},"max":{"1min":1.01,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.66500000000000004,"15min":0.621},"min":{"1min":0.58699999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Jul 14 16:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64800000000000002,"15min":0.60199999999999998},"min":{"1min":0.44900000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97899999999999998,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64100000000000001,"15min":0.59899999999999998},"min":{"1min":0.52300000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.53600000000000003,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95899999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97499999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.53500000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.72899999999999998,"15min":0.69299999999999995},"min":{"1min":0.63400000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.72099999999999997,"15min":0.67100000000000004},"min":{"1min":0.55900000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.113,"5min":1.244,"15min":1.244},"last":0.85599999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2050000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.81200000000000006,"15min":0.77200000000000002},"min":{"1min":0.75,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.82999999999999996}]},{"osd":9,"last update":"Fri Jul 14 16:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.872,"15min":0.875},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.69699999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.2030000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.90800000000000003}]},{"osd":10,"last update":"Fri Jul 14 16:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.91900000000000004,"15min":0.90900000000000003},"min":{"1min":0.72599999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2609999999999999,"5min":1.262,"15min":1.262},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93200000000000005,"15min":0.93000000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.84199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":460172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63800000000000001,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.137}]},{"osd":2,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69899999999999995,"15min":0.626},"min":{"1min":0.60099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.71199999999999997,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.03}]},{"osd":3,"last update":"Fri Jul 14 16:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74399999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.753},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.96399999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.89800000000000002}]},{"osd":5,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.45400000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.80700000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.5289999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.282}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.748,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83799999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.92100000000000004},"min":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.85899999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.1020000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.871,"15min":0.878},"min":{"1min":0.54300000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.649},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.93300000000000005,"15min":0.94999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.399,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.72499999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.92000000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":1.2370000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.95299999999999996,"15min":0.94599999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":3.4900000000000002,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.93799999999999994}]},{"osd":11,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.171},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.486}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":517527,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.54900000000000004},"min":{"1min":0.379,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.753},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.56299999999999994},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90500000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0549999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Jul 14 16:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.70199999999999996,"15min":0.65600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93000000000000005,"5min":0.999,"15min":0.999},"last":0.63}]},{"osd":4,"last update":"Fri Jul 14 16:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96399999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":1.022,"15min":1.022},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.76600000000000001,"15min":0.71999999999999997},"min":{"1min":0.57699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.7240000000000002,"5min":3.7240000000000002,"15min":3.7240000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.80600000000000005,"15min":0.75800000000000001},"min":{"1min":0.53200000000000003,"5min":0.439,"15min":0.439},"max":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.67000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.022,"5min":1.117,"15min":1.117},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99099999999999999,"5min":1.016,"15min":1.016},"last":0.56999999999999995}]},{"osd":9,"last update":"Fri Jul 14 16:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1890000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.72999999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.61799999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.048,"5min":1.246,"15min":1.246},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.89100000000000001,"15min":0.88600000000000001},"min":{"1min":0.69099999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.204,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":458943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.45400000000000001,"15min":0.40899999999999997},"min":{"1min":0.29399999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.375,"5min":2.375,"15min":2.375},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.436},"min":{"1min":0.316,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":0.96399999999999997}]},{"osd":1,"last update":"Fri Jul 14 16:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.67000000000000004,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.48799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Jul 14 16:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.67700000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90600000000000003,"5min":1.137,"15min":1.137},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.71699999999999997},"min":{"1min":0.52800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.3050000000000002,"5min":3.448,"15min":3.448},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.66000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":3.359,"15min":3.359},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71099999999999997,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96499999999999997,"5min":1.222,"15min":1.222},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.74199999999999999,"15min":0.71499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.628},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76500000000000001,"15min":0.73499999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Jul 14 16:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.042,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0329999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Jul 14 16:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.82499999999999996,"15min":0.79100000000000004},"min":{"1min":0.51500000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.1339999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.018},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.94799999999999995}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.876,"15min":0.88500000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1259999999999999,"5min":1.546,"15min":1.546},"last":0.62},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91800000000000004,"15min":0.92100000000000004},"min":{"1min":0.61899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.163,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1659999999999999,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.145,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.91100000000000003}]},{"osd":10,"last update":"Fri Jul 14 16:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.91800000000000004,"15min":0.91500000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.9860000000000002,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.016},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.0649999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":459680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.95599999999999996,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46800000000000003,"15min":0.432},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.501,"15min":0.45700000000000002},"min":{"1min":0.35499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68000000000000005,"15min":0.64200000000000002},"min":{"1min":0.505,"5min":0.439,"15min":0.439},"max":{"1min":1.048,"5min":3.698,"15min":3.698},"last":0.87},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.70199999999999996,"15min":0.67800000000000005},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1160000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.78400000000000003}]},{"osd":3,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.54900000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.995,"5min":1.03,"15min":1.03},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Jul 14 16:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.63700000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.006,"5min":1.506,"15min":1.506},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72599999999999998,"15min":0.71799999999999997},"min":{"1min":0.54300000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.079,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57099999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.147,"5min":1.321,"15min":1.321},"last":0.62},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.5}]},{"osd":7,"last update":"Fri Jul 14 16:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0189999999999999,"5min":1.038,"15min":1.038},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.751,"15min":0.751},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.042,"5min":1.129,"15min":1.129},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85999999999999999,"15min":0.84299999999999997},"min":{"1min":0.753,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79100000000000004,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.753}]},{"osd":9,"last update":"Fri Jul 14 16:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.91700000000000004,"15min":0.92300000000000004},"min":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.127,"5min":1.631,"15min":1.631},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86299999999999999,"15min":0.874},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0029999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.54800000000000004}]},{"osd":10,"last update":"Fri Jul 14 16:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":1050282,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92900000000000005,"5min":1.099,"15min":1.099},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65900000000000003,"15min":0.625},"min":{"1min":0.54600000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95299999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83499999999999996}]},{"osd":2,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.498,"15min":0.48499999999999999},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.75900000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.495,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.312,"15min":0.312},"max":{"1min":0.72899999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.41699999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66000000000000003,"15min":0.64200000000000002},"min":{"1min":0.505,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71699999999999997,"15min":0.66300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0289999999999999,"5min":7.056,"15min":7.056},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.73199999999999998,"15min":0.70399999999999996},"min":{"1min":0.59199999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.1100000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.84799999999999998}]},{"osd":6,"last update":"Fri Jul 14 16:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.2569999999999999,"5min":7.109,"15min":7.109},"last":0.45800000000000002}]},{"osd":7,"last update":"Fri Jul 14 16:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.751,"15min":0.745},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89800000000000002,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97499999999999998,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.67900000000000005}]},{"osd":8,"last update":"Fri Jul 14 16:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.1479999999999999,"5min":1.286,"15min":1.286},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97899999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.66300000000000003}]},{"osd":9,"last update":"Fri Jul 14 16:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.876,"15min":0.876},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86399999999999999,"15min":0.86499999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.165,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.75700000000000001}]},{"osd":10,"last update":"Fri Jul 14 16:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.198,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.879},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.91900000000000004,"15min":0.91600000000000004},"min":{"1min":0.73599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.167,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":57531,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58833,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":59451,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T16:14:53.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph pg dump --format=json 2023-07-14T16:14:54.047 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:54.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:54 smithi062 ceph-mon[95543]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:54.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:54 smithi062 ceph-mon[95543]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:54.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:54 smithi081 ceph-mon[127795]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:54.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:54 smithi081 ceph-mon[127795]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:55.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:54 smithi164 ceph-mon[97427]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:55.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:54 smithi164 ceph-mon[97427]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:14:55.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 -- 172.21.15.62:0/595603216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 msgr2=0x7f40b41001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:55.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 --2- 172.21.15.62:0/595603216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 0x7f40b41001d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40a80099c0 tx=0x7f40a8009cd0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 -- 172.21.15.62:0/595603216 shutdown_connections 2023-07-14T16:14:55.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 --2- 172.21.15.62:0/595603216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b4104ea0 0x7f40b410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 --2- 172.21.15.62:0/595603216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 0x7f40b41001d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 --2- 172.21.15.62:0/595603216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b41007a0 0x7f40b4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 -- 172.21.15.62:0/595603216 >> 172.21.15.62:0/595603216 conn(0x7f40b40fb660 msgr2=0x7f40b40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 -- 172.21.15.62:0/595603216 shutdown_connections 2023-07-14T16:14:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.594+0000 7f40bab4c700 1 -- 172.21.15.62:0/595603216 wait complete. 2023-07-14T16:14:55.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.595+0000 7f40bab4c700 1 Processor -- start 2023-07-14T16:14:55.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.595+0000 7f40bab4c700 1 -- start start 2023-07-14T16:14:55.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40bab4c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 0x7f40b419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40bab4c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b41007a0 0x7f40b419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40bab4c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b4104ea0 0x7f40b4195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40bab4c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4196090 con 0x7f40b40ffe00 2023-07-14T16:14:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40bab4c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4196200 con 0x7f40b4104ea0 2023-07-14T16:14:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40bab4c700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b41963c0 con 0x7f40b41007a0 2023-07-14T16:14:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b88e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 0x7f40b419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b88e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 0x7f40b419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59512/0 (socket says 172.21.15.62:59512) 2023-07-14T16:14:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b88e8700 1 -- 172.21.15.62:0/366087237 learned_addr learned my addr 172.21.15.62:0/366087237 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b3fff700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b41007a0 0x7f40b419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b88e8700 1 -- 172.21.15.62:0/366087237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b41007a0 msgr2=0x7f40b419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b90e9700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b4104ea0 0x7f40b4195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b88e8700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b41007a0 0x7f40b419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b88e8700 1 -- 172.21.15.62:0/366087237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b4104ea0 msgr2=0x7f40b4195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b88e8700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b4104ea0 0x7f40b4195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b88e8700 1 -- 172.21.15.62:0/366087237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a80095f0 con 0x7f40b40ffe00 2023-07-14T16:14:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.596+0000 7f40b90e9700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b4104ea0 0x7f40b4195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:14:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.597+0000 7f40b88e8700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 0x7f40b419b9a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005910 tx=0x7f40a800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.597+0000 7f40b1ffb700 1 -- 172.21.15.62:0/366087237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a801d070 con 0x7f40b40ffe00 2023-07-14T16:14:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.597+0000 7f40b1ffb700 1 -- 172.21.15.62:0/366087237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a80054e0 con 0x7f40b40ffe00 2023-07-14T16:14:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.597+0000 7f40b1ffb700 1 -- 172.21.15.62:0/366087237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8004b70 con 0x7f40b40ffe00 2023-07-14T16:14:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.597+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b4196640 con 0x7f40b40ffe00 2023-07-14T16:14:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.597+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b41a2990 con 0x7f40b40ffe00 2023-07-14T16:14:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.598+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b404e210 con 0x7f40b40ffe00 2023-07-14T16:14:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.600+0000 7f40b1ffb700 1 -- 172.21.15.62:0/366087237 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f40a8004cf0 con 0x7f40b40ffe00 2023-07-14T16:14:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.602+0000 7f40b1ffb700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a40a1260 0x7f40a40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.602+0000 7f40b1ffb700 1 -- 172.21.15.62:0/366087237 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40a80cdd60 con 0x7f40b40ffe00 2023-07-14T16:14:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.602+0000 7f40b3fff700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a40a1260 0x7f40a40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:55.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.603+0000 7f40b3fff700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a40a1260 0x7f40a40a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40b40fbc10 tx=0x7f40a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:55.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.604+0000 7f40b1ffb700 1 -- 172.21.15.62:0/366087237 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f40a8097390 con 0x7f40b40ffe00 2023-07-14T16:14:55.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.805+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40b40fc9a0 con 0x7f40a40a1260 2023-07-14T16:14:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.815+0000 7f40b1ffb700 1 -- 172.21.15.62:0/366087237 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339315 (secure 0 0 0) 0x7f40b40fc9a0 con 0x7f40a40a1260 2023-07-14T16:14:55.795 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:55.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a40a1260 msgr2=0x7f40a40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:55.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a40a1260 0x7f40a40a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f40b40fbc10 tx=0x7f40a0009260 comp rx=0 tx=0).stop 2023-07-14T16:14:55.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 msgr2=0x7f40b419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:55.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 0x7f40b419b9a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005910 tx=0x7f40a800f740 comp rx=0 tx=0).stop 2023-07-14T16:14:55.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 shutdown_connections 2023-07-14T16:14:55.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a40a1260 0x7f40a40a3710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b4104ea0 0x7f40b4195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b40ffe00 0x7f40b419b9a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 --2- 172.21.15.62:0/366087237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b41007a0 0x7f40b419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:55.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 >> 172.21.15.62:0/366087237 conn(0x7f40b40fb660 msgr2=0x7f40b40fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:55.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 shutdown_connections 2023-07-14T16:14:55.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:55.821+0000 7f40bab4c700 1 -- 172.21.15.62:0/366087237 wait complete. 2023-07-14T16:14:55.816 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-07-14T16:14:56.326 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2023-07-14T16:14:55.483163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652049,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":452,"num_read_kb":336,"num_write":331,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335052,"kb_used_data":11628,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606772,"statfs":{"total":1151940427776,"available":1151597334528,"internally_reserved":0,"allocated":11907072,"data_stored":7459582,"data_compressed":112137,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003085"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661040+0000","last_change":"2023-07-14T16:13:59.301892+0000","last_active":"2023-07-14T16:14:20.661040+0000","last_peered":"2023-07-14T16:14:20.661040+0000","last_clean":"2023-07-14T16:14:20.661040+0000","last_became_active":"2023-07-14T16:13:59.301379+0000","last_became_peered":"2023-07-14T16:13:59.301379+0000","last_unstale":"2023-07-14T16:14:20.661040+0000","last_undegraded":"2023-07-14T16:14:20.661040+0000","last_fullsized":"2023-07-14T16:14:20.661040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:49:12.243022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665853+0000","last_change":"2023-07-14T16:13:59.309322+0000","last_active":"2023-07-14T16:14:20.665853+0000","last_peered":"2023-07-14T16:14:20.665853+0000","last_clean":"2023-07-14T16:14:20.665853+0000","last_became_active":"2023-07-14T16:13:59.309035+0000","last_became_peered":"2023-07-14T16:13:59.309035+0000","last_unstale":"2023-07-14T16:14:20.665853+0000","last_undegraded":"2023-07-14T16:14:20.665853+0000","last_fullsized":"2023-07-14T16:14:20.665853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:46.013552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665545+0000","last_change":"2023-07-14T16:13:59.304371+0000","last_active":"2023-07-14T16:14:20.665545+0000","last_peered":"2023-07-14T16:14:20.665545+0000","last_clean":"2023-07-14T16:14:20.665545+0000","last_became_active":"2023-07-14T16:13:59.303965+0000","last_became_peered":"2023-07-14T16:13:59.303965+0000","last_unstale":"2023-07-14T16:14:20.665545+0000","last_undegraded":"2023-07-14T16:14:20.665545+0000","last_fullsized":"2023-07-14T16:14:20.665545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:19.938384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.674685+0000","last_change":"2023-07-14T16:14:20.670094+0000","last_active":"2023-07-14T16:14:20.674685+0000","last_peered":"2023-07-14T16:14:20.674685+0000","last_clean":"2023-07-14T16:14:20.674685+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:14:20.674685+0000","last_undegraded":"2023-07-14T16:14:20.674685+0000","last_fullsized":"2023-07-14T16:14:20.674685+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:32.039209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.662901+0000","last_change":"2023-07-14T16:13:59.298094+0000","last_active":"2023-07-14T16:14:20.662901+0000","last_peered":"2023-07-14T16:14:20.662901+0000","last_clean":"2023-07-14T16:14:20.662901+0000","last_became_active":"2023-07-14T16:13:59.297764+0000","last_became_peered":"2023-07-14T16:13:59.297764+0000","last_unstale":"2023-07-14T16:14:20.662901+0000","last_undegraded":"2023-07-14T16:14:20.662901+0000","last_fullsized":"2023-07-14T16:14:20.662901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:40:48.501912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666802+0000","last_change":"2023-07-14T16:13:59.299735+0000","last_active":"2023-07-14T16:14:20.666802+0000","last_peered":"2023-07-14T16:14:20.666802+0000","last_clean":"2023-07-14T16:14:20.666802+0000","last_became_active":"2023-07-14T16:13:59.299413+0000","last_became_peered":"2023-07-14T16:13:59.299413+0000","last_unstale":"2023-07-14T16:14:20.666802+0000","last_undegraded":"2023-07-14T16:14:20.666802+0000","last_fullsized":"2023-07-14T16:14:20.666802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:09:10.174873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660827+0000","last_change":"2023-07-14T16:13:59.299682+0000","last_active":"2023-07-14T16:14:20.660827+0000","last_peered":"2023-07-14T16:14:20.660827+0000","last_clean":"2023-07-14T16:14:20.660827+0000","last_became_active":"2023-07-14T16:13:59.299327+0000","last_became_peered":"2023-07-14T16:13:59.299327+0000","last_unstale":"2023-07-14T16:14:20.660827+0000","last_undegraded":"2023-07-14T16:14:20.660827+0000","last_fullsized":"2023-07-14T16:14:20.660827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:30.735405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663232+0000","last_change":"2023-07-14T16:13:59.311818+0000","last_active":"2023-07-14T16:14:20.663232+0000","last_peered":"2023-07-14T16:14:20.663232+0000","last_clean":"2023-07-14T16:14:20.663232+0000","last_became_active":"2023-07-14T16:13:59.311478+0000","last_became_peered":"2023-07-14T16:13:59.311478+0000","last_unstale":"2023-07-14T16:14:20.663232+0000","last_undegraded":"2023-07-14T16:14:20.663232+0000","last_fullsized":"2023-07-14T16:14:20.663232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:51.107741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763755+0000","last_change":"2023-07-14T16:13:59.309665+0000","last_active":"2023-07-14T16:14:20.763755+0000","last_peered":"2023-07-14T16:14:20.763755+0000","last_clean":"2023-07-14T16:14:20.763755+0000","last_became_active":"2023-07-14T16:13:59.309361+0000","last_became_peered":"2023-07-14T16:13:59.309361+0000","last_unstale":"2023-07-14T16:14:20.763755+0000","last_undegraded":"2023-07-14T16:14:20.763755+0000","last_fullsized":"2023-07-14T16:14:20.763755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:14.297747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666802+0000","last_change":"2023-07-14T16:13:59.296397+0000","last_active":"2023-07-14T16:14:20.666802+0000","last_peered":"2023-07-14T16:14:20.666802+0000","last_clean":"2023-07-14T16:14:20.666802+0000","last_became_active":"2023-07-14T16:13:59.295965+0000","last_became_peered":"2023-07-14T16:13:59.295965+0000","last_unstale":"2023-07-14T16:14:20.666802+0000","last_undegraded":"2023-07-14T16:14:20.666802+0000","last_fullsized":"2023-07-14T16:14:20.666802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:51.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661239+0000","last_change":"2023-07-14T16:13:59.302892+0000","last_active":"2023-07-14T16:14:20.661239+0000","last_peered":"2023-07-14T16:14:20.661239+0000","last_clean":"2023-07-14T16:14:20.661239+0000","last_became_active":"2023-07-14T16:13:59.302295+0000","last_became_peered":"2023-07-14T16:13:59.302295+0000","last_unstale":"2023-07-14T16:14:20.661239+0000","last_undegraded":"2023-07-14T16:14:20.661239+0000","last_fullsized":"2023-07-14T16:14:20.661239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:11.009590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664108+0000","last_change":"2023-07-14T16:13:59.301913+0000","last_active":"2023-07-14T16:14:20.664108+0000","last_peered":"2023-07-14T16:14:20.664108+0000","last_clean":"2023-07-14T16:14:20.664108+0000","last_became_active":"2023-07-14T16:13:59.301590+0000","last_became_peered":"2023-07-14T16:13:59.301590+0000","last_unstale":"2023-07-14T16:14:20.664108+0000","last_undegraded":"2023-07-14T16:14:20.664108+0000","last_fullsized":"2023-07-14T16:14:20.664108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:09:55.206983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763393+0000","last_change":"2023-07-14T16:13:59.302006+0000","last_active":"2023-07-14T16:14:20.763393+0000","last_peered":"2023-07-14T16:14:20.763393+0000","last_clean":"2023-07-14T16:14:20.763393+0000","last_became_active":"2023-07-14T16:13:59.301752+0000","last_became_peered":"2023-07-14T16:13:59.301752+0000","last_unstale":"2023-07-14T16:14:20.763393+0000","last_undegraded":"2023-07-14T16:14:20.763393+0000","last_fullsized":"2023-07-14T16:14:20.763393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:08.923165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665808+0000","last_change":"2023-07-14T16:13:59.309318+0000","last_active":"2023-07-14T16:14:20.665808+0000","last_peered":"2023-07-14T16:14:20.665808+0000","last_clean":"2023-07-14T16:14:20.665808+0000","last_became_active":"2023-07-14T16:13:59.308931+0000","last_became_peered":"2023-07-14T16:13:59.308931+0000","last_unstale":"2023-07-14T16:14:20.665808+0000","last_undegraded":"2023-07-14T16:14:20.665808+0000","last_fullsized":"2023-07-14T16:14:20.665808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:21:02.747919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.670208+0000","last_change":"2023-07-14T16:14:20.670208+0000","last_active":"2023-07-14T16:14:20.670208+0000","last_peered":"2023-07-14T16:14:20.670208+0000","last_clean":"2023-07-14T16:14:20.670208+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:14:20.670208+0000","last_undegraded":"2023-07-14T16:14:20.670208+0000","last_fullsized":"2023-07-14T16:14:20.670208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:53.265928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763182+0000","last_change":"2023-07-14T16:13:59.307986+0000","last_active":"2023-07-14T16:14:20.763182+0000","last_peered":"2023-07-14T16:14:20.763182+0000","last_clean":"2023-07-14T16:14:20.763182+0000","last_became_active":"2023-07-14T16:13:59.307701+0000","last_became_peered":"2023-07-14T16:13:59.307701+0000","last_unstale":"2023-07-14T16:14:20.763182+0000","last_undegraded":"2023-07-14T16:14:20.763182+0000","last_fullsized":"2023-07-14T16:14:20.763182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:56:03.104096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661208+0000","last_change":"2023-07-14T16:13:59.301685+0000","last_active":"2023-07-14T16:14:20.661208+0000","last_peered":"2023-07-14T16:14:20.661208+0000","last_clean":"2023-07-14T16:14:20.661208+0000","last_became_active":"2023-07-14T16:13:59.300905+0000","last_became_peered":"2023-07-14T16:13:59.300905+0000","last_unstale":"2023-07-14T16:14:20.661208+0000","last_undegraded":"2023-07-14T16:14:20.661208+0000","last_fullsized":"2023-07-14T16:14:20.661208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:48:24.721908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665233+0000","last_change":"2023-07-14T16:13:59.306027+0000","last_active":"2023-07-14T16:14:20.665233+0000","last_peered":"2023-07-14T16:14:20.665233+0000","last_clean":"2023-07-14T16:14:20.665233+0000","last_became_active":"2023-07-14T16:13:59.305718+0000","last_became_peered":"2023-07-14T16:13:59.305718+0000","last_unstale":"2023-07-14T16:14:20.665233+0000","last_undegraded":"2023-07-14T16:14:20.665233+0000","last_fullsized":"2023-07-14T16:14:20.665233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:20:47.951543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.668632+0000","last_change":"2023-07-14T16:13:59.294140+0000","last_active":"2023-07-14T16:14:20.668632+0000","last_peered":"2023-07-14T16:14:20.668632+0000","last_clean":"2023-07-14T16:14:20.668632+0000","last_became_active":"2023-07-14T16:13:59.293825+0000","last_became_peered":"2023-07-14T16:13:59.293825+0000","last_unstale":"2023-07-14T16:14:20.668632+0000","last_undegraded":"2023-07-14T16:14:20.668632+0000","last_fullsized":"2023-07-14T16:14:20.668632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:59.348563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665523+0000","last_change":"2023-07-14T16:13:58.277154+0000","last_active":"2023-07-14T16:14:20.665523+0000","last_peered":"2023-07-14T16:14:20.665523+0000","last_clean":"2023-07-14T16:14:20.665523+0000","last_became_active":"2023-07-14T16:13:58.276800+0000","last_became_peered":"2023-07-14T16:13:58.276800+0000","last_unstale":"2023-07-14T16:14:20.665523+0000","last_undegraded":"2023-07-14T16:14:20.665523+0000","last_fullsized":"2023-07-14T16:14:20.665523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:45.631137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663871+0000","last_change":"2023-07-14T16:13:59.299819+0000","last_active":"2023-07-14T16:14:20.663871+0000","last_peered":"2023-07-14T16:14:20.663871+0000","last_clean":"2023-07-14T16:14:20.663871+0000","last_became_active":"2023-07-14T16:13:59.299294+0000","last_became_peered":"2023-07-14T16:13:59.299294+0000","last_unstale":"2023-07-14T16:14:20.663871+0000","last_undegraded":"2023-07-14T16:14:20.663871+0000","last_fullsized":"2023-07-14T16:14:20.663871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:43.461827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664040+0000","last_change":"2023-07-14T16:13:58.278954+0000","last_active":"2023-07-14T16:14:20.664040+0000","last_peered":"2023-07-14T16:14:20.664040+0000","last_clean":"2023-07-14T16:14:20.664040+0000","last_became_active":"2023-07-14T16:13:58.278454+0000","last_became_peered":"2023-07-14T16:13:58.278454+0000","last_unstale":"2023-07-14T16:14:20.664040+0000","last_undegraded":"2023-07-14T16:14:20.664040+0000","last_fullsized":"2023-07-14T16:14:20.664040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:33:34.015342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665978+0000","last_change":"2023-07-14T16:13:59.299877+0000","last_active":"2023-07-14T16:14:20.665978+0000","last_peered":"2023-07-14T16:14:20.665978+0000","last_clean":"2023-07-14T16:14:20.665978+0000","last_became_active":"2023-07-14T16:13:59.299557+0000","last_became_peered":"2023-07-14T16:13:59.299557+0000","last_unstale":"2023-07-14T16:14:20.665978+0000","last_undegraded":"2023-07-14T16:14:20.665978+0000","last_fullsized":"2023-07-14T16:14:20.665978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:45.331826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.677912+0000","last_change":"2023-07-14T16:14:20.677912+0000","last_active":"2023-07-14T16:14:20.677912+0000","last_peered":"2023-07-14T16:14:20.677912+0000","last_clean":"2023-07-14T16:14:20.677912+0000","last_became_active":"2023-07-14T16:14:20.668156+0000","last_became_peered":"2023-07-14T16:14:20.668156+0000","last_unstale":"2023-07-14T16:14:20.677912+0000","last_undegraded":"2023-07-14T16:14:20.677912+0000","last_fullsized":"2023-07-14T16:14:20.677912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:51:50.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763275+0000","last_change":"2023-07-14T16:13:59.307987+0000","last_active":"2023-07-14T16:14:20.763275+0000","last_peered":"2023-07-14T16:14:20.763275+0000","last_clean":"2023-07-14T16:14:20.763275+0000","last_became_active":"2023-07-14T16:13:59.307457+0000","last_became_peered":"2023-07-14T16:13:59.307457+0000","last_unstale":"2023-07-14T16:14:20.763275+0000","last_undegraded":"2023-07-14T16:14:20.763275+0000","last_fullsized":"2023-07-14T16:14:20.763275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:32:28.569384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663585+0000","last_change":"2023-07-14T16:13:58.282332+0000","last_active":"2023-07-14T16:14:20.663585+0000","last_peered":"2023-07-14T16:14:20.663585+0000","last_clean":"2023-07-14T16:14:20.663585+0000","last_became_active":"2023-07-14T16:13:58.278103+0000","last_became_peered":"2023-07-14T16:13:58.278103+0000","last_unstale":"2023-07-14T16:14:20.663585+0000","last_undegraded":"2023-07-14T16:14:20.663585+0000","last_fullsized":"2023-07-14T16:14:20.663585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:00.081710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665002+0000","last_change":"2023-07-14T16:13:59.301036+0000","last_active":"2023-07-14T16:14:20.665002+0000","last_peered":"2023-07-14T16:14:20.665002+0000","last_clean":"2023-07-14T16:14:20.665002+0000","last_became_active":"2023-07-14T16:13:59.300787+0000","last_became_peered":"2023-07-14T16:13:59.300787+0000","last_unstale":"2023-07-14T16:14:20.665002+0000","last_undegraded":"2023-07-14T16:14:20.665002+0000","last_fullsized":"2023-07-14T16:14:20.665002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:40:56.654793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:51.643237+0000","last_change":"2023-07-14T16:13:58.282392+0000","last_active":"2023-07-14T16:14:51.643237+0000","last_peered":"2023-07-14T16:14:51.643237+0000","last_clean":"2023-07-14T16:14:51.643237+0000","last_became_active":"2023-07-14T16:13:58.282030+0000","last_became_peered":"2023-07-14T16:13:58.282030+0000","last_unstale":"2023-07-14T16:14:51.643237+0000","last_undegraded":"2023-07-14T16:14:51.643237+0000","last_fullsized":"2023-07-14T16:14:51.643237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:17.795386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663473+0000","last_change":"2023-07-14T16:13:59.292681+0000","last_active":"2023-07-14T16:14:20.663473+0000","last_peered":"2023-07-14T16:14:20.663473+0000","last_clean":"2023-07-14T16:14:20.663473+0000","last_became_active":"2023-07-14T16:13:59.292289+0000","last_became_peered":"2023-07-14T16:13:59.292289+0000","last_unstale":"2023-07-14T16:14:20.663473+0000","last_undegraded":"2023-07-14T16:14:20.663473+0000","last_fullsized":"2023-07-14T16:14:20.663473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:33.971255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666548+0000","last_change":"2023-07-14T16:13:58.282111+0000","last_active":"2023-07-14T16:14:20.666548+0000","last_peered":"2023-07-14T16:14:20.666548+0000","last_clean":"2023-07-14T16:14:20.666548+0000","last_became_active":"2023-07-14T16:13:58.281783+0000","last_became_peered":"2023-07-14T16:13:58.281783+0000","last_unstale":"2023-07-14T16:14:20.666548+0000","last_undegraded":"2023-07-14T16:14:20.666548+0000","last_fullsized":"2023-07-14T16:14:20.666548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:51:40.718143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664212+0000","last_change":"2023-07-14T16:13:59.297439+0000","last_active":"2023-07-14T16:14:20.664212+0000","last_peered":"2023-07-14T16:14:20.664212+0000","last_clean":"2023-07-14T16:14:20.664212+0000","last_became_active":"2023-07-14T16:13:59.297117+0000","last_became_peered":"2023-07-14T16:13:59.297117+0000","last_unstale":"2023-07-14T16:14:20.664212+0000","last_undegraded":"2023-07-14T16:14:20.664212+0000","last_fullsized":"2023-07-14T16:14:20.664212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:08.122607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763285+0000","last_change":"2023-07-14T16:13:58.274826+0000","last_active":"2023-07-14T16:14:20.763285+0000","last_peered":"2023-07-14T16:14:20.763285+0000","last_clean":"2023-07-14T16:14:20.763285+0000","last_became_active":"2023-07-14T16:13:58.274406+0000","last_became_peered":"2023-07-14T16:13:58.274406+0000","last_unstale":"2023-07-14T16:14:20.763285+0000","last_undegraded":"2023-07-14T16:14:20.763285+0000","last_fullsized":"2023-07-14T16:14:20.763285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:09.422218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663188+0000","last_change":"2023-07-14T16:13:59.296730+0000","last_active":"2023-07-14T16:14:20.663188+0000","last_peered":"2023-07-14T16:14:20.663188+0000","last_clean":"2023-07-14T16:14:20.663188+0000","last_became_active":"2023-07-14T16:13:59.296372+0000","last_became_peered":"2023-07-14T16:13:59.296372+0000","last_unstale":"2023-07-14T16:14:20.663188+0000","last_undegraded":"2023-07-14T16:14:20.663188+0000","last_fullsized":"2023-07-14T16:14:20.663188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:07.809329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664772+0000","last_change":"2023-07-14T16:13:58.269122+0000","last_active":"2023-07-14T16:14:20.664772+0000","last_peered":"2023-07-14T16:14:20.664772+0000","last_clean":"2023-07-14T16:14:20.664772+0000","last_became_active":"2023-07-14T16:13:58.268770+0000","last_became_peered":"2023-07-14T16:13:58.268770+0000","last_unstale":"2023-07-14T16:14:20.664772+0000","last_undegraded":"2023-07-14T16:14:20.664772+0000","last_fullsized":"2023-07-14T16:14:20.664772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:43.584997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666674+0000","last_change":"2023-07-14T16:13:59.306987+0000","last_active":"2023-07-14T16:14:20.666674+0000","last_peered":"2023-07-14T16:14:20.666674+0000","last_clean":"2023-07-14T16:14:20.666674+0000","last_became_active":"2023-07-14T16:13:59.306649+0000","last_became_peered":"2023-07-14T16:13:59.306649+0000","last_unstale":"2023-07-14T16:14:20.666674+0000","last_undegraded":"2023-07-14T16:14:20.666674+0000","last_fullsized":"2023-07-14T16:14:20.666674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:35:23.579814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:51.642599+0000","last_change":"2023-07-14T16:13:58.281319+0000","last_active":"2023-07-14T16:14:51.642599+0000","last_peered":"2023-07-14T16:14:51.642599+0000","last_clean":"2023-07-14T16:14:51.642599+0000","last_became_active":"2023-07-14T16:13:58.280928+0000","last_became_peered":"2023-07-14T16:13:58.280928+0000","last_unstale":"2023-07-14T16:14:51.642599+0000","last_undegraded":"2023-07-14T16:14:51.642599+0000","last_fullsized":"2023-07-14T16:14:51.642599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:41.337954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763717+0000","last_change":"2023-07-14T16:13:59.309586+0000","last_active":"2023-07-14T16:14:20.763717+0000","last_peered":"2023-07-14T16:14:20.763717+0000","last_clean":"2023-07-14T16:14:20.763717+0000","last_became_active":"2023-07-14T16:13:59.309243+0000","last_became_peered":"2023-07-14T16:13:59.309243+0000","last_unstale":"2023-07-14T16:14:20.763717+0000","last_undegraded":"2023-07-14T16:14:20.763717+0000","last_fullsized":"2023-07-14T16:14:20.763717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:30.700680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661328+0000","last_change":"2023-07-14T16:13:58.279769+0000","last_active":"2023-07-14T16:14:20.661328+0000","last_peered":"2023-07-14T16:14:20.661328+0000","last_clean":"2023-07-14T16:14:20.661328+0000","last_became_active":"2023-07-14T16:13:58.279409+0000","last_became_peered":"2023-07-14T16:13:58.279409+0000","last_unstale":"2023-07-14T16:14:20.661328+0000","last_undegraded":"2023-07-14T16:14:20.661328+0000","last_fullsized":"2023-07-14T16:14:20.661328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:32.751188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666121+0000","last_change":"2023-07-14T16:13:59.300382+0000","last_active":"2023-07-14T16:14:20.666121+0000","last_peered":"2023-07-14T16:14:20.666121+0000","last_clean":"2023-07-14T16:14:20.666121+0000","last_became_active":"2023-07-14T16:13:59.299981+0000","last_became_peered":"2023-07-14T16:13:59.299981+0000","last_unstale":"2023-07-14T16:14:20.666121+0000","last_undegraded":"2023-07-14T16:14:20.666121+0000","last_fullsized":"2023-07-14T16:14:20.666121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:46:04.769609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665031+0000","last_change":"2023-07-14T16:13:58.274128+0000","last_active":"2023-07-14T16:14:20.665031+0000","last_peered":"2023-07-14T16:14:20.665031+0000","last_clean":"2023-07-14T16:14:20.665031+0000","last_became_active":"2023-07-14T16:13:58.273446+0000","last_became_peered":"2023-07-14T16:13:58.273446+0000","last_unstale":"2023-07-14T16:14:20.665031+0000","last_undegraded":"2023-07-14T16:14:20.665031+0000","last_fullsized":"2023-07-14T16:14:20.665031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:12.510793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763503+0000","last_change":"2023-07-14T16:13:59.301551+0000","last_active":"2023-07-14T16:14:20.763503+0000","last_peered":"2023-07-14T16:14:20.763503+0000","last_clean":"2023-07-14T16:14:20.763503+0000","last_became_active":"2023-07-14T16:13:59.301206+0000","last_became_peered":"2023-07-14T16:13:59.301206+0000","last_unstale":"2023-07-14T16:14:20.763503+0000","last_undegraded":"2023-07-14T16:14:20.763503+0000","last_fullsized":"2023-07-14T16:14:20.763503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:16.227849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:52.644130+0000","last_change":"2023-07-14T16:13:58.283304+0000","last_active":"2023-07-14T16:14:52.644130+0000","last_peered":"2023-07-14T16:14:52.644130+0000","last_clean":"2023-07-14T16:14:52.644130+0000","last_became_active":"2023-07-14T16:13:58.282930+0000","last_became_peered":"2023-07-14T16:13:58.282930+0000","last_unstale":"2023-07-14T16:14:52.644130+0000","last_undegraded":"2023-07-14T16:14:52.644130+0000","last_fullsized":"2023-07-14T16:14:52.644130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:02.538305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57531,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":107,"num_read_kb":109,"num_write":27,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664117+0000","last_change":"2023-07-14T16:13:59.304144+0000","last_active":"2023-07-14T16:14:20.664117+0000","last_peered":"2023-07-14T16:14:20.664117+0000","last_clean":"2023-07-14T16:14:20.664117+0000","last_became_active":"2023-07-14T16:13:59.303745+0000","last_became_peered":"2023-07-14T16:13:59.303745+0000","last_unstale":"2023-07-14T16:14:20.664117+0000","last_undegraded":"2023-07-14T16:14:20.664117+0000","last_fullsized":"2023-07-14T16:14:20.664117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:20.828221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667830+0000","last_change":"2023-07-14T16:13:58.283734+0000","last_active":"2023-07-14T16:14:20.667830+0000","last_peered":"2023-07-14T16:14:20.667830+0000","last_clean":"2023-07-14T16:14:20.667830+0000","last_became_active":"2023-07-14T16:13:58.282765+0000","last_became_peered":"2023-07-14T16:13:58.282765+0000","last_unstale":"2023-07-14T16:14:20.667830+0000","last_undegraded":"2023-07-14T16:14:20.667830+0000","last_fullsized":"2023-07-14T16:14:20.667830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:13.217664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763262+0000","last_change":"2023-07-14T16:13:59.308445+0000","last_active":"2023-07-14T16:14:20.763262+0000","last_peered":"2023-07-14T16:14:20.763262+0000","last_clean":"2023-07-14T16:14:20.763262+0000","last_became_active":"2023-07-14T16:13:59.308130+0000","last_became_peered":"2023-07-14T16:13:59.308130+0000","last_unstale":"2023-07-14T16:14:20.763262+0000","last_undegraded":"2023-07-14T16:14:20.763262+0000","last_fullsized":"2023-07-14T16:14:20.763262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:45.004512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664202+0000","last_change":"2023-07-14T16:13:58.273973+0000","last_active":"2023-07-14T16:14:20.664202+0000","last_peered":"2023-07-14T16:14:20.664202+0000","last_clean":"2023-07-14T16:14:20.664202+0000","last_became_active":"2023-07-14T16:13:58.273606+0000","last_became_peered":"2023-07-14T16:13:58.273606+0000","last_unstale":"2023-07-14T16:14:20.664202+0000","last_undegraded":"2023-07-14T16:14:20.664202+0000","last_fullsized":"2023-07-14T16:14:20.664202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:12.304491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661141+0000","last_change":"2023-07-14T16:13:59.306428+0000","last_active":"2023-07-14T16:14:20.661141+0000","last_peered":"2023-07-14T16:14:20.661141+0000","last_clean":"2023-07-14T16:14:20.661141+0000","last_became_active":"2023-07-14T16:13:59.306090+0000","last_became_peered":"2023-07-14T16:13:59.306090+0000","last_unstale":"2023-07-14T16:14:20.661141+0000","last_undegraded":"2023-07-14T16:14:20.661141+0000","last_fullsized":"2023-07-14T16:14:20.661141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:16.033202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:53.703041+0000","last_change":"2023-07-14T16:13:58.277123+0000","last_active":"2023-07-14T16:14:53.703041+0000","last_peered":"2023-07-14T16:14:53.703041+0000","last_clean":"2023-07-14T16:14:53.703041+0000","last_became_active":"2023-07-14T16:13:58.276815+0000","last_became_peered":"2023-07-14T16:13:58.276815+0000","last_unstale":"2023-07-14T16:14:53.703041+0000","last_undegraded":"2023-07-14T16:14:53.703041+0000","last_fullsized":"2023-07-14T16:14:53.703041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:41.019233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660748+0000","last_change":"2023-07-14T16:13:59.310238+0000","last_active":"2023-07-14T16:14:20.660748+0000","last_peered":"2023-07-14T16:14:20.660748+0000","last_clean":"2023-07-14T16:14:20.660748+0000","last_became_active":"2023-07-14T16:13:59.309918+0000","last_became_peered":"2023-07-14T16:13:59.309918+0000","last_unstale":"2023-07-14T16:14:20.660748+0000","last_undegraded":"2023-07-14T16:14:20.660748+0000","last_fullsized":"2023-07-14T16:14:20.660748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:25.020575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663660+0000","last_change":"2023-07-14T16:13:58.280834+0000","last_active":"2023-07-14T16:14:20.663660+0000","last_peered":"2023-07-14T16:14:20.663660+0000","last_clean":"2023-07-14T16:14:20.663660+0000","last_became_active":"2023-07-14T16:13:58.280497+0000","last_became_peered":"2023-07-14T16:13:58.280497+0000","last_unstale":"2023-07-14T16:14:20.663660+0000","last_undegraded":"2023-07-14T16:14:20.663660+0000","last_fullsized":"2023-07-14T16:14:20.663660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:07:02.308388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.765253+0000","last_change":"2023-07-14T16:14:20.764846+0000","last_active":"2023-07-14T16:14:20.765253+0000","last_peered":"2023-07-14T16:14:20.765253+0000","last_clean":"2023-07-14T16:14:20.765253+0000","last_became_active":"2023-07-14T16:14:20.764485+0000","last_became_peered":"2023-07-14T16:14:20.764485+0000","last_unstale":"2023-07-14T16:14:20.765253+0000","last_undegraded":"2023-07-14T16:14:20.765253+0000","last_fullsized":"2023-07-14T16:14:20.765253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:29.245874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.682843+0000","last_change":"2023-07-14T16:14:20.682331+0000","last_active":"2023-07-14T16:14:20.682843+0000","last_peered":"2023-07-14T16:14:20.682843+0000","last_clean":"2023-07-14T16:14:20.682843+0000","last_became_active":"2023-07-14T16:14:20.669179+0000","last_became_peered":"2023-07-14T16:14:20.669179+0000","last_unstale":"2023-07-14T16:14:20.682843+0000","last_undegraded":"2023-07-14T16:14:20.682843+0000","last_fullsized":"2023-07-14T16:14:20.682843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:55:10.629011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660894+0000","last_change":"2023-07-14T16:13:59.297644+0000","last_active":"2023-07-14T16:14:20.660894+0000","last_peered":"2023-07-14T16:14:20.660894+0000","last_clean":"2023-07-14T16:14:20.660894+0000","last_became_active":"2023-07-14T16:13:59.297332+0000","last_became_peered":"2023-07-14T16:13:59.297332+0000","last_unstale":"2023-07-14T16:14:20.660894+0000","last_undegraded":"2023-07-14T16:14:20.660894+0000","last_fullsized":"2023-07-14T16:14:20.660894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:36.421075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665695+0000","last_change":"2023-07-14T16:13:58.280632+0000","last_active":"2023-07-14T16:14:20.665695+0000","last_peered":"2023-07-14T16:14:20.665695+0000","last_clean":"2023-07-14T16:14:20.665695+0000","last_became_active":"2023-07-14T16:13:58.280296+0000","last_became_peered":"2023-07-14T16:13:58.280296+0000","last_unstale":"2023-07-14T16:14:20.665695+0000","last_undegraded":"2023-07-14T16:14:20.665695+0000","last_fullsized":"2023-07-14T16:14:20.665695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:26.897678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666919+0000","last_change":"2023-07-14T16:13:59.297092+0000","last_active":"2023-07-14T16:14:20.666919+0000","last_peered":"2023-07-14T16:14:20.666919+0000","last_clean":"2023-07-14T16:14:20.666919+0000","last_became_active":"2023-07-14T16:13:59.296750+0000","last_became_peered":"2023-07-14T16:13:59.296750+0000","last_unstale":"2023-07-14T16:14:20.666919+0000","last_undegraded":"2023-07-14T16:14:20.666919+0000","last_fullsized":"2023-07-14T16:14:20.666919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:32.797912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660888+0000","last_change":"2023-07-14T16:13:58.281786+0000","last_active":"2023-07-14T16:14:20.660888+0000","last_peered":"2023-07-14T16:14:20.660888+0000","last_clean":"2023-07-14T16:14:20.660888+0000","last_became_active":"2023-07-14T16:13:58.281471+0000","last_became_peered":"2023-07-14T16:13:58.281471+0000","last_unstale":"2023-07-14T16:14:20.660888+0000","last_undegraded":"2023-07-14T16:14:20.660888+0000","last_fullsized":"2023-07-14T16:14:20.660888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:09.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.672813+0000","last_change":"2023-07-14T16:14:20.672813+0000","last_active":"2023-07-14T16:14:20.672813+0000","last_peered":"2023-07-14T16:14:20.672813+0000","last_clean":"2023-07-14T16:14:20.672813+0000","last_became_active":"2023-07-14T16:14:20.672447+0000","last_became_peered":"2023-07-14T16:14:20.672447+0000","last_unstale":"2023-07-14T16:14:20.672813+0000","last_undegraded":"2023-07-14T16:14:20.672813+0000","last_fullsized":"2023-07-14T16:14:20.672813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:33:07.582055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665347+0000","last_change":"2023-07-14T16:13:59.288836+0000","last_active":"2023-07-14T16:14:20.665347+0000","last_peered":"2023-07-14T16:14:20.665347+0000","last_clean":"2023-07-14T16:14:20.665347+0000","last_became_active":"2023-07-14T16:13:59.288517+0000","last_became_peered":"2023-07-14T16:13:59.288517+0000","last_unstale":"2023-07-14T16:14:20.665347+0000","last_undegraded":"2023-07-14T16:14:20.665347+0000","last_fullsized":"2023-07-14T16:14:20.665347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:48.293313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:52.700165+0000","last_change":"2023-07-14T16:13:58.282415+0000","last_active":"2023-07-14T16:14:52.700165+0000","last_peered":"2023-07-14T16:14:52.700165+0000","last_clean":"2023-07-14T16:14:52.700165+0000","last_became_active":"2023-07-14T16:13:58.282118+0000","last_became_peered":"2023-07-14T16:13:58.282118+0000","last_unstale":"2023-07-14T16:14:52.700165+0000","last_undegraded":"2023-07-14T16:14:52.700165+0000","last_fullsized":"2023-07-14T16:14:52.700165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:41.532341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663573+0000","last_change":"2023-07-14T16:13:59.302860+0000","last_active":"2023-07-14T16:14:20.663573+0000","last_peered":"2023-07-14T16:14:20.663573+0000","last_clean":"2023-07-14T16:14:20.663573+0000","last_became_active":"2023-07-14T16:13:59.302454+0000","last_became_peered":"2023-07-14T16:13:59.302454+0000","last_unstale":"2023-07-14T16:14:20.663573+0000","last_undegraded":"2023-07-14T16:14:20.663573+0000","last_fullsized":"2023-07-14T16:14:20.663573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:34.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664397+0000","last_change":"2023-07-14T16:13:58.273908+0000","last_active":"2023-07-14T16:14:20.664397+0000","last_peered":"2023-07-14T16:14:20.664397+0000","last_clean":"2023-07-14T16:14:20.664397+0000","last_became_active":"2023-07-14T16:13:58.273497+0000","last_became_peered":"2023-07-14T16:13:58.273497+0000","last_unstale":"2023-07-14T16:14:20.664397+0000","last_undegraded":"2023-07-14T16:14:20.664397+0000","last_fullsized":"2023-07-14T16:14:20.664397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:55:44.978027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660941+0000","last_change":"2023-07-14T16:12:36.546475+0000","last_active":"2023-07-14T16:14:20.660941+0000","last_peered":"2023-07-14T16:14:20.660941+0000","last_clean":"2023-07-14T16:14:20.660941+0000","last_became_active":"2023-07-14T16:12:36.545943+0000","last_became_peered":"2023-07-14T16:12:36.545943+0000","last_unstale":"2023-07-14T16:14:20.660941+0000","last_undegraded":"2023-07-14T16:14:20.660941+0000","last_fullsized":"2023-07-14T16:14:20.660941+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_clean_scrub_stamp":"2023-07-14T16:08:58.024408+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:27.166990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763112+0000","last_change":"2023-07-14T16:13:59.297669+0000","last_active":"2023-07-14T16:14:20.763112+0000","last_peered":"2023-07-14T16:14:20.763112+0000","last_clean":"2023-07-14T16:14:20.763112+0000","last_became_active":"2023-07-14T16:13:59.297364+0000","last_became_peered":"2023-07-14T16:13:59.297364+0000","last_unstale":"2023-07-14T16:14:20.763112+0000","last_undegraded":"2023-07-14T16:14:20.763112+0000","last_fullsized":"2023-07-14T16:14:20.763112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:50:00.972727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.673166+0000","last_change":"2023-07-14T16:14:20.673166+0000","last_active":"2023-07-14T16:14:20.673166+0000","last_peered":"2023-07-14T16:14:20.673166+0000","last_clean":"2023-07-14T16:14:20.673166+0000","last_became_active":"2023-07-14T16:14:20.667664+0000","last_became_peered":"2023-07-14T16:14:20.667664+0000","last_unstale":"2023-07-14T16:14:20.673166+0000","last_undegraded":"2023-07-14T16:14:20.673166+0000","last_fullsized":"2023-07-14T16:14:20.673166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:24.328768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664285+0000","last_change":"2023-07-14T16:13:58.276257+0000","last_active":"2023-07-14T16:14:20.664285+0000","last_peered":"2023-07-14T16:14:20.664285+0000","last_clean":"2023-07-14T16:14:20.664285+0000","last_became_active":"2023-07-14T16:13:58.275910+0000","last_became_peered":"2023-07-14T16:13:58.275910+0000","last_unstale":"2023-07-14T16:14:20.664285+0000","last_undegraded":"2023-07-14T16:14:20.664285+0000","last_fullsized":"2023-07-14T16:14:20.664285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:47.700225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763397+0000","last_change":"2023-07-14T16:13:59.299135+0000","last_active":"2023-07-14T16:14:20.763397+0000","last_peered":"2023-07-14T16:14:20.763397+0000","last_clean":"2023-07-14T16:14:20.763397+0000","last_became_active":"2023-07-14T16:13:59.298031+0000","last_became_peered":"2023-07-14T16:13:59.298031+0000","last_unstale":"2023-07-14T16:14:20.763397+0000","last_undegraded":"2023-07-14T16:14:20.763397+0000","last_fullsized":"2023-07-14T16:14:20.763397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:21.882960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665374+0000","last_change":"2023-07-14T16:13:58.279664+0000","last_active":"2023-07-14T16:14:20.665374+0000","last_peered":"2023-07-14T16:14:20.665374+0000","last_clean":"2023-07-14T16:14:20.665374+0000","last_became_active":"2023-07-14T16:13:58.279137+0000","last_became_peered":"2023-07-14T16:13:58.279137+0000","last_unstale":"2023-07-14T16:14:20.665374+0000","last_undegraded":"2023-07-14T16:14:20.665374+0000","last_fullsized":"2023-07-14T16:14:20.665374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:07.657781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663754+0000","last_change":"2023-07-14T16:13:59.305536+0000","last_active":"2023-07-14T16:14:20.663754+0000","last_peered":"2023-07-14T16:14:20.663754+0000","last_clean":"2023-07-14T16:14:20.663754+0000","last_became_active":"2023-07-14T16:13:59.305176+0000","last_became_peered":"2023-07-14T16:13:59.305176+0000","last_unstale":"2023-07-14T16:14:20.663754+0000","last_undegraded":"2023-07-14T16:14:20.663754+0000","last_fullsized":"2023-07-14T16:14:20.663754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:18:18.141957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664829+0000","last_change":"2023-07-14T16:13:58.269652+0000","last_active":"2023-07-14T16:14:20.664829+0000","last_peered":"2023-07-14T16:14:20.664829+0000","last_clean":"2023-07-14T16:14:20.664829+0000","last_became_active":"2023-07-14T16:13:58.269338+0000","last_became_peered":"2023-07-14T16:13:58.269338+0000","last_unstale":"2023-07-14T16:14:20.664829+0000","last_undegraded":"2023-07-14T16:14:20.664829+0000","last_fullsized":"2023-07-14T16:14:20.664829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:55:21.910971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665994+0000","last_change":"2023-07-14T16:13:59.297574+0000","last_active":"2023-07-14T16:14:20.665994+0000","last_peered":"2023-07-14T16:14:20.665994+0000","last_clean":"2023-07-14T16:14:20.665994+0000","last_became_active":"2023-07-14T16:13:59.297124+0000","last_became_peered":"2023-07-14T16:13:59.297124+0000","last_unstale":"2023-07-14T16:14:20.665994+0000","last_undegraded":"2023-07-14T16:14:20.665994+0000","last_fullsized":"2023-07-14T16:14:20.665994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:20.051395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763343+0000","last_change":"2023-07-14T16:13:58.278011+0000","last_active":"2023-07-14T16:14:20.763343+0000","last_peered":"2023-07-14T16:14:20.763343+0000","last_clean":"2023-07-14T16:14:20.763343+0000","last_became_active":"2023-07-14T16:13:58.277563+0000","last_became_peered":"2023-07-14T16:13:58.277563+0000","last_unstale":"2023-07-14T16:14:20.763343+0000","last_undegraded":"2023-07-14T16:14:20.763343+0000","last_fullsized":"2023-07-14T16:14:20.763343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:06.354928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.662905+0000","last_change":"2023-07-14T16:13:59.308836+0000","last_active":"2023-07-14T16:14:20.662905+0000","last_peered":"2023-07-14T16:14:20.662905+0000","last_clean":"2023-07-14T16:14:20.662905+0000","last_became_active":"2023-07-14T16:13:59.308510+0000","last_became_peered":"2023-07-14T16:13:59.308510+0000","last_unstale":"2023-07-14T16:14:20.662905+0000","last_undegraded":"2023-07-14T16:14:20.662905+0000","last_fullsized":"2023-07-14T16:14:20.662905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:48:07.973849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663352+0000","last_change":"2023-07-14T16:13:58.277438+0000","last_active":"2023-07-14T16:14:20.663352+0000","last_peered":"2023-07-14T16:14:20.663352+0000","last_clean":"2023-07-14T16:14:20.663352+0000","last_became_active":"2023-07-14T16:13:58.277114+0000","last_became_peered":"2023-07-14T16:13:58.277114+0000","last_unstale":"2023-07-14T16:14:20.663352+0000","last_undegraded":"2023-07-14T16:14:20.663352+0000","last_fullsized":"2023-07-14T16:14:20.663352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:40.636429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660728+0000","last_change":"2023-07-14T16:13:59.304491+0000","last_active":"2023-07-14T16:14:20.660728+0000","last_peered":"2023-07-14T16:14:20.660728+0000","last_clean":"2023-07-14T16:14:20.660728+0000","last_became_active":"2023-07-14T16:13:59.303925+0000","last_became_peered":"2023-07-14T16:13:59.303925+0000","last_unstale":"2023-07-14T16:14:20.660728+0000","last_undegraded":"2023-07-14T16:14:20.660728+0000","last_fullsized":"2023-07-14T16:14:20.660728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.410847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.685314+0000","last_change":"2023-07-14T16:14:20.684842+0000","last_active":"2023-07-14T16:14:20.685314+0000","last_peered":"2023-07-14T16:14:20.685314+0000","last_clean":"2023-07-14T16:14:20.685314+0000","last_became_active":"2023-07-14T16:14:20.670098+0000","last_became_peered":"2023-07-14T16:14:20.670098+0000","last_unstale":"2023-07-14T16:14:20.685314+0000","last_undegraded":"2023-07-14T16:14:20.685314+0000","last_fullsized":"2023-07-14T16:14:20.685314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:54:58.021152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663708+0000","last_change":"2023-07-14T16:13:59.310983+0000","last_active":"2023-07-14T16:14:20.663708+0000","last_peered":"2023-07-14T16:14:20.663708+0000","last_clean":"2023-07-14T16:14:20.663708+0000","last_became_active":"2023-07-14T16:13:59.310636+0000","last_became_peered":"2023-07-14T16:13:59.310636+0000","last_unstale":"2023-07-14T16:14:20.663708+0000","last_undegraded":"2023-07-14T16:14:20.663708+0000","last_fullsized":"2023-07-14T16:14:20.663708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:31.003517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663068+0000","last_change":"2023-07-14T16:13:58.280153+0000","last_active":"2023-07-14T16:14:20.663068+0000","last_peered":"2023-07-14T16:14:20.663068+0000","last_clean":"2023-07-14T16:14:20.663068+0000","last_became_active":"2023-07-14T16:13:58.279771+0000","last_became_peered":"2023-07-14T16:13:58.279771+0000","last_unstale":"2023-07-14T16:14:20.663068+0000","last_undegraded":"2023-07-14T16:14:20.663068+0000","last_fullsized":"2023-07-14T16:14:20.663068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:46.290429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660909+0000","last_change":"2023-07-14T16:13:59.304854+0000","last_active":"2023-07-14T16:14:20.660909+0000","last_peered":"2023-07-14T16:14:20.660909+0000","last_clean":"2023-07-14T16:14:20.660909+0000","last_became_active":"2023-07-14T16:13:59.304374+0000","last_became_peered":"2023-07-14T16:13:59.304374+0000","last_unstale":"2023-07-14T16:14:20.660909+0000","last_undegraded":"2023-07-14T16:14:20.660909+0000","last_fullsized":"2023-07-14T16:14:20.660909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:52.213737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667834+0000","last_change":"2023-07-14T16:13:58.281885+0000","last_active":"2023-07-14T16:14:20.667834+0000","last_peered":"2023-07-14T16:14:20.667834+0000","last_clean":"2023-07-14T16:14:20.667834+0000","last_became_active":"2023-07-14T16:13:58.278759+0000","last_became_peered":"2023-07-14T16:13:58.278759+0000","last_unstale":"2023-07-14T16:14:20.667834+0000","last_undegraded":"2023-07-14T16:14:20.667834+0000","last_fullsized":"2023-07-14T16:14:20.667834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:11.816765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661411+0000","last_change":"2023-07-14T16:13:59.302680+0000","last_active":"2023-07-14T16:14:20.661411+0000","last_peered":"2023-07-14T16:14:20.661411+0000","last_clean":"2023-07-14T16:14:20.661411+0000","last_became_active":"2023-07-14T16:13:59.301854+0000","last_became_peered":"2023-07-14T16:13:59.301854+0000","last_unstale":"2023-07-14T16:14:20.661411+0000","last_undegraded":"2023-07-14T16:14:20.661411+0000","last_fullsized":"2023-07-14T16:14:20.661411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:29:42.861394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664860+0000","last_change":"2023-07-14T16:13:58.274254+0000","last_active":"2023-07-14T16:14:20.664860+0000","last_peered":"2023-07-14T16:14:20.664860+0000","last_clean":"2023-07-14T16:14:20.664860+0000","last_became_active":"2023-07-14T16:13:58.273933+0000","last_became_peered":"2023-07-14T16:13:58.273933+0000","last_unstale":"2023-07-14T16:14:20.664860+0000","last_undegraded":"2023-07-14T16:14:20.664860+0000","last_fullsized":"2023-07-14T16:14:20.664860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:53.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661253+0000","last_change":"2023-07-14T16:13:59.305615+0000","last_active":"2023-07-14T16:14:20.661253+0000","last_peered":"2023-07-14T16:14:20.661253+0000","last_clean":"2023-07-14T16:14:20.661253+0000","last_became_active":"2023-07-14T16:13:59.305206+0000","last_became_peered":"2023-07-14T16:13:59.305206+0000","last_unstale":"2023-07-14T16:14:20.661253+0000","last_undegraded":"2023-07-14T16:14:20.661253+0000","last_fullsized":"2023-07-14T16:14:20.661253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:41.771061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763463+0000","last_change":"2023-07-14T16:13:58.277976+0000","last_active":"2023-07-14T16:14:20.763463+0000","last_peered":"2023-07-14T16:14:20.763463+0000","last_clean":"2023-07-14T16:14:20.763463+0000","last_became_active":"2023-07-14T16:13:58.277618+0000","last_became_peered":"2023-07-14T16:13:58.277618+0000","last_unstale":"2023-07-14T16:14:20.763463+0000","last_undegraded":"2023-07-14T16:14:20.763463+0000","last_fullsized":"2023-07-14T16:14:20.763463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:25.780591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661191+0000","last_change":"2023-07-14T16:13:59.302041+0000","last_active":"2023-07-14T16:14:20.661191+0000","last_peered":"2023-07-14T16:14:20.661191+0000","last_clean":"2023-07-14T16:14:20.661191+0000","last_became_active":"2023-07-14T16:13:59.301620+0000","last_became_peered":"2023-07-14T16:13:59.301620+0000","last_unstale":"2023-07-14T16:14:20.661191+0000","last_undegraded":"2023-07-14T16:14:20.661191+0000","last_fullsized":"2023-07-14T16:14:20.661191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:18.969064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660855+0000","last_change":"2023-07-14T16:13:59.304222+0000","last_active":"2023-07-14T16:14:20.660855+0000","last_peered":"2023-07-14T16:14:20.660855+0000","last_clean":"2023-07-14T16:14:20.660855+0000","last_became_active":"2023-07-14T16:13:59.303404+0000","last_became_peered":"2023-07-14T16:13:59.303404+0000","last_unstale":"2023-07-14T16:14:20.660855+0000","last_undegraded":"2023-07-14T16:14:20.660855+0000","last_fullsized":"2023-07-14T16:14:20.660855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:26.218664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666713+0000","last_change":"2023-07-14T16:13:59.296433+0000","last_active":"2023-07-14T16:14:20.666713+0000","last_peered":"2023-07-14T16:14:20.666713+0000","last_clean":"2023-07-14T16:14:20.666713+0000","last_became_active":"2023-07-14T16:13:59.295604+0000","last_became_peered":"2023-07-14T16:13:59.295604+0000","last_unstale":"2023-07-14T16:14:20.666713+0000","last_undegraded":"2023-07-14T16:14:20.666713+0000","last_fullsized":"2023-07-14T16:14:20.666713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:17:57.269484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667209+0000","last_change":"2023-07-14T16:13:59.294198+0000","last_active":"2023-07-14T16:14:20.667209+0000","last_peered":"2023-07-14T16:14:20.667209+0000","last_clean":"2023-07-14T16:14:20.667209+0000","last_became_active":"2023-07-14T16:13:59.293891+0000","last_became_peered":"2023-07-14T16:13:59.293891+0000","last_unstale":"2023-07-14T16:14:20.667209+0000","last_undegraded":"2023-07-14T16:14:20.667209+0000","last_fullsized":"2023-07-14T16:14:20.667209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:41.192604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665734+0000","last_change":"2023-07-14T16:13:59.298724+0000","last_active":"2023-07-14T16:14:20.665734+0000","last_peered":"2023-07-14T16:14:20.665734+0000","last_clean":"2023-07-14T16:14:20.665734+0000","last_became_active":"2023-07-14T16:13:59.298318+0000","last_became_peered":"2023-07-14T16:13:59.298318+0000","last_unstale":"2023-07-14T16:14:20.665734+0000","last_undegraded":"2023-07-14T16:14:20.665734+0000","last_fullsized":"2023-07-14T16:14:20.665734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:30:13.650949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763466+0000","last_change":"2023-07-14T16:13:59.294652+0000","last_active":"2023-07-14T16:14:20.763466+0000","last_peered":"2023-07-14T16:14:20.763466+0000","last_clean":"2023-07-14T16:14:20.763466+0000","last_became_active":"2023-07-14T16:13:59.294278+0000","last_became_peered":"2023-07-14T16:13:59.294278+0000","last_unstale":"2023-07-14T16:14:20.763466+0000","last_undegraded":"2023-07-14T16:14:20.763466+0000","last_fullsized":"2023-07-14T16:14:20.763466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:45:58.125276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660903+0000","last_change":"2023-07-14T16:13:59.299228+0000","last_active":"2023-07-14T16:14:20.660903+0000","last_peered":"2023-07-14T16:14:20.660903+0000","last_clean":"2023-07-14T16:14:20.660903+0000","last_became_active":"2023-07-14T16:13:59.297982+0000","last_became_peered":"2023-07-14T16:13:59.297982+0000","last_unstale":"2023-07-14T16:14:20.660903+0000","last_undegraded":"2023-07-14T16:14:20.660903+0000","last_fullsized":"2023-07-14T16:14:20.660903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:24.432565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.668368+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:14:20.668368+0000","last_peered":"2023-07-14T16:14:20.668368+0000","last_clean":"2023-07-14T16:14:20.668368+0000","last_became_active":"2023-07-14T16:13:59.299181+0000","last_became_peered":"2023-07-14T16:13:59.299181+0000","last_unstale":"2023-07-14T16:14:20.668368+0000","last_undegraded":"2023-07-14T16:14:20.668368+0000","last_fullsized":"2023-07-14T16:14:20.668368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:42.604278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664195+0000","last_change":"2023-07-14T16:13:59.303282+0000","last_active":"2023-07-14T16:14:20.664195+0000","last_peered":"2023-07-14T16:14:20.664195+0000","last_clean":"2023-07-14T16:14:20.664195+0000","last_became_active":"2023-07-14T16:13:59.302922+0000","last_became_peered":"2023-07-14T16:13:59.302922+0000","last_unstale":"2023-07-14T16:14:20.664195+0000","last_undegraded":"2023-07-14T16:14:20.664195+0000","last_fullsized":"2023-07-14T16:14:20.664195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.750328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664788+0000","last_change":"2023-07-14T16:13:59.296389+0000","last_active":"2023-07-14T16:14:20.664788+0000","last_peered":"2023-07-14T16:14:20.664788+0000","last_clean":"2023-07-14T16:14:20.664788+0000","last_became_active":"2023-07-14T16:13:59.296080+0000","last_became_peered":"2023-07-14T16:13:59.296080+0000","last_unstale":"2023-07-14T16:14:20.664788+0000","last_undegraded":"2023-07-14T16:14:20.664788+0000","last_fullsized":"2023-07-14T16:14:20.664788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:02.060808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663247+0000","last_change":"2023-07-14T16:13:59.297774+0000","last_active":"2023-07-14T16:14:20.663247+0000","last_peered":"2023-07-14T16:14:20.663247+0000","last_clean":"2023-07-14T16:14:20.663247+0000","last_became_active":"2023-07-14T16:13:59.297463+0000","last_became_peered":"2023-07-14T16:13:59.297463+0000","last_unstale":"2023-07-14T16:14:20.663247+0000","last_undegraded":"2023-07-14T16:14:20.663247+0000","last_fullsized":"2023-07-14T16:14:20.663247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:49.304022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763413+0000","last_change":"2023-07-14T16:13:59.306795+0000","last_active":"2023-07-14T16:14:20.763413+0000","last_peered":"2023-07-14T16:14:20.763413+0000","last_clean":"2023-07-14T16:14:20.763413+0000","last_became_active":"2023-07-14T16:13:59.306457+0000","last_became_peered":"2023-07-14T16:13:59.306457+0000","last_unstale":"2023-07-14T16:14:20.763413+0000","last_undegraded":"2023-07-14T16:14:20.763413+0000","last_fullsized":"2023-07-14T16:14:20.763413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:47.873750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763333+0000","last_change":"2023-07-14T16:13:59.305338+0000","last_active":"2023-07-14T16:14:20.763333+0000","last_peered":"2023-07-14T16:14:20.763333+0000","last_clean":"2023-07-14T16:14:20.763333+0000","last_became_active":"2023-07-14T16:13:59.305047+0000","last_became_peered":"2023-07-14T16:13:59.305047+0000","last_unstale":"2023-07-14T16:14:20.763333+0000","last_undegraded":"2023-07-14T16:14:20.763333+0000","last_fullsized":"2023-07-14T16:14:20.763333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:07:02.694956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660864+0000","last_change":"2023-07-14T16:13:59.299005+0000","last_active":"2023-07-14T16:14:20.660864+0000","last_peered":"2023-07-14T16:14:20.660864+0000","last_clean":"2023-07-14T16:14:20.660864+0000","last_became_active":"2023-07-14T16:13:59.298705+0000","last_became_peered":"2023-07-14T16:13:59.298705+0000","last_unstale":"2023-07-14T16:14:20.660864+0000","last_undegraded":"2023-07-14T16:14:20.660864+0000","last_fullsized":"2023-07-14T16:14:20.660864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:59:41.986408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247144,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61435,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":326,"num_read_kb":227,"num_write":148,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":184305,"data_compressed":4809,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":459298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.5},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.86699999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.036}]},{"osd":6,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.95699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.161}]},{"osd":10,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.91600000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1089536,"data_stored":1108355,"data_compressed":37379,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75800000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.505,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1080000000000001,"5min":1.173,"15min":1.173},"last":0.92400000000000004}]},{"osd":3,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.85099999999999998,"15min":0.84399999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.1440000000000001,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.82599999999999996,"15min":0.81599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.4429999999999996,"5min":4.4429999999999996,"15min":4.4429999999999996},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92400000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.755,"15min":0.76600000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":1.613,"15min":1.613},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.94299999999999995,"15min":0.92200000000000004},"min":{"1min":0.62,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":4.9749999999999996,"5min":4.9749999999999996,"15min":4.9749999999999996},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.91500000000000004,"15min":0.92100000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5289999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.0369999999999999}]},{"osd":6,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.026,"15min":0.95599999999999996},"min":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":6.609,"5min":6.609,"15min":6.609},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.228,"5min":0.97699999999999998,"15min":0.93500000000000005},"min":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":6.5759999999999996,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.83599999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.61699999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.98499999999999999,"15min":0.98599999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.006},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.996,"15min":1.012},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.159,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":0.99299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.1359999999999999,"15min":1.0660000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":7.3579999999999997,"5min":7.3579999999999997,"15min":7.3579999999999997},"last":1.0229999999999999}]},{"osd":11,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.097}]}]},{"osd":9,"up_from":68,"seq":292057776164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":459512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":1.492},{"interface":"front","average":{"1min":0.501,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70599999999999996,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82099999999999995,"15min":0.83299999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.032,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.94199999999999995,"5min":1.129,"15min":1.129},"last":0.995}]},{"osd":2,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.86099999999999999,"15min":0.875},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.86299999999999999,"15min":0.88400000000000001},"min":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.98899999999999999,"5min":1.151,"15min":1.151},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.64600000000000002,"15min":0.66400000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":1.6859999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.47399999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.228,"5min":1.268,"15min":1.268},"last":1.361}]},{"osd":4,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.92600000000000005,"15min":0.94799999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96199999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":2.2480000000000002}]},{"osd":5,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.099,"5min":1.125,"15min":1.125},"last":2.1259999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.97699999999999998,"15min":0.99399999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.3220000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.996,"15min":1.022},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.448},{"interface":"front","average":{"1min":0.79700000000000004,"5min":1,"15min":1.034},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1879999999999999,"5min":1.288,"15min":1.288},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.96799999999999997,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.157,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":2.21},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0680000000000001,"15min":1.087},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.3979999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0740000000000001,"15min":1.101},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2529999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.0269999999999999,"15min":1.0549999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":2.169}]},{"osd":10,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.004,"15min":1.0129999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.296,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.95699999999999996,"15min":0.96499999999999997},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":2.2879999999999998}]}]},{"osd":8,"up_from":60,"seq":257698037801,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":459410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66400000000000003,"15min":0.68899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88300000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72899999999999998,"15min":0.69799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":6.3410000000000002,"15min":6.3410000000000002},"last":0.621}]},{"osd":1,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98699999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.48799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.1100000000000001,"5min":1.216,"15min":1.216},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.081,"15min":1.081},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99099999999999999,"5min":1.175,"15min":1.175},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.78700000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92600000000000005,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.54200000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.878},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.61499999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.0289999999999999,"5min":1.218,"15min":1.218},"last":0.82799999999999996}]},{"osd":5,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69899999999999995,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73599999999999999,"15min":0.74399999999999999},"min":{"1min":0.46500000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.05,"5min":1.21,"15min":1.21},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84199999999999997,"15min":0.85799999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1839999999999999,"5min":1.29,"15min":1.29},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.63500000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.256,"15min":1.256},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.93799999999999994,"15min":0.89300000000000002},"min":{"1min":0.63200000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.222,"5min":6.3209999999999997,"15min":6.3209999999999997},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89000000000000001,"15min":0.89800000000000002},"min":{"1min":0.68400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.1919999999999999,"5min":1.373,"15min":1.373},"last":1.107}]},{"osd":9,"last update":"Fri Jul 14 16:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":0.90200000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.02,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0409999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.91800000000000004}]},{"osd":10,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.873,"15min":0.85599999999999998},"min":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.048},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.89000000000000001,"15min":0.877},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1619999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":1.024}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":7,"up_from":52,"seq":223338299438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":1051271,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.181,"15min":1.181},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76400000000000001,"5min":1.046,"15min":1.046},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95899999999999996,"5min":1.008,"15min":1.008},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93799999999999994,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70599999999999996}]},{"osd":3,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66600000000000004,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.85399999999999998,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96299999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.46500000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90900000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.89100000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.58899999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0589999999999999,"5min":1.246,"15min":1.246},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.77700000000000002},"min":{"1min":0.624,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.94399999999999995}]},{"osd":6,"last update":"Fri Jul 14 16:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.56499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0629999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78900000000000003,"15min":0.78100000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.997,"5min":1.137,"15min":1.137},"last":0.84999999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.67400000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0369999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.874,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.69999999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.079,"5min":1.331,"15min":1.331},"last":0.876}]},{"osd":9,"last update":"Fri Jul 14 16:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89400000000000002,"15min":0.89600000000000002},"min":{"1min":0.69799999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1910000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.86699999999999999,"15min":0.86899999999999999},"min":{"1min":0.61899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0660000000000001,"5min":1.262,"15min":1.262},"last":0.93000000000000005}]},{"osd":10,"last update":"Fri Jul 14 16:14:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.879,"15min":0.88500000000000001},"min":{"1min":0.628,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.036,"5min":1.238,"15min":1.238},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.90100000000000002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.042,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.96699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":458907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.68999999999999995},"min":{"1min":0.5,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.67800000000000005},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98599999999999999,"5min":1.111,"15min":1.111},"last":0.623}]},{"osd":2,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0009999999999999,"5min":1.121,"15min":1.121},"last":0.747},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94799999999999995,"5min":1.105,"15min":1.105},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.87},{"interface":"front","average":{"1min":0.877,"5min":0.79100000000000004,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82099999999999995,"15min":0.81000000000000005},"min":{"1min":0.53800000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0720000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.90400000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81599999999999995,"15min":0.78300000000000003},"min":{"1min":0.56999999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0469999999999999,"5min":1.28,"15min":1.28},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83899999999999997,"15min":0.80200000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.145,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.85399999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93000000000000005,"15min":0.93700000000000006},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2929999999999999,"5min":1.353,"15min":1.353},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.873,"15min":0.86499999999999999},"min":{"1min":0.72299999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2470000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.88100000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2170000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.97899999999999998,"15min":0.997},"min":{"1min":0.56499999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2330000000000001,"5min":1.331,"15min":1.331},"last":0.83099999999999996}]},{"osd":10,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.97799999999999998,"15min":0.99099999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.179,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.95299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":516225,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61399999999999999,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60699999999999998,"15min":0.53700000000000003},"min":{"1min":0.42099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.627}]},{"osd":2,"last update":"Fri Jul 14 16:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.65800000000000003,"15min":0.59899999999999998},"min":{"1min":0.54100000000000004,"5min":0.377,"15min":0.377},"max":{"1min":1.01,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.66500000000000004,"15min":0.621},"min":{"1min":0.58699999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":1.018}]},{"osd":3,"last update":"Fri Jul 14 16:14:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64800000000000002,"15min":0.60199999999999998},"min":{"1min":0.44900000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97899999999999998,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64100000000000001,"15min":0.59899999999999998},"min":{"1min":0.52300000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.442}]},{"osd":4,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"min":{"1min":0.42999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.53600000000000003,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95899999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Jul 14 16:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97499999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.53500000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82599999999999996}]},{"osd":6,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.753},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.72899999999999998,"15min":0.69299999999999995},"min":{"1min":0.63400000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.72099999999999997,"15min":0.67100000000000004},"min":{"1min":0.55900000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.113,"5min":1.244,"15min":1.244},"last":0.85199999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2050000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.81200000000000006,"15min":0.77200000000000002},"min":{"1min":0.75,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.86499999999999999}]},{"osd":9,"last update":"Fri Jul 14 16:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.872,"15min":0.875},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.69699999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.2030000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.95599999999999996}]},{"osd":10,"last update":"Fri Jul 14 16:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.91900000000000004,"15min":0.90900000000000003},"min":{"1min":0.72599999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2609999999999999,"5min":1.262,"15min":1.262},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93200000000000005,"15min":0.93000000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.90600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":460172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63800000000000001,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69899999999999995,"15min":0.626},"min":{"1min":0.60099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.71199999999999997,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.46000000000000002},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74399999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.37},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.47999999999999998},"min":{"1min":0.34499999999999997,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Jul 14 16:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.45400000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Jul 14 16:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.42999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.44},{"interface":"front","average":{"1min":0.749,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.748,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83799999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.92100000000000004},"min":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.85899999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.63500000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.871,"15min":0.878},"min":{"1min":0.54300000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.93300000000000005,"15min":0.94999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.399,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.65500000000000003}]},{"osd":10,"last update":"Fri Jul 14 16:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.92000000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.95299999999999996,"15min":0.94599999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":3.4900000000000002,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.72199999999999998}]},{"osd":11,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.75800000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":517527,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.54900000000000004},"min":{"1min":0.379,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.753},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.56299999999999994},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86499999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Jul 14 16:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90500000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0549999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Jul 14 16:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.70199999999999996,"15min":0.65600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65300000000000002,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93000000000000005,"5min":0.999,"15min":0.999},"last":0.63}]},{"osd":4,"last update":"Fri Jul 14 16:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96399999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":1.022,"15min":1.022},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.76600000000000001,"15min":0.71999999999999997},"min":{"1min":0.57699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.7240000000000002,"5min":3.7240000000000002,"15min":3.7240000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.80600000000000005,"15min":0.75800000000000001},"min":{"1min":0.53200000000000003,"5min":0.439,"15min":0.439},"max":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.67000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.022,"5min":1.117,"15min":1.117},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99099999999999999,"5min":1.016,"15min":1.016},"last":0.56999999999999995}]},{"osd":9,"last update":"Fri Jul 14 16:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1890000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.72999999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:14:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.61799999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.048,"5min":1.246,"15min":1.246},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.89100000000000001,"15min":0.88600000000000001},"min":{"1min":0.69099999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.204,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":458943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.45400000000000001,"15min":0.40899999999999997},"min":{"1min":0.29399999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.375,"5min":2.375,"15min":2.375},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.436},"min":{"1min":0.316,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":0.437}]},{"osd":1,"last update":"Fri Jul 14 16:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.67000000000000004,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.48799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.67700000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90600000000000003,"5min":1.137,"15min":1.137},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.71699999999999997},"min":{"1min":0.52800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.3050000000000002,"5min":3.448,"15min":3.448},"last":0.63800000000000001}]},{"osd":4,"last update":"Fri Jul 14 16:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.66000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92700000000000005,"5min":3.359,"15min":3.359},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71099999999999997,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96499999999999997,"5min":1.222,"15min":1.222},"last":0.83999999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.74199999999999999,"15min":0.71499999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76500000000000001,"15min":0.73499999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Fri Jul 14 16:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.66500000000000004},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.042,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0329999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.62}]},{"osd":7,"last update":"Fri Jul 14 16:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.82499999999999996,"15min":0.79100000000000004},"min":{"1min":0.51500000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.1339999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.006},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.89600000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.876,"15min":0.88500000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1259999999999999,"5min":1.546,"15min":1.546},"last":0.62},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91800000000000004,"15min":0.92100000000000004},"min":{"1min":0.61899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.163,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1659999999999999,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.145,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri Jul 14 16:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.91800000000000004,"15min":0.91500000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.9860000000000002,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0109999999999999,"15min":1.016},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.042}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":459680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.95599999999999996,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46800000000000003,"15min":0.432},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.501,"15min":0.45700000000000002},"min":{"1min":0.35499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68000000000000005,"15min":0.64200000000000002},"min":{"1min":0.505,"5min":0.439,"15min":0.439},"max":{"1min":1.048,"5min":3.698,"15min":3.698},"last":0.87},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.70199999999999996,"15min":0.67800000000000005},"min":{"1min":0.51600000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1160000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.78400000000000003}]},{"osd":3,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.54900000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.995,"5min":1.03,"15min":1.03},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Jul 14 16:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.63700000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.006,"5min":1.506,"15min":1.506},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72599999999999998,"15min":0.71799999999999997},"min":{"1min":0.54300000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.079,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57099999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.147,"5min":1.321,"15min":1.321},"last":0.62},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.5}]},{"osd":7,"last update":"Fri Jul 14 16:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0189999999999999,"5min":1.038,"15min":1.038},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.751,"15min":0.751},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.042,"5min":1.129,"15min":1.129},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85999999999999999,"15min":0.84299999999999997},"min":{"1min":0.753,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79100000000000004,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.753}]},{"osd":9,"last update":"Fri Jul 14 16:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.91700000000000004,"15min":0.92300000000000004},"min":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.127,"5min":1.631,"15min":1.631},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86299999999999999,"15min":0.874},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0029999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.54800000000000004}]},{"osd":10,"last update":"Fri Jul 14 16:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":1050282,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92900000000000005,"5min":1.099,"15min":1.099},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65900000000000003,"15min":0.625},"min":{"1min":0.54600000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95299999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.91200000000000003}]},{"osd":2,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.498,"15min":0.48499999999999999},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.75900000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.495,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.312,"15min":0.312},"max":{"1min":0.72899999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66000000000000003,"15min":0.64200000000000002},"min":{"1min":0.505,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71699999999999997,"15min":0.66300000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0289999999999999,"5min":7.056,"15min":7.056},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Jul 14 16:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.73199999999999998,"15min":0.70399999999999996},"min":{"1min":0.59199999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.1100000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Jul 14 16:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.081,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.2569999999999999,"5min":7.109,"15min":7.109},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.751,"15min":0.745},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89800000000000002,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97499999999999998,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.872}]},{"osd":8,"last update":"Fri Jul 14 16:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.1479999999999999,"5min":1.286,"15min":1.286},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97899999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.78500000000000003}]},{"osd":9,"last update":"Fri Jul 14 16:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.876,"15min":0.876},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86399999999999999,"15min":0.86499999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.165,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.85999999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.198,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.91900000000000004,"15min":0.91600000000000004},"min":{"1min":0.73599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.167,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.95199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":57531,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58833,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":59451,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T16:14:56.334 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-14T16:14:56.335 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-14T16:14:56.335 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-14T16:14:56.335 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph health --format=json 2023-07-14T16:14:56.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:56 smithi062 ceph-mon[95543]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:56.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:56 smithi081 ceph-mon[127795]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:57.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:56 smithi164 ceph-mon[97427]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:14:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:14:57.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:57 smithi062 ceph-mon[95543]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:57.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:57 smithi081 ceph-mon[127795]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:58.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:57 smithi164 ceph-mon[97427]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:14:58.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 -- 172.21.15.62:0/2732445495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c104ea0 msgr2=0x7fbd6c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:58.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2732445495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c104ea0 0x7fbd6c10ca20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400b0d0 tx=0x7fbd6400b3e0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 -- 172.21.15.62:0/2732445495 shutdown_connections 2023-07-14T16:14:58.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2732445495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c104ea0 0x7fbd6c10ca20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2732445495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c0ffe00 0x7fbd6c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2732445495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd6c1007a0 0x7fbd6c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 -- 172.21.15.62:0/2732445495 >> 172.21.15.62:0/2732445495 conn(0x7fbd6c0fb660 msgr2=0x7fbd6c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:58.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 -- 172.21.15.62:0/2732445495 shutdown_connections 2023-07-14T16:14:58.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.527+0000 7fbd709c1700 1 -- 172.21.15.62:0/2732445495 wait complete. 2023-07-14T16:14:58.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd709c1700 1 Processor -- start 2023-07-14T16:14:58.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd709c1700 1 -- start start 2023-07-14T16:14:58.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd709c1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c0ffe00 0x7fbd6c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:58.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd709c1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c1007a0 0x7fbd6c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:58.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd709c1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd6c104ea0 0x7fbd6c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:58.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd709c1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c19ce30 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd709c1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c19cfa0 con 0x7fbd6c1007a0 2023-07-14T16:14:58.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd709c1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c19d160 con 0x7fbd6c104ea0 2023-07-14T16:14:58.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.529+0000 7fbd6a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c0ffe00 0x7fbd6c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:58.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c0ffe00 0x7fbd6c197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59532/0 (socket says 172.21.15.62:59532) 2023-07-14T16:14:58.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6a59c700 1 -- 172.21.15.62:0/2878926892 learned_addr learned my addr 172.21.15.62:0/2878926892 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:58.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd69d9b700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c1007a0 0x7fbd6c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:58.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6ad9d700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd6c104ea0 0x7fbd6c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:58.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6a59c700 1 -- 172.21.15.62:0/2878926892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd6c104ea0 msgr2=0x7fbd6c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:58.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6a59c700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd6c104ea0 0x7fbd6c19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6a59c700 1 -- 172.21.15.62:0/2878926892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c1007a0 msgr2=0x7fbd6c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:58.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6a59c700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c1007a0 0x7fbd6c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6a59c700 1 -- 172.21.15.62:0/2878926892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009d00 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd69d9b700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c1007a0 0x7fbd6c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:58.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6ad9d700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd6c104ea0 0x7fbd6c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:58.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd6a59c700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c0ffe00 0x7fbd6c197e70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00e9a0 tx=0x7fbd5c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:58.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd5b7fe700 1 -- 172.21.15.62:0/2878926892 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c009790 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.530+0000 7fbd5b7fe700 1 -- 172.21.15.62:0/2878926892 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd5c00cbd0 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.531+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c19d440 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.531+0000 7fbd5b7fe700 1 -- 172.21.15.62:0/2878926892 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c018680 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.531+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c1a4ca0 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.532+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c04e210 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.534+0000 7fbd5b7fe700 1 -- 172.21.15.62:0/2878926892 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbd5c00cd40 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.535+0000 7fbd5b7fe700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd540a1260 0x7fbd540a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.535+0000 7fbd5b7fe700 1 -- 172.21.15.62:0/2878926892 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbd5c014070 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.535+0000 7fbd69d9b700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd540a1260 0x7fbd540a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.536+0000 7fbd69d9b700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd540a1260 0x7fbd540a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd60005e10 tx=0x7fbd60005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.539+0000 7fbd5b7fe700 1 -- 172.21.15.62:0/2878926892 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbd5c051790 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.791+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbd6c1991d0 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.792+0000 7fbd5b7fe700 1 -- 172.21.15.62:0/2878926892 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbd5c099e70 con 0x7fbd6c0ffe00 2023-07-14T16:14:58.770 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:58.771 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-14T16:14:58.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd540a1260 msgr2=0x7fbd540a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:58.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd540a1260 0x7fbd540a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd60005e10 tx=0x7fbd60005d60 comp rx=0 tx=0).stop 2023-07-14T16:14:58.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c0ffe00 msgr2=0x7fbd6c197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:58.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c0ffe00 0x7fbd6c197e70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00e9a0 tx=0x7fbd5c00ed60 comp rx=0 tx=0).stop 2023-07-14T16:14:58.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 shutdown_connections 2023-07-14T16:14:58.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd6c104ea0 0x7fbd6c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd6c0ffe00 0x7fbd6c197e70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd540a1260 0x7fbd540a3710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 --2- 172.21.15.62:0/2878926892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c1007a0 0x7fbd6c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.794+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 >> 172.21.15.62:0/2878926892 conn(0x7fbd6c0fb660 msgr2=0x7fbd6c0fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.795+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 shutdown_connections 2023-07-14T16:14:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:58.795+0000 7fbd709c1700 1 -- 172.21.15.62:0/2878926892 wait complete. 2023-07-14T16:14:58.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:58 smithi062 ceph-mon[95543]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:14:58.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:58 smithi081 ceph-mon[127795]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:14:59.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:58 smithi164 ceph-mon[97427]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:14:59.284 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-14T16:14:59.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:14:59.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.514+0000 7f8283dff700 1 -- 172.21.15.62:0/2066618956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c100cf0 msgr2=0x7f827c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:59.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.514+0000 7f8283dff700 1 --2- 172.21.15.62:0/2066618956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c100cf0 0x7f827c101140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f826c009990 tx=0x7f826c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.515+0000 7f8283dff700 1 -- 172.21.15.62:0/2066618956 shutdown_connections 2023-07-14T16:14:59.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.515+0000 7f8283dff700 1 --2- 172.21.15.62:0/2066618956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c101680 0x7f827c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.515+0000 7f8283dff700 1 --2- 172.21.15.62:0/2066618956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c0ffb00 0x7f827c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.515+0000 7f8283dff700 1 --2- 172.21.15.62:0/2066618956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c100cf0 0x7f827c101140 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.515+0000 7f8283dff700 1 -- 172.21.15.62:0/2066618956 >> 172.21.15.62:0/2066618956 conn(0x7f827c0fb0e0 msgr2=0x7f827c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:59.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.515+0000 7f8283dff700 1 -- 172.21.15.62:0/2066618956 shutdown_connections 2023-07-14T16:14:59.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.515+0000 7f8283dff700 1 -- 172.21.15.62:0/2066618956 wait complete. 2023-07-14T16:14:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8283dff700 1 Processor -- start 2023-07-14T16:14:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8283dff700 1 -- start start 2023-07-14T16:14:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8283dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0ffb00 0x7f827c195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8283dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c100cf0 0x7f827c195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8283dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c101680 0x7f827c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8283dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c19d2d0 con 0x7f827c0ffb00 2023-07-14T16:14:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8283dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c19d410 con 0x7f827c100cf0 2023-07-14T16:14:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8283dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c19d5a0 con 0x7f827c101680 2023-07-14T16:14:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8281b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0ffb00 0x7f827c195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8281b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0ffb00 0x7f827c195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59546/0 (socket says 172.21.15.62:59546) 2023-07-14T16:14:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.516+0000 7f8281b9b700 1 -- 172.21.15.62:0/3812079559 learned_addr learned my addr 172.21.15.62:0/3812079559 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f828139a700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c100cf0 0x7f827c195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f828239c700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c101680 0x7f827c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8281b9b700 1 -- 172.21.15.62:0/3812079559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c101680 msgr2=0x7f827c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8281b9b700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c101680 0x7f827c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8281b9b700 1 -- 172.21.15.62:0/3812079559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c100cf0 msgr2=0x7f827c195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8281b9b700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c100cf0 0x7f827c195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8281b9b700 1 -- 172.21.15.62:0/3812079559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c0095f0 con 0x7f827c0ffb00 2023-07-14T16:14:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f828239c700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c101680 0x7f827c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8281b9b700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0ffb00 0x7f827c195750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f827800d980 tx=0x7f827800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:59.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8272ffd700 1 -- 172.21.15.62:0/3812079559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278004020 con 0x7f827c0ffb00 2023-07-14T16:14:59.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8272ffd700 1 -- 172.21.15.62:0/3812079559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8278009730 con 0x7f827c0ffb00 2023-07-14T16:14:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c19d880 con 0x7f827c0ffb00 2023-07-14T16:14:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.517+0000 7f8272ffd700 1 -- 172.21.15.62:0/3812079559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278003a00 con 0x7f827c0ffb00 2023-07-14T16:14:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.518+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c19dde0 con 0x7f827c0ffb00 2023-07-14T16:14:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.518+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c1070b0 con 0x7f827c0ffb00 2023-07-14T16:14:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.520+0000 7f8272ffd700 1 -- 172.21.15.62:0/3812079559 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f82780098a0 con 0x7f827c0ffb00 2023-07-14T16:14:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.522+0000 7f8272ffd700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82680a12a0 0x7f82680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.522+0000 7f8272ffd700 1 -- 172.21.15.62:0/3812079559 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f82780cc760 con 0x7f827c0ffb00 2023-07-14T16:14:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.522+0000 7f828139a700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82680a12a0 0x7f82680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.523+0000 7f828139a700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82680a12a0 0x7f82680a3750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c010 tx=0x7f826c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.526+0000 7f8272ffd700 1 -- 172.21.15.62:0/3812079559 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8278095d00 con 0x7f827c0ffb00 2023-07-14T16:14:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.741+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f827c02c850 con 0x7f827c0ffb00 2023-07-14T16:14:59.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.743+0000 7f8272ffd700 1 -- 172.21.15.62:0/3812079559 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f82780232f0 con 0x7f827c0ffb00 2023-07-14T16:14:59.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:14:59.722 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:14:59.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82680a12a0 msgr2=0x7f82680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:59.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82680a12a0 0x7f82680a3750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c010 tx=0x7f826c005fd0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0ffb00 msgr2=0x7f827c195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0ffb00 0x7f827c195750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f827800d980 tx=0x7f827800dc90 comp rx=0 tx=0).stop 2023-07-14T16:14:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 shutdown_connections 2023-07-14T16:14:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c101680 0x7f827c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0ffb00 0x7f827c195750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82680a12a0 0x7f82680a3750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 --2- 172.21.15.62:0/3812079559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c100cf0 0x7f827c195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.745+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 >> 172.21.15.62:0/3812079559 conn(0x7f827c0fb0e0 msgr2=0x7f827c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.746+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 shutdown_connections 2023-07-14T16:14:59.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.746+0000 7f8283dff700 1 -- 172.21.15.62:0/3812079559 wait complete. 2023-07-14T16:14:59.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:14:59.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:14:59.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:14:59 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2878926892' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T16:14:59.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:14:59 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2878926892' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T16:14:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 -- 172.21.15.62:0/2275613587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd94100cd0 msgr2=0x7fcd94101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2275613587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd94100cd0 0x7fcd94101120 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009990 tx=0x7fcd80009ca0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 -- 172.21.15.62:0/2275613587 shutdown_connections 2023-07-14T16:14:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2275613587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94101660 0x7fcd94107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2275613587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd940ffae0 0x7fcd940ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2275613587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd94100cd0 0x7fcd94101120 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 -- 172.21.15.62:0/2275613587 >> 172.21.15.62:0/2275613587 conn(0x7fcd940fb0e0 msgr2=0x7fcd940fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:14:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 -- 172.21.15.62:0/2275613587 shutdown_connections 2023-07-14T16:14:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.973+0000 7fcd9abff700 1 -- 172.21.15.62:0/2275613587 wait complete. 2023-07-14T16:14:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.974+0000 7fcd9abff700 1 Processor -- start 2023-07-14T16:14:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.974+0000 7fcd9abff700 1 -- start start 2023-07-14T16:14:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.974+0000 7fcd9abff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd940ffae0 0x7fcd941979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9abff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd94100cd0 0x7fcd94197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9abff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94101660 0x7fcd9419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9abff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419f540 con 0x7fcd940ffae0 2023-07-14T16:14:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9abff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419f680 con 0x7fcd94101660 2023-07-14T16:14:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9abff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419f810 con 0x7fcd94100cd0 2023-07-14T16:14:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd940ffae0 0x7fcd941979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd940ffae0 0x7fcd941979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59570/0 (socket says 172.21.15.62:59570) 2023-07-14T16:14:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 -- 172.21.15.62:0/2330706171 learned_addr learned my addr 172.21.15.62:0/2330706171 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:14:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9919c700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94101660 0x7fcd9419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd93fff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd94100cd0 0x7fcd94197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 -- 172.21.15.62:0/2330706171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd94100cd0 msgr2=0x7fcd94197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd94100cd0 0x7fcd94197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 -- 172.21.15.62:0/2330706171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94101660 msgr2=0x7fcd9419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94101660 0x7fcd9419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 -- 172.21.15.62:0/2330706171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd800095f0 con 0x7fcd940ffae0 2023-07-14T16:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9919c700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94101660 0x7fcd9419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.975+0000 7fcd9899b700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd940ffae0 0x7fcd941979c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d980 tx=0x7fcd8800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.976+0000 7fcd91ffb700 1 -- 172.21.15.62:0/2330706171 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800de60 con 0x7fcd940ffae0 2023-07-14T16:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.976+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9419faf0 con 0x7fcd940ffae0 2023-07-14T16:14:59.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.976+0000 7fcd91ffb700 1 -- 172.21.15.62:0/2330706171 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd880044b0 con 0x7fcd940ffae0 2023-07-14T16:14:59.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.976+0000 7fcd91ffb700 1 -- 172.21.15.62:0/2330706171 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88003b40 con 0x7fcd940ffae0 2023-07-14T16:14:59.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.976+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd941a0000 con 0x7fcd940ffae0 2023-07-14T16:14:59.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.977+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9410b1b0 con 0x7fcd940ffae0 2023-07-14T16:14:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.979+0000 7fcd91ffb700 1 -- 172.21.15.62:0/2330706171 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcd88003ca0 con 0x7fcd940ffae0 2023-07-14T16:14:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.980+0000 7fcd91ffb700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcd840a11e0 0x7fcd840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:14:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.980+0000 7fcd91ffb700 1 -- 172.21.15.62:0/2330706171 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcd880cc820 con 0x7fcd940ffae0 2023-07-14T16:14:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.980+0000 7fcd93fff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcd840a11e0 0x7fcd840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:14:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.981+0000 7fcd93fff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcd840a11e0 0x7fcd840a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006010 tx=0x7fcd8000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:14:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:14:59.984+0000 7fcd91ffb700 1 -- 172.21.15.62:0/2330706171 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcd88095dc0 con 0x7fcd940ffae0 2023-07-14T16:15:00.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:14:59 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2878926892' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T16:15:00.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.192+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd941a05b0 con 0x7fcd940ffae0 2023-07-14T16:15:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.193+0000 7fcd91ffb700 1 -- 172.21.15.62:0/2330706171 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd88009940 con 0x7fcd940ffae0 2023-07-14T16:15:00.172 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:15:00.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcd840a11e0 msgr2=0x7fcd840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcd840a11e0 0x7fcd840a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006010 tx=0x7fcd8000b260 comp rx=0 tx=0).stop 2023-07-14T16:15:00.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd940ffae0 msgr2=0x7fcd941979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd940ffae0 0x7fcd941979c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d980 tx=0x7fcd8800dd40 comp rx=0 tx=0).stop 2023-07-14T16:15:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 shutdown_connections 2023-07-14T16:15:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcd840a11e0 0x7fcd840a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94101660 0x7fcd9419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd940ffae0 0x7fcd941979c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 --2- 172.21.15.62:0/2330706171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd94100cd0 0x7fcd94197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 >> 172.21.15.62:0/2330706171 conn(0x7fcd940fb0e0 msgr2=0x7fcd940fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 shutdown_connections 2023-07-14T16:15:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.195+0000 7fcd9abff700 1 -- 172.21.15.62:0/2330706171 wait complete. 2023-07-14T16:15:00.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:15:00.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/2725496468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28101670 msgr2=0x7f2c28107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/2725496468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28101670 0x7f2c28107f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000b0a0 tx=0x7f2c2000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/2725496468 shutdown_connections 2023-07-14T16:15:00.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/2725496468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28101670 0x7f2c28107f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/2725496468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c280ffaf0 0x7f2c280fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/2725496468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c28100ce0 0x7f2c28101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/2725496468 >> 172.21.15.62:0/2725496468 conn(0x7f2c280fb130 msgr2=0x7f2c280fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:00.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/2725496468 shutdown_connections 2023-07-14T16:15:00.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.423+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/2725496468 wait complete. 2023-07-14T16:15:00.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.424+0000 7f2c2e1ff700 1 Processor -- start 2023-07-14T16:15:00.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.424+0000 7f2c2e1ff700 1 -- start start 2023-07-14T16:15:00.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c2e1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c280ffaf0 0x7f2c280742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:00.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c2e1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28100ce0 0x7f2c28072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:00.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c2e1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c28101670 0x7f2c28072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:00.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c2e1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c280735e0 con 0x7f2c28100ce0 2023-07-14T16:15:00.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c2e1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c28073750 con 0x7f2c280ffaf0 2023-07-14T16:15:00.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c2e1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2819f810 con 0x7f2c28101670 2023-07-14T16:15:00.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c26ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28100ce0 0x7f2c28072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:00.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c26ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28100ce0 0x7f2c28072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59596/0 (socket says 172.21.15.62:59596) 2023-07-14T16:15:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c26ffd700 1 -- 172.21.15.62:0/3388190400 learned_addr learned my addr 172.21.15.62:0/3388190400 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c277fe700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c280ffaf0 0x7f2c280742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c27fff700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c28101670 0x7f2c28072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c26ffd700 1 -- 172.21.15.62:0/3388190400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c28101670 msgr2=0x7f2c28072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c26ffd700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c28101670 0x7f2c28072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c26ffd700 1 -- 172.21.15.62:0/3388190400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c280ffaf0 msgr2=0x7f2c280742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c26ffd700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c280ffaf0 0x7f2c280742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.425+0000 7f2c26ffd700 1 -- 172.21.15.62:0/3388190400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c20009d00 con 0x7f2c28100ce0 2023-07-14T16:15:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.426+0000 7f2c26ffd700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28100ce0 0x7f2c28072900 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00d950 tx=0x7f2c1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.426+0000 7f2c24ff9700 1 -- 172.21.15.62:0/3388190400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c009940 con 0x7f2c28100ce0 2023-07-14T16:15:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.426+0000 7f2c24ff9700 1 -- 172.21.15.62:0/3388190400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c1c010460 con 0x7f2c28100ce0 2023-07-14T16:15:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.426+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2819faf0 con 0x7f2c28100ce0 2023-07-14T16:15:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.426+0000 7f2c24ff9700 1 -- 172.21.15.62:0/3388190400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00f620 con 0x7f2c28100ce0 2023-07-14T16:15:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.426+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c281a0000 con 0x7f2c28100ce0 2023-07-14T16:15:00.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.427+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c28106fc0 con 0x7f2c28100ce0 2023-07-14T16:15:00.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.429+0000 7f2c24ff9700 1 -- 172.21.15.62:0/3388190400 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2c1c0105d0 con 0x7f2c28100ce0 2023-07-14T16:15:00.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.431+0000 7f2c24ff9700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c100a11e0 0x7f2c100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:00.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.431+0000 7f2c24ff9700 1 -- 172.21.15.62:0/3388190400 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c1c0cc900 con 0x7f2c28100ce0 2023-07-14T16:15:00.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.431+0000 7f2c277fe700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c100a11e0 0x7f2c100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.431+0000 7f2c277fe700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c100a11e0 0x7f2c100a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c28100b40 tx=0x7f2c1800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.434+0000 7f2c24ff9700 1 -- 172.21.15.62:0/3388190400 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c1c095f00 con 0x7f2c28100ce0 2023-07-14T16:15:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.647+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c2802ca90 con 0x7f2c28100ce0 2023-07-14T16:15:00.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.647+0000 7f2c24ff9700 1 -- 172.21.15.62:0/3388190400 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c1c016020 con 0x7f2c28100ce0 2023-07-14T16:15:00.626 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:15:00.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c100a11e0 msgr2=0x7f2c100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c100a11e0 0x7f2c100a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c28100b40 tx=0x7f2c1800a3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28100ce0 msgr2=0x7f2c28072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28100ce0 0x7f2c28072900 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00d950 tx=0x7f2c1c00dc60 comp rx=0 tx=0).stop 2023-07-14T16:15:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 shutdown_connections 2023-07-14T16:15:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c28101670 0x7f2c28072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c100a11e0 0x7f2c100a3690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c280ffaf0 0x7f2c280742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 --2- 172.21.15.62:0/3388190400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c28100ce0 0x7f2c28072900 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 >> 172.21.15.62:0/3388190400 conn(0x7f2c280fb130 msgr2=0x7f2c28104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 shutdown_connections 2023-07-14T16:15:00.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.650+0000 7f2c2e1ff700 1 -- 172.21.15.62:0/3388190400 wait complete. 2023-07-14T16:15:00.647 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:15:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 -- 172.21.15.62:0/3831557202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc1005c0 msgr2=0x7f78bc1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 --2- 172.21.15.62:0/3831557202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc1005c0 0x7f78bc1029a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78b800b0a0 tx=0x7f78b800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 -- 172.21.15.62:0/3831557202 shutdown_connections 2023-07-14T16:15:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 --2- 172.21.15.62:0/3831557202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78bc105800 0x7f78bc107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 --2- 172.21.15.62:0/3831557202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc1005c0 0x7f78bc1029a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 --2- 172.21.15.62:0/3831557202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78bc102ee0 0x7f78bc1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 -- 172.21.15.62:0/3831557202 >> 172.21.15.62:0/3831557202 conn(0x7f78bc0fa000 msgr2=0x7f78bc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 -- 172.21.15.62:0/3831557202 shutdown_connections 2023-07-14T16:15:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.876+0000 7f78c35ff700 1 -- 172.21.15.62:0/3831557202 wait complete. 2023-07-14T16:15:00.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.877+0000 7f78c35ff700 1 Processor -- start 2023-07-14T16:15:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.877+0000 7f78c35ff700 1 -- start start 2023-07-14T16:15:00.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c35ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78bc1005c0 0x7f78bc1958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c35ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78bc102ee0 0x7f78bc195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c35ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc105800 0x7f78bc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c35ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc19d420 con 0x7f78bc105800 2023-07-14T16:15:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c35ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc19d560 con 0x7f78bc102ee0 2023-07-14T16:15:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c35ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc19d6f0 con 0x7f78bc1005c0 2023-07-14T16:15:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c1b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc105800 0x7f78bc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c1b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc105800 0x7f78bc19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59608/0 (socket says 172.21.15.62:59608) 2023-07-14T16:15:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c1b9c700 1 -- 172.21.15.62:0/1365372708 learned_addr learned my addr 172.21.15.62:0/1365372708 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c0b9a700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78bc102ee0 0x7f78bc195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:00.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c139b700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78bc1005c0 0x7f78bc1958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c1b9c700 1 -- 172.21.15.62:0/1365372708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78bc1005c0 msgr2=0x7f78bc1958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c1b9c700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78bc1005c0 0x7f78bc1958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c1b9c700 1 -- 172.21.15.62:0/1365372708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78bc102ee0 msgr2=0x7f78bc195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c1b9c700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78bc102ee0 0x7f78bc195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c1b9c700 1 -- 172.21.15.62:0/1365372708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8009d00 con 0x7f78bc105800 2023-07-14T16:15:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.878+0000 7f78c0b9a700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78bc102ee0 0x7f78bc195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:00.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.879+0000 7f78c1b9c700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc105800 0x7f78bc19ce50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78b400e940 tx=0x7f78b400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.879+0000 7f78b27fc700 1 -- 172.21.15.62:0/1365372708 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b40097b0 con 0x7f78bc105800 2023-07-14T16:15:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.879+0000 7f78b27fc700 1 -- 172.21.15.62:0/1365372708 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78b4004cc0 con 0x7f78bc105800 2023-07-14T16:15:00.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.879+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78bc19d9d0 con 0x7f78bc105800 2023-07-14T16:15:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.879+0000 7f78b27fc700 1 -- 172.21.15.62:0/1365372708 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4005560 con 0x7f78bc105800 2023-07-14T16:15:00.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.879+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78bc19dee0 con 0x7f78bc105800 2023-07-14T16:15:00.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.880+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78bc0fbb70 con 0x7f78bc105800 2023-07-14T16:15:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.882+0000 7f78b27fc700 1 -- 172.21.15.62:0/1365372708 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f78b4004750 con 0x7f78bc105800 2023-07-14T16:15:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.884+0000 7f78b27fc700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f78a80a11e0 0x7f78a80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.884+0000 7f78b27fc700 1 -- 172.21.15.62:0/1365372708 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f78b4014070 con 0x7f78bc105800 2023-07-14T16:15:00.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.884+0000 7f78c139b700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f78a80a11e0 0x7f78a80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:00.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.885+0000 7f78c139b700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f78a80a11e0 0x7f78a80a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009fd0 tx=0x7f78b80095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:00.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:00.888+0000 7f78b27fc700 1 -- 172.21.15.62:0/1365372708 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f78b40964d0 con 0x7f78bc105800 2023-07-14T16:15:00.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:00 smithi062 ceph-mon[95543]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:00.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:00 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3812079559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:00.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:00 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2330706171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:00.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:00 smithi081 ceph-mon[127795]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:00.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:00 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3812079559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:00.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:00 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2330706171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:00 smithi164 ceph-mon[97427]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:00 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3812079559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:01.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:00 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2330706171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:01.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.093+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78bc02ca70 con 0x7f78bc105800 2023-07-14T16:15:01.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.094+0000 7f78b27fc700 1 -- 172.21.15.62:0/1365372708 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78b4099af0 con 0x7f78bc105800 2023-07-14T16:15:01.072 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:15:01.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f78a80a11e0 msgr2=0x7f78a80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f78a80a11e0 0x7f78a80a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009fd0 tx=0x7f78b80095a0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc105800 msgr2=0x7f78bc19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc105800 0x7f78bc19ce50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78b400e940 tx=0x7f78b400ed00 comp rx=0 tx=0).stop 2023-07-14T16:15:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 shutdown_connections 2023-07-14T16:15:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78bc105800 0x7f78bc19ce50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78bc1005c0 0x7f78bc1958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f78a80a11e0 0x7f78a80a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 --2- 172.21.15.62:0/1365372708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78bc102ee0 0x7f78bc195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 >> 172.21.15.62:0/1365372708 conn(0x7f78bc0fa000 msgr2=0x7f78bc101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 shutdown_connections 2023-07-14T16:15:01.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.096+0000 7f78c35ff700 1 -- 172.21.15.62:0/1365372708 wait complete. 2023-07-14T16:15:01.093 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:15:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.318+0000 7fabb51ff700 1 -- 172.21.15.62:0/3663597389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0101680 msgr2=0x7fabb0107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.318+0000 7fabb51ff700 1 --2- 172.21.15.62:0/3663597389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0101680 0x7fabb0107f40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faba800b0a0 tx=0x7faba800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.318+0000 7fabb51ff700 1 -- 172.21.15.62:0/3663597389 shutdown_connections 2023-07-14T16:15:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.318+0000 7fabb51ff700 1 --2- 172.21.15.62:0/3663597389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0101680 0x7fabb0107f40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.318+0000 7fabb51ff700 1 --2- 172.21.15.62:0/3663597389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb00ffb00 0x7fabb00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.318+0000 7fabb51ff700 1 --2- 172.21.15.62:0/3663597389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabb0100cf0 0x7fabb0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.318+0000 7fabb51ff700 1 -- 172.21.15.62:0/3663597389 >> 172.21.15.62:0/3663597389 conn(0x7fabb00fb0e0 msgr2=0x7fabb00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:01.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.319+0000 7fabb51ff700 1 -- 172.21.15.62:0/3663597389 shutdown_connections 2023-07-14T16:15:01.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.319+0000 7fabb51ff700 1 -- 172.21.15.62:0/3663597389 wait complete. 2023-07-14T16:15:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabb51ff700 1 Processor -- start 2023-07-14T16:15:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabb51ff700 1 -- start start 2023-07-14T16:15:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabb51ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabb00ffb00 0x7fabb01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:01.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabb51ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0100cf0 0x7fabb0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabb51ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb0101680 0x7fabb019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabb51ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb019f540 con 0x7fabb0100cf0 2023-07-14T16:15:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabb51ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb019f680 con 0x7fabb0101680 2023-07-14T16:15:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabb51ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb019f810 con 0x7fabb00ffb00 2023-07-14T16:15:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0100cf0 0x7fabb0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0100cf0 0x7fabb0197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59628/0 (socket says 172.21.15.62:59628) 2023-07-14T16:15:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.320+0000 7fabae59c700 1 -- 172.21.15.62:0/426480830 learned_addr learned my addr 172.21.15.62:0/426480830 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabaed9d700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabb00ffb00 0x7fabb01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabaf59e700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb0101680 0x7fabb019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabae59c700 1 -- 172.21.15.62:0/426480830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabb00ffb00 msgr2=0x7fabb01979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabae59c700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabb00ffb00 0x7fabb01979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabae59c700 1 -- 172.21.15.62:0/426480830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb0101680 msgr2=0x7fabb019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabae59c700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb0101680 0x7fabb019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabae59c700 1 -- 172.21.15.62:0/426480830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8009d00 con 0x7fabb0100cf0 2023-07-14T16:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabae59c700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0100cf0 0x7fabb0197f00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faba400c9b0 tx=0x7faba400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fab97fff700 1 -- 172.21.15.62:0/426480830 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4007b00 con 0x7fabb0100cf0 2023-07-14T16:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fab97fff700 1 -- 172.21.15.62:0/426480830 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faba4007c60 con 0x7fabb0100cf0 2023-07-14T16:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb019faf0 con 0x7fabb0100cf0 2023-07-14T16:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fab97fff700 1 -- 172.21.15.62:0/426480830 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4018660 con 0x7fabb0100cf0 2023-07-14T16:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.321+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb01a0000 con 0x7fabb0100cf0 2023-07-14T16:15:01.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.322+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb010b1e0 con 0x7fabb0100cf0 2023-07-14T16:15:01.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.324+0000 7fab97fff700 1 -- 172.21.15.62:0/426480830 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faba400f450 con 0x7fabb0100cf0 2023-07-14T16:15:01.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.326+0000 7fab97fff700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab980a1220 0x7fab980a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:01.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.326+0000 7fab97fff700 1 -- 172.21.15.62:0/426480830 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faba40cccd0 con 0x7fabb0100cf0 2023-07-14T16:15:01.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.326+0000 7fabaed9d700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab980a1220 0x7fab980a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.327+0000 7fabaed9d700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab980a1220 0x7fab980a36d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fabb0100b50 tx=0x7faba0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.328+0000 7fab97fff700 1 -- 172.21.15.62:0/426480830 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faba4096360 con 0x7fabb0100cf0 2023-07-14T16:15:01.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.583+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabb010b3d0 con 0x7fabb0100cf0 2023-07-14T16:15:01.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.584+0000 7fab97fff700 1 -- 172.21.15.62:0/426480830 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7faba400fe60 con 0x7fabb0100cf0 2023-07-14T16:15:01.562 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:01.562 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:15:01.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab980a1220 msgr2=0x7fab980a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab980a1220 0x7fab980a36d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fabb0100b50 tx=0x7faba0009260 comp rx=0 tx=0).stop 2023-07-14T16:15:01.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0100cf0 msgr2=0x7fabb0197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0100cf0 0x7fabb0197f00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faba400c9b0 tx=0x7faba400ccc0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 shutdown_connections 2023-07-14T16:15:01.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab980a1220 0x7fab980a36d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabb0101680 0x7fabb019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabb00ffb00 0x7fabb01979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.586+0000 7fabb51ff700 1 --2- 172.21.15.62:0/426480830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb0100cf0 0x7fabb0197f00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.587+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 >> 172.21.15.62:0/426480830 conn(0x7fabb00fb0e0 msgr2=0x7fabb0106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:01.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.587+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 shutdown_connections 2023-07-14T16:15:01.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.587+0000 7fabb51ff700 1 -- 172.21.15.62:0/426480830 wait complete. 2023-07-14T16:15:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:15:01.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:01.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 -- 172.21.15.62:0/2737248053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c101680 msgr2=0x7fa26c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 --2- 172.21.15.62:0/2737248053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c101680 0x7fa26c107f40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa26400b0a0 tx=0x7fa26400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 -- 172.21.15.62:0/2737248053 shutdown_connections 2023-07-14T16:15:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 --2- 172.21.15.62:0/2737248053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c101680 0x7fa26c107f40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 --2- 172.21.15.62:0/2737248053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26c0ffb00 0x7fa26c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 --2- 172.21.15.62:0/2737248053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa26c100cf0 0x7fa26c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 -- 172.21.15.62:0/2737248053 >> 172.21.15.62:0/2737248053 conn(0x7fa26c0fb0e0 msgr2=0x7fa26c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:01.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 -- 172.21.15.62:0/2737248053 shutdown_connections 2023-07-14T16:15:01.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.810+0000 7fa2721ff700 1 -- 172.21.15.62:0/2737248053 wait complete. 2023-07-14T16:15:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.811+0000 7fa2721ff700 1 Processor -- start 2023-07-14T16:15:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.811+0000 7fa2721ff700 1 -- start start 2023-07-14T16:15:01.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa2721ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c0ffb00 0x7fa26c195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa2721ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26c100cf0 0x7fa26c195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa2721ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa26c101680 0x7fa26c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa2721ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c19d3b0 con 0x7fa26c0ffb00 2023-07-14T16:15:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa2721ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c19d4f0 con 0x7fa26c100cf0 2023-07-14T16:15:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa2721ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c19d680 con 0x7fa26c101680 2023-07-14T16:15:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c0ffb00 0x7fa26c195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c0ffb00 0x7fa26c195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59648/0 (socket says 172.21.15.62:59648) 2023-07-14T16:15:01.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 -- 172.21.15.62:0/1509968444 learned_addr learned my addr 172.21.15.62:0/1509968444 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26affd700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26c100cf0 0x7fa26c195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 -- 172.21.15.62:0/1509968444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa26c101680 msgr2=0x7fa26c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa26c101680 0x7fa26c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 -- 172.21.15.62:0/1509968444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26c100cf0 msgr2=0x7fa26c195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26c100cf0 0x7fa26c195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 -- 172.21.15.62:0/1509968444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa264009d00 con 0x7fa26c0ffb00 2023-07-14T16:15:01.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.812+0000 7fa26b7fe700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c0ffb00 0x7fa26c195830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00c990 tx=0x7fa25c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:01.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.813+0000 7fa268ff9700 1 -- 172.21.15.62:0/1509968444 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c007b00 con 0x7fa26c0ffb00 2023-07-14T16:15:01.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.813+0000 7fa268ff9700 1 -- 172.21.15.62:0/1509968444 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa25c007c60 con 0x7fa26c0ffb00 2023-07-14T16:15:01.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.813+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c19d960 con 0x7fa26c0ffb00 2023-07-14T16:15:01.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.813+0000 7fa268ff9700 1 -- 172.21.15.62:0/1509968444 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c018660 con 0x7fa26c0ffb00 2023-07-14T16:15:01.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.813+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c19dec0 con 0x7fa26c0ffb00 2023-07-14T16:15:01.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.814+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c107000 con 0x7fa26c0ffb00 2023-07-14T16:15:01.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.816+0000 7fa268ff9700 1 -- 172.21.15.62:0/1509968444 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa25c00f450 con 0x7fa26c0ffb00 2023-07-14T16:15:01.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.817+0000 7fa268ff9700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa2540a12a0 0x7fa2540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:01.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.817+0000 7fa268ff9700 1 -- 172.21.15.62:0/1509968444 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa25c0cca80 con 0x7fa26c0ffb00 2023-07-14T16:15:01.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.817+0000 7fa26affd700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa2540a12a0 0x7fa2540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:01.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.818+0000 7fa26affd700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa2540a12a0 0x7fa2540a3750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa260005950 tx=0x7fa2600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:01.821+0000 7fa268ff9700 1 -- 172.21.15.62:0/1509968444 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa25c096020 con 0x7fa26c0ffb00 2023-07-14T16:15:01.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:01 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3388190400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:01.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:01 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1365372708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:01.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:01 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/426480830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:01.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:01 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3388190400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:01.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:01 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1365372708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:01.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:01 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/426480830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:02.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.031+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa26c02c850 con 0x7fa26c0ffb00 2023-07-14T16:15:02.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.033+0000 7fa268ff9700 1 -- 172.21.15.62:0/1509968444 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7fa25c099640 con 0x7fa26c0ffb00 2023-07-14T16:15:02.012 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:02.013 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:02.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.035+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa2540a12a0 msgr2=0x7fa2540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.035+0000 7fa2721ff700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa2540a12a0 0x7fa2540a3750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa260005950 tx=0x7fa2600058e0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c0ffb00 msgr2=0x7fa26c195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c0ffb00 0x7fa26c195830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00c990 tx=0x7fa25c00cd50 comp rx=0 tx=0).stop 2023-07-14T16:15:02.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 shutdown_connections 2023-07-14T16:15:02.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa26c101680 0x7fa26c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26c0ffb00 0x7fa26c195830 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa2540a12a0 0x7fa2540a3750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 --2- 172.21.15.62:0/1509968444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa26c100cf0 0x7fa26c195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 >> 172.21.15.62:0/1509968444 conn(0x7fa26c0fb0e0 msgr2=0x7fa26c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:02.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 shutdown_connections 2023-07-14T16:15:02.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.036+0000 7fa2721ff700 1 -- 172.21.15.62:0/1509968444 wait complete. 2023-07-14T16:15:02.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:01 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3388190400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:02.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:01 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1365372708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:02.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:01 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/426480830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:02.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:15:02.035 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:15:02.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 -- 172.21.15.62:0/2232443342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec101670 msgr2=0x7fa8ec107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/2232443342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec101670 0x7fa8ec107f30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009f90 tx=0x7fa8e400b330 comp rx=0 tx=0).stop 2023-07-14T16:15:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 -- 172.21.15.62:0/2232443342 shutdown_connections 2023-07-14T16:15:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/2232443342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec101670 0x7fa8ec107f30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/2232443342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8ec0ffaf0 0x7fa8ec0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/2232443342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8ec100ce0 0x7fa8ec101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 -- 172.21.15.62:0/2232443342 >> 172.21.15.62:0/2232443342 conn(0x7fa8ec0fb130 msgr2=0x7fa8ec0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:02.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 -- 172.21.15.62:0/2232443342 shutdown_connections 2023-07-14T16:15:02.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.264+0000 7fa8f3dff700 1 -- 172.21.15.62:0/2232443342 wait complete. 2023-07-14T16:15:02.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.265+0000 7fa8f3dff700 1 Processor -- start 2023-07-14T16:15:02.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.265+0000 7fa8f3dff700 1 -- start start 2023-07-14T16:15:02.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.265+0000 7fa8f3dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec0ffaf0 0x7fa8ec197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:02.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.265+0000 7fa8f3dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8ec100ce0 0x7fa8ec198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:02.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.265+0000 7fa8f3dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8ec101670 0x7fa8ec19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:02.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.265+0000 7fa8f3dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec19f6b0 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.265+0000 7fa8f3dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec19f7f0 con 0x7fa8ec101670 2023-07-14T16:15:02.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.265+0000 7fa8f3dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec19f980 con 0x7fa8ec100ce0 2023-07-14T16:15:02.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec0ffaf0 0x7fa8ec197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:02.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec0ffaf0 0x7fa8ec197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59668/0 (socket says 172.21.15.62:59668) 2023-07-14T16:15:02.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 -- 172.21.15.62:0/816015703 learned_addr learned my addr 172.21.15.62:0/816015703 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:02.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f139a700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8ec100ce0 0x7fa8ec198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:02.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f239c700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8ec101670 0x7fa8ec19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:02.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 -- 172.21.15.62:0/816015703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8ec100ce0 msgr2=0x7fa8ec198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8ec100ce0 0x7fa8ec198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 -- 172.21.15.62:0/816015703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8ec101670 msgr2=0x7fa8ec19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8ec101670 0x7fa8ec19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 -- 172.21.15.62:0/816015703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e4009c40 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.266+0000 7fa8f1b9b700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec0ffaf0 0x7fa8ec197b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e9f0 tx=0x7fa8e800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:02.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.267+0000 7fa8e2ffd700 1 -- 172.21.15.62:0/816015703 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8004220 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.267+0000 7fa8e2ffd700 1 -- 172.21.15.62:0/816015703 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8e8004380 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.267+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ec19fc60 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.267+0000 7fa8e2ffd700 1 -- 172.21.15.62:0/816015703 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8003a80 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.267+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ec1a0170 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.268+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ec192090 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.270+0000 7fa8e2ffd700 1 -- 172.21.15.62:0/816015703 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa8e8003c20 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.271+0000 7fa8e2ffd700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a11e0 0x7fa8d80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:02.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.271+0000 7fa8e2ffd700 1 -- 172.21.15.62:0/816015703 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8e8014070 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.271+0000 7fa8f139a700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a11e0 0x7fa8d80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:02.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.272+0000 7fa8f139a700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a11e0 0x7fa8d80a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc005950 tx=0x7fa8dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:02.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.275+0000 7fa8e2ffd700 1 -- 172.21.15.62:0/816015703 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa8e80961b0 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.481+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8ec02ca90 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.482+0000 7fa8e2ffd700 1 -- 172.21.15.62:0/816015703 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa8e80997d0 con 0x7fa8ec0ffaf0 2023-07-14T16:15:02.461 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:15:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.484+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a11e0 msgr2=0x7fa8d80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.484+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a11e0 0x7fa8d80a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc005950 tx=0x7fa8dc00b410 comp rx=0 tx=0).stop 2023-07-14T16:15:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec0ffaf0 msgr2=0x7fa8ec197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec0ffaf0 0x7fa8ec197b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e9f0 tx=0x7fa8e800ed00 comp rx=0 tx=0).stop 2023-07-14T16:15:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 shutdown_connections 2023-07-14T16:15:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a11e0 0x7fa8d80a3690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8ec101670 0x7fa8ec19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8ec0ffaf0 0x7fa8ec197b30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 --2- 172.21.15.62:0/816015703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8ec100ce0 0x7fa8ec198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 >> 172.21.15.62:0/816015703 conn(0x7fa8ec0fb130 msgr2=0x7fa8ec106640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:02.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 shutdown_connections 2023-07-14T16:15:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.485+0000 7fa8f3dff700 1 -- 172.21.15.62:0/816015703 wait complete. 2023-07-14T16:15:02.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:15:02.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.715+0000 7f060e1ff700 1 -- 172.21.15.62:0/795731832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 msgr2=0x7f0608102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.715+0000 7f060e1ff700 1 --2- 172.21.15.62:0/795731832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 0x7f0608102940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009990 tx=0x7f05f8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.716+0000 7f060e1ff700 1 -- 172.21.15.62:0/795731832 shutdown_connections 2023-07-14T16:15:02.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.716+0000 7f060e1ff700 1 --2- 172.21.15.62:0/795731832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06081057a0 0x7f0608107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.716+0000 7f060e1ff700 1 --2- 172.21.15.62:0/795731832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 0x7f0608102940 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.716+0000 7f060e1ff700 1 --2- 172.21.15.62:0/795731832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102e80 0x7f0608105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.716+0000 7f060e1ff700 1 -- 172.21.15.62:0/795731832 >> 172.21.15.62:0/795731832 conn(0x7f06080fa000 msgr2=0x7f06080fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:02.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.716+0000 7f060e1ff700 1 -- 172.21.15.62:0/795731832 shutdown_connections 2023-07-14T16:15:02.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.716+0000 7f060e1ff700 1 -- 172.21.15.62:0/795731832 wait complete. 2023-07-14T16:15:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.717+0000 7f060e1ff700 1 Processor -- start 2023-07-14T16:15:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.717+0000 7f060e1ff700 1 -- start start 2023-07-14T16:15:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.717+0000 7f060e1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 0x7f06080ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.717+0000 7f060e1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102e80 0x7f06080fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.717+0000 7f060e1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06081057a0 0x7f06080fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.717+0000 7f060e1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06080ff1f0 con 0x7f0608100560 2023-07-14T16:15:02.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.717+0000 7f060e1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06080ff360 con 0x7f06081057a0 2023-07-14T16:15:02.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.717+0000 7f060e1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819f820 con 0x7f0608102e80 2023-07-14T16:15:02.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 0x7f06080ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f0606ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102e80 0x7f06080fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 0x7f06080ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59684/0 (socket says 172.21.15.62:59684) 2023-07-14T16:15:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 -- 172.21.15.62:0/852170980 learned_addr learned my addr 172.21.15.62:0/852170980 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 -- 172.21.15.62:0/852170980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102e80 msgr2=0x7f06080fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102e80 0x7f06080fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 -- 172.21.15.62:0/852170980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06081057a0 msgr2=0x7f06080fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06081057a0 0x7f06080fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 -- 172.21.15.62:0/852170980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f80095f0 con 0x7f0608100560 2023-07-14T16:15:02.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f0606ffd700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102e80 0x7f06080fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:02.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.718+0000 7f06077fe700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 0x7f06080ffed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f05f80098e0 tx=0x7f05f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:02.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.719+0000 7f0604ff9700 1 -- 172.21.15.62:0/852170980 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f801d070 con 0x7f0608100560 2023-07-14T16:15:02.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.719+0000 7f0604ff9700 1 -- 172.21.15.62:0/852170980 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05f800bdb0 con 0x7f0608100560 2023-07-14T16:15:02.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.719+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060819faa0 con 0x7f0608100560 2023-07-14T16:15:02.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.719+0000 7f0604ff9700 1 -- 172.21.15.62:0/852170980 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8022d90 con 0x7f0608100560 2023-07-14T16:15:02.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.719+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060819fec0 con 0x7f0608100560 2023-07-14T16:15:02.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.721+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06080fbb70 con 0x7f0608100560 2023-07-14T16:15:02.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.723+0000 7f0604ff9700 1 -- 172.21.15.62:0/852170980 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f05f8022470 con 0x7f0608100560 2023-07-14T16:15:02.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.726+0000 7f0604ff9700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05f00a11e0 0x7f05f00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:02.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.726+0000 7f0604ff9700 1 -- 172.21.15.62:0/852170980 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05f80cdf80 con 0x7f0608100560 2023-07-14T16:15:02.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.726+0000 7f0606ffd700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05f00a11e0 0x7f05f00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:02.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.727+0000 7f0606ffd700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05f00a11e0 0x7f05f00a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f05fc011490 tx=0x7f05fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:02.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.730+0000 7f0604ff9700 1 -- 172.21.15.62:0/852170980 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f05f80976c0 con 0x7f0608100560 2023-07-14T16:15:02.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:02 smithi062 ceph-mon[95543]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:02.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:02 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1509968444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:02.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:02 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/816015703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:02 smithi081 ceph-mon[127795]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:02 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1509968444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:02 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/816015703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:02.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.945+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f060802ca90 con 0x7f0608100560 2023-07-14T16:15:02.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.946+0000 7f0604ff9700 1 -- 172.21.15.62:0/852170980 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05f809ace0 con 0x7f0608100560 2023-07-14T16:15:02.925 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:15:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05f00a11e0 msgr2=0x7f05f00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05f00a11e0 0x7f05f00a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f05fc011490 tx=0x7f05fc009260 comp rx=0 tx=0).stop 2023-07-14T16:15:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 msgr2=0x7f06080ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 0x7f06080ffed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f05f80098e0 tx=0x7f05f8004730 comp rx=0 tx=0).stop 2023-07-14T16:15:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 shutdown_connections 2023-07-14T16:15:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05f00a11e0 0x7f05f00a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06081057a0 0x7f06080fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0608100560 0x7f06080ffed0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 --2- 172.21.15.62:0/852170980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0608102e80 0x7f06080fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.948+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 >> 172.21.15.62:0/852170980 conn(0x7f06080fa000 msgr2=0x7f06081011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.949+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 shutdown_connections 2023-07-14T16:15:02.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:02.949+0000 7f060e1ff700 1 -- 172.21.15.62:0/852170980 wait complete. 2023-07-14T16:15:02.949 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:15:03.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:02 smithi164 ceph-mon[97427]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:03.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:02 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1509968444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:03.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:02 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/816015703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:03.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 -- 172.21.15.62:0/2921061265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa8107390 msgr2=0x7fdaa81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 --2- 172.21.15.62:0/2921061265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa8107390 0x7fdaa81077e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda98009990 tx=0x7fda98009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 -- 172.21.15.62:0/2921061265 shutdown_connections 2023-07-14T16:15:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 --2- 172.21.15.62:0/2921061265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaa8107d20 0x7fdaa810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 --2- 172.21.15.62:0/2921061265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa81061a0 0x7fdaa81065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 --2- 172.21.15.62:0/2921061265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa8107390 0x7fdaa81077e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 -- 172.21.15.62:0/2921061265 >> 172.21.15.62:0/2921061265 conn(0x7fdaa80751c0 msgr2=0x7fdaa8077610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:03.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 -- 172.21.15.62:0/2921061265 shutdown_connections 2023-07-14T16:15:03.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.187+0000 7fdaacfff700 1 -- 172.21.15.62:0/2921061265 wait complete. 2023-07-14T16:15:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.188+0000 7fdaacfff700 1 Processor -- start 2023-07-14T16:15:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaacfff700 1 -- start start 2023-07-14T16:15:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaacfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa81061a0 0x7fdaa819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaacfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa8107390 0x7fdaa819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaacfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaa8107d20 0x7fdaa81a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaacfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81a38f0 con 0x7fdaa81061a0 2023-07-14T16:15:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaacfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81a3a30 con 0x7fdaa8107d20 2023-07-14T16:15:03.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaacfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81a3bc0 con 0x7fdaa8107390 2023-07-14T16:15:03.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaa659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa81061a0 0x7fdaa819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:03.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaa5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa8107390 0x7fdaa819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:03.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaa659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa81061a0 0x7fdaa819bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60148/0 (socket says 172.21.15.62:60148) 2023-07-14T16:15:03.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.189+0000 7fdaa659c700 1 -- 172.21.15.62:0/4252227634 learned_addr learned my addr 172.21.15.62:0/4252227634 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:03.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaa6d9d700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaa8107d20 0x7fdaa81a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:03.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaa659c700 1 -- 172.21.15.62:0/4252227634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa8107390 msgr2=0x7fdaa819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaa659c700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa8107390 0x7fdaa819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaa659c700 1 -- 172.21.15.62:0/4252227634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaa8107d20 msgr2=0x7fdaa81a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaa659c700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaa8107d20 0x7fdaa81a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaa659c700 1 -- 172.21.15.62:0/4252227634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda980095f0 con 0x7fdaa81061a0 2023-07-14T16:15:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaa6d9d700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaa8107d20 0x7fdaa81a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaa659c700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa81061a0 0x7fdaa819bd70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0007ee0 tx=0x7fdaa000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:03.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fda977fe700 1 -- 172.21.15.62:0/4252227634 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000ddd0 con 0x7fdaa81061a0 2023-07-14T16:15:03.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa81a3ea0 con 0x7fdaa81061a0 2023-07-14T16:15:03.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fda977fe700 1 -- 172.21.15.62:0/4252227634 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdaa000f040 con 0x7fdaa81061a0 2023-07-14T16:15:03.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fda977fe700 1 -- 172.21.15.62:0/4252227634 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0014600 con 0x7fdaa81061a0 2023-07-14T16:15:03.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.190+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa81a42d0 con 0x7fdaa81061a0 2023-07-14T16:15:03.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.191+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa8111870 con 0x7fdaa81061a0 2023-07-14T16:15:03.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.193+0000 7fda977fe700 1 -- 172.21.15.62:0/4252227634 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdaa0015990 con 0x7fdaa81061a0 2023-07-14T16:15:03.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.195+0000 7fda977fe700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda900a11e0 0x7fda900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.195+0000 7fda977fe700 1 -- 172.21.15.62:0/4252227634 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdaa00cc820 con 0x7fdaa81061a0 2023-07-14T16:15:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.195+0000 7fdaa5d9b700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda900a11e0 0x7fda900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.196+0000 7fdaa5d9b700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda900a11e0 0x7fda900a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda98006010 tx=0x7fda9800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.197+0000 7fda977fe700 1 -- 172.21.15.62:0/4252227634 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdaa0095e20 con 0x7fdaa81061a0 2023-07-14T16:15:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.412+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdaa810d140 con 0x7fdaa81061a0 2023-07-14T16:15:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.413+0000 7fda977fe700 1 -- 172.21.15.62:0/4252227634 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdaa00240a0 con 0x7fdaa81061a0 2023-07-14T16:15:03.391 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:15:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda900a11e0 msgr2=0x7fda900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda900a11e0 0x7fda900a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda98006010 tx=0x7fda9800b260 comp rx=0 tx=0).stop 2023-07-14T16:15:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa81061a0 msgr2=0x7fdaa819bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa81061a0 0x7fdaa819bd70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0007ee0 tx=0x7fdaa000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 shutdown_connections 2023-07-14T16:15:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda900a11e0 0x7fda900a3690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaa8107d20 0x7fdaa81a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa81061a0 0x7fdaa819bd70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 --2- 172.21.15.62:0/4252227634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaa8107390 0x7fdaa819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 >> 172.21.15.62:0/4252227634 conn(0x7fdaa80751c0 msgr2=0x7fdaa80774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 shutdown_connections 2023-07-14T16:15:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.415+0000 7fdaacfff700 1 -- 172.21.15.62:0/4252227634 wait complete. 2023-07-14T16:15:03.413 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:15:03.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.652+0000 7f12a53ff700 1 -- 172.21.15.62:0/2653657370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a01005f0 msgr2=0x7f12a01029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.652+0000 7f12a53ff700 1 --2- 172.21.15.62:0/2653657370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a01005f0 0x7f12a01029d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1290009990 tx=0x7f1290009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.652+0000 7f12a53ff700 1 -- 172.21.15.62:0/2653657370 shutdown_connections 2023-07-14T16:15:03.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.652+0000 7f12a53ff700 1 --2- 172.21.15.62:0/2653657370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a0105830 0x7f12a0107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.652+0000 7f12a53ff700 1 --2- 172.21.15.62:0/2653657370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a01005f0 0x7f12a01029d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.652+0000 7f12a53ff700 1 --2- 172.21.15.62:0/2653657370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0102f10 0x7f12a01052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.652+0000 7f12a53ff700 1 -- 172.21.15.62:0/2653657370 >> 172.21.15.62:0/2653657370 conn(0x7f12a00f9fd0 msgr2=0x7f12a00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:03.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.653+0000 7f12a53ff700 1 -- 172.21.15.62:0/2653657370 shutdown_connections 2023-07-14T16:15:03.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.653+0000 7f12a53ff700 1 -- 172.21.15.62:0/2653657370 wait complete. 2023-07-14T16:15:03.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f12a53ff700 1 Processor -- start 2023-07-14T16:15:03.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f12a53ff700 1 -- start start 2023-07-14T16:15:03.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f12a53ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a01005f0 0x7f12a0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:03.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f12a53ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0102f10 0x7f12a0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:03.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f12a53ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0105830 0x7f12a019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:03.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f12a53ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a019f4f0 con 0x7f12a0102f10 2023-07-14T16:15:03.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f12a53ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a019f630 con 0x7f12a0105830 2023-07-14T16:15:03.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f12a53ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a019f7c0 con 0x7f12a01005f0 2023-07-14T16:15:03.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f129effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a01005f0 0x7f12a0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:03.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f129e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0102f10 0x7f12a0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:03.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f129effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a01005f0 0x7f12a0197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35582/0 (socket says 172.21.15.62:35582) 2023-07-14T16:15:03.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f129e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0102f10 0x7f12a0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60170/0 (socket says 172.21.15.62:60170) 2023-07-14T16:15:03.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f129f7fe700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0105830 0x7f12a019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:03.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.654+0000 7f129effd700 1 -- 172.21.15.62:0/1117641912 learned_addr learned my addr 172.21.15.62:0/1117641912 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:03.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f129effd700 1 -- 172.21.15.62:0/1117641912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0105830 msgr2=0x7f12a019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f129effd700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0105830 0x7f12a019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f129effd700 1 -- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0102f10 msgr2=0x7f12a0197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f129effd700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0102f10 0x7f12a0197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f129effd700 1 -- 172.21.15.62:0/1117641912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12900095f0 con 0x7f12a01005f0 2023-07-14T16:15:03.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f129f7fe700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0105830 0x7f12a019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:03.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f129effd700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a01005f0 0x7f12a0197970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12900098e0 tx=0x7f1290004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:03.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f1287fff700 1 -- 172.21.15.62:0/1117641912 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129001d070 con 0x7f12a01005f0 2023-07-14T16:15:03.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f1287fff700 1 -- 172.21.15.62:0/1117641912 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f129000bdb0 con 0x7f12a01005f0 2023-07-14T16:15:03.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a019fa40 con 0x7f12a01005f0 2023-07-14T16:15:03.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.655+0000 7f1287fff700 1 -- 172.21.15.62:0/1117641912 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290022d20 con 0x7f12a01005f0 2023-07-14T16:15:03.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.656+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a019fec0 con 0x7f12a01005f0 2023-07-14T16:15:03.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.657+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a01073a0 con 0x7f12a01005f0 2023-07-14T16:15:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.659+0000 7f1287fff700 1 -- 172.21.15.62:0/1117641912 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f129000b920 con 0x7f12a01005f0 2023-07-14T16:15:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.660+0000 7f1287fff700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12880a12a0 0x7f12880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.660+0000 7f1287fff700 1 -- 172.21.15.62:0/1117641912 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12900ce030 con 0x7f12a01005f0 2023-07-14T16:15:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.661+0000 7f129e7fc700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12880a12a0 0x7f12880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:03.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.661+0000 7f129e7fc700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12880a12a0 0x7f12880a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1294009c60 tx=0x7f1294009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.664+0000 7f1287fff700 1 -- 172.21.15.62:0/1117641912 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f12900976c0 con 0x7f12a01005f0 2023-07-14T16:15:03.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:15:03.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:03 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/852170980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:03.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:03 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/4252227634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:03.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:15:03.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:03 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/852170980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:03.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:03 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/4252227634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:03.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.931+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12a002c850 con 0x7f12a01005f0 2023-07-14T16:15:03.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.932+0000 7f1287fff700 1 -- 172.21.15.62:0/1117641912 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f129009ace0 con 0x7f12a01005f0 2023-07-14T16:15:03.911 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:03.911 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:15:03.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12880a12a0 msgr2=0x7f12880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12880a12a0 0x7f12880a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1294009c60 tx=0x7f1294009260 comp rx=0 tx=0).stop 2023-07-14T16:15:03.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a01005f0 msgr2=0x7f12a0197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:03.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a01005f0 0x7f12a0197970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12900098e0 tx=0x7f1290004730 comp rx=0 tx=0).stop 2023-07-14T16:15:03.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 shutdown_connections 2023-07-14T16:15:03.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12880a12a0 0x7f12880a3750 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0105830 0x7f12a019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12a01005f0 0x7f12a0197970 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 --2- 172.21.15.62:0/1117641912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0102f10 0x7f12a0197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:03.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 >> 172.21.15.62:0/1117641912 conn(0x7f12a00f9fd0 msgr2=0x7f12a00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:03.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 shutdown_connections 2023-07-14T16:15:03.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:03.935+0000 7f12a53ff700 1 -- 172.21.15.62:0/1117641912 wait complete. 2023-07-14T16:15:03.915 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:15:03.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:04.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:15:04.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:03 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/852170980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:04.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:03 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/4252227634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:04.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 -- 172.21.15.62:0/1202830875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c101680 msgr2=0x7f205c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 --2- 172.21.15.62:0/1202830875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c101680 0x7f205c107f40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f205400b0a0 tx=0x7f205400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 -- 172.21.15.62:0/1202830875 shutdown_connections 2023-07-14T16:15:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 --2- 172.21.15.62:0/1202830875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c101680 0x7f205c107f40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 --2- 172.21.15.62:0/1202830875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c0ffb00 0x7f205c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 --2- 172.21.15.62:0/1202830875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f205c100cf0 0x7f205c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 -- 172.21.15.62:0/1202830875 >> 172.21.15.62:0/1202830875 conn(0x7f205c0fb0e0 msgr2=0x7f205c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:04.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 -- 172.21.15.62:0/1202830875 shutdown_connections 2023-07-14T16:15:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.165+0000 7f20625ff700 1 -- 172.21.15.62:0/1202830875 wait complete. 2023-07-14T16:15:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.167+0000 7f20625ff700 1 Processor -- start 2023-07-14T16:15:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.167+0000 7f20625ff700 1 -- start start 2023-07-14T16:15:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f20625ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f205c0ffb00 0x7f205c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f20625ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c100cf0 0x7f205c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f20625ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c101680 0x7f205c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:04.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f20625ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0735e0 con 0x7f205c100cf0 2023-07-14T16:15:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f20625ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c073750 con 0x7f205c0ffb00 2023-07-14T16:15:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f20625ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c19f890 con 0x7f205c101680 2023-07-14T16:15:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c100cf0 0x7f205c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f205c0ffb00 0x7f205c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f2060b9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c101680 0x7f205c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c100cf0 0x7f205c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60188/0 (socket says 172.21.15.62:60188) 2023-07-14T16:15:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f205c0ffb00 0x7f205c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41592/0 (socket says 172.21.15.62:41592) 2023-07-14T16:15:04.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205b7fe700 1 -- 172.21.15.62:0/1253470685 learned_addr learned my addr 172.21.15.62:0/1253470685 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205b7fe700 1 -- 172.21.15.62:0/1253470685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c101680 msgr2=0x7f205c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205b7fe700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c101680 0x7f205c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205b7fe700 1 -- 172.21.15.62:0/1253470685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f205c0ffb00 msgr2=0x7f205c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.168+0000 7f205b7fe700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f205c0ffb00 0x7f205c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.169+0000 7f205b7fe700 1 -- 172.21.15.62:0/1253470685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009d00 con 0x7f205c100cf0 2023-07-14T16:15:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.169+0000 7f205b7fe700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c100cf0 0x7f205c072900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f205000e9a0 tx=0x7f205000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:04.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.169+0000 7f20597fa700 1 -- 172.21.15.62:0/1253470685 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050009790 con 0x7f205c100cf0 2023-07-14T16:15:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.169+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c19fb70 con 0x7f205c100cf0 2023-07-14T16:15:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.169+0000 7f20597fa700 1 -- 172.21.15.62:0/1253470685 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2050010430 con 0x7f205c100cf0 2023-07-14T16:15:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.169+0000 7f20597fa700 1 -- 172.21.15.62:0/1253470685 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205000cea0 con 0x7f205c100cf0 2023-07-14T16:15:04.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.169+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c1a0050 con 0x7f205c100cf0 2023-07-14T16:15:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.170+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c106ff0 con 0x7f205c100cf0 2023-07-14T16:15:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.172+0000 7f20597fa700 1 -- 172.21.15.62:0/1253470685 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f20500105a0 con 0x7f205c100cf0 2023-07-14T16:15:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.174+0000 7f20597fa700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f20440a1220 0x7f20440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:04.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.174+0000 7f20597fa700 1 -- 172.21.15.62:0/1253470685 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2050014070 con 0x7f205c100cf0 2023-07-14T16:15:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.174+0000 7f205bfff700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f20440a1220 0x7f20440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:04.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.175+0000 7f205bfff700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f20440a1220 0x7f20440a36d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f205c100b50 tx=0x7f204c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.178+0000 7f20597fa700 1 -- 172.21.15.62:0/1253470685 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2050051790 con 0x7f205c100cf0 2023-07-14T16:15:04.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.396+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f205c02c850 con 0x7f205c100cf0 2023-07-14T16:15:04.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.398+0000 7f20597fa700 1 -- 172.21.15.62:0/1253470685 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f2050078e60 con 0x7f205c100cf0 2023-07-14T16:15:04.376 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:04.377 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:04.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f20440a1220 msgr2=0x7f20440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f20440a1220 0x7f20440a36d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f205c100b50 tx=0x7f204c009260 comp rx=0 tx=0).stop 2023-07-14T16:15:04.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c100cf0 msgr2=0x7f205c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c100cf0 0x7f205c072900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f205000e9a0 tx=0x7f205000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 shutdown_connections 2023-07-14T16:15:04.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f205c101680 0x7f205c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f20440a1220 0x7f20440a36d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f205c0ffb00 0x7f205c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 --2- 172.21.15.62:0/1253470685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f205c100cf0 0x7f205c072900 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.400+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 >> 172.21.15.62:0/1253470685 conn(0x7f205c0fb0e0 msgr2=0x7f205c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:04.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.401+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 shutdown_connections 2023-07-14T16:15:04.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.401+0000 7f20625ff700 1 -- 172.21.15.62:0/1253470685 wait complete. 2023-07-14T16:15:04.401 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:15:04.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 -- 172.21.15.62:0/2680525776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa990074550 msgr2=0x7fa9900729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2680525776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa990074550 0x7fa9900729a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa980009990 tx=0x7fa980009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 -- 172.21.15.62:0/2680525776 shutdown_connections 2023-07-14T16:15:04.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2680525776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa99010c0d0 0x7fa99010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2680525776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa990074550 0x7fa9900729a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2680525776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa990072f70 0x7fa9900733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 -- 172.21.15.62:0/2680525776 >> 172.21.15.62:0/2680525776 conn(0x7fa9900fb0e0 msgr2=0x7fa9900fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:04.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 -- 172.21.15.62:0/2680525776 shutdown_connections 2023-07-14T16:15:04.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.636+0000 7fa9949ff700 1 -- 172.21.15.62:0/2680525776 wait complete. 2023-07-14T16:15:04.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.637+0000 7fa9949ff700 1 Processor -- start 2023-07-14T16:15:04.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.637+0000 7fa9949ff700 1 -- start start 2023-07-14T16:15:04.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa9949ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa990072f70 0x7fa99019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:04.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa9949ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990074550 0x7fa99019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:04.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa9949ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99010c0d0 0x7fa9901a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:04.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa9949ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9901a38f0 con 0x7fa99010c0d0 2023-07-14T16:15:04.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa9949ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9901a3a30 con 0x7fa990072f70 2023-07-14T16:15:04.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa9949ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9901a3bc0 con 0x7fa990074550 2023-07-14T16:15:04.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99010c0d0 0x7fa9901a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:04.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99010c0d0 0x7fa9901a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60214/0 (socket says 172.21.15.62:60214) 2023-07-14T16:15:04.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98ed9d700 1 -- 172.21.15.62:0/2775604654 learned_addr learned my addr 172.21.15.62:0/2775604654 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:04.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98e59c700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa990072f70 0x7fa99019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:04.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98dd9b700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990074550 0x7fa99019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:04.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98ed9d700 1 -- 172.21.15.62:0/2775604654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990074550 msgr2=0x7fa99019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98ed9d700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990074550 0x7fa99019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98ed9d700 1 -- 172.21.15.62:0/2775604654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa990072f70 msgr2=0x7fa99019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98ed9d700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa990072f70 0x7fa99019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.638+0000 7fa98ed9d700 1 -- 172.21.15.62:0/2775604654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800095f0 con 0x7fa99010c0d0 2023-07-14T16:15:04.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.639+0000 7fa98ed9d700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99010c0d0 0x7fa9901a3320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa988007ee0 tx=0x7fa98800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:04.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.639+0000 7fa97f7fe700 1 -- 172.21.15.62:0/2775604654 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988017070 con 0x7fa99010c0d0 2023-07-14T16:15:04.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.639+0000 7fa97f7fe700 1 -- 172.21.15.62:0/2775604654 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa98800f040 con 0x7fa99010c0d0 2023-07-14T16:15:04.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.639+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9901a3ea0 con 0x7fa99010c0d0 2023-07-14T16:15:04.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.639+0000 7fa97f7fe700 1 -- 172.21.15.62:0/2775604654 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988005570 con 0x7fa99010c0d0 2023-07-14T16:15:04.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.639+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9901a4320 con 0x7fa99010c0d0 2023-07-14T16:15:04.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.640+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa990101c20 con 0x7fa99010c0d0 2023-07-14T16:15:04.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.642+0000 7fa97f7fe700 1 -- 172.21.15.62:0/2775604654 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa98800db20 con 0x7fa99010c0d0 2023-07-14T16:15:04.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.644+0000 7fa97f7fe700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9780a1220 0x7fa9780a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:04.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.644+0000 7fa97f7fe700 1 -- 172.21.15.62:0/2775604654 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa9880cc200 con 0x7fa99010c0d0 2023-07-14T16:15:04.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.644+0000 7fa98e59c700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9780a1220 0x7fa9780a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:04.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.645+0000 7fa98e59c700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9780a1220 0x7fa9780a36d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa98000b5c0 tx=0x7fa980005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:04.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.648+0000 7fa97f7fe700 1 -- 172.21.15.62:0/2775604654 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa9880957a0 con 0x7fa99010c0d0 2023-07-14T16:15:04.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:04 smithi062 ceph-mon[95543]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:04.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:04 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1117641912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:04.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:04 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1253470685' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:04 smithi081 ceph-mon[127795]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:04 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1117641912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:04 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1253470685' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:04.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.935+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa99002c850 con 0x7fa99010c0d0 2023-07-14T16:15:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.936+0000 7fa97f7fe700 1 -- 172.21.15.62:0/2775604654 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fa988098dc0 con 0x7fa99010c0d0 2023-07-14T16:15:04.914 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:04.915 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:15:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9780a1220 msgr2=0x7fa9780a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9780a1220 0x7fa9780a36d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa98000b5c0 tx=0x7fa980005fb0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99010c0d0 msgr2=0x7fa9901a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99010c0d0 0x7fa9901a3320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa988007ee0 tx=0x7fa98800d3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 shutdown_connections 2023-07-14T16:15:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99010c0d0 0x7fa9901a3320 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9780a1220 0x7fa9780a36d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa990072f70 0x7fa99019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 --2- 172.21.15.62:0/2775604654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa990074550 0x7fa99019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 >> 172.21.15.62:0/2775604654 conn(0x7fa9900fb0e0 msgr2=0x7fa9900fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 shutdown_connections 2023-07-14T16:15:04.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:04.939+0000 7fa9949ff700 1 -- 172.21.15.62:0/2775604654 wait complete. 2023-07-14T16:15:04.920 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:15:04.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T16:13:58.805235+0000', 'modified': '2023-07-14T16:14:04.590671+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24602, 'mds_1': 24565, 'mds_2': 24512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24475': {'gid': 24475, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/903698978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 903698978}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 903698978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.81:6835/337855571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 337855571}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 337855571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/1558258971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 1558258971}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 1558258971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24551': {'gid': 24551, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6837/3627777030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 3627777030}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 3627777030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.164:6839/794647512', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 794647512}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 794647512}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6839/2113600738', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 2113600738}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 2113600738}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T16:15:04.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-14T16:15:04.938 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-14T16:15:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:04 smithi164 ceph-mon[97427]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:05.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:04 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1117641912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:05.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:04 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1253470685' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 -- 172.21.15.62:0/292265797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 msgr2=0x7f041c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 --2- 172.21.15.62:0/292265797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 0x7f041c101120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0404009990 tx=0x7f0404009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 -- 172.21.15.62:0/292265797 shutdown_connections 2023-07-14T16:15:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 --2- 172.21.15.62:0/292265797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c101660 0x7f041c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 --2- 172.21.15.62:0/292265797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c0ffae0 0x7f041c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 --2- 172.21.15.62:0/292265797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 0x7f041c101120 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 -- 172.21.15.62:0/292265797 >> 172.21.15.62:0/292265797 conn(0x7f041c0fb0e0 msgr2=0x7f041c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 -- 172.21.15.62:0/292265797 shutdown_connections 2023-07-14T16:15:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.174+0000 7f04209ff700 1 -- 172.21.15.62:0/292265797 wait complete. 2023-07-14T16:15:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.175+0000 7f04209ff700 1 Processor -- start 2023-07-14T16:15:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.175+0000 7f04209ff700 1 -- start start 2023-07-14T16:15:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.175+0000 7f04209ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c0ffae0 0x7f041c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.175+0000 7f04209ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 0x7f041c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.175+0000 7f04209ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c101660 0x7f041c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.175+0000 7f04209ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c19f540 con 0x7f041c100cd0 2023-07-14T16:15:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.175+0000 7f04209ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c19f680 con 0x7f041c0ffae0 2023-07-14T16:15:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.175+0000 7f04209ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c19f810 con 0x7f041c101660 2023-07-14T16:15:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 0x7f041c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 0x7f041c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60232/0 (socket says 172.21.15.62:60232) 2023-07-14T16:15:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f041a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c0ffae0 0x7f041c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 -- 172.21.15.62:0/1950613683 learned_addr learned my addr 172.21.15.62:0/1950613683 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f041ad9d700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c101660 0x7f041c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 -- 172.21.15.62:0/1950613683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c101660 msgr2=0x7f041c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c101660 0x7f041c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 -- 172.21.15.62:0/1950613683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c0ffae0 msgr2=0x7f041c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c0ffae0 0x7f041c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 -- 172.21.15.62:0/1950613683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c009520 con 0x7f041c100cd0 2023-07-14T16:15:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f041a59c700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c0ffae0 0x7f041c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f0413fff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 0x7f041c197f00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0404005910 tx=0x7f0404005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f04137fe700 1 -- 172.21.15.62:0/1950613683 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040401d070 con 0x7f041c100cd0 2023-07-14T16:15:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.176+0000 7f04137fe700 1 -- 172.21.15.62:0/1950613683 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f040400be10 con 0x7f041c100cd0 2023-07-14T16:15:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.177+0000 7f04209ff700 1 -- 172.21.15.62:0/1950613683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04040095f0 con 0x7f041c100cd0 2023-07-14T16:15:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.177+0000 7f04137fe700 1 -- 172.21.15.62:0/1950613683 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404022e00 con 0x7f041c100cd0 2023-07-14T16:15:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.177+0000 7f04209ff700 1 -- 172.21.15.62:0/1950613683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c19fd80 con 0x7f041c100cd0 2023-07-14T16:15:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.180+0000 7f04137fe700 1 -- 172.21.15.62:0/1950613683 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f040400b980 con 0x7f041c100cd0 2023-07-14T16:15:05.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.182+0000 7f043440bb80 1 Processor -- start 2023-07-14T16:15:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.182+0000 7f043440bb80 1 -- start start 2023-07-14T16:15:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.182+0000 7f04137fe700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04080a1260 0x7f04080a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.182+0000 7f04137fe700 1 -- 172.21.15.62:0/1950613683 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f04040cdfa0 con 0x7f041c100cd0 2023-07-14T16:15:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.182+0000 7f043440bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fc480 0x55c9f30fc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.182+0000 7f043440bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fcdd0 0x55c9f30fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.182+0000 7f043440bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fdc20 0x55c9f30fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.182+0000 7f043440bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f3001c60 con 0x55c9f30fdc20 2023-07-14T16:15:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f043440bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f30549f0 con 0x55c9f30fc480 2023-07-14T16:15:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f043440bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f304de80 con 0x55c9f30fcdd0 2023-07-14T16:15:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f041ad9d700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04080a1260 0x7f04080a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f0413fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fcdd0 0x55c9f30fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f041a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fc480 0x55c9f30fc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f0413fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fcdd0 0x55c9f30fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35644/0 (socket says 172.21.15.62:35644) 2023-07-14T16:15:05.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f0413fff700 1 -- 172.21.15.62:0/255194795 learned_addr learned my addr 172.21.15.62:0/255194795 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f041ad9d700 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fdc20 0x55c9f30fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f0413fff700 1 -- 172.21.15.62:0/255194795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fc480 msgr2=0x55c9f30fc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f0413fff700 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fc480 0x55c9f30fc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f0413fff700 1 -- 172.21.15.62:0/255194795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fdc20 msgr2=0x55c9f30fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f0413fff700 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fdc20 0x55c9f30fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f0413fff700 1 -- 172.21.15.62:0/255194795 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9f2e7bd90 con 0x55c9f30fcdd0 2023-07-14T16:15:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f041ad9d700 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fdc20 0x55c9f30fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.183+0000 7f041ad9d700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04080a1260 0x7f04080a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f041400f870 tx=0x7f041400fc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.184+0000 7f0413fff700 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fcdd0 0x55c9f30fd690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04040058b0 tx=0x7f040409aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba0b0c7f6e99a761 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.184+0000 7f0410ff9700 1 -- 172.21.15.62:0/255194795 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404095ea0 con 0x55c9f30fcdd0 2023-07-14T16:15:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.184+0000 7f0410ff9700 1 -- 172.21.15.62:0/255194795 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04040975d0 con 0x55c9f30fcdd0 2023-07-14T16:15:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.184+0000 7f0410ff9700 1 -- 172.21.15.62:0/255194795 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404022570 con 0x55c9f30fcdd0 2023-07-14T16:15:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.184+0000 7f043440bb80 1 -- 172.21.15.62:0/255194795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fcdd0 msgr2=0x55c9f30fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.184+0000 7f043440bb80 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fcdd0 0x55c9f30fd690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04040058b0 tx=0x7f040409aea0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 -- 172.21.15.62:0/255194795 shutdown_connections 2023-07-14T16:15:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fdc20 0x55c9f30fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fc480 0x55c9f30fc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 --2- 172.21.15.62:0/255194795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fcdd0 0x55c9f30fd690 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 -- 172.21.15.62:0/255194795 >> 172.21.15.62:0/255194795 conn(0x55c9f30f1d50 msgr2=0x55c9f30f7180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 -- 172.21.15.62:0/255194795 shutdown_connections 2023-07-14T16:15:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 -- 172.21.15.62:0/255194795 wait complete. 2023-07-14T16:15:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 -- start start 2023-07-14T16:15:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.185+0000 7f043440bb80 1 Processor -- start 2023-07-14T16:15:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f043440bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f3105d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f043440bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f31062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f043440bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f310d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f043440bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f2feda70 con 0x55c9f30fc480 2023-07-14T16:15:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f043440bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f2fed770 con 0x55c9f30fcdd0 2023-07-14T16:15:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f043440bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f2ed18d0 con 0x55c9f30fdc20 2023-07-14T16:15:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f043440bb80 10 client.0 _ll_register_callbacks cb 0x55c9f2feb710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f041a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f3105d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f041a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f3105d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60246/0 (socket says 172.21.15.62:60246) 2023-07-14T16:15:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f041a59c700 1 -- 172.21.15.62:0/3693663177 learned_addr learned my addr 172.21.15.62:0/3693663177 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f041ad9d700 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f310d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.186+0000 7f0413fff700 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f31062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f041a59c700 1 -- 172.21.15.62:0/3693663177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 msgr2=0x55c9f310d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f041a59c700 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f310d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f041a59c700 1 -- 172.21.15.62:0/3693663177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 msgr2=0x55c9f31062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f041a59c700 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f31062a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f041a59c700 1 -- 172.21.15.62:0/3693663177 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c19fd80 con 0x55c9f30fc480 2023-07-14T16:15:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f041a59c700 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f3105d60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f040c012cf0 tx=0x7f040c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bef569aac83591 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f0401ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f0401ffb700 1 -- 172.21.15.62:0/3693663177 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c017070 con 0x55c9f30fc480 2023-07-14T16:15:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f0401ffb700 1 -- 172.21.15.62:0/3693663177 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f040c00f830 con 0x55c9f30fc480 2023-07-14T16:15:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f043440bb80 1 -- 172.21.15.62:0/3693663177 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9f2fa74a0 con 0x55c9f30fc480 2023-07-14T16:15:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f0401ffb700 1 -- 172.21.15.62:0/3693663177 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f040c0122e0 con 0x55c9f30fc480 2023-07-14T16:15:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f043440bb80 10 client.15132 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.187+0000 7f043440bb80 1 -- 172.21.15.62:0/3693663177 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c9f2e64c10 con 0x55c9f30fc480 2023-07-14T16:15:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.188+0000 7f0401ffb700 1 -- 172.21.15.62:0/3693663177 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c9f2fa74a0 con 0x55c9f30fc480 2023-07-14T16:15:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.188+0000 7f0401ffb700 1 -- 172.21.15.62:0/3693663177 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f040c010bb0 con 0x55c9f30fc480 2023-07-14T16:15:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.188+0000 7f043440bb80 10 client.15132 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.188+0000 7f043440bb80 10 client.15132 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-14T16:15:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.188+0000 7f043440bb80 20 client.15132 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.188+0000 7f043440bb80 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c9f3111040 0x55c9f31103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.188+0000 7f043440bb80 4 client.15132 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:15:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.188+0000 7f043440bb80 1 -- 172.21.15.62:0/3693663177 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9f3046e60 con 0x55c9f3111040 2023-07-14T16:15:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.189+0000 7f0413fff700 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c9f3111040 0x55c9f31103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.189+0000 7f0413fff700 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c9f3111040 0x55c9f31103f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=571b22f890bea547 server_cookie=dce7885555b49c4c in_seq=0 out_seq=0 2023-07-14T16:15:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.189+0000 7f0401ffb700 10 client.15132 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:15:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f0401ffb700 1 -- 172.21.15.62:0/3693663177 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c9f3046e60 con 0x55c9f3111040 2023-07-14T16:15:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f0401ffb700 10 client.15132 handle_command_reply: tid=0 2023-07-14T16:15:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 client.15132 shutdown 2023-07-14T16:15:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 -- 172.21.15.62:0/3693663177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 msgr2=0x55c9f3105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f3105d60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f040c012cf0 tx=0x7f040c009e70 comp rx=0 tx=0).stop 2023-07-14T16:15:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 -- 172.21.15.62:0/3693663177 shutdown_connections 2023-07-14T16:15:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c9f3111040 0x55c9f31103f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f310d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f3105d60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 --2- 172.21.15.62:0/3693663177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f31062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 -- 172.21.15.62:0/3693663177 >> 172.21.15.62:0/3693663177 conn(0x55c9f30f1d50 msgr2=0x55c9f30f6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f0401ffb700 0 client.15132 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:15:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 -- 172.21.15.62:0/3693663177 shutdown_connections 2023-07-14T16:15:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 1 -- 172.21.15.62:0/3693663177 wait complete. 2023-07-14T16:15:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.190+0000 7f043440bb80 20 client.15132 trim_cache size 0 max 16384 2023-07-14T16:15:05.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f043440bb80 1 Processor -- start 2023-07-14T16:15:05.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f043440bb80 1 -- start start 2023-07-14T16:15:05.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f043440bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f30f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f043440bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f310d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f043440bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f3103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f043440bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f30fd520 con 0x55c9f30fc480 2023-07-14T16:15:05.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f043440bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f3106f50 con 0x55c9f30fcdd0 2023-07-14T16:15:05.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f043440bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f2ff7470 con 0x55c9f30fdc20 2023-07-14T16:15:05.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f041a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f30f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f041a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f30f6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60254/0 (socket says 172.21.15.62:60254) 2023-07-14T16:15:05.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.209+0000 7f041a59c700 1 -- 172.21.15.62:0/2829731007 learned_addr learned my addr 172.21.15.62:0/2829731007 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f0413fff700 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f310d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f041ad9d700 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f3103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f041a59c700 1 -- 172.21.15.62:0/2829731007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 msgr2=0x55c9f3103fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f041a59c700 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f3103fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f041a59c700 1 -- 172.21.15.62:0/2829731007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 msgr2=0x55c9f310d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f041a59c700 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f310d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f041a59c700 1 -- 172.21.15.62:0/2829731007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9f30f3590 con 0x55c9f30fc480 2023-07-14T16:15:05.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f0413fff700 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f310d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:05.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f041a59c700 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f30f6860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f040c005d60 tx=0x7f040c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2220f6d8bdb3f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f04117fa700 1 -- 172.21.15.62:0/2829731007 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c017040 con 0x55c9f30fc480 2023-07-14T16:15:05.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f04117fa700 1 -- 172.21.15.62:0/2829731007 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f040c01c610 con 0x55c9f30fc480 2023-07-14T16:15:05.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.210+0000 7f04117fa700 1 -- 172.21.15.62:0/2829731007 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c01cab0 con 0x55c9f30fc480 2023-07-14T16:15:05.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 -- 172.21.15.62:0/2829731007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 msgr2=0x55c9f30f6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f30f6860 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f040c005d60 tx=0x7f040c01c4c0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 -- 172.21.15.62:0/2829731007 shutdown_connections 2023-07-14T16:15:05.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f3103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f30f6860 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 --2- 172.21.15.62:0/2829731007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f310d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 -- 172.21.15.62:0/2829731007 >> 172.21.15.62:0/2829731007 conn(0x55c9f30f1d50 msgr2=0x55c9f3104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 -- 172.21.15.62:0/2829731007 shutdown_connections 2023-07-14T16:15:05.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 -- 172.21.15.62:0/2829731007 wait complete. 2023-07-14T16:15:05.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.211+0000 7f043440bb80 1 -- start start 2023-07-14T16:15:05.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f043440bb80 1 Processor -- start 2023-07-14T16:15:05.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f043440bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f310b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f043440bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f3162ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f043440bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f3165420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f043440bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f3165960 con 0x55c9f30fc480 2023-07-14T16:15:05.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f043440bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f3165ad0 con 0x55c9f30fcdd0 2023-07-14T16:15:05.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f043440bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f3165c90 con 0x55c9f30fdc20 2023-07-14T16:15:05.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f043440bb80 10 client.0 _ll_register_callbacks cb 0x55c9f2feb710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:05.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f041a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f310b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f041a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f310b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60266/0 (socket says 172.21.15.62:60266) 2023-07-14T16:15:05.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f041a59c700 1 -- 172.21.15.62:0/530620739 learned_addr learned my addr 172.21.15.62:0/530620739 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f041ad9d700 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f3165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.212+0000 7f0413fff700 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f3162ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f041a59c700 1 -- 172.21.15.62:0/530620739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 msgr2=0x55c9f3165420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f041a59c700 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f3165420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f041a59c700 1 -- 172.21.15.62:0/530620739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 msgr2=0x55c9f3162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f041a59c700 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f3162ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f041a59c700 1 -- 172.21.15.62:0/530620739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9f2e64c10 con 0x55c9f30fc480 2023-07-14T16:15:05.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f041a59c700 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f310b1f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f040c010f10 tx=0x7f040c005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c556e706840e3a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f0401ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:05.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f0401ffb700 1 -- 172.21.15.62:0/530620739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c017040 con 0x55c9f30fc480 2023-07-14T16:15:05.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f0401ffb700 1 -- 172.21.15.62:0/530620739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f040c00e3b0 con 0x55c9f30fc480 2023-07-14T16:15:05.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f043440bb80 1 -- 172.21.15.62:0/530620739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04040975d0 con 0x55c9f30fc480 2023-07-14T16:15:05.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f0401ffb700 1 -- 172.21.15.62:0/530620739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f040c01ca60 con 0x55c9f30fc480 2023-07-14T16:15:05.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f043440bb80 10 client.15138 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:05.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.213+0000 7f043440bb80 1 -- 172.21.15.62:0/530620739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c9f30fe510 con 0x55c9f30fc480 2023-07-14T16:15:05.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f0401ffb700 1 -- 172.21.15.62:0/530620739 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f040c040020 con 0x55c9f30fc480 2023-07-14T16:15:05.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f0401ffb700 1 -- 172.21.15.62:0/530620739 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f040c00e710 con 0x55c9f30fc480 2023-07-14T16:15:05.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f043440bb80 10 client.15138 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:05.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f043440bb80 10 client.15138 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-14T16:15:05.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f043440bb80 20 client.15138 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:05.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f043440bb80 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c9f316a640 0x55c9f3166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f043440bb80 4 client.15138 mds_command: new command op to 24512 tid=0[{"prefix": "status"}] 2023-07-14T16:15:05.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f043440bb80 1 -- 172.21.15.62:0/530620739 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c9f3009000 con 0x55c9f316a640 2023-07-14T16:15:05.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.214+0000 7f0413fff700 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c9f316a640 0x55c9f3166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f0413fff700 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c9f316a640 0x55c9f3166fb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33ee3e816cdb0e80 server_cookie=df7b86474f6edce5 in_seq=0 out_seq=0 2023-07-14T16:15:05.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f0401ffb700 10 client.15138 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:15:05.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f0401ffb700 1 -- 172.21.15.62:0/530620739 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c9f3009000 con 0x55c9f316a640 2023-07-14T16:15:05.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f0401ffb700 10 client.15138 handle_command_reply: tid=0 2023-07-14T16:15:05.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 client.15138 shutdown 2023-07-14T16:15:05.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 -- 172.21.15.62:0/530620739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 msgr2=0x55c9f310b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f310b1f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f040c010f10 tx=0x7f040c005680 comp rx=0 tx=0).stop 2023-07-14T16:15:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 -- 172.21.15.62:0/530620739 shutdown_connections 2023-07-14T16:15:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c9f316a640 0x55c9f3166fb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9f30fdc20 0x55c9f3165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c9f30fc480 0x55c9f310b1f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 --2- 172.21.15.62:0/530620739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c9f30fcdd0 0x55c9f3162ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f0401ffb700 0 client.15138 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:15:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.215+0000 7f043440bb80 1 -- 172.21.15.62:0/530620739 >> 172.21.15.62:0/530620739 conn(0x55c9f30f1d50 msgr2=0x55c9f3103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f043440bb80 1 -- 172.21.15.62:0/530620739 shutdown_connections 2023-07-14T16:15:05.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f043440bb80 1 -- 172.21.15.62:0/530620739 wait complete. 2023-07-14T16:15:05.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f043440bb80 20 client.15138 trim_cache size 0 max 16384 2023-07-14T16:15:05.214 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:15:05.214 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2023-07-14T16:15:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24512, 2023-07-14T16:15:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2023-07-14T16:15:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2023-07-14T16:15:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2023-07-14T16:15:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 61.651592645999997, 2023-07-14T16:15:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 23, 2023-07-14T16:15:05.215 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 88, 2023-07-14T16:15:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 86, 2023-07-14T16:15:05.216 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 94.715871839000002 2023-07-14T16:15:05.216 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:05.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 -- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04080a1260 msgr2=0x7f04080a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04080a1260 0x7f04080a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f041400f870 tx=0x7f041400fc30 comp rx=0 tx=0).stop 2023-07-14T16:15:05.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 -- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 msgr2=0x7f041c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 0x7f041c197f00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0404005910 tx=0x7f0404005740 comp rx=0 tx=0).stop 2023-07-14T16:15:05.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 -- 172.21.15.62:0/1950613683 shutdown_connections 2023-07-14T16:15:05.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c101660 0x7f041c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04080a1260 0x7f04080a3710 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c0ffae0 0x7f041c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 --2- 172.21.15.62:0/1950613683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c100cd0 0x7f041c197f00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 -- 172.21.15.62:0/1950613683 >> 172.21.15.62:0/1950613683 conn(0x7f041c0fb0e0 msgr2=0x7f041c0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.216+0000 7f04209ff700 1 -- 172.21.15.62:0/1950613683 shutdown_connections 2023-07-14T16:15:05.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.217+0000 7f04209ff700 1 -- 172.21.15.62:0/1950613683 wait complete. 2023-07-14T16:15:05.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T16:15:05.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 -- 172.21.15.62:0/1642920115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c010c0b0 msgr2=0x7f15c010e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 --2- 172.21.15.62:0/1642920115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c010c0b0 0x7f15c010e490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f15b800b0a0 tx=0x7f15b800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 -- 172.21.15.62:0/1642920115 shutdown_connections 2023-07-14T16:15:05.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 --2- 172.21.15.62:0/1642920115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c010c0b0 0x7f15c010e490 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 --2- 172.21.15.62:0/1642920115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15c0074550 0x7f15c00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 --2- 172.21.15.62:0/1642920115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c0072f70 0x7f15c00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 -- 172.21.15.62:0/1642920115 >> 172.21.15.62:0/1642920115 conn(0x7f15c00fb0e0 msgr2=0x7f15c00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 -- 172.21.15.62:0/1642920115 shutdown_connections 2023-07-14T16:15:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.456+0000 7f15c7fff700 1 -- 172.21.15.62:0/1642920115 wait complete. 2023-07-14T16:15:05.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.458+0000 7f15c7fff700 1 Processor -- start 2023-07-14T16:15:05.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.458+0000 7f15c7fff700 1 -- start start 2023-07-14T16:15:05.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.458+0000 7f15c7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15c0072f70 0x7f15c019be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.458+0000 7f15c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c0074550 0x7f15c019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.458+0000 7f15c7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c010c0b0 0x7f15c01a3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.458+0000 7f15c7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01a3a00 con 0x7f15c0074550 2023-07-14T16:15:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.458+0000 7f15c7fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01a3b40 con 0x7f15c0072f70 2023-07-14T16:15:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.458+0000 7f15c7fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01a3cd0 con 0x7f15c010c0b0 2023-07-14T16:15:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c0074550 0x7f15c019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c0074550 0x7f15c019c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60286/0 (socket says 172.21.15.62:60286) 2023-07-14T16:15:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 -- 172.21.15.62:0/3368242379 learned_addr learned my addr 172.21.15.62:0/3368242379 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15c0072f70 0x7f15c019be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c659c700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c010c0b0 0x7f15c01a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 -- 172.21.15.62:0/3368242379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c010c0b0 msgr2=0x7f15c01a3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c010c0b0 0x7f15c01a3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 -- 172.21.15.62:0/3368242379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15c0072f70 msgr2=0x7f15c019be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15c0072f70 0x7f15c019be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 -- 172.21.15.62:0/3368242379 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8009d00 con 0x7f15c0074550 2023-07-14T16:15:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.459+0000 7f15c559a700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c0074550 0x7f15c019c3c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f15b000e970 tx=0x7f15b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.460+0000 7f15b6ffd700 1 -- 172.21.15.62:0/3368242379 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0009790 con 0x7f15c0074550 2023-07-14T16:15:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.460+0000 7f15b6ffd700 1 -- 172.21.15.62:0/3368242379 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15b000c7e0 con 0x7f15c0074550 2023-07-14T16:15:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.460+0000 7f15c7fff700 1 -- 172.21.15.62:0/3368242379 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c01a3fb0 con 0x7f15c0074550 2023-07-14T16:15:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.460+0000 7f15b6ffd700 1 -- 172.21.15.62:0/3368242379 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0003bd0 con 0x7f15c0074550 2023-07-14T16:15:05.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.460+0000 7f15c7fff700 1 -- 172.21.15.62:0/3368242379 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c01a4480 con 0x7f15c0074550 2023-07-14T16:15:05.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.463+0000 7f15b6ffd700 1 -- 172.21.15.62:0/3368242379 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f15b000c950 con 0x7f15c0074550 2023-07-14T16:15:05.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.464+0000 7f15b6ffd700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f15ac0a1260 0x7f15ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.465+0000 7f15b6ffd700 1 -- 172.21.15.62:0/3368242379 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f15b0014070 con 0x7f15c0074550 2023-07-14T16:15:05.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.465+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f15ac0a1260 0x7f15ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.465+0000 7f15dba17b80 1 Processor -- start 2023-07-14T16:15:05.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.465+0000 7f15dba17b80 1 -- start start 2023-07-14T16:15:05.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a59e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f15ac0a1260 0x7f15ac0a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005950 tx=0x7f15bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a59f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15dba17b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a4a3c60 con 0x56271a59fc20 2023-07-14T16:15:05.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15dba17b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a4f69f0 con 0x56271a59e480 2023-07-14T16:15:05.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15dba17b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a4efe80 con 0x56271a59edd0 2023-07-14T16:15:05.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60290/0 (socket says 172.21.15.62:60290) 2023-07-14T16:15:05.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c659c700 1 -- 172.21.15.62:0/1684533313 learned_addr learned my addr 172.21.15.62:0/1684533313 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c559a700 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a59f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a59e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c659c700 1 -- 172.21.15.62:0/1684533313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 msgr2=0x56271a59f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c659c700 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a59f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c659c700 1 -- 172.21.15.62:0/1684533313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 msgr2=0x56271a59e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c659c700 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a59e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c659c700 1 -- 172.21.15.62:0/1684533313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56271a31dd90 con 0x56271a59fc20 2023-07-14T16:15:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c559a700 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a59f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.466+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a59e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.467+0000 7f15c659c700 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a0090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f15b80079c0 tx=0x7f15b80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877db0f0fbe2cf30 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.467+0000 7f15abfff700 1 -- 172.21.15.62:0/1684533313 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800e040 con 0x56271a59fc20 2023-07-14T16:15:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.467+0000 7f15abfff700 1 -- 172.21.15.62:0/1684533313 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15b80076e0 con 0x56271a59fc20 2023-07-14T16:15:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.467+0000 7f15abfff700 1 -- 172.21.15.62:0/1684533313 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8003d20 con 0x56271a59fc20 2023-07-14T16:15:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 -- 172.21.15.62:0/1684533313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 msgr2=0x56271a5a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a0090 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f15b80079c0 tx=0x7f15b80093d0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 -- 172.21.15.62:0/1684533313 shutdown_connections 2023-07-14T16:15:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a0090 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a59e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 --2- 172.21.15.62:0/1684533313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a59f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 -- 172.21.15.62:0/1684533313 >> 172.21.15.62:0/1684533313 conn(0x56271a593d50 msgr2=0x56271a599180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 -- 172.21.15.62:0/1684533313 shutdown_connections 2023-07-14T16:15:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 -- 172.21.15.62:0/1684533313 wait complete. 2023-07-14T16:15:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.468+0000 7f15dba17b80 1 -- start start 2023-07-14T16:15:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.469+0000 7f15dba17b80 1 Processor -- start 2023-07-14T16:15:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.469+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.469+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.469+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.469+0000 7f15dba17b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a48fa70 con 0x56271a59fc20 2023-07-14T16:15:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.469+0000 7f15dba17b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a48f770 con 0x56271a59edd0 2023-07-14T16:15:05.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.469+0000 7f15dba17b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a3738d0 con 0x56271a59e480 2023-07-14T16:15:05.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.469+0000 7f15dba17b80 10 client.0 _ll_register_callbacks cb 0x56271a48d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:05.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35710/0 (socket says 172.21.15.62:35710) 2023-07-14T16:15:05.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c5d9b700 1 -- 172.21.15.62:0/968119535 learned_addr learned my addr 172.21.15.62:0/968119535 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c559a700 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c659c700 1 -- 172.21.15.62:0/968119535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 msgr2=0x56271a5a8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c659c700 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c659c700 1 -- 172.21.15.62:0/968119535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 msgr2=0x56271a5a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c659c700 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5a85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c659c700 1 -- 172.21.15.62:0/968119535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c01a4480 con 0x56271a59fc20 2023-07-14T16:15:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c559a700 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15c659c700 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5af600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f15b800b780 tx=0x7f15b80078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244834326f95969d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.470+0000 7f15a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15a97fa700 1 -- 172.21.15.62:0/968119535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800e040 con 0x56271a59fc20 2023-07-14T16:15:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15dba17b80 1 -- 172.21.15.62:0/968119535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56271a4494a0 con 0x56271a59fc20 2023-07-14T16:15:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15a97fa700 1 -- 172.21.15.62:0/968119535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f15b80050c0 con 0x56271a59fc20 2023-07-14T16:15:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15a97fa700 1 -- 172.21.15.62:0/968119535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15b8005280 con 0x56271a59fc20 2023-07-14T16:15:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15dba17b80 10 client.15150 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15dba17b80 1 -- 172.21.15.62:0/968119535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f15c010ee40 con 0x56271a59fc20 2023-07-14T16:15:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15a97fa700 1 -- 172.21.15.62:0/968119535 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56271a4494a0 con 0x56271a59fc20 2023-07-14T16:15:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15a97fa700 1 -- 172.21.15.62:0/968119535 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f15b8019050 con 0x56271a59fc20 2023-07-14T16:15:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.471+0000 7f15dba17b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.472+0000 7f15dba17b80 10 client.15150 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T16:15:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.472+0000 7f15dba17b80 20 client.15150 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.472+0000 7f15dba17b80 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56271a5b2e10 0x56271a5b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.472+0000 7f15dba17b80 4 client.15150 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:15:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.472+0000 7f15dba17b80 1 -- 172.21.15.62:0/968119535 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56271a4e8e60 con 0x56271a5b2e10 2023-07-14T16:15:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.472+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56271a5b2e10 0x56271a5b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.473+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56271a5b2e10 0x56271a5b2530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f176d6781c1d3081 server_cookie=28217c48bc47b2eb in_seq=0 out_seq=0 2023-07-14T16:15:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.473+0000 7f15a97fa700 10 client.15150 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:15:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.473+0000 7f15a97fa700 1 -- 172.21.15.62:0/968119535 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56271a4e8e60 con 0x56271a5b2e10 2023-07-14T16:15:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.473+0000 7f15a97fa700 10 client.15150 handle_command_reply: tid=0 2023-07-14T16:15:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 client.15150 shutdown 2023-07-14T16:15:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 -- 172.21.15.62:0/968119535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 msgr2=0x56271a5af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5af600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f15b800b780 tx=0x7f15b80078d0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 -- 172.21.15.62:0/968119535 shutdown_connections 2023-07-14T16:15:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56271a5b2e10 0x56271a5b2530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5af600 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15a97fa700 0 client.15150 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:15:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 --2- 172.21.15.62:0/968119535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 -- 172.21.15.62:0/968119535 >> 172.21.15.62:0/968119535 conn(0x56271a593d50 msgr2=0x56271a598860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 -- 172.21.15.62:0/968119535 shutdown_connections 2023-07-14T16:15:05.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 1 -- 172.21.15.62:0/968119535 wait complete. 2023-07-14T16:15:05.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.474+0000 7f15dba17b80 20 client.15150 trim_cache size 0 max 16384 2023-07-14T16:15:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15dba17b80 1 Processor -- start 2023-07-14T16:15:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15dba17b80 1 -- start start 2023-07-14T16:15:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a5b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a5af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15dba17b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a5a8f80 con 0x56271a59fc20 2023-07-14T16:15:05.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15dba17b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a4e8e60 con 0x56271a59e480 2023-07-14T16:15:05.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15dba17b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a4494a0 con 0x56271a59edd0 2023-07-14T16:15:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60316/0 (socket says 172.21.15.62:60316) 2023-07-14T16:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.501+0000 7f15c659c700 1 -- 172.21.15.62:0/4080572808 learned_addr learned my addr 172.21.15.62:0/4080572808 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a5b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c559a700 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a5af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c5d9b700 1 -- 172.21.15.62:0/4080572808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 msgr2=0x56271a5af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a5af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c5d9b700 1 -- 172.21.15.62:0/4080572808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 msgr2=0x56271a5a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c5d9b700 1 -- 172.21.15.62:0/4080572808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56271a306c10 con 0x56271a59e480 2023-07-14T16:15:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c659c700 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a5b2530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x56271a5b4890 tx=0x7f15bc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca2a394a0b9fe1a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15b4ff9700 1 -- 172.21.15.62:0/4080572808 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc02b070 con 0x56271a59e480 2023-07-14T16:15:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15b4ff9700 1 -- 172.21.15.62:0/4080572808 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15bc015070 con 0x56271a59e480 2023-07-14T16:15:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.502+0000 7f15b4ff9700 1 -- 172.21.15.62:0/4080572808 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc025070 con 0x56271a59e480 2023-07-14T16:15:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 -- 172.21.15.62:0/4080572808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 msgr2=0x56271a5b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a5b2530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x56271a5b4890 tx=0x7f15bc013000 comp rx=0 tx=0).stop 2023-07-14T16:15:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 -- 172.21.15.62:0/4080572808 shutdown_connections 2023-07-14T16:15:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59e480 0x56271a5b2530 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4080572808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59edd0 0x56271a5af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 -- 172.21.15.62:0/4080572808 >> 172.21.15.62:0/4080572808 conn(0x56271a593d50 msgr2=0x56271a5bbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 -- 172.21.15.62:0/4080572808 shutdown_connections 2023-07-14T16:15:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.503+0000 7f15dba17b80 1 -- 172.21.15.62:0/4080572808 wait complete. 2023-07-14T16:15:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.504+0000 7f15dba17b80 1 -- start start 2023-07-14T16:15:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.504+0000 7f15dba17b80 1 Processor -- start 2023-07-14T16:15:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15dba17b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15dba17b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a5a5780 con 0x56271a59fc20 2023-07-14T16:15:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15dba17b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a5a58f0 con 0x56271a59edd0 2023-07-14T16:15:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15dba17b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271a606cc0 con 0x56271a59e480 2023-07-14T16:15:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15dba17b80 10 client.0 _ll_register_callbacks cb 0x7f15b0095b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15c5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15c5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35728/0 (socket says 172.21.15.62:35728) 2023-07-14T16:15:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15c5d9b700 1 -- 172.21.15.62:0/4284340754 learned_addr learned my addr 172.21.15.62:0/4284340754 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.505+0000 7f15c559a700 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15c5d9b700 1 -- 172.21.15.62:0/4284340754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 msgr2=0x56271a5af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15c5d9b700 1 -- 172.21.15.62:0/4284340754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 msgr2=0x56271a5a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15c5d9b700 1 -- 172.21.15.62:0/4284340754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c010ee40 con 0x56271a59e480 2023-07-14T16:15:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15c5d9b700 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a4af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15bc010e60 tx=0x7f15bc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb1c90d03471d1a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:15:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15a97fa700 1 -- 172.21.15.62:0/4284340754 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc025030 con 0x56271a59e480 2023-07-14T16:15:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15a97fa700 1 -- 172.21.15.62:0/4284340754 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f15bc017070 con 0x56271a59e480 2023-07-14T16:15:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15dba17b80 1 -- 172.21.15.62:0/4284340754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15b80076e0 con 0x56271a59e480 2023-07-14T16:15:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15a97fa700 1 -- 172.21.15.62:0/4284340754 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15bc02b070 con 0x56271a59e480 2023-07-14T16:15:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.506+0000 7f15dba17b80 10 client.24964 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15dba17b80 1 -- 172.21.15.62:0/4284340754 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f15bc020540 con 0x56271a59e480 2023-07-14T16:15:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15a97fa700 1 -- 172.21.15.62:0/4284340754 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15b80076e0 con 0x56271a59e480 2023-07-14T16:15:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15a97fa700 1 -- 172.21.15.62:0/4284340754 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f15bc01e9c0 con 0x56271a59e480 2023-07-14T16:15:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15dba17b80 10 client.24964 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:05.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15dba17b80 10 client.24964 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T16:15:05.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15dba17b80 20 client.24964 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:05.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56271a60b9d0 0x56271a608340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15dba17b80 4 client.24964 mds_command: new command op to 24565 tid=0[{"prefix": "status"}] 2023-07-14T16:15:05.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15dba17b80 1 -- 172.21.15.62:0/4284340754 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56271a4ab000 con 0x56271a60b9d0 2023-07-14T16:15:05.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.507+0000 7f15c559a700 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56271a60b9d0 0x56271a608340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15c559a700 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56271a60b9d0 0x56271a608340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14e2cfd199e89e22 server_cookie=8ac0ef90d65eccc9 in_seq=0 out_seq=0 2023-07-14T16:15:05.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15a97fa700 10 client.24964 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:15:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15a97fa700 1 -- 172.21.15.62:0/4284340754 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56271a4ab000 con 0x56271a60b9d0 2023-07-14T16:15:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15a97fa700 10 client.24964 handle_command_reply: tid=0 2023-07-14T16:15:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 client.24964 shutdown 2023-07-14T16:15:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 -- 172.21.15.62:0/4284340754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 msgr2=0x56271a5a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a4af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15bc010e60 tx=0x7f15bc004750 comp rx=0 tx=0).stop 2023-07-14T16:15:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 -- 172.21.15.62:0/4284340754 shutdown_connections 2023-07-14T16:15:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56271a60b9d0 0x56271a608340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56271a59fc20 0x56271a5a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56271a59e480 0x56271a5a4af0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 --2- 172.21.15.62:0/4284340754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56271a59edd0 0x56271a5af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15a97fa700 0 client.24964 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:15:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.508+0000 7f15dba17b80 1 -- 172.21.15.62:0/4284340754 >> 172.21.15.62:0/4284340754 conn(0x56271a593d50 msgr2=0x56271a5b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.509+0000 7f15dba17b80 1 -- 172.21.15.62:0/4284340754 shutdown_connections 2023-07-14T16:15:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.509+0000 7f15dba17b80 1 -- 172.21.15.62:0/4284340754 wait complete. 2023-07-14T16:15:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.509+0000 7f15dba17b80 20 client.24964 trim_cache size 0 max 16384 2023-07-14T16:15:05.495 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:15:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2023-07-14T16:15:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24565, 2023-07-14T16:15:05.495 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2023-07-14T16:15:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2023-07-14T16:15:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2023-07-14T16:15:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 62.924001668000002, 2023-07-14T16:15:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 23, 2023-07-14T16:15:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 88, 2023-07-14T16:15:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 86, 2023-07-14T16:15:05.496 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 82.097364631000005 2023-07-14T16:15:05.496 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.509+0000 7f15c7fff700 1 -- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f15ac0a1260 msgr2=0x7f15ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.509+0000 7f15c7fff700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f15ac0a1260 0x7f15ac0a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005950 tx=0x7f15bc00b410 comp rx=0 tx=0).stop 2023-07-14T16:15:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.509+0000 7f15c7fff700 1 -- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c0074550 msgr2=0x7f15c019c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.509+0000 7f15c7fff700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c0074550 0x7f15c019c3c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f15b000e970 tx=0x7f15b000ec80 comp rx=0 tx=0).stop 2023-07-14T16:15:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.510+0000 7f15c7fff700 1 -- 172.21.15.62:0/3368242379 shutdown_connections 2023-07-14T16:15:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.510+0000 7f15c7fff700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c010c0b0 0x7f15c01a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.510+0000 7f15c7fff700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f15ac0a1260 0x7f15ac0a3710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.510+0000 7f15c7fff700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15c0072f70 0x7f15c019be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.510+0000 7f15c7fff700 1 --2- 172.21.15.62:0/3368242379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15c0074550 0x7f15c019c3c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.510+0000 7f15c7fff700 1 -- 172.21.15.62:0/3368242379 >> 172.21.15.62:0/3368242379 conn(0x7f15c00fb0e0 msgr2=0x7f15c00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.510+0000 7f15c7fff700 1 -- 172.21.15.62:0/3368242379 shutdown_connections 2023-07-14T16:15:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.510+0000 7f15c7fff700 1 -- 172.21.15.62:0/3368242379 wait complete. 2023-07-14T16:15:05.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T16:15:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 -- 172.21.15.62:0/406507798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc80744f0 msgr2=0x7fbfc8072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/406507798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc80744f0 0x7fbfc8072940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009990 tx=0x7fbfb8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 -- 172.21.15.62:0/406507798 shutdown_connections 2023-07-14T16:15:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/406507798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8107880 0x7fbfc8109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/406507798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc80744f0 0x7fbfc8072940 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/406507798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfc8072f10 0x7fbfc8073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 -- 172.21.15.62:0/406507798 >> 172.21.15.62:0/406507798 conn(0x7fbfc80fb010 msgr2=0x7fbfc80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 -- 172.21.15.62:0/406507798 shutdown_connections 2023-07-14T16:15:05.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.747+0000 7fbfccf9d700 1 -- 172.21.15.62:0/406507798 wait complete. 2023-07-14T16:15:05.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.748+0000 7fbfccf9d700 1 Processor -- start 2023-07-14T16:15:05.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfccf9d700 1 -- start start 2023-07-14T16:15:05.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfccf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc8072f10 0x7fbfc81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfccf9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfc80744f0 0x7fbfc8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfccf9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8107880 0x7fbfc819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfccf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819f260 con 0x7fbfc8072f10 2023-07-14T16:15:05.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfccf9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819f3a0 con 0x7fbfc80744f0 2023-07-14T16:15:05.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfccf9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819f530 con 0x7fbfc8107880 2023-07-14T16:15:05.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfc80744f0 0x7fbfc8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc8072f10 0x7fbfc81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfc80744f0 0x7fbfc8197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41724/0 (socket says 172.21.15.62:41724) 2023-07-14T16:15:05.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfc6ffd700 1 -- 172.21.15.62:0/2515504019 learned_addr learned my addr 172.21.15.62:0/2515504019 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.749+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8107880 0x7fbfc819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc6ffd700 1 -- 172.21.15.62:0/2515504019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8107880 msgr2=0x7fbfc819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc6ffd700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8107880 0x7fbfc819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc6ffd700 1 -- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc8072f10 msgr2=0x7fbfc81976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc6ffd700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc8072f10 0x7fbfc81976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc6ffd700 1 -- 172.21.15.62:0/2515504019 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80095f0 con 0x7fbfc80744f0 2023-07-14T16:15:05.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8107880 0x7fbfc819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:05.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc8072f10 0x7fbfc81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:05.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc6ffd700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfc80744f0 0x7fbfc8197c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00e9a0 tx=0x7fbfbc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc4ff9700 1 -- 172.21.15.62:0/2515504019 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc009790 con 0x7fbfc80744f0 2023-07-14T16:15:05.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfccf9d700 1 -- 172.21.15.62:0/2515504019 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc819f810 con 0x7fbfc80744f0 2023-07-14T16:15:05.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc4ff9700 1 -- 172.21.15.62:0/2515504019 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfbc010430 con 0x7fbfc80744f0 2023-07-14T16:15:05.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.750+0000 7fbfc4ff9700 1 -- 172.21.15.62:0/2515504019 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00cea0 con 0x7fbfc80744f0 2023-07-14T16:15:05.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.751+0000 7fbfccf9d700 1 -- 172.21.15.62:0/2515504019 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc819fd70 con 0x7fbfc80744f0 2023-07-14T16:15:05.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.754+0000 7fbfc4ff9700 1 -- 172.21.15.62:0/2515504019 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbfbc0105a0 con 0x7fbfc80744f0 2023-07-14T16:15:05.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.755+0000 7fbfe1be2b80 1 Processor -- start 2023-07-14T16:15:05.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.755+0000 7fbfe1be2b80 1 -- start start 2023-07-14T16:15:05.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eae6d0 0x5608a1eaeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eaf020 0x5608a1eaf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfc4ff9700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbfb00a1290 0x7fbfb00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eafde0 0x5608a1eb0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1db3c60 con 0x5608a1eafde0 2023-07-14T16:15:05.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1e069f0 con 0x5608a1eaf020 2023-07-14T16:15:05.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1dffe80 con 0x5608a1eae6d0 2023-07-14T16:15:05.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfc4ff9700 1 -- 172.21.15.62:0/2515504019 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbfbc014070 con 0x7fbfc80744f0 2023-07-14T16:15:05.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eae6d0 0x5608a1eaeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eaf020 0x5608a1eaf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eae6d0 0x5608a1eaeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35758/0 (socket says 172.21.15.62:35758) 2023-07-14T16:15:05.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eaf020 0x5608a1eaf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41726/0 (socket says 172.21.15.62:41726) 2023-07-14T16:15:05.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfc7fff700 1 -- 172.21.15.62:0/1572195274 learned_addr learned my addr 172.21.15.62:0/1572195274 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.756+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/1572195274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eafde0 0x5608a1eb0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfc6ffd700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbfb00a1290 0x7fbfb00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfc77fe700 1 -- 172.21.15.62:0/1572195274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eae6d0 msgr2=0x5608a1eaeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/1572195274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eae6d0 0x5608a1eaeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfc77fe700 1 -- 172.21.15.62:0/1572195274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eafde0 msgr2=0x5608a1eb0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/1572195274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eafde0 0x5608a1eb0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfc77fe700 1 -- 172.21.15.62:0/1572195274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a1c2dd90 con 0x5608a1eaf020 2023-07-14T16:15:05.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/1572195274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eaf020 0x5608a1eaf850 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80098e0 tx=0x7fbfb8005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df25457c9372276 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfc6ffd700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbfb00a1290 0x7fbfb00a3740 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc096a10 tx=0x7fbfbc010bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfadffb700 1 -- 172.21.15.62:0/1572195274 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb801d070 con 0x5608a1eaf020 2023-07-14T16:15:05.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfadffb700 1 -- 172.21.15.62:0/1572195274 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfb801e040 con 0x5608a1eaf020 2023-07-14T16:15:05.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.757+0000 7fbfadffb700 1 -- 172.21.15.62:0/1572195274 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800ba00 con 0x5608a1eaf020 2023-07-14T16:15:05.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.758+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/1572195274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eaf020 msgr2=0x5608a1eaf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.758+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/1572195274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eaf020 0x5608a1eaf850 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80098e0 tx=0x7fbfb8005f00 comp rx=0 tx=0).stop 2023-07-14T16:15:05.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.759+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/1572195274 shutdown_connections 2023-07-14T16:15:05.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.759+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/1572195274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eafde0 0x5608a1eb0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.759+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/1572195274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eae6d0 0x5608a1eaeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.759+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/1572195274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eaf020 0x5608a1eaf850 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.759+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/1572195274 >> 172.21.15.62:0/1572195274 conn(0x5608a1ea3fa0 msgr2=0x5608a1ea9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.759+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/1572195274 shutdown_connections 2023-07-14T16:15:05.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.759+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/1572195274 wait complete. 2023-07-14T16:15:05.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.760+0000 7fbfe1be2b80 1 -- start start 2023-07-14T16:15:05.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.760+0000 7fbfe1be2b80 1 Processor -- start 2023-07-14T16:15:05.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eb8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1eb87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1d9fa70 con 0x5608a1eaf020 2023-07-14T16:15:05.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1d9f770 con 0x5608a1eae6d0 2023-07-14T16:15:05.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1c838d0 con 0x5608a1eafde0 2023-07-14T16:15:05.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfe1be2b80 10 client.0 _ll_register_callbacks cb 0x5608a1d9d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eb8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eb8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41732/0 (socket says 172.21.15.62:41732) 2023-07-14T16:15:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfc7fff700 1 -- 172.21.15.62:0/3835946823 learned_addr learned my addr 172.21.15.62:0/3835946823 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1eb87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.761+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfc7fff700 1 -- 172.21.15.62:0/3835946823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 msgr2=0x5608a1ebf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfc7fff700 1 -- 172.21.15.62:0/3835946823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 msgr2=0x5608a1eb87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1eb87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfc7fff700 1 -- 172.21.15.62:0/3835946823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc810a920 con 0x5608a1eae6d0 2023-07-14T16:15:05.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eb8260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000e7a0 tx=0x7fbfc0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f85a60d609d472a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:15:05.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/3835946823 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000ee80 con 0x5608a1eae6d0 2023-07-14T16:15:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3835946823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608a1d594a0 con 0x5608a1eae6d0 2023-07-14T16:15:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.762+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/3835946823 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbfc0003c10 con 0x5608a1eae6d0 2023-07-14T16:15:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/3835946823 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc0003dd0 con 0x5608a1eae6d0 2023-07-14T16:15:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfe1be2b80 10 client.24983 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3835946823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5608a1c16c10 con 0x5608a1eae6d0 2023-07-14T16:15:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/3835946823 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5608a1d594a0 con 0x5608a1eae6d0 2023-07-14T16:15:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/3835946823 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbfc001b450 con 0x5608a1eae6d0 2023-07-14T16:15:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfe1be2b80 10 client.24983 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfe1be2b80 10 client.24983 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T16:15:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfe1be2b80 20 client.24983 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5608a1ec3590 0x5608a1ec2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfe1be2b80 4 client.24983 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:15:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.763+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3835946823 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608a1df8e60 con 0x5608a1ec3590 2023-07-14T16:15:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.764+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5608a1ec3590 0x5608a1ec2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.764+0000 7fbfa6ffd700 10 client.24983 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.764+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5608a1ec3590 0x5608a1ec2990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d72f5d257062635 server_cookie=2bd085e5decc16ba in_seq=0 out_seq=0 2023-07-14T16:15:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/3835946823 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5608a1df8e60 con 0x5608a1ec3590 2023-07-14T16:15:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfa6ffd700 10 client.24983 handle_command_reply: tid=0 2023-07-14T16:15:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 client.24983 shutdown 2023-07-14T16:15:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3835946823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 msgr2=0x5608a1eb8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eb8260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000e7a0 tx=0x7fbfc0004b40 comp rx=0 tx=0).stop 2023-07-14T16:15:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3835946823 shutdown_connections 2023-07-14T16:15:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5608a1ec3590 0x5608a1ec2990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eb8260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3835946823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1eb87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfa6ffd700 0 client.24983 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.765+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3835946823 >> 172.21.15.62:0/3835946823 conn(0x5608a1ea3fa0 msgr2=0x5608a1ea8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.766+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3835946823 shutdown_connections 2023-07-14T16:15:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.766+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3835946823 wait complete. 2023-07-14T16:15:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.766+0000 7fbfe1be2b80 20 client.24983 trim_cache size 0 max 16384 2023-07-14T16:15:05.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfe1be2b80 1 Processor -- start 2023-07-14T16:15:05.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfe1be2b80 1 -- start start 2023-07-14T16:15:05.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1ec2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ec2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1eb7e30 con 0x5608a1eaf020 2023-07-14T16:15:05.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1dce790 con 0x5608a1eae6d0 2023-07-14T16:15:05.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1c8fde0 con 0x5608a1eafde0 2023-07-14T16:15:05.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ec2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ec2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60364/0 (socket says 172.21.15.62:60364) 2023-07-14T16:15:05.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfc77fe700 1 -- 172.21.15.62:0/3970940214 learned_addr learned my addr 172.21.15.62:0/3970940214 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.794+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1ec2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfc77fe700 1 -- 172.21.15.62:0/3970940214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 msgr2=0x5608a1ebf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfc77fe700 1 -- 172.21.15.62:0/3970940214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 msgr2=0x5608a1ec2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1ec2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfc77fe700 1 -- 172.21.15.62:0/3970940214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a1eb91b0 con 0x5608a1eaf020 2023-07-14T16:15:05.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:05.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ec2ed0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009990 tx=0x7fbfb8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e714177413f811c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfae7fc700 1 -- 172.21.15.62:0/3970940214 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb801d040 con 0x5608a1eaf020 2023-07-14T16:15:05.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfae7fc700 1 -- 172.21.15.62:0/3970940214 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfb800b310 con 0x5608a1eaf020 2023-07-14T16:15:05.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.795+0000 7fbfae7fc700 1 -- 172.21.15.62:0/3970940214 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8033b30 con 0x5608a1eaf020 2023-07-14T16:15:05.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3970940214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 msgr2=0x5608a1ec2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ec2ed0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009990 tx=0x7fbfb8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3970940214 shutdown_connections 2023-07-14T16:15:05.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1ebf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1ec2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/3970940214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ec2ed0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3970940214 >> 172.21.15.62:0/3970940214 conn(0x5608a1ea3fa0 msgr2=0x5608a1eb6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3970940214 shutdown_connections 2023-07-14T16:15:05.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.796+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/3970940214 wait complete. 2023-07-14T16:15:05.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 1 -- start start 2023-07-14T16:15:05.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 1 Processor -- start 2023-07-14T16:15:05.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ebf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1eba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1f17410 con 0x5608a1eaf020 2023-07-14T16:15:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1f17550 con 0x5608a1eae6d0 2023-07-14T16:15:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a1f17690 con 0x5608a1eafde0 2023-07-14T16:15:05.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfe1be2b80 10 client.0 _ll_register_callbacks cb 0x5608a1d9d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ebf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ebf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60376/0 (socket says 172.21.15.62:60376) 2023-07-14T16:15:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.797+0000 7fbfc77fe700 1 -- 172.21.15.62:0/4039907325 learned_addr learned my addr 172.21.15.62:0/4039907325 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1eba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc77fe700 1 -- 172.21.15.62:0/4039907325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 msgr2=0x5608a1eba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1eba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc77fe700 1 -- 172.21.15.62:0/4039907325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 msgr2=0x5608a1eba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:05.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eba180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc77fe700 1 -- 172.21.15.62:0/4039907325 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608a1c2dd90 con 0x5608a1eaf020 2023-07-14T16:15:05.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1eba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:05.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfc77fe700 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ebf6f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8036040 tx=0x7fbfb80337d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c042e600cf0c1358 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:05.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfa6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:05.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/4039907325 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb801d040 con 0x5608a1eaf020 2023-07-14T16:15:05.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/4039907325 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608a1dbb000 con 0x5608a1eaf020 2023-07-14T16:15:05.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.798+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/4039907325 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbfb8031070 con 0x5608a1eaf020 2023-07-14T16:15:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/4039907325 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb8005360 con 0x5608a1eaf020 2023-07-14T16:15:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfe1be2b80 10 client.15174 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/4039907325 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5608a1c54200 con 0x5608a1eaf020 2023-07-14T16:15:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/4039907325 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5608a1dbb000 con 0x5608a1eaf020 2023-07-14T16:15:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/4039907325 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbfb8045070 con 0x5608a1eaf020 2023-07-14T16:15:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfe1be2b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:05.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfe1be2b80 10 client.15174 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T16:15:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfe1be2b80 20 client.15174 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5608a1f1bf00 0x5608a1f18860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfe1be2b80 4 client.15174 mds_command: new command op to 24602 tid=0[{"prefix": "status"}] 2023-07-14T16:15:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.799+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/4039907325 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5608a1c4ef20 con 0x5608a1f1bf00 2023-07-14T16:15:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.800+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5608a1f1bf00 0x5608a1f18860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.800+0000 7fbfa6ffd700 10 client.15174 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.800+0000 7fbfc7fff700 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5608a1f1bf00 0x5608a1f18860 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7531b9f053e5de server_cookie=b6f4f880bdfca126 in_seq=0 out_seq=0 2023-07-14T16:15:05.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.800+0000 7fbfa6ffd700 1 -- 172.21.15.62:0/4039907325 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5608a1c4ef20 con 0x5608a1f1bf00 2023-07-14T16:15:05.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.800+0000 7fbfa6ffd700 10 client.15174 handle_command_reply: tid=0 2023-07-14T16:15:05.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.800+0000 7fbfe1be2b80 1 client.15174 shutdown 2023-07-14T16:15:05.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.800+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/4039907325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 msgr2=0x5608a1ebf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.800+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ebf6f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8036040 tx=0x7fbfb80337d0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/4039907325 shutdown_connections 2023-07-14T16:15:05.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5608a1eafde0 0x5608a1eba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5608a1f1bf00 0x5608a1f18860 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608a1eae6d0 0x5608a1eba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 1 --2- 172.21.15.62:0/4039907325 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5608a1eaf020 0x5608a1ebf6f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/4039907325 >> 172.21.15.62:0/4039907325 conn(0x5608a1ea3fa0 msgr2=0x5608a1ec2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfa6ffd700 0 client.15174 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:05.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/4039907325 shutdown_connections 2023-07-14T16:15:05.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 1 -- 172.21.15.62:0/4039907325 wait complete. 2023-07-14T16:15:05.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfe1be2b80 20 client.15174 trim_cache size 0 max 16384 2023-07-14T16:15:05.788 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:15:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2023-07-14T16:15:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24602, 2023-07-14T16:15:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2023-07-14T16:15:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2023-07-14T16:15:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2023-07-14T16:15:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 66.244053359999995, 2023-07-14T16:15:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 23, 2023-07-14T16:15:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 88, 2023-07-14T16:15:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 86, 2023-07-14T16:15:05.789 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 72.804237927000003 2023-07-14T16:15:05.789 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:05.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 -- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbfb00a1290 msgr2=0x7fbfb00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbfb00a1290 0x7fbfb00a3740 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc096a10 tx=0x7fbfbc010bf0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 -- 172.21.15.62:0/2515504019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfc80744f0 msgr2=0x7fbfc8197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:05.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfc80744f0 0x7fbfc8197c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00e9a0 tx=0x7fbfbc00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 -- 172.21.15.62:0/2515504019 shutdown_connections 2023-07-14T16:15:05.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfc8107880 0x7fbfc819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfc8072f10 0x7fbfc81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbfb00a1290 0x7fbfb00a3740 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 --2- 172.21.15.62:0/2515504019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbfc80744f0 0x7fbfc8197c20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:05.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.801+0000 7fbfccf9d700 1 -- 172.21.15.62:0/2515504019 >> 172.21.15.62:0/2515504019 conn(0x7fbfc80fb010 msgr2=0x7fbfc80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:05.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.802+0000 7fbfccf9d700 1 -- 172.21.15.62:0/2515504019 shutdown_connections 2023-07-14T16:15:05.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:05.802+0000 7fbfccf9d700 1 -- 172.21.15.62:0/2515504019 wait complete. 2023-07-14T16:15:05.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:05 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2775604654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:05.797 INFO:tasks.cephadm:Setup complete, yielding 2023-07-14T16:15:05.797 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T16:15:05.807 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2023-07-14T16:15:05.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- bash -c 'ceph orch status' 2023-07-14T16:15:05.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:05 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2775604654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:06.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:05 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2775604654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:06.549 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:15:06.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:06 smithi062 ceph-mon[95543]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:06.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:06 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:06 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:06 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:06 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:06 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:06 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:06 smithi081 ceph-mon[127795]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:06 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:06 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:06 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:06 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:06 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:06 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:07.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:06 smithi164 ceph-mon[97427]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:06 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:06 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:06 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:06 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:06 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:06 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:07.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:15:07.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:15:07.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:15:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:15:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:15:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:15:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:15:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.129+0000 7f3670ac1700 1 -- 172.21.15.62:0/461976699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c101720 msgr2=0x7f366c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.129+0000 7f3670ac1700 1 --2- 172.21.15.62:0/461976699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c101720 0x7f366c107fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f366400b0a0 tx=0x7f366400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.129+0000 7f3670ac1700 1 -- 172.21.15.62:0/461976699 shutdown_connections 2023-07-14T16:15:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.129+0000 7f3670ac1700 1 --2- 172.21.15.62:0/461976699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c101720 0x7f366c107fe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.129+0000 7f3670ac1700 1 --2- 172.21.15.62:0/461976699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c0ffb90 0x7f366c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.129+0000 7f3670ac1700 1 --2- 172.21.15.62:0/461976699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c100d90 0x7f366c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.129+0000 7f3670ac1700 1 -- 172.21.15.62:0/461976699 >> 172.21.15.62:0/461976699 conn(0x7f366c0fb1a0 msgr2=0x7f366c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:08.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.130+0000 7f3670ac1700 1 -- 172.21.15.62:0/461976699 shutdown_connections 2023-07-14T16:15:08.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.130+0000 7f3670ac1700 1 -- 172.21.15.62:0/461976699 wait complete. 2023-07-14T16:15:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f3670ac1700 1 Processor -- start 2023-07-14T16:15:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f3670ac1700 1 -- start start 2023-07-14T16:15:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f3670ac1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0ffb90 0x7f366c1958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f3670ac1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c100d90 0x7f366c195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f3670ac1700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c101720 0x7f366c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f3670ac1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c19d460 con 0x7f366c0ffb90 2023-07-14T16:15:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f3670ac1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c19d5a0 con 0x7f366c100d90 2023-07-14T16:15:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f3670ac1700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c19d730 con 0x7f366c101720 2023-07-14T16:15:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.131+0000 7f366a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0ffb90 0x7f366c1958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0ffb90 0x7f366c1958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60394/0 (socket says 172.21.15.62:60394) 2023-07-14T16:15:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366a59c700 1 -- 172.21.15.62:0/1442300914 learned_addr learned my addr 172.21.15.62:0/1442300914 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f3669d9b700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c100d90 0x7f366c195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366ad9d700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c101720 0x7f366c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366a59c700 1 -- 172.21.15.62:0/1442300914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c101720 msgr2=0x7f366c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366a59c700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c101720 0x7f366c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366a59c700 1 -- 172.21.15.62:0/1442300914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c100d90 msgr2=0x7f366c195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366a59c700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c100d90 0x7f366c195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366a59c700 1 -- 172.21.15.62:0/1442300914 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3664009d00 con 0x7f366c0ffb90 2023-07-14T16:15:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366ad9d700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c101720 0x7f366c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f366a59c700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0ffb90 0x7f366c1958d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f365c00e9a0 tx=0x7f365c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.132+0000 7f365b7fe700 1 -- 172.21.15.62:0/1442300914 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c009790 con 0x7f366c0ffb90 2023-07-14T16:15:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.133+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c19da10 con 0x7f366c0ffb90 2023-07-14T16:15:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.133+0000 7f365b7fe700 1 -- 172.21.15.62:0/1442300914 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f365c0044b0 con 0x7f366c0ffb90 2023-07-14T16:15:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.133+0000 7f365b7fe700 1 -- 172.21.15.62:0/1442300914 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c003c60 con 0x7f366c0ffb90 2023-07-14T16:15:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.133+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c19df10 con 0x7f366c0ffb90 2023-07-14T16:15:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.134+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c1071e0 con 0x7f366c0ffb90 2023-07-14T16:15:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.136+0000 7f365b7fe700 1 -- 172.21.15.62:0/1442300914 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f365c003dc0 con 0x7f366c0ffb90 2023-07-14T16:15:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.137+0000 7f365b7fe700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36540a11e0 0x7f36540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.137+0000 7f365b7fe700 1 -- 172.21.15.62:0/1442300914 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f365c014070 con 0x7f366c0ffb90 2023-07-14T16:15:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.137+0000 7f3669d9b700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36540a11e0 0x7f36540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.138+0000 7f3669d9b700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36540a11e0 0x7f36540a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3660005950 tx=0x7f366000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.141+0000 7f365b7fe700 1 -- 172.21.15.62:0/1442300914 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f365c096380 con 0x7f366c0ffb90 2023-07-14T16:15:08.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.362+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f366c060950 con 0x7f36540a11e0 2023-07-14T16:15:08.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.363+0000 7f365b7fe700 1 -- 172.21.15.62:0/1442300914 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f366c060950 con 0x7f36540a11e0 2023-07-14T16:15:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:Backend: cephadm 2023-07-14T16:15:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:Available: Yes 2023-07-14T16:15:08.342 INFO:teuthology.orchestra.run.smithi062.stdout:Paused: No 2023-07-14T16:15:08.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36540a11e0 msgr2=0x7f36540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:08.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36540a11e0 0x7f36540a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3660005950 tx=0x7f366000b410 comp rx=0 tx=0).stop 2023-07-14T16:15:08.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0ffb90 msgr2=0x7f366c1958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:08.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0ffb90 0x7f366c1958d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f365c00e9a0 tx=0x7f365c00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 shutdown_connections 2023-07-14T16:15:08.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f366c101720 0x7f366c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f366c0ffb90 0x7f366c1958d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36540a11e0 0x7f36540a3690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 --2- 172.21.15.62:0/1442300914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f366c100d90 0x7f366c195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:08.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 >> 172.21.15.62:0/1442300914 conn(0x7f366c0fb1a0 msgr2=0x7f366c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:08.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 shutdown_connections 2023-07-14T16:15:08.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:08.366+0000 7f3670ac1700 1 -- 172.21.15.62:0/1442300914 wait complete. 2023-07-14T16:15:08.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:08 smithi062 ceph-mon[95543]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:15:08.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-14T16:15:08.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:08 smithi081 ceph-mon[127795]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:15:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:08 smithi164 ceph-mon[97427]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:15:09.562 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:15:09.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:09 smithi062 ceph-mon[95543]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:09.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:09 smithi081 ceph-mon[127795]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:10.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:09 smithi164 ceph-mon[97427]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:10.822 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:10 smithi062 ceph-mon[95543]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:10.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:10 smithi081 ceph-mon[127795]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:11.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:10 smithi164 ceph-mon[97427]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:11.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.087+0000 7f81655d7700 1 -- 172.21.15.62:0/118000624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 msgr2=0x7f81601011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:11.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.087+0000 7f81655d7700 1 --2- 172.21.15.62:0/118000624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 0x7f81601011e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8154009990 tx=0x7f8154009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.087+0000 7f81655d7700 1 -- 172.21.15.62:0/118000624 shutdown_connections 2023-07-14T16:15:11.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.087+0000 7f81655d7700 1 --2- 172.21.15.62:0/118000624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8160101720 0x7f8160107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.087+0000 7f81655d7700 1 --2- 172.21.15.62:0/118000624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81600ffb90 0x7f81600fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.087+0000 7f81655d7700 1 --2- 172.21.15.62:0/118000624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 0x7f81601011e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.087+0000 7f81655d7700 1 -- 172.21.15.62:0/118000624 >> 172.21.15.62:0/118000624 conn(0x7f81600fb1a0 msgr2=0x7f81600fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.087+0000 7f81655d7700 1 -- 172.21.15.62:0/118000624 shutdown_connections 2023-07-14T16:15:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.088+0000 7f81655d7700 1 -- 172.21.15.62:0/118000624 wait complete. 2023-07-14T16:15:11.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.088+0000 7f81655d7700 1 Processor -- start 2023-07-14T16:15:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f81655d7700 1 -- start start 2023-07-14T16:15:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f81655d7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81600ffb90 0x7f8160197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f81655d7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 0x7f8160197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f81655d7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8160101720 0x7f816019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:11.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f81655d7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019f520 con 0x7f8160100d90 2023-07-14T16:15:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f81655d7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019f660 con 0x7f81600ffb90 2023-07-14T16:15:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f81655d7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019f7f0 con 0x7f8160101720 2023-07-14T16:15:11.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f815e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 0x7f8160197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:11.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f815e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 0x7f8160197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60416/0 (socket says 172.21.15.62:60416) 2023-07-14T16:15:11.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f815e7fc700 1 -- 172.21.15.62:0/2111197711 learned_addr learned my addr 172.21.15.62:0/2111197711 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:11.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.089+0000 7f815effd700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81600ffb90 0x7f8160197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:11.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f815f7fe700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8160101720 0x7f816019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:11.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f815e7fc700 1 -- 172.21.15.62:0/2111197711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8160101720 msgr2=0x7f816019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:11.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f815e7fc700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8160101720 0x7f816019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f815e7fc700 1 -- 172.21.15.62:0/2111197711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81600ffb90 msgr2=0x7f8160197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:11.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f815e7fc700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81600ffb90 0x7f8160197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f815e7fc700 1 -- 172.21.15.62:0/2111197711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540095f0 con 0x7f8160100d90 2023-07-14T16:15:11.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f815e7fc700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 0x7f8160197ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f815400c010 tx=0x7f8154003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f8147fff700 1 -- 172.21.15.62:0/2111197711 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815401d070 con 0x7f8160100d90 2023-07-14T16:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f8147fff700 1 -- 172.21.15.62:0/2111197711 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f815400bd70 con 0x7f8160100d90 2023-07-14T16:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f8147fff700 1 -- 172.21.15.62:0/2111197711 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154022c30 con 0x7f8160100d90 2023-07-14T16:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.090+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816019fa70 con 0x7f8160100d90 2023-07-14T16:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.091+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816019fe30 con 0x7f8160100d90 2023-07-14T16:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.092+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816010b2c0 con 0x7f8160100d90 2023-07-14T16:15:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.094+0000 7f8147fff700 1 -- 172.21.15.62:0/2111197711 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8154003f00 con 0x7f8160100d90 2023-07-14T16:15:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.095+0000 7f8147fff700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81480a1260 0x7f81480a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.095+0000 7f8147fff700 1 -- 172.21.15.62:0/2111197711 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f81540ce050 con 0x7f8160100d90 2023-07-14T16:15:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.095+0000 7f815effd700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81480a1260 0x7f81480a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.096+0000 7f815effd700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81480a1260 0x7f81480a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8160100bf0 tx=0x7f815000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:11.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.099+0000 7f8147fff700 1 -- 172.21.15.62:0/2111197711 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f81540976e0 con 0x7f8160100d90 2023-07-14T16:15:11.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.322+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f81601076f0 con 0x7f81480a1260 2023-07-14T16:15:11.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.335+0000 7f8147fff700 1 -- 172.21.15.62:0/2111197711 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f81601076f0 con 0x7f81480a1260 2023-07-14T16:15:11.315 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-14T16:15:11.315 INFO:teuthology.orchestra.run.smithi062.stdout:mds.a smithi062 running (81s) 4s ago 81s 13.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 94df584da400 2023-07-14T16:15:11.315 INFO:teuthology.orchestra.run.smithi062.stdout:mds.b smithi081 running (78s) 5s ago 78s 28.6M - 18.0.0-4757-g3fc0089c afd12e232ec0 3060dcfd8d17 2023-07-14T16:15:11.315 INFO:teuthology.orchestra.run.smithi062.stdout:mds.c smithi164 running (94s) 5s ago 94s 13.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 0f64132aaa49 2023-07-14T16:15:11.315 INFO:teuthology.orchestra.run.smithi062.stdout:mds.d smithi062 running (104s) 4s ago 103s 13.8M - 18.0.0-4757-g3fc0089c afd12e232ec0 13729483a565 2023-07-14T16:15:11.316 INFO:teuthology.orchestra.run.smithi062.stdout:mds.e smithi081 running (101s) 5s ago 100s 17.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 a6ec5512115b 2023-07-14T16:15:11.316 INFO:teuthology.orchestra.run.smithi062.stdout:mds.f smithi164 running (97s) 5s ago 97s 16.9M - 18.0.0-4757-g3fc0089c afd12e232ec0 ff33534edad2 2023-07-14T16:15:11.316 INFO:teuthology.orchestra.run.smithi062.stdout:mds.g smithi062 running (85s) 4s ago 84s 13.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 94ae63251fb0 2023-07-14T16:15:11.316 INFO:teuthology.orchestra.run.smithi062.stdout:mds.h smithi081 running (91s) 5s ago 91s 18.2M - 18.0.0-4757-g3fc0089c afd12e232ec0 f35b473a3f54 2023-07-14T16:15:11.316 INFO:teuthology.orchestra.run.smithi062.stdout:mds.i smithi164 running (88s) 5s ago 87s 17.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 9b9ef37605d2 2023-07-14T16:15:11.316 INFO:teuthology.orchestra.run.smithi062.stdout:mds.j smithi062 running (107s) 4s ago 107s 13.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 3c59f72942cf 2023-07-14T16:15:11.316 INFO:teuthology.orchestra.run.smithi062.stdout:mds.k smithi081 running (110s) 5s ago 110s 13.8M - 18.0.0-4757-g3fc0089c afd12e232ec0 c760b13c1d11 2023-07-14T16:15:11.317 INFO:teuthology.orchestra.run.smithi062.stdout:mds.l smithi164 running (113s) 5s ago 113s 17.0M - 18.0.0-4757-g3fc0089c afd12e232ec0 6d81e6da785a 2023-07-14T16:15:11.317 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi062 *:9283,8765 running (9m) 4s ago 9m 522M - 18.0.0-4757-g3fc0089c afd12e232ec0 e1ef15f63302 2023-07-14T16:15:11.317 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi081 *:8443,8765 running (7m) 5s ago 7m 449M - 18.0.0-4757-g3fc0089c afd12e232ec0 5ad013c04f61 2023-07-14T16:15:11.317 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.z smithi164 *:8443,8765 running (7m) 5s ago 7m 449M - 18.0.0-4757-g3fc0089c afd12e232ec0 a6854e882f80 2023-07-14T16:15:11.317 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (9m) 4s ago 9m 98.4M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 cdb498e2ac42 2023-07-14T16:15:11.317 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi081 running (7m) 5s ago 7m 64.7M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 5ac89713accb 2023-07-14T16:15:11.317 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi164 running (8m) 5s ago 8m 63.5M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 833b2587957d 2023-07-14T16:15:11.317 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (7m) 4s ago 7m 68.6M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 6375673afeb5 2023-07-14T16:15:11.318 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi081 running (6m) 5s ago 6m 77.1M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 b04346537cc7 2023-07-14T16:15:11.318 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi164 running (6m) 5s ago 6m 72.0M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 f899a64c00dd 2023-07-14T16:15:11.318 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (5m) 4s ago 5m 62.3M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 926773de5558 2023-07-14T16:15:11.318 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi081 running (5m) 5s ago 5m 61.0M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 1b2b10516ec0 2023-07-14T16:15:11.318 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi164 running (5m) 5s ago 5m 64.9M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 e8c644d3f321 2023-07-14T16:15:11.318 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi062 running (4m) 4s ago 4m 57.9M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 315b3e5b670c 2023-07-14T16:15:11.318 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi081 running (4m) 5s ago 4m 62.4M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 6c959b7bd064 2023-07-14T16:15:11.318 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 smithi164 running (3m) 5s ago 3m 59.6M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 c8335d51be7b 2023-07-14T16:15:11.319 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 smithi062 running (3m) 4s ago 3m 54.9M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 8a0d68ec83b1 2023-07-14T16:15:11.319 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 smithi081 running (2m) 5s ago 2m 60.9M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 a44d4fbdfa40 2023-07-14T16:15:11.319 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 smithi164 running (2m) 5s ago 2m 50.7M 4316M 18.0.0-4757-g3fc0089c afd12e232ec0 24313da9c00c 2023-07-14T16:15:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81480a1260 msgr2=0x7f81480a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81480a1260 0x7f81480a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8160100bf0 tx=0x7f815000b410 comp rx=0 tx=0).stop 2023-07-14T16:15:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 msgr2=0x7f8160197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:11.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 0x7f8160197ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f815400c010 tx=0x7f8154003a50 comp rx=0 tx=0).stop 2023-07-14T16:15:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 shutdown_connections 2023-07-14T16:15:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8160101720 0x7f816019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81480a1260 0x7f81480a3710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81600ffb90 0x7f8160197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 --2- 172.21.15.62:0/2111197711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8160100d90 0x7f8160197ed0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 >> 172.21.15.62:0/2111197711 conn(0x7f81600fb1a0 msgr2=0x7f81600fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:11.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 shutdown_connections 2023-07-14T16:15:11.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:11.338+0000 7f81655d7700 1 -- 172.21.15.62:0/2111197711 wait complete. 2023-07-14T16:15:11.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-14T16:15:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:15:12.869 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:12 smithi062 ceph-mon[95543]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:12.869 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:12 smithi062 ceph-mon[95543]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:12 smithi081 ceph-mon[127795]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:12.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:12 smithi081 ceph-mon[127795]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:13.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:12 smithi164 ceph-mon[97427]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:13.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:12 smithi164 ceph-mon[97427]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.103+0000 7ff6c6a70700 1 -- 172.21.15.62:0/1380494815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 msgr2=0x7ff6c00733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.103+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/1380494815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 0x7ff6c00733e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009990 tx=0x7ff6b0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.103+0000 7ff6c6a70700 1 -- 172.21.15.62:0/1380494815 shutdown_connections 2023-07-14T16:15:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.103+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/1380494815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c0107a60 0x7ff6c0109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.103+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/1380494815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c0074540 0x7ff6c00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.103+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/1380494815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 0x7ff6c00733e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.103+0000 7ff6c6a70700 1 -- 172.21.15.62:0/1380494815 >> 172.21.15.62:0/1380494815 conn(0x7ff6c00fb1a0 msgr2=0x7ff6c00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.104+0000 7ff6c6a70700 1 -- 172.21.15.62:0/1380494815 shutdown_connections 2023-07-14T16:15:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.104+0000 7ff6c6a70700 1 -- 172.21.15.62:0/1380494815 wait complete. 2023-07-14T16:15:14.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.105+0000 7ff6c6a70700 1 Processor -- start 2023-07-14T16:15:14.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.105+0000 7ff6c6a70700 1 -- start start 2023-07-14T16:15:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.105+0000 7ff6c6a70700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 0x7ff6c0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.105+0000 7ff6c6a70700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c0074540 0x7ff6c0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.105+0000 7ff6c6a70700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c0107a60 0x7ff6c019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.105+0000 7ff6c6a70700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019f5a0 con 0x7ff6c0072f70 2023-07-14T16:15:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.105+0000 7ff6c6a70700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019f6e0 con 0x7ff6c0107a60 2023-07-14T16:15:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.105+0000 7ff6c6a70700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019f870 con 0x7ff6c0074540 2023-07-14T16:15:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 0x7ff6c0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:14.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 0x7ff6c0197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60668/0 (socket says 172.21.15.62:60668) 2023-07-14T16:15:14.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 -- 172.21.15.62:0/4220744130 learned_addr learned my addr 172.21.15.62:0/4220744130 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:14.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6b7fff700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c0074540 0x7ff6c0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:14.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c500d700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c0107a60 0x7ff6c019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:14.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 -- 172.21.15.62:0/4220744130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c0074540 msgr2=0x7ff6c0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:14.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c0074540 0x7ff6c0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 -- 172.21.15.62:0/4220744130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c0107a60 msgr2=0x7ff6c019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c0107a60 0x7ff6c019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 -- 172.21.15.62:0/4220744130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b00095f0 con 0x7ff6c0072f70 2023-07-14T16:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6c480c700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 0x7ff6c0197a10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00e9d0 tx=0x7ff6bc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.106+0000 7ff6b5ffb700 1 -- 172.21.15.62:0/4220744130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc009790 con 0x7ff6c0072f70 2023-07-14T16:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.107+0000 7ff6b5ffb700 1 -- 172.21.15.62:0/4220744130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6bc00cbd0 con 0x7ff6c0072f70 2023-07-14T16:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.107+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c019fb50 con 0x7ff6c0072f70 2023-07-14T16:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.107+0000 7ff6b5ffb700 1 -- 172.21.15.62:0/4220744130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc018680 con 0x7ff6c0072f70 2023-07-14T16:15:14.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.107+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c01a0050 con 0x7ff6c0072f70 2023-07-14T16:15:14.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.108+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c0106110 con 0x7ff6c0072f70 2023-07-14T16:15:14.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.110+0000 7ff6b5ffb700 1 -- 172.21.15.62:0/4220744130 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff6bc00cd40 con 0x7ff6c0072f70 2023-07-14T16:15:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.111+0000 7ff6b5ffb700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff6a80a11e0 0x7ff6a80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.111+0000 7ff6b5ffb700 1 -- 172.21.15.62:0/4220744130 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff6bc014070 con 0x7ff6c0072f70 2023-07-14T16:15:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.111+0000 7ff6b7fff700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff6a80a11e0 0x7ff6a80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:14.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.112+0000 7ff6b7fff700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff6a80a11e0 0x7ff6a80a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0006010 tx=0x7ff6b000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:14.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.115+0000 7ff6b5ffb700 1 -- 172.21.15.62:0/4220744130 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff6bc051790 con 0x7ff6c0072f70 2023-07-14T16:15:14.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.330+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6c0060950 con 0x7ff6a80a11e0 2023-07-14T16:15:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.336+0000 7ff6b5ffb700 1 -- 172.21.15.62:0/4220744130 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7ff6c0060950 con 0x7ff6a80a11e0 2023-07-14T16:15:14.314 INFO:teuthology.orchestra.run.smithi062.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-14T16:15:14.315 INFO:teuthology.orchestra.run.smithi062.stdout:mds.all 12/12 8s ago 2m smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi081=b;smithi081=e;smithi081=h;smithi081=k;smithi164=c;smithi164=f;smithi164=i;smithi164=l;count:12 2023-07-14T16:15:14.315 INFO:teuthology.orchestra.run.smithi062.stdout:mgr 3/3 8s ago 7m smithi062=x;smithi081=y;smithi164=z;count:3 2023-07-14T16:15:14.315 INFO:teuthology.orchestra.run.smithi062.stdout:mon 3/3 8s ago 8m smithi062:172.21.15.62=a;smithi081:172.21.15.81=b;smithi164:172.21.15.164=c;count:3 2023-07-14T16:15:14.316 INFO:teuthology.orchestra.run.smithi062.stdout:osd 12 8s ago - 2023-07-14T16:15:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff6a80a11e0 msgr2=0x7ff6a80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff6a80a11e0 0x7ff6a80a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0006010 tx=0x7ff6b000b260 comp rx=0 tx=0).stop 2023-07-14T16:15:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 msgr2=0x7ff6c0197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 0x7ff6c0197a10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00e9d0 tx=0x7ff6bc00ece0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 shutdown_connections 2023-07-14T16:15:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff6a80a11e0 0x7ff6a80a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6c0107a60 0x7ff6c019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6c0072f70 0x7ff6c0197a10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 --2- 172.21.15.62:0/4220744130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6c0074540 0x7ff6c0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 >> 172.21.15.62:0/4220744130 conn(0x7ff6c00fb1a0 msgr2=0x7ff6c00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 shutdown_connections 2023-07-14T16:15:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:14.338+0000 7ff6c6a70700 1 -- 172.21.15.62:0/4220744130 wait complete. 2023-07-14T16:15:14.851 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-14T16:15:14.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:14 smithi062 ceph-mon[95543]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:14 smithi081 ceph-mon[127795]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:14 smithi164 ceph-mon[97427]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:15.580 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:15:15.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:15 smithi062 ceph-mon[95543]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:15.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:15 smithi081 ceph-mon[127795]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:16.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:15 smithi164 ceph-mon[97427]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:16.878 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:16 smithi062 ceph-mon[95543]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:16.906 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:16 smithi081 ceph-mon[127795]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:17.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:16 smithi164 ceph-mon[97427]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:15:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 -- 172.21.15.62:0/1883569274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934101720 msgr2=0x7f6934107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 --2- 172.21.15.62:0/1883569274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934101720 0x7f6934107fe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b0a0 tx=0x7f692c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 -- 172.21.15.62:0/1883569274 shutdown_connections 2023-07-14T16:15:17.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 --2- 172.21.15.62:0/1883569274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6934101720 0x7f6934107fe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 --2- 172.21.15.62:0/1883569274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69340ffb90 0x7f69340fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 --2- 172.21.15.62:0/1883569274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934100d90 0x7f69341011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 -- 172.21.15.62:0/1883569274 >> 172.21.15.62:0/1883569274 conn(0x7f69340fb1a0 msgr2=0x7f69340fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:17.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 -- 172.21.15.62:0/1883569274 shutdown_connections 2023-07-14T16:15:17.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.231+0000 7f6938a11700 1 -- 172.21.15.62:0/1883569274 wait complete. 2023-07-14T16:15:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.232+0000 7f6938a11700 1 Processor -- start 2023-07-14T16:15:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.232+0000 7f6938a11700 1 -- start start 2023-07-14T16:15:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.232+0000 7f6938a11700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69340ffb90 0x7f6934197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.232+0000 7f6938a11700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6934100d90 0x7f6934197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.232+0000 7f6938a11700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934101720 0x7f693419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.232+0000 7f6938a11700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419f5f0 con 0x7f69340ffb90 2023-07-14T16:15:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.232+0000 7f6938a11700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419f730 con 0x7f6934101720 2023-07-14T16:15:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.232+0000 7f6938a11700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693419f8c0 con 0x7f6934100d90 2023-07-14T16:15:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69340ffb90 0x7f6934197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69340ffb90 0x7f6934197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60694/0 (socket says 172.21.15.62:60694) 2023-07-14T16:15:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 -- 172.21.15.62:0/1243307816 learned_addr learned my addr 172.21.15.62:0/1243307816 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f6931d9b700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6934100d90 0x7f6934197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f6932d9d700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934101720 0x7f693419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:17.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 -- 172.21.15.62:0/1243307816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6934100d90 msgr2=0x7f6934197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:17.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6934100d90 0x7f6934197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 -- 172.21.15.62:0/1243307816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934101720 msgr2=0x7f693419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:17.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934101720 0x7f693419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 -- 172.21.15.62:0/1243307816 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c009d00 con 0x7f69340ffb90 2023-07-14T16:15:17.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f6932d9d700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934101720 0x7f693419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:17.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.233+0000 7f693259c700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69340ffb90 0x7f6934197a60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f692400d9b0 tx=0x7f692400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:17.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.234+0000 7f69237fe700 1 -- 172.21.15.62:0/1243307816 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924004020 con 0x7f69340ffb90 2023-07-14T16:15:17.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.234+0000 7f69237fe700 1 -- 172.21.15.62:0/1243307816 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6924009730 con 0x7f69340ffb90 2023-07-14T16:15:17.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.234+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693419fba0 con 0x7f69340ffb90 2023-07-14T16:15:17.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.234+0000 7f69237fe700 1 -- 172.21.15.62:0/1243307816 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240039c0 con 0x7f69340ffb90 2023-07-14T16:15:17.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.234+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69341a00a0 con 0x7f69340ffb90 2023-07-14T16:15:17.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.235+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6934192120 con 0x7f69340ffb90 2023-07-14T16:15:17.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.237+0000 7f69237fe700 1 -- 172.21.15.62:0/1243307816 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f69240098a0 con 0x7f69340ffb90 2023-07-14T16:15:17.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.238+0000 7f69237fe700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f691c0a11e0 0x7f691c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:17.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.239+0000 7f69237fe700 1 -- 172.21.15.62:0/1243307816 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69240cc7a0 con 0x7f69340ffb90 2023-07-14T16:15:17.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.239+0000 7f6931d9b700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f691c0a11e0 0x7f691c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:17.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.239+0000 7f6931d9b700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f691c0a11e0 0x7f691c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f692800ac50 tx=0x7f692800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:17.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.242+0000 7f69237fe700 1 -- 172.21.15.62:0/1243307816 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6924095d40 con 0x7f69340ffb90 2023-07-14T16:15:17.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.455+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6934060950 con 0x7f691c0a11e0 2023-07-14T16:15:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.456+0000 7f69237fe700 1 -- 172.21.15.62:0/1243307816 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f6934060950 con 0x7f691c0a11e0 2023-07-14T16:15:17.434 INFO:teuthology.orchestra.run.smithi062.stdout:HOST ADDR LABELS STATUS 2023-07-14T16:15:17.434 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 172.21.15.62 2023-07-14T16:15:17.434 INFO:teuthology.orchestra.run.smithi062.stdout:smithi081 172.21.15.81 2023-07-14T16:15:17.434 INFO:teuthology.orchestra.run.smithi062.stdout:smithi164 172.21.15.164 2023-07-14T16:15:17.434 INFO:teuthology.orchestra.run.smithi062.stdout:3 hosts in cluster 2023-07-14T16:15:17.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f691c0a11e0 msgr2=0x7f691c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:17.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f691c0a11e0 0x7f691c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f692800ac50 tx=0x7f692800a380 comp rx=0 tx=0).stop 2023-07-14T16:15:17.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69340ffb90 msgr2=0x7f6934197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:17.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69340ffb90 0x7f6934197a60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f692400d9b0 tx=0x7f692400dcc0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 shutdown_connections 2023-07-14T16:15:17.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f691c0a11e0 0x7f691c0a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6934101720 0x7f693419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69340ffb90 0x7f6934197a60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.458+0000 7f6938a11700 1 --2- 172.21.15.62:0/1243307816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6934100d90 0x7f6934197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:17.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.459+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 >> 172.21.15.62:0/1243307816 conn(0x7f69340fb1a0 msgr2=0x7f6934106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:17.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.459+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 shutdown_connections 2023-07-14T16:15:17.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:17.459+0000 7f6938a11700 1 -- 172.21.15.62:0/1243307816 wait complete. 2023-07-14T16:15:17.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-14T16:15:18.739 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:15:18.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:18 smithi081 ceph-mon[127795]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:18.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:18 smithi081 ceph-mon[127795]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:15:19.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:18 smithi062 ceph-mon[95543]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:19.011 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:18 smithi062 ceph-mon[95543]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:15:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:18 smithi164 ceph-mon[97427]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:18 smithi164 ceph-mon[97427]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:15:20.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 -- 172.21.15.62:0/421684826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 msgr2=0x7f88c4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:20.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 --2- 172.21.15.62:0/421684826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 0x7f88c4107fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00b0a0 tx=0x7f88bc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 -- 172.21.15.62:0/421684826 shutdown_connections 2023-07-14T16:15:20.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 --2- 172.21.15.62:0/421684826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 0x7f88c4107fe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 --2- 172.21.15.62:0/421684826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c40ffb90 0x7f88c40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 --2- 172.21.15.62:0/421684826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88c4100d90 0x7f88c41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 -- 172.21.15.62:0/421684826 >> 172.21.15.62:0/421684826 conn(0x7f88c40fb1a0 msgr2=0x7f88c40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:20.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 -- 172.21.15.62:0/421684826 shutdown_connections 2023-07-14T16:15:20.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.288+0000 7f88ca663700 1 -- 172.21.15.62:0/421684826 wait complete. 2023-07-14T16:15:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.289+0000 7f88ca663700 1 Processor -- start 2023-07-14T16:15:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.289+0000 7f88ca663700 1 -- start start 2023-07-14T16:15:20.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88ca663700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c40ffb90 0x7f88c41958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:20.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88ca663700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88c4100d90 0x7f88c4195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:20.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88ca663700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 0x7f88c419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:20.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88ca663700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c419d480 con 0x7f88c4101720 2023-07-14T16:15:20.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88ca663700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c419d5c0 con 0x7f88c4100d90 2023-07-14T16:15:20.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88ca663700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c419d750 con 0x7f88c40ffb90 2023-07-14T16:15:20.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c8c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 0x7f88c419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:20.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c8c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 0x7f88c419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60722/0 (socket says 172.21.15.62:60722) 2023-07-14T16:15:20.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c8c00700 1 -- 172.21.15.62:0/2702464940 learned_addr learned my addr 172.21.15.62:0/2702464940 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:20.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c3fff700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c40ffb90 0x7f88c41958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:20.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c37fe700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88c4100d90 0x7f88c4195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:20.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c8c00700 1 -- 172.21.15.62:0/2702464940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c40ffb90 msgr2=0x7f88c41958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:20.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c8c00700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c40ffb90 0x7f88c41958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c8c00700 1 -- 172.21.15.62:0/2702464940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88c4100d90 msgr2=0x7f88c4195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:20.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c8c00700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88c4100d90 0x7f88c4195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.290+0000 7f88c8c00700 1 -- 172.21.15.62:0/2702464940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc009d00 con 0x7f88c4101720 2023-07-14T16:15:20.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.291+0000 7f88c8c00700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 0x7f88c419ceb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009fd0 tx=0x7f88bc004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:20.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.291+0000 7f88c17fa700 1 -- 172.21.15.62:0/2702464940 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc00e040 con 0x7f88c4101720 2023-07-14T16:15:20.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.291+0000 7f88c17fa700 1 -- 172.21.15.62:0/2702464940 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88bc0094d0 con 0x7f88c4101720 2023-07-14T16:15:20.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.291+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c419d9d0 con 0x7f88c4101720 2023-07-14T16:15:20.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.291+0000 7f88c17fa700 1 -- 172.21.15.62:0/2702464940 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc004280 con 0x7f88c4101720 2023-07-14T16:15:20.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.291+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c419de70 con 0x7f88c4101720 2023-07-14T16:15:20.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.292+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c41070f0 con 0x7f88c4101720 2023-07-14T16:15:20.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.294+0000 7f88c17fa700 1 -- 172.21.15.62:0/2702464940 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f88bc01d690 con 0x7f88c4101720 2023-07-14T16:15:20.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.296+0000 7f88c17fa700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88ac0a1150 0x7f88ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:20.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.296+0000 7f88c17fa700 1 -- 172.21.15.62:0/2702464940 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f88bc0ce570 con 0x7f88c4101720 2023-07-14T16:15:20.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.296+0000 7f88c3fff700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88ac0a1150 0x7f88ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:20.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.296+0000 7f88c3fff700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88ac0a1150 0x7f88ac0a3600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88b4006fd0 tx=0x7f88b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.299+0000 7f88c17fa700 1 -- 172.21.15.62:0/2702464940 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f88bc097d60 con 0x7f88c4101720 2023-07-14T16:15:20.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.513+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f88c4060950 con 0x7f88ac0a1150 2023-07-14T16:15:20.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.515+0000 7f88c17fa700 1 -- 172.21.15.62:0/2702464940 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f88c4060950 con 0x7f88ac0a1150 2023-07-14T16:15:20.492 INFO:teuthology.orchestra.run.smithi062.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-14T16:15:20.493 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 3m ago LVM detected, locked 2023-07-14T16:15:20.493 INFO:teuthology.orchestra.run.smithi062.stdout:smithi081 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400N7400BGN 372G No 2m ago LVM detected, locked 2023-07-14T16:15:20.493 INFO:teuthology.orchestra.run.smithi062.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 2m ago LVM detected, locked 2023-07-14T16:15:20.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.517+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88ac0a1150 msgr2=0x7f88ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:20.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.517+0000 7f88ca663700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88ac0a1150 0x7f88ac0a3600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88b4006fd0 tx=0x7f88b4008040 comp rx=0 tx=0).stop 2023-07-14T16:15:20.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.517+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 msgr2=0x7f88c419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:20.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.517+0000 7f88ca663700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 0x7f88c419ceb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009fd0 tx=0x7f88bc004d30 comp rx=0 tx=0).stop 2023-07-14T16:15:20.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.518+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 shutdown_connections 2023-07-14T16:15:20.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.518+0000 7f88ca663700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c4101720 0x7f88c419ceb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.518+0000 7f88ca663700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88c40ffb90 0x7f88c41958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.518+0000 7f88ca663700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88ac0a1150 0x7f88ac0a3600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.518+0000 7f88ca663700 1 --2- 172.21.15.62:0/2702464940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88c4100d90 0x7f88c4195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:20.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.518+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 >> 172.21.15.62:0/2702464940 conn(0x7f88c40fb1a0 msgr2=0x7f88c4104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:20.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.518+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 shutdown_connections 2023-07-14T16:15:20.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:20.518+0000 7f88ca663700 1 -- 172.21.15.62:0/2702464940 wait complete. 2023-07-14T16:15:20.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:20 smithi062 ceph-mon[95543]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:20.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:20 smithi062 ceph-mon[95543]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-07-14T16:15:20.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:20 smithi081 ceph-mon[127795]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:20.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:20 smithi081 ceph-mon[127795]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-07-14T16:15:20.924 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T16:15:20.937 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2023-07-14T16:15:20.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-14T16:15:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:20 smithi164 ceph-mon[97427]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:21.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:20 smithi164 ceph-mon[97427]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-07-14T16:15:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:15:21.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:21 smithi081 ceph-mon[127795]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:21.943 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:21 smithi062 ceph-mon[95543]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:21 smithi164 ceph-mon[97427]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T16:15:22.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:22 smithi062 ceph-mon[95543]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T16:15:22.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:22 smithi081 ceph-mon[127795]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T16:15:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:22 smithi164 ceph-mon[97427]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T16:15:23.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 -- 172.21.15.62:0/1942374722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 msgr2=0x7f3ca80fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:23.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 --2- 172.21.15.62:0/1942374722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 0x7f3ca80fffa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009990 tx=0x7f3c98009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 -- 172.21.15.62:0/1942374722 shutdown_connections 2023-07-14T16:15:23.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 --2- 172.21.15.62:0/1942374722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ca8101720 0x7f3ca8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 --2- 172.21.15.62:0/1942374722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 0x7f3ca80fffa0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 --2- 172.21.15.62:0/1942374722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca8100d90 0x7f3ca81011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 -- 172.21.15.62:0/1942374722 >> 172.21.15.62:0/1942374722 conn(0x7f3ca80fb1a0 msgr2=0x7f3ca80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:23.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 -- 172.21.15.62:0/1942374722 shutdown_connections 2023-07-14T16:15:23.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.114+0000 7f3cadd87700 1 -- 172.21.15.62:0/1942374722 wait complete. 2023-07-14T16:15:23.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.115+0000 7f3cadd87700 1 Processor -- start 2023-07-14T16:15:23.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.115+0000 7f3cadd87700 1 -- start start 2023-07-14T16:15:23.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3cadd87700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 0x7f3ca81979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:23.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3cadd87700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca8100d90 0x7f3ca8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:23.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3cadd87700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ca8101720 0x7f3ca819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:23.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3cadd87700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca819f560 con 0x7f3ca80ffb90 2023-07-14T16:15:23.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3cadd87700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca819f6a0 con 0x7f3ca8100d90 2023-07-14T16:15:23.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3cadd87700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca819f830 con 0x7f3ca8101720 2023-07-14T16:15:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 0x7f3ca81979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 0x7f3ca81979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56348/0 (socket says 172.21.15.62:56348) 2023-07-14T16:15:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca77fe700 1 -- 172.21.15.62:0/277213190 learned_addr learned my addr 172.21.15.62:0/277213190 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca7fff700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ca8101720 0x7f3ca819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:23.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca6ffd700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca8100d90 0x7f3ca8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca77fe700 1 -- 172.21.15.62:0/277213190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ca8101720 msgr2=0x7f3ca819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca77fe700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ca8101720 0x7f3ca819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca77fe700 1 -- 172.21.15.62:0/277213190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca8100d90 msgr2=0x7f3ca8197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca77fe700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca8100d90 0x7f3ca8197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca77fe700 1 -- 172.21.15.62:0/277213190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c980095f0 con 0x7f3ca80ffb90 2023-07-14T16:15:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.116+0000 7f3ca6ffd700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca8100d90 0x7f3ca8197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.117+0000 7f3ca77fe700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 0x7f3ca81979d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098e0 tx=0x7f3c98003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.117+0000 7f3ca4ff9700 1 -- 172.21.15.62:0/277213190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9801d070 con 0x7f3ca80ffb90 2023-07-14T16:15:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.117+0000 7f3ca4ff9700 1 -- 172.21.15.62:0/277213190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c9800bd70 con 0x7f3ca80ffb90 2023-07-14T16:15:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.117+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca819fab0 con 0x7f3ca80ffb90 2023-07-14T16:15:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.117+0000 7f3ca4ff9700 1 -- 172.21.15.62:0/277213190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98022c60 con 0x7f3ca80ffb90 2023-07-14T16:15:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.117+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca819ff20 con 0x7f3ca80ffb90 2023-07-14T16:15:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.118+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca810b200 con 0x7f3ca80ffb90 2023-07-14T16:15:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.120+0000 7f3ca4ff9700 1 -- 172.21.15.62:0/277213190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3c98022470 con 0x7f3ca80ffb90 2023-07-14T16:15:23.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.121+0000 7f3ca4ff9700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3c900a11d0 0x7f3c900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:23.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.121+0000 7f3ca4ff9700 1 -- 172.21.15.62:0/277213190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3c980cdf40 con 0x7f3ca80ffb90 2023-07-14T16:15:23.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.122+0000 7f3ca6ffd700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3c900a11d0 0x7f3c900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:23.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.122+0000 7f3ca6ffd700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3c900a11d0 0x7f3c900a3680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005950 tx=0x7f3c9c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:23.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.125+0000 7f3ca4ff9700 1 -- 172.21.15.62:0/277213190 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3c980976d0 con 0x7f3ca80ffb90 2023-07-14T16:15:23.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.377+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3ca810b3f0 con 0x7f3ca80ffb90 2023-07-14T16:15:23.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.377+0000 7f3ca4ff9700 1 -- 172.21.15.62:0/277213190 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2886 (secure 0 0 0) 0x7f3c9809acf0 con 0x7f3ca80ffb90 2023-07-14T16:15:23.355 INFO:teuthology.orchestra.run.smithi062.stdout:e23 2023-07-14T16:15:23.355 INFO:teuthology.orchestra.run.smithi062.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T16:15:23.355 INFO:teuthology.orchestra.run.smithi062.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T16:15:23.355 INFO:teuthology.orchestra.run.smithi062.stdout:legacy client fscid: 1 2023-07-14T16:15:23.355 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:23.355 INFO:teuthology.orchestra.run.smithi062.stdout:Filesystem 'cephfs' (1) 2023-07-14T16:15:23.355 INFO:teuthology.orchestra.run.smithi062.stdout:fs_name cephfs 2023-07-14T16:15:23.356 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 23 2023-07-14T16:15:23.356 INFO:teuthology.orchestra.run.smithi062.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-14T16:15:23.356 INFO:teuthology.orchestra.run.smithi062.stdout:created 2023-07-14T16:13:58.805235+0000 2023-07-14T16:15:23.356 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2023-07-14T16:14:04.590671+0000 2023-07-14T16:15:23.356 INFO:teuthology.orchestra.run.smithi062.stdout:tableserver 0 2023-07-14T16:15:23.356 INFO:teuthology.orchestra.run.smithi062.stdout:root 0 2023-07-14T16:15:23.356 INFO:teuthology.orchestra.run.smithi062.stdout:session_timeout 300 2023-07-14T16:15:23.357 INFO:teuthology.orchestra.run.smithi062.stdout:session_autoclose 300 2023-07-14T16:15:23.357 INFO:teuthology.orchestra.run.smithi062.stdout:max_file_size 1099511627776 2023-07-14T16:15:23.357 INFO:teuthology.orchestra.run.smithi062.stdout:max_xattr_size 65536 2023-07-14T16:15:23.357 INFO:teuthology.orchestra.run.smithi062.stdout:required_client_features {} 2023-07-14T16:15:23.357 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure 0 2023-07-14T16:15:23.357 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure_osd_epoch 0 2023-07-14T16:15:23.357 INFO:teuthology.orchestra.run.smithi062.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T16:15:23.357 INFO:teuthology.orchestra.run.smithi062.stdout:max_mds 3 2023-07-14T16:15:23.358 INFO:teuthology.orchestra.run.smithi062.stdout:in 0,1,2 2023-07-14T16:15:23.358 INFO:teuthology.orchestra.run.smithi062.stdout:up {0=24602,1=24565,2=24512} 2023-07-14T16:15:23.358 INFO:teuthology.orchestra.run.smithi062.stdout:failed 2023-07-14T16:15:23.358 INFO:teuthology.orchestra.run.smithi062.stdout:damaged 2023-07-14T16:15:23.358 INFO:teuthology.orchestra.run.smithi062.stdout:stopped 2023-07-14T16:15:23.358 INFO:teuthology.orchestra.run.smithi062.stdout:data_pools [3] 2023-07-14T16:15:23.358 INFO:teuthology.orchestra.run.smithi062.stdout:metadata_pool 2 2023-07-14T16:15:23.358 INFO:teuthology.orchestra.run.smithi062.stdout:inline_data disabled 2023-07-14T16:15:23.359 INFO:teuthology.orchestra.run.smithi062.stdout:balancer 2023-07-14T16:15:23.359 INFO:teuthology.orchestra.run.smithi062.stdout:bal_rank_mask -1 2023-07-14T16:15:23.359 INFO:teuthology.orchestra.run.smithi062.stdout:standby_count_wanted 0 2023-07-14T16:15:23.359 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b{0:24602} state up:active seq 3 addr [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.359 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.h{0:24551} state up:standby-replay seq 1 addr [v2:172.21.15.81:6836/3627777030,v1:172.21.15.81:6837/3627777030] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.359 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.i{1:24565} state up:active seq 6 addr [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.359 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.f{1:24526} state up:standby-replay seq 1 addr [v2:172.21.15.164:6834/1558258971,v1:172.21.15.164:6835/1558258971] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.359 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.e{2:24512} state up:active seq 10 addr [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.360 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.l{2:24475} state up:standby-replay seq 1 addr [v2:172.21.15.164:6832/903698978,v1:172.21.15.164:6833/903698978] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.360 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:23.360 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:23.360 INFO:teuthology.orchestra.run.smithi062.stdout:Standby daemons: 2023-07-14T16:15:23.360 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:23.360 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.j{-1:14613} state up:standby seq 1 addr [v2:172.21.15.62:6834/3850024319,v1:172.21.15.62:6835/3850024319] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.360 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d{-1:14625} state up:standby seq 1 addr [v2:172.21.15.62:6836/1601453648,v1:172.21.15.62:6837/1601453648] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.361 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c{-1:14667} state up:standby seq 1 addr [v2:172.21.15.164:6836/194603776,v1:172.21.15.164:6837/194603776] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.361 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.g{-1:14709} state up:standby seq 1 addr [v2:172.21.15.62:6838/3506657450,v1:172.21.15.62:6839/3506657450] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.361 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a{-1:14721} state up:standby seq 1 addr [v2:172.21.15.62:6840/33062382,v1:172.21.15.62:6841/33062382] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.361 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.k{-1:24470} state up:standby seq 1 addr [v2:172.21.15.81:6832/2570105256,v1:172.21.15.81:6833/2570105256] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T16:15:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3c900a11d0 msgr2=0x7f3c900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3c900a11d0 0x7f3c900a3680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005950 tx=0x7f3c9c0058e0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 msgr2=0x7f3ca81979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 0x7f3ca81979d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098e0 tx=0x7f3c98003a50 comp rx=0 tx=0).stop 2023-07-14T16:15:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 shutdown_connections 2023-07-14T16:15:23.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ca8101720 0x7f3ca819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ca80ffb90 0x7f3ca81979d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3c900a11d0 0x7f3c900a3680 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 --2- 172.21.15.62:0/277213190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ca8100d90 0x7f3ca8197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:23.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 >> 172.21.15.62:0/277213190 conn(0x7f3ca80fb1a0 msgr2=0x7f3ca80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:23.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 shutdown_connections 2023-07-14T16:15:23.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:23.380+0000 7f3cadd87700 1 -- 172.21.15.62:0/277213190 wait complete. 2023-07-14T16:15:23.366 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:15:23.749 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-14T16:15:23.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:23 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/277213190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T16:15:24.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:23 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/277213190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T16:15:24.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:23 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/277213190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T16:15:24.482 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:15:24.903 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:24 smithi062 ceph-mon[95543]: pgmap v350: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:15:24.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:24 smithi081 ceph-mon[127795]: pgmap v350: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:15:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:24 smithi164 ceph-mon[97427]: pgmap v350: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:15:26.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 -- 172.21.15.62:0/2578123158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c107a60 msgr2=0x7f431c109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:26.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 --2- 172.21.15.62:0/2578123158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c107a60 0x7f431c109ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f431400b0a0 tx=0x7f431400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 -- 172.21.15.62:0/2578123158 shutdown_connections 2023-07-14T16:15:26.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 --2- 172.21.15.62:0/2578123158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c107a60 0x7f431c109ea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 --2- 172.21.15.62:0/2578123158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f431c074540 0x7f431c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 --2- 172.21.15.62:0/2578123158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f431c072f70 0x7f431c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 -- 172.21.15.62:0/2578123158 >> 172.21.15.62:0/2578123158 conn(0x7f431c0fb1a0 msgr2=0x7f431c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:26.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 -- 172.21.15.62:0/2578123158 shutdown_connections 2023-07-14T16:15:26.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.120+0000 7f43237c5700 1 -- 172.21.15.62:0/2578123158 wait complete. 2023-07-14T16:15:26.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.121+0000 7f43237c5700 1 Processor -- start 2023-07-14T16:15:26.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.121+0000 7f43237c5700 1 -- start start 2023-07-14T16:15:26.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f43237c5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c072f70 0x7f431c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:26.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f43237c5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f431c074540 0x7f431c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:26.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f43237c5700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f431c107a60 0x7f431c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:26.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f43237c5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c19f6f0 con 0x7f431c072f70 2023-07-14T16:15:26.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f43237c5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c19f830 con 0x7f431c074540 2023-07-14T16:15:26.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f43237c5700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c19f9c0 con 0x7f431c107a60 2023-07-14T16:15:26.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321561700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c072f70 0x7f431c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:26.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321561700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c072f70 0x7f431c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56366/0 (socket says 172.21.15.62:56366) 2023-07-14T16:15:26.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321561700 1 -- 172.21.15.62:0/2415890604 learned_addr learned my addr 172.21.15.62:0/2415890604 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:26.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321d62700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f431c107a60 0x7f431c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:26.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4320d60700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f431c074540 0x7f431c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:26.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321561700 1 -- 172.21.15.62:0/2415890604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f431c107a60 msgr2=0x7f431c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:26.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321561700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f431c107a60 0x7f431c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321561700 1 -- 172.21.15.62:0/2415890604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f431c074540 msgr2=0x7f431c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:26.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321561700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f431c074540 0x7f431c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321561700 1 -- 172.21.15.62:0/2415890604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009d00 con 0x7f431c072f70 2023-07-14T16:15:26.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4321d62700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f431c107a60 0x7f431c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:26.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.122+0000 7f4320d60700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f431c074540 0x7f431c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:26.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.123+0000 7f4321561700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c072f70 0x7f431c197b60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f431800d980 tx=0x7f431800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:26.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.123+0000 7f43127fc700 1 -- 172.21.15.62:0/2415890604 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318009b40 con 0x7f431c072f70 2023-07-14T16:15:26.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.123+0000 7f43127fc700 1 -- 172.21.15.62:0/2415890604 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f431800b6e0 con 0x7f431c072f70 2023-07-14T16:15:26.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.123+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c19fca0 con 0x7f431c072f70 2023-07-14T16:15:26.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.123+0000 7f43127fc700 1 -- 172.21.15.62:0/2415890604 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431800f620 con 0x7f431c072f70 2023-07-14T16:15:26.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.123+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1a01a0 con 0x7f431c072f70 2023-07-14T16:15:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.124+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c106110 con 0x7f431c072f70 2023-07-14T16:15:26.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.126+0000 7f43127fc700 1 -- 172.21.15.62:0/2415890604 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4318010460 con 0x7f431c072f70 2023-07-14T16:15:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.127+0000 7f43127fc700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43080a11d0 0x7f43080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.127+0000 7f43127fc700 1 -- 172.21.15.62:0/2415890604 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f43180cc7e0 con 0x7f431c072f70 2023-07-14T16:15:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.128+0000 7f4320d60700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43080a11d0 0x7f43080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:26.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.128+0000 7f4320d60700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43080a11d0 0x7f43080a3680 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f430c0099d0 tx=0x7f430c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:26.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.131+0000 7f43127fc700 1 -- 172.21.15.62:0/2415890604 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4318095ee0 con 0x7f431c072f70 2023-07-14T16:15:26.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.342+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f431c0611c0 con 0x7f431c072f70 2023-07-14T16:15:26.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.343+0000 7f43127fc700 1 -- 172.21.15.62:0/2415890604 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5262 (secure 0 0 0) 0x7f4318099500 con 0x7f431c072f70 2023-07-14T16:15:26.320 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 88 2023-07-14T16:15:26.320 INFO:teuthology.orchestra.run.smithi062.stdout:fsid 1356e48a-2260-11ee-9b33-001a4aab830c 2023-07-14T16:15:26.321 INFO:teuthology.orchestra.run.smithi062.stdout:created 2023-07-14T16:05:22.767545+0000 2023-07-14T16:15:26.321 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2023-07-14T16:14:20.655639+0000 2023-07-14T16:15:26.321 INFO:teuthology.orchestra.run.smithi062.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-14T16:15:26.321 INFO:teuthology.orchestra.run.smithi062.stdout:crush_version 26 2023-07-14T16:15:26.321 INFO:teuthology.orchestra.run.smithi062.stdout:full_ratio 0.95 2023-07-14T16:15:26.322 INFO:teuthology.orchestra.run.smithi062.stdout:backfillfull_ratio 0.9 2023-07-14T16:15:26.322 INFO:teuthology.orchestra.run.smithi062.stdout:nearfull_ratio 0.85 2023-07-14T16:15:26.322 INFO:teuthology.orchestra.run.smithi062.stdout:require_min_compat_client luminous 2023-07-14T16:15:26.322 INFO:teuthology.orchestra.run.smithi062.stdout:min_compat_client luminous 2023-07-14T16:15:26.323 INFO:teuthology.orchestra.run.smithi062.stdout:require_osd_release reef 2023-07-14T16:15:26.323 INFO:teuthology.orchestra.run.smithi062.stdout:stretch_mode_enabled false 2023-07-14T16:15:26.323 INFO:teuthology.orchestra.run.smithi062.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-14T16:15:26.323 INFO:teuthology.orchestra.run.smithi062.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-14T16:15:26.323 INFO:teuthology.orchestra.run.smithi062.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-14T16:15:26.324 INFO:teuthology.orchestra.run.smithi062.stdout:max_osd 12 2023-07-14T16:15:26.324 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6802/2960730798,v1:172.21.15.62:6803/2960730798] [v2:172.21.15.62:6804/2960730798,v1:172.21.15.62:6805/2960730798] exists,up 5ae1fdd9-0e93-4148-af93-87c47080d59d 2023-07-14T16:15:26.324 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6800/3123830761,v1:172.21.15.81:6801/3123830761] [v2:172.21.15.81:6802/3123830761,v1:172.21.15.81:6803/3123830761] exists,up ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e 2023-07-14T16:15:26.324 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/3167529694,v1:172.21.15.164:6801/3167529694] [v2:172.21.15.164:6802/3167529694,v1:172.21.15.164:6803/3167529694] exists,up 831ec85f-142a-4e71-bd2c-9f2c1e8d8d43 2023-07-14T16:15:26.324 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6810/1003536176,v1:172.21.15.62:6811/1003536176] [v2:172.21.15.62:6812/1003536176,v1:172.21.15.62:6813/1003536176] exists,up 9b2e9feb-b1c8-471d-8ded-98528b1b5890 2023-07-14T16:15:26.324 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6808/550163990,v1:172.21.15.81:6809/550163990] [v2:172.21.15.81:6810/550163990,v1:172.21.15.81:6811/550163990] exists,up d6647685-0173-4443-b612-13675c942486 2023-07-14T16:15:26.324 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/1499523865,v1:172.21.15.164:6809/1499523865] [v2:172.21.15.164:6810/1499523865,v1:172.21.15.164:6811/1499523865] exists,up ee43949b-be2c-40e7-8efa-b8768a6d0165 2023-07-14T16:15:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6818/348099216,v1:172.21.15.62:6819/348099216] [v2:172.21.15.62:6820/348099216,v1:172.21.15.62:6821/348099216] exists,up 47c5b778-8a5b-4a01-9976-c0ddf8635724 2023-07-14T16:15:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6816/1584123297,v1:172.21.15.81:6817/1584123297] [v2:172.21.15.81:6818/1584123297,v1:172.21.15.81:6819/1584123297] exists,up 02e215e4-f68b-44d5-8172-836fed42ce7c 2023-07-14T16:15:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/1299247730,v1:172.21.15.164:6817/1299247730] [v2:172.21.15.164:6818/1299247730,v1:172.21.15.164:6819/1299247730] exists,up 30aea523-6f39-4ad1-8852-ec62c90e4c7a 2023-07-14T16:15:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6826/1938724436,v1:172.21.15.62:6827/1938724436] [v2:172.21.15.62:6828/1938724436,v1:172.21.15.62:6829/1938724436] exists,up 52d65846-0e8d-4cef-b984-1e4a5e97f5b3 2023-07-14T16:15:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6824/737826799,v1:172.21.15.81:6825/737826799] [v2:172.21.15.81:6826/737826799,v1:172.21.15.81:6827/737826799] exists,up 1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1 2023-07-14T16:15:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/1185509832,v1:172.21.15.164:6825/1185509832] [v2:172.21.15.164:6826/1185509832,v1:172.21.15.164:6827/1185509832] exists,up 7fe4fc0d-eb60-4df7-869d-1925ffddd634 2023-07-14T16:15:26.325 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.0 [8,4] 2023-07-14T16:15:26.326 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.a [2,11] 2023-07-14T16:15:26.326 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.f [8,11] 2023-07-14T16:15:26.326 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.1d [2,4] 2023-07-14T16:15:26.326 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.3 [4,9] 2023-07-14T16:15:26.326 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.11 [5,9] 2023-07-14T16:15:26.326 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.31 [4,8] 2023-07-14T16:15:26.327 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.3c [4,9] 2023-07-14T16:15:26.327 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/105803689 expires 2023-07-15T16:06:19.369987+0000 2023-07-14T16:15:26.327 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/1873685153 expires 2023-07-15T16:06:19.369987+0000 2023-07-14T16:15:26.327 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/1177875314 expires 2023-07-15T16:05:52.663390+0000 2023-07-14T16:15:26.327 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/105803689 expires 2023-07-15T16:06:19.369987+0000 2023-07-14T16:15:26.327 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/1315520653 expires 2023-07-15T16:05:52.663390+0000 2023-07-14T16:15:26.327 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/1177875314 expires 2023-07-15T16:05:52.663390+0000 2023-07-14T16:15:26.327 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/926428906 expires 2023-07-15T16:05:52.663390+0000 2023-07-14T16:15:26.328 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/804633476 expires 2023-07-15T16:06:19.369987+0000 2023-07-14T16:15:26.328 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3081660148 expires 2023-07-15T16:05:52.663390+0000 2023-07-14T16:15:26.328 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/743698528 expires 2023-07-15T16:06:19.369987+0000 2023-07-14T16:15:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43080a11d0 msgr2=0x7f43080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43080a11d0 0x7f43080a3680 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f430c0099d0 tx=0x7f430c008040 comp rx=0 tx=0).stop 2023-07-14T16:15:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c072f70 msgr2=0x7f431c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c072f70 0x7f431c197b60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f431800d980 tx=0x7f431800dd40 comp rx=0 tx=0).stop 2023-07-14T16:15:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 shutdown_connections 2023-07-14T16:15:26.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f431c107a60 0x7f431c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f431c072f70 0x7f431c197b60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43080a11d0 0x7f43080a3680 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 --2- 172.21.15.62:0/2415890604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f431c074540 0x7f431c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:26.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.345+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 >> 172.21.15.62:0/2415890604 conn(0x7f431c0fb1a0 msgr2=0x7f431c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:26.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.346+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 shutdown_connections 2023-07-14T16:15:26.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:26.346+0000 7f43237c5700 1 -- 172.21.15.62:0/2415890604 wait complete. 2023-07-14T16:15:26.859 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-14T16:15:26.873 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:26 smithi081 ceph-mon[127795]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:15:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:26 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2415890604' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T16:15:26.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:26 smithi062 ceph-mon[95543]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:15:26.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:26 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2415890604' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T16:15:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:26 smithi164 ceph-mon[97427]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:15:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:26 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2415890604' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T16:15:27.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.103+0000 7f218dfff700 1 -- 172.21.15.62:0/310059824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880ffb00 msgr2=0x7f21880fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.103+0000 7f218dfff700 1 --2- 172.21.15.62:0/310059824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880ffb00 0x7f21880fff10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2178009990 tx=0x7f2178009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.104+0000 7f218dfff700 1 -- 172.21.15.62:0/310059824 shutdown_connections 2023-07-14T16:15:27.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.104+0000 7f218dfff700 1 --2- 172.21.15.62:0/310059824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2188101680 0x7f2188107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.104+0000 7f218dfff700 1 --2- 172.21.15.62:0/310059824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880ffb00 0x7f21880fff10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.104+0000 7f218dfff700 1 --2- 172.21.15.62:0/310059824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2188100cf0 0x7f2188101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.104+0000 7f218dfff700 1 -- 172.21.15.62:0/310059824 >> 172.21.15.62:0/310059824 conn(0x7f21880fb0e0 msgr2=0x7f21880fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:27.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.104+0000 7f218dfff700 1 -- 172.21.15.62:0/310059824 shutdown_connections 2023-07-14T16:15:27.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.104+0000 7f218dfff700 1 -- 172.21.15.62:0/310059824 wait complete. 2023-07-14T16:15:27.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.105+0000 7f218dfff700 1 Processor -- start 2023-07-14T16:15:27.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.105+0000 7f218dfff700 1 -- start start 2023-07-14T16:15:27.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f218dfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21880ffb00 0x7f21881957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f218dfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2188100cf0 0x7f2188195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f218dfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2188101680 0x7f218819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f218dfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218819d350 con 0x7f2188100cf0 2023-07-14T16:15:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f218dfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218819d490 con 0x7f21880ffb00 2023-07-14T16:15:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f218dfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218819d620 con 0x7f2188101680 2023-07-14T16:15:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2186ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2188100cf0 0x7f2188195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2186ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2188100cf0 0x7f2188195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56396/0 (socket says 172.21.15.62:56396) 2023-07-14T16:15:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2186ffd700 1 -- 172.21.15.62:0/1363181366 learned_addr learned my addr 172.21.15.62:0/1363181366 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f21877fe700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21880ffb00 0x7f21881957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2187fff700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2188101680 0x7f218819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2186ffd700 1 -- 172.21.15.62:0/1363181366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2188101680 msgr2=0x7f218819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2186ffd700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2188101680 0x7f218819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2186ffd700 1 -- 172.21.15.62:0/1363181366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21880ffb00 msgr2=0x7f21881957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2186ffd700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21880ffb00 0x7f21881957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.106+0000 7f2186ffd700 1 -- 172.21.15.62:0/1363181366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780095f0 con 0x7f2188100cf0 2023-07-14T16:15:27.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.107+0000 7f2186ffd700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2188100cf0 0x7f2188195d10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f217c00d950 tx=0x7f217c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:27.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.107+0000 7f2184ff9700 1 -- 172.21.15.62:0/1363181366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c009940 con 0x7f2188100cf0 2023-07-14T16:15:27.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.107+0000 7f2184ff9700 1 -- 172.21.15.62:0/1363181366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f217c010460 con 0x7f2188100cf0 2023-07-14T16:15:27.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.107+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218819d900 con 0x7f2188100cf0 2023-07-14T16:15:27.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.107+0000 7f2184ff9700 1 -- 172.21.15.62:0/1363181366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c00f620 con 0x7f2188100cf0 2023-07-14T16:15:27.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.107+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218819de60 con 0x7f2188100cf0 2023-07-14T16:15:27.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.108+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21881070e0 con 0x7f2188100cf0 2023-07-14T16:15:27.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.110+0000 7f2184ff9700 1 -- 172.21.15.62:0/1363181366 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f217c0105d0 con 0x7f2188100cf0 2023-07-14T16:15:27.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.111+0000 7f2184ff9700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21700a1220 0x7f21700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.112+0000 7f2184ff9700 1 -- 172.21.15.62:0/1363181366 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f217c0cc730 con 0x7f2188100cf0 2023-07-14T16:15:27.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.112+0000 7f21877fe700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21700a1220 0x7f21700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.112+0000 7f21877fe700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21700a1220 0x7f21700a36d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f217800c010 tx=0x7f2178005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:27.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.115+0000 7f2184ff9700 1 -- 172.21.15.62:0/1363181366 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f217c095e30 con 0x7f2188100cf0 2023-07-14T16:15:27.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.330+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f218802c850 con 0x7f2188100cf0 2023-07-14T16:15:27.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.331+0000 7f2184ff9700 1 -- 172.21.15.62:0/1363181366 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f217c099450 con 0x7f2188100cf0 2023-07-14T16:15:27.309 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:27.309 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:27.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21700a1220 msgr2=0x7f21700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21700a1220 0x7f21700a36d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f217800c010 tx=0x7f2178005fb0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2188100cf0 msgr2=0x7f2188195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2188100cf0 0x7f2188195d10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f217c00d950 tx=0x7f217c00dc60 comp rx=0 tx=0).stop 2023-07-14T16:15:27.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 shutdown_connections 2023-07-14T16:15:27.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2188101680 0x7f218819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21700a1220 0x7f21700a36d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21880ffb00 0x7f21881957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 --2- 172.21.15.62:0/1363181366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2188100cf0 0x7f2188195d10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 >> 172.21.15.62:0/1363181366 conn(0x7f21880fb0e0 msgr2=0x7f2188104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:27.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 shutdown_connections 2023-07-14T16:15:27.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.334+0000 7f218dfff700 1 -- 172.21.15.62:0/1363181366 wait complete. 2023-07-14T16:15:27.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:15:27.331 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:15:27.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 -- 172.21.15.62:0/527933338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf981005e0 msgr2=0x7fcf981029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/527933338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf981005e0 0x7fcf981029c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009990 tx=0x7fcf94009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 -- 172.21.15.62:0/527933338 shutdown_connections 2023-07-14T16:15:27.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/527933338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf98105820 0x7fcf98107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/527933338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf981005e0 0x7fcf981029c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/527933338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf98102f00 0x7fcf981052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 -- 172.21.15.62:0/527933338 >> 172.21.15.62:0/527933338 conn(0x7fcf980fa000 msgr2=0x7fcf980fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:27.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 -- 172.21.15.62:0/527933338 shutdown_connections 2023-07-14T16:15:27.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.560+0000 7fcfa01ff700 1 -- 172.21.15.62:0/527933338 wait complete. 2023-07-14T16:15:27.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.561+0000 7fcfa01ff700 1 Processor -- start 2023-07-14T16:15:27.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.561+0000 7fcfa01ff700 1 -- start start 2023-07-14T16:15:27.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.561+0000 7fcfa01ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf981005e0 0x7fcf981979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.561+0000 7fcfa01ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf98102f00 0x7fcf98197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.561+0000 7fcfa01ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf98105820 0x7fcf9819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.561+0000 7fcfa01ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9819f530 con 0x7fcf98105820 2023-07-14T16:15:27.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.561+0000 7fcfa01ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9819f670 con 0x7fcf98102f00 2023-07-14T16:15:27.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.561+0000 7fcfa01ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9819f800 con 0x7fcf981005e0 2023-07-14T16:15:27.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf98105820 0x7fcf9819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf98105820 0x7fcf9819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56404/0 (socket says 172.21.15.62:56404) 2023-07-14T16:15:27.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 -- 172.21.15.62:0/3485171714 learned_addr learned my addr 172.21.15.62:0/3485171714 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:27.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9df9b700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf981005e0 0x7fcf981979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9d79a700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf98102f00 0x7fcf98197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 -- 172.21.15.62:0/3485171714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf981005e0 msgr2=0x7fcf981979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf981005e0 0x7fcf981979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 -- 172.21.15.62:0/3485171714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf98102f00 msgr2=0x7fcf98197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf98102f00 0x7fcf98197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 -- 172.21.15.62:0/3485171714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940095f0 con 0x7fcf98105820 2023-07-14T16:15:27.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.562+0000 7fcf9e79c700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf98105820 0x7fcf9819ef60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf90007ee0 tx=0x7fcf9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:27.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.563+0000 7fcf8effd700 1 -- 172.21.15.62:0/3485171714 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90017070 con 0x7fcf98105820 2023-07-14T16:15:27.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.563+0000 7fcf8effd700 1 -- 172.21.15.62:0/3485171714 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf9000f040 con 0x7fcf98105820 2023-07-14T16:15:27.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.563+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9819fae0 con 0x7fcf98105820 2023-07-14T16:15:27.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.563+0000 7fcf8effd700 1 -- 172.21.15.62:0/3485171714 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90005570 con 0x7fcf98105820 2023-07-14T16:15:27.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.563+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9819fff0 con 0x7fcf98105820 2023-07-14T16:15:27.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.564+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf98107390 con 0x7fcf98105820 2023-07-14T16:15:27.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.566+0000 7fcf8effd700 1 -- 172.21.15.62:0/3485171714 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcf9000db20 con 0x7fcf98105820 2023-07-14T16:15:27.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.567+0000 7fcf8effd700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcf840a11d0 0x7fcf840a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.567+0000 7fcf8effd700 1 -- 172.21.15.62:0/3485171714 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcf900cc060 con 0x7fcf98105820 2023-07-14T16:15:27.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.567+0000 7fcf9df9b700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcf840a11d0 0x7fcf840a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.568+0000 7fcf9df9b700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcf840a11d0 0x7fcf840a3680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006010 tx=0x7fcf9400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:27.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.571+0000 7fcf8effd700 1 -- 172.21.15.62:0/3485171714 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcf900fe790 con 0x7fcf98105820 2023-07-14T16:15:27.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.783+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf98061120 con 0x7fcf98105820 2023-07-14T16:15:27.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.784+0000 7fcf8effd700 1 -- 172.21.15.62:0/3485171714 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf90095760 con 0x7fcf98105820 2023-07-14T16:15:27.761 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:15:27.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.786+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcf840a11d0 msgr2=0x7fcf840a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.786+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcf840a11d0 0x7fcf840a3680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006010 tx=0x7fcf9400b260 comp rx=0 tx=0).stop 2023-07-14T16:15:27.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.786+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf98105820 msgr2=0x7fcf9819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf98105820 0x7fcf9819ef60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf90007ee0 tx=0x7fcf9000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 shutdown_connections 2023-07-14T16:15:27.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf98105820 0x7fcf9819ef60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf981005e0 0x7fcf981979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcf840a11d0 0x7fcf840a3680 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 --2- 172.21.15.62:0/3485171714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf98102f00 0x7fcf98197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 >> 172.21.15.62:0/3485171714 conn(0x7fcf980fa000 msgr2=0x7fcf980fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:27.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 shutdown_connections 2023-07-14T16:15:27.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:27.787+0000 7fcfa01ff700 1 -- 172.21.15.62:0/3485171714 wait complete. 2023-07-14T16:15:27.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:15:27.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 -- 172.21.15.62:0/283301239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe28101670 msgr2=0x7efe28107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/283301239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe28101670 0x7efe28107f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe2000b0a0 tx=0x7efe2000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 -- 172.21.15.62:0/283301239 shutdown_connections 2023-07-14T16:15:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/283301239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe28101670 0x7efe28107f30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/283301239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe280ffaf0 0x7efe280fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/283301239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe28100ce0 0x7efe28101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 -- 172.21.15.62:0/283301239 >> 172.21.15.62:0/283301239 conn(0x7efe280fb130 msgr2=0x7efe280fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 -- 172.21.15.62:0/283301239 shutdown_connections 2023-07-14T16:15:27.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.014+0000 7efe2d9ff700 1 -- 172.21.15.62:0/283301239 wait complete. 2023-07-14T16:15:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.015+0000 7efe2d9ff700 1 Processor -- start 2023-07-14T16:15:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.015+0000 7efe2d9ff700 1 -- start start 2023-07-14T16:15:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe2d9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe280ffaf0 0x7efe28197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe2d9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe28100ce0 0x7efe28198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe2d9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe28101670 0x7efe2819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe2d9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819f6b0 con 0x7efe280ffaf0 2023-07-14T16:15:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe2d9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819f7f0 con 0x7efe28100ce0 2023-07-14T16:15:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe2d9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819f980 con 0x7efe28101670 2023-07-14T16:15:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe26ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe280ffaf0 0x7efe28197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe26ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe280ffaf0 0x7efe28197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56418/0 (socket says 172.21.15.62:56418) 2023-07-14T16:15:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe26ffd700 1 -- 172.21.15.62:0/1468040996 learned_addr learned my addr 172.21.15.62:0/1468040996 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:27.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe267fc700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe28100ce0 0x7efe28198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe26ffd700 1 -- 172.21.15.62:0/1468040996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe28101670 msgr2=0x7efe2819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe26ffd700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe28101670 0x7efe2819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe26ffd700 1 -- 172.21.15.62:0/1468040996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe28100ce0 msgr2=0x7efe28198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:27.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe26ffd700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe28100ce0 0x7efe28198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:27.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.016+0000 7efe26ffd700 1 -- 172.21.15.62:0/1468040996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20009d00 con 0x7efe280ffaf0 2023-07-14T16:15:27.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.017+0000 7efe26ffd700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe280ffaf0 0x7efe28197b30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efe1800e970 tx=0x7efe1800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:27.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.017+0000 7efe2c9fd700 1 -- 172.21.15.62:0/1468040996 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18009790 con 0x7efe280ffaf0 2023-07-14T16:15:27.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.017+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2819fc60 con 0x7efe280ffaf0 2023-07-14T16:15:27.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.017+0000 7efe2c9fd700 1 -- 172.21.15.62:0/1468040996 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe180044b0 con 0x7efe280ffaf0 2023-07-14T16:15:27.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.017+0000 7efe2c9fd700 1 -- 172.21.15.62:0/1468040996 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18003dd0 con 0x7efe280ffaf0 2023-07-14T16:15:27.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.017+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe281a01c0 con 0x7efe280ffaf0 2023-07-14T16:15:27.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.018+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe28192090 con 0x7efe280ffaf0 2023-07-14T16:15:27.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.020+0000 7efe2c9fd700 1 -- 172.21.15.62:0/1468040996 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efe18003f30 con 0x7efe280ffaf0 2023-07-14T16:15:27.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.021+0000 7efe2c9fd700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe100a1150 0x7efe100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:27.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.022+0000 7efe2c9fd700 1 -- 172.21.15.62:0/1468040996 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe18014070 con 0x7efe280ffaf0 2023-07-14T16:15:27.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.022+0000 7efe267fc700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe100a1150 0x7efe100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:27.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.022+0000 7efe267fc700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe100a1150 0x7efe100a3600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005950 tx=0x7efe1c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:28.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.025+0000 7efe2c9fd700 1 -- 172.21.15.62:0/1468040996 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efe180963a0 con 0x7efe280ffaf0 2023-07-14T16:15:28.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:27 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1363181366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:28.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:27 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1363181366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:28.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:27 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1363181366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:28.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.235+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe2802c850 con 0x7efe280ffaf0 2023-07-14T16:15:28.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.235+0000 7efe2c9fd700 1 -- 172.21.15.62:0/1468040996 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe2802c850 con 0x7efe280ffaf0 2023-07-14T16:15:28.213 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:15:28.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe100a1150 msgr2=0x7efe100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe100a1150 0x7efe100a3600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005950 tx=0x7efe1c0058e0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe280ffaf0 msgr2=0x7efe28197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe280ffaf0 0x7efe28197b30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efe1800e970 tx=0x7efe1800ed30 comp rx=0 tx=0).stop 2023-07-14T16:15:28.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 shutdown_connections 2023-07-14T16:15:28.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe28101670 0x7efe2819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe280ffaf0 0x7efe28197b30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe100a1150 0x7efe100a3600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 --2- 172.21.15.62:0/1468040996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe28100ce0 0x7efe28198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 >> 172.21.15.62:0/1468040996 conn(0x7efe280fb130 msgr2=0x7efe28106640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:28.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 shutdown_connections 2023-07-14T16:15:28.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.238+0000 7efe2d9ff700 1 -- 172.21.15.62:0/1468040996 wait complete. 2023-07-14T16:15:28.234 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:15:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/3035355796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a440ffae0 msgr2=0x7f0a440ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/3035355796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a440ffae0 0x7f0a440ffef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009990 tx=0x7f0a40009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/3035355796 shutdown_connections 2023-07-14T16:15:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/3035355796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a44101660 0x7f0a44107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/3035355796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a440ffae0 0x7f0a440ffef0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/3035355796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100cd0 0x7f0a44101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/3035355796 >> 172.21.15.62:0/3035355796 conn(0x7f0a440fb0e0 msgr2=0x7f0a440fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/3035355796 shutdown_connections 2023-07-14T16:15:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.463+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/3035355796 wait complete. 2023-07-14T16:15:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.464+0000 7f0a4b3ff700 1 Processor -- start 2023-07-14T16:15:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.464+0000 7f0a4b3ff700 1 -- start start 2023-07-14T16:15:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4b3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a440ffae0 0x7f0a441979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4b3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100cd0 0x7f0a44197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a44101660 0x7f0a4419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4419f520 con 0x7f0a44101660 2023-07-14T16:15:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4b3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4419f660 con 0x7f0a440ffae0 2023-07-14T16:15:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4b3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4419f7f0 con 0x7f0a44100cd0 2023-07-14T16:15:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4999c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a44101660 0x7f0a4419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4919b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a440ffae0 0x7f0a441979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4899a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100cd0 0x7f0a44197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4999c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a44101660 0x7f0a4419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56426/0 (socket says 172.21.15.62:56426) 2023-07-14T16:15:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4999c700 1 -- 172.21.15.62:0/4179702370 learned_addr learned my addr 172.21.15.62:0/4179702370 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4919b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a440ffae0 0x7f0a441979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:58102/0 (socket says 172.21.15.62:58102) 2023-07-14T16:15:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4999c700 1 -- 172.21.15.62:0/4179702370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100cd0 msgr2=0x7f0a44197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4999c700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100cd0 0x7f0a44197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4999c700 1 -- 172.21.15.62:0/4179702370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a440ffae0 msgr2=0x7f0a441979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4999c700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a440ffae0 0x7f0a441979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.465+0000 7f0a4999c700 1 -- 172.21.15.62:0/4179702370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a400095f0 con 0x7f0a44101660 2023-07-14T16:15:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.466+0000 7f0a4999c700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a44101660 0x7f0a4419ef50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c007ee0 tx=0x7f0a3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.466+0000 7f0a3a7fc700 1 -- 172.21.15.62:0/4179702370 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c017070 con 0x7f0a44101660 2023-07-14T16:15:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.466+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4419fad0 con 0x7f0a44101660 2023-07-14T16:15:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.466+0000 7f0a3a7fc700 1 -- 172.21.15.62:0/4179702370 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a3c00f040 con 0x7f0a44101660 2023-07-14T16:15:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.466+0000 7f0a3a7fc700 1 -- 172.21.15.62:0/4179702370 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c005570 con 0x7f0a44101660 2023-07-14T16:15:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.466+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4419ffe0 con 0x7f0a44101660 2023-07-14T16:15:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.468+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4410b1b0 con 0x7f0a44101660 2023-07-14T16:15:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.469+0000 7f0a3a7fc700 1 -- 172.21.15.62:0/4179702370 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0a3c00db20 con 0x7f0a44101660 2023-07-14T16:15:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.471+0000 7f0a3a7fc700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a300a1150 0x7f0a300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.471+0000 7f0a3a7fc700 1 -- 172.21.15.62:0/4179702370 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a3c0cc030 con 0x7f0a44101660 2023-07-14T16:15:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.471+0000 7f0a4919b700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a300a1150 0x7f0a300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:28.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.472+0000 7f0a4919b700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a300a1150 0x7f0a300a3600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000c010 tx=0x7f0a40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:28.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.474+0000 7f0a3a7fc700 1 -- 172.21.15.62:0/4179702370 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0a3c095790 con 0x7f0a44101660 2023-07-14T16:15:28.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.680+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a441a0590 con 0x7f0a44101660 2023-07-14T16:15:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.681+0000 7f0a3a7fc700 1 -- 172.21.15.62:0/4179702370 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a3c098db0 con 0x7f0a44101660 2023-07-14T16:15:28.658 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:15:28.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a300a1150 msgr2=0x7f0a300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a300a1150 0x7f0a300a3600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000c010 tx=0x7f0a40005fb0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a44101660 msgr2=0x7f0a4419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a44101660 0x7f0a4419ef50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c007ee0 tx=0x7f0a3c00d3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 shutdown_connections 2023-07-14T16:15:28.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a44101660 0x7f0a4419ef50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a300a1150 0x7f0a300a3600 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a440ffae0 0x7f0a441979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 --2- 172.21.15.62:0/4179702370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a44100cd0 0x7f0a44197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 >> 172.21.15.62:0/4179702370 conn(0x7f0a440fb0e0 msgr2=0x7f0a440fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:28.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 shutdown_connections 2023-07-14T16:15:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.683+0000 7f0a4b3ff700 1 -- 172.21.15.62:0/4179702370 wait complete. 2023-07-14T16:15:28.679 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:15:28.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 -- 172.21.15.62:0/1831362177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 msgr2=0x7f63cc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 --2- 172.21.15.62:0/1831362177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 0x7f63cc101140 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009990 tx=0x7f63c0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 -- 172.21.15.62:0/1831362177 shutdown_connections 2023-07-14T16:15:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 --2- 172.21.15.62:0/1831362177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63cc101680 0x7f63cc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 --2- 172.21.15.62:0/1831362177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ffb00 0x7f63cc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 --2- 172.21.15.62:0/1831362177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 0x7f63cc101140 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 -- 172.21.15.62:0/1831362177 >> 172.21.15.62:0/1831362177 conn(0x7f63cc0fb0e0 msgr2=0x7f63cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 -- 172.21.15.62:0/1831362177 shutdown_connections 2023-07-14T16:15:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.905+0000 7f63d19ff700 1 -- 172.21.15.62:0/1831362177 wait complete. 2023-07-14T16:15:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.906+0000 7f63d19ff700 1 Processor -- start 2023-07-14T16:15:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.906+0000 7f63d19ff700 1 -- start start 2023-07-14T16:15:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.906+0000 7f63d19ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ffb00 0x7f63cc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.906+0000 7f63d19ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 0x7f63cc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.906+0000 7f63d19ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63cc101680 0x7f63cc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.906+0000 7f63d19ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc19f520 con 0x7f63cc100cf0 2023-07-14T16:15:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.906+0000 7f63d19ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc19f660 con 0x7f63cc0ffb00 2023-07-14T16:15:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.906+0000 7f63d19ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc19f7f0 con 0x7f63cc101680 2023-07-14T16:15:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 0x7f63cc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63caffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ffb00 0x7f63cc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 0x7f63cc197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56438/0 (socket says 172.21.15.62:56438) 2023-07-14T16:15:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 -- 172.21.15.62:0/2640133497 learned_addr learned my addr 172.21.15.62:0/2640133497 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63cb7fe700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63cc101680 0x7f63cc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 -- 172.21.15.62:0/2640133497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63cc101680 msgr2=0x7f63cc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63cc101680 0x7f63cc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 -- 172.21.15.62:0/2640133497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ffb00 msgr2=0x7f63cc1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ffb00 0x7f63cc1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 -- 172.21.15.62:0/2640133497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c00095f0 con 0x7f63cc100cf0 2023-07-14T16:15:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63ca7fc700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 0x7f63cc197ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f63c00098e0 tx=0x7f63c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.907+0000 7f63d09fd700 1 -- 172.21.15.62:0/2640133497 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c001d070 con 0x7f63cc100cf0 2023-07-14T16:15:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.908+0000 7f63d09fd700 1 -- 172.21.15.62:0/2640133497 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63c000bdd0 con 0x7f63cc100cf0 2023-07-14T16:15:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.908+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc19fa70 con 0x7f63cc100cf0 2023-07-14T16:15:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.908+0000 7f63d09fd700 1 -- 172.21.15.62:0/2640133497 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0022da0 con 0x7f63cc100cf0 2023-07-14T16:15:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.908+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc19fe90 con 0x7f63cc100cf0 2023-07-14T16:15:28.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.909+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc192090 con 0x7f63cc100cf0 2023-07-14T16:15:28.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.911+0000 7f63d09fd700 1 -- 172.21.15.62:0/2640133497 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f63c000b940 con 0x7f63cc100cf0 2023-07-14T16:15:28.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.912+0000 7f63d09fd700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63b40a11a0 0x7f63b40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:28.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.912+0000 7f63d09fd700 1 -- 172.21.15.62:0/2640133497 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f63c00cde90 con 0x7f63cc100cf0 2023-07-14T16:15:28.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.912+0000 7f63caffd700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63b40a11a0 0x7f63b40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:28.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.913+0000 7f63caffd700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63b40a11a0 0x7f63b40a3650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f63cc100b50 tx=0x7f63bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:28.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:28.916+0000 7f63d09fd700 1 -- 172.21.15.62:0/2640133497 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f63c0097680 con 0x7f63cc100cf0 2023-07-14T16:15:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:28 smithi164 ceph-mon[97427]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T16:15:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:28 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3485171714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:28 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1468040996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:29.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:28 smithi062 ceph-mon[95543]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T16:15:29.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:28 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3485171714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:29.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:28 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1468040996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:29.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.169+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63cc02cc70 con 0x7f63cc100cf0 2023-07-14T16:15:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.170+0000 7f63d09fd700 1 -- 172.21.15.62:0/2640133497 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f63c009aca0 con 0x7f63cc100cf0 2023-07-14T16:15:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:29.149 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:15:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.172+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63b40a11a0 msgr2=0x7f63b40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.172+0000 7f63d19ff700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63b40a11a0 0x7f63b40a3650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f63cc100b50 tx=0x7f63bc00b410 comp rx=0 tx=0).stop 2023-07-14T16:15:29.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.172+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 msgr2=0x7f63cc197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.172+0000 7f63d19ff700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 0x7f63cc197ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f63c00098e0 tx=0x7f63c0004730 comp rx=0 tx=0).stop 2023-07-14T16:15:29.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.173+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 shutdown_connections 2023-07-14T16:15:29.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.173+0000 7f63d19ff700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63cc101680 0x7f63cc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.173+0000 7f63d19ff700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f63b40a11a0 0x7f63b40a3650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.173+0000 7f63d19ff700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ffb00 0x7f63cc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.173+0000 7f63d19ff700 1 --2- 172.21.15.62:0/2640133497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63cc100cf0 0x7f63cc197ee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.173+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 >> 172.21.15.62:0/2640133497 conn(0x7f63cc0fb0e0 msgr2=0x7f63cc106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:29.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.173+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 shutdown_connections 2023-07-14T16:15:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.173+0000 7f63d19ff700 1 -- 172.21.15.62:0/2640133497 wait complete. 2023-07-14T16:15:29.155 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:15:29.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:28 smithi081 ceph-mon[127795]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T16:15:29.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:28 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3485171714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:29.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:28 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1468040996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:29.170 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.396+0000 7f70609ff700 1 -- 172.21.15.62:0/4105671342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 msgr2=0x7f705c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.396+0000 7f70609ff700 1 --2- 172.21.15.62:0/4105671342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 0x7f705c107f40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f705400b0a0 tx=0x7f705400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.397+0000 7f70609ff700 1 -- 172.21.15.62:0/4105671342 shutdown_connections 2023-07-14T16:15:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.397+0000 7f70609ff700 1 --2- 172.21.15.62:0/4105671342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 0x7f705c107f40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.397+0000 7f70609ff700 1 --2- 172.21.15.62:0/4105671342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f705c0ffb00 0x7f705c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.397+0000 7f70609ff700 1 --2- 172.21.15.62:0/4105671342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f705c100cf0 0x7f705c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.397+0000 7f70609ff700 1 -- 172.21.15.62:0/4105671342 >> 172.21.15.62:0/4105671342 conn(0x7f705c0fb0e0 msgr2=0x7f705c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.397+0000 7f70609ff700 1 -- 172.21.15.62:0/4105671342 shutdown_connections 2023-07-14T16:15:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.397+0000 7f70609ff700 1 -- 172.21.15.62:0/4105671342 wait complete. 2023-07-14T16:15:29.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.398+0000 7f70609ff700 1 Processor -- start 2023-07-14T16:15:29.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.398+0000 7f70609ff700 1 -- start start 2023-07-14T16:15:29.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.398+0000 7f70609ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f705c0ffb00 0x7f705c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:29.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.398+0000 7f70609ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f705c100cf0 0x7f705c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:29.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.398+0000 7f70609ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 0x7f705c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:29.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.398+0000 7f70609ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c19f530 con 0x7f705c101680 2023-07-14T16:15:29.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.398+0000 7f70609ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c19f670 con 0x7f705c0ffb00 2023-07-14T16:15:29.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.398+0000 7f70609ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c19f800 con 0x7f705c100cf0 2023-07-14T16:15:29.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 0x7f705c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:29.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 0x7f705c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56460/0 (socket says 172.21.15.62:56460) 2023-07-14T16:15:29.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 -- 172.21.15.62:0/1893081941 learned_addr learned my addr 172.21.15.62:0/1893081941 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:29.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705a59c700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f705c0ffb00 0x7f705c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:29.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f7059d9b700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f705c100cf0 0x7f705c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:29.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 -- 172.21.15.62:0/1893081941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f705c100cf0 msgr2=0x7f705c197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f705c100cf0 0x7f705c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 -- 172.21.15.62:0/1893081941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f705c0ffb00 msgr2=0x7f705c1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f705c0ffb00 0x7f705c1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 -- 172.21.15.62:0/1893081941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054009d00 con 0x7f705c101680 2023-07-14T16:15:29.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f705ad9d700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 0x7f705c19ef60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7054009fd0 tx=0x7f705400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.399+0000 7f704b7fe700 1 -- 172.21.15.62:0/1893081941 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400e040 con 0x7f705c101680 2023-07-14T16:15:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.400+0000 7f704b7fe700 1 -- 172.21.15.62:0/1893081941 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7054007760 con 0x7f705c101680 2023-07-14T16:15:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.400+0000 7f704b7fe700 1 -- 172.21.15.62:0/1893081941 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705401b6c0 con 0x7f705c101680 2023-07-14T16:15:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.400+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c19fa80 con 0x7f705c101680 2023-07-14T16:15:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.400+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c19fe50 con 0x7f705c101680 2023-07-14T16:15:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.401+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c10b250 con 0x7f705c101680 2023-07-14T16:15:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.403+0000 7f704b7fe700 1 -- 172.21.15.62:0/1893081941 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f705401d470 con 0x7f705c101680 2023-07-14T16:15:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.404+0000 7f704b7fe700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70440a1220 0x7f70440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.404+0000 7f704b7fe700 1 -- 172.21.15.62:0/1893081941 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f70540cdc90 con 0x7f705c101680 2023-07-14T16:15:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.404+0000 7f705a59c700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70440a1220 0x7f70440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.405+0000 7f705a59c700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70440a1220 0x7f70440a36d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f705c100b50 tx=0x7f704c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.408+0000 7f704b7fe700 1 -- 172.21.15.62:0/1893081941 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7054097420 con 0x7f705c101680 2023-07-14T16:15:29.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.616+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f705c10b440 con 0x7f705c101680 2023-07-14T16:15:29.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.617+0000 7f704b7fe700 1 -- 172.21.15.62:0/1893081941 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f705409aa40 con 0x7f705c101680 2023-07-14T16:15:29.597 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:29.598 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70440a1220 msgr2=0x7f70440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70440a1220 0x7f70440a36d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f705c100b50 tx=0x7f704c009260 comp rx=0 tx=0).stop 2023-07-14T16:15:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 msgr2=0x7f705c19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 0x7f705c19ef60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7054009fd0 tx=0x7f705400bc80 comp rx=0 tx=0).stop 2023-07-14T16:15:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 shutdown_connections 2023-07-14T16:15:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f705c101680 0x7f705c19ef60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70440a1220 0x7f70440a36d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f705c0ffb00 0x7f705c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 --2- 172.21.15.62:0/1893081941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f705c100cf0 0x7f705c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 >> 172.21.15.62:0/1893081941 conn(0x7f705c0fb0e0 msgr2=0x7f705c0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 shutdown_connections 2023-07-14T16:15:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.620+0000 7f70609ff700 1 -- 172.21.15.62:0/1893081941 wait complete. 2023-07-14T16:15:29.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:15:29.619 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:15:29.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 -- 172.21.15.62:0/963070616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 msgr2=0x7fefd4101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 --2- 172.21.15.62:0/963070616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 0x7fefd4101120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009990 tx=0x7fefc8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 -- 172.21.15.62:0/963070616 shutdown_connections 2023-07-14T16:15:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 --2- 172.21.15.62:0/963070616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd4101660 0x7fefd4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 --2- 172.21.15.62:0/963070616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd40ffae0 0x7fefd40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 --2- 172.21.15.62:0/963070616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 0x7fefd4101120 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 -- 172.21.15.62:0/963070616 >> 172.21.15.62:0/963070616 conn(0x7fefd40fb0e0 msgr2=0x7fefd40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 -- 172.21.15.62:0/963070616 shutdown_connections 2023-07-14T16:15:29.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.852+0000 7fefda9ff700 1 -- 172.21.15.62:0/963070616 wait complete. 2023-07-14T16:15:29.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.853+0000 7fefda9ff700 1 Processor -- start 2023-07-14T16:15:29.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.853+0000 7fefda9ff700 1 -- start start 2023-07-14T16:15:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefda9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd40ffae0 0x7fefd41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefda9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 0x7fefd4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefda9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd4101660 0x7fefd419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:29.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefda9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd419f550 con 0x7fefd4100cd0 2023-07-14T16:15:29.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefda9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd419f690 con 0x7fefd4101660 2023-07-14T16:15:29.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefda9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd419f820 con 0x7fefd40ffae0 2023-07-14T16:15:29.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 0x7fefd4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:29.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 0x7fefd4197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56474/0 (socket says 172.21.15.62:56474) 2023-07-14T16:15:29.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 -- 172.21.15.62:0/2837303747 learned_addr learned my addr 172.21.15.62:0/2837303747 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd3fff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd40ffae0 0x7fefd41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd8f9c700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd4101660 0x7fefd419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 -- 172.21.15.62:0/2837303747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd40ffae0 msgr2=0x7fefd41979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd40ffae0 0x7fefd41979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 -- 172.21.15.62:0/2837303747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd4101660 msgr2=0x7fefd419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd4101660 0x7fefd419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 -- 172.21.15.62:0/2837303747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc80095f0 con 0x7fefd4100cd0 2023-07-14T16:15:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd3fff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd40ffae0 0x7fefd41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.854+0000 7fefd37fe700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 0x7fefd4197f10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000c00 tx=0x7fefc8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.855+0000 7fefd17fa700 1 -- 172.21.15.62:0/2837303747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc801d070 con 0x7fefd4100cd0 2023-07-14T16:15:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.855+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd419faa0 con 0x7fefd4100cd0 2023-07-14T16:15:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.855+0000 7fefd17fa700 1 -- 172.21.15.62:0/2837303747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefc8003fc0 con 0x7fefd4100cd0 2023-07-14T16:15:29.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.855+0000 7fefd17fa700 1 -- 172.21.15.62:0/2837303747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8022cd0 con 0x7fefd4100cd0 2023-07-14T16:15:29.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.855+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd419fec0 con 0x7fefd4100cd0 2023-07-14T16:15:29.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.856+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd410b220 con 0x7fefd4100cd0 2023-07-14T16:15:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.858+0000 7fefd17fa700 1 -- 172.21.15.62:0/2837303747 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fefc8022470 con 0x7fefd4100cd0 2023-07-14T16:15:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.860+0000 7fefd17fa700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fefbc0a1150 0x7fefbc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.860+0000 7fefd17fa700 1 -- 172.21.15.62:0/2837303747 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fefc80cdec0 con 0x7fefd4100cd0 2023-07-14T16:15:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.860+0000 7fefd3fff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fefbc0a1150 0x7fefbc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.861+0000 7fefd3fff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fefbc0a1150 0x7fefbc0a3600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fefc4005950 tx=0x7fefc40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:29.864+0000 7fefd17fa700 1 -- 172.21.15.62:0/2837303747 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fefc80976b0 con 0x7fefd4100cd0 2023-07-14T16:15:29.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:29 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/4179702370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:29.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:29 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2640133497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:29.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:29 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1893081941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:30.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:29 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/4179702370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:30.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:29 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2640133497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:30.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:29 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1893081941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.073+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fefd41a0470 con 0x7fefd4100cd0 2023-07-14T16:15:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.074+0000 7fefd17fa700 1 -- 172.21.15.62:0/2837303747 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fefc809acd0 con 0x7fefd4100cd0 2023-07-14T16:15:30.051 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:15:30.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fefbc0a1150 msgr2=0x7fefbc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fefbc0a1150 0x7fefbc0a3600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fefc4005950 tx=0x7fefc40058e0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 msgr2=0x7fefd4197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 0x7fefd4197f10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000c00 tx=0x7fefc8005ef0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 shutdown_connections 2023-07-14T16:15:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fefbc0a1150 0x7fefbc0a3600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefd4101660 0x7fefd419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefd40ffae0 0x7fefd41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 --2- 172.21.15.62:0/2837303747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd4100cd0 0x7fefd4197f10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 >> 172.21.15.62:0/2837303747 conn(0x7fefd40fb0e0 msgr2=0x7fefd40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 shutdown_connections 2023-07-14T16:15:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.076+0000 7fefda9ff700 1 -- 172.21.15.62:0/2837303747 wait complete. 2023-07-14T16:15:30.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:15:30.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:29 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/4179702370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:30.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:29 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2640133497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:30.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:29 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1893081941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 -- 172.21.15.62:0/3932926085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0105810 msgr2=0x7f21a0107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 --2- 172.21.15.62:0/3932926085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0105810 0x7f21a0107c70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f219800b0a0 tx=0x7f219800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 -- 172.21.15.62:0/3932926085 shutdown_connections 2023-07-14T16:15:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 --2- 172.21.15.62:0/3932926085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0105810 0x7f21a0107c70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 --2- 172.21.15.62:0/3932926085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a01005d0 0x7f21a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 --2- 172.21.15.62:0/3932926085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0102ef0 0x7f21a01052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 -- 172.21.15.62:0/3932926085 >> 172.21.15.62:0/3932926085 conn(0x7f21a00fa070 msgr2=0x7f21a00fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 -- 172.21.15.62:0/3932926085 shutdown_connections 2023-07-14T16:15:30.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.300+0000 7f21a4fff700 1 -- 172.21.15.62:0/3932926085 wait complete. 2023-07-14T16:15:30.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.302+0000 7f21a4fff700 1 Processor -- start 2023-07-14T16:15:30.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.302+0000 7f21a4fff700 1 -- start start 2023-07-14T16:15:30.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.302+0000 7f21a4fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a01005d0 0x7f21a01957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:30.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.302+0000 7f21a4fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0102ef0 0x7f21a0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.302+0000 7f21a4fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0105810 0x7f21a019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:30.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.302+0000 7f21a4fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019d370 con 0x7f21a01005d0 2023-07-14T16:15:30.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.302+0000 7f21a4fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019d4b0 con 0x7f21a0102ef0 2023-07-14T16:15:30.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.302+0000 7f21a4fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019d640 con 0x7f21a0105810 2023-07-14T16:15:30.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a01005d0 0x7f21a01957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:30.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a01005d0 0x7f21a01957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56482/0 (socket says 172.21.15.62:56482) 2023-07-14T16:15:30.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 -- 172.21.15.62:0/674346572 learned_addr learned my addr 172.21.15.62:0/674346572 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:30.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f2197fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0102ef0 0x7f21a0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:30.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 -- 172.21.15.62:0/674346572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0105810 msgr2=0x7f21a019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0105810 0x7f21a019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 -- 172.21.15.62:0/674346572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0102ef0 msgr2=0x7f21a0195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0102ef0 0x7f21a0195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 -- 172.21.15.62:0/674346572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2198009d00 con 0x7f21a01005d0 2023-07-14T16:15:30.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f2197fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0102ef0 0x7f21a0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:30.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f219e59c700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a01005d0 0x7f21a01957f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f219000d950 tx=0x7f219000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f21977fe700 1 -- 172.21.15.62:0/674346572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190009b40 con 0x7f21a01005d0 2023-07-14T16:15:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f21977fe700 1 -- 172.21.15.62:0/674346572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2190009ca0 con 0x7f21a01005d0 2023-07-14T16:15:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f21977fe700 1 -- 172.21.15.62:0/674346572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219000f620 con 0x7f21a01005d0 2023-07-14T16:15:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.303+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a019d920 con 0x7f21a01005d0 2023-07-14T16:15:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.304+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a019de30 con 0x7f21a01005d0 2023-07-14T16:15:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.304+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a00fbbe0 con 0x7f21a01005d0 2023-07-14T16:15:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.307+0000 7f21977fe700 1 -- 172.21.15.62:0/674346572 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2190010460 con 0x7f21a01005d0 2023-07-14T16:15:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.308+0000 7f21977fe700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f218c0a11d0 0x7f218c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.308+0000 7f21977fe700 1 -- 172.21.15.62:0/674346572 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f21900ccc90 con 0x7f21a01005d0 2023-07-14T16:15:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.308+0000 7f2197fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f218c0a11d0 0x7f218c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.309+0000 7f2197fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f218c0a11d0 0x7f218c0a3680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2188005d90 tx=0x7f2188005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.312+0000 7f21977fe700 1 -- 172.21.15.62:0/674346572 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2190096390 con 0x7f21a01005d0 2023-07-14T16:15:30.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.523+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21a0061220 con 0x7f21a01005d0 2023-07-14T16:15:30.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.523+0000 7f21977fe700 1 -- 172.21.15.62:0/674346572 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2190020020 con 0x7f21a01005d0 2023-07-14T16:15:30.501 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:15:30.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f218c0a11d0 msgr2=0x7f218c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f218c0a11d0 0x7f218c0a3680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2188005d90 tx=0x7f2188005d00 comp rx=0 tx=0).stop 2023-07-14T16:15:30.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a01005d0 msgr2=0x7f21a01957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a01005d0 0x7f21a01957f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f219000d950 tx=0x7f219000dd10 comp rx=0 tx=0).stop 2023-07-14T16:15:30.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 shutdown_connections 2023-07-14T16:15:30.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0105810 0x7f21a019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a01005d0 0x7f21a01957f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f218c0a11d0 0x7f218c0a3680 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 --2- 172.21.15.62:0/674346572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0102ef0 0x7f21a0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 >> 172.21.15.62:0/674346572 conn(0x7f21a00fa070 msgr2=0x7f21a0101230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:30.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 shutdown_connections 2023-07-14T16:15:30.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.526+0000 7f21a4fff700 1 -- 172.21.15.62:0/674346572 wait complete. 2023-07-14T16:15:30.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:15:30.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 -- 172.21.15.62:0/2850608996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 msgr2=0x7f9028101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 --2- 172.21.15.62:0/2850608996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 0x7f9028101120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9018009990 tx=0x7f9018009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 -- 172.21.15.62:0/2850608996 shutdown_connections 2023-07-14T16:15:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 --2- 172.21.15.62:0/2850608996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9028101660 0x7f9028107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 --2- 172.21.15.62:0/2850608996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90280ffae0 0x7f90280ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 --2- 172.21.15.62:0/2850608996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 0x7f9028101120 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 -- 172.21.15.62:0/2850608996 >> 172.21.15.62:0/2850608996 conn(0x7f90280fb0e0 msgr2=0x7f90280fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 -- 172.21.15.62:0/2850608996 shutdown_connections 2023-07-14T16:15:30.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.752+0000 7f90301ff700 1 -- 172.21.15.62:0/2850608996 wait complete. 2023-07-14T16:15:30.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.753+0000 7f90301ff700 1 Processor -- start 2023-07-14T16:15:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.753+0000 7f90301ff700 1 -- start start 2023-07-14T16:15:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.753+0000 7f90301ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90280ffae0 0x7f9028197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.753+0000 7f90301ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 0x7f9028197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.753+0000 7f90301ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9028101660 0x7f902819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.753+0000 7f90301ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902819f4c0 con 0x7f9028100cd0 2023-07-14T16:15:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f90301ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902819f600 con 0x7f9028101660 2023-07-14T16:15:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f90301ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902819f790 con 0x7f90280ffae0 2023-07-14T16:15:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 0x7f9028197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 0x7f9028197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56502/0 (socket says 172.21.15.62:56502) 2023-07-14T16:15:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 -- 172.21.15.62:0/3452776228 learned_addr learned my addr 172.21.15.62:0/3452776228 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902df9b700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90280ffae0 0x7f9028197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902e79c700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9028101660 0x7f902819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 -- 172.21.15.62:0/3452776228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90280ffae0 msgr2=0x7f9028197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90280ffae0 0x7f9028197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 -- 172.21.15.62:0/3452776228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9028101660 msgr2=0x7f902819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9028101660 0x7f902819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 -- 172.21.15.62:0/3452776228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180095f0 con 0x7f9028100cd0 2023-07-14T16:15:30.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.754+0000 7f902d79a700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 0x7f9028197e80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9018000c00 tx=0x7f9018005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:30.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.755+0000 7f901effd700 1 -- 172.21.15.62:0/3452776228 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901801d070 con 0x7f9028100cd0 2023-07-14T16:15:30.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.755+0000 7f901effd700 1 -- 172.21.15.62:0/3452776228 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9018003f40 con 0x7f9028100cd0 2023-07-14T16:15:30.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.755+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902819fa10 con 0x7f9028100cd0 2023-07-14T16:15:30.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.755+0000 7f901effd700 1 -- 172.21.15.62:0/3452776228 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018022c70 con 0x7f9028100cd0 2023-07-14T16:15:30.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.755+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902819fe90 con 0x7f9028100cd0 2023-07-14T16:15:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.756+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9028192070 con 0x7f9028100cd0 2023-07-14T16:15:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.758+0000 7f901effd700 1 -- 172.21.15.62:0/3452776228 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9018003ab0 con 0x7f9028100cd0 2023-07-14T16:15:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.759+0000 7f901effd700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90140a11d0 0x7f90140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.759+0000 7f901effd700 1 -- 172.21.15.62:0/3452776228 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f90180cde00 con 0x7f9028100cd0 2023-07-14T16:15:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.760+0000 7f902df9b700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90140a11d0 0x7f90140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.760+0000 7f902df9b700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90140a11d0 0x7f90140a3680 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9028100b30 tx=0x7f902400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:30.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.763+0000 7f901effd700 1 -- 172.21.15.62:0/3452776228 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f90180975f0 con 0x7f9028100cd0 2023-07-14T16:15:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:30 smithi062 ceph-mon[95543]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:15:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:30 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2837303747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:30.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:30 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/674346572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:30.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.972+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9028061120 con 0x7f9028100cd0 2023-07-14T16:15:30.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.972+0000 7f901effd700 1 -- 172.21.15.62:0/3452776228 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f901809ac10 con 0x7f9028100cd0 2023-07-14T16:15:30.949 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:15:30.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90140a11d0 msgr2=0x7f90140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90140a11d0 0x7f90140a3680 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9028100b30 tx=0x7f902400a3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 msgr2=0x7f9028197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:30.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 0x7f9028197e80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9018000c00 tx=0x7f9018005740 comp rx=0 tx=0).stop 2023-07-14T16:15:30.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 shutdown_connections 2023-07-14T16:15:30.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90140a11d0 0x7f90140a3680 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9028101660 0x7f902819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90280ffae0 0x7f9028197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 --2- 172.21.15.62:0/3452776228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9028100cd0 0x7f9028197e80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:30.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 >> 172.21.15.62:0/3452776228 conn(0x7f90280fb0e0 msgr2=0x7f9028106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:30.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 shutdown_connections 2023-07-14T16:15:30.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:30.975+0000 7f90301ff700 1 -- 172.21.15.62:0/3452776228 wait complete. 2023-07-14T16:15:30.970 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:15:31.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:30 smithi164 ceph-mon[97427]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:15:31.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:30 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2837303747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:31.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:30 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/674346572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:31.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:30 smithi081 ceph-mon[127795]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:15:31.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:30 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2837303747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:31.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:30 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/674346572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.195+0000 7f9b61bff700 1 -- 172.21.15.62:0/1537105065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c0ffb00 msgr2=0x7f9b5c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.195+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1537105065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c0ffb00 0x7f9b5c0fff10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009990 tx=0x7f9b4c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.196+0000 7f9b61bff700 1 -- 172.21.15.62:0/1537105065 shutdown_connections 2023-07-14T16:15:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.196+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1537105065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c101680 0x7f9b5c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.196+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1537105065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c0ffb00 0x7f9b5c0fff10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.196+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1537105065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b5c100cf0 0x7f9b5c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.196+0000 7f9b61bff700 1 -- 172.21.15.62:0/1537105065 >> 172.21.15.62:0/1537105065 conn(0x7f9b5c0fb0e0 msgr2=0x7f9b5c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.196+0000 7f9b61bff700 1 -- 172.21.15.62:0/1537105065 shutdown_connections 2023-07-14T16:15:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.196+0000 7f9b61bff700 1 -- 172.21.15.62:0/1537105065 wait complete. 2023-07-14T16:15:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b61bff700 1 Processor -- start 2023-07-14T16:15:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b61bff700 1 -- start start 2023-07-14T16:15:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b61bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c0ffb00 0x7f9b5c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b61bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c100cf0 0x7f9b5c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b61bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b5c101680 0x7f9b5c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b61bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c19f500 con 0x7f9b5c100cf0 2023-07-14T16:15:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b61bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c19f640 con 0x7f9b5c0ffb00 2023-07-14T16:15:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b61bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c19f7d0 con 0x7f9b5c101680 2023-07-14T16:15:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.197+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c100cf0 0x7f9b5c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c100cf0 0x7f9b5c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56522/0 (socket says 172.21.15.62:56522) 2023-07-14T16:15:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5affd700 1 -- 172.21.15.62:0/1235412870 learned_addr learned my addr 172.21.15.62:0/1235412870 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5bfff700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b5c101680 0x7f9b5c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5b7fe700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c0ffb00 0x7f9b5c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5affd700 1 -- 172.21.15.62:0/1235412870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b5c101680 msgr2=0x7f9b5c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5affd700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b5c101680 0x7f9b5c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5affd700 1 -- 172.21.15.62:0/1235412870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c0ffb00 msgr2=0x7f9b5c197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5affd700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c0ffb00 0x7f9b5c197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5affd700 1 -- 172.21.15.62:0/1235412870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c0095f0 con 0x7f9b5c100cf0 2023-07-14T16:15:31.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b5affd700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c100cf0 0x7f9b5c197ec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000e9a0 tx=0x7f9b5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:31.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.198+0000 7f9b58ff9700 1 -- 172.21.15.62:0/1235412870 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50009790 con 0x7f9b5c100cf0 2023-07-14T16:15:31.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.199+0000 7f9b58ff9700 1 -- 172.21.15.62:0/1235412870 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b50010430 con 0x7f9b5c100cf0 2023-07-14T16:15:31.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.199+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c19fab0 con 0x7f9b5c100cf0 2023-07-14T16:15:31.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.199+0000 7f9b58ff9700 1 -- 172.21.15.62:0/1235412870 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5000cea0 con 0x7f9b5c100cf0 2023-07-14T16:15:31.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.199+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c1a0010 con 0x7f9b5c100cf0 2023-07-14T16:15:31.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.200+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c10b1d0 con 0x7f9b5c100cf0 2023-07-14T16:15:31.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.201+0000 7f9b58ff9700 1 -- 172.21.15.62:0/1235412870 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9b500105a0 con 0x7f9b5c100cf0 2023-07-14T16:15:31.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.203+0000 7f9b58ff9700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b440a1220 0x7f9b440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:31.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.203+0000 7f9b58ff9700 1 -- 172.21.15.62:0/1235412870 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9b50014070 con 0x7f9b5c100cf0 2023-07-14T16:15:31.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.203+0000 7f9b5b7fe700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b440a1220 0x7f9b440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:31.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.204+0000 7f9b5b7fe700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b440a1220 0x7f9b440a36d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0098e0 tx=0x7f9b4c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:31.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.207+0000 7f9b58ff9700 1 -- 172.21.15.62:0/1235412870 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9b50051790 con 0x7f9b5c100cf0 2023-07-14T16:15:31.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.460+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b5c1a05c0 con 0x7f9b5c100cf0 2023-07-14T16:15:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.461+0000 7f9b58ff9700 1 -- 172.21.15.62:0/1235412870 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f9b500961c0 con 0x7f9b5c100cf0 2023-07-14T16:15:31.438 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:31.438 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:15:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b440a1220 msgr2=0x7f9b440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b440a1220 0x7f9b440a36d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0098e0 tx=0x7f9b4c00b260 comp rx=0 tx=0).stop 2023-07-14T16:15:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c100cf0 msgr2=0x7f9b5c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c100cf0 0x7f9b5c197ec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000e9a0 tx=0x7f9b5000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 shutdown_connections 2023-07-14T16:15:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b5c101680 0x7f9b5c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9b440a1220 0x7f9b440a36d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5c0ffb00 0x7f9b5c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 --2- 172.21.15.62:0/1235412870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b5c100cf0 0x7f9b5c197ec0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 >> 172.21.15.62:0/1235412870 conn(0x7f9b5c0fb0e0 msgr2=0x7f9b5c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 shutdown_connections 2023-07-14T16:15:31.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.463+0000 7f9b61bff700 1 -- 172.21.15.62:0/1235412870 wait complete. 2023-07-14T16:15:31.442 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:15:31.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.686+0000 7fc946dff700 1 -- 172.21.15.62:0/2506396950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9400ffb00 msgr2=0x7fc9400fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.686+0000 7fc946dff700 1 --2- 172.21.15.62:0/2506396950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9400ffb00 0x7fc9400fff10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9340098f0 tx=0x7fc934009c00 comp rx=0 tx=0).stop 2023-07-14T16:15:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.686+0000 7fc946dff700 1 -- 172.21.15.62:0/2506396950 shutdown_connections 2023-07-14T16:15:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.686+0000 7fc946dff700 1 --2- 172.21.15.62:0/2506396950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc940101680 0x7fc940107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.686+0000 7fc946dff700 1 --2- 172.21.15.62:0/2506396950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9400ffb00 0x7fc9400fff10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.687+0000 7fc946dff700 1 --2- 172.21.15.62:0/2506396950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc940100cf0 0x7fc940101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.687+0000 7fc946dff700 1 -- 172.21.15.62:0/2506396950 >> 172.21.15.62:0/2506396950 conn(0x7fc9400fb0e0 msgr2=0x7fc9400fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.687+0000 7fc946dff700 1 -- 172.21.15.62:0/2506396950 shutdown_connections 2023-07-14T16:15:31.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.687+0000 7fc946dff700 1 -- 172.21.15.62:0/2506396950 wait complete. 2023-07-14T16:15:31.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc946dff700 1 Processor -- start 2023-07-14T16:15:31.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc946dff700 1 -- start start 2023-07-14T16:15:31.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc946dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9400ffb00 0x7fc9400742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:31.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc946dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc940100cf0 0x7fc940072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:31.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc946dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc940101680 0x7fc940072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:31.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc946dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400735e0 con 0x7fc940100cf0 2023-07-14T16:15:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc946dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc940073750 con 0x7fc9400ffb00 2023-07-14T16:15:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc946dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94019f7f0 con 0x7fc940101680 2023-07-14T16:15:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.688+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc940100cf0 0x7fc940072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc940100cf0 0x7fc940072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56534/0 (socket says 172.21.15.62:56534) 2023-07-14T16:15:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93ffff700 1 -- 172.21.15.62:0/3000776330 learned_addr learned my addr 172.21.15.62:0/3000776330 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc944b9b700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9400ffb00 0x7fc9400742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc94539c700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc940101680 0x7fc940072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93ffff700 1 -- 172.21.15.62:0/3000776330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc940101680 msgr2=0x7fc940072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93ffff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc940101680 0x7fc940072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93ffff700 1 -- 172.21.15.62:0/3000776330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9400ffb00 msgr2=0x7fc9400742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93ffff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9400ffb00 0x7fc9400742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93ffff700 1 -- 172.21.15.62:0/3000776330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938009d00 con 0x7fc940100cf0 2023-07-14T16:15:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc94539c700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc940101680 0x7fc940072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93ffff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc940100cf0 0x7fc940072900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e9a0 tx=0x7fc92c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93dffb700 1 -- 172.21.15.62:0/3000776330 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c009790 con 0x7fc940100cf0 2023-07-14T16:15:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc934009520 con 0x7fc940100cf0 2023-07-14T16:15:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93dffb700 1 -- 172.21.15.62:0/3000776330 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc92c00cbd0 con 0x7fc940100cf0 2023-07-14T16:15:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.689+0000 7fc93dffb700 1 -- 172.21.15.62:0/3000776330 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c018680 con 0x7fc940100cf0 2023-07-14T16:15:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.690+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94019fe50 con 0x7fc940100cf0 2023-07-14T16:15:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.691+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940107120 con 0x7fc940100cf0 2023-07-14T16:15:31.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.693+0000 7fc93dffb700 1 -- 172.21.15.62:0/3000776330 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc92c00cd40 con 0x7fc940100cf0 2023-07-14T16:15:31.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.695+0000 7fc93dffb700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9300a11a0 0x7fc9300a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:31.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.695+0000 7fc93dffb700 1 -- 172.21.15.62:0/3000776330 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc92c014070 con 0x7fc940100cf0 2023-07-14T16:15:31.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.695+0000 7fc944b9b700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9300a11a0 0x7fc9300a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:31.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.696+0000 7fc944b9b700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9300a11a0 0x7fc9300a3650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc9340098c0 tx=0x7fc934019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:31.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.700+0000 7fc93dffb700 1 -- 172.21.15.62:0/3000776330 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc92c051790 con 0x7fc940100cf0 2023-07-14T16:15:31.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:31 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3452776228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:31.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:31 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1235412870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:31.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.911+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc94002c850 con 0x7fc940100cf0 2023-07-14T16:15:31.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.912+0000 7fc93dffb700 1 -- 172.21.15.62:0/3000776330 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7fc92c096300 con 0x7fc940100cf0 2023-07-14T16:15:31.890 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:31.890 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9300a11a0 msgr2=0x7fc9300a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9300a11a0 0x7fc9300a3650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc9340098c0 tx=0x7fc934019040 comp rx=0 tx=0).stop 2023-07-14T16:15:31.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc940100cf0 msgr2=0x7fc940072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:31.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc940100cf0 0x7fc940072900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e9a0 tx=0x7fc92c00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 shutdown_connections 2023-07-14T16:15:31.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc940101680 0x7fc940072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc9300a11a0 0x7fc9300a3650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9400ffb00 0x7fc9400742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 --2- 172.21.15.62:0/3000776330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc940100cf0 0x7fc940072900 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:31.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 >> 172.21.15.62:0/3000776330 conn(0x7fc9400fb0e0 msgr2=0x7fc940104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 shutdown_connections 2023-07-14T16:15:31.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:31.915+0000 7fc946dff700 1 -- 172.21.15.62:0/3000776330 wait complete. 2023-07-14T16:15:31.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T16:13:58.805235+0000', 'modified': '2023-07-14T16:14:04.590671+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24602, 'mds_1': 24565, 'mds_2': 24512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24475': {'gid': 24475, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/903698978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 903698978}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 903698978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.81:6835/337855571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 337855571}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 337855571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6835/1558258971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 1558258971}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 1558258971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24551': {'gid': 24551, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6837/3627777030', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 3627777030}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 3627777030}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.164:6839/794647512', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 794647512}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 794647512}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24602': {'gid': 24602, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6839/2113600738', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 2113600738}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 2113600738}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T16:15:31.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-14T16:15:31.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-14T16:15:32.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:31 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3452776228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:32.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:31 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1235412870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:32.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.138+0000 7f82cebff700 1 -- 172.21.15.62:0/2442455510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 msgr2=0x7f82c80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.138+0000 7f82cebff700 1 --2- 172.21.15.62:0/2442455510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 0x7f82c80ffef0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009990 tx=0x7f82bc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.139+0000 7f82cebff700 1 -- 172.21.15.62:0/2442455510 shutdown_connections 2023-07-14T16:15:32.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.139+0000 7f82cebff700 1 --2- 172.21.15.62:0/2442455510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82c8101660 0x7f82c8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.139+0000 7f82cebff700 1 --2- 172.21.15.62:0/2442455510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 0x7f82c80ffef0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.139+0000 7f82cebff700 1 --2- 172.21.15.62:0/2442455510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82c8100cd0 0x7f82c8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.139+0000 7f82cebff700 1 -- 172.21.15.62:0/2442455510 >> 172.21.15.62:0/2442455510 conn(0x7f82c80fb0e0 msgr2=0x7f82c80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.139+0000 7f82cebff700 1 -- 172.21.15.62:0/2442455510 shutdown_connections 2023-07-14T16:15:32.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.139+0000 7f82cebff700 1 -- 172.21.15.62:0/2442455510 wait complete. 2023-07-14T16:15:32.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cebff700 1 Processor -- start 2023-07-14T16:15:32.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cebff700 1 -- start start 2023-07-14T16:15:32.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cebff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 0x7f82c80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cebff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82c8100cd0 0x7f82c8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cebff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82c8101660 0x7f82c8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cebff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80735e0 con 0x7f82c80ffae0 2023-07-14T16:15:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cebff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c8073750 con 0x7f82c8100cd0 2023-07-14T16:15:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cebff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c819f860 con 0x7f82c8101660 2023-07-14T16:15:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.140+0000 7f82cc99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 0x7f82c80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cc99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 0x7f82c80742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56546/0 (socket says 172.21.15.62:56546) 2023-07-14T16:15:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cc99b700 1 -- 172.21.15.62:0/2724997103 learned_addr learned my addr 172.21.15.62:0/2724997103 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82c7fff700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82c8100cd0 0x7f82c8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cd19c700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82c8101660 0x7f82c8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cc99b700 1 -- 172.21.15.62:0/2724997103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82c8101660 msgr2=0x7f82c8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cc99b700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82c8101660 0x7f82c8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cc99b700 1 -- 172.21.15.62:0/2724997103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82c8100cd0 msgr2=0x7f82c8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cc99b700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82c8100cd0 0x7f82c8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cc99b700 1 -- 172.21.15.62:0/2724997103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc0095f0 con 0x7f82c80ffae0 2023-07-14T16:15:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82c7fff700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82c8100cd0 0x7f82c8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cc99b700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 0x7f82c80742c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82bc004730 tx=0x7f82bc004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82c5ffb700 1 -- 172.21.15.62:0/2724997103 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc01d070 con 0x7f82c80ffae0 2023-07-14T16:15:32.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82c5ffb700 1 -- 172.21.15.62:0/2724997103 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82bc004220 con 0x7f82c80ffae0 2023-07-14T16:15:32.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82c5ffb700 1 -- 172.21.15.62:0/2724997103 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc005290 con 0x7f82c80ffae0 2023-07-14T16:15:32.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.141+0000 7f82cebff700 1 -- 172.21.15.62:0/2724997103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c819fae0 con 0x7f82c80ffae0 2023-07-14T16:15:32.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.142+0000 7f82cebff700 1 -- 172.21.15.62:0/2724997103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c819ff30 con 0x7f82c80ffae0 2023-07-14T16:15:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.146+0000 7f82c5ffb700 1 -- 172.21.15.62:0/2724997103 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f82bc004960 con 0x7f82c80ffae0 2023-07-14T16:15:32.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.147+0000 7f82e2442b80 1 Processor -- start 2023-07-14T16:15:32.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.147+0000 7f82e2442b80 1 -- start start 2023-07-14T16:15:32.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09480 0x55b07ad09890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09dd0 0x55b07ad0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 0x55b07ad0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82c5ffb700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82b80a1150 0x7f82b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82e2442b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ac0ec60 con 0x55b07ad09480 2023-07-14T16:15:32.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82e2442b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ac619f0 con 0x55b07ad09dd0 2023-07-14T16:15:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82e2442b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ac5ae80 con 0x55b07ad0ac20 2023-07-14T16:15:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82c5ffb700 1 -- 172.21.15.62:0/2724997103 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f82bc0cdb30 con 0x7f82c80ffae0 2023-07-14T16:15:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82cd19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09480 0x55b07ad09890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82cd19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09480 0x55b07ad09890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56556/0 (socket says 172.21.15.62:56556) 2023-07-14T16:15:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82cd19c700 1 -- 172.21.15.62:0/3894306473 learned_addr learned my addr 172.21.15.62:0/3894306473 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82cc99b700 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09dd0 0x55b07ad0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82cd19c700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82b80a1150 0x7f82b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.148+0000 7f82c7fff700 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 0x55b07ad0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82cc99b700 1 -- 172.21.15.62:0/3894306473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 msgr2=0x55b07ad0b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82cc99b700 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 0x55b07ad0b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82cc99b700 1 -- 172.21.15.62:0/3894306473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09480 msgr2=0x55b07ad09890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82cc99b700 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09480 0x55b07ad09890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82cc99b700 1 -- 172.21.15.62:0/3894306473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b07aa88d90 con 0x55b07ad09dd0 2023-07-14T16:15:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82cd19c700 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09480 0x55b07ad09890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82cc99b700 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09dd0 0x55b07ad0a690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005f50 tx=0x7f82bc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700affa704021672 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82cd19c700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82b80a1150 0x7f82b80a3600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f82c00070b0 tx=0x7f82c00093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82b2ffd700 1 -- 172.21.15.62:0/3894306473 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc09efb0 con 0x55b07ad09dd0 2023-07-14T16:15:32.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82b2ffd700 1 -- 172.21.15.62:0/3894306473 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82bc0a3340 con 0x55b07ad09dd0 2023-07-14T16:15:32.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.149+0000 7f82b2ffd700 1 -- 172.21.15.62:0/3894306473 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0a3110 con 0x55b07ad09dd0 2023-07-14T16:15:32.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 -- 172.21.15.62:0/3894306473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09dd0 msgr2=0x55b07ad0a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09dd0 0x55b07ad0a690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005f50 tx=0x7f82bc009310 comp rx=0 tx=0).stop 2023-07-14T16:15:32.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 -- 172.21.15.62:0/3894306473 shutdown_connections 2023-07-14T16:15:32.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 0x55b07ad0b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09480 0x55b07ad09890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 --2- 172.21.15.62:0/3894306473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09dd0 0x55b07ad0a690 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 -- 172.21.15.62:0/3894306473 >> 172.21.15.62:0/3894306473 conn(0x55b07acfed50 msgr2=0x55b07ad04180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 -- 172.21.15.62:0/3894306473 shutdown_connections 2023-07-14T16:15:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.150+0000 7f82e2442b80 1 -- 172.21.15.62:0/3894306473 wait complete. 2023-07-14T16:15:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 1 -- start start 2023-07-14T16:15:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 1 Processor -- start 2023-07-14T16:15:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 0x55b07ad12d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad13260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 0x55b07ad1a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07abfaa70 con 0x55b07ad09dd0 2023-07-14T16:15:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07abfa770 con 0x55b07ad09480 2023-07-14T16:15:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07aade8d0 con 0x55b07ad0ac20 2023-07-14T16:15:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.151+0000 7f82e2442b80 10 client.0 _ll_register_callbacks cb 0x55b07abf8710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad13260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad13260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56572/0 (socket says 172.21.15.62:56572) 2023-07-14T16:15:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 -- 172.21.15.62:0/408246531 learned_addr learned my addr 172.21.15.62:0/408246531 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82cc99b700 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 0x55b07ad12d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 -- 172.21.15.62:0/408246531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 msgr2=0x55b07ad1a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 0x55b07ad1a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 -- 172.21.15.62:0/408246531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 msgr2=0x55b07ad12d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 0x55b07ad12d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 -- 172.21.15.62:0/408246531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c000aaa0 con 0x55b07ad09dd0 2023-07-14T16:15:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82c7fff700 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad13260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f82b4011800 tx=0x7f82b4011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c87b0d62eb7406c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.152+0000 7f82a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82a7fff700 1 -- 172.21.15.62:0/408246531 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b400fa50 con 0x55b07ad09dd0 2023-07-14T16:15:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82e2442b80 1 -- 172.21.15.62:0/408246531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b07abb44a0 con 0x55b07ad09dd0 2023-07-14T16:15:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82a7fff700 1 -- 172.21.15.62:0/408246531 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f82b400fbb0 con 0x55b07ad09dd0 2023-07-14T16:15:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82a7fff700 1 -- 172.21.15.62:0/408246531 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82b400fd70 con 0x55b07ad09dd0 2023-07-14T16:15:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82e2442b80 10 client.15294 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82e2442b80 1 -- 172.21.15.62:0/408246531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f82c819ff30 con 0x55b07ad09dd0 2023-07-14T16:15:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82a7fff700 1 -- 172.21.15.62:0/408246531 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b07abb44a0 con 0x55b07ad09dd0 2023-07-14T16:15:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82a7fff700 1 -- 172.21.15.62:0/408246531 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f82b400e3d0 con 0x55b07ad09dd0 2023-07-14T16:15:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82e2442b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82e2442b80 10 client.15294 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-14T16:15:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82e2442b80 20 client.15294 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82e2442b80 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b07ad1e020 0x55b07ad1d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.153+0000 7f82e2442b80 4 client.15294 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:15:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.154+0000 7f82e2442b80 1 -- 172.21.15.62:0/408246531 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b07ac53e60 con 0x55b07ad1e020 2023-07-14T16:15:32.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.154+0000 7f82cc99b700 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b07ad1e020 0x55b07ad1d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.154+0000 7f82cc99b700 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b07ad1e020 0x55b07ad1d400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd95ffda0f4b9452 server_cookie=d2b9ec330d9fc862 in_seq=0 out_seq=0 2023-07-14T16:15:32.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.154+0000 7f82a7fff700 10 client.15294 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:15:32.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.155+0000 7f82a7fff700 1 -- 172.21.15.62:0/408246531 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b07ac53e60 con 0x55b07ad1e020 2023-07-14T16:15:32.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.155+0000 7f82a7fff700 10 client.15294 handle_command_reply: tid=0 2023-07-14T16:15:32.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.155+0000 7f82e2442b80 1 client.15294 shutdown 2023-07-14T16:15:32.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.155+0000 7f82e2442b80 1 -- 172.21.15.62:0/408246531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 msgr2=0x55b07ad13260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.155+0000 7f82e2442b80 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad13260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f82b4011800 tx=0x7f82b4011b10 comp rx=0 tx=0).stop 2023-07-14T16:15:32.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 1 -- 172.21.15.62:0/408246531 shutdown_connections 2023-07-14T16:15:32.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b07ad1e020 0x55b07ad1d400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad0ac20 0x55b07ad1a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 0x55b07ad12d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82a7fff700 0 client.15294 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:15:32.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 1 --2- 172.21.15.62:0/408246531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad13260 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 1 -- 172.21.15.62:0/408246531 >> 172.21.15.62:0/408246531 conn(0x55b07acfed50 msgr2=0x55b07ad03860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 1 -- 172.21.15.62:0/408246531 shutdown_connections 2023-07-14T16:15:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 1 -- 172.21.15.62:0/408246531 wait complete. 2023-07-14T16:15:32.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.156+0000 7f82e2442b80 20 client.15294 trim_cache size 0 max 16384 2023-07-14T16:15:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.174+0000 7f82e2442b80 1 Processor -- start 2023-07-14T16:15:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.174+0000 7f82e2442b80 1 -- start start 2023-07-14T16:15:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09480 0x55b07ad1d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad1d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad0ac20 0x55b07ad1a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82e2442b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ac53e60 con 0x55b07ad09dd0 2023-07-14T16:15:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82e2442b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07abb44a0 con 0x55b07ad0ac20 2023-07-14T16:15:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82e2442b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07aaeade0 con 0x55b07ad09480 2023-07-14T16:15:32.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:31 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3452776228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:32.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:31 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1235412870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:32.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad1d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56576/0 (socket says 172.21.15.62:56576) 2023-07-14T16:15:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 -- 172.21.15.62:0/27455815 learned_addr learned my addr 172.21.15.62:0/27455815 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82cd19c700 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad0ac20 0x55b07ad1a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82cc99b700 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09480 0x55b07ad1d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 -- 172.21.15.62:0/27455815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09480 msgr2=0x55b07ad1d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09480 0x55b07ad1d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 -- 172.21.15.62:0/27455815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad0ac20 msgr2=0x55b07ad1a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad0ac20 0x55b07ad1a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 -- 172.21.15.62:0/27455815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b07aa71c10 con 0x55b07ad09dd0 2023-07-14T16:15:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82cd19c700 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad0ac20 0x55b07ad1a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.175+0000 7f82c7fff700 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad1d940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009010 tx=0x7f82b4005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e273c694b45291 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82b37fe700 1 -- 172.21.15.62:0/27455815 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b400e950 con 0x55b07ad09dd0 2023-07-14T16:15:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82b37fe700 1 -- 172.21.15.62:0/27455815 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82b400eab0 con 0x55b07ad09dd0 2023-07-14T16:15:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82b37fe700 1 -- 172.21.15.62:0/27455815 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b40053b0 con 0x55b07ad09dd0 2023-07-14T16:15:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82e2442b80 1 -- 172.21.15.62:0/27455815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 msgr2=0x55b07ad1d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82e2442b80 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad1d940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009010 tx=0x7f82b4005c30 comp rx=0 tx=0).stop 2023-07-14T16:15:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82e2442b80 1 -- 172.21.15.62:0/27455815 shutdown_connections 2023-07-14T16:15:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82e2442b80 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad0ac20 0x55b07ad1a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82e2442b80 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09480 0x55b07ad1d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82e2442b80 1 --2- 172.21.15.62:0/27455815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad09dd0 0x55b07ad1d940 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.176+0000 7f82e2442b80 1 -- 172.21.15.62:0/27455815 >> 172.21.15.62:0/27455815 conn(0x55b07acfed50 msgr2=0x55b07ad1aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.177+0000 7f82e2442b80 1 -- 172.21.15.62:0/27455815 shutdown_connections 2023-07-14T16:15:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.177+0000 7f82e2442b80 1 -- 172.21.15.62:0/27455815 wait complete. 2023-07-14T16:15:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.177+0000 7f82e2442b80 1 -- start start 2023-07-14T16:15:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.177+0000 7f82e2442b80 1 Processor -- start 2023-07-14T16:15:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 0x55b07ad181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09dd0 0x55b07ad6fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82e2442b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad0ac20 0x55b07ad72420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82e2442b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ad72960 con 0x55b07ad0ac20 2023-07-14T16:15:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82e2442b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ad72aa0 con 0x55b07ad09480 2023-07-14T16:15:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82e2442b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ad72c10 con 0x55b07ad09dd0 2023-07-14T16:15:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82e2442b80 10 client.0 _ll_register_callbacks cb 0x7f82b4008c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cd19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad0ac20 0x55b07ad72420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cd19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad0ac20 0x55b07ad72420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56580/0 (socket says 172.21.15.62:56580) 2023-07-14T16:15:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cd19c700 1 -- 172.21.15.62:0/394271363 learned_addr learned my addr 172.21.15.62:0/394271363 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cc99b700 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 0x55b07ad181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82c7fff700 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09dd0 0x55b07ad6fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cd19c700 1 -- 172.21.15.62:0/394271363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09dd0 msgr2=0x55b07ad6fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cd19c700 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09dd0 0x55b07ad6fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cd19c700 1 -- 172.21.15.62:0/394271363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 msgr2=0x55b07ad181b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cd19c700 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 0x55b07ad181b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82cd19c700 1 -- 172.21.15.62:0/394271363 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c819ff30 con 0x55b07ad0ac20 2023-07-14T16:15:32.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.178+0000 7f82c7fff700 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09dd0 0x55b07ad6fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:32.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82cd19c700 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad0ac20 0x55b07ad72420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82c0016bc0 tx=0x7f82c0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeee86929fd29e79 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:32.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82a7fff700 1 -- 172.21.15.62:0/394271363 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0019800 con 0x55b07ad0ac20 2023-07-14T16:15:32.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82a7fff700 1 -- 172.21.15.62:0/394271363 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f82c0006e40 con 0x55b07ad0ac20 2023-07-14T16:15:32.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82e2442b80 1 -- 172.21.15.62:0/394271363 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82bc0a3340 con 0x55b07ad0ac20 2023-07-14T16:15:32.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82a7fff700 1 -- 172.21.15.62:0/394271363 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c0019960 con 0x55b07ad0ac20 2023-07-14T16:15:32.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82e2442b80 10 client.15300 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:32.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82e2442b80 1 -- 172.21.15.62:0/394271363 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b07aaaf200 con 0x55b07ad0ac20 2023-07-14T16:15:32.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82a7fff700 1 -- 172.21.15.62:0/394271363 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82bc0a3340 con 0x55b07ad0ac20 2023-07-14T16:15:32.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82a7fff700 1 -- 172.21.15.62:0/394271363 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f82c0025a10 con 0x55b07ad0ac20 2023-07-14T16:15:32.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82e2442b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:32.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82e2442b80 10 client.15300 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-14T16:15:32.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.179+0000 7f82e2442b80 20 client.15300 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:32.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.180+0000 7f82e2442b80 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b07ad77a40 0x55b07ad74350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.180+0000 7f82e2442b80 4 client.15300 mds_command: new command op to 24512 tid=0[{"prefix": "status"}] 2023-07-14T16:15:32.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.180+0000 7f82e2442b80 1 -- 172.21.15.62:0/394271363 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b07ac16000 con 0x55b07ad77a40 2023-07-14T16:15:32.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.180+0000 7f82cc99b700 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b07ad77a40 0x55b07ad74350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.180+0000 7f82cc99b700 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b07ad77a40 0x55b07ad74350 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c28f364f8633b98a server_cookie=4590aeffa560a24f in_seq=0 out_seq=0 2023-07-14T16:15:32.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.180+0000 7f82a7fff700 10 client.15300 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:15:32.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82a7fff700 1 -- 172.21.15.62:0/394271363 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55b07ac16000 con 0x55b07ad77a40 2023-07-14T16:15:32.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82a7fff700 10 client.15300 handle_command_reply: tid=0 2023-07-14T16:15:32.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 client.15300 shutdown 2023-07-14T16:15:32.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 -- 172.21.15.62:0/394271363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad0ac20 msgr2=0x55b07ad72420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad0ac20 0x55b07ad72420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82c0016bc0 tx=0x7f82c0009a20 comp rx=0 tx=0).stop 2023-07-14T16:15:32.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 -- 172.21.15.62:0/394271363 shutdown_connections 2023-07-14T16:15:32.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b07ad77a40 0x55b07ad74350 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b07ad0ac20 0x55b07ad72420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b07ad09480 0x55b07ad181b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82a7fff700 0 client.15300 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:15:32.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 --2- 172.21.15.62:0/394271363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b07ad09dd0 0x55b07ad6fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 -- 172.21.15.62:0/394271363 >> 172.21.15.62:0/394271363 conn(0x55b07acfed50 msgr2=0x55b07ad1d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 -- 172.21.15.62:0/394271363 shutdown_connections 2023-07-14T16:15:32.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 1 -- 172.21.15.62:0/394271363 wait complete. 2023-07-14T16:15:32.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.181+0000 7f82e2442b80 20 client.15300 trim_cache size 0 max 16384 2023-07-14T16:15:32.167 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:32.168 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:15:32.168 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2023-07-14T16:15:32.168 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24512, 2023-07-14T16:15:32.168 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2023-07-14T16:15:32.168 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2023-07-14T16:15:32.168 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2023-07-14T16:15:32.168 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 88.616084147999999, 2023-07-14T16:15:32.168 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 23, 2023-07-14T16:15:32.169 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 88, 2023-07-14T16:15:32.169 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 86, 2023-07-14T16:15:32.169 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 121.68036447 2023-07-14T16:15:32.169 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:32.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 -- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82b80a1150 msgr2=0x7f82b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82b80a1150 0x7f82b80a3600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f82c00070b0 tx=0x7f82c00093d0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 -- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 msgr2=0x7f82c80742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 0x7f82c80742c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f82bc004730 tx=0x7f82bc004760 comp rx=0 tx=0).stop 2023-07-14T16:15:32.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 -- 172.21.15.62:0/2724997103 shutdown_connections 2023-07-14T16:15:32.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82c8101660 0x7f82c8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c80ffae0 0x7f82c80742c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82b80a1150 0x7f82b80a3600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 --2- 172.21.15.62:0/2724997103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82c8100cd0 0x7f82c8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 -- 172.21.15.62:0/2724997103 >> 172.21.15.62:0/2724997103 conn(0x7f82c80fb0e0 msgr2=0x7f82c80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 -- 172.21.15.62:0/2724997103 shutdown_connections 2023-07-14T16:15:32.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.182+0000 7f82cebff700 1 -- 172.21.15.62:0/2724997103 wait complete. 2023-07-14T16:15:32.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T16:15:32.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 -- 172.21.15.62:0/926322004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 msgr2=0x7efdec107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 --2- 172.21.15.62:0/926322004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 0x7efdec107f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efde400b0a0 tx=0x7efde400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 -- 172.21.15.62:0/926322004 shutdown_connections 2023-07-14T16:15:32.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 --2- 172.21.15.62:0/926322004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 0x7efdec107f20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 --2- 172.21.15.62:0/926322004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efdec0ffae0 0x7efdec0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 --2- 172.21.15.62:0/926322004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdec100cd0 0x7efdec101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 -- 172.21.15.62:0/926322004 >> 172.21.15.62:0/926322004 conn(0x7efdec0fb0e0 msgr2=0x7efdec0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 -- 172.21.15.62:0/926322004 shutdown_connections 2023-07-14T16:15:32.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.404+0000 7efdf1fff700 1 -- 172.21.15.62:0/926322004 wait complete. 2023-07-14T16:15:32.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.405+0000 7efdf1fff700 1 Processor -- start 2023-07-14T16:15:32.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdf1fff700 1 -- start start 2023-07-14T16:15:32.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdf1fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efdec0ffae0 0x7efdec1958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdf1fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdec100cd0 0x7efdec195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdf1fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 0x7efdec19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdf1fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec19d420 con 0x7efdec101660 2023-07-14T16:15:32.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdf1fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec19d560 con 0x7efdec0ffae0 2023-07-14T16:15:32.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdf1fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec19d6f0 con 0x7efdec100cd0 2023-07-14T16:15:32.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 0x7efdec19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 0x7efdec19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56592/0 (socket says 172.21.15.62:56592) 2023-07-14T16:15:32.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdebfff700 1 -- 172.21.15.62:0/2194627124 learned_addr learned my addr 172.21.15.62:0/2194627124 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.406+0000 7efdeaffd700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdec100cd0 0x7efdec195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efdec0ffae0 0x7efdec1958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdebfff700 1 -- 172.21.15.62:0/2194627124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdec100cd0 msgr2=0x7efdec195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdebfff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdec100cd0 0x7efdec195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdebfff700 1 -- 172.21.15.62:0/2194627124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efdec0ffae0 msgr2=0x7efdec1958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdebfff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efdec0ffae0 0x7efdec1958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdebfff700 1 -- 172.21.15.62:0/2194627124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde4009d00 con 0x7efdec101660 2023-07-14T16:15:32.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efdec0ffae0 0x7efdec1958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:32.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdebfff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 0x7efdec19ce50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efde4009fd0 tx=0x7efde400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efde8ff9700 1 -- 172.21.15.62:0/2194627124 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde400e040 con 0x7efdec101660 2023-07-14T16:15:32.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efde8ff9700 1 -- 172.21.15.62:0/2194627124 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efde4003e10 con 0x7efdec101660 2023-07-14T16:15:32.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdf1fff700 1 -- 172.21.15.62:0/2194627124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec19d970 con 0x7efdec101660 2023-07-14T16:15:32.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efde8ff9700 1 -- 172.21.15.62:0/2194627124 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde401b6c0 con 0x7efdec101660 2023-07-14T16:15:32.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.407+0000 7efdf1fff700 1 -- 172.21.15.62:0/2194627124 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec19dd40 con 0x7efdec101660 2023-07-14T16:15:32.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.411+0000 7efde8ff9700 1 -- 172.21.15.62:0/2194627124 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efde401d990 con 0x7efdec101660 2023-07-14T16:15:32.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efe058dbb80 1 Processor -- start 2023-07-14T16:15:32.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efe058dbb80 1 -- start start 2023-07-14T16:15:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a0103e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 0x559a0103f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efde8ff9700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efdd40a1150 0x7efdd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a01040090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efe058dbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00f43c60 con 0x559a0103e480 2023-07-14T16:15:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efe058dbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00f969f0 con 0x559a0103fc20 2023-07-14T16:15:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.412+0000 7efe058dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00f8fe80 con 0x559a0103edd0 2023-07-14T16:15:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a0103e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efde8ff9700 1 -- 172.21.15.62:0/2194627124 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efde40cddb0 con 0x7efdec101660 2023-07-14T16:15:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a0103e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56594/0 (socket says 172.21.15.62:56594) 2023-07-14T16:15:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdeaffd700 1 -- 172.21.15.62:0/275104271 learned_addr learned my addr 172.21.15.62:0/275104271 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efdd40a1150 0x7efdd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdebfff700 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a01040090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdeaffd700 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 0x559a0103f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdebfff700 1 -- 172.21.15.62:0/275104271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 msgr2=0x559a0103f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdebfff700 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 0x559a0103f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdebfff700 1 -- 172.21.15.62:0/275104271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 msgr2=0x559a0103e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdebfff700 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a0103e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdebfff700 1 -- 172.21.15.62:0/275104271 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a00dbdd90 con 0x559a0103fc20 2023-07-14T16:15:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdeaffd700 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a0103e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:15:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efdd40a1150 0x7efdd40a3600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efddc005fd0 tx=0x7efddc005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.413+0000 7efdebfff700 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a01040090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efde4097700 tx=0x7efde409adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=148794ea506561ff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efdd1ffb700 1 -- 172.21.15.62:0/275104271 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde40a10f0 con 0x559a0103fc20 2023-07-14T16:15:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efdd1ffb700 1 -- 172.21.15.62:0/275104271 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efde40ad670 con 0x559a0103fc20 2023-07-14T16:15:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efdd1ffb700 1 -- 172.21.15.62:0/275104271 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde409f2c0 con 0x559a0103fc20 2023-07-14T16:15:32.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efe058dbb80 1 -- 172.21.15.62:0/275104271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 msgr2=0x559a01040090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efe058dbb80 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a01040090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efde4097700 tx=0x7efde409adc0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efe058dbb80 1 -- 172.21.15.62:0/275104271 shutdown_connections 2023-07-14T16:15:32.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efe058dbb80 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a01040090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efe058dbb80 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a0103e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efe058dbb80 1 --2- 172.21.15.62:0/275104271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 0x559a0103f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.414+0000 7efe058dbb80 1 -- 172.21.15.62:0/275104271 >> 172.21.15.62:0/275104271 conn(0x559a01033d50 msgr2=0x559a01039180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.415+0000 7efe058dbb80 1 -- 172.21.15.62:0/275104271 shutdown_connections 2023-07-14T16:15:32.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.415+0000 7efe058dbb80 1 -- 172.21.15.62:0/275104271 wait complete. 2023-07-14T16:15:32.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.415+0000 7efe058dbb80 1 -- start start 2023-07-14T16:15:32.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.415+0000 7efe058dbb80 1 Processor -- start 2023-07-14T16:15:32.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103e480 0x559a0104c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a0104c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a010537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efe058dbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00f2fa70 con 0x559a0103edd0 2023-07-14T16:15:32.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efe058dbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00f2f770 con 0x559a0103fc20 2023-07-14T16:15:32.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efe058dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00e138d0 con 0x559a0103e480 2023-07-14T16:15:32.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efe058dbb80 10 client.0 _ll_register_callbacks cb 0x559a00f2d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:32.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a0104c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a0104c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56606/0 (socket says 172.21.15.62:56606) 2023-07-14T16:15:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeb7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103e480 0x559a0104c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeaffd700 1 -- 172.21.15.62:0/4140759732 learned_addr learned my addr 172.21.15.62:0/4140759732 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdebfff700 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a010537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeaffd700 1 -- 172.21.15.62:0/4140759732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103e480 msgr2=0x559a0104c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeaffd700 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103e480 0x559a0104c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeaffd700 1 -- 172.21.15.62:0/4140759732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 msgr2=0x559a010537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeaffd700 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a010537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.416+0000 7efdeaffd700 1 -- 172.21.15.62:0/4140759732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde0009db0 con 0x559a0103edd0 2023-07-14T16:15:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.417+0000 7efdeaffd700 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a0104c7b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efde0011b50 tx=0x7efde0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8f3d5b99d151af server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.417+0000 7efdcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.417+0000 7efdcaffd700 1 -- 172.21.15.62:0/4140759732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000fca0 con 0x559a0103edd0 2023-07-14T16:15:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.417+0000 7efe058dbb80 1 -- 172.21.15.62:0/4140759732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a00ee94a0 con 0x559a0103edd0 2023-07-14T16:15:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.417+0000 7efdcaffd700 1 -- 172.21.15.62:0/4140759732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efde000fe00 con 0x559a0103edd0 2023-07-14T16:15:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.417+0000 7efdcaffd700 1 -- 172.21.15.62:0/4140759732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efde0009610 con 0x559a0103edd0 2023-07-14T16:15:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.417+0000 7efe058dbb80 10 client.15312 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.417+0000 7efe058dbb80 1 -- 172.21.15.62:0/4140759732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7efdec19dd40 con 0x559a0103edd0 2023-07-14T16:15:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.418+0000 7efdcaffd700 1 -- 172.21.15.62:0/4140759732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559a00ee94a0 con 0x559a0103edd0 2023-07-14T16:15:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.418+0000 7efdcaffd700 1 -- 172.21.15.62:0/4140759732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efde0029590 con 0x559a0103edd0 2023-07-14T16:15:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.418+0000 7efe058dbb80 10 client.15312 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.418+0000 7efe058dbb80 10 client.15312 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T16:15:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.418+0000 7efe058dbb80 20 client.15312 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.418+0000 7efe058dbb80 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x559a01057550 0x559a01056950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.418+0000 7efe058dbb80 4 client.15312 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:15:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.418+0000 7efe058dbb80 1 -- 172.21.15.62:0/4140759732 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a00f88e60 con 0x559a01057550 2023-07-14T16:15:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.419+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x559a01057550 0x559a01056950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.419+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x559a01057550 0x559a01056950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2aed6ff468ed2982 server_cookie=7895b4f576e1b366 in_seq=0 out_seq=0 2023-07-14T16:15:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.419+0000 7efdcaffd700 10 client.15312 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:15:32.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efdcaffd700 1 -- 172.21.15.62:0/4140759732 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a00f88e60 con 0x559a01057550 2023-07-14T16:15:32.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efdcaffd700 10 client.15312 handle_command_reply: tid=0 2023-07-14T16:15:32.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 client.15312 shutdown 2023-07-14T16:15:32.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 -- 172.21.15.62:0/4140759732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 msgr2=0x559a0104c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a0104c7b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efde0011b50 tx=0x7efde0011e60 comp rx=0 tx=0).stop 2023-07-14T16:15:32.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 -- 172.21.15.62:0/4140759732 shutdown_connections 2023-07-14T16:15:32.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x559a01057550 0x559a01056950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a010537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103e480 0x559a0104c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efdcaffd700 0 client.15312 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:15:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 --2- 172.21.15.62:0/4140759732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a0104c7b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 -- 172.21.15.62:0/4140759732 >> 172.21.15.62:0/4140759732 conn(0x559a01033d50 msgr2=0x559a01038860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 -- 172.21.15.62:0/4140759732 shutdown_connections 2023-07-14T16:15:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.420+0000 7efe058dbb80 1 -- 172.21.15.62:0/4140759732 wait complete. 2023-07-14T16:15:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.421+0000 7efe058dbb80 20 client.15312 trim_cache size 0 max 16384 2023-07-14T16:15:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efe058dbb80 1 Processor -- start 2023-07-14T16:15:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efe058dbb80 1 -- start start 2023-07-14T16:15:32.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a01056950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 0x559a01056e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a010537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efe058dbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00f88e60 con 0x559a0103e480 2023-07-14T16:15:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efe058dbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00ee94a0 con 0x559a0103fc20 2023-07-14T16:15:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efe058dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a00f5e790 con 0x559a0103edd0 2023-07-14T16:15:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efdeb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a01056950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efdeb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a01056950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56620/0 (socket says 172.21.15.62:56620) 2023-07-14T16:15:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.439+0000 7efdeb7fe700 1 -- 172.21.15.62:0/779140517 learned_addr learned my addr 172.21.15.62:0/779140517 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdebfff700 1 --2- 172.21.15.62:0/779140517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a010537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdeb7fe700 1 -- 172.21.15.62:0/779140517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 msgr2=0x559a01056e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/779140517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 0x559a01056e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdeb7fe700 1 -- 172.21.15.62:0/779140517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 msgr2=0x559a010537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/779140517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a010537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdeb7fe700 1 -- 172.21.15.62:0/779140517 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a00da6c10 con 0x559a0103e480 2023-07-14T16:15:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/779140517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a01056950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x559a0104c210 tx=0x7efddc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfdc3c8c68b35fd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdd27fc700 1 -- 172.21.15.62:0/779140517 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc024040 con 0x559a0103e480 2023-07-14T16:15:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdd27fc700 1 -- 172.21.15.62:0/779140517 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efddc011be0 con 0x559a0103e480 2023-07-14T16:15:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.440+0000 7efdd27fc700 1 -- 172.21.15.62:0/779140517 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0257f0 con 0x559a0103e480 2023-07-14T16:15:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 -- 172.21.15.62:0/779140517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 msgr2=0x559a01056950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 --2- 172.21.15.62:0/779140517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a01056950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x559a0104c210 tx=0x7efddc015000 comp rx=0 tx=0).stop 2023-07-14T16:15:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 -- 172.21.15.62:0/779140517 shutdown_connections 2023-07-14T16:15:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 --2- 172.21.15.62:0/779140517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103fc20 0x559a010537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 --2- 172.21.15.62:0/779140517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103e480 0x559a01056950 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 --2- 172.21.15.62:0/779140517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103edd0 0x559a01056e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 -- 172.21.15.62:0/779140517 >> 172.21.15.62:0/779140517 conn(0x559a01033d50 msgr2=0x559a01054020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 -- 172.21.15.62:0/779140517 shutdown_connections 2023-07-14T16:15:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 -- 172.21.15.62:0/779140517 wait complete. 2023-07-14T16:15:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.441+0000 7efe058dbb80 1 -- start start 2023-07-14T16:15:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efe058dbb80 1 Processor -- start 2023-07-14T16:15:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103e480 0x559a010517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a010a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efe058dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103fc20 0x559a010ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efe058dbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a010abe50 con 0x559a0103edd0 2023-07-14T16:15:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efe058dbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a010abf90 con 0x559a0103e480 2023-07-14T16:15:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efe058dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a010ac150 con 0x559a0103fc20 2023-07-14T16:15:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efe058dbb80 10 client.0 _ll_register_callbacks cb 0x559a00f2d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efdeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a010a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efdeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a010a93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56624/0 (socket says 172.21.15.62:56624) 2023-07-14T16:15:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efdeaffd700 1 -- 172.21.15.62:0/2300666278 learned_addr learned my addr 172.21.15.62:0/2300666278 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.442+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103e480 0x559a010517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdeaffd700 1 -- 172.21.15.62:0/2300666278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103fc20 msgr2=0x559a010ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdeaffd700 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103fc20 0x559a010ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdeaffd700 1 -- 172.21.15.62:0/2300666278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103e480 msgr2=0x559a010517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdeaffd700 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103e480 0x559a010517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdeaffd700 1 -- 172.21.15.62:0/2300666278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc00ed50 con 0x559a0103edd0 2023-07-14T16:15:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdeaffd700 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a010a93c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efde0011ea0 tx=0x7efde0029d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ce72d55a7130cf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdcaffd700 1 -- 172.21.15.62:0/2300666278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0009610 con 0x559a0103edd0 2023-07-14T16:15:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efe058dbb80 1 -- 172.21.15.62:0/2300666278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efde40ad670 con 0x559a0103edd0 2023-07-14T16:15:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdcaffd700 1 -- 172.21.15.62:0/2300666278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efde0000e40 con 0x559a0103edd0 2023-07-14T16:15:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.443+0000 7efdcaffd700 1 -- 172.21.15.62:0/2300666278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efde0029e20 con 0x559a0103edd0 2023-07-14T16:15:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efe058dbb80 10 client.15318 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efe058dbb80 1 -- 172.21.15.62:0/2300666278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559a00de4200 con 0x559a0103edd0 2023-07-14T16:15:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efdcaffd700 1 -- 172.21.15.62:0/2300666278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efde40ad670 con 0x559a0103edd0 2023-07-14T16:15:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efdcaffd700 1 -- 172.21.15.62:0/2300666278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efde0005370 con 0x559a0103edd0 2023-07-14T16:15:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efe058dbb80 10 client.15318 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efe058dbb80 10 client.15318 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T16:15:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efe058dbb80 20 client.15318 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efe058dbb80 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x559a010b0f40 0x559a010ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efe058dbb80 4 client.15318 mds_command: new command op to 24565 tid=0[{"prefix": "status"}] 2023-07-14T16:15:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.444+0000 7efe058dbb80 1 -- 172.21.15.62:0/2300666278 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559a00f4b000 con 0x559a010b0f40 2023-07-14T16:15:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.445+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x559a010b0f40 0x559a010ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.445+0000 7efdeb7fe700 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x559a010b0f40 0x559a010ad870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8349ce7a38006f7a server_cookie=4e256d0bbda106e in_seq=0 out_seq=0 2023-07-14T16:15:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.445+0000 7efdcaffd700 10 client.15318 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:15:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.445+0000 7efdcaffd700 1 -- 172.21.15.62:0/2300666278 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x559a00f4b000 con 0x559a010b0f40 2023-07-14T16:15:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.445+0000 7efdcaffd700 10 client.15318 handle_command_reply: tid=0 2023-07-14T16:15:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 client.15318 shutdown 2023-07-14T16:15:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 -- 172.21.15.62:0/2300666278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 msgr2=0x559a010a93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a010a93c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efde0011ea0 tx=0x7efde0029d10 comp rx=0 tx=0).stop 2023-07-14T16:15:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 -- 172.21.15.62:0/2300666278 shutdown_connections 2023-07-14T16:15:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x559a010b0f40 0x559a010ad870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a0103fc20 0x559a010ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a0103e480 0x559a010517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efdcaffd700 0 client.15318 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:15:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 --2- 172.21.15.62:0/2300666278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a0103edd0 0x559a010a93c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 -- 172.21.15.62:0/2300666278 >> 172.21.15.62:0/2300666278 conn(0x559a01033d50 msgr2=0x559a01056950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 -- 172.21.15.62:0/2300666278 shutdown_connections 2023-07-14T16:15:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 1 -- 172.21.15.62:0/2300666278 wait complete. 2023-07-14T16:15:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.446+0000 7efe058dbb80 20 client.15318 trim_cache size 0 max 16384 2023-07-14T16:15:32.431 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:32.432 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:15:32.432 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2023-07-14T16:15:32.432 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24565, 2023-07-14T16:15:32.432 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2023-07-14T16:15:32.432 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2023-07-14T16:15:32.432 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2023-07-14T16:15:32.433 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 89.860179185000007, 2023-07-14T16:15:32.433 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 23, 2023-07-14T16:15:32.433 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 88, 2023-07-14T16:15:32.433 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 86, 2023-07-14T16:15:32.433 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 109.033540907 2023-07-14T16:15:32.433 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 -- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efdd40a1150 msgr2=0x7efdd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efdd40a1150 0x7efdd40a3600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efddc005fd0 tx=0x7efddc005f00 comp rx=0 tx=0).stop 2023-07-14T16:15:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 -- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 msgr2=0x7efdec19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 0x7efdec19ce50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efde4009fd0 tx=0x7efde400bd30 comp rx=0 tx=0).stop 2023-07-14T16:15:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 -- 172.21.15.62:0/2194627124 shutdown_connections 2023-07-14T16:15:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdec101660 0x7efdec19ce50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efdd40a1150 0x7efdd40a3600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efdec0ffae0 0x7efdec1958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 --2- 172.21.15.62:0/2194627124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efdec100cd0 0x7efdec195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 -- 172.21.15.62:0/2194627124 >> 172.21.15.62:0/2194627124 conn(0x7efdec0fb0e0 msgr2=0x7efdec0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 -- 172.21.15.62:0/2194627124 shutdown_connections 2023-07-14T16:15:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.447+0000 7efdf1fff700 1 -- 172.21.15.62:0/2194627124 wait complete. 2023-07-14T16:15:32.443 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T16:15:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 -- 172.21.15.62:0/4146205978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c10c0c0 msgr2=0x7f931c10e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 --2- 172.21.15.62:0/4146205978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c10c0c0 0x7f931c10e4a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f931400b0a0 tx=0x7f931400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 -- 172.21.15.62:0/4146205978 shutdown_connections 2023-07-14T16:15:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 --2- 172.21.15.62:0/4146205978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c10c0c0 0x7f931c10e4a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 --2- 172.21.15.62:0/4146205978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c074550 0x7f931c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 --2- 172.21.15.62:0/4146205978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f931c072f70 0x7f931c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 -- 172.21.15.62:0/4146205978 >> 172.21.15.62:0/4146205978 conn(0x7f931c0fb130 msgr2=0x7f931c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 -- 172.21.15.62:0/4146205978 shutdown_connections 2023-07-14T16:15:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.672+0000 7f93237ff700 1 -- 172.21.15.62:0/4146205978 wait complete. 2023-07-14T16:15:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.673+0000 7f93237ff700 1 Processor -- start 2023-07-14T16:15:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.673+0000 7f93237ff700 1 -- start start 2023-07-14T16:15:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f93237ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c072f70 0x7f931c19bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f93237ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c074550 0x7f931c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f93237ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f931c10c0c0 0x7f931c1a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f93237ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c1a3a20 con 0x7f931c074550 2023-07-14T16:15:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f93237ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c1a3b60 con 0x7f931c10c0c0 2023-07-14T16:15:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f93237ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c1a3cf0 con 0x7f931c072f70 2023-07-14T16:15:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9320d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c074550 0x7f931c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9320d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c074550 0x7f931c19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56640/0 (socket says 172.21.15.62:56640) 2023-07-14T16:15:32.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f932159b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c072f70 0x7f931c19bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9320d9a700 1 -- 172.21.15.62:0/2960828854 learned_addr learned my addr 172.21.15.62:0/2960828854 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9321d9c700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f931c10c0c0 0x7f931c1a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9320d9a700 1 -- 172.21.15.62:0/2960828854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c072f70 msgr2=0x7f931c19bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9320d9a700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c072f70 0x7f931c19bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9320d9a700 1 -- 172.21.15.62:0/2960828854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f931c10c0c0 msgr2=0x7f931c1a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9320d9a700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f931c10c0c0 0x7f931c1a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.674+0000 7f9320d9a700 1 -- 172.21.15.62:0/2960828854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009d00 con 0x7f931c074550 2023-07-14T16:15:32.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.675+0000 7f9321d9c700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f931c10c0c0 0x7f931c1a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:32.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.675+0000 7f9320d9a700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c074550 0x7f931c19c3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f930c00d980 tx=0x7f930c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.675+0000 7f93127fc700 1 -- 172.21.15.62:0/2960828854 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00b950 con 0x7f931c074550 2023-07-14T16:15:32.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.675+0000 7f93127fc700 1 -- 172.21.15.62:0/2960828854 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f930c007620 con 0x7f931c074550 2023-07-14T16:15:32.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.675+0000 7f93127fc700 1 -- 172.21.15.62:0/2960828854 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00f3c0 con 0x7f931c074550 2023-07-14T16:15:32.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.675+0000 7f93237ff700 1 -- 172.21.15.62:0/2960828854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c1a3fd0 con 0x7f931c074550 2023-07-14T16:15:32.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.675+0000 7f93237ff700 1 -- 172.21.15.62:0/2960828854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c1a4400 con 0x7f931c074550 2023-07-14T16:15:32.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.678+0000 7f93127fc700 1 -- 172.21.15.62:0/2960828854 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f930c00bab0 con 0x7f931c074550 2023-07-14T16:15:32.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93371c1b80 1 Processor -- start 2023-07-14T16:15:32.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93127fc700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93080a11d0 0x7f93080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93371c1b80 1 -- start start 2023-07-14T16:15:32.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93127fc700 1 -- 172.21.15.62:0/2960828854 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f930c0cca70 con 0x7f931c074550 2023-07-14T16:15:32.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f9321d9c700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93080a11d0 0x7f93080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc26d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ddd0 0x562dcc26e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ec20 0x562dcc26f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93371c1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc172c60 con 0x562dcc26d480 2023-07-14T16:15:32.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93371c1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc1c59f0 con 0x562dcc26ec20 2023-07-14T16:15:32.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.680+0000 7f93371c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc1bee80 con 0x562dcc26ddd0 2023-07-14T16:15:32.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f932159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc26d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f932159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc26d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56650/0 (socket says 172.21.15.62:56650) 2023-07-14T16:15:32.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f932159b700 1 -- 172.21.15.62:0/166054539 learned_addr learned my addr 172.21.15.62:0/166054539 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f9320d9a700 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ddd0 0x562dcc26e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f9321d9c700 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ec20 0x562dcc26f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f9321d9c700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93080a11d0 0x7f93080a3680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f93140062a0 tx=0x7f93140077a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f932159b700 1 -- 172.21.15.62:0/166054539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ddd0 msgr2=0x562dcc26e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f932159b700 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ddd0 0x562dcc26e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f932159b700 1 -- 172.21.15.62:0/166054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ec20 msgr2=0x562dcc26f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f932159b700 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ec20 0x562dcc26f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.681+0000 7f932159b700 1 -- 172.21.15.62:0/166054539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dcbfecd90 con 0x562dcc26d480 2023-07-14T16:15:32.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.682+0000 7f932159b700 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc26d890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f931800bcb0 tx=0x7f93180103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c79251979028dcb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.682+0000 7f93077fe700 1 -- 172.21.15.62:0/166054539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318010b90 con 0x562dcc26d480 2023-07-14T16:15:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.682+0000 7f93077fe700 1 -- 172.21.15.62:0/166054539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9318015070 con 0x562dcc26d480 2023-07-14T16:15:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.682+0000 7f93077fe700 1 -- 172.21.15.62:0/166054539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318003ce0 con 0x562dcc26d480 2023-07-14T16:15:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 -- 172.21.15.62:0/166054539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 msgr2=0x562dcc26d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc26d890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f931800bcb0 tx=0x7f93180103d0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 -- 172.21.15.62:0/166054539 shutdown_connections 2023-07-14T16:15:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ec20 0x562dcc26f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc26d890 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 --2- 172.21.15.62:0/166054539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ddd0 0x562dcc26e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 -- 172.21.15.62:0/166054539 >> 172.21.15.62:0/166054539 conn(0x562dcc262d50 msgr2=0x562dcc268180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 -- 172.21.15.62:0/166054539 shutdown_connections 2023-07-14T16:15:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 -- 172.21.15.62:0/166054539 wait complete. 2023-07-14T16:15:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.683+0000 7f93371c1b80 1 -- start start 2023-07-14T16:15:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f93371c1b80 1 Processor -- start 2023-07-14T16:15:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc2770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 0x562dcc277620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 0x562dcc27e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f93371c1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc15ea70 con 0x562dcc26d480 2023-07-14T16:15:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f93371c1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc15e770 con 0x562dcc26ddd0 2023-07-14T16:15:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f93371c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc0428d0 con 0x562dcc26ec20 2023-07-14T16:15:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f93371c1b80 10 client.0 _ll_register_callbacks cb 0x562dcc15c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f932159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc2770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f932159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc2770e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56658/0 (socket says 172.21.15.62:56658) 2023-07-14T16:15:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f932159b700 1 -- 172.21.15.62:0/1379961858 learned_addr learned my addr 172.21.15.62:0/1379961858 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f9321d9c700 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 0x562dcc27e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.684+0000 7f9320d9a700 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 0x562dcc277620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f932159b700 1 -- 172.21.15.62:0/1379961858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 msgr2=0x562dcc27e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f932159b700 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 0x562dcc27e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f932159b700 1 -- 172.21.15.62:0/1379961858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 msgr2=0x562dcc277620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f932159b700 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 0x562dcc277620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f932159b700 1 -- 172.21.15.62:0/1379961858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931800b8e0 con 0x562dcc26d480 2023-07-14T16:15:32.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f932159b700 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc2770e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f9318005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10ced25cb19c1c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f9304ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:32.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f9304ff9700 1 -- 172.21.15.62:0/1379961858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318005450 con 0x562dcc26d480 2023-07-14T16:15:32.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f9304ff9700 1 -- 172.21.15.62:0/1379961858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9318017070 con 0x562dcc26d480 2023-07-14T16:15:32.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.685+0000 7f93371c1b80 1 -- 172.21.15.62:0/1379961858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562dcc1184a0 con 0x562dcc26d480 2023-07-14T16:15:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f9304ff9700 1 -- 172.21.15.62:0/1379961858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93180055b0 con 0x562dcc26d480 2023-07-14T16:15:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f93371c1b80 10 client.15330 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f93371c1b80 1 -- 172.21.15.62:0/1379961858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562dcbfd5c10 con 0x562dcc26d480 2023-07-14T16:15:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f9304ff9700 1 -- 172.21.15.62:0/1379961858 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562dcc1184a0 con 0x562dcc26d480 2023-07-14T16:15:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f9304ff9700 1 -- 172.21.15.62:0/1379961858 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9318023070 con 0x562dcc26d480 2023-07-14T16:15:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f93371c1b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:32.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f93371c1b80 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T16:15:32.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f93371c1b80 20 client.15330 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:32.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562dcc282280 0x562dcc2816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f93371c1b80 4 client.15330 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:15:32.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.686+0000 7f93371c1b80 1 -- 172.21.15.62:0/1379961858 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dcc1b7e60 con 0x562dcc282280 2023-07-14T16:15:32.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.687+0000 7f9320d9a700 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562dcc282280 0x562dcc2816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.687+0000 7f9320d9a700 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562dcc282280 0x562dcc2816b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30f2e618722118b server_cookie=bdf14866dfce72cc in_seq=0 out_seq=0 2023-07-14T16:15:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.687+0000 7f9304ff9700 10 client.15330 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f9304ff9700 1 -- 172.21.15.62:0/1379961858 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562dcc1b7e60 con 0x562dcc282280 2023-07-14T16:15:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f9304ff9700 10 client.15330 handle_command_reply: tid=0 2023-07-14T16:15:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 client.15330 shutdown 2023-07-14T16:15:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 -- 172.21.15.62:0/1379961858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 msgr2=0x562dcc2770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc2770e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f9318005dd0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 -- 172.21.15.62:0/1379961858 shutdown_connections 2023-07-14T16:15:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 0x562dcc27e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26d480 0x562dcc2770e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562dcc282280 0x562dcc2816b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1379961858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 0x562dcc277620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 -- 172.21.15.62:0/1379961858 >> 172.21.15.62:0/1379961858 conn(0x562dcc262d50 msgr2=0x562dcc267860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f9304ff9700 0 client.15330 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:32.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.688+0000 7f93371c1b80 1 -- 172.21.15.62:0/1379961858 shutdown_connections 2023-07-14T16:15:32.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.689+0000 7f93371c1b80 1 -- 172.21.15.62:0/1379961858 wait complete. 2023-07-14T16:15:32.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.689+0000 7f93371c1b80 20 client.15330 trim_cache size 0 max 16384 2023-07-14T16:15:32.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.708+0000 7f93371c1b80 1 Processor -- start 2023-07-14T16:15:32.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.708+0000 7f93371c1b80 1 -- start start 2023-07-14T16:15:32.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26d480 0x562dcc275e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 0x562dcc27e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ec20 0x562dcc275030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f93371c1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc276cb0 con 0x562dcc26ec20 2023-07-14T16:15:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f93371c1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc18d790 con 0x562dcc26ddd0 2023-07-14T16:15:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f93371c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc04ede0 con 0x562dcc26d480 2023-07-14T16:15:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9321d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ec20 0x562dcc275030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9321d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ec20 0x562dcc275030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56666/0 (socket says 172.21.15.62:56666) 2023-07-14T16:15:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9321d9c700 1 -- 172.21.15.62:0/1664071059 learned_addr learned my addr 172.21.15.62:0/1664071059 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9320d9a700 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 0x562dcc27e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f932159b700 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26d480 0x562dcc275e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9321d9c700 1 -- 172.21.15.62:0/1664071059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26d480 msgr2=0x562dcc275e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9321d9c700 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26d480 0x562dcc275e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9321d9c700 1 -- 172.21.15.62:0/1664071059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 msgr2=0x562dcc27e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9321d9c700 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 0x562dcc27e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.709+0000 7f9321d9c700 1 -- 172.21.15.62:0/1664071059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dcc013200 con 0x562dcc26ec20 2023-07-14T16:15:32.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.710+0000 7f9321d9c700 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ec20 0x562dcc275030 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9314009510 tx=0x7f931401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb12846b0d51d68 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.710+0000 7f9307fff700 1 -- 172.21.15.62:0/1664071059 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314029670 con 0x562dcc26ec20 2023-07-14T16:15:32.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.710+0000 7f9307fff700 1 -- 172.21.15.62:0/1664071059 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9314029c60 con 0x562dcc26ec20 2023-07-14T16:15:32.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.710+0000 7f9307fff700 1 -- 172.21.15.62:0/1664071059 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931401db80 con 0x562dcc26ec20 2023-07-14T16:15:32.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 -- 172.21.15.62:0/1664071059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ec20 msgr2=0x562dcc275030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ec20 0x562dcc275030 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9314009510 tx=0x7f931401f040 comp rx=0 tx=0).stop 2023-07-14T16:15:32.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 -- 172.21.15.62:0/1664071059 shutdown_connections 2023-07-14T16:15:32.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ec20 0x562dcc275030 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26d480 0x562dcc275e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 --2- 172.21.15.62:0/1664071059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26ddd0 0x562dcc27e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 -- 172.21.15.62:0/1664071059 >> 172.21.15.62:0/1664071059 conn(0x562dcc262d50 msgr2=0x562dcc267860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 -- 172.21.15.62:0/1664071059 shutdown_connections 2023-07-14T16:15:32.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 -- 172.21.15.62:0/1664071059 wait complete. 2023-07-14T16:15:32.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.711+0000 7f93371c1b80 1 -- start start 2023-07-14T16:15:32.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f93371c1b80 1 Processor -- start 2023-07-14T16:15:32.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26d480 0x562dcc279a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ddd0 0x562dcc279f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f93371c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 0x562dcc27cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f93371c1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc27d4f0 con 0x562dcc26ddd0 2023-07-14T16:15:32.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f93371c1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc27d660 con 0x562dcc26d480 2023-07-14T16:15:32.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f93371c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc27d820 con 0x562dcc26ec20 2023-07-14T16:15:32.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f93371c1b80 10 client.0 _ll_register_callbacks cb 0x7f930c096170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:32.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f9320d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ddd0 0x562dcc279f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f9320d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ddd0 0x562dcc279f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56682/0 (socket says 172.21.15.62:56682) 2023-07-14T16:15:32.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f9320d9a700 1 -- 172.21.15.62:0/3010531338 learned_addr learned my addr 172.21.15.62:0/3010531338 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:32.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.712+0000 7f932159b700 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26d480 0x562dcc279a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9321d9c700 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 0x562dcc27cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9320d9a700 1 -- 172.21.15.62:0/3010531338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 msgr2=0x562dcc27cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9320d9a700 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 0x562dcc27cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9320d9a700 1 -- 172.21.15.62:0/3010531338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26d480 msgr2=0x562dcc279a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:32.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9320d9a700 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26d480 0x562dcc279a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9320d9a700 1 -- 172.21.15.62:0/3010531338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c10ee50 con 0x562dcc26ddd0 2023-07-14T16:15:32.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9320d9a700 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ddd0 0x562dcc279f90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f930c0968a0 tx=0x7f930c0a2400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343ac7e7ad49f84a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:32.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9304ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:32.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9304ff9700 1 -- 172.21.15.62:0/3010531338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00bd60 con 0x562dcc26ddd0 2023-07-14T16:15:32.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f93371c1b80 1 -- 172.21.15.62:0/3010531338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9318015070 con 0x562dcc26ddd0 2023-07-14T16:15:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9304ff9700 1 -- 172.21.15.62:0/3010531338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f930c099320 con 0x562dcc26ddd0 2023-07-14T16:15:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9304ff9700 1 -- 172.21.15.62:0/3010531338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f930c0994e0 con 0x562dcc26ddd0 2023-07-14T16:15:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f93371c1b80 10 client.15336 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f93371c1b80 1 -- 172.21.15.62:0/3010531338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562dcbfeb260 con 0x562dcc26ddd0 2023-07-14T16:15:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.713+0000 7f9304ff9700 1 -- 172.21.15.62:0/3010531338 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f930c0ca030 con 0x562dcc26ddd0 2023-07-14T16:15:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.714+0000 7f9304ff9700 1 -- 172.21.15.62:0/3010531338 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f930c095c20 con 0x562dcc26ddd0 2023-07-14T16:15:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.714+0000 7f93371c1b80 10 client.15336 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.714+0000 7f93371c1b80 10 client.15336 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T16:15:32.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.714+0000 7f93371c1b80 20 client.15336 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:32.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.714+0000 7f93371c1b80 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562dcc2dba70 0x562dcc2d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:32.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.714+0000 7f93371c1b80 4 client.15336 mds_command: new command op to 24602 tid=0[{"prefix": "status"}] 2023-07-14T16:15:32.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.714+0000 7f93371c1b80 1 -- 172.21.15.62:0/3010531338 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562dcc17a000 con 0x562dcc2dba70 2023-07-14T16:15:32.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.714+0000 7f932159b700 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562dcc2dba70 0x562dcc2d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:32.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.715+0000 7f932159b700 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562dcc2dba70 0x562dcc2d83e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61cf5be861f9f5a2 server_cookie=5230cc753abc955f in_seq=0 out_seq=0 2023-07-14T16:15:32.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.715+0000 7f9304ff9700 10 client.15336 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:32.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.715+0000 7f9304ff9700 1 -- 172.21.15.62:0/3010531338 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x562dcc17a000 con 0x562dcc2dba70 2023-07-14T16:15:32.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.715+0000 7f9304ff9700 10 client.15336 handle_command_reply: tid=0 2023-07-14T16:15:32.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.715+0000 7f93371c1b80 1 client.15336 shutdown 2023-07-14T16:15:32.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.715+0000 7f93371c1b80 1 -- 172.21.15.62:0/3010531338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ddd0 msgr2=0x562dcc279f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.715+0000 7f93371c1b80 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ddd0 0x562dcc279f90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f930c0968a0 tx=0x7f930c0a2400 comp rx=0 tx=0).stop 2023-07-14T16:15:32.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 1 -- 172.21.15.62:0/3010531338 shutdown_connections 2023-07-14T16:15:32.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562dcc26ec20 0x562dcc27cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562dcc2dba70 0x562dcc2d83e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562dcc26d480 0x562dcc279a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f9304ff9700 0 client.15336 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:32.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 1 --2- 172.21.15.62:0/3010531338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dcc26ddd0 0x562dcc279f90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 1 -- 172.21.15.62:0/3010531338 >> 172.21.15.62:0/3010531338 conn(0x562dcc262d50 msgr2=0x562dcc2816b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 1 -- 172.21.15.62:0/3010531338 shutdown_connections 2023-07-14T16:15:32.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 1 -- 172.21.15.62:0/3010531338 wait complete. 2023-07-14T16:15:32.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93371c1b80 20 client.15336 trim_cache size 0 max 16384 2023-07-14T16:15:32.702 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:32.702 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1356e48a-2260-11ee-9b33-001a4aab830c", 2023-07-14T16:15:32.702 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2023-07-14T16:15:32.702 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24602, 2023-07-14T16:15:32.702 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2023-07-14T16:15:32.702 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2023-07-14T16:15:32.703 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2023-07-14T16:15:32.703 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 93.157742807999995, 2023-07-14T16:15:32.703 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 23, 2023-07-14T16:15:32.703 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 88, 2023-07-14T16:15:32.703 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 86, 2023-07-14T16:15:32.703 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 99.717926921 2023-07-14T16:15:32.703 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:32.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93237ff700 1 -- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93080a11d0 msgr2=0x7f93080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93237ff700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93080a11d0 0x7f93080a3680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f93140062a0 tx=0x7f93140077a0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93237ff700 1 -- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c074550 msgr2=0x7f931c19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:32.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93237ff700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c074550 0x7f931c19c3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f930c00d980 tx=0x7f930c00dc90 comp rx=0 tx=0).stop 2023-07-14T16:15:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93237ff700 1 -- 172.21.15.62:0/2960828854 shutdown_connections 2023-07-14T16:15:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93237ff700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93080a11d0 0x7f93080a3680 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93237ff700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f931c10c0c0 0x7f931c1a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.716+0000 7f93237ff700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c072f70 0x7f931c19bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.717+0000 7f93237ff700 1 --2- 172.21.15.62:0/2960828854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f931c074550 0x7f931c19c3e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.717+0000 7f93237ff700 1 -- 172.21.15.62:0/2960828854 >> 172.21.15.62:0/2960828854 conn(0x7f931c0fb130 msgr2=0x7f931c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.717+0000 7f93237ff700 1 -- 172.21.15.62:0/2960828854 shutdown_connections 2023-07-14T16:15:32.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:32.717+0000 7f93237ff700 1 -- 172.21.15.62:0/2960828854 wait complete. 2023-07-14T16:15:32.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:32 smithi062 ceph-mon[95543]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:15:32.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:32 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3000776330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:32.711 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-07-14T16:15:32.737 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-07-14T16:15:32.737 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-07-14T16:15:32.738 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2023-07-14T16:15:32.738 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T16:15:32.738 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:32.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:32.739 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-07-14T16:15:32.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:32.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-07-14T16:15:32.904 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2023-07-14T16:15:32.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T16:15:32.908 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-07-14T16:15:32.908 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph fs ls 2023-07-14T16:15:33.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:32 smithi164 ceph-mon[97427]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:15:33.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:32 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3000776330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:33.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:32 smithi081 ceph-mon[127795]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:15:33.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:32 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3000776330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:15:34.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:34 smithi062 ceph-mon[95543]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:15:35.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:34 smithi164 ceph-mon[97427]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:15:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:34 smithi081 ceph-mon[127795]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:15:35.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 -- 172.21.15.62:0/1827584384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f8104ea0 msgr2=0x7f55f810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:35.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 --2- 172.21.15.62:0/1827584384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f8104ea0 0x7f55f810ca20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b0d0 tx=0x7f55f000b3e0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 -- 172.21.15.62:0/1827584384 shutdown_connections 2023-07-14T16:15:35.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 --2- 172.21.15.62:0/1827584384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f8104ea0 0x7f55f810ca20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 --2- 172.21.15.62:0/1827584384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f80ffe00 0x7f55f81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 --2- 172.21.15.62:0/1827584384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f81007a0 0x7f55f8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 -- 172.21.15.62:0/1827584384 >> 172.21.15.62:0/1827584384 conn(0x7f55f80fb660 msgr2=0x7f55f80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:35.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 -- 172.21.15.62:0/1827584384 shutdown_connections 2023-07-14T16:15:35.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.256+0000 7f55ff44d700 1 -- 172.21.15.62:0/1827584384 wait complete. 2023-07-14T16:15:35.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.257+0000 7f55ff44d700 1 Processor -- start 2023-07-14T16:15:35.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.257+0000 7f55ff44d700 1 -- start start 2023-07-14T16:15:35.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55ff44d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f80ffe00 0x7f55f8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:35.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55ff44d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f81007a0 0x7f55f8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:35.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55ff44d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f8104ea0 0x7f55f80732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:35.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55ff44d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f81a40e0 con 0x7f55f81007a0 2023-07-14T16:15:35.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55ff44d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f81a4250 con 0x7f55f8104ea0 2023-07-14T16:15:35.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55ff44d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f81a4410 con 0x7f55f80ffe00 2023-07-14T16:15:35.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55fc9e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f81007a0 0x7f55f8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:35.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55fc9e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f81007a0 0x7f55f8072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38456/0 (socket says 172.21.15.62:38456) 2023-07-14T16:15:35.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55fc9e8700 1 -- 172.21.15.62:0/4245256266 learned_addr learned my addr 172.21.15.62:0/4245256266 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:35.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55fd1e9700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f80ffe00 0x7f55f8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:35.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.258+0000 7f55fd9ea700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f8104ea0 0x7f55f80732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:35.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55fc9e8700 1 -- 172.21.15.62:0/4245256266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f80ffe00 msgr2=0x7f55f8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:35.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55fc9e8700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f80ffe00 0x7f55f8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55fc9e8700 1 -- 172.21.15.62:0/4245256266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f8104ea0 msgr2=0x7f55f80732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:35.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55fc9e8700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f8104ea0 0x7f55f80732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55fc9e8700 1 -- 172.21.15.62:0/4245256266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0009d00 con 0x7f55f81007a0 2023-07-14T16:15:35.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55fc9e8700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f81007a0 0x7f55f8072d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e970 tx=0x7f55e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:35.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55ee7fc700 1 -- 172.21.15.62:0/4245256266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8009790 con 0x7f55f81007a0 2023-07-14T16:15:35.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f81a46f0 con 0x7f55f81007a0 2023-07-14T16:15:35.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55ee7fc700 1 -- 172.21.15.62:0/4245256266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55e800c7e0 con 0x7f55f81007a0 2023-07-14T16:15:35.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55ee7fc700 1 -- 172.21.15.62:0/4245256266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8005440 con 0x7f55f81007a0 2023-07-14T16:15:35.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.259+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f81a4c40 con 0x7f55f81007a0 2023-07-14T16:15:35.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.261+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f804e210 con 0x7f55f81007a0 2023-07-14T16:15:35.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.263+0000 7f55ee7fc700 1 -- 172.21.15.62:0/4245256266 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f55e800c950 con 0x7f55f81007a0 2023-07-14T16:15:35.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.264+0000 7f55ee7fc700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f55e40a1150 0x7f55e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:35.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.264+0000 7f55ee7fc700 1 -- 172.21.15.62:0/4245256266 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f55e8014070 con 0x7f55f81007a0 2023-07-14T16:15:35.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.264+0000 7f55fd1e9700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f55e40a1150 0x7f55e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:35.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.265+0000 7f55fd1e9700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f55e40a1150 0x7f55e40a3600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009c60 tx=0x7f55f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:35.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.268+0000 7f55ee7fc700 1 -- 172.21.15.62:0/4245256266 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f55e80961c0 con 0x7f55f81007a0 2023-07-14T16:15:35.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.523+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f55f81a4ec0 con 0x7f55f81007a0 2023-07-14T16:15:35.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.524+0000 7f55ee7fc700 1 -- 172.21.15.62:0/4245256266 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f55e80997e0 con 0x7f55f81007a0 2023-07-14T16:15:35.502 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-14T16:15:35.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f55e40a1150 msgr2=0x7f55e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:35.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f55e40a1150 0x7f55e40a3600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009c60 tx=0x7f55f4009260 comp rx=0 tx=0).stop 2023-07-14T16:15:35.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f81007a0 msgr2=0x7f55f8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:35.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f81007a0 0x7f55f8072d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e970 tx=0x7f55e800ec80 comp rx=0 tx=0).stop 2023-07-14T16:15:35.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 shutdown_connections 2023-07-14T16:15:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f55e40a1150 0x7f55e40a3600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f8104ea0 0x7f55f80732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55f80ffe00 0x7f55f8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 --2- 172.21.15.62:0/4245256266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55f81007a0 0x7f55f8072d50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.526+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 >> 172.21.15.62:0/4245256266 conn(0x7f55f80fb660 msgr2=0x7f55f80fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.527+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 shutdown_connections 2023-07-14T16:15:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:35.527+0000 7f55ff44d700 1 -- 172.21.15.62:0/4245256266 wait complete. 2023-07-14T16:15:35.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:35 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/4245256266' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T16:15:35.922 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T16:15:35.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T16:15:35.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2023-07-14T16:15:35.923 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T16:15:35.923 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:35.923 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T16:15:35.923 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T16:15:35.923 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T16:15:35.924 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T16:15:35.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:35.924 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2023-07-14T16:15:35.969 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T16:15:35.969 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T16:15:35.970 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T16:15:35.970 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-07-14T16:15:35.970 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2023-07-14T16:15:35.970 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-07-14T16:15:35.970 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T16:15:35.970 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2023-07-14T16:15:35.971 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2023-07-14T16:15:35.971 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T16:15:35.971 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2023-07-14T16:15:35.971 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2023-07-14T16:15:35.971 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T16:15:35.971 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2023-07-14T16:15:35.971 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T16:15:35.972 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2023-07-14T16:15:35.972 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-14T16:15:35.972 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41404sec preferred_lft 41404sec 2023-07-14T16:15:35.972 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::f0a7:c988:251c:603/64 scope link noprefixroute 2023-07-14T16:15:35.972 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2023-07-14T16:15:35.973 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T16:15:35.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:35.974 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-07-14T16:15:35.974 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-07-14T16:15:35.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2023-07-14T16:15:35.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2023-07-14T16:15:35.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2023-07-14T16:15:35.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T16:15:35.976 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-07-14T16:15:36.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:35 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/4245256266' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T16:15:36.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:35 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/4245256266' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T16:15:36.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:36.461 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T16:15:36.532 INFO:teuthology.orchestra.run.smithi062.stdout:1 2023-07-14T16:15:36.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:36.535 DEBUG:teuthology.orchestra.run.smithi062:> route 2023-07-14T16:15:36.614 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2023-07-14T16:15:36.615 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T16:15:36.615 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T16:15:36.615 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-14T16:15:36.615 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T16:15:36.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:36.617 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-07-14T16:15:36.617 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-07-14T16:15:36.617 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T16:15:36.617 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T16:15:36.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T16:15:36.618 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-07-14T16:15:36.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:36 smithi062 ceph-mon[95543]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T16:15:37.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:36 smithi164 ceph-mon[97427]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T16:15:37.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:36 smithi081 ceph-mon[127795]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T16:15:37.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:37.292 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-07-14T16:15:37.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:37.337 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2023-07-14T16:15:37.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:37.421 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-07-14T16:15:37.422 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-07-14T16:15:37.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T16:15:37.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-14T16:15:37.423 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-07-14T16:15:37.606 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-14T16:15:37.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:37.607 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-07-14T16:15:37.607 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-07-14T16:15:37.607 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-14T16:15:37.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T16:15:37.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-14T16:15:37.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-14T16:15:37.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-14T16:15:37.609 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-07-14T16:15:37.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:37.892 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2023-07-14T16:15:37.892 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2023-07-14T16:15:37.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2023-07-14T16:15:37.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T16:15:37.893 DEBUG:teuthology.orchestra.run.smithi062:> ') 2023-07-14T16:15:38.071 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2023-07-14T16:15:38.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T16:15:38.071 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:38.115 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T16:15:38.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T16:15:38.116 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:38.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2023-07-14T16:15:38.309 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T16:15:38.309 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T16:15:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T16:15:38.381 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T16:15:38.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:38.382 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2023-07-14T16:15:38.426 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-14T16:15:38.426 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 --client_fs=cephfs) 2023-07-14T16:15:38.471 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2023-07-14T16:15:38.542 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T16:15:38.543 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T16:15:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T16:15:38.615 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T16:15:38.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:38.617 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2023-07-14T16:15:38.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:2023-07-14T16:15:38.723+0000 7f87ff9cb5c0 -1 init, newargv = 0x56437c0c0c20 newargc=15 2023-07-14T16:15:38.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:ceph-fuse[143835]: starting ceph client 2023-07-14T16:15:38.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:ceph-fuse[143835]: starting fuse 2023-07-14T16:15:39.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:38 smithi164 ceph-mon[97427]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T16:15:39.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:38 smithi062 ceph-mon[95543]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T16:15:39.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:38 smithi081 ceph-mon[127795]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T16:15:39.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2023-07-14T16:15:39.842 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-14T16:15:39.843 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-14T16:15:39.914 INFO:teuthology.orchestra.run.smithi062.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-14T16:15:39.917 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-14T16:15:39.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:39.918 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2023-07-14T16:15:39.962 INFO:teuthology.orchestra.run.smithi062.stdout:121 2023-07-14T16:15:39.963 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-07-14T16:15:39.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-14T16:15:39.963 DEBUG:teuthology.orchestra.run.smithi062:> import glob 2023-07-14T16:15:39.964 DEBUG:teuthology.orchestra.run.smithi062:> import re 2023-07-14T16:15:39.964 DEBUG:teuthology.orchestra.run.smithi062:> import os 2023-07-14T16:15:39.964 DEBUG:teuthology.orchestra.run.smithi062:> import subprocess 2023-07-14T16:15:39.964 DEBUG:teuthology.orchestra.run.smithi062:> 2023-07-14T16:15:39.965 DEBUG:teuthology.orchestra.run.smithi062:> def _find_admin_socket(client_name): 2023-07-14T16:15:39.965 DEBUG:teuthology.orchestra.run.smithi062:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-14T16:15:39.965 DEBUG:teuthology.orchestra.run.smithi062:> files = glob.glob(asok_path) 2023-07-14T16:15:39.968 DEBUG:teuthology.orchestra.run.smithi062:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-14T16:15:39.968 DEBUG:teuthology.orchestra.run.smithi062:> 2023-07-14T16:15:39.968 DEBUG:teuthology.orchestra.run.smithi062:> # Given a non-glob path, it better be there 2023-07-14T16:15:39.968 DEBUG:teuthology.orchestra.run.smithi062:> if "*" not in asok_path: 2023-07-14T16:15:39.968 DEBUG:teuthology.orchestra.run.smithi062:> assert(len(files) == 1) 2023-07-14T16:15:39.969 DEBUG:teuthology.orchestra.run.smithi062:> return files[0] 2023-07-14T16:15:39.969 DEBUG:teuthology.orchestra.run.smithi062:> 2023-07-14T16:15:39.969 DEBUG:teuthology.orchestra.run.smithi062:> for f in files: 2023-07-14T16:15:39.969 DEBUG:teuthology.orchestra.run.smithi062:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-14T16:15:39.970 DEBUG:teuthology.orchestra.run.smithi062:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-14T16:15:39.970 DEBUG:teuthology.orchestra.run.smithi062:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-14T16:15:39.970 DEBUG:teuthology.orchestra.run.smithi062:> contents = proc_f.read() 2023-07-14T16:15:39.970 DEBUG:teuthology.orchestra.run.smithi062:> if mountpoint in contents: 2023-07-14T16:15:39.971 DEBUG:teuthology.orchestra.run.smithi062:> return f 2023-07-14T16:15:39.971 DEBUG:teuthology.orchestra.run.smithi062:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-14T16:15:39.971 DEBUG:teuthology.orchestra.run.smithi062:> 2023-07-14T16:15:39.971 DEBUG:teuthology.orchestra.run.smithi062:> print(_find_admin_socket("client.0")) 2023-07-14T16:15:39.971 DEBUG:teuthology.orchestra.run.smithi062:> ' 2023-07-14T16:15:40.149 INFO:teuthology.orchestra.run.smithi062.stdout:/var/run/ceph/ceph-client.0.143835.asok 2023-07-14T16:15:40.326 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.143835.asok 2023-07-14T16:15:40.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:40.327 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.143835.asok status 2023-07-14T16:15:40.556 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:40.557 INFO:teuthology.orchestra.run.smithi062.stdout: "metadata": { 2023-07-14T16:15:40.557 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_sha1": "3fc0089cc1082313a98853d48b892acdc1385b8b", 2023-07-14T16:15:40.557 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-14T16:15:40.557 INFO:teuthology.orchestra.run.smithi062.stdout: "entity_id": "0", 2023-07-14T16:15:40.557 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": "smithi062", 2023-07-14T16:15:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-14T16:15:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "pid": "143835", 2023-07-14T16:15:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "root": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2" 2023-07-14T16:15:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:15:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_count": 0, 2023-07-14T16:15:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_pinned_count": 0, 2023-07-14T16:15:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 25145, 2023-07-14T16:15:40.559 INFO:teuthology.orchestra.run.smithi062.stdout: "inst": { 2023-07-14T16:15:40.559 INFO:teuthology.orchestra.run.smithi062.stdout: "name": { 2023-07-14T16:15:40.559 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "client", 2023-07-14T16:15:40.559 INFO:teuthology.orchestra.run.smithi062.stdout: "num": 25145 2023-07-14T16:15:40.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:15:40.560 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2023-07-14T16:15:40.560 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2023-07-14T16:15:40.560 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2023-07-14T16:15:40.560 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 902906291 2023-07-14T16:15:40.560 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:15:40.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:15:40.561 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2023-07-14T16:15:40.561 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2023-07-14T16:15:40.561 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2023-07-14T16:15:40.561 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 902906291 2023-07-14T16:15:40.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:15:40.562 INFO:teuthology.orchestra.run.smithi062.stdout: "inst_str": "client.25145 172.21.15.62:0/902906291", 2023-07-14T16:15:40.562 INFO:teuthology.orchestra.run.smithi062.stdout: "addr_str": "172.21.15.62:0/902906291", 2023-07-14T16:15:40.562 INFO:teuthology.orchestra.run.smithi062.stdout: "inode_count": 5, 2023-07-14T16:15:40.562 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_epoch": 23, 2023-07-14T16:15:40.562 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch": 88, 2023-07-14T16:15:40.562 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch_barrier": 0, 2023-07-14T16:15:40.562 INFO:teuthology.orchestra.run.smithi062.stdout: "blocklisted": false, 2023-07-14T16:15:40.563 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs" 2023-07-14T16:15:40.563 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:40.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:40.572 DEBUG:teuthology.orchestra.run.smithi062:> stat --file-system '--printf=%T 2023-07-14T16:15:40.572 DEBUG:teuthology.orchestra.run.smithi062:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:40.617 INFO:teuthology.orchestra.run.smithi062.stdout:fuseblk 2023-07-14T16:15:40.617 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:40.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:15:40.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:40.731 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-14T16:15:40.743 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-14T16:15:40.787 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:40.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 -- 172.21.15.62:0/3233509116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc100cf0 msgr2=0x7f0ddc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 --2- 172.21.15.62:0/3233509116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc100cf0 0x7f0ddc101140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009990 tx=0x7f0dcc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 -- 172.21.15.62:0/3233509116 shutdown_connections 2023-07-14T16:15:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 --2- 172.21.15.62:0/3233509116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc101680 0x7f0ddc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 --2- 172.21.15.62:0/3233509116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ddc0ffb00 0x7f0ddc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 --2- 172.21.15.62:0/3233509116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc100cf0 0x7f0ddc101140 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 -- 172.21.15.62:0/3233509116 >> 172.21.15.62:0/3233509116 conn(0x7f0ddc0fb0e0 msgr2=0x7f0ddc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 -- 172.21.15.62:0/3233509116 shutdown_connections 2023-07-14T16:15:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.013+0000 7f0de3fff700 1 -- 172.21.15.62:0/3233509116 wait complete. 2023-07-14T16:15:40.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.014+0000 7f0de3fff700 1 Processor -- start 2023-07-14T16:15:40.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.014+0000 7f0de3fff700 1 -- start start 2023-07-14T16:15:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ddc0ffb00 0x7f0ddc197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc100cf0 0x7f0ddc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc101680 0x7f0ddc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc19f4e0 con 0x7f0ddc101680 2023-07-14T16:15:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de3fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc19f620 con 0x7f0ddc0ffb00 2023-07-14T16:15:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de3fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc19f7b0 con 0x7f0ddc100cf0 2023-07-14T16:15:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc101680 0x7f0ddc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:40.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc101680 0x7f0ddc19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38462/0 (socket says 172.21.15.62:38462) 2023-07-14T16:15:40.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de259c700 1 -- 172.21.15.62:0/2847717759 learned_addr learned my addr 172.21.15.62:0/2847717759 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:40.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de1d9b700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ddc0ffb00 0x7f0ddc197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:40.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de159a700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc100cf0 0x7f0ddc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:40.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de259c700 1 -- 172.21.15.62:0/2847717759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc100cf0 msgr2=0x7f0ddc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:40.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de259c700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc100cf0 0x7f0ddc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:40.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de259c700 1 -- 172.21.15.62:0/2847717759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ddc0ffb00 msgr2=0x7f0ddc197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de259c700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ddc0ffb00 0x7f0ddc197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.015+0000 7f0de259c700 1 -- 172.21.15.62:0/2847717759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0095f0 con 0x7f0ddc101680 2023-07-14T16:15:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.016+0000 7f0de259c700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc101680 0x7f0ddc19ef10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4007ee0 tx=0x7f0dd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.016+0000 7f0dd2ffd700 1 -- 172.21.15.62:0/2847717759 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4017070 con 0x7f0ddc101680 2023-07-14T16:15:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.016+0000 7f0dd2ffd700 1 -- 172.21.15.62:0/2847717759 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0dd400f040 con 0x7f0ddc101680 2023-07-14T16:15:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.016+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc19fa90 con 0x7f0ddc101680 2023-07-14T16:15:40.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.016+0000 7f0dd2ffd700 1 -- 172.21.15.62:0/2847717759 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4005710 con 0x7f0ddc101680 2023-07-14T16:15:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.016+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc19fff0 con 0x7f0ddc101680 2023-07-14T16:15:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.017+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc192090 con 0x7f0ddc101680 2023-07-14T16:15:40.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.019+0000 7f0dd2ffd700 1 -- 172.21.15.62:0/2847717759 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0dd4005870 con 0x7f0ddc101680 2023-07-14T16:15:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.021+0000 7f0dd2ffd700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0dc80a1220 0x7f0dc80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.021+0000 7f0dd2ffd700 1 -- 172.21.15.62:0/2847717759 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0dd40cc080 con 0x7f0ddc101680 2023-07-14T16:15:40.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.021+0000 7f0de1d9b700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0dc80a1220 0x7f0dc80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:40.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.022+0000 7f0de1d9b700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0dc80a1220 0x7f0dc80a36d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc100b50 tx=0x7f0dd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:41.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.025+0000 7f0dd2ffd700 1 -- 172.21.15.62:0/2847717759 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0dd40957e0 con 0x7f0ddc101680 2023-07-14T16:15:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:40 smithi164 ceph-mon[97427]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:41.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:40 smithi062 ceph-mon[95543]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:40 smithi081 ceph-mon[127795]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:15:41.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.236+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ddc02c850 con 0x7f0ddc101680 2023-07-14T16:15:41.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.238+0000 7f0dd2ffd700 1 -- 172.21.15.62:0/2847717759 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f0dd4098e00 con 0x7f0ddc101680 2023-07-14T16:15:41.216 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:41.216 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0dc80a1220 msgr2=0x7f0dc80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0dc80a1220 0x7f0dc80a36d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc100b50 tx=0x7f0dd800b410 comp rx=0 tx=0).stop 2023-07-14T16:15:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc101680 msgr2=0x7f0ddc19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc101680 0x7f0ddc19ef10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4007ee0 tx=0x7f0dd400d3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 shutdown_connections 2023-07-14T16:15:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ddc101680 0x7f0ddc19ef10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0dc80a1220 0x7f0dc80a36d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ddc0ffb00 0x7f0ddc197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 --2- 172.21.15.62:0/2847717759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ddc100cf0 0x7f0ddc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.240+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 >> 172.21.15.62:0/2847717759 conn(0x7f0ddc0fb0e0 msgr2=0x7f0ddc106620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.241+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 shutdown_connections 2023-07-14T16:15:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.241+0000 7f0de3fff700 1 -- 172.21.15.62:0/2847717759 wait complete. 2023-07-14T16:15:41.237 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:15:41.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:15:41.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 -- 172.21.15.62:0/12256427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5390100cd0 msgr2=0x7f5390101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 --2- 172.21.15.62:0/12256427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5390100cd0 0x7f5390101120 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5380009990 tx=0x7f5380009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 -- 172.21.15.62:0/12256427 shutdown_connections 2023-07-14T16:15:41.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 --2- 172.21.15.62:0/12256427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5390101660 0x7f5390107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 --2- 172.21.15.62:0/12256427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53900ffae0 0x7f53900ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 --2- 172.21.15.62:0/12256427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5390100cd0 0x7f5390101120 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 -- 172.21.15.62:0/12256427 >> 172.21.15.62:0/12256427 conn(0x7f53900fb0e0 msgr2=0x7f53900fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:41.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 -- 172.21.15.62:0/12256427 shutdown_connections 2023-07-14T16:15:41.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.468+0000 7f53979ff700 1 -- 172.21.15.62:0/12256427 wait complete. 2023-07-14T16:15:41.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.469+0000 7f53979ff700 1 Processor -- start 2023-07-14T16:15:41.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.469+0000 7f53979ff700 1 -- start start 2023-07-14T16:15:41.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f53979ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53900ffae0 0x7f53900742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:41.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f53979ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5390100cd0 0x7f5390072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:41.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f53979ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5390101660 0x7f5390072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:41.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f53979ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900735e0 con 0x7f53900ffae0 2023-07-14T16:15:41.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f53979ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5390073750 con 0x7f5390100cd0 2023-07-14T16:15:41.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f53979ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539019f7b0 con 0x7f5390101660 2023-07-14T16:15:41.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f539579b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53900ffae0 0x7f53900742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:41.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f539579b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53900ffae0 0x7f53900742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38482/0 (socket says 172.21.15.62:38482) 2023-07-14T16:15:41.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f539579b700 1 -- 172.21.15.62:0/4163247044 learned_addr learned my addr 172.21.15.62:0/4163247044 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:41.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f5394f9a700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5390100cd0 0x7f5390072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:41.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f539579b700 1 -- 172.21.15.62:0/4163247044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5390101660 msgr2=0x7f5390072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f5395f9c700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5390101660 0x7f5390072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:41.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f539579b700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5390101660 0x7f5390072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f539579b700 1 -- 172.21.15.62:0/4163247044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5390100cd0 msgr2=0x7f5390072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f539579b700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5390100cd0 0x7f5390072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f539579b700 1 -- 172.21.15.62:0/4163247044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53800095f0 con 0x7f53900ffae0 2023-07-14T16:15:41.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f5395f9c700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5390101660 0x7f5390072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:41.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.470+0000 7f5394f9a700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5390100cd0 0x7f5390072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:41.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.471+0000 7f539579b700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53900ffae0 0x7f53900742c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f538c00e970 tx=0x7f538c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:41.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.471+0000 7f53867fc700 1 -- 172.21.15.62:0/4163247044 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c009790 con 0x7f53900ffae0 2023-07-14T16:15:41.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.471+0000 7f53867fc700 1 -- 172.21.15.62:0/4163247044 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f538c0044b0 con 0x7f53900ffae0 2023-07-14T16:15:41.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.471+0000 7f53867fc700 1 -- 172.21.15.62:0/4163247044 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c003dd0 con 0x7f53900ffae0 2023-07-14T16:15:41.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.471+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539019fa90 con 0x7f53900ffae0 2023-07-14T16:15:41.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.471+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539019ffa0 con 0x7f53900ffae0 2023-07-14T16:15:41.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.472+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5390106b50 con 0x7f53900ffae0 2023-07-14T16:15:41.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.474+0000 7f53867fc700 1 -- 172.21.15.62:0/4163247044 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f538c003f30 con 0x7f53900ffae0 2023-07-14T16:15:41.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.476+0000 7f53867fc700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f537c0a11d0 0x7f537c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:41.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.476+0000 7f53867fc700 1 -- 172.21.15.62:0/4163247044 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f538c014070 con 0x7f53900ffae0 2023-07-14T16:15:41.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.477+0000 7f5394f9a700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f537c0a11d0 0x7f537c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:41.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.477+0000 7f5394f9a700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f537c0a11d0 0x7f537c0a3680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f53800059d0 tx=0x7f5380005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:41.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.480+0000 7f53867fc700 1 -- 172.21.15.62:0/4163247044 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f538c096330 con 0x7f53900ffae0 2023-07-14T16:15:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.687+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53900611c0 con 0x7f53900ffae0 2023-07-14T16:15:41.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.687+0000 7f53867fc700 1 -- 172.21.15.62:0/4163247044 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f538c099950 con 0x7f53900ffae0 2023-07-14T16:15:41.664 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:15:41.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.689+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f537c0a11d0 msgr2=0x7f537c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.689+0000 7f53979ff700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f537c0a11d0 0x7f537c0a3680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f53800059d0 tx=0x7f5380005960 comp rx=0 tx=0).stop 2023-07-14T16:15:41.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.689+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53900ffae0 msgr2=0x7f53900742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.689+0000 7f53979ff700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53900ffae0 0x7f53900742c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f538c00e970 tx=0x7f538c00ed30 comp rx=0 tx=0).stop 2023-07-14T16:15:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.690+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 shutdown_connections 2023-07-14T16:15:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.690+0000 7f53979ff700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5390101660 0x7f5390072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.690+0000 7f53979ff700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53900ffae0 0x7f53900742c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.690+0000 7f53979ff700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f537c0a11d0 0x7f537c0a3680 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.690+0000 7f53979ff700 1 --2- 172.21.15.62:0/4163247044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5390100cd0 0x7f5390072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.690+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 >> 172.21.15.62:0/4163247044 conn(0x7f53900fb0e0 msgr2=0x7f53900fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:41.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.690+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 shutdown_connections 2023-07-14T16:15:41.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.690+0000 7f53979ff700 1 -- 172.21.15.62:0/4163247044 wait complete. 2023-07-14T16:15:41.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:15:41.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 -- 172.21.15.62:0/2859642442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 msgr2=0x7feb50107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 --2- 172.21.15.62:0/2859642442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 0x7feb50107f20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b0a0 tx=0x7feb4800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 -- 172.21.15.62:0/2859642442 shutdown_connections 2023-07-14T16:15:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 --2- 172.21.15.62:0/2859642442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 0x7feb50107f20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 --2- 172.21.15.62:0/2859642442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb500ffae0 0x7feb500ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 --2- 172.21.15.62:0/2859642442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb50100cd0 0x7feb50101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 -- 172.21.15.62:0/2859642442 >> 172.21.15.62:0/2859642442 conn(0x7feb500fb0e0 msgr2=0x7feb500fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 -- 172.21.15.62:0/2859642442 shutdown_connections 2023-07-14T16:15:41.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.915+0000 7feb577ff700 1 -- 172.21.15.62:0/2859642442 wait complete. 2023-07-14T16:15:41.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.916+0000 7feb577ff700 1 Processor -- start 2023-07-14T16:15:41.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.916+0000 7feb577ff700 1 -- start start 2023-07-14T16:15:41.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.916+0000 7feb577ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb500ffae0 0x7feb500742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.916+0000 7feb577ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb50100cd0 0x7feb50072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb577ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 0x7feb50072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb577ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500735e0 con 0x7feb50101660 2023-07-14T16:15:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb577ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50073750 con 0x7feb50100cd0 2023-07-14T16:15:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb577ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5019f8b0 con 0x7feb500ffae0 2023-07-14T16:15:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 0x7feb50072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:41.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 0x7feb50072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38498/0 (socket says 172.21.15.62:38498) 2023-07-14T16:15:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 -- 172.21.15.62:0/989483940 learned_addr learned my addr 172.21.15.62:0/989483940 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb5559b700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb500ffae0 0x7feb500742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 -- 172.21.15.62:0/989483940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb500ffae0 msgr2=0x7feb500742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb500ffae0 0x7feb500742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 -- 172.21.15.62:0/989483940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb50100cd0 msgr2=0x7feb50072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb50100cd0 0x7feb50072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 -- 172.21.15.62:0/989483940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb48009d00 con 0x7feb50101660 2023-07-14T16:15:41.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb5559b700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb500ffae0 0x7feb500742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.917+0000 7feb55d9c700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 0x7feb50072e40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b830 tx=0x7feb48004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.918+0000 7feb467fc700 1 -- 172.21.15.62:0/989483940 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800e040 con 0x7feb50101660 2023-07-14T16:15:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.918+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5019fb30 con 0x7feb50101660 2023-07-14T16:15:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.918+0000 7feb467fc700 1 -- 172.21.15.62:0/989483940 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb48009440 con 0x7feb50101660 2023-07-14T16:15:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.918+0000 7feb467fc700 1 -- 172.21.15.62:0/989483940 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4801dbe0 con 0x7feb50101660 2023-07-14T16:15:41.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.918+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5019ff60 con 0x7feb50101660 2023-07-14T16:15:41.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.919+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb501070c0 con 0x7feb50101660 2023-07-14T16:15:41.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.921+0000 7feb467fc700 1 -- 172.21.15.62:0/989483940 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feb48005450 con 0x7feb50101660 2023-07-14T16:15:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.923+0000 7feb467fc700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feb3c0a1150 0x7feb3c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.923+0000 7feb467fc700 1 -- 172.21.15.62:0/989483940 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feb480ce220 con 0x7feb50101660 2023-07-14T16:15:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.923+0000 7feb5559b700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feb3c0a1150 0x7feb3c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.924+0000 7feb5559b700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feb3c0a1150 0x7feb3c0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feb4c005fd0 tx=0x7feb4c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:41.927+0000 7feb467fc700 1 -- 172.21.15.62:0/989483940 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feb480979b0 con 0x7feb50101660 2023-07-14T16:15:42.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:41 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2847717759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:42.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:41 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/4163247044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:42.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.135+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb50061220 con 0x7feb50101660 2023-07-14T16:15:42.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.136+0000 7feb467fc700 1 -- 172.21.15.62:0/989483940 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb4809afd0 con 0x7feb50101660 2023-07-14T16:15:42.114 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:15:42.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feb3c0a1150 msgr2=0x7feb3c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feb3c0a1150 0x7feb3c0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feb4c005fd0 tx=0x7feb4c005f00 comp rx=0 tx=0).stop 2023-07-14T16:15:42.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 msgr2=0x7feb50072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 0x7feb50072e40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b830 tx=0x7feb48004d90 comp rx=0 tx=0).stop 2023-07-14T16:15:42.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 shutdown_connections 2023-07-14T16:15:42.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb50101660 0x7feb50072e40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb500ffae0 0x7feb500742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feb3c0a1150 0x7feb3c0a3600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 --2- 172.21.15.62:0/989483940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb50100cd0 0x7feb50072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 >> 172.21.15.62:0/989483940 conn(0x7feb500fb0e0 msgr2=0x7feb50104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:42.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 shutdown_connections 2023-07-14T16:15:42.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.138+0000 7feb577ff700 1 -- 172.21.15.62:0/989483940 wait complete. 2023-07-14T16:15:42.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:41 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2847717759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:42.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:41 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/4163247044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:42.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:15:42.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:41 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2847717759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:42.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:41 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/4163247044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/3267771285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a780ffae0 msgr2=0x7f5a780ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/3267771285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a780ffae0 0x7f5a780ffef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009990 tx=0x7f5a74009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/3267771285 shutdown_connections 2023-07-14T16:15:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/3267771285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a78101660 0x7f5a78107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/3267771285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a780ffae0 0x7f5a780ffef0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/3267771285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a78100cd0 0x7f5a78101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/3267771285 >> 172.21.15.62:0/3267771285 conn(0x7f5a780fb0e0 msgr2=0x7f5a780fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:42.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/3267771285 shutdown_connections 2023-07-14T16:15:42.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.362+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/3267771285 wait complete. 2023-07-14T16:15:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.363+0000 7f5a7f3ff700 1 Processor -- start 2023-07-14T16:15:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7f3ff700 1 -- start start 2023-07-14T16:15:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7f3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a780ffae0 0x7f5a780742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7f3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a78100cd0 0x7f5a78072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7f3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a78101660 0x7f5a78072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7f3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780735e0 con 0x7f5a78101660 2023-07-14T16:15:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7f3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78073750 con 0x7f5a78100cd0 2023-07-14T16:15:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7f3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7819f7a0 con 0x7f5a780ffae0 2023-07-14T16:15:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7d99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a78101660 0x7f5a78072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7d99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a78101660 0x7f5a78072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38502/0 (socket says 172.21.15.62:38502) 2023-07-14T16:15:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7d99c700 1 -- 172.21.15.62:0/2471521981 learned_addr learned my addr 172.21.15.62:0/2471521981 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7c99a700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a78100cd0 0x7f5a78072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.364+0000 7f5a7d19b700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a780ffae0 0x7f5a780742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a7d99c700 1 -- 172.21.15.62:0/2471521981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a780ffae0 msgr2=0x7f5a780742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a7d99c700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a780ffae0 0x7f5a780742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a7d99c700 1 -- 172.21.15.62:0/2471521981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a78100cd0 msgr2=0x7f5a78072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a7d99c700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a78100cd0 0x7f5a78072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a7d99c700 1 -- 172.21.15.62:0/2471521981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740095f0 con 0x7f5a78101660 2023-07-14T16:15:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a7d99c700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a78101660 0x7f5a78072e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5a70007ee0 tx=0x7f5a7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a6e7fc700 1 -- 172.21.15.62:0/2471521981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70017070 con 0x7f5a78101660 2023-07-14T16:15:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7819fa80 con 0x7f5a78101660 2023-07-14T16:15:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a6e7fc700 1 -- 172.21.15.62:0/2471521981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a7000f040 con 0x7f5a78101660 2023-07-14T16:15:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a6e7fc700 1 -- 172.21.15.62:0/2471521981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70005570 con 0x7f5a78101660 2023-07-14T16:15:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.365+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7819ff90 con 0x7f5a78101660 2023-07-14T16:15:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.366+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7810b1b0 con 0x7f5a78101660 2023-07-14T16:15:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.369+0000 7f5a6e7fc700 1 -- 172.21.15.62:0/2471521981 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5a7000db20 con 0x7f5a78101660 2023-07-14T16:15:42.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.370+0000 7f5a6e7fc700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a640a1150 0x7f5a640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:42.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.370+0000 7f5a6e7fc700 1 -- 172.21.15.62:0/2471521981 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5a700cc6e0 con 0x7f5a78101660 2023-07-14T16:15:42.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.370+0000 7f5a7d19b700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a640a1150 0x7f5a640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.371+0000 7f5a7d19b700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a640a1150 0x7f5a640a3600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400c010 tx=0x7f5a74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.372+0000 7f5a6e7fc700 1 -- 172.21.15.62:0/2471521981 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5a70095de0 con 0x7f5a78101660 2023-07-14T16:15:42.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.581+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a78106a30 con 0x7f5a78101660 2023-07-14T16:15:42.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.582+0000 7f5a6e7fc700 1 -- 172.21.15.62:0/2471521981 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a70099400 con 0x7f5a78101660 2023-07-14T16:15:42.559 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:15:42.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a640a1150 msgr2=0x7f5a640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a640a1150 0x7f5a640a3600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400c010 tx=0x7f5a74005fd0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a78101660 msgr2=0x7f5a78072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a78101660 0x7f5a78072e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5a70007ee0 tx=0x7f5a7000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 shutdown_connections 2023-07-14T16:15:42.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a78101660 0x7f5a78072e40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a780ffae0 0x7f5a780742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a640a1150 0x7f5a640a3600 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 --2- 172.21.15.62:0/2471521981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a78100cd0 0x7f5a78072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 >> 172.21.15.62:0/2471521981 conn(0x7f5a780fb0e0 msgr2=0x7f5a780fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:42.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 shutdown_connections 2023-07-14T16:15:42.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.584+0000 7f5a7f3ff700 1 -- 172.21.15.62:0/2471521981 wait complete. 2023-07-14T16:15:42.580 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:42.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 -- 172.21.15.62:0/4261228922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901016a0 msgr2=0x7f9d90107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 --2- 172.21.15.62:0/4261228922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901016a0 0x7f9d90107f60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800b0a0 tx=0x7f9d8800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 -- 172.21.15.62:0/4261228922 shutdown_connections 2023-07-14T16:15:42.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 --2- 172.21.15.62:0/4261228922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d901016a0 0x7f9d90107f60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 --2- 172.21.15.62:0/4261228922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d900ffb20 0x7f9d900fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 --2- 172.21.15.62:0/4261228922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d90100d10 0x7f9d90101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 -- 172.21.15.62:0/4261228922 >> 172.21.15.62:0/4261228922 conn(0x7f9d900fb160 msgr2=0x7f9d900fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:42.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 -- 172.21.15.62:0/4261228922 shutdown_connections 2023-07-14T16:15:42.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.806+0000 7f9d985ff700 1 -- 172.21.15.62:0/4261228922 wait complete. 2023-07-14T16:15:42.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.807+0000 7f9d985ff700 1 Processor -- start 2023-07-14T16:15:42.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.807+0000 7f9d985ff700 1 -- start start 2023-07-14T16:15:42.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d985ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d900ffb20 0x7f9d90197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:42.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d985ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90100d10 0x7f9d90197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:42.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d985ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d901016a0 0x7f9d9019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:42.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d985ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9019f580 con 0x7f9d900ffb20 2023-07-14T16:15:42.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d985ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9019f6c0 con 0x7f9d90100d10 2023-07-14T16:15:42.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d985ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9019f850 con 0x7f9d901016a0 2023-07-14T16:15:42.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d9639b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d900ffb20 0x7f9d90197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:42.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d9639b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d900ffb20 0x7f9d90197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34924/0 (socket says 172.21.15.62:34924) 2023-07-14T16:15:42.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d9639b700 1 -- 172.21.15.62:0/998405872 learned_addr learned my addr 172.21.15.62:0/998405872 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:42.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d95b9a700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90100d10 0x7f9d90197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:42.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d96b9c700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d901016a0 0x7f9d9019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:42.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d9639b700 1 -- 172.21.15.62:0/998405872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d901016a0 msgr2=0x7f9d9019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d9639b700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d901016a0 0x7f9d9019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d9639b700 1 -- 172.21.15.62:0/998405872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90100d10 msgr2=0x7f9d90197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:42.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d9639b700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90100d10 0x7f9d90197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:42.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d9639b700 1 -- 172.21.15.62:0/998405872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009d00 con 0x7f9d900ffb20 2023-07-14T16:15:42.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.808+0000 7f9d96b9c700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d901016a0 0x7f9d9019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:42.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.809+0000 7f9d9639b700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d900ffb20 0x7f9d90197a00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00d980 tx=0x7f9d8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:42.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.809+0000 7f9d877fe700 1 -- 172.21.15.62:0/998405872 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c004020 con 0x7f9d900ffb20 2023-07-14T16:15:42.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.809+0000 7f9d877fe700 1 -- 172.21.15.62:0/998405872 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d8c009730 con 0x7f9d900ffb20 2023-07-14T16:15:42.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.809+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9019fb30 con 0x7f9d900ffb20 2023-07-14T16:15:42.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.809+0000 7f9d877fe700 1 -- 172.21.15.62:0/998405872 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c003a00 con 0x7f9d900ffb20 2023-07-14T16:15:42.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.809+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d901a0090 con 0x7f9d900ffb20 2023-07-14T16:15:42.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.810+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9010b1b0 con 0x7f9d900ffb20 2023-07-14T16:15:42.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.812+0000 7f9d877fe700 1 -- 172.21.15.62:0/998405872 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9d8c0098a0 con 0x7f9d900ffb20 2023-07-14T16:15:42.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.813+0000 7f9d877fe700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d7c0a1220 0x7f9d7c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:42.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.814+0000 7f9d877fe700 1 -- 172.21.15.62:0/998405872 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d8c0cc5d0 con 0x7f9d900ffb20 2023-07-14T16:15:42.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.814+0000 7f9d95b9a700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d7c0a1220 0x7f9d7c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:42.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.814+0000 7f9d95b9a700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d7c0a1220 0x7f9d7c0a36d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005950 tx=0x7f9d8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:42.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:42.817+0000 7f9d877fe700 1 -- 172.21.15.62:0/998405872 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9d8c095cd0 con 0x7f9d900ffb20 2023-07-14T16:15:42.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:42 smithi062 ceph-mon[95543]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T16:15:42.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:42 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/989483940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:42.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:42 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2471521981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:43.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.029+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d9010b3d0 con 0x7f9d900ffb20 2023-07-14T16:15:43.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.030+0000 7f9d877fe700 1 -- 172.21.15.62:0/998405872 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f9d8c0232f0 con 0x7f9d900ffb20 2023-07-14T16:15:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:43.008 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:43.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.032+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d7c0a1220 msgr2=0x7f9d7c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.032+0000 7f9d985ff700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d7c0a1220 0x7f9d7c0a36d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005950 tx=0x7f9d8000b410 comp rx=0 tx=0).stop 2023-07-14T16:15:43.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.032+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d900ffb20 msgr2=0x7f9d90197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.032+0000 7f9d985ff700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d900ffb20 0x7f9d90197a00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00d980 tx=0x7f9d8c00dc90 comp rx=0 tx=0).stop 2023-07-14T16:15:43.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.033+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 shutdown_connections 2023-07-14T16:15:43.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.033+0000 7f9d985ff700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d901016a0 0x7f9d9019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.033+0000 7f9d985ff700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d900ffb20 0x7f9d90197a00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.033+0000 7f9d985ff700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d7c0a1220 0x7f9d7c0a36d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.033+0000 7f9d985ff700 1 --2- 172.21.15.62:0/998405872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d90100d10 0x7f9d90197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.033+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 >> 172.21.15.62:0/998405872 conn(0x7f9d900fb160 msgr2=0x7f9d900fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.033+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 shutdown_connections 2023-07-14T16:15:43.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.033+0000 7f9d985ff700 1 -- 172.21.15.62:0/998405872 wait complete. 2023-07-14T16:15:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:42 smithi164 ceph-mon[97427]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T16:15:43.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:42 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/989483940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:43.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:42 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2471521981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:43.028 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:15:43.029 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:15:43.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:42 smithi081 ceph-mon[127795]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T16:15:43.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:42 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/989483940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:43.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:42 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2471521981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:43.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 -- 172.21.15.62:0/3991195061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c1079b0 msgr2=0x7f2a0c109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 --2- 172.21.15.62:0/3991195061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c1079b0 0x7f2a0c109e30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400b0a0 tx=0x7f2a0400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 -- 172.21.15.62:0/3991195061 shutdown_connections 2023-07-14T16:15:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 --2- 172.21.15.62:0/3991195061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c1079b0 0x7f2a0c109e30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 --2- 172.21.15.62:0/3991195061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a0c074550 0x7f2a0c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 --2- 172.21.15.62:0/3991195061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a0c072f70 0x7f2a0c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 -- 172.21.15.62:0/3991195061 >> 172.21.15.62:0/3991195061 conn(0x7f2a0c0fb130 msgr2=0x7f2a0c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:43.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 -- 172.21.15.62:0/3991195061 shutdown_connections 2023-07-14T16:15:43.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.257+0000 7f2a143ff700 1 -- 172.21.15.62:0/3991195061 wait complete. 2023-07-14T16:15:43.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.258+0000 7f2a143ff700 1 Processor -- start 2023-07-14T16:15:43.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.258+0000 7f2a143ff700 1 -- start start 2023-07-14T16:15:43.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a143ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c072f70 0x7f2a0c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:43.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a143ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a0c074550 0x7f2a0c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:43.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a143ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a0c1079b0 0x7f2a0c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:43.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a143ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c19f650 con 0x7f2a0c072f70 2023-07-14T16:15:43.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a143ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c19f790 con 0x7f2a0c074550 2023-07-14T16:15:43.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a143ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c19f920 con 0x7f2a0c1079b0 2023-07-14T16:15:43.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a1219b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c072f70 0x7f2a0c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:43.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a1219b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c072f70 0x7f2a0c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34936/0 (socket says 172.21.15.62:34936) 2023-07-14T16:15:43.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a1219b700 1 -- 172.21.15.62:0/1548106020 learned_addr learned my addr 172.21.15.62:0/1548106020 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:43.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a1199a700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a0c074550 0x7f2a0c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:43.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.259+0000 7f2a1299c700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a0c1079b0 0x7f2a0c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:43.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a1219b700 1 -- 172.21.15.62:0/1548106020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a0c1079b0 msgr2=0x7f2a0c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a1219b700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a0c1079b0 0x7f2a0c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a1219b700 1 -- 172.21.15.62:0/1548106020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a0c074550 msgr2=0x7f2a0c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a1219b700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a0c074550 0x7f2a0c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a1219b700 1 -- 172.21.15.62:0/1548106020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04009d00 con 0x7f2a0c072f70 2023-07-14T16:15:43.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a1219b700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c072f70 0x7f2a0c197b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800d980 tx=0x7f2a0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:43.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a037fe700 1 -- 172.21.15.62:0/1548106020 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08009930 con 0x7f2a0c072f70 2023-07-14T16:15:43.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c19fc00 con 0x7f2a0c072f70 2023-07-14T16:15:43.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a037fe700 1 -- 172.21.15.62:0/1548106020 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a08010460 con 0x7f2a0c072f70 2023-07-14T16:15:43.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a037fe700 1 -- 172.21.15.62:0/1548106020 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800f620 con 0x7f2a0c072f70 2023-07-14T16:15:43.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.260+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c1a0110 con 0x7f2a0c072f70 2023-07-14T16:15:43.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.261+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c106060 con 0x7f2a0c072f70 2023-07-14T16:15:43.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.263+0000 7f2a037fe700 1 -- 172.21.15.62:0/1548106020 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2a080105d0 con 0x7f2a0c072f70 2023-07-14T16:15:43.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.266+0000 7f2a037fe700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f29f80a1150 0x7f29f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:43.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.266+0000 7f2a037fe700 1 -- 172.21.15.62:0/1548106020 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2a080cc410 con 0x7f2a0c072f70 2023-07-14T16:15:43.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.266+0000 7f2a1199a700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f29f80a1150 0x7f29f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:43.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.267+0000 7f2a1199a700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f29f80a1150 0x7f29f80a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c074270 tx=0x7f29fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:43.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.270+0000 7f2a037fe700 1 -- 172.21.15.62:0/1548106020 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2a08095b10 con 0x7f2a0c072f70 2023-07-14T16:15:43.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.477+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a0c0611c0 con 0x7f2a0c072f70 2023-07-14T16:15:43.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.478+0000 7f2a037fe700 1 -- 172.21.15.62:0/1548106020 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a08099130 con 0x7f2a0c072f70 2023-07-14T16:15:43.455 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:15:43.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.480+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f29f80a1150 msgr2=0x7f29f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.480+0000 7f2a143ff700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f29f80a1150 0x7f29f80a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c074270 tx=0x7f29fc00b410 comp rx=0 tx=0).stop 2023-07-14T16:15:43.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.480+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c072f70 msgr2=0x7f2a0c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.480+0000 7f2a143ff700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c072f70 0x7f2a0c197b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800d980 tx=0x7f2a0800dc90 comp rx=0 tx=0).stop 2023-07-14T16:15:43.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.481+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 shutdown_connections 2023-07-14T16:15:43.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.481+0000 7f2a143ff700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a0c1079b0 0x7f2a0c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.481+0000 7f2a143ff700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c072f70 0x7f2a0c197b20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.481+0000 7f2a143ff700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f29f80a1150 0x7f29f80a3600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.481+0000 7f2a143ff700 1 --2- 172.21.15.62:0/1548106020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a0c074550 0x7f2a0c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.481+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 >> 172.21.15.62:0/1548106020 conn(0x7f2a0c0fb130 msgr2=0x7f2a0c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:43.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.481+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 shutdown_connections 2023-07-14T16:15:43.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.481+0000 7f2a143ff700 1 -- 172.21.15.62:0/1548106020 wait complete. 2023-07-14T16:15:43.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:15:43.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 -- 172.21.15.62:0/1809964704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54101670 msgr2=0x7f5c54107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/1809964704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54101670 0x7f5c54107f30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00b0a0 tx=0x7f5c4c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 -- 172.21.15.62:0/1809964704 shutdown_connections 2023-07-14T16:15:43.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/1809964704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54101670 0x7f5c54107f30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/1809964704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c540ffaf0 0x7f5c540fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/1809964704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c54100ce0 0x7f5c54101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 -- 172.21.15.62:0/1809964704 >> 172.21.15.62:0/1809964704 conn(0x7f5c540fb130 msgr2=0x7f5c540fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:43.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 -- 172.21.15.62:0/1809964704 shutdown_connections 2023-07-14T16:15:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.706+0000 7f5c5bbff700 1 -- 172.21.15.62:0/1809964704 wait complete. 2023-07-14T16:15:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.707+0000 7f5c5bbff700 1 Processor -- start 2023-07-14T16:15:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.707+0000 7f5c5bbff700 1 -- start start 2023-07-14T16:15:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5bbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5bbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5bbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5bbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419f5a0 con 0x7f5c54100ce0 2023-07-14T16:15:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5bbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419f6e0 con 0x7f5c54101670 2023-07-14T16:15:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5bbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419f870 con 0x7f5c540ffaf0 2023-07-14T16:15:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5919a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5919a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34946/0 (socket says 172.21.15.62:34946) 2023-07-14T16:15:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5919a700 1 -- 172.21.15.62:0/2133478014 learned_addr learned my addr 172.21.15.62:0/2133478014 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5999b700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5a19c700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.708+0000 7f5c5919a700 1 -- 172.21.15.62:0/2133478014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c540ffaf0 msgr2=0x7f5c54197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5919a700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5919a700 1 -- 172.21.15.62:0/2133478014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c54101670 msgr2=0x7f5c5419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5919a700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5919a700 1 -- 172.21.15.62:0/2133478014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44009520 con 0x7f5c54100ce0 2023-07-14T16:15:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5999b700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:43.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5a19c700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:43.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5919a700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400eb30 tx=0x7f5c4400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:43.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c4affd700 1 -- 172.21.15.62:0/2133478014 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44009880 con 0x7f5c54100ce0 2023-07-14T16:15:43.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c009d00 con 0x7f5c54100ce0 2023-07-14T16:15:43.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c4affd700 1 -- 172.21.15.62:0/2133478014 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c440044b0 con 0x7f5c54100ce0 2023-07-14T16:15:43.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c4affd700 1 -- 172.21.15.62:0/2133478014 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44003e40 con 0x7f5c54100ce0 2023-07-14T16:15:43.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.709+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5419ff20 con 0x7f5c54100ce0 2023-07-14T16:15:43.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.710+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5410b180 con 0x7f5c54100ce0 2023-07-14T16:15:43.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.712+0000 7f5c4affd700 1 -- 172.21.15.62:0/2133478014 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5c44003fa0 con 0x7f5c54100ce0 2023-07-14T16:15:43.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.714+0000 7f5c4affd700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c400a1150 0x7f5c400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:43.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.714+0000 7f5c4affd700 1 -- 172.21.15.62:0/2133478014 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5c44014070 con 0x7f5c54100ce0 2023-07-14T16:15:43.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.714+0000 7f5c5999b700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c400a1150 0x7f5c400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:43.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.714+0000 7f5c5999b700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c400a1150 0x7f5c400a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c54100b40 tx=0x7f5c50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:43.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.718+0000 7f5c4affd700 1 -- 172.21.15.62:0/2133478014 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c44096500 con 0x7f5c54100ce0 2023-07-14T16:15:43.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:43 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/998405872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:43.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:43 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1548106020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.929+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c541a0590 con 0x7f5c54100ce0 2023-07-14T16:15:43.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.929+0000 7f5c4affd700 1 -- 172.21.15.62:0/2133478014 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c44099b20 con 0x7f5c54100ce0 2023-07-14T16:15:43.907 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:15:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c400a1150 msgr2=0x7f5c400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c400a1150 0x7f5c400a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c54100b40 tx=0x7f5c50009260 comp rx=0 tx=0).stop 2023-07-14T16:15:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54100ce0 msgr2=0x7f5c54197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400eb30 tx=0x7f5c4400eef0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 shutdown_connections 2023-07-14T16:15:43.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5c400a1150 0x7f5c400a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 --2- 172.21.15.62:0/2133478014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:43.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 >> 172.21.15.62:0/2133478014 conn(0x7f5c540fb130 msgr2=0x7f5c54106560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 shutdown_connections 2023-07-14T16:15:43.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:43.932+0000 7f5c5bbff700 1 -- 172.21.15.62:0/2133478014 wait complete. 2023-07-14T16:15:43.928 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:15:44.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:43 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/998405872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:44.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:43 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1548106020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.158+0000 7f58297ff700 1 -- 172.21.15.62:0/96642459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5824100ce0 msgr2=0x7f5824101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.158+0000 7f58297ff700 1 --2- 172.21.15.62:0/96642459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5824100ce0 0x7f5824101130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f580c009990 tx=0x7f580c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.159+0000 7f58297ff700 1 -- 172.21.15.62:0/96642459 shutdown_connections 2023-07-14T16:15:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.159+0000 7f58297ff700 1 --2- 172.21.15.62:0/96642459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824101670 0x7f5824107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.159+0000 7f58297ff700 1 --2- 172.21.15.62:0/96642459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58240ffaf0 0x7f58240fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.159+0000 7f58297ff700 1 --2- 172.21.15.62:0/96642459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5824100ce0 0x7f5824101130 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.159+0000 7f58297ff700 1 -- 172.21.15.62:0/96642459 >> 172.21.15.62:0/96642459 conn(0x7f58240fb130 msgr2=0x7f58240fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.159+0000 7f58297ff700 1 -- 172.21.15.62:0/96642459 shutdown_connections 2023-07-14T16:15:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.159+0000 7f58297ff700 1 -- 172.21.15.62:0/96642459 wait complete. 2023-07-14T16:15:44.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f58297ff700 1 Processor -- start 2023-07-14T16:15:44.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f58297ff700 1 -- start start 2023-07-14T16:15:44.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f58297ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58240ffaf0 0x7f58240742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:44.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f58297ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5824100ce0 0x7f5824072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:44.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f58297ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824101670 0x7f5824072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:44.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f58297ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58240735e0 con 0x7f58240ffaf0 2023-07-14T16:15:44.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f58297ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5824073750 con 0x7f5824100ce0 2023-07-14T16:15:44.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f58297ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582419f7d0 con 0x7f5824101670 2023-07-14T16:15:44.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f5822ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58240ffaf0 0x7f58240742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:44.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f5822ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58240ffaf0 0x7f58240742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34962/0 (socket says 172.21.15.62:34962) 2023-07-14T16:15:44.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.160+0000 7f5822ffd700 1 -- 172.21.15.62:0/1748829222 learned_addr learned my addr 172.21.15.62:0/1748829222 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:44.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f58237fe700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824101670 0x7f5824072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f581bfff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5824100ce0 0x7f5824072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5822ffd700 1 -- 172.21.15.62:0/1748829222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824101670 msgr2=0x7f5824072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5822ffd700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824101670 0x7f5824072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5822ffd700 1 -- 172.21.15.62:0/1748829222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5824100ce0 msgr2=0x7f5824072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5822ffd700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5824100ce0 0x7f5824072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5822ffd700 1 -- 172.21.15.62:0/1748829222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0095f0 con 0x7f58240ffaf0 2023-07-14T16:15:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f58237fe700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824101670 0x7f5824072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5822ffd700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58240ffaf0 0x7f58240742c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f581400d980 tx=0x7f581400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:44.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5820ff9700 1 -- 172.21.15.62:0/1748829222 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400b950 con 0x7f58240ffaf0 2023-07-14T16:15:44.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5820ff9700 1 -- 172.21.15.62:0/1748829222 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5814007620 con 0x7f58240ffaf0 2023-07-14T16:15:44.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f5820ff9700 1 -- 172.21.15.62:0/1748829222 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400f3c0 con 0x7f58240ffaf0 2023-07-14T16:15:44.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.161+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582419fab0 con 0x7f58240ffaf0 2023-07-14T16:15:44.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.162+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582419ff70 con 0x7f58240ffaf0 2023-07-14T16:15:44.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.163+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582410b170 con 0x7f58240ffaf0 2023-07-14T16:15:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.165+0000 7f5820ff9700 1 -- 172.21.15.62:0/1748829222 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f581400bab0 con 0x7f58240ffaf0 2023-07-14T16:15:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.166+0000 7f5820ff9700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58100a1150 0x7f58100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.166+0000 7f5820ff9700 1 -- 172.21.15.62:0/1748829222 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f58140cd030 con 0x7f58240ffaf0 2023-07-14T16:15:44.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.166+0000 7f581bfff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58100a1150 0x7f58100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:44.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.167+0000 7f581bfff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58100a1150 0x7f58100a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c010 tx=0x7f580c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:44.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.168+0000 7f5820ff9700 1 -- 172.21.15.62:0/1748829222 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5814096790 con 0x7f58240ffaf0 2023-07-14T16:15:44.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:43 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/998405872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:44.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:43 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1548106020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.377+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5824106a90 con 0x7f58240ffaf0 2023-07-14T16:15:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.377+0000 7f5820ff9700 1 -- 172.21.15.62:0/1748829222 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f581400e350 con 0x7f58240ffaf0 2023-07-14T16:15:44.355 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:15:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58100a1150 msgr2=0x7f58100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58100a1150 0x7f58100a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c010 tx=0x7f580c005960 comp rx=0 tx=0).stop 2023-07-14T16:15:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58240ffaf0 msgr2=0x7f58240742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58240ffaf0 0x7f58240742c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f581400d980 tx=0x7f581400dc90 comp rx=0 tx=0).stop 2023-07-14T16:15:44.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 shutdown_connections 2023-07-14T16:15:44.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5824101670 0x7f5824072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58240ffaf0 0x7f58240742c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58100a1150 0x7f58100a3600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 --2- 172.21.15.62:0/1748829222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5824100ce0 0x7f5824072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 >> 172.21.15.62:0/1748829222 conn(0x7f58240fb130 msgr2=0x7f58240fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 shutdown_connections 2023-07-14T16:15:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.380+0000 7f58297ff700 1 -- 172.21.15.62:0/1748829222 wait complete. 2023-07-14T16:15:44.375 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-14T16:15:44.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:15:44.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3307422750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb04101680 msgr2=0x7feb04107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3307422750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb04101680 0x7feb04107f40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feafc00b0a0 tx=0x7feafc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3307422750 shutdown_connections 2023-07-14T16:15:44.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3307422750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb04101680 0x7feb04107f40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3307422750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb040ffb00 0x7feb040fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3307422750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb04100cf0 0x7feb04101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3307422750 >> 172.21.15.62:0/3307422750 conn(0x7feb040fb0e0 msgr2=0x7feb040fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:44.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3307422750 shutdown_connections 2023-07-14T16:15:44.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.602+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3307422750 wait complete. 2023-07-14T16:15:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.603+0000 7feb0a3ff700 1 Processor -- start 2023-07-14T16:15:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.603+0000 7feb0a3ff700 1 -- start start 2023-07-14T16:15:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb0a3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb040ffb00 0x7feb040742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb0a3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb04100cf0 0x7feb04072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:44.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb0a3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb04101680 0x7feb04072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:44.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb0a3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb040735e0 con 0x7feb040ffb00 2023-07-14T16:15:44.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb0a3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04073750 con 0x7feb04101680 2023-07-14T16:15:44.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb0a3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0419f920 con 0x7feb04100cf0 2023-07-14T16:15:44.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb040ffb00 0x7feb040742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:44.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb040ffb00 0x7feb040742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34986/0 (socket says 172.21.15.62:34986) 2023-07-14T16:15:44.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb03fff700 1 -- 172.21.15.62:0/3443485695 learned_addr learned my addr 172.21.15.62:0/3443485695 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:44.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb037fe700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb04100cf0 0x7feb04072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:44.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb0899c700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb04101680 0x7feb04072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:44.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb03fff700 1 -- 172.21.15.62:0/3443485695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb04100cf0 msgr2=0x7feb04072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb03fff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb04100cf0 0x7feb04072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb03fff700 1 -- 172.21.15.62:0/3443485695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb04101680 msgr2=0x7feb04072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb03fff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb04101680 0x7feb04072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.604+0000 7feb03fff700 1 -- 172.21.15.62:0/3443485695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc009d00 con 0x7feb040ffb00 2023-07-14T16:15:44.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.605+0000 7feb037fe700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb04100cf0 0x7feb04072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:44.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.605+0000 7feb0899c700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb04101680 0x7feb04072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:44.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.605+0000 7feb03fff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb040ffb00 0x7feb040742c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e970 tx=0x7feaf400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:44.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.605+0000 7feb017fa700 1 -- 172.21.15.62:0/3443485695 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4009790 con 0x7feb040ffb00 2023-07-14T16:15:44.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.605+0000 7feb017fa700 1 -- 172.21.15.62:0/3443485695 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feaf40044b0 con 0x7feb040ffb00 2023-07-14T16:15:44.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.605+0000 7feb017fa700 1 -- 172.21.15.62:0/3443485695 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4003dd0 con 0x7feb040ffb00 2023-07-14T16:15:44.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.605+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0419fc00 con 0x7feb040ffb00 2023-07-14T16:15:44.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.605+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb041a00c0 con 0x7feb040ffb00 2023-07-14T16:15:44.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.606+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb04107040 con 0x7feb040ffb00 2023-07-14T16:15:44.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.608+0000 7feb017fa700 1 -- 172.21.15.62:0/3443485695 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feaf4003f30 con 0x7feb040ffb00 2023-07-14T16:15:44.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.610+0000 7feb017fa700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feaec0a11a0 0x7feaec0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:44.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.610+0000 7feb017fa700 1 -- 172.21.15.62:0/3443485695 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feaf4014070 con 0x7feb040ffb00 2023-07-14T16:15:44.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.610+0000 7feb037fe700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feaec0a11a0 0x7feaec0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:44.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.610+0000 7feb037fe700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feaec0a11a0 0x7feaec0a3650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009c90 tx=0x7feaf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:44.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.613+0000 7feb017fa700 1 -- 172.21.15.62:0/3443485695 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feaf40963a0 con 0x7feb040ffb00 2023-07-14T16:15:44.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.869+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb0402c850 con 0x7feb040ffb00 2023-07-14T16:15:44.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.870+0000 7feb017fa700 1 -- 172.21.15.62:0/3443485695 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7feaf40999c0 con 0x7feb040ffb00 2023-07-14T16:15:44.848 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:44.848 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:15:44.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.872+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feaec0a11a0 msgr2=0x7feaec0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.872+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feaec0a11a0 0x7feaec0a3650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009c90 tx=0x7feaf8009260 comp rx=0 tx=0).stop 2023-07-14T16:15:44.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.872+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb040ffb00 msgr2=0x7feb040742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:44.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.872+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb040ffb00 0x7feb040742c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e970 tx=0x7feaf400ed30 comp rx=0 tx=0).stop 2023-07-14T16:15:44.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.872+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 shutdown_connections 2023-07-14T16:15:44.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.872+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7feaec0a11a0 0x7feaec0a3650 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.872+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb04101680 0x7feb04072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.872+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb040ffb00 0x7feb040742c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.873+0000 7feb0a3ff700 1 --2- 172.21.15.62:0/3443485695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb04100cf0 0x7feb04072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:44.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.873+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 >> 172.21.15.62:0/3443485695 conn(0x7feb040fb0e0 msgr2=0x7feb04104180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:44.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.873+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 shutdown_connections 2023-07-14T16:15:44.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:44.873+0000 7feb0a3ff700 1 -- 172.21.15.62:0/3443485695 wait complete. 2023-07-14T16:15:44.852 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:15:44.868 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-14T16:15:44.869 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-14T16:15:44.869 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T16:15:45.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:44 smithi164 ceph-mon[97427]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-14T16:15:45.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:44 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2133478014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:45.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:44 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1748829222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:45.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 -- 172.21.15.62:0/2958932892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708100cf0 msgr2=0x7f8708101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2958932892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708100cf0 0x7f8708101140 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009990 tx=0x7f86fc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 -- 172.21.15.62:0/2958932892 shutdown_connections 2023-07-14T16:15:45.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2958932892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8708101680 0x7f8708107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2958932892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87080ffb00 0x7f87080fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2958932892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708100cf0 0x7f8708101140 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 -- 172.21.15.62:0/2958932892 >> 172.21.15.62:0/2958932892 conn(0x7f87080fb0e0 msgr2=0x7f87080fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:45.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 -- 172.21.15.62:0/2958932892 shutdown_connections 2023-07-14T16:15:45.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.095+0000 7f870d3ff700 1 -- 172.21.15.62:0/2958932892 wait complete. 2023-07-14T16:15:45.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.096+0000 7f870d3ff700 1 Processor -- start 2023-07-14T16:15:45.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.096+0000 7f870d3ff700 1 -- start start 2023-07-14T16:15:45.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f870d3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87080ffb00 0x7f8708197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:45.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f870d3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8708100cf0 0x7f8708197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:45.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f870d3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708101680 0x7f870819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:45.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f870d3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870819f4b0 con 0x7f8708101680 2023-07-14T16:15:45.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f870d3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870819f5f0 con 0x7f87080ffb00 2023-07-14T16:15:45.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f870d3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870819f780 con 0x7f8708100cf0 2023-07-14T16:15:45.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f87077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708101680 0x7f870819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:45.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f87077fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708101680 0x7f870819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35014/0 (socket says 172.21.15.62:35014) 2023-07-14T16:15:45.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f87077fe700 1 -- 172.21.15.62:0/2508902748 learned_addr learned my addr 172.21.15.62:0/2508902748 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:45.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f8706ffd700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87080ffb00 0x7f8708197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:45.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f87077fe700 1 -- 172.21.15.62:0/2508902748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8708100cf0 msgr2=0x7f8708197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f87077fe700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8708100cf0 0x7f8708197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f87077fe700 1 -- 172.21.15.62:0/2508902748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87080ffb00 msgr2=0x7f8708197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f87077fe700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87080ffb00 0x7f8708197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f87077fe700 1 -- 172.21.15.62:0/2508902748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc0095f0 con 0x7f8708101680 2023-07-14T16:15:45.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.097+0000 7f8706ffd700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87080ffb00 0x7f8708197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:45.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.098+0000 7f87077fe700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708101680 0x7f870819eee0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f870000ef30 tx=0x7f870000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:45.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.098+0000 7f86effff700 1 -- 172.21.15.62:0/2508902748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700018070 con 0x7f8708101680 2023-07-14T16:15:45.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.098+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870819fa00 con 0x7f8708101680 2023-07-14T16:15:45.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.098+0000 7f86effff700 1 -- 172.21.15.62:0/2508902748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f870000f040 con 0x7f8708101680 2023-07-14T16:15:45.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.098+0000 7f86effff700 1 -- 172.21.15.62:0/2508902748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700003d00 con 0x7f8708101680 2023-07-14T16:15:45.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.098+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870819fec0 con 0x7f8708101680 2023-07-14T16:15:45.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.099+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870810b1d0 con 0x7f8708101680 2023-07-14T16:15:45.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.101+0000 7f86effff700 1 -- 172.21.15.62:0/2508902748 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f870000caa0 con 0x7f8708101680 2023-07-14T16:15:45.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.103+0000 7f86effff700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86f00a11a0 0x7f86f00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:45.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.103+0000 7f86effff700 1 -- 172.21.15.62:0/2508902748 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87000ccff0 con 0x7f8708101680 2023-07-14T16:15:45.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.103+0000 7f8706ffd700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86f00a11a0 0x7f86f00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:45.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.104+0000 7f8706ffd700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86f00a11a0 0x7f86f00a3650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86f8005950 tx=0x7f86f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:45.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.104+0000 7f86effff700 1 -- 172.21.15.62:0/2508902748 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f87000967e0 con 0x7f8708101680 2023-07-14T16:15:45.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:44 smithi062 ceph-mon[95543]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-14T16:15:45.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:44 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2133478014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:45.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:44 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1748829222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:45.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:44 smithi081 ceph-mon[127795]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-14T16:15:45.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:44 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2133478014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:45.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:44 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1748829222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:45.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.307+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8708107630 con 0x7f86f00a11a0 2023-07-14T16:15:45.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.318+0000 7f86effff700 1 -- 172.21.15.62:0/2508902748 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343591 (secure 0 0 0) 0x7f8708107630 con 0x7f86f00a11a0 2023-07-14T16:15:45.299 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:45.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86f00a11a0 msgr2=0x7f86f00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86f00a11a0 0x7f86f00a3650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86f8005950 tx=0x7f86f80058e0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708101680 msgr2=0x7f870819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708101680 0x7f870819eee0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f870000ef30 tx=0x7f870000c450 comp rx=0 tx=0).stop 2023-07-14T16:15:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 shutdown_connections 2023-07-14T16:15:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8708101680 0x7f870819eee0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86f00a11a0 0x7f86f00a3650 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87080ffb00 0x7f8708197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 --2- 172.21.15.62:0/2508902748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8708100cf0 0x7f8708197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 >> 172.21.15.62:0/2508902748 conn(0x7f87080fb0e0 msgr2=0x7f87080fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 shutdown_connections 2023-07-14T16:15:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.323+0000 7f870d3ff700 1 -- 172.21.15.62:0/2508902748 wait complete. 2023-07-14T16:15:45.303 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-07-14T16:15:45.319 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2023-07-14T16:15:43.495253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658256,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":747,"num_read_kb":486,"num_write":345,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335060,"kb_used_data":11636,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606764,"statfs":{"total":1151940427776,"available":1151597326336,"internally_reserved":0,"allocated":11915264,"data_stored":7464744,"data_compressed":112137,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5817,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":43,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002834"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509907+0000","last_change":"2023-07-14T16:13:59.301892+0000","last_active":"2023-07-14T16:15:19.509907+0000","last_peered":"2023-07-14T16:15:19.509907+0000","last_clean":"2023-07-14T16:15:19.509907+0000","last_became_active":"2023-07-14T16:13:59.301379+0000","last_became_peered":"2023-07-14T16:13:59.301379+0000","last_unstale":"2023-07-14T16:15:19.509907+0000","last_undegraded":"2023-07-14T16:15:19.509907+0000","last_fullsized":"2023-07-14T16:15:19.509907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:49:12.243022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502665+0000","last_change":"2023-07-14T16:13:59.309322+0000","last_active":"2023-07-14T16:15:19.502665+0000","last_peered":"2023-07-14T16:15:19.502665+0000","last_clean":"2023-07-14T16:15:19.502665+0000","last_became_active":"2023-07-14T16:13:59.309035+0000","last_became_peered":"2023-07-14T16:13:59.309035+0000","last_unstale":"2023-07-14T16:15:19.502665+0000","last_undegraded":"2023-07-14T16:15:19.502665+0000","last_fullsized":"2023-07-14T16:15:19.502665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:46.013552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509601+0000","last_change":"2023-07-14T16:13:59.304371+0000","last_active":"2023-07-14T16:15:19.509601+0000","last_peered":"2023-07-14T16:15:19.509601+0000","last_clean":"2023-07-14T16:15:19.509601+0000","last_became_active":"2023-07-14T16:13:59.303965+0000","last_became_peered":"2023-07-14T16:13:59.303965+0000","last_unstale":"2023-07-14T16:15:19.509601+0000","last_undegraded":"2023-07-14T16:15:19.509601+0000","last_fullsized":"2023-07-14T16:15:19.509601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:19.938384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.494491+0000","last_change":"2023-07-14T16:14:20.670094+0000","last_active":"2023-07-14T16:15:19.494491+0000","last_peered":"2023-07-14T16:15:19.494491+0000","last_clean":"2023-07-14T16:15:19.494491+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:15:19.494491+0000","last_undegraded":"2023-07-14T16:15:19.494491+0000","last_fullsized":"2023-07-14T16:15:19.494491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:32.039209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499662+0000","last_change":"2023-07-14T16:13:59.298094+0000","last_active":"2023-07-14T16:15:19.499662+0000","last_peered":"2023-07-14T16:15:19.499662+0000","last_clean":"2023-07-14T16:15:19.499662+0000","last_became_active":"2023-07-14T16:13:59.297764+0000","last_became_peered":"2023-07-14T16:13:59.297764+0000","last_unstale":"2023-07-14T16:15:19.499662+0000","last_undegraded":"2023-07-14T16:15:19.499662+0000","last_fullsized":"2023-07-14T16:15:19.499662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:40:48.501912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503393+0000","last_change":"2023-07-14T16:13:59.299735+0000","last_active":"2023-07-14T16:15:19.503393+0000","last_peered":"2023-07-14T16:15:19.503393+0000","last_clean":"2023-07-14T16:15:19.503393+0000","last_became_active":"2023-07-14T16:13:59.299413+0000","last_became_peered":"2023-07-14T16:13:59.299413+0000","last_unstale":"2023-07-14T16:15:19.503393+0000","last_undegraded":"2023-07-14T16:15:19.503393+0000","last_fullsized":"2023-07-14T16:15:19.503393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:09:10.174873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.493151+0000","last_change":"2023-07-14T16:13:59.299682+0000","last_active":"2023-07-14T16:15:19.493151+0000","last_peered":"2023-07-14T16:15:19.493151+0000","last_clean":"2023-07-14T16:15:19.493151+0000","last_became_active":"2023-07-14T16:13:59.299327+0000","last_became_peered":"2023-07-14T16:13:59.299327+0000","last_unstale":"2023-07-14T16:15:19.493151+0000","last_undegraded":"2023-07-14T16:15:19.493151+0000","last_fullsized":"2023-07-14T16:15:19.493151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:30.735405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502074+0000","last_change":"2023-07-14T16:13:59.311818+0000","last_active":"2023-07-14T16:15:19.502074+0000","last_peered":"2023-07-14T16:15:19.502074+0000","last_clean":"2023-07-14T16:15:19.502074+0000","last_became_active":"2023-07-14T16:13:59.311478+0000","last_became_peered":"2023-07-14T16:13:59.311478+0000","last_unstale":"2023-07-14T16:15:19.502074+0000","last_undegraded":"2023-07-14T16:15:19.502074+0000","last_fullsized":"2023-07-14T16:15:19.502074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:51.107741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507313+0000","last_change":"2023-07-14T16:13:59.309665+0000","last_active":"2023-07-14T16:15:19.507313+0000","last_peered":"2023-07-14T16:15:19.507313+0000","last_clean":"2023-07-14T16:15:19.507313+0000","last_became_active":"2023-07-14T16:13:59.309361+0000","last_became_peered":"2023-07-14T16:13:59.309361+0000","last_unstale":"2023-07-14T16:15:19.507313+0000","last_undegraded":"2023-07-14T16:15:19.507313+0000","last_fullsized":"2023-07-14T16:15:19.507313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:14.297747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508327+0000","last_change":"2023-07-14T16:13:59.296397+0000","last_active":"2023-07-14T16:15:19.508327+0000","last_peered":"2023-07-14T16:15:19.508327+0000","last_clean":"2023-07-14T16:15:19.508327+0000","last_became_active":"2023-07-14T16:13:59.295965+0000","last_became_peered":"2023-07-14T16:13:59.295965+0000","last_unstale":"2023-07-14T16:15:19.508327+0000","last_undegraded":"2023-07-14T16:15:19.508327+0000","last_fullsized":"2023-07-14T16:15:19.508327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:51.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.511367+0000","last_change":"2023-07-14T16:13:59.302892+0000","last_active":"2023-07-14T16:15:19.511367+0000","last_peered":"2023-07-14T16:15:19.511367+0000","last_clean":"2023-07-14T16:15:19.511367+0000","last_became_active":"2023-07-14T16:13:59.302295+0000","last_became_peered":"2023-07-14T16:13:59.302295+0000","last_unstale":"2023-07-14T16:15:19.511367+0000","last_undegraded":"2023-07-14T16:15:19.511367+0000","last_fullsized":"2023-07-14T16:15:19.511367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:11.009590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499348+0000","last_change":"2023-07-14T16:13:59.301913+0000","last_active":"2023-07-14T16:15:19.499348+0000","last_peered":"2023-07-14T16:15:19.499348+0000","last_clean":"2023-07-14T16:15:19.499348+0000","last_became_active":"2023-07-14T16:13:59.301590+0000","last_became_peered":"2023-07-14T16:13:59.301590+0000","last_unstale":"2023-07-14T16:15:19.499348+0000","last_undegraded":"2023-07-14T16:15:19.499348+0000","last_fullsized":"2023-07-14T16:15:19.499348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:09:55.206983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507029+0000","last_change":"2023-07-14T16:13:59.302006+0000","last_active":"2023-07-14T16:15:19.507029+0000","last_peered":"2023-07-14T16:15:19.507029+0000","last_clean":"2023-07-14T16:15:19.507029+0000","last_became_active":"2023-07-14T16:13:59.301752+0000","last_became_peered":"2023-07-14T16:13:59.301752+0000","last_unstale":"2023-07-14T16:15:19.507029+0000","last_undegraded":"2023-07-14T16:15:19.507029+0000","last_fullsized":"2023-07-14T16:15:19.507029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:08.923165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503265+0000","last_change":"2023-07-14T16:13:59.309318+0000","last_active":"2023-07-14T16:15:19.503265+0000","last_peered":"2023-07-14T16:15:19.503265+0000","last_clean":"2023-07-14T16:15:19.503265+0000","last_became_active":"2023-07-14T16:13:59.308931+0000","last_became_peered":"2023-07-14T16:13:59.308931+0000","last_unstale":"2023-07-14T16:15:19.503265+0000","last_undegraded":"2023-07-14T16:15:19.503265+0000","last_fullsized":"2023-07-14T16:15:19.503265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:21:02.747919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498473+0000","last_change":"2023-07-14T16:14:20.670208+0000","last_active":"2023-07-14T16:15:19.498473+0000","last_peered":"2023-07-14T16:15:19.498473+0000","last_clean":"2023-07-14T16:15:19.498473+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:15:19.498473+0000","last_undegraded":"2023-07-14T16:15:19.498473+0000","last_fullsized":"2023-07-14T16:15:19.498473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:53.265928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509331+0000","last_change":"2023-07-14T16:13:59.307986+0000","last_active":"2023-07-14T16:15:19.509331+0000","last_peered":"2023-07-14T16:15:19.509331+0000","last_clean":"2023-07-14T16:15:19.509331+0000","last_became_active":"2023-07-14T16:13:59.307701+0000","last_became_peered":"2023-07-14T16:13:59.307701+0000","last_unstale":"2023-07-14T16:15:19.509331+0000","last_undegraded":"2023-07-14T16:15:19.509331+0000","last_fullsized":"2023-07-14T16:15:19.509331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:56:03.104096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510111+0000","last_change":"2023-07-14T16:13:59.301685+0000","last_active":"2023-07-14T16:15:19.510111+0000","last_peered":"2023-07-14T16:15:19.510111+0000","last_clean":"2023-07-14T16:15:19.510111+0000","last_became_active":"2023-07-14T16:13:59.300905+0000","last_became_peered":"2023-07-14T16:13:59.300905+0000","last_unstale":"2023-07-14T16:15:19.510111+0000","last_undegraded":"2023-07-14T16:15:19.510111+0000","last_fullsized":"2023-07-14T16:15:19.510111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:48:24.721908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498700+0000","last_change":"2023-07-14T16:13:59.306027+0000","last_active":"2023-07-14T16:15:19.498700+0000","last_peered":"2023-07-14T16:15:19.498700+0000","last_clean":"2023-07-14T16:15:19.498700+0000","last_became_active":"2023-07-14T16:13:59.305718+0000","last_became_peered":"2023-07-14T16:13:59.305718+0000","last_unstale":"2023-07-14T16:15:19.498700+0000","last_undegraded":"2023-07-14T16:15:19.498700+0000","last_fullsized":"2023-07-14T16:15:19.498700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:20:47.951543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.494287+0000","last_change":"2023-07-14T16:13:59.294140+0000","last_active":"2023-07-14T16:15:19.494287+0000","last_peered":"2023-07-14T16:15:19.494287+0000","last_clean":"2023-07-14T16:15:19.494287+0000","last_became_active":"2023-07-14T16:13:59.293825+0000","last_became_peered":"2023-07-14T16:13:59.293825+0000","last_unstale":"2023-07-14T16:15:19.494287+0000","last_undegraded":"2023-07-14T16:15:19.494287+0000","last_fullsized":"2023-07-14T16:15:19.494287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:59.348563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665523+0000","last_change":"2023-07-14T16:13:58.277154+0000","last_active":"2023-07-14T16:14:20.665523+0000","last_peered":"2023-07-14T16:14:20.665523+0000","last_clean":"2023-07-14T16:14:20.665523+0000","last_became_active":"2023-07-14T16:13:58.276800+0000","last_became_peered":"2023-07-14T16:13:58.276800+0000","last_unstale":"2023-07-14T16:14:20.665523+0000","last_undegraded":"2023-07-14T16:14:20.665523+0000","last_fullsized":"2023-07-14T16:14:20.665523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:45.631137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.488178+0000","last_change":"2023-07-14T16:13:59.299819+0000","last_active":"2023-07-14T16:15:19.488178+0000","last_peered":"2023-07-14T16:15:19.488178+0000","last_clean":"2023-07-14T16:15:19.488178+0000","last_became_active":"2023-07-14T16:13:59.299294+0000","last_became_peered":"2023-07-14T16:13:59.299294+0000","last_unstale":"2023-07-14T16:15:19.488178+0000","last_undegraded":"2023-07-14T16:15:19.488178+0000","last_fullsized":"2023-07-14T16:15:19.488178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:43.461827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664040+0000","last_change":"2023-07-14T16:13:58.278954+0000","last_active":"2023-07-14T16:14:20.664040+0000","last_peered":"2023-07-14T16:14:20.664040+0000","last_clean":"2023-07-14T16:14:20.664040+0000","last_became_active":"2023-07-14T16:13:58.278454+0000","last_became_peered":"2023-07-14T16:13:58.278454+0000","last_unstale":"2023-07-14T16:14:20.664040+0000","last_undegraded":"2023-07-14T16:14:20.664040+0000","last_fullsized":"2023-07-14T16:14:20.664040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:33:34.015342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510065+0000","last_change":"2023-07-14T16:13:59.299877+0000","last_active":"2023-07-14T16:15:19.510065+0000","last_peered":"2023-07-14T16:15:19.510065+0000","last_clean":"2023-07-14T16:15:19.510065+0000","last_became_active":"2023-07-14T16:13:59.299557+0000","last_became_peered":"2023-07-14T16:13:59.299557+0000","last_unstale":"2023-07-14T16:15:19.510065+0000","last_undegraded":"2023-07-14T16:15:19.510065+0000","last_fullsized":"2023-07-14T16:15:19.510065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:45.331826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.677912+0000","last_change":"2023-07-14T16:14:20.677912+0000","last_active":"2023-07-14T16:14:20.677912+0000","last_peered":"2023-07-14T16:14:20.677912+0000","last_clean":"2023-07-14T16:14:20.677912+0000","last_became_active":"2023-07-14T16:14:20.668156+0000","last_became_peered":"2023-07-14T16:14:20.668156+0000","last_unstale":"2023-07-14T16:14:20.677912+0000","last_undegraded":"2023-07-14T16:14:20.677912+0000","last_fullsized":"2023-07-14T16:14:20.677912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:51:50.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509983+0000","last_change":"2023-07-14T16:13:59.307987+0000","last_active":"2023-07-14T16:15:19.509983+0000","last_peered":"2023-07-14T16:15:19.509983+0000","last_clean":"2023-07-14T16:15:19.509983+0000","last_became_active":"2023-07-14T16:13:59.307457+0000","last_became_peered":"2023-07-14T16:13:59.307457+0000","last_unstale":"2023-07-14T16:15:19.509983+0000","last_undegraded":"2023-07-14T16:15:19.509983+0000","last_fullsized":"2023-07-14T16:15:19.509983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:32:28.569384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663585+0000","last_change":"2023-07-14T16:13:58.282332+0000","last_active":"2023-07-14T16:14:20.663585+0000","last_peered":"2023-07-14T16:14:20.663585+0000","last_clean":"2023-07-14T16:14:20.663585+0000","last_became_active":"2023-07-14T16:13:58.278103+0000","last_became_peered":"2023-07-14T16:13:58.278103+0000","last_unstale":"2023-07-14T16:14:20.663585+0000","last_undegraded":"2023-07-14T16:14:20.663585+0000","last_fullsized":"2023-07-14T16:14:20.663585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:00.081710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509816+0000","last_change":"2023-07-14T16:13:59.301036+0000","last_active":"2023-07-14T16:15:19.509816+0000","last_peered":"2023-07-14T16:15:19.509816+0000","last_clean":"2023-07-14T16:15:19.509816+0000","last_became_active":"2023-07-14T16:13:59.300787+0000","last_became_peered":"2023-07-14T16:13:59.300787+0000","last_unstale":"2023-07-14T16:15:19.509816+0000","last_undegraded":"2023-07-14T16:15:19.509816+0000","last_fullsized":"2023-07-14T16:15:19.509816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:40:56.654793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":127,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:41.719917+0000","last_change":"2023-07-14T16:13:58.282392+0000","last_active":"2023-07-14T16:15:41.719917+0000","last_peered":"2023-07-14T16:15:41.719917+0000","last_clean":"2023-07-14T16:15:41.719917+0000","last_became_active":"2023-07-14T16:13:58.282030+0000","last_became_peered":"2023-07-14T16:13:58.282030+0000","last_unstale":"2023-07-14T16:15:41.719917+0000","last_undegraded":"2023-07-14T16:15:41.719917+0000","last_fullsized":"2023-07-14T16:15:41.719917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:17.795386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.495045+0000","last_change":"2023-07-14T16:13:59.292681+0000","last_active":"2023-07-14T16:15:19.495045+0000","last_peered":"2023-07-14T16:15:19.495045+0000","last_clean":"2023-07-14T16:15:19.495045+0000","last_became_active":"2023-07-14T16:13:59.292289+0000","last_became_peered":"2023-07-14T16:13:59.292289+0000","last_unstale":"2023-07-14T16:15:19.495045+0000","last_undegraded":"2023-07-14T16:15:19.495045+0000","last_fullsized":"2023-07-14T16:15:19.495045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:33.971255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666548+0000","last_change":"2023-07-14T16:13:58.282111+0000","last_active":"2023-07-14T16:14:20.666548+0000","last_peered":"2023-07-14T16:14:20.666548+0000","last_clean":"2023-07-14T16:14:20.666548+0000","last_became_active":"2023-07-14T16:13:58.281783+0000","last_became_peered":"2023-07-14T16:13:58.281783+0000","last_unstale":"2023-07-14T16:14:20.666548+0000","last_undegraded":"2023-07-14T16:14:20.666548+0000","last_fullsized":"2023-07-14T16:14:20.666548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:51:40.718143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501115+0000","last_change":"2023-07-14T16:13:59.297439+0000","last_active":"2023-07-14T16:15:19.501115+0000","last_peered":"2023-07-14T16:15:19.501115+0000","last_clean":"2023-07-14T16:15:19.501115+0000","last_became_active":"2023-07-14T16:13:59.297117+0000","last_became_peered":"2023-07-14T16:13:59.297117+0000","last_unstale":"2023-07-14T16:15:19.501115+0000","last_undegraded":"2023-07-14T16:15:19.501115+0000","last_fullsized":"2023-07-14T16:15:19.501115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:08.122607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763285+0000","last_change":"2023-07-14T16:13:58.274826+0000","last_active":"2023-07-14T16:14:20.763285+0000","last_peered":"2023-07-14T16:14:20.763285+0000","last_clean":"2023-07-14T16:14:20.763285+0000","last_became_active":"2023-07-14T16:13:58.274406+0000","last_became_peered":"2023-07-14T16:13:58.274406+0000","last_unstale":"2023-07-14T16:14:20.763285+0000","last_undegraded":"2023-07-14T16:14:20.763285+0000","last_fullsized":"2023-07-14T16:14:20.763285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:09.422218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500875+0000","last_change":"2023-07-14T16:13:59.296730+0000","last_active":"2023-07-14T16:15:19.500875+0000","last_peered":"2023-07-14T16:15:19.500875+0000","last_clean":"2023-07-14T16:15:19.500875+0000","last_became_active":"2023-07-14T16:13:59.296372+0000","last_became_peered":"2023-07-14T16:13:59.296372+0000","last_unstale":"2023-07-14T16:15:19.500875+0000","last_undegraded":"2023-07-14T16:15:19.500875+0000","last_fullsized":"2023-07-14T16:15:19.500875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:07.809329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664772+0000","last_change":"2023-07-14T16:13:58.269122+0000","last_active":"2023-07-14T16:14:20.664772+0000","last_peered":"2023-07-14T16:14:20.664772+0000","last_clean":"2023-07-14T16:14:20.664772+0000","last_became_active":"2023-07-14T16:13:58.268770+0000","last_became_peered":"2023-07-14T16:13:58.268770+0000","last_unstale":"2023-07-14T16:14:20.664772+0000","last_undegraded":"2023-07-14T16:14:20.664772+0000","last_fullsized":"2023-07-14T16:14:20.664772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:43.584997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.506513+0000","last_change":"2023-07-14T16:13:59.306987+0000","last_active":"2023-07-14T16:15:19.506513+0000","last_peered":"2023-07-14T16:15:19.506513+0000","last_clean":"2023-07-14T16:15:19.506513+0000","last_became_active":"2023-07-14T16:13:59.306649+0000","last_became_peered":"2023-07-14T16:13:59.306649+0000","last_unstale":"2023-07-14T16:15:19.506513+0000","last_undegraded":"2023-07-14T16:15:19.506513+0000","last_fullsized":"2023-07-14T16:15:19.506513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:35:23.579814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:41.719379+0000","last_change":"2023-07-14T16:13:58.281319+0000","last_active":"2023-07-14T16:15:41.719379+0000","last_peered":"2023-07-14T16:15:41.719379+0000","last_clean":"2023-07-14T16:15:41.719379+0000","last_became_active":"2023-07-14T16:13:58.280928+0000","last_became_peered":"2023-07-14T16:13:58.280928+0000","last_unstale":"2023-07-14T16:15:41.719379+0000","last_undegraded":"2023-07-14T16:15:41.719379+0000","last_fullsized":"2023-07-14T16:15:41.719379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:41.337954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507694+0000","last_change":"2023-07-14T16:13:59.309586+0000","last_active":"2023-07-14T16:15:19.507694+0000","last_peered":"2023-07-14T16:15:19.507694+0000","last_clean":"2023-07-14T16:15:19.507694+0000","last_became_active":"2023-07-14T16:13:59.309243+0000","last_became_peered":"2023-07-14T16:13:59.309243+0000","last_unstale":"2023-07-14T16:15:19.507694+0000","last_undegraded":"2023-07-14T16:15:19.507694+0000","last_fullsized":"2023-07-14T16:15:19.507694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:30.700680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661328+0000","last_change":"2023-07-14T16:13:58.279769+0000","last_active":"2023-07-14T16:14:20.661328+0000","last_peered":"2023-07-14T16:14:20.661328+0000","last_clean":"2023-07-14T16:14:20.661328+0000","last_became_active":"2023-07-14T16:13:58.279409+0000","last_became_peered":"2023-07-14T16:13:58.279409+0000","last_unstale":"2023-07-14T16:14:20.661328+0000","last_undegraded":"2023-07-14T16:14:20.661328+0000","last_fullsized":"2023-07-14T16:14:20.661328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:32.751188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500584+0000","last_change":"2023-07-14T16:13:59.300382+0000","last_active":"2023-07-14T16:15:19.500584+0000","last_peered":"2023-07-14T16:15:19.500584+0000","last_clean":"2023-07-14T16:15:19.500584+0000","last_became_active":"2023-07-14T16:13:59.299981+0000","last_became_peered":"2023-07-14T16:13:59.299981+0000","last_unstale":"2023-07-14T16:15:19.500584+0000","last_undegraded":"2023-07-14T16:15:19.500584+0000","last_fullsized":"2023-07-14T16:15:19.500584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:46:04.769609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665031+0000","last_change":"2023-07-14T16:13:58.274128+0000","last_active":"2023-07-14T16:14:20.665031+0000","last_peered":"2023-07-14T16:14:20.665031+0000","last_clean":"2023-07-14T16:14:20.665031+0000","last_became_active":"2023-07-14T16:13:58.273446+0000","last_became_peered":"2023-07-14T16:13:58.273446+0000","last_unstale":"2023-07-14T16:14:20.665031+0000","last_undegraded":"2023-07-14T16:14:20.665031+0000","last_fullsized":"2023-07-14T16:14:20.665031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:12.510793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510736+0000","last_change":"2023-07-14T16:13:59.301551+0000","last_active":"2023-07-14T16:15:19.510736+0000","last_peered":"2023-07-14T16:15:19.510736+0000","last_clean":"2023-07-14T16:15:19.510736+0000","last_became_active":"2023-07-14T16:13:59.301206+0000","last_became_peered":"2023-07-14T16:13:59.301206+0000","last_unstale":"2023-07-14T16:15:19.510736+0000","last_undegraded":"2023-07-14T16:15:19.510736+0000","last_fullsized":"2023-07-14T16:15:19.510736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:16.227849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":259,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:42.721000+0000","last_change":"2023-07-14T16:13:58.283304+0000","last_active":"2023-07-14T16:15:42.721000+0000","last_peered":"2023-07-14T16:15:42.721000+0000","last_clean":"2023-07-14T16:15:42.721000+0000","last_became_active":"2023-07-14T16:13:58.282930+0000","last_became_peered":"2023-07-14T16:13:58.282930+0000","last_unstale":"2023-07-14T16:15:42.721000+0000","last_undegraded":"2023-07-14T16:15:42.721000+0000","last_fullsized":"2023-07-14T16:15:42.721000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:02.538305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62599,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":209,"num_read_kb":161,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500725+0000","last_change":"2023-07-14T16:13:59.304144+0000","last_active":"2023-07-14T16:15:19.500725+0000","last_peered":"2023-07-14T16:15:19.500725+0000","last_clean":"2023-07-14T16:15:19.500725+0000","last_became_active":"2023-07-14T16:13:59.303745+0000","last_became_peered":"2023-07-14T16:13:59.303745+0000","last_unstale":"2023-07-14T16:15:19.500725+0000","last_undegraded":"2023-07-14T16:15:19.500725+0000","last_fullsized":"2023-07-14T16:15:19.500725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:20.828221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667830+0000","last_change":"2023-07-14T16:13:58.283734+0000","last_active":"2023-07-14T16:14:20.667830+0000","last_peered":"2023-07-14T16:14:20.667830+0000","last_clean":"2023-07-14T16:14:20.667830+0000","last_became_active":"2023-07-14T16:13:58.282765+0000","last_became_peered":"2023-07-14T16:13:58.282765+0000","last_unstale":"2023-07-14T16:14:20.667830+0000","last_undegraded":"2023-07-14T16:14:20.667830+0000","last_fullsized":"2023-07-14T16:14:20.667830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:13.217664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500485+0000","last_change":"2023-07-14T16:13:59.308445+0000","last_active":"2023-07-14T16:15:19.500485+0000","last_peered":"2023-07-14T16:15:19.500485+0000","last_clean":"2023-07-14T16:15:19.500485+0000","last_became_active":"2023-07-14T16:13:59.308130+0000","last_became_peered":"2023-07-14T16:13:59.308130+0000","last_unstale":"2023-07-14T16:15:19.500485+0000","last_undegraded":"2023-07-14T16:15:19.500485+0000","last_fullsized":"2023-07-14T16:15:19.500485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:45.004512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664202+0000","last_change":"2023-07-14T16:13:58.273973+0000","last_active":"2023-07-14T16:14:20.664202+0000","last_peered":"2023-07-14T16:14:20.664202+0000","last_clean":"2023-07-14T16:14:20.664202+0000","last_became_active":"2023-07-14T16:13:58.273606+0000","last_became_peered":"2023-07-14T16:13:58.273606+0000","last_unstale":"2023-07-14T16:14:20.664202+0000","last_undegraded":"2023-07-14T16:14:20.664202+0000","last_fullsized":"2023-07-14T16:14:20.664202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:12.304491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501694+0000","last_change":"2023-07-14T16:13:59.306428+0000","last_active":"2023-07-14T16:15:19.501694+0000","last_peered":"2023-07-14T16:15:19.501694+0000","last_clean":"2023-07-14T16:15:19.501694+0000","last_became_active":"2023-07-14T16:13:59.306090+0000","last_became_peered":"2023-07-14T16:13:59.306090+0000","last_unstale":"2023-07-14T16:15:19.501694+0000","last_undegraded":"2023-07-14T16:15:19.501694+0000","last_fullsized":"2023-07-14T16:15:19.501694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:16.033202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":125,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:38.789441+0000","last_change":"2023-07-14T16:13:58.277123+0000","last_active":"2023-07-14T16:15:38.789441+0000","last_peered":"2023-07-14T16:15:38.789441+0000","last_clean":"2023-07-14T16:15:38.789441+0000","last_became_active":"2023-07-14T16:13:58.276815+0000","last_became_peered":"2023-07-14T16:13:58.276815+0000","last_unstale":"2023-07-14T16:15:38.789441+0000","last_undegraded":"2023-07-14T16:15:38.789441+0000","last_fullsized":"2023-07-14T16:15:38.789441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:41.019233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1570,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504574+0000","last_change":"2023-07-14T16:13:59.310238+0000","last_active":"2023-07-14T16:15:19.504574+0000","last_peered":"2023-07-14T16:15:19.504574+0000","last_clean":"2023-07-14T16:15:19.504574+0000","last_became_active":"2023-07-14T16:13:59.309918+0000","last_became_peered":"2023-07-14T16:13:59.309918+0000","last_unstale":"2023-07-14T16:15:19.504574+0000","last_undegraded":"2023-07-14T16:15:19.504574+0000","last_fullsized":"2023-07-14T16:15:19.504574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:25.020575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663660+0000","last_change":"2023-07-14T16:13:58.280834+0000","last_active":"2023-07-14T16:14:20.663660+0000","last_peered":"2023-07-14T16:14:20.663660+0000","last_clean":"2023-07-14T16:14:20.663660+0000","last_became_active":"2023-07-14T16:13:58.280497+0000","last_became_peered":"2023-07-14T16:13:58.280497+0000","last_unstale":"2023-07-14T16:14:20.663660+0000","last_undegraded":"2023-07-14T16:14:20.663660+0000","last_fullsized":"2023-07-14T16:14:20.663660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:07:02.308388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.489632+0000","last_change":"2023-07-14T16:14:20.764846+0000","last_active":"2023-07-14T16:15:19.489632+0000","last_peered":"2023-07-14T16:15:19.489632+0000","last_clean":"2023-07-14T16:15:19.489632+0000","last_became_active":"2023-07-14T16:14:20.764485+0000","last_became_peered":"2023-07-14T16:14:20.764485+0000","last_unstale":"2023-07-14T16:15:19.489632+0000","last_undegraded":"2023-07-14T16:15:19.489632+0000","last_fullsized":"2023-07-14T16:15:19.489632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:29.245874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.682843+0000","last_change":"2023-07-14T16:14:20.682331+0000","last_active":"2023-07-14T16:14:20.682843+0000","last_peered":"2023-07-14T16:14:20.682843+0000","last_clean":"2023-07-14T16:14:20.682843+0000","last_became_active":"2023-07-14T16:14:20.669179+0000","last_became_peered":"2023-07-14T16:14:20.669179+0000","last_unstale":"2023-07-14T16:14:20.682843+0000","last_undegraded":"2023-07-14T16:14:20.682843+0000","last_fullsized":"2023-07-14T16:14:20.682843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:55:10.629011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502602+0000","last_change":"2023-07-14T16:13:59.297644+0000","last_active":"2023-07-14T16:15:19.502602+0000","last_peered":"2023-07-14T16:15:19.502602+0000","last_clean":"2023-07-14T16:15:19.502602+0000","last_became_active":"2023-07-14T16:13:59.297332+0000","last_became_peered":"2023-07-14T16:13:59.297332+0000","last_unstale":"2023-07-14T16:15:19.502602+0000","last_undegraded":"2023-07-14T16:15:19.502602+0000","last_fullsized":"2023-07-14T16:15:19.502602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:36.421075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665695+0000","last_change":"2023-07-14T16:13:58.280632+0000","last_active":"2023-07-14T16:14:20.665695+0000","last_peered":"2023-07-14T16:14:20.665695+0000","last_clean":"2023-07-14T16:14:20.665695+0000","last_became_active":"2023-07-14T16:13:58.280296+0000","last_became_peered":"2023-07-14T16:13:58.280296+0000","last_unstale":"2023-07-14T16:14:20.665695+0000","last_undegraded":"2023-07-14T16:14:20.665695+0000","last_fullsized":"2023-07-14T16:14:20.665695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:26.897678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.505470+0000","last_change":"2023-07-14T16:13:59.297092+0000","last_active":"2023-07-14T16:15:19.505470+0000","last_peered":"2023-07-14T16:15:19.505470+0000","last_clean":"2023-07-14T16:15:19.505470+0000","last_became_active":"2023-07-14T16:13:59.296750+0000","last_became_peered":"2023-07-14T16:13:59.296750+0000","last_unstale":"2023-07-14T16:15:19.505470+0000","last_undegraded":"2023-07-14T16:15:19.505470+0000","last_fullsized":"2023-07-14T16:15:19.505470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:32.797912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660888+0000","last_change":"2023-07-14T16:13:58.281786+0000","last_active":"2023-07-14T16:14:20.660888+0000","last_peered":"2023-07-14T16:14:20.660888+0000","last_clean":"2023-07-14T16:14:20.660888+0000","last_became_active":"2023-07-14T16:13:58.281471+0000","last_became_peered":"2023-07-14T16:13:58.281471+0000","last_unstale":"2023-07-14T16:14:20.660888+0000","last_undegraded":"2023-07-14T16:14:20.660888+0000","last_fullsized":"2023-07-14T16:14:20.660888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:09.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502616+0000","last_change":"2023-07-14T16:14:20.672813+0000","last_active":"2023-07-14T16:15:19.502616+0000","last_peered":"2023-07-14T16:15:19.502616+0000","last_clean":"2023-07-14T16:15:19.502616+0000","last_became_active":"2023-07-14T16:14:20.672447+0000","last_became_peered":"2023-07-14T16:14:20.672447+0000","last_unstale":"2023-07-14T16:15:19.502616+0000","last_undegraded":"2023-07-14T16:15:19.502616+0000","last_fullsized":"2023-07-14T16:15:19.502616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:33:07.582055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.495526+0000","last_change":"2023-07-14T16:13:59.288836+0000","last_active":"2023-07-14T16:15:19.495526+0000","last_peered":"2023-07-14T16:15:19.495526+0000","last_clean":"2023-07-14T16:15:19.495526+0000","last_became_active":"2023-07-14T16:13:59.288517+0000","last_became_peered":"2023-07-14T16:13:59.288517+0000","last_unstale":"2023-07-14T16:15:19.495526+0000","last_undegraded":"2023-07-14T16:15:19.495526+0000","last_fullsized":"2023-07-14T16:15:19.495526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:48.293313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":119,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:38.741576+0000","last_change":"2023-07-14T16:13:58.282415+0000","last_active":"2023-07-14T16:15:38.741576+0000","last_peered":"2023-07-14T16:15:38.741576+0000","last_clean":"2023-07-14T16:15:38.741576+0000","last_became_active":"2023-07-14T16:13:58.282118+0000","last_became_peered":"2023-07-14T16:13:58.282118+0000","last_unstale":"2023-07-14T16:15:38.741576+0000","last_undegraded":"2023-07-14T16:15:38.741576+0000","last_fullsized":"2023-07-14T16:15:38.741576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:41.532341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501135+0000","last_change":"2023-07-14T16:13:59.302860+0000","last_active":"2023-07-14T16:15:19.501135+0000","last_peered":"2023-07-14T16:15:19.501135+0000","last_clean":"2023-07-14T16:15:19.501135+0000","last_became_active":"2023-07-14T16:13:59.302454+0000","last_became_peered":"2023-07-14T16:13:59.302454+0000","last_unstale":"2023-07-14T16:15:19.501135+0000","last_undegraded":"2023-07-14T16:15:19.501135+0000","last_fullsized":"2023-07-14T16:15:19.501135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:34.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664397+0000","last_change":"2023-07-14T16:13:58.273908+0000","last_active":"2023-07-14T16:14:20.664397+0000","last_peered":"2023-07-14T16:14:20.664397+0000","last_clean":"2023-07-14T16:14:20.664397+0000","last_became_active":"2023-07-14T16:13:58.273497+0000","last_became_peered":"2023-07-14T16:13:58.273497+0000","last_unstale":"2023-07-14T16:14:20.664397+0000","last_undegraded":"2023-07-14T16:14:20.664397+0000","last_fullsized":"2023-07-14T16:14:20.664397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:55:44.978027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661044+0000","last_change":"2023-07-14T16:12:36.546475+0000","last_active":"2023-07-14T16:14:20.661044+0000","last_peered":"2023-07-14T16:14:20.661044+0000","last_clean":"2023-07-14T16:14:20.661044+0000","last_became_active":"2023-07-14T16:12:36.545943+0000","last_became_peered":"2023-07-14T16:12:36.545943+0000","last_unstale":"2023-07-14T16:14:20.661044+0000","last_undegraded":"2023-07-14T16:14:20.661044+0000","last_fullsized":"2023-07-14T16:14:20.661044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_clean_scrub_stamp":"2023-07-14T16:08:58.024408+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:27.166990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500653+0000","last_change":"2023-07-14T16:13:59.297669+0000","last_active":"2023-07-14T16:15:19.500653+0000","last_peered":"2023-07-14T16:15:19.500653+0000","last_clean":"2023-07-14T16:15:19.500653+0000","last_became_active":"2023-07-14T16:13:59.297364+0000","last_became_peered":"2023-07-14T16:13:59.297364+0000","last_unstale":"2023-07-14T16:15:19.500653+0000","last_undegraded":"2023-07-14T16:15:19.500653+0000","last_fullsized":"2023-07-14T16:15:19.500653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:50:00.972727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.673166+0000","last_change":"2023-07-14T16:14:20.673166+0000","last_active":"2023-07-14T16:14:20.673166+0000","last_peered":"2023-07-14T16:14:20.673166+0000","last_clean":"2023-07-14T16:14:20.673166+0000","last_became_active":"2023-07-14T16:14:20.667664+0000","last_became_peered":"2023-07-14T16:14:20.667664+0000","last_unstale":"2023-07-14T16:14:20.673166+0000","last_undegraded":"2023-07-14T16:14:20.673166+0000","last_fullsized":"2023-07-14T16:14:20.673166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:24.328768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664285+0000","last_change":"2023-07-14T16:13:58.276257+0000","last_active":"2023-07-14T16:14:20.664285+0000","last_peered":"2023-07-14T16:14:20.664285+0000","last_clean":"2023-07-14T16:14:20.664285+0000","last_became_active":"2023-07-14T16:13:58.275910+0000","last_became_peered":"2023-07-14T16:13:58.275910+0000","last_unstale":"2023-07-14T16:14:20.664285+0000","last_undegraded":"2023-07-14T16:14:20.664285+0000","last_fullsized":"2023-07-14T16:14:20.664285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:47.700225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498722+0000","last_change":"2023-07-14T16:13:59.299135+0000","last_active":"2023-07-14T16:15:19.498722+0000","last_peered":"2023-07-14T16:15:19.498722+0000","last_clean":"2023-07-14T16:15:19.498722+0000","last_became_active":"2023-07-14T16:13:59.298031+0000","last_became_peered":"2023-07-14T16:13:59.298031+0000","last_unstale":"2023-07-14T16:15:19.498722+0000","last_undegraded":"2023-07-14T16:15:19.498722+0000","last_fullsized":"2023-07-14T16:15:19.498722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:21.882960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665411+0000","last_change":"2023-07-14T16:13:58.279664+0000","last_active":"2023-07-14T16:14:20.665411+0000","last_peered":"2023-07-14T16:14:20.665411+0000","last_clean":"2023-07-14T16:14:20.665411+0000","last_became_active":"2023-07-14T16:13:58.279137+0000","last_became_peered":"2023-07-14T16:13:58.279137+0000","last_unstale":"2023-07-14T16:14:20.665411+0000","last_undegraded":"2023-07-14T16:14:20.665411+0000","last_fullsized":"2023-07-14T16:14:20.665411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:07.657781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500067+0000","last_change":"2023-07-14T16:13:59.305536+0000","last_active":"2023-07-14T16:15:19.500067+0000","last_peered":"2023-07-14T16:15:19.500067+0000","last_clean":"2023-07-14T16:15:19.500067+0000","last_became_active":"2023-07-14T16:13:59.305176+0000","last_became_peered":"2023-07-14T16:13:59.305176+0000","last_unstale":"2023-07-14T16:15:19.500067+0000","last_undegraded":"2023-07-14T16:15:19.500067+0000","last_fullsized":"2023-07-14T16:15:19.500067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:18:18.141957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664829+0000","last_change":"2023-07-14T16:13:58.269652+0000","last_active":"2023-07-14T16:14:20.664829+0000","last_peered":"2023-07-14T16:14:20.664829+0000","last_clean":"2023-07-14T16:14:20.664829+0000","last_became_active":"2023-07-14T16:13:58.269338+0000","last_became_peered":"2023-07-14T16:13:58.269338+0000","last_unstale":"2023-07-14T16:14:20.664829+0000","last_undegraded":"2023-07-14T16:14:20.664829+0000","last_fullsized":"2023-07-14T16:14:20.664829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:55:21.910971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502139+0000","last_change":"2023-07-14T16:13:59.297574+0000","last_active":"2023-07-14T16:15:19.502139+0000","last_peered":"2023-07-14T16:15:19.502139+0000","last_clean":"2023-07-14T16:15:19.502139+0000","last_became_active":"2023-07-14T16:13:59.297124+0000","last_became_peered":"2023-07-14T16:13:59.297124+0000","last_unstale":"2023-07-14T16:15:19.502139+0000","last_undegraded":"2023-07-14T16:15:19.502139+0000","last_fullsized":"2023-07-14T16:15:19.502139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:20.051395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763343+0000","last_change":"2023-07-14T16:13:58.278011+0000","last_active":"2023-07-14T16:14:20.763343+0000","last_peered":"2023-07-14T16:14:20.763343+0000","last_clean":"2023-07-14T16:14:20.763343+0000","last_became_active":"2023-07-14T16:13:58.277563+0000","last_became_peered":"2023-07-14T16:13:58.277563+0000","last_unstale":"2023-07-14T16:14:20.763343+0000","last_undegraded":"2023-07-14T16:14:20.763343+0000","last_fullsized":"2023-07-14T16:14:20.763343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:06.354928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.463529+0000","last_change":"2023-07-14T16:13:59.308836+0000","last_active":"2023-07-14T16:15:19.463529+0000","last_peered":"2023-07-14T16:15:19.463529+0000","last_clean":"2023-07-14T16:15:19.463529+0000","last_became_active":"2023-07-14T16:13:59.308510+0000","last_became_peered":"2023-07-14T16:13:59.308510+0000","last_unstale":"2023-07-14T16:15:19.463529+0000","last_undegraded":"2023-07-14T16:15:19.463529+0000","last_fullsized":"2023-07-14T16:15:19.463529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:48:07.973849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663352+0000","last_change":"2023-07-14T16:13:58.277438+0000","last_active":"2023-07-14T16:14:20.663352+0000","last_peered":"2023-07-14T16:14:20.663352+0000","last_clean":"2023-07-14T16:14:20.663352+0000","last_became_active":"2023-07-14T16:13:58.277114+0000","last_became_peered":"2023-07-14T16:13:58.277114+0000","last_unstale":"2023-07-14T16:14:20.663352+0000","last_undegraded":"2023-07-14T16:14:20.663352+0000","last_fullsized":"2023-07-14T16:14:20.663352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:40.636429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502829+0000","last_change":"2023-07-14T16:13:59.304491+0000","last_active":"2023-07-14T16:15:19.502829+0000","last_peered":"2023-07-14T16:15:19.502829+0000","last_clean":"2023-07-14T16:15:19.502829+0000","last_became_active":"2023-07-14T16:13:59.303925+0000","last_became_peered":"2023-07-14T16:13:59.303925+0000","last_unstale":"2023-07-14T16:15:19.502829+0000","last_undegraded":"2023-07-14T16:15:19.502829+0000","last_fullsized":"2023-07-14T16:15:19.502829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.410847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.685314+0000","last_change":"2023-07-14T16:14:20.684842+0000","last_active":"2023-07-14T16:14:20.685314+0000","last_peered":"2023-07-14T16:14:20.685314+0000","last_clean":"2023-07-14T16:14:20.685314+0000","last_became_active":"2023-07-14T16:14:20.670098+0000","last_became_peered":"2023-07-14T16:14:20.670098+0000","last_unstale":"2023-07-14T16:14:20.685314+0000","last_undegraded":"2023-07-14T16:14:20.685314+0000","last_fullsized":"2023-07-14T16:14:20.685314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:54:58.021152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502027+0000","last_change":"2023-07-14T16:13:59.310983+0000","last_active":"2023-07-14T16:15:19.502027+0000","last_peered":"2023-07-14T16:15:19.502027+0000","last_clean":"2023-07-14T16:15:19.502027+0000","last_became_active":"2023-07-14T16:13:59.310636+0000","last_became_peered":"2023-07-14T16:13:59.310636+0000","last_unstale":"2023-07-14T16:15:19.502027+0000","last_undegraded":"2023-07-14T16:15:19.502027+0000","last_fullsized":"2023-07-14T16:15:19.502027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:31.003517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663068+0000","last_change":"2023-07-14T16:13:58.280153+0000","last_active":"2023-07-14T16:14:20.663068+0000","last_peered":"2023-07-14T16:14:20.663068+0000","last_clean":"2023-07-14T16:14:20.663068+0000","last_became_active":"2023-07-14T16:13:58.279771+0000","last_became_peered":"2023-07-14T16:13:58.279771+0000","last_unstale":"2023-07-14T16:14:20.663068+0000","last_undegraded":"2023-07-14T16:14:20.663068+0000","last_fullsized":"2023-07-14T16:14:20.663068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:46.290429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510162+0000","last_change":"2023-07-14T16:13:59.304854+0000","last_active":"2023-07-14T16:15:19.510162+0000","last_peered":"2023-07-14T16:15:19.510162+0000","last_clean":"2023-07-14T16:15:19.510162+0000","last_became_active":"2023-07-14T16:13:59.304374+0000","last_became_peered":"2023-07-14T16:13:59.304374+0000","last_unstale":"2023-07-14T16:15:19.510162+0000","last_undegraded":"2023-07-14T16:15:19.510162+0000","last_fullsized":"2023-07-14T16:15:19.510162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:52.213737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667834+0000","last_change":"2023-07-14T16:13:58.281885+0000","last_active":"2023-07-14T16:14:20.667834+0000","last_peered":"2023-07-14T16:14:20.667834+0000","last_clean":"2023-07-14T16:14:20.667834+0000","last_became_active":"2023-07-14T16:13:58.278759+0000","last_became_peered":"2023-07-14T16:13:58.278759+0000","last_unstale":"2023-07-14T16:14:20.667834+0000","last_undegraded":"2023-07-14T16:14:20.667834+0000","last_fullsized":"2023-07-14T16:14:20.667834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:11.816765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.511619+0000","last_change":"2023-07-14T16:13:59.302680+0000","last_active":"2023-07-14T16:15:19.511619+0000","last_peered":"2023-07-14T16:15:19.511619+0000","last_clean":"2023-07-14T16:15:19.511619+0000","last_became_active":"2023-07-14T16:13:59.301854+0000","last_became_peered":"2023-07-14T16:13:59.301854+0000","last_unstale":"2023-07-14T16:15:19.511619+0000","last_undegraded":"2023-07-14T16:15:19.511619+0000","last_fullsized":"2023-07-14T16:15:19.511619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:29:42.861394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664860+0000","last_change":"2023-07-14T16:13:58.274254+0000","last_active":"2023-07-14T16:14:20.664860+0000","last_peered":"2023-07-14T16:14:20.664860+0000","last_clean":"2023-07-14T16:14:20.664860+0000","last_became_active":"2023-07-14T16:13:58.273933+0000","last_became_peered":"2023-07-14T16:13:58.273933+0000","last_unstale":"2023-07-14T16:14:20.664860+0000","last_undegraded":"2023-07-14T16:14:20.664860+0000","last_fullsized":"2023-07-14T16:14:20.664860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:53.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501047+0000","last_change":"2023-07-14T16:13:59.305615+0000","last_active":"2023-07-14T16:15:19.501047+0000","last_peered":"2023-07-14T16:15:19.501047+0000","last_clean":"2023-07-14T16:15:19.501047+0000","last_became_active":"2023-07-14T16:13:59.305206+0000","last_became_peered":"2023-07-14T16:13:59.305206+0000","last_unstale":"2023-07-14T16:15:19.501047+0000","last_undegraded":"2023-07-14T16:15:19.501047+0000","last_fullsized":"2023-07-14T16:15:19.501047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:41.771061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763463+0000","last_change":"2023-07-14T16:13:58.277976+0000","last_active":"2023-07-14T16:14:20.763463+0000","last_peered":"2023-07-14T16:14:20.763463+0000","last_clean":"2023-07-14T16:14:20.763463+0000","last_became_active":"2023-07-14T16:13:58.277618+0000","last_became_peered":"2023-07-14T16:13:58.277618+0000","last_unstale":"2023-07-14T16:14:20.763463+0000","last_undegraded":"2023-07-14T16:14:20.763463+0000","last_fullsized":"2023-07-14T16:14:20.763463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:25.780591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504344+0000","last_change":"2023-07-14T16:13:59.302041+0000","last_active":"2023-07-14T16:15:19.504344+0000","last_peered":"2023-07-14T16:15:19.504344+0000","last_clean":"2023-07-14T16:15:19.504344+0000","last_became_active":"2023-07-14T16:13:59.301620+0000","last_became_peered":"2023-07-14T16:13:59.301620+0000","last_unstale":"2023-07-14T16:15:19.504344+0000","last_undegraded":"2023-07-14T16:15:19.504344+0000","last_fullsized":"2023-07-14T16:15:19.504344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:18.969064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502705+0000","last_change":"2023-07-14T16:13:59.304222+0000","last_active":"2023-07-14T16:15:19.502705+0000","last_peered":"2023-07-14T16:15:19.502705+0000","last_clean":"2023-07-14T16:15:19.502705+0000","last_became_active":"2023-07-14T16:13:59.303404+0000","last_became_peered":"2023-07-14T16:13:59.303404+0000","last_unstale":"2023-07-14T16:15:19.502705+0000","last_undegraded":"2023-07-14T16:15:19.502705+0000","last_fullsized":"2023-07-14T16:15:19.502705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:26.218664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.490011+0000","last_change":"2023-07-14T16:13:59.296433+0000","last_active":"2023-07-14T16:15:19.490011+0000","last_peered":"2023-07-14T16:15:19.490011+0000","last_clean":"2023-07-14T16:15:19.490011+0000","last_became_active":"2023-07-14T16:13:59.295604+0000","last_became_peered":"2023-07-14T16:13:59.295604+0000","last_unstale":"2023-07-14T16:15:19.490011+0000","last_undegraded":"2023-07-14T16:15:19.490011+0000","last_fullsized":"2023-07-14T16:15:19.490011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:17:57.269484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.493883+0000","last_change":"2023-07-14T16:13:59.294198+0000","last_active":"2023-07-14T16:15:19.493883+0000","last_peered":"2023-07-14T16:15:19.493883+0000","last_clean":"2023-07-14T16:15:19.493883+0000","last_became_active":"2023-07-14T16:13:59.293891+0000","last_became_peered":"2023-07-14T16:13:59.293891+0000","last_unstale":"2023-07-14T16:15:19.493883+0000","last_undegraded":"2023-07-14T16:15:19.493883+0000","last_fullsized":"2023-07-14T16:15:19.493883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:41.192604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503095+0000","last_change":"2023-07-14T16:13:59.298724+0000","last_active":"2023-07-14T16:15:19.503095+0000","last_peered":"2023-07-14T16:15:19.503095+0000","last_clean":"2023-07-14T16:15:19.503095+0000","last_became_active":"2023-07-14T16:13:59.298318+0000","last_became_peered":"2023-07-14T16:13:59.298318+0000","last_unstale":"2023-07-14T16:15:19.503095+0000","last_undegraded":"2023-07-14T16:15:19.503095+0000","last_fullsized":"2023-07-14T16:15:19.503095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:30:13.650949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503946+0000","last_change":"2023-07-14T16:13:59.294652+0000","last_active":"2023-07-14T16:15:19.503946+0000","last_peered":"2023-07-14T16:15:19.503946+0000","last_clean":"2023-07-14T16:15:19.503946+0000","last_became_active":"2023-07-14T16:13:59.294278+0000","last_became_peered":"2023-07-14T16:13:59.294278+0000","last_unstale":"2023-07-14T16:15:19.503946+0000","last_undegraded":"2023-07-14T16:15:19.503946+0000","last_fullsized":"2023-07-14T16:15:19.503946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:45:58.125276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502135+0000","last_change":"2023-07-14T16:13:59.299228+0000","last_active":"2023-07-14T16:15:19.502135+0000","last_peered":"2023-07-14T16:15:19.502135+0000","last_clean":"2023-07-14T16:15:19.502135+0000","last_became_active":"2023-07-14T16:13:59.297982+0000","last_became_peered":"2023-07-14T16:13:59.297982+0000","last_unstale":"2023-07-14T16:15:19.502135+0000","last_undegraded":"2023-07-14T16:15:19.502135+0000","last_fullsized":"2023-07-14T16:15:19.502135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:24.432565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508368+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:15:19.508368+0000","last_peered":"2023-07-14T16:15:19.508368+0000","last_clean":"2023-07-14T16:15:19.508368+0000","last_became_active":"2023-07-14T16:13:59.299181+0000","last_became_peered":"2023-07-14T16:13:59.299181+0000","last_unstale":"2023-07-14T16:15:19.508368+0000","last_undegraded":"2023-07-14T16:15:19.508368+0000","last_fullsized":"2023-07-14T16:15:19.508368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:42.604278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.488076+0000","last_change":"2023-07-14T16:13:59.303282+0000","last_active":"2023-07-14T16:15:19.488076+0000","last_peered":"2023-07-14T16:15:19.488076+0000","last_clean":"2023-07-14T16:15:19.488076+0000","last_became_active":"2023-07-14T16:13:59.302922+0000","last_became_peered":"2023-07-14T16:13:59.302922+0000","last_unstale":"2023-07-14T16:15:19.488076+0000","last_undegraded":"2023-07-14T16:15:19.488076+0000","last_fullsized":"2023-07-14T16:15:19.488076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.750328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499836+0000","last_change":"2023-07-14T16:13:59.296389+0000","last_active":"2023-07-14T16:15:19.499836+0000","last_peered":"2023-07-14T16:15:19.499836+0000","last_clean":"2023-07-14T16:15:19.499836+0000","last_became_active":"2023-07-14T16:13:59.296080+0000","last_became_peered":"2023-07-14T16:13:59.296080+0000","last_unstale":"2023-07-14T16:15:19.499836+0000","last_undegraded":"2023-07-14T16:15:19.499836+0000","last_fullsized":"2023-07-14T16:15:19.499836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:02.060808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500459+0000","last_change":"2023-07-14T16:13:59.297774+0000","last_active":"2023-07-14T16:15:19.500459+0000","last_peered":"2023-07-14T16:15:19.500459+0000","last_clean":"2023-07-14T16:15:19.500459+0000","last_became_active":"2023-07-14T16:13:59.297463+0000","last_became_peered":"2023-07-14T16:13:59.297463+0000","last_unstale":"2023-07-14T16:15:19.500459+0000","last_undegraded":"2023-07-14T16:15:19.500459+0000","last_fullsized":"2023-07-14T16:15:19.500459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:49.304022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508679+0000","last_change":"2023-07-14T16:13:59.306795+0000","last_active":"2023-07-14T16:15:19.508679+0000","last_peered":"2023-07-14T16:15:19.508679+0000","last_clean":"2023-07-14T16:15:19.508679+0000","last_became_active":"2023-07-14T16:13:59.306457+0000","last_became_peered":"2023-07-14T16:13:59.306457+0000","last_unstale":"2023-07-14T16:15:19.508679+0000","last_undegraded":"2023-07-14T16:15:19.508679+0000","last_fullsized":"2023-07-14T16:15:19.508679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:47.873750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504322+0000","last_change":"2023-07-14T16:13:59.305338+0000","last_active":"2023-07-14T16:15:19.504322+0000","last_peered":"2023-07-14T16:15:19.504322+0000","last_clean":"2023-07-14T16:15:19.504322+0000","last_became_active":"2023-07-14T16:13:59.305047+0000","last_became_peered":"2023-07-14T16:13:59.305047+0000","last_unstale":"2023-07-14T16:15:19.504322+0000","last_undegraded":"2023-07-14T16:15:19.504322+0000","last_fullsized":"2023-07-14T16:15:19.504322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:07:02.694956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503214+0000","last_change":"2023-07-14T16:13:59.299005+0000","last_active":"2023-07-14T16:15:19.503214+0000","last_peered":"2023-07-14T16:15:19.503214+0000","last_clean":"2023-07-14T16:15:19.503214+0000","last_became_active":"2023-07-14T16:13:59.298705+0000","last_became_peered":"2023-07-14T16:13:59.298705+0000","last_unstale":"2023-07-14T16:15:19.503214+0000","last_undegraded":"2023-07-14T16:15:19.503214+0000","last_fullsized":"2023-07-14T16:15:19.503214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:59:41.986408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247144,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67642,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":621,"num_read_kb":377,"num_write":162,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":193787,"data_compressed":4809,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":459298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97699999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80000000000000004,"15min":0.78800000000000003},"min":{"1min":0.61799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0069999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.135,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45900000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.84599999999999997,"15min":0.83099999999999996},"min":{"1min":0.70299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1919999999999999,"5min":1.385,"15min":1.385},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97499999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70299999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96999999999999997,"5min":1.361,"15min":1.361},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.165,"5min":1.264,"15min":1.264},"last":0.89300000000000002}]},{"osd":6,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.91700000000000004,"15min":0.90300000000000002},"min":{"1min":0.78300000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1359999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.89000000000000001,"15min":0.89600000000000002},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0629999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.88100000000000001}]},{"osd":8,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.04},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0600000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.94099999999999995,"15min":0.95399999999999996},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5880000000000001,"5min":1.992,"15min":1.992},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.96499999999999997,"15min":0.94899999999999995},"min":{"1min":0.79000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.397,"5min":1.653,"15min":1.653},"last":1.0189999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.92200000000000004,"15min":0.93000000000000005},"min":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.099,"5min":2.097,"15min":2.097},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.95099999999999996,"15min":0.96299999999999997},"min":{"1min":0.68500000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.081,"5min":2.024,"15min":2.024},"last":0.90600000000000003}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1093632,"data_stored":1109743,"data_compressed":37379,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.165,"5min":7.165,"15min":7.165},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.125,"5min":7.125,"15min":7.125},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79300000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73999999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.63100000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94599999999999995,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.501},"max":{"1min":0.92400000000000004,"5min":1.173,"15min":1.173},"last":0.745}]},{"osd":3,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.81999999999999995,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.80000000000000004,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.879,"5min":4.4429999999999996,"15min":4.4429999999999996},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.74299999999999999,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":1.613,"15min":1.613},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.92600000000000005,"15min":0.91600000000000004},"min":{"1min":0.68500000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.016,"5min":4.9749999999999996,"15min":4.9749999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89300000000000002,"15min":0.91400000000000003},"min":{"1min":0.65800000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0369999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.86499999999999999}]},{"osd":6,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":1.01,"15min":0.95099999999999996},"min":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.05,"5min":6.609,"15min":6.609},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96599999999999997,"15min":0.93200000000000005},"min":{"1min":0.67000000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1499999999999999,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.61699999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.97499999999999998,"15min":0.98299999999999998},"min":{"1min":0.754,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.103,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.98099999999999998,"15min":1.0069999999999999},"min":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0740000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.85299999999999998}]},{"osd":9,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.72699999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1830000000000001,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.121,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1579999999999999,"5min":7.3579999999999997,"15min":7.3579999999999997},"last":0.79700000000000004}]},{"osd":11,"last update":"Fri Jul 14 16:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.046,"15min":1.05},"min":{"1min":0.84799999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.1890000000000001,"5min":2.653,"15min":2.653},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0089999999999999,"15min":1.002},"min":{"1min":0.92200000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90900000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":459512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82599999999999996,"15min":0.86199999999999999},"min":{"1min":0.55200000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1259999999999999,"5min":1.129,"15min":1.129},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82899999999999996,"15min":0.86499999999999999},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.2809999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.878},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.82699999999999996,"15min":0.872},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63600000000000001,"15min":0.66100000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.376},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78300000000000003,"15min":0.83199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.629}]},{"osd":4,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.88600000000000001,"15min":0.93400000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.82899999999999996,"15min":0.85299999999999998},"min":{"1min":0.55000000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84499999999999997,"15min":0.877},"min":{"1min":0.502,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3220000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.94099999999999995,"15min":1.0029999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.448,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.93999999999999995,"15min":1.014},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1799999999999999,"5min":1.288,"15min":1.288},"last":0.59399999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94299999999999995,"15min":0.98299999999999998},"min":{"1min":0.58999999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.879,"5min":1.024,"15min":1.0720000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3979999999999999,"5min":1.575,"15min":1.575},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0229999999999999,"15min":1.0840000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.996,"15min":1.0449999999999999},"min":{"1min":0.69299999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.85799999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.96999999999999997,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.034,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.95999999999999996},"min":{"1min":0.66500000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.66500000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":459410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66400000000000003,"15min":0.68899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88300000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72899999999999998,"15min":0.69799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":6.3410000000000002,"15min":6.3410000000000002},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98699999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.48799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.1100000000000001,"5min":1.216,"15min":1.216},"last":1.052}]},{"osd":2,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.081,"15min":1.081},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99099999999999999,"5min":1.175,"15min":1.175},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.78700000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92600000000000005,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.54200000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.61499999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.0289999999999999,"5min":1.218,"15min":1.218},"last":0.98599999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69899999999999995,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73599999999999999,"15min":0.74399999999999999},"min":{"1min":0.46500000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.05,"5min":1.21,"15min":1.21},"last":0.81799999999999995}]},{"osd":6,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84199999999999997,"15min":0.85799999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1839999999999999,"5min":1.29,"15min":1.29},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.88,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.63500000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.256,"15min":1.256},"last":0.88200000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.93799999999999994,"15min":0.89300000000000002},"min":{"1min":0.63200000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.222,"5min":6.3209999999999997,"15min":6.3209999999999997},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89000000000000001,"15min":0.89800000000000002},"min":{"1min":0.68400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.1919999999999999,"5min":1.373,"15min":1.373},"last":1.0840000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.93899999999999995,"15min":0.91700000000000004},"min":{"1min":0.76700000000000002,"5min":0.625,"15min":0.625},"max":{"1min":3.4470000000000001,"5min":3.4470000000000001,"15min":3.4470000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90600000000000003,"15min":0.91500000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.2130000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.89800000000000002,"15min":0.86399999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1870000000000001,"5min":1.258,"15min":1.258},"last":1.006},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.90800000000000003,"15min":0.88300000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.2050000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.93999999999999995}]},{"osd":11,"last update":"Fri Jul 14 16:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":1.03}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":1052410,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.181,"15min":1.181},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52600000000000002,"15min":0.51200000000000001},"min":{"1min":0.40000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79100000000000004,"5min":1.046,"15min":1.046},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.755}]},{"osd":2,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71999999999999997,"15min":0.71899999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.60299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.52100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.72599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95899999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.024,"15min":1.024},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90800000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.443}]},{"osd":5,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0760000000000001,"5min":1.246,"15min":1.246},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.64700000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.93000000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.495,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93400000000000005,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84999999999999998,"5min":1.137,"15min":1.137},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Jul 14 16:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.67400000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0369999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.69999999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.079,"5min":1.331,"15min":1.331},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Jul 14 16:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.88800000000000001,"15min":0.89400000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.054,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.628,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0189999999999999,"5min":1.262,"15min":1.262},"last":0.748}]},{"osd":10,"last update":"Fri Jul 14 16:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.71699999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1080000000000001,"5min":1.238,"15min":1.238},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.89500000000000002,"15min":0.89900000000000002},"min":{"1min":0.70199999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0920000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.71199999999999997}]},{"osd":11,"last update":"Fri Jul 14 16:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.96599999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":458907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.68999999999999995},"min":{"1min":0.5,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.67800000000000005},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98599999999999999,"5min":1.111,"15min":1.111},"last":1}]},{"osd":2,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0009999999999999,"5min":1.121,"15min":1.121},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94799999999999995,"5min":1.105,"15min":1.105},"last":0.72499999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.79100000000000004,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.014}]},{"osd":5,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82099999999999995,"15min":0.81000000000000005},"min":{"1min":0.53800000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0720000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Jul 14 16:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.83399999999999996,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1739999999999999,"5min":1.28,"15min":1.28},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.85699999999999998,"15min":0.80800000000000005},"min":{"1min":0.65100000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.238,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.90800000000000003}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.93000000000000005,"15min":0.93700000000000006},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.2929999999999999,"5min":1.353,"15min":1.353},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.873,"15min":0.86499999999999999},"min":{"1min":0.72299999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2470000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.80700000000000005}]},{"osd":9,"last update":"Fri Jul 14 16:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.88100000000000001,"15min":0.92000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.141,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.93600000000000005,"15min":0.98299999999999998},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.129,"5min":1.331,"15min":1.331},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.97799999999999998,"15min":0.99099999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.179,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.0609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":516474,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.55300000000000005},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.34899999999999998},"max":{"1min":0.85699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Jul 14 16:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.71799999999999997,"15min":0.61899999999999999},"min":{"1min":0.52200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.0860000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.71299999999999997,"15min":0.63700000000000001},"min":{"1min":0.50700000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.69699999999999995,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57599999999999996,"15min":0.51800000000000002},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93500000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.47199999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68300000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Jul 14 16:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71099999999999997,"15min":0.66400000000000003},"min":{"1min":0.61299999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.71299999999999997,"15min":0.66500000000000004},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.754}]},{"osd":6,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76700000000000002,"15min":0.73099999999999998},"min":{"1min":0.61299999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0189999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.64100000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.77300000000000002,"15min":0.70699999999999996},"min":{"1min":0.68000000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1919999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.018},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.754,"15min":0.68200000000000005},"min":{"1min":0.48799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.1559999999999999,"5min":1.244,"15min":1.244},"last":1.0369999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2050000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.81200000000000006,"15min":0.77200000000000002},"min":{"1min":0.75,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.103}]},{"osd":9,"last update":"Fri Jul 14 16:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.871,"15min":0.875},"min":{"1min":0.68999999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.091,"5min":1.204,"15min":1.204},"last":1},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91700000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.262,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.083}]},{"osd":10,"last update":"Fri Jul 14 16:14:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.91900000000000004,"15min":0.90900000000000003},"min":{"1min":0.72599999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2609999999999999,"5min":1.262,"15min":1.262},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93200000000000005,"15min":0.93000000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.05}]},{"osd":11,"last update":"Fri Jul 14 16:15:02 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.141},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.121}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":460172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63800000000000001,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jul 14 16:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73799999999999999,"15min":0.63900000000000001},"min":{"1min":0.60099999999999998,"5min":0.38200000000000001,"15min":0.34399999999999997},"max":{"1min":0.96399999999999997,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.64900000000000002},"min":{"1min":0.53300000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.03,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.3599999999999999,"5min":3.3599999999999999,"15min":3.3599999999999999},"last":0.504}]},{"osd":4,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2030000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.68400000000000005,"15min":0.64800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":1.357,"15min":1.357},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.74199999999999999,"15min":0.69399999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70999999999999996,"15min":0.67800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.5670000000000002,"5min":3.5670000000000002,"15min":3.5670000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9630000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.748,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83799999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.92100000000000004},"min":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.85899999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.89700000000000002}]},{"osd":9,"last update":"Fri Jul 14 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.878,"15min":0.88},"min":{"1min":0.5,"5min":0.495,"15min":0.495},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.91500000000000004,"15min":0.94399999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.61499999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88600000000000001,"15min":0.91300000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2370000000000001,"5min":1.419,"15min":1.419},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.92600000000000005,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.93799999999999994,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.88300000000000001}]},{"osd":11,"last update":"Fri Jul 14 16:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89000000000000001,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.171,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.5350000000000001,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":0.82799999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":518774,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.376,"15min":0.34699999999999998},"max":{"1min":4.4400000000000004,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.47299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65100000000000002,"15min":0.66400000000000003},"min":{"1min":0.51400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":0.90500000000000003,"15min":0.90800000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.70599999999999996,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.41799999999999998},"max":{"1min":1.0569999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.038}]},{"osd":3,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.39200000000000002,"15min":0.39100000000000001},"max":{"1min":2.1309999999999998,"5min":3.274,"15min":3.274},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67200000000000004,"15min":0.629},"min":{"1min":0.52300000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Jul 14 16:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.64700000000000002,"15min":0.58799999999999997},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59899999999999998,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90100000000000002,"5min":1.022,"15min":1.022},"last":0.48999999999999999}]},{"osd":6,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.76600000000000001,"15min":0.71999999999999997},"min":{"1min":0.57699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.7240000000000002,"5min":3.7240000000000002,"15min":3.7240000000000002},"last":0.75},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.80600000000000005,"15min":0.75800000000000001},"min":{"1min":0.53200000000000003,"5min":0.439,"15min":0.439},"max":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.67000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.022,"5min":1.117,"15min":1.117},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99099999999999999,"5min":1.016,"15min":1.016},"last":0.61199999999999999}]},{"osd":9,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.86799999999999999,"15min":0.83799999999999997},"min":{"1min":0.70699999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":4.5579999999999998,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.91500000000000004,"15min":0.90000000000000002},"min":{"1min":0.67400000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.5060000000000002,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":0.96699999999999997}]},{"osd":10,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.84699999999999998,"15min":0.83099999999999996},"min":{"1min":0.66700000000000004,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1160000000000001,"5min":1.246,"15min":1.246},"last":0.997},{"interface":"front","average":{"1min":1,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.71999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0640000000000001}]},{"osd":11,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.86499999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":458943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.49399999999999999,"15min":0.42199999999999999},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.495,"15min":0.442},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.27900000000000003},"max":{"1min":0.96399999999999997,"5min":2.323,"15min":2.323},"last":0.497}]},{"osd":1,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1419999999999999,"5min":2.343,"15min":2.343},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70199999999999996,"15min":0.63800000000000001},"min":{"1min":0.50700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.9430000000000001,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.73799999999999999,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.72299999999999998},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2809999999999999,"5min":3.448,"15min":3.448},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.755,"15min":0.67400000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.048,"5min":3.359,"15min":3.359},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.749,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.84199999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75600000000000001,"15min":0.71999999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3120000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79000000000000004,"15min":0.74399999999999999},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.75,"15min":0.68500000000000005},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.627},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.76900000000000002,"15min":0.72899999999999998},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.2050000000000001,"5min":3.2050000000000001,"15min":3.2050000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.84599999999999997,"15min":0.79800000000000004},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.82999999999999996,"15min":0.78000000000000003},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.876,"15min":0.88500000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1259999999999999,"5min":1.546,"15min":1.546},"last":0.62},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91800000000000004,"15min":0.92100000000000004},"min":{"1min":0.61899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.163,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":0.98699999999999999,"15min":0.97899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.6299999999999999,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.85899999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.61399999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.004,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0860000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.79200000000000004}]},{"osd":11,"last update":"Fri Jul 14 16:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":0.75}]}]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":460320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.95599999999999996,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48499999999999999,"15min":0.437},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.46500000000000002},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79000000000000004,"5min":1.042,"15min":1.042},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.65100000000000002},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.1319999999999999,"5min":3.698,"15min":3.698},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72899999999999998,"15min":0.68700000000000006},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93400000000000005,"5min":1.153,"15min":1.153},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.014,"5min":1.03,"15min":1.03},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.012,"5min":1.506,"15min":1.506},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73899999999999999,"15min":0.72199999999999998},"min":{"1min":0.621,"5min":0.435,"15min":0.435},"max":{"1min":0.95099999999999996,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.83399999999999996}]},{"osd":6,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57099999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.147,"5min":1.321,"15min":1.321},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Jul 14 16:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.53800000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.012,"5min":1.038,"15min":1.038},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.752,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.107,"5min":1.129,"15min":1.129},"last":0.755}]},{"osd":8,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85999999999999999,"15min":0.84299999999999997},"min":{"1min":0.753,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79100000000000004,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.63800000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90700000000000003,"15min":0.91900000000000004},"min":{"1min":0.72799999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.24,"5min":1.631,"15min":1.631},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.67500000000000004}]},{"osd":10,"last update":"Fri Jul 14 16:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.878,"15min":0.88700000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.81899999999999995}]},{"osd":11,"last update":"Fri Jul 14 16:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.874},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.84999999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":1050781,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.39900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78600000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.68200000000000005,"15min":0.621},"min":{"1min":0.58599999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0189999999999999,"5min":1.099,"15min":1.099},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.69099999999999995,"15min":0.63600000000000001},"min":{"1min":0.55200000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51900000000000002,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86799999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.64400000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83199999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77700000000000002,"5min":7.056,"15min":7.056},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.752,"15min":0.70399999999999996},"min":{"1min":0.621,"5min":0.503,"15min":0.503},"max":{"1min":0.96499999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.76400000000000001,"15min":0.71399999999999997},"min":{"1min":0.70299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.0669999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73599999999999999,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":4.3739999999999997,"5min":7.109,"15min":7.109},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Jul 14 16:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.58099999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.82199999999999995,"15min":0.78800000000000003},"min":{"1min":0.65700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.94199999999999995}]},{"osd":8,"last update":"Fri Jul 14 16:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.1479999999999999,"5min":1.286,"15min":1.286},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97899999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Fri Jul 14 16:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89100000000000001,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.88400000000000001,"15min":0.872},"min":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.004}]},{"osd":10,"last update":"Fri Jul 14 16:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.86299999999999999,"15min":0.85799999999999998},"min":{"1min":0.67100000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0409999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.877},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.92300000000000004,"15min":0.91700000000000004},"min":{"1min":0.70499999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.105,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":57780,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60080,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65159,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T16:15:45.327 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T16:15:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 -- 172.21.15.62:0/747681617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c101680 msgr2=0x7fad9c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 --2- 172.21.15.62:0/747681617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c101680 0x7fad9c107f40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad9400b0a0 tx=0x7fad9400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 -- 172.21.15.62:0/747681617 shutdown_connections 2023-07-14T16:15:45.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 --2- 172.21.15.62:0/747681617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c101680 0x7fad9c107f40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 --2- 172.21.15.62:0/747681617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad9c0ffb00 0x7fad9c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 --2- 172.21.15.62:0/747681617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c100cf0 0x7fad9c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 -- 172.21.15.62:0/747681617 >> 172.21.15.62:0/747681617 conn(0x7fad9c0fb0e0 msgr2=0x7fad9c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:45.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 -- 172.21.15.62:0/747681617 shutdown_connections 2023-07-14T16:15:45.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.585+0000 7fada13ff700 1 -- 172.21.15.62:0/747681617 wait complete. 2023-07-14T16:15:45.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.586+0000 7fada13ff700 1 Processor -- start 2023-07-14T16:15:45.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fada13ff700 1 -- start start 2023-07-14T16:15:45.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fada13ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad9c0ffb00 0x7fad9c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fada13ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c100cf0 0x7fad9c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fada13ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c101680 0x7fad9c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fada13ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0735e0 con 0x7fad9c100cf0 2023-07-14T16:15:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fada13ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c073750 con 0x7fad9c0ffb00 2023-07-14T16:15:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fada13ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c19f920 con 0x7fad9c101680 2023-07-14T16:15:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fad93fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c100cf0 0x7fad9c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:45.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fad93fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c100cf0 0x7fad9c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35042/0 (socket says 172.21.15.62:35042) 2023-07-14T16:15:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fad93fff700 1 -- 172.21.15.62:0/1291448980 learned_addr learned my addr 172.21.15.62:0/1291448980 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.587+0000 7fad9affd700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad9c0ffb00 0x7fad9c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad93fff700 1 -- 172.21.15.62:0/1291448980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c101680 msgr2=0x7fad9c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad9b7fe700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c101680 0x7fad9c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad93fff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c101680 0x7fad9c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad93fff700 1 -- 172.21.15.62:0/1291448980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad9c0ffb00 msgr2=0x7fad9c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad93fff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad9c0ffb00 0x7fad9c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad93fff700 1 -- 172.21.15.62:0/1291448980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94009d00 con 0x7fad9c100cf0 2023-07-14T16:15:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad9affd700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad9c0ffb00 0x7fad9c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad9b7fe700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c101680 0x7fad9c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad93fff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c100cf0 0x7fad9c072900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e9a0 tx=0x7fad8400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:45.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad98ff9700 1 -- 172.21.15.62:0/1291448980 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84009790 con 0x7fad9c100cf0 2023-07-14T16:15:45.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad98ff9700 1 -- 172.21.15.62:0/1291448980 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad8400cbd0 con 0x7fad9c100cf0 2023-07-14T16:15:45.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fad98ff9700 1 -- 172.21.15.62:0/1291448980 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84018680 con 0x7fad9c100cf0 2023-07-14T16:15:45.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c19fc00 con 0x7fad9c100cf0 2023-07-14T16:15:45.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.588+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c1a00c0 con 0x7fad9c100cf0 2023-07-14T16:15:45.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.589+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c107040 con 0x7fad9c100cf0 2023-07-14T16:15:45.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.591+0000 7fad98ff9700 1 -- 172.21.15.62:0/1291448980 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fad8400cd40 con 0x7fad9c100cf0 2023-07-14T16:15:45.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.593+0000 7fad98ff9700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad880a11a0 0x7fad880a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:45.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.593+0000 7fad98ff9700 1 -- 172.21.15.62:0/1291448980 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fad84014070 con 0x7fad9c100cf0 2023-07-14T16:15:45.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.593+0000 7fad9affd700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad880a11a0 0x7fad880a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:45.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.594+0000 7fad9affd700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad880a11a0 0x7fad880a3650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005950 tx=0x7fad8c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:45.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.596+0000 7fad98ff9700 1 -- 172.21.15.62:0/1291448980 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fad84051790 con 0x7fad9c100cf0 2023-07-14T16:15:45.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.801+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad9c060950 con 0x7fad880a11a0 2023-07-14T16:15:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.812+0000 7fad98ff9700 1 -- 172.21.15.62:0/1291448980 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343728 (secure 0 0 0) 0x7fad9c060950 con 0x7fad880a11a0 2023-07-14T16:15:45.795 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.816+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad880a11a0 msgr2=0x7fad880a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.816+0000 7fada13ff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad880a11a0 0x7fad880a3650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005950 tx=0x7fad8c0058e0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.816+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c100cf0 msgr2=0x7fad9c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.816+0000 7fada13ff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c100cf0 0x7fad9c072900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e9a0 tx=0x7fad8400ed60 comp rx=0 tx=0).stop 2023-07-14T16:15:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.817+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 shutdown_connections 2023-07-14T16:15:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.817+0000 7fada13ff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad9c101680 0x7fad9c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.817+0000 7fada13ff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad880a11a0 0x7fad880a3650 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.817+0000 7fada13ff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad9c0ffb00 0x7fad9c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.817+0000 7fada13ff700 1 --2- 172.21.15.62:0/1291448980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad9c100cf0 0x7fad9c072900 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:45.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.817+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 >> 172.21.15.62:0/1291448980 conn(0x7fad9c0fb0e0 msgr2=0x7fad9c104180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:45.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.817+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 shutdown_connections 2023-07-14T16:15:45.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:45.817+0000 7fada13ff700 1 -- 172.21.15.62:0/1291448980 wait complete. 2023-07-14T16:15:45.799 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-07-14T16:15:45.813 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2023-07-14T16:15:45.495736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658256,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":757,"num_read_kb":491,"num_write":345,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335064,"kb_used_data":11640,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606760,"statfs":{"total":1151940427776,"available":1151597322240,"internally_reserved":0,"allocated":11919360,"data_stored":7465243,"data_compressed":112137,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5817,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":38,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002875"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509907+0000","last_change":"2023-07-14T16:13:59.301892+0000","last_active":"2023-07-14T16:15:19.509907+0000","last_peered":"2023-07-14T16:15:19.509907+0000","last_clean":"2023-07-14T16:15:19.509907+0000","last_became_active":"2023-07-14T16:13:59.301379+0000","last_became_peered":"2023-07-14T16:13:59.301379+0000","last_unstale":"2023-07-14T16:15:19.509907+0000","last_undegraded":"2023-07-14T16:15:19.509907+0000","last_fullsized":"2023-07-14T16:15:19.509907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:49:12.243022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502665+0000","last_change":"2023-07-14T16:13:59.309322+0000","last_active":"2023-07-14T16:15:19.502665+0000","last_peered":"2023-07-14T16:15:19.502665+0000","last_clean":"2023-07-14T16:15:19.502665+0000","last_became_active":"2023-07-14T16:13:59.309035+0000","last_became_peered":"2023-07-14T16:13:59.309035+0000","last_unstale":"2023-07-14T16:15:19.502665+0000","last_undegraded":"2023-07-14T16:15:19.502665+0000","last_fullsized":"2023-07-14T16:15:19.502665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:46.013552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509601+0000","last_change":"2023-07-14T16:13:59.304371+0000","last_active":"2023-07-14T16:15:19.509601+0000","last_peered":"2023-07-14T16:15:19.509601+0000","last_clean":"2023-07-14T16:15:19.509601+0000","last_became_active":"2023-07-14T16:13:59.303965+0000","last_became_peered":"2023-07-14T16:13:59.303965+0000","last_unstale":"2023-07-14T16:15:19.509601+0000","last_undegraded":"2023-07-14T16:15:19.509601+0000","last_fullsized":"2023-07-14T16:15:19.509601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:19.938384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.494491+0000","last_change":"2023-07-14T16:14:20.670094+0000","last_active":"2023-07-14T16:15:19.494491+0000","last_peered":"2023-07-14T16:15:19.494491+0000","last_clean":"2023-07-14T16:15:19.494491+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:15:19.494491+0000","last_undegraded":"2023-07-14T16:15:19.494491+0000","last_fullsized":"2023-07-14T16:15:19.494491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:32.039209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499662+0000","last_change":"2023-07-14T16:13:59.298094+0000","last_active":"2023-07-14T16:15:19.499662+0000","last_peered":"2023-07-14T16:15:19.499662+0000","last_clean":"2023-07-14T16:15:19.499662+0000","last_became_active":"2023-07-14T16:13:59.297764+0000","last_became_peered":"2023-07-14T16:13:59.297764+0000","last_unstale":"2023-07-14T16:15:19.499662+0000","last_undegraded":"2023-07-14T16:15:19.499662+0000","last_fullsized":"2023-07-14T16:15:19.499662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:40:48.501912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503393+0000","last_change":"2023-07-14T16:13:59.299735+0000","last_active":"2023-07-14T16:15:19.503393+0000","last_peered":"2023-07-14T16:15:19.503393+0000","last_clean":"2023-07-14T16:15:19.503393+0000","last_became_active":"2023-07-14T16:13:59.299413+0000","last_became_peered":"2023-07-14T16:13:59.299413+0000","last_unstale":"2023-07-14T16:15:19.503393+0000","last_undegraded":"2023-07-14T16:15:19.503393+0000","last_fullsized":"2023-07-14T16:15:19.503393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:09:10.174873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.493151+0000","last_change":"2023-07-14T16:13:59.299682+0000","last_active":"2023-07-14T16:15:19.493151+0000","last_peered":"2023-07-14T16:15:19.493151+0000","last_clean":"2023-07-14T16:15:19.493151+0000","last_became_active":"2023-07-14T16:13:59.299327+0000","last_became_peered":"2023-07-14T16:13:59.299327+0000","last_unstale":"2023-07-14T16:15:19.493151+0000","last_undegraded":"2023-07-14T16:15:19.493151+0000","last_fullsized":"2023-07-14T16:15:19.493151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:30.735405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502074+0000","last_change":"2023-07-14T16:13:59.311818+0000","last_active":"2023-07-14T16:15:19.502074+0000","last_peered":"2023-07-14T16:15:19.502074+0000","last_clean":"2023-07-14T16:15:19.502074+0000","last_became_active":"2023-07-14T16:13:59.311478+0000","last_became_peered":"2023-07-14T16:13:59.311478+0000","last_unstale":"2023-07-14T16:15:19.502074+0000","last_undegraded":"2023-07-14T16:15:19.502074+0000","last_fullsized":"2023-07-14T16:15:19.502074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:51.107741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507313+0000","last_change":"2023-07-14T16:13:59.309665+0000","last_active":"2023-07-14T16:15:19.507313+0000","last_peered":"2023-07-14T16:15:19.507313+0000","last_clean":"2023-07-14T16:15:19.507313+0000","last_became_active":"2023-07-14T16:13:59.309361+0000","last_became_peered":"2023-07-14T16:13:59.309361+0000","last_unstale":"2023-07-14T16:15:19.507313+0000","last_undegraded":"2023-07-14T16:15:19.507313+0000","last_fullsized":"2023-07-14T16:15:19.507313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:14.297747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508327+0000","last_change":"2023-07-14T16:13:59.296397+0000","last_active":"2023-07-14T16:15:19.508327+0000","last_peered":"2023-07-14T16:15:19.508327+0000","last_clean":"2023-07-14T16:15:19.508327+0000","last_became_active":"2023-07-14T16:13:59.295965+0000","last_became_peered":"2023-07-14T16:13:59.295965+0000","last_unstale":"2023-07-14T16:15:19.508327+0000","last_undegraded":"2023-07-14T16:15:19.508327+0000","last_fullsized":"2023-07-14T16:15:19.508327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:51.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.511367+0000","last_change":"2023-07-14T16:13:59.302892+0000","last_active":"2023-07-14T16:15:19.511367+0000","last_peered":"2023-07-14T16:15:19.511367+0000","last_clean":"2023-07-14T16:15:19.511367+0000","last_became_active":"2023-07-14T16:13:59.302295+0000","last_became_peered":"2023-07-14T16:13:59.302295+0000","last_unstale":"2023-07-14T16:15:19.511367+0000","last_undegraded":"2023-07-14T16:15:19.511367+0000","last_fullsized":"2023-07-14T16:15:19.511367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:11.009590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499348+0000","last_change":"2023-07-14T16:13:59.301913+0000","last_active":"2023-07-14T16:15:19.499348+0000","last_peered":"2023-07-14T16:15:19.499348+0000","last_clean":"2023-07-14T16:15:19.499348+0000","last_became_active":"2023-07-14T16:13:59.301590+0000","last_became_peered":"2023-07-14T16:13:59.301590+0000","last_unstale":"2023-07-14T16:15:19.499348+0000","last_undegraded":"2023-07-14T16:15:19.499348+0000","last_fullsized":"2023-07-14T16:15:19.499348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:09:55.206983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507029+0000","last_change":"2023-07-14T16:13:59.302006+0000","last_active":"2023-07-14T16:15:19.507029+0000","last_peered":"2023-07-14T16:15:19.507029+0000","last_clean":"2023-07-14T16:15:19.507029+0000","last_became_active":"2023-07-14T16:13:59.301752+0000","last_became_peered":"2023-07-14T16:13:59.301752+0000","last_unstale":"2023-07-14T16:15:19.507029+0000","last_undegraded":"2023-07-14T16:15:19.507029+0000","last_fullsized":"2023-07-14T16:15:19.507029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:08.923165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503265+0000","last_change":"2023-07-14T16:13:59.309318+0000","last_active":"2023-07-14T16:15:19.503265+0000","last_peered":"2023-07-14T16:15:19.503265+0000","last_clean":"2023-07-14T16:15:19.503265+0000","last_became_active":"2023-07-14T16:13:59.308931+0000","last_became_peered":"2023-07-14T16:13:59.308931+0000","last_unstale":"2023-07-14T16:15:19.503265+0000","last_undegraded":"2023-07-14T16:15:19.503265+0000","last_fullsized":"2023-07-14T16:15:19.503265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:21:02.747919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498473+0000","last_change":"2023-07-14T16:14:20.670208+0000","last_active":"2023-07-14T16:15:19.498473+0000","last_peered":"2023-07-14T16:15:19.498473+0000","last_clean":"2023-07-14T16:15:19.498473+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:15:19.498473+0000","last_undegraded":"2023-07-14T16:15:19.498473+0000","last_fullsized":"2023-07-14T16:15:19.498473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:53.265928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509331+0000","last_change":"2023-07-14T16:13:59.307986+0000","last_active":"2023-07-14T16:15:19.509331+0000","last_peered":"2023-07-14T16:15:19.509331+0000","last_clean":"2023-07-14T16:15:19.509331+0000","last_became_active":"2023-07-14T16:13:59.307701+0000","last_became_peered":"2023-07-14T16:13:59.307701+0000","last_unstale":"2023-07-14T16:15:19.509331+0000","last_undegraded":"2023-07-14T16:15:19.509331+0000","last_fullsized":"2023-07-14T16:15:19.509331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:56:03.104096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510111+0000","last_change":"2023-07-14T16:13:59.301685+0000","last_active":"2023-07-14T16:15:19.510111+0000","last_peered":"2023-07-14T16:15:19.510111+0000","last_clean":"2023-07-14T16:15:19.510111+0000","last_became_active":"2023-07-14T16:13:59.300905+0000","last_became_peered":"2023-07-14T16:13:59.300905+0000","last_unstale":"2023-07-14T16:15:19.510111+0000","last_undegraded":"2023-07-14T16:15:19.510111+0000","last_fullsized":"2023-07-14T16:15:19.510111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:48:24.721908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498700+0000","last_change":"2023-07-14T16:13:59.306027+0000","last_active":"2023-07-14T16:15:19.498700+0000","last_peered":"2023-07-14T16:15:19.498700+0000","last_clean":"2023-07-14T16:15:19.498700+0000","last_became_active":"2023-07-14T16:13:59.305718+0000","last_became_peered":"2023-07-14T16:13:59.305718+0000","last_unstale":"2023-07-14T16:15:19.498700+0000","last_undegraded":"2023-07-14T16:15:19.498700+0000","last_fullsized":"2023-07-14T16:15:19.498700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:20:47.951543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.494287+0000","last_change":"2023-07-14T16:13:59.294140+0000","last_active":"2023-07-14T16:15:19.494287+0000","last_peered":"2023-07-14T16:15:19.494287+0000","last_clean":"2023-07-14T16:15:19.494287+0000","last_became_active":"2023-07-14T16:13:59.293825+0000","last_became_peered":"2023-07-14T16:13:59.293825+0000","last_unstale":"2023-07-14T16:15:19.494287+0000","last_undegraded":"2023-07-14T16:15:19.494287+0000","last_fullsized":"2023-07-14T16:15:19.494287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:59.348563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665523+0000","last_change":"2023-07-14T16:13:58.277154+0000","last_active":"2023-07-14T16:14:20.665523+0000","last_peered":"2023-07-14T16:14:20.665523+0000","last_clean":"2023-07-14T16:14:20.665523+0000","last_became_active":"2023-07-14T16:13:58.276800+0000","last_became_peered":"2023-07-14T16:13:58.276800+0000","last_unstale":"2023-07-14T16:14:20.665523+0000","last_undegraded":"2023-07-14T16:14:20.665523+0000","last_fullsized":"2023-07-14T16:14:20.665523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:45.631137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.488178+0000","last_change":"2023-07-14T16:13:59.299819+0000","last_active":"2023-07-14T16:15:19.488178+0000","last_peered":"2023-07-14T16:15:19.488178+0000","last_clean":"2023-07-14T16:15:19.488178+0000","last_became_active":"2023-07-14T16:13:59.299294+0000","last_became_peered":"2023-07-14T16:13:59.299294+0000","last_unstale":"2023-07-14T16:15:19.488178+0000","last_undegraded":"2023-07-14T16:15:19.488178+0000","last_fullsized":"2023-07-14T16:15:19.488178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:43.461827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664040+0000","last_change":"2023-07-14T16:13:58.278954+0000","last_active":"2023-07-14T16:14:20.664040+0000","last_peered":"2023-07-14T16:14:20.664040+0000","last_clean":"2023-07-14T16:14:20.664040+0000","last_became_active":"2023-07-14T16:13:58.278454+0000","last_became_peered":"2023-07-14T16:13:58.278454+0000","last_unstale":"2023-07-14T16:14:20.664040+0000","last_undegraded":"2023-07-14T16:14:20.664040+0000","last_fullsized":"2023-07-14T16:14:20.664040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:33:34.015342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510065+0000","last_change":"2023-07-14T16:13:59.299877+0000","last_active":"2023-07-14T16:15:19.510065+0000","last_peered":"2023-07-14T16:15:19.510065+0000","last_clean":"2023-07-14T16:15:19.510065+0000","last_became_active":"2023-07-14T16:13:59.299557+0000","last_became_peered":"2023-07-14T16:13:59.299557+0000","last_unstale":"2023-07-14T16:15:19.510065+0000","last_undegraded":"2023-07-14T16:15:19.510065+0000","last_fullsized":"2023-07-14T16:15:19.510065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:45.331826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.677912+0000","last_change":"2023-07-14T16:14:20.677912+0000","last_active":"2023-07-14T16:14:20.677912+0000","last_peered":"2023-07-14T16:14:20.677912+0000","last_clean":"2023-07-14T16:14:20.677912+0000","last_became_active":"2023-07-14T16:14:20.668156+0000","last_became_peered":"2023-07-14T16:14:20.668156+0000","last_unstale":"2023-07-14T16:14:20.677912+0000","last_undegraded":"2023-07-14T16:14:20.677912+0000","last_fullsized":"2023-07-14T16:14:20.677912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:51:50.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509983+0000","last_change":"2023-07-14T16:13:59.307987+0000","last_active":"2023-07-14T16:15:19.509983+0000","last_peered":"2023-07-14T16:15:19.509983+0000","last_clean":"2023-07-14T16:15:19.509983+0000","last_became_active":"2023-07-14T16:13:59.307457+0000","last_became_peered":"2023-07-14T16:13:59.307457+0000","last_unstale":"2023-07-14T16:15:19.509983+0000","last_undegraded":"2023-07-14T16:15:19.509983+0000","last_fullsized":"2023-07-14T16:15:19.509983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:32:28.569384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663585+0000","last_change":"2023-07-14T16:13:58.282332+0000","last_active":"2023-07-14T16:14:20.663585+0000","last_peered":"2023-07-14T16:14:20.663585+0000","last_clean":"2023-07-14T16:14:20.663585+0000","last_became_active":"2023-07-14T16:13:58.278103+0000","last_became_peered":"2023-07-14T16:13:58.278103+0000","last_unstale":"2023-07-14T16:14:20.663585+0000","last_undegraded":"2023-07-14T16:14:20.663585+0000","last_fullsized":"2023-07-14T16:14:20.663585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:00.081710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509816+0000","last_change":"2023-07-14T16:13:59.301036+0000","last_active":"2023-07-14T16:15:19.509816+0000","last_peered":"2023-07-14T16:15:19.509816+0000","last_clean":"2023-07-14T16:15:19.509816+0000","last_became_active":"2023-07-14T16:13:59.300787+0000","last_became_peered":"2023-07-14T16:13:59.300787+0000","last_unstale":"2023-07-14T16:15:19.509816+0000","last_undegraded":"2023-07-14T16:15:19.509816+0000","last_fullsized":"2023-07-14T16:15:19.509816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:40:56.654793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":127,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:41.719917+0000","last_change":"2023-07-14T16:13:58.282392+0000","last_active":"2023-07-14T16:15:41.719917+0000","last_peered":"2023-07-14T16:15:41.719917+0000","last_clean":"2023-07-14T16:15:41.719917+0000","last_became_active":"2023-07-14T16:13:58.282030+0000","last_became_peered":"2023-07-14T16:13:58.282030+0000","last_unstale":"2023-07-14T16:15:41.719917+0000","last_undegraded":"2023-07-14T16:15:41.719917+0000","last_fullsized":"2023-07-14T16:15:41.719917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:17.795386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.495045+0000","last_change":"2023-07-14T16:13:59.292681+0000","last_active":"2023-07-14T16:15:19.495045+0000","last_peered":"2023-07-14T16:15:19.495045+0000","last_clean":"2023-07-14T16:15:19.495045+0000","last_became_active":"2023-07-14T16:13:59.292289+0000","last_became_peered":"2023-07-14T16:13:59.292289+0000","last_unstale":"2023-07-14T16:15:19.495045+0000","last_undegraded":"2023-07-14T16:15:19.495045+0000","last_fullsized":"2023-07-14T16:15:19.495045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:33.971255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666548+0000","last_change":"2023-07-14T16:13:58.282111+0000","last_active":"2023-07-14T16:14:20.666548+0000","last_peered":"2023-07-14T16:14:20.666548+0000","last_clean":"2023-07-14T16:14:20.666548+0000","last_became_active":"2023-07-14T16:13:58.281783+0000","last_became_peered":"2023-07-14T16:13:58.281783+0000","last_unstale":"2023-07-14T16:14:20.666548+0000","last_undegraded":"2023-07-14T16:14:20.666548+0000","last_fullsized":"2023-07-14T16:14:20.666548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:51:40.718143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501115+0000","last_change":"2023-07-14T16:13:59.297439+0000","last_active":"2023-07-14T16:15:19.501115+0000","last_peered":"2023-07-14T16:15:19.501115+0000","last_clean":"2023-07-14T16:15:19.501115+0000","last_became_active":"2023-07-14T16:13:59.297117+0000","last_became_peered":"2023-07-14T16:13:59.297117+0000","last_unstale":"2023-07-14T16:15:19.501115+0000","last_undegraded":"2023-07-14T16:15:19.501115+0000","last_fullsized":"2023-07-14T16:15:19.501115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:08.122607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763285+0000","last_change":"2023-07-14T16:13:58.274826+0000","last_active":"2023-07-14T16:14:20.763285+0000","last_peered":"2023-07-14T16:14:20.763285+0000","last_clean":"2023-07-14T16:14:20.763285+0000","last_became_active":"2023-07-14T16:13:58.274406+0000","last_became_peered":"2023-07-14T16:13:58.274406+0000","last_unstale":"2023-07-14T16:14:20.763285+0000","last_undegraded":"2023-07-14T16:14:20.763285+0000","last_fullsized":"2023-07-14T16:14:20.763285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:09.422218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500875+0000","last_change":"2023-07-14T16:13:59.296730+0000","last_active":"2023-07-14T16:15:19.500875+0000","last_peered":"2023-07-14T16:15:19.500875+0000","last_clean":"2023-07-14T16:15:19.500875+0000","last_became_active":"2023-07-14T16:13:59.296372+0000","last_became_peered":"2023-07-14T16:13:59.296372+0000","last_unstale":"2023-07-14T16:15:19.500875+0000","last_undegraded":"2023-07-14T16:15:19.500875+0000","last_fullsized":"2023-07-14T16:15:19.500875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:07.809329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664772+0000","last_change":"2023-07-14T16:13:58.269122+0000","last_active":"2023-07-14T16:14:20.664772+0000","last_peered":"2023-07-14T16:14:20.664772+0000","last_clean":"2023-07-14T16:14:20.664772+0000","last_became_active":"2023-07-14T16:13:58.268770+0000","last_became_peered":"2023-07-14T16:13:58.268770+0000","last_unstale":"2023-07-14T16:14:20.664772+0000","last_undegraded":"2023-07-14T16:14:20.664772+0000","last_fullsized":"2023-07-14T16:14:20.664772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:43.584997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.506513+0000","last_change":"2023-07-14T16:13:59.306987+0000","last_active":"2023-07-14T16:15:19.506513+0000","last_peered":"2023-07-14T16:15:19.506513+0000","last_clean":"2023-07-14T16:15:19.506513+0000","last_became_active":"2023-07-14T16:13:59.306649+0000","last_became_peered":"2023-07-14T16:13:59.306649+0000","last_unstale":"2023-07-14T16:15:19.506513+0000","last_undegraded":"2023-07-14T16:15:19.506513+0000","last_fullsized":"2023-07-14T16:15:19.506513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:35:23.579814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:41.719379+0000","last_change":"2023-07-14T16:13:58.281319+0000","last_active":"2023-07-14T16:15:41.719379+0000","last_peered":"2023-07-14T16:15:41.719379+0000","last_clean":"2023-07-14T16:15:41.719379+0000","last_became_active":"2023-07-14T16:13:58.280928+0000","last_became_peered":"2023-07-14T16:13:58.280928+0000","last_unstale":"2023-07-14T16:15:41.719379+0000","last_undegraded":"2023-07-14T16:15:41.719379+0000","last_fullsized":"2023-07-14T16:15:41.719379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:41.337954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507694+0000","last_change":"2023-07-14T16:13:59.309586+0000","last_active":"2023-07-14T16:15:19.507694+0000","last_peered":"2023-07-14T16:15:19.507694+0000","last_clean":"2023-07-14T16:15:19.507694+0000","last_became_active":"2023-07-14T16:13:59.309243+0000","last_became_peered":"2023-07-14T16:13:59.309243+0000","last_unstale":"2023-07-14T16:15:19.507694+0000","last_undegraded":"2023-07-14T16:15:19.507694+0000","last_fullsized":"2023-07-14T16:15:19.507694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:30.700680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661328+0000","last_change":"2023-07-14T16:13:58.279769+0000","last_active":"2023-07-14T16:14:20.661328+0000","last_peered":"2023-07-14T16:14:20.661328+0000","last_clean":"2023-07-14T16:14:20.661328+0000","last_became_active":"2023-07-14T16:13:58.279409+0000","last_became_peered":"2023-07-14T16:13:58.279409+0000","last_unstale":"2023-07-14T16:14:20.661328+0000","last_undegraded":"2023-07-14T16:14:20.661328+0000","last_fullsized":"2023-07-14T16:14:20.661328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:32.751188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500584+0000","last_change":"2023-07-14T16:13:59.300382+0000","last_active":"2023-07-14T16:15:19.500584+0000","last_peered":"2023-07-14T16:15:19.500584+0000","last_clean":"2023-07-14T16:15:19.500584+0000","last_became_active":"2023-07-14T16:13:59.299981+0000","last_became_peered":"2023-07-14T16:13:59.299981+0000","last_unstale":"2023-07-14T16:15:19.500584+0000","last_undegraded":"2023-07-14T16:15:19.500584+0000","last_fullsized":"2023-07-14T16:15:19.500584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:46:04.769609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665031+0000","last_change":"2023-07-14T16:13:58.274128+0000","last_active":"2023-07-14T16:14:20.665031+0000","last_peered":"2023-07-14T16:14:20.665031+0000","last_clean":"2023-07-14T16:14:20.665031+0000","last_became_active":"2023-07-14T16:13:58.273446+0000","last_became_peered":"2023-07-14T16:13:58.273446+0000","last_unstale":"2023-07-14T16:14:20.665031+0000","last_undegraded":"2023-07-14T16:14:20.665031+0000","last_fullsized":"2023-07-14T16:14:20.665031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:12.510793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510736+0000","last_change":"2023-07-14T16:13:59.301551+0000","last_active":"2023-07-14T16:15:19.510736+0000","last_peered":"2023-07-14T16:15:19.510736+0000","last_clean":"2023-07-14T16:15:19.510736+0000","last_became_active":"2023-07-14T16:13:59.301206+0000","last_became_peered":"2023-07-14T16:13:59.301206+0000","last_unstale":"2023-07-14T16:15:19.510736+0000","last_undegraded":"2023-07-14T16:15:19.510736+0000","last_fullsized":"2023-07-14T16:15:19.510736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:16.227849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":259,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:42.721000+0000","last_change":"2023-07-14T16:13:58.283304+0000","last_active":"2023-07-14T16:15:42.721000+0000","last_peered":"2023-07-14T16:15:42.721000+0000","last_clean":"2023-07-14T16:15:42.721000+0000","last_became_active":"2023-07-14T16:13:58.282930+0000","last_became_peered":"2023-07-14T16:13:58.282930+0000","last_unstale":"2023-07-14T16:15:42.721000+0000","last_undegraded":"2023-07-14T16:15:42.721000+0000","last_fullsized":"2023-07-14T16:15:42.721000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:02.538305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62599,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":209,"num_read_kb":161,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500725+0000","last_change":"2023-07-14T16:13:59.304144+0000","last_active":"2023-07-14T16:15:19.500725+0000","last_peered":"2023-07-14T16:15:19.500725+0000","last_clean":"2023-07-14T16:15:19.500725+0000","last_became_active":"2023-07-14T16:13:59.303745+0000","last_became_peered":"2023-07-14T16:13:59.303745+0000","last_unstale":"2023-07-14T16:15:19.500725+0000","last_undegraded":"2023-07-14T16:15:19.500725+0000","last_fullsized":"2023-07-14T16:15:19.500725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:20.828221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667830+0000","last_change":"2023-07-14T16:13:58.283734+0000","last_active":"2023-07-14T16:14:20.667830+0000","last_peered":"2023-07-14T16:14:20.667830+0000","last_clean":"2023-07-14T16:14:20.667830+0000","last_became_active":"2023-07-14T16:13:58.282765+0000","last_became_peered":"2023-07-14T16:13:58.282765+0000","last_unstale":"2023-07-14T16:14:20.667830+0000","last_undegraded":"2023-07-14T16:14:20.667830+0000","last_fullsized":"2023-07-14T16:14:20.667830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:13.217664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500485+0000","last_change":"2023-07-14T16:13:59.308445+0000","last_active":"2023-07-14T16:15:19.500485+0000","last_peered":"2023-07-14T16:15:19.500485+0000","last_clean":"2023-07-14T16:15:19.500485+0000","last_became_active":"2023-07-14T16:13:59.308130+0000","last_became_peered":"2023-07-14T16:13:59.308130+0000","last_unstale":"2023-07-14T16:15:19.500485+0000","last_undegraded":"2023-07-14T16:15:19.500485+0000","last_fullsized":"2023-07-14T16:15:19.500485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:45.004512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664202+0000","last_change":"2023-07-14T16:13:58.273973+0000","last_active":"2023-07-14T16:14:20.664202+0000","last_peered":"2023-07-14T16:14:20.664202+0000","last_clean":"2023-07-14T16:14:20.664202+0000","last_became_active":"2023-07-14T16:13:58.273606+0000","last_became_peered":"2023-07-14T16:13:58.273606+0000","last_unstale":"2023-07-14T16:14:20.664202+0000","last_undegraded":"2023-07-14T16:14:20.664202+0000","last_fullsized":"2023-07-14T16:14:20.664202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:12.304491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501694+0000","last_change":"2023-07-14T16:13:59.306428+0000","last_active":"2023-07-14T16:15:19.501694+0000","last_peered":"2023-07-14T16:15:19.501694+0000","last_clean":"2023-07-14T16:15:19.501694+0000","last_became_active":"2023-07-14T16:13:59.306090+0000","last_became_peered":"2023-07-14T16:13:59.306090+0000","last_unstale":"2023-07-14T16:15:19.501694+0000","last_undegraded":"2023-07-14T16:15:19.501694+0000","last_fullsized":"2023-07-14T16:15:19.501694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:16.033202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":130,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:43.799085+0000","last_change":"2023-07-14T16:13:58.277123+0000","last_active":"2023-07-14T16:15:43.799085+0000","last_peered":"2023-07-14T16:15:43.799085+0000","last_clean":"2023-07-14T16:15:43.799085+0000","last_became_active":"2023-07-14T16:13:58.276815+0000","last_became_peered":"2023-07-14T16:13:58.276815+0000","last_unstale":"2023-07-14T16:15:43.799085+0000","last_undegraded":"2023-07-14T16:15:43.799085+0000","last_fullsized":"2023-07-14T16:15:43.799085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:41.019233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1570,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504574+0000","last_change":"2023-07-14T16:13:59.310238+0000","last_active":"2023-07-14T16:15:19.504574+0000","last_peered":"2023-07-14T16:15:19.504574+0000","last_clean":"2023-07-14T16:15:19.504574+0000","last_became_active":"2023-07-14T16:13:59.309918+0000","last_became_peered":"2023-07-14T16:13:59.309918+0000","last_unstale":"2023-07-14T16:15:19.504574+0000","last_undegraded":"2023-07-14T16:15:19.504574+0000","last_fullsized":"2023-07-14T16:15:19.504574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:25.020575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663660+0000","last_change":"2023-07-14T16:13:58.280834+0000","last_active":"2023-07-14T16:14:20.663660+0000","last_peered":"2023-07-14T16:14:20.663660+0000","last_clean":"2023-07-14T16:14:20.663660+0000","last_became_active":"2023-07-14T16:13:58.280497+0000","last_became_peered":"2023-07-14T16:13:58.280497+0000","last_unstale":"2023-07-14T16:14:20.663660+0000","last_undegraded":"2023-07-14T16:14:20.663660+0000","last_fullsized":"2023-07-14T16:14:20.663660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:07:02.308388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.489632+0000","last_change":"2023-07-14T16:14:20.764846+0000","last_active":"2023-07-14T16:15:19.489632+0000","last_peered":"2023-07-14T16:15:19.489632+0000","last_clean":"2023-07-14T16:15:19.489632+0000","last_became_active":"2023-07-14T16:14:20.764485+0000","last_became_peered":"2023-07-14T16:14:20.764485+0000","last_unstale":"2023-07-14T16:15:19.489632+0000","last_undegraded":"2023-07-14T16:15:19.489632+0000","last_fullsized":"2023-07-14T16:15:19.489632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:29.245874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.682843+0000","last_change":"2023-07-14T16:14:20.682331+0000","last_active":"2023-07-14T16:14:20.682843+0000","last_peered":"2023-07-14T16:14:20.682843+0000","last_clean":"2023-07-14T16:14:20.682843+0000","last_became_active":"2023-07-14T16:14:20.669179+0000","last_became_peered":"2023-07-14T16:14:20.669179+0000","last_unstale":"2023-07-14T16:14:20.682843+0000","last_undegraded":"2023-07-14T16:14:20.682843+0000","last_fullsized":"2023-07-14T16:14:20.682843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:55:10.629011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502602+0000","last_change":"2023-07-14T16:13:59.297644+0000","last_active":"2023-07-14T16:15:19.502602+0000","last_peered":"2023-07-14T16:15:19.502602+0000","last_clean":"2023-07-14T16:15:19.502602+0000","last_became_active":"2023-07-14T16:13:59.297332+0000","last_became_peered":"2023-07-14T16:13:59.297332+0000","last_unstale":"2023-07-14T16:15:19.502602+0000","last_undegraded":"2023-07-14T16:15:19.502602+0000","last_fullsized":"2023-07-14T16:15:19.502602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:36.421075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665695+0000","last_change":"2023-07-14T16:13:58.280632+0000","last_active":"2023-07-14T16:14:20.665695+0000","last_peered":"2023-07-14T16:14:20.665695+0000","last_clean":"2023-07-14T16:14:20.665695+0000","last_became_active":"2023-07-14T16:13:58.280296+0000","last_became_peered":"2023-07-14T16:13:58.280296+0000","last_unstale":"2023-07-14T16:14:20.665695+0000","last_undegraded":"2023-07-14T16:14:20.665695+0000","last_fullsized":"2023-07-14T16:14:20.665695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:26.897678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.505470+0000","last_change":"2023-07-14T16:13:59.297092+0000","last_active":"2023-07-14T16:15:19.505470+0000","last_peered":"2023-07-14T16:15:19.505470+0000","last_clean":"2023-07-14T16:15:19.505470+0000","last_became_active":"2023-07-14T16:13:59.296750+0000","last_became_peered":"2023-07-14T16:13:59.296750+0000","last_unstale":"2023-07-14T16:15:19.505470+0000","last_undegraded":"2023-07-14T16:15:19.505470+0000","last_fullsized":"2023-07-14T16:15:19.505470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:32.797912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660888+0000","last_change":"2023-07-14T16:13:58.281786+0000","last_active":"2023-07-14T16:14:20.660888+0000","last_peered":"2023-07-14T16:14:20.660888+0000","last_clean":"2023-07-14T16:14:20.660888+0000","last_became_active":"2023-07-14T16:13:58.281471+0000","last_became_peered":"2023-07-14T16:13:58.281471+0000","last_unstale":"2023-07-14T16:14:20.660888+0000","last_undegraded":"2023-07-14T16:14:20.660888+0000","last_fullsized":"2023-07-14T16:14:20.660888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:09.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502616+0000","last_change":"2023-07-14T16:14:20.672813+0000","last_active":"2023-07-14T16:15:19.502616+0000","last_peered":"2023-07-14T16:15:19.502616+0000","last_clean":"2023-07-14T16:15:19.502616+0000","last_became_active":"2023-07-14T16:14:20.672447+0000","last_became_peered":"2023-07-14T16:14:20.672447+0000","last_unstale":"2023-07-14T16:15:19.502616+0000","last_undegraded":"2023-07-14T16:15:19.502616+0000","last_fullsized":"2023-07-14T16:15:19.502616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:33:07.582055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.495526+0000","last_change":"2023-07-14T16:13:59.288836+0000","last_active":"2023-07-14T16:15:19.495526+0000","last_peered":"2023-07-14T16:15:19.495526+0000","last_clean":"2023-07-14T16:15:19.495526+0000","last_became_active":"2023-07-14T16:13:59.288517+0000","last_became_peered":"2023-07-14T16:13:59.288517+0000","last_unstale":"2023-07-14T16:15:19.495526+0000","last_undegraded":"2023-07-14T16:15:19.495526+0000","last_fullsized":"2023-07-14T16:15:19.495526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:48.293313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":124,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:42.796187+0000","last_change":"2023-07-14T16:13:58.282415+0000","last_active":"2023-07-14T16:15:42.796187+0000","last_peered":"2023-07-14T16:15:42.796187+0000","last_clean":"2023-07-14T16:15:42.796187+0000","last_became_active":"2023-07-14T16:13:58.282118+0000","last_became_peered":"2023-07-14T16:13:58.282118+0000","last_unstale":"2023-07-14T16:15:42.796187+0000","last_undegraded":"2023-07-14T16:15:42.796187+0000","last_fullsized":"2023-07-14T16:15:42.796187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:41.532341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501135+0000","last_change":"2023-07-14T16:13:59.302860+0000","last_active":"2023-07-14T16:15:19.501135+0000","last_peered":"2023-07-14T16:15:19.501135+0000","last_clean":"2023-07-14T16:15:19.501135+0000","last_became_active":"2023-07-14T16:13:59.302454+0000","last_became_peered":"2023-07-14T16:13:59.302454+0000","last_unstale":"2023-07-14T16:15:19.501135+0000","last_undegraded":"2023-07-14T16:15:19.501135+0000","last_fullsized":"2023-07-14T16:15:19.501135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:34.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664397+0000","last_change":"2023-07-14T16:13:58.273908+0000","last_active":"2023-07-14T16:14:20.664397+0000","last_peered":"2023-07-14T16:14:20.664397+0000","last_clean":"2023-07-14T16:14:20.664397+0000","last_became_active":"2023-07-14T16:13:58.273497+0000","last_became_peered":"2023-07-14T16:13:58.273497+0000","last_unstale":"2023-07-14T16:14:20.664397+0000","last_undegraded":"2023-07-14T16:14:20.664397+0000","last_fullsized":"2023-07-14T16:14:20.664397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:55:44.978027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661044+0000","last_change":"2023-07-14T16:12:36.546475+0000","last_active":"2023-07-14T16:14:20.661044+0000","last_peered":"2023-07-14T16:14:20.661044+0000","last_clean":"2023-07-14T16:14:20.661044+0000","last_became_active":"2023-07-14T16:12:36.545943+0000","last_became_peered":"2023-07-14T16:12:36.545943+0000","last_unstale":"2023-07-14T16:14:20.661044+0000","last_undegraded":"2023-07-14T16:14:20.661044+0000","last_fullsized":"2023-07-14T16:14:20.661044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_clean_scrub_stamp":"2023-07-14T16:08:58.024408+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:27.166990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500653+0000","last_change":"2023-07-14T16:13:59.297669+0000","last_active":"2023-07-14T16:15:19.500653+0000","last_peered":"2023-07-14T16:15:19.500653+0000","last_clean":"2023-07-14T16:15:19.500653+0000","last_became_active":"2023-07-14T16:13:59.297364+0000","last_became_peered":"2023-07-14T16:13:59.297364+0000","last_unstale":"2023-07-14T16:15:19.500653+0000","last_undegraded":"2023-07-14T16:15:19.500653+0000","last_fullsized":"2023-07-14T16:15:19.500653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:50:00.972727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.673166+0000","last_change":"2023-07-14T16:14:20.673166+0000","last_active":"2023-07-14T16:14:20.673166+0000","last_peered":"2023-07-14T16:14:20.673166+0000","last_clean":"2023-07-14T16:14:20.673166+0000","last_became_active":"2023-07-14T16:14:20.667664+0000","last_became_peered":"2023-07-14T16:14:20.667664+0000","last_unstale":"2023-07-14T16:14:20.673166+0000","last_undegraded":"2023-07-14T16:14:20.673166+0000","last_fullsized":"2023-07-14T16:14:20.673166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:24.328768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664285+0000","last_change":"2023-07-14T16:13:58.276257+0000","last_active":"2023-07-14T16:14:20.664285+0000","last_peered":"2023-07-14T16:14:20.664285+0000","last_clean":"2023-07-14T16:14:20.664285+0000","last_became_active":"2023-07-14T16:13:58.275910+0000","last_became_peered":"2023-07-14T16:13:58.275910+0000","last_unstale":"2023-07-14T16:14:20.664285+0000","last_undegraded":"2023-07-14T16:14:20.664285+0000","last_fullsized":"2023-07-14T16:14:20.664285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:47.700225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498722+0000","last_change":"2023-07-14T16:13:59.299135+0000","last_active":"2023-07-14T16:15:19.498722+0000","last_peered":"2023-07-14T16:15:19.498722+0000","last_clean":"2023-07-14T16:15:19.498722+0000","last_became_active":"2023-07-14T16:13:59.298031+0000","last_became_peered":"2023-07-14T16:13:59.298031+0000","last_unstale":"2023-07-14T16:15:19.498722+0000","last_undegraded":"2023-07-14T16:15:19.498722+0000","last_fullsized":"2023-07-14T16:15:19.498722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:21.882960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665411+0000","last_change":"2023-07-14T16:13:58.279664+0000","last_active":"2023-07-14T16:14:20.665411+0000","last_peered":"2023-07-14T16:14:20.665411+0000","last_clean":"2023-07-14T16:14:20.665411+0000","last_became_active":"2023-07-14T16:13:58.279137+0000","last_became_peered":"2023-07-14T16:13:58.279137+0000","last_unstale":"2023-07-14T16:14:20.665411+0000","last_undegraded":"2023-07-14T16:14:20.665411+0000","last_fullsized":"2023-07-14T16:14:20.665411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:07.657781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500067+0000","last_change":"2023-07-14T16:13:59.305536+0000","last_active":"2023-07-14T16:15:19.500067+0000","last_peered":"2023-07-14T16:15:19.500067+0000","last_clean":"2023-07-14T16:15:19.500067+0000","last_became_active":"2023-07-14T16:13:59.305176+0000","last_became_peered":"2023-07-14T16:13:59.305176+0000","last_unstale":"2023-07-14T16:15:19.500067+0000","last_undegraded":"2023-07-14T16:15:19.500067+0000","last_fullsized":"2023-07-14T16:15:19.500067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:18:18.141957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664829+0000","last_change":"2023-07-14T16:13:58.269652+0000","last_active":"2023-07-14T16:14:20.664829+0000","last_peered":"2023-07-14T16:14:20.664829+0000","last_clean":"2023-07-14T16:14:20.664829+0000","last_became_active":"2023-07-14T16:13:58.269338+0000","last_became_peered":"2023-07-14T16:13:58.269338+0000","last_unstale":"2023-07-14T16:14:20.664829+0000","last_undegraded":"2023-07-14T16:14:20.664829+0000","last_fullsized":"2023-07-14T16:14:20.664829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:55:21.910971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502139+0000","last_change":"2023-07-14T16:13:59.297574+0000","last_active":"2023-07-14T16:15:19.502139+0000","last_peered":"2023-07-14T16:15:19.502139+0000","last_clean":"2023-07-14T16:15:19.502139+0000","last_became_active":"2023-07-14T16:13:59.297124+0000","last_became_peered":"2023-07-14T16:13:59.297124+0000","last_unstale":"2023-07-14T16:15:19.502139+0000","last_undegraded":"2023-07-14T16:15:19.502139+0000","last_fullsized":"2023-07-14T16:15:19.502139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:20.051395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763343+0000","last_change":"2023-07-14T16:13:58.278011+0000","last_active":"2023-07-14T16:14:20.763343+0000","last_peered":"2023-07-14T16:14:20.763343+0000","last_clean":"2023-07-14T16:14:20.763343+0000","last_became_active":"2023-07-14T16:13:58.277563+0000","last_became_peered":"2023-07-14T16:13:58.277563+0000","last_unstale":"2023-07-14T16:14:20.763343+0000","last_undegraded":"2023-07-14T16:14:20.763343+0000","last_fullsized":"2023-07-14T16:14:20.763343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:06.354928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.463529+0000","last_change":"2023-07-14T16:13:59.308836+0000","last_active":"2023-07-14T16:15:19.463529+0000","last_peered":"2023-07-14T16:15:19.463529+0000","last_clean":"2023-07-14T16:15:19.463529+0000","last_became_active":"2023-07-14T16:13:59.308510+0000","last_became_peered":"2023-07-14T16:13:59.308510+0000","last_unstale":"2023-07-14T16:15:19.463529+0000","last_undegraded":"2023-07-14T16:15:19.463529+0000","last_fullsized":"2023-07-14T16:15:19.463529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:48:07.973849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663352+0000","last_change":"2023-07-14T16:13:58.277438+0000","last_active":"2023-07-14T16:14:20.663352+0000","last_peered":"2023-07-14T16:14:20.663352+0000","last_clean":"2023-07-14T16:14:20.663352+0000","last_became_active":"2023-07-14T16:13:58.277114+0000","last_became_peered":"2023-07-14T16:13:58.277114+0000","last_unstale":"2023-07-14T16:14:20.663352+0000","last_undegraded":"2023-07-14T16:14:20.663352+0000","last_fullsized":"2023-07-14T16:14:20.663352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:40.636429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502829+0000","last_change":"2023-07-14T16:13:59.304491+0000","last_active":"2023-07-14T16:15:19.502829+0000","last_peered":"2023-07-14T16:15:19.502829+0000","last_clean":"2023-07-14T16:15:19.502829+0000","last_became_active":"2023-07-14T16:13:59.303925+0000","last_became_peered":"2023-07-14T16:13:59.303925+0000","last_unstale":"2023-07-14T16:15:19.502829+0000","last_undegraded":"2023-07-14T16:15:19.502829+0000","last_fullsized":"2023-07-14T16:15:19.502829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.410847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.685314+0000","last_change":"2023-07-14T16:14:20.684842+0000","last_active":"2023-07-14T16:14:20.685314+0000","last_peered":"2023-07-14T16:14:20.685314+0000","last_clean":"2023-07-14T16:14:20.685314+0000","last_became_active":"2023-07-14T16:14:20.670098+0000","last_became_peered":"2023-07-14T16:14:20.670098+0000","last_unstale":"2023-07-14T16:14:20.685314+0000","last_undegraded":"2023-07-14T16:14:20.685314+0000","last_fullsized":"2023-07-14T16:14:20.685314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:54:58.021152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502027+0000","last_change":"2023-07-14T16:13:59.310983+0000","last_active":"2023-07-14T16:15:19.502027+0000","last_peered":"2023-07-14T16:15:19.502027+0000","last_clean":"2023-07-14T16:15:19.502027+0000","last_became_active":"2023-07-14T16:13:59.310636+0000","last_became_peered":"2023-07-14T16:13:59.310636+0000","last_unstale":"2023-07-14T16:15:19.502027+0000","last_undegraded":"2023-07-14T16:15:19.502027+0000","last_fullsized":"2023-07-14T16:15:19.502027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:31.003517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663068+0000","last_change":"2023-07-14T16:13:58.280153+0000","last_active":"2023-07-14T16:14:20.663068+0000","last_peered":"2023-07-14T16:14:20.663068+0000","last_clean":"2023-07-14T16:14:20.663068+0000","last_became_active":"2023-07-14T16:13:58.279771+0000","last_became_peered":"2023-07-14T16:13:58.279771+0000","last_unstale":"2023-07-14T16:14:20.663068+0000","last_undegraded":"2023-07-14T16:14:20.663068+0000","last_fullsized":"2023-07-14T16:14:20.663068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:46.290429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510162+0000","last_change":"2023-07-14T16:13:59.304854+0000","last_active":"2023-07-14T16:15:19.510162+0000","last_peered":"2023-07-14T16:15:19.510162+0000","last_clean":"2023-07-14T16:15:19.510162+0000","last_became_active":"2023-07-14T16:13:59.304374+0000","last_became_peered":"2023-07-14T16:13:59.304374+0000","last_unstale":"2023-07-14T16:15:19.510162+0000","last_undegraded":"2023-07-14T16:15:19.510162+0000","last_fullsized":"2023-07-14T16:15:19.510162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:52.213737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667834+0000","last_change":"2023-07-14T16:13:58.281885+0000","last_active":"2023-07-14T16:14:20.667834+0000","last_peered":"2023-07-14T16:14:20.667834+0000","last_clean":"2023-07-14T16:14:20.667834+0000","last_became_active":"2023-07-14T16:13:58.278759+0000","last_became_peered":"2023-07-14T16:13:58.278759+0000","last_unstale":"2023-07-14T16:14:20.667834+0000","last_undegraded":"2023-07-14T16:14:20.667834+0000","last_fullsized":"2023-07-14T16:14:20.667834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:11.816765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.511619+0000","last_change":"2023-07-14T16:13:59.302680+0000","last_active":"2023-07-14T16:15:19.511619+0000","last_peered":"2023-07-14T16:15:19.511619+0000","last_clean":"2023-07-14T16:15:19.511619+0000","last_became_active":"2023-07-14T16:13:59.301854+0000","last_became_peered":"2023-07-14T16:13:59.301854+0000","last_unstale":"2023-07-14T16:15:19.511619+0000","last_undegraded":"2023-07-14T16:15:19.511619+0000","last_fullsized":"2023-07-14T16:15:19.511619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:29:42.861394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664860+0000","last_change":"2023-07-14T16:13:58.274254+0000","last_active":"2023-07-14T16:14:20.664860+0000","last_peered":"2023-07-14T16:14:20.664860+0000","last_clean":"2023-07-14T16:14:20.664860+0000","last_became_active":"2023-07-14T16:13:58.273933+0000","last_became_peered":"2023-07-14T16:13:58.273933+0000","last_unstale":"2023-07-14T16:14:20.664860+0000","last_undegraded":"2023-07-14T16:14:20.664860+0000","last_fullsized":"2023-07-14T16:14:20.664860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:53.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501047+0000","last_change":"2023-07-14T16:13:59.305615+0000","last_active":"2023-07-14T16:15:19.501047+0000","last_peered":"2023-07-14T16:15:19.501047+0000","last_clean":"2023-07-14T16:15:19.501047+0000","last_became_active":"2023-07-14T16:13:59.305206+0000","last_became_peered":"2023-07-14T16:13:59.305206+0000","last_unstale":"2023-07-14T16:15:19.501047+0000","last_undegraded":"2023-07-14T16:15:19.501047+0000","last_fullsized":"2023-07-14T16:15:19.501047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:41.771061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763463+0000","last_change":"2023-07-14T16:13:58.277976+0000","last_active":"2023-07-14T16:14:20.763463+0000","last_peered":"2023-07-14T16:14:20.763463+0000","last_clean":"2023-07-14T16:14:20.763463+0000","last_became_active":"2023-07-14T16:13:58.277618+0000","last_became_peered":"2023-07-14T16:13:58.277618+0000","last_unstale":"2023-07-14T16:14:20.763463+0000","last_undegraded":"2023-07-14T16:14:20.763463+0000","last_fullsized":"2023-07-14T16:14:20.763463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:25.780591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504344+0000","last_change":"2023-07-14T16:13:59.302041+0000","last_active":"2023-07-14T16:15:19.504344+0000","last_peered":"2023-07-14T16:15:19.504344+0000","last_clean":"2023-07-14T16:15:19.504344+0000","last_became_active":"2023-07-14T16:13:59.301620+0000","last_became_peered":"2023-07-14T16:13:59.301620+0000","last_unstale":"2023-07-14T16:15:19.504344+0000","last_undegraded":"2023-07-14T16:15:19.504344+0000","last_fullsized":"2023-07-14T16:15:19.504344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:18.969064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502705+0000","last_change":"2023-07-14T16:13:59.304222+0000","last_active":"2023-07-14T16:15:19.502705+0000","last_peered":"2023-07-14T16:15:19.502705+0000","last_clean":"2023-07-14T16:15:19.502705+0000","last_became_active":"2023-07-14T16:13:59.303404+0000","last_became_peered":"2023-07-14T16:13:59.303404+0000","last_unstale":"2023-07-14T16:15:19.502705+0000","last_undegraded":"2023-07-14T16:15:19.502705+0000","last_fullsized":"2023-07-14T16:15:19.502705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:26.218664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.490011+0000","last_change":"2023-07-14T16:13:59.296433+0000","last_active":"2023-07-14T16:15:19.490011+0000","last_peered":"2023-07-14T16:15:19.490011+0000","last_clean":"2023-07-14T16:15:19.490011+0000","last_became_active":"2023-07-14T16:13:59.295604+0000","last_became_peered":"2023-07-14T16:13:59.295604+0000","last_unstale":"2023-07-14T16:15:19.490011+0000","last_undegraded":"2023-07-14T16:15:19.490011+0000","last_fullsized":"2023-07-14T16:15:19.490011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:17:57.269484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.493883+0000","last_change":"2023-07-14T16:13:59.294198+0000","last_active":"2023-07-14T16:15:19.493883+0000","last_peered":"2023-07-14T16:15:19.493883+0000","last_clean":"2023-07-14T16:15:19.493883+0000","last_became_active":"2023-07-14T16:13:59.293891+0000","last_became_peered":"2023-07-14T16:13:59.293891+0000","last_unstale":"2023-07-14T16:15:19.493883+0000","last_undegraded":"2023-07-14T16:15:19.493883+0000","last_fullsized":"2023-07-14T16:15:19.493883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:41.192604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503095+0000","last_change":"2023-07-14T16:13:59.298724+0000","last_active":"2023-07-14T16:15:19.503095+0000","last_peered":"2023-07-14T16:15:19.503095+0000","last_clean":"2023-07-14T16:15:19.503095+0000","last_became_active":"2023-07-14T16:13:59.298318+0000","last_became_peered":"2023-07-14T16:13:59.298318+0000","last_unstale":"2023-07-14T16:15:19.503095+0000","last_undegraded":"2023-07-14T16:15:19.503095+0000","last_fullsized":"2023-07-14T16:15:19.503095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:30:13.650949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503946+0000","last_change":"2023-07-14T16:13:59.294652+0000","last_active":"2023-07-14T16:15:19.503946+0000","last_peered":"2023-07-14T16:15:19.503946+0000","last_clean":"2023-07-14T16:15:19.503946+0000","last_became_active":"2023-07-14T16:13:59.294278+0000","last_became_peered":"2023-07-14T16:13:59.294278+0000","last_unstale":"2023-07-14T16:15:19.503946+0000","last_undegraded":"2023-07-14T16:15:19.503946+0000","last_fullsized":"2023-07-14T16:15:19.503946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:45:58.125276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502135+0000","last_change":"2023-07-14T16:13:59.299228+0000","last_active":"2023-07-14T16:15:19.502135+0000","last_peered":"2023-07-14T16:15:19.502135+0000","last_clean":"2023-07-14T16:15:19.502135+0000","last_became_active":"2023-07-14T16:13:59.297982+0000","last_became_peered":"2023-07-14T16:13:59.297982+0000","last_unstale":"2023-07-14T16:15:19.502135+0000","last_undegraded":"2023-07-14T16:15:19.502135+0000","last_fullsized":"2023-07-14T16:15:19.502135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:24.432565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508368+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:15:19.508368+0000","last_peered":"2023-07-14T16:15:19.508368+0000","last_clean":"2023-07-14T16:15:19.508368+0000","last_became_active":"2023-07-14T16:13:59.299181+0000","last_became_peered":"2023-07-14T16:13:59.299181+0000","last_unstale":"2023-07-14T16:15:19.508368+0000","last_undegraded":"2023-07-14T16:15:19.508368+0000","last_fullsized":"2023-07-14T16:15:19.508368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:42.604278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.488076+0000","last_change":"2023-07-14T16:13:59.303282+0000","last_active":"2023-07-14T16:15:19.488076+0000","last_peered":"2023-07-14T16:15:19.488076+0000","last_clean":"2023-07-14T16:15:19.488076+0000","last_became_active":"2023-07-14T16:13:59.302922+0000","last_became_peered":"2023-07-14T16:13:59.302922+0000","last_unstale":"2023-07-14T16:15:19.488076+0000","last_undegraded":"2023-07-14T16:15:19.488076+0000","last_fullsized":"2023-07-14T16:15:19.488076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.750328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499836+0000","last_change":"2023-07-14T16:13:59.296389+0000","last_active":"2023-07-14T16:15:19.499836+0000","last_peered":"2023-07-14T16:15:19.499836+0000","last_clean":"2023-07-14T16:15:19.499836+0000","last_became_active":"2023-07-14T16:13:59.296080+0000","last_became_peered":"2023-07-14T16:13:59.296080+0000","last_unstale":"2023-07-14T16:15:19.499836+0000","last_undegraded":"2023-07-14T16:15:19.499836+0000","last_fullsized":"2023-07-14T16:15:19.499836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:02.060808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500459+0000","last_change":"2023-07-14T16:13:59.297774+0000","last_active":"2023-07-14T16:15:19.500459+0000","last_peered":"2023-07-14T16:15:19.500459+0000","last_clean":"2023-07-14T16:15:19.500459+0000","last_became_active":"2023-07-14T16:13:59.297463+0000","last_became_peered":"2023-07-14T16:13:59.297463+0000","last_unstale":"2023-07-14T16:15:19.500459+0000","last_undegraded":"2023-07-14T16:15:19.500459+0000","last_fullsized":"2023-07-14T16:15:19.500459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:49.304022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508679+0000","last_change":"2023-07-14T16:13:59.306795+0000","last_active":"2023-07-14T16:15:19.508679+0000","last_peered":"2023-07-14T16:15:19.508679+0000","last_clean":"2023-07-14T16:15:19.508679+0000","last_became_active":"2023-07-14T16:13:59.306457+0000","last_became_peered":"2023-07-14T16:13:59.306457+0000","last_unstale":"2023-07-14T16:15:19.508679+0000","last_undegraded":"2023-07-14T16:15:19.508679+0000","last_fullsized":"2023-07-14T16:15:19.508679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:47.873750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504322+0000","last_change":"2023-07-14T16:13:59.305338+0000","last_active":"2023-07-14T16:15:19.504322+0000","last_peered":"2023-07-14T16:15:19.504322+0000","last_clean":"2023-07-14T16:15:19.504322+0000","last_became_active":"2023-07-14T16:13:59.305047+0000","last_became_peered":"2023-07-14T16:13:59.305047+0000","last_unstale":"2023-07-14T16:15:19.504322+0000","last_undegraded":"2023-07-14T16:15:19.504322+0000","last_fullsized":"2023-07-14T16:15:19.504322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:07:02.694956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503214+0000","last_change":"2023-07-14T16:13:59.299005+0000","last_active":"2023-07-14T16:15:19.503214+0000","last_peered":"2023-07-14T16:15:19.503214+0000","last_clean":"2023-07-14T16:15:19.503214+0000","last_became_active":"2023-07-14T16:13:59.298705+0000","last_became_peered":"2023-07-14T16:13:59.298705+0000","last_unstale":"2023-07-14T16:15:19.503214+0000","last_undegraded":"2023-07-14T16:15:19.503214+0000","last_fullsized":"2023-07-14T16:15:19.503214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:59:41.986408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247144,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67642,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":631,"num_read_kb":382,"num_write":162,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":198606,"data_compressed":4809,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":459298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97699999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80000000000000004,"15min":0.78800000000000003},"min":{"1min":0.61799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0069999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.135,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45900000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.84599999999999997,"15min":0.83099999999999996},"min":{"1min":0.70299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1919999999999999,"5min":1.385,"15min":1.385},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97499999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70299999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96999999999999997,"5min":1.361,"15min":1.361},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.165,"5min":1.264,"15min":1.264},"last":0.89300000000000002}]},{"osd":6,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.91700000000000004,"15min":0.90300000000000002},"min":{"1min":0.78300000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1359999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.89000000000000001,"15min":0.89600000000000002},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0629999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.88100000000000001}]},{"osd":8,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.04},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0600000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.94099999999999995,"15min":0.95399999999999996},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5880000000000001,"5min":1.992,"15min":1.992},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.96499999999999997,"15min":0.94899999999999995},"min":{"1min":0.79000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.397,"5min":1.653,"15min":1.653},"last":1.0189999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.92200000000000004,"15min":0.93000000000000005},"min":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.099,"5min":2.097,"15min":2.097},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.95099999999999996,"15min":0.96299999999999997},"min":{"1min":0.68500000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.081,"5min":2.024,"15min":2.024},"last":0.90600000000000003}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1093632,"data_stored":1109743,"data_compressed":37379,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.165,"5min":7.165,"15min":7.165},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.125,"5min":7.125,"15min":7.125},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79300000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73999999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.63100000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94599999999999995,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.501},"max":{"1min":0.92400000000000004,"5min":1.173,"15min":1.173},"last":0.745}]},{"osd":3,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.81999999999999995,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.80000000000000004,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.879,"5min":4.4429999999999996,"15min":4.4429999999999996},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.74299999999999999,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":1.613,"15min":1.613},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.92600000000000005,"15min":0.91600000000000004},"min":{"1min":0.68500000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.016,"5min":4.9749999999999996,"15min":4.9749999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89300000000000002,"15min":0.91400000000000003},"min":{"1min":0.65800000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0369999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.86499999999999999}]},{"osd":6,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":1.01,"15min":0.95099999999999996},"min":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.05,"5min":6.609,"15min":6.609},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96599999999999997,"15min":0.93200000000000005},"min":{"1min":0.67000000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1499999999999999,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.61699999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.97499999999999998,"15min":0.98299999999999998},"min":{"1min":0.754,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.103,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.98099999999999998,"15min":1.0069999999999999},"min":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0740000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.85299999999999998}]},{"osd":9,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.72699999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1830000000000001,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.121,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1579999999999999,"5min":7.3579999999999997,"15min":7.3579999999999997},"last":0.79700000000000004}]},{"osd":11,"last update":"Fri Jul 14 16:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.046,"15min":1.05},"min":{"1min":0.84799999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.1890000000000001,"5min":2.653,"15min":2.653},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0089999999999999,"15min":1.002},"min":{"1min":0.92200000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90900000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":459512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82599999999999996,"15min":0.86199999999999999},"min":{"1min":0.55200000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1259999999999999,"5min":1.129,"15min":1.129},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82899999999999996,"15min":0.86499999999999999},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.2809999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.878},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.82699999999999996,"15min":0.872},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63600000000000001,"15min":0.66100000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.376},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78300000000000003,"15min":0.83199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.629}]},{"osd":4,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.88600000000000001,"15min":0.93400000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.82899999999999996,"15min":0.85299999999999998},"min":{"1min":0.55000000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84499999999999997,"15min":0.877},"min":{"1min":0.502,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3220000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.94099999999999995,"15min":1.0029999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.448,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.93999999999999995,"15min":1.014},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1799999999999999,"5min":1.288,"15min":1.288},"last":0.59399999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94299999999999995,"15min":0.98299999999999998},"min":{"1min":0.58999999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.879,"5min":1.024,"15min":1.0720000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3979999999999999,"5min":1.575,"15min":1.575},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0229999999999999,"15min":1.0840000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.996,"15min":1.0449999999999999},"min":{"1min":0.69299999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.85799999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.96999999999999997,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.034,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.95999999999999996},"min":{"1min":0.66500000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.66500000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":459410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66400000000000003,"15min":0.68899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88300000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72899999999999998,"15min":0.69799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":6.3410000000000002,"15min":6.3410000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98699999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.48799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.1100000000000001,"5min":1.216,"15min":1.216},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.081,"15min":1.081},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.89700000000000002}]},{"osd":3,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99099999999999999,"5min":1.175,"15min":1.175},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.78700000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92600000000000005,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.54200000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.61499999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.0289999999999999,"5min":1.218,"15min":1.218},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69899999999999995,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73599999999999999,"15min":0.74399999999999999},"min":{"1min":0.46500000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.05,"5min":1.21,"15min":1.21},"last":0.624}]},{"osd":6,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84199999999999997,"15min":0.85799999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1839999999999999,"5min":1.29,"15min":1.29},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.63500000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.256,"15min":1.256},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.93799999999999994,"15min":0.89300000000000002},"min":{"1min":0.63200000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.222,"5min":6.3209999999999997,"15min":6.3209999999999997},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89000000000000001,"15min":0.89800000000000002},"min":{"1min":0.68400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.1919999999999999,"5min":1.373,"15min":1.373},"last":0.92200000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.93899999999999995,"15min":0.91700000000000004},"min":{"1min":0.76700000000000002,"5min":0.625,"15min":0.625},"max":{"1min":3.4470000000000001,"5min":3.4470000000000001,"15min":3.4470000000000001},"last":0.875},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90600000000000003,"15min":0.91500000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.2130000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Jul 14 16:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.89800000000000002,"15min":0.86399999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1870000000000001,"5min":1.258,"15min":1.258},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.90800000000000003,"15min":0.88300000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.2050000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86199999999999999}]},{"osd":11,"last update":"Fri Jul 14 16:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.96199999999999997}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":1052410,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.181,"15min":1.181},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52600000000000002,"15min":0.51200000000000001},"min":{"1min":0.40000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79100000000000004,"5min":1.046,"15min":1.046},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.755}]},{"osd":2,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71999999999999997,"15min":0.71899999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.60299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.52100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.72599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95899999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.024,"15min":1.024},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90800000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.443}]},{"osd":5,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0760000000000001,"5min":1.246,"15min":1.246},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.64700000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.93000000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.495,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93400000000000005,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84999999999999998,"5min":1.137,"15min":1.137},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Jul 14 16:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.67400000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0369999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.69999999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.079,"5min":1.331,"15min":1.331},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Jul 14 16:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.88800000000000001,"15min":0.89400000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.054,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.628,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0189999999999999,"5min":1.262,"15min":1.262},"last":0.748}]},{"osd":10,"last update":"Fri Jul 14 16:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.71699999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1080000000000001,"5min":1.238,"15min":1.238},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.89500000000000002,"15min":0.89900000000000002},"min":{"1min":0.70199999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0920000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.71199999999999997}]},{"osd":11,"last update":"Fri Jul 14 16:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.96599999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":458907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.48199999999999998}]},{"osd":1,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.68999999999999995},"min":{"1min":0.5,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.67800000000000005},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98599999999999999,"5min":1.111,"15min":1.111},"last":0.82099999999999995}]},{"osd":2,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0009999999999999,"5min":1.121,"15min":1.121},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94799999999999995,"5min":1.105,"15min":1.105},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.62},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.79100000000000004,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82099999999999995,"15min":0.81000000000000005},"min":{"1min":0.53800000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0720000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jul 14 16:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.83399999999999996,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1739999999999999,"5min":1.28,"15min":1.28},"last":1.048},{"interface":"front","average":{"1min":0.874,"5min":0.85699999999999998,"15min":0.80800000000000005},"min":{"1min":0.65100000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.238,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0129999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.90700000000000003,"15min":0.92900000000000005},"min":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1180000000000001,"5min":1.353,"15min":1.353},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1020000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.94699999999999995}]},{"osd":9,"last update":"Fri Jul 14 16:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.88100000000000001,"15min":0.92000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.141,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.93600000000000005,"15min":0.98299999999999998},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.129,"5min":1.331,"15min":1.331},"last":1.0880000000000001}]},{"osd":10,"last update":"Fri Jul 14 16:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.92000000000000004,"15min":0.93400000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.149,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.95199999999999996,"15min":0.98199999999999998},"min":{"1min":0.77400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.073,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":516973,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.55300000000000005},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.34899999999999998},"max":{"1min":0.85699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.503}]},{"osd":2,"last update":"Fri Jul 14 16:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.71799999999999997,"15min":0.61899999999999999},"min":{"1min":0.52200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.0860000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.71299999999999997,"15min":0.63700000000000001},"min":{"1min":0.50700000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.69699999999999995,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57599999999999996,"15min":0.51800000000000002},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93500000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.47199999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68300000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71099999999999997,"15min":0.66400000000000003},"min":{"1min":0.61299999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.71299999999999997,"15min":0.66500000000000004},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.751}]},{"osd":6,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76700000000000002,"15min":0.73099999999999998},"min":{"1min":0.61299999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0189999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.64100000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.77300000000000002,"15min":0.70699999999999996},"min":{"1min":0.68000000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1919999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.754,"15min":0.68200000000000005},"min":{"1min":0.48799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.1559999999999999,"5min":1.244,"15min":1.244},"last":0.76300000000000001}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2050000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.81200000000000006,"15min":0.77200000000000002},"min":{"1min":0.75,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.73699999999999999}]},{"osd":9,"last update":"Fri Jul 14 16:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.871,"15min":0.875},"min":{"1min":0.68999999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.091,"5min":1.204,"15min":1.204},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91700000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.262,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.58099999999999996}]},{"osd":10,"last update":"Fri Jul 14 16:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.66700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92400000000000004,"15min":0.92800000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.05,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.65300000000000002}]},{"osd":11,"last update":"Fri Jul 14 16:15:02 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.872},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.83099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":460172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63800000000000001,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jul 14 16:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73799999999999999,"15min":0.63900000000000001},"min":{"1min":0.60099999999999998,"5min":0.38200000000000001,"15min":0.34399999999999997},"max":{"1min":0.96399999999999997,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.64900000000000002},"min":{"1min":0.53300000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.03,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.3599999999999999,"5min":3.3599999999999999,"15min":3.3599999999999999},"last":0.504}]},{"osd":4,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2030000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.68400000000000005,"15min":0.64800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":1.357,"15min":1.357},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.74199999999999999,"15min":0.69399999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70999999999999996,"15min":0.67800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.5670000000000002,"5min":3.5670000000000002,"15min":3.5670000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9630000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.748,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83799999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.92100000000000004},"min":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.85899999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.89700000000000002}]},{"osd":9,"last update":"Fri Jul 14 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.878,"15min":0.88},"min":{"1min":0.5,"5min":0.495,"15min":0.495},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.91500000000000004,"15min":0.94399999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.61499999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88600000000000001,"15min":0.91300000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2370000000000001,"5min":1.419,"15min":1.419},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.92600000000000005,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.93799999999999994,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.88300000000000001}]},{"osd":11,"last update":"Fri Jul 14 16:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89000000000000001,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.171,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.5350000000000001,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":0.82799999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":518774,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.376,"15min":0.34699999999999998},"max":{"1min":4.4400000000000004,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.47299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65100000000000002,"15min":0.66400000000000003},"min":{"1min":0.51400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":0.90500000000000003,"15min":0.90800000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.70599999999999996,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.41799999999999998},"max":{"1min":1.0569999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.038}]},{"osd":3,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.39200000000000002,"15min":0.39100000000000001},"max":{"1min":2.1309999999999998,"5min":3.274,"15min":3.274},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67200000000000004,"15min":0.629},"min":{"1min":0.52300000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Jul 14 16:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.64700000000000002,"15min":0.58799999999999997},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59899999999999998,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90100000000000002,"5min":1.022,"15min":1.022},"last":0.48999999999999999}]},{"osd":6,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.76600000000000001,"15min":0.71999999999999997},"min":{"1min":0.57699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.7240000000000002,"5min":3.7240000000000002,"15min":3.7240000000000002},"last":0.75},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.80600000000000005,"15min":0.75800000000000001},"min":{"1min":0.53200000000000003,"5min":0.439,"15min":0.439},"max":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.67000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.022,"5min":1.117,"15min":1.117},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99099999999999999,"5min":1.016,"15min":1.016},"last":0.61199999999999999}]},{"osd":9,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.86799999999999999,"15min":0.83799999999999997},"min":{"1min":0.70699999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":4.5579999999999998,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.91500000000000004,"15min":0.90000000000000002},"min":{"1min":0.67400000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.5060000000000002,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":0.96699999999999997}]},{"osd":10,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.84699999999999998,"15min":0.83099999999999996},"min":{"1min":0.66700000000000004,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1160000000000001,"5min":1.246,"15min":1.246},"last":0.997},{"interface":"front","average":{"1min":1,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.71999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0640000000000001}]},{"osd":11,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.86499999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":458943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.49399999999999999,"15min":0.42199999999999999},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.495,"15min":0.442},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.27900000000000003},"max":{"1min":0.96399999999999997,"5min":2.323,"15min":2.323},"last":0.497}]},{"osd":1,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1419999999999999,"5min":2.343,"15min":2.343},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70199999999999996,"15min":0.63800000000000001},"min":{"1min":0.50700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.9430000000000001,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.73799999999999999,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.72299999999999998},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2809999999999999,"5min":3.448,"15min":3.448},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.755,"15min":0.67400000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.048,"5min":3.359,"15min":3.359},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.749,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.84199999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75600000000000001,"15min":0.71999999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3120000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79000000000000004,"15min":0.74399999999999999},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.75,"15min":0.68500000000000005},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.627},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.76900000000000002,"15min":0.72899999999999998},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.2050000000000001,"5min":3.2050000000000001,"15min":3.2050000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.84599999999999997,"15min":0.79800000000000004},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.82999999999999996,"15min":0.78000000000000003},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.876,"15min":0.88500000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1259999999999999,"5min":1.546,"15min":1.546},"last":0.62},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91800000000000004,"15min":0.92100000000000004},"min":{"1min":0.61899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.163,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":0.98699999999999999,"15min":0.97899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.6299999999999999,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.85899999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.61399999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.004,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0860000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.79200000000000004}]},{"osd":11,"last update":"Fri Jul 14 16:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":0.75}]}]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":460320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.95599999999999996,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48499999999999999,"15min":0.437},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.46500000000000002},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79000000000000004,"5min":1.042,"15min":1.042},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.65100000000000002},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.1319999999999999,"5min":3.698,"15min":3.698},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72899999999999998,"15min":0.68700000000000006},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.873}]},{"osd":3,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93400000000000005,"5min":1.153,"15min":1.153},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.014,"5min":1.03,"15min":1.03},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.012,"5min":1.506,"15min":1.506},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73899999999999999,"15min":0.72199999999999998},"min":{"1min":0.621,"5min":0.435,"15min":0.435},"max":{"1min":0.95099999999999996,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57099999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.147,"5min":1.321,"15min":1.321},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.53800000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.012,"5min":1.038,"15min":1.038},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.752,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.107,"5min":1.129,"15min":1.129},"last":0.85699999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85999999999999999,"15min":0.84299999999999997},"min":{"1min":0.753,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79100000000000004,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.95699999999999996}]},{"osd":9,"last update":"Fri Jul 14 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90700000000000003,"15min":0.91900000000000004},"min":{"1min":0.72799999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.24,"5min":1.631,"15min":1.631},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.77100000000000002}]},{"osd":10,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.59099999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0840000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.88800000000000001,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0229999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Fri Jul 14 16:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.94499999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":1050781,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.39900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78600000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.68200000000000005,"15min":0.621},"min":{"1min":0.58599999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0189999999999999,"5min":1.099,"15min":1.099},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.69099999999999995,"15min":0.63600000000000001},"min":{"1min":0.55200000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51900000000000002,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86799999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.64400000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83199999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77700000000000002,"5min":7.056,"15min":7.056},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.752,"15min":0.70399999999999996},"min":{"1min":0.621,"5min":0.503,"15min":0.503},"max":{"1min":0.96499999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.76400000000000001,"15min":0.71399999999999997},"min":{"1min":0.70299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.0669999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73599999999999999,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":4.3739999999999997,"5min":7.109,"15min":7.109},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Jul 14 16:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.58099999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.82199999999999995,"15min":0.78800000000000003},"min":{"1min":0.65700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.94199999999999995}]},{"osd":8,"last update":"Fri Jul 14 16:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.1479999999999999,"5min":1.286,"15min":1.286},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97899999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Fri Jul 14 16:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89100000000000001,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.88400000000000001,"15min":0.872},"min":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.004}]},{"osd":10,"last update":"Fri Jul 14 16:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.86299999999999999,"15min":0.85799999999999998},"min":{"1min":0.67100000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0409999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.877},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.92300000000000004,"15min":0.91700000000000004},"min":{"1min":0.70499999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.105,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62599,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60080,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65159,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T16:15:45.825 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-14T16:15:45.825 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T16:15:46.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:45 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3443485695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:46.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.078+0000 7f9e76dff700 1 -- 172.21.15.62:0/90345221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70100cf0 msgr2=0x7f9e70101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.078+0000 7f9e76dff700 1 --2- 172.21.15.62:0/90345221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70100cf0 0x7f9e70101140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009990 tx=0x7f9e5c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.078+0000 7f9e76dff700 1 -- 172.21.15.62:0/90345221 shutdown_connections 2023-07-14T16:15:46.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.078+0000 7f9e76dff700 1 --2- 172.21.15.62:0/90345221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e70101680 0x7f9e70107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.078+0000 7f9e76dff700 1 --2- 172.21.15.62:0/90345221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e700ffb00 0x7f9e700fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.078+0000 7f9e76dff700 1 --2- 172.21.15.62:0/90345221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70100cf0 0x7f9e70101140 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.078+0000 7f9e76dff700 1 -- 172.21.15.62:0/90345221 >> 172.21.15.62:0/90345221 conn(0x7f9e700fb0e0 msgr2=0x7f9e700fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:46.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.079+0000 7f9e76dff700 1 -- 172.21.15.62:0/90345221 shutdown_connections 2023-07-14T16:15:46.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.079+0000 7f9e76dff700 1 -- 172.21.15.62:0/90345221 wait complete. 2023-07-14T16:15:46.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.080+0000 7f9e76dff700 1 Processor -- start 2023-07-14T16:15:46.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.080+0000 7f9e76dff700 1 -- start start 2023-07-14T16:15:46.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.080+0000 7f9e76dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e700ffb00 0x7f9e700742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:46.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.080+0000 7f9e76dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e70100cf0 0x7f9e70072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:46.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.080+0000 7f9e76dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70101680 0x7f9e70072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:46.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.080+0000 7f9e76dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700735e0 con 0x7f9e70101680 2023-07-14T16:15:46.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.080+0000 7f9e76dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70073750 con 0x7f9e70100cf0 2023-07-14T16:15:46.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.080+0000 7f9e76dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019f700 con 0x7f9e700ffb00 2023-07-14T16:15:46.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70101680 0x7f9e70072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:46.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70101680 0x7f9e70072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35066/0 (socket says 172.21.15.62:35066) 2023-07-14T16:15:46.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 -- 172.21.15.62:0/524011728 learned_addr learned my addr 172.21.15.62:0/524011728 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:46.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e74b9b700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e700ffb00 0x7f9e700742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:46.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e6ffff700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e70100cf0 0x7f9e70072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:46.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 -- 172.21.15.62:0/524011728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e700ffb00 msgr2=0x7f9e700742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e700ffb00 0x7f9e700742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 -- 172.21.15.62:0/524011728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e70100cf0 msgr2=0x7f9e70072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e70100cf0 0x7f9e70072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 -- 172.21.15.62:0/524011728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c0095f0 con 0x7f9e70101680 2023-07-14T16:15:46.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e7539c700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70101680 0x7f9e70072e40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800c900 tx=0x7f9e6800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:46.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e6dffb700 1 -- 172.21.15.62:0/524011728 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e680078d0 con 0x7f9e70101680 2023-07-14T16:15:46.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e6dffb700 1 -- 172.21.15.62:0/524011728 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e6800f450 con 0x7f9e70101680 2023-07-14T16:15:46.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.081+0000 7f9e6dffb700 1 -- 172.21.15.62:0/524011728 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68018660 con 0x7f9e70101680 2023-07-14T16:15:46.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.082+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7019f9e0 con 0x7f9e70101680 2023-07-14T16:15:46.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.082+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7019fea0 con 0x7f9e70101680 2023-07-14T16:15:46.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.083+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e70107070 con 0x7f9e70101680 2023-07-14T16:15:46.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.085+0000 7f9e6dffb700 1 -- 172.21.15.62:0/524011728 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9e6800f5c0 con 0x7f9e70101680 2023-07-14T16:15:46.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.087+0000 7f9e6dffb700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1220 0x7f9e600a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:46.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.088+0000 7f9e6dffb700 1 -- 172.21.15.62:0/524011728 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e680cccd0 con 0x7f9e70101680 2023-07-14T16:15:46.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.088+0000 7f9e74b9b700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1220 0x7f9e600a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:46.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.088+0000 7f9e74b9b700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1220 0x7f9e600a36d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009550 tx=0x7f9e640091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:46.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.090+0000 7f9e6dffb700 1 -- 172.21.15.62:0/524011728 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9e68096450 con 0x7f9e70101680 2023-07-14T16:15:46.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:45 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3443485695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:46.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:45 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3443485695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.293+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 --> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e70060950 con 0x7f9e600a1220 2023-07-14T16:15:46.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.304+0000 7f9e6dffb700 1 -- 172.21.15.62:0/524011728 <== mgr.14152 v2:172.21.15.62:6800/4048727837 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343728 (secure 0 0 0) 0x7f9e70060950 con 0x7f9e600a1220 2023-07-14T16:15:46.287 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:46.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1220 msgr2=0x7f9e600a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1220 0x7f9e600a36d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009550 tx=0x7f9e640091b0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70101680 msgr2=0x7f9e70072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70101680 0x7f9e70072e40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800c900 tx=0x7f9e6800cc10 comp rx=0 tx=0).stop 2023-07-14T16:15:46.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 shutdown_connections 2023-07-14T16:15:46.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e70101680 0x7f9e70072e40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e700ffb00 0x7f9e700742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e600a1220 0x7f9e600a36d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 --2- 172.21.15.62:0/524011728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e70100cf0 0x7f9e70072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 >> 172.21.15.62:0/524011728 conn(0x7f9e700fb0e0 msgr2=0x7f9e701041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:46.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 shutdown_connections 2023-07-14T16:15:46.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.309+0000 7f9e76dff700 1 -- 172.21.15.62:0/524011728 wait complete. 2023-07-14T16:15:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-07-14T16:15:46.304 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2023-07-14T16:15:45.495736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658256,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":757,"num_read_kb":491,"num_write":345,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335064,"kb_used_data":11640,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606760,"statfs":{"total":1151940427776,"available":1151597322240,"internally_reserved":0,"allocated":11919360,"data_stored":7465243,"data_compressed":112137,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5817,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":38,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002875"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509907+0000","last_change":"2023-07-14T16:13:59.301892+0000","last_active":"2023-07-14T16:15:19.509907+0000","last_peered":"2023-07-14T16:15:19.509907+0000","last_clean":"2023-07-14T16:15:19.509907+0000","last_became_active":"2023-07-14T16:13:59.301379+0000","last_became_peered":"2023-07-14T16:13:59.301379+0000","last_unstale":"2023-07-14T16:15:19.509907+0000","last_undegraded":"2023-07-14T16:15:19.509907+0000","last_fullsized":"2023-07-14T16:15:19.509907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:49:12.243022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502665+0000","last_change":"2023-07-14T16:13:59.309322+0000","last_active":"2023-07-14T16:15:19.502665+0000","last_peered":"2023-07-14T16:15:19.502665+0000","last_clean":"2023-07-14T16:15:19.502665+0000","last_became_active":"2023-07-14T16:13:59.309035+0000","last_became_peered":"2023-07-14T16:13:59.309035+0000","last_unstale":"2023-07-14T16:15:19.502665+0000","last_undegraded":"2023-07-14T16:15:19.502665+0000","last_fullsized":"2023-07-14T16:15:19.502665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:46.013552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509601+0000","last_change":"2023-07-14T16:13:59.304371+0000","last_active":"2023-07-14T16:15:19.509601+0000","last_peered":"2023-07-14T16:15:19.509601+0000","last_clean":"2023-07-14T16:15:19.509601+0000","last_became_active":"2023-07-14T16:13:59.303965+0000","last_became_peered":"2023-07-14T16:13:59.303965+0000","last_unstale":"2023-07-14T16:15:19.509601+0000","last_undegraded":"2023-07-14T16:15:19.509601+0000","last_fullsized":"2023-07-14T16:15:19.509601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:21:19.938384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.494491+0000","last_change":"2023-07-14T16:14:20.670094+0000","last_active":"2023-07-14T16:15:19.494491+0000","last_peered":"2023-07-14T16:15:19.494491+0000","last_clean":"2023-07-14T16:15:19.494491+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:15:19.494491+0000","last_undegraded":"2023-07-14T16:15:19.494491+0000","last_fullsized":"2023-07-14T16:15:19.494491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:32.039209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499662+0000","last_change":"2023-07-14T16:13:59.298094+0000","last_active":"2023-07-14T16:15:19.499662+0000","last_peered":"2023-07-14T16:15:19.499662+0000","last_clean":"2023-07-14T16:15:19.499662+0000","last_became_active":"2023-07-14T16:13:59.297764+0000","last_became_peered":"2023-07-14T16:13:59.297764+0000","last_unstale":"2023-07-14T16:15:19.499662+0000","last_undegraded":"2023-07-14T16:15:19.499662+0000","last_fullsized":"2023-07-14T16:15:19.499662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:40:48.501912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503393+0000","last_change":"2023-07-14T16:13:59.299735+0000","last_active":"2023-07-14T16:15:19.503393+0000","last_peered":"2023-07-14T16:15:19.503393+0000","last_clean":"2023-07-14T16:15:19.503393+0000","last_became_active":"2023-07-14T16:13:59.299413+0000","last_became_peered":"2023-07-14T16:13:59.299413+0000","last_unstale":"2023-07-14T16:15:19.503393+0000","last_undegraded":"2023-07-14T16:15:19.503393+0000","last_fullsized":"2023-07-14T16:15:19.503393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:09:10.174873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.493151+0000","last_change":"2023-07-14T16:13:59.299682+0000","last_active":"2023-07-14T16:15:19.493151+0000","last_peered":"2023-07-14T16:15:19.493151+0000","last_clean":"2023-07-14T16:15:19.493151+0000","last_became_active":"2023-07-14T16:13:59.299327+0000","last_became_peered":"2023-07-14T16:13:59.299327+0000","last_unstale":"2023-07-14T16:15:19.493151+0000","last_undegraded":"2023-07-14T16:15:19.493151+0000","last_fullsized":"2023-07-14T16:15:19.493151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:30.735405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502074+0000","last_change":"2023-07-14T16:13:59.311818+0000","last_active":"2023-07-14T16:15:19.502074+0000","last_peered":"2023-07-14T16:15:19.502074+0000","last_clean":"2023-07-14T16:15:19.502074+0000","last_became_active":"2023-07-14T16:13:59.311478+0000","last_became_peered":"2023-07-14T16:13:59.311478+0000","last_unstale":"2023-07-14T16:15:19.502074+0000","last_undegraded":"2023-07-14T16:15:19.502074+0000","last_fullsized":"2023-07-14T16:15:19.502074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:51.107741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507313+0000","last_change":"2023-07-14T16:13:59.309665+0000","last_active":"2023-07-14T16:15:19.507313+0000","last_peered":"2023-07-14T16:15:19.507313+0000","last_clean":"2023-07-14T16:15:19.507313+0000","last_became_active":"2023-07-14T16:13:59.309361+0000","last_became_peered":"2023-07-14T16:13:59.309361+0000","last_unstale":"2023-07-14T16:15:19.507313+0000","last_undegraded":"2023-07-14T16:15:19.507313+0000","last_fullsized":"2023-07-14T16:15:19.507313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:56:14.297747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508327+0000","last_change":"2023-07-14T16:13:59.296397+0000","last_active":"2023-07-14T16:15:19.508327+0000","last_peered":"2023-07-14T16:15:19.508327+0000","last_clean":"2023-07-14T16:15:19.508327+0000","last_became_active":"2023-07-14T16:13:59.295965+0000","last_became_peered":"2023-07-14T16:13:59.295965+0000","last_unstale":"2023-07-14T16:15:19.508327+0000","last_undegraded":"2023-07-14T16:15:19.508327+0000","last_fullsized":"2023-07-14T16:15:19.508327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:51.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.511367+0000","last_change":"2023-07-14T16:13:59.302892+0000","last_active":"2023-07-14T16:15:19.511367+0000","last_peered":"2023-07-14T16:15:19.511367+0000","last_clean":"2023-07-14T16:15:19.511367+0000","last_became_active":"2023-07-14T16:13:59.302295+0000","last_became_peered":"2023-07-14T16:13:59.302295+0000","last_unstale":"2023-07-14T16:15:19.511367+0000","last_undegraded":"2023-07-14T16:15:19.511367+0000","last_fullsized":"2023-07-14T16:15:19.511367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:11.009590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499348+0000","last_change":"2023-07-14T16:13:59.301913+0000","last_active":"2023-07-14T16:15:19.499348+0000","last_peered":"2023-07-14T16:15:19.499348+0000","last_clean":"2023-07-14T16:15:19.499348+0000","last_became_active":"2023-07-14T16:13:59.301590+0000","last_became_peered":"2023-07-14T16:13:59.301590+0000","last_unstale":"2023-07-14T16:15:19.499348+0000","last_undegraded":"2023-07-14T16:15:19.499348+0000","last_fullsized":"2023-07-14T16:15:19.499348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:09:55.206983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507029+0000","last_change":"2023-07-14T16:13:59.302006+0000","last_active":"2023-07-14T16:15:19.507029+0000","last_peered":"2023-07-14T16:15:19.507029+0000","last_clean":"2023-07-14T16:15:19.507029+0000","last_became_active":"2023-07-14T16:13:59.301752+0000","last_became_peered":"2023-07-14T16:13:59.301752+0000","last_unstale":"2023-07-14T16:15:19.507029+0000","last_undegraded":"2023-07-14T16:15:19.507029+0000","last_fullsized":"2023-07-14T16:15:19.507029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:08.923165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503265+0000","last_change":"2023-07-14T16:13:59.309318+0000","last_active":"2023-07-14T16:15:19.503265+0000","last_peered":"2023-07-14T16:15:19.503265+0000","last_clean":"2023-07-14T16:15:19.503265+0000","last_became_active":"2023-07-14T16:13:59.308931+0000","last_became_peered":"2023-07-14T16:13:59.308931+0000","last_unstale":"2023-07-14T16:15:19.503265+0000","last_undegraded":"2023-07-14T16:15:19.503265+0000","last_fullsized":"2023-07-14T16:15:19.503265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:21:02.747919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498473+0000","last_change":"2023-07-14T16:14:20.670208+0000","last_active":"2023-07-14T16:15:19.498473+0000","last_peered":"2023-07-14T16:15:19.498473+0000","last_clean":"2023-07-14T16:15:19.498473+0000","last_became_active":"2023-07-14T16:14:20.669742+0000","last_became_peered":"2023-07-14T16:14:20.669742+0000","last_unstale":"2023-07-14T16:15:19.498473+0000","last_undegraded":"2023-07-14T16:15:19.498473+0000","last_fullsized":"2023-07-14T16:15:19.498473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:53.265928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509331+0000","last_change":"2023-07-14T16:13:59.307986+0000","last_active":"2023-07-14T16:15:19.509331+0000","last_peered":"2023-07-14T16:15:19.509331+0000","last_clean":"2023-07-14T16:15:19.509331+0000","last_became_active":"2023-07-14T16:13:59.307701+0000","last_became_peered":"2023-07-14T16:13:59.307701+0000","last_unstale":"2023-07-14T16:15:19.509331+0000","last_undegraded":"2023-07-14T16:15:19.509331+0000","last_fullsized":"2023-07-14T16:15:19.509331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:56:03.104096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510111+0000","last_change":"2023-07-14T16:13:59.301685+0000","last_active":"2023-07-14T16:15:19.510111+0000","last_peered":"2023-07-14T16:15:19.510111+0000","last_clean":"2023-07-14T16:15:19.510111+0000","last_became_active":"2023-07-14T16:13:59.300905+0000","last_became_peered":"2023-07-14T16:13:59.300905+0000","last_unstale":"2023-07-14T16:15:19.510111+0000","last_undegraded":"2023-07-14T16:15:19.510111+0000","last_fullsized":"2023-07-14T16:15:19.510111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:48:24.721908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498700+0000","last_change":"2023-07-14T16:13:59.306027+0000","last_active":"2023-07-14T16:15:19.498700+0000","last_peered":"2023-07-14T16:15:19.498700+0000","last_clean":"2023-07-14T16:15:19.498700+0000","last_became_active":"2023-07-14T16:13:59.305718+0000","last_became_peered":"2023-07-14T16:13:59.305718+0000","last_unstale":"2023-07-14T16:15:19.498700+0000","last_undegraded":"2023-07-14T16:15:19.498700+0000","last_fullsized":"2023-07-14T16:15:19.498700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:20:47.951543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.494287+0000","last_change":"2023-07-14T16:13:59.294140+0000","last_active":"2023-07-14T16:15:19.494287+0000","last_peered":"2023-07-14T16:15:19.494287+0000","last_clean":"2023-07-14T16:15:19.494287+0000","last_became_active":"2023-07-14T16:13:59.293825+0000","last_became_peered":"2023-07-14T16:13:59.293825+0000","last_unstale":"2023-07-14T16:15:19.494287+0000","last_undegraded":"2023-07-14T16:15:19.494287+0000","last_fullsized":"2023-07-14T16:15:19.494287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:38:59.348563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665523+0000","last_change":"2023-07-14T16:13:58.277154+0000","last_active":"2023-07-14T16:14:20.665523+0000","last_peered":"2023-07-14T16:14:20.665523+0000","last_clean":"2023-07-14T16:14:20.665523+0000","last_became_active":"2023-07-14T16:13:58.276800+0000","last_became_peered":"2023-07-14T16:13:58.276800+0000","last_unstale":"2023-07-14T16:14:20.665523+0000","last_undegraded":"2023-07-14T16:14:20.665523+0000","last_fullsized":"2023-07-14T16:14:20.665523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:17:45.631137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.488178+0000","last_change":"2023-07-14T16:13:59.299819+0000","last_active":"2023-07-14T16:15:19.488178+0000","last_peered":"2023-07-14T16:15:19.488178+0000","last_clean":"2023-07-14T16:15:19.488178+0000","last_became_active":"2023-07-14T16:13:59.299294+0000","last_became_peered":"2023-07-14T16:13:59.299294+0000","last_unstale":"2023-07-14T16:15:19.488178+0000","last_undegraded":"2023-07-14T16:15:19.488178+0000","last_fullsized":"2023-07-14T16:15:19.488178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:43.461827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664040+0000","last_change":"2023-07-14T16:13:58.278954+0000","last_active":"2023-07-14T16:14:20.664040+0000","last_peered":"2023-07-14T16:14:20.664040+0000","last_clean":"2023-07-14T16:14:20.664040+0000","last_became_active":"2023-07-14T16:13:58.278454+0000","last_became_peered":"2023-07-14T16:13:58.278454+0000","last_unstale":"2023-07-14T16:14:20.664040+0000","last_undegraded":"2023-07-14T16:14:20.664040+0000","last_fullsized":"2023-07-14T16:14:20.664040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:33:34.015342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510065+0000","last_change":"2023-07-14T16:13:59.299877+0000","last_active":"2023-07-14T16:15:19.510065+0000","last_peered":"2023-07-14T16:15:19.510065+0000","last_clean":"2023-07-14T16:15:19.510065+0000","last_became_active":"2023-07-14T16:13:59.299557+0000","last_became_peered":"2023-07-14T16:13:59.299557+0000","last_unstale":"2023-07-14T16:15:19.510065+0000","last_undegraded":"2023-07-14T16:15:19.510065+0000","last_fullsized":"2023-07-14T16:15:19.510065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:46:45.331826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.677912+0000","last_change":"2023-07-14T16:14:20.677912+0000","last_active":"2023-07-14T16:14:20.677912+0000","last_peered":"2023-07-14T16:14:20.677912+0000","last_clean":"2023-07-14T16:14:20.677912+0000","last_became_active":"2023-07-14T16:14:20.668156+0000","last_became_peered":"2023-07-14T16:14:20.668156+0000","last_unstale":"2023-07-14T16:14:20.677912+0000","last_undegraded":"2023-07-14T16:14:20.677912+0000","last_fullsized":"2023-07-14T16:14:20.677912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:51:50.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509983+0000","last_change":"2023-07-14T16:13:59.307987+0000","last_active":"2023-07-14T16:15:19.509983+0000","last_peered":"2023-07-14T16:15:19.509983+0000","last_clean":"2023-07-14T16:15:19.509983+0000","last_became_active":"2023-07-14T16:13:59.307457+0000","last_became_peered":"2023-07-14T16:13:59.307457+0000","last_unstale":"2023-07-14T16:15:19.509983+0000","last_undegraded":"2023-07-14T16:15:19.509983+0000","last_fullsized":"2023-07-14T16:15:19.509983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:32:28.569384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663585+0000","last_change":"2023-07-14T16:13:58.282332+0000","last_active":"2023-07-14T16:14:20.663585+0000","last_peered":"2023-07-14T16:14:20.663585+0000","last_clean":"2023-07-14T16:14:20.663585+0000","last_became_active":"2023-07-14T16:13:58.278103+0000","last_became_peered":"2023-07-14T16:13:58.278103+0000","last_unstale":"2023-07-14T16:14:20.663585+0000","last_undegraded":"2023-07-14T16:14:20.663585+0000","last_fullsized":"2023-07-14T16:14:20.663585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:24:00.081710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.509816+0000","last_change":"2023-07-14T16:13:59.301036+0000","last_active":"2023-07-14T16:15:19.509816+0000","last_peered":"2023-07-14T16:15:19.509816+0000","last_clean":"2023-07-14T16:15:19.509816+0000","last_became_active":"2023-07-14T16:13:59.300787+0000","last_became_peered":"2023-07-14T16:13:59.300787+0000","last_unstale":"2023-07-14T16:15:19.509816+0000","last_undegraded":"2023-07-14T16:15:19.509816+0000","last_fullsized":"2023-07-14T16:15:19.509816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:40:56.654793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":127,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:41.719917+0000","last_change":"2023-07-14T16:13:58.282392+0000","last_active":"2023-07-14T16:15:41.719917+0000","last_peered":"2023-07-14T16:15:41.719917+0000","last_clean":"2023-07-14T16:15:41.719917+0000","last_became_active":"2023-07-14T16:13:58.282030+0000","last_became_peered":"2023-07-14T16:13:58.282030+0000","last_unstale":"2023-07-14T16:15:41.719917+0000","last_undegraded":"2023-07-14T16:15:41.719917+0000","last_fullsized":"2023-07-14T16:15:41.719917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:23:17.795386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.495045+0000","last_change":"2023-07-14T16:13:59.292681+0000","last_active":"2023-07-14T16:15:19.495045+0000","last_peered":"2023-07-14T16:15:19.495045+0000","last_clean":"2023-07-14T16:15:19.495045+0000","last_became_active":"2023-07-14T16:13:59.292289+0000","last_became_peered":"2023-07-14T16:13:59.292289+0000","last_unstale":"2023-07-14T16:15:19.495045+0000","last_undegraded":"2023-07-14T16:15:19.495045+0000","last_fullsized":"2023-07-14T16:15:19.495045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:33.971255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.666548+0000","last_change":"2023-07-14T16:13:58.282111+0000","last_active":"2023-07-14T16:14:20.666548+0000","last_peered":"2023-07-14T16:14:20.666548+0000","last_clean":"2023-07-14T16:14:20.666548+0000","last_became_active":"2023-07-14T16:13:58.281783+0000","last_became_peered":"2023-07-14T16:13:58.281783+0000","last_unstale":"2023-07-14T16:14:20.666548+0000","last_undegraded":"2023-07-14T16:14:20.666548+0000","last_fullsized":"2023-07-14T16:14:20.666548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:51:40.718143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501115+0000","last_change":"2023-07-14T16:13:59.297439+0000","last_active":"2023-07-14T16:15:19.501115+0000","last_peered":"2023-07-14T16:15:19.501115+0000","last_clean":"2023-07-14T16:15:19.501115+0000","last_became_active":"2023-07-14T16:13:59.297117+0000","last_became_peered":"2023-07-14T16:13:59.297117+0000","last_unstale":"2023-07-14T16:15:19.501115+0000","last_undegraded":"2023-07-14T16:15:19.501115+0000","last_fullsized":"2023-07-14T16:15:19.501115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:08.122607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763285+0000","last_change":"2023-07-14T16:13:58.274826+0000","last_active":"2023-07-14T16:14:20.763285+0000","last_peered":"2023-07-14T16:14:20.763285+0000","last_clean":"2023-07-14T16:14:20.763285+0000","last_became_active":"2023-07-14T16:13:58.274406+0000","last_became_peered":"2023-07-14T16:13:58.274406+0000","last_unstale":"2023-07-14T16:14:20.763285+0000","last_undegraded":"2023-07-14T16:14:20.763285+0000","last_fullsized":"2023-07-14T16:14:20.763285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:30:09.422218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500875+0000","last_change":"2023-07-14T16:13:59.296730+0000","last_active":"2023-07-14T16:15:19.500875+0000","last_peered":"2023-07-14T16:15:19.500875+0000","last_clean":"2023-07-14T16:15:19.500875+0000","last_became_active":"2023-07-14T16:13:59.296372+0000","last_became_peered":"2023-07-14T16:13:59.296372+0000","last_unstale":"2023-07-14T16:15:19.500875+0000","last_undegraded":"2023-07-14T16:15:19.500875+0000","last_fullsized":"2023-07-14T16:15:19.500875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:07.809329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664772+0000","last_change":"2023-07-14T16:13:58.269122+0000","last_active":"2023-07-14T16:14:20.664772+0000","last_peered":"2023-07-14T16:14:20.664772+0000","last_clean":"2023-07-14T16:14:20.664772+0000","last_became_active":"2023-07-14T16:13:58.268770+0000","last_became_peered":"2023-07-14T16:13:58.268770+0000","last_unstale":"2023-07-14T16:14:20.664772+0000","last_undegraded":"2023-07-14T16:14:20.664772+0000","last_fullsized":"2023-07-14T16:14:20.664772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:55:43.584997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.506513+0000","last_change":"2023-07-14T16:13:59.306987+0000","last_active":"2023-07-14T16:15:19.506513+0000","last_peered":"2023-07-14T16:15:19.506513+0000","last_clean":"2023-07-14T16:15:19.506513+0000","last_became_active":"2023-07-14T16:13:59.306649+0000","last_became_peered":"2023-07-14T16:13:59.306649+0000","last_unstale":"2023-07-14T16:15:19.506513+0000","last_undegraded":"2023-07-14T16:15:19.506513+0000","last_fullsized":"2023-07-14T16:15:19.506513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:35:23.579814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:41.719379+0000","last_change":"2023-07-14T16:13:58.281319+0000","last_active":"2023-07-14T16:15:41.719379+0000","last_peered":"2023-07-14T16:15:41.719379+0000","last_clean":"2023-07-14T16:15:41.719379+0000","last_became_active":"2023-07-14T16:13:58.280928+0000","last_became_peered":"2023-07-14T16:13:58.280928+0000","last_unstale":"2023-07-14T16:15:41.719379+0000","last_undegraded":"2023-07-14T16:15:41.719379+0000","last_fullsized":"2023-07-14T16:15:41.719379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:57:41.337954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.507694+0000","last_change":"2023-07-14T16:13:59.309586+0000","last_active":"2023-07-14T16:15:19.507694+0000","last_peered":"2023-07-14T16:15:19.507694+0000","last_clean":"2023-07-14T16:15:19.507694+0000","last_became_active":"2023-07-14T16:13:59.309243+0000","last_became_peered":"2023-07-14T16:13:59.309243+0000","last_unstale":"2023-07-14T16:15:19.507694+0000","last_undegraded":"2023-07-14T16:15:19.507694+0000","last_fullsized":"2023-07-14T16:15:19.507694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:41:30.700680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661328+0000","last_change":"2023-07-14T16:13:58.279769+0000","last_active":"2023-07-14T16:14:20.661328+0000","last_peered":"2023-07-14T16:14:20.661328+0000","last_clean":"2023-07-14T16:14:20.661328+0000","last_became_active":"2023-07-14T16:13:58.279409+0000","last_became_peered":"2023-07-14T16:13:58.279409+0000","last_unstale":"2023-07-14T16:14:20.661328+0000","last_undegraded":"2023-07-14T16:14:20.661328+0000","last_fullsized":"2023-07-14T16:14:20.661328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:44:32.751188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500584+0000","last_change":"2023-07-14T16:13:59.300382+0000","last_active":"2023-07-14T16:15:19.500584+0000","last_peered":"2023-07-14T16:15:19.500584+0000","last_clean":"2023-07-14T16:15:19.500584+0000","last_became_active":"2023-07-14T16:13:59.299981+0000","last_became_peered":"2023-07-14T16:13:59.299981+0000","last_unstale":"2023-07-14T16:15:19.500584+0000","last_undegraded":"2023-07-14T16:15:19.500584+0000","last_fullsized":"2023-07-14T16:15:19.500584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:46:04.769609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665031+0000","last_change":"2023-07-14T16:13:58.274128+0000","last_active":"2023-07-14T16:14:20.665031+0000","last_peered":"2023-07-14T16:14:20.665031+0000","last_clean":"2023-07-14T16:14:20.665031+0000","last_became_active":"2023-07-14T16:13:58.273446+0000","last_became_peered":"2023-07-14T16:13:58.273446+0000","last_unstale":"2023-07-14T16:14:20.665031+0000","last_undegraded":"2023-07-14T16:14:20.665031+0000","last_fullsized":"2023-07-14T16:14:20.665031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:12.510793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510736+0000","last_change":"2023-07-14T16:13:59.301551+0000","last_active":"2023-07-14T16:15:19.510736+0000","last_peered":"2023-07-14T16:15:19.510736+0000","last_clean":"2023-07-14T16:15:19.510736+0000","last_became_active":"2023-07-14T16:13:59.301206+0000","last_became_peered":"2023-07-14T16:13:59.301206+0000","last_unstale":"2023-07-14T16:15:19.510736+0000","last_undegraded":"2023-07-14T16:15:19.510736+0000","last_fullsized":"2023-07-14T16:15:19.510736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:35:16.227849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":259,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:42.721000+0000","last_change":"2023-07-14T16:13:58.283304+0000","last_active":"2023-07-14T16:15:42.721000+0000","last_peered":"2023-07-14T16:15:42.721000+0000","last_clean":"2023-07-14T16:15:42.721000+0000","last_became_active":"2023-07-14T16:13:58.282930+0000","last_became_peered":"2023-07-14T16:13:58.282930+0000","last_unstale":"2023-07-14T16:15:42.721000+0000","last_undegraded":"2023-07-14T16:15:42.721000+0000","last_fullsized":"2023-07-14T16:15:42.721000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:41:02.538305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62599,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":209,"num_read_kb":161,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500725+0000","last_change":"2023-07-14T16:13:59.304144+0000","last_active":"2023-07-14T16:15:19.500725+0000","last_peered":"2023-07-14T16:15:19.500725+0000","last_clean":"2023-07-14T16:15:19.500725+0000","last_became_active":"2023-07-14T16:13:59.303745+0000","last_became_peered":"2023-07-14T16:13:59.303745+0000","last_unstale":"2023-07-14T16:15:19.500725+0000","last_undegraded":"2023-07-14T16:15:19.500725+0000","last_fullsized":"2023-07-14T16:15:19.500725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:20.828221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667830+0000","last_change":"2023-07-14T16:13:58.283734+0000","last_active":"2023-07-14T16:14:20.667830+0000","last_peered":"2023-07-14T16:14:20.667830+0000","last_clean":"2023-07-14T16:14:20.667830+0000","last_became_active":"2023-07-14T16:13:58.282765+0000","last_became_peered":"2023-07-14T16:13:58.282765+0000","last_unstale":"2023-07-14T16:14:20.667830+0000","last_undegraded":"2023-07-14T16:14:20.667830+0000","last_fullsized":"2023-07-14T16:14:20.667830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:59:13.217664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500485+0000","last_change":"2023-07-14T16:13:59.308445+0000","last_active":"2023-07-14T16:15:19.500485+0000","last_peered":"2023-07-14T16:15:19.500485+0000","last_clean":"2023-07-14T16:15:19.500485+0000","last_became_active":"2023-07-14T16:13:59.308130+0000","last_became_peered":"2023-07-14T16:13:59.308130+0000","last_unstale":"2023-07-14T16:15:19.500485+0000","last_undegraded":"2023-07-14T16:15:19.500485+0000","last_fullsized":"2023-07-14T16:15:19.500485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:45.004512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664202+0000","last_change":"2023-07-14T16:13:58.273973+0000","last_active":"2023-07-14T16:14:20.664202+0000","last_peered":"2023-07-14T16:14:20.664202+0000","last_clean":"2023-07-14T16:14:20.664202+0000","last_became_active":"2023-07-14T16:13:58.273606+0000","last_became_peered":"2023-07-14T16:13:58.273606+0000","last_unstale":"2023-07-14T16:14:20.664202+0000","last_undegraded":"2023-07-14T16:14:20.664202+0000","last_fullsized":"2023-07-14T16:14:20.664202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:12.304491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501694+0000","last_change":"2023-07-14T16:13:59.306428+0000","last_active":"2023-07-14T16:15:19.501694+0000","last_peered":"2023-07-14T16:15:19.501694+0000","last_clean":"2023-07-14T16:15:19.501694+0000","last_became_active":"2023-07-14T16:13:59.306090+0000","last_became_peered":"2023-07-14T16:13:59.306090+0000","last_unstale":"2023-07-14T16:15:19.501694+0000","last_undegraded":"2023-07-14T16:15:19.501694+0000","last_fullsized":"2023-07-14T16:15:19.501694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:27:16.033202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":130,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:43.799085+0000","last_change":"2023-07-14T16:13:58.277123+0000","last_active":"2023-07-14T16:15:43.799085+0000","last_peered":"2023-07-14T16:15:43.799085+0000","last_clean":"2023-07-14T16:15:43.799085+0000","last_became_active":"2023-07-14T16:13:58.276815+0000","last_became_peered":"2023-07-14T16:13:58.276815+0000","last_unstale":"2023-07-14T16:15:43.799085+0000","last_undegraded":"2023-07-14T16:15:43.799085+0000","last_fullsized":"2023-07-14T16:15:43.799085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:19:41.019233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1570,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504574+0000","last_change":"2023-07-14T16:13:59.310238+0000","last_active":"2023-07-14T16:15:19.504574+0000","last_peered":"2023-07-14T16:15:19.504574+0000","last_clean":"2023-07-14T16:15:19.504574+0000","last_became_active":"2023-07-14T16:13:59.309918+0000","last_became_peered":"2023-07-14T16:13:59.309918+0000","last_unstale":"2023-07-14T16:15:19.504574+0000","last_undegraded":"2023-07-14T16:15:19.504574+0000","last_fullsized":"2023-07-14T16:15:19.504574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:14:25.020575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663660+0000","last_change":"2023-07-14T16:13:58.280834+0000","last_active":"2023-07-14T16:14:20.663660+0000","last_peered":"2023-07-14T16:14:20.663660+0000","last_clean":"2023-07-14T16:14:20.663660+0000","last_became_active":"2023-07-14T16:13:58.280497+0000","last_became_peered":"2023-07-14T16:13:58.280497+0000","last_unstale":"2023-07-14T16:14:20.663660+0000","last_undegraded":"2023-07-14T16:14:20.663660+0000","last_fullsized":"2023-07-14T16:14:20.663660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:07:02.308388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.489632+0000","last_change":"2023-07-14T16:14:20.764846+0000","last_active":"2023-07-14T16:15:19.489632+0000","last_peered":"2023-07-14T16:15:19.489632+0000","last_clean":"2023-07-14T16:15:19.489632+0000","last_became_active":"2023-07-14T16:14:20.764485+0000","last_became_peered":"2023-07-14T16:14:20.764485+0000","last_unstale":"2023-07-14T16:15:19.489632+0000","last_undegraded":"2023-07-14T16:15:19.489632+0000","last_fullsized":"2023-07-14T16:15:19.489632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:21:29.245874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.682843+0000","last_change":"2023-07-14T16:14:20.682331+0000","last_active":"2023-07-14T16:14:20.682843+0000","last_peered":"2023-07-14T16:14:20.682843+0000","last_clean":"2023-07-14T16:14:20.682843+0000","last_became_active":"2023-07-14T16:14:20.669179+0000","last_became_peered":"2023-07-14T16:14:20.669179+0000","last_unstale":"2023-07-14T16:14:20.682843+0000","last_undegraded":"2023-07-14T16:14:20.682843+0000","last_fullsized":"2023-07-14T16:14:20.682843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:55:10.629011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502602+0000","last_change":"2023-07-14T16:13:59.297644+0000","last_active":"2023-07-14T16:15:19.502602+0000","last_peered":"2023-07-14T16:15:19.502602+0000","last_clean":"2023-07-14T16:15:19.502602+0000","last_became_active":"2023-07-14T16:13:59.297332+0000","last_became_peered":"2023-07-14T16:13:59.297332+0000","last_unstale":"2023-07-14T16:15:19.502602+0000","last_undegraded":"2023-07-14T16:15:19.502602+0000","last_fullsized":"2023-07-14T16:15:19.502602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:36.421075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665695+0000","last_change":"2023-07-14T16:13:58.280632+0000","last_active":"2023-07-14T16:14:20.665695+0000","last_peered":"2023-07-14T16:14:20.665695+0000","last_clean":"2023-07-14T16:14:20.665695+0000","last_became_active":"2023-07-14T16:13:58.280296+0000","last_became_peered":"2023-07-14T16:13:58.280296+0000","last_unstale":"2023-07-14T16:14:20.665695+0000","last_undegraded":"2023-07-14T16:14:20.665695+0000","last_fullsized":"2023-07-14T16:14:20.665695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:13:26.897678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.505470+0000","last_change":"2023-07-14T16:13:59.297092+0000","last_active":"2023-07-14T16:15:19.505470+0000","last_peered":"2023-07-14T16:15:19.505470+0000","last_clean":"2023-07-14T16:15:19.505470+0000","last_became_active":"2023-07-14T16:13:59.296750+0000","last_became_peered":"2023-07-14T16:13:59.296750+0000","last_unstale":"2023-07-14T16:15:19.505470+0000","last_undegraded":"2023-07-14T16:15:19.505470+0000","last_fullsized":"2023-07-14T16:15:19.505470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:13:32.797912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.660888+0000","last_change":"2023-07-14T16:13:58.281786+0000","last_active":"2023-07-14T16:14:20.660888+0000","last_peered":"2023-07-14T16:14:20.660888+0000","last_clean":"2023-07-14T16:14:20.660888+0000","last_became_active":"2023-07-14T16:13:58.281471+0000","last_became_peered":"2023-07-14T16:13:58.281471+0000","last_unstale":"2023-07-14T16:14:20.660888+0000","last_undegraded":"2023-07-14T16:14:20.660888+0000","last_fullsized":"2023-07-14T16:14:20.660888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:27:09.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502616+0000","last_change":"2023-07-14T16:14:20.672813+0000","last_active":"2023-07-14T16:15:19.502616+0000","last_peered":"2023-07-14T16:15:19.502616+0000","last_clean":"2023-07-14T16:15:19.502616+0000","last_became_active":"2023-07-14T16:14:20.672447+0000","last_became_peered":"2023-07-14T16:14:20.672447+0000","last_unstale":"2023-07-14T16:15:19.502616+0000","last_undegraded":"2023-07-14T16:15:19.502616+0000","last_fullsized":"2023-07-14T16:15:19.502616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:33:07.582055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.495526+0000","last_change":"2023-07-14T16:13:59.288836+0000","last_active":"2023-07-14T16:15:19.495526+0000","last_peered":"2023-07-14T16:15:19.495526+0000","last_clean":"2023-07-14T16:15:19.495526+0000","last_became_active":"2023-07-14T16:13:59.288517+0000","last_became_peered":"2023-07-14T16:13:59.288517+0000","last_unstale":"2023-07-14T16:15:19.495526+0000","last_undegraded":"2023-07-14T16:15:19.495526+0000","last_fullsized":"2023-07-14T16:15:19.495526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:58:48.293313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":124,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:42.796187+0000","last_change":"2023-07-14T16:13:58.282415+0000","last_active":"2023-07-14T16:15:42.796187+0000","last_peered":"2023-07-14T16:15:42.796187+0000","last_clean":"2023-07-14T16:15:42.796187+0000","last_became_active":"2023-07-14T16:13:58.282118+0000","last_became_peered":"2023-07-14T16:13:58.282118+0000","last_unstale":"2023-07-14T16:15:42.796187+0000","last_undegraded":"2023-07-14T16:15:42.796187+0000","last_fullsized":"2023-07-14T16:15:42.796187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:52:41.532341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501135+0000","last_change":"2023-07-14T16:13:59.302860+0000","last_active":"2023-07-14T16:15:19.501135+0000","last_peered":"2023-07-14T16:15:19.501135+0000","last_clean":"2023-07-14T16:15:19.501135+0000","last_became_active":"2023-07-14T16:13:59.302454+0000","last_became_peered":"2023-07-14T16:13:59.302454+0000","last_unstale":"2023-07-14T16:15:19.501135+0000","last_undegraded":"2023-07-14T16:15:19.501135+0000","last_fullsized":"2023-07-14T16:15:19.501135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:34.340132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664397+0000","last_change":"2023-07-14T16:13:58.273908+0000","last_active":"2023-07-14T16:14:20.664397+0000","last_peered":"2023-07-14T16:14:20.664397+0000","last_clean":"2023-07-14T16:14:20.664397+0000","last_became_active":"2023-07-14T16:13:58.273497+0000","last_became_peered":"2023-07-14T16:13:58.273497+0000","last_unstale":"2023-07-14T16:14:20.664397+0000","last_undegraded":"2023-07-14T16:14:20.664397+0000","last_fullsized":"2023-07-14T16:14:20.664397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:55:44.978027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.661044+0000","last_change":"2023-07-14T16:12:36.546475+0000","last_active":"2023-07-14T16:14:20.661044+0000","last_peered":"2023-07-14T16:14:20.661044+0000","last_clean":"2023-07-14T16:14:20.661044+0000","last_became_active":"2023-07-14T16:12:36.545943+0000","last_became_peered":"2023-07-14T16:12:36.545943+0000","last_unstale":"2023-07-14T16:14:20.661044+0000","last_undegraded":"2023-07-14T16:14:20.661044+0000","last_fullsized":"2023-07-14T16:14:20.661044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:08:58.024408+0000","last_clean_scrub_stamp":"2023-07-14T16:08:58.024408+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:27.166990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500653+0000","last_change":"2023-07-14T16:13:59.297669+0000","last_active":"2023-07-14T16:15:19.500653+0000","last_peered":"2023-07-14T16:15:19.500653+0000","last_clean":"2023-07-14T16:15:19.500653+0000","last_became_active":"2023-07-14T16:13:59.297364+0000","last_became_peered":"2023-07-14T16:13:59.297364+0000","last_unstale":"2023-07-14T16:15:19.500653+0000","last_undegraded":"2023-07-14T16:15:19.500653+0000","last_fullsized":"2023-07-14T16:15:19.500653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:50:00.972727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.673166+0000","last_change":"2023-07-14T16:14:20.673166+0000","last_active":"2023-07-14T16:14:20.673166+0000","last_peered":"2023-07-14T16:14:20.673166+0000","last_clean":"2023-07-14T16:14:20.673166+0000","last_became_active":"2023-07-14T16:14:20.667664+0000","last_became_peered":"2023-07-14T16:14:20.667664+0000","last_unstale":"2023-07-14T16:14:20.673166+0000","last_undegraded":"2023-07-14T16:14:20.673166+0000","last_fullsized":"2023-07-14T16:14:20.673166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T16:26:24.328768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664285+0000","last_change":"2023-07-14T16:13:58.276257+0000","last_active":"2023-07-14T16:14:20.664285+0000","last_peered":"2023-07-14T16:14:20.664285+0000","last_clean":"2023-07-14T16:14:20.664285+0000","last_became_active":"2023-07-14T16:13:58.275910+0000","last_became_peered":"2023-07-14T16:13:58.275910+0000","last_unstale":"2023-07-14T16:14:20.664285+0000","last_undegraded":"2023-07-14T16:14:20.664285+0000","last_fullsized":"2023-07-14T16:14:20.664285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:33:47.700225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.498722+0000","last_change":"2023-07-14T16:13:59.299135+0000","last_active":"2023-07-14T16:15:19.498722+0000","last_peered":"2023-07-14T16:15:19.498722+0000","last_clean":"2023-07-14T16:15:19.498722+0000","last_became_active":"2023-07-14T16:13:59.298031+0000","last_became_peered":"2023-07-14T16:13:59.298031+0000","last_unstale":"2023-07-14T16:15:19.498722+0000","last_undegraded":"2023-07-14T16:15:19.498722+0000","last_fullsized":"2023-07-14T16:15:19.498722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:48:21.882960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.665411+0000","last_change":"2023-07-14T16:13:58.279664+0000","last_active":"2023-07-14T16:14:20.665411+0000","last_peered":"2023-07-14T16:14:20.665411+0000","last_clean":"2023-07-14T16:14:20.665411+0000","last_became_active":"2023-07-14T16:13:58.279137+0000","last_became_peered":"2023-07-14T16:13:58.279137+0000","last_unstale":"2023-07-14T16:14:20.665411+0000","last_undegraded":"2023-07-14T16:14:20.665411+0000","last_fullsized":"2023-07-14T16:14:20.665411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:47:07.657781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500067+0000","last_change":"2023-07-14T16:13:59.305536+0000","last_active":"2023-07-14T16:15:19.500067+0000","last_peered":"2023-07-14T16:15:19.500067+0000","last_clean":"2023-07-14T16:15:19.500067+0000","last_became_active":"2023-07-14T16:13:59.305176+0000","last_became_peered":"2023-07-14T16:13:59.305176+0000","last_unstale":"2023-07-14T16:15:19.500067+0000","last_undegraded":"2023-07-14T16:15:19.500067+0000","last_fullsized":"2023-07-14T16:15:19.500067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:18:18.141957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664829+0000","last_change":"2023-07-14T16:13:58.269652+0000","last_active":"2023-07-14T16:14:20.664829+0000","last_peered":"2023-07-14T16:14:20.664829+0000","last_clean":"2023-07-14T16:14:20.664829+0000","last_became_active":"2023-07-14T16:13:58.269338+0000","last_became_peered":"2023-07-14T16:13:58.269338+0000","last_unstale":"2023-07-14T16:14:20.664829+0000","last_undegraded":"2023-07-14T16:14:20.664829+0000","last_fullsized":"2023-07-14T16:14:20.664829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:55:21.910971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502139+0000","last_change":"2023-07-14T16:13:59.297574+0000","last_active":"2023-07-14T16:15:19.502139+0000","last_peered":"2023-07-14T16:15:19.502139+0000","last_clean":"2023-07-14T16:15:19.502139+0000","last_became_active":"2023-07-14T16:13:59.297124+0000","last_became_peered":"2023-07-14T16:13:59.297124+0000","last_unstale":"2023-07-14T16:15:19.502139+0000","last_undegraded":"2023-07-14T16:15:19.502139+0000","last_fullsized":"2023-07-14T16:15:19.502139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:59:20.051395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763343+0000","last_change":"2023-07-14T16:13:58.278011+0000","last_active":"2023-07-14T16:14:20.763343+0000","last_peered":"2023-07-14T16:14:20.763343+0000","last_clean":"2023-07-14T16:14:20.763343+0000","last_became_active":"2023-07-14T16:13:58.277563+0000","last_became_peered":"2023-07-14T16:13:58.277563+0000","last_unstale":"2023-07-14T16:14:20.763343+0000","last_undegraded":"2023-07-14T16:14:20.763343+0000","last_fullsized":"2023-07-14T16:14:20.763343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:21:06.354928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.463529+0000","last_change":"2023-07-14T16:13:59.308836+0000","last_active":"2023-07-14T16:15:19.463529+0000","last_peered":"2023-07-14T16:15:19.463529+0000","last_clean":"2023-07-14T16:15:19.463529+0000","last_became_active":"2023-07-14T16:13:59.308510+0000","last_became_peered":"2023-07-14T16:13:59.308510+0000","last_unstale":"2023-07-14T16:15:19.463529+0000","last_undegraded":"2023-07-14T16:15:19.463529+0000","last_fullsized":"2023-07-14T16:15:19.463529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:48:07.973849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663352+0000","last_change":"2023-07-14T16:13:58.277438+0000","last_active":"2023-07-14T16:14:20.663352+0000","last_peered":"2023-07-14T16:14:20.663352+0000","last_clean":"2023-07-14T16:14:20.663352+0000","last_became_active":"2023-07-14T16:13:58.277114+0000","last_became_peered":"2023-07-14T16:13:58.277114+0000","last_unstale":"2023-07-14T16:14:20.663352+0000","last_undegraded":"2023-07-14T16:14:20.663352+0000","last_fullsized":"2023-07-14T16:14:20.663352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:22:40.636429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502829+0000","last_change":"2023-07-14T16:13:59.304491+0000","last_active":"2023-07-14T16:15:19.502829+0000","last_peered":"2023-07-14T16:15:19.502829+0000","last_clean":"2023-07-14T16:15:19.502829+0000","last_became_active":"2023-07-14T16:13:59.303925+0000","last_became_peered":"2023-07-14T16:13:59.303925+0000","last_unstale":"2023-07-14T16:15:19.502829+0000","last_undegraded":"2023-07-14T16:15:19.502829+0000","last_fullsized":"2023-07-14T16:15:19.502829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:39.410847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.685314+0000","last_change":"2023-07-14T16:14:20.684842+0000","last_active":"2023-07-14T16:14:20.685314+0000","last_peered":"2023-07-14T16:14:20.685314+0000","last_clean":"2023-07-14T16:14:20.685314+0000","last_became_active":"2023-07-14T16:14:20.670098+0000","last_became_peered":"2023-07-14T16:14:20.670098+0000","last_unstale":"2023-07-14T16:14:20.685314+0000","last_undegraded":"2023-07-14T16:14:20.685314+0000","last_fullsized":"2023-07-14T16:14:20.685314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:54:58.021152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502027+0000","last_change":"2023-07-14T16:13:59.310983+0000","last_active":"2023-07-14T16:15:19.502027+0000","last_peered":"2023-07-14T16:15:19.502027+0000","last_clean":"2023-07-14T16:15:19.502027+0000","last_became_active":"2023-07-14T16:13:59.310636+0000","last_became_peered":"2023-07-14T16:13:59.310636+0000","last_unstale":"2023-07-14T16:15:19.502027+0000","last_undegraded":"2023-07-14T16:15:19.502027+0000","last_fullsized":"2023-07-14T16:15:19.502027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:31.003517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.663068+0000","last_change":"2023-07-14T16:13:58.280153+0000","last_active":"2023-07-14T16:14:20.663068+0000","last_peered":"2023-07-14T16:14:20.663068+0000","last_clean":"2023-07-14T16:14:20.663068+0000","last_became_active":"2023-07-14T16:13:58.279771+0000","last_became_peered":"2023-07-14T16:13:58.279771+0000","last_unstale":"2023-07-14T16:14:20.663068+0000","last_undegraded":"2023-07-14T16:14:20.663068+0000","last_fullsized":"2023-07-14T16:14:20.663068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:53:46.290429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.510162+0000","last_change":"2023-07-14T16:13:59.304854+0000","last_active":"2023-07-14T16:15:19.510162+0000","last_peered":"2023-07-14T16:15:19.510162+0000","last_clean":"2023-07-14T16:15:19.510162+0000","last_became_active":"2023-07-14T16:13:59.304374+0000","last_became_peered":"2023-07-14T16:13:59.304374+0000","last_unstale":"2023-07-14T16:15:19.510162+0000","last_undegraded":"2023-07-14T16:15:19.510162+0000","last_fullsized":"2023-07-14T16:15:19.510162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:52.213737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.667834+0000","last_change":"2023-07-14T16:13:58.281885+0000","last_active":"2023-07-14T16:14:20.667834+0000","last_peered":"2023-07-14T16:14:20.667834+0000","last_clean":"2023-07-14T16:14:20.667834+0000","last_became_active":"2023-07-14T16:13:58.278759+0000","last_became_peered":"2023-07-14T16:13:58.278759+0000","last_unstale":"2023-07-14T16:14:20.667834+0000","last_undegraded":"2023-07-14T16:14:20.667834+0000","last_fullsized":"2023-07-14T16:14:20.667834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:06:11.816765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.511619+0000","last_change":"2023-07-14T16:13:59.302680+0000","last_active":"2023-07-14T16:15:19.511619+0000","last_peered":"2023-07-14T16:15:19.511619+0000","last_clean":"2023-07-14T16:15:19.511619+0000","last_became_active":"2023-07-14T16:13:59.301854+0000","last_became_peered":"2023-07-14T16:13:59.301854+0000","last_unstale":"2023-07-14T16:15:19.511619+0000","last_undegraded":"2023-07-14T16:15:19.511619+0000","last_fullsized":"2023-07-14T16:15:19.511619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:29:42.861394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.664860+0000","last_change":"2023-07-14T16:13:58.274254+0000","last_active":"2023-07-14T16:14:20.664860+0000","last_peered":"2023-07-14T16:14:20.664860+0000","last_clean":"2023-07-14T16:14:20.664860+0000","last_became_active":"2023-07-14T16:13:58.273933+0000","last_became_peered":"2023-07-14T16:13:58.273933+0000","last_unstale":"2023-07-14T16:14:20.664860+0000","last_undegraded":"2023-07-14T16:14:20.664860+0000","last_fullsized":"2023-07-14T16:14:20.664860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:54:53.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.501047+0000","last_change":"2023-07-14T16:13:59.305615+0000","last_active":"2023-07-14T16:15:19.501047+0000","last_peered":"2023-07-14T16:15:19.501047+0000","last_clean":"2023-07-14T16:15:19.501047+0000","last_became_active":"2023-07-14T16:13:59.305206+0000","last_became_peered":"2023-07-14T16:13:59.305206+0000","last_unstale":"2023-07-14T16:15:19.501047+0000","last_undegraded":"2023-07-14T16:15:19.501047+0000","last_fullsized":"2023-07-14T16:15:19.501047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T17:49:41.771061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:14:20.763463+0000","last_change":"2023-07-14T16:13:58.277976+0000","last_active":"2023-07-14T16:14:20.763463+0000","last_peered":"2023-07-14T16:14:20.763463+0000","last_clean":"2023-07-14T16:14:20.763463+0000","last_became_active":"2023-07-14T16:13:58.277618+0000","last_became_peered":"2023-07-14T16:13:58.277618+0000","last_unstale":"2023-07-14T16:14:20.763463+0000","last_undegraded":"2023-07-14T16:14:20.763463+0000","last_fullsized":"2023-07-14T16:14:20.763463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:57.233726+0000","last_clean_scrub_stamp":"2023-07-14T16:13:57.233726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:25.780591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504344+0000","last_change":"2023-07-14T16:13:59.302041+0000","last_active":"2023-07-14T16:15:19.504344+0000","last_peered":"2023-07-14T16:15:19.504344+0000","last_clean":"2023-07-14T16:15:19.504344+0000","last_became_active":"2023-07-14T16:13:59.301620+0000","last_became_peered":"2023-07-14T16:13:59.301620+0000","last_unstale":"2023-07-14T16:15:19.504344+0000","last_undegraded":"2023-07-14T16:15:19.504344+0000","last_fullsized":"2023-07-14T16:15:19.504344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:18.969064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502705+0000","last_change":"2023-07-14T16:13:59.304222+0000","last_active":"2023-07-14T16:15:19.502705+0000","last_peered":"2023-07-14T16:15:19.502705+0000","last_clean":"2023-07-14T16:15:19.502705+0000","last_became_active":"2023-07-14T16:13:59.303404+0000","last_became_peered":"2023-07-14T16:13:59.303404+0000","last_unstale":"2023-07-14T16:15:19.502705+0000","last_undegraded":"2023-07-14T16:15:19.502705+0000","last_fullsized":"2023-07-14T16:15:19.502705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:26.218664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.490011+0000","last_change":"2023-07-14T16:13:59.296433+0000","last_active":"2023-07-14T16:15:19.490011+0000","last_peered":"2023-07-14T16:15:19.490011+0000","last_clean":"2023-07-14T16:15:19.490011+0000","last_became_active":"2023-07-14T16:13:59.295604+0000","last_became_peered":"2023-07-14T16:13:59.295604+0000","last_unstale":"2023-07-14T16:15:19.490011+0000","last_undegraded":"2023-07-14T16:15:19.490011+0000","last_fullsized":"2023-07-14T16:15:19.490011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:17:57.269484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.493883+0000","last_change":"2023-07-14T16:13:59.294198+0000","last_active":"2023-07-14T16:15:19.493883+0000","last_peered":"2023-07-14T16:15:19.493883+0000","last_clean":"2023-07-14T16:15:19.493883+0000","last_became_active":"2023-07-14T16:13:59.293891+0000","last_became_peered":"2023-07-14T16:13:59.293891+0000","last_unstale":"2023-07-14T16:15:19.493883+0000","last_undegraded":"2023-07-14T16:15:19.493883+0000","last_fullsized":"2023-07-14T16:15:19.493883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:41.192604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503095+0000","last_change":"2023-07-14T16:13:59.298724+0000","last_active":"2023-07-14T16:15:19.503095+0000","last_peered":"2023-07-14T16:15:19.503095+0000","last_clean":"2023-07-14T16:15:19.503095+0000","last_became_active":"2023-07-14T16:13:59.298318+0000","last_became_peered":"2023-07-14T16:13:59.298318+0000","last_unstale":"2023-07-14T16:15:19.503095+0000","last_undegraded":"2023-07-14T16:15:19.503095+0000","last_fullsized":"2023-07-14T16:15:19.503095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:30:13.650949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503946+0000","last_change":"2023-07-14T16:13:59.294652+0000","last_active":"2023-07-14T16:15:19.503946+0000","last_peered":"2023-07-14T16:15:19.503946+0000","last_clean":"2023-07-14T16:15:19.503946+0000","last_became_active":"2023-07-14T16:13:59.294278+0000","last_became_peered":"2023-07-14T16:13:59.294278+0000","last_unstale":"2023-07-14T16:15:19.503946+0000","last_undegraded":"2023-07-14T16:15:19.503946+0000","last_fullsized":"2023-07-14T16:15:19.503946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:45:58.125276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.502135+0000","last_change":"2023-07-14T16:13:59.299228+0000","last_active":"2023-07-14T16:15:19.502135+0000","last_peered":"2023-07-14T16:15:19.502135+0000","last_clean":"2023-07-14T16:15:19.502135+0000","last_became_active":"2023-07-14T16:13:59.297982+0000","last_became_peered":"2023-07-14T16:13:59.297982+0000","last_unstale":"2023-07-14T16:15:19.502135+0000","last_undegraded":"2023-07-14T16:15:19.502135+0000","last_fullsized":"2023-07-14T16:15:19.502135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:34:24.432565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508368+0000","last_change":"2023-07-14T16:13:59.299482+0000","last_active":"2023-07-14T16:15:19.508368+0000","last_peered":"2023-07-14T16:15:19.508368+0000","last_clean":"2023-07-14T16:15:19.508368+0000","last_became_active":"2023-07-14T16:13:59.299181+0000","last_became_peered":"2023-07-14T16:13:59.299181+0000","last_unstale":"2023-07-14T16:15:19.508368+0000","last_undegraded":"2023-07-14T16:15:19.508368+0000","last_fullsized":"2023-07-14T16:15:19.508368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:42.604278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.488076+0000","last_change":"2023-07-14T16:13:59.303282+0000","last_active":"2023-07-14T16:15:19.488076+0000","last_peered":"2023-07-14T16:15:19.488076+0000","last_clean":"2023-07-14T16:15:19.488076+0000","last_became_active":"2023-07-14T16:13:59.302922+0000","last_became_peered":"2023-07-14T16:13:59.302922+0000","last_unstale":"2023-07-14T16:15:19.488076+0000","last_undegraded":"2023-07-14T16:15:19.488076+0000","last_fullsized":"2023-07-14T16:15:19.488076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:26:08.750328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.499836+0000","last_change":"2023-07-14T16:13:59.296389+0000","last_active":"2023-07-14T16:15:19.499836+0000","last_peered":"2023-07-14T16:15:19.499836+0000","last_clean":"2023-07-14T16:15:19.499836+0000","last_became_active":"2023-07-14T16:13:59.296080+0000","last_became_peered":"2023-07-14T16:13:59.296080+0000","last_unstale":"2023-07-14T16:15:19.499836+0000","last_undegraded":"2023-07-14T16:15:19.499836+0000","last_fullsized":"2023-07-14T16:15:19.499836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T18:25:02.060808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.500459+0000","last_change":"2023-07-14T16:13:59.297774+0000","last_active":"2023-07-14T16:15:19.500459+0000","last_peered":"2023-07-14T16:15:19.500459+0000","last_clean":"2023-07-14T16:15:19.500459+0000","last_became_active":"2023-07-14T16:13:59.297463+0000","last_became_peered":"2023-07-14T16:13:59.297463+0000","last_unstale":"2023-07-14T16:15:19.500459+0000","last_undegraded":"2023-07-14T16:15:19.500459+0000","last_fullsized":"2023-07-14T16:15:19.500459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:34:49.304022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.508679+0000","last_change":"2023-07-14T16:13:59.306795+0000","last_active":"2023-07-14T16:15:19.508679+0000","last_peered":"2023-07-14T16:15:19.508679+0000","last_clean":"2023-07-14T16:15:19.508679+0000","last_became_active":"2023-07-14T16:13:59.306457+0000","last_became_peered":"2023-07-14T16:13:59.306457+0000","last_unstale":"2023-07-14T16:15:19.508679+0000","last_undegraded":"2023-07-14T16:15:19.508679+0000","last_fullsized":"2023-07-14T16:15:19.508679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:47.873750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.504322+0000","last_change":"2023-07-14T16:13:59.305338+0000","last_active":"2023-07-14T16:15:19.504322+0000","last_peered":"2023-07-14T16:15:19.504322+0000","last_clean":"2023-07-14T16:15:19.504322+0000","last_became_active":"2023-07-14T16:13:59.305047+0000","last_became_peered":"2023-07-14T16:13:59.305047+0000","last_unstale":"2023-07-14T16:15:19.504322+0000","last_undegraded":"2023-07-14T16:15:19.504322+0000","last_fullsized":"2023-07-14T16:15:19.504322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:07:02.694956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-07-14T16:15:19.503214+0000","last_change":"2023-07-14T16:13:59.299005+0000","last_active":"2023-07-14T16:15:19.503214+0000","last_peered":"2023-07-14T16:15:19.503214+0000","last_clean":"2023-07-14T16:15:19.503214+0000","last_became_active":"2023-07-14T16:13:59.298705+0000","last_became_peered":"2023-07-14T16:13:59.298705+0000","last_unstale":"2023-07-14T16:15:19.503214+0000","last_undegraded":"2023-07-14T16:15:19.503214+0000","last_fullsized":"2023-07-14T16:15:19.503214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T16:13:58.253790+0000","last_clean_scrub_stamp":"2023-07-14T16:13:58.253790+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:59:41.986408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247144,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67642,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":631,"num_read_kb":382,"num_write":162,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":198606,"data_compressed":4809,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":459298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97699999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80000000000000004,"15min":0.78800000000000003},"min":{"1min":0.61799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0069999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.135,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41399999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45900000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88400000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.84599999999999997,"15min":0.83099999999999996},"min":{"1min":0.70299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1919999999999999,"5min":1.385,"15min":1.385},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97499999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.70299999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96999999999999997,"5min":1.361,"15min":1.361},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.78400000000000003,"15min":0.77200000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.165,"5min":1.264,"15min":1.264},"last":0.89300000000000002}]},{"osd":6,"last update":"Fri Jul 14 16:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.91700000000000004,"15min":0.90300000000000002},"min":{"1min":0.78300000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1359999999999999,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.89000000000000001,"15min":0.89600000000000002},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0629999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.88100000000000001}]},{"osd":8,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.04},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0600000000000001}]},{"osd":9,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.94099999999999995,"15min":0.95399999999999996},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5880000000000001,"5min":1.992,"15min":1.992},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.96499999999999997,"15min":0.94899999999999995},"min":{"1min":0.79000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.397,"5min":1.653,"15min":1.653},"last":1.0189999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:03 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.92200000000000004,"15min":0.93000000000000005},"min":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.099,"5min":2.097,"15min":2.097},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.95099999999999996,"15min":0.96299999999999997},"min":{"1min":0.68500000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.081,"5min":2.024,"15min":2.024},"last":0.90600000000000003}]}]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1093632,"data_stored":1109743,"data_compressed":37379,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.165,"5min":7.165,"15min":7.165},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.125,"5min":7.125,"15min":7.125},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.41799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79300000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73999999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.63100000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94599999999999995,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.501},"max":{"1min":0.92400000000000004,"5min":1.173,"15min":1.173},"last":0.745}]},{"osd":3,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.81999999999999995,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":4.1440000000000001,"15min":4.1440000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.80000000000000004,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.879,"5min":4.4429999999999996,"15min":4.4429999999999996},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93799999999999994,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.74299999999999999,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":1.613,"15min":1.613},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.92600000000000005,"15min":0.91600000000000004},"min":{"1min":0.68500000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.016,"5min":4.9749999999999996,"15min":4.9749999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89300000000000002,"15min":0.91400000000000003},"min":{"1min":0.65800000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0369999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.86499999999999999}]},{"osd":6,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":1.01,"15min":0.95099999999999996},"min":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.05,"5min":6.609,"15min":6.609},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96599999999999997,"15min":0.93200000000000005},"min":{"1min":0.67000000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1499999999999999,"5min":6.5759999999999996,"15min":6.5759999999999996},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Jul 14 16:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.61699999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.97499999999999998,"15min":0.98299999999999998},"min":{"1min":0.754,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.103,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.98099999999999998,"15min":1.0069999999999999},"min":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0740000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.85299999999999998}]},{"osd":9,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.72699999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1830000000000001,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.121,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1579999999999999,"5min":7.3579999999999997,"15min":7.3579999999999997},"last":0.79700000000000004}]},{"osd":11,"last update":"Fri Jul 14 16:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.046,"15min":1.05},"min":{"1min":0.84799999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.1890000000000001,"5min":2.653,"15min":2.653},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0089999999999999,"15min":1.002},"min":{"1min":0.92200000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.90900000000000003}]}]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":459512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":0.82399999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82599999999999996,"15min":0.86199999999999999},"min":{"1min":0.55200000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1259999999999999,"5min":1.129,"15min":1.129},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82899999999999996,"15min":0.86499999999999999},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.2809999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.878},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.82699999999999996,"15min":0.872},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63600000000000001,"15min":0.66100000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.376},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.78300000000000003,"15min":0.83199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.629}]},{"osd":4,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.88600000000000001,"15min":0.93400000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.82899999999999996,"15min":0.85299999999999998},"min":{"1min":0.55000000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84499999999999997,"15min":0.877},"min":{"1min":0.502,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3220000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.94099999999999995,"15min":1.0029999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.448,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.93999999999999995,"15min":1.014},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1799999999999999,"5min":1.288,"15min":1.288},"last":0.59399999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94299999999999995,"15min":0.98299999999999998},"min":{"1min":0.58999999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.879,"5min":1.024,"15min":1.0720000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3979999999999999,"5min":1.575,"15min":1.575},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0229999999999999,"15min":1.0840000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.996,"15min":1.0449999999999999},"min":{"1min":0.69299999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.85799999999999998}]},{"osd":10,"last update":"Fri Jul 14 16:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.96999999999999997,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.034,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.94299999999999995,"15min":0.95999999999999996},"min":{"1min":0.66500000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.66500000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":459410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66400000000000003,"15min":0.68899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88300000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.72899999999999998,"15min":0.69799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":6.3410000000000002,"15min":6.3410000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98699999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.48799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.1100000000000001,"5min":1.216,"15min":1.216},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.081,"15min":1.081},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.89700000000000002}]},{"osd":3,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99099999999999999,"5min":1.175,"15min":1.175},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.78700000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92600000000000005,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.54200000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.002,"5min":1.149,"15min":1.149},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.61499999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.0289999999999999,"5min":1.218,"15min":1.218},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69899999999999995,"15min":0.69199999999999995},"min":{"1min":0.52600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73599999999999999,"15min":0.74399999999999999},"min":{"1min":0.46500000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.05,"5min":1.21,"15min":1.21},"last":0.624}]},{"osd":6,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84199999999999997,"15min":0.85799999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1839999999999999,"5min":1.29,"15min":1.29},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.63500000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.256,"15min":1.256},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.93799999999999994,"15min":0.89300000000000002},"min":{"1min":0.63200000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.222,"5min":6.3209999999999997,"15min":6.3209999999999997},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89000000000000001,"15min":0.89800000000000002},"min":{"1min":0.68400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.1919999999999999,"5min":1.373,"15min":1.373},"last":0.92200000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.93899999999999995,"15min":0.91700000000000004},"min":{"1min":0.76700000000000002,"5min":0.625,"15min":0.625},"max":{"1min":3.4470000000000001,"5min":3.4470000000000001,"15min":3.4470000000000001},"last":0.875},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90600000000000003,"15min":0.91500000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.2130000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Jul 14 16:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.89800000000000002,"15min":0.86399999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1870000000000001,"5min":1.258,"15min":1.258},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.90800000000000003,"15min":0.88300000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.2050000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86199999999999999}]},{"osd":11,"last update":"Fri Jul 14 16:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.96199999999999997}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":1052410,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":1.181,"15min":1.181},"last":0.44}]},{"osd":1,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52600000000000002,"15min":0.51200000000000001},"min":{"1min":0.40000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79100000000000004,"5min":1.046,"15min":1.046},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.755}]},{"osd":2,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71999999999999997,"15min":0.71899999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.60299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.52100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71899999999999997,"15min":0.72599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95899999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.024,"15min":1.024},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90800000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.443}]},{"osd":5,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0760000000000001,"5min":1.246,"15min":1.246},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78800000000000003,"15min":0.78200000000000003},"min":{"1min":0.64700000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.93000000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.495,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93400000000000005,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84999999999999998,"5min":1.137,"15min":1.137},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Jul 14 16:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84299999999999997,"15min":0.83899999999999997},"min":{"1min":0.67400000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0369999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.69999999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.079,"5min":1.331,"15min":1.331},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Jul 14 16:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.88800000000000001,"15min":0.89400000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.054,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.628,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0189999999999999,"5min":1.262,"15min":1.262},"last":0.748}]},{"osd":10,"last update":"Fri Jul 14 16:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.71699999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1080000000000001,"5min":1.238,"15min":1.238},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.89500000000000002,"15min":0.89900000000000002},"min":{"1min":0.70199999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.0920000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.71199999999999997}]},{"osd":11,"last update":"Fri Jul 14 16:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.96599999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":458907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.48199999999999998}]},{"osd":1,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71199999999999997,"15min":0.68999999999999995},"min":{"1min":0.5,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.67800000000000005},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98599999999999999,"5min":1.111,"15min":1.111},"last":0.82099999999999995}]},{"osd":2,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0009999999999999,"5min":1.121,"15min":1.121},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94799999999999995,"5min":1.105,"15min":1.105},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.62},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.79100000000000004,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82099999999999995,"15min":0.81000000000000005},"min":{"1min":0.53800000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0720000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jul 14 16:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.83399999999999996,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1739999999999999,"5min":1.28,"15min":1.28},"last":1.048},{"interface":"front","average":{"1min":0.874,"5min":0.85699999999999998,"15min":0.80800000000000005},"min":{"1min":0.65100000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.238,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0129999999999999}]},{"osd":8,"last update":"Fri Jul 14 16:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.90700000000000003,"15min":0.92900000000000005},"min":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1180000000000001,"5min":1.353,"15min":1.353},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1020000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.94699999999999995}]},{"osd":9,"last update":"Fri Jul 14 16:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.88100000000000001,"15min":0.92000000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.141,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.93600000000000005,"15min":0.98299999999999998},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.129,"5min":1.331,"15min":1.331},"last":1.0880000000000001}]},{"osd":10,"last update":"Fri Jul 14 16:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.92000000000000004,"15min":0.93400000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.149,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.95199999999999996,"15min":0.98199999999999998},"min":{"1min":0.77400000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.073,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.78200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":516973,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61499999999999999,"15min":0.55300000000000005},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.34899999999999998},"max":{"1min":0.85699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.503}]},{"osd":2,"last update":"Fri Jul 14 16:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.71799999999999997,"15min":0.61899999999999999},"min":{"1min":0.52200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.0860000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.71299999999999997,"15min":0.63700000000000001},"min":{"1min":0.50700000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.69699999999999995,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57599999999999996,"15min":0.51800000000000002},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93500000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.504},"min":{"1min":0.47199999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68300000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 14 16:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.71099999999999997,"15min":0.66400000000000003},"min":{"1min":0.61299999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.71299999999999997,"15min":0.66500000000000004},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.751}]},{"osd":6,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76700000000000002,"15min":0.73099999999999998},"min":{"1min":0.61299999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0189999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.64100000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Jul 14 16:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.77300000000000002,"15min":0.70699999999999996},"min":{"1min":0.68000000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1919999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.754,"15min":0.68200000000000005},"min":{"1min":0.48799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.1559999999999999,"5min":1.244,"15min":1.244},"last":0.76300000000000001}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.86499999999999999,"15min":0.84699999999999998},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2050000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.81200000000000006,"15min":0.77200000000000002},"min":{"1min":0.75,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.73699999999999999}]},{"osd":9,"last update":"Fri Jul 14 16:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.871,"15min":0.875},"min":{"1min":0.68999999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.091,"5min":1.204,"15min":1.204},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92000000000000004,"15min":0.91700000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.262,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.58099999999999996}]},{"osd":10,"last update":"Fri Jul 14 16:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.66700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92400000000000004,"15min":0.92800000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.05,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.65300000000000002}]},{"osd":11,"last update":"Fri Jul 14 16:15:02 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.872},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.83099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":460172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63800000000000001,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jul 14 16:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73799999999999999,"15min":0.63900000000000001},"min":{"1min":0.60099999999999998,"5min":0.38200000000000001,"15min":0.34399999999999997},"max":{"1min":0.96399999999999997,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.64900000000000002},"min":{"1min":0.53300000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.03,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Fri Jul 14 16:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.3599999999999999,"5min":3.3599999999999999,"15min":3.3599999999999999},"last":0.504}]},{"osd":4,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2030000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.68400000000000005,"15min":0.64800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Jul 14 16:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":1.357,"15min":1.357},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.74199999999999999,"15min":0.69399999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.70999999999999996,"15min":0.67800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.5670000000000002,"5min":3.5670000000000002,"15min":3.5670000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.9630000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.748,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.83799999999999997}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.92100000000000004},"min":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.85899999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.89700000000000002}]},{"osd":9,"last update":"Fri Jul 14 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.878,"15min":0.88},"min":{"1min":0.5,"5min":0.495,"15min":0.495},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.91500000000000004,"15min":0.94399999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.61499999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88600000000000001,"15min":0.91300000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2370000000000001,"5min":1.419,"15min":1.419},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.92600000000000005,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.93799999999999994,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.88300000000000001}]},{"osd":11,"last update":"Fri Jul 14 16:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89000000000000001,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.171,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.94899999999999995,"15min":0.94099999999999995},"min":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.5350000000000001,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":0.82799999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":518774,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.69099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.376,"15min":0.34699999999999998},"max":{"1min":4.4400000000000004,"5min":4.4400000000000004,"15min":4.4400000000000004},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.67200000000000004,"15min":0.58399999999999996},"min":{"1min":0.47299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Jul 14 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65100000000000002,"15min":0.66400000000000003},"min":{"1min":0.51400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":0.90500000000000003,"15min":0.90800000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.70599999999999996,"15min":0.67300000000000004},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.41799999999999998},"max":{"1min":1.0569999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.038}]},{"osd":3,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.66500000000000004},"min":{"1min":0.45900000000000002,"5min":0.39200000000000002,"15min":0.39100000000000001},"max":{"1min":2.1309999999999998,"5min":3.274,"15min":3.274},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67200000000000004,"15min":0.629},"min":{"1min":0.52300000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Jul 14 16:15:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.64700000000000002,"15min":0.58799999999999997},"min":{"1min":0.44700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59899999999999998,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90100000000000002,"5min":1.022,"15min":1.022},"last":0.48999999999999999}]},{"osd":6,"last update":"Fri Jul 14 16:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.76600000000000001,"15min":0.71999999999999997},"min":{"1min":0.57699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.7240000000000002,"5min":3.7240000000000002,"15min":3.7240000000000002},"last":0.75},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.80600000000000005,"15min":0.75800000000000001},"min":{"1min":0.53200000000000003,"5min":0.439,"15min":0.439},"max":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jul 14 16:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.94999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78300000000000003,"15min":0.77500000000000002},"min":{"1min":0.67000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.022,"5min":1.117,"15min":1.117},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Jul 14 16:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99099999999999999,"5min":1.016,"15min":1.016},"last":0.61199999999999999}]},{"osd":9,"last update":"Fri Jul 14 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.86799999999999999,"15min":0.83799999999999997},"min":{"1min":0.70699999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":4.5579999999999998,"5min":4.5579999999999998,"15min":4.5579999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.91500000000000004,"15min":0.90000000000000002},"min":{"1min":0.67400000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.5060000000000002,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":0.96699999999999997}]},{"osd":10,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.84699999999999998,"15min":0.83099999999999996},"min":{"1min":0.66700000000000004,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1160000000000001,"5min":1.246,"15min":1.246},"last":0.997},{"interface":"front","average":{"1min":1,"5min":0.91400000000000003,"15min":0.89400000000000002},"min":{"1min":0.71999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.0640000000000001}]},{"osd":11,"last update":"Fri Jul 14 16:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.86499999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":458943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.49399999999999999,"15min":0.42199999999999999},"min":{"1min":0.33400000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.495,"15min":0.442},"min":{"1min":0.34100000000000003,"5min":0.30299999999999999,"15min":0.27900000000000003},"max":{"1min":0.96399999999999997,"5min":2.323,"15min":2.323},"last":0.497}]},{"osd":1,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.63400000000000001},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1419999999999999,"5min":2.343,"15min":2.343},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70199999999999996,"15min":0.63800000000000001},"min":{"1min":0.50700000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.9430000000000001,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.73799999999999999,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.72299999999999998},"min":{"1min":0.46100000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2809999999999999,"5min":3.448,"15min":3.448},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Jul 14 16:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.755,"15min":0.67400000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.048,"5min":3.359,"15min":3.359},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.749,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.84199999999999997}]},{"osd":5,"last update":"Fri Jul 14 16:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75600000000000001,"15min":0.71999999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3120000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79000000000000004,"15min":0.74399999999999999},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.434},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Jul 14 16:14:54 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.75,"15min":0.68500000000000005},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":2.8879999999999999},"last":0.627},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.76900000000000002,"15min":0.72899999999999998},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.2050000000000001,"5min":3.2050000000000001,"15min":3.2050000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.84599999999999997,"15min":0.79800000000000004},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.82999999999999996,"15min":0.78000000000000003},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Jul 14 16:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.876,"15min":0.88500000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1259999999999999,"5min":1.546,"15min":1.546},"last":0.62},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91800000000000004,"15min":0.92100000000000004},"min":{"1min":0.61899999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.163,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91600000000000004}]},{"osd":9,"last update":"Fri Jul 14 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":0.98699999999999999,"15min":0.97899999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.6299999999999999,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.85899999999999999,"15min":0.86699999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.61399999999999999}]},{"osd":10,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.61699999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.004,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.97699999999999998,"15min":1.0049999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0860000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.79200000000000004}]},{"osd":11,"last update":"Fri Jul 14 16:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":2.4580000000000002},"last":0.75}]}]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":460320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.628,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.95599999999999996,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48499999999999999,"15min":0.437},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72999999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.46500000000000002},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79000000000000004,"5min":1.042,"15min":1.042},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.65100000000000002},"min":{"1min":0.46899999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.1319999999999999,"5min":3.698,"15min":3.698},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72899999999999998,"15min":0.68700000000000006},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.873}]},{"osd":3,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93400000000000005,"5min":1.153,"15min":1.153},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.014,"5min":1.03,"15min":1.03},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Jul 14 16:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76600000000000001,"15min":0.74199999999999999},"min":{"1min":0.64700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.012,"5min":1.506,"15min":1.506},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73899999999999999,"15min":0.72199999999999998},"min":{"1min":0.621,"5min":0.435,"15min":0.435},"max":{"1min":0.95099999999999996,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57099999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.147,"5min":1.321,"15min":1.321},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Jul 14 16:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.53800000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.012,"5min":1.038,"15min":1.038},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.752,"15min":0.751},"min":{"1min":0.51200000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.107,"5min":1.129,"15min":1.129},"last":0.85699999999999998}]},{"osd":8,"last update":"Fri Jul 14 16:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85999999999999999,"15min":0.84299999999999997},"min":{"1min":0.753,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1970000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.79100000000000004,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.95699999999999996}]},{"osd":9,"last update":"Fri Jul 14 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90700000000000003,"15min":0.91900000000000004},"min":{"1min":0.72799999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.24,"5min":1.631,"15min":1.631},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.77100000000000002}]},{"osd":10,"last update":"Fri Jul 14 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.59099999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0840000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.88800000000000001,"15min":0.91000000000000003},"min":{"1min":0.67100000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0229999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Fri Jul 14 16:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.94499999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":1050781,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.39900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78600000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.68200000000000005,"15min":0.621},"min":{"1min":0.58599999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0189999999999999,"5min":1.099,"15min":1.099},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.69099999999999995,"15min":0.63600000000000001},"min":{"1min":0.55200000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51900000000000002,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86799999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77400000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.64400000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.83199999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.71299999999999997,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77700000000000002,"5min":7.056,"15min":7.056},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jul 14 16:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.752,"15min":0.70399999999999996},"min":{"1min":0.621,"5min":0.503,"15min":0.503},"max":{"1min":0.96499999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.76400000000000001,"15min":0.71399999999999997},"min":{"1min":0.70299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.0669999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Jul 14 16:14:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73599999999999999,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":4.3739999999999997,"5min":7.109,"15min":7.109},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Jul 14 16:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.58099999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.82199999999999995,"15min":0.78800000000000003},"min":{"1min":0.65700000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.94199999999999995}]},{"osd":8,"last update":"Fri Jul 14 16:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.1479999999999999,"5min":1.286,"15min":1.286},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.79300000000000004},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97899999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Fri Jul 14 16:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.89100000000000001,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.88400000000000001,"15min":0.872},"min":{"1min":0.622,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.004}]},{"osd":10,"last update":"Fri Jul 14 16:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.86299999999999999,"15min":0.85799999999999998},"min":{"1min":0.67100000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0409999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.877},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.92300000000000004,"15min":0.91700000000000004},"min":{"1min":0.70499999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.105,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34132,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35776,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62599,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60080,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65159,"data_compressed":1603,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T16:15:46.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.570+0000 7fb5937ff700 1 -- 172.21.15.62:0/1292360963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 msgr2=0x7fb58c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.571+0000 7fb5937ff700 1 --2- 172.21.15.62:0/1292360963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 0x7fb58c101140 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009990 tx=0x7fb57c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.571+0000 7fb5937ff700 1 -- 172.21.15.62:0/1292360963 shutdown_connections 2023-07-14T16:15:46.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.571+0000 7fb5937ff700 1 --2- 172.21.15.62:0/1292360963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb58c101680 0x7fb58c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.571+0000 7fb5937ff700 1 --2- 172.21.15.62:0/1292360963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb58c0ffb00 0x7fb58c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.571+0000 7fb5937ff700 1 --2- 172.21.15.62:0/1292360963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 0x7fb58c101140 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.571+0000 7fb5937ff700 1 -- 172.21.15.62:0/1292360963 >> 172.21.15.62:0/1292360963 conn(0x7fb58c0fb0e0 msgr2=0x7fb58c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:46.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.571+0000 7fb5937ff700 1 -- 172.21.15.62:0/1292360963 shutdown_connections 2023-07-14T16:15:46.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.571+0000 7fb5937ff700 1 -- 172.21.15.62:0/1292360963 wait complete. 2023-07-14T16:15:46.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.572+0000 7fb5937ff700 1 Processor -- start 2023-07-14T16:15:46.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.572+0000 7fb5937ff700 1 -- start start 2023-07-14T16:15:46.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.572+0000 7fb5937ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb58c0ffb00 0x7fb58c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:46.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.572+0000 7fb5937ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 0x7fb58c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:46.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.572+0000 7fb5937ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb58c101680 0x7fb58c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:46.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.572+0000 7fb5937ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c19f4c0 con 0x7fb58c100cf0 2023-07-14T16:15:46.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.572+0000 7fb5937ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c19f600 con 0x7fb58c0ffb00 2023-07-14T16:15:46.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.572+0000 7fb5937ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c19f790 con 0x7fb58c101680 2023-07-14T16:15:46.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 0x7fb58c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:46.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 0x7fb58c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35094/0 (socket says 172.21.15.62:35094) 2023-07-14T16:15:46.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 -- 172.21.15.62:0/181018044 learned_addr learned my addr 172.21.15.62:0/181018044 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:46.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb59159b700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb58c0ffb00 0x7fb58c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:46.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb591d9c700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb58c101680 0x7fb58c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:46.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 -- 172.21.15.62:0/181018044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb58c101680 msgr2=0x7fb58c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb58c101680 0x7fb58c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 -- 172.21.15.62:0/181018044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb58c0ffb00 msgr2=0x7fb58c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb58c0ffb00 0x7fb58c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 -- 172.21.15.62:0/181018044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0095f0 con 0x7fb58c100cf0 2023-07-14T16:15:46.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb591d9c700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb58c101680 0x7fb58c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:46.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.573+0000 7fb590d9a700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 0x7fb58c197e80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb57c006010 tx=0x7fb57c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:46.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.574+0000 7fb5827fc700 1 -- 172.21.15.62:0/181018044 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c01d070 con 0x7fb58c100cf0 2023-07-14T16:15:46.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.574+0000 7fb5827fc700 1 -- 172.21.15.62:0/181018044 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb57c004100 con 0x7fb58c100cf0 2023-07-14T16:15:46.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.574+0000 7fb5827fc700 1 -- 172.21.15.62:0/181018044 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c005290 con 0x7fb58c100cf0 2023-07-14T16:15:46.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.574+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c19fa10 con 0x7fb58c100cf0 2023-07-14T16:15:46.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.574+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c19fde0 con 0x7fb58c100cf0 2023-07-14T16:15:46.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.575+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c10b1d0 con 0x7fb58c100cf0 2023-07-14T16:15:46.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.577+0000 7fb5827fc700 1 -- 172.21.15.62:0/181018044 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb57c0047e0 con 0x7fb58c100cf0 2023-07-14T16:15:46.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.578+0000 7fb5827fc700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5780a1220 0x7fb5780a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:46.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.578+0000 7fb5827fc700 1 -- 172.21.15.62:0/181018044 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb57c0cd9e0 con 0x7fb58c100cf0 2023-07-14T16:15:46.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.578+0000 7fb59159b700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5780a1220 0x7fb5780a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:46.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.579+0000 7fb59159b700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5780a1220 0x7fb5780a36d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb588005950 tx=0x7fb5880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:46.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.582+0000 7fb5827fc700 1 -- 172.21.15.62:0/181018044 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb57c097170 con 0x7fb58c100cf0 2023-07-14T16:15:46.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.791+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb58c1a04e0 con 0x7fb58c100cf0 2023-07-14T16:15:46.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.793+0000 7fb5827fc700 1 -- 172.21.15.62:0/181018044 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7fb57c09a790 con 0x7fb58c100cf0 2023-07-14T16:15:46.773 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:46.773 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:46.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5780a1220 msgr2=0x7fb5780a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5780a1220 0x7fb5780a36d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb588005950 tx=0x7fb5880058e0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 msgr2=0x7fb58c197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 0x7fb58c197e80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb57c006010 tx=0x7fb57c005ef0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 shutdown_connections 2023-07-14T16:15:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb58c101680 0x7fb58c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5780a1220 0x7fb5780a36d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb58c0ffb00 0x7fb58c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 --2- 172.21.15.62:0/181018044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb58c100cf0 0x7fb58c197e80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.795+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 >> 172.21.15.62:0/181018044 conn(0x7fb58c0fb0e0 msgr2=0x7fb58c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.796+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 shutdown_connections 2023-07-14T16:15:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:46.796+0000 7fb5937ff700 1 -- 172.21.15.62:0/181018044 wait complete. 2023-07-14T16:15:46.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:15:46.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:15:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2788071200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec100ce0 msgr2=0x7f1cec101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2788071200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec100ce0 0x7f1cec101130 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009990 tx=0x7f1ce0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2788071200 shutdown_connections 2023-07-14T16:15:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2788071200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cec101670 0x7f1cec107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2788071200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cec0ffaf0 0x7f1cec0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2788071200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec100ce0 0x7f1cec101130 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2788071200 >> 172.21.15.62:0/2788071200 conn(0x7f1cec0fb130 msgr2=0x7f1cec0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2788071200 shutdown_connections 2023-07-14T16:15:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.021+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2788071200 wait complete. 2023-07-14T16:15:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.022+0000 7f1cf29ff700 1 Processor -- start 2023-07-14T16:15:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cf29ff700 1 -- start start 2023-07-14T16:15:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cf29ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec0ffaf0 0x7f1cec0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cf29ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cec100ce0 0x7f1cec072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cf29ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cec101670 0x7f1cec072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cf29ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec0735e0 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cf29ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec073750 con 0x7f1cec100ce0 2023-07-14T16:15:47.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cf29ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec19f810 con 0x7f1cec101670 2023-07-14T16:15:47.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec0ffaf0 0x7f1cec0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec0ffaf0 0x7f1cec0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35114/0 (socket says 172.21.15.62:35114) 2023-07-14T16:15:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cebfff700 1 -- 172.21.15.62:0/2339588246 learned_addr learned my addr 172.21.15.62:0/2339588246 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.023+0000 7f1cf0f9c700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cec101670 0x7f1cec072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1ceb7fe700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cec100ce0 0x7f1cec072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1cebfff700 1 -- 172.21.15.62:0/2339588246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cec101670 msgr2=0x7f1cec072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1cebfff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cec101670 0x7f1cec072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1cebfff700 1 -- 172.21.15.62:0/2339588246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cec100ce0 msgr2=0x7f1cec072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1cebfff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cec100ce0 0x7f1cec072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1cebfff700 1 -- 172.21.15.62:0/2339588246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce00095f0 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1ceb7fe700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cec100ce0 0x7f1cec072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:47.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1cebfff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec0ffaf0 0x7f1cec0742c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00e970 tx=0x7f1cdc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:47.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1ce97fa700 1 -- 172.21.15.62:0/2339588246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc009790 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1ce97fa700 1 -- 172.21.15.62:0/2339588246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cdc00eec0 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1ce97fa700 1 -- 172.21.15.62:0/2339588246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc003c10 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.024+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cec19faf0 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.025+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cec19ffb0 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.026+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cec10b1c0 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.028+0000 7f1ce97fa700 1 -- 172.21.15.62:0/2339588246 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1cdc003db0 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.029+0000 7f1ce97fa700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1cd40a11d0 0x7f1cd40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.029+0000 7f1ce97fa700 1 -- 172.21.15.62:0/2339588246 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cdc014070 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.030+0000 7f1ceb7fe700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1cd40a11d0 0x7f1cd40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.030+0000 7f1ceb7fe700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1cd40a11d0 0x7f1cd40a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000b2e0 tx=0x7f1ce0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.033+0000 7f1ce97fa700 1 -- 172.21.15.62:0/2339588246 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1cdc096260 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:46 smithi164 ceph-mon[97427]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:47.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:46 smithi164 ceph-mon[97427]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-07-14T16:15:47.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:46 smithi062 ceph-mon[95543]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:47.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:46 smithi062 ceph-mon[95543]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-07-14T16:15:47.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:46 smithi081 ceph-mon[127795]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:47.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:46 smithi081 ceph-mon[127795]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-07-14T16:15:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.244+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cec1a0820 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.244+0000 7f1ce97fa700 1 -- 172.21.15.62:0/2339588246 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cdc099880 con 0x7f1cec0ffaf0 2023-07-14T16:15:47.223 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:15:47.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1cd40a11d0 msgr2=0x7f1cd40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1cd40a11d0 0x7f1cd40a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000b2e0 tx=0x7f1ce0005fd0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec0ffaf0 msgr2=0x7f1cec0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec0ffaf0 0x7f1cec0742c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00e970 tx=0x7f1cdc00ec80 comp rx=0 tx=0).stop 2023-07-14T16:15:47.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 shutdown_connections 2023-07-14T16:15:47.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1cec101670 0x7f1cec072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cec0ffaf0 0x7f1cec0742c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1cd40a11d0 0x7f1cd40a3680 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 --2- 172.21.15.62:0/2339588246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cec100ce0 0x7f1cec072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.246+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 >> 172.21.15.62:0/2339588246 conn(0x7f1cec0fb130 msgr2=0x7f1cec0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.247+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 shutdown_connections 2023-07-14T16:15:47.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.247+0000 7f1cf29ff700 1 -- 172.21.15.62:0/2339588246 wait complete. 2023-07-14T16:15:47.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:15:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 -- 172.21.15.62:0/3430671604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 msgr2=0x7f4630107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 --2- 172.21.15.62:0/3430671604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 0x7f4630107f20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f462800b0a0 tx=0x7f462800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 -- 172.21.15.62:0/3430671604 shutdown_connections 2023-07-14T16:15:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 --2- 172.21.15.62:0/3430671604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 0x7f4630107f20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 --2- 172.21.15.62:0/3430671604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46300ffae0 0x7f46300ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 --2- 172.21.15.62:0/3430671604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4630100cd0 0x7f4630101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 -- 172.21.15.62:0/3430671604 >> 172.21.15.62:0/3430671604 conn(0x7f46300fb0e0 msgr2=0x7f46300fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 -- 172.21.15.62:0/3430671604 shutdown_connections 2023-07-14T16:15:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.471+0000 7f46381ff700 1 -- 172.21.15.62:0/3430671604 wait complete. 2023-07-14T16:15:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.472+0000 7f46381ff700 1 Processor -- start 2023-07-14T16:15:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.472+0000 7f46381ff700 1 -- start start 2023-07-14T16:15:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f46381ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46300ffae0 0x7f4630197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f46381ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4630100cd0 0x7f4630197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f46381ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 0x7f463019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f46381ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463019f5a0 con 0x7f4630101660 2023-07-14T16:15:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f46381ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463019f6e0 con 0x7f4630100cd0 2023-07-14T16:15:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f46381ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463019f870 con 0x7f46300ffae0 2023-07-14T16:15:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 0x7f463019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 0x7f463019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35144/0 (socket says 172.21.15.62:35144) 2023-07-14T16:15:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463679c700 1 -- 172.21.15.62:0/803401340 learned_addr learned my addr 172.21.15.62:0/803401340 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f4635f9b700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46300ffae0 0x7f4630197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463579a700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4630100cd0 0x7f4630197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463679c700 1 -- 172.21.15.62:0/803401340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46300ffae0 msgr2=0x7f4630197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463679c700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46300ffae0 0x7f4630197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463679c700 1 -- 172.21.15.62:0/803401340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4630100cd0 msgr2=0x7f4630197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463679c700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4630100cd0 0x7f4630197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.473+0000 7f463679c700 1 -- 172.21.15.62:0/803401340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628009d00 con 0x7f4630101660 2023-07-14T16:15:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.474+0000 7f463679c700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 0x7f463019efd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f462800b070 tx=0x7f4628004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.474+0000 7f4626ffd700 1 -- 172.21.15.62:0/803401340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800e040 con 0x7f4630101660 2023-07-14T16:15:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.474+0000 7f4626ffd700 1 -- 172.21.15.62:0/803401340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4628003e90 con 0x7f4630101660 2023-07-14T16:15:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.474+0000 7f4626ffd700 1 -- 172.21.15.62:0/803401340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628004020 con 0x7f4630101660 2023-07-14T16:15:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.474+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463019faf0 con 0x7f4630101660 2023-07-14T16:15:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.474+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463019ff20 con 0x7f4630101660 2023-07-14T16:15:47.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.475+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463010b1c0 con 0x7f4630101660 2023-07-14T16:15:47.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.477+0000 7f4626ffd700 1 -- 172.21.15.62:0/803401340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4628003a00 con 0x7f4630101660 2023-07-14T16:15:47.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.478+0000 7f4626ffd700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f461c0a1150 0x7f461c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.479+0000 7f4626ffd700 1 -- 172.21.15.62:0/803401340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f46280ce2e0 con 0x7f4630101660 2023-07-14T16:15:47.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.479+0000 7f4635f9b700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f461c0a1150 0x7f461c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.479+0000 7f4635f9b700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f461c0a1150 0x7f461c0a3600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4630100b30 tx=0x7f462c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:47.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.482+0000 7f4626ffd700 1 -- 172.21.15.62:0/803401340 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4628097ad0 con 0x7f4630101660 2023-07-14T16:15:47.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.692+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46301a05f0 con 0x7f4630101660 2023-07-14T16:15:47.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.693+0000 7f4626ffd700 1 -- 172.21.15.62:0/803401340 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4628019350 con 0x7f4630101660 2023-07-14T16:15:47.671 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:15:47.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f461c0a1150 msgr2=0x7f461c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f461c0a1150 0x7f461c0a3600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4630100b30 tx=0x7f462c00a3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 msgr2=0x7f463019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 0x7f463019efd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f462800b070 tx=0x7f4628004c80 comp rx=0 tx=0).stop 2023-07-14T16:15:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 shutdown_connections 2023-07-14T16:15:47.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4630101660 0x7f463019efd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46300ffae0 0x7f4630197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f461c0a1150 0x7f461c0a3600 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 --2- 172.21.15.62:0/803401340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4630100cd0 0x7f4630197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 >> 172.21.15.62:0/803401340 conn(0x7f46300fb0e0 msgr2=0x7f4630106570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 shutdown_connections 2023-07-14T16:15:47.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.695+0000 7f46381ff700 1 -- 172.21.15.62:0/803401340 wait complete. 2023-07-14T16:15:47.692 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:15:47.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 -- 172.21.15.62:0/3286456136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0100ce0 msgr2=0x7f38f0101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 --2- 172.21.15.62:0/3286456136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0100ce0 0x7f38f0101130 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009990 tx=0x7f38e4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 -- 172.21.15.62:0/3286456136 shutdown_connections 2023-07-14T16:15:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 --2- 172.21.15.62:0/3286456136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f0101670 0x7f38f0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 --2- 172.21.15.62:0/3286456136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f00ffaf0 0x7f38f00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 --2- 172.21.15.62:0/3286456136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0100ce0 0x7f38f0101130 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 -- 172.21.15.62:0/3286456136 >> 172.21.15.62:0/3286456136 conn(0x7f38f00fb130 msgr2=0x7f38f00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 -- 172.21.15.62:0/3286456136 shutdown_connections 2023-07-14T16:15:47.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.928+0000 7f38f77ff700 1 -- 172.21.15.62:0/3286456136 wait complete. 2023-07-14T16:15:47.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.929+0000 7f38f77ff700 1 Processor -- start 2023-07-14T16:15:47.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.929+0000 7f38f77ff700 1 -- start start 2023-07-14T16:15:47.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f77ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f00ffaf0 0x7f38f00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f77ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100ce0 0x7f38f0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f77ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0101670 0x7f38f0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f77ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00735e0 con 0x7f38f0101670 2023-07-14T16:15:47.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f77ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0073750 con 0x7f38f00ffaf0 2023-07-14T16:15:47.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f77ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f019f8a0 con 0x7f38f0100ce0 2023-07-14T16:15:47.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f5d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0101670 0x7f38f0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f5d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0101670 0x7f38f0072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35168/0 (socket says 172.21.15.62:35168) 2023-07-14T16:15:47.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f5d9c700 1 -- 172.21.15.62:0/2026471724 learned_addr learned my addr 172.21.15.62:0/2026471724 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:47.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f559b700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f00ffaf0 0x7f38f00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f4d9a700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100ce0 0x7f38f0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f5d9c700 1 -- 172.21.15.62:0/2026471724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100ce0 msgr2=0x7f38f0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f5d9c700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100ce0 0x7f38f0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f5d9c700 1 -- 172.21.15.62:0/2026471724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f00ffaf0 msgr2=0x7f38f00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:47.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f5d9c700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f00ffaf0 0x7f38f00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:47.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.930+0000 7f38f5d9c700 1 -- 172.21.15.62:0/2026471724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8009c40 con 0x7f38f0101670 2023-07-14T16:15:47.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.931+0000 7f38f5d9c700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0101670 0x7f38f0072e40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c370 tx=0x7f38e800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:47.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.931+0000 7f38ee7fc700 1 -- 172.21.15.62:0/2026471724 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8017070 con 0x7f38f0101670 2023-07-14T16:15:47.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.931+0000 7f38ee7fc700 1 -- 172.21.15.62:0/2026471724 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38e800f040 con 0x7f38f0101670 2023-07-14T16:15:47.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.931+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e40095f0 con 0x7f38f0101670 2023-07-14T16:15:47.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.931+0000 7f38ee7fc700 1 -- 172.21.15.62:0/2026471724 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e800e050 con 0x7f38f0101670 2023-07-14T16:15:47.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.931+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f019fe50 con 0x7f38f0101670 2023-07-14T16:15:47.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.932+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f0107180 con 0x7f38f0101670 2023-07-14T16:15:47.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.934+0000 7f38ee7fc700 1 -- 172.21.15.62:0/2026471724 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f38e8003a00 con 0x7f38f0101670 2023-07-14T16:15:47.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.935+0000 7f38ee7fc700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:47.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.936+0000 7f38ee7fc700 1 -- 172.21.15.62:0/2026471724 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f38e80cc2b0 con 0x7f38f0101670 2023-07-14T16:15:47.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.936+0000 7f38f559b700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:47.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.936+0000 7f38f559b700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38f0100b40 tx=0x7f38e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:47.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:47.939+0000 7f38ee7fc700 1 -- 172.21.15.62:0/2026471724 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f38e80959b0 con 0x7f38f0101670 2023-07-14T16:15:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:47 smithi164 ceph-mon[97427]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:47 smithi164 ceph-mon[97427]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:47 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/181018044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:47 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2339588246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:48.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:47 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/803401340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:47 smithi062 ceph-mon[95543]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:47 smithi062 ceph-mon[95543]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:47 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/181018044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:47 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2339588246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:47 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/803401340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:48.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.147+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38f0061150 con 0x7f38f0101670 2023-07-14T16:15:48.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.148+0000 7f38ee7fc700 1 -- 172.21.15.62:0/2026471724 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38e8098fd0 con 0x7f38f0101670 2023-07-14T16:15:48.126 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:15:48.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 msgr2=0x7f38d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38f0100b40 tx=0x7f38e0009260 comp rx=0 tx=0).stop 2023-07-14T16:15:48.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0101670 msgr2=0x7f38f0072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0101670 0x7f38f0072e40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c370 tx=0x7f38e800c680 comp rx=0 tx=0).stop 2023-07-14T16:15:48.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 shutdown_connections 2023-07-14T16:15:48.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f0101670 0x7f38f0072e40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f00ffaf0 0x7f38f00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 --2- 172.21.15.62:0/2026471724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100ce0 0x7f38f0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 >> 172.21.15.62:0/2026471724 conn(0x7f38f00fb130 msgr2=0x7f38f01042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:48.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 shutdown_connections 2023-07-14T16:15:48.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.150+0000 7f38f77ff700 1 -- 172.21.15.62:0/2026471724 wait complete. 2023-07-14T16:15:48.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:15:48.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:47 smithi081 ceph-mon[127795]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:48.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:47 smithi081 ceph-mon[127795]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T16:15:48.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:47 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/181018044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:48.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:47 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2339588246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:15:48.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:47 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/803401340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:15:48.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 -- 172.21.15.62:0/3829621194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 msgr2=0x7fab440729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 --2- 172.21.15.62:0/3829621194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 0x7fab440729a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fab34009990 tx=0x7fab34009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 -- 172.21.15.62:0/3829621194 shutdown_connections 2023-07-14T16:15:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 --2- 172.21.15.62:0/3829621194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab441079c0 0x7fab44109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 --2- 172.21.15.62:0/3829621194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 0x7fab440729a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 --2- 172.21.15.62:0/3829621194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab44072f70 0x7fab440733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 -- 172.21.15.62:0/3829621194 >> 172.21.15.62:0/3829621194 conn(0x7fab440fb0e0 msgr2=0x7fab440fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 -- 172.21.15.62:0/3829621194 shutdown_connections 2023-07-14T16:15:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.378+0000 7fab49bff700 1 -- 172.21.15.62:0/3829621194 wait complete. 2023-07-14T16:15:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.379+0000 7fab49bff700 1 Processor -- start 2023-07-14T16:15:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.379+0000 7fab49bff700 1 -- start start 2023-07-14T16:15:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab49bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab44072f70 0x7fab441979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab49bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 0x7fab44197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab49bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab441079c0 0x7fab4419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab49bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4419f550 con 0x7fab44074550 2023-07-14T16:15:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab49bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4419f690 con 0x7fab44072f70 2023-07-14T16:15:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab49bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4419f820 con 0x7fab441079c0 2023-07-14T16:15:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab42ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 0x7fab44197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab42ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 0x7fab44197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35184/0 (socket says 172.21.15.62:35184) 2023-07-14T16:15:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab42ffd700 1 -- 172.21.15.62:0/332594884 learned_addr learned my addr 172.21.15.62:0/332594884 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab437fe700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab44072f70 0x7fab441979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab43fff700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab441079c0 0x7fab4419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab42ffd700 1 -- 172.21.15.62:0/332594884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab441079c0 msgr2=0x7fab4419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab42ffd700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab441079c0 0x7fab4419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab42ffd700 1 -- 172.21.15.62:0/332594884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab44072f70 msgr2=0x7fab441979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab42ffd700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab44072f70 0x7fab441979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab42ffd700 1 -- 172.21.15.62:0/332594884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340095f0 con 0x7fab44074550 2023-07-14T16:15:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.380+0000 7fab43fff700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab441079c0 0x7fab4419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:48.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.381+0000 7fab42ffd700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 0x7fab44197f10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fab3800d980 tx=0x7fab3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:48.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.381+0000 7fab40ff9700 1 -- 172.21.15.62:0/332594884 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38004020 con 0x7fab44074550 2023-07-14T16:15:48.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.381+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4419fb00 con 0x7fab44074550 2023-07-14T16:15:48.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.381+0000 7fab40ff9700 1 -- 172.21.15.62:0/332594884 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab38009730 con 0x7fab44074550 2023-07-14T16:15:48.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.381+0000 7fab40ff9700 1 -- 172.21.15.62:0/332594884 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380039c0 con 0x7fab44074550 2023-07-14T16:15:48.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.381+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab441a0060 con 0x7fab44074550 2023-07-14T16:15:48.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.382+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab44106070 con 0x7fab44074550 2023-07-14T16:15:48.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.384+0000 7fab40ff9700 1 -- 172.21.15.62:0/332594884 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fab380098a0 con 0x7fab44074550 2023-07-14T16:15:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.386+0000 7fab40ff9700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab2c0a11a0 0x7fab2c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.386+0000 7fab40ff9700 1 -- 172.21.15.62:0/332594884 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fab380cc940 con 0x7fab44074550 2023-07-14T16:15:48.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.386+0000 7fab437fe700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab2c0a11a0 0x7fab2c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:48.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.386+0000 7fab437fe700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab2c0a11a0 0x7fab2c0a3650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fab3400b5c0 tx=0x7fab34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:48.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.388+0000 7fab40ff9700 1 -- 172.21.15.62:0/332594884 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fab380960a0 con 0x7fab44074550 2023-07-14T16:15:48.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.640+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab4402c850 con 0x7fab44074550 2023-07-14T16:15:48.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.641+0000 7fab40ff9700 1 -- 172.21.15.62:0/332594884 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fab380996c0 con 0x7fab44074550 2023-07-14T16:15:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:48.620 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:15:48.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab2c0a11a0 msgr2=0x7fab2c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab2c0a11a0 0x7fab2c0a3650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fab3400b5c0 tx=0x7fab34005fb0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 msgr2=0x7fab44197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 0x7fab44197f10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fab3800d980 tx=0x7fab3800dc90 comp rx=0 tx=0).stop 2023-07-14T16:15:48.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 shutdown_connections 2023-07-14T16:15:48.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab441079c0 0x7fab4419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab2c0a11a0 0x7fab2c0a3650 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab44072f70 0x7fab441979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 --2- 172.21.15.62:0/332594884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab44074550 0x7fab44197f10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 >> 172.21.15.62:0/332594884 conn(0x7fab440fb0e0 msgr2=0x7fab440fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 shutdown_connections 2023-07-14T16:15:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.643+0000 7fab49bff700 1 -- 172.21.15.62:0/332594884 wait complete. 2023-07-14T16:15:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:15:48.641 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:15:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.869+0000 7f10699ff700 1 -- 172.21.15.62:0/502738929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 msgr2=0x7f1064109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.869+0000 7f10699ff700 1 --2- 172.21.15.62:0/502738929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 0x7f1064109e40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f105c00b0a0 tx=0x7f105c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.869+0000 7f10699ff700 1 -- 172.21.15.62:0/502738929 shutdown_connections 2023-07-14T16:15:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.869+0000 7f10699ff700 1 --2- 172.21.15.62:0/502738929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 0x7f1064109e40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.869+0000 7f10699ff700 1 --2- 172.21.15.62:0/502738929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1064074550 0x7f10640729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.869+0000 7f10699ff700 1 --2- 172.21.15.62:0/502738929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1064072f70 0x7f10640733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.870+0000 7f10699ff700 1 -- 172.21.15.62:0/502738929 >> 172.21.15.62:0/502738929 conn(0x7f10640fb0e0 msgr2=0x7f10640fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:48.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.870+0000 7f10699ff700 1 -- 172.21.15.62:0/502738929 shutdown_connections 2023-07-14T16:15:48.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.870+0000 7f10699ff700 1 -- 172.21.15.62:0/502738929 wait complete. 2023-07-14T16:15:48.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10699ff700 1 Processor -- start 2023-07-14T16:15:48.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10699ff700 1 -- start start 2023-07-14T16:15:48.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10699ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1064072f70 0x7f1064197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:48.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10699ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1064074550 0x7f1064197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:48.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10699ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 0x7f106419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:48.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10699ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106419f630 con 0x7f10641079c0 2023-07-14T16:15:48.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10699ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106419f770 con 0x7f1064072f70 2023-07-14T16:15:48.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10699ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106419f900 con 0x7f1064074550 2023-07-14T16:15:48.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10637fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 0x7f106419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:48.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10637fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 0x7f106419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35208/0 (socket says 172.21.15.62:35208) 2023-07-14T16:15:48.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.871+0000 7f10637fe700 1 -- 172.21.15.62:0/1966709105 learned_addr learned my addr 172.21.15.62:0/1966709105 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:48.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f1062ffd700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1064072f70 0x7f1064197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:48.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10627fc700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1064074550 0x7f1064197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:48.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10637fe700 1 -- 172.21.15.62:0/1966709105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1064074550 msgr2=0x7f1064197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10637fe700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1064074550 0x7f1064197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10637fe700 1 -- 172.21.15.62:0/1966709105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1064072f70 msgr2=0x7f1064197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:48.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10637fe700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1064072f70 0x7f1064197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:48.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10637fe700 1 -- 172.21.15.62:0/1966709105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c009d00 con 0x7f10641079c0 2023-07-14T16:15:48.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10627fc700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1064074550 0x7f1064197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:48.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10637fe700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 0x7f106419f060 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f105c00d010 tx=0x7f105c00bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:48.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10689fd700 1 -- 172.21.15.62:0/1966709105 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00e040 con 0x7f10641079c0 2023-07-14T16:15:48.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10689fd700 1 -- 172.21.15.62:0/1966709105 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f105c003e10 con 0x7f10641079c0 2023-07-14T16:15:48.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10689fd700 1 -- 172.21.15.62:0/1966709105 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c005260 con 0x7f10641079c0 2023-07-14T16:15:48.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.872+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106419fb80 con 0x7f10641079c0 2023-07-14T16:15:48.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.873+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106419ff50 con 0x7f10641079c0 2023-07-14T16:15:48.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.874+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1064106070 con 0x7f10641079c0 2023-07-14T16:15:48.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.875+0000 7f10689fd700 1 -- 172.21.15.62:0/1966709105 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f105c01da10 con 0x7f10641079c0 2023-07-14T16:15:48.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.877+0000 7f10689fd700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f104c0a1220 0x7f104c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:48.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.877+0000 7f10689fd700 1 -- 172.21.15.62:0/1966709105 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f105c0cdd40 con 0x7f10641079c0 2023-07-14T16:15:48.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.877+0000 7f1062ffd700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f104c0a1220 0x7f104c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:48.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.878+0000 7f1062ffd700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f104c0a1220 0x7f104c0a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1064074920 tx=0x7f1054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:48.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:48.881+0000 7f10689fd700 1 -- 172.21.15.62:0/1966709105 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f105c0974d0 con 0x7f10641079c0 2023-07-14T16:15:49.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:48 smithi164 ceph-mon[97427]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-07-14T16:15:49.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:48 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2026471724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:49.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:48 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/332594884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:49.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.090+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f106402c850 con 0x7f10641079c0 2023-07-14T16:15:49.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.092+0000 7f10689fd700 1 -- 172.21.15.62:0/1966709105 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f105c09aaf0 con 0x7f10641079c0 2023-07-14T16:15:49.070 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:15:49.071 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:15:49.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f104c0a1220 msgr2=0x7f104c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f104c0a1220 0x7f104c0a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1064074920 tx=0x7f1054009260 comp rx=0 tx=0).stop 2023-07-14T16:15:49.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 msgr2=0x7f106419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 0x7f106419f060 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f105c00d010 tx=0x7f105c00bcc0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 shutdown_connections 2023-07-14T16:15:49.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10641079c0 0x7f106419f060 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f104c0a1220 0x7f104c0a36d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1064072f70 0x7f1064197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 --2- 172.21.15.62:0/1966709105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1064074550 0x7f1064197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 >> 172.21.15.62:0/1966709105 conn(0x7f10640fb0e0 msgr2=0x7f10640fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 shutdown_connections 2023-07-14T16:15:49.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.094+0000 7f10699ff700 1 -- 172.21.15.62:0/1966709105 wait complete. 2023-07-14T16:15:49.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:48 smithi062 ceph-mon[95543]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-07-14T16:15:49.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:48 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2026471724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:49.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:48 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/332594884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:49.091 DEBUG:tasks.fwd_scrub:Yielding 2023-07-14T16:15:49.092 INFO:teuthology.run_tasks:Running task background_exec... 2023-07-14T16:15:49.105 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2023-07-14T16:15:49.105 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-07-14T16:15:49.106 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-14T16:15:49.106 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:15:49.106 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:15:49.113 INFO:teuthology.run_tasks:Running task workunit... 2023-07-14T16:15:49.142 INFO:tasks.workunit:Pulling workunits from ref 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:15:49.143 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-14T16:15:49.143 INFO:tasks.workunit:timeout=3h 2023-07-14T16:15:49.143 INFO:tasks.workunit:cleanup=True 2023-07-14T16:15:49.143 DEBUG:teuthology.orchestra.run.smithi062:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:49.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:48 smithi081 ceph-mon[127795]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-07-14T16:15:49.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:48 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2026471724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:15:49.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:48 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/332594884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:15:49.191 INFO:teuthology.orchestra.run.smithi062.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:49.192 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2023-07-14T16:15:49.192 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 79h/121d Inode: 1 Links: 2 2023-07-14T16:15:49.192 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T16:15:49.192 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:fusefs_t:s0 2023-07-14T16:15:49.192 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-07-14 16:14:05.102618611 +0000 2023-07-14T16:15:49.192 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-07-14 16:14:05.102618611 +0000 2023-07-14T16:15:49.193 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-07-14 16:15:40.749317953 +0000 2023-07-14T16:15:49.193 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-07-14T16:15:49.193 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-14T16:15:49.193 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-14T16:15:49.314 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-14T16:15:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 -- 172.21.15.62:0/3493118874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 msgr2=0x7f4c78100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/3493118874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 0x7f4c78100020 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009990 tx=0x7f4c6c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 -- 172.21.15.62:0/3493118874 shutdown_connections 2023-07-14T16:15:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/3493118874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c78100560 0x7f4c78106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/3493118874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c780fe9e0 0x7f4c780fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/3493118874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 0x7f4c78100020 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 -- 172.21.15.62:0/3493118874 >> 172.21.15.62:0/3493118874 conn(0x7f4c780fa000 msgr2=0x7f4c780fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 -- 172.21.15.62:0/3493118874 shutdown_connections 2023-07-14T16:15:49.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.354+0000 7f4c7cbff700 1 -- 172.21.15.62:0/3493118874 wait complete. 2023-07-14T16:15:49.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.355+0000 7f4c7cbff700 1 Processor -- start 2023-07-14T16:15:49.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.355+0000 7f4c7cbff700 1 -- start start 2023-07-14T16:15:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.355+0000 7f4c7cbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c780fe9e0 0x7f4c781957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.355+0000 7f4c7cbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 0x7f4c78195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c7cbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c78100560 0x7f4c7819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c7cbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7819d350 con 0x7f4c780ffbd0 2023-07-14T16:15:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c7cbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7819d490 con 0x7f4c780fe9e0 2023-07-14T16:15:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c7cbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7819d620 con 0x7f4c78100560 2023-07-14T16:15:49.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c75d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 0x7f4c78195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c75d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 0x7f4c78195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35228/0 (socket says 172.21.15.62:35228) 2023-07-14T16:15:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c75d9b700 1 -- 172.21.15.62:0/353554240 learned_addr learned my addr 172.21.15.62:0/353554240 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c7659c700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c780fe9e0 0x7f4c781957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c76d9d700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c78100560 0x7f4c7819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c75d9b700 1 -- 172.21.15.62:0/353554240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c78100560 msgr2=0x7f4c7819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c78100560 0x7f4c7819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c75d9b700 1 -- 172.21.15.62:0/353554240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c780fe9e0 msgr2=0x7f4c781957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c780fe9e0 0x7f4c781957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c75d9b700 1 -- 172.21.15.62:0/353554240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c0095f0 con 0x7f4c780ffbd0 2023-07-14T16:15:49.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.356+0000 7f4c76d9d700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c78100560 0x7f4c7819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:49.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.357+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 0x7f4c78195d10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c005910 tx=0x7f4c6c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.357+0000 7f4c677fe700 1 -- 172.21.15.62:0/353554240 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c01d070 con 0x7f4c780ffbd0 2023-07-14T16:15:49.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.357+0000 7f4c7cbff700 1 -- 172.21.15.62:0/353554240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c781011a0 con 0x7f4c780ffbd0 2023-07-14T16:15:49.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.357+0000 7f4c677fe700 1 -- 172.21.15.62:0/353554240 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c6c00be90 con 0x7f4c780ffbd0 2023-07-14T16:15:49.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.357+0000 7f4c677fe700 1 -- 172.21.15.62:0/353554240 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c022e60 con 0x7f4c780ffbd0 2023-07-14T16:15:49.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.357+0000 7f4c7cbff700 1 -- 172.21.15.62:0/353554240 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c78101650 con 0x7f4c780ffbd0 2023-07-14T16:15:49.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.360+0000 7f4c677fe700 1 -- 172.21.15.62:0/353554240 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4c6c022600 con 0x7f4c780ffbd0 2023-07-14T16:15:49.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.362+0000 7f4c677fe700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c600a1150 0x7f4c600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.362+0000 7f4c904e0b80 1 Processor -- start 2023-07-14T16:15:49.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.362+0000 7f4c677fe700 1 -- 172.21.15.62:0/353554240 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c6c0cdd60 con 0x7f4c780ffbd0 2023-07-14T16:15:49.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.362+0000 7f4c904e0b80 1 -- start start 2023-07-14T16:15:49.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.362+0000 7f4c7659c700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c600a1150 0x7f4c600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.362+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211de4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 0x563211de4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211de5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211ce0d30 con 0x563211de3d60 2023-07-14T16:15:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211d33ac0 con 0x563211de46b0 2023-07-14T16:15:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211d2cf50 con 0x563211de5500 2023-07-14T16:15:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c7659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211de4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c7659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211de4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35236/0 (socket says 172.21.15.62:35236) 2023-07-14T16:15:49.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c7659c700 1 -- 172.21.15.62:0/2758014445 learned_addr learned my addr 172.21.15.62:0/2758014445 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 0x563211de4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c7659c700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c600a1150 0x7f4c600a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4c780ffa30 tx=0x7f4c68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c76d9d700 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211de5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.363+0000 7f4c7659c700 1 -- 172.21.15.62:0/2758014445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 msgr2=0x563211de5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.364+0000 7f4c7659c700 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211de5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.364+0000 7f4c7659c700 1 -- 172.21.15.62:0/2758014445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 msgr2=0x563211de4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.364+0000 7f4c7659c700 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 0x563211de4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.364+0000 7f4c7659c700 1 -- 172.21.15.62:0/2758014445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563211b5ae60 con 0x563211de3d60 2023-07-14T16:15:49.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.364+0000 7f4c7659c700 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211de4170 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005c50 tx=0x7f4c6801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34930ff0f3aba7e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.364+0000 7f4c64ff9700 1 -- 172.21.15.62:0/2758014445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6801cb80 con 0x563211de3d60 2023-07-14T16:15:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.364+0000 7f4c64ff9700 1 -- 172.21.15.62:0/2758014445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c68016070 con 0x563211de3d60 2023-07-14T16:15:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.364+0000 7f4c64ff9700 1 -- 172.21.15.62:0/2758014445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6801b5d0 con 0x563211de3d60 2023-07-14T16:15:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 -- 172.21.15.62:0/2758014445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 msgr2=0x563211de4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211de4170 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005c50 tx=0x7f4c6801c000 comp rx=0 tx=0).stop 2023-07-14T16:15:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 -- 172.21.15.62:0/2758014445 shutdown_connections 2023-07-14T16:15:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211de5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211de4170 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/2758014445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 0x563211de4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 -- 172.21.15.62:0/2758014445 >> 172.21.15.62:0/2758014445 conn(0x563211dd0e20 msgr2=0x563211dd6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 -- 172.21.15.62:0/2758014445 shutdown_connections 2023-07-14T16:15:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.365+0000 7f4c904e0b80 1 -- 172.21.15.62:0/2758014445 wait complete. 2023-07-14T16:15:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.366+0000 7f4c904e0b80 1 -- start start 2023-07-14T16:15:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.366+0000 7f4c904e0b80 1 Processor -- start 2023-07-14T16:15:49.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211ded9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de46b0 0x563211dedf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211df4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211cccb40 con 0x563211de46b0 2023-07-14T16:15:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211ccc840 con 0x563211de3d60 2023-07-14T16:15:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211bb09a0 con 0x563211de5500 2023-07-14T16:15:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c904e0b80 10 client.0 _ll_register_callbacks cb 0x563211cca7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c75d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de46b0 0x563211dedf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c75d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de46b0 0x563211dedf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35244/0 (socket says 172.21.15.62:35244) 2023-07-14T16:15:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c75d9b700 1 -- 172.21.15.62:0/4292177542 learned_addr learned my addr 172.21.15.62:0/4292177542 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c7659c700 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211ded9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c76d9d700 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211df4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.367+0000 7f4c75d9b700 1 -- 172.21.15.62:0/4292177542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 msgr2=0x563211df4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211df4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c75d9b700 1 -- 172.21.15.62:0/4292177542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 msgr2=0x563211ded9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211ded9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c75d9b700 1 -- 172.21.15.62:0/4292177542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c68009ae0 con 0x563211de46b0 2023-07-14T16:15:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de46b0 0x563211dedf20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c09b140 tx=0x7f4c6c09b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46852c2f081c2185 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c5dffb700 1 -- 172.21.15.62:0/4292177542 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0a32e0 con 0x563211de46b0 2023-07-14T16:15:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c904e0b80 1 -- 172.21.15.62:0/4292177542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563211c86570 con 0x563211de46b0 2023-07-14T16:15:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c5dffb700 1 -- 172.21.15.62:0/4292177542 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c6c09f1e0 con 0x563211de46b0 2023-07-14T16:15:49.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c5dffb700 1 -- 172.21.15.62:0/4292177542 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c6c0a3570 con 0x563211de46b0 2023-07-14T16:15:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c904e0b80 10 client.15468 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:49.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.368+0000 7f4c904e0b80 1 -- 172.21.15.62:0/4292177542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563211b43ce0 con 0x563211de46b0 2023-07-14T16:15:49.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c5dffb700 1 -- 172.21.15.62:0/4292177542 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c6c0c9020 con 0x563211de46b0 2023-07-14T16:15:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c5dffb700 1 -- 172.21.15.62:0/4292177542 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4c6c0b3b90 con 0x563211de46b0 2023-07-14T16:15:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c904e0b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c904e0b80 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:15:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c904e0b80 20 client.15468 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:49.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563211df93f0 0x563211dfb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c904e0b80 4 client.15468 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:15:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c904e0b80 1 -- 172.21.15.62:0/4292177542 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563211d25f30 con 0x563211df93f0 2023-07-14T16:15:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.369+0000 7f4c7659c700 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563211df93f0 0x563211dfb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.370+0000 7f4c7659c700 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563211df93f0 0x563211dfb7d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a7854d13a3a57d server_cookie=17dc1d238889f0e3 in_seq=0 out_seq=0 2023-07-14T16:15:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.370+0000 7f4c5dffb700 10 client.15468 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c5dffb700 1 -- 172.21.15.62:0/4292177542 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563211d25f30 con 0x563211df93f0 2023-07-14T16:15:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c5dffb700 10 client.15468 handle_command_reply: tid=0 2023-07-14T16:15:49.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 client.15468 shutdown 2023-07-14T16:15:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 -- 172.21.15.62:0/4292177542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de46b0 msgr2=0x563211dedf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de46b0 0x563211dedf20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c09b140 tx=0x7f4c6c09b6f0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 -- 172.21.15.62:0/4292177542 shutdown_connections 2023-07-14T16:15:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211df4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563211df93f0 0x563211dfb7d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211ded9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/4292177542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de46b0 0x563211dedf20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 -- 172.21.15.62:0/4292177542 >> 172.21.15.62:0/4292177542 conn(0x563211dd0e20 msgr2=0x563211dd5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c5dffb700 0 client.15468 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:49.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 -- 172.21.15.62:0/4292177542 shutdown_connections 2023-07-14T16:15:49.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 1 -- 172.21.15.62:0/4292177542 wait complete. 2023-07-14T16:15:49.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.371+0000 7f4c904e0b80 20 client.15468 trim_cache size 0 max 16384 2023-07-14T16:15:49.366 INFO:tasks.workunit.client.0.smithi062.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-14T16:15:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.389+0000 7f4c904e0b80 1 Processor -- start 2023-07-14T16:15:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.389+0000 7f4c904e0b80 1 -- start start 2023-07-14T16:15:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.389+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211dfb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.389+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 0x563211df4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.389+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211deb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.389+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211dd2f80 con 0x563211de3d60 2023-07-14T16:15:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.389+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211dee910 con 0x563211de46b0 2023-07-14T16:15:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.389+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211ded550 con 0x563211de5500 2023-07-14T16:15:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211dfb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211dfb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35256/0 (socket says 172.21.15.62:35256) 2023-07-14T16:15:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 -- 172.21.15.62:0/3165244653 learned_addr learned my addr 172.21.15.62:0/3165244653 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 0x563211df4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c76d9d700 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211deb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 -- 172.21.15.62:0/3165244653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 msgr2=0x563211deb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211deb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 -- 172.21.15.62:0/3165244653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 msgr2=0x563211df4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 0x563211df4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 -- 172.21.15.62:0/3165244653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563211dd2660 con 0x563211de3d60 2023-07-14T16:15:49.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.390+0000 7f4c7659c700 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211dfb400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4c68010c90 tx=0x7f4c6801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6517deceefec04c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.391+0000 7f4c657fa700 1 -- 172.21.15.62:0/3165244653 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6801cd80 con 0x563211de3d60 2023-07-14T16:15:49.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.391+0000 7f4c657fa700 1 -- 172.21.15.62:0/3165244653 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c68009db0 con 0x563211de3d60 2023-07-14T16:15:49.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.391+0000 7f4c657fa700 1 -- 172.21.15.62:0/3165244653 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68005040 con 0x563211de3d60 2023-07-14T16:15:49.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.391+0000 7f4c904e0b80 1 -- 172.21.15.62:0/3165244653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 msgr2=0x563211dfb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.391+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211dfb400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4c68010c90 tx=0x7f4c6801c000 comp rx=0 tx=0).stop 2023-07-14T16:15:49.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 -- 172.21.15.62:0/3165244653 shutdown_connections 2023-07-14T16:15:49.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de5500 0x563211deb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de3d60 0x563211dfb400 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/3165244653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de46b0 0x563211df4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 -- 172.21.15.62:0/3165244653 >> 172.21.15.62:0/3165244653 conn(0x563211dd0e20 msgr2=0x563211dd5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 -- 172.21.15.62:0/3165244653 shutdown_connections 2023-07-14T16:15:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 -- 172.21.15.62:0/3165244653 wait complete. 2023-07-14T16:15:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 -- start start 2023-07-14T16:15:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.392+0000 7f4c904e0b80 1 Processor -- start 2023-07-14T16:15:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211df4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de46b0 0x563211df17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c904e0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de5500 0x563211df1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211df22b0 con 0x563211de5500 2023-07-14T16:15:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211df2420 con 0x563211de3d60 2023-07-14T16:15:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c904e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563211e4d2f0 con 0x563211de46b0 2023-07-14T16:15:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c904e0b80 10 client.0 _ll_register_callbacks cb 0x7f4c6c09af80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c76d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de5500 0x563211df1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c76d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de5500 0x563211df1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35264/0 (socket says 172.21.15.62:35264) 2023-07-14T16:15:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c76d9d700 1 -- 172.21.15.62:0/1169331921 learned_addr learned my addr 172.21.15.62:0/1169331921 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.393+0000 7f4c75d9b700 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de46b0 0x563211df17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c7659c700 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211df4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c76d9d700 1 -- 172.21.15.62:0/1169331921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de46b0 msgr2=0x563211df17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c76d9d700 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de46b0 0x563211df17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c76d9d700 1 -- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 msgr2=0x563211df4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c76d9d700 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211df4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c76d9d700 1 -- 172.21.15.62:0/1169331921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563211b5ae60 con 0x563211de5500 2023-07-14T16:15:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c7659c700 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211df4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:15:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c76d9d700 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de5500 0x563211df1cf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4c70007840 tx=0x7f4c70007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bdc7226dc4904e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c5dffb700 1 -- 172.21.15.62:0/1169331921 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70015810 con 0x563211de5500 2023-07-14T16:15:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c5dffb700 1 -- 172.21.15.62:0/1169331921 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c70015970 con 0x563211de5500 2023-07-14T16:15:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.394+0000 7f4c904e0b80 1 -- 172.21.15.62:0/1169331921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c68016070 con 0x563211de5500 2023-07-14T16:15:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c5dffb700 1 -- 172.21.15.62:0/1169331921 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c70015b70 con 0x563211de5500 2023-07-14T16:15:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c904e0b80 10 client.15474 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c904e0b80 1 -- 172.21.15.62:0/1169331921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563211de5df0 con 0x563211de5500 2023-07-14T16:15:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c5dffb700 1 -- 172.21.15.62:0/1169331921 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c68016070 con 0x563211de5500 2023-07-14T16:15:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c5dffb700 1 -- 172.21.15.62:0/1169331921 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4c7002a070 con 0x563211de5500 2023-07-14T16:15:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c904e0b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c904e0b80 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:15:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c904e0b80 20 client.15474 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563211e4ee10 0x563211e51220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c904e0b80 4 client.15474 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:15:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.395+0000 7f4c904e0b80 1 -- 172.21.15.62:0/1169331921 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563211ce80d0 con 0x563211e4ee10 2023-07-14T16:15:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.396+0000 7f4c7659c700 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563211e4ee10 0x563211e51220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.396+0000 7f4c5dffb700 10 client.15474 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.396+0000 7f4c7659c700 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563211e4ee10 0x563211e51220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85c5b4f6d6ecd5e server_cookie=2735bc3a54dac248 in_seq=0 out_seq=0 2023-07-14T16:15:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c5dffb700 1 -- 172.21.15.62:0/1169331921 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563211ce80d0 con 0x563211e4ee10 2023-07-14T16:15:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c5dffb700 10 client.15474 handle_command_reply: tid=0 2023-07-14T16:15:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 client.15474 shutdown 2023-07-14T16:15:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 -- 172.21.15.62:0/1169331921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de5500 msgr2=0x563211df1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de5500 0x563211df1cf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4c70007840 tx=0x7f4c70007c00 comp rx=0 tx=0).stop 2023-07-14T16:15:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 -- 172.21.15.62:0/1169331921 shutdown_connections 2023-07-14T16:15:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563211de5500 0x563211df1cf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563211e4ee10 0x563211e51220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563211de3d60 0x563211df4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 --2- 172.21.15.62:0/1169331921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563211de46b0 0x563211df17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 -- 172.21.15.62:0/1169331921 >> 172.21.15.62:0/1169331921 conn(0x563211dd0e20 msgr2=0x563211dfb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c5dffb700 0 client.15474 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 -- 172.21.15.62:0/1169331921 shutdown_connections 2023-07-14T16:15:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 1 -- 172.21.15.62:0/1169331921 wait complete. 2023-07-14T16:15:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.398+0000 7f4c904e0b80 20 client.15474 trim_cache size 0 max 16384 2023-07-14T16:15:49.382 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:49.383 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:15:49.383 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "d523ca8f-615b-4121-bf9c-f88b9ceafbb7", 2023-07-14T16:15:49.383 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:15:49.383 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 -- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c600a1150 msgr2=0x7f4c600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c600a1150 0x7f4c600a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4c780ffa30 tx=0x7f4c68009260 comp rx=0 tx=0).stop 2023-07-14T16:15:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 -- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 msgr2=0x7f4c78195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 0x7f4c78195d10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c005910 tx=0x7f4c6c005720 comp rx=0 tx=0).stop 2023-07-14T16:15:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 -- 172.21.15.62:0/353554240 shutdown_connections 2023-07-14T16:15:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c78100560 0x7f4c7819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c600a1150 0x7f4c600a3600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c780fe9e0 0x7f4c781957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 --2- 172.21.15.62:0/353554240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c780ffbd0 0x7f4c78195d10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 -- 172.21.15.62:0/353554240 >> 172.21.15.62:0/353554240 conn(0x7f4c780fa000 msgr2=0x7f4c780fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 -- 172.21.15.62:0/353554240 shutdown_connections 2023-07-14T16:15:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.399+0000 7f4c7cbff700 1 -- 172.21.15.62:0/353554240 wait complete. 2023-07-14T16:15:49.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:15:49.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 -- 172.21.15.62:0/1628516478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0072f70 msgr2=0x7f8df00733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 --2- 172.21.15.62:0/1628516478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0072f70 0x7f8df00733e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009990 tx=0x7f8de0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 -- 172.21.15.62:0/1628516478 shutdown_connections 2023-07-14T16:15:49.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 --2- 172.21.15.62:0/1628516478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8df010c0b0 0x7f8df010e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 --2- 172.21.15.62:0/1628516478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df0074550 0x7f8df00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 --2- 172.21.15.62:0/1628516478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0072f70 0x7f8df00733e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 -- 172.21.15.62:0/1628516478 >> 172.21.15.62:0/1628516478 conn(0x7f8df00fb0e0 msgr2=0x7f8df00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 -- 172.21.15.62:0/1628516478 shutdown_connections 2023-07-14T16:15:49.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.673+0000 7f8df85ff700 1 -- 172.21.15.62:0/1628516478 wait complete. 2023-07-14T16:15:49.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.674+0000 7f8df85ff700 1 Processor -- start 2023-07-14T16:15:49.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.674+0000 7f8df85ff700 1 -- start start 2023-07-14T16:15:49.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df85ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8df0072f70 0x7f8df019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df85ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0074550 0x7f8df019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df85ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df010c0b0 0x7f8df01a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df85ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df01a3940 con 0x7f8df0074550 2023-07-14T16:15:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df85ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df01a3a80 con 0x7f8df010c0b0 2023-07-14T16:15:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df85ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df01a3c10 con 0x7f8df0072f70 2023-07-14T16:15:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df5b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0074550 0x7f8df019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df5b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0074550 0x7f8df019c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35280/0 (socket says 172.21.15.62:35280) 2023-07-14T16:15:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df5b9a700 1 -- 172.21.15.62:0/651772924 learned_addr learned my addr 172.21.15.62:0/651772924 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df639b700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8df0072f70 0x7f8df019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df010c0b0 0x7f8df01a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df5b9a700 1 -- 172.21.15.62:0/651772924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8df0072f70 msgr2=0x7f8df019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8df0072f70 0x7f8df019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df5b9a700 1 -- 172.21.15.62:0/651772924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df010c0b0 msgr2=0x7f8df01a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df010c0b0 0x7f8df01a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.675+0000 7f8df5b9a700 1 -- 172.21.15.62:0/651772924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00095f0 con 0x7f8df0074550 2023-07-14T16:15:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.676+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0074550 0x7f8df019c300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8de00059d0 tx=0x7f8de0003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.676+0000 7f8de77fe700 1 -- 172.21.15.62:0/651772924 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de001d070 con 0x7f8df0074550 2023-07-14T16:15:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.676+0000 7f8de77fe700 1 -- 172.21.15.62:0/651772924 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8de000bd70 con 0x7f8df0074550 2023-07-14T16:15:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.676+0000 7f8df85ff700 1 -- 172.21.15.62:0/651772924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df01a3e90 con 0x7f8df0074550 2023-07-14T16:15:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.676+0000 7f8de77fe700 1 -- 172.21.15.62:0/651772924 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0022cb0 con 0x7f8df0074550 2023-07-14T16:15:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.676+0000 7f8df85ff700 1 -- 172.21.15.62:0/651772924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df01a4200 con 0x7f8df0074550 2023-07-14T16:15:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.681+0000 7f8de77fe700 1 -- 172.21.15.62:0/651772924 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8de0022470 con 0x7f8df0074550 2023-07-14T16:15:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.682+0000 7f8e0be56b80 1 Processor -- start 2023-07-14T16:15:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.682+0000 7f8e0be56b80 1 -- start start 2023-07-14T16:15:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.682+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dc28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.682+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dc36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.682+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dc40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.682+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7cc7cb0 con 0x55ceb7dc24d0 2023-07-14T16:15:49.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.682+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7d1aa40 con 0x55ceb7dc3c70 2023-07-14T16:15:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.682+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7d13ed0 con 0x55ceb7dc2e20 2023-07-14T16:15:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8de77fe700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8ddc0a1150 0x7f8ddc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df5b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dc40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df639b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dc36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df639b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dc36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56418/0 (socket says 172.21.15.62:56418) 2023-07-14T16:15:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df639b700 1 -- 172.21.15.62:0/1270093502 learned_addr learned my addr 172.21.15.62:0/1270093502 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dc28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8de77fe700 1 -- 172.21.15.62:0/651772924 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8de00cdf80 con 0x7f8df0074550 2023-07-14T16:15:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8ddc0a1150 0x7f8ddc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df639b700 1 -- 172.21.15.62:0/1270093502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 msgr2=0x55ceb7dc40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df639b700 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dc40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df639b700 1 -- 172.21.15.62:0/1270093502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 msgr2=0x55ceb7dc28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df639b700 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dc28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df639b700 1 -- 172.21.15.62:0/1270093502 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb7b41de0 con 0x55ceb7dc2e20 2023-07-14T16:15:49.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.683+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dc40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:15:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.684+0000 7f8df639b700 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dc36e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00bd30 tx=0x7f8dec0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2efe17ff9be73bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.684+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8ddc0a1150 0x7f8ddc0a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009690 tx=0x7f8de8009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.684+0000 7f8de4bfe700 1 -- 172.21.15.62:0/1270093502 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec010e10 con 0x55ceb7dc2e20 2023-07-14T16:15:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.684+0000 7f8de4bfe700 1 -- 172.21.15.62:0/1270093502 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8dec015070 con 0x55ceb7dc2e20 2023-07-14T16:15:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.684+0000 7f8de4bfe700 1 -- 172.21.15.62:0/1270093502 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec003d20 con 0x55ceb7dc2e20 2023-07-14T16:15:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1270093502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 msgr2=0x55ceb7dc36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dc36e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00bd30 tx=0x7f8dec0106b0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1270093502 shutdown_connections 2023-07-14T16:15:49.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dc40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dc28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/1270093502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dc36e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1270093502 >> 172.21.15.62:0/1270093502 conn(0x55ceb7db7da0 msgr2=0x55ceb7dbd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1270093502 shutdown_connections 2023-07-14T16:15:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.685+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1270093502 wait complete. 2023-07-14T16:15:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.686+0000 7f8e0be56b80 1 -- start start 2023-07-14T16:15:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.686+0000 7f8e0be56b80 1 Processor -- start 2023-07-14T16:15:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dcc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dcc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dd36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7cb3ac0 con 0x55ceb7dc24d0 2023-07-14T16:15:49.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7cb37c0 con 0x55ceb7dc3c70 2023-07-14T16:15:49.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7b97920 con 0x55ceb7dc2e20 2023-07-14T16:15:49.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8e0be56b80 10 client.0 _ll_register_callbacks cb 0x55ceb7cb1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:49.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df639b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dcc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df639b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dcc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35302/0 (socket says 172.21.15.62:35302) 2023-07-14T16:15:49.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df639b700 1 -- 172.21.15.62:0/2403214592 learned_addr learned my addr 172.21.15.62:0/2403214592 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dcc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dd36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df639b700 1 -- 172.21.15.62:0/2403214592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 msgr2=0x55ceb7dcc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df639b700 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dcc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df639b700 1 -- 172.21.15.62:0/2403214592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 msgr2=0x55ceb7dd36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df639b700 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dd36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.687+0000 7f8df639b700 1 -- 172.21.15.62:0/2403214592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dec00b960 con 0x55ceb7dc24d0 2023-07-14T16:15:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.688+0000 7f8df639b700 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dcc150 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00b6c0 tx=0x7f8dec010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b8a5845c166fe2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.688+0000 7f8dd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:49.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.688+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2403214592 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec020070 con 0x55ceb7dc24d0 2023-07-14T16:15:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.688+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2403214592 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8dec017070 con 0x55ceb7dc24d0 2023-07-14T16:15:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.688+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2403214592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceb7c6d4f0 con 0x55ceb7dc24d0 2023-07-14T16:15:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.688+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2403214592 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dec0053d0 con 0x55ceb7dc24d0 2023-07-14T16:15:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.688+0000 7f8e0be56b80 10 client.15486 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.688+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2403214592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ceb7b2ac60 con 0x55ceb7dc24d0 2023-07-14T16:15:49.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2403214592 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ceb7c6d4f0 con 0x55ceb7dc24d0 2023-07-14T16:15:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2403214592 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8dec011560 con 0x55ceb7dc24d0 2023-07-14T16:15:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8e0be56b80 10 client.15486 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8e0be56b80 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:15:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8e0be56b80 20 client.15486 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ceb7dd7e50 0x55ceb7dda260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8e0be56b80 4 client.15486 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:15:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2403214592 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ceb7d0ceb0 con 0x55ceb7dd7e50 2023-07-14T16:15:49.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.689+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ceb7dd7e50 0x55ceb7dda260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.690+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ceb7dd7e50 0x55ceb7dda260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505240824ecac636 server_cookie=a2d1c2f06765dd93 in_seq=0 out_seq=0 2023-07-14T16:15:49.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.690+0000 7f8dd9ffb700 10 client.15486 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:49.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.690+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2403214592 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ceb7d0ceb0 con 0x55ceb7dd7e50 2023-07-14T16:15:49.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.690+0000 7f8dd9ffb700 10 client.15486 handle_command_reply: tid=0 2023-07-14T16:15:49.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 client.15486 shutdown 2023-07-14T16:15:49.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2403214592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 msgr2=0x55ceb7dcc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dcc150 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00b6c0 tx=0x7f8dec010ef0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2403214592 shutdown_connections 2023-07-14T16:15:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ceb7dd7e50 0x55ceb7dda260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dd36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dcc150 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2403214592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dcc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8dd9ffb700 0 client.15486 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2403214592 >> 172.21.15.62:0/2403214592 conn(0x55ceb7db7da0 msgr2=0x55ceb7dbc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2403214592 shutdown_connections 2023-07-14T16:15:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2403214592 wait complete. 2023-07-14T16:15:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.691+0000 7f8e0be56b80 20 client.15486 trim_cache size 0 max 16384 2023-07-14T16:15:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8e0be56b80 1 Processor -- start 2023-07-14T16:15:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8e0be56b80 1 -- start start 2023-07-14T16:15:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dc40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc3c70 0x55ceb7ddeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7db7970 con 0x55ceb7dc3c70 2023-07-14T16:15:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7ddf0e0 con 0x55ceb7dc24d0 2023-07-14T16:15:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7dd7b20 con 0x55ceb7dc2e20 2023-07-14T16:15:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8df6b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc3c70 0x55ceb7ddeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8df6b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc3c70 0x55ceb7ddeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35314/0 (socket says 172.21.15.62:35314) 2023-07-14T16:15:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8df6b9c700 1 -- 172.21.15.62:0/1199011828 learned_addr learned my addr 172.21.15.62:0/1199011828 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.709+0000 7f8df639b700 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dc40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8df6b9c700 1 -- 172.21.15.62:0/1199011828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 msgr2=0x55ceb7dd3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8df6b9c700 1 -- 172.21.15.62:0/1199011828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc24d0 msgr2=0x55ceb7dc40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dc40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8df6b9c700 1 -- 172.21.15.62:0/1199011828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb7b68250 con 0x55ceb7dc3c70 2023-07-14T16:15:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc3c70 0x55ceb7ddeba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009c20 tx=0x7f8de8009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6a32dd6ebb6182 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8de53ff700 1 -- 172.21.15.62:0/1199011828 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8028070 con 0x55ceb7dc3c70 2023-07-14T16:15:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8de53ff700 1 -- 172.21.15.62:0/1199011828 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8de8020040 con 0x55ceb7dc3c70 2023-07-14T16:15:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.710+0000 7f8de53ff700 1 -- 172.21.15.62:0/1199011828 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de801f070 con 0x55ceb7dc3c70 2023-07-14T16:15:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1199011828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc3c70 msgr2=0x55ceb7ddeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc3c70 0x55ceb7ddeba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009c20 tx=0x7f8de8009270 comp rx=0 tx=0).stop 2023-07-14T16:15:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1199011828 shutdown_connections 2023-07-14T16:15:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc3c70 0x55ceb7ddeba0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dc40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/1199011828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1199011828 >> 172.21.15.62:0/1199011828 conn(0x55ceb7db7da0 msgr2=0x55ceb7dcae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1199011828 shutdown_connections 2023-07-14T16:15:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.711+0000 7f8e0be56b80 1 -- 172.21.15.62:0/1199011828 wait complete. 2023-07-14T16:15:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 1 -- start start 2023-07-14T16:15:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 1 Processor -- start 2023-07-14T16:15:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dd30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dcfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7dd02e0 con 0x55ceb7dc2e20 2023-07-14T16:15:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7dd0450 con 0x55ceb7dc3c70 2023-07-14T16:15:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb7dd0610 con 0x55ceb7dc24d0 2023-07-14T16:15:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.712+0000 7f8e0be56b80 10 client.0 _ll_register_callbacks cb 0x7f8de8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:15:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35328/0 (socket says 172.21.15.62:35328) 2023-07-14T16:15:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 -- 172.21.15.62:0/2052086288 learned_addr learned my addr 172.21.15.62:0/2052086288 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:15:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df6b9c700 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dcfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df639b700 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dd30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 -- 172.21.15.62:0/2052086288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc24d0 msgr2=0x55ceb7dd30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dd30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 -- 172.21.15.62:0/2052086288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 msgr2=0x55ceb7dcfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:15:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dcfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 -- 172.21.15.62:0/2052086288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de8007890 con 0x55ceb7dc2e20 2023-07-14T16:15:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8df5b9a700 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3630 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8de00cba30 tx=0x7f8de0097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b49354f7976e83 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:15:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8dd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:15:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2052086288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de006a020 con 0x55ceb7dc2e20 2023-07-14T16:15:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2052086288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8de009b3d0 con 0x55ceb7dc2e20 2023-07-14T16:15:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.713+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2052086288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dec015070 con 0x55ceb7dc2e20 2023-07-14T16:15:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2052086288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8de009f400 con 0x55ceb7dc2e20 2023-07-14T16:15:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8e0be56b80 10 client.15492 fetch_fsmap learned FSMap version 23 2023-07-14T16:15:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2052086288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ceb7b402b0 con 0x55ceb7dc2e20 2023-07-14T16:15:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2052086288 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8dec015070 con 0x55ceb7dc2e20 2023-07-14T16:15:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2052086288 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8de009b630 con 0x55ceb7dc2e20 2023-07-14T16:15:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8e0be56b80 10 client.15492 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:15:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8e0be56b80 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:15:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8e0be56b80 20 client.15492 populate_metadata read hostname 'smithi062' 2023-07-14T16:15:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ceb7e2d090 0x55ceb7e2f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:15:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8e0be56b80 4 client.15492 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:15:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2052086288 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ceb7ccf050 con 0x55ceb7e2d090 2023-07-14T16:15:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.714+0000 7f8df639b700 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ceb7e2d090 0x55ceb7e2f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:15:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.715+0000 7f8dd9ffb700 10 client.15492 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.715+0000 7f8df639b700 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ceb7e2d090 0x55ceb7e2f4a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe8f61c274ad9be server_cookie=8a3d26ca4088edbb in_seq=0 out_seq=0 2023-07-14T16:15:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.715+0000 7f8dd9ffb700 1 -- 172.21.15.62:0/2052086288 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ceb7ccf050 con 0x55ceb7e2d090 2023-07-14T16:15:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.715+0000 7f8dd9ffb700 10 client.15492 handle_command_reply: tid=0 2023-07-14T16:15:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.715+0000 7f8e0be56b80 1 client.15492 shutdown 2023-07-14T16:15:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.715+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2052086288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc2e20 msgr2=0x55ceb7dd3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.715+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3630 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8de00cba30 tx=0x7f8de0097040 comp rx=0 tx=0).stop 2023-07-14T16:15:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2052086288 shutdown_connections 2023-07-14T16:15:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ceb7e2d090 0x55ceb7e2f4a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ceb7dc3c70 0x55ceb7dcfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ceb7dc24d0 0x55ceb7dd30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8dd9ffb700 0 client.15492 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:15:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 1 --2- 172.21.15.62:0/2052086288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ceb7dc2e20 0x55ceb7dd3630 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2052086288 >> 172.21.15.62:0/2052086288 conn(0x55ceb7db7da0 msgr2=0x55ceb7dca040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2052086288 shutdown_connections 2023-07-14T16:15:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 1 -- 172.21.15.62:0/2052086288 wait complete. 2023-07-14T16:15:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8e0be56b80 20 client.15492 trim_cache size 0 max 16384 2023-07-14T16:15:49.710 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:15:49.711 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:15:49.711 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:15:49.711 INFO:teuthology.orchestra.run.smithi062.stdout: "d523ca8f-615b-4121-bf9c-f88b9ceafbb7": { 2023-07-14T16:15:49.711 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:15:49.711 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "d523ca8f-615b-4121-bf9c-f88b9ceafbb7", 2023-07-14T16:15:49.711 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:15:49.712 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:15:49.712 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:15:49.712 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:15:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8df85ff700 1 -- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8ddc0a1150 msgr2=0x7f8ddc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8df85ff700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8ddc0a1150 0x7f8ddc0a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009690 tx=0x7f8de8009620 comp rx=0 tx=0).stop 2023-07-14T16:15:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8df85ff700 1 -- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0074550 msgr2=0x7f8df019c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:15:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8df85ff700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0074550 0x7f8df019c300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8de00059d0 tx=0x7f8de0003a50 comp rx=0 tx=0).stop 2023-07-14T16:15:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8df85ff700 1 -- 172.21.15.62:0/651772924 shutdown_connections 2023-07-14T16:15:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8df85ff700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8ddc0a1150 0x7f8ddc0a3600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8df85ff700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8df010c0b0 0x7f8df01a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.716+0000 7f8df85ff700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8df0072f70 0x7f8df019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.717+0000 7f8df85ff700 1 --2- 172.21.15.62:0/651772924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8df0074550 0x7f8df019c300 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:15:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.717+0000 7f8df85ff700 1 -- 172.21.15.62:0/651772924 >> 172.21.15.62:0/651772924 conn(0x7f8df00fb0e0 msgr2=0x7f8df00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:15:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.717+0000 7f8df85ff700 1 -- 172.21.15.62:0/651772924 shutdown_connections 2023-07-14T16:15:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:15:49.717+0000 7f8df85ff700 1 -- 172.21.15.62:0/651772924 wait complete. 2023-07-14T16:15:49.716 INFO:tasks.cephfs.filesystem:scrub status for tag:d523ca8f-615b-4121-bf9c-f88b9ceafbb7 - {'path': '/', 'tag': 'd523ca8f-615b-4121-bf9c-f88b9ceafbb7', 'options': 'recursive,force'} 2023-07-14T16:15:50.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:49 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1966709105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:50.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:49 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1966709105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:50.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:49 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1966709105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:15:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:50 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:15:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:50 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:15:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:50 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:15:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:50 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:15:51.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:50 smithi164 ceph-mon[97427]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-07-14T16:15:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:50 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:15:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:50 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:15:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:50 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:15:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:50 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:15:51.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:50 smithi062 ceph-mon[95543]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-07-14T16:15:51.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:50 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:15:51.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:50 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:15:51.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:50 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:15:51.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:50 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:15:51.158 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:50 smithi081 ceph-mon[127795]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-07-14T16:15:53.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:52 smithi164 ceph-mon[97427]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-07-14T16:15:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:52 smithi062 ceph-mon[95543]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-07-14T16:15:53.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:52 smithi081 ceph-mon[127795]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-07-14T16:15:54.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:53 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:15:54.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:53 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:15:54.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:53 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:15:55.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:54 smithi164 ceph-mon[97427]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-07-14T16:15:55.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:54 smithi062 ceph-mon[95543]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-07-14T16:15:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:54 smithi081 ceph-mon[127795]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-07-14T16:15:56.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:55 smithi164 ceph-mon[97427]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2023-07-14T16:15:56.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:55 smithi062 ceph-mon[95543]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2023-07-14T16:15:56.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:55 smithi081 ceph-mon[127795]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2023-07-14T16:15:58.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:15:58 smithi062 ceph-mon[95543]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 5 op/s 2023-07-14T16:15:58.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:15:58 smithi081 ceph-mon[127795]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 5 op/s 2023-07-14T16:15:59.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:15:58 smithi164 ceph-mon[97427]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 5 op/s 2023-07-14T16:16:00.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:00 smithi062 ceph-mon[95543]: pgmap v368: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-07-14T16:16:00.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:00 smithi081 ceph-mon[127795]: pgmap v368: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-07-14T16:16:01.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:00 smithi164 ceph-mon[97427]: pgmap v368: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-07-14T16:16:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.397+0000 7ff5633ff700 1 -- 172.21.15.62:0/1980425517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c100d00 msgr2=0x7ff55c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.397+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1980425517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c100d00 0x7ff55c101150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009990 tx=0x7ff54c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.398+0000 7ff5633ff700 1 -- 172.21.15.62:0/1980425517 shutdown_connections 2023-07-14T16:16:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.398+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1980425517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff55c101690 0x7ff55c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.398+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1980425517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff55c0ffb10 0x7ff55c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.398+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1980425517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c100d00 0x7ff55c101150 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.398+0000 7ff5633ff700 1 -- 172.21.15.62:0/1980425517 >> 172.21.15.62:0/1980425517 conn(0x7ff55c0fb130 msgr2=0x7ff55c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.398+0000 7ff5633ff700 1 -- 172.21.15.62:0/1980425517 shutdown_connections 2023-07-14T16:16:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.398+0000 7ff5633ff700 1 -- 172.21.15.62:0/1980425517 wait complete. 2023-07-14T16:16:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff5633ff700 1 Processor -- start 2023-07-14T16:16:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff5633ff700 1 -- start start 2023-07-14T16:16:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff5633ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c0ffb10 0x7ff55c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff5633ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff55c100d00 0x7ff55c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff5633ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff55c101690 0x7ff55c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff5633ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1994a0 con 0x7ff55c0ffb10 2023-07-14T16:16:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff5633ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c199610 con 0x7ff55c101690 2023-07-14T16:16:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff5633ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1997d0 con 0x7ff55c100d00 2023-07-14T16:16:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff56119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c0ffb10 0x7ff55c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff56119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c0ffb10 0x7ff55c197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55608/0 (socket says 172.21.15.62:55608) 2023-07-14T16:16:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.399+0000 7ff56119b700 1 -- 172.21.15.62:0/1265370999 learned_addr learned my addr 172.21.15.62:0/1265370999 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff56199c700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff55c101690 0x7ff55c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff56099a700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff55c100d00 0x7ff55c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff56119b700 1 -- 172.21.15.62:0/1265370999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff55c100d00 msgr2=0x7ff55c197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff56119b700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff55c100d00 0x7ff55c197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff56119b700 1 -- 172.21.15.62:0/1265370999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff55c101690 msgr2=0x7ff55c198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff56119b700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff55c101690 0x7ff55c198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff56119b700 1 -- 172.21.15.62:0/1265370999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0095f0 con 0x7ff55c0ffb10 2023-07-14T16:16:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff56119b700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c0ffb10 0x7ff55c197a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff55800d980 tx=0x7ff55800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff5527fc700 1 -- 172.21.15.62:0/1265370999 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558004020 con 0x7ff55c0ffb10 2023-07-14T16:16:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff5633ff700 1 -- 172.21.15.62:0/1265370999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c199ab0 con 0x7ff55c0ffb10 2023-07-14T16:16:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff5527fc700 1 -- 172.21.15.62:0/1265370999 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff55800f690 con 0x7ff55c0ffb10 2023-07-14T16:16:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.400+0000 7ff5527fc700 1 -- 172.21.15.62:0/1265370999 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5580039e0 con 0x7ff55c0ffb10 2023-07-14T16:16:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.401+0000 7ff5633ff700 1 -- 172.21.15.62:0/1265370999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c1a0270 con 0x7ff55c0ffb10 2023-07-14T16:16:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.404+0000 7ff5527fc700 1 -- 172.21.15.62:0/1265370999 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff55800f800 con 0x7ff55c0ffb10 2023-07-14T16:16:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.405+0000 7ff576d2ab80 1 Processor -- start 2023-07-14T16:16:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.405+0000 7ff5527fc700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5480a1150 0x7ff5480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.405+0000 7ff576d2ab80 1 -- start start 2023-07-14T16:16:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.405+0000 7ff5527fc700 1 -- 172.21.15.62:0/1265370999 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff5580cc550 con 0x7ff55c0ffb10 2023-07-14T16:16:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff56199c700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5480a1150 0x7ff5480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0efc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0efd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 0x562be0efdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0e01c50 con 0x562be0efc840 2023-07-14T16:16:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0e549e0 con 0x562be0d586b0 2023-07-14T16:16:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0e4de70 con 0x562be0efd6e0 2023-07-14T16:16:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff56099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0efd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff56099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0efd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55610/0 (socket says 172.21.15.62:55610) 2023-07-14T16:16:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff56099a700 1 -- 172.21.15.62:0/4119226375 learned_addr learned my addr 172.21.15.62:0/4119226375 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff56199c700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5480a1150 0x7ff5480a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff55400f590 tx=0x7ff55400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.406+0000 7ff56119b700 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0efc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff56199c700 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 0x562be0efdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff56099a700 1 -- 172.21.15.62:0/4119226375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 msgr2=0x562be0efdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff56099a700 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 0x562be0efdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff56099a700 1 -- 172.21.15.62:0/4119226375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 msgr2=0x562be0efc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff56099a700 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0efc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff56099a700 1 -- 172.21.15.62:0/4119226375 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be0c7bd80 con 0x562be0efc840 2023-07-14T16:16:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff56119b700 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0efc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff56099a700 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0efd1a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009990 tx=0x7ff54c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3b408d3b2ce644 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff5477fe700 1 -- 172.21.15.62:0/4119226375 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c01d070 con 0x562be0efc840 2023-07-14T16:16:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff5477fe700 1 -- 172.21.15.62:0/4119226375 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff54c01e040 con 0x562be0efc840 2023-07-14T16:16:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.407+0000 7ff5477fe700 1 -- 172.21.15.62:0/4119226375 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00b910 con 0x562be0efc840 2023-07-14T16:16:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 -- 172.21.15.62:0/4119226375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 msgr2=0x562be0efd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0efd1a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009990 tx=0x7ff54c005f00 comp rx=0 tx=0).stop 2023-07-14T16:16:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 -- 172.21.15.62:0/4119226375 shutdown_connections 2023-07-14T16:16:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 0x562be0efdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0efc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/4119226375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0efd1a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 -- 172.21.15.62:0/4119226375 >> 172.21.15.62:0/4119226375 conn(0x562be0ef1be0 msgr2=0x562be0ef6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 -- 172.21.15.62:0/4119226375 shutdown_connections 2023-07-14T16:16:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.408+0000 7ff576d2ab80 1 -- 172.21.15.62:0/4119226375 wait complete. 2023-07-14T16:16:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.409+0000 7ff576d2ab80 1 -- start start 2023-07-14T16:16:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.409+0000 7ff576d2ab80 1 Processor -- start 2023-07-14T16:16:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.409+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f05790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.409+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0f05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.409+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 0x562be0f0cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.409+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0deda60 con 0x562be0efc840 2023-07-14T16:16:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.409+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0ded760 con 0x562be0d586b0 2023-07-14T16:16:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.409+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0cd18c0 con 0x562be0efd6e0 2023-07-14T16:16:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff576d2ab80 10 client.0 _ll_register_callbacks cb 0x562be0deb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0f05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0f05cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55622/0 (socket says 172.21.15.62:55622) 2023-07-14T16:16:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 -- 172.21.15.62:0/448587651 learned_addr learned my addr 172.21.15.62:0/448587651 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56119b700 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f05790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56199c700 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 0x562be0f0cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 -- 172.21.15.62:0/448587651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 msgr2=0x562be0f0cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 0x562be0f0cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 -- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 msgr2=0x562be0f05790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f05790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 -- 172.21.15.62:0/448587651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c108880 con 0x562be0efc840 2023-07-14T16:16:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56119b700 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f05790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff56099a700 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0f05cd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0095c0 tx=0x7ff54c005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f418da19491e45 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.410+0000 7ff544ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.411+0000 7ff544ff9700 1 -- 172.21.15.62:0/448587651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c01d030 con 0x562be0efc840 2023-07-14T16:16:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.411+0000 7ff576d2ab80 1 -- 172.21.15.62:0/448587651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562be0da7490 con 0x562be0efc840 2023-07-14T16:16:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.411+0000 7ff544ff9700 1 -- 172.21.15.62:0/448587651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff54c00bac0 con 0x562be0efc840 2023-07-14T16:16:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.411+0000 7ff544ff9700 1 -- 172.21.15.62:0/448587651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff54c003700 con 0x562be0efc840 2023-07-14T16:16:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.411+0000 7ff576d2ab80 10 client.15504 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.411+0000 7ff576d2ab80 1 -- 172.21.15.62:0/448587651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562be0c64c00 con 0x562be0efc840 2023-07-14T16:16:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.411+0000 7ff544ff9700 1 -- 172.21.15.62:0/448587651 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562be0da7490 con 0x562be0efc840 2023-07-14T16:16:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.411+0000 7ff544ff9700 1 -- 172.21.15.62:0/448587651 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff54c00f460 con 0x562be0efc840 2023-07-14T16:16:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.412+0000 7ff576d2ab80 10 client.15504 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.412+0000 7ff576d2ab80 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.412+0000 7ff576d2ab80 20 client.15504 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.412+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562be0f110a0 0x562be0f134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.412+0000 7ff576d2ab80 4 client.15504 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.412+0000 7ff576d2ab80 1 -- 172.21.15.62:0/448587651 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562be0e46e50 con 0x562be0f110a0 2023-07-14T16:16:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.412+0000 7ff56119b700 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562be0f110a0 0x562be0f134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.413+0000 7ff56119b700 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562be0f110a0 0x562be0f134b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3ac15603ee5846 server_cookie=7942b162ae0eb43f in_seq=0 out_seq=0 2023-07-14T16:16:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.413+0000 7ff544ff9700 10 client.15504 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff544ff9700 1 -- 172.21.15.62:0/448587651 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562be0e46e50 con 0x562be0f110a0 2023-07-14T16:16:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff544ff9700 10 client.15504 handle_command_reply: tid=0 2023-07-14T16:16:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 client.15504 shutdown 2023-07-14T16:16:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 -- 172.21.15.62:0/448587651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 msgr2=0x562be0f05cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0f05cd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0095c0 tx=0x7ff54c005910 comp rx=0 tx=0).stop 2023-07-14T16:16:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 -- 172.21.15.62:0/448587651 shutdown_connections 2023-07-14T16:16:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efd6e0 0x562be0f0cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562be0f110a0 0x562be0f134b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f05790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/448587651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efc840 0x562be0f05cd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff544ff9700 0 client.15504 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 -- 172.21.15.62:0/448587651 >> 172.21.15.62:0/448587651 conn(0x562be0ef1be0 msgr2=0x562be0ef64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 -- 172.21.15.62:0/448587651 shutdown_connections 2023-07-14T16:16:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 1 -- 172.21.15.62:0/448587651 wait complete. 2023-07-14T16:16:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.414+0000 7ff576d2ab80 20 client.15504 trim_cache size 0 max 16384 2023-07-14T16:16:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff576d2ab80 1 Processor -- start 2023-07-14T16:16:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff576d2ab80 1 -- start start 2023-07-14T16:16:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0d586b0 0x562be0efcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0efc840 0x562be0f0ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0ef6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0f042d0 con 0x562be0efd6e0 2023-07-14T16:16:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0f05300 con 0x562be0efc840 2023-07-14T16:16:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0ef84e0 con 0x562be0d586b0 2023-07-14T16:16:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff56199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0ef6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff56199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0ef6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55634/0 (socket says 172.21.15.62:55634) 2023-07-14T16:16:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.430+0000 7ff56199c700 1 -- 172.21.15.62:0/1597081295 learned_addr learned my addr 172.21.15.62:0/1597081295 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff56099a700 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0efc840 0x562be0f0ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff56119b700 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0d586b0 0x562be0efcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff56199c700 1 -- 172.21.15.62:0/1597081295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0d586b0 msgr2=0x562be0efcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff56199c700 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0d586b0 0x562be0efcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff56199c700 1 -- 172.21.15.62:0/1597081295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0efc840 msgr2=0x562be0f0ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff56199c700 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0efc840 0x562be0f0ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff56199c700 1 -- 172.21.15.62:0/1597081295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be0ca21f0 con 0x562be0efd6e0 2023-07-14T16:16:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff56199c700 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0ef6c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff554009410 tx=0x7ff554014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d849d5b24e4462 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff547fff700 1 -- 172.21.15.62:0/1597081295 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55400e040 con 0x562be0efd6e0 2023-07-14T16:16:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff547fff700 1 -- 172.21.15.62:0/1597081295 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff55400f040 con 0x562be0efd6e0 2023-07-14T16:16:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.431+0000 7ff547fff700 1 -- 172.21.15.62:0/1597081295 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554015670 con 0x562be0efd6e0 2023-07-14T16:16:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 -- 172.21.15.62:0/1597081295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 msgr2=0x562be0ef6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0ef6c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff554009410 tx=0x7ff554014000 comp rx=0 tx=0).stop 2023-07-14T16:16:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 -- 172.21.15.62:0/1597081295 shutdown_connections 2023-07-14T16:16:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0ef6c80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0d586b0 0x562be0efcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/1597081295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0efc840 0x562be0f0ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 -- 172.21.15.62:0/1597081295 >> 172.21.15.62:0/1597081295 conn(0x562be0ef1be0 msgr2=0x562be0f044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 -- 172.21.15.62:0/1597081295 shutdown_connections 2023-07-14T16:16:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 -- 172.21.15.62:0/1597081295 wait complete. 2023-07-14T16:16:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.432+0000 7ff576d2ab80 1 -- start start 2023-07-14T16:16:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff576d2ab80 1 Processor -- start 2023-07-14T16:16:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efc840 0x562be0f08ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff576d2ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0f0bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0f0c450 con 0x562be0efd6e0 2023-07-14T16:16:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0f0c5c0 con 0x562be0d586b0 2023-07-14T16:16:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff576d2ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be0f0c780 con 0x562be0efc840 2023-07-14T16:16:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff576d2ab80 10 client.0 _ll_register_callbacks cb 0x7ff54c005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.433+0000 7ff56199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0f0bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56119b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0f0bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55636/0 (socket says 172.21.15.62:55636) 2023-07-14T16:16:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56199c700 1 -- 172.21.15.62:0/2605584749 learned_addr learned my addr 172.21.15.62:0/2605584749 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56099a700 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efc840 0x562be0f08ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56119b700 1 -- 172.21.15.62:0/2605584749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efc840 msgr2=0x562be0f08ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56119b700 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efc840 0x562be0f08ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56119b700 1 -- 172.21.15.62:0/2605584749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 msgr2=0x562be0f0bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56119b700 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0f0bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56119b700 1 -- 172.21.15.62:0/2605584749 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5540090c0 con 0x562be0d586b0 2023-07-14T16:16:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56199c700 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0f0bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff56119b700 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f089b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff558000c00 tx=0x7ff55800d8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ea8a328286b6e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff544ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff544ff9700 1 -- 172.21.15.62:0/2605584749 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558099910 con 0x562be0d586b0 2023-07-14T16:16:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.434+0000 7ff576d2ab80 1 -- 172.21.15.62:0/2605584749 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff54c01e040 con 0x562be0d586b0 2023-07-14T16:16:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff544ff9700 1 -- 172.21.15.62:0/2605584749 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff55809d940 con 0x562be0d586b0 2023-07-14T16:16:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff544ff9700 1 -- 172.21.15.62:0/2605584749 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5580a1d60 con 0x562be0d586b0 2023-07-14T16:16:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff576d2ab80 10 client.25289 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff576d2ab80 1 -- 172.21.15.62:0/2605584749 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562be0c7a250 con 0x562be0d586b0 2023-07-14T16:16:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff544ff9700 1 -- 172.21.15.62:0/2605584749 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff54c01e040 con 0x562be0d586b0 2023-07-14T16:16:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff544ff9700 1 -- 172.21.15.62:0/2605584749 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff5580c8020 con 0x562be0d586b0 2023-07-14T16:16:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff576d2ab80 10 client.25289 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff576d2ab80 10 client.25289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff576d2ab80 20 client.25289 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562be0f65e60 0x562be0f68270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff576d2ab80 4 client.25289 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:16:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.435+0000 7ff576d2ab80 1 -- 172.21.15.62:0/2605584749 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562be0e08ff0 con 0x562be0f65e60 2023-07-14T16:16:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.436+0000 7ff56099a700 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562be0f65e60 0x562be0f68270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.436+0000 7ff56099a700 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562be0f65e60 0x562be0f68270 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d8f8721fdde8f5 server_cookie=ebb5aba2995b8863 in_seq=0 out_seq=0 2023-07-14T16:16:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.436+0000 7ff544ff9700 10 client.25289 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff544ff9700 1 -- 172.21.15.62:0/2605584749 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562be0e08ff0 con 0x562be0f65e60 2023-07-14T16:16:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff544ff9700 10 client.25289 handle_command_reply: tid=0 2023-07-14T16:16:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 client.25289 shutdown 2023-07-14T16:16:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 -- 172.21.15.62:0/2605584749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 msgr2=0x562be0f089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f089b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff558000c00 tx=0x7ff55800d8d0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 -- 172.21.15.62:0/2605584749 shutdown_connections 2023-07-14T16:16:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562be0efd6e0 0x562be0f0bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562be0f65e60 0x562be0f68270 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562be0d586b0 0x562be0f089b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 --2- 172.21.15.62:0/2605584749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562be0efc840 0x562be0f08ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff544ff9700 0 client.25289 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 -- 172.21.15.62:0/2605584749 >> 172.21.15.62:0/2605584749 conn(0x562be0ef1be0 msgr2=0x562be0f03a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 -- 172.21.15.62:0/2605584749 shutdown_connections 2023-07-14T16:16:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 1 -- 172.21.15.62:0/2605584749 wait complete. 2023-07-14T16:16:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.449+0000 7ff576d2ab80 20 client.25289 trim_cache size 0 max 16384 2023-07-14T16:16:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:16:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:16:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:16:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:16:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 -- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5480a1150 msgr2=0x7ff5480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5480a1150 0x7ff5480a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff55400f590 tx=0x7ff55400b040 comp rx=0 tx=0).stop 2023-07-14T16:16:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 -- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c0ffb10 msgr2=0x7ff55c197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c0ffb10 0x7ff55c197a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff55800d980 tx=0x7ff55800dc90 comp rx=0 tx=0).stop 2023-07-14T16:16:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 -- 172.21.15.62:0/1265370999 shutdown_connections 2023-07-14T16:16:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff5480a1150 0x7ff5480a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff55c101690 0x7ff55c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff55c0ffb10 0x7ff55c197a80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 --2- 172.21.15.62:0/1265370999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff55c100d00 0x7ff55c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 -- 172.21.15.62:0/1265370999 >> 172.21.15.62:0/1265370999 conn(0x7ff55c0fb130 msgr2=0x7ff55c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 -- 172.21.15.62:0/1265370999 shutdown_connections 2023-07-14T16:16:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:02.450+0000 7ff5633ff700 1 -- 172.21.15.62:0/1265370999 wait complete. 2023-07-14T16:16:02.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:02 smithi062 ceph-mon[95543]: pgmap v369: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-14T16:16:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:02 smithi081 ceph-mon[127795]: pgmap v369: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-14T16:16:03.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:02 smithi164 ceph-mon[97427]: pgmap v369: 97 pgs: 97 active+clean; 653 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-07-14T16:16:04.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:04 smithi062 ceph-mon[95543]: pgmap v370: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-07-14T16:16:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:04 smithi081 ceph-mon[127795]: pgmap v370: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-07-14T16:16:05.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:04 smithi164 ceph-mon[97427]: pgmap v370: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-07-14T16:16:06.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:06 smithi062 ceph-mon[95543]: pgmap v371: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2023-07-14T16:16:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:06 smithi081 ceph-mon[127795]: pgmap v371: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2023-07-14T16:16:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:06 smithi164 ceph-mon[97427]: pgmap v371: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2023-07-14T16:16:07.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:07 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:16:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:07 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:16:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:07 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:16:08.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:08 smithi062 ceph-mon[95543]: pgmap v372: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-07-14T16:16:08.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:08.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:08 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:08.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:08 smithi081 ceph-mon[127795]: pgmap v372: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-07-14T16:16:08.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:08.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:08 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:08 smithi164 ceph-mon[97427]: pgmap v372: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-07-14T16:16:09.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:09.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:08 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:10.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:10 smithi062 ceph-mon[95543]: pgmap v373: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2023-07-14T16:16:10.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:10 smithi081 ceph-mon[127795]: pgmap v373: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2023-07-14T16:16:11.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:10 smithi164 ceph-mon[97427]: pgmap v373: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2023-07-14T16:16:12.374 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:12 smithi062 ceph-mon[95543]: pgmap v374: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-07-14T16:16:12.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:12 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.407 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.408 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.408 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.408 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.408 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:12 smithi081 ceph-mon[127795]: pgmap v374: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-07-14T16:16:12.408 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.408 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:12 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:12 smithi164 ceph-mon[97427]: pgmap v374: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-07-14T16:16:12.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:12.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:12 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:13.525 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:13 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:16:13.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:13 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:16:13.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:13 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:13.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:13 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:16:13.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:13 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:16:13.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:13 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:13.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:13 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:16:13.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:13 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:16:13.654 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:13 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:16:14.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:14 smithi062 ceph-mon[95543]: pgmap v375: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-14T16:16:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:14 smithi081 ceph-mon[127795]: pgmap v375: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-14T16:16:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:14 smithi164 ceph-mon[97427]: pgmap v375: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-14T16:16:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.610+0000 7f62ddbff700 1 -- 172.21.15.62:0/1281667995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8072f70 msgr2=0x7f62d80733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.610+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1281667995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8072f70 0x7f62d80733e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098f0 tx=0x7f62cc009c00 comp rx=0 tx=0).stop 2023-07-14T16:16:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.610+0000 7f62ddbff700 1 -- 172.21.15.62:0/1281667995 shutdown_connections 2023-07-14T16:16:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.610+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1281667995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d810c0d0 0x7f62d810e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.610+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1281667995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62d8074550 0x7f62d80729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.610+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1281667995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8072f70 0x7f62d80733e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.610+0000 7f62ddbff700 1 -- 172.21.15.62:0/1281667995 >> 172.21.15.62:0/1281667995 conn(0x7f62d80fb0e0 msgr2=0x7f62d80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.610+0000 7f62ddbff700 1 -- 172.21.15.62:0/1281667995 shutdown_connections 2023-07-14T16:16:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.611+0000 7f62ddbff700 1 -- 172.21.15.62:0/1281667995 wait complete. 2023-07-14T16:16:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62ddbff700 1 Processor -- start 2023-07-14T16:16:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62ddbff700 1 -- start start 2023-07-14T16:16:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62ddbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62d8072f70 0x7f62d819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62ddbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8074550 0x7f62d819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62ddbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d810c0d0 0x7f62d819d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62ddbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d819d970 con 0x7f62d810c0d0 2023-07-14T16:16:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62ddbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d819dae0 con 0x7f62d8074550 2023-07-14T16:16:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62ddbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d819dca0 con 0x7f62d8072f70 2023-07-14T16:16:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62d7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d810c0d0 0x7f62d819d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62d7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d810c0d0 0x7f62d819d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35560/0 (socket says 172.21.15.62:35560) 2023-07-14T16:16:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62d7fff700 1 -- 172.21.15.62:0/1713319560 learned_addr learned my addr 172.21.15.62:0/1713319560 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.612+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8074550 0x7f62d819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d77fe700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62d8072f70 0x7f62d819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d7fff700 1 -- 172.21.15.62:0/1713319560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62d8072f70 msgr2=0x7f62d819bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d7fff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62d8072f70 0x7f62d819bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d7fff700 1 -- 172.21.15.62:0/1713319560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8074550 msgr2=0x7f62d819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d7fff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8074550 0x7f62d819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d7fff700 1 -- 172.21.15.62:0/1713319560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc009520 con 0x7f62d810c0d0 2023-07-14T16:16:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8074550 0x7f62d819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d7fff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d810c0d0 0x7f62d819d310 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c110 tx=0x7f62d000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d4ff9700 1 -- 172.21.15.62:0/1713319560 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0017070 con 0x7f62d810c0d0 2023-07-14T16:16:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d4ff9700 1 -- 172.21.15.62:0/1713319560 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62d000e050 con 0x7f62d810c0d0 2023-07-14T16:16:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62ddbff700 1 -- 172.21.15.62:0/1713319560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d819df20 con 0x7f62d810c0d0 2023-07-14T16:16:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.613+0000 7f62d4ff9700 1 -- 172.21.15.62:0/1713319560 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0007820 con 0x7f62d810c0d0 2023-07-14T16:16:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.614+0000 7f62ddbff700 1 -- 172.21.15.62:0/1713319560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d81a4740 con 0x7f62d810c0d0 2023-07-14T16:16:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.617+0000 7f62d4ff9700 1 -- 172.21.15.62:0/1713319560 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f62d00092e0 con 0x7f62d810c0d0 2023-07-14T16:16:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.618+0000 7f62f15dbb80 1 Processor -- start 2023-07-14T16:16:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.619+0000 7f62f15dbb80 1 -- start start 2023-07-14T16:16:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.619+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a215300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.619+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a215840 0x55bd8a2161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.619+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 0x55bd8a216b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.619+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a11ac50 con 0x55bd8a215840 2023-07-14T16:16:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.619+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a16d9e0 con 0x55bd8a0716b0 2023-07-14T16:16:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.619+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a166e70 con 0x55bd8a2166e0 2023-07-14T16:16:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.619+0000 7f62d4ff9700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62c00a11d0 0x7f62c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a215840 0x55bd8a2161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a215300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a215300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50904/0 (socket says 172.21.15.62:50904) 2023-07-14T16:16:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 -- 172.21.15.62:0/3766918126 learned_addr learned my addr 172.21.15.62:0/3766918126 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d4ff9700 1 -- 172.21.15.62:0/1713319560 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f62d00cbff0 con 0x7f62d810c0d0 2023-07-14T16:16:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 0x55bd8a216b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 -- 172.21.15.62:0/3766918126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 msgr2=0x55bd8a216b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 0x55bd8a216b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 -- 172.21.15.62:0/3766918126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a215840 msgr2=0x55bd8a2161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a215840 0x55bd8a2161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d6ffd700 1 -- 172.21.15.62:0/3766918126 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd89f94d80 con 0x55bd8a0716b0 2023-07-14T16:16:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d7fff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62c00a11d0 0x7f62c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.620+0000 7f62d77fe700 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a215840 0x55bd8a2161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.621+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a215300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62d8074270 tx=0x7f62cc019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a5af75acaf33d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.621+0000 7f62d7fff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62c00a11d0 0x7f62c00a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f62d0095170 tx=0x7f62d0006260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.621+0000 7f62bdffb700 1 -- 172.21.15.62:0/3766918126 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00f710 con 0x55bd8a0716b0 2023-07-14T16:16:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.621+0000 7f62bdffb700 1 -- 172.21.15.62:0/3766918126 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62cc0055a0 con 0x55bd8a0716b0 2023-07-14T16:16:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.621+0000 7f62bdffb700 1 -- 172.21.15.62:0/3766918126 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc003680 con 0x55bd8a0716b0 2023-07-14T16:16:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 -- 172.21.15.62:0/3766918126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 msgr2=0x55bd8a215300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a215300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62d8074270 tx=0x7f62cc019040 comp rx=0 tx=0).stop 2023-07-14T16:16:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 -- 172.21.15.62:0/3766918126 shutdown_connections 2023-07-14T16:16:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 0x55bd8a216b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a215300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/3766918126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a215840 0x55bd8a2161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 -- 172.21.15.62:0/3766918126 >> 172.21.15.62:0/3766918126 conn(0x55bd8a20abe0 msgr2=0x55bd8a20fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 -- 172.21.15.62:0/3766918126 shutdown_connections 2023-07-14T16:16:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 -- 172.21.15.62:0/3766918126 wait complete. 2023-07-14T16:16:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.622+0000 7f62f15dbb80 1 -- start start 2023-07-14T16:16:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.623+0000 7f62f15dbb80 1 Processor -- start 2023-07-14T16:16:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.623+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a21e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.623+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a21edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.623+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.623+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a106a60 con 0x55bd8a2166e0 2023-07-14T16:16:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.623+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a106760 con 0x55bd8a0716b0 2023-07-14T16:16:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.623+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd89fea8c0 con 0x55bd8a215840 2023-07-14T16:16:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.623+0000 7f62f15dbb80 10 client.0 _ll_register_callbacks cb 0x55bd8a104700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a21edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a21edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45160/0 (socket says 172.21.15.62:45160) 2023-07-14T16:16:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 -- 172.21.15.62:0/2124513136 learned_addr learned my addr 172.21.15.62:0/2124513136 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a21e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 -- 172.21.15.62:0/2124513136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 msgr2=0x55bd8a21e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a21e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 -- 172.21.15.62:0/2124513136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 msgr2=0x55bd8a225df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 -- 172.21.15.62:0/2124513136 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c80098e0 con 0x55bd8a215840 2023-07-14T16:16:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a21edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62c80128a0 tx=0x7f62c8012bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec1e9ba06a7fe23b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.624+0000 7f62b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:16:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.625+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2124513136 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8017070 con 0x55bd8a215840 2023-07-14T16:16:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.625+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2124513136 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f62c8004020 con 0x55bd8a215840 2023-07-14T16:16:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.625+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2124513136 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd8a0c0490 con 0x55bd8a215840 2023-07-14T16:16:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.625+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2124513136 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c8022030 con 0x55bd8a215840 2023-07-14T16:16:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.625+0000 7f62f15dbb80 10 client.25249 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.625+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2124513136 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f62cc01f040 con 0x55bd8a215840 2023-07-14T16:16:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.625+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2124513136 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bd8a0c0490 con 0x55bd8a215840 2023-07-14T16:16:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.626+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2124513136 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f62c800f640 con 0x55bd8a215840 2023-07-14T16:16:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.626+0000 7f62f15dbb80 10 client.25249 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.626+0000 7f62f15dbb80 10 client.25249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.626+0000 7f62f15dbb80 20 client.25249 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.626+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a22a780 0x55bd8a22cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.626+0000 7f62f15dbb80 4 client.25249 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.626+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2124513136 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd8a15fe50 con 0x55bd8a22a780 2023-07-14T16:16:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.626+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a22a780 0x55bd8a22cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.627+0000 7f62b6ffd700 10 client.25249 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.627+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a22a780 0x55bd8a22cb90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c283be9ba8527b server_cookie=c385de280564089c in_seq=0 out_seq=0 2023-07-14T16:16:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.627+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2124513136 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bd8a15fe50 con 0x55bd8a22a780 2023-07-14T16:16:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.627+0000 7f62b6ffd700 10 client.25249 handle_command_reply: tid=0 2023-07-14T16:16:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.627+0000 7f62f15dbb80 1 client.25249 shutdown 2023-07-14T16:16:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2124513136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 msgr2=0x55bd8a21edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a21edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62c80128a0 tx=0x7f62c8012bb0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2124513136 shutdown_connections 2023-07-14T16:16:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a22a780 0x55bd8a22cb90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a21e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2124513136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a21edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62b6ffd700 0 client.25249 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2124513136 >> 172.21.15.62:0/2124513136 conn(0x55bd8a20abe0 msgr2=0x55bd8a20f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2124513136 shutdown_connections 2023-07-14T16:16:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2124513136 wait complete. 2023-07-14T16:16:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.628+0000 7f62f15dbb80 20 client.25249 trim_cache size 0 max 16384 2023-07-14T16:16:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.643+0000 7f62f15dbb80 1 Processor -- start 2023-07-14T16:16:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.643+0000 7f62f15dbb80 1 -- start start 2023-07-14T16:16:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a2195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a219b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a0c0490 con 0x55bd8a2166e0 2023-07-14T16:16:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a21fa60 con 0x55bd8a0716b0 2023-07-14T16:16:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a21dd20 con 0x55bd8a215840 2023-07-14T16:16:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35592/0 (socket says 172.21.15.62:35592) 2023-07-14T16:16:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a219b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62d6ffd700 1 -- 172.21.15.62:0/645921459 learned_addr learned my addr 172.21.15.62:0/645921459 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.644+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a2195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62d77fe700 1 -- 172.21.15.62:0/645921459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 msgr2=0x55bd8a2195f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62d77fe700 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a2195f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62d77fe700 1 -- 172.21.15.62:0/645921459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 msgr2=0x55bd8a225d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62d77fe700 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62d77fe700 1 -- 172.21.15.62:0/645921459 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd89f7dc00 con 0x55bd8a215840 2023-07-14T16:16:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62d77fe700 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a219b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f62c800fa20 tx=0x7f62c800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca0c4cbab2d4fd26 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62be7fc700 1 -- 172.21.15.62:0/645921459 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c802b020 con 0x55bd8a215840 2023-07-14T16:16:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62be7fc700 1 -- 172.21.15.62:0/645921459 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62c800fbc0 con 0x55bd8a215840 2023-07-14T16:16:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.645+0000 7f62be7fc700 1 -- 172.21.15.62:0/645921459 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8017040 con 0x55bd8a215840 2023-07-14T16:16:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 -- 172.21.15.62:0/645921459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 msgr2=0x55bd8a219b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a219b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f62c800fa20 tx=0x7f62c800f3c0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 -- 172.21.15.62:0/645921459 shutdown_connections 2023-07-14T16:16:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a2166e0 0x55bd8a225d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a0716b0 0x55bd8a2195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/645921459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a215840 0x55bd8a219b30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 -- 172.21.15.62:0/645921459 >> 172.21.15.62:0/645921459 conn(0x55bd8a20abe0 msgr2=0x55bd8a2314b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 -- 172.21.15.62:0/645921459 shutdown_connections 2023-07-14T16:16:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.646+0000 7f62f15dbb80 1 -- 172.21.15.62:0/645921459 wait complete. 2023-07-14T16:16:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 1 -- start start 2023-07-14T16:16:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 1 Processor -- start 2023-07-14T16:16:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a0716b0 0x55bd8a221ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a215840 0x55bd8a2220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 0x55bd8a225100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a225640 con 0x55bd8a0716b0 2023-07-14T16:16:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a2257b0 con 0x55bd8a215840 2023-07-14T16:16:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a225970 con 0x55bd8a2166e0 2023-07-14T16:16:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.647+0000 7f62f15dbb80 10 client.0 _ll_register_callbacks cb 0x7f62c8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a0716b0 0x55bd8a221ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a0716b0 0x55bd8a221ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35604/0 (socket says 172.21.15.62:35604) 2023-07-14T16:16:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2943542190 learned_addr learned my addr 172.21.15.62:0/2943542190 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a215840 0x55bd8a2220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2943542190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 msgr2=0x55bd8a225100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 0x55bd8a225100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2943542190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a215840 msgr2=0x55bd8a2220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a215840 0x55bd8a2220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2943542190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd89f94d80 con 0x55bd8a0716b0 2023-07-14T16:16:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a0716b0 0x55bd8a221ba0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009810 tx=0x7f62cc036380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3335e0d35f9fef77 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.648+0000 7f62b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2943542190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc036520 con 0x55bd8a0716b0 2023-07-14T16:16:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2943542190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f62cc0050b0 con 0x55bd8a0716b0 2023-07-14T16:16:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2943542190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62cc0055a0 con 0x55bd8a0716b0 2023-07-14T16:16:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2943542190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62cc005270 con 0x55bd8a0716b0 2023-07-14T16:16:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62f15dbb80 10 client.15528 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2943542190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f62cc01f040 con 0x55bd8a0716b0 2023-07-14T16:16:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2943542190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62cc047020 con 0x55bd8a0716b0 2023-07-14T16:16:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2943542190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f62cc015020 con 0x55bd8a0716b0 2023-07-14T16:16:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62f15dbb80 10 client.15528 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62f15dbb80 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62f15dbb80 20 client.15528 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.649+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a27f230 0x55bd8a281640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.650+0000 7f62f15dbb80 4 client.15528 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:16:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.650+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2943542190 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bd8a121ff0 con 0x55bd8a27f230 2023-07-14T16:16:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.650+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a27f230 0x55bd8a281640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.651+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a27f230 0x55bd8a281640 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc6d04c2e10d1ca server_cookie=33a78831a91f6b17 in_seq=0 out_seq=0 2023-07-14T16:16:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.651+0000 7f62b6ffd700 10 client.15528 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.657+0000 7f62b6ffd700 1 -- 172.21.15.62:0/2943542190 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bd8a121ff0 con 0x55bd8a27f230 2023-07-14T16:16:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.657+0000 7f62b6ffd700 10 client.15528 handle_command_reply: tid=0 2023-07-14T16:16:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.657+0000 7f62f15dbb80 1 client.15528 shutdown 2023-07-14T16:16:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2943542190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a0716b0 msgr2=0x55bd8a221ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a0716b0 0x55bd8a221ba0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009810 tx=0x7f62cc036380 comp rx=0 tx=0).stop 2023-07-14T16:16:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2943542190 shutdown_connections 2023-07-14T16:16:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a2166e0 0x55bd8a225100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a0716b0 0x55bd8a221ba0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a27f230 0x55bd8a281640 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 --2- 172.21.15.62:0/2943542190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a215840 0x55bd8a2220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2943542190 >> 172.21.15.62:0/2943542190 conn(0x55bd8a20abe0 msgr2=0x55bd8a2195f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62b6ffd700 0 client.15528 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2943542190 shutdown_connections 2023-07-14T16:16:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 1 -- 172.21.15.62:0/2943542190 wait complete. 2023-07-14T16:16:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.658+0000 7f62f15dbb80 20 client.15528 trim_cache size 0 max 16384 2023-07-14T16:16:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:16:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:16:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:16:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:16:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 -- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62c00a11d0 msgr2=0x7f62c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62c00a11d0 0x7f62c00a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f62d0095170 tx=0x7f62d0006260 comp rx=0 tx=0).stop 2023-07-14T16:16:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 -- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d810c0d0 msgr2=0x7f62d819d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d810c0d0 0x7f62d819d310 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c110 tx=0x7f62d000c420 comp rx=0 tx=0).stop 2023-07-14T16:16:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 -- 172.21.15.62:0/1713319560 shutdown_connections 2023-07-14T16:16:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d810c0d0 0x7f62d819d310 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62d8072f70 0x7f62d819bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62c00a11d0 0x7f62c00a3680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 --2- 172.21.15.62:0/1713319560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62d8074550 0x7f62d819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 -- 172.21.15.62:0/1713319560 >> 172.21.15.62:0/1713319560 conn(0x7f62d80fb0e0 msgr2=0x7f62d80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 -- 172.21.15.62:0/1713319560 shutdown_connections 2023-07-14T16:16:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:15.659+0000 7f62ddbff700 1 -- 172.21.15.62:0/1713319560 wait complete. 2023-07-14T16:16:16.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:16 smithi062 ceph-mon[95543]: pgmap v376: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 8 op/s 2023-07-14T16:16:16.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:16 smithi081 ceph-mon[127795]: pgmap v376: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 8 op/s 2023-07-14T16:16:17.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:16 smithi164 ceph-mon[97427]: pgmap v376: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 8 op/s 2023-07-14T16:16:18.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:18 smithi062 ceph-mon[95543]: pgmap v377: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:16:18.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:18 smithi081 ceph-mon[127795]: pgmap v377: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:16:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:18 smithi164 ceph-mon[97427]: pgmap v377: 97 pgs: 97 active+clean; 656 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:16:19.717 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:16:19.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 -- 172.21.15.62:0/1759337352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040fece0 msgr2=0x7fe3040ff150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:19.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 --2- 172.21.15.62:0/1759337352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040fece0 0x7fe3040ff150 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009990 tx=0x7fe2f8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 -- 172.21.15.62:0/1759337352 shutdown_connections 2023-07-14T16:16:19.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 --2- 172.21.15.62:0/1759337352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3040ff690 0x7fe3040ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 --2- 172.21.15.62:0/1759337352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3040fe390 0x7fe3040fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 --2- 172.21.15.62:0/1759337352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040fece0 0x7fe3040ff150 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 -- 172.21.15.62:0/1759337352 >> 172.21.15.62:0/1759337352 conn(0x7fe3040fa000 msgr2=0x7fe3040fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:19.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 -- 172.21.15.62:0/1759337352 shutdown_connections 2023-07-14T16:16:19.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.970+0000 7fe308bff700 1 -- 172.21.15.62:0/1759337352 wait complete. 2023-07-14T16:16:19.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.971+0000 7fe308bff700 1 Processor -- start 2023-07-14T16:16:19.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe308bff700 1 -- start start 2023-07-14T16:16:19.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe308bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3040fe390 0x7fe3041979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe308bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3040fece0 0x7fe304197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe308bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040ff690 0x7fe30419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe308bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30419f530 con 0x7fe3040ff690 2023-07-14T16:16:19.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe308bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30419f670 con 0x7fe3040fece0 2023-07-14T16:16:19.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe308bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30419f800 con 0x7fe3040fe390 2023-07-14T16:16:19.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe302d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040ff690 0x7fe30419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe302d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040ff690 0x7fe30419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35624/0 (socket says 172.21.15.62:35624) 2023-07-14T16:16:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe302d9d700 1 -- 172.21.15.62:0/277545248 learned_addr learned my addr 172.21.15.62:0/277545248 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe301d9b700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3040fece0 0x7fe304197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.972+0000 7fe30259c700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3040fe390 0x7fe3041979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe302d9d700 1 -- 172.21.15.62:0/277545248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3040fe390 msgr2=0x7fe3041979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe302d9d700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3040fe390 0x7fe3041979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe302d9d700 1 -- 172.21.15.62:0/277545248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3040fece0 msgr2=0x7fe304197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe302d9d700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3040fece0 0x7fe304197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe302d9d700 1 -- 172.21.15.62:0/277545248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc009c40 con 0x7fe3040ff690 2023-07-14T16:16:19.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe301d9b700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3040fece0 0x7fe304197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:19.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe302d9d700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040ff690 0x7fe30419ef60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c370 tx=0x7fe2fc00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:19.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe2f37fe700 1 -- 172.21.15.62:0/277545248 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc017070 con 0x7fe3040ff690 2023-07-14T16:16:19.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe2f37fe700 1 -- 172.21.15.62:0/277545248 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2fc00f040 con 0x7fe3040ff690 2023-07-14T16:16:19.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe2f37fe700 1 -- 172.21.15.62:0/277545248 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00e050 con 0x7fe3040ff690 2023-07-14T16:16:19.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.973+0000 7fe308bff700 1 -- 172.21.15.62:0/277545248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f80095f0 con 0x7fe3040ff690 2023-07-14T16:16:19.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.974+0000 7fe308bff700 1 -- 172.21.15.62:0/277545248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30419fdb0 con 0x7fe3040ff690 2023-07-14T16:16:19.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.977+0000 7fe2f37fe700 1 -- 172.21.15.62:0/277545248 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe2fc00cb60 con 0x7fe3040ff690 2023-07-14T16:16:19.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.979+0000 7fe2f37fe700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe2ec0a1150 0x7fe2ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.979+0000 7fe2f37fe700 1 -- 172.21.15.62:0/277545248 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe2fc0cc2c0 con 0x7fe3040ff690 2023-07-14T16:16:19.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.979+0000 7fe30259c700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe2ec0a1150 0x7fe2ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.980+0000 7fe30259c700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe2ec0a1150 0x7fe2ec0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009c60 tx=0x7fe2f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:19.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.981+0000 7fe31c58cb80 1 Processor -- start 2023-07-14T16:16:19.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.981+0000 7fe31c58cb80 1 -- start start 2023-07-14T16:16:19.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a3c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a3d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4941cb0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4994a40 con 0x55f9d4a3dc70 2023-07-14T16:16:19.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d498ded0 con 0x55f9d4a3c4d0 2023-07-14T16:16:19.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe301d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a3d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe301d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a3d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35640/0 (socket says 172.21.15.62:35640) 2023-07-14T16:16:19.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe301d9b700 1 -- 172.21.15.62:0/1729215949 learned_addr learned my addr 172.21.15.62:0/1729215949 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:19.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe302d9d700 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe30259c700 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a3c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe301d9b700 1 -- 172.21.15.62:0/1729215949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 msgr2=0x55f9d4a3c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:19.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe301d9b700 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a3c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe301d9b700 1 -- 172.21.15.62:0/1729215949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 msgr2=0x55f9d4a3e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:19.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe301d9b700 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a3e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.982+0000 7fe301d9b700 1 -- 172.21.15.62:0/1729215949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d47bbde0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.983+0000 7fe301d9b700 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a3d6e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8015000 tx=0x7fe2f8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99001fdb4ab5c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:19.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.983+0000 7fe2f0ff9700 1 -- 172.21.15.62:0/1729215949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f801d050 con 0x55f9d4a3ce20 2023-07-14T16:16:19.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.983+0000 7fe2f0ff9700 1 -- 172.21.15.62:0/1729215949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2f801e040 con 0x55f9d4a3ce20 2023-07-14T16:16:19.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.983+0000 7fe2f0ff9700 1 -- 172.21.15.62:0/1729215949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8003680 con 0x55f9d4a3ce20 2023-07-14T16:16:19.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 -- 172.21.15.62:0/1729215949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 msgr2=0x55f9d4a3d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:19.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a3d6e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8015000 tx=0x7fe2f8005c00 comp rx=0 tx=0).stop 2023-07-14T16:16:19.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 -- 172.21.15.62:0/1729215949 shutdown_connections 2023-07-14T16:16:19.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a3e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a3c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/1729215949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a3d6e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 -- 172.21.15.62:0/1729215949 >> 172.21.15.62:0/1729215949 conn(0x55f9d4a31da0 msgr2=0x55f9d4a371d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:19.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 -- 172.21.15.62:0/1729215949 shutdown_connections 2023-07-14T16:16:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.984+0000 7fe31c58cb80 1 -- 172.21.15.62:0/1729215949 wait complete. 2023-07-14T16:16:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.985+0000 7fe31c58cb80 1 -- start start 2023-07-14T16:16:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.985+0000 7fe31c58cb80 1 Processor -- start 2023-07-14T16:16:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a46150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a46690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a4d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d492dac0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d492d7c0 con 0x55f9d4a3c4d0 2023-07-14T16:16:19.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4811920 con 0x55f9d4a3dc70 2023-07-14T16:16:19.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe31c58cb80 10 client.0 _ll_register_callbacks cb 0x55f9d492b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:19.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe301d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a46690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe301d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a46690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35654/0 (socket says 172.21.15.62:35654) 2023-07-14T16:16:19.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe301d9b700 1 -- 172.21.15.62:0/2194984232 learned_addr learned my addr 172.21.15.62:0/2194984232 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:19.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.986+0000 7fe30259c700 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a46150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe301d9b700 1 -- 172.21.15.62:0/2194984232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 msgr2=0x55f9d4a4d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:19.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe301d9b700 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a4d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe301d9b700 1 -- 172.21.15.62:0/2194984232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3c4d0 msgr2=0x55f9d4a46150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe301d9b700 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a46150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe301d9b700 1 -- 172.21.15.62:0/2194984232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30410aa30 con 0x55f9d4a3ce20 2023-07-14T16:16:19.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe301d9b700 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a46690 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009990 tx=0x7fe2f8004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b8cfad1bb549b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:19.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe2e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:19.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/2194984232 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8003880 con 0x55f9d4a3ce20 2023-07-14T16:16:19.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe31c58cb80 1 -- 172.21.15.62:0/2194984232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9d48e74f0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.987+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/2194984232 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe2f80039e0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/2194984232 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2f8003ba0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe31c58cb80 10 client.15540 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:19.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe31c58cb80 1 -- 172.21.15.62:0/2194984232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f9d47a4c60 con 0x55f9d4a3ce20 2023-07-14T16:16:19.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/2194984232 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f9d48e74f0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/2194984232 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2f8023600 con 0x55f9d4a3ce20 2023-07-14T16:16:19.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe31c58cb80 10 client.15540 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:19.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe31c58cb80 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:19.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe31c58cb80 20 client.15540 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:19.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f9d4a51990 0x55f9d4a53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe31c58cb80 4 client.15540 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:19.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.988+0000 7fe31c58cb80 1 -- 172.21.15.62:0/2194984232 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9d4986eb0 con 0x55f9d4a51990 2023-07-14T16:16:19.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.989+0000 7fe30259c700 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f9d4a51990 0x55f9d4a53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.989+0000 7fe2e9ffb700 10 client.15540 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:19.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.989+0000 7fe30259c700 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f9d4a51990 0x55f9d4a53da0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab230159e1d22ec0 server_cookie=8ba5a635a1f1c6c0 in_seq=0 out_seq=0 2023-07-14T16:16:19.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.990+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/2194984232 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f9d4986eb0 con 0x55f9d4a51990 2023-07-14T16:16:19.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.990+0000 7fe2e9ffb700 10 client.15540 handle_command_reply: tid=0 2023-07-14T16:16:19.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.990+0000 7fe31c58cb80 1 client.15540 shutdown 2023-07-14T16:16:19.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.990+0000 7fe31c58cb80 1 -- 172.21.15.62:0/2194984232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 msgr2=0x55f9d4a46690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:19.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.990+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a46690 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009990 tx=0x7fe2f8004e40 comp rx=0 tx=0).stop 2023-07-14T16:16:19.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.990+0000 7fe31c58cb80 1 -- 172.21.15.62:0/2194984232 shutdown_connections 2023-07-14T16:16:19.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.990+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a4d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.990+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f9d4a51990 0x55f9d4a53da0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.991+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a46150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.991+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/2194984232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a46690 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.991+0000 7fe2e9ffb700 0 client.15540 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:19.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.991+0000 7fe31c58cb80 1 -- 172.21.15.62:0/2194984232 >> 172.21.15.62:0/2194984232 conn(0x55f9d4a31da0 msgr2=0x55f9d4a368b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:19.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.991+0000 7fe31c58cb80 1 -- 172.21.15.62:0/2194984232 shutdown_connections 2023-07-14T16:16:19.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.991+0000 7fe31c58cb80 1 -- 172.21.15.62:0/2194984232 wait complete. 2023-07-14T16:16:19.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:19.991+0000 7fe31c58cb80 20 client.15540 trim_cache size 0 max 16384 2023-07-14T16:16:19.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.012+0000 7fe31c58cb80 1 Processor -- start 2023-07-14T16:16:19.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.012+0000 7fe31c58cb80 1 -- start start 2023-07-14T16:16:19.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.012+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.012+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a4d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.012+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a46150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4a31970 con 0x55f9d4a3c4d0 2023-07-14T16:16:19.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4a46dd0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4a43e30 con 0x55f9d4a3dc70 2023-07-14T16:16:19.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe30259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe30259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35666/0 (socket says 172.21.15.62:35666) 2023-07-14T16:16:19.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe30259c700 1 -- 172.21.15.62:0/3267866718 learned_addr learned my addr 172.21.15.62:0/3267866718 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:19.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe302d9d700 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a46150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe301d9b700 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a4d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe30259c700 1 -- 172.21.15.62:0/3267866718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 msgr2=0x55f9d4a46150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:19.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe30259c700 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a46150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe30259c700 1 -- 172.21.15.62:0/3267866718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3ce20 msgr2=0x55f9d4a4d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:19.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe30259c700 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a4d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe30259c700 1 -- 172.21.15.62:0/3267866718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d47e2250 con 0x55f9d4a3c4d0 2023-07-14T16:16:19.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe302d9d700 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a46150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:19.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.013+0000 7fe301d9b700 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a4d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:19.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.014+0000 7fe30259c700 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4dfc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55f9d4a54680 tx=0x7fe2f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382fefe2459cfbe8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:19.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.014+0000 7fe2f17fa700 1 -- 172.21.15.62:0/3267866718 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4023040 con 0x55f9d4a3c4d0 2023-07-14T16:16:19.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.014+0000 7fe2f17fa700 1 -- 172.21.15.62:0/3267866718 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2f4004550 con 0x55f9d4a3c4d0 2023-07-14T16:16:19.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.014+0000 7fe2f17fa700 1 -- 172.21.15.62:0/3267866718 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f401d670 con 0x55f9d4a3c4d0 2023-07-14T16:16:19.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 -- 172.21.15.62:0/3267866718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3c4d0 msgr2=0x55f9d4a4dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:19.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4dfc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55f9d4a54680 tx=0x7fe2f4014000 comp rx=0 tx=0).stop 2023-07-14T16:16:19.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 -- 172.21.15.62:0/3267866718 shutdown_connections 2023-07-14T16:16:19.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a46150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4dfc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/3267866718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a4d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:19.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 -- 172.21.15.62:0/3267866718 >> 172.21.15.62:0/3267866718 conn(0x55f9d4a31da0 msgr2=0x55f9d4a44df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:19.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 -- 172.21.15.62:0/3267866718 shutdown_connections 2023-07-14T16:16:19.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 -- 172.21.15.62:0/3267866718 wait complete. 2023-07-14T16:16:19.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 -- start start 2023-07-14T16:16:19.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.015+0000 7fe31c58cb80 1 Processor -- start 2023-07-14T16:16:19.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a49e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe31c58cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a4a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:19.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4a4a8e0 con 0x55f9d4a3ce20 2023-07-14T16:16:19.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4a4aa50 con 0x55f9d4a3dc70 2023-07-14T16:16:19.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe31c58cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d4aa5840 con 0x55f9d4a3c4d0 2023-07-14T16:16:19.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe31c58cb80 10 client.0 _ll_register_callbacks cb 0x7fe2fc004ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:19.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe301d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a49e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:19.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe301d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a49e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35680/0 (socket says 172.21.15.62:35680) 2023-07-14T16:16:19.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe301d9b700 1 -- 172.21.15.62:0/71204652 learned_addr learned my addr 172.21.15.62:0/71204652 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:19.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.016+0000 7fe302d9d700 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a4a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe30259c700 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe301d9b700 1 -- 172.21.15.62:0/71204652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 msgr2=0x55f9d4a4d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe301d9b700 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe301d9b700 1 -- 172.21.15.62:0/71204652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 msgr2=0x55f9d4a4a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe301d9b700 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a4a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe301d9b700 1 -- 172.21.15.62:0/71204652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f4017070 con 0x55f9d4a3ce20 2023-07-14T16:16:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe301d9b700 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a49e60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80047e0 tx=0x7fe2f8005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e41b57c3ede40b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe2e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/71204652 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f801d070 con 0x55f9d4a3ce20 2023-07-14T16:16:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe31c58cb80 1 -- 172.21.15.62:0/71204652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2f801e040 con 0x55f9d4a3ce20 2023-07-14T16:16:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.017+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/71204652 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe2f800b2c0 con 0x55f9d4a3ce20 2023-07-14T16:16:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/71204652 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2f8023c80 con 0x55f9d4a3ce20 2023-07-14T16:16:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe31c58cb80 10 client.15546 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe31c58cb80 1 -- 172.21.15.62:0/71204652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f9d47ba2b0 con 0x55f9d4a3ce20 2023-07-14T16:16:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/71204652 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2f801e040 con 0x55f9d4a3ce20 2023-07-14T16:16:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/71204652 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2f800b940 con 0x55f9d4a3ce20 2023-07-14T16:16:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe31c58cb80 10 client.15546 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe31c58cb80 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe31c58cb80 20 client.15546 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f9d4aa6d40 0x55f9d4aa9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe31c58cb80 4 client.15546 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:16:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.018+0000 7fe31c58cb80 1 -- 172.21.15.62:0/71204652 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f9d4949050 con 0x55f9d4aa6d40 2023-07-14T16:16:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.019+0000 7fe30259c700 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f9d4aa6d40 0x55f9d4aa9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.019+0000 7fe30259c700 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f9d4aa6d40 0x55f9d4aa9120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55da37df7bb0394 server_cookie=475bd2611d33c2d5 in_seq=0 out_seq=0 2023-07-14T16:16:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.019+0000 7fe2e9ffb700 10 client.15546 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.019+0000 7fe2e9ffb700 1 -- 172.21.15.62:0/71204652 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f9d4949050 con 0x55f9d4aa6d40 2023-07-14T16:16:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.019+0000 7fe2e9ffb700 10 client.15546 handle_command_reply: tid=0 2023-07-14T16:16:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.019+0000 7fe31c58cb80 1 client.15546 shutdown 2023-07-14T16:16:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 -- 172.21.15.62:0/71204652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 msgr2=0x55f9d4a49e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a49e60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80047e0 tx=0x7fe2f8005500 comp rx=0 tx=0).stop 2023-07-14T16:16:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 -- 172.21.15.62:0/71204652 shutdown_connections 2023-07-14T16:16:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f9d4aa6d40 0x55f9d4aa9120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f9d4a3dc70 0x55f9d4a4a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f9d4a3c4d0 0x55f9d4a4d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 --2- 172.21.15.62:0/71204652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9d4a3ce20 0x55f9d4a49e60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 -- 172.21.15.62:0/71204652 >> 172.21.15.62:0/71204652 conn(0x55f9d4a31da0 msgr2=0x55f9d4a4dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe2e9ffb700 0 client.15546 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 -- 172.21.15.62:0/71204652 shutdown_connections 2023-07-14T16:16:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 1 -- 172.21.15.62:0/71204652 wait complete. 2023-07-14T16:16:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.020+0000 7fe31c58cb80 20 client.15546 trim_cache size 0 max 16384 2023-07-14T16:16:20.008 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:16:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:16:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:16:20.009 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:16:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 -- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe2ec0a1150 msgr2=0x7fe2ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe2ec0a1150 0x7fe2ec0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009c60 tx=0x7fe2f4009260 comp rx=0 tx=0).stop 2023-07-14T16:16:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 -- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040ff690 msgr2=0x7fe30419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040ff690 0x7fe30419ef60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c370 tx=0x7fe2fc00c680 comp rx=0 tx=0).stop 2023-07-14T16:16:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 -- 172.21.15.62:0/277545248 shutdown_connections 2023-07-14T16:16:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3040ff690 0x7fe30419ef60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3040fe390 0x7fe3041979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe2ec0a1150 0x7fe2ec0a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 --2- 172.21.15.62:0/277545248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3040fece0 0x7fe304197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 -- 172.21.15.62:0/277545248 >> 172.21.15.62:0/277545248 conn(0x7fe3040fa000 msgr2=0x7fe3040fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 -- 172.21.15.62:0/277545248 shutdown_connections 2023-07-14T16:16:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.021+0000 7fe308bff700 1 -- 172.21.15.62:0/277545248 wait complete. 2023-07-14T16:16:20.017 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:16:20.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:16:20.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 -- 172.21.15.62:0/1147564941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 msgr2=0x7fe1bc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/1147564941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 0x7fe1bc106e60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400b0a0 tx=0x7fe1b400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 -- 172.21.15.62:0/1147564941 shutdown_connections 2023-07-14T16:16:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/1147564941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 0x7fe1bc106e60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/1147564941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1bc0fea20 0x7fe1bc0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/1147564941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1bc0ffc10 0x7fe1bc100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 -- 172.21.15.62:0/1147564941 >> 172.21.15.62:0/1147564941 conn(0x7fe1bc0fa060 msgr2=0x7fe1bc0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 -- 172.21.15.62:0/1147564941 shutdown_connections 2023-07-14T16:16:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.253+0000 7fe1c19ff700 1 -- 172.21.15.62:0/1147564941 wait complete. 2023-07-14T16:16:20.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.254+0000 7fe1c19ff700 1 Processor -- start 2023-07-14T16:16:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.255+0000 7fe1c19ff700 1 -- start start 2023-07-14T16:16:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.255+0000 7fe1c19ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1bc0fea20 0x7fe1bc0691b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.255+0000 7fe1c19ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1bc0ffc10 0x7fe1bc19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.255+0000 7fe1c19ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 0x7fe1bc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.255+0000 7fe1c19ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc19d360 con 0x7fe1bc1005a0 2023-07-14T16:16:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.255+0000 7fe1c19ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc19d4d0 con 0x7fe1bc0ffc10 2023-07-14T16:16:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.255+0000 7fe1c19ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc19d690 con 0x7fe1bc0fea20 2023-07-14T16:16:20.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1bc0fea20 0x7fe1bc0691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 0x7fe1bc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1bc0fea20 0x7fe1bc0691b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45254/0 (socket says 172.21.15.62:45254) 2023-07-14T16:16:20.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 -- 172.21.15.62:0/3377398717 learned_addr learned my addr 172.21.15.62:0/3377398717 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:20.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1ba7fc700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1bc0ffc10 0x7fe1bc19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 -- 172.21.15.62:0/3377398717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1bc0ffc10 msgr2=0x7fe1bc19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1bc0ffc10 0x7fe1bc19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 -- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 msgr2=0x7fe1bc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 0x7fe1bc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 -- 172.21.15.62:0/3377398717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4009d00 con 0x7fe1bc0fea20 2023-07-14T16:16:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1bb7fe700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 0x7fe1bc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.256+0000 7fe1baffd700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1bc0fea20 0x7fe1bc0691b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00e970 tx=0x7fe1ac00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.257+0000 7fe1c09fd700 1 -- 172.21.15.62:0/3377398717 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac009790 con 0x7fe1bc0fea20 2023-07-14T16:16:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.257+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc19d970 con 0x7fe1bc0fea20 2023-07-14T16:16:20.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.257+0000 7fe1c09fd700 1 -- 172.21.15.62:0/3377398717 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1ac00c7e0 con 0x7fe1bc0fea20 2023-07-14T16:16:20.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.257+0000 7fe1c09fd700 1 -- 172.21.15.62:0/3377398717 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac003a80 con 0x7fe1bc0fea20 2023-07-14T16:16:20.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.257+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc19ded0 con 0x7fe1bc0fea20 2023-07-14T16:16:20.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.258+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc105f10 con 0x7fe1bc0fea20 2023-07-14T16:16:20.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.260+0000 7fe1c09fd700 1 -- 172.21.15.62:0/3377398717 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe1ac003c60 con 0x7fe1bc0fea20 2023-07-14T16:16:20.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.262+0000 7fe1c09fd700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1a40a1220 0x7fe1a40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.262+0000 7fe1c09fd700 1 -- 172.21.15.62:0/3377398717 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe1ac014070 con 0x7fe1bc0fea20 2023-07-14T16:16:20.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.262+0000 7fe1ba7fc700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1a40a1220 0x7fe1a40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.262+0000 7fe1ba7fc700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1a40a1220 0x7fe1a40a36d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc068f20 tx=0x7fe1b00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.264+0000 7fe1c09fd700 1 -- 172.21.15.62:0/3377398717 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe1ac096770 con 0x7fe1bc0fea20 2023-07-14T16:16:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.522+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1bc02c850 con 0x7fe1bc0fea20 2023-07-14T16:16:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.523+0000 7fe1c09fd700 1 -- 172.21.15.62:0/3377398717 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fe1ac099d90 con 0x7fe1bc0fea20 2023-07-14T16:16:20.500 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:16:20.500 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:16:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1a40a1220 msgr2=0x7fe1a40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1a40a1220 0x7fe1a40a36d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc068f20 tx=0x7fe1b00073d0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1bc0fea20 msgr2=0x7fe1bc0691b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1bc0fea20 0x7fe1bc0691b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00e970 tx=0x7fe1ac00ec80 comp rx=0 tx=0).stop 2023-07-14T16:16:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 shutdown_connections 2023-07-14T16:16:20.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1bc1005a0 0x7fe1bc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1bc0fea20 0x7fe1bc0691b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1a40a1220 0x7fe1a40a36d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 --2- 172.21.15.62:0/3377398717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1bc0ffc10 0x7fe1bc19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 >> 172.21.15.62:0/3377398717 conn(0x7fe1bc0fa060 msgr2=0x7fe1bc103050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.525+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 shutdown_connections 2023-07-14T16:16:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.526+0000 7fe1c19ff700 1 -- 172.21.15.62:0/3377398717 wait complete. 2023-07-14T16:16:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:16:20.523 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:16:20.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.762+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3831404397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe680fec10 msgr2=0x7fbe680ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.762+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3831404397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe680fec10 0x7fbe680ff080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe580098f0 tx=0x7fbe58009c00 comp rx=0 tx=0).stop 2023-07-14T16:16:20.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.762+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3831404397 shutdown_connections 2023-07-14T16:16:20.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.762+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3831404397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe680ff5c0 0x7fbe680ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.762+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3831404397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe680fe2c0 0x7fbe680fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.762+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3831404397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe680fec10 0x7fbe680ff080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.762+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3831404397 >> 172.21.15.62:0/3831404397 conn(0x7fbe680f9ed0 msgr2=0x7fbe680fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.763+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3831404397 shutdown_connections 2023-07-14T16:16:20.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.763+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3831404397 wait complete. 2023-07-14T16:16:20.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.764+0000 7fbe6e59d700 1 Processor -- start 2023-07-14T16:16:20.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6e59d700 1 -- start start 2023-07-14T16:16:20.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6e59d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe680fe2c0 0x7fbe68197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6e59d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe680fec10 0x7fbe68197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6e59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe680ff5c0 0x7fbe6819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6e59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819f2d0 con 0x7fbe680ff5c0 2023-07-14T16:16:20.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6e59d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819f410 con 0x7fbe680fec10 2023-07-14T16:16:20.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6e59d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819f5a0 con 0x7fbe680fe2c0 2023-07-14T16:16:20.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6dd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe680ff5c0 0x7fbe6819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6dd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe680ff5c0 0x7fbe6819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35728/0 (socket says 172.21.15.62:35728) 2023-07-14T16:16:20.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6dd9c700 1 -- 172.21.15.62:0/3313553218 learned_addr learned my addr 172.21.15.62:0/3313553218 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:20.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.765+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe680fec10 0x7fbe68197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.766+0000 7fbe6dd9c700 1 -- 172.21.15.62:0/3313553218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe680fe2c0 msgr2=0x7fbe68197750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:16:20.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.766+0000 7fbe6dd9c700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe680fe2c0 0x7fbe68197750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.766+0000 7fbe6dd9c700 1 -- 172.21.15.62:0/3313553218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe680fec10 msgr2=0x7fbe68197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.766+0000 7fbe6dd9c700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe680fec10 0x7fbe68197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.766+0000 7fbe6dd9c700 1 -- 172.21.15.62:0/3313553218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58009520 con 0x7fbe680ff5c0 2023-07-14T16:16:20.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.766+0000 7fbe6dd9c700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe680ff5c0 0x7fbe6819ed00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbe60007ee0 tx=0x7fbe6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.767+0000 7fbe5e7fc700 1 -- 172.21.15.62:0/3313553218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6000ddd0 con 0x7fbe680ff5c0 2023-07-14T16:16:20.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.767+0000 7fbe5e7fc700 1 -- 172.21.15.62:0/3313553218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe6000f040 con 0x7fbe680ff5c0 2023-07-14T16:16:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.767+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3313553218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6819f880 con 0x7fbe680ff5c0 2023-07-14T16:16:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.767+0000 7fbe5e7fc700 1 -- 172.21.15.62:0/3313553218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe600047d0 con 0x7fbe680ff5c0 2023-07-14T16:16:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.767+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3313553218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6819fde0 con 0x7fbe680ff5c0 2023-07-14T16:16:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.772+0000 7fbe5e7fc700 1 -- 172.21.15.62:0/3313553218 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbe60003a00 con 0x7fbe680ff5c0 2023-07-14T16:16:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.773+0000 7fbe8315eb80 1 Processor -- start 2023-07-14T16:16:20.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.774+0000 7fbe8315eb80 1 -- start start 2023-07-14T16:16:20.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.774+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6c8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.774+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.774+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.774+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb5c4cb0 con 0x5621eb6c9670 2023-07-14T16:16:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.774+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb617a40 con 0x5621eb6c88a0 2023-07-14T16:16:20.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.774+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb610ed0 con 0x5621eb6c7f30 2023-07-14T16:16:20.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe5e7fc700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbe540a1270 0x7fbe540a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6d59b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6d59b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51058/0 (socket says 172.21.15.62:51058) 2023-07-14T16:16:20.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6d59b700 1 -- 172.21.15.62:0/2467529860 learned_addr learned my addr 172.21.15.62:0/2467529860 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:20.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe5e7fc700 1 -- 172.21.15.62:0/3313553218 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbe600cc1d0 con 0x7fbe680ff5c0 2023-07-14T16:16:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6d59b700 1 -- 172.21.15.62:0/2467529860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 msgr2=0x5621eb6c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6d59b700 1 -- 172.21.15.62:0/2467529860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 msgr2=0x5621eb6c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6d59b700 1 -- 172.21.15.62:0/2467529860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621eb43ede0 con 0x5621eb6c88a0 2023-07-14T16:16:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.775+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.776+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6c9090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009b50 tx=0x7fbe64009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c70575e252ea15f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.776+0000 7fbe6dd9c700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbe540a1270 0x7fbe540a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.776+0000 7fbe537fe700 1 -- 172.21.15.62:0/2467529860 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6400eae0 con 0x5621eb6c88a0 2023-07-14T16:16:20.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.776+0000 7fbe537fe700 1 -- 172.21.15.62:0/2467529860 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe64004cc0 con 0x5621eb6c88a0 2023-07-14T16:16:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.776+0000 7fbe537fe700 1 -- 172.21.15.62:0/2467529860 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64005620 con 0x5621eb6c88a0 2023-07-14T16:16:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2467529860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 msgr2=0x5621eb6c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe6dd9c700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbe540a1270 0x7fbe540a3720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbe60096000 tx=0x7fbe60009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6c9090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009b50 tx=0x7fbe64009e60 comp rx=0 tx=0).stop 2023-07-14T16:16:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2467529860 shutdown_connections 2023-07-14T16:16:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2467529860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6c9090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2467529860 >> 172.21.15.62:0/2467529860 conn(0x5621eb6b4ff0 msgr2=0x5621eb6ba460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2467529860 shutdown_connections 2023-07-14T16:16:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.777+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2467529860 wait complete. 2023-07-14T16:16:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.778+0000 7fbe8315eb80 1 -- start start 2023-07-14T16:16:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.779+0000 7fbe8315eb80 1 Processor -- start 2023-07-14T16:16:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.779+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.779+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.779+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6d90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.779+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb5b0ac0 con 0x5621eb6c9670 2023-07-14T16:16:20.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.779+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb5b07c0 con 0x5621eb6c88a0 2023-07-14T16:16:20.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.779+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb494920 con 0x5621eb6c7f30 2023-07-14T16:16:20.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.779+0000 7fbe8315eb80 10 client.0 _ll_register_callbacks cb 0x5621eb5ae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:20.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6cd9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6d59b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6cd9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45284/0 (socket says 172.21.15.62:45284) 2023-07-14T16:16:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6d59b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51068/0 (socket says 172.21.15.62:51068) 2023-07-14T16:16:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/2915280569 learned_addr learned my addr 172.21.15.62:0/2915280569 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6d90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6d59b700 1 -- 172.21.15.62:0/2915280569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 msgr2=0x5621eb6d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6d59b700 1 -- 172.21.15.62:0/2915280569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 msgr2=0x5621eb6d90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6d90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.780+0000 7fbe6d59b700 1 -- 172.21.15.62:0/2915280569 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe64009800 con 0x5621eb6c88a0 2023-07-14T16:16:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.781+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d2080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009b50 tx=0x7fbe64004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a8acae132052c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.781+0000 7fbe50ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.781+0000 7fbe50ff9700 1 -- 172.21.15.62:0/2915280569 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6400eae0 con 0x5621eb6c88a0 2023-07-14T16:16:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.781+0000 7fbe50ff9700 1 -- 172.21.15.62:0/2915280569 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbe64014070 con 0x5621eb6c88a0 2023-07-14T16:16:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.781+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2915280569 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621eb56a4f0 con 0x5621eb6c88a0 2023-07-14T16:16:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe50ff9700 1 -- 172.21.15.62:0/2915280569 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe64004b50 con 0x5621eb6c88a0 2023-07-14T16:16:20.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe8315eb80 10 client.25343 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:20.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2915280569 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5621eb427c60 con 0x5621eb6c88a0 2023-07-14T16:16:20.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe50ff9700 1 -- 172.21.15.62:0/2915280569 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5621eb56a4f0 con 0x5621eb6c88a0 2023-07-14T16:16:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe50ff9700 1 -- 172.21.15.62:0/2915280569 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe64018af0 con 0x5621eb6c88a0 2023-07-14T16:16:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe8315eb80 10 client.25343 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe8315eb80 10 client.25343 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:16:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe8315eb80 20 client.25343 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5621eb6dd770 0x5621eb6dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe8315eb80 4 client.25343 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.782+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2915280569 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621eb609eb0 con 0x5621eb6dd770 2023-07-14T16:16:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.783+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5621eb6dd770 0x5621eb6dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.783+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5621eb6dd770 0x5621eb6dfb80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8ecc7159e559fd5 server_cookie=77d11f5aa581acc in_seq=0 out_seq=0 2023-07-14T16:16:20.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.783+0000 7fbe50ff9700 10 client.25343 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:20.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.784+0000 7fbe50ff9700 1 -- 172.21.15.62:0/2915280569 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5621eb609eb0 con 0x5621eb6dd770 2023-07-14T16:16:20.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.784+0000 7fbe50ff9700 10 client.25343 handle_command_reply: tid=0 2023-07-14T16:16:20.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.784+0000 7fbe8315eb80 1 client.25343 shutdown 2023-07-14T16:16:20.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2915280569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 msgr2=0x5621eb6d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d2080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009b50 tx=0x7fbe64004960 comp rx=0 tx=0).stop 2023-07-14T16:16:20.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2915280569 shutdown_connections 2023-07-14T16:16:20.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5621eb6dd770 0x5621eb6dfb80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6d90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/2915280569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d2080 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe50ff9700 0 client.25343 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:20.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2915280569 >> 172.21.15.62:0/2915280569 conn(0x5621eb6b4ff0 msgr2=0x5621eb6b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2915280569 shutdown_connections 2023-07-14T16:16:20.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 1 -- 172.21.15.62:0/2915280569 wait complete. 2023-07-14T16:16:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.785+0000 7fbe8315eb80 20 client.25343 trim_cache size 0 max 16384 2023-07-14T16:16:20.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.811+0000 7fbe8315eb80 1 Processor -- start 2023-07-14T16:16:20.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.812+0000 7fbe8315eb80 1 -- start start 2023-07-14T16:16:20.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.812+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.812+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.812+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.812+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb6d0ff0 con 0x5621eb6c9670 2023-07-14T16:16:20.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.812+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb6bb160 con 0x5621eb6c88a0 2023-07-14T16:16:20.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.812+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb6cf7e0 con 0x5621eb6c7f30 2023-07-14T16:16:20.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.812+0000 7fbe6cd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35764/0 (socket says 172.21.15.62:35764) 2023-07-14T16:16:20.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/1502882941 learned_addr learned my addr 172.21.15.62:0/1502882941 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:20.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/1502882941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 msgr2=0x5621eb6c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/1502882941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 msgr2=0x5621eb6d9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/1502882941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621eb465250 con 0x5621eb6c9670 2023-07-14T16:16:20.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.813+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6cfa30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbe580098f0 tx=0x7fbe58009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b8abe3135824ae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.814+0000 7fbe53fff700 1 -- 172.21.15.62:0/1502882941 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5801c070 con 0x5621eb6c9670 2023-07-14T16:16:20.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.814+0000 7fbe53fff700 1 -- 172.21.15.62:0/1502882941 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe5801d040 con 0x5621eb6c9670 2023-07-14T16:16:20.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.814+0000 7fbe53fff700 1 -- 172.21.15.62:0/1502882941 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58005600 con 0x5621eb6c9670 2023-07-14T16:16:20.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 -- 172.21.15.62:0/1502882941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 msgr2=0x5621eb6cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6cfa30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbe580098f0 tx=0x7fbe58009260 comp rx=0 tx=0).stop 2023-07-14T16:16:20.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 -- 172.21.15.62:0/1502882941 shutdown_connections 2023-07-14T16:16:20.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c9670 0x5621eb6cfa30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c7f30 0x5621eb6c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/1502882941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 -- 172.21.15.62:0/1502882941 >> 172.21.15.62:0/1502882941 conn(0x5621eb6b4ff0 msgr2=0x5621eb6d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 -- 172.21.15.62:0/1502882941 shutdown_connections 2023-07-14T16:16:20.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 -- 172.21.15.62:0/1502882941 wait complete. 2023-07-14T16:16:20.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.815+0000 7fbe8315eb80 1 -- start start 2023-07-14T16:16:20.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.816+0000 7fbe8315eb80 1 Processor -- start 2023-07-14T16:16:20.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.816+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c7f30 0x5621eb6d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.816+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.816+0000 7fbe8315eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c9670 0x5621eb6d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.816+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb6d5b50 con 0x5621eb6c7f30 2023-07-14T16:16:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.817+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb6d5c90 con 0x5621eb6c88a0 2023-07-14T16:16:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.817+0000 7fbe8315eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621eb6d5e50 con 0x5621eb6c9670 2023-07-14T16:16:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.817+0000 7fbe8315eb80 10 client.0 _ll_register_callbacks cb 0x7fbe64009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.817+0000 7fbe6cd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c7f30 0x5621eb6d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.817+0000 7fbe6cd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c7f30 0x5621eb6d89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35778/0 (socket says 172.21.15.62:35778) 2023-07-14T16:16:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.817+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/595388996 learned_addr learned my addr 172.21.15.62:0/595388996 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.817+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.817+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c9670 0x5621eb6d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/595388996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c9670 msgr2=0x5621eb6d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c9670 0x5621eb6d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/595388996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 msgr2=0x5621eb6d8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:20.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe6cd9a700 1 -- 172.21.15.62:0/595388996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6810a910 con 0x5621eb6c7f30 2023-07-14T16:16:20.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe6cd9a700 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c7f30 0x5621eb6d89d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbe58000c00 tx=0x7fbe58005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd952e9ec400479 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe50ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe50ff9700 1 -- 172.21.15.62:0/595388996 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5801c070 con 0x5621eb6c7f30 2023-07-14T16:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe50ff9700 1 -- 172.21.15.62:0/595388996 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbe58005380 con 0x5621eb6c7f30 2023-07-14T16:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.818+0000 7fbe8315eb80 1 -- 172.21.15.62:0/595388996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe64004cc0 con 0x5621eb6c7f30 2023-07-14T16:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.819+0000 7fbe50ff9700 1 -- 172.21.15.62:0/595388996 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe58005540 con 0x5621eb6c7f30 2023-07-14T16:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.819+0000 7fbe8315eb80 10 client.15567 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.819+0000 7fbe8315eb80 1 -- 172.21.15.62:0/595388996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5621eb43d2b0 con 0x5621eb6c7f30 2023-07-14T16:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.819+0000 7fbe50ff9700 1 -- 172.21.15.62:0/595388996 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe64004cc0 con 0x5621eb6c7f30 2023-07-14T16:16:20.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.819+0000 7fbe50ff9700 1 -- 172.21.15.62:0/595388996 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe58015020 con 0x5621eb6c7f30 2023-07-14T16:16:20.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.819+0000 7fbe8315eb80 10 client.15567 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:20.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.819+0000 7fbe8315eb80 10 client.15567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:16:20.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.819+0000 7fbe8315eb80 20 client.15567 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:20.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.820+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5621eb7326f0 0x5621eb734b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:20.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.820+0000 7fbe8315eb80 4 client.15567 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:20.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.820+0000 7fbe8315eb80 1 -- 172.21.15.62:0/595388996 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5621eb5cc050 con 0x5621eb7326f0 2023-07-14T16:16:20.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.820+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5621eb7326f0 0x5621eb734b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:20.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe6d59b700 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5621eb7326f0 0x5621eb734b00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eaa8b9b02bce7fb6 server_cookie=3d980de91d13447a in_seq=0 out_seq=0 2023-07-14T16:16:20.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe50ff9700 10 client.15567 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:20.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe50ff9700 1 -- 172.21.15.62:0/595388996 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5621eb5cc050 con 0x5621eb7326f0 2023-07-14T16:16:20.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe50ff9700 10 client.15567 handle_command_reply: tid=0 2023-07-14T16:16:20.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 client.15567 shutdown 2023-07-14T16:16:20.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 -- 172.21.15.62:0/595388996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c7f30 msgr2=0x5621eb6d89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c7f30 0x5621eb6d89d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbe58000c00 tx=0x7fbe58005040 comp rx=0 tx=0).stop 2023-07-14T16:16:20.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 -- 172.21.15.62:0/595388996 shutdown_connections 2023-07-14T16:16:20.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5621eb7326f0 0x5621eb734b00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5621eb6c9670 0x5621eb6d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eb6c7f30 0x5621eb6d89d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 --2- 172.21.15.62:0/595388996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621eb6c88a0 0x5621eb6d8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe8315eb80 1 -- 172.21.15.62:0/595388996 >> 172.21.15.62:0/595388996 conn(0x5621eb6b4ff0 msgr2=0x5621eb6cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.821+0000 7fbe50ff9700 0 client.15567 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:20.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.822+0000 7fbe8315eb80 1 -- 172.21.15.62:0/595388996 shutdown_connections 2023-07-14T16:16:20.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.822+0000 7fbe8315eb80 1 -- 172.21.15.62:0/595388996 wait complete. 2023-07-14T16:16:20.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.822+0000 7fbe8315eb80 20 client.15567 trim_cache size 0 max 16384 2023-07-14T16:16:20.806 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:20.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.822+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbe540a1270 msgr2=0x7fbe540a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbe540a1270 0x7fbe540a3720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbe60096000 tx=0x7fbe60009550 comp rx=0 tx=0).stop 2023-07-14T16:16:20.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe680ff5c0 msgr2=0x7fbe6819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:20.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe680ff5c0 0x7fbe6819ed00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbe60007ee0 tx=0x7fbe6000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3313553218 shutdown_connections 2023-07-14T16:16:20.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe680ff5c0 0x7fbe6819ed00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe680fe2c0 0x7fbe68197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbe540a1270 0x7fbe540a3720 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 --2- 172.21.15.62:0/3313553218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe680fec10 0x7fbe68197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:20.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3313553218 >> 172.21.15.62:0/3313553218 conn(0x7fbe680f9ed0 msgr2=0x7fbe680fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:20.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3313553218 shutdown_connections 2023-07-14T16:16:20.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:20.823+0000 7fbe6e59d700 1 -- 172.21.15.62:0/3313553218 wait complete. 2023-07-14T16:16:20.820 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:20 smithi062 ceph-mon[95543]: pgmap v378: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:20.820 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:20 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3377398717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:20.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:16:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:20 smithi081 ceph-mon[127795]: pgmap v378: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:20.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:20 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3377398717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:20 smithi164 ceph-mon[97427]: pgmap v378: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:21.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:20 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3377398717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:21.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 -- 172.21.15.62:0/1554883582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c100cb0 msgr2=0x7f6b1c103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 --2- 172.21.15.62:0/1554883582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c100cb0 0x7f6b1c103130 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009990 tx=0x7f6b10009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 -- 172.21.15.62:0/1554883582 shutdown_connections 2023-07-14T16:16:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 --2- 172.21.15.62:0/1554883582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b1c103670 0x7f6b1c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 --2- 172.21.15.62:0/1554883582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b1c0fe350 0x7f6b1c100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 --2- 172.21.15.62:0/1554883582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c100cb0 0x7f6b1c103130 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 -- 172.21.15.62:0/1554883582 >> 172.21.15.62:0/1554883582 conn(0x7f6b1c0fa000 msgr2=0x7f6b1c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 -- 172.21.15.62:0/1554883582 shutdown_connections 2023-07-14T16:16:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.065+0000 7f6b219ff700 1 -- 172.21.15.62:0/1554883582 wait complete. 2023-07-14T16:16:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.066+0000 7f6b219ff700 1 Processor -- start 2023-07-14T16:16:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.067+0000 7f6b219ff700 1 -- start start 2023-07-14T16:16:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.067+0000 7f6b219ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c0fe350 0x7f6b1c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.067+0000 7f6b219ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b1c100cb0 0x7f6b1c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.067+0000 7f6b219ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b1c103670 0x7f6b1c1a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.067+0000 7f6b219ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1a38f0 con 0x7f6b1c0fe350 2023-07-14T16:16:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.067+0000 7f6b219ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1a3a60 con 0x7f6b1c100cb0 2023-07-14T16:16:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.067+0000 7f6b219ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1a3c20 con 0x7f6b1c103670 2023-07-14T16:16:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b1c100cb0 0x7f6b1c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c0fe350 0x7f6b1c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b1c100cb0 0x7f6b1c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51104/0 (socket says 172.21.15.62:51104) 2023-07-14T16:16:21.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1a7fc700 1 -- 172.21.15.62:0/3524490590 learned_addr learned my addr 172.21.15.62:0/3524490590 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1b7fe700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b1c103670 0x7f6b1c1a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1affd700 1 -- 172.21.15.62:0/3524490590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b1c103670 msgr2=0x7f6b1c1a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1affd700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b1c103670 0x7f6b1c1a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1affd700 1 -- 172.21.15.62:0/3524490590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b1c100cb0 msgr2=0x7f6b1c19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1affd700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b1c100cb0 0x7f6b1c19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.068+0000 7f6b1affd700 1 -- 172.21.15.62:0/3524490590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100095f0 con 0x7f6b1c0fe350 2023-07-14T16:16:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.069+0000 7f6b1affd700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c0fe350 0x7f6b1c19bdb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e9f0 tx=0x7f6b0c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.069+0000 7f6b209fd700 1 -- 172.21.15.62:0/3524490590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00c9c0 con 0x7f6b1c0fe350 2023-07-14T16:16:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.069+0000 7f6b209fd700 1 -- 172.21.15.62:0/3524490590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b0c010430 con 0x7f6b1c0fe350 2023-07-14T16:16:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.069+0000 7f6b219ff700 1 -- 172.21.15.62:0/3524490590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c1a3f00 con 0x7f6b1c0fe350 2023-07-14T16:16:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.069+0000 7f6b209fd700 1 -- 172.21.15.62:0/3524490590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c018680 con 0x7f6b1c0fe350 2023-07-14T16:16:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.069+0000 7f6b219ff700 1 -- 172.21.15.62:0/3524490590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c1a4330 con 0x7f6b1c0fe350 2023-07-14T16:16:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.073+0000 7f6b209fd700 1 -- 172.21.15.62:0/3524490590 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6b0c0105a0 con 0x7f6b1c0fe350 2023-07-14T16:16:21.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.076+0000 7f6b209fd700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6b040a11d0 0x7f6b040a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.076+0000 7f6b209fd700 1 -- 172.21.15.62:0/3524490590 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6b0c014070 con 0x7f6b1c0fe350 2023-07-14T16:16:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.076+0000 7f6b1a7fc700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6b040a11d0 0x7f6b040a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.076+0000 7f6b35246b80 1 Processor -- start 2023-07-14T16:16:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b35246b80 1 -- start start 2023-07-14T16:16:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402c4d0 0x561e3402c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b1a7fc700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6b040a11d0 0x7f6b040a3680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006010 tx=0x7f6b1000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402ce20 0x561e3402d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e3402e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b35246b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f31cb0 con 0x561e3402c4d0 2023-07-14T16:16:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b35246b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f84a40 con 0x561e3402ce20 2023-07-14T16:16:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b35246b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f7ded0 con 0x561e3402dc70 2023-07-14T16:16:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.077+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402c4d0 0x561e3402c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402c4d0 0x561e3402c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35796/0 (socket says 172.21.15.62:35796) 2023-07-14T16:16:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1affd700 1 -- 172.21.15.62:0/2513728454 learned_addr learned my addr 172.21.15.62:0/2513728454 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1a7fc700 1 --2- 172.21.15.62:0/2513728454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402ce20 0x561e3402d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1affd700 1 -- 172.21.15.62:0/2513728454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 msgr2=0x561e3402e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1affd700 1 --2- 172.21.15.62:0/2513728454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e3402e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1affd700 1 -- 172.21.15.62:0/2513728454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402ce20 msgr2=0x561e3402d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1affd700 1 --2- 172.21.15.62:0/2513728454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402ce20 0x561e3402d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1affd700 1 -- 172.21.15.62:0/2513728454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e33dabde0 con 0x561e3402c4d0 2023-07-14T16:16:21.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.078+0000 7f6b1affd700 1 --2- 172.21.15.62:0/2513728454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402c4d0 0x561e3402c8e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c096710 tx=0x7f6b0c011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b005296385ba2d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.079+0000 7f6b01ffb700 1 -- 172.21.15.62:0/2513728454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c099600 con 0x561e3402c4d0 2023-07-14T16:16:21.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.079+0000 7f6b01ffb700 1 -- 172.21.15.62:0/2513728454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b0c010880 con 0x561e3402c4d0 2023-07-14T16:16:21.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.079+0000 7f6b01ffb700 1 -- 172.21.15.62:0/2513728454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0b2470 con 0x561e3402c4d0 2023-07-14T16:16:21.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 -- 172.21.15.62:0/2513728454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402c4d0 msgr2=0x561e3402c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2513728454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402c4d0 0x561e3402c8e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c096710 tx=0x7f6b0c011fd0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 -- 172.21.15.62:0/2513728454 shutdown_connections 2023-07-14T16:16:21.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2513728454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e3402e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2513728454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402c4d0 0x561e3402c8e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2513728454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402ce20 0x561e3402d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 -- 172.21.15.62:0/2513728454 >> 172.21.15.62:0/2513728454 conn(0x561e34021da0 msgr2=0x561e340271d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 -- 172.21.15.62:0/2513728454 shutdown_connections 2023-07-14T16:16:21.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.080+0000 7f6b35246b80 1 -- 172.21.15.62:0/2513728454 wait complete. 2023-07-14T16:16:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.081+0000 7f6b35246b80 1 -- start start 2023-07-14T16:16:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.081+0000 7f6b35246b80 1 Processor -- start 2023-07-14T16:16:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.082+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e34036180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.082+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e340366c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.082+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.082+0000 7f6b35246b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f1dac0 con 0x561e3402dc70 2023-07-14T16:16:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.082+0000 7f6b35246b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33f1d7c0 con 0x561e3402c4d0 2023-07-14T16:16:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.082+0000 7f6b35246b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33e01920 con 0x561e3402ce20 2023-07-14T16:16:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.082+0000 7f6b35246b80 10 client.0 _ll_register_callbacks cb 0x561e33f1b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35808/0 (socket says 172.21.15.62:35808) 2023-07-14T16:16:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 -- 172.21.15.62:0/1850832035 learned_addr learned my addr 172.21.15.62:0/1850832035 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1affd700 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e34036180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1a7fc700 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e340366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 -- 172.21.15.62:0/1850832035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 msgr2=0x561e340366c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e340366c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 -- 172.21.15.62:0/1850832035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 msgr2=0x561e34036180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e34036180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 -- 172.21.15.62:0/1850832035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c1a4330 con 0x561e3402dc70 2023-07-14T16:16:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6b1b7fe700 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403d700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6b14011ce0 tx=0x7f6b1400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a782669827e922 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.083+0000 7f6afaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.084+0000 7f6afaffd700 1 -- 172.21.15.62:0/1850832035 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400e040 con 0x561e3402dc70 2023-07-14T16:16:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.084+0000 7f6b35246b80 1 -- 172.21.15.62:0/1850832035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e33ed74f0 con 0x561e3402dc70 2023-07-14T16:16:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.084+0000 7f6afaffd700 1 -- 172.21.15.62:0/1850832035 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6b14003a00 con 0x561e3402dc70 2023-07-14T16:16:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.084+0000 7f6afaffd700 1 -- 172.21.15.62:0/1850832035 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b14003bc0 con 0x561e3402dc70 2023-07-14T16:16:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.084+0000 7f6b35246b80 10 client.15579 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.084+0000 7f6b35246b80 1 -- 172.21.15.62:0/1850832035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561e33d94c60 con 0x561e3402dc70 2023-07-14T16:16:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.084+0000 7f6afaffd700 1 -- 172.21.15.62:0/1850832035 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561e33ed74f0 con 0x561e3402dc70 2023-07-14T16:16:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.085+0000 7f6afaffd700 1 -- 172.21.15.62:0/1850832035 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6b140089c0 con 0x561e3402dc70 2023-07-14T16:16:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.085+0000 7f6b35246b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.085+0000 7f6b35246b80 10 client.15579 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:16:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.085+0000 7f6b35246b80 20 client.15579 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.085+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561e34041f60 0x561e34044370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.085+0000 7f6b35246b80 4 client.15579 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.085+0000 7f6b35246b80 1 -- 172.21.15.62:0/1850832035 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e33f76eb0 con 0x561e34041f60 2023-07-14T16:16:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.085+0000 7f6b1affd700 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561e34041f60 0x561e34044370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.086+0000 7f6b1affd700 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561e34041f60 0x561e34044370 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad0dc444916c8800 server_cookie=f4b9933333fbb48d in_seq=0 out_seq=0 2023-07-14T16:16:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.086+0000 7f6afaffd700 10 client.15579 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6afaffd700 1 -- 172.21.15.62:0/1850832035 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561e33f76eb0 con 0x561e34041f60 2023-07-14T16:16:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6afaffd700 10 client.15579 handle_command_reply: tid=0 2023-07-14T16:16:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 client.15579 shutdown 2023-07-14T16:16:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 -- 172.21.15.62:0/1850832035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 msgr2=0x561e3403d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403d700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6b14011ce0 tx=0x7f6b1400f1b0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 -- 172.21.15.62:0/1850832035 shutdown_connections 2023-07-14T16:16:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561e34041f60 0x561e34044370 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403d700 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e34036180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1850832035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e340366c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 -- 172.21.15.62:0/1850832035 >> 172.21.15.62:0/1850832035 conn(0x561e34021da0 msgr2=0x561e340268b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6afaffd700 0 client.15579 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 -- 172.21.15.62:0/1850832035 shutdown_connections 2023-07-14T16:16:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.087+0000 7f6b35246b80 1 -- 172.21.15.62:0/1850832035 wait complete. 2023-07-14T16:16:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.088+0000 7f6b35246b80 20 client.15579 trim_cache size 0 max 16384 2023-07-14T16:16:21.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.110+0000 7f6b35246b80 1 Processor -- start 2023-07-14T16:16:21.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.111+0000 7f6b35246b80 1 -- start start 2023-07-14T16:16:21.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.111+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3402e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.111+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402ce20 0x561e3403d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.111+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e340340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.111+0000 7f6b35246b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3403db60 con 0x561e3402ce20 2023-07-14T16:16:21.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.111+0000 7f6b35246b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e34033e80 con 0x561e3402c4d0 2023-07-14T16:16:21.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.111+0000 7f6b35246b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e34041c60 con 0x561e3402dc70 2023-07-14T16:16:21.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.111+0000 7f6b1b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e340340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3402e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e340340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45346/0 (socket says 172.21.15.62:45346) 2023-07-14T16:16:21.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3402e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51130/0 (socket says 172.21.15.62:51130) 2023-07-14T16:16:21.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1b7fe700 1 -- 172.21.15.62:0/1339714922 learned_addr learned my addr 172.21.15.62:0/1339714922 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1a7fc700 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402ce20 0x561e3403d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1b7fe700 1 -- 172.21.15.62:0/1339714922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 msgr2=0x561e3402e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1b7fe700 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3402e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1b7fe700 1 -- 172.21.15.62:0/1339714922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402ce20 msgr2=0x561e3403d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1b7fe700 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402ce20 0x561e3403d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1b7fe700 1 -- 172.21.15.62:0/1339714922 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e33dd2250 con 0x561e3402dc70 2023-07-14T16:16:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.112+0000 7f6b1affd700 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3402e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.113+0000 7f6b1b7fe700 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e340340d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b14008d50 tx=0x7f6b14008fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ccc669ecb9b2148 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.113+0000 7f6b027fc700 1 -- 172.21.15.62:0/1339714922 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400e040 con 0x561e3402dc70 2023-07-14T16:16:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.113+0000 7f6b027fc700 1 -- 172.21.15.62:0/1339714922 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b1400f040 con 0x561e3402dc70 2023-07-14T16:16:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.113+0000 7f6b027fc700 1 -- 172.21.15.62:0/1339714922 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14015c90 con 0x561e3402dc70 2023-07-14T16:16:21.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 -- 172.21.15.62:0/1339714922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 msgr2=0x561e340340d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e340340d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b14008d50 tx=0x7f6b14008fd0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 -- 172.21.15.62:0/1339714922 shutdown_connections 2023-07-14T16:16:21.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402dc70 0x561e340340d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3402e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 --2- 172.21.15.62:0/1339714922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402ce20 0x561e3403d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 -- 172.21.15.62:0/1339714922 >> 172.21.15.62:0/1339714922 conn(0x561e34021da0 msgr2=0x561e34034df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 -- 172.21.15.62:0/1339714922 shutdown_connections 2023-07-14T16:16:21.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.114+0000 7f6b35246b80 1 -- 172.21.15.62:0/1339714922 wait complete. 2023-07-14T16:16:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 1 -- start start 2023-07-14T16:16:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 1 Processor -- start 2023-07-14T16:16:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e3403b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3403ce70 con 0x561e3402dc70 2023-07-14T16:16:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3403cfe0 con 0x561e3402c4d0 2023-07-14T16:16:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3403d150 con 0x561e3402ce20 2023-07-14T16:16:21.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.115+0000 7f6b35246b80 10 client.0 _ll_register_callbacks cb 0x7f6b140115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:21.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1a7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e3403b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1a7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e3403b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45362/0 (socket says 172.21.15.62:45362) 2023-07-14T16:16:21.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1a7fc700 1 -- 172.21.15.62:0/2095182258 learned_addr learned my addr 172.21.15.62:0/2095182258 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1b7fe700 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1affd700 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1affd700 1 -- 172.21.15.62:0/2095182258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 msgr2=0x561e3403b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1affd700 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e3403b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1affd700 1 -- 172.21.15.62:0/2095182258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 msgr2=0x561e3403c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1affd700 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1affd700 1 -- 172.21.15.62:0/2095182258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e33dabde0 con 0x561e3402c4d0 2023-07-14T16:16:21.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6b1affd700 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3403b3e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00c5b0 tx=0x7f6b0c009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e782c430d651ffc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.116+0000 7f6afaffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:21.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.117+0000 7f6afaffd700 1 -- 172.21.15.62:0/2095182258 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0b2410 con 0x561e3402c4d0 2023-07-14T16:16:21.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.117+0000 7f6afaffd700 1 -- 172.21.15.62:0/2095182258 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6b0c09dcd0 con 0x561e3402c4d0 2023-07-14T16:16:21.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.117+0000 7f6b35246b80 1 -- 172.21.15.62:0/2095182258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b0c010880 con 0x561e3402c4d0 2023-07-14T16:16:21.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.117+0000 7f6afaffd700 1 -- 172.21.15.62:0/2095182258 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b0c098040 con 0x561e3402c4d0 2023-07-14T16:16:21.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.117+0000 7f6b35246b80 10 client.25367 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:21.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.117+0000 7f6b35246b80 1 -- 172.21.15.62:0/2095182258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561e33d94c60 con 0x561e3402c4d0 2023-07-14T16:16:21.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.117+0000 7f6afaffd700 1 -- 172.21.15.62:0/2095182258 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b0c07c020 con 0x561e3402c4d0 2023-07-14T16:16:21.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.117+0000 7f6afaffd700 1 -- 172.21.15.62:0/2095182258 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6b0c0a1c60 con 0x561e3402c4d0 2023-07-14T16:16:21.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.118+0000 7f6b35246b80 10 client.25367 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:21.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.118+0000 7f6b35246b80 10 client.25367 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:16:21.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.118+0000 7f6b35246b80 20 client.25367 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:21.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.118+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561e340a7b50 0x561e340a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.118+0000 7f6b35246b80 4 client.25367 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:21.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.118+0000 7f6b35246b80 1 -- 172.21.15.62:0/2095182258 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e33f39050 con 0x561e340a7b50 2023-07-14T16:16:21.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.118+0000 7f6b1a7fc700 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561e340a7b50 0x561e340a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.119+0000 7f6afaffd700 10 client.25367 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:21.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.119+0000 7f6b1a7fc700 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561e340a7b50 0x561e340a9f60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=495b554171087b47 server_cookie=f6e1717e3a425971 in_seq=0 out_seq=0 2023-07-14T16:16:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6afaffd700 1 -- 172.21.15.62:0/2095182258 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e33f39050 con 0x561e340a7b50 2023-07-14T16:16:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6afaffd700 10 client.25367 handle_command_reply: tid=0 2023-07-14T16:16:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 client.25367 shutdown 2023-07-14T16:16:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 -- 172.21.15.62:0/2095182258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 msgr2=0x561e3403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3403b3e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00c5b0 tx=0x7f6b0c009ef0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 -- 172.21.15.62:0/2095182258 shutdown_connections 2023-07-14T16:16:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561e340a7b50 0x561e340a9f60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e3402dc70 0x561e3403c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e3402c4d0 0x561e3403b3e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 --2- 172.21.15.62:0/2095182258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e3402ce20 0x561e3403b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 -- 172.21.15.62:0/2095182258 >> 172.21.15.62:0/2095182258 conn(0x561e34021da0 msgr2=0x561e34043f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6afaffd700 0 client.25367 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 -- 172.21.15.62:0/2095182258 shutdown_connections 2023-07-14T16:16:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 1 -- 172.21.15.62:0/2095182258 wait complete. 2023-07-14T16:16:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.120+0000 7f6b35246b80 20 client.25367 trim_cache size 0 max 16384 2023-07-14T16:16:21.104 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:21.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 -- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6b040a11d0 msgr2=0x7f6b040a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6b040a11d0 0x7f6b040a3680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006010 tx=0x7f6b1000b260 comp rx=0 tx=0).stop 2023-07-14T16:16:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 -- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c0fe350 msgr2=0x7f6b1c19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c0fe350 0x7f6b1c19bdb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e9f0 tx=0x7f6b0c00ed00 comp rx=0 tx=0).stop 2023-07-14T16:16:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 -- 172.21.15.62:0/3524490590 shutdown_connections 2023-07-14T16:16:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b1c103670 0x7f6b1c1a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1c0fe350 0x7f6b1c19bdb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6b040a11d0 0x7f6b040a3680 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 --2- 172.21.15.62:0/3524490590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b1c100cb0 0x7f6b1c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.121+0000 7f6b219ff700 1 -- 172.21.15.62:0/3524490590 >> 172.21.15.62:0/3524490590 conn(0x7f6b1c0fa000 msgr2=0x7f6b1c0feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.122+0000 7f6b219ff700 1 -- 172.21.15.62:0/3524490590 shutdown_connections 2023-07-14T16:16:21.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.122+0000 7f6b219ff700 1 -- 172.21.15.62:0/3524490590 wait complete. 2023-07-14T16:16:21.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:16:21.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 -- 172.21.15.62:0/3527256817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b4102e80 msgr2=0x7ff4b4105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/3527256817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b4102e80 0x7ff4b4105260 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009990 tx=0x7ff4a8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 -- 172.21.15.62:0/3527256817 shutdown_connections 2023-07-14T16:16:21.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/3527256817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b41057a0 0x7ff4b4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/3527256817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4b4100560 0x7ff4b4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/3527256817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b4102e80 0x7ff4b4105260 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 -- 172.21.15.62:0/3527256817 >> 172.21.15.62:0/3527256817 conn(0x7ff4b40fa000 msgr2=0x7ff4b40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 -- 172.21.15.62:0/3527256817 shutdown_connections 2023-07-14T16:16:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.374+0000 7ff4b95ff700 1 -- 172.21.15.62:0/3527256817 wait complete. 2023-07-14T16:16:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.376+0000 7ff4b95ff700 1 Processor -- start 2023-07-14T16:16:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.376+0000 7ff4b95ff700 1 -- start start 2023-07-14T16:16:21.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.376+0000 7ff4b95ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4b4100560 0x7ff4b4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.376+0000 7ff4b95ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b4102e80 0x7ff4b4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.376+0000 7ff4b95ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41057a0 0x7ff4b419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.376+0000 7ff4b95ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b419f4c0 con 0x7ff4b41057a0 2023-07-14T16:16:21.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b95ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b419f600 con 0x7ff4b4102e80 2023-07-14T16:16:21.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b95ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b419f790 con 0x7ff4b4100560 2023-07-14T16:16:21.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4b4100560 0x7ff4b4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4b4100560 0x7ff4b4197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45388/0 (socket says 172.21.15.62:45388) 2023-07-14T16:16:21.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/874232848 learned_addr learned my addr 172.21.15.62:0/874232848 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41057a0 0x7ff4b419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b4102e80 0x7ff4b4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b37fe700 1 -- 172.21.15.62:0/874232848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4b4100560 msgr2=0x7ff4b4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4b4100560 0x7ff4b4197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.377+0000 7ff4b37fe700 1 -- 172.21.15.62:0/874232848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b4102e80 msgr2=0x7ff4b4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b4102e80 0x7ff4b4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff4b37fe700 1 -- 172.21.15.62:0/874232848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009520 con 0x7ff4b41057a0 2023-07-14T16:16:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4b4100560 0x7ff4b4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b4102e80 0x7ff4b4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41057a0 0x7ff4b419eef0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00efc0 tx=0x7ff4ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff49bfff700 1 -- 172.21.15.62:0/874232848 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac018070 con 0x7ff4b41057a0 2023-07-14T16:16:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff49bfff700 1 -- 172.21.15.62:0/874232848 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4ac00f040 con 0x7ff4b41057a0 2023-07-14T16:16:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff49bfff700 1 -- 172.21.15.62:0/874232848 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00cba0 con 0x7ff4b41057a0 2023-07-14T16:16:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff4b95ff700 1 -- 172.21.15.62:0/874232848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a80095f0 con 0x7ff4b41057a0 2023-07-14T16:16:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.378+0000 7ff4b95ff700 1 -- 172.21.15.62:0/874232848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b419fd40 con 0x7ff4b41057a0 2023-07-14T16:16:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.383+0000 7ff4cceb1b80 1 Processor -- start 2023-07-14T16:16:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.383+0000 7ff49bfff700 1 -- 172.21.15.62:0/874232848 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff4ac003a00 con 0x7ff4b41057a0 2023-07-14T16:16:21.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.384+0000 7ff4cceb1b80 1 -- start start 2023-07-14T16:16:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.384+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.384+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b9e20 0x55c22b1ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.384+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1bac70 0x55c22b1bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.384+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b0becb0 con 0x55c22b1bac70 2023-07-14T16:16:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.384+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b111a40 con 0x55c22b1b94d0 2023-07-14T16:16:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.384+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b10aed0 con 0x55c22b1b9e20 2023-07-14T16:16:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1b98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51164/0 (socket says 172.21.15.62:51164) 2023-07-14T16:16:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b27fc700 1 -- 172.21.15.62:0/2859074940 learned_addr learned my addr 172.21.15.62:0/2859074940 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1bac70 0x55c22b1bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b9e20 0x55c22b1ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b27fc700 1 -- 172.21.15.62:0/2859074940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b9e20 msgr2=0x55c22b1ba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b9e20 0x55c22b1ba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b27fc700 1 -- 172.21.15.62:0/2859074940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1bac70 msgr2=0x55c22b1bb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.385+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1bac70 0x55c22b1bb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.386+0000 7ff4b27fc700 1 -- 172.21.15.62:0/2859074940 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c22af38de0 con 0x55c22b1b94d0 2023-07-14T16:16:21.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.386+0000 7ff49bfff700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff49c0a11d0 0x7ff49c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.386+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1b98e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005930 tx=0x7ff4a801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f57adfd4d61552b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.386+0000 7ff49bfff700 1 -- 172.21.15.62:0/874232848 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4ac0cc3f0 con 0x7ff4b41057a0 2023-07-14T16:16:21.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.386+0000 7ff4997fa700 1 -- 172.21.15.62:0/2859074940 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80041e0 con 0x55c22b1b94d0 2023-07-14T16:16:21.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.386+0000 7ff4997fa700 1 -- 172.21.15.62:0/2859074940 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4a8003d10 con 0x55c22b1b94d0 2023-07-14T16:16:21.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.386+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff49c0a11d0 0x7ff49c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.386+0000 7ff4997fa700 1 -- 172.21.15.62:0/2859074940 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800bed0 con 0x55c22b1b94d0 2023-07-14T16:16:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.387+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff49c0a11d0 0x7ff49c0a3680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4012900 tx=0x7ff4a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.387+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2859074940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 msgr2=0x55c22b1b98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.387+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1b98e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005930 tx=0x7ff4a801a040 comp rx=0 tx=0).stop 2023-07-14T16:16:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.387+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2859074940 shutdown_connections 2023-07-14T16:16:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.387+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1bac70 0x55c22b1bb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.387+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1b98e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.387+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2859074940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b9e20 0x55c22b1ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.387+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2859074940 >> 172.21.15.62:0/2859074940 conn(0x55c22b1aeda0 msgr2=0x55c22b1b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.388+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2859074940 shutdown_connections 2023-07-14T16:16:21.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.388+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2859074940 wait complete. 2023-07-14T16:16:21.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.388+0000 7ff4cceb1b80 1 -- start start 2023-07-14T16:16:21.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.389+0000 7ff4cceb1b80 1 Processor -- start 2023-07-14T16:16:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.389+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.389+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.389+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.389+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b0aaac0 con 0x55c22b1b9e20 2023-07-14T16:16:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.389+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b0aa7c0 con 0x55c22b1b94d0 2023-07-14T16:16:21.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.389+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22af8e920 con 0x55c22b1bac70 2023-07-14T16:16:21.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.389+0000 7ff4cceb1b80 10 client.0 _ll_register_callbacks cb 0x55c22b0a8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:21.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1c3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51172/0 (socket says 172.21.15.62:51172) 2023-07-14T16:16:21.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/2043608417 learned_addr learned my addr 172.21.15.62:0/2043608417 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/2043608417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 msgr2=0x55c22b1ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/2043608417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 msgr2=0x55c22b1c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/2043608417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a8020040 con 0x55c22b1b94d0 2023-07-14T16:16:21.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.390+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:21.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.391+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1c3150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400fe00 tx=0x7ff4a40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c59571e89898fd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.391+0000 7ff496ff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:21.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.391+0000 7ff496ff5700 1 -- 172.21.15.62:0/2043608417 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4023070 con 0x55c22b1b94d0 2023-07-14T16:16:21.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.391+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2043608417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c22b0644f0 con 0x55c22b1b94d0 2023-07-14T16:16:21.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.391+0000 7ff496ff5700 1 -- 172.21.15.62:0/2043608417 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4a4012dd0 con 0x55c22b1b94d0 2023-07-14T16:16:21.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.391+0000 7ff496ff5700 1 -- 172.21.15.62:0/2043608417 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4a4017070 con 0x55c22b1b94d0 2023-07-14T16:16:21.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.391+0000 7ff4cceb1b80 10 client.25379 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:21.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2043608417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c22af21c60 con 0x55c22b1b94d0 2023-07-14T16:16:21.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff496ff5700 1 -- 172.21.15.62:0/2043608417 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c22b0644f0 con 0x55c22b1b94d0 2023-07-14T16:16:21.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff496ff5700 1 -- 172.21.15.62:0/2043608417 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4a4034b80 con 0x55c22b1b94d0 2023-07-14T16:16:21.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff4cceb1b80 10 client.25379 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:21.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff4cceb1b80 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:21.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff4cceb1b80 20 client.25379 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:21.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c22b1ceeb0 0x55c22b1d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff4cceb1b80 4 client.25379 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:21.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.392+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2043608417 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c22b103eb0 con 0x55c22b1ceeb0 2023-07-14T16:16:21.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.393+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c22b1ceeb0 0x55c22b1d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.394+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c22b1ceeb0 0x55c22b1d12c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa3ad4c67006b5 server_cookie=68e01d557fccef09 in_seq=0 out_seq=0 2023-07-14T16:16:21.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.394+0000 7ff496ff5700 10 client.25379 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:21.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.394+0000 7ff496ff5700 1 -- 172.21.15.62:0/2043608417 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c22b103eb0 con 0x55c22b1ceeb0 2023-07-14T16:16:21.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.394+0000 7ff496ff5700 10 client.25379 handle_command_reply: tid=0 2023-07-14T16:16:21.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.394+0000 7ff4cceb1b80 1 client.25379 shutdown 2023-07-14T16:16:21.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2043608417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 msgr2=0x55c22b1c3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1c3150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400fe00 tx=0x7ff4a40047c0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2043608417 shutdown_connections 2023-07-14T16:16:21.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c22b1ceeb0 0x55c22b1d12c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b94d0 0x55c22b1c3150 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/2043608417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2043608417 >> 172.21.15.62:0/2043608417 conn(0x55c22b1aeda0 msgr2=0x55c22b1b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff496ff5700 0 client.25379 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:21.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2043608417 shutdown_connections 2023-07-14T16:16:21.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/2043608417 wait complete. 2023-07-14T16:16:21.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.395+0000 7ff4cceb1b80 20 client.25379 trim_cache size 0 max 16384 2023-07-14T16:16:21.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.409+0000 7ff4cceb1b80 1 Processor -- start 2023-07-14T16:16:21.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.409+0000 7ff4cceb1b80 1 -- start start 2023-07-14T16:16:21.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.409+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b94d0 0x55c22b1c2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.409+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b1b5cd0 con 0x55c22b1b94d0 2023-07-14T16:16:21.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b1cebb0 con 0x55c22b1b9e20 2023-07-14T16:16:21.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b1c2360 con 0x55c22b1bac70 2023-07-14T16:16:21.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b94d0 0x55c22b1c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b94d0 0x55c22b1c2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35876/0 (socket says 172.21.15.62:35876) 2023-07-14T16:16:21.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/1313855941 learned_addr learned my addr 172.21.15.62:0/1313855941 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/1313855941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 msgr2=0x55c22b1c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/1313855941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b9e20 msgr2=0x55c22b1ca690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:16:21.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b2ffd700 1 -- 172.21.15.62:0/1313855941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c22af5f250 con 0x55c22b1b94d0 2023-07-14T16:16:21.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.410+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:21.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.411+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b94d0 0x55c22b1c2800 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400f720 tx=0x7ff4a4012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95029edcea0f6746 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.411+0000 7ff499ffb700 1 -- 172.21.15.62:0/1313855941 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4023040 con 0x55c22b1b94d0 2023-07-14T16:16:21.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.411+0000 7ff499ffb700 1 -- 172.21.15.62:0/1313855941 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4a401c590 con 0x55c22b1b94d0 2023-07-14T16:16:21.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.411+0000 7ff499ffb700 1 -- 172.21.15.62:0/1313855941 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40043f0 con 0x55c22b1b94d0 2023-07-14T16:16:21.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/1313855941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b94d0 msgr2=0x55c22b1c2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b94d0 0x55c22b1c2800 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400f720 tx=0x7ff4a4012680 comp rx=0 tx=0).stop 2023-07-14T16:16:21.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/1313855941 shutdown_connections 2023-07-14T16:16:21.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1bac70 0x55c22b1c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b94d0 0x55c22b1c2800 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/1313855941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/1313855941 >> 172.21.15.62:0/1313855941 conn(0x55c22b1aeda0 msgr2=0x55c22b1d5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/1313855941 shutdown_connections 2023-07-14T16:16:21.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.412+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/1313855941 wait complete. 2023-07-14T16:16:21.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.413+0000 7ff4cceb1b80 1 -- start start 2023-07-14T16:16:21.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.413+0000 7ff4cceb1b80 1 Processor -- start 2023-07-14T16:16:21.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b94d0 0x55c22b1c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4cceb1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1bac70 0x55c22b1c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b1c6e70 con 0x55c22b1b9e20 2023-07-14T16:16:21.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b1c6fe0 con 0x55c22b1bac70 2023-07-14T16:16:21.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4cceb1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22b1c7150 con 0x55c22b1b94d0 2023-07-14T16:16:21.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4cceb1b80 10 client.0 _ll_register_callbacks cb 0x7ff4a400fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:21.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4b27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4b27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35888/0 (socket says 172.21.15.62:35888) 2023-07-14T16:16:21.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4b27fc700 1 -- 172.21.15.62:0/83084128 learned_addr learned my addr 172.21.15.62:0/83084128 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:21.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4b37fe700 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1bac70 0x55c22b1c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.414+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b94d0 0x55c22b1c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff4b27fc700 1 -- 172.21.15.62:0/83084128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b94d0 msgr2=0x55c22b1c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b94d0 0x55c22b1c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff4b27fc700 1 -- 172.21.15.62:0/83084128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1bac70 msgr2=0x55c22b1c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1bac70 0x55c22b1c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff4b27fc700 1 -- 172.21.15.62:0/83084128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c22af21c60 con 0x55c22b1b9e20 2023-07-14T16:16:21.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff4b27fc700 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca3d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009ce0 tx=0x7ff4a8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe46737b8d7f0f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:21.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff496ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:21.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff496ff5700 1 -- 172.21.15.62:0/83084128 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a802d040 con 0x55c22b1b9e20 2023-07-14T16:16:21.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.415+0000 7ff496ff5700 1 -- 172.21.15.62:0/83084128 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4a8005640 con 0x55c22b1b9e20 2023-07-14T16:16:21.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.416+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/83084128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4a8003d10 con 0x55c22b1b9e20 2023-07-14T16:16:21.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.416+0000 7ff496ff5700 1 -- 172.21.15.62:0/83084128 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4a8036cc0 con 0x55c22b1b9e20 2023-07-14T16:16:21.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.416+0000 7ff4cceb1b80 10 client.15603 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:21.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.416+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/83084128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c22af372b0 con 0x55c22b1b9e20 2023-07-14T16:16:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.416+0000 7ff496ff5700 1 -- 172.21.15.62:0/83084128 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4a8003d10 con 0x55c22b1b9e20 2023-07-14T16:16:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.416+0000 7ff496ff5700 1 -- 172.21.15.62:0/83084128 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4a800bac0 con 0x55c22b1b9e20 2023-07-14T16:16:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.417+0000 7ff4cceb1b80 10 client.15603 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.417+0000 7ff4cceb1b80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:21.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.417+0000 7ff4cceb1b80 20 client.15603 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:21.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.417+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c22b223d40 0x55c22b226150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:21.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.417+0000 7ff4cceb1b80 4 client.15603 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:21.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.417+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/83084128 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c22b0c6050 con 0x55c22b223d40 2023-07-14T16:16:21.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.417+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c22b223d40 0x55c22b226150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:21.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4b2ffd700 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c22b223d40 0x55c22b226150 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3ad9f1708bf0b8 server_cookie=42d693a11f935ab2 in_seq=0 out_seq=0 2023-07-14T16:16:21.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff496ff5700 10 client.15603 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:21.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff496ff5700 1 -- 172.21.15.62:0/83084128 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c22b0c6050 con 0x55c22b223d40 2023-07-14T16:16:21.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff496ff5700 10 client.15603 handle_command_reply: tid=0 2023-07-14T16:16:21.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 client.15603 shutdown 2023-07-14T16:16:21.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/83084128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 msgr2=0x55c22b1ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca3d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009ce0 tx=0x7ff4a8009290 comp rx=0 tx=0).stop 2023-07-14T16:16:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/83084128 shutdown_connections 2023-07-14T16:16:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c22b223d40 0x55c22b226150 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c22b1bac70 0x55c22b1c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c22b1b94d0 0x55c22b1c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 --2- 172.21.15.62:0/83084128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c22b1b9e20 0x55c22b1ca3d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff496ff5700 0 client.15603 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:21.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.418+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/83084128 >> 172.21.15.62:0/83084128 conn(0x55c22b1aeda0 msgr2=0x55c22b1c1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.419+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/83084128 shutdown_connections 2023-07-14T16:16:21.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.419+0000 7ff4cceb1b80 1 -- 172.21.15.62:0/83084128 wait complete. 2023-07-14T16:16:21.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.419+0000 7ff4cceb1b80 20 client.15603 trim_cache size 0 max 16384 2023-07-14T16:16:21.419 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:21.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 -- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff49c0a11d0 msgr2=0x7ff49c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff49c0a11d0 0x7ff49c0a3680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4012900 tx=0x7ff4a4009260 comp rx=0 tx=0).stop 2023-07-14T16:16:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 -- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41057a0 msgr2=0x7ff4b419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41057a0 0x7ff4b419eef0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00efc0 tx=0x7ff4ac00c3a0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 -- 172.21.15.62:0/874232848 shutdown_connections 2023-07-14T16:16:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41057a0 0x7ff4b419eef0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4b4100560 0x7ff4b4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff49c0a11d0 0x7ff49c0a3680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 --2- 172.21.15.62:0/874232848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4b4102e80 0x7ff4b4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 -- 172.21.15.62:0/874232848 >> 172.21.15.62:0/874232848 conn(0x7ff4b40fa000 msgr2=0x7ff4b40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 -- 172.21.15.62:0/874232848 shutdown_connections 2023-07-14T16:16:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:21.420+0000 7ff4b95ff700 1 -- 172.21.15.62:0/874232848 wait complete. 2023-07-14T16:16:22.423 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:16:22.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:16:22.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.662+0000 7f41f4dff700 1 -- 172.21.15.62:0/2201664581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0100560 msgr2=0x7f41f0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.662+0000 7f41f4dff700 1 --2- 172.21.15.62:0/2201664581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0100560 0x7f41f0102940 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009990 tx=0x7f41e0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.662+0000 7f41f4dff700 1 -- 172.21.15.62:0/2201664581 shutdown_connections 2023-07-14T16:16:22.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.662+0000 7f41f4dff700 1 --2- 172.21.15.62:0/2201664581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41f01057a0 0x7f41f0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.662+0000 7f41f4dff700 1 --2- 172.21.15.62:0/2201664581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0100560 0x7f41f0102940 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.663+0000 7f41f4dff700 1 --2- 172.21.15.62:0/2201664581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41f0102e80 0x7f41f0105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.663+0000 7f41f4dff700 1 -- 172.21.15.62:0/2201664581 >> 172.21.15.62:0/2201664581 conn(0x7f41f00fa000 msgr2=0x7f41f00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:22.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.663+0000 7f41f4dff700 1 -- 172.21.15.62:0/2201664581 shutdown_connections 2023-07-14T16:16:22.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.663+0000 7f41f4dff700 1 -- 172.21.15.62:0/2201664581 wait complete. 2023-07-14T16:16:22.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.664+0000 7f41f4dff700 1 Processor -- start 2023-07-14T16:16:22.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.664+0000 7f41f4dff700 1 -- start start 2023-07-14T16:16:22.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41f4dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41f0100560 0x7f41f00ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41f4dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0102e80 0x7f41f00fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41f4dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41f01057a0 0x7f41f00fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41f4dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0199630 con 0x7f41f0102e80 2023-07-14T16:16:22.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41f4dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f00fef90 con 0x7f41f01057a0 2023-07-14T16:16:22.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41f4dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f00ff150 con 0x7f41f0100560 2023-07-14T16:16:22.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41edd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0102e80 0x7f41f00fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41ee59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41f0100560 0x7f41f00ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41ee59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41f0100560 0x7f41f00ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45440/0 (socket says 172.21.15.62:45440) 2023-07-14T16:16:22.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41ee59c700 1 -- 172.21.15.62:0/1198866892 learned_addr learned my addr 172.21.15.62:0/1198866892 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:22.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.665+0000 7f41eed9d700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41f01057a0 0x7f41f00fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41edd9b700 1 -- 172.21.15.62:0/1198866892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41f0100560 msgr2=0x7f41f00ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41edd9b700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41f0100560 0x7f41f00ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41edd9b700 1 -- 172.21.15.62:0/1198866892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41f01057a0 msgr2=0x7f41f00fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41edd9b700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41f01057a0 0x7f41f00fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41edd9b700 1 -- 172.21.15.62:0/1198866892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00095f0 con 0x7f41f0102e80 2023-07-14T16:16:22.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41edd9b700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0102e80 0x7f41f00fe510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9f0 tx=0x7f41e400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41df7fe700 1 -- 172.21.15.62:0/1198866892 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400c9c0 con 0x7f41f0102e80 2023-07-14T16:16:22.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41df7fe700 1 -- 172.21.15.62:0/1198866892 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41e4010430 con 0x7f41f0102e80 2023-07-14T16:16:22.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.666+0000 7f41f4dff700 1 -- 172.21.15.62:0/1198866892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f01a3fe0 con 0x7f41f0102e80 2023-07-14T16:16:22.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.667+0000 7f41f4dff700 1 -- 172.21.15.62:0/1198866892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f01a4400 con 0x7f41f0102e80 2023-07-14T16:16:22.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.667+0000 7f41df7fe700 1 -- 172.21.15.62:0/1198866892 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4018680 con 0x7f41f0102e80 2023-07-14T16:16:22.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.671+0000 7f41df7fe700 1 -- 172.21.15.62:0/1198866892 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f41e400cb20 con 0x7f41f0102e80 2023-07-14T16:16:22.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.673+0000 7f42087a1b80 1 Processor -- start 2023-07-14T16:16:22.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.673+0000 7f42087a1b80 1 -- start start 2023-07-14T16:16:22.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.673+0000 7f41df7fe700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41d80a11d0 0x7f41d80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f41df7fe700 1 -- 172.21.15.62:0/1198866892 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41e4014070 con 0x7f41f0102e80 2023-07-14T16:16:22.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad273960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad274760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 0x562cad275160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f42087a1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad178d30 con 0x562cad273550 2023-07-14T16:16:22.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f42087a1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad1cbac0 con 0x562cad274cf0 2023-07-14T16:16:22.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f42087a1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad1c4f50 con 0x562cad273ea0 2023-07-14T16:16:22.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f41eed9d700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41d80a11d0 0x7f41d80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f41edd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad274760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f41edd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad274760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45448/0 (socket says 172.21.15.62:45448) 2023-07-14T16:16:22.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.674+0000 7f41edd9b700 1 -- 172.21.15.62:0/2744094431 learned_addr learned my addr 172.21.15.62:0/2744094431 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:22.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41ee59c700 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad273960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41edd9b700 1 -- 172.21.15.62:0/2744094431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 msgr2=0x562cad275160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41edd9b700 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 0x562cad275160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41edd9b700 1 -- 172.21.15.62:0/2744094431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 msgr2=0x562cad273960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41edd9b700 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad273960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41edd9b700 1 -- 172.21.15.62:0/2744094431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cacff2e60 con 0x562cad273ea0 2023-07-14T16:16:22.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41ee59c700 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad273960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:22.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41edd9b700 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad274760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41e4096770 tx=0x7f41e40996e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7026d7f44bf0055 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41eed9d700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41d80a11d0 0x7f41d80a3680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f41f0100530 tx=0x7f41e8009ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41dcff9700 1 -- 172.21.15.62:0/2744094431 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4099d00 con 0x562cad273ea0 2023-07-14T16:16:22.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.675+0000 7f41dcff9700 1 -- 172.21.15.62:0/2744094431 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41e409dd30 con 0x562cad273ea0 2023-07-14T16:16:22.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f41dcff9700 1 -- 172.21.15.62:0/2744094431 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e40a1f20 con 0x562cad273ea0 2023-07-14T16:16:22.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f42087a1b80 1 -- 172.21.15.62:0/2744094431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 msgr2=0x562cad274760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f42087a1b80 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad274760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41e4096770 tx=0x7f41e40996e0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f42087a1b80 1 -- 172.21.15.62:0/2744094431 shutdown_connections 2023-07-14T16:16:22.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f42087a1b80 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 0x562cad275160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f42087a1b80 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad273960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f42087a1b80 1 --2- 172.21.15.62:0/2744094431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad274760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f42087a1b80 1 -- 172.21.15.62:0/2744094431 >> 172.21.15.62:0/2744094431 conn(0x562cad268e20 msgr2=0x562cad26e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:22.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.676+0000 7f42087a1b80 1 -- 172.21.15.62:0/2744094431 shutdown_connections 2023-07-14T16:16:22.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.677+0000 7f42087a1b80 1 -- 172.21.15.62:0/2744094431 wait complete. 2023-07-14T16:16:22.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.677+0000 7f42087a1b80 1 -- start start 2023-07-14T16:16:22.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.677+0000 7f42087a1b80 1 Processor -- start 2023-07-14T16:16:22.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad27d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273ea0 0x562cad27d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad274cf0 0x562cad2846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f42087a1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad164b40 con 0x562cad273550 2023-07-14T16:16:22.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f42087a1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad164840 con 0x562cad273ea0 2023-07-14T16:16:22.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f42087a1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad0489a0 con 0x562cad274cf0 2023-07-14T16:16:22.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f42087a1b80 10 client.0 _ll_register_callbacks cb 0x562cad1627e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:22.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f41edd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273ea0 0x562cad27d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f41edd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273ea0 0x562cad27d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51228/0 (socket says 172.21.15.62:51228) 2023-07-14T16:16:22.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f41edd9b700 1 -- 172.21.15.62:0/864438220 learned_addr learned my addr 172.21.15.62:0/864438220 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:22.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.678+0000 7f41ee59c700 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad27d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41eed9d700 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad274cf0 0x562cad2846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41edd9b700 1 -- 172.21.15.62:0/864438220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad274cf0 msgr2=0x562cad2846a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41edd9b700 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad274cf0 0x562cad2846a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41edd9b700 1 -- 172.21.15.62:0/864438220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 msgr2=0x562cad27d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41edd9b700 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad27d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41edd9b700 1 -- 172.21.15.62:0/864438220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f01a4400 con 0x562cad273ea0 2023-07-14T16:16:22.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41edd9b700 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273ea0 0x562cad27d660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41e4000c00 tx=0x7f41e4099260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2070086db4b31cb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:22.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41d5ffb700 1 -- 172.21.15.62:0/864438220 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4099d00 con 0x562cad273ea0 2023-07-14T16:16:22.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f42087a1b80 1 -- 172.21.15.62:0/864438220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cad11e570 con 0x562cad273ea0 2023-07-14T16:16:22.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.679+0000 7f41d5ffb700 1 -- 172.21.15.62:0/864438220 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41e40b2270 con 0x562cad273ea0 2023-07-14T16:16:22.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f41d5ffb700 1 -- 172.21.15.62:0/864438220 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e40c9030 con 0x562cad273ea0 2023-07-14T16:16:22.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f42087a1b80 10 client.25391 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:22.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f42087a1b80 1 -- 172.21.15.62:0/864438220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562cacfdbce0 con 0x562cad273ea0 2023-07-14T16:16:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f41d5ffb700 1 -- 172.21.15.62:0/864438220 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41e407c020 con 0x562cad273ea0 2023-07-14T16:16:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f41d5ffb700 1 -- 172.21.15.62:0/864438220 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f41e4098dd0 con 0x562cad273ea0 2023-07-14T16:16:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f42087a1b80 10 client.25391 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f42087a1b80 10 client.25391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f42087a1b80 20 client.25391 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f42087a1b80 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562cad288c50 0x562cad28b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f42087a1b80 4 client.25391 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f42087a1b80 1 -- 172.21.15.62:0/864438220 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cad1bdf30 con 0x562cad288c50 2023-07-14T16:16:22.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.680+0000 7f41ee59c700 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562cad288c50 0x562cad28b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.681+0000 7f41ee59c700 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562cad288c50 0x562cad28b060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4afea60c0262d server_cookie=ef1a80e72da33f2b in_seq=0 out_seq=0 2023-07-14T16:16:22.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.681+0000 7f41d5ffb700 10 client.25391 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:22.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f41d5ffb700 1 -- 172.21.15.62:0/864438220 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562cad1bdf30 con 0x562cad288c50 2023-07-14T16:16:22.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f41d5ffb700 10 client.25391 handle_command_reply: tid=0 2023-07-14T16:16:22.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 client.25391 shutdown 2023-07-14T16:16:22.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 -- 172.21.15.62:0/864438220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273ea0 msgr2=0x562cad27d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273ea0 0x562cad27d660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41e4000c00 tx=0x7f41e4099260 comp rx=0 tx=0).stop 2023-07-14T16:16:22.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 -- 172.21.15.62:0/864438220 shutdown_connections 2023-07-14T16:16:22.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad274cf0 0x562cad2846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad27d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562cad288c50 0x562cad28b060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 --2- 172.21.15.62:0/864438220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273ea0 0x562cad27d660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 -- 172.21.15.62:0/864438220 >> 172.21.15.62:0/864438220 conn(0x562cad268e20 msgr2=0x562cad26d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:22.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f41d5ffb700 0 client.25391 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:22.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 -- 172.21.15.62:0/864438220 shutdown_connections 2023-07-14T16:16:22.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.682+0000 7f42087a1b80 1 -- 172.21.15.62:0/864438220 wait complete. 2023-07-14T16:16:22.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.683+0000 7f42087a1b80 20 client.25391 trim_cache size 0 max 16384 2023-07-14T16:16:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f42087a1b80 1 Processor -- start 2023-07-14T16:16:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f42087a1b80 1 -- start start 2023-07-14T16:16:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273550 0x562cad27b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad2845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad274cf0 0x562cad27ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f42087a1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad26f540 con 0x562cad274cf0 2023-07-14T16:16:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f42087a1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad27cc90 con 0x562cad273550 2023-07-14T16:16:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f42087a1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad26ef50 con 0x562cad273ea0 2023-07-14T16:16:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f41eed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad274cf0 0x562cad27ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f41eed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad274cf0 0x562cad27ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35932/0 (socket says 172.21.15.62:35932) 2023-07-14T16:16:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.702+0000 7f41eed9d700 1 -- 172.21.15.62:0/2496077168 learned_addr learned my addr 172.21.15.62:0/2496077168 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.703+0000 7f41edd9b700 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad2845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.703+0000 7f41ee59c700 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273550 0x562cad27b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.703+0000 7f41ee59c700 1 -- 172.21.15.62:0/2496077168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 msgr2=0x562cad2845c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.703+0000 7f41ee59c700 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad2845c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.703+0000 7f41ee59c700 1 -- 172.21.15.62:0/2496077168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad274cf0 msgr2=0x562cad27ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.703+0000 7f41ee59c700 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad274cf0 0x562cad27ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.703+0000 7f41ee59c700 1 -- 172.21.15.62:0/2496077168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cad0192d0 con 0x562cad273550 2023-07-14T16:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.703+0000 7f41ee59c700 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273550 0x562cad27b810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x562cad28b940 tx=0x7f41e0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42dfaa4348c8cb8f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.704+0000 7f41dd7fa700 1 -- 172.21.15.62:0/2496077168 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e001d070 con 0x562cad273550 2023-07-14T16:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.704+0000 7f41dd7fa700 1 -- 172.21.15.62:0/2496077168 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41e001e040 con 0x562cad273550 2023-07-14T16:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.704+0000 7f41dd7fa700 1 -- 172.21.15.62:0/2496077168 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0003680 con 0x562cad273550 2023-07-14T16:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.704+0000 7f42087a1b80 1 -- 172.21.15.62:0/2496077168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273550 msgr2=0x562cad27b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.704+0000 7f42087a1b80 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273550 0x562cad27b810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x562cad28b940 tx=0x7f41e0009310 comp rx=0 tx=0).stop 2023-07-14T16:16:22.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 -- 172.21.15.62:0/2496077168 shutdown_connections 2023-07-14T16:16:22.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad274cf0 0x562cad27ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad273550 0x562cad27b810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 --2- 172.21.15.62:0/2496077168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad2845c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 -- 172.21.15.62:0/2496077168 >> 172.21.15.62:0/2496077168 conn(0x562cad268e20 msgr2=0x562cad27b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:22.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 -- 172.21.15.62:0/2496077168 shutdown_connections 2023-07-14T16:16:22.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 -- 172.21.15.62:0/2496077168 wait complete. 2023-07-14T16:16:22.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 -- start start 2023-07-14T16:16:22.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.705+0000 7f42087a1b80 1 Processor -- start 2023-07-14T16:16:22.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.706+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad281560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.706+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad281aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.706+0000 7f42087a1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 0x562cad2dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.706+0000 7f42087a1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad2dd040 con 0x562cad273550 2023-07-14T16:16:22.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.706+0000 7f42087a1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad2dd180 con 0x562cad274cf0 2023-07-14T16:16:22.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.706+0000 7f42087a1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cad2dd310 con 0x562cad273ea0 2023-07-14T16:16:22.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.706+0000 7f42087a1b80 10 client.0 _ll_register_callbacks cb 0x7f41e4099860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:22.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad281aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad281560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad281aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45472/0 (socket says 172.21.15.62:45472) 2023-07-14T16:16:22.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 -- 172.21.15.62:0/465304191 learned_addr learned my addr 172.21.15.62:0/465304191 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:22.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41eed9d700 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 0x562cad2dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 -- 172.21.15.62:0/465304191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 msgr2=0x562cad2dcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 0x562cad2dcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 -- 172.21.15.62:0/465304191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 msgr2=0x562cad281560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad281560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 -- 172.21.15.62:0/465304191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f01087f0 con 0x562cad273ea0 2023-07-14T16:16:22.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41ee59c700 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad281560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:22.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41edd9b700 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad281aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9c0 tx=0x7f41e4096040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cae4bbb5b46474 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.707+0000 7f41d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:16:22.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.708+0000 7f41d5ffb700 1 -- 172.21.15.62:0/465304191 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4098f90 con 0x562cad273ea0 2023-07-14T16:16:22.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.708+0000 7f41d5ffb700 1 -- 172.21.15.62:0/465304191 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41e407d080 con 0x562cad273ea0 2023-07-14T16:16:22.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.708+0000 7f42087a1b80 1 -- 172.21.15.62:0/465304191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41e409dd30 con 0x562cad273ea0 2023-07-14T16:16:22.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.708+0000 7f41d5ffb700 1 -- 172.21.15.62:0/465304191 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41e40992d0 con 0x562cad273ea0 2023-07-14T16:16:22.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.708+0000 7f42087a1b80 10 client.25333 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:22.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.708+0000 7f42087a1b80 1 -- 172.21.15.62:0/465304191 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562cacff1330 con 0x562cad273ea0 2023-07-14T16:16:22.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.708+0000 7f41d5ffb700 1 -- 172.21.15.62:0/465304191 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41e409dd30 con 0x562cad273ea0 2023-07-14T16:16:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.709+0000 7f41d5ffb700 1 -- 172.21.15.62:0/465304191 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f41e407ad90 con 0x562cad273ea0 2023-07-14T16:16:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.709+0000 7f42087a1b80 10 client.25333 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.709+0000 7f42087a1b80 10 client.25333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.709+0000 7f42087a1b80 20 client.25333 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.709+0000 7f42087a1b80 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562cad2de6d0 0x562cad2e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.709+0000 7f42087a1b80 4 client.25333 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:16:22.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.709+0000 7f42087a1b80 1 -- 172.21.15.62:0/465304191 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562cad1800d0 con 0x562cad2de6d0 2023-07-14T16:16:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.710+0000 7f41ee59c700 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562cad2de6d0 0x562cad2e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.710+0000 7f41ee59c700 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562cad2de6d0 0x562cad2e0ae0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdfe49ea2b577f6d server_cookie=9d0a9f01d473b18 in_seq=0 out_seq=0 2023-07-14T16:16:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.710+0000 7f41d5ffb700 10 client.25333 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f41d5ffb700 1 -- 172.21.15.62:0/465304191 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562cad1800d0 con 0x562cad2de6d0 2023-07-14T16:16:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f41d5ffb700 10 client.25333 handle_command_reply: tid=0 2023-07-14T16:16:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 client.25333 shutdown 2023-07-14T16:16:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 -- 172.21.15.62:0/465304191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 msgr2=0x562cad281aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad281aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9c0 tx=0x7f41e4096040 comp rx=0 tx=0).stop 2023-07-14T16:16:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 -- 172.21.15.62:0/465304191 shutdown_connections 2023-07-14T16:16:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562cad2de6d0 0x562cad2e0ae0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562cad274cf0 0x562cad2dcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562cad273550 0x562cad281560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 --2- 172.21.15.62:0/465304191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cad273ea0 0x562cad281aa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f41d5ffb700 0 client.25333 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.712+0000 7f42087a1b80 1 -- 172.21.15.62:0/465304191 >> 172.21.15.62:0/465304191 conn(0x562cad268e20 msgr2=0x562cad27ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.713+0000 7f42087a1b80 1 -- 172.21.15.62:0/465304191 shutdown_connections 2023-07-14T16:16:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.713+0000 7f42087a1b80 1 -- 172.21.15.62:0/465304191 wait complete. 2023-07-14T16:16:22.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.713+0000 7f42087a1b80 20 client.25333 trim_cache size 0 max 16384 2023-07-14T16:16:22.697 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:16:22.697 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:16:22.697 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "c4257cf5-d917-4768-97d6-cf71eb406616", 2023-07-14T16:16:22.698 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:16:22.698 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:16:22.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 -- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41d80a11d0 msgr2=0x7f41d80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41d80a11d0 0x7f41d80a3680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f41f0100530 tx=0x7f41e8009ba0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 -- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0102e80 msgr2=0x7f41f00fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0102e80 0x7f41f00fe510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9f0 tx=0x7f41e400ed00 comp rx=0 tx=0).stop 2023-07-14T16:16:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 -- 172.21.15.62:0/1198866892 shutdown_connections 2023-07-14T16:16:22.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41d80a11d0 0x7f41d80a3680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41f01057a0 0x7f41f00fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41f0100560 0x7f41f00ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 --2- 172.21.15.62:0/1198866892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41f0102e80 0x7f41f00fe510 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 -- 172.21.15.62:0/1198866892 >> 172.21.15.62:0/1198866892 conn(0x7f41f00fa000 msgr2=0x7f41f01011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:22.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 -- 172.21.15.62:0/1198866892 shutdown_connections 2023-07-14T16:16:22.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:22.714+0000 7f41f4dff700 1 -- 172.21.15.62:0/1198866892 wait complete. 2023-07-14T16:16:22.715 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:16:22.790 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:22 smithi062 ceph-mon[95543]: pgmap v379: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:16:22.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:22 smithi081 ceph-mon[127795]: pgmap v379: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:16:22.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.006+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/1444936462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc8100560 msgr2=0x7f8dc8102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.006+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/1444936462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc8100560 0x7f8dc8102940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009990 tx=0x7f8db8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.006+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/1444936462 shutdown_connections 2023-07-14T16:16:22.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.006+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/1444936462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dc81057a0 0x7f8dc8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.006+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/1444936462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc8100560 0x7f8dc8102940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.006+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/1444936462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc8102e80 0x7f8dc8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.006+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/1444936462 >> 172.21.15.62:0/1444936462 conn(0x7f8dc80fa000 msgr2=0x7f8dc80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:22.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.007+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/1444936462 shutdown_connections 2023-07-14T16:16:22.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.007+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/1444936462 wait complete. 2023-07-14T16:16:22.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.008+0000 7f8dcd9ff700 1 Processor -- start 2023-07-14T16:16:22.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.008+0000 7f8dcd9ff700 1 -- start start 2023-07-14T16:16:22.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dcd9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc8100560 0x7f8dc8195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dcd9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dc8102e80 0x7f8dc8195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dcd9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc81057a0 0x7f8dc819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dcd9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc819d2b0 con 0x7f8dc8100560 2023-07-14T16:16:22.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dcd9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc819d3f0 con 0x7f8dc8102e80 2023-07-14T16:16:22.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dcd9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc819d580 con 0x7f8dc81057a0 2023-07-14T16:16:22.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc8100560 0x7f8dc8195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc8100560 0x7f8dc8195730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48596/0 (socket says 172.21.15.62:48596) 2023-07-14T16:16:22.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/831568926 learned_addr learned my addr 172.21.15.62:0/831568926 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:22.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.009+0000 7f8dc77fe700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc81057a0 0x7f8dc819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/831568926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc81057a0 msgr2=0x7f8dc819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc81057a0 0x7f8dc819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/831568926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dc8102e80 msgr2=0x7f8dc8195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dc8102e80 0x7f8dc8195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/831568926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db80095f0 con 0x7f8dc8100560 2023-07-14T16:16:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc8100560 0x7f8dc8195730 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8db80059d0 tx=0x7f8db80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dcc9fd700 1 -- 172.21.15.62:0/831568926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db801d070 con 0x7f8dc8100560 2023-07-14T16:16:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/831568926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc819d800 con 0x7f8dc8100560 2023-07-14T16:16:22.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dcc9fd700 1 -- 172.21.15.62:0/831568926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8db800beb0 con 0x7f8dc8100560 2023-07-14T16:16:22.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dcc9fd700 1 -- 172.21.15.62:0/831568926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800fe80 con 0x7f8dc8100560 2023-07-14T16:16:22.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.010+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/831568926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc819dd00 con 0x7f8dc8100560 2023-07-14T16:16:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.016+0000 7f8dcc9fd700 1 -- 172.21.15.62:0/831568926 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8db800f640 con 0x7f8dc8100560 2023-07-14T16:16:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.017+0000 7f8de1390b80 1 Processor -- start 2023-07-14T16:16:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.017+0000 7f8dcc9fd700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8db00a1150 0x7f8db00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.017+0000 7f8de1390b80 1 -- start start 2023-07-14T16:16:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8dcc9fd700 1 -- 172.21.15.62:0/831568926 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8db80cdd70 con 0x7f8dc8100560 2023-07-14T16:16:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 0x557939d220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:22.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8de1390b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939c25cb0 con 0x557939d20e20 2023-07-14T16:16:22.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8de1390b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939c78a40 con 0x557939d204d0 2023-07-14T16:16:22.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8de1390b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939c71ed0 con 0x557939d21c70 2023-07-14T16:16:22.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d208e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48062/0 (socket says 172.21.15.62:48062) 2023-07-14T16:16:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.018+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/3751056371 learned_addr learned my addr 172.21.15.62:0/3751056371 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc77fe700 1 --2- 172.21.15.62:0/3751056371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 0x557939d220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc77fe700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8db00a1150 0x7f8db00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:22.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/3751056371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 msgr2=0x557939d220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/3751056371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 0x557939d220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/3751056371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 msgr2=0x557939d216e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:22.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/3751056371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d216e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:22.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/3751056371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557939a9fde0 con 0x557939d204d0 2023-07-14T16:16:22.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/3751056371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d208e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8db809aec0 tx=0x7f8db809ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77dbe3e22b2e8a2a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.019+0000 7f8dc77fe700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8db00a1150 0x7f8db00a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009550 tx=0x7f8dc0006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:22.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.020+0000 7f8dadffb700 1 -- 172.21.15.62:0/3751056371 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800f460 con 0x557939d204d0 2023-07-14T16:16:22.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.020+0000 7f8dadffb700 1 -- 172.21.15.62:0/3751056371 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8db809b1c0 con 0x557939d204d0 2023-07-14T16:16:22.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.020+0000 7f8dadffb700 1 -- 172.21.15.62:0/3751056371 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db809a6b0 con 0x557939d204d0 2023-07-14T16:16:22.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 -- 172.21.15.62:0/3751056371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 msgr2=0x557939d208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 --2- 172.21.15.62:0/3751056371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d208e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8db809aec0 tx=0x7f8db809ad80 comp rx=0 tx=0).stop 2023-07-14T16:16:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 -- 172.21.15.62:0/3751056371 shutdown_connections 2023-07-14T16:16:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 --2- 172.21.15.62:0/3751056371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 0x557939d220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 --2- 172.21.15.62:0/3751056371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d208e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 --2- 172.21.15.62:0/3751056371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 -- 172.21.15.62:0/3751056371 >> 172.21.15.62:0/3751056371 conn(0x557939d15da0 msgr2=0x557939d1b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 -- 172.21.15.62:0/3751056371 shutdown_connections 2023-07-14T16:16:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.021+0000 7f8de1390b80 1 -- 172.21.15.62:0/3751056371 wait complete. 2023-07-14T16:16:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.022+0000 7f8de1390b80 1 -- start start 2023-07-14T16:16:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.022+0000 7f8de1390b80 1 Processor -- start 2023-07-14T16:16:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d204d0 0x557939d25810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d25d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 0x557939d26d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8de1390b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939c11ac0 con 0x557939d20e20 2023-07-14T16:16:23.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8de1390b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939c117c0 con 0x557939d21c70 2023-07-14T16:16:23.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8de1390b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939af5920 con 0x557939d204d0 2023-07-14T16:16:23.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8de1390b80 10 client.0 _ll_register_callbacks cb 0x557939c0f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d25d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d204d0 0x557939d25810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 0x557939d26d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d204d0 0x557939d25810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35822/0 (socket says 172.21.15.62:35822) 2023-07-14T16:16:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.023+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/1484989107 learned_addr learned my addr 172.21.15.62:0/1484989107 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/1484989107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 msgr2=0x557939d26d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 0x557939d26d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/1484989107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 msgr2=0x557939d25d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d25d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/1484989107 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc819dd00 con 0x557939d204d0 2023-07-14T16:16:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d204d0 0x557939d25810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8db8097670 tx=0x7f8db809f1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217791eddc3d5cc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8da6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:16:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8da6ffd700 1 -- 172.21.15.62:0/1484989107 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db80a32f0 con 0x557939d204d0 2023-07-14T16:16:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8da6ffd700 1 -- 172.21.15.62:0/1484989107 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8db800ba20 con 0x557939d204d0 2023-07-14T16:16:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.024+0000 7f8de1390b80 1 -- 172.21.15.62:0/1484989107 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557939bcb4f0 con 0x557939d204d0 2023-07-14T16:16:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.025+0000 7f8da6ffd700 1 -- 172.21.15.62:0/1484989107 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8db80b9030 con 0x557939d204d0 2023-07-14T16:16:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.025+0000 7f8de1390b80 10 client.25345 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.025+0000 7f8de1390b80 1 -- 172.21.15.62:0/1484989107 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557939a88c60 con 0x557939d204d0 2023-07-14T16:16:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.025+0000 7f8da6ffd700 1 -- 172.21.15.62:0/1484989107 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8db80c9020 con 0x557939d204d0 2023-07-14T16:16:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.025+0000 7f8da6ffd700 1 -- 172.21.15.62:0/1484989107 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8db80b9190 con 0x557939d204d0 2023-07-14T16:16:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.025+0000 7f8de1390b80 10 client.25345 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.025+0000 7f8de1390b80 10 client.25345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.025+0000 7f8de1390b80 20 client.25345 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.026+0000 7f8de1390b80 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557939d3dfc0 0x557939d403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.026+0000 7f8de1390b80 4 client.25345 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.026+0000 7f8de1390b80 1 -- 172.21.15.62:0/1484989107 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557939c6aeb0 con 0x557939d3dfc0 2023-07-14T16:16:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.026+0000 7f8dc67fc700 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557939d3dfc0 0x557939d403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.027+0000 7f8dc67fc700 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557939d3dfc0 0x557939d403a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a4f140d7236985 server_cookie=b663b7df3fed4d62 in_seq=0 out_seq=0 2023-07-14T16:16:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.027+0000 7f8da6ffd700 10 client.25345 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8da6ffd700 1 -- 172.21.15.62:0/1484989107 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557939c6aeb0 con 0x557939d3dfc0 2023-07-14T16:16:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8da6ffd700 10 client.25345 handle_command_reply: tid=0 2023-07-14T16:16:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 client.25345 shutdown 2023-07-14T16:16:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 -- 172.21.15.62:0/1484989107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d204d0 msgr2=0x557939d25810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d204d0 0x557939d25810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8db8097670 tx=0x7f8db809f1f0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 -- 172.21.15.62:0/1484989107 shutdown_connections 2023-07-14T16:16:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557939d3dfc0 0x557939d403a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 0x557939d26d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d204d0 0x557939d25810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 --2- 172.21.15.62:0/1484989107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 -- 172.21.15.62:0/1484989107 >> 172.21.15.62:0/1484989107 conn(0x557939d15da0 msgr2=0x557939d1a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8da6ffd700 0 client.25345 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 -- 172.21.15.62:0/1484989107 shutdown_connections 2023-07-14T16:16:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.028+0000 7f8de1390b80 1 -- 172.21.15.62:0/1484989107 wait complete. 2023-07-14T16:16:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.029+0000 7f8de1390b80 20 client.25345 trim_cache size 0 max 16384 2023-07-14T16:16:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:22 smithi164 ceph-mon[97427]: pgmap v379: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-14T16:16:23.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.050+0000 7f8de1390b80 1 Processor -- start 2023-07-14T16:16:23.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.050+0000 7f8de1390b80 1 -- start start 2023-07-14T16:16:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.050+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d204d0 0x557939d22900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.050+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d20e20 0x557939d22e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.050+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 0x557939d255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.050+0000 7f8de1390b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939d1c4b0 con 0x557939d204d0 2023-07-14T16:16:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.050+0000 7f8de1390b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939d15990 con 0x557939d21c70 2023-07-14T16:16:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.050+0000 7f8de1390b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939d1c070 con 0x557939d20e20 2023-07-14T16:16:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d204d0 0x557939d22900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d204d0 0x557939d22900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48624/0 (socket says 172.21.15.62:48624) 2023-07-14T16:16:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/776556757 learned_addr learned my addr 172.21.15.62:0/776556757 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc67fc700 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d20e20 0x557939d22e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc77fe700 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 0x557939d255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/776556757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d20e20 msgr2=0x557939d22e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d20e20 0x557939d22e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/776556757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 msgr2=0x557939d255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 0x557939d255c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc6ffd700 1 -- 172.21.15.62:0/776556757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557939ac6250 con 0x557939d204d0 2023-07-14T16:16:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.051+0000 7f8dc67fc700 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d20e20 0x557939d22e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.052+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d204d0 0x557939d22900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8db809b150 tx=0x7f8db8099770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409a0e0c81b9bdfa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.052+0000 7f8dae7fc700 1 -- 172.21.15.62:0/776556757 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db80b9040 con 0x557939d204d0 2023-07-14T16:16:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.052+0000 7f8dae7fc700 1 -- 172.21.15.62:0/776556757 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8db80c9070 con 0x557939d204d0 2023-07-14T16:16:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.052+0000 7f8dae7fc700 1 -- 172.21.15.62:0/776556757 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8096fb0 con 0x557939d204d0 2023-07-14T16:16:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 -- 172.21.15.62:0/776556757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d204d0 msgr2=0x557939d22900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d204d0 0x557939d22900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8db809b150 tx=0x7f8db8099770 comp rx=0 tx=0).stop 2023-07-14T16:16:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 -- 172.21.15.62:0/776556757 shutdown_connections 2023-07-14T16:16:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d21c70 0x557939d255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d204d0 0x557939d22900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 --2- 172.21.15.62:0/776556757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d20e20 0x557939d22e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 -- 172.21.15.62:0/776556757 >> 172.21.15.62:0/776556757 conn(0x557939d15da0 msgr2=0x557939d2acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 -- 172.21.15.62:0/776556757 shutdown_connections 2023-07-14T16:16:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 -- 172.21.15.62:0/776556757 wait complete. 2023-07-14T16:16:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.053+0000 7f8de1390b80 1 -- start start 2023-07-14T16:16:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.054+0000 7f8de1390b80 1 Processor -- start 2023-07-14T16:16:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.054+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d3ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.054+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.054+0000 7f8de1390b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 0x557939d37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.054+0000 7f8de1390b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939d3b650 con 0x557939d20e20 2023-07-14T16:16:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.054+0000 7f8de1390b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939d38350 con 0x557939d204d0 2023-07-14T16:16:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.054+0000 7f8de1390b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557939d38510 con 0x557939d21c70 2023-07-14T16:16:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.054+0000 7f8de1390b80 10 client.0 _ll_register_callbacks cb 0x7f8dbc009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d378d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48634/0 (socket says 172.21.15.62:48634) 2023-07-14T16:16:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc67fc700 1 -- 172.21.15.62:0/2237856627 learned_addr learned my addr 172.21.15.62:0/2237856627 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d3ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc77fe700 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 0x557939d37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc67fc700 1 -- 172.21.15.62:0/2237856627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 msgr2=0x557939d37e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc67fc700 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 0x557939d37e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc67fc700 1 -- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 msgr2=0x557939d3ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc67fc700 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d3ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc67fc700 1 -- 172.21.15.62:0/2237856627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557939a88c60 con 0x557939d20e20 2023-07-14T16:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.055+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d3ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.056+0000 7f8dc67fc700 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d378d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x557939d40b70 tx=0x7f8dbc01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70acc038f88202b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.056+0000 7f8da6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.056+0000 7f8da6ffd700 1 -- 172.21.15.62:0/2237856627 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc017040 con 0x557939d20e20 2023-07-14T16:16:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.056+0000 7f8de1390b80 1 -- 172.21.15.62:0/2237856627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8db809b1c0 con 0x557939d20e20 2023-07-14T16:16:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.056+0000 7f8da6ffd700 1 -- 172.21.15.62:0/2237856627 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8dbc004050 con 0x557939d20e20 2023-07-14T16:16:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.056+0000 7f8da6ffd700 1 -- 172.21.15.62:0/2237856627 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dbc004210 con 0x557939d20e20 2023-07-14T16:16:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8de1390b80 10 client.15639 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8de1390b80 1 -- 172.21.15.62:0/2237856627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557939a9e2b0 con 0x557939d20e20 2023-07-14T16:16:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8da6ffd700 1 -- 172.21.15.62:0/2237856627 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8db809b1c0 con 0x557939d20e20 2023-07-14T16:16:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8da6ffd700 1 -- 172.21.15.62:0/2237856627 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8dbc02e070 con 0x557939d20e20 2023-07-14T16:16:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8de1390b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8de1390b80 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8de1390b80 20 client.15639 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8de1390b80 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557939d936b0 0x557939d95ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8de1390b80 4 client.15639 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:16:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.057+0000 7f8de1390b80 1 -- 172.21.15.62:0/2237856627 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557939c2d050 con 0x557939d936b0 2023-07-14T16:16:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.058+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557939d936b0 0x557939d95ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.058+0000 7f8dc6ffd700 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557939d936b0 0x557939d95ac0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0cf7d0628f6e7e0 server_cookie=6784bc9c58e40b85 in_seq=0 out_seq=0 2023-07-14T16:16:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.058+0000 7f8da6ffd700 10 client.15639 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.058+0000 7f8da6ffd700 1 -- 172.21.15.62:0/2237856627 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557939c2d050 con 0x557939d936b0 2023-07-14T16:16:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.058+0000 7f8da6ffd700 10 client.15639 handle_command_reply: tid=0 2023-07-14T16:16:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.058+0000 7f8de1390b80 1 client.15639 shutdown 2023-07-14T16:16:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 -- 172.21.15.62:0/2237856627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 msgr2=0x557939d378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d378d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x557939d40b70 tx=0x7f8dbc01cc30 comp rx=0 tx=0).stop 2023-07-14T16:16:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 -- 172.21.15.62:0/2237856627 shutdown_connections 2023-07-14T16:16:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557939d21c70 0x557939d37e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557939d936b0 0x557939d95ac0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557939d204d0 0x557939d3ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 --2- 172.21.15.62:0/2237856627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557939d20e20 0x557939d378d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 -- 172.21.15.62:0/2237856627 >> 172.21.15.62:0/2237856627 conn(0x557939d15da0 msgr2=0x557939d22900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8da6ffd700 0 client.15639 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 -- 172.21.15.62:0/2237856627 shutdown_connections 2023-07-14T16:16:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 1 -- 172.21.15.62:0/2237856627 wait complete. 2023-07-14T16:16:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.059+0000 7f8de1390b80 20 client.15639 trim_cache size 0 max 16384 2023-07-14T16:16:23.047 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:16:23.047 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:16:23.048 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:16:23.048 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:16:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8db00a1150 msgr2=0x7f8db00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8db00a1150 0x7f8db00a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009550 tx=0x7f8dc0006040 comp rx=0 tx=0).stop 2023-07-14T16:16:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc8100560 msgr2=0x7f8dc8195730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc8100560 0x7f8dc8195730 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8db80059d0 tx=0x7f8db80047b0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/831568926 shutdown_connections 2023-07-14T16:16:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8dc81057a0 0x7f8dc819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc8100560 0x7f8dc8195730 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8db00a1150 0x7f8db00a3600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 --2- 172.21.15.62:0/831568926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8dc8102e80 0x7f8dc8195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/831568926 >> 172.21.15.62:0/831568926 conn(0x7f8dc80fa000 msgr2=0x7f8dc81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/831568926 shutdown_connections 2023-07-14T16:16:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.060+0000 7f8dcd9ff700 1 -- 172.21.15.62:0/831568926 wait complete. 2023-07-14T16:16:23.058 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:16:23.058 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:16:23.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 -- 172.21.15.62:0/3990389552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 msgr2=0x7fec441051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 --2- 172.21.15.62:0/3990389552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 0x7fec441051c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fec38009990 tx=0x7fec38009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 -- 172.21.15.62:0/3990389552 shutdown_connections 2023-07-14T16:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 --2- 172.21.15.62:0/3990389552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec44105700 0x7fec44107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 --2- 172.21.15.62:0/3990389552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec441004c0 0x7fec441028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 --2- 172.21.15.62:0/3990389552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 0x7fec441051c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 -- 172.21.15.62:0/3990389552 >> 172.21.15.62:0/3990389552 conn(0x7fec440f9ee0 msgr2=0x7fec440fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 -- 172.21.15.62:0/3990389552 shutdown_connections 2023-07-14T16:16:23.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.345+0000 7fec4359e700 1 -- 172.21.15.62:0/3990389552 wait complete. 2023-07-14T16:16:23.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.346+0000 7fec4359e700 1 Processor -- start 2023-07-14T16:16:23.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.347+0000 7fec4359e700 1 -- start start 2023-07-14T16:16:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.347+0000 7fec4359e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec441004c0 0x7fec44193250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.347+0000 7fec4359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 0x7fec44193790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.347+0000 7fec4359e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec44105700 0x7fec4419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.347+0000 7fec4359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4419add0 con 0x7fec44102de0 2023-07-14T16:16:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.347+0000 7fec4359e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4419af10 con 0x7fec441004c0 2023-07-14T16:16:23.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.347+0000 7fec4359e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4419b0a0 con 0x7fec44105700 2023-07-14T16:16:23.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.347+0000 7fec41d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 0x7fec44193790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec42d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec44105700 0x7fec4419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec41d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 0x7fec44193790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48658/0 (socket says 172.21.15.62:48658) 2023-07-14T16:16:23.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec41d9b700 1 -- 172.21.15.62:0/1168617193 learned_addr learned my addr 172.21.15.62:0/1168617193 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec4259c700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec441004c0 0x7fec44193250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec42d9d700 1 -- 172.21.15.62:0/1168617193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec441004c0 msgr2=0x7fec44193250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec42d9d700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec441004c0 0x7fec44193250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec42d9d700 1 -- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 msgr2=0x7fec44193790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec42d9d700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 0x7fec44193790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec42d9d700 1 -- 172.21.15.62:0/1168617193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec380095f0 con 0x7fec44105700 2023-07-14T16:16:23.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec41d9b700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 0x7fec44193790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:23.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.348+0000 7fec42d9d700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec44105700 0x7fec4419a800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00efc0 tx=0x7fec3c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.349+0000 7fec337fe700 1 -- 172.21.15.62:0/1168617193 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c018070 con 0x7fec44105700 2023-07-14T16:16:23.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.349+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4419b320 con 0x7fec44105700 2023-07-14T16:16:23.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.349+0000 7fec337fe700 1 -- 172.21.15.62:0/1168617193 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec3c00f040 con 0x7fec44105700 2023-07-14T16:16:23.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.349+0000 7fec337fe700 1 -- 172.21.15.62:0/1168617193 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00cba0 con 0x7fec44105700 2023-07-14T16:16:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.349+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4419b830 con 0x7fec44105700 2023-07-14T16:16:23.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.350+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec440fba50 con 0x7fec44105700 2023-07-14T16:16:23.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.354+0000 7fec337fe700 1 -- 172.21.15.62:0/1168617193 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fec3c003a00 con 0x7fec44105700 2023-07-14T16:16:23.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.355+0000 7fec337fe700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fec2c0a1240 0x7fec2c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.355+0000 7fec337fe700 1 -- 172.21.15.62:0/1168617193 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fec3c0cc570 con 0x7fec44105700 2023-07-14T16:16:23.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.355+0000 7fec4259c700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fec2c0a1240 0x7fec2c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.356+0000 7fec337fe700 1 -- 172.21.15.62:0/1168617193 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fec3c095d60 con 0x7fec44105700 2023-07-14T16:16:23.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.357+0000 7fec4259c700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fec2c0a1240 0x7fec2c0a36f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec3400ff50 tx=0x7fec34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.704+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec4402c850 con 0x7fec44105700 2023-07-14T16:16:23.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.706+0000 7fec337fe700 1 -- 172.21.15.62:0/1168617193 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fec3c099380 con 0x7fec44105700 2023-07-14T16:16:23.683 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:16:23.683 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:16:23.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.708+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fec2c0a1240 msgr2=0x7fec2c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.708+0000 7fec4359e700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fec2c0a1240 0x7fec2c0a36f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec3400ff50 tx=0x7fec34009260 comp rx=0 tx=0).stop 2023-07-14T16:16:23.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.708+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec44105700 msgr2=0x7fec4419a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.708+0000 7fec4359e700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec44105700 0x7fec4419a800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00efc0 tx=0x7fec3c00c3a0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.709+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 shutdown_connections 2023-07-14T16:16:23.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.709+0000 7fec4359e700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec44105700 0x7fec4419a800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.709+0000 7fec4359e700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fec2c0a1240 0x7fec2c0a36f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.709+0000 7fec4359e700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec441004c0 0x7fec44193250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.709+0000 7fec4359e700 1 --2- 172.21.15.62:0/1168617193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec44102de0 0x7fec44193790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.709+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 >> 172.21.15.62:0/1168617193 conn(0x7fec440f9ee0 msgr2=0x7fec44101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.709+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 shutdown_connections 2023-07-14T16:16:23.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.709+0000 7fec4359e700 1 -- 172.21.15.62:0/1168617193 wait complete. 2023-07-14T16:16:23.688 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:16:23.707 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:16:23.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:23 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:16:23.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:23 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:23.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:23 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:16:23.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:23 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:23.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:23 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:16:23.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:23 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:16:23.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:23 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:23.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:23 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:16:23.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:23 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:23.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:23 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:16:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.954+0000 7f69f279d700 1 -- 172.21.15.62:0/2462381167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec102d70 msgr2=0x7f69ec105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.954+0000 7f69f279d700 1 --2- 172.21.15.62:0/2462381167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec102d70 0x7f69ec105150 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009990 tx=0x7f69dc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.954+0000 7f69f279d700 1 -- 172.21.15.62:0/2462381167 shutdown_connections 2023-07-14T16:16:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.954+0000 7f69f279d700 1 --2- 172.21.15.62:0/2462381167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69ec105690 0x7f69ec107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.954+0000 7f69f279d700 1 --2- 172.21.15.62:0/2462381167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69ec100450 0x7f69ec102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.954+0000 7f69f279d700 1 --2- 172.21.15.62:0/2462381167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec102d70 0x7f69ec105150 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.954+0000 7f69f279d700 1 -- 172.21.15.62:0/2462381167 >> 172.21.15.62:0/2462381167 conn(0x7f69ec0f9ed0 msgr2=0x7f69ec0fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.954+0000 7f69f279d700 1 -- 172.21.15.62:0/2462381167 shutdown_connections 2023-07-14T16:16:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.955+0000 7f69f279d700 1 -- 172.21.15.62:0/2462381167 wait complete. 2023-07-14T16:16:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.956+0000 7f69f279d700 1 Processor -- start 2023-07-14T16:16:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.956+0000 7f69f279d700 1 -- start start 2023-07-14T16:16:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.956+0000 7f69f279d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69ec100450 0x7f69ec1954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.956+0000 7f69f279d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec102d70 0x7f69ec195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.956+0000 7f69f279d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69ec105690 0x7f69ec19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.956+0000 7f69f279d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec19d060 con 0x7f69ec102d70 2023-07-14T16:16:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.956+0000 7f69f279d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec19d1a0 con 0x7f69ec105690 2023-07-14T16:16:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.956+0000 7f69f279d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec19d330 con 0x7f69ec100450 2023-07-14T16:16:23.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69ec100450 0x7f69ec1954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69ec100450 0x7f69ec1954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35868/0 (socket says 172.21.15.62:35868) 2023-07-14T16:16:23.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f179b700 1 -- 172.21.15.62:0/1029809491 learned_addr learned my addr 172.21.15.62:0/1029809491 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69ec105690 0x7f69ec19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f179b700 1 -- 172.21.15.62:0/1029809491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69ec105690 msgr2=0x7f69ec19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f179b700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69ec105690 0x7f69ec19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f179b700 1 -- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec102d70 msgr2=0x7f69ec195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f179b700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec102d70 0x7f69ec195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.957+0000 7f69f179b700 1 -- 172.21.15.62:0/1029809491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc0095f0 con 0x7f69ec100450 2023-07-14T16:16:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.958+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69ec105690 0x7f69ec19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.958+0000 7f69f179b700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69ec100450 0x7f69ec1954e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e9a0 tx=0x7f69e800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.958+0000 7f69e27fc700 1 -- 172.21.15.62:0/1029809491 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8009790 con 0x7f69ec100450 2023-07-14T16:16:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.958+0000 7f69e27fc700 1 -- 172.21.15.62:0/1029809491 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69e800cbd0 con 0x7f69ec100450 2023-07-14T16:16:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.958+0000 7f69f279d700 1 -- 172.21.15.62:0/1029809491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec19d610 con 0x7f69ec100450 2023-07-14T16:16:23.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.958+0000 7f69e27fc700 1 -- 172.21.15.62:0/1029809491 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8018680 con 0x7f69ec100450 2023-07-14T16:16:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.958+0000 7f69f279d700 1 -- 172.21.15.62:0/1029809491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec19db70 con 0x7f69ec100450 2023-07-14T16:16:23.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.963+0000 7f69e27fc700 1 -- 172.21.15.62:0/1029809491 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f69e800cd40 con 0x7f69ec100450 2023-07-14T16:16:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.965+0000 7f6a07278b80 1 Processor -- start 2023-07-14T16:16:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.965+0000 7f6a07278b80 1 -- start start 2023-07-14T16:16:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa2fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 0x56236fa312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f6a07278b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f934cb0 con 0x56236fa30070 2023-07-14T16:16:23.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f6a07278b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f987a40 con 0x56236fa2f720 2023-07-14T16:16:23.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f6a07278b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f980ed0 con 0x56236fa30e30 2023-07-14T16:16:23.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f69e27fc700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69d80a1270 0x7f69d80a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f69f1f9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa2fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f69e27fc700 1 -- 172.21.15.62:0/1029809491 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69e8014070 con 0x7f69ec100450 2023-07-14T16:16:23.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f69f179b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f69f1f9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa2fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48152/0 (socket says 172.21.15.62:48152) 2023-07-14T16:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f69f1f9c700 1 -- 172.21.15.62:0/1037223207 learned_addr learned my addr 172.21.15.62:0/1037223207 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.966+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69d80a1270 0x7f69d80a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69f0f9a700 1 --2- 172.21.15.62:0/1037223207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 0x56236fa312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69f179b700 1 -- 172.21.15.62:0/1037223207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 msgr2=0x56236fa312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69f179b700 1 --2- 172.21.15.62:0/1037223207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 0x56236fa312a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69f179b700 1 -- 172.21.15.62:0/1037223207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 msgr2=0x56236fa2fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69f179b700 1 --2- 172.21.15.62:0/1037223207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa2fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69f179b700 1 -- 172.21.15.62:0/1037223207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236f7aede0 con 0x56236fa30070 2023-07-14T16:16:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69d80a1270 0x7f69d80a3720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009550 tx=0x7f69e4015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69f179b700 1 --2- 172.21.15.62:0/1037223207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa308a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f69e80ca6f0 tx=0x7f69e8099f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ecddc42df4c250 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.967+0000 7f69d77fe700 1 -- 172.21.15.62:0/1037223207 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e809e030 con 0x56236fa30070 2023-07-14T16:16:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.968+0000 7f69d77fe700 1 -- 172.21.15.62:0/1037223207 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69e80a2450 con 0x56236fa30070 2023-07-14T16:16:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.968+0000 7f69d77fe700 1 -- 172.21.15.62:0/1037223207 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e80a2220 con 0x56236fa30070 2023-07-14T16:16:23.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.968+0000 7f6a07278b80 1 -- 172.21.15.62:0/1037223207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 msgr2=0x56236fa308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.968+0000 7f6a07278b80 1 --2- 172.21.15.62:0/1037223207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa308a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f69e80ca6f0 tx=0x7f69e8099f00 comp rx=0 tx=0).stop 2023-07-14T16:16:23.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.969+0000 7f6a07278b80 1 -- 172.21.15.62:0/1037223207 shutdown_connections 2023-07-14T16:16:23.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.969+0000 7f6a07278b80 1 --2- 172.21.15.62:0/1037223207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 0x56236fa312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.969+0000 7f6a07278b80 1 --2- 172.21.15.62:0/1037223207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa2fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.969+0000 7f6a07278b80 1 --2- 172.21.15.62:0/1037223207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa308a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.969+0000 7f6a07278b80 1 -- 172.21.15.62:0/1037223207 >> 172.21.15.62:0/1037223207 conn(0x56236fa24ff0 msgr2=0x56236fa2a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.969+0000 7f6a07278b80 1 -- 172.21.15.62:0/1037223207 shutdown_connections 2023-07-14T16:16:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.969+0000 7f6a07278b80 1 -- 172.21.15.62:0/1037223207 wait complete. 2023-07-14T16:16:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.970+0000 7f6a07278b80 1 -- start start 2023-07-14T16:16:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.970+0000 7f6a07278b80 1 Processor -- start 2023-07-14T16:16:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.970+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30070 0x56236fa39810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa40850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f6a07278b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f920ac0 con 0x56236fa30e30 2023-07-14T16:16:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f6a07278b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f9207c0 con 0x56236fa2f720 2023-07-14T16:16:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f6a07278b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f804920 con 0x56236fa30070 2023-07-14T16:16:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f6a07278b80 10 client.0 _ll_register_callbacks cb 0x56236f91e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f69f1f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa40850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f69f1f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa40850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48682/0 (socket says 172.21.15.62:48682) 2023-07-14T16:16:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f69f1f9c700 1 -- 172.21.15.62:0/2628870863 learned_addr learned my addr 172.21.15.62:0/2628870863 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f69f179b700 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.971+0000 7f69f0f9a700 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30070 0x56236fa39810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69f179b700 1 -- 172.21.15.62:0/2628870863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30070 msgr2=0x56236fa39810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69f179b700 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30070 0x56236fa39810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69f179b700 1 -- 172.21.15.62:0/2628870863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 msgr2=0x56236fa40850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69f179b700 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa40850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69f179b700 1 -- 172.21.15.62:0/2628870863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec19db70 con 0x56236fa2f720 2023-07-14T16:16:23.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa40850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:23.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69f179b700 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa392d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c5b0 tx=0x7f69e80a2370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab420058476ce61 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:23.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2628870863 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e809e030 con 0x56236fa2f720 2023-07-14T16:16:23.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f6a07278b80 1 -- 172.21.15.62:0/2628870863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56236f8da4f0 con 0x56236fa2f720 2023-07-14T16:16:23.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.972+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2628870863 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69e80cadc0 con 0x56236fa2f720 2023-07-14T16:16:23.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.973+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2628870863 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69e8095df0 con 0x56236fa2f720 2023-07-14T16:16:23.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.973+0000 7f6a07278b80 10 client.25430 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.973+0000 7f6a07278b80 1 -- 172.21.15.62:0/2628870863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56236f797c60 con 0x56236fa2f720 2023-07-14T16:16:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.973+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2628870863 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56236f8da4f0 con 0x56236fa2f720 2023-07-14T16:16:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.974+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2628870863 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f69e80b2a70 con 0x56236fa2f720 2023-07-14T16:16:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.974+0000 7f6a07278b80 10 client.25430 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.974+0000 7f6a07278b80 10 client.25430 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:16:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.974+0000 7f6a07278b80 20 client.25430 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.974+0000 7f6a07278b80 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x56236fa44c40 0x56236fa47050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.974+0000 7f6a07278b80 4 client.25430 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.974+0000 7f6a07278b80 1 -- 172.21.15.62:0/2628870863 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56236f979eb0 con 0x56236fa44c40 2023-07-14T16:16:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.974+0000 7f69f0f9a700 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x56236fa44c40 0x56236fa47050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.975+0000 7f69f0f9a700 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x56236fa44c40 0x56236fa47050 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67b6570b49417cce server_cookie=306946c5dda67c31 in_seq=0 out_seq=0 2023-07-14T16:16:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.975+0000 7f69d4ff9700 10 client.25430 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f69d4ff9700 1 -- 172.21.15.62:0/2628870863 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56236f979eb0 con 0x56236fa44c40 2023-07-14T16:16:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f69d4ff9700 10 client.25430 handle_command_reply: tid=0 2023-07-14T16:16:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 client.25430 shutdown 2023-07-14T16:16:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 -- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 msgr2=0x56236fa392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa392d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c5b0 tx=0x7f69e80a2370 comp rx=0 tx=0).stop 2023-07-14T16:16:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 -- 172.21.15.62:0/2628870863 shutdown_connections 2023-07-14T16:16:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x56236fa44c40 0x56236fa47050 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa40850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa392d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 --2- 172.21.15.62:0/2628870863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30070 0x56236fa39810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 -- 172.21.15.62:0/2628870863 >> 172.21.15.62:0/2628870863 conn(0x56236fa24ff0 msgr2=0x56236fa29b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f69d4ff9700 0 client.25430 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.976+0000 7f6a07278b80 1 -- 172.21.15.62:0/2628870863 shutdown_connections 2023-07-14T16:16:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.977+0000 7f6a07278b80 1 -- 172.21.15.62:0/2628870863 wait complete. 2023-07-14T16:16:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.977+0000 7f6a07278b80 20 client.25430 trim_cache size 0 max 16384 2023-07-14T16:16:23.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.990+0000 7f6a07278b80 1 Processor -- start 2023-07-14T16:16:23.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f6a07278b80 1 -- start start 2023-07-14T16:16:23.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa41140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa40810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 0x56236fa371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f6a07278b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236fa38780 con 0x56236fa30070 2023-07-14T16:16:23.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f6a07278b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236fa38e20 con 0x56236fa2f720 2023-07-14T16:16:23.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f6a07278b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236fa2bbd0 con 0x56236fa30e30 2023-07-14T16:16:23.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f69f0f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa40810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f69f0f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa40810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48698/0 (socket says 172.21.15.62:48698) 2023-07-14T16:16:23.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f69f0f9a700 1 -- 172.21.15.62:0/4107316141 learned_addr learned my addr 172.21.15.62:0/4107316141 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.991+0000 7f69f179b700 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa41140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 0x56236fa371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69f0f9a700 1 -- 172.21.15.62:0/4107316141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 msgr2=0x56236fa371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69f0f9a700 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 0x56236fa371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69f0f9a700 1 -- 172.21.15.62:0/4107316141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 msgr2=0x56236fa41140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69f0f9a700 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa41140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69f0f9a700 1 -- 172.21.15.62:0/4107316141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236fa2b680 con 0x56236fa30070 2023-07-14T16:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69f0f9a700 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa40810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005b40 tx=0x7f69dc005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe756fab261ae139 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69d7fff700 1 -- 172.21.15.62:0/4107316141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0055a0 con 0x56236fa30070 2023-07-14T16:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69d7fff700 1 -- 172.21.15.62:0/4107316141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69dc01e040 con 0x56236fa30070 2023-07-14T16:16:23.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.992+0000 7f69d7fff700 1 -- 172.21.15.62:0/4107316141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc003680 con 0x56236fa30070 2023-07-14T16:16:23.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 -- 172.21.15.62:0/4107316141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 msgr2=0x56236fa40810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa40810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005b40 tx=0x7f69dc005a90 comp rx=0 tx=0).stop 2023-07-14T16:16:23.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 -- 172.21.15.62:0/4107316141 shutdown_connections 2023-07-14T16:16:23.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa30e30 0x56236fa371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa2f720 0x56236fa41140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 --2- 172.21.15.62:0/4107316141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30070 0x56236fa40810 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 -- 172.21.15.62:0/4107316141 >> 172.21.15.62:0/4107316141 conn(0x56236fa24ff0 msgr2=0x56236fa37fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 -- 172.21.15.62:0/4107316141 shutdown_connections 2023-07-14T16:16:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.993+0000 7f6a07278b80 1 -- 172.21.15.62:0/4107316141 wait complete. 2023-07-14T16:16:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.994+0000 7f6a07278b80 1 -- start start 2023-07-14T16:16:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.994+0000 7f6a07278b80 1 Processor -- start 2023-07-14T16:16:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.995+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa2f720 0x56236fa3c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.995+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa30070 0x56236fa3cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.995+0000 7f6a07278b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa3fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.995+0000 7f6a07278b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236fa40400 con 0x56236fa30e30 2023-07-14T16:16:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.995+0000 7f6a07278b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236fa40570 con 0x56236fa30070 2023-07-14T16:16:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.995+0000 7f6a07278b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236fa40730 con 0x56236fa2f720 2023-07-14T16:16:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.995+0000 7f6a07278b80 10 client.0 _ll_register_callbacks cb 0x56236f91e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.995+0000 7f69f1f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa3fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa2f720 0x56236fa3c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f1f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa3fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48700/0 (socket says 172.21.15.62:48700) 2023-07-14T16:16:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f1f9c700 1 -- 172.21.15.62:0/3887696085 learned_addr learned my addr 172.21.15.62:0/3887696085 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f0f9a700 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa30070 0x56236fa3cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f1f9c700 1 -- 172.21.15.62:0/3887696085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa2f720 msgr2=0x56236fa3c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa2f720 0x56236fa3c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f1f9c700 1 -- 172.21.15.62:0/3887696085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa30070 msgr2=0x56236fa3cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa30070 0x56236fa3cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f1f9c700 1 -- 172.21.15.62:0/3887696085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec1086e0 con 0x56236fa30e30 2023-07-14T16:16:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f179b700 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa2f720 0x56236fa3c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69f1f9c700 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa3fec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f69e4004b10 tx=0x7f69e40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf6e6c9d778c5d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.996+0000 7f69d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.997+0000 7f69d4ff9700 1 -- 172.21.15.62:0/3887696085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400e050 con 0x56236fa30e30 2023-07-14T16:16:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.997+0000 7f6a07278b80 1 -- 172.21.15.62:0/3887696085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69e80a2450 con 0x56236fa30e30 2023-07-14T16:16:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.997+0000 7f69d4ff9700 1 -- 172.21.15.62:0/3887696085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69e4009900 con 0x56236fa30e30 2023-07-14T16:16:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.997+0000 7f69d4ff9700 1 -- 172.21.15.62:0/3887696085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69e4004bb0 con 0x56236fa30e30 2023-07-14T16:16:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.997+0000 7f6a07278b80 10 client.15660 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.997+0000 7f6a07278b80 1 -- 172.21.15.62:0/3887696085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56236f7d5250 con 0x56236fa30e30 2023-07-14T16:16:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.997+0000 7f69d4ff9700 1 -- 172.21.15.62:0/3887696085 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69e80a2450 con 0x56236fa30e30 2023-07-14T16:16:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.998+0000 7f69d4ff9700 1 -- 172.21.15.62:0/3887696085 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f69e4005020 con 0x56236fa30e30 2023-07-14T16:16:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.998+0000 7f6a07278b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.998+0000 7f6a07278b80 10 client.15660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:16:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.998+0000 7f6a07278b80 20 client.15660 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.998+0000 7f6a07278b80 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x56236fa99f00 0x56236fa9c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.998+0000 7f6a07278b80 4 client.15660 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.998+0000 7f6a07278b80 1 -- 172.21.15.62:0/3887696085 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56236f93c050 con 0x56236fa99f00 2023-07-14T16:16:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.998+0000 7f69f179b700 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x56236fa99f00 0x56236fa9c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f69f179b700 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x56236fa99f00 0x56236fa9c310 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13fe4e3de98cae15 server_cookie=61a01fa4287bbdca in_seq=0 out_seq=0 2023-07-14T16:16:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f69d4ff9700 10 client.15660 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f69d4ff9700 1 -- 172.21.15.62:0/3887696085 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56236f93c050 con 0x56236fa99f00 2023-07-14T16:16:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f69d4ff9700 10 client.15660 handle_command_reply: tid=0 2023-07-14T16:16:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 client.15660 shutdown 2023-07-14T16:16:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 -- 172.21.15.62:0/3887696085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 msgr2=0x56236fa3fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa3fec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f69e4004b10 tx=0x7f69e40093d0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 -- 172.21.15.62:0/3887696085 shutdown_connections 2023-07-14T16:16:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x56236fa99f00 0x56236fa9c310 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56236fa30e30 0x56236fa3fec0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56236fa2f720 0x56236fa3c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 --2- 172.21.15.62:0/3887696085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56236fa30070 0x56236fa3cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f69d4ff9700 0 client.15660 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:23.999+0000 7f6a07278b80 1 -- 172.21.15.62:0/3887696085 >> 172.21.15.62:0/3887696085 conn(0x56236fa24ff0 msgr2=0x56236fa371a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.000+0000 7f6a07278b80 1 -- 172.21.15.62:0/3887696085 shutdown_connections 2023-07-14T16:16:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.000+0000 7f6a07278b80 1 -- 172.21.15.62:0/3887696085 wait complete. 2023-07-14T16:16:23.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.000+0000 7f6a07278b80 20 client.15660 trim_cache size 0 max 16384 2023-07-14T16:16:23.990 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:23.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.000+0000 7f69f279d700 1 -- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69d80a1270 msgr2=0x7f69d80a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.000+0000 7f69f279d700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69d80a1270 0x7f69d80a3720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009550 tx=0x7f69e4015040 comp rx=0 tx=0).stop 2023-07-14T16:16:23.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.000+0000 7f69f279d700 1 -- 172.21.15.62:0/1029809491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69ec100450 msgr2=0x7f69ec1954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:23.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.000+0000 7f69f279d700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69ec100450 0x7f69ec1954e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e9a0 tx=0x7f69e800ed60 comp rx=0 tx=0).stop 2023-07-14T16:16:23.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.001+0000 7f69f279d700 1 -- 172.21.15.62:0/1029809491 shutdown_connections 2023-07-14T16:16:23.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.001+0000 7f69f279d700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f69d80a1270 0x7f69d80a3720 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.001+0000 7f69f279d700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f69ec105690 0x7f69ec19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.001+0000 7f69f279d700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69ec100450 0x7f69ec1954e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.001+0000 7f69f279d700 1 --2- 172.21.15.62:0/1029809491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec102d70 0x7f69ec195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.001+0000 7f69f279d700 1 -- 172.21.15.62:0/1029809491 >> 172.21.15.62:0/1029809491 conn(0x7f69ec0f9ed0 msgr2=0x7f69ec101110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.001+0000 7f69f279d700 1 -- 172.21.15.62:0/1029809491 shutdown_connections 2023-07-14T16:16:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.001+0000 7f69f279d700 1 -- 172.21.15.62:0/1029809491 wait complete. 2023-07-14T16:16:24.002 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:16:24.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:23 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:16:24.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:23 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:24.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:23 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:16:24.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:23 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:24.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:23 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:16:24.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 -- 172.21.15.62:0/1276822951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 msgr2=0x7fa3dc107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/1276822951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 0x7fa3dc107f20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400b0a0 tx=0x7fa3d400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 -- 172.21.15.62:0/1276822951 shutdown_connections 2023-07-14T16:16:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/1276822951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 0x7fa3dc107f20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/1276822951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc0ffae0 0x7fa3dc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/1276822951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc100cd0 0x7fa3dc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 -- 172.21.15.62:0/1276822951 >> 172.21.15.62:0/1276822951 conn(0x7fa3dc0fb0e0 msgr2=0x7fa3dc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 -- 172.21.15.62:0/1276822951 shutdown_connections 2023-07-14T16:16:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.304+0000 7fa3e11ff700 1 -- 172.21.15.62:0/1276822951 wait complete. 2023-07-14T16:16:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.306+0000 7fa3e11ff700 1 Processor -- start 2023-07-14T16:16:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.306+0000 7fa3e11ff700 1 -- start start 2023-07-14T16:16:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.306+0000 7fa3e11ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc0ffae0 0x7fa3dc1957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3e11ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc100cd0 0x7fa3dc195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3e11ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 0x7fa3dc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3e11ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc19d320 con 0x7fa3dc101660 2023-07-14T16:16:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3e11ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc19d460 con 0x7fa3dc0ffae0 2023-07-14T16:16:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3e11ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc19d5f0 con 0x7fa3dc100cd0 2023-07-14T16:16:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3db59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 0x7fa3dc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3db59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 0x7fa3dc19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48706/0 (socket says 172.21.15.62:48706) 2023-07-14T16:16:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3db59e700 1 -- 172.21.15.62:0/973881552 learned_addr learned my addr 172.21.15.62:0/973881552 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc0ffae0 0x7fa3dc1957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3da59c700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc100cd0 0x7fa3dc195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3db59e700 1 -- 172.21.15.62:0/973881552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc100cd0 msgr2=0x7fa3dc195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3db59e700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc100cd0 0x7fa3dc195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3db59e700 1 -- 172.21.15.62:0/973881552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc0ffae0 msgr2=0x7fa3dc1957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3db59e700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc0ffae0 0x7fa3dc1957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.307+0000 7fa3db59e700 1 -- 172.21.15.62:0/973881552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4009d00 con 0x7fa3dc101660 2023-07-14T16:16:24.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.308+0000 7fa3db59e700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 0x7fa3dc19cd50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400bad0 tx=0x7fa3d400bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.308+0000 7fa3c3fff700 1 -- 172.21.15.62:0/973881552 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400e040 con 0x7fa3dc101660 2023-07-14T16:16:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.308+0000 7fa3c3fff700 1 -- 172.21.15.62:0/973881552 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3d4003c00 con 0x7fa3dc101660 2023-07-14T16:16:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.308+0000 7fa3c3fff700 1 -- 172.21.15.62:0/973881552 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d40052a0 con 0x7fa3dc101660 2023-07-14T16:16:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.308+0000 7fa3e11ff700 1 -- 172.21.15.62:0/973881552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc19d870 con 0x7fa3dc101660 2023-07-14T16:16:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.308+0000 7fa3e11ff700 1 -- 172.21.15.62:0/973881552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc19dca0 con 0x7fa3dc101660 2023-07-14T16:16:24.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.314+0000 7fa3c3fff700 1 -- 172.21.15.62:0/973881552 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa3d401d470 con 0x7fa3dc101660 2023-07-14T16:16:24.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.314+0000 7fa3f4b0eb80 1 Processor -- start 2023-07-14T16:16:24.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.315+0000 7fa3f4b0eb80 1 -- start start 2023-07-14T16:16:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.315+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc484d0 0x563e5fc488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.315+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.315+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc49c70 0x563e5fc4a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.315+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fb4dcb0 con 0x563e5fc49c70 2023-07-14T16:16:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.315+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fba0a40 con 0x563e5fc48e20 2023-07-14T16:16:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.315+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fb99ed0 con 0x563e5fc484d0 2023-07-14T16:16:24.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.316+0000 7fa3da59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc484d0 0x563e5fc488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.316+0000 7fa3da59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc484d0 0x563e5fc488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35934/0 (socket says 172.21.15.62:35934) 2023-07-14T16:16:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.316+0000 7fa3da59c700 1 -- 172.21.15.62:0/1815852133 learned_addr learned my addr 172.21.15.62:0/1815852133 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.316+0000 7fa3da59c700 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc49c70 0x563e5fc4a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.316+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3da59c700 1 -- 172.21.15.62:0/1815852133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 msgr2=0x563e5fc496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3da59c700 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3da59c700 1 -- 172.21.15.62:0/1815852133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc49c70 msgr2=0x563e5fc4a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3da59c700 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc49c70 0x563e5fc4a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3da59c700 1 -- 172.21.15.62:0/1815852133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e5f9c7de0 con 0x563e5fc484d0 2023-07-14T16:16:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3c3fff700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3c40a1250 0x7fa3c40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3c3fff700 1 -- 172.21.15.62:0/973881552 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa3d40cdbd0 con 0x7fa3dc101660 2023-07-14T16:16:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3da59c700 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc484d0 0x563e5fc488e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0011860 tx=0x7fa3d000a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec022e5a565d79e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.317+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3c40a1250 0x7fa3c40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.318+0000 7fa3c17fa700 1 -- 172.21.15.62:0/1815852133 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0011e80 con 0x563e5fc484d0 2023-07-14T16:16:24.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.318+0000 7fa3c17fa700 1 -- 172.21.15.62:0/1815852133 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3d00044b0 con 0x563e5fc484d0 2023-07-14T16:16:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.318+0000 7fa3c17fa700 1 -- 172.21.15.62:0/1815852133 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0005220 con 0x563e5fc484d0 2023-07-14T16:16:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1815852133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc484d0 msgr2=0x563e5fc488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc484d0 0x563e5fc488e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0011860 tx=0x7fa3d000a7f0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3c40a1250 0x7fa3c40a3700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc100b30 tx=0x7fa3cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1815852133 shutdown_connections 2023-07-14T16:16:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc49c70 0x563e5fc4a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc484d0 0x563e5fc488e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1815852133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1815852133 >> 172.21.15.62:0/1815852133 conn(0x563e5fc3dda0 msgr2=0x563e5fc431d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1815852133 shutdown_connections 2023-07-14T16:16:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.319+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1815852133 wait complete. 2023-07-14T16:16:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.320+0000 7fa3f4b0eb80 1 -- start start 2023-07-14T16:16:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.320+0000 7fa3f4b0eb80 1 Processor -- start 2023-07-14T16:16:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.320+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc56530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc48e20 0x563e5fc56a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc49c70 0x563e5fc5dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fb39ac0 con 0x563e5fc484d0 2023-07-14T16:16:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fb397c0 con 0x563e5fc49c70 2023-07-14T16:16:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fa1d920 con 0x563e5fc48e20 2023-07-14T16:16:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3f4b0eb80 10 client.0 _ll_register_callbacks cb 0x563e5fb37760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3dad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc56530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3dad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc56530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48716/0 (socket says 172.21.15.62:48716) 2023-07-14T16:16:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3dad9d700 1 -- 172.21.15.62:0/3703906950 learned_addr learned my addr 172.21.15.62:0/3703906950 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3da59c700 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc48e20 0x563e5fc56a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3db59e700 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc49c70 0x563e5fc5dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3dad9d700 1 -- 172.21.15.62:0/3703906950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc48e20 msgr2=0x563e5fc56a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc48e20 0x563e5fc56a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3dad9d700 1 -- 172.21.15.62:0/3703906950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc49c70 msgr2=0x563e5fc5dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc49c70 0x563e5fc5dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.321+0000 7fa3dad9d700 1 -- 172.21.15.62:0/3703906950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc0098e0 con 0x563e5fc484d0 2023-07-14T16:16:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.322+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc56530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097c0 tx=0x7fa3cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d9655e07b813c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.322+0000 7fa3beff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.322+0000 7fa3beff5700 1 -- 172.21.15.62:0/3703906950 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc023070 con 0x563e5fc484d0 2023-07-14T16:16:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.322+0000 7fa3beff5700 1 -- 172.21.15.62:0/3703906950 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa3cc01ed60 con 0x563e5fc484d0 2023-07-14T16:16:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.322+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/3703906950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e5faf34f0 con 0x563e5fc484d0 2023-07-14T16:16:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.322+0000 7fa3beff5700 1 -- 172.21.15.62:0/3703906950 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3cc017070 con 0x563e5fc484d0 2023-07-14T16:16:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.322+0000 7fa3f4b0eb80 10 client.15672 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.322+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/3703906950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fa3d0019040 con 0x563e5fc484d0 2023-07-14T16:16:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.323+0000 7fa3beff5700 1 -- 172.21.15.62:0/3703906950 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563e5faf34f0 con 0x563e5fc484d0 2023-07-14T16:16:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.323+0000 7fa3beff5700 1 -- 172.21.15.62:0/3703906950 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa3cc0296d0 con 0x563e5fc484d0 2023-07-14T16:16:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.323+0000 7fa3f4b0eb80 10 client.15672 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.323+0000 7fa3f4b0eb80 10 client.15672 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:16:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.323+0000 7fa3f4b0eb80 20 client.15672 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.323+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563e5fc622c0 0x563e5fc646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.323+0000 7fa3f4b0eb80 4 client.15672 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.323+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/3703906950 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e5fb92eb0 con 0x563e5fc622c0 2023-07-14T16:16:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.324+0000 7fa3da59c700 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563e5fc622c0 0x563e5fc646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.324+0000 7fa3beff5700 10 client.15672 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:24.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.324+0000 7fa3da59c700 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563e5fc622c0 0x563e5fc646d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71f18a3789912b8 server_cookie=1d4541b966a74bcc in_seq=0 out_seq=0 2023-07-14T16:16:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.325+0000 7fa3beff5700 1 -- 172.21.15.62:0/3703906950 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563e5fb92eb0 con 0x563e5fc622c0 2023-07-14T16:16:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.325+0000 7fa3beff5700 10 client.15672 handle_command_reply: tid=0 2023-07-14T16:16:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.325+0000 7fa3f4b0eb80 1 client.15672 shutdown 2023-07-14T16:16:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.325+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/3703906950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 msgr2=0x563e5fc56530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.325+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc56530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097c0 tx=0x7fa3cc0047c0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/3703906950 shutdown_connections 2023-07-14T16:16:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563e5fc622c0 0x563e5fc646d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc49c70 0x563e5fc5dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc56530 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3beff5700 0 client.15672 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/3703906950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc48e20 0x563e5fc56a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/3703906950 >> 172.21.15.62:0/3703906950 conn(0x563e5fc3dda0 msgr2=0x563e5fc428b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/3703906950 shutdown_connections 2023-07-14T16:16:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/3703906950 wait complete. 2023-07-14T16:16:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.326+0000 7fa3f4b0eb80 20 client.15672 trim_cache size 0 max 16384 2023-07-14T16:16:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.341+0000 7fa3f4b0eb80 1 Processor -- start 2023-07-14T16:16:24.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.341+0000 7fa3f4b0eb80 1 -- start start 2023-07-14T16:16:24.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.341+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc484d0 0x563e5fc488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.341+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc48e20 0x563e5fc5da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.341+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 0x563e5fc50e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.341+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fc44a00 con 0x563e5fc48e20 2023-07-14T16:16:24.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.341+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fc44410 con 0x563e5fc484d0 2023-07-14T16:16:24.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.341+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fc61fc0 con 0x563e5fc49c70 2023-07-14T16:16:24.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3da59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc48e20 0x563e5fc5da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3da59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc48e20 0x563e5fc5da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48726/0 (socket says 172.21.15.62:48726) 2023-07-14T16:16:24.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3da59c700 1 -- 172.21.15.62:0/2149830842 learned_addr learned my addr 172.21.15.62:0/2149830842 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc484d0 0x563e5fc488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3db59e700 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 0x563e5fc50e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3da59c700 1 -- 172.21.15.62:0/2149830842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 msgr2=0x563e5fc50e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3da59c700 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 0x563e5fc50e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3da59c700 1 -- 172.21.15.62:0/2149830842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc484d0 msgr2=0x563e5fc488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3da59c700 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc484d0 0x563e5fc488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.342+0000 7fa3da59c700 1 -- 172.21.15.62:0/2149830842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e5f9b0c60 con 0x563e5fc48e20 2023-07-14T16:16:24.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.343+0000 7fa3da59c700 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc48e20 0x563e5fc5da70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0011e50 tx=0x7fa3d0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ae1720682ab0d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.343+0000 7fa3c1ffb700 1 -- 172.21.15.62:0/2149830842 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0023030 con 0x563e5fc48e20 2023-07-14T16:16:24.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.343+0000 7fa3c1ffb700 1 -- 172.21.15.62:0/2149830842 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3d0015040 con 0x563e5fc48e20 2023-07-14T16:16:24.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.343+0000 7fa3c1ffb700 1 -- 172.21.15.62:0/2149830842 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0005050 con 0x563e5fc48e20 2023-07-14T16:16:24.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/2149830842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc48e20 msgr2=0x563e5fc5da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc48e20 0x563e5fc5da70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0011e50 tx=0x7fa3d0013040 comp rx=0 tx=0).stop 2023-07-14T16:16:24.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/2149830842 shutdown_connections 2023-07-14T16:16:24.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 0x563e5fc50e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc484d0 0x563e5fc488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/2149830842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc48e20 0x563e5fc5da70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/2149830842 >> 172.21.15.62:0/2149830842 conn(0x563e5fc3dda0 msgr2=0x563e5fc69030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/2149830842 shutdown_connections 2023-07-14T16:16:24.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.344+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/2149830842 wait complete. 2023-07-14T16:16:24.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.345+0000 7fa3f4b0eb80 1 -- start start 2023-07-14T16:16:24.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.345+0000 7fa3f4b0eb80 1 Processor -- start 2023-07-14T16:16:24.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc59d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3f4b0eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 0x563e5fc5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fc5d270 con 0x563e5fc484d0 2023-07-14T16:16:24.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fc5d3e0 con 0x563e5fc48e20 2023-07-14T16:16:24.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3f4b0eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5fc5d5a0 con 0x563e5fc49c70 2023-07-14T16:16:24.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3f4b0eb80 10 client.0 _ll_register_callbacks cb 0x7fa3cc01e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:24.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3dad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3dad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48730/0 (socket says 172.21.15.62:48730) 2023-07-14T16:16:24.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3dad9d700 1 -- 172.21.15.62:0/1551709471 learned_addr learned my addr 172.21.15.62:0/1551709471 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3da59c700 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc59d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.346+0000 7fa3db59e700 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 0x563e5fc5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.347+0000 7fa3dad9d700 1 -- 172.21.15.62:0/1551709471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 msgr2=0x563e5fc5cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.347+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 0x563e5fc5cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.347+0000 7fa3dad9d700 1 -- 172.21.15.62:0/1551709471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 msgr2=0x563e5fc59d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.347+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc59d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.347+0000 7fa3dad9d700 1 -- 172.21.15.62:0/1551709471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e5f9c7de0 con 0x563e5fc484d0 2023-07-14T16:16:24.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.347+0000 7fa3da59c700 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc59d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.347+0000 7fa3dad9d700 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc597d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0296d0 tx=0x7fa3cc029700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d93b6b343322162 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.347+0000 7fa3beff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.348+0000 7fa3beff5700 1 -- 172.21.15.62:0/1551709471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc023070 con 0x563e5fc484d0 2023-07-14T16:16:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.348+0000 7fa3beff5700 1 -- 172.21.15.62:0/1551709471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa3cc009cf0 con 0x563e5fc484d0 2023-07-14T16:16:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.348+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1551709471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3d00044b0 con 0x563e5fc484d0 2023-07-14T16:16:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.348+0000 7fa3beff5700 1 -- 172.21.15.62:0/1551709471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3cc017040 con 0x563e5fc484d0 2023-07-14T16:16:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.348+0000 7fa3f4b0eb80 10 client.15678 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.348+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1551709471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563e5f9ee250 con 0x563e5fc484d0 2023-07-14T16:16:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.348+0000 7fa3beff5700 1 -- 172.21.15.62:0/1551709471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3d00044b0 con 0x563e5fc484d0 2023-07-14T16:16:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.349+0000 7fa3beff5700 1 -- 172.21.15.62:0/1551709471 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa3cc043020 con 0x563e5fc484d0 2023-07-14T16:16:24.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.349+0000 7fa3f4b0eb80 10 client.15678 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:24.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.349+0000 7fa3f4b0eb80 10 client.15678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:16:24.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.349+0000 7fa3f4b0eb80 20 client.15678 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:24.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.349+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563e5fcb7030 0x563e5fcb9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.349+0000 7fa3f4b0eb80 4 client.15678 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:24.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.349+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1551709471 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e5fb55050 con 0x563e5fcb7030 2023-07-14T16:16:24.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.349+0000 7fa3da59c700 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563e5fcb7030 0x563e5fcb9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3da59c700 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563e5fcb7030 0x563e5fcb9440 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1304c341ef4b8c3d server_cookie=69f1a07d44fa100c in_seq=0 out_seq=0 2023-07-14T16:16:24.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3beff5700 10 client.15678 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3beff5700 1 -- 172.21.15.62:0/1551709471 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e5fb55050 con 0x563e5fcb7030 2023-07-14T16:16:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3beff5700 10 client.15678 handle_command_reply: tid=0 2023-07-14T16:16:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3f4b0eb80 1 client.15678 shutdown 2023-07-14T16:16:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1551709471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 msgr2=0x563e5fc597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc597d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0296d0 tx=0x7fa3cc029700 comp rx=0 tx=0).stop 2023-07-14T16:16:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1551709471 shutdown_connections 2023-07-14T16:16:24.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563e5fcb7030 0x563e5fcb9440 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.350+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5fc49c70 0x563e5fc5cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563e5fc484d0 0x563e5fc597d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3f4b0eb80 1 --2- 172.21.15.62:0/1551709471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e5fc48e20 0x563e5fc59d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1551709471 >> 172.21.15.62:0/1551709471 conn(0x563e5fc3dda0 msgr2=0x563e5fc50e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3beff5700 0 client.15678 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1551709471 shutdown_connections 2023-07-14T16:16:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3f4b0eb80 1 -- 172.21.15.62:0/1551709471 wait complete. 2023-07-14T16:16:24.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3f4b0eb80 20 client.15678 trim_cache size 0 max 16384 2023-07-14T16:16:24.334 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:24.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3e11ff700 1 -- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3c40a1250 msgr2=0x7fa3c40a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3c40a1250 0x7fa3c40a3700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc100b30 tx=0x7fa3cc009260 comp rx=0 tx=0).stop 2023-07-14T16:16:24.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3e11ff700 1 -- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 msgr2=0x7fa3dc19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.351+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 0x7fa3dc19cd50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400bad0 tx=0x7fa3d400bb00 comp rx=0 tx=0).stop 2023-07-14T16:16:24.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.352+0000 7fa3e11ff700 1 -- 172.21.15.62:0/973881552 shutdown_connections 2023-07-14T16:16:24.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.352+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3dc101660 0x7fa3dc19cd50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.352+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3c40a1250 0x7fa3c40a3700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.352+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3dc0ffae0 0x7fa3dc1957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.352+0000 7fa3e11ff700 1 --2- 172.21.15.62:0/973881552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3dc100cd0 0x7fa3dc195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.352+0000 7fa3e11ff700 1 -- 172.21.15.62:0/973881552 >> 172.21.15.62:0/973881552 conn(0x7fa3dc0fb0e0 msgr2=0x7fa3dc0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.352+0000 7fa3e11ff700 1 -- 172.21.15.62:0/973881552 shutdown_connections 2023-07-14T16:16:24.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.352+0000 7fa3e11ff700 1 -- 172.21.15.62:0/973881552 wait complete. 2023-07-14T16:16:24.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:16:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.608+0000 7ff40bfff700 1 -- 172.21.15.62:0/3899597375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c105690 msgr2=0x7ff40c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.608+0000 7ff40bfff700 1 --2- 172.21.15.62:0/3899597375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c105690 0x7ff40c107af0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b0a0 tx=0x7ff40400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.609+0000 7ff40bfff700 1 -- 172.21.15.62:0/3899597375 shutdown_connections 2023-07-14T16:16:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.609+0000 7ff40bfff700 1 --2- 172.21.15.62:0/3899597375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c105690 0x7ff40c107af0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.609+0000 7ff40bfff700 1 --2- 172.21.15.62:0/3899597375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff40c100450 0x7ff40c102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.609+0000 7ff40bfff700 1 --2- 172.21.15.62:0/3899597375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40c102d70 0x7ff40c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.609+0000 7ff40bfff700 1 -- 172.21.15.62:0/3899597375 >> 172.21.15.62:0/3899597375 conn(0x7ff40c0f9ed0 msgr2=0x7ff40c0fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.609+0000 7ff40bfff700 1 -- 172.21.15.62:0/3899597375 shutdown_connections 2023-07-14T16:16:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.609+0000 7ff40bfff700 1 -- 172.21.15.62:0/3899597375 wait complete. 2023-07-14T16:16:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.610+0000 7ff40bfff700 1 Processor -- start 2023-07-14T16:16:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.610+0000 7ff40bfff700 1 -- start start 2023-07-14T16:16:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c100450 0x7ff40c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff40c102d70 0x7ff40c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40c105690 0x7ff40c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c19f320 con 0x7ff40c100450 2023-07-14T16:16:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40bfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c19f460 con 0x7ff40c102d70 2023-07-14T16:16:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40bfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c19f5f0 con 0x7ff40c105690 2023-07-14T16:16:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c100450 0x7ff40c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c100450 0x7ff40c1977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48756/0 (socket says 172.21.15.62:48756) 2023-07-14T16:16:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40affd700 1 -- 172.21.15.62:0/2030144120 learned_addr learned my addr 172.21.15.62:0/2030144120 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff40c102d70 0x7ff40c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.611+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40c105690 0x7ff40c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff40affd700 1 -- 172.21.15.62:0/2030144120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40c105690 msgr2=0x7ff40c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff40affd700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40c105690 0x7ff40c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff40affd700 1 -- 172.21.15.62:0/2030144120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff40c102d70 msgr2=0x7ff40c197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff40affd700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff40c102d70 0x7ff40c197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff40affd700 1 -- 172.21.15.62:0/2030144120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc009520 con 0x7ff40c100450 2023-07-14T16:16:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff40c102d70 0x7ff40c197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff40affd700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c100450 0x7ff40c1977a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00eb30 tx=0x7ff3fc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff3f3fff700 1 -- 172.21.15.62:0/2030144120 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc009880 con 0x7ff40c100450 2023-07-14T16:16:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff3f3fff700 1 -- 172.21.15.62:0/2030144120 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3fc0044b0 con 0x7ff40c100450 2023-07-14T16:16:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff3f3fff700 1 -- 172.21.15.62:0/2030144120 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc003ce0 con 0x7ff40c100450 2023-07-14T16:16:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.612+0000 7ff40bfff700 1 -- 172.21.15.62:0/2030144120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404009d00 con 0x7ff40c100450 2023-07-14T16:16:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.613+0000 7ff40bfff700 1 -- 172.21.15.62:0/2030144120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c19fc50 con 0x7ff40c100450 2023-07-14T16:16:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.619+0000 7ff424d15b80 1 Processor -- start 2023-07-14T16:16:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.619+0000 7ff3f3fff700 1 -- 172.21.15.62:0/2030144120 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff3fc003e40 con 0x7ff40c100450 2023-07-14T16:16:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.619+0000 7ff424d15b80 1 -- start start 2023-07-14T16:16:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff137b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff1392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff424d15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff03ccb0 con 0x55baff137720 2023-07-14T16:16:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff424d15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff08fa40 con 0x55baff138e30 2023-07-14T16:16:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff424d15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff088ed0 con 0x55baff138070 2023-07-14T16:16:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff40affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff40affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1388a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35978/0 (socket says 172.21.15.62:35978) 2023-07-14T16:16:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff40affd700 1 -- 172.21.15.62:0/930513582 learned_addr learned my addr 172.21.15.62:0/930513582 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.620+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff1392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40affd700 1 -- 172.21.15.62:0/930513582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 msgr2=0x55baff1392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40affd700 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff1392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40affd700 1 -- 172.21.15.62:0/930513582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 msgr2=0x55baff137b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40affd700 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff137b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40affd700 1 -- 172.21.15.62:0/930513582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bafeeb6de0 con 0x55baff138070 2023-07-14T16:16:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff1392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.621+0000 7ff40affd700 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1388a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0d0df0 tx=0x7ff3fc0d53c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a0578a592524f27 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.622+0000 7ff3f3fff700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3f40a1270 0x7ff3f40a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.622+0000 7ff3f17fa700 1 -- 172.21.15.62:0/930513582 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0d5bb0 con 0x55baff138070 2023-07-14T16:16:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.622+0000 7ff3f17fa700 1 -- 172.21.15.62:0/930513582 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3fc0d5d10 con 0x55baff138070 2023-07-14T16:16:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.622+0000 7ff3f17fa700 1 -- 172.21.15.62:0/930513582 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0fc610 con 0x55baff138070 2023-07-14T16:16:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.622+0000 7ff3f3fff700 1 -- 172.21.15.62:0/2030144120 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3fc014070 con 0x7ff40c100450 2023-07-14T16:16:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.622+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3f40a1270 0x7ff3f40a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.623+0000 7ff424d15b80 1 -- 172.21.15.62:0/930513582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 msgr2=0x55baff1388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.623+0000 7ff424d15b80 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1388a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0d0df0 tx=0x7ff3fc0d53c0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.623+0000 7ff424d15b80 1 -- 172.21.15.62:0/930513582 shutdown_connections 2023-07-14T16:16:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.623+0000 7ff424d15b80 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff1392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.623+0000 7ff424d15b80 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.623+0000 7ff424d15b80 1 --2- 172.21.15.62:0/930513582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1388a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.624+0000 7ff424d15b80 1 -- 172.21.15.62:0/930513582 >> 172.21.15.62:0/930513582 conn(0x55baff12cff0 msgr2=0x55baff132460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.624+0000 7ff424d15b80 1 -- 172.21.15.62:0/930513582 shutdown_connections 2023-07-14T16:16:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.624+0000 7ff424d15b80 1 -- 172.21.15.62:0/930513582 wait complete. 2023-07-14T16:16:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.624+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3f40a1270 0x7ff3f40a3720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b780 tx=0x7ff404006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.624+0000 7ff424d15b80 1 -- start start 2023-07-14T16:16:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.625+0000 7ff424d15b80 1 Processor -- start 2023-07-14T16:16:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.625+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff141180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.625+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.625+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff148700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.625+0000 7ff424d15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff028ac0 con 0x55baff137720 2023-07-14T16:16:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.625+0000 7ff424d15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff0287c0 con 0x55baff138e30 2023-07-14T16:16:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.625+0000 7ff424d15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bafef0c920 con 0x55baff138070 2023-07-14T16:16:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.625+0000 7ff424d15b80 10 client.0 _ll_register_callbacks cb 0x55baff026760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.626+0000 7ff40affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.626+0000 7ff40affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff141180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48784/0 (socket says 172.21.15.62:48784) 2023-07-14T16:16:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.626+0000 7ff40affd700 1 -- 172.21.15.62:0/2388718399 learned_addr learned my addr 172.21.15.62:0/2388718399 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.626+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.626+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.627+0000 7ff40affd700 1 -- 172.21.15.62:0/2388718399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 msgr2=0x55baff1416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.627+0000 7ff40affd700 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.627+0000 7ff40affd700 1 -- 172.21.15.62:0/2388718399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 msgr2=0x55baff148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.627+0000 7ff40affd700 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.627+0000 7ff40affd700 1 -- 172.21.15.62:0/2388718399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c1086e0 con 0x55baff137720 2023-07-14T16:16:24.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.627+0000 7ff40affd700 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff141180 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0a23b0 tx=0x7ff3fc0cd170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539f58e3368d78a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.627+0000 7ff3eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:24.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.627+0000 7ff3eeff5700 1 -- 172.21.15.62:0/2388718399 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0d5bb0 con 0x55baff137720 2023-07-14T16:16:24.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.628+0000 7ff3eeff5700 1 -- 172.21.15.62:0/2388718399 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3fc0af020 con 0x55baff137720 2023-07-14T16:16:24.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.628+0000 7ff424d15b80 1 -- 172.21.15.62:0/2388718399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bafefe24f0 con 0x55baff137720 2023-07-14T16:16:24.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.628+0000 7ff3eeff5700 1 -- 172.21.15.62:0/2388718399 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3fc0d0460 con 0x55baff137720 2023-07-14T16:16:24.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.628+0000 7ff424d15b80 10 client.15690 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:24.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.628+0000 7ff424d15b80 1 -- 172.21.15.62:0/2388718399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff40c19fc50 con 0x55baff137720 2023-07-14T16:16:24.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.628+0000 7ff3eeff5700 1 -- 172.21.15.62:0/2388718399 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3fc0b6070 con 0x55baff137720 2023-07-14T16:16:24.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.629+0000 7ff3eeff5700 1 -- 172.21.15.62:0/2388718399 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff3fc09e120 con 0x55baff137720 2023-07-14T16:16:24.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.629+0000 7ff424d15b80 10 client.15690 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:24.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.629+0000 7ff424d15b80 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:24.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.629+0000 7ff424d15b80 20 client.15690 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:24.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.629+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55baff14cb70 0x55baff14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.629+0000 7ff424d15b80 4 client.15690 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:24.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.629+0000 7ff424d15b80 1 -- 172.21.15.62:0/2388718399 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55baff081eb0 con 0x55baff14cb70 2023-07-14T16:16:24.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.629+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55baff14cb70 0x55baff14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.630+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55baff14cb70 0x55baff14ef80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd614e320938b1b server_cookie=c1c3f95b4b42fc8c in_seq=0 out_seq=0 2023-07-14T16:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.630+0000 7ff3eeff5700 10 client.15690 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff3eeff5700 1 -- 172.21.15.62:0/2388718399 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55baff081eb0 con 0x55baff14cb70 2023-07-14T16:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff3eeff5700 10 client.15690 handle_command_reply: tid=0 2023-07-14T16:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 client.15690 shutdown 2023-07-14T16:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 -- 172.21.15.62:0/2388718399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 msgr2=0x55baff141180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff141180 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0a23b0 tx=0x7ff3fc0cd170 comp rx=0 tx=0).stop 2023-07-14T16:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 -- 172.21.15.62:0/2388718399 shutdown_connections 2023-07-14T16:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55baff14cb70 0x55baff14ef80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff141180 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2388718399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff424d15b80 1 -- 172.21.15.62:0/2388718399 >> 172.21.15.62:0/2388718399 conn(0x55baff12cff0 msgr2=0x55baff131b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.631+0000 7ff3eeff5700 0 client.15690 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:24.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.632+0000 7ff424d15b80 1 -- 172.21.15.62:0/2388718399 shutdown_connections 2023-07-14T16:16:24.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.632+0000 7ff424d15b80 1 -- 172.21.15.62:0/2388718399 wait complete. 2023-07-14T16:16:24.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.632+0000 7ff424d15b80 20 client.15690 trim_cache size 0 max 16384 2023-07-14T16:16:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.646+0000 7ff424d15b80 1 Processor -- start 2023-07-14T16:16:24.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.646+0000 7ff424d15b80 1 -- start start 2023-07-14T16:16:24.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.646+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff140830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.646+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.646+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff13bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.646+0000 7ff424d15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff12cb70 con 0x55baff137720 2023-07-14T16:16:24.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.646+0000 7ff424d15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff12e920 con 0x55baff138e30 2023-07-14T16:16:24.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.646+0000 7ff424d15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff12eae0 con 0x55baff138070 2023-07-14T16:16:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40a7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff140830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48790/0 (socket says 172.21.15.62:48790) 2023-07-14T16:16:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40a7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35998/0 (socket says 172.21.15.62:35998) 2023-07-14T16:16:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40affd700 1 -- 172.21.15.62:0/2144022537 learned_addr learned my addr 172.21.15.62:0/2144022537 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/2144022537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff13bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40a7fc700 1 -- 172.21.15.62:0/2144022537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 msgr2=0x55baff13bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/2144022537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff13bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40a7fc700 1 -- 172.21.15.62:0/2144022537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 msgr2=0x55baff140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/2144022537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.647+0000 7ff40a7fc700 1 -- 172.21.15.62:0/2144022537 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bafee9fc60 con 0x55baff138070 2023-07-14T16:16:24.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:24 smithi062 ceph-mon[95543]: pgmap v380: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-14T16:16:24.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:24 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1168617193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.648+0000 7ff40a7fc700 1 --2- 172.21.15.62:0/2144022537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1486c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc096eb0 tx=0x7ff400009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2307976902e27d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.648+0000 7ff3f1ffb700 1 -- 172.21.15.62:0/2144022537 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400017070 con 0x55baff138070 2023-07-14T16:16:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.648+0000 7ff3f1ffb700 1 -- 172.21.15.62:0/2144022537 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff400018040 con 0x55baff138070 2023-07-14T16:16:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.648+0000 7ff3f1ffb700 1 -- 172.21.15.62:0/2144022537 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400004510 con 0x55baff138070 2023-07-14T16:16:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 -- 172.21.15.62:0/2144022537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 msgr2=0x55baff1486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2144022537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1486c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc096eb0 tx=0x7ff400009260 comp rx=0 tx=0).stop 2023-07-14T16:16:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 -- 172.21.15.62:0/2144022537 shutdown_connections 2023-07-14T16:16:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2144022537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138e30 0x55baff13bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2144022537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff137720 0x55baff140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 --2- 172.21.15.62:0/2144022537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff138070 0x55baff1486c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 -- 172.21.15.62:0/2144022537 >> 172.21.15.62:0/2144022537 conn(0x55baff12cff0 msgr2=0x55baff13fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 -- 172.21.15.62:0/2144022537 shutdown_connections 2023-07-14T16:16:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.649+0000 7ff424d15b80 1 -- 172.21.15.62:0/2144022537 wait complete. 2023-07-14T16:16:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.650+0000 7ff424d15b80 1 -- start start 2023-07-14T16:16:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.650+0000 7ff424d15b80 1 Processor -- start 2023-07-14T16:16:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.650+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff137720 0x55baff143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.650+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138070 0x55baff143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.650+0000 7ff424d15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff138e30 0x55baff1a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.650+0000 7ff424d15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff144080 con 0x55baff138e30 2023-07-14T16:16:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.650+0000 7ff424d15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff1a8bf0 con 0x55baff138070 2023-07-14T16:16:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.650+0000 7ff424d15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baff1a8d30 con 0x55baff137720 2023-07-14T16:16:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff424d15b80 10 client.0 _ll_register_callbacks cb 0x7ff3fc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138070 0x55baff143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138070 0x55baff143aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48284/0 (socket says 172.21.15.62:48284) 2023-07-14T16:16:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40a7fc700 1 -- 172.21.15.62:0/3380268755 learned_addr learned my addr 172.21.15.62:0/3380268755 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff138e30 0x55baff1a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40affd700 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff137720 0x55baff143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40b7fe700 1 -- 172.21.15.62:0/3380268755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff137720 msgr2=0x55baff143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff137720 0x55baff143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40b7fe700 1 -- 172.21.15.62:0/3380268755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138070 msgr2=0x55baff143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138070 0x55baff143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.651+0000 7ff40b7fe700 1 -- 172.21.15.62:0/3380268755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff400012050 con 0x55baff138e30 2023-07-14T16:16:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.652+0000 7ff40b7fe700 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff138e30 0x55baff1a86b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b070 tx=0x7ff4040099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c200fa34945f0bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.652+0000 7ff3eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.652+0000 7ff3eeff5700 1 -- 172.21.15.62:0/3380268755 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40400e040 con 0x55baff138e30 2023-07-14T16:16:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.652+0000 7ff424d15b80 1 -- 172.21.15.62:0/3380268755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3fc0d5d10 con 0x55baff138e30 2023-07-14T16:16:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.652+0000 7ff3eeff5700 1 -- 172.21.15.62:0/3380268755 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff4040052c0 con 0x55baff138e30 2023-07-14T16:16:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.652+0000 7ff3eeff5700 1 -- 172.21.15.62:0/3380268755 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff404005480 con 0x55baff138e30 2023-07-14T16:16:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.653+0000 7ff424d15b80 10 client.15696 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.653+0000 7ff424d15b80 1 -- 172.21.15.62:0/3380268755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bafeedd250 con 0x55baff138e30 2023-07-14T16:16:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.653+0000 7ff3eeff5700 1 -- 172.21.15.62:0/3380268755 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3fc0d5d10 con 0x55baff138e30 2023-07-14T16:16:24.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.654+0000 7ff3eeff5700 1 -- 172.21.15.62:0/3380268755 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4040055e0 con 0x55baff138e30 2023-07-14T16:16:24.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.654+0000 7ff424d15b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:24.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.654+0000 7ff424d15b80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:24.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.654+0000 7ff424d15b80 20 client.15696 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:24.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.654+0000 7ff424d15b80 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55baff1aa670 0x55baff1aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:24.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.654+0000 7ff424d15b80 4 client.15696 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:24.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.654+0000 7ff424d15b80 1 -- 172.21.15.62:0/3380268755 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55baff044050 con 0x55baff1aa670 2023-07-14T16:16:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.654+0000 7ff40affd700 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55baff1aa670 0x55baff1aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff40affd700 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55baff1aa670 0x55baff1aca80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7acd5a7494de6a3 server_cookie=80b3e93b445ceb9d in_seq=0 out_seq=0 2023-07-14T16:16:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff3eeff5700 10 client.15696 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff3eeff5700 1 -- 172.21.15.62:0/3380268755 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55baff044050 con 0x55baff1aa670 2023-07-14T16:16:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff3eeff5700 10 client.15696 handle_command_reply: tid=0 2023-07-14T16:16:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 client.15696 shutdown 2023-07-14T16:16:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 -- 172.21.15.62:0/3380268755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff138e30 msgr2=0x55baff1a86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff138e30 0x55baff1a86b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b070 tx=0x7ff4040099e0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 -- 172.21.15.62:0/3380268755 shutdown_connections 2023-07-14T16:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baff138e30 0x55baff1a86b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55baff137720 0x55baff143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55baff1aa670 0x55baff1aca80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 --2- 172.21.15.62:0/3380268755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55baff138070 0x55baff143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff424d15b80 1 -- 172.21.15.62:0/3380268755 >> 172.21.15.62:0/3380268755 conn(0x55baff12cff0 msgr2=0x55baff13bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.655+0000 7ff3eeff5700 0 client.15696 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:24.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.656+0000 7ff424d15b80 1 -- 172.21.15.62:0/3380268755 shutdown_connections 2023-07-14T16:16:24.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.656+0000 7ff424d15b80 1 -- 172.21.15.62:0/3380268755 wait complete. 2023-07-14T16:16:24.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.656+0000 7ff424d15b80 20 client.15696 trim_cache size 0 max 16384 2023-07-14T16:16:24.641 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:24.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.656+0000 7ff40bfff700 1 -- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3f40a1270 msgr2=0x7ff3f40a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.656+0000 7ff40bfff700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3f40a1270 0x7ff3f40a3720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff40400b780 tx=0x7ff404006210 comp rx=0 tx=0).stop 2023-07-14T16:16:24.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.656+0000 7ff40bfff700 1 -- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c100450 msgr2=0x7ff40c1977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.656+0000 7ff40bfff700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c100450 0x7ff40c1977a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00eb30 tx=0x7ff3fc00ee40 comp rx=0 tx=0).stop 2023-07-14T16:16:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.657+0000 7ff40bfff700 1 -- 172.21.15.62:0/2030144120 shutdown_connections 2023-07-14T16:16:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.657+0000 7ff40bfff700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff40c105690 0x7ff40c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.657+0000 7ff40bfff700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c100450 0x7ff40c1977a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.657+0000 7ff40bfff700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3f40a1270 0x7ff3f40a3720 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.657+0000 7ff40bfff700 1 --2- 172.21.15.62:0/2030144120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff40c102d70 0x7ff40c197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.657+0000 7ff40bfff700 1 -- 172.21.15.62:0/2030144120 >> 172.21.15.62:0/2030144120 conn(0x7ff40c0f9ed0 msgr2=0x7ff40c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:24.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.657+0000 7ff40bfff700 1 -- 172.21.15.62:0/2030144120 shutdown_connections 2023-07-14T16:16:24.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:24.657+0000 7ff40bfff700 1 -- 172.21.15.62:0/2030144120 wait complete. 2023-07-14T16:16:24.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:24 smithi081 ceph-mon[127795]: pgmap v380: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-14T16:16:24.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:24 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1168617193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:25.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:24 smithi164 ceph-mon[97427]: pgmap v380: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-14T16:16:25.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:24 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1168617193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:25.657 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:16:25.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:16:25.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.925+0000 7f223dd9d700 1 -- 172.21.15.62:0/2670845429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2238102dd0 msgr2=0x7f22381051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.925+0000 7f223dd9d700 1 --2- 172.21.15.62:0/2670845429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2238102dd0 0x7f22381051b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2224009990 tx=0x7f2224009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.926+0000 7f223dd9d700 1 -- 172.21.15.62:0/2670845429 shutdown_connections 2023-07-14T16:16:25.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.926+0000 7f223dd9d700 1 --2- 172.21.15.62:0/2670845429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22381056f0 0x7f2238107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.926+0000 7f223dd9d700 1 --2- 172.21.15.62:0/2670845429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22381004b0 0x7f2238102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.926+0000 7f223dd9d700 1 --2- 172.21.15.62:0/2670845429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2238102dd0 0x7f22381051b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.926+0000 7f223dd9d700 1 -- 172.21.15.62:0/2670845429 >> 172.21.15.62:0/2670845429 conn(0x7f22380f9ed0 msgr2=0x7f22380fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:25.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.926+0000 7f223dd9d700 1 -- 172.21.15.62:0/2670845429 shutdown_connections 2023-07-14T16:16:25.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.926+0000 7f223dd9d700 1 -- 172.21.15.62:0/2670845429 wait complete. 2023-07-14T16:16:25.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.927+0000 7f223dd9d700 1 Processor -- start 2023-07-14T16:16:25.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223dd9d700 1 -- start start 2023-07-14T16:16:25.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223dd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22381004b0 0x7f22380ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223dd9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2238102dd0 0x7f22380fe460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223dd9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22381056f0 0x7f22380fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223dd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380ff140 con 0x7f22381004b0 2023-07-14T16:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223dd9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380ff2b0 con 0x7f2238102dd0 2023-07-14T16:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223dd9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223819f5a0 con 0x7f22381056f0 2023-07-14T16:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223cd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22381004b0 0x7f22380ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.928+0000 7f223cd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22381004b0 0x7f22380ffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48818/0 (socket says 172.21.15.62:48818) 2023-07-14T16:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223cd9b700 1 -- 172.21.15.62:0/1242527401 learned_addr learned my addr 172.21.15.62:0/1242527401 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:25.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223d59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22381056f0 0x7f22380fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f2237fff700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2238102dd0 0x7f22380fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223cd9b700 1 -- 172.21.15.62:0/1242527401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22381056f0 msgr2=0x7f22380fe9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22381056f0 0x7f22380fe9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223cd9b700 1 -- 172.21.15.62:0/1242527401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2238102dd0 msgr2=0x7f22380fe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2238102dd0 0x7f22380fe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223cd9b700 1 -- 172.21.15.62:0/1242527401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22240095f0 con 0x7f22381004b0 2023-07-14T16:16:25.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223d59c700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22381056f0 0x7f22380fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:25.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.929+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22381004b0 0x7f22380ffe20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f222c00e9a0 tx=0x7f222c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:25.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.930+0000 7f2235ffb700 1 -- 172.21.15.62:0/1242527401 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c009790 con 0x7f22381004b0 2023-07-14T16:16:25.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.930+0000 7f223dd9d700 1 -- 172.21.15.62:0/1242527401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223819f880 con 0x7f22381004b0 2023-07-14T16:16:25.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.930+0000 7f2235ffb700 1 -- 172.21.15.62:0/1242527401 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f222c010430 con 0x7f22381004b0 2023-07-14T16:16:25.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.930+0000 7f2235ffb700 1 -- 172.21.15.62:0/1242527401 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00cea0 con 0x7f22381004b0 2023-07-14T16:16:25.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.930+0000 7f223dd9d700 1 -- 172.21.15.62:0/1242527401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223819fd90 con 0x7f22381004b0 2023-07-14T16:16:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.935+0000 7f2235ffb700 1 -- 172.21.15.62:0/1242527401 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f222c0105a0 con 0x7f22381004b0 2023-07-14T16:16:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f22529eab80 1 Processor -- start 2023-07-14T16:16:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f22529eab80 1 -- start start 2023-07-14T16:16:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f22529eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f22529eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f22529eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f22529eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e6b0d30 con 0x55b50e7aceb0 2023-07-14T16:16:25.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f22529eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e703ac0 con 0x55b50e7ac0f0 2023-07-14T16:16:25.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f22529eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e6fcf50 con 0x55b50e7ab7a0 2023-07-14T16:16:25.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f2235ffb700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22280a11f0 0x7f22280a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.937+0000 7f2235ffb700 1 -- 172.21.15.62:0/1242527401 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f222c014070 con 0x7f22381004b0 2023-07-14T16:16:25.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f223d59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f2237fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f2237fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7ad320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48822/0 (socket says 172.21.15.62:48822) 2023-07-14T16:16:25.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f2237fff700 1 -- 172.21.15.62:0/2196319376 learned_addr learned my addr 172.21.15.62:0/2196319376 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:25.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f223cd9b700 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f223d59c700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22280a11f0 0x7f22280a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f2237fff700 1 -- 172.21.15.62:0/2196319376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 msgr2=0x55b50e7abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:25.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f2237fff700 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f2237fff700 1 -- 172.21.15.62:0/2196319376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 msgr2=0x55b50e7ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:25.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f2237fff700 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f2237fff700 1 -- 172.21.15.62:0/2196319376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b50e52ae60 con 0x55b50e7aceb0 2023-07-14T16:16:25.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.938+0000 7f223d59c700 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:25.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.939+0000 7f2237fff700 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7ad320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2224009990 tx=0x7f2224005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe25583092f64df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:25.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.939+0000 7f223d59c700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22280a11f0 0x7f22280a36a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2230009690 tx=0x7f2230009130 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:25.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.939+0000 7f2222ffd700 1 -- 172.21.15.62:0/2196319376 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222401d070 con 0x55b50e7aceb0 2023-07-14T16:16:25.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.939+0000 7f2222ffd700 1 -- 172.21.15.62:0/2196319376 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f222401e040 con 0x55b50e7aceb0 2023-07-14T16:16:25.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.939+0000 7f2222ffd700 1 -- 172.21.15.62:0/2196319376 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400b710 con 0x55b50e7aceb0 2023-07-14T16:16:25.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 -- 172.21.15.62:0/2196319376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 msgr2=0x55b50e7ad320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7ad320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2224009990 tx=0x7f2224005f00 comp rx=0 tx=0).stop 2023-07-14T16:16:25.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 -- 172.21.15.62:0/2196319376 shutdown_connections 2023-07-14T16:16:25.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7ad320 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 --2- 172.21.15.62:0/2196319376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 -- 172.21.15.62:0/2196319376 >> 172.21.15.62:0/2196319376 conn(0x55b50e7a1070 msgr2=0x55b50e7a64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:25.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 -- 172.21.15.62:0/2196319376 shutdown_connections 2023-07-14T16:16:25.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.940+0000 7f22529eab80 1 -- 172.21.15.62:0/2196319376 wait complete. 2023-07-14T16:16:25.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.941+0000 7f22529eab80 1 -- start start 2023-07-14T16:16:25.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.941+0000 7f22529eab80 1 Processor -- start 2023-07-14T16:16:25.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f22529eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f22529eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7b5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f22529eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7bc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f22529eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e69cb40 con 0x55b50e7aceb0 2023-07-14T16:16:25.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f22529eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e69c840 con 0x55b50e7ac0f0 2023-07-14T16:16:25.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f22529eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e5809a0 con 0x55b50e7ab7a0 2023-07-14T16:16:25.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f22529eab80 10 client.0 _ll_register_callbacks cb 0x55b50e69a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:25.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f223d59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7bc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f223d59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7bc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48828/0 (socket says 172.21.15.62:48828) 2023-07-14T16:16:25.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f223d59c700 1 -- 172.21.15.62:0/3644212534 learned_addr learned my addr 172.21.15.62:0/3644212534 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:25.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.942+0000 7f223cd9b700 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f2237fff700 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f223d59c700 1 -- 172.21.15.62:0/3644212534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 msgr2=0x55b50e7b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:25.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f223d59c700 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f223d59c700 1 -- 172.21.15.62:0/3644212534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 msgr2=0x55b50e7b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:25.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f223d59c700 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f223d59c700 1 -- 172.21.15.62:0/3644212534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2230011040 con 0x55b50e7aceb0 2023-07-14T16:16:25.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f223cd9b700 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:25.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f223d59c700 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7bc870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2230019e40 tx=0x7f2230004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faf5e00e5cd1809 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:25.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.943+0000 7f2217fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:25.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.944+0000 7f2217fff700 1 -- 172.21.15.62:0/3644212534 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230022070 con 0x55b50e7aceb0 2023-07-14T16:16:25.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.944+0000 7f22529eab80 1 -- 172.21.15.62:0/3644212534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b50e656570 con 0x55b50e7aceb0 2023-07-14T16:16:25.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.944+0000 7f2217fff700 1 -- 172.21.15.62:0/3644212534 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2230004240 con 0x55b50e7aceb0 2023-07-14T16:16:25.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.944+0000 7f2217fff700 1 -- 172.21.15.62:0/3644212534 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2230004bb0 con 0x55b50e7aceb0 2023-07-14T16:16:25.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.944+0000 7f22529eab80 10 client.15708 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:25.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.944+0000 7f22529eab80 1 -- 172.21.15.62:0/3644212534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f22381086e0 con 0x55b50e7aceb0 2023-07-14T16:16:25.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.944+0000 7f2217fff700 1 -- 172.21.15.62:0/3644212534 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b50e656570 con 0x55b50e7aceb0 2023-07-14T16:16:25.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.945+0000 7f2217fff700 1 -- 172.21.15.62:0/3644212534 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f22300314c0 con 0x55b50e7aceb0 2023-07-14T16:16:25.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.945+0000 7f22529eab80 10 client.15708 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:25.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.945+0000 7f22529eab80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:25.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.945+0000 7f22529eab80 20 client.15708 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:25.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.945+0000 7f22529eab80 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b50e7c1090 0x55b50e7c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.945+0000 7f22529eab80 4 client.15708 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:25.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.945+0000 7f22529eab80 1 -- 172.21.15.62:0/3644212534 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b50e6f5f30 con 0x55b50e7c1090 2023-07-14T16:16:25.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.945+0000 7f223cd9b700 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b50e7c1090 0x55b50e7c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.946+0000 7f223cd9b700 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b50e7c1090 0x55b50e7c34a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61c3c1256ea40f6 server_cookie=8623d1e91ac73d7d in_seq=0 out_seq=0 2023-07-14T16:16:25.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.946+0000 7f2217fff700 10 client.15708 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:25.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f2217fff700 1 -- 172.21.15.62:0/3644212534 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b50e6f5f30 con 0x55b50e7c1090 2023-07-14T16:16:25.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f2217fff700 10 client.15708 handle_command_reply: tid=0 2023-07-14T16:16:25.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 client.15708 shutdown 2023-07-14T16:16:25.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 -- 172.21.15.62:0/3644212534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 msgr2=0x55b50e7bc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7bc870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2230019e40 tx=0x7f2230004b40 comp rx=0 tx=0).stop 2023-07-14T16:16:25.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 -- 172.21.15.62:0/3644212534 shutdown_connections 2023-07-14T16:16:25.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7aceb0 0x55b50e7bc870 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ab7a0 0x55b50e7b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b50e7c1090 0x55b50e7c34a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 --2- 172.21.15.62:0/3644212534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 -- 172.21.15.62:0/3644212534 >> 172.21.15.62:0/3644212534 conn(0x55b50e7a1070 msgr2=0x55b50e7a5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:25.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f2217fff700 0 client.15708 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:25.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 -- 172.21.15.62:0/3644212534 shutdown_connections 2023-07-14T16:16:25.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.947+0000 7f22529eab80 1 -- 172.21.15.62:0/3644212534 wait complete. 2023-07-14T16:16:25.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.948+0000 7f22529eab80 20 client.15708 trim_cache size 0 max 16384 2023-07-14T16:16:25.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.970+0000 7f22529eab80 1 Processor -- start 2023-07-14T16:16:25.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.970+0000 7f22529eab80 1 -- start start 2023-07-14T16:16:25.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.970+0000 7f22529eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f22529eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ac0f0 0x55b50e7bc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f22529eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7aceb0 0x55b50e7b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f22529eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e7bccd0 con 0x55b50e7ab7a0 2023-07-14T16:16:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f22529eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e7a0bf0 con 0x55b50e7aceb0 2023-07-14T16:16:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f22529eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e7b2fc0 con 0x55b50e7ac0f0 2023-07-14T16:16:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f223cd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f223cd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7c30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48844/0 (socket says 172.21.15.62:48844) 2023-07-14T16:16:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f223cd9b700 1 -- 172.21.15.62:0/1263987263 learned_addr learned my addr 172.21.15.62:0/1263987263 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f223d59c700 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7aceb0 0x55b50e7b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.971+0000 7f2237fff700 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ac0f0 0x55b50e7bc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f223cd9b700 1 -- 172.21.15.62:0/1263987263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ac0f0 msgr2=0x55b50e7bc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:25.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ac0f0 0x55b50e7bc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f223cd9b700 1 -- 172.21.15.62:0/1263987263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7aceb0 msgr2=0x55b50e7b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:25.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7aceb0 0x55b50e7b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f223cd9b700 1 -- 172.21.15.62:0/1263987263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b50e7b4330 con 0x55b50e7ab7a0 2023-07-14T16:16:25.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7c30a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f222c005bf0 tx=0x7f222c099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa999c53eb95040 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f22237fe700 1 -- 172.21.15.62:0/1263987263 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c09dfa0 con 0x55b50e7ab7a0 2023-07-14T16:16:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f22237fe700 1 -- 172.21.15.62:0/1263987263 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f222c0a23c0 con 0x55b50e7ab7a0 2023-07-14T16:16:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.972+0000 7f22237fe700 1 -- 172.21.15.62:0/1263987263 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c09a430 con 0x55b50e7ab7a0 2023-07-14T16:16:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.973+0000 7f22529eab80 1 -- 172.21.15.62:0/1263987263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 msgr2=0x55b50e7c30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.973+0000 7f22529eab80 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7c30a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f222c005bf0 tx=0x7f222c099bd0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.973+0000 7f22529eab80 1 -- 172.21.15.62:0/1263987263 shutdown_connections 2023-07-14T16:16:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.973+0000 7f22529eab80 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7aceb0 0x55b50e7b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.973+0000 7f22529eab80 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7c30a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.973+0000 7f22529eab80 1 --2- 172.21.15.62:0/1263987263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7ac0f0 0x55b50e7bc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.973+0000 7f22529eab80 1 -- 172.21.15.62:0/1263987263 >> 172.21.15.62:0/1263987263 conn(0x55b50e7a1070 msgr2=0x55b50e7bd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.973+0000 7f22529eab80 1 -- 172.21.15.62:0/1263987263 shutdown_connections 2023-07-14T16:16:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.974+0000 7f22529eab80 1 -- 172.21.15.62:0/1263987263 wait complete. 2023-07-14T16:16:25.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.974+0000 7f22529eab80 1 -- start start 2023-07-14T16:16:25.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.974+0000 7f22529eab80 1 Processor -- start 2023-07-14T16:16:25.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f22529eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f22529eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f22529eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7aceb0 0x55b50e7b9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f22529eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e814f40 con 0x55b50e7ab7a0 2023-07-14T16:16:25.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f22529eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e815080 con 0x55b50e7ac0f0 2023-07-14T16:16:25.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f22529eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50e815240 con 0x55b50e7aceb0 2023-07-14T16:16:25.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f22529eab80 10 client.0 _ll_register_callbacks cb 0x7f2230019c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f223cd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f223cd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7bc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48856/0 (socket says 172.21.15.62:48856) 2023-07-14T16:16:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f223cd9b700 1 -- 172.21.15.62:0/1362314803 learned_addr learned my addr 172.21.15.62:0/1362314803 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.975+0000 7f223d59c700 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7aceb0 0x55b50e7b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f2237fff700 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f223cd9b700 1 -- 172.21.15.62:0/1362314803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7aceb0 msgr2=0x55b50e7b9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7aceb0 0x55b50e7b9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f223cd9b700 1 -- 172.21.15.62:0/1362314803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 msgr2=0x55b50e7b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f223cd9b700 1 -- 172.21.15.62:0/1362314803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223819fd90 con 0x55b50e7ab7a0 2023-07-14T16:16:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f223d59c700 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7aceb0 0x55b50e7b9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f223cd9b700 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7bc760 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f222c0ca9d0 tx=0x7f222c010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0dd50b6e06d61b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.976+0000 7f2217fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.977+0000 7f2217fff700 1 -- 172.21.15.62:0/1362314803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c098130 con 0x55b50e7ab7a0 2023-07-14T16:16:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.977+0000 7f22529eab80 1 -- 172.21.15.62:0/1362314803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f222401e040 con 0x55b50e7ab7a0 2023-07-14T16:16:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.977+0000 7f2217fff700 1 -- 172.21.15.62:0/1362314803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f222c09a1d0 con 0x55b50e7ab7a0 2023-07-14T16:16:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.977+0000 7f2217fff700 1 -- 172.21.15.62:0/1362314803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f222c09a390 con 0x55b50e7ab7a0 2023-07-14T16:16:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.977+0000 7f22529eab80 10 client.15714 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.977+0000 7f22529eab80 1 -- 172.21.15.62:0/1362314803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b50e513ce0 con 0x55b50e7ab7a0 2023-07-14T16:16:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.977+0000 7f2217fff700 1 -- 172.21.15.62:0/1362314803 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f222c0b6080 con 0x55b50e7ab7a0 2023-07-14T16:16:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.977+0000 7f2217fff700 1 -- 172.21.15.62:0/1362314803 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f222c00ca10 con 0x55b50e7ab7a0 2023-07-14T16:16:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.978+0000 7f22529eab80 10 client.15714 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.978+0000 7f22529eab80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.978+0000 7f22529eab80 20 client.15714 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.978+0000 7f22529eab80 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b50e816940 0x55b50e818d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.978+0000 7f22529eab80 4 client.15714 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:16:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.978+0000 7f22529eab80 1 -- 172.21.15.62:0/1362314803 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b50e6b80d0 con 0x55b50e816940 2023-07-14T16:16:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.978+0000 7f2237fff700 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b50e816940 0x55b50e818d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:25.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.979+0000 7f2217fff700 10 client.15714 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:25.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.979+0000 7f2237fff700 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b50e816940 0x55b50e818d50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b663815da412ec5 server_cookie=c0fc4b1ee361a8c2 in_seq=0 out_seq=0 2023-07-14T16:16:25.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.980+0000 7f2217fff700 1 -- 172.21.15.62:0/1362314803 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b50e6b80d0 con 0x55b50e816940 2023-07-14T16:16:25.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.980+0000 7f2217fff700 10 client.15714 handle_command_reply: tid=0 2023-07-14T16:16:25.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.980+0000 7f22529eab80 1 client.15714 shutdown 2023-07-14T16:16:25.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 -- 172.21.15.62:0/1362314803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 msgr2=0x55b50e7bc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7bc760 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f222c0ca9d0 tx=0x7f222c010a70 comp rx=0 tx=0).stop 2023-07-14T16:16:25.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 -- 172.21.15.62:0/1362314803 shutdown_connections 2023-07-14T16:16:25.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b50e7aceb0 0x55b50e7b9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b50e7ab7a0 0x55b50e7bc760 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b50e816940 0x55b50e818d50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 --2- 172.21.15.62:0/1362314803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b50e7ac0f0 0x55b50e7b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 -- 172.21.15.62:0/1362314803 >> 172.21.15.62:0/1362314803 conn(0x55b50e7a1070 msgr2=0x55b50e7c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f2217fff700 0 client.15714 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 -- 172.21.15.62:0/1362314803 shutdown_connections 2023-07-14T16:16:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 1 -- 172.21.15.62:0/1362314803 wait complete. 2023-07-14T16:16:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.981+0000 7f22529eab80 20 client.15714 trim_cache size 0 max 16384 2023-07-14T16:16:25.964 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:16:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:16:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "72684e35-10db-4e76-bedb-c5e037384aeb", 2023-07-14T16:16:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:16:25.964 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:16:25.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 -- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22280a11f0 msgr2=0x7f22280a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22280a11f0 0x7f22280a36a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2230009690 tx=0x7f2230009130 comp rx=0 tx=0).stop 2023-07-14T16:16:25.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 -- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22381004b0 msgr2=0x7f22380ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:25.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22381004b0 0x7f22380ffe20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f222c00e9a0 tx=0x7f222c00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 -- 172.21.15.62:0/1242527401 shutdown_connections 2023-07-14T16:16:25.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22381056f0 0x7f22380fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22381004b0 0x7f22380ffe20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22280a11f0 0x7f22280a36a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 --2- 172.21.15.62:0/1242527401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2238102dd0 0x7f22380fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:25.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.982+0000 7f223dd9d700 1 -- 172.21.15.62:0/1242527401 >> 172.21.15.62:0/1242527401 conn(0x7f22380f9ed0 msgr2=0x7f2238101110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:25.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.983+0000 7f223dd9d700 1 -- 172.21.15.62:0/1242527401 shutdown_connections 2023-07-14T16:16:25.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:25.983+0000 7f223dd9d700 1 -- 172.21.15.62:0/1242527401 wait complete. 2023-07-14T16:16:25.986 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:16:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.299+0000 7f37697ff700 1 -- 172.21.15.62:0/4048080345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 msgr2=0x7f3764072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.299+0000 7f37697ff700 1 --2- 172.21.15.62:0/4048080345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 0x7f3764072940 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3754009990 tx=0x7f3754009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.299+0000 7f37697ff700 1 -- 172.21.15.62:0/4048080345 shutdown_connections 2023-07-14T16:16:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.299+0000 7f37697ff700 1 --2- 172.21.15.62:0/4048080345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37641079a0 0x7f3764109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.299+0000 7f37697ff700 1 --2- 172.21.15.62:0/4048080345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 0x7f3764072940 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.299+0000 7f37697ff700 1 --2- 172.21.15.62:0/4048080345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3764072f10 0x7f3764073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.299+0000 7f37697ff700 1 -- 172.21.15.62:0/4048080345 >> 172.21.15.62:0/4048080345 conn(0x7f37640fb130 msgr2=0x7f37640fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.299+0000 7f37697ff700 1 -- 172.21.15.62:0/4048080345 shutdown_connections 2023-07-14T16:16:26.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.300+0000 7f37697ff700 1 -- 172.21.15.62:0/4048080345 wait complete. 2023-07-14T16:16:26.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.300+0000 7f37697ff700 1 Processor -- start 2023-07-14T16:16:26.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.301+0000 7f37697ff700 1 -- start start 2023-07-14T16:16:26.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.301+0000 7f37697ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3764072f10 0x7f3764197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.301+0000 7f37697ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 0x7f3764197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.301+0000 7f37697ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37641079a0 0x7f376419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.301+0000 7f37697ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419f510 con 0x7f37640744f0 2023-07-14T16:16:26.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.301+0000 7f37697ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419f650 con 0x7f37641079a0 2023-07-14T16:16:26.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.301+0000 7f37697ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376419f7e0 con 0x7f3764072f10 2023-07-14T16:16:26.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.301+0000 7f37627fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 0x7f3764197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3764072f10 0x7f3764197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3764072f10 0x7f3764197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:36058/0 (socket says 172.21.15.62:36058) 2023-07-14T16:16:26.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 -- 172.21.15.62:0/3659043896 learned_addr learned my addr 172.21.15.62:0/3659043896 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:26.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f37637fe700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37641079a0 0x7f376419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 -- 172.21.15.62:0/3659043896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37641079a0 msgr2=0x7f376419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37641079a0 0x7f376419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 -- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 msgr2=0x7f3764197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 0x7f3764197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 -- 172.21.15.62:0/3659043896 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37540095f0 con 0x7f3764072f10 2023-07-14T16:16:26.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f37627fc700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 0x7f3764197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:26.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.302+0000 7f3762ffd700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3764072f10 0x7f3764197990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3754000c00 tx=0x7f3754003a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:26.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.303+0000 7f374bfff700 1 -- 172.21.15.62:0/3659043896 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375401d070 con 0x7f3764072f10 2023-07-14T16:16:26.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.303+0000 7f37697ff700 1 -- 172.21.15.62:0/3659043896 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376419fa60 con 0x7f3764072f10 2023-07-14T16:16:26.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.303+0000 7f374bfff700 1 -- 172.21.15.62:0/3659043896 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f375400bd70 con 0x7f3764072f10 2023-07-14T16:16:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.303+0000 7f374bfff700 1 -- 172.21.15.62:0/3659043896 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754022c20 con 0x7f3764072f10 2023-07-14T16:16:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.303+0000 7f37697ff700 1 -- 172.21.15.62:0/3659043896 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376419fee0 con 0x7f3764072f10 2023-07-14T16:16:26.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.307+0000 7f374bfff700 1 -- 172.21.15.62:0/3659043896 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3754003e90 con 0x7f3764072f10 2023-07-14T16:16:26.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.309+0000 7f374bfff700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f374c0a1150 0x7f374c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.309+0000 7f377d185b80 1 Processor -- start 2023-07-14T16:16:26.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.309+0000 7f374bfff700 1 -- 172.21.15.62:0/3659043896 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f37540cdf00 con 0x7f3764072f10 2023-07-14T16:16:26.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f377d185b80 1 -- start start 2023-07-14T16:16:26.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f37627fc700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f374c0a1150 0x7f374c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f377d185b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 0x562bbd4a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f377d185b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f377d185b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f377d185b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd3a5cb0 con 0x562bbd4a0e20 2023-07-14T16:16:26.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f377d185b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd3f8a40 con 0x562bbd4a1c70 2023-07-14T16:16:26.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f377d185b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd3f1ed0 con 0x562bbd4a04d0 2023-07-14T16:16:26.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f37627fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f37627fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48870/0 (socket says 172.21.15.62:48870) 2023-07-14T16:16:26.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.310+0000 7f37627fc700 1 -- 172.21.15.62:0/3655045347 learned_addr learned my addr 172.21.15.62:0/3655045347 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:26.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f37637fe700 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f3762ffd700 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 0x562bbd4a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f37637fe700 1 -- 172.21.15.62:0/3655045347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 msgr2=0x562bbd4a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:26.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f37637fe700 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 0x562bbd4a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f37637fe700 1 -- 172.21.15.62:0/3655045347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 msgr2=0x562bbd4a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:26.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f37637fe700 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f37637fe700 1 -- 172.21.15.62:0/3655045347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bbd21fde0 con 0x562bbd4a1c70 2023-07-14T16:16:26.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f37627fc700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f374c0a1150 0x7f374c0a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3758012e10 tx=0x7f3758012470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:26.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.311+0000 7f37637fe700 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4a20e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f375c003f20 tx=0x7f375c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a809ecfca726700b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:26.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.312+0000 7f37497fa700 1 -- 172.21.15.62:0/3655045347 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c019070 con 0x562bbd4a1c70 2023-07-14T16:16:26.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.312+0000 7f37497fa700 1 -- 172.21.15.62:0/3655045347 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f375c00f040 con 0x562bbd4a1c70 2023-07-14T16:16:26.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.312+0000 7f37497fa700 1 -- 172.21.15.62:0/3655045347 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c0078c0 con 0x562bbd4a1c70 2023-07-14T16:16:26.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 -- 172.21.15.62:0/3655045347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 msgr2=0x562bbd4a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4a20e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f375c003f20 tx=0x7f375c009ba0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 -- 172.21.15.62:0/3655045347 shutdown_connections 2023-07-14T16:16:26.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4a20e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 0x562bbd4a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 --2- 172.21.15.62:0/3655045347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 -- 172.21.15.62:0/3655045347 >> 172.21.15.62:0/3655045347 conn(0x562bbd495da0 msgr2=0x562bbd49b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:26.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 -- 172.21.15.62:0/3655045347 shutdown_connections 2023-07-14T16:16:26.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.313+0000 7f377d185b80 1 -- 172.21.15.62:0/3655045347 wait complete. 2023-07-14T16:16:26.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.314+0000 7f377d185b80 1 -- start start 2023-07-14T16:16:26.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.314+0000 7f377d185b80 1 Processor -- start 2023-07-14T16:16:26.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f377d185b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f377d185b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a0e20 0x562bbd4aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f377d185b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a1c70 0x562bbd4b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f377d185b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd391ac0 con 0x562bbd4a1c70 2023-07-14T16:16:26.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f377d185b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd3917c0 con 0x562bbd4a04d0 2023-07-14T16:16:26.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f377d185b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd275920 con 0x562bbd4a0e20 2023-07-14T16:16:26.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f377d185b80 10 client.0 _ll_register_callbacks cb 0x562bbd38f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:26.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f37637fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a1c70 0x562bbd4b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f3762ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f37637fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a1c70 0x562bbd4b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48880/0 (socket says 172.21.15.62:48880) 2023-07-14T16:16:26.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f37637fe700 1 -- 172.21.15.62:0/1598128529 learned_addr learned my addr 172.21.15.62:0/1598128529 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:26.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.315+0000 7f37627fc700 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a0e20 0x562bbd4aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3762ffd700 1 -- 172.21.15.62:0/1598128529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a0e20 msgr2=0x562bbd4aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:26.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3762ffd700 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a0e20 0x562bbd4aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3762ffd700 1 -- 172.21.15.62:0/1598128529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a1c70 msgr2=0x562bbd4b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:26.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3762ffd700 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a1c70 0x562bbd4b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3762ffd700 1 -- 172.21.15.62:0/1598128529 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c003b50 con 0x562bbd4a04d0 2023-07-14T16:16:26.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3762ffd700 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4aa140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3754097970 tx=0x7f3754009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab8ee43d361bedcc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:26.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3746ff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:26.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3746ff5700 1 -- 172.21.15.62:0/1598128529 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375409add0 con 0x562bbd4a04d0 2023-07-14T16:16:26.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.316+0000 7f3746ff5700 1 -- 172.21.15.62:0/1598128529 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3754022470 con 0x562bbd4a04d0 2023-07-14T16:16:26.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.317+0000 7f377d185b80 1 -- 172.21.15.62:0/1598128529 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bbd34b4f0 con 0x562bbd4a04d0 2023-07-14T16:16:26.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.317+0000 7f3746ff5700 1 -- 172.21.15.62:0/1598128529 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f375409b470 con 0x562bbd4a04d0 2023-07-14T16:16:26.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.317+0000 7f377d185b80 10 client.25499 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:26.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.317+0000 7f377d185b80 1 -- 172.21.15.62:0/1598128529 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562bbd208c60 con 0x562bbd4a04d0 2023-07-14T16:16:26.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.317+0000 7f3746ff5700 1 -- 172.21.15.62:0/1598128529 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562bbd34b4f0 con 0x562bbd4a04d0 2023-07-14T16:16:26.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.318+0000 7f3746ff5700 1 -- 172.21.15.62:0/1598128529 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f37540a33a0 con 0x562bbd4a04d0 2023-07-14T16:16:26.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.318+0000 7f377d185b80 10 client.25499 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:26.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.318+0000 7f377d185b80 10 client.25499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:26.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.318+0000 7f377d185b80 20 client.25499 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:26.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.318+0000 7f377d185b80 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562bbd4b5bf0 0x562bbd4b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.318+0000 7f377d185b80 4 client.25499 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:26.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.318+0000 7f377d185b80 1 -- 172.21.15.62:0/1598128529 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bbd3eaeb0 con 0x562bbd4b5bf0 2023-07-14T16:16:26.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.319+0000 7f37627fc700 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562bbd4b5bf0 0x562bbd4b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.319+0000 7f37627fc700 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562bbd4b5bf0 0x562bbd4b8000 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baafaa615af9f15e server_cookie=107105b39696d2e7 in_seq=0 out_seq=0 2023-07-14T16:16:26.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.319+0000 7f3746ff5700 10 client.25499 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:26.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.320+0000 7f3746ff5700 1 -- 172.21.15.62:0/1598128529 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562bbd3eaeb0 con 0x562bbd4b5bf0 2023-07-14T16:16:26.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.320+0000 7f3746ff5700 10 client.25499 handle_command_reply: tid=0 2023-07-14T16:16:26.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.320+0000 7f377d185b80 1 client.25499 shutdown 2023-07-14T16:16:26.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 -- 172.21.15.62:0/1598128529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 msgr2=0x562bbd4aa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4aa140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3754097970 tx=0x7f3754009310 comp rx=0 tx=0).stop 2023-07-14T16:16:26.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 -- 172.21.15.62:0/1598128529 shutdown_connections 2023-07-14T16:16:26.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a1c70 0x562bbd4b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562bbd4b5bf0 0x562bbd4b8000 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4aa140 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 --2- 172.21.15.62:0/1598128529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a0e20 0x562bbd4aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 -- 172.21.15.62:0/1598128529 >> 172.21.15.62:0/1598128529 conn(0x562bbd495da0 msgr2=0x562bbd49a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:26.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f3746ff5700 0 client.25499 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:26.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 -- 172.21.15.62:0/1598128529 shutdown_connections 2023-07-14T16:16:26.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 1 -- 172.21.15.62:0/1598128529 wait complete. 2023-07-14T16:16:26.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.321+0000 7f377d185b80 20 client.25499 trim_cache size 0 max 16384 2023-07-14T16:16:26.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.354+0000 7f377d185b80 1 Processor -- start 2023-07-14T16:16:26.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.354+0000 7f377d185b80 1 -- start start 2023-07-14T16:16:26.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.354+0000 7f377d185b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 0x562bbd4a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.354+0000 7f377d185b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.354+0000 7f377d185b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.354+0000 7f377d185b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd4a9d10 con 0x562bbd4a0e20 2023-07-14T16:16:26.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.354+0000 7f377d185b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd4a1570 con 0x562bbd4a1c70 2023-07-14T16:16:26.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.354+0000 7f377d185b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd495990 con 0x562bbd4a04d0 2023-07-14T16:16:26.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37627fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37627fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48884/0 (socket says 172.21.15.62:48884) 2023-07-14T16:16:26.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37627fc700 1 -- 172.21.15.62:0/3190398335 learned_addr learned my addr 172.21.15.62:0/3190398335 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:26.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f3762ffd700 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 0x562bbd4a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37627fc700 1 -- 172.21.15.62:0/3190398335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 msgr2=0x562bbd4a8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:26.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37627fc700 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 0x562bbd4a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37637fe700 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37627fc700 1 -- 172.21.15.62:0/3190398335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 msgr2=0x562bbd4b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:26.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37627fc700 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37627fc700 1 -- 172.21.15.62:0/3190398335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bbd246250 con 0x562bbd4a0e20 2023-07-14T16:16:26.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.355+0000 7f37637fe700 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:26.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.356+0000 7f37627fc700 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4b1680 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x562bbd4b8910 tx=0x7f375801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7198fae21e9be818 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:26.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.356+0000 7f3749ffb700 1 -- 172.21.15.62:0/3190398335 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375802c070 con 0x562bbd4a0e20 2023-07-14T16:16:26.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.356+0000 7f3749ffb700 1 -- 172.21.15.62:0/3190398335 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f375800ee30 con 0x562bbd4a0e20 2023-07-14T16:16:26.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.356+0000 7f3749ffb700 1 -- 172.21.15.62:0/3190398335 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758025070 con 0x562bbd4a0e20 2023-07-14T16:16:26.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 -- 172.21.15.62:0/3190398335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 msgr2=0x562bbd4b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4b1680 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x562bbd4b8910 tx=0x7f375801f000 comp rx=0 tx=0).stop 2023-07-14T16:16:26.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 -- 172.21.15.62:0/3190398335 shutdown_connections 2023-07-14T16:16:26.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a1c70 0x562bbd4b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a04d0 0x562bbd4a8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 --2- 172.21.15.62:0/3190398335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4b1680 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 -- 172.21.15.62:0/3190398335 >> 172.21.15.62:0/3190398335 conn(0x562bbd495da0 msgr2=0x562bbd4a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:26.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 -- 172.21.15.62:0/3190398335 shutdown_connections 2023-07-14T16:16:26.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.357+0000 7f377d185b80 1 -- 172.21.15.62:0/3190398335 wait complete. 2023-07-14T16:16:26.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.358+0000 7f377d185b80 1 -- start start 2023-07-14T16:16:26.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.358+0000 7f377d185b80 1 Processor -- start 2023-07-14T16:16:26.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f377d185b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f377d185b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f377d185b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a1c70 0x562bbd4ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f377d185b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd4ae8f0 con 0x562bbd4a0e20 2023-07-14T16:16:26.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f377d185b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd4aea60 con 0x562bbd4a04d0 2023-07-14T16:16:26.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f377d185b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bbd509aa0 con 0x562bbd4a1c70 2023-07-14T16:16:26.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f377d185b80 10 client.0 _ll_register_callbacks cb 0x7f375409b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:26.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f37627fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f37627fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48888/0 (socket says 172.21.15.62:48888) 2023-07-14T16:16:26.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f37627fc700 1 -- 172.21.15.62:0/3982856354 learned_addr learned my addr 172.21.15.62:0/3982856354 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:26.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f3762ffd700 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.360+0000 7f37637fe700 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a1c70 0x562bbd4ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f37627fc700 1 -- 172.21.15.62:0/3982856354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a1c70 msgr2=0x562bbd4ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:26.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f37627fc700 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a1c70 0x562bbd4ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f37627fc700 1 -- 172.21.15.62:0/3982856354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 msgr2=0x562bbd4b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:26.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f37627fc700 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f37627fc700 1 -- 172.21.15.62:0/3982856354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375801a040 con 0x562bbd4a0e20 2023-07-14T16:16:26.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f37637fe700 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a1c70 0x562bbd4ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:26.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f37627fc700 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4ade70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3758028290 tx=0x7f37580282c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afed1cd53053ba8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:26.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f3746ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:26.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f3746ff5700 1 -- 172.21.15.62:0/3982856354 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375802c070 con 0x562bbd4a0e20 2023-07-14T16:16:26.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f377d185b80 1 -- 172.21.15.62:0/3982856354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f375c00f040 con 0x562bbd4a0e20 2023-07-14T16:16:26.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.361+0000 7f3746ff5700 1 -- 172.21.15.62:0/3982856354 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3758028590 con 0x562bbd4a0e20 2023-07-14T16:16:26.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f3746ff5700 1 -- 172.21.15.62:0/3982856354 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3758025040 con 0x562bbd4a0e20 2023-07-14T16:16:26.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f377d185b80 10 client.15732 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:26.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f377d185b80 1 -- 172.21.15.62:0/3982856354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562bbd21e2b0 con 0x562bbd4a0e20 2023-07-14T16:16:26.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f3746ff5700 1 -- 172.21.15.62:0/3982856354 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f375c00f040 con 0x562bbd4a0e20 2023-07-14T16:16:26.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f3746ff5700 1 -- 172.21.15.62:0/3982856354 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3758010930 con 0x562bbd4a0e20 2023-07-14T16:16:26.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f377d185b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:26.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f377d185b80 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:26.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f377d185b80 20 client.15732 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:26.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f377d185b80 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562bbd50b350 0x562bbd50d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:26.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f377d185b80 4 client.15732 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:16:26.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.362+0000 7f377d185b80 1 -- 172.21.15.62:0/3982856354 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562bbd3ad050 con 0x562bbd50b350 2023-07-14T16:16:26.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.363+0000 7f3762ffd700 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562bbd50b350 0x562bbd50d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:26.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f3762ffd700 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562bbd50b350 0x562bbd50d760 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51181ea1c8a71a9c server_cookie=bfbb219f2e4f2877 in_seq=0 out_seq=0 2023-07-14T16:16:26.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f3746ff5700 10 client.15732 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:26.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f3746ff5700 1 -- 172.21.15.62:0/3982856354 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562bbd3ad050 con 0x562bbd50b350 2023-07-14T16:16:26.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f3746ff5700 10 client.15732 handle_command_reply: tid=0 2023-07-14T16:16:26.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f377d185b80 1 client.15732 shutdown 2023-07-14T16:16:26.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f377d185b80 1 -- 172.21.15.62:0/3982856354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 msgr2=0x562bbd4ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f377d185b80 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4ade70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3758028290 tx=0x7f37580282c0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f377d185b80 1 -- 172.21.15.62:0/3982856354 shutdown_connections 2023-07-14T16:16:26.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f377d185b80 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562bbd4a1c70 0x562bbd4ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f377d185b80 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562bbd50b350 0x562bbd50d760 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.364+0000 7f377d185b80 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562bbd4a04d0 0x562bbd4b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.365+0000 7f377d185b80 1 --2- 172.21.15.62:0/3982856354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562bbd4a0e20 0x562bbd4ade70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.365+0000 7f377d185b80 1 -- 172.21.15.62:0/3982856354 >> 172.21.15.62:0/3982856354 conn(0x562bbd495da0 msgr2=0x562bbd4a8110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:26.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.365+0000 7f3746ff5700 0 client.15732 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:26.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.365+0000 7f377d185b80 1 -- 172.21.15.62:0/3982856354 shutdown_connections 2023-07-14T16:16:26.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.365+0000 7f377d185b80 1 -- 172.21.15.62:0/3982856354 wait complete. 2023-07-14T16:16:26.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.365+0000 7f377d185b80 20 client.15732 trim_cache size 0 max 16384 2023-07-14T16:16:26.347 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:16:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:16:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:16:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "72684e35-10db-4e76-bedb-c5e037384aeb": { 2023-07-14T16:16:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:16:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "72684e35-10db-4e76-bedb-c5e037384aeb", 2023-07-14T16:16:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:16:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:16:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:16:26.349 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:16:26.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 -- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f374c0a1150 msgr2=0x7f374c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f374c0a1150 0x7f374c0a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3758012e10 tx=0x7f3758012470 comp rx=0 tx=0).stop 2023-07-14T16:16:26.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 -- 172.21.15.62:0/3659043896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3764072f10 msgr2=0x7f3764197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:26.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3764072f10 0x7f3764197990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3754000c00 tx=0x7f3754003a50 comp rx=0 tx=0).stop 2023-07-14T16:16:26.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 -- 172.21.15.62:0/3659043896 shutdown_connections 2023-07-14T16:16:26.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f374c0a1150 0x7f374c0a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37641079a0 0x7f376419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3764072f10 0x7f3764197990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 --2- 172.21.15.62:0/3659043896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37640744f0 0x7f3764197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:26.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 -- 172.21.15.62:0/3659043896 >> 172.21.15.62:0/3659043896 conn(0x7f37640fb130 msgr2=0x7f37640fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:26.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 -- 172.21.15.62:0/3659043896 shutdown_connections 2023-07-14T16:16:26.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:26.366+0000 7f37697ff700 1 -- 172.21.15.62:0/3659043896 wait complete. 2023-07-14T16:16:26.362 INFO:tasks.cephfs.filesystem:scrub status for tag:72684e35-10db-4e76-bedb-c5e037384aeb - {'path': '/', 'tag': '72684e35-10db-4e76-bedb-c5e037384aeb', 'options': 'recursive,force'} 2023-07-14T16:16:26.624 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:26 smithi062 ceph-mon[95543]: pgmap v381: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:26.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:26 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:16:26.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:26 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:26.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:26 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:16:26.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:26 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:26 smithi081 ceph-mon[127795]: pgmap v381: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:26.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:26 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:16:26.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:26 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:26.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:26 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:16:26.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:26 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:27.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:26 smithi164 ceph-mon[97427]: pgmap v381: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:26 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:16:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:26 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:26 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:16:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:26 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:16:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.894+0000 7f49125ff700 1 -- 172.21.15.62:0/1281359078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c072f70 msgr2=0x7f490c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.894+0000 7f49125ff700 1 --2- 172.21.15.62:0/1281359078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c072f70 0x7f490c0733e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4900009990 tx=0x7f4900009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.894+0000 7f49125ff700 1 -- 172.21.15.62:0/1281359078 shutdown_connections 2023-07-14T16:16:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.894+0000 7f49125ff700 1 --2- 172.21.15.62:0/1281359078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f490c1079c0 0x7f490c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.894+0000 7f49125ff700 1 --2- 172.21.15.62:0/1281359078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f490c074550 0x7f490c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.894+0000 7f49125ff700 1 --2- 172.21.15.62:0/1281359078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c072f70 0x7f490c0733e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.894+0000 7f49125ff700 1 -- 172.21.15.62:0/1281359078 >> 172.21.15.62:0/1281359078 conn(0x7f490c0fb0e0 msgr2=0x7f490c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.895+0000 7f49125ff700 1 -- 172.21.15.62:0/1281359078 shutdown_connections 2023-07-14T16:16:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.895+0000 7f49125ff700 1 -- 172.21.15.62:0/1281359078 wait complete. 2023-07-14T16:16:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.896+0000 7f49125ff700 1 Processor -- start 2023-07-14T16:16:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.896+0000 7f49125ff700 1 -- start start 2023-07-14T16:16:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.897+0000 7f49125ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c072f70 0x7f490c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.897+0000 7f49125ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f490c074550 0x7f490c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.897+0000 7f49125ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f490c1079c0 0x7f490c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.897+0000 7f49125ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c199530 con 0x7f490c072f70 2023-07-14T16:16:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.897+0000 7f49125ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c1996a0 con 0x7f490c074550 2023-07-14T16:16:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.897+0000 7f49125ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c199860 con 0x7f490c1079c0 2023-07-14T16:16:28.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:28 smithi062 ceph-mon[95543]: pgmap v382: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:28.877 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:16:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f4910b9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f490c1079c0 0x7f490c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f4910b9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f490c1079c0 0x7f490c198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:36102/0 (socket says 172.21.15.62:36102) 2023-07-14T16:16:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f4910b9c700 1 -- 172.21.15.62:0/1386503232 learned_addr learned my addr 172.21.15.62:0/1386503232 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f490b7fe700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f490c074550 0x7f490c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f490bfff700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c072f70 0x7f490c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f4910b9c700 1 -- 172.21.15.62:0/1386503232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f490c074550 msgr2=0x7f490c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f4910b9c700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f490c074550 0x7f490c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f4910b9c700 1 -- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c072f70 msgr2=0x7f490c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f4910b9c700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c072f70 0x7f490c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.898+0000 7f4910b9c700 1 -- 172.21.15.62:0/1386503232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000095f0 con 0x7f490c1079c0 2023-07-14T16:16:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.899+0000 7f4910b9c700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f490c1079c0 0x7f490c198ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f490400ef30 tx=0x7f490400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.899+0000 7f49097fa700 1 -- 172.21.15.62:0/1386503232 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904018070 con 0x7f490c1079c0 2023-07-14T16:16:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.899+0000 7f49097fa700 1 -- 172.21.15.62:0/1386503232 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f490400f040 con 0x7f490c1079c0 2023-07-14T16:16:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.899+0000 7f49125ff700 1 -- 172.21.15.62:0/1386503232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c199ae0 con 0x7f490c1079c0 2023-07-14T16:16:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.899+0000 7f49097fa700 1 -- 172.21.15.62:0/1386503232 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904003d00 con 0x7f490c1079c0 2023-07-14T16:16:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.899+0000 7f49125ff700 1 -- 172.21.15.62:0/1386503232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c1a0300 con 0x7f490c1079c0 2023-07-14T16:16:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.905+0000 7f49097fa700 1 -- 172.21.15.62:0/1386503232 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f490400caa0 con 0x7f490c1079c0 2023-07-14T16:16:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.906+0000 7f4925f22b80 1 Processor -- start 2023-07-14T16:16:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.906+0000 7f4925f22b80 1 -- start start 2023-07-14T16:16:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f4925f22b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b0c1c50 con 0x55e68b0186b0 2023-07-14T16:16:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f4925f22b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b1149e0 con 0x55e68b1bc840 2023-07-14T16:16:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f4925f22b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b10de70 con 0x55e68b1bd6e0 2023-07-14T16:16:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f490bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f490bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1bd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48410/0 (socket says 172.21.15.62:48410) 2023-07-14T16:16:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f490bfff700 1 -- 172.21.15.62:0/2465738130 learned_addr learned my addr 172.21.15.62:0/2465738130 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f49097fa700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48f40a1150 0x7f48f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.907+0000 7f490b7fe700 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f49097fa700 1 -- 172.21.15.62:0/1386503232 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f49040cc5c0 con 0x7f490c1079c0 2023-07-14T16:16:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f490b7fe700 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f490bfff700 1 -- 172.21.15.62:0/2465738130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 msgr2=0x55e68b1bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f490bfff700 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1bdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f490bfff700 1 -- 172.21.15.62:0/2465738130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 msgr2=0x55e68b1bc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f490bfff700 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f490bfff700 1 -- 172.21.15.62:0/2465738130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e68af3bd80 con 0x55e68b1bc840 2023-07-14T16:16:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f4910b9c700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48f40a1150 0x7f48f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.908+0000 7f490bfff700 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1bd1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00dca0 tx=0x7f48fc00f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f0357d38cf6861 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.909+0000 7f48f27fc700 1 -- 172.21.15.62:0/2465738130 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00fc90 con 0x55e68b1bc840 2023-07-14T16:16:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.909+0000 7f48f27fc700 1 -- 172.21.15.62:0/2465738130 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48fc00fdf0 con 0x55e68b1bc840 2023-07-14T16:16:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.909+0000 7f48f27fc700 1 -- 172.21.15.62:0/2465738130 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc005680 con 0x55e68b1bc840 2023-07-14T16:16:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 -- 172.21.15.62:0/2465738130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 msgr2=0x55e68b1bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4910b9c700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48f40a1150 0x7f48f40a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4904009c10 tx=0x7f49040092e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1bd1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00dca0 tx=0x7f48fc00f360 comp rx=0 tx=0).stop 2023-07-14T16:16:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 -- 172.21.15.62:0/2465738130 shutdown_connections 2023-07-14T16:16:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 --2- 172.21.15.62:0/2465738130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1bd1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 -- 172.21.15.62:0/2465738130 >> 172.21.15.62:0/2465738130 conn(0x55e68b1b1be0 msgr2=0x55e68b1b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 -- 172.21.15.62:0/2465738130 shutdown_connections 2023-07-14T16:16:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.910+0000 7f4925f22b80 1 -- 172.21.15.62:0/2465738130 wait complete. 2023-07-14T16:16:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.911+0000 7f4925f22b80 1 -- start start 2023-07-14T16:16:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.911+0000 7f4925f22b80 1 Processor -- start 2023-07-14T16:16:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.912+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b0186b0 0x55e68b1c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.912+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.912+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bd6e0 0x55e68b1ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.912+0000 7f4925f22b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b0ada60 con 0x55e68b1bd6e0 2023-07-14T16:16:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.912+0000 7f4925f22b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b0ad760 con 0x55e68b1bc840 2023-07-14T16:16:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.912+0000 7f4925f22b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68af918c0 con 0x55e68b0186b0 2023-07-14T16:16:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.912+0000 7f4925f22b80 10 client.0 _ll_register_callbacks cb 0x55e68b0ab700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.912+0000 7f490b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b0186b0 0x55e68b1c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b0186b0 0x55e68b1c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:36108/0 (socket says 172.21.15.62:36108) 2023-07-14T16:16:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490b7fe700 1 -- 172.21.15.62:0/3556271753 learned_addr learned my addr 172.21.15.62:0/3556271753 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490b7fe700 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bd6e0 0x55e68b1ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490bfff700 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490bfff700 1 -- 172.21.15.62:0/3556271753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b0186b0 msgr2=0x55e68b1c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490bfff700 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b0186b0 0x55e68b1c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490bfff700 1 -- 172.21.15.62:0/3556271753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bd6e0 msgr2=0x55e68b1ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490bfff700 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bd6e0 0x55e68b1ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.913+0000 7f490bfff700 1 -- 172.21.15.62:0/3556271753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc00d900 con 0x55e68b1bc840 2023-07-14T16:16:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.914+0000 7f490bfff700 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1c5d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48fc000c00 tx=0x7f48fc00fcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5e545ab872451 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.914+0000 7f48eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.914+0000 7f48eb7fe700 1 -- 172.21.15.62:0/3556271753 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc021070 con 0x55e68b1bc840 2023-07-14T16:16:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.914+0000 7f48eb7fe700 1 -- 172.21.15.62:0/3556271753 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f48fc005610 con 0x55e68b1bc840 2023-07-14T16:16:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.914+0000 7f4925f22b80 1 -- 172.21.15.62:0/3556271753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e68b067490 con 0x55e68b1bc840 2023-07-14T16:16:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.915+0000 7f48eb7fe700 1 -- 172.21.15.62:0/3556271753 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48fc004040 con 0x55e68b1bc840 2023-07-14T16:16:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.915+0000 7f4925f22b80 10 client.25514 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.915+0000 7f4925f22b80 1 -- 172.21.15.62:0/3556271753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e68af24c00 con 0x55e68b1bc840 2023-07-14T16:16:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.915+0000 7f48eb7fe700 1 -- 172.21.15.62:0/3556271753 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e68b067490 con 0x55e68b1bc840 2023-07-14T16:16:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.916+0000 7f48eb7fe700 1 -- 172.21.15.62:0/3556271753 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f48fc02d070 con 0x55e68b1bc840 2023-07-14T16:16:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.916+0000 7f4925f22b80 10 client.25514 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.916+0000 7f4925f22b80 10 client.25514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.916+0000 7f4925f22b80 20 client.25514 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.916+0000 7f4925f22b80 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e68b1d1480 0x55e68b1d3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.916+0000 7f4925f22b80 4 client.25514 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.916+0000 7f4925f22b80 1 -- 172.21.15.62:0/3556271753 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e68b106e50 con 0x55e68b1d1480 2023-07-14T16:16:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.916+0000 7f490b7fe700 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e68b1d1480 0x55e68b1d3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.917+0000 7f490b7fe700 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e68b1d1480 0x55e68b1d3890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71a5559274ce581 server_cookie=6cf9574297bbf8bf in_seq=0 out_seq=0 2023-07-14T16:16:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.917+0000 7f48eb7fe700 10 client.25514 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.918+0000 7f48eb7fe700 1 -- 172.21.15.62:0/3556271753 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e68b106e50 con 0x55e68b1d1480 2023-07-14T16:16:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.918+0000 7f48eb7fe700 10 client.25514 handle_command_reply: tid=0 2023-07-14T16:16:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.918+0000 7f4925f22b80 1 client.25514 shutdown 2023-07-14T16:16:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.918+0000 7f4925f22b80 1 -- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 msgr2=0x55e68b1c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.918+0000 7f4925f22b80 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1c5d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48fc000c00 tx=0x7f48fc00fcc0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 1 -- 172.21.15.62:0/3556271753 shutdown_connections 2023-07-14T16:16:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bd6e0 0x55e68b1ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b0186b0 0x55e68b1c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e68b1d1480 0x55e68b1d3890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 1 --2- 172.21.15.62:0/3556271753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bc840 0x55e68b1c5d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 1 -- 172.21.15.62:0/3556271753 >> 172.21.15.62:0/3556271753 conn(0x55e68b1b1be0 msgr2=0x55e68b1b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f48eb7fe700 0 client.25514 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 1 -- 172.21.15.62:0/3556271753 shutdown_connections 2023-07-14T16:16:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 1 -- 172.21.15.62:0/3556271753 wait complete. 2023-07-14T16:16:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.919+0000 7f4925f22b80 20 client.25514 trim_cache size 0 max 16384 2023-07-14T16:16:28.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:28 smithi081 ceph-mon[127795]: pgmap v382: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:28.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:16:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.949+0000 7f4925f22b80 1 Processor -- start 2023-07-14T16:16:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.949+0000 7f4925f22b80 1 -- start start 2023-07-14T16:16:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.949+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.949+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bc840 0x55e68b1ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bd6e0 0x55e68b1bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f4925f22b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b1be070 con 0x55e68b0186b0 2023-07-14T16:16:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f4925f22b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b1aebe0 con 0x55e68b1bd6e0 2023-07-14T16:16:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f4925f22b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b1bcd00 con 0x55e68b1bc840 2023-07-14T16:16:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48938/0 (socket says 172.21.15.62:48938) 2023-07-14T16:16:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490b7fe700 1 -- 172.21.15.62:0/440295763 learned_addr learned my addr 172.21.15.62:0/440295763 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490bfff700 1 --2- 172.21.15.62:0/440295763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bc840 0x55e68b1ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490b7fe700 1 -- 172.21.15.62:0/440295763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bc840 msgr2=0x55e68b1ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490b7fe700 1 --2- 172.21.15.62:0/440295763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bc840 0x55e68b1ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490b7fe700 1 -- 172.21.15.62:0/440295763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bd6e0 msgr2=0x55e68b1bdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490b7fe700 1 --2- 172.21.15.62:0/440295763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bd6e0 0x55e68b1bdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.950+0000 7f490b7fe700 1 -- 172.21.15.62:0/440295763 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e68af621f0 con 0x55e68b0186b0 2023-07-14T16:16:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.951+0000 7f490b7fe700 1 --2- 172.21.15.62:0/440295763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc2c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4900003f60 tx=0x7f4900005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe95725e10bd4218 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.951+0000 7f48f2ffd700 1 -- 172.21.15.62:0/440295763 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900003c20 con 0x55e68b0186b0 2023-07-14T16:16:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.951+0000 7f48f2ffd700 1 -- 172.21.15.62:0/440295763 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f490002f070 con 0x55e68b0186b0 2023-07-14T16:16:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.951+0000 7f48f2ffd700 1 -- 172.21.15.62:0/440295763 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490000f6c0 con 0x55e68b0186b0 2023-07-14T16:16:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.952+0000 7f4925f22b80 1 -- 172.21.15.62:0/440295763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 msgr2=0x55e68b1bc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.952+0000 7f4925f22b80 1 --2- 172.21.15.62:0/440295763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc2c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4900003f60 tx=0x7f4900005510 comp rx=0 tx=0).stop 2023-07-14T16:16:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.952+0000 7f4925f22b80 1 -- 172.21.15.62:0/440295763 shutdown_connections 2023-07-14T16:16:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.952+0000 7f4925f22b80 1 --2- 172.21.15.62:0/440295763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b1bd6e0 0x55e68b1bdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.952+0000 7f4925f22b80 1 --2- 172.21.15.62:0/440295763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b0186b0 0x55e68b1bc2c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.952+0000 7f4925f22b80 1 --2- 172.21.15.62:0/440295763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bc840 0x55e68b1ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.952+0000 7f4925f22b80 1 -- 172.21.15.62:0/440295763 >> 172.21.15.62:0/440295763 conn(0x55e68b1b1be0 msgr2=0x55e68b1d81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.953+0000 7f4925f22b80 1 -- 172.21.15.62:0/440295763 shutdown_connections 2023-07-14T16:16:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.953+0000 7f4925f22b80 1 -- 172.21.15.62:0/440295763 wait complete. 2023-07-14T16:16:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.953+0000 7f4925f22b80 1 -- start start 2023-07-14T16:16:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.953+0000 7f4925f22b80 1 Processor -- start 2023-07-14T16:16:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.954+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b0186b0 0x55e68b1c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.954+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bc840 0x55e68b1c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.954+0000 7f4925f22b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.954+0000 7f4925f22b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b1cc600 con 0x55e68b1bc840 2023-07-14T16:16:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.954+0000 7f4925f22b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b1cc770 con 0x55e68b0186b0 2023-07-14T16:16:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.954+0000 7f4925f22b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b1cc930 con 0x55e68b1bd6e0 2023-07-14T16:16:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.954+0000 7f4925f22b80 10 client.0 _ll_register_callbacks cb 0x7f48fc00db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:36116/0 (socket says 172.21.15.62:36116) 2023-07-14T16:16:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 -- 172.21.15.62:0/181884097 learned_addr learned my addr 172.21.15.62:0/181884097 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b0186b0 0x55e68b1c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490bfff700 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bc840 0x55e68b1c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 -- 172.21.15.62:0/181884097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 msgr2=0x55e68b1cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 -- 172.21.15.62:0/181884097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bc840 msgr2=0x55e68b1c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bc840 0x55e68b1c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.955+0000 7f490b7fe700 1 -- 172.21.15.62:0/181884097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c10aa60 con 0x55e68b0186b0 2023-07-14T16:16:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.956+0000 7f490b7fe700 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b0186b0 0x55e68b1c8b60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4900000c00 tx=0x7f4900009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaf962b8899eb5fe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.956+0000 7f48eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.956+0000 7f48eb7fe700 1 -- 172.21.15.62:0/181884097 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900003b20 con 0x55e68b0186b0 2023-07-14T16:16:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.956+0000 7f48eb7fe700 1 -- 172.21.15.62:0/181884097 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4900003c80 con 0x55e68b0186b0 2023-07-14T16:16:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.956+0000 7f4925f22b80 1 -- 172.21.15.62:0/181884097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48fc00fdf0 con 0x55e68b0186b0 2023-07-14T16:16:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.956+0000 7f48eb7fe700 1 -- 172.21.15.62:0/181884097 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f490000b710 con 0x55e68b0186b0 2023-07-14T16:16:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.956+0000 7f4925f22b80 10 client.25517 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.956+0000 7f4925f22b80 1 -- 172.21.15.62:0/181884097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e68af3a250 con 0x55e68b0186b0 2023-07-14T16:16:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.957+0000 7f48eb7fe700 1 -- 172.21.15.62:0/181884097 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48fc00fdf0 con 0x55e68b0186b0 2023-07-14T16:16:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.957+0000 7f48eb7fe700 1 -- 172.21.15.62:0/181884097 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f490002a450 con 0x55e68b0186b0 2023-07-14T16:16:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.958+0000 7f4925f22b80 10 client.25517 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.958+0000 7f4925f22b80 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.958+0000 7f4925f22b80 20 client.25517 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.958+0000 7f4925f22b80 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e68b22e620 0x55e68b230a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.958+0000 7f4925f22b80 4 client.25517 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:16:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.958+0000 7f4925f22b80 1 -- 172.21.15.62:0/181884097 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e68b0c8ff0 con 0x55e68b22e620 2023-07-14T16:16:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.958+0000 7f490bfff700 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e68b22e620 0x55e68b230a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.959+0000 7f490bfff700 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e68b22e620 0x55e68b230a30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c29875e6447e1f3 server_cookie=faa7b68c46e87d5c in_seq=0 out_seq=0 2023-07-14T16:16:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.959+0000 7f48eb7fe700 10 client.25517 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f48eb7fe700 1 -- 172.21.15.62:0/181884097 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e68b0c8ff0 con 0x55e68b22e620 2023-07-14T16:16:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f48eb7fe700 10 client.25517 handle_command_reply: tid=0 2023-07-14T16:16:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 client.25517 shutdown 2023-07-14T16:16:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 -- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b0186b0 msgr2=0x55e68b1c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b0186b0 0x55e68b1c8b60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4900000c00 tx=0x7f4900009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 -- 172.21.15.62:0/181884097 shutdown_connections 2023-07-14T16:16:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e68b1bd6e0 0x55e68b1cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e68b22e620 0x55e68b230a30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e68b0186b0 0x55e68b1c8b60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 --2- 172.21.15.62:0/181884097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e68b1bc840 0x55e68b1c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f48eb7fe700 0 client.25517 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 -- 172.21.15.62:0/181884097 >> 172.21.15.62:0/181884097 conn(0x55e68b1b1be0 msgr2=0x55e68b1c0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 -- 172.21.15.62:0/181884097 shutdown_connections 2023-07-14T16:16:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.963+0000 7f4925f22b80 1 -- 172.21.15.62:0/181884097 wait complete. 2023-07-14T16:16:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.964+0000 7f4925f22b80 20 client.25517 trim_cache size 0 max 16384 2023-07-14T16:16:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:16:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:16:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:16:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:16:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 -- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48f40a1150 msgr2=0x7f48f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48f40a1150 0x7f48f40a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4904009c10 tx=0x7f49040092e0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 -- 172.21.15.62:0/1386503232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f490c1079c0 msgr2=0x7f490c198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f490c1079c0 0x7f490c198ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f490400ef30 tx=0x7f490400c450 comp rx=0 tx=0).stop 2023-07-14T16:16:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 -- 172.21.15.62:0/1386503232 shutdown_connections 2023-07-14T16:16:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f490c1079c0 0x7f490c198ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c072f70 0x7f490c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48f40a1150 0x7f48f40a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 --2- 172.21.15.62:0/1386503232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f490c074550 0x7f490c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 -- 172.21.15.62:0/1386503232 >> 172.21.15.62:0/1386503232 conn(0x7f490c0fb0e0 msgr2=0x7f490c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 -- 172.21.15.62:0/1386503232 shutdown_connections 2023-07-14T16:16:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:28.965+0000 7f49125ff700 1 -- 172.21.15.62:0/1386503232 wait complete. 2023-07-14T16:16:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:28 smithi164 ceph-mon[97427]: pgmap v382: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T16:16:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:16:30.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:30 smithi062 ceph-mon[95543]: pgmap v383: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-14T16:16:30.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:30 smithi081 ceph-mon[127795]: pgmap v383: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-14T16:16:31.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:30 smithi164 ceph-mon[97427]: pgmap v383: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-14T16:16:32.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:32 smithi062 ceph-mon[95543]: pgmap v384: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2023-07-14T16:16:32.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:32 smithi081 ceph-mon[127795]: pgmap v384: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2023-07-14T16:16:33.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:32 smithi164 ceph-mon[97427]: pgmap v384: 97 pgs: 97 active+clean; 657 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2023-07-14T16:16:34.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:34 smithi062 ceph-mon[95543]: pgmap v385: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T16:16:34.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:34 smithi081 ceph-mon[127795]: pgmap v385: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T16:16:35.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:34 smithi164 ceph-mon[97427]: pgmap v385: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-14T16:16:36.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:36 smithi062 ceph-mon[95543]: pgmap v386: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:16:36.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:36 smithi081 ceph-mon[127795]: pgmap v386: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:16:37.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:36 smithi164 ceph-mon[97427]: pgmap v386: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:16:38.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:38 smithi062 ceph-mon[95543]: pgmap v387: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:38.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:38 smithi081 ceph-mon[127795]: pgmap v387: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:39.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:38 smithi164 ceph-mon[97427]: pgmap v387: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:40.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:40 smithi062 ceph-mon[95543]: pgmap v388: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:16:40.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:40 smithi081 ceph-mon[127795]: pgmap v388: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:16:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:40 smithi164 ceph-mon[97427]: pgmap v388: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T16:16:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.208+0000 7f1b40b9d700 1 -- 172.21.15.62:0/323006205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c105520 msgr2=0x7f1b3c1079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.208+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/323006205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c105520 0x7f1b3c1079d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400b0a0 tx=0x7f1b3400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.209+0000 7f1b40b9d700 1 -- 172.21.15.62:0/323006205 shutdown_connections 2023-07-14T16:16:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.209+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/323006205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c105520 0x7f1b3c1079d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.209+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/323006205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3c068640 0x7f1b3c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.209+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/323006205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b3c102a10 0x7f1b3c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.209+0000 7f1b40b9d700 1 -- 172.21.15.62:0/323006205 >> 172.21.15.62:0/323006205 conn(0x7f1b3c0f9e40 msgr2=0x7f1b3c0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.209+0000 7f1b40b9d700 1 -- 172.21.15.62:0/323006205 shutdown_connections 2023-07-14T16:16:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.209+0000 7f1b40b9d700 1 -- 172.21.15.62:0/323006205 wait complete. 2023-07-14T16:16:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.210+0000 7f1b40b9d700 1 Processor -- start 2023-07-14T16:16:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.211+0000 7f1b40b9d700 1 -- start start 2023-07-14T16:16:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.211+0000 7f1b40b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c068640 0x7f1b3c0ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.211+0000 7f1b40b9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b3c102a10 0x7f1b3c0fe470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.211+0000 7f1b40b9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3c105520 0x7f1b3c0fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.211+0000 7f1b40b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c0ff150 con 0x7f1b3c068640 2023-07-14T16:16:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.211+0000 7f1b40b9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c0ff2c0 con 0x7f1b3c102a10 2023-07-14T16:16:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.211+0000 7f1b40b9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c19f740 con 0x7f1b3c105520 2023-07-14T16:16:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b3c102a10 0x7f1b3c0fe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c068640 0x7f1b3c0ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b3c102a10 0x7f1b3c0fe470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:37106/0 (socket says 172.21.15.62:37106) 2023-07-14T16:16:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3affd700 1 -- 172.21.15.62:0/2193752142 learned_addr learned my addr 172.21.15.62:0/2193752142 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3bfff700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3c105520 0x7f1b3c0fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3affd700 1 -- 172.21.15.62:0/2193752142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3c105520 msgr2=0x7f1b3c0fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3affd700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3c105520 0x7f1b3c0fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3affd700 1 -- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c068640 msgr2=0x7f1b3c0ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3affd700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c068640 0x7f1b3c0ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3affd700 1 -- 172.21.15.62:0/2193752142 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b34009d00 con 0x7f1b3c102a10 2023-07-14T16:16:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.212+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c068640 0x7f1b3c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.213+0000 7f1b3affd700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b3c102a10 0x7f1b3c0fe470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000d950 tx=0x7f1b3000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.213+0000 7f1b38ff9700 1 -- 172.21.15.62:0/2193752142 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000f890 con 0x7f1b3c102a10 2023-07-14T16:16:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.213+0000 7f1b40b9d700 1 -- 172.21.15.62:0/2193752142 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3c19fa20 con 0x7f1b3c102a10 2023-07-14T16:16:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.213+0000 7f1b38ff9700 1 -- 172.21.15.62:0/2193752142 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b3000fe80 con 0x7f1b3c102a10 2023-07-14T16:16:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.213+0000 7f1b40b9d700 1 -- 172.21.15.62:0/2193752142 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c19ff50 con 0x7f1b3c102a10 2023-07-14T16:16:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.214+0000 7f1b38ff9700 1 -- 172.21.15.62:0/2193752142 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000e3b0 con 0x7f1b3c102a10 2023-07-14T16:16:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.219+0000 7f1b38ff9700 1 -- 172.21.15.62:0/2193752142 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1b3000f9f0 con 0x7f1b3c102a10 2023-07-14T16:16:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.220+0000 7f1b55681b80 1 Processor -- start 2023-07-14T16:16:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.221+0000 7f1b55681b80 1 -- start start 2023-07-14T16:16:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.221+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cb5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.221+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cb62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.221+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cb6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.221+0000 7f1b55681b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1bbac50 con 0x55ebc1b116b0 2023-07-14T16:16:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.221+0000 7f1b55681b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1c0d9e0 con 0x55ebc1cb67e0 2023-07-14T16:16:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.221+0000 7f1b55681b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1c06e70 con 0x55ebc1cb5a90 2023-07-14T16:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cb5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cb6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cb5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51820/0 (socket says 172.21.15.62:51820) 2023-07-14T16:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3bfff700 1 -- 172.21.15.62:0/2829840573 learned_addr learned my addr 172.21.15.62:0/2829840573 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/2829840573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cb62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3affd700 1 -- 172.21.15.62:0/2829840573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 msgr2=0x55ebc1cb62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3affd700 1 --2- 172.21.15.62:0/2829840573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cb62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3affd700 1 -- 172.21.15.62:0/2829840573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 msgr2=0x55ebc1cb5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3affd700 1 --2- 172.21.15.62:0/2829840573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cb5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b3affd700 1 -- 172.21.15.62:0/2829840573 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebc1a34d80 con 0x55ebc1cb67e0 2023-07-14T16:16:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.222+0000 7f1b38ff9700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1b240a11d0 0x7f1b240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.223+0000 7f1b3affd700 1 --2- 172.21.15.62:0/2829840573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cb6c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b300ca380 tx=0x7f1b300d2a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbdd91a72f98a568 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.223+0000 7f1b38ff9700 1 -- 172.21.15.62:0/2193752142 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1b300cc930 con 0x7f1b3c102a10 2023-07-14T16:16:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.223+0000 7f1b3bfff700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1b240a11d0 0x7f1b240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.223+0000 7f1b21ffb700 1 -- 172.21.15.62:0/2829840573 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b300d26d0 con 0x55ebc1cb67e0 2023-07-14T16:16:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.223+0000 7f1b21ffb700 1 -- 172.21.15.62:0/2829840573 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b30096030 con 0x55ebc1cb67e0 2023-07-14T16:16:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.223+0000 7f1b21ffb700 1 -- 172.21.15.62:0/2829840573 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000fca0 con 0x55ebc1cb67e0 2023-07-14T16:16:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.224+0000 7f1b3bfff700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1b240a11d0 0x7f1b240a3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55ebc1cb52b0 tx=0x7f1b34006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.224+0000 7f1b55681b80 1 -- 172.21.15.62:0/2829840573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb67e0 msgr2=0x55ebc1cb6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.224+0000 7f1b55681b80 1 --2- 172.21.15.62:0/2829840573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cb6c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b300ca380 tx=0x7f1b300d2a60 comp rx=0 tx=0).stop 2023-07-14T16:16:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.224+0000 7f1b55681b80 1 -- 172.21.15.62:0/2829840573 shutdown_connections 2023-07-14T16:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.224+0000 7f1b55681b80 1 --2- 172.21.15.62:0/2829840573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cb6c50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.224+0000 7f1b55681b80 1 --2- 172.21.15.62:0/2829840573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cb5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.224+0000 7f1b55681b80 1 --2- 172.21.15.62:0/2829840573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cb62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.224+0000 7f1b55681b80 1 -- 172.21.15.62:0/2829840573 >> 172.21.15.62:0/2829840573 conn(0x55ebc1caad40 msgr2=0x55ebc1cafe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.225+0000 7f1b55681b80 1 -- 172.21.15.62:0/2829840573 shutdown_connections 2023-07-14T16:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.225+0000 7f1b55681b80 1 -- 172.21.15.62:0/2829840573 wait complete. 2023-07-14T16:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.225+0000 7f1b55681b80 1 -- start start 2023-07-14T16:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.226+0000 7f1b55681b80 1 Processor -- start 2023-07-14T16:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.226+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1b116b0 0x55ebc1cbeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.226+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cbf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.226+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.226+0000 7f1b55681b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1ba6a60 con 0x55ebc1cb67e0 2023-07-14T16:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.226+0000 7f1b55681b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1ba6760 con 0x55ebc1b116b0 2023-07-14T16:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.226+0000 7f1b55681b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1a8a8c0 con 0x55ebc1cb5a90 2023-07-14T16:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.227+0000 7f1b55681b80 10 client.0 _ll_register_callbacks cb 0x55ebc1ba4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.227+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.227+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51832/0 (socket says 172.21.15.62:51832) 2023-07-14T16:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.227+0000 7f1b3bfff700 1 -- 172.21.15.62:0/3935386824 learned_addr learned my addr 172.21.15.62:0/3935386824 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.227+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cbf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.227+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1b116b0 0x55ebc1cbeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b3affd700 1 -- 172.21.15.62:0/3935386824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1b116b0 msgr2=0x55ebc1cbeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1b116b0 0x55ebc1cbeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b3affd700 1 -- 172.21.15.62:0/3935386824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1cb67e0 msgr2=0x55ebc1cc6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b3affd700 1 -- 172.21.15.62:0/3935386824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c19ff50 con 0x55ebc1cb5a90 2023-07-14T16:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cbf040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1b300997c0 tx=0x7f1b300d2380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=805b15e3b68a0d77 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b1affd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b1affd700 1 -- 172.21.15.62:0/3935386824 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30099ac0 con 0x55ebc1cb5a90 2023-07-14T16:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b1affd700 1 -- 172.21.15.62:0/3935386824 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1b3000fca0 con 0x55ebc1cb5a90 2023-07-14T16:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b55681b80 1 -- 172.21.15.62:0/3935386824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebc1b60490 con 0x55ebc1cb5a90 2023-07-14T16:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.228+0000 7f1b1affd700 1 -- 172.21.15.62:0/3935386824 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b30099cf0 con 0x55ebc1cb5a90 2023-07-14T16:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.229+0000 7f1b55681b80 10 client.25468 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.229+0000 7f1b55681b80 1 -- 172.21.15.62:0/3935386824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ebc1a1dc00 con 0x55ebc1cb5a90 2023-07-14T16:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.229+0000 7f1b1affd700 1 -- 172.21.15.62:0/3935386824 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ebc1b60490 con 0x55ebc1cb5a90 2023-07-14T16:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.230+0000 7f1b1affd700 1 -- 172.21.15.62:0/3935386824 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1b30099f50 con 0x55ebc1cb5a90 2023-07-14T16:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.230+0000 7f1b55681b80 10 client.25468 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.230+0000 7f1b55681b80 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.230+0000 7f1b55681b80 20 client.25468 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.230+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1cca340 0x55ebc1ccc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.230+0000 7f1b55681b80 4 client.25468 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.230+0000 7f1b55681b80 1 -- 172.21.15.62:0/3935386824 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebc1bffe50 con 0x55ebc1cca340 2023-07-14T16:16:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.231+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1cca340 0x55ebc1ccc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.231+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1cca340 0x55ebc1ccc750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d593e51a6b0034ba server_cookie=669750ef16296390 in_seq=0 out_seq=0 2023-07-14T16:16:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.231+0000 7f1b1affd700 10 client.25468 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.232+0000 7f1b1affd700 1 -- 172.21.15.62:0/3935386824 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ebc1bffe50 con 0x55ebc1cca340 2023-07-14T16:16:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.232+0000 7f1b1affd700 10 client.25468 handle_command_reply: tid=0 2023-07-14T16:16:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.232+0000 7f1b55681b80 1 client.25468 shutdown 2023-07-14T16:16:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.232+0000 7f1b55681b80 1 -- 172.21.15.62:0/3935386824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 msgr2=0x55ebc1cbf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cbf040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1b300997c0 tx=0x7f1b300d2380 comp rx=0 tx=0).stop 2023-07-14T16:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 -- 172.21.15.62:0/3935386824 shutdown_connections 2023-07-14T16:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1cca340 0x55ebc1ccc750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1b116b0 0x55ebc1cbeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3935386824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cbf040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 -- 172.21.15.62:0/3935386824 >> 172.21.15.62:0/3935386824 conn(0x55ebc1caad40 msgr2=0x55ebc1caf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b1affd700 0 client.25468 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 -- 172.21.15.62:0/3935386824 shutdown_connections 2023-07-14T16:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 1 -- 172.21.15.62:0/3935386824 wait complete. 2023-07-14T16:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.233+0000 7f1b55681b80 20 client.25468 trim_cache size 0 max 16384 2023-07-14T16:16:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.263+0000 7f1b55681b80 1 Processor -- start 2023-07-14T16:16:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.264+0000 7f1b55681b80 1 -- start start 2023-07-14T16:16:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.264+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cbcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.264+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.264+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cbd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.264+0000 7f1b55681b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1cbd600 con 0x55ebc1b116b0 2023-07-14T16:16:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.264+0000 7f1b55681b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1b604c0 con 0x55ebc1cb5a90 2023-07-14T16:16:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.264+0000 7f1b55681b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1cca130 con 0x55ebc1cb67e0 2023-07-14T16:16:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.264+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:37124/0 (socket says 172.21.15.62:37124) 2023-07-14T16:16:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3affd700 1 -- 172.21.15.62:0/3863215664 learned_addr learned my addr 172.21.15.62:0/3863215664 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cbcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3bfff700 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cbd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3affd700 1 -- 172.21.15.62:0/3863215664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 msgr2=0x55ebc1cbd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cbd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3affd700 1 -- 172.21.15.62:0/3863215664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 msgr2=0x55ebc1cbcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cbcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.265+0000 7f1b3affd700 1 -- 172.21.15.62:0/3863215664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebc1a5b1f0 con 0x55ebc1cb5a90 2023-07-14T16:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.266+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc6040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1b300d2fd0 tx=0x7f1b3000dfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb549ee5f3c86dd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.266+0000 7f1b227fc700 1 -- 172.21.15.62:0/3863215664 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b300ccb70 con 0x55ebc1cb5a90 2023-07-14T16:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.266+0000 7f1b227fc700 1 -- 172.21.15.62:0/3863215664 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b30099ac0 con 0x55ebc1cb5a90 2023-07-14T16:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.266+0000 7f1b227fc700 1 -- 172.21.15.62:0/3863215664 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b300a2140 con 0x55ebc1cb5a90 2023-07-14T16:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 -- 172.21.15.62:0/3863215664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 msgr2=0x55ebc1cc6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc6040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1b300d2fd0 tx=0x7f1b3000dfb0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 -- 172.21.15.62:0/3863215664 shutdown_connections 2023-07-14T16:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cbd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cbcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3863215664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc6040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 -- 172.21.15.62:0/3863215664 >> 172.21.15.62:0/3863215664 conn(0x55ebc1caad40 msgr2=0x55ebc1cb9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 -- 172.21.15.62:0/3863215664 shutdown_connections 2023-07-14T16:16:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.267+0000 7f1b55681b80 1 -- 172.21.15.62:0/3863215664 wait complete. 2023-07-14T16:16:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.268+0000 7f1b55681b80 1 -- start start 2023-07-14T16:16:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.268+0000 7f1b55681b80 1 Processor -- start 2023-07-14T16:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cc1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b55681b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b55681b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1cc5610 con 0x55ebc1b116b0 2023-07-14T16:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b55681b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1cc5780 con 0x55ebc1cb5a90 2023-07-14T16:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b55681b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc1cc5940 con 0x55ebc1cb67e0 2023-07-14T16:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b55681b80 10 client.0 _ll_register_callbacks cb 0x7f1b34009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b3b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cc1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b3b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cc1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51838/0 (socket says 172.21.15.62:51838) 2023-07-14T16:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b3b7fe700 1 -- 172.21.15.62:0/3136993969 learned_addr learned my addr 172.21.15.62:0/3136993969 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.269+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b3bfff700 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b3b7fe700 1 -- 172.21.15.62:0/3136993969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 msgr2=0x55ebc1cc50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b3b7fe700 1 -- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 msgr2=0x55ebc1cc20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b3b7fe700 1 -- 172.21.15.62:0/3136993969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebc1a1dc00 con 0x55ebc1b116b0 2023-07-14T16:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b3b7fe700 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cc1b70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c005d90 tx=0x7f1b2c01dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f81072121fc54d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.270+0000 7f1b1affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.271+0000 7f1b1affd700 1 -- 172.21.15.62:0/3136993969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c017040 con 0x55ebc1b116b0 2023-07-14T16:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.271+0000 7f1b1affd700 1 -- 172.21.15.62:0/3136993969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1b2c004220 con 0x55ebc1b116b0 2023-07-14T16:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.271+0000 7f1b55681b80 1 -- 172.21.15.62:0/3136993969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b30096030 con 0x55ebc1b116b0 2023-07-14T16:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.271+0000 7f1b1affd700 1 -- 172.21.15.62:0/3136993969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b2c004400 con 0x55ebc1b116b0 2023-07-14T16:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.271+0000 7f1b55681b80 10 client.15768 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.271+0000 7f1b55681b80 1 -- 172.21.15.62:0/3136993969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ebc1a33250 con 0x55ebc1b116b0 2023-07-14T16:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.272+0000 7f1b1affd700 1 -- 172.21.15.62:0/3136993969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b30096030 con 0x55ebc1b116b0 2023-07-14T16:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.272+0000 7f1b1affd700 1 -- 172.21.15.62:0/3136993969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1b2c003a20 con 0x55ebc1b116b0 2023-07-14T16:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.273+0000 7f1b55681b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.273+0000 7f1b55681b80 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.273+0000 7f1b55681b80 20 client.15768 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.273+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1d1ee80 0x55ebc1d21290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.273+0000 7f1b55681b80 4 client.15768 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.273+0000 7f1b55681b80 1 -- 172.21.15.62:0/3136993969 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ebc1bc1ff0 con 0x55ebc1d1ee80 2023-07-14T16:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.273+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1d1ee80 0x55ebc1d21290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.274+0000 7f1b1affd700 10 client.15768 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.274+0000 7f1b3affd700 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1d1ee80 0x55ebc1d21290 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63eff5b8ae1c1f96 server_cookie=dacd12432fa342f0 in_seq=0 out_seq=0 2023-07-14T16:16:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.278+0000 7f1b1affd700 1 -- 172.21.15.62:0/3136993969 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ebc1bc1ff0 con 0x55ebc1d1ee80 2023-07-14T16:16:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.278+0000 7f1b1affd700 10 client.15768 handle_command_reply: tid=0 2023-07-14T16:16:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.278+0000 7f1b55681b80 1 client.15768 shutdown 2023-07-14T16:16:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 -- 172.21.15.62:0/3136993969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 msgr2=0x55ebc1cc1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cc1b70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c005d90 tx=0x7f1b2c01dca0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 -- 172.21.15.62:0/3136993969 shutdown_connections 2023-07-14T16:16:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc1cb67e0 0x55ebc1cc50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc1b116b0 0x55ebc1cc1b70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1d1ee80 0x55ebc1d21290 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 --2- 172.21.15.62:0/3136993969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc1cb5a90 0x55ebc1cc20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 -- 172.21.15.62:0/3136993969 >> 172.21.15.62:0/3136993969 conn(0x55ebc1caad40 msgr2=0x55ebc1cbcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b1affd700 0 client.15768 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 -- 172.21.15.62:0/3136993969 shutdown_connections 2023-07-14T16:16:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.279+0000 7f1b55681b80 1 -- 172.21.15.62:0/3136993969 wait complete. 2023-07-14T16:16:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.280+0000 7f1b55681b80 20 client.15768 trim_cache size 0 max 16384 2023-07-14T16:16:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:16:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:16:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:16:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:16:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.280+0000 7f1b40b9d700 1 -- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1b240a11d0 msgr2=0x7f1b240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.280+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1b240a11d0 0x7f1b240a3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55ebc1cb52b0 tx=0x7f1b34006210 comp rx=0 tx=0).stop 2023-07-14T16:16:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.280+0000 7f1b40b9d700 1 -- 172.21.15.62:0/2193752142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b3c102a10 msgr2=0x7f1b3c0fe470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.280+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b3c102a10 0x7f1b3c0fe470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000d950 tx=0x7f1b3000dc60 comp rx=0 tx=0).stop 2023-07-14T16:16:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.281+0000 7f1b40b9d700 1 -- 172.21.15.62:0/2193752142 shutdown_connections 2023-07-14T16:16:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.281+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b3c105520 0x7f1b3c0fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.281+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b3c068640 0x7f1b3c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.281+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1b240a11d0 0x7f1b240a3680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.281+0000 7f1b40b9d700 1 --2- 172.21.15.62:0/2193752142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b3c102a10 0x7f1b3c0fe470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.281+0000 7f1b40b9d700 1 -- 172.21.15.62:0/2193752142 >> 172.21.15.62:0/2193752142 conn(0x7f1b3c0f9e40 msgr2=0x7f1b3c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.281+0000 7f1b40b9d700 1 -- 172.21.15.62:0/2193752142 shutdown_connections 2023-07-14T16:16:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:42.281+0000 7f1b40b9d700 1 -- 172.21.15.62:0/2193752142 wait complete. 2023-07-14T16:16:42.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:42 smithi081 ceph-mon[127795]: pgmap v389: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:43.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:42 smithi164 ceph-mon[97427]: pgmap v389: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:43.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:42 smithi062 ceph-mon[95543]: pgmap v389: 97 pgs: 97 active+clean; 658 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:44.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:44 smithi081 ceph-mon[127795]: pgmap v390: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-07-14T16:16:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:44 smithi164 ceph-mon[97427]: pgmap v390: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-07-14T16:16:45.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:44 smithi062 ceph-mon[95543]: pgmap v390: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-07-14T16:16:46.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:46 smithi081 ceph-mon[127795]: pgmap v391: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:47.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:46 smithi164 ceph-mon[97427]: pgmap v391: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:47.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:46 smithi062 ceph-mon[95543]: pgmap v391: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:48.548 INFO:tasks.workunit.client.0.smithi062.stderr:Updating files: 62% (7324/11710) Updating files: 63% (7378/11710) Updating files: 64% (7495/11710) Updating files: 65% (7612/11710) Updating files: 66% (7729/11710) Updating files: 67% (7846/11710) Updating files: 68% (7963/11710) Updating files: 69% (8080/11710) Updating files: 70% (8197/11710) Updating files: 71% (8315/11710) Updating files: 72% (8432/11710) Updating files: 73% (8549/11710) Updating files: 74% (8666/11710) Updating files: 75% (8783/11710) Updating files: 76% (8900/11710) Updating files: 77% (9017/11710) Updating files: 78% (9134/11710) Updating files: 79% (9251/11710) Updating files: 80% (9368/11710) Updating files: 81% (9486/11710) Updating files: 82% (9603/11710) Updating files: 83% (9720/11710) Updating files: 84% (9837/11710) Updating files: 85% (9954/11710) Updating files: 86% (10071/11710) Updating files: 87% (10188/11710) Updating files: 88% (10305/11710) Updating files: 89% (10422/11710) Updating files: 90% (10539/11710) Updating files: 91% (10657/11710) Updating files: 92% (10774/11710) Updating files: 93% (10891/11710) Updating files: 94% (11008/11710) Updating files: 95% (11125/11710) Updating files: 96% (11242/11710) Updating files: 97% (11359/11710) Updating files: 98% (11476/11710) Updating files: 99% (11593/11710) Updating files: 99% (11691/11710) Updating files: 100% (11710/11710) Updating files: 100% (11710/11710), done. 2023-07-14T16:16:48.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:48 smithi081 ceph-mon[127795]: pgmap v392: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T16:16:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:48 smithi164 ceph-mon[97427]: pgmap v392: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T16:16:49.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:48 smithi062 ceph-mon[95543]: pgmap v392: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-07-14T16:16:49.392 INFO:tasks.workunit.client.0.smithi062.stderr:Note: switching to '3fc0089cc1082313a98853d48b892acdc1385b8b'. 2023-07-14T16:16:49.392 INFO:tasks.workunit.client.0.smithi062.stderr: 2023-07-14T16:16:49.392 INFO:tasks.workunit.client.0.smithi062.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-14T16:16:49.392 INFO:tasks.workunit.client.0.smithi062.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-14T16:16:49.392 INFO:tasks.workunit.client.0.smithi062.stderr:state without impacting any branches by switching back to a branch. 2023-07-14T16:16:49.393 INFO:tasks.workunit.client.0.smithi062.stderr: 2023-07-14T16:16:49.393 INFO:tasks.workunit.client.0.smithi062.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-14T16:16:49.393 INFO:tasks.workunit.client.0.smithi062.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-14T16:16:49.393 INFO:tasks.workunit.client.0.smithi062.stderr: 2023-07-14T16:16:49.393 INFO:tasks.workunit.client.0.smithi062.stderr: git switch -c 2023-07-14T16:16:49.393 INFO:tasks.workunit.client.0.smithi062.stderr: 2023-07-14T16:16:49.393 INFO:tasks.workunit.client.0.smithi062.stderr:Or undo this operation with: 2023-07-14T16:16:49.394 INFO:tasks.workunit.client.0.smithi062.stderr: 2023-07-14T16:16:49.394 INFO:tasks.workunit.client.0.smithi062.stderr: git switch - 2023-07-14T16:16:49.394 INFO:tasks.workunit.client.0.smithi062.stderr: 2023-07-14T16:16:49.394 INFO:tasks.workunit.client.0.smithi062.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-14T16:16:49.394 INFO:tasks.workunit.client.0.smithi062.stderr: 2023-07-14T16:16:49.394 INFO:tasks.workunit.client.0.smithi062.stderr:HEAD is now at 3fc0089cc10 Merge remote-tracking branch 'rishabh/fs-auth-subcmd' into wip-rishabh-2023Jul13 2023-07-14T16:16:49.408 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-14T16:16:49.460 INFO:tasks.workunit.client.0.smithi062.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-14T16:16:49.465 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-14T16:16:49.466 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-14T16:16:50.057 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-14T16:16:50.174 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-14T16:16:50.238 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-14T16:16:50.242 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-14T16:16:50.242 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-14T16:16:50.308 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-14T16:16:50.313 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:16:50.313 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-14T16:16:50.357 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2023-07-14T16:16:50.358 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2023-07-14T16:16:50.359 DEBUG:teuthology.orchestra.run.smithi062:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3fc0089cc1082313a98853d48b892acdc1385b8b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2023-07-14T16:16:50.466 INFO:tasks.workunit.client.0.smithi062.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2023-07-14T16:16:50.468 INFO:tasks.workunit.client.0.smithi062.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2023-07-14T16:16:50.468 INFO:tasks.workunit.client.0.smithi062.stderr:+ trap -- retry EXIT 2023-07-14T16:16:50.468 INFO:tasks.workunit.client.0.smithi062.stderr:+ rm -rf ffsb 2023-07-14T16:16:50.471 INFO:tasks.workunit.client.0.smithi062.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2023-07-14T16:16:50.481 INFO:tasks.workunit.client.0.smithi062.stderr:Cloning into 'ffsb'... 2023-07-14T16:16:50.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:50 smithi081 ceph-mon[127795]: pgmap v393: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:51.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:50 smithi164 ceph-mon[97427]: pgmap v393: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:51.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:50 smithi062 ceph-mon[95543]: pgmap v393: 97 pgs: 97 active+clean; 659 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:16:52.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:52 smithi081 ceph-mon[127795]: pgmap v394: 97 pgs: 97 active+clean; 670 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T16:16:53.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:52 smithi164 ceph-mon[97427]: pgmap v394: 97 pgs: 97 active+clean; 670 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T16:16:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:52 smithi062 ceph-mon[95543]: pgmap v394: 97 pgs: 97 active+clean; 670 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T16:16:53.365 INFO:tasks.workunit.client.0.smithi062.stderr:+ trap - EXIT 2023-07-14T16:16:53.365 INFO:tasks.workunit.client.0.smithi062.stderr:+ cd ffsb 2023-07-14T16:16:53.367 INFO:tasks.workunit.client.0.smithi062.stderr:+ ./configure 2023-07-14T16:16:53.682 INFO:tasks.workunit.client.0.smithi062.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-07-14T16:16:54.701 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether build environment is sane... yes 2023-07-14T16:16:54.738 INFO:tasks.workunit.client.0.smithi062.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2023-07-14T16:16:54.739 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gawk... gawk 2023-07-14T16:16:54.792 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether make sets $(MAKE)... yes 2023-07-14T16:16:54.897 INFO:tasks.workunit.client.0.smithi062.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-07-14T16:16:54.898 INFO:tasks.workunit.client.0.smithi062.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-07-14T16:16:54.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:54 smithi081 ceph-mon[127795]: pgmap v395: 97 pgs: 97 active+clean; 2.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 148 KiB/s wr, 29 op/s 2023-07-14T16:16:54.930 INFO:tasks.workunit.client.0.smithi062.stdout:checking for style of include used by make... GNU 2023-07-14T16:16:54.938 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc... gcc 2023-07-14T16:16:55.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:54 smithi164 ceph-mon[97427]: pgmap v395: 97 pgs: 97 active+clean; 2.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 148 KiB/s wr, 29 op/s 2023-07-14T16:16:55.092 INFO:tasks.workunit.client.0.smithi062.stdout:checking for C compiler default output file name... a.out 2023-07-14T16:16:55.099 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether the C compiler works... yes 2023-07-14T16:16:55.105 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are cross compiling... no 2023-07-14T16:16:55.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:54 smithi062 ceph-mon[95543]: pgmap v395: 97 pgs: 97 active+clean; 2.3 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 148 KiB/s wr, 29 op/s 2023-07-14T16:16:55.196 INFO:tasks.workunit.client.0.smithi062.stdout:checking for suffix of executables... 2023-07-14T16:16:55.279 INFO:tasks.workunit.client.0.smithi062.stdout:checking for suffix of object files... o 2023-07-14T16:16:55.369 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are using the GNU C compiler... yes 2023-07-14T16:16:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.449+0000 7f762d9ff700 1 -- 172.21.15.62:0/1215968229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76280ffb40 msgr2=0x7f76280fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.449+0000 7f762d9ff700 1 --2- 172.21.15.62:0/1215968229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76280ffb40 0x7f76280fff50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7618009990 tx=0x7f7618009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.450+0000 7f762d9ff700 1 -- 172.21.15.62:0/1215968229 shutdown_connections 2023-07-14T16:16:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.450+0000 7f762d9ff700 1 --2- 172.21.15.62:0/1215968229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76281016c0 0x7f7628107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.450+0000 7f762d9ff700 1 --2- 172.21.15.62:0/1215968229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76280ffb40 0x7f76280fff50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.450+0000 7f762d9ff700 1 --2- 172.21.15.62:0/1215968229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7628100d30 0x7f7628101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.450+0000 7f762d9ff700 1 -- 172.21.15.62:0/1215968229 >> 172.21.15.62:0/1215968229 conn(0x7f76280fb160 msgr2=0x7f76280fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.450+0000 7f762d9ff700 1 -- 172.21.15.62:0/1215968229 shutdown_connections 2023-07-14T16:16:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.450+0000 7f762d9ff700 1 -- 172.21.15.62:0/1215968229 wait complete. 2023-07-14T16:16:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.451+0000 7f762d9ff700 1 Processor -- start 2023-07-14T16:16:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.451+0000 7f762d9ff700 1 -- start start 2023-07-14T16:16:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.451+0000 7f762d9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76280ffb40 0x7f7628072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.451+0000 7f762d9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7628100d30 0x7f7628072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.451+0000 7f762d9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76281016c0 0x7f7628073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.451+0000 7f762d9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762819f860 con 0x7f76281016c0 2023-07-14T16:16:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.451+0000 7f762d9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762819f9a0 con 0x7f7628100d30 2023-07-14T16:16:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.451+0000 7f762d9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762819fae0 con 0x7f76280ffb40 2023-07-14T16:16:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76280ffb40 0x7f7628072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76267fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7628100d30 0x7f7628072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76267fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7628100d30 0x7f7628072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43428/0 (socket says 172.21.15.62:43428) 2023-07-14T16:16:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76267fc700 1 -- 172.21.15.62:0/302388048 learned_addr learned my addr 172.21.15.62:0/302388048 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76277fe700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76281016c0 0x7f7628073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76267fc700 1 -- 172.21.15.62:0/302388048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76280ffb40 msgr2=0x7f7628072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76267fc700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76280ffb40 0x7f7628072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76267fc700 1 -- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76281016c0 msgr2=0x7f7628073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76267fc700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76281016c0 0x7f7628073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76267fc700 1 -- 172.21.15.62:0/302388048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180095f0 con 0x7f7628100d30 2023-07-14T16:16:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.452+0000 7f76277fe700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76281016c0 0x7f7628073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.453+0000 7f76267fc700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7628100d30 0x7f7628072e40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f761c00e9a0 tx=0x7f761c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.453+0000 7f762c9fd700 1 -- 172.21.15.62:0/302388048 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c009790 con 0x7f7628100d30 2023-07-14T16:16:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.453+0000 7f762d9ff700 1 -- 172.21.15.62:0/302388048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762819fd70 con 0x7f7628100d30 2023-07-14T16:16:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.453+0000 7f762d9ff700 1 -- 172.21.15.62:0/302388048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76281a0240 con 0x7f7628100d30 2023-07-14T16:16:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.453+0000 7f762c9fd700 1 -- 172.21.15.62:0/302388048 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f761c010430 con 0x7f7628100d30 2023-07-14T16:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.453+0000 7f762c9fd700 1 -- 172.21.15.62:0/302388048 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c00cea0 con 0x7f7628100d30 2023-07-14T16:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.457+0000 7f762c9fd700 1 -- 172.21.15.62:0/302388048 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f761c00ca10 con 0x7f7628100d30 2023-07-14T16:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76413cfb80 1 Processor -- start 2023-07-14T16:16:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76413cfb80 1 -- start start 2023-07-14T16:16:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f762c9fd700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76100a11d0 0x7f76100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90e8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f762c9fd700 1 -- 172.21.15.62:0/302388048 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f761c014070 con 0x7f7628100d30 2023-07-14T16:16:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76413cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d8fedc50 con 0x5564d90e8840 2023-07-14T16:16:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76413cfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d90409e0 con 0x5564d90e96e0 2023-07-14T16:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76413cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d9039e70 con 0x5564d8f446b0 2023-07-14T16:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.458+0000 7f76277fe700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76100a11d0 0x7f76100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f76277fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f76267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90e8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:58590/0 (socket says 172.21.15.62:58590) 2023-07-14T16:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f7626ffd700 1 -- 172.21.15.62:0/3814533909 learned_addr learned my addr 172.21.15.62:0/3814533909 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f76267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90e91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45614/0 (socket says 172.21.15.62:45614) 2023-07-14T16:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f7626ffd700 1 -- 172.21.15.62:0/3814533909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 msgr2=0x5564d90e9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f7626ffd700 1 --2- 172.21.15.62:0/3814533909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90e9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f7626ffd700 1 -- 172.21.15.62:0/3814533909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 msgr2=0x5564d90e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f7626ffd700 1 --2- 172.21.15.62:0/3814533909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.459+0000 7f7626ffd700 1 -- 172.21.15.62:0/3814533909 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564d8e67d80 con 0x5564d8f446b0 2023-07-14T16:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.460+0000 7f76277fe700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76100a11d0 0x7f76100a3680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f762000aec0 tx=0x7f762000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.460+0000 7f7626ffd700 1 --2- 172.21.15.62:0/3814533909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90e8300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7618009990 tx=0x7f7618004000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=706ff9254cddf0eb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.460+0000 7f760dffb700 1 -- 172.21.15.62:0/3814533909 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761801d070 con 0x5564d8f446b0 2023-07-14T16:16:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.460+0000 7f760dffb700 1 -- 172.21.15.62:0/3814533909 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f761801e040 con 0x5564d8f446b0 2023-07-14T16:16:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.460+0000 7f760dffb700 1 -- 172.21.15.62:0/3814533909 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800b6b0 con 0x5564d8f446b0 2023-07-14T16:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 -- 172.21.15.62:0/3814533909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 msgr2=0x5564d90e8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 --2- 172.21.15.62:0/3814533909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90e8300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7618009990 tx=0x7f7618004000 comp rx=0 tx=0).stop 2023-07-14T16:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 -- 172.21.15.62:0/3814533909 shutdown_connections 2023-07-14T16:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 --2- 172.21.15.62:0/3814533909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90e9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 --2- 172.21.15.62:0/3814533909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90e8300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 --2- 172.21.15.62:0/3814533909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 -- 172.21.15.62:0/3814533909 >> 172.21.15.62:0/3814533909 conn(0x5564d90ddbe0 msgr2=0x5564d90e2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 -- 172.21.15.62:0/3814533909 shutdown_connections 2023-07-14T16:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 -- 172.21.15.62:0/3814533909 wait complete. 2023-07-14T16:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.461+0000 7f76413cfb80 1 -- start start 2023-07-14T16:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.462+0000 7f76413cfb80 1 Processor -- start 2023-07-14T16:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.462+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.462+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e8840 0x5564d90f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.462+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e96e0 0x5564d90f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.462+0000 7f76413cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d8fd9a60 con 0x5564d90e96e0 2023-07-14T16:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.462+0000 7f76413cfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d8fd9760 con 0x5564d8f446b0 2023-07-14T16:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.462+0000 7f76413cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d8ebd8c0 con 0x5564d90e8840 2023-07-14T16:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76413cfb80 10 client.0 _ll_register_callbacks cb 0x5564d8fd7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e96e0 0x5564d90f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e96e0 0x5564d90f8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45616/0 (socket says 172.21.15.62:45616) 2023-07-14T16:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43450/0 (socket says 172.21.15.62:43450) 2023-07-14T16:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76277fe700 1 -- 172.21.15.62:0/2768067728 learned_addr learned my addr 172.21.15.62:0/2768067728 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76267fc700 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e8840 0x5564d90f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76277fe700 1 -- 172.21.15.62:0/2768067728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e8840 msgr2=0x5564d90f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76277fe700 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e8840 0x5564d90f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76277fe700 1 -- 172.21.15.62:0/2768067728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 msgr2=0x5564d90f17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76277fe700 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.463+0000 7f76277fe700 1 -- 172.21.15.62:0/2768067728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76281088b0 con 0x5564d90e96e0 2023-07-14T16:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f76277fe700 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e96e0 0x5564d90f8d60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f762000a780 tx=0x7f7620004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e091039d210a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f7606ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f7606ffd700 1 -- 172.21.15.62:0/2768067728 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620004bb0 con 0x5564d90e96e0 2023-07-14T16:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f76413cfb80 1 -- 172.21.15.62:0/2768067728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564d8f93490 con 0x5564d90e96e0 2023-07-14T16:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f7606ffd700 1 -- 172.21.15.62:0/2768067728 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7620004d10 con 0x5564d90e96e0 2023-07-14T16:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f7606ffd700 1 -- 172.21.15.62:0/2768067728 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7620003a00 con 0x5564d90e96e0 2023-07-14T16:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f76413cfb80 10 client.15780 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f76413cfb80 1 -- 172.21.15.62:0/2768067728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5564d8e50c00 con 0x5564d90e96e0 2023-07-14T16:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f7606ffd700 1 -- 172.21.15.62:0/2768067728 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5564d8f93490 con 0x5564d90e96e0 2023-07-14T16:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.464+0000 7f7606ffd700 1 -- 172.21.15.62:0/2768067728 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f76200043f0 con 0x5564d90e96e0 2023-07-14T16:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.465+0000 7f76413cfb80 10 client.15780 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.465+0000 7f76413cfb80 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.465+0000 7f76413cfb80 20 client.15780 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.465+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5564d90fd640 0x5564d90ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.465+0000 7f76413cfb80 4 client.15780 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.465+0000 7f76413cfb80 1 -- 172.21.15.62:0/2768067728 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564d9032e50 con 0x5564d90fd640 2023-07-14T16:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.465+0000 7f7626ffd700 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5564d90fd640 0x5564d90ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.466+0000 7f7626ffd700 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5564d90fd640 0x5564d90ffa50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1353b2896ac29e1 server_cookie=a7447bcd6528aaab in_seq=0 out_seq=0 2023-07-14T16:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.466+0000 7f7606ffd700 10 client.15780 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.466+0000 7f7606ffd700 1 -- 172.21.15.62:0/2768067728 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5564d9032e50 con 0x5564d90fd640 2023-07-14T16:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.466+0000 7f7606ffd700 10 client.15780 handle_command_reply: tid=0 2023-07-14T16:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.466+0000 7f76413cfb80 1 client.15780 shutdown 2023-07-14T16:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.466+0000 7f76413cfb80 1 -- 172.21.15.62:0/2768067728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e96e0 msgr2=0x5564d90f8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.466+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e96e0 0x5564d90f8d60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f762000a780 tx=0x7f7620004b40 comp rx=0 tx=0).stop 2023-07-14T16:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 1 -- 172.21.15.62:0/2768067728 shutdown_connections 2023-07-14T16:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e96e0 0x5564d90f8d60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5564d90fd640 0x5564d90ffa50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2768067728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e8840 0x5564d90f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 1 -- 172.21.15.62:0/2768067728 >> 172.21.15.62:0/2768067728 conn(0x5564d90ddbe0 msgr2=0x5564d90e24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f7606ffd700 0 client.15780 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 1 -- 172.21.15.62:0/2768067728 shutdown_connections 2023-07-14T16:16:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 1 -- 172.21.15.62:0/2768067728 wait complete. 2023-07-14T16:16:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.467+0000 7f76413cfb80 20 client.15780 trim_cache size 0 max 16384 2023-07-14T16:16:55.459 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether gcc accepts -g... yes 2023-07-14T16:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76413cfb80 1 Processor -- start 2023-07-14T16:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76413cfb80 1 -- start start 2023-07-14T16:16:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90ec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76413cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d90e3a40 con 0x5564d90e8840 2023-07-14T16:16:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76413cfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d90f1350 con 0x5564d90e96e0 2023-07-14T16:16:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76413cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d90dfd00 con 0x5564d8f446b0 2023-07-14T16:16:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.485+0000 7f76267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45618/0 (socket says 172.21.15.62:45618) 2023-07-14T16:16:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76267fc700 1 -- 172.21.15.62:0/2542355097 learned_addr learned my addr 172.21.15.62:0/2542355097 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76277fe700 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f7626ffd700 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76277fe700 1 -- 172.21.15.62:0/2542355097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 msgr2=0x5564d90f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76277fe700 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76277fe700 1 -- 172.21.15.62:0/2542355097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 msgr2=0x5564d90f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76277fe700 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76277fe700 1 -- 172.21.15.62:0/2542355097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564d8e8e1f0 con 0x5564d90e96e0 2023-07-14T16:16:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76267fc700 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.486+0000 7f76277fe700 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90ec550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7620004b10 tx=0x7f76200078d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a6b8e78ad08f3c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f760e7fc700 1 -- 172.21.15.62:0/2542355097 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762001ba30 con 0x5564d90e96e0 2023-07-14T16:16:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f760e7fc700 1 -- 172.21.15.62:0/2542355097 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7620003e90 con 0x5564d90e96e0 2023-07-14T16:16:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f760e7fc700 1 -- 172.21.15.62:0/2542355097 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620004260 con 0x5564d90e96e0 2023-07-14T16:16:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f76413cfb80 1 -- 172.21.15.62:0/2542355097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 msgr2=0x5564d90ec550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90ec550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7620004b10 tx=0x7f76200078d0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f76413cfb80 1 -- 172.21.15.62:0/2542355097 shutdown_connections 2023-07-14T16:16:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d90e96e0 0x5564d90ec550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d8f446b0 0x5564d90f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f76413cfb80 1 --2- 172.21.15.62:0/2542355097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f76413cfb80 1 -- 172.21.15.62:0/2542355097 >> 172.21.15.62:0/2542355097 conn(0x5564d90ddbe0 msgr2=0x5564d91043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.487+0000 7f76413cfb80 1 -- 172.21.15.62:0/2542355097 shutdown_connections 2023-07-14T16:16:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.488+0000 7f76413cfb80 1 -- 172.21.15.62:0/2542355097 wait complete. 2023-07-14T16:16:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.488+0000 7f76413cfb80 1 -- start start 2023-07-14T16:16:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.488+0000 7f76413cfb80 1 Processor -- start 2023-07-14T16:16:55.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76413cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e96e0 0x5564d90f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76413cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d90f84a0 con 0x5564d90e8840 2023-07-14T16:16:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76413cfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d90f8610 con 0x5564d8f446b0 2023-07-14T16:16:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76413cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d90f87d0 con 0x5564d90e96e0 2023-07-14T16:16:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76413cfb80 10 client.0 _ll_register_callbacks cb 0x7f762000a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43468/0 (socket says 172.21.15.62:43468) 2023-07-14T16:16:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45630/0 (socket says 172.21.15.62:45630) 2023-07-14T16:16:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f7626ffd700 1 -- 172.21.15.62:0/1913861938 learned_addr learned my addr 172.21.15.62:0/1913861938 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f76277fe700 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e96e0 0x5564d90f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f7626ffd700 1 -- 172.21.15.62:0/1913861938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e96e0 msgr2=0x5564d90f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f7626ffd700 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e96e0 0x5564d90f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f7626ffd700 1 -- 172.21.15.62:0/1913861938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 msgr2=0x5564d90f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f7626ffd700 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.489+0000 7f7626ffd700 1 -- 172.21.15.62:0/1913861938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76281a0240 con 0x5564d8f446b0 2023-07-14T16:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.490+0000 7f7626ffd700 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f4a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7618003f60 tx=0x7f7618005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de59a189a395c475 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.490+0000 7f7606ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.490+0000 7f7606ffd700 1 -- 172.21.15.62:0/1913861938 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761801d040 con 0x5564d8f446b0 2023-07-14T16:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.490+0000 7f76413cfb80 1 -- 172.21.15.62:0/1913861938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f761801e040 con 0x5564d8f446b0 2023-07-14T16:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.490+0000 7f7606ffd700 1 -- 172.21.15.62:0/1913861938 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7618032040 con 0x5564d8f446b0 2023-07-14T16:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.490+0000 7f7606ffd700 1 -- 172.21.15.62:0/1913861938 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7618035390 con 0x5564d8f446b0 2023-07-14T16:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.490+0000 7f76413cfb80 10 client.25553 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.490+0000 7f76413cfb80 1 -- 172.21.15.62:0/1913861938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5564d8e50c00 con 0x5564d8f446b0 2023-07-14T16:16:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f7606ffd700 1 -- 172.21.15.62:0/1913861938 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f761801e040 con 0x5564d8f446b0 2023-07-14T16:16:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f7606ffd700 1 -- 172.21.15.62:0/1913861938 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f761800b6b0 con 0x5564d8f446b0 2023-07-14T16:16:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f76413cfb80 10 client.25553 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f76413cfb80 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f76413cfb80 20 client.25553 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f76413cfb80 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5564d9151f20 0x5564d9154330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f76413cfb80 4 client.25553 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f76413cfb80 1 -- 172.21.15.62:0/1913861938 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5564d8ff4ff0 con 0x5564d9151f20 2023-07-14T16:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.491+0000 7f76267fc700 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5564d9151f20 0x5564d9154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.492+0000 7f76267fc700 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5564d9151f20 0x5564d9154330 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ae66d70509da845 server_cookie=dd63993576341d8b in_seq=0 out_seq=0 2023-07-14T16:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.492+0000 7f7606ffd700 10 client.25553 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:55.585 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc option to accept ISO C89... none needed 2023-07-14T16:16:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f7606ffd700 1 -- 172.21.15.62:0/1913861938 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5564d8ff4ff0 con 0x5564d9151f20 2023-07-14T16:16:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f7606ffd700 10 client.25553 handle_command_reply: tid=0 2023-07-14T16:16:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 client.25553 shutdown 2023-07-14T16:16:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 -- 172.21.15.62:0/1913861938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 msgr2=0x5564d90f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f4a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7618003f60 tx=0x7f7618005590 comp rx=0 tx=0).stop 2023-07-14T16:16:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 -- 172.21.15.62:0/1913861938 shutdown_connections 2023-07-14T16:16:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564d90e96e0 0x5564d90f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5564d9151f20 0x5564d9154330 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5564d8f446b0 0x5564d90f4a00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 --2- 172.21.15.62:0/1913861938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5564d90e8840 0x5564d90f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f7606ffd700 0 client.25553 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 -- 172.21.15.62:0/1913861938 >> 172.21.15.62:0/1913861938 conn(0x5564d90ddbe0 msgr2=0x5564d90efa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 -- 172.21.15.62:0/1913861938 shutdown_connections 2023-07-14T16:16:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.851+0000 7f76413cfb80 1 -- 172.21.15.62:0/1913861938 wait complete. 2023-07-14T16:16:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f76413cfb80 20 client.25553 trim_cache size 0 max 16384 2023-07-14T16:16:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:16:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:16:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:16:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:16:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 -- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76100a11d0 msgr2=0x7f76100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76100a11d0 0x7f76100a3680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f762000aec0 tx=0x7f762000a4c0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 -- 172.21.15.62:0/302388048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7628100d30 msgr2=0x7f7628072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7628100d30 0x7f7628072e40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f761c00e9a0 tx=0x7f761c00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 -- 172.21.15.62:0/302388048 shutdown_connections 2023-07-14T16:16:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76281016c0 0x7f7628073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76280ffb40 0x7f7628072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76100a11d0 0x7f76100a3680 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 --2- 172.21.15.62:0/302388048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7628100d30 0x7f7628072e40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.852+0000 7f762d9ff700 1 -- 172.21.15.62:0/302388048 >> 172.21.15.62:0/302388048 conn(0x7f76280fb160 msgr2=0x7f76281042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.853+0000 7f762d9ff700 1 -- 172.21.15.62:0/302388048 shutdown_connections 2023-07-14T16:16:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:16:55.853+0000 7f762d9ff700 1 -- 172.21.15.62:0/302388048 wait complete. 2023-07-14T16:16:55.888 INFO:tasks.workunit.client.0.smithi062.stdout:checking dependency style of gcc... gcc3 2023-07-14T16:16:56.012 INFO:tasks.workunit.client.0.smithi062.stdout:checking how to run the C preprocessor... gcc -E 2023-07-14T16:16:56.121 INFO:tasks.workunit.client.0.smithi062.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-07-14T16:16:56.125 INFO:tasks.workunit.client.0.smithi062.stdout:checking for egrep... /usr/bin/grep -E 2023-07-14T16:16:56.152 INFO:tasks.workunit.client.0.smithi062.stdout:checking for AIX... no 2023-07-14T16:16:56.180 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc... (cached) gcc 2023-07-14T16:16:56.209 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-07-14T16:16:56.211 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether gcc accepts -g... (cached) yes 2023-07-14T16:16:56.212 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-07-14T16:16:56.216 INFO:tasks.workunit.client.0.smithi062.stdout:checking dependency style of gcc... (cached) gcc3 2023-07-14T16:16:56.364 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:16:56.369 INFO:tasks.workunit.client.0.smithi062.stdout:checking for main in -lm... yes 2023-07-14T16:16:56.508 INFO:tasks.workunit.client.0.smithi062.stdout:checking for main in -lpthread... yes 2023-07-14T16:16:56.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 -- 172.21.15.62:0/479577190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4102f20 msgr2=0x7faff4105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 --2- 172.21.15.62:0/479577190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4102f20 0x7faff4105300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009990 tx=0x7fafe8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 -- 172.21.15.62:0/479577190 shutdown_connections 2023-07-14T16:16:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 --2- 172.21.15.62:0/479577190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff4105840 0x7faff4107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 --2- 172.21.15.62:0/479577190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff4100600 0x7faff41029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 --2- 172.21.15.62:0/479577190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4102f20 0x7faff4105300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 -- 172.21.15.62:0/479577190 >> 172.21.15.62:0/479577190 conn(0x7faff40fa000 msgr2=0x7faff40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 -- 172.21.15.62:0/479577190 shutdown_connections 2023-07-14T16:16:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.614+0000 7faff8bff700 1 -- 172.21.15.62:0/479577190 wait complete. 2023-07-14T16:16:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.615+0000 7faff8bff700 1 Processor -- start 2023-07-14T16:16:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.615+0000 7faff8bff700 1 -- start start 2023-07-14T16:16:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.615+0000 7faff8bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4100600 0x7faff41979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.615+0000 7faff8bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff4102f20 0x7faff4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.615+0000 7faff8bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff4105840 0x7faff419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.615+0000 7faff8bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff419f540 con 0x7faff4100600 2023-07-14T16:16:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.615+0000 7faff8bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff419f680 con 0x7faff4105840 2023-07-14T16:16:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.615+0000 7faff8bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff419f810 con 0x7faff4102f20 2023-07-14T16:16:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4100600 0x7faff41979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4100600 0x7faff41979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45648/0 (socket says 172.21.15.62:45648) 2023-07-14T16:16:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 -- 172.21.15.62:0/1360874854 learned_addr learned my addr 172.21.15.62:0/1360874854 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff2d9d700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff4105840 0x7faff419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff1d9b700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff4102f20 0x7faff4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 -- 172.21.15.62:0/1360874854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff4102f20 msgr2=0x7faff4197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff4102f20 0x7faff4197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 -- 172.21.15.62:0/1360874854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff4105840 msgr2=0x7faff419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff4105840 0x7faff419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 -- 172.21.15.62:0/1360874854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe80095f0 con 0x7faff4100600 2023-07-14T16:16:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff2d9d700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff4105840 0x7faff419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.616+0000 7faff259c700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4100600 0x7faff41979c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fafe400d950 tx=0x7fafe400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.617+0000 7fafe37fe700 1 -- 172.21.15.62:0/1360874854 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400f890 con 0x7faff4100600 2023-07-14T16:16:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.617+0000 7faff8bff700 1 -- 172.21.15.62:0/1360874854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff419faf0 con 0x7faff4100600 2023-07-14T16:16:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.617+0000 7fafe37fe700 1 -- 172.21.15.62:0/1360874854 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe400fe80 con 0x7faff4100600 2023-07-14T16:16:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.617+0000 7fafe37fe700 1 -- 172.21.15.62:0/1360874854 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400e3b0 con 0x7faff4100600 2023-07-14T16:16:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.617+0000 7faff8bff700 1 -- 172.21.15.62:0/1360874854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff41a0050 con 0x7faff4100600 2023-07-14T16:16:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.620+0000 7fafe37fe700 1 -- 172.21.15.62:0/1360874854 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fafe4010460 con 0x7faff4100600 2023-07-14T16:16:56.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fb00c58db80 1 Processor -- start 2023-07-14T16:16:56.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fb00c58db80 1 -- start start 2023-07-14T16:16:56.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fafe37fe700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fafdc0a11d0 0x7fafdc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fafe37fe700 1 -- 172.21.15.62:0/1360874854 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fafe40cc6b0 con 0x7faff4100600 2023-07-14T16:16:56.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4e4d0 0x55ab65d4e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d4f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 0x55ab65d500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fb00c58db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65c53cb0 con 0x55ab65d4fc70 2023-07-14T16:16:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fb00c58db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65ca6a40 con 0x55ab65d4e4d0 2023-07-14T16:16:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7fb00c58db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65c9fed0 con 0x55ab65d4ee20 2023-07-14T16:16:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.622+0000 7faff2d9d700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fafdc0a11d0 0x7fafdc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 0x55ab65d500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4e4d0 0x55ab65d4e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4e4d0 0x55ab65d4e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43504/0 (socket says 172.21.15.62:43504) 2023-07-14T16:16:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 -- 172.21.15.62:0/649664771 learned_addr learned my addr 172.21.15.62:0/649664771 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff1d9b700 1 --2- 172.21.15.62:0/649664771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d4f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 -- 172.21.15.62:0/649664771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 msgr2=0x55ab65d4f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 --2- 172.21.15.62:0/649664771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d4f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 -- 172.21.15.62:0/649664771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 msgr2=0x55ab65d500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 --2- 172.21.15.62:0/649664771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 0x55ab65d500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 -- 172.21.15.62:0/649664771 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab65acdde0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff2d9d700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fafdc0a11d0 0x7fafdc0a3680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fafec007f40 tx=0x7fafec007580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:56.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.623+0000 7faff259c700 1 --2- 172.21.15.62:0/649664771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4e4d0 0x55ab65d4e8e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fafe400e010 tx=0x7fafe4099500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45fa42344a390d68 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:56.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.624+0000 7fafe0ff9700 1 -- 172.21.15.62:0/649664771 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe409daa0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.624+0000 7fafe0ff9700 1 -- 172.21.15.62:0/649664771 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe40a1ec0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.624+0000 7fafe0ff9700 1 -- 172.21.15.62:0/649664771 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe40a1c90 con 0x55ab65d4e4d0 2023-07-14T16:16:56.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.624+0000 7fb00c58db80 1 -- 172.21.15.62:0/649664771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4e4d0 msgr2=0x55ab65d4e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.624+0000 7fb00c58db80 1 --2- 172.21.15.62:0/649664771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4e4d0 0x55ab65d4e8e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fafe400e010 tx=0x7fafe4099500 comp rx=0 tx=0).stop 2023-07-14T16:16:56.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.625+0000 7fb00c58db80 1 -- 172.21.15.62:0/649664771 shutdown_connections 2023-07-14T16:16:56.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.625+0000 7fb00c58db80 1 --2- 172.21.15.62:0/649664771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 0x55ab65d500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.625+0000 7fb00c58db80 1 --2- 172.21.15.62:0/649664771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4e4d0 0x55ab65d4e8e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.625+0000 7fb00c58db80 1 --2- 172.21.15.62:0/649664771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d4f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.625+0000 7fb00c58db80 1 -- 172.21.15.62:0/649664771 >> 172.21.15.62:0/649664771 conn(0x55ab65d43da0 msgr2=0x55ab65d491d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.625+0000 7fb00c58db80 1 -- 172.21.15.62:0/649664771 shutdown_connections 2023-07-14T16:16:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.625+0000 7fb00c58db80 1 -- 172.21.15.62:0/649664771 wait complete. 2023-07-14T16:16:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.625+0000 7fb00c58db80 1 -- start start 2023-07-14T16:16:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7fb00c58db80 1 Processor -- start 2023-07-14T16:16:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d60780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d60cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 0x55ab65d67d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7fb00c58db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65c3fac0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7fb00c58db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65c3f7c0 con 0x55ab65d4fc70 2023-07-14T16:16:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7fb00c58db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65b23920 con 0x55ab65d4ee20 2023-07-14T16:16:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7fb00c58db80 10 client.0 _ll_register_callbacks cb 0x55ab65c3d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7faff259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d60780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7faff259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d60780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45674/0 (socket says 172.21.15.62:45674) 2023-07-14T16:16:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7faff259c700 1 -- 172.21.15.62:0/3982749281 learned_addr learned my addr 172.21.15.62:0/3982749281 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.626+0000 7faff1d9b700 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d60cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7faff2d9d700 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 0x55ab65d67d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7faff259c700 1 -- 172.21.15.62:0/3982749281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 msgr2=0x55ab65d60cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7faff259c700 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d60cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7faff259c700 1 -- 172.21.15.62:0/3982749281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 msgr2=0x55ab65d67d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7faff259c700 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 0x55ab65d67d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7faff259c700 1 -- 172.21.15.62:0/3982749281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff41a0050 con 0x55ab65d4e4d0 2023-07-14T16:16:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7faff1d9b700 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d60cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7faff259c700 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d60780 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fafe409a0d0 tx=0x7fafe4010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7c34002a845d0d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.627+0000 7fafd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.628+0000 7fafd9ffb700 1 -- 172.21.15.62:0/3982749281 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe409daa0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.628+0000 7fafd9ffb700 1 -- 172.21.15.62:0/3982749281 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fafe40a1a30 con 0x55ab65d4e4d0 2023-07-14T16:16:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.628+0000 7fb00c58db80 1 -- 172.21.15.62:0/3982749281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab65bf94f0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.628+0000 7fafd9ffb700 1 -- 172.21.15.62:0/3982749281 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafe40a1bf0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.628+0000 7fb00c58db80 10 client.15798 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.628+0000 7fb00c58db80 1 -- 172.21.15.62:0/3982749281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ab65ab6c60 con 0x55ab65d4e4d0 2023-07-14T16:16:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.628+0000 7fafd9ffb700 1 -- 172.21.15.62:0/3982749281 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe407c020 con 0x55ab65d4e4d0 2023-07-14T16:16:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.628+0000 7fafd9ffb700 1 -- 172.21.15.62:0/3982749281 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fafe40989d0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.629+0000 7fb00c58db80 10 client.15798 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.629+0000 7fb00c58db80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.629+0000 7fb00c58db80 20 client.15798 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.629+0000 7fb00c58db80 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ab65d6c030 0x55ab65d6e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.629+0000 7fb00c58db80 4 client.15798 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.629+0000 7fb00c58db80 1 -- 172.21.15.62:0/3982749281 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab65c98eb0 con 0x55ab65d6c030 2023-07-14T16:16:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.629+0000 7faff1d9b700 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ab65d6c030 0x55ab65d6e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.630+0000 7fafd9ffb700 10 client.15798 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.630+0000 7faff1d9b700 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ab65d6c030 0x55ab65d6e440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63687c23b3bbbaf9 server_cookie=747a12daa2f10261 in_seq=0 out_seq=0 2023-07-14T16:16:56.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.630+0000 7fafd9ffb700 1 -- 172.21.15.62:0/3982749281 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab65c98eb0 con 0x55ab65d6c030 2023-07-14T16:16:56.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.630+0000 7fafd9ffb700 10 client.15798 handle_command_reply: tid=0 2023-07-14T16:16:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.630+0000 7fb00c58db80 1 client.15798 shutdown 2023-07-14T16:16:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 -- 172.21.15.62:0/3982749281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 msgr2=0x55ab65d60780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d60780 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fafe409a0d0 tx=0x7fafe4010970 comp rx=0 tx=0).stop 2023-07-14T16:16:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 -- 172.21.15.62:0/3982749281 shutdown_connections 2023-07-14T16:16:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ab65d6c030 0x55ab65d6e440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 0x55ab65d67d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d60780 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fafd9ffb700 0 client.15798 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 --2- 172.21.15.62:0/3982749281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d60cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 -- 172.21.15.62:0/3982749281 >> 172.21.15.62:0/3982749281 conn(0x55ab65d43da0 msgr2=0x55ab65d488b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 -- 172.21.15.62:0/3982749281 shutdown_connections 2023-07-14T16:16:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 1 -- 172.21.15.62:0/3982749281 wait complete. 2023-07-14T16:16:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.631+0000 7fb00c58db80 20 client.15798 trim_cache size 0 max 16384 2023-07-14T16:16:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7fb00c58db80 1 Processor -- start 2023-07-14T16:16:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7fb00c58db80 1 -- start start 2023-07-14T16:16:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4e4d0 0x55ab65d55ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4ee20 0x55ab65d67cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 0x55ab65d52b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7fb00c58db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65d49ed0 con 0x55ab65d4fc70 2023-07-14T16:16:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7fb00c58db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65d6bdb0 con 0x55ab65d4ee20 2023-07-14T16:16:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7fb00c58db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65d456d0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7faff1d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4ee20 0x55ab65d67cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7faff2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 0x55ab65d52b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.650+0000 7faff1d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4ee20 0x55ab65d67cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43524/0 (socket says 172.21.15.62:43524) 2023-07-14T16:16:56.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7faff1d9b700 1 -- 172.21.15.62:0/913165798 learned_addr learned my addr 172.21.15.62:0/913165798 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:56.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7faff259c700 1 --2- 172.21.15.62:0/913165798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4e4d0 0x55ab65d55ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7faff1d9b700 1 -- 172.21.15.62:0/913165798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4e4d0 msgr2=0x55ab65d55ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7faff1d9b700 1 --2- 172.21.15.62:0/913165798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4e4d0 0x55ab65d55ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7faff1d9b700 1 -- 172.21.15.62:0/913165798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 msgr2=0x55ab65d52b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7faff1d9b700 1 --2- 172.21.15.62:0/913165798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 0x55ab65d52b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7faff1d9b700 1 -- 172.21.15.62:0/913165798 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab65af4250 con 0x55ab65d4ee20 2023-07-14T16:16:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7faff1d9b700 1 --2- 172.21.15.62:0/913165798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4ee20 0x55ab65d67cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fafe80098e0 tx=0x7fafe8009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38297c7976eb1ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.651+0000 7fafe17fa700 1 -- 172.21.15.62:0/913165798 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe801d040 con 0x55ab65d4ee20 2023-07-14T16:16:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fafe17fa700 1 -- 172.21.15.62:0/913165798 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe801e040 con 0x55ab65d4ee20 2023-07-14T16:16:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fafe17fa700 1 -- 172.21.15.62:0/913165798 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8005670 con 0x55ab65d4ee20 2023-07-14T16:16:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 -- 172.21.15.62:0/913165798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4ee20 msgr2=0x55ab65d67cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 --2- 172.21.15.62:0/913165798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4ee20 0x55ab65d67cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fafe80098e0 tx=0x7fafe8009510 comp rx=0 tx=0).stop 2023-07-14T16:16:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 -- 172.21.15.62:0/913165798 shutdown_connections 2023-07-14T16:16:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 --2- 172.21.15.62:0/913165798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4fc70 0x55ab65d52b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 --2- 172.21.15.62:0/913165798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4e4d0 0x55ab65d55ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 --2- 172.21.15.62:0/913165798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4ee20 0x55ab65d67cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 -- 172.21.15.62:0/913165798 >> 172.21.15.62:0/913165798 conn(0x55ab65d43da0 msgr2=0x55ab65d5f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:56.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 -- 172.21.15.62:0/913165798 shutdown_connections 2023-07-14T16:16:56.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.652+0000 7fb00c58db80 1 -- 172.21.15.62:0/913165798 wait complete. 2023-07-14T16:16:56.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.653+0000 7fb00c58db80 1 -- start start 2023-07-14T16:16:56.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.653+0000 7fb00c58db80 1 Processor -- start 2023-07-14T16:16:56.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.653+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.653+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d64250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.653+0000 7fb00c58db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 0x55ab65d64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7fb00c58db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65d64e70 con 0x55ab65d4e4d0 2023-07-14T16:16:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7fb00c58db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65d67e30 con 0x55ab65d4fc70 2023-07-14T16:16:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7fb00c58db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab65d67fc0 con 0x55ab65d4ee20 2023-07-14T16:16:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7fb00c58db80 10 client.0 _ll_register_callbacks cb 0x7fafec007a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45700/0 (socket says 172.21.15.62:45700) 2023-07-14T16:16:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff259c700 1 -- 172.21.15.62:0/4224039614 learned_addr learned my addr 172.21.15.62:0/4224039614 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff2d9d700 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 0x55ab65d64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff1d9b700 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d64250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff259c700 1 -- 172.21.15.62:0/4224039614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 msgr2=0x55ab65d64250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff259c700 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d64250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff259c700 1 -- 172.21.15.62:0/4224039614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 msgr2=0x55ab65d64790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff259c700 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 0x55ab65d64790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.654+0000 7faff259c700 1 -- 172.21.15.62:0/4224039614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff41087f0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7faff259c700 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d677b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fafe4098d30 tx=0x7fafe400dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe8bd79b23f575f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7fafd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7fafd9ffb700 1 -- 172.21.15.62:0/4224039614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe40a1c30 con 0x55ab65d4e4d0 2023-07-14T16:16:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7fafd9ffb700 1 -- 172.21.15.62:0/4224039614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fafe407d080 con 0x55ab65d4e4d0 2023-07-14T16:16:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7fb00c58db80 1 -- 172.21.15.62:0/4224039614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafe40a1ec0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7fafd9ffb700 1 -- 172.21.15.62:0/4224039614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafe40b91f0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7fb00c58db80 10 client.15804 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7fb00c58db80 1 -- 172.21.15.62:0/4224039614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ab65acc2b0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.655+0000 7fafd9ffb700 1 -- 172.21.15.62:0/4224039614 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafe40a1ec0 con 0x55ab65d4e4d0 2023-07-14T16:16:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.656+0000 7fafd9ffb700 1 -- 172.21.15.62:0/4224039614 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fafe4095860 con 0x55ab65d4e4d0 2023-07-14T16:16:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.656+0000 7fb00c58db80 10 client.15804 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.656+0000 7fb00c58db80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.656+0000 7fb00c58db80 20 client.15804 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.656+0000 7fb00c58db80 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ab65dc11a0 0x55ab65dc35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.656+0000 7fb00c58db80 4 client.15804 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:16:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.656+0000 7fb00c58db80 1 -- 172.21.15.62:0/4224039614 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab65c5b050 con 0x55ab65dc11a0 2023-07-14T16:16:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.656+0000 7faff1d9b700 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ab65dc11a0 0x55ab65dc35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7faff1d9b700 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ab65dc11a0 0x55ab65dc35b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe9ea556c89ecca server_cookie=f7d4a6f074e9235e in_seq=0 out_seq=0 2023-07-14T16:16:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fafd9ffb700 10 client.15804 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fafd9ffb700 1 -- 172.21.15.62:0/4224039614 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ab65c5b050 con 0x55ab65dc11a0 2023-07-14T16:16:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fafd9ffb700 10 client.15804 handle_command_reply: tid=0 2023-07-14T16:16:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 client.15804 shutdown 2023-07-14T16:16:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 -- 172.21.15.62:0/4224039614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 msgr2=0x55ab65d677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d677b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fafe4098d30 tx=0x7fafe400dfd0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 -- 172.21.15.62:0/4224039614 shutdown_connections 2023-07-14T16:16:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ab65dc11a0 0x55ab65dc35b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab65d4fc70 0x55ab65d64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab65d4e4d0 0x55ab65d677b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 --2- 172.21.15.62:0/4224039614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ab65d4ee20 0x55ab65d64250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 -- 172.21.15.62:0/4224039614 >> 172.21.15.62:0/4224039614 conn(0x55ab65d43da0 msgr2=0x55ab65d52b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fafd9ffb700 0 client.15804 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 -- 172.21.15.62:0/4224039614 shutdown_connections 2023-07-14T16:16:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.657+0000 7fb00c58db80 1 -- 172.21.15.62:0/4224039614 wait complete. 2023-07-14T16:16:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7fb00c58db80 20 client.15804 trim_cache size 0 max 16384 2023-07-14T16:16:56.643 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:16:56.644 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:16:56.644 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:16:56.644 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:16:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 -- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fafdc0a11d0 msgr2=0x7fafdc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fafdc0a11d0 0x7fafdc0a3680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fafec007f40 tx=0x7fafec007580 comp rx=0 tx=0).stop 2023-07-14T16:16:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 -- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4100600 msgr2=0x7faff41979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4100600 0x7faff41979c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fafe400d950 tx=0x7fafe400dc60 comp rx=0 tx=0).stop 2023-07-14T16:16:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 -- 172.21.15.62:0/1360874854 shutdown_connections 2023-07-14T16:16:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fafdc0a11d0 0x7fafdc0a3680 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faff4105840 0x7faff419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff4100600 0x7faff41979c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 --2- 172.21.15.62:0/1360874854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faff4102f20 0x7faff4197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 -- 172.21.15.62:0/1360874854 >> 172.21.15.62:0/1360874854 conn(0x7faff40fa000 msgr2=0x7faff40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 -- 172.21.15.62:0/1360874854 shutdown_connections 2023-07-14T16:16:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.658+0000 7faff8bff700 1 -- 172.21.15.62:0/1360874854 wait complete. 2023-07-14T16:16:56.658 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:16:56.658 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:16:56.763 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:56 smithi062 ceph-mon[95543]: pgmap v396: 97 pgs: 97 active+clean; 2.6 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-07-14T16:16:56.870 INFO:tasks.workunit.client.0.smithi062.stdout:checking for ANSI C header files... yes 2023-07-14T16:16:56.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:56 smithi081 ceph-mon[127795]: pgmap v396: 97 pgs: 97 active+clean; 2.6 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-07-14T16:16:56.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 -- 172.21.15.62:0/1136152440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 msgr2=0x7fa380072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 --2- 172.21.15.62:0/1136152440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 0x7fa380072920 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009990 tx=0x7fa37c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 -- 172.21.15.62:0/1136152440 shutdown_connections 2023-07-14T16:16:56.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 --2- 172.21.15.62:0/1136152440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3801079b0 0x7fa380109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 --2- 172.21.15.62:0/1136152440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 0x7fa380072920 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 --2- 172.21.15.62:0/1136152440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa380072ef0 0x7fa380073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 -- 172.21.15.62:0/1136152440 >> 172.21.15.62:0/1136152440 conn(0x7fa3800fb0e0 msgr2=0x7fa3800fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:56.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 -- 172.21.15.62:0/1136152440 shutdown_connections 2023-07-14T16:16:56.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.939+0000 7fa387dff700 1 -- 172.21.15.62:0/1136152440 wait complete. 2023-07-14T16:16:56.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.941+0000 7fa387dff700 1 Processor -- start 2023-07-14T16:16:56.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.941+0000 7fa387dff700 1 -- start start 2023-07-14T16:16:56.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.941+0000 7fa387dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa380072ef0 0x7fa3801978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.941+0000 7fa387dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 0x7fa380197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.941+0000 7fa387dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3801079b0 0x7fa38019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.941+0000 7fa387dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38019f450 con 0x7fa3800744d0 2023-07-14T16:16:56.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.941+0000 7fa387dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38019f590 con 0x7fa3801079b0 2023-07-14T16:16:56.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.941+0000 7fa387dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38019f720 con 0x7fa380072ef0 2023-07-14T16:16:56.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38539a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 0x7fa380197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38639c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3801079b0 0x7fa38019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38539a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 0x7fa380197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45710/0 (socket says 172.21.15.62:45710) 2023-07-14T16:16:56.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38539a700 1 -- 172.21.15.62:0/4198848565 learned_addr learned my addr 172.21.15.62:0/4198848565 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:56.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa385b9b700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa380072ef0 0x7fa3801978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38639c700 1 -- 172.21.15.62:0/4198848565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa380072ef0 msgr2=0x7fa3801978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38639c700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa380072ef0 0x7fa3801978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38639c700 1 -- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 msgr2=0x7fa380197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:56.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38639c700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 0x7fa380197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:56.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38639c700 1 -- 172.21.15.62:0/4198848565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0095f0 con 0x7fa3801079b0 2023-07-14T16:16:56.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.942+0000 7fa38539a700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 0x7fa380197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:56.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.943+0000 7fa38639c700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3801079b0 0x7fa38019ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa37800cc50 tx=0x7fa37800cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:56.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.943+0000 7fa376ffd700 1 -- 172.21.15.62:0/4198848565 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800aa90 con 0x7fa3801079b0 2023-07-14T16:16:56.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.943+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38019fa00 con 0x7fa3801079b0 2023-07-14T16:16:56.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.943+0000 7fa376ffd700 1 -- 172.21.15.62:0/4198848565 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa378007a80 con 0x7fa3801079b0 2023-07-14T16:16:56.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.943+0000 7fa376ffd700 1 -- 172.21.15.62:0/4198848565 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37801b400 con 0x7fa3801079b0 2023-07-14T16:16:56.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.943+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38019ff60 con 0x7fa3801079b0 2023-07-14T16:16:56.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.944+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3801060f0 con 0x7fa3801079b0 2023-07-14T16:16:56.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.946+0000 7fa376ffd700 1 -- 172.21.15.62:0/4198848565 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa378012420 con 0x7fa3801079b0 2023-07-14T16:16:56.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.947+0000 7fa376ffd700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa36c0a11a0 0x7fa36c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:56.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.948+0000 7fa376ffd700 1 -- 172.21.15.62:0/4198848565 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa3780cc730 con 0x7fa3801079b0 2023-07-14T16:16:56.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.948+0000 7fa385b9b700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa36c0a11a0 0x7fa36c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:56.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.948+0000 7fa385b9b700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa36c0a11a0 0x7fa36c0a3650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00b5c0 tx=0x7fa37c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:56.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:56.951+0000 7fa376ffd700 1 -- 172.21.15.62:0/4198848565 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa378095e30 con 0x7fa3801079b0 2023-07-14T16:16:56.971 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2023-07-14T16:16:57.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:56 smithi164 ceph-mon[97427]: pgmap v396: 97 pgs: 97 active+clean; 2.6 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-07-14T16:16:57.076 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/types.h... yes 2023-07-14T16:16:57.202 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/stat.h... yes 2023-07-14T16:16:57.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.247+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa380061120 con 0x7fa3801079b0 2023-07-14T16:16:57.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.248+0000 7fa376ffd700 1 -- 172.21.15.62:0/4198848565 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fa378099450 con 0x7fa3801079b0 2023-07-14T16:16:57.226 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:16:57.226 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:14:04.590671+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:16:57.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa36c0a11a0 msgr2=0x7fa36c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa36c0a11a0 0x7fa36c0a3650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00b5c0 tx=0x7fa37c005fb0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3801079b0 msgr2=0x7fa38019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3801079b0 0x7fa38019ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa37800cc50 tx=0x7fa37800cf60 comp rx=0 tx=0).stop 2023-07-14T16:16:57.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 shutdown_connections 2023-07-14T16:16:57.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa36c0a11a0 0x7fa36c0a3650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3801079b0 0x7fa38019ee80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa380072ef0 0x7fa3801978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 --2- 172.21.15.62:0/4198848565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3800744d0 0x7fa380197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 >> 172.21.15.62:0/4198848565 conn(0x7fa3800fb0e0 msgr2=0x7fa3800fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 shutdown_connections 2023-07-14T16:16:57.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.251+0000 7fa387dff700 1 -- 172.21.15.62:0/4198848565 wait complete. 2023-07-14T16:16:57.231 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2023-07-14T16:16:57.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:16:57.325 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdlib.h... yes 2023-07-14T16:16:57.449 INFO:tasks.workunit.client.0.smithi062.stdout:checking for string.h... yes 2023-07-14T16:16:57.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 -- 172.21.15.62:0/1948720742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 msgr2=0x7f0430101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 --2- 172.21.15.62:0/1948720742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 0x7f0430101120 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0418009990 tx=0x7f0418009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 -- 172.21.15.62:0/1948720742 shutdown_connections 2023-07-14T16:16:57.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 --2- 172.21.15.62:0/1948720742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0430101660 0x7f0430107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 --2- 172.21.15.62:0/1948720742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04300ffae0 0x7f04300ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 --2- 172.21.15.62:0/1948720742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 0x7f0430101120 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 -- 172.21.15.62:0/1948720742 >> 172.21.15.62:0/1948720742 conn(0x7f04300fb0e0 msgr2=0x7f04300fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 -- 172.21.15.62:0/1948720742 shutdown_connections 2023-07-14T16:16:57.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.499+0000 7f0434fff700 1 -- 172.21.15.62:0/1948720742 wait complete. 2023-07-14T16:16:57.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.500+0000 7f0434fff700 1 Processor -- start 2023-07-14T16:16:57.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.500+0000 7f0434fff700 1 -- start start 2023-07-14T16:16:57.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.500+0000 7f0434fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04300ffae0 0x7f0430197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.500+0000 7f0434fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 0x7f0430197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0434fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0430101660 0x7f043019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0434fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019f4c0 con 0x7f0430100cd0 2023-07-14T16:16:57.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0434fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019f600 con 0x7f04300ffae0 2023-07-14T16:16:57.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0434fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043019f790 con 0x7f0430101660 2023-07-14T16:16:57.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0427fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 0x7f0430197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0427fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 0x7f0430197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45724/0 (socket says 172.21.15.62:45724) 2023-07-14T16:16:57.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0427fff700 1 -- 172.21.15.62:0/1178065968 learned_addr learned my addr 172.21.15.62:0/1178065968 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f042ed9d700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0430101660 0x7f043019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f042e59c700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04300ffae0 0x7f0430197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0427fff700 1 -- 172.21.15.62:0/1178065968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0430101660 msgr2=0x7f043019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0427fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0430101660 0x7f043019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0427fff700 1 -- 172.21.15.62:0/1178065968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04300ffae0 msgr2=0x7f0430197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0427fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04300ffae0 0x7f0430197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f0427fff700 1 -- 172.21.15.62:0/1178065968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0420009520 con 0x7f0430100cd0 2023-07-14T16:16:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f042ed9d700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0430101660 0x7f043019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.501+0000 7f042e59c700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04300ffae0 0x7f0430197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.502+0000 7f0427fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 0x7f0430197e80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0418005910 tx=0x7f04180057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.502+0000 7f04277fe700 1 -- 172.21.15.62:0/1178065968 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041801d070 con 0x7f0430100cd0 2023-07-14T16:16:57.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.502+0000 7f04277fe700 1 -- 172.21.15.62:0/1178065968 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0418022470 con 0x7f0430100cd0 2023-07-14T16:16:57.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.502+0000 7f04277fe700 1 -- 172.21.15.62:0/1178065968 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800f6c0 con 0x7f0430100cd0 2023-07-14T16:16:57.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.502+0000 7f0434fff700 1 -- 172.21.15.62:0/1178065968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04180095f0 con 0x7f0430100cd0 2023-07-14T16:16:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.502+0000 7f0434fff700 1 -- 172.21.15.62:0/1178065968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043019fc50 con 0x7f0430100cd0 2023-07-14T16:16:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.506+0000 7f04277fe700 1 -- 172.21.15.62:0/1178065968 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f04180225e0 con 0x7f0430100cd0 2023-07-14T16:16:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.507+0000 7f04488aab80 1 Processor -- start 2023-07-14T16:16:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.507+0000 7f04277fe700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f041c0a11d0 0x7f041c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.507+0000 7f04488aab80 1 -- start start 2023-07-14T16:16:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.507+0000 7f04277fe700 1 -- 172.21.15.62:0/1178065968 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f04180cded0 con 0x7f0430100cd0 2023-07-14T16:16:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.508+0000 7f04488aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.508+0000 7f04488aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 0x55f932e026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.508+0000 7f04488aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e02c70 0x55f932e030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.508+0000 7f04488aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932d06cb0 con 0x55f932e014d0 2023-07-14T16:16:57.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.508+0000 7f04488aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932d59a40 con 0x55f932e02c70 2023-07-14T16:16:57.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.508+0000 7f04488aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932d52ed0 con 0x55f932e01e20 2023-07-14T16:16:57.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.508+0000 7f042ed9d700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f041c0a11d0 0x7f041c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45736/0 (socket says 172.21.15.62:45736) 2023-07-14T16:16:57.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e02c70 0x55f932e030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042e59c700 1 -- 172.21.15.62:0/3073898423 learned_addr learned my addr 172.21.15.62:0/3073898423 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f0427fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 0x55f932e026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042e59c700 1 -- 172.21.15.62:0/3073898423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 msgr2=0x55f932e026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042e59c700 1 --2- 172.21.15.62:0/3073898423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 0x55f932e026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042e59c700 1 -- 172.21.15.62:0/3073898423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e02c70 msgr2=0x55f932e030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042e59c700 1 --2- 172.21.15.62:0/3073898423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e02c70 0x55f932e030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042e59c700 1 -- 172.21.15.62:0/3073898423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f932b80de0 con 0x55f932e014d0 2023-07-14T16:16:57.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.509+0000 7f042ed9d700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f041c0a11d0 0x7f041c0a3680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0428011750 tx=0x7f0428011b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.510+0000 7f042e59c700 1 --2- 172.21.15.62:0/3073898423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e018e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0420009e30 tx=0x7f0420010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324cbc364e7f0226 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.510+0000 7f0424ff9700 1 -- 172.21.15.62:0/3073898423 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420010d30 con 0x55f932e014d0 2023-07-14T16:16:57.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.510+0000 7f0424ff9700 1 -- 172.21.15.62:0/3073898423 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0420010e90 con 0x55f932e014d0 2023-07-14T16:16:57.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.510+0000 7f0424ff9700 1 -- 172.21.15.62:0/3073898423 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04200127d0 con 0x55f932e014d0 2023-07-14T16:16:57.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 -- 172.21.15.62:0/3073898423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 msgr2=0x55f932e018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 --2- 172.21.15.62:0/3073898423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e018e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0420009e30 tx=0x7f0420010650 comp rx=0 tx=0).stop 2023-07-14T16:16:57.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 -- 172.21.15.62:0/3073898423 shutdown_connections 2023-07-14T16:16:57.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 --2- 172.21.15.62:0/3073898423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e02c70 0x55f932e030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 --2- 172.21.15.62:0/3073898423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e018e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 --2- 172.21.15.62:0/3073898423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 0x55f932e026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 -- 172.21.15.62:0/3073898423 >> 172.21.15.62:0/3073898423 conn(0x55f932df6da0 msgr2=0x55f932dfc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 -- 172.21.15.62:0/3073898423 shutdown_connections 2023-07-14T16:16:57.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.511+0000 7f04488aab80 1 -- 172.21.15.62:0/3073898423 wait complete. 2023-07-14T16:16:57.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.512+0000 7f04488aab80 1 -- start start 2023-07-14T16:16:57.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.512+0000 7f04488aab80 1 Processor -- start 2023-07-14T16:16:57.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f04488aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e014d0 0x55f932e13960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f04488aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e13ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f04488aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f04488aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932cf2ac0 con 0x55f932e02c70 2023-07-14T16:16:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f04488aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932cf27c0 con 0x55f932e01e20 2023-07-14T16:16:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f04488aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932bd6920 con 0x55f932e014d0 2023-07-14T16:16:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f04488aab80 10 client.0 _ll_register_callbacks cb 0x55f932cf0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f0427fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e13ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f042e59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e014d0 0x55f932e13960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45748/0 (socket says 172.21.15.62:45748) 2023-07-14T16:16:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.513+0000 7f042ed9d700 1 -- 172.21.15.62:0/560229011 learned_addr learned my addr 172.21.15.62:0/560229011 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f042e59c700 1 -- 172.21.15.62:0/560229011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 msgr2=0x55f932e13ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f042e59c700 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e13ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f042e59c700 1 -- 172.21.15.62:0/560229011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 msgr2=0x55f932e1aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f042e59c700 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f042e59c700 1 -- 172.21.15.62:0/560229011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0420009a90 con 0x55f932e014d0 2023-07-14T16:16:57.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f0427fff700 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e13ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:57.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f042ed9d700 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:57.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f042e59c700 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e014d0 0x55f932e13960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0420009e30 tx=0x7f042001b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8ac91951069b29 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.514+0000 7f0415ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:16:57.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.515+0000 7f0415ffb700 1 -- 172.21.15.62:0/560229011 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420005030 con 0x55f932e014d0 2023-07-14T16:16:57.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.515+0000 7f0415ffb700 1 -- 172.21.15.62:0/560229011 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0420016070 con 0x55f932e014d0 2023-07-14T16:16:57.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.515+0000 7f04488aab80 1 -- 172.21.15.62:0/560229011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f932cac4f0 con 0x55f932e014d0 2023-07-14T16:16:57.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.515+0000 7f0415ffb700 1 -- 172.21.15.62:0/560229011 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0420005190 con 0x55f932e014d0 2023-07-14T16:16:57.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.515+0000 7f04488aab80 10 client.25516 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:57.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.515+0000 7f04488aab80 1 -- 172.21.15.62:0/560229011 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f932b69c60 con 0x55f932e014d0 2023-07-14T16:16:57.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f0415ffb700 1 -- 172.21.15.62:0/560229011 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f932cac4f0 con 0x55f932e014d0 2023-07-14T16:16:57.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f0415ffb700 1 -- 172.21.15.62:0/560229011 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0420012550 con 0x55f932e014d0 2023-07-14T16:16:57.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f04488aab80 10 client.25516 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:57.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f04488aab80 10 client.25516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:16:57.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f04488aab80 20 client.25516 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:57.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f04488aab80 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f932e1f600 0x55f932e21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f04488aab80 4 client.25516 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:57.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f04488aab80 1 -- 172.21.15.62:0/560229011 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f932d4beb0 con 0x55f932e1f600 2023-07-14T16:16:57.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.516+0000 7f0427fff700 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f932e1f600 0x55f932e21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.517+0000 7f0427fff700 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f932e1f600 0x55f932e21a10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a14ce766faa709c5 server_cookie=6fc8609e595ec439 in_seq=0 out_seq=0 2023-07-14T16:16:57.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.517+0000 7f0415ffb700 10 client.25516 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:57.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.517+0000 7f0415ffb700 1 -- 172.21.15.62:0/560229011 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f932d4beb0 con 0x55f932e1f600 2023-07-14T16:16:57.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.517+0000 7f0415ffb700 10 client.25516 handle_command_reply: tid=0 2023-07-14T16:16:57.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 client.25516 shutdown 2023-07-14T16:16:57.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 -- 172.21.15.62:0/560229011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e014d0 msgr2=0x55f932e13960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e014d0 0x55f932e13960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0420009e30 tx=0x7f042001b6b0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 -- 172.21.15.62:0/560229011 shutdown_connections 2023-07-14T16:16:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f932e1f600 0x55f932e21a10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e014d0 0x55f932e13960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 --2- 172.21.15.62:0/560229011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e13ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f0415ffb700 0 client.25516 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 -- 172.21.15.62:0/560229011 >> 172.21.15.62:0/560229011 conn(0x55f932df6da0 msgr2=0x55f932dfb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 -- 172.21.15.62:0/560229011 shutdown_connections 2023-07-14T16:16:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 1 -- 172.21.15.62:0/560229011 wait complete. 2023-07-14T16:16:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.518+0000 7f04488aab80 20 client.25516 trim_cache size 0 max 16384 2023-07-14T16:16:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.533+0000 7f04488aab80 1 Processor -- start 2023-07-14T16:16:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.533+0000 7f04488aab80 1 -- start start 2023-07-14T16:16:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.533+0000 7f04488aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e014d0 0x55f932e05bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.533+0000 7f04488aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 0x55f932e060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.533+0000 7f04488aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.533+0000 7f04488aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932df6990 con 0x55f932e02c70 2023-07-14T16:16:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.533+0000 7f04488aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932e12e10 con 0x55f932e014d0 2023-07-14T16:16:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.533+0000 7f04488aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932e021f0 con 0x55f932e01e20 2023-07-14T16:16:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45750/0 (socket says 172.21.15.62:45750) 2023-07-14T16:16:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042ed9d700 1 -- 172.21.15.62:0/3873215676 learned_addr learned my addr 172.21.15.62:0/3873215676 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042e59c700 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e014d0 0x55f932e05bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f0427fff700 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 0x55f932e060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042e59c700 1 -- 172.21.15.62:0/3873215676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 msgr2=0x55f932e060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042e59c700 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 0x55f932e060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042e59c700 1 -- 172.21.15.62:0/3873215676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 msgr2=0x55f932e1aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042e59c700 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042e59c700 1 -- 172.21.15.62:0/3873215676 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f932ba7250 con 0x55f932e014d0 2023-07-14T16:16:57.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042ed9d700 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:57.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.534+0000 7f042e59c700 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e014d0 0x55f932e05bb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0420005d00 tx=0x7f0420010650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e8d080e401b225 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04257fa700 1 -- 172.21.15.62:0/3873215676 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420012ba0 con 0x55f932e014d0 2023-07-14T16:16:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04257fa700 1 -- 172.21.15.62:0/3873215676 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0420020040 con 0x55f932e014d0 2023-07-14T16:16:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04257fa700 1 -- 172.21.15.62:0/3873215676 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042000e630 con 0x55f932e014d0 2023-07-14T16:16:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04488aab80 1 -- 172.21.15.62:0/3873215676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e014d0 msgr2=0x55f932e05bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04488aab80 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e014d0 0x55f932e05bb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0420005d00 tx=0x7f0420010650 comp rx=0 tx=0).stop 2023-07-14T16:16:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04488aab80 1 -- 172.21.15.62:0/3873215676 shutdown_connections 2023-07-14T16:16:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04488aab80 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e02c70 0x55f932e1aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04488aab80 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e014d0 0x55f932e05bb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04488aab80 1 --2- 172.21.15.62:0/3873215676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e01e20 0x55f932e060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.535+0000 7f04488aab80 1 -- 172.21.15.62:0/3873215676 >> 172.21.15.62:0/3873215676 conn(0x55f932df6da0 msgr2=0x55f932e09080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.536+0000 7f04488aab80 1 -- 172.21.15.62:0/3873215676 shutdown_connections 2023-07-14T16:16:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.536+0000 7f04488aab80 1 -- 172.21.15.62:0/3873215676 wait complete. 2023-07-14T16:16:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.536+0000 7f04488aab80 1 -- start start 2023-07-14T16:16:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.536+0000 7f04488aab80 1 Processor -- start 2023-07-14T16:16:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f04488aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f04488aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f04488aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e02c70 0x55f932e1a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f04488aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932e1ab50 con 0x55f932e014d0 2023-07-14T16:16:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f04488aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932e1acc0 con 0x55f932e01e20 2023-07-14T16:16:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f04488aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f932e1ae80 con 0x55f932e02c70 2023-07-14T16:16:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f04488aab80 10 client.0 _ll_register_callbacks cb 0x7f0428007ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e170b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45752/0 (socket says 172.21.15.62:45752) 2023-07-14T16:16:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042e59c700 1 -- 172.21.15.62:0/2459483333 learned_addr learned my addr 172.21.15.62:0/2459483333 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f0427fff700 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042ed9d700 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e02c70 0x55f932e1a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042e59c700 1 -- 172.21.15.62:0/2459483333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e02c70 msgr2=0x55f932e1a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042e59c700 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e02c70 0x55f932e1a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042e59c700 1 -- 172.21.15.62:0/2459483333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 msgr2=0x55f932e175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042e59c700 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.537+0000 7f042e59c700 1 -- 172.21.15.62:0/2459483333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04280148a0 con 0x55f932e014d0 2023-07-14T16:16:57.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.538+0000 7f042e59c700 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e170b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0420005e90 tx=0x7f042001bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0d3618be838e60 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.538+0000 7f0415ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:57.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.538+0000 7f0415ffb700 1 -- 172.21.15.62:0/2459483333 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420005430 con 0x55f932e014d0 2023-07-14T16:16:57.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.538+0000 7f04488aab80 1 -- 172.21.15.62:0/2459483333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0420010e90 con 0x55f932e014d0 2023-07-14T16:16:57.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.538+0000 7f0415ffb700 1 -- 172.21.15.62:0/2459483333 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0420000e40 con 0x55f932e014d0 2023-07-14T16:16:57.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.538+0000 7f0415ffb700 1 -- 172.21.15.62:0/2459483333 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04200126c0 con 0x55f932e014d0 2023-07-14T16:16:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.538+0000 7f04488aab80 10 client.15828 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f04488aab80 1 -- 172.21.15.62:0/2459483333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f932b69c60 con 0x55f932e014d0 2023-07-14T16:16:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f0415ffb700 1 -- 172.21.15.62:0/2459483333 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0420010e90 con 0x55f932e014d0 2023-07-14T16:16:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f0415ffb700 1 -- 172.21.15.62:0/2459483333 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0420030450 con 0x55f932e014d0 2023-07-14T16:16:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f04488aab80 10 client.15828 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f04488aab80 10 client.15828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:16:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f04488aab80 20 client.15828 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f04488aab80 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f932e743a0 0x55f932e767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f04488aab80 4 client.15828 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.539+0000 7f04488aab80 1 -- 172.21.15.62:0/2459483333 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f932d0e050 con 0x55f932e743a0 2023-07-14T16:16:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.540+0000 7f0427fff700 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f932e743a0 0x55f932e767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.540+0000 7f0415ffb700 10 client.15828 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.540+0000 7f0427fff700 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f932e743a0 0x55f932e767b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72938d529ff9cf1e server_cookie=2953c45e4fb38adc in_seq=0 out_seq=0 2023-07-14T16:16:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.540+0000 7f0415ffb700 1 -- 172.21.15.62:0/2459483333 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f932d0e050 con 0x55f932e743a0 2023-07-14T16:16:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f0415ffb700 10 client.15828 handle_command_reply: tid=0 2023-07-14T16:16:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 client.15828 shutdown 2023-07-14T16:16:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 -- 172.21.15.62:0/2459483333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 msgr2=0x55f932e170b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e170b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0420005e90 tx=0x7f042001bce0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 -- 172.21.15.62:0/2459483333 shutdown_connections 2023-07-14T16:16:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f932e743a0 0x55f932e767b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f932e02c70 0x55f932e1a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f932e014d0 0x55f932e170b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 --2- 172.21.15.62:0/2459483333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f932e01e20 0x55f932e175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 -- 172.21.15.62:0/2459483333 >> 172.21.15.62:0/2459483333 conn(0x55f932df6da0 msgr2=0x55f932e05bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f0415ffb700 0 client.15828 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:16:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 -- 172.21.15.62:0/2459483333 shutdown_connections 2023-07-14T16:16:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 1 -- 172.21.15.62:0/2459483333 wait complete. 2023-07-14T16:16:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.541+0000 7f04488aab80 20 client.15828 trim_cache size 0 max 16384 2023-07-14T16:16:57.529 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 -- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f041c0a11d0 msgr2=0x7f041c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f041c0a11d0 0x7f041c0a3680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0428011750 tx=0x7f0428011b10 comp rx=0 tx=0).stop 2023-07-14T16:16:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 -- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 msgr2=0x7f0430197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 0x7f0430197e80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0418005910 tx=0x7f04180057d0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 -- 172.21.15.62:0/1178065968 shutdown_connections 2023-07-14T16:16:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0430101660 0x7f043019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f041c0a11d0 0x7f041c0a3680 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04300ffae0 0x7f0430197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 --2- 172.21.15.62:0/1178065968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0430100cd0 0x7f0430197e80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 -- 172.21.15.62:0/1178065968 >> 172.21.15.62:0/1178065968 conn(0x7f04300fb0e0 msgr2=0x7f0430106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 -- 172.21.15.62:0/1178065968 shutdown_connections 2023-07-14T16:16:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.542+0000 7f0434fff700 1 -- 172.21.15.62:0/1178065968 wait complete. 2023-07-14T16:16:57.538 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:16:57.578 INFO:tasks.workunit.client.0.smithi062.stdout:checking for memory.h... yes 2023-07-14T16:16:57.708 INFO:tasks.workunit.client.0.smithi062.stdout:checking for strings.h... yes 2023-07-14T16:16:57.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 -- 172.21.15.62:0/3093385332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 msgr2=0x7f8660105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 --2- 172.21.15.62:0/3093385332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 0x7f8660105260 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8650009990 tx=0x7f8650009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 -- 172.21.15.62:0/3093385332 shutdown_connections 2023-07-14T16:16:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 --2- 172.21.15.62:0/3093385332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86601057a0 0x7f8660107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 --2- 172.21.15.62:0/3093385332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8660100560 0x7f8660102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 --2- 172.21.15.62:0/3093385332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 0x7f8660105260 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 -- 172.21.15.62:0/3093385332 >> 172.21.15.62:0/3093385332 conn(0x7f86600fa000 msgr2=0x7f86600fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 -- 172.21.15.62:0/3093385332 shutdown_connections 2023-07-14T16:16:57.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.793+0000 7f86675ff700 1 -- 172.21.15.62:0/3093385332 wait complete. 2023-07-14T16:16:57.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.794+0000 7f86675ff700 1 Processor -- start 2023-07-14T16:16:57.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.794+0000 7f86675ff700 1 -- start start 2023-07-14T16:16:57.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f86675ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8660100560 0x7f86600ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f86675ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 0x7f86600fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f86675ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86601057a0 0x7f86600fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f86675ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600ff1f0 con 0x7f8660102e80 2023-07-14T16:16:57.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f86675ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600ff360 con 0x7f8660100560 2023-07-14T16:16:57.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f86675ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866019f740 con 0x7f86601057a0 2023-07-14T16:16:57.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8664b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 0x7f86600fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8664b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 0x7f86600fe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45764/0 (socket says 172.21.15.62:45764) 2023-07-14T16:16:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8664b9a700 1 -- 172.21.15.62:0/1265850010 learned_addr learned my addr 172.21.15.62:0/1265850010 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f866539b700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8660100560 0x7f86600ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8665b9c700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86601057a0 0x7f86600fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8664b9a700 1 -- 172.21.15.62:0/1265850010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86601057a0 msgr2=0x7f86600fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8664b9a700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86601057a0 0x7f86600fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8664b9a700 1 -- 172.21.15.62:0/1265850010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8660100560 msgr2=0x7f86600ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8664b9a700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8660100560 0x7f86600ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8664b9a700 1 -- 172.21.15.62:0/1265850010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500095f0 con 0x7f8660102e80 2023-07-14T16:16:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f8665b9c700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86601057a0 0x7f86600fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:16:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.795+0000 7f866539b700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8660100560 0x7f86600ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.796+0000 7f8664b9a700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 0x7f86600fe510 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8650000c00 tx=0x7f8650003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.796+0000 7f86567fc700 1 -- 172.21.15.62:0/1265850010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865001d070 con 0x7f8660102e80 2023-07-14T16:16:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.796+0000 7f86675ff700 1 -- 172.21.15.62:0/1265850010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866019f9c0 con 0x7f8660102e80 2023-07-14T16:16:57.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.796+0000 7f86567fc700 1 -- 172.21.15.62:0/1265850010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f865000bd70 con 0x7f8660102e80 2023-07-14T16:16:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.796+0000 7f86567fc700 1 -- 172.21.15.62:0/1265850010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650022c80 con 0x7f8660102e80 2023-07-14T16:16:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.796+0000 7f86675ff700 1 -- 172.21.15.62:0/1265850010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866019fdf0 con 0x7f8660102e80 2023-07-14T16:16:57.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.800+0000 7f86567fc700 1 -- 172.21.15.62:0/1265850010 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8650022470 con 0x7f8660102e80 2023-07-14T16:16:57.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.801+0000 7f86567fc700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f864c0a1150 0x7f864c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.801+0000 7f86567fc700 1 -- 172.21.15.62:0/1265850010 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86500cdf80 con 0x7f8660102e80 2023-07-14T16:16:57.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.801+0000 7f867aef1b80 1 Processor -- start 2023-07-14T16:16:57.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.801+0000 7f866539b700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f864c0a1150 0x7f864c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f867aef1b80 1 -- start start 2023-07-14T16:16:57.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d4e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d4f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f867aef1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67c53cb0 con 0x55be67d4fc70 2023-07-14T16:16:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f867aef1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67ca6a40 con 0x55be67d4ee20 2023-07-14T16:16:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f867aef1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67c9fed0 con 0x55be67d4e4d0 2023-07-14T16:16:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f8665b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f8665b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45766/0 (socket says 172.21.15.62:45766) 2023-07-14T16:16:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f8665b9c700 1 -- 172.21.15.62:0/797854648 learned_addr learned my addr 172.21.15.62:0/797854648 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f866539b700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f864c0a1150 0x7f864c0a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f865c00afd0 tx=0x7f865c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.802+0000 7f8664b9a700 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d4f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f866539b700 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d4e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f8664b9a700 1 -- 172.21.15.62:0/797854648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 msgr2=0x55be67d4e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f8664b9a700 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d4e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f8664b9a700 1 -- 172.21.15.62:0/797854648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 msgr2=0x55be67d500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f8664b9a700 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f8664b9a700 1 -- 172.21.15.62:0/797854648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be67acdde0 con 0x55be67d4ee20 2023-07-14T16:16:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f8665b9c700 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f8664b9a700 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d4f6e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f86500cba30 tx=0x7f8650009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=745ca8a90d18fb74 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f864b7fe700 1 -- 172.21.15.62:0/797854648 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650003f40 con 0x55be67d4ee20 2023-07-14T16:16:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f864b7fe700 1 -- 172.21.15.62:0/797854648 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86500860b0 con 0x55be67d4ee20 2023-07-14T16:16:57.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.803+0000 7f864b7fe700 1 -- 172.21.15.62:0/797854648 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86500a3800 con 0x55be67d4ee20 2023-07-14T16:16:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 -- 172.21.15.62:0/797854648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 msgr2=0x55be67d4f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d4f6e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f86500cba30 tx=0x7f8650009310 comp rx=0 tx=0).stop 2023-07-14T16:16:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 -- 172.21.15.62:0/797854648 shutdown_connections 2023-07-14T16:16:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d4e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 --2- 172.21.15.62:0/797854648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d4f6e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 -- 172.21.15.62:0/797854648 >> 172.21.15.62:0/797854648 conn(0x55be67d43da0 msgr2=0x55be67d491d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 -- 172.21.15.62:0/797854648 shutdown_connections 2023-07-14T16:16:57.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.804+0000 7f867aef1b80 1 -- 172.21.15.62:0/797854648 wait complete. 2023-07-14T16:16:57.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.805+0000 7f867aef1b80 1 -- start start 2023-07-14T16:16:57.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.805+0000 7f867aef1b80 1 Processor -- start 2023-07-14T16:16:57.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d57da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f867aef1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67c3fac0 con 0x55be67d4fc70 2023-07-14T16:16:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f867aef1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67c3f7c0 con 0x55be67d4ee20 2023-07-14T16:16:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f867aef1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67b23920 con 0x55be67d4e4d0 2023-07-14T16:16:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f867aef1b80 10 client.0 _ll_register_callbacks cb 0x55be67c3d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f8665b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f866539b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d57da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f866539b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d57da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:58776/0 (socket says 172.21.15.62:58776) 2023-07-14T16:16:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f866539b700 1 -- 172.21.15.62:0/918771307 learned_addr learned my addr 172.21.15.62:0/918771307 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f8664b9a700 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f8665b9c700 1 -- 172.21.15.62:0/918771307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 msgr2=0x55be67d57da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f8665b9c700 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d57da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f8665b9c700 1 -- 172.21.15.62:0/918771307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 msgr2=0x55be67d582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f8665b9c700 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.806+0000 7f8665b9c700 1 -- 172.21.15.62:0/918771307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8658009a70 con 0x55be67d4fc70 2023-07-14T16:16:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.807+0000 7f8665b9c700 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5f320 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8658007c30 tx=0x7f8658007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaf5941b15d5293 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.807+0000 7f8648ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:57.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.807+0000 7f8648ff9700 1 -- 172.21.15.62:0/918771307 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658004db0 con 0x55be67d4fc70 2023-07-14T16:16:57.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.807+0000 7f867aef1b80 1 -- 172.21.15.62:0/918771307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be67bf94f0 con 0x55be67d4fc70 2023-07-14T16:16:57.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.807+0000 7f8648ff9700 1 -- 172.21.15.62:0/918771307 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8658012b20 con 0x55be67d4fc70 2023-07-14T16:16:57.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.807+0000 7f8648ff9700 1 -- 172.21.15.62:0/918771307 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8658012ce0 con 0x55be67d4fc70 2023-07-14T16:16:57.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.807+0000 7f867aef1b80 10 client.15840 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:57.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.807+0000 7f867aef1b80 1 -- 172.21.15.62:0/918771307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f866019fdf0 con 0x55be67d4fc70 2023-07-14T16:16:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f8648ff9700 1 -- 172.21.15.62:0/918771307 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55be67bf94f0 con 0x55be67d4fc70 2023-07-14T16:16:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f8648ff9700 1 -- 172.21.15.62:0/918771307 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8658014040 con 0x55be67d4fc70 2023-07-14T16:16:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f867aef1b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f867aef1b80 10 client.15840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:16:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f867aef1b80 20 client.15840 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f867aef1b80 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55be67d63c20 0x55be67d66030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f867aef1b80 4 client.15840 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:57.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f867aef1b80 1 -- 172.21.15.62:0/918771307 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be67c98eb0 con 0x55be67d63c20 2023-07-14T16:16:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.808+0000 7f866539b700 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55be67d63c20 0x55be67d66030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.809+0000 7f866539b700 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55be67d63c20 0x55be67d66030 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f2ac36f6460fb8a server_cookie=c0d874f1026c76f7 in_seq=0 out_seq=0 2023-07-14T16:16:57.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.809+0000 7f8648ff9700 10 client.15840 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f8648ff9700 1 -- 172.21.15.62:0/918771307 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55be67c98eb0 con 0x55be67d63c20 2023-07-14T16:16:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f8648ff9700 10 client.15840 handle_command_reply: tid=0 2023-07-14T16:16:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 client.15840 shutdown 2023-07-14T16:16:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 -- 172.21.15.62:0/918771307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 msgr2=0x55be67d5f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5f320 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8658007c30 tx=0x7f8658007f40 comp rx=0 tx=0).stop 2023-07-14T16:16:57.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 -- 172.21.15.62:0/918771307 shutdown_connections 2023-07-14T16:16:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55be67d63c20 0x55be67d66030 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5f320 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d57da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f8648ff9700 0 client.15840 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 --2- 172.21.15.62:0/918771307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 -- 172.21.15.62:0/918771307 >> 172.21.15.62:0/918771307 conn(0x55be67d43da0 msgr2=0x55be67d488b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 -- 172.21.15.62:0/918771307 shutdown_connections 2023-07-14T16:16:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.810+0000 7f867aef1b80 1 -- 172.21.15.62:0/918771307 wait complete. 2023-07-14T16:16:57.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.811+0000 7f867aef1b80 20 client.15840 trim_cache size 0 max 16384 2023-07-14T16:16:57.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.825+0000 7f867aef1b80 1 Processor -- start 2023-07-14T16:16:57.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.825+0000 7f867aef1b80 1 -- start start 2023-07-14T16:16:57.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.825+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d66030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.825+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.825+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d55fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.825+0000 7f867aef1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67d4a4c0 con 0x55be67d4fc70 2023-07-14T16:16:57.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.825+0000 7f867aef1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67d57250 con 0x55be67d4ee20 2023-07-14T16:16:57.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.825+0000 7f867aef1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67d57910 con 0x55be67d4e4d0 2023-07-14T16:16:57.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f8665b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d55fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f866539b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d66030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f866539b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d66030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:58790/0 (socket says 172.21.15.62:58790) 2023-07-14T16:16:57.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f866539b700 1 -- 172.21.15.62:0/1870055134 learned_addr learned my addr 172.21.15.62:0/1870055134 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f8664b9a700 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f866539b700 1 -- 172.21.15.62:0/1870055134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 msgr2=0x55be67d5f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f866539b700 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f866539b700 1 -- 172.21.15.62:0/1870055134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 msgr2=0x55be67d55fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f866539b700 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d55fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f866539b700 1 -- 172.21.15.62:0/1870055134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be67ab6c60 con 0x55be67d4e4d0 2023-07-14T16:16:57.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f8664b9a700 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:57.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.826+0000 7f8665b9c700 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d55fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:57.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.827+0000 7f866539b700 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d66030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f865c00ae00 tx=0x7f865c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf6480a160e33868 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.827+0000 7f864bfff700 1 -- 172.21.15.62:0/1870055134 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c022040 con 0x55be67d4e4d0 2023-07-14T16:16:57.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.827+0000 7f864bfff700 1 -- 172.21.15.62:0/1870055134 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f865c014070 con 0x55be67d4e4d0 2023-07-14T16:16:57.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.827+0000 7f864bfff700 1 -- 172.21.15.62:0/1870055134 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c010690 con 0x55be67d4e4d0 2023-07-14T16:16:57.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 -- 172.21.15.62:0/1870055134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 msgr2=0x55be67d66030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d66030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f865c00ae00 tx=0x7f865c012000 comp rx=0 tx=0).stop 2023-07-14T16:16:57.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 -- 172.21.15.62:0/1870055134 shutdown_connections 2023-07-14T16:16:57.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d55fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d66030 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 --2- 172.21.15.62:0/1870055134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 -- 172.21.15.62:0/1870055134 >> 172.21.15.62:0/1870055134 conn(0x55be67d43da0 msgr2=0x55be67d6a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 -- 172.21.15.62:0/1870055134 shutdown_connections 2023-07-14T16:16:57.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 -- 172.21.15.62:0/1870055134 wait complete. 2023-07-14T16:16:57.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.828+0000 7f867aef1b80 1 -- start start 2023-07-14T16:16:57.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f867aef1b80 1 Processor -- start 2023-07-14T16:16:57.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d5b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f867aef1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f867aef1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67d5ee40 con 0x55be67d4fc70 2023-07-14T16:16:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f867aef1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67d5efb0 con 0x55be67d4ee20 2023-07-14T16:16:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f867aef1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be67d5f170 con 0x55be67d4e4d0 2023-07-14T16:16:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f867aef1b80 10 client.0 _ll_register_callbacks cb 0x55be67c3d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f8665b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f8664b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f8665b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45782/0 (socket says 172.21.15.62:45782) 2023-07-14T16:16:57.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f8665b9c700 1 -- 172.21.15.62:0/3729937501 learned_addr learned my addr 172.21.15.62:0/3729937501 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:57.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.829+0000 7f866539b700 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d5b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8664b9a700 1 -- 172.21.15.62:0/3729937501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 msgr2=0x55be67d5b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8664b9a700 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d5b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8664b9a700 1 -- 172.21.15.62:0/3729937501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 msgr2=0x55be67d5e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:57.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8664b9a700 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8664b9a700 1 -- 172.21.15.62:0/3729937501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be67acdde0 con 0x55be67d4ee20 2023-07-14T16:16:57.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8664b9a700 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5b8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f865009f5c0 tx=0x7f8650009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ceea821dd29f3f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:57.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8648ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:57.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8648ff9700 1 -- 172.21.15.62:0/3729937501 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650003f40 con 0x55be67d4ee20 2023-07-14T16:16:57.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f867aef1b80 1 -- 172.21.15.62:0/3729937501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86500860b0 con 0x55be67d4ee20 2023-07-14T16:16:57.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8648ff9700 1 -- 172.21.15.62:0/3729937501 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8650022720 con 0x55be67d4ee20 2023-07-14T16:16:57.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f8648ff9700 1 -- 172.21.15.62:0/3729937501 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f865009b3d0 con 0x55be67d4ee20 2023-07-14T16:16:57.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f867aef1b80 10 client.25610 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:57.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.830+0000 7f867aef1b80 1 -- 172.21.15.62:0/3729937501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f866019fdf0 con 0x55be67d4ee20 2023-07-14T16:16:57.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f8648ff9700 1 -- 172.21.15.62:0/3729937501 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86500860b0 con 0x55be67d4ee20 2023-07-14T16:16:57.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f8648ff9700 1 -- 172.21.15.62:0/3729937501 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86500b3d20 con 0x55be67d4ee20 2023-07-14T16:16:57.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f867aef1b80 10 client.25610 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:57.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f867aef1b80 10 client.25610 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:16:57.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f867aef1b80 20 client.25610 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:57.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f867aef1b80 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55be67db8aa0 0x55be67dbaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:57.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f867aef1b80 4 client.25610 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:57.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f867aef1b80 1 -- 172.21.15.62:0/3729937501 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be67c5b050 con 0x55be67db8aa0 2023-07-14T16:16:57.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.831+0000 7f866539b700 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55be67db8aa0 0x55be67dbaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:57.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f866539b700 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55be67db8aa0 0x55be67dbaeb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3626263d2ccb42a server_cookie=53a17e1dd0392b0e in_seq=0 out_seq=0 2023-07-14T16:16:57.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f8648ff9700 10 client.25610 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:57.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f8648ff9700 1 -- 172.21.15.62:0/3729937501 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be67c5b050 con 0x55be67db8aa0 2023-07-14T16:16:57.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f8648ff9700 10 client.25610 handle_command_reply: tid=0 2023-07-14T16:16:57.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 client.25610 shutdown 2023-07-14T16:16:57.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 -- 172.21.15.62:0/3729937501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 msgr2=0x55be67d5b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5b8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f865009f5c0 tx=0x7f8650009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 -- 172.21.15.62:0/3729937501 shutdown_connections 2023-07-14T16:16:57.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55be67db8aa0 0x55be67dbaeb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be67d4fc70 0x55be67d5e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be67d4e4d0 0x55be67d5b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 --2- 172.21.15.62:0/3729937501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be67d4ee20 0x55be67d5b8e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f8648ff9700 0 client.25610 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:16:57.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.832+0000 7f867aef1b80 1 -- 172.21.15.62:0/3729937501 >> 172.21.15.62:0/3729937501 conn(0x55be67d43da0 msgr2=0x55be67d55fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f867aef1b80 1 -- 172.21.15.62:0/3729937501 shutdown_connections 2023-07-14T16:16:57.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f867aef1b80 1 -- 172.21.15.62:0/3729937501 wait complete. 2023-07-14T16:16:57.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f867aef1b80 20 client.25610 trim_cache size 0 max 16384 2023-07-14T16:16:57.819 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:57.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f86675ff700 1 -- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f864c0a1150 msgr2=0x7f864c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f86675ff700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f864c0a1150 0x7f864c0a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f865c00afd0 tx=0x7f865c00a380 comp rx=0 tx=0).stop 2023-07-14T16:16:57.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f86675ff700 1 -- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 msgr2=0x7f86600fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:57.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f86675ff700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 0x7f86600fe510 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8650000c00 tx=0x7f8650003a50 comp rx=0 tx=0).stop 2023-07-14T16:16:57.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f86675ff700 1 -- 172.21.15.62:0/1265850010 shutdown_connections 2023-07-14T16:16:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.833+0000 7f86675ff700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86601057a0 0x7f86600fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.834+0000 7f86675ff700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f864c0a1150 0x7f864c0a3600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.834+0000 7f86675ff700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8660100560 0x7f86600ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.834+0000 7f86675ff700 1 --2- 172.21.15.62:0/1265850010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8660102e80 0x7f86600fe510 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.834+0000 7f86675ff700 1 -- 172.21.15.62:0/1265850010 >> 172.21.15.62:0/1265850010 conn(0x7f86600fa000 msgr2=0x7f86601011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.834+0000 7f86675ff700 1 -- 172.21.15.62:0/1265850010 shutdown_connections 2023-07-14T16:16:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:57.834+0000 7f86675ff700 1 -- 172.21.15.62:0/1265850010 wait complete. 2023-07-14T16:16:57.828 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:57 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/4198848565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:57.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:16:57.842 INFO:tasks.workunit.client.0.smithi062.stdout:checking for inttypes.h... yes 2023-07-14T16:16:57.968 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdint.h... yes 2023-07-14T16:16:58.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:57 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/4198848565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.092+0000 7fabdd5ff700 1 -- 172.21.15.62:0/1422011248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd8074550 msgr2=0x7fabd80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.092+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/1422011248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd8074550 0x7fabd80729a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009990 tx=0x7fabc8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.092+0000 7fabdd5ff700 1 -- 172.21.15.62:0/1422011248 shutdown_connections 2023-07-14T16:16:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.092+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/1422011248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd810c0b0 0x7fabd810e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.093+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/1422011248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd8074550 0x7fabd80729a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.093+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/1422011248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd8072f70 0x7fabd80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.093+0000 7fabdd5ff700 1 -- 172.21.15.62:0/1422011248 >> 172.21.15.62:0/1422011248 conn(0x7fabd80fb0e0 msgr2=0x7fabd80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.093+0000 7fabdd5ff700 1 -- 172.21.15.62:0/1422011248 shutdown_connections 2023-07-14T16:16:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.093+0000 7fabdd5ff700 1 -- 172.21.15.62:0/1422011248 wait complete. 2023-07-14T16:16:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.094+0000 7fabdd5ff700 1 Processor -- start 2023-07-14T16:16:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.094+0000 7fabdd5ff700 1 -- start start 2023-07-14T16:16:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.094+0000 7fabdd5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd8072f70 0x7fabd819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.094+0000 7fabdd5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8074550 0x7fabd819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.094+0000 7fabdd5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd810c0b0 0x7fabd81a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.094+0000 7fabdd5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd81a3990 con 0x7fabd810c0b0 2023-07-14T16:16:58.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.094+0000 7fabdd5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd81a3ad0 con 0x7fabd8074550 2023-07-14T16:16:58.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.094+0000 7fabdd5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd81a3c60 con 0x7fabd8072f70 2023-07-14T16:16:58.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd810c0b0 0x7fabd81a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd6ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd8072f70 0x7fabd819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd810c0b0 0x7fabd81a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45788/0 (socket says 172.21.15.62:45788) 2023-07-14T16:16:58.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 -- 172.21.15.62:0/2803650357 learned_addr learned my addr 172.21.15.62:0/2803650357 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:58.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd67fc700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8074550 0x7fabd819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 -- 172.21.15.62:0/2803650357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd8072f70 msgr2=0x7fabd819be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd8072f70 0x7fabd819be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 -- 172.21.15.62:0/2803650357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8074550 msgr2=0x7fabd819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8074550 0x7fabd819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 -- 172.21.15.62:0/2803650357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80095f0 con 0x7fabd810c0b0 2023-07-14T16:16:58.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.095+0000 7fabd77fe700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd810c0b0 0x7fabd81a33c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fabd0007ee0 tx=0x7fabd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:58.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.096+0000 7fabbffff700 1 -- 172.21.15.62:0/2803650357 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000dd20 con 0x7fabd810c0b0 2023-07-14T16:16:58.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.096+0000 7fabbffff700 1 -- 172.21.15.62:0/2803650357 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabd000f040 con 0x7fabd810c0b0 2023-07-14T16:16:58.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.096+0000 7fabbffff700 1 -- 172.21.15.62:0/2803650357 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0004750 con 0x7fabd810c0b0 2023-07-14T16:16:58.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.096+0000 7fabdd5ff700 1 -- 172.21.15.62:0/2803650357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd81a3f40 con 0x7fabd810c0b0 2023-07-14T16:16:58.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.096+0000 7fabdd5ff700 1 -- 172.21.15.62:0/2803650357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd81a4370 con 0x7fabd810c0b0 2023-07-14T16:16:58.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.101+0000 7fabbffff700 1 -- 172.21.15.62:0/2803650357 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fabd0003a00 con 0x7fabd810c0b0 2023-07-14T16:16:58.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabf0f61b80 1 Processor -- start 2023-07-14T16:16:58.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabf0f61b80 1 -- start start 2023-07-14T16:16:58.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabbffff700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fabc00a1150 0x7fabc00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf57c70 0x55761cf580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761ce5bcb0 con 0x55761cf564d0 2023-07-14T16:16:58.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761ceaea40 con 0x55761cf57c70 2023-07-14T16:16:58.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.102+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761cea7ed0 con 0x55761cf56e20 2023-07-14T16:16:58.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabbffff700 1 -- 172.21.15.62:0/2803650357 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fabd00cc170 con 0x7fabd810c0b0 2023-07-14T16:16:58.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd67fc700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fabc00a1150 0x7fabc00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45794/0 (socket says 172.21.15.62:45794) 2023-07-14T16:16:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd6ffd700 1 -- 172.21.15.62:0/2107530351 learned_addr learned my addr 172.21.15.62:0/2107530351 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd67fc700 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd77fe700 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf57c70 0x55761cf580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd6ffd700 1 -- 172.21.15.62:0/2107530351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 msgr2=0x55761cf576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:58.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd6ffd700 1 -- 172.21.15.62:0/2107530351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf57c70 msgr2=0x55761cf580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:58.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf57c70 0x55761cf580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd6ffd700 1 -- 172.21.15.62:0/2107530351 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55761ccd5de0 con 0x55761cf564d0 2023-07-14T16:16:58.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.103+0000 7fabd77fe700 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf57c70 0x55761cf580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.104+0000 7fabd67fc700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fabc00a1150 0x7fabc00a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fabd8074270 tx=0x7fabcc009bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.104+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf568e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fabd8074920 tx=0x7fabc8005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6681e1d9981f46c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.104+0000 7fabbd7fa700 1 -- 172.21.15.62:0/2107530351 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc801d070 con 0x55761cf564d0 2023-07-14T16:16:58.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.104+0000 7fabbd7fa700 1 -- 172.21.15.62:0/2107530351 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabc801e040 con 0x55761cf564d0 2023-07-14T16:16:58.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.104+0000 7fabbd7fa700 1 -- 172.21.15.62:0/2107530351 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800b710 con 0x55761cf564d0 2023-07-14T16:16:58.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2107530351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 msgr2=0x55761cf568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf568e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fabd8074920 tx=0x7fabc8005f00 comp rx=0 tx=0).stop 2023-07-14T16:16:58.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2107530351 shutdown_connections 2023-07-14T16:16:58.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf57c70 0x55761cf580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf568e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2107530351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2107530351 >> 172.21.15.62:0/2107530351 conn(0x55761cf4bda0 msgr2=0x55761cf511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:58.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2107530351 shutdown_connections 2023-07-14T16:16:58.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.105+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2107530351 wait complete. 2023-07-14T16:16:58.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.106+0000 7fabf0f61b80 1 -- start start 2023-07-14T16:16:58.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.106+0000 7fabf0f61b80 1 Processor -- start 2023-07-14T16:16:58.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf67630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761ce47ac0 con 0x55761cf57c70 2023-07-14T16:16:58.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761ce477c0 con 0x55761cf564d0 2023-07-14T16:16:58.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761cd2b920 con 0x55761cf56e20 2023-07-14T16:16:58.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabf0f61b80 10 client.0 _ll_register_callbacks cb 0x55761ce45760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:58.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43688/0 (socket says 172.21.15.62:43688) 2023-07-14T16:16:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd6ffd700 1 -- 172.21.15.62:0/2989202371 learned_addr learned my addr 172.21.15.62:0/2989202371 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd77fe700 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf67630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd67fc700 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd6ffd700 1 -- 172.21.15.62:0/2989202371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 msgr2=0x55761cf605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd6ffd700 1 -- 172.21.15.62:0/2989202371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 msgr2=0x55761cf67630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf67630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd6ffd700 1 -- 172.21.15.62:0/2989202371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd81a4370 con 0x55761cf564d0 2023-07-14T16:16:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.107+0000 7fabd67fc700 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.108+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf600b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabc8000c00 tx=0x7fabc80050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2167b293385b7d0e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.108+0000 7fabbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.108+0000 7fabbaff5700 1 -- 172.21.15.62:0/2989202371 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc801d070 con 0x55761cf564d0 2023-07-14T16:16:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.108+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2989202371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55761ce014f0 con 0x55761cf564d0 2023-07-14T16:16:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.108+0000 7fabbaff5700 1 -- 172.21.15.62:0/2989202371 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fabc800b910 con 0x55761cf564d0 2023-07-14T16:16:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabbaff5700 1 -- 172.21.15.62:0/2989202371 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabc800bad0 con 0x55761cf564d0 2023-07-14T16:16:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabf0f61b80 10 client.25622 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2989202371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fabd810ee40 con 0x55761cf564d0 2023-07-14T16:16:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabbaff5700 1 -- 172.21.15.62:0/2989202371 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55761ce014f0 con 0x55761cf564d0 2023-07-14T16:16:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabbaff5700 1 -- 172.21.15.62:0/2989202371 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fabc8004c50 con 0x55761cf564d0 2023-07-14T16:16:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabf0f61b80 10 client.25622 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabf0f61b80 10 client.25622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabf0f61b80 20 client.25622 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55761cf6b9b0 0x55761cf6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabf0f61b80 4 client.25622 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.109+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2989202371 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55761cea0eb0 con 0x55761cf6b9b0 2023-07-14T16:16:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.110+0000 7fabd67fc700 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55761cf6b9b0 0x55761cf6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.110+0000 7fabd67fc700 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55761cf6b9b0 0x55761cf6ddc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91d34843c51010c8 server_cookie=488bec6e126ff2cf in_seq=0 out_seq=0 2023-07-14T16:16:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.110+0000 7fabbaff5700 10 client.25622 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabbaff5700 1 -- 172.21.15.62:0/2989202371 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55761cea0eb0 con 0x55761cf6b9b0 2023-07-14T16:16:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabbaff5700 10 client.25622 handle_command_reply: tid=0 2023-07-14T16:16:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 client.25622 shutdown 2023-07-14T16:16:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2989202371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 msgr2=0x55761cf600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf600b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabc8000c00 tx=0x7fabc80050e0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2989202371 shutdown_connections 2023-07-14T16:16:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf67630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55761cf6b9b0 0x55761cf6ddc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf600b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/2989202371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabbaff5700 0 client.25622 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2989202371 >> 172.21.15.62:0/2989202371 conn(0x55761cf4bda0 msgr2=0x55761cf508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2989202371 shutdown_connections 2023-07-14T16:16:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 1 -- 172.21.15.62:0/2989202371 wait complete. 2023-07-14T16:16:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.111+0000 7fabf0f61b80 20 client.25622 trim_cache size 0 max 16384 2023-07-14T16:16:58.103 INFO:tasks.workunit.client.0.smithi062.stdout:checking for unistd.h... yes 2023-07-14T16:16:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabf0f61b80 1 Processor -- start 2023-07-14T16:16:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabf0f61b80 1 -- start start 2023-07-14T16:16:58.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf5f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf72700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761cf72c40 con 0x55761cf57c70 2023-07-14T16:16:58.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761cf56af0 con 0x55761cf564d0 2023-07-14T16:16:58.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761cf56cb0 con 0x55761cf56e20 2023-07-14T16:16:58.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabd77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf72700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabd77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf72700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45808/0 (socket says 172.21.15.62:45808) 2023-07-14T16:16:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabd77fe700 1 -- 172.21.15.62:0/3576149606 learned_addr learned my addr 172.21.15.62:0/3576149606 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.127+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf5f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabd67fc700 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabd6ffd700 1 -- 172.21.15.62:0/3576149606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 msgr2=0x55761cf675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabd6ffd700 1 -- 172.21.15.62:0/3576149606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 msgr2=0x55761cf72700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf72700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabd6ffd700 1 -- 172.21.15.62:0/3576149606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55761ccbec60 con 0x55761cf564d0 2023-07-14T16:16:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabd77fe700 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf72700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf5f760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fabc80098e0 tx=0x7fabc8003990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d73bd4edd544c410 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabbdffb700 1 -- 172.21.15.62:0/3576149606 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc801d070 con 0x55761cf564d0 2023-07-14T16:16:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabbdffb700 1 -- 172.21.15.62:0/3576149606 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabc8005200 con 0x55761cf564d0 2023-07-14T16:16:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.128+0000 7fabbdffb700 1 -- 172.21.15.62:0/3576149606 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800bb00 con 0x55761cf564d0 2023-07-14T16:16:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 -- 172.21.15.62:0/3576149606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 msgr2=0x55761cf5f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf5f760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fabc80098e0 tx=0x7fabc8003990 comp rx=0 tx=0).stop 2023-07-14T16:16:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 -- 172.21.15.62:0/3576149606 shutdown_connections 2023-07-14T16:16:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf57c70 0x55761cf72700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf564d0 0x55761cf5f760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/3576149606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf56e20 0x55761cf675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 -- 172.21.15.62:0/3576149606 >> 172.21.15.62:0/3576149606 conn(0x55761cf4bda0 msgr2=0x55761cf5ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 -- 172.21.15.62:0/3576149606 shutdown_connections 2023-07-14T16:16:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.129+0000 7fabf0f61b80 1 -- 172.21.15.62:0/3576149606 wait complete. 2023-07-14T16:16:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 1 -- start start 2023-07-14T16:16:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 1 Processor -- start 2023-07-14T16:16:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf63440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf56e20 0x55761cf63980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf57c70 0x55761cf669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761cf66ee0 con 0x55761cf564d0 2023-07-14T16:16:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761cf67050 con 0x55761cf56e20 2023-07-14T16:16:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761cf67210 con 0x55761cf57c70 2023-07-14T16:16:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.130+0000 7fabf0f61b80 10 client.0 _ll_register_callbacks cb 0x7fabd00951b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf56e20 0x55761cf63980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf56e20 0x55761cf63980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43712/0 (socket says 172.21.15.62:43712) 2023-07-14T16:16:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf63440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd67fc700 1 -- 172.21.15.62:0/1930801520 learned_addr learned my addr 172.21.15.62:0/1930801520 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd77fe700 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf57c70 0x55761cf669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd67fc700 1 -- 172.21.15.62:0/1930801520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf57c70 msgr2=0x55761cf669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd67fc700 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf57c70 0x55761cf669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd67fc700 1 -- 172.21.15.62:0/1930801520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 msgr2=0x55761cf63440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd67fc700 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf63440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd67fc700 1 -- 172.21.15.62:0/1930801520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd81a3f40 con 0x55761cf56e20 2023-07-14T16:16:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.131+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf63440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.132+0000 7fabd67fc700 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf56e20 0x55761cf63980 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55761cf6e6b0 tx=0x7fabcc01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad209e5e3c7a9ff2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.132+0000 7fabbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.132+0000 7fabbaff5700 1 -- 172.21.15.62:0/1930801520 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc0295b0 con 0x55761cf56e20 2023-07-14T16:16:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.132+0000 7fabbaff5700 1 -- 172.21.15.62:0/1930801520 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fabcc029970 con 0x55761cf56e20 2023-07-14T16:16:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.132+0000 7fabf0f61b80 1 -- 172.21.15.62:0/1930801520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55761ce63050 con 0x55761cf56e20 2023-07-14T16:16:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.132+0000 7fabbaff5700 1 -- 172.21.15.62:0/1930801520 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabcc029b30 con 0x55761cf56e20 2023-07-14T16:16:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.132+0000 7fabf0f61b80 10 client.25628 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.132+0000 7fabf0f61b80 1 -- 172.21.15.62:0/1930801520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fabd810ee40 con 0x55761cf56e20 2023-07-14T16:16:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabbaff5700 1 -- 172.21.15.62:0/1930801520 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55761ce63050 con 0x55761cf56e20 2023-07-14T16:16:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabbaff5700 1 -- 172.21.15.62:0/1930801520 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fabcc00f830 con 0x55761cf56e20 2023-07-14T16:16:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabf0f61b80 10 client.25628 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabf0f61b80 10 client.25628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:58.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabf0f61b80 20 client.25628 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:58.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55761cfc08f0 0x55761cfc2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:58.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabf0f61b80 4 client.25628 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:16:58.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabf0f61b80 1 -- 172.21.15.62:0/1930801520 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55761ccf6f70 con 0x55761cfc08f0 2023-07-14T16:16:58.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.133+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55761cfc08f0 0x55761cfc2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:58.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabd6ffd700 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55761cfc08f0 0x55761cfc2d00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95628ffe9e6496c server_cookie=fc57b63fd3a1437a in_seq=0 out_seq=0 2023-07-14T16:16:58.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabbaff5700 10 client.25628 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:58.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabbaff5700 1 -- 172.21.15.62:0/1930801520 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55761ccf6f70 con 0x55761cfc08f0 2023-07-14T16:16:58.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabbaff5700 10 client.25628 handle_command_reply: tid=0 2023-07-14T16:16:58.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 client.25628 shutdown 2023-07-14T16:16:58.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 -- 172.21.15.62:0/1930801520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf56e20 msgr2=0x55761cf63980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf56e20 0x55761cf63980 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55761cf6e6b0 tx=0x7fabcc01d000 comp rx=0 tx=0).stop 2023-07-14T16:16:58.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 -- 172.21.15.62:0/1930801520 shutdown_connections 2023-07-14T16:16:58.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55761cf57c70 0x55761cf669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55761cf564d0 0x55761cf63440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55761cfc08f0 0x55761cfc2d00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 --2- 172.21.15.62:0/1930801520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55761cf56e20 0x55761cf63980 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabbaff5700 0 client.25628 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:58.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.134+0000 7fabf0f61b80 1 -- 172.21.15.62:0/1930801520 >> 172.21.15.62:0/1930801520 conn(0x55761cf4bda0 msgr2=0x55761cf5dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:58.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabf0f61b80 1 -- 172.21.15.62:0/1930801520 shutdown_connections 2023-07-14T16:16:58.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabf0f61b80 1 -- 172.21.15.62:0/1930801520 wait complete. 2023-07-14T16:16:58.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabf0f61b80 20 client.25628 trim_cache size 0 max 16384 2023-07-14T16:16:58.121 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:16:58.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 -- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fabc00a1150 msgr2=0x7fabc00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fabc00a1150 0x7fabc00a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fabd8074270 tx=0x7fabcc009bf0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 -- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd810c0b0 msgr2=0x7fabd81a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:58.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd810c0b0 0x7fabd81a33c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fabd0007ee0 tx=0x7fabd000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 -- 172.21.15.62:0/2803650357 shutdown_connections 2023-07-14T16:16:58.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabd810c0b0 0x7fabd81a33c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabd8072f70 0x7fabd819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fabc00a1150 0x7fabc00a3600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 --2- 172.21.15.62:0/2803650357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8074550 0x7fabd819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:58.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 -- 172.21.15.62:0/2803650357 >> 172.21.15.62:0/2803650357 conn(0x7fabd80fb0e0 msgr2=0x7fabd80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:58.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.135+0000 7fabdd5ff700 1 -- 172.21.15.62:0/2803650357 shutdown_connections 2023-07-14T16:16:58.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:58.136+0000 7fabdd5ff700 1 -- 172.21.15.62:0/2803650357 wait complete. 2023-07-14T16:16:58.160 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:57 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/4198848565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:16:58.249 INFO:tasks.workunit.client.0.smithi062.stdout:checking pthread.h usability... yes 2023-07-14T16:16:58.299 INFO:tasks.workunit.client.0.smithi062.stdout:checking pthread.h presence... yes 2023-07-14T16:16:58.301 INFO:tasks.workunit.client.0.smithi062.stdout:checking for pthread.h... yes 2023-07-14T16:16:58.427 INFO:tasks.workunit.client.0.smithi062.stdout:checking fcntl.h usability... yes 2023-07-14T16:16:58.478 INFO:tasks.workunit.client.0.smithi062.stdout:checking fcntl.h presence... yes 2023-07-14T16:16:58.479 INFO:tasks.workunit.client.0.smithi062.stdout:checking for fcntl.h... yes 2023-07-14T16:16:58.605 INFO:tasks.workunit.client.0.smithi062.stdout:checking limits.h usability... yes 2023-07-14T16:16:58.654 INFO:tasks.workunit.client.0.smithi062.stdout:checking limits.h presence... yes 2023-07-14T16:16:58.655 INFO:tasks.workunit.client.0.smithi062.stdout:checking for limits.h... yes 2023-07-14T16:16:58.701 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdint.h... (cached) yes 2023-07-14T16:16:58.824 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/time.h usability... yes 2023-07-14T16:16:58.874 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/time.h presence... yes 2023-07-14T16:16:58.876 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/time.h... yes 2023-07-14T16:16:58.923 INFO:tasks.workunit.client.0.smithi062.stdout:checking for unistd.h... (cached) yes 2023-07-14T16:16:59.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:16:58 smithi164 ceph-mon[97427]: pgmap v397: 97 pgs: 97 active+clean; 3.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 227 KiB/s wr, 67 op/s 2023-07-14T16:16:59.035 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/vfs.h usability... yes 2023-07-14T16:16:59.085 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/vfs.h presence... yes 2023-07-14T16:16:59.086 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/vfs.h... yes 2023-07-14T16:16:59.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:16:58 smithi062 ceph-mon[95543]: pgmap v397: 97 pgs: 97 active+clean; 3.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 227 KiB/s wr, 67 op/s 2023-07-14T16:16:59.133 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:16:59.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:16:59.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:16:58 smithi081 ceph-mon[127795]: pgmap v397: 97 pgs: 97 active+clean; 3.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 227 KiB/s wr, 67 op/s 2023-07-14T16:16:59.204 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/limits.h usability... no 2023-07-14T16:16:59.261 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/limits.h presence... no 2023-07-14T16:16:59.263 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/limits.h... no 2023-07-14T16:16:59.350 INFO:tasks.workunit.client.0.smithi062.stdout:checking for an ANSI C-conforming const... yes 2023-07-14T16:16:59.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.385+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3219945832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8280ffbd0 msgr2=0x7fb828100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.385+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3219945832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8280ffbd0 0x7fb828100020 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009990 tx=0x7fb81c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.386+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3219945832 shutdown_connections 2023-07-14T16:16:59.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.386+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3219945832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb828100560 0x7fb828106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.386+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3219945832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8280fe9e0 0x7fb8280fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.386+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3219945832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8280ffbd0 0x7fb828100020 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.386+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3219945832 >> 172.21.15.62:0/3219945832 conn(0x7fb8280fa000 msgr2=0x7fb8280fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.386+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3219945832 shutdown_connections 2023-07-14T16:16:59.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.386+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3219945832 wait complete. 2023-07-14T16:16:59.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.387+0000 7fb82d5ff700 1 Processor -- start 2023-07-14T16:16:59.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.387+0000 7fb82d5ff700 1 -- start start 2023-07-14T16:16:59.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.387+0000 7fb82d5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8280fe9e0 0x7fb8281979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.387+0000 7fb82d5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8280ffbd0 0x7fb828197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.387+0000 7fb82d5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb828100560 0x7fb82819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.387+0000 7fb82d5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819f540 con 0x7fb828100560 2023-07-14T16:16:59.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.387+0000 7fb82d5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819f680 con 0x7fb8280fe9e0 2023-07-14T16:16:59.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.387+0000 7fb82d5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819f810 con 0x7fb8280ffbd0 2023-07-14T16:16:59.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8280fe9e0 0x7fb8281979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb8277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb828100560 0x7fb82819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8280fe9e0 0x7fb8281979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43728/0 (socket says 172.21.15.62:43728) 2023-07-14T16:16:59.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 -- 172.21.15.62:0/3279756835 learned_addr learned my addr 172.21.15.62:0/3279756835 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb8267fc700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8280ffbd0 0x7fb828197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 -- 172.21.15.62:0/3279756835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8280ffbd0 msgr2=0x7fb828197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8280ffbd0 0x7fb828197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 -- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb828100560 msgr2=0x7fb82819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb828100560 0x7fb82819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 -- 172.21.15.62:0/3279756835 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0095f0 con 0x7fb8280fe9e0 2023-07-14T16:16:59.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb8277fe700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb828100560 0x7fb82819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:59.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.388+0000 7fb826ffd700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8280fe9e0 0x7fb8281979c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb81800d980 tx=0x7fb81800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.389+0000 7fb80ffff700 1 -- 172.21.15.62:0/3279756835 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81800f890 con 0x7fb8280fe9e0 2023-07-14T16:16:59.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.389+0000 7fb80ffff700 1 -- 172.21.15.62:0/3279756835 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb81800fe80 con 0x7fb8280fe9e0 2023-07-14T16:16:59.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.389+0000 7fb80ffff700 1 -- 172.21.15.62:0/3279756835 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818019360 con 0x7fb8280fe9e0 2023-07-14T16:16:59.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.389+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3279756835 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82819faf0 con 0x7fb8280fe9e0 2023-07-14T16:16:59.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.389+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3279756835 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82819ffb0 con 0x7fb8280fe9e0 2023-07-14T16:16:59.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.392+0000 7fb80ffff700 1 -- 172.21.15.62:0/3279756835 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb81800f9f0 con 0x7fb8280fe9e0 2023-07-14T16:16:59.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.393+0000 7fb80ffff700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8100a11d0 0x7fb8100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.393+0000 7fb840e81b80 1 Processor -- start 2023-07-14T16:16:59.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb80ffff700 1 -- 172.21.15.62:0/3279756835 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8180cc2f0 con 0x7fb8280fe9e0 2023-07-14T16:16:59.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb840e81b80 1 -- start start 2023-07-14T16:16:59.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb8267fc700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8100a11d0 0x7fb8100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecbb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb840e81b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cebbfd30 con 0x5649cecbaea0 2023-07-14T16:16:59.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb840e81b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cec12ac0 con 0x5649cecba550 2023-07-14T16:16:59.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.394+0000 7fb840e81b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cec0bf50 con 0x5649cecbbcf0 2023-07-14T16:16:59.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8277fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8277fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:58906/0 (socket says 172.21.15.62:58906) 2023-07-14T16:16:59.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8277fe700 1 -- 172.21.15.62:0/3713744507 learned_addr learned my addr 172.21.15.62:0/3713744507 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb826ffd700 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8267fc700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8100a11d0 0x7fb8100a3680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00c010 tx=0x7fb81c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8267fc700 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecbb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8277fe700 1 -- 172.21.15.62:0/3713744507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 msgr2=0x5649cecba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8277fe700 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecba960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8277fe700 1 -- 172.21.15.62:0/3713744507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 msgr2=0x5649cecbb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8277fe700 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecbb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb8277fe700 1 -- 172.21.15.62:0/3713744507 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649cea39e60 con 0x5649cecbbcf0 2023-07-14T16:16:59.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.395+0000 7fb826ffd700 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:59.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.396+0000 7fb8277fe700 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbc160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb820007740 tx=0x7fb820009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d60c583d30bc6654 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.396+0000 7fb80d7fa700 1 -- 172.21.15.62:0/3713744507 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820015850 con 0x5649cecbbcf0 2023-07-14T16:16:59.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.396+0000 7fb80d7fa700 1 -- 172.21.15.62:0/3713744507 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb82000f040 con 0x5649cecbbcf0 2023-07-14T16:16:59.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.396+0000 7fb80d7fa700 1 -- 172.21.15.62:0/3713744507 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820014d30 con 0x5649cecbbcf0 2023-07-14T16:16:59.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 -- 172.21.15.62:0/3713744507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 msgr2=0x5649cecbc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbc160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb820007740 tx=0x7fb820009f90 comp rx=0 tx=0).stop 2023-07-14T16:16:59.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 -- 172.21.15.62:0/3713744507 shutdown_connections 2023-07-14T16:16:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbc160 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 --2- 172.21.15.62:0/3713744507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecbb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 -- 172.21.15.62:0/3713744507 >> 172.21.15.62:0/3713744507 conn(0x5649cecafe20 msgr2=0x5649cecb5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 -- 172.21.15.62:0/3713744507 shutdown_connections 2023-07-14T16:16:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 -- 172.21.15.62:0/3713744507 wait complete. 2023-07-14T16:16:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.397+0000 7fb840e81b80 1 -- start start 2023-07-14T16:16:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb840e81b80 1 Processor -- start 2023-07-14T16:16:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 0x5649cecc4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecc46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbbcf0 0x5649ceccb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb840e81b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cebabb40 con 0x5649cecbaea0 2023-07-14T16:16:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb840e81b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cebab840 con 0x5649cecbbcf0 2023-07-14T16:16:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb840e81b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cea8f9a0 con 0x5649cecba550 2023-07-14T16:16:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb840e81b80 10 client.0 _ll_register_callbacks cb 0x5649ceba97e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.398+0000 7fb8267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecc46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecc46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45864/0 (socket says 172.21.15.62:45864) 2023-07-14T16:16:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8267fc700 1 -- 172.21.15.62:0/1014755991 learned_addr learned my addr 172.21.15.62:0/1014755991 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8277fe700 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbbcf0 0x5649ceccb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb826ffd700 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 0x5649cecc4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8277fe700 1 -- 172.21.15.62:0/1014755991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 msgr2=0x5649cecc4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8277fe700 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 0x5649cecc4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8277fe700 1 -- 172.21.15.62:0/1014755991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 msgr2=0x5649cecc46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8277fe700 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecc46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8277fe700 1 -- 172.21.15.62:0/1014755991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82819ffb0 con 0x5649cecbbcf0 2023-07-14T16:16:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8267fc700 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecc46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb8277fe700 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbbcf0 0x5649ceccb710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb820014940 tx=0x7fb820009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33abeb1fe907a6b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.399+0000 7fb80aff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.400+0000 7fb80aff5700 1 -- 172.21.15.62:0/1014755991 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820014f00 con 0x5649cecbbcf0 2023-07-14T16:16:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.400+0000 7fb840e81b80 1 -- 172.21.15.62:0/1014755991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649ceb65570 con 0x5649cecbbcf0 2023-07-14T16:16:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.400+0000 7fb80aff5700 1 -- 172.21.15.62:0/1014755991 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb820015650 con 0x5649cecbbcf0 2023-07-14T16:16:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.400+0000 7fb80aff5700 1 -- 172.21.15.62:0/1014755991 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb820015810 con 0x5649cecbbcf0 2023-07-14T16:16:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.400+0000 7fb840e81b80 10 client.25640 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.400+0000 7fb840e81b80 1 -- 172.21.15.62:0/1014755991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fb820009e00 con 0x5649cecbbcf0 2023-07-14T16:16:59.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.400+0000 7fb80aff5700 1 -- 172.21.15.62:0/1014755991 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5649ceb65570 con 0x5649cecbbcf0 2023-07-14T16:16:59.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.401+0000 7fb80aff5700 1 -- 172.21.15.62:0/1014755991 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb820031070 con 0x5649cecbbcf0 2023-07-14T16:16:59.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.401+0000 7fb840e81b80 10 client.25640 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:59.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.401+0000 7fb840e81b80 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:59.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.401+0000 7fb840e81b80 20 client.25640 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:59.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.401+0000 7fb840e81b80 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5649ceccfe20 0x5649cecd2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.401+0000 7fb840e81b80 4 client.25640 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.401+0000 7fb840e81b80 1 -- 172.21.15.62:0/1014755991 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649cec04f30 con 0x5649ceccfe20 2023-07-14T16:16:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.401+0000 7fb826ffd700 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5649ceccfe20 0x5649cecd2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.402+0000 7fb80aff5700 10 client.25640 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.402+0000 7fb826ffd700 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5649ceccfe20 0x5649cecd2230 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d228b94996c04cc server_cookie=2f2a95c5ecf450ea in_seq=0 out_seq=0 2023-07-14T16:16:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.402+0000 7fb80aff5700 1 -- 172.21.15.62:0/1014755991 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5649cec04f30 con 0x5649ceccfe20 2023-07-14T16:16:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.402+0000 7fb80aff5700 10 client.25640 handle_command_reply: tid=0 2023-07-14T16:16:59.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 client.25640 shutdown 2023-07-14T16:16:59.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 -- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbbcf0 msgr2=0x5649ceccb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbbcf0 0x5649ceccb710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb820014940 tx=0x7fb820009620 comp rx=0 tx=0).stop 2023-07-14T16:16:59.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 -- 172.21.15.62:0/1014755991 shutdown_connections 2023-07-14T16:16:59.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5649ceccfe20 0x5649cecd2230 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbbcf0 0x5649ceccb710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb80aff5700 0 client.25640 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:59.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 0x5649cecc4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 --2- 172.21.15.62:0/1014755991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649cecc46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 -- 172.21.15.62:0/1014755991 >> 172.21.15.62:0/1014755991 conn(0x5649cecafe20 msgr2=0x5649cecb4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 -- 172.21.15.62:0/1014755991 shutdown_connections 2023-07-14T16:16:59.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 1 -- 172.21.15.62:0/1014755991 wait complete. 2023-07-14T16:16:59.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.403+0000 7fb840e81b80 20 client.25640 trim_cache size 0 max 16384 2023-07-14T16:16:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.422+0000 7fb840e81b80 1 Processor -- start 2023-07-14T16:16:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.422+0000 7fb840e81b80 1 -- start start 2023-07-14T16:16:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecbeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649ceccb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb840e81b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cecc3d00 con 0x5649cecbaea0 2023-07-14T16:16:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb840e81b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cecb5f50 con 0x5649cecba550 2023-07-14T16:16:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb840e81b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cecb6a80 con 0x5649cecbbcf0 2023-07-14T16:16:59.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb8267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649ceccb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb826ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecbeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb8267fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649ceccb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45876/0 (socket says 172.21.15.62:45876) 2023-07-14T16:16:59.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb826ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecbeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43746/0 (socket says 172.21.15.62:43746) 2023-07-14T16:16:59.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb8267fc700 1 -- 172.21.15.62:0/4188496628 learned_addr learned my addr 172.21.15.62:0/4188496628 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb8277fe700 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb826ffd700 1 -- 172.21.15.62:0/4188496628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 msgr2=0x5649cecbf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb826ffd700 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb826ffd700 1 -- 172.21.15.62:0/4188496628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 msgr2=0x5649ceccb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb826ffd700 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649ceccb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb826ffd700 1 -- 172.21.15.62:0/4188496628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649cea22ce0 con 0x5649cecba550 2023-07-14T16:16:59.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.423+0000 7fb8277fe700 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:59.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.424+0000 7fb826ffd700 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecbeb50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb818099640 tx=0x7fb81801d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7e4e016870e58 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.424+0000 7fb80dffb700 1 -- 172.21.15.62:0/4188496628 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180b4020 con 0x5649cecba550 2023-07-14T16:16:59.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.424+0000 7fb80dffb700 1 -- 172.21.15.62:0/4188496628 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb818010460 con 0x5649cecba550 2023-07-14T16:16:59.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.424+0000 7fb80dffb700 1 -- 172.21.15.62:0/4188496628 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818097a50 con 0x5649cecba550 2023-07-14T16:16:59.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 -- 172.21.15.62:0/4188496628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 msgr2=0x5649cecbeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecbeb50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb818099640 tx=0x7fb81801d040 comp rx=0 tx=0).stop 2023-07-14T16:16:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 -- 172.21.15.62:0/4188496628 shutdown_connections 2023-07-14T16:16:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecbbcf0 0x5649cecbf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecba550 0x5649cecbeb50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 --2- 172.21.15.62:0/4188496628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbaea0 0x5649ceccb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 -- 172.21.15.62:0/4188496628 >> 172.21.15.62:0/4188496628 conn(0x5649cecafe20 msgr2=0x5649cecc2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 -- 172.21.15.62:0/4188496628 shutdown_connections 2023-07-14T16:16:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 -- 172.21.15.62:0/4188496628 wait complete. 2023-07-14T16:16:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 -- start start 2023-07-14T16:16:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.425+0000 7fb840e81b80 1 Processor -- start 2023-07-14T16:16:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 0x5649cecc87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbaea0 0x5649cecc8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb840e81b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbbcf0 0x5649cecc9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb840e81b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cecca280 con 0x5649cecbbcf0 2023-07-14T16:16:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb840e81b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cecca3f0 con 0x5649cecbaea0 2023-07-14T16:16:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb840e81b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649cecca5b0 con 0x5649cecba550 2023-07-14T16:16:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb840e81b80 10 client.0 _ll_register_callbacks cb 0x7fb818095330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:59.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb8277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbbcf0 0x5649cecc9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb8277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbbcf0 0x5649cecc9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45890/0 (socket says 172.21.15.62:45890) 2023-07-14T16:16:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb8267fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbaea0 0x5649cecc8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb8277fe700 1 -- 172.21.15.62:0/97681232 learned_addr learned my addr 172.21.15.62:0/97681232 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.426+0000 7fb826ffd700 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 0x5649cecc87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb8277fe700 1 -- 172.21.15.62:0/97681232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 msgr2=0x5649cecc87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb8277fe700 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 0x5649cecc87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb8277fe700 1 -- 172.21.15.62:0/97681232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbaea0 msgr2=0x5649cecc8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb8277fe700 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbaea0 0x5649cecc8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb8277fe700 1 -- 172.21.15.62:0/97681232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82819faf0 con 0x5649cecbbcf0 2023-07-14T16:16:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb8267fc700 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbaea0 0x5649cecc8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb8277fe700 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbbcf0 0x5649cecc9d40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb820002230 tx=0x7fb82001ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2d026ec3429326 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb80aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:16:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb80aff5700 1 -- 172.21.15.62:0/97681232 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820014970 con 0x5649cecbbcf0 2023-07-14T16:16:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb840e81b80 1 -- 172.21.15.62:0/97681232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb82000f040 con 0x5649cecbbcf0 2023-07-14T16:16:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.427+0000 7fb80aff5700 1 -- 172.21.15.62:0/97681232 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb82001ed20 con 0x5649cecbbcf0 2023-07-14T16:16:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb80aff5700 1 -- 172.21.15.62:0/97681232 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb82001e450 con 0x5649cecbbcf0 2023-07-14T16:16:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb840e81b80 10 client.15882 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb840e81b80 1 -- 172.21.15.62:0/97681232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5649cea602d0 con 0x5649cecbbcf0 2023-07-14T16:16:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb80aff5700 1 -- 172.21.15.62:0/97681232 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb82000f040 con 0x5649cecbbcf0 2023-07-14T16:16:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb80aff5700 1 -- 172.21.15.62:0/97681232 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb820007a50 con 0x5649cecbbcf0 2023-07-14T16:16:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb840e81b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb840e81b80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb840e81b80 20 client.15882 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb840e81b80 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5649ced25ce0 0x5649ced280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb840e81b80 4 client.15882 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:16:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.428+0000 7fb840e81b80 1 -- 172.21.15.62:0/97681232 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5649cebc70d0 con 0x5649ced25ce0 2023-07-14T16:16:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.429+0000 7fb826ffd700 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5649ced25ce0 0x5649ced280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.429+0000 7fb826ffd700 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5649ced25ce0 0x5649ced280c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466cfd0b20ff1504 server_cookie=44e94189c7f79b87 in_seq=0 out_seq=0 2023-07-14T16:16:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.429+0000 7fb80aff5700 10 client.15882 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb80aff5700 1 -- 172.21.15.62:0/97681232 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5649cebc70d0 con 0x5649ced25ce0 2023-07-14T16:16:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb80aff5700 10 client.15882 handle_command_reply: tid=0 2023-07-14T16:16:59.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 client.15882 shutdown 2023-07-14T16:16:59.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 -- 172.21.15.62:0/97681232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbbcf0 msgr2=0x5649cecc9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbbcf0 0x5649cecc9d40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb820002230 tx=0x7fb82001ebd0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 -- 172.21.15.62:0/97681232 shutdown_connections 2023-07-14T16:16:59.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649cecbbcf0 0x5649cecc9d40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5649cecba550 0x5649cecc87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5649ced25ce0 0x5649ced280c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 --2- 172.21.15.62:0/97681232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649cecbaea0 0x5649cecc8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 -- 172.21.15.62:0/97681232 >> 172.21.15.62:0/97681232 conn(0x5649cecafe20 msgr2=0x5649cecc2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb80aff5700 0 client.15882 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:59.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 -- 172.21.15.62:0/97681232 shutdown_connections 2023-07-14T16:16:59.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.430+0000 7fb840e81b80 1 -- 172.21.15.62:0/97681232 wait complete. 2023-07-14T16:16:59.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb840e81b80 20 client.15882 trim_cache size 0 max 16384 2023-07-14T16:16:59.429 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:16:59.429 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:16:59.429 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "8070e88b-f9b0-46e1-9288-6c62c620cbfc", 2023-07-14T16:16:59.429 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:16:59.429 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:16:59.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8100a11d0 msgr2=0x7fb8100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8100a11d0 0x7fb8100a3680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00c010 tx=0x7fb81c005fb0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3279756835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8280fe9e0 msgr2=0x7fb8281979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8280fe9e0 0x7fb8281979c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb81800d980 tx=0x7fb81800dc90 comp rx=0 tx=0).stop 2023-07-14T16:16:59.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3279756835 shutdown_connections 2023-07-14T16:16:59.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb828100560 0x7fb82819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8100a11d0 0x7fb8100a3680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8280fe9e0 0x7fb8281979c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 --2- 172.21.15.62:0/3279756835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8280ffbd0 0x7fb828197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3279756835 >> 172.21.15.62:0/3279756835 conn(0x7fb8280fa000 msgr2=0x7fb828067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3279756835 shutdown_connections 2023-07-14T16:16:59.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.431+0000 7fb82d5ff700 1 -- 172.21.15.62:0/3279756835 wait complete. 2023-07-14T16:16:59.432 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:16:59.450 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-07-14T16:16:59.455 INFO:tasks.workunit.client.0.smithi062.stdout:checking for function prototypes... yes 2023-07-14T16:16:59.561 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether setvbuf arguments are reversed... no 2023-07-14T16:16:59.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 -- 172.21.15.62:0/2768226797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c100ce0 msgr2=0x7f622c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 --2- 172.21.15.62:0/2768226797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c100ce0 0x7f622c101130 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f621c009990 tx=0x7f621c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 -- 172.21.15.62:0/2768226797 shutdown_connections 2023-07-14T16:16:59.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 --2- 172.21.15.62:0/2768226797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c101670 0x7f622c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 --2- 172.21.15.62:0/2768226797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c0ffaf0 0x7f622c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 --2- 172.21.15.62:0/2768226797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c100ce0 0x7f622c101130 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 -- 172.21.15.62:0/2768226797 >> 172.21.15.62:0/2768226797 conn(0x7f622c0fb130 msgr2=0x7f622c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 -- 172.21.15.62:0/2768226797 shutdown_connections 2023-07-14T16:16:59.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.681+0000 7f62343ff700 1 -- 172.21.15.62:0/2768226797 wait complete. 2023-07-14T16:16:59.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.683+0000 7f62343ff700 1 Processor -- start 2023-07-14T16:16:59.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.683+0000 7f62343ff700 1 -- start start 2023-07-14T16:16:59.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.683+0000 7f62343ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c0ffaf0 0x7f622c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.683+0000 7f62343ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c100ce0 0x7f622c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f62343ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c101670 0x7f622c1a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f62343ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1a38f0 con 0x7f622c100ce0 2023-07-14T16:16:59.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f62343ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1a3a30 con 0x7f622c0ffaf0 2023-07-14T16:16:59.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f62343ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1a3bc0 con 0x7f622c101670 2023-07-14T16:16:59.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f623219b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c0ffaf0 0x7f622c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f623219b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c0ffaf0 0x7f622c19bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43776/0 (socket says 172.21.15.62:43776) 2023-07-14T16:16:59.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f623219b700 1 -- 172.21.15.62:0/1804649478 learned_addr learned my addr 172.21.15.62:0/1804649478 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f623299c700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c101670 0x7f622c1a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.684+0000 7f623199a700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c100ce0 0x7f622c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.685+0000 7f623219b700 1 -- 172.21.15.62:0/1804649478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c101670 msgr2=0x7f622c1a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.685+0000 7f623219b700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c101670 0x7f622c1a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.685+0000 7f623219b700 1 -- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c100ce0 msgr2=0x7f622c19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.685+0000 7f623219b700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c100ce0 0x7f622c19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.685+0000 7f623219b700 1 -- 172.21.15.62:0/1804649478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c0095f0 con 0x7f622c0ffaf0 2023-07-14T16:16:59.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.685+0000 7f623299c700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c101670 0x7f622c1a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:59.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.686+0000 7f623219b700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c0ffaf0 0x7f622c19bd70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f622800e970 tx=0x7f622800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.686+0000 7f62237fe700 1 -- 172.21.15.62:0/1804649478 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228009790 con 0x7f622c0ffaf0 2023-07-14T16:16:59.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.686+0000 7f62237fe700 1 -- 172.21.15.62:0/1804649478 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f622800c7e0 con 0x7f622c0ffaf0 2023-07-14T16:16:59.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.686+0000 7f62343ff700 1 -- 172.21.15.62:0/1804649478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c1a3ea0 con 0x7f622c0ffaf0 2023-07-14T16:16:59.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.686+0000 7f62237fe700 1 -- 172.21.15.62:0/1804649478 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228003bd0 con 0x7f622c0ffaf0 2023-07-14T16:16:59.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.686+0000 7f62343ff700 1 -- 172.21.15.62:0/1804649478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c1a42d0 con 0x7f622c0ffaf0 2023-07-14T16:16:59.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.691+0000 7f62237fe700 1 -- 172.21.15.62:0/1804649478 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6228003d30 con 0x7f622c0ffaf0 2023-07-14T16:16:59.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.694+0000 7f6247ce1b80 1 Processor -- start 2023-07-14T16:16:59.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.694+0000 7f6247ce1b80 1 -- start start 2023-07-14T16:16:59.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.695+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c2048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.695+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c204e20 0x557e6c2056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.695+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.695+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c109cb0 con 0x557e6c2044d0 2023-07-14T16:16:59.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.695+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c15ca40 con 0x557e6c204e20 2023-07-14T16:16:59.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.695+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c155ed0 con 0x557e6c205c70 2023-07-14T16:16:59.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.695+0000 7f623199a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.696+0000 7f623199a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2060e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:58948/0 (socket says 172.21.15.62:58948) 2023-07-14T16:16:59.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.696+0000 7f623199a700 1 -- 172.21.15.62:0/386866959 learned_addr learned my addr 172.21.15.62:0/386866959 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.696+0000 7f623199a700 1 -- 172.21.15.62:0/386866959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c204e20 msgr2=0x557e6c2056e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.696+0000 7f623199a700 1 --2- 172.21.15.62:0/386866959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c204e20 0x557e6c2056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.696+0000 7f623199a700 1 -- 172.21.15.62:0/386866959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 msgr2=0x557e6c2048e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.696+0000 7f623199a700 1 --2- 172.21.15.62:0/386866959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c2048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.696+0000 7f623199a700 1 -- 172.21.15.62:0/386866959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6bf83de0 con 0x557e6c205c70 2023-07-14T16:16:59.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.697+0000 7f623199a700 1 --2- 172.21.15.62:0/386866959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2060e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f621c009990 tx=0x7f621c005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff98fd572be0da1c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.697+0000 7f62237fe700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a11d0 0x7f62180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.697+0000 7f62237fe700 1 -- 172.21.15.62:0/1804649478 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6228014070 con 0x7f622c0ffaf0 2023-07-14T16:16:59.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.697+0000 7f623299c700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a11d0 0x7f62180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.698+0000 7f6220bfe700 1 -- 172.21.15.62:0/386866959 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c01d070 con 0x557e6c205c70 2023-07-14T16:16:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.698+0000 7f6220bfe700 1 -- 172.21.15.62:0/386866959 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f621c01e040 con 0x557e6c205c70 2023-07-14T16:16:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.698+0000 7f6220bfe700 1 -- 172.21.15.62:0/386866959 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00b910 con 0x557e6c205c70 2023-07-14T16:16:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 -- 172.21.15.62:0/386866959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 msgr2=0x557e6c2060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f623299c700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a11d0 0x7f62180a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6224009550 tx=0x7f6224009cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/386866959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2060e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f621c009990 tx=0x7f621c005c90 comp rx=0 tx=0).stop 2023-07-14T16:16:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 -- 172.21.15.62:0/386866959 shutdown_connections 2023-07-14T16:16:59.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/386866959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2060e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/386866959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c2048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/386866959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c204e20 0x557e6c2056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 -- 172.21.15.62:0/386866959 >> 172.21.15.62:0/386866959 conn(0x557e6c1f9da0 msgr2=0x557e6c1ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 -- 172.21.15.62:0/386866959 shutdown_connections 2023-07-14T16:16:59.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.699+0000 7f6247ce1b80 1 -- 172.21.15.62:0/386866959 wait complete. 2023-07-14T16:16:59.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.700+0000 7f6247ce1b80 1 -- start start 2023-07-14T16:16:59.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.700+0000 7f6247ce1b80 1 Processor -- start 2023-07-14T16:16:59.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.700+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c20e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.700+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c20e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.700+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.700+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c0f5ac0 con 0x557e6c204e20 2023-07-14T16:16:59.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.700+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c0f57c0 con 0x557e6c2044d0 2023-07-14T16:16:59.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.700+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6bfd9920 con 0x557e6c205c70 2023-07-14T16:16:59.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f6247ce1b80 10 client.0 _ll_register_callbacks cb 0x557e6c0f3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:59.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623219b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c20e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623199a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c20e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623219b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c20e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43784/0 (socket says 172.21.15.62:43784) 2023-07-14T16:16:59.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623299c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623219b700 1 -- 172.21.15.62:0/3009397282 learned_addr learned my addr 172.21.15.62:0/3009397282 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623219b700 1 -- 172.21.15.62:0/3009397282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 msgr2=0x557e6c2156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623219b700 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2156e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623219b700 1 -- 172.21.15.62:0/3009397282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 msgr2=0x557e6c20e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623219b700 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c20e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.701+0000 7f623219b700 1 -- 172.21.15.62:0/3009397282 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c108860 con 0x557e6c2044d0 2023-07-14T16:16:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f623219b700 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c20e160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6228099df0 tx=0x7f6228099a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d72f557b6c807d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f6215ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f6215ffb700 1 -- 172.21.15.62:0/3009397282 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622809a2c0 con 0x557e6c2044d0 2023-07-14T16:16:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f6215ffb700 1 -- 172.21.15.62:0/3009397282 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6228016330 con 0x557e6c2044d0 2023-07-14T16:16:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3009397282 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e6c0af4f0 con 0x557e6c2044d0 2023-07-14T16:16:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f6215ffb700 1 -- 172.21.15.62:0/3009397282 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6228099e60 con 0x557e6c2044d0 2023-07-14T16:16:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f6247ce1b80 10 client.25655 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3009397282 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557e6bf6cc60 con 0x557e6c2044d0 2023-07-14T16:16:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.702+0000 7f6215ffb700 1 -- 172.21.15.62:0/3009397282 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557e6c0af4f0 con 0x557e6c2044d0 2023-07-14T16:16:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.703+0000 7f6215ffb700 1 -- 172.21.15.62:0/3009397282 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6228099470 con 0x557e6c2044d0 2023-07-14T16:16:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.703+0000 7f6247ce1b80 10 client.25655 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.703+0000 7f6247ce1b80 10 client.25655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.703+0000 7f6247ce1b80 20 client.25655 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.703+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e6c219dd0 0x557e6c21c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.703+0000 7f6247ce1b80 4 client.25655 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:16:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.703+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3009397282 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e6c14eeb0 con 0x557e6c219dd0 2023-07-14T16:16:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.703+0000 7f623199a700 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e6c219dd0 0x557e6c21c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.704+0000 7f623199a700 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e6c219dd0 0x557e6c21c1e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b6c4a1c373c1aa6 server_cookie=8c9cf21cf5762b in_seq=0 out_seq=0 2023-07-14T16:16:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.704+0000 7f6215ffb700 10 client.25655 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.704+0000 7f6215ffb700 1 -- 172.21.15.62:0/3009397282 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557e6c14eeb0 con 0x557e6c219dd0 2023-07-14T16:16:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.704+0000 7f6215ffb700 10 client.25655 handle_command_reply: tid=0 2023-07-14T16:16:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.704+0000 7f6247ce1b80 1 client.25655 shutdown 2023-07-14T16:16:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3009397282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 msgr2=0x557e6c20e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c20e160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6228099df0 tx=0x7f6228099a20 comp rx=0 tx=0).stop 2023-07-14T16:16:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3009397282 shutdown_connections 2023-07-14T16:16:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c2156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e6c219dd0 0x557e6c21c1e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c20e160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3009397282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c20e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3009397282 >> 172.21.15.62:0/3009397282 conn(0x557e6c1f9da0 msgr2=0x557e6c1fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6215ffb700 0 client.25655 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:59.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3009397282 shutdown_connections 2023-07-14T16:16:59.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.705+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3009397282 wait complete. 2023-07-14T16:16:59.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.706+0000 7f6247ce1b80 20 client.25655 trim_cache size 0 max 16384 2023-07-14T16:16:59.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f6247ce1b80 1 Processor -- start 2023-07-14T16:16:59.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f6247ce1b80 1 -- start start 2023-07-14T16:16:59.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c20c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c204e20 0x557e6c2156a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c205c70 0x557e6c208c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c20d560 con 0x557e6c2044d0 2023-07-14T16:16:59.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c200690 con 0x557e6c205c70 2023-07-14T16:16:59.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c200c50 con 0x557e6c204e20 2023-07-14T16:16:59.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f623219b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c20c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f623219b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c20c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45930/0 (socket says 172.21.15.62:45930) 2023-07-14T16:16:59.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f623219b700 1 -- 172.21.15.62:0/1265361818 learned_addr learned my addr 172.21.15.62:0/1265361818 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.727+0000 7f623299c700 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c205c70 0x557e6c208c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f623199a700 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c204e20 0x557e6c2156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f623219b700 1 -- 172.21.15.62:0/1265361818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c204e20 msgr2=0x557e6c2156a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f623219b700 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c204e20 0x557e6c2156a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f623219b700 1 -- 172.21.15.62:0/1265361818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c205c70 msgr2=0x557e6c208c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f623219b700 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c205c70 0x557e6c208c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f623219b700 1 -- 172.21.15.62:0/1265361818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6bfaa250 con 0x557e6c2044d0 2023-07-14T16:16:59.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f623219b700 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c20c140 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f62280968d0 tx=0x7f6228098410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cc305581c5107e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f62213ff700 1 -- 172.21.15.62:0/1265361818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62280997c0 con 0x557e6c2044d0 2023-07-14T16:16:59.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f62213ff700 1 -- 172.21.15.62:0/1265361818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6228016330 con 0x557e6c2044d0 2023-07-14T16:16:59.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.728+0000 7f62213ff700 1 -- 172.21.15.62:0/1265361818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622809a320 con 0x557e6c2044d0 2023-07-14T16:16:59.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 -- 172.21.15.62:0/1265361818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 msgr2=0x557e6c20c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c20c140 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f62280968d0 tx=0x7f6228098410 comp rx=0 tx=0).stop 2023-07-14T16:16:59.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 -- 172.21.15.62:0/1265361818 shutdown_connections 2023-07-14T16:16:59.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c205c70 0x557e6c208c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c2044d0 0x557e6c20c140 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/1265361818 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c204e20 0x557e6c2156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 -- 172.21.15.62:0/1265361818 >> 172.21.15.62:0/1265361818 conn(0x557e6c1f9da0 msgr2=0x557e6c20ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 -- 172.21.15.62:0/1265361818 shutdown_connections 2023-07-14T16:16:59.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.729+0000 7f6247ce1b80 1 -- 172.21.15.62:0/1265361818 wait complete. 2023-07-14T16:16:59.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.730+0000 7f6247ce1b80 1 -- start start 2023-07-14T16:16:59.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.730+0000 7f6247ce1b80 1 Processor -- start 2023-07-14T16:16:59.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.730+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c214e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.730+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c2153b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.730+0000 7f6247ce1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c213480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.730+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c2139c0 con 0x557e6c204e20 2023-07-14T16:16:59.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.730+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c213b30 con 0x557e6c2044d0 2023-07-14T16:16:59.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.730+0000 7f6247ce1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c213cf0 con 0x557e6c205c70 2023-07-14T16:16:59.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f6247ce1b80 10 client.0 _ll_register_callbacks cb 0x7f6228096270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:16:59.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623199a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c2153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623199a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c2153b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45934/0 (socket says 172.21.15.62:45934) 2023-07-14T16:16:59.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623219b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c214e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623199a700 1 -- 172.21.15.62:0/3590533288 learned_addr learned my addr 172.21.15.62:0/3590533288 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:16:59.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623299c700 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c213480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623219b700 1 -- 172.21.15.62:0/3590533288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 msgr2=0x557e6c213480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623219b700 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c213480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623219b700 1 -- 172.21.15.62:0/3590533288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 msgr2=0x557e6c2153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:16:59.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623219b700 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c2153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623219b700 1 -- 172.21.15.62:0/3590533288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e6bf6cc60 con 0x557e6c2044d0 2023-07-14T16:16:59.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623199a700 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c2153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:16:59.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623299c700 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c213480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:16:59.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f623219b700 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c214e70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x557e6c205b60 tx=0x7f6228005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b40e31554f76b63 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:16:59.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.731+0000 7f6215ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:16:59.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6215ffb700 1 -- 172.21.15.62:0/3590533288 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62280ca3e0 con 0x557e6c2044d0 2023-07-14T16:16:59.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6215ffb700 1 -- 172.21.15.62:0/3590533288 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6228099e60 con 0x557e6c2044d0 2023-07-14T16:16:59.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3590533288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f621c01e040 con 0x557e6c2044d0 2023-07-14T16:16:59.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6215ffb700 1 -- 172.21.15.62:0/3590533288 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62280a22b0 con 0x557e6c2044d0 2023-07-14T16:16:59.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 10 client.25661 fetch_fsmap learned FSMap version 23 2023-07-14T16:16:59.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3590533288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557e6bf822b0 con 0x557e6c2044d0 2023-07-14T16:16:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6215ffb700 1 -- 172.21.15.62:0/3590533288 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f621c01e040 con 0x557e6c2044d0 2023-07-14T16:16:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6215ffb700 1 -- 172.21.15.62:0/3590533288 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f62280adb60 con 0x557e6c2044d0 2023-07-14T16:16:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 10 client.25661 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:16:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:16:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 20 client.25661 populate_metadata read hostname 'smithi062' 2023-07-14T16:16:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e6c26e7a0 0x557e6c270b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:16:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 4 client.25661 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:16:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.732+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3590533288 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e6c111050 con 0x557e6c26e7a0 2023-07-14T16:16:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.733+0000 7f623199a700 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e6c26e7a0 0x557e6c270b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:16:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.733+0000 7f623199a700 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e6c26e7a0 0x557e6c270b80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8f53964d36ed40 server_cookie=404e76a13d63f619 in_seq=0 out_seq=0 2023-07-14T16:16:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.733+0000 7f6215ffb700 10 client.25661 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:59.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.733+0000 7f6215ffb700 1 -- 172.21.15.62:0/3590533288 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557e6c111050 con 0x557e6c26e7a0 2023-07-14T16:16:59.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.733+0000 7f6215ffb700 10 client.25661 handle_command_reply: tid=0 2023-07-14T16:16:59.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 client.25661 shutdown 2023-07-14T16:16:59.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3590533288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 msgr2=0x557e6c214e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c214e70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x557e6c205b60 tx=0x7f6228005910 comp rx=0 tx=0).stop 2023-07-14T16:16:59.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3590533288 shutdown_connections 2023-07-14T16:16:59.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e6c205c70 0x557e6c213480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e6c26e7a0 0x557e6c270b80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e6c2044d0 0x557e6c214e70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 --2- 172.21.15.62:0/3590533288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e6c204e20 0x557e6c2153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6215ffb700 0 client.25661 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:16:59.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3590533288 >> 172.21.15.62:0/3590533288 conn(0x557e6c1f9da0 msgr2=0x557e6c20c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3590533288 shutdown_connections 2023-07-14T16:16:59.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 1 -- 172.21.15.62:0/3590533288 wait complete. 2023-07-14T16:16:59.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.734+0000 7f6247ce1b80 20 client.25661 trim_cache size 0 max 16384 2023-07-14T16:16:59.722 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:16:59.722 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:16:59.722 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:16:59.722 INFO:teuthology.orchestra.run.smithi062.stdout: "8070e88b-f9b0-46e1-9288-6c62c620cbfc": { 2023-07-14T16:16:59.722 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:16:59.723 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "8070e88b-f9b0-46e1-9288-6c62c620cbfc", 2023-07-14T16:16:59.723 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:16:59.723 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:16:59.723 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:16:59.723 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:16:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 -- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a11d0 msgr2=0x7f62180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a11d0 0x7f62180a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6224009550 tx=0x7f6224009cf0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 -- 172.21.15.62:0/1804649478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c0ffaf0 msgr2=0x7f622c19bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:16:59.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c0ffaf0 0x7f622c19bd70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f622800e970 tx=0x7f622800ec80 comp rx=0 tx=0).stop 2023-07-14T16:16:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 -- 172.21.15.62:0/1804649478 shutdown_connections 2023-07-14T16:16:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c101670 0x7f622c1a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a11d0 0x7f62180a3680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c0ffaf0 0x7f622c19bd70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 --2- 172.21.15.62:0/1804649478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c100ce0 0x7f622c19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:16:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 -- 172.21.15.62:0/1804649478 >> 172.21.15.62:0/1804649478 conn(0x7f622c0fb130 msgr2=0x7f622c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:16:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 -- 172.21.15.62:0/1804649478 shutdown_connections 2023-07-14T16:16:59.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:16:59.735+0000 7f62343ff700 1 -- 172.21.15.62:0/1804649478 wait complete. 2023-07-14T16:16:59.727 INFO:tasks.workunit.client.0.smithi062.stdout:checking for system... yes 2023-07-14T16:16:59.733 INFO:tasks.cephfs.filesystem:scrub status for tag:8070e88b-f9b0-46e1-9288-6c62c620cbfc - {'path': '/', 'tag': '8070e88b-f9b0-46e1-9288-6c62c620cbfc', 'options': 'recursive,force'} 2023-07-14T16:16:59.885 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gettimeofday... yes 2023-07-14T16:17:00.042 INFO:tasks.workunit.client.0.smithi062.stdout:checking for mkdir... yes 2023-07-14T16:17:00.196 INFO:tasks.workunit.client.0.smithi062.stdout:checking for strerror... yes 2023-07-14T16:17:00.353 INFO:tasks.workunit.client.0.smithi062.stdout:checking for open64... yes 2023-07-14T16:17:00.512 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stat64... yes 2023-07-14T16:17:00.671 INFO:tasks.workunit.client.0.smithi062.stdout:checking for fseeko64... yes 2023-07-14T16:17:00.839 INFO:tasks.workunit.client.0.smithi062.stdout:checking for lrand48_r... yes 2023-07-14T16:17:00.995 INFO:tasks.workunit.client.0.smithi062.stdout:checking for srand48_r... yes 2023-07-14T16:17:01.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:00 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:17:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:00 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:00 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:17:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:00 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:00 smithi164 ceph-mon[97427]: pgmap v398: 97 pgs: 97 active+clean; 4.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 341 KiB/s wr, 106 op/s 2023-07-14T16:17:01.051 INFO:tasks.workunit.client.0.smithi062.stdout:configure: creating ./config.status 2023-07-14T16:17:01.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:00 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:17:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:00 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:00 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:17:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:00 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:01.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:00 smithi062 ceph-mon[95543]: pgmap v398: 97 pgs: 97 active+clean; 4.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 341 KiB/s wr, 106 op/s 2023-07-14T16:17:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:00 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:17:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:00 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:00 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:17:01.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:00 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:01.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:00 smithi081 ceph-mon[127795]: pgmap v398: 97 pgs: 97 active+clean; 4.2 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 341 KiB/s wr, 106 op/s 2023-07-14T16:17:01.349 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating Makefile 2023-07-14T16:17:01.389 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating config.h 2023-07-14T16:17:01.441 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: executing depfiles commands 2023-07-14T16:17:01.593 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: executing default commands 2023-07-14T16:17:01.693 INFO:tasks.workunit.client.0.smithi062.stderr:+ make 2023-07-14T16:17:01.748 INFO:tasks.workunit.client.0.smithi062.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2023-07-14T16:17:01.764 INFO:tasks.workunit.client.0.smithi062.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2023-07-14T16:17:01.767 INFO:tasks.workunit.client.0.smithi062.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2023-07-14T16:17:01.767 INFO:tasks.workunit.client.0.smithi062.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2023-07-14T16:17:01.767 INFO:tasks.workunit.client.0.smithi062.stderr: to install the `Automake' and `Perl' packages. Grab them from 2023-07-14T16:17:01.768 INFO:tasks.workunit.client.0.smithi062.stderr: any GNU archive site. 2023-07-14T16:17:01.777 INFO:tasks.workunit.client.0.smithi062.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2023-07-14T16:17:01.789 INFO:tasks.workunit.client.0.smithi062.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2023-07-14T16:17:01.791 INFO:tasks.workunit.client.0.smithi062.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2023-07-14T16:17:01.792 INFO:tasks.workunit.client.0.smithi062.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2023-07-14T16:17:01.792 INFO:tasks.workunit.client.0.smithi062.stderr: You might want to install the `Automake' and `Perl' packages. 2023-07-14T16:17:01.792 INFO:tasks.workunit.client.0.smithi062.stderr: Grab them from any GNU archive site. 2023-07-14T16:17:02.078 INFO:tasks.workunit.client.0.smithi062.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2023-07-14T16:17:02.447 INFO:tasks.workunit.client.0.smithi062.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2023-07-14T16:17:02.448 INFO:tasks.workunit.client.0.smithi062.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2023-07-14T16:17:02.448 INFO:tasks.workunit.client.0.smithi062.stderr:If you have problems, you may need to regenerate the build system entirely. 2023-07-14T16:17:02.448 INFO:tasks.workunit.client.0.smithi062.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2023-07-14T16:17:02.610 INFO:tasks.workunit.client.0.smithi062.stdout:/bin/sh ./config.status --recheck 2023-07-14T16:17:02.763 INFO:tasks.workunit.client.0.smithi062.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2023-07-14T16:17:03.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:02 smithi164 ceph-mon[97427]: pgmap v399: 97 pgs: 97 active+clean; 4.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 367 KiB/s wr, 144 op/s 2023-07-14T16:17:03.057 INFO:tasks.workunit.client.0.smithi062.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-07-14T16:17:03.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:02 smithi062 ceph-mon[95543]: pgmap v399: 97 pgs: 97 active+clean; 4.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 367 KiB/s wr, 144 op/s 2023-07-14T16:17:03.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:02 smithi081 ceph-mon[127795]: pgmap v399: 97 pgs: 97 active+clean; 4.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 367 KiB/s wr, 144 op/s 2023-07-14T16:17:04.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:17:04.075 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether build environment is sane... yes 2023-07-14T16:17:04.099 INFO:tasks.workunit.client.0.smithi062.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2023-07-14T16:17:04.100 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gawk... gawk 2023-07-14T16:17:04.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:17:04.136 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether make sets $(MAKE)... yes 2023-07-14T16:17:04.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:17:04.239 INFO:tasks.workunit.client.0.smithi062.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-07-14T16:17:04.240 INFO:tasks.workunit.client.0.smithi062.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-07-14T16:17:04.269 INFO:tasks.workunit.client.0.smithi062.stdout:checking for style of include used by make... GNU 2023-07-14T16:17:04.277 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc... gcc 2023-07-14T16:17:04.478 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether the C compiler works... yes 2023-07-14T16:17:04.478 INFO:tasks.workunit.client.0.smithi062.stdout:checking for C compiler default output file name... a.out 2023-07-14T16:17:04.571 INFO:tasks.workunit.client.0.smithi062.stdout:checking for suffix of executables... 2023-07-14T16:17:04.665 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are cross compiling... no 2023-07-14T16:17:04.748 INFO:tasks.workunit.client.0.smithi062.stdout:checking for suffix of object files... o 2023-07-14T16:17:04.807 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are using the GNU C compiler... yes 2023-07-14T16:17:04.867 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether gcc accepts -g... yes 2023-07-14T16:17:04.941 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc option to accept ISO C89... none needed 2023-07-14T16:17:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:04 smithi164 ceph-mon[97427]: pgmap v400: 97 pgs: 97 active+clean; 5.9 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 532 KiB/s wr, 175 op/s 2023-07-14T16:17:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:04 smithi062 ceph-mon[95543]: pgmap v400: 97 pgs: 97 active+clean; 5.9 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 532 KiB/s wr, 175 op/s 2023-07-14T16:17:05.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:04 smithi081 ceph-mon[127795]: pgmap v400: 97 pgs: 97 active+clean; 5.9 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 532 KiB/s wr, 175 op/s 2023-07-14T16:17:05.181 INFO:tasks.workunit.client.0.smithi062.stdout:checking dependency style of gcc... gcc3 2023-07-14T16:17:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:06 smithi164 ceph-mon[97427]: pgmap v401: 97 pgs: 97 active+clean; 5.9 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 158 op/s 2023-07-14T16:17:07.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:06 smithi062 ceph-mon[95543]: pgmap v401: 97 pgs: 97 active+clean; 5.9 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 158 op/s 2023-07-14T16:17:07.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:06 smithi081 ceph-mon[127795]: pgmap v401: 97 pgs: 97 active+clean; 5.9 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 158 op/s 2023-07-14T16:17:08.022 INFO:tasks.workunit.client.0.smithi062.stdout:checking how to run the C preprocessor... gcc -E 2023-07-14T16:17:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1399494127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad40ffb10 msgr2=0x7f2ad40fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1399494127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad40ffb10 0x7f2ad40fff20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009990 tx=0x7f2ac4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1399494127 shutdown_connections 2023-07-14T16:17:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1399494127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad4101690 0x7f2ad4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1399494127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad40ffb10 0x7f2ad40fff20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1399494127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ad4100d00 0x7f2ad4101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1399494127 >> 172.21.15.62:0/1399494127 conn(0x7f2ad40fb130 msgr2=0x7f2ad40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1399494127 shutdown_connections 2023-07-14T16:17:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.013+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1399494127 wait complete. 2023-07-14T16:17:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.015+0000 7f2ada9ff700 1 Processor -- start 2023-07-14T16:17:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.015+0000 7f2ada9ff700 1 -- start start 2023-07-14T16:17:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ada9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad40ffb10 0x7f2ad4195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ada9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ad4100d00 0x7f2ad4195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ada9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad4101690 0x7f2ad4196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ada9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad4197250 con 0x7f2ad4101690 2023-07-14T16:17:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ada9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41973c0 con 0x7f2ad4100d00 2023-07-14T16:17:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ada9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad4197580 con 0x7f2ad40ffb10 2023-07-14T16:17:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad8f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad4101690 0x7f2ad4196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ad4100d00 0x7f2ad4195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad8f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad4101690 0x7f2ad4196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52026/0 (socket says 172.21.15.62:52026) 2023-07-14T16:17:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad8f9c700 1 -- 172.21.15.62:0/1612354415 learned_addr learned my addr 172.21.15.62:0/1612354415 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad40ffb10 0x7f2ad4195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad8f9c700 1 -- 172.21.15.62:0/1612354415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad40ffb10 msgr2=0x7f2ad4195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad8f9c700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad40ffb10 0x7f2ad4195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad8f9c700 1 -- 172.21.15.62:0/1612354415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ad4100d00 msgr2=0x7f2ad4195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad8f9c700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ad4100d00 0x7f2ad4195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.016+0000 7f2ad8f9c700 1 -- 172.21.15.62:0/1612354415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40095f0 con 0x7f2ad4101690 2023-07-14T16:17:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.017+0000 7f2ad8f9c700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad4101690 0x7f2ad4196bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2acc007ee0 tx=0x7f2acc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.017+0000 7f2ad17fa700 1 -- 172.21.15.62:0/1612354415 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc017070 con 0x7f2ad4101690 2023-07-14T16:17:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.017+0000 7f2ad17fa700 1 -- 172.21.15.62:0/1612354415 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2acc00f040 con 0x7f2ad4101690 2023-07-14T16:17:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.017+0000 7f2ad17fa700 1 -- 172.21.15.62:0/1612354415 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc005570 con 0x7f2ad4101690 2023-07-14T16:17:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.017+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1612354415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad4197860 con 0x7f2ad4101690 2023-07-14T16:17:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.017+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1612354415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad419e020 con 0x7f2ad4101690 2023-07-14T16:17:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.022+0000 7f2ad17fa700 1 -- 172.21.15.62:0/1612354415 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2acc00db20 con 0x7f2ad4101690 2023-07-14T16:17:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.023+0000 7f2aee389b80 1 Processor -- start 2023-07-14T16:17:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.023+0000 7f2aee389b80 1 -- start start 2023-07-14T16:17:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.023+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595437ad6b0 0x5595439557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.023+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543955cf0 0x559543956570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.023+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543956ab0 0x559543956f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.023+0000 7f2aee389b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559543856c50 con 0x559543956ab0 2023-07-14T16:17:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.023+0000 7f2aee389b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595438a99e0 con 0x559543955cf0 2023-07-14T16:17:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.023+0000 7f2aee389b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595438a2e70 con 0x5595437ad6b0 2023-07-14T16:17:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543955cf0 0x559543956570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543956ab0 0x559543956f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543955cf0 0x559543956570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46458/0 (socket says 172.21.15.62:46458) 2023-07-14T16:17:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad3fff700 1 -- 172.21.15.62:0/3617094577 learned_addr learned my addr 172.21.15.62:0/3617094577 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595437ad6b0 0x5595439557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad17fa700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2abc0a11d0 0x7f2abc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad3fff700 1 -- 172.21.15.62:0/3617094577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595437ad6b0 msgr2=0x5595439557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595437ad6b0 0x5595439557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad3fff700 1 -- 172.21.15.62:0/3617094577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543956ab0 msgr2=0x559543956f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543956ab0 0x559543956f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad3fff700 1 -- 172.21.15.62:0/3617094577 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595436d0d80 con 0x559543955cf0 2023-07-14T16:17:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad17fa700 1 -- 172.21.15.62:0/1612354415 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2acc0cc060 con 0x7f2ad4101690 2023-07-14T16:17:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543956ab0 0x559543956f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.024+0000 7f2ad8f9c700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2abc0a11d0 0x7f2abc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.025+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543955cf0 0x559543956570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4100b60 tx=0x7f2ac4009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76f06dc369e4f1f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.025+0000 7f2aba7fc700 1 -- 172.21.15.62:0/3617094577 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac401d070 con 0x559543955cf0 2023-07-14T16:17:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.025+0000 7f2ad8f9c700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2abc0a11d0 0x7f2abc0a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0c9b10 tx=0x7f2acc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.025+0000 7f2aba7fc700 1 -- 172.21.15.62:0/3617094577 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ac401e040 con 0x559543955cf0 2023-07-14T16:17:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.025+0000 7f2aba7fc700 1 -- 172.21.15.62:0/3617094577 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4004500 con 0x559543955cf0 2023-07-14T16:17:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 -- 172.21.15.62:0/3617094577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543955cf0 msgr2=0x559543956570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543955cf0 0x559543956570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4100b60 tx=0x7f2ac4009ee0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 -- 172.21.15.62:0/3617094577 shutdown_connections 2023-07-14T16:17:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543956ab0 0x559543956f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5595437ad6b0 0x5595439557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 --2- 172.21.15.62:0/3617094577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543955cf0 0x559543956570 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 -- 172.21.15.62:0/3617094577 >> 172.21.15.62:0/3617094577 conn(0x559543946be0 msgr2=0x55954394bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 -- 172.21.15.62:0/3617094577 shutdown_connections 2023-07-14T16:17:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 -- 172.21.15.62:0/3617094577 wait complete. 2023-07-14T16:17:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.026+0000 7f2aee389b80 1 -- start start 2023-07-14T16:17:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.027+0000 7f2aee389b80 1 Processor -- start 2023-07-14T16:17:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5595437ad6b0 0x55954395ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543955cf0 0x55954395f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543956ab0 0x559543966150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2aee389b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559543842a60 con 0x559543955cf0 2023-07-14T16:17:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2aee389b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559543842760 con 0x5595437ad6b0 2023-07-14T16:17:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2aee389b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595437268c0 con 0x559543956ab0 2023-07-14T16:17:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2aee389b80 10 client.0 _ll_register_callbacks cb 0x559543840700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543955cf0 0x55954395f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543955cf0 0x55954395f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52038/0 (socket says 172.21.15.62:52038) 2023-07-14T16:17:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad3fff700 1 -- 172.21.15.62:0/4144455734 learned_addr learned my addr 172.21.15.62:0/4144455734 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5595437ad6b0 0x55954395ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543956ab0 0x559543966150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad3fff700 1 -- 172.21.15.62:0/4144455734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543956ab0 msgr2=0x559543966150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543956ab0 0x559543966150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad3fff700 1 -- 172.21.15.62:0/4144455734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5595437ad6b0 msgr2=0x55954395ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5595437ad6b0 0x55954395ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad3fff700 1 -- 172.21.15.62:0/4144455734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac8019040 con 0x559543955cf0 2023-07-14T16:17:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.028+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543956ab0 0x559543966150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.029+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543955cf0 0x55954395f110 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009ce0 tx=0x7f2ac400b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3af88e803e1a82 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.029+0000 7f2ab37fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.029+0000 7f2ab37fe700 1 -- 172.21.15.62:0/4144455734 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac401d030 con 0x559543955cf0 2023-07-14T16:17:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.029+0000 7f2aee389b80 1 -- 172.21.15.62:0/4144455734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595437fc490 con 0x559543955cf0 2023-07-14T16:17:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.029+0000 7f2ab37fe700 1 -- 172.21.15.62:0/4144455734 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2ac4005980 con 0x559543955cf0 2023-07-14T16:17:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.029+0000 7f2ab37fe700 1 -- 172.21.15.62:0/4144455734 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac4003a20 con 0x559543955cf0 2023-07-14T16:17:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.029+0000 7f2aee389b80 10 client.15909 fetch_fsmap learned FSMap version 23 2023-07-14T16:17:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.029+0000 7f2aee389b80 1 -- 172.21.15.62:0/4144455734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f2ad4108880 con 0x559543955cf0 2023-07-14T16:17:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2ab37fe700 1 -- 172.21.15.62:0/4144455734 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5595437fc490 con 0x559543955cf0 2023-07-14T16:17:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2ab37fe700 1 -- 172.21.15.62:0/4144455734 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2ac400f940 con 0x559543955cf0 2023-07-14T16:17:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2aee389b80 10 client.15909 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:17:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2aee389b80 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2aee389b80 20 client.15909 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2aee389b80 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55954396a550 0x55954396c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2aee389b80 4 client.15909 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2aee389b80 1 -- 172.21.15.62:0/4144455734 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55954389be50 con 0x55954396a550 2023-07-14T16:17:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.030+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55954396a550 0x55954396c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.031+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55954396a550 0x55954396c960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921572b3b3e763c6 server_cookie=e21c6d00e8027db6 in_seq=0 out_seq=0 2023-07-14T16:17:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.031+0000 7f2ab37fe700 10 client.15909 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2ab37fe700 1 -- 172.21.15.62:0/4144455734 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55954389be50 con 0x55954396a550 2023-07-14T16:17:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2ab37fe700 10 client.15909 handle_command_reply: tid=0 2023-07-14T16:17:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 client.15909 shutdown 2023-07-14T16:17:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 -- 172.21.15.62:0/4144455734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543955cf0 msgr2=0x55954395f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543955cf0 0x55954395f110 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009ce0 tx=0x7f2ac400b980 comp rx=0 tx=0).stop 2023-07-14T16:17:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 -- 172.21.15.62:0/4144455734 shutdown_connections 2023-07-14T16:17:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543956ab0 0x559543966150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55954396a550 0x55954396c960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5595437ad6b0 0x55954395ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 --2- 172.21.15.62:0/4144455734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559543955cf0 0x55954395f110 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 -- 172.21.15.62:0/4144455734 >> 172.21.15.62:0/4144455734 conn(0x559543946be0 msgr2=0x55954394b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2ab37fe700 0 client.15909 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 -- 172.21.15.62:0/4144455734 shutdown_connections 2023-07-14T16:17:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 1 -- 172.21.15.62:0/4144455734 wait complete. 2023-07-14T16:17:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.032+0000 7f2aee389b80 20 client.15909 trim_cache size 0 max 16384 2023-07-14T16:17:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:08 smithi164 ceph-mon[97427]: pgmap v402: 97 pgs: 97 active+clean; 5.9 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 389 KiB/s wr, 162 op/s 2023-07-14T16:17:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2aee389b80 1 Processor -- start 2023-07-14T16:17:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2aee389b80 1 -- start start 2023-07-14T16:17:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543959880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x559543966110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 0x559543959dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2aee389b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954394d4e0 con 0x5595437ad6b0 2023-07-14T16:17:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2aee389b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954395e080 con 0x559543956ab0 2023-07-14T16:17:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2aee389b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954395e740 con 0x559543955cf0 2023-07-14T16:17:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2ad37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543959880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2ad37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543959880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52054/0 (socket says 172.21.15.62:52054) 2023-07-14T16:17:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.048+0000 7f2ad37fe700 1 -- 172.21.15.62:0/359231824 learned_addr learned my addr 172.21.15.62:0/359231824 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/359231824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x559543966110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2ad37fe700 1 -- 172.21.15.62:0/359231824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 msgr2=0x559543966110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/359231824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x559543966110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2ad37fe700 1 -- 172.21.15.62:0/359231824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 msgr2=0x559543959dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/359231824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 0x559543959dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2ad37fe700 1 -- 172.21.15.62:0/359231824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595436b9c00 con 0x5595437ad6b0 2023-07-14T16:17:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/359231824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543959880 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8001eb0 tx=0x7f2ac8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b7f8bd0979684c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2abaffd700 1 -- 172.21.15.62:0/359231824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80241c0 con 0x5595437ad6b0 2023-07-14T16:17:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2abaffd700 1 -- 172.21.15.62:0/359231824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ac8004cc0 con 0x5595437ad6b0 2023-07-14T16:17:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.049+0000 7f2abaffd700 1 -- 172.21.15.62:0/359231824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80053e0 con 0x5595437ad6b0 2023-07-14T16:17:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 -- 172.21.15.62:0/359231824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 msgr2=0x559543959880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 --2- 172.21.15.62:0/359231824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543959880 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8001eb0 tx=0x7f2ac8004750 comp rx=0 tx=0).stop 2023-07-14T16:17:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 -- 172.21.15.62:0/359231824 shutdown_connections 2023-07-14T16:17:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 --2- 172.21.15.62:0/359231824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 0x559543959dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 --2- 172.21.15.62:0/359231824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543959880 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 --2- 172.21.15.62:0/359231824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x559543966110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 -- 172.21.15.62:0/359231824 >> 172.21.15.62:0/359231824 conn(0x559543946be0 msgr2=0x559543971280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 -- 172.21.15.62:0/359231824 shutdown_connections 2023-07-14T16:17:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.050+0000 7f2aee389b80 1 -- 172.21.15.62:0/359231824 wait complete. 2023-07-14T16:17:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 1 -- start start 2023-07-14T16:17:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 1 Processor -- start 2023-07-14T16:17:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543961db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x5595439622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 0x559543965310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559543965850 con 0x5595437ad6b0 2023-07-14T16:17:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595439659c0 con 0x559543956ab0 2023-07-14T16:17:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559543965b80 con 0x559543955cf0 2023-07-14T16:17:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.051+0000 7f2aee389b80 10 client.0 _ll_register_callbacks cb 0x7f2ac800aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543961db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543961db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52062/0 (socket says 172.21.15.62:52062) 2023-07-14T16:17:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad37fe700 1 -- 172.21.15.62:0/1154819987 learned_addr learned my addr 172.21.15.62:0/1154819987 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x5595439622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 0x559543965310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad3fff700 1 -- 172.21.15.62:0/1154819987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 msgr2=0x559543965310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 0x559543965310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad3fff700 1 -- 172.21.15.62:0/1154819987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 msgr2=0x559543961db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543961db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.052+0000 7f2ad3fff700 1 -- 172.21.15.62:0/1154819987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac8004160 con 0x559543955cf0 2023-07-14T16:17:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2ad3fff700 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x5595439622f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4000c00 tx=0x7f2ac4009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf0046445a2cbfd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2ab37fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:17:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1154819987 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac401d040 con 0x559543955cf0 2023-07-14T16:17:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1154819987 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2ac4000e40 con 0x559543955cf0 2023-07-14T16:17:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2aee389b80 1 -- 172.21.15.62:0/1154819987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ac401e040 con 0x559543955cf0 2023-07-14T16:17:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1154819987 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac40058e0 con 0x559543955cf0 2023-07-14T16:17:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2aee389b80 10 client.25600 fetch_fsmap learned FSMap version 23 2023-07-14T16:17:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2aee389b80 1 -- 172.21.15.62:0/1154819987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5595436f71f0 con 0x559543955cf0 2023-07-14T16:17:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1154819987 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ac401e040 con 0x559543955cf0 2023-07-14T16:17:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.053+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1154819987 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2ac4003820 con 0x559543955cf0 2023-07-14T16:17:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.054+0000 7f2aee389b80 10 client.25600 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T16:17:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.054+0000 7f2aee389b80 10 client.25600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.054+0000 7f2aee389b80 20 client.25600 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.054+0000 7f2aee389b80 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5595439bedf0 0x5595439c1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.054+0000 7f2aee389b80 4 client.25600 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:17:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.054+0000 7f2aee389b80 1 -- 172.21.15.62:0/1154819987 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55954385dff0 con 0x5595439bedf0 2023-07-14T16:17:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.054+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5595439bedf0 0x5595439c1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.055+0000 7f2ad37fe700 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5595439bedf0 0x5595439c1200 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5286d3895939dd7f server_cookie=dd7c39ce1780d4df in_seq=0 out_seq=0 2023-07-14T16:17:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.055+0000 7f2ab37fe700 10 client.25600 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:09.082 INFO:tasks.workunit.client.0.smithi062.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-07-14T16:17:09.086 INFO:tasks.workunit.client.0.smithi062.stdout:checking for egrep... /usr/bin/grep -E 2023-07-14T16:17:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2ab37fe700 1 -- 172.21.15.62:0/1154819987 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55954385dff0 con 0x5595439bedf0 2023-07-14T16:17:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2ab37fe700 10 client.25600 handle_command_reply: tid=0 2023-07-14T16:17:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 client.25600 shutdown 2023-07-14T16:17:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 -- 172.21.15.62:0/1154819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 msgr2=0x5595439622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x5595439622f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4000c00 tx=0x7f2ac4009290 comp rx=0 tx=0).stop 2023-07-14T16:17:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 -- 172.21.15.62:0/1154819987 shutdown_connections 2023-07-14T16:17:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5595439bedf0 0x5595439c1200 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559543956ab0 0x559543965310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5595437ad6b0 0x559543961db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2ab37fe700 0 client.25600 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 --2- 172.21.15.62:0/1154819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559543955cf0 0x5595439622f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 -- 172.21.15.62:0/1154819987 >> 172.21.15.62:0/1154819987 conn(0x559543946be0 msgr2=0x559543959880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.111+0000 7f2aee389b80 1 -- 172.21.15.62:0/1154819987 shutdown_connections 2023-07-14T16:17:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2aee389b80 1 -- 172.21.15.62:0/1154819987 wait complete. 2023-07-14T16:17:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2aee389b80 20 client.25600 trim_cache size 0 max 16384 2023-07-14T16:17:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:17:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:17:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:17:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:17:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2abc0a11d0 msgr2=0x7f2abc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2abc0a11d0 0x7f2abc0a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0c9b10 tx=0x7f2acc00b040 comp rx=0 tx=0).stop 2023-07-14T16:17:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad4101690 msgr2=0x7f2ad4196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad4101690 0x7f2ad4196bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2acc007ee0 tx=0x7f2acc00d3b0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1612354415 shutdown_connections 2023-07-14T16:17:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ad4101690 0x7f2ad4196bf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad40ffb10 0x7f2ad4195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2abc0a11d0 0x7f2abc0a3680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 --2- 172.21.15.62:0/1612354415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ad4100d00 0x7f2ad4195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.112+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1612354415 >> 172.21.15.62:0/1612354415 conn(0x7f2ad40fb130 msgr2=0x7f2ad40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.113+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1612354415 shutdown_connections 2023-07-14T16:17:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:09.113+0000 7f2ada9ff700 1 -- 172.21.15.62:0/1612354415 wait complete. 2023-07-14T16:17:09.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:08 smithi062 ceph-mon[95543]: pgmap v402: 97 pgs: 97 active+clean; 5.9 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 389 KiB/s wr, 162 op/s 2023-07-14T16:17:09.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:08 smithi081 ceph-mon[127795]: pgmap v402: 97 pgs: 97 active+clean; 5.9 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 389 KiB/s wr, 162 op/s 2023-07-14T16:17:09.362 INFO:tasks.workunit.client.0.smithi062.stdout:checking for ANSI C header files... yes 2023-07-14T16:17:09.444 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/types.h... yes 2023-07-14T16:17:09.538 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/stat.h... yes 2023-07-14T16:17:09.634 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdlib.h... yes 2023-07-14T16:17:09.729 INFO:tasks.workunit.client.0.smithi062.stdout:checking for string.h... yes 2023-07-14T16:17:09.824 INFO:tasks.workunit.client.0.smithi062.stdout:checking for memory.h... yes 2023-07-14T16:17:09.922 INFO:tasks.workunit.client.0.smithi062.stdout:checking for strings.h... yes 2023-07-14T16:17:10.020 INFO:tasks.workunit.client.0.smithi062.stdout:checking for inttypes.h... yes 2023-07-14T16:17:10.115 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdint.h... yes 2023-07-14T16:17:10.215 INFO:tasks.workunit.client.0.smithi062.stdout:checking for unistd.h... yes 2023-07-14T16:17:10.316 INFO:tasks.workunit.client.0.smithi062.stdout:checking minix/config.h usability... no 2023-07-14T16:17:11.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:10 smithi164 ceph-mon[97427]: pgmap v403: 97 pgs: 97 active+clean; 6.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 393 KiB/s wr, 137 op/s 2023-07-14T16:17:11.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:10 smithi062 ceph-mon[95543]: pgmap v403: 97 pgs: 97 active+clean; 6.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 393 KiB/s wr, 137 op/s 2023-07-14T16:17:11.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:10 smithi081 ceph-mon[127795]: pgmap v403: 97 pgs: 97 active+clean; 6.3 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 393 KiB/s wr, 137 op/s 2023-07-14T16:17:13.012 INFO:tasks.workunit.client.0.smithi062.stdout:checking minix/config.h presence... no 2023-07-14T16:17:13.013 INFO:tasks.workunit.client.0.smithi062.stdout:checking for minix/config.h... no 2023-07-14T16:17:13.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:12 smithi164 ceph-mon[97427]: pgmap v404: 97 pgs: 97 active+clean; 6.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 285 KiB/s wr, 109 op/s 2023-07-14T16:17:13.095 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2023-07-14T16:17:13.109 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc... (cached) gcc 2023-07-14T16:17:13.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:12 smithi062 ceph-mon[95543]: pgmap v404: 97 pgs: 97 active+clean; 6.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 285 KiB/s wr, 109 op/s 2023-07-14T16:17:13.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:12 smithi081 ceph-mon[127795]: pgmap v404: 97 pgs: 97 active+clean; 6.3 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 285 KiB/s wr, 109 op/s 2023-07-14T16:17:13.200 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-07-14T16:17:13.207 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether gcc accepts -g... (cached) yes 2023-07-14T16:17:13.208 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-07-14T16:17:13.208 INFO:tasks.workunit.client.0.smithi062.stdout:checking dependency style of gcc... (cached) gcc3 2023-07-14T16:17:13.317 INFO:tasks.workunit.client.0.smithi062.stdout:checking for main in -lm... yes 2023-07-14T16:17:13.434 INFO:tasks.workunit.client.0.smithi062.stdout:checking for main in -lpthread... yes 2023-07-14T16:17:13.439 INFO:tasks.workunit.client.0.smithi062.stdout:checking for ANSI C header files... (cached) yes 2023-07-14T16:17:13.524 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2023-07-14T16:17:13.626 INFO:tasks.workunit.client.0.smithi062.stdout:checking pthread.h usability... yes 2023-07-14T16:17:13.665 INFO:tasks.workunit.client.0.smithi062.stdout:checking pthread.h presence... yes 2023-07-14T16:17:13.665 INFO:tasks.workunit.client.0.smithi062.stdout:checking for pthread.h... yes 2023-07-14T16:17:13.774 INFO:tasks.workunit.client.0.smithi062.stdout:checking fcntl.h usability... yes 2023-07-14T16:17:13.817 INFO:tasks.workunit.client.0.smithi062.stdout:checking fcntl.h presence... yes 2023-07-14T16:17:13.818 INFO:tasks.workunit.client.0.smithi062.stdout:checking for fcntl.h... yes 2023-07-14T16:17:13.944 INFO:tasks.workunit.client.0.smithi062.stdout:checking limits.h usability... yes 2023-07-14T16:17:13.979 INFO:tasks.workunit.client.0.smithi062.stdout:checking limits.h presence... yes 2023-07-14T16:17:13.980 INFO:tasks.workunit.client.0.smithi062.stdout:checking for limits.h... yes 2023-07-14T16:17:14.017 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdint.h... (cached) yes 2023-07-14T16:17:14.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:13 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:17:14.116 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/time.h usability... yes 2023-07-14T16:17:14.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:13 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:17:14.151 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/time.h presence... yes 2023-07-14T16:17:14.152 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/time.h... yes 2023-07-14T16:17:14.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:13 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:17:14.183 INFO:tasks.workunit.client.0.smithi062.stdout:checking for unistd.h... (cached) yes 2023-07-14T16:17:14.276 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/vfs.h usability... yes 2023-07-14T16:17:14.311 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/vfs.h presence... yes 2023-07-14T16:17:14.311 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/vfs.h... yes 2023-07-14T16:17:14.426 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/limits.h usability... no 2023-07-14T16:17:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:14 smithi164 ceph-mon[97427]: pgmap v405: 97 pgs: 97 active+clean; 3.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 321 KiB/s wr, 85 op/s 2023-07-14T16:17:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:17:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:17:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:14 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:14 smithi062 ceph-mon[95543]: pgmap v405: 97 pgs: 97 active+clean; 3.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 321 KiB/s wr, 85 op/s 2023-07-14T16:17:15.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:17:15.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:17:15.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:14 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:14 smithi081 ceph-mon[127795]: pgmap v405: 97 pgs: 97 active+clean; 3.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 321 KiB/s wr, 85 op/s 2023-07-14T16:17:15.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:15.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:17:15.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:17:15.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:14 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:17:17.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:16 smithi164 ceph-mon[97427]: pgmap v406: 97 pgs: 97 active+clean; 3.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 158 KiB/s wr, 57 op/s 2023-07-14T16:17:17.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:16 smithi062 ceph-mon[95543]: pgmap v406: 97 pgs: 97 active+clean; 3.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 158 KiB/s wr, 57 op/s 2023-07-14T16:17:17.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:16 smithi081 ceph-mon[127795]: pgmap v406: 97 pgs: 97 active+clean; 3.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 158 KiB/s wr, 57 op/s 2023-07-14T16:17:18.015 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/limits.h presence... no 2023-07-14T16:17:18.015 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/limits.h... no 2023-07-14T16:17:18.097 INFO:tasks.workunit.client.0.smithi062.stdout:checking for an ANSI C-conforming const... yes 2023-07-14T16:17:18.517 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-07-14T16:17:19.018 INFO:tasks.workunit.client.0.smithi062.stdout:checking for system... yes 2023-07-14T16:17:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:18 smithi164 ceph-mon[97427]: pgmap v407: 97 pgs: 97 active+clean; 3.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 144 KiB/s wr, 57 op/s 2023-07-14T16:17:19.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:18 smithi062 ceph-mon[95543]: pgmap v407: 97 pgs: 97 active+clean; 3.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 144 KiB/s wr, 57 op/s 2023-07-14T16:17:19.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:18 smithi081 ceph-mon[127795]: pgmap v407: 97 pgs: 97 active+clean; 3.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 144 KiB/s wr, 57 op/s 2023-07-14T16:17:19.162 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gettimeofday... yes 2023-07-14T16:17:19.300 INFO:tasks.workunit.client.0.smithi062.stdout:checking for mkdir... yes 2023-07-14T16:17:19.437 INFO:tasks.workunit.client.0.smithi062.stdout:checking for strerror... yes 2023-07-14T16:17:19.575 INFO:tasks.workunit.client.0.smithi062.stdout:checking for open64... yes 2023-07-14T16:17:19.713 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stat64... yes 2023-07-14T16:17:19.852 INFO:tasks.workunit.client.0.smithi062.stdout:checking for fseeko64... yes 2023-07-14T16:17:19.988 INFO:tasks.workunit.client.0.smithi062.stdout:checking for lrand48_r... yes 2023-07-14T16:17:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:19 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:17:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:19 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:17:20.127 INFO:tasks.workunit.client.0.smithi062.stdout:checking for srand48_r... yes 2023-07-14T16:17:20.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:19 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:17:20.176 INFO:tasks.workunit.client.0.smithi062.stdout:configure: creating ./config.status 2023-07-14T16:17:20.380 INFO:tasks.workunit.client.0.smithi062.stdout: /bin/sh ./config.status 2023-07-14T16:17:20.518 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating Makefile 2023-07-14T16:17:20.556 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating config.h 2023-07-14T16:17:20.604 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: executing depfiles commands 2023-07-14T16:17:20.680 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: executing default commands 2023-07-14T16:17:20.762 INFO:tasks.workunit.client.0.smithi062.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2023-07-14T16:17:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:20 smithi164 ceph-mon[97427]: pgmap v408: 97 pgs: 97 active+clean; 3.3 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 173 KiB/s wr, 56 op/s 2023-07-14T16:17:21.040 INFO:tasks.workunit.client.0.smithi062.stdout:rm -f stamp-h1 2023-07-14T16:17:21.044 INFO:tasks.workunit.client.0.smithi062.stdout:touch config.h.in 2023-07-14T16:17:21.049 INFO:tasks.workunit.client.0.smithi062.stdout:cd . && /bin/sh ./config.status config.h 2023-07-14T16:17:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:20 smithi062 ceph-mon[95543]: pgmap v408: 97 pgs: 97 active+clean; 3.3 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 173 KiB/s wr, 56 op/s 2023-07-14T16:17:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:20 smithi081 ceph-mon[127795]: pgmap v408: 97 pgs: 97 active+clean; 3.3 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 173 KiB/s wr, 56 op/s 2023-07-14T16:17:21.163 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating config.h 2023-07-14T16:17:21.226 INFO:tasks.workunit.client.0.smithi062.stdout:make all-am 2023-07-14T16:17:21.274 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2023-07-14T16:17:21.275 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2023-07-14T16:17:21.517 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2023-07-14T16:17:21.535 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2023-07-14T16:17:21.671 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2023-07-14T16:17:21.695 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2023-07-14T16:17:21.869 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/main.Tpo .deps/main.Po 2023-07-14T16:17:21.889 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2023-07-14T16:17:22.067 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2023-07-14T16:17:22.086 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2023-07-14T16:17:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.271+0000 7ff8473ff700 1 -- 172.21.15.62:0/1830509177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff840074550 msgr2=0x7ff8400729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.271+0000 7ff8473ff700 1 --2- 172.21.15.62:0/1830509177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff840074550 0x7ff8400729a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009990 tx=0x7ff83c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.272+0000 7ff8473ff700 1 -- 172.21.15.62:0/1830509177 shutdown_connections 2023-07-14T16:17:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.272+0000 7ff8473ff700 1 --2- 172.21.15.62:0/1830509177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff84010c0d0 0x7ff84010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.272+0000 7ff8473ff700 1 --2- 172.21.15.62:0/1830509177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff840074550 0x7ff8400729a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.272+0000 7ff8473ff700 1 --2- 172.21.15.62:0/1830509177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff840072f70 0x7ff8400733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.272+0000 7ff8473ff700 1 -- 172.21.15.62:0/1830509177 >> 172.21.15.62:0/1830509177 conn(0x7ff8400fb0e0 msgr2=0x7ff8400fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.272+0000 7ff8473ff700 1 -- 172.21.15.62:0/1830509177 shutdown_connections 2023-07-14T16:17:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.272+0000 7ff8473ff700 1 -- 172.21.15.62:0/1830509177 wait complete. 2023-07-14T16:17:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.273+0000 7ff8473ff700 1 Processor -- start 2023-07-14T16:17:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.273+0000 7ff8473ff700 1 -- start start 2023-07-14T16:17:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff8473ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff840072f70 0x7ff84019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff8473ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff840074550 0x7ff84019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff8473ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff84010c0d0 0x7ff84019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff8473ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84019d910 con 0x7ff840072f70 2023-07-14T16:17:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff8473ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84019da80 con 0x7ff84010c0d0 2023-07-14T16:17:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff8473ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84019dc40 con 0x7ff840074550 2023-07-14T16:17:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff84519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff840072f70 0x7ff84019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff84519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff840072f70 0x7ff84019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51306/0 (socket says 172.21.15.62:51306) 2023-07-14T16:17:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff84519b700 1 -- 172.21.15.62:0/2531481326 learned_addr learned my addr 172.21.15.62:0/2531481326 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.274+0000 7ff84599c700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff84010c0d0 0x7ff84019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff84499a700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff840074550 0x7ff84019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff84519b700 1 -- 172.21.15.62:0/2531481326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff840074550 msgr2=0x7ff84019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff84519b700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff840074550 0x7ff84019c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff84519b700 1 -- 172.21.15.62:0/2531481326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff84010c0d0 msgr2=0x7ff84019d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff84519b700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff84010c0d0 0x7ff84019d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff84519b700 1 -- 172.21.15.62:0/2531481326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c0095f0 con 0x7ff840072f70 2023-07-14T16:17:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff84519b700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff840072f70 0x7ff84019bef0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0098e0 tx=0x7ff83c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff8367fc700 1 -- 172.21.15.62:0/2531481326 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c01d070 con 0x7ff840072f70 2023-07-14T16:17:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff8367fc700 1 -- 172.21.15.62:0/2531481326 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff83c00be30 con 0x7ff840072f70 2023-07-14T16:17:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff8473ff700 1 -- 172.21.15.62:0/2531481326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84019dec0 con 0x7ff840072f70 2023-07-14T16:17:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.275+0000 7ff8367fc700 1 -- 172.21.15.62:0/2531481326 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c022df0 con 0x7ff840072f70 2023-07-14T16:17:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.276+0000 7ff8473ff700 1 -- 172.21.15.62:0/2531481326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8401a46e0 con 0x7ff840072f70 2023-07-14T16:17:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.279+0000 7ff8367fc700 1 -- 172.21.15.62:0/2531481326 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff83c022470 con 0x7ff840072f70 2023-07-14T16:17:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.280+0000 7ff85ace0b80 1 Processor -- start 2023-07-14T16:17:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff85ace0b80 1 -- start start 2023-07-14T16:17:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0124aec50 con 0x55c0125aa6e0 2023-07-14T16:17:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0125019e0 con 0x55c0125a9840 2023-07-14T16:17:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0124fae70 con 0x55c0124056b0 2023-07-14T16:17:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff84499a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff84499a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125aab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51322/0 (socket says 172.21.15.62:51322) 2023-07-14T16:17:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.281+0000 7ff84499a700 1 -- 172.21.15.62:0/2710742405 learned_addr learned my addr 172.21.15.62:0/2710742405 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff8367fc700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff82c0a11d0 0x7ff82c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84599c700 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84519b700 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff8367fc700 1 -- 172.21.15.62:0/2531481326 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff83c0cde10 con 0x7ff840072f70 2023-07-14T16:17:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84499a700 1 -- 172.21.15.62:0/2710742405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 msgr2=0x55c0125a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84599c700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff82c0a11d0 0x7ff82c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84499a700 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125a9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84499a700 1 -- 172.21.15.62:0/2710742405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 msgr2=0x55c0125aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84499a700 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84499a700 1 -- 172.21.15.62:0/2710742405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c012328d80 con 0x55c0125aa6e0 2023-07-14T16:17:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.282+0000 7ff84499a700 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125aab80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff830009cb0 tx=0x7ff8300113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e664ba662545f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.283+0000 7ff82b7fe700 1 -- 172.21.15.62:0/2710742405 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8300097b0 con 0x55c0125aa6e0 2023-07-14T16:17:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.283+0000 7ff84599c700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff82c0a11d0 0x7ff82c0a3680 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55c0125a9060 tx=0x7ff838004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.283+0000 7ff82b7fe700 1 -- 172.21.15.62:0/2710742405 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff830011c60 con 0x55c0125aa6e0 2023-07-14T16:17:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.283+0000 7ff82b7fe700 1 -- 172.21.15.62:0/2710742405 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83000f6a0 con 0x55c0125aa6e0 2023-07-14T16:17:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2710742405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 msgr2=0x55c0125aab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125aab80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff830009cb0 tx=0x7ff8300113e0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2710742405 shutdown_connections 2023-07-14T16:17:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125aab80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2710742405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2710742405 >> 172.21.15.62:0/2710742405 conn(0x55c01259ebe0 msgr2=0x55c0125a3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2710742405 shutdown_connections 2023-07-14T16:17:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2710742405 wait complete. 2023-07-14T16:17:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.284+0000 7ff85ace0b80 1 -- start start 2023-07-14T16:17:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.285+0000 7ff85ace0b80 1 Processor -- start 2023-07-14T16:17:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01249aa60 con 0x55c0125aa6e0 2023-07-14T16:17:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01249a760 con 0x55c0125a9840 2023-07-14T16:17:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01237e8c0 con 0x55c0124056b0 2023-07-14T16:17:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff85ace0b80 10 client.0 _ll_register_callbacks cb 0x55c012498700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff84599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff84499a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff84599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51338/0 (socket says 172.21.15.62:51338) 2023-07-14T16:17:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff84599c700 1 -- 172.21.15.62:0/2256894254 learned_addr learned my addr 172.21.15.62:0/2256894254 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff84519b700 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff84499a700 1 -- 172.21.15.62:0/2256894254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 msgr2=0x55c0125b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.286+0000 7ff84499a700 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff84499a700 1 -- 172.21.15.62:0/2256894254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 msgr2=0x55c0125b9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff84499a700 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff84499a700 1 -- 172.21.15.62:0/2256894254 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff830009910 con 0x55c0125a9840 2023-07-14T16:17:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff84599c700 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff84519b700 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff84499a700 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b2d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff83001c040 tx=0x7ff8300097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ffecf71c3f4ad4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff828ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:17:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff828ff9700 1 -- 172.21.15.62:0/2256894254 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830005050 con 0x55c0125a9840 2023-07-14T16:17:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2256894254 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c012454490 con 0x55c0125a9840 2023-07-14T16:17:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.287+0000 7ff828ff9700 1 -- 172.21.15.62:0/2256894254 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff8300051b0 con 0x55c0125a9840 2023-07-14T16:17:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff828ff9700 1 -- 172.21.15.62:0/2256894254 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff830005370 con 0x55c0125a9840 2023-07-14T16:17:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff85ace0b80 10 client.25685 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2256894254 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c012311c00 con 0x55c0125a9840 2023-07-14T16:17:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff828ff9700 1 -- 172.21.15.62:0/2256894254 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c012454490 con 0x55c0125a9840 2023-07-14T16:17:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff828ff9700 1 -- 172.21.15.62:0/2256894254 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff830022070 con 0x55c0125a9840 2023-07-14T16:17:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff85ace0b80 10 client.25685 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff85ace0b80 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff85ace0b80 20 client.25685 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c0125be670 0x55c0125c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff85ace0b80 4 client.25685 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.288+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2256894254 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0124f3e50 con 0x55c0125be670 2023-07-14T16:17:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.289+0000 7ff84519b700 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c0125be670 0x55c0125c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.289+0000 7ff84519b700 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c0125be670 0x55c0125c0a80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be91d924384942b8 server_cookie=42b29c562da27ddf in_seq=0 out_seq=0 2023-07-14T16:17:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.289+0000 7ff828ff9700 10 client.25685 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff828ff9700 1 -- 172.21.15.62:0/2256894254 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c0124f3e50 con 0x55c0125be670 2023-07-14T16:17:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff828ff9700 10 client.25685 handle_command_reply: tid=0 2023-07-14T16:17:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 client.25685 shutdown 2023-07-14T16:17:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2256894254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 msgr2=0x55c0125b2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b2d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff83001c040 tx=0x7ff8300097f0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2256894254 shutdown_connections 2023-07-14T16:17:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c0125be670 0x55c0125c0a80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/2256894254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b2d80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff828ff9700 0 client.25685 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2256894254 >> 172.21.15.62:0/2256894254 conn(0x55c01259ebe0 msgr2=0x55c0125a34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2256894254 shutdown_connections 2023-07-14T16:17:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 1 -- 172.21.15.62:0/2256894254 wait complete. 2023-07-14T16:17:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.290+0000 7ff85ace0b80 20 client.25685 trim_cache size 0 max 16384 2023-07-14T16:17:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.308+0000 7ff85ace0b80 1 Processor -- start 2023-07-14T16:17:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.308+0000 7ff85ace0b80 1 -- start start 2023-07-14T16:17:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0125a0d00 con 0x55c0125aa6e0 2023-07-14T16:17:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0125a54e0 con 0x55c0125a9840 2023-07-14T16:17:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0125b39c0 con 0x55c0124056b0 2023-07-14T16:17:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff84499a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff84599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff84499a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:55230/0 (socket says 172.21.15.62:55230) 2023-07-14T16:17:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff84499a700 1 -- 172.21.15.62:0/1475263695 learned_addr learned my addr 172.21.15.62:0/1475263695 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff84519b700 1 --2- 172.21.15.62:0/1475263695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff84499a700 1 -- 172.21.15.62:0/1475263695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 msgr2=0x55c0125ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.309+0000 7ff84499a700 1 --2- 172.21.15.62:0/1475263695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.310+0000 7ff84499a700 1 -- 172.21.15.62:0/1475263695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 msgr2=0x55c0125b9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.310+0000 7ff84499a700 1 --2- 172.21.15.62:0/1475263695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.310+0000 7ff84499a700 1 -- 172.21.15.62:0/1475263695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c01234f1f0 con 0x55c0125a9840 2023-07-14T16:17:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.310+0000 7ff84499a700 1 --2- 172.21.15.62:0/1475263695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125adad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff8300092a0 tx=0x7ff8300093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad893e6964c6027 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.310+0000 7ff82bfff700 1 -- 172.21.15.62:0/1475263695 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830024850 con 0x55c0125a9840 2023-07-14T16:17:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.310+0000 7ff82bfff700 1 -- 172.21.15.62:0/1475263695 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8300249b0 con 0x55c0125a9840 2023-07-14T16:17:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.310+0000 7ff82bfff700 1 -- 172.21.15.62:0/1475263695 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8300233b0 con 0x55c0125a9840 2023-07-14T16:17:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1475263695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 msgr2=0x55c0125adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1475263695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125adad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff8300092a0 tx=0x7ff8300093d0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1475263695 shutdown_connections 2023-07-14T16:17:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1475263695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1475263695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1475263695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125adad0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1475263695 >> 172.21.15.62:0/1475263695 conn(0x55c01259ebe0 msgr2=0x55c0125a3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1475263695 shutdown_connections 2023-07-14T16:17:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1475263695 wait complete. 2023-07-14T16:17:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.311+0000 7ff85ace0b80 1 -- start start 2023-07-14T16:17:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff85ace0b80 1 Processor -- start 2023-07-14T16:17:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff85ace0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0125b8f80 con 0x55c0125aa6e0 2023-07-14T16:17:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0125b90f0 con 0x55c0125a9840 2023-07-14T16:17:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff85ace0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0125b92b0 con 0x55c0124056b0 2023-07-14T16:17:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff85ace0b80 10 client.0 _ll_register_callbacks cb 0x7ff83c09ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff84599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff84599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51364/0 (socket says 172.21.15.62:51364) 2023-07-14T16:17:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff84599c700 1 -- 172.21.15.62:0/1777053270 learned_addr learned my addr 172.21.15.62:0/1777053270 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff84519b700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.312+0000 7ff84499a700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff84599c700 1 -- 172.21.15.62:0/1777053270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 msgr2=0x55c0125b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff84599c700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff84599c700 1 -- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 msgr2=0x55c0125b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff84599c700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff84599c700 1 -- 172.21.15.62:0/1777053270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c012311c00 con 0x55c0125aa6e0 2023-07-14T16:17:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff84519b700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff84499a700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff84599c700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b8a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff838004a60 tx=0x7ff838004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc158b8e04c0847 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff828ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff828ff9700 1 -- 172.21.15.62:0/1777053270 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83800e040 con 0x55c0125aa6e0 2023-07-14T16:17:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.313+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1777053270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff830011c60 con 0x55c0125aa6e0 2023-07-14T16:17:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff828ff9700 1 -- 172.21.15.62:0/1777053270 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff838004bb0 con 0x55c0125aa6e0 2023-07-14T16:17:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff828ff9700 1 -- 172.21.15.62:0/1777053270 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff838004d70 con 0x55c0125aa6e0 2023-07-14T16:17:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff85ace0b80 10 client.15933 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1777053270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c012327250 con 0x55c0125aa6e0 2023-07-14T16:17:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff828ff9700 1 -- 172.21.15.62:0/1777053270 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff830011c60 con 0x55c0125aa6e0 2023-07-14T16:17:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff828ff9700 1 -- 172.21.15.62:0/1777053270 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff838025050 con 0x55c0125aa6e0 2023-07-14T16:17:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff85ace0b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff85ace0b80 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff85ace0b80 20 client.15933 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c012612f50 0x55c012615360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff85ace0b80 4 client.15933 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:17:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.314+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1777053270 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c0124b5ff0 con 0x55c012612f50 2023-07-14T16:17:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.315+0000 7ff84519b700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c012612f50 0x55c012615360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.315+0000 7ff84519b700 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c012612f50 0x55c012615360 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64efd23e65ea2fb3 server_cookie=fcd717b6275313be in_seq=0 out_seq=0 2023-07-14T16:17:22.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.315+0000 7ff828ff9700 10 client.15933 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:22.311 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2023-07-14T16:17:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff828ff9700 1 -- 172.21.15.62:0/1777053270 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c0124b5ff0 con 0x55c012612f50 2023-07-14T16:17:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff828ff9700 10 client.15933 handle_command_reply: tid=0 2023-07-14T16:17:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 client.15933 shutdown 2023-07-14T16:17:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1777053270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 msgr2=0x55c0125b8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b8a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff838004a60 tx=0x7ff838004ad0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1777053270 shutdown_connections 2023-07-14T16:17:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0125aa6e0 0x55c0125b8a40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c0124056b0 0x55c0125b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c012612f50 0x55c012615360 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 --2- 172.21.15.62:0/1777053270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c0125a9840 0x55c0125b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1777053270 >> 172.21.15.62:0/1777053270 conn(0x55c01259ebe0 msgr2=0x55c0125b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff828ff9700 0 client.15933 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1777053270 shutdown_connections 2023-07-14T16:17:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.364+0000 7ff85ace0b80 1 -- 172.21.15.62:0/1777053270 wait complete. 2023-07-14T16:17:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff85ace0b80 20 client.15933 trim_cache size 0 max 16384 2023-07-14T16:17:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:17:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:17:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:17:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:17:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff8473ff700 1 -- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff82c0a11d0 msgr2=0x7ff82c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff8473ff700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff82c0a11d0 0x7ff82c0a3680 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55c0125a9060 tx=0x7ff838004b40 comp rx=0 tx=0).stop 2023-07-14T16:17:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff8473ff700 1 -- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff840072f70 msgr2=0x7ff84019bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff8473ff700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff840072f70 0x7ff84019bef0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0098e0 tx=0x7ff83c004730 comp rx=0 tx=0).stop 2023-07-14T16:17:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff8473ff700 1 -- 172.21.15.62:0/2531481326 shutdown_connections 2023-07-14T16:17:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff8473ff700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff82c0a11d0 0x7ff82c0a3680 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff8473ff700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff84010c0d0 0x7ff84019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.365+0000 7ff8473ff700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff840072f70 0x7ff84019bef0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.366+0000 7ff8473ff700 1 --2- 172.21.15.62:0/2531481326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff840074550 0x7ff84019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.366+0000 7ff8473ff700 1 -- 172.21.15.62:0/2531481326 >> 172.21.15.62:0/2531481326 conn(0x7ff8400fb0e0 msgr2=0x7ff8400fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.366+0000 7ff8473ff700 1 -- 172.21.15.62:0/2531481326 shutdown_connections 2023-07-14T16:17:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:22.366+0000 7ff8473ff700 1 -- 172.21.15.62:0/2531481326 wait complete. 2023-07-14T16:17:22.348 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2023-07-14T16:17:22.470 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2023-07-14T16:17:22.487 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2023-07-14T16:17:22.582 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2023-07-14T16:17:22.601 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2023-07-14T16:17:22.714 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2023-07-14T16:17:22.732 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2023-07-14T16:17:22.962 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2023-07-14T16:17:22.982 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2023-07-14T16:17:23.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:22 smithi164 ceph-mon[97427]: pgmap v409: 97 pgs: 97 active+clean; 3.3 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 130 KiB/s wr, 53 op/s 2023-07-14T16:17:23.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:22 smithi062 ceph-mon[95543]: pgmap v409: 97 pgs: 97 active+clean; 3.3 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 130 KiB/s wr, 53 op/s 2023-07-14T16:17:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:22 smithi081 ceph-mon[127795]: pgmap v409: 97 pgs: 97 active+clean; 3.3 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 130 KiB/s wr, 53 op/s 2023-07-14T16:17:23.173 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2023-07-14T16:17:23.192 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2023-07-14T16:17:23.421 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2023-07-14T16:17:23.439 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2023-07-14T16:17:23.577 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2023-07-14T16:17:23.597 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2023-07-14T16:17:23.796 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2023-07-14T16:17:23.815 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2023-07-14T16:17:24.562 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/util.Tpo .deps/util.Po 2023-07-14T16:17:24.583 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2023-07-14T16:17:24.660 INFO:tasks.workunit.client.0.smithi062.stderr:In file included from parser.h:22, 2023-07-14T16:17:24.660 INFO:tasks.workunit.client.0.smithi062.stderr: from parser.c:25: 2023-07-14T16:17:24.660 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c: In function ‘init_filesys’: 2023-07-14T16:17:24.664 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-07-14T16:17:24.665 INFO:tasks.workunit.client.0.smithi062.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-07-14T16:17:24.665 INFO:tasks.workunit.client.0.smithi062.stderr: ^ 2023-07-14T16:17:24.665 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-07-14T16:17:24.665 INFO:tasks.workunit.client.0.smithi062.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-07-14T16:17:24.665 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~ 2023-07-14T16:17:24.665 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-07-14T16:17:24.666 INFO:tasks.workunit.client.0.smithi062.stderr: container_of(ptr, type, member) 2023-07-14T16:17:24.666 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~ 2023-07-14T16:17:24.666 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-07-14T16:17:24.666 INFO:tasks.workunit.client.0.smithi062.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-07-14T16:17:24.666 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~ 2023-07-14T16:17:24.667 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2023-07-14T16:17:24.667 INFO:tasks.workunit.client.0.smithi062.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2023-07-14T16:17:24.667 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.667 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-07-14T16:17:24.667 INFO:tasks.workunit.client.0.smithi062.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-07-14T16:17:24.667 INFO:tasks.workunit.client.0.smithi062.stderr: ^ 2023-07-14T16:17:24.667 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-07-14T16:17:24.668 INFO:tasks.workunit.client.0.smithi062.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-07-14T16:17:24.668 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~ 2023-07-14T16:17:24.668 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-07-14T16:17:24.668 INFO:tasks.workunit.client.0.smithi062.stderr: container_of(ptr, type, member) 2023-07-14T16:17:24.668 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~ 2023-07-14T16:17:24.668 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-07-14T16:17:24.668 INFO:tasks.workunit.client.0.smithi062.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-07-14T16:17:24.669 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~ 2023-07-14T16:17:24.669 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2023-07-14T16:17:24.669 INFO:tasks.workunit.client.0.smithi062.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2023-07-14T16:17:24.669 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.669 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-07-14T16:17:24.669 INFO:tasks.workunit.client.0.smithi062.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-07-14T16:17:24.669 INFO:tasks.workunit.client.0.smithi062.stderr: ^ 2023-07-14T16:17:24.669 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-07-14T16:17:24.670 INFO:tasks.workunit.client.0.smithi062.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-07-14T16:17:24.670 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~ 2023-07-14T16:17:24.670 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-07-14T16:17:24.670 INFO:tasks.workunit.client.0.smithi062.stderr: container_of(ptr, type, member) 2023-07-14T16:17:24.670 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~ 2023-07-14T16:17:24.670 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-07-14T16:17:24.670 INFO:tasks.workunit.client.0.smithi062.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-07-14T16:17:24.670 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~ 2023-07-14T16:17:24.671 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2023-07-14T16:17:24.671 INFO:tasks.workunit.client.0.smithi062.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2023-07-14T16:17:24.671 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.671 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-07-14T16:17:24.671 INFO:tasks.workunit.client.0.smithi062.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-07-14T16:17:24.671 INFO:tasks.workunit.client.0.smithi062.stderr: ^ 2023-07-14T16:17:24.671 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-07-14T16:17:24.671 INFO:tasks.workunit.client.0.smithi062.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-07-14T16:17:24.672 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~ 2023-07-14T16:17:24.672 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-07-14T16:17:24.672 INFO:tasks.workunit.client.0.smithi062.stderr: container_of(ptr, type, member) 2023-07-14T16:17:24.672 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~ 2023-07-14T16:17:24.672 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-07-14T16:17:24.672 INFO:tasks.workunit.client.0.smithi062.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-07-14T16:17:24.672 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~ 2023-07-14T16:17:24.673 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2023-07-14T16:17:24.673 INFO:tasks.workunit.client.0.smithi062.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2023-07-14T16:17:24.673 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.673 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c: In function ‘init_tg_stats’: 2023-07-14T16:17:24.673 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-07-14T16:17:24.674 INFO:tasks.workunit.client.0.smithi062.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-07-14T16:17:24.674 INFO:tasks.workunit.client.0.smithi062.stderr: ^ 2023-07-14T16:17:24.674 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-07-14T16:17:24.674 INFO:tasks.workunit.client.0.smithi062.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-07-14T16:17:24.674 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~ 2023-07-14T16:17:24.674 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-07-14T16:17:24.674 INFO:tasks.workunit.client.0.smithi062.stderr: container_of(ptr, type, member) 2023-07-14T16:17:24.675 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~ 2023-07-14T16:17:24.675 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-07-14T16:17:24.675 INFO:tasks.workunit.client.0.smithi062.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-07-14T16:17:24.675 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~ 2023-07-14T16:17:24.675 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2023-07-14T16:17:24.675 INFO:tasks.workunit.client.0.smithi062.stderr: list_for_each_entry(tmp_list, 2023-07-14T16:17:24.675 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.675 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-07-14T16:17:24.676 INFO:tasks.workunit.client.0.smithi062.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-07-14T16:17:24.676 INFO:tasks.workunit.client.0.smithi062.stderr: ^ 2023-07-14T16:17:24.676 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-07-14T16:17:24.676 INFO:tasks.workunit.client.0.smithi062.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-07-14T16:17:24.676 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~ 2023-07-14T16:17:24.676 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-07-14T16:17:24.676 INFO:tasks.workunit.client.0.smithi062.stderr: container_of(ptr, type, member) 2023-07-14T16:17:24.677 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~ 2023-07-14T16:17:24.677 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-07-14T16:17:24.677 INFO:tasks.workunit.client.0.smithi062.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-07-14T16:17:24.677 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~ 2023-07-14T16:17:24.677 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2023-07-14T16:17:24.677 INFO:tasks.workunit.client.0.smithi062.stderr: list_for_each_entry(tmp_list, 2023-07-14T16:17:24.677 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.677 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-07-14T16:17:24.678 INFO:tasks.workunit.client.0.smithi062.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-07-14T16:17:24.678 INFO:tasks.workunit.client.0.smithi062.stderr: ^ 2023-07-14T16:17:24.678 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-07-14T16:17:24.678 INFO:tasks.workunit.client.0.smithi062.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-07-14T16:17:24.678 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~ 2023-07-14T16:17:24.678 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-07-14T16:17:24.678 INFO:tasks.workunit.client.0.smithi062.stderr: container_of(ptr, type, member) 2023-07-14T16:17:24.678 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~ 2023-07-14T16:17:24.679 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2023-07-14T16:17:24.679 INFO:tasks.workunit.client.0.smithi062.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2023-07-14T16:17:24.679 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~ 2023-07-14T16:17:24.679 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2023-07-14T16:17:24.679 INFO:tasks.workunit.client.0.smithi062.stderr: list_for_each_entry(tmp_list, 2023-07-14T16:17:24.679 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.679 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2023-07-14T16:17:24.679 INFO:tasks.workunit.client.0.smithi062.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2023-07-14T16:17:24.680 INFO:tasks.workunit.client.0.smithi062.stderr: ^ 2023-07-14T16:17:24.680 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2023-07-14T16:17:24.680 INFO:tasks.workunit.client.0.smithi062.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2023-07-14T16:17:24.680 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~ 2023-07-14T16:17:24.680 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2023-07-14T16:17:24.680 INFO:tasks.workunit.client.0.smithi062.stderr: container_of(ptr, type, member) 2023-07-14T16:17:24.680 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~ 2023-07-14T16:17:24.680 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2023-07-14T16:17:24.681 INFO:tasks.workunit.client.0.smithi062.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2023-07-14T16:17:24.681 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~ 2023-07-14T16:17:24.681 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2023-07-14T16:17:24.681 INFO:tasks.workunit.client.0.smithi062.stderr: list_for_each_entry(tmp_list, 2023-07-14T16:17:24.681 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.786 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c: In function ‘set_weight’: 2023-07-14T16:17:24.786 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:623:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2023-07-14T16:17:24.787 INFO:tasks.workunit.client.0.smithi062.stderr: strncpy (op, tmp_config->name, len - 7); 2023-07-14T16:17:24.787 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.787 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:620:10: note: length computed here 2023-07-14T16:17:24.787 INFO:tasks.workunit.client.0.smithi062.stderr: len = strlen(tmp_config->name); 2023-07-14T16:17:24.787 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.791 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c: In function ‘get_weight_total’: 2023-07-14T16:17:24.791 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:645:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2023-07-14T16:17:24.792 INFO:tasks.workunit.client.0.smithi062.stderr: strncpy (op, tmp_config->name, len - 7); 2023-07-14T16:17:24.792 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.792 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:642:10: note: length computed here 2023-07-14T16:17:24.792 INFO:tasks.workunit.client.0.smithi062.stderr: len = strlen(tmp_config->name); 2023-07-14T16:17:24.792 INFO:tasks.workunit.client.0.smithi062.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-07-14T16:17:24.959 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2023-07-14T16:17:24.980 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2023-07-14T16:17:25.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:24 smithi164 ceph-mon[97427]: pgmap v410: 97 pgs: 97 active+clean; 4.6 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 251 KiB/s wr, 67 op/s 2023-07-14T16:17:25.109 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2023-07-14T16:17:25.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:24 smithi062 ceph-mon[95543]: pgmap v410: 97 pgs: 97 active+clean; 4.6 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 251 KiB/s wr, 67 op/s 2023-07-14T16:17:25.127 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2023-07-14T16:17:25.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:24 smithi081 ceph-mon[127795]: pgmap v410: 97 pgs: 97 active+clean; 4.6 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 251 KiB/s wr, 67 op/s 2023-07-14T16:17:25.285 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2023-07-14T16:17:25.306 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2023-07-14T16:17:25.382 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/list.Tpo .deps/list.Po 2023-07-14T16:17:25.402 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2023-07-14T16:17:25.604 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2023-07-14T16:17:25.606 INFO:tasks.workunit.client.0.smithi062.stderr:+ cd .. 2023-07-14T16:17:25.609 INFO:tasks.workunit.client.0.smithi062.stderr:+ mkdir tmp 2023-07-14T16:17:25.615 INFO:tasks.workunit.client.0.smithi062.stderr:+ cd tmp 2023-07-14T16:17:25.617 INFO:tasks.workunit.client.0.smithi062.stderr:+ for f in $mydir/*.ffsb 2023-07-14T16:17:25.617 INFO:tasks.workunit.client.0.smithi062.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2023-07-14T16:17:25.619 INFO:tasks.workunit.client.0.smithi062.stdout:FFSB version 6.0-RC2 started 2023-07-14T16:17:25.620 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:17:25.626 INFO:tasks.workunit.client.0.smithi062.stdout:benchmark time = 300 2023-07-14T16:17:25.626 INFO:tasks.workunit.client.0.smithi062.stdout:ThreadGroup 0 2023-07-14T16:17:25.626 INFO:tasks.workunit.client.0.smithi062.stdout:================ 2023-07-14T16:17:25.626 INFO:tasks.workunit.client.0.smithi062.stdout: num_threads = 32 2023-07-14T16:17:25.627 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:17:25.627 INFO:tasks.workunit.client.0.smithi062.stdout: read_random = off 2023-07-14T16:17:25.627 INFO:tasks.workunit.client.0.smithi062.stdout: read_size = 0 (0B) 2023-07-14T16:17:25.627 INFO:tasks.workunit.client.0.smithi062.stdout: read_blocksize = 0 (0B) 2023-07-14T16:17:25.627 INFO:tasks.workunit.client.0.smithi062.stdout: read_skip = off 2023-07-14T16:17:25.627 INFO:tasks.workunit.client.0.smithi062.stdout: read_skipsize = 0 (0B) 2023-07-14T16:17:25.627 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:17:25.627 INFO:tasks.workunit.client.0.smithi062.stdout: write_random = on 2023-07-14T16:17:25.628 INFO:tasks.workunit.client.0.smithi062.stdout: write_size = 5242880 (5MB) 2023-07-14T16:17:25.628 INFO:tasks.workunit.client.0.smithi062.stdout: fsync_file = 0 2023-07-14T16:17:25.628 INFO:tasks.workunit.client.0.smithi062.stdout: write_blocksize = 4096 (4KB) 2023-07-14T16:17:25.628 INFO:tasks.workunit.client.0.smithi062.stdout: wait time = 0 2023-07-14T16:17:25.628 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:17:25.628 INFO:tasks.workunit.client.0.smithi062.stdout: op weights 2023-07-14T16:17:25.628 INFO:tasks.workunit.client.0.smithi062.stdout: read = 0 (0.00%) 2023-07-14T16:17:25.629 INFO:tasks.workunit.client.0.smithi062.stdout: readall = 0 (0.00%) 2023-07-14T16:17:25.629 INFO:tasks.workunit.client.0.smithi062.stdout: write = 1 (100.00%) 2023-07-14T16:17:25.629 INFO:tasks.workunit.client.0.smithi062.stdout: create = 0 (0.00%) 2023-07-14T16:17:25.629 INFO:tasks.workunit.client.0.smithi062.stdout: append = 0 (0.00%) 2023-07-14T16:17:25.629 INFO:tasks.workunit.client.0.smithi062.stdout: delete = 0 (0.00%) 2023-07-14T16:17:25.629 INFO:tasks.workunit.client.0.smithi062.stdout: metaop = 0 (0.00%) 2023-07-14T16:17:25.629 INFO:tasks.workunit.client.0.smithi062.stdout: createdir = 0 (0.00%) 2023-07-14T16:17:25.629 INFO:tasks.workunit.client.0.smithi062.stdout: stat = 0 (0.00%) 2023-07-14T16:17:25.630 INFO:tasks.workunit.client.0.smithi062.stdout: writeall = 0 (0.00%) 2023-07-14T16:17:25.630 INFO:tasks.workunit.client.0.smithi062.stdout: writeall_fsync = 0 (0.00%) 2023-07-14T16:17:25.630 INFO:tasks.workunit.client.0.smithi062.stdout: open_close = 0 (0.00%) 2023-07-14T16:17:25.630 INFO:tasks.workunit.client.0.smithi062.stdout: write_fsync = 0 (0.00%) 2023-07-14T16:17:25.630 INFO:tasks.workunit.client.0.smithi062.stdout: create_fsync = 0 (0.00%) 2023-07-14T16:17:25.630 INFO:tasks.workunit.client.0.smithi062.stdout: append_fsync = 0 (0.00%) 2023-07-14T16:17:25.630 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:17:25.630 INFO:tasks.workunit.client.0.smithi062.stdout:FileSystem . 2023-07-14T16:17:25.631 INFO:tasks.workunit.client.0.smithi062.stdout:========== 2023-07-14T16:17:25.631 INFO:tasks.workunit.client.0.smithi062.stdout: num_dirs = 0 2023-07-14T16:17:25.631 INFO:tasks.workunit.client.0.smithi062.stdout: starting files = 128 2023-07-14T16:17:25.631 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:17:25.632 INFO:tasks.workunit.client.0.smithi062.stdout: min file size = 104857600 (100MB) 2023-07-14T16:17:25.632 INFO:tasks.workunit.client.0.smithi062.stdout: max file size = 104857600 (100MB) 2023-07-14T16:17:25.632 INFO:tasks.workunit.client.0.smithi062.stdout: directio = off 2023-07-14T16:17:25.632 INFO:tasks.workunit.client.0.smithi062.stdout: alignedio = on 2023-07-14T16:17:25.632 INFO:tasks.workunit.client.0.smithi062.stdout: bufferedio = off 2023-07-14T16:17:25.632 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:17:25.632 INFO:tasks.workunit.client.0.smithi062.stdout: aging is off 2023-07-14T16:17:25.634 INFO:tasks.workunit.client.0.smithi062.stdout: current utilization = 0.00% 2023-07-14T16:17:25.634 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:17:25.634 INFO:tasks.workunit.client.0.smithi062.stdout:checking existing fs: . 2023-07-14T16:17:25.634 INFO:tasks.workunit.client.0.smithi062.stdout:recreating new fileset 2023-07-14T16:17:25.634 INFO:tasks.workunit.client.0.smithi062.stderr:opendir: No such file or directory 2023-07-14T16:17:27.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:26 smithi164 ceph-mon[97427]: pgmap v411: 97 pgs: 97 active+clean; 4.8 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 218 KiB/s wr, 62 op/s 2023-07-14T16:17:27.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:26 smithi062 ceph-mon[95543]: pgmap v411: 97 pgs: 97 active+clean; 4.8 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 218 KiB/s wr, 62 op/s 2023-07-14T16:17:27.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:26 smithi081 ceph-mon[127795]: pgmap v411: 97 pgs: 97 active+clean; 4.8 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 218 KiB/s wr, 62 op/s 2023-07-14T16:17:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:28 smithi164 ceph-mon[97427]: pgmap v412: 97 pgs: 97 active+clean; 9.6 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 630 KiB/s wr, 69 op/s 2023-07-14T16:17:29.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:28 smithi062 ceph-mon[95543]: pgmap v412: 97 pgs: 97 active+clean; 9.6 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 630 KiB/s wr, 69 op/s 2023-07-14T16:17:29.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:28 smithi081 ceph-mon[127795]: pgmap v412: 97 pgs: 97 active+clean; 9.6 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 630 KiB/s wr, 69 op/s 2023-07-14T16:17:29.735 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:17:29.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 -- 172.21.15.62:0/58313023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c072ef0 msgr2=0x7f326c073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:29.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 --2- 172.21.15.62:0/58313023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c072ef0 0x7f326c073360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3260009990 tx=0x7f3260009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 -- 172.21.15.62:0/58313023 shutdown_connections 2023-07-14T16:17:29.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 --2- 172.21.15.62:0/58313023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f326c1079b0 0x7f326c109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 --2- 172.21.15.62:0/58313023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f326c0744d0 0x7f326c072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 --2- 172.21.15.62:0/58313023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c072ef0 0x7f326c073360 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 -- 172.21.15.62:0/58313023 >> 172.21.15.62:0/58313023 conn(0x7f326c0fb120 msgr2=0x7f326c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:29.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 -- 172.21.15.62:0/58313023 shutdown_connections 2023-07-14T16:17:29.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.992+0000 7f32725ff700 1 -- 172.21.15.62:0/58313023 wait complete. 2023-07-14T16:17:29.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.994+0000 7f32725ff700 1 Processor -- start 2023-07-14T16:17:29.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.994+0000 7f32725ff700 1 -- start start 2023-07-14T16:17:29.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.994+0000 7f32725ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f326c072ef0 0x7f326c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.994+0000 7f32725ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c0744d0 0x7f326c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f32725ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f326c1079b0 0x7f326c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f32725ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c19f540 con 0x7f326c0744d0 2023-07-14T16:17:29.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f32725ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c19f680 con 0x7f326c1079b0 2023-07-14T16:17:29.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f32725ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c19f810 con 0x7f326c072ef0 2023-07-14T16:17:29.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f326c072ef0 0x7f326c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c0744d0 0x7f326c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f326c072ef0 0x7f326c1979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34866/0 (socket says 172.21.15.62:34866) 2023-07-14T16:17:29.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326bfff700 1 -- 172.21.15.62:0/2096004521 learned_addr learned my addr 172.21.15.62:0/2096004521 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:29.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f3270b9c700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f326c1079b0 0x7f326c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326b7fe700 1 -- 172.21.15.62:0/2096004521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f326c072ef0 msgr2=0x7f326c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:29.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326b7fe700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f326c072ef0 0x7f326c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326b7fe700 1 -- 172.21.15.62:0/2096004521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f326c1079b0 msgr2=0x7f326c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:29.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326b7fe700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f326c1079b0 0x7f326c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326b7fe700 1 -- 172.21.15.62:0/2096004521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600095f0 con 0x7f326c0744d0 2023-07-14T16:17:29.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.995+0000 7f326bfff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f326c072ef0 0x7f326c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:29.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.996+0000 7f326b7fe700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c0744d0 0x7f326c197f00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f32600098e0 tx=0x7f3260005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:29.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.996+0000 7f32697fa700 1 -- 172.21.15.62:0/2096004521 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326001d070 con 0x7f326c0744d0 2023-07-14T16:17:29.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.996+0000 7f32697fa700 1 -- 172.21.15.62:0/2096004521 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f326000beb0 con 0x7f326c0744d0 2023-07-14T16:17:29.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.996+0000 7f32725ff700 1 -- 172.21.15.62:0/2096004521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c19fa90 con 0x7f326c0744d0 2023-07-14T16:17:29.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.996+0000 7f32697fa700 1 -- 172.21.15.62:0/2096004521 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260022de0 con 0x7f326c0744d0 2023-07-14T16:17:29.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:29.996+0000 7f32725ff700 1 -- 172.21.15.62:0/2096004521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c19feb0 con 0x7f326c0744d0 2023-07-14T16:17:29.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.000+0000 7f32697fa700 1 -- 172.21.15.62:0/2096004521 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f32600225e0 con 0x7f326c0744d0 2023-07-14T16:17:29.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.002+0000 7f32697fa700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32540a1150 0x7f32540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.002+0000 7f3285f5ab80 1 Processor -- start 2023-07-14T16:17:29.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.002+0000 7f32697fa700 1 -- 172.21.15.62:0/2096004521 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f32600cdc90 con 0x7f326c0744d0 2023-07-14T16:17:29.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f3285f5ab80 1 -- start start 2023-07-14T16:17:29.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f326bfff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32540a1150 0x7f32540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fc4d0 0x563b022fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fce20 0x563b022fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 0x563b022fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02201cb0 con 0x563b022fce20 2023-07-14T16:17:29.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02254a40 con 0x563b022fc4d0 2023-07-14T16:17:29.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0224ded0 con 0x563b022fdc70 2023-07-14T16:17:29.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f326b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fce20 0x563b022fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f326b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fce20 0x563b022fd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46046/0 (socket says 172.21.15.62:46046) 2023-07-14T16:17:29.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f326b7fe700 1 -- 172.21.15.62:0/2659496308 learned_addr learned my addr 172.21.15.62:0/2659496308 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:29.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.003+0000 7f326bfff700 1 --2- 172.21.15.62:0/2659496308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fc4d0 0x563b022fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f326bfff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32540a1150 0x7f32540a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f325c005950 tx=0x7f325c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:29.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f326b7fe700 1 -- 172.21.15.62:0/2659496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 msgr2=0x563b022fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:29.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f326b7fe700 1 --2- 172.21.15.62:0/2659496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 0x563b022fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f326b7fe700 1 -- 172.21.15.62:0/2659496308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fc4d0 msgr2=0x563b022fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:29.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f326b7fe700 1 --2- 172.21.15.62:0/2659496308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fc4d0 0x563b022fc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f326b7fe700 1 -- 172.21.15.62:0/2659496308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b0207bde0 con 0x563b022fce20 2023-07-14T16:17:29.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f326b7fe700 1 --2- 172.21.15.62:0/2659496308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fce20 0x563b022fd6e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f326009b0d0 tx=0x7f3260005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bec553401eca883 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:29.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f32527fc700 1 -- 172.21.15.62:0/2659496308 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600b2070 con 0x563b022fce20 2023-07-14T16:17:29.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.004+0000 7f32527fc700 1 -- 172.21.15.62:0/2659496308 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f326009f170 con 0x563b022fce20 2023-07-14T16:17:29.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f32527fc700 1 -- 172.21.15.62:0/2659496308 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326009a870 con 0x563b022fce20 2023-07-14T16:17:29.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f3285f5ab80 1 -- 172.21.15.62:0/2659496308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fce20 msgr2=0x563b022fd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:29.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/2659496308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fce20 0x563b022fd6e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f326009b0d0 tx=0x7f3260005ad0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f3285f5ab80 1 -- 172.21.15.62:0/2659496308 shutdown_connections 2023-07-14T16:17:29.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/2659496308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 0x563b022fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/2659496308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fc4d0 0x563b022fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/2659496308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fce20 0x563b022fd6e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f3285f5ab80 1 -- 172.21.15.62:0/2659496308 >> 172.21.15.62:0/2659496308 conn(0x563b022f1da0 msgr2=0x563b022f71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:29.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.005+0000 7f3285f5ab80 1 -- 172.21.15.62:0/2659496308 shutdown_connections 2023-07-14T16:17:29.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.006+0000 7f3285f5ab80 1 -- 172.21.15.62:0/2659496308 wait complete. 2023-07-14T16:17:29.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.006+0000 7f3285f5ab80 1 -- start start 2023-07-14T16:17:29.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.006+0000 7f3285f5ab80 1 Processor -- start 2023-07-14T16:17:29.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fc4d0 0x563b02305dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b02306300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fdc70 0x563b0230d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b021edac0 con 0x563b022fdc70 2023-07-14T16:17:29.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b021ed7c0 con 0x563b022fce20 2023-07-14T16:17:29.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b020d1920 con 0x563b022fc4d0 2023-07-14T16:17:29.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3285f5ab80 10 client.0 _ll_register_callbacks cb 0x563b021eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:29.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3270b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fdc70 0x563b0230d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3270b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fdc70 0x563b0230d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46052/0 (socket says 172.21.15.62:46052) 2023-07-14T16:17:29.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f3270b9c700 1 -- 172.21.15.62:0/272275380 learned_addr learned my addr 172.21.15.62:0/272275380 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:29.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f326b7fe700 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b02306300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.007+0000 7f326bfff700 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fc4d0 0x563b02305dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f3270b9c700 1 -- 172.21.15.62:0/272275380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fc4d0 msgr2=0x563b02305dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:29.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f3270b9c700 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fc4d0 0x563b02305dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f3270b9c700 1 -- 172.21.15.62:0/272275380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 msgr2=0x563b02306300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:29.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f3270b9c700 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b02306300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:29.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f3270b9c700 1 -- 172.21.15.62:0/272275380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c19feb0 con 0x563b022fdc70 2023-07-14T16:17:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f326bfff700 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fc4d0 0x563b02305dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f3270b9c700 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fdc70 0x563b0230d340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3264011ce0 tx=0x7f326400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c471ab66f3b77b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f324b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f324b7fe700 1 -- 172.21.15.62:0/272275380 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400e040 con 0x563b022fdc70 2023-07-14T16:17:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f3285f5ab80 1 -- 172.21.15.62:0/272275380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b021a74f0 con 0x563b022fdc70 2023-07-14T16:17:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.008+0000 7f324b7fe700 1 -- 172.21.15.62:0/272275380 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3264003a00 con 0x563b022fdc70 2023-07-14T16:17:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f324b7fe700 1 -- 172.21.15.62:0/272275380 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3264003bc0 con 0x563b022fdc70 2023-07-14T16:17:29.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f3285f5ab80 10 client.15945 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:29.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f3285f5ab80 1 -- 172.21.15.62:0/272275380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563b02064c60 con 0x563b022fdc70 2023-07-14T16:17:29.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f324b7fe700 1 -- 172.21.15.62:0/272275380 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563b021a74f0 con 0x563b022fdc70 2023-07-14T16:17:29.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f324b7fe700 1 -- 172.21.15.62:0/272275380 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3264007ab0 con 0x563b022fdc70 2023-07-14T16:17:29.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f3285f5ab80 10 client.15945 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:29.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f3285f5ab80 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:29.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f3285f5ab80 20 client.15945 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:29.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b02311c20 0x563b02314030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:29.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f3285f5ab80 4 client.15945 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:29.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.009+0000 7f3285f5ab80 1 -- 172.21.15.62:0/272275380 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b02246eb0 con 0x563b02311c20 2023-07-14T16:17:29.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.010+0000 7f326bfff700 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b02311c20 0x563b02314030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:29.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.011+0000 7f324b7fe700 10 client.15945 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:29.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.011+0000 7f326bfff700 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b02311c20 0x563b02314030 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9a80478a511cea server_cookie=a8e1eae2ec82c90a in_seq=0 out_seq=0 2023-07-14T16:17:29.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f324b7fe700 1 -- 172.21.15.62:0/272275380 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b02246eb0 con 0x563b02311c20 2023-07-14T16:17:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f324b7fe700 10 client.15945 handle_command_reply: tid=0 2023-07-14T16:17:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 client.15945 shutdown 2023-07-14T16:17:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 -- 172.21.15.62:0/272275380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fdc70 msgr2=0x563b0230d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fdc70 0x563b0230d340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3264011ce0 tx=0x7f326400f1b0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 -- 172.21.15.62:0/272275380 shutdown_connections 2023-07-14T16:17:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fdc70 0x563b0230d340 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fc4d0 0x563b02305dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b02311c20 0x563b02314030 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/272275380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b02306300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f324b7fe700 0 client.15945 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:30.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 -- 172.21.15.62:0/272275380 >> 172.21.15.62:0/272275380 conn(0x563b022f1da0 msgr2=0x563b022f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 -- 172.21.15.62:0/272275380 shutdown_connections 2023-07-14T16:17:30.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.012+0000 7f3285f5ab80 1 -- 172.21.15.62:0/272275380 wait complete. 2023-07-14T16:17:30.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.013+0000 7f3285f5ab80 20 client.15945 trim_cache size 0 max 16384 2023-07-14T16:17:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.030+0000 7f3285f5ab80 1 Processor -- start 2023-07-14T16:17:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.030+0000 7f3285f5ab80 1 -- start start 2023-07-14T16:17:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b0230d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 0x563b02300b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0230d7a0 con 0x563b022fc4d0 2023-07-14T16:17:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b022f8a00 con 0x563b022fce20 2023-07-14T16:17:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b022fd210 con 0x563b022fdc70 2023-07-14T16:17:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f326bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f326bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46060/0 (socket says 172.21.15.62:46060) 2023-07-14T16:17:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f326bfff700 1 -- 172.21.15.62:0/972055254 learned_addr learned my addr 172.21.15.62:0/972055254 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.031+0000 7f3270b9c700 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 0x563b02300b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f326b7fe700 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b0230d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f326bfff700 1 -- 172.21.15.62:0/972055254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 msgr2=0x563b02300b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f326bfff700 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 0x563b02300b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f326bfff700 1 -- 172.21.15.62:0/972055254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 msgr2=0x563b0230d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f326bfff700 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b0230d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f326bfff700 1 -- 172.21.15.62:0/972055254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b020a2250 con 0x563b022fc4d0 2023-07-14T16:17:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f326b7fe700 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b0230d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f3270b9c700 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 0x563b02300b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:30.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f326bfff700 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230dc30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f325c01e4f0 tx=0x7f325c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d5ef6be15c2fa3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.032+0000 7f3252ffd700 1 -- 172.21.15.62:0/972055254 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c023040 con 0x563b022fc4d0 2023-07-14T16:17:30.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3252ffd700 1 -- 172.21.15.62:0/972055254 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f325c020560 con 0x563b022fc4d0 2023-07-14T16:17:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3252ffd700 1 -- 172.21.15.62:0/972055254 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c004830 con 0x563b022fc4d0 2023-07-14T16:17:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 -- 172.21.15.62:0/972055254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 msgr2=0x563b0230dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230dc30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f325c01e4f0 tx=0x7f325c01b000 comp rx=0 tx=0).stop 2023-07-14T16:17:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 -- 172.21.15.62:0/972055254 shutdown_connections 2023-07-14T16:17:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fdc70 0x563b02300b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230dc30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/972055254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fce20 0x563b0230d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 -- 172.21.15.62:0/972055254 >> 172.21.15.62:0/972055254 conn(0x563b022f1da0 msgr2=0x563b02318990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 -- 172.21.15.62:0/972055254 shutdown_connections 2023-07-14T16:17:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.033+0000 7f3285f5ab80 1 -- 172.21.15.62:0/972055254 wait complete. 2023-07-14T16:17:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.034+0000 7f3285f5ab80 1 -- start start 2023-07-14T16:17:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.034+0000 7f3285f5ab80 1 Processor -- start 2023-07-14T16:17:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fce20 0x563b02309b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f3285f5ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fdc70 0x563b0230a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0230a780 con 0x563b022fc4d0 2023-07-14T16:17:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02365ad0 con 0x563b022fdc70 2023-07-14T16:17:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f3285f5ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02365c10 con 0x563b022fce20 2023-07-14T16:17:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f3285f5ab80 10 client.0 _ll_register_callbacks cb 0x7f32640115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46072/0 (socket says 172.21.15.62:46072) 2023-07-14T16:17:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326bfff700 1 -- 172.21.15.62:0/3365484145 learned_addr learned my addr 172.21.15.62:0/3365484145 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326b7fe700 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fce20 0x563b02309b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f3270b9c700 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fdc70 0x563b0230a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326bfff700 1 -- 172.21.15.62:0/3365484145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fce20 msgr2=0x563b02309b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326bfff700 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fce20 0x563b02309b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326bfff700 1 -- 172.21.15.62:0/3365484145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fdc70 msgr2=0x563b0230a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326bfff700 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fdc70 0x563b0230a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.035+0000 7f326bfff700 1 -- 172.21.15.62:0/3365484145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c01e220 con 0x563b022fc4d0 2023-07-14T16:17:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.036+0000 7f326bfff700 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230d090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f325c029750 tx=0x7f325c029410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07d56b8279dfb69 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.036+0000 7f324b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:30.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.036+0000 7f324b7fe700 1 -- 172.21.15.62:0/3365484145 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c023030 con 0x563b022fc4d0 2023-07-14T16:17:30.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.036+0000 7f324b7fe700 1 -- 172.21.15.62:0/3365484145 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f325c017040 con 0x563b022fc4d0 2023-07-14T16:17:30.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.036+0000 7f3285f5ab80 1 -- 172.21.15.62:0/3365484145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f326009f170 con 0x563b022fc4d0 2023-07-14T16:17:30.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.036+0000 7f324b7fe700 1 -- 172.21.15.62:0/3365484145 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f325c041030 con 0x563b022fc4d0 2023-07-14T16:17:30.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.036+0000 7f3285f5ab80 10 client.15951 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:30.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.036+0000 7f3285f5ab80 1 -- 172.21.15.62:0/3365484145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563b0207a2b0 con 0x563b022fc4d0 2023-07-14T16:17:30.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f324b7fe700 1 -- 172.21.15.62:0/3365484145 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f326009f170 con 0x563b022fc4d0 2023-07-14T16:17:30.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f324b7fe700 1 -- 172.21.15.62:0/3365484145 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f325c03b470 con 0x563b022fc4d0 2023-07-14T16:17:30.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f3285f5ab80 10 client.15951 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:30.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f3285f5ab80 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f3285f5ab80 20 client.15951 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b023675e0 0x563b023699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f3285f5ab80 4 client.15951 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:17:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f3285f5ab80 1 -- 172.21.15.62:0/3365484145 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563b02209050 con 0x563b023675e0 2023-07-14T16:17:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.037+0000 7f326b7fe700 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b023675e0 0x563b023699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.038+0000 7f326b7fe700 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b023675e0 0x563b023699f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fc8c114caecf55 server_cookie=7c1019051976d398 in_seq=0 out_seq=0 2023-07-14T16:17:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.038+0000 7f324b7fe700 10 client.15951 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.038+0000 7f324b7fe700 1 -- 172.21.15.62:0/3365484145 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563b02209050 con 0x563b023675e0 2023-07-14T16:17:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.038+0000 7f324b7fe700 10 client.15951 handle_command_reply: tid=0 2023-07-14T16:17:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.038+0000 7f3285f5ab80 1 client.15951 shutdown 2023-07-14T16:17:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.038+0000 7f3285f5ab80 1 -- 172.21.15.62:0/3365484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 msgr2=0x563b0230d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.038+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230d090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f325c029750 tx=0x7f325c029410 comp rx=0 tx=0).stop 2023-07-14T16:17:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 1 -- 172.21.15.62:0/3365484145 shutdown_connections 2023-07-14T16:17:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b023675e0 0x563b023699f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b022fdc70 0x563b0230a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b022fc4d0 0x563b0230d090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f324b7fe700 0 client.15951 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 1 --2- 172.21.15.62:0/3365484145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b022fce20 0x563b02309b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 1 -- 172.21.15.62:0/3365484145 >> 172.21.15.62:0/3365484145 conn(0x563b022f1da0 msgr2=0x563b0230dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 1 -- 172.21.15.62:0/3365484145 shutdown_connections 2023-07-14T16:17:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 1 -- 172.21.15.62:0/3365484145 wait complete. 2023-07-14T16:17:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.039+0000 7f3285f5ab80 20 client.15951 trim_cache size 0 max 16384 2023-07-14T16:17:30.026 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:17:30.027 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:17:30.027 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:17:30.027 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:17:30.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 -- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32540a1150 msgr2=0x7f32540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32540a1150 0x7f32540a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f325c005950 tx=0x7f325c00b500 comp rx=0 tx=0).stop 2023-07-14T16:17:30.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 -- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c0744d0 msgr2=0x7f326c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c0744d0 0x7f326c197f00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f32600098e0 tx=0x7f3260005740 comp rx=0 tx=0).stop 2023-07-14T16:17:30.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 -- 172.21.15.62:0/2096004521 shutdown_connections 2023-07-14T16:17:30.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32540a1150 0x7f32540a3600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f326c1079b0 0x7f326c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f326c072ef0 0x7f326c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 --2- 172.21.15.62:0/2096004521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f326c0744d0 0x7f326c197f00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 -- 172.21.15.62:0/2096004521 >> 172.21.15.62:0/2096004521 conn(0x7f326c0fb120 msgr2=0x7f326c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 -- 172.21.15.62:0/2096004521 shutdown_connections 2023-07-14T16:17:30.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.040+0000 7f32725ff700 1 -- 172.21.15.62:0/2096004521 wait complete. 2023-07-14T16:17:30.041 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:17:30.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:17:30.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.293+0000 7fbf81bff700 1 -- 172.21.15.62:0/1546009227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c102e70 msgr2=0x7fbf7c105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.293+0000 7fbf81bff700 1 --2- 172.21.15.62:0/1546009227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c102e70 0x7fbf7c105250 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009990 tx=0x7fbf70009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.293+0000 7fbf81bff700 1 -- 172.21.15.62:0/1546009227 shutdown_connections 2023-07-14T16:17:30.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.293+0000 7fbf81bff700 1 --2- 172.21.15.62:0/1546009227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c105790 0x7fbf7c107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.293+0000 7fbf81bff700 1 --2- 172.21.15.62:0/1546009227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7c100550 0x7fbf7c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.293+0000 7fbf81bff700 1 --2- 172.21.15.62:0/1546009227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c102e70 0x7fbf7c105250 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.293+0000 7fbf81bff700 1 -- 172.21.15.62:0/1546009227 >> 172.21.15.62:0/1546009227 conn(0x7fbf7c0f9fd0 msgr2=0x7fbf7c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.294+0000 7fbf81bff700 1 -- 172.21.15.62:0/1546009227 shutdown_connections 2023-07-14T16:17:30.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.294+0000 7fbf81bff700 1 -- 172.21.15.62:0/1546009227 wait complete. 2023-07-14T16:17:30.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.295+0000 7fbf81bff700 1 Processor -- start 2023-07-14T16:17:30.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.295+0000 7fbf81bff700 1 -- start start 2023-07-14T16:17:30.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf81bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c100550 0x7fbf7c0ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf81bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7c102e70 0x7fbf7c0fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf81bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c105790 0x7fbf7c0fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf81bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0ff1e0 con 0x7fbf7c100550 2023-07-14T16:17:30.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf81bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0ff350 con 0x7fbf7c105790 2023-07-14T16:17:30.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf81bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c19f740 con 0x7fbf7c102e70 2023-07-14T16:17:30.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c100550 0x7fbf7c0ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c100550 0x7fbf7c0ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46090/0 (socket says 172.21.15.62:46090) 2023-07-14T16:17:30.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7b7fe700 1 -- 172.21.15.62:0/2756040500 learned_addr learned my addr 172.21.15.62:0/2756040500 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:30.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7affd700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7c102e70 0x7fbf7c0fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7bfff700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c105790 0x7fbf7c0fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7b7fe700 1 -- 172.21.15.62:0/2756040500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7c102e70 msgr2=0x7fbf7c0fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7b7fe700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7c102e70 0x7fbf7c0fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7b7fe700 1 -- 172.21.15.62:0/2756040500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c105790 msgr2=0x7fbf7c0fea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7b7fe700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c105790 0x7fbf7c0fea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.296+0000 7fbf7b7fe700 1 -- 172.21.15.62:0/2756040500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700095f0 con 0x7fbf7c100550 2023-07-14T16:17:30.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.297+0000 7fbf7b7fe700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c100550 0x7fbf7c0ffec0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00d950 tx=0x7fbf6c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.297+0000 7fbf78ff9700 1 -- 172.21.15.62:0/2756040500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c009b40 con 0x7fbf7c100550 2023-07-14T16:17:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.297+0000 7fbf78ff9700 1 -- 172.21.15.62:0/2756040500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf6c009ca0 con 0x7fbf7c100550 2023-07-14T16:17:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.297+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c19fa20 con 0x7fbf7c100550 2023-07-14T16:17:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.297+0000 7fbf78ff9700 1 -- 172.21.15.62:0/2756040500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00f620 con 0x7fbf7c100550 2023-07-14T16:17:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.297+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c19ff80 con 0x7fbf7c100550 2023-07-14T16:17:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.299+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c0fbb40 con 0x7fbf7c100550 2023-07-14T16:17:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.301+0000 7fbf78ff9700 1 -- 172.21.15.62:0/2756040500 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbf6c010460 con 0x7fbf7c100550 2023-07-14T16:17:30.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.303+0000 7fbf78ff9700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf640a1220 0x7fbf640a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.303+0000 7fbf78ff9700 1 -- 172.21.15.62:0/2756040500 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf6c0ccc90 con 0x7fbf7c100550 2023-07-14T16:17:30.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.303+0000 7fbf7affd700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf640a1220 0x7fbf640a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.304+0000 7fbf7affd700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf640a1220 0x7fbf640a36d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000b5c0 tx=0x7fbf70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.308+0000 7fbf78ff9700 1 -- 172.21.15.62:0/2756040500 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbf6c096390 con 0x7fbf7c100550 2023-07-14T16:17:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.599+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf7c02c850 con 0x7fbf7c100550 2023-07-14T16:17:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.600+0000 7fbf78ff9700 1 -- 172.21.15.62:0/2756040500 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10826 (secure 0 0 0) 0x7fbf6c020090 con 0x7fbf7c100550 2023-07-14T16:17:30.578 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:17:30.578 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:17:18.727946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:17:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf640a1220 msgr2=0x7fbf640a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf640a1220 0x7fbf640a36d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000b5c0 tx=0x7fbf70005fb0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c100550 msgr2=0x7fbf7c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c100550 0x7fbf7c0ffec0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00d950 tx=0x7fbf6c00dd10 comp rx=0 tx=0).stop 2023-07-14T16:17:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 shutdown_connections 2023-07-14T16:17:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf640a1220 0x7fbf640a36d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf7c105790 0x7fbf7c0fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7c100550 0x7fbf7c0ffec0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 --2- 172.21.15.62:0/2756040500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf7c102e70 0x7fbf7c0fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 >> 172.21.15.62:0/2756040500 conn(0x7fbf7c0f9fd0 msgr2=0x7fbf7c1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 shutdown_connections 2023-07-14T16:17:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.603+0000 7fbf81bff700 1 -- 172.21.15.62:0/2756040500 wait complete. 2023-07-14T16:17:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2023-07-14T16:17:30.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:17:30.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.857+0000 7fad4f7ff700 1 -- 172.21.15.62:0/742098622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 msgr2=0x7fad48072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.857+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/742098622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 0x7fad48072940 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad44009990 tx=0x7fad44009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.857+0000 7fad4f7ff700 1 -- 172.21.15.62:0/742098622 shutdown_connections 2023-07-14T16:17:30.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.857+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/742098622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad481079a0 0x7fad48109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.857+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/742098622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 0x7fad48072940 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.857+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/742098622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad48072f10 0x7fad48073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.857+0000 7fad4f7ff700 1 -- 172.21.15.62:0/742098622 >> 172.21.15.62:0/742098622 conn(0x7fad480fb130 msgr2=0x7fad480fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.857+0000 7fad4f7ff700 1 -- 172.21.15.62:0/742098622 shutdown_connections 2023-07-14T16:17:30.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.858+0000 7fad4f7ff700 1 -- 172.21.15.62:0/742098622 wait complete. 2023-07-14T16:17:30.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.859+0000 7fad4f7ff700 1 Processor -- start 2023-07-14T16:17:30.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.859+0000 7fad4f7ff700 1 -- start start 2023-07-14T16:17:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4f7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad48072f10 0x7fad48197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4f7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 0x7fad48197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4f7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad481079a0 0x7fad4819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4f7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4819f4e0 con 0x7fad480744f0 2023-07-14T16:17:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4f7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4819f620 con 0x7fad481079a0 2023-07-14T16:17:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4f7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4819f7b0 con 0x7fad48072f10 2023-07-14T16:17:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4cd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 0x7fad48197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4cd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 0x7fad48197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46104/0 (socket says 172.21.15.62:46104) 2023-07-14T16:17:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4cd9a700 1 -- 172.21.15.62:0/1714391312 learned_addr learned my addr 172.21.15.62:0/1714391312 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4d59b700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad48072f10 0x7fad48197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad481079a0 0x7fad4819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4cd9a700 1 -- 172.21.15.62:0/1714391312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad48072f10 msgr2=0x7fad48197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad48072f10 0x7fad48197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4cd9a700 1 -- 172.21.15.62:0/1714391312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad481079a0 msgr2=0x7fad4819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad481079a0 0x7fad4819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.860+0000 7fad4cd9a700 1 -- 172.21.15.62:0/1714391312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad440095f0 con 0x7fad480744f0 2023-07-14T16:17:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.861+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 0x7fad48197ea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fad3800d950 tx=0x7fad3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.861+0000 7fad3e7fc700 1 -- 172.21.15.62:0/1714391312 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38009940 con 0x7fad480744f0 2023-07-14T16:17:30.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.861+0000 7fad3e7fc700 1 -- 172.21.15.62:0/1714391312 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad38010460 con 0x7fad480744f0 2023-07-14T16:17:30.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.861+0000 7fad4f7ff700 1 -- 172.21.15.62:0/1714391312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4819fa90 con 0x7fad480744f0 2023-07-14T16:17:30.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.861+0000 7fad3e7fc700 1 -- 172.21.15.62:0/1714391312 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3800f620 con 0x7fad480744f0 2023-07-14T16:17:30.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.861+0000 7fad4f7ff700 1 -- 172.21.15.62:0/1714391312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4819fff0 con 0x7fad480744f0 2023-07-14T16:17:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.865+0000 7fad3e7fc700 1 -- 172.21.15.62:0/1714391312 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fad380105d0 con 0x7fad480744f0 2023-07-14T16:17:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.866+0000 7fad63095b80 1 Processor -- start 2023-07-14T16:17:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.866+0000 7fad63095b80 1 -- start start 2023-07-14T16:17:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad63095b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163f4d0 0x561ee163f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad63095b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163fe20 0x561ee16406e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad63095b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee1640c70 0x561ee16410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad63095b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee1544cb0 con 0x561ee163fe20 2023-07-14T16:17:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad63095b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee1597a40 con 0x561ee1640c70 2023-07-14T16:17:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad63095b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee1590ed0 con 0x561ee163f4d0 2023-07-14T16:17:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad4d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163fe20 0x561ee16406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad4dd9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163f4d0 0x561ee163f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad4d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163fe20 0x561ee16406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46114/0 (socket says 172.21.15.62:46114) 2023-07-14T16:17:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad4dd9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163f4d0 0x561ee163f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34928/0 (socket says 172.21.15.62:34928) 2023-07-14T16:17:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad4d59b700 1 -- 172.21.15.62:0/2948196025 learned_addr learned my addr 172.21.15.62:0/2948196025 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.867+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee1640c70 0x561ee16410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4dd9c700 1 -- 172.21.15.62:0/2948196025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee1640c70 msgr2=0x561ee16410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee1640c70 0x561ee16410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4dd9c700 1 -- 172.21.15.62:0/2948196025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163fe20 msgr2=0x561ee16406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163fe20 0x561ee16406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4dd9c700 1 -- 172.21.15.62:0/2948196025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ee13bede0 con 0x561ee163f4d0 2023-07-14T16:17:30.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee1640c70 0x561ee16410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:30.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4d59b700 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163fe20 0x561ee16406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:30.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad3e7fc700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad340a11d0 0x7fad340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163f4d0 0x561ee163f8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad40009310 tx=0x7fad40009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21cce9b5a0d4d9f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad3e7fc700 1 -- 172.21.15.62:0/1714391312 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fad380cc1b0 con 0x7fad480744f0 2023-07-14T16:17:30.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.868+0000 7fad4d59b700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad340a11d0 0x7fad340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.869+0000 7fad337fe700 1 -- 172.21.15.62:0/2948196025 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000e070 con 0x561ee163f4d0 2023-07-14T16:17:30.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.869+0000 7fad337fe700 1 -- 172.21.15.62:0/2948196025 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad4000f040 con 0x561ee163f4d0 2023-07-14T16:17:30.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.869+0000 7fad337fe700 1 -- 172.21.15.62:0/2948196025 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40014710 con 0x561ee163f4d0 2023-07-14T16:17:30.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.869+0000 7fad63095b80 1 -- 172.21.15.62:0/2948196025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163f4d0 msgr2=0x561ee163f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.869+0000 7fad63095b80 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163f4d0 0x561ee163f8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad40009310 tx=0x7fad40009920 comp rx=0 tx=0).stop 2023-07-14T16:17:30.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.869+0000 7fad4d59b700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad340a11d0 0x7fad340a3680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fad44003f60 tx=0x7fad4400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 -- 172.21.15.62:0/2948196025 shutdown_connections 2023-07-14T16:17:30.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee1640c70 0x561ee16410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163f4d0 0x561ee163f8e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 --2- 172.21.15.62:0/2948196025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163fe20 0x561ee16406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 -- 172.21.15.62:0/2948196025 >> 172.21.15.62:0/2948196025 conn(0x561ee1634da0 msgr2=0x561ee163a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 -- 172.21.15.62:0/2948196025 shutdown_connections 2023-07-14T16:17:30.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 -- 172.21.15.62:0/2948196025 wait complete. 2023-07-14T16:17:30.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 -- start start 2023-07-14T16:17:30.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.870+0000 7fad63095b80 1 Processor -- start 2023-07-14T16:17:30.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad63095b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163f4d0 0x561ee1649140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad63095b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163fe20 0x561ee1649680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad63095b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee1640c70 0x561ee16506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad63095b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee1530ac0 con 0x561ee1640c70 2023-07-14T16:17:30.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad63095b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee15307c0 con 0x561ee163f4d0 2023-07-14T16:17:30.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad63095b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee1414920 con 0x561ee163fe20 2023-07-14T16:17:30.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad63095b80 10 client.0 _ll_register_callbacks cb 0x561ee152e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:30.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad4dd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee1640c70 0x561ee16506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad4cd9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163fe20 0x561ee1649680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad4cd9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163fe20 0x561ee1649680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34936/0 (socket says 172.21.15.62:34936) 2023-07-14T16:17:30.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad4cd9a700 1 -- 172.21.15.62:0/209219181 learned_addr learned my addr 172.21.15.62:0/209219181 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:30.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.871+0000 7fad4d59b700 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163f4d0 0x561ee1649140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.872+0000 7fad4dd9c700 1 -- 172.21.15.62:0/209219181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163fe20 msgr2=0x561ee1649680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.872+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163fe20 0x561ee1649680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.872+0000 7fad4dd9c700 1 -- 172.21.15.62:0/209219181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163f4d0 msgr2=0x561ee1649140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.872+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163f4d0 0x561ee1649140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.872+0000 7fad4dd9c700 1 -- 172.21.15.62:0/209219181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4810aa40 con 0x561ee1640c70 2023-07-14T16:17:30.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.872+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163fe20 0x561ee1649680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:30.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.872+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee1640c70 0x561ee16506c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fad40015f40 tx=0x7fad400156f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d3ea3bfe2102f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.872+0000 7fad30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:30.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad30ff9700 1 -- 172.21.15.62:0/209219181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000e040 con 0x561ee1640c70 2023-07-14T16:17:30.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad63095b80 1 -- 172.21.15.62:0/209219181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ee14ea4f0 con 0x561ee1640c70 2023-07-14T16:17:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad30ff9700 1 -- 172.21.15.62:0/209219181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fad40025070 con 0x561ee1640c70 2023-07-14T16:17:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad30ff9700 1 -- 172.21.15.62:0/209219181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad40030030 con 0x561ee1640c70 2023-07-14T16:17:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad63095b80 10 client.15969 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad63095b80 1 -- 172.21.15.62:0/209219181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fad40009050 con 0x561ee1640c70 2023-07-14T16:17:30.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad30ff9700 1 -- 172.21.15.62:0/209219181 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561ee14ea4f0 con 0x561ee1640c70 2023-07-14T16:17:30.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad30ff9700 1 -- 172.21.15.62:0/209219181 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fad40005040 con 0x561ee1640c70 2023-07-14T16:17:30.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad63095b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:30.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad63095b80 10 client.15969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:17:30.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.873+0000 7fad63095b80 20 client.15969 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:30.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.874+0000 7fad63095b80 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x561ee1654e90 0x561ee16572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.874+0000 7fad63095b80 4 client.15969 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:30.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.874+0000 7fad63095b80 1 -- 172.21.15.62:0/209219181 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ee1589eb0 con 0x561ee1654e90 2023-07-14T16:17:30.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.874+0000 7fad4d59b700 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x561ee1654e90 0x561ee16572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.875+0000 7fad4d59b700 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x561ee1654e90 0x561ee16572a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abe8e6c0df6a3451 server_cookie=861d2edd336cf7fe in_seq=0 out_seq=0 2023-07-14T16:17:30.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.875+0000 7fad30ff9700 10 client.15969 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:17:30.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.875+0000 7fad30ff9700 1 -- 172.21.15.62:0/209219181 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561ee1589eb0 con 0x561ee1654e90 2023-07-14T16:17:30.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.875+0000 7fad30ff9700 10 client.15969 handle_command_reply: tid=0 2023-07-14T16:17:30.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.875+0000 7fad63095b80 1 client.15969 shutdown 2023-07-14T16:17:30.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 -- 172.21.15.62:0/209219181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee1640c70 msgr2=0x561ee16506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee1640c70 0x561ee16506c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fad40015f40 tx=0x7fad400156f0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 -- 172.21.15.62:0/209219181 shutdown_connections 2023-07-14T16:17:30.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x561ee1654e90 0x561ee16572a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee1640c70 0x561ee16506c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163f4d0 0x561ee1649140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 --2- 172.21.15.62:0/209219181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee163fe20 0x561ee1649680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 -- 172.21.15.62:0/209219181 >> 172.21.15.62:0/209219181 conn(0x561ee1634da0 msgr2=0x561ee16398b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad30ff9700 0 client.15969 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:17:30.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 -- 172.21.15.62:0/209219181 shutdown_connections 2023-07-14T16:17:30.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 1 -- 172.21.15.62:0/209219181 wait complete. 2023-07-14T16:17:30.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.876+0000 7fad63095b80 20 client.15969 trim_cache size 0 max 16384 2023-07-14T16:17:30.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.893+0000 7fad63095b80 1 Processor -- start 2023-07-14T16:17:30.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.893+0000 7fad63095b80 1 -- start start 2023-07-14T16:17:30.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.893+0000 7fad63095b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee1650ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.893+0000 7fad63095b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee1650680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.893+0000 7fad63095b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 0x561ee1656ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.893+0000 7fad63095b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee1634970 con 0x561ee163f4d0 2023-07-14T16:17:30.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.893+0000 7fad63095b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee163b270 con 0x561ee163fe20 2023-07-14T16:17:30.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.893+0000 7fad63095b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee163ba00 con 0x561ee1640c70 2023-07-14T16:17:30.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee1650ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee1650ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46124/0 (socket says 172.21.15.62:46124) 2023-07-14T16:17:30.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4d59b700 1 -- 172.21.15.62:0/2389790152 learned_addr learned my addr 172.21.15.62:0/2389790152 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:30.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 0x561ee1656ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee1650680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4d59b700 1 -- 172.21.15.62:0/2389790152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 msgr2=0x561ee1656ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4d59b700 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 0x561ee1656ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4d59b700 1 -- 172.21.15.62:0/2389790152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 msgr2=0x561ee1650680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4d59b700 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee1650680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4d59b700 1 -- 172.21.15.62:0/2389790152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ee13a7c60 con 0x561ee163f4d0 2023-07-14T16:17:30.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.894+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee1650680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:30.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.895+0000 7fad4d59b700 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee1650ef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fad44009990 tx=0x7fad44023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcba331dd5fdef73 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.895+0000 7fad33fff700 1 -- 172.21.15.62:0/2389790152 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44003d60 con 0x561ee163f4d0 2023-07-14T16:17:30.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.895+0000 7fad33fff700 1 -- 172.21.15.62:0/2389790152 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad4401e040 con 0x561ee163f4d0 2023-07-14T16:17:30.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.895+0000 7fad33fff700 1 -- 172.21.15.62:0/2389790152 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400bd40 con 0x561ee163f4d0 2023-07-14T16:17:30.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 -- 172.21.15.62:0/2389790152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 msgr2=0x561ee1650ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee1650ef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fad44009990 tx=0x7fad44023000 comp rx=0 tx=0).stop 2023-07-14T16:17:30.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 -- 172.21.15.62:0/2389790152 shutdown_connections 2023-07-14T16:17:30.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 0x561ee1656ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee1650ef0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 --2- 172.21.15.62:0/2389790152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee1650680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 -- 172.21.15.62:0/2389790152 >> 172.21.15.62:0/2389790152 conn(0x561ee1634da0 msgr2=0x561ee165bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 -- 172.21.15.62:0/2389790152 shutdown_connections 2023-07-14T16:17:30.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 -- 172.21.15.62:0/2389790152 wait complete. 2023-07-14T16:17:30.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 -- start start 2023-07-14T16:17:30.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.896+0000 7fad63095b80 1 Processor -- start 2023-07-14T16:17:30.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad63095b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee164ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad63095b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee16504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad63095b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 0x561ee164ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad63095b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee164d0e0 con 0x561ee163f4d0 2023-07-14T16:17:30.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad63095b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee164d250 con 0x561ee163fe20 2023-07-14T16:17:30.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad63095b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ee164d410 con 0x561ee1640c70 2023-07-14T16:17:30.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad63095b80 10 client.0 _ll_register_callbacks cb 0x7fad400097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:30.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad4d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee164ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad4d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee164ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46134/0 (socket says 172.21.15.62:46134) 2023-07-14T16:17:30.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad4d59b700 1 -- 172.21.15.62:0/610614237 learned_addr learned my addr 172.21.15.62:0/610614237 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:30.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee16504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.897+0000 7fad4dd9c700 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 0x561ee164ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:30 smithi062 ceph-mon[95543]: pgmap v413: 97 pgs: 97 active+clean; 14 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 991 KiB/s wr, 70 op/s 2023-07-14T16:17:30.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:30 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2756040500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:17:30.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.898+0000 7fad4d59b700 1 -- 172.21.15.62:0/610614237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 msgr2=0x561ee164ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.898+0000 7fad4d59b700 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 0x561ee164ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.898+0000 7fad4d59b700 1 -- 172.21.15.62:0/610614237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 msgr2=0x561ee16504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.898+0000 7fad4d59b700 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee16504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.898+0000 7fad4d59b700 1 -- 172.21.15.62:0/610614237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad440055b0 con 0x561ee163f4d0 2023-07-14T16:17:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.898+0000 7fad4d59b700 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee164ffb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fad4400b990 tx=0x7fad44003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d521bd075f7ee5c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.898+0000 7fad30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.899+0000 7fad30ff9700 1 -- 172.21.15.62:0/610614237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400bed0 con 0x561ee163f4d0 2023-07-14T16:17:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.899+0000 7fad30ff9700 1 -- 172.21.15.62:0/610614237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fad44037160 con 0x561ee163f4d0 2023-07-14T16:17:30.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.899+0000 7fad63095b80 1 -- 172.21.15.62:0/610614237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad4000f040 con 0x561ee163f4d0 2023-07-14T16:17:30.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.899+0000 7fad30ff9700 1 -- 172.21.15.62:0/610614237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad44037320 con 0x561ee163f4d0 2023-07-14T16:17:30.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.899+0000 7fad63095b80 10 client.15975 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:30.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.899+0000 7fad63095b80 1 -- 172.21.15.62:0/610614237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561ee13e5250 con 0x561ee163f4d0 2023-07-14T16:17:30.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.899+0000 7fad30ff9700 1 -- 172.21.15.62:0/610614237 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad4000f040 con 0x561ee163f4d0 2023-07-14T16:17:30.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.899+0000 7fad30ff9700 1 -- 172.21.15.62:0/610614237 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fad44040070 con 0x561ee163f4d0 2023-07-14T16:17:30.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.900+0000 7fad63095b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:30.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.900+0000 7fad63095b80 10 client.15975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:17:30.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.900+0000 7fad63095b80 20 client.15975 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:30.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.900+0000 7fad63095b80 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x561ee16a9f10 0x561ee16ac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:30.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.900+0000 7fad63095b80 4 client.15975 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:17:30.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.900+0000 7fad63095b80 1 -- 172.21.15.62:0/610614237 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ee154c050 con 0x561ee16a9f10 2023-07-14T16:17:30.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.900+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x561ee16a9f10 0x561ee16ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:30.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad4cd9a700 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x561ee16a9f10 0x561ee16ac320 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=788be628ef3946bf server_cookie=e20d6842343da316 in_seq=0 out_seq=0 2023-07-14T16:17:30.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad30ff9700 10 client.15975 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:17:30.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad30ff9700 1 -- 172.21.15.62:0/610614237 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ee154c050 con 0x561ee16a9f10 2023-07-14T16:17:30.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad30ff9700 10 client.15975 handle_command_reply: tid=0 2023-07-14T16:17:30.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 client.15975 shutdown 2023-07-14T16:17:30.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 -- 172.21.15.62:0/610614237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 msgr2=0x561ee164ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee164ffb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fad4400b990 tx=0x7fad44003b60 comp rx=0 tx=0).stop 2023-07-14T16:17:30.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 -- 172.21.15.62:0/610614237 shutdown_connections 2023-07-14T16:17:30.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x561ee16a9f10 0x561ee16ac320 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561ee1640c70 0x561ee164ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ee163f4d0 0x561ee164ffb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 --2- 172.21.15.62:0/610614237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ee163fe20 0x561ee16504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad30ff9700 0 client.15975 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:17:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 -- 172.21.15.62:0/610614237 >> 172.21.15.62:0/610614237 conn(0x561ee1634da0 msgr2=0x561ee1656ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 -- 172.21.15.62:0/610614237 shutdown_connections 2023-07-14T16:17:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 1 -- 172.21.15.62:0/610614237 wait complete. 2023-07-14T16:17:30.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.901+0000 7fad63095b80 20 client.15975 trim_cache size 0 max 16384 2023-07-14T16:17:30.894 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:17:30.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 -- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad340a11d0 msgr2=0x7fad340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad340a11d0 0x7fad340a3680 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fad44003f60 tx=0x7fad4400b260 comp rx=0 tx=0).stop 2023-07-14T16:17:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 -- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 msgr2=0x7fad48197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 0x7fad48197ea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fad3800d950 tx=0x7fad3800dc60 comp rx=0 tx=0).stop 2023-07-14T16:17:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 -- 172.21.15.62:0/1714391312 shutdown_connections 2023-07-14T16:17:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fad340a11d0 0x7fad340a3680 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad481079a0 0x7fad4819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad48072f10 0x7fad48197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 --2- 172.21.15.62:0/1714391312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad480744f0 0x7fad48197ea0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:30.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 -- 172.21.15.62:0/1714391312 >> 172.21.15.62:0/1714391312 conn(0x7fad480fb130 msgr2=0x7fad480fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:30.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 -- 172.21.15.62:0/1714391312 shutdown_connections 2023-07-14T16:17:30.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:30.902+0000 7fad4f7ff700 1 -- 172.21.15.62:0/1714391312 wait complete. 2023-07-14T16:17:30.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:17:31.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:30 smithi164 ceph-mon[97427]: pgmap v413: 97 pgs: 97 active+clean; 14 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 991 KiB/s wr, 70 op/s 2023-07-14T16:17:31.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:30 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2756040500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:17:31.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.147+0000 7f03f199d700 1 -- 172.21.15.62:0/1730404989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 msgr2=0x7f03ec10a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.147+0000 7f03f199d700 1 --2- 172.21.15.62:0/1730404989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 0x7f03ec10a040 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f03e400b0a0 tx=0x7f03e400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.148+0000 7f03f199d700 1 -- 172.21.15.62:0/1730404989 shutdown_connections 2023-07-14T16:17:31.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.148+0000 7f03f199d700 1 --2- 172.21.15.62:0/1730404989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 0x7f03ec10a040 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.148+0000 7f03f199d700 1 --2- 172.21.15.62:0/1730404989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03ec101c00 0x7f03ec102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.148+0000 7f03f199d700 1 --2- 172.21.15.62:0/1730404989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ec102df0 0x7f03ec103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.148+0000 7f03f199d700 1 -- 172.21.15.62:0/1730404989 >> 172.21.15.62:0/1730404989 conn(0x7f03ec0fd220 msgr2=0x7f03ec0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.148+0000 7f03f199d700 1 -- 172.21.15.62:0/1730404989 shutdown_connections 2023-07-14T16:17:31.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.148+0000 7f03f199d700 1 -- 172.21.15.62:0/1730404989 wait complete. 2023-07-14T16:17:31.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.150+0000 7f03f199d700 1 Processor -- start 2023-07-14T16:17:31.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.150+0000 7f03f199d700 1 -- start start 2023-07-14T16:17:31.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f199d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ec101c00 0x7f03ec197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f199d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03ec102df0 0x7f03ec197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f199d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 0x7f03ec19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f199d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19f380 con 0x7f03ec103780 2023-07-14T16:17:31.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f199d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19f4c0 con 0x7f03ec101c00 2023-07-14T16:17:31.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f199d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19f650 con 0x7f03ec102df0 2023-07-14T16:17:31.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f119c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 0x7f03ec19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f119c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 0x7f03ec19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46150/0 (socket says 172.21.15.62:46150) 2023-07-14T16:17:31.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f119c700 1 -- 172.21.15.62:0/1948107184 learned_addr learned my addr 172.21.15.62:0/1948107184 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03f099b700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ec101c00 0x7f03ec197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.151+0000 7f03ebfff700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03ec102df0 0x7f03ec197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03f119c700 1 -- 172.21.15.62:0/1948107184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03ec102df0 msgr2=0x7f03ec197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03f119c700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03ec102df0 0x7f03ec197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03f119c700 1 -- 172.21.15.62:0/1948107184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ec101c00 msgr2=0x7f03ec197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03f119c700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ec101c00 0x7f03ec197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03f119c700 1 -- 172.21.15.62:0/1948107184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e4009d00 con 0x7f03ec103780 2023-07-14T16:17:31.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03f099b700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ec101c00 0x7f03ec197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:31.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03f119c700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 0x7f03ec19edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009fd0 tx=0x7f03e400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03e9ffb700 1 -- 172.21.15.62:0/1948107184 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e400e040 con 0x7f03ec103780 2023-07-14T16:17:31.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03f199d700 1 -- 172.21.15.62:0/1948107184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec19f8d0 con 0x7f03ec103780 2023-07-14T16:17:31.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03e9ffb700 1 -- 172.21.15.62:0/1948107184 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03e4003e10 con 0x7f03ec103780 2023-07-14T16:17:31.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.152+0000 7f03e9ffb700 1 -- 172.21.15.62:0/1948107184 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e401b6c0 con 0x7f03ec103780 2023-07-14T16:17:31.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.153+0000 7f03f199d700 1 -- 172.21.15.62:0/1948107184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec19fd00 con 0x7f03ec103780 2023-07-14T16:17:31.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.158+0000 7f03e9ffb700 1 -- 172.21.15.62:0/1948107184 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f03e401d990 con 0x7f03ec103780 2023-07-14T16:17:31.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.158+0000 7f040655cb80 1 Processor -- start 2023-07-14T16:17:31.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.159+0000 7f040655cb80 1 -- start start 2023-07-14T16:17:31.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.159+0000 7f040655cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c739720 0x55601c739b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.159+0000 7f040655cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c73a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.159+0000 7f040655cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73ae30 0x55601c73b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.159+0000 7f040655cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c63ecb0 con 0x55601c73a070 2023-07-14T16:17:31.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.159+0000 7f040655cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c691a40 con 0x55601c73ae30 2023-07-14T16:17:31.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.159+0000 7f040655cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c68aed0 con 0x55601c739720 2023-07-14T16:17:31.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73ae30 0x55601c73b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73ae30 0x55601c73b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40840/0 (socket says 172.21.15.62:40840) 2023-07-14T16:17:31.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 -- 172.21.15.62:0/1376320921 learned_addr learned my addr 172.21.15.62:0/1376320921 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03f099b700 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c73a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03e9ffb700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03dc0a11f0 0x7f03dc0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c739720 0x55601c739b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03e9ffb700 1 -- 172.21.15.62:0/1948107184 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03e40cdb20 con 0x7f03ec103780 2023-07-14T16:17:31.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03f119c700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03dc0a11f0 0x7f03dc0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 -- 172.21.15.62:0/1376320921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73ae30 msgr2=0x55601c73b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73ae30 0x55601c73b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 -- 172.21.15.62:0/1376320921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 msgr2=0x55601c73a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c73a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.160+0000 7f03ebfff700 1 -- 172.21.15.62:0/1376320921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55601c4b8de0 con 0x55601c739720 2023-07-14T16:17:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.161+0000 7f03ebfff700 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c739720 0x55601c739b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03d800f6e0 tx=0x7f03d800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0d565abd0ba30e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.161+0000 7f03f119c700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03dc0a11f0 0x7f03dc0a36a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f03e4097470 tx=0x7f03e400bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.161+0000 7f03d6ffd700 1 -- 172.21.15.62:0/1376320921 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800fe40 con 0x55601c739720 2023-07-14T16:17:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.161+0000 7f03d6ffd700 1 -- 172.21.15.62:0/1376320921 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03d80268b0 con 0x55601c739720 2023-07-14T16:17:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.161+0000 7f03d6ffd700 1 -- 172.21.15.62:0/1376320921 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800d5a0 con 0x55601c739720 2023-07-14T16:17:31.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 -- 172.21.15.62:0/1376320921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c739720 msgr2=0x55601c739b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c739720 0x55601c739b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03d800f6e0 tx=0x7f03d800b260 comp rx=0 tx=0).stop 2023-07-14T16:17:31.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 -- 172.21.15.62:0/1376320921 shutdown_connections 2023-07-14T16:17:31.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73ae30 0x55601c73b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c739720 0x55601c739b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 --2- 172.21.15.62:0/1376320921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c73a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 -- 172.21.15.62:0/1376320921 >> 172.21.15.62:0/1376320921 conn(0x55601c72eff0 msgr2=0x55601c734460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 -- 172.21.15.62:0/1376320921 shutdown_connections 2023-07-14T16:17:31.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.162+0000 7f040655cb80 1 -- 172.21.15.62:0/1376320921 wait complete. 2023-07-14T16:17:31.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.163+0000 7f040655cb80 1 -- start start 2023-07-14T16:17:31.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.163+0000 7f040655cb80 1 Processor -- start 2023-07-14T16:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.163+0000 7f040655cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c7432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.163+0000 7f040655cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c743800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.163+0000 7f040655cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 0x55601c74a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.163+0000 7f040655cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c62aac0 con 0x55601c73a070 2023-07-14T16:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.163+0000 7f040655cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c62a7c0 con 0x55601c739720 2023-07-14T16:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.163+0000 7f040655cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c50e920 con 0x55601c73ae30 2023-07-14T16:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f040655cb80 10 client.0 _ll_register_callbacks cb 0x55601c628760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03f099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c743800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03f099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c743800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46164/0 (socket says 172.21.15.62:46164) 2023-07-14T16:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03f099b700 1 -- 172.21.15.62:0/3627430394 learned_addr learned my addr 172.21.15.62:0/3627430394 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03ebfff700 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c7432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03ebfff700 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 0x55601c74a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03f099b700 1 -- 172.21.15.62:0/3627430394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 msgr2=0x55601c74a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03f099b700 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 0x55601c74a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03f099b700 1 -- 172.21.15.62:0/3627430394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 msgr2=0x55601c7432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03f099b700 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c7432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.164+0000 7f03f099b700 1 -- 172.21.15.62:0/3627430394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d8021040 con 0x55601c73a070 2023-07-14T16:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f03f099b700 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c743800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f03e0012840 tx=0x7f03e0012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963f6e9963f0ea88 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f03cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f03cbfff700 1 -- 172.21.15.62:0/3627430394 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0017070 con 0x55601c73a070 2023-07-14T16:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f040655cb80 1 -- 172.21.15.62:0/3627430394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55601c5e44f0 con 0x55601c73a070 2023-07-14T16:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f03cbfff700 1 -- 172.21.15.62:0/3627430394 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03e0011850 con 0x55601c73a070 2023-07-14T16:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f03cbfff700 1 -- 172.21.15.62:0/3627430394 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03e0011a10 con 0x55601c73a070 2023-07-14T16:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f040655cb80 10 client.15987 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f040655cb80 1 -- 172.21.15.62:0/3627430394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55601c4a1c60 con 0x55601c73a070 2023-07-14T16:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.165+0000 7f03cbfff700 1 -- 172.21.15.62:0/3627430394 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55601c5e44f0 con 0x55601c73a070 2023-07-14T16:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.166+0000 7f03cbfff700 1 -- 172.21.15.62:0/3627430394 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f03e000fba0 con 0x55601c73a070 2023-07-14T16:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.166+0000 7f040655cb80 10 client.15987 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.166+0000 7f040655cb80 10 client.15987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.166+0000 7f040655cb80 20 client.15987 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.166+0000 7f040655cb80 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55601c74f050 0x55601c751460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.166+0000 7f040655cb80 4 client.15987 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.166+0000 7f040655cb80 1 -- 172.21.15.62:0/3627430394 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55601c683eb0 con 0x55601c74f050 2023-07-14T16:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.166+0000 7f03ebfff700 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55601c74f050 0x55601c751460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.167+0000 7f03ebfff700 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55601c74f050 0x55601c751460 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3093f484c0e57293 server_cookie=496575dc22a4f77a in_seq=0 out_seq=0 2023-07-14T16:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.167+0000 7f03cbfff700 10 client.15987 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f03cbfff700 1 -- 172.21.15.62:0/3627430394 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55601c683eb0 con 0x55601c74f050 2023-07-14T16:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f03cbfff700 10 client.15987 handle_command_reply: tid=0 2023-07-14T16:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 client.15987 shutdown 2023-07-14T16:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 -- 172.21.15.62:0/3627430394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 msgr2=0x55601c743800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c743800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f03e0012840 tx=0x7f03e0012b50 comp rx=0 tx=0).stop 2023-07-14T16:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 -- 172.21.15.62:0/3627430394 shutdown_connections 2023-07-14T16:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55601c74f050 0x55601c751460 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 0x55601c74a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c7432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 --2- 172.21.15.62:0/3627430394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73a070 0x55601c743800 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f03cbfff700 0 client.15987 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:17:31.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 -- 172.21.15.62:0/3627430394 >> 172.21.15.62:0/3627430394 conn(0x55601c72eff0 msgr2=0x55601c733b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 -- 172.21.15.62:0/3627430394 shutdown_connections 2023-07-14T16:17:31.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.168+0000 7f040655cb80 1 -- 172.21.15.62:0/3627430394 wait complete. 2023-07-14T16:17:31.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.169+0000 7f040655cb80 20 client.15987 trim_cache size 0 max 16384 2023-07-14T16:17:31.158 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:30 smithi081 ceph-mon[127795]: pgmap v413: 97 pgs: 97 active+clean; 14 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 991 KiB/s wr, 70 op/s 2023-07-14T16:17:31.159 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:30 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2756040500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:17:31.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.182+0000 7f040655cb80 1 Processor -- start 2023-07-14T16:17:31.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f040655cb80 1 -- start start 2023-07-14T16:17:31.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f040655cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c74b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f040655cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73a070 0x55601c74a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f040655cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73ae30 0x55601c73dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f040655cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c740f90 con 0x55601c73ae30 2023-07-14T16:17:31.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f040655cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c74eda0 con 0x55601c739720 2023-07-14T16:17:31.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f040655cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c735e70 con 0x55601c73a070 2023-07-14T16:17:31.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73ae30 0x55601c73dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c74b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c74b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40864/0 (socket says 172.21.15.62:40864) 2023-07-14T16:17:31.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f03ebfff700 1 -- 172.21.15.62:0/2859611470 learned_addr learned my addr 172.21.15.62:0/2859611470 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.183+0000 7f03f099b700 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73a070 0x55601c74a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03ebfff700 1 -- 172.21.15.62:0/2859611470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73a070 msgr2=0x55601c74a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03ebfff700 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73a070 0x55601c74a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03ebfff700 1 -- 172.21.15.62:0/2859611470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73ae30 msgr2=0x55601c73dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03ebfff700 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73ae30 0x55601c73dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03ebfff700 1 -- 172.21.15.62:0/2859611470 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55601c4df250 con 0x55601c739720 2023-07-14T16:17:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03f099b700 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73a070 0x55601c74a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03ebfff700 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c74b130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f03d800f510 tx=0x7f03d800c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cf4c2d0fe7b0f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03d77fe700 1 -- 172.21.15.62:0/2859611470 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8034410 con 0x55601c739720 2023-07-14T16:17:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03d77fe700 1 -- 172.21.15.62:0/2859611470 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03d802b040 con 0x55601c739720 2023-07-14T16:17:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.184+0000 7f03d77fe700 1 -- 172.21.15.62:0/2859611470 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8034b80 con 0x55601c739720 2023-07-14T16:17:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 -- 172.21.15.62:0/2859611470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 msgr2=0x55601c74b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c74b130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f03d800f510 tx=0x7f03d800c510 comp rx=0 tx=0).stop 2023-07-14T16:17:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 -- 172.21.15.62:0/2859611470 shutdown_connections 2023-07-14T16:17:31.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c73ae30 0x55601c73dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c739720 0x55601c74b130 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 --2- 172.21.15.62:0/2859611470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73a070 0x55601c74a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 -- 172.21.15.62:0/2859611470 >> 172.21.15.62:0/2859611470 conn(0x55601c72eff0 msgr2=0x55601c741fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 -- 172.21.15.62:0/2859611470 shutdown_connections 2023-07-14T16:17:31.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.185+0000 7f040655cb80 1 -- 172.21.15.62:0/2859611470 wait complete. 2023-07-14T16:17:31.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.186+0000 7f040655cb80 1 -- start start 2023-07-14T16:17:31.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.186+0000 7f040655cb80 1 Processor -- start 2023-07-14T16:17:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f040655cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c739720 0x55601c74a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f040655cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73a070 0x55601c74a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f040655cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 0x55601c746cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f040655cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c7473d0 con 0x55601c739720 2023-07-14T16:17:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f040655cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c747540 con 0x55601c73a070 2023-07-14T16:17:31.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f040655cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601c747700 con 0x55601c73ae30 2023-07-14T16:17:31.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f040655cb80 10 client.0 _ll_register_callbacks cb 0x7f03e0009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:31.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c739720 0x55601c74a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c739720 0x55601c74a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46186/0 (socket says 172.21.15.62:46186) 2023-07-14T16:17:31.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f03ebfff700 1 -- 172.21.15.62:0/4026846725 learned_addr learned my addr 172.21.15.62:0/4026846725 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f03f099b700 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73a070 0x55601c74a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f03ebfff700 1 -- 172.21.15.62:0/4026846725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 msgr2=0x55601c746cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f03ebfff700 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 0x55601c746cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.187+0000 7f03ebfff700 1 -- 172.21.15.62:0/4026846725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73a070 msgr2=0x55601c74a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f03ebfff700 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73a070 0x55601c74a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f03ebfff700 1 -- 172.21.15.62:0/4026846725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d802d040 con 0x55601c739720 2023-07-14T16:17:31.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f03ebfff700 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c739720 0x55601c74a1e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005d20 tx=0x7f03d800fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b6fc3f31fadbe4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f03cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:31.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f03cbfff700 1 -- 172.21.15.62:0/4026846725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8026a20 con 0x55601c739720 2023-07-14T16:17:31.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f040655cb80 1 -- 172.21.15.62:0/4026846725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03d80268b0 con 0x55601c739720 2023-07-14T16:17:31.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f03cbfff700 1 -- 172.21.15.62:0/4026846725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03d8026b80 con 0x55601c739720 2023-07-14T16:17:31.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f03cbfff700 1 -- 172.21.15.62:0/4026846725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03d8026420 con 0x55601c739720 2023-07-14T16:17:31.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f040655cb80 10 client.15993 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:31.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.188+0000 7f040655cb80 1 -- 172.21.15.62:0/4026846725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55601c4b72b0 con 0x55601c739720 2023-07-14T16:17:31.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f03cbfff700 1 -- 172.21.15.62:0/4026846725 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03d80268b0 con 0x55601c739720 2023-07-14T16:17:31.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f03cbfff700 1 -- 172.21.15.62:0/4026846725 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f03d8004300 con 0x55601c739720 2023-07-14T16:17:31.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f040655cb80 10 client.15993 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f040655cb80 10 client.15993 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:17:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f040655cb80 20 client.15993 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f040655cb80 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55601c7a4180 0x55601c7a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f040655cb80 4 client.15993 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:17:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f040655cb80 1 -- 172.21.15.62:0/4026846725 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55601c646050 con 0x55601c7a4180 2023-07-14T16:17:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.189+0000 7f03f099b700 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55601c7a4180 0x55601c7a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.190+0000 7f03f099b700 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55601c7a4180 0x55601c7a6590 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a54089ffcb5b117 server_cookie=1b597bbf32312190 in_seq=0 out_seq=0 2023-07-14T16:17:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.190+0000 7f03cbfff700 10 client.15993 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:17:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.190+0000 7f03cbfff700 1 -- 172.21.15.62:0/4026846725 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55601c646050 con 0x55601c7a4180 2023-07-14T16:17:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.190+0000 7f03cbfff700 10 client.15993 handle_command_reply: tid=0 2023-07-14T16:17:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.190+0000 7f040655cb80 1 client.15993 shutdown 2023-07-14T16:17:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.190+0000 7f040655cb80 1 -- 172.21.15.62:0/4026846725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c739720 msgr2=0x55601c74a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.190+0000 7f040655cb80 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c739720 0x55601c74a1e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005d20 tx=0x7f03d800fe40 comp rx=0 tx=0).stop 2023-07-14T16:17:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 1 -- 172.21.15.62:0/4026846725 shutdown_connections 2023-07-14T16:17:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55601c7a4180 0x55601c7a6590 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55601c73ae30 0x55601c746cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55601c739720 0x55601c74a1e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f03cbfff700 0 client.15993 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:17:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 1 --2- 172.21.15.62:0/4026846725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55601c73a070 0x55601c74a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 1 -- 172.21.15.62:0/4026846725 >> 172.21.15.62:0/4026846725 conn(0x55601c72eff0 msgr2=0x55601c74b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 1 -- 172.21.15.62:0/4026846725 shutdown_connections 2023-07-14T16:17:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 1 -- 172.21.15.62:0/4026846725 wait complete. 2023-07-14T16:17:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f040655cb80 20 client.15993 trim_cache size 0 max 16384 2023-07-14T16:17:31.176 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:17:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f03f199d700 1 -- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03dc0a11f0 msgr2=0x7f03dc0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f03f199d700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03dc0a11f0 0x7f03dc0a36a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f03e4097470 tx=0x7f03e400bf90 comp rx=0 tx=0).stop 2023-07-14T16:17:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f03f199d700 1 -- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 msgr2=0x7f03ec19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.191+0000 7f03f199d700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 0x7f03ec19edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009fd0 tx=0x7f03e400bd30 comp rx=0 tx=0).stop 2023-07-14T16:17:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.192+0000 7f03f199d700 1 -- 172.21.15.62:0/1948107184 shutdown_connections 2023-07-14T16:17:31.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.192+0000 7f03f199d700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03ec103780 0x7f03ec19edb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.192+0000 7f03f199d700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03dc0a11f0 0x7f03dc0a36a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.192+0000 7f03f199d700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03ec101c00 0x7f03ec197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.192+0000 7f03f199d700 1 --2- 172.21.15.62:0/1948107184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03ec102df0 0x7f03ec197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.192+0000 7f03f199d700 1 -- 172.21.15.62:0/1948107184 >> 172.21.15.62:0/1948107184 conn(0x7f03ec0fd220 msgr2=0x7f03ec0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.192+0000 7f03f199d700 1 -- 172.21.15.62:0/1948107184 shutdown_connections 2023-07-14T16:17:31.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.192+0000 7f03f199d700 1 -- 172.21.15.62:0/1948107184 wait complete. 2023-07-14T16:17:31.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:17:31.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1676051486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198101d10 msgr2=0x7fb198102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1676051486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198101d10 0x7fb198102120 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb194009990 tx=0x7fb194009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1676051486 shutdown_connections 2023-07-14T16:17:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1676051486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb198103890 0x7fb19810a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1676051486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198101d10 0x7fb198102120 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1676051486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb198102f00 0x7fb198103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1676051486 >> 172.21.15.62:0/1676051486 conn(0x7fb1980fd350 msgr2=0x7fb1980ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1676051486 shutdown_connections 2023-07-14T16:17:31.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.429+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1676051486 wait complete. 2023-07-14T16:17:31.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.430+0000 7fb1a03ff700 1 Processor -- start 2023-07-14T16:17:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.431+0000 7fb1a03ff700 1 -- start start 2023-07-14T16:17:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.431+0000 7fb1a03ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb198101d10 0x7fb1981979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.431+0000 7fb1a03ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb198102f00 0x7fb198197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.431+0000 7fb1a03ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198103890 0x7fb19819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.431+0000 7fb1a03ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19819f550 con 0x7fb198103890 2023-07-14T16:17:31.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.431+0000 7fb1a03ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19819f690 con 0x7fb198101d10 2023-07-14T16:17:31.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.431+0000 7fb1a03ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19819f820 con 0x7fb198102f00 2023-07-14T16:17:31.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198103890 0x7fb19819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198103890 0x7fb19819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46204/0 (socket says 172.21.15.62:46204) 2023-07-14T16:17:31.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e99c700 1 -- 172.21.15.62:0/1681126828 learned_addr learned my addr 172.21.15.62:0/1681126828 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e19b700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb198101d10 0x7fb1981979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19d99a700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb198102f00 0x7fb198197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e99c700 1 -- 172.21.15.62:0/1681126828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb198102f00 msgr2=0x7fb198197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e99c700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb198102f00 0x7fb198197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e99c700 1 -- 172.21.15.62:0/1681126828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb198101d10 msgr2=0x7fb1981979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e99c700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb198101d10 0x7fb1981979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.432+0000 7fb19e99c700 1 -- 172.21.15.62:0/1681126828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940095f0 con 0x7fb198103890 2023-07-14T16:17:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.433+0000 7fb19e99c700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198103890 0x7fb19819ef80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb190007ee0 tx=0x7fb19000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.433+0000 7fb18f7fe700 1 -- 172.21.15.62:0/1681126828 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190017070 con 0x7fb198103890 2023-07-14T16:17:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.433+0000 7fb18f7fe700 1 -- 172.21.15.62:0/1681126828 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb19000f040 con 0x7fb198103890 2023-07-14T16:17:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.433+0000 7fb18f7fe700 1 -- 172.21.15.62:0/1681126828 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190005710 con 0x7fb198103890 2023-07-14T16:17:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.433+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1681126828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19819fb00 con 0x7fb198103890 2023-07-14T16:17:31.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.433+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1681126828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19819ffc0 con 0x7fb198103890 2023-07-14T16:17:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.438+0000 7fb18f7fe700 1 -- 172.21.15.62:0/1681126828 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb190005870 con 0x7fb198103890 2023-07-14T16:17:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.439+0000 7fb1b3d79b80 1 Processor -- start 2023-07-14T16:17:31.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.439+0000 7fb1b3d79b80 1 -- start start 2023-07-14T16:17:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.439+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb84d0 0x5573c5bb88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.439+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bb96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.439+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb9c70 0x5573c5bba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5abdcb0 con 0x5573c5bb8e20 2023-07-14T16:17:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5b10a40 con 0x5573c5bb84d0 2023-07-14T16:17:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5b09ed0 con 0x5573c5bb9c70 2023-07-14T16:17:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19e19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bb96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19e19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bb96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46216/0 (socket says 172.21.15.62:46216) 2023-07-14T16:17:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19e19b700 1 -- 172.21.15.62:0/4115809289 learned_addr learned my addr 172.21.15.62:0/4115809289 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19d99a700 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb84d0 0x5573c5bb88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19d99a700 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb9c70 0x5573c5bba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19e19b700 1 -- 172.21.15.62:0/4115809289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb9c70 msgr2=0x5573c5bba0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19e19b700 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb9c70 0x5573c5bba0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19e19b700 1 -- 172.21.15.62:0/4115809289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb84d0 msgr2=0x5573c5bb88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19e19b700 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb84d0 0x5573c5bb88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.440+0000 7fb19e19b700 1 -- 172.21.15.62:0/4115809289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573c5937de0 con 0x5573c5bb8e20 2023-07-14T16:17:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.441+0000 7fb19e19b700 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bb96e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb198102d60 tx=0x7fb194009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4092cd41c9792244 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.441+0000 7fb18cbfe700 1 -- 172.21.15.62:0/4115809289 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19401d070 con 0x5573c5bb8e20 2023-07-14T16:17:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.441+0000 7fb18cbfe700 1 -- 172.21.15.62:0/4115809289 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb19401e040 con 0x5573c5bb8e20 2023-07-14T16:17:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.441+0000 7fb18cbfe700 1 -- 172.21.15.62:0/4115809289 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194004500 con 0x5573c5bb8e20 2023-07-14T16:17:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.441+0000 7fb18f7fe700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1840a1150 0x7fb1840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.441+0000 7fb18f7fe700 1 -- 172.21.15.62:0/1681126828 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1900cc080 con 0x7fb198103890 2023-07-14T16:17:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.441+0000 7fb19d99a700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1840a1150 0x7fb1840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/4115809289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 msgr2=0x5573c5bb96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bb96e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb198102d60 tx=0x7fb194009570 comp rx=0 tx=0).stop 2023-07-14T16:17:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/4115809289 shutdown_connections 2023-07-14T16:17:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb9c70 0x5573c5bba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb84d0 0x5573c5bb88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/4115809289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bb96e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/4115809289 >> 172.21.15.62:0/4115809289 conn(0x5573c5badda0 msgr2=0x5573c5bb31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/4115809289 shutdown_connections 2023-07-14T16:17:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb19d99a700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1840a1150 0x7fb1840a3600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb18800d010 tx=0x7fb188016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.442+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/4115809289 wait complete. 2023-07-14T16:17:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.443+0000 7fb1b3d79b80 1 -- start start 2023-07-14T16:17:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.443+0000 7fb1b3d79b80 1 Processor -- start 2023-07-14T16:17:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb84d0 0x5573c5bc2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb8e20 0x5573c5bc25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 0x5573c5bc9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5aa9ac0 con 0x5573c5bb84d0 2023-07-14T16:17:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5aa97c0 con 0x5573c5bb9c70 2023-07-14T16:17:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c598d920 con 0x5573c5bb8e20 2023-07-14T16:17:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb1b3d79b80 10 client.0 _ll_register_callbacks cb 0x5573c5aa7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19e19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb84d0 0x5573c5bc2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19e19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb84d0 0x5573c5bc2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46224/0 (socket says 172.21.15.62:46224) 2023-07-14T16:17:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19e19b700 1 -- 172.21.15.62:0/1377266437 learned_addr learned my addr 172.21.15.62:0/1377266437 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19d99a700 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb8e20 0x5573c5bc25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19e19b700 1 -- 172.21.15.62:0/1377266437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb8e20 msgr2=0x5573c5bc25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19e19b700 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb8e20 0x5573c5bc25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19e19b700 1 -- 172.21.15.62:0/1377266437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 msgr2=0x5573c5bc9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19e19b700 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 0x5573c5bc9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19e19b700 1 -- 172.21.15.62:0/1377266437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19810aa80 con 0x5573c5bb84d0 2023-07-14T16:17:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.444+0000 7fb19d99a700 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb8e20 0x5573c5bc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:17:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.445+0000 7fb19e19b700 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb84d0 0x5573c5bc2080 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb194009990 tx=0x7fb19400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753469dfb355fb38 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.445+0000 7fb181ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.445+0000 7fb181ffb700 1 -- 172.21.15.62:0/1377266437 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19401d030 con 0x5573c5bb84d0 2023-07-14T16:17:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.445+0000 7fb181ffb700 1 -- 172.21.15.62:0/1377266437 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1940059a0 con 0x5573c5bb84d0 2023-07-14T16:17:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.445+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/1377266437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573c5a634f0 con 0x5573c5bb84d0 2023-07-14T16:17:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.445+0000 7fb181ffb700 1 -- 172.21.15.62:0/1377266437 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb194003a20 con 0x5573c5bb84d0 2023-07-14T16:17:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.445+0000 7fb1b3d79b80 10 client.16005 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.445+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/1377266437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5573c5920c60 con 0x5573c5bb84d0 2023-07-14T16:17:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.446+0000 7fb181ffb700 1 -- 172.21.15.62:0/1377266437 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb19403d020 con 0x5573c5bb84d0 2023-07-14T16:17:31.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.446+0000 7fb181ffb700 1 -- 172.21.15.62:0/1377266437 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb19400f9e0 con 0x5573c5bb84d0 2023-07-14T16:17:31.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.446+0000 7fb1b3d79b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:31.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.446+0000 7fb1b3d79b80 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.446+0000 7fb1b3d79b80 20 client.16005 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.446+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5573c5bcda30 0x5573c5bcfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.446+0000 7fb1b3d79b80 4 client.16005 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.446+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/1377266437 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573c5b02eb0 con 0x5573c5bcda30 2023-07-14T16:17:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.447+0000 7fb19d99a700 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5573c5bcda30 0x5573c5bcfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.447+0000 7fb19d99a700 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5573c5bcda30 0x5573c5bcfe40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10db5ff521b6f39 server_cookie=fcecb112bb0bbe11 in_seq=0 out_seq=0 2023-07-14T16:17:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.447+0000 7fb181ffb700 10 client.16005 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:31.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb181ffb700 1 -- 172.21.15.62:0/1377266437 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573c5b02eb0 con 0x5573c5bcda30 2023-07-14T16:17:31.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb181ffb700 10 client.16005 handle_command_reply: tid=0 2023-07-14T16:17:31.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 client.16005 shutdown 2023-07-14T16:17:31.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/1377266437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb84d0 msgr2=0x5573c5bc2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb84d0 0x5573c5bc2080 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb194009990 tx=0x7fb19400b9d0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/1377266437 shutdown_connections 2023-07-14T16:17:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5573c5bcda30 0x5573c5bcfe40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 0x5573c5bc9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb84d0 0x5573c5bc2080 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/1377266437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb8e20 0x5573c5bc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb181ffb700 0 client.16005 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/1377266437 >> 172.21.15.62:0/1377266437 conn(0x5573c5badda0 msgr2=0x5573c5bb28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.448+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/1377266437 shutdown_connections 2023-07-14T16:17:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.449+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/1377266437 wait complete. 2023-07-14T16:17:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.449+0000 7fb1b3d79b80 20 client.16005 trim_cache size 0 max 16384 2023-07-14T16:17:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.470+0000 7fb1b3d79b80 1 Processor -- start 2023-07-14T16:17:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.470+0000 7fb1b3d79b80 1 -- start start 2023-07-14T16:17:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.470+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.470+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb8e20 0x5573c5bc95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.470+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb9c70 0x5573c5bbca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.470+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5bbfd10 con 0x5573c5bb9c70 2023-07-14T16:17:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.470+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5bcfe40 con 0x5573c5bb8e20 2023-07-14T16:17:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.470+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5bb98f0 con 0x5573c5bb84d0 2023-07-14T16:17:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19e99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb9c70 0x5573c5bbca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19d99a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb8e20 0x5573c5bc95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19d99a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb8e20 0x5573c5bc95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40934/0 (socket says 172.21.15.62:40934) 2023-07-14T16:17:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19d99a700 1 -- 172.21.15.62:0/2886558405 learned_addr learned my addr 172.21.15.62:0/2886558405 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19e19b700 1 --2- 172.21.15.62:0/2886558405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19e99c700 1 -- 172.21.15.62:0/2886558405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 msgr2=0x5573c5bc0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19e99c700 1 --2- 172.21.15.62:0/2886558405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19e99c700 1 -- 172.21.15.62:0/2886558405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb8e20 msgr2=0x5573c5bc95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19e99c700 1 --2- 172.21.15.62:0/2886558405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb8e20 0x5573c5bc95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.471+0000 7fb19e99c700 1 -- 172.21.15.62:0/2886558405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573c595e250 con 0x5573c5bb9c70 2023-07-14T16:17:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.472+0000 7fb19e99c700 1 --2- 172.21.15.62:0/2886558405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb9c70 0x5573c5bbca90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb19000d7b0 tx=0x7fb190007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c9fa9631a14249 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.472+0000 7fb18d3ff700 1 -- 172.21.15.62:0/2886558405 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190098e00 con 0x5573c5bb9c70 2023-07-14T16:17:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.472+0000 7fb18d3ff700 1 -- 172.21.15.62:0/2886558405 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb190004ad0 con 0x5573c5bb9c70 2023-07-14T16:17:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.472+0000 7fb18d3ff700 1 -- 172.21.15.62:0/2886558405 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19009d170 con 0x5573c5bb9c70 2023-07-14T16:17:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2886558405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb9c70 msgr2=0x5573c5bbca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2886558405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb9c70 0x5573c5bbca90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb19000d7b0 tx=0x7fb190007420 comp rx=0 tx=0).stop 2023-07-14T16:17:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2886558405 shutdown_connections 2023-07-14T16:17:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2886558405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb9c70 0x5573c5bbca90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2886558405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2886558405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb8e20 0x5573c5bc95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2886558405 >> 172.21.15.62:0/2886558405 conn(0x5573c5badda0 msgr2=0x5573c5bd47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2886558405 shutdown_connections 2023-07-14T16:17:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2886558405 wait complete. 2023-07-14T16:17:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.473+0000 7fb1b3d79b80 1 -- start start 2023-07-14T16:17:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.474+0000 7fb1b3d79b80 1 Processor -- start 2023-07-14T16:17:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.474+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.474+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bc94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.474+0000 7fb1b3d79b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 0x5573c5bc5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.474+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5bc6150 con 0x5573c5bb8e20 2023-07-14T16:17:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.474+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5bc62c0 con 0x5573c5bb9c70 2023-07-14T16:17:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.474+0000 7fb1b3d79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c5bc6480 con 0x5573c5bb84d0 2023-07-14T16:17:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb1b3d79b80 10 client.0 _ll_register_callbacks cb 0x5573c5aa7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19d99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bc94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19d99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bc94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46244/0 (socket says 172.21.15.62:46244) 2023-07-14T16:17:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19d99a700 1 -- 172.21.15.62:0/2338079242 learned_addr learned my addr 172.21.15.62:0/2338079242 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19e19b700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19e99c700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 0x5573c5bc5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19d99a700 1 -- 172.21.15.62:0/2338079242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 msgr2=0x5573c5bc8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19d99a700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19d99a700 1 -- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 msgr2=0x5573c5bc5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19d99a700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 0x5573c5bc5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19d99a700 1 -- 172.21.15.62:0/2338079242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19819ffc0 con 0x5573c5bb8e20 2023-07-14T16:17:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19e19b700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.475+0000 7fb19e99c700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 0x5573c5bc5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb19d99a700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bc94c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb188003b70 tx=0x7fb1880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f697c687727a88bf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb181ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb181ffb700 1 -- 172.21.15.62:0/2338079242 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188013660 con 0x5573c5bb8e20 2023-07-14T16:17:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2338079242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb19401e040 con 0x5573c5bb8e20 2023-07-14T16:17:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb181ffb700 1 -- 172.21.15.62:0/2338079242 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb188026030 con 0x5573c5bb8e20 2023-07-14T16:17:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb181ffb700 1 -- 172.21.15.62:0/2338079242 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1880295f0 con 0x5573c5bb8e20 2023-07-14T16:17:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb1b3d79b80 10 client.16011 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2338079242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5573c59362b0 con 0x5573c5bb8e20 2023-07-14T16:17:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.476+0000 7fb181ffb700 1 -- 172.21.15.62:0/2338079242 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb19401e040 con 0x5573c5bb8e20 2023-07-14T16:17:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.477+0000 7fb181ffb700 1 -- 172.21.15.62:0/2338079242 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb18803b070 con 0x5573c5bb8e20 2023-07-14T16:17:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.477+0000 7fb1b3d79b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.477+0000 7fb1b3d79b80 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.477+0000 7fb1b3d79b80 20 client.16011 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.477+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5573c5c22fb0 0x5573c5c253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.477+0000 7fb1b3d79b80 4 client.16011 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:17:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.477+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2338079242 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573c5ac5050 con 0x5573c5c22fb0 2023-07-14T16:17:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.477+0000 7fb19e19b700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5573c5c22fb0 0x5573c5c253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb19e19b700 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5573c5c22fb0 0x5573c5c253c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1657e08170ac47 server_cookie=f33e8880cff143ef in_seq=0 out_seq=0 2023-07-14T16:17:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb181ffb700 10 client.16011 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb181ffb700 1 -- 172.21.15.62:0/2338079242 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573c5ac5050 con 0x5573c5c22fb0 2023-07-14T16:17:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb181ffb700 10 client.16011 handle_command_reply: tid=0 2023-07-14T16:17:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 client.16011 shutdown 2023-07-14T16:17:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2338079242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 msgr2=0x5573c5bc94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bc94c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb188003b70 tx=0x7fb1880047c0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2338079242 shutdown_connections 2023-07-14T16:17:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5573c5c22fb0 0x5573c5c253c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573c5bb9c70 0x5573c5bc5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb181ffb700 0 client.16011 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573c5bb84d0 0x5573c5bc8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 --2- 172.21.15.62:0/2338079242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573c5bb8e20 0x5573c5bc94c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.478+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2338079242 >> 172.21.15.62:0/2338079242 conn(0x5573c5badda0 msgr2=0x5573c5bbca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2338079242 shutdown_connections 2023-07-14T16:17:31.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1b3d79b80 1 -- 172.21.15.62:0/2338079242 wait complete. 2023-07-14T16:17:31.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1b3d79b80 20 client.16011 trim_cache size 0 max 16384 2023-07-14T16:17:31.464 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:17:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1840a1150 msgr2=0x7fb1840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1840a1150 0x7fb1840a3600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb18800d010 tx=0x7fb188016040 comp rx=0 tx=0).stop 2023-07-14T16:17:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198103890 msgr2=0x7fb19819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198103890 0x7fb19819ef80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb190007ee0 tx=0x7fb19000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1681126828 shutdown_connections 2023-07-14T16:17:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb198103890 0x7fb19819ef80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1840a1150 0x7fb1840a3600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb198101d10 0x7fb1981979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 --2- 172.21.15.62:0/1681126828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb198102f00 0x7fb198197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.479+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1681126828 >> 172.21.15.62:0/1681126828 conn(0x7fb1980fd350 msgr2=0x7fb1980ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.480+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1681126828 shutdown_connections 2023-07-14T16:17:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:31.480+0000 7fb1a03ff700 1 -- 172.21.15.62:0/1681126828 wait complete. 2023-07-14T16:17:32.477 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:17:32.477 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:17:32.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.730+0000 7f40af7ff700 1 -- 172.21.15.62:0/3450320801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8102e80 msgr2=0x7f40a8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.730+0000 7f40af7ff700 1 --2- 172.21.15.62:0/3450320801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8102e80 0x7f40a8105260 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4098009990 tx=0x7f4098009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.730+0000 7f40af7ff700 1 -- 172.21.15.62:0/3450320801 shutdown_connections 2023-07-14T16:17:32.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.730+0000 7f40af7ff700 1 --2- 172.21.15.62:0/3450320801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a81057a0 0x7f40a8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.730+0000 7f40af7ff700 1 --2- 172.21.15.62:0/3450320801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a8100560 0x7f40a8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.730+0000 7f40af7ff700 1 --2- 172.21.15.62:0/3450320801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8102e80 0x7f40a8105260 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.730+0000 7f40af7ff700 1 -- 172.21.15.62:0/3450320801 >> 172.21.15.62:0/3450320801 conn(0x7f40a80fa000 msgr2=0x7f40a80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:32.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.731+0000 7f40af7ff700 1 -- 172.21.15.62:0/3450320801 shutdown_connections 2023-07-14T16:17:32.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.731+0000 7f40af7ff700 1 -- 172.21.15.62:0/3450320801 wait complete. 2023-07-14T16:17:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40af7ff700 1 Processor -- start 2023-07-14T16:17:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40af7ff700 1 -- start start 2023-07-14T16:17:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40af7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8100560 0x7f40a8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40af7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a8102e80 0x7f40a8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40af7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a81057a0 0x7f40a819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40af7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a819f4d0 con 0x7f40a8100560 2023-07-14T16:17:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40af7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a819f610 con 0x7f40a81057a0 2023-07-14T16:17:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40af7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a819f7a0 con 0x7f40a8102e80 2023-07-14T16:17:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40ad59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8100560 0x7f40a8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40ad59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8100560 0x7f40a8197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46248/0 (socket says 172.21.15.62:46248) 2023-07-14T16:17:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.732+0000 7f40ad59b700 1 -- 172.21.15.62:0/736596162 learned_addr learned my addr 172.21.15.62:0/736596162 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40acd9a700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a8102e80 0x7f40a8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40ad59b700 1 -- 172.21.15.62:0/736596162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a8102e80 msgr2=0x7f40a8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40ad59b700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a8102e80 0x7f40a8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40add9c700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a81057a0 0x7f40a819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40ad59b700 1 -- 172.21.15.62:0/736596162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a81057a0 msgr2=0x7f40a819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40ad59b700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a81057a0 0x7f40a819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40ad59b700 1 -- 172.21.15.62:0/736596162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40980095f0 con 0x7f40a8100560 2023-07-14T16:17:32.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40add9c700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a81057a0 0x7f40a819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:17:32.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40ad59b700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8100560 0x7f40a8197950 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f40a400e9a0 tx=0x7f40a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:32.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f409e7fc700 1 -- 172.21.15.62:0/736596162 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4009790 con 0x7f40a8100560 2023-07-14T16:17:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f409e7fc700 1 -- 172.21.15.62:0/736596162 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a400cbd0 con 0x7f40a8100560 2023-07-14T16:17:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f40af7ff700 1 -- 172.21.15.62:0/736596162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a819fa80 con 0x7f40a8100560 2023-07-14T16:17:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.733+0000 7f409e7fc700 1 -- 172.21.15.62:0/736596162 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4018680 con 0x7f40a8100560 2023-07-14T16:17:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.734+0000 7f40af7ff700 1 -- 172.21.15.62:0/736596162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a819ffe0 con 0x7f40a8100560 2023-07-14T16:17:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.737+0000 7f409e7fc700 1 -- 172.21.15.62:0/736596162 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f40a400cd40 con 0x7f40a8100560 2023-07-14T16:17:32.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.739+0000 7f409e7fc700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40940a11d0 0x7f40940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.739+0000 7f40c3170b80 1 Processor -- start 2023-07-14T16:17:32.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.739+0000 7f409e7fc700 1 -- 172.21.15.62:0/736596162 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40a4014070 con 0x7f40a8100560 2023-07-14T16:17:32.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.739+0000 7f40c3170b80 1 -- start start 2023-07-14T16:17:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.739+0000 7f40acd9a700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40940a11d0 0x7f40940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40c3170b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d14b6d30 con 0x55c2d15b1ea0 2023-07-14T16:17:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40c3170b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d1509ac0 con 0x55c2d15b2cf0 2023-07-14T16:17:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40c3170b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d1502f50 con 0x55c2d15b1550 2023-07-14T16:17:32.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40acd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40ad59b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40acd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46262/0 (socket says 172.21.15.62:46262) 2023-07-14T16:17:32.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40acd9a700 1 -- 172.21.15.62:0/3155063929 learned_addr learned my addr 172.21.15.62:0/3155063929 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:32.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40add9c700 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40ad59b700 1 -- 172.21.15.62:0/3155063929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 msgr2=0x55c2d15b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:32.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40ad59b700 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40ad59b700 1 -- 172.21.15.62:0/3155063929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 msgr2=0x55c2d15b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:32.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40ad59b700 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.740+0000 7f40ad59b700 1 -- 172.21.15.62:0/3155063929 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2d1330e60 con 0x55c2d15b1550 2023-07-14T16:17:32.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.741+0000 7f40add9c700 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:32.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.741+0000 7f40acd9a700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40940a11d0 0x7f40940a3680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f409800b2e0 tx=0x7f4098005190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:32.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.741+0000 7f40ad59b700 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b1960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c5b0 tx=0x7f40a4099b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601f94f9725e25d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:32.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.741+0000 7f40937fe700 1 -- 172.21.15.62:0/3155063929 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4099f40 con 0x55c2d15b1550 2023-07-14T16:17:32.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.741+0000 7f40937fe700 1 -- 172.21.15.62:0/3155063929 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a409df70 con 0x55c2d15b1550 2023-07-14T16:17:32.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.741+0000 7f40937fe700 1 -- 172.21.15.62:0/3155063929 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40a2160 con 0x55c2d15b1550 2023-07-14T16:17:32.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 -- 172.21.15.62:0/3155063929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 msgr2=0x55c2d15b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b1960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c5b0 tx=0x7f40a4099b00 comp rx=0 tx=0).stop 2023-07-14T16:17:32.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 -- 172.21.15.62:0/3155063929 shutdown_connections 2023-07-14T16:17:32.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b1960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 --2- 172.21.15.62:0/3155063929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 -- 172.21.15.62:0/3155063929 >> 172.21.15.62:0/3155063929 conn(0x55c2d15a6e20 msgr2=0x55c2d15ac250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:32.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 -- 172.21.15.62:0/3155063929 shutdown_connections 2023-07-14T16:17:32.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.742+0000 7f40c3170b80 1 -- 172.21.15.62:0/3155063929 wait complete. 2023-07-14T16:17:32.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.743+0000 7f40c3170b80 1 -- start start 2023-07-14T16:17:32.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.743+0000 7f40c3170b80 1 Processor -- start 2023-07-14T16:17:32.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40c3170b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d14a2b40 con 0x55c2d15b2cf0 2023-07-14T16:17:32.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40c3170b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d14a2840 con 0x55c2d15b1ea0 2023-07-14T16:17:32.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40c3170b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d13869a0 con 0x55c2d15b1550 2023-07-14T16:17:32.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40c3170b80 10 client.0 _ll_register_callbacks cb 0x55c2d14a07e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:32.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40add9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40add9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46278/0 (socket says 172.21.15.62:46278) 2023-07-14T16:17:32.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40add9c700 1 -- 172.21.15.62:0/2757138385 learned_addr learned my addr 172.21.15.62:0/2757138385 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:32.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40ad59b700 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40acd9a700 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.744+0000 7f40ad59b700 1 -- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 msgr2=0x55c2d15bb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:32.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40ad59b700 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40ad59b700 1 -- 172.21.15.62:0/2757138385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 msgr2=0x55c2d15c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:32.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40ad59b700 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40ad59b700 1 -- 172.21.15.62:0/2757138385 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a819ffe0 con 0x55c2d15b1550 2023-07-14T16:17:32.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40acd9a700 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:32.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40ad59b700 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bb1c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c5b0 tx=0x7f40a4005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4a45d4e5369967 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:32.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f4090ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:17:32.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f4090ff9700 1 -- 172.21.15.62:0/2757138385 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4099f40 con 0x55c2d15b1550 2023-07-14T16:17:32.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f4090ff9700 1 -- 172.21.15.62:0/2757138385 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40a4094b50 con 0x55c2d15b1550 2023-07-14T16:17:32.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40c3170b80 1 -- 172.21.15.62:0/2757138385 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2d145c570 con 0x55c2d15b1550 2023-07-14T16:17:32.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f4090ff9700 1 -- 172.21.15.62:0/2757138385 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a40b82a0 con 0x55c2d15b1550 2023-07-14T16:17:32.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40c3170b80 10 client.25699 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:32.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.745+0000 7f40c3170b80 1 -- 172.21.15.62:0/2757138385 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c2d1319ce0 con 0x55c2d15b1550 2023-07-14T16:17:32.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.746+0000 7f4090ff9700 1 -- 172.21.15.62:0/2757138385 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c2d145c570 con 0x55c2d15b1550 2023-07-14T16:17:32.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.746+0000 7f4090ff9700 1 -- 172.21.15.62:0/2757138385 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f40a40caaf0 con 0x55c2d15b1550 2023-07-14T16:17:32.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.746+0000 7f40c3170b80 10 client.25699 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:32.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.746+0000 7f40c3170b80 10 client.25699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:32.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.746+0000 7f40c3170b80 20 client.25699 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:32.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.746+0000 7f40c3170b80 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c2d15c6ad0 0x55c2d15c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.746+0000 7f40c3170b80 4 client.25699 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:32.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.746+0000 7f40c3170b80 1 -- 172.21.15.62:0/2757138385 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2d14fbf30 con 0x55c2d15c6ad0 2023-07-14T16:17:32.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.747+0000 7f40acd9a700 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c2d15c6ad0 0x55c2d15c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.747+0000 7f40acd9a700 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c2d15c6ad0 0x55c2d15c8ee0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854188b28de72f54 server_cookie=73ad8a44bded05d9 in_seq=0 out_seq=0 2023-07-14T16:17:32.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.747+0000 7f4090ff9700 10 client.25699 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:32.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f4090ff9700 1 -- 172.21.15.62:0/2757138385 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c2d14fbf30 con 0x55c2d15c6ad0 2023-07-14T16:17:32.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f4090ff9700 10 client.25699 handle_command_reply: tid=0 2023-07-14T16:17:32.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 client.25699 shutdown 2023-07-14T16:17:32.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 -- 172.21.15.62:0/2757138385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 msgr2=0x55c2d15bb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bb1c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c5b0 tx=0x7f40a4005760 comp rx=0 tx=0).stop 2023-07-14T16:17:32.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 -- 172.21.15.62:0/2757138385 shutdown_connections 2023-07-14T16:17:32.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bb1c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c2d15c6ad0 0x55c2d15c8ee0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 --2- 172.21.15.62:0/2757138385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f4090ff9700 0 client.25699 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:32.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 -- 172.21.15.62:0/2757138385 >> 172.21.15.62:0/2757138385 conn(0x55c2d15a6e20 msgr2=0x55c2d15ab930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:32.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.748+0000 7f40c3170b80 1 -- 172.21.15.62:0/2757138385 shutdown_connections 2023-07-14T16:17:32.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.749+0000 7f40c3170b80 1 -- 172.21.15.62:0/2757138385 wait complete. 2023-07-14T16:17:32.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.749+0000 7f40c3170b80 20 client.25699 trim_cache size 0 max 16384 2023-07-14T16:17:32.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.772+0000 7f40c3170b80 1 Processor -- start 2023-07-14T16:17:32.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.772+0000 7f40c3170b80 1 -- start start 2023-07-14T16:17:32.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40c3170b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d15ad540 con 0x55c2d15b2cf0 2023-07-14T16:17:32.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40c3170b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d15add80 con 0x55c2d15b1ea0 2023-07-14T16:17:32.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40c3170b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d15ac040 con 0x55c2d15b1550 2023-07-14T16:17:32.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40add9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40add9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58098/0 (socket says 172.21.15.62:58098) 2023-07-14T16:17:32.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40add9c700 1 -- 172.21.15.62:0/3054924389 learned_addr learned my addr 172.21.15.62:0/3054924389 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:32.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40ad59b700 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40acd9a700 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40add9c700 1 -- 172.21.15.62:0/3054924389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 msgr2=0x55c2d15b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:32.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40add9c700 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40add9c700 1 -- 172.21.15.62:0/3054924389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 msgr2=0x55c2d15b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:32.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40add9c700 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40add9c700 1 -- 172.21.15.62:0/3054924389 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2d13572d0 con 0x55c2d15b2cf0 2023-07-14T16:17:32.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.773+0000 7f40acd9a700 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:32.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.774+0000 7f40add9c700 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f40a00085e0 tx=0x7f40a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e122bbe3da3e389 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:32.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.774+0000 7f4093fff700 1 -- 172.21.15.62:0/3054924389 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0009280 con 0x55c2d15b2cf0 2023-07-14T16:17:32.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.774+0000 7f4093fff700 1 -- 172.21.15.62:0/3054924389 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a000f030 con 0x55c2d15b2cf0 2023-07-14T16:17:32.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.774+0000 7f4093fff700 1 -- 172.21.15.62:0/3054924389 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a00079a0 con 0x55c2d15b2cf0 2023-07-14T16:17:32.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 -- 172.21.15.62:0/3054924389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 msgr2=0x55c2d15c2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f40a00085e0 tx=0x7f40a000b040 comp rx=0 tx=0).stop 2023-07-14T16:17:32.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 -- 172.21.15.62:0/3054924389 shutdown_connections 2023-07-14T16:17:32.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c2700 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 --2- 172.21.15.62:0/3054924389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b1ea0 0x55c2d15b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 -- 172.21.15.62:0/3054924389 >> 172.21.15.62:0/3054924389 conn(0x55c2d15a6e20 msgr2=0x55c2d15ab930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:32.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 -- 172.21.15.62:0/3054924389 shutdown_connections 2023-07-14T16:17:32.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.775+0000 7f40c3170b80 1 -- 172.21.15.62:0/3054924389 wait complete. 2023-07-14T16:17:32.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.776+0000 7f40c3170b80 1 -- start start 2023-07-14T16:17:32.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.776+0000 7f40c3170b80 1 Processor -- start 2023-07-14T16:17:32.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.776+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.776+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.776+0000 7f40c3170b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.776+0000 7f40c3170b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d15c12d0 con 0x55c2d15b1ea0 2023-07-14T16:17:32.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.776+0000 7f40c3170b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d15c1440 con 0x55c2d15b2cf0 2023-07-14T16:17:32.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.776+0000 7f40c3170b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2d15c1600 con 0x55c2d15b1550 2023-07-14T16:17:32.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40c3170b80 10 client.0 _ll_register_callbacks cb 0x7f40a4099dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:32.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40ad59b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40acd9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40add9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40add9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53670/0 (socket says 172.21.15.62:53670) 2023-07-14T16:17:32.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40add9c700 1 -- 172.21.15.62:0/4219792604 learned_addr learned my addr 172.21.15.62:0/4219792604 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:32.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40add9c700 1 -- 172.21.15.62:0/4219792604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 msgr2=0x55c2d15bf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:32.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40add9c700 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40add9c700 1 -- 172.21.15.62:0/4219792604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 msgr2=0x55c2d15bfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:32.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40add9c700 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.777+0000 7f40add9c700 1 -- 172.21.15.62:0/4219792604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409802e040 con 0x55c2d15b2cf0 2023-07-14T16:17:32.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.778+0000 7f40ad59b700 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:32.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.778+0000 7f40add9c700 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c0d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f40a0000f80 tx=0x7f40a0017ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd878d1a266628c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:32.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.778+0000 7f4090ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:17:32.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.778+0000 7f4090ff9700 1 -- 172.21.15.62:0/4219792604 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0007b30 con 0x55c2d15b2cf0 2023-07-14T16:17:32.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.778+0000 7f4090ff9700 1 -- 172.21.15.62:0/4219792604 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40a0008e80 con 0x55c2d15b2cf0 2023-07-14T16:17:32.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.778+0000 7f40c3170b80 1 -- 172.21.15.62:0/4219792604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40a409df70 con 0x55c2d15b2cf0 2023-07-14T16:17:32.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f4090ff9700 1 -- 172.21.15.62:0/4219792604 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a0009260 con 0x55c2d15b2cf0 2023-07-14T16:17:32.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f40c3170b80 10 client.25778 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:32.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f40c3170b80 1 -- 172.21.15.62:0/4219792604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c2d132f330 con 0x55c2d15b2cf0 2023-07-14T16:17:32.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f4090ff9700 1 -- 172.21.15.62:0/4219792604 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a409df70 con 0x55c2d15b2cf0 2023-07-14T16:17:32.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f4090ff9700 1 -- 172.21.15.62:0/4219792604 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f40a002ad20 con 0x55c2d15b2cf0 2023-07-14T16:17:32.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f40c3170b80 10 client.25778 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:32.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f40c3170b80 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:32.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f40c3170b80 20 client.25778 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:32.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f40c3170b80 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c2d161cc10 0x55c2d161f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:32.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f40c3170b80 4 client.25778 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:17:32.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.779+0000 7f40c3170b80 1 -- 172.21.15.62:0/4219792604 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c2d14be0d0 con 0x55c2d161cc10 2023-07-14T16:17:32.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.780+0000 7f40ad59b700 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c2d161cc10 0x55c2d161f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:32.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.780+0000 7f40ad59b700 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c2d161cc10 0x55c2d161f020 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7841766b4287012c server_cookie=6786e69f7503fb0b in_seq=0 out_seq=0 2023-07-14T16:17:32.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.780+0000 7f4090ff9700 10 client.25778 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:32.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.781+0000 7f4090ff9700 1 -- 172.21.15.62:0/4219792604 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c2d14be0d0 con 0x55c2d161cc10 2023-07-14T16:17:32.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.781+0000 7f4090ff9700 10 client.25778 handle_command_reply: tid=0 2023-07-14T16:17:32.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.781+0000 7f40c3170b80 1 client.25778 shutdown 2023-07-14T16:17:32.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.781+0000 7f40c3170b80 1 -- 172.21.15.62:0/4219792604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 msgr2=0x55c2d15c0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.781+0000 7f40c3170b80 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c0d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f40a0000f80 tx=0x7f40a0017ce0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 1 -- 172.21.15.62:0/4219792604 shutdown_connections 2023-07-14T16:17:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c2d161cc10 0x55c2d161f020 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2d15b2cf0 0x55c2d15c0d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2d15b1550 0x55c2d15bf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f4090ff9700 0 client.25778 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 1 --2- 172.21.15.62:0/4219792604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2d15b1ea0 0x55c2d15bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 1 -- 172.21.15.62:0/4219792604 >> 172.21.15.62:0/4219792604 conn(0x55c2d15a6e20 msgr2=0x55c2d15b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:32.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 1 -- 172.21.15.62:0/4219792604 shutdown_connections 2023-07-14T16:17:32.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 1 -- 172.21.15.62:0/4219792604 wait complete. 2023-07-14T16:17:32.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.782+0000 7f40c3170b80 20 client.25778 trim_cache size 0 max 16384 2023-07-14T16:17:32.767 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:17:32.767 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:17:32.768 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "05ccbc1a-99cd-4d8e-828e-2b363d45bc21", 2023-07-14T16:17:32.768 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:17:32.768 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:17:32.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 -- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40940a11d0 msgr2=0x7f40940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40940a11d0 0x7f40940a3680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f409800b2e0 tx=0x7f4098005190 comp rx=0 tx=0).stop 2023-07-14T16:17:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 -- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8100560 msgr2=0x7f40a8197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8100560 0x7f40a8197950 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f40a400e9a0 tx=0x7f40a400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 -- 172.21.15.62:0/736596162 shutdown_connections 2023-07-14T16:17:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40940a11d0 0x7f40940a3680 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40a81057a0 0x7f40a819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40a8100560 0x7f40a8197950 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 --2- 172.21.15.62:0/736596162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40a8102e80 0x7f40a8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:32.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 -- 172.21.15.62:0/736596162 >> 172.21.15.62:0/736596162 conn(0x7f40a80fa000 msgr2=0x7f40a81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:32.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.783+0000 7f40af7ff700 1 -- 172.21.15.62:0/736596162 shutdown_connections 2023-07-14T16:17:32.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:32.784+0000 7f40af7ff700 1 -- 172.21.15.62:0/736596162 wait complete. 2023-07-14T16:17:32.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:17:32.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:32 smithi062 ceph-mon[95543]: pgmap v414: 97 pgs: 97 active+clean; 50 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.9 MiB/s wr, 68 op/s 2023-07-14T16:17:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 -- 172.21.15.62:0/3717047753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0102e80 msgr2=0x7f8fc0105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/3717047753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0102e80 0x7f8fc0105260 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009990 tx=0x7f8fb4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 -- 172.21.15.62:0/3717047753 shutdown_connections 2023-07-14T16:17:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/3717047753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc01057a0 0x7f8fc0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/3717047753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc0100560 0x7f8fc0102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/3717047753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0102e80 0x7f8fc0105260 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 -- 172.21.15.62:0/3717047753 >> 172.21.15.62:0/3717047753 conn(0x7f8fc00fa000 msgr2=0x7f8fc00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 -- 172.21.15.62:0/3717047753 shutdown_connections 2023-07-14T16:17:33.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.030+0000 7f8fc69ff700 1 -- 172.21.15.62:0/3717047753 wait complete. 2023-07-14T16:17:33.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.031+0000 7f8fc69ff700 1 Processor -- start 2023-07-14T16:17:33.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.031+0000 7f8fc69ff700 1 -- start start 2023-07-14T16:17:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fc69ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0100560 0x7f8fc00ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fc69ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc0102e80 0x7f8fc00fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fc69ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc01057a0 0x7f8fc00fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fc69ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00ff1f0 con 0x7f8fc0100560 2023-07-14T16:17:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fc69ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00ff360 con 0x7f8fc0102e80 2023-07-14T16:17:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fc69ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc019f7c0 con 0x7f8fc01057a0 2023-07-14T16:17:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0100560 0x7f8fc00ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0100560 0x7f8fc00ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58124/0 (socket says 172.21.15.62:58124) 2023-07-14T16:17:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fbffff700 1 -- 172.21.15.62:0/4241380521 learned_addr learned my addr 172.21.15.62:0/4241380521 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc0102e80 0x7f8fc00fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fc4f9c700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc01057a0 0x7f8fc00fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fbffff700 1 -- 172.21.15.62:0/4241380521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc01057a0 msgr2=0x7f8fc00fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fbffff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc01057a0 0x7f8fc00fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fbffff700 1 -- 172.21.15.62:0/4241380521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc0102e80 msgr2=0x7f8fc00fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.032+0000 7f8fbffff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc0102e80 0x7f8fc00fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.033+0000 7f8fbffff700 1 -- 172.21.15.62:0/4241380521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40095f0 con 0x7f8fc0100560 2023-07-14T16:17:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.033+0000 7f8fc4f9c700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc01057a0 0x7f8fc00fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.033+0000 7f8fbffff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0100560 0x7f8fc00ffed0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000e9a0 tx=0x7f8fb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.033+0000 7f8fbd7fa700 1 -- 172.21.15.62:0/4241380521 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0009790 con 0x7f8fc0100560 2023-07-14T16:17:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.033+0000 7f8fbd7fa700 1 -- 172.21.15.62:0/4241380521 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fb000cbd0 con 0x7f8fc0100560 2023-07-14T16:17:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.033+0000 7f8fc69ff700 1 -- 172.21.15.62:0/4241380521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc019faa0 con 0x7f8fc0100560 2023-07-14T16:17:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.033+0000 7f8fbd7fa700 1 -- 172.21.15.62:0/4241380521 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0018680 con 0x7f8fc0100560 2023-07-14T16:17:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.033+0000 7f8fc69ff700 1 -- 172.21.15.62:0/4241380521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc01a0000 con 0x7f8fc0100560 2023-07-14T16:17:33.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.038+0000 7f8fbd7fa700 1 -- 172.21.15.62:0/4241380521 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8fb000cd40 con 0x7f8fc0100560 2023-07-14T16:17:33.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.039+0000 7f8fda240b80 1 Processor -- start 2023-07-14T16:17:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.039+0000 7f8fda240b80 1 -- start start 2023-07-14T16:17:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269db8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fda240b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d268e0cb0 con 0x555d269dcc70 2023-07-14T16:17:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fda240b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d26933a40 con 0x555d269dbe20 2023-07-14T16:17:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fda240b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2692ced0 con 0x555d269db4d0 2023-07-14T16:17:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fbd7fa700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8fa80a11d0 0x7f8fa80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fc4f9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fbd7fa700 1 -- 172.21.15.62:0/4241380521 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8fb0014070 con 0x7f8fc0100560 2023-07-14T16:17:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fc4f9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269db8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47884/0 (socket says 172.21.15.62:47884) 2023-07-14T16:17:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.040+0000 7f8fc4f9c700 1 -- 172.21.15.62:0/2769515653 learned_addr learned my addr 172.21.15.62:0/2769515653 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.041+0000 7f8fc4f9c700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8fa80a11d0 0x7f8fa80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.041+0000 7f8fbffff700 1 -- 172.21.15.62:0/2769515653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 msgr2=0x555d269db8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.041+0000 7f8fbffff700 1 --2- 172.21.15.62:0/2769515653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269db8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.041+0000 7f8fbffff700 1 -- 172.21.15.62:0/2769515653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 msgr2=0x555d269dd0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:17:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.041+0000 7f8fbffff700 1 --2- 172.21.15.62:0/2769515653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.041+0000 7f8fbffff700 1 -- 172.21.15.62:0/2769515653 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d2675ade0 con 0x555d269dbe20 2023-07-14T16:17:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.041+0000 7f8fc4f9c700 1 --2- 172.21.15.62:0/2769515653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269db8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.041+0000 7f8fbffff700 1 --2- 172.21.15.62:0/2769515653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269dc6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00ca650 tx=0x7f8fb008f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5dae69615483116 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.042+0000 7f8fc4f9c700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8fa80a11d0 0x7f8fa80a3680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8012920 tx=0x7f8fb800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.042+0000 7f8fa67fc700 1 -- 172.21.15.62:0/2769515653 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb001c1c0 con 0x555d269dbe20 2023-07-14T16:17:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.042+0000 7f8fa67fc700 1 -- 172.21.15.62:0/2769515653 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fb0099f10 con 0x555d269dbe20 2023-07-14T16:17:33.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.042+0000 7f8fa67fc700 1 -- 172.21.15.62:0/2769515653 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0099400 con 0x555d269dbe20 2023-07-14T16:17:33.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.042+0000 7f8fda240b80 1 -- 172.21.15.62:0/2769515653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 msgr2=0x555d269dc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.042+0000 7f8fda240b80 1 --2- 172.21.15.62:0/2769515653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269dc6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00ca650 tx=0x7f8fb008f8f0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 -- 172.21.15.62:0/2769515653 shutdown_connections 2023-07-14T16:17:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 --2- 172.21.15.62:0/2769515653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 --2- 172.21.15.62:0/2769515653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269db8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 --2- 172.21.15.62:0/2769515653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269dc6e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 -- 172.21.15.62:0/2769515653 >> 172.21.15.62:0/2769515653 conn(0x555d269d0da0 msgr2=0x555d269d61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:33.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 -- 172.21.15.62:0/2769515653 shutdown_connections 2023-07-14T16:17:33.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 -- 172.21.15.62:0/2769515653 wait complete. 2023-07-14T16:17:33.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 -- start start 2023-07-14T16:17:33.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.043+0000 7f8fda240b80 1 Processor -- start 2023-07-14T16:17:33.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fda240b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d268ccac0 con 0x555d269dcc70 2023-07-14T16:17:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fda240b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d268cc7c0 con 0x555d269dbe20 2023-07-14T16:17:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fda240b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d267b0920 con 0x555d269db4d0 2023-07-14T16:17:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fda240b80 10 client.0 _ll_register_callbacks cb 0x555d268ca760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fc4f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fbf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fbf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269e56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53718/0 (socket says 172.21.15.62:53718) 2023-07-14T16:17:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.044+0000 7f8fbf7fe700 1 -- 172.21.15.62:0/1713782574 learned_addr learned my addr 172.21.15.62:0/1713782574 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8fbffff700 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8fbf7fe700 1 -- 172.21.15.62:0/1713782574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 msgr2=0x555d269e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8fbf7fe700 1 -- 172.21.15.62:0/1713782574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 msgr2=0x555d269ec700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269ec700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8fbf7fe700 1 -- 172.21.15.62:0/1713782574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc01a0000 con 0x555d269dbe20 2023-07-14T16:17:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269e56c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a1dd2cd0f7e952b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8f9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:17:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.045+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/1713782574 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb401d070 con 0x555d269dbe20 2023-07-14T16:17:33.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.046+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/1713782574 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8fb4003a90 con 0x555d269dbe20 2023-07-14T16:17:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.046+0000 7f8fda240b80 1 -- 172.21.15.62:0/1713782574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d268864f0 con 0x555d269dbe20 2023-07-14T16:17:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.046+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/1713782574 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fb4003c50 con 0x555d269dbe20 2023-07-14T16:17:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.046+0000 7f8fda240b80 10 client.25790 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.046+0000 7f8fda240b80 1 -- 172.21.15.62:0/1713782574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555d26743c60 con 0x555d269dbe20 2023-07-14T16:17:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.046+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/1713782574 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555d268864f0 con 0x555d269dbe20 2023-07-14T16:17:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.046+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/1713782574 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8fb4004590 con 0x555d269dbe20 2023-07-14T16:17:33.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.047+0000 7f8fda240b80 10 client.25790 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.047+0000 7f8fda240b80 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.047+0000 7f8fda240b80 20 client.25790 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.047+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555d269f1050 0x555d269f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.047+0000 7f8fda240b80 4 client.25790 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.047+0000 7f8fda240b80 1 -- 172.21.15.62:0/1713782574 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d26925eb0 con 0x555d269f1050 2023-07-14T16:17:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.047+0000 7f8fbffff700 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555d269f1050 0x555d269f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.032 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:32 smithi164 ceph-mon[97427]: pgmap v414: 97 pgs: 97 active+clean; 50 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.9 MiB/s wr, 68 op/s 2023-07-14T16:17:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.048+0000 7f8fbffff700 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555d269f1050 0x555d269f3460 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d0ebd879149d18 server_cookie=2588ccd9da5f2d2b in_seq=0 out_seq=0 2023-07-14T16:17:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.048+0000 7f8f9f7fe700 10 client.25790 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.048+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/1713782574 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d26925eb0 con 0x555d269f1050 2023-07-14T16:17:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.048+0000 7f8f9f7fe700 10 client.25790 handle_command_reply: tid=0 2023-07-14T16:17:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.048+0000 7f8fda240b80 1 client.25790 shutdown 2023-07-14T16:17:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 -- 172.21.15.62:0/1713782574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 msgr2=0x555d269e56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269e56c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb4005270 comp rx=0 tx=0).stop 2023-07-14T16:17:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 -- 172.21.15.62:0/1713782574 shutdown_connections 2023-07-14T16:17:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269ec700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555d269f1050 0x555d269f3460 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1713782574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dbe20 0x555d269e56c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 -- 172.21.15.62:0/1713782574 >> 172.21.15.62:0/1713782574 conn(0x555d269d0da0 msgr2=0x555d269d58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8f9f7fe700 0 client.25790 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 -- 172.21.15.62:0/1713782574 shutdown_connections 2023-07-14T16:17:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 1 -- 172.21.15.62:0/1713782574 wait complete. 2023-07-14T16:17:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.049+0000 7f8fda240b80 20 client.25790 trim_cache size 0 max 16384 2023-07-14T16:17:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fda240b80 1 Processor -- start 2023-07-14T16:17:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fda240b80 1 -- start start 2023-07-14T16:17:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269db4d0 0x555d269e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269dbe20 0x555d269ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269e5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fda240b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d269e4d50 con 0x555d269dcc70 2023-07-14T16:17:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fda240b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d269d09c0 con 0x555d269db4d0 2023-07-14T16:17:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fda240b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d269e4630 con 0x555d269dbe20 2023-07-14T16:17:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.071+0000 7f8fc4f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269db4d0 0x555d269e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269db4d0 0x555d269e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53728/0 (socket says 172.21.15.62:53728) 2023-07-14T16:17:33.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 -- 172.21.15.62:0/1000451303 learned_addr learned my addr 172.21.15.62:0/1000451303 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:33.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269dbe20 0x555d269ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 -- 172.21.15.62:0/1000451303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269dbe20 msgr2=0x555d269ec6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269dbe20 0x555d269ec6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 -- 172.21.15.62:0/1000451303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 msgr2=0x555d269e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269e5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 -- 172.21.15.62:0/1000451303 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d26781250 con 0x555d269db4d0 2023-07-14T16:17:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269dbe20 0x555d269ec6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fc4f9c700 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.072+0000 7f8fbffff700 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269db4d0 0x555d269e3090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c5b0 tx=0x7f8fb0099b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c809cf9ed8bdeb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:33.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fa6ffd700 1 -- 172.21.15.62:0/1000451303 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00ccbf0 con 0x555d269db4d0 2023-07-14T16:17:33.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fa6ffd700 1 -- 172.21.15.62:0/1000451303 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fb0099600 con 0x555d269db4d0 2023-07-14T16:17:33.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fa6ffd700 1 -- 172.21.15.62:0/1000451303 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00a1ed0 con 0x555d269db4d0 2023-07-14T16:17:33.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fda240b80 1 -- 172.21.15.62:0/1000451303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269db4d0 msgr2=0x555d269e3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269db4d0 0x555d269e3090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c5b0 tx=0x7f8fb0099b50 comp rx=0 tx=0).stop 2023-07-14T16:17:33.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fda240b80 1 -- 172.21.15.62:0/1000451303 shutdown_connections 2023-07-14T16:17:33.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dcc70 0x555d269e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269db4d0 0x555d269e3090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fda240b80 1 --2- 172.21.15.62:0/1000451303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269dbe20 0x555d269ec6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fda240b80 1 -- 172.21.15.62:0/1000451303 >> 172.21.15.62:0/1000451303 conn(0x555d269d0da0 msgr2=0x555d269e3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:33.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.073+0000 7f8fda240b80 1 -- 172.21.15.62:0/1000451303 shutdown_connections 2023-07-14T16:17:33.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.074+0000 7f8fda240b80 1 -- 172.21.15.62:0/1000451303 wait complete. 2023-07-14T16:17:33.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.074+0000 7f8fda240b80 1 -- start start 2023-07-14T16:17:33.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.074+0000 7f8fda240b80 1 Processor -- start 2023-07-14T16:17:33.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dbe20 0x555d269e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fda240b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dcc70 0x555d269ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fda240b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d269ec050 con 0x555d269dbe20 2023-07-14T16:17:33.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fda240b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d269ec1c0 con 0x555d269dcc70 2023-07-14T16:17:33.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fda240b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d269ec380 con 0x555d269db4d0 2023-07-14T16:17:33.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fda240b80 10 client.0 _ll_register_callbacks cb 0x7f8fb4009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:33.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fbf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dbe20 0x555d269e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fbf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dbe20 0x555d269e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58168/0 (socket says 172.21.15.62:58168) 2023-07-14T16:17:33.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fbf7fe700 1 -- 172.21.15.62:0/3505864062 learned_addr learned my addr 172.21.15.62:0/3505864062 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:33.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.075+0000 7f8fc4f9c700 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dcc70 0x555d269ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fbffff700 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fbf7fe700 1 -- 172.21.15.62:0/3505864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 msgr2=0x555d269e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:33.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fbf7fe700 1 -- 172.21.15.62:0/3505864062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dcc70 msgr2=0x555d269ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:33.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dcc70 0x555d269ebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fbf7fe700 1 -- 172.21.15.62:0/3505864062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d26743c60 con 0x555d269dbe20 2023-07-14T16:17:33.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fbffff700 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:17:33.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fbf7fe700 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dbe20 0x555d269e8af0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009990 tx=0x7f8fb4004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a486719b5b01ddc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:33.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8f9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:33.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/3505864062 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb401d070 con 0x555d269dbe20 2023-07-14T16:17:33.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8fda240b80 1 -- 172.21.15.62:0/3505864062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fb0099f10 con 0x555d269dbe20 2023-07-14T16:17:33.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.076+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/3505864062 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8fb4000e40 con 0x555d269dbe20 2023-07-14T16:17:33.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/3505864062 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fb4004710 con 0x555d269dbe20 2023-07-14T16:17:33.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8fda240b80 10 client.16044 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8fda240b80 1 -- 172.21.15.62:0/3505864062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f8fb80090d0 con 0x555d269dbe20 2023-07-14T16:17:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/3505864062 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fb0099f10 con 0x555d269dbe20 2023-07-14T16:17:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/3505864062 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8fb400b710 con 0x555d269dbe20 2023-07-14T16:17:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8fda240b80 10 client.16044 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8fda240b80 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8fda240b80 20 client.16044 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8fda240b80 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555d26a4e2a0 0x555d26a506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8fda240b80 4 client.16044 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:17:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.077+0000 7f8fda240b80 1 -- 172.21.15.62:0/3505864062 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d268e8050 con 0x555d26a4e2a0 2023-07-14T16:17:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.078+0000 7f8fbffff700 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555d26a4e2a0 0x555d26a506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.078+0000 7f8f9f7fe700 10 client.16044 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.078+0000 7f8fbffff700 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555d26a4e2a0 0x555d26a506b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ac5d1d77ab9f server_cookie=2e16962fe162ac73 in_seq=0 out_seq=0 2023-07-14T16:17:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.078+0000 7f8f9f7fe700 1 -- 172.21.15.62:0/3505864062 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555d268e8050 con 0x555d26a4e2a0 2023-07-14T16:17:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.078+0000 7f8f9f7fe700 10 client.16044 handle_command_reply: tid=0 2023-07-14T16:17:33.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.078+0000 7f8fda240b80 1 client.16044 shutdown 2023-07-14T16:17:33.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 -- 172.21.15.62:0/3505864062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dbe20 msgr2=0x555d269e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dbe20 0x555d269e8af0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009990 tx=0x7f8fb4004f20 comp rx=0 tx=0).stop 2023-07-14T16:17:33.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 -- 172.21.15.62:0/3505864062 shutdown_connections 2023-07-14T16:17:33.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555d26a4e2a0 0x555d26a506b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d269dcc70 0x555d269ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d269db4d0 0x555d269e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8f9f7fe700 0 client.16044 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:33.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 --2- 172.21.15.62:0/3505864062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d269dbe20 0x555d269e8af0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 -- 172.21.15.62:0/3505864062 >> 172.21.15.62:0/3505864062 conn(0x555d269d0da0 msgr2=0x555d269e3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:33.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 -- 172.21.15.62:0/3505864062 shutdown_connections 2023-07-14T16:17:33.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 1 -- 172.21.15.62:0/3505864062 wait complete. 2023-07-14T16:17:33.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.079+0000 7f8fda240b80 20 client.16044 trim_cache size 0 max 16384 2023-07-14T16:17:33.066 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:17:33.066 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:17:33.066 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:17:33.066 INFO:teuthology.orchestra.run.smithi062.stdout: "05ccbc1a-99cd-4d8e-828e-2b363d45bc21": { 2023-07-14T16:17:33.066 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:17:33.066 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "05ccbc1a-99cd-4d8e-828e-2b363d45bc21", 2023-07-14T16:17:33.066 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:17:33.067 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:17:33.067 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:17:33.067 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:17:33.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 -- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8fa80a11d0 msgr2=0x7f8fa80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8fa80a11d0 0x7f8fa80a3680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8012920 tx=0x7f8fb800b040 comp rx=0 tx=0).stop 2023-07-14T16:17:33.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 -- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0100560 msgr2=0x7f8fc00ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:33.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0100560 0x7f8fc00ffed0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000e9a0 tx=0x7f8fb000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 -- 172.21.15.62:0/4241380521 shutdown_connections 2023-07-14T16:17:33.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc01057a0 0x7f8fc00fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fc0100560 0x7f8fc00ffed0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8fa80a11d0 0x7f8fa80a3680 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 --2- 172.21.15.62:0/4241380521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc0102e80 0x7f8fc00fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:33.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 -- 172.21.15.62:0/4241380521 >> 172.21.15.62:0/4241380521 conn(0x7f8fc00fa000 msgr2=0x7f8fc01011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:33.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 -- 172.21.15.62:0/4241380521 shutdown_connections 2023-07-14T16:17:33.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:17:33.080+0000 7f8fc69ff700 1 -- 172.21.15.62:0/4241380521 wait complete. 2023-07-14T16:17:33.076 INFO:tasks.cephfs.filesystem:scrub status for tag:05ccbc1a-99cd-4d8e-828e-2b363d45bc21 - {'path': '/', 'tag': '05ccbc1a-99cd-4d8e-828e-2b363d45bc21', 'options': 'recursive,force'} 2023-07-14T16:17:33.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:32 smithi081 ceph-mon[127795]: pgmap v414: 97 pgs: 97 active+clean; 50 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.9 MiB/s wr, 68 op/s 2023-07-14T16:17:34.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:17:34.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:17:34.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:17:34.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:17:34.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:17:34.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:17:34.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:33 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:17:34.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:17:34.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:17:34.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:17:34.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:17:34.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:17:34.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:33 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:17:34.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:17:34.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:17:34.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:17:34.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:17:34.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:17:34.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:34.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:33 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:17:35.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:34 smithi164 ceph-mon[97427]: pgmap v415: 97 pgs: 97 active+clean; 82 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.6 MiB/s wr, 70 op/s 2023-07-14T16:17:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:34 smithi062 ceph-mon[95543]: pgmap v415: 97 pgs: 97 active+clean; 82 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.6 MiB/s wr, 70 op/s 2023-07-14T16:17:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:34 smithi081 ceph-mon[127795]: pgmap v415: 97 pgs: 97 active+clean; 82 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.6 MiB/s wr, 70 op/s 2023-07-14T16:17:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.536+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2178216204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628102bd0 msgr2=0x7fd628105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.537+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2178216204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628102bd0 0x7fd628105050 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009990 tx=0x7fd61c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.537+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2178216204 shutdown_connections 2023-07-14T16:17:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.537+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2178216204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6281056e0 0x7fd628107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.537+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2178216204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd628068640 0x7fd628102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.537+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2178216204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628102bd0 0x7fd628105050 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.537+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2178216204 >> 172.21.15.62:0/2178216204 conn(0x7fd6280fa000 msgr2=0x7fd6280fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.537+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2178216204 shutdown_connections 2023-07-14T16:17:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.537+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2178216204 wait complete. 2023-07-14T16:17:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.538+0000 7fd62e3ff700 1 Processor -- start 2023-07-14T16:17:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.538+0000 7fd62e3ff700 1 -- start start 2023-07-14T16:17:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd62e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628068640 0x7fd6280fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd62e3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd628102bd0 0x7fd6280feb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd62e3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6281056e0 0x7fd6280ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd62e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62819f7e0 con 0x7fd628068640 2023-07-14T16:17:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd62e3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62819f920 con 0x7fd628102bd0 2023-07-14T16:17:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd62e3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62819fa60 con 0x7fd6281056e0 2023-07-14T16:17:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd627fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628068640 0x7fd6280fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd627fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628068640 0x7fd6280fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58182/0 (socket says 172.21.15.62:58182) 2023-07-14T16:17:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd627fff700 1 -- 172.21.15.62:0/2839410783 learned_addr learned my addr 172.21.15.62:0/2839410783 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd62c99c700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6281056e0 0x7fd6280ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd6277fe700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd628102bd0 0x7fd6280feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd627fff700 1 -- 172.21.15.62:0/2839410783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6281056e0 msgr2=0x7fd6280ff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd627fff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6281056e0 0x7fd6280ff0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd627fff700 1 -- 172.21.15.62:0/2839410783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd628102bd0 msgr2=0x7fd6280feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd627fff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd628102bd0 0x7fd6280feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.539+0000 7fd627fff700 1 -- 172.21.15.62:0/2839410783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0095f0 con 0x7fd628068640 2023-07-14T16:17:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.540+0000 7fd627fff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628068640 0x7fd6280fe630 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd61800d950 tx=0x7fd61800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.540+0000 7fd6257fa700 1 -- 172.21.15.62:0/2839410783 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618004220 con 0x7fd628068640 2023-07-14T16:17:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.540+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2839410783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62819fcf0 con 0x7fd628068640 2023-07-14T16:17:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.540+0000 7fd6257fa700 1 -- 172.21.15.62:0/2839410783 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd618007620 con 0x7fd628068640 2023-07-14T16:17:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.540+0000 7fd6257fa700 1 -- 172.21.15.62:0/2839410783 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618005530 con 0x7fd628068640 2023-07-14T16:17:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.540+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2839410783 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6281a01a0 con 0x7fd628068640 2023-07-14T16:17:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.544+0000 7fd6257fa700 1 -- 172.21.15.62:0/2839410783 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd618004750 con 0x7fd628068640 2023-07-14T16:17:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.546+0000 7fd641d00b80 1 Processor -- start 2023-07-14T16:17:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.546+0000 7fd641d00b80 1 -- start start 2023-07-14T16:17:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.546+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c4826b0 0x55dd2c626300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.546+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c6271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.546+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c6276e0 0x55dd2c627b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.546+0000 7fd641d00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c52bc50 con 0x55dd2c626840 2023-07-14T16:17:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.546+0000 7fd641d00b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c57e9e0 con 0x55dd2c6276e0 2023-07-14T16:17:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.546+0000 7fd641d00b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c577e70 con 0x55dd2c4826b0 2023-07-14T16:17:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.547+0000 7fd6257fa700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd6100a11d0 0x7fd6100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.547+0000 7fd6257fa700 1 -- 172.21.15.62:0/2839410783 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd6180cc750 con 0x7fd628068640 2023-07-14T16:17:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.547+0000 7fd62c99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c4826b0 0x55dd2c626300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.547+0000 7fd62c99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c4826b0 0x55dd2c626300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47958/0 (socket says 172.21.15.62:47958) 2023-07-14T16:17:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.547+0000 7fd62c99c700 1 -- 172.21.15.62:0/16737249 learned_addr learned my addr 172.21.15.62:0/16737249 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.547+0000 7fd627fff700 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c6271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.547+0000 7fd62c99c700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd6100a11d0 0x7fd6100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.547+0000 7fd6277fe700 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c6276e0 0x55dd2c627b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd627fff700 1 -- 172.21.15.62:0/16737249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c4826b0 msgr2=0x55dd2c626300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd627fff700 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c4826b0 0x55dd2c626300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd627fff700 1 -- 172.21.15.62:0/16737249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c6276e0 msgr2=0x55dd2c627b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd627fff700 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c6276e0 0x55dd2c627b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd627fff700 1 -- 172.21.15.62:0/16737249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2c3a5d80 con 0x55dd2c626840 2023-07-14T16:17:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd627fff700 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c6271a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd6180ca5d0 tx=0x7fd618000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c674495edf3ea54 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd62c99c700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd6100a11d0 0x7fd6100a3680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd620007b90 tx=0x7fd620007950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd60e7fc700 1 -- 172.21.15.62:0/16737249 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618099b70 con 0x55dd2c626840 2023-07-14T16:17:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd60e7fc700 1 -- 172.21.15.62:0/16737249 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd61809dba0 con 0x55dd2c626840 2023-07-14T16:17:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.548+0000 7fd60e7fc700 1 -- 172.21.15.62:0/16737249 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61809a040 con 0x55dd2c626840 2023-07-14T16:17:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.549+0000 7fd641d00b80 1 -- 172.21.15.62:0/16737249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 msgr2=0x55dd2c6271a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.549+0000 7fd641d00b80 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c6271a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd6180ca5d0 tx=0x7fd618000bc0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.549+0000 7fd641d00b80 1 -- 172.21.15.62:0/16737249 shutdown_connections 2023-07-14T16:17:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.549+0000 7fd641d00b80 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c6276e0 0x55dd2c627b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.549+0000 7fd641d00b80 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c4826b0 0x55dd2c626300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.549+0000 7fd641d00b80 1 --2- 172.21.15.62:0/16737249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c6271a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.549+0000 7fd641d00b80 1 -- 172.21.15.62:0/16737249 >> 172.21.15.62:0/16737249 conn(0x55dd2c61bbe0 msgr2=0x55dd2c620cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.550+0000 7fd641d00b80 1 -- 172.21.15.62:0/16737249 shutdown_connections 2023-07-14T16:17:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.550+0000 7fd641d00b80 1 -- 172.21.15.62:0/16737249 wait complete. 2023-07-14T16:17:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.550+0000 7fd641d00b80 1 -- start start 2023-07-14T16:17:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.550+0000 7fd641d00b80 1 Processor -- start 2023-07-14T16:17:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c62f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c626840 0x55dd2c62fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c6276e0 0x55dd2c636c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd641d00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c517a60 con 0x55dd2c6276e0 2023-07-14T16:17:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd641d00b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c517760 con 0x55dd2c4826b0 2023-07-14T16:17:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd641d00b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c3fb8c0 con 0x55dd2c626840 2023-07-14T16:17:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd641d00b80 10 client.0 _ll_register_callbacks cb 0x55dd2c515700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd62c99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c6276e0 0x55dd2c636c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd6277fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c626840 0x55dd2c62fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd62c99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c6276e0 0x55dd2c636c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58206/0 (socket says 172.21.15.62:58206) 2023-07-14T16:17:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd62c99c700 1 -- 172.21.15.62:0/3485055864 learned_addr learned my addr 172.21.15.62:0/3485055864 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.551+0000 7fd627fff700 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c62f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.552+0000 7fd6277fe700 1 -- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 msgr2=0x55dd2c62f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.552+0000 7fd6277fe700 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c62f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.552+0000 7fd6277fe700 1 -- 172.21.15.62:0/3485055864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c6276e0 msgr2=0x55dd2c636c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.552+0000 7fd6277fe700 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c6276e0 0x55dd2c636c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.552+0000 7fd6277fe700 1 -- 172.21.15.62:0/3485055864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6281a01a0 con 0x55dd2c626840 2023-07-14T16:17:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.552+0000 7fd62c99c700 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c6276e0 0x55dd2c636c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.552+0000 7fd627fff700 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c62f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.553+0000 7fd6277fe700 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c626840 0x55dd2c62fbc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c010 tx=0x7fd61c00bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eae9fb85a364804 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.553+0000 7fd6077fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:17:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.553+0000 7fd6077fe700 1 -- 172.21.15.62:0/3485055864 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c005450 con 0x55dd2c626840 2023-07-14T16:17:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.553+0000 7fd6077fe700 1 -- 172.21.15.62:0/3485055864 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd61c0055b0 con 0x55dd2c626840 2023-07-14T16:17:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.553+0000 7fd641d00b80 1 -- 172.21.15.62:0/3485055864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd2c4d1490 con 0x55dd2c626840 2023-07-14T16:17:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.554+0000 7fd6077fe700 1 -- 172.21.15.62:0/3485055864 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd61c004500 con 0x55dd2c626840 2023-07-14T16:17:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.554+0000 7fd641d00b80 10 client.25729 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.554+0000 7fd641d00b80 1 -- 172.21.15.62:0/3485055864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dd2c38ec00 con 0x55dd2c626840 2023-07-14T16:17:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.554+0000 7fd6077fe700 1 -- 172.21.15.62:0/3485055864 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dd2c4d1490 con 0x55dd2c626840 2023-07-14T16:17:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.555+0000 7fd6077fe700 1 -- 172.21.15.62:0/3485055864 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd61c0039b0 con 0x55dd2c626840 2023-07-14T16:17:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.555+0000 7fd641d00b80 10 client.25729 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.555+0000 7fd641d00b80 10 client.25729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.555+0000 7fd641d00b80 20 client.25729 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.555+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55dd2c63b520 0x55dd2c63d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.555+0000 7fd641d00b80 4 client.25729 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.555+0000 7fd641d00b80 1 -- 172.21.15.62:0/3485055864 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd2c570e50 con 0x55dd2c63b520 2023-07-14T16:17:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.556+0000 7fd627fff700 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55dd2c63b520 0x55dd2c63d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.556+0000 7fd627fff700 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55dd2c63b520 0x55dd2c63d930 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb57f976ac4cdd7 server_cookie=9de84eb0856d7e09 in_seq=0 out_seq=0 2023-07-14T16:17:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.556+0000 7fd6077fe700 10 client.25729 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd6077fe700 1 -- 172.21.15.62:0/3485055864 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd2c570e50 con 0x55dd2c63b520 2023-07-14T16:17:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd6077fe700 10 client.25729 handle_command_reply: tid=0 2023-07-14T16:17:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 client.25729 shutdown 2023-07-14T16:17:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 -- 172.21.15.62:0/3485055864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c626840 msgr2=0x55dd2c62fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c626840 0x55dd2c62fbc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c010 tx=0x7fd61c00bee0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 -- 172.21.15.62:0/3485055864 shutdown_connections 2023-07-14T16:17:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c6276e0 0x55dd2c636c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55dd2c63b520 0x55dd2c63d930 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c62f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3485055864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c626840 0x55dd2c62fbc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd6077fe700 0 client.25729 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 -- 172.21.15.62:0/3485055864 >> 172.21.15.62:0/3485055864 conn(0x55dd2c61bbe0 msgr2=0x55dd2c6204e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 -- 172.21.15.62:0/3485055864 shutdown_connections 2023-07-14T16:17:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.557+0000 7fd641d00b80 1 -- 172.21.15.62:0/3485055864 wait complete. 2023-07-14T16:17:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.558+0000 7fd641d00b80 20 client.25729 trim_cache size 0 max 16384 2023-07-14T16:17:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.578+0000 7fd641d00b80 1 Processor -- start 2023-07-14T16:17:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.578+0000 7fd641d00b80 1 -- start start 2023-07-14T16:17:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.578+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c627ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.578+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c636bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.578+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c62a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.578+0000 7fd641d00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c62f250 con 0x55dd2c626840 2023-07-14T16:17:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.578+0000 7fd641d00b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c4d1490 con 0x55dd2c4826b0 2023-07-14T16:17:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.578+0000 7fd641d00b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c621bd0 con 0x55dd2c6276e0 2023-07-14T16:17:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd6277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c636bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd62c99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c62a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd62c99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c62a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47976/0 (socket says 172.21.15.62:47976) 2023-07-14T16:17:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd62c99c700 1 -- 172.21.15.62:0/3117669450 learned_addr learned my addr 172.21.15.62:0/3117669450 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd627fff700 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c627ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd6277fe700 1 -- 172.21.15.62:0/3117669450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 msgr2=0x55dd2c62a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd6277fe700 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c62a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd6277fe700 1 -- 172.21.15.62:0/3117669450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 msgr2=0x55dd2c627ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd6277fe700 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c627ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd6277fe700 1 -- 172.21.15.62:0/3117669450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2c3cc1f0 con 0x55dd2c626840 2023-07-14T16:17:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd62c99c700 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c62a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd627fff700 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c627ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:17:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.579+0000 7fd6277fe700 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c636bc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000c00 tx=0x7fd61c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfd1a94efc5b78c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.580+0000 7fd60effd700 1 -- 172.21.15.62:0/3117669450 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c009ee0 con 0x55dd2c626840 2023-07-14T16:17:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.580+0000 7fd60effd700 1 -- 172.21.15.62:0/3117669450 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd61c01e050 con 0x55dd2c626840 2023-07-14T16:17:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.580+0000 7fd60effd700 1 -- 172.21.15.62:0/3117669450 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c004760 con 0x55dd2c626840 2023-07-14T16:17:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.580+0000 7fd641d00b80 1 -- 172.21.15.62:0/3117669450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 msgr2=0x55dd2c636bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.580+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c636bc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000c00 tx=0x7fd61c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 -- 172.21.15.62:0/3117669450 shutdown_connections 2023-07-14T16:17:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c62a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c627ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 --2- 172.21.15.62:0/3117669450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c636bc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 -- 172.21.15.62:0/3117669450 >> 172.21.15.62:0/3117669450 conn(0x55dd2c61bbe0 msgr2=0x55dd2c642290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 -- 172.21.15.62:0/3117669450 shutdown_connections 2023-07-14T16:17:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 -- 172.21.15.62:0/3117669450 wait complete. 2023-07-14T16:17:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 -- start start 2023-07-14T16:17:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.581+0000 7fd641d00b80 1 Processor -- start 2023-07-14T16:17:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c6326c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c632c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd641d00b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c635c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd641d00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c636160 con 0x55dd2c626840 2023-07-14T16:17:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd641d00b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c6362d0 con 0x55dd2c4826b0 2023-07-14T16:17:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd641d00b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2c636490 con 0x55dd2c6276e0 2023-07-14T16:17:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd641d00b80 10 client.0 _ll_register_callbacks cb 0x7fd61c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd6277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c632c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd6277fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c632c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58216/0 (socket says 172.21.15.62:58216) 2023-07-14T16:17:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd6277fe700 1 -- 172.21.15.62:0/4242621920 learned_addr learned my addr 172.21.15.62:0/4242621920 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.582+0000 7fd627fff700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c6326c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd62c99c700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c635c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6277fe700 1 -- 172.21.15.62:0/4242621920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 msgr2=0x55dd2c635c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6277fe700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c635c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6277fe700 1 -- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 msgr2=0x55dd2c6326c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6277fe700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c6326c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6277fe700 1 -- 172.21.15.62:0/4242621920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd2c3a5d80 con 0x55dd2c626840 2023-07-14T16:17:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd627fff700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c6326c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd62c99c700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c635c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:17:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6277fe700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c632c00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009990 tx=0x7fd61c005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1cb455a69490bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6077fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6077fe700 1 -- 172.21.15.62:0/4242621920 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0058e0 con 0x55dd2c626840 2023-07-14T16:17:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd641d00b80 1 -- 172.21.15.62:0/4242621920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd61809dba0 con 0x55dd2c626840 2023-07-14T16:17:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.583+0000 7fd6077fe700 1 -- 172.21.15.62:0/4242621920 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd61c000e40 con 0x55dd2c626840 2023-07-14T16:17:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd6077fe700 1 -- 172.21.15.62:0/4242621920 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd61c048030 con 0x55dd2c626840 2023-07-14T16:17:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd641d00b80 10 client.16062 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd641d00b80 1 -- 172.21.15.62:0/4242621920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fd620005e70 con 0x55dd2c626840 2023-07-14T16:17:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd6077fe700 1 -- 172.21.15.62:0/4242621920 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd61809dba0 con 0x55dd2c626840 2023-07-14T16:17:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd6077fe700 1 -- 172.21.15.62:0/4242621920 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd61c036070 con 0x55dd2c626840 2023-07-14T16:17:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd641d00b80 10 client.16062 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd641d00b80 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd641d00b80 20 client.16062 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd641d00b80 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55dd2c68fc20 0x55dd2c692030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd641d00b80 4 client.16062 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:17:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.584+0000 7fd641d00b80 1 -- 172.21.15.62:0/4242621920 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dd2c532ff0 con 0x55dd2c68fc20 2023-07-14T16:17:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.585+0000 7fd627fff700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55dd2c68fc20 0x55dd2c692030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.585+0000 7fd627fff700 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55dd2c68fc20 0x55dd2c692030 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be913a6ce6e37b2c server_cookie=e8c878f0dfda0994 in_seq=0 out_seq=0 2023-07-14T16:17:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.585+0000 7fd6077fe700 10 client.16062 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.603+0000 7fd6077fe700 1 -- 172.21.15.62:0/4242621920 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dd2c532ff0 con 0x55dd2c68fc20 2023-07-14T16:17:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd6077fe700 10 client.16062 handle_command_reply: tid=0 2023-07-14T16:17:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 client.16062 shutdown 2023-07-14T16:17:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 -- 172.21.15.62:0/4242621920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 msgr2=0x55dd2c632c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c632c00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009990 tx=0x7fd61c005cb0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 -- 172.21.15.62:0/4242621920 shutdown_connections 2023-07-14T16:17:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd2c6276e0 0x55dd2c635c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55dd2c68fc20 0x55dd2c692030 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd2c4826b0 0x55dd2c6326c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 --2- 172.21.15.62:0/4242621920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd2c626840 0x55dd2c632c00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 -- 172.21.15.62:0/4242621920 >> 172.21.15.62:0/4242621920 conn(0x55dd2c61bbe0 msgr2=0x55dd2c642290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd6077fe700 0 client.16062 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 -- 172.21.15.62:0/4242621920 shutdown_connections 2023-07-14T16:17:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 1 -- 172.21.15.62:0/4242621920 wait complete. 2023-07-14T16:17:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.604+0000 7fd641d00b80 20 client.16062 trim_cache size 0 max 16384 2023-07-14T16:17:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:17:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:17:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:17:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:17:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd6100a11d0 msgr2=0x7fd6100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd6100a11d0 0x7fd6100a3680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd620007b90 tx=0x7fd620007950 comp rx=0 tx=0).stop 2023-07-14T16:17:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628068640 msgr2=0x7fd6280fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628068640 0x7fd6280fe630 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd61800d950 tx=0x7fd61800dc60 comp rx=0 tx=0).stop 2023-07-14T16:17:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2839410783 shutdown_connections 2023-07-14T16:17:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6281056e0 0x7fd6280ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628068640 0x7fd6280fe630 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd6100a11d0 0x7fd6100a3680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 --2- 172.21.15.62:0/2839410783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd628102bd0 0x7fd6280feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2839410783 >> 172.21.15.62:0/2839410783 conn(0x7fd6280fa000 msgr2=0x7fd628100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2839410783 shutdown_connections 2023-07-14T16:17:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:35.605+0000 7fd62e3ff700 1 -- 172.21.15.62:0/2839410783 wait complete. 2023-07-14T16:17:37.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:36 smithi164 ceph-mon[97427]: pgmap v416: 97 pgs: 97 active+clean; 102 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.1 MiB/s wr, 51 op/s 2023-07-14T16:17:37.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:36 smithi062 ceph-mon[95543]: pgmap v416: 97 pgs: 97 active+clean; 102 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.1 MiB/s wr, 51 op/s 2023-07-14T16:17:37.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:36 smithi081 ceph-mon[127795]: pgmap v416: 97 pgs: 97 active+clean; 102 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.1 MiB/s wr, 51 op/s 2023-07-14T16:17:39.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:38 smithi164 ceph-mon[97427]: pgmap v417: 97 pgs: 97 active+clean; 146 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 54 op/s 2023-07-14T16:17:39.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:38 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:17:39.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:38 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:17:39.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:38 smithi062 ceph-mon[95543]: pgmap v417: 97 pgs: 97 active+clean; 146 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 54 op/s 2023-07-14T16:17:39.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:38 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:17:39.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:38 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:17:39.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:38 smithi081 ceph-mon[127795]: pgmap v417: 97 pgs: 97 active+clean; 146 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 54 op/s 2023-07-14T16:17:39.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:38 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:17:39.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:38 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:17:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:40 smithi164 ceph-mon[97427]: pgmap v418: 97 pgs: 97 active+clean; 178 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 49 op/s 2023-07-14T16:17:41.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:40 smithi062 ceph-mon[95543]: pgmap v418: 97 pgs: 97 active+clean; 178 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 49 op/s 2023-07-14T16:17:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:40 smithi081 ceph-mon[127795]: pgmap v418: 97 pgs: 97 active+clean; 178 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 49 op/s 2023-07-14T16:17:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:42 smithi164 ceph-mon[97427]: pgmap v419: 97 pgs: 97 active+clean; 206 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 44 op/s 2023-07-14T16:17:43.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:42 smithi062 ceph-mon[95543]: pgmap v419: 97 pgs: 97 active+clean; 206 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 44 op/s 2023-07-14T16:17:43.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:42 smithi081 ceph-mon[127795]: pgmap v419: 97 pgs: 97 active+clean; 206 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 44 op/s 2023-07-14T16:17:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:44 smithi164 ceph-mon[97427]: pgmap v420: 97 pgs: 97 active+clean; 242 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 44 op/s 2023-07-14T16:17:45.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:44 smithi062 ceph-mon[95543]: pgmap v420: 97 pgs: 97 active+clean; 242 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 44 op/s 2023-07-14T16:17:45.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:44 smithi081 ceph-mon[127795]: pgmap v420: 97 pgs: 97 active+clean; 242 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 16 MiB/s wr, 44 op/s 2023-07-14T16:17:47.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:46 smithi164 ceph-mon[97427]: pgmap v421: 97 pgs: 97 active+clean; 254 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:17:47.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:46 smithi062 ceph-mon[95543]: pgmap v421: 97 pgs: 97 active+clean; 254 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:17:47.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:46 smithi081 ceph-mon[127795]: pgmap v421: 97 pgs: 97 active+clean; 254 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:17:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 -- 172.21.15.62:0/898699202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c100b30 msgr2=0x7fc80c100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 --2- 172.21.15.62:0/898699202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c100b30 0x7fc80c100f80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc800009990 tx=0x7fc800009ca0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 -- 172.21.15.62:0/898699202 shutdown_connections 2023-07-14T16:17:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 --2- 172.21.15.62:0/898699202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc80c1014c0 0x7fc80c107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 --2- 172.21.15.62:0/898699202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc80c0ff940 0x7fc80c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 --2- 172.21.15.62:0/898699202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c100b30 0x7fc80c100f80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 -- 172.21.15.62:0/898699202 >> 172.21.15.62:0/898699202 conn(0x7fc80c0faf80 msgr2=0x7fc80c0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 -- 172.21.15.62:0/898699202 shutdown_connections 2023-07-14T16:17:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.769+0000 7fc81139d700 1 -- 172.21.15.62:0/898699202 wait complete. 2023-07-14T16:17:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.770+0000 7fc81139d700 1 Processor -- start 2023-07-14T16:17:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.770+0000 7fc81139d700 1 -- start start 2023-07-14T16:17:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc81139d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c0ff940 0x7fc80c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc81139d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc80c100b30 0x7fc80c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc81139d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc80c1014c0 0x7fc80c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc81139d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c19f2f0 con 0x7fc80c0ff940 2023-07-14T16:17:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc81139d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c19f430 con 0x7fc80c1014c0 2023-07-14T16:17:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc81139d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c19f5c0 con 0x7fc80c100b30 2023-07-14T16:17:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c0ff940 0x7fc80c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c0ff940 0x7fc80c197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34304/0 (socket says 172.21.15.62:34304) 2023-07-14T16:17:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80bfff700 1 -- 172.21.15.62:0/2755610975 learned_addr learned my addr 172.21.15.62:0/2755610975 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc810b9c700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc80c1014c0 0x7fc80c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc80c100b30 0x7fc80c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80bfff700 1 -- 172.21.15.62:0/2755610975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc80c100b30 msgr2=0x7fc80c197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80bfff700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc80c100b30 0x7fc80c197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80bfff700 1 -- 172.21.15.62:0/2755610975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc80c1014c0 msgr2=0x7fc80c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80bfff700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc80c1014c0 0x7fc80c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.771+0000 7fc80bfff700 1 -- 172.21.15.62:0/2755610975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000095f0 con 0x7fc80c0ff940 2023-07-14T16:17:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.772+0000 7fc80bfff700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c0ff940 0x7fc80c197770 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00e970 tx=0x7fc7fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.772+0000 7fc8097fa700 1 -- 172.21.15.62:0/2755610975 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc009790 con 0x7fc80c0ff940 2023-07-14T16:17:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.772+0000 7fc8097fa700 1 -- 172.21.15.62:0/2755610975 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7fc00c7e0 con 0x7fc80c0ff940 2023-07-14T16:17:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.772+0000 7fc81139d700 1 -- 172.21.15.62:0/2755610975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c19f8a0 con 0x7fc80c0ff940 2023-07-14T16:17:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.772+0000 7fc8097fa700 1 -- 172.21.15.62:0/2755610975 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc003bd0 con 0x7fc80c0ff940 2023-07-14T16:17:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.772+0000 7fc81139d700 1 -- 172.21.15.62:0/2755610975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c19fe00 con 0x7fc80c0ff940 2023-07-14T16:17:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.777+0000 7fc8097fa700 1 -- 172.21.15.62:0/2755610975 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc7fc00c950 con 0x7fc80c0ff940 2023-07-14T16:17:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.778+0000 7fc825ff6b80 1 Processor -- start 2023-07-14T16:17:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc825ff6b80 1 -- start start 2023-07-14T16:17:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc8097fa700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc7f40a11d0 0x7fc7f40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f960550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc8097fa700 1 -- 172.21.15.62:0/2755610975 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc7fc014070 con 0x7fc80c0ff940 2023-07-14T16:17:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f9612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f961c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f865c50 con 0x55606f960a90 2023-07-14T16:17:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f8b89e0 con 0x55606f7bc6b0 2023-07-14T16:17:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f8b1e70 con 0x55606f9617e0 2023-07-14T16:17:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc810b9c700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc7f40a11d0 0x7fc7f40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc80b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f9612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.779+0000 7fc80b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f9612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34310/0 (socket says 172.21.15.62:34310) 2023-07-14T16:17:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc80b7fe700 1 -- 172.21.15.62:0/1937354489 learned_addr learned my addr 172.21.15.62:0/1937354489 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc80bfff700 1 --2- 172.21.15.62:0/1937354489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f960550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc80b7fe700 1 -- 172.21.15.62:0/1937354489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 msgr2=0x55606f961c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/1937354489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f961c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc80b7fe700 1 -- 172.21.15.62:0/1937354489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 msgr2=0x55606f960550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/1937354489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f960550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc80b7fe700 1 -- 172.21.15.62:0/1937354489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55606f6dfd80 con 0x55606f960a90 2023-07-14T16:17:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc810b9c700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc7f40a11d0 0x7fc7f40a3680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc80400f540 tx=0x7fc804009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.780+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/1937354489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f9612a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc800009990 tx=0x7fc800005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7772ebea55f2f40e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.781+0000 7fc7f27fc700 1 -- 172.21.15.62:0/1937354489 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80001d070 con 0x55606f960a90 2023-07-14T16:17:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.781+0000 7fc7f27fc700 1 -- 172.21.15.62:0/1937354489 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc80001e040 con 0x55606f960a90 2023-07-14T16:17:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.781+0000 7fc7f27fc700 1 -- 172.21.15.62:0/1937354489 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80000b6b0 con 0x55606f960a90 2023-07-14T16:17:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.781+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1937354489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 msgr2=0x55606f9612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.781+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1937354489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f9612a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc800009990 tx=0x7fc800005c90 comp rx=0 tx=0).stop 2023-07-14T16:17:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.782+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1937354489 shutdown_connections 2023-07-14T16:17:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.782+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1937354489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f961c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.782+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1937354489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f960550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.782+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1937354489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f9612a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.782+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1937354489 >> 172.21.15.62:0/1937354489 conn(0x55606f955d40 msgr2=0x55606f95ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.782+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1937354489 shutdown_connections 2023-07-14T16:17:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.782+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1937354489 wait complete. 2023-07-14T16:17:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.782+0000 7fc825ff6b80 1 -- start start 2023-07-14T16:17:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.783+0000 7fc825ff6b80 1 Processor -- start 2023-07-14T16:17:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.783+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f969970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.783+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f969eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.783+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.783+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f851a60 con 0x55606f960a90 2023-07-14T16:17:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.783+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f851760 con 0x55606f7bc6b0 2023-07-14T16:17:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.783+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f7358c0 con 0x55606f9617e0 2023-07-14T16:17:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.783+0000 7fc825ff6b80 10 client.0 _ll_register_callbacks cb 0x55606f84f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f969eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f969970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f969970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46740/0 (socket says 172.21.15.62:46740) 2023-07-14T16:17:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 -- 172.21.15.62:0/1288539567 learned_addr learned my addr 172.21.15.62:0/1288539567 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc810b9c700 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 -- 172.21.15.62:0/1288539567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 msgr2=0x55606f970ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 -- 172.21.15.62:0/1288539567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 msgr2=0x55606f969eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f969eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 -- 172.21.15.62:0/1288539567 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c1086b0 con 0x55606f7bc6b0 2023-07-14T16:17:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc80bfff700 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f969970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc099b30 tx=0x7fc7fc016330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559dedbd104fce0d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.784+0000 7fc7eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:17:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/1288539567 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc09a290 con 0x55606f7bc6b0 2023-07-14T16:17:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/1288539567 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc7fc099e30 con 0x55606f7bc6b0 2023-07-14T16:17:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1288539567 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55606f80b490 con 0x55606f7bc6b0 2023-07-14T16:17:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/1288539567 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7fc0b8250 con 0x55606f7bc6b0 2023-07-14T16:17:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 10 client.25826 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1288539567 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55606f6c8c00 con 0x55606f7bc6b0 2023-07-14T16:17:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/1288539567 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55606f80b490 con 0x55606f7bc6b0 2023-07-14T16:17:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/1288539567 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc7fc0a1df0 con 0x55606f7bc6b0 2023-07-14T16:17:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 10 client.25826 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 20 client.25826 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55606f975720 0x55606f977b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 4 client.25826 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:17:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.785+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1288539567 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55606f8aae50 con 0x55606f975720 2023-07-14T16:17:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.786+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55606f975720 0x55606f977b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.786+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55606f975720 0x55606f977b30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc65ac49fc91806c server_cookie=a1790d651edb4104 in_seq=0 out_seq=0 2023-07-14T16:17:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.786+0000 7fc7eb7fe700 10 client.25826 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.787+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/1288539567 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55606f8aae50 con 0x55606f975720 2023-07-14T16:17:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.787+0000 7fc7eb7fe700 10 client.25826 handle_command_reply: tid=0 2023-07-14T16:17:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.787+0000 7fc825ff6b80 1 client.25826 shutdown 2023-07-14T16:17:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.787+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1288539567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 msgr2=0x55606f969970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.787+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f969970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc099b30 tx=0x7fc7fc016330 comp rx=0 tx=0).stop 2023-07-14T16:17:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1288539567 shutdown_connections 2023-07-14T16:17:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55606f975720 0x55606f977b30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f969970 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1288539567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f969eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1288539567 >> 172.21.15.62:0/1288539567 conn(0x55606f955d40 msgr2=0x55606f95a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc7eb7fe700 0 client.25826 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1288539567 shutdown_connections 2023-07-14T16:17:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1288539567 wait complete. 2023-07-14T16:17:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.788+0000 7fc825ff6b80 20 client.25826 trim_cache size 0 max 16384 2023-07-14T16:17:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.805+0000 7fc825ff6b80 1 Processor -- start 2023-07-14T16:17:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.805+0000 7fc825ff6b80 1 -- start start 2023-07-14T16:17:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.805+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f9646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f964bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f968e70 con 0x55606f960a90 2023-07-14T16:17:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f957e90 con 0x55606f7bc6b0 2023-07-14T16:17:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f9610d0 con 0x55606f9617e0 2023-07-14T16:17:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f964bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f964bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34328/0 (socket says 172.21.15.62:34328) 2023-07-14T16:17:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80b7fe700 1 -- 172.21.15.62:0/1897593228 learned_addr learned my addr 172.21.15.62:0/1897593228 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80bfff700 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f9646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc810b9c700 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80b7fe700 1 -- 172.21.15.62:0/1897593228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 msgr2=0x55606f970eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80b7fe700 1 -- 172.21.15.62:0/1897593228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 msgr2=0x55606f9646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f9646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80b7fe700 1 -- 172.21.15.62:0/1897593228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55606f7061f0 con 0x55606f960a90 2023-07-14T16:17:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc80bfff700 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f9646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:17:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.806+0000 7fc810b9c700 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:17:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.807+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f964bf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc800026000 tx=0x7fc800005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836533bdc4bd3f99 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.807+0000 7fc7f2ffd700 1 -- 172.21.15.62:0/1897593228 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80001d040 con 0x55606f960a90 2023-07-14T16:17:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.807+0000 7fc7f2ffd700 1 -- 172.21.15.62:0/1897593228 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc800005910 con 0x55606f960a90 2023-07-14T16:17:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.807+0000 7fc7f2ffd700 1 -- 172.21.15.62:0/1897593228 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800005050 con 0x55606f960a90 2023-07-14T16:17:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1897593228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 msgr2=0x55606f964bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f964bf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc800026000 tx=0x7fc800005c20 comp rx=0 tx=0).stop 2023-07-14T16:17:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1897593228 shutdown_connections 2023-07-14T16:17:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f9617e0 0x55606f970eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f7bc6b0 0x55606f9646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/1897593228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f960a90 0x55606f964bf0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1897593228 >> 172.21.15.62:0/1897593228 conn(0x55606f955d40 msgr2=0x55606f97c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1897593228 shutdown_connections 2023-07-14T16:17:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 -- 172.21.15.62:0/1897593228 wait complete. 2023-07-14T16:17:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.808+0000 7fc825ff6b80 1 -- start start 2023-07-14T16:17:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc825ff6b80 1 Processor -- start 2023-07-14T16:17:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f7bc6b0 0x55606f96eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f960a90 0x55606f9c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc825ff6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f9617e0 0x55606f9c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f9c8060 con 0x55606f9617e0 2023-07-14T16:17:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f9c81d0 con 0x55606f960a90 2023-07-14T16:17:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc825ff6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606f9c8390 con 0x55606f7bc6b0 2023-07-14T16:17:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc825ff6b80 10 client.0 _ll_register_callbacks cb 0x55606f84f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:17:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc810b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f9617e0 0x55606f9c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc810b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f9617e0 0x55606f9c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34332/0 (socket says 172.21.15.62:34332) 2023-07-14T16:17:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.809+0000 7fc810b9c700 1 -- 172.21.15.62:0/2515212703 learned_addr learned my addr 172.21.15.62:0/2515212703 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:17:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc80b7fe700 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f960a90 0x55606f9c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc80bfff700 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f7bc6b0 0x55606f96eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc810b9c700 1 -- 172.21.15.62:0/2515212703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f7bc6b0 msgr2=0x55606f96eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc810b9c700 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f7bc6b0 0x55606f96eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc810b9c700 1 -- 172.21.15.62:0/2515212703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f960a90 msgr2=0x55606f9c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:17:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc810b9c700 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f960a90 0x55606f9c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc810b9c700 1 -- 172.21.15.62:0/2515212703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55606f6dfd80 con 0x55606f9617e0 2023-07-14T16:17:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc810b9c700 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f9617e0 0x55606f9c7b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc804019f00 tx=0x7fc804004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6050e6a93f5d8c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:17:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.810+0000 7fc7eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:17:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.811+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/2515212703 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80400e040 con 0x55606f9617e0 2023-07-14T16:17:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.811+0000 7fc825ff6b80 1 -- 172.21.15.62:0/2515212703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55606f86cff0 con 0x55606f9617e0 2023-07-14T16:17:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.811+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/2515212703 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc804017ba0 con 0x55606f9617e0 2023-07-14T16:17:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.811+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/2515212703 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc804017d60 con 0x55606f9617e0 2023-07-14T16:17:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.811+0000 7fc825ff6b80 10 client.16080 fetch_fsmap learned FSMap version 24 2023-07-14T16:17:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.811+0000 7fc825ff6b80 1 -- 172.21.15.62:0/2515212703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55606f6de250 con 0x55606f9617e0 2023-07-14T16:17:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.811+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/2515212703 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc804034020 con 0x55606f9617e0 2023-07-14T16:17:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.811+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/2515212703 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc804004520 con 0x55606f9617e0 2023-07-14T16:17:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.812+0000 7fc825ff6b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:17:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.812+0000 7fc825ff6b80 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:17:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.812+0000 7fc825ff6b80 20 client.16080 populate_metadata read hostname 'smithi062' 2023-07-14T16:17:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.812+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55606f9c9e40 0x55606f9cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:17:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.812+0000 7fc825ff6b80 4 client.16080 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:17:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.812+0000 7fc825ff6b80 1 -- 172.21.15.62:0/2515212703 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55606f700f10 con 0x55606f9c9e40 2023-07-14T16:17:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.812+0000 7fc80bfff700 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55606f9c9e40 0x55606f9cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:17:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.813+0000 7fc80bfff700 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55606f9c9e40 0x55606f9cc250 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290f84f6cf87402b server_cookie=24b0692d60d116f0 in_seq=0 out_seq=0 2023-07-14T16:17:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.813+0000 7fc7eb7fe700 10 client.16080 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc7eb7fe700 1 -- 172.21.15.62:0/2515212703 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55606f700f10 con 0x55606f9c9e40 2023-07-14T16:17:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc7eb7fe700 10 client.16080 handle_command_reply: tid=0 2023-07-14T16:17:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 client.16080 shutdown 2023-07-14T16:17:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 -- 172.21.15.62:0/2515212703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f9617e0 msgr2=0x55606f9c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f9617e0 0x55606f9c7b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc804019f00 tx=0x7fc804004b40 comp rx=0 tx=0).stop 2023-07-14T16:17:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 -- 172.21.15.62:0/2515212703 shutdown_connections 2023-07-14T16:17:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606f9617e0 0x55606f9c7b20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606f7bc6b0 0x55606f96eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55606f9c9e40 0x55606f9cc250 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 --2- 172.21.15.62:0/2515212703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606f960a90 0x55606f9c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 -- 172.21.15.62:0/2515212703 >> 172.21.15.62:0/2515212703 conn(0x55606f955d40 msgr2=0x55606f9646b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc7eb7fe700 0 client.16080 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:17:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 -- 172.21.15.62:0/2515212703 shutdown_connections 2023-07-14T16:17:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 1 -- 172.21.15.62:0/2515212703 wait complete. 2023-07-14T16:17:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.825+0000 7fc825ff6b80 20 client.16080 trim_cache size 0 max 16384 2023-07-14T16:17:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:17:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:17:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:17:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:17:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 -- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc7f40a11d0 msgr2=0x7fc7f40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc7f40a11d0 0x7fc7f40a3680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc80400f540 tx=0x7fc804009920 comp rx=0 tx=0).stop 2023-07-14T16:17:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 -- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c0ff940 msgr2=0x7fc80c197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:17:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c0ff940 0x7fc80c197770 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00e970 tx=0x7fc7fc00ec80 comp rx=0 tx=0).stop 2023-07-14T16:17:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 -- 172.21.15.62:0/2755610975 shutdown_connections 2023-07-14T16:17:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc7f40a11d0 0x7fc7f40a3680 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc80c1014c0 0x7fc80c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc80c0ff940 0x7fc80c197770 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 --2- 172.21.15.62:0/2755610975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc80c100b30 0x7fc80c197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:17:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 -- 172.21.15.62:0/2755610975 >> 172.21.15.62:0/2755610975 conn(0x7fc80c0faf80 msgr2=0x7fc80c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:17:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 -- 172.21.15.62:0/2755610975 shutdown_connections 2023-07-14T16:17:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:17:48.826+0000 7fc81139d700 1 -- 172.21.15.62:0/2755610975 wait complete. 2023-07-14T16:17:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:48 smithi164 ceph-mon[97427]: pgmap v422: 97 pgs: 97 active+clean; 290 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 29 op/s 2023-07-14T16:17:49.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:48 smithi062 ceph-mon[95543]: pgmap v422: 97 pgs: 97 active+clean; 290 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 29 op/s 2023-07-14T16:17:49.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:48 smithi081 ceph-mon[127795]: pgmap v422: 97 pgs: 97 active+clean; 290 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 29 op/s 2023-07-14T16:17:51.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:50 smithi164 ceph-mon[97427]: pgmap v423: 97 pgs: 97 active+clean; 314 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 18 op/s 2023-07-14T16:17:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:50 smithi062 ceph-mon[95543]: pgmap v423: 97 pgs: 97 active+clean; 314 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 18 op/s 2023-07-14T16:17:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:50 smithi081 ceph-mon[127795]: pgmap v423: 97 pgs: 97 active+clean; 314 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 18 op/s 2023-07-14T16:17:53.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:52 smithi164 ceph-mon[97427]: pgmap v424: 97 pgs: 97 active+clean; 358 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:17:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:52 smithi062 ceph-mon[95543]: pgmap v424: 97 pgs: 97 active+clean; 358 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:17:53.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:52 smithi081 ceph-mon[127795]: pgmap v424: 97 pgs: 97 active+clean; 358 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:17:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:54 smithi062 ceph-mon[95543]: pgmap v425: 97 pgs: 97 active+clean; 394 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:17:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:54 smithi081 ceph-mon[127795]: pgmap v425: 97 pgs: 97 active+clean; 394 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:17:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:54 smithi164 ceph-mon[97427]: pgmap v425: 97 pgs: 97 active+clean; 394 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:17:56.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:55 smithi062 ceph-mon[95543]: pgmap v426: 97 pgs: 97 active+clean; 398 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:17:56.156 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:55 smithi081 ceph-mon[127795]: pgmap v426: 97 pgs: 97 active+clean; 398 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:17:56.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:55 smithi164 ceph-mon[97427]: pgmap v426: 97 pgs: 97 active+clean; 398 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:17:58.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:17:58 smithi062 ceph-mon[95543]: pgmap v427: 97 pgs: 97 active+clean; 442 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:17:58.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:17:58 smithi081 ceph-mon[127795]: pgmap v427: 97 pgs: 97 active+clean; 442 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:17:59.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:17:58 smithi164 ceph-mon[97427]: pgmap v427: 97 pgs: 97 active+clean; 442 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:18:00.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:00 smithi062 ceph-mon[95543]: pgmap v428: 97 pgs: 97 active+clean; 466 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:18:00.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:00 smithi081 ceph-mon[127795]: pgmap v428: 97 pgs: 97 active+clean; 466 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:18:01.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:00 smithi164 ceph-mon[97427]: pgmap v428: 97 pgs: 97 active+clean; 466 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:18:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 -- 172.21.15.62:0/4129021959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60102f40 msgr2=0x7f2b60105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 --2- 172.21.15.62:0/4129021959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60102f40 0x7f2b60105320 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009990 tx=0x7f2b50009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 -- 172.21.15.62:0/4129021959 shutdown_connections 2023-07-14T16:18:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 --2- 172.21.15.62:0/4129021959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b60105860 0x7f2b60107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 --2- 172.21.15.62:0/4129021959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b60100620 0x7f2b60102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 --2- 172.21.15.62:0/4129021959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60102f40 0x7f2b60105320 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 -- 172.21.15.62:0/4129021959 >> 172.21.15.62:0/4129021959 conn(0x7f2b600fa000 msgr2=0x7f2b600fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 -- 172.21.15.62:0/4129021959 shutdown_connections 2023-07-14T16:18:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.000+0000 7f2b681ff700 1 -- 172.21.15.62:0/4129021959 wait complete. 2023-07-14T16:18:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.001+0000 7f2b681ff700 1 Processor -- start 2023-07-14T16:18:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b681ff700 1 -- start start 2023-07-14T16:18:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b681ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b60100620 0x7f2b60197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b681ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b60102f40 0x7f2b60197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b681ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60105860 0x7f2b60198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b681ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b601994c0 con 0x7f2b60105860 2023-07-14T16:18:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b681ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60199630 con 0x7f2b60100620 2023-07-14T16:18:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b681ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b601997f0 con 0x7f2b60102f40 2023-07-14T16:18:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b6679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60105860 0x7f2b60198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b6679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60105860 0x7f2b60198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48892/0 (socket says 172.21.15.62:48892) 2023-07-14T16:18:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.002+0000 7f2b6679c700 1 -- 172.21.15.62:0/1365197662 learned_addr learned my addr 172.21.15.62:0/1365197662 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b6579a700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b60102f40 0x7f2b60197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b60100620 0x7f2b60197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b6679c700 1 -- 172.21.15.62:0/1365197662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b60102f40 msgr2=0x7f2b60197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b6679c700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b60102f40 0x7f2b60197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b6679c700 1 -- 172.21.15.62:0/1365197662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b60100620 msgr2=0x7f2b60197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b6679c700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b60100620 0x7f2b60197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b6679c700 1 -- 172.21.15.62:0/1365197662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500095f0 con 0x7f2b60105860 2023-07-14T16:18:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b60100620 0x7f2b60197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b6679c700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60105860 0x7f2b60198e60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2b58007f10 tx=0x7f2b5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b56ffd700 1 -- 172.21.15.62:0/1365197662 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58017070 con 0x7f2b60105860 2023-07-14T16:18:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b56ffd700 1 -- 172.21.15.62:0/1365197662 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b5800f040 con 0x7f2b60105860 2023-07-14T16:18:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b56ffd700 1 -- 172.21.15.62:0/1365197662 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58013e40 con 0x7f2b60105860 2023-07-14T16:18:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.003+0000 7f2b681ff700 1 -- 172.21.15.62:0/1365197662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b60199ad0 con 0x7f2b60105860 2023-07-14T16:18:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.004+0000 7f2b681ff700 1 -- 172.21.15.62:0/1365197662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b601a0290 con 0x7f2b60105860 2023-07-14T16:18:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.007+0000 7f2b56ffd700 1 -- 172.21.15.62:0/1365197662 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2b580133f0 con 0x7f2b60105860 2023-07-14T16:18:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.009+0000 7f2b7bb8fb80 1 Processor -- start 2023-07-14T16:18:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.009+0000 7f2b7bb8fb80 1 -- start start 2023-07-14T16:18:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.009+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 0x55fe87273300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.009+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe87273840 0x55fe872741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.009+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 0x55fe87274b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.009+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe87178c50 con 0x55fe87273840 2023-07-14T16:18:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.009+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe871cb9e0 con 0x55fe872746e0 2023-07-14T16:18:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.009+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe871c4e70 con 0x55fe870cf6b0 2023-07-14T16:18:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b65f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe87273840 0x55fe872741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b56ffd700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2b4c0a11d0 0x7f2b4c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b65f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe87273840 0x55fe872741a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48894/0 (socket says 172.21.15.62:48894) 2023-07-14T16:18:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b65f9b700 1 -- 172.21.15.62:0/1852830984 learned_addr learned my addr 172.21.15.62:0/1852830984 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b6579a700 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 0x55fe87273300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b6579a700 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 0x55fe87274b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b56ffd700 1 -- 172.21.15.62:0/1365197662 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b580cc310 con 0x7f2b60105860 2023-07-14T16:18:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b6679c700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2b4c0a11d0 0x7f2b4c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b65f9b700 1 -- 172.21.15.62:0/1852830984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 msgr2=0x55fe87273300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 0x55fe87273300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b65f9b700 1 -- 172.21.15.62:0/1852830984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 msgr2=0x55fe87274b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 0x55fe87274b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.010+0000 7f2b65f9b700 1 -- 172.21.15.62:0/1852830984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe86ff2d80 con 0x55fe87273840 2023-07-14T16:18:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.011+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe87273840 0x55fe872741a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009d80 tx=0x7f2b5c0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd60ff5a20b8933b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.011+0000 7f2b6679c700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2b4c0a11d0 0x7f2b4c0a3680 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b58096140 tx=0x7f2b58004ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.011+0000 7f2b4bfff700 1 -- 172.21.15.62:0/1852830984 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c011e20 con 0x55fe87273840 2023-07-14T16:18:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.011+0000 7f2b4bfff700 1 -- 172.21.15.62:0/1852830984 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b5c0128e0 con 0x55fe87273840 2023-07-14T16:18:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.011+0000 7f2b4bfff700 1 -- 172.21.15.62:0/1852830984 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00f6a0 con 0x55fe87273840 2023-07-14T16:18:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/1852830984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe87273840 msgr2=0x55fe872741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe87273840 0x55fe872741a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009d80 tx=0x7f2b5c0116c0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/1852830984 shutdown_connections 2023-07-14T16:18:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 0x55fe87274b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 0x55fe87273300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/1852830984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe87273840 0x55fe872741a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/1852830984 >> 172.21.15.62:0/1852830984 conn(0x55fe87268be0 msgr2=0x55fe8726dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/1852830984 shutdown_connections 2023-07-14T16:18:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.012+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/1852830984 wait complete. 2023-07-14T16:18:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.013+0000 7f2b7bb8fb80 1 -- start start 2023-07-14T16:18:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.013+0000 7f2b7bb8fb80 1 Processor -- start 2023-07-14T16:18:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 0x55fe8727c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe87273840 0x55fe8727cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe87283d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe87164a60 con 0x55fe872746e0 2023-07-14T16:18:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe87164760 con 0x55fe87273840 2023-07-14T16:18:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe870488c0 con 0x55fe870cf6b0 2023-07-14T16:18:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b7bb8fb80 10 client.0 _ll_register_callbacks cb 0x55fe87162700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b6579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe87283d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b6579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe87283d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48906/0 (socket says 172.21.15.62:48906) 2023-07-14T16:18:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b6579a700 1 -- 172.21.15.62:0/2848855918 learned_addr learned my addr 172.21.15.62:0/2848855918 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.014+0000 7f2b6579a700 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 0x55fe8727c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe87273840 0x55fe8727cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b6579a700 1 -- 172.21.15.62:0/2848855918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 msgr2=0x55fe8727c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b6579a700 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 0x55fe8727c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b6579a700 1 -- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe87273840 msgr2=0x55fe8727cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b6579a700 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe87273840 0x55fe8727cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b6579a700 1 -- 172.21.15.62:0/2848855918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c0099b0 con 0x55fe872746e0 2023-07-14T16:18:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe87273840 0x55fe8727cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b6579a700 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe87283d60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005910 tx=0x7f2b50009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc1b5464fc4dc66 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b497fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b497fa700 1 -- 172.21.15.62:0/2848855918 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000baf0 con 0x55fe872746e0 2023-07-14T16:18:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2848855918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe8711e490 con 0x55fe872746e0 2023-07-14T16:18:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.015+0000 7f2b497fa700 1 -- 172.21.15.62:0/2848855918 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b5000bc50 con 0x55fe872746e0 2023-07-14T16:18:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b497fa700 1 -- 172.21.15.62:0/2848855918 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b5000be10 con 0x55fe872746e0 2023-07-14T16:18:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b7bb8fb80 10 client.16092 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2848855918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fe86fdbc00 con 0x55fe872746e0 2023-07-14T16:18:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b497fa700 1 -- 172.21.15.62:0/2848855918 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe8711e490 con 0x55fe872746e0 2023-07-14T16:18:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b497fa700 1 -- 172.21.15.62:0/2848855918 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2b50028400 con 0x55fe872746e0 2023-07-14T16:18:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b7bb8fb80 10 client.16092 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b7bb8fb80 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b7bb8fb80 20 client.16092 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe87288590 0x55fe8728a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b7bb8fb80 4 client.16092 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.016+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2848855918 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe871bde50 con 0x55fe87288590 2023-07-14T16:18:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.017+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe87288590 0x55fe8728a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.017+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe87288590 0x55fe8728a9a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7578d148c83b2cd server_cookie=6bcf049f193669ca in_seq=0 out_seq=0 2023-07-14T16:18:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.017+0000 7f2b497fa700 10 client.16092 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b497fa700 1 -- 172.21.15.62:0/2848855918 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fe871bde50 con 0x55fe87288590 2023-07-14T16:18:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b497fa700 10 client.16092 handle_command_reply: tid=0 2023-07-14T16:18:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 client.16092 shutdown 2023-07-14T16:18:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2848855918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 msgr2=0x55fe87283d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe87283d60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005910 tx=0x7f2b50009310 comp rx=0 tx=0).stop 2023-07-14T16:18:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2848855918 shutdown_connections 2023-07-14T16:18:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe87283d60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe870cf6b0 0x55fe8727c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe87288590 0x55fe8728a9a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2848855918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe87273840 0x55fe8727cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2848855918 >> 172.21.15.62:0/2848855918 conn(0x55fe87268be0 msgr2=0x55fe8726d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.018+0000 7f2b497fa700 0 client.16092 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.019+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2848855918 shutdown_connections 2023-07-14T16:18:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.019+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2848855918 wait complete. 2023-07-14T16:18:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.019+0000 7f2b7bb8fb80 20 client.16092 trim_cache size 0 max 16384 2023-07-14T16:18:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.043+0000 7f2b7bb8fb80 1 Processor -- start 2023-07-14T16:18:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.043+0000 7f2b7bb8fb80 1 -- start start 2023-07-14T16:18:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.043+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe870cf6b0 0x55fe87277550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.043+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87283d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.043+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 0x55fe87277a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.043+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8711e490 con 0x55fe870cf6b0 2023-07-14T16:18:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.043+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8727d9d0 con 0x55fe872746e0 2023-07-14T16:18:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.043+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8726ea50 con 0x55fe87273840 2023-07-14T16:18:02.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe870cf6b0 0x55fe87277550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe870cf6b0 0x55fe87277550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48922/0 (socket says 172.21.15.62:48922) 2023-07-14T16:18:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 -- 172.21.15.62:0/2687713099 learned_addr learned my addr 172.21.15.62:0/2687713099 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87283d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 0x55fe87277a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 -- 172.21.15.62:0/2687713099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 msgr2=0x55fe87283d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87283d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 -- 172.21.15.62:0/2687713099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 msgr2=0x55fe87277a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 0x55fe87277a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.044+0000 7f2b6579a700 1 -- 172.21.15.62:0/2687713099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe870191f0 con 0x55fe870cf6b0 2023-07-14T16:18:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.045+0000 7f2b6579a700 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe870cf6b0 0x55fe87277550 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2b50028980 tx=0x7f2b50005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5244ae34ef8fc8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.045+0000 7f2b54ff9700 1 -- 172.21.15.62:0/2687713099 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50028a30 con 0x55fe870cf6b0 2023-07-14T16:18:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.045+0000 7f2b54ff9700 1 -- 172.21.15.62:0/2687713099 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b50028400 con 0x55fe870cf6b0 2023-07-14T16:18:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.045+0000 7f2b54ff9700 1 -- 172.21.15.62:0/2687713099 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000e3e0 con 0x55fe870cf6b0 2023-07-14T16:18:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2687713099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe870cf6b0 msgr2=0x55fe87277550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe870cf6b0 0x55fe87277550 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2b50028980 tx=0x7f2b50005fb0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2687713099 shutdown_connections 2023-07-14T16:18:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe872746e0 0x55fe87277a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe870cf6b0 0x55fe87277550 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/2687713099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87283d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2687713099 >> 172.21.15.62:0/2687713099 conn(0x55fe87268be0 msgr2=0x55fe8728f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2687713099 shutdown_connections 2023-07-14T16:18:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/2687713099 wait complete. 2023-07-14T16:18:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.046+0000 7f2b7bb8fb80 1 -- start start 2023-07-14T16:18:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.047+0000 7f2b7bb8fb80 1 Processor -- start 2023-07-14T16:18:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.047+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe870cf6b0 0x55fe8727fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.047+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87280190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.047+0000 7f2b7bb8fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe872831b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.047+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe872836f0 con 0x55fe872746e0 2023-07-14T16:18:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.047+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe87283860 con 0x55fe870cf6b0 2023-07-14T16:18:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.047+0000 7f2b7bb8fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe87283a20 con 0x55fe87273840 2023-07-14T16:18:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.047+0000 7f2b7bb8fb80 10 client.0 _ll_register_callbacks cb 0x55fe87162700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b6579a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe870cf6b0 0x55fe8727fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b65f9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87280190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b65f9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87280190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50254/0 (socket says 172.21.15.62:50254) 2023-07-14T16:18:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b65f9b700 1 -- 172.21.15.62:0/3396574001 learned_addr learned my addr 172.21.15.62:0/3396574001 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b6579a700 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe872831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b65f9b700 1 -- 172.21.15.62:0/3396574001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe870cf6b0 msgr2=0x55fe8727fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe870cf6b0 0x55fe8727fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b65f9b700 1 -- 172.21.15.62:0/3396574001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 msgr2=0x55fe872831b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe872831b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.048+0000 7f2b65f9b700 1 -- 172.21.15.62:0/3396574001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe86fdbc00 con 0x55fe87273840 2023-07-14T16:18:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.049+0000 7f2b65f9b700 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87280190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0058f0 tx=0x7f2b5c009870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec0972e5108a2972 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.049+0000 7f2b497fa700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:18:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.049+0000 7f2b497fa700 1 -- 172.21.15.62:0/3396574001 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c029b50 con 0x55fe87273840 2023-07-14T16:18:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.049+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/3396574001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b5c0128e0 con 0x55fe87273840 2023-07-14T16:18:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.049+0000 7f2b497fa700 1 -- 172.21.15.62:0/3396574001 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b5c029cb0 con 0x55fe87273840 2023-07-14T16:18:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.050+0000 7f2b497fa700 1 -- 172.21.15.62:0/3396574001 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b5c037030 con 0x55fe87273840 2023-07-14T16:18:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.050+0000 7f2b7bb8fb80 10 client.25762 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.050+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/3396574001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fe86ff1250 con 0x55fe87273840 2023-07-14T16:18:02.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.050+0000 7f2b497fa700 1 -- 172.21.15.62:0/3396574001 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b5c0128e0 con 0x55fe87273840 2023-07-14T16:18:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.050+0000 7f2b497fa700 1 -- 172.21.15.62:0/3396574001 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2b5c0113e0 con 0x55fe87273840 2023-07-14T16:18:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.050+0000 7f2b7bb8fb80 10 client.25762 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.050+0000 7f2b7bb8fb80 10 client.25762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.050+0000 7f2b7bb8fb80 20 client.25762 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.051+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe872dd4c0 0x55fe872df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.051+0000 7f2b7bb8fb80 4 client.25762 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:18:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.051+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/3396574001 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe8717fff0 con 0x55fe872dd4c0 2023-07-14T16:18:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.051+0000 7f2b6579a700 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe872dd4c0 0x55fe872df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.052+0000 7f2b497fa700 10 client.25762 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.052+0000 7f2b6579a700 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe872dd4c0 0x55fe872df8d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ef1fe06872d870 server_cookie=ea7fe5e5bba0a783 in_seq=0 out_seq=0 2023-07-14T16:18:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.063+0000 7f2b497fa700 1 -- 172.21.15.62:0/3396574001 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe8717fff0 con 0x55fe872dd4c0 2023-07-14T16:18:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.063+0000 7f2b497fa700 10 client.25762 handle_command_reply: tid=0 2023-07-14T16:18:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.063+0000 7f2b7bb8fb80 1 client.25762 shutdown 2023-07-14T16:18:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/3396574001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 msgr2=0x55fe87280190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87280190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0058f0 tx=0x7f2b5c009870 comp rx=0 tx=0).stop 2023-07-14T16:18:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/3396574001 shutdown_connections 2023-07-14T16:18:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe872746e0 0x55fe872831b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe872dd4c0 0x55fe872df8d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe870cf6b0 0x55fe8727fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 --2- 172.21.15.62:0/3396574001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe87273840 0x55fe87280190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b497fa700 0 client.25762 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/3396574001 >> 172.21.15.62:0/3396574001 conn(0x55fe87268be0 msgr2=0x55fe8727aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/3396574001 shutdown_connections 2023-07-14T16:18:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 1 -- 172.21.15.62:0/3396574001 wait complete. 2023-07-14T16:18:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.064+0000 7f2b7bb8fb80 20 client.25762 trim_cache size 0 max 16384 2023-07-14T16:18:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:18:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:18:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:18:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:18:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 -- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2b4c0a11d0 msgr2=0x7f2b4c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2b4c0a11d0 0x7f2b4c0a3680 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b58096140 tx=0x7f2b58004ce0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 -- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60105860 msgr2=0x7f2b60198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60105860 0x7f2b60198e60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2b58007f10 tx=0x7f2b5800d3b0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 -- 172.21.15.62:0/1365197662 shutdown_connections 2023-07-14T16:18:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b60105860 0x7f2b60198e60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2b4c0a11d0 0x7f2b4c0a3680 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b60100620 0x7f2b60197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 --2- 172.21.15.62:0/1365197662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b60102f40 0x7f2b60197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 -- 172.21.15.62:0/1365197662 >> 172.21.15.62:0/1365197662 conn(0x7f2b600fa000 msgr2=0x7f2b600feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 -- 172.21.15.62:0/1365197662 shutdown_connections 2023-07-14T16:18:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:02.065+0000 7f2b681ff700 1 -- 172.21.15.62:0/1365197662 wait complete. 2023-07-14T16:18:02.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:02 smithi062 ceph-mon[95543]: pgmap v429: 97 pgs: 97 active+clean; 494 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:18:02.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:02 smithi081 ceph-mon[127795]: pgmap v429: 97 pgs: 97 active+clean; 494 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:18:03.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:02 smithi164 ceph-mon[97427]: pgmap v429: 97 pgs: 97 active+clean; 494 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:18:03.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:18:03.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 -- 172.21.15.62:0/2303816585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 msgr2=0x7f40b8101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 --2- 172.21.15.62:0/2303816585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 0x7f40b8101120 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009990 tx=0x7f40a4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 -- 172.21.15.62:0/2303816585 shutdown_connections 2023-07-14T16:18:03.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 --2- 172.21.15.62:0/2303816585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b8101660 0x7f40b8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 --2- 172.21.15.62:0/2303816585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b80ffae0 0x7f40b80ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 --2- 172.21.15.62:0/2303816585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 0x7f40b8101120 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 -- 172.21.15.62:0/2303816585 >> 172.21.15.62:0/2303816585 conn(0x7f40b80fb0e0 msgr2=0x7f40b80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:03.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 -- 172.21.15.62:0/2303816585 shutdown_connections 2023-07-14T16:18:03.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.324+0000 7f40bedff700 1 -- 172.21.15.62:0/2303816585 wait complete. 2023-07-14T16:18:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.325+0000 7f40bedff700 1 Processor -- start 2023-07-14T16:18:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.325+0000 7f40bedff700 1 -- start start 2023-07-14T16:18:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40bedff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b80ffae0 0x7f40b81957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40bedff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 0x7f40b8195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40bedff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b8101660 0x7f40b819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40bedff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b819d310 con 0x7f40b8100cd0 2023-07-14T16:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40bedff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b819d450 con 0x7f40b80ffae0 2023-07-14T16:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40bedff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b819d5e0 con 0x7f40b8101660 2023-07-14T16:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 0x7f40b8195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 0x7f40b8195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52390/0 (socket says 172.21.15.62:52390) 2023-07-14T16:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40b7fff700 1 -- 172.21.15.62:0/877701580 learned_addr learned my addr 172.21.15.62:0/877701580 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40bd39c700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b8101660 0x7f40b819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b80ffae0 0x7f40b81957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40b7fff700 1 -- 172.21.15.62:0/877701580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b8101660 msgr2=0x7f40b819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40b7fff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b8101660 0x7f40b819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40b7fff700 1 -- 172.21.15.62:0/877701580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b80ffae0 msgr2=0x7f40b81957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40b7fff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b80ffae0 0x7f40b81957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.326+0000 7f40b7fff700 1 -- 172.21.15.62:0/877701580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40095f0 con 0x7f40b8100cd0 2023-07-14T16:18:03.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.327+0000 7f40b7fff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 0x7f40b8195d20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f40a4005910 tx=0x7f40a4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.327+0000 7f40b5ffb700 1 -- 172.21.15.62:0/877701580 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a401d070 con 0x7f40b8100cd0 2023-07-14T16:18:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.327+0000 7f40b5ffb700 1 -- 172.21.15.62:0/877701580 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a400be90 con 0x7f40b8100cd0 2023-07-14T16:18:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.327+0000 7f40b5ffb700 1 -- 172.21.15.62:0/877701580 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4022dd0 con 0x7f40b8100cd0 2023-07-14T16:18:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.327+0000 7f40bedff700 1 -- 172.21.15.62:0/877701580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b8072740 con 0x7f40b8100cd0 2023-07-14T16:18:03.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.327+0000 7f40bedff700 1 -- 172.21.15.62:0/877701580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b8072ba0 con 0x7f40b8100cd0 2023-07-14T16:18:03.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.331+0000 7f40b5ffb700 1 -- 172.21.15.62:0/877701580 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f40a400ba00 con 0x7f40b8100cd0 2023-07-14T16:18:03.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.332+0000 7f40d276eb80 1 Processor -- start 2023-07-14T16:18:03.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40d276eb80 1 -- start start 2023-07-14T16:18:03.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8bd58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40b5ffb700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a80a11d0 0x7f40a80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bd66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd6c70 0x559bf8bd70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40d276eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8adacb0 con 0x559bf8bd54d0 2023-07-14T16:18:03.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40d276eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8b2da40 con 0x559bf8bd6c70 2023-07-14T16:18:03.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40d276eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8b26ed0 con 0x559bf8bd5e20 2023-07-14T16:18:03.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40b5ffb700 1 -- 172.21.15.62:0/877701580 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40a40cde90 con 0x7f40b8100cd0 2023-07-14T16:18:03.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.333+0000 7f40bd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8bd58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8bd58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52402/0 (socket says 172.21.15.62:52402) 2023-07-14T16:18:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 -- 172.21.15.62:0/555801886 learned_addr learned my addr 172.21.15.62:0/555801886 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a80a11d0 0x7f40a80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd6c70 0x559bf8bd70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40b7fff700 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bd66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 -- 172.21.15.62:0/555801886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 msgr2=0x559bf8bd66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:03.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bd66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 -- 172.21.15.62:0/555801886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd6c70 msgr2=0x559bf8bd70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:03.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd6c70 0x559bf8bd70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 -- 172.21.15.62:0/555801886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bf8954de0 con 0x559bf8bd54d0 2023-07-14T16:18:03.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40b7fff700 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bd66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:18:03.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.334+0000 7f40bd39c700 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8bd58e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f40b0011aa0 tx=0x7f40b0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30332d309eeb9642 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:03.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40a2ffd700 1 -- 172.21.15.62:0/555801886 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000e040 con 0x559bf8bd54d0 2023-07-14T16:18:03.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a80a11d0 0x7f40a80a3680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f40b8100b30 tx=0x7f40ac007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:03.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40a2ffd700 1 -- 172.21.15.62:0/555801886 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40b000f040 con 0x559bf8bd54d0 2023-07-14T16:18:03.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40a2ffd700 1 -- 172.21.15.62:0/555801886 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0007760 con 0x559bf8bd54d0 2023-07-14T16:18:03.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40d276eb80 1 -- 172.21.15.62:0/555801886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 msgr2=0x559bf8bd58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40d276eb80 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8bd58e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f40b0011aa0 tx=0x7f40b0009920 comp rx=0 tx=0).stop 2023-07-14T16:18:03.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40d276eb80 1 -- 172.21.15.62:0/555801886 shutdown_connections 2023-07-14T16:18:03.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40d276eb80 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd6c70 0x559bf8bd70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40d276eb80 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8bd58e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.335+0000 7f40d276eb80 1 --2- 172.21.15.62:0/555801886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bd66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.336+0000 7f40d276eb80 1 -- 172.21.15.62:0/555801886 >> 172.21.15.62:0/555801886 conn(0x559bf8bcada0 msgr2=0x559bf8bd01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:03.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.336+0000 7f40d276eb80 1 -- 172.21.15.62:0/555801886 shutdown_connections 2023-07-14T16:18:03.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.336+0000 7f40d276eb80 1 -- 172.21.15.62:0/555801886 wait complete. 2023-07-14T16:18:03.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.336+0000 7f40d276eb80 1 -- start start 2023-07-14T16:18:03.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.336+0000 7f40d276eb80 1 Processor -- start 2023-07-14T16:18:03.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd54d0 0x559bf8bdf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd6c70 0x559bf8be6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40d276eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8ac6ac0 con 0x559bf8bd6c70 2023-07-14T16:18:03.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40d276eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8ac67c0 con 0x559bf8bd54d0 2023-07-14T16:18:03.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40d276eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf89aa920 con 0x559bf8bd5e20 2023-07-14T16:18:03.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40d276eb80 10 client.0 _ll_register_callbacks cb 0x559bf8ac4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:03.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40bd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd6c70 0x559bf8be6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40bd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd6c70 0x559bf8be6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52410/0 (socket says 172.21.15.62:52410) 2023-07-14T16:18:03.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40bd39c700 1 -- 172.21.15.62:0/1222267413 learned_addr learned my addr 172.21.15.62:0/1222267413 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.337+0000 7f40b7fff700 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd54d0 0x559bf8bdf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f40bd39c700 1 -- 172.21.15.62:0/1222267413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 msgr2=0x559bf8bdf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f40bd39c700 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f40bd39c700 1 -- 172.21.15.62:0/1222267413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd54d0 msgr2=0x559bf8bdf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f40bd39c700 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd54d0 0x559bf8bdf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f40bd39c700 1 -- 172.21.15.62:0/1222267413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b00116d0 con 0x559bf8bd6c70 2023-07-14T16:18:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f40bd39c700 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd6c70 0x559bf8be6740 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f40b00112c0 tx=0x7f40b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5db6481b5fe663 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f4097fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f4097fff700 1 -- 172.21.15.62:0/1222267413 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000e040 con 0x559bf8bd6c70 2023-07-14T16:18:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f40d276eb80 1 -- 172.21.15.62:0/1222267413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bf8a804f0 con 0x559bf8bd6c70 2023-07-14T16:18:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.338+0000 7f4097fff700 1 -- 172.21.15.62:0/1222267413 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40b0013a70 con 0x559bf8bd6c70 2023-07-14T16:18:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f4097fff700 1 -- 172.21.15.62:0/1222267413 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b0007500 con 0x559bf8bd6c70 2023-07-14T16:18:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f40d276eb80 10 client.16110 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f40d276eb80 1 -- 172.21.15.62:0/1222267413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559bf893dc60 con 0x559bf8bd6c70 2023-07-14T16:18:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f4097fff700 1 -- 172.21.15.62:0/1222267413 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40b0033020 con 0x559bf8bd6c70 2023-07-14T16:18:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f4097fff700 1 -- 172.21.15.62:0/1222267413 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f40b002f760 con 0x559bf8bd6c70 2023-07-14T16:18:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f40d276eb80 10 client.16110 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f40d276eb80 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f40d276eb80 20 client.16110 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559bf8beae50 0x559bf8bed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f40d276eb80 4 client.16110 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.339+0000 7f40d276eb80 1 -- 172.21.15.62:0/1222267413 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bf8b1feb0 con 0x559bf8beae50 2023-07-14T16:18:03.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.340+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559bf8beae50 0x559bf8bed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.341+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559bf8beae50 0x559bf8bed260 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3c0fd203781bb3 server_cookie=ed37c59b9c4176e9 in_seq=0 out_seq=0 2023-07-14T16:18:03.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.341+0000 7f4097fff700 10 client.16110 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:03.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.341+0000 7f4097fff700 1 -- 172.21.15.62:0/1222267413 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559bf8b1feb0 con 0x559bf8beae50 2023-07-14T16:18:03.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.341+0000 7f4097fff700 10 client.16110 handle_command_reply: tid=0 2023-07-14T16:18:03.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.341+0000 7f40d276eb80 1 client.16110 shutdown 2023-07-14T16:18:03.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 -- 172.21.15.62:0/1222267413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd6c70 msgr2=0x559bf8be6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd6c70 0x559bf8be6740 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f40b00112c0 tx=0x7f40b0004ad0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 -- 172.21.15.62:0/1222267413 shutdown_connections 2023-07-14T16:18:03.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd6c70 0x559bf8be6740 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559bf8beae50 0x559bf8bed260 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd54d0 0x559bf8bdf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1222267413 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 -- 172.21.15.62:0/1222267413 >> 172.21.15.62:0/1222267413 conn(0x559bf8bcada0 msgr2=0x559bf8bcf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:03.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f4097fff700 0 client.16110 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:03.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 -- 172.21.15.62:0/1222267413 shutdown_connections 2023-07-14T16:18:03.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 1 -- 172.21.15.62:0/1222267413 wait complete. 2023-07-14T16:18:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.342+0000 7f40d276eb80 20 client.16110 trim_cache size 0 max 16384 2023-07-14T16:18:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.360+0000 7f40d276eb80 1 Processor -- start 2023-07-14T16:18:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.360+0000 7f40d276eb80 1 -- start start 2023-07-14T16:18:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.360+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bcada0 0x559bf8bcf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.360+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40d276eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8bded30 con 0x559bf8bd54d0 2023-07-14T16:18:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40d276eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8bd61f0 con 0x559bf8bcada0 2023-07-14T16:18:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40d276eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8bed8c0 con 0x559bf8bd5e20 2023-07-14T16:18:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52412/0 (socket says 172.21.15.62:52412) 2023-07-14T16:18:03.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40b7fff700 1 -- 172.21.15.62:0/1401768715 learned_addr learned my addr 172.21.15.62:0/1401768715 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:03.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bcada0 0x559bf8bcf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40bd39c700 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40b7fff700 1 -- 172.21.15.62:0/1401768715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 msgr2=0x559bf8bdd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:03.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40b7fff700 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40b7fff700 1 -- 172.21.15.62:0/1401768715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bcada0 msgr2=0x559bf8bcf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:03.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40b7fff700 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bcada0 0x559bf8bcf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40b7fff700 1 -- 172.21.15.62:0/1401768715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bf897b250 con 0x559bf8bd54d0 2023-07-14T16:18:03.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.361+0000 7f40bd39c700 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:18:03.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.362+0000 7f40b7fff700 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be6700 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f40a40977e0 tx=0x7f40a40a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d319fcde51ec5cd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:03.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.362+0000 7f40a37fe700 1 -- 172.21.15.62:0/1401768715 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40b2070 con 0x559bf8bd54d0 2023-07-14T16:18:03.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.362+0000 7f40a37fe700 1 -- 172.21.15.62:0/1401768715 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a409ac40 con 0x559bf8bd54d0 2023-07-14T16:18:03.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.362+0000 7f40a37fe700 1 -- 172.21.15.62:0/1401768715 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40a36a0 con 0x559bf8bd54d0 2023-07-14T16:18:03.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.362+0000 7f40d276eb80 1 -- 172.21.15.62:0/1401768715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 msgr2=0x559bf8be6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.362+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be6700 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f40a40977e0 tx=0x7f40a40a1040 comp rx=0 tx=0).stop 2023-07-14T16:18:03.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 -- 172.21.15.62:0/1401768715 shutdown_connections 2023-07-14T16:18:03.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bd5e20 0x559bf8bdd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bcada0 0x559bf8bcf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 --2- 172.21.15.62:0/1401768715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be6700 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 -- 172.21.15.62:0/1401768715 >> 172.21.15.62:0/1401768715 conn(0x559bf8bcc690 msgr2=0x559bf8bdde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:03.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 -- 172.21.15.62:0/1401768715 shutdown_connections 2023-07-14T16:18:03.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 -- 172.21.15.62:0/1401768715 wait complete. 2023-07-14T16:18:03.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 -- start start 2023-07-14T16:18:03.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.363+0000 7f40d276eb80 1 Processor -- start 2023-07-14T16:18:03.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bcada0 0x559bf8be5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40d276eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd5e20 0x559bf8be5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40d276eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8c3ee40 con 0x559bf8bd54d0 2023-07-14T16:18:03.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40d276eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8c3efb0 con 0x559bf8bd5e20 2023-07-14T16:18:03.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40d276eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf8c3f170 con 0x559bf8bcada0 2023-07-14T16:18:03.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40d276eb80 10 client.0 _ll_register_callbacks cb 0x7f40b0011980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:03.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52418/0 (socket says 172.21.15.62:52418) 2023-07-14T16:18:03.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40b7fff700 1 -- 172.21.15.62:0/2162217060 learned_addr learned my addr 172.21.15.62:0/2162217060 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:03.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.364+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bcada0 0x559bf8be5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f40b7fff700 1 -- 172.21.15.62:0/2162217060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bcada0 msgr2=0x559bf8be5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:03.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f40b7fff700 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bcada0 0x559bf8be5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f40b7fff700 1 -- 172.21.15.62:0/2162217060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd5e20 msgr2=0x559bf8be5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:03.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f40bd39c700 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd5e20 0x559bf8be5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f40b7fff700 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd5e20 0x559bf8be5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f40b7fff700 1 -- 172.21.15.62:0/2162217060 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b8072ba0 con 0x559bf8bd54d0 2023-07-14T16:18:03.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f40b7fff700 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be56b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f40a409adb0 tx=0x7f40a4097750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b354a158251be75 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:03.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f4097fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:03.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f4097fff700 1 -- 172.21.15.62:0/2162217060 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40b2070 con 0x559bf8bd54d0 2023-07-14T16:18:03.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f4097fff700 1 -- 172.21.15.62:0/2162217060 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40a40a36a0 con 0x559bf8bd54d0 2023-07-14T16:18:03.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f40d276eb80 1 -- 172.21.15.62:0/2162217060 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40b000f040 con 0x559bf8bd54d0 2023-07-14T16:18:03.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.365+0000 7f4097fff700 1 -- 172.21.15.62:0/2162217060 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a409b2e0 con 0x559bf8bd54d0 2023-07-14T16:18:03.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f40d276eb80 10 client.16116 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:03.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f40d276eb80 1 -- 172.21.15.62:0/2162217060 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559bf89532b0 con 0x559bf8bd54d0 2023-07-14T16:18:03.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f4097fff700 1 -- 172.21.15.62:0/2162217060 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40b000f040 con 0x559bf8bd54d0 2023-07-14T16:18:03.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f4097fff700 1 -- 172.21.15.62:0/2162217060 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f40a40970d0 con 0x559bf8bd54d0 2023-07-14T16:18:03.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f40d276eb80 10 client.16116 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:03.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f40d276eb80 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:03.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f40d276eb80 20 client.16116 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:03.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f40d276eb80 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559bf8bd6c70 0x559bf8c42780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f40d276eb80 4 client.16116 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:18:03.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.366+0000 7f40d276eb80 1 -- 172.21.15.62:0/2162217060 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559bf8ae2050 con 0x559bf8bd6c70 2023-07-14T16:18:03.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.367+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559bf8bd6c70 0x559bf8c42780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.367+0000 7f40bcb9b700 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559bf8bd6c70 0x559bf8c42780 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295683b8b86a2a88 server_cookie=6663ebc1ba87dbb4 in_seq=0 out_seq=0 2023-07-14T16:18:03.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.367+0000 7f4097fff700 10 client.16116 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:03.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f4097fff700 1 -- 172.21.15.62:0/2162217060 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559bf8ae2050 con 0x559bf8bd6c70 2023-07-14T16:18:03.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f4097fff700 10 client.16116 handle_command_reply: tid=0 2023-07-14T16:18:03.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 client.16116 shutdown 2023-07-14T16:18:03.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 -- 172.21.15.62:0/2162217060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 msgr2=0x559bf8be56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be56b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f40a409adb0 tx=0x7f40a4097750 comp rx=0 tx=0).stop 2023-07-14T16:18:03.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 -- 172.21.15.62:0/2162217060 shutdown_connections 2023-07-14T16:18:03.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559bf8bd6c70 0x559bf8c42780 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559bf8bd5e20 0x559bf8be5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559bf8bcada0 0x559bf8be5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 --2- 172.21.15.62:0/2162217060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559bf8bd54d0 0x559bf8be56b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f4097fff700 0 client.16116 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:03.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 -- 172.21.15.62:0/2162217060 >> 172.21.15.62:0/2162217060 conn(0x559bf8bcc690 msgr2=0x559bf8bcf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:03.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 -- 172.21.15.62:0/2162217060 shutdown_connections 2023-07-14T16:18:03.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 1 -- 172.21.15.62:0/2162217060 wait complete. 2023-07-14T16:18:03.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.368+0000 7f40d276eb80 20 client.16116 trim_cache size 0 max 16384 2023-07-14T16:18:03.354 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:18:03.354 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:18:03.354 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:18:03.354 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:18:03.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 -- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a80a11d0 msgr2=0x7f40a80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a80a11d0 0x7f40a80a3680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f40b8100b30 tx=0x7f40ac007400 comp rx=0 tx=0).stop 2023-07-14T16:18:03.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 -- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 msgr2=0x7f40b8195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 0x7f40b8195d20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f40a4005910 tx=0x7f40a4005720 comp rx=0 tx=0).stop 2023-07-14T16:18:03.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 -- 172.21.15.62:0/877701580 shutdown_connections 2023-07-14T16:18:03.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40b8101660 0x7f40b819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40a80a11d0 0x7f40a80a3680 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40b80ffae0 0x7f40b81957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 --2- 172.21.15.62:0/877701580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b8100cd0 0x7f40b8195d20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 -- 172.21.15.62:0/877701580 >> 172.21.15.62:0/877701580 conn(0x7f40b80fb0e0 msgr2=0x7f40b81042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:03.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 -- 172.21.15.62:0/877701580 shutdown_connections 2023-07-14T16:18:03.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.369+0000 7f40bedff700 1 -- 172.21.15.62:0/877701580 wait complete. 2023-07-14T16:18:03.365 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:18:03.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:18:03.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.608+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3477105450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa00ffb00 msgr2=0x7f7fa00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.608+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3477105450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa00ffb00 0x7f7fa00fff10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009990 tx=0x7f7f90009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.608+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3477105450 shutdown_connections 2023-07-14T16:18:03.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.608+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3477105450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa0101680 0x7f7fa0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.608+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3477105450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa00ffb00 0x7f7fa00fff10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.608+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3477105450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa0100cf0 0x7f7fa0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.608+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3477105450 >> 172.21.15.62:0/3477105450 conn(0x7f7fa00fb0e0 msgr2=0x7f7fa00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:03.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.609+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3477105450 shutdown_connections 2023-07-14T16:18:03.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.609+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3477105450 wait complete. 2023-07-14T16:18:03.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.610+0000 7f7fa5dff700 1 Processor -- start 2023-07-14T16:18:03.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.610+0000 7f7fa5dff700 1 -- start start 2023-07-14T16:18:03.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7fa5dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa00ffb00 0x7f7fa0197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7fa5dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0100cf0 0x7f7fa0197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7fa5dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa0101680 0x7f7fa019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7fa5dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa019f4a0 con 0x7f7fa0100cf0 2023-07-14T16:18:03.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7fa5dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa019f5e0 con 0x7f7fa00ffb00 2023-07-14T16:18:03.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7fa5dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa019f770 con 0x7f7fa0101680 2023-07-14T16:18:03.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7f9effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0100cf0 0x7f7fa0197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7f9effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0100cf0 0x7f7fa0197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52426/0 (socket says 172.21.15.62:52426) 2023-07-14T16:18:03.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7f9effd700 1 -- 172.21.15.62:0/3358919435 learned_addr learned my addr 172.21.15.62:0/3358919435 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:03.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7f9f7fe700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa00ffb00 0x7f7fa0197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.611+0000 7f7f9ffff700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa0101680 0x7f7fa019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9effd700 1 -- 172.21.15.62:0/3358919435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa0101680 msgr2=0x7f7fa019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9effd700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa0101680 0x7f7fa019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9effd700 1 -- 172.21.15.62:0/3358919435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa00ffb00 msgr2=0x7f7fa0197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9effd700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa00ffb00 0x7f7fa0197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9effd700 1 -- 172.21.15.62:0/3358919435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900095f0 con 0x7f7fa0100cf0 2023-07-14T16:18:03.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9effd700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0100cf0 0x7f7fa0197e60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c9b0 tx=0x7f7f9400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9cff9700 1 -- 172.21.15.62:0/3358919435 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94007b00 con 0x7f7fa0100cf0 2023-07-14T16:18:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9cff9700 1 -- 172.21.15.62:0/3358919435 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f9400ce50 con 0x7f7fa0100cf0 2023-07-14T16:18:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7f9cff9700 1 -- 172.21.15.62:0/3358919435 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94018420 con 0x7f7fa0100cf0 2023-07-14T16:18:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.612+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa019fa50 con 0x7f7fa0100cf0 2023-07-14T16:18:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.613+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa019ff10 con 0x7f7fa0100cf0 2023-07-14T16:18:03.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.614+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa0192090 con 0x7f7fa0100cf0 2023-07-14T16:18:03.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.616+0000 7f7f9cff9700 1 -- 172.21.15.62:0/3358919435 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7f9400f450 con 0x7f7fa0100cf0 2023-07-14T16:18:03.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.618+0000 7f7f9cff9700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f880a1220 0x7f7f880a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:03.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.619+0000 7f7f9cff9700 1 -- 172.21.15.62:0/3358919435 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7f940ccaf0 con 0x7f7fa0100cf0 2023-07-14T16:18:03.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.619+0000 7f7f9f7fe700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f880a1220 0x7f7f880a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:03.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.619+0000 7f7f9f7fe700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f880a1220 0x7f7f880a36d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c010 tx=0x7f7f90005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:03.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.621+0000 7f7f9cff9700 1 -- 172.21.15.62:0/3358919435 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7f94096300 con 0x7f7fa0100cf0 2023-07-14T16:18:03.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.928+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fa002c850 con 0x7f7fa0100cf0 2023-07-14T16:18:03.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.928+0000 7f7f9cff9700 1 -- 172.21.15.62:0/3358919435 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10826 (secure 0 0 0) 0x7f7f9400f700 con 0x7f7fa0100cf0 2023-07-14T16:18:03.907 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:18:03.907 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:17:18.727946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:18:03.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f880a1220 msgr2=0x7f7f880a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f880a1220 0x7f7f880a36d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c010 tx=0x7f7f90005fd0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0100cf0 msgr2=0x7f7fa0197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:03.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0100cf0 0x7f7fa0197e60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c9b0 tx=0x7f7f9400ccc0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 shutdown_connections 2023-07-14T16:18:03.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fa0101680 0x7f7fa019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7f880a1220 0x7f7f880a36d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7fa00ffb00 0x7f7fa0197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 --2- 172.21.15.62:0/3358919435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0100cf0 0x7f7fa0197e60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:03.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 >> 172.21.15.62:0/3358919435 conn(0x7f7fa00fb0e0 msgr2=0x7f7fa01065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:03.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 shutdown_connections 2023-07-14T16:18:03.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:03.931+0000 7f7fa5dff700 1 -- 172.21.15.62:0/3358919435 wait complete. 2023-07-14T16:18:03.912 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2023-07-14T16:18:03.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:18:04.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 -- 172.21.15.62:0/2213407810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 msgr2=0x7fe1d4105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/2213407810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 0x7fe1d4105260 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009990 tx=0x7fe1c8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 -- 172.21.15.62:0/2213407810 shutdown_connections 2023-07-14T16:18:04.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/2213407810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d41057a0 0x7fe1d4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/2213407810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1d4100560 0x7fe1d4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/2213407810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 0x7fe1d4105260 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 -- 172.21.15.62:0/2213407810 >> 172.21.15.62:0/2213407810 conn(0x7fe1d40fa000 msgr2=0x7fe1d40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 -- 172.21.15.62:0/2213407810 shutdown_connections 2023-07-14T16:18:04.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.191+0000 7fe1d99ff700 1 -- 172.21.15.62:0/2213407810 wait complete. 2023-07-14T16:18:04.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.192+0000 7fe1d99ff700 1 Processor -- start 2023-07-14T16:18:04.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.193+0000 7fe1d99ff700 1 -- start start 2023-07-14T16:18:04.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.193+0000 7fe1d99ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1d4100560 0x7fe1d4195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.193+0000 7fe1d99ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 0x7fe1d4195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.193+0000 7fe1d99ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d41057a0 0x7fe1d419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.193+0000 7fe1d99ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d419d280 con 0x7fe1d4102e80 2023-07-14T16:18:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.193+0000 7fe1d99ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d419d3c0 con 0x7fe1d41057a0 2023-07-14T16:18:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.193+0000 7fe1d99ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d419d550 con 0x7fe1d4100560 2023-07-14T16:18:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1d4100560 0x7fe1d4195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1d4100560 0x7fe1d4195700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:43988/0 (socket says 172.21.15.62:43988) 2023-07-14T16:18:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d2ffd700 1 -- 172.21.15.62:0/3739796001 learned_addr learned my addr 172.21.15.62:0/3739796001 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 0x7fe1d4195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d37fe700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d41057a0 0x7fe1d419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d27fc700 1 -- 172.21.15.62:0/3739796001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1d4100560 msgr2=0x7fe1d4195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1d4100560 0x7fe1d4195700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d27fc700 1 -- 172.21.15.62:0/3739796001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d41057a0 msgr2=0x7fe1d419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d41057a0 0x7fe1d419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d27fc700 1 -- 172.21.15.62:0/3739796001 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80095f0 con 0x7fe1d4102e80 2023-07-14T16:18:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1d4100560 0x7fe1d4195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d37fe700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d41057a0 0x7fe1d419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.194+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 0x7fe1d4195c40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80098e0 tx=0x7fe1c8003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.195+0000 7fe1d89fd700 1 -- 172.21.15.62:0/3739796001 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c801d070 con 0x7fe1d4102e80 2023-07-14T16:18:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.195+0000 7fe1d89fd700 1 -- 172.21.15.62:0/3739796001 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1c800bde0 con 0x7fe1d4102e80 2023-07-14T16:18:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.195+0000 7fe1d99ff700 1 -- 172.21.15.62:0/3739796001 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d419d7d0 con 0x7fe1d4102e80 2023-07-14T16:18:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.195+0000 7fe1d89fd700 1 -- 172.21.15.62:0/3739796001 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8022c60 con 0x7fe1d4102e80 2023-07-14T16:18:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.195+0000 7fe1d99ff700 1 -- 172.21.15.62:0/3739796001 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d419dbf0 con 0x7fe1d4102e80 2023-07-14T16:18:04.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.198+0000 7fe1d89fd700 1 -- 172.21.15.62:0/3739796001 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe1c8022470 con 0x7fe1d4102e80 2023-07-14T16:18:04.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.200+0000 7fe1d89fd700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1bc0a1150 0x7fe1bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.200+0000 7fe1d89fd700 1 -- 172.21.15.62:0/3739796001 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe1c80cde90 con 0x7fe1d4102e80 2023-07-14T16:18:04.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.200+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1bc0a1150 0x7fe1bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1ed35cb80 1 Processor -- start 2023-07-14T16:18:04.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1bc0a1150 0x7fe1bc0a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4005950 tx=0x7fe1c400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1ed35cb80 1 -- start start 2023-07-14T16:18:04.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd18824d0 0x558bd18828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd18836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1883c70 0x558bd18840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd1787cb0 con 0x558bd1883c70 2023-07-14T16:18:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd17daa40 con 0x558bd18824d0 2023-07-14T16:18:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.201+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd17d3ed0 con 0x558bd1882e20 2023-07-14T16:18:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1883c70 0x558bd18840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d27fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd18836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1883c70 0x558bd18840e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52450/0 (socket says 172.21.15.62:52450) 2023-07-14T16:18:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 -- 172.21.15.62:0/3890780835 learned_addr learned my addr 172.21.15.62:0/3890780835 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd18824d0 0x558bd18828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 -- 172.21.15.62:0/3890780835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 msgr2=0x558bd18836e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd18836e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 -- 172.21.15.62:0/3890780835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd18824d0 msgr2=0x558bd18828e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd18824d0 0x558bd18828e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 -- 172.21.15.62:0/3890780835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bd1601de0 con 0x558bd1883c70 2023-07-14T16:18:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd18836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.202+0000 7fe1d37fe700 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1883c70 0x558bd18840e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00e260 tx=0x7fe1cc00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811df3aa265abd20 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.203+0000 7fe1b9ffb700 1 -- 172.21.15.62:0/3890780835 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc018070 con 0x558bd1883c70 2023-07-14T16:18:04.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.203+0000 7fe1b9ffb700 1 -- 172.21.15.62:0/3890780835 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1cc007d00 con 0x558bd1883c70 2023-07-14T16:18:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.203+0000 7fe1b9ffb700 1 -- 172.21.15.62:0/3890780835 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc014ca0 con 0x558bd1883c70 2023-07-14T16:18:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.203+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3890780835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1883c70 msgr2=0x558bd18840e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.203+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1883c70 0x558bd18840e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00e260 tx=0x7fe1cc00e570 comp rx=0 tx=0).stop 2023-07-14T16:18:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.204+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3890780835 shutdown_connections 2023-07-14T16:18:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.204+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1883c70 0x558bd18840e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.204+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd18824d0 0x558bd18828e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.204+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3890780835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd18836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.204+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3890780835 >> 172.21.15.62:0/3890780835 conn(0x558bd1877da0 msgr2=0x558bd187d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.204+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3890780835 shutdown_connections 2023-07-14T16:18:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.204+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3890780835 wait complete. 2023-07-14T16:18:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.204+0000 7fe1ed35cb80 1 -- start start 2023-07-14T16:18:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1ed35cb80 1 Processor -- start 2023-07-14T16:18:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd188c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd18936a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd1773ac0 con 0x558bd1882e20 2023-07-14T16:18:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd17737c0 con 0x558bd1883c70 2023-07-14T16:18:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd1657920 con 0x558bd18824d0 2023-07-14T16:18:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1ed35cb80 10 client.0 _ll_register_callbacks cb 0x558bd1771760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1d27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd188c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1d27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd188c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52458/0 (socket says 172.21.15.62:52458) 2023-07-14T16:18:04.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.205+0000 7fe1d27fc700 1 -- 172.21.15.62:0/233596556 learned_addr learned my addr 172.21.15.62:0/233596556 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d37fe700 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd18936a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d27fc700 1 -- 172.21.15.62:0/233596556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 msgr2=0x558bd188c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d27fc700 1 -- 172.21.15.62:0/233596556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 msgr2=0x558bd18936a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd18936a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d27fc700 1 -- 172.21.15.62:0/233596556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d419dbf0 con 0x558bd1882e20 2023-07-14T16:18:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd188c660 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8097e70 tx=0x7fe1c80a1390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779ec567d2eb0b02 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/233596556 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80a37c0 con 0x558bd1882e20 2023-07-14T16:18:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/233596556 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe1c809f030 con 0x558bd1882e20 2023-07-14T16:18:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.206+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/233596556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bd172d4f0 con 0x558bd1882e20 2023-07-14T16:18:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/233596556 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1c809f1f0 con 0x558bd1882e20 2023-07-14T16:18:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1ed35cb80 10 client.16134 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/233596556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fe1cc00f040 con 0x558bd1882e20 2023-07-14T16:18:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/233596556 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558bd172d4f0 con 0x558bd1882e20 2023-07-14T16:18:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/233596556 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe1c80a3410 con 0x558bd1882e20 2023-07-14T16:18:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1ed35cb80 10 client.16134 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1ed35cb80 10 client.16134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:18:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1ed35cb80 20 client.16134 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558bd1897a40 0x558bd1899e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1ed35cb80 4 client.16134 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.207+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/233596556 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bd17cceb0 con 0x558bd1897a40 2023-07-14T16:18:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.208+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558bd1897a40 0x558bd1899e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.208+0000 7fe1b2ffd700 10 client.16134 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:18:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.208+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558bd1897a40 0x558bd1899e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42e2c7cae1cbab3d server_cookie=a5585d39834215ea in_seq=0 out_seq=0 2023-07-14T16:18:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.209+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/233596556 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558bd17cceb0 con 0x558bd1897a40 2023-07-14T16:18:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.209+0000 7fe1b2ffd700 10 client.16134 handle_command_reply: tid=0 2023-07-14T16:18:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.209+0000 7fe1ed35cb80 1 client.16134 shutdown 2023-07-14T16:18:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/233596556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 msgr2=0x558bd188c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd188c660 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8097e70 tx=0x7fe1c80a1390 comp rx=0 tx=0).stop 2023-07-14T16:18:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/233596556 shutdown_connections 2023-07-14T16:18:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558bd1897a40 0x558bd1899e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd18936a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1b2ffd700 0 client.16134 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:18:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/233596556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd188c660 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/233596556 >> 172.21.15.62:0/233596556 conn(0x558bd1877da0 msgr2=0x558bd187c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/233596556 shutdown_connections 2023-07-14T16:18:04.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/233596556 wait complete. 2023-07-14T16:18:04.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.210+0000 7fe1ed35cb80 20 client.16134 trim_cache size 0 max 16384 2023-07-14T16:18:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.224+0000 7fe1ed35cb80 1 Processor -- start 2023-07-14T16:18:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.224+0000 7fe1ed35cb80 1 -- start start 2023-07-14T16:18:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.224+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.224+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd1893660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.224+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd187c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.224+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd188b450 con 0x558bd1882e20 2023-07-14T16:18:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.224+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd188b7d0 con 0x558bd1883c70 2023-07-14T16:18:04.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.224+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd188b940 con 0x558bd18824d0 2023-07-14T16:18:04.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd1893660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd1893660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52470/0 (socket says 172.21.15.62:52470) 2023-07-14T16:18:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 -- 172.21.15.62:0/3015017448 learned_addr learned my addr 172.21.15.62:0/3015017448 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d37fe700 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd187c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 -- 172.21.15.62:0/3015017448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 msgr2=0x558bd188ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 -- 172.21.15.62:0/3015017448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 msgr2=0x558bd187c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd187c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 -- 172.21.15.62:0/3015017448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bd15eac60 con 0x558bd1882e20 2023-07-14T16:18:04.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.225+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd1893660 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8097900 tx=0x7fe1c8097870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d70f479afd527b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ba7fc700 1 -- 172.21.15.62:0/3015017448 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8097390 con 0x558bd1882e20 2023-07-14T16:18:04.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ba7fc700 1 -- 172.21.15.62:0/3015017448 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1c80a37c0 con 0x558bd1882e20 2023-07-14T16:18:04.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ba7fc700 1 -- 172.21.15.62:0/3015017448 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80a3570 con 0x558bd1882e20 2023-07-14T16:18:04.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3015017448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 msgr2=0x558bd1893660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd1893660 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8097900 tx=0x7fe1c8097870 comp rx=0 tx=0).stop 2023-07-14T16:18:04.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3015017448 shutdown_connections 2023-07-14T16:18:04.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd187c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd18824d0 0x558bd188ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3015017448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd1882e20 0x558bd1893660 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.226+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3015017448 >> 172.21.15.62:0/3015017448 conn(0x558bd1877da0 msgr2=0x558bd189e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.227+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3015017448 shutdown_connections 2023-07-14T16:18:04.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.227+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3015017448 wait complete. 2023-07-14T16:18:04.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.227+0000 7fe1ed35cb80 1 -- start start 2023-07-14T16:18:04.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.227+0000 7fe1ed35cb80 1 Processor -- start 2023-07-14T16:18:04.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd18824d0 0x558bd1893690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd1890130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1ed35cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd18906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd18eb8c0 con 0x558bd18824d0 2023-07-14T16:18:04.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd18eba00 con 0x558bd1883c70 2023-07-14T16:18:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1ed35cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bd18ebb40 con 0x558bd1882e20 2023-07-14T16:18:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1ed35cb80 10 client.0 _ll_register_callbacks cb 0x7fe1c809af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd18824d0 0x558bd1893690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd18824d0 0x558bd1893690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52484/0 (socket says 172.21.15.62:52484) 2023-07-14T16:18:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1d2ffd700 1 -- 172.21.15.62:0/3524476647 learned_addr learned my addr 172.21.15.62:0/3524476647 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd1890130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1d2ffd700 1 -- 172.21.15.62:0/3524476647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 msgr2=0x558bd1890130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.228+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd1890130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1d2ffd700 1 -- 172.21.15.62:0/3524476647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 msgr2=0x558bd18906f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd18906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1d2ffd700 1 -- 172.21.15.62:0/3524476647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc00f040 con 0x558bd18824d0 2023-07-14T16:18:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1d2ffd700 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd18824d0 0x558bd1893690 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x558bd189a730 tx=0x7fe1c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac06d6b739e6323 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/3524476647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4004020 con 0x558bd18824d0 2023-07-14T16:18:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3524476647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1cc007d00 con 0x558bd18824d0 2023-07-14T16:18:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/3524476647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe1c4017070 con 0x558bd18824d0 2023-07-14T16:18:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.229+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/3524476647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1c4004180 con 0x558bd18824d0 2023-07-14T16:18:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1ed35cb80 10 client.16140 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3524476647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558bd1628250 con 0x558bd18824d0 2023-07-14T16:18:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/3524476647 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1cc007d00 con 0x558bd18824d0 2023-07-14T16:18:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/3524476647 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe1c402d020 con 0x558bd18824d0 2023-07-14T16:18:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1ed35cb80 10 client.16140 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1ed35cb80 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:18:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1ed35cb80 20 client.16140 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558bd18ed580 0x558bd18ef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1ed35cb80 4 client.16140 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:18:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.230+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3524476647 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558bd178f050 con 0x558bd18ed580 2023-07-14T16:18:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.231+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558bd18ed580 0x558bd18ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.231+0000 7fe1d27fc700 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558bd18ed580 0x558bd18ef990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=930358aeb55b91ee server_cookie=3a7264ab6e959b53 in_seq=0 out_seq=0 2023-07-14T16:18:04.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.231+0000 7fe1b2ffd700 10 client.16140 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:18:04.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.231+0000 7fe1b2ffd700 1 -- 172.21.15.62:0/3524476647 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558bd178f050 con 0x558bd18ed580 2023-07-14T16:18:04.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.231+0000 7fe1b2ffd700 10 client.16140 handle_command_reply: tid=0 2023-07-14T16:18:04.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.231+0000 7fe1ed35cb80 1 client.16140 shutdown 2023-07-14T16:18:04.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.231+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3524476647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd18824d0 msgr2=0x558bd1893690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.231+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd18824d0 0x558bd1893690 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x558bd189a730 tx=0x7fe1c40047c0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3524476647 shutdown_connections 2023-07-14T16:18:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558bd18ed580 0x558bd18ef990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558bd1883c70 0x558bd18906f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bd18824d0 0x558bd1893690 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 1 --2- 172.21.15.62:0/3524476647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558bd1882e20 0x558bd1890130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1b2ffd700 0 client.16140 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:18:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3524476647 >> 172.21.15.62:0/3524476647 conn(0x558bd1877da0 msgr2=0x558bd187c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3524476647 shutdown_connections 2023-07-14T16:18:04.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 1 -- 172.21.15.62:0/3524476647 wait complete. 2023-07-14T16:18:04.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.232+0000 7fe1ed35cb80 20 client.16140 trim_cache size 0 max 16384 2023-07-14T16:18:04.234 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:18:04.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 -- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1bc0a1150 msgr2=0x7fe1bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1bc0a1150 0x7fe1bc0a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4005950 tx=0x7fe1c400b500 comp rx=0 tx=0).stop 2023-07-14T16:18:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 -- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 msgr2=0x7fe1d4195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 0x7fe1d4195c40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80098e0 tx=0x7fe1c8003a50 comp rx=0 tx=0).stop 2023-07-14T16:18:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 -- 172.21.15.62:0/3739796001 shutdown_connections 2023-07-14T16:18:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe1bc0a1150 0x7fe1bc0a3600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d41057a0 0x7fe1d419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1d4100560 0x7fe1d4195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 --2- 172.21.15.62:0/3739796001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1d4102e80 0x7fe1d4195c40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 -- 172.21.15.62:0/3739796001 >> 172.21.15.62:0/3739796001 conn(0x7fe1d40fa000 msgr2=0x7fe1d41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 -- 172.21.15.62:0/3739796001 shutdown_connections 2023-07-14T16:18:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.233+0000 7fe1d99ff700 1 -- 172.21.15.62:0/3739796001 wait complete. 2023-07-14T16:18:04.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:18:04.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 -- 172.21.15.62:0/3599670293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c90101660 msgr2=0x7f2c90107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 --2- 172.21.15.62:0/3599670293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c90101660 0x7f2c90107f20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b0a0 tx=0x7f2c8800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 -- 172.21.15.62:0/3599670293 shutdown_connections 2023-07-14T16:18:04.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 --2- 172.21.15.62:0/3599670293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c90101660 0x7f2c90107f20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 --2- 172.21.15.62:0/3599670293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c900ffae0 0x7f2c900ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 --2- 172.21.15.62:0/3599670293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c90100cd0 0x7f2c90101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 -- 172.21.15.62:0/3599670293 >> 172.21.15.62:0/3599670293 conn(0x7f2c900fb0e0 msgr2=0x7f2c900fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 -- 172.21.15.62:0/3599670293 shutdown_connections 2023-07-14T16:18:04.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.491+0000 7f2c955ff700 1 -- 172.21.15.62:0/3599670293 wait complete. 2023-07-14T16:18:04.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.493+0000 7f2c955ff700 1 Processor -- start 2023-07-14T16:18:04.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.493+0000 7f2c955ff700 1 -- start start 2023-07-14T16:18:04.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.493+0000 7f2c955ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900ffae0 0x7f2c90195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.493+0000 7f2c955ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c90100cd0 0x7f2c90195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.493+0000 7f2c955ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c90101660 0x7f2c9019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.493+0000 7f2c955ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9019d3d0 con 0x7f2c900ffae0 2023-07-14T16:18:04.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.493+0000 7f2c955ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9019d510 con 0x7f2c90101660 2023-07-14T16:18:04.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.493+0000 7f2c955ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9019d6a0 con 0x7f2c90100cd0 2023-07-14T16:18:04.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900ffae0 0x7f2c90195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900ffae0 0x7f2c90195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52500/0 (socket says 172.21.15.62:52500) 2023-07-14T16:18:04.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8effd700 1 -- 172.21.15.62:0/963080942 learned_addr learned my addr 172.21.15.62:0/963080942 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c90100cd0 0x7f2c90195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8f7fe700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c90101660 0x7f2c9019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8effd700 1 -- 172.21.15.62:0/963080942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c90100cd0 msgr2=0x7f2c90195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8effd700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c90100cd0 0x7f2c90195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8effd700 1 -- 172.21.15.62:0/963080942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c90101660 msgr2=0x7f2c9019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8effd700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c90101660 0x7f2c9019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.494+0000 7f2c8effd700 1 -- 172.21.15.62:0/963080942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c88009d00 con 0x7f2c900ffae0 2023-07-14T16:18:04.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.495+0000 7f2c8effd700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900ffae0 0x7f2c90195850 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000e970 tx=0x7f2c8000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.495+0000 7f2c77fff700 1 -- 172.21.15.62:0/963080942 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80009790 con 0x7f2c900ffae0 2023-07-14T16:18:04.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.495+0000 7f2c955ff700 1 -- 172.21.15.62:0/963080942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9019d980 con 0x7f2c900ffae0 2023-07-14T16:18:04.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.495+0000 7f2c77fff700 1 -- 172.21.15.62:0/963080942 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c8000c7e0 con 0x7f2c900ffae0 2023-07-14T16:18:04.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.495+0000 7f2c77fff700 1 -- 172.21.15.62:0/963080942 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80003bd0 con 0x7f2c900ffae0 2023-07-14T16:18:04.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.495+0000 7f2c955ff700 1 -- 172.21.15.62:0/963080942 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9019de90 con 0x7f2c900ffae0 2023-07-14T16:18:04.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.500+0000 7f2c77fff700 1 -- 172.21.15.62:0/963080942 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c8000c950 con 0x7f2c900ffae0 2023-07-14T16:18:04.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.501+0000 7f2ca901db80 1 Processor -- start 2023-07-14T16:18:04.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.501+0000 7f2ca901db80 1 -- start start 2023-07-14T16:18:04.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.501+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c702d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.501+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702de20 0x55e9c702e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.501+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702ec70 0x55e9c702f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.501+0000 7f2ca901db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c6f32cb0 con 0x55e9c702d4d0 2023-07-14T16:18:04.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.501+0000 7f2ca901db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c6f85a40 con 0x55e9c702ec70 2023-07-14T16:18:04.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.501+0000 7f2ca901db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c6f7eed0 con 0x55e9c702de20 2023-07-14T16:18:04.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c702d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c702d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52514/0 (socket says 172.21.15.62:52514) 2023-07-14T16:18:04.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/1734697563 learned_addr learned my addr 172.21.15.62:0/1734697563 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702ec70 0x55e9c702f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8effd700 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702de20 0x55e9c702e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c77fff700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c780a1150 0x7f2c780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8e7fc700 1 -- 172.21.15.62:0/1734697563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702de20 msgr2=0x55e9c702e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702de20 0x55e9c702e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8e7fc700 1 -- 172.21.15.62:0/1734697563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 msgr2=0x55e9c702d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c702d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8e7fc700 1 -- 172.21.15.62:0/1734697563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9c6dacde0 con 0x55e9c702ec70 2023-07-14T16:18:04.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c77fff700 1 -- 172.21.15.62:0/963080942 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c80014070 con 0x7f2c900ffae0 2023-07-14T16:18:04.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.502+0000 7f2c8f7fe700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c780a1150 0x7f2c780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.503+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702ec70 0x55e9c702f0e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009cb0 tx=0x7f2c84011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38f43db3f22dafc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.503+0000 7f2c757fa700 1 -- 172.21.15.62:0/1734697563 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84011d20 con 0x55e9c702ec70 2023-07-14T16:18:04.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.503+0000 7f2c757fa700 1 -- 172.21.15.62:0/1734697563 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c84011e80 con 0x55e9c702ec70 2023-07-14T16:18:04.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.503+0000 7f2c757fa700 1 -- 172.21.15.62:0/1734697563 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84005280 con 0x55e9c702ec70 2023-07-14T16:18:04.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 -- 172.21.15.62:0/1734697563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702ec70 msgr2=0x55e9c702f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702ec70 0x55e9c702f0e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009cb0 tx=0x7f2c84011440 comp rx=0 tx=0).stop 2023-07-14T16:18:04.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2c8f7fe700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c780a1150 0x7f2c780a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009600 tx=0x7f2c880099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 -- 172.21.15.62:0/1734697563 shutdown_connections 2023-07-14T16:18:04.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702ec70 0x55e9c702f0e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c702d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 --2- 172.21.15.62:0/1734697563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702de20 0x55e9c702e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 -- 172.21.15.62:0/1734697563 >> 172.21.15.62:0/1734697563 conn(0x55e9c7022da0 msgr2=0x55e9c70281d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 -- 172.21.15.62:0/1734697563 shutdown_connections 2023-07-14T16:18:04.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.504+0000 7f2ca901db80 1 -- 172.21.15.62:0/1734697563 wait complete. 2023-07-14T16:18:04.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.505+0000 7f2ca901db80 1 -- start start 2023-07-14T16:18:04.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.505+0000 7f2ca901db80 1 Processor -- start 2023-07-14T16:18:04.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7037180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c70376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c703e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2ca901db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c6f1eac0 con 0x55e9c702ec70 2023-07-14T16:18:04.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2ca901db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c6f1e7c0 con 0x55e9c702de20 2023-07-14T16:18:04.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2ca901db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c6e02920 con 0x55e9c702d4d0 2023-07-14T16:18:04.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2ca901db80 10 client.0 _ll_register_callbacks cb 0x55e9c6f1c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:04.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2c8f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c703e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2c8f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c703e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52516/0 (socket says 172.21.15.62:52516) 2023-07-14T16:18:04.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2c8f7fe700 1 -- 172.21.15.62:0/3200487880 learned_addr learned my addr 172.21.15.62:0/3200487880 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7037180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c70376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.506+0000 7f2c8effd700 1 -- 172.21.15.62:0/3200487880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 msgr2=0x55e9c70376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c8effd700 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c70376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c8effd700 1 -- 172.21.15.62:0/3200487880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 msgr2=0x55e9c703e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c8effd700 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c703e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c8effd700 1 -- 172.21.15.62:0/3200487880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c840098e0 con 0x55e9c702d4d0 2023-07-14T16:18:04.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c8f7fe700 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c703e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:04.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c8effd700 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7037180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c80096980 tx=0x7f2c80003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41e26132632df8a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:18:04.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c72ff5700 1 -- 172.21.15.62:0/3200487880 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8009a370 con 0x55e9c702d4d0 2023-07-14T16:18:04.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c72ff5700 1 -- 172.21.15.62:0/3200487880 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c8000cc50 con 0x55e9c702d4d0 2023-07-14T16:18:04.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2ca901db80 1 -- 172.21.15.62:0/3200487880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9c6f77eb0 con 0x55e9c702d4d0 2023-07-14T16:18:04.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2c72ff5700 1 -- 172.21.15.62:0/3200487880 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c80099f10 con 0x55e9c702d4d0 2023-07-14T16:18:04.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2ca901db80 10 client.25807 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:04.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.507+0000 7f2ca901db80 1 -- 172.21.15.62:0/3200487880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e9c6d95c60 con 0x55e9c702d4d0 2023-07-14T16:18:04.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.508+0000 7f2c72ff5700 1 -- 172.21.15.62:0/3200487880 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c800b8030 con 0x55e9c702d4d0 2023-07-14T16:18:04.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.508+0000 7f2c72ff5700 1 -- 172.21.15.62:0/3200487880 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2c80099490 con 0x55e9c702d4d0 2023-07-14T16:18:04.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.508+0000 7f2ca901db80 10 client.25807 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:04.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.508+0000 7f2ca901db80 10 client.25807 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:18:04.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.508+0000 7f2ca901db80 20 client.25807 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:04.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.508+0000 7f2ca901db80 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55e9c7042f40 0x55e9c7045350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.508+0000 7f2ca901db80 4 client.25807 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:04.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.508+0000 7f2ca901db80 1 -- 172.21.15.62:0/3200487880 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9c6f3a050 con 0x55e9c7042f40 2023-07-14T16:18:04.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.509+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55e9c7042f40 0x55e9c7045350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.509+0000 7f2c72ff5700 10 client.25807 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:18:04.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.509+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55e9c7042f40 0x55e9c7045350 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=612d1b362bf7e9f server_cookie=7bc73b1ebad9bb2c in_seq=0 out_seq=0 2023-07-14T16:18:04.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.510+0000 7f2c72ff5700 1 -- 172.21.15.62:0/3200487880 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e9c6f3a050 con 0x55e9c7042f40 2023-07-14T16:18:04.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.510+0000 7f2c72ff5700 10 client.25807 handle_command_reply: tid=0 2023-07-14T16:18:04.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.510+0000 7f2ca901db80 1 client.25807 shutdown 2023-07-14T16:18:04.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.510+0000 7f2ca901db80 1 -- 172.21.15.62:0/3200487880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 msgr2=0x55e9c7037180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.510+0000 7f2ca901db80 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7037180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c80096980 tx=0x7f2c80003680 comp rx=0 tx=0).stop 2023-07-14T16:18:04.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 1 -- 172.21.15.62:0/3200487880 shutdown_connections 2023-07-14T16:18:04.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55e9c7042f40 0x55e9c7045350 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c703e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2c72ff5700 0 client.25807 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:18:04.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7037180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 1 --2- 172.21.15.62:0/3200487880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c70376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 1 -- 172.21.15.62:0/3200487880 >> 172.21.15.62:0/3200487880 conn(0x55e9c7022da0 msgr2=0x55e9c70278b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 1 -- 172.21.15.62:0/3200487880 shutdown_connections 2023-07-14T16:18:04.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 1 -- 172.21.15.62:0/3200487880 wait complete. 2023-07-14T16:18:04.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.511+0000 7f2ca901db80 20 client.25807 trim_cache size 0 max 16384 2023-07-14T16:18:04.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.530+0000 7f2ca901db80 1 Processor -- start 2023-07-14T16:18:04.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.530+0000 7f2ca901db80 1 -- start start 2023-07-14T16:18:04.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.530+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c7036830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.530+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7049cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.530+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702ec70 0x55e9c7035e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.530+0000 7f2ca901db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c7042d90 con 0x55e9c702d4d0 2023-07-14T16:18:04.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.530+0000 7f2ca901db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c704a220 con 0x55e9c702de20 2023-07-14T16:18:04.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.530+0000 7f2ca901db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c70363c0 con 0x55e9c702ec70 2023-07-14T16:18:04.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c7036830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c7036830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52530/0 (socket says 172.21.15.62:52530) 2023-07-14T16:18:04.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 -- 172.21.15.62:0/1034835604 learned_addr learned my addr 172.21.15.62:0/1034835604 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8f7fe700 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702ec70 0x55e9c7035e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7049cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 -- 172.21.15.62:0/1034835604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702ec70 msgr2=0x55e9c7035e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702ec70 0x55e9c7035e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 -- 172.21.15.62:0/1034835604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 msgr2=0x55e9c7049cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7049cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 -- 172.21.15.62:0/1034835604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9c6dd3250 con 0x55e9c702d4d0 2023-07-14T16:18:04.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7049cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:04.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.531+0000 7f2c8effd700 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c7036830 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2c80099c10 tx=0x7f2c8000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45474519fef7952d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2c75ffb700 1 -- 172.21.15.62:0/1034835604 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800c8070 con 0x55e9c702d4d0 2023-07-14T16:18:04.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2c75ffb700 1 -- 172.21.15.62:0/1034835604 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c800b8040 con 0x55e9c702d4d0 2023-07-14T16:18:04.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2c75ffb700 1 -- 172.21.15.62:0/1034835604 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800ca2c0 con 0x55e9c702d4d0 2023-07-14T16:18:04.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2ca901db80 1 -- 172.21.15.62:0/1034835604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 msgr2=0x55e9c7036830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2ca901db80 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c7036830 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2c80099c10 tx=0x7f2c8000cf50 comp rx=0 tx=0).stop 2023-07-14T16:18:04.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2ca901db80 1 -- 172.21.15.62:0/1034835604 shutdown_connections 2023-07-14T16:18:04.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2ca901db80 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702ec70 0x55e9c7035e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2ca901db80 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702d4d0 0x55e9c7036830 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2ca901db80 1 --2- 172.21.15.62:0/1034835604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7049cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2ca901db80 1 -- 172.21.15.62:0/1034835604 >> 172.21.15.62:0/1034835604 conn(0x55e9c7022da0 msgr2=0x55e9c70281a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.532+0000 7f2ca901db80 1 -- 172.21.15.62:0/1034835604 shutdown_connections 2023-07-14T16:18:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.533+0000 7f2ca901db80 1 -- 172.21.15.62:0/1034835604 wait complete. 2023-07-14T16:18:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.533+0000 7f2ca901db80 1 -- start start 2023-07-14T16:18:04.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.533+0000 7f2ca901db80 1 Processor -- start 2023-07-14T16:18:04.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.534+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7094fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.534+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7097520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.534+0000 7f2ca901db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c7099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.534+0000 7f2ca901db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c709a190 con 0x55e9c702ec70 2023-07-14T16:18:04.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.534+0000 7f2ca901db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c709a300 con 0x55e9c702de20 2023-07-14T16:18:04.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.534+0000 7f2ca901db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c709a4c0 con 0x55e9c702d4d0 2023-07-14T16:18:04.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.534+0000 7f2ca901db80 10 client.0 _ll_register_callbacks cb 0x55e9c6f1c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:04.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.534+0000 7f2c8e7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7097520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c7099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8e7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7097520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57444/0 (socket says 172.21.15.62:57444) 2023-07-14T16:18:04.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8e7fc700 1 -- 172.21.15.62:0/860791572 learned_addr learned my addr 172.21.15.62:0/860791572 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8effd700 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7094fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8e7fc700 1 -- 172.21.15.62:0/860791572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 msgr2=0x55e9c7094fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7094fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8e7fc700 1 -- 172.21.15.62:0/860791572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 msgr2=0x55e9c7099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c7099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8e7fc700 1 -- 172.21.15.62:0/860791572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9c6d95c60 con 0x55e9c702de20 2023-07-14T16:18:04.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8f7fe700 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c7099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:04.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c8e7fc700 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7097520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009c80 tx=0x7f2c84009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d02059f0b9ca27 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.535+0000 7f2c72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:18:04.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.536+0000 7f2c72ff5700 1 -- 172.21.15.62:0/860791572 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84029070 con 0x55e9c702de20 2023-07-14T16:18:04.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.536+0000 7f2ca901db80 1 -- 172.21.15.62:0/860791572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9c6ed84f0 con 0x55e9c702de20 2023-07-14T16:18:04.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.536+0000 7f2c72ff5700 1 -- 172.21.15.62:0/860791572 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c84013bc0 con 0x55e9c702de20 2023-07-14T16:18:04.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.536+0000 7f2c72ff5700 1 -- 172.21.15.62:0/860791572 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c84013d80 con 0x55e9c702de20 2023-07-14T16:18:04.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.536+0000 7f2ca901db80 10 client.25901 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:04.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.536+0000 7f2ca901db80 1 -- 172.21.15.62:0/860791572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e9c6dab2b0 con 0x55e9c702de20 2023-07-14T16:18:04.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.536+0000 7f2c72ff5700 1 -- 172.21.15.62:0/860791572 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c84038020 con 0x55e9c702de20 2023-07-14T16:18:04.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.537+0000 7f2c72ff5700 1 -- 172.21.15.62:0/860791572 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2c84003680 con 0x55e9c702de20 2023-07-14T16:18:04.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.537+0000 7f2ca901db80 10 client.25901 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:04.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.537+0000 7f2ca901db80 10 client.25901 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:18:04.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.537+0000 7f2ca901db80 20 client.25901 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:04.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.537+0000 7f2ca901db80 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55e9c709be80 0x55e9c709e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.537+0000 7f2ca901db80 4 client.25901 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:18:04.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.537+0000 7f2ca901db80 1 -- 172.21.15.62:0/860791572 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e9c6dcdf70 con 0x55e9c709be80 2023-07-14T16:18:04.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.537+0000 7f2c8effd700 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55e9c709be80 0x55e9c709e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2c8effd700 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55e9c709be80 0x55e9c709e290 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce5b22b8491fd6fe server_cookie=3a152bb99f7f90a5 in_seq=0 out_seq=0 2023-07-14T16:18:04.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2c72ff5700 10 client.25901 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:18:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2c72ff5700 1 -- 172.21.15.62:0/860791572 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e9c6dcdf70 con 0x55e9c709be80 2023-07-14T16:18:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2c72ff5700 10 client.25901 handle_command_reply: tid=0 2023-07-14T16:18:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 client.25901 shutdown 2023-07-14T16:18:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 -- 172.21.15.62:0/860791572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 msgr2=0x55e9c7097520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7097520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009c80 tx=0x7f2c84009490 comp rx=0 tx=0).stop 2023-07-14T16:18:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 -- 172.21.15.62:0/860791572 shutdown_connections 2023-07-14T16:18:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55e9c709be80 0x55e9c709e290 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e9c702ec70 0x55e9c7099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c702d4d0 0x55e9c7094fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2c72ff5700 0 client.25901 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:18:04.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 --2- 172.21.15.62:0/860791572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9c702de20 0x55e9c7097520 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 -- 172.21.15.62:0/860791572 >> 172.21.15.62:0/860791572 conn(0x55e9c7022da0 msgr2=0x55e9c7031b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 -- 172.21.15.62:0/860791572 shutdown_connections 2023-07-14T16:18:04.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.538+0000 7f2ca901db80 1 -- 172.21.15.62:0/860791572 wait complete. 2023-07-14T16:18:04.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2ca901db80 20 client.25901 trim_cache size 0 max 16384 2023-07-14T16:18:04.524 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:18:04.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 -- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c780a1150 msgr2=0x7f2c780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c780a1150 0x7f2c780a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009600 tx=0x7f2c880099e0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 -- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900ffae0 msgr2=0x7f2c90195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900ffae0 0x7f2c90195850 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000e970 tx=0x7f2c8000ec80 comp rx=0 tx=0).stop 2023-07-14T16:18:04.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 -- 172.21.15.62:0/963080942 shutdown_connections 2023-07-14T16:18:04.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2c780a1150 0x7f2c780a3600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c90101660 0x7f2c9019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900ffae0 0x7f2c90195850 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 --2- 172.21.15.62:0/963080942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c90100cd0 0x7f2c90195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.539+0000 7f2c955ff700 1 -- 172.21.15.62:0/963080942 >> 172.21.15.62:0/963080942 conn(0x7f2c900fb0e0 msgr2=0x7f2c90104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.540+0000 7f2c955ff700 1 -- 172.21.15.62:0/963080942 shutdown_connections 2023-07-14T16:18:04.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.540+0000 7f2c955ff700 1 -- 172.21.15.62:0/963080942 wait complete. 2023-07-14T16:18:04.538 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:18:04.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 -- 172.21.15.62:0/3306117287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841057a0 msgr2=0x7f1384107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 --2- 172.21.15.62:0/3306117287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841057a0 0x7f1384107c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b0a0 tx=0x7f137c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 -- 172.21.15.62:0/3306117287 shutdown_connections 2023-07-14T16:18:04.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 --2- 172.21.15.62:0/3306117287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13841057a0 0x7f1384107c00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 --2- 172.21.15.62:0/3306117287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1384100560 0x7f1384102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 --2- 172.21.15.62:0/3306117287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1384102e80 0x7f1384105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 -- 172.21.15.62:0/3306117287 >> 172.21.15.62:0/3306117287 conn(0x7f13840fa000 msgr2=0x7f13840fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 -- 172.21.15.62:0/3306117287 shutdown_connections 2023-07-14T16:18:04.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.802+0000 7f138b5ff700 1 -- 172.21.15.62:0/3306117287 wait complete. 2023-07-14T16:18:04.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.804+0000 7f138b5ff700 1 Processor -- start 2023-07-14T16:18:04.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.804+0000 7f138b5ff700 1 -- start start 2023-07-14T16:18:04.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.804+0000 7f138b5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1384100560 0x7f13840ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.804+0000 7f138b5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1384102e80 0x7f13840fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138b5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13841057a0 0x7f13840fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138b5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13840ff1f0 con 0x7f1384100560 2023-07-14T16:18:04.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138b5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13840ff360 con 0x7f1384102e80 2023-07-14T16:18:04.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138b5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138419f8d0 con 0x7f13841057a0 2023-07-14T16:18:04.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138939b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1384100560 0x7f13840ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138939b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1384100560 0x7f13840ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52556/0 (socket says 172.21.15.62:52556) 2023-07-14T16:18:04.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138939b700 1 -- 172.21.15.62:0/1650381957 learned_addr learned my addr 172.21.15.62:0/1650381957 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f1388b9a700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1384102e80 0x7f13840fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f1389b9c700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13841057a0 0x7f13840fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138939b700 1 -- 172.21.15.62:0/1650381957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13841057a0 msgr2=0x7f13840fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138939b700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13841057a0 0x7f13840fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138939b700 1 -- 172.21.15.62:0/1650381957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1384102e80 msgr2=0x7f13840fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138939b700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1384102e80 0x7f13840fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.805+0000 7f138939b700 1 -- 172.21.15.62:0/1650381957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c009d00 con 0x7f1384100560 2023-07-14T16:18:04.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.806+0000 7f138939b700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1384100560 0x7f13840ffed0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f138000d950 tx=0x7f138000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.806+0000 7f137a7fc700 1 -- 172.21.15.62:0/1650381957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380009b40 con 0x7f1384100560 2023-07-14T16:18:04.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.806+0000 7f137a7fc700 1 -- 172.21.15.62:0/1650381957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1380009ca0 con 0x7f1384100560 2023-07-14T16:18:04.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.806+0000 7f137a7fc700 1 -- 172.21.15.62:0/1650381957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000f620 con 0x7f1384100560 2023-07-14T16:18:04.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.806+0000 7f138b5ff700 1 -- 172.21.15.62:0/1650381957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138419fbb0 con 0x7f1384100560 2023-07-14T16:18:04.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.806+0000 7f138b5ff700 1 -- 172.21.15.62:0/1650381957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13841a0070 con 0x7f1384100560 2023-07-14T16:18:04.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.810+0000 7f137a7fc700 1 -- 172.21.15.62:0/1650381957 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1380010460 con 0x7f1384100560 2023-07-14T16:18:04.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.811+0000 7f139ef79b80 1 Processor -- start 2023-07-14T16:18:04.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f139ef79b80 1 -- start start 2023-07-14T16:18:04.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc507a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc507c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f139ef79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc4f7fcb0 con 0x55edc507bc70 2023-07-14T16:18:04.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f139ef79b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc4fd2a40 con 0x55edc507ae20 2023-07-14T16:18:04.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f139ef79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc4fcbed0 con 0x55edc507a4d0 2023-07-14T16:18:04.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f137a7fc700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f13700a11d0 0x7f13700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f1388b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc507c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f1388b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc507c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52564/0 (socket says 172.21.15.62:52564) 2023-07-14T16:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f1388b9a700 1 -- 172.21.15.62:0/1557833358 learned_addr learned my addr 172.21.15.62:0/1557833358 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.812+0000 7f137a7fc700 1 -- 172.21.15.62:0/1650381957 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f13800ccc90 con 0x7f1384100560 2023-07-14T16:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f1389b9c700 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc507a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f138939b700 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f1389b9c700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f13700a11d0 0x7f13700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f138939b700 1 -- 172.21.15.62:0/1557833358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 msgr2=0x55edc507a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f138939b700 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc507a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f138939b700 1 -- 172.21.15.62:0/1557833358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 msgr2=0x55edc507c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f138939b700 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc507c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f138939b700 1 -- 172.21.15.62:0/1557833358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edc4df9de0 con 0x55edc507ae20 2023-07-14T16:18:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f138939b700 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507b6e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1380099fe0 tx=0x7f1380099ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f697fe9f8f3f07af server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.813+0000 7f1389b9c700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f13700a11d0 0x7f13700a3680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1384100530 tx=0x7f137c0099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.814+0000 7f136f7fe700 1 -- 172.21.15.62:0/1557833358 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138009a050 con 0x55edc507ae20 2023-07-14T16:18:04.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.814+0000 7f136f7fe700 1 -- 172.21.15.62:0/1557833358 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f138009e080 con 0x55edc507ae20 2023-07-14T16:18:04.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.814+0000 7f136f7fe700 1 -- 172.21.15.62:0/1557833358 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138009a2b0 con 0x55edc507ae20 2023-07-14T16:18:04.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.814+0000 7f139ef79b80 1 -- 172.21.15.62:0/1557833358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 msgr2=0x55edc507b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.814+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507b6e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1380099fe0 tx=0x7f1380099ea0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.814+0000 7f139ef79b80 1 -- 172.21.15.62:0/1557833358 shutdown_connections 2023-07-14T16:18:04.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.815+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc507c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.815+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc507a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.815+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1557833358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507b6e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.815+0000 7f139ef79b80 1 -- 172.21.15.62:0/1557833358 >> 172.21.15.62:0/1557833358 conn(0x55edc506fda0 msgr2=0x55edc50751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.815+0000 7f139ef79b80 1 -- 172.21.15.62:0/1557833358 shutdown_connections 2023-07-14T16:18:04.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.815+0000 7f139ef79b80 1 -- 172.21.15.62:0/1557833358 wait complete. 2023-07-14T16:18:04.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.815+0000 7f139ef79b80 1 -- start start 2023-07-14T16:18:04.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.815+0000 7f139ef79b80 1 Processor -- start 2023-07-14T16:18:04.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507a4d0 0x55edc50840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc5084630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507bc70 0x55edc508b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f139ef79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc4f6bac0 con 0x55edc507ae20 2023-07-14T16:18:04.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f139ef79b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc4f6b7c0 con 0x55edc507a4d0 2023-07-14T16:18:04.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f139ef79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc4e4f920 con 0x55edc507bc70 2023-07-14T16:18:04.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f139ef79b80 10 client.0 _ll_register_callbacks cb 0x55edc4f69760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:04.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f1388b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc5084630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f1388b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc5084630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52578/0 (socket says 172.21.15.62:52578) 2023-07-14T16:18:04.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f1388b9a700 1 -- 172.21.15.62:0/1855702062 learned_addr learned my addr 172.21.15.62:0/1855702062 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.816+0000 7f138939b700 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507a4d0 0x55edc50840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f1388b9a700 1 -- 172.21.15.62:0/1855702062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507bc70 msgr2=0x55edc508b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f1388b9a700 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507bc70 0x55edc508b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f1388b9a700 1 -- 172.21.15.62:0/1855702062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507a4d0 msgr2=0x55edc50840f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f1388b9a700 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507a4d0 0x55edc50840f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f1388b9a700 1 -- 172.21.15.62:0/1855702062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13841a0070 con 0x55edc507ae20 2023-07-14T16:18:04.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f1388b9a700 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc5084630 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f137400dfa0 tx=0x7f1374012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae68128d2b9643d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f136cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:04.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f136cff9700 1 -- 172.21.15.62:0/1855702062 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137400f800 con 0x55edc507ae20 2023-07-14T16:18:04.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f139ef79b80 1 -- 172.21.15.62:0/1855702062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edc4f254f0 con 0x55edc507ae20 2023-07-14T16:18:04.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.817+0000 7f136cff9700 1 -- 172.21.15.62:0/1855702062 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f137400f960 con 0x55edc507ae20 2023-07-14T16:18:04.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f136cff9700 1 -- 172.21.15.62:0/1855702062 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f137400fb20 con 0x55edc507ae20 2023-07-14T16:18:04.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f139ef79b80 10 client.16170 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:04.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f139ef79b80 1 -- 172.21.15.62:0/1855702062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55edc4de2c60 con 0x55edc507ae20 2023-07-14T16:18:04.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f136cff9700 1 -- 172.21.15.62:0/1855702062 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55edc4f254f0 con 0x55edc507ae20 2023-07-14T16:18:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f136cff9700 1 -- 172.21.15.62:0/1855702062 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f137401a3e0 con 0x55edc507ae20 2023-07-14T16:18:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f139ef79b80 10 client.16170 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f139ef79b80 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f139ef79b80 20 client.16170 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55edc508ff40 0x55edc5092350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f139ef79b80 4 client.16170 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:04.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.818+0000 7f139ef79b80 1 -- 172.21.15.62:0/1855702062 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edc4fc4eb0 con 0x55edc508ff40 2023-07-14T16:18:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.819+0000 7f138939b700 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55edc508ff40 0x55edc5092350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.819+0000 7f138939b700 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55edc508ff40 0x55edc5092350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c2d2faf447d1d6 server_cookie=c394670ff9a653bb in_seq=0 out_seq=0 2023-07-14T16:18:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.819+0000 7f136cff9700 10 client.16170 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f136cff9700 1 -- 172.21.15.62:0/1855702062 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55edc4fc4eb0 con 0x55edc508ff40 2023-07-14T16:18:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f136cff9700 10 client.16170 handle_command_reply: tid=0 2023-07-14T16:18:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 client.16170 shutdown 2023-07-14T16:18:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 -- 172.21.15.62:0/1855702062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 msgr2=0x55edc5084630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc5084630 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f137400dfa0 tx=0x7f1374012730 comp rx=0 tx=0).stop 2023-07-14T16:18:04.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 -- 172.21.15.62:0/1855702062 shutdown_connections 2023-07-14T16:18:04.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507bc70 0x55edc508b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55edc508ff40 0x55edc5092350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507a4d0 0x55edc50840f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1855702062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc5084630 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f136cff9700 0 client.16170 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.820+0000 7f139ef79b80 1 -- 172.21.15.62:0/1855702062 >> 172.21.15.62:0/1855702062 conn(0x55edc506fda0 msgr2=0x55edc50748b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.821+0000 7f139ef79b80 1 -- 172.21.15.62:0/1855702062 shutdown_connections 2023-07-14T16:18:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.821+0000 7f139ef79b80 1 -- 172.21.15.62:0/1855702062 wait complete. 2023-07-14T16:18:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.821+0000 7f139ef79b80 20 client.16170 trim_cache size 0 max 16384 2023-07-14T16:18:04.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.835+0000 7f139ef79b80 1 Processor -- start 2023-07-14T16:18:04.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.835+0000 7f139ef79b80 1 -- start start 2023-07-14T16:18:04.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.835+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc507eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.835+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.835+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc508b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.835+0000 7f139ef79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5083c60 con 0x55edc507bc70 2023-07-14T16:18:04.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.835+0000 7f139ef79b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5075ed0 con 0x55edc507ae20 2023-07-14T16:18:04.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.835+0000 7f139ef79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5084d70 con 0x55edc507a4d0 2023-07-14T16:18:04.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1389b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc508b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1388b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1389b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc508b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52586/0 (socket says 172.21.15.62:52586) 2023-07-14T16:18:04.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1388b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57484/0 (socket says 172.21.15.62:57484) 2023-07-14T16:18:04.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1389b9c700 1 -- 172.21.15.62:0/405190577 learned_addr learned my addr 172.21.15.62:0/405190577 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f138939b700 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc507eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1388b9a700 1 -- 172.21.15.62:0/405190577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 msgr2=0x55edc507eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1388b9a700 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc507eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1388b9a700 1 -- 172.21.15.62:0/405190577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 msgr2=0x55edc508b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1388b9a700 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc508b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1388b9a700 1 -- 172.21.15.62:0/405190577 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edc507c560 con 0x55edc507ae20 2023-07-14T16:18:04.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.836+0000 7f1389b9c700 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc508b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:04.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.837+0000 7f1388b9a700 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507f070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1374000c00 tx=0x7f137400d800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b4164e0f7abe92 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.837+0000 7f136ffff700 1 -- 172.21.15.62:0/405190577 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13740253a0 con 0x55edc507ae20 2023-07-14T16:18:04.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.837+0000 7f136ffff700 1 -- 172.21.15.62:0/405190577 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1374025500 con 0x55edc507ae20 2023-07-14T16:18:04.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.837+0000 7f136ffff700 1 -- 172.21.15.62:0/405190577 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374025710 con 0x55edc507ae20 2023-07-14T16:18:04.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 -- 172.21.15.62:0/405190577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 msgr2=0x55edc507f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507f070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1374000c00 tx=0x7f137400d800 comp rx=0 tx=0).stop 2023-07-14T16:18:04.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 -- 172.21.15.62:0/405190577 shutdown_connections 2023-07-14T16:18:04.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507bc70 0x55edc508b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc507eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 --2- 172.21.15.62:0/405190577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507ae20 0x55edc507f070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 -- 172.21.15.62:0/405190577 >> 172.21.15.62:0/405190577 conn(0x55edc506fda0 msgr2=0x55edc5082000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 -- 172.21.15.62:0/405190577 shutdown_connections 2023-07-14T16:18:04.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 -- 172.21.15.62:0/405190577 wait complete. 2023-07-14T16:18:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 -- start start 2023-07-14T16:18:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.838+0000 7f139ef79b80 1 Processor -- start 2023-07-14T16:18:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.839+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc508afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.839+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc508b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.839+0000 7f139ef79b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507bc70 0x55edc5087a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.839+0000 7f139ef79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc5088170 con 0x55edc507ae20 2023-07-14T16:18:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.839+0000 7f139ef79b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc50882e0 con 0x55edc507bc70 2023-07-14T16:18:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.839+0000 7f139ef79b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edc50884a0 con 0x55edc507a4d0 2023-07-14T16:18:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.839+0000 7f139ef79b80 10 client.0 _ll_register_callbacks cb 0x7f137400daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc508b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc508b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52600/0 (socket says 172.21.15.62:52600) 2023-07-14T16:18:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f138939b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc508afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 -- 172.21.15.62:0/1945624294 learned_addr learned my addr 172.21.15.62:0/1945624294 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1389b9c700 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507bc70 0x55edc5087a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 -- 172.21.15.62:0/1945624294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 msgr2=0x55edc508afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc508afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 -- 172.21.15.62:0/1945624294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507bc70 msgr2=0x55edc5087a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507bc70 0x55edc5087a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 -- 172.21.15.62:0/1945624294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13841087f0 con 0x55edc507ae20 2023-07-14T16:18:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f138939b700 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc508afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f1388b9a700 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc508b500 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f137400d870 tx=0x7f137401ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef602ff56a5fdc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.840+0000 7f136cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.841+0000 7f136cff9700 1 -- 172.21.15.62:0/1945624294 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13740253a0 con 0x55edc507ae20 2023-07-14T16:18:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.841+0000 7f136cff9700 1 -- 172.21.15.62:0/1945624294 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f13740256f0 con 0x55edc507ae20 2023-07-14T16:18:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.841+0000 7f139ef79b80 1 -- 172.21.15.62:0/1945624294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f138009e080 con 0x55edc507ae20 2023-07-14T16:18:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.841+0000 7f136cff9700 1 -- 172.21.15.62:0/1945624294 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13740258b0 con 0x55edc507ae20 2023-07-14T16:18:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.841+0000 7f139ef79b80 10 client.16176 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.841+0000 7f139ef79b80 1 -- 172.21.15.62:0/1945624294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55edc4de2c60 con 0x55edc507ae20 2023-07-14T16:18:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.841+0000 7f136cff9700 1 -- 172.21.15.62:0/1945624294 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f138009e080 con 0x55edc507ae20 2023-07-14T16:18:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f136cff9700 1 -- 172.21.15.62:0/1945624294 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f137401b980 con 0x55edc507ae20 2023-07-14T16:18:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f139ef79b80 10 client.16176 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f139ef79b80 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f139ef79b80 20 client.16176 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55edc50e5080 0x55edc50e7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f139ef79b80 4 client.16176 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:18:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f139ef79b80 1 -- 172.21.15.62:0/1945624294 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55edc4f87050 con 0x55edc50e5080 2023-07-14T16:18:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f138939b700 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55edc50e5080 0x55edc50e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f138939b700 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55edc50e5080 0x55edc50e7490 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f362ee5ffa97d38a server_cookie=61d870610a13cba6 in_seq=0 out_seq=0 2023-07-14T16:18:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.842+0000 7f136cff9700 10 client.16176 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f136cff9700 1 -- 172.21.15.62:0/1945624294 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55edc4f87050 con 0x55edc50e5080 2023-07-14T16:18:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f136cff9700 10 client.16176 handle_command_reply: tid=0 2023-07-14T16:18:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 client.16176 shutdown 2023-07-14T16:18:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 -- 172.21.15.62:0/1945624294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 msgr2=0x55edc508b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc508b500 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f137400d870 tx=0x7f137401ae40 comp rx=0 tx=0).stop 2023-07-14T16:18:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 -- 172.21.15.62:0/1945624294 shutdown_connections 2023-07-14T16:18:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55edc50e5080 0x55edc50e7490 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55edc507bc70 0x55edc5087a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55edc507a4d0 0x55edc508afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f136cff9700 0 client.16176 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 --2- 172.21.15.62:0/1945624294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edc507ae20 0x55edc508b500 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 -- 172.21.15.62:0/1945624294 >> 172.21.15.62:0/1945624294 conn(0x55edc506fda0 msgr2=0x55edc507eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 -- 172.21.15.62:0/1945624294 shutdown_connections 2023-07-14T16:18:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 1 -- 172.21.15.62:0/1945624294 wait complete. 2023-07-14T16:18:04.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.843+0000 7f139ef79b80 20 client.16176 trim_cache size 0 max 16384 2023-07-14T16:18:04.829 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:18:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 -- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f13700a11d0 msgr2=0x7f13700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f13700a11d0 0x7f13700a3680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1384100530 tx=0x7f137c0099e0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 -- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1384100560 msgr2=0x7f13840ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1384100560 0x7f13840ffed0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f138000d950 tx=0x7f138000dd10 comp rx=0 tx=0).stop 2023-07-14T16:18:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 -- 172.21.15.62:0/1650381957 shutdown_connections 2023-07-14T16:18:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13841057a0 0x7f13840fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1384100560 0x7f13840ffed0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f13700a11d0 0x7f13700a3680 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 --2- 172.21.15.62:0/1650381957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1384102e80 0x7f13840fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 -- 172.21.15.62:0/1650381957 >> 172.21.15.62:0/1650381957 conn(0x7f13840fa000 msgr2=0x7f13841011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 -- 172.21.15.62:0/1650381957 shutdown_connections 2023-07-14T16:18:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:04.844+0000 7f138b5ff700 1 -- 172.21.15.62:0/1650381957 wait complete. 2023-07-14T16:18:04.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:04 smithi062 ceph-mon[95543]: pgmap v430: 97 pgs: 97 active+clean; 534 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:18:04.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:04 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3358919435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:18:04.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:04 smithi081 ceph-mon[127795]: pgmap v430: 97 pgs: 97 active+clean; 534 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:18:04.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:04 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3358919435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:18:05.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:04 smithi164 ceph-mon[97427]: pgmap v430: 97 pgs: 97 active+clean; 534 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:18:05.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:04 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3358919435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:18:05.841 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:18:05.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:18:06.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 -- 172.21.15.62:0/3392690706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc101630 msgr2=0x7f96dc107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 --2- 172.21.15.62:0/3392690706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc101630 0x7f96dc107ed0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f96d400b0a0 tx=0x7f96d400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 -- 172.21.15.62:0/3392690706 shutdown_connections 2023-07-14T16:18:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 --2- 172.21.15.62:0/3392690706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc101630 0x7f96dc107ed0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 --2- 172.21.15.62:0/3392690706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96dc0ff9f0 0x7f96dc0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 --2- 172.21.15.62:0/3392690706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96dc100c80 0x7f96dc1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 -- 172.21.15.62:0/3392690706 >> 172.21.15.62:0/3392690706 conn(0x7f96dc0fb010 msgr2=0x7f96dc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 -- 172.21.15.62:0/3392690706 shutdown_connections 2023-07-14T16:18:06.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.084+0000 7f96e199d700 1 -- 172.21.15.62:0/3392690706 wait complete. 2023-07-14T16:18:06.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.086+0000 7f96e199d700 1 Processor -- start 2023-07-14T16:18:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.086+0000 7f96e199d700 1 -- start start 2023-07-14T16:18:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.086+0000 7f96e199d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96dc0ff9f0 0x7f96dc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96e199d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc100c80 0x7f96dc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96e199d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96dc101630 0x7f96dc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96e199d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0735e0 con 0x7f96dc100c80 2023-07-14T16:18:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96e199d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc073750 con 0x7f96dc101630 2023-07-14T16:18:06.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96e199d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc19f660 con 0x7f96dc0ff9f0 2023-07-14T16:18:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc100c80 0x7f96dc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc100c80 0x7f96dc072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52618/0 (socket says 172.21.15.62:52618) 2023-07-14T16:18:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96dbfff700 1 -- 172.21.15.62:0/1559046281 learned_addr learned my addr 172.21.15.62:0/1559046281 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96e119c700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96dc101630 0x7f96dc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96e099b700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96dc0ff9f0 0x7f96dc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96dbfff700 1 -- 172.21.15.62:0/1559046281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96dc0ff9f0 msgr2=0x7f96dc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96dc0ff9f0 0x7f96dc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96dbfff700 1 -- 172.21.15.62:0/1559046281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96dc101630 msgr2=0x7f96dc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96dc101630 0x7f96dc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.087+0000 7f96dbfff700 1 -- 172.21.15.62:0/1559046281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009d00 con 0x7f96dc100c80 2023-07-14T16:18:06.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.088+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc100c80 0x7f96dc072900 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f96c800d950 tx=0x7f96c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.088+0000 7f96d9ffb700 1 -- 172.21.15.62:0/1559046281 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800f890 con 0x7f96dc100c80 2023-07-14T16:18:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.088+0000 7f96e199d700 1 -- 172.21.15.62:0/1559046281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc19f940 con 0x7f96dc100c80 2023-07-14T16:18:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.088+0000 7f96e199d700 1 -- 172.21.15.62:0/1559046281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc19fe70 con 0x7f96dc100c80 2023-07-14T16:18:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.088+0000 7f96d9ffb700 1 -- 172.21.15.62:0/1559046281 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96c800fe80 con 0x7f96dc100c80 2023-07-14T16:18:06.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.088+0000 7f96d9ffb700 1 -- 172.21.15.62:0/1559046281 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800e3b0 con 0x7f96dc100c80 2023-07-14T16:18:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.092+0000 7f96d9ffb700 1 -- 172.21.15.62:0/1559046281 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f96c800f9f0 con 0x7f96dc100c80 2023-07-14T16:18:06.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.093+0000 7f96f64c8b80 1 Processor -- start 2023-07-14T16:18:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96f64c8b80 1 -- start start 2023-07-14T16:18:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96d9ffb700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f96cc0a1270 0x7f96cc0a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96d9ffb700 1 -- 172.21.15.62:0/1559046281 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96c80cc8a0 con 0x7f96dc100c80 2023-07-14T16:18:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545d4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1544d8d30 con 0x55d1545d40f0 2023-07-14T16:18:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15452bac0 con 0x55d1545d37a0 2023-07-14T16:18:06.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d154524f50 con 0x55d1545d4eb0 2023-07-14T16:18:06.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545d4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96e119c700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f96cc0a1270 0x7f96cc0a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545d4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52628/0 (socket says 172.21.15.62:52628) 2023-07-14T16:18:06.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.094+0000 7f96dbfff700 1 -- 172.21.15.62:0/1005987035 learned_addr learned my addr 172.21.15.62:0/1005987035 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96e099b700 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96e119c700 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96dbfff700 1 -- 172.21.15.62:0/1005987035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 msgr2=0x55d1545d5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545d5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96dbfff700 1 -- 172.21.15.62:0/1005987035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 msgr2=0x55d1545d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96dbfff700 1 -- 172.21.15.62:0/1005987035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d154352e60 con 0x55d1545d40f0 2023-07-14T16:18:06.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545d4920 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f96c8096160 tx=0x7f96c800fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd3150a22c8d4c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96e119c700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f96cc0a1270 0x7f96cc0a3720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f96d400b830 tx=0x7f96d4006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96c6ffd700 1 -- 172.21.15.62:0/1005987035 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c80a20b0 con 0x55d1545d40f0 2023-07-14T16:18:06.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.095+0000 7f96c6ffd700 1 -- 172.21.15.62:0/1005987035 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96c8098000 con 0x55d1545d40f0 2023-07-14T16:18:06.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.096+0000 7f96c6ffd700 1 -- 172.21.15.62:0/1005987035 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8099400 con 0x55d1545d40f0 2023-07-14T16:18:06.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.096+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1005987035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 msgr2=0x55d1545d4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.096+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545d4920 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f96c8096160 tx=0x7f96c800fdd0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.096+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1005987035 shutdown_connections 2023-07-14T16:18:06.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.096+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545d5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.096+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.096+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1005987035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545d4920 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.096+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1005987035 >> 172.21.15.62:0/1005987035 conn(0x55d1545c9070 msgr2=0x55d1545ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.097+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1005987035 shutdown_connections 2023-07-14T16:18:06.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.097+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1005987035 wait complete. 2023-07-14T16:18:06.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.097+0000 7f96f64c8b80 1 -- start start 2023-07-14T16:18:06.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.097+0000 7f96f64c8b80 1 Processor -- start 2023-07-14T16:18:06.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545dd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545dd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1544c4b40 con 0x55d1545d40f0 2023-07-14T16:18:06.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1544c4840 con 0x55d1545d37a0 2023-07-14T16:18:06.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1543a89a0 con 0x55d1545d4eb0 2023-07-14T16:18:06.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96f64c8b80 10 client.0 _ll_register_callbacks cb 0x55d1544c27e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:06.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545dd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545dd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52636/0 (socket says 172.21.15.62:52636) 2023-07-14T16:18:06.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.098+0000 7f96dbfff700 1 -- 172.21.15.62:0/1991092210 learned_addr learned my addr 172.21.15.62:0/1991092210 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96e119c700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96e099b700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545dd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96dbfff700 1 -- 172.21.15.62:0/1991092210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 msgr2=0x55d1545e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545e48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96dbfff700 1 -- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 msgr2=0x55d1545dd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545dd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96dbfff700 1 -- 172.21.15.62:0/1991092210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc19fe70 con 0x55d1545d40f0 2023-07-14T16:18:06.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96e119c700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:06.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96e099b700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545dd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:06.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96dbfff700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545dd860 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f96c8010c70 tx=0x7f96c80a20b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d94114251c753 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:06.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96bbfff700 1 -- 172.21.15.62:0/1991092210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c80993c0 con 0x55d1545d40f0 2023-07-14T16:18:06.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96bbfff700 1 -- 172.21.15.62:0/1991092210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96c80a1c20 con 0x55d1545d40f0 2023-07-14T16:18:06.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.099+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1991092210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d15447e570 con 0x55d1545d40f0 2023-07-14T16:18:06.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96bbfff700 1 -- 172.21.15.62:0/1991092210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96c8099fc0 con 0x55d1545d40f0 2023-07-14T16:18:06.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96f64c8b80 10 client.16188 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:06.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1991092210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d15433bce0 con 0x55d1545d40f0 2023-07-14T16:18:06.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96bbfff700 1 -- 172.21.15.62:0/1991092210 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96c807c080 con 0x55d1545d40f0 2023-07-14T16:18:06.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96bbfff700 1 -- 172.21.15.62:0/1991092210 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f96c80b9310 con 0x55d1545d40f0 2023-07-14T16:18:06.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96f64c8b80 10 client.16188 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:06.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96f64c8b80 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:06.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96f64c8b80 20 client.16188 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:06.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d1545e8cc0 0x55d1545eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96f64c8b80 4 client.16188 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:06.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.100+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1991092210 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d15451df30 con 0x55d1545e8cc0 2023-07-14T16:18:06.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.101+0000 7f96e099b700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d1545e8cc0 0x55d1545eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.101+0000 7f96e099b700 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d1545e8cc0 0x55d1545eb0d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a8d92567ee4678 server_cookie=8d095a2f8ddcbc71 in_seq=0 out_seq=0 2023-07-14T16:18:06.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.101+0000 7f96bbfff700 10 client.16188 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:06.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96bbfff700 1 -- 172.21.15.62:0/1991092210 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d15451df30 con 0x55d1545e8cc0 2023-07-14T16:18:06.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96bbfff700 10 client.16188 handle_command_reply: tid=0 2023-07-14T16:18:06.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 client.16188 shutdown 2023-07-14T16:18:06.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1991092210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 msgr2=0x55d1545dd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545dd860 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f96c8010c70 tx=0x7f96c80a20b0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1991092210 shutdown_connections 2023-07-14T16:18:06.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d4eb0 0x55d1545e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d1545e8cc0 0x55d1545eb0d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d37a0 0x55d1545dd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/1991092210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545dd860 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96bbfff700 0 client.16188 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:06.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.102+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1991092210 >> 172.21.15.62:0/1991092210 conn(0x55d1545c9070 msgr2=0x55d1545cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.103+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1991092210 shutdown_connections 2023-07-14T16:18:06.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.103+0000 7f96f64c8b80 1 -- 172.21.15.62:0/1991092210 wait complete. 2023-07-14T16:18:06.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.103+0000 7f96f64c8b80 20 client.16188 trim_cache size 0 max 16384 2023-07-14T16:18:06.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.126+0000 7f96f64c8b80 1 Processor -- start 2023-07-14T16:18:06.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.126+0000 7f96f64c8b80 1 -- start start 2023-07-14T16:18:06.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.126+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d37a0 0x55d1545d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.126+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d40f0 0x55d1545d82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.126+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.126+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1545db050 con 0x55d1545d37a0 2023-07-14T16:18:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.126+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1545cf830 con 0x55d1545d4eb0 2023-07-14T16:18:06.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.126+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1545d4710 con 0x55d1545d40f0 2023-07-14T16:18:06.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d37a0 0x55d1545d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d37a0 0x55d1545d7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52650/0 (socket says 172.21.15.62:52650) 2023-07-14T16:18:06.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e099b700 1 -- 172.21.15.62:0/3318747474 learned_addr learned my addr 172.21.15.62:0/3318747474 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96dbfff700 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d40f0 0x55d1545d82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e119c700 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e099b700 1 -- 172.21.15.62:0/3318747474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d40f0 msgr2=0x55d1545d82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e099b700 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d40f0 0x55d1545d82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e099b700 1 -- 172.21.15.62:0/3318747474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 msgr2=0x55d1545e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e099b700 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e099b700 1 -- 172.21.15.62:0/3318747474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1545d5550 con 0x55d1545d37a0 2023-07-14T16:18:06.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.127+0000 7f96e119c700 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:06.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.128+0000 7f96e099b700 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d37a0 0x55d1545d7db0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55d1545eb980 tx=0x7f96d000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ab1d75b38af286 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.128+0000 7f96c77fe700 1 -- 172.21.15.62:0/3318747474 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d000bea0 con 0x55d1545d37a0 2023-07-14T16:18:06.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.128+0000 7f96c77fe700 1 -- 172.21.15.62:0/3318747474 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96d0017040 con 0x55d1545d37a0 2023-07-14T16:18:06.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.128+0000 7f96c77fe700 1 -- 172.21.15.62:0/3318747474 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0005020 con 0x55d1545d37a0 2023-07-14T16:18:06.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3318747474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d37a0 msgr2=0x55d1545d7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d37a0 0x55d1545d7db0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55d1545eb980 tx=0x7f96d000b410 comp rx=0 tx=0).stop 2023-07-14T16:18:06.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3318747474 shutdown_connections 2023-07-14T16:18:06.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d37a0 0x55d1545d7db0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3318747474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d40f0 0x55d1545d82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3318747474 >> 172.21.15.62:0/3318747474 conn(0x55d1545c9070 msgr2=0x55d1545dd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3318747474 shutdown_connections 2023-07-14T16:18:06.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3318747474 wait complete. 2023-07-14T16:18:06.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 -- start start 2023-07-14T16:18:06.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.129+0000 7f96f64c8b80 1 Processor -- start 2023-07-14T16:18:06.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d37a0 0x55d1545e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545e1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96f64c8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15463cb70 con 0x55d1545d40f0 2023-07-14T16:18:06.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15463ccb0 con 0x55d1545d4eb0 2023-07-14T16:18:06.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96f64c8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15463ce40 con 0x55d1545d37a0 2023-07-14T16:18:06.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96f64c8b80 10 client.0 _ll_register_callbacks cb 0x7f96d000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:06.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545e1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52656/0 (socket says 172.21.15.62:52656) 2023-07-14T16:18:06.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96dbfff700 1 -- 172.21.15.62:0/3859561021 learned_addr learned my addr 172.21.15.62:0/3859561021 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96e099b700 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d37a0 0x55d1545e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.130+0000 7f96e119c700 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96dbfff700 1 -- 172.21.15.62:0/3859561021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d37a0 msgr2=0x55d1545e4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96dbfff700 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d37a0 0x55d1545e4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96dbfff700 1 -- 172.21.15.62:0/3859561021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 msgr2=0x55d1545e19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96dbfff700 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96dbfff700 1 -- 172.21.15.62:0/3859561021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d00042c0 con 0x55d1545d40f0 2023-07-14T16:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96dbfff700 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545e1330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f96c8099960 tx=0x7f96c8098df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa29ed22a4b7393 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:06.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96bbfff700 1 -- 172.21.15.62:0/3859561021 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c80ca1d0 con 0x55d1545d40f0 2023-07-14T16:18:06.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96bbfff700 1 -- 172.21.15.62:0/3859561021 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96c8099ec0 con 0x55d1545d40f0 2023-07-14T16:18:06.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.131+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3859561021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96c8098000 con 0x55d1545d40f0 2023-07-14T16:18:06.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.132+0000 7f96bbfff700 1 -- 172.21.15.62:0/3859561021 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96c8098ee0 con 0x55d1545d40f0 2023-07-14T16:18:06.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.132+0000 7f96f64c8b80 10 client.16194 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:06.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.132+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3859561021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d1543792d0 con 0x55d1545d40f0 2023-07-14T16:18:06.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.132+0000 7f96bbfff700 1 -- 172.21.15.62:0/3859561021 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96c80bd020 con 0x55d1545d40f0 2023-07-14T16:18:06.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.132+0000 7f96bbfff700 1 -- 172.21.15.62:0/3859561021 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f96c8099150 con 0x55d1545d40f0 2023-07-14T16:18:06.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.132+0000 7f96f64c8b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:06.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.132+0000 7f96f64c8b80 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:06.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.132+0000 7f96f64c8b80 20 client.16194 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:06.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.133+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d15463e2e0 0x55d1546406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.133+0000 7f96f64c8b80 4 client.16194 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:18:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.133+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3859561021 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d1544e00d0 con 0x55d15463e2e0 2023-07-14T16:18:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.133+0000 7f96e099b700 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d15463e2e0 0x55d1546406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.134+0000 7f96bbfff700 10 client.16194 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.134+0000 7f96e099b700 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d15463e2e0 0x55d1546406c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3217d6f82de5c25e server_cookie=5a8e1eb1e6cb58f0 in_seq=0 out_seq=0 2023-07-14T16:18:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96bbfff700 1 -- 172.21.15.62:0/3859561021 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d1544e00d0 con 0x55d15463e2e0 2023-07-14T16:18:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96bbfff700 10 client.16194 handle_command_reply: tid=0 2023-07-14T16:18:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 client.16194 shutdown 2023-07-14T16:18:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3859561021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 msgr2=0x55d1545e1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545e1330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f96c8099960 tx=0x7f96c8098df0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3859561021 shutdown_connections 2023-07-14T16:18:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d15463e2e0 0x55d1546406c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1545d4eb0 0x55d1545e19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1545d37a0 0x55d1545e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 --2- 172.21.15.62:0/3859561021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1545d40f0 0x55d1545e1330 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3859561021 >> 172.21.15.62:0/3859561021 conn(0x55d1545c9070 msgr2=0x55d1545dd210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96bbfff700 0 client.16194 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:06.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3859561021 shutdown_connections 2023-07-14T16:18:06.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 1 -- 172.21.15.62:0/3859561021 wait complete. 2023-07-14T16:18:06.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.135+0000 7f96f64c8b80 20 client.16194 trim_cache size 0 max 16384 2023-07-14T16:18:06.124 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:18:06.124 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:18:06.124 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "71a2ac63-fde8-440c-913b-a6ae8757a759", 2023-07-14T16:18:06.124 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:18:06.124 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:18:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 -- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f96cc0a1270 msgr2=0x7f96cc0a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f96cc0a1270 0x7f96cc0a3720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f96d400b830 tx=0x7f96d4006000 comp rx=0 tx=0).stop 2023-07-14T16:18:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 -- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc100c80 msgr2=0x7f96dc072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc100c80 0x7f96dc072900 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f96c800d950 tx=0x7f96c800dc60 comp rx=0 tx=0).stop 2023-07-14T16:18:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 -- 172.21.15.62:0/1559046281 shutdown_connections 2023-07-14T16:18:06.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f96cc0a1270 0x7f96cc0a3720 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96dc101630 0x7f96dc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96dc0ff9f0 0x7f96dc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 --2- 172.21.15.62:0/1559046281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96dc100c80 0x7f96dc072900 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 -- 172.21.15.62:0/1559046281 >> 172.21.15.62:0/1559046281 conn(0x7f96dc0fb010 msgr2=0x7f96dc0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 -- 172.21.15.62:0/1559046281 shutdown_connections 2023-07-14T16:18:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.136+0000 7f96e199d700 1 -- 172.21.15.62:0/1559046281 wait complete. 2023-07-14T16:18:06.137 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:18:06.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 -- 172.21.15.62:0/2453858491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd770100ce0 msgr2=0x7fd770101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 --2- 172.21.15.62:0/2453858491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd770100ce0 0x7fd770101130 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd760009990 tx=0x7fd760009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 -- 172.21.15.62:0/2453858491 shutdown_connections 2023-07-14T16:18:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 --2- 172.21.15.62:0/2453858491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd770101670 0x7fd770107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 --2- 172.21.15.62:0/2453858491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7700ffaf0 0x7fd7700fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 --2- 172.21.15.62:0/2453858491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd770100ce0 0x7fd770101130 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 -- 172.21.15.62:0/2453858491 >> 172.21.15.62:0/2453858491 conn(0x7fd7700fb130 msgr2=0x7fd7700fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 -- 172.21.15.62:0/2453858491 shutdown_connections 2023-07-14T16:18:06.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.398+0000 7fd7775ff700 1 -- 172.21.15.62:0/2453858491 wait complete. 2023-07-14T16:18:06.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.399+0000 7fd7775ff700 1 Processor -- start 2023-07-14T16:18:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.399+0000 7fd7775ff700 1 -- start start 2023-07-14T16:18:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd7775ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7700ffaf0 0x7fd770195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd7775ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd770100ce0 0x7fd770195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd7775ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd770101670 0x7fd77019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd7775ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77019d2e0 con 0x7fd7700ffaf0 2023-07-14T16:18:06.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd7775ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77019d420 con 0x7fd770100ce0 2023-07-14T16:18:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd7775ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77019d5b0 con 0x7fd770101670 2023-07-14T16:18:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd77539b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7700ffaf0 0x7fd770195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd77539b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7700ffaf0 0x7fd770195760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52678/0 (socket says 172.21.15.62:52678) 2023-07-14T16:18:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd77539b700 1 -- 172.21.15.62:0/3266523748 learned_addr learned my addr 172.21.15.62:0/3266523748 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd774b9a700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd770100ce0 0x7fd770195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.400+0000 7fd775b9c700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd770101670 0x7fd77019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd77539b700 1 -- 172.21.15.62:0/3266523748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd770101670 msgr2=0x7fd77019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd77539b700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd770101670 0x7fd77019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd77539b700 1 -- 172.21.15.62:0/3266523748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd770100ce0 msgr2=0x7fd770195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd77539b700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd770100ce0 0x7fd770195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd77539b700 1 -- 172.21.15.62:0/3266523748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600095f0 con 0x7fd7700ffaf0 2023-07-14T16:18:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd775b9c700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd770101670 0x7fd77019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd77539b700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7700ffaf0 0x7fd770195760 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00d980 tx=0x7fd76c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd7667fc700 1 -- 172.21.15.62:0/3266523748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c004020 con 0x7fd7700ffaf0 2023-07-14T16:18:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd7775ff700 1 -- 172.21.15.62:0/3266523748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77019d890 con 0x7fd7700ffaf0 2023-07-14T16:18:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd7667fc700 1 -- 172.21.15.62:0/3266523748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd76c009730 con 0x7fd7700ffaf0 2023-07-14T16:18:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd7667fc700 1 -- 172.21.15.62:0/3266523748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c003a00 con 0x7fd7700ffaf0 2023-07-14T16:18:06.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.401+0000 7fd7775ff700 1 -- 172.21.15.62:0/3266523748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77019dda0 con 0x7fd7700ffaf0 2023-07-14T16:18:06.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.406+0000 7fd7667fc700 1 -- 172.21.15.62:0/3266523748 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd76c0098a0 con 0x7fd7700ffaf0 2023-07-14T16:18:06.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.406+0000 7fd78af63b80 1 Processor -- start 2023-07-14T16:18:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.406+0000 7fd78af63b80 1 -- start start 2023-07-14T16:18:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960d8e0 0x55f16960dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960e230 0x55f16960eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16960f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd78af63b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16950ecb0 con 0x55f16960e230 2023-07-14T16:18:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd78af63b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169561a40 con 0x55f16960d8e0 2023-07-14T16:18:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd78af63b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16955aed0 con 0x55f16960f080 2023-07-14T16:18:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd77539b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960e230 0x55f16960eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd77539b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960e230 0x55f16960eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52688/0 (socket says 172.21.15.62:52688) 2023-07-14T16:18:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd77539b700 1 -- 172.21.15.62:0/3133275582 learned_addr learned my addr 172.21.15.62:0/3133275582 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd774b9a700 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16960f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.407+0000 7fd775b9c700 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960d8e0 0x55f16960dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd7667fc700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd75c0a1150 0x7fd75c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd774b9a700 1 -- 172.21.15.62:0/3133275582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960d8e0 msgr2=0x55f16960dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd774b9a700 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960d8e0 0x55f16960dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd774b9a700 1 -- 172.21.15.62:0/3133275582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960e230 msgr2=0x55f16960eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd774b9a700 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960e230 0x55f16960eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd7667fc700 1 -- 172.21.15.62:0/3266523748 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd76c0cc690 con 0x7fd7700ffaf0 2023-07-14T16:18:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd774b9a700 1 -- 172.21.15.62:0/3133275582 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f169388de0 con 0x55f16960f080 2023-07-14T16:18:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd775b9c700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd75c0a1150 0x7fd75c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.408+0000 7fd774b9a700 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16960f4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd7600059d0 tx=0x7fd760009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b1e4e3cf8cd8ac1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.409+0000 7fd75b7fe700 1 -- 172.21.15.62:0/3133275582 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76001d070 con 0x55f16960f080 2023-07-14T16:18:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.409+0000 7fd75b7fe700 1 -- 172.21.15.62:0/3133275582 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd76001e040 con 0x55f16960f080 2023-07-14T16:18:06.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.409+0000 7fd75b7fe700 1 -- 172.21.15.62:0/3133275582 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760004500 con 0x55f16960f080 2023-07-14T16:18:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.409+0000 7fd78af63b80 1 -- 172.21.15.62:0/3133275582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 msgr2=0x55f16960f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.409+0000 7fd775b9c700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd75c0a1150 0x7fd75c0a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd768009ef0 tx=0x7fd768004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.409+0000 7fd78af63b80 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16960f4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd7600059d0 tx=0x7fd760009ee0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.410+0000 7fd78af63b80 1 -- 172.21.15.62:0/3133275582 shutdown_connections 2023-07-14T16:18:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.410+0000 7fd78af63b80 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16960f4f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.410+0000 7fd78af63b80 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960d8e0 0x55f16960dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.410+0000 7fd78af63b80 1 --2- 172.21.15.62:0/3133275582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960e230 0x55f16960eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.410+0000 7fd78af63b80 1 -- 172.21.15.62:0/3133275582 >> 172.21.15.62:0/3133275582 conn(0x55f1695feda0 msgr2=0x55f1696041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.410+0000 7fd78af63b80 1 -- 172.21.15.62:0/3133275582 shutdown_connections 2023-07-14T16:18:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.410+0000 7fd78af63b80 1 -- 172.21.15.62:0/3133275582 wait complete. 2023-07-14T16:18:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.410+0000 7fd78af63b80 1 -- start start 2023-07-14T16:18:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.411+0000 7fd78af63b80 1 Processor -- start 2023-07-14T16:18:06.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.411+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f1696174a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.411+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 0x55f1696179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.411+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16961ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.411+0000 7fd78af63b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1694faac0 con 0x55f16960d8e0 2023-07-14T16:18:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.411+0000 7fd78af63b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1694fa7c0 con 0x55f16960e230 2023-07-14T16:18:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.411+0000 7fd78af63b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1693de920 con 0x55f16960f080 2023-07-14T16:18:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.411+0000 7fd78af63b80 10 client.0 _ll_register_callbacks cb 0x55f1694f8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:06.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f1696174a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f1696174a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52698/0 (socket says 172.21.15.62:52698) 2023-07-14T16:18:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 -- 172.21.15.62:0/229538152 learned_addr learned my addr 172.21.15.62:0/229538152 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd775b9c700 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16961ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd774b9a700 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 0x55f1696179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 -- 172.21.15.62:0/229538152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 msgr2=0x55f16961ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16961ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 -- 172.21.15.62:0/229538152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 msgr2=0x55f1696179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 0x55f1696179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 -- 172.21.15.62:0/229538152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd770108860 con 0x55f16960d8e0 2023-07-14T16:18:06.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd77539b700 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f1696174a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0046a0 tx=0x7fd76c0ca100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c35abf19e84c15 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.412+0000 7fd758ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd758ff9700 1 -- 172.21.15.62:0/229538152 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c099140 con 0x55f16960d8e0 2023-07-14T16:18:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd758ff9700 1 -- 172.21.15.62:0/229538152 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd76c009b80 con 0x55f16960d8e0 2023-07-14T16:18:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 1 -- 172.21.15.62:0/229538152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1694b44f0 con 0x55f16960d8e0 2023-07-14T16:18:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd758ff9700 1 -- 172.21.15.62:0/229538152 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd76c099590 con 0x55f16960d8e0 2023-07-14T16:18:06.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 10 client.16206 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 1 -- 172.21.15.62:0/229538152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f169371c60 con 0x55f16960d8e0 2023-07-14T16:18:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd758ff9700 1 -- 172.21.15.62:0/229538152 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd76c0b6080 con 0x55f16960d8e0 2023-07-14T16:18:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd758ff9700 1 -- 172.21.15.62:0/229538152 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd76c0a1a10 con 0x55f16960d8e0 2023-07-14T16:18:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 10 client.16206 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:06.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 20 client.16206 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169623200 0x55f169625610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 4 client.16206 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.413+0000 7fd78af63b80 1 -- 172.21.15.62:0/229538152 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f169553eb0 con 0x55f169623200 2023-07-14T16:18:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.414+0000 7fd774b9a700 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169623200 0x55f169625610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.414+0000 7fd774b9a700 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169623200 0x55f169625610 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d409237ff7f5908 server_cookie=f6317f0c7e55edc8 in_seq=0 out_seq=0 2023-07-14T16:18:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.414+0000 7fd758ff9700 10 client.16206 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.415+0000 7fd758ff9700 1 -- 172.21.15.62:0/229538152 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f169553eb0 con 0x55f169623200 2023-07-14T16:18:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.415+0000 7fd758ff9700 10 client.16206 handle_command_reply: tid=0 2023-07-14T16:18:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.415+0000 7fd78af63b80 1 client.16206 shutdown 2023-07-14T16:18:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.415+0000 7fd78af63b80 1 -- 172.21.15.62:0/229538152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 msgr2=0x55f1696174a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.415+0000 7fd78af63b80 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f1696174a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0046a0 tx=0x7fd76c0ca100 comp rx=0 tx=0).stop 2023-07-14T16:18:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 1 -- 172.21.15.62:0/229538152 shutdown_connections 2023-07-14T16:18:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960f080 0x55f16961ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f1696174a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169623200 0x55f169625610 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 1 --2- 172.21.15.62:0/229538152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 0x55f1696179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd758ff9700 0 client.16206 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:06.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 1 -- 172.21.15.62:0/229538152 >> 172.21.15.62:0/229538152 conn(0x55f1695feda0 msgr2=0x55f1696038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 1 -- 172.21.15.62:0/229538152 shutdown_connections 2023-07-14T16:18:06.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 1 -- 172.21.15.62:0/229538152 wait complete. 2023-07-14T16:18:06.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.416+0000 7fd78af63b80 20 client.16206 trim_cache size 0 max 16384 2023-07-14T16:18:06.404 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:18:06.147+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v93>,<0x10000000003/client.0 v94>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v105>,<0x10000000001/sv_0 v120>,<0x10000000000/_nogroup v113>,<0x1/volumes v102>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:18:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.435+0000 7fd78af63b80 1 Processor -- start 2023-07-14T16:18:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd78af63b80 1 -- start start 2023-07-14T16:18:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f169616200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960e230 0x55f16961e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960f080 0x55f16961f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd78af63b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169605d30 con 0x55f16960d8e0 2023-07-14T16:18:06.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd78af63b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169622f50 con 0x55f16960f080 2023-07-14T16:18:06.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd78af63b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169615f30 con 0x55f16960e230 2023-07-14T16:18:06.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd77539b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f169616200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd77539b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f169616200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52700/0 (socket says 172.21.15.62:52700) 2023-07-14T16:18:06.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.436+0000 7fd77539b700 1 -- 172.21.15.62:0/1689644156 learned_addr learned my addr 172.21.15.62:0/1689644156 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd774b9a700 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960e230 0x55f16961e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd775b9c700 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960f080 0x55f16961f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd77539b700 1 -- 172.21.15.62:0/1689644156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960e230 msgr2=0x55f16961e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd77539b700 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960e230 0x55f16961e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd77539b700 1 -- 172.21.15.62:0/1689644156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960f080 msgr2=0x55f16961f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd77539b700 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960f080 0x55f16961f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd77539b700 1 -- 172.21.15.62:0/1689644156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1693af250 con 0x55f16960d8e0 2023-07-14T16:18:06.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd774b9a700 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960e230 0x55f16961e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:06.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd77539b700 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f169616200 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0999e0 tx=0x7fd76c00d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d246a7f3f061d0b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd75bfff700 1 -- 172.21.15.62:0/1689644156 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0a1c10 con 0x55f16960d8e0 2023-07-14T16:18:06.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd75bfff700 1 -- 172.21.15.62:0/1689644156 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f1694b44f0 con 0x55f16960d8e0 2023-07-14T16:18:06.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.437+0000 7fd75bfff700 1 -- 172.21.15.62:0/1689644156 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c099ae0 con 0x55f16960d8e0 2023-07-14T16:18:06.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 -- 172.21.15.62:0/1689644156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 msgr2=0x55f169616200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f169616200 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0999e0 tx=0x7fd76c00d8d0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 -- 172.21.15.62:0/1689644156 shutdown_connections 2023-07-14T16:18:06.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960f080 0x55f16961f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960d8e0 0x55f169616200 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 --2- 172.21.15.62:0/1689644156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960e230 0x55f16961e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 -- 172.21.15.62:0/1689644156 >> 172.21.15.62:0/1689644156 conn(0x55f1695feda0 msgr2=0x55f169629f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 -- 172.21.15.62:0/1689644156 shutdown_connections 2023-07-14T16:18:06.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.438+0000 7fd78af63b80 1 -- 172.21.15.62:0/1689644156 wait complete. 2023-07-14T16:18:06.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.439+0000 7fd78af63b80 1 -- start start 2023-07-14T16:18:06.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.439+0000 7fd78af63b80 1 Processor -- start 2023-07-14T16:18:06.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.439+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960d8e0 0x55f16961e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 0x55f16961b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd78af63b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960f080 0x55f16961b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd78af63b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16961bc10 con 0x55f16960f080 2023-07-14T16:18:06.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd78af63b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16961bd80 con 0x55f16960e230 2023-07-14T16:18:06.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd78af63b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1696770b0 con 0x55f16960d8e0 2023-07-14T16:18:06.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd78af63b80 10 client.0 _ll_register_callbacks cb 0x7fd76c004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:06.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd775b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960f080 0x55f16961b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd775b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960f080 0x55f16961b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52712/0 (socket says 172.21.15.62:52712) 2023-07-14T16:18:06.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd775b9c700 1 -- 172.21.15.62:0/451460612 learned_addr learned my addr 172.21.15.62:0/451460612 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:06.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd774b9a700 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 0x55f16961b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd77539b700 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960d8e0 0x55f16961e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd775b9c700 1 -- 172.21.15.62:0/451460612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960d8e0 msgr2=0x55f16961e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd775b9c700 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960d8e0 0x55f16961e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd775b9c700 1 -- 172.21.15.62:0/451460612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 msgr2=0x55f16961b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd775b9c700 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 0x55f16961b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.440+0000 7fd775b9c700 1 -- 172.21.15.62:0/451460612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f169371c60 con 0x55f16960f080 2023-07-14T16:18:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.441+0000 7fd775b9c700 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960f080 0x55f16961b6d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd768017ca0 tx=0x7fd768009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae227fe9fd9878a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.441+0000 7fd758ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.441+0000 7fd758ff9700 1 -- 172.21.15.62:0/451460612 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768016c80 con 0x55f16960f080 2023-07-14T16:18:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.441+0000 7fd758ff9700 1 -- 172.21.15.62:0/451460612 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd768016de0 con 0x55f16960f080 2023-07-14T16:18:06.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.441+0000 7fd78af63b80 1 -- 172.21.15.62:0/451460612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd76001e040 con 0x55f16960f080 2023-07-14T16:18:06.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.441+0000 7fd758ff9700 1 -- 172.21.15.62:0/451460612 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd768025470 con 0x55f16960f080 2023-07-14T16:18:06.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.441+0000 7fd78af63b80 10 client.16212 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:06.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.441+0000 7fd78af63b80 1 -- 172.21.15.62:0/451460612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f1693872b0 con 0x55f16960f080 2023-07-14T16:18:06.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.442+0000 7fd758ff9700 1 -- 172.21.15.62:0/451460612 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd76001e040 con 0x55f16960f080 2023-07-14T16:18:06.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.442+0000 7fd758ff9700 1 -- 172.21.15.62:0/451460612 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd768024560 con 0x55f16960f080 2023-07-14T16:18:06.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.442+0000 7fd78af63b80 10 client.16212 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:06.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.442+0000 7fd78af63b80 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:06.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.442+0000 7fd78af63b80 20 client.16212 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:06.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.442+0000 7fd78af63b80 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169678b70 0x55f16967af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:06.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.442+0000 7fd78af63b80 4 client.16212 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:18:06.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.442+0000 7fd78af63b80 1 -- 172.21.15.62:0/451460612 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f169516050 con 0x55f169678b70 2023-07-14T16:18:06.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.443+0000 7fd77539b700 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169678b70 0x55f16967af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:06.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.443+0000 7fd77539b700 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169678b70 0x55f16967af80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a169041765b1a9 server_cookie=4494508bb817f7c4 in_seq=0 out_seq=0 2023-07-14T16:18:06.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.443+0000 7fd758ff9700 10 client.16212 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:06.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.443+0000 7fd758ff9700 1 -- 172.21.15.62:0/451460612 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f169516050 con 0x55f169678b70 2023-07-14T16:18:06.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.443+0000 7fd758ff9700 10 client.16212 handle_command_reply: tid=0 2023-07-14T16:18:06.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.443+0000 7fd78af63b80 1 client.16212 shutdown 2023-07-14T16:18:06.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.443+0000 7fd78af63b80 1 -- 172.21.15.62:0/451460612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960f080 msgr2=0x55f16961b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.443+0000 7fd78af63b80 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960f080 0x55f16961b6d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd768017ca0 tx=0x7fd768009040 comp rx=0 tx=0).stop 2023-07-14T16:18:06.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 1 -- 172.21.15.62:0/451460612 shutdown_connections 2023-07-14T16:18:06.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f16960f080 0x55f16961b6d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f16960d8e0 0x55f16961e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169678b70 0x55f16967af80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 1 --2- 172.21.15.62:0/451460612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f16960e230 0x55f16961b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd758ff9700 0 client.16212 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:06.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 1 -- 172.21.15.62:0/451460612 >> 172.21.15.62:0/451460612 conn(0x55f1695feda0 msgr2=0x55f169616200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 1 -- 172.21.15.62:0/451460612 shutdown_connections 2023-07-14T16:18:06.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 1 -- 172.21.15.62:0/451460612 wait complete. 2023-07-14T16:18:06.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.444+0000 7fd78af63b80 20 client.16212 trim_cache size 0 max 16384 2023-07-14T16:18:06.430 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:18:06.430 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:18:06.430 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:18:06.431 INFO:teuthology.orchestra.run.smithi062.stdout: "71a2ac63-fde8-440c-913b-a6ae8757a759": { 2023-07-14T16:18:06.431 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:18:06.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "71a2ac63-fde8-440c-913b-a6ae8757a759", 2023-07-14T16:18:06.431 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:18:06.431 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:18:06.431 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:18:06.431 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:18:06.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 -- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd75c0a1150 msgr2=0x7fd75c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd75c0a1150 0x7fd75c0a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd768009ef0 tx=0x7fd768004ad0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 -- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7700ffaf0 msgr2=0x7fd770195760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:06.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7700ffaf0 0x7fd770195760 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00d980 tx=0x7fd76c00dc90 comp rx=0 tx=0).stop 2023-07-14T16:18:06.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 -- 172.21.15.62:0/3266523748 shutdown_connections 2023-07-14T16:18:06.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd770101670 0x7fd77019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7700ffaf0 0x7fd770195760 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd75c0a1150 0x7fd75c0a3600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 --2- 172.21.15.62:0/3266523748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd770100ce0 0x7fd770195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:06.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 -- 172.21.15.62:0/3266523748 >> 172.21.15.62:0/3266523748 conn(0x7fd7700fb130 msgr2=0x7fd7700fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:06.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 -- 172.21.15.62:0/3266523748 shutdown_connections 2023-07-14T16:18:06.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:06.445+0000 7fd7775ff700 1 -- 172.21.15.62:0/3266523748 wait complete. 2023-07-14T16:18:06.443 INFO:tasks.cephfs.filesystem:scrub status for tag:71a2ac63-fde8-440c-913b-a6ae8757a759 - {'path': '/', 'tag': '71a2ac63-fde8-440c-913b-a6ae8757a759', 'options': 'recursive,force'} 2023-07-14T16:18:06.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: pgmap v431: 97 pgs: 97 active+clean; 542 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:18:06.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:06.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:06.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:06.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:06.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:06.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:06 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: pgmap v431: 97 pgs: 97 active+clean; 542 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:18:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:06.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:06.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:06.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:06.905 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:06 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: pgmap v431: 97 pgs: 97 active+clean; 542 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:18:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:07.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:07.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:07.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:07.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:07.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:06 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:07 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:18:07.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:07 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:07 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:07.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:07 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:18:07.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:07 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:07 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:18:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:07 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:07.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:07 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:07.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:07 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:18:07.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:07 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:07 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:18:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:07 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:08.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:07 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:08.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:07 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:18:08.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:07 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:08.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:08 smithi062 ceph-mon[95543]: pgmap v432: 97 pgs: 97 active+clean; 582 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 19 op/s 2023-07-14T16:18:08.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:08 smithi081 ceph-mon[127795]: pgmap v432: 97 pgs: 97 active+clean; 582 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 19 op/s 2023-07-14T16:18:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:08 smithi164 ceph-mon[97427]: pgmap v432: 97 pgs: 97 active+clean; 582 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 19 op/s 2023-07-14T16:18:10.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:10 smithi062 ceph-mon[95543]: pgmap v433: 97 pgs: 97 active+clean; 622 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:18:10.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:10 smithi081 ceph-mon[127795]: pgmap v433: 97 pgs: 97 active+clean; 622 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:18:11.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:10 smithi164 ceph-mon[97427]: pgmap v433: 97 pgs: 97 active+clean; 622 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:18:12.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:12 smithi081 ceph-mon[127795]: pgmap v434: 97 pgs: 97 active+clean; 642 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:18:13.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:12 smithi164 ceph-mon[97427]: pgmap v434: 97 pgs: 97 active+clean; 642 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:18:13.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:12 smithi062 ceph-mon[95543]: pgmap v434: 97 pgs: 97 active+clean; 642 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:18:13.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:13 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:18:13.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:13 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:18:14.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:13 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:18:14.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:13 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:18:14.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:13 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:18:14.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:13 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:18:14.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:14 smithi081 ceph-mon[127795]: pgmap v435: 97 pgs: 97 active+clean; 666 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:18:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:14 smithi164 ceph-mon[97427]: pgmap v435: 97 pgs: 97 active+clean; 666 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:18:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:14 smithi062 ceph-mon[95543]: pgmap v435: 97 pgs: 97 active+clean; 666 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:18:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 -- 172.21.15.62:0/2063605034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 msgr2=0x7f4200106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 --2- 172.21.15.62:0/2063605034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 0x7f4200106560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098f0 tx=0x7f41f0009c00 comp rx=0 tx=0).stop 2023-07-14T16:18:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 -- 172.21.15.62:0/2063605034 shutdown_connections 2023-07-14T16:18:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 --2- 172.21.15.62:0/2063605034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200107cd0 0x7f420010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 --2- 172.21.15.62:0/2063605034 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 0x7f4200106560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 --2- 172.21.15.62:0/2063605034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200107340 0x7f4200107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 -- 172.21.15.62:0/2063605034 >> 172.21.15.62:0/2063605034 conn(0x7f4200075190 msgr2=0x7f42000775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 -- 172.21.15.62:0/2063605034 shutdown_connections 2023-07-14T16:18:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.239+0000 7f4205bff700 1 -- 172.21.15.62:0/2063605034 wait complete. 2023-07-14T16:18:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.240+0000 7f4205bff700 1 Processor -- start 2023-07-14T16:18:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.240+0000 7f4205bff700 1 -- start start 2023-07-14T16:18:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f4205bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 0x7f420019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f4205bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200107340 0x7f420019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f4205bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200107cd0 0x7f420019d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f4205bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420019d900 con 0x7f4200107cd0 2023-07-14T16:18:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f4205bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420019da70 con 0x7f4200106150 2023-07-14T16:18:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f4205bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420019dc30 con 0x7f4200107340 2023-07-14T16:18:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f41ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 0x7f420019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f41fffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200107cd0 0x7f420019d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f41ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 0x7f420019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51606/0 (socket says 172.21.15.62:51606) 2023-07-14T16:18:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f41ff7fe700 1 -- 172.21.15.62:0/2287565224 learned_addr learned my addr 172.21.15.62:0/2287565224 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f41feffd700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200107340 0x7f420019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f41ff7fe700 1 -- 172.21.15.62:0/2287565224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200107340 msgr2=0x7f420019c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.241+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200107340 0x7f420019c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f41ff7fe700 1 -- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200107cd0 msgr2=0x7f420019d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200107cd0 0x7f420019d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f41ff7fe700 1 -- 172.21.15.62:0/2287565224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40095f0 con 0x7f4200106150 2023-07-14T16:18:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 0x7f420019bee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009fd0 tx=0x7f41f000bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f41fcff9700 1 -- 172.21.15.62:0/2287565224 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f001c070 con 0x7f4200106150 2023-07-14T16:18:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f4205bff700 1 -- 172.21.15.62:0/2287565224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f0009520 con 0x7f4200106150 2023-07-14T16:18:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f41fcff9700 1 -- 172.21.15.62:0/2287565224 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41f000bb10 con 0x7f4200106150 2023-07-14T16:18:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f41fcff9700 1 -- 172.21.15.62:0/2287565224 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00038c0 con 0x7f4200106150 2023-07-14T16:18:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.242+0000 7f4205bff700 1 -- 172.21.15.62:0/2287565224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42001a44f0 con 0x7f4200106150 2023-07-14T16:18:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.246+0000 7f41fcff9700 1 -- 172.21.15.62:0/2287565224 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f41f0020030 con 0x7f4200106150 2023-07-14T16:18:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.247+0000 7f41fcff9700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1150 0x7f41e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f41fcff9700 1 -- 172.21.15.62:0/2287565224 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41f00cd9f0 con 0x7f4200106150 2023-07-14T16:18:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f42195f4b80 1 Processor -- start 2023-07-14T16:18:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f41feffd700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1150 0x7f41e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f42195f4b80 1 -- start start 2023-07-14T16:18:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea826300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea826840 0x5630ea8271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea8276e0 0x5630ea827b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f42195f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea72bc50 con 0x5630ea8276e0 2023-07-14T16:18:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f42195f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea77e9e0 con 0x5630ea826840 2023-07-14T16:18:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.248+0000 7f42195f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea777e70 con 0x5630ea6826b0 2023-07-14T16:18:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41ff7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea826300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea826840 0x5630ea8271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41ff7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea826300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52446/0 (socket says 172.21.15.62:52446) 2023-07-14T16:18:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41ff7fe700 1 -- 172.21.15.62:0/4277129731 learned_addr learned my addr 172.21.15.62:0/4277129731 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41fffff700 1 --2- 172.21.15.62:0/4277129731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea8276e0 0x5630ea827b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41feffd700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1150 0x7f41e80a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005fd0 tx=0x7f41f4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41ff7fe700 1 -- 172.21.15.62:0/4277129731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea826840 msgr2=0x5630ea8271a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/4277129731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea826840 0x5630ea8271a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41ff7fe700 1 -- 172.21.15.62:0/4277129731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea8276e0 msgr2=0x5630ea827b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/4277129731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea8276e0 0x5630ea827b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.249+0000 7f41ff7fe700 1 -- 172.21.15.62:0/4277129731 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ea5a5d80 con 0x5630ea6826b0 2023-07-14T16:18:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.250+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/4277129731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea826300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f41f00a2cd0 tx=0x7f41f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c962151969e0ba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.250+0000 7f41e5ffb700 1 -- 172.21.15.62:0/4277129731 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00b2070 con 0x5630ea6826b0 2023-07-14T16:18:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.250+0000 7f41e5ffb700 1 -- 172.21.15.62:0/4277129731 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41f00a3200 con 0x5630ea6826b0 2023-07-14T16:18:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.250+0000 7f41e5ffb700 1 -- 172.21.15.62:0/4277129731 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0005260 con 0x5630ea6826b0 2023-07-14T16:18:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 -- 172.21.15.62:0/4277129731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 msgr2=0x5630ea826300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 --2- 172.21.15.62:0/4277129731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea826300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f41f00a2cd0 tx=0x7f41f0009c00 comp rx=0 tx=0).stop 2023-07-14T16:18:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 -- 172.21.15.62:0/4277129731 shutdown_connections 2023-07-14T16:18:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 --2- 172.21.15.62:0/4277129731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea8276e0 0x5630ea827b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 --2- 172.21.15.62:0/4277129731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea826300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 --2- 172.21.15.62:0/4277129731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea826840 0x5630ea8271a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 -- 172.21.15.62:0/4277129731 >> 172.21.15.62:0/4277129731 conn(0x5630ea81bbe0 msgr2=0x5630ea820cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 -- 172.21.15.62:0/4277129731 shutdown_connections 2023-07-14T16:18:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.251+0000 7f42195f4b80 1 -- 172.21.15.62:0/4277129731 wait complete. 2023-07-14T16:18:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.252+0000 7f42195f4b80 1 -- start start 2023-07-14T16:18:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.252+0000 7f42195f4b80 1 Processor -- start 2023-07-14T16:18:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.252+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea82f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.252+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea82fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.252+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea836a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.252+0000 7f42195f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea717a60 con 0x5630ea826840 2023-07-14T16:18:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.252+0000 7f42195f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea717760 con 0x5630ea6826b0 2023-07-14T16:18:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.252+0000 7f42195f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea5fb8c0 con 0x5630ea8276e0 2023-07-14T16:18:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f42195f4b80 10 client.0 _ll_register_callbacks cb 0x5630ea715700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea836a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea82fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea836a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52450/0 (socket says 172.21.15.62:52450) 2023-07-14T16:18:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41fffff700 1 -- 172.21.15.62:0/233976533 learned_addr learned my addr 172.21.15.62:0/233976533 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea82f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41ff7fe700 1 -- 172.21.15.62:0/233976533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 msgr2=0x5630ea836a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea836a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41ff7fe700 1 -- 172.21.15.62:0/233976533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 msgr2=0x5630ea82fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea82fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.253+0000 7f41ff7fe700 1 -- 172.21.15.62:0/233976533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42001a44f0 con 0x5630ea6826b0 2023-07-14T16:18:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.254+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea82f510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f41f009add0 tx=0x7f41f00977f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7341b721f1b7144c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.254+0000 7f41deffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:18:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.254+0000 7f41deffd700 1 -- 172.21.15.62:0/233976533 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00b2070 con 0x5630ea6826b0 2023-07-14T16:18:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.254+0000 7f41deffd700 1 -- 172.21.15.62:0/233976533 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41f00cb1e0 con 0x5630ea6826b0 2023-07-14T16:18:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.254+0000 7f42195f4b80 1 -- 172.21.15.62:0/233976533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630ea6d1490 con 0x5630ea6826b0 2023-07-14T16:18:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.254+0000 7f41deffd700 1 -- 172.21.15.62:0/233976533 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41f0005470 con 0x5630ea6826b0 2023-07-14T16:18:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.254+0000 7f42195f4b80 10 client.25967 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f42195f4b80 1 -- 172.21.15.62:0/233976533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5630ea58ec00 con 0x5630ea6826b0 2023-07-14T16:18:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f41deffd700 1 -- 172.21.15.62:0/233976533 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5630ea6d1490 con 0x5630ea6826b0 2023-07-14T16:18:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f41deffd700 1 -- 172.21.15.62:0/233976533 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f41f00a2d70 con 0x5630ea6826b0 2023-07-14T16:18:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f42195f4b80 10 client.25967 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f42195f4b80 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f42195f4b80 20 client.25967 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f42195f4b80 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630ea83adc0 0x5630ea83d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f42195f4b80 4 client.25967 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f42195f4b80 1 -- 172.21.15.62:0/233976533 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630ea770e50 con 0x5630ea83adc0 2023-07-14T16:18:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.255+0000 7f41feffd700 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630ea83adc0 0x5630ea83d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.256+0000 7f41deffd700 10 client.25967 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.256+0000 7f41feffd700 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630ea83adc0 0x5630ea83d1d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52ab4f238f32c9f server_cookie=a0cf7efda9f91feb in_seq=0 out_seq=0 2023-07-14T16:18:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f41deffd700 1 -- 172.21.15.62:0/233976533 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5630ea770e50 con 0x5630ea83adc0 2023-07-14T16:18:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f41deffd700 10 client.25967 handle_command_reply: tid=0 2023-07-14T16:18:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 client.25967 shutdown 2023-07-14T16:18:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 -- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 msgr2=0x5630ea82f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea82f510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f41f009add0 tx=0x7f41f00977f0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 -- 172.21.15.62:0/233976533 shutdown_connections 2023-07-14T16:18:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea836a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630ea83adc0 0x5630ea83d1d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea82f510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 --2- 172.21.15.62:0/233976533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea82fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f41deffd700 0 client.25967 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 -- 172.21.15.62:0/233976533 >> 172.21.15.62:0/233976533 conn(0x5630ea81bbe0 msgr2=0x5630ea8204e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 -- 172.21.15.62:0/233976533 shutdown_connections 2023-07-14T16:18:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.257+0000 7f42195f4b80 1 -- 172.21.15.62:0/233976533 wait complete. 2023-07-14T16:18:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.258+0000 7f42195f4b80 20 client.25967 trim_cache size 0 max 16384 2023-07-14T16:18:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.273+0000 7f42195f4b80 1 Processor -- start 2023-07-14T16:18:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.273+0000 7f42195f4b80 1 -- start start 2023-07-14T16:18:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.273+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea841b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.273+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea836a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.273+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea82f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.273+0000 7f42195f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea82f0e0 con 0x5630ea826840 2023-07-14T16:18:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.273+0000 7f42195f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea830700 con 0x5630ea6826b0 2023-07-14T16:18:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.273+0000 7f42195f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea7ec060 con 0x5630ea8276e0 2023-07-14T16:18:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea836a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea836a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43752/0 (socket says 172.21.15.62:43752) 2023-07-14T16:18:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41feffd700 1 -- 172.21.15.62:0/198279705 learned_addr learned my addr 172.21.15.62:0/198279705 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41fffff700 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea82f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea841b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41feffd700 1 -- 172.21.15.62:0/198279705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 msgr2=0x5630ea82f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41feffd700 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea82f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41feffd700 1 -- 172.21.15.62:0/198279705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 msgr2=0x5630ea841b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41feffd700 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea841b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.274+0000 7f41feffd700 1 -- 172.21.15.62:0/198279705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ea5cc1f0 con 0x5630ea826840 2023-07-14T16:18:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.275+0000 7f41feffd700 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea836a50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5630ea83dae0 tx=0x7f41f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e954d23dda3f57 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.275+0000 7f41e67fc700 1 -- 172.21.15.62:0/198279705 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4022070 con 0x5630ea826840 2023-07-14T16:18:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.275+0000 7f41e67fc700 1 -- 172.21.15.62:0/198279705 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41f4023040 con 0x5630ea826840 2023-07-14T16:18:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.275+0000 7f41e67fc700 1 -- 172.21.15.62:0/198279705 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f400ecb0 con 0x5630ea826840 2023-07-14T16:18:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 -- 172.21.15.62:0/198279705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 msgr2=0x5630ea836a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea836a50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5630ea83dae0 tx=0x7f41f4014000 comp rx=0 tx=0).stop 2023-07-14T16:18:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 -- 172.21.15.62:0/198279705 shutdown_connections 2023-07-14T16:18:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea8276e0 0x5630ea82f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea6826b0 0x5630ea841b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 --2- 172.21.15.62:0/198279705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea836a50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 -- 172.21.15.62:0/198279705 >> 172.21.15.62:0/198279705 conn(0x5630ea81bbe0 msgr2=0x5630ea820c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 -- 172.21.15.62:0/198279705 shutdown_connections 2023-07-14T16:18:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 -- 172.21.15.62:0/198279705 wait complete. 2023-07-14T16:18:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.276+0000 7f42195f4b80 1 -- start start 2023-07-14T16:18:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.277+0000 7f42195f4b80 1 Processor -- start 2023-07-14T16:18:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.277+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea832830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.277+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea832d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.277+0000 7f42195f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea8276e0 0x5630ea835d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.277+0000 7f42195f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea8362d0 con 0x5630ea826840 2023-07-14T16:18:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.277+0000 7f42195f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea836440 con 0x5630ea8276e0 2023-07-14T16:18:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.277+0000 7f42195f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ea836600 con 0x5630ea6826b0 2023-07-14T16:18:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.277+0000 7f42195f4b80 10 client.0 _ll_register_callbacks cb 0x7f41f8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea832d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41ff7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea832830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea8276e0 0x5630ea835d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41ff7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea832830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52460/0 (socket says 172.21.15.62:52460) 2023-07-14T16:18:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41ff7fe700 1 -- 172.21.15.62:0/3967753517 learned_addr learned my addr 172.21.15.62:0/3967753517 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41fffff700 1 -- 172.21.15.62:0/3967753517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 msgr2=0x5630ea832830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41fffff700 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea832830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41fffff700 1 -- 172.21.15.62:0/3967753517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 msgr2=0x5630ea832d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41fffff700 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea832d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41fffff700 1 -- 172.21.15.62:0/3967753517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f4017070 con 0x5630ea8276e0 2023-07-14T16:18:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41feffd700 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea832d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41fffff700 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea8276e0 0x5630ea835d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009ec0 tx=0x7f41f8007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2869278b7da27d93 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.278+0000 7f41deffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:18:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.279+0000 7f41deffd700 1 -- 172.21.15.62:0/3967753517 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f801c070 con 0x5630ea8276e0 2023-07-14T16:18:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.279+0000 7f42195f4b80 1 -- 172.21.15.62:0/3967753517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41f00a3200 con 0x5630ea8276e0 2023-07-14T16:18:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.279+0000 7f41deffd700 1 -- 172.21.15.62:0/3967753517 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41f8004bb0 con 0x5630ea8276e0 2023-07-14T16:18:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.279+0000 7f41deffd700 1 -- 172.21.15.62:0/3967753517 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41f800e040 con 0x5630ea8276e0 2023-07-14T16:18:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.279+0000 7f42195f4b80 10 client.25973 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.279+0000 7f42195f4b80 1 -- 172.21.15.62:0/3967753517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5630ea5a4250 con 0x5630ea8276e0 2023-07-14T16:18:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.279+0000 7f41deffd700 1 -- 172.21.15.62:0/3967753517 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41f802f080 con 0x5630ea8276e0 2023-07-14T16:18:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.280+0000 7f41deffd700 1 -- 172.21.15.62:0/3967753517 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f41f80203f0 con 0x5630ea8276e0 2023-07-14T16:18:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.280+0000 7f42195f4b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.280+0000 7f42195f4b80 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.280+0000 7f42195f4b80 20 client.25973 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.280+0000 7f42195f4b80 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630ea88fdf0 0x5630ea892200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.280+0000 7f42195f4b80 4 client.25973 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:18:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.280+0000 7f42195f4b80 1 -- 172.21.15.62:0/3967753517 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5630ea732ff0 con 0x5630ea88fdf0 2023-07-14T16:18:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.280+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630ea88fdf0 0x5630ea892200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.281+0000 7f41ff7fe700 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630ea88fdf0 0x5630ea892200 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d998ea73a6309139 server_cookie=81ec2b69687f63c in_seq=0 out_seq=0 2023-07-14T16:18:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.281+0000 7f41deffd700 10 client.25973 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f41deffd700 1 -- 172.21.15.62:0/3967753517 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5630ea732ff0 con 0x5630ea88fdf0 2023-07-14T16:18:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f41deffd700 10 client.25973 handle_command_reply: tid=0 2023-07-14T16:18:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 client.25973 shutdown 2023-07-14T16:18:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 -- 172.21.15.62:0/3967753517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea8276e0 msgr2=0x5630ea835d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea8276e0 0x5630ea835d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009ec0 tx=0x7f41f8007f60 comp rx=0 tx=0).stop 2023-07-14T16:18:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 -- 172.21.15.62:0/3967753517 shutdown_connections 2023-07-14T16:18:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630ea88fdf0 0x5630ea892200 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630ea8276e0 0x5630ea835d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630ea6826b0 0x5630ea832830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f41deffd700 0 client.25973 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 --2- 172.21.15.62:0/3967753517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630ea826840 0x5630ea832d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 -- 172.21.15.62:0/3967753517 >> 172.21.15.62:0/3967753517 conn(0x5630ea81bbe0 msgr2=0x5630ea82f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 -- 172.21.15.62:0/3967753517 shutdown_connections 2023-07-14T16:18:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 1 -- 172.21.15.62:0/3967753517 wait complete. 2023-07-14T16:18:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.290+0000 7f42195f4b80 20 client.25973 trim_cache size 0 max 16384 2023-07-14T16:18:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:18:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:18:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:18:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:18:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 -- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1150 msgr2=0x7f41e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1150 0x7f41e80a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005fd0 tx=0x7f41f4009310 comp rx=0 tx=0).stop 2023-07-14T16:18:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 -- 172.21.15.62:0/2287565224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 msgr2=0x7f420019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 0x7f420019bee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009fd0 tx=0x7f41f000bf80 comp rx=0 tx=0).stop 2023-07-14T16:18:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 -- 172.21.15.62:0/2287565224 shutdown_connections 2023-07-14T16:18:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200107cd0 0x7f420019d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1150 0x7f41e80a3600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200106150 0x7f420019bee0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 --2- 172.21.15.62:0/2287565224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200107340 0x7f420019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 -- 172.21.15.62:0/2287565224 >> 172.21.15.62:0/2287565224 conn(0x7f4200075190 msgr2=0x7f4200077450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 -- 172.21.15.62:0/2287565224 shutdown_connections 2023-07-14T16:18:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:15.291+0000 7f4205bff700 1 -- 172.21.15.62:0/2287565224 wait complete. 2023-07-14T16:18:15.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:15 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:18:16.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:15 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:18:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:15 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:18:16.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:16 smithi081 ceph-mon[127795]: pgmap v436: 97 pgs: 97 active+clean; 686 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:18:16.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:16 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:18:16.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:16 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:18:16.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:16 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:18:17.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:16 smithi164 ceph-mon[97427]: pgmap v436: 97 pgs: 97 active+clean; 686 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:18:17.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:16 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:18:17.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:16 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:18:17.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:16 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:18:17.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:16 smithi062 ceph-mon[95543]: pgmap v436: 97 pgs: 97 active+clean; 686 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:18:17.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:16 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:18:17.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:16 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:18:17.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:16 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:18:18.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:18 smithi081 ceph-mon[127795]: pgmap v437: 97 pgs: 97 active+clean; 742 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 34 op/s 2023-07-14T16:18:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:18 smithi164 ceph-mon[97427]: pgmap v437: 97 pgs: 97 active+clean; 742 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 34 op/s 2023-07-14T16:18:19.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:18 smithi062 ceph-mon[95543]: pgmap v437: 97 pgs: 97 active+clean; 742 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 34 op/s 2023-07-14T16:18:20.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:20 smithi081 ceph-mon[127795]: pgmap v438: 97 pgs: 97 active+clean; 758 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:18:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:20 smithi164 ceph-mon[97427]: pgmap v438: 97 pgs: 97 active+clean; 758 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:18:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:20 smithi062 ceph-mon[95543]: pgmap v438: 97 pgs: 97 active+clean; 758 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:18:22.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:22 smithi081 ceph-mon[127795]: pgmap v439: 97 pgs: 97 active+clean; 782 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 20 op/s 2023-07-14T16:18:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:22 smithi164 ceph-mon[97427]: pgmap v439: 97 pgs: 97 active+clean; 782 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 20 op/s 2023-07-14T16:18:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:22 smithi062 ceph-mon[95543]: pgmap v439: 97 pgs: 97 active+clean; 782 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 20 op/s 2023-07-14T16:18:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:24 smithi164 ceph-mon[97427]: pgmap v440: 97 pgs: 97 active+clean; 826 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:18:25.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:24 smithi062 ceph-mon[95543]: pgmap v440: 97 pgs: 97 active+clean; 826 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:18:25.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:24 smithi081 ceph-mon[127795]: pgmap v440: 97 pgs: 97 active+clean; 826 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:18:27.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:26 smithi164 ceph-mon[97427]: pgmap v441: 97 pgs: 97 active+clean; 834 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:18:27.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:26 smithi062 ceph-mon[95543]: pgmap v441: 97 pgs: 97 active+clean; 834 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:18:27.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:26 smithi081 ceph-mon[127795]: pgmap v441: 97 pgs: 97 active+clean; 834 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:18:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 -- 172.21.15.62:0/3798422213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f41056e0 msgr2=0x7f53f4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 --2- 172.21.15.62:0/3798422213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f41056e0 0x7f53f4107b90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009ec0 tx=0x7f53ec00b330 comp rx=0 tx=0).stop 2023-07-14T16:18:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 -- 172.21.15.62:0/3798422213 shutdown_connections 2023-07-14T16:18:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 --2- 172.21.15.62:0/3798422213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f41056e0 0x7f53f4107b90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 --2- 172.21.15.62:0/3798422213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53f4068640 0x7f53f4102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 --2- 172.21.15.62:0/3798422213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f4102bd0 0x7f53f4105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 -- 172.21.15.62:0/3798422213 >> 172.21.15.62:0/3798422213 conn(0x7f53f40fa000 msgr2=0x7f53f40fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 -- 172.21.15.62:0/3798422213 shutdown_connections 2023-07-14T16:18:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.466+0000 7f53f9bff700 1 -- 172.21.15.62:0/3798422213 wait complete. 2023-07-14T16:18:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.468+0000 7f53f9bff700 1 Processor -- start 2023-07-14T16:18:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.468+0000 7f53f9bff700 1 -- start start 2023-07-14T16:18:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.468+0000 7f53f9bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4068640 0x7f53f4195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.468+0000 7f53f9bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f4102bd0 0x7f53f4195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.468+0000 7f53f9bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53f41056e0 0x7f53f4196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.468+0000 7f53f9bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41973a0 con 0x7f53f4068640 2023-07-14T16:18:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.468+0000 7f53f9bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4197510 con 0x7f53f41056e0 2023-07-14T16:18:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.468+0000 7f53f9bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41976d0 con 0x7f53f4102bd0 2023-07-14T16:18:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4068640 0x7f53f4195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4068640 0x7f53f4195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37940/0 (socket says 172.21.15.62:37940) 2023-07-14T16:18:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 -- 172.21.15.62:0/2329136281 learned_addr learned my addr 172.21.15.62:0/2329136281 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f4102bd0 0x7f53f4195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f3fff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53f41056e0 0x7f53f4196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 -- 172.21.15.62:0/2329136281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f4102bd0 msgr2=0x7f53f4195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f4102bd0 0x7f53f4195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 -- 172.21.15.62:0/2329136281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53f41056e0 msgr2=0x7f53f4196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53f41056e0 0x7f53f4196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 -- 172.21.15.62:0/2329136281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec005e70 con 0x7f53f4068640 2023-07-14T16:18:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.469+0000 7f53f37fe700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4068640 0x7f53f4195980 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e970 tx=0x7f53e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.470+0000 7f53f0ff9700 1 -- 172.21.15.62:0/2329136281 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4009790 con 0x7f53f4068640 2023-07-14T16:18:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.470+0000 7f53f0ff9700 1 -- 172.21.15.62:0/2329136281 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53e400c7e0 con 0x7f53f4068640 2023-07-14T16:18:28.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.470+0000 7f53f9bff700 1 -- 172.21.15.62:0/2329136281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f41979b0 con 0x7f53f4068640 2023-07-14T16:18:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.470+0000 7f53f0ff9700 1 -- 172.21.15.62:0/2329136281 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4005440 con 0x7f53f4068640 2023-07-14T16:18:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.470+0000 7f53f9bff700 1 -- 172.21.15.62:0/2329136281 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f419e170 con 0x7f53f4068640 2023-07-14T16:18:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.474+0000 7f53f0ff9700 1 -- 172.21.15.62:0/2329136281 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f53e400c950 con 0x7f53f4068640 2023-07-14T16:18:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.475+0000 7f540d540b80 1 Processor -- start 2023-07-14T16:18:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f540d540b80 1 -- start start 2023-07-14T16:18:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f53f0ff9700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53dc0a11d0 0x7f53dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f53f0ff9700 1 -- 172.21.15.62:0/2329136281 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53e4014070 con 0x7f53f4068640 2023-07-14T16:18:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f540d540b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f540d540b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d203df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f540d540b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d203dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f540d540b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d202e3c50 con 0x561d203de840 2023-07-14T16:18:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f540d540b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d203369e0 con 0x561d203df6e0 2023-07-14T16:18:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f540d540b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2032fe70 con 0x561d2023a6b0 2023-07-14T16:18:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f53f3fff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53dc0a11d0 0x7f53dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d203df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d203df1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37946/0 (socket says 172.21.15.62:37946) 2023-07-14T16:18:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.476+0000 7f53f2ffd700 1 -- 172.21.15.62:0/2728423544 learned_addr learned my addr 172.21.15.62:0/2728423544 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f3fff700 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d203dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f37fe700 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f2ffd700 1 -- 172.21.15.62:0/2728423544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 msgr2=0x561d203de300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203de300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f2ffd700 1 -- 172.21.15.62:0/2728423544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 msgr2=0x561d203dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d203dfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f2ffd700 1 -- 172.21.15.62:0/2728423544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d2015dd80 con 0x561d203de840 2023-07-14T16:18:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d203df1a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009d80 tx=0x7f53e8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1b486ffff805b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53f3fff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53dc0a11d0 0x7f53dc0a3680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009850 tx=0x7f53ec007580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.477+0000 7f53d9ffb700 1 -- 172.21.15.62:0/2728423544 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8004230 con 0x561d203de840 2023-07-14T16:18:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.478+0000 7f53d9ffb700 1 -- 172.21.15.62:0/2728423544 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53e8004390 con 0x561d203de840 2023-07-14T16:18:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.478+0000 7f53d9ffb700 1 -- 172.21.15.62:0/2728423544 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8019750 con 0x561d203de840 2023-07-14T16:18:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.478+0000 7f540d540b80 1 -- 172.21.15.62:0/2728423544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 msgr2=0x561d203df1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.478+0000 7f540d540b80 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d203df1a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009d80 tx=0x7f53e8011720 comp rx=0 tx=0).stop 2023-07-14T16:18:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.479+0000 7f540d540b80 1 -- 172.21.15.62:0/2728423544 shutdown_connections 2023-07-14T16:18:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.479+0000 7f540d540b80 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d203dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.479+0000 7f540d540b80 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203de300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.479+0000 7f540d540b80 1 --2- 172.21.15.62:0/2728423544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d203df1a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.479+0000 7f540d540b80 1 -- 172.21.15.62:0/2728423544 >> 172.21.15.62:0/2728423544 conn(0x561d203d3be0 msgr2=0x561d203d8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.479+0000 7f540d540b80 1 -- 172.21.15.62:0/2728423544 shutdown_connections 2023-07-14T16:18:28.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.479+0000 7f540d540b80 1 -- 172.21.15.62:0/2728423544 wait complete. 2023-07-14T16:18:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.479+0000 7f540d540b80 1 -- start start 2023-07-14T16:18:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.480+0000 7f540d540b80 1 Processor -- start 2023-07-14T16:18:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.480+0000 7f540d540b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f540d540b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203de840 0x561d203e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f540d540b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203df6e0 0x561d203eedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f540d540b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d202cfa60 con 0x561d203df6e0 2023-07-14T16:18:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f540d540b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d202cf760 con 0x561d203de840 2023-07-14T16:18:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f540d540b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d201b38c0 con 0x561d2023a6b0 2023-07-14T16:18:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f540d540b80 10 client.0 _ll_register_callbacks cb 0x561d202cd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203df6e0 0x561d203eedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203de840 0x561d203e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203de840 0x561d203e7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:38502/0 (socket says 172.21.15.62:38502) 2023-07-14T16:18:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f53f2ffd700 1 -- 172.21.15.62:0/4254960860 learned_addr learned my addr 172.21.15.62:0/4254960860 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203df6e0 0x561d203eedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37954/0 (socket says 172.21.15.62:37954) 2023-07-14T16:18:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f53f37fe700 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f53f3fff700 1 -- 172.21.15.62:0/4254960860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 msgr2=0x561d203e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.481+0000 7f53f3fff700 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203e7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f53f3fff700 1 -- 172.21.15.62:0/4254960860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203de840 msgr2=0x561d203e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f53f3fff700 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203de840 0x561d203e7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f53f3fff700 1 -- 172.21.15.62:0/4254960860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e80099e0 con 0x561d203df6e0 2023-07-14T16:18:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f53f37fe700 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f53f3fff700 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203df6e0 0x561d203eedb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f53ec007840 tx=0x7f53ec00b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2017d288f79f592c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f53d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4254960860 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec026070 con 0x561d203df6e0 2023-07-14T16:18:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4254960860 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53ec01e430 con 0x561d203df6e0 2023-07-14T16:18:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.482+0000 7f540d540b80 1 -- 172.21.15.62:0/4254960860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d20289490 con 0x561d203df6e0 2023-07-14T16:18:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4254960860 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53ec00e040 con 0x561d203df6e0 2023-07-14T16:18:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f540d540b80 10 client.16242 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f540d540b80 1 -- 172.21.15.62:0/4254960860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561d20146c00 con 0x561d203df6e0 2023-07-14T16:18:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4254960860 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53ec03b020 con 0x561d203df6e0 2023-07-14T16:18:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4254960860 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f53ec035450 con 0x561d203df6e0 2023-07-14T16:18:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f540d540b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f540d540b80 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f540d540b80 20 client.16242 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f540d540b80 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561d203f3680 0x561d203f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f540d540b80 4 client.16242 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.483+0000 7f540d540b80 1 -- 172.21.15.62:0/4254960860 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d20328e50 con 0x561d203f3680 2023-07-14T16:18:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.484+0000 7f53f37fe700 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561d203f3680 0x561d203f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.484+0000 7f53f37fe700 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561d203f3680 0x561d203f5a90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f119a79652303e73 server_cookie=63d9ba16c72afcab in_seq=0 out_seq=0 2023-07-14T16:18:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.484+0000 7f53d2ffd700 10 client.16242 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4254960860 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561d20328e50 con 0x561d203f3680 2023-07-14T16:18:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f53d2ffd700 10 client.16242 handle_command_reply: tid=0 2023-07-14T16:18:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 client.16242 shutdown 2023-07-14T16:18:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 -- 172.21.15.62:0/4254960860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203df6e0 msgr2=0x561d203eedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203df6e0 0x561d203eedb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f53ec007840 tx=0x7f53ec00b680 comp rx=0 tx=0).stop 2023-07-14T16:18:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 -- 172.21.15.62:0/4254960860 shutdown_connections 2023-07-14T16:18:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203df6e0 0x561d203eedb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d203e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561d203f3680 0x561d203f5a90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 --2- 172.21.15.62:0/4254960860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203de840 0x561d203e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f540d540b80 1 -- 172.21.15.62:0/4254960860 >> 172.21.15.62:0/4254960860 conn(0x561d203d3be0 msgr2=0x561d203d84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.485+0000 7f53d2ffd700 0 client.16242 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.486+0000 7f540d540b80 1 -- 172.21.15.62:0/4254960860 shutdown_connections 2023-07-14T16:18:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.486+0000 7f540d540b80 1 -- 172.21.15.62:0/4254960860 wait complete. 2023-07-14T16:18:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.486+0000 7f540d540b80 20 client.16242 trim_cache size 0 max 16384 2023-07-14T16:18:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.506+0000 7f540d540b80 1 Processor -- start 2023-07-14T16:18:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f540d540b80 1 -- start start 2023-07-14T16:18:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f540d540b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d2023a6b0 0x561d203fa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f540d540b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d203de840 0x561d203e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f540d540b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d203e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f540d540b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d203e6370 con 0x561d2023a6b0 2023-07-14T16:18:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f540d540b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d203da7a0 con 0x561d203df6e0 2023-07-14T16:18:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f540d540b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d203e3180 con 0x561d203de840 2023-07-14T16:18:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d2023a6b0 0x561d203fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d2023a6b0 0x561d203fa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37970/0 (socket says 172.21.15.62:37970) 2023-07-14T16:18:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f53f37fe700 1 -- 172.21.15.62:0/1127717495 learned_addr learned my addr 172.21.15.62:0/1127717495 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.507+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d203de840 0x561d203e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53f3fff700 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d203e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53f37fe700 1 -- 172.21.15.62:0/1127717495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d203de840 msgr2=0x561d203e65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d203de840 0x561d203e65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53f37fe700 1 -- 172.21.15.62:0/1127717495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 msgr2=0x561d203e25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d203e25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53f37fe700 1 -- 172.21.15.62:0/1127717495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d201841f0 con 0x561d2023a6b0 2023-07-14T16:18:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53f37fe700 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d2023a6b0 0x561d203fa410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f53e4099df0 tx=0x7f53e40962d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27384fd5a721ffbe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53da7fc700 1 -- 172.21.15.62:0/1127717495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4097fa0 con 0x561d2023a6b0 2023-07-14T16:18:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53da7fc700 1 -- 172.21.15.62:0/1127717495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53e40ca660 con 0x561d2023a6b0 2023-07-14T16:18:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.508+0000 7f53da7fc700 1 -- 172.21.15.62:0/1127717495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e409a390 con 0x561d2023a6b0 2023-07-14T16:18:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 -- 172.21.15.62:0/1127717495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d2023a6b0 msgr2=0x561d203fa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d2023a6b0 0x561d203fa410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f53e4099df0 tx=0x7f53e40962d0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 -- 172.21.15.62:0/1127717495 shutdown_connections 2023-07-14T16:18:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d203e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d2023a6b0 0x561d203fa410 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 --2- 172.21.15.62:0/1127717495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d203de840 0x561d203e65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 -- 172.21.15.62:0/1127717495 >> 172.21.15.62:0/1127717495 conn(0x561d203d3be0 msgr2=0x561d203d8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 -- 172.21.15.62:0/1127717495 shutdown_connections 2023-07-14T16:18:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.509+0000 7f540d540b80 1 -- 172.21.15.62:0/1127717495 wait complete. 2023-07-14T16:18:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 1 -- start start 2023-07-14T16:18:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 1 Processor -- start 2023-07-14T16:18:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d20443780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d20443cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d2044a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2044ad90 con 0x561d203de840 2023-07-14T16:18:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2044af00 con 0x561d203df6e0 2023-07-14T16:18:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2044b0c0 con 0x561d2023a6b0 2023-07-14T16:18:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.510+0000 7f540d540b80 10 client.0 _ll_register_callbacks cb 0x561d202cd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d20443cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d20443cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37982/0 (socket says 172.21.15.62:37982) 2023-07-14T16:18:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f2ffd700 1 -- 172.21.15.62:0/4053664884 learned_addr learned my addr 172.21.15.62:0/4053664884 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f3fff700 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d2044a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f37fe700 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d20443780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f37fe700 1 -- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 msgr2=0x561d2044a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f37fe700 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d2044a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f37fe700 1 -- 172.21.15.62:0/4053664884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 msgr2=0x561d20443cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f37fe700 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d20443cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f37fe700 1 -- 172.21.15.62:0/4053664884 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f419e170 con 0x561d2023a6b0 2023-07-14T16:18:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.511+0000 7f53f3fff700 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d2044a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f53f37fe700 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d20443780 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000c00 tx=0x7f53e400cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=359873b681489087 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f53d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:18:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4053664884 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4099350 con 0x561d2023a6b0 2023-07-14T16:18:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f540d540b80 1 -- 172.21.15.62:0/4053664884 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e8004390 con 0x561d2023a6b0 2023-07-14T16:18:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4053664884 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f53e40994b0 con 0x561d2023a6b0 2023-07-14T16:18:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4053664884 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e40a1e20 con 0x561d2023a6b0 2023-07-14T16:18:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f540d540b80 10 client.25900 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f540d540b80 1 -- 172.21.15.62:0/4053664884 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561d2015c250 con 0x561d2023a6b0 2023-07-14T16:18:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4053664884 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53e407e020 con 0x561d2023a6b0 2023-07-14T16:18:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.512+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4053664884 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f53e4004a30 con 0x561d2023a6b0 2023-07-14T16:18:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.513+0000 7f540d540b80 10 client.25900 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.513+0000 7f540d540b80 10 client.25900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.513+0000 7f540d540b80 20 client.25900 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.513+0000 7f540d540b80 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561d2044c830 0x561d2044ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.513+0000 7f540d540b80 4 client.25900 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:18:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.513+0000 7f540d540b80 1 -- 172.21.15.62:0/4053664884 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561d202eaff0 con 0x561d2044c830 2023-07-14T16:18:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.513+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561d2044c830 0x561d2044ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.514+0000 7f53f2ffd700 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561d2044c830 0x561d2044ec40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc016d7c5393dbc server_cookie=1fe91ee10430b412 in_seq=0 out_seq=0 2023-07-14T16:18:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.514+0000 7f53d2ffd700 10 client.25900 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f53d2ffd700 1 -- 172.21.15.62:0/4053664884 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561d202eaff0 con 0x561d2044c830 2023-07-14T16:18:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f53d2ffd700 10 client.25900 handle_command_reply: tid=0 2023-07-14T16:18:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 client.25900 shutdown 2023-07-14T16:18:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 -- 172.21.15.62:0/4053664884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 msgr2=0x561d20443780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d20443780 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000c00 tx=0x7f53e400cf50 comp rx=0 tx=0).stop 2023-07-14T16:18:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 -- 172.21.15.62:0/4053664884 shutdown_connections 2023-07-14T16:18:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561d2044c830 0x561d2044ec40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561d203df6e0 0x561d2044a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561d2023a6b0 0x561d20443780 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 --2- 172.21.15.62:0/4053664884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561d203de840 0x561d20443cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f53d2ffd700 0 client.25900 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 -- 172.21.15.62:0/4053664884 >> 172.21.15.62:0/4053664884 conn(0x561d203d3be0 msgr2=0x561d203e25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 -- 172.21.15.62:0/4053664884 shutdown_connections 2023-07-14T16:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 1 -- 172.21.15.62:0/4053664884 wait complete. 2023-07-14T16:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.524+0000 7f540d540b80 20 client.25900 trim_cache size 0 max 16384 2023-07-14T16:18:28.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:18:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:18:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:18:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:18:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 -- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53dc0a11d0 msgr2=0x7f53dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53dc0a11d0 0x7f53dc0a3680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009850 tx=0x7f53ec007580 comp rx=0 tx=0).stop 2023-07-14T16:18:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 -- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4068640 msgr2=0x7f53f4195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4068640 0x7f53f4195980 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e970 tx=0x7f53e400ec80 comp rx=0 tx=0).stop 2023-07-14T16:18:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 -- 172.21.15.62:0/2329136281 shutdown_connections 2023-07-14T16:18:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f53dc0a11d0 0x7f53dc0a3680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53f41056e0 0x7f53f4196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53f4068640 0x7f53f4195980 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 --2- 172.21.15.62:0/2329136281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f4102bd0 0x7f53f4195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 -- 172.21.15.62:0/2329136281 >> 172.21.15.62:0/2329136281 conn(0x7f53f40fa000 msgr2=0x7f53f4101730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 -- 172.21.15.62:0/2329136281 shutdown_connections 2023-07-14T16:18:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:28.525+0000 7f53f9bff700 1 -- 172.21.15.62:0/2329136281 wait complete. 2023-07-14T16:18:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:28 smithi164 ceph-mon[97427]: pgmap v442: 97 pgs: 97 active+clean; 886 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:18:29.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:28 smithi062 ceph-mon[95543]: pgmap v442: 97 pgs: 97 active+clean; 886 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:18:29.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:28 smithi081 ceph-mon[127795]: pgmap v442: 97 pgs: 97 active+clean; 886 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:18:31.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:30 smithi164 ceph-mon[97427]: pgmap v443: 97 pgs: 97 active+clean; 902 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:18:31.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:30 smithi062 ceph-mon[95543]: pgmap v443: 97 pgs: 97 active+clean; 902 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:18:31.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:30 smithi081 ceph-mon[127795]: pgmap v443: 97 pgs: 97 active+clean; 902 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:18:33.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:32 smithi164 ceph-mon[97427]: pgmap v444: 97 pgs: 97 active+clean; 946 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:18:33.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:32 smithi062 ceph-mon[95543]: pgmap v444: 97 pgs: 97 active+clean; 946 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:18:33.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:32 smithi081 ceph-mon[127795]: pgmap v444: 97 pgs: 97 active+clean; 946 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:18:35.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:34 smithi164 ceph-mon[97427]: pgmap v445: 97 pgs: 97 active+clean; 974 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:18:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:34 smithi062 ceph-mon[95543]: pgmap v445: 97 pgs: 97 active+clean; 974 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:18:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:34 smithi081 ceph-mon[127795]: pgmap v445: 97 pgs: 97 active+clean; 974 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:18:36.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:18:36.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/235122268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a40ffae0 msgr2=0x7fb1a40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/235122268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a40ffae0 0x7fb1a40ffef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1980098f0 tx=0x7fb198009c00 comp rx=0 tx=0).stop 2023-07-14T16:18:36.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/235122268 shutdown_connections 2023-07-14T16:18:36.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/235122268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4101660 0x7fb1a4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/235122268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a40ffae0 0x7fb1a40ffef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/235122268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a4100cd0 0x7fb1a4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/235122268 >> 172.21.15.62:0/235122268 conn(0x7fb1a40fb0e0 msgr2=0x7fb1a40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:36.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/235122268 shutdown_connections 2023-07-14T16:18:36.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.703+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/235122268 wait complete. 2023-07-14T16:18:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.704+0000 7fb1ab1ff700 1 Processor -- start 2023-07-14T16:18:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.704+0000 7fb1ab1ff700 1 -- start start 2023-07-14T16:18:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1ab1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a40ffae0 0x7fb1a41957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1ab1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4100cd0 0x7fb1a4195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1ab1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a4101660 0x7fb1a419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1ab1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a419d360 con 0x7fb1a4100cd0 2023-07-14T16:18:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1ab1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a419d4a0 con 0x7fb1a40ffae0 2023-07-14T16:18:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1ab1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a419d630 con 0x7fb1a4101660 2023-07-14T16:18:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1a3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4100cd0 0x7fb1a4195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1a8f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a40ffae0 0x7fb1a41957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1a8f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a40ffae0 0x7fb1a41957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59090/0 (socket says 172.21.15.62:59090) 2023-07-14T16:18:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/2816986661 learned_addr learned my addr 172.21.15.62:0/2816986661 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.705+0000 7fb1a979c700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a4101660 0x7fb1a419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.706+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/2816986661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a4101660 msgr2=0x7fb1a419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.706+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a4101660 0x7fb1a419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.706+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4100cd0 msgr2=0x7fb1a4195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.706+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4100cd0 0x7fb1a4195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.706+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/2816986661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c009d00 con 0x7fb1a40ffae0 2023-07-14T16:18:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.706+0000 7fb1a979c700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a4101660 0x7fb1a419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.706+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4100cd0 0x7fb1a4195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.706+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a40ffae0 0x7fb1a41957e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb198009fd0 tx=0x7fb19800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.707+0000 7fb1a1ffb700 1 -- 172.21.15.62:0/2816986661 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19801c070 con 0x7fb1a40ffae0 2023-07-14T16:18:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.707+0000 7fb1a1ffb700 1 -- 172.21.15.62:0/2816986661 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb19800f7b0 con 0x7fb1a40ffae0 2023-07-14T16:18:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.707+0000 7fb1a1ffb700 1 -- 172.21.15.62:0/2816986661 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198003a00 con 0x7fb1a40ffae0 2023-07-14T16:18:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.707+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/2816986661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb198009520 con 0x7fb1a40ffae0 2023-07-14T16:18:36.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.707+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/2816986661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a4072af0 con 0x7fb1a40ffae0 2023-07-14T16:18:36.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.711+0000 7fb1a1ffb700 1 -- 172.21.15.62:0/2816986661 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb198020030 con 0x7fb1a40ffae0 2023-07-14T16:18:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.712+0000 7fb1a1ffb700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1940a11d0 0x7fb1940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.712+0000 7fb1a1ffb700 1 -- 172.21.15.62:0/2816986661 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1980cd8f0 con 0x7fb1a40ffae0 2023-07-14T16:18:36.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1beb48b80 1 Processor -- start 2023-07-14T16:18:36.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1940a11d0 0x7fb1940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1beb48b80 1 -- start start 2023-07-14T16:18:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b147e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147ee20 0x5637b147f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b14800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b1383cb0 con 0x5637b147e4d0 2023-07-14T16:18:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b13d6a40 con 0x5637b147ee20 2023-07-14T16:18:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.713+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b13cfed0 con 0x5637b147fc70 2023-07-14T16:18:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a8f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b147e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1940a11d0 0x7fb1940a3680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb190011490 tx=0x7fb190009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147ee20 0x5637b147f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a8f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b147e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59928/0 (socket says 172.21.15.62:59928) 2023-07-14T16:18:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/3219770789 learned_addr learned my addr 172.21.15.62:0/3219770789 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:36.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a979c700 1 --2- 172.21.15.62:0/3219770789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b14800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/3219770789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 msgr2=0x5637b14800e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:36.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/3219770789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b14800e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/3219770789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147ee20 msgr2=0x5637b147f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:36.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/3219770789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147ee20 0x5637b147f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.714+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/3219770789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b11fdde0 con 0x5637b147e4d0 2023-07-14T16:18:36.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.715+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/3219770789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b147e8e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb198097240 tx=0x7fb19809a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8df5fa43e8abee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:36.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.715+0000 7fb18effd700 1 -- 172.21.15.62:0/3219770789 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980b2070 con 0x5637b147e4d0 2023-07-14T16:18:36.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.715+0000 7fb18effd700 1 -- 172.21.15.62:0/3219770789 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb19809ed70 con 0x5637b147e4d0 2023-07-14T16:18:36.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.715+0000 7fb18effd700 1 -- 172.21.15.62:0/3219770789 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980b3270 con 0x5637b147e4d0 2023-07-14T16:18:36.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 -- 172.21.15.62:0/3219770789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 msgr2=0x5637b147e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/3219770789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b147e8e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb198097240 tx=0x7fb19809a880 comp rx=0 tx=0).stop 2023-07-14T16:18:36.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 -- 172.21.15.62:0/3219770789 shutdown_connections 2023-07-14T16:18:36.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/3219770789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b14800e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/3219770789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b147e8e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/3219770789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147ee20 0x5637b147f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 -- 172.21.15.62:0/3219770789 >> 172.21.15.62:0/3219770789 conn(0x5637b1473da0 msgr2=0x5637b14791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:36.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 -- 172.21.15.62:0/3219770789 shutdown_connections 2023-07-14T16:18:36.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 -- 172.21.15.62:0/3219770789 wait complete. 2023-07-14T16:18:36.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.716+0000 7fb1beb48b80 1 -- start start 2023-07-14T16:18:36.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.717+0000 7fb1beb48b80 1 Processor -- start 2023-07-14T16:18:36.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.717+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147e4d0 0x5637b1487de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.717+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b1488320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.717+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b148f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.717+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b136fac0 con 0x5637b147ee20 2023-07-14T16:18:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.717+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b136f7c0 con 0x5637b147e4d0 2023-07-14T16:18:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.717+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b1253920 con 0x5637b147fc70 2023-07-14T16:18:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.717+0000 7fb1beb48b80 10 client.0 _ll_register_callbacks cb 0x5637b136d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a8f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147e4d0 0x5637b1487de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b1488320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a8f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147e4d0 0x5637b1487de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59100/0 (socket says 172.21.15.62:59100) 2023-07-14T16:18:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/846280483 learned_addr learned my addr 172.21.15.62:0/846280483 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a979c700 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b148f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a3fff700 1 -- 172.21.15.62:0/846280483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 msgr2=0x5637b148f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:36.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b148f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a3fff700 1 -- 172.21.15.62:0/846280483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147e4d0 msgr2=0x5637b1487de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:36.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147e4d0 0x5637b1487de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a3fff700 1 -- 172.21.15.62:0/846280483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a4072af0 con 0x5637b147ee20 2023-07-14T16:18:36.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a979c700 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b148f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:36.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.718+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147e4d0 0x5637b1487de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:36.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.719+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b1488320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1900094f0 tx=0x7fb1900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8b851a4374e16c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:36.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.719+0000 7fb183fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:36.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.719+0000 7fb183fff700 1 -- 172.21.15.62:0/846280483 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19001f9d0 con 0x5637b147ee20 2023-07-14T16:18:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.719+0000 7fb183fff700 1 -- 172.21.15.62:0/846280483 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb19001fb30 con 0x5637b147ee20 2023-07-14T16:18:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.719+0000 7fb1beb48b80 1 -- 172.21.15.62:0/846280483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637b13294f0 con 0x5637b147ee20 2023-07-14T16:18:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.719+0000 7fb183fff700 1 -- 172.21.15.62:0/846280483 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb19001fcf0 con 0x5637b147ee20 2023-07-14T16:18:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.719+0000 7fb1beb48b80 10 client.16260 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.719+0000 7fb1beb48b80 1 -- 172.21.15.62:0/846280483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5637b11e6c60 con 0x5637b147ee20 2023-07-14T16:18:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb183fff700 1 -- 172.21.15.62:0/846280483 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5637b13294f0 con 0x5637b147ee20 2023-07-14T16:18:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb183fff700 1 -- 172.21.15.62:0/846280483 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb1900323f0 con 0x5637b147ee20 2023-07-14T16:18:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb1beb48b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb1beb48b80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb1beb48b80 20 client.16260 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637b1493c60 0x5637b1496070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb1beb48b80 4 client.16260 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb1beb48b80 1 -- 172.21.15.62:0/846280483 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637b13c8eb0 con 0x5637b1493c60 2023-07-14T16:18:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.720+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637b1493c60 0x5637b1496070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.721+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637b1493c60 0x5637b1496070 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492aa87afa8cf634 server_cookie=5905a71a20263f13 in_seq=0 out_seq=0 2023-07-14T16:18:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.721+0000 7fb183fff700 10 client.16260 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb183fff700 1 -- 172.21.15.62:0/846280483 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5637b13c8eb0 con 0x5637b1493c60 2023-07-14T16:18:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb183fff700 10 client.16260 handle_command_reply: tid=0 2023-07-14T16:18:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 client.16260 shutdown 2023-07-14T16:18:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 -- 172.21.15.62:0/846280483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 msgr2=0x5637b1488320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b1488320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1900094f0 tx=0x7fb1900047c0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 -- 172.21.15.62:0/846280483 shutdown_connections 2023-07-14T16:18:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147fc70 0x5637b148f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637b1493c60 0x5637b1496070 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147e4d0 0x5637b1487de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/846280483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b1488320 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb183fff700 0 client.16260 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 -- 172.21.15.62:0/846280483 >> 172.21.15.62:0/846280483 conn(0x5637b1473da0 msgr2=0x5637b14788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 -- 172.21.15.62:0/846280483 shutdown_connections 2023-07-14T16:18:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.722+0000 7fb1beb48b80 1 -- 172.21.15.62:0/846280483 wait complete. 2023-07-14T16:18:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.723+0000 7fb1beb48b80 20 client.16260 trim_cache size 0 max 16384 2023-07-14T16:18:36.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.750+0000 7fb1beb48b80 1 Processor -- start 2023-07-14T16:18:36.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.750+0000 7fb1beb48b80 1 -- start start 2023-07-14T16:18:36.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.750+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147e4d0 0x5637b148fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.750+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b148f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.750+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b1482bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b1491020 con 0x5637b147ee20 2023-07-14T16:18:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b147f400 con 0x5637b147fc70 2023-07-14T16:18:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b1473970 con 0x5637b147e4d0 2023-07-14T16:18:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b148f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b148f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59942/0 (socket says 172.21.15.62:59942) 2023-07-14T16:18:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a3fff700 1 -- 172.21.15.62:0/529476549 learned_addr learned my addr 172.21.15.62:0/529476549 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:36.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a979c700 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b1482bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147e4d0 0x5637b148fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a3fff700 1 -- 172.21.15.62:0/529476549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147e4d0 msgr2=0x5637b148fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:36.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147e4d0 0x5637b148fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a3fff700 1 -- 172.21.15.62:0/529476549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 msgr2=0x5637b1482bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:36.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b1482bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a3fff700 1 -- 172.21.15.62:0/529476549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b1224250 con 0x5637b147ee20 2023-07-14T16:18:36.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.751+0000 7fb1a979c700 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b1482bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:36.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.752+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b148f320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb1900092d0 tx=0x7fb190009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b117572222595f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:36.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.752+0000 7fb18f7fe700 1 -- 172.21.15.62:0/529476549 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190032970 con 0x5637b147ee20 2023-07-14T16:18:36.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.752+0000 7fb18f7fe700 1 -- 172.21.15.62:0/529476549 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb190032ad0 con 0x5637b147ee20 2023-07-14T16:18:36.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.752+0000 7fb18f7fe700 1 -- 172.21.15.62:0/529476549 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1900286a0 con 0x5637b147ee20 2023-07-14T16:18:36.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 -- 172.21.15.62:0/529476549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 msgr2=0x5637b148f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b148f320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb1900092d0 tx=0x7fb190009300 comp rx=0 tx=0).stop 2023-07-14T16:18:36.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 -- 172.21.15.62:0/529476549 shutdown_connections 2023-07-14T16:18:36.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b1482bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147e4d0 0x5637b148fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/529476549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147ee20 0x5637b148f320 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 -- 172.21.15.62:0/529476549 >> 172.21.15.62:0/529476549 conn(0x5637b1473da0 msgr2=0x5637b149a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:36.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 -- 172.21.15.62:0/529476549 shutdown_connections 2023-07-14T16:18:36.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 -- 172.21.15.62:0/529476549 wait complete. 2023-07-14T16:18:36.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.753+0000 7fb1beb48b80 1 -- start start 2023-07-14T16:18:36.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.754+0000 7fb1beb48b80 1 Processor -- start 2023-07-14T16:18:36.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.754+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b148b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.754+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147ee20 0x5637b148bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.754+0000 7fb1beb48b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b148eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.754+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b148f030 con 0x5637b147e4d0 2023-07-14T16:18:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.754+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b148f1a0 con 0x5637b147fc70 2023-07-14T16:18:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.754+0000 7fb1beb48b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b148f360 con 0x5637b147ee20 2023-07-14T16:18:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.754+0000 7fb1beb48b80 10 client.0 _ll_register_callbacks cb 0x7fb190004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b148b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b148b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59956/0 (socket says 172.21.15.62:59956) 2023-07-14T16:18:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/1864504083 learned_addr learned my addr 172.21.15.62:0/1864504083 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:36.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147ee20 0x5637b148bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a979c700 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b148eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/1864504083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147ee20 msgr2=0x5637b148bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:36.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147ee20 0x5637b148bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/1864504083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 msgr2=0x5637b148eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:36.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b148eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 -- 172.21.15.62:0/1864504083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637b11e6c60 con 0x5637b147e4d0 2023-07-14T16:18:36.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb1a8f9b700 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b148b590 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb19809acd0 tx=0x7fb1980990e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc263c33f6d4da server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:36.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.755+0000 7fb183fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:36.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb183fff700 1 -- 172.21.15.62:0/1864504083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980b2040 con 0x5637b147e4d0 2023-07-14T16:18:36.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb183fff700 1 -- 172.21.15.62:0/1864504083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1980a3100 con 0x5637b147e4d0 2023-07-14T16:18:36.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 1 -- 172.21.15.62:0/1864504083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb19809ed70 con 0x5637b147e4d0 2023-07-14T16:18:36.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb183fff700 1 -- 172.21.15.62:0/1864504083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1980cb250 con 0x5637b147e4d0 2023-07-14T16:18:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 10 client.16266 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 1 -- 172.21.15.62:0/1864504083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5637b11fc2b0 con 0x5637b147e4d0 2023-07-14T16:18:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb183fff700 1 -- 172.21.15.62:0/1864504083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb19809ed70 con 0x5637b147e4d0 2023-07-14T16:18:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb183fff700 1 -- 172.21.15.62:0/1864504083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb1980b3330 con 0x5637b147e4d0 2023-07-14T16:18:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 10 client.16266 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 20 client.16266 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:36.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637b14e88c0 0x5637b14eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 4 client.16266 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:18:36.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.756+0000 7fb1beb48b80 1 -- 172.21.15.62:0/1864504083 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5637b138b050 con 0x5637b14e88c0 2023-07-14T16:18:36.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.757+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637b14e88c0 0x5637b14eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.757+0000 7fb183fff700 10 client.16266 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:36.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.757+0000 7fb1a3fff700 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637b14e88c0 0x5637b14eaca0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2663971c0f1dacc5 server_cookie=aab548c49ca65298 in_seq=0 out_seq=0 2023-07-14T16:18:36.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb183fff700 1 -- 172.21.15.62:0/1864504083 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5637b138b050 con 0x5637b14e88c0 2023-07-14T16:18:36.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb183fff700 10 client.16266 handle_command_reply: tid=0 2023-07-14T16:18:36.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 client.16266 shutdown 2023-07-14T16:18:36.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 -- 172.21.15.62:0/1864504083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 msgr2=0x5637b148b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b148b590 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb19809acd0 tx=0x7fb1980990e0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 -- 172.21.15.62:0/1864504083 shutdown_connections 2023-07-14T16:18:36.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637b14e88c0 0x5637b14eaca0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637b147fc70 0x5637b148eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637b147e4d0 0x5637b148b590 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 --2- 172.21.15.62:0/1864504083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637b147ee20 0x5637b148bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 -- 172.21.15.62:0/1864504083 >> 172.21.15.62:0/1864504083 conn(0x5637b1473da0 msgr2=0x5637b148fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:36.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb183fff700 0 client.16266 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:36.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 -- 172.21.15.62:0/1864504083 shutdown_connections 2023-07-14T16:18:36.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 1 -- 172.21.15.62:0/1864504083 wait complete. 2023-07-14T16:18:36.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.758+0000 7fb1beb48b80 20 client.16266 trim_cache size 0 max 16384 2023-07-14T16:18:36.743 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:18:36.743 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:18:36.743 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:18:36.743 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:18:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1940a11d0 msgr2=0x7fb1940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1940a11d0 0x7fb1940a3680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb190011490 tx=0x7fb190009260 comp rx=0 tx=0).stop 2023-07-14T16:18:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/2816986661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a40ffae0 msgr2=0x7fb1a41957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a40ffae0 0x7fb1a41957e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb198009fd0 tx=0x7fb19800b790 comp rx=0 tx=0).stop 2023-07-14T16:18:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/2816986661 shutdown_connections 2023-07-14T16:18:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1a4101660 0x7fb1a419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb1940a11d0 0x7fb1940a3680 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a40ffae0 0x7fb1a41957e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 --2- 172.21.15.62:0/2816986661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4100cd0 0x7fb1a4195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/2816986661 >> 172.21.15.62:0/2816986661 conn(0x7fb1a40fb0e0 msgr2=0x7fb1a41042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/2816986661 shutdown_connections 2023-07-14T16:18:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:36.759+0000 7fb1ab1ff700 1 -- 172.21.15.62:0/2816986661 wait complete. 2023-07-14T16:18:36.756 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:18:36.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:18:36.824 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:36 smithi062 ceph-mon[95543]: pgmap v446: 97 pgs: 97 active+clean; 990 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:18:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2170422255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 msgr2=0x7fa2640fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2170422255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 0x7fa2640fff10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa254009990 tx=0x7fa254009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2170422255 shutdown_connections 2023-07-14T16:18:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2170422255 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa264101680 0x7fa264107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2170422255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 0x7fa2640fff10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2170422255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa264100cf0 0x7fa264101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2170422255 >> 172.21.15.62:0/2170422255 conn(0x7fa2640fb0e0 msgr2=0x7fa2640fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2170422255 shutdown_connections 2023-07-14T16:18:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.001+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2170422255 wait complete. 2023-07-14T16:18:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.003+0000 7fa26a9ff700 1 Processor -- start 2023-07-14T16:18:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.003+0000 7fa26a9ff700 1 -- start start 2023-07-14T16:18:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.003+0000 7fa26a9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 0x7fa264193590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.003+0000 7fa26a9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa264100cf0 0x7fa264193ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.003+0000 7fa26a9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa264101680 0x7fa26419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.003+0000 7fa26a9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26419b110 con 0x7fa2640ffb00 2023-07-14T16:18:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.003+0000 7fa26a9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26419b250 con 0x7fa264100cf0 2023-07-14T16:18:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.003+0000 7fa26a9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26419b3e0 con 0x7fa264101680 2023-07-14T16:18:36.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 0x7fa264193590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 0x7fa264193590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59970/0 (socket says 172.21.15.62:59970) 2023-07-14T16:18:36.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 -- 172.21.15.62:0/2611209066 learned_addr learned my addr 172.21.15.62:0/2611209066 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:36.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa2637fe700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa264100cf0 0x7fa264193ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa268f9c700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa264101680 0x7fa26419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 -- 172.21.15.62:0/2611209066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa264101680 msgr2=0x7fa26419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa264101680 0x7fa26419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 -- 172.21.15.62:0/2611209066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa264100cf0 msgr2=0x7fa264193ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa264100cf0 0x7fa264193ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 -- 172.21.15.62:0/2611209066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2540095f0 con 0x7fa2640ffb00 2023-07-14T16:18:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa2637fe700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa264100cf0 0x7fa264193ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:36.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.004+0000 7fa263fff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 0x7fa264193590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa254005910 tx=0x7fa254005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:36.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.005+0000 7fa2617fa700 1 -- 172.21.15.62:0/2611209066 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25401d070 con 0x7fa2640ffb00 2023-07-14T16:18:36.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.005+0000 7fa2617fa700 1 -- 172.21.15.62:0/2611209066 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa25400bd50 con 0x7fa2640ffb00 2023-07-14T16:18:36.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.005+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26419b660 con 0x7fa2640ffb00 2023-07-14T16:18:36.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.005+0000 7fa2617fa700 1 -- 172.21.15.62:0/2611209066 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254022d00 con 0x7fa2640ffb00 2023-07-14T16:18:36.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.005+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26419bae0 con 0x7fa2640ffb00 2023-07-14T16:18:36.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.006+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa264107040 con 0x7fa2640ffb00 2023-07-14T16:18:36.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.008+0000 7fa2617fa700 1 -- 172.21.15.62:0/2611209066 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa25400b8c0 con 0x7fa2640ffb00 2023-07-14T16:18:36.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.011+0000 7fa2617fa700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa24c0a1220 0x7fa24c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.011+0000 7fa2617fa700 1 -- 172.21.15.62:0/2611209066 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa2540cdd20 con 0x7fa2640ffb00 2023-07-14T16:18:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.011+0000 7fa2637fe700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa24c0a1220 0x7fa24c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:36.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.012+0000 7fa2637fe700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa24c0a1220 0x7fa24c0a36d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa258011490 tx=0x7fa258009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:36.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.015+0000 7fa2617fa700 1 -- 172.21.15.62:0/2611209066 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa2540974b0 con 0x7fa2640ffb00 2023-07-14T16:18:37.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:36 smithi164 ceph-mon[97427]: pgmap v446: 97 pgs: 97 active+clean; 990 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:18:37.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:36 smithi081 ceph-mon[127795]: pgmap v446: 97 pgs: 97 active+clean; 990 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:18:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.313+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa26402c850 con 0x7fa2640ffb00 2023-07-14T16:18:37.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.314+0000 7fa2617fa700 1 -- 172.21.15.62:0/2611209066 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10826 (secure 0 0 0) 0x7fa254022e60 con 0x7fa2640ffb00 2023-07-14T16:18:37.291 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:18:37.291 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:17:18.727946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:18:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa24c0a1220 msgr2=0x7fa24c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa24c0a1220 0x7fa24c0a36d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa258011490 tx=0x7fa258009260 comp rx=0 tx=0).stop 2023-07-14T16:18:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 msgr2=0x7fa264193590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 0x7fa264193590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa254005910 tx=0x7fa254005740 comp rx=0 tx=0).stop 2023-07-14T16:18:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 shutdown_connections 2023-07-14T16:18:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa264101680 0x7fa26419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2640ffb00 0x7fa264193590 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa24c0a1220 0x7fa24c0a36d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 --2- 172.21.15.62:0/2611209066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa264100cf0 0x7fa264193ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 >> 172.21.15.62:0/2611209066 conn(0x7fa2640fb0e0 msgr2=0x7fa264104180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 shutdown_connections 2023-07-14T16:18:37.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.317+0000 7fa26a9ff700 1 -- 172.21.15.62:0/2611209066 wait complete. 2023-07-14T16:18:37.296 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2023-07-14T16:18:37.312 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:18:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 -- 172.21.15.62:0/2435953820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c101690 msgr2=0x7f090c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 --2- 172.21.15.62:0/2435953820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c101690 0x7f090c107f50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f090400b0a0 tx=0x7f090400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 -- 172.21.15.62:0/2435953820 shutdown_connections 2023-07-14T16:18:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 --2- 172.21.15.62:0/2435953820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c101690 0x7f090c107f50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 --2- 172.21.15.62:0/2435953820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f090c0ffb10 0x7f090c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 --2- 172.21.15.62:0/2435953820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f090c100d00 0x7f090c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 -- 172.21.15.62:0/2435953820 >> 172.21.15.62:0/2435953820 conn(0x7f090c0fb130 msgr2=0x7f090c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 -- 172.21.15.62:0/2435953820 shutdown_connections 2023-07-14T16:18:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.573+0000 7f09133ff700 1 -- 172.21.15.62:0/2435953820 wait complete. 2023-07-14T16:18:37.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.574+0000 7f09133ff700 1 Processor -- start 2023-07-14T16:18:37.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f09133ff700 1 -- start start 2023-07-14T16:18:37.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f09133ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c0ffb10 0x7f090c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f09133ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f090c100d00 0x7f090c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f09133ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f090c101690 0x7f090c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f09133ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0735e0 con 0x7f090c0ffb10 2023-07-14T16:18:37.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f09133ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c073750 con 0x7f090c100d00 2023-07-14T16:18:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f09133ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c19f8e0 con 0x7f090c101690 2023-07-14T16:18:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f091119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c0ffb10 0x7f090c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f091119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c0ffb10 0x7f090c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59992/0 (socket says 172.21.15.62:59992) 2023-07-14T16:18:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f091119b700 1 -- 172.21.15.62:0/1030126535 learned_addr learned my addr 172.21.15.62:0/1030126535 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.575+0000 7f091099a700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f090c100d00 0x7f090c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f091199c700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f090c101690 0x7f090c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f091119b700 1 -- 172.21.15.62:0/1030126535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f090c101690 msgr2=0x7f090c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f091119b700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f090c101690 0x7f090c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f091119b700 1 -- 172.21.15.62:0/1030126535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f090c100d00 msgr2=0x7f090c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f091119b700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f090c100d00 0x7f090c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f091119b700 1 -- 172.21.15.62:0/1030126535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904009d00 con 0x7f090c0ffb10 2023-07-14T16:18:37.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f091199c700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f090c101690 0x7f090c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:37.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f091119b700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c0ffb10 0x7f090c0742c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f090800e970 tx=0x7f090800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f09027fc700 1 -- 172.21.15.62:0/1030126535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908009790 con 0x7f090c0ffb10 2023-07-14T16:18:37.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f09133ff700 1 -- 172.21.15.62:0/1030126535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c19fbc0 con 0x7f090c0ffb10 2023-07-14T16:18:37.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f09027fc700 1 -- 172.21.15.62:0/1030126535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f090800eec0 con 0x7f090c0ffb10 2023-07-14T16:18:37.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f09027fc700 1 -- 172.21.15.62:0/1030126535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908003c10 con 0x7f090c0ffb10 2023-07-14T16:18:37.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.576+0000 7f09133ff700 1 -- 172.21.15.62:0/1030126535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c1a00d0 con 0x7f090c0ffb10 2023-07-14T16:18:37.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.581+0000 7f09027fc700 1 -- 172.21.15.62:0/1030126535 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0908003db0 con 0x7f090c0ffb10 2023-07-14T16:18:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.582+0000 7f0926c7fb80 1 Processor -- start 2023-07-14T16:18:37.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f09027fc700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08f80a1150 0x7f08f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f0926c7fb80 1 -- start start 2023-07-14T16:18:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f09027fc700 1 -- 172.21.15.62:0/1030126535 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0908014070 con 0x7f090c0ffb10 2023-07-14T16:18:37.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f55e20 0x55b417f566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f56c70 0x55b417f570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417e5acb0 con 0x55b417f56c70 2023-07-14T16:18:37.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417eada40 con 0x55b417f55e20 2023-07-14T16:18:37.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417ea6ed0 con 0x55b417f554d0 2023-07-14T16:18:37.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.583+0000 7f091199c700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08f80a1150 0x7f08f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091099a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f55e20 0x55b417f566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091099a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f55e20 0x55b417f566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59180/0 (socket says 172.21.15.62:59180) 2023-07-14T16:18:37.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091099a700 1 -- 172.21.15.62:0/331253015 learned_addr learned my addr 172.21.15.62:0/331253015 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091119b700 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091199c700 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f56c70 0x55b417f570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091099a700 1 -- 172.21.15.62:0/331253015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 msgr2=0x55b417f558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091099a700 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091099a700 1 -- 172.21.15.62:0/331253015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f56c70 msgr2=0x55b417f570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091099a700 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f56c70 0x55b417f570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091099a700 1 -- 172.21.15.62:0/331253015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b417cd4de0 con 0x55b417f55e20 2023-07-14T16:18:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091119b700 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.584+0000 7f091199c700 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f56c70 0x55b417f570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:18:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.585+0000 7f091199c700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08f80a1150 0x7f08f80a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f090c074920 tx=0x7f0904007540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.585+0000 7f091099a700 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f55e20 0x55b417f566e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009d80 tx=0x7f08fc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4876b5efb1a736c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.586+0000 7f08f77fe700 1 -- 172.21.15.62:0/331253015 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc004230 con 0x55b417f55e20 2023-07-14T16:18:37.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.586+0000 7f08f77fe700 1 -- 172.21.15.62:0/331253015 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08fc004390 con 0x55b417f55e20 2023-07-14T16:18:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.586+0000 7f08f77fe700 1 -- 172.21.15.62:0/331253015 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc019800 con 0x55b417f55e20 2023-07-14T16:18:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 -- 172.21.15.62:0/331253015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f55e20 msgr2=0x55b417f566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f55e20 0x55b417f566e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009d80 tx=0x7f08fc011720 comp rx=0 tx=0).stop 2023-07-14T16:18:37.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 -- 172.21.15.62:0/331253015 shutdown_connections 2023-07-14T16:18:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f56c70 0x55b417f570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/331253015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f55e20 0x55b417f566e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 -- 172.21.15.62:0/331253015 >> 172.21.15.62:0/331253015 conn(0x55b417f4ada0 msgr2=0x55b417f501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 -- 172.21.15.62:0/331253015 shutdown_connections 2023-07-14T16:18:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.587+0000 7f0926c7fb80 1 -- 172.21.15.62:0/331253015 wait complete. 2023-07-14T16:18:37.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.588+0000 7f0926c7fb80 1 -- start start 2023-07-14T16:18:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.588+0000 7f0926c7fb80 1 Processor -- start 2023-07-14T16:18:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.590+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f67950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.590+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f67e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.590+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 0x55b417f6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.590+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417e46ac0 con 0x55b417f55e20 2023-07-14T16:18:37.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.590+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417e467c0 con 0x55b417f554d0 2023-07-14T16:18:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.590+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417d2a920 con 0x55b417f56c70 2023-07-14T16:18:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.590+0000 7f0926c7fb80 10 client.0 _ll_register_callbacks cb 0x55b417e44760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f67950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f67950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59182/0 (socket says 172.21.15.62:59182) 2023-07-14T16:18:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 -- 172.21.15.62:0/797888647 learned_addr learned my addr 172.21.15.62:0/797888647 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091099a700 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f67e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091199c700 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 0x55b417f6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 -- 172.21.15.62:0/797888647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 msgr2=0x55b417f6eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 0x55b417f6eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 -- 172.21.15.62:0/797888647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 msgr2=0x55b417f67e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f67e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 -- 172.21.15.62:0/797888647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c1a00d0 con 0x55b417f554d0 2023-07-14T16:18:37.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f091119b700 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f67950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f090800c5b0 tx=0x7f090800cdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=487bbe28c6150227 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.591+0000 7f08f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:18:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.592+0000 7f08f4ff9700 1 -- 172.21.15.62:0/797888647 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908099660 con 0x55b417f554d0 2023-07-14T16:18:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.592+0000 7f08f4ff9700 1 -- 172.21.15.62:0/797888647 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0908099f70 con 0x55b417f554d0 2023-07-14T16:18:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.592+0000 7f0926c7fb80 1 -- 172.21.15.62:0/797888647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b417e004f0 con 0x55b417f554d0 2023-07-14T16:18:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.592+0000 7f08f4ff9700 1 -- 172.21.15.62:0/797888647 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f090809dfa0 con 0x55b417f554d0 2023-07-14T16:18:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.592+0000 7f0926c7fb80 10 client.26027 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.592+0000 7f0926c7fb80 1 -- 172.21.15.62:0/797888647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f08fc0099e0 con 0x55b417f554d0 2023-07-14T16:18:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f08f4ff9700 1 -- 172.21.15.62:0/797888647 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b417e004f0 con 0x55b417f554d0 2023-07-14T16:18:37.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f08f4ff9700 1 -- 172.21.15.62:0/797888647 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f09080a1f30 con 0x55b417f554d0 2023-07-14T16:18:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f0926c7fb80 10 client.26027 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f0926c7fb80 10 client.26027 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:18:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f0926c7fb80 20 client.26027 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b417f735a0 0x55b417f759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f0926c7fb80 4 client.26027 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f0926c7fb80 1 -- 172.21.15.62:0/797888647 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b417e9feb0 con 0x55b417f735a0 2023-07-14T16:18:37.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.593+0000 7f091099a700 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b417f735a0 0x55b417f759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.594+0000 7f091099a700 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b417f735a0 0x55b417f759b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db0a1fb5ff2abdc7 server_cookie=fa14662b3a319dbb in_seq=0 out_seq=0 2023-07-14T16:18:37.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.594+0000 7f08f4ff9700 10 client.26027 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:18:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f08f4ff9700 1 -- 172.21.15.62:0/797888647 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b417e9feb0 con 0x55b417f735a0 2023-07-14T16:18:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f08f4ff9700 10 client.26027 handle_command_reply: tid=0 2023-07-14T16:18:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 client.26027 shutdown 2023-07-14T16:18:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 -- 172.21.15.62:0/797888647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 msgr2=0x55b417f67950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f67950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f090800c5b0 tx=0x7f090800cdc0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 -- 172.21.15.62:0/797888647 shutdown_connections 2023-07-14T16:18:37.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b417f735a0 0x55b417f759b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 0x55b417f6eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f67950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/797888647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f67e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f08f4ff9700 0 client.26027 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:18:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 -- 172.21.15.62:0/797888647 >> 172.21.15.62:0/797888647 conn(0x55b417f4ada0 msgr2=0x55b417f4f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 -- 172.21.15.62:0/797888647 shutdown_connections 2023-07-14T16:18:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.595+0000 7f0926c7fb80 1 -- 172.21.15.62:0/797888647 wait complete. 2023-07-14T16:18:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.596+0000 7f0926c7fb80 20 client.26027 trim_cache size 0 max 16384 2023-07-14T16:18:37.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.609+0000 7f0926c7fb80 1 Processor -- start 2023-07-14T16:18:37.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.609+0000 7f0926c7fb80 1 -- start start 2023-07-14T16:18:37.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f5d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f56c70 0x55b417f59c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417f561f0 con 0x55b417f55e20 2023-07-14T16:18:37.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417f4a970 con 0x55b417f56c70 2023-07-14T16:18:37.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417f51060 con 0x55b417f554d0 2023-07-14T16:18:37.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f091099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f091099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60022/0 (socket says 172.21.15.62:60022) 2023-07-14T16:18:37.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f091099a700 1 -- 172.21.15.62:0/3036901628 learned_addr learned my addr 172.21.15.62:0/3036901628 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.610+0000 7f091199c700 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f56c70 0x55b417f59c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091119b700 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f5d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091199c700 1 -- 172.21.15.62:0/3036901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 msgr2=0x55b417f5d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091199c700 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f5d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091199c700 1 -- 172.21.15.62:0/3036901628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 msgr2=0x55b417f6ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091199c700 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091199c700 1 -- 172.21.15.62:0/3036901628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b417cbdc60 con 0x55b417f56c70 2023-07-14T16:18:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091099a700 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091119b700 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f5d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.611+0000 7f091199c700 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f56c70 0x55b417f59c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f090400b070 tx=0x7f090400e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769492244dbcadee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f08f7fff700 1 -- 172.21.15.62:0/3036901628 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090400e070 con 0x55b417f56c70 2023-07-14T16:18:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f08f7fff700 1 -- 172.21.15.62:0/3036901628 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0904007e00 con 0x55b417f56c70 2023-07-14T16:18:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f08f7fff700 1 -- 172.21.15.62:0/3036901628 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904020be0 con 0x55b417f56c70 2023-07-14T16:18:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f0926c7fb80 1 -- 172.21.15.62:0/3036901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f56c70 msgr2=0x55b417f59c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f56c70 0x55b417f59c70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f090400b070 tx=0x7f090400e000 comp rx=0 tx=0).stop 2023-07-14T16:18:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f0926c7fb80 1 -- 172.21.15.62:0/3036901628 shutdown_connections 2023-07-14T16:18:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f56c70 0x55b417f59c70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f554d0 0x55b417f5d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/3036901628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f0926c7fb80 1 -- 172.21.15.62:0/3036901628 >> 172.21.15.62:0/3036901628 conn(0x55b417f4ada0 msgr2=0x55b417f66620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.612+0000 7f0926c7fb80 1 -- 172.21.15.62:0/3036901628 shutdown_connections 2023-07-14T16:18:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.613+0000 7f0926c7fb80 1 -- 172.21.15.62:0/3036901628 wait complete. 2023-07-14T16:18:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.613+0000 7f0926c7fb80 1 -- start start 2023-07-14T16:18:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.613+0000 7f0926c7fb80 1 Processor -- start 2023-07-14T16:18:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f6ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f0926c7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 0x55b417f6d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417f6f2f0 con 0x55b417f55e20 2023-07-14T16:18:37.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417f6db60 con 0x55b417f554d0 2023-07-14T16:18:37.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f0926c7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b417f6dd20 con 0x55b417f56c70 2023-07-14T16:18:37.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f0926c7fb80 10 client.0 _ll_register_callbacks cb 0x7f0908099b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:37.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f091119b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f6ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f091119b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f6ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59196/0 (socket says 172.21.15.62:59196) 2023-07-14T16:18:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.614+0000 7f091119b700 1 -- 172.21.15.62:0/2241495128 learned_addr learned my addr 172.21.15.62:0/2241495128 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091099a700 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091199c700 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 0x55b417f6d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091119b700 1 -- 172.21.15.62:0/2241495128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 msgr2=0x55b417f6d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091119b700 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 0x55b417f6d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091119b700 1 -- 172.21.15.62:0/2241495128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 msgr2=0x55b417f6d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091119b700 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091119b700 1 -- 172.21.15.62:0/2241495128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b417cd4de0 con 0x55b417f554d0 2023-07-14T16:18:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091099a700 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f091119b700 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f6ead0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f090809a5d0 tx=0x7f090809a4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7ab578dc581d10 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f08f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:18:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f08f4ff9700 1 -- 172.21.15.62:0/2241495128 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090809dfa0 con 0x55b417f554d0 2023-07-14T16:18:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f0926c7fb80 1 -- 172.21.15.62:0/2241495128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08fc004390 con 0x55b417f554d0 2023-07-14T16:18:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.615+0000 7f08f4ff9700 1 -- 172.21.15.62:0/2241495128 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f09080a23c0 con 0x55b417f554d0 2023-07-14T16:18:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f08f4ff9700 1 -- 172.21.15.62:0/2241495128 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09080a1f30 con 0x55b417f554d0 2023-07-14T16:18:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f0926c7fb80 10 client.26033 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f0926c7fb80 1 -- 172.21.15.62:0/2241495128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f0904025040 con 0x55b417f554d0 2023-07-14T16:18:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f08f4ff9700 1 -- 172.21.15.62:0/2241495128 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0908078e90 con 0x55b417f554d0 2023-07-14T16:18:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f08f4ff9700 1 -- 172.21.15.62:0/2241495128 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f09080994f0 con 0x55b417f554d0 2023-07-14T16:18:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f0926c7fb80 10 client.26033 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f0926c7fb80 10 client.26033 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:18:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f0926c7fb80 20 client.26033 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b417fc87f0 0x55b417fcac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f0926c7fb80 4 client.26033 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:18:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.616+0000 7f0926c7fb80 1 -- 172.21.15.62:0/2241495128 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b417e62050 con 0x55b417fc87f0 2023-07-14T16:18:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.617+0000 7f091099a700 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b417fc87f0 0x55b417fcac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.617+0000 7f091099a700 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b417fc87f0 0x55b417fcac00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e145f9df6d1b10ab server_cookie=d9f0b305ab3341c7 in_seq=0 out_seq=0 2023-07-14T16:18:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.617+0000 7f08f4ff9700 10 client.26033 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:18:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f08f4ff9700 1 -- 172.21.15.62:0/2241495128 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b417e62050 con 0x55b417fc87f0 2023-07-14T16:18:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f08f4ff9700 10 client.26033 handle_command_reply: tid=0 2023-07-14T16:18:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 client.26033 shutdown 2023-07-14T16:18:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 -- 172.21.15.62:0/2241495128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 msgr2=0x55b417f6ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f6ead0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f090809a5d0 tx=0x7f090809a4b0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 -- 172.21.15.62:0/2241495128 shutdown_connections 2023-07-14T16:18:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55b417fc87f0 0x55b417fcac00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b417f56c70 0x55b417f6d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b417f554d0 0x55b417f6ead0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f08f4ff9700 0 client.26033 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:18:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 --2- 172.21.15.62:0/2241495128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b417f55e20 0x55b417f6d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 -- 172.21.15.62:0/2241495128 >> 172.21.15.62:0/2241495128 conn(0x55b417f4ada0 msgr2=0x55b417f5d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 -- 172.21.15.62:0/2241495128 shutdown_connections 2023-07-14T16:18:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 1 -- 172.21.15.62:0/2241495128 wait complete. 2023-07-14T16:18:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.618+0000 7f0926c7fb80 20 client.26033 trim_cache size 0 max 16384 2023-07-14T16:18:37.603 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:18:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 -- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08f80a1150 msgr2=0x7f08f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08f80a1150 0x7f08f80a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f090c074920 tx=0x7f0904007540 comp rx=0 tx=0).stop 2023-07-14T16:18:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 -- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c0ffb10 msgr2=0x7f090c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c0ffb10 0x7f090c0742c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f090800e970 tx=0x7f090800ec80 comp rx=0 tx=0).stop 2023-07-14T16:18:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 -- 172.21.15.62:0/1030126535 shutdown_connections 2023-07-14T16:18:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f090c101690 0x7f090c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f090c0ffb10 0x7f090c0742c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08f80a1150 0x7f08f80a3600 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 --2- 172.21.15.62:0/1030126535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f090c100d00 0x7f090c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 -- 172.21.15.62:0/1030126535 >> 172.21.15.62:0/1030126535 conn(0x7f090c0fb130 msgr2=0x7f090c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 -- 172.21.15.62:0/1030126535 shutdown_connections 2023-07-14T16:18:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.619+0000 7f09133ff700 1 -- 172.21.15.62:0/1030126535 wait complete. 2023-07-14T16:18:37.614 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:18:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/360275819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 msgr2=0x7f6cc0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/360275819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 0x7f6cc0101030 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009990 tx=0x7f6cb4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/360275819 shutdown_connections 2023-07-14T16:18:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/360275819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cc0101570 0x7f6cc0107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/360275819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cc00ff9f0 0x7f6cc00ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/360275819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 0x7f6cc0101030 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/360275819 >> 172.21.15.62:0/360275819 conn(0x7f6cc00fb010 msgr2=0x7f6cc00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/360275819 shutdown_connections 2023-07-14T16:18:37.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.879+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/360275819 wait complete. 2023-07-14T16:18:37.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.880+0000 7f6cc4f9d700 1 Processor -- start 2023-07-14T16:18:37.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.880+0000 7f6cc4f9d700 1 -- start start 2023-07-14T16:18:37.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cc4f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cc00ff9f0 0x7f6cc00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cc4f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 0x7f6cc0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cc4f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cc0101570 0x7f6cc0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cc4f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc00735e0 con 0x7f6cc0100be0 2023-07-14T16:18:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cc4f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0073750 con 0x7f6cc00ff9f0 2023-07-14T16:18:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cc4f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc019f5a0 con 0x7f6cc0101570 2023-07-14T16:18:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 0x7f6cc0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 0x7f6cc0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60050/0 (socket says 172.21.15.62:60050) 2023-07-14T16:18:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbeffd700 1 -- 172.21.15.62:0/1003657275 learned_addr learned my addr 172.21.15.62:0/1003657275 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbffff700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cc0101570 0x7f6cc0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cc00ff9f0 0x7f6cc00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbeffd700 1 -- 172.21.15.62:0/1003657275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cc0101570 msgr2=0x7f6cc0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cc0101570 0x7f6cc0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbeffd700 1 -- 172.21.15.62:0/1003657275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cc00ff9f0 msgr2=0x7f6cc00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cc00ff9f0 0x7f6cc00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.881+0000 7f6cbeffd700 1 -- 172.21.15.62:0/1003657275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40095f0 con 0x7f6cc0100be0 2023-07-14T16:18:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.882+0000 7f6cbffff700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cc0101570 0x7f6cc0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:18:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.882+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cc00ff9f0 0x7f6cc00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:18:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.882+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 0x7f6cc0072900 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40095c0 tx=0x7f6cb4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.882+0000 7f6cbcff9700 1 -- 172.21.15.62:0/1003657275 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb401d070 con 0x7f6cc0100be0 2023-07-14T16:18:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.882+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/1003657275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc019f820 con 0x7f6cc0100be0 2023-07-14T16:18:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.882+0000 7f6cbcff9700 1 -- 172.21.15.62:0/1003657275 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6cb4003fc0 con 0x7f6cc0100be0 2023-07-14T16:18:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.882+0000 7f6cbcff9700 1 -- 172.21.15.62:0/1003657275 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4022d00 con 0x7f6cc0100be0 2023-07-14T16:18:37.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.882+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/1003657275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc019fca0 con 0x7f6cc0100be0 2023-07-14T16:18:37.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.888+0000 7f6cbcff9700 1 -- 172.21.15.62:0/1003657275 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6cb4003b30 con 0x7f6cc0100be0 2023-07-14T16:18:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.888+0000 7f6cd9aaab80 1 Processor -- start 2023-07-14T16:18:37.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cd9aaab80 1 -- start start 2023-07-14T16:18:37.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21ce070 0x5573d21ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cee30 0x5573d21cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d20d2cb0 con 0x5573d21ce070 2023-07-14T16:18:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d2125a40 con 0x5573d21cd720 2023-07-14T16:18:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d211eed0 con 0x5573d21cee30 2023-07-14T16:18:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cbf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21ce070 0x5573d21ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.889+0000 7f6cbf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21ce070 0x5573d21ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60064/0 (socket says 172.21.15.62:60064) 2023-07-14T16:18:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbf7fe700 1 -- 172.21.15.62:0/1397514279 learned_addr learned my addr 172.21.15.62:0/1397514279 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cee30 0x5573d21cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbffff700 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbcff9700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6ca80a11f0 0x7f6ca80a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbf7fe700 1 -- 172.21.15.62:0/1397514279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cee30 msgr2=0x5573d21cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cee30 0x5573d21cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbf7fe700 1 -- 172.21.15.62:0/1397514279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 msgr2=0x5573d21cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbcff9700 1 -- 172.21.15.62:0/1003657275 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cb40cdd50 con 0x7f6cc0100be0 2023-07-14T16:18:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbf7fe700 1 -- 172.21.15.62:0/1397514279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573d1f4cde0 con 0x5573d21ce070 2023-07-14T16:18:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbffff700 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbffff700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6ca80a11f0 0x7f6ca80a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.890+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21ce070 0x5573d21ce8a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009cc0 tx=0x7f6cb000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6f7ead64103835 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.891+0000 7f6ca5ffb700 1 -- 172.21.15.62:0/1397514279 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000eb40 con 0x5573d21ce070 2023-07-14T16:18:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.891+0000 7f6ca5ffb700 1 -- 172.21.15.62:0/1397514279 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6cb000eca0 con 0x5573d21ce070 2023-07-14T16:18:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.891+0000 7f6ca5ffb700 1 -- 172.21.15.62:0/1397514279 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0010630 con 0x5573d21ce070 2023-07-14T16:18:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/1397514279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21ce070 msgr2=0x5573d21ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21ce070 0x5573d21ce8a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009cc0 tx=0x7f6cb000e2f0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cbffff700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6ca80a11f0 0x7f6ca80a36a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0074920 tx=0x7f6cb800a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/1397514279 shutdown_connections 2023-07-14T16:18:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cee30 0x5573d21cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/1397514279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21ce070 0x5573d21ce8a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/1397514279 >> 172.21.15.62:0/1397514279 conn(0x5573d21c2ff0 msgr2=0x5573d21c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/1397514279 shutdown_connections 2023-07-14T16:18:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/1397514279 wait complete. 2023-07-14T16:18:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.892+0000 7f6cd9aaab80 1 -- start start 2023-07-14T16:18:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.893+0000 7f6cd9aaab80 1 Processor -- start 2023-07-14T16:18:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.893+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cd720 0x5573d21d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.893+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21d7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.893+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cee30 0x5573d21de890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.893+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d20beac0 con 0x5573d21cd720 2023-07-14T16:18:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.893+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d20be7c0 con 0x5573d21cee30 2023-07-14T16:18:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.893+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d1fa2920 con 0x5573d21ce070 2023-07-14T16:18:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.893+0000 7f6cd9aaab80 10 client.0 _ll_register_callbacks cb 0x5573d20bc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cd720 0x5573d21d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cd720 0x5573d21d7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60072/0 (socket says 172.21.15.62:60072) 2023-07-14T16:18:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 -- 172.21.15.62:0/2787947628 learned_addr learned my addr 172.21.15.62:0/2787947628 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbffff700 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cee30 0x5573d21de890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21d7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 -- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 msgr2=0x5573d21d7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21d7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 -- 172.21.15.62:0/2787947628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cee30 msgr2=0x5573d21de890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cee30 0x5573d21de890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 -- 172.21.15.62:0/2787947628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb0009920 con 0x5573d21cd720 2023-07-14T16:18:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cd720 0x5573d21d7310 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00098f0 tx=0x7f6cb0011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d0e72720ac1c40 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.894+0000 7f6c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.895+0000 7f6c9effd700 1 -- 172.21.15.62:0/2787947628 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0021070 con 0x5573d21cd720 2023-07-14T16:18:37.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.895+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/2787947628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573d20784f0 con 0x5573d21cd720 2023-07-14T16:18:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.895+0000 7f6c9effd700 1 -- 172.21.15.62:0/2787947628 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cb0011680 con 0x5573d21cd720 2023-07-14T16:18:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.895+0000 7f6c9effd700 1 -- 172.21.15.62:0/2787947628 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cb0011840 con 0x5573d21cd720 2023-07-14T16:18:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.895+0000 7f6cd9aaab80 10 client.16299 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.895+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/2787947628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5573d1f35c60 con 0x5573d21cd720 2023-07-14T16:18:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.895+0000 7f6c9effd700 1 -- 172.21.15.62:0/2787947628 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5573d20784f0 con 0x5573d21cd720 2023-07-14T16:18:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.896+0000 7f6c9effd700 1 -- 172.21.15.62:0/2787947628 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6cb000c070 con 0x5573d21cd720 2023-07-14T16:18:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.896+0000 7f6cd9aaab80 10 client.16299 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.896+0000 7f6cd9aaab80 10 client.16299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:18:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.896+0000 7f6cd9aaab80 20 client.16299 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.896+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5573d21e3010 0x5573d21e5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.896+0000 7f6cd9aaab80 4 client.16299 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.896+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/2787947628 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573d2117eb0 con 0x5573d21e3010 2023-07-14T16:18:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.896+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5573d21e3010 0x5573d21e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.897+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5573d21e3010 0x5573d21e5420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d61ad84f20765ff1 server_cookie=cabfb08b32c389a3 in_seq=0 out_seq=0 2023-07-14T16:18:37.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.897+0000 7f6c9effd700 10 client.16299 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:18:37.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:37 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2611209066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:18:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6c9effd700 1 -- 172.21.15.62:0/2787947628 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573d2117eb0 con 0x5573d21e3010 2023-07-14T16:18:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6c9effd700 10 client.16299 handle_command_reply: tid=0 2023-07-14T16:18:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 client.16299 shutdown 2023-07-14T16:18:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/2787947628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cd720 msgr2=0x5573d21d7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cd720 0x5573d21d7310 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00098f0 tx=0x7f6cb0011d00 comp rx=0 tx=0).stop 2023-07-14T16:18:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/2787947628 shutdown_connections 2023-07-14T16:18:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5573d21e3010 0x5573d21e5420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cee30 0x5573d21de890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cd720 0x5573d21d7310 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6c9effd700 0 client.16299 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/2787947628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/2787947628 >> 172.21.15.62:0/2787947628 conn(0x5573d21c2ff0 msgr2=0x5573d21c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/2787947628 shutdown_connections 2023-07-14T16:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.898+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/2787947628 wait complete. 2023-07-14T16:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.899+0000 7f6cd9aaab80 20 client.16299 trim_cache size 0 max 16384 2023-07-14T16:18:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.917+0000 7f6cd9aaab80 1 Processor -- start 2023-07-14T16:18:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.917+0000 7f6cd9aaab80 1 -- start start 2023-07-14T16:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cd720 0x5573d21d69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21ce070 0x5573d21de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d21c91f0 con 0x5573d21cee30 2023-07-14T16:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d21ce6c0 con 0x5573d21ce070 2023-07-14T16:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d21e2d10 con 0x5573d21cd720 2023-07-14T16:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cbffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cbffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21d6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60086/0 (socket says 172.21.15.62:60086) 2023-07-14T16:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cbffff700 1 -- 172.21.15.62:0/4206771058 learned_addr learned my addr 172.21.15.62:0/4206771058 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cd720 0x5573d21d69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21ce070 0x5573d21de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cbffff700 1 -- 172.21.15.62:0/4206771058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cd720 msgr2=0x5573d21d69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.918+0000 7f6cbffff700 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cd720 0x5573d21d69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.919+0000 7f6cbffff700 1 -- 172.21.15.62:0/4206771058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21ce070 msgr2=0x5573d21de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.919+0000 7f6cbffff700 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21ce070 0x5573d21de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.919+0000 7f6cbffff700 1 -- 172.21.15.62:0/4206771058 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573d1f73250 con 0x5573d21cee30 2023-07-14T16:18:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.919+0000 7f6cbffff700 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21d6010 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80157b0 tx=0x7f6cb8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333516ee78156444 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.919+0000 7f6ca67fc700 1 -- 172.21.15.62:0/4206771058 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8022600 con 0x5573d21cee30 2023-07-14T16:18:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.919+0000 7f6ca67fc700 1 -- 172.21.15.62:0/4206771058 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6cb8022bf0 con 0x5573d21cee30 2023-07-14T16:18:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.919+0000 7f6ca67fc700 1 -- 172.21.15.62:0/4206771058 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8014a60 con 0x5573d21cee30 2023-07-14T16:18:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/4206771058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 msgr2=0x5573d21d6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21d6010 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80157b0 tx=0x7f6cb8012000 comp rx=0 tx=0).stop 2023-07-14T16:18:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/4206771058 shutdown_connections 2023-07-14T16:18:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21d6010 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21cd720 0x5573d21d69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/4206771058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21ce070 0x5573d21de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/4206771058 >> 172.21.15.62:0/4206771058 conn(0x5573d21c2ff0 msgr2=0x5573d21e9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/4206771058 shutdown_connections 2023-07-14T16:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.920+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/4206771058 wait complete. 2023-07-14T16:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 1 -- start start 2023-07-14T16:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 1 Processor -- start 2023-07-14T16:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21daa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d21de1d0 con 0x5573d21cee30 2023-07-14T16:18:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d21de340 con 0x5573d21cd720 2023-07-14T16:18:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573d21de500 con 0x5573d21ce070 2023-07-14T16:18:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.921+0000 7f6cd9aaab80 10 client.0 _ll_register_callbacks cb 0x7f6cb4097370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21ddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60100/0 (socket says 172.21.15.62:60100) 2023-07-14T16:18:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbffff700 1 -- 172.21.15.62:0/3561407345 learned_addr learned my addr 172.21.15.62:0/3561407345 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbeffd700 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbffff700 1 -- 172.21.15.62:0/3561407345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 msgr2=0x5573d21dafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbffff700 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21dafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbffff700 1 -- 172.21.15.62:0/3561407345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 msgr2=0x5573d21daa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbffff700 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21daa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.922+0000 7f6cbffff700 1 -- 172.21.15.62:0/3561407345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb8015460 con 0x5573d21cee30 2023-07-14T16:18:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6cbffff700 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21ddc90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800a010 tx=0x7f6cb8022a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd539207bdfee36 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6c9effd700 1 -- 172.21.15.62:0/3561407345 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8014b90 con 0x5573d21cee30 2023-07-14T16:18:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6c9effd700 1 -- 172.21.15.62:0/3561407345 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cb8032070 con 0x5573d21cee30 2023-07-14T16:18:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/3561407345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cb000eca0 con 0x5573d21cee30 2023-07-14T16:18:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6c9effd700 1 -- 172.21.15.62:0/3561407345 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cb8014cf0 con 0x5573d21cee30 2023-07-14T16:18:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6cd9aaab80 10 client.16305 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/3561407345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5573d1f4b2b0 con 0x5573d21cee30 2023-07-14T16:18:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.923+0000 7f6c9effd700 1 -- 172.21.15.62:0/3561407345 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6cb000eca0 con 0x5573d21cee30 2023-07-14T16:18:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.924+0000 7f6c9effd700 1 -- 172.21.15.62:0/3561407345 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6cb8040070 con 0x5573d21cee30 2023-07-14T16:18:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.924+0000 7f6cd9aaab80 10 client.16305 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.924+0000 7f6cd9aaab80 10 client.16305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:18:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.924+0000 7f6cd9aaab80 20 client.16305 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:37.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.924+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5573d2237fb0 0x5573d223a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:37.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.924+0000 7f6cd9aaab80 4 client.16305 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:18:37.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.924+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/3561407345 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573d20da050 con 0x5573d2237fb0 2023-07-14T16:18:37.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.924+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5573d2237fb0 0x5573d223a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:37.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cbf7fe700 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5573d2237fb0 0x5573d223a3c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87114108df467c0b server_cookie=74232ef79335f96a in_seq=0 out_seq=0 2023-07-14T16:18:37.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6c9effd700 10 client.16305 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:18:37.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6c9effd700 1 -- 172.21.15.62:0/3561407345 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573d20da050 con 0x5573d2237fb0 2023-07-14T16:18:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6c9effd700 10 client.16305 handle_command_reply: tid=0 2023-07-14T16:18:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 client.16305 shutdown 2023-07-14T16:18:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/3561407345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 msgr2=0x5573d21ddc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21ddc90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800a010 tx=0x7f6cb8022a60 comp rx=0 tx=0).stop 2023-07-14T16:18:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/3561407345 shutdown_connections 2023-07-14T16:18:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5573d2237fb0 0x5573d223a3c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573d21cee30 0x5573d21ddc90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573d21cd720 0x5573d21daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 --2- 172.21.15.62:0/3561407345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573d21ce070 0x5573d21dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/3561407345 >> 172.21.15.62:0/3561407345 conn(0x5573d21c2ff0 msgr2=0x5573d21d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6c9effd700 0 client.16305 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:18:37.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/3561407345 shutdown_connections 2023-07-14T16:18:37.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.925+0000 7f6cd9aaab80 1 -- 172.21.15.62:0/3561407345 wait complete. 2023-07-14T16:18:37.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cd9aaab80 20 client.16305 trim_cache size 0 max 16384 2023-07-14T16:18:37.910 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:18:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6ca80a11f0 msgr2=0x7f6ca80a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6ca80a11f0 0x7f6ca80a36a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0074920 tx=0x7f6cb800a040 comp rx=0 tx=0).stop 2023-07-14T16:18:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 msgr2=0x7f6cc0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 0x7f6cc0072900 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40095c0 tx=0x7f6cb4005ef0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/1003657275 shutdown_connections 2023-07-14T16:18:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6cc0101570 0x7f6cc0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6ca80a11f0 0x7f6ca80a36a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cc00ff9f0 0x7f6cc00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 --2- 172.21.15.62:0/1003657275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cc0100be0 0x7f6cc0072900 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.926+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/1003657275 >> 172.21.15.62:0/1003657275 conn(0x7f6cc00fb010 msgr2=0x7f6cc01040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.927+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/1003657275 shutdown_connections 2023-07-14T16:18:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:37.927+0000 7f6cc4f9d700 1 -- 172.21.15.62:0/1003657275 wait complete. 2023-07-14T16:18:37.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:18:38.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:37 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2611209066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:18:38.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:37 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2611209066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:18:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 -- 172.21.15.62:0/3083937360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 msgr2=0x7f7b38101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/3083937360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 0x7f7b38101120 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009990 tx=0x7f7b20009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 -- 172.21.15.62:0/3083937360 shutdown_connections 2023-07-14T16:18:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/3083937360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b38101660 0x7f7b38107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/3083937360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b380ffae0 0x7f7b380ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/3083937360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 0x7f7b38101120 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 -- 172.21.15.62:0/3083937360 >> 172.21.15.62:0/3083937360 conn(0x7f7b380fb0e0 msgr2=0x7f7b380fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:38.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 -- 172.21.15.62:0/3083937360 shutdown_connections 2023-07-14T16:18:38.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.184+0000 7f7b3dbff700 1 -- 172.21.15.62:0/3083937360 wait complete. 2023-07-14T16:18:38.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.185+0000 7f7b3dbff700 1 Processor -- start 2023-07-14T16:18:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b3dbff700 1 -- start start 2023-07-14T16:18:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b3dbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b380ffae0 0x7f7b38197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b3dbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 0x7f7b38197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b3dbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b38101660 0x7f7b3819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b3dbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3819f590 con 0x7f7b38100cd0 2023-07-14T16:18:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b3dbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3819f6d0 con 0x7f7b380ffae0 2023-07-14T16:18:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b3dbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3819f860 con 0x7f7b38101660 2023-07-14T16:18:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b36ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 0x7f7b38197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b36ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 0x7f7b38197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60116/0 (socket says 172.21.15.62:60116) 2023-07-14T16:18:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b36ffd700 1 -- 172.21.15.62:0/2620878457 learned_addr learned my addr 172.21.15.62:0/2620878457 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.186+0000 7f7b377fe700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b380ffae0 0x7f7b38197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b37fff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b38101660 0x7f7b3819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b36ffd700 1 -- 172.21.15.62:0/2620878457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b38101660 msgr2=0x7f7b3819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b36ffd700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b38101660 0x7f7b3819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b36ffd700 1 -- 172.21.15.62:0/2620878457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b380ffae0 msgr2=0x7f7b38197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b36ffd700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b380ffae0 0x7f7b38197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b36ffd700 1 -- 172.21.15.62:0/2620878457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200095f0 con 0x7f7b38100cd0 2023-07-14T16:18:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b37fff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b38101660 0x7f7b3819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:18:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b36ffd700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 0x7f7b38197f50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005b40 tx=0x7f7b20003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b34ff9700 1 -- 172.21.15.62:0/2620878457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2001d070 con 0x7f7b38100cd0 2023-07-14T16:18:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b3dbff700 1 -- 172.21.15.62:0/2620878457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3819fae0 con 0x7f7b38100cd0 2023-07-14T16:18:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b34ff9700 1 -- 172.21.15.62:0/2620878457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b200056e0 con 0x7f7b38100cd0 2023-07-14T16:18:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.187+0000 7f7b34ff9700 1 -- 172.21.15.62:0/2620878457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20021670 con 0x7f7b38100cd0 2023-07-14T16:18:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.188+0000 7f7b3dbff700 1 -- 172.21.15.62:0/2620878457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3819ffe0 con 0x7f7b38100cd0 2023-07-14T16:18:38.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.192+0000 7f7b34ff9700 1 -- 172.21.15.62:0/2620878457 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7b2000f460 con 0x7f7b38100cd0 2023-07-14T16:18:38.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.192+0000 7f7b515d9b80 1 Processor -- start 2023-07-14T16:18:38.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.192+0000 7f7b515d9b80 1 -- start start 2023-07-14T16:18:38.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177abe20 0x55af177ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177acc70 0x55af177ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af176b0cb0 con 0x55af177abe20 2023-07-14T16:18:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af17703a40 con 0x55af177acc70 2023-07-14T16:18:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af176fced0 con 0x55af177ab4d0 2023-07-14T16:18:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54792/0 (socket says 172.21.15.62:54792) 2023-07-14T16:18:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.193+0000 7f7b37fff700 1 -- 172.21.15.62:0/2508065506 learned_addr learned my addr 172.21.15.62:0/2508065506 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b377fe700 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177abe20 0x55af177ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b36ffd700 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177acc70 0x55af177ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b34ff9700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b240a1150 0x7f7b240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b34ff9700 1 -- 172.21.15.62:0/2620878457 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b200cdda0 con 0x7f7b38100cd0 2023-07-14T16:18:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b377fe700 1 -- 172.21.15.62:0/2508065506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 msgr2=0x55af177ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b377fe700 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b377fe700 1 -- 172.21.15.62:0/2508065506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177acc70 msgr2=0x55af177ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:38.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b377fe700 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177acc70 0x55af177ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b377fe700 1 -- 172.21.15.62:0/2508065506 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af1752ade0 con 0x55af177abe20 2023-07-14T16:18:38.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b37fff700 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:38.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b37fff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b240a1150 0x7f7b240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.194+0000 7f7b377fe700 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177abe20 0x55af177ac6e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800a400 tx=0x7f7b280096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc7c84e036df4f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.195+0000 7f7b2dffb700 1 -- 172.21.15.62:0/2508065506 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28010040 con 0x55af177abe20 2023-07-14T16:18:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.195+0000 7f7b2dffb700 1 -- 172.21.15.62:0/2508065506 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b28014ab0 con 0x55af177abe20 2023-07-14T16:18:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.195+0000 7f7b2dffb700 1 -- 172.21.15.62:0/2508065506 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28013940 con 0x55af177abe20 2023-07-14T16:18:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.195+0000 7f7b515d9b80 1 -- 172.21.15.62:0/2508065506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177abe20 msgr2=0x55af177ac6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.195+0000 7f7b37fff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b240a1150 0x7f7b240a3600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b300094f0 tx=0x7f7b30004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.195+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177abe20 0x55af177ac6e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800a400 tx=0x7f7b280096d0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.196+0000 7f7b515d9b80 1 -- 172.21.15.62:0/2508065506 shutdown_connections 2023-07-14T16:18:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.196+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177acc70 0x55af177ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.196+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.196+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/2508065506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177abe20 0x55af177ac6e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.196+0000 7f7b515d9b80 1 -- 172.21.15.62:0/2508065506 >> 172.21.15.62:0/2508065506 conn(0x55af177a0da0 msgr2=0x55af177a61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.196+0000 7f7b515d9b80 1 -- 172.21.15.62:0/2508065506 shutdown_connections 2023-07-14T16:18:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.196+0000 7f7b515d9b80 1 -- 172.21.15.62:0/2508065506 wait complete. 2023-07-14T16:18:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.196+0000 7f7b515d9b80 1 -- start start 2023-07-14T16:18:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b515d9b80 1 Processor -- start 2023-07-14T16:18:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177ab4d0 0x55af177b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177abe20 0x55af177b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af1769cac0 con 0x55af177acc70 2023-07-14T16:18:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af1769c7c0 con 0x55af177ab4d0 2023-07-14T16:18:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af17580920 con 0x55af177abe20 2023-07-14T16:18:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b515d9b80 10 client.0 _ll_register_callbacks cb 0x55af1769a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.197+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60134/0 (socket says 172.21.15.62:60134) 2023-07-14T16:18:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b37fff700 1 -- 172.21.15.62:0/459018422 learned_addr learned my addr 172.21.15.62:0/459018422 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b377fe700 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177ab4d0 0x55af177b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b36ffd700 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177abe20 0x55af177b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b37fff700 1 -- 172.21.15.62:0/459018422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177abe20 msgr2=0x55af177b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b37fff700 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177abe20 0x55af177b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b37fff700 1 -- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177ab4d0 msgr2=0x55af177b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b37fff700 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177ab4d0 0x55af177b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b37fff700 1 -- 172.21.15.62:0/459018422 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b30009970 con 0x55af177acc70 2023-07-14T16:18:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b377fe700 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177ab4d0 0x55af177b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b37fff700 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bc630 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009440 tx=0x7f7b300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec127e633408f5e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:38.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.198+0000 7f7b16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:38.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b16ffd700 1 -- 172.21.15.62:0/459018422 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3001f220 con 0x55af177acc70 2023-07-14T16:18:38.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b515d9b80 1 -- 172.21.15.62:0/459018422 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af176564f0 con 0x55af177acc70 2023-07-14T16:18:38.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b16ffd700 1 -- 172.21.15.62:0/459018422 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b30011a80 con 0x55af177acc70 2023-07-14T16:18:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b16ffd700 1 -- 172.21.15.62:0/459018422 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b30011c40 con 0x55af177acc70 2023-07-14T16:18:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b515d9b80 10 client.16317 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b515d9b80 1 -- 172.21.15.62:0/459018422 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f7b28009c40 con 0x55af177acc70 2023-07-14T16:18:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b16ffd700 1 -- 172.21.15.62:0/459018422 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af176564f0 con 0x55af177acc70 2023-07-14T16:18:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b16ffd700 1 -- 172.21.15.62:0/459018422 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7b3001c410 con 0x55af177acc70 2023-07-14T16:18:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b515d9b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.199+0000 7f7b515d9b80 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.200+0000 7f7b515d9b80 20 client.16317 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.200+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55af177c0ee0 0x55af177c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.200+0000 7f7b515d9b80 4 client.16317 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.200+0000 7f7b515d9b80 1 -- 172.21.15.62:0/459018422 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af176f5eb0 con 0x55af177c0ee0 2023-07-14T16:18:38.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.200+0000 7f7b377fe700 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55af177c0ee0 0x55af177c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.201+0000 7f7b377fe700 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55af177c0ee0 0x55af177c32f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e84ee51d88043 server_cookie=d2696617c1b93342 in_seq=0 out_seq=0 2023-07-14T16:18:38.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.201+0000 7f7b16ffd700 10 client.16317 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:38.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.201+0000 7f7b16ffd700 1 -- 172.21.15.62:0/459018422 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af176f5eb0 con 0x55af177c0ee0 2023-07-14T16:18:38.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.201+0000 7f7b16ffd700 10 client.16317 handle_command_reply: tid=0 2023-07-14T16:18:38.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.201+0000 7f7b515d9b80 1 client.16317 shutdown 2023-07-14T16:18:38.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 -- 172.21.15.62:0/459018422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 msgr2=0x55af177bc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bc630 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009440 tx=0x7f7b300047c0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 -- 172.21.15.62:0/459018422 shutdown_connections 2023-07-14T16:18:38.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bc630 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55af177c0ee0 0x55af177c32f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177ab4d0 0x55af177b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/459018422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177abe20 0x55af177b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 -- 172.21.15.62:0/459018422 >> 172.21.15.62:0/459018422 conn(0x55af177a0da0 msgr2=0x55af177a58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b16ffd700 0 client.16317 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 -- 172.21.15.62:0/459018422 shutdown_connections 2023-07-14T16:18:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 1 -- 172.21.15.62:0/459018422 wait complete. 2023-07-14T16:18:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.202+0000 7f7b515d9b80 20 client.16317 trim_cache size 0 max 16384 2023-07-14T16:18:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.216+0000 7f7b515d9b80 1 Processor -- start 2023-07-14T16:18:38.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b515d9b80 1 -- start start 2023-07-14T16:18:38.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 0x55af177bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af177a6ed0 con 0x55af177acc70 2023-07-14T16:18:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af177a0970 con 0x55af177abe20 2023-07-14T16:18:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af177be2f0 con 0x55af177ab4d0 2023-07-14T16:18:38.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60144/0 (socket says 172.21.15.62:60144) 2023-07-14T16:18:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b37fff700 1 -- 172.21.15.62:0/295397303 learned_addr learned my addr 172.21.15.62:0/295397303 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b377fe700 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.217+0000 7f7b36ffd700 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 0x55af177bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b37fff700 1 -- 172.21.15.62:0/295397303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 msgr2=0x55af177ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b37fff700 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b37fff700 1 -- 172.21.15.62:0/295397303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 msgr2=0x55af177bc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b37fff700 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 0x55af177bc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b37fff700 1 -- 172.21.15.62:0/295397303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af17513c60 con 0x55af177acc70 2023-07-14T16:18:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b37fff700 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177a2310 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7b3001c7c0 tx=0x7f7b30009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b6ea936cebfe5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b2e7fc700 1 -- 172.21.15.62:0/295397303 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3001c960 con 0x55af177acc70 2023-07-14T16:18:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b2e7fc700 1 -- 172.21.15.62:0/295397303 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b3001cac0 con 0x55af177acc70 2023-07-14T16:18:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.218+0000 7f7b2e7fc700 1 -- 172.21.15.62:0/295397303 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3003ace0 con 0x55af177acc70 2023-07-14T16:18:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 -- 172.21.15.62:0/295397303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 msgr2=0x55af177a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177a2310 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7b3001c7c0 tx=0x7f7b30009300 comp rx=0 tx=0).stop 2023-07-14T16:18:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 -- 172.21.15.62:0/295397303 shutdown_connections 2023-07-14T16:18:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177a2310 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/295397303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 0x55af177bc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 -- 172.21.15.62:0/295397303 >> 172.21.15.62:0/295397303 conn(0x55af177a0da0 msgr2=0x55af177c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 -- 172.21.15.62:0/295397303 shutdown_connections 2023-07-14T16:18:38.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.219+0000 7f7b515d9b80 1 -- 172.21.15.62:0/295397303 wait complete. 2023-07-14T16:18:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.220+0000 7f7b515d9b80 1 -- start start 2023-07-14T16:18:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.220+0000 7f7b515d9b80 1 Processor -- start 2023-07-14T16:18:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.220+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.220+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 0x55af177b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b515d9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af177bb9f0 con 0x55af177acc70 2023-07-14T16:18:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af177bbb60 con 0x55af177abe20 2023-07-14T16:18:38.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b515d9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af177bbd20 con 0x55af177ab4d0 2023-07-14T16:18:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b515d9b80 10 client.0 _ll_register_callbacks cb 0x7f7b3001f830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60150/0 (socket says 172.21.15.62:60150) 2023-07-14T16:18:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b377fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b37fff700 1 -- 172.21.15.62:0/3764121437 learned_addr learned my addr 172.21.15.62:0/3764121437 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b36ffd700 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 0x55af177b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b37fff700 1 -- 172.21.15.62:0/3764121437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 msgr2=0x55af177b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b37fff700 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b37fff700 1 -- 172.21.15.62:0/3764121437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 msgr2=0x55af177b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b37fff700 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 0x55af177b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b37fff700 1 -- 172.21.15.62:0/3764121437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b28009c40 con 0x55af177acc70 2023-07-14T16:18:38.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.221+0000 7f7b377fe700 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b37fff700 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bb4b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7b30013be0 tx=0x7f7b3001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e01210c403f60bb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b16ffd700 1 -- 172.21.15.62:0/3764121437 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30011c80 con 0x55af177acc70 2023-07-14T16:18:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b16ffd700 1 -- 172.21.15.62:0/3764121437 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b3003a650 con 0x55af177acc70 2023-07-14T16:18:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b515d9b80 1 -- 172.21.15.62:0/3764121437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b28014ab0 con 0x55af177acc70 2023-07-14T16:18:38.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b16ffd700 1 -- 172.21.15.62:0/3764121437 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b30011de0 con 0x55af177acc70 2023-07-14T16:18:38.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b515d9b80 10 client.16323 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:38.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b515d9b80 1 -- 172.21.15.62:0/3764121437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af17551250 con 0x55af177acc70 2023-07-14T16:18:38.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.222+0000 7f7b16ffd700 1 -- 172.21.15.62:0/3764121437 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b28014ab0 con 0x55af177acc70 2023-07-14T16:18:38.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.223+0000 7f7b16ffd700 1 -- 172.21.15.62:0/3764121437 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7b30043aa0 con 0x55af177acc70 2023-07-14T16:18:38.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.223+0000 7f7b515d9b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:38.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.223+0000 7f7b515d9b80 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:38.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.223+0000 7f7b515d9b80 20 client.16323 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.223+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55af17815620 0x55af17817a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.223+0000 7f7b515d9b80 4 client.16323 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:18:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.223+0000 7f7b515d9b80 1 -- 172.21.15.62:0/3764121437 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55af176b8050 con 0x55af17815620 2023-07-14T16:18:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.223+0000 7f7b377fe700 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55af17815620 0x55af17817a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.224+0000 7f7b377fe700 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55af17815620 0x55af17817a30 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af790cfbaac1281 server_cookie=a12b4df6bf6c5f43 in_seq=0 out_seq=0 2023-07-14T16:18:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.224+0000 7f7b16ffd700 10 client.16323 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:38.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.224+0000 7f7b16ffd700 1 -- 172.21.15.62:0/3764121437 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55af176b8050 con 0x55af17815620 2023-07-14T16:18:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.224+0000 7f7b16ffd700 10 client.16323 handle_command_reply: tid=0 2023-07-14T16:18:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.224+0000 7f7b515d9b80 1 client.16323 shutdown 2023-07-14T16:18:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.224+0000 7f7b515d9b80 1 -- 172.21.15.62:0/3764121437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 msgr2=0x55af177bb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.224+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bb4b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7b30013be0 tx=0x7f7b3001c960 comp rx=0 tx=0).stop 2023-07-14T16:18:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 1 -- 172.21.15.62:0/3764121437 shutdown_connections 2023-07-14T16:18:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af177acc70 0x55af177bb4b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55af177ab4d0 0x55af177b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55af17815620 0x55af17817a30 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 1 --2- 172.21.15.62:0/3764121437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af177abe20 0x55af177b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b16ffd700 0 client.16323 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:38.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 1 -- 172.21.15.62:0/3764121437 >> 172.21.15.62:0/3764121437 conn(0x55af177a0da0 msgr2=0x55af177c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:38.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 1 -- 172.21.15.62:0/3764121437 shutdown_connections 2023-07-14T16:18:38.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 1 -- 172.21.15.62:0/3764121437 wait complete. 2023-07-14T16:18:38.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.225+0000 7f7b515d9b80 20 client.16323 trim_cache size 0 max 16384 2023-07-14T16:18:38.223 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:18:38.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 -- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b240a1150 msgr2=0x7f7b240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b240a1150 0x7f7b240a3600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b300094f0 tx=0x7f7b30004750 comp rx=0 tx=0).stop 2023-07-14T16:18:38.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 -- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 msgr2=0x7f7b38197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 0x7f7b38197f50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005b40 tx=0x7f7b20003c20 comp rx=0 tx=0).stop 2023-07-14T16:18:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 -- 172.21.15.62:0/2620878457 shutdown_connections 2023-07-14T16:18:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b38101660 0x7f7b3819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b240a1150 0x7f7b240a3600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b380ffae0 0x7f7b38197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 --2- 172.21.15.62:0/2620878457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b38100cd0 0x7f7b38197f50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 -- 172.21.15.62:0/2620878457 >> 172.21.15.62:0/2620878457 conn(0x7f7b380fb0e0 msgr2=0x7f7b380fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 -- 172.21.15.62:0/2620878457 shutdown_connections 2023-07-14T16:18:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:38.226+0000 7f7b3dbff700 1 -- 172.21.15.62:0/2620878457 wait complete. 2023-07-14T16:18:39.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:38 smithi164 ceph-mon[97427]: pgmap v447: 97 pgs: 97 active+clean; 1018 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:18:39.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:38 smithi062 ceph-mon[95543]: pgmap v447: 97 pgs: 97 active+clean; 1018 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:18:39.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:38 smithi081 ceph-mon[127795]: pgmap v447: 97 pgs: 97 active+clean; 1018 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:18:39.227 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:18:39.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:18:39.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 -- 172.21.15.62:0/3562758739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c105800 msgr2=0x7ff91c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 --2- 172.21.15.62:0/3562758739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c105800 0x7ff91c107c00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff91400b0a0 tx=0x7ff91400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 -- 172.21.15.62:0/3562758739 shutdown_connections 2023-07-14T16:18:39.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 --2- 172.21.15.62:0/3562758739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c105800 0x7ff91c107c00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 --2- 172.21.15.62:0/3562758739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c1005c0 0x7ff91c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 --2- 172.21.15.62:0/3562758739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff91c102ee0 0x7ff91c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 -- 172.21.15.62:0/3562758739 >> 172.21.15.62:0/3562758739 conn(0x7ff91c0fa000 msgr2=0x7ff91c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 -- 172.21.15.62:0/3562758739 shutdown_connections 2023-07-14T16:18:39.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.505+0000 7ff9245ff700 1 -- 172.21.15.62:0/3562758739 wait complete. 2023-07-14T16:18:39.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.506+0000 7ff9245ff700 1 Processor -- start 2023-07-14T16:18:39.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.506+0000 7ff9245ff700 1 -- start start 2023-07-14T16:18:39.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff9245ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c1005c0 0x7ff91c195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff9245ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c102ee0 0x7ff91c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff9245ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff91c105800 0x7ff91c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff9245ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c19d3a0 con 0x7ff91c1005c0 2023-07-14T16:18:39.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff9245ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c19d4e0 con 0x7ff91c105800 2023-07-14T16:18:39.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff9245ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c19d670 con 0x7ff91c102ee0 2023-07-14T16:18:39.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff92239b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c1005c0 0x7ff91c195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff92239b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c1005c0 0x7ff91c195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60166/0 (socket says 172.21.15.62:60166) 2023-07-14T16:18:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff92239b700 1 -- 172.21.15.62:0/1177884070 learned_addr learned my addr 172.21.15.62:0/1177884070 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff921b9a700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c102ee0 0x7ff91c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff922b9c700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff91c105800 0x7ff91c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff92239b700 1 -- 172.21.15.62:0/1177884070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c102ee0 msgr2=0x7ff91c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff92239b700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c102ee0 0x7ff91c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff92239b700 1 -- 172.21.15.62:0/1177884070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff91c105800 msgr2=0x7ff91c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff92239b700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff91c105800 0x7ff91c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff92239b700 1 -- 172.21.15.62:0/1177884070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914009d00 con 0x7ff91c1005c0 2023-07-14T16:18:39.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.507+0000 7ff922b9c700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff91c105800 0x7ff91c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:39.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.508+0000 7ff921b9a700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c102ee0 0x7ff91c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:39.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.508+0000 7ff92239b700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c1005c0 0x7ff91c195820 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9a0 tx=0x7ff91800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.508+0000 7ff9137fe700 1 -- 172.21.15.62:0/1177884070 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918009790 con 0x7ff91c1005c0 2023-07-14T16:18:39.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.508+0000 7ff9137fe700 1 -- 172.21.15.62:0/1177884070 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff91800cbd0 con 0x7ff91c1005c0 2023-07-14T16:18:39.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.508+0000 7ff9137fe700 1 -- 172.21.15.62:0/1177884070 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918018680 con 0x7ff91c1005c0 2023-07-14T16:18:39.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.508+0000 7ff9245ff700 1 -- 172.21.15.62:0/1177884070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c19d950 con 0x7ff91c1005c0 2023-07-14T16:18:39.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.508+0000 7ff9245ff700 1 -- 172.21.15.62:0/1177884070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c19de10 con 0x7ff91c1005c0 2023-07-14T16:18:39.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.511+0000 7ff9137fe700 1 -- 172.21.15.62:0/1177884070 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff91800cd40 con 0x7ff91c1005c0 2023-07-14T16:18:39.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.513+0000 7ff9137fe700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9080a1150 0x7ff9080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.513+0000 7ff937e9fb80 1 Processor -- start 2023-07-14T16:18:39.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.513+0000 7ff9137fe700 1 -- 172.21.15.62:0/1177884070 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff918014070 con 0x7ff91c1005c0 2023-07-14T16:18:39.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.514+0000 7ff921b9a700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9080a1150 0x7ff9080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.514+0000 7ff937e9fb80 1 -- start start 2023-07-14T16:18:39.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.514+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb32d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.514+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.514+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb34570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.514+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14ea33d30 con 0x55f14eb332b0 2023-07-14T16:18:39.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.514+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14ea86ac0 con 0x55f14eb34100 2023-07-14T16:18:39.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.514+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14ea7ff50 con 0x55f14eb32960 2023-07-14T16:18:39.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb32d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb32d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54852/0 (socket says 172.21.15.62:54852) 2023-07-14T16:18:39.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 -- 172.21.15.62:0/877978064 learned_addr learned my addr 172.21.15.62:0/877978064 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff921b9a700 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff922b9c700 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb34570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff921b9a700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9080a1150 0x7ff9080a3600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009cb0 tx=0x7ff90c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 -- 172.21.15.62:0/877978064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 msgr2=0x55f14eb34570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb34570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 -- 172.21.15.62:0/877978064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 msgr2=0x55f14eb33b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb33b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 -- 172.21.15.62:0/877978064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f14e8ade60 con 0x55f14eb32960 2023-07-14T16:18:39.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.515+0000 7ff92239b700 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb32d70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff918099ff0 tx=0x7ff91809a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e4b54404e1a1a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.516+0000 7ff910bfe700 1 -- 172.21.15.62:0/877978064 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180a24b0 con 0x55f14eb32960 2023-07-14T16:18:39.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.516+0000 7ff910bfe700 1 -- 172.21.15.62:0/877978064 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff918098400 con 0x55f14eb32960 2023-07-14T16:18:39.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.516+0000 7ff910bfe700 1 -- 172.21.15.62:0/877978064 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180a2280 con 0x55f14eb32960 2023-07-14T16:18:39.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 -- 172.21.15.62:0/877978064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 msgr2=0x55f14eb32d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb32d70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff918099ff0 tx=0x7ff91809a060 comp rx=0 tx=0).stop 2023-07-14T16:18:39.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 -- 172.21.15.62:0/877978064 shutdown_connections 2023-07-14T16:18:39.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb34570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb32d70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/877978064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb33b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 -- 172.21.15.62:0/877978064 >> 172.21.15.62:0/877978064 conn(0x55f14eb23e20 msgr2=0x55f14eb29250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 -- 172.21.15.62:0/877978064 shutdown_connections 2023-07-14T16:18:39.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 -- 172.21.15.62:0/877978064 wait complete. 2023-07-14T16:18:39.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.517+0000 7ff937e9fb80 1 -- start start 2023-07-14T16:18:39.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff937e9fb80 1 Processor -- start 2023-07-14T16:18:39.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb3c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb3cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb43b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14ea1fb40 con 0x55f14eb332b0 2023-07-14T16:18:39.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14ea1f840 con 0x55f14eb34100 2023-07-14T16:18:39.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14e9039a0 con 0x55f14eb32960 2023-07-14T16:18:39.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff937e9fb80 10 client.0 _ll_register_callbacks cb 0x55f14ea1d7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:39.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff921b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb3cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff921b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb3cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60190/0 (socket says 172.21.15.62:60190) 2023-07-14T16:18:39.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.518+0000 7ff921b9a700 1 -- 172.21.15.62:0/372129250 learned_addr learned my addr 172.21.15.62:0/372129250 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff92239b700 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb3c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff922b9c700 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb43b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff921b9a700 1 -- 172.21.15.62:0/372129250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 msgr2=0x55f14eb3c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff921b9a700 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb3c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff921b9a700 1 -- 172.21.15.62:0/372129250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 msgr2=0x55f14eb43b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff921b9a700 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb43b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff921b9a700 1 -- 172.21.15.62:0/372129250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c19de10 con 0x55f14eb332b0 2023-07-14T16:18:39.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff92239b700 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb3c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:39.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff921b9a700 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb3cae0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff90c012930 tx=0x7ff90c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a56fe368146f41 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.519+0000 7ff905ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:39.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.520+0000 7ff905ffb700 1 -- 172.21.15.62:0/372129250 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c020940 con 0x55f14eb332b0 2023-07-14T16:18:39.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.520+0000 7ff937e9fb80 1 -- 172.21.15.62:0/372129250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f14e9d9570 con 0x55f14eb332b0 2023-07-14T16:18:39.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.520+0000 7ff905ffb700 1 -- 172.21.15.62:0/372129250 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff90c018070 con 0x55f14eb332b0 2023-07-14T16:18:39.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.520+0000 7ff905ffb700 1 -- 172.21.15.62:0/372129250 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff90c020aa0 con 0x55f14eb332b0 2023-07-14T16:18:39.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.520+0000 7ff937e9fb80 10 client.16335 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:39.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.520+0000 7ff937e9fb80 1 -- 172.21.15.62:0/372129250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f14e896ce0 con 0x55f14eb332b0 2023-07-14T16:18:39.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.520+0000 7ff905ffb700 1 -- 172.21.15.62:0/372129250 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f14e9d9570 con 0x55f14eb332b0 2023-07-14T16:18:39.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.520+0000 7ff905ffb700 1 -- 172.21.15.62:0/372129250 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff90c029d10 con 0x55f14eb332b0 2023-07-14T16:18:39.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.521+0000 7ff937e9fb80 10 client.16335 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:39.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.521+0000 7ff937e9fb80 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:39.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.521+0000 7ff937e9fb80 20 client.16335 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:39.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.521+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f14eb48380 0x55f14eb4a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.521+0000 7ff937e9fb80 4 client.16335 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:39.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.521+0000 7ff937e9fb80 1 -- 172.21.15.62:0/372129250 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f14ea78f30 con 0x55f14eb48380 2023-07-14T16:18:39.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.521+0000 7ff92239b700 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f14eb48380 0x55f14eb4a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.522+0000 7ff92239b700 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f14eb48380 0x55f14eb4a790 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8797dcf5e834d14 server_cookie=998bc5c1689b977a in_seq=0 out_seq=0 2023-07-14T16:18:39.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.522+0000 7ff905ffb700 10 client.16335 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:39.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.522+0000 7ff905ffb700 1 -- 172.21.15.62:0/372129250 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f14ea78f30 con 0x55f14eb48380 2023-07-14T16:18:39.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.522+0000 7ff905ffb700 10 client.16335 handle_command_reply: tid=0 2023-07-14T16:18:39.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.522+0000 7ff937e9fb80 1 client.16335 shutdown 2023-07-14T16:18:39.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 -- 172.21.15.62:0/372129250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 msgr2=0x55f14eb3cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb3cae0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff90c012930 tx=0x7ff90c0047c0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 -- 172.21.15.62:0/372129250 shutdown_connections 2023-07-14T16:18:39.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f14eb48380 0x55f14eb4a790 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb43b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb3c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff905ffb700 0 client.16335 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:39.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/372129250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb3cae0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 -- 172.21.15.62:0/372129250 >> 172.21.15.62:0/372129250 conn(0x55f14eb23e20 msgr2=0x55f14eb28930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 -- 172.21.15.62:0/372129250 shutdown_connections 2023-07-14T16:18:39.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 1 -- 172.21.15.62:0/372129250 wait complete. 2023-07-14T16:18:39.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.523+0000 7ff937e9fb80 20 client.16335 trim_cache size 0 max 16384 2023-07-14T16:18:39.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.543+0000 7ff937e9fb80 1 Processor -- start 2023-07-14T16:18:39.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.543+0000 7ff937e9fb80 1 -- start start 2023-07-14T16:18:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.543+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb36fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.543+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb332b0 0x55f14eb43ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.543+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb34100 0x55f14eb37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.543+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14eb2ace0 con 0x55f14eb34100 2023-07-14T16:18:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.543+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14eb239f0 con 0x55f14eb332b0 2023-07-14T16:18:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.543+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14eb2a2f0 con 0x55f14eb32960 2023-07-14T16:18:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff922b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb34100 0x55f14eb37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff921b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb332b0 0x55f14eb43ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff921b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb332b0 0x55f14eb43ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59332/0 (socket says 172.21.15.62:59332) 2023-07-14T16:18:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff922b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb34100 0x55f14eb37500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60196/0 (socket says 172.21.15.62:60196) 2023-07-14T16:18:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff921b9a700 1 -- 172.21.15.62:0/2480537105 learned_addr learned my addr 172.21.15.62:0/2480537105 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff92239b700 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb36fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff92239b700 1 -- 172.21.15.62:0/2480537105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb332b0 msgr2=0x55f14eb43ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff92239b700 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb332b0 0x55f14eb43ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff92239b700 1 -- 172.21.15.62:0/2480537105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb34100 msgr2=0x55f14eb37500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff92239b700 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb34100 0x55f14eb37500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff92239b700 1 -- 172.21.15.62:0/2480537105 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f14e8d42d0 con 0x55f14eb32960 2023-07-14T16:18:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff922b9c700 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb34100 0x55f14eb37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:39.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.544+0000 7ff921b9a700 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb332b0 0x55f14eb43ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.545+0000 7ff92239b700 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb36fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff91800c5b0 tx=0x7ff91809a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b41038705996738 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.545+0000 7ff9113ff700 1 -- 172.21.15.62:0/2480537105 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918099550 con 0x55f14eb32960 2023-07-14T16:18:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.545+0000 7ff9113ff700 1 -- 172.21.15.62:0/2480537105 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff918095e50 con 0x55f14eb32960 2023-07-14T16:18:39.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.545+0000 7ff9113ff700 1 -- 172.21.15.62:0/2480537105 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180996b0 con 0x55f14eb32960 2023-07-14T16:18:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 -- 172.21.15.62:0/2480537105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 msgr2=0x55f14eb36fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb36fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff91800c5b0 tx=0x7ff91809a0d0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 -- 172.21.15.62:0/2480537105 shutdown_connections 2023-07-14T16:18:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb34100 0x55f14eb37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb36fc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/2480537105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb332b0 0x55f14eb43ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 -- 172.21.15.62:0/2480537105 >> 172.21.15.62:0/2480537105 conn(0x55f14eb23e20 msgr2=0x55f14eb3a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 -- 172.21.15.62:0/2480537105 shutdown_connections 2023-07-14T16:18:39.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.546+0000 7ff937e9fb80 1 -- 172.21.15.62:0/2480537105 wait complete. 2023-07-14T16:18:39.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.547+0000 7ff937e9fb80 1 -- start start 2023-07-14T16:18:39.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.547+0000 7ff937e9fb80 1 Processor -- start 2023-07-14T16:18:39.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.547+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb40eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff937e9fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb42400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14eb42940 con 0x55f14eb332b0 2023-07-14T16:18:39.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14eb42ab0 con 0x55f14eb34100 2023-07-14T16:18:39.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff937e9fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f14eb42c70 con 0x55f14eb32960 2023-07-14T16:18:39.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff937e9fb80 10 client.0 _ll_register_callbacks cb 0x7ff90c0124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:39.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff921b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff921b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60208/0 (socket says 172.21.15.62:60208) 2023-07-14T16:18:39.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff921b9a700 1 -- 172.21.15.62:0/324318817 learned_addr learned my addr 172.21.15.62:0/324318817 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff92239b700 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb40eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff921b9a700 1 -- 172.21.15.62:0/324318817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 msgr2=0x55f14eb40eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.548+0000 7ff921b9a700 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb40eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff921b9a700 1 -- 172.21.15.62:0/324318817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 msgr2=0x55f14eb42400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff921b9a700 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb42400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff921b9a700 1 -- 172.21.15.62:0/324318817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f14e896ce0 con 0x55f14eb332b0 2023-07-14T16:18:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff921b9a700 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb413f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff90c011a50 tx=0x7ff90c0339f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97591c8ad95fc7f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff905ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff905ffb700 1 -- 172.21.15.62:0/324318817 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c020740 con 0x55f14eb332b0 2023-07-14T16:18:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff937e9fb80 1 -- 172.21.15.62:0/324318817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff918098400 con 0x55f14eb332b0 2023-07-14T16:18:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff905ffb700 1 -- 172.21.15.62:0/324318817 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff90c038030 con 0x55f14eb332b0 2023-07-14T16:18:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff905ffb700 1 -- 172.21.15.62:0/324318817 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff90c0208a0 con 0x55f14eb332b0 2023-07-14T16:18:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.549+0000 7ff937e9fb80 10 client.16341 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff937e9fb80 1 -- 172.21.15.62:0/324318817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7ff91c1087f0 con 0x55f14eb332b0 2023-07-14T16:18:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff905ffb700 1 -- 172.21.15.62:0/324318817 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff918098400 con 0x55f14eb332b0 2023-07-14T16:18:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff905ffb700 1 -- 172.21.15.62:0/324318817 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff90c046070 con 0x55f14eb332b0 2023-07-14T16:18:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff937e9fb80 10 client.16341 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff937e9fb80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff937e9fb80 20 client.16341 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f14eb9e890 0x55f14eba0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff937e9fb80 4 client.16341 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:18:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.550+0000 7ff937e9fb80 1 -- 172.21.15.62:0/324318817 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f14ea3b0d0 con 0x55f14eb9e890 2023-07-14T16:18:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.551+0000 7ff92239b700 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f14eb9e890 0x55f14eba0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.552+0000 7ff905ffb700 10 client.16341 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:39.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.552+0000 7ff92239b700 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f14eb9e890 0x55f14eba0ca0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=687a957dedf063d6 server_cookie=c43329786505be6d in_seq=0 out_seq=0 2023-07-14T16:18:39.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff905ffb700 1 -- 172.21.15.62:0/324318817 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f14ea3b0d0 con 0x55f14eb9e890 2023-07-14T16:18:39.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff905ffb700 10 client.16341 handle_command_reply: tid=0 2023-07-14T16:18:39.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 client.16341 shutdown 2023-07-14T16:18:39.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 -- 172.21.15.62:0/324318817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 msgr2=0x55f14eb413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb413f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff90c011a50 tx=0x7ff90c0339f0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 -- 172.21.15.62:0/324318817 shutdown_connections 2023-07-14T16:18:39.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f14eb9e890 0x55f14eba0ca0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f14eb34100 0x55f14eb42400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f14eb32960 0x55f14eb40eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 --2- 172.21.15.62:0/324318817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f14eb332b0 0x55f14eb413f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff905ffb700 0 client.16341 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:39.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.554+0000 7ff937e9fb80 1 -- 172.21.15.62:0/324318817 >> 172.21.15.62:0/324318817 conn(0x55f14eb23e20 msgr2=0x55f14eb36fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.555+0000 7ff937e9fb80 1 -- 172.21.15.62:0/324318817 shutdown_connections 2023-07-14T16:18:39.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.555+0000 7ff937e9fb80 1 -- 172.21.15.62:0/324318817 wait complete. 2023-07-14T16:18:39.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.555+0000 7ff937e9fb80 20 client.16341 trim_cache size 0 max 16384 2023-07-14T16:18:39.537 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:18:39.537 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:18:39.537 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "23d81de3-c204-4a85-9447-2e70dda88623", 2023-07-14T16:18:39.537 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:18:39.538 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:18:39.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 -- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9080a1150 msgr2=0x7ff9080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9080a1150 0x7ff9080a3600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009cb0 tx=0x7ff90c009260 comp rx=0 tx=0).stop 2023-07-14T16:18:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 -- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c1005c0 msgr2=0x7ff91c195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c1005c0 0x7ff91c195820 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9a0 tx=0x7ff91800ed60 comp rx=0 tx=0).stop 2023-07-14T16:18:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 -- 172.21.15.62:0/1177884070 shutdown_connections 2023-07-14T16:18:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9080a1150 0x7ff9080a3600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff91c105800 0x7ff91c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff91c1005c0 0x7ff91c195820 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 --2- 172.21.15.62:0/1177884070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c102ee0 0x7ff91c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 -- 172.21.15.62:0/1177884070 >> 172.21.15.62:0/1177884070 conn(0x7ff91c0fa000 msgr2=0x7ff91c1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 -- 172.21.15.62:0/1177884070 shutdown_connections 2023-07-14T16:18:39.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.556+0000 7ff9245ff700 1 -- 172.21.15.62:0/1177884070 wait complete. 2023-07-14T16:18:39.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:18:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 -- 172.21.15.62:0/2554342398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8102ef0 msgr2=0x7f9bc8103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/2554342398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8102ef0 0x7f9bc8103340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009990 tx=0x7f9bbc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 -- 172.21.15.62:0/2554342398 shutdown_connections 2023-07-14T16:18:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/2554342398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bc8103880 0x7f9bc810a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/2554342398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bc8101d00 0x7f9bc8102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/2554342398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8102ef0 0x7f9bc8103340 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 -- 172.21.15.62:0/2554342398 >> 172.21.15.62:0/2554342398 conn(0x7f9bc80fd320 msgr2=0x7f9bc80ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 -- 172.21.15.62:0/2554342398 shutdown_connections 2023-07-14T16:18:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.826+0000 7f9bcdbff700 1 -- 172.21.15.62:0/2554342398 wait complete. 2023-07-14T16:18:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.827+0000 7f9bcdbff700 1 Processor -- start 2023-07-14T16:18:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.827+0000 7f9bcdbff700 1 -- start start 2023-07-14T16:18:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.827+0000 7f9bcdbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8101d00 0x7f9bc81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.827+0000 7f9bcdbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bc8102ef0 0x7f9bc8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bcdbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bc8103880 0x7f9bc819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bcdbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc819f530 con 0x7f9bc8101d00 2023-07-14T16:18:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bcdbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc819f670 con 0x7f9bc8102ef0 2023-07-14T16:18:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bcdbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc819f800 con 0x7f9bc8103880 2023-07-14T16:18:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8101d00 0x7f9bc81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8101d00 0x7f9bc81979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60222/0 (socket says 172.21.15.62:60222) 2023-07-14T16:18:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc77fe700 1 -- 172.21.15.62:0/3134388251 learned_addr learned my addr 172.21.15.62:0/3134388251 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bc8103880 0x7f9bc819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bc8102ef0 0x7f9bc8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc77fe700 1 -- 172.21.15.62:0/3134388251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bc8103880 msgr2=0x7f9bc819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bc8103880 0x7f9bc819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc77fe700 1 -- 172.21.15.62:0/3134388251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bc8102ef0 msgr2=0x7f9bc8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bc8102ef0 0x7f9bc8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.828+0000 7f9bc77fe700 1 -- 172.21.15.62:0/3134388251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0095f0 con 0x7f9bc8101d00 2023-07-14T16:18:39.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.829+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8101d00 0x7f9bc81979b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800e970 tx=0x7f9bb800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.829+0000 7f9bc4ff9700 1 -- 172.21.15.62:0/3134388251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8009790 con 0x7f9bc8101d00 2023-07-14T16:18:39.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.829+0000 7f9bc4ff9700 1 -- 172.21.15.62:0/3134388251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bb800eec0 con 0x7f9bc8101d00 2023-07-14T16:18:39.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.829+0000 7f9bcdbff700 1 -- 172.21.15.62:0/3134388251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc819fae0 con 0x7f9bc8101d00 2023-07-14T16:18:39.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.829+0000 7f9bc4ff9700 1 -- 172.21.15.62:0/3134388251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8003c10 con 0x7f9bc8101d00 2023-07-14T16:18:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.829+0000 7f9bcdbff700 1 -- 172.21.15.62:0/3134388251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc81a0040 con 0x7f9bc8101d00 2023-07-14T16:18:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.833+0000 7f9bc4ff9700 1 -- 172.21.15.62:0/3134388251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9bb8003db0 con 0x7f9bc8101d00 2023-07-14T16:18:39.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.834+0000 7f9be1508b80 1 Processor -- start 2023-07-14T16:18:39.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.834+0000 7f9be1508b80 1 -- start start 2023-07-14T16:18:39.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d7748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d774e20 0x55bd9d7756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d775c70 0x55bd9d7760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9be1508b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d679cb0 con 0x55bd9d775c70 2023-07-14T16:18:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9be1508b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d6cca40 con 0x55bd9d774e20 2023-07-14T16:18:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9be1508b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d6c5ed0 con 0x55bd9d7744d0 2023-07-14T16:18:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9bc4ff9700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bb00a11d0 0x7f9bb00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9bc7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d7748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9bc4ff9700 1 -- 172.21.15.62:0/3134388251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9bb8014070 con 0x7f9bc8101d00 2023-07-14T16:18:39.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9bc7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d7748e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54914/0 (socket says 172.21.15.62:54914) 2023-07-14T16:18:39.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9bc7fff700 1 -- 172.21.15.62:0/2451598921 learned_addr learned my addr 172.21.15.62:0/2451598921 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bb00a11d0 0x7f9bb00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.835+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d774e20 0x55bd9d7756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.836+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d775c70 0x55bd9d7760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.836+0000 7f9bc7fff700 1 -- 172.21.15.62:0/2451598921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d774e20 msgr2=0x55bd9d7756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.836+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d774e20 0x55bd9d7756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.836+0000 7f9bc7fff700 1 -- 172.21.15.62:0/2451598921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d775c70 msgr2=0x55bd9d7760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.836+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d775c70 0x55bd9d7760e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.836+0000 7f9bc7fff700 1 -- 172.21.15.62:0/2451598921 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9d4f3de0 con 0x55bd9d7744d0 2023-07-14T16:18:39.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.836+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d7748e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0007d30 tx=0x7f9bc0019650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68769b65e3cfd0e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.836+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bb00a11d0 0x7f9bb00a3680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00199a0 tx=0x7f9bc000d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.837+0000 7f9badffb700 1 -- 172.21.15.62:0/2451598921 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0007590 con 0x55bd9d7744d0 2023-07-14T16:18:39.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.837+0000 7f9badffb700 1 -- 172.21.15.62:0/2451598921 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bc000f040 con 0x55bd9d7744d0 2023-07-14T16:18:39.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.837+0000 7f9badffb700 1 -- 172.21.15.62:0/2451598921 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0004e10 con 0x55bd9d7744d0 2023-07-14T16:18:39.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.837+0000 7f9be1508b80 1 -- 172.21.15.62:0/2451598921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 msgr2=0x55bd9d7748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.837+0000 7f9be1508b80 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d7748e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0007d30 tx=0x7f9bc0019650 comp rx=0 tx=0).stop 2023-07-14T16:18:39.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.838+0000 7f9be1508b80 1 -- 172.21.15.62:0/2451598921 shutdown_connections 2023-07-14T16:18:39.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.838+0000 7f9be1508b80 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d775c70 0x55bd9d7760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.838+0000 7f9be1508b80 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d7748e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.838+0000 7f9be1508b80 1 --2- 172.21.15.62:0/2451598921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d774e20 0x55bd9d7756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.838+0000 7f9be1508b80 1 -- 172.21.15.62:0/2451598921 >> 172.21.15.62:0/2451598921 conn(0x55bd9d769da0 msgr2=0x55bd9d76f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.838+0000 7f9be1508b80 1 -- 172.21.15.62:0/2451598921 shutdown_connections 2023-07-14T16:18:39.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.838+0000 7f9be1508b80 1 -- 172.21.15.62:0/2451598921 wait complete. 2023-07-14T16:18:39.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.838+0000 7f9be1508b80 1 -- start start 2023-07-14T16:18:39.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.839+0000 7f9be1508b80 1 Processor -- start 2023-07-14T16:18:39.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.839+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d77e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.839+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d77e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.839+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d785680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.839+0000 7f9be1508b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d665ac0 con 0x55bd9d774e20 2023-07-14T16:18:39.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.839+0000 7f9be1508b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d6657c0 con 0x55bd9d775c70 2023-07-14T16:18:39.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.839+0000 7f9be1508b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d549920 con 0x55bd9d7744d0 2023-07-14T16:18:39.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.839+0000 7f9be1508b80 10 client.0 _ll_register_callbacks cb 0x55bd9d663760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:39.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d77e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d77e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d77e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54924/0 (socket says 172.21.15.62:54924) 2023-07-14T16:18:39.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc77fe700 1 -- 172.21.15.62:0/560753041 learned_addr learned my addr 172.21.15.62:0/560753041 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d785680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc77fe700 1 -- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 msgr2=0x55bd9d785680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d785680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc77fe700 1 -- 172.21.15.62:0/560753041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 msgr2=0x55bd9d77e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d77e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc77fe700 1 -- 172.21.15.62:0/560753041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc0011040 con 0x55bd9d7744d0 2023-07-14T16:18:39.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d785680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:39.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.840+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d77e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:18:39.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.841+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d77e100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8096990 tx=0x7f9bb800cdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8897e97b3b2323 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.841+0000 7f9ba6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:18:39.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.841+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/560753041 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb80b1070 con 0x55bd9d7744d0 2023-07-14T16:18:39.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.841+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/560753041 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9bb80a2370 con 0x55bd9d7744d0 2023-07-14T16:18:39.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.841+0000 7f9be1508b80 1 -- 172.21.15.62:0/560753041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd9d61f4f0 con 0x55bd9d7744d0 2023-07-14T16:18:39.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.841+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/560753041 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bb80b8170 con 0x55bd9d7744d0 2023-07-14T16:18:39.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.841+0000 7f9be1508b80 10 client.25999 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:39.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.841+0000 7f9be1508b80 1 -- 172.21.15.62:0/560753041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bd9d4dcc60 con 0x55bd9d7744d0 2023-07-14T16:18:39.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.842+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/560753041 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bb80c8080 con 0x55bd9d7744d0 2023-07-14T16:18:39.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.842+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/560753041 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9bb80a1ee0 con 0x55bd9d7744d0 2023-07-14T16:18:39.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.842+0000 7f9be1508b80 10 client.25999 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:39.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.842+0000 7f9be1508b80 10 client.25999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:39.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.842+0000 7f9be1508b80 20 client.25999 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:39.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.842+0000 7f9be1508b80 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd9d789d80 0x55bd9d78c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.842+0000 7f9be1508b80 4 client.25999 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:39.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.842+0000 7f9be1508b80 1 -- 172.21.15.62:0/560753041 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd9d6beeb0 con 0x55bd9d789d80 2023-07-14T16:18:39.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.843+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd9d789d80 0x55bd9d78c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.843+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd9d789d80 0x55bd9d78c190 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38a1c28aa5501a92 server_cookie=8bd91026000477fa in_seq=0 out_seq=0 2023-07-14T16:18:39.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.843+0000 7f9ba6ffd700 10 client.25999 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:39.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.844+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/560753041 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bd9d6beeb0 con 0x55bd9d789d80 2023-07-14T16:18:39.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.844+0000 7f9ba6ffd700 10 client.25999 handle_command_reply: tid=0 2023-07-14T16:18:39.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.844+0000 7f9be1508b80 1 client.25999 shutdown 2023-07-14T16:18:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 -- 172.21.15.62:0/560753041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 msgr2=0x55bd9d77e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d77e100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8096990 tx=0x7f9bb800cdc0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 -- 172.21.15.62:0/560753041 shutdown_connections 2023-07-14T16:18:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd9d789d80 0x55bd9d78c190 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d785680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d77e100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9ba6ffd700 0 client.25999 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:39.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 --2- 172.21.15.62:0/560753041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d77e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 -- 172.21.15.62:0/560753041 >> 172.21.15.62:0/560753041 conn(0x55bd9d769da0 msgr2=0x55bd9d76e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 -- 172.21.15.62:0/560753041 shutdown_connections 2023-07-14T16:18:39.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 1 -- 172.21.15.62:0/560753041 wait complete. 2023-07-14T16:18:39.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.845+0000 7f9be1508b80 20 client.25999 trim_cache size 0 max 16384 2023-07-14T16:18:39.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9be1508b80 1 Processor -- start 2023-07-14T16:18:39.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9be1508b80 1 -- start start 2023-07-14T16:18:39.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d7744d0 0x55bd9d785f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d774e20 0x55bd9d785640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d778c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9be1508b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d77dcb0 con 0x55bd9d7744d0 2023-07-14T16:18:39.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9be1508b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d775220 con 0x55bd9d775c70 2023-07-14T16:18:39.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9be1508b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d789940 con 0x55bd9d774e20 2023-07-14T16:18:39.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9bc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d7744d0 0x55bd9d785f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9bc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d7744d0 0x55bd9d785f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60262/0 (socket says 172.21.15.62:60262) 2023-07-14T16:18:39.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.865+0000 7f9bc77fe700 1 -- 172.21.15.62:0/1418620100 learned_addr learned my addr 172.21.15.62:0/1418620100 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/1418620100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d774e20 0x55bd9d785640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bc77fe700 1 -- 172.21.15.62:0/1418620100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d774e20 msgr2=0x55bd9d785640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/1418620100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d774e20 0x55bd9d785640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bc77fe700 1 -- 172.21.15.62:0/1418620100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 msgr2=0x55bd9d778c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/1418620100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d778c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bc77fe700 1 -- 172.21.15.62:0/1418620100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9d51a250 con 0x55bd9d7744d0 2023-07-14T16:18:39.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/1418620100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d7744d0 0x55bd9d785f70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8099c20 tx=0x7f9bb8096390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337015f939a890e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bae7fc700 1 -- 172.21.15.62:0/1418620100 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb80b1070 con 0x55bd9d7744d0 2023-07-14T16:18:39.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bae7fc700 1 -- 172.21.15.62:0/1418620100 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bd9d61f4f0 con 0x55bd9d7744d0 2023-07-14T16:18:39.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.866+0000 7f9bae7fc700 1 -- 172.21.15.62:0/1418620100 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb80b8360 con 0x55bd9d7744d0 2023-07-14T16:18:39.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.867+0000 7f9be1508b80 1 -- 172.21.15.62:0/1418620100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d7744d0 msgr2=0x55bd9d785f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.867+0000 7f9be1508b80 1 --2- 172.21.15.62:0/1418620100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d7744d0 0x55bd9d785f70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8099c20 tx=0x7f9bb8096390 comp rx=0 tx=0).stop 2023-07-14T16:18:39.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.867+0000 7f9be1508b80 1 -- 172.21.15.62:0/1418620100 shutdown_connections 2023-07-14T16:18:39.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.867+0000 7f9be1508b80 1 --2- 172.21.15.62:0/1418620100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d778c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.867+0000 7f9be1508b80 1 --2- 172.21.15.62:0/1418620100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d7744d0 0x55bd9d785f70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.867+0000 7f9be1508b80 1 --2- 172.21.15.62:0/1418620100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d774e20 0x55bd9d785640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.867+0000 7f9be1508b80 1 -- 172.21.15.62:0/1418620100 >> 172.21.15.62:0/1418620100 conn(0x55bd9d769da0 msgr2=0x55bd9d77cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.867+0000 7f9be1508b80 1 -- 172.21.15.62:0/1418620100 shutdown_connections 2023-07-14T16:18:39.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.868+0000 7f9be1508b80 1 -- 172.21.15.62:0/1418620100 wait complete. 2023-07-14T16:18:39.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.868+0000 7f9be1508b80 1 -- start start 2023-07-14T16:18:39.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.868+0000 7f9be1508b80 1 Processor -- start 2023-07-14T16:18:39.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d784fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d785500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9be1508b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d7835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9be1508b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d783ca0 con 0x55bd9d774e20 2023-07-14T16:18:39.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9be1508b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d783e10 con 0x55bd9d775c70 2023-07-14T16:18:39.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9be1508b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d783fd0 con 0x55bd9d7744d0 2023-07-14T16:18:39.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9be1508b80 10 client.0 _ll_register_callbacks cb 0x7f9bb8099a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:39.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d785500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d785500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60266/0 (socket says 172.21.15.62:60266) 2023-07-14T16:18:39.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc6ffd700 1 -- 172.21.15.62:0/856352977 learned_addr learned my addr 172.21.15.62:0/856352977 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:39.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d784fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc7fff700 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d7835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc6ffd700 1 -- 172.21.15.62:0/856352977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 msgr2=0x55bd9d784fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d784fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc6ffd700 1 -- 172.21.15.62:0/856352977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 msgr2=0x55bd9d7835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:39.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d7835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.869+0000 7f9bc6ffd700 1 -- 172.21.15.62:0/856352977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd9d4dcc60 con 0x55bd9d774e20 2023-07-14T16:18:39.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.870+0000 7f9bc6ffd700 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d785500 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc005910 tx=0x7f9bbc0051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b9a0488cb13056 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:39.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.870+0000 7f9ba6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:39.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.870+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/856352977 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc0285c0 con 0x55bd9d774e20 2023-07-14T16:18:39.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.870+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/856352977 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9bbc028a50 con 0x55bd9d774e20 2023-07-14T16:18:39.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.870+0000 7f9be1508b80 1 -- 172.21.15.62:0/856352977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bc000f040 con 0x55bd9d774e20 2023-07-14T16:18:39.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.870+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/856352977 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bbc028c10 con 0x55bd9d774e20 2023-07-14T16:18:39.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.870+0000 7f9be1508b80 10 client.16359 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:39.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.870+0000 7f9be1508b80 1 -- 172.21.15.62:0/856352977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bd9d4f22b0 con 0x55bd9d774e20 2023-07-14T16:18:39.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/856352977 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bc000f040 con 0x55bd9d774e20 2023-07-14T16:18:39.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/856352977 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9bbc00b870 con 0x55bd9d774e20 2023-07-14T16:18:39.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9be1508b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:39.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9be1508b80 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:39.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9be1508b80 20 client.16359 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:39.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9be1508b80 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd9d7df110 0x55bd9d7e1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:39.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9be1508b80 4 client.16359 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:18:39.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9be1508b80 1 -- 172.21.15.62:0/856352977 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bd9d681050 con 0x55bd9d7df110 2023-07-14T16:18:39.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.871+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd9d7df110 0x55bd9d7e1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:39.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9bc77fe700 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd9d7df110 0x55bd9d7e1520 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cae72d40f304e00 server_cookie=29308e7a40f24ad0 in_seq=0 out_seq=0 2023-07-14T16:18:39.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9ba6ffd700 10 client.16359 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:39.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9ba6ffd700 1 -- 172.21.15.62:0/856352977 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bd9d681050 con 0x55bd9d7df110 2023-07-14T16:18:39.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9ba6ffd700 10 client.16359 handle_command_reply: tid=0 2023-07-14T16:18:39.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 client.16359 shutdown 2023-07-14T16:18:39.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 -- 172.21.15.62:0/856352977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 msgr2=0x55bd9d785500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d785500 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc005910 tx=0x7f9bbc0051f0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 -- 172.21.15.62:0/856352977 shutdown_connections 2023-07-14T16:18:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd9d7df110 0x55bd9d7e1520 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd9d775c70 0x55bd9d7835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd9d7744d0 0x55bd9d784fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 --2- 172.21.15.62:0/856352977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd9d774e20 0x55bd9d785500 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9ba6ffd700 0 client.16359 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:39.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.872+0000 7f9be1508b80 1 -- 172.21.15.62:0/856352977 >> 172.21.15.62:0/856352977 conn(0x55bd9d769da0 msgr2=0x55bd9d785f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.873+0000 7f9be1508b80 1 -- 172.21.15.62:0/856352977 shutdown_connections 2023-07-14T16:18:39.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.873+0000 7f9be1508b80 1 -- 172.21.15.62:0/856352977 wait complete. 2023-07-14T16:18:39.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.873+0000 7f9be1508b80 20 client.16359 trim_cache size 0 max 16384 2023-07-14T16:18:39.861 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:18:39.861 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:18:39.861 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:18:39.862 INFO:teuthology.orchestra.run.smithi062.stdout: "23d81de3-c204-4a85-9447-2e70dda88623": { 2023-07-14T16:18:39.862 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:18:39.862 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "23d81de3-c204-4a85-9447-2e70dda88623", 2023-07-14T16:18:39.862 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:18:39.862 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:18:39.863 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:18:39.863 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:18:39.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.873+0000 7f9bcdbff700 1 -- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bb00a11d0 msgr2=0x7f9bb00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.873+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bb00a11d0 0x7f9bb00a3680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00199a0 tx=0x7f9bc000d000 comp rx=0 tx=0).stop 2023-07-14T16:18:39.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.873+0000 7f9bcdbff700 1 -- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8101d00 msgr2=0x7f9bc81979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:39.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.873+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8101d00 0x7f9bc81979b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800e970 tx=0x7f9bb800ec80 comp rx=0 tx=0).stop 2023-07-14T16:18:39.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.874+0000 7f9bcdbff700 1 -- 172.21.15.62:0/3134388251 shutdown_connections 2023-07-14T16:18:39.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.874+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bc8103880 0x7f9bc819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.874+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bc8101d00 0x7f9bc81979b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.874+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bb00a11d0 0x7f9bb00a3680 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.874+0000 7f9bcdbff700 1 --2- 172.21.15.62:0/3134388251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9bc8102ef0 0x7f9bc8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:39.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.874+0000 7f9bcdbff700 1 -- 172.21.15.62:0/3134388251 >> 172.21.15.62:0/3134388251 conn(0x7f9bc80fd320 msgr2=0x7f9bc80ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:39.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.874+0000 7f9bcdbff700 1 -- 172.21.15.62:0/3134388251 shutdown_connections 2023-07-14T16:18:39.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:18:39.874+0000 7f9bcdbff700 1 -- 172.21.15.62:0/3134388251 wait complete. 2023-07-14T16:18:39.869 INFO:tasks.cephfs.filesystem:scrub status for tag:23d81de3-c204-4a85-9447-2e70dda88623 - {'path': '/', 'tag': '23d81de3-c204-4a85-9447-2e70dda88623', 'options': 'recursive,force'} 2023-07-14T16:18:39.903 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:18:39 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:18:39.563+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v101>,<0x10000000003/client.0 v102>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v114>,<0x10000000001/sv_0 v128>,<0x10000000000/_nogroup v122>,<0x1/volumes v112>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:18:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: pgmap v448: 97 pgs: 97 active+clean; 1.0 GiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:18:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:18:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:41.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:18:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:18:41.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: pgmap v448: 97 pgs: 97 active+clean; 1.0 GiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:18:41.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:18:41.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:41.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:41.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:41.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:18:41.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:41.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:18:41.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: pgmap v448: 97 pgs: 97 active+clean; 1.0 GiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:18:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:18:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:41.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:41.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:41.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:18:41.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:41.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:18:41.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:18:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 -- 172.21.15.62:0/1135250824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec100620 msgr2=0x7f2aec102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 --2- 172.21.15.62:0/1135250824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec100620 0x7f2aec102a00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009990 tx=0x7f2adc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 -- 172.21.15.62:0/1135250824 shutdown_connections 2023-07-14T16:18:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 --2- 172.21.15.62:0/1135250824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec105860 0x7f2aec107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 --2- 172.21.15.62:0/1135250824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec100620 0x7f2aec102a00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 --2- 172.21.15.62:0/1135250824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec102f40 0x7f2aec105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 -- 172.21.15.62:0/1135250824 >> 172.21.15.62:0/1135250824 conn(0x7f2aec0fa000 msgr2=0x7f2aec0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 -- 172.21.15.62:0/1135250824 shutdown_connections 2023-07-14T16:18:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.698+0000 7f2af1fff700 1 -- 172.21.15.62:0/1135250824 wait complete. 2023-07-14T16:18:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.699+0000 7f2af1fff700 1 Processor -- start 2023-07-14T16:18:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.699+0000 7f2af1fff700 1 -- start start 2023-07-14T16:18:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2af1fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100620 0x7f2aec197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2af1fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec102f40 0x7f2aec197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2af1fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105860 0x7f2aec198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2af1fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec199470 con 0x7f2aec105860 2023-07-14T16:18:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2af1fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec1995e0 con 0x7f2aec102f40 2023-07-14T16:18:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2af1fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec1997a0 con 0x7f2aec100620 2023-07-14T16:18:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2aebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105860 0x7f2aec198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2aebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105860 0x7f2aec198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60280/0 (socket says 172.21.15.62:60280) 2023-07-14T16:18:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.700+0000 7f2aebfff700 1 -- 172.21.15.62:0/2902237629 learned_addr learned my addr 172.21.15.62:0/2902237629 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100620 0x7f2aec197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec102f40 0x7f2aec197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2aebfff700 1 -- 172.21.15.62:0/2902237629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100620 msgr2=0x7f2aec197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2aebfff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100620 0x7f2aec197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2aebfff700 1 -- 172.21.15.62:0/2902237629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec102f40 msgr2=0x7f2aec197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2aebfff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec102f40 0x7f2aec197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2aebfff700 1 -- 172.21.15.62:0/2902237629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0095f0 con 0x7f2aec105860 2023-07-14T16:18:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2aebfff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105860 0x7f2aec198e10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400efc0 tx=0x7f2ae400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2ae8ff9700 1 -- 172.21.15.62:0/2902237629 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4018070 con 0x7f2aec105860 2023-07-14T16:18:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2ae8ff9700 1 -- 172.21.15.62:0/2902237629 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ae400f040 con 0x7f2aec105860 2023-07-14T16:18:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2af1fff700 1 -- 172.21.15.62:0/2902237629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec199a20 con 0x7f2aec105860 2023-07-14T16:18:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.701+0000 7f2ae8ff9700 1 -- 172.21.15.62:0/2902237629 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4007b60 con 0x7f2aec105860 2023-07-14T16:18:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.702+0000 7f2af1fff700 1 -- 172.21.15.62:0/2902237629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec1a0240 con 0x7f2aec105860 2023-07-14T16:18:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.705+0000 7f2ae8ff9700 1 -- 172.21.15.62:0/2902237629 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2ae4007cc0 con 0x7f2aec105860 2023-07-14T16:18:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2b059fdb80 1 Processor -- start 2023-07-14T16:18:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2b059fdb80 1 -- start start 2023-07-14T16:18:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f0fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f10970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f11320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8e0cc50 con 0x555eb8d636b0 2023-07-14T16:18:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8e5f9e0 con 0x555eb8f100f0 2023-07-14T16:18:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8e58e70 con 0x555eb8f10eb0 2023-07-14T16:18:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.707+0000 7f2ae8ff9700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ad40a11d0 0x7f2ad40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2ae8ff9700 1 -- 172.21.15.62:0/2902237629 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2ae40cc900 con 0x7f2aec105860 2023-07-14T16:18:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f10970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f10970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59420/0 (socket says 172.21.15.62:59420) 2023-07-14T16:18:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeb7fe700 1 -- 172.21.15.62:0/1968739638 learned_addr learned my addr 172.21.15.62:0/1968739638 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aebfff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ad40a11d0 0x7f2ad40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f11320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f0fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeb7fe700 1 -- 172.21.15.62:0/1968739638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 msgr2=0x555eb8f11320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f11320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeb7fe700 1 -- 172.21.15.62:0/1968739638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 msgr2=0x555eb8f0fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f0fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeb7fe700 1 -- 172.21.15.62:0/1968739638 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb8c86d80 con 0x555eb8f100f0 2023-07-14T16:18:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.708+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f11320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.709+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f10970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0059d0 tx=0x7f2adc005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4d24d4e82872e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.709+0000 7f2aebfff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ad40a11d0 0x7f2ad40a3680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4096250 tx=0x7f2ae4007f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.712+0000 7f2ad1ffb700 1 -- 172.21.15.62:0/1968739638 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc01d070 con 0x555eb8f100f0 2023-07-14T16:18:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.712+0000 7f2ad1ffb700 1 -- 172.21.15.62:0/1968739638 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2adc01e040 con 0x555eb8f100f0 2023-07-14T16:18:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.712+0000 7f2ad1ffb700 1 -- 172.21.15.62:0/1968739638 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00b710 con 0x555eb8f100f0 2023-07-14T16:18:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 -- 172.21.15.62:0/1968739638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 msgr2=0x555eb8f10970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f10970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0059d0 tx=0x7f2adc005bc0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 -- 172.21.15.62:0/1968739638 shutdown_connections 2023-07-14T16:18:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f11320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/1968739638 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f10970 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 -- 172.21.15.62:0/1968739638 >> 172.21.15.62:0/1968739638 conn(0x555eb8efcbe0 msgr2=0x555eb8f01cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 -- 172.21.15.62:0/1968739638 shutdown_connections 2023-07-14T16:18:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.713+0000 7f2b059fdb80 1 -- 172.21.15.62:0/1968739638 wait complete. 2023-07-14T16:18:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.714+0000 7f2b059fdb80 1 -- start start 2023-07-14T16:18:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.714+0000 7f2b059fdb80 1 Processor -- start 2023-07-14T16:18:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 0x555eb8f19060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f100f0 0x555eb8f195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f10eb0 0x555eb8f205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8df8a60 con 0x555eb8f10eb0 2023-07-14T16:18:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8df8760 con 0x555eb8d636b0 2023-07-14T16:18:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8cdc8c0 con 0x555eb8f100f0 2023-07-14T16:18:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2b059fdb80 10 client.0 _ll_register_callbacks cb 0x555eb8df6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2aeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f10eb0 0x555eb8f205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2aeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f10eb0 0x555eb8f205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60298/0 (socket says 172.21.15.62:60298) 2023-07-14T16:18:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2aeaffd700 1 -- 172.21.15.62:0/4151235919 learned_addr learned my addr 172.21.15.62:0/4151235919 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 0x555eb8f19060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.715+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f100f0 0x555eb8f195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2aeaffd700 1 -- 172.21.15.62:0/4151235919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f100f0 msgr2=0x555eb8f195a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f100f0 0x555eb8f195a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2aeaffd700 1 -- 172.21.15.62:0/4151235919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 msgr2=0x555eb8f19060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 0x555eb8f19060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2aeaffd700 1 -- 172.21.15.62:0/4151235919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec108860 con 0x555eb8f10eb0 2023-07-14T16:18:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f10eb0 0x555eb8f205e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0011b40 tx=0x7f2ae0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb98405302f0d4ca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2acaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2acaffd700 1 -- 172.21.15.62:0/4151235919 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0016070 con 0x555eb8f10eb0 2023-07-14T16:18:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2acaffd700 1 -- 172.21.15.62:0/4151235919 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2ae0004230 con 0x555eb8f10eb0 2023-07-14T16:18:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.716+0000 7f2b059fdb80 1 -- 172.21.15.62:0/4151235919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eb8db2490 con 0x555eb8f10eb0 2023-07-14T16:18:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.717+0000 7f2acaffd700 1 -- 172.21.15.62:0/4151235919 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ae0011e50 con 0x555eb8f10eb0 2023-07-14T16:18:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.717+0000 7f2b059fdb80 10 client.16371 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.717+0000 7f2b059fdb80 1 -- 172.21.15.62:0/4151235919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555eb8c6fc00 con 0x555eb8f10eb0 2023-07-14T16:18:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.717+0000 7f2acaffd700 1 -- 172.21.15.62:0/4151235919 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555eb8db2490 con 0x555eb8f10eb0 2023-07-14T16:18:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.717+0000 7f2acaffd700 1 -- 172.21.15.62:0/4151235919 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2ae00037d0 con 0x555eb8f10eb0 2023-07-14T16:18:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.717+0000 7f2b059fdb80 10 client.16371 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.717+0000 7f2b059fdb80 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.718+0000 7f2b059fdb80 20 client.16371 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.718+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555eb8f24e60 0x555eb8f27270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.718+0000 7f2b059fdb80 4 client.16371 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.718+0000 7f2b059fdb80 1 -- 172.21.15.62:0/4151235919 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eb8e51e50 con 0x555eb8f24e60 2023-07-14T16:18:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.718+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555eb8f24e60 0x555eb8f27270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.719+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555eb8f24e60 0x555eb8f27270 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c7f5c6889249f5 server_cookie=62eff02761410e00 in_seq=0 out_seq=0 2023-07-14T16:18:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.719+0000 7f2acaffd700 10 client.16371 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:41.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2acaffd700 1 -- 172.21.15.62:0/4151235919 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555eb8e51e50 con 0x555eb8f24e60 2023-07-14T16:18:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2acaffd700 10 client.16371 handle_command_reply: tid=0 2023-07-14T16:18:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 client.16371 shutdown 2023-07-14T16:18:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 -- 172.21.15.62:0/4151235919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f10eb0 msgr2=0x555eb8f205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f10eb0 0x555eb8f205e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0011b40 tx=0x7f2ae0013040 comp rx=0 tx=0).stop 2023-07-14T16:18:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 -- 172.21.15.62:0/4151235919 shutdown_connections 2023-07-14T16:18:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f10eb0 0x555eb8f205e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555eb8f24e60 0x555eb8f27270 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 0x555eb8f19060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/4151235919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f100f0 0x555eb8f195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 -- 172.21.15.62:0/4151235919 >> 172.21.15.62:0/4151235919 conn(0x555eb8efcbe0 msgr2=0x555eb8f014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2acaffd700 0 client.16371 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 -- 172.21.15.62:0/4151235919 shutdown_connections 2023-07-14T16:18:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.720+0000 7f2b059fdb80 1 -- 172.21.15.62:0/4151235919 wait complete. 2023-07-14T16:18:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.721+0000 7f2b059fdb80 20 client.16371 trim_cache size 0 max 16384 2023-07-14T16:18:41.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2b059fdb80 1 Processor -- start 2023-07-14T16:18:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2b059fdb80 1 -- start start 2023-07-14T16:18:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 0x555eb8f171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f100f0 0x555eb8f17720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8db2490 con 0x555eb8f100f0 2023-07-14T16:18:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8f1a250 con 0x555eb8d636b0 2023-07-14T16:18:41.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8f18510 con 0x555eb8f10eb0 2023-07-14T16:18:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2aeb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f100f0 0x555eb8f17720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2aeb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f100f0 0x555eb8f17720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60300/0 (socket says 172.21.15.62:60300) 2023-07-14T16:18:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2aeb7fe700 1 -- 172.21.15.62:0/2230692805 learned_addr learned my addr 172.21.15.62:0/2230692805 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:41.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.741+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 0x555eb8f171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2aeb7fe700 1 -- 172.21.15.62:0/2230692805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 msgr2=0x555eb8f205a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f205a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2aeb7fe700 1 -- 172.21.15.62:0/2230692805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 msgr2=0x555eb8f171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 0x555eb8f171e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2aeb7fe700 1 -- 172.21.15.62:0/2230692805 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb8cad1f0 con 0x555eb8f100f0 2023-07-14T16:18:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:18:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f100f0 0x555eb8f17720 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0098e0 tx=0x7f2adc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30aeb846face4ec2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2ad27fc700 1 -- 172.21.15.62:0/2230692805 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc01d040 con 0x555eb8f100f0 2023-07-14T16:18:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2ad27fc700 1 -- 172.21.15.62:0/2230692805 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2adc034550 con 0x555eb8f100f0 2023-07-14T16:18:41.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.742+0000 7f2ad27fc700 1 -- 172.21.15.62:0/2230692805 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00b710 con 0x555eb8f100f0 2023-07-14T16:18:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2230692805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f100f0 msgr2=0x555eb8f17720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f100f0 0x555eb8f17720 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0098e0 tx=0x7f2adc01a040 comp rx=0 tx=0).stop 2023-07-14T16:18:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2230692805 shutdown_connections 2023-07-14T16:18:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8d636b0 0x555eb8f171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2230692805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8f100f0 0x555eb8f17720 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2230692805 >> 172.21.15.62:0/2230692805 conn(0x555eb8efcbe0 msgr2=0x555eb8f17d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2230692805 shutdown_connections 2023-07-14T16:18:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.743+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2230692805 wait complete. 2023-07-14T16:18:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.744+0000 7f2b059fdb80 1 -- start start 2023-07-14T16:18:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.744+0000 7f2b059fdb80 1 Processor -- start 2023-07-14T16:18:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.744+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f1bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.744+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.744+0000 7f2b059fdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8f1f660 con 0x555eb8d636b0 2023-07-14T16:18:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8f1f7d0 con 0x555eb8f100f0 2023-07-14T16:18:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2b059fdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb8f1f990 con 0x555eb8f10eb0 2023-07-14T16:18:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2b059fdb80 10 client.0 _ll_register_callbacks cb 0x7f2ae0011830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f1bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f1bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60312/0 (socket says 172.21.15.62:60312) 2023-07-14T16:18:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeaffd700 1 -- 172.21.15.62:0/2852951115 learned_addr learned my addr 172.21.15.62:0/2852951115 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeaffd700 1 -- 172.21.15.62:0/2852951115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 msgr2=0x555eb8f1f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeaffd700 1 -- 172.21.15.62:0/2852951115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 msgr2=0x555eb8f1c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f1c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.745+0000 7f2aeaffd700 1 -- 172.21.15.62:0/2852951115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eb8c86d80 con 0x555eb8d636b0 2023-07-14T16:18:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2aeaffd700 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f1bbc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0020490 tx=0x7f2ae0011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856d504e81b3b699 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:41.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2acaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2acaffd700 1 -- 172.21.15.62:0/2852951115 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0016040 con 0x555eb8d636b0 2023-07-14T16:18:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2852951115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2adc01e040 con 0x555eb8d636b0 2023-07-14T16:18:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2acaffd700 1 -- 172.21.15.62:0/2852951115 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2ae0005b00 con 0x555eb8d636b0 2023-07-14T16:18:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2acaffd700 1 -- 172.21.15.62:0/2852951115 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ae0003880 con 0x555eb8d636b0 2023-07-14T16:18:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2b059fdb80 10 client.16377 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2852951115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555eb8c85250 con 0x555eb8d636b0 2023-07-14T16:18:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.746+0000 7f2acaffd700 1 -- 172.21.15.62:0/2852951115 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2adc01e040 con 0x555eb8d636b0 2023-07-14T16:18:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.747+0000 7f2acaffd700 1 -- 172.21.15.62:0/2852951115 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2ae0020b70 con 0x555eb8d636b0 2023-07-14T16:18:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.747+0000 7f2b059fdb80 10 client.16377 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.747+0000 7f2b059fdb80 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.747+0000 7f2b059fdb80 20 client.16377 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.747+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555eb8f795a0 0x555eb8f7b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.747+0000 7f2b059fdb80 4 client.16377 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:18:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.747+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2852951115 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555eb8e13ff0 con 0x555eb8f795a0 2023-07-14T16:18:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.747+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555eb8f795a0 0x555eb8f7b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.748+0000 7f2aeb7fe700 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555eb8f795a0 0x555eb8f7b9b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b49dfa230cb2a1e server_cookie=db2ae1e9cc037cf1 in_seq=0 out_seq=0 2023-07-14T16:18:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.748+0000 7f2acaffd700 10 client.16377 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2acaffd700 1 -- 172.21.15.62:0/2852951115 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555eb8e13ff0 con 0x555eb8f795a0 2023-07-14T16:18:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2acaffd700 10 client.16377 handle_command_reply: tid=0 2023-07-14T16:18:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 client.16377 shutdown 2023-07-14T16:18:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2852951115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 msgr2=0x555eb8f1bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f1bbc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0020490 tx=0x7f2ae0011580 comp rx=0 tx=0).stop 2023-07-14T16:18:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2852951115 shutdown_connections 2023-07-14T16:18:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555eb8f10eb0 0x555eb8f1f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb8d636b0 0x555eb8f1bbc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555eb8f795a0 0x555eb8f7b9b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 --2- 172.21.15.62:0/2852951115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555eb8f100f0 0x555eb8f1c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2852951115 >> 172.21.15.62:0/2852951115 conn(0x555eb8efcbe0 msgr2=0x555eb8f171e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:41.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2acaffd700 0 client.16377 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2852951115 shutdown_connections 2023-07-14T16:18:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 1 -- 172.21.15.62:0/2852951115 wait complete. 2023-07-14T16:18:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.757+0000 7f2b059fdb80 20 client.16377 trim_cache size 0 max 16384 2023-07-14T16:18:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:18:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:18:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:18:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:18:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 -- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ad40a11d0 msgr2=0x7f2ad40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ad40a11d0 0x7f2ad40a3680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4096250 tx=0x7f2ae4007f70 comp rx=0 tx=0).stop 2023-07-14T16:18:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 -- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105860 msgr2=0x7f2aec198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105860 0x7f2aec198e10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400efc0 tx=0x7f2ae400c3a0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 -- 172.21.15.62:0/2902237629 shutdown_connections 2023-07-14T16:18:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec105860 0x7f2aec198e10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec100620 0x7f2aec197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ad40a11d0 0x7f2ad40a3680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 --2- 172.21.15.62:0/2902237629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2aec102f40 0x7f2aec197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 -- 172.21.15.62:0/2902237629 >> 172.21.15.62:0/2902237629 conn(0x7f2aec0fa000 msgr2=0x7f2aec0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 -- 172.21.15.62:0/2902237629 shutdown_connections 2023-07-14T16:18:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:41.758+0000 7f2af1fff700 1 -- 172.21.15.62:0/2902237629 wait complete. 2023-07-14T16:18:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:42 smithi164 ceph-mon[97427]: pgmap v449: 97 pgs: 97 active+clean; 1.1 GiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:18:43.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:42 smithi062 ceph-mon[95543]: pgmap v449: 97 pgs: 97 active+clean; 1.1 GiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:18:43.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:42 smithi081 ceph-mon[127795]: pgmap v449: 97 pgs: 97 active+clean; 1.1 GiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:18:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:44 smithi164 ceph-mon[97427]: pgmap v450: 97 pgs: 97 active+clean; 1.1 GiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:18:45.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:44 smithi062 ceph-mon[95543]: pgmap v450: 97 pgs: 97 active+clean; 1.1 GiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:18:45.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:44 smithi081 ceph-mon[127795]: pgmap v450: 97 pgs: 97 active+clean; 1.1 GiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:18:47.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:46 smithi164 ceph-mon[97427]: pgmap v451: 97 pgs: 97 active+clean; 1.1 GiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:18:47.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:46 smithi062 ceph-mon[95543]: pgmap v451: 97 pgs: 97 active+clean; 1.1 GiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:18:47.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:46 smithi081 ceph-mon[127795]: pgmap v451: 97 pgs: 97 active+clean; 1.1 GiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:18:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:48 smithi164 ceph-mon[97427]: pgmap v452: 97 pgs: 97 active+clean; 1.1 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:18:49.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:48 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:18:49.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:48 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:18:49.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:48 smithi062 ceph-mon[95543]: pgmap v452: 97 pgs: 97 active+clean; 1.1 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:18:49.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:48 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:18:49.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:48 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:18:49.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:48 smithi081 ceph-mon[127795]: pgmap v452: 97 pgs: 97 active+clean; 1.1 GiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:18:49.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:48 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:18:49.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:48 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:18:51.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:50 smithi164 ceph-mon[97427]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:18:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:50 smithi062 ceph-mon[95543]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:18:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:50 smithi081 ceph-mon[127795]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:18:53.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:52 smithi164 ceph-mon[97427]: pgmap v454: 97 pgs: 97 active+clean; 1.2 GiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:18:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:52 smithi062 ceph-mon[95543]: pgmap v454: 97 pgs: 97 active+clean; 1.2 GiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:18:53.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:52 smithi081 ceph-mon[127795]: pgmap v454: 97 pgs: 97 active+clean; 1.2 GiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:18:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.935+0000 7fa46af9d700 1 -- 172.21.15.62:0/502407014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 msgr2=0x7fa464107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.935+0000 7fa46af9d700 1 --2- 172.21.15.62:0/502407014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 0x7fa464107d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00b0a0 tx=0x7fa45c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.935+0000 7fa46af9d700 1 -- 172.21.15.62:0/502407014 shutdown_connections 2023-07-14T16:18:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.935+0000 7fa46af9d700 1 --2- 172.21.15.62:0/502407014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 0x7fa464107d80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.935+0000 7fa46af9d700 1 --2- 172.21.15.62:0/502407014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4640ff940 0x7fa4640ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.935+0000 7fa46af9d700 1 --2- 172.21.15.62:0/502407014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa464100b30 0x7fa464100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.935+0000 7fa46af9d700 1 -- 172.21.15.62:0/502407014 >> 172.21.15.62:0/502407014 conn(0x7fa4640faf60 msgr2=0x7fa4640fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.936+0000 7fa46af9d700 1 -- 172.21.15.62:0/502407014 shutdown_connections 2023-07-14T16:18:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.936+0000 7fa46af9d700 1 -- 172.21.15.62:0/502407014 wait complete. 2023-07-14T16:18:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.937+0000 7fa46af9d700 1 Processor -- start 2023-07-14T16:18:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.937+0000 7fa46af9d700 1 -- start start 2023-07-14T16:18:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.937+0000 7fa46af9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4640ff940 0x7fa4640742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.937+0000 7fa46af9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa464100b30 0x7fa464072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.937+0000 7fa46af9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 0x7fa464072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.937+0000 7fa46af9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4640735e0 con 0x7fa4641014c0 2023-07-14T16:18:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.937+0000 7fa46af9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464073750 con 0x7fa4640ff940 2023-07-14T16:18:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.937+0000 7fa46af9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46419f6d0 con 0x7fa464100b30 2023-07-14T16:18:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa469f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4640ff940 0x7fa4640742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 0x7fa464072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46979a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa464100b30 0x7fa464072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa469f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4640ff940 0x7fa4640742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47116/0 (socket says 172.21.15.62:47116) 2023-07-14T16:18:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa469f9b700 1 -- 172.21.15.62:0/1999112400 learned_addr learned my addr 172.21.15.62:0/1999112400 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46979a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa464100b30 0x7fa464072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35302/0 (socket says 172.21.15.62:35302) 2023-07-14T16:18:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46a79c700 1 -- 172.21.15.62:0/1999112400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa464100b30 msgr2=0x7fa464072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46a79c700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa464100b30 0x7fa464072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46a79c700 1 -- 172.21.15.62:0/1999112400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4640ff940 msgr2=0x7fa4640742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46a79c700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4640ff940 0x7fa4640742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46a79c700 1 -- 172.21.15.62:0/1999112400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c009d00 con 0x7fa4641014c0 2023-07-14T16:18:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.938+0000 7fa46a79c700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 0x7fa464072e40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa45c015000 tx=0x7fa45c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.939+0000 7fa45affd700 1 -- 172.21.15.62:0/1999112400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00e040 con 0x7fa4641014c0 2023-07-14T16:18:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.939+0000 7fa46af9d700 1 -- 172.21.15.62:0/1999112400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46419f950 con 0x7fa4641014c0 2023-07-14T16:18:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.939+0000 7fa45affd700 1 -- 172.21.15.62:0/1999112400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa45c00be80 con 0x7fa4641014c0 2023-07-14T16:18:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.939+0000 7fa45affd700 1 -- 172.21.15.62:0/1999112400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c005040 con 0x7fa4641014c0 2023-07-14T16:18:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.939+0000 7fa46af9d700 1 -- 172.21.15.62:0/1999112400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46419fe20 con 0x7fa4641014c0 2023-07-14T16:18:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.942+0000 7fa45affd700 1 -- 172.21.15.62:0/1999112400 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa45c01d470 con 0x7fa4641014c0 2023-07-14T16:18:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.944+0000 7fa45affd700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa4500a1150 0x7fa4500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.944+0000 7fa45affd700 1 -- 172.21.15.62:0/1999112400 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa45c0cd990 con 0x7fa4641014c0 2023-07-14T16:18:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.944+0000 7fa47fbdab80 1 Processor -- start 2023-07-14T16:18:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.944+0000 7fa469f9b700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa4500a1150 0x7fa4500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.944+0000 7fa47fbdab80 1 -- start start 2023-07-14T16:18:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 0x55b0509ee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0508f3c50 con 0x55b0509ef7e0 2023-07-14T16:18:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0509469e0 con 0x55b0509eea90 2023-07-14T16:18:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05093fe70 con 0x55b05084a6b0 2023-07-14T16:18:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa469f9b700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa4500a1150 0x7fa4500a3600 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa460005950 tx=0x7fa460009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa46a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa46a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509efc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50680/0 (socket says 172.21.15.62:50680) 2023-07-14T16:18:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa46a79c700 1 -- 172.21.15.62:0/1237476364 learned_addr learned my addr 172.21.15.62:0/1237476364 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa46979a700 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa469f9b700 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 0x55b0509ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa46a79c700 1 -- 172.21.15.62:0/1237476364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 msgr2=0x55b0509ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa46a79c700 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 0x55b0509ee550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.945+0000 7fa46a79c700 1 -- 172.21.15.62:0/1237476364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 msgr2=0x55b0509ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.946+0000 7fa46a79c700 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509ef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.946+0000 7fa46a79c700 1 -- 172.21.15.62:0/1237476364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b05076dd80 con 0x55b0509ef7e0 2023-07-14T16:18:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.946+0000 7fa46979a700 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:18:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.946+0000 7fa46a79c700 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509efc50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa45c097850 tx=0x7fa45c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3295e6bad4671a3e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.946+0000 7fa44ffff700 1 -- 172.21.15.62:0/1237476364 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c09ee10 con 0x55b0509ef7e0 2023-07-14T16:18:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.946+0000 7fa44ffff700 1 -- 172.21.15.62:0/1237476364 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa45c0a31a0 con 0x55b0509ef7e0 2023-07-14T16:18:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.946+0000 7fa44ffff700 1 -- 172.21.15.62:0/1237476364 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c01d720 con 0x55b0509ef7e0 2023-07-14T16:18:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1237476364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 msgr2=0x55b0509efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509efc50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa45c097850 tx=0x7fa45c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1237476364 shutdown_connections 2023-07-14T16:18:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509efc50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 0x55b0509ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/1237476364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1237476364 >> 172.21.15.62:0/1237476364 conn(0x55b0509e3d40 msgr2=0x55b0509e8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1237476364 shutdown_connections 2023-07-14T16:18:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.947+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1237476364 wait complete. 2023-07-14T16:18:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.948+0000 7fa47fbdab80 1 -- start start 2023-07-14T16:18:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.948+0000 7fa47fbdab80 1 Processor -- start 2023-07-14T16:18:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.948+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b05084a6b0 0x55b0509f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.948+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.948+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509ef7e0 0x55b0509febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.948+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0508dfa60 con 0x55b05084a6b0 2023-07-14T16:18:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.948+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0508df760 con 0x55b0509eea90 2023-07-14T16:18:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.948+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0507c38c0 con 0x55b0509ef7e0 2023-07-14T16:18:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa47fbdab80 10 client.0 _ll_register_callbacks cb 0x55b0508dd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa469f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b05084a6b0 0x55b0509f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa469f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b05084a6b0 0x55b0509f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50692/0 (socket says 172.21.15.62:50692) 2023-07-14T16:18:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa469f9b700 1 -- 172.21.15.62:0/4237743074 learned_addr learned my addr 172.21.15.62:0/4237743074 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa46979a700 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa46a79c700 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509ef7e0 0x55b0509febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa469f9b700 1 -- 172.21.15.62:0/4237743074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509ef7e0 msgr2=0x55b0509febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa469f9b700 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509ef7e0 0x55b0509febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa469f9b700 1 -- 172.21.15.62:0/4237743074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 msgr2=0x55b0509f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa469f9b700 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.949+0000 7fa469f9b700 1 -- 172.21.15.62:0/4237743074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46419fe20 con 0x55b05084a6b0 2023-07-14T16:18:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.950+0000 7fa469f9b700 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b05084a6b0 0x55b0509f7660 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa46001ec10 tx=0x7fa4600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450d8fce668348f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.950+0000 7fa44d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.950+0000 7fa44d7fa700 1 -- 172.21.15.62:0/4237743074 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46000ec50 con 0x55b05084a6b0 2023-07-14T16:18:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.950+0000 7fa44d7fa700 1 -- 172.21.15.62:0/4237743074 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa46000edb0 con 0x55b05084a6b0 2023-07-14T16:18:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.950+0000 7fa47fbdab80 1 -- 172.21.15.62:0/4237743074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b050899490 con 0x55b05084a6b0 2023-07-14T16:18:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.950+0000 7fa44d7fa700 1 -- 172.21.15.62:0/4237743074 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4600263f0 con 0x55b05084a6b0 2023-07-14T16:18:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.950+0000 7fa47fbdab80 10 client.16389 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.950+0000 7fa47fbdab80 1 -- 172.21.15.62:0/4237743074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b050756c00 con 0x55b05084a6b0 2023-07-14T16:18:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.951+0000 7fa44d7fa700 1 -- 172.21.15.62:0/4237743074 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b050899490 con 0x55b05084a6b0 2023-07-14T16:18:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.951+0000 7fa44d7fa700 1 -- 172.21.15.62:0/4237743074 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa46001c5d0 con 0x55b05084a6b0 2023-07-14T16:18:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.951+0000 7fa47fbdab80 10 client.16389 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.951+0000 7fa47fbdab80 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.951+0000 7fa47fbdab80 20 client.16389 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.951+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b050a03510 0x55b050a05920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.951+0000 7fa47fbdab80 4 client.16389 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:18:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.951+0000 7fa47fbdab80 1 -- 172.21.15.62:0/4237743074 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b050938e50 con 0x55b050a03510 2023-07-14T16:18:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.952+0000 7fa46979a700 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b050a03510 0x55b050a05920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.952+0000 7fa46979a700 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b050a03510 0x55b050a05920 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f102ec17a9ac0ea5 server_cookie=22cb7d052fcc596f in_seq=0 out_seq=0 2023-07-14T16:18:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.952+0000 7fa44d7fa700 10 client.16389 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa44d7fa700 1 -- 172.21.15.62:0/4237743074 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b050938e50 con 0x55b050a03510 2023-07-14T16:18:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa44d7fa700 10 client.16389 handle_command_reply: tid=0 2023-07-14T16:18:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 client.16389 shutdown 2023-07-14T16:18:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 -- 172.21.15.62:0/4237743074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b05084a6b0 msgr2=0x55b0509f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b05084a6b0 0x55b0509f7660 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa46001ec10 tx=0x7fa4600047c0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 -- 172.21.15.62:0/4237743074 shutdown_connections 2023-07-14T16:18:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509ef7e0 0x55b0509febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b05084a6b0 0x55b0509f7660 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b050a03510 0x55b050a05920 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/4237743074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa44d7fa700 0 client.16389 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 -- 172.21.15.62:0/4237743074 >> 172.21.15.62:0/4237743074 conn(0x55b0509e3d40 msgr2=0x55b0509e8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 -- 172.21.15.62:0/4237743074 shutdown_connections 2023-07-14T16:18:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.953+0000 7fa47fbdab80 1 -- 172.21.15.62:0/4237743074 wait complete. 2023-07-14T16:18:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.954+0000 7fa47fbdab80 20 client.16389 trim_cache size 0 max 16384 2023-07-14T16:18:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.972+0000 7fa47fbdab80 1 Processor -- start 2023-07-14T16:18:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.972+0000 7fa47fbdab80 1 -- start start 2023-07-14T16:18:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.972+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b05084a6b0 0x55b0509f7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.972+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509eea90 0x55b0509feba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.972+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509f7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.972+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0509f7230 con 0x55b0509ef7e0 2023-07-14T16:18:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.972+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050899490 con 0x55b05084a6b0 2023-07-14T16:18:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.972+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0509f6b10 con 0x55b0509eea90 2023-07-14T16:18:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa46a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa469f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b05084a6b0 0x55b0509f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa469f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b05084a6b0 0x55b0509f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47148/0 (socket says 172.21.15.62:47148) 2023-07-14T16:18:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa469f9b700 1 -- 172.21.15.62:0/1165492144 learned_addr learned my addr 172.21.15.62:0/1165492144 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa46979a700 1 --2- 172.21.15.62:0/1165492144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509eea90 0x55b0509feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa469f9b700 1 -- 172.21.15.62:0/1165492144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509eea90 msgr2=0x55b0509feba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa469f9b700 1 --2- 172.21.15.62:0/1165492144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509eea90 0x55b0509feba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa469f9b700 1 -- 172.21.15.62:0/1165492144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 msgr2=0x55b0509f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa469f9b700 1 --2- 172.21.15.62:0/1165492144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.973+0000 7fa469f9b700 1 -- 172.21.15.62:0/1165492144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0507941f0 con 0x55b05084a6b0 2023-07-14T16:18:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.974+0000 7fa469f9b700 1 --2- 172.21.15.62:0/1165492144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b05084a6b0 0x55b0509f7410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa460000c00 tx=0x7fa460011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0e249842d4213b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.974+0000 7fa458ff9700 1 -- 172.21.15.62:0/1165492144 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46001c430 con 0x55b05084a6b0 2023-07-14T16:18:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.974+0000 7fa458ff9700 1 -- 172.21.15.62:0/1165492144 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa46001c590 con 0x55b05084a6b0 2023-07-14T16:18:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.974+0000 7fa458ff9700 1 -- 172.21.15.62:0/1165492144 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460004e90 con 0x55b05084a6b0 2023-07-14T16:18:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1165492144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b05084a6b0 msgr2=0x55b0509f7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/1165492144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b05084a6b0 0x55b0509f7410 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa460000c00 tx=0x7fa460011af0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1165492144 shutdown_connections 2023-07-14T16:18:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/1165492144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/1165492144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b05084a6b0 0x55b0509f7410 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/1165492144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0509eea90 0x55b0509feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1165492144 >> 172.21.15.62:0/1165492144 conn(0x55b0509e3d40 msgr2=0x55b050a0a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1165492144 shutdown_connections 2023-07-14T16:18:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 -- 172.21.15.62:0/1165492144 wait complete. 2023-07-14T16:18:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.975+0000 7fa47fbdab80 1 -- start start 2023-07-14T16:18:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.976+0000 7fa47fbdab80 1 Processor -- start 2023-07-14T16:18:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.976+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 0x55b0509faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.976+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509faf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.976+0000 7fa47fbdab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.976+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0509fe4e0 con 0x55b0509ef7e0 2023-07-14T16:18:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.976+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0509fe650 con 0x55b0509eea90 2023-07-14T16:18:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.976+0000 7fa47fbdab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0509fe810 con 0x55b05084a6b0 2023-07-14T16:18:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.976+0000 7fa47fbdab80 10 client.0 _ll_register_callbacks cb 0x7fa46001e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:18:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa469f9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 0x55b0509faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509fdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50716/0 (socket says 172.21.15.62:50716) 2023-07-14T16:18:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 -- 172.21.15.62:0/3637113195 learned_addr learned my addr 172.21.15.62:0/3637113195 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:18:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46979a700 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 -- 172.21.15.62:0/3637113195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 msgr2=0x55b0509faa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 0x55b0509faa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 -- 172.21.15.62:0/3637113195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 msgr2=0x55b0509faf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:18:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509faf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 -- 172.21.15.62:0/3637113195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b050756c00 con 0x55b0509ef7e0 2023-07-14T16:18:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa46a79c700 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509fdfa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa45c097850 tx=0x7fa45c097120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc83a8d31d7c604 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:18:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.977+0000 7fa44d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:18:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa44d7fa700 1 -- 172.21.15.62:0/3637113195 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c09ee10 con 0x55b0509ef7e0 2023-07-14T16:18:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa44d7fa700 1 -- 172.21.15.62:0/3637113195 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa45c0cb1e0 con 0x55b0509ef7e0 2023-07-14T16:18:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 1 -- 172.21.15.62:0/3637113195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa45c0a31a0 con 0x55b0509ef7e0 2023-07-14T16:18:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa44d7fa700 1 -- 172.21.15.62:0/3637113195 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa45c0974f0 con 0x55b0509ef7e0 2023-07-14T16:18:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 10 client.16395 fetch_fsmap learned FSMap version 24 2023-07-14T16:18:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 1 -- 172.21.15.62:0/3637113195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b05076c250 con 0x55b0509ef7e0 2023-07-14T16:18:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa44d7fa700 1 -- 172.21.15.62:0/3637113195 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa45c07d080 con 0x55b0509ef7e0 2023-07-14T16:18:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa44d7fa700 1 -- 172.21.15.62:0/3637113195 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa45c09b0b0 con 0x55b0509ef7e0 2023-07-14T16:18:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 10 client.16395 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:18:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:18:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 20 client.16395 populate_metadata read hostname 'smithi062' 2023-07-14T16:18:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b050a57f90 0x55b050a5a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:18:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 4 client.16395 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:18:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.978+0000 7fa47fbdab80 1 -- 172.21.15.62:0/3637113195 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b0508faff0 con 0x55b050a57f90 2023-07-14T16:18:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.979+0000 7fa469f9b700 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b050a57f90 0x55b050a5a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:18:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.979+0000 7fa44d7fa700 10 client.16395 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.979+0000 7fa469f9b700 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b050a57f90 0x55b050a5a370 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d383e47fa35dd3 server_cookie=30118dc2609e3654 in_seq=0 out_seq=0 2023-07-14T16:18:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa44d7fa700 1 -- 172.21.15.62:0/3637113195 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b0508faff0 con 0x55b050a57f90 2023-07-14T16:18:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa44d7fa700 10 client.16395 handle_command_reply: tid=0 2023-07-14T16:18:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 client.16395 shutdown 2023-07-14T16:18:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 -- 172.21.15.62:0/3637113195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 msgr2=0x55b0509fdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509fdfa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa45c097850 tx=0x7fa45c097120 comp rx=0 tx=0).stop 2023-07-14T16:18:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 -- 172.21.15.62:0/3637113195 shutdown_connections 2023-07-14T16:18:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b0509ef7e0 0x55b0509fdfa0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b05084a6b0 0x55b0509faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b050a57f90 0x55b050a5a370 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 --2- 172.21.15.62:0/3637113195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b0509eea90 0x55b0509faf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa44d7fa700 0 client.16395 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:18:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 -- 172.21.15.62:0/3637113195 >> 172.21.15.62:0/3637113195 conn(0x55b0509e3d40 msgr2=0x55b0509f7410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 -- 172.21.15.62:0/3637113195 shutdown_connections 2023-07-14T16:18:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 1 -- 172.21.15.62:0/3637113195 wait complete. 2023-07-14T16:18:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.989+0000 7fa47fbdab80 20 client.16395 trim_cache size 0 max 16384 2023-07-14T16:18:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:18:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:18:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:18:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:18:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 -- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa4500a1150 msgr2=0x7fa4500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa4500a1150 0x7fa4500a3600 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa460005950 tx=0x7fa460009260 comp rx=0 tx=0).stop 2023-07-14T16:18:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 -- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 msgr2=0x7fa464072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:18:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 0x7fa464072e40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa45c015000 tx=0x7fa45c00ba50 comp rx=0 tx=0).stop 2023-07-14T16:18:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 -- 172.21.15.62:0/1999112400 shutdown_connections 2023-07-14T16:18:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4641014c0 0x7fa464072e40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa4500a1150 0x7fa4500a3600 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa4640ff940 0x7fa4640742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 --2- 172.21.15.62:0/1999112400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa464100b30 0x7fa464072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:18:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 -- 172.21.15.62:0/1999112400 >> 172.21.15.62:0/1999112400 conn(0x7fa4640faf60 msgr2=0x7fa464103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:18:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 -- 172.21.15.62:0/1999112400 shutdown_connections 2023-07-14T16:18:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:18:54.990+0000 7fa46af9d700 1 -- 172.21.15.62:0/1999112400 wait complete. 2023-07-14T16:18:55.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:54 smithi164 ceph-mon[97427]: pgmap v455: 97 pgs: 97 active+clean; 1.2 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:18:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:54 smithi062 ceph-mon[95543]: pgmap v455: 97 pgs: 97 active+clean; 1.2 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:18:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:54 smithi081 ceph-mon[127795]: pgmap v455: 97 pgs: 97 active+clean; 1.2 GiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:18:57.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:56 smithi164 ceph-mon[97427]: pgmap v456: 97 pgs: 97 active+clean; 1.2 GiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 14 op/s 2023-07-14T16:18:57.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:56 smithi062 ceph-mon[95543]: pgmap v456: 97 pgs: 97 active+clean; 1.2 GiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 14 op/s 2023-07-14T16:18:57.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:56 smithi081 ceph-mon[127795]: pgmap v456: 97 pgs: 97 active+clean; 1.2 GiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 14 op/s 2023-07-14T16:18:59.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:18:58 smithi164 ceph-mon[97427]: pgmap v457: 97 pgs: 97 active+clean; 1.3 GiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 MiB/s wr, 11 op/s 2023-07-14T16:18:59.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:18:58 smithi062 ceph-mon[95543]: pgmap v457: 97 pgs: 97 active+clean; 1.3 GiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 MiB/s wr, 11 op/s 2023-07-14T16:18:59.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:18:58 smithi081 ceph-mon[127795]: pgmap v457: 97 pgs: 97 active+clean; 1.3 GiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 MiB/s wr, 11 op/s 2023-07-14T16:19:01.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:00 smithi164 ceph-mon[97427]: pgmap v458: 97 pgs: 97 active+clean; 1.3 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:01.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:00 smithi062 ceph-mon[95543]: pgmap v458: 97 pgs: 97 active+clean; 1.3 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:00 smithi081 ceph-mon[127795]: pgmap v458: 97 pgs: 97 active+clean; 1.3 GiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:03.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:02 smithi164 ceph-mon[97427]: pgmap v459: 97 pgs: 97 active+clean; 1.3 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:19:03.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:02 smithi062 ceph-mon[95543]: pgmap v459: 97 pgs: 97 active+clean; 1.3 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:19:03.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:02 smithi081 ceph-mon[127795]: pgmap v459: 97 pgs: 97 active+clean; 1.3 GiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:19:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:04 smithi164 ceph-mon[97427]: pgmap v460: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:19:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:04 smithi062 ceph-mon[95543]: pgmap v460: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:19:05.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:04 smithi081 ceph-mon[127795]: pgmap v460: 97 pgs: 97 active+clean; 1.4 GiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:19:07.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:06 smithi164 ceph-mon[97427]: pgmap v461: 97 pgs: 97 active+clean; 1.4 GiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:19:07.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:06 smithi062 ceph-mon[95543]: pgmap v461: 97 pgs: 97 active+clean; 1.4 GiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:19:07.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:06 smithi081 ceph-mon[127795]: pgmap v461: 97 pgs: 97 active+clean; 1.4 GiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:19:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 -- 172.21.15.62:0/3470125194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 msgr2=0x7fb8f00733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/3470125194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 0x7fb8f00733e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009990 tx=0x7fb8e4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 -- 172.21.15.62:0/3470125194 shutdown_connections 2023-07-14T16:19:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/3470125194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f0107800 0x7fb8f0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/3470125194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8f0074550 0x7fb8f00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/3470125194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 0x7fb8f00733e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 -- 172.21.15.62:0/3470125194 >> 172.21.15.62:0/3470125194 conn(0x7fb8f00faf60 msgr2=0x7fb8f00fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 -- 172.21.15.62:0/3470125194 shutdown_connections 2023-07-14T16:19:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.140+0000 7fb8ef59e700 1 -- 172.21.15.62:0/3470125194 wait complete. 2023-07-14T16:19:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ef59e700 1 Processor -- start 2023-07-14T16:19:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ef59e700 1 -- start start 2023-07-14T16:19:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ef59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 0x7fb8f01977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ef59e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f0074550 0x7fb8f0197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ef59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8f0107800 0x7fb8f019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ef59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019f330 con 0x7fb8f0072f70 2023-07-14T16:19:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ef59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019f470 con 0x7fb8f0107800 2023-07-14T16:19:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ef59e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019f600 con 0x7fb8f0074550 2023-07-14T16:19:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.142+0000 7fb8ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 0x7fb8f01977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 0x7fb8f01977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38112/0 (socket says 172.21.15.62:38112) 2023-07-14T16:19:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8ee59c700 1 -- 172.21.15.62:0/2105038492 learned_addr learned my addr 172.21.15.62:0/2105038492 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f0074550 0x7fb8f0197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8eed9d700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8f0107800 0x7fb8f019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8ee59c700 1 -- 172.21.15.62:0/2105038492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f0074550 msgr2=0x7fb8f0197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f0074550 0x7fb8f0197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8ee59c700 1 -- 172.21.15.62:0/2105038492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8f0107800 msgr2=0x7fb8f019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8f0107800 0x7fb8f019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8ee59c700 1 -- 172.21.15.62:0/2105038492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40095f0 con 0x7fb8f0072f70 2023-07-14T16:19:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8eed9d700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8f0107800 0x7fb8f019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:19:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.143+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 0x7fb8f01977b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000e970 tx=0x7fb8e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.144+0000 7fb8df7fe700 1 -- 172.21.15.62:0/2105038492 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0009790 con 0x7fb8f0072f70 2023-07-14T16:19:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.144+0000 7fb8ef59e700 1 -- 172.21.15.62:0/2105038492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f019f8e0 con 0x7fb8f0072f70 2023-07-14T16:19:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.144+0000 7fb8df7fe700 1 -- 172.21.15.62:0/2105038492 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8e0004cc0 con 0x7fb8f0072f70 2023-07-14T16:19:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.144+0000 7fb8df7fe700 1 -- 172.21.15.62:0/2105038492 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00055b0 con 0x7fb8f0072f70 2023-07-14T16:19:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.144+0000 7fb8ef59e700 1 -- 172.21.15.62:0/2105038492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f019fdf0 con 0x7fb8f0072f70 2023-07-14T16:19:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.148+0000 7fb908455b80 1 Processor -- start 2023-07-14T16:19:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.148+0000 7fb8df7fe700 1 -- 172.21.15.62:0/2105038492 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb8e0004750 con 0x7fb8f0072f70 2023-07-14T16:19:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb908455b80 1 -- start start 2023-07-14T16:19:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb908455b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f38048b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb908455b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 0x55f38048c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb908455b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb908455b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f380390c50 con 0x55f3802e76b0 2023-07-14T16:19:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb908455b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3803e39e0 con 0x55f38048ba90 2023-07-14T16:19:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb908455b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3803dce70 con 0x55f38048c7e0 2023-07-14T16:19:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb8eed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f38048b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb8eed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f38048b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38126/0 (socket says 172.21.15.62:38126) 2023-07-14T16:19:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb8eed9d700 1 -- 172.21.15.62:0/2290511232 learned_addr learned my addr 172.21.15.62:0/2290511232 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.149+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.150+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 0x55f38048c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.150+0000 7fb8edd9b700 1 -- 172.21.15.62:0/2290511232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 msgr2=0x55f38048c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.150+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 0x55f38048c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.150+0000 7fb8edd9b700 1 -- 172.21.15.62:0/2290511232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 msgr2=0x55f38048b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.150+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f38048b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.150+0000 7fb8edd9b700 1 -- 172.21.15.62:0/2290511232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f38020ad80 con 0x55f38048c7e0 2023-07-14T16:19:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.150+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048cc50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4000c00 tx=0x7fb8e4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=111377d865f4c140 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.151+0000 7fb8df7fe700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8d80a11d0 0x7fb8d80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.151+0000 7fb8dcff9700 1 -- 172.21.15.62:0/2290511232 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4005240 con 0x55f38048c7e0 2023-07-14T16:19:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.151+0000 7fb8dcff9700 1 -- 172.21.15.62:0/2290511232 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8e401e040 con 0x55f38048c7e0 2023-07-14T16:19:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.151+0000 7fb8dcff9700 1 -- 172.21.15.62:0/2290511232 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400b970 con 0x55f38048c7e0 2023-07-14T16:19:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.151+0000 7fb8df7fe700 1 -- 172.21.15.62:0/2105038492 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8e0014070 con 0x7fb8f0072f70 2023-07-14T16:19:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.151+0000 7fb8eed9d700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8d80a11d0 0x7fb8d80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 -- 172.21.15.62:0/2290511232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 msgr2=0x55f38048cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048cc50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4000c00 tx=0x7fb8e4005ef0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb8eed9d700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8d80a11d0 0x7fb8d80a3680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55f38048b2b0 tx=0x7fb8e800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 -- 172.21.15.62:0/2290511232 shutdown_connections 2023-07-14T16:19:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048cc50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f38048b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 --2- 172.21.15.62:0/2290511232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 0x55f38048c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 -- 172.21.15.62:0/2290511232 >> 172.21.15.62:0/2290511232 conn(0x55f380480d40 msgr2=0x55f380485e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 -- 172.21.15.62:0/2290511232 shutdown_connections 2023-07-14T16:19:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.152+0000 7fb908455b80 1 -- 172.21.15.62:0/2290511232 wait complete. 2023-07-14T16:19:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 1 -- start start 2023-07-14T16:19:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 1 Processor -- start 2023-07-14T16:19:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048ba90 0x55f380494ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048c7e0 0x55f38049bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38037ca60 con 0x55f3802e76b0 2023-07-14T16:19:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38037c760 con 0x55f38048c7e0 2023-07-14T16:19:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3802608c0 con 0x55f38048ba90 2023-07-14T16:19:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.153+0000 7fb908455b80 10 client.0 _ll_register_callbacks cb 0x55f38037a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8edd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048ba90 0x55f380494ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804949b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38136/0 (socket says 172.21.15.62:38136) 2023-07-14T16:19:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 -- 172.21.15.62:0/1864505415 learned_addr learned my addr 172.21.15.62:0/1864505415 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8eed9d700 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048c7e0 0x55f38049bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 -- 172.21.15.62:0/1864505415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048ba90 msgr2=0x55f380494ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048ba90 0x55f380494ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 -- 172.21.15.62:0/1864505415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048c7e0 msgr2=0x55f38049bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048c7e0 0x55f38049bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 -- 172.21.15.62:0/1864505415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f010a8a0 con 0x55f3802e76b0 2023-07-14T16:19:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804949b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00a1f40 tx=0x7fb8e0005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505656b1e7537e75 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.154+0000 7fb8d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1864505415 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0099df0 con 0x55f3802e76b0 2023-07-14T16:19:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1864505415 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8e000ccd0 con 0x55f3802e76b0 2023-07-14T16:19:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 1 -- 172.21.15.62:0/1864505415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f380336490 con 0x55f3802e76b0 2023-07-14T16:19:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1864505415 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8e00ceea0 con 0x55f3802e76b0 2023-07-14T16:19:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 10 client.16407 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 1 -- 172.21.15.62:0/1864505415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f3801f3c00 con 0x55f3802e76b0 2023-07-14T16:19:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1864505415 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f380336490 con 0x55f3802e76b0 2023-07-14T16:19:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1864505415 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb8e00add20 con 0x55f3802e76b0 2023-07-14T16:19:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 20 client.16407 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3804a0700 0x55f3804a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 4 client.16407 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.155+0000 7fb908455b80 1 -- 172.21.15.62:0/1864505415 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3803d5e50 con 0x55f3804a0700 2023-07-14T16:19:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.156+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3804a0700 0x55f3804a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.156+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3804a0700 0x55f3804a2b10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=873c2cf9626ac13c server_cookie=5f483cb681f45896 in_seq=0 out_seq=0 2023-07-14T16:19:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.156+0000 7fb8d5ffb700 10 client.16407 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.157+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1864505415 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f3803d5e50 con 0x55f3804a0700 2023-07-14T16:19:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.157+0000 7fb8d5ffb700 10 client.16407 handle_command_reply: tid=0 2023-07-14T16:19:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.157+0000 7fb908455b80 1 client.16407 shutdown 2023-07-14T16:19:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.157+0000 7fb908455b80 1 -- 172.21.15.62:0/1864505415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 msgr2=0x55f3804949b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.157+0000 7fb908455b80 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804949b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00a1f40 tx=0x7fb8e0005000 comp rx=0 tx=0).stop 2023-07-14T16:19:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 1 -- 172.21.15.62:0/1864505415 shutdown_connections 2023-07-14T16:19:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3804a0700 0x55f3804a2b10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048c7e0 0x55f38049bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804949b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb8d5ffb700 0 client.16407 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 1 --2- 172.21.15.62:0/1864505415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048ba90 0x55f380494ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 1 -- 172.21.15.62:0/1864505415 >> 172.21.15.62:0/1864505415 conn(0x55f380480d40 msgr2=0x55f380485640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 1 -- 172.21.15.62:0/1864505415 shutdown_connections 2023-07-14T16:19:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 1 -- 172.21.15.62:0/1864505415 wait complete. 2023-07-14T16:19:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.158+0000 7fb908455b80 20 client.16407 trim_cache size 0 max 16384 2023-07-14T16:19:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.174+0000 7fb908455b80 1 Processor -- start 2023-07-14T16:19:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb908455b80 1 -- start start 2023-07-14T16:19:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb908455b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb908455b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 0x55f38049bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb908455b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb908455b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3804a02a0 con 0x55f3802e76b0 2023-07-14T16:19:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb908455b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3804876a0 con 0x55f38048ba90 2023-07-14T16:19:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb908455b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38048c0d0 con 0x55f38048c7e0 2023-07-14T16:19:08.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb8ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb8ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38150/0 (socket says 172.21.15.62:38150) 2023-07-14T16:19:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb8ee59c700 1 -- 172.21.15.62:0/2221597163 learned_addr learned my addr 172.21.15.62:0/2221597163 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.175+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 0x55f38049bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8eed9d700 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8ee59c700 1 -- 172.21.15.62:0/2221597163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 msgr2=0x55f38048f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8ee59c700 1 -- 172.21.15.62:0/2221597163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 msgr2=0x55f38049bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 0x55f38049bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8ee59c700 1 -- 172.21.15.62:0/2221597163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3802311f0 con 0x55f3802e76b0 2023-07-14T16:19:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804a7490 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0099fb0 tx=0x7fb8e009e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ba1a0de322c51d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8dd7fa700 1 -- 172.21.15.62:0/2221597163 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0099df0 con 0x55f3802e76b0 2023-07-14T16:19:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8dd7fa700 1 -- 172.21.15.62:0/2221597163 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8e0099b60 con 0x55f3802e76b0 2023-07-14T16:19:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.176+0000 7fb8dd7fa700 1 -- 172.21.15.62:0/2221597163 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e009a020 con 0x55f3802e76b0 2023-07-14T16:19:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 -- 172.21.15.62:0/2221597163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 msgr2=0x55f3804a7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804a7490 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0099fb0 tx=0x7fb8e009e0d0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 -- 172.21.15.62:0/2221597163 shutdown_connections 2023-07-14T16:19:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38048f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3802e76b0 0x55f3804a7490 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 --2- 172.21.15.62:0/2221597163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f38048ba90 0x55f38049bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 -- 172.21.15.62:0/2221597163 >> 172.21.15.62:0/2221597163 conn(0x55f380480d40 msgr2=0x55f380493730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 -- 172.21.15.62:0/2221597163 shutdown_connections 2023-07-14T16:19:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.177+0000 7fb908455b80 1 -- 172.21.15.62:0/2221597163 wait complete. 2023-07-14T16:19:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.178+0000 7fb908455b80 1 -- start start 2023-07-14T16:19:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.178+0000 7fb908455b80 1 Processor -- start 2023-07-14T16:19:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.179+0000 7fb908455b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3802e76b0 0x55f380497d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.179+0000 7fb908455b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f38048ba90 0x55f3804982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.179+0000 7fb908455b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38049b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.179+0000 7fb908455b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38049b810 con 0x55f38048ba90 2023-07-14T16:19:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.179+0000 7fb908455b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38049b980 con 0x55f3802e76b0 2023-07-14T16:19:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.179+0000 7fb908455b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38049bb40 con 0x55f38048c7e0 2023-07-14T16:19:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.179+0000 7fb908455b80 10 client.0 _ll_register_callbacks cb 0x55f38037a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.179+0000 7fb8edd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f38048ba90 0x55f3804982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8edd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f38048ba90 0x55f3804982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38154/0 (socket says 172.21.15.62:38154) 2023-07-14T16:19:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8edd9b700 1 -- 172.21.15.62:0/1722511502 learned_addr learned my addr 172.21.15.62:0/1722511502 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8eed9d700 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38049b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3802e76b0 0x55f380497d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8edd9b700 1 -- 172.21.15.62:0/1722511502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 msgr2=0x55f38049b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38049b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8edd9b700 1 -- 172.21.15.62:0/1722511502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3802e76b0 msgr2=0x55f380497d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3802e76b0 0x55f380497d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8edd9b700 1 -- 172.21.15.62:0/1722511502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3801f3c00 con 0x55f38048ba90 2023-07-14T16:19:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8edd9b700 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f38048ba90 0x55f3804982b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009ce0 tx=0x7fb8e400b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c808ab1a422a359 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.180+0000 7fb8d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.181+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1722511502 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4030b20 con 0x55f38048ba90 2023-07-14T16:19:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.181+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1722511502 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8e4030c80 con 0x55f38048ba90 2023-07-14T16:19:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.181+0000 7fb908455b80 1 -- 172.21.15.62:0/1722511502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8e401e040 con 0x55f38048ba90 2023-07-14T16:19:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.181+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1722511502 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8e4030e40 con 0x55f38048ba90 2023-07-14T16:19:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.181+0000 7fb908455b80 10 client.16413 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.181+0000 7fb908455b80 1 -- 172.21.15.62:0/1722511502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f380209250 con 0x55f38048ba90 2023-07-14T16:19:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1722511502 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8e401e040 con 0x55f38048ba90 2023-07-14T16:19:08.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1722511502 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb8e400b710 con 0x55f38048ba90 2023-07-14T16:19:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb908455b80 10 client.16413 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb908455b80 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb908455b80 20 client.16413 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb908455b80 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3804f51a0 0x55f3804f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb908455b80 4 client.16413 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:19:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb908455b80 1 -- 172.21.15.62:0/1722511502 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f380397ff0 con 0x55f3804f51a0 2023-07-14T16:19:08.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.182+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3804f51a0 0x55f3804f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.183+0000 7fb8ee59c700 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3804f51a0 0x55f3804f75b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9864651f6e095c server_cookie=ece9d5d9a409e285 in_seq=0 out_seq=0 2023-07-14T16:19:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.183+0000 7fb8d5ffb700 10 client.16413 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb8d5ffb700 1 -- 172.21.15.62:0/1722511502 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f380397ff0 con 0x55f3804f51a0 2023-07-14T16:19:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb8d5ffb700 10 client.16413 handle_command_reply: tid=0 2023-07-14T16:19:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 client.16413 shutdown 2023-07-14T16:19:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 -- 172.21.15.62:0/1722511502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f38048ba90 msgr2=0x55f3804982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f38048ba90 0x55f3804982b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009ce0 tx=0x7fb8e400b380 comp rx=0 tx=0).stop 2023-07-14T16:19:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 -- 172.21.15.62:0/1722511502 shutdown_connections 2023-07-14T16:19:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f38048c7e0 0x55f38049b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3804f51a0 0x55f3804f75b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3802e76b0 0x55f380497d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 --2- 172.21.15.62:0/1722511502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f38048ba90 0x55f3804982b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb8d5ffb700 0 client.16413 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 -- 172.21.15.62:0/1722511502 >> 172.21.15.62:0/1722511502 conn(0x55f380480d40 msgr2=0x55f3804a7490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 -- 172.21.15.62:0/1722511502 shutdown_connections 2023-07-14T16:19:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.194+0000 7fb908455b80 1 -- 172.21.15.62:0/1722511502 wait complete. 2023-07-14T16:19:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb908455b80 20 client.16413 trim_cache size 0 max 16384 2023-07-14T16:19:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:19:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:19:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:19:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:19:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 -- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8d80a11d0 msgr2=0x7fb8d80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8d80a11d0 0x7fb8d80a3680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55f38048b2b0 tx=0x7fb8e800b040 comp rx=0 tx=0).stop 2023-07-14T16:19:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 -- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 msgr2=0x7fb8f01977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 0x7fb8f01977b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000e970 tx=0x7fb8e000ed30 comp rx=0 tx=0).stop 2023-07-14T16:19:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 -- 172.21.15.62:0/2105038492 shutdown_connections 2023-07-14T16:19:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8d80a11d0 0x7fb8d80a3680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8f0107800 0x7fb8f019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8f0072f70 0x7fb8f01977b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 --2- 172.21.15.62:0/2105038492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f0074550 0x7fb8f0197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.195+0000 7fb8ef59e700 1 -- 172.21.15.62:0/2105038492 >> 172.21.15.62:0/2105038492 conn(0x7fb8f00faf60 msgr2=0x7fb8f00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.196+0000 7fb8ef59e700 1 -- 172.21.15.62:0/2105038492 shutdown_connections 2023-07-14T16:19:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:08.196+0000 7fb8ef59e700 1 -- 172.21.15.62:0/2105038492 wait complete. 2023-07-14T16:19:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:08 smithi164 ceph-mon[97427]: pgmap v462: 97 pgs: 97 active+clean; 1.4 GiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:19:09.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:08 smithi062 ceph-mon[95543]: pgmap v462: 97 pgs: 97 active+clean; 1.4 GiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:19:09.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:08 smithi081 ceph-mon[127795]: pgmap v462: 97 pgs: 97 active+clean; 1.4 GiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:19:09.870 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:19:10.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.117+0000 7fd5f87ff700 1 -- 172.21.15.62:0/2251533878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0101670 msgr2=0x7fd5f0107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.117+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/2251533878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0101670 0x7fd5f0107f30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800b0a0 tx=0x7fd5e800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.117+0000 7fd5f87ff700 1 -- 172.21.15.62:0/2251533878 shutdown_connections 2023-07-14T16:19:10.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.117+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/2251533878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0101670 0x7fd5f0107f30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.117+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/2251533878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5f00ffaf0 0x7fd5f00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.117+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/2251533878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5f0100ce0 0x7fd5f0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.117+0000 7fd5f87ff700 1 -- 172.21.15.62:0/2251533878 >> 172.21.15.62:0/2251533878 conn(0x7fd5f00fb130 msgr2=0x7fd5f00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.118+0000 7fd5f87ff700 1 -- 172.21.15.62:0/2251533878 shutdown_connections 2023-07-14T16:19:10.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.118+0000 7fd5f87ff700 1 -- 172.21.15.62:0/2251533878 wait complete. 2023-07-14T16:19:10.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.119+0000 7fd5f87ff700 1 Processor -- start 2023-07-14T16:19:10.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.119+0000 7fd5f87ff700 1 -- start start 2023-07-14T16:19:10.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f87ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5f00ffaf0 0x7fd5f01958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f87ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0100ce0 0x7fd5f0195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f87ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5f0101670 0x7fd5f019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f87ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f019d430 con 0x7fd5f0100ce0 2023-07-14T16:19:10.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f87ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f019d570 con 0x7fd5f00ffaf0 2023-07-14T16:19:10.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f87ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f019d700 con 0x7fd5f0101670 2023-07-14T16:19:10.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f5d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0100ce0 0x7fd5f0195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f5d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0100ce0 0x7fd5f0195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38174/0 (socket says 172.21.15.62:38174) 2023-07-14T16:19:10.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f5d9a700 1 -- 172.21.15.62:0/16243766 learned_addr learned my addr 172.21.15.62:0/16243766 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:10.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.120+0000 7fd5f659b700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5f00ffaf0 0x7fd5f01958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5f0101670 0x7fd5f019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f5d9a700 1 -- 172.21.15.62:0/16243766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5f0101670 msgr2=0x7fd5f019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5f0101670 0x7fd5f019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f5d9a700 1 -- 172.21.15.62:0/16243766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5f00ffaf0 msgr2=0x7fd5f01958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5f00ffaf0 0x7fd5f01958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f5d9a700 1 -- 172.21.15.62:0/16243766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8009d00 con 0x7fd5f0100ce0 2023-07-14T16:19:10.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5f0101670 0x7fd5f019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:19:10.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0100ce0 0x7fd5f0195df0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9a0 tx=0x7fd5e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5e77fe700 1 -- 172.21.15.62:0/16243766 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0009790 con 0x7fd5f0100ce0 2023-07-14T16:19:10.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5e77fe700 1 -- 172.21.15.62:0/16243766 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5e000cbd0 con 0x7fd5f0100ce0 2023-07-14T16:19:10.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5e77fe700 1 -- 172.21.15.62:0/16243766 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0018680 con 0x7fd5f0100ce0 2023-07-14T16:19:10.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f87ff700 1 -- 172.21.15.62:0/16243766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f019d9e0 con 0x7fd5f0100ce0 2023-07-14T16:19:10.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.121+0000 7fd5f87ff700 1 -- 172.21.15.62:0/16243766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f019dea0 con 0x7fd5f0100ce0 2023-07-14T16:19:10.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.125+0000 7fd5e77fe700 1 -- 172.21.15.62:0/16243766 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd5e000cd40 con 0x7fd5f0100ce0 2023-07-14T16:19:10.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.126+0000 7fd60c0afb80 1 Processor -- start 2023-07-14T16:19:10.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.126+0000 7fd60c0afb80 1 -- start start 2023-07-14T16:19:10.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432c94cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 0x55a432c95af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 0x55a432c964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432b95cb0 con 0x55a432c948e0 2023-07-14T16:19:10.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432be8a40 con 0x55a432c96080 2023-07-14T16:19:10.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432be1ed0 con 0x55a432c95230 2023-07-14T16:19:10.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd5e77fe700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5dc0a11d0 0x7fd5dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd5e77fe700 1 -- 172.21.15.62:0/16243766 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5e0014070 con 0x7fd5f0100ce0 2023-07-14T16:19:10.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd5f6d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432c94cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd5f6d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432c94cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38180/0 (socket says 172.21.15.62:38180) 2023-07-14T16:19:10.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd5f6d9c700 1 -- 172.21.15.62:0/305692569 learned_addr learned my addr 172.21.15.62:0/305692569 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:10.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5dc0a11d0 0x7fd5dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.127+0000 7fd5f659b700 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 0x55a432c95af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.128+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 0x55a432c964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.128+0000 7fd5f6d9c700 1 -- 172.21.15.62:0/305692569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 msgr2=0x55a432c95af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.128+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 0x55a432c95af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.128+0000 7fd5f6d9c700 1 -- 172.21.15.62:0/305692569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 msgr2=0x55a432c964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.128+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 0x55a432c964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.128+0000 7fd5f6d9c700 1 -- 172.21.15.62:0/305692569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a432a0fde0 con 0x55a432c948e0 2023-07-14T16:19:10.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.128+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432c94cf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8008d00 tx=0x7fd5e8008d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7b287fd7558f9e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.128+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5dc0a11d0 0x7fd5dc0a3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800b070 tx=0x7fd5e8013000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd5e4ff9700 1 -- 172.21.15.62:0/305692569 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e802c070 con 0x55a432c948e0 2023-07-14T16:19:10.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd5e4ff9700 1 -- 172.21.15.62:0/305692569 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5e8007720 con 0x55a432c948e0 2023-07-14T16:19:10.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd5e4ff9700 1 -- 172.21.15.62:0/305692569 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800e040 con 0x55a432c948e0 2023-07-14T16:19:10.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd60c0afb80 1 -- 172.21.15.62:0/305692569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 msgr2=0x55a432c94cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432c94cf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8008d00 tx=0x7fd5e8008d30 comp rx=0 tx=0).stop 2023-07-14T16:19:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd60c0afb80 1 -- 172.21.15.62:0/305692569 shutdown_connections 2023-07-14T16:19:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 0x55a432c964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432c94cf0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/305692569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 0x55a432c95af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.129+0000 7fd60c0afb80 1 -- 172.21.15.62:0/305692569 >> 172.21.15.62:0/305692569 conn(0x55a432c85da0 msgr2=0x55a432c8b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.130+0000 7fd60c0afb80 1 -- 172.21.15.62:0/305692569 shutdown_connections 2023-07-14T16:19:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.130+0000 7fd60c0afb80 1 -- 172.21.15.62:0/305692569 wait complete. 2023-07-14T16:19:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.130+0000 7fd60c0afb80 1 -- start start 2023-07-14T16:19:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.130+0000 7fd60c0afb80 1 Processor -- start 2023-07-14T16:19:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c948e0 0x55a432c9e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c95230 0x55a432c9ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c96080 0x55a432ca5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432b81ac0 con 0x55a432c95230 2023-07-14T16:19:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432b817c0 con 0x55a432c948e0 2023-07-14T16:19:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432a65920 con 0x55a432c96080 2023-07-14T16:19:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd60c0afb80 10 client.0 _ll_register_callbacks cb 0x55a432b7f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd5f5d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c95230 0x55a432c9ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd5f5d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c95230 0x55a432c9ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38196/0 (socket says 172.21.15.62:38196) 2023-07-14T16:19:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd5f5d9a700 1 -- 172.21.15.62:0/3871129567 learned_addr learned my addr 172.21.15.62:0/3871129567 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd5f659b700 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c948e0 0x55a432c9e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.131+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c96080 0x55a432ca5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5f5d9a700 1 -- 172.21.15.62:0/3871129567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c96080 msgr2=0x55a432ca5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c96080 0x55a432ca5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5f5d9a700 1 -- 172.21.15.62:0/3871129567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c948e0 msgr2=0x55a432c9e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c948e0 0x55a432c9e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5f5d9a700 1 -- 172.21.15.62:0/3871129567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f0108860 con 0x55a432c95230 2023-07-14T16:19:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c96080 0x55a432ca5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c95230 0x55a432c9ead0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e970 tx=0x7fd5e0019a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633df2fee342271b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3871129567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0004dc0 con 0x55a432c95230 2023-07-14T16:19:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3871129567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5e0004b90 con 0x55a432c95230 2023-07-14T16:19:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.132+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3871129567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a432b3b4f0 con 0x55a432c95230 2023-07-14T16:19:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3871129567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e0004750 con 0x55a432c95230 2023-07-14T16:19:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd60c0afb80 10 client.16425 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3871129567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a4329f8c60 con 0x55a432c95230 2023-07-14T16:19:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3871129567 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a432b3b4f0 con 0x55a432c95230 2023-07-14T16:19:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3871129567 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd5e0099430 con 0x55a432c95230 2023-07-14T16:19:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd60c0afb80 10 client.16425 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd60c0afb80 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd60c0afb80 20 client.16425 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a432caa3c0 0x55a432cac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd60c0afb80 4 client.16425 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.133+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3871129567 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a432bdaeb0 con 0x55a432caa3c0 2023-07-14T16:19:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.134+0000 7fd5f659b700 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a432caa3c0 0x55a432cac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.134+0000 7fd5f659b700 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a432caa3c0 0x55a432cac7d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a0c60a31d25d63 server_cookie=a35474a66e1d5eb5 in_seq=0 out_seq=0 2023-07-14T16:19:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.134+0000 7fd5d9ffb700 10 client.16425 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.135+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3871129567 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a432bdaeb0 con 0x55a432caa3c0 2023-07-14T16:19:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.135+0000 7fd5d9ffb700 10 client.16425 handle_command_reply: tid=0 2023-07-14T16:19:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.135+0000 7fd60c0afb80 1 client.16425 shutdown 2023-07-14T16:19:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.135+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3871129567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c95230 msgr2=0x55a432c9ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.135+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c95230 0x55a432c9ead0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e970 tx=0x7fd5e0019a50 comp rx=0 tx=0).stop 2023-07-14T16:19:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.135+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3871129567 shutdown_connections 2023-07-14T16:19:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c96080 0x55a432ca5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a432caa3c0 0x55a432cac7d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c948e0 0x55a432c9e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3871129567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c95230 0x55a432c9ead0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3871129567 >> 172.21.15.62:0/3871129567 conn(0x55a432c85da0 msgr2=0x55a432c8a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd5d9ffb700 0 client.16425 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3871129567 shutdown_connections 2023-07-14T16:19:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3871129567 wait complete. 2023-07-14T16:19:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.136+0000 7fd60c0afb80 20 client.16425 trim_cache size 0 max 16384 2023-07-14T16:19:10.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.157+0000 7fd60c0afb80 1 Processor -- start 2023-07-14T16:19:10.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd60c0afb80 1 -- start start 2023-07-14T16:19:10.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c948e0 0x55a432c9d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c95230 0x55a432ca5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c96080 0x55a432ca6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432c8cd00 con 0x55a432c96080 2023-07-14T16:19:10.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432c95750 con 0x55a432c95230 2023-07-14T16:19:10.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432c9f210 con 0x55a432c948e0 2023-07-14T16:19:10.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd5f6d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c96080 0x55a432ca6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd5f6d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c96080 0x55a432ca6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38200/0 (socket says 172.21.15.62:38200) 2023-07-14T16:19:10.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd5f6d9c700 1 -- 172.21.15.62:0/1183008028 learned_addr learned my addr 172.21.15.62:0/1183008028 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:10.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.158+0000 7fd5f659b700 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c948e0 0x55a432c9d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c95230 0x55a432ca5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5f6d9c700 1 -- 172.21.15.62:0/1183008028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c948e0 msgr2=0x55a432c9d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c948e0 0x55a432c9d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5f6d9c700 1 -- 172.21.15.62:0/1183008028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c95230 msgr2=0x55a432ca5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c95230 0x55a432ca5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5f6d9c700 1 -- 172.21.15.62:0/1183008028 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a432a36250 con 0x55a432c96080 2023-07-14T16:19:10.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c95230 0x55a432ca5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:10.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5f6d9c700 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c96080 0x55a432ca6340 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800b7b0 tx=0x7fd5e80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c570ac41bacbd8f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5e57fa700 1 -- 172.21.15.62:0/1183008028 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e802c030 con 0x55a432c96080 2023-07-14T16:19:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5e57fa700 1 -- 172.21.15.62:0/1183008028 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5e8004a50 con 0x55a432c96080 2023-07-14T16:19:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.159+0000 7fd5e57fa700 1 -- 172.21.15.62:0/1183008028 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800e040 con 0x55a432c96080 2023-07-14T16:19:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 -- 172.21.15.62:0/1183008028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c96080 msgr2=0x55a432ca6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c96080 0x55a432ca6340 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800b7b0 tx=0x7fd5e80099e0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 -- 172.21.15.62:0/1183008028 shutdown_connections 2023-07-14T16:19:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c96080 0x55a432ca6340 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c948e0 0x55a432c9d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/1183008028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c95230 0x55a432ca5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 -- 172.21.15.62:0/1183008028 >> 172.21.15.62:0/1183008028 conn(0x55a432c85da0 msgr2=0x55a432cb1130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 -- 172.21.15.62:0/1183008028 shutdown_connections 2023-07-14T16:19:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.160+0000 7fd60c0afb80 1 -- 172.21.15.62:0/1183008028 wait complete. 2023-07-14T16:19:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.161+0000 7fd60c0afb80 1 -- start start 2023-07-14T16:19:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.161+0000 7fd60c0afb80 1 Processor -- start 2023-07-14T16:19:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.161+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432ca1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.161+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 0x55a432ca20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.161+0000 7fd60c0afb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 0x55a432ca5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432ca5650 con 0x55a432c948e0 2023-07-14T16:19:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432ca57c0 con 0x55a432c96080 2023-07-14T16:19:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd60c0afb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a432ca5980 con 0x55a432c95230 2023-07-14T16:19:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd60c0afb80 10 client.0 _ll_register_callbacks cb 0x7fd5e0099de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432ca1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432ca1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38204/0 (socket says 172.21.15.62:38204) 2023-07-14T16:19:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f659b700 1 -- 172.21.15.62:0/3436398331 learned_addr learned my addr 172.21.15.62:0/3436398331 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 0x55a432ca20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f659b700 1 -- 172.21.15.62:0/3436398331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 msgr2=0x55a432ca20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f659b700 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 0x55a432ca20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f659b700 1 -- 172.21.15.62:0/3436398331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 msgr2=0x55a432ca5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f659b700 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 0x55a432ca5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.162+0000 7fd5f659b700 1 -- 172.21.15.62:0/3436398331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a432a0fde0 con 0x55a432c948e0 2023-07-14T16:19:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd5f659b700 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432ca1bb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec004790 tx=0x7fd5ec0107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8344ff0042c95c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd5d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3436398331 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec017070 con 0x55a432c948e0 2023-07-14T16:19:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3436398331 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5ec01ebd0 con 0x55a432c948e0 2023-07-14T16:19:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3436398331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5e8007720 con 0x55a432c948e0 2023-07-14T16:19:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3436398331 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5ec01ed90 con 0x55a432c948e0 2023-07-14T16:19:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd60c0afb80 10 client.16431 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3436398331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a432a0e2b0 con 0x55a432c948e0 2023-07-14T16:19:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.163+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3436398331 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5e8007720 con 0x55a432c948e0 2023-07-14T16:19:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.164+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3436398331 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd5ec02f070 con 0x55a432c948e0 2023-07-14T16:19:10.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.164+0000 7fd60c0afb80 10 client.16431 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:10.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.164+0000 7fd60c0afb80 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:10.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.164+0000 7fd60c0afb80 20 client.16431 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:10.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.164+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a432cff620 0x55a432d01a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.164+0000 7fd60c0afb80 4 client.16431 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:19:10.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.164+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3436398331 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a432b9d050 con 0x55a432cff620 2023-07-14T16:19:10.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.164+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a432cff620 0x55a432d01a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd5d9ffb700 10 client.16431 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:10.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd5f5d9a700 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a432cff620 0x55a432d01a30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3b9292313cd69a server_cookie=91e8d2e2d1397fd4 in_seq=0 out_seq=0 2023-07-14T16:19:10.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd5d9ffb700 1 -- 172.21.15.62:0/3436398331 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a432b9d050 con 0x55a432cff620 2023-07-14T16:19:10.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd5d9ffb700 10 client.16431 handle_command_reply: tid=0 2023-07-14T16:19:10.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 client.16431 shutdown 2023-07-14T16:19:10.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3436398331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 msgr2=0x55a432ca1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432ca1bb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec004790 tx=0x7fd5ec0107c0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3436398331 shutdown_connections 2023-07-14T16:19:10.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a432cff620 0x55a432d01a30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a432c96080 0x55a432ca5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a432c948e0 0x55a432ca1bb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 --2- 172.21.15.62:0/3436398331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a432c95230 0x55a432ca20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3436398331 >> 172.21.15.62:0/3436398331 conn(0x55a432c85da0 msgr2=0x55a432c9d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.165+0000 7fd5d9ffb700 0 client.16431 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.166+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3436398331 shutdown_connections 2023-07-14T16:19:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.166+0000 7fd60c0afb80 1 -- 172.21.15.62:0/3436398331 wait complete. 2023-07-14T16:19:10.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.166+0000 7fd60c0afb80 20 client.16431 trim_cache size 0 max 16384 2023-07-14T16:19:10.151 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:19:10.151 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:19:10.151 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:19:10.151 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:19:10.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.166+0000 7fd5f87ff700 1 -- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5dc0a11d0 msgr2=0x7fd5dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.166+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5dc0a11d0 0x7fd5dc0a3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800b070 tx=0x7fd5e8013000 comp rx=0 tx=0).stop 2023-07-14T16:19:10.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.166+0000 7fd5f87ff700 1 -- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0100ce0 msgr2=0x7fd5f0195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.166+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0100ce0 0x7fd5f0195df0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9a0 tx=0x7fd5e000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.167+0000 7fd5f87ff700 1 -- 172.21.15.62:0/16243766 shutdown_connections 2023-07-14T16:19:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.167+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5f0101670 0x7fd5f019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.167+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5dc0a11d0 0x7fd5dc0a3680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.167+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5f00ffaf0 0x7fd5f01958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.167+0000 7fd5f87ff700 1 --2- 172.21.15.62:0/16243766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5f0100ce0 0x7fd5f0195df0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.167+0000 7fd5f87ff700 1 -- 172.21.15.62:0/16243766 >> 172.21.15.62:0/16243766 conn(0x7fd5f00fb130 msgr2=0x7fd5f00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.167+0000 7fd5f87ff700 1 -- 172.21.15.62:0/16243766 shutdown_connections 2023-07-14T16:19:10.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.167+0000 7fd5f87ff700 1 -- 172.21.15.62:0/16243766 wait complete. 2023-07-14T16:19:10.164 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:19:10.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:19:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 -- 172.21.15.62:0/2317435661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc102e70 msgr2=0x7f1adc105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/2317435661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc102e70 0x7f1adc105250 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009990 tx=0x7f1ad0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 -- 172.21.15.62:0/2317435661 shutdown_connections 2023-07-14T16:19:10.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/2317435661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1adc105790 0x7f1adc107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/2317435661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1adc100550 0x7f1adc102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/2317435661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc102e70 0x7f1adc105250 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 -- 172.21.15.62:0/2317435661 >> 172.21.15.62:0/2317435661 conn(0x7f1adc0f9fd0 msgr2=0x7f1adc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 -- 172.21.15.62:0/2317435661 shutdown_connections 2023-07-14T16:19:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.423+0000 7f1ae1bff700 1 -- 172.21.15.62:0/2317435661 wait complete. 2023-07-14T16:19:10.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.424+0000 7f1ae1bff700 1 Processor -- start 2023-07-14T16:19:10.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.424+0000 7f1ae1bff700 1 -- start start 2023-07-14T16:19:10.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.424+0000 7f1ae1bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1adc100550 0x7f1adc195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1ae1bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1adc102e70 0x7f1adc195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1ae1bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc105790 0x7f1adc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1ae1bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc19d2d0 con 0x7f1adc105790 2023-07-14T16:19:10.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1ae1bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc19d410 con 0x7f1adc102e70 2023-07-14T16:19:10.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1ae1bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc19d5a0 con 0x7f1adc100550 2023-07-14T16:19:10.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc105790 0x7f1adc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1adc102e70 0x7f1adc195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc105790 0x7f1adc19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38226/0 (socket says 172.21.15.62:38226) 2023-07-14T16:19:10.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adbfff700 1 -- 172.21.15.62:0/1637526864 learned_addr learned my addr 172.21.15.62:0/1637526864 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:10.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adb7fe700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1adc100550 0x7f1adc195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adaffd700 1 -- 172.21.15.62:0/1637526864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1adc100550 msgr2=0x7f1adc195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adaffd700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1adc100550 0x7f1adc195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adaffd700 1 -- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc105790 msgr2=0x7f1adc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adaffd700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc105790 0x7f1adc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adaffd700 1 -- 172.21.15.62:0/1637526864 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00095f0 con 0x7f1adc102e70 2023-07-14T16:19:10.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.425+0000 7f1adbfff700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc105790 0x7f1adc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:10.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.426+0000 7f1adaffd700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1adc102e70 0x7f1adc195c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0000c00 tx=0x7f1ad0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.426+0000 7f1ad8ff9700 1 -- 172.21.15.62:0/1637526864 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001d070 con 0x7f1adc102e70 2023-07-14T16:19:10.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.426+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc19d820 con 0x7f1adc102e70 2023-07-14T16:19:10.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.426+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc19dca0 con 0x7f1adc102e70 2023-07-14T16:19:10.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.426+0000 7f1ad8ff9700 1 -- 172.21.15.62:0/1637526864 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ad000bdb0 con 0x7f1adc102e70 2023-07-14T16:19:10.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.426+0000 7f1ad8ff9700 1 -- 172.21.15.62:0/1637526864 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000fc10 con 0x7f1adc102e70 2023-07-14T16:19:10.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.428+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8005320 con 0x7f1adc102e70 2023-07-14T16:19:10.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.431+0000 7f1ad8ff9700 1 -- 172.21.15.62:0/1637526864 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1ad000b920 con 0x7f1adc102e70 2023-07-14T16:19:10.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.432+0000 7f1ad8ff9700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac40a1220 0x7f1ac40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.433+0000 7f1ad8ff9700 1 -- 172.21.15.62:0/1637526864 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ad00cdce0 con 0x7f1adc102e70 2023-07-14T16:19:10.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.433+0000 7f1adb7fe700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac40a1220 0x7f1ac40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.433+0000 7f1adb7fe700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac40a1220 0x7f1ac40a36d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009c60 tx=0x7f1acc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.436+0000 7f1ad8ff9700 1 -- 172.21.15.62:0/1637526864 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1ad0097470 con 0x7f1adc102e70 2023-07-14T16:19:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.744+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ac8005f70 con 0x7f1adc102e70 2023-07-14T16:19:10.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.745+0000 7f1ad8ff9700 1 -- 172.21.15.62:0/1637526864 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10826 (secure 0 0 0) 0x7f1ad009aa90 con 0x7f1adc102e70 2023-07-14T16:19:10.723 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:19:10.724 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:17:18.727946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:19:10.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.748+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac40a1220 msgr2=0x7f1ac40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.748+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac40a1220 0x7f1ac40a36d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009c60 tx=0x7f1acc009260 comp rx=0 tx=0).stop 2023-07-14T16:19:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.748+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1adc102e70 msgr2=0x7f1adc195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.748+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1adc102e70 0x7f1adc195c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0000c00 tx=0x7f1ad0004730 comp rx=0 tx=0).stop 2023-07-14T16:19:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.749+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 shutdown_connections 2023-07-14T16:19:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.749+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1adc105790 0x7f1adc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.749+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1adc100550 0x7f1adc195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.749+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac40a1220 0x7f1ac40a36d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.749+0000 7f1ae1bff700 1 --2- 172.21.15.62:0/1637526864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1adc102e70 0x7f1adc195c90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.749+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 >> 172.21.15.62:0/1637526864 conn(0x7f1adc0f9fd0 msgr2=0x7f1adc1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.749+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 shutdown_connections 2023-07-14T16:19:10.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:10.749+0000 7f1ae1bff700 1 -- 172.21.15.62:0/1637526864 wait complete. 2023-07-14T16:19:10.728 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2023-07-14T16:19:10.748 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:19:10.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 -- 172.21.15.62:0/4081947704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40ff5c0 msgr2=0x7f1ed40ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 --2- 172.21.15.62:0/4081947704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40ff5c0 0x7f1ed40ffa30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00b0a0 tx=0x7f1ecc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 -- 172.21.15.62:0/4081947704 shutdown_connections 2023-07-14T16:19:10.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 --2- 172.21.15.62:0/4081947704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40ff5c0 0x7f1ed40ffa30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 --2- 172.21.15.62:0/4081947704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed40fe2c0 0x7f1ed40fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 --2- 172.21.15.62:0/4081947704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ed40fec10 0x7f1ed40ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 -- 172.21.15.62:0/4081947704 >> 172.21.15.62:0/4081947704 conn(0x7f1ed40f9ed0 msgr2=0x7f1ed40fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:10.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 -- 172.21.15.62:0/4081947704 shutdown_connections 2023-07-14T16:19:10.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.005+0000 7f1eda79d700 1 -- 172.21.15.62:0/4081947704 wait complete. 2023-07-14T16:19:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.006+0000 7f1eda79d700 1 Processor -- start 2023-07-14T16:19:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.006+0000 7f1eda79d700 1 -- start start 2023-07-14T16:19:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1eda79d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed40fe2c0 0x7f1ed4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1eda79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40fec10 0x7f1ed4197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1eda79d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ed40ff5c0 0x7f1ed419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1eda79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed419f2e0 con 0x7f1ed40fec10 2023-07-14T16:19:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1eda79d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed419f420 con 0x7f1ed40ff5c0 2023-07-14T16:19:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1eda79d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed419f5b0 con 0x7f1ed40fe2c0 2023-07-14T16:19:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1ed8f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40fec10 0x7f1ed4197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1ed8f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40fec10 0x7f1ed4197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38256/0 (socket says 172.21.15.62:38256) 2023-07-14T16:19:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1ed8f9a700 1 -- 172.21.15.62:0/1712360858 learned_addr learned my addr 172.21.15.62:0/1712360858 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.007+0000 7f1ed979b700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed40fe2c0 0x7f1ed4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1ed8f9a700 1 -- 172.21.15.62:0/1712360858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed40fe2c0 msgr2=0x7f1ed4197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed40fe2c0 0x7f1ed4197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1ed8f9a700 1 -- 172.21.15.62:0/1712360858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ed40ff5c0 msgr2=0x7f1ed419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ed40ff5c0 0x7f1ed419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1ed8f9a700 1 -- 172.21.15.62:0/1712360858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc009d00 con 0x7f1ed40fec10 2023-07-14T16:19:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40fec10 0x7f1ed4197ca0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400d950 tx=0x7f1ec400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1eca7fc700 1 -- 172.21.15.62:0/1712360858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4009b40 con 0x7f1ed40fec10 2023-07-14T16:19:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1eda79d700 1 -- 172.21.15.62:0/1712360858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed419f890 con 0x7f1ed40fec10 2023-07-14T16:19:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1eca7fc700 1 -- 172.21.15.62:0/1712360858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ec4009ca0 con 0x7f1ed40fec10 2023-07-14T16:19:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1eca7fc700 1 -- 172.21.15.62:0/1712360858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec400f620 con 0x7f1ed40fec10 2023-07-14T16:19:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.008+0000 7f1eda79d700 1 -- 172.21.15.62:0/1712360858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed419fdf0 con 0x7f1ed40fec10 2023-07-14T16:19:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.012+0000 7f1eca7fc700 1 -- 172.21.15.62:0/1712360858 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1ec4010460 con 0x7f1ed40fec10 2023-07-14T16:19:10.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.013+0000 7f1eca7fc700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ec00a11f0 0x7f1ec00a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.013+0000 7f1eca7fc700 1 -- 172.21.15.62:0/1712360858 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ec40ccc60 con 0x7f1ed40fec10 2023-07-14T16:19:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.013+0000 7f1eef3bbb80 1 Processor -- start 2023-07-14T16:19:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1ed979b700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ec00a11f0 0x7f1ec00a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1eef3bbb80 1 -- start start 2023-07-14T16:19:10.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e7293b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:10.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e7198cb0 con 0x5616e7294e30 2023-07-14T16:19:10.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e71eba40 con 0x5616e7293720 2023-07-14T16:19:10.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e71e4ed0 con 0x5616e7294070 2023-07-14T16:19:10.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.014+0000 7f1ed979b700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ec00a11f0 0x7f1ec00a36a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000bbd0 tx=0x7f1ed000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed979b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e7293b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38268/0 (socket says 172.21.15.62:38268) 2023-07-14T16:19:10.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 -- 172.21.15.62:0/3147841101 learned_addr learned my addr 172.21.15.62:0/3147841101 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:10.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:10.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 -- 172.21.15.62:0/3147841101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 msgr2=0x5616e72948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 -- 172.21.15.62:0/3147841101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 msgr2=0x5616e7293b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:10.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e7293b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:10.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 -- 172.21.15.62:0/3147841101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e7012de0 con 0x5616e7294e30 2023-07-14T16:19:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed979b700 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e7293b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.015+0000 7f1ed9f9c700 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72952a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc007a30 tx=0x7f1ecc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2f956fff39e0f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:10.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.016+0000 7f1ebf7fe700 1 -- 172.21.15.62:0/3147841101 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00e040 con 0x5616e7294e30 2023-07-14T16:19:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.016+0000 7f1ebf7fe700 1 -- 172.21.15.62:0/3147841101 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ecc00cea0 con 0x5616e7294e30 2023-07-14T16:19:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.016+0000 7f1ebf7fe700 1 -- 172.21.15.62:0/3147841101 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc01d480 con 0x5616e7294e30 2023-07-14T16:19:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.016+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/3147841101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 msgr2=0x5616e72952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.016+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72952a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc007a30 tx=0x7f1ecc009c20 comp rx=0 tx=0).stop 2023-07-14T16:19:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.017+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/3147841101 shutdown_connections 2023-07-14T16:19:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.017+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72952a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.017+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e7293b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.017+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/3147841101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.017+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/3147841101 >> 172.21.15.62:0/3147841101 conn(0x5616e7288ff0 msgr2=0x5616e728e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.017+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/3147841101 shutdown_connections 2023-07-14T16:19:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.017+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/3147841101 wait complete. 2023-07-14T16:19:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.017+0000 7f1eef3bbb80 1 -- start start 2023-07-14T16:19:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1eef3bbb80 1 Processor -- start 2023-07-14T16:19:11.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e729d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294e30 0x5616e72a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e7184ac0 con 0x5616e7293720 2023-07-14T16:19:11.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e71847c0 con 0x5616e7294e30 2023-07-14T16:19:11.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e7068920 con 0x5616e7294070 2023-07-14T16:19:11.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1eef3bbb80 10 client.0 _ll_register_callbacks cb 0x5616e7182760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:11.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1ed979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1ed979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38272/0 (socket says 172.21.15.62:38272) 2023-07-14T16:19:11.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1ed979b700 1 -- 172.21.15.62:0/1683240443 learned_addr learned my addr 172.21.15.62:0/1683240443 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.018+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e729d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ed9f9c700 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294e30 0x5616e72a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ed979b700 1 -- 172.21.15.62:0/1683240443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 msgr2=0x5616e729d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ed979b700 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e729d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ed979b700 1 -- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294e30 msgr2=0x5616e72a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ed979b700 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294e30 0x5616e72a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ed979b700 1 -- 172.21.15.62:0/1683240443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed000f960 con 0x5616e7293720 2023-07-14T16:19:11.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ed9f9c700 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294e30 0x5616e72a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:11.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ed979b700 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729d280 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00116f0 tx=0x7f1ed00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3573bb5b2c54b514 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ebcff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:11.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.019+0000 7f1ebcff9700 1 -- 172.21.15.62:0/1683240443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed001f840 con 0x5616e7293720 2023-07-14T16:19:11.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/1683240443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616e713e4f0 con 0x5616e7293720 2023-07-14T16:19:11.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1ebcff9700 1 -- 172.21.15.62:0/1683240443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ed001f9a0 con 0x5616e7293720 2023-07-14T16:19:11.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1ebcff9700 1 -- 172.21.15.62:0/1683240443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ed000feb0 con 0x5616e7293720 2023-07-14T16:19:11.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 10 client.16449 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:11.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/1683240443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f1ed410a910 con 0x5616e7293720 2023-07-14T16:19:11.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1ebcff9700 1 -- 172.21.15.62:0/1683240443 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5616e713e4f0 con 0x5616e7293720 2023-07-14T16:19:11.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1ebcff9700 1 -- 172.21.15.62:0/1683240443 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1ed0031070 con 0x5616e7293720 2023-07-14T16:19:11.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 10 client.16449 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:11.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 10 client.16449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:19:11.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 20 client.16449 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:11.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5616e72a90e0 0x5616e72ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 4 client.16449 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:11.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.020+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/1683240443 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616e71ddeb0 con 0x5616e72a90e0 2023-07-14T16:19:11.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.021+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5616e72a90e0 0x5616e72ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.021+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5616e72a90e0 0x5616e72ab4f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abc393143ace7d28 server_cookie=f6dd6b01a9c6c3f0 in_seq=0 out_seq=0 2023-07-14T16:19:11.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.021+0000 7f1ebcff9700 10 client.16449 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:19:11.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.022+0000 7f1ebcff9700 1 -- 172.21.15.62:0/1683240443 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5616e71ddeb0 con 0x5616e72a90e0 2023-07-14T16:19:11.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.022+0000 7f1ebcff9700 10 client.16449 handle_command_reply: tid=0 2023-07-14T16:19:11.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.022+0000 7f1eef3bbb80 1 client.16449 shutdown 2023-07-14T16:19:11.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/1683240443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 msgr2=0x5616e729d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729d280 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00116f0 tx=0x7f1ed00047c0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/1683240443 shutdown_connections 2023-07-14T16:19:11.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5616e72a90e0 0x5616e72ab4f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294e30 0x5616e72a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729d280 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/1683240443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e729d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/1683240443 >> 172.21.15.62:0/1683240443 conn(0x5616e7288ff0 msgr2=0x5616e728db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1ebcff9700 0 client.16449 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:19:11.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/1683240443 shutdown_connections 2023-07-14T16:19:11.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/1683240443 wait complete. 2023-07-14T16:19:11.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.023+0000 7f1eef3bbb80 20 client.16449 trim_cache size 0 max 16384 2023-07-14T16:19:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1eef3bbb80 1 Processor -- start 2023-07-14T16:19:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1eef3bbb80 1 -- start start 2023-07-14T16:19:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294070 0x5616e72a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294e30 0x5616e728db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e728f260 con 0x5616e7293720 2023-07-14T16:19:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e729e270 con 0x5616e7294070 2023-07-14T16:19:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e729af40 con 0x5616e7294e30 2023-07-14T16:19:11.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1ed979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1ed979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38274/0 (socket says 172.21.15.62:38274) 2023-07-14T16:19:11.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.039+0000 7f1ed979b700 1 -- 172.21.15.62:0/4269573810 learned_addr learned my addr 172.21.15.62:0/4269573810 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ed979b700 1 -- 172.21.15.62:0/4269573810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294e30 msgr2=0x5616e728db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ed979b700 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294e30 0x5616e728db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294070 0x5616e72a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ed979b700 1 -- 172.21.15.62:0/4269573810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294070 msgr2=0x5616e72a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ed979b700 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294070 0x5616e72a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ed979b700 1 -- 172.21.15.62:0/4269573810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e6ffbc60 con 0x5616e7293720 2023-07-14T16:19:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294070 0x5616e72a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:19:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ed979b700 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729c930 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0029fd0 tx=0x7f1ed000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b74e313bffe4431 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ebffff700 1 -- 172.21.15.62:0/4269573810 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0011d90 con 0x5616e7293720 2023-07-14T16:19:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ebffff700 1 -- 172.21.15.62:0/4269573810 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ed002e070 con 0x5616e7293720 2023-07-14T16:19:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.040+0000 7f1ebffff700 1 -- 172.21.15.62:0/4269573810 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed001ed40 con 0x5616e7293720 2023-07-14T16:19:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/4269573810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 msgr2=0x5616e729c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729c930 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0029fd0 tx=0x7f1ed000b7c0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/4269573810 shutdown_connections 2023-07-14T16:19:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294e30 0x5616e728db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7293720 0x5616e729c930 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/4269573810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7294070 0x5616e72a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/4269573810 >> 172.21.15.62:0/4269573810 conn(0x5616e7288ff0 msgr2=0x5616e72afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/4269573810 shutdown_connections 2023-07-14T16:19:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.041+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/4269573810 wait complete. 2023-07-14T16:19:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.042+0000 7f1eef3bbb80 1 -- start start 2023-07-14T16:19:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.042+0000 7f1eef3bbb80 1 Processor -- start 2023-07-14T16:19:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.042+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e72a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.042+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.042+0000 7f1eef3bbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.042+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e72a4240 con 0x5616e7294e30 2023-07-14T16:19:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.042+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e72a43b0 con 0x5616e7293720 2023-07-14T16:19:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.042+0000 7f1eef3bbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e72a4570 con 0x5616e7294070 2023-07-14T16:19:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1eef3bbb80 10 client.0 _ll_register_callbacks cb 0x7f1ed000fd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed8f9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed979b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e72a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed9f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed8f9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:59700/0 (socket says 172.21.15.62:59700) 2023-07-14T16:19:11.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed8f9a700 1 -- 172.21.15.62:0/13842140 learned_addr learned my addr 172.21.15.62:0/13842140 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed979b700 1 -- 172.21.15.62:0/13842140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 msgr2=0x5616e72a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed979b700 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed979b700 1 -- 172.21.15.62:0/13842140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 msgr2=0x5616e72a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed979b700 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed979b700 1 -- 172.21.15.62:0/13842140 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed410a910 con 0x5616e7293720 2023-07-14T16:19:11.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed9f9c700 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.043+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.044+0000 7f1ed979b700 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e72a07a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000f6f0 tx=0x7f1ed0028e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f48818fa5438bf6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.044+0000 7f1ebcff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:19:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.044+0000 7f1ebcff9700 1 -- 172.21.15.62:0/13842140 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed001edc0 con 0x5616e7293720 2023-07-14T16:19:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.044+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/13842140 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ecc00cea0 con 0x5616e7293720 2023-07-14T16:19:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.044+0000 7f1ebcff9700 1 -- 172.21.15.62:0/13842140 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ed0005040 con 0x5616e7293720 2023-07-14T16:19:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.044+0000 7f1ebcff9700 1 -- 172.21.15.62:0/13842140 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ed0005240 con 0x5616e7293720 2023-07-14T16:19:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.044+0000 7f1eef3bbb80 10 client.26177 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/13842140 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5616e7039250 con 0x5616e7293720 2023-07-14T16:19:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1ebcff9700 1 -- 172.21.15.62:0/13842140 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ecc00cea0 con 0x5616e7293720 2023-07-14T16:19:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1ebcff9700 1 -- 172.21.15.62:0/13842140 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1ed0011720 con 0x5616e7293720 2023-07-14T16:19:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1eef3bbb80 10 client.26177 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1eef3bbb80 10 client.26177 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:19:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1eef3bbb80 20 client.26177 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5616e72fdd20 0x5616e7300130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1eef3bbb80 4 client.26177 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:19:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.045+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/13842140 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5616e71a0050 con 0x5616e72fdd20 2023-07-14T16:19:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.046+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5616e72fdd20 0x5616e7300130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.046+0000 7f1ed8f9a700 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5616e72fdd20 0x5616e7300130 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f43955a66f6ac247 server_cookie=927d918919c78e68 in_seq=0 out_seq=0 2023-07-14T16:19:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.046+0000 7f1ebcff9700 10 client.26177 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:19:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.046+0000 7f1ebcff9700 1 -- 172.21.15.62:0/13842140 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5616e71a0050 con 0x5616e72fdd20 2023-07-14T16:19:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.046+0000 7f1ebcff9700 10 client.26177 handle_command_reply: tid=0 2023-07-14T16:19:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.046+0000 7f1eef3bbb80 1 client.26177 shutdown 2023-07-14T16:19:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.046+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/13842140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 msgr2=0x5616e72a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.046+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e72a07a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000f6f0 tx=0x7f1ed0028e90 comp rx=0 tx=0).stop 2023-07-14T16:19:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/13842140 shutdown_connections 2023-07-14T16:19:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5616e72fdd20 0x5616e7300130 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616e7294e30 0x5616e72a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5616e7293720 0x5616e72a07a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 1 --2- 172.21.15.62:0/13842140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616e7294070 0x5616e72a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1ebcff9700 0 client.26177 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:19:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/13842140 >> 172.21.15.62:0/13842140 conn(0x5616e7288ff0 msgr2=0x5616e728db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/13842140 shutdown_connections 2023-07-14T16:19:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 1 -- 172.21.15.62:0/13842140 wait complete. 2023-07-14T16:19:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eef3bbb80 20 client.26177 trim_cache size 0 max 16384 2023-07-14T16:19:11.034 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:19:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eda79d700 1 -- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ec00a11f0 msgr2=0x7f1ec00a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eda79d700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ec00a11f0 0x7f1ec00a36a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000bbd0 tx=0x7f1ed000b5b0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eda79d700 1 -- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40fec10 msgr2=0x7f1ed4197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.047+0000 7f1eda79d700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40fec10 0x7f1ed4197ca0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400d950 tx=0x7f1ec400dd10 comp rx=0 tx=0).stop 2023-07-14T16:19:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.048+0000 7f1eda79d700 1 -- 172.21.15.62:0/1712360858 shutdown_connections 2023-07-14T16:19:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.048+0000 7f1eda79d700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ec00a11f0 0x7f1ec00a36a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.048+0000 7f1eda79d700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ed40ff5c0 0x7f1ed419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.048+0000 7f1eda79d700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ed40fe2c0 0x7f1ed4197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.048+0000 7f1eda79d700 1 --2- 172.21.15.62:0/1712360858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed40fec10 0x7f1ed4197ca0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.048+0000 7f1eda79d700 1 -- 172.21.15.62:0/1712360858 >> 172.21.15.62:0/1712360858 conn(0x7f1ed40f9ed0 msgr2=0x7f1ed40fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.048+0000 7f1eda79d700 1 -- 172.21.15.62:0/1712360858 shutdown_connections 2023-07-14T16:19:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.048+0000 7f1eda79d700 1 -- 172.21.15.62:0/1712360858 wait complete. 2023-07-14T16:19:11.037 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:10 smithi164 ceph-mon[97427]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:19:11.037 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:10 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1637526864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:19:11.045 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:10 smithi062 ceph-mon[95543]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:19:11.046 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:10 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1637526864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:19:11.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:19:11.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:10 smithi081 ceph-mon[127795]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:19:11.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:10 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1637526864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:19:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 -- 172.21.15.62:0/67985059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8103890 msgr2=0x7f05e810a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/67985059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8103890 0x7f05e810a150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f05e000b0a0 tx=0x7f05e000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 -- 172.21.15.62:0/67985059 shutdown_connections 2023-07-14T16:19:11.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/67985059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8103890 0x7f05e810a150 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/67985059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05e8101d10 0x7f05e8102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/67985059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05e8102f00 0x7f05e8103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 -- 172.21.15.62:0/67985059 >> 172.21.15.62:0/67985059 conn(0x7f05e80fd350 msgr2=0x7f05e80ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 -- 172.21.15.62:0/67985059 shutdown_connections 2023-07-14T16:19:11.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.310+0000 7f05ed5ff700 1 -- 172.21.15.62:0/67985059 wait complete. 2023-07-14T16:19:11.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.311+0000 7f05ed5ff700 1 Processor -- start 2023-07-14T16:19:11.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05ed5ff700 1 -- start start 2023-07-14T16:19:11.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05ed5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05e8101d10 0x7f05e819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05ed5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8102f00 0x7f05e819c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05ed5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05e8103890 0x7f05e81a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05ed5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e81a3ab0 con 0x7f05e8103890 2023-07-14T16:19:11.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05ed5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e81a3bf0 con 0x7f05e8102f00 2023-07-14T16:19:11.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05ed5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e81a3d80 con 0x7f05e8101d10 2023-07-14T16:19:11.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05e8103890 0x7f05e81a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05e67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8102f00 0x7f05e819c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05e8103890 0x7f05e81a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38292/0 (socket says 172.21.15.62:38292) 2023-07-14T16:19:11.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05e77fe700 1 -- 172.21.15.62:0/3738899947 learned_addr learned my addr 172.21.15.62:0/3738899947 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.312+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05e8101d10 0x7f05e819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05e67fc700 1 -- 172.21.15.62:0/3738899947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05e8101d10 msgr2=0x7f05e819bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05e67fc700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05e8101d10 0x7f05e819bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05e67fc700 1 -- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05e8103890 msgr2=0x7f05e81a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05e67fc700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05e8103890 0x7f05e81a34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05e67fc700 1 -- 172.21.15.62:0/3738899947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0009d00 con 0x7f05e8102f00 2023-07-14T16:19:11.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05e67fc700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8102f00 0x7f05e819c470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00b450 tx=0x7f05dc00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05cffff700 1 -- 172.21.15.62:0/3738899947 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc011830 con 0x7f05e8102f00 2023-07-14T16:19:11.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05cffff700 1 -- 172.21.15.62:0/3738899947 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05dc00be60 con 0x7f05e8102f00 2023-07-14T16:19:11.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05ed5ff700 1 -- 172.21.15.62:0/3738899947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e81a4060 con 0x7f05e8102f00 2023-07-14T16:19:11.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.313+0000 7f05cffff700 1 -- 172.21.15.62:0/3738899947 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc010ca0 con 0x7f05e8102f00 2023-07-14T16:19:11.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.314+0000 7f05ed5ff700 1 -- 172.21.15.62:0/3738899947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e81a4640 con 0x7f05e8102f00 2023-07-14T16:19:11.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.318+0000 7f05cffff700 1 -- 172.21.15.62:0/3738899947 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f05dc011990 con 0x7f05e8102f00 2023-07-14T16:19:11.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.318+0000 7f0601021b80 1 Processor -- start 2023-07-14T16:19:11.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.318+0000 7f0601021b80 1 -- start start 2023-07-14T16:19:11.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f0601021b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f0601021b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f0601021b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f0601021b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19b37cb0 con 0x55de19c33c70 2023-07-14T16:19:11.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f0601021b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19b8aa40 con 0x55de19c32e20 2023-07-14T16:19:11.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f0601021b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19b83ed0 con 0x55de19c324d0 2023-07-14T16:19:11.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f05e67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f05e67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c340e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38304/0 (socket says 172.21.15.62:38304) 2023-07-14T16:19:11.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f05e67fc700 1 -- 172.21.15.62:0/3053945153 learned_addr learned my addr 172.21.15.62:0/3053945153 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.319+0000 7f05e77fe700 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05e67fc700 1 -- 172.21.15.62:0/3053945153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 msgr2=0x55de19c328e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05e67fc700 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c328e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05e67fc700 1 -- 172.21.15.62:0/3053945153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 msgr2=0x55de19c336e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05e67fc700 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c336e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05e67fc700 1 -- 172.21.15.62:0/3053945153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de199b1de0 con 0x55de19c33c70 2023-07-14T16:19:11.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05cffff700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05d00a11d0 0x7f05d00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05e67fc700 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c340e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0d4190 tx=0x7f05dc0d44a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811a84a7e38e5ee3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05cffff700 1 -- 172.21.15.62:0/3738899947 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05dc0cc1d0 con 0x7f05e8102f00 2023-07-14T16:19:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.320+0000 7f05e77fe700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05d00a11d0 0x7f05d00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f05cd7fa700 1 -- 172.21.15.62:0/3053945153 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc0b90c0 con 0x55de19c33c70 2023-07-14T16:19:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f05cd7fa700 1 -- 172.21.15.62:0/3053945153 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05dc0958d0 con 0x55de19c33c70 2023-07-14T16:19:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f05cd7fa700 1 -- 172.21.15.62:0/3053945153 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc009280 con 0x55de19c33c70 2023-07-14T16:19:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f0601021b80 1 -- 172.21.15.62:0/3053945153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 msgr2=0x55de19c340e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f0601021b80 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c340e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0d4190 tx=0x7f05dc0d44a0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f05e77fe700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05d00a11d0 0x7f05d00a3680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f05e000d010 tx=0x7f05e000cf80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f0601021b80 1 -- 172.21.15.62:0/3053945153 shutdown_connections 2023-07-14T16:19:11.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f0601021b80 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c340e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f0601021b80 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c328e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f0601021b80 1 --2- 172.21.15.62:0/3053945153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c336e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.321+0000 7f0601021b80 1 -- 172.21.15.62:0/3053945153 >> 172.21.15.62:0/3053945153 conn(0x55de19c27da0 msgr2=0x55de19c2d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.322+0000 7f0601021b80 1 -- 172.21.15.62:0/3053945153 shutdown_connections 2023-07-14T16:19:11.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.322+0000 7f0601021b80 1 -- 172.21.15.62:0/3053945153 wait complete. 2023-07-14T16:19:11.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.322+0000 7f0601021b80 1 -- start start 2023-07-14T16:19:11.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.322+0000 7f0601021b80 1 Processor -- start 2023-07-14T16:19:11.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f0601021b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f0601021b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f0601021b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c33c70 0x55de19c436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f0601021b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19b23ac0 con 0x55de19c324d0 2023-07-14T16:19:11.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f0601021b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19b237c0 con 0x55de19c32e20 2023-07-14T16:19:11.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f0601021b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19a07920 con 0x55de19c33c70 2023-07-14T16:19:11.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f0601021b80 10 client.0 _ll_register_callbacks cb 0x55de19b21760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:11.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f05e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f05e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38312/0 (socket says 172.21.15.62:38312) 2023-07-14T16:19:11.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f05e6ffd700 1 -- 172.21.15.62:0/2665081993 learned_addr learned my addr 172.21.15.62:0/2665081993 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f05e67fc700 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.323+0000 7f05e77fe700 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c33c70 0x55de19c436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05e6ffd700 1 -- 172.21.15.62:0/2665081993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c33c70 msgr2=0x55de19c436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c33c70 0x55de19c436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05e6ffd700 1 -- 172.21.15.62:0/2665081993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 msgr2=0x55de19c3c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c3c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05e6ffd700 1 -- 172.21.15.62:0/2665081993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e81a4640 con 0x55de19c324d0 2023-07-14T16:19:11.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3c140 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f05d800b860 tx=0x7f05d800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6417d05920fdae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05caff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:11.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05caff5700 1 -- 172.21.15.62:0/2665081993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8003d30 con 0x55de19c324d0 2023-07-14T16:19:11.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f0601021b80 1 -- 172.21.15.62:0/2665081993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de19add4f0 con 0x55de19c324d0 2023-07-14T16:19:11.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.324+0000 7f05caff5700 1 -- 172.21.15.62:0/2665081993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05d801b440 con 0x55de19c324d0 2023-07-14T16:19:11.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f05caff5700 1 -- 172.21.15.62:0/2665081993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05d8021030 con 0x55de19c324d0 2023-07-14T16:19:11.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f0601021b80 10 client.16464 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:11.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f0601021b80 1 -- 172.21.15.62:0/2665081993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55de1999ac60 con 0x55de19c324d0 2023-07-14T16:19:11.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f05caff5700 1 -- 172.21.15.62:0/2665081993 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de19add4f0 con 0x55de19c324d0 2023-07-14T16:19:11.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f05caff5700 1 -- 172.21.15.62:0/2665081993 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f05d8011560 con 0x55de19c324d0 2023-07-14T16:19:11.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f0601021b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:11.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f0601021b80 10 client.16464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:19:11.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f0601021b80 20 client.16464 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:11.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f0601021b80 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de19c47f30 0x55de19c4a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f0601021b80 4 client.16464 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:11.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.325+0000 7f0601021b80 1 -- 172.21.15.62:0/2665081993 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de19b7ceb0 con 0x55de19c47f30 2023-07-14T16:19:11.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.326+0000 7f05e67fc700 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de19c47f30 0x55de19c4a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.326+0000 7f05e67fc700 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de19c47f30 0x55de19c4a340 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d9579514a4a8887 server_cookie=1d9d4470097f513 in_seq=0 out_seq=0 2023-07-14T16:19:11.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.326+0000 7f05caff5700 10 client.16464 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:19:11.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.327+0000 7f05caff5700 1 -- 172.21.15.62:0/2665081993 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de19b7ceb0 con 0x55de19c47f30 2023-07-14T16:19:11.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.327+0000 7f05caff5700 10 client.16464 handle_command_reply: tid=0 2023-07-14T16:19:11.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.327+0000 7f0601021b80 1 client.16464 shutdown 2023-07-14T16:19:11.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.327+0000 7f0601021b80 1 -- 172.21.15.62:0/2665081993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 msgr2=0x55de19c3c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3c140 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f05d800b860 tx=0x7f05d800bb70 comp rx=0 tx=0).stop 2023-07-14T16:19:11.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 -- 172.21.15.62:0/2665081993 shutdown_connections 2023-07-14T16:19:11.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de19c47f30 0x55de19c4a340 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c33c70 0x55de19c436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3c140 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 --2- 172.21.15.62:0/2665081993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 -- 172.21.15.62:0/2665081993 >> 172.21.15.62:0/2665081993 conn(0x55de19c27da0 msgr2=0x55de19c2c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f05caff5700 0 client.16464 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:19:11.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 -- 172.21.15.62:0/2665081993 shutdown_connections 2023-07-14T16:19:11.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 1 -- 172.21.15.62:0/2665081993 wait complete. 2023-07-14T16:19:11.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.328+0000 7f0601021b80 20 client.16464 trim_cache size 0 max 16384 2023-07-14T16:19:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.348+0000 7f0601021b80 1 Processor -- start 2023-07-14T16:19:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.348+0000 7f0601021b80 1 -- start start 2023-07-14T16:19:11.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f0601021b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f0601021b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c32e20 0x55de19c43680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f0601021b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c33c70 0x55de19c36b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f0601021b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19c2ece0 con 0x55de19c324d0 2023-07-14T16:19:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f0601021b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19c2e060 con 0x55de19c33c70 2023-07-14T16:19:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f0601021b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19c24f60 con 0x55de19c32e20 2023-07-14T16:19:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f05e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f05e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38326/0 (socket says 172.21.15.62:38326) 2023-07-14T16:19:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f05e6ffd700 1 -- 172.21.15.62:0/1948328452 learned_addr learned my addr 172.21.15.62:0/1948328452 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f05e77fe700 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c33c70 0x55de19c36b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.349+0000 7f05e67fc700 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c32e20 0x55de19c43680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05e6ffd700 1 -- 172.21.15.62:0/1948328452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c32e20 msgr2=0x55de19c43680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c32e20 0x55de19c43680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05e6ffd700 1 -- 172.21.15.62:0/1948328452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c33c70 msgr2=0x55de19c36b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c33c70 0x55de19c36b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05e6ffd700 1 -- 172.21.15.62:0/1948328452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de199d8250 con 0x55de19c324d0 2023-07-14T16:19:11.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3a020 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f05d8000c00 tx=0x7f05d8003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb434f659318f2c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05cdffb700 1 -- 172.21.15.62:0/1948328452 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8021070 con 0x55de19c324d0 2023-07-14T16:19:11.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05cdffb700 1 -- 172.21.15.62:0/1948328452 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05d8011440 con 0x55de19c324d0 2023-07-14T16:19:11.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.350+0000 7f05cdffb700 1 -- 172.21.15.62:0/1948328452 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8005580 con 0x55de19c324d0 2023-07-14T16:19:11.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 -- 172.21.15.62:0/1948328452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 msgr2=0x55de19c3a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3a020 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f05d8000c00 tx=0x7f05d8003780 comp rx=0 tx=0).stop 2023-07-14T16:19:11.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 -- 172.21.15.62:0/1948328452 shutdown_connections 2023-07-14T16:19:11.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c33c70 0x55de19c36b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c324d0 0x55de19c3a020 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 --2- 172.21.15.62:0/1948328452 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c32e20 0x55de19c43680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 -- 172.21.15.62:0/1948328452 >> 172.21.15.62:0/1948328452 conn(0x55de19c27da0 msgr2=0x55de19c3ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 -- 172.21.15.62:0/1948328452 shutdown_connections 2023-07-14T16:19:11.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.351+0000 7f0601021b80 1 -- 172.21.15.62:0/1948328452 wait complete. 2023-07-14T16:19:11.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.352+0000 7f0601021b80 1 -- start start 2023-07-14T16:19:11.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.352+0000 7f0601021b80 1 Processor -- start 2023-07-14T16:19:11.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.352+0000 7f0601021b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c42e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.352+0000 7f0601021b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f0601021b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c3f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f0601021b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19c3fe70 con 0x55de19c33c70 2023-07-14T16:19:11.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f0601021b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19c3ffe0 con 0x55de19c32e20 2023-07-14T16:19:11.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f0601021b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de19c401a0 con 0x55de19c324d0 2023-07-14T16:19:11.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f0601021b80 10 client.0 _ll_register_callbacks cb 0x7f05d8004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:11.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c3f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e6ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c42e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e6ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c42e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:59744/0 (socket says 172.21.15.62:59744) 2023-07-14T16:19:11.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e6ffd700 1 -- 172.21.15.62:0/389936646 learned_addr learned my addr 172.21.15.62:0/389936646 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e67fc700 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e6ffd700 1 -- 172.21.15.62:0/389936646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 msgr2=0x55de19c433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e6ffd700 1 -- 172.21.15.62:0/389936646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 msgr2=0x55de19c3f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c3f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e6ffd700 1 -- 172.21.15.62:0/389936646 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de1999ac60 con 0x55de19c324d0 2023-07-14T16:19:11.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.353+0000 7f05e67fc700 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:11.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.354+0000 7f05e6ffd700 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c42e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05d8004150 tx=0x7f05d8005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf85d38bf7f32fd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.354+0000 7f05caff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:19:11.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.354+0000 7f05caff5700 1 -- 172.21.15.62:0/389936646 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8021030 con 0x55de19c324d0 2023-07-14T16:19:11.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.354+0000 7f05caff5700 1 -- 172.21.15.62:0/389936646 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05d8000e40 con 0x55de19c324d0 2023-07-14T16:19:11.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.354+0000 7f0601021b80 1 -- 172.21.15.62:0/389936646 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05dc0958d0 con 0x55de19c324d0 2023-07-14T16:19:11.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.354+0000 7f05caff5700 1 -- 172.21.15.62:0/389936646 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05d8005500 con 0x55de19c324d0 2023-07-14T16:19:11.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.354+0000 7f0601021b80 10 client.26107 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:11.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.354+0000 7f0601021b80 1 -- 172.21.15.62:0/389936646 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55de199b02b0 con 0x55de19c324d0 2023-07-14T16:19:11.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f05caff5700 1 -- 172.21.15.62:0/389936646 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05dc0958d0 con 0x55de19c324d0 2023-07-14T16:19:11.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f05caff5700 1 -- 172.21.15.62:0/389936646 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f05d801b540 con 0x55de19c324d0 2023-07-14T16:19:11.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f0601021b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:11.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f0601021b80 10 client.26107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:19:11.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f0601021b80 20 client.26107 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:11.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f0601021b80 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de19c9cf20 0x55de19c9f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f0601021b80 4 client.26107 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:19:11.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f0601021b80 1 -- 172.21.15.62:0/389936646 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de19b3f050 con 0x55de19c9cf20 2023-07-14T16:19:11.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.355+0000 7f05e67fc700 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de19c9cf20 0x55de19c9f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.356+0000 7f05e67fc700 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de19c9cf20 0x55de19c9f330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9b93ec1faad3c5e server_cookie=3cc781bca90ff947 in_seq=0 out_seq=0 2023-07-14T16:19:11.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.356+0000 7f05caff5700 10 client.26107 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:19:11.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.356+0000 7f05caff5700 1 -- 172.21.15.62:0/389936646 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de19b3f050 con 0x55de19c9cf20 2023-07-14T16:19:11.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.356+0000 7f05caff5700 10 client.26107 handle_command_reply: tid=0 2023-07-14T16:19:11.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.356+0000 7f0601021b80 1 client.26107 shutdown 2023-07-14T16:19:11.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 -- 172.21.15.62:0/389936646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 msgr2=0x55de19c42e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c42e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05d8004150 tx=0x7f05d8005a00 comp rx=0 tx=0).stop 2023-07-14T16:19:11.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 -- 172.21.15.62:0/389936646 shutdown_connections 2023-07-14T16:19:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de19c9cf20 0x55de19c9f330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de19c33c70 0x55de19c3f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de19c324d0 0x55de19c42e90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f05caff5700 0 client.26107 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:19:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 --2- 172.21.15.62:0/389936646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de19c32e20 0x55de19c433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 -- 172.21.15.62:0/389936646 >> 172.21.15.62:0/389936646 conn(0x55de19c27da0 msgr2=0x55de19c3a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 -- 172.21.15.62:0/389936646 shutdown_connections 2023-07-14T16:19:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 1 -- 172.21.15.62:0/389936646 wait complete. 2023-07-14T16:19:11.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.357+0000 7f0601021b80 20 client.26107 trim_cache size 0 max 16384 2023-07-14T16:19:11.345 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:19:11.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 -- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05d00a11d0 msgr2=0x7f05d00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05d00a11d0 0x7f05d00a3680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f05e000d010 tx=0x7f05e000cf80 comp rx=0 tx=0).stop 2023-07-14T16:19:11.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 -- 172.21.15.62:0/3738899947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8102f00 msgr2=0x7f05e819c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8102f00 0x7f05e819c470 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00b450 tx=0x7f05dc00b760 comp rx=0 tx=0).stop 2023-07-14T16:19:11.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 -- 172.21.15.62:0/3738899947 shutdown_connections 2023-07-14T16:19:11.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05e8103890 0x7f05e81a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05e8101d10 0x7f05e819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05d00a11d0 0x7f05d00a3680 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 --2- 172.21.15.62:0/3738899947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05e8102f00 0x7f05e819c470 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 -- 172.21.15.62:0/3738899947 >> 172.21.15.62:0/3738899947 conn(0x7f05e80fd350 msgr2=0x7f05e80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 -- 172.21.15.62:0/3738899947 shutdown_connections 2023-07-14T16:19:11.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.358+0000 7f05ed5ff700 1 -- 172.21.15.62:0/3738899947 wait complete. 2023-07-14T16:19:11.355 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:19:11.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 -- 172.21.15.62:0/1210750107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b8100c60 msgr2=0x7fb7b81010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/1210750107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b8100c60 0x7fb7b81010d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009990 tx=0x7fb7ac009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 -- 172.21.15.62:0/1210750107 shutdown_connections 2023-07-14T16:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/1210750107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b8101610 0x7fb7b8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/1210750107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7b80ff9d0 0x7fb7b80ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/1210750107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b8100c60 0x7fb7b81010d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 -- 172.21.15.62:0/1210750107 >> 172.21.15.62:0/1210750107 conn(0x7fb7b80fb010 msgr2=0x7fb7b80fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 -- 172.21.15.62:0/1210750107 shutdown_connections 2023-07-14T16:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.613+0000 7fb7b7fff700 1 -- 172.21.15.62:0/1210750107 wait complete. 2023-07-14T16:19:11.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.614+0000 7fb7b7fff700 1 Processor -- start 2023-07-14T16:19:11.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.614+0000 7fb7b7fff700 1 -- start start 2023-07-14T16:19:11.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b80ff9d0 0x7fb7b80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b8100c60 0x7fb7b8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7b8101610 0x7fb7b8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b80735e0 con 0x7fb7b8100c60 2023-07-14T16:19:11.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b7fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b8073750 con 0x7fb7b80ff9d0 2023-07-14T16:19:11.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b7fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b819f5e0 con 0x7fb7b8101610 2023-07-14T16:19:11.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b8100c60 0x7fb7b8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b8100c60 0x7fb7b8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38364/0 (socket says 172.21.15.62:38364) 2023-07-14T16:19:11.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b67fc700 1 -- 172.21.15.62:0/399851397 learned_addr learned my addr 172.21.15.62:0/399851397 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.615+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b80ff9d0 0x7fb7b80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b77fe700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7b8101610 0x7fb7b8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b67fc700 1 -- 172.21.15.62:0/399851397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7b8101610 msgr2=0x7fb7b8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7b8101610 0x7fb7b8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b67fc700 1 -- 172.21.15.62:0/399851397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b80ff9d0 msgr2=0x7fb7b80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b80ff9d0 0x7fb7b80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b67fc700 1 -- 172.21.15.62:0/399851397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0009c40 con 0x7fb7b8100c60 2023-07-14T16:19:11.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b80ff9d0 0x7fb7b80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:11.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b8100c60 0x7fb7b8072900 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0098e0 tx=0x7fb7ac009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7bc9fd700 1 -- 172.21.15.62:0/399851397 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac01d070 con 0x7fb7b8100c60 2023-07-14T16:19:11.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7bc9fd700 1 -- 172.21.15.62:0/399851397 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7ac022470 con 0x7fb7b8100c60 2023-07-14T16:19:11.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7bc9fd700 1 -- 172.21.15.62:0/399851397 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00f6c0 con 0x7fb7b8100c60 2023-07-14T16:19:11.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.616+0000 7fb7b7fff700 1 -- 172.21.15.62:0/399851397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac0095f0 con 0x7fb7b8100c60 2023-07-14T16:19:11.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.617+0000 7fb7b7fff700 1 -- 172.21.15.62:0/399851397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b819faf0 con 0x7fb7b8100c60 2023-07-14T16:19:11.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.620+0000 7fb7bc9fd700 1 -- 172.21.15.62:0/399851397 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb7ac00baa0 con 0x7fb7b8100c60 2023-07-14T16:19:11.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7d141cb80 1 Processor -- start 2023-07-14T16:19:11.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7d141cb80 1 -- start start 2023-07-14T16:19:11.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7bc9fd700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7a00a1270 0x7fb7a00a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7bc9fd700 1 -- 172.21.15.62:0/399851397 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7ac0cde50 con 0x7fb7b8100c60 2023-07-14T16:19:11.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e30718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e30722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e2f75cb0 con 0x5624e3070720 2023-07-14T16:19:11.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e2fc8a40 con 0x5624e3071e30 2023-07-14T16:19:11.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.622+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e2fc1ed0 con 0x5624e3071070 2023-07-14T16:19:11.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b77fe700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7a00a1270 0x7fb7a00a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38378/0 (socket says 172.21.15.62:38378) 2023-07-14T16:19:11.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b6ffd700 1 -- 172.21.15.62:0/3256343001 learned_addr learned my addr 172.21.15.62:0/3256343001 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e30718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b77fe700 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e30722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b67fc700 1 -- 172.21.15.62:0/3256343001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 msgr2=0x5624e30722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e30722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b67fc700 1 -- 172.21.15.62:0/3256343001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 msgr2=0x5624e3070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b67fc700 1 -- 172.21.15.62:0/3256343001 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624e2defde0 con 0x5624e3071070 2023-07-14T16:19:11.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e30718a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0cb900 tx=0x7fb7ac005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f1213623b5492d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.623+0000 7fb7b77fe700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7a00a1270 0x7fb7a00a3720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009c10 tx=0x7fb7b0010530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.624+0000 7fb79dffb700 1 -- 172.21.15.62:0/3256343001 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac09f2d0 con 0x5624e3071070 2023-07-14T16:19:11.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.624+0000 7fb79dffb700 1 -- 172.21.15.62:0/3256343001 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7ac0a3660 con 0x5624e3071070 2023-07-14T16:19:11.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.624+0000 7fb79dffb700 1 -- 172.21.15.62:0/3256343001 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac022890 con 0x5624e3071070 2023-07-14T16:19:11.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.624+0000 7fb7d141cb80 1 -- 172.21.15.62:0/3256343001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 msgr2=0x5624e30718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.624+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e30718a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0cb900 tx=0x7fb7ac005fd0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.625+0000 7fb7d141cb80 1 -- 172.21.15.62:0/3256343001 shutdown_connections 2023-07-14T16:19:11.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.625+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e30722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.625+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.625+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/3256343001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e30718a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.625+0000 7fb7d141cb80 1 -- 172.21.15.62:0/3256343001 >> 172.21.15.62:0/3256343001 conn(0x5624e3065ff0 msgr2=0x5624e306b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.625+0000 7fb7d141cb80 1 -- 172.21.15.62:0/3256343001 shutdown_connections 2023-07-14T16:19:11.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.625+0000 7fb7d141cb80 1 -- 172.21.15.62:0/3256343001 wait complete. 2023-07-14T16:19:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.625+0000 7fb7d141cb80 1 -- start start 2023-07-14T16:19:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7d141cb80 1 Processor -- start 2023-07-14T16:19:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3079f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e307a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e30814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e2f61ac0 con 0x5624e3070720 2023-07-14T16:19:11.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e2f617c0 con 0x5624e3071e30 2023-07-14T16:19:11.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e2e45920 con 0x5624e3071070 2023-07-14T16:19:11.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7d141cb80 10 client.0 _ll_register_callbacks cb 0x5624e2f5f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:11.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7b6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3079f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7b6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3079f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38394/0 (socket says 172.21.15.62:38394) 2023-07-14T16:19:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7b6ffd700 1 -- 172.21.15.62:0/341254805 learned_addr learned my addr 172.21.15.62:0/341254805 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.626+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e307a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb7b77fe700 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e30814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb7b6ffd700 1 -- 172.21.15.62:0/341254805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 msgr2=0x5624e307a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e307a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb7b6ffd700 1 -- 172.21.15.62:0/341254805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 msgr2=0x5624e30814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e30814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb7b6ffd700 1 -- 172.21.15.62:0/341254805 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b819faf0 con 0x5624e3070720 2023-07-14T16:19:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3079f30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8012840 tx=0x7fb7a8012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59645659af32664f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb796ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb796ffd700 1 -- 172.21.15.62:0/341254805 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8017070 con 0x5624e3070720 2023-07-14T16:19:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.627+0000 7fb7d141cb80 1 -- 172.21.15.62:0/341254805 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624e2f1b4f0 con 0x5624e3070720 2023-07-14T16:19:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb796ffd700 1 -- 172.21.15.62:0/341254805 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7a8011770 con 0x5624e3070720 2023-07-14T16:19:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb796ffd700 1 -- 172.21.15.62:0/341254805 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7a8009620 con 0x5624e3070720 2023-07-14T16:19:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb7d141cb80 10 client.16482 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb7d141cb80 1 -- 172.21.15.62:0/341254805 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5624e2dd8c60 con 0x5624e3070720 2023-07-14T16:19:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb796ffd700 1 -- 172.21.15.62:0/341254805 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5624e2f1b4f0 con 0x5624e3070720 2023-07-14T16:19:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb796ffd700 1 -- 172.21.15.62:0/341254805 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb7a801cb70 con 0x5624e3070720 2023-07-14T16:19:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb7d141cb80 10 client.16482 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb7d141cb80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb7d141cb80 20 client.16482 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5624e3085cc0 0x5624e30880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb7d141cb80 4 client.16482 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:11.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.628+0000 7fb7d141cb80 1 -- 172.21.15.62:0/341254805 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624e2fbaeb0 con 0x5624e3085cc0 2023-07-14T16:19:11.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.629+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5624e3085cc0 0x5624e30880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.629+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5624e3085cc0 0x5624e30880d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f940c1e6336a0d server_cookie=bd13217071a70b27 in_seq=0 out_seq=0 2023-07-14T16:19:11.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.629+0000 7fb796ffd700 10 client.16482 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:11.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.630+0000 7fb796ffd700 1 -- 172.21.15.62:0/341254805 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5624e2fbaeb0 con 0x5624e3085cc0 2023-07-14T16:19:11.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.630+0000 7fb796ffd700 10 client.16482 handle_command_reply: tid=0 2023-07-14T16:19:11.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.630+0000 7fb7d141cb80 1 client.16482 shutdown 2023-07-14T16:19:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.630+0000 7fb7d141cb80 1 -- 172.21.15.62:0/341254805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 msgr2=0x5624e3079f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.630+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3079f30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8012840 tx=0x7fb7a8012b50 comp rx=0 tx=0).stop 2023-07-14T16:19:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 1 -- 172.21.15.62:0/341254805 shutdown_connections 2023-07-14T16:19:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5624e3085cc0 0x5624e30880d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e30814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3070720 0x5624e3079f30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/341254805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e307a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb796ffd700 0 client.16482 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 1 -- 172.21.15.62:0/341254805 >> 172.21.15.62:0/341254805 conn(0x5624e3065ff0 msgr2=0x5624e306ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 1 -- 172.21.15.62:0/341254805 shutdown_connections 2023-07-14T16:19:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 1 -- 172.21.15.62:0/341254805 wait complete. 2023-07-14T16:19:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.631+0000 7fb7d141cb80 20 client.16482 trim_cache size 0 max 16384 2023-07-14T16:19:11.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.644+0000 7fb7d141cb80 1 Processor -- start 2023-07-14T16:19:11.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.644+0000 7fb7d141cb80 1 -- start start 2023-07-14T16:19:11.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3070720 0x5624e306a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e3081470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071e30 0x5624e3074c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e3079b00 con 0x5624e3071e30 2023-07-14T16:19:11.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e2f1b4f0 con 0x5624e3070720 2023-07-14T16:19:11.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e30789f0 con 0x5624e3071070 2023-07-14T16:19:11.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071e30 0x5624e3074c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071e30 0x5624e3074c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38408/0 (socket says 172.21.15.62:38408) 2023-07-14T16:19:11.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7b77fe700 1 -- 172.21.15.62:0/797613218 learned_addr learned my addr 172.21.15.62:0/797613218 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e3081470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.645+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3070720 0x5624e306a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb7b77fe700 1 -- 172.21.15.62:0/797613218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 msgr2=0x5624e3081470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb7b77fe700 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e3081470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb7b77fe700 1 -- 172.21.15.62:0/797613218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3070720 msgr2=0x5624e306a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb7b77fe700 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3070720 0x5624e306a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb7b77fe700 1 -- 172.21.15.62:0/797613218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624e2e16250 con 0x5624e3071e30 2023-07-14T16:19:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e3081470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb7b77fe700 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071e30 0x5624e3074c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009340 tx=0x7fb7b001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b8c1cf0a963cd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb79e7fc700 1 -- 172.21.15.62:0/797613218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0029070 con 0x5624e3071e30 2023-07-14T16:19:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb79e7fc700 1 -- 172.21.15.62:0/797613218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7b0003e90 con 0x5624e3071e30 2023-07-14T16:19:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.646+0000 7fb79e7fc700 1 -- 172.21.15.62:0/797613218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000e040 con 0x5624e3071e30 2023-07-14T16:19:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 -- 172.21.15.62:0/797613218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071e30 msgr2=0x5624e3074c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071e30 0x5624e3074c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009340 tx=0x7fb7b001f000 comp rx=0 tx=0).stop 2023-07-14T16:19:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 -- 172.21.15.62:0/797613218 shutdown_connections 2023-07-14T16:19:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071e30 0x5624e3074c90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3070720 0x5624e306a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/797613218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3071070 0x5624e3081470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 -- 172.21.15.62:0/797613218 >> 172.21.15.62:0/797613218 conn(0x5624e3065ff0 msgr2=0x5624e3081da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 -- 172.21.15.62:0/797613218 shutdown_connections 2023-07-14T16:19:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.647+0000 7fb7d141cb80 1 -- 172.21.15.62:0/797613218 wait complete. 2023-07-14T16:19:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.648+0000 7fb7d141cb80 1 -- start start 2023-07-14T16:19:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.648+0000 7fb7d141cb80 1 Processor -- start 2023-07-14T16:19:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.648+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3070720 0x5624e3080d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.648+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071070 0x5624e30812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7d141cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e307d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e307df00 con 0x5624e3071070 2023-07-14T16:19:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e307e070 con 0x5624e3071e30 2023-07-14T16:19:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7d141cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624e307e230 con 0x5624e3070720 2023-07-14T16:19:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7d141cb80 10 client.0 _ll_register_callbacks cb 0x7fb7a8009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e307d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071070 0x5624e30812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e307d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:55496/0 (socket says 172.21.15.62:55496) 2023-07-14T16:19:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b77fe700 1 -- 172.21.15.62:0/469330333 learned_addr learned my addr 172.21.15.62:0/469330333 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3070720 0x5624e3080d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b67fc700 1 -- 172.21.15.62:0/469330333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3070720 msgr2=0x5624e3080d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3070720 0x5624e3080d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b67fc700 1 -- 172.21.15.62:0/469330333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 msgr2=0x5624e307d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e307d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.649+0000 7fb7b67fc700 1 -- 172.21.15.62:0/469330333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624e2defde0 con 0x5624e3071070 2023-07-14T16:19:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb7b67fc700 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071070 0x5624e30812c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac005f50 tx=0x7fb7ac00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5f7ac6e5711618 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb796ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb796ffd700 1 -- 172.21.15.62:0/469330333 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac09f2d0 con 0x5624e3071070 2023-07-14T16:19:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb796ffd700 1 -- 172.21.15.62:0/469330333 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7ac0225e0 con 0x5624e3071070 2023-07-14T16:19:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb7d141cb80 1 -- 172.21.15.62:0/469330333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7ac0a3660 con 0x5624e3071070 2023-07-14T16:19:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb796ffd700 1 -- 172.21.15.62:0/469330333 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7ac022890 con 0x5624e3071070 2023-07-14T16:19:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb7d141cb80 10 client.16488 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb7d141cb80 1 -- 172.21.15.62:0/469330333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fb7b00090b0 con 0x5624e3071070 2023-07-14T16:19:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.650+0000 7fb796ffd700 1 -- 172.21.15.62:0/469330333 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7ac0a3660 con 0x5624e3071070 2023-07-14T16:19:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.651+0000 7fb796ffd700 1 -- 172.21.15.62:0/469330333 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb7ac0a31d0 con 0x5624e3071070 2023-07-14T16:19:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.651+0000 7fb7d141cb80 10 client.16488 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.651+0000 7fb7d141cb80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.651+0000 7fb7d141cb80 20 client.16488 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.651+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5624e30dad90 0x5624e30dd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.651+0000 7fb7d141cb80 4 client.16488 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:19:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.651+0000 7fb7d141cb80 1 -- 172.21.15.62:0/469330333 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5624e2f7d050 con 0x5624e30dad90 2023-07-14T16:19:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.651+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5624e30dad90 0x5624e30dd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb796ffd700 10 client.16488 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7b6ffd700 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5624e30dad90 0x5624e30dd1a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af2abd92815f84b7 server_cookie=c8d6cb7e783bec91 in_seq=0 out_seq=0 2023-07-14T16:19:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb796ffd700 1 -- 172.21.15.62:0/469330333 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5624e2f7d050 con 0x5624e30dad90 2023-07-14T16:19:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb796ffd700 10 client.16488 handle_command_reply: tid=0 2023-07-14T16:19:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7d141cb80 1 client.16488 shutdown 2023-07-14T16:19:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7d141cb80 1 -- 172.21.15.62:0/469330333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071070 msgr2=0x5624e30812c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071070 0x5624e30812c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac005f50 tx=0x7fb7ac00b810 comp rx=0 tx=0).stop 2023-07-14T16:19:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7d141cb80 1 -- 172.21.15.62:0/469330333 shutdown_connections 2023-07-14T16:19:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5624e30dad90 0x5624e30dd1a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5624e3071e30 0x5624e307d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5624e3070720 0x5624e3080d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.652+0000 7fb7d141cb80 1 --2- 172.21.15.62:0/469330333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5624e3071070 0x5624e30812c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb7d141cb80 1 -- 172.21.15.62:0/469330333 >> 172.21.15.62:0/469330333 conn(0x5624e3065ff0 msgr2=0x5624e3078c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb796ffd700 0 client.16488 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:11.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb7d141cb80 1 -- 172.21.15.62:0/469330333 shutdown_connections 2023-07-14T16:19:11.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb7d141cb80 1 -- 172.21.15.62:0/469330333 wait complete. 2023-07-14T16:19:11.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb7d141cb80 20 client.16488 trim_cache size 0 max 16384 2023-07-14T16:19:11.638 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:19:11.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb7b7fff700 1 -- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7a00a1270 msgr2=0x7fb7a00a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7a00a1270 0x7fb7a00a3720 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009c10 tx=0x7fb7b0010530 comp rx=0 tx=0).stop 2023-07-14T16:19:11.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb7b7fff700 1 -- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b8100c60 msgr2=0x7fb7b8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:11.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.653+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b8100c60 0x7fb7b8072900 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0098e0 tx=0x7fb7ac009570 comp rx=0 tx=0).stop 2023-07-14T16:19:11.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.654+0000 7fb7b7fff700 1 -- 172.21.15.62:0/399851397 shutdown_connections 2023-07-14T16:19:11.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.654+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7b8101610 0x7fb7b8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.654+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7a00a1270 0x7fb7a00a3720 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.654+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7b80ff9d0 0x7fb7b80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.654+0000 7fb7b7fff700 1 --2- 172.21.15.62:0/399851397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b8100c60 0x7fb7b8072900 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:11.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.654+0000 7fb7b7fff700 1 -- 172.21.15.62:0/399851397 >> 172.21.15.62:0/399851397 conn(0x7fb7b80fb010 msgr2=0x7fb7b81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:11.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.654+0000 7fb7b7fff700 1 -- 172.21.15.62:0/399851397 shutdown_connections 2023-07-14T16:19:11.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:11.654+0000 7fb7b7fff700 1 -- 172.21.15.62:0/399851397 wait complete. 2023-07-14T16:19:12.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:19:12.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:19:12.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.905+0000 7fedfcdff700 1 -- 172.21.15.62:0/1923422609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf8100560 msgr2=0x7fedf8102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.905+0000 7fedfcdff700 1 --2- 172.21.15.62:0/1923422609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf8100560 0x7fedf8102940 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fede8009990 tx=0x7fede8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.905+0000 7fedfcdff700 1 -- 172.21.15.62:0/1923422609 shutdown_connections 2023-07-14T16:19:12.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.905+0000 7fedfcdff700 1 --2- 172.21.15.62:0/1923422609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf81057a0 0x7fedf8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.905+0000 7fedfcdff700 1 --2- 172.21.15.62:0/1923422609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf8100560 0x7fedf8102940 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.905+0000 7fedfcdff700 1 --2- 172.21.15.62:0/1923422609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf8102e80 0x7fedf8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.905+0000 7fedfcdff700 1 -- 172.21.15.62:0/1923422609 >> 172.21.15.62:0/1923422609 conn(0x7fedf80fa000 msgr2=0x7fedf80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:12.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.906+0000 7fedfcdff700 1 -- 172.21.15.62:0/1923422609 shutdown_connections 2023-07-14T16:19:12.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.906+0000 7fedfcdff700 1 -- 172.21.15.62:0/1923422609 wait complete. 2023-07-14T16:19:12.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedfcdff700 1 Processor -- start 2023-07-14T16:19:12.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedfcdff700 1 -- start start 2023-07-14T16:19:12.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedfcdff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf8100560 0x7fedf81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedfcdff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf8102e80 0x7fedf8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedfcdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf81057a0 0x7fedf819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedfcdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819f570 con 0x7fedf81057a0 2023-07-14T16:19:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedfcdff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819f6b0 con 0x7fedf8102e80 2023-07-14T16:19:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedfcdff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819f840 con 0x7fedf8100560 2023-07-14T16:19:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedf6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf81057a0 0x7fedf819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedf6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf81057a0 0x7fedf819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57778/0 (socket says 172.21.15.62:57778) 2023-07-14T16:19:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedf6d9d700 1 -- 172.21.15.62:0/3510283636 learned_addr learned my addr 172.21.15.62:0/3510283636 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.907+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf8102e80 0x7fedf8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.908+0000 7fedf659c700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf8100560 0x7fedf81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.908+0000 7fedf6d9d700 1 -- 172.21.15.62:0/3510283636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf8100560 msgr2=0x7fedf81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.908+0000 7fedf6d9d700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf8100560 0x7fedf81979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.908+0000 7fedf6d9d700 1 -- 172.21.15.62:0/3510283636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf8102e80 msgr2=0x7fedf8197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.908+0000 7fedf6d9d700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf8102e80 0x7fedf8197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.908+0000 7fedf6d9d700 1 -- 172.21.15.62:0/3510283636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0009c40 con 0x7fedf81057a0 2023-07-14T16:19:12.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.908+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf8102e80 0x7fedf8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:12.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.908+0000 7fedf6d9d700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf81057a0 0x7fedf819efa0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fedf000c370 tx=0x7fedf000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:12.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.909+0000 7fede77fe700 1 -- 172.21.15.62:0/3510283636 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0017070 con 0x7fedf81057a0 2023-07-14T16:19:12.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.909+0000 7fedfcdff700 1 -- 172.21.15.62:0/3510283636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede80095f0 con 0x7fedf81057a0 2023-07-14T16:19:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.909+0000 7fede77fe700 1 -- 172.21.15.62:0/3510283636 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedf000f040 con 0x7fedf81057a0 2023-07-14T16:19:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.909+0000 7fede77fe700 1 -- 172.21.15.62:0/3510283636 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000e050 con 0x7fedf81057a0 2023-07-14T16:19:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.909+0000 7fedfcdff700 1 -- 172.21.15.62:0/3510283636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf819fe90 con 0x7fedf81057a0 2023-07-14T16:19:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.912+0000 7fede77fe700 1 -- 172.21.15.62:0/3510283636 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fedf000cb60 con 0x7fedf81057a0 2023-07-14T16:19:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.914+0000 7fee1076db80 1 Processor -- start 2023-07-14T16:19:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.914+0000 7fee1076db80 1 -- start start 2023-07-14T16:19:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fee1076db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711550 0x55a9f8711960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fee1076db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8712760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fee1076db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8712cf0 0x55a9f8713160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fee1076db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8616d30 con 0x55a9f8711ea0 2023-07-14T16:19:12.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fee1076db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8669ac0 con 0x55a9f8712cf0 2023-07-14T16:19:12.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fee1076db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8662f50 con 0x55a9f8711550 2023-07-14T16:19:12.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fede77fe700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fede00a11d0 0x7fede00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fedf5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711550 0x55a9f8711960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fedf659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8712760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fede77fe700 1 -- 172.21.15.62:0/3510283636 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fedf00cc230 con 0x7fedf81057a0 2023-07-14T16:19:12.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fedf5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711550 0x55a9f8711960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34144/0 (socket says 172.21.15.62:34144) 2023-07-14T16:19:12.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fedf5d9b700 1 -- 172.21.15.62:0/3834952290 learned_addr learned my addr 172.21.15.62:0/3834952290 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:12.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3834952290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8712cf0 0x55a9f8713160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.915+0000 7fedf6d9d700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fede00a11d0 0x7fede00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.916+0000 7fedf5d9b700 1 -- 172.21.15.62:0/3834952290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8712cf0 msgr2=0x55a9f8713160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:12.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.916+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3834952290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8712cf0 0x55a9f8713160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.916+0000 7fedf5d9b700 1 -- 172.21.15.62:0/3834952290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 msgr2=0x55a9f8712760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:12.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.916+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3834952290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8712760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.916+0000 7fedf5d9b700 1 -- 172.21.15.62:0/3834952290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f8490e60 con 0x55a9f8711550 2023-07-14T16:19:12.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.916+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3834952290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711550 0x55a9f8711960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedec011cd0 tx=0x7fedec009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49663e1b095716b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:12.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.916+0000 7fedf6d9d700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fede00a11d0 0x7fede00a3680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009690 tx=0x7fedf001c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:12.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.916+0000 7fede4ff9700 1 -- 172.21.15.62:0/3834952290 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00f6d0 con 0x55a9f8711550 2023-07-14T16:19:12.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fede4ff9700 1 -- 172.21.15.62:0/3834952290 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedec00fcc0 con 0x55a9f8711550 2023-07-14T16:19:12.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fede4ff9700 1 -- 172.21.15.62:0/3834952290 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec01fb00 con 0x55a9f8711550 2023-07-14T16:19:12.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fee1076db80 1 -- 172.21.15.62:0/3834952290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711550 msgr2=0x55a9f8711960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fee1076db80 1 --2- 172.21.15.62:0/3834952290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711550 0x55a9f8711960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedec011cd0 tx=0x7fedec009c50 comp rx=0 tx=0).stop 2023-07-14T16:19:12.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fee1076db80 1 -- 172.21.15.62:0/3834952290 shutdown_connections 2023-07-14T16:19:12.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fee1076db80 1 --2- 172.21.15.62:0/3834952290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8712cf0 0x55a9f8713160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fee1076db80 1 --2- 172.21.15.62:0/3834952290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711550 0x55a9f8711960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fee1076db80 1 --2- 172.21.15.62:0/3834952290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8712760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.917+0000 7fee1076db80 1 -- 172.21.15.62:0/3834952290 >> 172.21.15.62:0/3834952290 conn(0x55a9f8706e20 msgr2=0x55a9f870c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:12.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.918+0000 7fee1076db80 1 -- 172.21.15.62:0/3834952290 shutdown_connections 2023-07-14T16:19:12.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.918+0000 7fee1076db80 1 -- 172.21.15.62:0/3834952290 wait complete. 2023-07-14T16:19:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.918+0000 7fee1076db80 1 -- start start 2023-07-14T16:19:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fee1076db80 1 Processor -- start 2023-07-14T16:19:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fee1076db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f871b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fee1076db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f871b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fee1076db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8722720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fee1076db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8602b40 con 0x55a9f8712cf0 2023-07-14T16:19:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fee1076db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8602840 con 0x55a9f8711550 2023-07-14T16:19:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fee1076db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f84e69a0 con 0x55a9f8711ea0 2023-07-14T16:19:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fee1076db80 10 client.0 _ll_register_callbacks cb 0x55a9f86007e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.919+0000 7fedf5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8722720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f871b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8722720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57806/0 (socket says 172.21.15.62:57806) 2023-07-14T16:19:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf5d9b700 1 -- 172.21.15.62:0/3432420508 learned_addr learned my addr 172.21.15.62:0/3432420508 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f871b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34154/0 (socket says 172.21.15.62:34154) 2023-07-14T16:19:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f871b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf659c700 1 -- 172.21.15.62:0/3432420508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 msgr2=0x55a9f871b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf659c700 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f871b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf659c700 1 -- 172.21.15.62:0/3432420508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 msgr2=0x55a9f8722720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf659c700 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8722720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.920+0000 7fedf659c700 1 -- 172.21.15.62:0/3432420508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedec019040 con 0x55a9f8711ea0 2023-07-14T16:19:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fedf659c700 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f871b6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fede800b2e0 tx=0x7fede80052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f5736913024a98c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fedddffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:19:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fedddffb700 1 -- 172.21.15.62:0/3432420508 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede801d070 con 0x55a9f8711ea0 2023-07-14T16:19:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fedddffb700 1 -- 172.21.15.62:0/3432420508 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fede8003b50 con 0x55a9f8711ea0 2023-07-14T16:19:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fee1076db80 1 -- 172.21.15.62:0/3432420508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9f85bc570 con 0x55a9f8711ea0 2023-07-14T16:19:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fedddffb700 1 -- 172.21.15.62:0/3432420508 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede8003d10 con 0x55a9f8711ea0 2023-07-14T16:19:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fee1076db80 10 client.26137 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fee1076db80 1 -- 172.21.15.62:0/3432420508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a9f8479ce0 con 0x55a9f8711ea0 2023-07-14T16:19:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.921+0000 7fedddffb700 1 -- 172.21.15.62:0/3432420508 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a9f85bc570 con 0x55a9f8711ea0 2023-07-14T16:19:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.922+0000 7fedddffb700 1 -- 172.21.15.62:0/3432420508 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fede8004870 con 0x55a9f8711ea0 2023-07-14T16:19:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.922+0000 7fee1076db80 10 client.26137 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.922+0000 7fee1076db80 10 client.26137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.922+0000 7fee1076db80 20 client.26137 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.922+0000 7fee1076db80 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a9f8726f90 0x55a9f87293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.922+0000 7fee1076db80 4 client.26137 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.922+0000 7fee1076db80 1 -- 172.21.15.62:0/3432420508 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9f865bf30 con 0x55a9f8726f90 2023-07-14T16:19:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.922+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a9f8726f90 0x55a9f87293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.923+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a9f8726f90 0x55a9f87293a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998584793364e0a4 server_cookie=f190a75fd97a4bac in_seq=0 out_seq=0 2023-07-14T16:19:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.923+0000 7fedddffb700 10 client.26137 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fedddffb700 1 -- 172.21.15.62:0/3432420508 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a9f865bf30 con 0x55a9f8726f90 2023-07-14T16:19:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fedddffb700 10 client.26137 handle_command_reply: tid=0 2023-07-14T16:19:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 client.26137 shutdown 2023-07-14T16:19:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 -- 172.21.15.62:0/3432420508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 msgr2=0x55a9f871b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f871b6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fede800b2e0 tx=0x7fede80052e0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 -- 172.21.15.62:0/3432420508 shutdown_connections 2023-07-14T16:19:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8722720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a9f8726f90 0x55a9f87293a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f871b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 --2- 172.21.15.62:0/3432420508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f871b6e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fedddffb700 0 client.26137 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 -- 172.21.15.62:0/3432420508 >> 172.21.15.62:0/3432420508 conn(0x55a9f8706e20 msgr2=0x55a9f870b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:12.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 -- 172.21.15.62:0/3432420508 shutdown_connections 2023-07-14T16:19:12.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.924+0000 7fee1076db80 1 -- 172.21.15.62:0/3432420508 wait complete. 2023-07-14T16:19:12.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.925+0000 7fee1076db80 20 client.26137 trim_cache size 0 max 16384 2023-07-14T16:19:12.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.942+0000 7fee1076db80 1 Processor -- start 2023-07-14T16:19:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fee1076db80 1 -- start start 2023-07-14T16:19:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fee1076db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f8715bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fee1076db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8716130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fee1076db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8712cf0 0x55a9f87226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fee1076db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f871a650 con 0x55a9f8711ea0 2023-07-14T16:19:12.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fee1076db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f871ad10 con 0x55a9f8711550 2023-07-14T16:19:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fee1076db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f870cfa0 con 0x55a9f8712cf0 2023-07-14T16:19:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fedf659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8716130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fedf659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8716130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57820/0 (socket says 172.21.15.62:57820) 2023-07-14T16:19:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fedf659c700 1 -- 172.21.15.62:0/3634671751 learned_addr learned my addr 172.21.15.62:0/3634671751 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.943+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f8715bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8712cf0 0x55a9f87226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fedf659c700 1 -- 172.21.15.62:0/3634671751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8712cf0 msgr2=0x55a9f87226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fedf659c700 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8712cf0 0x55a9f87226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fedf659c700 1 -- 172.21.15.62:0/3634671751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 msgr2=0x55a9f8715bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fedf659c700 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f8715bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fedf659c700 1 -- 172.21.15.62:0/3634671751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f84b72d0 con 0x55a9f8711ea0 2023-07-14T16:19:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f8715bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fedf659c700 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8716130 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fede8000c00 tx=0x7fede8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ff22c3748cdfe9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fede57fa700 1 -- 172.21.15.62:0/3634671751 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede801d070 con 0x55a9f8711ea0 2023-07-14T16:19:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fede57fa700 1 -- 172.21.15.62:0/3634671751 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fede801e040 con 0x55a9f8711ea0 2023-07-14T16:19:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.944+0000 7fede57fa700 1 -- 172.21.15.62:0/3634671751 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8004760 con 0x55a9f8711ea0 2023-07-14T16:19:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 -- 172.21.15.62:0/3634671751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 msgr2=0x55a9f8716130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8716130 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fede8000c00 tx=0x7fede8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 -- 172.21.15.62:0/3634671751 shutdown_connections 2023-07-14T16:19:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8712cf0 0x55a9f87226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f8715bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 --2- 172.21.15.62:0/3634671751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8711ea0 0x55a9f8716130 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 -- 172.21.15.62:0/3634671751 >> 172.21.15.62:0/3634671751 conn(0x55a9f8706e20 msgr2=0x55a9f871b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 -- 172.21.15.62:0/3634671751 shutdown_connections 2023-07-14T16:19:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.945+0000 7fee1076db80 1 -- 172.21.15.62:0/3634671751 wait complete. 2023-07-14T16:19:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.946+0000 7fee1076db80 1 -- start start 2023-07-14T16:19:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.946+0000 7fee1076db80 1 Processor -- start 2023-07-14T16:19:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.946+0000 7fee1076db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f871fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.946+0000 7fee1076db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f8720040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.946+0000 7fee1076db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8721050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.946+0000 7fee1076db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8721590 con 0x55a9f8712cf0 2023-07-14T16:19:12.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.946+0000 7fee1076db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8721700 con 0x55a9f8711550 2023-07-14T16:19:12.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.946+0000 7fee1076db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f87218c0 con 0x55a9f8711ea0 2023-07-14T16:19:12.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fee1076db80 10 client.0 _ll_register_callbacks cb 0x55a9f86007e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:12.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f871fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8721050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f8720040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8721050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57826/0 (socket says 172.21.15.62:57826) 2023-07-14T16:19:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf5d9b700 1 -- 172.21.15.62:0/329554457 learned_addr learned my addr 172.21.15.62:0/329554457 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf659c700 1 -- 172.21.15.62:0/329554457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 msgr2=0x55a9f871fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf659c700 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f871fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf659c700 1 -- 172.21.15.62:0/329554457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 msgr2=0x55a9f8721050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf659c700 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8721050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf659c700 1 -- 172.21.15.62:0/329554457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9f8490e60 con 0x55a9f8711ea0 2023-07-14T16:19:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.947+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8721050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.948+0000 7fedf659c700 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f8720040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fede8004bd0 tx=0x7fede8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670de389e57515d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.948+0000 7fedddffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:19:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.948+0000 7fedddffb700 1 -- 172.21.15.62:0/329554457 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede801d040 con 0x55a9f8711ea0 2023-07-14T16:19:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.948+0000 7fedddffb700 1 -- 172.21.15.62:0/329554457 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fede8000e40 con 0x55a9f8711ea0 2023-07-14T16:19:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.948+0000 7fee1076db80 1 -- 172.21.15.62:0/329554457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedec00fcc0 con 0x55a9f8711ea0 2023-07-14T16:19:12.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.948+0000 7fedddffb700 1 -- 172.21.15.62:0/329554457 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede80056e0 con 0x55a9f8711ea0 2023-07-14T16:19:12.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.948+0000 7fee1076db80 10 client.26143 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:12.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.948+0000 7fee1076db80 1 -- 172.21.15.62:0/329554457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a9f8479ce0 con 0x55a9f8711ea0 2023-07-14T16:19:12.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fedddffb700 1 -- 172.21.15.62:0/329554457 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedec00fcc0 con 0x55a9f8711ea0 2023-07-14T16:19:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fedddffb700 1 -- 172.21.15.62:0/329554457 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fede800b7b0 con 0x55a9f8711ea0 2023-07-14T16:19:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fee1076db80 10 client.26143 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fee1076db80 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fee1076db80 20 client.26143 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fee1076db80 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a9f877d060 0x55a9f877f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fee1076db80 4 client.26143 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:19:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fee1076db80 1 -- 172.21.15.62:0/329554457 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a9f861e0d0 con 0x55a9f877d060 2023-07-14T16:19:12.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.949+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a9f877d060 0x55a9f877f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:12.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.950+0000 7fedf5d9b700 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a9f877d060 0x55a9f877f440 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e99f22edf95ccb server_cookie=642bb797539705a4 in_seq=0 out_seq=0 2023-07-14T16:19:12.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.950+0000 7fedddffb700 10 client.26143 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:12.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fedddffb700 1 -- 172.21.15.62:0/329554457 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a9f861e0d0 con 0x55a9f877d060 2023-07-14T16:19:12.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fedddffb700 10 client.26143 handle_command_reply: tid=0 2023-07-14T16:19:12.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 client.26143 shutdown 2023-07-14T16:19:12.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 -- 172.21.15.62:0/329554457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 msgr2=0x55a9f8720040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f8720040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fede8004bd0 tx=0x7fede8004730 comp rx=0 tx=0).stop 2023-07-14T16:19:12.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 -- 172.21.15.62:0/329554457 shutdown_connections 2023-07-14T16:19:12.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a9f8712cf0 0x55a9f8721050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a9f877d060 0x55a9f877f440 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a9f8711550 0x55a9f871fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 --2- 172.21.15.62:0/329554457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a9f8711ea0 0x55a9f8720040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fedddffb700 0 client.26143 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:12.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 -- 172.21.15.62:0/329554457 >> 172.21.15.62:0/329554457 conn(0x55a9f8706e20 msgr2=0x55a9f871b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:12.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 -- 172.21.15.62:0/329554457 shutdown_connections 2023-07-14T16:19:12.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.951+0000 7fee1076db80 1 -- 172.21.15.62:0/329554457 wait complete. 2023-07-14T16:19:12.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fee1076db80 20 client.26143 trim_cache size 0 max 16384 2023-07-14T16:19:12.939 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:19:12.939 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:19:12.939 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "902b84b5-f53e-4d7c-b2ee-98e2e012878b", 2023-07-14T16:19:12.939 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:19:12.940 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:19:12.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 -- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fede00a11d0 msgr2=0x7fede00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fede00a11d0 0x7fede00a3680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009690 tx=0x7fedf001c000 comp rx=0 tx=0).stop 2023-07-14T16:19:12.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 -- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf81057a0 msgr2=0x7fedf819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:12.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf81057a0 0x7fedf819efa0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fedf000c370 tx=0x7fedf000c680 comp rx=0 tx=0).stop 2023-07-14T16:19:12.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 -- 172.21.15.62:0/3510283636 shutdown_connections 2023-07-14T16:19:12.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedf81057a0 0x7fedf819efa0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedf8100560 0x7fedf81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fede00a11d0 0x7fede00a3680 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 --2- 172.21.15.62:0/3510283636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedf8102e80 0x7fedf8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:12.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.952+0000 7fedfcdff700 1 -- 172.21.15.62:0/3510283636 >> 172.21.15.62:0/3510283636 conn(0x7fedf80fa000 msgr2=0x7fedf81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:12.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.953+0000 7fedfcdff700 1 -- 172.21.15.62:0/3510283636 shutdown_connections 2023-07-14T16:19:12.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:12.953+0000 7fedfcdff700 1 -- 172.21.15.62:0/3510283636 wait complete. 2023-07-14T16:19:12.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:19:13.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:12 smithi062 ceph-mon[95543]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:19:13.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:12 smithi164 ceph-mon[97427]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:19:13.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:12 smithi081 ceph-mon[127795]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:19:13.154 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:19:12 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:19:12.959+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v109>,<0x10000000003/client.0 v110>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v122>,<0x10000000001/sv_0 v136>,<0x10000000000/_nogroup v130>,<0x1/volumes v120>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:19:13.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 -- 172.21.15.62:0/2758037527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf40744f0 msgr2=0x7febf4072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 --2- 172.21.15.62:0/2758037527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf40744f0 0x7febf4072940 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7febe8009990 tx=0x7febe8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 -- 172.21.15.62:0/2758037527 shutdown_connections 2023-07-14T16:19:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 --2- 172.21.15.62:0/2758037527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febf4107980 0x7febf4109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 --2- 172.21.15.62:0/2758037527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf40744f0 0x7febf4072940 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 --2- 172.21.15.62:0/2758037527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf4072f10 0x7febf4073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 -- 172.21.15.62:0/2758037527 >> 172.21.15.62:0/2758037527 conn(0x7febf40fb130 msgr2=0x7febf40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 -- 172.21.15.62:0/2758037527 shutdown_connections 2023-07-14T16:19:13.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.211+0000 7febfabff700 1 -- 172.21.15.62:0/2758037527 wait complete. 2023-07-14T16:19:13.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.212+0000 7febfabff700 1 Processor -- start 2023-07-14T16:19:13.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febfabff700 1 -- start start 2023-07-14T16:19:13.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febfabff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4072f10 0x7febf41979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febfabff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febf40744f0 0x7febf4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febfabff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf4107980 0x7febf419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febfabff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf419f520 con 0x7febf4072f10 2023-07-14T16:19:13.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febfabff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf419f660 con 0x7febf4107980 2023-07-14T16:19:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febfabff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf419f7f0 con 0x7febf40744f0 2023-07-14T16:19:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febf899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4072f10 0x7febf41979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febf899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4072f10 0x7febf41979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57834/0 (socket says 172.21.15.62:57834) 2023-07-14T16:19:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febf899b700 1 -- 172.21.15.62:0/4087024547 learned_addr learned my addr 172.21.15.62:0/4087024547 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.213+0000 7febf919c700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf4107980 0x7febf419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.214+0000 7febf3fff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febf40744f0 0x7febf4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.214+0000 7febf899b700 1 -- 172.21.15.62:0/4087024547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febf40744f0 msgr2=0x7febf4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.214+0000 7febf899b700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febf40744f0 0x7febf4197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.214+0000 7febf899b700 1 -- 172.21.15.62:0/4087024547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf4107980 msgr2=0x7febf419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.214+0000 7febf899b700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf4107980 0x7febf419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.214+0000 7febf899b700 1 -- 172.21.15.62:0/4087024547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80095f0 con 0x7febf4072f10 2023-07-14T16:19:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.214+0000 7febf899b700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4072f10 0x7febf41979a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7febe8005910 tx=0x7febe8005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.214+0000 7febf1ffb700 1 -- 172.21.15.62:0/4087024547 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe801d070 con 0x7febf4072f10 2023-07-14T16:19:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.215+0000 7febfabff700 1 -- 172.21.15.62:0/4087024547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf419fa70 con 0x7febf4072f10 2023-07-14T16:19:13.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.215+0000 7febf1ffb700 1 -- 172.21.15.62:0/4087024547 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe8022470 con 0x7febf4072f10 2023-07-14T16:19:13.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.215+0000 7febf1ffb700 1 -- 172.21.15.62:0/4087024547 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8022ea0 con 0x7febf4072f10 2023-07-14T16:19:13.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.215+0000 7febfabff700 1 -- 172.21.15.62:0/4087024547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf419fe90 con 0x7febf4072f10 2023-07-14T16:19:13.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.218+0000 7febf1ffb700 1 -- 172.21.15.62:0/4087024547 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7febe800ba60 con 0x7febf4072f10 2023-07-14T16:19:13.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7fec0e52ab80 1 Processor -- start 2023-07-14T16:19:13.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7fec0e52ab80 1 -- start start 2023-07-14T16:19:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb512f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb51306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7febf1ffb700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febe40a1150 0x7febe40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb5130c70 0x561fb51310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb5034cb0 con 0x561fb512fe20 2023-07-14T16:19:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb5087a40 con 0x561fb5130c70 2023-07-14T16:19:13.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.220+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb5080ed0 con 0x561fb512f4d0 2023-07-14T16:19:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf1ffb700 1 -- 172.21.15.62:0/4087024547 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7febe80cdc50 con 0x7febf4072f10 2023-07-14T16:19:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb51306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb51306e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57840/0 (socket says 172.21.15.62:57840) 2023-07-14T16:19:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf899b700 1 -- 172.21.15.62:0/259425551 learned_addr learned my addr 172.21.15.62:0/259425551 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf919c700 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb512f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf919c700 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb5130c70 0x561fb51310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf899b700 1 -- 172.21.15.62:0/259425551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 msgr2=0x561fb512f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:13.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf899b700 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb512f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf899b700 1 -- 172.21.15.62:0/259425551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb5130c70 msgr2=0x561fb51310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:13.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf899b700 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb5130c70 0x561fb51310e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf3fff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febe40a1150 0x7febe40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.221+0000 7febf899b700 1 -- 172.21.15.62:0/259425551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fb4eaede0 con 0x561fb512fe20 2023-07-14T16:19:13.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.222+0000 7febf899b700 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb51306e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7febe809b030 tx=0x7febe8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb2374d3dd47a3e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:13.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.222+0000 7febdeffd700 1 -- 172.21.15.62:0/259425551 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8022800 con 0x561fb512fe20 2023-07-14T16:19:13.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.222+0000 7febdeffd700 1 -- 172.21.15.62:0/259425551 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe809b0a0 con 0x561fb512fe20 2023-07-14T16:19:13.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.222+0000 7febdeffd700 1 -- 172.21.15.62:0/259425551 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe809b300 con 0x561fb512fe20 2023-07-14T16:19:13.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 -- 172.21.15.62:0/259425551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 msgr2=0x561fb51306e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7febf3fff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febe40a1150 0x7febe40a3600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7febe0009c60 tx=0x7febe0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb51306e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7febe809b030 tx=0x7febe8000bc0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 -- 172.21.15.62:0/259425551 shutdown_connections 2023-07-14T16:19:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb5130c70 0x561fb51310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb512f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/259425551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb51306e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 -- 172.21.15.62:0/259425551 >> 172.21.15.62:0/259425551 conn(0x561fb5124da0 msgr2=0x561fb512a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:13.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 -- 172.21.15.62:0/259425551 shutdown_connections 2023-07-14T16:19:13.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.223+0000 7fec0e52ab80 1 -- 172.21.15.62:0/259425551 wait complete. 2023-07-14T16:19:13.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.224+0000 7fec0e52ab80 1 -- start start 2023-07-14T16:19:13.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.224+0000 7fec0e52ab80 1 Processor -- start 2023-07-14T16:19:13.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.224+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5138e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.224+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5139340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.224+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5130c70 0x561fb5140380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb5020ac0 con 0x561fb512fe20 2023-07-14T16:19:13.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb50207c0 con 0x561fb512f4d0 2023-07-14T16:19:13.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4f04920 con 0x561fb5130c70 2023-07-14T16:19:13.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7fec0e52ab80 10 client.0 _ll_register_callbacks cb 0x561fb501e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:13.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57842/0 (socket says 172.21.15.62:57842) 2023-07-14T16:19:13.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf3fff700 1 -- 172.21.15.62:0/533716532 learned_addr learned my addr 172.21.15.62:0/533716532 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:13.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf899b700 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5138e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf899b700 1 -- 172.21.15.62:0/533716532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5130c70 msgr2=0x561fb5140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:13.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf899b700 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5130c70 0x561fb5140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf899b700 1 -- 172.21.15.62:0/533716532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 msgr2=0x561fb5139340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:13.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf899b700 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5139340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf899b700 1 -- 172.21.15.62:0/533716532 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf419fe90 con 0x561fb512f4d0 2023-07-14T16:19:13.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.225+0000 7febf3fff700 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:13.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7febf899b700 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5138e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7febe8009350 tx=0x7febe8005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859e6b3330a86a58 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:13.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7febd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:19:13.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7febd3fff700 1 -- 172.21.15.62:0/533716532 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe80b9070 con 0x561fb512f4d0 2023-07-14T16:19:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7fec0e52ab80 1 -- 172.21.15.62:0/533716532 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fb4fda4f0 con 0x561fb512f4d0 2023-07-14T16:19:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7febd3fff700 1 -- 172.21.15.62:0/533716532 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7febe809f0d0 con 0x561fb512f4d0 2023-07-14T16:19:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7febd3fff700 1 -- 172.21.15.62:0/533716532 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe80225e0 con 0x561fb512f4d0 2023-07-14T16:19:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7fec0e52ab80 10 client.26243 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7fec0e52ab80 1 -- 172.21.15.62:0/533716532 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561fb4e97c60 con 0x561fb512f4d0 2023-07-14T16:19:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.226+0000 7febd3fff700 1 -- 172.21.15.62:0/533716532 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe807d020 con 0x561fb512f4d0 2023-07-14T16:19:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.227+0000 7febd3fff700 1 -- 172.21.15.62:0/533716532 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7febe80a3060 con 0x561fb512f4d0 2023-07-14T16:19:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.227+0000 7fec0e52ab80 10 client.26243 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.227+0000 7fec0e52ab80 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.227+0000 7fec0e52ab80 20 client.26243 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.227+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561fb5144640 0x561fb5146a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.227+0000 7fec0e52ab80 4 client.26243 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.227+0000 7fec0e52ab80 1 -- 172.21.15.62:0/533716532 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fb5079eb0 con 0x561fb5144640 2023-07-14T16:19:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.228+0000 7febf3fff700 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561fb5144640 0x561fb5146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.228+0000 7febf3fff700 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561fb5144640 0x561fb5146a20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55d1ada71ab5011 server_cookie=62d4ca92f52d751d in_seq=0 out_seq=0 2023-07-14T16:19:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.228+0000 7febd3fff700 10 client.26243 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7febd3fff700 1 -- 172.21.15.62:0/533716532 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561fb5079eb0 con 0x561fb5144640 2023-07-14T16:19:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7febd3fff700 10 client.26243 handle_command_reply: tid=0 2023-07-14T16:19:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 client.26243 shutdown 2023-07-14T16:19:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 -- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 msgr2=0x561fb5138e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5138e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7febe8009350 tx=0x7febe8005a00 comp rx=0 tx=0).stop 2023-07-14T16:19:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 -- 172.21.15.62:0/533716532 shutdown_connections 2023-07-14T16:19:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5130c70 0x561fb5140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561fb5144640 0x561fb5146a20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5138e00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/533716532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7febd3fff700 0 client.26243 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 -- 172.21.15.62:0/533716532 >> 172.21.15.62:0/533716532 conn(0x561fb5124da0 msgr2=0x561fb51298b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 -- 172.21.15.62:0/533716532 shutdown_connections 2023-07-14T16:19:13.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.229+0000 7fec0e52ab80 1 -- 172.21.15.62:0/533716532 wait complete. 2023-07-14T16:19:13.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.230+0000 7fec0e52ab80 20 client.26243 trim_cache size 0 max 16384 2023-07-14T16:19:13.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.253+0000 7fec0e52ab80 1 Processor -- start 2023-07-14T16:19:13.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.253+0000 7fec0e52ab80 1 -- start start 2023-07-14T16:19:13.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5124da0 0x561fb5140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5136f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb5130570 con 0x561fb512fe20 2023-07-14T16:19:13.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb512b680 con 0x561fb512f4d0 2023-07-14T16:19:13.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb5139ff0 con 0x561fb5124da0 2023-07-14T16:19:13.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf919c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf919c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5136f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57856/0 (socket says 172.21.15.62:57856) 2023-07-14T16:19:13.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf919c700 1 -- 172.21.15.62:0/3412445639 learned_addr learned my addr 172.21.15.62:0/3412445639 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:13.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf3fff700 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf899b700 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5124da0 0x561fb5140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf919c700 1 -- 172.21.15.62:0/3412445639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5124da0 msgr2=0x561fb5140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:13.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf919c700 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5124da0 0x561fb5140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf919c700 1 -- 172.21.15.62:0/3412445639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 msgr2=0x561fb5140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:13.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf919c700 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.254+0000 7febf919c700 1 -- 172.21.15.62:0/3412445639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fb4ed5250 con 0x561fb512fe20 2023-07-14T16:19:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.255+0000 7febf919c700 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5136f70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7febec009940 tx=0x7febec009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe2603fa6b46702 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.255+0000 7febdf7fe700 1 -- 172.21.15.62:0/3412445639 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec007de0 con 0x561fb512fe20 2023-07-14T16:19:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.255+0000 7febdf7fe700 1 -- 172.21.15.62:0/3412445639 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febec00f070 con 0x561fb512fe20 2023-07-14T16:19:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.255+0000 7febdf7fe700 1 -- 172.21.15.62:0/3412445639 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec004870 con 0x561fb512fe20 2023-07-14T16:19:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 -- 172.21.15.62:0/3412445639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 msgr2=0x561fb5136f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5136f70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7febec009940 tx=0x7febec009c30 comp rx=0 tx=0).stop 2023-07-14T16:19:13.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 -- 172.21.15.62:0/3412445639 shutdown_connections 2023-07-14T16:19:13.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb512fe20 0x561fb5136f70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb5124da0 0x561fb5140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/3412445639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512f4d0 0x561fb5140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 -- 172.21.15.62:0/3412445639 >> 172.21.15.62:0/3412445639 conn(0x561fb5126600 msgr2=0x561fb5137b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 -- 172.21.15.62:0/3412445639 shutdown_connections 2023-07-14T16:19:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 -- 172.21.15.62:0/3412445639 wait complete. 2023-07-14T16:19:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.256+0000 7fec0e52ab80 1 -- start start 2023-07-14T16:19:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.257+0000 7fec0e52ab80 1 Processor -- start 2023-07-14T16:19:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.257+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb5124da0 0x561fb513fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.257+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb5140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.257+0000 7fec0e52ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512fe20 0x561fb513c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.257+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb513d090 con 0x561fb5124da0 2023-07-14T16:19:13.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.257+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb513d200 con 0x561fb512fe20 2023-07-14T16:19:13.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.257+0000 7fec0e52ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb513d3c0 con 0x561fb512f4d0 2023-07-14T16:19:13.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.257+0000 7fec0e52ab80 10 client.0 _ll_register_callbacks cb 0x7febec009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:13.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb5124da0 0x561fb513fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb5124da0 0x561fb513fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57866/0 (socket says 172.21.15.62:57866) 2023-07-14T16:19:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf899b700 1 -- 172.21.15.62:0/564391081 learned_addr learned my addr 172.21.15.62:0/564391081 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf3fff700 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb5140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf919c700 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512fe20 0x561fb513c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf3fff700 1 -- 172.21.15.62:0/564391081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512fe20 msgr2=0x561fb513c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf3fff700 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512fe20 0x561fb513c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf3fff700 1 -- 172.21.15.62:0/564391081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb5124da0 msgr2=0x561fb513fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf3fff700 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb5124da0 0x561fb513fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf3fff700 1 -- 172.21.15.62:0/564391081 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec011040 con 0x561fb512f4d0 2023-07-14T16:19:13.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.258+0000 7febf899b700 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb5124da0 0x561fb513fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:13.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.259+0000 7febf3fff700 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb5140350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x561fb5147000 tx=0x7febe00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be7316b471e5be38 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:13.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.259+0000 7febd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:19:13.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.259+0000 7febd3fff700 1 -- 172.21.15.62:0/564391081 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0022040 con 0x561fb512f4d0 2023-07-14T16:19:13.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.259+0000 7fec0e52ab80 1 -- 172.21.15.62:0/564391081 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febe809b0a0 con 0x561fb512f4d0 2023-07-14T16:19:13.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.259+0000 7febd3fff700 1 -- 172.21.15.62:0/564391081 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7febe0028dd0 con 0x561fb512f4d0 2023-07-14T16:19:13.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.259+0000 7febd3fff700 1 -- 172.21.15.62:0/564391081 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe001d410 con 0x561fb512f4d0 2023-07-14T16:19:13.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.259+0000 7fec0e52ab80 10 client.26158 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:13.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.259+0000 7fec0e52ab80 1 -- 172.21.15.62:0/564391081 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561fb4ead2b0 con 0x561fb512f4d0 2023-07-14T16:19:13.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7febd3fff700 1 -- 172.21.15.62:0/564391081 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe809b0a0 con 0x561fb512f4d0 2023-07-14T16:19:13.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7febd3fff700 1 -- 172.21.15.62:0/564391081 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7febe003c490 con 0x561fb512f4d0 2023-07-14T16:19:13.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7fec0e52ab80 10 client.26158 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:13.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7fec0e52ab80 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7fec0e52ab80 20 client.26158 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561fb5130c70 0x561fb519bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7fec0e52ab80 4 client.26158 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:19:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7fec0e52ab80 1 -- 172.21.15.62:0/564391081 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561fb503c050 con 0x561fb5130c70 2023-07-14T16:19:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.260+0000 7febf899b700 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561fb5130c70 0x561fb519bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7febf899b700 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561fb5130c70 0x561fb519bc30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe253fe5067a7e05 server_cookie=a67b9f38e238a7ca in_seq=0 out_seq=0 2023-07-14T16:19:13.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7febd3fff700 10 client.26158 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7febd3fff700 1 -- 172.21.15.62:0/564391081 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561fb503c050 con 0x561fb5130c70 2023-07-14T16:19:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7febd3fff700 10 client.26158 handle_command_reply: tid=0 2023-07-14T16:19:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 client.26158 shutdown 2023-07-14T16:19:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 -- 172.21.15.62:0/564391081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 msgr2=0x561fb5140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb5140350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x561fb5147000 tx=0x7febe00047c0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 -- 172.21.15.62:0/564391081 shutdown_connections 2023-07-14T16:19:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561fb5130c70 0x561fb519bc30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561fb512fe20 0x561fb513c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fb5124da0 0x561fb513fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7febd3fff700 0 client.26158 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 --2- 172.21.15.62:0/564391081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561fb512f4d0 0x561fb5140350 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 -- 172.21.15.62:0/564391081 >> 172.21.15.62:0/564391081 conn(0x561fb5126600 msgr2=0x561fb5140bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 -- 172.21.15.62:0/564391081 shutdown_connections 2023-07-14T16:19:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.261+0000 7fec0e52ab80 1 -- 172.21.15.62:0/564391081 wait complete. 2023-07-14T16:19:13.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7fec0e52ab80 20 client.26158 trim_cache size 0 max 16384 2023-07-14T16:19:13.247 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:19:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:19:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:19:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "902b84b5-f53e-4d7c-b2ee-98e2e012878b": { 2023-07-14T16:19:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:19:13.247 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "902b84b5-f53e-4d7c-b2ee-98e2e012878b", 2023-07-14T16:19:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:19:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:19:13.248 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:19:13.248 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:19:13.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 -- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febe40a1150 msgr2=0x7febe40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febe40a1150 0x7febe40a3600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7febe0009c60 tx=0x7febe0009260 comp rx=0 tx=0).stop 2023-07-14T16:19:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 -- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4072f10 msgr2=0x7febf41979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4072f10 0x7febf41979a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7febe8005910 tx=0x7febe8005720 comp rx=0 tx=0).stop 2023-07-14T16:19:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 -- 172.21.15.62:0/4087024547 shutdown_connections 2023-07-14T16:19:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febe40a1150 0x7febe40a3600 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf4107980 0x7febf419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febf4072f10 0x7febf41979a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 --2- 172.21.15.62:0/4087024547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febf40744f0 0x7febf4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 -- 172.21.15.62:0/4087024547 >> 172.21.15.62:0/4087024547 conn(0x7febf40fb130 msgr2=0x7febf40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.262+0000 7febfabff700 1 -- 172.21.15.62:0/4087024547 shutdown_connections 2023-07-14T16:19:13.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:13.263+0000 7febfabff700 1 -- 172.21.15.62:0/4087024547 wait complete. 2023-07-14T16:19:13.260 INFO:tasks.cephfs.filesystem:scrub status for tag:902b84b5-f53e-4d7c-b2ee-98e2e012878b - {'path': '/', 'tag': '902b84b5-f53e-4d7c-b2ee-98e2e012878b', 'options': 'recursive,force'} 2023-07-14T16:19:14.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:19:14.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:14.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:14.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:14.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:19:14.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:14.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:14.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:13 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:19:14.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:14.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:14.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:14.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:19:14.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:14.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:14.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:13 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:19:14.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:14.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:14.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:14.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:19:14.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:14.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:14.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:14.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:13 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:14 smithi164 ceph-mon[97427]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 23 op/s 2023-07-14T16:19:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:14 smithi062 ceph-mon[95543]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 23 op/s 2023-07-14T16:19:15.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:14 smithi081 ceph-mon[127795]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 23 op/s 2023-07-14T16:19:17.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:16 smithi164 ceph-mon[97427]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:19:17.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:16 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:19:17.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:16 smithi062 ceph-mon[95543]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:19:17.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:16 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:19:17.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:16 smithi081 ceph-mon[127795]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:19:17.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:16 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:19:18.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:19:18.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:19:18.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:19:18.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:19:18.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:19:18.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:19:18.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:19:18.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:19:18.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:19:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:18 smithi164 ceph-mon[97427]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:19:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:18 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:19:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:18 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:19:19.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:18 smithi062 ceph-mon[95543]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:19:19.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:18 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:19:19.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:18 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:19:19.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:18 smithi081 ceph-mon[127795]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:19:19.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:18 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:19:19.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:18 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:19:21.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:20 smithi164 ceph-mon[97427]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 38 op/s 2023-07-14T16:19:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:20 smithi062 ceph-mon[95543]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 38 op/s 2023-07-14T16:19:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:20 smithi081 ceph-mon[127795]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 38 op/s 2023-07-14T16:19:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/4223449367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf4102d50 msgr2=0x7fbaf41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/4223449367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf4102d50 0x7fbaf41031a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009990 tx=0x7fbae8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/4223449367 shutdown_connections 2023-07-14T16:19:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/4223449367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaf41036e0 0x7fbaf4109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/4223449367 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaf4101b60 0x7fbaf4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/4223449367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf4102d50 0x7fbaf41031a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/4223449367 >> 172.21.15.62:0/4223449367 conn(0x7fbaf40fd180 msgr2=0x7fbaf40ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/4223449367 shutdown_connections 2023-07-14T16:19:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.363+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/4223449367 wait complete. 2023-07-14T16:19:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.365+0000 7fbaf8b9d700 1 Processor -- start 2023-07-14T16:19:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.365+0000 7fbaf8b9d700 1 -- start start 2023-07-14T16:19:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.365+0000 7fbaf8b9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaf4101b60 0x7fbaf41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.365+0000 7fbaf8b9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaf4102d50 0x7fbaf4197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.365+0000 7fbaf8b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf41036e0 0x7fbaf419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.365+0000 7fbaf8b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf419f350 con 0x7fbaf41036e0 2023-07-14T16:19:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.365+0000 7fbaf8b9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf419f490 con 0x7fbaf4101b60 2023-07-14T16:19:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.365+0000 7fbaf8b9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf419f620 con 0x7fbaf4102d50 2023-07-14T16:19:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaf4101b60 0x7fbaf41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaf4101b60 0x7fbaf41977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:56520/0 (socket says 172.21.15.62:56520) 2023-07-14T16:19:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 -- 172.21.15.62:0/890745276 learned_addr learned my addr 172.21.15.62:0/890745276 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf3fff700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf41036e0 0x7fbaf419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaf4102d50 0x7fbaf4197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 -- 172.21.15.62:0/890745276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaf4102d50 msgr2=0x7fbaf4197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaf4102d50 0x7fbaf4197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 -- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf41036e0 msgr2=0x7fbaf419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf41036e0 0x7fbaf419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 -- 172.21.15.62:0/890745276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae80095f0 con 0x7fbaf4101b60 2023-07-14T16:19:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.366+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaf4101b60 0x7fbaf41977d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbae400e970 tx=0x7fbae400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.367+0000 7fbaf0ff9700 1 -- 172.21.15.62:0/890745276 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4009790 con 0x7fbaf4101b60 2023-07-14T16:19:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.367+0000 7fbaf0ff9700 1 -- 172.21.15.62:0/890745276 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbae400c7e0 con 0x7fbaf4101b60 2023-07-14T16:19:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.367+0000 7fbaf0ff9700 1 -- 172.21.15.62:0/890745276 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4003bd0 con 0x7fbaf4101b60 2023-07-14T16:19:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.367+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/890745276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf419f900 con 0x7fbaf4101b60 2023-07-14T16:19:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.367+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/890745276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf419fe10 con 0x7fbaf4101b60 2023-07-14T16:19:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.370+0000 7fbaf0ff9700 1 -- 172.21.15.62:0/890745276 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbae400c950 con 0x7fbaf4101b60 2023-07-14T16:19:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.372+0000 7fbb0d7c7b80 1 Processor -- start 2023-07-14T16:19:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.372+0000 7fbb0d7c7b80 1 -- start start 2023-07-14T16:19:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.372+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf93434550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.372+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf934352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93435c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf93339c50 con 0x55bf932906b0 2023-07-14T16:19:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9338c9e0 con 0x55bf93434a90 2023-07-14T16:19:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf93385e70 con 0x55bf934357e0 2023-07-14T16:19:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbaf0ff9700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbadc0a11d0 0x7fbadc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf93434550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbaf0ff9700 1 -- 172.21.15.62:0/890745276 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbae4014070 con 0x7fbaf4101b60 2023-07-14T16:19:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf934352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf934352a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:56532/0 (socket says 172.21.15.62:56532) 2023-07-14T16:19:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1526563169 learned_addr learned my addr 172.21.15.62:0/1526563169 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbaf3fff700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbadc0a11d0 0x7fbadc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.373+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93435c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1526563169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 msgr2=0x55bf93435c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93435c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1526563169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 msgr2=0x55bf93434550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf93434550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1526563169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf931b3d80 con 0x55bf93434a90 2023-07-14T16:19:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbaf3fff700 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf93434550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf934352a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbae40a1da0 tx=0x7fbae4099860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f49d0ded176293 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbaf3fff700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbadc0a11d0 0x7fbadc0a3680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55bf934342b0 tx=0x7fbaec004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbad9ffb700 1 -- 172.21.15.62:0/1526563169 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae409deb0 con 0x55bf93434a90 2023-07-14T16:19:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbad9ffb700 1 -- 172.21.15.62:0/1526563169 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbae40a22d0 con 0x55bf93434a90 2023-07-14T16:19:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.374+0000 7fbad9ffb700 1 -- 172.21.15.62:0/1526563169 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae409a3b0 con 0x55bf93434a90 2023-07-14T16:19:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.375+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1526563169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 msgr2=0x55bf934352a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.375+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf934352a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbae40a1da0 tx=0x7fbae4099860 comp rx=0 tx=0).stop 2023-07-14T16:19:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.375+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1526563169 shutdown_connections 2023-07-14T16:19:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.375+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93435c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.375+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf93434550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.375+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1526563169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf934352a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.375+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1526563169 >> 172.21.15.62:0/1526563169 conn(0x55bf93429d40 msgr2=0x55bf9342ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.375+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1526563169 shutdown_connections 2023-07-14T16:19:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.376+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1526563169 wait complete. 2023-07-14T16:19:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.376+0000 7fbb0d7c7b80 1 -- start start 2023-07-14T16:19:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.376+0000 7fbb0d7c7b80 1 Processor -- start 2023-07-14T16:19:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf9343db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf9343e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93445080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf93325a60 con 0x55bf932906b0 2023-07-14T16:19:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf93325760 con 0x55bf93434a90 2023-07-14T16:19:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf932098c0 con 0x55bf934357e0 2023-07-14T16:19:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbb0d7c7b80 10 client.0 _ll_register_callbacks cb 0x55bf93323700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf9343db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf9343db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57904/0 (socket says 172.21.15.62:57904) 2023-07-14T16:19:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1877069726 learned_addr learned my addr 172.21.15.62:0/1877069726 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.377+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf9343e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbaf3fff700 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93445080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1877069726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 msgr2=0x55bf93445080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93445080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1877069726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 msgr2=0x55bf9343e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf9343e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1877069726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec020040 con 0x55bf932906b0 2023-07-14T16:19:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf9343db00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbae4099e10 tx=0x7fbae40ccb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0d37485f1d797a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbad2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbad2ffd700 1 -- 172.21.15.62:0/1877069726 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae409deb0 con 0x55bf932906b0 2023-07-14T16:19:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbad2ffd700 1 -- 172.21.15.62:0/1877069726 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbae4099580 con 0x55bf932906b0 2023-07-14T16:19:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.378+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1877069726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf932df490 con 0x55bf932906b0 2023-07-14T16:19:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbad2ffd700 1 -- 172.21.15.62:0/1877069726 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbae40b8080 con 0x55bf932906b0 2023-07-14T16:19:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbb0d7c7b80 10 client.16536 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1877069726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fbaf419fe10 con 0x55bf932906b0 2023-07-14T16:19:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbad2ffd700 1 -- 172.21.15.62:0/1877069726 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bf932df490 con 0x55bf932906b0 2023-07-14T16:19:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbad2ffd700 1 -- 172.21.15.62:0/1877069726 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbae40ca120 con 0x55bf932906b0 2023-07-14T16:19:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbb0d7c7b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbb0d7c7b80 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbb0d7c7b80 20 client.16536 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bf93449480 0x55bf9344b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbb0d7c7b80 4 client.16536 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.379+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1877069726 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf9337ee50 con 0x55bf93449480 2023-07-14T16:19:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.380+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bf93449480 0x55bf9344b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.380+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bf93449480 0x55bf9344b890 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbce949b7fc55548 server_cookie=cee1f2cbec91f7ab in_seq=0 out_seq=0 2023-07-14T16:19:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.380+0000 7fbad2ffd700 10 client.16536 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.381+0000 7fbad2ffd700 1 -- 172.21.15.62:0/1877069726 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bf9337ee50 con 0x55bf93449480 2023-07-14T16:19:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.381+0000 7fbad2ffd700 10 client.16536 handle_command_reply: tid=0 2023-07-14T16:19:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.381+0000 7fbb0d7c7b80 1 client.16536 shutdown 2023-07-14T16:19:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.381+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1877069726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 msgr2=0x55bf9343db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.381+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf9343db00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbae4099e10 tx=0x7fbae40ccb40 comp rx=0 tx=0).stop 2023-07-14T16:19:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1877069726 shutdown_connections 2023-07-14T16:19:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93445080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf932906b0 0x55bf9343db00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bf93449480 0x55bf9344b890 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1877069726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf9343e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbad2ffd700 0 client.16536 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1877069726 >> 172.21.15.62:0/1877069726 conn(0x55bf93429d40 msgr2=0x55bf9342e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1877069726 shutdown_connections 2023-07-14T16:19:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1877069726 wait complete. 2023-07-14T16:19:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.382+0000 7fbb0d7c7b80 20 client.16536 trim_cache size 0 max 16384 2023-07-14T16:19:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.399+0000 7fbb0d7c7b80 1 Processor -- start 2023-07-14T16:19:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.399+0000 7fbb0d7c7b80 1 -- start start 2023-07-14T16:19:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.399+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf932906b0 0x55bf9343c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.399+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf93445040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.399+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf934357e0 0x55bf93438730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.399+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9343d6d0 con 0x55bf934357e0 2023-07-14T16:19:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.399+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9342be60 con 0x55bf93434a90 2023-07-14T16:19:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf93426d80 con 0x55bf932906b0 2023-07-14T16:19:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf934357e0 0x55bf93438730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf934357e0 0x55bf93438730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57918/0 (socket says 172.21.15.62:57918) 2023-07-14T16:19:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf932906b0 0x55bf9343c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf3fff700 1 -- 172.21.15.62:0/1616487646 learned_addr learned my addr 172.21.15.62:0/1616487646 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf93445040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1616487646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 msgr2=0x55bf93445040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf93445040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1616487646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf934357e0 msgr2=0x55bf93438730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf934357e0 0x55bf93438730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf37fe700 1 -- 172.21.15.62:0/1616487646 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf9319cc00 con 0x55bf932906b0 2023-07-14T16:19:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.400+0000 7fbaf3fff700 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf934357e0 0x55bf93438730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.401+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf932906b0 0x55bf9343c790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbae400cc00 tx=0x7fbae4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=314f753fe9f85a61 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.401+0000 7fbada7fc700 1 -- 172.21.15.62:0/1616487646 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae40ade90 con 0x55bf932906b0 2023-07-14T16:19:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.401+0000 7fbada7fc700 1 -- 172.21.15.62:0/1616487646 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbae40adca0 con 0x55bf932906b0 2023-07-14T16:19:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.401+0000 7fbada7fc700 1 -- 172.21.15.62:0/1616487646 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae40a1e40 con 0x55bf932906b0 2023-07-14T16:19:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1616487646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf932906b0 msgr2=0x55bf9343c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf932906b0 0x55bf9343c790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbae400cc00 tx=0x7fbae4000bc0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1616487646 shutdown_connections 2023-07-14T16:19:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf934357e0 0x55bf93438730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf932906b0 0x55bf9343c790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/1616487646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf93434a90 0x55bf93445040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1616487646 >> 172.21.15.62:0/1616487646 conn(0x55bf93429d40 msgr2=0x55bf934501b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1616487646 shutdown_connections 2023-07-14T16:19:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/1616487646 wait complete. 2023-07-14T16:19:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.402+0000 7fbb0d7c7b80 1 -- start start 2023-07-14T16:19:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbb0d7c7b80 1 Processor -- start 2023-07-14T16:19:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf932906b0 0x55bf93443030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf93434a90 0x55bf93443590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbb0d7c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93443c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf93444370 con 0x55bf93434a90 2023-07-14T16:19:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf934444e0 con 0x55bf932906b0 2023-07-14T16:19:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbb0d7c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf934446a0 con 0x55bf934357e0 2023-07-14T16:19:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbb0d7c7b80 10 client.0 _ll_register_callbacks cb 0x7fbaec003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbaf2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf93434a90 0x55bf93443590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbaf2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf93434a90 0x55bf93443590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57924/0 (socket says 172.21.15.62:57924) 2023-07-14T16:19:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbaf2ffd700 1 -- 172.21.15.62:0/2735286719 learned_addr learned my addr 172.21.15.62:0/2735286719 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.403+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf932906b0 0x55bf93443030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf3fff700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93443c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf2ffd700 1 -- 172.21.15.62:0/2735286719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 msgr2=0x55bf93443c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93443c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf2ffd700 1 -- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf932906b0 msgr2=0x55bf93443030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf932906b0 0x55bf93443030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf2ffd700 1 -- 172.21.15.62:0/2735286719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec016950 con 0x55bf93434a90 2023-07-14T16:19:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf932906b0 0x55bf93443030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf3fff700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93443c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbaf2ffd700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf93434a90 0x55bf93443590 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbae8005910 tx=0x7fbae8005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bec6b46ad5ddaf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.404+0000 7fbad2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.405+0000 7fbad2ffd700 1 -- 172.21.15.62:0/2735286719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae801d040 con 0x55bf93434a90 2023-07-14T16:19:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.405+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/2735286719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbae40a22d0 con 0x55bf93434a90 2023-07-14T16:19:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.405+0000 7fbad2ffd700 1 -- 172.21.15.62:0/2735286719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbae8003680 con 0x55bf93434a90 2023-07-14T16:19:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.405+0000 7fbad2ffd700 1 -- 172.21.15.62:0/2735286719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbae8003840 con 0x55bf93434a90 2023-07-14T16:19:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.405+0000 7fbb0d7c7b80 10 client.16542 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.405+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/2735286719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fbaf419fe10 con 0x55bf93434a90 2023-07-14T16:19:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.405+0000 7fbad2ffd700 1 -- 172.21.15.62:0/2735286719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbae40a22d0 con 0x55bf93434a90 2023-07-14T16:19:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.405+0000 7fbad2ffd700 1 -- 172.21.15.62:0/2735286719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbae803c070 con 0x55bf93434a90 2023-07-14T16:19:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.406+0000 7fbb0d7c7b80 10 client.16542 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.406+0000 7fbb0d7c7b80 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.406+0000 7fbb0d7c7b80 20 client.16542 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.406+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bf9349deb0 0x55bf934a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.406+0000 7fbb0d7c7b80 4 client.16542 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:19:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.406+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/2735286719 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bf93340ff0 con 0x55bf9349deb0 2023-07-14T16:19:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.406+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bf9349deb0 0x55bf934a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.407+0000 7fbaf37fe700 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bf9349deb0 0x55bf934a02c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63efa69285bcb40 server_cookie=bc6493a8eca4f849 in_seq=0 out_seq=0 2023-07-14T16:19:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.407+0000 7fbad2ffd700 10 client.16542 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbad2ffd700 1 -- 172.21.15.62:0/2735286719 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bf93340ff0 con 0x55bf9349deb0 2023-07-14T16:19:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbad2ffd700 10 client.16542 handle_command_reply: tid=0 2023-07-14T16:19:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 client.16542 shutdown 2023-07-14T16:19:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/2735286719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf93434a90 msgr2=0x55bf93443590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf93434a90 0x55bf93443590 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbae8005910 tx=0x7fbae8005160 comp rx=0 tx=0).stop 2023-07-14T16:19:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/2735286719 shutdown_connections 2023-07-14T16:19:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bf934357e0 0x55bf93443c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bf9349deb0 0x55bf934a02c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bf932906b0 0x55bf93443030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 --2- 172.21.15.62:0/2735286719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf93434a90 0x55bf93443590 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/2735286719 >> 172.21.15.62:0/2735286719 conn(0x55bf93429d40 msgr2=0x55bf9343c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbad2ffd700 0 client.16542 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/2735286719 shutdown_connections 2023-07-14T16:19:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.416+0000 7fbb0d7c7b80 1 -- 172.21.15.62:0/2735286719 wait complete. 2023-07-14T16:19:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbb0d7c7b80 20 client.16542 trim_cache size 0 max 16384 2023-07-14T16:19:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:19:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:19:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:19:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:19:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbadc0a11d0 msgr2=0x7fbadc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbadc0a11d0 0x7fbadc0a3680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55bf934342b0 tx=0x7fbaec004b40 comp rx=0 tx=0).stop 2023-07-14T16:19:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/890745276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaf4101b60 msgr2=0x7fbaf41977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaf4101b60 0x7fbaf41977d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbae400e970 tx=0x7fbae400ec80 comp rx=0 tx=0).stop 2023-07-14T16:19:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/890745276 shutdown_connections 2023-07-14T16:19:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbaf41036e0 0x7fbaf419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbadc0a11d0 0x7fbadc0a3680 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaf4101b60 0x7fbaf41977d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 --2- 172.21.15.62:0/890745276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaf4102d50 0x7fbaf4197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.417+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/890745276 >> 172.21.15.62:0/890745276 conn(0x7fbaf40fd180 msgr2=0x7fbaf41062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.418+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/890745276 shutdown_connections 2023-07-14T16:19:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:21.418+0000 7fbaf8b9d700 1 -- 172.21.15.62:0/890745276 wait complete. 2023-07-14T16:19:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:22 smithi164 ceph-mon[97427]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:19:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:22 smithi062 ceph-mon[95543]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:19:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:22 smithi081 ceph-mon[127795]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:19:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:24 smithi164 ceph-mon[97427]: pgmap v470: 97 pgs: 97 active+clean; 1.7 GiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:19:25.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:24 smithi062 ceph-mon[95543]: pgmap v470: 97 pgs: 97 active+clean; 1.7 GiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:19:25.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:24 smithi081 ceph-mon[127795]: pgmap v470: 97 pgs: 97 active+clean; 1.7 GiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:19:27.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:26 smithi164 ceph-mon[97427]: pgmap v471: 97 pgs: 97 active+clean; 1.7 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:19:27.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:26 smithi062 ceph-mon[95543]: pgmap v471: 97 pgs: 97 active+clean; 1.7 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:19:27.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:26 smithi081 ceph-mon[127795]: pgmap v471: 97 pgs: 97 active+clean; 1.7 GiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:19:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:28 smithi164 ceph-mon[97427]: pgmap v472: 97 pgs: 97 active+clean; 1.7 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 24 op/s 2023-07-14T16:19:29.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:28 smithi062 ceph-mon[95543]: pgmap v472: 97 pgs: 97 active+clean; 1.7 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 24 op/s 2023-07-14T16:19:29.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:28 smithi081 ceph-mon[127795]: pgmap v472: 97 pgs: 97 active+clean; 1.7 GiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 24 op/s 2023-07-14T16:19:31.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:30 smithi164 ceph-mon[97427]: pgmap v473: 97 pgs: 97 active+clean; 1.8 GiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:31.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:30 smithi062 ceph-mon[95543]: pgmap v473: 97 pgs: 97 active+clean; 1.8 GiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:31.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:30 smithi081 ceph-mon[127795]: pgmap v473: 97 pgs: 97 active+clean; 1.8 GiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:33.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:32 smithi062 ceph-mon[95543]: pgmap v474: 97 pgs: 97 active+clean; 1.8 GiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:19:33.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:32 smithi081 ceph-mon[127795]: pgmap v474: 97 pgs: 97 active+clean; 1.8 GiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:19:33.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:32 smithi164 ceph-mon[97427]: pgmap v474: 97 pgs: 97 active+clean; 1.8 GiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:19:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.585+0000 7fdd30fff700 1 -- 172.21.15.62:0/3057442968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c0ffb10 msgr2=0x7fdd2c0fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.585+0000 7fdd30fff700 1 --2- 172.21.15.62:0/3057442968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c0ffb10 0x7fdd2c0fff20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009990 tx=0x7fdd1c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.586+0000 7fdd30fff700 1 -- 172.21.15.62:0/3057442968 shutdown_connections 2023-07-14T16:19:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.586+0000 7fdd30fff700 1 --2- 172.21.15.62:0/3057442968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd2c101690 0x7fdd2c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.586+0000 7fdd30fff700 1 --2- 172.21.15.62:0/3057442968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c0ffb10 0x7fdd2c0fff20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.586+0000 7fdd30fff700 1 --2- 172.21.15.62:0/3057442968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd2c100d00 0x7fdd2c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.586+0000 7fdd30fff700 1 -- 172.21.15.62:0/3057442968 >> 172.21.15.62:0/3057442968 conn(0x7fdd2c0fb130 msgr2=0x7fdd2c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.586+0000 7fdd30fff700 1 -- 172.21.15.62:0/3057442968 shutdown_connections 2023-07-14T16:19:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.586+0000 7fdd30fff700 1 -- 172.21.15.62:0/3057442968 wait complete. 2023-07-14T16:19:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.587+0000 7fdd30fff700 1 Processor -- start 2023-07-14T16:19:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.587+0000 7fdd30fff700 1 -- start start 2023-07-14T16:19:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.587+0000 7fdd30fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd2c0ffb10 0x7fdd2c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.587+0000 7fdd30fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd2c100d00 0x7fdd2c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.587+0000 7fdd30fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c101690 0x7fdd2c073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.587+0000 7fdd30fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c19f820 con 0x7fdd2c101690 2023-07-14T16:19:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.587+0000 7fdd30fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c19f960 con 0x7fdd2c100d00 2023-07-14T16:19:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.587+0000 7fdd30fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c19faa0 con 0x7fdd2c0ffb10 2023-07-14T16:19:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c101690 0x7fdd2c073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c101690 0x7fdd2c073380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55850/0 (socket says 172.21.15.62:55850) 2023-07-14T16:19:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd29d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd2c100d00 0x7fdd2c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2ad9d700 1 -- 172.21.15.62:0/880510986 learned_addr learned my addr 172.21.15.62:0/880510986 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd2c0ffb10 0x7fdd2c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2ad9d700 1 -- 172.21.15.62:0/880510986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd2c0ffb10 msgr2=0x7fdd2c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd2c0ffb10 0x7fdd2c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2ad9d700 1 -- 172.21.15.62:0/880510986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd2c100d00 msgr2=0x7fdd2c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd2c100d00 0x7fdd2c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.588+0000 7fdd2ad9d700 1 -- 172.21.15.62:0/880510986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0095f0 con 0x7fdd2c101690 2023-07-14T16:19:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.589+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c101690 0x7fdd2c073380 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400ef30 tx=0x7fdd2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.589+0000 7fdd1b7fe700 1 -- 172.21.15.62:0/880510986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24018070 con 0x7fdd2c101690 2023-07-14T16:19:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.589+0000 7fdd1b7fe700 1 -- 172.21.15.62:0/880510986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd2400f040 con 0x7fdd2c101690 2023-07-14T16:19:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.589+0000 7fdd1b7fe700 1 -- 172.21.15.62:0/880510986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400cbc0 con 0x7fdd2c101690 2023-07-14T16:19:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.589+0000 7fdd30fff700 1 -- 172.21.15.62:0/880510986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c19fcd0 con 0x7fdd2c101690 2023-07-14T16:19:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.589+0000 7fdd30fff700 1 -- 172.21.15.62:0/880510986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c1a0160 con 0x7fdd2c101690 2023-07-14T16:19:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.593+0000 7fdd1b7fe700 1 -- 172.21.15.62:0/880510986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdd2400cd80 con 0x7fdd2c101690 2023-07-14T16:19:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.594+0000 7fdd4484eb80 1 Processor -- start 2023-07-14T16:19:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.594+0000 7fdd4484eb80 1 -- start start 2023-07-14T16:19:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.594+0000 7fdd1b7fe700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd140a11d0 0x7fdd140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd1b7fe700 1 -- 172.21.15.62:0/880510986 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd240cc350 con 0x7fdd2c101690 2023-07-14T16:19:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ab876b0 0x562e2ad2b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad2c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ac30c50 con 0x562e2ad2c6e0 2023-07-14T16:19:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ac839e0 con 0x562e2ad2b840 2023-07-14T16:19:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ac7ce70 con 0x562e2ab876b0 2023-07-14T16:19:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd140a11d0 0x7fdd140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd2ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd2a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ab876b0 0x562e2ad2b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd2a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ab876b0 0x562e2ad2b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50800/0 (socket says 172.21.15.62:50800) 2023-07-14T16:19:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd2ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad2cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55852/0 (socket says 172.21.15.62:55852) 2023-07-14T16:19:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd2a59c700 1 -- 172.21.15.62:0/1527505744 learned_addr learned my addr 172.21.15.62:0/1527505744 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.595+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad2c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd2a59c700 1 -- 172.21.15.62:0/1527505744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 msgr2=0x562e2ad2c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad2c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd2a59c700 1 -- 172.21.15.62:0/1527505744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 msgr2=0x562e2ad2cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad2cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd2a59c700 1 -- 172.21.15.62:0/1527505744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e2aaaad80 con 0x562e2ab876b0 2023-07-14T16:19:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad2cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd140a11d0 0x7fdd140a3680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b6c0 tx=0x7fdd2000bbf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ab876b0 0x562e2ad2b300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009990 tx=0x7fdd1c005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197e3362f2c92e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.596+0000 7fdd18ff9700 1 -- 172.21.15.62:0/1527505744 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c01d070 con 0x562e2ab876b0 2023-07-14T16:19:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd18ff9700 1 -- 172.21.15.62:0/1527505744 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd1c01e040 con 0x562e2ab876b0 2023-07-14T16:19:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd18ff9700 1 -- 172.21.15.62:0/1527505744 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00b6b0 con 0x562e2ab876b0 2023-07-14T16:19:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd4484eb80 1 -- 172.21.15.62:0/1527505744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ab876b0 msgr2=0x562e2ad2b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ab876b0 0x562e2ad2b300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009990 tx=0x7fdd1c005c70 comp rx=0 tx=0).stop 2023-07-14T16:19:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd4484eb80 1 -- 172.21.15.62:0/1527505744 shutdown_connections 2023-07-14T16:19:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad2cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ab876b0 0x562e2ad2b300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/1527505744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad2c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.597+0000 7fdd4484eb80 1 -- 172.21.15.62:0/1527505744 >> 172.21.15.62:0/1527505744 conn(0x562e2ad20be0 msgr2=0x562e2ad25cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.598+0000 7fdd4484eb80 1 -- 172.21.15.62:0/1527505744 shutdown_connections 2023-07-14T16:19:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.598+0000 7fdd4484eb80 1 -- 172.21.15.62:0/1527505744 wait complete. 2023-07-14T16:19:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.598+0000 7fdd4484eb80 1 -- start start 2023-07-14T16:19:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.598+0000 7fdd4484eb80 1 Processor -- start 2023-07-14T16:19:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 0x562e2ad34cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ac1ca60 con 0x562e2ab876b0 2023-07-14T16:19:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ac1c760 con 0x562e2ad2c6e0 2023-07-14T16:19:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ab008c0 con 0x562e2ad2b840 2023-07-14T16:19:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd4484eb80 10 client.0 _ll_register_callbacks cb 0x562e2ac1a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd2a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd2a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55864/0 (socket says 172.21.15.62:55864) 2023-07-14T16:19:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd2a59c700 1 -- 172.21.15.62:0/2730330971 learned_addr learned my addr 172.21.15.62:0/2730330971 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.599+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 0x562e2ad34cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd2a59c700 1 -- 172.21.15.62:0/2730330971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 msgr2=0x562e2ad34cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 0x562e2ad34cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd2a59c700 1 -- 172.21.15.62:0/2730330971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2c6e0 msgr2=0x562e2ad3bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd2a59c700 1 -- 172.21.15.62:0/2730330971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c1a0160 con 0x562e2ab876b0 2023-07-14T16:19:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad347b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c003ea0 tx=0x7fdd1c0059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc63b1a9f8e563d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2730330971 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c01d030 con 0x562e2ab876b0 2023-07-14T16:19:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2730330971 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd1c00b6b0 con 0x562e2ab876b0 2023-07-14T16:19:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.600+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2730330971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e2abd6490 con 0x562e2ab876b0 2023-07-14T16:19:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2730330971 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd1c005710 con 0x562e2ab876b0 2023-07-14T16:19:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd4484eb80 10 client.16554 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2730330971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fdd2c108880 con 0x562e2ab876b0 2023-07-14T16:19:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2730330971 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd1c03d020 con 0x562e2ab876b0 2023-07-14T16:19:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2730330971 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdd1c004a30 con 0x562e2ab876b0 2023-07-14T16:19:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd4484eb80 10 client.16554 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd4484eb80 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd4484eb80 20 client.16554 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562e2ad400d0 0x562e2ad424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd4484eb80 4 client.16554 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.601+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2730330971 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e2ac75e50 con 0x562e2ad400d0 2023-07-14T16:19:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.602+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562e2ad400d0 0x562e2ad424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.602+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562e2ad400d0 0x562e2ad424e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0e37c1c0a4c596 server_cookie=6d1f43e30ac880cb in_seq=0 out_seq=0 2023-07-14T16:19:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.602+0000 7fdd11ffb700 10 client.16554 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2730330971 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562e2ac75e50 con 0x562e2ad400d0 2023-07-14T16:19:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd11ffb700 10 client.16554 handle_command_reply: tid=0 2023-07-14T16:19:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 client.16554 shutdown 2023-07-14T16:19:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2730330971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 msgr2=0x562e2ad347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad347b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c003ea0 tx=0x7fdd1c0059f0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2730330971 shutdown_connections 2023-07-14T16:19:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562e2ad400d0 0x562e2ad424e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd11ffb700 0 client.16554 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad347b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2730330971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 0x562e2ad34cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.603+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2730330971 >> 172.21.15.62:0/2730330971 conn(0x562e2ad20be0 msgr2=0x562e2ad254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.604+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2730330971 shutdown_connections 2023-07-14T16:19:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.604+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2730330971 wait complete. 2023-07-14T16:19:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.604+0000 7fdd4484eb80 20 client.16554 trim_cache size 0 max 16384 2023-07-14T16:19:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd4484eb80 1 Processor -- start 2023-07-14T16:19:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd4484eb80 1 -- start start 2023-07-14T16:19:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad2b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad3bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2c6e0 0x562e2ad33e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ad277a0 con 0x562e2ab876b0 2023-07-14T16:19:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ad22d00 con 0x562e2ad2b840 2023-07-14T16:19:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ad301a0 con 0x562e2ad2c6e0 2023-07-14T16:19:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd2a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad2b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd2a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad2b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55868/0 (socket says 172.21.15.62:55868) 2023-07-14T16:19:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd2a59c700 1 -- 172.21.15.62:0/3763026305 learned_addr learned my addr 172.21.15.62:0/3763026305 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.623+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad3bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.624+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2c6e0 0x562e2ad33e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.624+0000 7fdd29d9b700 1 -- 172.21.15.62:0/3763026305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2c6e0 msgr2=0x562e2ad33e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.624+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2c6e0 0x562e2ad33e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.624+0000 7fdd29d9b700 1 -- 172.21.15.62:0/3763026305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 msgr2=0x562e2ad2b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.624+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad2b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.624+0000 7fdd29d9b700 1 -- 172.21.15.62:0/3763026305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e2aa93c00 con 0x562e2ad2b840 2023-07-14T16:19:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.624+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad2b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.624+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad3bcf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x562e2ad42d90 tx=0x7fdd2000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f46d297a83dbe35 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.625+0000 7fdd197fa700 1 -- 172.21.15.62:0/3763026305 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20029410 con 0x562e2ad2b840 2023-07-14T16:19:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.625+0000 7fdd197fa700 1 -- 172.21.15.62:0/3763026305 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd20013e90 con 0x562e2ad2b840 2023-07-14T16:19:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.625+0000 7fdd197fa700 1 -- 172.21.15.62:0/3763026305 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20020670 con 0x562e2ad2b840 2023-07-14T16:19:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.625+0000 7fdd4484eb80 1 -- 172.21.15.62:0/3763026305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 msgr2=0x562e2ad3bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.625+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad3bcf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x562e2ad42d90 tx=0x7fdd2000b410 comp rx=0 tx=0).stop 2023-07-14T16:19:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 -- 172.21.15.62:0/3763026305 shutdown_connections 2023-07-14T16:19:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2c6e0 0x562e2ad33e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ab876b0 0x562e2ad2b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/3763026305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ad2b840 0x562e2ad3bcf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 -- 172.21.15.62:0/3763026305 >> 172.21.15.62:0/3763026305 conn(0x562e2ad20be0 msgr2=0x562e2ad46e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 -- 172.21.15.62:0/3763026305 shutdown_connections 2023-07-14T16:19:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 -- 172.21.15.62:0/3763026305 wait complete. 2023-07-14T16:19:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 -- start start 2023-07-14T16:19:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.626+0000 7fdd4484eb80 1 Processor -- start 2023-07-14T16:19:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ab876b0 0x562e2ad37f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 0x562e2ad38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd4484eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ad3b510 con 0x562e2ad2c6e0 2023-07-14T16:19:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ad3b680 con 0x562e2ab876b0 2023-07-14T16:19:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd4484eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2ad3b840 con 0x562e2ad2b840 2023-07-14T16:19:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd4484eb80 10 client.0 _ll_register_callbacks cb 0x7fdd24004c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd2ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd2ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55884/0 (socket says 172.21.15.62:55884) 2023-07-14T16:19:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd2ad9d700 1 -- 172.21.15.62:0/2757005100 learned_addr learned my addr 172.21.15.62:0/2757005100 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.627+0000 7fdd29d9b700 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 0x562e2ad38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ab876b0 0x562e2ad37f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd2ad9d700 1 -- 172.21.15.62:0/2757005100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 msgr2=0x562e2ad38460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 0x562e2ad38460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd2ad9d700 1 -- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ab876b0 msgr2=0x562e2ad37f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ab876b0 0x562e2ad37f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd2ad9d700 1 -- 172.21.15.62:0/2757005100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c108880 con 0x562e2ad2c6e0 2023-07-14T16:19:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ab876b0 0x562e2ad37f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd2ad9d700 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3afd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdd240994a0 tx=0x7fdd24099810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0d3fb2d76f5fff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2757005100 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24098c90 con 0x562e2ad2c6e0 2023-07-14T16:19:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2757005100 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd24098df0 con 0x562e2ad2c6e0 2023-07-14T16:19:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2757005100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd1c01e040 con 0x562e2ad2c6e0 2023-07-14T16:19:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.628+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2757005100 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd240c9030 con 0x562e2ad2c6e0 2023-07-14T16:19:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd4484eb80 10 client.16560 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2757005100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fdd2001a040 con 0x562e2ad2c6e0 2023-07-14T16:19:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2757005100 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd2407c020 con 0x562e2ad2c6e0 2023-07-14T16:19:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2757005100 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdd24098790 con 0x562e2ad2c6e0 2023-07-14T16:19:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd4484eb80 10 client.16560 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd4484eb80 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd4484eb80 20 client.16560 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562e2ad94fe0 0x562e2ad973f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd4484eb80 4 client.16560 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:19:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.629+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2757005100 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e2ac37ff0 con 0x562e2ad94fe0 2023-07-14T16:19:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.630+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562e2ad94fe0 0x562e2ad973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.630+0000 7fdd11ffb700 10 client.16560 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.630+0000 7fdd2a59c700 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562e2ad94fe0 0x562e2ad973f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc824f94cc6760b0 server_cookie=727a07838920342e in_seq=0 out_seq=0 2023-07-14T16:19:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd11ffb700 1 -- 172.21.15.62:0/2757005100 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e2ac37ff0 con 0x562e2ad94fe0 2023-07-14T16:19:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd11ffb700 10 client.16560 handle_command_reply: tid=0 2023-07-14T16:19:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 client.16560 shutdown 2023-07-14T16:19:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2757005100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 msgr2=0x562e2ad3afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3afd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdd240994a0 tx=0x7fdd24099810 comp rx=0 tx=0).stop 2023-07-14T16:19:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2757005100 shutdown_connections 2023-07-14T16:19:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562e2ad2c6e0 0x562e2ad3afd0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562e2ad94fe0 0x562e2ad973f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e2ab876b0 0x562e2ad37f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 --2- 172.21.15.62:0/2757005100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562e2ad2b840 0x562e2ad38460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd11ffb700 0 client.16560 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2757005100 >> 172.21.15.62:0/2757005100 conn(0x562e2ad20be0 msgr2=0x562e2ad2f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2757005100 shutdown_connections 2023-07-14T16:19:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.640+0000 7fdd4484eb80 1 -- 172.21.15.62:0/2757005100 wait complete. 2023-07-14T16:19:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd4484eb80 20 client.16560 trim_cache size 0 max 16384 2023-07-14T16:19:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:19:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:19:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:19:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:19:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 -- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd140a11d0 msgr2=0x7fdd140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd140a11d0 0x7fdd140a3680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b6c0 tx=0x7fdd2000bbf0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 -- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c101690 msgr2=0x7fdd2c073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c101690 0x7fdd2c073380 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400ef30 tx=0x7fdd2400c3a0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 -- 172.21.15.62:0/880510986 shutdown_connections 2023-07-14T16:19:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd2c101690 0x7fdd2c073380 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd2c0ffb10 0x7fdd2c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd140a11d0 0x7fdd140a3680 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 --2- 172.21.15.62:0/880510986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd2c100d00 0x7fdd2c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 -- 172.21.15.62:0/880510986 >> 172.21.15.62:0/880510986 conn(0x7fdd2c0fb130 msgr2=0x7fdd2c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 -- 172.21.15.62:0/880510986 shutdown_connections 2023-07-14T16:19:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:34.641+0000 7fdd30fff700 1 -- 172.21.15.62:0/880510986 wait complete. 2023-07-14T16:19:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:34 smithi062 ceph-mon[95543]: pgmap v475: 97 pgs: 97 active+clean; 1.8 GiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:34 smithi081 ceph-mon[127795]: pgmap v475: 97 pgs: 97 active+clean; 1.8 GiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:35.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:34 smithi164 ceph-mon[97427]: pgmap v475: 97 pgs: 97 active+clean; 1.8 GiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:37.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:36 smithi062 ceph-mon[95543]: pgmap v476: 97 pgs: 97 active+clean; 1.8 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:19:37.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:36 smithi081 ceph-mon[127795]: pgmap v476: 97 pgs: 97 active+clean; 1.8 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:19:37.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:36 smithi164 ceph-mon[97427]: pgmap v476: 97 pgs: 97 active+clean; 1.8 GiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:19:39.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:38 smithi062 ceph-mon[95543]: pgmap v477: 97 pgs: 97 active+clean; 1.9 GiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:19:39.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:38 smithi081 ceph-mon[127795]: pgmap v477: 97 pgs: 97 active+clean; 1.9 GiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:19:39.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:38 smithi164 ceph-mon[97427]: pgmap v477: 97 pgs: 97 active+clean; 1.9 GiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:19:40.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:39 smithi062 ceph-mon[95543]: pgmap v478: 97 pgs: 97 active+clean; 1.9 GiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:40.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:39 smithi081 ceph-mon[127795]: pgmap v478: 97 pgs: 97 active+clean; 1.9 GiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:40.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:39 smithi164 ceph-mon[97427]: pgmap v478: 97 pgs: 97 active+clean; 1.9 GiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:19:42.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:42 smithi062 ceph-mon[95543]: pgmap v479: 97 pgs: 97 active+clean; 1.9 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:19:42.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:42 smithi081 ceph-mon[127795]: pgmap v479: 97 pgs: 97 active+clean; 1.9 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:19:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:42 smithi164 ceph-mon[97427]: pgmap v479: 97 pgs: 97 active+clean; 1.9 GiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:19:43.261 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:19:43.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 -- 172.21.15.62:0/2068133033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30180744f0 msgr2=0x7f3018072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 --2- 172.21.15.62:0/2068133033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30180744f0 0x7f3018072940 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3008009990 tx=0x7f3008009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 -- 172.21.15.62:0/2068133033 shutdown_connections 2023-07-14T16:19:43.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 --2- 172.21.15.62:0/2068133033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018107980 0x7f3018109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 --2- 172.21.15.62:0/2068133033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30180744f0 0x7f3018072940 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 --2- 172.21.15.62:0/2068133033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3018072f10 0x7f3018073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 -- 172.21.15.62:0/2068133033 >> 172.21.15.62:0/2068133033 conn(0x7f30180fb130 msgr2=0x7f30180fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:43.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 -- 172.21.15.62:0/2068133033 shutdown_connections 2023-07-14T16:19:43.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.527+0000 7f301ddff700 1 -- 172.21.15.62:0/2068133033 wait complete. 2023-07-14T16:19:43.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.528+0000 7f301ddff700 1 Processor -- start 2023-07-14T16:19:43.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.528+0000 7f301ddff700 1 -- start start 2023-07-14T16:19:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f301ddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3018072f10 0x7f3018197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f301ddff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30180744f0 0x7f3018197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f301ddff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018107980 0x7f301819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f301ddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301819f490 con 0x7f3018072f10 2023-07-14T16:19:43.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f301ddff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301819f5d0 con 0x7f30180744f0 2023-07-14T16:19:43.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f301ddff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301819f760 con 0x7f3018107980 2023-07-14T16:19:43.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f30177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3018072f10 0x7f3018197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f30177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3018072f10 0x7f3018197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35066/0 (socket says 172.21.15.62:35066) 2023-07-14T16:19:43.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f30177fe700 1 -- 172.21.15.62:0/2498987594 learned_addr learned my addr 172.21.15.62:0/2498987594 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f30177fe700 1 -- 172.21.15.62:0/2498987594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018107980 msgr2=0x7f301819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f30177fe700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018107980 0x7f301819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f30177fe700 1 -- 172.21.15.62:0/2498987594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30180744f0 msgr2=0x7f3018197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f3016ffd700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30180744f0 0x7f3018197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f30177fe700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30180744f0 0x7f3018197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f30177fe700 1 -- 172.21.15.62:0/2498987594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080095f0 con 0x7f3018072f10 2023-07-14T16:19:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.529+0000 7f3016ffd700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30180744f0 0x7f3018197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:19:43.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.530+0000 7f30177fe700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3018072f10 0x7f3018197910 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3008000c00 tx=0x7f3008003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.530+0000 7f3014ff9700 1 -- 172.21.15.62:0/2498987594 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300801d070 con 0x7f3018072f10 2023-07-14T16:19:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.530+0000 7f3014ff9700 1 -- 172.21.15.62:0/2498987594 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f300800bd70 con 0x7f3018072f10 2023-07-14T16:19:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.530+0000 7f3014ff9700 1 -- 172.21.15.62:0/2498987594 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008022cf0 con 0x7f3018072f10 2023-07-14T16:19:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.530+0000 7f301ddff700 1 -- 172.21.15.62:0/2498987594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301819f9e0 con 0x7f3018072f10 2023-07-14T16:19:43.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.530+0000 7f301ddff700 1 -- 172.21.15.62:0/2498987594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301819fdb0 con 0x7f3018072f10 2023-07-14T16:19:43.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.534+0000 7f3014ff9700 1 -- 172.21.15.62:0/2498987594 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3008022470 con 0x7f3018072f10 2023-07-14T16:19:43.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f30317b0b80 1 Processor -- start 2023-07-14T16:19:43.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f3014ff9700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30000a11d0 0x7f30000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f30317b0b80 1 -- start start 2023-07-14T16:19:43.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f3014ff9700 1 -- 172.21.15.62:0/2498987594 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f30080cdf20 con 0x7f3018072f10 2023-07-14T16:19:43.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f3017fff700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30000a11d0 0x7f30000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f9e20 0x55bd8a4fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 0x55bd8a4fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f30317b0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a3fecb0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f30317b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a451a40 con 0x55bd8a4f9e20 2023-07-14T16:19:43.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.536+0000 7f30317b0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a44aed0 con 0x55bd8a4fac70 2023-07-14T16:19:43.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f30177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f30177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35074/0 (socket says 172.21.15.62:35074) 2023-07-14T16:19:43.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f30177fe700 1 -- 172.21.15.62:0/2711040871 learned_addr learned my addr 172.21.15.62:0/2711040871 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:43.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f3016ffd700 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f9e20 0x55bd8a4fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f30177fe700 1 -- 172.21.15.62:0/2711040871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 msgr2=0x55bd8a4fb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:43.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f30177fe700 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 0x55bd8a4fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f30177fe700 1 -- 172.21.15.62:0/2711040871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f9e20 msgr2=0x55bd8a4fa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:43.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f30177fe700 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f9e20 0x55bd8a4fa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f30177fe700 1 -- 172.21.15.62:0/2711040871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd8a278de0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f3016ffd700 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f9e20 0x55bd8a4fa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:43.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.537+0000 7f3017fff700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30000a11d0 0x7f30000a3680 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3010008fd0 tx=0x7f301000d470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:43.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.538+0000 7f30177fe700 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f98e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3008097870 tx=0x7f3008009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aad1ae20acefd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:43.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.538+0000 7f2ffdffb700 1 -- 172.21.15.62:0/2711040871 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300809b370 con 0x55bd8a4f94d0 2023-07-14T16:19:43.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.538+0000 7f2ffdffb700 1 -- 172.21.15.62:0/2711040871 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30080cbe90 con 0x55bd8a4f94d0 2023-07-14T16:19:43.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.538+0000 7f2ffdffb700 1 -- 172.21.15.62:0/2711040871 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30080a10b0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.538+0000 7f30317b0b80 1 -- 172.21.15.62:0/2711040871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 msgr2=0x55bd8a4f98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f98e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3008097870 tx=0x7f3008009310 comp rx=0 tx=0).stop 2023-07-14T16:19:43.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 -- 172.21.15.62:0/2711040871 shutdown_connections 2023-07-14T16:19:43.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 0x55bd8a4fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f98e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 --2- 172.21.15.62:0/2711040871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f9e20 0x55bd8a4fa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 -- 172.21.15.62:0/2711040871 >> 172.21.15.62:0/2711040871 conn(0x55bd8a4eeda0 msgr2=0x55bd8a4f41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:43.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 -- 172.21.15.62:0/2711040871 shutdown_connections 2023-07-14T16:19:43.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 -- 172.21.15.62:0/2711040871 wait complete. 2023-07-14T16:19:43.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.539+0000 7f30317b0b80 1 -- start start 2023-07-14T16:19:43.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.540+0000 7f30317b0b80 1 Processor -- start 2023-07-14T16:19:43.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.540+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a502de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.540+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 0x55bd8a503320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.540+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4fac70 0x55bd8a50a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.540+0000 7f30317b0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a3eaac0 con 0x55bd8a4fac70 2023-07-14T16:19:43.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.540+0000 7f30317b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a3ea7c0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.540+0000 7f30317b0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a2ce920 con 0x55bd8a4f9e20 2023-07-14T16:19:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.540+0000 7f30317b0b80 10 client.0 _ll_register_callbacks cb 0x55bd8a3e8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f30177fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a502de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f30177fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a502de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:33826/0 (socket says 172.21.15.62:33826) 2023-07-14T16:19:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f30177fe700 1 -- 172.21.15.62:0/3361386010 learned_addr learned my addr 172.21.15.62:0/3361386010 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f3017fff700 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4fac70 0x55bd8a50a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f3016ffd700 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 0x55bd8a503320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f30177fe700 1 -- 172.21.15.62:0/3361386010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 msgr2=0x55bd8a503320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f30177fe700 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 0x55bd8a503320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f30177fe700 1 -- 172.21.15.62:0/3361386010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4fac70 msgr2=0x55bd8a50a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f30177fe700 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4fac70 0x55bd8a50a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.541+0000 7f30177fe700 1 -- 172.21.15.62:0/3361386010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301819fdb0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f30177fe700 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a502de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f300809b2b0 tx=0x7f3008009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c7304d794c595d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f2ff6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:19:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/3361386010 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300809f3a0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f30317b0b80 1 -- 172.21.15.62:0/3361386010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd8a3a44f0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/3361386010 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f30080a3730 con 0x55bd8a4f94d0 2023-07-14T16:19:43.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/3361386010 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f300809b370 con 0x55bd8a4f94d0 2023-07-14T16:19:43.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f30317b0b80 10 client.26294 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:43.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f30317b0b80 1 -- 172.21.15.62:0/3361386010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bd8a261c60 con 0x55bd8a4f94d0 2023-07-14T16:19:43.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.542+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/3361386010 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bd8a3a44f0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.543+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/3361386010 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f300809a530 con 0x55bd8a4f94d0 2023-07-14T16:19:43.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.543+0000 7f30317b0b80 10 client.26294 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:43.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.543+0000 7f30317b0b80 10 client.26294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:43.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.543+0000 7f30317b0b80 20 client.26294 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.543+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a50e710 0x55bd8a510b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.543+0000 7f30317b0b80 4 client.26294 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.543+0000 7f30317b0b80 1 -- 172.21.15.62:0/3361386010 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd8a443eb0 con 0x55bd8a50e710 2023-07-14T16:19:43.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.543+0000 7f3016ffd700 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a50e710 0x55bd8a510b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.544+0000 7f3016ffd700 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a50e710 0x55bd8a510b20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=107d4fa4e3fac743 server_cookie=bdc833424e1ca895 in_seq=0 out_seq=0 2023-07-14T16:19:43.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.544+0000 7f2ff6ffd700 10 client.26294 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/3361386010 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bd8a443eb0 con 0x55bd8a50e710 2023-07-14T16:19:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f2ff6ffd700 10 client.26294 handle_command_reply: tid=0 2023-07-14T16:19:43.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 client.26294 shutdown 2023-07-14T16:19:43.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 -- 172.21.15.62:0/3361386010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 msgr2=0x55bd8a502de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a502de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f300809b2b0 tx=0x7f3008009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 -- 172.21.15.62:0/3361386010 shutdown_connections 2023-07-14T16:19:43.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4fac70 0x55bd8a50a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a50e710 0x55bd8a510b20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a502de0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3361386010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 0x55bd8a503320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f2ff6ffd700 0 client.26294 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 -- 172.21.15.62:0/3361386010 >> 172.21.15.62:0/3361386010 conn(0x55bd8a4eeda0 msgr2=0x55bd8a4f38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 -- 172.21.15.62:0/3361386010 shutdown_connections 2023-07-14T16:19:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.545+0000 7f30317b0b80 1 -- 172.21.15.62:0/3361386010 wait complete. 2023-07-14T16:19:43.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.546+0000 7f30317b0b80 20 client.26294 trim_cache size 0 max 16384 2023-07-14T16:19:43.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.569+0000 7f30317b0b80 1 Processor -- start 2023-07-14T16:19:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.569+0000 7f30317b0b80 1 -- start start 2023-07-14T16:19:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.569+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.569+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f9e20 0x55bd8a50a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.569+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 0x55bd8a501070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.569+0000 7f30317b0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a443eb0 con 0x55bd8a4f9e20 2023-07-14T16:19:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.569+0000 7f30317b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a3a44f0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.569+0000 7f30317b0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a503fd0 con 0x55bd8a4fac70 2023-07-14T16:19:43.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f9e20 0x55bd8a50a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f9e20 0x55bd8a50a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35098/0 (socket says 172.21.15.62:35098) 2023-07-14T16:19:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 -- 172.21.15.62:0/3809308188 learned_addr learned my addr 172.21.15.62:0/3809308188 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3017fff700 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 0x55bd8a501070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f30177fe700 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 -- 172.21.15.62:0/3809308188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 msgr2=0x55bd8a501070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 0x55bd8a501070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 -- 172.21.15.62:0/3809308188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 msgr2=0x55bd8a4f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 -- 172.21.15.62:0/3809308188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd8a29f250 con 0x55bd8a4f9e20 2023-07-14T16:19:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.570+0000 7f3016ffd700 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f9e20 0x55bd8a50a320 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x55bd8a511400 tx=0x7f300c009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33673c06afc6049 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.571+0000 7f2ffe7fc700 1 -- 172.21.15.62:0/3809308188 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c018040 con 0x55bd8a4f9e20 2023-07-14T16:19:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.571+0000 7f2ffe7fc700 1 -- 172.21.15.62:0/3809308188 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f300c01a370 con 0x55bd8a4f9e20 2023-07-14T16:19:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.571+0000 7f2ffe7fc700 1 -- 172.21.15.62:0/3809308188 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c012450 con 0x55bd8a4f9e20 2023-07-14T16:19:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.571+0000 7f30317b0b80 1 -- 172.21.15.62:0/3809308188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f9e20 msgr2=0x55bd8a50a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.571+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f9e20 0x55bd8a50a320 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x55bd8a511400 tx=0x7f300c009600 comp rx=0 tx=0).stop 2023-07-14T16:19:43.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.572+0000 7f30317b0b80 1 -- 172.21.15.62:0/3809308188 shutdown_connections 2023-07-14T16:19:43.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.572+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4fac70 0x55bd8a501070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.572+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4f94d0 0x55bd8a4f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.572+0000 7f30317b0b80 1 --2- 172.21.15.62:0/3809308188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f9e20 0x55bd8a50a320 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.572+0000 7f30317b0b80 1 -- 172.21.15.62:0/3809308188 >> 172.21.15.62:0/3809308188 conn(0x55bd8a4eeda0 msgr2=0x55bd8a501b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:43.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.572+0000 7f30317b0b80 1 -- 172.21.15.62:0/3809308188 shutdown_connections 2023-07-14T16:19:43.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.572+0000 7f30317b0b80 1 -- 172.21.15.62:0/3809308188 wait complete. 2023-07-14T16:19:43.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.572+0000 7f30317b0b80 1 -- start start 2023-07-14T16:19:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30317b0b80 1 Processor -- start 2023-07-14T16:19:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a50a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 0x55bd8a506e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30317b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4fac70 0x55bd8a5073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30317b0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a5625c0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30317b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a562700 con 0x55bd8a4fac70 2023-07-14T16:19:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30317b0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd8a562840 con 0x55bd8a4f9e20 2023-07-14T16:19:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30317b0b80 10 client.0 _ll_register_callbacks cb 0x7f300809afa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a50a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a50a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35112/0 (socket says 172.21.15.62:35112) 2023-07-14T16:19:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.573+0000 7f30177fe700 1 -- 172.21.15.62:0/858646371 learned_addr learned my addr 172.21.15.62:0/858646371 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f3017fff700 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4fac70 0x55bd8a5073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f3016ffd700 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 0x55bd8a506e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f30177fe700 1 -- 172.21.15.62:0/858646371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 msgr2=0x55bd8a506e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f30177fe700 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 0x55bd8a506e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f30177fe700 1 -- 172.21.15.62:0/858646371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4fac70 msgr2=0x55bd8a5073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f30177fe700 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4fac70 0x55bd8a5073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f30177fe700 1 -- 172.21.15.62:0/858646371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c009840 con 0x55bd8a4f94d0 2023-07-14T16:19:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f30177fe700 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a50a360 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3008006010 tx=0x7f3008099600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430b8e7f4b437914 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f2ff6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/858646371 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300809c360 con 0x55bd8a4f94d0 2023-07-14T16:19:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/858646371 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f30080a3730 con 0x55bd8a4f94d0 2023-07-14T16:19:43.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.574+0000 7f30317b0b80 1 -- 172.21.15.62:0/858646371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30080cbe90 con 0x55bd8a4f94d0 2023-07-14T16:19:43.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/858646371 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f300809f3a0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f30317b0b80 10 client.16578 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:43.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f30317b0b80 1 -- 172.21.15.62:0/858646371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bd8a2772b0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/858646371 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30080cbe90 con 0x55bd8a4f94d0 2023-07-14T16:19:43.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/858646371 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f30080a32a0 con 0x55bd8a4f94d0 2023-07-14T16:19:43.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f30317b0b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:43.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f30317b0b80 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f30317b0b80 20 client.16578 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f30317b0b80 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a563c60 0x55bd8a566040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f30317b0b80 4 client.16578 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:19:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.575+0000 7f30317b0b80 1 -- 172.21.15.62:0/858646371 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bd8a406050 con 0x55bd8a563c60 2023-07-14T16:19:43.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.576+0000 7f3016ffd700 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a563c60 0x55bd8a566040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.576+0000 7f3016ffd700 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a563c60 0x55bd8a566040 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b740491ae156ae server_cookie=87f2cfb40c7b7d06 in_seq=0 out_seq=0 2023-07-14T16:19:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.576+0000 7f2ff6ffd700 10 client.16578 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:43.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f2ff6ffd700 1 -- 172.21.15.62:0/858646371 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bd8a406050 con 0x55bd8a563c60 2023-07-14T16:19:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f2ff6ffd700 10 client.16578 handle_command_reply: tid=0 2023-07-14T16:19:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 client.16578 shutdown 2023-07-14T16:19:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 -- 172.21.15.62:0/858646371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 msgr2=0x55bd8a50a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a50a360 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3008006010 tx=0x7f3008099600 comp rx=0 tx=0).stop 2023-07-14T16:19:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 -- 172.21.15.62:0/858646371 shutdown_connections 2023-07-14T16:19:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bd8a563c60 0x55bd8a566040 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bd8a4fac70 0x55bd8a5073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd8a4f94d0 0x55bd8a50a360 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 --2- 172.21.15.62:0/858646371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bd8a4f9e20 0x55bd8a506e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 -- 172.21.15.62:0/858646371 >> 172.21.15.62:0/858646371 conn(0x55bd8a4eeda0 msgr2=0x55bd8a4f38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f2ff6ffd700 0 client.16578 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 -- 172.21.15.62:0/858646371 shutdown_connections 2023-07-14T16:19:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 1 -- 172.21.15.62:0/858646371 wait complete. 2023-07-14T16:19:43.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.577+0000 7f30317b0b80 20 client.16578 trim_cache size 0 max 16384 2023-07-14T16:19:43.564 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:19:43.564 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:19:43.564 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:19:43.564 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:19:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 -- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30000a11d0 msgr2=0x7f30000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30000a11d0 0x7f30000a3680 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3010008fd0 tx=0x7f301000d470 comp rx=0 tx=0).stop 2023-07-14T16:19:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 -- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3018072f10 msgr2=0x7f3018197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3018072f10 0x7f3018197910 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3008000c00 tx=0x7f3008003b00 comp rx=0 tx=0).stop 2023-07-14T16:19:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 -- 172.21.15.62:0/2498987594 shutdown_connections 2023-07-14T16:19:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3018107980 0x7f301819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3018072f10 0x7f3018197910 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f30000a11d0 0x7f30000a3680 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 --2- 172.21.15.62:0/2498987594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30180744f0 0x7f3018197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 -- 172.21.15.62:0/2498987594 >> 172.21.15.62:0/2498987594 conn(0x7f30180fb130 msgr2=0x7f30180fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 -- 172.21.15.62:0/2498987594 shutdown_connections 2023-07-14T16:19:43.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.578+0000 7f301ddff700 1 -- 172.21.15.62:0/2498987594 wait complete. 2023-07-14T16:19:43.574 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:19:43.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:19:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 -- 172.21.15.62:0/2668913162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 msgr2=0x7f7520103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 --2- 172.21.15.62:0/2668913162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 0x7f7520103360 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7510009990 tx=0x7f7510009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 -- 172.21.15.62:0/2668913162 shutdown_connections 2023-07-14T16:19:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 --2- 172.21.15.62:0/2668913162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75201038a0 0x7f752010a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 --2- 172.21.15.62:0/2668913162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7520101d20 0x7f7520102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 --2- 172.21.15.62:0/2668913162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 0x7f7520103360 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 -- 172.21.15.62:0/2668913162 >> 172.21.15.62:0/2668913162 conn(0x7f75200fd360 msgr2=0x7f75200ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 -- 172.21.15.62:0/2668913162 shutdown_connections 2023-07-14T16:19:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.831+0000 7f75285ff700 1 -- 172.21.15.62:0/2668913162 wait complete. 2023-07-14T16:19:43.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.833+0000 7f75285ff700 1 Processor -- start 2023-07-14T16:19:43.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.833+0000 7f75285ff700 1 -- start start 2023-07-14T16:19:43.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.833+0000 7f75285ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7520101d20 0x7f7520197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.833+0000 7f75285ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 0x7f7520197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f75285ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75201038a0 0x7f752019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f75285ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752019f4c0 con 0x7f7520102f10 2023-07-14T16:19:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f75285ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752019f600 con 0x7f75201038a0 2023-07-14T16:19:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f75285ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752019f790 con 0x7f7520101d20 2023-07-14T16:19:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7525b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 0x7f7520197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7525b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 0x7f7520197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35118/0 (socket says 172.21.15.62:35118) 2023-07-14T16:19:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7525b9a700 1 -- 172.21.15.62:0/2946167966 learned_addr learned my addr 172.21.15.62:0/2946167966 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f752639b700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7520101d20 0x7f7520197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7526b9c700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75201038a0 0x7f752019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7525b9a700 1 -- 172.21.15.62:0/2946167966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7520101d20 msgr2=0x7f7520197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7525b9a700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7520101d20 0x7f7520197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7525b9a700 1 -- 172.21.15.62:0/2946167966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75201038a0 msgr2=0x7f752019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7525b9a700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75201038a0 0x7f752019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7525b9a700 1 -- 172.21.15.62:0/2946167966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75100095f0 con 0x7f7520102f10 2023-07-14T16:19:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.834+0000 7f7526b9c700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75201038a0 0x7f752019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.835+0000 7f7525b9a700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 0x7f7520197e80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7510003ec0 tx=0x7f7510004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.835+0000 7f75177fe700 1 -- 172.21.15.62:0/2946167966 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751001d070 con 0x7f7520102f10 2023-07-14T16:19:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.835+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752019fa10 con 0x7f7520102f10 2023-07-14T16:19:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.835+0000 7f75177fe700 1 -- 172.21.15.62:0/2946167966 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f751000be90 con 0x7f7520102f10 2023-07-14T16:19:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.835+0000 7f75177fe700 1 -- 172.21.15.62:0/2946167966 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510022e80 con 0x7f7520102f10 2023-07-14T16:19:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.835+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752019fe90 con 0x7f7520102f10 2023-07-14T16:19:43.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.836+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75201092e0 con 0x7f7520102f10 2023-07-14T16:19:43.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.838+0000 7f75177fe700 1 -- 172.21.15.62:0/2946167966 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7510022650 con 0x7f7520102f10 2023-07-14T16:19:43.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.840+0000 7f75177fe700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f750c0a1220 0x7f750c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.840+0000 7f75177fe700 1 -- 172.21.15.62:0/2946167966 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f75100cdc90 con 0x7f7520102f10 2023-07-14T16:19:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.840+0000 7f752639b700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f750c0a1220 0x7f750c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.841+0000 7f752639b700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f750c0a1220 0x7f750c0a36d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7520102d70 tx=0x7f751c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:43.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:43.844+0000 7f75177fe700 1 -- 172.21.15.62:0/2946167966 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f751005c790 con 0x7f7520102f10 2023-07-14T16:19:44.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.110+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f752002c850 con 0x7f7520102f10 2023-07-14T16:19:44.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.111+0000 7f75177fe700 1 -- 172.21.15.62:0/2946167966 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10826 (secure 0 0 0) 0x7f751009aeb0 con 0x7f7520102f10 2023-07-14T16:19:44.088 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:19:44.088 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:17:18.727946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:19:44.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f750c0a1220 msgr2=0x7f750c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f750c0a1220 0x7f750c0a36d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7520102d70 tx=0x7f751c009260 comp rx=0 tx=0).stop 2023-07-14T16:19:44.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 msgr2=0x7f7520197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 0x7f7520197e80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7510003ec0 tx=0x7f7510004730 comp rx=0 tx=0).stop 2023-07-14T16:19:44.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 shutdown_connections 2023-07-14T16:19:44.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f750c0a1220 0x7f750c0a36d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75201038a0 0x7f752019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7520101d20 0x7f7520197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 --2- 172.21.15.62:0/2946167966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7520102f10 0x7f7520197e80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 >> 172.21.15.62:0/2946167966 conn(0x7f75200fd360 msgr2=0x7f7520106420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 shutdown_connections 2023-07-14T16:19:44.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.114+0000 7f75285ff700 1 -- 172.21.15.62:0/2946167966 wait complete. 2023-07-14T16:19:44.093 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 24 2023-07-14T16:19:44.115 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:19:44.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 -- 172.21.15.62:0/1838364704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 msgr2=0x7f67c8072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 --2- 172.21.15.62:0/1838364704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 0x7f67c8072940 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009990 tx=0x7f67c4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 -- 172.21.15.62:0/1838364704 shutdown_connections 2023-07-14T16:19:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 --2- 172.21.15.62:0/1838364704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67c8107880 0x7f67c8109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 --2- 172.21.15.62:0/1838364704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 0x7f67c8072940 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 --2- 172.21.15.62:0/1838364704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c8072f10 0x7f67c8073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 -- 172.21.15.62:0/1838364704 >> 172.21.15.62:0/1838364704 conn(0x7f67c80fb010 msgr2=0x7f67c80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 -- 172.21.15.62:0/1838364704 shutdown_connections 2023-07-14T16:19:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.379+0000 7f67ce79d700 1 -- 172.21.15.62:0/1838364704 wait complete. 2023-07-14T16:19:44.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.381+0000 7f67ce79d700 1 Processor -- start 2023-07-14T16:19:44.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.381+0000 7f67ce79d700 1 -- start start 2023-07-14T16:19:44.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.381+0000 7f67ce79d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67c8072f10 0x7f67c81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.381+0000 7f67ce79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 0x7f67c8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.381+0000 7f67ce79d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c8107880 0x7f67c819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67ce79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c819f260 con 0x7f67c80744f0 2023-07-14T16:19:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67ce79d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c819f3a0 con 0x7f67c8072f10 2023-07-14T16:19:44.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67ce79d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c819f530 con 0x7f67c8107880 2023-07-14T16:19:44.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67ccf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 0x7f67c8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cd79b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67c8072f10 0x7f67c81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67ccf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 0x7f67c8197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35144/0 (socket says 172.21.15.62:35144) 2023-07-14T16:19:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67ccf9a700 1 -- 172.21.15.62:0/3698592990 learned_addr learned my addr 172.21.15.62:0/3698592990 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67c8072f10 0x7f67c81976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:33888/0 (socket says 172.21.15.62:33888) 2023-07-14T16:19:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c8107880 0x7f67c819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cd79b700 1 -- 172.21.15.62:0/3698592990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c8107880 msgr2=0x7f67c819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c8107880 0x7f67c819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cd79b700 1 -- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 msgr2=0x7f67c8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 0x7f67c8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cd79b700 1 -- 172.21.15.62:0/3698592990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c40095f0 con 0x7f67c8072f10 2023-07-14T16:19:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.382+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c8107880 0x7f67c819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.383+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67c8072f10 0x7f67c81976e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67c4000c00 tx=0x7f67c4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.383+0000 7f67be7fc700 1 -- 172.21.15.62:0/3698592990 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c401d070 con 0x7f67c8072f10 2023-07-14T16:19:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.383+0000 7f67be7fc700 1 -- 172.21.15.62:0/3698592990 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67c400be30 con 0x7f67c8072f10 2023-07-14T16:19:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.383+0000 7f67be7fc700 1 -- 172.21.15.62:0/3698592990 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4022e10 con 0x7f67c8072f10 2023-07-14T16:19:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.383+0000 7f67ce79d700 1 -- 172.21.15.62:0/3698592990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c819f7b0 con 0x7f67c8072f10 2023-07-14T16:19:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.383+0000 7f67ce79d700 1 -- 172.21.15.62:0/3698592990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c819fbd0 con 0x7f67c8072f10 2023-07-14T16:19:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.387+0000 7f67be7fc700 1 -- 172.21.15.62:0/3698592990 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f67c40225e0 con 0x7f67c8072f10 2023-07-14T16:19:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.388+0000 7f67e33f0b80 1 Processor -- start 2023-07-14T16:19:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.388+0000 7f67e33f0b80 1 -- start start 2023-07-14T16:19:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a936fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370070 0x5633a93708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a93712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a9274cb0 con 0x5633a9370e30 2023-07-14T16:19:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a92c7a40 con 0x5633a9370070 2023-07-14T16:19:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a92c0ed0 con 0x5633a936f720 2023-07-14T16:19:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67ccf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a93712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67ccf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a93712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35150/0 (socket says 172.21.15.62:35150) 2023-07-14T16:19:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67ccf9a700 1 -- 172.21.15.62:0/3368692892 learned_addr learned my addr 172.21.15.62:0/3368692892 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67be7fc700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67b40a1270 0x7f67b40a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67be7fc700 1 -- 172.21.15.62:0/3698592990 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67c40cdc20 con 0x7f67c8072f10 2023-07-14T16:19:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a936fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.389+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370070 0x5633a93708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67b40a1270 0x7f67b40a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67ccf9a700 1 -- 172.21.15.62:0/3368692892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 msgr2=0x5633a936fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a936fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67ccf9a700 1 -- 172.21.15.62:0/3368692892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370070 msgr2=0x5633a93708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370070 0x5633a93708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67ccf9a700 1 -- 172.21.15.62:0/3368692892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633a90eede0 con 0x5633a9370e30 2023-07-14T16:19:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a936fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370070 0x5633a93708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a93712a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f67b800bea0 tx=0x7f67b80106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e43b0aac2fc383 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67b37fe700 1 -- 172.21.15.62:0/3368692892 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8011440 con 0x5633a9370e30 2023-07-14T16:19:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.390+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67b40a1270 0x7f67b40a3720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67c00126c0 tx=0x7f67c000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67b37fe700 1 -- 172.21.15.62:0/3368692892 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67b8015070 con 0x5633a9370e30 2023-07-14T16:19:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67b37fe700 1 -- 172.21.15.62:0/3368692892 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8019890 con 0x5633a9370e30 2023-07-14T16:19:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3368692892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 msgr2=0x5633a93712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a93712a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f67b800bea0 tx=0x7f67b80106b0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3368692892 shutdown_connections 2023-07-14T16:19:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a93712a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a936fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3368692892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370070 0x5633a93708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3368692892 >> 172.21.15.62:0/3368692892 conn(0x5633a9364ff0 msgr2=0x5633a936a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3368692892 shutdown_connections 2023-07-14T16:19:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.391+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3368692892 wait complete. 2023-07-14T16:19:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.392+0000 7f67e33f0b80 1 -- start start 2023-07-14T16:19:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.392+0000 7f67e33f0b80 1 Processor -- start 2023-07-14T16:19:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a936f720 0x5633a93792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 0x5633a93797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a9380830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a9260ac0 con 0x5633a9370e30 2023-07-14T16:19:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a92607c0 con 0x5633a936f720 2023-07-14T16:19:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a9144920 con 0x5633a9370070 2023-07-14T16:19:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67e33f0b80 10 client.0 _ll_register_callbacks cb 0x5633a925e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67cdf9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a9380830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67cd79b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a936f720 0x5633a93792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67cd79b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a936f720 0x5633a93792b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:33894/0 (socket says 172.21.15.62:33894) 2023-07-14T16:19:44.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67cd79b700 1 -- 172.21.15.62:0/2079868859 learned_addr learned my addr 172.21.15.62:0/2079868859 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.393+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 0x5633a93797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67cdf9c700 1 -- 172.21.15.62:0/2079868859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 msgr2=0x5633a93797f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 0x5633a93797f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67cdf9c700 1 -- 172.21.15.62:0/2079868859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a936f720 msgr2=0x5633a93792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a936f720 0x5633a93792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67cdf9c700 1 -- 172.21.15.62:0/2079868859 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b800bad0 con 0x5633a9370e30 2023-07-14T16:19:44.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67cd79b700 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a936f720 0x5633a93792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:44.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a9380830 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007bc0 tx=0x7f67c0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6abd95b5de9c1a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:44.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67b0ff9700 1 -- 172.21.15.62:0/2079868859 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c000e040 con 0x5633a9370e30 2023-07-14T16:19:44.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67e33f0b80 1 -- 172.21.15.62:0/2079868859 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633a921a4f0 con 0x5633a9370e30 2023-07-14T16:19:44.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.394+0000 7f67b0ff9700 1 -- 172.21.15.62:0/2079868859 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67c001e050 con 0x5633a9370e30 2023-07-14T16:19:44.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67b0ff9700 1 -- 172.21.15.62:0/2079868859 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67c00168d0 con 0x5633a9370e30 2023-07-14T16:19:44.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67e33f0b80 10 client.16596 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:44.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67e33f0b80 1 -- 172.21.15.62:0/2079868859 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5633a90d7c60 con 0x5633a9370e30 2023-07-14T16:19:44.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67b0ff9700 1 -- 172.21.15.62:0/2079868859 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5633a921a4f0 con 0x5633a9370e30 2023-07-14T16:19:44.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67b0ff9700 1 -- 172.21.15.62:0/2079868859 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f67c002fb30 con 0x5633a9370e30 2023-07-14T16:19:44.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67e33f0b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:44.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67e33f0b80 10 client.16596 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:19:44.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67e33f0b80 20 client.16596 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5633a9384fa0 0x5633a93873b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67e33f0b80 4 client.16596 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.395+0000 7f67e33f0b80 1 -- 172.21.15.62:0/2079868859 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633a92b9eb0 con 0x5633a9384fa0 2023-07-14T16:19:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.396+0000 7f67cd79b700 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5633a9384fa0 0x5633a93873b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.396+0000 7f67cd79b700 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5633a9384fa0 0x5633a93873b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef54bb970a804647 server_cookie=b45aa894867b7e9d in_seq=0 out_seq=0 2023-07-14T16:19:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.396+0000 7f67b0ff9700 10 client.16596 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:19:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.397+0000 7f67b0ff9700 1 -- 172.21.15.62:0/2079868859 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5633a92b9eb0 con 0x5633a9384fa0 2023-07-14T16:19:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.397+0000 7f67b0ff9700 10 client.16596 handle_command_reply: tid=0 2023-07-14T16:19:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.397+0000 7f67e33f0b80 1 client.16596 shutdown 2023-07-14T16:19:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 -- 172.21.15.62:0/2079868859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 msgr2=0x5633a9380830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a9380830 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007bc0 tx=0x7f67c0004b40 comp rx=0 tx=0).stop 2023-07-14T16:19:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 -- 172.21.15.62:0/2079868859 shutdown_connections 2023-07-14T16:19:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5633a9384fa0 0x5633a93873b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370e30 0x5633a9380830 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a936f720 0x5633a93792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/2079868859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 0x5633a93797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 -- 172.21.15.62:0/2079868859 >> 172.21.15.62:0/2079868859 conn(0x5633a9364ff0 msgr2=0x5633a9369b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67b0ff9700 0 client.16596 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:19:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 -- 172.21.15.62:0/2079868859 shutdown_connections 2023-07-14T16:19:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 1 -- 172.21.15.62:0/2079868859 wait complete. 2023-07-14T16:19:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.398+0000 7f67e33f0b80 20 client.16596 trim_cache size 0 max 16384 2023-07-14T16:19:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.413+0000 7f67e33f0b80 1 Processor -- start 2023-07-14T16:19:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67e33f0b80 1 -- start start 2023-07-14T16:19:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a9369b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370070 0x5633a93807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a9386fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a936b160 con 0x5633a9370070 2023-07-14T16:19:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a9382490 con 0x5633a9370e30 2023-07-14T16:19:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a936bbd0 con 0x5633a936f720 2023-07-14T16:19:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67ccf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370070 0x5633a93807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67ccf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370070 0x5633a93807c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35164/0 (socket says 172.21.15.62:35164) 2023-07-14T16:19:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67ccf9a700 1 -- 172.21.15.62:0/4282149626 learned_addr learned my addr 172.21.15.62:0/4282149626 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.414+0000 7f67cd79b700 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a9369b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a9386fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67ccf9a700 1 -- 172.21.15.62:0/4282149626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 msgr2=0x5633a9369b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a9369b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67ccf9a700 1 -- 172.21.15.62:0/4282149626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 msgr2=0x5633a9386fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a9386fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67ccf9a700 1 -- 172.21.15.62:0/4282149626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633a9115250 con 0x5633a9370070 2023-07-14T16:19:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370070 0x5633a93807c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f67b801a000 tx=0x7f67b800b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befdf079a778c25e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67b3fff700 1 -- 172.21.15.62:0/4282149626 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80052b0 con 0x5633a9370070 2023-07-14T16:19:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67b3fff700 1 -- 172.21.15.62:0/4282149626 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67b8005410 con 0x5633a9370070 2023-07-14T16:19:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.415+0000 7f67b3fff700 1 -- 172.21.15.62:0/4282149626 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8019740 con 0x5633a9370070 2023-07-14T16:19:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 -- 172.21.15.62:0/4282149626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370070 msgr2=0x5633a93807c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370070 0x5633a93807c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f67b801a000 tx=0x7f67b800b320 comp rx=0 tx=0).stop 2023-07-14T16:19:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 -- 172.21.15.62:0/4282149626 shutdown_connections 2023-07-14T16:19:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a9386fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a936f720 0x5633a9369b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/4282149626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a9370070 0x5633a93807c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 -- 172.21.15.62:0/4282149626 >> 172.21.15.62:0/4282149626 conn(0x5633a9364ff0 msgr2=0x5633a938bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 -- 172.21.15.62:0/4282149626 shutdown_connections 2023-07-14T16:19:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.416+0000 7f67e33f0b80 1 -- 172.21.15.62:0/4282149626 wait complete. 2023-07-14T16:19:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.417+0000 7f67e33f0b80 1 -- start start 2023-07-14T16:19:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.417+0000 7f67e33f0b80 1 Processor -- start 2023-07-14T16:19:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.417+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a936f720 0x5633a937c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.417+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 0x5633a937cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67e33f0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a937fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a9380190 con 0x5633a936f720 2023-07-14T16:19:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a9380300 con 0x5633a9370e30 2023-07-14T16:19:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67e33f0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a93804c0 con 0x5633a9370070 2023-07-14T16:19:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67e33f0b80 10 client.0 _ll_register_callbacks cb 0x7f67c0007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a936f720 0x5633a937c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67ccf9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 0x5633a937cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a936f720 0x5633a937c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35176/0 (socket says 172.21.15.62:35176) 2023-07-14T16:19:44.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cd79b700 1 -- 172.21.15.62:0/3761072543 learned_addr learned my addr 172.21.15.62:0/3761072543 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a937fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cd79b700 1 -- 172.21.15.62:0/3761072543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 msgr2=0x5633a937cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 0x5633a937cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cd79b700 1 -- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 msgr2=0x5633a937fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a937fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cd79b700 1 -- 172.21.15.62:0/3761072543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633a90eede0 con 0x5633a936f720 2023-07-14T16:19:44.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.418+0000 7f67cdf9c700 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a937fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:19:44.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67cd79b700 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a936f720 0x5633a937c9d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67c4003f60 tx=0x7f67c4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9970009aaab1e6bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:44.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67b0ff9700 1 -- 172.21.15.62:0/3761072543 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c409b160 con 0x5633a936f720 2023-07-14T16:19:44.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67b0ff9700 1 -- 172.21.15.62:0/3761072543 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67c409f190 con 0x5633a936f720 2023-07-14T16:19:44.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3761072543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67b8015070 con 0x5633a936f720 2023-07-14T16:19:44.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67b0ff9700 1 -- 172.21.15.62:0/3761072543 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67c40a3520 con 0x5633a936f720 2023-07-14T16:19:44.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67e33f0b80 10 client.16602 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:44.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3761072543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5633a90ed2b0 con 0x5633a936f720 2023-07-14T16:19:44.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67b0ff9700 1 -- 172.21.15.62:0/3761072543 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67b8015070 con 0x5633a936f720 2023-07-14T16:19:44.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.419+0000 7f67b0ff9700 1 -- 172.21.15.62:0/3761072543 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f67c40b9230 con 0x5633a936f720 2023-07-14T16:19:44.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.420+0000 7f67e33f0b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:44.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.420+0000 7f67e33f0b80 10 client.16602 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:19:44.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.420+0000 7f67e33f0b80 20 client.16602 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.420+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5633a93d9c40 0x5633a93dc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.420+0000 7f67e33f0b80 4 client.16602 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:19:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.420+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3761072543 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5633a927c050 con 0x5633a93d9c40 2023-07-14T16:19:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.420+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5633a93d9c40 0x5633a93dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67ccf9a700 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5633a93d9c40 0x5633a93dc050 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9f4f55fba622bc0 server_cookie=8769aadc36fc4e09 in_seq=0 out_seq=0 2023-07-14T16:19:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67b0ff9700 10 client.16602 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:19:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67b0ff9700 1 -- 172.21.15.62:0/3761072543 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5633a927c050 con 0x5633a93d9c40 2023-07-14T16:19:44.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67b0ff9700 10 client.16602 handle_command_reply: tid=0 2023-07-14T16:19:44.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 client.16602 shutdown 2023-07-14T16:19:44.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3761072543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a936f720 msgr2=0x5633a937c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a936f720 0x5633a937c9d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67c4003f60 tx=0x7f67c4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3761072543 shutdown_connections 2023-07-14T16:19:44.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5633a93d9c40 0x5633a93dc050 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5633a9370e30 0x5633a937fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5633a936f720 0x5633a937c9d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67b0ff9700 0 client.16602 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:19:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 --2- 172.21.15.62:0/3761072543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5633a9370070 0x5633a937cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3761072543 >> 172.21.15.62:0/3761072543 conn(0x5633a9364ff0 msgr2=0x5633a9369b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3761072543 shutdown_connections 2023-07-14T16:19:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.421+0000 7f67e33f0b80 1 -- 172.21.15.62:0/3761072543 wait complete. 2023-07-14T16:19:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67e33f0b80 20 client.16602 trim_cache size 0 max 16384 2023-07-14T16:19:44.407 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:19:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 -- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67b40a1270 msgr2=0x7f67b40a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67b40a1270 0x7f67b40a3720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67c00126c0 tx=0x7f67c000b040 comp rx=0 tx=0).stop 2023-07-14T16:19:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 -- 172.21.15.62:0/3698592990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67c8072f10 msgr2=0x7f67c81976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67c8072f10 0x7f67c81976e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67c4000c00 tx=0x7f67c4004730 comp rx=0 tx=0).stop 2023-07-14T16:19:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 -- 172.21.15.62:0/3698592990 shutdown_connections 2023-07-14T16:19:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c8107880 0x7f67c819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67b40a1270 0x7f67b40a3720 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67c8072f10 0x7f67c81976e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 --2- 172.21.15.62:0/3698592990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67c80744f0 0x7f67c8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.422+0000 7f67ce79d700 1 -- 172.21.15.62:0/3698592990 >> 172.21.15.62:0/3698592990 conn(0x7f67c80fb010 msgr2=0x7f67c80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.423+0000 7f67ce79d700 1 -- 172.21.15.62:0/3698592990 shutdown_connections 2023-07-14T16:19:44.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.423+0000 7f67ce79d700 1 -- 172.21.15.62:0/3698592990 wait complete. 2023-07-14T16:19:44.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:19:44.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.678+0000 7ff3a2bff700 1 -- 172.21.15.62:0/1331918180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 msgr2=0x7ff39c103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.678+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/1331918180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 0x7ff39c103130 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff388009990 tx=0x7ff388009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.678+0000 7ff3a2bff700 1 -- 172.21.15.62:0/1331918180 shutdown_connections 2023-07-14T16:19:44.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.678+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/1331918180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c103670 0x7ff39c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.678+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/1331918180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c0fe350 0x7ff39c100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.678+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/1331918180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 0x7ff39c103130 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.678+0000 7ff3a2bff700 1 -- 172.21.15.62:0/1331918180 >> 172.21.15.62:0/1331918180 conn(0x7ff39c0fa000 msgr2=0x7ff39c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.679+0000 7ff3a2bff700 1 -- 172.21.15.62:0/1331918180 shutdown_connections 2023-07-14T16:19:44.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.679+0000 7ff3a2bff700 1 -- 172.21.15.62:0/1331918180 wait complete. 2023-07-14T16:19:44.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.680+0000 7ff3a2bff700 1 Processor -- start 2023-07-14T16:19:44.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.680+0000 7ff3a2bff700 1 -- start start 2023-07-14T16:19:44.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff3a2bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c0fe350 0x7ff39c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff3a2bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 0x7ff39c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff3a2bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c103670 0x7ff39c1a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff3a2bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c1a3940 con 0x7ff39c100cb0 2023-07-14T16:19:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff3a2bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c1a3a80 con 0x7ff39c0fe350 2023-07-14T16:19:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff3a2bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c1a3c10 con 0x7ff39c103670 2023-07-14T16:19:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff39bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 0x7ff39c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff3a099b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c0fe350 0x7ff39c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff39bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 0x7ff39c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35206/0 (socket says 172.21.15.62:35206) 2023-07-14T16:19:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff39bfff700 1 -- 172.21.15.62:0/434312309 learned_addr learned my addr 172.21.15.62:0/434312309 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.681+0000 7ff3a119c700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c103670 0x7ff39c1a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff39bfff700 1 -- 172.21.15.62:0/434312309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c103670 msgr2=0x7ff39c1a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff39bfff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c103670 0x7ff39c1a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff39bfff700 1 -- 172.21.15.62:0/434312309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c0fe350 msgr2=0x7ff39c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff39bfff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c0fe350 0x7ff39c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff39bfff700 1 -- 172.21.15.62:0/434312309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3880095f0 con 0x7ff39c100cb0 2023-07-14T16:19:44.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff39bfff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 0x7ff39c19c300 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff388005910 tx=0x7ff388004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff399ffb700 1 -- 172.21.15.62:0/434312309 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38801d070 con 0x7ff39c100cb0 2023-07-14T16:19:44.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff399ffb700 1 -- 172.21.15.62:0/434312309 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff388004c70 con 0x7ff39c100cb0 2023-07-14T16:19:44.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff3a2bff700 1 -- 172.21.15.62:0/434312309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39c1a3e90 con 0x7ff39c100cb0 2023-07-14T16:19:44.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.682+0000 7ff399ffb700 1 -- 172.21.15.62:0/434312309 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388003a30 con 0x7ff39c100cb0 2023-07-14T16:19:44.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.683+0000 7ff3a2bff700 1 -- 172.21.15.62:0/434312309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39c1a42b0 con 0x7ff39c100cb0 2023-07-14T16:19:44.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.686+0000 7ff399ffb700 1 -- 172.21.15.62:0/434312309 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff388005000 con 0x7ff39c100cb0 2023-07-14T16:19:44.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.687+0000 7ff3b6603b80 1 Processor -- start 2023-07-14T16:19:44.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.687+0000 7ff3b6603b80 1 -- start start 2023-07-14T16:19:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89c4d0 0x55c3de89c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89ce20 0x55c3de89d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89dc70 0x55c3de89e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de7a1cb0 con 0x55c3de89ce20 2023-07-14T16:19:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de7f4a40 con 0x55c3de89c4d0 2023-07-14T16:19:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de7eded0 con 0x55c3de89dc70 2023-07-14T16:19:44.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff399ffb700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff38c0a11d0 0x7ff38c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3a099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89ce20 0x55c3de89d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3a099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89ce20 0x55c3de89d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35208/0 (socket says 172.21.15.62:35208) 2023-07-14T16:19:44.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff3a099b700 1 -- 172.21.15.62:0/2484636191 learned_addr learned my addr 172.21.15.62:0/2484636191 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.688+0000 7ff399ffb700 1 -- 172.21.15.62:0/434312309 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3880cd820 con 0x7ff39c100cb0 2023-07-14T16:19:44.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a119c700 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89c4d0 0x55c3de89c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff39bfff700 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89dc70 0x55c3de89e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a119c700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff38c0a11d0 0x7ff38c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a099b700 1 -- 172.21.15.62:0/2484636191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89dc70 msgr2=0x55c3de89e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89dc70 0x55c3de89e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a099b700 1 -- 172.21.15.62:0/2484636191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89c4d0 msgr2=0x55c3de89c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89c4d0 0x55c3de89c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a099b700 1 -- 172.21.15.62:0/2484636191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3de61bde0 con 0x55c3de89ce20 2023-07-14T16:19:44.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a119c700 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89c4d0 0x55c3de89c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:44.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89ce20 0x55c3de89d6e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff39000bd70 tx=0x7ff3900106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c438e3ed55178345 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.689+0000 7ff386ffd700 1 -- 172.21.15.62:0/2484636191 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390010ec0 con 0x55c3de89ce20 2023-07-14T16:19:44.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.690+0000 7ff386ffd700 1 -- 172.21.15.62:0/2484636191 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff390015070 con 0x55c3de89ce20 2023-07-14T16:19:44.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.690+0000 7ff386ffd700 1 -- 172.21.15.62:0/2484636191 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900116a0 con 0x55c3de89ce20 2023-07-14T16:19:44.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.690+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2484636191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89ce20 msgr2=0x55c3de89d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.690+0000 7ff3a119c700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff38c0a11d0 0x7ff38c0a3680 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff394009f20 tx=0x7ff394009100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.690+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89ce20 0x55c3de89d6e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff39000bd70 tx=0x7ff3900106b0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.690+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2484636191 shutdown_connections 2023-07-14T16:19:44.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.690+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89dc70 0x55c3de89e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.691+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89c4d0 0x55c3de89c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.691+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2484636191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89ce20 0x55c3de89d6e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.691+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2484636191 >> 172.21.15.62:0/2484636191 conn(0x55c3de891da0 msgr2=0x55c3de8971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.691+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2484636191 shutdown_connections 2023-07-14T16:19:44.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.691+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2484636191 wait complete. 2023-07-14T16:19:44.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.691+0000 7ff3b6603b80 1 -- start start 2023-07-14T16:19:44.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.691+0000 7ff3b6603b80 1 Processor -- start 2023-07-14T16:19:44.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89c4d0 0x55c3de8a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89ce20 0x55c3de8a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89dc70 0x55c3de8ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de78dac0 con 0x55c3de89dc70 2023-07-14T16:19:44.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de78d7c0 con 0x55c3de89ce20 2023-07-14T16:19:44.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de671920 con 0x55c3de89c4d0 2023-07-14T16:19:44.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3b6603b80 10 client.0 _ll_register_callbacks cb 0x55c3de78b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:44.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3a119c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89dc70 0x55c3de8ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3a119c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89dc70 0x55c3de8ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35214/0 (socket says 172.21.15.62:35214) 2023-07-14T16:19:44.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff39bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89ce20 0x55c3de8a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3a119c700 1 -- 172.21.15.62:0/2944584813 learned_addr learned my addr 172.21.15.62:0/2944584813 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.692+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89c4d0 0x55c3de8a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.693+0000 7ff3a119c700 1 -- 172.21.15.62:0/2944584813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89c4d0 msgr2=0x55c3de8a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.693+0000 7ff3a119c700 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89c4d0 0x55c3de8a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.693+0000 7ff3a119c700 1 -- 172.21.15.62:0/2944584813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89ce20 msgr2=0x55c3de8a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.693+0000 7ff3a119c700 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89ce20 0x55c3de8a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.693+0000 7ff3a119c700 1 -- 172.21.15.62:0/2944584813 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39000b9a0 con 0x55c3de89dc70 2023-07-14T16:19:44.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.693+0000 7ff3a119c700 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89dc70 0x55c3de8ad700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff394017c50 tx=0x7ff394004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb96a24e2eb5784 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.693+0000 7ff37bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:44.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.693+0000 7ff37bfff700 1 -- 172.21.15.62:0/2944584813 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394007ea0 con 0x55c3de89dc70 2023-07-14T16:19:44.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff37bfff700 1 -- 172.21.15.62:0/2944584813 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff394018ab0 con 0x55c3de89dc70 2023-07-14T16:19:44.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2944584813 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3de7474f0 con 0x55c3de89dc70 2023-07-14T16:19:44.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff37bfff700 1 -- 172.21.15.62:0/2944584813 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff394018c70 con 0x55c3de89dc70 2023-07-14T16:19:44.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 10 client.16614 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:44.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2944584813 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c3de604c60 con 0x55c3de89dc70 2023-07-14T16:19:44.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff37bfff700 1 -- 172.21.15.62:0/2944584813 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c3de7474f0 con 0x55c3de89dc70 2023-07-14T16:19:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff37bfff700 1 -- 172.21.15.62:0/2944584813 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff39402c400 con 0x55c3de89dc70 2023-07-14T16:19:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 10 client.16614 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:19:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 20 client.16614 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c3de8b2000 0x55c3de8b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 4 client.16614 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.694+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2944584813 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3de7e6eb0 con 0x55c3de8b2000 2023-07-14T16:19:44.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.695+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c3de8b2000 0x55c3de8b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.696+0000 7ff37bfff700 10 client.16614 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:19:44.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.696+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c3de8b2000 0x55c3de8b4410 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9311cf5aa070c43 server_cookie=3d077aa9f7ef350a in_seq=0 out_seq=0 2023-07-14T16:19:44.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff37bfff700 1 -- 172.21.15.62:0/2944584813 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c3de7e6eb0 con 0x55c3de8b2000 2023-07-14T16:19:44.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff37bfff700 10 client.16614 handle_command_reply: tid=0 2023-07-14T16:19:44.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 client.16614 shutdown 2023-07-14T16:19:44.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2944584813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89dc70 msgr2=0x55c3de8ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89dc70 0x55c3de8ad700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff394017c50 tx=0x7ff394004b40 comp rx=0 tx=0).stop 2023-07-14T16:19:44.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2944584813 shutdown_connections 2023-07-14T16:19:44.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c3de8b2000 0x55c3de8b4410 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89dc70 0x55c3de8ad700 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89c4d0 0x55c3de8a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2944584813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89ce20 0x55c3de8a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff37bfff700 0 client.16614 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:19:44.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2944584813 >> 172.21.15.62:0/2944584813 conn(0x55c3de891da0 msgr2=0x55c3de8968b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2944584813 shutdown_connections 2023-07-14T16:19:44.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2944584813 wait complete. 2023-07-14T16:19:44.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.697+0000 7ff3b6603b80 20 client.16614 trim_cache size 0 max 16384 2023-07-14T16:19:44.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.711+0000 7ff3b6603b80 1 Processor -- start 2023-07-14T16:19:44.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.711+0000 7ff3b6603b80 1 -- start start 2023-07-14T16:19:44.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.711+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.711+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 0x55c3de8ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.711+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 0x55c3de8a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de8af420 con 0x55c3de89c4d0 2023-07-14T16:19:44.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de8a3dd0 con 0x55c3de89dc70 2023-07-14T16:19:44.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de8a5590 con 0x55c3de89ce20 2023-07-14T16:19:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8a4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35230/0 (socket says 172.21.15.62:35230) 2023-07-14T16:19:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a099b700 1 -- 172.21.15.62:0/98372203 learned_addr learned my addr 172.21.15.62:0/98372203 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a119c700 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 0x55c3de8a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff39bfff700 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 0x55c3de8ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a099b700 1 -- 172.21.15.62:0/98372203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 msgr2=0x55c3de8ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a099b700 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 0x55c3de8ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a099b700 1 -- 172.21.15.62:0/98372203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 msgr2=0x55c3de8a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a099b700 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 0x55c3de8a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.712+0000 7ff3a099b700 1 -- 172.21.15.62:0/98372203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3de642250 con 0x55c3de89c4d0 2023-07-14T16:19:44.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.713+0000 7ff3a099b700 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8a4020 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff390010ab0 tx=0x7ff39000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9047b282653cc1f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.713+0000 7ff3877fe700 1 -- 172.21.15.62:0/98372203 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390020040 con 0x55c3de89c4d0 2023-07-14T16:19:44.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.713+0000 7ff3877fe700 1 -- 172.21.15.62:0/98372203 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff390010ec0 con 0x55c3de89c4d0 2023-07-14T16:19:44.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.713+0000 7ff3877fe700 1 -- 172.21.15.62:0/98372203 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390028290 con 0x55c3de89c4d0 2023-07-14T16:19:44.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.713+0000 7ff3b6603b80 1 -- 172.21.15.62:0/98372203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 msgr2=0x55c3de8a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8a4020 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff390010ab0 tx=0x7ff39000b410 comp rx=0 tx=0).stop 2023-07-14T16:19:44.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 -- 172.21.15.62:0/98372203 shutdown_connections 2023-07-14T16:19:44.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 0x55c3de8a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8a4020 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/98372203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 0x55c3de8ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 -- 172.21.15.62:0/98372203 >> 172.21.15.62:0/98372203 conn(0x55c3de891da0 msgr2=0x55c3de8a4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 -- 172.21.15.62:0/98372203 shutdown_connections 2023-07-14T16:19:44.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 -- 172.21.15.62:0/98372203 wait complete. 2023-07-14T16:19:44.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 -- start start 2023-07-14T16:19:44.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.714+0000 7ff3b6603b80 1 Processor -- start 2023-07-14T16:19:44.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 0x55c3de8ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3b6603b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 0x55c3de8a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de8a9e10 con 0x55c3de89c4d0 2023-07-14T16:19:44.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de8a9f80 con 0x55c3de89dc70 2023-07-14T16:19:44.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3b6603b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de8aa0f0 con 0x55c3de89ce20 2023-07-14T16:19:44.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3b6603b80 10 client.0 _ll_register_callbacks cb 0x7ff394017800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:44.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3a099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3a099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35240/0 (socket says 172.21.15.62:35240) 2023-07-14T16:19:44.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff3a099b700 1 -- 172.21.15.62:0/2834637675 learned_addr learned my addr 172.21.15.62:0/2834637675 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.715+0000 7ff39bfff700 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 0x55c3de8ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff3a119c700 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 0x55c3de8a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff3a099b700 1 -- 172.21.15.62:0/2834637675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 msgr2=0x55c3de8ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 0x55c3de8ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff3a099b700 1 -- 172.21.15.62:0/2834637675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 msgr2=0x55c3de8a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 0x55c3de8a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff3a099b700 1 -- 172.21.15.62:0/2834637675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3de61bde0 con 0x55c3de89c4d0 2023-07-14T16:19:44.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff3a099b700 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8ace20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff39000bd40 tx=0x7ff3900285b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46dba65ce1cd9c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff37bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:44.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff37bfff700 1 -- 172.21.15.62:0/2834637675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390020070 con 0x55c3de89c4d0 2023-07-14T16:19:44.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff37bfff700 1 -- 172.21.15.62:0/2834637675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff390017070 con 0x55c3de89c4d0 2023-07-14T16:19:44.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.716+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2834637675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3de7a9050 con 0x55c3de89c4d0 2023-07-14T16:19:44.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff37bfff700 1 -- 172.21.15.62:0/2834637675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff390005270 con 0x55c3de89c4d0 2023-07-14T16:19:44.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff3b6603b80 10 client.16620 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:44.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2834637675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c3de61a2b0 con 0x55c3de89c4d0 2023-07-14T16:19:44.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff37bfff700 1 -- 172.21.15.62:0/2834637675 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c3de7a9050 con 0x55c3de89c4d0 2023-07-14T16:19:44.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff37bfff700 1 -- 172.21.15.62:0/2834637675 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff39001c480 con 0x55c3de89c4d0 2023-07-14T16:19:44.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff3b6603b80 10 client.16620 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:44.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff3b6603b80 10 client.16620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:19:44.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff3b6603b80 20 client.16620 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:44.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c3de906f70 0x55c3de909380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff3b6603b80 4 client.16620 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:19:44.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.717+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2834637675 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3de63cf70 con 0x55c3de906f70 2023-07-14T16:19:44.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.718+0000 7ff39bfff700 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c3de906f70 0x55c3de909380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.718+0000 7ff39bfff700 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c3de906f70 0x55c3de909380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ba23aa8a0ed33c server_cookie=c01c0e08634d7bdb in_seq=0 out_seq=0 2023-07-14T16:19:44.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.718+0000 7ff37bfff700 10 client.16620 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:19:44.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.718+0000 7ff37bfff700 1 -- 172.21.15.62:0/2834637675 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3de63cf70 con 0x55c3de906f70 2023-07-14T16:19:44.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.718+0000 7ff37bfff700 10 client.16620 handle_command_reply: tid=0 2023-07-14T16:19:44.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.718+0000 7ff3b6603b80 1 client.16620 shutdown 2023-07-14T16:19:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2834637675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 msgr2=0x55c3de8ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8ace20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff39000bd40 tx=0x7ff3900285b0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2834637675 shutdown_connections 2023-07-14T16:19:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c3de906f70 0x55c3de909380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3de89dc70 0x55c3de8a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3de89c4d0 0x55c3de8ace20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff37bfff700 0 client.16620 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:19:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 --2- 172.21.15.62:0/2834637675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c3de89ce20 0x55c3de8ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2834637675 >> 172.21.15.62:0/2834637675 conn(0x55c3de891da0 msgr2=0x55c3de8a4020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2834637675 shutdown_connections 2023-07-14T16:19:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 1 -- 172.21.15.62:0/2834637675 wait complete. 2023-07-14T16:19:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.719+0000 7ff3b6603b80 20 client.16620 trim_cache size 0 max 16384 2023-07-14T16:19:44.703 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:19:44.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 -- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff38c0a11d0 msgr2=0x7ff38c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff38c0a11d0 0x7ff38c0a3680 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff394009f20 tx=0x7ff394009100 comp rx=0 tx=0).stop 2023-07-14T16:19:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 -- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 msgr2=0x7ff39c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 0x7ff39c19c300 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff388005910 tx=0x7ff388004730 comp rx=0 tx=0).stop 2023-07-14T16:19:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 -- 172.21.15.62:0/434312309 shutdown_connections 2023-07-14T16:19:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c103670 0x7ff39c1a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff38c0a11d0 0x7ff38c0a3680 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c0fe350 0x7ff39c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 --2- 172.21.15.62:0/434312309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c100cb0 0x7ff39c19c300 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 -- 172.21.15.62:0/434312309 >> 172.21.15.62:0/434312309 conn(0x7ff39c0fa000 msgr2=0x7ff39c0feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 -- 172.21.15.62:0/434312309 shutdown_connections 2023-07-14T16:19:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.720+0000 7ff3a2bff700 1 -- 172.21.15.62:0/434312309 wait complete. 2023-07-14T16:19:44.713 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:44 smithi062 ceph-mon[95543]: pgmap v480: 97 pgs: 97 active+clean; 2.0 GiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:19:44.713 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:44 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2946167966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:19:44.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:19:44.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:44 smithi081 ceph-mon[127795]: pgmap v480: 97 pgs: 97 active+clean; 2.0 GiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:19:44.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:44 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2946167966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:19:44.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 -- 172.21.15.62:0/1844217776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a4102e80 msgr2=0x7f16a4105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 --2- 172.21.15.62:0/1844217776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a4102e80 0x7f16a4105260 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1694009990 tx=0x7f1694009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 -- 172.21.15.62:0/1844217776 shutdown_connections 2023-07-14T16:19:44.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 --2- 172.21.15.62:0/1844217776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16a41057a0 0x7f16a4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 --2- 172.21.15.62:0/1844217776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16a4100560 0x7f16a4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 --2- 172.21.15.62:0/1844217776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a4102e80 0x7f16a4105260 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 -- 172.21.15.62:0/1844217776 >> 172.21.15.62:0/1844217776 conn(0x7f16a40fa000 msgr2=0x7f16a40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 -- 172.21.15.62:0/1844217776 shutdown_connections 2023-07-14T16:19:44.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.972+0000 7f16abbff700 1 -- 172.21.15.62:0/1844217776 wait complete. 2023-07-14T16:19:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.973+0000 7f16abbff700 1 Processor -- start 2023-07-14T16:19:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.974+0000 7f16abbff700 1 -- start start 2023-07-14T16:19:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.974+0000 7f16abbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16a4100560 0x7f16a4195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.974+0000 7f16abbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16a4102e80 0x7f16a4195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.974+0000 7f16abbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a41057a0 0x7f16a419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.974+0000 7f16abbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a419d2c0 con 0x7f16a41057a0 2023-07-14T16:19:44.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.974+0000 7f16abbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a419d400 con 0x7f16a4100560 2023-07-14T16:19:44.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.974+0000 7f16abbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a419d590 con 0x7f16a4102e80 2023-07-14T16:19:44.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.974+0000 7f16aa19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a41057a0 0x7f16a419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16a919a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16a4102e80 0x7f16a4195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16aa19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a41057a0 0x7f16a419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35254/0 (socket says 172.21.15.62:35254) 2023-07-14T16:19:44.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16aa19c700 1 -- 172.21.15.62:0/20225926 learned_addr learned my addr 172.21.15.62:0/20225926 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16a999b700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16a4100560 0x7f16a4195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16aa19c700 1 -- 172.21.15.62:0/20225926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16a4102e80 msgr2=0x7f16a4195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16aa19c700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16a4102e80 0x7f16a4195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16aa19c700 1 -- 172.21.15.62:0/20225926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16a4100560 msgr2=0x7f16a4195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16aa19c700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16a4100560 0x7f16a4195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16aa19c700 1 -- 172.21.15.62:0/20225926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16940095f0 con 0x7f16a41057a0 2023-07-14T16:19:44.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f16aa19c700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a41057a0 0x7f16a419ccf0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f169c00ef30 tx=0x7f169c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.975+0000 7f169affd700 1 -- 172.21.15.62:0/20225926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c018070 con 0x7f16a41057a0 2023-07-14T16:19:44.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.976+0000 7f16abbff700 1 -- 172.21.15.62:0/20225926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a419d810 con 0x7f16a41057a0 2023-07-14T16:19:44.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.976+0000 7f169affd700 1 -- 172.21.15.62:0/20225926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f169c00f040 con 0x7f16a41057a0 2023-07-14T16:19:44.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.976+0000 7f169affd700 1 -- 172.21.15.62:0/20225926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00cb80 con 0x7f16a41057a0 2023-07-14T16:19:44.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.976+0000 7f16abbff700 1 -- 172.21.15.62:0/20225926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a419dd70 con 0x7f16a41057a0 2023-07-14T16:19:44.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.981+0000 7f169affd700 1 -- 172.21.15.62:0/20225926 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f169c00cd60 con 0x7f16a41057a0 2023-07-14T16:19:44.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.981+0000 7f16bf591b80 1 Processor -- start 2023-07-14T16:19:44.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16bf591b80 1 -- start start 2023-07-14T16:19:44.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dc8e0 0x55c5c79dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79de4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16bf591b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c78ddcb0 con 0x55c5c79de080 2023-07-14T16:19:44.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16bf591b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7930a40 con 0x55c5c79dc8e0 2023-07-14T16:19:44.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16bf591b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7929ed0 con 0x55c5c79dd230 2023-07-14T16:19:44.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16a919a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79de4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16a919a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79de4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35264/0 (socket says 172.21.15.62:35264) 2023-07-14T16:19:44.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16a919a700 1 -- 172.21.15.62:0/2686784996 learned_addr learned my addr 172.21.15.62:0/2686784996 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.982+0000 7f16a999b700 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f16a919a700 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dc8e0 0x55c5c79dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f16a999b700 1 -- 172.21.15.62:0/2686784996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dc8e0 msgr2=0x55c5c79dccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f16a999b700 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dc8e0 0x55c5c79dccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f16a999b700 1 -- 172.21.15.62:0/2686784996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 msgr2=0x55c5c79de4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f16a999b700 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79de4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f16a999b700 1 -- 172.21.15.62:0/2686784996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c7757de0 con 0x55c5c79dd230 2023-07-14T16:19:44.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f169affd700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16900a11d0 0x7f16900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f169affd700 1 -- 172.21.15.62:0/20225926 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f169c0cc310 con 0x7f16a41057a0 2023-07-14T16:19:44.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f16a999b700 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ddaf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009cb0 tx=0x7f16a0011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbfb998ab65b2d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.983+0000 7f16a919a700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16900a11d0 0x7f16900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.984+0000 7f168ffff700 1 -- 172.21.15.62:0/2686784996 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0011d20 con 0x55c5c79dd230 2023-07-14T16:19:44.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.984+0000 7f168ffff700 1 -- 172.21.15.62:0/2686784996 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16a0011e80 con 0x55c5c79dd230 2023-07-14T16:19:44.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.984+0000 7f168ffff700 1 -- 172.21.15.62:0/2686784996 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0005280 con 0x55c5c79dd230 2023-07-14T16:19:44.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 -- 172.21.15.62:0/2686784996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 msgr2=0x55c5c79ddaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ddaf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009cb0 tx=0x7f16a0011440 comp rx=0 tx=0).stop 2023-07-14T16:19:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16a919a700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16900a11d0 0x7f16900a3680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1694005930 tx=0x7f169401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 -- 172.21.15.62:0/2686784996 shutdown_connections 2023-07-14T16:19:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79de4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dc8e0 0x55c5c79dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2686784996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ddaf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 -- 172.21.15.62:0/2686784996 >> 172.21.15.62:0/2686784996 conn(0x55c5c79cdda0 msgr2=0x55c5c79d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 -- 172.21.15.62:0/2686784996 shutdown_connections 2023-07-14T16:19:44.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 -- 172.21.15.62:0/2686784996 wait complete. 2023-07-14T16:19:44.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.985+0000 7f16bf591b80 1 -- start start 2023-07-14T16:19:44.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.986+0000 7f16bf591b80 1 Processor -- start 2023-07-14T16:19:44.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.986+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.986+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.986+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79de080 0x55c5c79eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.986+0000 7f16bf591b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c78c9ac0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.986+0000 7f16bf591b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c78c97c0 con 0x55c5c79dd230 2023-07-14T16:19:44.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.986+0000 7f16bf591b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c77ad920 con 0x55c5c79de080 2023-07-14T16:19:44.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.986+0000 7f16bf591b80 10 client.0 _ll_register_callbacks cb 0x55c5c78c7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:44.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35278/0 (socket says 172.21.15.62:35278) 2023-07-14T16:19:44.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 -- 172.21.15.62:0/2801489271 learned_addr learned my addr 172.21.15.62:0/2801489271 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a919a700 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16aa19c700 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79de080 0x55c5c79eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 -- 172.21.15.62:0/2801489271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79de080 msgr2=0x55c5c79eda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79de080 0x55c5c79eda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 -- 172.21.15.62:0/2801489271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 msgr2=0x55c5c79e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 -- 172.21.15.62:0/2801489271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a00098e0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.987+0000 7f16a999b700 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e64e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f16a0005c50 tx=0x7f16a0011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc0153c9865d3dc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.988+0000 7f168d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:44.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.988+0000 7f168d7fa700 1 -- 172.21.15.62:0/2801489271 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0004260 con 0x55c5c79dc8e0 2023-07-14T16:19:44.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.988+0000 7f168d7fa700 1 -- 172.21.15.62:0/2801489271 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16a00043c0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.988+0000 7f16bf591b80 1 -- 172.21.15.62:0/2801489271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5c7922eb0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.988+0000 7f168d7fa700 1 -- 172.21.15.62:0/2801489271 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a0004580 con 0x55c5c79dc8e0 2023-07-14T16:19:44.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.988+0000 7f16bf591b80 10 client.16632 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:44.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.988+0000 7f16bf591b80 1 -- 172.21.15.62:0/2801489271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c5c7740c60 con 0x55c5c79dc8e0 2023-07-14T16:19:44.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.988+0000 7f168d7fa700 1 -- 172.21.15.62:0/2801489271 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c5c7922eb0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.989+0000 7f168d7fa700 1 -- 172.21.15.62:0/2801489271 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f16a00051e0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.989+0000 7f16bf591b80 10 client.16632 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:44.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.989+0000 7f16bf591b80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:44.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.989+0000 7f16bf591b80 20 client.16632 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:44.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.989+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c5c79f2170 0x55c5c79f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.989+0000 7f16bf591b80 4 client.16632 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:44.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.989+0000 7f16bf591b80 1 -- 172.21.15.62:0/2801489271 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5c78e5050 con 0x55c5c79f2170 2023-07-14T16:19:44.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.989+0000 7f16a919a700 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c5c79f2170 0x55c5c79f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.990+0000 7f16a919a700 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c5c79f2170 0x55c5c79f4580 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=561fefa33cdd4eb4 server_cookie=d3470d08ee38c8cf in_seq=0 out_seq=0 2023-07-14T16:19:44.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.990+0000 7f168d7fa700 10 client.16632 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:44.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f168d7fa700 1 -- 172.21.15.62:0/2801489271 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c5c78e5050 con 0x55c5c79f2170 2023-07-14T16:19:44.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f168d7fa700 10 client.16632 handle_command_reply: tid=0 2023-07-14T16:19:44.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 client.16632 shutdown 2023-07-14T16:19:44.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 -- 172.21.15.62:0/2801489271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 msgr2=0x55c5c79e64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e64e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f16a0005c50 tx=0x7f16a0011c30 comp rx=0 tx=0).stop 2023-07-14T16:19:44.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 -- 172.21.15.62:0/2801489271 shutdown_connections 2023-07-14T16:19:44.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79de080 0x55c5c79eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e64e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c5c79f2170 0x55c5c79f4580 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 --2- 172.21.15.62:0/2801489271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f168d7fa700 0 client.16632 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:44.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.991+0000 7f16bf591b80 1 -- 172.21.15.62:0/2801489271 >> 172.21.15.62:0/2801489271 conn(0x55c5c79cdda0 msgr2=0x55c5c79d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.992+0000 7f16bf591b80 1 -- 172.21.15.62:0/2801489271 shutdown_connections 2023-07-14T16:19:44.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.992+0000 7f16bf591b80 1 -- 172.21.15.62:0/2801489271 wait complete. 2023-07-14T16:19:44.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:44.992+0000 7f16bf591b80 20 client.16632 trim_cache size 0 max 16384 2023-07-14T16:19:44.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.009+0000 7f16bf591b80 1 Processor -- start 2023-07-14T16:19:44.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.009+0000 7f16bf591b80 1 -- start start 2023-07-14T16:19:44.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79eda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16bf591b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c79dd9b0 con 0x55c5c79de080 2023-07-14T16:19:44.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16bf591b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c79f1ea0 con 0x55c5c79dd230 2023-07-14T16:19:44.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16bf591b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c79e5b90 con 0x55c5c79dc8e0 2023-07-14T16:19:44.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16aa19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16aa19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79e67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35284/0 (socket says 172.21.15.62:35284) 2023-07-14T16:19:44.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16aa19c700 1 -- 172.21.15.62:0/1945199126 learned_addr learned my addr 172.21.15.62:0/1945199126 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16a919a700 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.010+0000 7f16a999b700 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f16aa19c700 1 -- 172.21.15.62:0/1945199126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dc8e0 msgr2=0x55c5c79e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f16aa19c700 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f16aa19c700 1 -- 172.21.15.62:0/1945199126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 msgr2=0x55c5c79eda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f16aa19c700 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79eda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f16aa19c700 1 -- 172.21.15.62:0/1945199126 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c777e250 con 0x55c5c79de080 2023-07-14T16:19:44.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f16a919a700 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79eda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:44.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f16a999b700 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:44.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f16aa19c700 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79e67d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f169c095c70 tx=0x7f169c099240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3b978471c40008 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f1698bff700 1 -- 172.21.15.62:0/1945199126 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0092e0 con 0x55c5c79de080 2023-07-14T16:19:44.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f1698bff700 1 -- 172.21.15.62:0/1945199126 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f169c097900 con 0x55c5c79de080 2023-07-14T16:19:44.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.011+0000 7f1698bff700 1 -- 172.21.15.62:0/1945199126 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c098f10 con 0x55c5c79de080 2023-07-14T16:19:44.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 -- 172.21.15.62:0/1945199126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 msgr2=0x55c5c79e67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:44.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79e67d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f169c095c70 tx=0x7f169c099240 comp rx=0 tx=0).stop 2023-07-14T16:19:44.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 -- 172.21.15.62:0/1945199126 shutdown_connections 2023-07-14T16:19:44.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79de080 0x55c5c79e67d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dc8e0 0x55c5c79e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 --2- 172.21.15.62:0/1945199126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79dd230 0x55c5c79eda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 -- 172.21.15.62:0/1945199126 >> 172.21.15.62:0/1945199126 conn(0x55c5c79cdda0 msgr2=0x55c5c79e0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:44.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 -- 172.21.15.62:0/1945199126 shutdown_connections 2023-07-14T16:19:44.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 -- 172.21.15.62:0/1945199126 wait complete. 2023-07-14T16:19:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.012+0000 7f16bf591b80 1 -- start start 2023-07-14T16:19:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16bf591b80 1 Processor -- start 2023-07-14T16:19:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79eda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16bf591b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79de080 0x55c5c79eab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16bf591b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7a46020 con 0x55c5c79dc8e0 2023-07-14T16:19:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16bf591b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7a46160 con 0x55c5c79de080 2023-07-14T16:19:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16bf591b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c7a462f0 con 0x55c5c79dd230 2023-07-14T16:19:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16bf591b80 10 client.0 _ll_register_callbacks cb 0x7f16a0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16a999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16a999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79eda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35294/0 (socket says 172.21.15.62:35294) 2023-07-14T16:19:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.013+0000 7f16a999b700 1 -- 172.21.15.62:0/3401780552 learned_addr learned my addr 172.21.15.62:0/3401780552 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16aa19c700 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79de080 0x55c5c79eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16a919a700 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16a999b700 1 -- 172.21.15.62:0/3401780552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 msgr2=0x55c5c79ea4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16a999b700 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ea4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16a999b700 1 -- 172.21.15.62:0/3401780552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79de080 msgr2=0x55c5c79eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16a999b700 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79de080 0x55c5c79eab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16a999b700 1 -- 172.21.15.62:0/3401780552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a419dd70 con 0x55c5c79dc8e0 2023-07-14T16:19:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16a919a700 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ea4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f16a999b700 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79eda20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f16a0005a80 tx=0x7f16a0004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaeb258bbc31f955 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f168d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.014+0000 7f168d7fa700 1 -- 172.21.15.62:0/3401780552 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0009260 con 0x55c5c79dc8e0 2023-07-14T16:19:44.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f168d7fa700 1 -- 172.21.15.62:0/3401780552 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16a0000e40 con 0x55c5c79dc8e0 2023-07-14T16:19:44.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f16bf591b80 1 -- 172.21.15.62:0/3401780552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5c78834f0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f168d7fa700 1 -- 172.21.15.62:0/3401780552 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a0011ca0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f16bf591b80 10 client.16638 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:44.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f16bf591b80 1 -- 172.21.15.62:0/3401780552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c5c77562b0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f168d7fa700 1 -- 172.21.15.62:0/3401780552 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c5c78834f0 con 0x55c5c79dc8e0 2023-07-14T16:19:44.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f168d7fa700 1 -- 172.21.15.62:0/3401780552 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f16a0041070 con 0x55c5c79dc8e0 2023-07-14T16:19:44.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f16bf591b80 10 client.16638 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:44.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f16bf591b80 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:44.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f16bf591b80 20 client.16638 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:44.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.015+0000 7f16bf591b80 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c5c7a479d0 0x55c5c7a49de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:44.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.016+0000 7f16bf591b80 4 client.16638 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:19:44.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.016+0000 7f16bf591b80 1 -- 172.21.15.62:0/3401780552 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5c7778f70 con 0x55c5c7a479d0 2023-07-14T16:19:44.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.016+0000 7f16a919a700 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c5c7a479d0 0x55c5c7a49de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:45.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f168d7fa700 10 client.16638 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:45.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16a919a700 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c5c7a479d0 0x55c5c7a49de0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16775efb141c3e40 server_cookie=88a5ef87698beb61 in_seq=0 out_seq=0 2023-07-14T16:19:45.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f168d7fa700 1 -- 172.21.15.62:0/3401780552 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5c7778f70 con 0x55c5c7a479d0 2023-07-14T16:19:45.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f168d7fa700 10 client.16638 handle_command_reply: tid=0 2023-07-14T16:19:45.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 client.16638 shutdown 2023-07-14T16:19:45.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 -- 172.21.15.62:0/3401780552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 msgr2=0x55c5c79eda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:45.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79eda20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f16a0005a80 tx=0x7f16a0004de0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 -- 172.21.15.62:0/3401780552 shutdown_connections 2023-07-14T16:19:45.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c5c7a479d0 0x55c5c7a49de0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c5c79de080 0x55c5c79eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c5c79dc8e0 0x55c5c79eda20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 --2- 172.21.15.62:0/3401780552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5c79dd230 0x55c5c79ea4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 -- 172.21.15.62:0/3401780552 >> 172.21.15.62:0/3401780552 conn(0x55c5c79cdda0 msgr2=0x55c5c79e6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:45.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f168d7fa700 0 client.16638 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:45.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 -- 172.21.15.62:0/3401780552 shutdown_connections 2023-07-14T16:19:45.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.017+0000 7f16bf591b80 1 -- 172.21.15.62:0/3401780552 wait complete. 2023-07-14T16:19:45.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16bf591b80 20 client.16638 trim_cache size 0 max 16384 2023-07-14T16:19:45.003 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:19:45.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 -- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16900a11d0 msgr2=0x7f16900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16900a11d0 0x7f16900a3680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1694005930 tx=0x7f169401a040 comp rx=0 tx=0).stop 2023-07-14T16:19:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 -- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a41057a0 msgr2=0x7f16a419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a41057a0 0x7f16a419ccf0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f169c00ef30 tx=0x7f169c00c3a0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 -- 172.21.15.62:0/20225926 shutdown_connections 2023-07-14T16:19:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a41057a0 0x7f16a419ccf0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16900a11d0 0x7f16900a3680 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16a4100560 0x7f16a4195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 --2- 172.21.15.62:0/20225926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16a4102e80 0x7f16a4195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.018+0000 7f16abbff700 1 -- 172.21.15.62:0/20225926 >> 172.21.15.62:0/20225926 conn(0x7f16a40fa000 msgr2=0x7f16a41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.019+0000 7f16abbff700 1 -- 172.21.15.62:0/20225926 shutdown_connections 2023-07-14T16:19:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:45.019+0000 7f16abbff700 1 -- 172.21.15.62:0/20225926 wait complete. 2023-07-14T16:19:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:44 smithi164 ceph-mon[97427]: pgmap v480: 97 pgs: 97 active+clean; 2.0 GiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:19:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:44 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2946167966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:19:46.017 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:19:46.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:19:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 -- 172.21.15.62:0/2490808549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 msgr2=0x7f9f54102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2490808549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 0x7f9f54102950 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009990 tx=0x7f9f44009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 -- 172.21.15.62:0/2490808549 shutdown_connections 2023-07-14T16:19:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2490808549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f541057b0 0x7f9f54107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2490808549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 0x7f9f54102950 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2490808549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f54102e90 0x7f9f54105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 -- 172.21.15.62:0/2490808549 >> 172.21.15.62:0/2490808549 conn(0x7f9f540fa010 msgr2=0x7f9f540fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 -- 172.21.15.62:0/2490808549 shutdown_connections 2023-07-14T16:19:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.272+0000 7f9f59dff700 1 -- 172.21.15.62:0/2490808549 wait complete. 2023-07-14T16:19:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.273+0000 7f9f59dff700 1 Processor -- start 2023-07-14T16:19:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.273+0000 7f9f59dff700 1 -- start start 2023-07-14T16:19:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.273+0000 7f9f59dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 0x7f9f54197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f59dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f54102e90 0x7f9f54197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f59dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f541057b0 0x7f9f5419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f59dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419f490 con 0x7f9f54100570 2023-07-14T16:19:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f59dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419f5d0 con 0x7f9f54102e90 2023-07-14T16:19:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f59dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419f760 con 0x7f9f541057b0 2023-07-14T16:19:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 0x7f9f54197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 0x7f9f54197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35310/0 (socket says 172.21.15.62:35310) 2023-07-14T16:19:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f537fe700 1 -- 172.21.15.62:0/2369969510 learned_addr learned my addr 172.21.15.62:0/2369969510 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f54102e90 0x7f9f54197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f537fe700 1 -- 172.21.15.62:0/2369969510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f541057b0 msgr2=0x7f9f5419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f537fe700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f541057b0 0x7f9f5419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f537fe700 1 -- 172.21.15.62:0/2369969510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f54102e90 msgr2=0x7f9f54197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f537fe700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f54102e90 0x7f9f54197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f537fe700 1 -- 172.21.15.62:0/2369969510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f440095f0 con 0x7f9f54100570 2023-07-14T16:19:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.274+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f54102e90 0x7f9f54197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.275+0000 7f9f537fe700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 0x7f9f54197910 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9f44006010 tx=0x7f9f44003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.275+0000 7f9f50ff9700 1 -- 172.21.15.62:0/2369969510 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4401d070 con 0x7f9f54100570 2023-07-14T16:19:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.275+0000 7f9f50ff9700 1 -- 172.21.15.62:0/2369969510 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f4400bd70 con 0x7f9f54100570 2023-07-14T16:19:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.275+0000 7f9f59dff700 1 -- 172.21.15.62:0/2369969510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5419f9e0 con 0x7f9f54100570 2023-07-14T16:19:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.275+0000 7f9f50ff9700 1 -- 172.21.15.62:0/2369969510 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44022da0 con 0x7f9f54100570 2023-07-14T16:19:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.275+0000 7f9f59dff700 1 -- 172.21.15.62:0/2369969510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5419fe60 con 0x7f9f54100570 2023-07-14T16:19:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.280+0000 7f9f50ff9700 1 -- 172.21.15.62:0/2369969510 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9f44003f30 con 0x7f9f54100570 2023-07-14T16:19:46.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.281+0000 7f9f6d6b4b80 1 Processor -- start 2023-07-14T16:19:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.281+0000 7f9f6d6b4b80 1 -- start start 2023-07-14T16:19:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.281+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8669960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.281+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e866a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.281+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e866b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.281+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e856ed30 con 0x5599e8669ea0 2023-07-14T16:19:46.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.281+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e85c1ac0 con 0x5599e866acf0 2023-07-14T16:19:46.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.281+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e85baf50 con 0x5599e8669550 2023-07-14T16:19:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f53fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8669960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f50ff9700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9f3c0a11d0 0x7f9f3c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f53fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8669960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56974/0 (socket says 172.21.15.62:56974) 2023-07-14T16:19:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f53fff700 1 -- 172.21.15.62:0/793647416 learned_addr learned my addr 172.21.15.62:0/793647416 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f537fe700 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e866a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e866b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f50ff9700 1 -- 172.21.15.62:0/2369969510 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9f440cdd80 con 0x7f9f54100570 2023-07-14T16:19:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f53fff700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9f3c0a11d0 0x7f9f3c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f537fe700 1 -- 172.21.15.62:0/793647416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 msgr2=0x5599e8669960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f537fe700 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8669960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f537fe700 1 -- 172.21.15.62:0/793647416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 msgr2=0x5599e866b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f537fe700 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e866b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f537fe700 1 -- 172.21.15.62:0/793647416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e83e8e60 con 0x5599e8669ea0 2023-07-14T16:19:46.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.282+0000 7f9f53fff700 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8669960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.283+0000 7f9f537fe700 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e866a760 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9f440cbc00 tx=0x7f9f4406d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e78f5fcb4efcc6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.283+0000 7f9f39ffb700 1 -- 172.21.15.62:0/793647416 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4409b000 con 0x5599e8669ea0 2023-07-14T16:19:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.283+0000 7f9f53fff700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9f3c0a11d0 0x7f9f3c0a3680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009690 tx=0x7f9f4c009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.283+0000 7f9f39ffb700 1 -- 172.21.15.62:0/793647416 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f4409ad70 con 0x5599e8669ea0 2023-07-14T16:19:46.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.283+0000 7f9f39ffb700 1 -- 172.21.15.62:0/793647416 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f440a3680 con 0x5599e8669ea0 2023-07-14T16:19:46.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/793647416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 msgr2=0x5599e866a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e866a760 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9f440cbc00 tx=0x7f9f4406d2f0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/793647416 shutdown_connections 2023-07-14T16:19:46.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e866b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8669960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/793647416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e866a760 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/793647416 >> 172.21.15.62:0/793647416 conn(0x5599e865ee20 msgr2=0x5599e8664250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/793647416 shutdown_connections 2023-07-14T16:19:46.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.284+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/793647416 wait complete. 2023-07-14T16:19:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.285+0000 7f9f6d6b4b80 1 -- start start 2023-07-14T16:19:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.285+0000 7f9f6d6b4b80 1 Processor -- start 2023-07-14T16:19:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669550 0x5599e8672e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669ea0 0x5599e8673350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e866acf0 0x5599e867a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e855ab40 con 0x5599e8669550 2023-07-14T16:19:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e855a840 con 0x5599e8669ea0 2023-07-14T16:19:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e843e9a0 con 0x5599e866acf0 2023-07-14T16:19:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f6d6b4b80 10 client.0 _ll_register_callbacks cb 0x5599e85587e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669550 0x5599e8672e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669550 0x5599e8672e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35330/0 (socket says 172.21.15.62:35330) 2023-07-14T16:19:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f537fe700 1 -- 172.21.15.62:0/4171065298 learned_addr learned my addr 172.21.15.62:0/4171065298 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f53fff700 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e866acf0 0x5599e867a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.286+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669ea0 0x5599e8673350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f537fe700 1 -- 172.21.15.62:0/4171065298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e866acf0 msgr2=0x5599e867a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f537fe700 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e866acf0 0x5599e867a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f537fe700 1 -- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669ea0 msgr2=0x5599e8673350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f537fe700 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669ea0 0x5599e8673350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f537fe700 1 -- 172.21.15.62:0/4171065298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c007500 con 0x5599e8669550 2023-07-14T16:19:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669ea0 0x5599e8673350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:19:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f537fe700 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669550 0x5599e8672e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9f44000c00 tx=0x7f9f44009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bd5e18f9e76afc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:46.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f32ffd700 1 -- 172.21.15.62:0/4171065298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4409b000 con 0x5599e8669550 2023-07-14T16:19:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f32ffd700 1 -- 172.21.15.62:0/4171065298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9f44095e40 con 0x5599e8669550 2023-07-14T16:19:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/4171065298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599e8514570 con 0x5599e8669550 2023-07-14T16:19:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.287+0000 7f9f32ffd700 1 -- 172.21.15.62:0/4171065298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4409ab90 con 0x5599e8669550 2023-07-14T16:19:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f6d6b4b80 10 client.16650 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/4171065298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f9f5419fe60 con 0x5599e8669550 2023-07-14T16:19:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f32ffd700 1 -- 172.21.15.62:0/4171065298 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5599e8514570 con 0x5599e8669550 2023-07-14T16:19:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f32ffd700 1 -- 172.21.15.62:0/4171065298 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9f44084020 con 0x5599e8669550 2023-07-14T16:19:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f6d6b4b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f6d6b4b80 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f6d6b4b80 20 client.16650 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5599e867e6a0 0x5599e8680a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f6d6b4b80 4 client.16650 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.288+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/4171065298 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599e85b3f30 con 0x5599e867e6a0 2023-07-14T16:19:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.289+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5599e867e6a0 0x5599e8680a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.289+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5599e867e6a0 0x5599e8680a80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1dd97d24ec264a server_cookie=10e83c45b9f94f95 in_seq=0 out_seq=0 2023-07-14T16:19:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.289+0000 7f9f32ffd700 10 client.16650 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f32ffd700 1 -- 172.21.15.62:0/4171065298 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5599e85b3f30 con 0x5599e867e6a0 2023-07-14T16:19:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f32ffd700 10 client.16650 handle_command_reply: tid=0 2023-07-14T16:19:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 client.16650 shutdown 2023-07-14T16:19:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/4171065298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669550 msgr2=0x5599e8672e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669550 0x5599e8672e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9f44000c00 tx=0x7f9f44009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/4171065298 shutdown_connections 2023-07-14T16:19:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e866acf0 0x5599e867a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669550 0x5599e8672e10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5599e867e6a0 0x5599e8680a80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/4171065298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669ea0 0x5599e8673350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f32ffd700 0 client.16650 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/4171065298 >> 172.21.15.62:0/4171065298 conn(0x5599e865ee20 msgr2=0x5599e8663930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/4171065298 shutdown_connections 2023-07-14T16:19:46.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.290+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/4171065298 wait complete. 2023-07-14T16:19:46.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.291+0000 7f9f6d6b4b80 20 client.16650 trim_cache size 0 max 16384 2023-07-14T16:19:46.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f6d6b4b80 1 Processor -- start 2023-07-14T16:19:46.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f6d6b4b80 1 -- start start 2023-07-14T16:19:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8672bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e867a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e8673100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e85c1ac0 con 0x5599e8669ea0 2023-07-14T16:19:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e85b3f30 con 0x5599e866acf0 2023-07-14T16:19:46.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e8514570 con 0x5599e8669550 2023-07-14T16:19:46.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e867a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e867a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35346/0 (socket says 172.21.15.62:35346) 2023-07-14T16:19:46.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.316+0000 7f9f52ffd700 1 -- 172.21.15.62:0/18978092 learned_addr learned my addr 172.21.15.62:0/18978092 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f537fe700 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8672bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f53fff700 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e8673100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f52ffd700 1 -- 172.21.15.62:0/18978092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 msgr2=0x5599e8672bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8672bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f52ffd700 1 -- 172.21.15.62:0/18978092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 msgr2=0x5599e8673100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e8673100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f52ffd700 1 -- 172.21.15.62:0/18978092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e8660660 con 0x5599e8669ea0 2023-07-14T16:19:46.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f53fff700 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e8673100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:46.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e867a350 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9f48000c00 tx=0x7f9f4800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c71fdb3edcdad0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.317+0000 7f9f3a7fc700 1 -- 172.21.15.62:0/18978092 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48018040 con 0x5599e8669ea0 2023-07-14T16:19:46.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f3a7fc700 1 -- 172.21.15.62:0/18978092 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f4801e070 con 0x5599e8669ea0 2023-07-14T16:19:46.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f3a7fc700 1 -- 172.21.15.62:0/18978092 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480047c0 con 0x5599e8669ea0 2023-07-14T16:19:46.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/18978092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 msgr2=0x5599e867a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e867a350 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9f48000c00 tx=0x7f9f4800b5c0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/18978092 shutdown_connections 2023-07-14T16:19:46.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e866acf0 0x5599e8673100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669550 0x5599e8672bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/18978092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e8669ea0 0x5599e867a350 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/18978092 >> 172.21.15.62:0/18978092 conn(0x5599e865ee20 msgr2=0x5599e8670f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.318+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/18978092 shutdown_connections 2023-07-14T16:19:46.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.319+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/18978092 wait complete. 2023-07-14T16:19:46.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.319+0000 7f9f6d6b4b80 1 -- start start 2023-07-14T16:19:46.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.319+0000 7f9f6d6b4b80 1 Processor -- start 2023-07-14T16:19:46.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669550 0x5599e86773f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669ea0 0x5599e8677930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f6d6b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e866acf0 0x5599e86d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e86d2a80 con 0x5599e866acf0 2023-07-14T16:19:46.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e86d2bc0 con 0x5599e8669550 2023-07-14T16:19:46.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f6d6b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e86d2d50 con 0x5599e8669ea0 2023-07-14T16:19:46.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f6d6b4b80 10 client.0 _ll_register_callbacks cb 0x7f9f4800b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:46.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f537fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669550 0x5599e86773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f53fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e866acf0 0x5599e86d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f52ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669ea0 0x5599e8677930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f537fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669550 0x5599e86773f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34082/0 (socket says 172.21.15.62:34082) 2023-07-14T16:19:46.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.320+0000 7f9f537fe700 1 -- 172.21.15.62:0/1593073117 learned_addr learned my addr 172.21.15.62:0/1593073117 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f537fe700 1 -- 172.21.15.62:0/1593073117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669ea0 msgr2=0x5599e8677930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f537fe700 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669ea0 0x5599e8677930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f537fe700 1 -- 172.21.15.62:0/1593073117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e866acf0 msgr2=0x5599e86d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f537fe700 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e866acf0 0x5599e86d2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f537fe700 1 -- 172.21.15.62:0/1593073117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4801a860 con 0x5599e8669550 2023-07-14T16:19:46.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f537fe700 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669550 0x5599e86773f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9f440b9e60 tx=0x7f9f44097250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57401861850250c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:19:46.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f32ffd700 1 -- 172.21.15.62:0/1593073117 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f440aeea0 con 0x5599e8669550 2023-07-14T16:19:46.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f32ffd700 1 -- 172.21.15.62:0/1593073117 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9f44082070 con 0x5599e8669550 2023-07-14T16:19:46.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.321+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/1593073117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f4409ad70 con 0x5599e8669550 2023-07-14T16:19:46.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f32ffd700 1 -- 172.21.15.62:0/1593073117 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4409b000 con 0x5599e8669550 2023-07-14T16:19:46.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f6d6b4b80 10 client.26372 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/1593073117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5599e866b5e0 con 0x5599e8669550 2023-07-14T16:19:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f32ffd700 1 -- 172.21.15.62:0/1593073117 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f4409ad70 con 0x5599e8669550 2023-07-14T16:19:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f32ffd700 1 -- 172.21.15.62:0/1593073117 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9f44022470 con 0x5599e8669550 2023-07-14T16:19:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f6d6b4b80 10 client.26372 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f6d6b4b80 10 client.26372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f6d6b4b80 20 client.26372 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5599e86d42d0 0x5599e86d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f6d6b4b80 4 client.26372 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:19:46.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.322+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/1593073117 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5599e85760d0 con 0x5599e86d42d0 2023-07-14T16:19:46.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.323+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5599e86d42d0 0x5599e86d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.323+0000 7f9f52ffd700 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5599e86d42d0 0x5599e86d66b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff86b0d3496f2fe1 server_cookie=402a8f6353ae6408 in_seq=0 out_seq=0 2023-07-14T16:19:46.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.323+0000 7f9f32ffd700 10 client.26372 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:46.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.324+0000 7f9f32ffd700 1 -- 172.21.15.62:0/1593073117 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5599e85760d0 con 0x5599e86d42d0 2023-07-14T16:19:46.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.324+0000 7f9f32ffd700 10 client.26372 handle_command_reply: tid=0 2023-07-14T16:19:46.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.324+0000 7f9f6d6b4b80 1 client.26372 shutdown 2023-07-14T16:19:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/1593073117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669550 msgr2=0x5599e86773f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669550 0x5599e86773f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9f440b9e60 tx=0x7f9f44097250 comp rx=0 tx=0).stop 2023-07-14T16:19:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/1593073117 shutdown_connections 2023-07-14T16:19:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5599e866acf0 0x5599e86d2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5599e86d42d0 0x5599e86d66b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5599e8669550 0x5599e86773f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 --2- 172.21.15.62:0/1593073117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5599e8669ea0 0x5599e8677930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/1593073117 >> 172.21.15.62:0/1593073117 conn(0x5599e865ee20 msgr2=0x5599e8672bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f32ffd700 0 client.26372 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/1593073117 shutdown_connections 2023-07-14T16:19:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 1 -- 172.21.15.62:0/1593073117 wait complete. 2023-07-14T16:19:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.325+0000 7f9f6d6b4b80 20 client.26372 trim_cache size 0 max 16384 2023-07-14T16:19:46.308 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:19:46.309 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:19:46.309 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "956995e7-3e6a-4a26-8e51-78a8c9ed56a2", 2023-07-14T16:19:46.309 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:19:46.309 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:19:46.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 -- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9f3c0a11d0 msgr2=0x7f9f3c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9f3c0a11d0 0x7f9f3c0a3680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009690 tx=0x7f9f4c009620 comp rx=0 tx=0).stop 2023-07-14T16:19:46.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 -- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 msgr2=0x7f9f54197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 0x7f9f54197910 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9f44006010 tx=0x7f9f44003b00 comp rx=0 tx=0).stop 2023-07-14T16:19:46.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 -- 172.21.15.62:0/2369969510 shutdown_connections 2023-07-14T16:19:46.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f541057b0 0x7f9f5419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f54100570 0x7f9f54197910 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9f3c0a11d0 0x7f9f3c0a3680 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 --2- 172.21.15.62:0/2369969510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f54102e90 0x7f9f54197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 -- 172.21.15.62:0/2369969510 >> 172.21.15.62:0/2369969510 conn(0x7f9f540fa010 msgr2=0x7f9f541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 -- 172.21.15.62:0/2369969510 shutdown_connections 2023-07-14T16:19:46.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.326+0000 7f9f59dff700 1 -- 172.21.15.62:0/2369969510 wait complete. 2023-07-14T16:19:46.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:19:46.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.579+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3647082615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 msgr2=0x7fcbec105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.579+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3647082615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 0x7fcbec105260 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009990 tx=0x7fcbe0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.579+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3647082615 shutdown_connections 2023-07-14T16:19:46.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.579+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3647082615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbec1057a0 0x7fcbec107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.579+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3647082615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbec100560 0x7fcbec102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.579+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3647082615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 0x7fcbec105260 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.579+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3647082615 >> 172.21.15.62:0/3647082615 conn(0x7fcbec0fa000 msgr2=0x7fcbec0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.579+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3647082615 shutdown_connections 2023-07-14T16:19:46.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.580+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3647082615 wait complete. 2023-07-14T16:19:46.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.581+0000 7fcbf1dff700 1 Processor -- start 2023-07-14T16:19:46.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.581+0000 7fcbf1dff700 1 -- start start 2023-07-14T16:19:46.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.581+0000 7fcbf1dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbec100560 0x7fcbec197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.581+0000 7fcbf1dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 0x7fcbec197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.581+0000 7fcbf1dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbec1057a0 0x7fcbec19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.581+0000 7fcbf1dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec19f510 con 0x7fcbec102e80 2023-07-14T16:19:46.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.581+0000 7fcbf1dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec19f650 con 0x7fcbec100560 2023-07-14T16:19:46.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.581+0000 7fcbf1dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec19f7e0 con 0x7fcbec1057a0 2023-07-14T16:19:46.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 0x7fcbec197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 0x7fcbec197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35366/0 (socket says 172.21.15.62:35366) 2023-07-14T16:19:46.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 -- 172.21.15.62:0/3040818376 learned_addr learned my addr 172.21.15.62:0/3040818376 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbec100560 0x7fcbec197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbebfff700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbec1057a0 0x7fcbec19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 -- 172.21.15.62:0/3040818376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbec1057a0 msgr2=0x7fcbec19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbec1057a0 0x7fcbec19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 -- 172.21.15.62:0/3040818376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbec100560 msgr2=0x7fcbec197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbec100560 0x7fcbec197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 -- 172.21.15.62:0/3040818376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc009520 con 0x7fcbec102e80 2023-07-14T16:19:46.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.582+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 0x7fcbec197ed0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00098e0 tx=0x7fcbe0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.583+0000 7fcbe8ff9700 1 -- 172.21.15.62:0/3040818376 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe001d070 con 0x7fcbec102e80 2023-07-14T16:19:46.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.583+0000 7fcbe8ff9700 1 -- 172.21.15.62:0/3040818376 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbe000bdb0 con 0x7fcbec102e80 2023-07-14T16:19:46.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.583+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3040818376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe00095f0 con 0x7fcbec102e80 2023-07-14T16:19:46.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.583+0000 7fcbe8ff9700 1 -- 172.21.15.62:0/3040818376 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0022d20 con 0x7fcbec102e80 2023-07-14T16:19:46.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.583+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3040818376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec19fd50 con 0x7fcbec102e80 2023-07-14T16:19:46.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.587+0000 7fcbe8ff9700 1 -- 172.21.15.62:0/3040818376 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcbe000b920 con 0x7fcbec102e80 2023-07-14T16:19:46.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.588+0000 7fcbe8ff9700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcbd40a11d0 0x7fcbd40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.588+0000 7fcbe8ff9700 1 -- 172.21.15.62:0/3040818376 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcbe00cdf20 con 0x7fcbec102e80 2023-07-14T16:19:46.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcbd40a11d0 0x7fcbd40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcc05809b80 1 Processor -- start 2023-07-14T16:19:46.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcc05809b80 1 -- start start 2023-07-14T16:19:46.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcbd40a11d0 0x7fcbd40a3680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009df0 tx=0x7fcbdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c526e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcc05809b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80b56cb0 con 0x555b80c52c70 2023-07-14T16:19:46.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcc05809b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80ba9a40 con 0x555b80c51e20 2023-07-14T16:19:46.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.589+0000 7fcc05809b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80ba2ed0 con 0x555b80c514d0 2023-07-14T16:19:46.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35376/0 (socket says 172.21.15.62:35376) 2023-07-14T16:19:46.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbeb7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 -- 172.21.15.62:0/1315506181 learned_addr learned my addr 172.21.15.62:0/1315506181 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 -- 172.21.15.62:0/1315506181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 msgr2=0x555b80c518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 -- 172.21.15.62:0/1315506181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 msgr2=0x555b80c526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 -- 172.21.15.62:0/1315506181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b809d0de0 con 0x555b80c52c70 2023-07-14T16:19:46.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:46.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.590+0000 7fcbebfff700 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c530e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4011680 tx=0x7fcbe4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7b9ce59b05d0e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.591+0000 7fcbd1ffb700 1 -- 172.21.15.62:0/1315506181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400e040 con 0x555b80c52c70 2023-07-14T16:19:46.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.591+0000 7fcbd1ffb700 1 -- 172.21.15.62:0/1315506181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbe400f040 con 0x555b80c52c70 2023-07-14T16:19:46.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.591+0000 7fcbd1ffb700 1 -- 172.21.15.62:0/1315506181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4004730 con 0x555b80c52c70 2023-07-14T16:19:46.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 -- 172.21.15.62:0/1315506181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 msgr2=0x555b80c530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c530e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4011680 tx=0x7fcbe4009920 comp rx=0 tx=0).stop 2023-07-14T16:19:46.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 -- 172.21.15.62:0/1315506181 shutdown_connections 2023-07-14T16:19:46.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c530e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 --2- 172.21.15.62:0/1315506181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 -- 172.21.15.62:0/1315506181 >> 172.21.15.62:0/1315506181 conn(0x555b80c46da0 msgr2=0x555b80c4c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 -- 172.21.15.62:0/1315506181 shutdown_connections 2023-07-14T16:19:46.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.592+0000 7fcc05809b80 1 -- 172.21.15.62:0/1315506181 wait complete. 2023-07-14T16:19:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.593+0000 7fcc05809b80 1 -- start start 2023-07-14T16:19:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.593+0000 7fcc05809b80 1 Processor -- start 2023-07-14T16:19:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c51e20 0x555b80c5b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c62630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcc05809b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80b42ac0 con 0x555b80c51e20 2023-07-14T16:19:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcc05809b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80b427c0 con 0x555b80c52c70 2023-07-14T16:19:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcc05809b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80a26920 con 0x555b80c514d0 2023-07-14T16:19:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcc05809b80 10 client.0 _ll_register_callbacks cb 0x555b80b40760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c51e20 0x555b80c5b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcbeaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c51e20 0x555b80c5b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35384/0 (socket says 172.21.15.62:35384) 2023-07-14T16:19:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcbeaffd700 1 -- 172.21.15.62:0/2122696064 learned_addr learned my addr 172.21.15.62:0/2122696064 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcbebfff700 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c62630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcbeaffd700 1 -- 172.21.15.62:0/2122696064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 msgr2=0x555b80c5b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.594+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbeaffd700 1 -- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 msgr2=0x555b80c62630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c62630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbeaffd700 1 -- 172.21.15.62:0/2122696064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe4011330 con 0x555b80c51e20 2023-07-14T16:19:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbebfff700 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c62630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:46.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c51e20 0x555b80c5b5f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0003ec0 tx=0x7fcbe0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c07ab0526c83fb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:46.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2122696064 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000bbd0 con 0x555b80c51e20 2023-07-14T16:19:46.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2122696064 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcbe009b370 con 0x555b80c51e20 2023-07-14T16:19:46.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcc05809b80 1 -- 172.21.15.62:0/2122696064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b80afc4f0 con 0x555b80c51e20 2023-07-14T16:19:46.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2122696064 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbe009f3a0 con 0x555b80c51e20 2023-07-14T16:19:46.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcc05809b80 10 client.16668 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:46.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.595+0000 7fcc05809b80 1 -- 172.21.15.62:0/2122696064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555b809b9c60 con 0x555b80c51e20 2023-07-14T16:19:46.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2122696064 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555b80afc4f0 con 0x555b80c51e20 2023-07-14T16:19:46.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2122696064 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fcbe00a32a0 con 0x555b80c51e20 2023-07-14T16:19:46.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcc05809b80 10 client.16668 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:46.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcc05809b80 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:46.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcc05809b80 20 client.16668 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b80c66ad0 0x555b80c68ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcc05809b80 4 client.16668 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcc05809b80 1 -- 172.21.15.62:0/2122696064 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b80b9beb0 con 0x555b80c66ad0 2023-07-14T16:19:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.596+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b80c66ad0 0x555b80c68ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.597+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b80c66ad0 0x555b80c68ee0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f27b7a634765fc6 server_cookie=e1a67a796acfd09 in_seq=0 out_seq=0 2023-07-14T16:19:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.597+0000 7fcbcaffd700 10 client.16668 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2122696064 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555b80b9beb0 con 0x555b80c66ad0 2023-07-14T16:19:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcbcaffd700 10 client.16668 handle_command_reply: tid=0 2023-07-14T16:19:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 client.16668 shutdown 2023-07-14T16:19:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 -- 172.21.15.62:0/2122696064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c51e20 msgr2=0x555b80c5b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c51e20 0x555b80c5b5f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0003ec0 tx=0x7fcbe0009310 comp rx=0 tx=0).stop 2023-07-14T16:19:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 -- 172.21.15.62:0/2122696064 shutdown_connections 2023-07-14T16:19:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b80c66ad0 0x555b80c68ee0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c62630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcbcaffd700 0 client.16668 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2122696064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c51e20 0x555b80c5b5f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 -- 172.21.15.62:0/2122696064 >> 172.21.15.62:0/2122696064 conn(0x555b80c46da0 msgr2=0x555b80c4b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 -- 172.21.15.62:0/2122696064 shutdown_connections 2023-07-14T16:19:46.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.598+0000 7fcc05809b80 1 -- 172.21.15.62:0/2122696064 wait complete. 2023-07-14T16:19:46.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.599+0000 7fcc05809b80 20 client.16668 trim_cache size 0 max 16384 2023-07-14T16:19:46.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.621+0000 7fcc05809b80 1 Processor -- start 2023-07-14T16:19:46.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.621+0000 7fcc05809b80 1 -- start start 2023-07-14T16:19:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.621+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.621+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.621+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c5b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.621+0000 7fcc05809b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80c4da00 con 0x555b80c52c70 2023-07-14T16:19:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.621+0000 7fcc05809b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80c4dc20 con 0x555b80c51e20 2023-07-14T16:19:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.621+0000 7fcc05809b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80c5bd30 con 0x555b80c514d0 2023-07-14T16:19:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c5b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c5b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35394/0 (socket says 172.21.15.62:35394) 2023-07-14T16:19:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 -- 172.21.15.62:0/3058497393 learned_addr learned my addr 172.21.15.62:0/3058497393 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 -- 172.21.15.62:0/3058497393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 msgr2=0x555b80c5b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 -- 172.21.15.62:0/3058497393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 msgr2=0x555b80c625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 -- 172.21.15.62:0/3058497393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b809f7250 con 0x555b80c52c70 2023-07-14T16:19:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:19:46.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:46.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.622+0000 7fcbebfff700 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c5b5f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4011680 tx=0x7fcbe4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7fbb12cfa9378a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcbd27fc700 1 -- 172.21.15.62:0/3058497393 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400e050 con 0x555b80c52c70 2023-07-14T16:19:46.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcbd27fc700 1 -- 172.21.15.62:0/3058497393 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbe4003e90 con 0x555b80c52c70 2023-07-14T16:19:46.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcbd27fc700 1 -- 172.21.15.62:0/3058497393 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40059e0 con 0x555b80c52c70 2023-07-14T16:19:46.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcc05809b80 1 -- 172.21.15.62:0/3058497393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 msgr2=0x555b80c5b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcc05809b80 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c5b5f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4011680 tx=0x7fcbe4009e50 comp rx=0 tx=0).stop 2023-07-14T16:19:46.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcc05809b80 1 -- 172.21.15.62:0/3058497393 shutdown_connections 2023-07-14T16:19:46.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcc05809b80 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c52c70 0x555b80c5b5f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcc05809b80 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c514d0 0x555b80c5b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcc05809b80 1 --2- 172.21.15.62:0/3058497393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c51e20 0x555b80c625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.623+0000 7fcc05809b80 1 -- 172.21.15.62:0/3058497393 >> 172.21.15.62:0/3058497393 conn(0x555b80c46da0 msgr2=0x555b80c58fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.624+0000 7fcc05809b80 1 -- 172.21.15.62:0/3058497393 shutdown_connections 2023-07-14T16:19:46.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.624+0000 7fcc05809b80 1 -- 172.21.15.62:0/3058497393 wait complete. 2023-07-14T16:19:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.624+0000 7fcc05809b80 1 -- start start 2023-07-14T16:19:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.624+0000 7fcc05809b80 1 Processor -- start 2023-07-14T16:19:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c514d0 0x555b80c62610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c51e20 0x555b80c5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcc05809b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcc05809b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80cba940 con 0x555b80c514d0 2023-07-14T16:19:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcc05809b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80cbaa80 con 0x555b80c52c70 2023-07-14T16:19:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcc05809b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b80cbabc0 con 0x555b80c51e20 2023-07-14T16:19:46.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcc05809b80 10 client.0 _ll_register_callbacks cb 0x7fcbe009b1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:46.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c514d0 0x555b80c62610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c514d0 0x555b80c62610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35402/0 (socket says 172.21.15.62:35402) 2023-07-14T16:19:46.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeb7fe700 1 -- 172.21.15.62:0/2106142466 learned_addr learned my addr 172.21.15.62:0/2106142466 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbebfff700 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c51e20 0x555b80c5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeb7fe700 1 -- 172.21.15.62:0/2106142466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c51e20 msgr2=0x555b80c5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c51e20 0x555b80c5f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeb7fe700 1 -- 172.21.15.62:0/2106142466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 msgr2=0x555b80c5f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:46.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c5f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.625+0000 7fcbeb7fe700 1 -- 172.21.15.62:0/2106142466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b809d0de0 con 0x555b80c514d0 2023-07-14T16:19:46.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.626+0000 7fcbeb7fe700 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c514d0 0x555b80c62610 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009ba0 tx=0x7fcbdc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b178939401022d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:46.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.626+0000 7fcbcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:46.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.626+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2106142466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc022040 con 0x555b80c514d0 2023-07-14T16:19:46.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.626+0000 7fcc05809b80 1 -- 172.21.15.62:0/2106142466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbe400f040 con 0x555b80c514d0 2023-07-14T16:19:46.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.626+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2106142466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcbdc00ec10 con 0x555b80c514d0 2023-07-14T16:19:46.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.626+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2106142466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbdc03a030 con 0x555b80c514d0 2023-07-14T16:19:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcc05809b80 10 client.16674 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcc05809b80 1 -- 172.21.15.62:0/2106142466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555b809cf2b0 con 0x555b80c514d0 2023-07-14T16:19:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2106142466 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbe400f040 con 0x555b80c514d0 2023-07-14T16:19:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2106142466 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fcbdc003980 con 0x555b80c514d0 2023-07-14T16:19:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcc05809b80 10 client.16674 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcc05809b80 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcc05809b80 20 client.16674 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b80cbc540 0x555b80cbe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcc05809b80 4 client.16674 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:19:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcc05809b80 1 -- 172.21.15.62:0/2106142466 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555b80b5e050 con 0x555b80cbc540 2023-07-14T16:19:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.627+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b80cbc540 0x555b80cbe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.628+0000 7fcbeaffd700 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b80cbc540 0x555b80cbe950 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85010b197afd95d8 server_cookie=d5c6c7eb03c676b3 in_seq=0 out_seq=0 2023-07-14T16:19:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.628+0000 7fcbcaffd700 10 client.16674 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.628+0000 7fcbcaffd700 1 -- 172.21.15.62:0/2106142466 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555b80b5e050 con 0x555b80cbc540 2023-07-14T16:19:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.628+0000 7fcbcaffd700 10 client.16674 handle_command_reply: tid=0 2023-07-14T16:19:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.628+0000 7fcc05809b80 1 client.16674 shutdown 2023-07-14T16:19:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.628+0000 7fcc05809b80 1 -- 172.21.15.62:0/2106142466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c514d0 msgr2=0x555b80c62610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.628+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c514d0 0x555b80c62610 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009ba0 tx=0x7fcbdc0047c0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 1 -- 172.21.15.62:0/2106142466 shutdown_connections 2023-07-14T16:19:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b80cbc540 0x555b80cbe950 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b80c52c70 0x555b80c5f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b80c514d0 0x555b80c62610 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 1 --2- 172.21.15.62:0/2106142466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b80c51e20 0x555b80c5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcbcaffd700 0 client.16674 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:46.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 1 -- 172.21.15.62:0/2106142466 >> 172.21.15.62:0/2106142466 conn(0x555b80c46da0 msgr2=0x555b80c55ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 1 -- 172.21.15.62:0/2106142466 shutdown_connections 2023-07-14T16:19:46.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 1 -- 172.21.15.62:0/2106142466 wait complete. 2023-07-14T16:19:46.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.629+0000 7fcc05809b80 20 client.16674 trim_cache size 0 max 16384 2023-07-14T16:19:46.618 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:19:46.618 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:19:46.618 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:19:46.618 INFO:teuthology.orchestra.run.smithi062.stdout: "956995e7-3e6a-4a26-8e51-78a8c9ed56a2": { 2023-07-14T16:19:46.618 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:19:46.618 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "956995e7-3e6a-4a26-8e51-78a8c9ed56a2", 2023-07-14T16:19:46.619 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:19:46.619 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:19:46.619 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:19:46.619 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:19:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcbd40a11d0 msgr2=0x7fcbd40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcbd40a11d0 0x7fcbd40a3680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009df0 tx=0x7fcbdc009260 comp rx=0 tx=0).stop 2023-07-14T16:19:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 msgr2=0x7fcbec197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 0x7fcbec197ed0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00098e0 tx=0x7fcbe0004730 comp rx=0 tx=0).stop 2023-07-14T16:19:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3040818376 shutdown_connections 2023-07-14T16:19:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbec1057a0 0x7fcbec19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcbd40a11d0 0x7fcbd40a3680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbec100560 0x7fcbec197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 --2- 172.21.15.62:0/3040818376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbec102e80 0x7fcbec197ed0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3040818376 >> 172.21.15.62:0/3040818376 conn(0x7fcbec0fa000 msgr2=0x7fcbec0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3040818376 shutdown_connections 2023-07-14T16:19:46.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:19:46.630+0000 7fcbf1dff700 1 -- 172.21.15.62:0/3040818376 wait complete. 2023-07-14T16:19:46.625 INFO:tasks.cephfs.filesystem:scrub status for tag:956995e7-3e6a-4a26-8e51-78a8c9ed56a2 - {'path': '/', 'tag': '956995e7-3e6a-4a26-8e51-78a8c9ed56a2', 'options': 'recursive,force'} 2023-07-14T16:19:46.633 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:19:46.333+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v117>,<0x10000000003/client.0 v118>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v130>,<0x10000000001/sv_0 v144>,<0x10000000000/_nogroup v138>,<0x1/volumes v128>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:19:46.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: pgmap v481: 97 pgs: 97 active+clean; 2.0 GiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:19:46.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:46.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:46.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:46.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:46.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:46.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:46.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:46.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:46 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:46.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: pgmap v481: 97 pgs: 97 active+clean; 2.0 GiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:19:46.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:46.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:46.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:46.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:46.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:46.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:46.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:46.904 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:46 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:47.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: pgmap v481: 97 pgs: 97 active+clean; 2.0 GiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:19:47.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:47.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:47.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:47.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:47.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:47.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:47.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:47.029 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:46 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.808+0000 7f2fafdff700 1 -- 172.21.15.62:0/986771300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa8102f30 msgr2=0x7f2fa8103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.808+0000 7f2fafdff700 1 --2- 172.21.15.62:0/986771300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa8102f30 0x7f2fa8103380 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009990 tx=0x7f2f98009ca0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.808+0000 7f2fafdff700 1 -- 172.21.15.62:0/986771300 shutdown_connections 2023-07-14T16:19:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.808+0000 7f2fafdff700 1 --2- 172.21.15.62:0/986771300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa81038c0 0x7f2fa810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.808+0000 7f2fafdff700 1 --2- 172.21.15.62:0/986771300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa8101d40 0x7f2fa8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.808+0000 7f2fafdff700 1 --2- 172.21.15.62:0/986771300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa8102f30 0x7f2fa8103380 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.808+0000 7f2fafdff700 1 -- 172.21.15.62:0/986771300 >> 172.21.15.62:0/986771300 conn(0x7f2fa80fd360 msgr2=0x7f2fa80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.808+0000 7f2fafdff700 1 -- 172.21.15.62:0/986771300 shutdown_connections 2023-07-14T16:19:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.809+0000 7f2fafdff700 1 -- 172.21.15.62:0/986771300 wait complete. 2023-07-14T16:19:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.810+0000 7f2fafdff700 1 Processor -- start 2023-07-14T16:19:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.810+0000 7f2fafdff700 1 -- start start 2023-07-14T16:19:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.810+0000 7f2fafdff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa8101d40 0x7f2fa8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.810+0000 7f2fafdff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa8102f30 0x7f2fa8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.810+0000 7f2fafdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa81038c0 0x7f2fa8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.810+0000 7f2fafdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81994b0 con 0x7f2fa81038c0 2023-07-14T16:19:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.810+0000 7f2fafdff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8199620 con 0x7f2fa8102f30 2023-07-14T16:19:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.810+0000 7f2fafdff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81997e0 con 0x7f2fa8101d40 2023-07-14T16:19:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fadb9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa8101d40 0x7f2fa8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fae39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa81038c0 0x7f2fa8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fadb9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa8101d40 0x7f2fa8197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:57056/0 (socket says 172.21.15.62:57056) 2023-07-14T16:19:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fadb9b700 1 -- 172.21.15.62:0/2676715109 learned_addr learned my addr 172.21.15.62:0/2676715109 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fae39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa81038c0 0x7f2fa8198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35412/0 (socket says 172.21.15.62:35412) 2023-07-14T16:19:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fad39a700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa8102f30 0x7f2fa8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fae39c700 1 -- 172.21.15.62:0/2676715109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa8101d40 msgr2=0x7f2fa8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fae39c700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa8101d40 0x7f2fa8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fae39c700 1 -- 172.21.15.62:0/2676715109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa8102f30 msgr2=0x7f2fa8197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fae39c700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa8102f30 0x7f2fa8197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fae39c700 1 -- 172.21.15.62:0/2676715109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980095f0 con 0x7f2fa81038c0 2023-07-14T16:19:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.811+0000 7f2fae39c700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa81038c0 0x7f2fa8198e50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0007f10 tx=0x7f2fa000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.812+0000 7f2f9effd700 1 -- 172.21.15.62:0/2676715109 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0004db0 con 0x7f2fa81038c0 2023-07-14T16:19:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.812+0000 7f2f9effd700 1 -- 172.21.15.62:0/2676715109 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fa000f040 con 0x7f2fa81038c0 2023-07-14T16:19:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.812+0000 7f2fafdff700 1 -- 172.21.15.62:0/2676715109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa8199ac0 con 0x7f2fa81038c0 2023-07-14T16:19:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.812+0000 7f2f9effd700 1 -- 172.21.15.62:0/2676715109 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0005960 con 0x7f2fa81038c0 2023-07-14T16:19:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.812+0000 7f2fafdff700 1 -- 172.21.15.62:0/2676715109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa81a0280 con 0x7f2fa81038c0 2023-07-14T16:19:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.816+0000 7f2f9effd700 1 -- 172.21.15.62:0/2676715109 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2fa000db20 con 0x7f2fa81038c0 2023-07-14T16:19:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.817+0000 7f2fc3686b80 1 Processor -- start 2023-07-14T16:19:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.817+0000 7f2fc3686b80 1 -- start start 2023-07-14T16:19:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.817+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641dddcc6b0 0x5641ddf70300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.817+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.817+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf71b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.817+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dde75c50 con 0x5641dddcc6b0 2023-07-14T16:19:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.817+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddec89e0 con 0x5641ddf716e0 2023-07-14T16:19:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.817+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddec1e70 con 0x5641ddf70840 2023-07-14T16:19:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fad39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641dddcc6b0 0x5641ddf70300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fad39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641dddcc6b0 0x5641ddf70300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35426/0 (socket says 172.21.15.62:35426) 2023-07-14T16:19:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fad39a700 1 -- 172.21.15.62:0/3618072165 learned_addr learned my addr 172.21.15.62:0/3618072165 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fad39a700 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2f9effd700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f940a1150 0x7f2f940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2f9effd700 1 -- 172.21.15.62:0/2676715109 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2fa00cc0d0 con 0x7f2fa81038c0 2023-07-14T16:19:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fadb9b700 1 -- 172.21.15.62:0/3618072165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 msgr2=0x5641ddf71b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf71b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fadb9b700 1 -- 172.21.15.62:0/3618072165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641dddcc6b0 msgr2=0x5641ddf70300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641dddcc6b0 0x5641ddf70300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fadb9b700 1 -- 172.21.15.62:0/3618072165 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ddcefd80 con 0x5641ddf70840 2023-07-14T16:19:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.818+0000 7f2fad39a700 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641dddcc6b0 0x5641ddf70300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:19:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.819+0000 7f2fae39c700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f940a1150 0x7f2f940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.819+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf711a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007d00 tx=0x7f2fa40106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea666d552b91dc24 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.819+0000 7f2fae39c700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f940a1150 0x7f2f940a3600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2fa001b000 tx=0x7f2fa0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.819+0000 7f2f93fff700 1 -- 172.21.15.62:0/3618072165 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4010e10 con 0x5641ddf70840 2023-07-14T16:19:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.819+0000 7f2f93fff700 1 -- 172.21.15.62:0/3618072165 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fa4015070 con 0x5641ddf70840 2023-07-14T16:19:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.819+0000 7f2f93fff700 1 -- 172.21.15.62:0/3618072165 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4019630 con 0x5641ddf70840 2023-07-14T16:19:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 -- 172.21.15.62:0/3618072165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 msgr2=0x5641ddf711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf711a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007d00 tx=0x7f2fa40106b0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 -- 172.21.15.62:0/3618072165 shutdown_connections 2023-07-14T16:19:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf71b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641dddcc6b0 0x5641ddf70300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/3618072165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf711a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 -- 172.21.15.62:0/3618072165 >> 172.21.15.62:0/3618072165 conn(0x5641ddf65be0 msgr2=0x5641ddf6acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 -- 172.21.15.62:0/3618072165 shutdown_connections 2023-07-14T16:19:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.820+0000 7f2fc3686b80 1 -- 172.21.15.62:0/3618072165 wait complete. 2023-07-14T16:19:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.821+0000 7f2fc3686b80 1 -- start start 2023-07-14T16:19:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.821+0000 7f2fc3686b80 1 Processor -- start 2023-07-14T16:19:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641dddcc6b0 0x5641ddf79890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf79dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf716e0 0x5641ddf80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dde61a60 con 0x5641ddf716e0 2023-07-14T16:19:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641dde61760 con 0x5641dddcc6b0 2023-07-14T16:19:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddd458c0 con 0x5641ddf70840 2023-07-14T16:19:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fc3686b80 10 client.0 _ll_register_callbacks cb 0x5641dde5f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf716e0 0x5641ddf80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf716e0 0x5641ddf80e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35430/0 (socket says 172.21.15.62:35430) 2023-07-14T16:19:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 -- 172.21.15.62:0/1391672232 learned_addr learned my addr 172.21.15.62:0/1391672232 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641dddcc6b0 0x5641ddf79890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf79dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 -- 172.21.15.62:0/1391672232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 msgr2=0x5641ddf79dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf79dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 -- 172.21.15.62:0/1391672232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641dddcc6b0 msgr2=0x5641ddf79890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641dddcc6b0 0x5641ddf79890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.822+0000 7f2fad39a700 1 -- 172.21.15.62:0/1391672232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f98020040 con 0x5641ddf716e0 2023-07-14T16:19:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.823+0000 7f2fad39a700 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf716e0 0x5641ddf80e10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2f98015000 tx=0x7f2f98005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9ff6e41f94be0e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.823+0000 7f2f917fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.823+0000 7f2f917fa700 1 -- 172.21.15.62:0/1391672232 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9801d030 con 0x5641ddf716e0 2023-07-14T16:19:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.823+0000 7f2fc3686b80 1 -- 172.21.15.62:0/1391672232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641dde1b490 con 0x5641ddf716e0 2023-07-14T16:19:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.823+0000 7f2f917fa700 1 -- 172.21.15.62:0/1391672232 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f9801e070 con 0x5641ddf716e0 2023-07-14T16:19:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.823+0000 7f2f917fa700 1 -- 172.21.15.62:0/1391672232 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f98003d70 con 0x5641ddf716e0 2023-07-14T16:19:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.823+0000 7f2fc3686b80 10 client.16686 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.823+0000 7f2fc3686b80 1 -- 172.21.15.62:0/1391672232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f2fa4007930 con 0x5641ddf716e0 2023-07-14T16:19:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.824+0000 7f2f917fa700 1 -- 172.21.15.62:0/1391672232 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5641dde1b490 con 0x5641ddf716e0 2023-07-14T16:19:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.824+0000 7f2f917fa700 1 -- 172.21.15.62:0/1391672232 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2f9800fa90 con 0x5641ddf716e0 2023-07-14T16:19:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.824+0000 7f2fc3686b80 10 client.16686 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.824+0000 7f2fc3686b80 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.824+0000 7f2fc3686b80 20 client.16686 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.824+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5641ddf85690 0x5641ddf87aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.824+0000 7f2fc3686b80 4 client.16686 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:19:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.824+0000 7f2fc3686b80 1 -- 172.21.15.62:0/1391672232 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641ddebae50 con 0x5641ddf85690 2023-07-14T16:19:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.825+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5641ddf85690 0x5641ddf87aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.825+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5641ddf85690 0x5641ddf87aa0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1a2cb0c2709441 server_cookie=11ec9343bdbeb1b2 in_seq=0 out_seq=0 2023-07-14T16:19:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.825+0000 7f2f917fa700 10 client.16686 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2f917fa700 1 -- 172.21.15.62:0/1391672232 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5641ddebae50 con 0x5641ddf85690 2023-07-14T16:19:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2f917fa700 10 client.16686 handle_command_reply: tid=0 2023-07-14T16:19:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 client.16686 shutdown 2023-07-14T16:19:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 -- 172.21.15.62:0/1391672232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf716e0 msgr2=0x5641ddf80e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf716e0 0x5641ddf80e10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2f98015000 tx=0x7f2f98005fb0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 -- 172.21.15.62:0/1391672232 shutdown_connections 2023-07-14T16:19:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf716e0 0x5641ddf80e10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5641ddf85690 0x5641ddf87aa0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641dddcc6b0 0x5641ddf79890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/1391672232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641ddf70840 0x5641ddf79dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2f917fa700 0 client.16686 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 -- 172.21.15.62:0/1391672232 >> 172.21.15.62:0/1391672232 conn(0x5641ddf65be0 msgr2=0x5641ddf6a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 -- 172.21.15.62:0/1391672232 shutdown_connections 2023-07-14T16:19:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.826+0000 7f2fc3686b80 1 -- 172.21.15.62:0/1391672232 wait complete. 2023-07-14T16:19:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.827+0000 7f2fc3686b80 20 client.16686 trim_cache size 0 max 16384 2023-07-14T16:19:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.847+0000 7f2fc3686b80 1 Processor -- start 2023-07-14T16:19:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.847+0000 7f2fc3686b80 1 -- start start 2023-07-14T16:19:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf78560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf80dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf74630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddf7aab0 con 0x5641ddf70840 2023-07-14T16:19:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddf78390 con 0x5641ddf716e0 2023-07-14T16:19:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddf70c10 con 0x5641dddcc6b0 2023-07-14T16:19:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fadb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf80dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fadb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf80dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35434/0 (socket says 172.21.15.62:35434) 2023-07-14T16:19:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fadb9b700 1 -- 172.21.15.62:0/4114924848 learned_addr learned my addr 172.21.15.62:0/4114924848 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fad39a700 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf74630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fadb9b700 1 -- 172.21.15.62:0/4114924848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 msgr2=0x5641ddf78560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fad39a700 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf78560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf78560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fadb9b700 1 -- 172.21.15.62:0/4114924848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 msgr2=0x5641ddf74630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf74630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fadb9b700 1 -- 172.21.15.62:0/4114924848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641ddcd8c00 con 0x5641ddf70840 2023-07-14T16:19:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.848+0000 7f2fad39a700 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf78560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:19:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.849+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf80dd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007c20 tx=0x7f2fa4005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea40a178caabc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.849+0000 7f2f9cbff700 1 -- 172.21.15.62:0/4114924848 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4027c80 con 0x5641ddf70840 2023-07-14T16:19:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.849+0000 7f2f9cbff700 1 -- 172.21.15.62:0/4114924848 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fa402b070 con 0x5641ddf70840 2023-07-14T16:19:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.849+0000 7f2f9cbff700 1 -- 172.21.15.62:0/4114924848 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4005280 con 0x5641ddf70840 2023-07-14T16:19:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 -- 172.21.15.62:0/4114924848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 msgr2=0x5641ddf80dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf80dd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007c20 tx=0x7f2fa4005ce0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 -- 172.21.15.62:0/4114924848 shutdown_connections 2023-07-14T16:19:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf74630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf78560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/4114924848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf80dd0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 -- 172.21.15.62:0/4114924848 >> 172.21.15.62:0/4114924848 conn(0x5641ddf65be0 msgr2=0x5641ddf8c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 -- 172.21.15.62:0/4114924848 shutdown_connections 2023-07-14T16:19:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 -- 172.21.15.62:0/4114924848 wait complete. 2023-07-14T16:19:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.850+0000 7f2fc3686b80 1 -- start start 2023-07-14T16:19:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fc3686b80 1 Processor -- start 2023-07-14T16:19:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf7cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fc3686b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf80460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddf809a0 con 0x5641ddf70840 2023-07-14T16:19:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddf80b10 con 0x5641ddf716e0 2023-07-14T16:19:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fc3686b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641ddf80cd0 con 0x5641dddcc6b0 2023-07-14T16:19:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fc3686b80 10 client.0 _ll_register_callbacks cb 0x5641dde5f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:19:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fadb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fadb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf7d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35440/0 (socket says 172.21.15.62:35440) 2023-07-14T16:19:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.851+0000 7f2fadb9b700 1 -- 172.21.15.62:0/2726723527 learned_addr learned my addr 172.21.15.62:0/2726723527 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:19:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fad39a700 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf7cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fad39a700 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf80460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fadb9b700 1 -- 172.21.15.62:0/2726723527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 msgr2=0x5641ddf7cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf7cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fadb9b700 1 -- 172.21.15.62:0/2726723527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 msgr2=0x5641ddf80460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:19:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf80460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fadb9b700 1 -- 172.21.15.62:0/2726723527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa401e040 con 0x5641ddf70840 2023-07-14T16:19:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fad39a700 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf7cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:19:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fadb9b700 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf7d440 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4010a00 tx=0x7f2fa4007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21624071ed615ad9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:19:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2f917fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:19:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2f917fa700 1 -- 172.21.15.62:0/2726723527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4027c80 con 0x5641ddf70840 2023-07-14T16:19:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2fc3686b80 1 -- 172.21.15.62:0/2726723527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fa4015070 con 0x5641ddf70840 2023-07-14T16:19:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.852+0000 7f2f917fa700 1 -- 172.21.15.62:0/2726723527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2fa4027370 con 0x5641ddf70840 2023-07-14T16:19:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2f917fa700 1 -- 172.21.15.62:0/2726723527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa4027de0 con 0x5641ddf70840 2023-07-14T16:19:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2fc3686b80 10 client.16692 fetch_fsmap learned FSMap version 24 2023-07-14T16:19:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2fc3686b80 1 -- 172.21.15.62:0/2726723527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5641ddd161f0 con 0x5641ddf70840 2023-07-14T16:19:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2f917fa700 1 -- 172.21.15.62:0/2726723527 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fa4015070 con 0x5641ddf70840 2023-07-14T16:19:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2f917fa700 1 -- 172.21.15.62:0/2726723527 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2fa4004980 con 0x5641ddf70840 2023-07-14T16:19:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2fc3686b80 10 client.16692 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:19:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2fc3686b80 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:19:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2fc3686b80 20 client.16692 populate_metadata read hostname 'smithi062' 2023-07-14T16:19:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5641ddfda600 0x5641ddfdca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:19:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2fc3686b80 4 client.16692 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:19:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.853+0000 7f2fc3686b80 1 -- 172.21.15.62:0/2726723527 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5641dde7cff0 con 0x5641ddfda600 2023-07-14T16:19:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.854+0000 7f2fad39a700 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5641ddfda600 0x5641ddfdca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:19:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.854+0000 7f2fad39a700 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5641ddfda600 0x5641ddfdca10 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a297448c67e5b2f server_cookie=a7fd81edc8c6d0a8 in_seq=0 out_seq=0 2023-07-14T16:19:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.854+0000 7f2f917fa700 10 client.16692 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.863+0000 7f2f917fa700 1 -- 172.21.15.62:0/2726723527 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5641dde7cff0 con 0x5641ddfda600 2023-07-14T16:19:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.863+0000 7f2f917fa700 10 client.16692 handle_command_reply: tid=0 2023-07-14T16:19:47.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.863+0000 7f2fc3686b80 1 client.16692 shutdown 2023-07-14T16:19:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 -- 172.21.15.62:0/2726723527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 msgr2=0x5641ddf7d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf7d440 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4010a00 tx=0x7f2fa4007f00 comp rx=0 tx=0).stop 2023-07-14T16:19:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 -- 172.21.15.62:0/2726723527 shutdown_connections 2023-07-14T16:19:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5641ddfda600 0x5641ddfdca10 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5641ddf716e0 0x5641ddf80460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5641dddcc6b0 0x5641ddf7cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 --2- 172.21.15.62:0/2726723527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641ddf70840 0x5641ddf7d440 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2f917fa700 0 client.16692 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:19:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 -- 172.21.15.62:0/2726723527 >> 172.21.15.62:0/2726723527 conn(0x5641ddf65be0 msgr2=0x5641ddf78560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 -- 172.21.15.62:0/2726723527 shutdown_connections 2023-07-14T16:19:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 1 -- 172.21.15.62:0/2726723527 wait complete. 2023-07-14T16:19:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.864+0000 7f2fc3686b80 20 client.16692 trim_cache size 0 max 16384 2023-07-14T16:19:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:19:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:19:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:19:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:19:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 -- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f940a1150 msgr2=0x7f2f940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f940a1150 0x7f2f940a3600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2fa001b000 tx=0x7f2fa0000f40 comp rx=0 tx=0).stop 2023-07-14T16:19:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 -- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa81038c0 msgr2=0x7f2fa8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:19:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa81038c0 0x7f2fa8198e50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0007f10 tx=0x7f2fa000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 -- 172.21.15.62:0/2676715109 shutdown_connections 2023-07-14T16:19:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa81038c0 0x7f2fa8198e50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fa8101d40 0x7f2fa8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f940a1150 0x7f2f940a3600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 --2- 172.21.15.62:0/2676715109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fa8102f30 0x7f2fa8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:19:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 -- 172.21.15.62:0/2676715109 >> 172.21.15.62:0/2676715109 conn(0x7f2fa80fd360 msgr2=0x7f2fa8106470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:19:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 -- 172.21.15.62:0/2676715109 shutdown_connections 2023-07-14T16:19:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:19:47.865+0000 7f2fafdff700 1 -- 172.21.15.62:0/2676715109 wait complete. 2023-07-14T16:19:47.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:47 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:19:47.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:47 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:47.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:47 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:19:47.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:47 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:19:47.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:47 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:48.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:47 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:19:48.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:47 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:47 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:19:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:47 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:19:48.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:47 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:48.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:47 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:19:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:47 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:47 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:19:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:47 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:19:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:47 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:19:48.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:48 smithi081 ceph-mon[127795]: pgmap v482: 97 pgs: 97 active+clean; 2.0 GiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:19:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:48 smithi164 ceph-mon[97427]: pgmap v482: 97 pgs: 97 active+clean; 2.0 GiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:19:49.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:48 smithi062 ceph-mon[95543]: pgmap v482: 97 pgs: 97 active+clean; 2.0 GiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:19:50.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:50 smithi081 ceph-mon[127795]: pgmap v483: 97 pgs: 97 active+clean; 2.0 GiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 29 op/s 2023-07-14T16:19:51.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:50 smithi164 ceph-mon[97427]: pgmap v483: 97 pgs: 97 active+clean; 2.0 GiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 29 op/s 2023-07-14T16:19:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:50 smithi062 ceph-mon[95543]: pgmap v483: 97 pgs: 97 active+clean; 2.0 GiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 29 op/s 2023-07-14T16:19:52.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:52 smithi081 ceph-mon[127795]: pgmap v484: 97 pgs: 97 active+clean; 2.1 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:19:53.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:52 smithi164 ceph-mon[97427]: pgmap v484: 97 pgs: 97 active+clean; 2.1 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:19:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:52 smithi062 ceph-mon[95543]: pgmap v484: 97 pgs: 97 active+clean; 2.1 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:19:54.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:53 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:19:54.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:53 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:19:54.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:53 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:19:54.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:53 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:19:54.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:53 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:19:54.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:53 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:19:55.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:54 smithi164 ceph-mon[97427]: pgmap v485: 97 pgs: 97 active+clean; 2.1 GiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:19:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:54 smithi062 ceph-mon[95543]: pgmap v485: 97 pgs: 97 active+clean; 2.1 GiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:19:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:54 smithi081 ceph-mon[127795]: pgmap v485: 97 pgs: 97 active+clean; 2.1 GiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:19:57.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:56 smithi164 ceph-mon[97427]: pgmap v486: 97 pgs: 97 active+clean; 2.1 GiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:19:57.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:56 smithi062 ceph-mon[95543]: pgmap v486: 97 pgs: 97 active+clean; 2.1 GiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:19:57.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:56 smithi081 ceph-mon[127795]: pgmap v486: 97 pgs: 97 active+clean; 2.1 GiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:19:59.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:19:58 smithi164 ceph-mon[97427]: pgmap v487: 97 pgs: 97 active+clean; 2.2 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-14T16:19:59.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:19:58 smithi062 ceph-mon[95543]: pgmap v487: 97 pgs: 97 active+clean; 2.2 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-14T16:19:59.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:19:58 smithi081 ceph-mon[127795]: pgmap v487: 97 pgs: 97 active+clean; 2.2 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-14T16:20:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.024+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/4091397284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 msgr2=0x7fb6e4101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.024+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/4091397284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 0x7fb6e4101150 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009990 tx=0x7fb6d4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.025+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/4091397284 shutdown_connections 2023-07-14T16:20:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.025+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/4091397284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e4101690 0x7fb6e4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.025+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/4091397284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6e40ffb10 0x7fb6e40fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.025+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/4091397284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 0x7fb6e4101150 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.025+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/4091397284 >> 172.21.15.62:0/4091397284 conn(0x7fb6e40fb130 msgr2=0x7fb6e40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.025+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/4091397284 shutdown_connections 2023-07-14T16:20:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.025+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/4091397284 wait complete. 2023-07-14T16:20:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.026+0000 7fb6eb7ff700 1 Processor -- start 2023-07-14T16:20:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.026+0000 7fb6eb7ff700 1 -- start start 2023-07-14T16:20:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.026+0000 7fb6eb7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6e40ffb10 0x7fb6e4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.026+0000 7fb6eb7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 0x7fb6e4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.026+0000 7fb6eb7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e4101690 0x7fb6e4073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.026+0000 7fb6eb7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e419f810 con 0x7fb6e4100d00 2023-07-14T16:20:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.026+0000 7fb6eb7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e419f950 con 0x7fb6e4101690 2023-07-14T16:20:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.026+0000 7fb6eb7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e419fa90 con 0x7fb6e40ffb10 2023-07-14T16:20:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 0x7fb6e4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 0x7fb6e4072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56424/0 (socket says 172.21.15.62:56424) 2023-07-14T16:20:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 -- 172.21.15.62:0/1536452852 learned_addr learned my addr 172.21.15.62:0/1536452852 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e959b700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6e40ffb10 0x7fb6e4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e4101690 0x7fb6e4073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 -- 172.21.15.62:0/1536452852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6e40ffb10 msgr2=0x7fb6e4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6e40ffb10 0x7fb6e4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 -- 172.21.15.62:0/1536452852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e4101690 msgr2=0x7fb6e4073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e4101690 0x7fb6e4073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 -- 172.21.15.62:0/1536452852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40095f0 con 0x7fb6e4100d00 2023-07-14T16:20:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e4101690 0x7fb6e4073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.027+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 0x7fb6e4072e40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005910 tx=0x7fb6d4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.028+0000 7fb6da7fc700 1 -- 172.21.15.62:0/1536452852 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d401d070 con 0x7fb6e4100d00 2023-07-14T16:20:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.028+0000 7fb6da7fc700 1 -- 172.21.15.62:0/1536452852 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6d400be90 con 0x7fb6e4100d00 2023-07-14T16:20:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.028+0000 7fb6da7fc700 1 -- 172.21.15.62:0/1536452852 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4022e60 con 0x7fb6e4100d00 2023-07-14T16:20:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.028+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/1536452852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e419fcc0 con 0x7fb6e4100d00 2023-07-14T16:20:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.028+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/1536452852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e41a0090 con 0x7fb6e4100d00 2023-07-14T16:20:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.031+0000 7fb6da7fc700 1 -- 172.21.15.62:0/1536452852 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb6d4022600 con 0x7fb6e4100d00 2023-07-14T16:20:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.033+0000 7fb6da7fc700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6d00a11d0 0x7fb6d00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.033+0000 7fb6da7fc700 1 -- 172.21.15.62:0/1536452852 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6d40cdcc0 con 0x7fb6e4100d00 2023-07-14T16:20:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.033+0000 7fb6e959b700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6d00a11d0 0x7fb6d00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6ff071b80 1 Processor -- start 2023-07-14T16:20:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6ff071b80 1 -- start start 2023-07-14T16:20:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6e959b700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6d00a11d0 0x7fb6d00a3680 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4074210 tx=0x7fb6e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1be840 0x55cd8e1bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e0c3c50 con 0x55cd8e1be840 2023-07-14T16:20:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e1169e0 con 0x55cd8e1bf6e0 2023-07-14T16:20:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e10fe70 con 0x55cd8e01a6b0 2023-07-14T16:20:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.034+0000 7fb6e8d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1be840 0x55cd8e1bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e959b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e8d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1be840 0x55cd8e1bf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56432/0 (socket says 172.21.15.62:56432) 2023-07-14T16:20:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e9d9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e8d9a700 1 -- 172.21.15.62:0/1752495055 learned_addr learned my addr 172.21.15.62:0/1752495055 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e8d9a700 1 -- 172.21.15.62:0/1752495055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 msgr2=0x55cd8e1be300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/1752495055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1be300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e8d9a700 1 -- 172.21.15.62:0/1752495055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1bf6e0 msgr2=0x55cd8e1bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/1752495055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1bfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e8d9a700 1 -- 172.21.15.62:0/1752495055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd8df3dd80 con 0x55cd8e1be840 2023-07-14T16:20:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/1752495055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.035+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/1752495055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1be840 0x55cd8e1bf1a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40a2fa0 tx=0x7fb6d409b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdac68b88d6fc03a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6cf7fe700 1 -- 172.21.15.62:0/1752495055 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d409f140 con 0x55cd8e1be840 2023-07-14T16:20:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6cf7fe700 1 -- 172.21.15.62:0/1752495055 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6d40a34d0 con 0x55cd8e1be840 2023-07-14T16:20:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6cf7fe700 1 -- 172.21.15.62:0/1752495055 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d409a820 con 0x55cd8e1be840 2023-07-14T16:20:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6ff071b80 1 -- 172.21.15.62:0/1752495055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1be840 msgr2=0x55cd8e1bf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/1752495055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1be840 0x55cd8e1bf1a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40a2fa0 tx=0x7fb6d409b000 comp rx=0 tx=0).stop 2023-07-14T16:20:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6ff071b80 1 -- 172.21.15.62:0/1752495055 shutdown_connections 2023-07-14T16:20:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/1752495055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/1752495055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1be300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/1752495055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1be840 0x55cd8e1bf1a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.036+0000 7fb6ff071b80 1 -- 172.21.15.62:0/1752495055 >> 172.21.15.62:0/1752495055 conn(0x55cd8e1b3be0 msgr2=0x55cd8e1b8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.037+0000 7fb6ff071b80 1 -- 172.21.15.62:0/1752495055 shutdown_connections 2023-07-14T16:20:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.037+0000 7fb6ff071b80 1 -- 172.21.15.62:0/1752495055 wait complete. 2023-07-14T16:20:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.037+0000 7fb6ff071b80 1 -- start start 2023-07-14T16:20:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.037+0000 7fb6ff071b80 1 Processor -- start 2023-07-14T16:20:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e0afa60 con 0x55cd8e1bf6e0 2023-07-14T16:20:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e0af760 con 0x55cd8e1be840 2023-07-14T16:20:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8df938c0 con 0x55cd8e01a6b0 2023-07-14T16:20:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6ff071b80 10 client.0 _ll_register_callbacks cb 0x55cd8e0ad700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6e9d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6e959b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6e9d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56442/0 (socket says 172.21.15.62:56442) 2023-07-14T16:20:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.038+0000 7fb6e9d9c700 1 -- 172.21.15.62:0/2362238395 learned_addr learned my addr 172.21.15.62:0/2362238395 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e959b700 1 -- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 msgr2=0x55cd8e1c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e959b700 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e959b700 1 -- 172.21.15.62:0/2362238395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1bf6e0 msgr2=0x55cd8e1cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e959b700 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e959b700 1 -- 172.21.15.62:0/2362238395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e41a0090 con 0x55cd8e01a6b0 2023-07-14T16:20:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6e959b700 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c74f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4100b60 tx=0x7fb6e00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda80a04cfb6442d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.039+0000 7fb6ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:20:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.040+0000 7fb6ccff9700 1 -- 172.21.15.62:0/2362238395 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0004020 con 0x55cd8e01a6b0 2023-07-14T16:20:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.040+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2362238395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd8e069490 con 0x55cd8e01a6b0 2023-07-14T16:20:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.040+0000 7fb6ccff9700 1 -- 172.21.15.62:0/2362238395 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6e0017070 con 0x55cd8e01a6b0 2023-07-14T16:20:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.040+0000 7fb6ccff9700 1 -- 172.21.15.62:0/2362238395 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6e002d030 con 0x55cd8e01a6b0 2023-07-14T16:20:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.040+0000 7fb6ff071b80 10 client.26320 fetch_fsmap learned FSMap version 24 2023-07-14T16:20:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.040+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2362238395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cd8df26c00 con 0x55cd8e01a6b0 2023-07-14T16:20:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6ccff9700 1 -- 172.21.15.62:0/2362238395 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cd8e069490 con 0x55cd8e01a6b0 2023-07-14T16:20:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6ccff9700 1 -- 172.21.15.62:0/2362238395 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb6e001f430 con 0x55cd8e01a6b0 2023-07-14T16:20:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6ff071b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:20:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6ff071b80 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6ff071b80 20 client.26320 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd8e1d33d0 0x55cd8e1d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6ff071b80 4 client.26320 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2362238395 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd8e108e50 con 0x55cd8e1d33d0 2023-07-14T16:20:01.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:00 smithi164 ceph-mon[97427]: pgmap v488: 97 pgs: 97 active+clean; 2.2 GiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:20:01.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:00 smithi164 ceph-mon[97427]: overall HEALTH_OK 2023-07-14T16:20:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.041+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd8e1d33d0 0x55cd8e1d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.042+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd8e1d33d0 0x55cd8e1d57e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f092cc20a8e76bcc server_cookie=abc3186f7be8dc1b in_seq=0 out_seq=0 2023-07-14T16:20:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.042+0000 7fb6ccff9700 10 client.26320 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ccff9700 1 -- 172.21.15.62:0/2362238395 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd8e108e50 con 0x55cd8e1d33d0 2023-07-14T16:20:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ccff9700 10 client.26320 handle_command_reply: tid=0 2023-07-14T16:20:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 client.26320 shutdown 2023-07-14T16:20:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2362238395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 msgr2=0x55cd8e1c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c74f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4100b60 tx=0x7fb6e00047c0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2362238395 shutdown_connections 2023-07-14T16:20:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c74f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd8e1d33d0 0x55cd8e1d57e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ccff9700 0 client.26320 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2362238395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2362238395 >> 172.21.15.62:0/2362238395 conn(0x55cd8e1b3be0 msgr2=0x55cd8e1b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2362238395 shutdown_connections 2023-07-14T16:20:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.043+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2362238395 wait complete. 2023-07-14T16:20:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.044+0000 7fb6ff071b80 20 client.26320 trim_cache size 0 max 16384 2023-07-14T16:20:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6ff071b80 1 Processor -- start 2023-07-14T16:20:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6ff071b80 1 -- start start 2023-07-14T16:20:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e1c70c0 con 0x55cd8e01a6b0 2023-07-14T16:20:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e1ba840 con 0x55cd8e1be840 2023-07-14T16:20:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e108e80 con 0x55cd8e1bf6e0 2023-07-14T16:20:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.062+0000 7fb6e959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56444/0 (socket says 172.21.15.62:56444) 2023-07-14T16:20:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e959b700 1 -- 172.21.15.62:0/2622879266 learned_addr learned my addr 172.21.15.62:0/2622879266 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e9d9c700 1 -- 172.21.15.62:0/2622879266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 msgr2=0x55cd8e1cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e9d9c700 1 -- 172.21.15.62:0/2622879266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 msgr2=0x55cd8e1c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e9d9c700 1 -- 172.21.15.62:0/2622879266 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd8df641f0 con 0x55cd8e1bf6e0 2023-07-14T16:20:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e959b700 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.063+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1c77e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009fd0 tx=0x7fb6dc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d15eab1df70b601 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6cffff700 1 -- 172.21.15.62:0/2622879266 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00e040 con 0x55cd8e1bf6e0 2023-07-14T16:20:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6cffff700 1 -- 172.21.15.62:0/2622879266 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6dc00f040 con 0x55cd8e1bf6e0 2023-07-14T16:20:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6cffff700 1 -- 172.21.15.62:0/2622879266 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc017860 con 0x55cd8e1bf6e0 2023-07-14T16:20:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2622879266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 msgr2=0x55cd8e1c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1c77e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009fd0 tx=0x7fb6dc00b040 comp rx=0 tx=0).stop 2023-07-14T16:20:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2622879266 shutdown_connections 2023-07-14T16:20:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1c77e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/2622879266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.064+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2622879266 >> 172.21.15.62:0/2622879266 conn(0x55cd8e1b3be0 msgr2=0x55cd8e1da100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.065+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2622879266 shutdown_connections 2023-07-14T16:20:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.065+0000 7fb6ff071b80 1 -- 172.21.15.62:0/2622879266 wait complete. 2023-07-14T16:20:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.065+0000 7fb6ff071b80 1 -- start start 2023-07-14T16:20:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.065+0000 7fb6ff071b80 1 Processor -- start 2023-07-14T16:20:01.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6ff071b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e1cde30 con 0x55cd8e01a6b0 2023-07-14T16:20:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e1cdfa0 con 0x55cd8e1be840 2023-07-14T16:20:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6ff071b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd8e1ce160 con 0x55cd8e1bf6e0 2023-07-14T16:20:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6ff071b80 10 client.0 _ll_register_callbacks cb 0x7fb6e0010d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6e959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6e959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56454/0 (socket says 172.21.15.62:56454) 2023-07-14T16:20:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6e959b700 1 -- 172.21.15.62:0/3431219938 learned_addr learned my addr 172.21.15.62:0/3431219938 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.066+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6e959b700 1 -- 172.21.15.62:0/3431219938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 msgr2=0x55cd8e1cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6e959b700 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6e959b700 1 -- 172.21.15.62:0/3431219938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 msgr2=0x55cd8e1ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6e959b700 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6e959b700 1 -- 172.21.15.62:0/3431219938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd8df26c00 con 0x55cd8e01a6b0 2023-07-14T16:20:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6e9d9c700 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6e959b700 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1ca390 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb6e001f6e0 tx=0x7fb6e000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a328121caaecae3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6ccff9700 1 -- 172.21.15.62:0/3431219938 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e002d040 con 0x55cd8e01a6b0 2023-07-14T16:20:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6ff071b80 1 -- 172.21.15.62:0/3431219938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6d40a34d0 con 0x55cd8e01a6b0 2023-07-14T16:20:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.067+0000 7fb6ccff9700 1 -- 172.21.15.62:0/3431219938 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6e0017040 con 0x55cd8e01a6b0 2023-07-14T16:20:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ccff9700 1 -- 172.21.15.62:0/3431219938 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6e001fb40 con 0x55cd8e01a6b0 2023-07-14T16:20:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ff071b80 10 client.16710 fetch_fsmap learned FSMap version 24 2023-07-14T16:20:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ff071b80 1 -- 172.21.15.62:0/3431219938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fb6dc007500 con 0x55cd8e01a6b0 2023-07-14T16:20:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ccff9700 1 -- 172.21.15.62:0/3431219938 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6d40a34d0 con 0x55cd8e01a6b0 2023-07-14T16:20:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ccff9700 1 -- 172.21.15.62:0/3431219938 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb6e0027420 con 0x55cd8e01a6b0 2023-07-14T16:20:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ff071b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T16:20:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ff071b80 10 client.16710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ff071b80 20 client.16710 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd8e227e40 0x55cd8e22a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ff071b80 4 client.16710 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:20:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.068+0000 7fb6ff071b80 1 -- 172.21.15.62:0/3431219938 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd8e0caff0 con 0x55cd8e227e40 2023-07-14T16:20:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.069+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd8e227e40 0x55cd8e22a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.069+0000 7fb6ccff9700 10 client.16710 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.069+0000 7fb6e8d9a700 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd8e227e40 0x55cd8e22a250 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a18d86b41ededff server_cookie=b9269f17c896100c in_seq=0 out_seq=0 2023-07-14T16:20:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.078+0000 7fb6ccff9700 1 -- 172.21.15.62:0/3431219938 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd8e0caff0 con 0x55cd8e227e40 2023-07-14T16:20:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.078+0000 7fb6ccff9700 10 client.16710 handle_command_reply: tid=0 2023-07-14T16:20:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.078+0000 7fb6ff071b80 1 client.16710 shutdown 2023-07-14T16:20:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 -- 172.21.15.62:0/3431219938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 msgr2=0x55cd8e1ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1ca390 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb6e001f6e0 tx=0x7fb6e000b680 comp rx=0 tx=0).stop 2023-07-14T16:20:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 -- 172.21.15.62:0/3431219938 shutdown_connections 2023-07-14T16:20:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd8e1bf6e0 0x55cd8e1cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd8e01a6b0 0x55cd8e1ca390 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd8e227e40 0x55cd8e22a250 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 --2- 172.21.15.62:0/3431219938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd8e1be840 0x55cd8e1ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 -- 172.21.15.62:0/3431219938 >> 172.21.15.62:0/3431219938 conn(0x55cd8e1b3be0 msgr2=0x55cd8e1c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ccff9700 0 client.16710 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 -- 172.21.15.62:0/3431219938 shutdown_connections 2023-07-14T16:20:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 1 -- 172.21.15.62:0/3431219938 wait complete. 2023-07-14T16:20:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.079+0000 7fb6ff071b80 20 client.16710 trim_cache size 0 max 16384 2023-07-14T16:20:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:20:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:20:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:20:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:20:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6d00a11d0 msgr2=0x7fb6d00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6d00a11d0 0x7fb6d00a3680 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4074210 tx=0x7fb6e000b410 comp rx=0 tx=0).stop 2023-07-14T16:20:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 msgr2=0x7fb6e4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 0x7fb6e4072e40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005910 tx=0x7fb6d4005720 comp rx=0 tx=0).stop 2023-07-14T16:20:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/1536452852 shutdown_connections 2023-07-14T16:20:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6d00a11d0 0x7fb6d00a3680 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6e4101690 0x7fb6e4073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6e40ffb10 0x7fb6e4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 --2- 172.21.15.62:0/1536452852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6e4100d00 0x7fb6e4072e40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/1536452852 >> 172.21.15.62:0/1536452852 conn(0x7fb6e40fb130 msgr2=0x7fb6e4104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/1536452852 shutdown_connections 2023-07-14T16:20:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:01.080+0000 7fb6eb7ff700 1 -- 172.21.15.62:0/1536452852 wait complete. 2023-07-14T16:20:01.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:00 smithi062 ceph-mon[95543]: pgmap v488: 97 pgs: 97 active+clean; 2.2 GiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:20:01.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:00 smithi062 ceph-mon[95543]: overall HEALTH_OK 2023-07-14T16:20:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:00 smithi081 ceph-mon[127795]: pgmap v488: 97 pgs: 97 active+clean; 2.2 GiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:20:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:00 smithi081 ceph-mon[127795]: overall HEALTH_OK 2023-07-14T16:20:03.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:02 smithi164 ceph-mon[97427]: pgmap v489: 97 pgs: 97 active+clean; 2.2 GiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-14T16:20:03.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:02 smithi062 ceph-mon[95543]: pgmap v489: 97 pgs: 97 active+clean; 2.2 GiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-14T16:20:03.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:02 smithi081 ceph-mon[127795]: pgmap v489: 97 pgs: 97 active+clean; 2.2 GiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-14T16:20:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:04 smithi164 ceph-mon[97427]: pgmap v490: 97 pgs: 97 active+clean; 2.3 GiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 15 op/s 2023-07-14T16:20:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:04 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:20:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:04 smithi062 ceph-mon[95543]: pgmap v490: 97 pgs: 97 active+clean; 2.3 GiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 15 op/s 2023-07-14T16:20:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:04 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:20:05.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:04 smithi081 ceph-mon[127795]: pgmap v490: 97 pgs: 97 active+clean; 2.3 GiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 15 op/s 2023-07-14T16:20:05.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:04 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:20:07.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:06 smithi164 ceph-mon[97427]: pgmap v491: 97 pgs: 97 active+clean; 2.3 GiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:07.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:06 smithi062 ceph-mon[95543]: pgmap v491: 97 pgs: 97 active+clean; 2.3 GiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:07.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:06 smithi081 ceph-mon[127795]: pgmap v491: 97 pgs: 97 active+clean; 2.3 GiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:08 smithi164 ceph-mon[97427]: pgmap v492: 97 pgs: 97 active+clean; 2.3 GiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:09.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:08 smithi062 ceph-mon[95543]: pgmap v492: 97 pgs: 97 active+clean; 2.3 GiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:09.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:08 smithi081 ceph-mon[127795]: pgmap v492: 97 pgs: 97 active+clean; 2.3 GiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:11.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:10 smithi164 ceph-mon[97427]: pgmap v493: 97 pgs: 97 active+clean; 2.3 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:20:11.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:10 smithi062 ceph-mon[95543]: pgmap v493: 97 pgs: 97 active+clean; 2.3 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:20:11.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:10 smithi081 ceph-mon[127795]: pgmap v493: 97 pgs: 97 active+clean; 2.3 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:20:13.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:12 smithi164 ceph-mon[97427]: pgmap v494: 97 pgs: 97 active+clean; 2.4 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:20:13.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:12 smithi062 ceph-mon[95543]: pgmap v494: 97 pgs: 97 active+clean; 2.4 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:20:13.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:12 smithi081 ceph-mon[127795]: pgmap v494: 97 pgs: 97 active+clean; 2.4 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:20:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 -- 172.21.15.62:0/2875395692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc0744f0 msgr2=0x7f87dc072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 --2- 172.21.15.62:0/2875395692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc0744f0 0x7f87dc072940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0098f0 tx=0x7f87cc009c00 comp rx=0 tx=0).stop 2023-07-14T16:20:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 -- 172.21.15.62:0/2875395692 shutdown_connections 2023-07-14T16:20:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 --2- 172.21.15.62:0/2875395692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87dc1077d0 0x7f87dc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 --2- 172.21.15.62:0/2875395692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc0744f0 0x7f87dc072940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 --2- 172.21.15.62:0/2875395692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc072f10 0x7f87dc073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 -- 172.21.15.62:0/2875395692 >> 172.21.15.62:0/2875395692 conn(0x7f87dc0faf80 msgr2=0x7f87dc0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 -- 172.21.15.62:0/2875395692 shutdown_connections 2023-07-14T16:20:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.254+0000 7f87e099d700 1 -- 172.21.15.62:0/2875395692 wait complete. 2023-07-14T16:20:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.255+0000 7f87e099d700 1 Processor -- start 2023-07-14T16:20:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.255+0000 7f87e099d700 1 -- start start 2023-07-14T16:20:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87e099d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc072f10 0x7f87dc1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87e099d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0744f0 0x7f87dc197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87e099d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87dc1077d0 0x7f87dc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87e099d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc19f350 con 0x7f87dc1077d0 2023-07-14T16:20:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87e099d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc19f490 con 0x7f87dc072f10 2023-07-14T16:20:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87e099d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc19f620 con 0x7f87dc0744f0 2023-07-14T16:20:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87dc1077d0 0x7f87dc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc072f10 0x7f87dc1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87dc1077d0 0x7f87dc19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34306/0 (socket says 172.21.15.62:34306) 2023-07-14T16:20:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc072f10 0x7f87dc1977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50642/0 (socket says 172.21.15.62:50642) 2023-07-14T16:20:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87dbfff700 1 -- 172.21.15.62:0/3775788631 learned_addr learned my addr 172.21.15.62:0/3775788631 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.256+0000 7f87daffd700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0744f0 0x7f87dc197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87dbfff700 1 -- 172.21.15.62:0/3775788631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0744f0 msgr2=0x7f87dc197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87dbfff700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0744f0 0x7f87dc197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87dbfff700 1 -- 172.21.15.62:0/3775788631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc072f10 msgr2=0x7f87dc1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87dbfff700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc072f10 0x7f87dc1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87dbfff700 1 -- 172.21.15.62:0/3775788631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc009520 con 0x7f87dc1077d0 2023-07-14T16:20:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87db7fe700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc072f10 0x7f87dc1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87dbfff700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87dc1077d0 0x7f87dc19ed80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f87d400c110 tx=0x7f87d400c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87d8ff9700 1 -- 172.21.15.62:0/3775788631 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4017070 con 0x7f87dc1077d0 2023-07-14T16:20:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87d8ff9700 1 -- 172.21.15.62:0/3775788631 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87d400e050 con 0x7f87dc1077d0 2023-07-14T16:20:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87e099d700 1 -- 172.21.15.62:0/3775788631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc19f8a0 con 0x7f87dc1077d0 2023-07-14T16:20:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.257+0000 7f87d8ff9700 1 -- 172.21.15.62:0/3775788631 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4007820 con 0x7f87dc1077d0 2023-07-14T16:20:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.258+0000 7f87e099d700 1 -- 172.21.15.62:0/3775788631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc19fe00 con 0x7f87dc1077d0 2023-07-14T16:20:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.261+0000 7f87d8ff9700 1 -- 172.21.15.62:0/3775788631 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f87d400ca20 con 0x7f87dc1077d0 2023-07-14T16:20:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.263+0000 7f87f54b5b80 1 Processor -- start 2023-07-14T16:20:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.263+0000 7f87d8ff9700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87c40a11d0 0x7f87c40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.263+0000 7f87d8ff9700 1 -- 172.21.15.62:0/3775788631 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87d40cc210 con 0x7f87dc1077d0 2023-07-14T16:20:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.263+0000 7f87f54b5b80 1 -- start start 2023-07-14T16:20:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87daffd700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87c40a11d0 0x7f87c40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9e7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e87e0 0x564a0e9e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e8ecc50 con 0x564a0e8436b0 2023-07-14T16:20:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e93f9e0 con 0x564a0e9e87e0 2023-07-14T16:20:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e938e70 con 0x564a0e9e7a90 2023-07-14T16:20:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9e7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34312/0 (socket says 172.21.15.62:34312) 2023-07-14T16:20:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87db7fe700 1 -- 172.21.15.62:0/617812628 learned_addr learned my addr 172.21.15.62:0/617812628 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87daffd700 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.264+0000 7f87dbfff700 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e87e0 0x564a0e9e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.265+0000 7f87daffd700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87c40a11d0 0x7f87c40a3680 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87d0007cb0 tx=0x7f87d00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.265+0000 7f87db7fe700 1 -- 172.21.15.62:0/617812628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 msgr2=0x564a0e9e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.265+0000 7f87db7fe700 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.265+0000 7f87db7fe700 1 -- 172.21.15.62:0/617812628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e87e0 msgr2=0x564a0e9e8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.265+0000 7f87db7fe700 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e87e0 0x564a0e9e8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.265+0000 7f87db7fe700 1 -- 172.21.15.62:0/617812628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a0e766d80 con 0x564a0e8436b0 2023-07-14T16:20:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.265+0000 7f87db7fe700 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9e7550 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0098c0 tx=0x7f87cc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134499f52c9d353c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.265+0000 7f87c1ffb700 1 -- 172.21.15.62:0/617812628 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc01c050 con 0x564a0e8436b0 2023-07-14T16:20:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87c1ffb700 1 -- 172.21.15.62:0/617812628 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87cc01d040 con 0x564a0e8436b0 2023-07-14T16:20:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87c1ffb700 1 -- 172.21.15.62:0/617812628 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0053a0 con 0x564a0e8436b0 2023-07-14T16:20:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87f54b5b80 1 -- 172.21.15.62:0/617812628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 msgr2=0x564a0e9e7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9e7550 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0098c0 tx=0x7f87cc005dc0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87f54b5b80 1 -- 172.21.15.62:0/617812628 shutdown_connections 2023-07-14T16:20:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e87e0 0x564a0e9e8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9e7550 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/617812628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.266+0000 7f87f54b5b80 1 -- 172.21.15.62:0/617812628 >> 172.21.15.62:0/617812628 conn(0x564a0e9dcd40 msgr2=0x564a0e9e1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.267+0000 7f87f54b5b80 1 -- 172.21.15.62:0/617812628 shutdown_connections 2023-07-14T16:20:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.267+0000 7f87f54b5b80 1 -- 172.21.15.62:0/617812628 wait complete. 2023-07-14T16:20:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.267+0000 7f87f54b5b80 1 -- start start 2023-07-14T16:20:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.267+0000 7f87f54b5b80 1 Processor -- start 2023-07-14T16:20:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e7a90 0x564a0e9f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e87e0 0x564a0e9f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e8d8a60 con 0x564a0e8436b0 2023-07-14T16:20:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e8d8760 con 0x564a0e9e7a90 2023-07-14T16:20:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e7bc8c0 con 0x564a0e9e87e0 2023-07-14T16:20:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87f54b5b80 10 client.0 _ll_register_callbacks cb 0x564a0e8d6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9f09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34320/0 (socket says 172.21.15.62:34320) 2023-07-14T16:20:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.268+0000 7f87db7fe700 1 -- 172.21.15.62:0/601396334 learned_addr learned my addr 172.21.15.62:0/601396334 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87dbfff700 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e87e0 0x564a0e9f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87daffd700 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e7a90 0x564a0e9f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87db7fe700 1 -- 172.21.15.62:0/601396334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e87e0 msgr2=0x564a0e9f7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87db7fe700 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e87e0 0x564a0e9f7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87db7fe700 1 -- 172.21.15.62:0/601396334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e7a90 msgr2=0x564a0e9f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87db7fe700 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e7a90 0x564a0e9f0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87db7fe700 1 -- 172.21.15.62:0/601396334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc10a870 con 0x564a0e8436b0 2023-07-14T16:20:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87db7fe700 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9f09f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f87cc005e90 tx=0x7f87cc00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462a74c002054e12 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87baffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87baffd700 1 -- 172.21.15.62:0/601396334 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc01c070 con 0x564a0e8436b0 2023-07-14T16:20:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87f54b5b80 1 -- 172.21.15.62:0/601396334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a0e892490 con 0x564a0e8436b0 2023-07-14T16:20:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.269+0000 7f87baffd700 1 -- 172.21.15.62:0/601396334 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87cc005530 con 0x564a0e8436b0 2023-07-14T16:20:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87baffd700 1 -- 172.21.15.62:0/601396334 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87cc00bac0 con 0x564a0e8436b0 2023-07-14T16:20:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87f54b5b80 10 client.16722 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87f54b5b80 1 -- 172.21.15.62:0/601396334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564a0e74fc00 con 0x564a0e8436b0 2023-07-14T16:20:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87baffd700 1 -- 172.21.15.62:0/601396334 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564a0e892490 con 0x564a0e8436b0 2023-07-14T16:20:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87baffd700 1 -- 172.21.15.62:0/601396334 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f87cc009c70 con 0x564a0e8436b0 2023-07-14T16:20:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87f54b5b80 10 client.16722 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87f54b5b80 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87f54b5b80 20 client.16722 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a0e9fc3c0 0x564a0e9fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87f54b5b80 4 client.16722 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.270+0000 7f87f54b5b80 1 -- 172.21.15.62:0/601396334 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a0e931e50 con 0x564a0e9fc3c0 2023-07-14T16:20:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.271+0000 7f87daffd700 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a0e9fc3c0 0x564a0e9fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.271+0000 7f87daffd700 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a0e9fc3c0 0x564a0e9fe7d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0ee249f01aab64 server_cookie=ed8e554800cc449f in_seq=0 out_seq=0 2023-07-14T16:20:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.271+0000 7f87baffd700 10 client.16722 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.272+0000 7f87baffd700 1 -- 172.21.15.62:0/601396334 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a0e931e50 con 0x564a0e9fc3c0 2023-07-14T16:20:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.272+0000 7f87baffd700 10 client.16722 handle_command_reply: tid=0 2023-07-14T16:20:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.272+0000 7f87f54b5b80 1 client.16722 shutdown 2023-07-14T16:20:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.272+0000 7f87f54b5b80 1 -- 172.21.15.62:0/601396334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 msgr2=0x564a0e9f09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.272+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9f09f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f87cc005e90 tx=0x7f87cc00b820 comp rx=0 tx=0).stop 2023-07-14T16:20:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 1 -- 172.21.15.62:0/601396334 shutdown_connections 2023-07-14T16:20:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e87e0 0x564a0e9f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e8436b0 0x564a0e9f09f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a0e9fc3c0 0x564a0e9fe7d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/601396334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e9e7a90 0x564a0e9f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87baffd700 0 client.16722 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 1 -- 172.21.15.62:0/601396334 >> 172.21.15.62:0/601396334 conn(0x564a0e9dcd40 msgr2=0x564a0e9e1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 1 -- 172.21.15.62:0/601396334 shutdown_connections 2023-07-14T16:20:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 1 -- 172.21.15.62:0/601396334 wait complete. 2023-07-14T16:20:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.273+0000 7f87f54b5b80 20 client.16722 trim_cache size 0 max 16384 2023-07-14T16:20:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.295+0000 7f87f54b5b80 1 Processor -- start 2023-07-14T16:20:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.295+0000 7f87f54b5b80 1 -- start start 2023-07-14T16:20:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.295+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.295+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.295+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0ea030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.295+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e9f05c0 con 0x564a0e9e87e0 2023-07-14T16:20:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.295+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e9dee60 con 0x564a0e8436b0 2023-07-14T16:20:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.295+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e9e8100 con 0x564a0e9e7a90 2023-07-14T16:20:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0ea030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0ea030f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34332/0 (socket says 172.21.15.62:34332) 2023-07-14T16:20:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87dbfff700 1 -- 172.21.15.62:0/301737150 learned_addr learned my addr 172.21.15.62:0/301737150 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87db7fe700 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87daffd700 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87db7fe700 1 -- 172.21.15.62:0/301737150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 msgr2=0x564a0e9f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87db7fe700 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87db7fe700 1 -- 172.21.15.62:0/301737150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 msgr2=0x564a0ea030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87db7fe700 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0ea030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87db7fe700 1 -- 172.21.15.62:0/301737150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a0e78d1f0 con 0x564a0e8436b0 2023-07-14T16:20:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.296+0000 7f87dbfff700 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0ea030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.297+0000 7f87db7fe700 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9e1e40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009810 tx=0x7f87cc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8177043d54762a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.297+0000 7f87c27fc700 1 -- 172.21.15.62:0/301737150 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc01c040 con 0x564a0e8436b0 2023-07-14T16:20:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.297+0000 7f87c27fc700 1 -- 172.21.15.62:0/301737150 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87cc02c430 con 0x564a0e8436b0 2023-07-14T16:20:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.297+0000 7f87c27fc700 1 -- 172.21.15.62:0/301737150 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc02ce90 con 0x564a0e8436b0 2023-07-14T16:20:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 -- 172.21.15.62:0/301737150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 msgr2=0x564a0e9e1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9e1e40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009810 tx=0x7f87cc000bc0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 -- 172.21.15.62:0/301737150 shutdown_connections 2023-07-14T16:20:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0ea030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9e1e40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/301737150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 -- 172.21.15.62:0/301737150 >> 172.21.15.62:0/301737150 conn(0x564a0e9dcd40 msgr2=0x564a0e9ef760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:14.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 -- 172.21.15.62:0/301737150 shutdown_connections 2023-07-14T16:20:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 -- 172.21.15.62:0/301737150 wait complete. 2023-07-14T16:20:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 -- start start 2023-07-14T16:20:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.298+0000 7f87f54b5b80 1 Processor -- start 2023-07-14T16:20:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.299+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.299+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.299+0000 7f87f54b5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0e9f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.299+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e9f7190 con 0x564a0e9e87e0 2023-07-14T16:20:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.299+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e9f7300 con 0x564a0e8436b0 2023-07-14T16:20:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.299+0000 7f87f54b5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0e9f74c0 con 0x564a0e9e7a90 2023-07-14T16:20:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.299+0000 7f87f54b5b80 10 client.0 _ll_register_callbacks cb 0x7f87d40952d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.299+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0e9f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0e9f6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34338/0 (socket says 172.21.15.62:34338) 2023-07-14T16:20:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87dbfff700 1 -- 172.21.15.62:0/4054251239 learned_addr learned my addr 172.21.15.62:0/4054251239 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87daffd700 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87db7fe700 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87dbfff700 1 -- 172.21.15.62:0/4054251239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 msgr2=0x564a0e9f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87dbfff700 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87dbfff700 1 -- 172.21.15.62:0/4054251239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 msgr2=0x564a0e9f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87dbfff700 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87dbfff700 1 -- 172.21.15.62:0/4054251239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc19fe00 con 0x564a0e9e87e0 2023-07-14T16:20:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87dbfff700 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0e9f6c50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009fd0 tx=0x7f87d40959f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e99cdd08cc9dfa5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87baffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87baffd700 1 -- 172.21.15.62:0/4054251239 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4099630 con 0x564a0e9e87e0 2023-07-14T16:20:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87baffd700 1 -- 172.21.15.62:0/4054251239 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87d409d660 con 0x564a0e9e87e0 2023-07-14T16:20:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.300+0000 7f87f54b5b80 1 -- 172.21.15.62:0/4054251239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87cc01d040 con 0x564a0e9e87e0 2023-07-14T16:20:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87baffd700 1 -- 172.21.15.62:0/4054251239 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d40a1a80 con 0x564a0e9e87e0 2023-07-14T16:20:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87f54b5b80 10 client.16728 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87f54b5b80 1 -- 172.21.15.62:0/4054251239 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564a0e74fc00 con 0x564a0e9e87e0 2023-07-14T16:20:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87baffd700 1 -- 172.21.15.62:0/4054251239 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87d40c7080 con 0x564a0e9e87e0 2023-07-14T16:20:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87baffd700 1 -- 172.21.15.62:0/4054251239 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f87d40840b0 con 0x564a0e9e87e0 2023-07-14T16:20:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87f54b5b80 10 client.16728 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87f54b5b80 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87f54b5b80 20 client.16728 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a0ea50cc0 0x564a0ea530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87f54b5b80 4 client.16728 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:20:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.301+0000 7f87f54b5b80 1 -- 172.21.15.62:0/4054251239 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a0e8f3ff0 con 0x564a0ea50cc0 2023-07-14T16:20:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.302+0000 7f87db7fe700 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a0ea50cc0 0x564a0ea530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.302+0000 7f87db7fe700 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a0ea50cc0 0x564a0ea530d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e03acc50ea6c5d server_cookie=fcdc82ed6ec136a9 in_seq=0 out_seq=0 2023-07-14T16:20:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.302+0000 7f87baffd700 10 client.16728 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87baffd700 1 -- 172.21.15.62:0/4054251239 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a0e8f3ff0 con 0x564a0ea50cc0 2023-07-14T16:20:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87baffd700 10 client.16728 handle_command_reply: tid=0 2023-07-14T16:20:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 client.16728 shutdown 2023-07-14T16:20:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 -- 172.21.15.62:0/4054251239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 msgr2=0x564a0e9f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0e9f6c50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009fd0 tx=0x7f87d40959f0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 -- 172.21.15.62:0/4054251239 shutdown_connections 2023-07-14T16:20:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a0e9e87e0 0x564a0e9f6c50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a0ea50cc0 0x564a0ea530d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a0e8436b0 0x564a0e9f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 --2- 172.21.15.62:0/4054251239 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a0e9e7a90 0x564a0e9f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87baffd700 0 client.16728 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.312+0000 7f87f54b5b80 1 -- 172.21.15.62:0/4054251239 >> 172.21.15.62:0/4054251239 conn(0x564a0e9dcd40 msgr2=0x564a0e9f07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.313+0000 7f87f54b5b80 1 -- 172.21.15.62:0/4054251239 shutdown_connections 2023-07-14T16:20:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.313+0000 7f87f54b5b80 1 -- 172.21.15.62:0/4054251239 wait complete. 2023-07-14T16:20:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.313+0000 7f87f54b5b80 20 client.16728 trim_cache size 0 max 16384 2023-07-14T16:20:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:20:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:20:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:20:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:20:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.313+0000 7f87e099d700 1 -- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87c40a11d0 msgr2=0x7f87c40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.313+0000 7f87e099d700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87c40a11d0 0x7f87c40a3680 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87d0007cb0 tx=0x7f87d00073d0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.313+0000 7f87e099d700 1 -- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87dc1077d0 msgr2=0x7f87dc19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.313+0000 7f87e099d700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87dc1077d0 0x7f87dc19ed80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f87d400c110 tx=0x7f87d400c420 comp rx=0 tx=0).stop 2023-07-14T16:20:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.314+0000 7f87e099d700 1 -- 172.21.15.62:0/3775788631 shutdown_connections 2023-07-14T16:20:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.314+0000 7f87e099d700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87dc1077d0 0x7f87dc19ed80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.314+0000 7f87e099d700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87c40a11d0 0x7f87c40a3680 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.314+0000 7f87e099d700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87dc072f10 0x7f87dc1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.314+0000 7f87e099d700 1 --2- 172.21.15.62:0/3775788631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0744f0 0x7f87dc197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.314+0000 7f87e099d700 1 -- 172.21.15.62:0/3775788631 >> 172.21.15.62:0/3775788631 conn(0x7f87dc0faf80 msgr2=0x7f87dc0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.314+0000 7f87e099d700 1 -- 172.21.15.62:0/3775788631 shutdown_connections 2023-07-14T16:20:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:14.314+0000 7f87e099d700 1 -- 172.21.15.62:0/3775788631 wait complete. 2023-07-14T16:20:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:14 smithi164 ceph-mon[97427]: pgmap v495: 97 pgs: 97 active+clean; 2.4 GiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:20:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:14 smithi062 ceph-mon[95543]: pgmap v495: 97 pgs: 97 active+clean; 2.4 GiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:20:15.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:14 smithi081 ceph-mon[127795]: pgmap v495: 97 pgs: 97 active+clean; 2.4 GiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:20:16.627 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:20:16.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 -- 172.21.15.62:0/4263745385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220102f00 msgr2=0x7f2220103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 --2- 172.21.15.62:0/4263745385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220102f00 0x7f2220103350 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2210009990 tx=0x7f2210009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 -- 172.21.15.62:0/4263745385 shutdown_connections 2023-07-14T16:20:16.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 --2- 172.21.15.62:0/4263745385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2220103890 0x7f222010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 --2- 172.21.15.62:0/4263745385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2220101d10 0x7f2220102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 --2- 172.21.15.62:0/4263745385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220102f00 0x7f2220103350 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 -- 172.21.15.62:0/4263745385 >> 172.21.15.62:0/4263745385 conn(0x7f22200fd350 msgr2=0x7f22200ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:16.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 -- 172.21.15.62:0/4263745385 shutdown_connections 2023-07-14T16:20:16.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.885+0000 7f22273ff700 1 -- 172.21.15.62:0/4263745385 wait complete. 2023-07-14T16:20:16.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.886+0000 7f22273ff700 1 Processor -- start 2023-07-14T16:20:16.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.886+0000 7f22273ff700 1 -- start start 2023-07-14T16:20:16.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f22273ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220101d10 0x7f2220197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f22273ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2220102f00 0x7f2220197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f22273ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2220103890 0x7f222019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f22273ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f4a0 con 0x7f2220101d10 2023-07-14T16:20:16.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f22273ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f5e0 con 0x7f2220102f00 2023-07-14T16:20:16.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f22273ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f770 con 0x7f2220103890 2023-07-14T16:20:16.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220101d10 0x7f2220197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220101d10 0x7f2220197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34358/0 (socket says 172.21.15.62:34358) 2023-07-14T16:20:16.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222519b700 1 -- 172.21.15.62:0/217483700 learned_addr learned my addr 172.21.15.62:0/217483700 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:16.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222499a700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2220102f00 0x7f2220197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222599c700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2220103890 0x7f222019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222519b700 1 -- 172.21.15.62:0/217483700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2220103890 msgr2=0x7f222019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222519b700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2220103890 0x7f222019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222519b700 1 -- 172.21.15.62:0/217483700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2220102f00 msgr2=0x7f2220197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.887+0000 7f222519b700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2220102f00 0x7f2220197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.888+0000 7f222519b700 1 -- 172.21.15.62:0/217483700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100095f0 con 0x7f2220101d10 2023-07-14T16:20:16.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.888+0000 7f222599c700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2220103890 0x7f222019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:16.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.888+0000 7f222519b700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220101d10 0x7f2220197920 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f221c00e970 tx=0x7f221c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:16.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.888+0000 7f22167fc700 1 -- 172.21.15.62:0/217483700 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c009790 con 0x7f2220101d10 2023-07-14T16:20:16.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.888+0000 7f22273ff700 1 -- 172.21.15.62:0/217483700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222019fa50 con 0x7f2220101d10 2023-07-14T16:20:16.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.888+0000 7f22167fc700 1 -- 172.21.15.62:0/217483700 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221c00eec0 con 0x7f2220101d10 2023-07-14T16:20:16.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.888+0000 7f22167fc700 1 -- 172.21.15.62:0/217483700 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c003c10 con 0x7f2220101d10 2023-07-14T16:20:16.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.888+0000 7f22273ff700 1 -- 172.21.15.62:0/217483700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222019ff60 con 0x7f2220101d10 2023-07-14T16:20:16.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.892+0000 7f22167fc700 1 -- 172.21.15.62:0/217483700 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f221c003db0 con 0x7f2220101d10 2023-07-14T16:20:16.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.893+0000 7f223adfbb80 1 Processor -- start 2023-07-14T16:20:16.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f223adfbb80 1 -- start start 2023-07-14T16:20:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f22167fc700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f220c0a1150 0x7f220c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe316414d0 0x55fe316418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f22167fc700 1 -- 172.21.15.62:0/217483700 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f221c014070 con 0x7f2220101d10 2023-07-14T16:20:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe316426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31642c70 0x55fe316430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f223adfbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31546cb0 con 0x55fe31641e20 2023-07-14T16:20:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f223adfbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31599a40 con 0x55fe316414d0 2023-07-14T16:20:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f223adfbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31592ed0 con 0x55fe31642c70 2023-07-14T16:20:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.894+0000 7f222599c700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f220c0a1150 0x7f220c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31642c70 0x55fe316430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31642c70 0x55fe316430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50122/0 (socket says 172.21.15.62:50122) 2023-07-14T16:20:16.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 -- 172.21.15.62:0/1829660402 learned_addr learned my addr 172.21.15.62:0/1829660402 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:16.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222499a700 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe316426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222519b700 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe316414d0 0x55fe316418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f220c0a1150 0x7f220c0a3600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2218007810 tx=0x7f2218007bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:16.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 -- 172.21.15.62:0/1829660402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe316414d0 msgr2=0x55fe316418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe316414d0 0x55fe316418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 -- 172.21.15.62:0/1829660402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 msgr2=0x55fe316426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe316426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 -- 172.21.15.62:0/1829660402 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe313c0de0 con 0x55fe31642c70 2023-07-14T16:20:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.895+0000 7f222599c700 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31642c70 0x55fe316430e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22180177c0 tx=0x7f221801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3dc4442f4c468c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.896+0000 7f220b7fe700 1 -- 172.21.15.62:0/1829660402 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221800e050 con 0x55fe31642c70 2023-07-14T16:20:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.896+0000 7f220b7fe700 1 -- 172.21.15.62:0/1829660402 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221800f040 con 0x55fe31642c70 2023-07-14T16:20:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.896+0000 7f220b7fe700 1 -- 172.21.15.62:0/1829660402 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218024880 con 0x55fe31642c70 2023-07-14T16:20:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 -- 172.21.15.62:0/1829660402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31642c70 msgr2=0x55fe316430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31642c70 0x55fe316430e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22180177c0 tx=0x7f221801e000 comp rx=0 tx=0).stop 2023-07-14T16:20:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 -- 172.21.15.62:0/1829660402 shutdown_connections 2023-07-14T16:20:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31642c70 0x55fe316430e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe316414d0 0x55fe316418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1829660402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe316426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 -- 172.21.15.62:0/1829660402 >> 172.21.15.62:0/1829660402 conn(0x55fe31636da0 msgr2=0x55fe3163c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 -- 172.21.15.62:0/1829660402 shutdown_connections 2023-07-14T16:20:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 -- 172.21.15.62:0/1829660402 wait complete. 2023-07-14T16:20:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.897+0000 7f223adfbb80 1 -- start start 2023-07-14T16:20:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f223adfbb80 1 Processor -- start 2023-07-14T16:20:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe3164b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe3164b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f223adfbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31532ac0 con 0x55fe31641e20 2023-07-14T16:20:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f223adfbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe315327c0 con 0x55fe31642c70 2023-07-14T16:20:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f223adfbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31416920 con 0x55fe316414d0 2023-07-14T16:20:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f223adfbb80 10 client.0 _ll_register_callbacks cb 0x55fe31530760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f222499a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe3164b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f222499a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe3164b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34378/0 (socket says 172.21.15.62:34378) 2023-07-14T16:20:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f222499a700 1 -- 172.21.15.62:0/1728936288 learned_addr learned my addr 172.21.15.62:0/1728936288 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.898+0000 7f222599c700 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f222519b700 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe3164b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f222599c700 1 -- 172.21.15.62:0/1728936288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 msgr2=0x55fe3164b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f222599c700 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe3164b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f222599c700 1 -- 172.21.15.62:0/1728936288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 msgr2=0x55fe3164b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f222599c700 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe3164b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f222599c700 1 -- 172.21.15.62:0/1728936288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2218017530 con 0x55fe31642c70 2023-07-14T16:20:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f222499a700 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe3164b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:16.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f222599c700 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316526b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22180244d0 tx=0x7f221801a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d105722ab50226ad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:16.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.899+0000 7f2208ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.900+0000 7f2208ff9700 1 -- 172.21.15.62:0/1728936288 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22180095e0 con 0x55fe31642c70 2023-07-14T16:20:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.900+0000 7f223adfbb80 1 -- 172.21.15.62:0/1728936288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe314ec4f0 con 0x55fe31642c70 2023-07-14T16:20:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.900+0000 7f2208ff9700 1 -- 172.21.15.62:0/1728936288 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2218024800 con 0x55fe31642c70 2023-07-14T16:20:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.900+0000 7f2208ff9700 1 -- 172.21.15.62:0/1728936288 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2218009d60 con 0x55fe31642c70 2023-07-14T16:20:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.900+0000 7f223adfbb80 10 client.26456 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.900+0000 7f223adfbb80 1 -- 172.21.15.62:0/1728936288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fe313a9c60 con 0x55fe31642c70 2023-07-14T16:20:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f2208ff9700 1 -- 172.21.15.62:0/1728936288 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe314ec4f0 con 0x55fe31642c70 2023-07-14T16:20:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f2208ff9700 1 -- 172.21.15.62:0/1728936288 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f22180052e0 con 0x55fe31642c70 2023-07-14T16:20:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f223adfbb80 10 client.26456 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f223adfbb80 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f223adfbb80 20 client.26456 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe31656d80 0x55fe31659190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f223adfbb80 4 client.26456 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f223adfbb80 1 -- 172.21.15.62:0/1728936288 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe3158beb0 con 0x55fe31656d80 2023-07-14T16:20:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.901+0000 7f222519b700 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe31656d80 0x55fe31659190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.902+0000 7f222519b700 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe31656d80 0x55fe31659190 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1722ab6f6dbff61d server_cookie=39ab7ee96d84ea73 in_seq=0 out_seq=0 2023-07-14T16:20:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.902+0000 7f2208ff9700 10 client.26456 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f2208ff9700 1 -- 172.21.15.62:0/1728936288 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fe3158beb0 con 0x55fe31656d80 2023-07-14T16:20:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f2208ff9700 10 client.26456 handle_command_reply: tid=0 2023-07-14T16:20:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 client.26456 shutdown 2023-07-14T16:20:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 -- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 msgr2=0x55fe316526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316526b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22180244d0 tx=0x7f221801a720 comp rx=0 tx=0).stop 2023-07-14T16:20:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 -- 172.21.15.62:0/1728936288 shutdown_connections 2023-07-14T16:20:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe31656d80 0x55fe31659190 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316526b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe3164b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1728936288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe3164b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f2208ff9700 0 client.26456 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 -- 172.21.15.62:0/1728936288 >> 172.21.15.62:0/1728936288 conn(0x55fe31636da0 msgr2=0x55fe3163b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 -- 172.21.15.62:0/1728936288 shutdown_connections 2023-07-14T16:20:16.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 1 -- 172.21.15.62:0/1728936288 wait complete. 2023-07-14T16:20:16.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.903+0000 7f223adfbb80 20 client.26456 trim_cache size 0 max 16384 2023-07-14T16:20:16.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f223adfbb80 1 Processor -- start 2023-07-14T16:20:16.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f223adfbb80 1 -- start start 2023-07-14T16:20:16.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe31649df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe31652670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f223adfbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31636970 con 0x55fe31641e20 2023-07-14T16:20:16.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f223adfbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3163ced0 con 0x55fe31642c70 2023-07-14T16:20:16.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f223adfbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3164bdb0 con 0x55fe316414d0 2023-07-14T16:20:16.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f222499a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe31652670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f222499a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe31652670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34382/0 (socket says 172.21.15.62:34382) 2023-07-14T16:20:16.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f222499a700 1 -- 172.21.15.62:0/3564151785 learned_addr learned my addr 172.21.15.62:0/3564151785 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:16.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.924+0000 7f222519b700 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe31649df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222599c700 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222499a700 1 -- 172.21.15.62:0/3564151785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 msgr2=0x55fe31649df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:16.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222499a700 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe31649df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222499a700 1 -- 172.21.15.62:0/3564151785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 msgr2=0x55fe316490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:16.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222499a700 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222499a700 1 -- 172.21.15.62:0/3564151785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe313e7250 con 0x55fe31641e20 2023-07-14T16:20:16.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222519b700 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe31649df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:16.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222599c700 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.925+0000 7f222499a700 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe31652670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2210003f60 tx=0x7f221000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d40cd27ddc24cf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.926+0000 7f220bfff700 1 -- 172.21.15.62:0/3564151785 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221001d030 con 0x55fe31641e20 2023-07-14T16:20:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.926+0000 7f220bfff700 1 -- 172.21.15.62:0/3564151785 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221001e040 con 0x55fe31641e20 2023-07-14T16:20:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.926+0000 7f220bfff700 1 -- 172.21.15.62:0/3564151785 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221000b910 con 0x55fe31641e20 2023-07-14T16:20:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.926+0000 7f223adfbb80 1 -- 172.21.15.62:0/3564151785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 msgr2=0x55fe31652670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.926+0000 7f223adfbb80 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe31652670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2210003f60 tx=0x7f221000b260 comp rx=0 tx=0).stop 2023-07-14T16:20:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 -- 172.21.15.62:0/3564151785 shutdown_connections 2023-07-14T16:20:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe316414d0 0x55fe31649df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 --2- 172.21.15.62:0/3564151785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe31641e20 0x55fe31652670 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 -- 172.21.15.62:0/3564151785 >> 172.21.15.62:0/3564151785 conn(0x55fe31636da0 msgr2=0x55fe3165dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 -- 172.21.15.62:0/3564151785 shutdown_connections 2023-07-14T16:20:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 -- 172.21.15.62:0/3564151785 wait complete. 2023-07-14T16:20:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 -- start start 2023-07-14T16:20:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.927+0000 7f223adfbb80 1 Processor -- start 2023-07-14T16:20:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe316414d0 0x55fe3164e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31641e20 0x55fe3164e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f223adfbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f223adfbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31651ee0 con 0x55fe316414d0 2023-07-14T16:20:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f223adfbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31652050 con 0x55fe31642c70 2023-07-14T16:20:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f223adfbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31652210 con 0x55fe31641e20 2023-07-14T16:20:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f223adfbb80 10 client.0 _ll_register_callbacks cb 0x7f22180097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f222519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe316414d0 0x55fe3164e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f222519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe316414d0 0x55fe3164e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34392/0 (socket says 172.21.15.62:34392) 2023-07-14T16:20:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.928+0000 7f222519b700 1 -- 172.21.15.62:0/1899297538 learned_addr learned my addr 172.21.15.62:0/1899297538 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f222499a700 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31641e20 0x55fe3164e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f222599c700 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f222519b700 1 -- 172.21.15.62:0/1899297538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31641e20 msgr2=0x55fe3164e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f222519b700 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31641e20 0x55fe3164e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f222519b700 1 -- 172.21.15.62:0/1899297538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 msgr2=0x55fe316519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f222519b700 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f222519b700 1 -- 172.21.15.62:0/1899297538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222019ff60 con 0x55fe316414d0 2023-07-14T16:20:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f222519b700 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe316414d0 0x55fe3164e440 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c5b0 tx=0x7f221c09a020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ed29c30aed2f68 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f2208ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f2208ff9700 1 -- 172.21.15.62:0/1899297538 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0b8330 con 0x55fe316414d0 2023-07-14T16:20:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f2208ff9700 1 -- 172.21.15.62:0/1899297538 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f221c09a1d0 con 0x55fe316414d0 2023-07-14T16:20:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.929+0000 7f223adfbb80 1 -- 172.21.15.62:0/1899297538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f221800f040 con 0x55fe316414d0 2023-07-14T16:20:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f2208ff9700 1 -- 172.21.15.62:0/1899297538 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f221c09a390 con 0x55fe316414d0 2023-07-14T16:20:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f223adfbb80 10 client.16746 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f223adfbb80 1 -- 172.21.15.62:0/1899297538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fe313a9c60 con 0x55fe316414d0 2023-07-14T16:20:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f2208ff9700 1 -- 172.21.15.62:0/1899297538 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221800f040 con 0x55fe316414d0 2023-07-14T16:20:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f2208ff9700 1 -- 172.21.15.62:0/1899297538 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f221c099460 con 0x55fe316414d0 2023-07-14T16:20:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f223adfbb80 10 client.16746 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f223adfbb80 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f223adfbb80 20 client.16746 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe316abb00 0x55fe316adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f223adfbb80 4 client.16746 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:20:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f223adfbb80 1 -- 172.21.15.62:0/1899297538 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe3154e050 con 0x55fe316abb00 2023-07-14T16:20:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.930+0000 7f222499a700 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe316abb00 0x55fe316adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.931+0000 7f222499a700 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe316abb00 0x55fe316adf10 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b6e20ae48a4a64 server_cookie=f0d902ff2b4723e5 in_seq=0 out_seq=0 2023-07-14T16:20:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.931+0000 7f2208ff9700 10 client.16746 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.931+0000 7f2208ff9700 1 -- 172.21.15.62:0/1899297538 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fe3154e050 con 0x55fe316abb00 2023-07-14T16:20:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.931+0000 7f2208ff9700 10 client.16746 handle_command_reply: tid=0 2023-07-14T16:20:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.931+0000 7f223adfbb80 1 client.16746 shutdown 2023-07-14T16:20:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.931+0000 7f223adfbb80 1 -- 172.21.15.62:0/1899297538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe316414d0 msgr2=0x55fe3164e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.931+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe316414d0 0x55fe3164e440 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c5b0 tx=0x7f221c09a020 comp rx=0 tx=0).stop 2023-07-14T16:20:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 1 -- 172.21.15.62:0/1899297538 shutdown_connections 2023-07-14T16:20:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fe316abb00 0x55fe316adf10 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fe31642c70 0x55fe316519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe316414d0 0x55fe3164e440 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 1 --2- 172.21.15.62:0/1899297538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fe31641e20 0x55fe3164e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f2208ff9700 0 client.16746 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 1 -- 172.21.15.62:0/1899297538 >> 172.21.15.62:0/1899297538 conn(0x55fe31636da0 msgr2=0x55fe316490a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 1 -- 172.21.15.62:0/1899297538 shutdown_connections 2023-07-14T16:20:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 1 -- 172.21.15.62:0/1899297538 wait complete. 2023-07-14T16:20:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.932+0000 7f223adfbb80 20 client.16746 trim_cache size 0 max 16384 2023-07-14T16:20:16.917 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:20:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:20:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:20:16.918 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:20:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 -- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f220c0a1150 msgr2=0x7f220c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f220c0a1150 0x7f220c0a3600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2218007810 tx=0x7f2218007bd0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 -- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220101d10 msgr2=0x7f2220197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220101d10 0x7f2220197920 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f221c00e970 tx=0x7f221c00ec80 comp rx=0 tx=0).stop 2023-07-14T16:20:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 -- 172.21.15.62:0/217483700 shutdown_connections 2023-07-14T16:20:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2220103890 0x7f222019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2220101d10 0x7f2220197920 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f220c0a1150 0x7f220c0a3600 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 --2- 172.21.15.62:0/217483700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2220102f00 0x7f2220197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 -- 172.21.15.62:0/217483700 >> 172.21.15.62:0/217483700 conn(0x7f22200fd350 msgr2=0x7f22200ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 -- 172.21.15.62:0/217483700 shutdown_connections 2023-07-14T16:20:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:16.933+0000 7f22273ff700 1 -- 172.21.15.62:0/217483700 wait complete. 2023-07-14T16:20:16.929 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:20:16.930 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:20:17.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:16 smithi062 ceph-mon[95543]: pgmap v496: 97 pgs: 97 active+clean; 2.4 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 MiB/s wr, 11 op/s 2023-07-14T16:20:17.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:16 smithi164 ceph-mon[97427]: pgmap v496: 97 pgs: 97 active+clean; 2.4 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 MiB/s wr, 11 op/s 2023-07-14T16:20:17.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:16 smithi081 ceph-mon[127795]: pgmap v496: 97 pgs: 97 active+clean; 2.4 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 MiB/s wr, 11 op/s 2023-07-14T16:20:17.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 -- 172.21.15.62:0/3498990351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe604074550 msgr2=0x7fe6040729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/3498990351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe604074550 0x7fe6040729a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe600009990 tx=0x7fe600009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 -- 172.21.15.62:0/3498990351 shutdown_connections 2023-07-14T16:20:17.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/3498990351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6041079c0 0x7fe604109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/3498990351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe604074550 0x7fe6040729a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/3498990351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe604072f70 0x7fe6040733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 -- 172.21.15.62:0/3498990351 >> 172.21.15.62:0/3498990351 conn(0x7fe6040fb0e0 msgr2=0x7fe6040fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 -- 172.21.15.62:0/3498990351 shutdown_connections 2023-07-14T16:20:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.195+0000 7fe60c1ff700 1 -- 172.21.15.62:0/3498990351 wait complete. 2023-07-14T16:20:17.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.196+0000 7fe60c1ff700 1 Processor -- start 2023-07-14T16:20:17.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.196+0000 7fe60c1ff700 1 -- start start 2023-07-14T16:20:17.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60c1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe604072f70 0x7fe604197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60c1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604074550 0x7fe604197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60c1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6041079c0 0x7fe60419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60c1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60419f600 con 0x7fe6041079c0 2023-07-14T16:20:17.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60c1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60419f740 con 0x7fe604074550 2023-07-14T16:20:17.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60c1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60419f8d0 con 0x7fe604072f70 2023-07-14T16:20:17.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6041079c0 0x7fe60419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60979a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604074550 0x7fe604197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60979a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604074550 0x7fe604197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50752/0 (socket says 172.21.15.62:50752) 2023-07-14T16:20:17.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6041079c0 0x7fe60419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34414/0 (socket says 172.21.15.62:34414) 2023-07-14T16:20:17.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe60979a700 1 -- 172.21.15.62:0/2701215395 learned_addr learned my addr 172.21.15.62:0/2701215395 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:17.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.197+0000 7fe609f9b700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe604072f70 0x7fe604197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe60a79c700 1 -- 172.21.15.62:0/2701215395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe604072f70 msgr2=0x7fe604197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe60a79c700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe604072f70 0x7fe604197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe60a79c700 1 -- 172.21.15.62:0/2701215395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604074550 msgr2=0x7fe604197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe60a79c700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604074550 0x7fe604197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe60a79c700 1 -- 172.21.15.62:0/2701215395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6000095f0 con 0x7fe6041079c0 2023-07-14T16:20:17.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe60979a700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604074550 0x7fe604197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:17.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe60a79c700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6041079c0 0x7fe60419f030 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc007f10 tx=0x7fe5fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:17.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe5faffd700 1 -- 172.21.15.62:0/2701215395 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc004db0 con 0x7fe6041079c0 2023-07-14T16:20:17.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60419fbb0 con 0x7fe6041079c0 2023-07-14T16:20:17.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe5faffd700 1 -- 172.21.15.62:0/2701215395 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5fc00f040 con 0x7fe6041079c0 2023-07-14T16:20:17.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.198+0000 7fe5faffd700 1 -- 172.21.15.62:0/2701215395 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc005960 con 0x7fe6041079c0 2023-07-14T16:20:17.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.199+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6041a0110 con 0x7fe6041079c0 2023-07-14T16:20:17.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.200+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe604106070 con 0x7fe6041079c0 2023-07-14T16:20:17.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.202+0000 7fe5faffd700 1 -- 172.21.15.62:0/2701215395 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe5fc00db20 con 0x7fe6041079c0 2023-07-14T16:20:17.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.203+0000 7fe5faffd700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5f00a1220 0x7fe5f00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.203+0000 7fe5faffd700 1 -- 172.21.15.62:0/2701215395 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe5fc0cc0b0 con 0x7fe6041079c0 2023-07-14T16:20:17.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.203+0000 7fe609f9b700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5f00a1220 0x7fe5f00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.204+0000 7fe609f9b700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5f00a1220 0x7fe5f00a36d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe60000b5c0 tx=0x7fe600005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:17.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.208+0000 7fe5faffd700 1 -- 172.21.15.62:0/2701215395 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe5fc04e790 con 0x7fe6041079c0 2023-07-14T16:20:17.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.505+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe60402c850 con 0x7fe6041079c0 2023-07-14T16:20:17.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.505+0000 7fe5faffd700 1 -- 172.21.15.62:0/2701215395 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fe5fc095810 con 0x7fe6041079c0 2023-07-14T16:20:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:20:17.483 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:20:17.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5f00a1220 msgr2=0x7fe5f00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5f00a1220 0x7fe5f00a36d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe60000b5c0 tx=0x7fe600005fb0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6041079c0 msgr2=0x7fe60419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6041079c0 0x7fe60419f030 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc007f10 tx=0x7fe5fc00d3b0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 shutdown_connections 2023-07-14T16:20:17.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6041079c0 0x7fe60419f030 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe604072f70 0x7fe604197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5f00a1220 0x7fe5f00a36d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 --2- 172.21.15.62:0/2701215395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe604074550 0x7fe604197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 >> 172.21.15.62:0/2701215395 conn(0x7fe6040fb0e0 msgr2=0x7fe6040fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:17.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 shutdown_connections 2023-07-14T16:20:17.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.508+0000 7fe60c1ff700 1 -- 172.21.15.62:0/2701215395 wait complete. 2023-07-14T16:20:17.487 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:20:17.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:20:17.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.788+0000 7f7bb87ff700 1 -- 172.21.15.62:0/343734558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0074550 msgr2=0x7f7bb00729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.788+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/343734558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0074550 0x7f7bb00729a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009990 tx=0x7f7bac009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.789+0000 7f7bb87ff700 1 -- 172.21.15.62:0/343734558 shutdown_connections 2023-07-14T16:20:17.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.789+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/343734558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb010c0c0 0x7f7bb010e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.789+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/343734558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0074550 0x7f7bb00729a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.789+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/343734558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb0072f70 0x7f7bb00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.789+0000 7f7bb87ff700 1 -- 172.21.15.62:0/343734558 >> 172.21.15.62:0/343734558 conn(0x7f7bb00fb130 msgr2=0x7f7bb00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:17.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.789+0000 7f7bb87ff700 1 -- 172.21.15.62:0/343734558 shutdown_connections 2023-07-14T16:20:17.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.789+0000 7f7bb87ff700 1 -- 172.21.15.62:0/343734558 wait complete. 2023-07-14T16:20:17.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.790+0000 7f7bb87ff700 1 Processor -- start 2023-07-14T16:20:17.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb87ff700 1 -- start start 2023-07-14T16:20:17.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb87ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0072f70 0x7f7bb019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb87ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb0074550 0x7f7bb019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb87ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb010c0c0 0x7f7bb01a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb87ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01a3940 con 0x7f7bb0072f70 2023-07-14T16:20:17.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb87ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01a3a80 con 0x7f7bb010c0c0 2023-07-14T16:20:17.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb87ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01a3c10 con 0x7f7bb0074550 2023-07-14T16:20:17.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0072f70 0x7f7bb019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0072f70 0x7f7bb019bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34430/0 (socket says 172.21.15.62:34430) 2023-07-14T16:20:17.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb6d9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb010c0c0 0x7f7bb01a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb659b700 1 -- 172.21.15.62:0/3536281589 learned_addr learned my addr 172.21.15.62:0/3536281589 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:17.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.791+0000 7f7bb5d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb0074550 0x7f7bb019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3536281589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb0074550 msgr2=0x7f7bb019c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb0074550 0x7f7bb019c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0072f70 msgr2=0x7f7bb019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0072f70 0x7f7bb019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3536281589 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0009520 con 0x7f7bb010c0c0 2023-07-14T16:20:17.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7bb659b700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0072f70 0x7f7bb019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:17.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb010c0c0 0x7f7bb01a3370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800efc0 tx=0x7f7ba800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:17.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7ba77fe700 1 -- 172.21.15.62:0/3536281589 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8018070 con 0x7f7bb010c0c0 2023-07-14T16:20:17.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7bb87ff700 1 -- 172.21.15.62:0/3536281589 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac0095f0 con 0x7f7bb010c0c0 2023-07-14T16:20:17.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7ba77fe700 1 -- 172.21.15.62:0/3536281589 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ba800f040 con 0x7f7bb010c0c0 2023-07-14T16:20:17.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.792+0000 7f7ba77fe700 1 -- 172.21.15.62:0/3536281589 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800cba0 con 0x7f7bb010c0c0 2023-07-14T16:20:17.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.793+0000 7f7bb87ff700 1 -- 172.21.15.62:0/3536281589 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb01a41c0 con 0x7f7bb010c0c0 2023-07-14T16:20:17.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.797+0000 7f7ba77fe700 1 -- 172.21.15.62:0/3536281589 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7ba8003a00 con 0x7f7bb010c0c0 2023-07-14T16:20:17.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.798+0000 7f7bcc188b80 1 Processor -- start 2023-07-14T16:20:17.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.798+0000 7f7bcc188b80 1 -- start start 2023-07-14T16:20:17.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e8821868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882186e20 0x55e8821876e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e8821880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88208bcb0 con 0x55e882186e20 2023-07-14T16:20:17.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8820dea40 con 0x55e882187c70 2023-07-14T16:20:17.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8820d7ed0 con 0x55e8821864d0 2023-07-14T16:20:17.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bb5d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e8821868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bb5d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e8821868e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50186/0 (socket says 172.21.15.62:50186) 2023-07-14T16:20:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bb5d9a700 1 -- 172.21.15.62:0/1589286206 learned_addr learned my addr 172.21.15.62:0/1589286206 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bb5d9a700 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e8821880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.799+0000 7f7bb659b700 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882186e20 0x55e8821876e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7bb5d9a700 1 -- 172.21.15.62:0/1589286206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 msgr2=0x55e8821880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7bb5d9a700 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e8821880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7bb5d9a700 1 -- 172.21.15.62:0/1589286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882186e20 msgr2=0x55e8821876e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7bb5d9a700 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882186e20 0x55e8821876e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7bb5d9a700 1 -- 172.21.15.62:0/1589286206 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e881f05de0 con 0x55e8821864d0 2023-07-14T16:20:17.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7bb659b700 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882186e20 0x55e8821876e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:17.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7ba77fe700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b9c0a1150 0x7f7b9c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7bb5d9a700 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e8821868e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0005950 tx=0x7f7ba0009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a334a9b1201a0cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:17.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7ba77fe700 1 -- 172.21.15.62:0/3536281589 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7ba80cc410 con 0x7f7bb010c0c0 2023-07-14T16:20:17.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7ba4ff9700 1 -- 172.21.15.62:0/1589286206 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0011110 con 0x55e8821864d0 2023-07-14T16:20:17.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7ba4ff9700 1 -- 172.21.15.62:0/1589286206 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ba00044c0 con 0x55e8821864d0 2023-07-14T16:20:17.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7ba4ff9700 1 -- 172.21.15.62:0/1589286206 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0003cb0 con 0x55e8821864d0 2023-07-14T16:20:17.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.800+0000 7f7bb659b700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b9c0a1150 0x7f7b9c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.801+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1589286206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 msgr2=0x55e8821868e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.801+0000 7f7bb659b700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b9c0a1150 0x7f7b9c0a3600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7bac006010 tx=0x7f7bac00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:17.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.801+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e8821868e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0005950 tx=0x7f7ba0009420 comp rx=0 tx=0).stop 2023-07-14T16:20:17.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.801+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1589286206 shutdown_connections 2023-07-14T16:20:17.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.801+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e8821880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.801+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e8821868e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.801+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1589286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882186e20 0x55e8821876e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.801+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1589286206 >> 172.21.15.62:0/1589286206 conn(0x55e88217bda0 msgr2=0x55e8821811d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:17.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.802+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1589286206 shutdown_connections 2023-07-14T16:20:17.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.802+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1589286206 wait complete. 2023-07-14T16:20:17.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.802+0000 7f7bcc188b80 1 -- start start 2023-07-14T16:20:17.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.803+0000 7f7bcc188b80 1 Processor -- start 2023-07-14T16:20:17.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.803+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e8821901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.803+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882186e20 0x55e882190700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.803+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e882197740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.803+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e882077ac0 con 0x55e8821864d0 2023-07-14T16:20:17.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.803+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8820777c0 con 0x55e882187c70 2023-07-14T16:20:17.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.803+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e881f5b920 con 0x55e882186e20 2023-07-14T16:20:17.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bcc188b80 10 client.0 _ll_register_callbacks cb 0x55e882075760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb6d9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e882197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e8821901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb6d9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e882197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50790/0 (socket says 172.21.15.62:50790) 2023-07-14T16:20:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3173843365 learned_addr learned my addr 172.21.15.62:0/3173843365 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb5d9a700 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882186e20 0x55e882190700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3173843365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882186e20 msgr2=0x55e882190700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882186e20 0x55e882190700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3173843365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 msgr2=0x55e8821901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:17.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e8821901c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.804+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3173843365 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba0019040 con 0x55e882187c70 2023-07-14T16:20:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.805+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e882197740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8004fd0 tx=0x7f7ba8099490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e0c56e9e00c344c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.805+0000 7f7b99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.805+0000 7f7b99ffb700 1 -- 172.21.15.62:0/3173843365 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8099860 con 0x55e882187c70 2023-07-14T16:20:17.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.805+0000 7f7b99ffb700 1 -- 172.21.15.62:0/3173843365 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7ba809d890 con 0x55e882187c70 2023-07-14T16:20:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.805+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3173843365 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8820314f0 con 0x55e882187c70 2023-07-14T16:20:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.805+0000 7f7b99ffb700 1 -- 172.21.15.62:0/3173843365 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba80a1c20 con 0x55e882187c70 2023-07-14T16:20:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.805+0000 7f7bcc188b80 10 client.26480 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.805+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3173843365 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e881eeec60 con 0x55e882187c70 2023-07-14T16:20:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7b99ffb700 1 -- 172.21.15.62:0/3173843365 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e8820314f0 con 0x55e882187c70 2023-07-14T16:20:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7b99ffb700 1 -- 172.21.15.62:0/3173843365 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ba8098d50 con 0x55e882187c70 2023-07-14T16:20:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7bcc188b80 10 client.26480 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:17.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7bcc188b80 10 client.26480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:20:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7bcc188b80 20 client.26480 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e88219bf30 0x55e88219e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7bcc188b80 4 client.26480 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3173843365 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8820d0eb0 con 0x55e88219bf30 2023-07-14T16:20:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.806+0000 7f7bb659b700 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e88219bf30 0x55e88219e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.807+0000 7f7bb659b700 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e88219bf30 0x55e88219e340 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e70441a274d4df93 server_cookie=23761188ab1a92a1 in_seq=0 out_seq=0 2023-07-14T16:20:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.807+0000 7f7b99ffb700 10 client.26480 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:20:17.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7b99ffb700 1 -- 172.21.15.62:0/3173843365 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e8820d0eb0 con 0x55e88219bf30 2023-07-14T16:20:17.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7b99ffb700 10 client.26480 handle_command_reply: tid=0 2023-07-14T16:20:17.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 client.26480 shutdown 2023-07-14T16:20:17.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3173843365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 msgr2=0x55e882197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e882197740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8004fd0 tx=0x7f7ba8099490 comp rx=0 tx=0).stop 2023-07-14T16:20:17.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3173843365 shutdown_connections 2023-07-14T16:20:17.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e88219bf30 0x55e88219e340 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882187c70 0x55e882197740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e8821901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7b99ffb700 0 client.26480 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:20:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3173843365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882186e20 0x55e882190700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3173843365 >> 172.21.15.62:0/3173843365 conn(0x55e88217bda0 msgr2=0x55e8821808b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3173843365 shutdown_connections 2023-07-14T16:20:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.808+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3173843365 wait complete. 2023-07-14T16:20:17.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.809+0000 7f7bcc188b80 20 client.26480 trim_cache size 0 max 16384 2023-07-14T16:20:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.826+0000 7f7bcc188b80 1 Processor -- start 2023-07-14T16:20:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.826+0000 7f7bcc188b80 1 -- start start 2023-07-14T16:20:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.826+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e88218e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.826+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 0x55e882197700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.826+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882187c70 0x55e88218ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.826+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88218fd30 con 0x55e8821864d0 2023-07-14T16:20:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.826+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88217b970 con 0x55e882186e20 2023-07-14T16:20:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.826+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e882182580 con 0x55e882187c70 2023-07-14T16:20:17.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e88218e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e88218e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34456/0 (socket says 172.21.15.62:34456) 2023-07-14T16:20:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb5d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 0x55e882197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb659b700 1 -- 172.21.15.62:0/3975722867 learned_addr learned my addr 172.21.15.62:0/3975722867 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3975722867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882187c70 0x55e88218ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3975722867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 msgr2=0x55e882197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3975722867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 0x55e882197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3975722867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 msgr2=0x55e88218e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3975722867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e88218e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.827+0000 7f7bb6d9c700 1 -- 172.21.15.62:0/3975722867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e881f2c250 con 0x55e882187c70 2023-07-14T16:20:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.828+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/3975722867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882187c70 0x55e88218ab30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800ef90 tx=0x7f7ba80a1820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd647cb36adf59f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.828+0000 7f7ba57fa700 1 -- 172.21.15.62:0/3975722867 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba809d890 con 0x55e882187c70 2023-07-14T16:20:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.828+0000 7f7ba57fa700 1 -- 172.21.15.62:0/3975722867 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ba80a1900 con 0x55e882187c70 2023-07-14T16:20:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.828+0000 7f7ba57fa700 1 -- 172.21.15.62:0/3975722867 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8098d50 con 0x55e882187c70 2023-07-14T16:20:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3975722867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882187c70 msgr2=0x55e88218ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3975722867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882187c70 0x55e88218ab30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800ef90 tx=0x7f7ba80a1820 comp rx=0 tx=0).stop 2023-07-14T16:20:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3975722867 shutdown_connections 2023-07-14T16:20:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3975722867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e882187c70 0x55e88218ab30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3975722867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8821864d0 0x55e88218e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/3975722867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 0x55e882197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3975722867 >> 172.21.15.62:0/3975722867 conn(0x55e88217bda0 msgr2=0x55e88218ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3975722867 shutdown_connections 2023-07-14T16:20:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 -- 172.21.15.62:0/3975722867 wait complete. 2023-07-14T16:20:17.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.829+0000 7f7bcc188b80 1 -- start start 2023-07-14T16:20:17.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.830+0000 7f7bcc188b80 1 Processor -- start 2023-07-14T16:20:17.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.830+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e882193620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.830+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 0x55e882193b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.830+0000 7f7bcc188b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882187c70 0x55e882196b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.830+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8821970c0 con 0x55e882187c70 2023-07-14T16:20:17.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.830+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e882197230 con 0x55e882186e20 2023-07-14T16:20:17.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.830+0000 7f7bcc188b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8821973f0 con 0x55e8821864d0 2023-07-14T16:20:17.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.830+0000 7f7bcc188b80 10 client.0 _ll_register_callbacks cb 0x7f7ba80996c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:17.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e882193620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb5d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 0x55e882193b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e882193620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50206/0 (socket says 172.21.15.62:50206) 2023-07-14T16:20:17.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 -- 172.21.15.62:0/1167407336 learned_addr learned my addr 172.21.15.62:0/1167407336 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:17.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb6d9c700 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882187c70 0x55e882196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 -- 172.21.15.62:0/1167407336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 msgr2=0x55e882193b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:17.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 0x55e882193b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 -- 172.21.15.62:0/1167407336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882187c70 msgr2=0x55e882196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:17.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882187c70 0x55e882196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 -- 172.21.15.62:0/1167407336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e881eeec60 con 0x55e8821864d0 2023-07-14T16:20:17.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7bb659b700 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e882193620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7bac000c00 tx=0x7f7bac009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d90ca72237589c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:17.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.831+0000 7f7b99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:20:17.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.832+0000 7f7b99ffb700 1 -- 172.21.15.62:0/1167407336 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac01d050 con 0x55e8821864d0 2023-07-14T16:20:17.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.832+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1167407336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ba00044c0 con 0x55e8821864d0 2023-07-14T16:20:17.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.832+0000 7f7b99ffb700 1 -- 172.21.15.62:0/1167407336 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7bac033d60 con 0x55e8821864d0 2023-07-14T16:20:17.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.832+0000 7f7b99ffb700 1 -- 172.21.15.62:0/1167407336 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bac033600 con 0x55e8821864d0 2023-07-14T16:20:17.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.832+0000 7f7bcc188b80 10 client.26383 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:17.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.832+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1167407336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e881f042b0 con 0x55e8821864d0 2023-07-14T16:20:17.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7b99ffb700 1 -- 172.21.15.62:0/1167407336 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ba00044c0 con 0x55e8821864d0 2023-07-14T16:20:17.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7b99ffb700 1 -- 172.21.15.62:0/1167407336 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7bac004600 con 0x55e8821864d0 2023-07-14T16:20:17.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7bcc188b80 10 client.26383 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:17.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7bcc188b80 10 client.26383 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:20:17.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7bcc188b80 20 client.26383 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:17.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e8821f0ec0 0x55e8821f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:17.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7bcc188b80 4 client.26383 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:20:17.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1167407336 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e882093050 con 0x55e8821f0ec0 2023-07-14T16:20:17.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.833+0000 7f7bb5d9a700 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e8821f0ec0 0x55e8821f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:17.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bb5d9a700 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e8821f0ec0 0x55e8821f32d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=924f455edb6c0c15 server_cookie=d6f32f5e138c3825 in_seq=0 out_seq=0 2023-07-14T16:20:17.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7b99ffb700 10 client.26383 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:20:17.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7b99ffb700 1 -- 172.21.15.62:0/1167407336 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e882093050 con 0x55e8821f0ec0 2023-07-14T16:20:17.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7b99ffb700 10 client.26383 handle_command_reply: tid=0 2023-07-14T16:20:17.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 client.26383 shutdown 2023-07-14T16:20:17.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1167407336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 msgr2=0x55e882193620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e882193620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7bac000c00 tx=0x7f7bac009310 comp rx=0 tx=0).stop 2023-07-14T16:20:17.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1167407336 shutdown_connections 2023-07-14T16:20:17.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e8821f0ec0 0x55e8821f32d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e882187c70 0x55e882196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8821864d0 0x55e882193620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 --2- 172.21.15.62:0/1167407336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e882186e20 0x55e882193b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1167407336 >> 172.21.15.62:0/1167407336 conn(0x55e88217bda0 msgr2=0x55e88218ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:17.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.834+0000 7f7b99ffb700 0 client.26383 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:20:17.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.835+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1167407336 shutdown_connections 2023-07-14T16:20:17.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.835+0000 7f7bcc188b80 1 -- 172.21.15.62:0/1167407336 wait complete. 2023-07-14T16:20:17.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.835+0000 7f7bcc188b80 20 client.26383 trim_cache size 0 max 16384 2023-07-14T16:20:17.820 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:20:17.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.835+0000 7f7bb87ff700 1 -- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b9c0a1150 msgr2=0x7f7b9c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.835+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b9c0a1150 0x7f7b9c0a3600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7bac006010 tx=0x7f7bac00b260 comp rx=0 tx=0).stop 2023-07-14T16:20:17.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.835+0000 7f7bb87ff700 1 -- 172.21.15.62:0/3536281589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb010c0c0 msgr2=0x7f7bb01a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:17.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.835+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb010c0c0 0x7f7bb01a3370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800efc0 tx=0x7f7ba800c3a0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.836+0000 7f7bb87ff700 1 -- 172.21.15.62:0/3536281589 shutdown_connections 2023-07-14T16:20:17.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.836+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b9c0a1150 0x7f7b9c0a3600 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.836+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb010c0c0 0x7f7bb01a3370 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.836+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb0072f70 0x7f7bb019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.836+0000 7f7bb87ff700 1 --2- 172.21.15.62:0/3536281589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb0074550 0x7f7bb019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:17.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.836+0000 7f7bb87ff700 1 -- 172.21.15.62:0/3536281589 >> 172.21.15.62:0/3536281589 conn(0x7f7bb00fb130 msgr2=0x7f7bb00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:17.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.836+0000 7f7bb87ff700 1 -- 172.21.15.62:0/3536281589 shutdown_connections 2023-07-14T16:20:17.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:17.836+0000 7f7bb87ff700 1 -- 172.21.15.62:0/3536281589 wait complete. 2023-07-14T16:20:17.831 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:17 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2701215395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:20:17.831 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:17 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:20:17.834 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:20:18.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:17 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2701215395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:20:18.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:17 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:20:18.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 -- 172.21.15.62:0/1494153632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 msgr2=0x7fdeb80fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 --2- 172.21.15.62:0/1494153632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 0x7fdeb80fece0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009990 tx=0x7fdea8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 -- 172.21.15.62:0/1494153632 shutdown_connections 2023-07-14T16:20:18.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 --2- 172.21.15.62:0/1494153632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb8100450 0x7fdeb8106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 --2- 172.21.15.62:0/1494153632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 0x7fdeb80fece0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 --2- 172.21.15.62:0/1494153632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeb80ffac0 0x7fdeb80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 -- 172.21.15.62:0/1494153632 >> 172.21.15.62:0/1494153632 conn(0x7fdeb80f9ed0 msgr2=0x7fdeb80fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 -- 172.21.15.62:0/1494153632 shutdown_connections 2023-07-14T16:20:18.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.094+0000 7fdeb759e700 1 -- 172.21.15.62:0/1494153632 wait complete. 2023-07-14T16:20:18.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.095+0000 7fdeb759e700 1 Processor -- start 2023-07-14T16:20:18.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.095+0000 7fdeb759e700 1 -- start start 2023-07-14T16:20:18.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 0x7fdeb81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeb80ffac0 0x7fdeb8197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb759e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb8100450 0x7fdeb819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb819f270 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb819f3b0 con 0x7fdeb80ffac0 2023-07-14T16:20:18.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb759e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb819f540 con 0x7fdeb8100450 2023-07-14T16:20:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 0x7fdeb81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeb80ffac0 0x7fdeb8197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 0x7fdeb81976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34490/0 (socket says 172.21.15.62:34490) 2023-07-14T16:20:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeb80ffac0 0x7fdeb8197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50838/0 (socket says 172.21.15.62:50838) 2023-07-14T16:20:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb659c700 1 -- 172.21.15.62:0/2061983504 learned_addr learned my addr 172.21.15.62:0/2061983504 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb6d9d700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb8100450 0x7fdeb819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb659c700 1 -- 172.21.15.62:0/2061983504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb8100450 msgr2=0x7fdeb819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb659c700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb8100450 0x7fdeb819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb659c700 1 -- 172.21.15.62:0/2061983504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeb80ffac0 msgr2=0x7fdeb8197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb659c700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeb80ffac0 0x7fdeb8197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.096+0000 7fdeb659c700 1 -- 172.21.15.62:0/2061983504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80095f0 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.097+0000 7fdeb659c700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 0x7fdeb81976f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdea80098e0 tx=0x7fdea8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.097+0000 7fdea77fe700 1 -- 172.21.15.62:0/2061983504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea801d070 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.097+0000 7fdeb759e700 1 -- 172.21.15.62:0/2061983504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb819f7c0 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.097+0000 7fdea77fe700 1 -- 172.21.15.62:0/2061983504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdea800bd50 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.097+0000 7fdea77fe700 1 -- 172.21.15.62:0/2061983504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8022cf0 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.097+0000 7fdeb759e700 1 -- 172.21.15.62:0/2061983504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb819fc40 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.101+0000 7fdea77fe700 1 -- 172.21.15.62:0/2061983504 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdea8022470 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.102+0000 7fded0431b80 1 Processor -- start 2023-07-14T16:20:18.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.102+0000 7fded0431b80 1 -- start start 2023-07-14T16:20:18.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fded0431b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d13ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fded0431b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d13b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fded0431b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d13c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fded0431b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d03fcb0 con 0x55da7d13b070 2023-07-14T16:20:18.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fded0431b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d092a40 con 0x55da7d13be30 2023-07-14T16:20:18.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fded0431b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d08bed0 con 0x55da7d13a720 2023-07-14T16:20:18.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d13b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fdeb6d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d13ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fdeb6d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d13ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50226/0 (socket says 172.21.15.62:50226) 2023-07-14T16:20:18.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fdeb6d9d700 1 -- 172.21.15.62:0/901036941 learned_addr learned my addr 172.21.15.62:0/901036941 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/901036941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d13c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.103+0000 7fdea77fe700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdea00a1270 0x7fdea00a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdeb5d9b700 1 -- 172.21.15.62:0/901036941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 msgr2=0x55da7d13ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/901036941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d13ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdeb5d9b700 1 -- 172.21.15.62:0/901036941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 msgr2=0x55da7d13b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/901036941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d13b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdeb5d9b700 1 -- 172.21.15.62:0/901036941 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da7ceb9de0 con 0x55da7d13be30 2023-07-14T16:20:18.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdea77fe700 1 -- 172.21.15.62:0/2061983504 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdea80cdf30 con 0x7fdeb80fe8d0 2023-07-14T16:20:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdeb6d9d700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdea00a1270 0x7fdea00a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/901036941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d13c2a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009cb0 tx=0x7fdeac011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb8061c98561eb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdea4ff9700 1 -- 172.21.15.62:0/901036941 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac011d20 con 0x55da7d13be30 2023-07-14T16:20:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdea4ff9700 1 -- 172.21.15.62:0/901036941 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdeac011e80 con 0x55da7d13be30 2023-07-14T16:20:18.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.104+0000 7fdea4ff9700 1 -- 172.21.15.62:0/901036941 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac005280 con 0x55da7d13be30 2023-07-14T16:20:18.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 -- 172.21.15.62:0/901036941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 msgr2=0x55da7d13c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 --2- 172.21.15.62:0/901036941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d13c2a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009cb0 tx=0x7fdeac011440 comp rx=0 tx=0).stop 2023-07-14T16:20:18.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fdeb6d9d700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdea00a1270 0x7fdea00a3720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00075c0 tx=0x7fdeb0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 -- 172.21.15.62:0/901036941 shutdown_connections 2023-07-14T16:20:18.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 --2- 172.21.15.62:0/901036941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d13c2a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 --2- 172.21.15.62:0/901036941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d13ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 --2- 172.21.15.62:0/901036941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d13b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 -- 172.21.15.62:0/901036941 >> 172.21.15.62:0/901036941 conn(0x55da7d12fff0 msgr2=0x55da7d135460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 -- 172.21.15.62:0/901036941 shutdown_connections 2023-07-14T16:20:18.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.105+0000 7fded0431b80 1 -- 172.21.15.62:0/901036941 wait complete. 2023-07-14T16:20:18.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.106+0000 7fded0431b80 1 -- start start 2023-07-14T16:20:18.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.106+0000 7fded0431b80 1 Processor -- start 2023-07-14T16:20:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fded0431b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d144310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fded0431b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d144850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fded0431b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13be30 0x55da7d14b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fded0431b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d02bac0 con 0x55da7d13b070 2023-07-14T16:20:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fded0431b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d02b7c0 con 0x55da7d13a720 2023-07-14T16:20:18.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fded0431b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7cf0f920 con 0x55da7d13be30 2023-07-14T16:20:18.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fded0431b80 10 client.0 _ll_register_callbacks cb 0x55da7d029760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:18.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d144310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50852/0 (socket says 172.21.15.62:50852) 2023-07-14T16:20:18.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fdeb659c700 1 -- 172.21.15.62:0/3441818215 learned_addr learned my addr 172.21.15.62:0/3441818215 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fdeb6d9d700 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13be30 0x55da7d14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.107+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fdeb659c700 1 -- 172.21.15.62:0/3441818215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13be30 msgr2=0x55da7d14b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fdeb659c700 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13be30 0x55da7d14b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fdeb659c700 1 -- 172.21.15.62:0/3441818215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 msgr2=0x55da7d144850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fdeb659c700 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d144850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fdeb659c700 1 -- 172.21.15.62:0/3441818215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac0098e0 con 0x55da7d13a720 2023-07-14T16:20:18.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fdeb659c700 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d144310 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdea80a3210 tx=0x7fdea8004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e059d81bd39242b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fde9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fde9dffb700 1 -- 172.21.15.62:0/3441818215 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea80cbea0 con 0x55da7d13a720 2023-07-14T16:20:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fde9dffb700 1 -- 172.21.15.62:0/3441818215 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdea8086070 con 0x55da7d13a720 2023-07-14T16:20:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fded0431b80 1 -- 172.21.15.62:0/3441818215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da7d084eb0 con 0x55da7d13a720 2023-07-14T16:20:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fde9dffb700 1 -- 172.21.15.62:0/3441818215 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea809ace0 con 0x55da7d13a720 2023-07-14T16:20:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.108+0000 7fded0431b80 10 client.26498 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fded0431b80 1 -- 172.21.15.62:0/3441818215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55da7cea2c60 con 0x55da7d13a720 2023-07-14T16:20:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fde9dffb700 1 -- 172.21.15.62:0/3441818215 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55da7d084eb0 con 0x55da7d13a720 2023-07-14T16:20:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fde9dffb700 1 -- 172.21.15.62:0/3441818215 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdea809a870 con 0x55da7d13a720 2023-07-14T16:20:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fded0431b80 10 client.26498 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fded0431b80 10 client.26498 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:20:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fded0431b80 20 client.26498 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fded0431b80 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55da7d14fd60 0x55da7d152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fded0431b80 4 client.26498 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:18.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.109+0000 7fded0431b80 1 -- 172.21.15.62:0/3441818215 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da7d047050 con 0x55da7d14fd60 2023-07-14T16:20:18.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.110+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55da7d14fd60 0x55da7d152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.110+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55da7d14fd60 0x55da7d152140 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e147be27550ea57d server_cookie=adac5b11e66f488a in_seq=0 out_seq=0 2023-07-14T16:20:18.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.110+0000 7fde9dffb700 10 client.26498 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:20:18.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fde9dffb700 1 -- 172.21.15.62:0/3441818215 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da7d047050 con 0x55da7d14fd60 2023-07-14T16:20:18.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fde9dffb700 10 client.26498 handle_command_reply: tid=0 2023-07-14T16:20:18.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 client.26498 shutdown 2023-07-14T16:20:18.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 -- 172.21.15.62:0/3441818215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 msgr2=0x55da7d144310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d144310 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdea80a3210 tx=0x7fdea8004320 comp rx=0 tx=0).stop 2023-07-14T16:20:18.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 -- 172.21.15.62:0/3441818215 shutdown_connections 2023-07-14T16:20:18.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55da7d14fd60 0x55da7d152140 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13be30 0x55da7d14b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d144310 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fde9dffb700 0 client.26498 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:20:18.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 --2- 172.21.15.62:0/3441818215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 -- 172.21.15.62:0/3441818215 >> 172.21.15.62:0/3441818215 conn(0x55da7d12fff0 msgr2=0x55da7d134b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.111+0000 7fded0431b80 1 -- 172.21.15.62:0/3441818215 shutdown_connections 2023-07-14T16:20:18.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.112+0000 7fded0431b80 1 -- 172.21.15.62:0/3441818215 wait complete. 2023-07-14T16:20:18.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.112+0000 7fded0431b80 20 client.26498 trim_cache size 0 max 16384 2023-07-14T16:20:18.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.127+0000 7fded0431b80 1 Processor -- start 2023-07-14T16:20:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.127+0000 7fded0431b80 1 -- start start 2023-07-14T16:20:18.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.127+0000 7fded0431b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d1421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.127+0000 7fded0431b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13b070 0x55da7d14b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.127+0000 7fded0431b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13be30 0x55da7d142700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.127+0000 7fded0431b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d12fe00 con 0x55da7d13be30 2023-07-14T16:20:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.127+0000 7fded0431b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d1437c0 con 0x55da7d13a720 2023-07-14T16:20:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.127+0000 7fded0431b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d145370 con 0x55da7d13b070 2023-07-14T16:20:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d1421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13be30 0x55da7d142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13b070 0x55da7d14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d1421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50860/0 (socket says 172.21.15.62:50860) 2023-07-14T16:20:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 -- 172.21.15.62:0/2179681264 learned_addr learned my addr 172.21.15.62:0/2179681264 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb6d9d700 1 --2- 172.21.15.62:0/2179681264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13be30 0x55da7d142700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34512/0 (socket says 172.21.15.62:34512) 2023-07-14T16:20:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 -- 172.21.15.62:0/2179681264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13b070 msgr2=0x55da7d14b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 --2- 172.21.15.62:0/2179681264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13b070 0x55da7d14b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 -- 172.21.15.62:0/2179681264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13be30 msgr2=0x55da7d142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 --2- 172.21.15.62:0/2179681264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13be30 0x55da7d142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 -- 172.21.15.62:0/2179681264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da7cee0250 con 0x55da7d13a720 2023-07-14T16:20:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.128+0000 7fdeb659c700 1 --2- 172.21.15.62:0/2179681264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d1421c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005cb0 tx=0x7fdea8005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26229ebc3d3cc0c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fdea57fa700 1 -- 172.21.15.62:0/2179681264 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea80cbea0 con 0x55da7d13a720 2023-07-14T16:20:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fdea57fa700 1 -- 172.21.15.62:0/2179681264 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdea8004e90 con 0x55da7d13a720 2023-07-14T16:20:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fdea57fa700 1 -- 172.21.15.62:0/2179681264 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea809f3b0 con 0x55da7d13a720 2023-07-14T16:20:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fded0431b80 1 -- 172.21.15.62:0/2179681264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 msgr2=0x55da7d1421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fded0431b80 1 --2- 172.21.15.62:0/2179681264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d1421c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005cb0 tx=0x7fdea8005a00 comp rx=0 tx=0).stop 2023-07-14T16:20:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fded0431b80 1 -- 172.21.15.62:0/2179681264 shutdown_connections 2023-07-14T16:20:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fded0431b80 1 --2- 172.21.15.62:0/2179681264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13be30 0x55da7d142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fded0431b80 1 --2- 172.21.15.62:0/2179681264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13a720 0x55da7d1421c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fded0431b80 1 --2- 172.21.15.62:0/2179681264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13b070 0x55da7d14b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.129+0000 7fded0431b80 1 -- 172.21.15.62:0/2179681264 >> 172.21.15.62:0/2179681264 conn(0x55da7d12fff0 msgr2=0x55da7d143010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.130+0000 7fded0431b80 1 -- 172.21.15.62:0/2179681264 shutdown_connections 2023-07-14T16:20:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.130+0000 7fded0431b80 1 -- 172.21.15.62:0/2179681264 wait complete. 2023-07-14T16:20:18.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.130+0000 7fded0431b80 1 -- start start 2023-07-14T16:20:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.130+0000 7fded0431b80 1 Processor -- start 2023-07-14T16:20:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fded0431b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d147740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fded0431b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d147c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fded0431b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fded0431b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d14b1e0 con 0x55da7d13b070 2023-07-14T16:20:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fded0431b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d14b350 con 0x55da7d13be30 2023-07-14T16:20:18.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fded0431b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7d14b510 con 0x55da7d13a720 2023-07-14T16:20:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fded0431b80 10 client.0 _ll_register_callbacks cb 0x7fdea8005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34526/0 (socket says 172.21.15.62:34526) 2023-07-14T16:20:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb5d9b700 1 -- 172.21.15.62:0/2552762717 learned_addr learned my addr 172.21.15.62:0/2552762717 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb6d9d700 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb5d9b700 1 -- 172.21.15.62:0/2552762717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 msgr2=0x55da7d147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb5d9b700 1 -- 172.21.15.62:0/2552762717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 msgr2=0x55da7d14aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d14aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.131+0000 7fdeb5d9b700 1 -- 172.21.15.62:0/2552762717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da7cea2c60 con 0x55da7d13b070 2023-07-14T16:20:18.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.132+0000 7fdeb5d9b700 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d147c80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdeac011790 tx=0x7fdeac012cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf8d99177482e2e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.132+0000 7fde9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:18.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.132+0000 7fde9dffb700 1 -- 172.21.15.62:0/2552762717 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac012ee0 con 0x55da7d13b070 2023-07-14T16:20:18.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.132+0000 7fded0431b80 1 -- 172.21.15.62:0/2552762717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da7cfe54f0 con 0x55da7d13b070 2023-07-14T16:20:18.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.132+0000 7fde9dffb700 1 -- 172.21.15.62:0/2552762717 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdeac028b00 con 0x55da7d13b070 2023-07-14T16:20:18.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.132+0000 7fde9dffb700 1 -- 172.21.15.62:0/2552762717 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeac028cc0 con 0x55da7d13b070 2023-07-14T16:20:18.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.132+0000 7fded0431b80 10 client.16788 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:18.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.132+0000 7fded0431b80 1 -- 172.21.15.62:0/2552762717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55da7ceb82b0 con 0x55da7d13b070 2023-07-14T16:20:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fde9dffb700 1 -- 172.21.15.62:0/2552762717 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55da7cfe54f0 con 0x55da7d13b070 2023-07-14T16:20:18.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fde9dffb700 1 -- 172.21.15.62:0/2552762717 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdeac004510 con 0x55da7d13b070 2023-07-14T16:20:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fded0431b80 10 client.16788 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fded0431b80 10 client.16788 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:20:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fded0431b80 20 client.16788 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fded0431b80 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55da7d1a4c60 0x55da7d1a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fded0431b80 4 client.16788 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:20:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fded0431b80 1 -- 172.21.15.62:0/2552762717 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55da7cedaf70 con 0x55da7d1a4c60 2023-07-14T16:20:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.133+0000 7fdeb659c700 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55da7d1a4c60 0x55da7d1a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fde9dffb700 10 client.16788 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:20:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fdeb659c700 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55da7d1a4c60 0x55da7d1a7070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=beb3a4050ce2606 server_cookie=e44078e078818ea9 in_seq=0 out_seq=0 2023-07-14T16:20:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fde9dffb700 1 -- 172.21.15.62:0/2552762717 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55da7cedaf70 con 0x55da7d1a4c60 2023-07-14T16:20:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fde9dffb700 10 client.16788 handle_command_reply: tid=0 2023-07-14T16:20:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 client.16788 shutdown 2023-07-14T16:20:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 -- 172.21.15.62:0/2552762717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 msgr2=0x55da7d147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d147c80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdeac011790 tx=0x7fdeac012cc0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 -- 172.21.15.62:0/2552762717 shutdown_connections 2023-07-14T16:20:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55da7d1a4c60 0x55da7d1a7070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55da7d13be30 0x55da7d14aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55da7d13a720 0x55da7d147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 --2- 172.21.15.62:0/2552762717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da7d13b070 0x55da7d147c80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fde9dffb700 0 client.16788 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:20:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 -- 172.21.15.62:0/2552762717 >> 172.21.15.62:0/2552762717 conn(0x55da7d12fff0 msgr2=0x55da7d1421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.134+0000 7fded0431b80 1 -- 172.21.15.62:0/2552762717 shutdown_connections 2023-07-14T16:20:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fded0431b80 1 -- 172.21.15.62:0/2552762717 wait complete. 2023-07-14T16:20:18.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fded0431b80 20 client.16788 trim_cache size 0 max 16384 2023-07-14T16:20:18.120 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:20:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 -- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdea00a1270 msgr2=0x7fdea00a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdea00a1270 0x7fdea00a3720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00075c0 tx=0x7fdeb0009040 comp rx=0 tx=0).stop 2023-07-14T16:20:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 -- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 msgr2=0x7fdeb81976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 0x7fdeb81976f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdea80098e0 tx=0x7fdea8004730 comp rx=0 tx=0).stop 2023-07-14T16:20:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 -- 172.21.15.62:0/2061983504 shutdown_connections 2023-07-14T16:20:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdeb8100450 0x7fdeb819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdeb80fe8d0 0x7fdeb81976f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdea00a1270 0x7fdea00a3720 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 --2- 172.21.15.62:0/2061983504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeb80ffac0 0x7fdeb8197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.135+0000 7fdeb759e700 1 -- 172.21.15.62:0/2061983504 >> 172.21.15.62:0/2061983504 conn(0x7fdeb80f9ed0 msgr2=0x7fdeb81099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.136+0000 7fdeb759e700 1 -- 172.21.15.62:0/2061983504 shutdown_connections 2023-07-14T16:20:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.136+0000 7fdeb759e700 1 -- 172.21.15.62:0/2061983504 wait complete. 2023-07-14T16:20:18.136 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:20:18.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:17 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2701215395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:20:18.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:17 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:20:18.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3800413420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c81057a0 msgr2=0x7fd5c8107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3800413420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c81057a0 0x7fd5c8107c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009f90 tx=0x7fd5c000b330 comp rx=0 tx=0).stop 2023-07-14T16:20:18.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3800413420 shutdown_connections 2023-07-14T16:20:18.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3800413420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c81057a0 0x7fd5c8107c00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3800413420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5c8100560 0x7fd5c8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3800413420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5c8102e80 0x7fd5c8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3800413420 >> 172.21.15.62:0/3800413420 conn(0x7fd5c80fa000 msgr2=0x7fd5c80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3800413420 shutdown_connections 2023-07-14T16:20:18.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.408+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3800413420 wait complete. 2023-07-14T16:20:18.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.410+0000 7fd5ccfff700 1 Processor -- start 2023-07-14T16:20:18.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.410+0000 7fd5ccfff700 1 -- start start 2023-07-14T16:20:18.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.410+0000 7fd5ccfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c8100560 0x7fd5c80ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.410+0000 7fd5ccfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5c8102e80 0x7fd5c80fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.410+0000 7fd5ccfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5c81057a0 0x7fd5c80fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.410+0000 7fd5ccfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80ff1f0 con 0x7fd5c81057a0 2023-07-14T16:20:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.410+0000 7fd5ccfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80ff360 con 0x7fd5c8102e80 2023-07-14T16:20:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.410+0000 7fd5ccfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c819f8d0 con 0x7fd5c8100560 2023-07-14T16:20:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c8100560 0x7fd5c80ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c8100560 0x7fd5c80ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50256/0 (socket says 172.21.15.62:50256) 2023-07-14T16:20:18.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c659c700 1 -- 172.21.15.62:0/3015476952 learned_addr learned my addr 172.21.15.62:0/3015476952 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5c81057a0 0x7fd5c80fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5c8102e80 0x7fd5c80fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c6d9d700 1 -- 172.21.15.62:0/3015476952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c8100560 msgr2=0x7fd5c80ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c8100560 0x7fd5c80ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c6d9d700 1 -- 172.21.15.62:0/3015476952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5c8102e80 msgr2=0x7fd5c80fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5c8102e80 0x7fd5c80fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.411+0000 7fd5c6d9d700 1 -- 172.21.15.62:0/3015476952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0009c40 con 0x7fd5c81057a0 2023-07-14T16:20:18.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.412+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5c81057a0 0x7fd5c80fea50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0007890 tx=0x7fd5c0008c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.412+0000 7fd5bf7fe700 1 -- 172.21.15.62:0/3015476952 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000e040 con 0x7fd5c81057a0 2023-07-14T16:20:18.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.412+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3015476952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c819fb50 con 0x7fd5c81057a0 2023-07-14T16:20:18.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.412+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3015476952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c819ffd0 con 0x7fd5c81057a0 2023-07-14T16:20:18.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.413+0000 7fd5bf7fe700 1 -- 172.21.15.62:0/3015476952 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5c000bc00 con 0x7fd5c81057a0 2023-07-14T16:20:18.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.413+0000 7fd5bf7fe700 1 -- 172.21.15.62:0/3015476952 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0005040 con 0x7fd5c81057a0 2023-07-14T16:20:18.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.417+0000 7fd5bf7fe700 1 -- 172.21.15.62:0/3015476952 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd5c001d470 con 0x7fd5c81057a0 2023-07-14T16:20:18.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.419+0000 7fd5e0901b80 1 Processor -- start 2023-07-14T16:20:18.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.419+0000 7fd5e0901b80 1 -- start start 2023-07-14T16:20:18.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.419+0000 7fd5bf7fe700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5b40a11d0 0x7fd5b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5bf7fe700 1 -- 172.21.15.62:0/3015476952 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5c00cdca0 con 0x7fd5c81057a0 2023-07-14T16:20:18.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ec6bcb0 con 0x55f81ed664d0 2023-07-14T16:20:18.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ecbea40 con 0x55f81ed67c70 2023-07-14T16:20:18.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ecb7ed0 con 0x55f81ed66e20 2023-07-14T16:20:18.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5c5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5c5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed676e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50266/0 (socket says 172.21.15.62:50266) 2023-07-14T16:20:18.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/2552621228 learned_addr learned my addr 172.21.15.62:0/2552621228 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5c659c700 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.420+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5b40a11d0 0x7fd5b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.421+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.421+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/2552621228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 msgr2=0x55f81ed680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.421+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed680e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.421+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/2552621228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 msgr2=0x55f81ed668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.421+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.421+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/2552621228 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f81eae5de0 con 0x55f81ed66e20 2023-07-14T16:20:18.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.421+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:18.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.421+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed676e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8011670 tx=0x7fd5b8011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22c4a67d2e0abe18 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.422+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5b40a11d0 0x7fd5b40a3680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd5b801c000 tx=0x7fd5b801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.422+0000 7fd5bcff9700 1 -- 172.21.15.62:0/2552621228 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800ed20 con 0x55f81ed66e20 2023-07-14T16:20:18.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.422+0000 7fd5bcff9700 1 -- 172.21.15.62:0/2552621228 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5b800ee80 con 0x55f81ed66e20 2023-07-14T16:20:18.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.422+0000 7fd5bcff9700 1 -- 172.21.15.62:0/2552621228 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b801d640 con 0x55f81ed66e20 2023-07-14T16:20:18.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.422+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2552621228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 msgr2=0x55f81ed676e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.422+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed676e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8011670 tx=0x7fd5b8011980 comp rx=0 tx=0).stop 2023-07-14T16:20:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.423+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2552621228 shutdown_connections 2023-07-14T16:20:18.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.423+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.423+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.423+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2552621228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed676e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.423+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2552621228 >> 172.21.15.62:0/2552621228 conn(0x55f81ed5bda0 msgr2=0x55f81ed611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.423+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2552621228 shutdown_connections 2023-07-14T16:20:18.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.423+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2552621228 wait complete. 2023-07-14T16:20:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.423+0000 7fd5e0901b80 1 -- start start 2023-07-14T16:20:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.424+0000 7fd5e0901b80 1 Processor -- start 2023-07-14T16:20:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed664d0 0x55f81ed701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed66e20 0x55f81ed706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed67c70 0x55f81ed77720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ec57ac0 con 0x55f81ed67c70 2023-07-14T16:20:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ec577c0 con 0x55f81ed66e20 2023-07-14T16:20:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81eb3b920 con 0x55f81ed664d0 2023-07-14T16:20:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5e0901b80 10 client.0 _ll_register_callbacks cb 0x55f81ec55760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5c659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed664d0 0x55f81ed701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5c659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed664d0 0x55f81ed701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50268/0 (socket says 172.21.15.62:50268) 2023-07-14T16:20:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5c659c700 1 -- 172.21.15.62:0/2486797682 learned_addr learned my addr 172.21.15.62:0/2486797682 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.425+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed66e20 0x55f81ed706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed67c70 0x55f81ed77720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/2486797682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed664d0 msgr2=0x55f81ed701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed664d0 0x55f81ed701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/2486797682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed67c70 msgr2=0x55f81ed77720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed67c70 0x55f81ed77720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/2486797682 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b8019040 con 0x55f81ed66e20 2023-07-14T16:20:18.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed66e20 0x55f81ed706e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8011cd0 tx=0x7fd5b8009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95fa3f7be8d5333a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5adffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:18.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.426+0000 7fd5c659c700 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed664d0 0x55f81ed701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:18.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.427+0000 7fd5adffb700 1 -- 172.21.15.62:0/2486797682 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b801e690 con 0x55f81ed66e20 2023-07-14T16:20:18.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.427+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2486797682 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f81ecb0eb0 con 0x55f81ed66e20 2023-07-14T16:20:18.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.427+0000 7fd5adffb700 1 -- 172.21.15.62:0/2486797682 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5b801e7f0 con 0x55f81ed66e20 2023-07-14T16:20:18.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.427+0000 7fd5adffb700 1 -- 172.21.15.62:0/2486797682 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b8005040 con 0x55f81ed66e20 2023-07-14T16:20:18.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.427+0000 7fd5e0901b80 10 client.26516 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:18.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.427+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2486797682 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f81eacec60 con 0x55f81ed66e20 2023-07-14T16:20:18.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.427+0000 7fd5adffb700 1 -- 172.21.15.62:0/2486797682 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f81ecb0eb0 con 0x55f81ed66e20 2023-07-14T16:20:18.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.427+0000 7fd5adffb700 1 -- 172.21.15.62:0/2486797682 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5b8004b60 con 0x55f81ed66e20 2023-07-14T16:20:18.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.428+0000 7fd5e0901b80 10 client.26516 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:18.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.428+0000 7fd5e0901b80 10 client.26516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:18.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.428+0000 7fd5e0901b80 20 client.26516 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:18.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.428+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f81ed7bf00 0x55f81ed7e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.428+0000 7fd5e0901b80 4 client.26516 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:18.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.428+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2486797682 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f81ec73050 con 0x55f81ed7bf00 2023-07-14T16:20:18.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.428+0000 7fd5c659c700 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f81ed7bf00 0x55f81ed7e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.429+0000 7fd5adffb700 10 client.26516 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:18.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.429+0000 7fd5c659c700 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f81ed7bf00 0x55f81ed7e310 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd89c415892ff4a server_cookie=442e7290fa0e9d73 in_seq=0 out_seq=0 2023-07-14T16:20:18.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.429+0000 7fd5adffb700 1 -- 172.21.15.62:0/2486797682 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f81ec73050 con 0x55f81ed7bf00 2023-07-14T16:20:18.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.429+0000 7fd5adffb700 10 client.26516 handle_command_reply: tid=0 2023-07-14T16:20:18.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.429+0000 7fd5e0901b80 1 client.26516 shutdown 2023-07-14T16:20:18.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed66e20 msgr2=0x55f81ed706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed66e20 0x55f81ed706e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8011cd0 tx=0x7fd5b8009b10 comp rx=0 tx=0).stop 2023-07-14T16:20:18.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2486797682 shutdown_connections 2023-07-14T16:20:18.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed67c70 0x55f81ed77720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed664d0 0x55f81ed701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f81ed7bf00 0x55f81ed7e310 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/2486797682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed66e20 0x55f81ed706e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2486797682 >> 172.21.15.62:0/2486797682 conn(0x55f81ed5bda0 msgr2=0x55f81ed608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5adffb700 0 client.26516 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:18.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2486797682 shutdown_connections 2023-07-14T16:20:18.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 1 -- 172.21.15.62:0/2486797682 wait complete. 2023-07-14T16:20:18.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.430+0000 7fd5e0901b80 20 client.26516 trim_cache size 0 max 16384 2023-07-14T16:20:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.449+0000 7fd5e0901b80 1 Processor -- start 2023-07-14T16:20:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.449+0000 7fd5e0901b80 1 -- start start 2023-07-14T16:20:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.449+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed6e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.449+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.449+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed6ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.449+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ed71390 con 0x55f81ed664d0 2023-07-14T16:20:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.449+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ed6fd10 con 0x55f81ed67c70 2023-07-14T16:20:18.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.449+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ed68400 con 0x55f81ed66e20 2023-07-14T16:20:18.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed6e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed6ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed6e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34566/0 (socket says 172.21.15.62:34566) 2023-07-14T16:20:18.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 -- 172.21.15.62:0/390168764 learned_addr learned my addr 172.21.15.62:0/390168764 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 -- 172.21.15.62:0/390168764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 msgr2=0x55f81ed776e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed776e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 -- 172.21.15.62:0/390168764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 msgr2=0x55f81ed6ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed6ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 -- 172.21.15.62:0/390168764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f81eb0c250 con 0x55f81ed664d0 2023-07-14T16:20:18.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed6ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:18.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.450+0000 7fd5c659c700 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed6e020 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0005d90 tx=0x7fd5b0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ade678d3348cb9f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.451+0000 7fd5bd7fa700 1 -- 172.21.15.62:0/390168764 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0004220 con 0x55f81ed664d0 2023-07-14T16:20:18.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.451+0000 7fd5bd7fa700 1 -- 172.21.15.62:0/390168764 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5b0018040 con 0x55f81ed664d0 2023-07-14T16:20:18.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.451+0000 7fd5bd7fa700 1 -- 172.21.15.62:0/390168764 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b001f4c0 con 0x55f81ed664d0 2023-07-14T16:20:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 -- 172.21.15.62:0/390168764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 msgr2=0x55f81ed6e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed6e020 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0005d90 tx=0x7fd5b0009260 comp rx=0 tx=0).stop 2023-07-14T16:20:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 -- 172.21.15.62:0/390168764 shutdown_connections 2023-07-14T16:20:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed67c70 0x55f81ed6ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed664d0 0x55f81ed6e020 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/390168764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed66e20 0x55f81ed776e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 -- 172.21.15.62:0/390168764 >> 172.21.15.62:0/390168764 conn(0x55f81ed5bda0 msgr2=0x55f81ed78010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 -- 172.21.15.62:0/390168764 shutdown_connections 2023-07-14T16:20:18.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.452+0000 7fd5e0901b80 1 -- 172.21.15.62:0/390168764 wait complete. 2023-07-14T16:20:18.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.453+0000 7fd5e0901b80 1 -- start start 2023-07-14T16:20:18.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.453+0000 7fd5e0901b80 1 Processor -- start 2023-07-14T16:20:18.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.453+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed664d0 0x55f81ed77290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.453+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed66e20 0x55f81ed73d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5e0901b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed67c70 0x55f81ed742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ed77ab0 con 0x55f81ed66e20 2023-07-14T16:20:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ed747e0 con 0x55f81ed664d0 2023-07-14T16:20:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5e0901b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81ed749a0 con 0x55f81ed67c70 2023-07-14T16:20:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5e0901b80 10 client.0 _ll_register_callbacks cb 0x55f81ec55760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed66e20 0x55f81ed73d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed66e20 0x55f81ed73d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34580/0 (socket says 172.21.15.62:34580) 2023-07-14T16:20:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/1787157328 learned_addr learned my addr 172.21.15.62:0/1787157328 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed67c70 0x55f81ed742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c659c700 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed664d0 0x55f81ed77290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/1787157328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed67c70 msgr2=0x55f81ed742a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed67c70 0x55f81ed742a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/1787157328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed664d0 msgr2=0x55f81ed77290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.454+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed664d0 0x55f81ed77290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.455+0000 7fd5c5d9b700 1 -- 172.21.15.62:0/1787157328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c819ffd0 con 0x55f81ed66e20 2023-07-14T16:20:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.455+0000 7fd5c6d9d700 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed67c70 0x55f81ed742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.455+0000 7fd5c5d9b700 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed66e20 0x55f81ed73d60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009720 tx=0x7fd5b80096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30e99f83b057031 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.455+0000 7fd5adffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.455+0000 7fd5adffb700 1 -- 172.21.15.62:0/1787157328 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80275b0 con 0x55f81ed66e20 2023-07-14T16:20:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.455+0000 7fd5adffb700 1 -- 172.21.15.62:0/1787157328 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5b80048a0 con 0x55f81ed66e20 2023-07-14T16:20:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.455+0000 7fd5e0901b80 1 -- 172.21.15.62:0/1787157328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f81ec114f0 con 0x55f81ed66e20 2023-07-14T16:20:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5adffb700 1 -- 172.21.15.62:0/1787157328 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b8009ea0 con 0x55f81ed66e20 2023-07-14T16:20:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5e0901b80 10 client.16806 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5e0901b80 1 -- 172.21.15.62:0/1787157328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fd5b0009d50 con 0x55f81ed66e20 2023-07-14T16:20:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5adffb700 1 -- 172.21.15.62:0/1787157328 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f81ec114f0 con 0x55f81ed66e20 2023-07-14T16:20:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5adffb700 1 -- 172.21.15.62:0/1787157328 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5b8004f90 con 0x55f81ed66e20 2023-07-14T16:20:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5e0901b80 10 client.16806 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5e0901b80 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5e0901b80 20 client.16806 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f81edd1320 0x55f81edd3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5e0901b80 4 client.16806 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:20:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.456+0000 7fd5e0901b80 1 -- 172.21.15.62:0/1787157328 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f81eb06f70 con 0x55f81edd1320 2023-07-14T16:20:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.457+0000 7fd5c659c700 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f81edd1320 0x55f81edd3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.457+0000 7fd5c659c700 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f81edd1320 0x55f81edd3730 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b47ad37f092a69 server_cookie=31f36c0a80a75112 in_seq=0 out_seq=0 2023-07-14T16:20:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.457+0000 7fd5adffb700 10 client.16806 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5adffb700 1 -- 172.21.15.62:0/1787157328 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f81eb06f70 con 0x55f81edd1320 2023-07-14T16:20:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5adffb700 10 client.16806 handle_command_reply: tid=0 2023-07-14T16:20:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 client.16806 shutdown 2023-07-14T16:20:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 -- 172.21.15.62:0/1787157328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed66e20 msgr2=0x55f81ed73d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed66e20 0x55f81ed73d60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009720 tx=0x7fd5b80096b0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 -- 172.21.15.62:0/1787157328 shutdown_connections 2023-07-14T16:20:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f81ed67c70 0x55f81ed742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f81edd1320 0x55f81edd3730 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f81ed664d0 0x55f81ed77290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 --2- 172.21.15.62:0/1787157328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f81ed66e20 0x55f81ed73d60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 -- 172.21.15.62:0/1787157328 >> 172.21.15.62:0/1787157328 conn(0x55f81ed5bda0 msgr2=0x55f81ed6e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5adffb700 0 client.16806 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:18.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 -- 172.21.15.62:0/1787157328 shutdown_connections 2023-07-14T16:20:18.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 1 -- 172.21.15.62:0/1787157328 wait complete. 2023-07-14T16:20:18.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.458+0000 7fd5e0901b80 20 client.16806 trim_cache size 0 max 16384 2023-07-14T16:20:18.445 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:20:18.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5b40a11d0 msgr2=0x7fd5b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5b40a11d0 0x7fd5b40a3680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd5b801c000 tx=0x7fd5b801e000 comp rx=0 tx=0).stop 2023-07-14T16:20:18.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5c81057a0 msgr2=0x7fd5c80fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:18.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5c81057a0 0x7fd5c80fea50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0007890 tx=0x7fd5c0008c80 comp rx=0 tx=0).stop 2023-07-14T16:20:18.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3015476952 shutdown_connections 2023-07-14T16:20:18.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5c81057a0 0x7fd5c80fea50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5c8100560 0x7fd5c80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd5b40a11d0 0x7fd5b40a3680 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 --2- 172.21.15.62:0/3015476952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5c8102e80 0x7fd5c80fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:18.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.459+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3015476952 >> 172.21.15.62:0/3015476952 conn(0x7fd5c80fa000 msgr2=0x7fd5c8101240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:18.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.460+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3015476952 shutdown_connections 2023-07-14T16:20:18.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:18.460+0000 7fd5ccfff700 1 -- 172.21.15.62:0/3015476952 wait complete. 2023-07-14T16:20:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:18 smithi164 ceph-mon[97427]: pgmap v497: 97 pgs: 97 active+clean; 2.4 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 MiB/s wr, 11 op/s 2023-07-14T16:20:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:20:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:20:19.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:18 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:20:19.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:18 smithi062 ceph-mon[95543]: pgmap v497: 97 pgs: 97 active+clean; 2.4 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 MiB/s wr, 11 op/s 2023-07-14T16:20:19.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:20:19.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:20:19.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:18 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:20:19.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:18 smithi081 ceph-mon[127795]: pgmap v497: 97 pgs: 97 active+clean; 2.4 GiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 14 MiB/s wr, 11 op/s 2023-07-14T16:20:19.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:20:19.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:20:19.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:18 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:20:19.458 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:20:19.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:20:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 -- 172.21.15.62:0/1261819887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 msgr2=0x7fc43c109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 --2- 172.21.15.62:0/1261819887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 0x7fc43c109e20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc43400b0a0 tx=0x7fc43400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 -- 172.21.15.62:0/1261819887 shutdown_connections 2023-07-14T16:20:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 --2- 172.21.15.62:0/1261819887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 0x7fc43c109e20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 --2- 172.21.15.62:0/1261819887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc43c074550 0x7fc43c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 --2- 172.21.15.62:0/1261819887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43c072f70 0x7fc43c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 -- 172.21.15.62:0/1261819887 >> 172.21.15.62:0/1261819887 conn(0x7fc43c0fb0e0 msgr2=0x7fc43c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:19.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 -- 172.21.15.62:0/1261819887 shutdown_connections 2023-07-14T16:20:19.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.716+0000 7fc443bff700 1 -- 172.21.15.62:0/1261819887 wait complete. 2023-07-14T16:20:19.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.717+0000 7fc443bff700 1 Processor -- start 2023-07-14T16:20:19.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.717+0000 7fc443bff700 1 -- start start 2023-07-14T16:20:19.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc443bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc43c072f70 0x7fc43c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc443bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43c074550 0x7fc43c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc443bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 0x7fc43c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc443bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c19f5e0 con 0x7fc43c1079a0 2023-07-14T16:20:19.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc443bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c19f720 con 0x7fc43c072f70 2023-07-14T16:20:19.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc443bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c19f8b0 con 0x7fc43c074550 2023-07-14T16:20:19.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc44219c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 0x7fc43c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc44219c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 0x7fc43c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34600/0 (socket says 172.21.15.62:34600) 2023-07-14T16:20:19.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc44219c700 1 -- 172.21.15.62:0/1101938505 learned_addr learned my addr 172.21.15.62:0/1101938505 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:19.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc44119a700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43c074550 0x7fc43c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.718+0000 7fc44199b700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc43c072f70 0x7fc43c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc44219c700 1 -- 172.21.15.62:0/1101938505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43c074550 msgr2=0x7fc43c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc44219c700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43c074550 0x7fc43c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc44219c700 1 -- 172.21.15.62:0/1101938505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc43c072f70 msgr2=0x7fc43c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc44219c700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc43c072f70 0x7fc43c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc44219c700 1 -- 172.21.15.62:0/1101938505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc434009d00 con 0x7fc43c1079a0 2023-07-14T16:20:19.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc44199b700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc43c072f70 0x7fc43c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:19.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc44219c700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 0x7fc43c19f010 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc43400d010 tx=0x7fc43400bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:19.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc432ffd700 1 -- 172.21.15.62:0/1101938505 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43400e040 con 0x7fc43c1079a0 2023-07-14T16:20:19.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc432ffd700 1 -- 172.21.15.62:0/1101938505 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc434009480 con 0x7fc43c1079a0 2023-07-14T16:20:19.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc443bff700 1 -- 172.21.15.62:0/1101938505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c19fb30 con 0x7fc43c1079a0 2023-07-14T16:20:19.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc432ffd700 1 -- 172.21.15.62:0/1101938505 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4340052a0 con 0x7fc43c1079a0 2023-07-14T16:20:19.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.719+0000 7fc443bff700 1 -- 172.21.15.62:0/1101938505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c19ffb0 con 0x7fc43c1079a0 2023-07-14T16:20:19.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.723+0000 7fc432ffd700 1 -- 172.21.15.62:0/1101938505 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc434005400 con 0x7fc43c1079a0 2023-07-14T16:20:19.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc4575abb80 1 Processor -- start 2023-07-14T16:20:19.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc432ffd700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc4280a11d0 0x7fc4280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc4575abb80 1 -- start start 2023-07-14T16:20:19.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc432ffd700 1 -- 172.21.15.62:0/1101938505 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc4340cda30 con 0x7fc43c1079a0 2023-07-14T16:20:19.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469d5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc44119a700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc4280a11d0 0x7fc4280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469d7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc4575abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f468dad30 con 0x561f469d5550 2023-07-14T16:20:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc4575abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4692dac0 con 0x561f469d5ea0 2023-07-14T16:20:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.725+0000 7fc4575abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f46926f50 con 0x561f469d6cf0 2023-07-14T16:20:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469d5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34612/0 (socket says 172.21.15.62:34612) 2023-07-14T16:20:19.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44199b700 1 -- 172.21.15.62:0/254586541 learned_addr learned my addr 172.21.15.62:0/254586541 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44219c700 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469d7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44119a700 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44199b700 1 -- 172.21.15.62:0/254586541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 msgr2=0x561f469d7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44199b700 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469d7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44199b700 1 -- 172.21.15.62:0/254586541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 msgr2=0x561f469d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44199b700 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44199b700 1 -- 172.21.15.62:0/254586541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f46754e60 con 0x561f469d5550 2023-07-14T16:20:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44219c700 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469d7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.726+0000 7fc44119a700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc4280a11d0 0x7fc4280a3680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc43c074270 tx=0x7fc42c011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.727+0000 7fc44199b700 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469d5960 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc438009d80 tx=0x7fc4380116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d0059538fe88a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.727+0000 7fc427fff700 1 -- 172.21.15.62:0/254586541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438011ed0 con 0x561f469d5550 2023-07-14T16:20:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.727+0000 7fc427fff700 1 -- 172.21.15.62:0/254586541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4380128e0 con 0x561f469d5550 2023-07-14T16:20:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.727+0000 7fc427fff700 1 -- 172.21.15.62:0/254586541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43800f7b0 con 0x561f469d5550 2023-07-14T16:20:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 -- 172.21.15.62:0/254586541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 msgr2=0x561f469d5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469d5960 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc438009d80 tx=0x7fc4380116c0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 -- 172.21.15.62:0/254586541 shutdown_connections 2023-07-14T16:20:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469d7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469d5960 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 --2- 172.21.15.62:0/254586541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 -- 172.21.15.62:0/254586541 >> 172.21.15.62:0/254586541 conn(0x561f469cae20 msgr2=0x561f469d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 -- 172.21.15.62:0/254586541 shutdown_connections 2023-07-14T16:20:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 -- 172.21.15.62:0/254586541 wait complete. 2023-07-14T16:20:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.728+0000 7fc4575abb80 1 -- start start 2023-07-14T16:20:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.729+0000 7fc4575abb80 1 Processor -- start 2023-07-14T16:20:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.729+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 0x561f469df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.729+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469df6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.729+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f469e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.729+0000 7fc4575abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f468c6b40 con 0x561f469d5ea0 2023-07-14T16:20:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.729+0000 7fc4575abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f468c6840 con 0x561f469d6cf0 2023-07-14T16:20:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.729+0000 7fc4575abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f467aa9a0 con 0x561f469d5550 2023-07-14T16:20:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.729+0000 7fc4575abb80 10 client.0 _ll_register_callbacks cb 0x561f468c47e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469df6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469df6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34622/0 (socket says 172.21.15.62:34622) 2023-07-14T16:20:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44119a700 1 -- 172.21.15.62:0/3150473983 learned_addr learned my addr 172.21.15.62:0/3150473983 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44199b700 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 0x561f469df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44219c700 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f469e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44119a700 1 -- 172.21.15.62:0/3150473983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 msgr2=0x561f469df1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44119a700 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 0x561f469df1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44119a700 1 -- 172.21.15.62:0/3150473983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 msgr2=0x561f469e6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44119a700 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f469e6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.730+0000 7fc44119a700 1 -- 172.21.15.62:0/3150473983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4380099b0 con 0x561f469d5ea0 2023-07-14T16:20:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.731+0000 7fc44119a700 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469df6e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc42c01e970 tx=0x7fc42c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da2a8db2d6f3d21 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.731+0000 7fc4257fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.731+0000 7fc4257fa700 1 -- 172.21.15.62:0/3150473983 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c011e90 con 0x561f469d5ea0 2023-07-14T16:20:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.731+0000 7fc4575abb80 1 -- 172.21.15.62:0/3150473983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f46880570 con 0x561f469d5ea0 2023-07-14T16:20:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.731+0000 7fc4257fa700 1 -- 172.21.15.62:0/3150473983 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc42c00f7d0 con 0x561f469d5ea0 2023-07-14T16:20:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.731+0000 7fc4257fa700 1 -- 172.21.15.62:0/3150473983 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc42c00f990 con 0x561f469d5ea0 2023-07-14T16:20:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.731+0000 7fc4575abb80 10 client.16818 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.731+0000 7fc4575abb80 1 -- 172.21.15.62:0/3150473983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561f4673dce0 con 0x561f469d5ea0 2023-07-14T16:20:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc4257fa700 1 -- 172.21.15.62:0/3150473983 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561f46880570 con 0x561f469d5ea0 2023-07-14T16:20:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc4257fa700 1 -- 172.21.15.62:0/3150473983 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc42c028bb0 con 0x561f469d5ea0 2023-07-14T16:20:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc4575abb80 10 client.16818 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc4575abb80 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc4575abb80 20 client.16818 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561f469eaf90 0x561f469ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc4575abb80 4 client.16818 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc4575abb80 1 -- 172.21.15.62:0/3150473983 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f4691ff30 con 0x561f469eaf90 2023-07-14T16:20:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.732+0000 7fc44199b700 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561f469eaf90 0x561f469ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.733+0000 7fc44199b700 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561f469eaf90 0x561f469ed3a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983f1f7fcf6002b6 server_cookie=f6a94e81f02106ba in_seq=0 out_seq=0 2023-07-14T16:20:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.733+0000 7fc4257fa700 10 client.16818 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4257fa700 1 -- 172.21.15.62:0/3150473983 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561f4691ff30 con 0x561f469eaf90 2023-07-14T16:20:19.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4257fa700 10 client.16818 handle_command_reply: tid=0 2023-07-14T16:20:19.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 client.16818 shutdown 2023-07-14T16:20:19.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 -- 172.21.15.62:0/3150473983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 msgr2=0x561f469df6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469df6e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc42c01e970 tx=0x7fc42c0047c0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 -- 172.21.15.62:0/3150473983 shutdown_connections 2023-07-14T16:20:19.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561f469eaf90 0x561f469ed3a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f469e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 0x561f469df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3150473983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469df6e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 -- 172.21.15.62:0/3150473983 >> 172.21.15.62:0/3150473983 conn(0x561f469cae20 msgr2=0x561f469cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4257fa700 0 client.16818 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.734+0000 7fc4575abb80 1 -- 172.21.15.62:0/3150473983 shutdown_connections 2023-07-14T16:20:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.735+0000 7fc4575abb80 1 -- 172.21.15.62:0/3150473983 wait complete. 2023-07-14T16:20:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.735+0000 7fc4575abb80 20 client.16818 trim_cache size 0 max 16384 2023-07-14T16:20:19.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.763+0000 7fc4575abb80 1 Processor -- start 2023-07-14T16:20:19.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.763+0000 7fc4575abb80 1 -- start start 2023-07-14T16:20:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.763+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469e7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469dd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc4575abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f469d6270 con 0x561f469d5550 2023-07-14T16:20:19.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc4575abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f469d0f50 con 0x561f469d5ea0 2023-07-14T16:20:19.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc4575abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f469e8380 con 0x561f469d6cf0 2023-07-14T16:20:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34626/0 (socket says 172.21.15.62:34626) 2023-07-14T16:20:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44199b700 1 -- 172.21.15.62:0/1460559669 learned_addr learned my addr 172.21.15.62:0/1460559669 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44219c700 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469dd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44119a700 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44199b700 1 -- 172.21.15.62:0/1460559669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 msgr2=0x561f469dd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44199b700 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469dd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44199b700 1 -- 172.21.15.62:0/1460559669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 msgr2=0x561f469e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44199b700 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469e66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44199b700 1 -- 172.21.15.62:0/1460559669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f469de1d0 con 0x561f469d5550 2023-07-14T16:20:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.764+0000 7fc44119a700 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:20:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.765+0000 7fc44199b700 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469e7010 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc4380058f0 tx=0x7fc4380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb939f0ac34327a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:19.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.765+0000 7fc430bff700 1 -- 172.21.15.62:0/1460559669 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438011e90 con 0x561f469d5550 2023-07-14T16:20:19.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.765+0000 7fc430bff700 1 -- 172.21.15.62:0/1460559669 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc438029b50 con 0x561f469d5550 2023-07-14T16:20:19.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.765+0000 7fc430bff700 1 -- 172.21.15.62:0/1460559669 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380050e0 con 0x561f469d5550 2023-07-14T16:20:19.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 -- 172.21.15.62:0/1460559669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 msgr2=0x561f469e7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469e7010 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc4380058f0 tx=0x7fc4380095c0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 -- 172.21.15.62:0/1460559669 shutdown_connections 2023-07-14T16:20:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d6cf0 0x561f469dd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5550 0x561f469e7010 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 --2- 172.21.15.62:0/1460559669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d5ea0 0x561f469e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 -- 172.21.15.62:0/1460559669 >> 172.21.15.62:0/1460559669 conn(0x561f469cae20 msgr2=0x561f469f1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 -- 172.21.15.62:0/1460559669 shutdown_connections 2023-07-14T16:20:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 -- 172.21.15.62:0/1460559669 wait complete. 2023-07-14T16:20:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.766+0000 7fc4575abb80 1 -- start start 2023-07-14T16:20:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc4575abb80 1 Processor -- start 2023-07-14T16:20:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 0x561f469e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469e3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc4575abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f46a3ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc4575abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f469e6840 con 0x561f469d5ea0 2023-07-14T16:20:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc4575abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f469e69b0 con 0x561f469d6cf0 2023-07-14T16:20:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc4575abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f46a3f340 con 0x561f469d5550 2023-07-14T16:20:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc4575abb80 10 client.0 _ll_register_callbacks cb 0x7fc42c01e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc44119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc44119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469e3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34630/0 (socket says 172.21.15.62:34630) 2023-07-14T16:20:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.767+0000 7fc44119a700 1 -- 172.21.15.62:0/3128769646 learned_addr learned my addr 172.21.15.62:0/3128769646 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44199b700 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 0x561f469e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44219c700 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f46a3ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44119a700 1 -- 172.21.15.62:0/3128769646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 msgr2=0x561f469e32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44119a700 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 0x561f469e32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44119a700 1 -- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 msgr2=0x561f46a3ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44119a700 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f46a3ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44119a700 1 -- 172.21.15.62:0/3128769646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f46754e60 con 0x561f469d5ea0 2023-07-14T16:20:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44219c700 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f46a3ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc44119a700 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469e3820 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005950 tx=0x7fc42c00e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51b9174356a1443 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.768+0000 7fc4257fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4257fa700 1 -- 172.21.15.62:0/3128769646 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00edd0 con 0x561f469d5ea0 2023-07-14T16:20:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 1 -- 172.21.15.62:0/3128769646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4380128e0 con 0x561f469d5ea0 2023-07-14T16:20:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4257fa700 1 -- 172.21.15.62:0/3128769646 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc42c004020 con 0x561f469d5ea0 2023-07-14T16:20:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4257fa700 1 -- 172.21.15.62:0/3128769646 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc42c0041e0 con 0x561f469d5ea0 2023-07-14T16:20:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 10 client.16824 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 1 -- 172.21.15.62:0/3128769646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561f469d7390 con 0x561f469d5ea0 2023-07-14T16:20:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4257fa700 1 -- 172.21.15.62:0/3128769646 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4380128e0 con 0x561f469d5ea0 2023-07-14T16:20:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4257fa700 1 -- 172.21.15.62:0/3128769646 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc42c003680 con 0x561f469d5ea0 2023-07-14T16:20:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 10 client.16824 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 20 client.16824 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561f46a40c00 0x561f46a43010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 4 client.16824 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:20:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.769+0000 7fc4575abb80 1 -- 172.21.15.62:0/3128769646 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561f468e20d0 con 0x561f46a40c00 2023-07-14T16:20:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.770+0000 7fc44199b700 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561f46a40c00 0x561f46a43010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.770+0000 7fc4257fa700 10 client.16824 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.770+0000 7fc44199b700 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561f46a40c00 0x561f46a43010 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466cc45c03ed256e server_cookie=438408e414b889c2 in_seq=0 out_seq=0 2023-07-14T16:20:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.771+0000 7fc4257fa700 1 -- 172.21.15.62:0/3128769646 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561f468e20d0 con 0x561f46a40c00 2023-07-14T16:20:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.771+0000 7fc4257fa700 10 client.16824 handle_command_reply: tid=0 2023-07-14T16:20:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.771+0000 7fc4575abb80 1 client.16824 shutdown 2023-07-14T16:20:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 -- 172.21.15.62:0/3128769646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 msgr2=0x561f469e3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469e3820 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005950 tx=0x7fc42c00e860 comp rx=0 tx=0).stop 2023-07-14T16:20:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 -- 172.21.15.62:0/3128769646 shutdown_connections 2023-07-14T16:20:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561f46a40c00 0x561f46a43010 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f469d6cf0 0x561f46a3ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f469d5550 0x561f469e32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 --2- 172.21.15.62:0/3128769646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f469d5ea0 0x561f469e3820 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4257fa700 0 client.16824 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 -- 172.21.15.62:0/3128769646 >> 172.21.15.62:0/3128769646 conn(0x561f469cae20 msgr2=0x561f469e7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 -- 172.21.15.62:0/3128769646 shutdown_connections 2023-07-14T16:20:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 1 -- 172.21.15.62:0/3128769646 wait complete. 2023-07-14T16:20:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.772+0000 7fc4575abb80 20 client.16824 trim_cache size 0 max 16384 2023-07-14T16:20:19.762 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:20:19.762 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:20:19.763 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "d44aa9e5-4764-4dc3-a67d-1dedeabb727e", 2023-07-14T16:20:19.763 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:20:19.763 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:20:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 -- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc4280a11d0 msgr2=0x7fc4280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc4280a11d0 0x7fc4280a3680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc43c074270 tx=0x7fc42c011440 comp rx=0 tx=0).stop 2023-07-14T16:20:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 -- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 msgr2=0x7fc43c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 0x7fc43c19f010 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc43400d010 tx=0x7fc43400bcc0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 -- 172.21.15.62:0/1101938505 shutdown_connections 2023-07-14T16:20:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc43c1079a0 0x7fc43c19f010 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc4280a11d0 0x7fc4280a3680 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc43c072f70 0x7fc43c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 --2- 172.21.15.62:0/1101938505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc43c074550 0x7fc43c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 -- 172.21.15.62:0/1101938505 >> 172.21.15.62:0/1101938505 conn(0x7fc43c0fb0e0 msgr2=0x7fc43c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 -- 172.21.15.62:0/1101938505 shutdown_connections 2023-07-14T16:20:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:19.773+0000 7fc443bff700 1 -- 172.21.15.62:0/1101938505 wait complete. 2023-07-14T16:20:19.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:20:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 -- 172.21.15.62:0/3056593352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 msgr2=0x7f3e80101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 --2- 172.21.15.62:0/3056593352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 0x7f3e80101130 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009990 tx=0x7f3e70009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 -- 172.21.15.62:0/3056593352 shutdown_connections 2023-07-14T16:20:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 --2- 172.21.15.62:0/3056593352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e80101670 0x7f3e80107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 --2- 172.21.15.62:0/3056593352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e800ffaf0 0x7f3e800fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 --2- 172.21.15.62:0/3056593352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 0x7f3e80101130 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 -- 172.21.15.62:0/3056593352 >> 172.21.15.62:0/3056593352 conn(0x7f3e800fb130 msgr2=0x7f3e800fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 -- 172.21.15.62:0/3056593352 shutdown_connections 2023-07-14T16:20:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.028+0000 7f3e883ff700 1 -- 172.21.15.62:0/3056593352 wait complete. 2023-07-14T16:20:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.029+0000 7f3e883ff700 1 Processor -- start 2023-07-14T16:20:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.029+0000 7f3e883ff700 1 -- start start 2023-07-14T16:20:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e883ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e800ffaf0 0x7f3e800742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e883ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 0x7f3e80072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e883ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e80101670 0x7f3e80072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e883ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800735e0 con 0x7f3e80100ce0 2023-07-14T16:20:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e883ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80073750 con 0x7f3e800ffaf0 2023-07-14T16:20:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e883ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8019f7e0 con 0x7f3e80101670 2023-07-14T16:20:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e8599a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 0x7f3e80072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e8599a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 0x7f3e80072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34648/0 (socket says 172.21.15.62:34648) 2023-07-14T16:20:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e8599a700 1 -- 172.21.15.62:0/2722836083 learned_addr learned my addr 172.21.15.62:0/2722836083 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.030+0000 7f3e8619b700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e800ffaf0 0x7f3e800742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8599a700 1 -- 172.21.15.62:0/2722836083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e80101670 msgr2=0x7f3e80072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8699c700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e80101670 0x7f3e80072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8599a700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e80101670 0x7f3e80072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8599a700 1 -- 172.21.15.62:0/2722836083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e800ffaf0 msgr2=0x7f3e800742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8599a700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e800ffaf0 0x7f3e800742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8599a700 1 -- 172.21.15.62:0/2722836083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700095f0 con 0x7f3e80100ce0 2023-07-14T16:20:20.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8619b700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e800ffaf0 0x7f3e800742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:20.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8699c700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e80101670 0x7f3e80072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:20:20.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e8599a700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 0x7f3e80072900 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3e70005910 tx=0x7f3e70005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:20.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e777fe700 1 -- 172.21.15.62:0/2722836083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7001d070 con 0x7f3e80100ce0 2023-07-14T16:20:20.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e883ff700 1 -- 172.21.15.62:0/2722836083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8019fa60 con 0x7f3e80100ce0 2023-07-14T16:20:20.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e777fe700 1 -- 172.21.15.62:0/2722836083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e7000be90 con 0x7f3e80100ce0 2023-07-14T16:20:20.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.031+0000 7f3e777fe700 1 -- 172.21.15.62:0/2722836083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70022e60 con 0x7f3e80100ce0 2023-07-14T16:20:20.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.032+0000 7f3e883ff700 1 -- 172.21.15.62:0/2722836083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8019fe80 con 0x7f3e80100ce0 2023-07-14T16:20:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.035+0000 7f3e777fe700 1 -- 172.21.15.62:0/2722836083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3e70022600 con 0x7f3e80100ce0 2023-07-14T16:20:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.037+0000 7f3e777fe700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3e6c0a1150 0x7f3e6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.037+0000 7f3e777fe700 1 -- 172.21.15.62:0/2722836083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3e700cdd60 con 0x7f3e80100ce0 2023-07-14T16:20:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.037+0000 7f3e9bc6db80 1 Processor -- start 2023-07-14T16:20:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.037+0000 7f3e8619b700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3e6c0a1150 0x7f3e6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.037+0000 7f3e9bc6db80 1 -- start start 2023-07-14T16:20:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x56439936f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x5643993706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x5643993710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564399274cb0 con 0x56439936f4d0 2023-07-14T16:20:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643992c7a40 con 0x564399370c70 2023-07-14T16:20:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643992c0ed0 con 0x56439936fe20 2023-07-14T16:20:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e8599a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x5643993706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e8599a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x5643993706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50398/0 (socket says 172.21.15.62:50398) 2023-07-14T16:20:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e8599a700 1 -- 172.21.15.62:0/674591409 learned_addr learned my addr 172.21.15.62:0/674591409 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.038+0000 7f3e8619b700 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x56439936f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8699c700 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x5643993710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8619b700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3e6c0a1150 0x7f3e6c0a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e80100b40 tx=0x7f3e7c009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8599a700 1 -- 172.21.15.62:0/674591409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 msgr2=0x5643993710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8599a700 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x5643993710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8599a700 1 -- 172.21.15.62:0/674591409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 msgr2=0x56439936f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8599a700 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x56439936f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8599a700 1 -- 172.21.15.62:0/674591409 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643990eede0 con 0x56439936fe20 2023-07-14T16:20:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8619b700 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x56439936f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e8599a700 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x5643993706e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e700a3040 tx=0x7f3e7009b0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f0380ffc23bc0c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.039+0000 7f3e74bfe700 1 -- 172.21.15.62:0/674591409 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7009ab10 con 0x56439936fe20 2023-07-14T16:20:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e74bfe700 1 -- 172.21.15.62:0/674591409 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e7009b1b0 con 0x56439936fe20 2023-07-14T16:20:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e74bfe700 1 -- 172.21.15.62:0/674591409 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7009a970 con 0x56439936fe20 2023-07-14T16:20:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/674591409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 msgr2=0x5643993706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x5643993706e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3e700a3040 tx=0x7f3e7009b0a0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/674591409 shutdown_connections 2023-07-14T16:20:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x5643993710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x56439936f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/674591409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x5643993706e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.040+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/674591409 >> 172.21.15.62:0/674591409 conn(0x564399364da0 msgr2=0x56439936a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.041+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/674591409 shutdown_connections 2023-07-14T16:20:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.041+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/674591409 wait complete. 2023-07-14T16:20:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.041+0000 7f3e9bc6db80 1 -- start start 2023-07-14T16:20:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e9bc6db80 1 Processor -- start 2023-07-14T16:20:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399378df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x564399379330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x564399380370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564399260ac0 con 0x56439936f4d0 2023-07-14T16:20:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643992607c0 con 0x564399370c70 2023-07-14T16:20:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564399144920 con 0x56439936fe20 2023-07-14T16:20:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e9bc6db80 10 client.0 _ll_register_callbacks cb 0x56439925e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e8619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399378df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e8619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399378df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34652/0 (socket says 172.21.15.62:34652) 2023-07-14T16:20:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.042+0000 7f3e8619b700 1 -- 172.21.15.62:0/840510537 learned_addr learned my addr 172.21.15.62:0/840510537 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8599a700 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x564399379330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8699c700 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x564399380370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8619b700 1 -- 172.21.15.62:0/840510537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 msgr2=0x564399379330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8619b700 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x564399379330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8619b700 1 -- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 msgr2=0x564399380370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8619b700 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x564399380370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8619b700 1 -- 172.21.15.62:0/840510537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c01d440 con 0x56439936f4d0 2023-07-14T16:20:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8699c700 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x564399380370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:20.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e8619b700 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399378df0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c01ebd0 tx=0x7f3e7c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e781f4b06bd71b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:20.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.043+0000 7f3e69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:20.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.044+0000 7f3e69ffb700 1 -- 172.21.15.62:0/840510537 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c00fd50 con 0x56439936f4d0 2023-07-14T16:20:20.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.044+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/840510537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56439921a4f0 con 0x56439936f4d0 2023-07-14T16:20:20.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.044+0000 7f3e69ffb700 1 -- 172.21.15.62:0/840510537 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3e7c004830 con 0x56439936f4d0 2023-07-14T16:20:20.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.044+0000 7f3e69ffb700 1 -- 172.21.15.62:0/840510537 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e7c02e030 con 0x56439936f4d0 2023-07-14T16:20:20.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.044+0000 7f3e9bc6db80 10 client.16836 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:20.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.044+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/840510537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f3e8019fe80 con 0x56439936f4d0 2023-07-14T16:20:20.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.044+0000 7f3e69ffb700 1 -- 172.21.15.62:0/840510537 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56439921a4f0 con 0x56439936f4d0 2023-07-14T16:20:20.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.044+0000 7f3e69ffb700 1 -- 172.21.15.62:0/840510537 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e7c005490 con 0x56439936f4d0 2023-07-14T16:20:20.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.045+0000 7f3e9bc6db80 10 client.16836 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:20.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.045+0000 7f3e9bc6db80 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:20.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.045+0000 7f3e9bc6db80 20 client.16836 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:20.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.045+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564399384c30 0x564399387040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.045+0000 7f3e9bc6db80 4 client.16836 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.045+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/840510537 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643992b9eb0 con 0x564399384c30 2023-07-14T16:20:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.045+0000 7f3e8599a700 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564399384c30 0x564399387040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.046+0000 7f3e8599a700 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564399384c30 0x564399387040 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da62a164991cc6b2 server_cookie=4053fb911203337f in_seq=0 out_seq=0 2023-07-14T16:20:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.046+0000 7f3e69ffb700 10 client.16836 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.046+0000 7f3e69ffb700 1 -- 172.21.15.62:0/840510537 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5643992b9eb0 con 0x564399384c30 2023-07-14T16:20:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.046+0000 7f3e69ffb700 10 client.16836 handle_command_reply: tid=0 2023-07-14T16:20:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.046+0000 7f3e9bc6db80 1 client.16836 shutdown 2023-07-14T16:20:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/840510537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 msgr2=0x564399378df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399378df0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c01ebd0 tx=0x7f3e7c016000 comp rx=0 tx=0).stop 2023-07-14T16:20:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/840510537 shutdown_connections 2023-07-14T16:20:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564399384c30 0x564399387040 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564399370c70 0x564399380370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399378df0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e69ffb700 0 client.16836 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:20.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/840510537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x564399379330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/840510537 >> 172.21.15.62:0/840510537 conn(0x564399364da0 msgr2=0x5643993698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:20.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/840510537 shutdown_connections 2023-07-14T16:20:20.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/840510537 wait complete. 2023-07-14T16:20:20.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.047+0000 7f3e9bc6db80 20 client.16836 trim_cache size 0 max 16384 2023-07-14T16:20:20.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.070+0000 7f3e9bc6db80 1 Processor -- start 2023-07-14T16:20:20.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.070+0000 7f3e9bc6db80 1 -- start start 2023-07-14T16:20:20.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.070+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399380ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.070+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936fe20 0x564399380330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564399370c70 0x564399376fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439936bc60 con 0x56439936f4d0 2023-07-14T16:20:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564399364970 con 0x56439936fe20 2023-07-14T16:20:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564399376d90 con 0x564399370c70 2023-07-14T16:20:20.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399380ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399380ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34656/0 (socket says 172.21.15.62:34656) 2023-07-14T16:20:20.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8619b700 1 -- 172.21.15.62:0/3927679189 learned_addr learned my addr 172.21.15.62:0/3927679189 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:20.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8699c700 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564399370c70 0x564399376fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8599a700 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936fe20 0x564399380330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8619b700 1 -- 172.21.15.62:0/3927679189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564399370c70 msgr2=0x564399376fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8619b700 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564399370c70 0x564399376fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8619b700 1 -- 172.21.15.62:0/3927679189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936fe20 msgr2=0x564399380330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8619b700 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936fe20 0x564399380330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8619b700 1 -- 172.21.15.62:0/3927679189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643990d7c60 con 0x56439936f4d0 2023-07-14T16:20:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.071+0000 7f3e8599a700 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936fe20 0x564399380330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.072+0000 7f3e8619b700 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399380ba0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0092e0 tx=0x7f3e7c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff83802826a7ec9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.072+0000 7f3e753ff700 1 -- 172.21.15.62:0/3927679189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c02e040 con 0x56439936f4d0 2023-07-14T16:20:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.072+0000 7f3e753ff700 1 -- 172.21.15.62:0/3927679189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e7c0286f0 con 0x56439936f4d0 2023-07-14T16:20:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.072+0000 7f3e753ff700 1 -- 172.21.15.62:0/3927679189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c028b90 con 0x56439936f4d0 2023-07-14T16:20:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3927679189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 msgr2=0x564399380ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399380ba0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0092e0 tx=0x7f3e7c005c40 comp rx=0 tx=0).stop 2023-07-14T16:20:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3927679189 shutdown_connections 2023-07-14T16:20:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564399370c70 0x564399376fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56439936f4d0 0x564399380ba0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3927679189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936fe20 0x564399380330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3927679189 >> 172.21.15.62:0/3927679189 conn(0x564399364da0 msgr2=0x56439938b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3927679189 shutdown_connections 2023-07-14T16:20:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3927679189 wait complete. 2023-07-14T16:20:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.073+0000 7f3e9bc6db80 1 -- start start 2023-07-14T16:20:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e9bc6db80 1 Processor -- start 2023-07-14T16:20:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936f4d0 0x56439937fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x564399380220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e9bc6db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564399370c70 0x56439937c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439937ceb0 con 0x564399370c70 2023-07-14T16:20:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439937d020 con 0x56439936f4d0 2023-07-14T16:20:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e9bc6db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439937d1e0 con 0x56439936fe20 2023-07-14T16:20:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e9bc6db80 10 client.0 _ll_register_callbacks cb 0x7f3e7c01e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e8619b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936f4d0 0x56439937fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e8699c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564399370c70 0x56439937c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.074+0000 7f3e8619b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936f4d0 0x56439937fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51018/0 (socket says 172.21.15.62:51018) 2023-07-14T16:20:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e8619b700 1 -- 172.21.15.62:0/3755931663 learned_addr learned my addr 172.21.15.62:0/3755931663 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e8599a700 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x564399380220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e8619b700 1 -- 172.21.15.62:0/3755931663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 msgr2=0x564399380220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e8619b700 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x564399380220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e8619b700 1 -- 172.21.15.62:0/3755931663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564399370c70 msgr2=0x56439937c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e8619b700 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564399370c70 0x56439937c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e8619b700 1 -- 172.21.15.62:0/3755931663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8019fe80 con 0x56439936f4d0 2023-07-14T16:20:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e8619b700 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936f4d0 0x56439937fce0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005850 tx=0x7f3e7c00ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b8e457184c38c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.075+0000 7f3e69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.076+0000 7f3e69ffb700 1 -- 172.21.15.62:0/3755931663 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c03f070 con 0x56439936f4d0 2023-07-14T16:20:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.076+0000 7f3e69ffb700 1 -- 172.21.15.62:0/3755931663 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3e7c02f070 con 0x56439936f4d0 2023-07-14T16:20:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.076+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3755931663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e7009b1b0 con 0x56439936f4d0 2023-07-14T16:20:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.076+0000 7f3e69ffb700 1 -- 172.21.15.62:0/3755931663 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e7c02e040 con 0x56439936f4d0 2023-07-14T16:20:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.076+0000 7f3e9bc6db80 10 client.26555 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.076+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3755931663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564399115250 con 0x56439936f4d0 2023-07-14T16:20:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.076+0000 7f3e69ffb700 1 -- 172.21.15.62:0/3755931663 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e7009b1b0 con 0x56439936f4d0 2023-07-14T16:20:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e69ffb700 1 -- 172.21.15.62:0/3755931663 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e7c042030 con 0x56439936f4d0 2023-07-14T16:20:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e9bc6db80 10 client.26555 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e9bc6db80 10 client.26555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e9bc6db80 20 client.26555 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5643993d9d00 0x5643993dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e9bc6db80 4 client.26555 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:20:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3755931663 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56439927c050 con 0x5643993d9d00 2023-07-14T16:20:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e8599a700 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5643993d9d00 0x5643993dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e69ffb700 10 client.26555 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.077+0000 7f3e8599a700 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5643993d9d00 0x5643993dc110 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacfa3fd20f3a120 server_cookie=9ae74f3424672d1d in_seq=0 out_seq=0 2023-07-14T16:20:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e69ffb700 1 -- 172.21.15.62:0/3755931663 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56439927c050 con 0x5643993d9d00 2023-07-14T16:20:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e69ffb700 10 client.26555 handle_command_reply: tid=0 2023-07-14T16:20:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 client.26555 shutdown 2023-07-14T16:20:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3755931663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936f4d0 msgr2=0x56439937fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936f4d0 0x56439937fce0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005850 tx=0x7f3e7c00ff70 comp rx=0 tx=0).stop 2023-07-14T16:20:20.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3755931663 shutdown_connections 2023-07-14T16:20:20.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564399370c70 0x56439937c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5643993d9d00 0x5643993dc110 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56439936f4d0 0x56439937fce0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 --2- 172.21.15.62:0/3755931663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56439936fe20 0x564399380220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e69ffb700 0 client.26555 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3755931663 >> 172.21.15.62:0/3755931663 conn(0x564399364da0 msgr2=0x564399380ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3755931663 shutdown_connections 2023-07-14T16:20:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 1 -- 172.21.15.62:0/3755931663 wait complete. 2023-07-14T16:20:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.078+0000 7f3e9bc6db80 20 client.26555 trim_cache size 0 max 16384 2023-07-14T16:20:20.071 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:20:20.071 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:20:20.072 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:20:20.072 INFO:teuthology.orchestra.run.smithi062.stdout: "d44aa9e5-4764-4dc3-a67d-1dedeabb727e": { 2023-07-14T16:20:20.072 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:20:20.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "d44aa9e5-4764-4dc3-a67d-1dedeabb727e", 2023-07-14T16:20:20.073 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:20:20.073 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:20:20.073 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:20:20.073 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:20:20.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 -- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3e6c0a1150 msgr2=0x7f3e6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3e6c0a1150 0x7f3e6c0a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e80100b40 tx=0x7f3e7c009f90 comp rx=0 tx=0).stop 2023-07-14T16:20:20.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 -- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 msgr2=0x7f3e80072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:20.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 0x7f3e80072900 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3e70005910 tx=0x7f3e70005720 comp rx=0 tx=0).stop 2023-07-14T16:20:20.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 -- 172.21.15.62:0/2722836083 shutdown_connections 2023-07-14T16:20:20.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e80101670 0x7f3e80072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3e6c0a1150 0x7f3e6c0a3600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e800ffaf0 0x7f3e800742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 --2- 172.21.15.62:0/2722836083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e80100ce0 0x7f3e80072900 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:20.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 -- 172.21.15.62:0/2722836083 >> 172.21.15.62:0/2722836083 conn(0x7f3e800fb130 msgr2=0x7f3e800fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:20.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 -- 172.21.15.62:0/2722836083 shutdown_connections 2023-07-14T16:20:20.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:20.079+0000 7f3e883ff700 1 -- 172.21.15.62:0/2722836083 wait complete. 2023-07-14T16:20:20.078 INFO:tasks.cephfs.filesystem:scrub status for tag:d44aa9e5-4764-4dc3-a67d-1dedeabb727e - {'path': '/', 'tag': 'd44aa9e5-4764-4dc3-a67d-1dedeabb727e', 'options': 'recursive,force'} 2023-07-14T16:20:20.153 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:20:19 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:20:19.780+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v125>,<0x10000000003/client.0 v126>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v138>,<0x10000000001/sv_0 v152>,<0x10000000000/_nogroup v146>,<0x1/volumes v136>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:20:21.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: pgmap v498: 97 pgs: 97 active+clean; 2.5 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:21.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:20:21.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:20:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:20:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:20 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: pgmap v498: 97 pgs: 97 active+clean; 2.5 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:20:21.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:20:21.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:20:21.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:20:21.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:20:21.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:20:21.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:20 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: pgmap v498: 97 pgs: 97 active+clean; 2.5 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:20:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:20:21.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:20:21.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:20:21.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:20:21.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:20:21.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:21.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:20 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:22 smithi164 ceph-mon[97427]: pgmap v499: 97 pgs: 97 active+clean; 2.5 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 23 op/s 2023-07-14T16:20:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:22 smithi062 ceph-mon[95543]: pgmap v499: 97 pgs: 97 active+clean; 2.5 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 23 op/s 2023-07-14T16:20:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:22 smithi081 ceph-mon[127795]: pgmap v499: 97 pgs: 97 active+clean; 2.5 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 23 op/s 2023-07-14T16:20:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:24 smithi164 ceph-mon[97427]: pgmap v500: 97 pgs: 97 active+clean; 2.5 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:20:25.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:24 smithi062 ceph-mon[95543]: pgmap v500: 97 pgs: 97 active+clean; 2.5 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:20:25.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:24 smithi081 ceph-mon[127795]: pgmap v500: 97 pgs: 97 active+clean; 2.5 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:20:27.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:26 smithi164 ceph-mon[97427]: pgmap v501: 97 pgs: 97 active+clean; 2.5 GiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:20:27.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:26 smithi062 ceph-mon[95543]: pgmap v501: 97 pgs: 97 active+clean; 2.5 GiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:20:27.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:26 smithi081 ceph-mon[127795]: pgmap v501: 97 pgs: 97 active+clean; 2.5 GiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:20:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.478+0000 7f56d49ff700 1 -- 172.21.15.62:0/4038220511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00ffb00 msgr2=0x7f56d00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.479+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4038220511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00ffb00 0x7f56d00fff10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009990 tx=0x7f56c0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.479+0000 7f56d49ff700 1 -- 172.21.15.62:0/4038220511 shutdown_connections 2023-07-14T16:20:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.479+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4038220511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d0101680 0x7f56d0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.479+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4038220511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d00ffb00 0x7f56d00fff10 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.479+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4038220511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56d0100cf0 0x7f56d0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.479+0000 7f56d49ff700 1 -- 172.21.15.62:0/4038220511 >> 172.21.15.62:0/4038220511 conn(0x7f56d00fb0e0 msgr2=0x7f56d00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.479+0000 7f56d49ff700 1 -- 172.21.15.62:0/4038220511 shutdown_connections 2023-07-14T16:20:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.479+0000 7f56d49ff700 1 -- 172.21.15.62:0/4038220511 wait complete. 2023-07-14T16:20:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.480+0000 7f56d49ff700 1 Processor -- start 2023-07-14T16:20:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.480+0000 7f56d49ff700 1 -- start start 2023-07-14T16:20:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.480+0000 7f56d49ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d00ffb00 0x7f56d0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.480+0000 7f56d49ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0100cf0 0x7f56d0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56d49ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56d0101680 0x7f56d0198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56d49ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0199510 con 0x7f56d0100cf0 2023-07-14T16:20:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56d49ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0199680 con 0x7f56d00ffb00 2023-07-14T16:20:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56d49ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0199840 con 0x7f56d0101680 2023-07-14T16:20:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0100cf0 0x7f56d0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0100cf0 0x7f56d0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43556/0 (socket says 172.21.15.62:43556) 2023-07-14T16:20:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56cdd9b700 1 -- 172.21.15.62:0/4240564861 learned_addr learned my addr 172.21.15.62:0/4240564861 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56ce59c700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d00ffb00 0x7f56d0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56ced9d700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56d0101680 0x7f56d0198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56cdd9b700 1 -- 172.21.15.62:0/4240564861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56d0101680 msgr2=0x7f56d0198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56d0101680 0x7f56d0198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56cdd9b700 1 -- 172.21.15.62:0/4240564861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d00ffb00 msgr2=0x7f56d0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d00ffb00 0x7f56d0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.481+0000 7f56cdd9b700 1 -- 172.21.15.62:0/4240564861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00095f0 con 0x7f56d0100cf0 2023-07-14T16:20:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.482+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0100cf0 0x7f56d0198030 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f56c400e9a0 tx=0x7f56c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.482+0000 7f56bf7fe700 1 -- 172.21.15.62:0/4240564861 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4009790 con 0x7f56d0100cf0 2023-07-14T16:20:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.482+0000 7f56d49ff700 1 -- 172.21.15.62:0/4240564861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d0199b20 con 0x7f56d0100cf0 2023-07-14T16:20:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.482+0000 7f56bf7fe700 1 -- 172.21.15.62:0/4240564861 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56c400cbd0 con 0x7f56d0100cf0 2023-07-14T16:20:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.482+0000 7f56d49ff700 1 -- 172.21.15.62:0/4240564861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d01a02e0 con 0x7f56d0100cf0 2023-07-14T16:20:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.482+0000 7f56bf7fe700 1 -- 172.21.15.62:0/4240564861 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4018680 con 0x7f56d0100cf0 2023-07-14T16:20:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.486+0000 7f56bf7fe700 1 -- 172.21.15.62:0/4240564861 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f56c400cd40 con 0x7f56d0100cf0 2023-07-14T16:20:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.487+0000 7f56e840ab80 1 Processor -- start 2023-07-14T16:20:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.487+0000 7f56e840ab80 1 -- start start 2023-07-14T16:20:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e6106b0 0x55782e7b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56e840ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e6b9c50 con 0x55782e6106b0 2023-07-14T16:20:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56e840ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e70c9e0 con 0x55782e7b4840 2023-07-14T16:20:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56e840ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e705e70 con 0x55782e7b56e0 2023-07-14T16:20:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e6106b0 0x55782e7b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e6106b0 0x55782e7b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43564/0 (socket says 172.21.15.62:43564) 2023-07-14T16:20:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56ced9d700 1 -- 172.21.15.62:0/1719435892 learned_addr learned my addr 172.21.15.62:0/1719435892 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56bf7fe700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f56b80a11d0 0x7f56b80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56ced9d700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f56b80a11d0 0x7f56b80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.488+0000 7f56bf7fe700 1 -- 172.21.15.62:0/4240564861 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f56c4014070 con 0x7f56d0100cf0 2023-07-14T16:20:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56cdd9b700 1 -- 172.21.15.62:0/1719435892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 msgr2=0x55782e7b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56ce59c700 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56cdd9b700 1 -- 172.21.15.62:0/1719435892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e6106b0 msgr2=0x55782e7b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e6106b0 0x55782e7b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56cdd9b700 1 -- 172.21.15.62:0/1719435892 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55782e533d80 con 0x55782e7b56e0 2023-07-14T16:20:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56ced9d700 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e6106b0 0x55782e7b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56ce59c700 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:20:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7b5b80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f56c4096440 tx=0x7f56c401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11e48ef321624f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56ced9d700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f56b80a11d0 0x7f56b80a3680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55782e7b4060 tx=0x7f56c8004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56bcff9700 1 -- 172.21.15.62:0/1719435892 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4099f40 con 0x55782e7b56e0 2023-07-14T16:20:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56bcff9700 1 -- 172.21.15.62:0/1719435892 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56c409df70 con 0x55782e7b56e0 2023-07-14T16:20:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.489+0000 7f56bcff9700 1 -- 172.21.15.62:0/1719435892 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4099690 con 0x55782e7b56e0 2023-07-14T16:20:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 -- 172.21.15.62:0/1719435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 msgr2=0x55782e7b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7b5b80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f56c4096440 tx=0x7f56c401f000 comp rx=0 tx=0).stop 2023-07-14T16:20:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 -- 172.21.15.62:0/1719435892 shutdown_connections 2023-07-14T16:20:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7b5b80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e6106b0 0x55782e7b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 --2- 172.21.15.62:0/1719435892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 -- 172.21.15.62:0/1719435892 >> 172.21.15.62:0/1719435892 conn(0x55782e7a9be0 msgr2=0x55782e7aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 -- 172.21.15.62:0/1719435892 shutdown_connections 2023-07-14T16:20:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.490+0000 7f56e840ab80 1 -- 172.21.15.62:0/1719435892 wait complete. 2023-07-14T16:20:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.491+0000 7f56e840ab80 1 -- start start 2023-07-14T16:20:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.491+0000 7f56e840ab80 1 Processor -- start 2023-07-14T16:20:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b4840 0x55782e7bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56e840ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e6a5a60 con 0x55782e7b4840 2023-07-14T16:20:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56e840ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e6a5760 con 0x55782e6106b0 2023-07-14T16:20:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56e840ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e5898c0 con 0x55782e7b56e0 2023-07-14T16:20:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56e840ab80 10 client.0 _ll_register_callbacks cb 0x55782e6a3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b4840 0x55782e7bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56ce59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b4840 0x55782e7bddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43566/0 (socket says 172.21.15.62:43566) 2023-07-14T16:20:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56cdd9b700 1 -- 172.21.15.62:0/3558710509 learned_addr learned my addr 172.21.15.62:0/3558710509 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56ced9d700 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.492+0000 7f56cdd9b700 1 -- 172.21.15.62:0/3558710509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 msgr2=0x55782e7c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56cdd9b700 1 -- 172.21.15.62:0/3558710509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 msgr2=0x55782e7bd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7bd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56cdd9b700 1 -- 172.21.15.62:0/3558710509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c8007620 con 0x55782e7b4840 2023-07-14T16:20:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56ced9d700 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56ce59c700 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b4840 0x55782e7bddc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f56c4099ed0 tx=0x7f56c409a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a839331ae5b2508d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56b5ffb700 1 -- 172.21.15.62:0/3558710509 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4099f40 con 0x55782e7b4840 2023-07-14T16:20:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56b5ffb700 1 -- 172.21.15.62:0/3558710509 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f56c40982e0 con 0x55782e7b4840 2023-07-14T16:20:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56e840ab80 1 -- 172.21.15.62:0/3558710509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55782e65f490 con 0x55782e7b4840 2023-07-14T16:20:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56b5ffb700 1 -- 172.21.15.62:0/3558710509 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c40a2110 con 0x55782e7b4840 2023-07-14T16:20:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56e840ab80 10 client.16854 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.493+0000 7f56e840ab80 1 -- 172.21.15.62:0/3558710509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f56d01a02e0 con 0x55782e7b4840 2023-07-14T16:20:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.494+0000 7f56b5ffb700 1 -- 172.21.15.62:0/3558710509 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55782e65f490 con 0x55782e7b4840 2023-07-14T16:20:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.494+0000 7f56b5ffb700 1 -- 172.21.15.62:0/3558710509 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f56c4095e50 con 0x55782e7b4840 2023-07-14T16:20:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.494+0000 7f56e840ab80 10 client.16854 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.494+0000 7f56e840ab80 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.494+0000 7f56e840ab80 20 client.16854 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.494+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55782e7c9200 0x55782e7cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.494+0000 7f56e840ab80 4 client.16854 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.494+0000 7f56e840ab80 1 -- 172.21.15.62:0/3558710509 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55782e6fee50 con 0x55782e7c9200 2023-07-14T16:20:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.495+0000 7f56ce59c700 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55782e7c9200 0x55782e7cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.495+0000 7f56ce59c700 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55782e7c9200 0x55782e7cb610 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15e4b1325afc354 server_cookie=d5f89c3411ceabc in_seq=0 out_seq=0 2023-07-14T16:20:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.495+0000 7f56b5ffb700 10 client.16854 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.496+0000 7f56b5ffb700 1 -- 172.21.15.62:0/3558710509 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55782e6fee50 con 0x55782e7c9200 2023-07-14T16:20:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.496+0000 7f56b5ffb700 10 client.16854 handle_command_reply: tid=0 2023-07-14T16:20:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.496+0000 7f56e840ab80 1 client.16854 shutdown 2023-07-14T16:20:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.496+0000 7f56e840ab80 1 -- 172.21.15.62:0/3558710509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b4840 msgr2=0x55782e7bddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.496+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b4840 0x55782e7bddc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f56c4099ed0 tx=0x7f56c409a230 comp rx=0 tx=0).stop 2023-07-14T16:20:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.496+0000 7f56e840ab80 1 -- 172.21.15.62:0/3558710509 shutdown_connections 2023-07-14T16:20:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.496+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b56e0 0x55782e7c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.497+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55782e7c9200 0x55782e7cb610 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.497+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.497+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3558710509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b4840 0x55782e7bddc0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.497+0000 7f56b5ffb700 0 client.16854 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.497+0000 7f56e840ab80 1 -- 172.21.15.62:0/3558710509 >> 172.21.15.62:0/3558710509 conn(0x55782e7a9be0 msgr2=0x55782e7ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.497+0000 7f56e840ab80 1 -- 172.21.15.62:0/3558710509 shutdown_connections 2023-07-14T16:20:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.497+0000 7f56e840ab80 1 -- 172.21.15.62:0/3558710509 wait complete. 2023-07-14T16:20:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.497+0000 7f56e840ab80 20 client.16854 trim_cache size 0 max 16384 2023-07-14T16:20:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.518+0000 7f56e840ab80 1 Processor -- start 2023-07-14T16:20:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.518+0000 7f56e840ab80 1 -- start start 2023-07-14T16:20:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.518+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e6106b0 0x55782e7bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.518+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.518+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.518+0000 7f56e840ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e7bc320 con 0x55782e7b56e0 2023-07-14T16:20:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.518+0000 7f56e840ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e7a6bb0 con 0x55782e7b4840 2023-07-14T16:20:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.518+0000 7f56e840ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e7b0780 con 0x55782e6106b0 2023-07-14T16:20:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7c4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35912/0 (socket says 172.21.15.62:35912) 2023-07-14T16:20:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 -- 172.21.15.62:0/3044521633 learned_addr learned my addr 172.21.15.62:0/3044521633 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56ce59c700 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e6106b0 0x55782e7bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 -- 172.21.15.62:0/3044521633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e6106b0 msgr2=0x55782e7bcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e6106b0 0x55782e7bcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 -- 172.21.15.62:0/3044521633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 msgr2=0x55782e7bc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7bc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 -- 172.21.15.62:0/3044521633 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55782e51cc00 con 0x55782e7b4840 2023-07-14T16:20:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56ce59c700 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e6106b0 0x55782e7bcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.519+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7c4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f56c4010b20 tx=0x7f56c409a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc7db96e12732a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56bd7fa700 1 -- 172.21.15.62:0/3044521633 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c409a2a0 con 0x55782e7b4840 2023-07-14T16:20:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56bd7fa700 1 -- 172.21.15.62:0/3044521633 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56c4099f40 con 0x55782e7b4840 2023-07-14T16:20:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56bd7fa700 1 -- 172.21.15.62:0/3044521633 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4095f30 con 0x55782e7b4840 2023-07-14T16:20:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56e840ab80 1 -- 172.21.15.62:0/3044521633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 msgr2=0x55782e7c4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7c4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f56c4010b20 tx=0x7f56c409a550 comp rx=0 tx=0).stop 2023-07-14T16:20:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56e840ab80 1 -- 172.21.15.62:0/3044521633 shutdown_connections 2023-07-14T16:20:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e6106b0 0x55782e7bcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56e840ab80 1 --2- 172.21.15.62:0/3044521633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e7b4840 0x55782e7c4dc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.520+0000 7f56e840ab80 1 -- 172.21.15.62:0/3044521633 >> 172.21.15.62:0/3044521633 conn(0x55782e7a9be0 msgr2=0x55782e7cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.521+0000 7f56e840ab80 1 -- 172.21.15.62:0/3044521633 shutdown_connections 2023-07-14T16:20:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.521+0000 7f56e840ab80 1 -- 172.21.15.62:0/3044521633 wait complete. 2023-07-14T16:20:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.521+0000 7f56e840ab80 1 -- start start 2023-07-14T16:20:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.521+0000 7f56e840ab80 1 Processor -- start 2023-07-14T16:20:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7c0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b4840 0x55782e7c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56e840ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56e840ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e7c3f00 con 0x55782e7b56e0 2023-07-14T16:20:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56e840ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e7c4070 con 0x55782e6106b0 2023-07-14T16:20:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56e840ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782e7c4230 con 0x55782e7b4840 2023-07-14T16:20:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56e840ab80 10 client.0 _ll_register_callbacks cb 0x7f56c8009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56ced9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7c39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43594/0 (socket says 172.21.15.62:43594) 2023-07-14T16:20:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56ced9d700 1 -- 172.21.15.62:0/2455596988 learned_addr learned my addr 172.21.15.62:0/2455596988 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56cdd9b700 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b4840 0x55782e7c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.522+0000 7f56ce59c700 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7c0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56ced9d700 1 -- 172.21.15.62:0/2455596988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b4840 msgr2=0x55782e7c09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56ced9d700 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b4840 0x55782e7c09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56ced9d700 1 -- 172.21.15.62:0/2455596988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 msgr2=0x55782e7c0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56ced9d700 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7c0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56ced9d700 1 -- 172.21.15.62:0/2455596988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d01a02e0 con 0x55782e7b56e0 2023-07-14T16:20:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56ced9d700 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7c39c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f56c8017db0 tx=0x7f56c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b5a918440b7b24 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56b5ffb700 1 -- 172.21.15.62:0/2455596988 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c800e040 con 0x55782e7b56e0 2023-07-14T16:20:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56e840ab80 1 -- 172.21.15.62:0/2455596988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56c409df70 con 0x55782e7b56e0 2023-07-14T16:20:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.523+0000 7f56b5ffb700 1 -- 172.21.15.62:0/2455596988 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f56c801e040 con 0x55782e7b56e0 2023-07-14T16:20:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56b5ffb700 1 -- 172.21.15.62:0/2455596988 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c8017de0 con 0x55782e7b56e0 2023-07-14T16:20:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56e840ab80 10 client.16860 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56e840ab80 1 -- 172.21.15.62:0/2455596988 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55782e55a1f0 con 0x55782e7b56e0 2023-07-14T16:20:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56b5ffb700 1 -- 172.21.15.62:0/2455596988 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56c409df70 con 0x55782e7b56e0 2023-07-14T16:20:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56b5ffb700 1 -- 172.21.15.62:0/2455596988 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f56c8036070 con 0x55782e7b56e0 2023-07-14T16:20:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56e840ab80 10 client.16860 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56e840ab80 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56e840ab80 20 client.16860 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56e840ab80 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55782e81da20 0x55782e81fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56e840ab80 4 client.16860 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:20:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.524+0000 7f56e840ab80 1 -- 172.21.15.62:0/2455596988 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55782e6c0ff0 con 0x55782e81da20 2023-07-14T16:20:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.525+0000 7f56ce59c700 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55782e81da20 0x55782e81fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.525+0000 7f56ce59c700 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55782e81da20 0x55782e81fe30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a84161a617ea212 server_cookie=9b6678ab5b610f05 in_seq=0 out_seq=0 2023-07-14T16:20:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.525+0000 7f56b5ffb700 10 client.16860 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.534+0000 7f56b5ffb700 1 -- 172.21.15.62:0/2455596988 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55782e6c0ff0 con 0x55782e81da20 2023-07-14T16:20:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.534+0000 7f56b5ffb700 10 client.16860 handle_command_reply: tid=0 2023-07-14T16:20:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 client.16860 shutdown 2023-07-14T16:20:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 -- 172.21.15.62:0/2455596988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 msgr2=0x55782e7c39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7c39c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f56c8017db0 tx=0x7f56c8004ad0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 -- 172.21.15.62:0/2455596988 shutdown_connections 2023-07-14T16:20:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55782e7b56e0 0x55782e7c39c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55782e81da20 0x55782e81fe30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55782e6106b0 0x55782e7c0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56b5ffb700 0 client.16860 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 --2- 172.21.15.62:0/2455596988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55782e7b4840 0x55782e7c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 -- 172.21.15.62:0/2455596988 >> 172.21.15.62:0/2455596988 conn(0x55782e7a9be0 msgr2=0x55782e7bc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 -- 172.21.15.62:0/2455596988 shutdown_connections 2023-07-14T16:20:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 1 -- 172.21.15.62:0/2455596988 wait complete. 2023-07-14T16:20:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.535+0000 7f56e840ab80 20 client.16860 trim_cache size 0 max 16384 2023-07-14T16:20:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:20:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:20:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:20:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:20:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 -- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f56b80a11d0 msgr2=0x7f56b80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f56b80a11d0 0x7f56b80a3680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55782e7b4060 tx=0x7f56c8004b40 comp rx=0 tx=0).stop 2023-07-14T16:20:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 -- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0100cf0 msgr2=0x7f56d0198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0100cf0 0x7f56d0198030 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f56c400e9a0 tx=0x7f56c400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 -- 172.21.15.62:0/4240564861 shutdown_connections 2023-07-14T16:20:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56d0101680 0x7f56d0198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f56b80a11d0 0x7f56b80a3680 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d00ffb00 0x7f56d0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 --2- 172.21.15.62:0/4240564861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0100cf0 0x7f56d0198030 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 -- 172.21.15.62:0/4240564861 >> 172.21.15.62:0/4240564861 conn(0x7f56d00fb0e0 msgr2=0x7f56d01066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 -- 172.21.15.62:0/4240564861 shutdown_connections 2023-07-14T16:20:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:27.536+0000 7f56d49ff700 1 -- 172.21.15.62:0/4240564861 wait complete. 2023-07-14T16:20:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:28 smithi164 ceph-mon[97427]: pgmap v502: 97 pgs: 97 active+clean; 2.6 GiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:20:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:20:29.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:20:29.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:28 smithi062 ceph-mon[95543]: pgmap v502: 97 pgs: 97 active+clean; 2.6 GiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:20:29.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:20:29.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:20:29.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:28 smithi081 ceph-mon[127795]: pgmap v502: 97 pgs: 97 active+clean; 2.6 GiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:20:29.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:20:29.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:20:31.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:30 smithi164 ceph-mon[97427]: pgmap v503: 97 pgs: 97 active+clean; 2.6 GiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:20:31.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:30 smithi062 ceph-mon[95543]: pgmap v503: 97 pgs: 97 active+clean; 2.6 GiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:20:31.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:30 smithi081 ceph-mon[127795]: pgmap v503: 97 pgs: 97 active+clean; 2.6 GiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:20:33.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:32 smithi164 ceph-mon[97427]: pgmap v504: 97 pgs: 97 active+clean; 2.6 GiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:20:33.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:32 smithi062 ceph-mon[95543]: pgmap v504: 97 pgs: 97 active+clean; 2.6 GiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:20:33.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:32 smithi081 ceph-mon[127795]: pgmap v504: 97 pgs: 97 active+clean; 2.6 GiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:20:35.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:34 smithi164 ceph-mon[97427]: pgmap v505: 97 pgs: 97 active+clean; 2.7 GiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:20:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:34 smithi062 ceph-mon[95543]: pgmap v505: 97 pgs: 97 active+clean; 2.7 GiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:20:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:34 smithi081 ceph-mon[127795]: pgmap v505: 97 pgs: 97 active+clean; 2.7 GiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:20:37.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:36 smithi164 ceph-mon[97427]: pgmap v506: 97 pgs: 97 active+clean; 2.7 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 17 op/s 2023-07-14T16:20:37.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:36 smithi062 ceph-mon[95543]: pgmap v506: 97 pgs: 97 active+clean; 2.7 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 17 op/s 2023-07-14T16:20:37.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:36 smithi081 ceph-mon[127795]: pgmap v506: 97 pgs: 97 active+clean; 2.7 GiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 17 op/s 2023-07-14T16:20:39.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:38 smithi164 ceph-mon[97427]: pgmap v507: 97 pgs: 97 active+clean; 2.7 GiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:39.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:38 smithi062 ceph-mon[95543]: pgmap v507: 97 pgs: 97 active+clean; 2.7 GiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:39.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:38 smithi081 ceph-mon[127795]: pgmap v507: 97 pgs: 97 active+clean; 2.7 GiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 -- 172.21.15.62:0/2003247723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec068620 msgr2=0x7ff8ec1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/2003247723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec068620 0x7ff8ec1026b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009990 tx=0x7ff8dc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 -- 172.21.15.62:0/2003247723 shutdown_connections 2023-07-14T16:20:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/2003247723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8ec1056e0 0x7ff8ec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/2003247723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec068620 0x7ff8ec1026b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/2003247723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec102bf0 0x7ff8ec105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 -- 172.21.15.62:0/2003247723 >> 172.21.15.62:0/2003247723 conn(0x7ff8ec0fa000 msgr2=0x7ff8ec0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 -- 172.21.15.62:0/2003247723 shutdown_connections 2023-07-14T16:20:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.699+0000 7ff8f1bff700 1 -- 172.21.15.62:0/2003247723 wait complete. 2023-07-14T16:20:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.700+0000 7ff8f1bff700 1 Processor -- start 2023-07-14T16:20:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.700+0000 7ff8f1bff700 1 -- start start 2023-07-14T16:20:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8f1bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8ec068620 0x7ff8ec197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8f1bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec102bf0 0x7ff8ec197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8f1bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec1056e0 0x7ff8ec198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8f1bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec199450 con 0x7ff8ec1056e0 2023-07-14T16:20:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8f1bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec1995c0 con 0x7ff8ec068620 2023-07-14T16:20:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8f1bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec199780 con 0x7ff8ec102bf0 2023-07-14T16:20:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8ebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec1056e0 0x7ff8ec198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8ebfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec1056e0 0x7ff8ec198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48134/0 (socket says 172.21.15.62:48134) 2023-07-14T16:20:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8ebfff700 1 -- 172.21.15.62:0/3570558863 learned_addr learned my addr 172.21.15.62:0/3570558863 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8eb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8ec068620 0x7ff8ec197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec102bf0 0x7ff8ec197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8ebfff700 1 -- 172.21.15.62:0/3570558863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec102bf0 msgr2=0x7ff8ec197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8ebfff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec102bf0 0x7ff8ec197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8ebfff700 1 -- 172.21.15.62:0/3570558863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8ec068620 msgr2=0x7ff8ec197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8ebfff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8ec068620 0x7ff8ec197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8ebfff700 1 -- 172.21.15.62:0/3570558863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc0095f0 con 0x7ff8ec1056e0 2023-07-14T16:20:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec102bf0 0x7ff8ec197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.701+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8ec068620 0x7ff8ec197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.702+0000 7ff8ebfff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec1056e0 0x7ff8ec198df0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400ef30 tx=0x7ff8e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.702+0000 7ff8e8ff9700 1 -- 172.21.15.62:0/3570558863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4018070 con 0x7ff8ec1056e0 2023-07-14T16:20:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.702+0000 7ff8f1bff700 1 -- 172.21.15.62:0/3570558863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec199a00 con 0x7ff8ec1056e0 2023-07-14T16:20:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.702+0000 7ff8e8ff9700 1 -- 172.21.15.62:0/3570558863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8e400f040 con 0x7ff8ec1056e0 2023-07-14T16:20:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.702+0000 7ff8e8ff9700 1 -- 172.21.15.62:0/3570558863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4007b80 con 0x7ff8ec1056e0 2023-07-14T16:20:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.702+0000 7ff8f1bff700 1 -- 172.21.15.62:0/3570558863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec1a0220 con 0x7ff8ec1056e0 2023-07-14T16:20:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.706+0000 7ff8e8ff9700 1 -- 172.21.15.62:0/3570558863 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff8e400c9f0 con 0x7ff8ec1056e0 2023-07-14T16:20:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.707+0000 7ff905441b80 1 Processor -- start 2023-07-14T16:20:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.707+0000 7ff905441b80 1 -- start start 2023-07-14T16:20:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff905441b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af17300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff905441b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff905441b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af18b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff905441b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68ae1cc50 con 0x55c68af17840 2023-07-14T16:20:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff905441b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68ae6f9e0 con 0x55c68af186e0 2023-07-14T16:20:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff905441b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68ae68e70 con 0x55c68ad736b0 2023-07-14T16:20:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff8eb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff8eb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af181a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48144/0 (socket says 172.21.15.62:48144) 2023-07-14T16:20:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/1925681901 learned_addr learned my addr 172.21.15.62:0/1925681901 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.708+0000 7ff8e8ff9700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff8d40a1150 0x7ff8d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af18b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8e8ff9700 1 -- 172.21.15.62:0/3570558863 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff8e40ccb50 con 0x7ff8ec1056e0 2023-07-14T16:20:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af17300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/1925681901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 msgr2=0x55c68af17300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af17300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/1925681901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 msgr2=0x55c68af18b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af18b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/1925681901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c68ac96d80 con 0x55c68af17840 2023-07-14T16:20:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af17300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:20:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8ebfff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff8d40a1150 0x7ff8d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.709+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af181a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec068340 tx=0x7ff8dc004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb26c74591318b76 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.710+0000 7ff8ebfff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff8d40a1150 0x7ff8d40a3600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004d90 tx=0x7ff8e4004ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.710+0000 7ff8d1ffb700 1 -- 172.21.15.62:0/1925681901 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc01d070 con 0x55c68af17840 2023-07-14T16:20:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.710+0000 7ff8d1ffb700 1 -- 172.21.15.62:0/1925681901 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8dc01e040 con 0x55c68af17840 2023-07-14T16:20:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.710+0000 7ff8d1ffb700 1 -- 172.21.15.62:0/1925681901 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00bcd0 con 0x55c68af17840 2023-07-14T16:20:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.710+0000 7ff905441b80 1 -- 172.21.15.62:0/1925681901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 msgr2=0x55c68af181a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.710+0000 7ff905441b80 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af181a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec068340 tx=0x7ff8dc004000 comp rx=0 tx=0).stop 2023-07-14T16:20:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.711+0000 7ff905441b80 1 -- 172.21.15.62:0/1925681901 shutdown_connections 2023-07-14T16:20:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.711+0000 7ff905441b80 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af18b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.711+0000 7ff905441b80 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af17300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.711+0000 7ff905441b80 1 --2- 172.21.15.62:0/1925681901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af181a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.711+0000 7ff905441b80 1 -- 172.21.15.62:0/1925681901 >> 172.21.15.62:0/1925681901 conn(0x55c68af0cbe0 msgr2=0x55c68af11cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.711+0000 7ff905441b80 1 -- 172.21.15.62:0/1925681901 shutdown_connections 2023-07-14T16:20:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.711+0000 7ff905441b80 1 -- 172.21.15.62:0/1925681901 wait complete. 2023-07-14T16:20:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.711+0000 7ff905441b80 1 -- start start 2023-07-14T16:20:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.712+0000 7ff905441b80 1 Processor -- start 2023-07-14T16:20:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.712+0000 7ff905441b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af20810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.712+0000 7ff905441b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af20d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.712+0000 7ff905441b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af27d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.712+0000 7ff905441b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68ae08a60 con 0x55c68af17840 2023-07-14T16:20:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.712+0000 7ff905441b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68ae08760 con 0x55c68af186e0 2023-07-14T16:20:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.712+0000 7ff905441b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68acec8c0 con 0x55c68ad736b0 2023-07-14T16:20:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.712+0000 7ff905441b80 10 client.0 _ll_register_callbacks cb 0x55c68ae06700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af20d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af20d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48150/0 (socket says 172.21.15.62:48150) 2023-07-14T16:20:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/3145741156 learned_addr learned my addr 172.21.15.62:0/3145741156 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af27d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af20810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/3145741156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 msgr2=0x55c68af20810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af20810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/3145741156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 msgr2=0x55c68af27d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af27d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/3145741156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec108850 con 0x55c68af17840 2023-07-14T16:20:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af20d50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc000c00 tx=0x7ff8dc00b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44096b8bfcc306db server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.713+0000 7ff8caffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.714+0000 7ff8caffd700 1 -- 172.21.15.62:0/3145741156 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc01d040 con 0x55c68af17840 2023-07-14T16:20:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.714+0000 7ff8caffd700 1 -- 172.21.15.62:0/3145741156 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff8dc005520 con 0x55c68af17840 2023-07-14T16:20:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.714+0000 7ff905441b80 1 -- 172.21.15.62:0/3145741156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c68adc2490 con 0x55c68af17840 2023-07-14T16:20:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.714+0000 7ff8caffd700 1 -- 172.21.15.62:0/3145741156 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8dc004500 con 0x55c68af17840 2023-07-14T16:20:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.714+0000 7ff905441b80 10 client.16872 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.714+0000 7ff905441b80 1 -- 172.21.15.62:0/3145741156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c68ac7fc00 con 0x55c68af17840 2023-07-14T16:20:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.714+0000 7ff8caffd700 1 -- 172.21.15.62:0/3145741156 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c68adc2490 con 0x55c68af17840 2023-07-14T16:20:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.715+0000 7ff8caffd700 1 -- 172.21.15.62:0/3145741156 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff8dc00f580 con 0x55c68af17840 2023-07-14T16:20:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.715+0000 7ff905441b80 10 client.16872 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.715+0000 7ff905441b80 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.715+0000 7ff905441b80 20 client.16872 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.715+0000 7ff905441b80 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c68af2c080 0x55c68af2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.715+0000 7ff905441b80 4 client.16872 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.715+0000 7ff905441b80 1 -- 172.21.15.62:0/3145741156 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c68ae61e50 con 0x55c68af2c080 2023-07-14T16:20:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.715+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c68af2c080 0x55c68af2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.716+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c68af2c080 0x55c68af2e460 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35398439514a5f04 server_cookie=ed76b92d6517955e in_seq=0 out_seq=0 2023-07-14T16:20:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.716+0000 7ff8caffd700 10 client.16872 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff8caffd700 1 -- 172.21.15.62:0/3145741156 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c68ae61e50 con 0x55c68af2c080 2023-07-14T16:20:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff8caffd700 10 client.16872 handle_command_reply: tid=0 2023-07-14T16:20:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 client.16872 shutdown 2023-07-14T16:20:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 -- 172.21.15.62:0/3145741156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 msgr2=0x55c68af20d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af20d50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc000c00 tx=0x7ff8dc00b770 comp rx=0 tx=0).stop 2023-07-14T16:20:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 -- 172.21.15.62:0/3145741156 shutdown_connections 2023-07-14T16:20:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c68af2c080 0x55c68af2e460 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af27d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af20810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 --2- 172.21.15.62:0/3145741156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af20d50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 -- 172.21.15.62:0/3145741156 >> 172.21.15.62:0/3145741156 conn(0x55c68af0cbe0 msgr2=0x55c68af114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff8caffd700 0 client.16872 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 -- 172.21.15.62:0/3145741156 shutdown_connections 2023-07-14T16:20:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.717+0000 7ff905441b80 1 -- 172.21.15.62:0/3145741156 wait complete. 2023-07-14T16:20:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.718+0000 7ff905441b80 20 client.16872 trim_cache size 0 max 16384 2023-07-14T16:20:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.738+0000 7ff905441b80 1 Processor -- start 2023-07-14T16:20:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.738+0000 7ff905441b80 1 -- start start 2023-07-14T16:20:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.738+0000 7ff905441b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af1fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff905441b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af27d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff905441b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af1ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff905441b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68aedd060 con 0x55c68af17840 2023-07-14T16:20:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff905441b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68af1fcc0 con 0x55c68af186e0 2023-07-14T16:20:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff905441b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68af0ed00 con 0x55c68ad736b0 2023-07-14T16:20:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af27d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af27d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48152/0 (socket says 172.21.15.62:48152) 2023-07-14T16:20:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/307265776 learned_addr learned my addr 172.21.15.62:0/307265776 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af1ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af1fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/307265776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 msgr2=0x55c68af1fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af1fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/307265776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 msgr2=0x55c68af1ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af1ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/307265776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c68acbd1f0 con 0x55c68af17840 2023-07-14T16:20:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.739+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af1ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.740+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af27d50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc03a040 tx=0x7ff8dc004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e56c74e4d14819e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.740+0000 7ff8d27fc700 1 -- 172.21.15.62:0/307265776 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc01d040 con 0x55c68af17840 2023-07-14T16:20:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.740+0000 7ff8d27fc700 1 -- 172.21.15.62:0/307265776 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8dc00b2c0 con 0x55c68af17840 2023-07-14T16:20:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.740+0000 7ff8d27fc700 1 -- 172.21.15.62:0/307265776 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc017650 con 0x55c68af17840 2023-07-14T16:20:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 -- 172.21.15.62:0/307265776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 msgr2=0x55c68af27d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af27d50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc03a040 tx=0x7ff8dc004000 comp rx=0 tx=0).stop 2023-07-14T16:20:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 -- 172.21.15.62:0/307265776 shutdown_connections 2023-07-14T16:20:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af186e0 0x55c68af1ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af1fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 --2- 172.21.15.62:0/307265776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af17840 0x55c68af27d50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 -- 172.21.15.62:0/307265776 >> 172.21.15.62:0/307265776 conn(0x55c68af0cbe0 msgr2=0x55c68af32dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 -- 172.21.15.62:0/307265776 shutdown_connections 2023-07-14T16:20:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 -- 172.21.15.62:0/307265776 wait complete. 2023-07-14T16:20:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.741+0000 7ff905441b80 1 -- start start 2023-07-14T16:20:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.742+0000 7ff905441b80 1 Processor -- start 2023-07-14T16:20:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.742+0000 7ff905441b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af23060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.742+0000 7ff905441b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af17840 0x55c68af235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.742+0000 7ff905441b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af186e0 0x55c68af265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.742+0000 7ff905441b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68af26b00 con 0x55c68af186e0 2023-07-14T16:20:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.742+0000 7ff905441b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68af26c70 con 0x55c68af17840 2023-07-14T16:20:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.742+0000 7ff905441b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68af26e30 con 0x55c68ad736b0 2023-07-14T16:20:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.742+0000 7ff905441b80 10 client.0 _ll_register_callbacks cb 0x7ff8dc005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eaffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af23060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eaffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af23060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38834/0 (socket says 172.21.15.62:38834) 2023-07-14T16:20:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eaffd700 1 -- 172.21.15.62:0/3975245043 learned_addr learned my addr 172.21.15.62:0/3975245043 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af186e0 0x55c68af265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af17840 0x55c68af235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/3975245043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 msgr2=0x55c68af23060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af23060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/3975245043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af186e0 msgr2=0x55c68af265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af186e0 0x55c68af265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eb7fe700 1 -- 172.21.15.62:0/3975245043 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c68ac7fc00 con 0x55c68af17840 2023-07-14T16:20:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.743+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af23060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.744+0000 7ff8eb7fe700 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af17840 0x55c68af235a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00f960 tx=0x7ff8dc005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13785925b6688c9c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.744+0000 7ff8caffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.744+0000 7ff8caffd700 1 -- 172.21.15.62:0/3975245043 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc045070 con 0x55c68af17840 2023-07-14T16:20:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.744+0000 7ff8caffd700 1 -- 172.21.15.62:0/3975245043 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff8dc000e40 con 0x55c68af17840 2023-07-14T16:20:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.744+0000 7ff905441b80 1 -- 172.21.15.62:0/3975245043 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8dc01e040 con 0x55c68af17840 2023-07-14T16:20:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.744+0000 7ff8caffd700 1 -- 172.21.15.62:0/3975245043 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8dc01d040 con 0x55c68af17840 2023-07-14T16:20:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.744+0000 7ff905441b80 10 client.26588 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.744+0000 7ff905441b80 1 -- 172.21.15.62:0/3975245043 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c68ac95250 con 0x55c68af17840 2023-07-14T16:20:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.745+0000 7ff8caffd700 1 -- 172.21.15.62:0/3975245043 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8dc050020 con 0x55c68af17840 2023-07-14T16:20:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.745+0000 7ff8caffd700 1 -- 172.21.15.62:0/3975245043 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff8dc048020 con 0x55c68af17840 2023-07-14T16:20:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.745+0000 7ff905441b80 10 client.26588 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.745+0000 7ff905441b80 10 client.26588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.745+0000 7ff905441b80 20 client.26588 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.745+0000 7ff905441b80 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c68af80030 0x55c68af82440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.745+0000 7ff905441b80 4 client.26588 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:20:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.745+0000 7ff905441b80 1 -- 172.21.15.62:0/3975245043 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c68ae23ff0 con 0x55c68af80030 2023-07-14T16:20:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.746+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c68af80030 0x55c68af82440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.746+0000 7ff8eaffd700 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c68af80030 0x55c68af82440 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc039c7852204617 server_cookie=44f40b875072b759 in_seq=0 out_seq=0 2023-07-14T16:20:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.746+0000 7ff8caffd700 10 client.26588 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff8caffd700 1 -- 172.21.15.62:0/3975245043 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c68ae23ff0 con 0x55c68af80030 2023-07-14T16:20:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff8caffd700 10 client.26588 handle_command_reply: tid=0 2023-07-14T16:20:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 client.26588 shutdown 2023-07-14T16:20:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 -- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af17840 msgr2=0x55c68af235a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af17840 0x55c68af235a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00f960 tx=0x7ff8dc005810 comp rx=0 tx=0).stop 2023-07-14T16:20:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 -- 172.21.15.62:0/3975245043 shutdown_connections 2023-07-14T16:20:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c68af186e0 0x55c68af265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c68ad736b0 0x55c68af23060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c68af80030 0x55c68af82440 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 --2- 172.21.15.62:0/3975245043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c68af17840 0x55c68af235a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff8caffd700 0 client.26588 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 -- 172.21.15.62:0/3975245043 >> 172.21.15.62:0/3975245043 conn(0x55c68af0cbe0 msgr2=0x55c68af1ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 -- 172.21.15.62:0/3975245043 shutdown_connections 2023-07-14T16:20:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 1 -- 172.21.15.62:0/3975245043 wait complete. 2023-07-14T16:20:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.756+0000 7ff905441b80 20 client.26588 trim_cache size 0 max 16384 2023-07-14T16:20:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:20:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:20:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:20:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:20:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 -- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff8d40a1150 msgr2=0x7ff8d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff8d40a1150 0x7ff8d40a3600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004d90 tx=0x7ff8e4004ce0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 -- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec1056e0 msgr2=0x7ff8ec198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec1056e0 0x7ff8ec198df0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400ef30 tx=0x7ff8e400c3a0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 -- 172.21.15.62:0/3570558863 shutdown_connections 2023-07-14T16:20:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ec1056e0 0x7ff8ec198df0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff8d40a1150 0x7ff8d40a3600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8ec068620 0x7ff8ec197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 --2- 172.21.15.62:0/3570558863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8ec102bf0 0x7ff8ec197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 -- 172.21.15.62:0/3570558863 >> 172.21.15.62:0/3570558863 conn(0x7ff8ec0fa000 msgr2=0x7ff8ec0fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 -- 172.21.15.62:0/3570558863 shutdown_connections 2023-07-14T16:20:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:40.757+0000 7ff8f1bff700 1 -- 172.21.15.62:0/3570558863 wait complete. 2023-07-14T16:20:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:40 smithi164 ceph-mon[97427]: pgmap v508: 97 pgs: 97 active+clean; 2.7 GiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:41.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:40 smithi062 ceph-mon[95543]: pgmap v508: 97 pgs: 97 active+clean; 2.7 GiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:40 smithi081 ceph-mon[127795]: pgmap v508: 97 pgs: 97 active+clean; 2.7 GiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:42 smithi164 ceph-mon[97427]: pgmap v509: 97 pgs: 97 active+clean; 2.8 GiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:43.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:42 smithi062 ceph-mon[95543]: pgmap v509: 97 pgs: 97 active+clean; 2.8 GiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:43.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:42 smithi081 ceph-mon[127795]: pgmap v509: 97 pgs: 97 active+clean; 2.8 GiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:44 smithi164 ceph-mon[97427]: pgmap v510: 97 pgs: 97 active+clean; 2.8 GiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:20:45.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:44 smithi062 ceph-mon[95543]: pgmap v510: 97 pgs: 97 active+clean; 2.8 GiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:20:45.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:44 smithi081 ceph-mon[127795]: pgmap v510: 97 pgs: 97 active+clean; 2.8 GiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:20:47.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:46 smithi164 ceph-mon[97427]: pgmap v511: 97 pgs: 97 active+clean; 2.8 GiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:47.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:46 smithi062 ceph-mon[95543]: pgmap v511: 97 pgs: 97 active+clean; 2.8 GiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:47.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:46 smithi081 ceph-mon[127795]: pgmap v511: 97 pgs: 97 active+clean; 2.8 GiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:20:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:48 smithi164 ceph-mon[97427]: pgmap v512: 97 pgs: 97 active+clean; 2.9 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:49.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:48 smithi062 ceph-mon[95543]: pgmap v512: 97 pgs: 97 active+clean; 2.9 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:49.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:48 smithi081 ceph-mon[127795]: pgmap v512: 97 pgs: 97 active+clean; 2.9 GiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:20:50.079 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:20:50.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 -- 172.21.15.62:0/893654287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b201057a0 msgr2=0x7f5b20107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 --2- 172.21.15.62:0/893654287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b201057a0 0x7f5b20107c00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b0a0 tx=0x7f5b1800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 -- 172.21.15.62:0/893654287 shutdown_connections 2023-07-14T16:20:50.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 --2- 172.21.15.62:0/893654287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b201057a0 0x7f5b20107c00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 --2- 172.21.15.62:0/893654287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b20100560 0x7f5b20102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 --2- 172.21.15.62:0/893654287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b20102e80 0x7f5b20105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 -- 172.21.15.62:0/893654287 >> 172.21.15.62:0/893654287 conn(0x7f5b200fa000 msgr2=0x7f5b200fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:50.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 -- 172.21.15.62:0/893654287 shutdown_connections 2023-07-14T16:20:50.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.342+0000 7f5b253ff700 1 -- 172.21.15.62:0/893654287 wait complete. 2023-07-14T16:20:50.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.343+0000 7f5b253ff700 1 Processor -- start 2023-07-14T16:20:50.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.343+0000 7f5b253ff700 1 -- start start 2023-07-14T16:20:50.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.343+0000 7f5b253ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b20100560 0x7f5b20197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.343+0000 7f5b253ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b20102e80 0x7f5b20197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b253ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b201057a0 0x7f5b2019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b253ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019f5e0 con 0x7f5b20102e80 2023-07-14T16:20:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b253ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019f720 con 0x7f5b201057a0 2023-07-14T16:20:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b253ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019f8b0 con 0x7f5b20100560 2023-07-14T16:20:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b20102e80 0x7f5b20197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b20102e80 0x7f5b20197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55476/0 (socket says 172.21.15.62:55476) 2023-07-14T16:20:50.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/2417052690 learned_addr learned my addr 172.21.15.62:0/2417052690 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1effd700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b20100560 0x7f5b20197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b201057a0 0x7f5b2019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/2417052690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b20100560 msgr2=0x7f5b20197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b20100560 0x7f5b20197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/2417052690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b201057a0 msgr2=0x7f5b2019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b201057a0 0x7f5b2019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.344+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/2417052690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18009d00 con 0x7f5b20102e80 2023-07-14T16:20:50.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.345+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b20102e80 0x7f5b20197fa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400e9a0 tx=0x7f5b1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:50.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.345+0000 7f5b07fff700 1 -- 172.21.15.62:0/2417052690 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14009790 con 0x7f5b20102e80 2023-07-14T16:20:50.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.345+0000 7f5b07fff700 1 -- 172.21.15.62:0/2417052690 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b14010430 con 0x7f5b20102e80 2023-07-14T16:20:50.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.345+0000 7f5b253ff700 1 -- 172.21.15.62:0/2417052690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2019fb90 con 0x7f5b20102e80 2023-07-14T16:20:50.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.345+0000 7f5b07fff700 1 -- 172.21.15.62:0/2417052690 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400cea0 con 0x7f5b20102e80 2023-07-14T16:20:50.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.345+0000 7f5b253ff700 1 -- 172.21.15.62:0/2417052690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a00f0 con 0x7f5b20102e80 2023-07-14T16:20:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.349+0000 7f5b07fff700 1 -- 172.21.15.62:0/2417052690 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b140105a0 con 0x7f5b20102e80 2023-07-14T16:20:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.350+0000 7f5b38d7ab80 1 Processor -- start 2023-07-14T16:20:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b38d7ab80 1 -- start start 2023-07-14T16:20:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e7e20 0x5602813e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b07fff700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b080a11d0 0x7f5b080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602812eccb0 con 0x5602813e8c70 2023-07-14T16:20:50.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028133fa40 con 0x5602813e7e20 2023-07-14T16:20:50.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560281338ed0 con 0x5602813e74d0 2023-07-14T16:20:50.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.351+0000 7f5b07fff700 1 -- 172.21.15.62:0/2417052690 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b14014070 con 0x7f5b20102e80 2023-07-14T16:20:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e7e20 0x5602813e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e7e20 0x5602813e86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43078/0 (socket says 172.21.15.62:43078) 2023-07-14T16:20:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 -- 172.21.15.62:0/1413416978 learned_addr learned my addr 172.21.15.62:0/1413416978 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b080a11d0 0x7f5b080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 -- 172.21.15.62:0/1413416978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 msgr2=0x5602813e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 -- 172.21.15.62:0/1413416978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 msgr2=0x5602813e90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813e90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 -- 172.21.15.62:0/1413416978 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560281166de0 con 0x5602813e7e20 2023-07-14T16:20:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.352+0000 7f5b1effd700 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e7e20 0x5602813e86e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009d80 tx=0x7f5b10011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=920c617d0d79867c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.353+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b080a11d0 0x7f5b080a3680 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b780 tx=0x7f5b18006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.353+0000 7f5b057fa700 1 -- 172.21.15.62:0/1413416978 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10004230 con 0x5602813e7e20 2023-07-14T16:20:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.353+0000 7f5b057fa700 1 -- 172.21.15.62:0/1413416978 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b10004390 con 0x5602813e7e20 2023-07-14T16:20:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.353+0000 7f5b057fa700 1 -- 172.21.15.62:0/1413416978 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100197e0 con 0x5602813e7e20 2023-07-14T16:20:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1413416978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e7e20 msgr2=0x5602813e86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e7e20 0x5602813e86e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009d80 tx=0x7f5b10011720 comp rx=0 tx=0).stop 2023-07-14T16:20:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1413416978 shutdown_connections 2023-07-14T16:20:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1413416978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e7e20 0x5602813e86e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1413416978 >> 172.21.15.62:0/1413416978 conn(0x5602813dcda0 msgr2=0x5602813e21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1413416978 shutdown_connections 2023-07-14T16:20:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1413416978 wait complete. 2023-07-14T16:20:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.354+0000 7f5b38d7ab80 1 -- start start 2023-07-14T16:20:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.355+0000 7f5b38d7ab80 1 Processor -- start 2023-07-14T16:20:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.355+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.355+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.355+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.355+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602812d8ac0 con 0x5602813e7e20 2023-07-14T16:20:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.355+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602812d87c0 con 0x5602813e8c70 2023-07-14T16:20:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.355+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602811bc920 con 0x5602813e74d0 2023-07-14T16:20:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.355+0000 7f5b38d7ab80 10 client.0 _ll_register_callbacks cb 0x5602812d6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55490/0 (socket says 172.21.15.62:55490) 2023-07-14T16:20:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/3749424095 learned_addr learned my addr 172.21.15.62:0/3749424095 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1effd700 1 -- 172.21.15.62:0/3749424095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 msgr2=0x5602813f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1effd700 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1effd700 1 -- 172.21.15.62:0/3749424095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 msgr2=0x5602813f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1effd700 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.356+0000 7f5b1effd700 1 -- 172.21.15.62:0/3749424095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b100099e0 con 0x5602813e74d0 2023-07-14T16:20:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.357+0000 7f5b1effd700 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813f10c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b10006010 tx=0x7f5b10003930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb120e9250d6b3e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.357+0000 7f5b02ff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:20:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.357+0000 7f5b02ff5700 1 -- 172.21.15.62:0/3749424095 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10005450 con 0x5602813e74d0 2023-07-14T16:20:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.357+0000 7f5b02ff5700 1 -- 172.21.15.62:0/3749424095 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b10004500 con 0x5602813e74d0 2023-07-14T16:20:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.357+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/3749424095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602812924f0 con 0x5602813e74d0 2023-07-14T16:20:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.357+0000 7f5b02ff5700 1 -- 172.21.15.62:0/3749424095 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b10030030 con 0x5602813e74d0 2023-07-14T16:20:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.357+0000 7f5b38d7ab80 10 client.26494 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.357+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/3749424095 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56028114fc60 con 0x5602813e74d0 2023-07-14T16:20:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b02ff5700 1 -- 172.21.15.62:0/3749424095 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5602812924f0 con 0x5602813e74d0 2023-07-14T16:20:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b02ff5700 1 -- 172.21.15.62:0/3749424095 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5b10028020 con 0x5602813e74d0 2023-07-14T16:20:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b38d7ab80 10 client.26494 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b38d7ab80 10 client.26494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b38d7ab80 20 client.26494 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5602813fce20 0x5602813ff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b38d7ab80 4 client.26494 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/3749424095 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560281331eb0 con 0x5602813fce20 2023-07-14T16:20:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.358+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5602813fce20 0x5602813ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.359+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5602813fce20 0x5602813ff230 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929cc7809b6cdbd server_cookie=fda6cdc5ad87a38e in_seq=0 out_seq=0 2023-07-14T16:20:50.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.359+0000 7f5b02ff5700 10 client.26494 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b02ff5700 1 -- 172.21.15.62:0/3749424095 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560281331eb0 con 0x5602813fce20 2023-07-14T16:20:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b02ff5700 10 client.26494 handle_command_reply: tid=0 2023-07-14T16:20:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 client.26494 shutdown 2023-07-14T16:20:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/3749424095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 msgr2=0x5602813f10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813f10c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b10006010 tx=0x7f5b10003930 comp rx=0 tx=0).stop 2023-07-14T16:20:50.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/3749424095 shutdown_connections 2023-07-14T16:20:50.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5602813fce20 0x5602813ff230 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813f10c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/3749424095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b02ff5700 0 client.26494 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:50.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/3749424095 >> 172.21.15.62:0/3749424095 conn(0x5602813dcda0 msgr2=0x5602813e18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:50.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/3749424095 shutdown_connections 2023-07-14T16:20:50.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.360+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/3749424095 wait complete. 2023-07-14T16:20:50.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.361+0000 7f5b38d7ab80 20 client.26494 trim_cache size 0 max 16384 2023-07-14T16:20:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.383+0000 7f5b38d7ab80 1 Processor -- start 2023-07-14T16:20:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b38d7ab80 1 -- start start 2023-07-14T16:20:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813ef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602813f0570 con 0x5602813e7e20 2023-07-14T16:20:50.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602813e2ed0 con 0x5602813e8c70 2023-07-14T16:20:50.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602813f2240 con 0x5602813e74d0 2023-07-14T16:20:50.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b1e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b1e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813ef500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55496/0 (socket says 172.21.15.62:55496) 2023-07-14T16:20:50.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/2226161719 learned_addr learned my addr 172.21.15.62:0/2226161719 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:50.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.384+0000 7f5b1effd700 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/2226161719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 msgr2=0x5602813eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/2226161719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 msgr2=0x5602813f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:50.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b1e7fc700 1 -- 172.21.15.62:0/2226161719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56028118d250 con 0x5602813e7e20 2023-07-14T16:20:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:20:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813ef500 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5b14096380 tx=0x7f5b140961c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9afd760a847a54 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b05ffb700 1 -- 172.21.15.62:0/2226161719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14098060 con 0x5602813e7e20 2023-07-14T16:20:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b05ffb700 1 -- 172.21.15.62:0/2226161719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b14097e00 con 0x5602813e7e20 2023-07-14T16:20:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.385+0000 7f5b05ffb700 1 -- 172.21.15.62:0/2226161719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b140a22d0 con 0x5602813e7e20 2023-07-14T16:20:50.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/2226161719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 msgr2=0x5602813ef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813ef500 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5b14096380 tx=0x7f5b140961c0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/2226161719 shutdown_connections 2023-07-14T16:20:50.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e8c70 0x5602813f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e74d0 0x5602813eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/2226161719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e7e20 0x5602813ef500 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/2226161719 >> 172.21.15.62:0/2226161719 conn(0x5602813dcda0 msgr2=0x560281403b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:50.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/2226161719 shutdown_connections 2023-07-14T16:20:50.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.386+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/2226161719 wait complete. 2023-07-14T16:20:50.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 1 -- start start 2023-07-14T16:20:50.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 1 Processor -- start 2023-07-14T16:20:50.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e74d0 0x5602813f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e7e20 0x5602813f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602813f54a0 con 0x5602813e8c70 2023-07-14T16:20:50.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602813f5610 con 0x5602813e74d0 2023-07-14T16:20:50.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602813f57d0 con 0x5602813e7e20 2023-07-14T16:20:50.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.387+0000 7f5b38d7ab80 10 client.0 _ll_register_callbacks cb 0x7f5b14005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:50.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e74d0 0x5602813f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e74d0 0x5602813f81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43108/0 (socket says 172.21.15.62:43108) 2023-07-14T16:20:50.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1effd700 1 -- 172.21.15.62:0/1356319500 learned_addr learned my addr 172.21.15.62:0/1356319500 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:50.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1e7fc700 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e7e20 0x5602813f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1f7fe700 1 -- 172.21.15.62:0/1356319500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e7e20 msgr2=0x5602813f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:50.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e7e20 0x5602813f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1f7fe700 1 -- 172.21.15.62:0/1356319500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e74d0 msgr2=0x5602813f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:50.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e74d0 0x5602813f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1f7fe700 1 -- 172.21.15.62:0/1356319500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a00f0 con 0x5602813e8c70 2023-07-14T16:20:50.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.388+0000 7f5b1effd700 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e74d0 0x5602813f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:50.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.389+0000 7f5b1f7fe700 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813f4d90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b070 tx=0x7f5b1800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ba40002c99da48 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.389+0000 7f5b02ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.389+0000 7f5b02ff5700 1 -- 172.21.15.62:0/1356319500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800e070 con 0x5602813e8c70 2023-07-14T16:20:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.389+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1356319500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b10004390 con 0x5602813e8c70 2023-07-14T16:20:50.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.389+0000 7f5b02ff5700 1 -- 172.21.15.62:0/1356319500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b1801f650 con 0x5602813e8c70 2023-07-14T16:20:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.389+0000 7f5b02ff5700 1 -- 172.21.15.62:0/1356319500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b180094a0 con 0x5602813e8c70 2023-07-14T16:20:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.389+0000 7f5b38d7ab80 10 client.16896 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.389+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1356319500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5602811652b0 con 0x5602813e8c70 2023-07-14T16:20:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b02ff5700 1 -- 172.21.15.62:0/1356319500 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b10004390 con 0x5602813e8c70 2023-07-14T16:20:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b02ff5700 1 -- 172.21.15.62:0/1356319500 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5b180054b0 con 0x5602813e8c70 2023-07-14T16:20:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b38d7ab80 10 client.16896 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b38d7ab80 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:50.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b38d7ab80 20 client.16896 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5602814523d0 0x5602814547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b38d7ab80 4 client.16896 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:20:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1356319500 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5602812f4050 con 0x5602814523d0 2023-07-14T16:20:50.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.390+0000 7f5b1effd700 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5602814523d0 0x5602814547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.391+0000 7f5b1effd700 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5602814523d0 0x5602814547e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10a0c5ff141943 server_cookie=20ccd9edcebc2c18 in_seq=0 out_seq=0 2023-07-14T16:20:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.391+0000 7f5b02ff5700 10 client.16896 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.391+0000 7f5b02ff5700 1 -- 172.21.15.62:0/1356319500 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5602812f4050 con 0x5602814523d0 2023-07-14T16:20:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.391+0000 7f5b02ff5700 10 client.16896 handle_command_reply: tid=0 2023-07-14T16:20:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.391+0000 7f5b38d7ab80 1 client.16896 shutdown 2023-07-14T16:20:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1356319500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 msgr2=0x5602813f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813f4d90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b070 tx=0x7f5b1800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1356319500 shutdown_connections 2023-07-14T16:20:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602813e8c70 0x5602813f4d90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5602814523d0 0x5602814547e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602813e74d0 0x5602813f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 --2- 172.21.15.62:0/1356319500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5602813e7e20 0x5602813f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1356319500 >> 172.21.15.62:0/1356319500 conn(0x5602813dcda0 msgr2=0x5602813eefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b02ff5700 0 client.16896 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1356319500 shutdown_connections 2023-07-14T16:20:50.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 1 -- 172.21.15.62:0/1356319500 wait complete. 2023-07-14T16:20:50.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.392+0000 7f5b38d7ab80 20 client.16896 trim_cache size 0 max 16384 2023-07-14T16:20:50.379 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:20:50.379 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:20:50.379 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:20:50.379 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:20:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 -- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b080a11d0 msgr2=0x7f5b080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b080a11d0 0x7f5b080a3680 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b780 tx=0x7f5b18006210 comp rx=0 tx=0).stop 2023-07-14T16:20:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 -- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b20102e80 msgr2=0x7f5b20197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b20102e80 0x7f5b20197fa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400e9a0 tx=0x7f5b1400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 -- 172.21.15.62:0/2417052690 shutdown_connections 2023-07-14T16:20:50.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b080a11d0 0x7f5b080a3680 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b201057a0 0x7f5b2019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b20100560 0x7f5b20197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 --2- 172.21.15.62:0/2417052690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b20102e80 0x7f5b20197fa0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 -- 172.21.15.62:0/2417052690 >> 172.21.15.62:0/2417052690 conn(0x7f5b200fa000 msgr2=0x7f5b200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:50.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 -- 172.21.15.62:0/2417052690 shutdown_connections 2023-07-14T16:20:50.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.393+0000 7f5b253ff700 1 -- 172.21.15.62:0/2417052690 wait complete. 2023-07-14T16:20:50.391 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:20:50.391 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:20:50.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 -- 172.21.15.62:0/2255946429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0100550 msgr2=0x7f77b0102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 --2- 172.21.15.62:0/2255946429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0100550 0x7f77b0102930 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009990 tx=0x7f77a0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 -- 172.21.15.62:0/2255946429 shutdown_connections 2023-07-14T16:20:50.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 --2- 172.21.15.62:0/2255946429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b0105790 0x7f77b0107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 --2- 172.21.15.62:0/2255946429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0100550 0x7f77b0102930 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 --2- 172.21.15.62:0/2255946429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77b0102e70 0x7f77b0105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 -- 172.21.15.62:0/2255946429 >> 172.21.15.62:0/2255946429 conn(0x7f77b00f9fd0 msgr2=0x7f77b00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:50.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 -- 172.21.15.62:0/2255946429 shutdown_connections 2023-07-14T16:20:50.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.654+0000 7f77b53ff700 1 -- 172.21.15.62:0/2255946429 wait complete. 2023-07-14T16:20:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.655+0000 7f77b53ff700 1 Processor -- start 2023-07-14T16:20:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77b53ff700 1 -- start start 2023-07-14T16:20:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77b53ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b0100550 0x7f77b00ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77b53ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77b0102e70 0x7f77b00fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77b53ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0105790 0x7f77b00fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77b53ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b00ff1e0 con 0x7f77b0105790 2023-07-14T16:20:50.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77b53ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b00ff350 con 0x7f77b0102e70 2023-07-14T16:20:50.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77b53ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b019f760 con 0x7f77b0100550 2023-07-14T16:20:50.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77af7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0105790 0x7f77b00fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77aeffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b0100550 0x7f77b00ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77af7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0105790 0x7f77b00fea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55534/0 (socket says 172.21.15.62:55534) 2023-07-14T16:20:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.656+0000 7f77af7fe700 1 -- 172.21.15.62:0/3829449597 learned_addr learned my addr 172.21.15.62:0/3829449597 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77ae7fc700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77b0102e70 0x7f77b00fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77af7fe700 1 -- 172.21.15.62:0/3829449597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b0100550 msgr2=0x7f77b00ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77af7fe700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b0100550 0x7f77b00ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77af7fe700 1 -- 172.21.15.62:0/3829449597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77b0102e70 msgr2=0x7f77b00fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77af7fe700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77b0102e70 0x7f77b00fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77af7fe700 1 -- 172.21.15.62:0/3829449597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a00095f0 con 0x7f77b0105790 2023-07-14T16:20:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77ae7fc700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77b0102e70 0x7f77b00fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77af7fe700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0105790 0x7f77b00fea40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b970 tx=0x7f77a800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f7797fff700 1 -- 172.21.15.62:0/3829449597 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a800c930 con 0x7f77b0105790 2023-07-14T16:20:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f7797fff700 1 -- 172.21.15.62:0/3829449597 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77a8014440 con 0x7f77b0105790 2023-07-14T16:20:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b019fa40 con 0x7f77b0105790 2023-07-14T16:20:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.657+0000 7f7797fff700 1 -- 172.21.15.62:0/3829449597 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a80075d0 con 0x7f77b0105790 2023-07-14T16:20:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.658+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b019ff80 con 0x7f77b0105790 2023-07-14T16:20:50.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.659+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b00fbb40 con 0x7f77b0105790 2023-07-14T16:20:50.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.661+0000 7f7797fff700 1 -- 172.21.15.62:0/3829449597 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f77a80145b0 con 0x7f77b0105790 2023-07-14T16:20:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.663+0000 7f7797fff700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f77980a11a0 0x7f77980a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.663+0000 7f7797fff700 1 -- 172.21.15.62:0/3829449597 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77a8006e40 con 0x7f77b0105790 2023-07-14T16:20:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.663+0000 7f77aeffd700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f77980a11a0 0x7f77980a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:50.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.664+0000 7f77aeffd700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f77980a11a0 0x7f77980a3650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f77a000b5c0 tx=0x7f77a0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:50.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.667+0000 7f7797fff700 1 -- 172.21.15.62:0/3829449597 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f77a8095980 con 0x7f77b0105790 2023-07-14T16:20:50.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.982+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77b002c850 con 0x7f77b0105790 2023-07-14T16:20:50.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.983+0000 7f7797fff700 1 -- 172.21.15.62:0/3829449597 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f77a8098fa0 con 0x7f77b0105790 2023-07-14T16:20:50.960 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:20:50.960 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:20:50.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f77980a11a0 msgr2=0x7f77980a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f77980a11a0 0x7f77980a3650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f77a000b5c0 tx=0x7f77a0005ee0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0105790 msgr2=0x7f77b00fea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0105790 0x7f77b00fea40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b970 tx=0x7f77a800bc80 comp rx=0 tx=0).stop 2023-07-14T16:20:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 shutdown_connections 2023-07-14T16:20:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77b0105790 0x7f77b00fea40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b0100550 0x7f77b00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f77980a11a0 0x7f77980a3650 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 --2- 172.21.15.62:0/3829449597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f77b0102e70 0x7f77b00fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.985+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 >> 172.21.15.62:0/3829449597 conn(0x7f77b00f9fd0 msgr2=0x7f77b01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.986+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 shutdown_connections 2023-07-14T16:20:50.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:50.986+0000 7f77b53ff700 1 -- 172.21.15.62:0/3829449597 wait complete. 2023-07-14T16:20:50.964 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:20:50.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:20:51.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:50 smithi164 ceph-mon[97427]: pgmap v513: 97 pgs: 97 active+clean; 2.9 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:51.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:50 smithi062 ceph-mon[95543]: pgmap v513: 97 pgs: 97 active+clean; 2.9 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:51.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:50 smithi081 ceph-mon[127795]: pgmap v513: 97 pgs: 97 active+clean; 2.9 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:20:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/3719505041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107300 msgr2=0x7fe5a8107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/3719505041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107300 0x7fe5a8107750 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009990 tx=0x7fe59c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/3719505041 shutdown_connections 2023-07-14T16:20:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/3719505041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8107c90 0x7fe5a810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/3719505041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5a8106110 0x7fe5a8106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/3719505041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107300 0x7fe5a8107750 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/3719505041 >> 172.21.15.62:0/3719505041 conn(0x7fe5a8075160 msgr2=0x7fe5a8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/3719505041 shutdown_connections 2023-07-14T16:20:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.238+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/3719505041 wait complete. 2023-07-14T16:20:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.239+0000 7fe5ad3ff700 1 Processor -- start 2023-07-14T16:20:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5ad3ff700 1 -- start start 2023-07-14T16:20:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5ad3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8106110 0x7fe5a819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5ad3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5a8107300 0x7fe5a819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5ad3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107c90 0x7fe5a81a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5ad3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a81a3950 con 0x7fe5a8107c90 2023-07-14T16:20:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5ad3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a81a3a90 con 0x7fe5a8107300 2023-07-14T16:20:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5ad3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a81a3c20 con 0x7fe5a8106110 2023-07-14T16:20:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5a6ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8106110 0x7fe5a819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.240+0000 7fe5a6ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8106110 0x7fe5a819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34080/0 (socket says 172.21.15.62:34080) 2023-07-14T16:20:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5a8107300 0x7fe5a819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107c90 0x7fe5a81a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a6ffd700 1 -- 172.21.15.62:0/4007231758 learned_addr learned my addr 172.21.15.62:0/4007231758 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a67fc700 1 -- 172.21.15.62:0/4007231758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8106110 msgr2=0x7fe5a819bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8106110 0x7fe5a819bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a67fc700 1 -- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107c90 msgr2=0x7fe5a81a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107c90 0x7fe5a81a3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a67fc700 1 -- 172.21.15.62:0/4007231758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0095f0 con 0x7fe5a8107300 2023-07-14T16:20:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8106110 0x7fe5a819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107c90 0x7fe5a81a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.241+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5a8107300 0x7fe5a819c310 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe59c000c00 tx=0x7fe59c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.242+0000 7fe58ffff700 1 -- 172.21.15.62:0/4007231758 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c01d070 con 0x7fe5a8107300 2023-07-14T16:20:51.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.242+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/4007231758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a81a3ea0 con 0x7fe5a8107300 2023-07-14T16:20:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.242+0000 7fe58ffff700 1 -- 172.21.15.62:0/4007231758 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe59c00bdd0 con 0x7fe5a8107300 2023-07-14T16:20:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.242+0000 7fe58ffff700 1 -- 172.21.15.62:0/4007231758 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c022d60 con 0x7fe5a8107300 2023-07-14T16:20:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.242+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/4007231758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a81a4270 con 0x7fe5a8107300 2023-07-14T16:20:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.246+0000 7fe58ffff700 1 -- 172.21.15.62:0/4007231758 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe59c022470 con 0x7fe5a8107300 2023-07-14T16:20:51.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.247+0000 7fe58ffff700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5900a1150 0x7fe5900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.247+0000 7fe58ffff700 1 -- 172.21.15.62:0/4007231758 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe59c0cde60 con 0x7fe5a8107300 2023-07-14T16:20:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.247+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5900a1150 0x7fe5900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.248+0000 7fe5c0d29b80 1 Processor -- start 2023-07-14T16:20:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.248+0000 7fe5c0d29b80 1 -- start start 2023-07-14T16:20:51.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.248+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5900a1150 0x7fe5900a3600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe598005950 tx=0x7fe598009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.248+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b0e8cb0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b13ba40 con 0x55bb3b1e34d0 2023-07-14T16:20:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b134ed0 con 0x55bb3b1e3e20 2023-07-14T16:20:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1e50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55568/0 (socket says 172.21.15.62:55568) 2023-07-14T16:20:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a77fe700 1 -- 172.21.15.62:0/3494592245 learned_addr learned my addr 172.21.15.62:0/3494592245 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a77fe700 1 -- 172.21.15.62:0/3494592245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e3e20 msgr2=0x55bb3b1e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1e46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a77fe700 1 -- 172.21.15.62:0/3494592245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e34d0 msgr2=0x55bb3b1e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.249+0000 7fe5a77fe700 1 -- 172.21.15.62:0/3494592245 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb3af62de0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.250+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1e50e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00078d0 tx=0x7fe5a0007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ed1b22756ddf3c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.250+0000 7fe58d7fa700 1 -- 172.21.15.62:0/3494592245 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0004bb0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.250+0000 7fe58d7fa700 1 -- 172.21.15.62:0/3494592245 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5a000f040 con 0x55bb3b1e4c70 2023-07-14T16:20:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.250+0000 7fe58d7fa700 1 -- 172.21.15.62:0/3494592245 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00052a0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3494592245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 msgr2=0x55bb3b1e50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1e50e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00078d0 tx=0x7fe5a0007be0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3494592245 shutdown_connections 2023-07-14T16:20:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1e50e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3494592245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3494592245 >> 172.21.15.62:0/3494592245 conn(0x55bb3b1d8da0 msgr2=0x55bb3b1de1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3494592245 shutdown_connections 2023-07-14T16:20:51.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.251+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3494592245 wait complete. 2023-07-14T16:20:51.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.252+0000 7fe5c0d29b80 1 -- start start 2023-07-14T16:20:51.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.252+0000 7fe5c0d29b80 1 Processor -- start 2023-07-14T16:20:51.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1ed5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b0d4ac0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b0d47c0 con 0x55bb3b1e3e20 2023-07-14T16:20:51.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3afb8920 con 0x55bb3b1e34d0 2023-07-14T16:20:51.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5c0d29b80 10 client.0 _ll_register_callbacks cb 0x55bb3b0d2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:51.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55572/0 (socket says 172.21.15.62:55572) 2023-07-14T16:20:51.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5a77fe700 1 -- 172.21.15.62:0/3532642790 learned_addr learned my addr 172.21.15.62:0/3532642790 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1ed5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.253+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe5a77fe700 1 -- 172.21.15.62:0/3532642790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 msgr2=0x55bb3b1ed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1ed0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe5a77fe700 1 -- 172.21.15.62:0/3532642790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 msgr2=0x55bb3b1ed5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1ed5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe5a77fe700 1 -- 172.21.15.62:0/3532642790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a0007500 con 0x55bb3b1e4c70 2023-07-14T16:20:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1f4620 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0017000 tx=0x7fe5a00158e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ade8feea7fd454d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe58aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe58aff5700 1 -- 172.21.15.62:0/3532642790 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0015c00 con 0x55bb3b1e4c70 2023-07-14T16:20:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe58aff5700 1 -- 172.21.15.62:0/3532642790 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe5a0003c10 con 0x55bb3b1e4c70 2023-07-14T16:20:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.254+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3532642790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb3b08e4f0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe58aff5700 1 -- 172.21.15.62:0/3532642790 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5a0003dd0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe5c0d29b80 10 client.16914 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3532642790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb3af4bc60 con 0x55bb3b1e4c70 2023-07-14T16:20:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe58aff5700 1 -- 172.21.15.62:0/3532642790 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bb3b08e4f0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe58aff5700 1 -- 172.21.15.62:0/3532642790 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe5a001f5e0 con 0x55bb3b1e4c70 2023-07-14T16:20:51.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe5c0d29b80 10 client.16914 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:51.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe5c0d29b80 10 client.16914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:20:51.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe5c0d29b80 20 client.16914 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:51.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bb3b1f8c90 0x55bb3b1fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe5c0d29b80 4 client.16914 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:51.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.255+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3532642790 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb3b12deb0 con 0x55bb3b1f8c90 2023-07-14T16:20:51.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.256+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bb3b1f8c90 0x55bb3b1fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.256+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bb3b1f8c90 0x55bb3b1fb0a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f28c0b92466b1ac7 server_cookie=ba2984620ec13679 in_seq=0 out_seq=0 2023-07-14T16:20:51.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.256+0000 7fe58aff5700 10 client.16914 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:20:51.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.257+0000 7fe58aff5700 1 -- 172.21.15.62:0/3532642790 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb3b12deb0 con 0x55bb3b1f8c90 2023-07-14T16:20:51.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.257+0000 7fe58aff5700 10 client.16914 handle_command_reply: tid=0 2023-07-14T16:20:51.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.257+0000 7fe5c0d29b80 1 client.16914 shutdown 2023-07-14T16:20:51.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.257+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3532642790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 msgr2=0x55bb3b1f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.257+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1f4620 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0017000 tx=0x7fe5a00158e0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3532642790 shutdown_connections 2023-07-14T16:20:51.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bb3b1f8c90 0x55bb3b1fb0a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e4c70 0x55bb3b1f4620 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1ed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe58aff5700 0 client.16914 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:20:51.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/3532642790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1ed5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3532642790 >> 172.21.15.62:0/3532642790 conn(0x55bb3b1d8da0 msgr2=0x55bb3b1dd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3532642790 shutdown_connections 2023-07-14T16:20:51.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/3532642790 wait complete. 2023-07-14T16:20:51.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.258+0000 7fe5c0d29b80 20 client.16914 trim_cache size 0 max 16384 2023-07-14T16:20:51.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.275+0000 7fe5c0d29b80 1 Processor -- start 2023-07-14T16:20:51.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.275+0000 7fe5c0d29b80 1 -- start start 2023-07-14T16:20:51.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.275+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.275+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.275+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1f4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.275+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b1f4a80 con 0x55bb3b1e34d0 2023-07-14T16:20:51.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.275+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b1dfa00 con 0x55bb3b1e3e20 2023-07-14T16:20:51.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b1ead50 con 0x55bb3b1d8da0 2023-07-14T16:20:51.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55582/0 (socket says 172.21.15.62:55582) 2023-07-14T16:20:51.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a67fc700 1 -- 172.21.15.62:0/962557906 learned_addr learned my addr 172.21.15.62:0/962557906 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a67fc700 1 -- 172.21.15.62:0/962557906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1d8da0 msgr2=0x55bb3b1e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a67fc700 1 -- 172.21.15.62:0/962557906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 msgr2=0x55bb3b1f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.276+0000 7fe5a67fc700 1 -- 172.21.15.62:0/962557906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb3af89250 con 0x55bb3b1e34d0 2023-07-14T16:20:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.277+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e8010 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe59c097e40 tx=0x7fe59c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c49b2e083d84db7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.277+0000 7fe58dffb700 1 -- 172.21.15.62:0/962557906 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c09ad30 con 0x55bb3b1e34d0 2023-07-14T16:20:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.277+0000 7fe58dffb700 1 -- 172.21.15.62:0/962557906 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe59c09b3d0 con 0x55bb3b1e34d0 2023-07-14T16:20:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.277+0000 7fe58dffb700 1 -- 172.21.15.62:0/962557906 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c099770 con 0x55bb3b1e34d0 2023-07-14T16:20:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/962557906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e34d0 msgr2=0x55bb3b1e8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e8010 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe59c097e40 tx=0x7fe59c01a040 comp rx=0 tx=0).stop 2023-07-14T16:20:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/962557906 shutdown_connections 2023-07-14T16:20:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1e3e20 0x55bb3b1f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/962557906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1e8010 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/962557906 >> 172.21.15.62:0/962557906 conn(0x55bb3b1ec150 msgr2=0x55bb3b1f4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/962557906 shutdown_connections 2023-07-14T16:20:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/962557906 wait complete. 2023-07-14T16:20:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.278+0000 7fe5c0d29b80 1 -- start start 2023-07-14T16:20:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5c0d29b80 1 Processor -- start 2023-07-14T16:20:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5c0d29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e3e20 0x55bb3b24cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b1f46c0 con 0x55bb3b1e3e20 2023-07-14T16:20:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b1f4830 con 0x55bb3b1d8da0 2023-07-14T16:20:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5c0d29b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3b24d080 con 0x55bb3b1e34d0 2023-07-14T16:20:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5c0d29b80 10 client.0 _ll_register_callbacks cb 0x7fe598009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e3e20 0x55bb3b24cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5a6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e3e20 0x55bb3b24cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55596/0 (socket says 172.21.15.62:55596) 2023-07-14T16:20:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.279+0000 7fe5a77fe700 1 -- 172.21.15.62:0/2969643402 learned_addr learned my addr 172.21.15.62:0/2969643402 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe5a6ffd700 1 -- 172.21.15.62:0/2969643402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 msgr2=0x55bb3b1f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe5a6ffd700 1 -- 172.21.15.62:0/2969643402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e3e20 msgr2=0x55bb3b24cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e3e20 0x55bb3b24cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe5a6ffd700 1 -- 172.21.15.62:0/2969643402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a81a4270 con 0x55bb3b1d8da0 2023-07-14T16:20:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe5a77fe700 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e3e20 0x55bb3b24cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe5a6ffd700 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1f1160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55bb3b1f9060 tx=0x7fe5980047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d2d7f89d7a563c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.280+0000 7fe58aff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.281+0000 7fe58aff5700 1 -- 172.21.15.62:0/2969643402 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59800eb40 con 0x55bb3b1d8da0 2023-07-14T16:20:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.281+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/2969643402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5a000f040 con 0x55bb3b1d8da0 2023-07-14T16:20:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.281+0000 7fe58aff5700 1 -- 172.21.15.62:0/2969643402 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe598030070 con 0x55bb3b1d8da0 2023-07-14T16:20:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.281+0000 7fe58aff5700 1 -- 172.21.15.62:0/2969643402 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe59800eca0 con 0x55bb3b1d8da0 2023-07-14T16:20:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.281+0000 7fe5c0d29b80 10 client.26624 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.281+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/2969643402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb3af612b0 con 0x55bb3b1d8da0 2023-07-14T16:20:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe58aff5700 1 -- 172.21.15.62:0/2969643402 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5a000f040 con 0x55bb3b1d8da0 2023-07-14T16:20:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe58aff5700 1 -- 172.21.15.62:0/2969643402 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe59803d070 con 0x55bb3b1d8da0 2023-07-14T16:20:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe5c0d29b80 10 client.26624 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe5c0d29b80 10 client.26624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:20:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe5c0d29b80 20 client.26624 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bb3b1e4c70 0x55bb3b250a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe5c0d29b80 4 client.26624 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:20:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/2969643402 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb3b0f0050 con 0x55bb3b1e4c70 2023-07-14T16:20:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.282+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bb3b1e4c70 0x55bb3b250a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5a67fc700 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bb3b1e4c70 0x55bb3b250a80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8f0fafd461c1744 server_cookie=63ba087f06212040 in_seq=0 out_seq=0 2023-07-14T16:20:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe58aff5700 10 client.26624 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:20:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe58aff5700 1 -- 172.21.15.62:0/2969643402 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb3b0f0050 con 0x55bb3b1e4c70 2023-07-14T16:20:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe58aff5700 10 client.26624 handle_command_reply: tid=0 2023-07-14T16:20:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 client.26624 shutdown 2023-07-14T16:20:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/2969643402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1d8da0 msgr2=0x55bb3b1f1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1f1160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55bb3b1f9060 tx=0x7fe5980047c0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/2969643402 shutdown_connections 2023-07-14T16:20:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bb3b1e4c70 0x55bb3b250a80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb3b1e3e20 0x55bb3b24cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb3b1d8da0 0x55bb3b1f1160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe58aff5700 0 client.26624 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:20:51.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 --2- 172.21.15.62:0/2969643402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb3b1e34d0 0x55bb3b1f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.283+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/2969643402 >> 172.21.15.62:0/2969643402 conn(0x55bb3b1ec150 msgr2=0x55bb3b1e7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.284+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/2969643402 shutdown_connections 2023-07-14T16:20:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.284+0000 7fe5c0d29b80 1 -- 172.21.15.62:0/2969643402 wait complete. 2023-07-14T16:20:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.284+0000 7fe5c0d29b80 20 client.26624 trim_cache size 0 max 16384 2023-07-14T16:20:51.267 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:20:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.284+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5900a1150 msgr2=0x7fe5900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.284+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5900a1150 0x7fe5900a3600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe598005950 tx=0x7fe598009260 comp rx=0 tx=0).stop 2023-07-14T16:20:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.284+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/4007231758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5a8107300 msgr2=0x7fe5a819c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.284+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5a8107300 0x7fe5a819c310 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe59c000c00 tx=0x7fe59c004730 comp rx=0 tx=0).stop 2023-07-14T16:20:51.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.285+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/4007231758 shutdown_connections 2023-07-14T16:20:51.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.285+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5a8107c90 0x7fe5a81a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.285+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5a8106110 0x7fe5a819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.285+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5900a1150 0x7fe5900a3600 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.285+0000 7fe5ad3ff700 1 --2- 172.21.15.62:0/4007231758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5a8107300 0x7fe5a819c310 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.285+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/4007231758 >> 172.21.15.62:0/4007231758 conn(0x7fe5a8075160 msgr2=0x7fe5a8077470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.285+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/4007231758 shutdown_connections 2023-07-14T16:20:51.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.285+0000 7fe5ad3ff700 1 -- 172.21.15.62:0/4007231758 wait complete. 2023-07-14T16:20:51.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:20:51.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 -- 172.21.15.62:0/120838718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34102d70 msgr2=0x7f4c34105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/120838718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34102d70 0x7f4c34105150 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009990 tx=0x7f4c24009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 -- 172.21.15.62:0/120838718 shutdown_connections 2023-07-14T16:20:51.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/120838718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c34105690 0x7f4c34107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/120838718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c34100450 0x7f4c34102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/120838718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34102d70 0x7f4c34105150 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 -- 172.21.15.62:0/120838718 >> 172.21.15.62:0/120838718 conn(0x7f4c340f9ed0 msgr2=0x7f4c340fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 -- 172.21.15.62:0/120838718 shutdown_connections 2023-07-14T16:20:51.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.533+0000 7f4c3a79d700 1 -- 172.21.15.62:0/120838718 wait complete. 2023-07-14T16:20:51.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.534+0000 7f4c3a79d700 1 Processor -- start 2023-07-14T16:20:51.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.534+0000 7f4c3a79d700 1 -- start start 2023-07-14T16:20:51.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3a79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34100450 0x7f4c341976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3a79d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c34102d70 0x7f4c34197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3a79d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c34105690 0x7f4c3419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3a79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3419f270 con 0x7f4c34100450 2023-07-14T16:20:51.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3a79d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3419f3b0 con 0x7f4c34105690 2023-07-14T16:20:51.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3a79d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3419f540 con 0x7f4c34102d70 2023-07-14T16:20:51.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34100450 0x7f4c341976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34100450 0x7f4c341976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55612/0 (socket says 172.21.15.62:55612) 2023-07-14T16:20:51.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c3979b700 1 -- 172.21.15.62:0/3489995026 learned_addr learned my addr 172.21.15.62:0/3489995026 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c34102d70 0x7f4c34197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.535+0000 7f4c39f9c700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c34105690 0x7f4c3419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c3979b700 1 -- 172.21.15.62:0/3489995026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c34102d70 msgr2=0x7f4c34197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c3979b700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c34102d70 0x7f4c34197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c3979b700 1 -- 172.21.15.62:0/3489995026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c34105690 msgr2=0x7f4c3419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c3979b700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c34105690 0x7f4c3419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c3979b700 1 -- 172.21.15.62:0/3489995026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240095f0 con 0x7f4c34100450 2023-07-14T16:20:51.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c34102d70 0x7f4c34197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:51.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c3979b700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34100450 0x7f4c341976f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000e9a0 tx=0x7f4c3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c2a7fc700 1 -- 172.21.15.62:0/3489995026 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30009790 con 0x7f4c34100450 2023-07-14T16:20:51.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c2a7fc700 1 -- 172.21.15.62:0/3489995026 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c3000cbd0 con 0x7f4c34100450 2023-07-14T16:20:51.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c3a79d700 1 -- 172.21.15.62:0/3489995026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3419f820 con 0x7f4c34100450 2023-07-14T16:20:51.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c2a7fc700 1 -- 172.21.15.62:0/3489995026 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30018680 con 0x7f4c34100450 2023-07-14T16:20:51.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.536+0000 7f4c3a79d700 1 -- 172.21.15.62:0/3489995026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3419fd80 con 0x7f4c34100450 2023-07-14T16:20:51.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.540+0000 7f4c2a7fc700 1 -- 172.21.15.62:0/3489995026 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4c3000cd40 con 0x7f4c34100450 2023-07-14T16:20:51.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.541+0000 7f4c4f256b80 1 Processor -- start 2023-07-14T16:20:51.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c2a7fc700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c200a1270 0x7f4c200a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c4f256b80 1 -- start start 2023-07-14T16:20:51.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c2a7fc700 1 -- 172.21.15.62:0/3489995026 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c30014070 con 0x7f4c34100450 2023-07-14T16:20:51.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 0x5639641d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639640d5cb0 con 0x5639641d0720 2023-07-14T16:20:51.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563964128a40 con 0x5639641d1e30 2023-07-14T16:20:51.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563964121ed0 con 0x5639641d1070 2023-07-14T16:20:51.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.542+0000 7f4c39f9c700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c200a1270 0x7f4c200a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c3979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34152/0 (socket says 172.21.15.62:34152) 2023-07-14T16:20:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 -- 172.21.15.62:0/2671242016 learned_addr learned my addr 172.21.15.62:0/2671242016 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c39f9c700 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 0x5639641d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 -- 172.21.15.62:0/2671242016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 msgr2=0x5639641d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 0x5639641d22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 -- 172.21.15.62:0/2671242016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 msgr2=0x5639641d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 -- 172.21.15.62:0/2671242016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563963f4fde0 con 0x5639641d1070 2023-07-14T16:20:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c3979b700 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c39f9c700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c200a1270 0x7f4c200a3720 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00f540 tx=0x7f4c2c009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.543+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641d18a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c24005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36671b99fcee839e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.544+0000 7f4c1f7fe700 1 -- 172.21.15.62:0/2671242016 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240047e0 con 0x5639641d1070 2023-07-14T16:20:51.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.544+0000 7f4c1f7fe700 1 -- 172.21.15.62:0/2671242016 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c2401e040 con 0x5639641d1070 2023-07-14T16:20:51.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.544+0000 7f4c1f7fe700 1 -- 172.21.15.62:0/2671242016 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24003680 con 0x5639641d1070 2023-07-14T16:20:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 -- 172.21.15.62:0/2671242016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 msgr2=0x5639641d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641d18a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c24005c80 comp rx=0 tx=0).stop 2023-07-14T16:20:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 -- 172.21.15.62:0/2671242016 shutdown_connections 2023-07-14T16:20:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 0x5639641d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/2671242016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641d18a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 -- 172.21.15.62:0/2671242016 >> 172.21.15.62:0/2671242016 conn(0x5639641c5ff0 msgr2=0x5639641cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 -- 172.21.15.62:0/2671242016 shutdown_connections 2023-07-14T16:20:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 -- 172.21.15.62:0/2671242016 wait complete. 2023-07-14T16:20:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.545+0000 7f4c4f256b80 1 -- start start 2023-07-14T16:20:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.546+0000 7f4c4f256b80 1 Processor -- start 2023-07-14T16:20:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.546+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641da300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.546+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1070 0x5639641da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.546+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1e30 0x5639641e1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.546+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639640c1ac0 con 0x5639641d0720 2023-07-14T16:20:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.546+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639640c17c0 con 0x5639641d1070 2023-07-14T16:20:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.546+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963fa5920 con 0x5639641d1e30 2023-07-14T16:20:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.546+0000 7f4c4f256b80 10 client.0 _ll_register_callbacks cb 0x5639640bf760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:51.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641da300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c38f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1070 0x5639641da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641da300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55636/0 (socket says 172.21.15.62:55636) 2023-07-14T16:20:51.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 -- 172.21.15.62:0/4066848996 learned_addr learned my addr 172.21.15.62:0/4066848996 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c39f9c700 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1e30 0x5639641e1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 -- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1e30 msgr2=0x5639641e1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1e30 0x5639641e1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 -- 172.21.15.62:0/4066848996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1070 msgr2=0x5639641da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1070 0x5639641da840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 -- 172.21.15.62:0/4066848996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3419fd80 con 0x5639641d0720 2023-07-14T16:20:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c39f9c700 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1e30 0x5639641e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1070 0x5639641da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c3979b700 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641da300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4c300ca630 tx=0x7f4c3009a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4bd5707c24a2b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.547+0000 7f4c1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c1cff9700 1 -- 172.21.15.62:0/4066848996 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300a2100 con 0x5639641d0720 2023-07-14T16:20:51.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c1cff9700 1 -- 172.21.15.62:0/4066848996 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c300a2390 con 0x5639641d0720 2023-07-14T16:20:51.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c4f256b80 1 -- 172.21.15.62:0/4066848996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56396407b4f0 con 0x5639641d0720 2023-07-14T16:20:51.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c1cff9700 1 -- 172.21.15.62:0/4066848996 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c300b8030 con 0x5639641d0720 2023-07-14T16:20:51.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c4f256b80 10 client.16932 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:51.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c4f256b80 1 -- 172.21.15.62:0/4066848996 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f4c341086e0 con 0x5639641d0720 2023-07-14T16:20:51.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c1cff9700 1 -- 172.21.15.62:0/4066848996 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c300c8020 con 0x5639641d0720 2023-07-14T16:20:51.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c1cff9700 1 -- 172.21.15.62:0/4066848996 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4c300cacf0 con 0x5639641d0720 2023-07-14T16:20:51.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c4f256b80 10 client.16932 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:51.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.548+0000 7f4c4f256b80 10 client.16932 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:20:51.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.549+0000 7f4c4f256b80 20 client.16932 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:51.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.549+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5639641e60e0 0x5639641e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.549+0000 7f4c4f256b80 4 client.16932 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:51.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.549+0000 7f4c4f256b80 1 -- 172.21.15.62:0/4066848996 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56396411aeb0 con 0x5639641e60e0 2023-07-14T16:20:51.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.549+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5639641e60e0 0x5639641e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.550+0000 7f4c1cff9700 10 client.16932 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:20:51.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.550+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5639641e60e0 0x5639641e84f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f39b06ec3f5ed1e1 server_cookie=24c2e6095c0dc29b in_seq=0 out_seq=0 2023-07-14T16:20:51.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.550+0000 7f4c1cff9700 1 -- 172.21.15.62:0/4066848996 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56396411aeb0 con 0x5639641e60e0 2023-07-14T16:20:51.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c1cff9700 10 client.16932 handle_command_reply: tid=0 2023-07-14T16:20:51.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 client.16932 shutdown 2023-07-14T16:20:51.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 -- 172.21.15.62:0/4066848996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 msgr2=0x5639641da300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641da300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4c300ca630 tx=0x7f4c3009a480 comp rx=0 tx=0).stop 2023-07-14T16:20:51.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 -- 172.21.15.62:0/4066848996 shutdown_connections 2023-07-14T16:20:51.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5639641e60e0 0x5639641e84f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1e30 0x5639641e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641da300 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c1cff9700 0 client.16932 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:20:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/4066848996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1070 0x5639641da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 -- 172.21.15.62:0/4066848996 >> 172.21.15.62:0/4066848996 conn(0x5639641c5ff0 msgr2=0x5639641cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 -- 172.21.15.62:0/4066848996 shutdown_connections 2023-07-14T16:20:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 1 -- 172.21.15.62:0/4066848996 wait complete. 2023-07-14T16:20:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.551+0000 7f4c4f256b80 20 client.16932 trim_cache size 0 max 16384 2023-07-14T16:20:51.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c4f256b80 1 Processor -- start 2023-07-14T16:20:51.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c4f256b80 1 -- start start 2023-07-14T16:20:51.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641e1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 0x5639641cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639641ccc10 con 0x5639641d0720 2023-07-14T16:20:51.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639641c5b70 con 0x5639641d1e30 2023-07-14T16:20:51.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639641cc660 con 0x5639641d1070 2023-07-14T16:20:51.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.570+0000 7f4c3979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c3979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55648/0 (socket says 172.21.15.62:55648) 2023-07-14T16:20:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c3979b700 1 -- 172.21.15.62:0/3547407066 learned_addr learned my addr 172.21.15.62:0/3547407066 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641e1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c39f9c700 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 0x5639641cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c3979b700 1 -- 172.21.15.62:0/3547407066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 msgr2=0x5639641e1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c3979b700 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641e1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c3979b700 1 -- 172.21.15.62:0/3547407066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 msgr2=0x5639641cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c3979b700 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 0x5639641cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c3979b700 1 -- 172.21.15.62:0/3547407066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563963f38c60 con 0x5639641d0720 2023-07-14T16:20:51.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.571+0000 7f4c3979b700 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d8fe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4c300969b0 tx=0x7f4c300984f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d914e33e70c839 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.572+0000 7f4c1ffff700 1 -- 172.21.15.62:0/3547407066 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300b8040 con 0x5639641d0720 2023-07-14T16:20:51.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.572+0000 7f4c1ffff700 1 -- 172.21.15.62:0/3547407066 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c300c8070 con 0x5639641d0720 2023-07-14T16:20:51.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.572+0000 7f4c1ffff700 1 -- 172.21.15.62:0/3547407066 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300a1f00 con 0x5639641d0720 2023-07-14T16:20:51.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.572+0000 7f4c4f256b80 1 -- 172.21.15.62:0/3547407066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 msgr2=0x5639641d8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.572+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d8fe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4c300969b0 tx=0x7f4c300984f0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 -- 172.21.15.62:0/3547407066 shutdown_connections 2023-07-14T16:20:51.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d1e30 0x5639641cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d0720 0x5639641d8fe0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/3547407066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641e1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 -- 172.21.15.62:0/3547407066 >> 172.21.15.62:0/3547407066 conn(0x5639641c5ff0 msgr2=0x5639641ece50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 -- 172.21.15.62:0/3547407066 shutdown_connections 2023-07-14T16:20:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 -- 172.21.15.62:0/3547407066 wait complete. 2023-07-14T16:20:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 -- start start 2023-07-14T16:20:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.573+0000 7f4c4f256b80 1 Processor -- start 2023-07-14T16:20:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d0720 0x5639641dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c4f256b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d1e30 0x5639641e08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639641e0e00 con 0x5639641d1e30 2023-07-14T16:20:51.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639641e0f70 con 0x5639641d0720 2023-07-14T16:20:51.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c4f256b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639641e1130 con 0x5639641d1070 2023-07-14T16:20:51.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c4f256b80 10 client.0 _ll_register_callbacks cb 0x7f4c30099aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:51.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c39f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d1e30 0x5639641e08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c39f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d1e30 0x5639641e08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55650/0 (socket says 172.21.15.62:55650) 2023-07-14T16:20:51.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c38f9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c39f9c700 1 -- 172.21.15.62:0/1182526607 learned_addr learned my addr 172.21.15.62:0/1182526607 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.574+0000 7f4c3979b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d0720 0x5639641dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c3979b700 1 -- 172.21.15.62:0/1182526607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 msgr2=0x5639641dd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c3979b700 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641dd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c3979b700 1 -- 172.21.15.62:0/1182526607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d1e30 msgr2=0x5639641e08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c3979b700 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d1e30 0x5639641e08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c3979b700 1 -- 172.21.15.62:0/1182526607 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c341086e0 con 0x5639641d0720 2023-07-14T16:20:51.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c3979b700 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d0720 0x5639641dd360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4c3009a100 tx=0x7f4c300ca4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=575173beebc79260 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:51.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c1cff9700 1 -- 172.21.15.62:0/1182526607 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300b8040 con 0x5639641d0720 2023-07-14T16:20:51.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c1cff9700 1 -- 172.21.15.62:0/1182526607 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c300c9080 con 0x5639641d0720 2023-07-14T16:20:51.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.575+0000 7f4c4f256b80 1 -- 172.21.15.62:0/1182526607 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c2401e040 con 0x5639641d0720 2023-07-14T16:20:51.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c1cff9700 1 -- 172.21.15.62:0/1182526607 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c300a2390 con 0x5639641d0720 2023-07-14T16:20:51.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c4f256b80 10 client.26642 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:51.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c4f256b80 1 -- 172.21.15.62:0/1182526607 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563963f76250 con 0x5639641d0720 2023-07-14T16:20:51.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c1cff9700 1 -- 172.21.15.62:0/1182526607 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c2401e040 con 0x5639641d0720 2023-07-14T16:20:51.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c1cff9700 1 -- 172.21.15.62:0/1182526607 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4c30018900 con 0x5639641d0720 2023-07-14T16:20:51.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c4f256b80 10 client.26642 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:51.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c4f256b80 10 client.26642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:20:51.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c4f256b80 20 client.26642 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:51.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56396423a760 0x56396423cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c4f256b80 4 client.26642 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:20:51.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.576+0000 7f4c4f256b80 1 -- 172.21.15.62:0/1182526607 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5639640dd050 con 0x56396423a760 2023-07-14T16:20:51.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.577+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56396423a760 0x56396423cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.577+0000 7f4c38f9a700 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56396423a760 0x56396423cb40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=410a3a4108c5631a server_cookie=80e9c895653a89d8 in_seq=0 out_seq=0 2023-07-14T16:20:51.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.577+0000 7f4c1cff9700 10 client.26642 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:20:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c1cff9700 1 -- 172.21.15.62:0/1182526607 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5639640dd050 con 0x56396423a760 2023-07-14T16:20:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c1cff9700 10 client.26642 handle_command_reply: tid=0 2023-07-14T16:20:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 client.26642 shutdown 2023-07-14T16:20:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 -- 172.21.15.62:0/1182526607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d0720 msgr2=0x5639641dd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d0720 0x5639641dd360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4c3009a100 tx=0x7f4c300ca4e0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 -- 172.21.15.62:0/1182526607 shutdown_connections 2023-07-14T16:20:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56396423a760 0x56396423cb40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5639641d1e30 0x5639641e08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639641d0720 0x5639641dd360 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 --2- 172.21.15.62:0/1182526607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5639641d1070 0x5639641dd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c1cff9700 0 client.26642 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:20:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 -- 172.21.15.62:0/1182526607 >> 172.21.15.62:0/1182526607 conn(0x5639641c5ff0 msgr2=0x5639641d8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 -- 172.21.15.62:0/1182526607 shutdown_connections 2023-07-14T16:20:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 1 -- 172.21.15.62:0/1182526607 wait complete. 2023-07-14T16:20:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.578+0000 7f4c4f256b80 20 client.26642 trim_cache size 0 max 16384 2023-07-14T16:20:51.562 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:20:51.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 -- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c200a1270 msgr2=0x7f4c200a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c200a1270 0x7f4c200a3720 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00f540 tx=0x7f4c2c009920 comp rx=0 tx=0).stop 2023-07-14T16:20:51.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 -- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34100450 msgr2=0x7f4c341976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34100450 0x7f4c341976f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000e9a0 tx=0x7f4c3000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 -- 172.21.15.62:0/3489995026 shutdown_connections 2023-07-14T16:20:51.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c200a1270 0x7f4c200a3720 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c34105690 0x7f4c3419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c34100450 0x7f4c341976f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 --2- 172.21.15.62:0/3489995026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c34102d70 0x7f4c34197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 -- 172.21.15.62:0/3489995026 >> 172.21.15.62:0/3489995026 conn(0x7f4c340f9ed0 msgr2=0x7f4c340fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 -- 172.21.15.62:0/3489995026 shutdown_connections 2023-07-14T16:20:51.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.579+0000 7f4c3a79d700 1 -- 172.21.15.62:0/3489995026 wait complete. 2023-07-14T16:20:51.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:20:51.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 -- 172.21.15.62:0/2199926081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62440ffaf0 msgr2=0x7f62440fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 --2- 172.21.15.62:0/2199926081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62440ffaf0 0x7f62440fff00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6240009990 tx=0x7f6240009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 -- 172.21.15.62:0/2199926081 shutdown_connections 2023-07-14T16:20:51.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 --2- 172.21.15.62:0/2199926081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6244101670 0x7f6244107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 --2- 172.21.15.62:0/2199926081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62440ffaf0 0x7f62440fff00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 --2- 172.21.15.62:0/2199926081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6244100ce0 0x7f6244101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 -- 172.21.15.62:0/2199926081 >> 172.21.15.62:0/2199926081 conn(0x7f62440fb130 msgr2=0x7f62440fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 -- 172.21.15.62:0/2199926081 shutdown_connections 2023-07-14T16:20:51.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.833+0000 7f624c5ff700 1 -- 172.21.15.62:0/2199926081 wait complete. 2023-07-14T16:20:51.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.834+0000 7f624c5ff700 1 Processor -- start 2023-07-14T16:20:51.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624c5ff700 1 -- start start 2023-07-14T16:20:51.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624c5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440ffaf0 0x7f62440742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624c5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6244100ce0 0x7f6244072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624c5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6244101670 0x7f6244072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624c5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440735e0 con 0x7f62440ffaf0 2023-07-14T16:20:51.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624c5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244073750 con 0x7f6244101670 2023-07-14T16:20:51.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624c5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624419f810 con 0x7f6244100ce0 2023-07-14T16:20:51.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440ffaf0 0x7f62440742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440ffaf0 0x7f62440742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55666/0 (socket says 172.21.15.62:55666) 2023-07-14T16:20:51.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.835+0000 7f624a39b700 1 -- 172.21.15.62:0/3197058969 learned_addr learned my addr 172.21.15.62:0/3197058969 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f624ab9c700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6244101670 0x7f6244072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f6249b9a700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6244100ce0 0x7f6244072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f624a39b700 1 -- 172.21.15.62:0/3197058969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6244100ce0 msgr2=0x7f6244072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f624a39b700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6244100ce0 0x7f6244072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f624a39b700 1 -- 172.21.15.62:0/3197058969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6244101670 msgr2=0x7f6244072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f624a39b700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6244101670 0x7f6244072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f624a39b700 1 -- 172.21.15.62:0/3197058969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62400095f0 con 0x7f62440ffaf0 2023-07-14T16:20:51.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f624ab9c700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6244101670 0x7f6244072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:51.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.836+0000 7f624a39b700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440ffaf0 0x7f62440742c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f62400098e0 tx=0x7f6240004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.837+0000 7f623b7fe700 1 -- 172.21.15.62:0/3197058969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624001d070 con 0x7f62440ffaf0 2023-07-14T16:20:51.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.837+0000 7f624c5ff700 1 -- 172.21.15.62:0/3197058969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624419fa90 con 0x7f62440ffaf0 2023-07-14T16:20:51.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.837+0000 7f623b7fe700 1 -- 172.21.15.62:0/3197058969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f624000bd50 con 0x7f62440ffaf0 2023-07-14T16:20:51.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.837+0000 7f623b7fe700 1 -- 172.21.15.62:0/3197058969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240022cf0 con 0x7f62440ffaf0 2023-07-14T16:20:51.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.837+0000 7f624c5ff700 1 -- 172.21.15.62:0/3197058969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624419feb0 con 0x7f62440ffaf0 2023-07-14T16:20:51.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.841+0000 7f623b7fe700 1 -- 172.21.15.62:0/3197058969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6240022470 con 0x7f62440ffaf0 2023-07-14T16:20:51.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.842+0000 7f625fe92b80 1 Processor -- start 2023-07-14T16:20:51.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.842+0000 7f625fe92b80 1 -- start start 2023-07-14T16:20:51.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.842+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f1708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.842+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f1716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.842+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f171c70 0x558a1f1720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.842+0000 7f625fe92b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f075cb0 con 0x558a1f170e20 2023-07-14T16:20:51.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.842+0000 7f625fe92b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f0c8a40 con 0x558a1f1704d0 2023-07-14T16:20:51.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.842+0000 7f625fe92b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f0c1ed0 con 0x558a1f171c70 2023-07-14T16:20:51.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f1716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624ab9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f1708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f1716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55668/0 (socket says 172.21.15.62:55668) 2023-07-14T16:20:51.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f6249b9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f171c70 0x558a1f1720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624a39b700 1 -- 172.21.15.62:0/2472764228 learned_addr learned my addr 172.21.15.62:0/2472764228 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f623b7fe700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62300a1150 0x7f62300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624ab9c700 1 -- 172.21.15.62:0/2472764228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f171c70 msgr2=0x558a1f1720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624ab9c700 1 --2- 172.21.15.62:0/2472764228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f171c70 0x558a1f1720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624ab9c700 1 -- 172.21.15.62:0/2472764228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 msgr2=0x558a1f1716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624ab9c700 1 --2- 172.21.15.62:0/2472764228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f1716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624ab9c700 1 -- 172.21.15.62:0/2472764228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a1eeefde0 con 0x558a1f1704d0 2023-07-14T16:20:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f623b7fe700 1 -- 172.21.15.62:0/3197058969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f62400cdf30 con 0x7f62440ffaf0 2023-07-14T16:20:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.843+0000 7f624a39b700 1 --2- 172.21.15.62:0/2472764228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f1716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.844+0000 7f624ab9c700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62300a1150 0x7f62300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.844+0000 7f624ab9c700 1 --2- 172.21.15.62:0/2472764228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f1708e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f623c003e90 tx=0x7f623c004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b163d533dcf8002e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.844+0000 7f6238bfe700 1 -- 172.21.15.62:0/2472764228 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c007e70 con 0x558a1f1704d0 2023-07-14T16:20:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.844+0000 7f6238bfe700 1 -- 172.21.15.62:0/2472764228 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f623c00f040 con 0x558a1f1704d0 2023-07-14T16:20:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.844+0000 7f6238bfe700 1 -- 172.21.15.62:0/2472764228 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c018680 con 0x558a1f1704d0 2023-07-14T16:20:51.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.844+0000 7f624ab9c700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62300a1150 0x7f62300a3600 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f623c022000 tx=0x7f623c01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 -- 172.21.15.62:0/2472764228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 msgr2=0x558a1f1708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 --2- 172.21.15.62:0/2472764228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f1708e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f623c003e90 tx=0x7f623c004320 comp rx=0 tx=0).stop 2023-07-14T16:20:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 -- 172.21.15.62:0/2472764228 shutdown_connections 2023-07-14T16:20:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 --2- 172.21.15.62:0/2472764228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f171c70 0x558a1f1720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 --2- 172.21.15.62:0/2472764228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f1708e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 --2- 172.21.15.62:0/2472764228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f1716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 -- 172.21.15.62:0/2472764228 >> 172.21.15.62:0/2472764228 conn(0x558a1f165da0 msgr2=0x558a1f16b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 -- 172.21.15.62:0/2472764228 shutdown_connections 2023-07-14T16:20:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.845+0000 7f625fe92b80 1 -- 172.21.15.62:0/2472764228 wait complete. 2023-07-14T16:20:51.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.846+0000 7f625fe92b80 1 -- start start 2023-07-14T16:20:51.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.846+0000 7f625fe92b80 1 Processor -- start 2023-07-14T16:20:51.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 0x558a1f17a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f17a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f171c70 0x558a1f1816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f625fe92b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f061ac0 con 0x558a1f170e20 2023-07-14T16:20:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f625fe92b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f0617c0 con 0x558a1f171c70 2023-07-14T16:20:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f625fe92b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1ef45920 con 0x558a1f1704d0 2023-07-14T16:20:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f625fe92b80 10 client.0 _ll_register_callbacks cb 0x558a1f05f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f6249b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f17a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f6249b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f17a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55684/0 (socket says 172.21.15.62:55684) 2023-07-14T16:20:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f6249b9a700 1 -- 172.21.15.62:0/3600601382 learned_addr learned my addr 172.21.15.62:0/3600601382 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f624ab9c700 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f171c70 0x558a1f1816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.847+0000 7f624a39b700 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 0x558a1f17a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f6249b9a700 1 -- 172.21.15.62:0/3600601382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 msgr2=0x558a1f17a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f6249b9a700 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 0x558a1f17a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f6249b9a700 1 -- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f171c70 msgr2=0x558a1f1816b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f6249b9a700 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f171c70 0x558a1f1816b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f6249b9a700 1 -- 172.21.15.62:0/3600601382 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624419feb0 con 0x558a1f170e20 2023-07-14T16:20:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f624ab9c700 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f171c70 0x558a1f1816b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f6249b9a700 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f17a670 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6234011830 tx=0x7f6234011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f64c57904305e9b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f622dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f622dffb700 1 -- 172.21.15.62:0/3600601382 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234016070 con 0x558a1f170e20 2023-07-14T16:20:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f625fe92b80 1 -- 172.21.15.62:0/3600601382 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a1f01b4f0 con 0x558a1f170e20 2023-07-14T16:20:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.848+0000 7f622dffb700 1 -- 172.21.15.62:0/3600601382 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f623400f7d0 con 0x558a1f170e20 2023-07-14T16:20:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.849+0000 7f622dffb700 1 -- 172.21.15.62:0/3600601382 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f623400f990 con 0x558a1f170e20 2023-07-14T16:20:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.849+0000 7f625fe92b80 10 client.16950 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.849+0000 7f625fe92b80 1 -- 172.21.15.62:0/3600601382 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f623c003ab0 con 0x558a1f170e20 2023-07-14T16:20:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.849+0000 7f622dffb700 1 -- 172.21.15.62:0/3600601382 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558a1f01b4f0 con 0x558a1f170e20 2023-07-14T16:20:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.849+0000 7f622dffb700 1 -- 172.21.15.62:0/3600601382 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f623401aae0 con 0x558a1f170e20 2023-07-14T16:20:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.850+0000 7f625fe92b80 10 client.16950 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.850+0000 7f625fe92b80 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.850+0000 7f625fe92b80 20 client.16950 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.850+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a1f186060 0x558a1f188470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.850+0000 7f625fe92b80 4 client.16950 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.850+0000 7f625fe92b80 1 -- 172.21.15.62:0/3600601382 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a1f0baeb0 con 0x558a1f186060 2023-07-14T16:20:51.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.850+0000 7f624a39b700 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a1f186060 0x558a1f188470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.851+0000 7f624a39b700 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a1f186060 0x558a1f188470 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8142c04985b381e1 server_cookie=82572dc3c9a09640 in_seq=0 out_seq=0 2023-07-14T16:20:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.851+0000 7f622dffb700 10 client.16950 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f622dffb700 1 -- 172.21.15.62:0/3600601382 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a1f0baeb0 con 0x558a1f186060 2023-07-14T16:20:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f622dffb700 10 client.16950 handle_command_reply: tid=0 2023-07-14T16:20:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 client.16950 shutdown 2023-07-14T16:20:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 -- 172.21.15.62:0/3600601382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 msgr2=0x558a1f17a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f17a670 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6234011830 tx=0x7f6234011b40 comp rx=0 tx=0).stop 2023-07-14T16:20:51.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 -- 172.21.15.62:0/3600601382 shutdown_connections 2023-07-14T16:20:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a1f186060 0x558a1f188470 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f171c70 0x558a1f1816b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 0x558a1f17a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3600601382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f170e20 0x558a1f17a670 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 -- 172.21.15.62:0/3600601382 >> 172.21.15.62:0/3600601382 conn(0x558a1f165da0 msgr2=0x558a1f16a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f622dffb700 0 client.16950 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 -- 172.21.15.62:0/3600601382 shutdown_connections 2023-07-14T16:20:51.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.852+0000 7f625fe92b80 1 -- 172.21.15.62:0/3600601382 wait complete. 2023-07-14T16:20:51.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.853+0000 7f625fe92b80 20 client.16950 trim_cache size 0 max 16384 2023-07-14T16:20:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.867+0000 7f625fe92b80 1 Processor -- start 2023-07-14T16:20:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.867+0000 7f625fe92b80 1 -- start start 2023-07-14T16:20:51.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 0x558a1f178080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f170e20 0x558a1f181670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f17a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f625fe92b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f171570 con 0x558a1f171c70 2023-07-14T16:20:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f625fe92b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f16c4c0 con 0x558a1f170e20 2023-07-14T16:20:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f625fe92b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f179ca0 con 0x558a1f1704d0 2023-07-14T16:20:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f624ab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f17a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f624ab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f17a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55686/0 (socket says 172.21.15.62:55686) 2023-07-14T16:20:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f624ab9c700 1 -- 172.21.15.62:0/65140424 learned_addr learned my addr 172.21.15.62:0/65140424 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f624a39b700 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 0x558a1f178080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.868+0000 7f6249b9a700 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f170e20 0x558a1f181670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f624ab9c700 1 -- 172.21.15.62:0/65140424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 msgr2=0x558a1f178080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f624ab9c700 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 0x558a1f178080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f624ab9c700 1 -- 172.21.15.62:0/65140424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f170e20 msgr2=0x558a1f181670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f624ab9c700 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f170e20 0x558a1f181670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f624ab9c700 1 -- 172.21.15.62:0/65140424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a1eed8c60 con 0x558a1f171c70 2023-07-14T16:20:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f624ab9c700 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f17a130 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f623c018810 tx=0x7f623c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dee2defa85b8824 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f62393ff700 1 -- 172.21.15.62:0/65140424 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c02fa10 con 0x558a1f171c70 2023-07-14T16:20:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f62393ff700 1 -- 172.21.15.62:0/65140424 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f623c0049a0 con 0x558a1f171c70 2023-07-14T16:20:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.869+0000 7f62393ff700 1 -- 172.21.15.62:0/65140424 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c018ca0 con 0x558a1f171c70 2023-07-14T16:20:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 -- 172.21.15.62:0/65140424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 msgr2=0x558a1f17a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f17a130 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f623c018810 tx=0x7f623c009520 comp rx=0 tx=0).stop 2023-07-14T16:20:51.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 -- 172.21.15.62:0/65140424 shutdown_connections 2023-07-14T16:20:51.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f17a130 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f1704d0 0x558a1f178080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 --2- 172.21.15.62:0/65140424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f170e20 0x558a1f181670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 -- 172.21.15.62:0/65140424 >> 172.21.15.62:0/65140424 conn(0x558a1f165da0 msgr2=0x558a1f178e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 -- 172.21.15.62:0/65140424 shutdown_connections 2023-07-14T16:20:51.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.870+0000 7f625fe92b80 1 -- 172.21.15.62:0/65140424 wait complete. 2023-07-14T16:20:51.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.871+0000 7f625fe92b80 1 -- start start 2023-07-14T16:20:51.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.871+0000 7f625fe92b80 1 Processor -- start 2023-07-14T16:20:51.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f17d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f170e20 0x558a1f17dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f625fe92b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f180ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f625fe92b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f181010 con 0x558a1f171c70 2023-07-14T16:20:51.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f625fe92b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f181150 con 0x558a1f1704d0 2023-07-14T16:20:51.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f625fe92b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1f181310 con 0x558a1f170e20 2023-07-14T16:20:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f625fe92b80 10 client.0 _ll_register_callbacks cb 0x7f6234009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f624a39b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f17d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f624a39b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f17d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43298/0 (socket says 172.21.15.62:43298) 2023-07-14T16:20:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f624a39b700 1 -- 172.21.15.62:0/3191088903 learned_addr learned my addr 172.21.15.62:0/3191088903 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f624ab9c700 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f180ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.872+0000 7f6249b9a700 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f170e20 0x558a1f17dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f624a39b700 1 -- 172.21.15.62:0/3191088903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f170e20 msgr2=0x558a1f17dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f624a39b700 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f170e20 0x558a1f17dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f624a39b700 1 -- 172.21.15.62:0/3191088903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 msgr2=0x558a1f180ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f624a39b700 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f180ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f624a39b700 1 -- 172.21.15.62:0/3191088903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c027070 con 0x558a1f1704d0 2023-07-14T16:20:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f6249b9a700 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f170e20 0x558a1f17dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f624a39b700 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f17d8a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6240003ec0 tx=0x7f62400d19d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54abe4cbefc5e46b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f622dffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f622dffb700 1 -- 172.21.15.62:0/3191088903 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624009b1b0 con 0x558a1f1704d0 2023-07-14T16:20:51.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f625fe92b80 1 -- 172.21.15.62:0/3191088903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f623c00f040 con 0x558a1f1704d0 2023-07-14T16:20:51.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.873+0000 7f622dffb700 1 -- 172.21.15.62:0/3191088903 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f62400a37a0 con 0x558a1f1704d0 2023-07-14T16:20:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f622dffb700 1 -- 172.21.15.62:0/3191088903 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f624009af20 con 0x558a1f1704d0 2023-07-14T16:20:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f625fe92b80 10 client.26660 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f625fe92b80 1 -- 172.21.15.62:0/3191088903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558a1ef16250 con 0x558a1f1704d0 2023-07-14T16:20:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f622dffb700 1 -- 172.21.15.62:0/3191088903 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62400c9080 con 0x558a1f1704d0 2023-07-14T16:20:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f622dffb700 1 -- 172.21.15.62:0/3191088903 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f624009a8d0 con 0x558a1f1704d0 2023-07-14T16:20:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f625fe92b80 10 client.26660 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f625fe92b80 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f625fe92b80 20 client.26660 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a1f1dac50 0x558a1f1dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f625fe92b80 4 client.26660 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:20:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.874+0000 7f625fe92b80 1 -- 172.21.15.62:0/3191088903 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a1f07d050 con 0x558a1f1dac50 2023-07-14T16:20:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.875+0000 7f6249b9a700 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a1f1dac50 0x558a1f1dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.875+0000 7f6249b9a700 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a1f1dac50 0x558a1f1dd030 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=973e45b095adea7 server_cookie=e68670956e7cca50 in_seq=0 out_seq=0 2023-07-14T16:20:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.875+0000 7f622dffb700 10 client.26660 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.875+0000 7f622dffb700 1 -- 172.21.15.62:0/3191088903 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a1f07d050 con 0x558a1f1dac50 2023-07-14T16:20:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f622dffb700 10 client.26660 handle_command_reply: tid=0 2023-07-14T16:20:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 client.26660 shutdown 2023-07-14T16:20:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 -- 172.21.15.62:0/3191088903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 msgr2=0x558a1f17d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f17d8a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6240003ec0 tx=0x7f62400d19d0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 -- 172.21.15.62:0/3191088903 shutdown_connections 2023-07-14T16:20:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a1f171c70 0x558a1f180ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a1f1dac50 0x558a1f1dd030 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a1f1704d0 0x558a1f17d8a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 --2- 172.21.15.62:0/3191088903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a1f170e20 0x558a1f17dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f622dffb700 0 client.26660 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:51.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 -- 172.21.15.62:0/3191088903 >> 172.21.15.62:0/3191088903 conn(0x558a1f165da0 msgr2=0x558a1f178080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 -- 172.21.15.62:0/3191088903 shutdown_connections 2023-07-14T16:20:51.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 1 -- 172.21.15.62:0/3191088903 wait complete. 2023-07-14T16:20:51.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.876+0000 7f625fe92b80 20 client.26660 trim_cache size 0 max 16384 2023-07-14T16:20:51.861 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:20:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 -- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62300a1150 msgr2=0x7f62300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62300a1150 0x7f62300a3600 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f623c022000 tx=0x7f623c01f000 comp rx=0 tx=0).stop 2023-07-14T16:20:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 -- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440ffaf0 msgr2=0x7f62440742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440ffaf0 0x7f62440742c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f62400098e0 tx=0x7f6240004730 comp rx=0 tx=0).stop 2023-07-14T16:20:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 -- 172.21.15.62:0/3197058969 shutdown_connections 2023-07-14T16:20:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62300a1150 0x7f62300a3600 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6244101670 0x7f6244072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440ffaf0 0x7f62440742c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 --2- 172.21.15.62:0/3197058969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6244100ce0 0x7f6244072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 -- 172.21.15.62:0/3197058969 >> 172.21.15.62:0/3197058969 conn(0x7f62440fb130 msgr2=0x7f62440fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 -- 172.21.15.62:0/3197058969 shutdown_connections 2023-07-14T16:20:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:51.877+0000 7f624c5ff700 1 -- 172.21.15.62:0/3197058969 wait complete. 2023-07-14T16:20:52.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:51 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3829449597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:20:52.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:51 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3829449597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:20:52.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:51 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3829449597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:20:52.874 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:20:52.875 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:20:53.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:52 smithi164 ceph-mon[97427]: pgmap v514: 97 pgs: 97 active+clean; 2.9 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:20:53.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 -- 172.21.15.62:0/3039653177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 msgr2=0x7fb740102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 --2- 172.21.15.62:0/3039653177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 0x7fb740102010 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb730009990 tx=0x7fb730009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 -- 172.21.15.62:0/3039653177 shutdown_connections 2023-07-14T16:20:53.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 --2- 172.21.15.62:0/3039653177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb740103780 0x7fb74010a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 --2- 172.21.15.62:0/3039653177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 0x7fb740102010 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 --2- 172.21.15.62:0/3039653177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb740102df0 0x7fb740103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 -- 172.21.15.62:0/3039653177 >> 172.21.15.62:0/3039653177 conn(0x7fb7400fd220 msgr2=0x7fb7400ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 -- 172.21.15.62:0/3039653177 shutdown_connections 2023-07-14T16:20:53.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.137+0000 7fb74519d700 1 -- 172.21.15.62:0/3039653177 wait complete. 2023-07-14T16:20:53.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.139+0000 7fb74519d700 1 Processor -- start 2023-07-14T16:20:53.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.139+0000 7fb74519d700 1 -- start start 2023-07-14T16:20:53.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.139+0000 7fb74519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 0x7fb740197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.139+0000 7fb74519d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb740102df0 0x7fb740197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.139+0000 7fb74519d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb740103780 0x7fb74019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.139+0000 7fb74519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74019f2b0 con 0x7fb740101c00 2023-07-14T16:20:53.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.139+0000 7fb74519d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74019f3f0 con 0x7fb740103780 2023-07-14T16:20:53.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.139+0000 7fb74519d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74019f580 con 0x7fb740102df0 2023-07-14T16:20:53.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 0x7fb740197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb740102df0 0x7fb740197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb740102df0 0x7fb740197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39870/0 (socket says 172.21.15.62:39870) 2023-07-14T16:20:53.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73f7fe700 1 -- 172.21.15.62:0/1935796304 learned_addr learned my addr 172.21.15.62:0/1935796304 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb74499c700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb740103780 0x7fb74019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73f7fe700 1 -- 172.21.15.62:0/1935796304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb740103780 msgr2=0x7fb74019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb740103780 0x7fb74019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73f7fe700 1 -- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 msgr2=0x7fb740197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 0x7fb740197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73f7fe700 1 -- 172.21.15.62:0/1935796304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300095f0 con 0x7fb740102df0 2023-07-14T16:20:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb74499c700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb740103780 0x7fb74019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.140+0000 7fb73ffff700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 0x7fb740197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:53.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.141+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb740102df0 0x7fb740197c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb73400e9a0 tx=0x7fb73400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.141+0000 7fb73d7fa700 1 -- 172.21.15.62:0/1935796304 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734009790 con 0x7fb740102df0 2023-07-14T16:20:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.141+0000 7fb74519d700 1 -- 172.21.15.62:0/1935796304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74019f860 con 0x7fb740102df0 2023-07-14T16:20:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.141+0000 7fb73d7fa700 1 -- 172.21.15.62:0/1935796304 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb73400cbd0 con 0x7fb740102df0 2023-07-14T16:20:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.141+0000 7fb73d7fa700 1 -- 172.21.15.62:0/1935796304 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734018680 con 0x7fb740102df0 2023-07-14T16:20:53.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.141+0000 7fb74519d700 1 -- 172.21.15.62:0/1935796304 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74019fdc0 con 0x7fb740102df0 2023-07-14T16:20:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.145+0000 7fb73d7fa700 1 -- 172.21.15.62:0/1935796304 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb73400cd40 con 0x7fb740102df0 2023-07-14T16:20:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.146+0000 7fb759df3b80 1 Processor -- start 2023-07-14T16:20:53.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.146+0000 7fb759df3b80 1 -- start start 2023-07-14T16:20:53.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.146+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.146+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.146+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c978c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.146+0000 7fb759df3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c977cad30 con 0x564c978c57a0 2023-07-14T16:20:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.146+0000 7fb759df3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c9781dac0 con 0x564c978c60f0 2023-07-14T16:20:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.146+0000 7fb759df3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c97816f50 con 0x564c978c6eb0 2023-07-14T16:20:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb74499c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb74499c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40388/0 (socket says 172.21.15.62:40388) 2023-07-14T16:20:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb74499c700 1 -- 172.21.15.62:0/3790369510 learned_addr learned my addr 172.21.15.62:0/3790369510 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c978c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb73ffff700 1 -- 172.21.15.62:0/3790369510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 msgr2=0x564c978c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c978c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb73ffff700 1 -- 172.21.15.62:0/3790369510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 msgr2=0x564c978c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb73ffff700 1 -- 172.21.15.62:0/3790369510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c97644e60 con 0x564c978c60f0 2023-07-14T16:20:53.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:52 smithi062 ceph-mon[95543]: pgmap v514: 97 pgs: 97 active+clean; 2.9 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:20:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.147+0000 7fb74499c700 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.148+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978c6920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb730003f60 tx=0x7fb730005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ad37c4979bced82 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.148+0000 7fb73d7fa700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7280a11f0 0x7fb7280a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.148+0000 7fb73d7fa700 1 -- 172.21.15.62:0/1935796304 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb734014070 con 0x7fb740102df0 2023-07-14T16:20:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.148+0000 7fb74499c700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7280a11f0 0x7fb7280a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.148+0000 7fb7267fc700 1 -- 172.21.15.62:0/3790369510 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73001d050 con 0x564c978c60f0 2023-07-14T16:20:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.148+0000 7fb7267fc700 1 -- 172.21.15.62:0/3790369510 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb73001e040 con 0x564c978c60f0 2023-07-14T16:20:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.148+0000 7fb7267fc700 1 -- 172.21.15.62:0/3790369510 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730005290 con 0x564c978c60f0 2023-07-14T16:20:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.149+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790369510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 msgr2=0x564c978c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.149+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978c6920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb730003f60 tx=0x7fb730005c00 comp rx=0 tx=0).stop 2023-07-14T16:20:53.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.149+0000 7fb74499c700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7280a11f0 0x7fb7280a36a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb7380076f0 tx=0x7fb738009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.149+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790369510 shutdown_connections 2023-07-14T16:20:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.149+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c978c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.149+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.149+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3790369510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978c6920 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.149+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790369510 >> 172.21.15.62:0/3790369510 conn(0x564c978bb070 msgr2=0x564c978c04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.150+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790369510 shutdown_connections 2023-07-14T16:20:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.150+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790369510 wait complete. 2023-07-14T16:20:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.150+0000 7fb759df3b80 1 -- start start 2023-07-14T16:20:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.150+0000 7fb759df3b80 1 Processor -- start 2023-07-14T16:20:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c978d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb759df3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c977b6b40 con 0x564c978c57a0 2023-07-14T16:20:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb759df3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c977b6840 con 0x564c978c60f0 2023-07-14T16:20:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb759df3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c9769a9a0 con 0x564c978c6eb0 2023-07-14T16:20:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb759df3b80 10 client.0 _ll_register_callbacks cb 0x564c977b47e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40404/0 (socket says 172.21.15.62:40404) 2023-07-14T16:20:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb73ffff700 1 -- 172.21.15.62:0/3765779882 learned_addr learned my addr 172.21.15.62:0/3765779882 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.151+0000 7fb74499c700 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c978d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.152+0000 7fb73ffff700 1 -- 172.21.15.62:0/3765779882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 msgr2=0x564c978d68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.152+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c978d68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.152+0000 7fb73ffff700 1 -- 172.21.15.62:0/3765779882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 msgr2=0x564c978cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.152+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.152+0000 7fb73ffff700 1 -- 172.21.15.62:0/3765779882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb738009a80 con 0x564c978c57a0 2023-07-14T16:20:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.152+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978cf320 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb7300098e0 tx=0x7fb730005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb986521d5e1f00 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.152+0000 7fb71f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:53.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.152+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3765779882 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730005030 con 0x564c978c57a0 2023-07-14T16:20:53.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb759df3b80 1 -- 172.21.15.62:0/3765779882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c97770570 con 0x564c978c57a0 2023-07-14T16:20:53.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3765779882 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb730005190 con 0x564c978c57a0 2023-07-14T16:20:53.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3765779882 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb73001d020 con 0x564c978c57a0 2023-07-14T16:20:53.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb759df3b80 10 client.16968 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:53.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb759df3b80 1 -- 172.21.15.62:0/3765779882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb74010a970 con 0x564c978c57a0 2023-07-14T16:20:53.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3765779882 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb73003b020 con 0x564c978c57a0 2023-07-14T16:20:53.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3765779882 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb73000f460 con 0x564c978c57a0 2023-07-14T16:20:53.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb759df3b80 10 client.16968 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:53.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.153+0000 7fb759df3b80 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:53.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.154+0000 7fb759df3b80 20 client.16968 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:53.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.154+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564c978dab80 0x564c978dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.154+0000 7fb759df3b80 4 client.16968 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:53.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.154+0000 7fb759df3b80 1 -- 172.21.15.62:0/3765779882 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c9780ff30 con 0x564c978dab80 2023-07-14T16:20:53.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.154+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564c978dab80 0x564c978dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.155+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564c978dab80 0x564c978dcf90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1109b008e28f6d server_cookie=c89a375266e6939b in_seq=0 out_seq=0 2023-07-14T16:20:53.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.155+0000 7fb71f7fe700 10 client.16968 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.155+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3765779882 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c9780ff30 con 0x564c978dab80 2023-07-14T16:20:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.155+0000 7fb71f7fe700 10 client.16968 handle_command_reply: tid=0 2023-07-14T16:20:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 client.16968 shutdown 2023-07-14T16:20:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 -- 172.21.15.62:0/3765779882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 msgr2=0x564c978cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978cf320 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb7300098e0 tx=0x7fb730005910 comp rx=0 tx=0).stop 2023-07-14T16:20:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 -- 172.21.15.62:0/3765779882 shutdown_connections 2023-07-14T16:20:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c978d68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978cf320 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564c978dab80 0x564c978dcf90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3765779882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 -- 172.21.15.62:0/3765779882 >> 172.21.15.62:0/3765779882 conn(0x564c978bb070 msgr2=0x564c978bfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb71f7fe700 0 client.16968 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 -- 172.21.15.62:0/3765779882 shutdown_connections 2023-07-14T16:20:53.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 1 -- 172.21.15.62:0/3765779882 wait complete. 2023-07-14T16:20:53.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.156+0000 7fb759df3b80 20 client.16968 trim_cache size 0 max 16384 2023-07-14T16:20:53.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:52 smithi081 ceph-mon[127795]: pgmap v514: 97 pgs: 97 active+clean; 2.9 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:20:53.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.184+0000 7fb759df3b80 1 Processor -- start 2023-07-14T16:20:53.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.184+0000 7fb759df3b80 1 -- start start 2023-07-14T16:20:53.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c60f0 0x564c978ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c6eb0 0x564c978d6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb759df3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c978ceef0 con 0x564c978c57a0 2023-07-14T16:20:53.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb759df3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c978da9b0 con 0x564c978c6eb0 2023-07-14T16:20:53.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb759df3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c978cdd60 con 0x564c978c60f0 2023-07-14T16:20:53.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb74499c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c6eb0 0x564c978d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40420/0 (socket says 172.21.15.62:40420) 2023-07-14T16:20:53.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb73ffff700 1 -- 172.21.15.62:0/3790194721 learned_addr learned my addr 172.21.15.62:0/3790194721 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c60f0 0x564c978ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb74499c700 1 -- 172.21.15.62:0/3790194721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c60f0 msgr2=0x564c978ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb74499c700 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c60f0 0x564c978ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb74499c700 1 -- 172.21.15.62:0/3790194721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 msgr2=0x564c978c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.185+0000 7fb74499c700 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.186+0000 7fb74499c700 1 -- 172.21.15.62:0/3790194721 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c978c7550 con 0x564c978c6eb0 2023-07-14T16:20:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.186+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c60f0 0x564c978ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:53.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.186+0000 7fb74499c700 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c6eb0 0x564c978d6860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb738009c60 tx=0x7fb738015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf586fa810b651ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.186+0000 7fb726ffd700 1 -- 172.21.15.62:0/3790194721 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738003c00 con 0x564c978c6eb0 2023-07-14T16:20:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.186+0000 7fb726ffd700 1 -- 172.21.15.62:0/3790194721 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb73801c070 con 0x564c978c6eb0 2023-07-14T16:20:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.186+0000 7fb726ffd700 1 -- 172.21.15.62:0/3790194721 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738016680 con 0x564c978c6eb0 2023-07-14T16:20:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790194721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c6eb0 msgr2=0x564c978d6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c6eb0 0x564c978d6860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb738009c60 tx=0x7fb738015000 comp rx=0 tx=0).stop 2023-07-14T16:20:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790194721 shutdown_connections 2023-07-14T16:20:53.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c6eb0 0x564c978d6860 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3790194721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c60f0 0x564c978ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790194721 >> 172.21.15.62:0/3790194721 conn(0x564c978bb070 msgr2=0x564c978d7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790194721 shutdown_connections 2023-07-14T16:20:53.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.187+0000 7fb759df3b80 1 -- 172.21.15.62:0/3790194721 wait complete. 2023-07-14T16:20:53.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 1 -- start start 2023-07-14T16:20:53.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 1 Processor -- start 2023-07-14T16:20:53.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c9792ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c978d40d0 con 0x564c978c57a0 2023-07-14T16:20:53.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c978d4240 con 0x564c978c60f0 2023-07-14T16:20:53.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c9792ef90 con 0x564c978c6eb0 2023-07-14T16:20:53.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.188+0000 7fb759df3b80 10 client.0 _ll_register_callbacks cb 0x7fb738009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:53.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978d3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40424/0 (socket says 172.21.15.62:40424) 2023-07-14T16:20:53.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 -- 172.21.15.62:0/3501017964 learned_addr learned my addr 172.21.15.62:0/3501017964 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb74499c700 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c9792ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 -- 172.21.15.62:0/3501017964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 msgr2=0x564c9792ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c9792ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 -- 172.21.15.62:0/3501017964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 msgr2=0x564c978d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978d3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 -- 172.21.15.62:0/3501017964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb738018470 con 0x564c978c57a0 2023-07-14T16:20:53.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:53.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb73ffff700 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978d3650 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb73000f790 tx=0x7fb73000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28e32a161060b87 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.189+0000 7fb71f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:53.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.190+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3501017964 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73001d020 con 0x564c978c57a0 2023-07-14T16:20:53.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.190+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3501017964 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb730000e40 con 0x564c978c57a0 2023-07-14T16:20:53.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.190+0000 7fb759df3b80 1 -- 172.21.15.62:0/3501017964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb73001e040 con 0x564c978c57a0 2023-07-14T16:20:53.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.190+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3501017964 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb730005bc0 con 0x564c978c57a0 2023-07-14T16:20:53.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.190+0000 7fb759df3b80 10 client.16974 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:53.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.190+0000 7fb759df3b80 1 -- 172.21.15.62:0/3501017964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564c9762dce0 con 0x564c978c57a0 2023-07-14T16:20:53.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.190+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3501017964 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb730046020 con 0x564c978c57a0 2023-07-14T16:20:53.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.191+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3501017964 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb73000bcd0 con 0x564c978c57a0 2023-07-14T16:20:53.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.191+0000 7fb759df3b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:53.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.191+0000 7fb759df3b80 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:53.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.191+0000 7fb759df3b80 20 client.16974 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:53.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.191+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564c97930570 0x564c97932980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.191+0000 7fb759df3b80 4 client.16974 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:20:53.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.191+0000 7fb759df3b80 1 -- 172.21.15.62:0/3501017964 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564c977d20d0 con 0x564c97930570 2023-07-14T16:20:53.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.191+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564c97930570 0x564c97932980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.192+0000 7fb73f7fe700 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564c97930570 0x564c97932980 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fcc7def558e5800 server_cookie=9f07b807eb69b8fb in_seq=0 out_seq=0 2023-07-14T16:20:53.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.192+0000 7fb71f7fe700 10 client.16974 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb71f7fe700 1 -- 172.21.15.62:0/3501017964 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564c977d20d0 con 0x564c97930570 2023-07-14T16:20:53.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb71f7fe700 10 client.16974 handle_command_reply: tid=0 2023-07-14T16:20:53.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 client.16974 shutdown 2023-07-14T16:20:53.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 -- 172.21.15.62:0/3501017964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 msgr2=0x564c978d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978d3650 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb73000f790 tx=0x7fb73000f690 comp rx=0 tx=0).stop 2023-07-14T16:20:53.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 -- 172.21.15.62:0/3501017964 shutdown_connections 2023-07-14T16:20:53.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564c978c6eb0 0x564c9792ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564c978c57a0 0x564c978d3650 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564c97930570 0x564c97932980 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 --2- 172.21.15.62:0/3501017964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564c978c60f0 0x564c978d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 -- 172.21.15.62:0/3501017964 >> 172.21.15.62:0/3501017964 conn(0x564c978bb070 msgr2=0x564c978c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb71f7fe700 0 client.16974 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 -- 172.21.15.62:0/3501017964 shutdown_connections 2023-07-14T16:20:53.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.193+0000 7fb759df3b80 1 -- 172.21.15.62:0/3501017964 wait complete. 2023-07-14T16:20:53.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb759df3b80 20 client.16974 trim_cache size 0 max 16384 2023-07-14T16:20:53.178 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:20:53.178 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:20:53.178 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "c9e1915b-d5d7-47f7-aae9-3a87062e3dc8", 2023-07-14T16:20:53.178 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:20:53.178 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:20:53.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 -- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7280a11f0 msgr2=0x7fb7280a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7280a11f0 0x7fb7280a36a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb7380076f0 tx=0x7fb738009040 comp rx=0 tx=0).stop 2023-07-14T16:20:53.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 -- 172.21.15.62:0/1935796304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb740102df0 msgr2=0x7fb740197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb740102df0 0x7fb740197c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb73400e9a0 tx=0x7fb73400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 -- 172.21.15.62:0/1935796304 shutdown_connections 2023-07-14T16:20:53.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7280a11f0 0x7fb7280a36a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb740103780 0x7fb74019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb740101c00 0x7fb740197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 --2- 172.21.15.62:0/1935796304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb740102df0 0x7fb740197c70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.194+0000 7fb74519d700 1 -- 172.21.15.62:0/1935796304 >> 172.21.15.62:0/1935796304 conn(0x7fb7400fd220 msgr2=0x7fb7400ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.195+0000 7fb74519d700 1 -- 172.21.15.62:0/1935796304 shutdown_connections 2023-07-14T16:20:53.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.195+0000 7fb74519d700 1 -- 172.21.15.62:0/1935796304 wait complete. 2023-07-14T16:20:53.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:20:53.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 -- 172.21.15.62:0/2947953877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41004b0 msgr2=0x7f9fc4102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 --2- 172.21.15.62:0/2947953877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41004b0 0x7f9fc4102890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009990 tx=0x7f9fb4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 -- 172.21.15.62:0/2947953877 shutdown_connections 2023-07-14T16:20:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 --2- 172.21.15.62:0/2947953877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fc41056f0 0x7f9fc4107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 --2- 172.21.15.62:0/2947953877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41004b0 0x7f9fc4102890 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 --2- 172.21.15.62:0/2947953877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4102dd0 0x7f9fc41051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 -- 172.21.15.62:0/2947953877 >> 172.21.15.62:0/2947953877 conn(0x7f9fc40f9ed0 msgr2=0x7f9fc40fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 -- 172.21.15.62:0/2947953877 shutdown_connections 2023-07-14T16:20:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.462+0000 7f9fc359e700 1 -- 172.21.15.62:0/2947953877 wait complete. 2023-07-14T16:20:53.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.463+0000 7f9fc359e700 1 Processor -- start 2023-07-14T16:20:53.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.463+0000 7f9fc359e700 1 -- start start 2023-07-14T16:20:53.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc359e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fc41004b0 0x7f9fc41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4102dd0 0x7f9fc4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc359e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41056f0 0x7f9fc419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc419f260 con 0x7f9fc4102dd0 2023-07-14T16:20:53.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc359e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc419f3a0 con 0x7f9fc41004b0 2023-07-14T16:20:53.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc359e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc419f530 con 0x7f9fc41056f0 2023-07-14T16:20:53.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fc41004b0 0x7f9fc41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4102dd0 0x7f9fc4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fc41004b0 0x7f9fc41976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52184/0 (socket says 172.21.15.62:52184) 2023-07-14T16:20:53.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc259c700 1 -- 172.21.15.62:0/52100238 learned_addr learned my addr 172.21.15.62:0/52100238 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.464+0000 7f9fc2d9d700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41056f0 0x7f9fc419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fc259c700 1 -- 172.21.15.62:0/52100238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41056f0 msgr2=0x7f9fc419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fc259c700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41056f0 0x7f9fc419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fc259c700 1 -- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4102dd0 msgr2=0x7f9fc4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fc259c700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4102dd0 0x7f9fc4197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fc259c700 1 -- 172.21.15.62:0/52100238 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40095f0 con 0x7f9fc41004b0 2023-07-14T16:20:53.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4102dd0 0x7f9fc4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:53.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fc259c700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fc41004b0 0x7f9fc41976e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40095c0 tx=0x7f9fb4003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fb37fe700 1 -- 172.21.15.62:0/52100238 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb401d070 con 0x7f9fc41004b0 2023-07-14T16:20:53.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fb37fe700 1 -- 172.21.15.62:0/52100238 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fb4003da0 con 0x7f9fc41004b0 2023-07-14T16:20:53.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fc359e700 1 -- 172.21.15.62:0/52100238 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc419f7b0 con 0x7f9fc41004b0 2023-07-14T16:20:53.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.465+0000 7f9fb37fe700 1 -- 172.21.15.62:0/52100238 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4005280 con 0x7f9fc41004b0 2023-07-14T16:20:53.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.466+0000 7f9fc359e700 1 -- 172.21.15.62:0/52100238 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc419fb70 con 0x7f9fc41004b0 2023-07-14T16:20:53.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.470+0000 7f9fb37fe700 1 -- 172.21.15.62:0/52100238 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9fb40053e0 con 0x7f9fc41004b0 2023-07-14T16:20:53.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.470+0000 7f9fdc3c1b80 1 Processor -- start 2023-07-14T16:20:53.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.471+0000 7f9fdc3c1b80 1 -- start start 2023-07-14T16:20:53.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.471+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.471+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.471+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1e30 0x55f73f3c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.471+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f2c5cb0 con 0x55f73f3c1e30 2023-07-14T16:20:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.471+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f318a40 con 0x55f73f3c0720 2023-07-14T16:20:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.471+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f311ed0 con 0x55f73f3c1070 2023-07-14T16:20:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.471+0000 7f9fc259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1e30 0x55f73f3c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39948/0 (socket says 172.21.15.62:39948) 2023-07-14T16:20:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc259c700 1 -- 172.21.15.62:0/785696388 learned_addr learned my addr 172.21.15.62:0/785696388 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc2d9d700 1 --2- 172.21.15.62:0/785696388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc259c700 1 -- 172.21.15.62:0/785696388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 msgr2=0x55f73f3c0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc259c700 1 --2- 172.21.15.62:0/785696388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc259c700 1 -- 172.21.15.62:0/785696388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1e30 msgr2=0x55f73f3c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc259c700 1 --2- 172.21.15.62:0/785696388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1e30 0x55f73f3c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fc259c700 1 -- 172.21.15.62:0/785696388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f73f13fde0 con 0x55f73f3c1070 2023-07-14T16:20:53.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fb37fe700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9fac0a11f0 0x7f9fac0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.472+0000 7f9fb37fe700 1 -- 172.21.15.62:0/52100238 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9fb40cdc70 con 0x7f9fc41004b0 2023-07-14T16:20:53.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.473+0000 7f9fc2d9d700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9fac0a11f0 0x7f9fac0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.473+0000 7f9fc259c700 1 --2- 172.21.15.62:0/785696388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3c18a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40d1bd0 tx=0x7f9fb40d1c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39a4b1ac70875fca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.473+0000 7f9fb0ff9700 1 -- 172.21.15.62:0/785696388 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4097520 con 0x55f73f3c1070 2023-07-14T16:20:53.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.473+0000 7f9fb0ff9700 1 -- 172.21.15.62:0/785696388 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fb40d1810 con 0x55f73f3c1070 2023-07-14T16:20:53.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.473+0000 7f9fb0ff9700 1 -- 172.21.15.62:0/785696388 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb409ad20 con 0x55f73f3c1070 2023-07-14T16:20:53.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fc2d9d700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9fac0a11f0 0x7f9fac0a36a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0127f0 tx=0x7f9fbc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/785696388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 msgr2=0x55f73f3c18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/785696388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3c18a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40d1bd0 tx=0x7f9fb40d1c00 comp rx=0 tx=0).stop 2023-07-14T16:20:53.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/785696388 shutdown_connections 2023-07-14T16:20:53.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/785696388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1e30 0x55f73f3c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/785696388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/785696388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3c18a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/785696388 >> 172.21.15.62:0/785696388 conn(0x55f73f3b5ff0 msgr2=0x55f73f3bb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/785696388 shutdown_connections 2023-07-14T16:20:53.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.474+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/785696388 wait complete. 2023-07-14T16:20:53.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.475+0000 7f9fdc3c1b80 1 -- start start 2023-07-14T16:20:53.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.475+0000 7f9fdc3c1b80 1 Processor -- start 2023-07-14T16:20:53.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.475+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.475+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.475+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1e30 0x55f73f3d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f2b1ac0 con 0x55f73f3c1070 2023-07-14T16:20:53.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f2b17c0 con 0x55f73f3c0720 2023-07-14T16:20:53.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f195920 con 0x55f73f3c1e30 2023-07-14T16:20:53.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fdc3c1b80 10 client.0 _ll_register_callbacks cb 0x55f73f2af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:53.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40470/0 (socket says 172.21.15.62:40470) 2023-07-14T16:20:53.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc1d9b700 1 -- 172.21.15.62:0/1033322141 learned_addr learned my addr 172.21.15.62:0/1033322141 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc259c700 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc2d9d700 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1e30 0x55f73f3d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc259c700 1 -- 172.21.15.62:0/1033322141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1e30 msgr2=0x55f73f3d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc259c700 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1e30 0x55f73f3d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc259c700 1 -- 172.21.15.62:0/1033322141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 msgr2=0x55f73f3ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc259c700 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc259c700 1 -- 172.21.15.62:0/1033322141 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc419fb70 con 0x55f73f3c0720 2023-07-14T16:20:53.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.476+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:53.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fc259c700 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c9f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40048e0 tx=0x7f9fb400b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664e86edc1c03a56 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:53.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/1033322141 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40d1810 con 0x55f73f3c0720 2023-07-14T16:20:53.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/1033322141 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f73f30aeb0 con 0x55f73f3c0720 2023-07-14T16:20:53.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/1033322141 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9fb4005c00 con 0x55f73f3c0720 2023-07-14T16:20:53.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/1033322141 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fb409ab40 con 0x55f73f3c0720 2023-07-14T16:20:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fdc3c1b80 10 client.26690 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/1033322141 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f73f128c60 con 0x55f73f3c0720 2023-07-14T16:20:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.477+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/1033322141 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fb40b9030 con 0x55f73f3c0720 2023-07-14T16:20:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.478+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/1033322141 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9fb40cbbe0 con 0x55f73f3c0720 2023-07-14T16:20:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.478+0000 7f9fdc3c1b80 10 client.26690 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.478+0000 7f9fdc3c1b80 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.478+0000 7f9fdc3c1b80 20 client.26690 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.478+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f73f3d5820 0x55f73f3d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.478+0000 7f9fdc3c1b80 4 client.26690 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.478+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/1033322141 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f73f2cd050 con 0x55f73f3d5820 2023-07-14T16:20:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.478+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f73f3d5820 0x55f73f3d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.479+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f73f3d5820 0x55f73f3d7c30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2451c490d7396bd2 server_cookie=9595398d99987061 in_seq=0 out_seq=0 2023-07-14T16:20:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.479+0000 7f9fa9ffb700 10 client.26690 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/1033322141 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f73f2cd050 con 0x55f73f3d5820 2023-07-14T16:20:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fa9ffb700 10 client.26690 handle_command_reply: tid=0 2023-07-14T16:20:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 client.26690 shutdown 2023-07-14T16:20:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 msgr2=0x55f73f3c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c9f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40048e0 tx=0x7f9fb400b490 comp rx=0 tx=0).stop 2023-07-14T16:20:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/1033322141 shutdown_connections 2023-07-14T16:20:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1e30 0x55f73f3d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f73f3d5820 0x55f73f3d7c30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c0720 0x55f73f3c9f70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/1033322141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fa9ffb700 0 client.26690 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/1033322141 >> 172.21.15.62:0/1033322141 conn(0x55f73f3b5ff0 msgr2=0x55f73f3bab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/1033322141 shutdown_connections 2023-07-14T16:20:53.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.480+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/1033322141 wait complete. 2023-07-14T16:20:53.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.481+0000 7f9fdc3c1b80 20 client.26690 trim_cache size 0 max 16384 2023-07-14T16:20:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.499+0000 7f9fdc3c1b80 1 Processor -- start 2023-07-14T16:20:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.499+0000 7f9fdc3c1b80 1 -- start start 2023-07-14T16:20:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.499+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c0720 0x55f73f3c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.499+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.499+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 0x55f73f3d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.499+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f3bc880 con 0x55f73f3c1070 2023-07-14T16:20:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.499+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f2cd080 con 0x55f73f3c1e30 2023-07-14T16:20:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.499+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f3cafc0 con 0x55f73f3c0720 2023-07-14T16:20:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c0720 0x55f73f3c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c0720 0x55f73f3c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39972/0 (socket says 172.21.15.62:39972) 2023-07-14T16:20:53.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc259c700 1 -- 172.21.15.62:0/4182195896 learned_addr learned my addr 172.21.15.62:0/4182195896 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc2d9d700 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 0x55f73f3d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc259c700 1 -- 172.21.15.62:0/4182195896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 msgr2=0x55f73f3d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc259c700 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 0x55f73f3d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc259c700 1 -- 172.21.15.62:0/4182195896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 msgr2=0x55f73f3c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc259c700 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc259c700 1 -- 172.21.15.62:0/4182195896 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f73f166250 con 0x55f73f3c0720 2023-07-14T16:20:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.500+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.501+0000 7f9fc259c700 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c0720 0x55f73f3c4cf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009ce0 tx=0x7f9fb4004c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=decc253cde146fa8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.501+0000 7f9fb17fa700 1 -- 172.21.15.62:0/4182195896 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40a3610 con 0x55f73f3c0720 2023-07-14T16:20:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.501+0000 7f9fb17fa700 1 -- 172.21.15.62:0/4182195896 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fb40b9050 con 0x55f73f3c0720 2023-07-14T16:20:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.501+0000 7f9fb17fa700 1 -- 172.21.15.62:0/4182195896 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4005c00 con 0x55f73f3c0720 2023-07-14T16:20:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4182195896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c0720 msgr2=0x55f73f3c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c0720 0x55f73f3c4cf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009ce0 tx=0x7f9fb4004c00 comp rx=0 tx=0).stop 2023-07-14T16:20:53.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4182195896 shutdown_connections 2023-07-14T16:20:53.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 0x55f73f3d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c0720 0x55f73f3c4cf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4182195896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c1070 0x55f73f3c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4182195896 >> 172.21.15.62:0/4182195896 conn(0x55f73f3b5ff0 msgr2=0x55f73f3c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4182195896 shutdown_connections 2023-07-14T16:20:53.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4182195896 wait complete. 2023-07-14T16:20:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 -- start start 2023-07-14T16:20:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.502+0000 7f9fdc3c1b80 1 Processor -- start 2023-07-14T16:20:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c0720 0x55f73f3cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fdc3c1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 0x55f73f3d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f3d1260 con 0x55f73f3c0720 2023-07-14T16:20:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f3d13d0 con 0x55f73f3c1e30 2023-07-14T16:20:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fdc3c1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73f3d1590 con 0x55f73f3c1070 2023-07-14T16:20:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fdc3c1b80 10 client.0 _ll_register_callbacks cb 0x7f9fbc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fc259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c0720 0x55f73f3cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fc259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c0720 0x55f73f3cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40478/0 (socket says 172.21.15.62:40478) 2023-07-14T16:20:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fc259c700 1 -- 172.21.15.62:0/4131269889 learned_addr learned my addr 172.21.15.62:0/4131269889 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.503+0000 7f9fc2d9d700 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 0x55f73f3d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fc259c700 1 -- 172.21.15.62:0/4131269889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 msgr2=0x55f73f3cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fc259c700 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fc259c700 1 -- 172.21.15.62:0/4131269889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 msgr2=0x55f73f3d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fc259c700 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 0x55f73f3d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fc259c700 1 -- 172.21.15.62:0/4131269889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb8009e00 con 0x55f73f3c0720 2023-07-14T16:20:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fc259c700 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c0720 0x55f73f3cd7c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9fb409b170 tx=0x7f9fb400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8be610d52efbc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/4131269889 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40a35f0 con 0x55f73f3c0720 2023-07-14T16:20:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/4131269889 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9fb4005c00 con 0x55f73f3c0720 2023-07-14T16:20:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.504+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4131269889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f73f26b4f0 con 0x55f73f3c0720 2023-07-14T16:20:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/4131269889 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fb409f210 con 0x55f73f3c0720 2023-07-14T16:20:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fdc3c1b80 10 client.16992 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4131269889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f73f128c60 con 0x55f73f3c0720 2023-07-14T16:20:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/4131269889 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f73f26b4f0 con 0x55f73f3c0720 2023-07-14T16:20:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/4131269889 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9fb40a3110 con 0x55f73f3c0720 2023-07-14T16:20:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fdc3c1b80 10 client.16992 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fdc3c1b80 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fdc3c1b80 20 client.16992 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f73f42a560 0x55f73f42c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fdc3c1b80 4 client.16992 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:20:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.505+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4131269889 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f73f160f70 con 0x55f73f42a560 2023-07-14T16:20:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.506+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f73f42a560 0x55f73f42c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.506+0000 7f9fc1d9b700 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f73f42a560 0x55f73f42c970 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a931d0f866584c13 server_cookie=85d8c8689f39bc8b in_seq=0 out_seq=0 2023-07-14T16:20:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.506+0000 7f9fa9ffb700 10 client.16992 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.506+0000 7f9fa9ffb700 1 -- 172.21.15.62:0/4131269889 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f73f160f70 con 0x55f73f42a560 2023-07-14T16:20:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.506+0000 7f9fa9ffb700 10 client.16992 handle_command_reply: tid=0 2023-07-14T16:20:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.506+0000 7f9fdc3c1b80 1 client.16992 shutdown 2023-07-14T16:20:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4131269889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c0720 msgr2=0x55f73f3cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c0720 0x55f73f3cd7c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9fb409b170 tx=0x7f9fb400b490 comp rx=0 tx=0).stop 2023-07-14T16:20:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4131269889 shutdown_connections 2023-07-14T16:20:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f73f42a560 0x55f73f42c970 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f73f3c1e30 0x55f73f3d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f73f3c0720 0x55f73f3cd7c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fa9ffb700 0 client.16992 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 --2- 172.21.15.62:0/4131269889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f73f3c1070 0x55f73f3cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4131269889 >> 172.21.15.62:0/4131269889 conn(0x55f73f3b5ff0 msgr2=0x55f73f3c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4131269889 shutdown_connections 2023-07-14T16:20:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 1 -- 172.21.15.62:0/4131269889 wait complete. 2023-07-14T16:20:53.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.507+0000 7f9fdc3c1b80 20 client.16992 trim_cache size 0 max 16384 2023-07-14T16:20:53.491 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:20:53.491 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:20:53.491 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:20:53.491 INFO:teuthology.orchestra.run.smithi062.stdout: "c9e1915b-d5d7-47f7-aae9-3a87062e3dc8": { 2023-07-14T16:20:53.491 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:20:53.492 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "c9e1915b-d5d7-47f7-aae9-3a87062e3dc8", 2023-07-14T16:20:53.492 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:20:53.492 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:20:53.492 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:20:53.492 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:20:53.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 -- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9fac0a11f0 msgr2=0x7f9fac0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9fac0a11f0 0x7f9fac0a36a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0127f0 tx=0x7f9fbc00b040 comp rx=0 tx=0).stop 2023-07-14T16:20:53.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 -- 172.21.15.62:0/52100238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fc41004b0 msgr2=0x7f9fc41976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fc41004b0 0x7f9fc41976e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40095c0 tx=0x7f9fb4003890 comp rx=0 tx=0).stop 2023-07-14T16:20:53.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 -- 172.21.15.62:0/52100238 shutdown_connections 2023-07-14T16:20:53.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fc41056f0 0x7f9fc419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9fac0a11f0 0x7f9fac0a36a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fc41004b0 0x7f9fc41976e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 --2- 172.21.15.62:0/52100238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc4102dd0 0x7f9fc4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 -- 172.21.15.62:0/52100238 >> 172.21.15.62:0/52100238 conn(0x7f9fc40f9ed0 msgr2=0x7f9fc40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 -- 172.21.15.62:0/52100238 shutdown_connections 2023-07-14T16:20:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:20:53.508+0000 7f9fc359e700 1 -- 172.21.15.62:0/52100238 wait complete. 2023-07-14T16:20:53.504 INFO:tasks.cephfs.filesystem:scrub status for tag:c9e1915b-d5d7-47f7-aae9-3a87062e3dc8 - {'path': '/', 'tag': 'c9e1915b-d5d7-47f7-aae9-3a87062e3dc8', 'options': 'recursive,force'} 2023-07-14T16:20:53.652 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:20:53 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:20:53.201+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v133>,<0x10000000003/client.0 v134>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v146>,<0x10000000001/sv_0 v160>,<0x10000000000/_nogroup v154>,<0x1/volumes v144>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:20:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 -- 172.21.15.62:0/934376992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 msgr2=0x7fbc48072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/934376992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 0x7fbc48072940 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009990 tx=0x7fbc3c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 -- 172.21.15.62:0/934376992 shutdown_connections 2023-07-14T16:20:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/934376992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc481077d0 0x7fbc48109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/934376992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 0x7fbc48072940 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/934376992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc48072f10 0x7fbc48073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 -- 172.21.15.62:0/934376992 >> 172.21.15.62:0/934376992 conn(0x7fbc480faf40 msgr2=0x7fbc480fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 -- 172.21.15.62:0/934376992 shutdown_connections 2023-07-14T16:20:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.941+0000 7fbc4df9d700 1 -- 172.21.15.62:0/934376992 wait complete. 2023-07-14T16:20:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.942+0000 7fbc4df9d700 1 Processor -- start 2023-07-14T16:20:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.942+0000 7fbc4df9d700 1 -- start start 2023-07-14T16:20:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4df9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc48072f10 0x7fbc48197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4df9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 0x7fbc48197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4df9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc481077d0 0x7fbc4819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4df9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819f2c0 con 0x7fbc480744f0 2023-07-14T16:20:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4df9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819f400 con 0x7fbc48072f10 2023-07-14T16:20:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4df9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819f590 con 0x7fbc481077d0 2023-07-14T16:20:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 0x7fbc48197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 0x7fbc48197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40492/0 (socket says 172.21.15.62:40492) 2023-07-14T16:20:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc47fff700 1 -- 172.21.15.62:0/673793137 learned_addr learned my addr 172.21.15.62:0/673793137 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc48072f10 0x7fbc48197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4d79c700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc481077d0 0x7fbc4819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc47fff700 1 -- 172.21.15.62:0/673793137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc481077d0 msgr2=0x7fbc4819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc47fff700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc481077d0 0x7fbc4819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc47fff700 1 -- 172.21.15.62:0/673793137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc48072f10 msgr2=0x7fbc48197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc47fff700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc48072f10 0x7fbc48197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc47fff700 1 -- 172.21.15.62:0/673793137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0095f0 con 0x7fbc480744f0 2023-07-14T16:20:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.943+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc48072f10 0x7fbc48197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.944+0000 7fbc47fff700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 0x7fbc48197c80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400d950 tx=0x7fbc3400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.944+0000 7fbc45ffb700 1 -- 172.21.15.62:0/673793137 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34009b40 con 0x7fbc480744f0 2023-07-14T16:20:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.944+0000 7fbc45ffb700 1 -- 172.21.15.62:0/673793137 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc34009ca0 con 0x7fbc480744f0 2023-07-14T16:20:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.944+0000 7fbc45ffb700 1 -- 172.21.15.62:0/673793137 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400f620 con 0x7fbc480744f0 2023-07-14T16:20:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.944+0000 7fbc4df9d700 1 -- 172.21.15.62:0/673793137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4819f870 con 0x7fbc480744f0 2023-07-14T16:20:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.944+0000 7fbc4df9d700 1 -- 172.21.15.62:0/673793137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4819fd30 con 0x7fbc480744f0 2023-07-14T16:20:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.948+0000 7fbc45ffb700 1 -- 172.21.15.62:0/673793137 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbc34010460 con 0x7fbc480744f0 2023-07-14T16:20:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.949+0000 7fbc62adab80 1 Processor -- start 2023-07-14T16:20:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc62adab80 1 -- start start 2023-07-14T16:20:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f05b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f05bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc62adab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95ef5fc50 con 0x55d95eeb66b0 2023-07-14T16:20:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc62adab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95efb29e0 con 0x55d95f05b7e0 2023-07-14T16:20:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc62adab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95efabe70 con 0x55d95f05aa90 2023-07-14T16:20:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc4d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc4d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40498/0 (socket says 172.21.15.62:40498) 2023-07-14T16:20:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc4d79c700 1 -- 172.21.15.62:0/28306409 learned_addr learned my addr 172.21.15.62:0/28306409 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.950+0000 7fbc47fff700 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f05bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f05b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc45ffb700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbc380a11d0 0x7fbc380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc47fff700 1 -- 172.21.15.62:0/28306409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 msgr2=0x55d95f05b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc47fff700 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f05b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc47fff700 1 -- 172.21.15.62:0/28306409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 msgr2=0x55d95f05a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc47fff700 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc45ffb700 1 -- 172.21.15.62:0/673793137 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc340ccc90 con 0x7fbc480744f0 2023-07-14T16:20:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc47fff700 1 -- 172.21.15.62:0/28306409 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d95edd9d80 con 0x55d95f05b7e0 2023-07-14T16:20:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f05b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc4d79c700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbc380a11d0 0x7fbc380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.951+0000 7fbc47fff700 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f05bc50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc340ca6e0 tx=0x7fbc34010a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af605ba280c1658e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.952+0000 7fbc32ffd700 1 -- 172.21.15.62:0/28306409 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3405e0d0 con 0x55d95f05b7e0 2023-07-14T16:20:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.952+0000 7fbc32ffd700 1 -- 172.21.15.62:0/28306409 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc34099e20 con 0x55d95f05b7e0 2023-07-14T16:20:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.952+0000 7fbc32ffd700 1 -- 172.21.15.62:0/28306409 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc340a24a0 con 0x55d95f05b7e0 2023-07-14T16:20:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.952+0000 7fbc62adab80 1 -- 172.21.15.62:0/28306409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 msgr2=0x55d95f05bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.952+0000 7fbc4d79c700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbc380a11d0 0x7fbc380a3680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55d95f05a2b0 tx=0x7fbc40004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.952+0000 7fbc62adab80 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f05bc50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc340ca6e0 tx=0x7fbc34010a00 comp rx=0 tx=0).stop 2023-07-14T16:20:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.953+0000 7fbc62adab80 1 -- 172.21.15.62:0/28306409 shutdown_connections 2023-07-14T16:20:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.953+0000 7fbc62adab80 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f05bc50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.953+0000 7fbc62adab80 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.953+0000 7fbc62adab80 1 --2- 172.21.15.62:0/28306409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f05b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.953+0000 7fbc62adab80 1 -- 172.21.15.62:0/28306409 >> 172.21.15.62:0/28306409 conn(0x55d95f04fd40 msgr2=0x55d95f054e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.953+0000 7fbc62adab80 1 -- 172.21.15.62:0/28306409 shutdown_connections 2023-07-14T16:20:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.953+0000 7fbc62adab80 1 -- 172.21.15.62:0/28306409 wait complete. 2023-07-14T16:20:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.953+0000 7fbc62adab80 1 -- start start 2023-07-14T16:20:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.954+0000 7fbc62adab80 1 Processor -- start 2023-07-14T16:20:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.954+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95eeb66b0 0x55d95f063a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.954+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f063f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.954+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05b7e0 0x55d95f06af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.954+0000 7fbc62adab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95ef4ba60 con 0x55d95f05b7e0 2023-07-14T16:20:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.954+0000 7fbc62adab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95ef4b760 con 0x55d95eeb66b0 2023-07-14T16:20:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.954+0000 7fbc62adab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95ee2f8c0 con 0x55d95f05aa90 2023-07-14T16:20:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.954+0000 7fbc62adab80 10 client.0 _ll_register_callbacks cb 0x55d95ef49700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05b7e0 0x55d95f06af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4cf9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95eeb66b0 0x55d95f063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05b7e0 0x55d95f06af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40502/0 (socket says 172.21.15.62:40502) 2023-07-14T16:20:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4d79c700 1 -- 172.21.15.62:0/3681119584 learned_addr learned my addr 172.21.15.62:0/3681119584 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc47fff700 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4cf9b700 1 -- 172.21.15.62:0/3681119584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 msgr2=0x55d95f063f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f063f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4cf9b700 1 -- 172.21.15.62:0/3681119584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05b7e0 msgr2=0x55d95f06af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05b7e0 0x55d95f06af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4cf9b700 1 -- 172.21.15.62:0/3681119584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4810a870 con 0x55d95eeb66b0 2023-07-14T16:20:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.955+0000 7fbc4d79c700 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05b7e0 0x55d95f06af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:20:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.956+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95eeb66b0 0x55d95f063a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c000c00 tx=0x7fbc3c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27cc4b888bb55b14 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.956+0000 7fbc27fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:20:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.956+0000 7fbc27fff700 1 -- 172.21.15.62:0/3681119584 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c009ee0 con 0x55d95eeb66b0 2023-07-14T16:20:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.956+0000 7fbc62adab80 1 -- 172.21.15.62:0/3681119584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d95ef05490 con 0x55d95eeb66b0 2023-07-14T16:20:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.956+0000 7fbc27fff700 1 -- 172.21.15.62:0/3681119584 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc3c005670 con 0x55d95eeb66b0 2023-07-14T16:20:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.956+0000 7fbc27fff700 1 -- 172.21.15.62:0/3681119584 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc3c004500 con 0x55d95eeb66b0 2023-07-14T16:20:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.956+0000 7fbc62adab80 10 client.26708 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.956+0000 7fbc62adab80 1 -- 172.21.15.62:0/3681119584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fbc4819fd30 con 0x55d95eeb66b0 2023-07-14T16:20:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc27fff700 1 -- 172.21.15.62:0/3681119584 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d95ef05490 con 0x55d95eeb66b0 2023-07-14T16:20:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc27fff700 1 -- 172.21.15.62:0/3681119584 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc3c003a50 con 0x55d95eeb66b0 2023-07-14T16:20:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc62adab80 10 client.26708 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc62adab80 10 client.26708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc62adab80 20 client.26708 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc62adab80 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d95f06f750 0x55d95f071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc62adab80 4 client.26708 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:20:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc62adab80 1 -- 172.21.15.62:0/3681119584 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d95efa4e50 con 0x55d95f06f750 2023-07-14T16:20:53.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.957+0000 7fbc47fff700 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d95f06f750 0x55d95f071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.958+0000 7fbc27fff700 10 client.26708 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.958+0000 7fbc47fff700 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d95f06f750 0x55d95f071b60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53a0db647cc534ee server_cookie=35622320a65bd87 in_seq=0 out_seq=0 2023-07-14T16:20:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc27fff700 1 -- 172.21.15.62:0/3681119584 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d95efa4e50 con 0x55d95f06f750 2023-07-14T16:20:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc27fff700 10 client.26708 handle_command_reply: tid=0 2023-07-14T16:20:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 client.26708 shutdown 2023-07-14T16:20:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 -- 172.21.15.62:0/3681119584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95eeb66b0 msgr2=0x55d95f063a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95eeb66b0 0x55d95f063a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c000c00 tx=0x7fbc3c005270 comp rx=0 tx=0).stop 2023-07-14T16:20:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 -- 172.21.15.62:0/3681119584 shutdown_connections 2023-07-14T16:20:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05b7e0 0x55d95f06af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d95f06f750 0x55d95f071b60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95eeb66b0 0x55d95f063a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 --2- 172.21.15.62:0/3681119584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f063f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc27fff700 0 client.26708 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 -- 172.21.15.62:0/3681119584 >> 172.21.15.62:0/3681119584 conn(0x55d95f04fd40 msgr2=0x55d95f054640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 -- 172.21.15.62:0/3681119584 shutdown_connections 2023-07-14T16:20:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 1 -- 172.21.15.62:0/3681119584 wait complete. 2023-07-14T16:20:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.959+0000 7fbc62adab80 20 client.26708 trim_cache size 0 max 16384 2023-07-14T16:20:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc62adab80 1 Processor -- start 2023-07-14T16:20:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc62adab80 1 -- start start 2023-07-14T16:20:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f06af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f0764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc62adab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95f061920 con 0x55d95eeb66b0 2023-07-14T16:20:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc62adab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95f05f210 con 0x55d95f05b7e0 2023-07-14T16:20:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc62adab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95f0566a0 con 0x55d95f05aa90 2023-07-14T16:20:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc4cf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc4cf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40516/0 (socket says 172.21.15.62:40516) 2023-07-14T16:20:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc4cf9b700 1 -- 172.21.15.62:0/2613662674 learned_addr learned my addr 172.21.15.62:0/2613662674 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.979+0000 7fbc4d79c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f0764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc47fff700 1 --2- 172.21.15.62:0/2613662674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f06af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc4cf9b700 1 -- 172.21.15.62:0/2613662674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 msgr2=0x55d95f06af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/2613662674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f06af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc4cf9b700 1 -- 172.21.15.62:0/2613662674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 msgr2=0x55d95f0764c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/2613662674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f0764c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc4cf9b700 1 -- 172.21.15.62:0/2613662674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d95edc2c00 con 0x55d95eeb66b0 2023-07-14T16:20:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/2613662674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c005b40 tx=0x7fbc3c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ddec889631ebbe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc337fe700 1 -- 172.21.15.62:0/2613662674 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c009ee0 con 0x55d95eeb66b0 2023-07-14T16:20:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc337fe700 1 -- 172.21.15.62:0/2613662674 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc3c01e040 con 0x55d95eeb66b0 2023-07-14T16:20:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.980+0000 7fbc337fe700 1 -- 172.21.15.62:0/2613662674 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c017900 con 0x55d95eeb66b0 2023-07-14T16:20:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 -- 172.21.15.62:0/2613662674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 msgr2=0x55d95f05a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2613662674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c005b40 tx=0x7fbc3c009310 comp rx=0 tx=0).stop 2023-07-14T16:20:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 -- 172.21.15.62:0/2613662674 shutdown_connections 2023-07-14T16:20:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2613662674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f0764c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2613662674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95eeb66b0 0x55d95f05a510 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2613662674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95f05aa90 0x55d95f06af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 -- 172.21.15.62:0/2613662674 >> 172.21.15.62:0/2613662674 conn(0x55d95f04fd40 msgr2=0x55d95f062730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 -- 172.21.15.62:0/2613662674 shutdown_connections 2023-07-14T16:20:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.981+0000 7fbc62adab80 1 -- 172.21.15.62:0/2613662674 wait complete. 2023-07-14T16:20:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 1 -- start start 2023-07-14T16:20:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 1 Processor -- start 2023-07-14T16:20:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95eeb66b0 0x55d95f0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05aa90 0x55d95f066b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f069b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95f06a060 con 0x55d95f05aa90 2023-07-14T16:20:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95f06a1d0 con 0x55d95f05b7e0 2023-07-14T16:20:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95f06a390 con 0x55d95eeb66b0 2023-07-14T16:20:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.982+0000 7fbc62adab80 10 client.0 _ll_register_callbacks cb 0x7fbc3c00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:20:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05aa90 0x55d95f066b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc4cf9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95eeb66b0 0x55d95f0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05aa90 0x55d95f066b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40530/0 (socket says 172.21.15.62:40530) 2023-07-14T16:20:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 -- 172.21.15.62:0/2975749732 learned_addr learned my addr 172.21.15.62:0/2975749732 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:20:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc4d79c700 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 -- 172.21.15.62:0/2975749732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95eeb66b0 msgr2=0x55d95f0665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95eeb66b0 0x55d95f0665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 -- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 msgr2=0x55d95f069b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:20:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f069b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 -- 172.21.15.62:0/2975749732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4819fd30 con 0x55d95f05aa90 2023-07-14T16:20:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc4d79c700 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f069b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:20:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc47fff700 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05aa90 0x55d95f066b00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbc340a1f70 tx=0x7fbc340cced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd375b8805d9dc97 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:20:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.983+0000 7fbc27fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:20:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc27fff700 1 -- 172.21.15.62:0/2975749732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc340b1070 con 0x55d95f05aa90 2023-07-14T16:20:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc27fff700 1 -- 172.21.15.62:0/2975749732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc340a24a0 con 0x55d95f05aa90 2023-07-14T16:20:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc62adab80 1 -- 172.21.15.62:0/2975749732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc34099e20 con 0x55d95f05aa90 2023-07-14T16:20:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc27fff700 1 -- 172.21.15.62:0/2975749732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc340999b0 con 0x55d95f05aa90 2023-07-14T16:20:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc62adab80 10 client.17010 fetch_fsmap learned FSMap version 25 2023-07-14T16:20:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc62adab80 1 -- 172.21.15.62:0/2975749732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d95ee001f0 con 0x55d95f05aa90 2023-07-14T16:20:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc27fff700 1 -- 172.21.15.62:0/2975749732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc340c8080 con 0x55d95f05aa90 2023-07-14T16:20:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc27fff700 1 -- 172.21.15.62:0/2975749732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc340d01e0 con 0x55d95f05aa90 2023-07-14T16:20:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc62adab80 10 client.17010 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:20:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc62adab80 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:20:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.984+0000 7fbc62adab80 20 client.17010 populate_metadata read hostname 'smithi062' 2023-07-14T16:20:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.985+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d95f0c39b0 0x55d95f0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:20:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.985+0000 7fbc62adab80 4 client.17010 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:20:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.985+0000 7fbc62adab80 1 -- 172.21.15.62:0/2975749732 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d95ef66ff0 con 0x55d95f0c39b0 2023-07-14T16:20:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.985+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d95f0c39b0 0x55d95f0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:20:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.986+0000 7fbc27fff700 10 client.17010 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.986+0000 7fbc4cf9b700 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d95f0c39b0 0x55d95f0c5dc0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36cf464468e018d2 server_cookie=d79bce140d813b6e in_seq=0 out_seq=0 2023-07-14T16:20:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc27fff700 1 -- 172.21.15.62:0/2975749732 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d95ef66ff0 con 0x55d95f0c39b0 2023-07-14T16:20:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc27fff700 10 client.17010 handle_command_reply: tid=0 2023-07-14T16:20:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 client.17010 shutdown 2023-07-14T16:20:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 -- 172.21.15.62:0/2975749732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05aa90 msgr2=0x55d95f066b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05aa90 0x55d95f066b00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbc340a1f70 tx=0x7fbc340cced0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 -- 172.21.15.62:0/2975749732 shutdown_connections 2023-07-14T16:20:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d95f0c39b0 0x55d95f0c5dc0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d95f05b7e0 0x55d95f069b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc27fff700 0 client.17010 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:20:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d95eeb66b0 0x55d95f0665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 --2- 172.21.15.62:0/2975749732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d95f05aa90 0x55d95f066b00 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 -- 172.21.15.62:0/2975749732 >> 172.21.15.62:0/2975749732 conn(0x55d95f04fd40 msgr2=0x55d95f0764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 -- 172.21.15.62:0/2975749732 shutdown_connections 2023-07-14T16:20:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 1 -- 172.21.15.62:0/2975749732 wait complete. 2023-07-14T16:20:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.995+0000 7fbc62adab80 20 client.17010 trim_cache size 0 max 16384 2023-07-14T16:20:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:20:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:20:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:20:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:20:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 -- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbc380a11d0 msgr2=0x7fbc380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbc380a11d0 0x7fbc380a3680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55d95f05a2b0 tx=0x7fbc40004ad0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 -- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 msgr2=0x7fbc48197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:20:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 0x7fbc48197c80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400d950 tx=0x7fbc3400dd10 comp rx=0 tx=0).stop 2023-07-14T16:20:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 -- 172.21.15.62:0/673793137 shutdown_connections 2023-07-14T16:20:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc481077d0 0x7fbc4819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbc380a11d0 0x7fbc380a3680 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc48072f10 0x7fbc48197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 --2- 172.21.15.62:0/673793137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc480744f0 0x7fbc48197c80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:20:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 -- 172.21.15.62:0/673793137 >> 172.21.15.62:0/673793137 conn(0x7fbc480faf40 msgr2=0x7fbc480fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:20:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 -- 172.21.15.62:0/673793137 shutdown_connections 2023-07-14T16:20:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:20:53.996+0000 7fbc4df9d700 1 -- 172.21.15.62:0/673793137 wait complete. 2023-07-14T16:20:54.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:53 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:20:54.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:53 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:54.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:53 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:20:54.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:53 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:54.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:53 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:20:54.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:53 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:54.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:53 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:20:54.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:53 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:54.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:53 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:20:54.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:53 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:54.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:53 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:20:54.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:53 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:54.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:53 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:20:54.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:53 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:54.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:53 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:20:54.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:53 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:54.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:53 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:20:54.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:53 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:54 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:20:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:54 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:54 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:20:55.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:54 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:20:55.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:54 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:55.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:54 smithi062 ceph-mon[95543]: pgmap v515: 97 pgs: 97 active+clean; 3.0 GiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:20:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:54 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:20:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:54 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:54 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:20:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:54 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:20:55.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:54 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:55.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:54 smithi081 ceph-mon[127795]: pgmap v515: 97 pgs: 97 active+clean; 3.0 GiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:20:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:54 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:20:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:54 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:54 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:20:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:54 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:20:55.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:54 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:20:55.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:54 smithi164 ceph-mon[97427]: pgmap v515: 97 pgs: 97 active+clean; 3.0 GiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:20:57.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:56 smithi062 ceph-mon[95543]: pgmap v516: 97 pgs: 97 active+clean; 3.0 GiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:20:57.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:56 smithi081 ceph-mon[127795]: pgmap v516: 97 pgs: 97 active+clean; 3.0 GiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:20:57.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:56 smithi164 ceph-mon[97427]: pgmap v516: 97 pgs: 97 active+clean; 3.0 GiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:20:59.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:20:58 smithi062 ceph-mon[95543]: pgmap v517: 97 pgs: 97 active+clean; 3.0 GiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:20:59.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:20:58 smithi081 ceph-mon[127795]: pgmap v517: 97 pgs: 97 active+clean; 3.0 GiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:20:59.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:20:58 smithi164 ceph-mon[97427]: pgmap v517: 97 pgs: 97 active+clean; 3.0 GiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:21:01.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:00 smithi062 ceph-mon[95543]: pgmap v518: 97 pgs: 97 active+clean; 3.0 GiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:21:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:00 smithi081 ceph-mon[127795]: pgmap v518: 97 pgs: 97 active+clean; 3.0 GiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:21:01.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:00 smithi164 ceph-mon[97427]: pgmap v518: 97 pgs: 97 active+clean; 3.0 GiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:21:03.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:02 smithi062 ceph-mon[95543]: pgmap v519: 97 pgs: 97 active+clean; 3.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:21:03.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:02 smithi081 ceph-mon[127795]: pgmap v519: 97 pgs: 97 active+clean; 3.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:21:03.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:02 smithi164 ceph-mon[97427]: pgmap v519: 97 pgs: 97 active+clean; 3.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:21:04.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:21:04.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:21:04.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:21:04.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:21:04.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:21:04.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:21:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:04 smithi062 ceph-mon[95543]: pgmap v520: 97 pgs: 97 active+clean; 3.1 GiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:21:05.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:04 smithi081 ceph-mon[127795]: pgmap v520: 97 pgs: 97 active+clean; 3.1 GiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:21:05.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:04 smithi164 ceph-mon[97427]: pgmap v520: 97 pgs: 97 active+clean; 3.1 GiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:21:06.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:05 smithi062 ceph-mon[95543]: pgmap v521: 97 pgs: 97 active+clean; 3.1 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 28 op/s 2023-07-14T16:21:06.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:05 smithi081 ceph-mon[127795]: pgmap v521: 97 pgs: 97 active+clean; 3.1 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 28 op/s 2023-07-14T16:21:06.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:05 smithi164 ceph-mon[97427]: pgmap v521: 97 pgs: 97 active+clean; 3.1 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 28 op/s 2023-07-14T16:21:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 -- 172.21.15.62:0/4270458903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc100620 msgr2=0x7f10cc102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 --2- 172.21.15.62:0/4270458903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc100620 0x7f10cc102a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10c80098f0 tx=0x7f10c8009c00 comp rx=0 tx=0).stop 2023-07-14T16:21:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 -- 172.21.15.62:0/4270458903 shutdown_connections 2023-07-14T16:21:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 --2- 172.21.15.62:0/4270458903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10cc105860 0x7f10cc107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 --2- 172.21.15.62:0/4270458903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc100620 0x7f10cc102a00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 --2- 172.21.15.62:0/4270458903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10cc102f40 0x7f10cc105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 -- 172.21.15.62:0/4270458903 >> 172.21.15.62:0/4270458903 conn(0x7f10cc0fa000 msgr2=0x7f10cc0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 -- 172.21.15.62:0/4270458903 shutdown_connections 2023-07-14T16:21:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.154+0000 7f10d33ff700 1 -- 172.21.15.62:0/4270458903 wait complete. 2023-07-14T16:21:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.155+0000 7f10d33ff700 1 Processor -- start 2023-07-14T16:21:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.156+0000 7f10d33ff700 1 -- start start 2023-07-14T16:21:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.156+0000 7f10d33ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc100620 0x7f10cc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.156+0000 7f10d33ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10cc102f40 0x7f10cc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.156+0000 7f10d33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10cc105860 0x7f10cc198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.156+0000 7f10d33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc199530 con 0x7f10cc105860 2023-07-14T16:21:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.156+0000 7f10d33ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc1996a0 con 0x7f10cc102f40 2023-07-14T16:21:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.156+0000 7f10d33ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc199860 con 0x7f10cc100620 2023-07-14T16:21:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10cc105860 0x7f10cc198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d119b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc100620 0x7f10cc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10cc105860 0x7f10cc198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53284/0 (socket says 172.21.15.62:53284) 2023-07-14T16:21:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 -- 172.21.15.62:0/118427675 learned_addr learned my addr 172.21.15.62:0/118427675 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d099a700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10cc102f40 0x7f10cc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 -- 172.21.15.62:0/118427675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc100620 msgr2=0x7f10cc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc100620 0x7f10cc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 -- 172.21.15.62:0/118427675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10cc102f40 msgr2=0x7f10cc198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10cc102f40 0x7f10cc198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 -- 172.21.15.62:0/118427675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc0095f0 con 0x7f10cc105860 2023-07-14T16:21:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.157+0000 7f10d199c700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10cc105860 0x7f10cc198ed0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f10c400efc0 tx=0x7f10c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.158+0000 7f10c27fc700 1 -- 172.21.15.62:0/118427675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4018070 con 0x7f10cc105860 2023-07-14T16:21:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.158+0000 7f10c27fc700 1 -- 172.21.15.62:0/118427675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10c400f040 con 0x7f10cc105860 2023-07-14T16:21:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.158+0000 7f10c27fc700 1 -- 172.21.15.62:0/118427675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4007c10 con 0x7f10cc105860 2023-07-14T16:21:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.158+0000 7f10d33ff700 1 -- 172.21.15.62:0/118427675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c8009520 con 0x7f10cc105860 2023-07-14T16:21:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.158+0000 7f10d33ff700 1 -- 172.21.15.62:0/118427675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10cc1a0120 con 0x7f10cc105860 2023-07-14T16:21:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.161+0000 7f10c27fc700 1 -- 172.21.15.62:0/118427675 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f10c400c9f0 con 0x7f10cc105860 2023-07-14T16:21:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.164+0000 7f10c27fc700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b80a11d0 0x7f10b80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.164+0000 7f10e6c9ab80 1 Processor -- start 2023-07-14T16:21:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.164+0000 7f10c27fc700 1 -- 172.21.15.62:0/118427675 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10c40cc9f0 con 0x7f10cc105860 2023-07-14T16:21:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.164+0000 7f10d119b700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b80a11d0 0x7f10b80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.164+0000 7f10e6c9ab80 1 -- start start 2023-07-14T16:21:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.164+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfdfb8c50 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfe00b9e0 con 0x55cdfe0b3840 2023-07-14T16:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfe004e70 con 0x55cdfe0b46e0 2023-07-14T16:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10d119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10d119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53288/0 (socket says 172.21.15.62:53288) 2023-07-14T16:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10d119b700 1 -- 172.21.15.62:0/1279076175 learned_addr learned my addr 172.21.15.62:0/1279076175 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10d199c700 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10d099a700 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10d119b700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b80a11d0 0x7f10b80a3680 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009fd0 tx=0x7f10c8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.165+0000 7f10d119b700 1 -- 172.21.15.62:0/1279076175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 msgr2=0x55cdfe0b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10d119b700 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10d119b700 1 -- 172.21.15.62:0/1279076175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 msgr2=0x55cdfe0b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10d119b700 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10d119b700 1 -- 172.21.15.62:0/1279076175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdfde32d80 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10d099a700 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10d119b700 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0b3300 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f10c80098f0 tx=0x7f10c801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77242a25f22e1c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10b77fe700 1 -- 172.21.15.62:0/1279076175 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c802a070 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10b77fe700 1 -- 172.21.15.62:0/1279076175 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10c801d040 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.166+0000 7f10b77fe700 1 -- 172.21.15.62:0/1279076175 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c8005260 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1279076175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 msgr2=0x55cdfe0b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0b3300 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f10c80098f0 tx=0x7f10c801e000 comp rx=0 tx=0).stop 2023-07-14T16:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1279076175 shutdown_connections 2023-07-14T16:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0b3300 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/1279076175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1279076175 >> 172.21.15.62:0/1279076175 conn(0x55cdfe0a8be0 msgr2=0x55cdfe0adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1279076175 shutdown_connections 2023-07-14T16:21:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.167+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1279076175 wait complete. 2023-07-14T16:21:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.168+0000 7f10e6c9ab80 1 -- start start 2023-07-14T16:21:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.168+0000 7f10e6c9ab80 1 Processor -- start 2023-07-14T16:21:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b3840 0x55cdfe0bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfdfa4a60 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfdfa4760 con 0x55cdfe0b46e0 2023-07-14T16:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfde888c0 con 0x55cdfe0b3840 2023-07-14T16:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10e6c9ab80 10 client.0 _ll_register_callbacks cb 0x55cdfdfa2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10d119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10d119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53302/0 (socket says 172.21.15.62:53302) 2023-07-14T16:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.169+0000 7f10d119b700 1 -- 172.21.15.62:0/4008926634 learned_addr learned my addr 172.21.15.62:0/4008926634 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10d099a700 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b3840 0x55cdfe0bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10d199c700 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10d119b700 1 -- 172.21.15.62:0/4008926634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b3840 msgr2=0x55cdfe0bcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10d119b700 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b3840 0x55cdfe0bcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10d119b700 1 -- 172.21.15.62:0/4008926634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b46e0 msgr2=0x55cdfe0c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10d119b700 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10d119b700 1 -- 172.21.15.62:0/4008926634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc009ad0 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10d119b700 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bc860 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f10c8005e00 tx=0x7f10c800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9363cc5ab051c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10b4ff9700 1 -- 172.21.15.62:0/4008926634 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c802a040 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/4008926634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdfdf5e490 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.170+0000 7f10b4ff9700 1 -- 172.21.15.62:0/4008926634 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10c8003680 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10b4ff9700 1 -- 172.21.15.62:0/4008926634 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10c803d030 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10e6c9ab80 10 client.17022 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/4008926634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f10cc108860 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10b4ff9700 1 -- 172.21.15.62:0/4008926634 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cdfdf5e490 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10b4ff9700 1 -- 172.21.15.62:0/4008926634 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f10c8037720 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10e6c9ab80 10 client.17022 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10e6c9ab80 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10e6c9ab80 20 client.17022 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdfe0c8180 0x55cdfe0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10e6c9ab80 4 client.17022 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.171+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/4008926634 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdfdffde50 con 0x55cdfe0c8180 2023-07-14T16:21:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.172+0000 7f10d099a700 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdfe0c8180 0x55cdfe0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.172+0000 7f10d099a700 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdfe0c8180 0x55cdfe0ca590 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb630bf7b9c6699 server_cookie=e71a0be9a1a3ad2f in_seq=0 out_seq=0 2023-07-14T16:21:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.172+0000 7f10b4ff9700 10 client.17022 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.173+0000 7f10b4ff9700 1 -- 172.21.15.62:0/4008926634 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cdfdffde50 con 0x55cdfe0c8180 2023-07-14T16:21:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.173+0000 7f10b4ff9700 10 client.17022 handle_command_reply: tid=0 2023-07-14T16:21:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.173+0000 7f10e6c9ab80 1 client.17022 shutdown 2023-07-14T16:21:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/4008926634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 msgr2=0x55cdfe0bc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bc860 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f10c8005e00 tx=0x7f10c800b8d0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/4008926634 shutdown_connections 2023-07-14T16:21:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdfe0c8180 0x55cdfe0ca590 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bc860 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/4008926634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b3840 0x55cdfe0bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10b4ff9700 0 client.17022 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:07.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/4008926634 >> 172.21.15.62:0/4008926634 conn(0x55cdfe0a8be0 msgr2=0x55cdfe0ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/4008926634 shutdown_connections 2023-07-14T16:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/4008926634 wait complete. 2023-07-14T16:21:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.174+0000 7f10e6c9ab80 20 client.17022 trim_cache size 0 max 16384 2023-07-14T16:21:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.192+0000 7f10e6c9ab80 1 Processor -- start 2023-07-14T16:21:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.192+0000 7f10e6c9ab80 1 -- start start 2023-07-14T16:21:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.192+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.192+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.192+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.192+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfe0aeb80 con 0x55cdfe0b3840 2023-07-14T16:21:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.192+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfe0bb330 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.192+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfe0aad00 con 0x55cdfe0b46e0 2023-07-14T16:21:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d199c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d199c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55836/0 (socket says 172.21.15.62:55836) 2023-07-14T16:21:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53308/0 (socket says 172.21.15.62:53308) 2023-07-14T16:21:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d199c700 1 -- 172.21.15.62:0/1300817214 learned_addr learned my addr 172.21.15.62:0/1300817214 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d119b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d099a700 1 -- 172.21.15.62:0/1300817214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 msgr2=0x55cdfe0b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d099a700 1 --2- 172.21.15.62:0/1300817214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d099a700 1 -- 172.21.15.62:0/1300817214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfdf0f6b0 msgr2=0x55cdfe0bb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d099a700 1 --2- 172.21.15.62:0/1300817214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.193+0000 7f10d099a700 1 -- 172.21.15.62:0/1300817214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdfde1bc00 con 0x55cdfe0b3840 2023-07-14T16:21:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.194+0000 7f10d099a700 1 --2- 172.21.15.62:0/1300817214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c3da0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55cdfe0cab70 tx=0x7f10bc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b923a572c20b643 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.194+0000 7f10b7fff700 1 -- 172.21.15.62:0/1300817214 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc004230 con 0x55cdfe0b3840 2023-07-14T16:21:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.194+0000 7f10b7fff700 1 -- 172.21.15.62:0/1300817214 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10bc018040 con 0x55cdfe0b3840 2023-07-14T16:21:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.194+0000 7f10b7fff700 1 -- 172.21.15.62:0/1300817214 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc01f4d0 con 0x55cdfe0b3840 2023-07-14T16:21:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1300817214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b3840 msgr2=0x55cdfe0c3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/1300817214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c3da0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55cdfe0cab70 tx=0x7f10bc009310 comp rx=0 tx=0).stop 2023-07-14T16:21:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1300817214 shutdown_connections 2023-07-14T16:21:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/1300817214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/1300817214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/1300817214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c3da0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1300817214 >> 172.21.15.62:0/1300817214 conn(0x55cdfe0a8be0 msgr2=0x55cdfe0ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1300817214 shutdown_connections 2023-07-14T16:21:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.195+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/1300817214 wait complete. 2023-07-14T16:21:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.196+0000 7f10e6c9ab80 1 -- start start 2023-07-14T16:21:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.196+0000 7f10e6c9ab80 1 Processor -- start 2023-07-14T16:21:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10e6c9ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfe0c3850 con 0x55cdfe0b46e0 2023-07-14T16:21:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfe0c39c0 con 0x55cdfe0b3840 2023-07-14T16:21:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10e6c9ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdfe0c3b80 con 0x55cdfdf0f6b0 2023-07-14T16:21:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10e6c9ab80 10 client.0 _ll_register_callbacks cb 0x7f10bc0098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10d199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10d119b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10d119b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55838/0 (socket says 172.21.15.62:55838) 2023-07-14T16:21:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10d119b700 1 -- 172.21.15.62:0/3318425460 learned_addr learned my addr 172.21.15.62:0/3318425460 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.197+0000 7f10d099a700 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10d199c700 1 -- 172.21.15.62:0/3318425460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfdf0f6b0 msgr2=0x55cdfe0bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10d199c700 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10d199c700 1 -- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 msgr2=0x55cdfe0c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10d199c700 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10d199c700 1 -- 172.21.15.62:0/3318425460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc009d80 con 0x55cdfe0b46e0 2023-07-14T16:21:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10d099a700 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10d199c700 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3310 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f10c4000f80 tx=0x7f10c400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbe252505ff43e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10b4ff9700 1 -- 172.21.15.62:0/3318425460 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4099e40 con 0x55cdfe0b46e0 2023-07-14T16:21:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10b4ff9700 1 -- 172.21.15.62:0/3318425460 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10c409de70 con 0x55cdfe0b46e0 2023-07-14T16:21:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.198+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/3318425460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10c801d040 con 0x55cdfe0b46e0 2023-07-14T16:21:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10b4ff9700 1 -- 172.21.15.62:0/3318425460 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10c40ca1a0 con 0x55cdfe0b46e0 2023-07-14T16:21:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10e6c9ab80 10 client.17028 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/3318425460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cdfde591f0 con 0x55cdfe0b46e0 2023-07-14T16:21:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10b4ff9700 1 -- 172.21.15.62:0/3318425460 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10c801d040 con 0x55cdfe0b46e0 2023-07-14T16:21:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10b4ff9700 1 -- 172.21.15.62:0/3318425460 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f10c4099410 con 0x55cdfe0b46e0 2023-07-14T16:21:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10e6c9ab80 10 client.17028 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10e6c9ab80 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10e6c9ab80 20 client.17028 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdfe11d260 0x55cdfe11f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10e6c9ab80 4 client.17028 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:21:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.199+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/3318425460 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cdfdfbfff0 con 0x55cdfe11d260 2023-07-14T16:21:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.200+0000 7f10d119b700 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdfe11d260 0x55cdfe11f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.200+0000 7f10d119b700 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdfe11d260 0x55cdfe11f670 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ec9cedb8b6ae6a server_cookie=b9a081facaebcf27 in_seq=0 out_seq=0 2023-07-14T16:21:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.200+0000 7f10b4ff9700 10 client.17028 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10b4ff9700 1 -- 172.21.15.62:0/3318425460 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cdfdfbfff0 con 0x55cdfe11d260 2023-07-14T16:21:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10b4ff9700 10 client.17028 handle_command_reply: tid=0 2023-07-14T16:21:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 client.17028 shutdown 2023-07-14T16:21:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/3318425460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b46e0 msgr2=0x55cdfe0c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3310 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f10c4000f80 tx=0x7f10c400cf80 comp rx=0 tx=0).stop 2023-07-14T16:21:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/3318425460 shutdown_connections 2023-07-14T16:21:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdfe0b46e0 0x55cdfe0c3310 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdfdf0f6b0 0x55cdfe0bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdfe11d260 0x55cdfe11f670 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 --2- 172.21.15.62:0/3318425460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdfe0b3840 0x55cdfe0c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/3318425460 >> 172.21.15.62:0/3318425460 conn(0x55cdfe0a8be0 msgr2=0x55cdfe0bb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10b4ff9700 0 client.17028 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/3318425460 shutdown_connections 2023-07-14T16:21:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 1 -- 172.21.15.62:0/3318425460 wait complete. 2023-07-14T16:21:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.212+0000 7f10e6c9ab80 20 client.17028 trim_cache size 0 max 16384 2023-07-14T16:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:21:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 -- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b80a11d0 msgr2=0x7f10b80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b80a11d0 0x7f10b80a3680 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009fd0 tx=0x7f10c8019040 comp rx=0 tx=0).stop 2023-07-14T16:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 -- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10cc105860 msgr2=0x7f10cc198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10cc105860 0x7f10cc198ed0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f10c400efc0 tx=0x7f10c400c3a0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 -- 172.21.15.62:0/118427675 shutdown_connections 2023-07-14T16:21:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10cc105860 0x7f10cc198ed0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc100620 0x7f10cc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b80a11d0 0x7f10b80a3680 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 --2- 172.21.15.62:0/118427675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10cc102f40 0x7f10cc198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 -- 172.21.15.62:0/118427675 >> 172.21.15.62:0/118427675 conn(0x7f10cc0fa000 msgr2=0x7f10cc101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 -- 172.21.15.62:0/118427675 shutdown_connections 2023-07-14T16:21:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:07.213+0000 7f10d33ff700 1 -- 172.21.15.62:0/118427675 wait complete. 2023-07-14T16:21:08.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:08 smithi062 ceph-mon[95543]: pgmap v522: 97 pgs: 97 active+clean; 3.2 GiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 22 op/s 2023-07-14T16:21:08.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:08 smithi081 ceph-mon[127795]: pgmap v522: 97 pgs: 97 active+clean; 3.2 GiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 22 op/s 2023-07-14T16:21:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:08 smithi164 ceph-mon[97427]: pgmap v522: 97 pgs: 97 active+clean; 3.2 GiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 22 op/s 2023-07-14T16:21:10.903 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:10 smithi081 ceph-mon[127795]: pgmap v523: 97 pgs: 97 active+clean; 3.2 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:21:11.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:10 smithi164 ceph-mon[97427]: pgmap v523: 97 pgs: 97 active+clean; 3.2 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:21:11.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:10 smithi062 ceph-mon[95543]: pgmap v523: 97 pgs: 97 active+clean; 3.2 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:21:12.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:12 smithi081 ceph-mon[127795]: pgmap v524: 97 pgs: 97 active+clean; 3.2 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:21:13.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:12 smithi164 ceph-mon[97427]: pgmap v524: 97 pgs: 97 active+clean; 3.2 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:21:13.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:12 smithi062 ceph-mon[95543]: pgmap v524: 97 pgs: 97 active+clean; 3.2 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:21:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:14 smithi164 ceph-mon[97427]: pgmap v525: 97 pgs: 97 active+clean; 3.3 GiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:14 smithi062 ceph-mon[95543]: pgmap v525: 97 pgs: 97 active+clean; 3.3 GiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:14 smithi081 ceph-mon[127795]: pgmap v525: 97 pgs: 97 active+clean; 3.3 GiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:17.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:16 smithi164 ceph-mon[97427]: pgmap v526: 97 pgs: 97 active+clean; 3.3 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:21:17.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:16 smithi062 ceph-mon[95543]: pgmap v526: 97 pgs: 97 active+clean; 3.3 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:21:17.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:16 smithi081 ceph-mon[127795]: pgmap v526: 97 pgs: 97 active+clean; 3.3 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:21:19.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:18 smithi164 ceph-mon[97427]: pgmap v527: 97 pgs: 97 active+clean; 3.3 GiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:21:19.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:18 smithi062 ceph-mon[95543]: pgmap v527: 97 pgs: 97 active+clean; 3.3 GiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:21:19.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:18 smithi081 ceph-mon[127795]: pgmap v527: 97 pgs: 97 active+clean; 3.3 GiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:21:20.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:19 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:21:20.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:19 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:21:20.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:19 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:21:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 -- 172.21.15.62:0/181892097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c072f70 msgr2=0x7ff77c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 --2- 172.21.15.62:0/181892097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c072f70 0x7ff77c0733e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff770009990 tx=0x7ff770009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 -- 172.21.15.62:0/181892097 shutdown_connections 2023-07-14T16:21:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 --2- 172.21.15.62:0/181892097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff77c10c0e0 0x7ff77c10e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 --2- 172.21.15.62:0/181892097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff77c074550 0x7ff77c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 --2- 172.21.15.62:0/181892097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c072f70 0x7ff77c0733e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 -- 172.21.15.62:0/181892097 >> 172.21.15.62:0/181892097 conn(0x7ff77c0fb130 msgr2=0x7ff77c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 -- 172.21.15.62:0/181892097 shutdown_connections 2023-07-14T16:21:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.378+0000 7ff780fff700 1 -- 172.21.15.62:0/181892097 wait complete. 2023-07-14T16:21:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.379+0000 7ff780fff700 1 Processor -- start 2023-07-14T16:21:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.380+0000 7ff780fff700 1 -- start start 2023-07-14T16:21:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.380+0000 7ff780fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff77c072f70 0x7ff77c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.380+0000 7ff780fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c074550 0x7ff77c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.380+0000 7ff780fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff77c10c0e0 0x7ff77c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.380+0000 7ff780fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c19d8d0 con 0x7ff77c074550 2023-07-14T16:21:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.380+0000 7ff780fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c19da40 con 0x7ff77c10c0e0 2023-07-14T16:21:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.380+0000 7ff780fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c19dc00 con 0x7ff77c072f70 2023-07-14T16:21:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.380+0000 7ff779d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c074550 0x7ff77c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff779d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c074550 0x7ff77c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34058/0 (socket says 172.21.15.62:34058) 2023-07-14T16:21:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff779d9b700 1 -- 172.21.15.62:0/5072401 learned_addr learned my addr 172.21.15.62:0/5072401 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff77ad9d700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff77c10c0e0 0x7ff77c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff77a59c700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff77c072f70 0x7ff77c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff779d9b700 1 -- 172.21.15.62:0/5072401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff77c072f70 msgr2=0x7ff77c19beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff779d9b700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff77c072f70 0x7ff77c19beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff779d9b700 1 -- 172.21.15.62:0/5072401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff77c10c0e0 msgr2=0x7ff77c19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff779d9b700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff77c10c0e0 0x7ff77c19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff779d9b700 1 -- 172.21.15.62:0/5072401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7700095f0 con 0x7ff77c074550 2023-07-14T16:21:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff77a59c700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff77c072f70 0x7ff77c19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.381+0000 7ff779d9b700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c074550 0x7ff77c19c3f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff770005910 tx=0x7ff770005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.382+0000 7ff76b7fe700 1 -- 172.21.15.62:0/5072401 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77001d070 con 0x7ff77c074550 2023-07-14T16:21:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.382+0000 7ff76b7fe700 1 -- 172.21.15.62:0/5072401 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff770022470 con 0x7ff77c074550 2023-07-14T16:21:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.382+0000 7ff780fff700 1 -- 172.21.15.62:0/5072401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c19de80 con 0x7ff77c074550 2023-07-14T16:21:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.382+0000 7ff76b7fe700 1 -- 172.21.15.62:0/5072401 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77000f6c0 con 0x7ff77c074550 2023-07-14T16:21:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.382+0000 7ff780fff700 1 -- 172.21.15.62:0/5072401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c1a46a0 con 0x7ff77c074550 2023-07-14T16:21:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.386+0000 7ff76b7fe700 1 -- 172.21.15.62:0/5072401 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff770022850 con 0x7ff77c074550 2023-07-14T16:21:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.386+0000 7ff7948ceb80 1 Processor -- start 2023-07-14T16:21:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.387+0000 7ff7948ceb80 1 -- start start 2023-07-14T16:21:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.387+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a267815300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.387+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a267815840 0x55a2678161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.387+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2678166e0 0x55a267816b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.387+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26771ac50 con 0x55a267815840 2023-07-14T16:21:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.387+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26776d9e0 con 0x55a2678166e0 2023-07-14T16:21:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.387+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a267766e70 con 0x55a2676716b0 2023-07-14T16:21:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a267815840 0x55a2678161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a267815300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff779d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2678166e0 0x55a267816b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a267815840 0x55a2678161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34068/0 (socket says 172.21.15.62:34068) 2023-07-14T16:21:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a267815300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55684/0 (socket says 172.21.15.62:55684) 2023-07-14T16:21:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77a59c700 1 -- 172.21.15.62:0/3089413290 learned_addr learned my addr 172.21.15.62:0/3089413290 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77ad9d700 1 -- 172.21.15.62:0/3089413290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2678166e0 msgr2=0x55a267816b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77ad9d700 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2678166e0 0x55a267816b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77ad9d700 1 -- 172.21.15.62:0/3089413290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a267815840 msgr2=0x55a2678161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77ad9d700 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a267815840 0x55a2678161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77ad9d700 1 -- 172.21.15.62:0/3089413290 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a267594d80 con 0x55a2676716b0 2023-07-14T16:21:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff779d9b700 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2678166e0 0x55a267816b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.388+0000 7ff77a59c700 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a267815840 0x55a2678161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.389+0000 7ff76b7fe700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff7640a11d0 0x7ff7640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.389+0000 7ff77ad9d700 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a267815300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff774011580 tx=0x7ff774009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e01fbdab09d04e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.389+0000 7ff76b7fe700 1 -- 172.21.15.62:0/5072401 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff7700cdfc0 con 0x7ff77c074550 2023-07-14T16:21:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.389+0000 7ff77a59c700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff7640a11d0 0x7ff7640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.389+0000 7ff768ff9700 1 -- 172.21.15.62:0/3089413290 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77400e040 con 0x55a2676716b0 2023-07-14T16:21:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.389+0000 7ff768ff9700 1 -- 172.21.15.62:0/3089413290 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff77400f040 con 0x55a2676716b0 2023-07-14T16:21:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.389+0000 7ff768ff9700 1 -- 172.21.15.62:0/3089413290 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774004320 con 0x55a2676716b0 2023-07-14T16:21:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 -- 172.21.15.62:0/3089413290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 msgr2=0x55a267815300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff77a59c700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff7640a11d0 0x7ff7640a3680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff76c005c50 tx=0x7ff76c00b680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a267815300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff774011580 tx=0x7ff774009920 comp rx=0 tx=0).stop 2023-07-14T16:21:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 -- 172.21.15.62:0/3089413290 shutdown_connections 2023-07-14T16:21:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2678166e0 0x55a267816b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a267815300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/3089413290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a267815840 0x55a2678161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 -- 172.21.15.62:0/3089413290 >> 172.21.15.62:0/3089413290 conn(0x55a26780abe0 msgr2=0x55a26780fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 -- 172.21.15.62:0/3089413290 shutdown_connections 2023-07-14T16:21:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.390+0000 7ff7948ceb80 1 -- 172.21.15.62:0/3089413290 wait complete. 2023-07-14T16:21:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.391+0000 7ff7948ceb80 1 -- start start 2023-07-14T16:21:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.391+0000 7ff7948ceb80 1 Processor -- start 2023-07-14T16:21:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a26781e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a267815840 0x55a26781edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a267825e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a267706a60 con 0x55a2678166e0 2023-07-14T16:21:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a267706760 con 0x55a267815840 2023-07-14T16:21:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2675ea8c0 con 0x55a2676716b0 2023-07-14T16:21:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff7948ceb80 10 client.0 _ll_register_callbacks cb 0x55a267704700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a267825e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a26781e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff779d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a267815840 0x55a26781edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a26781e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55686/0 (socket says 172.21.15.62:55686) 2023-07-14T16:21:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a267825e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34072/0 (socket says 172.21.15.62:34072) 2023-07-14T16:21:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.392+0000 7ff77a59c700 1 -- 172.21.15.62:0/2924113246 learned_addr learned my addr 172.21.15.62:0/2924113246 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff779d9b700 1 -- 172.21.15.62:0/2924113246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 msgr2=0x55a26781e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff779d9b700 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a26781e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff779d9b700 1 -- 172.21.15.62:0/2924113246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 msgr2=0x55a267825e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff779d9b700 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a267825e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff779d9b700 1 -- 172.21.15.62:0/2924113246 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c00b9c0 con 0x55a267815840 2023-07-14T16:21:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff779d9b700 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a267815840 0x55a26781edd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff77009b110 tx=0x7ff77009b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6351ef75a366222 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff761ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:21:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff761ffb700 1 -- 172.21.15.62:0/2924113246 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770097750 con 0x55a267815840 2023-07-14T16:21:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff761ffb700 1 -- 172.21.15.62:0/2924113246 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff77009b410 con 0x55a267815840 2023-07-14T16:21:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff7948ceb80 1 -- 172.21.15.62:0/2924113246 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2676c0490 con 0x55a267815840 2023-07-14T16:21:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff761ffb700 1 -- 172.21.15.62:0/2924113246 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff77009b1e0 con 0x55a267815840 2023-07-14T16:21:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff7948ceb80 10 client.26744 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.393+0000 7ff7948ceb80 1 -- 172.21.15.62:0/2924113246 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7ff77c1a46a0 con 0x55a267815840 2023-07-14T16:21:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff761ffb700 1 -- 172.21.15.62:0/2924113246 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a2676c0490 con 0x55a267815840 2023-07-14T16:21:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff761ffb700 1 -- 172.21.15.62:0/2924113246 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7700a10f0 con 0x55a267815840 2023-07-14T16:21:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff7948ceb80 10 client.26744 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff7948ceb80 10 client.26744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff7948ceb80 20 client.26744 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a26782a340 0x55a26782c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff7948ceb80 4 client.26744 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff7948ceb80 1 -- 172.21.15.62:0/2924113246 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a26775fe50 con 0x55a26782a340 2023-07-14T16:21:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.394+0000 7ff77a59c700 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a26782a340 0x55a26782c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.395+0000 7ff77a59c700 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a26782a340 0x55a26782c750 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27780049d1002a4 server_cookie=49f144c955f20f8b in_seq=0 out_seq=0 2023-07-14T16:21:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.395+0000 7ff761ffb700 10 client.26744 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff761ffb700 1 -- 172.21.15.62:0/2924113246 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a26775fe50 con 0x55a26782a340 2023-07-14T16:21:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff761ffb700 10 client.26744 handle_command_reply: tid=0 2023-07-14T16:21:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 client.26744 shutdown 2023-07-14T16:21:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 -- 172.21.15.62:0/2924113246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a267815840 msgr2=0x55a26781edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a267815840 0x55a26781edd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff77009b110 tx=0x7ff77009b950 comp rx=0 tx=0).stop 2023-07-14T16:21:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 -- 172.21.15.62:0/2924113246 shutdown_connections 2023-07-14T16:21:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a267825e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2676716b0 0x55a26781e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a26782a340 0x55a26782c750 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/2924113246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a267815840 0x55a26781edd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff761ffb700 0 client.26744 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:20.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 -- 172.21.15.62:0/2924113246 >> 172.21.15.62:0/2924113246 conn(0x55a26780abe0 msgr2=0x55a26780f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:20.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 -- 172.21.15.62:0/2924113246 shutdown_connections 2023-07-14T16:21:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 1 -- 172.21.15.62:0/2924113246 wait complete. 2023-07-14T16:21:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.396+0000 7ff7948ceb80 20 client.26744 trim_cache size 0 max 16384 2023-07-14T16:21:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.418+0000 7ff7948ceb80 1 Processor -- start 2023-07-14T16:21:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.418+0000 7ff7948ceb80 1 -- start start 2023-07-14T16:21:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.418+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a26781d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.418+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a267825dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.418+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.418+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2676c0490 con 0x55a2678166e0 2023-07-14T16:21:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.418+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26781fa80 con 0x55a2676716b0 2023-07-14T16:21:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.418+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2677db060 con 0x55a267815840 2023-07-14T16:21:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff779d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a267825dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a26781d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678195f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34074/0 (socket says 172.21.15.62:34074) 2023-07-14T16:21:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff779d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a267825dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55696/0 (socket says 172.21.15.62:55696) 2023-07-14T16:21:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77ad9d700 1 -- 172.21.15.62:0/716726712 learned_addr learned my addr 172.21.15.62:0/716726712 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77a59c700 1 -- 172.21.15.62:0/716726712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 msgr2=0x55a267825dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77a59c700 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a267825dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77a59c700 1 -- 172.21.15.62:0/716726712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 msgr2=0x55a2678195f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77a59c700 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678195f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77a59c700 1 -- 172.21.15.62:0/716726712 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff774011230 con 0x55a2676716b0 2023-07-14T16:21:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff779d9b700 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a267825dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.419+0000 7ff77ad9d700 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.420+0000 7ff77a59c700 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a26781d5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff770099760 tx=0x7ff76c014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2291af128ab6088 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.420+0000 7ff7697fa700 1 -- 172.21.15.62:0/716726712 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c022040 con 0x55a2676716b0 2023-07-14T16:21:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.420+0000 7ff7697fa700 1 -- 172.21.15.62:0/716726712 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff76c012e90 con 0x55a2676716b0 2023-07-14T16:21:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.420+0000 7ff7697fa700 1 -- 172.21.15.62:0/716726712 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0273f0 con 0x55a2676716b0 2023-07-14T16:21:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 -- 172.21.15.62:0/716726712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 msgr2=0x55a26781d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a26781d5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff770099760 tx=0x7ff76c014040 comp rx=0 tx=0).stop 2023-07-14T16:21:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 -- 172.21.15.62:0/716726712 shutdown_connections 2023-07-14T16:21:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a26781d5a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/716726712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a267825dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 -- 172.21.15.62:0/716726712 >> 172.21.15.62:0/716726712 conn(0x55a26780abe0 msgr2=0x55a2678310b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 -- 172.21.15.62:0/716726712 shutdown_connections 2023-07-14T16:21:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 -- 172.21.15.62:0/716726712 wait complete. 2023-07-14T16:21:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.421+0000 7ff7948ceb80 1 -- start start 2023-07-14T16:21:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff7948ceb80 1 Processor -- start 2023-07-14T16:21:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a267821c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a2678221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff7948ceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a267825710 con 0x55a2678166e0 2023-07-14T16:21:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a267825880 con 0x55a2676716b0 2023-07-14T16:21:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff7948ceb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a267825a40 con 0x55a267815840 2023-07-14T16:21:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff7948ceb80 10 client.0 _ll_register_callbacks cb 0x55a267704700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a267821c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.422+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a267821c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:54916/0 (socket says 172.21.15.62:54916) 2023-07-14T16:21:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678251d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34086/0 (socket says 172.21.15.62:34086) 2023-07-14T16:21:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff77a59c700 1 -- 172.21.15.62:0/695092361 learned_addr learned my addr 172.21.15.62:0/695092361 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff779d9b700 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a2678221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff77a59c700 1 -- 172.21.15.62:0/695092361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 msgr2=0x55a2678221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff77a59c700 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a2678221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff77a59c700 1 -- 172.21.15.62:0/695092361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 msgr2=0x55a2678251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff77a59c700 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678251d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff77a59c700 1 -- 172.21.15.62:0/695092361 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c1a46a0 con 0x55a2676716b0 2023-07-14T16:21:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff77a59c700 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a267821c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff76c005da0 tx=0x7ff76c01dd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=533a34e9d7e8659c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff761ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:21:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.423+0000 7ff761ffb700 1 -- 172.21.15.62:0/695092361 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c022040 con 0x55a2676716b0 2023-07-14T16:21:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff761ffb700 1 -- 172.21.15.62:0/695092361 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff76c028030 con 0x55a2676716b0 2023-07-14T16:21:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 1 -- 172.21.15.62:0/695092361 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff77400f040 con 0x55a2676716b0 2023-07-14T16:21:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff761ffb700 1 -- 172.21.15.62:0/695092361 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff76c0281f0 con 0x55a2676716b0 2023-07-14T16:21:20.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 10 client.26750 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 1 -- 172.21.15.62:0/695092361 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a267594d80 con 0x55a2676716b0 2023-07-14T16:21:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff761ffb700 1 -- 172.21.15.62:0/695092361 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff77400f040 con 0x55a2676716b0 2023-07-14T16:21:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff761ffb700 1 -- 172.21.15.62:0/695092361 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff76c032c20 con 0x55a2676716b0 2023-07-14T16:21:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 10 client.26750 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 10 client.26750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 20 client.26750 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a26787f2c0 0x55a2678816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 4 client.26750 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:21:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.424+0000 7ff7948ceb80 1 -- 172.21.15.62:0/695092361 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a267721ff0 con 0x55a26787f2c0 2023-07-14T16:21:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.425+0000 7ff779d9b700 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a26787f2c0 0x55a2678816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.425+0000 7ff779d9b700 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a26787f2c0 0x55a2678816d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5217a5415467a43d server_cookie=cb7c27535a06384 in_seq=0 out_seq=0 2023-07-14T16:21:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.425+0000 7ff761ffb700 10 client.26750 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff761ffb700 1 -- 172.21.15.62:0/695092361 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a267721ff0 con 0x55a26787f2c0 2023-07-14T16:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff761ffb700 10 client.26750 handle_command_reply: tid=0 2023-07-14T16:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 client.26750 shutdown 2023-07-14T16:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 -- 172.21.15.62:0/695092361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 msgr2=0x55a267821c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a267821c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff76c005da0 tx=0x7ff76c01dd80 comp rx=0 tx=0).stop 2023-07-14T16:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 -- 172.21.15.62:0/695092361 shutdown_connections 2023-07-14T16:21:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2678166e0 0x55a2678251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a26787f2c0 0x55a2678816d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2676716b0 0x55a267821c70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 --2- 172.21.15.62:0/695092361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a267815840 0x55a2678221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff761ffb700 0 client.26750 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 -- 172.21.15.62:0/695092361 >> 172.21.15.62:0/695092361 conn(0x55a26780abe0 msgr2=0x55a26781d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 -- 172.21.15.62:0/695092361 shutdown_connections 2023-07-14T16:21:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 1 -- 172.21.15.62:0/695092361 wait complete. 2023-07-14T16:21:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.433+0000 7ff7948ceb80 20 client.26750 trim_cache size 0 max 16384 2023-07-14T16:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:21:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 -- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff7640a11d0 msgr2=0x7ff7640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff7640a11d0 0x7ff7640a3680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff76c005c50 tx=0x7ff76c00b680 comp rx=0 tx=0).stop 2023-07-14T16:21:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 -- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c074550 msgr2=0x7ff77c19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c074550 0x7ff77c19c3f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff770005910 tx=0x7ff770005740 comp rx=0 tx=0).stop 2023-07-14T16:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 -- 172.21.15.62:0/5072401 shutdown_connections 2023-07-14T16:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff7640a11d0 0x7ff7640a3680 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff77c10c0e0 0x7ff77c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff77c072f70 0x7ff77c19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 --2- 172.21.15.62:0/5072401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c074550 0x7ff77c19c3f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 -- 172.21.15.62:0/5072401 >> 172.21.15.62:0/5072401 conn(0x7ff77c0fb130 msgr2=0x7ff77c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 -- 172.21.15.62:0/5072401 shutdown_connections 2023-07-14T16:21:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:20.434+0000 7ff780fff700 1 -- 172.21.15.62:0/5072401 wait complete. 2023-07-14T16:21:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:20 smithi164 ceph-mon[97427]: pgmap v528: 97 pgs: 97 active+clean; 3.3 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:21:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:21:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:21:21.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:21:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:20 smithi062 ceph-mon[95543]: pgmap v528: 97 pgs: 97 active+clean; 3.3 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:21:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:21:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:21:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:21:21.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:20 smithi081 ceph-mon[127795]: pgmap v528: 97 pgs: 97 active+clean; 3.3 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:21:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:21:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:21:21.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:21:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:22 smithi164 ceph-mon[97427]: pgmap v529: 97 pgs: 97 active+clean; 3.4 GiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:21:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:22 smithi062 ceph-mon[95543]: pgmap v529: 97 pgs: 97 active+clean; 3.4 GiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:21:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:22 smithi081 ceph-mon[127795]: pgmap v529: 97 pgs: 97 active+clean; 3.4 GiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:21:23.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:21:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 -- 172.21.15.62:0/3725814700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 msgr2=0x7f3224109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3725814700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 0x7f3224109e60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f321c00b0a0 tx=0x7f321c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 -- 172.21.15.62:0/3725814700 shutdown_connections 2023-07-14T16:21:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3725814700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 0x7f3224109e60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3725814700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32240744d0 0x7f3224072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3725814700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3224072ef0 0x7f3224073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 -- 172.21.15.62:0/3725814700 >> 172.21.15.62:0/3725814700 conn(0x7f32240fb170 msgr2=0x7f32240fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 -- 172.21.15.62:0/3725814700 shutdown_connections 2023-07-14T16:21:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.774+0000 7f322c1ff700 1 -- 172.21.15.62:0/3725814700 wait complete. 2023-07-14T16:21:23.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.776+0000 7f322c1ff700 1 Processor -- start 2023-07-14T16:21:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.776+0000 7f322c1ff700 1 -- start start 2023-07-14T16:21:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.776+0000 7f322c1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3224072ef0 0x7f32241979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.776+0000 7f322c1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32240744d0 0x7f3224197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.776+0000 7f322c1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 0x7f322419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.776+0000 7f322c1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419f560 con 0x7f3224107a00 2023-07-14T16:21:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.776+0000 7f322c1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419f6a0 con 0x7f3224072ef0 2023-07-14T16:21:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.776+0000 7f322c1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419f830 con 0x7f32240744d0 2023-07-14T16:21:23.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3224072ef0 0x7f32241979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f322a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 0x7f322419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3224072ef0 0x7f32241979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:54440/0 (socket says 172.21.15.62:54440) 2023-07-14T16:21:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 -- 172.21.15.62:0/3362261673 learned_addr learned my addr 172.21.15.62:0/3362261673 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f322979a700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32240744d0 0x7f3224197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 -- 172.21.15.62:0/3362261673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32240744d0 msgr2=0x7f3224197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32240744d0 0x7f3224197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 -- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 msgr2=0x7f322419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 0x7f322419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 -- 172.21.15.62:0/3362261673 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c009d00 con 0x7f3224072ef0 2023-07-14T16:21:23.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f322a79c700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 0x7f322419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.777+0000 7f3229f9b700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3224072ef0 0x7f32241979e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9a0 tx=0x7f322000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.778+0000 7f321affd700 1 -- 172.21.15.62:0/3362261673 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220009790 con 0x7f3224072ef0 2023-07-14T16:21:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.778+0000 7f321affd700 1 -- 172.21.15.62:0/3362261673 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3220010430 con 0x7f3224072ef0 2023-07-14T16:21:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.778+0000 7f322c1ff700 1 -- 172.21.15.62:0/3362261673 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322419fb10 con 0x7f3224072ef0 2023-07-14T16:21:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.778+0000 7f321affd700 1 -- 172.21.15.62:0/3362261673 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000cea0 con 0x7f3224072ef0 2023-07-14T16:21:23.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.778+0000 7f322c1ff700 1 -- 172.21.15.62:0/3362261673 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32241a0020 con 0x7f3224072ef0 2023-07-14T16:21:23.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.782+0000 7f321affd700 1 -- 172.21.15.62:0/3362261673 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f32200105a0 con 0x7f3224072ef0 2023-07-14T16:21:23.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.783+0000 7f323fbbeb80 1 Processor -- start 2023-07-14T16:21:23.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.783+0000 7f323fbbeb80 1 -- start start 2023-07-14T16:21:23.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cbe20 0x56206d4cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4ccc70 0x56206d4cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d3d0cb0 con 0x56206d4cbe20 2023-07-14T16:21:23.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d423a40 con 0x56206d4cb4d0 2023-07-14T16:21:23.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d41ced0 con 0x56206d4ccc70 2023-07-14T16:21:23.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f321affd700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32100a1150 0x7f32100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f321affd700 1 -- 172.21.15.62:0/3362261673 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3220014070 con 0x7f3224072ef0 2023-07-14T16:21:23.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f3229f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cbe20 0x56206d4cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f3229f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cbe20 0x56206d4cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33778/0 (socket says 172.21.15.62:33778) 2023-07-14T16:21:23.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f3229f9b700 1 -- 172.21.15.62:0/3819796244 learned_addr learned my addr 172.21.15.62:0/3819796244 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:23.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f322979a700 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4ccc70 0x56206d4cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f322a79c700 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.784+0000 7f322a79c700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32100a1150 0x7f32100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f3229f9b700 1 -- 172.21.15.62:0/3819796244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4ccc70 msgr2=0x56206d4cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f3229f9b700 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4ccc70 0x56206d4cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f3229f9b700 1 -- 172.21.15.62:0/3819796244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 msgr2=0x56206d4cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f3229f9b700 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f3229f9b700 1 -- 172.21.15.62:0/3819796244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206d24ade0 con 0x56206d4cbe20 2023-07-14T16:21:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f322a79c700 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f3229f9b700 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cbe20 0x56206d4cc6e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f32200a1e90 tx=0x7f3220099e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14447d64e804dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f322a79c700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32100a1150 0x7f32100a3600 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f321c00b780 tx=0x7f321c006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f320ffff700 1 -- 172.21.15.62:0/3819796244 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200be5a0 con 0x56206d4cbe20 2023-07-14T16:21:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f320ffff700 1 -- 172.21.15.62:0/3819796244 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3220010880 con 0x56206d4cbe20 2023-07-14T16:21:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.785+0000 7f320ffff700 1 -- 172.21.15.62:0/3819796244 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322009a1d0 con 0x56206d4cbe20 2023-07-14T16:21:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.786+0000 7f323fbbeb80 1 -- 172.21.15.62:0/3819796244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cbe20 msgr2=0x56206d4cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.786+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cbe20 0x56206d4cc6e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f32200a1e90 tx=0x7f3220099e10 comp rx=0 tx=0).stop 2023-07-14T16:21:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.786+0000 7f323fbbeb80 1 -- 172.21.15.62:0/3819796244 shutdown_connections 2023-07-14T16:21:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.786+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4ccc70 0x56206d4cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.786+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.786+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/3819796244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cbe20 0x56206d4cc6e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.786+0000 7f323fbbeb80 1 -- 172.21.15.62:0/3819796244 >> 172.21.15.62:0/3819796244 conn(0x56206d4c0da0 msgr2=0x56206d4c61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.787+0000 7f323fbbeb80 1 -- 172.21.15.62:0/3819796244 shutdown_connections 2023-07-14T16:21:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.787+0000 7f323fbbeb80 1 -- 172.21.15.62:0/3819796244 wait complete. 2023-07-14T16:21:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.787+0000 7f323fbbeb80 1 -- start start 2023-07-14T16:21:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.788+0000 7f323fbbeb80 1 Processor -- start 2023-07-14T16:21:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.788+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.788+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.788+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 0x56206d4dc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.788+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d3bcac0 con 0x56206d4cb4d0 2023-07-14T16:21:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.788+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d3bc7c0 con 0x56206d4ccc70 2023-07-14T16:21:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.788+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d2a0920 con 0x56206d4cbe20 2023-07-14T16:21:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.788+0000 7f323fbbeb80 10 client.0 _ll_register_callbacks cb 0x56206d3ba760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33784/0 (socket says 172.21.15.62:33784) 2023-07-14T16:21:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 -- 172.21.15.62:0/2910568809 learned_addr learned my addr 172.21.15.62:0/2910568809 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f322979a700 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f322a79c700 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 0x56206d4dc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 -- 172.21.15.62:0/2910568809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 msgr2=0x56206d4d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 -- 172.21.15.62:0/2910568809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 msgr2=0x56206d4dc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 0x56206d4dc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 -- 172.21.15.62:0/2910568809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32241a0020 con 0x56206d4cb4d0 2023-07-14T16:21:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f3229f9b700 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d5140 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c00 tx=0x7f3220010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d5a049f4c7f68e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.789+0000 7f320d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.790+0000 7f320d7fa700 1 -- 172.21.15.62:0/2910568809 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322009dfa0 con 0x56206d4cb4d0 2023-07-14T16:21:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.790+0000 7f320d7fa700 1 -- 172.21.15.62:0/2910568809 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f32200a23c0 con 0x56206d4cb4d0 2023-07-14T16:21:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.790+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2910568809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56206d3764f0 con 0x56206d4cb4d0 2023-07-14T16:21:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.790+0000 7f320d7fa700 1 -- 172.21.15.62:0/2910568809 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3220099f70 con 0x56206d4cb4d0 2023-07-14T16:21:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.790+0000 7f323fbbeb80 10 client.17058 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.790+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2910568809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56206d233c60 con 0x56206d4cb4d0 2023-07-14T16:21:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.790+0000 7f320d7fa700 1 -- 172.21.15.62:0/2910568809 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56206d3764f0 con 0x56206d4cb4d0 2023-07-14T16:21:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.791+0000 7f320d7fa700 1 -- 172.21.15.62:0/2910568809 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f32200b9d20 con 0x56206d4cb4d0 2023-07-14T16:21:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.791+0000 7f323fbbeb80 10 client.17058 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.791+0000 7f323fbbeb80 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.791+0000 7f323fbbeb80 20 client.17058 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.791+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56206d4e0b50 0x56206d4e2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.791+0000 7f323fbbeb80 4 client.17058 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.791+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2910568809 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56206d415eb0 con 0x56206d4e0b50 2023-07-14T16:21:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.791+0000 7f322979a700 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56206d4e0b50 0x56206d4e2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.792+0000 7f322979a700 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56206d4e0b50 0x56206d4e2f60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf31201367cc6ea server_cookie=1a0559554242ef8f in_seq=0 out_seq=0 2023-07-14T16:21:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.792+0000 7f320d7fa700 10 client.17058 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f320d7fa700 1 -- 172.21.15.62:0/2910568809 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56206d415eb0 con 0x56206d4e0b50 2023-07-14T16:21:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f320d7fa700 10 client.17058 handle_command_reply: tid=0 2023-07-14T16:21:23.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 client.17058 shutdown 2023-07-14T16:21:23.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2910568809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 msgr2=0x56206d4d5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d5140 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c00 tx=0x7f3220010ae0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2910568809 shutdown_connections 2023-07-14T16:21:23.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56206d4e0b50 0x56206d4e2f60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 0x56206d4dc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d5140 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f320d7fa700 0 client.17058 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2910568809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2910568809 >> 172.21.15.62:0/2910568809 conn(0x56206d4c0da0 msgr2=0x56206d4c58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2910568809 shutdown_connections 2023-07-14T16:21:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2910568809 wait complete. 2023-07-14T16:21:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.793+0000 7f323fbbeb80 20 client.17058 trim_cache size 0 max 16384 2023-07-14T16:21:23.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.815+0000 7f323fbbeb80 1 Processor -- start 2023-07-14T16:21:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.815+0000 7f323fbbeb80 1 -- start start 2023-07-14T16:21:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.815+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.815+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4dc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.815+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 0x56206d4cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.815+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d4d2e10 con 0x56206d4cb4d0 2023-07-14T16:21:23.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.815+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d4cb8a0 con 0x56206d4ccc70 2023-07-14T16:21:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.815+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d4c2310 con 0x56206d4cbe20 2023-07-14T16:21:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33800/0 (socket says 172.21.15.62:33800) 2023-07-14T16:21:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 -- 172.21.15.62:0/250793053 learned_addr learned my addr 172.21.15.62:0/250793053 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f322a79c700 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 0x56206d4cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f322979a700 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 -- 172.21.15.62:0/250793053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 msgr2=0x56206d4dc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4dc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 -- 172.21.15.62:0/250793053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 msgr2=0x56206d4cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 0x56206d4cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 -- 172.21.15.62:0/250793053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206d271250 con 0x56206d4cb4d0 2023-07-14T16:21:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f322979a700 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4dc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:21:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.816+0000 7f3229f9b700 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d3060 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3220099f00 tx=0x7f3220005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436e86a90cdcb39e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.817+0000 7f3218ff9700 1 -- 172.21.15.62:0/250793053 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200996f0 con 0x56206d4cb4d0 2023-07-14T16:21:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.817+0000 7f3218ff9700 1 -- 172.21.15.62:0/250793053 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32200a23c0 con 0x56206d4cb4d0 2023-07-14T16:21:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.817+0000 7f3218ff9700 1 -- 172.21.15.62:0/250793053 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322009e030 con 0x56206d4cb4d0 2023-07-14T16:21:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.817+0000 7f323fbbeb80 1 -- 172.21.15.62:0/250793053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 msgr2=0x56206d4d3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.817+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d3060 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3220099f00 tx=0x7f3220005b40 comp rx=0 tx=0).stop 2023-07-14T16:21:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.817+0000 7f323fbbeb80 1 -- 172.21.15.62:0/250793053 shutdown_connections 2023-07-14T16:21:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.817+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4ccc70 0x56206d4cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.818+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4cb4d0 0x56206d4d3060 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.818+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/250793053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4dc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.818+0000 7f323fbbeb80 1 -- 172.21.15.62:0/250793053 >> 172.21.15.62:0/250793053 conn(0x56206d4c0da0 msgr2=0x56206d4d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.818+0000 7f323fbbeb80 1 -- 172.21.15.62:0/250793053 shutdown_connections 2023-07-14T16:21:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.818+0000 7f323fbbeb80 1 -- 172.21.15.62:0/250793053 wait complete. 2023-07-14T16:21:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.818+0000 7f323fbbeb80 1 -- start start 2023-07-14T16:21:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.818+0000 7f323fbbeb80 1 Processor -- start 2023-07-14T16:21:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4dc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f323fbbeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4ccc70 0x56206d4d94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d4d9b60 con 0x56206d4ccc70 2023-07-14T16:21:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d534a00 con 0x56206d4cb4d0 2023-07-14T16:21:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f323fbbeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206d534b40 con 0x56206d4cbe20 2023-07-14T16:21:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f323fbbeb80 10 client.0 _ll_register_callbacks cb 0x56206d3ba760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f322a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4ccc70 0x56206d4d94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f322a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4ccc70 0x56206d4d94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33806/0 (socket says 172.21.15.62:33806) 2023-07-14T16:21:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f322979a700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f322a79c700 1 -- 172.21.15.62:0/2373249464 learned_addr learned my addr 172.21.15.62:0/2373249464 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.819+0000 7f3229f9b700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4dc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f322979a700 1 -- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 msgr2=0x56206d4dc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f322979a700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4dc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f322979a700 1 -- 172.21.15.62:0/2373249464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4ccc70 msgr2=0x56206d4d94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f322979a700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4ccc70 0x56206d4d94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f322979a700 1 -- 172.21.15.62:0/2373249464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56206d233c60 con 0x56206d4cbe20 2023-07-14T16:21:23.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f3229f9b700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4dc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f322a79c700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4ccc70 0x56206d4d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f322979a700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d8f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x56206d4e3540 tx=0x7f3214004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286ffe1cff69fed server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f320d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:21:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f320d7fa700 1 -- 172.21.15.62:0/2373249464 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214017040 con 0x56206d4cbe20 2023-07-14T16:21:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2373249464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3220010880 con 0x56206d4cbe20 2023-07-14T16:21:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.820+0000 7f320d7fa700 1 -- 172.21.15.62:0/2373249464 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f321401ebd0 con 0x56206d4cbe20 2023-07-14T16:21:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.821+0000 7f320d7fa700 1 -- 172.21.15.62:0/2373249464 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f321402c030 con 0x56206d4cbe20 2023-07-14T16:21:23.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.821+0000 7f323fbbeb80 10 client.26659 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.821+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2373249464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56206d2492b0 con 0x56206d4cbe20 2023-07-14T16:21:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.821+0000 7f320d7fa700 1 -- 172.21.15.62:0/2373249464 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3220010880 con 0x56206d4cbe20 2023-07-14T16:21:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.821+0000 7f320d7fa700 1 -- 172.21.15.62:0/2373249464 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3214024020 con 0x56206d4cbe20 2023-07-14T16:21:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.821+0000 7f323fbbeb80 10 client.26659 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.822+0000 7f323fbbeb80 10 client.26659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.822+0000 7f323fbbeb80 20 client.26659 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.822+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56206d5364c0 0x56206d5388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:23.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.822+0000 7f323fbbeb80 4 client.26659 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:21:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.822+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2373249464 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56206d3d8050 con 0x56206d5364c0 2023-07-14T16:21:23.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.822+0000 7f3229f9b700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56206d5364c0 0x56206d5388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.822+0000 7f3229f9b700 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56206d5364c0 0x56206d5388d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77cc9d79d5bcb665 server_cookie=16b1dbaf12d46684 in_seq=0 out_seq=0 2023-07-14T16:21:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.822+0000 7f320d7fa700 10 client.26659 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f320d7fa700 1 -- 172.21.15.62:0/2373249464 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56206d3d8050 con 0x56206d5364c0 2023-07-14T16:21:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f320d7fa700 10 client.26659 handle_command_reply: tid=0 2023-07-14T16:21:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 client.26659 shutdown 2023-07-14T16:21:23.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2373249464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 msgr2=0x56206d4d8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d8f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x56206d4e3540 tx=0x7f3214004690 comp rx=0 tx=0).stop 2023-07-14T16:21:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2373249464 shutdown_connections 2023-07-14T16:21:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56206d4ccc70 0x56206d4d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56206d5364c0 0x56206d5388d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56206d4cb4d0 0x56206d4dc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 --2- 172.21.15.62:0/2373249464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56206d4cbe20 0x56206d4d8f90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f320d7fa700 0 client.26659 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:23.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2373249464 >> 172.21.15.62:0/2373249464 conn(0x56206d4c0da0 msgr2=0x56206d4d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:23.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2373249464 shutdown_connections 2023-07-14T16:21:23.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 1 -- 172.21.15.62:0/2373249464 wait complete. 2023-07-14T16:21:23.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.823+0000 7f323fbbeb80 20 client.26659 trim_cache size 0 max 16384 2023-07-14T16:21:23.812 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:21:23.812 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:21:23.813 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:21:23.813 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:21:23.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 -- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32100a1150 msgr2=0x7f32100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32100a1150 0x7f32100a3600 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f321c00b780 tx=0x7f321c006210 comp rx=0 tx=0).stop 2023-07-14T16:21:23.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 -- 172.21.15.62:0/3362261673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3224072ef0 msgr2=0x7f32241979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:23.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3224072ef0 0x7f32241979e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9a0 tx=0x7f322000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 -- 172.21.15.62:0/3362261673 shutdown_connections 2023-07-14T16:21:23.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3224107a00 0x7f322419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f32100a1150 0x7f32100a3600 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3224072ef0 0x7f32241979e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 --2- 172.21.15.62:0/3362261673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32240744d0 0x7f3224197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:23.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 -- 172.21.15.62:0/3362261673 >> 172.21.15.62:0/3362261673 conn(0x7f32240fb170 msgr2=0x7f32240fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:23.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 -- 172.21.15.62:0/3362261673 shutdown_connections 2023-07-14T16:21:23.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:23.824+0000 7f322c1ff700 1 -- 172.21.15.62:0/3362261673 wait complete. 2023-07-14T16:21:23.821 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:21:23.821 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:21:24.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 -- 172.21.15.62:0/375273576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54105790 msgr2=0x7f5e54107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 --2- 172.21.15.62:0/375273576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54105790 0x7f5e54107bf0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00b0a0 tx=0x7f5e4c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 -- 172.21.15.62:0/375273576 shutdown_connections 2023-07-14T16:21:24.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 --2- 172.21.15.62:0/375273576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54105790 0x7f5e54107bf0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 --2- 172.21.15.62:0/375273576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e54100550 0x7f5e54102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 --2- 172.21.15.62:0/375273576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e54102e70 0x7f5e54105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 -- 172.21.15.62:0/375273576 >> 172.21.15.62:0/375273576 conn(0x7f5e540f9fd0 msgr2=0x7f5e540fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 -- 172.21.15.62:0/375273576 shutdown_connections 2023-07-14T16:21:24.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.075+0000 7f5e597ff700 1 -- 172.21.15.62:0/375273576 wait complete. 2023-07-14T16:21:24.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.076+0000 7f5e597ff700 1 Processor -- start 2023-07-14T16:21:24.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.076+0000 7f5e597ff700 1 -- start start 2023-07-14T16:21:24.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e597ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e54100550 0x7f5e541957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e597ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54102e70 0x7f5e54195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e597ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e54105790 0x7f5e5419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e597ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5419d340 con 0x7f5e54102e70 2023-07-14T16:21:24.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e597ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5419d480 con 0x7f5e54105790 2023-07-14T16:21:24.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e597ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5419d610 con 0x7f5e54100550 2023-07-14T16:21:24.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e527fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54102e70 0x7f5e54195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e52ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e54100550 0x7f5e541957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e527fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54102e70 0x7f5e54195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33816/0 (socket says 172.21.15.62:33816) 2023-07-14T16:21:24.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e52ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e54100550 0x7f5e541957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38854/0 (socket says 172.21.15.62:38854) 2023-07-14T16:21:24.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e527fc700 1 -- 172.21.15.62:0/1179305616 learned_addr learned my addr 172.21.15.62:0/1179305616 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.077+0000 7f5e537fe700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e54105790 0x7f5e5419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e527fc700 1 -- 172.21.15.62:0/1179305616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e54100550 msgr2=0x7f5e541957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e527fc700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e54100550 0x7f5e541957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e527fc700 1 -- 172.21.15.62:0/1179305616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e54105790 msgr2=0x7f5e5419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e527fc700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e54105790 0x7f5e5419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e527fc700 1 -- 172.21.15.62:0/1179305616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c009d00 con 0x7f5e54102e70 2023-07-14T16:21:24.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e527fc700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54102e70 0x7f5e54195d00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800d950 tx=0x7f5e4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e3bfff700 1 -- 172.21.15.62:0/1179305616 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800f890 con 0x7f5e54102e70 2023-07-14T16:21:24.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5419d8f0 con 0x7f5e54102e70 2023-07-14T16:21:24.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e3bfff700 1 -- 172.21.15.62:0/1179305616 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e4800fe80 con 0x7f5e54102e70 2023-07-14T16:21:24.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.078+0000 7f5e3bfff700 1 -- 172.21.15.62:0/1179305616 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800e3b0 con 0x7f5e54102e70 2023-07-14T16:21:24.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.079+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5419de50 con 0x7f5e54102e70 2023-07-14T16:21:24.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.080+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e540fbb40 con 0x7f5e54102e70 2023-07-14T16:21:24.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.082+0000 7f5e3bfff700 1 -- 172.21.15.62:0/1179305616 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5e48010460 con 0x7f5e54102e70 2023-07-14T16:21:24.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.084+0000 7f5e3bfff700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e3c0a1220 0x7f5e3c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.084+0000 7f5e3bfff700 1 -- 172.21.15.62:0/1179305616 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5e480cca00 con 0x7f5e54102e70 2023-07-14T16:21:24.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.084+0000 7f5e52ffd700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e3c0a1220 0x7f5e3c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.085+0000 7f5e52ffd700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e3c0a1220 0x7f5e3c0a36d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009c60 tx=0x7f5e44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.089+0000 7f5e3bfff700 1 -- 172.21.15.62:0/1179305616 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5e48096100 con 0x7f5e54102e70 2023-07-14T16:21:24.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.363+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e5402cce0 con 0x7f5e54102e70 2023-07-14T16:21:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.364+0000 7f5e3bfff700 1 -- 172.21.15.62:0/1179305616 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f5e48020090 con 0x7f5e54102e70 2023-07-14T16:21:24.341 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:21:24.342 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:21:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e3c0a1220 msgr2=0x7f5e3c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e3c0a1220 0x7f5e3c0a36d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009c60 tx=0x7f5e44009260 comp rx=0 tx=0).stop 2023-07-14T16:21:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54102e70 msgr2=0x7f5e54195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54102e70 0x7f5e54195d00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800d950 tx=0x7f5e4800dc60 comp rx=0 tx=0).stop 2023-07-14T16:21:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 shutdown_connections 2023-07-14T16:21:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e3c0a1220 0x7f5e3c0a36d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e54105790 0x7f5e5419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e54100550 0x7f5e541957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 --2- 172.21.15.62:0/1179305616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e54102e70 0x7f5e54195d00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 >> 172.21.15.62:0/1179305616 conn(0x7f5e540f9fd0 msgr2=0x7f5e541011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 shutdown_connections 2023-07-14T16:21:24.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.366+0000 7f5e597ff700 1 -- 172.21.15.62:0/1179305616 wait complete. 2023-07-14T16:21:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:21:24.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:21:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 -- 172.21.15.62:0/1861209576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed34101610 msgr2=0x7fed34107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 --2- 172.21.15.62:0/1861209576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed34101610 0x7fed34107eb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00b0a0 tx=0x7fed2c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 -- 172.21.15.62:0/1861209576 shutdown_connections 2023-07-14T16:21:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 --2- 172.21.15.62:0/1861209576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed34101610 0x7fed34107eb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 --2- 172.21.15.62:0/1861209576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed340ff9d0 0x7fed340ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 --2- 172.21.15.62:0/1861209576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed34100c60 0x7fed341010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 -- 172.21.15.62:0/1861209576 >> 172.21.15.62:0/1861209576 conn(0x7fed340fb010 msgr2=0x7fed340fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 -- 172.21.15.62:0/1861209576 shutdown_connections 2023-07-14T16:21:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.605+0000 7fed3359e700 1 -- 172.21.15.62:0/1861209576 wait complete. 2023-07-14T16:21:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.607+0000 7fed3359e700 1 Processor -- start 2023-07-14T16:21:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.607+0000 7fed3359e700 1 -- start start 2023-07-14T16:21:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.607+0000 7fed3359e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed340ff9d0 0x7fed34195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.607+0000 7fed3359e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed34100c60 0x7fed34195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.607+0000 7fed3359e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed34101610 0x7fed3419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.607+0000 7fed3359e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3419d100 con 0x7fed340ff9d0 2023-07-14T16:21:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.607+0000 7fed3359e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3419d240 con 0x7fed34101610 2023-07-14T16:21:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.607+0000 7fed3359e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3419d3d0 con 0x7fed34100c60 2023-07-14T16:21:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed340ff9d0 0x7fed34195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed340ff9d0 0x7fed34195580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33838/0 (socket says 172.21.15.62:33838) 2023-07-14T16:21:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 -- 172.21.15.62:0/2172998652 learned_addr learned my addr 172.21.15.62:0/2172998652 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed34100c60 0x7fed34195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed32d9d700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed34101610 0x7fed3419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 -- 172.21.15.62:0/2172998652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed34100c60 msgr2=0x7fed34195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed34100c60 0x7fed34195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 -- 172.21.15.62:0/2172998652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed34101610 msgr2=0x7fed3419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed34101610 0x7fed3419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 -- 172.21.15.62:0/2172998652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c009d00 con 0x7fed340ff9d0 2023-07-14T16:21:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed32d9d700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed34101610 0x7fed3419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:21:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.608+0000 7fed3259c700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed340ff9d0 0x7fed34195580 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fed2400e9a0 tx=0x7fed2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.609+0000 7fed237fe700 1 -- 172.21.15.62:0/2172998652 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24009790 con 0x7fed340ff9d0 2023-07-14T16:21:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.609+0000 7fed237fe700 1 -- 172.21.15.62:0/2172998652 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed2400cbd0 con 0x7fed340ff9d0 2023-07-14T16:21:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.609+0000 7fed3359e700 1 -- 172.21.15.62:0/2172998652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3419d6b0 con 0x7fed340ff9d0 2023-07-14T16:21:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.609+0000 7fed237fe700 1 -- 172.21.15.62:0/2172998652 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24018680 con 0x7fed340ff9d0 2023-07-14T16:21:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.609+0000 7fed3359e700 1 -- 172.21.15.62:0/2172998652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3419dbc0 con 0x7fed340ff9d0 2023-07-14T16:21:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.613+0000 7fed237fe700 1 -- 172.21.15.62:0/2172998652 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fed2400cd40 con 0x7fed340ff9d0 2023-07-14T16:21:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.614+0000 7fed4c451b80 1 Processor -- start 2023-07-14T16:21:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.614+0000 7fed4c451b80 1 -- start start 2023-07-14T16:21:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.614+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9e720 0x55ea3ef9eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.614+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 0x55ea3ef9f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.614+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9fe30 0x55ea3efa02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.614+0000 7fed4c451b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3eea3cb0 con 0x55ea3ef9fe30 2023-07-14T16:21:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.614+0000 7fed4c451b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3eef6a40 con 0x55ea3ef9f070 2023-07-14T16:21:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.614+0000 7fed4c451b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3eeefed0 con 0x55ea3ef9e720 2023-07-14T16:21:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed31d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9fe30 0x55ea3efa02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed31d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9fe30 0x55ea3efa02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33854/0 (socket says 172.21.15.62:33854) 2023-07-14T16:21:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed32d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9e720 0x55ea3ef9eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed3259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 0x55ea3ef9f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed31d9b700 1 -- 172.21.15.62:0/2749949980 learned_addr learned my addr 172.21.15.62:0/2749949980 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed31d9b700 1 -- 172.21.15.62:0/2749949980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9e720 msgr2=0x55ea3ef9eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2749949980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9e720 0x55ea3ef9eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed31d9b700 1 -- 172.21.15.62:0/2749949980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 msgr2=0x55ea3ef9f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2749949980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 0x55ea3ef9f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed31d9b700 1 -- 172.21.15.62:0/2749949980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea3ed1dde0 con 0x55ea3ef9fe30 2023-07-14T16:21:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.615+0000 7fed32d9d700 1 --2- 172.21.15.62:0/2749949980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9e720 0x55ea3ef9eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.616+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2749949980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9fe30 0x55ea3efa02a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fed28009d80 tx=0x7fed28011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26f2e2c98c7764b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.616+0000 7fed20ff9700 1 -- 172.21.15.62:0/2749949980 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28004230 con 0x55ea3ef9fe30 2023-07-14T16:21:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.616+0000 7fed20ff9700 1 -- 172.21.15.62:0/2749949980 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed28004390 con 0x55ea3ef9fe30 2023-07-14T16:21:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.616+0000 7fed20ff9700 1 -- 172.21.15.62:0/2749949980 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28019800 con 0x55ea3ef9fe30 2023-07-14T16:21:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.616+0000 7fed237fe700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed1c0a11f0 0x7fed1c0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.616+0000 7fed237fe700 1 -- 172.21.15.62:0/2172998652 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed24014070 con 0x7fed340ff9d0 2023-07-14T16:21:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.616+0000 7fed32d9d700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed1c0a11f0 0x7fed1c0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 -- 172.21.15.62:0/2749949980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9fe30 msgr2=0x55ea3efa02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2749949980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9fe30 0x55ea3efa02a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fed28009d80 tx=0x7fed28011720 comp rx=0 tx=0).stop 2023-07-14T16:21:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 -- 172.21.15.62:0/2749949980 shutdown_connections 2023-07-14T16:21:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2749949980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9fe30 0x55ea3efa02a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2749949980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9e720 0x55ea3ef9eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2749949980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 0x55ea3ef9f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 -- 172.21.15.62:0/2749949980 >> 172.21.15.62:0/2749949980 conn(0x55ea3ef93ff0 msgr2=0x55ea3ef99460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 -- 172.21.15.62:0/2749949980 shutdown_connections 2023-07-14T16:21:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed32d9d700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed1c0a11f0 0x7fed1c0a36a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009850 tx=0x7fed2c009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.617+0000 7fed4c451b80 1 -- 172.21.15.62:0/2749949980 wait complete. 2023-07-14T16:21:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.618+0000 7fed4c451b80 1 -- start start 2023-07-14T16:21:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.618+0000 7fed4c451b80 1 Processor -- start 2023-07-14T16:21:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 0x55ea3efacbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 0x55ea3efb3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed4c451b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3ee8fac0 con 0x55ea3ef9e720 2023-07-14T16:21:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed4c451b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3ee8f7c0 con 0x55ea3ef9fe30 2023-07-14T16:21:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed4c451b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3ed73920 con 0x55ea3ef9f070 2023-07-14T16:21:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed4c451b80 10 client.0 _ll_register_callbacks cb 0x55ea3ee8d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33860/0 (socket says 172.21.15.62:33860) 2023-07-14T16:21:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed3259c700 1 -- 172.21.15.62:0/2987813786 learned_addr learned my addr 172.21.15.62:0/2987813786 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed32d9d700 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 0x55ea3efb3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 0x55ea3efacbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed3259c700 1 -- 172.21.15.62:0/2987813786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 msgr2=0x55ea3efacbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed3259c700 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 0x55ea3efacbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed3259c700 1 -- 172.21.15.62:0/2987813786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 msgr2=0x55ea3efb3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed3259c700 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 0x55ea3efb3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.619+0000 7fed3259c700 1 -- 172.21.15.62:0/2987813786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed34108720 con 0x55ea3ef9e720 2023-07-14T16:21:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed3259c700 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efac680 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fed240a1ee0 tx=0x7fed24096430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca21b2542c01e81b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed19ffb700 1 -- 172.21.15.62:0/2987813786 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24099920 con 0x55ea3ef9e720 2023-07-14T16:21:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed19ffb700 1 -- 172.21.15.62:0/2987813786 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed24099fc0 con 0x55ea3ef9e720 2023-07-14T16:21:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed4c451b80 1 -- 172.21.15.62:0/2987813786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea3ee494f0 con 0x55ea3ef9e720 2023-07-14T16:21:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed19ffb700 1 -- 172.21.15.62:0/2987813786 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed2409dff0 con 0x55ea3ef9e720 2023-07-14T16:21:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed4c451b80 10 client.17082 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed4c451b80 1 -- 172.21.15.62:0/2987813786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fed280099e0 con 0x55ea3ef9e720 2023-07-14T16:21:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.620+0000 7fed19ffb700 1 -- 172.21.15.62:0/2987813786 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ea3ee494f0 con 0x55ea3ef9e720 2023-07-14T16:21:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.621+0000 7fed19ffb700 1 -- 172.21.15.62:0/2987813786 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fed240a1f80 con 0x55ea3ef9e720 2023-07-14T16:21:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.621+0000 7fed4c451b80 10 client.17082 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.621+0000 7fed4c451b80 10 client.17082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:21:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.621+0000 7fed4c451b80 20 client.17082 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.621+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55ea3efb83f0 0x55ea3efba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.621+0000 7fed4c451b80 4 client.17082 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.621+0000 7fed4c451b80 1 -- 172.21.15.62:0/2987813786 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea3eee8eb0 con 0x55ea3efb83f0 2023-07-14T16:21:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.621+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55ea3efb83f0 0x55ea3efba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.622+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55ea3efb83f0 0x55ea3efba800 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b944e1d7cd85d07 server_cookie=365558c8e11b4022 in_seq=0 out_seq=0 2023-07-14T16:21:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.622+0000 7fed19ffb700 10 client.17082 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:21:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed19ffb700 1 -- 172.21.15.62:0/2987813786 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ea3eee8eb0 con 0x55ea3efb83f0 2023-07-14T16:21:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed19ffb700 10 client.17082 handle_command_reply: tid=0 2023-07-14T16:21:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 client.17082 shutdown 2023-07-14T16:21:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 -- 172.21.15.62:0/2987813786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 msgr2=0x55ea3efac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efac680 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fed240a1ee0 tx=0x7fed24096430 comp rx=0 tx=0).stop 2023-07-14T16:21:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 -- 172.21.15.62:0/2987813786 shutdown_connections 2023-07-14T16:21:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55ea3efb83f0 0x55ea3efba800 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 0x55ea3efb3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efac680 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2987813786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 0x55ea3efacbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed19ffb700 0 client.17082 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:21:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 -- 172.21.15.62:0/2987813786 >> 172.21.15.62:0/2987813786 conn(0x55ea3ef93ff0 msgr2=0x55ea3ef98b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 -- 172.21.15.62:0/2987813786 shutdown_connections 2023-07-14T16:21:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.623+0000 7fed4c451b80 1 -- 172.21.15.62:0/2987813786 wait complete. 2023-07-14T16:21:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.624+0000 7fed4c451b80 20 client.17082 trim_cache size 0 max 16384 2023-07-14T16:21:24.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.639+0000 7fed4c451b80 1 Processor -- start 2023-07-14T16:21:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed4c451b80 1 -- start start 2023-07-14T16:21:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3ef99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 0x55ea3efb3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9fe30 0x55ea3efbf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed4c451b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3efbf6a0 con 0x55ea3ef9e720 2023-07-14T16:21:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed4c451b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3efab150 con 0x55ea3ef9f070 2023-07-14T16:21:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed4c451b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3efba7d0 con 0x55ea3ef9fe30 2023-07-14T16:21:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3ef99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3ef99430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33866/0 (socket says 172.21.15.62:33866) 2023-07-14T16:21:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.640+0000 7fed3259c700 1 -- 172.21.15.62:0/1155658056 learned_addr learned my addr 172.21.15.62:0/1155658056 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed31d9b700 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 0x55ea3efb3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed32d9d700 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9fe30 0x55ea3efbf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed3259c700 1 -- 172.21.15.62:0/1155658056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9fe30 msgr2=0x55ea3efbf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed3259c700 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9fe30 0x55ea3efbf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed3259c700 1 -- 172.21.15.62:0/1155658056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 msgr2=0x55ea3efb3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed3259c700 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 0x55ea3efb3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed3259c700 1 -- 172.21.15.62:0/1155658056 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea3ed06c60 con 0x55ea3ef9e720 2023-07-14T16:21:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed32d9d700 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9fe30 0x55ea3efbf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed3259c700 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3ef99430 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fed24096a30 tx=0x7fed24005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28b5195e7092249 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed217fa700 1 -- 172.21.15.62:0/1155658056 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24099920 con 0x55ea3ef9e720 2023-07-14T16:21:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed217fa700 1 -- 172.21.15.62:0/1155658056 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed24099fc0 con 0x55ea3ef9e720 2023-07-14T16:21:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.641+0000 7fed217fa700 1 -- 172.21.15.62:0/1155658056 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240105d0 con 0x55ea3ef9e720 2023-07-14T16:21:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.642+0000 7fed4c451b80 1 -- 172.21.15.62:0/1155658056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 msgr2=0x55ea3ef99430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.642+0000 7fed4c451b80 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3ef99430 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fed24096a30 tx=0x7fed24005d00 comp rx=0 tx=0).stop 2023-07-14T16:21:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.642+0000 7fed4c451b80 1 -- 172.21.15.62:0/1155658056 shutdown_connections 2023-07-14T16:21:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.642+0000 7fed4c451b80 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9fe30 0x55ea3efbf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.642+0000 7fed4c451b80 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3ef99430 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.642+0000 7fed4c451b80 1 --2- 172.21.15.62:0/1155658056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9f070 0x55ea3efb3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.642+0000 7fed4c451b80 1 -- 172.21.15.62:0/1155658056 >> 172.21.15.62:0/1155658056 conn(0x55ea3ef93ff0 msgr2=0x55ea3ef95560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.642+0000 7fed4c451b80 1 -- 172.21.15.62:0/1155658056 shutdown_connections 2023-07-14T16:21:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.643+0000 7fed4c451b80 1 -- 172.21.15.62:0/1155658056 wait complete. 2023-07-14T16:21:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.643+0000 7fed4c451b80 1 -- start start 2023-07-14T16:21:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.643+0000 7fed4c451b80 1 Processor -- start 2023-07-14T16:21:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efafde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 0x55ea3efb0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed4c451b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 0x55ea3efb3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed4c451b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3efb3880 con 0x55ea3ef9e720 2023-07-14T16:21:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed4c451b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3efb39f0 con 0x55ea3ef9fe30 2023-07-14T16:21:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed4c451b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3efb3bb0 con 0x55ea3ef9f070 2023-07-14T16:21:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed4c451b80 10 client.0 _ll_register_callbacks cb 0x7fed24010960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efafde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efafde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33874/0 (socket says 172.21.15.62:33874) 2023-07-14T16:21:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed3259c700 1 -- 172.21.15.62:0/2518335278 learned_addr learned my addr 172.21.15.62:0/2518335278 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.644+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 0x55ea3efb0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed32d9d700 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 0x55ea3efb3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed3259c700 1 -- 172.21.15.62:0/2518335278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 msgr2=0x55ea3efb0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed3259c700 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 0x55ea3efb0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed3259c700 1 -- 172.21.15.62:0/2518335278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 msgr2=0x55ea3efb3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed3259c700 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 0x55ea3efb3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed3259c700 1 -- 172.21.15.62:0/2518335278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed280099e0 con 0x55ea3ef9e720 2023-07-14T16:21:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed3259c700 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efafde0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fed240964c0 tx=0x7fed24096430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4004a45ee1ab42f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed19ffb700 1 -- 172.21.15.62:0/2518335278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24010730 con 0x55ea3ef9e720 2023-07-14T16:21:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed19ffb700 1 -- 172.21.15.62:0/2518335278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed240a2410 con 0x55ea3ef9e720 2023-07-14T16:21:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed4c451b80 1 -- 172.21.15.62:0/2518335278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed28004390 con 0x55ea3ef9e720 2023-07-14T16:21:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.645+0000 7fed19ffb700 1 -- 172.21.15.62:0/2518335278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed2409dff0 con 0x55ea3ef9e720 2023-07-14T16:21:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed4c451b80 10 client.17088 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed4c451b80 1 -- 172.21.15.62:0/2518335278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ea3ed44250 con 0x55ea3ef9e720 2023-07-14T16:21:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed19ffb700 1 -- 172.21.15.62:0/2518335278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed240b8020 con 0x55ea3ef9e720 2023-07-14T16:21:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed19ffb700 1 -- 172.21.15.62:0/2518335278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fed240b2140 con 0x55ea3ef9e720 2023-07-14T16:21:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed4c451b80 10 client.17088 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed4c451b80 10 client.17088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:21:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed4c451b80 20 client.17088 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55ea3f00d190 0x55ea3f00f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed4c451b80 4 client.17088 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:21:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.646+0000 7fed4c451b80 1 -- 172.21.15.62:0/2518335278 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea3eeab050 con 0x55ea3f00d190 2023-07-14T16:21:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.647+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55ea3f00d190 0x55ea3f00f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.647+0000 7fed31d9b700 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55ea3f00d190 0x55ea3f00f570 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5b5fb94865b7430 server_cookie=8773f530fdff49a6 in_seq=0 out_seq=0 2023-07-14T16:21:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.647+0000 7fed19ffb700 10 client.17088 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:21:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.647+0000 7fed19ffb700 1 -- 172.21.15.62:0/2518335278 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea3eeab050 con 0x55ea3f00d190 2023-07-14T16:21:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.647+0000 7fed19ffb700 10 client.17088 handle_command_reply: tid=0 2023-07-14T16:21:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 client.17088 shutdown 2023-07-14T16:21:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 -- 172.21.15.62:0/2518335278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 msgr2=0x55ea3efafde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efafde0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fed240964c0 tx=0x7fed24096430 comp rx=0 tx=0).stop 2023-07-14T16:21:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 -- 172.21.15.62:0/2518335278 shutdown_connections 2023-07-14T16:21:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55ea3f00d190 0x55ea3f00f570 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ea3ef9fe30 0x55ea3efb3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3ef9e720 0x55ea3efafde0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 --2- 172.21.15.62:0/2518335278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ea3ef9f070 0x55ea3efb0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 -- 172.21.15.62:0/2518335278 >> 172.21.15.62:0/2518335278 conn(0x55ea3ef93ff0 msgr2=0x55ea3efa2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed19ffb700 0 client.17088 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:21:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 -- 172.21.15.62:0/2518335278 shutdown_connections 2023-07-14T16:21:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 1 -- 172.21.15.62:0/2518335278 wait complete. 2023-07-14T16:21:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.648+0000 7fed4c451b80 20 client.17088 trim_cache size 0 max 16384 2023-07-14T16:21:24.634 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:21:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 -- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed1c0a11f0 msgr2=0x7fed1c0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed1c0a11f0 0x7fed1c0a36a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009850 tx=0x7fed2c009780 comp rx=0 tx=0).stop 2023-07-14T16:21:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 -- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed340ff9d0 msgr2=0x7fed34195580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed340ff9d0 0x7fed34195580 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fed2400e9a0 tx=0x7fed2400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 -- 172.21.15.62:0/2172998652 shutdown_connections 2023-07-14T16:21:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed1c0a11f0 0x7fed1c0a36a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed34101610 0x7fed3419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed340ff9d0 0x7fed34195580 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 --2- 172.21.15.62:0/2172998652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed34100c60 0x7fed34195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 -- 172.21.15.62:0/2172998652 >> 172.21.15.62:0/2172998652 conn(0x7fed340fb010 msgr2=0x7fed340fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 -- 172.21.15.62:0/2172998652 shutdown_connections 2023-07-14T16:21:24.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.649+0000 7fed3359e700 1 -- 172.21.15.62:0/2172998652 wait complete. 2023-07-14T16:21:24.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:21:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 -- 172.21.15.62:0/3706768588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81004b0 msgr2=0x7f18b8102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 --2- 172.21.15.62:0/3706768588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81004b0 0x7f18b8102890 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009990 tx=0x7f18a8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 -- 172.21.15.62:0/3706768588 shutdown_connections 2023-07-14T16:21:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 --2- 172.21.15.62:0/3706768588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18b81056f0 0x7f18b8107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 --2- 172.21.15.62:0/3706768588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81004b0 0x7f18b8102890 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 --2- 172.21.15.62:0/3706768588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18b8102dd0 0x7f18b81051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 -- 172.21.15.62:0/3706768588 >> 172.21.15.62:0/3706768588 conn(0x7f18b80f9ed0 msgr2=0x7f18b80fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 -- 172.21.15.62:0/3706768588 shutdown_connections 2023-07-14T16:21:24.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.899+0000 7f18b7fff700 1 -- 172.21.15.62:0/3706768588 wait complete. 2023-07-14T16:21:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.900+0000 7f18b7fff700 1 Processor -- start 2023-07-14T16:21:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.900+0000 7f18b7fff700 1 -- start start 2023-07-14T16:21:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.900+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18b81004b0 0x7f18b80ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18b8102dd0 0x7f18b80fe460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81056f0 0x7f18b80fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b7fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80ff140 con 0x7f18b81056f0 2023-07-14T16:21:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b7fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80ff2b0 con 0x7f18b81004b0 2023-07-14T16:21:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b7fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b819f520 con 0x7f18b8102dd0 2023-07-14T16:21:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81056f0 0x7f18b80fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81056f0 0x7f18b80fe9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33878/0 (socket says 172.21.15.62:33878) 2023-07-14T16:21:24.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b77fe700 1 -- 172.21.15.62:0/2466348943 learned_addr learned my addr 172.21.15.62:0/2466348943 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18b81004b0 0x7f18b80ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.901+0000 7f18b67fc700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18b8102dd0 0x7f18b80fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b77fe700 1 -- 172.21.15.62:0/2466348943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18b8102dd0 msgr2=0x7f18b80fe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b77fe700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18b8102dd0 0x7f18b80fe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b77fe700 1 -- 172.21.15.62:0/2466348943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18b81004b0 msgr2=0x7f18b80ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b77fe700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18b81004b0 0x7f18b80ffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b77fe700 1 -- 172.21.15.62:0/2466348943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80095f0 con 0x7f18b81056f0 2023-07-14T16:21:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18b81004b0 0x7f18b80ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b77fe700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81056f0 0x7f18b80fe9a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f18b0007ee0 tx=0x7f18b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18bc9fd700 1 -- 172.21.15.62:0/2466348943 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0017070 con 0x7f18b81056f0 2023-07-14T16:21:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18bc9fd700 1 -- 172.21.15.62:0/2466348943 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18b000f040 con 0x7f18b81056f0 2023-07-14T16:21:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18bc9fd700 1 -- 172.21.15.62:0/2466348943 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0005570 con 0x7f18b81056f0 2023-07-14T16:21:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b7fff700 1 -- 172.21.15.62:0/2466348943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b819f800 con 0x7f18b81056f0 2023-07-14T16:21:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.902+0000 7f18b7fff700 1 -- 172.21.15.62:0/2466348943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b819fcc0 con 0x7f18b81056f0 2023-07-14T16:21:24.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.907+0000 7f18bc9fd700 1 -- 172.21.15.62:0/2466348943 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f18b000db20 con 0x7f18b81056f0 2023-07-14T16:21:24.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.907+0000 7f18d1314b80 1 Processor -- start 2023-07-14T16:21:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.908+0000 7f18d1314b80 1 -- start start 2023-07-14T16:21:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.908+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 0x55bdac738b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.908+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.908+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac73a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.908+0000 7f18d1314b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac63dcb0 con 0x55bdac739e30 2023-07-14T16:21:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.908+0000 7f18d1314b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac690a40 con 0x55bdac738720 2023-07-14T16:21:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.908+0000 7f18d1314b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac689ed0 con 0x55bdac739070 2023-07-14T16:21:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac73a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac73a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33894/0 (socket says 172.21.15.62:33894) 2023-07-14T16:21:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 -- 172.21.15.62:0/635036078 learned_addr learned my addr 172.21.15.62:0/635036078 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 0x55bdac738b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18bc9fd700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18a00a1270 0x7f18a00a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 -- 172.21.15.62:0/635036078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 msgr2=0x55bdac7398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 -- 172.21.15.62:0/635036078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 msgr2=0x55bdac738b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 0x55bdac738b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18b67fc700 1 -- 172.21.15.62:0/635036078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdac4b7de0 con 0x55bdac739e30 2023-07-14T16:21:24.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.909+0000 7f18bc9fd700 1 -- 172.21.15.62:0/2466348943 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18b00cbaa0 con 0x7f18b81056f0 2023-07-14T16:21:24.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.910+0000 7f18b77fe700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18a00a1270 0x7f18a00a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.910+0000 7f18b67fc700 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac73a2a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0118e0 tx=0x7f18ac009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4285792f07995f6b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.910+0000 7f189dffb700 1 -- 172.21.15.62:0/635036078 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac004030 con 0x55bdac739e30 2023-07-14T16:21:24.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.910+0000 7f189dffb700 1 -- 172.21.15.62:0/635036078 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18ac003680 con 0x55bdac739e30 2023-07-14T16:21:24.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.910+0000 7f189dffb700 1 -- 172.21.15.62:0/635036078 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac01e600 con 0x55bdac739e30 2023-07-14T16:21:24.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 -- 172.21.15.62:0/635036078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 msgr2=0x55bdac73a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18b77fe700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18a00a1270 0x7f18a00a3720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f18b0007fd0 tx=0x7f18b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac73a2a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0118e0 tx=0x7f18ac009880 comp rx=0 tx=0).stop 2023-07-14T16:21:24.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 -- 172.21.15.62:0/635036078 shutdown_connections 2023-07-14T16:21:24.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac73a2a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 0x55bdac738b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 --2- 172.21.15.62:0/635036078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 -- 172.21.15.62:0/635036078 >> 172.21.15.62:0/635036078 conn(0x55bdac72dff0 msgr2=0x55bdac733460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 -- 172.21.15.62:0/635036078 shutdown_connections 2023-07-14T16:21:24.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.911+0000 7f18d1314b80 1 -- 172.21.15.62:0/635036078 wait complete. 2023-07-14T16:21:24.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.912+0000 7f18d1314b80 1 -- start start 2023-07-14T16:21:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.912+0000 7f18d1314b80 1 Processor -- start 2023-07-14T16:21:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.912+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 0x55bdac742380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.912+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.912+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac749900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.912+0000 7f18d1314b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac629ac0 con 0x55bdac739e30 2023-07-14T16:21:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.912+0000 7f18d1314b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac6297c0 con 0x55bdac738720 2023-07-14T16:21:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.912+0000 7f18d1314b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac50d920 con 0x55bdac739070 2023-07-14T16:21:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18d1314b80 10 client.0 _ll_register_callbacks cb 0x55bdac627760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac749900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac749900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33910/0 (socket says 172.21.15.62:33910) 2023-07-14T16:21:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b67fc700 1 -- 172.21.15.62:0/1217741190 learned_addr learned my addr 172.21.15.62:0/1217741190 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b67fc700 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 0x55bdac742380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b6ffd700 1 -- 172.21.15.62:0/1217741190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 msgr2=0x55bdac742380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 0x55bdac742380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b6ffd700 1 -- 172.21.15.62:0/1217741190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 msgr2=0x55bdac749900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac749900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.913+0000 7f18b6ffd700 1 -- 172.21.15.62:0/1217741190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac019040 con 0x55bdac739070 2023-07-14T16:21:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.914+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7428c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18a8000c00 tx=0x7f18a8004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f925f0546ef687a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.914+0000 7f1896ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:21:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.914+0000 7f1896ffd700 1 -- 172.21.15.62:0/1217741190 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a801d070 con 0x55bdac739070 2023-07-14T16:21:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.914+0000 7f1896ffd700 1 -- 172.21.15.62:0/1217741190 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18a800b910 con 0x55bdac739070 2023-07-14T16:21:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.914+0000 7f18d1314b80 1 -- 172.21.15.62:0/1217741190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdac5e34f0 con 0x55bdac739070 2023-07-14T16:21:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f1896ffd700 1 -- 172.21.15.62:0/1217741190 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a80056f0 con 0x55bdac739070 2023-07-14T16:21:24.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f18d1314b80 10 client.26695 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:24.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f18d1314b80 1 -- 172.21.15.62:0/1217741190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bdac4a0c60 con 0x55bdac739070 2023-07-14T16:21:24.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f1896ffd700 1 -- 172.21.15.62:0/1217741190 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bdac5e34f0 con 0x55bdac739070 2023-07-14T16:21:24.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f1896ffd700 1 -- 172.21.15.62:0/1217741190 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f18a8004ba0 con 0x55bdac739070 2023-07-14T16:21:24.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f18d1314b80 10 client.26695 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:24.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f18d1314b80 10 client.26695 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:21:24.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f18d1314b80 20 client.26695 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:24.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f18d1314b80 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bdac74e150 0x55bdac750560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f18d1314b80 4 client.26695 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:24.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.915+0000 7f18d1314b80 1 -- 172.21.15.62:0/1217741190 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdac682eb0 con 0x55bdac74e150 2023-07-14T16:21:24.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.916+0000 7f18b67fc700 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bdac74e150 0x55bdac750560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.916+0000 7f1896ffd700 10 client.26695 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:21:24.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.916+0000 7f18b67fc700 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bdac74e150 0x55bdac750560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23ede3ba1e4ca9eb server_cookie=8f5d5204b1f46166 in_seq=0 out_seq=0 2023-07-14T16:21:24.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f1896ffd700 1 -- 172.21.15.62:0/1217741190 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bdac682eb0 con 0x55bdac74e150 2023-07-14T16:21:24.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f1896ffd700 10 client.26695 handle_command_reply: tid=0 2023-07-14T16:21:24.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 client.26695 shutdown 2023-07-14T16:21:24.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 -- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 msgr2=0x55bdac7428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7428c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18a8000c00 tx=0x7f18a8004310 comp rx=0 tx=0).stop 2023-07-14T16:21:24.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 -- 172.21.15.62:0/1217741190 shutdown_connections 2023-07-14T16:21:24.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bdac74e150 0x55bdac750560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739e30 0x55bdac749900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac738720 0x55bdac742380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f1896ffd700 0 client.26695 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:21:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 --2- 172.21.15.62:0/1217741190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739070 0x55bdac7428c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 -- 172.21.15.62:0/1217741190 >> 172.21.15.62:0/1217741190 conn(0x55bdac72dff0 msgr2=0x55bdac732b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.917+0000 7f18d1314b80 1 -- 172.21.15.62:0/1217741190 shutdown_connections 2023-07-14T16:21:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.918+0000 7f18d1314b80 1 -- 172.21.15.62:0/1217741190 wait complete. 2023-07-14T16:21:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.918+0000 7f18d1314b80 20 client.26695 trim_cache size 0 max 16384 2023-07-14T16:21:24.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.931+0000 7f18d1314b80 1 Processor -- start 2023-07-14T16:21:24.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.931+0000 7f18d1314b80 1 -- start start 2023-07-14T16:21:24.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac738720 0x55bdac741010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739070 0x55bdac7498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739e30 0x55bdac742380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18d1314b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac741ef0 con 0x55bdac738720 2023-07-14T16:21:24.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18d1314b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac739700 con 0x55bdac739070 2023-07-14T16:21:24.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18d1314b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac734300 con 0x55bdac739e30 2023-07-14T16:21:24.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac738720 0x55bdac741010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac738720 0x55bdac741010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33916/0 (socket says 172.21.15.62:33916) 2023-07-14T16:21:24.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b67fc700 1 -- 172.21.15.62:0/3465551118 learned_addr learned my addr 172.21.15.62:0/3465551118 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/3465551118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739070 0x55bdac7498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b67fc700 1 -- 172.21.15.62:0/3465551118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739e30 msgr2=0x55bdac742380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b67fc700 1 --2- 172.21.15.62:0/3465551118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739e30 0x55bdac742380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b67fc700 1 -- 172.21.15.62:0/3465551118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739070 msgr2=0x55bdac7498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b67fc700 1 --2- 172.21.15.62:0/3465551118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739070 0x55bdac7498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.932+0000 7f18b67fc700 1 -- 172.21.15.62:0/3465551118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdac4de250 con 0x55bdac738720 2023-07-14T16:21:24.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.933+0000 7f18b67fc700 1 --2- 172.21.15.62:0/3465551118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac738720 0x55bdac741010 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0099f0 tx=0x7f18ac009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8071e7356a6b1d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.933+0000 7f189e7fc700 1 -- 172.21.15.62:0/3465551118 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0039d0 con 0x55bdac738720 2023-07-14T16:21:24.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.933+0000 7f189e7fc700 1 -- 172.21.15.62:0/3465551118 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18ac003b30 con 0x55bdac738720 2023-07-14T16:21:24.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.933+0000 7f189e7fc700 1 -- 172.21.15.62:0/3465551118 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac02f480 con 0x55bdac738720 2023-07-14T16:21:24.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 -- 172.21.15.62:0/3465551118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac738720 msgr2=0x55bdac741010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3465551118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac738720 0x55bdac741010 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0099f0 tx=0x7f18ac009f40 comp rx=0 tx=0).stop 2023-07-14T16:21:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 -- 172.21.15.62:0/3465551118 shutdown_connections 2023-07-14T16:21:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3465551118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac739e30 0x55bdac742380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3465551118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac738720 0x55bdac741010 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3465551118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739070 0x55bdac7498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 -- 172.21.15.62:0/3465551118 >> 172.21.15.62:0/3465551118 conn(0x55bdac72dff0 msgr2=0x55bdac740260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 -- 172.21.15.62:0/3465551118 shutdown_connections 2023-07-14T16:21:24.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 -- 172.21.15.62:0/3465551118 wait complete. 2023-07-14T16:21:24.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.934+0000 7f18d1314b80 1 -- start start 2023-07-14T16:21:24.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18d1314b80 1 Processor -- start 2023-07-14T16:21:24.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac738720 0x55bdac749100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739070 0x55bdac749640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18d1314b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739e30 0x55bdac745ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18d1314b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac746170 con 0x55bdac739070 2023-07-14T16:21:24.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18d1314b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac7462b0 con 0x55bdac739e30 2023-07-14T16:21:24.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18d1314b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdac746470 con 0x55bdac738720 2023-07-14T16:21:24.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18d1314b80 10 client.0 _ll_register_callbacks cb 0x7f18a8003a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:24.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18b6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739070 0x55bdac749640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18b6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739070 0x55bdac749640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33918/0 (socket says 172.21.15.62:33918) 2023-07-14T16:21:24.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18b6ffd700 1 -- 172.21.15.62:0/3888041392 learned_addr learned my addr 172.21.15.62:0/3888041392 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:24.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.935+0000 7f18b67fc700 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac738720 0x55bdac749100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18b67fc700 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739e30 0x55bdac745ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18b6ffd700 1 -- 172.21.15.62:0/3888041392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac738720 msgr2=0x55bdac749100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac738720 0x55bdac749100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18b6ffd700 1 -- 172.21.15.62:0/3888041392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739e30 msgr2=0x55bdac745ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:24.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739e30 0x55bdac745ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18b6ffd700 1 -- 172.21.15.62:0/3888041392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdac4b7de0 con 0x55bdac739070 2023-07-14T16:21:24.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18b67fc700 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739e30 0x55bdac745ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:21:24.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18b6ffd700 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739070 0x55bdac749640 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f18a8029040 tx=0x7f18a80316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bb4508a6795ed3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:24.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f1896ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:24.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f1896ffd700 1 -- 172.21.15.62:0/3888041392 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a801d040 con 0x55bdac739070 2023-07-14T16:21:24.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f18d1314b80 1 -- 172.21.15.62:0/3888041392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18ac003680 con 0x55bdac739070 2023-07-14T16:21:24.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.936+0000 7f1896ffd700 1 -- 172.21.15.62:0/3888041392 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18a8000e40 con 0x55bdac739070 2023-07-14T16:21:24.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f1896ffd700 1 -- 172.21.15.62:0/3888041392 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a8009290 con 0x55bdac739070 2023-07-14T16:21:24.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f18d1314b80 10 client.17106 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:24.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f18d1314b80 1 -- 172.21.15.62:0/3888041392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bdac4b62b0 con 0x55bdac739070 2023-07-14T16:21:24.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f1896ffd700 1 -- 172.21.15.62:0/3888041392 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18ac003680 con 0x55bdac739070 2023-07-14T16:21:24.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f1896ffd700 1 -- 172.21.15.62:0/3888041392 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f18a800bb60 con 0x55bdac739070 2023-07-14T16:21:24.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f18d1314b80 10 client.17106 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:24.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f18d1314b80 10 client.17106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:21:24.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f18d1314b80 20 client.17106 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:24.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bdac7a30a0 0x55bdac7a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:24.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f18d1314b80 4 client.17106 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:21:24.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.937+0000 7f18d1314b80 1 -- 172.21.15.62:0/3888041392 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bdac645050 con 0x55bdac7a30a0 2023-07-14T16:21:24.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.938+0000 7f18b67fc700 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bdac7a30a0 0x55bdac7a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:24.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.938+0000 7f1896ffd700 10 client.17106 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:21:24.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.938+0000 7f18b67fc700 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bdac7a30a0 0x55bdac7a54b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8fb06d04d0f22a5 server_cookie=9439f5248134edb in_seq=0 out_seq=0 2023-07-14T16:21:24.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f1896ffd700 1 -- 172.21.15.62:0/3888041392 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bdac645050 con 0x55bdac7a30a0 2023-07-14T16:21:24.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f1896ffd700 10 client.17106 handle_command_reply: tid=0 2023-07-14T16:21:24.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 client.17106 shutdown 2023-07-14T16:21:24.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 -- 172.21.15.62:0/3888041392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739070 msgr2=0x55bdac749640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739070 0x55bdac749640 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f18a8029040 tx=0x7f18a80316b0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 -- 172.21.15.62:0/3888041392 shutdown_connections 2023-07-14T16:21:24.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bdac7a30a0 0x55bdac7a54b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bdac739e30 0x55bdac745ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bdac738720 0x55bdac749100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 --2- 172.21.15.62:0/3888041392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdac739070 0x55bdac749640 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f1896ffd700 0 client.17106 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:21:24.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 -- 172.21.15.62:0/3888041392 >> 172.21.15.62:0/3888041392 conn(0x55bdac72dff0 msgr2=0x55bdac741010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 -- 172.21.15.62:0/3888041392 shutdown_connections 2023-07-14T16:21:24.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 1 -- 172.21.15.62:0/3888041392 wait complete. 2023-07-14T16:21:24.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.939+0000 7f18d1314b80 20 client.17106 trim_cache size 0 max 16384 2023-07-14T16:21:24.928 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:21:24.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 -- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18a00a1270 msgr2=0x7f18a00a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18a00a1270 0x7f18a00a3720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f18b0007fd0 tx=0x7f18b000b040 comp rx=0 tx=0).stop 2023-07-14T16:21:24.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 -- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81056f0 msgr2=0x7f18b80fe9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:24.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81056f0 0x7f18b80fe9a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f18b0007ee0 tx=0x7f18b000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 -- 172.21.15.62:0/2466348943 shutdown_connections 2023-07-14T16:21:24.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b81056f0 0x7f18b80fe9a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18a00a1270 0x7f18a00a3720 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18b81004b0 0x7f18b80ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 --2- 172.21.15.62:0/2466348943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18b8102dd0 0x7f18b80fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:24.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 -- 172.21.15.62:0/2466348943 >> 172.21.15.62:0/2466348943 conn(0x7f18b80f9ed0 msgr2=0x7f18b8101110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:24.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 -- 172.21.15.62:0/2466348943 shutdown_connections 2023-07-14T16:21:24.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:24.940+0000 7f18b7fff700 1 -- 172.21.15.62:0/2466348943 wait complete. 2023-07-14T16:21:24.934 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:24 smithi062 ceph-mon[95543]: pgmap v530: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:24.934 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:24 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1179305616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:21:24.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:21:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:24 smithi164 ceph-mon[97427]: pgmap v530: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:24 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1179305616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:21:25.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:24 smithi081 ceph-mon[127795]: pgmap v530: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:25.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:24 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1179305616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:21:25.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.199+0000 7fcdec9ff700 1 -- 172.21.15.62:0/2865054858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8102e80 msgr2=0x7fcde8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.199+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/2865054858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8102e80 0x7fcde8105260 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009990 tx=0x7fcddc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.199+0000 7fcdec9ff700 1 -- 172.21.15.62:0/2865054858 shutdown_connections 2023-07-14T16:21:25.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.199+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/2865054858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcde81057a0 0x7fcde8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.199+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/2865054858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde8100560 0x7fcde8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.199+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/2865054858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8102e80 0x7fcde8105260 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.199+0000 7fcdec9ff700 1 -- 172.21.15.62:0/2865054858 >> 172.21.15.62:0/2865054858 conn(0x7fcde80fa000 msgr2=0x7fcde80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:25.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.199+0000 7fcdec9ff700 1 -- 172.21.15.62:0/2865054858 shutdown_connections 2023-07-14T16:21:25.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.200+0000 7fcdec9ff700 1 -- 172.21.15.62:0/2865054858 wait complete. 2023-07-14T16:21:25.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.200+0000 7fcdec9ff700 1 Processor -- start 2023-07-14T16:21:25.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcdec9ff700 1 -- start start 2023-07-14T16:21:25.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcdec9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8100560 0x7fcde80ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcdec9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde8102e80 0x7fcde80fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcdec9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcde81057a0 0x7fcde80fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcdec9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80ff1f0 con 0x7fcde8100560 2023-07-14T16:21:25.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcdec9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80ff360 con 0x7fcde81057a0 2023-07-14T16:21:25.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcdec9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde819f810 con 0x7fcde8102e80 2023-07-14T16:21:25.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcde659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8100560 0x7fcde80ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcde659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8100560 0x7fcde80ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33926/0 (socket says 172.21.15.62:33926) 2023-07-14T16:21:25.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.201+0000 7fcde659c700 1 -- 172.21.15.62:0/3809371062 learned_addr learned my addr 172.21.15.62:0/3809371062 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:25.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde6d9d700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcde81057a0 0x7fcde80fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde8102e80 0x7fcde80fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde659c700 1 -- 172.21.15.62:0/3809371062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde8102e80 msgr2=0x7fcde80fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde659c700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde8102e80 0x7fcde80fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde659c700 1 -- 172.21.15.62:0/3809371062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcde81057a0 msgr2=0x7fcde80fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde659c700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcde81057a0 0x7fcde80fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde659c700 1 -- 172.21.15.62:0/3809371062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0095f0 con 0x7fcde8100560 2023-07-14T16:21:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde8102e80 0x7fcde80fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.202+0000 7fcde659c700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8100560 0x7fcde80ffed0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800e9a0 tx=0x7fcdd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:25.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.203+0000 7fcdd77fe700 1 -- 172.21.15.62:0/3809371062 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8009790 con 0x7fcde8100560 2023-07-14T16:21:25.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.203+0000 7fcdd77fe700 1 -- 172.21.15.62:0/3809371062 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdd8010430 con 0x7fcde8100560 2023-07-14T16:21:25.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.203+0000 7fcdd77fe700 1 -- 172.21.15.62:0/3809371062 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800cea0 con 0x7fcde8100560 2023-07-14T16:21:25.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.203+0000 7fcdec9ff700 1 -- 172.21.15.62:0/3809371062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde819faf0 con 0x7fcde8100560 2023-07-14T16:21:25.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.203+0000 7fcdec9ff700 1 -- 172.21.15.62:0/3809371062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde81a0000 con 0x7fcde8100560 2023-07-14T16:21:25.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.207+0000 7fcdd77fe700 1 -- 172.21.15.62:0/3809371062 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcdd80105a0 con 0x7fcde8100560 2023-07-14T16:21:25.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.208+0000 7fcdd77fe700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcdd00a11d0 0x7fcdd00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.209+0000 7fcdd77fe700 1 -- 172.21.15.62:0/3809371062 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdd8014070 con 0x7fcde8100560 2023-07-14T16:21:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.209+0000 7fce00298b80 1 Processor -- start 2023-07-14T16:21:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.209+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcdd00a11d0 0x7fcdd00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.209+0000 7fce00298b80 1 -- start start 2023-07-14T16:21:25.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.209+0000 7fce00298b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655076fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcdd00a11d0 0x7fcdd00a3680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006010 tx=0x7fcddc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:25.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fce00298b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655076fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fce00298b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x5655076fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fce00298b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x565507601cb0 con 0x5655076fce20 2023-07-14T16:21:25.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fce00298b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x565507654a40 con 0x5655076fdc70 2023-07-14T16:21:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fce00298b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550764ded0 con 0x5655076fc4d0 2023-07-14T16:21:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fcde5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655076fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fcde5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655076fd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33940/0 (socket says 172.21.15.62:33940) 2023-07-14T16:21:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fcde5d9b700 1 -- 172.21.15.62:0/1388941632 learned_addr learned my addr 172.21.15.62:0/1388941632 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fcde659c700 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655076fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.210+0000 7fcde6d9d700 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x5655076fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcde659c700 1 -- 172.21.15.62:0/1388941632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 msgr2=0x5655076fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcde659c700 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x5655076fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcde659c700 1 -- 172.21.15.62:0/1388941632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 msgr2=0x5655076fd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcde659c700 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655076fd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcde659c700 1 -- 172.21.15.62:0/1388941632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56550747bde0 con 0x5655076fc4d0 2023-07-14T16:21:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcde6d9d700 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x5655076fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655076fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcde659c700 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655076fc8e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800e970 tx=0x7fcdd8099860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe537dd23d88ed5c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcdd4ff9700 1 -- 172.21.15.62:0/1388941632 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8099e80 con 0x5655076fc4d0 2023-07-14T16:21:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcdd4ff9700 1 -- 172.21.15.62:0/1388941632 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdd809deb0 con 0x5655076fc4d0 2023-07-14T16:21:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.211+0000 7fcdd4ff9700 1 -- 172.21.15.62:0/1388941632 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80a20a0 con 0x5655076fc4d0 2023-07-14T16:21:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 -- 172.21.15.62:0/1388941632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 msgr2=0x5655076fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655076fc8e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800e970 tx=0x7fcdd8099860 comp rx=0 tx=0).stop 2023-07-14T16:21:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 -- 172.21.15.62:0/1388941632 shutdown_connections 2023-07-14T16:21:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x5655076fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655076fc8e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 --2- 172.21.15.62:0/1388941632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655076fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 -- 172.21.15.62:0/1388941632 >> 172.21.15.62:0/1388941632 conn(0x5655076f1da0 msgr2=0x5655076f71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 -- 172.21.15.62:0/1388941632 shutdown_connections 2023-07-14T16:21:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.212+0000 7fce00298b80 1 -- 172.21.15.62:0/1388941632 wait complete. 2023-07-14T16:21:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.213+0000 7fce00298b80 1 -- start start 2023-07-14T16:21:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.213+0000 7fce00298b80 1 Processor -- start 2023-07-14T16:21:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.213+0000 7fce00298b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x565507706170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fce00298b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655077066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fce00298b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x56550770d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fce00298b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655075edac0 con 0x5655076fce20 2023-07-14T16:21:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fce00298b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655075ed7c0 con 0x5655076fdc70 2023-07-14T16:21:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fce00298b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655074d1920 con 0x5655076fc4d0 2023-07-14T16:21:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fce00298b80 10 client.0 _ll_register_callbacks cb 0x5655075eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fcde5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655077066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fcde5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655077066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33952/0 (socket says 172.21.15.62:33952) 2023-07-14T16:21:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fcde5d9b700 1 -- 172.21.15.62:0/745069806 learned_addr learned my addr 172.21.15.62:0/745069806 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fcde659c700 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x565507706170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fcde6d9d700 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x56550770d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.214+0000 7fcde5d9b700 1 -- 172.21.15.62:0/745069806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 msgr2=0x565507706170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x565507706170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fcde5d9b700 1 -- 172.21.15.62:0/745069806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 msgr2=0x56550770d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x56550770d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fcde5d9b700 1 -- 172.21.15.62:0/745069806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde81087f0 con 0x5655076fce20 2023-07-14T16:21:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655077066b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0098e0 tx=0x7fcddc01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828bb62d3a9e0175 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fcdcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fcdcdffb700 1 -- 172.21.15.62:0/745069806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc02b050 con 0x5655076fce20 2023-07-14T16:21:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fcdcdffb700 1 -- 172.21.15.62:0/745069806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcddc00fd50 con 0x5655076fce20 2023-07-14T16:21:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.215+0000 7fce00298b80 1 -- 172.21.15.62:0/745069806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5655075a74f0 con 0x5655076fce20 2023-07-14T16:21:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fcdcdffb700 1 -- 172.21.15.62:0/745069806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcddc03c480 con 0x5655076fce20 2023-07-14T16:21:25.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fce00298b80 10 client.17118 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:25.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fce00298b80 1 -- 172.21.15.62:0/745069806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fcde0003b50 con 0x5655076fce20 2023-07-14T16:21:25.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fcdcdffb700 1 -- 172.21.15.62:0/745069806 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5655075a74f0 con 0x5655076fce20 2023-07-14T16:21:25.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fcdcdffb700 1 -- 172.21.15.62:0/745069806 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcddc024cf0 con 0x5655076fce20 2023-07-14T16:21:25.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fce00298b80 10 client.17118 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:25.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fce00298b80 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:25.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fce00298b80 20 client.17118 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:25.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fce00298b80 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x565507711fc0 0x5655077143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fce00298b80 4 client.17118 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:25.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.216+0000 7fce00298b80 1 -- 172.21.15.62:0/745069806 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565507646eb0 con 0x565507711fc0 2023-07-14T16:21:25.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.217+0000 7fcde659c700 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x565507711fc0 0x5655077143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.217+0000 7fcde659c700 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x565507711fc0 0x5655077143d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552422e29a4d5ae7 server_cookie=27f79773c0128e2e in_seq=0 out_seq=0 2023-07-14T16:21:25.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.217+0000 7fcdcdffb700 10 client.17118 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:25.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fcdcdffb700 1 -- 172.21.15.62:0/745069806 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565507646eb0 con 0x565507711fc0 2023-07-14T16:21:25.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fcdcdffb700 10 client.17118 handle_command_reply: tid=0 2023-07-14T16:21:25.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fce00298b80 1 client.17118 shutdown 2023-07-14T16:21:25.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fce00298b80 1 -- 172.21.15.62:0/745069806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 msgr2=0x5655077066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fce00298b80 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655077066b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0098e0 tx=0x7fcddc01a050 comp rx=0 tx=0).stop 2023-07-14T16:21:25.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fce00298b80 1 -- 172.21.15.62:0/745069806 shutdown_connections 2023-07-14T16:21:25.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fce00298b80 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x565507711fc0 0x5655077143d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fce00298b80 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fdc70 0x56550770d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fce00298b80 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x565507706170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fcdcdffb700 0 client.17118 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:25.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.218+0000 7fce00298b80 1 --2- 172.21.15.62:0/745069806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fce20 0x5655077066b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.219+0000 7fce00298b80 1 -- 172.21.15.62:0/745069806 >> 172.21.15.62:0/745069806 conn(0x5655076f1da0 msgr2=0x5655076f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:25.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.219+0000 7fce00298b80 1 -- 172.21.15.62:0/745069806 shutdown_connections 2023-07-14T16:21:25.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.219+0000 7fce00298b80 1 -- 172.21.15.62:0/745069806 wait complete. 2023-07-14T16:21:25.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.219+0000 7fce00298b80 20 client.17118 trim_cache size 0 max 16384 2023-07-14T16:21:25.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.234+0000 7fce00298b80 1 Processor -- start 2023-07-14T16:21:25.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fce00298b80 1 -- start start 2023-07-14T16:21:25.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fce00298b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655077143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fce00298b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fce00298b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fdc70 0x565507704020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fce00298b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655076f80f0 con 0x5655076fdc70 2023-07-14T16:21:25.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fce00298b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655076f82a0 con 0x5655076fce20 2023-07-14T16:21:25.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fce00298b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x565507705620 con 0x5655076fc4d0 2023-07-14T16:21:25.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fcde6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fdc70 0x565507704020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fcde6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fdc70 0x565507704020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33962/0 (socket says 172.21.15.62:33962) 2023-07-14T16:21:25.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.235+0000 7fcde6d9d700 1 -- 172.21.15.62:0/615874628 learned_addr learned my addr 172.21.15.62:0/615874628 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:25.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcde659c700 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655077143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcde6d9d700 1 -- 172.21.15.62:0/615874628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 msgr2=0x5655077143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:25.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcde6d9d700 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655077143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcde6d9d700 1 -- 172.21.15.62:0/615874628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 msgr2=0x56550770d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:25.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcde6d9d700 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcde6d9d700 1 -- 172.21.15.62:0/615874628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde81a0000 con 0x5655076fdc70 2023-07-14T16:21:25.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcde6d9d700 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fdc70 0x565507704020 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009d20 tx=0x7fcde0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc4dfcf8d7094c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:25.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.236+0000 7fcdd57fa700 1 -- 172.21.15.62:0/615874628 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0004db0 con 0x5655076fdc70 2023-07-14T16:21:25.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fcdd57fa700 1 -- 172.21.15.62:0/615874628 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcde000f040 con 0x5655076fdc70 2023-07-14T16:21:25.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fcdd57fa700 1 -- 172.21.15.62:0/615874628 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde001fd50 con 0x5655076fdc70 2023-07-14T16:21:25.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fce00298b80 1 -- 172.21.15.62:0/615874628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fdc70 msgr2=0x565507704020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fce00298b80 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fdc70 0x565507704020 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009d20 tx=0x7fcde0009d50 comp rx=0 tx=0).stop 2023-07-14T16:21:25.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fce00298b80 1 -- 172.21.15.62:0/615874628 shutdown_connections 2023-07-14T16:21:25.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fce00298b80 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fdc70 0x565507704020 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fce00298b80 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fc4d0 0x5655077143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fce00298b80 1 --2- 172.21.15.62:0/615874628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fce00298b80 1 -- 172.21.15.62:0/615874628 >> 172.21.15.62:0/615874628 conn(0x5655076f1da0 msgr2=0x565507704e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:25.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.237+0000 7fce00298b80 1 -- 172.21.15.62:0/615874628 shutdown_connections 2023-07-14T16:21:25.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.238+0000 7fce00298b80 1 -- 172.21.15.62:0/615874628 wait complete. 2023-07-14T16:21:25.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.238+0000 7fce00298b80 1 -- start start 2023-07-14T16:21:25.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.238+0000 7fce00298b80 1 Processor -- start 2023-07-14T16:21:25.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fce00298b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fc4d0 0x56550770cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fce00298b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fce00298b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fdc70 0x5655077099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fce00298b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x565507709f70 con 0x5655076fc4d0 2023-07-14T16:21:25.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fce00298b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550770a0e0 con 0x5655076fce20 2023-07-14T16:21:25.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fce00298b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550770a2a0 con 0x5655076fdc70 2023-07-14T16:21:25.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fce00298b80 10 client.0 _ll_register_callbacks cb 0x7fcddc00ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:25.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fc4d0 0x56550770cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fc4d0 0x56550770cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33972/0 (socket says 172.21.15.62:33972) 2023-07-14T16:21:25.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde659c700 1 -- 172.21.15.62:0/2503177838 learned_addr learned my addr 172.21.15.62:0/2503177838 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:25.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde659c700 1 -- 172.21.15.62:0/2503177838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fdc70 msgr2=0x5655077099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:25.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde659c700 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fdc70 0x5655077099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde659c700 1 -- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 msgr2=0x56550770d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:25.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde659c700 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde659c700 1 -- 172.21.15.62:0/2503177838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde00099d0 con 0x5655076fc4d0 2023-07-14T16:21:25.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.239+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:21:25.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fcde659c700 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fc4d0 0x56550770cf00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcdd809a4e0 tx=0x7fcdd8010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6989ec3015f0991 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:25.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fcdcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:25.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fcdcdffb700 1 -- 172.21.15.62:0/2503177838 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8098220 con 0x5655076fc4d0 2023-07-14T16:21:25.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fcdcdffb700 1 -- 172.21.15.62:0/2503177838 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcdd8099410 con 0x5655076fc4d0 2023-07-14T16:21:25.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fce00298b80 1 -- 172.21.15.62:0/2503177838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdd809deb0 con 0x5655076fc4d0 2023-07-14T16:21:25.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fcdcdffb700 1 -- 172.21.15.62:0/2503177838 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdd80995d0 con 0x5655076fc4d0 2023-07-14T16:21:25.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fce00298b80 10 client.17124 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:25.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fce00298b80 1 -- 172.21.15.62:0/2503177838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x565507464c60 con 0x5655076fc4d0 2023-07-14T16:21:25.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.240+0000 7fcdcdffb700 1 -- 172.21.15.62:0/2503177838 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdd809deb0 con 0x5655076fc4d0 2023-07-14T16:21:25.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.241+0000 7fcdcdffb700 1 -- 172.21.15.62:0/2503177838 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcdd80adb80 con 0x5655076fc4d0 2023-07-14T16:21:25.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.241+0000 7fce00298b80 10 client.17124 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:25.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.241+0000 7fce00298b80 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:25.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.241+0000 7fce00298b80 20 client.17124 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:25.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.241+0000 7fce00298b80 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x565507767070 0x565507769480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:25.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.241+0000 7fce00298b80 4 client.17124 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:21:25.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.241+0000 7fce00298b80 1 -- 172.21.15.62:0/2503177838 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565507609050 con 0x565507767070 2023-07-14T16:21:25.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.241+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x565507767070 0x565507769480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:25.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fcde5d9b700 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x565507767070 0x565507769480 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e604306621fded8f server_cookie=8d1c3346b866422a in_seq=0 out_seq=0 2023-07-14T16:21:25.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fcdcdffb700 10 client.17124 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:25.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fcdcdffb700 1 -- 172.21.15.62:0/2503177838 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565507609050 con 0x565507767070 2023-07-14T16:21:25.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fcdcdffb700 10 client.17124 handle_command_reply: tid=0 2023-07-14T16:21:25.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 client.17124 shutdown 2023-07-14T16:21:25.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 -- 172.21.15.62:0/2503177838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fc4d0 msgr2=0x56550770cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fc4d0 0x56550770cf00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcdd809a4e0 tx=0x7fcdd8010880 comp rx=0 tx=0).stop 2023-07-14T16:21:25.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 -- 172.21.15.62:0/2503177838 shutdown_connections 2023-07-14T16:21:25.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5655076fdc70 0x5655077099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5655076fc4d0 0x56550770cf00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x565507767070 0x565507769480 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 --2- 172.21.15.62:0/2503177838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5655076fce20 0x56550770d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fcdcdffb700 0 client.17124 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:25.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.242+0000 7fce00298b80 1 -- 172.21.15.62:0/2503177838 >> 172.21.15.62:0/2503177838 conn(0x5655076f1da0 msgr2=0x565507704020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:25.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.243+0000 7fce00298b80 1 -- 172.21.15.62:0/2503177838 shutdown_connections 2023-07-14T16:21:25.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.243+0000 7fce00298b80 1 -- 172.21.15.62:0/2503177838 wait complete. 2023-07-14T16:21:25.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.243+0000 7fce00298b80 20 client.17124 trim_cache size 0 max 16384 2023-07-14T16:21:25.230 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:21:25.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.243+0000 7fcdec9ff700 1 -- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcdd00a11d0 msgr2=0x7fcdd00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.243+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcdd00a11d0 0x7fcdd00a3680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006010 tx=0x7fcddc00b260 comp rx=0 tx=0).stop 2023-07-14T16:21:25.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.243+0000 7fcdec9ff700 1 -- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8100560 msgr2=0x7fcde80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:25.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.243+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8100560 0x7fcde80ffed0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800e9a0 tx=0x7fcdd800ecb0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.244+0000 7fcdec9ff700 1 -- 172.21.15.62:0/3809371062 shutdown_connections 2023-07-14T16:21:25.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.244+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcdd00a11d0 0x7fcdd00a3680 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.244+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcde81057a0 0x7fcde80fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.244+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde8100560 0x7fcde80ffed0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.244+0000 7fcdec9ff700 1 --2- 172.21.15.62:0/3809371062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcde8102e80 0x7fcde80fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:25.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.244+0000 7fcdec9ff700 1 -- 172.21.15.62:0/3809371062 >> 172.21.15.62:0/3809371062 conn(0x7fcde80fa000 msgr2=0x7fcde81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:25.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.244+0000 7fcdec9ff700 1 -- 172.21.15.62:0/3809371062 shutdown_connections 2023-07-14T16:21:25.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:25.244+0000 7fcdec9ff700 1 -- 172.21.15.62:0/3809371062 wait complete. 2023-07-14T16:21:26.240 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:21:26.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:21:26.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 -- 172.21.15.62:0/2205841773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8102f00 msgr2=0x7f17d8103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2205841773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8102f00 0x7f17d8103350 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009990 tx=0x7f17cc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 -- 172.21.15.62:0/2205841773 shutdown_connections 2023-07-14T16:21:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2205841773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d8103890 0x7f17d810a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2205841773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d8101d10 0x7f17d8102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2205841773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8102f00 0x7f17d8103350 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 -- 172.21.15.62:0/2205841773 >> 172.21.15.62:0/2205841773 conn(0x7f17d80fd350 msgr2=0x7f17d80ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 -- 172.21.15.62:0/2205841773 shutdown_connections 2023-07-14T16:21:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.481+0000 7f17de5ff700 1 -- 172.21.15.62:0/2205841773 wait complete. 2023-07-14T16:21:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.482+0000 7f17de5ff700 1 Processor -- start 2023-07-14T16:21:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17de5ff700 1 -- start start 2023-07-14T16:21:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17de5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8101d10 0x7f17d81979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17de5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d8102f00 0x7f17d8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17de5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d8103890 0x7f17d819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17de5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d819f540 con 0x7f17d8101d10 2023-07-14T16:21:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17de5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d819f680 con 0x7f17d8102f00 2023-07-14T16:21:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17de5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d819f810 con 0x7f17d8103890 2023-07-14T16:21:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17d7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8101d10 0x7f17d81979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17d7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8101d10 0x7f17d81979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34000/0 (socket says 172.21.15.62:34000) 2023-07-14T16:21:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17d7fff700 1 -- 172.21.15.62:0/2287706631 learned_addr learned my addr 172.21.15.62:0/2287706631 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.483+0000 7f17d77fe700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d8102f00 0x7f17d8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d8103890 0x7f17d819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d7fff700 1 -- 172.21.15.62:0/2287706631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d8103890 msgr2=0x7f17d819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d7fff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d8103890 0x7f17d819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d7fff700 1 -- 172.21.15.62:0/2287706631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d8102f00 msgr2=0x7f17d8197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d7fff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d8102f00 0x7f17d8197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d7fff700 1 -- 172.21.15.62:0/2287706631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0095f0 con 0x7f17d8101d10 2023-07-14T16:21:26.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d77fe700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d8102f00 0x7f17d8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:26.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d7fff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8101d10 0x7f17d81979c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f17c800e970 tx=0x7f17c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d57fa700 1 -- 172.21.15.62:0/2287706631 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8009790 con 0x7f17d8101d10 2023-07-14T16:21:26.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d57fa700 1 -- 172.21.15.62:0/2287706631 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17c800c7e0 con 0x7f17d8101d10 2023-07-14T16:21:26.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.484+0000 7f17d57fa700 1 -- 172.21.15.62:0/2287706631 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8003bd0 con 0x7f17d8101d10 2023-07-14T16:21:26.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.485+0000 7f17de5ff700 1 -- 172.21.15.62:0/2287706631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d819faf0 con 0x7f17d8101d10 2023-07-14T16:21:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.485+0000 7f17de5ff700 1 -- 172.21.15.62:0/2287706631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d819ffb0 con 0x7f17d8101d10 2023-07-14T16:21:26.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.489+0000 7f17d57fa700 1 -- 172.21.15.62:0/2287706631 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f17c800c950 con 0x7f17d8101d10 2023-07-14T16:21:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.489+0000 7f17f1ee9b80 1 Processor -- start 2023-07-14T16:21:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17f1ee9b80 1 -- start start 2023-07-14T16:21:26.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4a7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 0x55db6c4a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c3acd30 con 0x55db6c4a7ea0 2023-07-14T16:21:26.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c3ffac0 con 0x55db6c4a8cf0 2023-07-14T16:21:26.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c3f8f50 con 0x55db6c4a7550 2023-07-14T16:21:26.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17d7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17d7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4a8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34008/0 (socket says 172.21.15.62:34008) 2023-07-14T16:21:26.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17d77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 0x55db6c4a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17d7fff700 1 -- 172.21.15.62:0/1226197591 learned_addr learned my addr 172.21.15.62:0/1226197591 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.490+0000 7f17dcb9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17dcb9c700 1 -- 172.21.15.62:0/1226197591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 msgr2=0x55db6c4a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1226197591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 0x55db6c4a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17dcb9c700 1 -- 172.21.15.62:0/1226197591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 msgr2=0x55db6c4a8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1226197591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4a8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17dcb9c700 1 -- 172.21.15.62:0/1226197591 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db6c226e60 con 0x55db6c4a7550 2023-07-14T16:21:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17d57fa700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c00a1150 0x7f17c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17d57fa700 1 -- 172.21.15.62:0/2287706631 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17c8014070 con 0x7f17d8101d10 2023-07-14T16:21:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1226197591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4a7960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f17d0003f20 tx=0x7f17d0009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bd8ebbd8d5888ae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.491+0000 7f17d7fff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c00a1150 0x7f17c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.492+0000 7f17be7fc700 1 -- 172.21.15.62:0/1226197591 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0019070 con 0x55db6c4a7550 2023-07-14T16:21:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.492+0000 7f17be7fc700 1 -- 172.21.15.62:0/1226197591 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17d000f040 con 0x55db6c4a7550 2023-07-14T16:21:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.492+0000 7f17be7fc700 1 -- 172.21.15.62:0/1226197591 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0007930 con 0x55db6c4a7550 2023-07-14T16:21:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.492+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1226197591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 msgr2=0x55db6c4a7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.492+0000 7f17d7fff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c00a1150 0x7f17c00a3600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17c80a1dc0 tx=0x7f17c80d2550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.492+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1226197591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4a7960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f17d0003f20 tx=0x7f17d0009ba0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.493+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1226197591 shutdown_connections 2023-07-14T16:21:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.493+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1226197591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 0x55db6c4a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.493+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1226197591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4a7960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.493+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1226197591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.493+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1226197591 >> 172.21.15.62:0/1226197591 conn(0x55db6c49ce20 msgr2=0x55db6c4a2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.493+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1226197591 shutdown_connections 2023-07-14T16:21:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.493+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1226197591 wait complete. 2023-07-14T16:21:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.493+0000 7f17f1ee9b80 1 -- start start 2023-07-14T16:21:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17f1ee9b80 1 Processor -- start 2023-07-14T16:21:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 0x55db6c4b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c398b40 con 0x55db6c4a7ea0 2023-07-14T16:21:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c398840 con 0x55db6c4a7550 2023-07-14T16:21:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c27c9a0 con 0x55db6c4a8cf0 2023-07-14T16:21:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17f1ee9b80 10 client.0 _ll_register_callbacks cb 0x55db6c3967e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34010/0 (socket says 172.21.15.62:34010) 2023-07-14T16:21:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.494+0000 7f17d77fe700 1 -- 172.21.15.62:0/571821972 learned_addr learned my addr 172.21.15.62:0/571821972 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 0x55db6c4b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17d77fe700 1 -- 172.21.15.62:0/571821972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a8cf0 msgr2=0x55db6c4b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17d77fe700 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17d77fe700 1 -- 172.21.15.62:0/571821972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 msgr2=0x55db6c4b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17d77fe700 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 0x55db6c4b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17d77fe700 1 -- 172.21.15.62:0/571821972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d0003b50 con 0x55db6c4a7ea0 2023-07-14T16:21:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17d77fe700 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b1710 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009990 tx=0x7f17cc0050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb2f352c8404cb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.495+0000 7f17b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17b77fe700 1 -- 172.21.15.62:0/571821972 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc01d070 con 0x55db6c4a7ea0 2023-07-14T16:21:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/571821972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db6c352570 con 0x55db6c4a7ea0 2023-07-14T16:21:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17b77fe700 1 -- 172.21.15.62:0/571821972 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17cc00b8b0 con 0x55db6c4a7ea0 2023-07-14T16:21:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17b77fe700 1 -- 172.21.15.62:0/571821972 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17cc00ba70 con 0x55db6c4a7ea0 2023-07-14T16:21:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17f1ee9b80 10 client.17136 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/571821972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55db6c20fce0 con 0x55db6c4a7ea0 2023-07-14T16:21:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17b77fe700 1 -- 172.21.15.62:0/571821972 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55db6c352570 con 0x55db6c4a7ea0 2023-07-14T16:21:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17b77fe700 1 -- 172.21.15.62:0/571821972 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f17cc004ca0 con 0x55db6c4a7ea0 2023-07-14T16:21:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.496+0000 7f17f1ee9b80 10 client.17136 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.497+0000 7f17f1ee9b80 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.497+0000 7f17f1ee9b80 20 client.17136 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.497+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55db6c4bced0 0x55db6c4bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.497+0000 7f17f1ee9b80 4 client.17136 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.497+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/571821972 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db6c3f1f30 con 0x55db6c4bced0 2023-07-14T16:21:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.497+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55db6c4bced0 0x55db6c4bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.498+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55db6c4bced0 0x55db6c4bf2e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75fed5fc032b901f server_cookie=63e2d50d72992d1b in_seq=0 out_seq=0 2023-07-14T16:21:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.498+0000 7f17b77fe700 10 client.17136 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.498+0000 7f17b77fe700 1 -- 172.21.15.62:0/571821972 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55db6c3f1f30 con 0x55db6c4bced0 2023-07-14T16:21:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.498+0000 7f17b77fe700 10 client.17136 handle_command_reply: tid=0 2023-07-14T16:21:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.498+0000 7f17f1ee9b80 1 client.17136 shutdown 2023-07-14T16:21:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/571821972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 msgr2=0x55db6c4b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b1710 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009990 tx=0x7f17cc0050f0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/571821972 shutdown_connections 2023-07-14T16:21:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55db6c4bced0 0x55db6c4bf2e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 0x55db6c4b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/571821972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b1710 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/571821972 >> 172.21.15.62:0/571821972 conn(0x55db6c49ce20 msgr2=0x55db6c4a1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17b77fe700 0 client.17136 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/571821972 shutdown_connections 2023-07-14T16:21:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/571821972 wait complete. 2023-07-14T16:21:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.499+0000 7f17f1ee9b80 20 client.17136 trim_cache size 0 max 16384 2023-07-14T16:21:26.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.521+0000 7f17f1ee9b80 1 Processor -- start 2023-07-14T16:21:26.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.521+0000 7f17f1ee9b80 1 -- start start 2023-07-14T16:21:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 0x55db6c4b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c4a82e0 con 0x55db6c4a8cf0 2023-07-14T16:21:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c4b0d70 con 0x55db6c4a7550 2023-07-14T16:21:26.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c4a3140 con 0x55db6c4a7ea0 2023-07-14T16:21:26.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17dcb9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17dcb9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34026/0 (socket says 172.21.15.62:34026) 2023-07-14T16:21:26.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17dcb9c700 1 -- 172.21.15.62:0/1316907117 learned_addr learned my addr 172.21.15.62:0/1316907117 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17d77fe700 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.522+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 0x55db6c4b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17dcb9c700 1 -- 172.21.15.62:0/1316907117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7ea0 msgr2=0x55db6c4b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17dcb9c700 1 -- 172.21.15.62:0/1316907117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 msgr2=0x55db6c4b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 0x55db6c4b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17dcb9c700 1 -- 172.21.15.62:0/1316907117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db6c4a9390 con 0x55db6c4a8cf0 2023-07-14T16:21:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8710 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009690 tx=0x7f17d0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51ef83f13feb5c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17beffd700 1 -- 172.21.15.62:0/1316907117 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0019040 con 0x55db6c4a8cf0 2023-07-14T16:21:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17beffd700 1 -- 172.21.15.62:0/1316907117 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17d002a030 con 0x55db6c4a8cf0 2023-07-14T16:21:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.523+0000 7f17beffd700 1 -- 172.21.15.62:0/1316907117 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d00156e0 con 0x55db6c4a8cf0 2023-07-14T16:21:26.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1316907117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a8cf0 msgr2=0x55db6c4b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8710 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009690 tx=0x7f17d0000f40 comp rx=0 tx=0).stop 2023-07-14T16:21:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1316907117 shutdown_connections 2023-07-14T16:21:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a8cf0 0x55db6c4b8710 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a7550 0x55db6c4b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1316907117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1316907117 >> 172.21.15.62:0/1316907117 conn(0x55db6c49ce20 msgr2=0x55db6c4b9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1316907117 shutdown_connections 2023-07-14T16:21:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.524+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1316907117 wait complete. 2023-07-14T16:21:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.525+0000 7f17f1ee9b80 1 -- start start 2023-07-14T16:21:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.525+0000 7f17f1ee9b80 1 Processor -- start 2023-07-14T16:21:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.525+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.525+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.525+0000 7f17f1ee9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 0x55db6c510d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.525+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c4b89c0 con 0x55db6c4a7ea0 2023-07-14T16:21:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.525+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c4b8b30 con 0x55db6c4a8cf0 2023-07-14T16:21:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17f1ee9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db6c5112c0 con 0x55db6c4a7550 2023-07-14T16:21:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17f1ee9b80 10 client.0 _ll_register_callbacks cb 0x7f17cc003a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34040/0 (socket says 172.21.15.62:34040) 2023-07-14T16:21:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17d77fe700 1 -- 172.21.15.62:0/1328966246 learned_addr learned my addr 172.21.15.62:0/1328966246 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 0x55db6c510d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17d77fe700 1 -- 172.21.15.62:0/1328966246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 msgr2=0x55db6c4b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17d77fe700 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17d77fe700 1 -- 172.21.15.62:0/1328966246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 msgr2=0x55db6c510d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17d77fe700 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 0x55db6c510d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17d77fe700 1 -- 172.21.15.62:0/1328966246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d002c040 con 0x55db6c4a7ea0 2023-07-14T16:21:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.526+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:21:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17d77fe700 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b59a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009350 tx=0x7f17cc00fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4802575003d37b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17b77fe700 1 -- 172.21.15.62:0/1328966246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc01d040 con 0x55db6c4a7ea0 2023-07-14T16:21:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1328966246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17d000f040 con 0x55db6c4a7ea0 2023-07-14T16:21:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17b77fe700 1 -- 172.21.15.62:0/1328966246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17cc00f650 con 0x55db6c4a7ea0 2023-07-14T16:21:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17b77fe700 1 -- 172.21.15.62:0/1328966246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17cc00e7f0 con 0x55db6c4a7ea0 2023-07-14T16:21:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17f1ee9b80 10 client.17142 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1328966246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55db6c24d2d0 con 0x55db6c4a7ea0 2023-07-14T16:21:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.527+0000 7f17b77fe700 1 -- 172.21.15.62:0/1328966246 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17d000f040 con 0x55db6c4a7ea0 2023-07-14T16:21:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.528+0000 7f17b77fe700 1 -- 172.21.15.62:0/1328966246 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f17cc00bbe0 con 0x55db6c4a7ea0 2023-07-14T16:21:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.528+0000 7f17f1ee9b80 10 client.17142 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.528+0000 7f17f1ee9b80 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.528+0000 7f17f1ee9b80 20 client.17142 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.528+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55db6c512810 0x55db6c514c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.528+0000 7f17f1ee9b80 4 client.17142 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:21:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.528+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1328966246 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55db6c3b40d0 con 0x55db6c512810 2023-07-14T16:21:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.528+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55db6c512810 0x55db6c514c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.529+0000 7f17b77fe700 10 client.17142 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.529+0000 7f17dcb9c700 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55db6c512810 0x55db6c514c20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab6a03dacca68fa server_cookie=6c56cf883d4c4caf in_seq=0 out_seq=0 2023-07-14T16:21:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.530+0000 7f17b77fe700 1 -- 172.21.15.62:0/1328966246 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55db6c3b40d0 con 0x55db6c512810 2023-07-14T16:21:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.530+0000 7f17b77fe700 10 client.17142 handle_command_reply: tid=0 2023-07-14T16:21:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.530+0000 7f17f1ee9b80 1 client.17142 shutdown 2023-07-14T16:21:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1328966246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 msgr2=0x55db6c4b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b59a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009350 tx=0x7f17cc00fb70 comp rx=0 tx=0).stop 2023-07-14T16:21:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1328966246 shutdown_connections 2023-07-14T16:21:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55db6c512810 0x55db6c514c20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55db6c4a8cf0 0x55db6c510d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55db6c4a7550 0x55db6c4b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 --2- 172.21.15.62:0/1328966246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55db6c4a7ea0 0x55db6c4b59a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1328966246 >> 172.21.15.62:0/1328966246 conn(0x55db6c49ce20 msgr2=0x55db6c4b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17b77fe700 0 client.17142 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:26.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1328966246 shutdown_connections 2023-07-14T16:21:26.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 1 -- 172.21.15.62:0/1328966246 wait complete. 2023-07-14T16:21:26.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.531+0000 7f17f1ee9b80 20 client.17142 trim_cache size 0 max 16384 2023-07-14T16:21:26.518 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:21:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:21:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "ff5c95e0-0016-45dc-a69f-bbe06aed3b00", 2023-07-14T16:21:26.519 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:21:26.519 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:21:26.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 -- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c00a1150 msgr2=0x7f17c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c00a1150 0x7f17c00a3600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17c80a1dc0 tx=0x7f17c80d2550 comp rx=0 tx=0).stop 2023-07-14T16:21:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 -- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8101d10 msgr2=0x7f17d81979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8101d10 0x7f17d81979c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f17c800e970 tx=0x7f17c800ec80 comp rx=0 tx=0).stop 2023-07-14T16:21:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 -- 172.21.15.62:0/2287706631 shutdown_connections 2023-07-14T16:21:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d8103890 0x7f17d819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d8101d10 0x7f17d81979c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c00a1150 0x7f17c00a3600 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 --2- 172.21.15.62:0/2287706631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d8102f00 0x7f17d8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 -- 172.21.15.62:0/2287706631 >> 172.21.15.62:0/2287706631 conn(0x7f17d80fd350 msgr2=0x7f17d80ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 -- 172.21.15.62:0/2287706631 shutdown_connections 2023-07-14T16:21:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.532+0000 7f17de5ff700 1 -- 172.21.15.62:0/2287706631 wait complete. 2023-07-14T16:21:26.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:21:26.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 -- 172.21.15.62:0/1341416018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d901056f0 msgr2=0x7f5d90107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/1341416018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d901056f0 0x7f5d90107af0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800b0a0 tx=0x7f5d8800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 -- 172.21.15.62:0/1341416018 shutdown_connections 2023-07-14T16:21:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/1341416018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d901056f0 0x7f5d90107af0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/1341416018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d901004b0 0x7f5d90102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/1341416018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d90102dd0 0x7f5d901051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 -- 172.21.15.62:0/1341416018 >> 172.21.15.62:0/1341416018 conn(0x7f5d900f9ed0 msgr2=0x7f5d900fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 -- 172.21.15.62:0/1341416018 shutdown_connections 2023-07-14T16:21:26.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.784+0000 7f5d94f9d700 1 -- 172.21.15.62:0/1341416018 wait complete. 2023-07-14T16:21:26.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.785+0000 7f5d94f9d700 1 Processor -- start 2023-07-14T16:21:26.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d94f9d700 1 -- start start 2023-07-14T16:21:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d94f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d901004b0 0x7f5d901955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d94f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90102dd0 0x7f5d90195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d94f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d901056f0 0x7f5d9019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d94f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019d170 con 0x7f5d90102dd0 2023-07-14T16:21:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d94f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019d2b0 con 0x7f5d901004b0 2023-07-14T16:21:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d94f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019d440 con 0x7f5d901056f0 2023-07-14T16:21:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90102dd0 0x7f5d90195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90102dd0 0x7f5d90195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34066/0 (socket says 172.21.15.62:34066) 2023-07-14T16:21:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.786+0000 7f5d8effd700 1 -- 172.21.15.62:0/2077170414 learned_addr learned my addr 172.21.15.62:0/2077170414 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d901056f0 0x7f5d9019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d901004b0 0x7f5d901955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8effd700 1 -- 172.21.15.62:0/2077170414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d901056f0 msgr2=0x7f5d9019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8effd700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d901056f0 0x7f5d9019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8effd700 1 -- 172.21.15.62:0/2077170414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d901004b0 msgr2=0x7f5d901955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8effd700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d901004b0 0x7f5d901955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8effd700 1 -- 172.21.15.62:0/2077170414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d88009d00 con 0x7f5d90102dd0 2023-07-14T16:21:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d901056f0 0x7f5d9019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.787+0000 7f5d8effd700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90102dd0 0x7f5d90195b30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400d950 tx=0x7f5d8400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.788+0000 7f5d8cff9700 1 -- 172.21.15.62:0/2077170414 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400f890 con 0x7f5d90102dd0 2023-07-14T16:21:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.788+0000 7f5d8cff9700 1 -- 172.21.15.62:0/2077170414 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d8400fe80 con 0x7f5d90102dd0 2023-07-14T16:21:26.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.788+0000 7f5d94f9d700 1 -- 172.21.15.62:0/2077170414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9019d720 con 0x7f5d90102dd0 2023-07-14T16:21:26.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.788+0000 7f5d8cff9700 1 -- 172.21.15.62:0/2077170414 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400e3b0 con 0x7f5d90102dd0 2023-07-14T16:21:26.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.788+0000 7f5d94f9d700 1 -- 172.21.15.62:0/2077170414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9019dc30 con 0x7f5d90102dd0 2023-07-14T16:21:26.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.792+0000 7f5d8cff9700 1 -- 172.21.15.62:0/2077170414 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5d84010460 con 0x7f5d90102dd0 2023-07-14T16:21:26.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.792+0000 7f5da9ba2b80 1 Processor -- start 2023-07-14T16:21:26.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5da9ba2b80 1 -- start start 2023-07-14T16:21:26.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea370b30 0x557eea370f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea371480 0x557eea371cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea3726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea271cb0 con 0x557eea370b30 2023-07-14T16:21:26.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea2c4a40 con 0x557eea372240 2023-07-14T16:21:26.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea2bded0 con 0x557eea371480 2023-07-14T16:21:26.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5d8ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea370b30 0x557eea370f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5d8ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea370b30 0x557eea370f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34070/0 (socket says 172.21.15.62:34070) 2023-07-14T16:21:26.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.793+0000 7f5d8ffff700 1 -- 172.21.15.62:0/2030955509 learned_addr learned my addr 172.21.15.62:0/2030955509 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.794+0000 7f5d8effd700 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea3726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.794+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea371480 0x557eea371cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.794+0000 7f5d8ffff700 1 -- 172.21.15.62:0/2030955509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea371480 msgr2=0x557eea371cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.794+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea371480 0x557eea371cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.794+0000 7f5d8ffff700 1 -- 172.21.15.62:0/2030955509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 msgr2=0x557eea3726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.794+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea3726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.794+0000 7f5d8ffff700 1 -- 172.21.15.62:0/2030955509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eea0ebde0 con 0x557eea370b30 2023-07-14T16:21:26.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.794+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea370b30 0x557eea370f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007a30 tx=0x7f5d88009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc994a59757b54f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.795+0000 7f5d8cff9700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5d780a11f0 0x7f5d780a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.795+0000 7f5d75ffb700 1 -- 172.21.15.62:0/2030955509 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800e040 con 0x557eea370b30 2023-07-14T16:21:26.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.795+0000 7f5d75ffb700 1 -- 172.21.15.62:0/2030955509 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d880092d0 con 0x557eea370b30 2023-07-14T16:21:26.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.795+0000 7f5d75ffb700 1 -- 172.21.15.62:0/2030955509 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88005610 con 0x557eea370b30 2023-07-14T16:21:26.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.795+0000 7f5d8cff9700 1 -- 172.21.15.62:0/2077170414 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5d840cc9d0 con 0x7f5d90102dd0 2023-07-14T16:21:26.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.795+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5d780a11f0 0x7f5d780a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5d780a11f0 0x7f5d780a36a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5d80007d40 tx=0x7f5d800073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/2030955509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea370b30 msgr2=0x557eea370f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea370b30 0x557eea370f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007a30 tx=0x7f5d88009c30 comp rx=0 tx=0).stop 2023-07-14T16:21:26.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/2030955509 shutdown_connections 2023-07-14T16:21:26.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea3726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea370b30 0x557eea370f40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/2030955509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea371480 0x557eea371cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/2030955509 >> 172.21.15.62:0/2030955509 conn(0x557eea361ff0 msgr2=0x557eea367460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/2030955509 shutdown_connections 2023-07-14T16:21:26.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.796+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/2030955509 wait complete. 2023-07-14T16:21:26.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.797+0000 7f5da9ba2b80 1 -- start start 2023-07-14T16:21:26.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.797+0000 7f5da9ba2b80 1 Processor -- start 2023-07-14T16:21:26.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.797+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea37a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.797+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea37ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.797+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea381ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.797+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea25dac0 con 0x557eea371480 2023-07-14T16:21:26.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.797+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea25d7c0 con 0x557eea372240 2023-07-14T16:21:26.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.797+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea141920 con 0x557eea370b30 2023-07-14T16:21:26.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5da9ba2b80 10 client.0 _ll_register_callbacks cb 0x557eea25b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:26.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea37a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea37ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea37a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39106/0 (socket says 172.21.15.62:39106) 2023-07-14T16:21:26.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8f7fe700 1 -- 172.21.15.62:0/446871536 learned_addr learned my addr 172.21.15.62:0/446871536 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea381ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8f7fe700 1 -- 172.21.15.62:0/446871536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 msgr2=0x557eea381ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea381ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8f7fe700 1 -- 172.21.15.62:0/446871536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 msgr2=0x557eea37ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea37ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8f7fe700 1 -- 172.21.15.62:0/446871536 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d901086e0 con 0x557eea370b30 2023-07-14T16:21:26.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.798+0000 7f5d8effd700 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea37ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:26.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.799+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea37a720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5d800108f0 tx=0x7f5d80004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18821dc9aade276 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.799+0000 7f5d6effd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:21:26.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.799+0000 7f5d6effd700 1 -- 172.21.15.62:0/446871536 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80004460 con 0x557eea370b30 2023-07-14T16:21:26.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.799+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/446871536 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557eea2174f0 con 0x557eea370b30 2023-07-14T16:21:26.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.799+0000 7f5d6effd700 1 -- 172.21.15.62:0/446871536 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5d80003680 con 0x557eea370b30 2023-07-14T16:21:26.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.799+0000 7f5d6effd700 1 -- 172.21.15.62:0/446871536 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d8002c030 con 0x557eea370b30 2023-07-14T16:21:26.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.799+0000 7f5da9ba2b80 10 client.26740 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:26.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.799+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/446871536 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557eea0d4c60 con 0x557eea370b30 2023-07-14T16:21:26.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.800+0000 7f5d6effd700 1 -- 172.21.15.62:0/446871536 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557eea2174f0 con 0x557eea370b30 2023-07-14T16:21:26.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.800+0000 7f5d6effd700 1 -- 172.21.15.62:0/446871536 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5d8001b900 con 0x557eea370b30 2023-07-14T16:21:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.800+0000 7f5da9ba2b80 10 client.26740 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.800+0000 7f5da9ba2b80 10 client.26740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.800+0000 7f5da9ba2b80 20 client.26740 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:26.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.800+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557eea386560 0x557eea388970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.800+0000 7f5da9ba2b80 4 client.26740 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.800+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/446871536 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557eea2b6eb0 con 0x557eea386560 2023-07-14T16:21:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.801+0000 7f5d8effd700 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557eea386560 0x557eea388970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.801+0000 7f5d8effd700 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557eea386560 0x557eea388970 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=251067461b97558d server_cookie=86e50c73b39709cc in_seq=0 out_seq=0 2023-07-14T16:21:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.801+0000 7f5d6effd700 10 client.26740 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5d6effd700 1 -- 172.21.15.62:0/446871536 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557eea2b6eb0 con 0x557eea386560 2023-07-14T16:21:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5d6effd700 10 client.26740 handle_command_reply: tid=0 2023-07-14T16:21:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 client.26740 shutdown 2023-07-14T16:21:26.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/446871536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 msgr2=0x557eea37a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea37a720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5d800108f0 tx=0x7f5d80004750 comp rx=0 tx=0).stop 2023-07-14T16:21:26.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/446871536 shutdown_connections 2023-07-14T16:21:26.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557eea386560 0x557eea388970 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea381ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea37a720 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/446871536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea37ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/446871536 >> 172.21.15.62:0/446871536 conn(0x557eea361ff0 msgr2=0x557eea366b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5d6effd700 0 client.26740 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:26.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/446871536 shutdown_connections 2023-07-14T16:21:26.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.802+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/446871536 wait complete. 2023-07-14T16:21:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.803+0000 7f5da9ba2b80 20 client.26740 trim_cache size 0 max 16384 2023-07-14T16:21:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.820+0000 7f5da9ba2b80 1 Processor -- start 2023-07-14T16:21:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.820+0000 7f5da9ba2b80 1 -- start start 2023-07-14T16:21:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.820+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea370b30 0x557eea378630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.820+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.820+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea372240 0x557eea375180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.820+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea379c00 con 0x557eea371480 2023-07-14T16:21:26.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.820+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea361ba0 con 0x557eea370b30 2023-07-14T16:21:26.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.820+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea379120 con 0x557eea372240 2023-07-14T16:21:26.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea370b30 0x557eea378630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea370b30 0x557eea378630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:54764/0 (socket says 172.21.15.62:54764) 2023-07-14T16:21:26.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 -- 172.21.15.62:0/3018153966 learned_addr learned my addr 172.21.15.62:0/3018153966 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/3018153966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea372240 0x557eea375180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 -- 172.21.15.62:0/3018153966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea372240 msgr2=0x557eea375180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/3018153966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea372240 0x557eea375180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 -- 172.21.15.62:0/3018153966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 msgr2=0x557eea381c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/3018153966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 -- 172.21.15.62:0/3018153966 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eea112250 con 0x557eea370b30 2023-07-14T16:21:26.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.821+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/3018153966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea370b30 0x557eea378630 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d80000c00 tx=0x7f5d80007580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b4830f6f3f29ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.822+0000 7f5d767fc700 1 -- 172.21.15.62:0/3018153966 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8002c040 con 0x557eea370b30 2023-07-14T16:21:26.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.822+0000 7f5d767fc700 1 -- 172.21.15.62:0/3018153966 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d8001be50 con 0x557eea370b30 2023-07-14T16:21:26.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.822+0000 7f5d767fc700 1 -- 172.21.15.62:0/3018153966 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d800054a0 con 0x557eea370b30 2023-07-14T16:21:26.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.822+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3018153966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea370b30 msgr2=0x557eea378630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.822+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3018153966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea370b30 0x557eea378630 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d80000c00 tx=0x7f5d80007580 comp rx=0 tx=0).stop 2023-07-14T16:21:26.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3018153966 shutdown_connections 2023-07-14T16:21:26.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3018153966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea372240 0x557eea375180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3018153966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea370b30 0x557eea378630 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3018153966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3018153966 >> 172.21.15.62:0/3018153966 conn(0x557eea361ff0 msgr2=0x557eea3793c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3018153966 shutdown_connections 2023-07-14T16:21:26.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3018153966 wait complete. 2023-07-14T16:21:26.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 -- start start 2023-07-14T16:21:26.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.823+0000 7f5da9ba2b80 1 Processor -- start 2023-07-14T16:21:26.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea381570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5da9ba2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea37e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea37e6f0 con 0x557eea371480 2023-07-14T16:21:26.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea37e860 con 0x557eea372240 2023-07-14T16:21:26.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5da9ba2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eea37ea20 con 0x557eea370b30 2023-07-14T16:21:26.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5da9ba2b80 10 client.0 _ll_register_callbacks cb 0x7f5d800104f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:26.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5d8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5d8effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34086/0 (socket says 172.21.15.62:34086) 2023-07-14T16:21:26.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5d8ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea37e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.824+0000 7f5d8effd700 1 -- 172.21.15.62:0/3440244796 learned_addr learned my addr 172.21.15.62:0/3440244796 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:26.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea381570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d8ffff700 1 -- 172.21.15.62:0/3440244796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 msgr2=0x557eea381570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea381570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d8ffff700 1 -- 172.21.15.62:0/3440244796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 msgr2=0x557eea381ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:26.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d8ffff700 1 -- 172.21.15.62:0/3440244796 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557eea0d4c60 con 0x557eea372240 2023-07-14T16:21:26.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d8effd700 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:26.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d8ffff700 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea37e010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800b830 tx=0x7f5d88003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f7520e9c7ddd2c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:26.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.825+0000 7f5d6effd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:21:26.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.826+0000 7f5d6effd700 1 -- 172.21.15.62:0/3440244796 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800e040 con 0x557eea372240 2023-07-14T16:21:26.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.826+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3440244796 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d880092d0 con 0x557eea372240 2023-07-14T16:21:26.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.826+0000 7f5d6effd700 1 -- 172.21.15.62:0/3440244796 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5d880050c0 con 0x557eea372240 2023-07-14T16:21:26.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.826+0000 7f5d6effd700 1 -- 172.21.15.62:0/3440244796 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d880052a0 con 0x557eea372240 2023-07-14T16:21:26.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.826+0000 7f5da9ba2b80 10 client.26852 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:26.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.826+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3440244796 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557eea0ea2b0 con 0x557eea372240 2023-07-14T16:21:26.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.826+0000 7f5d6effd700 1 -- 172.21.15.62:0/3440244796 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d880092d0 con 0x557eea372240 2023-07-14T16:21:26.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.827+0000 7f5d6effd700 1 -- 172.21.15.62:0/3440244796 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5d8800b0d0 con 0x557eea372240 2023-07-14T16:21:26.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.827+0000 7f5da9ba2b80 10 client.26852 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:26.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.827+0000 7f5da9ba2b80 10 client.26852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:26.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.827+0000 7f5da9ba2b80 20 client.26852 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:26.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.827+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557eea3db5f0 0x557eea3dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:26.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.827+0000 7f5da9ba2b80 4 client.26852 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:21:26.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.827+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3440244796 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557eea279050 con 0x557eea3db5f0 2023-07-14T16:21:26.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.827+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557eea3db5f0 0x557eea3dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:26.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5d8f7fe700 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557eea3db5f0 0x557eea3dda00 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d095ad02e684c6 server_cookie=6b85dfd6163eec65 in_seq=0 out_seq=0 2023-07-14T16:21:26.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5d6effd700 10 client.26852 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:26.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5d6effd700 1 -- 172.21.15.62:0/3440244796 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557eea279050 con 0x557eea3db5f0 2023-07-14T16:21:26.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5d6effd700 10 client.26852 handle_command_reply: tid=0 2023-07-14T16:21:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 client.26852 shutdown 2023-07-14T16:21:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3440244796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 msgr2=0x557eea37e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea37e010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800b830 tx=0x7f5d88003c30 comp rx=0 tx=0).stop 2023-07-14T16:21:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3440244796 shutdown_connections 2023-07-14T16:21:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557eea3db5f0 0x557eea3dda00 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557eea372240 0x557eea37e010 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557eea370b30 0x557eea381570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5d6effd700 0 client.26852 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 --2- 172.21.15.62:0/3440244796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557eea371480 0x557eea381ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3440244796 >> 172.21.15.62:0/3440244796 conn(0x557eea361ff0 msgr2=0x557eea378630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3440244796 shutdown_connections 2023-07-14T16:21:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 1 -- 172.21.15.62:0/3440244796 wait complete. 2023-07-14T16:21:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.828+0000 7f5da9ba2b80 20 client.26852 trim_cache size 0 max 16384 2023-07-14T16:21:26.835 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:21:26.835 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:21:26.835 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:21:26.835 INFO:teuthology.orchestra.run.smithi062.stdout: "ff5c95e0-0016-45dc-a69f-bbe06aed3b00": { 2023-07-14T16:21:26.835 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:21:26.835 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "ff5c95e0-0016-45dc-a69f-bbe06aed3b00", 2023-07-14T16:21:26.835 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:21:26.836 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:21:26.836 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:21:26.836 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:21:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 -- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5d780a11f0 msgr2=0x7f5d780a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5d780a11f0 0x7f5d780a36a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5d80007d40 tx=0x7f5d800073d0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 -- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90102dd0 msgr2=0x7f5d90195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90102dd0 0x7f5d90195b30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400d950 tx=0x7f5d8400dc60 comp rx=0 tx=0).stop 2023-07-14T16:21:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 -- 172.21.15.62:0/2077170414 shutdown_connections 2023-07-14T16:21:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d901056f0 0x7f5d9019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5d780a11f0 0x7f5d780a36a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d901004b0 0x7f5d901955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 --2- 172.21.15.62:0/2077170414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d90102dd0 0x7f5d90195b30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 -- 172.21.15.62:0/2077170414 >> 172.21.15.62:0/2077170414 conn(0x7f5d900f9ed0 msgr2=0x7f5d90101110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 -- 172.21.15.62:0/2077170414 shutdown_connections 2023-07-14T16:21:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:26.829+0000 7f5d94f9d700 1 -- 172.21.15.62:0/2077170414 wait complete. 2023-07-14T16:21:26.839 INFO:tasks.cephfs.filesystem:scrub status for tag:ff5c95e0-0016-45dc-a69f-bbe06aed3b00 - {'path': '/', 'tag': 'ff5c95e0-0016-45dc-a69f-bbe06aed3b00', 'options': 'recursive,force'} 2023-07-14T16:21:26.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:26 smithi062 ceph-mon[95543]: pgmap v531: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:21:26.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:26 smithi081 ceph-mon[127795]: pgmap v531: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:21:26.903 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:21:26 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:21:26.541+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v141>,<0x10000000003/client.0 v142>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v154>,<0x10000000001/sv_0 v168>,<0x10000000000/_nogroup v162>,<0x1/volumes v152>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:21:27.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:26 smithi164 ceph-mon[97427]: pgmap v531: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:21:28.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:21:28.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:21:28.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:21:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:21:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:21:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:21:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:27 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:21:28.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:21:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:21:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:21:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:21:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:21:28.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:27 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:21:28.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:21:28.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:21:28.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:21:28.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:21:28.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:21:28.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:28.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:27 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:21:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:28 smithi164 ceph-mon[97427]: pgmap v532: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:29.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:28 smithi062 ceph-mon[95543]: pgmap v532: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:29.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:28 smithi081 ceph-mon[127795]: pgmap v532: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:31.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:30 smithi164 ceph-mon[97427]: pgmap v533: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2023-07-14T16:21:31.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:30 smithi062 ceph-mon[95543]: pgmap v533: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2023-07-14T16:21:31.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:30 smithi081 ceph-mon[127795]: pgmap v533: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 26 op/s 2023-07-14T16:21:33.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:32 smithi164 ceph-mon[97427]: pgmap v534: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:21:33.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:32 smithi062 ceph-mon[95543]: pgmap v534: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:21:33.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:32 smithi081 ceph-mon[127795]: pgmap v534: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:21:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 -- 172.21.15.62:0/2342927268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 msgr2=0x7f2180073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 --2- 172.21.15.62:0/2342927268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 0x7f2180073380 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2174009990 tx=0x7f2174009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 -- 172.21.15.62:0/2342927268 shutdown_connections 2023-07-14T16:21:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 --2- 172.21.15.62:0/2342927268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21801079d0 0x7f2180109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 --2- 172.21.15.62:0/2342927268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21800744f0 0x7f2180072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 --2- 172.21.15.62:0/2342927268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 0x7f2180073380 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 -- 172.21.15.62:0/2342927268 >> 172.21.15.62:0/2342927268 conn(0x7f21800fb160 msgr2=0x7f21800fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 -- 172.21.15.62:0/2342927268 shutdown_connections 2023-07-14T16:21:33.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.595+0000 7f21857ff700 1 -- 172.21.15.62:0/2342927268 wait complete. 2023-07-14T16:21:33.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.596+0000 7f21857ff700 1 Processor -- start 2023-07-14T16:21:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.596+0000 7f21857ff700 1 -- start start 2023-07-14T16:21:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f21857ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 0x7f21801979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f21857ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800744f0 0x7f2180197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f21857ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21801079d0 0x7f2180198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f21857ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21801993d0 con 0x7f2180072f10 2023-07-14T16:21:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f21857ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180199540 con 0x7f21800744f0 2023-07-14T16:21:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f21857ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180199700 con 0x7f21801079d0 2023-07-14T16:21:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 0x7f21801979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 0x7f21801979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33290/0 (socket says 172.21.15.62:33290) 2023-07-14T16:21:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217effd700 1 -- 172.21.15.62:0/1514960203 learned_addr learned my addr 172.21.15.62:0/1514960203 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800744f0 0x7f2180197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217f7fe700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21801079d0 0x7f2180198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217effd700 1 -- 172.21.15.62:0/1514960203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21801079d0 msgr2=0x7f2180198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217effd700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21801079d0 0x7f2180198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217effd700 1 -- 172.21.15.62:0/1514960203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800744f0 msgr2=0x7f2180197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217effd700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800744f0 0x7f2180197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217effd700 1 -- 172.21.15.62:0/1514960203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740095f0 con 0x7f2180072f10 2023-07-14T16:21:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.597+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800744f0 0x7f2180197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:33.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.598+0000 7f217effd700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 0x7f21801979b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f217000c9b0 tx=0x7f217000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.598+0000 7f2167fff700 1 -- 172.21.15.62:0/1514960203 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170007b00 con 0x7f2180072f10 2023-07-14T16:21:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.598+0000 7f2167fff700 1 -- 172.21.15.62:0/1514960203 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2170007c60 con 0x7f2180072f10 2023-07-14T16:21:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.598+0000 7f21857ff700 1 -- 172.21.15.62:0/1514960203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21801999e0 con 0x7f2180072f10 2023-07-14T16:21:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.598+0000 7f2167fff700 1 -- 172.21.15.62:0/1514960203 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170018660 con 0x7f2180072f10 2023-07-14T16:21:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.598+0000 7f21857ff700 1 -- 172.21.15.62:0/1514960203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21801a01a0 con 0x7f2180072f10 2023-07-14T16:21:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.602+0000 7f2167fff700 1 -- 172.21.15.62:0/1514960203 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f217000f450 con 0x7f2180072f10 2023-07-14T16:21:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.604+0000 7f2167fff700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21680a11d0 0x7f21680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.604+0000 7f2167fff700 1 -- 172.21.15.62:0/1514960203 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f21700cc950 con 0x7f2180072f10 2023-07-14T16:21:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.604+0000 7f21990b7b80 1 Processor -- start 2023-07-14T16:21:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.604+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21680a11d0 0x7f21680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f21990b7b80 1 -- start start 2023-07-14T16:21:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 0x563b348cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cb840 0x563b348cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f21990b7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b347d0c50 con 0x563b348cb840 2023-07-14T16:21:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f21990b7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b348239e0 con 0x563b347276b0 2023-07-14T16:21:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f21990b7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3481ce70 con 0x563b348cc6e0 2023-07-14T16:21:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f217e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cb840 0x563b348cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f217f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f217f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:58728/0 (socket says 172.21.15.62:58728) 2023-07-14T16:21:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.605+0000 7f217f7fe700 1 -- 172.21.15.62:0/1037448662 learned_addr learned my addr 172.21.15.62:0/1037448662 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.606+0000 7f217effd700 1 --2- 172.21.15.62:0/1037448662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 0x563b348cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.606+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21680a11d0 0x7f21680a3680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f217400b5c0 tx=0x7f2174005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.606+0000 7f217e7fc700 1 -- 172.21.15.62:0/1037448662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 msgr2=0x563b348ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.606+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1037448662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.606+0000 7f217e7fc700 1 -- 172.21.15.62:0/1037448662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 msgr2=0x563b348cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.606+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1037448662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 0x563b348cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.606+0000 7f217e7fc700 1 -- 172.21.15.62:0/1037448662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b3464ad80 con 0x563b348cb840 2023-07-14T16:21:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.606+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1037448662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cb840 0x563b348cc1a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2174000c00 tx=0x7f217401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39528609cc729afd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.607+0000 7f21657fa700 1 -- 172.21.15.62:0/1037448662 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217402b070 con 0x563b348cb840 2023-07-14T16:21:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.607+0000 7f21657fa700 1 -- 172.21.15.62:0/1037448662 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f217401e040 con 0x563b348cb840 2023-07-14T16:21:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.607+0000 7f21657fa700 1 -- 172.21.15.62:0/1037448662 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174004b00 con 0x563b348cb840 2023-07-14T16:21:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.607+0000 7f21990b7b80 1 -- 172.21.15.62:0/1037448662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cb840 msgr2=0x563b348cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.607+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1037448662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cb840 0x563b348cc1a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2174000c00 tx=0x7f217401f000 comp rx=0 tx=0).stop 2023-07-14T16:21:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 -- 172.21.15.62:0/1037448662 shutdown_connections 2023-07-14T16:21:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1037448662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1037448662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 0x563b348cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1037448662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cb840 0x563b348cc1a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 -- 172.21.15.62:0/1037448662 >> 172.21.15.62:0/1037448662 conn(0x563b348c0be0 msgr2=0x563b348c5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 -- 172.21.15.62:0/1037448662 shutdown_connections 2023-07-14T16:21:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 -- 172.21.15.62:0/1037448662 wait complete. 2023-07-14T16:21:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 -- start start 2023-07-14T16:21:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.608+0000 7f21990b7b80 1 Processor -- start 2023-07-14T16:21:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 0x563b348d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cc6e0 0x563b348dbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f21990b7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b347bca60 con 0x563b348cc6e0 2023-07-14T16:21:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f21990b7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b347bc760 con 0x563b347276b0 2023-07-14T16:21:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f21990b7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b346a08c0 con 0x563b348cb840 2023-07-14T16:21:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f21990b7b80 10 client.0 _ll_register_callbacks cb 0x563b347ba700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f217f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cc6e0 0x563b348dbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f217f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cc6e0 0x563b348dbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33300/0 (socket says 172.21.15.62:33300) 2023-07-14T16:21:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f217f7fe700 1 -- 172.21.15.62:0/889246956 learned_addr learned my addr 172.21.15.62:0/889246956 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f217effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 0x563b348d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.609+0000 7f217e7fc700 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f217f7fe700 1 -- 172.21.15.62:0/889246956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 msgr2=0x563b348d4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f217f7fe700 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f217f7fe700 1 -- 172.21.15.62:0/889246956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 msgr2=0x563b348d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f217f7fe700 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 0x563b348d4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f217f7fe700 1 -- 172.21.15.62:0/889246956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218010aa70 con 0x563b348cc6e0 2023-07-14T16:21:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f217f7fe700 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cc6e0 0x563b348dbd90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f21780060b0 tx=0x7f2178007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349dee9306bce8c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f2162ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f2162ff5700 1 -- 172.21.15.62:0/889246956 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178004db0 con 0x563b348cc6e0 2023-07-14T16:21:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f2162ff5700 1 -- 172.21.15.62:0/889246956 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2178012940 con 0x563b348cc6e0 2023-07-14T16:21:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.610+0000 7f21990b7b80 1 -- 172.21.15.62:0/889246956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b34776490 con 0x563b348cc6e0 2023-07-14T16:21:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.611+0000 7f2162ff5700 1 -- 172.21.15.62:0/889246956 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2178012b00 con 0x563b348cc6e0 2023-07-14T16:21:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.611+0000 7f21990b7b80 10 client.17172 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.611+0000 7f21990b7b80 1 -- 172.21.15.62:0/889246956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563b34633c00 con 0x563b348cc6e0 2023-07-14T16:21:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.611+0000 7f2162ff5700 1 -- 172.21.15.62:0/889246956 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563b34776490 con 0x563b348cc6e0 2023-07-14T16:21:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.611+0000 7f2162ff5700 1 -- 172.21.15.62:0/889246956 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2178014040 con 0x563b348cc6e0 2023-07-14T16:21:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.611+0000 7f21990b7b80 10 client.17172 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.611+0000 7f21990b7b80 10 client.17172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.611+0000 7f21990b7b80 20 client.17172 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.612+0000 7f21990b7b80 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b348e06c0 0x563b348e2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.612+0000 7f21990b7b80 4 client.17172 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.612+0000 7f21990b7b80 1 -- 172.21.15.62:0/889246956 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b34815e50 con 0x563b348e06c0 2023-07-14T16:21:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.612+0000 7f217effd700 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b348e06c0 0x563b348e2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.613+0000 7f2162ff5700 10 client.17172 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.613+0000 7f217effd700 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b348e06c0 0x563b348e2ad0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14bee0c22c309502 server_cookie=1be266feea26a0f7 in_seq=0 out_seq=0 2023-07-14T16:21:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.613+0000 7f2162ff5700 1 -- 172.21.15.62:0/889246956 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b34815e50 con 0x563b348e06c0 2023-07-14T16:21:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.613+0000 7f2162ff5700 10 client.17172 handle_command_reply: tid=0 2023-07-14T16:21:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 client.17172 shutdown 2023-07-14T16:21:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 -- 172.21.15.62:0/889246956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cc6e0 msgr2=0x563b348dbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cc6e0 0x563b348dbd90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f21780060b0 tx=0x7f2178007df0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 -- 172.21.15.62:0/889246956 shutdown_connections 2023-07-14T16:21:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b348cc6e0 0x563b348dbd90 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b348e06c0 0x563b348e2ad0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b347276b0 0x563b348d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 --2- 172.21.15.62:0/889246956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 -- 172.21.15.62:0/889246956 >> 172.21.15.62:0/889246956 conn(0x563b348c0be0 msgr2=0x563b348c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f2162ff5700 0 client.17172 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 -- 172.21.15.62:0/889246956 shutdown_connections 2023-07-14T16:21:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 1 -- 172.21.15.62:0/889246956 wait complete. 2023-07-14T16:21:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.614+0000 7f21990b7b80 20 client.17172 trim_cache size 0 max 16384 2023-07-14T16:21:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.634+0000 7f21990b7b80 1 Processor -- start 2023-07-14T16:21:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.634+0000 7f21990b7b80 1 -- start start 2023-07-14T16:21:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.634+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.634+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cb840 0x563b348cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.634+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.634+0000 7f21990b7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b348c69c0 con 0x563b347276b0 2023-07-14T16:21:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.634+0000 7f21990b7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b348c67a0 con 0x563b348cb840 2023-07-14T16:21:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.634+0000 7f21990b7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b348c2d00 con 0x563b348cc6e0 2023-07-14T16:21:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33314/0 (socket says 172.21.15.62:33314) 2023-07-14T16:21:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 -- 172.21.15.62:0/259540642 learned_addr learned my addr 172.21.15.62:0/259540642 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217e7fc700 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cb840 0x563b348cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217f7fe700 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 -- 172.21.15.62:0/259540642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 msgr2=0x563b348dbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348dbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 -- 172.21.15.62:0/259540642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cb840 msgr2=0x563b348cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cb840 0x563b348cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 -- 172.21.15.62:0/259540642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b346711f0 con 0x563b347276b0 2023-07-14T16:21:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217e7fc700 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cb840 0x563b348cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.635+0000 7f217effd700 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348cf550 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f21700074a0 tx=0x7f21700960d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5bb596f4708c81 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f2165ffb700 1 -- 172.21.15.62:0/259540642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170097e50 con 0x563b347276b0 2023-07-14T16:21:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f2165ffb700 1 -- 172.21.15.62:0/259540642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21700ca510 con 0x563b347276b0 2023-07-14T16:21:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f2165ffb700 1 -- 172.21.15.62:0/259540642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170099f70 con 0x563b347276b0 2023-07-14T16:21:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f21990b7b80 1 -- 172.21.15.62:0/259540642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 msgr2=0x563b348cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f21990b7b80 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348cf550 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f21700074a0 tx=0x7f21700960d0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f21990b7b80 1 -- 172.21.15.62:0/259540642 shutdown_connections 2023-07-14T16:21:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f21990b7b80 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cc6e0 0x563b348dbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f21990b7b80 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348cf550 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f21990b7b80 1 --2- 172.21.15.62:0/259540642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cb840 0x563b348cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.636+0000 7f21990b7b80 1 -- 172.21.15.62:0/259540642 >> 172.21.15.62:0/259540642 conn(0x563b348c0be0 msgr2=0x563b348d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.637+0000 7f21990b7b80 1 -- 172.21.15.62:0/259540642 shutdown_connections 2023-07-14T16:21:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.637+0000 7f21990b7b80 1 -- 172.21.15.62:0/259540642 wait complete. 2023-07-14T16:21:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.637+0000 7f21990b7b80 1 -- start start 2023-07-14T16:21:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.637+0000 7f21990b7b80 1 Processor -- start 2023-07-14T16:21:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.638+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.638+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.638+0000 7f21990b7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cc6e0 0x563b348db290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.638+0000 7f21990b7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b348db7d0 con 0x563b347276b0 2023-07-14T16:21:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.638+0000 7f21990b7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b348db940 con 0x563b348cc6e0 2023-07-14T16:21:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.638+0000 7f21990b7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b348dbb00 con 0x563b348cb840 2023-07-14T16:21:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.638+0000 7f21990b7b80 10 client.0 _ll_register_callbacks cb 0x7f2178009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.638+0000 7f217effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348d7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33316/0 (socket says 172.21.15.62:33316) 2023-07-14T16:21:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217effd700 1 -- 172.21.15.62:0/1857707697 learned_addr learned my addr 172.21.15.62:0/1857707697 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217f7fe700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cc6e0 0x563b348db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217effd700 1 -- 172.21.15.62:0/1857707697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 msgr2=0x563b348d8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217effd700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217effd700 1 -- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cc6e0 msgr2=0x563b348db290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217effd700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cc6e0 0x563b348db290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217effd700 1 -- 172.21.15.62:0/1857707697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21801a01a0 con 0x563b347276b0 2023-07-14T16:21:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217f7fe700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cc6e0 0x563b348db290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f217effd700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348d7d30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f21700997e0 tx=0x7f2170007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378e6aea6fe1a392 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.639+0000 7f2162ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f2162ff5700 1 -- 172.21.15.62:0/1857707697 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000f900 con 0x563b347276b0 2023-07-14T16:21:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f2162ff5700 1 -- 172.21.15.62:0/1857707697 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2170099200 con 0x563b347276b0 2023-07-14T16:21:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 1 -- 172.21.15.62:0/1857707697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f217401e040 con 0x563b347276b0 2023-07-14T16:21:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f2162ff5700 1 -- 172.21.15.62:0/1857707697 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21700993c0 con 0x563b347276b0 2023-07-14T16:21:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 10 client.17178 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 1 -- 172.21.15.62:0/1857707697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563b34649250 con 0x563b347276b0 2023-07-14T16:21:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f2162ff5700 1 -- 172.21.15.62:0/1857707697 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f217401e040 con 0x563b347276b0 2023-07-14T16:21:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f2162ff5700 1 -- 172.21.15.62:0/1857707697 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f21700b9ac0 con 0x563b347276b0 2023-07-14T16:21:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 10 client.17178 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 10 client.17178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 20 client.17178 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b3493dc50 0x563b34940060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 4 client.17178 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:21:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.640+0000 7f21990b7b80 1 -- 172.21.15.62:0/1857707697 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b347d7ff0 con 0x563b3493dc50 2023-07-14T16:21:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.641+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b3493dc50 0x563b34940060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.641+0000 7f2162ff5700 10 client.17178 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.641+0000 7f217e7fc700 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b3493dc50 0x563b34940060 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336d35c14014467c server_cookie=85d34925cd83aa27 in_seq=0 out_seq=0 2023-07-14T16:21:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f2162ff5700 1 -- 172.21.15.62:0/1857707697 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b347d7ff0 con 0x563b3493dc50 2023-07-14T16:21:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f2162ff5700 10 client.17178 handle_command_reply: tid=0 2023-07-14T16:21:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 client.17178 shutdown 2023-07-14T16:21:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 -- 172.21.15.62:0/1857707697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 msgr2=0x563b348d7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348d7d30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f21700997e0 tx=0x7f2170007eb0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 -- 172.21.15.62:0/1857707697 shutdown_connections 2023-07-14T16:21:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563b3493dc50 0x563b34940060 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563b348cc6e0 0x563b348db290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f2162ff5700 0 client.17178 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563b347276b0 0x563b348d7d30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 --2- 172.21.15.62:0/1857707697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b348cb840 0x563b348d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.651+0000 7f21990b7b80 1 -- 172.21.15.62:0/1857707697 >> 172.21.15.62:0/1857707697 conn(0x563b348c0be0 msgr2=0x563b348d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.652+0000 7f21990b7b80 1 -- 172.21.15.62:0/1857707697 shutdown_connections 2023-07-14T16:21:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.652+0000 7f21990b7b80 1 -- 172.21.15.62:0/1857707697 wait complete. 2023-07-14T16:21:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.652+0000 7f21990b7b80 20 client.17178 trim_cache size 0 max 16384 2023-07-14T16:21:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:21:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:21:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:21:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:21:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.652+0000 7f21857ff700 1 -- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21680a11d0 msgr2=0x7f21680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.652+0000 7f21857ff700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21680a11d0 0x7f21680a3680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f217400b5c0 tx=0x7f2174005fd0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.652+0000 7f21857ff700 1 -- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 msgr2=0x7f21801979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.652+0000 7f21857ff700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 0x7f21801979b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f217000c9b0 tx=0x7f217000cd70 comp rx=0 tx=0).stop 2023-07-14T16:21:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.653+0000 7f21857ff700 1 -- 172.21.15.62:0/1514960203 shutdown_connections 2023-07-14T16:21:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.653+0000 7f21857ff700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21801079d0 0x7f2180198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.653+0000 7f21857ff700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2180072f10 0x7f21801979b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.653+0000 7f21857ff700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21680a11d0 0x7f21680a3680 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.653+0000 7f21857ff700 1 --2- 172.21.15.62:0/1514960203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21800744f0 0x7f2180197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.653+0000 7f21857ff700 1 -- 172.21.15.62:0/1514960203 >> 172.21.15.62:0/1514960203 conn(0x7f21800fb160 msgr2=0x7f2180109d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.653+0000 7f21857ff700 1 -- 172.21.15.62:0/1514960203 shutdown_connections 2023-07-14T16:21:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:33.653+0000 7f21857ff700 1 -- 172.21.15.62:0/1514960203 wait complete. 2023-07-14T16:21:34.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:33 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:21:34.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:33 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:21:34.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:33 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:21:34.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:33 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:21:34.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:33 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:21:34.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:33 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:21:35.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:34 smithi164 ceph-mon[97427]: pgmap v535: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:21:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:34 smithi062 ceph-mon[95543]: pgmap v535: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:21:35.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:34 smithi081 ceph-mon[127795]: pgmap v535: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:21:37.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:36 smithi164 ceph-mon[97427]: pgmap v536: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:21:37.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:36 smithi062 ceph-mon[95543]: pgmap v536: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:21:37.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:36 smithi081 ceph-mon[127795]: pgmap v536: 97 pgs: 97 active+clean; 3.5 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:21:39.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:38 smithi164 ceph-mon[97427]: pgmap v537: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:21:39.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:38 smithi062 ceph-mon[95543]: pgmap v537: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:21:39.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:38 smithi081 ceph-mon[127795]: pgmap v537: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:21:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:40 smithi164 ceph-mon[97427]: pgmap v538: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:21:41.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:40 smithi062 ceph-mon[95543]: pgmap v538: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:21:41.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:40 smithi081 ceph-mon[127795]: pgmap v538: 97 pgs: 97 active+clean; 3.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:21:43.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:42 smithi164 ceph-mon[97427]: pgmap v539: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:21:43.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:42 smithi062 ceph-mon[95543]: pgmap v539: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:21:43.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:42 smithi081 ceph-mon[127795]: pgmap v539: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:21:45.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:44 smithi164 ceph-mon[97427]: pgmap v540: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:45.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:44 smithi062 ceph-mon[95543]: pgmap v540: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:45.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:44 smithi081 ceph-mon[127795]: pgmap v540: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:21:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 -- 172.21.15.62:0/2157057263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 msgr2=0x7f1698107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 --2- 172.21.15.62:0/2157057263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 0x7f1698107b90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1690009ec0 tx=0x7f169000b330 comp rx=0 tx=0).stop 2023-07-14T16:21:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 -- 172.21.15.62:0/2157057263 shutdown_connections 2023-07-14T16:21:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 --2- 172.21.15.62:0/2157057263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 0x7f1698107b90 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 --2- 172.21.15.62:0/2157057263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1698068620 0x7f16981026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 --2- 172.21.15.62:0/2157057263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698102bf0 0x7f1698105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 -- 172.21.15.62:0/2157057263 >> 172.21.15.62:0/2157057263 conn(0x7f16980fa000 msgr2=0x7f16980fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 -- 172.21.15.62:0/2157057263 shutdown_connections 2023-07-14T16:21:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.823+0000 7f169f7ff700 1 -- 172.21.15.62:0/2157057263 wait complete. 2023-07-14T16:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.824+0000 7f169f7ff700 1 Processor -- start 2023-07-14T16:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.824+0000 7f169f7ff700 1 -- start start 2023-07-14T16:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169f7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1698068620 0x7f1698197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169f7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698102bf0 0x7f16981980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169f7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 0x7f1698198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169f7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1698199590 con 0x7f16981056e0 2023-07-14T16:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169f7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1698199700 con 0x7f1698068620 2023-07-14T16:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169f7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16981998c0 con 0x7f1698102bf0 2023-07-14T16:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169dd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 0x7f1698198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169dd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 0x7f1698198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51172/0 (socket says 172.21.15.62:51172) 2023-07-14T16:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169dd9c700 1 -- 172.21.15.62:0/3487034589 learned_addr learned my addr 172.21.15.62:0/3487034589 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169d59b700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1698068620 0x7f1698197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.825+0000 7f169cd9a700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698102bf0 0x7f16981980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f169dd9c700 1 -- 172.21.15.62:0/3487034589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698102bf0 msgr2=0x7f16981980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f169dd9c700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698102bf0 0x7f16981980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f169dd9c700 1 -- 172.21.15.62:0/3487034589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1698068620 msgr2=0x7f1698197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f169dd9c700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1698068620 0x7f1698197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f169dd9c700 1 -- 172.21.15.62:0/3487034589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690005e70 con 0x7f16981056e0 2023-07-14T16:21:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f169dd9c700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 0x7f1698198f30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1690007cc0 tx=0x7f1690007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f168e7fc700 1 -- 172.21.15.62:0/3487034589 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000e040 con 0x7f16981056e0 2023-07-14T16:21:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f169f7ff700 1 -- 172.21.15.62:0/3487034589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1698199b40 con 0x7f16981056e0 2023-07-14T16:21:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f168e7fc700 1 -- 172.21.15.62:0/3487034589 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f169001dd20 con 0x7f16981056e0 2023-07-14T16:21:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f168e7fc700 1 -- 172.21.15.62:0/3487034589 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169001baf0 con 0x7f16981056e0 2023-07-14T16:21:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.826+0000 7f169f7ff700 1 -- 172.21.15.62:0/3487034589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16981a0360 con 0x7f16981056e0 2023-07-14T16:21:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.830+0000 7f168e7fc700 1 -- 172.21.15.62:0/3487034589 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f169001bc50 con 0x7f16981056e0 2023-07-14T16:21:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.831+0000 7f168e7fc700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16840a11d0 0x7f16840a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.831+0000 7f168e7fc700 1 -- 172.21.15.62:0/3487034589 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16900ce1c0 con 0x7f16981056e0 2023-07-14T16:21:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.831+0000 7f169d59b700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16840a11d0 0x7f16840a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.832+0000 7f16b309cb80 1 Processor -- start 2023-07-14T16:21:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.832+0000 7f169d59b700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16840a11d0 0x7f16840a3680 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1694005950 tx=0x7f169400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.832+0000 7f16b309cb80 1 -- start start 2023-07-14T16:21:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.832+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb9984d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.832+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 0x55bb9984e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.832+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984e6e0 0x55bb9984eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f16b309cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb99752c50 con 0x55bb996a96b0 2023-07-14T16:21:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f16b309cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb997a59e0 con 0x55bb9984d840 2023-07-14T16:21:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f16b309cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9979ee70 con 0x55bb9984e6e0 2023-07-14T16:21:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb9984d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb9984d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51174/0 (socket says 172.21.15.62:51174) 2023-07-14T16:21:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169d59b700 1 -- 172.21.15.62:0/3363518539 learned_addr learned my addr 172.21.15.62:0/3363518539 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169cd9a700 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 0x55bb9984e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169dd9c700 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984e6e0 0x55bb9984eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169d59b700 1 -- 172.21.15.62:0/3363518539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984e6e0 msgr2=0x55bb9984eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169d59b700 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984e6e0 0x55bb9984eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169d59b700 1 -- 172.21.15.62:0/3363518539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 msgr2=0x55bb9984e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169d59b700 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 0x55bb9984e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169d59b700 1 -- 172.21.15.62:0/3363518539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb995ccd80 con 0x55bb996a96b0 2023-07-14T16:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.833+0000 7f169dd9c700 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984e6e0 0x55bb9984eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.834+0000 7f169d59b700 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb9984d300 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1694010f70 tx=0x7f1694013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893a62168b14364f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.834+0000 7f16837fe700 1 -- 172.21.15.62:0/3363518539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694011db0 con 0x55bb996a96b0 2023-07-14T16:21:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.834+0000 7f16837fe700 1 -- 172.21.15.62:0/3363518539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1694015070 con 0x55bb996a96b0 2023-07-14T16:21:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.834+0000 7f16837fe700 1 -- 172.21.15.62:0/3363518539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169401c610 con 0x55bb996a96b0 2023-07-14T16:21:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 -- 172.21.15.62:0/3363518539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 msgr2=0x55bb9984d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb9984d300 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1694010f70 tx=0x7f1694013000 comp rx=0 tx=0).stop 2023-07-14T16:21:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 -- 172.21.15.62:0/3363518539 shutdown_connections 2023-07-14T16:21:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984e6e0 0x55bb9984eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb9984d300 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 --2- 172.21.15.62:0/3363518539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 0x55bb9984e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 -- 172.21.15.62:0/3363518539 >> 172.21.15.62:0/3363518539 conn(0x55bb99842be0 msgr2=0x55bb99847cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 -- 172.21.15.62:0/3363518539 shutdown_connections 2023-07-14T16:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.835+0000 7f16b309cb80 1 -- 172.21.15.62:0/3363518539 wait complete. 2023-07-14T16:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.836+0000 7f16b309cb80 1 -- start start 2023-07-14T16:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.836+0000 7f16b309cb80 1 Processor -- start 2023-07-14T16:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb996a96b0 0x55bb998568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 0x55bb99856e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb9984e6e0 0x55bb9985de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f16b309cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9973ea60 con 0x55bb9984e6e0 2023-07-14T16:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f16b309cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9973e760 con 0x55bb9984d840 2023-07-14T16:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f16b309cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb996228c0 con 0x55bb996a96b0 2023-07-14T16:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f16b309cb80 10 client.0 _ll_register_callbacks cb 0x55bb9973c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f169dd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb9984e6e0 0x55bb9985de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f169dd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb9984e6e0 0x55bb9985de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51182/0 (socket says 172.21.15.62:51182) 2023-07-14T16:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f169dd9c700 1 -- 172.21.15.62:0/2224755704 learned_addr learned my addr 172.21.15.62:0/2224755704 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f169cd9a700 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 0x55bb99856e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.837+0000 7f169d59b700 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb996a96b0 0x55bb998568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f169dd9c700 1 -- 172.21.15.62:0/2224755704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb996a96b0 msgr2=0x55bb998568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f169dd9c700 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb996a96b0 0x55bb998568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f169dd9c700 1 -- 172.21.15.62:0/2224755704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 msgr2=0x55bb99856e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f169dd9c700 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 0x55bb99856e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f169dd9c700 1 -- 172.21.15.62:0/2224755704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1694010c20 con 0x55bb9984e6e0 2023-07-14T16:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f169dd9c700 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb9984e6e0 0x55bb9985de50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f16900cbd20 tx=0x7f169000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22604a0735b7a195 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f1680ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f1680ff9700 1 -- 172.21.15.62:0/2224755704 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169009bad0 con 0x55bb9984e6e0 2023-07-14T16:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f1680ff9700 1 -- 172.21.15.62:0/2224755704 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f16900cd070 con 0x55bb9984e6e0 2023-07-14T16:21:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f16b309cb80 1 -- 172.21.15.62:0/2224755704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb996f8490 con 0x55bb9984e6e0 2023-07-14T16:21:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f1680ff9700 1 -- 172.21.15.62:0/2224755704 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f169009afd0 con 0x55bb9984e6e0 2023-07-14T16:21:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f16b309cb80 10 client.17190 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.838+0000 7f16b309cb80 1 -- 172.21.15.62:0/2224755704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb995b5c00 con 0x55bb9984e6e0 2023-07-14T16:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.839+0000 7f1680ff9700 1 -- 172.21.15.62:0/2224755704 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bb996f8490 con 0x55bb9984e6e0 2023-07-14T16:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.839+0000 7f1680ff9700 1 -- 172.21.15.62:0/2224755704 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f169009ab60 con 0x55bb9984e6e0 2023-07-14T16:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.839+0000 7f16b309cb80 10 client.17190 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.839+0000 7f16b309cb80 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.839+0000 7f16b309cb80 20 client.17190 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.839+0000 7f16b309cb80 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bb998624a0 0x55bb998648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.839+0000 7f16b309cb80 4 client.17190 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.839+0000 7f16b309cb80 1 -- 172.21.15.62:0/2224755704 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb99797e50 con 0x55bb998624a0 2023-07-14T16:21:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.840+0000 7f169d59b700 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bb998624a0 0x55bb998648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.840+0000 7f1680ff9700 10 client.17190 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.840+0000 7f169d59b700 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bb998624a0 0x55bb998648b0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ad54612c4a6b16 server_cookie=b6ae0f646033a920 in_seq=0 out_seq=0 2023-07-14T16:21:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f1680ff9700 1 -- 172.21.15.62:0/2224755704 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb99797e50 con 0x55bb998624a0 2023-07-14T16:21:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f1680ff9700 10 client.17190 handle_command_reply: tid=0 2023-07-14T16:21:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 client.17190 shutdown 2023-07-14T16:21:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 -- 172.21.15.62:0/2224755704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb9984e6e0 msgr2=0x55bb9985de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb9984e6e0 0x55bb9985de50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f16900cbd20 tx=0x7f169000b330 comp rx=0 tx=0).stop 2023-07-14T16:21:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 -- 172.21.15.62:0/2224755704 shutdown_connections 2023-07-14T16:21:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb9984e6e0 0x55bb9985de50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb996a96b0 0x55bb998568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bb998624a0 0x55bb998648b0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 --2- 172.21.15.62:0/2224755704 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984d840 0x55bb99856e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 -- 172.21.15.62:0/2224755704 >> 172.21.15.62:0/2224755704 conn(0x55bb99842be0 msgr2=0x55bb998474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f1680ff9700 0 client.17190 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.841+0000 7f16b309cb80 1 -- 172.21.15.62:0/2224755704 shutdown_connections 2023-07-14T16:21:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.842+0000 7f16b309cb80 1 -- 172.21.15.62:0/2224755704 wait complete. 2023-07-14T16:21:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.842+0000 7f16b309cb80 20 client.17190 trim_cache size 0 max 16384 2023-07-14T16:21:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.863+0000 7f16b309cb80 1 Processor -- start 2023-07-14T16:21:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.863+0000 7f16b309cb80 1 -- start start 2023-07-14T16:21:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99854b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 0x55bb9985de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 0x55bb99851670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f16b309cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb99797e50 con 0x55bb996a96b0 2023-07-14T16:21:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f16b309cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb996f8490 con 0x55bb9984e6e0 2023-07-14T16:21:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f16b309cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb99844d00 con 0x55bb9984d840 2023-07-14T16:21:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99854b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99854b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51194/0 (socket says 172.21.15.62:51194) 2023-07-14T16:21:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169d59b700 1 -- 172.21.15.62:0/460634024 learned_addr learned my addr 172.21.15.62:0/460634024 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169dd9c700 1 --2- 172.21.15.62:0/460634024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 0x55bb99851670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169d59b700 1 -- 172.21.15.62:0/460634024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 msgr2=0x55bb9985de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169d59b700 1 --2- 172.21.15.62:0/460634024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 0x55bb9985de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169d59b700 1 -- 172.21.15.62:0/460634024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 msgr2=0x55bb99851670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169d59b700 1 --2- 172.21.15.62:0/460634024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 0x55bb99851670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.864+0000 7f169d59b700 1 -- 172.21.15.62:0/460634024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb995f31f0 con 0x55bb996a96b0 2023-07-14T16:21:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.865+0000 7f169d59b700 1 --2- 172.21.15.62:0/460634024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99854b20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1694011010 tx=0x7f1694010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a306436d5468002 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.865+0000 7f1683fff700 1 -- 172.21.15.62:0/460634024 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694011ab0 con 0x55bb996a96b0 2023-07-14T16:21:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.865+0000 7f1683fff700 1 -- 172.21.15.62:0/460634024 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f169402c040 con 0x55bb996a96b0 2023-07-14T16:21:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.865+0000 7f1683fff700 1 -- 172.21.15.62:0/460634024 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940335d0 con 0x55bb996a96b0 2023-07-14T16:21:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 -- 172.21.15.62:0/460634024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 msgr2=0x55bb99854b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 --2- 172.21.15.62:0/460634024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99854b20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1694011010 tx=0x7f1694010fa0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 -- 172.21.15.62:0/460634024 shutdown_connections 2023-07-14T16:21:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 --2- 172.21.15.62:0/460634024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 0x55bb99851670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 --2- 172.21.15.62:0/460634024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99854b20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 --2- 172.21.15.62:0/460634024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 0x55bb9985de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 -- 172.21.15.62:0/460634024 >> 172.21.15.62:0/460634024 conn(0x55bb99842be0 msgr2=0x55bb998555a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 -- 172.21.15.62:0/460634024 shutdown_connections 2023-07-14T16:21:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.866+0000 7f16b309cb80 1 -- 172.21.15.62:0/460634024 wait complete. 2023-07-14T16:21:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 1 -- start start 2023-07-14T16:21:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 1 Processor -- start 2023-07-14T16:21:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99859fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 0x55bb9985a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 0x55bb9985cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9985d3e0 con 0x55bb996a96b0 2023-07-14T16:21:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9985d550 con 0x55bb9984e6e0 2023-07-14T16:21:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9985d710 con 0x55bb9984d840 2023-07-14T16:21:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.867+0000 7f16b309cb80 10 client.0 _ll_register_callbacks cb 0x7f1690009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99859fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99859fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51198/0 (socket says 172.21.15.62:51198) 2023-07-14T16:21:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 -- 172.21.15.62:0/683043199 learned_addr learned my addr 172.21.15.62:0/683043199 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169cd9a700 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 0x55bb9985a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169dd9c700 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 0x55bb9985cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 -- 172.21.15.62:0/683043199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 msgr2=0x55bb9985a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 0x55bb9985a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 -- 172.21.15.62:0/683043199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 msgr2=0x55bb9985cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 0x55bb9985cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 -- 172.21.15.62:0/683043199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb995ccd80 con 0x55bb996a96b0 2023-07-14T16:21:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f169d59b700 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99859fa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1694011a00 tx=0x7f1694005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43c40da2e9db734 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.868+0000 7f1680ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.869+0000 7f1680ff9700 1 -- 172.21.15.62:0/683043199 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694033510 con 0x55bb996a96b0 2023-07-14T16:21:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.869+0000 7f16b309cb80 1 -- 172.21.15.62:0/683043199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1694015070 con 0x55bb996a96b0 2023-07-14T16:21:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.869+0000 7f1680ff9700 1 -- 172.21.15.62:0/683043199 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1694017070 con 0x55bb996a96b0 2023-07-14T16:21:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.869+0000 7f1680ff9700 1 -- 172.21.15.62:0/683043199 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1694033a40 con 0x55bb996a96b0 2023-07-14T16:21:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.869+0000 7f16b309cb80 10 client.17196 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.869+0000 7f16b309cb80 1 -- 172.21.15.62:0/683043199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb995cb250 con 0x55bb996a96b0 2023-07-14T16:21:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.869+0000 7f1680ff9700 1 -- 172.21.15.62:0/683043199 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1694015070 con 0x55bb996a96b0 2023-07-14T16:21:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.870+0000 7f1680ff9700 1 -- 172.21.15.62:0/683043199 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f169401e420 con 0x55bb996a96b0 2023-07-14T16:21:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.870+0000 7f16b309cb80 10 client.17196 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.870+0000 7f16b309cb80 10 client.17196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.870+0000 7f16b309cb80 20 client.17196 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.870+0000 7f16b309cb80 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bb998b6c70 0x55bb998b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.870+0000 7f16b309cb80 4 client.17196 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:21:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.870+0000 7f16b309cb80 1 -- 172.21.15.62:0/683043199 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bb99759ff0 con 0x55bb998b6c70 2023-07-14T16:21:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.870+0000 7f169cd9a700 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bb998b6c70 0x55bb998b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.871+0000 7f1680ff9700 10 client.17196 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.871+0000 7f169cd9a700 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bb998b6c70 0x55bb998b9080 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222b260374ed2c36 server_cookie=c9a174a6e7608c96 in_seq=0 out_seq=0 2023-07-14T16:21:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f1680ff9700 1 -- 172.21.15.62:0/683043199 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bb99759ff0 con 0x55bb998b6c70 2023-07-14T16:21:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f1680ff9700 10 client.17196 handle_command_reply: tid=0 2023-07-14T16:21:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 client.17196 shutdown 2023-07-14T16:21:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 -- 172.21.15.62:0/683043199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 msgr2=0x55bb99859fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99859fa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1694011a00 tx=0x7f1694005a00 comp rx=0 tx=0).stop 2023-07-14T16:21:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 -- 172.21.15.62:0/683043199 shutdown_connections 2023-07-14T16:21:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bb998b6c70 0x55bb998b9080 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bb9984e6e0 0x55bb9985cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bb996a96b0 0x55bb99859fa0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f1680ff9700 0 client.17196 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 --2- 172.21.15.62:0/683043199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bb9984d840 0x55bb9985a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.880+0000 7f16b309cb80 1 -- 172.21.15.62:0/683043199 >> 172.21.15.62:0/683043199 conn(0x55bb99842be0 msgr2=0x55bb99854b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.881+0000 7f16b309cb80 1 -- 172.21.15.62:0/683043199 shutdown_connections 2023-07-14T16:21:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.881+0000 7f16b309cb80 1 -- 172.21.15.62:0/683043199 wait complete. 2023-07-14T16:21:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.881+0000 7f16b309cb80 20 client.17196 trim_cache size 0 max 16384 2023-07-14T16:21:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:21:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:21:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:21:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:21:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.881+0000 7f169f7ff700 1 -- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16840a11d0 msgr2=0x7f16840a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.881+0000 7f169f7ff700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16840a11d0 0x7f16840a3680 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1694005950 tx=0x7f169400b410 comp rx=0 tx=0).stop 2023-07-14T16:21:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.881+0000 7f169f7ff700 1 -- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 msgr2=0x7f1698198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.881+0000 7f169f7ff700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 0x7f1698198f30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1690007cc0 tx=0x7f1690007cf0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.882+0000 7f169f7ff700 1 -- 172.21.15.62:0/3487034589 shutdown_connections 2023-07-14T16:21:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.882+0000 7f169f7ff700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16981056e0 0x7f1698198f30 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.882+0000 7f169f7ff700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f16840a11d0 0x7f16840a3680 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.882+0000 7f169f7ff700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1698068620 0x7f1698197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.882+0000 7f169f7ff700 1 --2- 172.21.15.62:0/3487034589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1698102bf0 0x7f16981980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.882+0000 7f169f7ff700 1 -- 172.21.15.62:0/3487034589 >> 172.21.15.62:0/3487034589 conn(0x7f16980fa000 msgr2=0x7f16980fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.882+0000 7f169f7ff700 1 -- 172.21.15.62:0/3487034589 shutdown_connections 2023-07-14T16:21:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:21:46.882+0000 7f169f7ff700 1 -- 172.21.15.62:0/3487034589 wait complete. 2023-07-14T16:21:47.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:46 smithi164 ceph-mon[97427]: pgmap v541: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:21:47.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:46 smithi062 ceph-mon[95543]: pgmap v541: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:21:47.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:46 smithi081 ceph-mon[127795]: pgmap v541: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:21:49.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:48 smithi164 ceph-mon[97427]: pgmap v542: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:21:49.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:48 smithi062 ceph-mon[95543]: pgmap v542: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:21:49.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:48 smithi081 ceph-mon[127795]: pgmap v542: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:21:51.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:50 smithi164 ceph-mon[97427]: pgmap v543: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:21:51.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:50 smithi062 ceph-mon[95543]: pgmap v543: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:21:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:50 smithi081 ceph-mon[127795]: pgmap v543: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:21:53.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:52 smithi164 ceph-mon[97427]: pgmap v544: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:21:53.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:52 smithi062 ceph-mon[95543]: pgmap v544: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:21:53.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:52 smithi081 ceph-mon[127795]: pgmap v544: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:21:55.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:54 smithi164 ceph-mon[97427]: pgmap v545: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:21:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:54 smithi062 ceph-mon[95543]: pgmap v545: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:21:55.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:54 smithi081 ceph-mon[127795]: pgmap v545: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:21:56.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:21:57.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:56 smithi164 ceph-mon[97427]: pgmap v546: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:21:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 -- 172.21.15.62:0/2814620659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244100ce0 msgr2=0x7fb244101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2814620659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244100ce0 0x7fb244101130 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb234009990 tx=0x7fb234009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 -- 172.21.15.62:0/2814620659 shutdown_connections 2023-07-14T16:21:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2814620659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb244101670 0x7fb244107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2814620659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2440ffaf0 0x7fb2440fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2814620659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244100ce0 0x7fb244101130 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 -- 172.21.15.62:0/2814620659 >> 172.21.15.62:0/2814620659 conn(0x7fb2440fb130 msgr2=0x7fb2440fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 -- 172.21.15.62:0/2814620659 shutdown_connections 2023-07-14T16:21:57.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.105+0000 7fb24bdff700 1 -- 172.21.15.62:0/2814620659 wait complete. 2023-07-14T16:21:57.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.106+0000 7fb24bdff700 1 Processor -- start 2023-07-14T16:21:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.106+0000 7fb24bdff700 1 -- start start 2023-07-14T16:21:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24bdff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2440ffaf0 0x7fb244197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24bdff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb244100ce0 0x7fb244197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24bdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101670 0x7fb24419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24bdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24419f500 con 0x7fb244101670 2023-07-14T16:21:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24bdff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24419f640 con 0x7fb2440ffaf0 2023-07-14T16:21:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24bdff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24419f7d0 con 0x7fb244100ce0 2023-07-14T16:21:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24a39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101670 0x7fb24419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24a39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101670 0x7fb24419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49082/0 (socket says 172.21.15.62:49082) 2023-07-14T16:21:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24a39c700 1 -- 172.21.15.62:0/2839010409 learned_addr learned my addr 172.21.15.62:0/2839010409 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24939a700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb244100ce0 0x7fb244197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb249b9b700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2440ffaf0 0x7fb244197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24a39c700 1 -- 172.21.15.62:0/2839010409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb244100ce0 msgr2=0x7fb244197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24a39c700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb244100ce0 0x7fb244197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24a39c700 1 -- 172.21.15.62:0/2839010409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2440ffaf0 msgr2=0x7fb244197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.107+0000 7fb24a39c700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2440ffaf0 0x7fb244197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.108+0000 7fb24a39c700 1 -- 172.21.15.62:0/2839010409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2340095f0 con 0x7fb244101670 2023-07-14T16:21:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.108+0000 7fb24a39c700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101670 0x7fb24419ef30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb23c007ee0 tx=0x7fb23c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.108+0000 7fb23affd700 1 -- 172.21.15.62:0/2839010409 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c017070 con 0x7fb244101670 2023-07-14T16:21:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.108+0000 7fb23affd700 1 -- 172.21.15.62:0/2839010409 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb23c00f040 con 0x7fb244101670 2023-07-14T16:21:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.108+0000 7fb23affd700 1 -- 172.21.15.62:0/2839010409 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c005710 con 0x7fb244101670 2023-07-14T16:21:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.108+0000 7fb24bdff700 1 -- 172.21.15.62:0/2839010409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24419fab0 con 0x7fb244101670 2023-07-14T16:21:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.108+0000 7fb24bdff700 1 -- 172.21.15.62:0/2839010409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24419ffc0 con 0x7fb244101670 2023-07-14T16:21:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.113+0000 7fb23affd700 1 -- 172.21.15.62:0/2839010409 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb23c005870 con 0x7fb244101670 2023-07-14T16:21:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.113+0000 7fb25f629b80 1 Processor -- start 2023-07-14T16:21:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.113+0000 7fb25f629b80 1 -- start start 2023-07-14T16:21:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027ac28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027ac36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ac40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb25f629b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590279c7cb0 con 0x559027ac3c70 2023-07-14T16:21:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb25f629b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027a1aa40 con 0x559027ac24d0 2023-07-14T16:21:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb25f629b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027a13ed0 con 0x559027ac2e20 2023-07-14T16:21:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb24939a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ac40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb249b9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027ac36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb24939a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ac40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49096/0 (socket says 172.21.15.62:49096) 2023-07-14T16:21:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb24939a700 1 -- 172.21.15.62:0/2947313445 learned_addr learned my addr 172.21.15.62:0/2947313445 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.114+0000 7fb24939a700 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027ac28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb249b9b700 1 -- 172.21.15.62:0/2947313445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 msgr2=0x559027ac28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb249b9b700 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027ac28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb249b9b700 1 -- 172.21.15.62:0/2947313445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 msgr2=0x559027ac40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb249b9b700 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ac40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb249b9b700 1 -- 172.21.15.62:0/2947313445 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559027841de0 con 0x559027ac2e20 2023-07-14T16:21:57.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb24939a700 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ac40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:57.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb249b9b700 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027ac36e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb240009bd0 tx=0x7fb240009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeeafb85cea60df1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb23affd700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2300a11d0 0x7fb2300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb23affd700 1 -- 172.21.15.62:0/2839010409 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb23c0cc080 con 0x7fb244101670 2023-07-14T16:21:57.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.115+0000 7fb24939a700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2300a11d0 0x7fb2300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.116+0000 7fb22ffff700 1 -- 172.21.15.62:0/2947313445 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240011b90 con 0x559027ac2e20 2023-07-14T16:21:57.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.116+0000 7fb22ffff700 1 -- 172.21.15.62:0/2947313445 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb240011cf0 con 0x559027ac2e20 2023-07-14T16:21:57.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.116+0000 7fb22ffff700 1 -- 172.21.15.62:0/2947313445 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000f6a0 con 0x559027ac2e20 2023-07-14T16:21:57.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 -- 172.21.15.62:0/2947313445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 msgr2=0x559027ac36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb24939a700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2300a11d0 0x7fb2300a3680 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb234005850 tx=0x7fb23401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027ac36e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb240009bd0 tx=0x7fb240009ee0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 -- 172.21.15.62:0/2947313445 shutdown_connections 2023-07-14T16:21:57.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ac40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027ac28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 --2- 172.21.15.62:0/2947313445 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027ac36e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 -- 172.21.15.62:0/2947313445 >> 172.21.15.62:0/2947313445 conn(0x559027ab7da0 msgr2=0x559027abd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 -- 172.21.15.62:0/2947313445 shutdown_connections 2023-07-14T16:21:57.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 -- 172.21.15.62:0/2947313445 wait complete. 2023-07-14T16:21:57.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.117+0000 7fb25f629b80 1 -- start start 2023-07-14T16:21:57.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.118+0000 7fb25f629b80 1 Processor -- start 2023-07-14T16:21:57.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.118+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027acc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.118+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027acc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.118+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ad36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.118+0000 7fb25f629b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590279b3ac0 con 0x559027ac3c70 2023-07-14T16:21:57.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.118+0000 7fb25f629b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590279b37c0 con 0x559027ac24d0 2023-07-14T16:21:57.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.118+0000 7fb25f629b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027897920 con 0x559027ac2e20 2023-07-14T16:21:57.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.118+0000 7fb25f629b80 10 client.0 _ll_register_callbacks cb 0x5590279b1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:57.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ad36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24939a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027acc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb249b9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027acc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ad36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49110/0 (socket says 172.21.15.62:49110) 2023-07-14T16:21:57.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 -- 172.21.15.62:0/4095938085 learned_addr learned my addr 172.21.15.62:0/4095938085 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 -- 172.21.15.62:0/4095938085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 msgr2=0x559027acc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027acc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 -- 172.21.15.62:0/4095938085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 msgr2=0x559027acc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027acc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 -- 172.21.15.62:0/4095938085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb234020040 con 0x559027ac3c70 2023-07-14T16:21:57.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb24a39c700 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ad36c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb23c099430 tx=0x7fb23c0990d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93236eb3aa12dbb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.119+0000 7fb22d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:57.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4095938085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0994a0 con 0x559027ac3c70 2023-07-14T16:21:57.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4095938085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb23c09d4d0 con 0x559027ac3c70 2023-07-14T16:21:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb25f629b80 1 -- 172.21.15.62:0/4095938085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55902796d4f0 con 0x559027ac3c70 2023-07-14T16:21:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4095938085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb23c0a18f0 con 0x559027ac3c70 2023-07-14T16:21:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb25f629b80 10 client.17208 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb25f629b80 1 -- 172.21.15.62:0/4095938085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb240009830 con 0x559027ac3c70 2023-07-14T16:21:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4095938085 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb23c0b5080 con 0x559027ac3c70 2023-07-14T16:21:57.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4095938085 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb23c099700 con 0x559027ac3c70 2023-07-14T16:21:57.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb25f629b80 10 client.17208 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:57.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb25f629b80 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:57.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.120+0000 7fb25f629b80 20 client.17208 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:57.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.121+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559027ad7e40 0x559027ada250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.121+0000 7fb25f629b80 4 client.17208 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:57.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.121+0000 7fb25f629b80 1 -- 172.21.15.62:0/4095938085 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559027a0ceb0 con 0x559027ad7e40 2023-07-14T16:21:57.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.121+0000 7fb249b9b700 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559027ad7e40 0x559027ada250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.122+0000 7fb22d7fa700 10 client.17208 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:57.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.122+0000 7fb249b9b700 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559027ad7e40 0x559027ada250 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d5e82635a0afb0a server_cookie=238de268e1df9379 in_seq=0 out_seq=0 2023-07-14T16:21:57.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.122+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4095938085 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559027a0ceb0 con 0x559027ad7e40 2023-07-14T16:21:57.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.122+0000 7fb22d7fa700 10 client.17208 handle_command_reply: tid=0 2023-07-14T16:21:57.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.122+0000 7fb25f629b80 1 client.17208 shutdown 2023-07-14T16:21:57.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 -- 172.21.15.62:0/4095938085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 msgr2=0x559027ad36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ad36c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb23c099430 tx=0x7fb23c0990d0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 -- 172.21.15.62:0/4095938085 shutdown_connections 2023-07-14T16:21:57.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac3c70 0x559027ad36c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559027ad7e40 0x559027ada250 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027acc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4095938085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac2e20 0x559027acc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 -- 172.21.15.62:0/4095938085 >> 172.21.15.62:0/4095938085 conn(0x559027ab7da0 msgr2=0x559027abc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb22d7fa700 0 client.17208 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:57.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 -- 172.21.15.62:0/4095938085 shutdown_connections 2023-07-14T16:21:57.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 1 -- 172.21.15.62:0/4095938085 wait complete. 2023-07-14T16:21:57.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.123+0000 7fb25f629b80 20 client.17208 trim_cache size 0 max 16384 2023-07-14T16:21:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.145+0000 7fb25f629b80 1 Processor -- start 2023-07-14T16:21:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb25f629b80 1 -- start start 2023-07-14T16:21:57.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac24d0 0x559027acae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac3c70 0x559027abc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb25f629b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027ad7c20 con 0x559027ac2e20 2023-07-14T16:21:57.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb25f629b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027ac9d90 con 0x559027ac3c70 2023-07-14T16:21:57.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb25f629b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027acb390 con 0x559027ac24d0 2023-07-14T16:21:57.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb24939a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb24939a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49112/0 (socket says 172.21.15.62:49112) 2023-07-14T16:21:57.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.146+0000 7fb24939a700 1 -- 172.21.15.62:0/778440957 learned_addr learned my addr 172.21.15.62:0/778440957 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb249b9b700 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac24d0 0x559027acae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb24a39c700 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac3c70 0x559027abc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb24939a700 1 -- 172.21.15.62:0/778440957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac24d0 msgr2=0x559027acae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb24939a700 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac24d0 0x559027acae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb24939a700 1 -- 172.21.15.62:0/778440957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac3c70 msgr2=0x559027abc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb24939a700 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac3c70 0x559027abc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb24939a700 1 -- 172.21.15.62:0/778440957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55902782ac60 con 0x559027ac2e20 2023-07-14T16:21:57.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb24a39c700 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac3c70 0x559027abc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:57.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.147+0000 7fb24939a700 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad3680 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb234015000 tx=0x7fb234009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a1c55728c1473c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:56 smithi062 ceph-mon[95543]: pgmap v546: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:21:57.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.148+0000 7fb238bff700 1 -- 172.21.15.62:0/778440957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23400be20 con 0x559027ac2e20 2023-07-14T16:21:57.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.148+0000 7fb238bff700 1 -- 172.21.15.62:0/778440957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb23400f8f0 con 0x559027ac2e20 2023-07-14T16:21:57.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.148+0000 7fb238bff700 1 -- 172.21.15.62:0/778440957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234028580 con 0x559027ac2e20 2023-07-14T16:21:57.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.148+0000 7fb25f629b80 1 -- 172.21.15.62:0/778440957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 msgr2=0x559027ad3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.148+0000 7fb25f629b80 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad3680 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb234015000 tx=0x7fb234009310 comp rx=0 tx=0).stop 2023-07-14T16:21:57.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 -- 172.21.15.62:0/778440957 shutdown_connections 2023-07-14T16:21:57.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac3c70 0x559027abc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac24d0 0x559027acae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 --2- 172.21.15.62:0/778440957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad3680 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 -- 172.21.15.62:0/778440957 >> 172.21.15.62:0/778440957 conn(0x559027ab7da0 msgr2=0x559027adebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 -- 172.21.15.62:0/778440957 shutdown_connections 2023-07-14T16:21:57.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 -- 172.21.15.62:0/778440957 wait complete. 2023-07-14T16:21:57.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 -- start start 2023-07-14T16:21:57.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.149+0000 7fb25f629b80 1 Processor -- start 2023-07-14T16:21:57.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027ad3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb25f629b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac3c70 0x559027ad0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb25f629b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027ad0c20 con 0x559027ac2e20 2023-07-14T16:21:57.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb25f629b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027b2bcf0 con 0x559027ac24d0 2023-07-14T16:21:57.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb25f629b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559027b2be80 con 0x559027ac3c70 2023-07-14T16:21:57.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb25f629b80 10 client.0 _ll_register_callbacks cb 0x7fb23c099270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:57.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24939a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24939a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49126/0 (socket says 172.21.15.62:49126) 2023-07-14T16:21:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24939a700 1 -- 172.21.15.62:0/4193545482 learned_addr learned my addr 172.21.15.62:0/4193545482 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb249b9b700 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027ad3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24a39c700 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac3c70 0x559027ad0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24939a700 1 -- 172.21.15.62:0/4193545482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac3c70 msgr2=0x559027ad0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24939a700 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac3c70 0x559027ad0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24939a700 1 -- 172.21.15.62:0/4193545482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 msgr2=0x559027ad3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24939a700 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027ad3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.150+0000 7fb24939a700 1 -- 172.21.15.62:0/4193545482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb244108860 con 0x559027ac2e20 2023-07-14T16:21:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.151+0000 7fb24939a700 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad0010 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb234009990 tx=0x7fb234004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d7eef27db1ce96 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.151+0000 7fb22d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.151+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4193545482 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23400be20 con 0x559027ac2e20 2023-07-14T16:21:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.151+0000 7fb25f629b80 1 -- 172.21.15.62:0/4193545482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb240011cf0 con 0x559027ac2e20 2023-07-14T16:21:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.151+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4193545482 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb23400f670 con 0x559027ac2e20 2023-07-14T16:21:57.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.151+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4193545482 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb234028500 con 0x559027ac2e20 2023-07-14T16:21:57.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb25f629b80 10 client.17214 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:57.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb25f629b80 1 -- 172.21.15.62:0/4193545482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559027868250 con 0x559027ac2e20 2023-07-14T16:21:57.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4193545482 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb240011cf0 con 0x559027ac2e20 2023-07-14T16:21:57.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4193545482 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb23403b070 con 0x559027ac2e20 2023-07-14T16:21:57.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb25f629b80 10 client.17214 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:57.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb25f629b80 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:57.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb25f629b80 20 client.17214 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:57.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559027b2d380 0x559027b2f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb25f629b80 4 client.17214 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:21:57.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.152+0000 7fb25f629b80 1 -- 172.21.15.62:0/4193545482 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590279cf050 con 0x559027b2d380 2023-07-14T16:21:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.153+0000 7fb249b9b700 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559027b2d380 0x559027b2f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.153+0000 7fb22d7fa700 10 client.17214 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.153+0000 7fb249b9b700 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559027b2d380 0x559027b2f790 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a8bc80973c56e99 server_cookie=2a0913a3cbad062d in_seq=0 out_seq=0 2023-07-14T16:21:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.153+0000 7fb22d7fa700 1 -- 172.21.15.62:0/4193545482 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5590279cf050 con 0x559027b2d380 2023-07-14T16:21:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.153+0000 7fb22d7fa700 10 client.17214 handle_command_reply: tid=0 2023-07-14T16:21:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.153+0000 7fb25f629b80 1 client.17214 shutdown 2023-07-14T16:21:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 -- 172.21.15.62:0/4193545482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 msgr2=0x559027ad0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad0010 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb234009990 tx=0x7fb234004220 comp rx=0 tx=0).stop 2023-07-14T16:21:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 -- 172.21.15.62:0/4193545482 shutdown_connections 2023-07-14T16:21:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559027ac3c70 0x559027ad0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559027b2d380 0x559027b2f790 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559027ac24d0 0x559027ad3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 --2- 172.21.15.62:0/4193545482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559027ac2e20 0x559027ad0010 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 -- 172.21.15.62:0/4193545482 >> 172.21.15.62:0/4193545482 conn(0x559027ab7da0 msgr2=0x559027abc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb22d7fa700 0 client.17214 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:57.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 -- 172.21.15.62:0/4193545482 shutdown_connections 2023-07-14T16:21:57.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 1 -- 172.21.15.62:0/4193545482 wait complete. 2023-07-14T16:21:57.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.154+0000 7fb25f629b80 20 client.17214 trim_cache size 0 max 16384 2023-07-14T16:21:57.138 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:21:57.139 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:21:57.139 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:21:57.139 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:21:57.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 -- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2300a11d0 msgr2=0x7fb2300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2300a11d0 0x7fb2300a3680 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb234005850 tx=0x7fb23401a040 comp rx=0 tx=0).stop 2023-07-14T16:21:57.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 -- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101670 msgr2=0x7fb24419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101670 0x7fb24419ef30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb23c007ee0 tx=0x7fb23c00d3b0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 -- 172.21.15.62:0/2839010409 shutdown_connections 2023-07-14T16:21:57.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101670 0x7fb24419ef30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb2300a11d0 0x7fb2300a3680 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2440ffaf0 0x7fb244197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 --2- 172.21.15.62:0/2839010409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb244100ce0 0x7fb244197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 -- 172.21.15.62:0/2839010409 >> 172.21.15.62:0/2839010409 conn(0x7fb2440fb130 msgr2=0x7fb2440fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 -- 172.21.15.62:0/2839010409 shutdown_connections 2023-07-14T16:21:57.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.155+0000 7fb24bdff700 1 -- 172.21.15.62:0/2839010409 wait complete. 2023-07-14T16:21:57.151 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:21:57.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:21:57.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:56 smithi081 ceph-mon[127795]: pgmap v546: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:21:57.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.403+0000 7fa1349ff700 1 -- 172.21.15.62:0/1340878658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1300ffb00 msgr2=0x7fa1300fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.403+0000 7fa1349ff700 1 --2- 172.21.15.62:0/1340878658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1300ffb00 0x7fa1300fff10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa120009990 tx=0x7fa120009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.403+0000 7fa1349ff700 1 -- 172.21.15.62:0/1340878658 shutdown_connections 2023-07-14T16:21:57.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.403+0000 7fa1349ff700 1 --2- 172.21.15.62:0/1340878658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa130101680 0x7fa130107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.403+0000 7fa1349ff700 1 --2- 172.21.15.62:0/1340878658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1300ffb00 0x7fa1300fff10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.403+0000 7fa1349ff700 1 --2- 172.21.15.62:0/1340878658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa130100cf0 0x7fa130101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.403+0000 7fa1349ff700 1 -- 172.21.15.62:0/1340878658 >> 172.21.15.62:0/1340878658 conn(0x7fa1300fb0e0 msgr2=0x7fa1300fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.404+0000 7fa1349ff700 1 -- 172.21.15.62:0/1340878658 shutdown_connections 2023-07-14T16:21:57.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.404+0000 7fa1349ff700 1 -- 172.21.15.62:0/1340878658 wait complete. 2023-07-14T16:21:57.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.405+0000 7fa1349ff700 1 Processor -- start 2023-07-14T16:21:57.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.405+0000 7fa1349ff700 1 -- start start 2023-07-14T16:21:57.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.405+0000 7fa1349ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1300ffb00 0x7fa1300742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.405+0000 7fa1349ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa130100cf0 0x7fa130072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa1349ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa130101680 0x7fa130072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa1349ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300735e0 con 0x7fa130100cf0 2023-07-14T16:21:57.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa1349ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa130073750 con 0x7fa1300ffb00 2023-07-14T16:21:57.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa1349ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13019f830 con 0x7fa130101680 2023-07-14T16:21:57.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa130100cf0 0x7fa130072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa130100cf0 0x7fa130072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49152/0 (socket says 172.21.15.62:49152) 2023-07-14T16:21:57.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12dd9b700 1 -- 172.21.15.62:0/3965286793 learned_addr learned my addr 172.21.15.62:0/3965286793 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12e59c700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1300ffb00 0x7fa1300742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12ed9d700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa130101680 0x7fa130072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12dd9b700 1 -- 172.21.15.62:0/3965286793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa130101680 msgr2=0x7fa130072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12dd9b700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa130101680 0x7fa130072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12dd9b700 1 -- 172.21.15.62:0/3965286793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1300ffb00 msgr2=0x7fa1300742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12dd9b700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1300ffb00 0x7fa1300742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12dd9b700 1 -- 172.21.15.62:0/3965286793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200095f0 con 0x7fa130100cf0 2023-07-14T16:21:57.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12ed9d700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa130101680 0x7fa130072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:21:57.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.406+0000 7fa12e59c700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1300ffb00 0x7fa1300742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:57.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.407+0000 7fa12dd9b700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa130100cf0 0x7fa130072900 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa12400e9a0 tx=0x7fa12400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.407+0000 7fa11f7fe700 1 -- 172.21.15.62:0/3965286793 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124009790 con 0x7fa130100cf0 2023-07-14T16:21:57.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.407+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13019fb10 con 0x7fa130100cf0 2023-07-14T16:21:57.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.407+0000 7fa11f7fe700 1 -- 172.21.15.62:0/3965286793 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa12400cbd0 con 0x7fa130100cf0 2023-07-14T16:21:57.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.407+0000 7fa11f7fe700 1 -- 172.21.15.62:0/3965286793 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124018680 con 0x7fa130100cf0 2023-07-14T16:21:57.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.407+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1301a0070 con 0x7fa130100cf0 2023-07-14T16:21:57.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.408+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1301070e0 con 0x7fa130100cf0 2023-07-14T16:21:57.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.411+0000 7fa11f7fe700 1 -- 172.21.15.62:0/3965286793 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa12400cd40 con 0x7fa130100cf0 2023-07-14T16:21:57.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.412+0000 7fa11f7fe700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa1180a1220 0x7fa1180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.413+0000 7fa11f7fe700 1 -- 172.21.15.62:0/3965286793 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa124014070 con 0x7fa130100cf0 2023-07-14T16:21:57.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.413+0000 7fa12e59c700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa1180a1220 0x7fa1180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.413+0000 7fa12e59c700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa1180a1220 0x7fa1180a36d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098e0 tx=0x7fa120005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.417+0000 7fa11f7fe700 1 -- 172.21.15.62:0/3965286793 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa124051790 con 0x7fa130100cf0 2023-07-14T16:21:57.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.727+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa13002c850 con 0x7fa130100cf0 2023-07-14T16:21:57.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.728+0000 7fa11f7fe700 1 -- 172.21.15.62:0/3965286793 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fa124096320 con 0x7fa130100cf0 2023-07-14T16:21:57.705 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:21:57.705 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:21:57.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa1180a1220 msgr2=0x7fa1180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa1180a1220 0x7fa1180a36d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098e0 tx=0x7fa120005fd0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa130100cf0 msgr2=0x7fa130072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa130100cf0 0x7fa130072900 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa12400e9a0 tx=0x7fa12400ed60 comp rx=0 tx=0).stop 2023-07-14T16:21:57.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 shutdown_connections 2023-07-14T16:21:57.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa130101680 0x7fa130072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa1180a1220 0x7fa1180a36d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1300ffb00 0x7fa1300742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 --2- 172.21.15.62:0/3965286793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa130100cf0 0x7fa130072900 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 >> 172.21.15.62:0/3965286793 conn(0x7fa1300fb0e0 msgr2=0x7fa130104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 shutdown_connections 2023-07-14T16:21:57.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.730+0000 7fa1349ff700 1 -- 172.21.15.62:0/3965286793 wait complete. 2023-07-14T16:21:57.711 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:21:57.728 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:21:57.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 -- 172.21.15.62:0/310134758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 msgr2=0x7fdaac073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 --2- 172.21.15.62:0/310134758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 0x7fdaac073360 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fda98009990 tx=0x7fda98009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 -- 172.21.15.62:0/310134758 shutdown_connections 2023-07-14T16:21:57.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 --2- 172.21.15.62:0/310134758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaac1079a0 0x7fdaac109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 --2- 172.21.15.62:0/310134758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaac0744d0 0x7fdaac072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 --2- 172.21.15.62:0/310134758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 0x7fdaac073360 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 -- 172.21.15.62:0/310134758 >> 172.21.15.62:0/310134758 conn(0x7fdaac0fb130 msgr2=0x7fdaac0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 -- 172.21.15.62:0/310134758 shutdown_connections 2023-07-14T16:21:57.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.978+0000 7fdab31ff700 1 -- 172.21.15.62:0/310134758 wait complete. 2023-07-14T16:21:57.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.980+0000 7fdab31ff700 1 Processor -- start 2023-07-14T16:21:57.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.980+0000 7fdab31ff700 1 -- start start 2023-07-14T16:21:57.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.980+0000 7fdab31ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 0x7fdaac197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab31ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaac0744d0 0x7fdaac197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab31ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaac1079a0 0x7fdaac19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab31ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac19f4d0 con 0x7fdaac072ef0 2023-07-14T16:21:57.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab31ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac19f610 con 0x7fdaac1079a0 2023-07-14T16:21:57.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab31ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac19f7a0 con 0x7fdaac0744d0 2023-07-14T16:21:57.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab0f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 0x7fdaac197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab0f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 0x7fdaac197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49174/0 (socket says 172.21.15.62:49174) 2023-07-14T16:21:57.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab0f9b700 1 -- 172.21.15.62:0/2644106453 learned_addr learned my addr 172.21.15.62:0/2644106453 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab179c700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaac1079a0 0x7fdaac19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaac0744d0 0x7fdaac197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab0f9b700 1 -- 172.21.15.62:0/2644106453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaac0744d0 msgr2=0x7fdaac197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaac0744d0 0x7fdaac197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab0f9b700 1 -- 172.21.15.62:0/2644106453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaac1079a0 msgr2=0x7fdaac19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaac1079a0 0x7fdaac19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdab0f9b700 1 -- 172.21.15.62:0/2644106453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda980095f0 con 0x7fdaac072ef0 2023-07-14T16:21:57.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.981+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaac0744d0 0x7fdaac197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:57.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.982+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 0x7fdaac197950 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e9a0 tx=0x7fdaa000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.982+0000 7fdaa9ffb700 1 -- 172.21.15.62:0/2644106453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0009790 con 0x7fdaac072ef0 2023-07-14T16:21:57.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.982+0000 7fdaa9ffb700 1 -- 172.21.15.62:0/2644106453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdaa000cbd0 con 0x7fdaac072ef0 2023-07-14T16:21:57.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.982+0000 7fdab31ff700 1 -- 172.21.15.62:0/2644106453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac19fa80 con 0x7fdaac072ef0 2023-07-14T16:21:57.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.982+0000 7fdaa9ffb700 1 -- 172.21.15.62:0/2644106453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0018680 con 0x7fdaac072ef0 2023-07-14T16:21:57.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.982+0000 7fdab31ff700 1 -- 172.21.15.62:0/2644106453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac19ffe0 con 0x7fdaac072ef0 2023-07-14T16:21:57.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.986+0000 7fdaa9ffb700 1 -- 172.21.15.62:0/2644106453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdaa000cd40 con 0x7fdaac072ef0 2023-07-14T16:21:57.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.987+0000 7fdac6b3eb80 1 Processor -- start 2023-07-14T16:21:57.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.987+0000 7fdac6b3eb80 1 -- start start 2023-07-14T16:21:57.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.987+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e734d0 0x558e02e738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.987+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.987+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.987+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02d78cb0 con 0x558e02e74c70 2023-07-14T16:21:57.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.987+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02dcba40 con 0x558e02e734d0 2023-07-14T16:21:57.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.987+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02dc4ed0 con 0x558e02e73e20 2023-07-14T16:21:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdab179c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e734d0 0x558e02e738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdab179c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e734d0 0x558e02e738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57118/0 (socket says 172.21.15.62:57118) 2023-07-14T16:21:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdab179c700 1 -- 172.21.15.62:0/1080109454 learned_addr learned my addr 172.21.15.62:0/1080109454 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdaa9ffb700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda9c0a11d0 0x7fda9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdaabfff700 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdaa9ffb700 1 -- 172.21.15.62:0/2644106453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdaa0014070 con 0x7fdaac072ef0 2023-07-14T16:21:57.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdab179c700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda9c0a11d0 0x7fda9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdaabfff700 1 -- 172.21.15.62:0/1080109454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 msgr2=0x558e02e746e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.988+0000 7fdaabfff700 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e746e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.989+0000 7fdaabfff700 1 -- 172.21.15.62:0/1080109454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e734d0 msgr2=0x558e02e738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.989+0000 7fdaabfff700 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e734d0 0x558e02e738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.989+0000 7fdaabfff700 1 -- 172.21.15.62:0/1080109454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e02bf2de0 con 0x558e02e74c70 2023-07-14T16:21:57.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.989+0000 7fdaabfff700 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e750e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fda980098e0 tx=0x7fda98005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39ca7c0b5235848 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.989+0000 7fdab179c700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda9c0a11d0 0x7fda9c0a3680 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4003fd0 tx=0x7fdaa4004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.989+0000 7fda96ffd700 1 -- 172.21.15.62:0/1080109454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda980052c0 con 0x558e02e74c70 2023-07-14T16:21:57.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.989+0000 7fda96ffd700 1 -- 172.21.15.62:0/1080109454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda9801e040 con 0x558e02e74c70 2023-07-14T16:21:57.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.989+0000 7fda96ffd700 1 -- 172.21.15.62:0/1080109454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9800b970 con 0x558e02e74c70 2023-07-14T16:21:57.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/1080109454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 msgr2=0x558e02e750e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e750e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fda980098e0 tx=0x7fda98005c00 comp rx=0 tx=0).stop 2023-07-14T16:21:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/1080109454 shutdown_connections 2023-07-14T16:21:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e750e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e734d0 0x558e02e738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/1080109454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/1080109454 >> 172.21.15.62:0/1080109454 conn(0x558e02e68da0 msgr2=0x558e02e6e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/1080109454 shutdown_connections 2023-07-14T16:21:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.990+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/1080109454 wait complete. 2023-07-14T16:21:57.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.991+0000 7fdac6b3eb80 1 -- start start 2023-07-14T16:21:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.991+0000 7fdac6b3eb80 1 Processor -- start 2023-07-14T16:21:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e734d0 0x558e02e7d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e7d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 0x558e02e84620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02d64ac0 con 0x558e02e734d0 2023-07-14T16:21:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02d647c0 con 0x558e02e74c70 2023-07-14T16:21:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02c48920 con 0x558e02e73e20 2023-07-14T16:21:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdac6b3eb80 10 client.0 _ll_register_callbacks cb 0x558e02d62760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab0f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e734d0 0x558e02e7d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab0f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e734d0 0x558e02e7d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49182/0 (socket says 172.21.15.62:49182) 2023-07-14T16:21:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab0f9b700 1 -- 172.21.15.62:0/2518358869 learned_addr learned my addr 172.21.15.62:0/2518358869 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e7d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab179c700 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 0x558e02e84620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab0f9b700 1 -- 172.21.15.62:0/2518358869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 msgr2=0x558e02e7d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e7d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab0f9b700 1 -- 172.21.15.62:0/2518358869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 msgr2=0x558e02e84620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 0x558e02e84620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdab0f9b700 1 -- 172.21.15.62:0/2518358869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac10aa20 con 0x558e02e734d0 2023-07-14T16:21:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.992+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e7d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e734d0 0x558e02e7d0a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0010a10 tx=0x7fdaa009a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c71c85c726ca5a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fda8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fda8bfff700 1 -- 172.21.15.62:0/2518358869 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00a2100 con 0x558e02e734d0 2023-07-14T16:21:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fda8bfff700 1 -- 172.21.15.62:0/2518358869 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdaa00cee40 con 0x558e02e734d0 2023-07-14T16:21:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2518358869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e02d1e4f0 con 0x558e02e734d0 2023-07-14T16:21:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fda8bfff700 1 -- 172.21.15.62:0/2518358869 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa00a2390 con 0x558e02e734d0 2023-07-14T16:21:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fdac6b3eb80 10 client.17232 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2518358869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558e02bdbc60 con 0x558e02e734d0 2023-07-14T16:21:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.993+0000 7fda8bfff700 1 -- 172.21.15.62:0/2518358869 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558e02d1e4f0 con 0x558e02e734d0 2023-07-14T16:21:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.994+0000 7fda8bfff700 1 -- 172.21.15.62:0/2518358869 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdaa00cac80 con 0x558e02e734d0 2023-07-14T16:21:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.994+0000 7fdac6b3eb80 10 client.17232 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.994+0000 7fdac6b3eb80 10 client.17232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:21:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.994+0000 7fdac6b3eb80 20 client.17232 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.994+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558e02e88d70 0x558e02e8b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.994+0000 7fdac6b3eb80 4 client.17232 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.994+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2518358869 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e02dbdeb0 con 0x558e02e88d70 2023-07-14T16:21:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.994+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558e02e88d70 0x558e02e8b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.995+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558e02e88d70 0x558e02e8b180 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=627f9859c3d983a6 server_cookie=98eaf615efdaccc2 in_seq=0 out_seq=0 2023-07-14T16:21:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.995+0000 7fda8bfff700 10 client.17232 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:21:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fda8bfff700 1 -- 172.21.15.62:0/2518358869 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e02dbdeb0 con 0x558e02e88d70 2023-07-14T16:21:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fda8bfff700 10 client.17232 handle_command_reply: tid=0 2023-07-14T16:21:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 client.17232 shutdown 2023-07-14T16:21:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2518358869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e734d0 msgr2=0x558e02e7d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e734d0 0x558e02e7d0a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0010a10 tx=0x7fdaa009a480 comp rx=0 tx=0).stop 2023-07-14T16:21:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2518358869 shutdown_connections 2023-07-14T16:21:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558e02e88d70 0x558e02e8b180 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 0x558e02e84620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e734d0 0x558e02e7d0a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2518358869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e73e20 0x558e02e7d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2518358869 >> 172.21.15.62:0/2518358869 conn(0x558e02e68da0 msgr2=0x558e02e6d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fda8bfff700 0 client.17232 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:21:57.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2518358869 shutdown_connections 2023-07-14T16:21:57.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2518358869 wait complete. 2023-07-14T16:21:57.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:57.996+0000 7fdac6b3eb80 20 client.17232 trim_cache size 0 max 16384 2023-07-14T16:21:57.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdac6b3eb80 1 Processor -- start 2023-07-14T16:21:57.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdac6b3eb80 1 -- start start 2023-07-14T16:21:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 0x558e02e77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e73e20 0x558e02e845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 0x558e02e780b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02e6f2e0 con 0x558e02e73e20 2023-07-14T16:21:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02e65ef0 con 0x558e02e74c70 2023-07-14T16:21:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02e6f610 con 0x558e02e734d0 2023-07-14T16:21:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdaabfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e73e20 0x558e02e845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdaabfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e73e20 0x558e02e845e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49196/0 (socket says 172.21.15.62:49196) 2023-07-14T16:21:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.014+0000 7fdaabfff700 1 -- 172.21.15.62:0/2891905661 learned_addr learned my addr 172.21.15.62:0/2891905661 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 0x558e02e77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fdab179c700 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 0x558e02e780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fdaabfff700 1 -- 172.21.15.62:0/2891905661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 msgr2=0x558e02e77b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 0x558e02e77b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fdaabfff700 1 -- 172.21.15.62:0/2891905661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 msgr2=0x558e02e780b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 0x558e02e780b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fdaabfff700 1 -- 172.21.15.62:0/2891905661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e02e75560 con 0x558e02e73e20 2023-07-14T16:21:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fdaabfff700 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e73e20 0x558e02e845e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fda98004360 tx=0x7fda98005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13268f89c4997130 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.015+0000 7fda977fe700 1 -- 172.21.15.62:0/2891905661 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98035070 con 0x558e02e73e20 2023-07-14T16:21:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fda977fe700 1 -- 172.21.15.62:0/2891905661 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda980050c0 con 0x558e02e73e20 2023-07-14T16:21:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fda977fe700 1 -- 172.21.15.62:0/2891905661 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98003b00 con 0x558e02e73e20 2023-07-14T16:21:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2891905661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e73e20 msgr2=0x558e02e845e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e73e20 0x558e02e845e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fda98004360 tx=0x7fda98005aa0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2891905661 shutdown_connections 2023-07-14T16:21:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e74c70 0x558e02e780b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 0x558e02e77b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/2891905661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e73e20 0x558e02e845e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2891905661 >> 172.21.15.62:0/2891905661 conn(0x558e02e68da0 msgr2=0x558e02e7b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.016+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2891905661 shutdown_connections 2023-07-14T16:21:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.017+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/2891905661 wait complete. 2023-07-14T16:21:57.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.017+0000 7fdac6b3eb80 1 -- start start 2023-07-14T16:21:58.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.017+0000 7fdac6b3eb80 1 Processor -- start 2023-07-14T16:21:58.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 0x558e02e80810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e73e20 0x558e02e80d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdac6b3eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e83bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02e84130 con 0x558e02e74c70 2023-07-14T16:21:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02e842a0 con 0x558e02e73e20 2023-07-14T16:21:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdac6b3eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e02e84460 con 0x558e02e734d0 2023-07-14T16:21:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdac6b3eb80 10 client.0 _ll_register_callbacks cb 0x7fdaa0099b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdab179c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e83bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdab179c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e83bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49206/0 (socket says 172.21.15.62:49206) 2023-07-14T16:21:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdab179c700 1 -- 172.21.15.62:0/389751786 learned_addr learned my addr 172.21.15.62:0/389751786 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdaabfff700 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e73e20 0x558e02e80d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.018+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 0x558e02e80810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fdab179c700 1 -- 172.21.15.62:0/389751786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 msgr2=0x558e02e80810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fdab179c700 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 0x558e02e80810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fdab179c700 1 -- 172.21.15.62:0/389751786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e73e20 msgr2=0x558e02e80d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fdab179c700 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e73e20 0x558e02e80d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fdab179c700 1 -- 172.21.15.62:0/389751786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e02bf2de0 con 0x558e02e74c70 2023-07-14T16:21:58.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fdab179c700 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e83bf0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdaa401a670 tx=0x7fdaa4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e75b1a927ef88ce server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fda8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:58.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fda8bfff700 1 -- 172.21.15.62:0/389751786 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa401ae10 con 0x558e02e74c70 2023-07-14T16:21:58.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fda8bfff700 1 -- 172.21.15.62:0/389751786 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdaa4004bb0 con 0x558e02e74c70 2023-07-14T16:21:58.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.019+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/389751786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda9801e040 con 0x558e02e74c70 2023-07-14T16:21:58.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fda8bfff700 1 -- 172.21.15.62:0/389751786 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa4004d70 con 0x558e02e74c70 2023-07-14T16:21:58.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fdac6b3eb80 10 client.17238 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/389751786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558e02c19250 con 0x558e02e74c70 2023-07-14T16:21:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fda8bfff700 1 -- 172.21.15.62:0/389751786 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdaa4037020 con 0x558e02e74c70 2023-07-14T16:21:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fda8bfff700 1 -- 172.21.15.62:0/389751786 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdaa4031610 con 0x558e02e74c70 2023-07-14T16:21:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fdac6b3eb80 10 client.17238 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fdac6b3eb80 10 client.17238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:21:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fdac6b3eb80 20 client.17238 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558e02eddfe0 0x558e02ee03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fdac6b3eb80 4 client.17238 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:21:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.020+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/389751786 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e02d80050 con 0x558e02eddfe0 2023-07-14T16:21:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.021+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558e02eddfe0 0x558e02ee03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.021+0000 7fda8bfff700 10 client.17238 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:21:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.021+0000 7fdab0f9b700 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558e02eddfe0 0x558e02ee03f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=282448762b920095 server_cookie=6c49e4991d58168c in_seq=0 out_seq=0 2023-07-14T16:21:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.021+0000 7fda8bfff700 1 -- 172.21.15.62:0/389751786 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e02d80050 con 0x558e02eddfe0 2023-07-14T16:21:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.021+0000 7fda8bfff700 10 client.17238 handle_command_reply: tid=0 2023-07-14T16:21:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.021+0000 7fdac6b3eb80 1 client.17238 shutdown 2023-07-14T16:21:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/389751786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 msgr2=0x558e02e83bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e83bf0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdaa401a670 tx=0x7fdaa4004ad0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/389751786 shutdown_connections 2023-07-14T16:21:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558e02eddfe0 0x558e02ee03f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558e02e74c70 0x558e02e83bf0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558e02e734d0 0x558e02e80810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 --2- 172.21.15.62:0/389751786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e02e73e20 0x558e02e80d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/389751786 >> 172.21.15.62:0/389751786 conn(0x558e02e68da0 msgr2=0x558e02e77b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fda8bfff700 0 client.17238 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:21:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/389751786 shutdown_connections 2023-07-14T16:21:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 1 -- 172.21.15.62:0/389751786 wait complete. 2023-07-14T16:21:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.022+0000 7fdac6b3eb80 20 client.17238 trim_cache size 0 max 16384 2023-07-14T16:21:58.008 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:21:58.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 -- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda9c0a11d0 msgr2=0x7fda9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda9c0a11d0 0x7fda9c0a3680 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4003fd0 tx=0x7fdaa4004b40 comp rx=0 tx=0).stop 2023-07-14T16:21:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 -- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 msgr2=0x7fdaac197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 0x7fdaac197950 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e9a0 tx=0x7fdaa000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 -- 172.21.15.62:0/2644106453 shutdown_connections 2023-07-14T16:21:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda9c0a11d0 0x7fda9c0a3680 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaac1079a0 0x7fdaac19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaac072ef0 0x7fdaac197950 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 --2- 172.21.15.62:0/2644106453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdaac0744d0 0x7fdaac197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 -- 172.21.15.62:0/2644106453 >> 172.21.15.62:0/2644106453 conn(0x7fdaac0fb130 msgr2=0x7fdaac0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 -- 172.21.15.62:0/2644106453 shutdown_connections 2023-07-14T16:21:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.023+0000 7fdab31ff700 1 -- 172.21.15.62:0/2644106453 wait complete. 2023-07-14T16:21:58.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:57 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3965286793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:21:58.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:21:58.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:57 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3965286793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:21:58.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:57 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3965286793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:21:58.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 -- 172.21.15.62:0/1983898558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18600ffae0 msgr2=0x7f18600ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 --2- 172.21.15.62:0/1983898558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18600ffae0 0x7f18600ffef0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1850009990 tx=0x7f1850009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 -- 172.21.15.62:0/1983898558 shutdown_connections 2023-07-14T16:21:58.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 --2- 172.21.15.62:0/1983898558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1860101660 0x7f1860107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 --2- 172.21.15.62:0/1983898558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18600ffae0 0x7f18600ffef0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 --2- 172.21.15.62:0/1983898558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1860100cd0 0x7f1860101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 -- 172.21.15.62:0/1983898558 >> 172.21.15.62:0/1983898558 conn(0x7f18600fb0e0 msgr2=0x7f18600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 -- 172.21.15.62:0/1983898558 shutdown_connections 2023-07-14T16:21:58.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.275+0000 7f18655ff700 1 -- 172.21.15.62:0/1983898558 wait complete. 2023-07-14T16:21:58.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.277+0000 7f18655ff700 1 Processor -- start 2023-07-14T16:21:58.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.277+0000 7f18655ff700 1 -- start start 2023-07-14T16:21:58.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.277+0000 7f18655ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18600ffae0 0x7f18601935a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.277+0000 7f18655ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1860100cd0 0x7f1860193ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.277+0000 7f18655ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1860101660 0x7f186019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f18655ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019b120 con 0x7f1860101660 2023-07-14T16:21:58.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f18655ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019b260 con 0x7f1860100cd0 2023-07-14T16:21:58.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f18655ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019b3f0 con 0x7f18600ffae0 2023-07-14T16:21:58.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f185f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1860101660 0x7f186019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f185effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18600ffae0 0x7f18601935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f185f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1860101660 0x7f186019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49220/0 (socket says 172.21.15.62:49220) 2023-07-14T16:21:58.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f185f7fe700 1 -- 172.21.15.62:0/1687432487 learned_addr learned my addr 172.21.15.62:0/1687432487 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f185effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18600ffae0 0x7f18601935a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44142/0 (socket says 172.21.15.62:44142) 2023-07-14T16:21:58.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.278+0000 7f185e7fc700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1860100cd0 0x7f1860193ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.279+0000 7f185f7fe700 1 -- 172.21.15.62:0/1687432487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18600ffae0 msgr2=0x7f18601935a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.279+0000 7f185f7fe700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18600ffae0 0x7f18601935a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.279+0000 7f185f7fe700 1 -- 172.21.15.62:0/1687432487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1860100cd0 msgr2=0x7f1860193ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.279+0000 7f185f7fe700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1860100cd0 0x7f1860193ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.279+0000 7f185f7fe700 1 -- 172.21.15.62:0/1687432487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18500095f0 con 0x7f1860101660 2023-07-14T16:21:58.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.279+0000 7f185f7fe700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1860101660 0x7f186019ab50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1858007ee0 tx=0x7f185800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.280+0000 7f1847fff700 1 -- 172.21.15.62:0/1687432487 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800dd20 con 0x7f1860101660 2023-07-14T16:21:58.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.280+0000 7f1847fff700 1 -- 172.21.15.62:0/1687432487 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f185800f040 con 0x7f1860101660 2023-07-14T16:21:58.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.280+0000 7f1847fff700 1 -- 172.21.15.62:0/1687432487 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858004750 con 0x7f1860101660 2023-07-14T16:21:58.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.280+0000 7f18655ff700 1 -- 172.21.15.62:0/1687432487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186019b6d0 con 0x7f1860101660 2023-07-14T16:21:58.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.280+0000 7f18655ff700 1 -- 172.21.15.62:0/1687432487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186019bb90 con 0x7f1860101660 2023-07-14T16:21:58.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.285+0000 7f1847fff700 1 -- 172.21.15.62:0/1687432487 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1858003a00 con 0x7f1860101660 2023-07-14T16:21:58.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.286+0000 7f1878ef1b80 1 Processor -- start 2023-07-14T16:21:58.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.286+0000 7f1878ef1b80 1 -- start start 2023-07-14T16:21:58.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.286+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56b66cb0 con 0x561c56c61e20 2023-07-14T16:21:58.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56bb9a40 con 0x561c56c62c70 2023-07-14T16:21:58.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56bb2ed0 con 0x561c56c614d0 2023-07-14T16:21:58.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f185effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f185e7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f185effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c626e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49232/0 (socket says 172.21.15.62:49232) 2023-07-14T16:21:58.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f185effd700 1 -- 172.21.15.62:0/802903611 learned_addr learned my addr 172.21.15.62:0/802903611 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f185e7fc700 1 --2- 172.21.15.62:0/802903611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.287+0000 7f1847fff700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18480a11d0 0x7f18480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f185effd700 1 -- 172.21.15.62:0/802903611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 msgr2=0x561c56c618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f1847fff700 1 -- 172.21.15.62:0/1687432487 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18580cc120 con 0x7f1860101660 2023-07-14T16:21:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f185effd700 1 --2- 172.21.15.62:0/802903611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f185effd700 1 -- 172.21.15.62:0/802903611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 msgr2=0x561c56c630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f185effd700 1 --2- 172.21.15.62:0/802903611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f185effd700 1 -- 172.21.15.62:0/802903611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c569e0de0 con 0x561c56c61e20 2023-07-14T16:21:58.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f185f7fe700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18480a11d0 0x7f18480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f185effd700 1 --2- 172.21.15.62:0/802903611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c626e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1850009990 tx=0x7f1850005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fea707031637ba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f185f7fe700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18480a11d0 0x7f18480a3680 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1858004b80 tx=0x7f1858004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f18457fa700 1 -- 172.21.15.62:0/802903611 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850010800 con 0x561c56c61e20 2023-07-14T16:21:58.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.288+0000 7f18457fa700 1 -- 172.21.15.62:0/802903611 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1850003e90 con 0x561c56c61e20 2023-07-14T16:21:58.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f18457fa700 1 -- 172.21.15.62:0/802903611 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850005610 con 0x561c56c61e20 2023-07-14T16:21:58.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f1878ef1b80 1 -- 172.21.15.62:0/802903611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 msgr2=0x561c56c626e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/802903611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c626e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1850009990 tx=0x7f1850005c40 comp rx=0 tx=0).stop 2023-07-14T16:21:58.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f1878ef1b80 1 -- 172.21.15.62:0/802903611 shutdown_connections 2023-07-14T16:21:58.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/802903611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/802903611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/802903611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c626e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f1878ef1b80 1 -- 172.21.15.62:0/802903611 >> 172.21.15.62:0/802903611 conn(0x561c56c56da0 msgr2=0x561c56c5c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.289+0000 7f1878ef1b80 1 -- 172.21.15.62:0/802903611 shutdown_connections 2023-07-14T16:21:58.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.290+0000 7f1878ef1b80 1 -- 172.21.15.62:0/802903611 wait complete. 2023-07-14T16:21:58.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.290+0000 7f1878ef1b80 1 -- start start 2023-07-14T16:21:58.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f1878ef1b80 1 Processor -- start 2023-07-14T16:21:58.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c72650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56b52ac0 con 0x561c56c61e20 2023-07-14T16:21:58.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56b527c0 con 0x561c56c62c70 2023-07-14T16:21:58.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56a36920 con 0x561c56c614d0 2023-07-14T16:21:58.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f1878ef1b80 10 client.0 _ll_register_callbacks cb 0x561c56b50760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:58.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.291+0000 7f185effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49234/0 (socket says 172.21.15.62:49234) 2023-07-14T16:21:58.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185effd700 1 -- 172.21.15.62:0/745457699 learned_addr learned my addr 172.21.15.62:0/745457699 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185e7fc700 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185e7fc700 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c72650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185effd700 1 -- 172.21.15.62:0/745457699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 msgr2=0x561c56c6b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185effd700 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185effd700 1 -- 172.21.15.62:0/745457699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 msgr2=0x561c56c72650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185effd700 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c72650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.292+0000 7f185effd700 1 -- 172.21.15.62:0/745457699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1860108850 con 0x561c56c61e20 2023-07-14T16:21:58.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.293+0000 7f185effd700 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6b610 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1850005cd0 tx=0x7f1850003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73fdc1e261d0c12 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.293+0000 7f1842ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:58.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.293+0000 7f1842ff5700 1 -- 172.21.15.62:0/745457699 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500055a0 con 0x561c56c61e20 2023-07-14T16:21:58.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.293+0000 7f1842ff5700 1 -- 172.21.15.62:0/745457699 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1850018430 con 0x561c56c61e20 2023-07-14T16:21:58.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.293+0000 7f1878ef1b80 1 -- 172.21.15.62:0/745457699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c56b0c4f0 con 0x561c56c61e20 2023-07-14T16:21:58.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.293+0000 7f1842ff5700 1 -- 172.21.15.62:0/745457699 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1850005700 con 0x561c56c61e20 2023-07-14T16:21:58.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.293+0000 7f1878ef1b80 10 client.17250 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:58.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.293+0000 7f1878ef1b80 1 -- 172.21.15.62:0/745457699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561c569c9c60 con 0x561c56c61e20 2023-07-14T16:21:58.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f1842ff5700 1 -- 172.21.15.62:0/745457699 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561c56b0c4f0 con 0x561c56c61e20 2023-07-14T16:21:58.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f1842ff5700 1 -- 172.21.15.62:0/745457699 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1850036440 con 0x561c56c61e20 2023-07-14T16:21:58.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f1878ef1b80 10 client.17250 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:58.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f1878ef1b80 10 client.17250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:21:58.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f1878ef1b80 20 client.17250 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:58.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561c56c76ae0 0x561c56c78ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f1878ef1b80 4 client.17250 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:58.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f1878ef1b80 1 -- 172.21.15.62:0/745457699 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c56babeb0 con 0x561c56c76ae0 2023-07-14T16:21:58.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.294+0000 7f185e7fc700 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561c56c76ae0 0x561c56c78ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.295+0000 7f1842ff5700 10 client.17250 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:21:58.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.295+0000 7f185e7fc700 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561c56c76ae0 0x561c56c78ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a075d2c6aced32a9 server_cookie=a456fd9784ec8a0e in_seq=0 out_seq=0 2023-07-14T16:21:58.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1842ff5700 1 -- 172.21.15.62:0/745457699 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c56babeb0 con 0x561c56c76ae0 2023-07-14T16:21:58.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1842ff5700 10 client.17250 handle_command_reply: tid=0 2023-07-14T16:21:58.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 client.17250 shutdown 2023-07-14T16:21:58.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 -- 172.21.15.62:0/745457699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 msgr2=0x561c56c6b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6b610 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1850005cd0 tx=0x7f1850003b80 comp rx=0 tx=0).stop 2023-07-14T16:21:58.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 -- 172.21.15.62:0/745457699 shutdown_connections 2023-07-14T16:21:58.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561c56c76ae0 0x561c56c78ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c72650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/745457699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6b610 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1842ff5700 0 client.17250 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:21:58.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 -- 172.21.15.62:0/745457699 >> 172.21.15.62:0/745457699 conn(0x561c56c56da0 msgr2=0x561c56c5b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 -- 172.21.15.62:0/745457699 shutdown_connections 2023-07-14T16:21:58.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.296+0000 7f1878ef1b80 1 -- 172.21.15.62:0/745457699 wait complete. 2023-07-14T16:21:58.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.297+0000 7f1878ef1b80 20 client.17250 trim_cache size 0 max 16384 2023-07-14T16:21:58.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.311+0000 7f1878ef1b80 1 Processor -- start 2023-07-14T16:21:58.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f1878ef1b80 1 -- start start 2023-07-14T16:21:58.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c72610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c69020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56c5da80 con 0x561c56c61e20 2023-07-14T16:21:58.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56c68dd0 con 0x561c56c62c70 2023-07-14T16:21:58.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56c6c220 con 0x561c56c614d0 2023-07-14T16:21:58.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f185effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c72610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f185effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c72610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49242/0 (socket says 172.21.15.62:49242) 2023-07-14T16:21:58.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f185effd700 1 -- 172.21.15.62:0/2319406427 learned_addr learned my addr 172.21.15.62:0/2319406427 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.312+0000 7f185e7fc700 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c69020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f185e7fc700 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f185effd700 1 -- 172.21.15.62:0/2319406427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 msgr2=0x561c56c69df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f185effd700 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c69df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f185effd700 1 -- 172.21.15.62:0/2319406427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 msgr2=0x561c56c69020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f185effd700 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c69020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f185effd700 1 -- 172.21.15.62:0/2319406427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c56a07250 con 0x561c56c61e20 2023-07-14T16:21:58.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f185e7fc700 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:58.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f185effd700 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c72610 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1850009990 tx=0x7f1850012fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da55d9db0b4f7ac0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.313+0000 7f1845ffb700 1 -- 172.21.15.62:0/2319406427 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850009290 con 0x561c56c61e20 2023-07-14T16:21:58.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1845ffb700 1 -- 172.21.15.62:0/2319406427 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1850036ac0 con 0x561c56c61e20 2023-07-14T16:21:58.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1845ffb700 1 -- 172.21.15.62:0/2319406427 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850018850 con 0x561c56c61e20 2023-07-14T16:21:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 -- 172.21.15.62:0/2319406427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 msgr2=0x561c56c72610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c72610 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1850009990 tx=0x7f1850012fe0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 -- 172.21.15.62:0/2319406427 shutdown_connections 2023-07-14T16:21:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c69020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/2319406427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c72610 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 -- 172.21.15.62:0/2319406427 >> 172.21.15.62:0/2319406427 conn(0x561c56c56da0 msgr2=0x561c56c7d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 -- 172.21.15.62:0/2319406427 shutdown_connections 2023-07-14T16:21:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.314+0000 7f1878ef1b80 1 -- 172.21.15.62:0/2319406427 wait complete. 2023-07-14T16:21:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.315+0000 7f1878ef1b80 1 -- start start 2023-07-14T16:21:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.315+0000 7f1878ef1b80 1 Processor -- start 2023-07-14T16:21:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.315+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.315+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.315+0000 7f1878ef1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c71ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.315+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56c721e0 con 0x561c56c61e20 2023-07-14T16:21:58.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.315+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56c72350 con 0x561c56c62c70 2023-07-14T16:21:58.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.315+0000 7f1878ef1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c56c72510 con 0x561c56c614d0 2023-07-14T16:21:58.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f1878ef1b80 10 client.0 _ll_register_callbacks cb 0x7f1854009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:58.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185e7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49252/0 (socket says 172.21.15.62:49252) 2023-07-14T16:21:58.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185effd700 1 -- 172.21.15.62:0/193442788 learned_addr learned my addr 172.21.15.62:0/193442788 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185e7fc700 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c71ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185effd700 1 -- 172.21.15.62:0/193442788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 msgr2=0x561c56c6e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185effd700 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185effd700 1 -- 172.21.15.62:0/193442788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 msgr2=0x561c56c71ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185effd700 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c71ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185effd700 1 -- 172.21.15.62:0/193442788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c569c9c60 con 0x561c56c61e20 2023-07-14T16:21:58.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.316+0000 7f185e7fc700 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.317+0000 7f185effd700 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6ed80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f18500368f0 tx=0x7f1850005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6f69419913a22a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.317+0000 7f1842ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.317+0000 7f1842ff5700 1 -- 172.21.15.62:0/193442788 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850009290 con 0x561c56c61e20 2023-07-14T16:21:58.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.317+0000 7f1878ef1b80 1 -- 172.21.15.62:0/193442788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1850003e90 con 0x561c56c61e20 2023-07-14T16:21:58.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.317+0000 7f1842ff5700 1 -- 172.21.15.62:0/193442788 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1850000e40 con 0x561c56c61e20 2023-07-14T16:21:58.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.317+0000 7f1842ff5700 1 -- 172.21.15.62:0/193442788 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1850010b10 con 0x561c56c61e20 2023-07-14T16:21:58.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.317+0000 7f1878ef1b80 10 client.17256 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:58.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.317+0000 7f1878ef1b80 1 -- 172.21.15.62:0/193442788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561c569df2b0 con 0x561c56c61e20 2023-07-14T16:21:58.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f1842ff5700 1 -- 172.21.15.62:0/193442788 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1850003e90 con 0x561c56c61e20 2023-07-14T16:21:58.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f1842ff5700 1 -- 172.21.15.62:0/193442788 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1850036d20 con 0x561c56c61e20 2023-07-14T16:21:58.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f1878ef1b80 10 client.17256 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f1878ef1b80 10 client.17256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:21:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f1878ef1b80 20 client.17256 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561c56ccb860 0x561c56ccdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f1878ef1b80 4 client.17256 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:21:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f1878ef1b80 1 -- 172.21.15.62:0/193442788 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561c56b6e050 con 0x561c56ccb860 2023-07-14T16:21:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.318+0000 7f185e7fc700 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561c56ccb860 0x561c56ccdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f185e7fc700 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561c56ccb860 0x561c56ccdc70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aac0dd088cddfa4c server_cookie=93c87d15cff8cd7e in_seq=0 out_seq=0 2023-07-14T16:21:58.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1842ff5700 10 client.17256 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:21:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1842ff5700 1 -- 172.21.15.62:0/193442788 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561c56b6e050 con 0x561c56ccb860 2023-07-14T16:21:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1842ff5700 10 client.17256 handle_command_reply: tid=0 2023-07-14T16:21:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 client.17256 shutdown 2023-07-14T16:21:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 -- 172.21.15.62:0/193442788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 msgr2=0x561c56c6ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6ed80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f18500368f0 tx=0x7f1850005050 comp rx=0 tx=0).stop 2023-07-14T16:21:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 -- 172.21.15.62:0/193442788 shutdown_connections 2023-07-14T16:21:58.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561c56ccb860 0x561c56ccdc70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c56c62c70 0x561c56c71ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c56c614d0 0x561c56c6e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 --2- 172.21.15.62:0/193442788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c56c61e20 0x561c56c6ed80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1878ef1b80 1 -- 172.21.15.62:0/193442788 >> 172.21.15.62:0/193442788 conn(0x561c56c56da0 msgr2=0x561c56c69df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.319+0000 7f1842ff5700 0 client.17256 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:21:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.320+0000 7f1878ef1b80 1 -- 172.21.15.62:0/193442788 shutdown_connections 2023-07-14T16:21:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.320+0000 7f1878ef1b80 1 -- 172.21.15.62:0/193442788 wait complete. 2023-07-14T16:21:58.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.320+0000 7f1878ef1b80 20 client.17256 trim_cache size 0 max 16384 2023-07-14T16:21:58.304 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:21:58.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.320+0000 7f18655ff700 1 -- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18480a11d0 msgr2=0x7f18480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.320+0000 7f18655ff700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18480a11d0 0x7f18480a3680 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1858004b80 tx=0x7f1858004ad0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.320+0000 7f18655ff700 1 -- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1860101660 msgr2=0x7f186019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.320+0000 7f18655ff700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1860101660 0x7f186019ab50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1858007ee0 tx=0x7f185800d3b0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.321+0000 7f18655ff700 1 -- 172.21.15.62:0/1687432487 shutdown_connections 2023-07-14T16:21:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.321+0000 7f18655ff700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1860101660 0x7f186019ab50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.321+0000 7f18655ff700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18600ffae0 0x7f18601935a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.321+0000 7f18655ff700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f18480a11d0 0x7f18480a3680 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.321+0000 7f18655ff700 1 --2- 172.21.15.62:0/1687432487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1860100cd0 0x7f1860193ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.321+0000 7f18655ff700 1 -- 172.21.15.62:0/1687432487 >> 172.21.15.62:0/1687432487 conn(0x7f18600fb0e0 msgr2=0x7f18600fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.321+0000 7f18655ff700 1 -- 172.21.15.62:0/1687432487 shutdown_connections 2023-07-14T16:21:58.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.321+0000 7f18655ff700 1 -- 172.21.15.62:0/1687432487 wait complete. 2023-07-14T16:21:58.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:21:58.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 -- 172.21.15.62:0/2751876428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1478102f00 msgr2=0x7f1478103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 --2- 172.21.15.62:0/2751876428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1478102f00 0x7f1478103350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f146c009990 tx=0x7f146c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 -- 172.21.15.62:0/2751876428 shutdown_connections 2023-07-14T16:21:58.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 --2- 172.21.15.62:0/2751876428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1478103890 0x7f147810a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 --2- 172.21.15.62:0/2751876428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1478101d10 0x7f1478102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 --2- 172.21.15.62:0/2751876428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1478102f00 0x7f1478103350 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 -- 172.21.15.62:0/2751876428 >> 172.21.15.62:0/2751876428 conn(0x7f14780fd350 msgr2=0x7f14780ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 -- 172.21.15.62:0/2751876428 shutdown_connections 2023-07-14T16:21:58.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.586+0000 7f147cbff700 1 -- 172.21.15.62:0/2751876428 wait complete. 2023-07-14T16:21:58.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.588+0000 7f147cbff700 1 Processor -- start 2023-07-14T16:21:58.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.588+0000 7f147cbff700 1 -- start start 2023-07-14T16:21:58.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.588+0000 7f147cbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1478101d10 0x7f1478197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.588+0000 7f147cbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1478102f00 0x7f1478197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.588+0000 7f147cbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1478103890 0x7f147819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.588+0000 7f147cbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147819f500 con 0x7f1478102f00 2023-07-14T16:21:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.588+0000 7f147cbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147819f640 con 0x7f1478101d10 2023-07-14T16:21:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.588+0000 7f147cbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147819f7d0 con 0x7f1478103890 2023-07-14T16:21:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1475d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1478102f00 0x7f1478197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1475d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1478102f00 0x7f1478197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49282/0 (socket says 172.21.15.62:49282) 2023-07-14T16:21:58.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1475d9b700 1 -- 172.21.15.62:0/3764627291 learned_addr learned my addr 172.21.15.62:0/3764627291 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f147659c700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1478101d10 0x7f1478197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1476d9d700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1478103890 0x7f147819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1475d9b700 1 -- 172.21.15.62:0/3764627291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1478103890 msgr2=0x7f147819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1475d9b700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1478103890 0x7f147819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1475d9b700 1 -- 172.21.15.62:0/3764627291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1478101d10 msgr2=0x7f1478197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1475d9b700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1478101d10 0x7f1478197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f1475d9b700 1 -- 172.21.15.62:0/3764627291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0095f0 con 0x7f1478102f00 2023-07-14T16:21:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.589+0000 7f147659c700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1478101d10 0x7f1478197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.590+0000 7f1475d9b700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1478102f00 0x7f1478197ec0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f146c005910 tx=0x7f146c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.590+0000 7f14677fe700 1 -- 172.21.15.62:0/3764627291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c01d070 con 0x7f1478102f00 2023-07-14T16:21:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.590+0000 7f14677fe700 1 -- 172.21.15.62:0/3764627291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f146c022470 con 0x7f1478102f00 2023-07-14T16:21:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.590+0000 7f14677fe700 1 -- 172.21.15.62:0/3764627291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c022ea0 con 0x7f1478102f00 2023-07-14T16:21:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.590+0000 7f147cbff700 1 -- 172.21.15.62:0/3764627291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147819fa50 con 0x7f1478102f00 2023-07-14T16:21:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.590+0000 7f147cbff700 1 -- 172.21.15.62:0/3764627291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147819fe80 con 0x7f1478102f00 2023-07-14T16:21:58.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.596+0000 7f14677fe700 1 -- 172.21.15.62:0/3764627291 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f146c00ba60 con 0x7f1478102f00 2023-07-14T16:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.597+0000 7f14905efb80 1 Processor -- start 2023-07-14T16:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.597+0000 7f14905efb80 1 -- start start 2023-07-14T16:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.597+0000 7f14905efb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d588368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.597+0000 7f14905efb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d588376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.597+0000 7f14905efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.597+0000 7f14905efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5873bcb0 con 0x562d58837c70 2023-07-14T16:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.597+0000 7f14905efb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5878ea40 con 0x562d58836e20 2023-07-14T16:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.597+0000 7f14905efb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d58787ed0 con 0x562d588364d0 2023-07-14T16:21:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.598+0000 7f1475d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.598+0000 7f1475d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49284/0 (socket says 172.21.15.62:49284) 2023-07-14T16:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.598+0000 7f1475d9b700 1 -- 172.21.15.62:0/274008147 learned_addr learned my addr 172.21.15.62:0/274008147 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f1476d9d700 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d588368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f147659c700 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d588376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f14677fe700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14600a11d0 0x7f14600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f1475d9b700 1 -- 172.21.15.62:0/274008147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 msgr2=0x562d588368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f1475d9b700 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d588368e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f1475d9b700 1 -- 172.21.15.62:0/274008147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 msgr2=0x562d588376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f1475d9b700 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d588376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f1475d9b700 1 -- 172.21.15.62:0/274008147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d585b5de0 con 0x562d58837c70 2023-07-14T16:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f147659c700 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d588376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f1476d9d700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14600a11d0 0x7f14600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.599+0000 7f14677fe700 1 -- 172.21.15.62:0/3764627291 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f146c0cdbd0 con 0x7f1478102f00 2023-07-14T16:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.600+0000 7f1475d9b700 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588380e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f146c059dc0 tx=0x7f146c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda8e580c047309e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.600+0000 7f1476d9d700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14600a11d0 0x7f14600a3680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1470009ec0 tx=0x7f1470004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.600+0000 7f1464ff9700 1 -- 172.21.15.62:0/274008147 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c09ae50 con 0x562d58837c70 2023-07-14T16:21:58.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.600+0000 7f1464ff9700 1 -- 172.21.15.62:0/274008147 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f146c09abc0 con 0x562d58837c70 2023-07-14T16:21:58.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.600+0000 7f1464ff9700 1 -- 172.21.15.62:0/274008147 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c09a9e0 con 0x562d58837c70 2023-07-14T16:21:58.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.601+0000 7f14905efb80 1 -- 172.21.15.62:0/274008147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 msgr2=0x562d588380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.601+0000 7f14905efb80 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588380e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f146c059dc0 tx=0x7f146c02f000 comp rx=0 tx=0).stop 2023-07-14T16:21:58.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.602+0000 7f14905efb80 1 -- 172.21.15.62:0/274008147 shutdown_connections 2023-07-14T16:21:58.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.602+0000 7f14905efb80 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588380e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.602+0000 7f14905efb80 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d588368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.602+0000 7f14905efb80 1 --2- 172.21.15.62:0/274008147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d588376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.602+0000 7f14905efb80 1 -- 172.21.15.62:0/274008147 >> 172.21.15.62:0/274008147 conn(0x562d5882bda0 msgr2=0x562d588311d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.602+0000 7f14905efb80 1 -- 172.21.15.62:0/274008147 shutdown_connections 2023-07-14T16:21:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.602+0000 7f14905efb80 1 -- 172.21.15.62:0/274008147 wait complete. 2023-07-14T16:21:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.602+0000 7f14905efb80 1 -- start start 2023-07-14T16:21:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f14905efb80 1 Processor -- start 2023-07-14T16:21:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f14905efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d588364d0 0x562d5883fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f14905efb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d58840260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f14905efb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d58837c70 0x562d588472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f14905efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d58727ac0 con 0x562d588364d0 2023-07-14T16:21:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f14905efb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d587277c0 con 0x562d58836e20 2023-07-14T16:21:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f14905efb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5860b920 con 0x562d58837c70 2023-07-14T16:21:58.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f14905efb80 10 client.0 _ll_register_callbacks cb 0x562d58725760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:58.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.603+0000 7f147659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d588364d0 0x562d5883fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f147659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d588364d0 0x562d5883fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49298/0 (socket says 172.21.15.62:49298) 2023-07-14T16:21:58.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f147659c700 1 -- 172.21.15.62:0/146486053 learned_addr learned my addr 172.21.15.62:0/146486053 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f1476d9d700 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d58837c70 0x562d588472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f1475d9b700 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d58840260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f147659c700 1 -- 172.21.15.62:0/146486053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d58837c70 msgr2=0x562d588472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f147659c700 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d58837c70 0x562d588472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f147659c700 1 -- 172.21.15.62:0/146486053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 msgr2=0x562d58840260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f147659c700 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d58840260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.604+0000 7f147659c700 1 -- 172.21.15.62:0/146486053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147819fe80 con 0x562d588364d0 2023-07-14T16:21:58.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.605+0000 7f147659c700 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d588364d0 0x562d5883fd20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1468012780 tx=0x7f1468012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8e5ad903a71389 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.605+0000 7f145dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:58.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.605+0000 7f145dffb700 1 -- 172.21.15.62:0/146486053 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468004220 con 0x562d588364d0 2023-07-14T16:21:58.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.605+0000 7f14905efb80 1 -- 172.21.15.62:0/146486053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d586e14f0 con 0x562d588364d0 2023-07-14T16:21:58.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.605+0000 7f145dffb700 1 -- 172.21.15.62:0/146486053 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1468004380 con 0x562d588364d0 2023-07-14T16:21:58.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.605+0000 7f145dffb700 1 -- 172.21.15.62:0/146486053 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1468004540 con 0x562d588364d0 2023-07-14T16:21:58.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.605+0000 7f14905efb80 10 client.17268 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:58.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.605+0000 7f14905efb80 1 -- 172.21.15.62:0/146486053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562d5859ec60 con 0x562d588364d0 2023-07-14T16:21:58.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f145dffb700 1 -- 172.21.15.62:0/146486053 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562d586e14f0 con 0x562d588364d0 2023-07-14T16:21:58.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f145dffb700 1 -- 172.21.15.62:0/146486053 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1468003ac0 con 0x562d588364d0 2023-07-14T16:21:58.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f14905efb80 10 client.17268 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f14905efb80 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f14905efb80 20 client.17268 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f14905efb80 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562d5884bab0 0x562d5884dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f14905efb80 4 client.17268 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f14905efb80 1 -- 172.21.15.62:0/146486053 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d58780eb0 con 0x562d5884bab0 2023-07-14T16:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.606+0000 7f1475d9b700 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562d5884bab0 0x562d5884dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.607+0000 7f1475d9b700 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562d5884bab0 0x562d5884dec0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6461b2e782cbdca1 server_cookie=953ecce4c702cdd4 in_seq=0 out_seq=0 2023-07-14T16:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.607+0000 7f145dffb700 10 client.17268 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f145dffb700 1 -- 172.21.15.62:0/146486053 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d58780eb0 con 0x562d5884bab0 2023-07-14T16:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f145dffb700 10 client.17268 handle_command_reply: tid=0 2023-07-14T16:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 client.17268 shutdown 2023-07-14T16:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 -- 172.21.15.62:0/146486053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d588364d0 msgr2=0x562d5883fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d588364d0 0x562d5883fd20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1468012780 tx=0x7f1468012a90 comp rx=0 tx=0).stop 2023-07-14T16:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 -- 172.21.15.62:0/146486053 shutdown_connections 2023-07-14T16:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d58837c70 0x562d588472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d588364d0 0x562d5883fd20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562d5884bab0 0x562d5884dec0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 --2- 172.21.15.62:0/146486053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d58840260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 -- 172.21.15.62:0/146486053 >> 172.21.15.62:0/146486053 conn(0x562d5882bda0 msgr2=0x562d588308b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f145dffb700 0 client.17268 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 -- 172.21.15.62:0/146486053 shutdown_connections 2023-07-14T16:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.608+0000 7f14905efb80 1 -- 172.21.15.62:0/146486053 wait complete. 2023-07-14T16:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.609+0000 7f14905efb80 20 client.17268 trim_cache size 0 max 16384 2023-07-14T16:21:58.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.627+0000 7f14905efb80 1 Processor -- start 2023-07-14T16:21:58.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.627+0000 7f14905efb80 1 -- start start 2023-07-14T16:21:58.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f14905efb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d5883aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f14905efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58836e20 0x562d58847260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f14905efb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58837c70 0x562d5883afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f14905efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d586e14f0 con 0x562d58836e20 2023-07-14T16:21:58.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f14905efb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d588322e0 con 0x562d58837c70 2023-07-14T16:21:58.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f14905efb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5883dcf0 con 0x562d588364d0 2023-07-14T16:21:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f1475d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58836e20 0x562d58847260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f1475d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58836e20 0x562d58847260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49308/0 (socket says 172.21.15.62:49308) 2023-07-14T16:21:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f1475d9b700 1 -- 172.21.15.62:0/1065403380 learned_addr learned my addr 172.21.15.62:0/1065403380 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f147659c700 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d5883aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.628+0000 7f1476d9d700 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58837c70 0x562d5883afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f147659c700 1 -- 172.21.15.62:0/1065403380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58837c70 msgr2=0x562d5883afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f147659c700 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58837c70 0x562d5883afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f147659c700 1 -- 172.21.15.62:0/1065403380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58836e20 msgr2=0x562d58847260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f147659c700 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58836e20 0x562d58847260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f147659c700 1 -- 172.21.15.62:0/1065403380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d585dc250 con 0x562d588364d0 2023-07-14T16:21:58.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f1475d9b700 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58836e20 0x562d58847260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:58.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f1476d9d700 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58837c70 0x562d5883afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:58.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f147659c700 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d5883aa90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1468005950 tx=0x7f1468003ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd353656ec5d9b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f14657fa700 1 -- 172.21.15.62:0/1065403380 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468013ea0 con 0x562d588364d0 2023-07-14T16:21:58.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f14657fa700 1 -- 172.21.15.62:0/1065403380 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1468003680 con 0x562d588364d0 2023-07-14T16:21:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.629+0000 7f14657fa700 1 -- 172.21.15.62:0/1065403380 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14680052d0 con 0x562d588364d0 2023-07-14T16:21:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 -- 172.21.15.62:0/1065403380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 msgr2=0x562d5883aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d5883aa90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1468005950 tx=0x7f1468003ac0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 -- 172.21.15.62:0/1065403380 shutdown_connections 2023-07-14T16:21:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58837c70 0x562d5883afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d5883aa90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 --2- 172.21.15.62:0/1065403380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58836e20 0x562d58847260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 -- 172.21.15.62:0/1065403380 >> 172.21.15.62:0/1065403380 conn(0x562d5882bda0 msgr2=0x562d5883eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 -- 172.21.15.62:0/1065403380 shutdown_connections 2023-07-14T16:21:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.630+0000 7f14905efb80 1 -- 172.21.15.62:0/1065403380 wait complete. 2023-07-14T16:21:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.631+0000 7f14905efb80 1 -- start start 2023-07-14T16:21:58.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.631+0000 7f14905efb80 1 Processor -- start 2023-07-14T16:21:58.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f14905efb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d588470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f14905efb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d58843b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f14905efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f14905efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d588447b0 con 0x562d58837c70 2023-07-14T16:21:58.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f14905efb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5889f960 con 0x562d58836e20 2023-07-14T16:21:58.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f14905efb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d5889fad0 con 0x562d588364d0 2023-07-14T16:21:58.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f14905efb80 10 client.0 _ll_register_callbacks cb 0x7f1468007cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:58.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1476d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1476d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49310/0 (socket says 172.21.15.62:49310) 2023-07-14T16:21:58.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1476d9d700 1 -- 172.21.15.62:0/4034042851 learned_addr learned my addr 172.21.15.62:0/4034042851 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:58.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f147659c700 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d588470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1475d9b700 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d58843b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1476d9d700 1 -- 172.21.15.62:0/4034042851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 msgr2=0x562d588470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1476d9d700 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d588470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1476d9d700 1 -- 172.21.15.62:0/4034042851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 msgr2=0x562d58843b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:58.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1476d9d700 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d58843b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.632+0000 7f1476d9d700 1 -- 172.21.15.62:0/4034042851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d585b5de0 con 0x562d58837c70 2023-07-14T16:21:58.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.633+0000 7f1476d9d700 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588440d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1470007bd0 tx=0x7f147000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a64c825d39ee8a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:58.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.633+0000 7f145dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:58.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.633+0000 7f145dffb700 1 -- 172.21.15.62:0/4034042851 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000e040 con 0x562d58837c70 2023-07-14T16:21:58.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.633+0000 7f145dffb700 1 -- 172.21.15.62:0/4034042851 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1470015cc0 con 0x562d58837c70 2023-07-14T16:21:58.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.633+0000 7f14905efb80 1 -- 172.21.15.62:0/4034042851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f146c09abc0 con 0x562d58837c70 2023-07-14T16:21:58.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.633+0000 7f145dffb700 1 -- 172.21.15.62:0/4034042851 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147002c420 con 0x562d58837c70 2023-07-14T16:21:58.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.633+0000 7f14905efb80 10 client.17274 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:58.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.633+0000 7f14905efb80 1 -- 172.21.15.62:0/4034042851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562d5859ec60 con 0x562d58837c70 2023-07-14T16:21:58.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.634+0000 7f145dffb700 1 -- 172.21.15.62:0/4034042851 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f146c09abc0 con 0x562d58837c70 2023-07-14T16:21:58.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.634+0000 7f145dffb700 1 -- 172.21.15.62:0/4034042851 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1470022400 con 0x562d58837c70 2023-07-14T16:21:58.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.634+0000 7f14905efb80 10 client.17274 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:58.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.634+0000 7f14905efb80 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:58.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.634+0000 7f14905efb80 20 client.17274 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:58.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.634+0000 7f14905efb80 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562d588a15f0 0x562d588a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:58.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.634+0000 7f14905efb80 4 client.17274 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:21:58.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.634+0000 7f14905efb80 1 -- 172.21.15.62:0/4034042851 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d58743050 con 0x562d588a15f0 2023-07-14T16:21:58.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.635+0000 7f147659c700 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562d588a15f0 0x562d588a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:58.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.635+0000 7f147659c700 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562d588a15f0 0x562d588a3a00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ba4a99c407c127 server_cookie=8ee0aa11c3fa7ba8 in_seq=0 out_seq=0 2023-07-14T16:21:58.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.635+0000 7f145dffb700 10 client.17274 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:58.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.635+0000 7f145dffb700 1 -- 172.21.15.62:0/4034042851 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d58743050 con 0x562d588a15f0 2023-07-14T16:21:58.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.635+0000 7f145dffb700 10 client.17274 handle_command_reply: tid=0 2023-07-14T16:21:58.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.635+0000 7f14905efb80 1 client.17274 shutdown 2023-07-14T16:21:58.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 -- 172.21.15.62:0/4034042851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 msgr2=0x562d588440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588440d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1470007bd0 tx=0x7f147000b040 comp rx=0 tx=0).stop 2023-07-14T16:21:58.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 -- 172.21.15.62:0/4034042851 shutdown_connections 2023-07-14T16:21:58.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d58837c70 0x562d588440d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562d588364d0 0x562d588470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562d588a15f0 0x562d588a3a00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 --2- 172.21.15.62:0/4034042851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562d58836e20 0x562d58843b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f145dffb700 0 client.17274 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:58.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 -- 172.21.15.62:0/4034042851 >> 172.21.15.62:0/4034042851 conn(0x562d5882bda0 msgr2=0x562d5883aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 -- 172.21.15.62:0/4034042851 shutdown_connections 2023-07-14T16:21:58.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 1 -- 172.21.15.62:0/4034042851 wait complete. 2023-07-14T16:21:58.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.636+0000 7f14905efb80 20 client.17274 trim_cache size 0 max 16384 2023-07-14T16:21:58.642 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:21:58.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 -- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14600a11d0 msgr2=0x7f14600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14600a11d0 0x7f14600a3680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1470009ec0 tx=0x7f1470004ad0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 -- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1478102f00 msgr2=0x7f1478197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:58.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1478102f00 0x7f1478197ec0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f146c005910 tx=0x7f146c005720 comp rx=0 tx=0).stop 2023-07-14T16:21:58.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 -- 172.21.15.62:0/3764627291 shutdown_connections 2023-07-14T16:21:58.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1478103890 0x7f147819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14600a11d0 0x7f14600a3680 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1478101d10 0x7f1478197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 --2- 172.21.15.62:0/3764627291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1478102f00 0x7f1478197ec0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:58.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 -- 172.21.15.62:0/3764627291 >> 172.21.15.62:0/3764627291 conn(0x7f14780fd350 msgr2=0x7f1478106430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:58.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 -- 172.21.15.62:0/3764627291 shutdown_connections 2023-07-14T16:21:58.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:58.637+0000 7f147cbff700 1 -- 172.21.15.62:0/3764627291 wait complete. 2023-07-14T16:21:59.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:21:58 smithi164 ceph-mon[97427]: pgmap v547: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:21:59.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:21:58 smithi062 ceph-mon[95543]: pgmap v547: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:21:59.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:21:58 smithi081 ceph-mon[127795]: pgmap v547: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:21:59.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:21:59.646 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:21:59.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 -- 172.21.15.62:0/106160479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 msgr2=0x7f7e14101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 --2- 172.21.15.62:0/106160479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 0x7f7e14101120 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009990 tx=0x7f7e08009ca0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 -- 172.21.15.62:0/106160479 shutdown_connections 2023-07-14T16:21:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 --2- 172.21.15.62:0/106160479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e14101660 0x7f7e14107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 --2- 172.21.15.62:0/106160479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e140ffae0 0x7f7e140ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 --2- 172.21.15.62:0/106160479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 0x7f7e14101120 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 -- 172.21.15.62:0/106160479 >> 172.21.15.62:0/106160479 conn(0x7f7e140fb0e0 msgr2=0x7f7e140fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 -- 172.21.15.62:0/106160479 shutdown_connections 2023-07-14T16:21:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.926+0000 7f7e19dff700 1 -- 172.21.15.62:0/106160479 wait complete. 2023-07-14T16:21:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.927+0000 7f7e19dff700 1 Processor -- start 2023-07-14T16:21:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.927+0000 7f7e19dff700 1 -- start start 2023-07-14T16:21:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e19dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e140ffae0 0x7f7e141979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e19dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 0x7f7e14197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e19dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e14101660 0x7f7e1419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e19dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419f530 con 0x7f7e14100cd0 2023-07-14T16:21:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e19dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419f670 con 0x7f7e14101660 2023-07-14T16:21:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e19dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419f800 con 0x7f7e140ffae0 2023-07-14T16:21:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 0x7f7e14197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 0x7f7e14197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49322/0 (socket says 172.21.15.62:49322) 2023-07-14T16:21:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2922080043 learned_addr learned my addr 172.21.15.62:0/2922080043 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.928+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e140ffae0 0x7f7e141979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e13fff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e14101660 0x7f7e1419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2922080043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e140ffae0 msgr2=0x7f7e141979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e140ffae0 0x7f7e141979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2922080043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e14101660 msgr2=0x7f7e1419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e14101660 0x7f7e1419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2922080043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080095f0 con 0x7f7e14100cd0 2023-07-14T16:21:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e140ffae0 0x7f7e141979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e13fff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e14101660 0x7f7e1419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:21:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.929+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 0x7f7e14197ef0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098e0 tx=0x7f7e08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.930+0000 7f7e10ff9700 1 -- 172.21.15.62:0/2922080043 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0801d070 con 0x7f7e14100cd0 2023-07-14T16:21:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.930+0000 7f7e10ff9700 1 -- 172.21.15.62:0/2922080043 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e0800bdd0 con 0x7f7e14100cd0 2023-07-14T16:21:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.930+0000 7f7e19dff700 1 -- 172.21.15.62:0/2922080043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1419fa80 con 0x7f7e14100cd0 2023-07-14T16:21:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.930+0000 7f7e10ff9700 1 -- 172.21.15.62:0/2922080043 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08022db0 con 0x7f7e14100cd0 2023-07-14T16:21:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.930+0000 7f7e19dff700 1 -- 172.21.15.62:0/2922080043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1419fea0 con 0x7f7e14100cd0 2023-07-14T16:21:59.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.934+0000 7f7e10ff9700 1 -- 172.21.15.62:0/2922080043 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7e08022470 con 0x7f7e14100cd0 2023-07-14T16:21:59.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.935+0000 7f7e2d811b80 1 Processor -- start 2023-07-14T16:21:59.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.935+0000 7f7e2d811b80 1 -- start start 2023-07-14T16:21:59.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.935+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ed6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.935+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ed7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.935+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ed8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.935+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516ddbd30 con 0x55d516ed7cf0 2023-07-14T16:21:59.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.935+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516e2eac0 con 0x55d516ed6550 2023-07-14T16:21:59.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.935+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516e27f50 con 0x55d516ed6ea0 2023-07-14T16:21:59.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ed6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ed6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57264/0 (socket says 172.21.15.62:57264) 2023-07-14T16:21:59.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e13fff700 1 -- 172.21.15.62:0/3684903998 learned_addr learned my addr 172.21.15.62:0/3684903998 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:59.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e137fe700 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ed7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ed8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e10ff9700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e137fe700 1 -- 172.21.15.62:0/3684903998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 msgr2=0x55d516ed6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e137fe700 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ed6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e137fe700 1 -- 172.21.15.62:0/3684903998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 msgr2=0x55d516ed8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e137fe700 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ed8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.936+0000 7f7e137fe700 1 -- 172.21.15.62:0/3684903998 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d516c55e60 con 0x55d516ed6ea0 2023-07-14T16:21:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.937+0000 7f7e10ff9700 1 -- 172.21.15.62:0/2922080043 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e080cdf20 con 0x7f7e14100cd0 2023-07-14T16:21:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.937+0000 7f7e137fe700 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ed7760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e040114c0 tx=0x7f7e040117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f430e5ed1bce8a1f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:59.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.937+0000 7f7e13fff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.937+0000 7f7df9ffb700 1 -- 172.21.15.62:0/3684903998 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04004310 con 0x55d516ed6ea0 2023-07-14T16:21:59.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.937+0000 7f7df9ffb700 1 -- 172.21.15.62:0/3684903998 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e04004470 con 0x55d516ed6ea0 2023-07-14T16:21:59.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.937+0000 7f7df9ffb700 1 -- 172.21.15.62:0/3684903998 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04003680 con 0x55d516ed6ea0 2023-07-14T16:21:59.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 -- 172.21.15.62:0/3684903998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 msgr2=0x55d516ed7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ed7760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e040114c0 tx=0x7f7e040117d0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e13fff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0076a0 tx=0x7f7e0c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:59.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 -- 172.21.15.62:0/3684903998 shutdown_connections 2023-07-14T16:21:59.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ed8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ed6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/3684903998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ed7760 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 -- 172.21.15.62:0/3684903998 >> 172.21.15.62:0/3684903998 conn(0x55d516ecbe20 msgr2=0x55d516ed1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:59.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 -- 172.21.15.62:0/3684903998 shutdown_connections 2023-07-14T16:21:59.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.938+0000 7f7e2d811b80 1 -- 172.21.15.62:0/3684903998 wait complete. 2023-07-14T16:21:59.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.939+0000 7f7e2d811b80 1 -- start start 2023-07-14T16:21:59.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.939+0000 7f7e2d811b80 1 Processor -- start 2023-07-14T16:21:59.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ee01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ee0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ee7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516dc7b40 con 0x55d516ed7cf0 2023-07-14T16:21:59.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516dc7840 con 0x55d516ed6550 2023-07-14T16:21:59.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516cab9a0 con 0x55d516ed6ea0 2023-07-14T16:21:59.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e2d811b80 10 client.0 _ll_register_callbacks cb 0x55d516dc57e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:59.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ee7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ee0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ee7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49344/0 (socket says 172.21.15.62:49344) 2023-07-14T16:21:59.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e13fff700 1 -- 172.21.15.62:0/2336637371 learned_addr learned my addr 172.21.15.62:0/2336637371 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:59.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ee01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2336637371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 msgr2=0x55d516ee01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:59.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ee01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2336637371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 msgr2=0x55d516ee7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:59.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.940+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ee7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.941+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2336637371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e04009c90 con 0x55d516ed6ea0 2023-07-14T16:21:59.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.941+0000 7f7e13fff700 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ee7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:21:59.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.941+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ee0720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e080cba80 tx=0x7f7e0809b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e3b4258f04ae33 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:59.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.941+0000 7f7df2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:21:59.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.941+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2336637371 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0809b1a0 con 0x55d516ed6ea0 2023-07-14T16:21:59.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.941+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2336637371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d516d81570 con 0x55d516ed6ea0 2023-07-14T16:21:59.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.941+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2336637371 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e080860b0 con 0x55d516ed6ea0 2023-07-14T16:21:59.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.941+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2336637371 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0809af10 con 0x55d516ed6ea0 2023-07-14T16:21:59.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7e2d811b80 10 client.26860 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:59.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2336637371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d516c3ece0 con 0x55d516ed6ea0 2023-07-14T16:21:59.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2336637371 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d516d81570 con 0x55d516ed6ea0 2023-07-14T16:21:59.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2336637371 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7e080a10f0 con 0x55d516ed6ea0 2023-07-14T16:21:59.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7e2d811b80 10 client.26860 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:59.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7e2d811b80 10 client.26860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:59.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7e2d811b80 20 client.26860 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:59.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d516eebba0 0x55d516eedf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7e2d811b80 4 client.26860 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:21:59.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.942+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2336637371 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d516e20f30 con 0x55d516eebba0 2023-07-14T16:21:59.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.943+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d516eebba0 0x55d516eedf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.943+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d516eebba0 0x55d516eedf80 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4bfa21ccf8125c server_cookie=4308e3e7ed32a2c6 in_seq=0 out_seq=0 2023-07-14T16:21:59.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.943+0000 7f7df2ffd700 10 client.26860 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:59.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.944+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2336637371 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d516e20f30 con 0x55d516eebba0 2023-07-14T16:21:59.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.944+0000 7f7df2ffd700 10 client.26860 handle_command_reply: tid=0 2023-07-14T16:21:59.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.944+0000 7f7e2d811b80 1 client.26860 shutdown 2023-07-14T16:21:59.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2336637371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 msgr2=0x55d516ee0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ee0720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e080cba80 tx=0x7f7e0809b910 comp rx=0 tx=0).stop 2023-07-14T16:21:59.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2336637371 shutdown_connections 2023-07-14T16:21:59.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed7cf0 0x55d516ee7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d516eebba0 0x55d516eedf80 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed6550 0x55d516ee01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2336637371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6ea0 0x55d516ee0720 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2336637371 >> 172.21.15.62:0/2336637371 conn(0x55d516ecbe20 msgr2=0x55d516ed0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7df2ffd700 0 client.26860 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2336637371 shutdown_connections 2023-07-14T16:21:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2336637371 wait complete. 2023-07-14T16:21:59.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.945+0000 7f7e2d811b80 20 client.26860 trim_cache size 0 max 16384 2023-07-14T16:21:59.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.964+0000 7f7e2d811b80 1 Processor -- start 2023-07-14T16:21:59.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.964+0000 7f7e2d811b80 1 -- start start 2023-07-14T16:21:59.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 0x55d516ede0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 0x55d516edabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516ee13d0 con 0x55d516ed6ea0 2023-07-14T16:21:59.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516ecba40 con 0x55d516ed7cf0 2023-07-14T16:21:59.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516ecdf40 con 0x55d516ed6550 2023-07-14T16:21:59.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49360/0 (socket says 172.21.15.62:49360) 2023-07-14T16:21:59.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e12ffd700 1 -- 172.21.15.62:0/72538695 learned_addr learned my addr 172.21.15.62:0/72538695 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:59.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e137fe700 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 0x55d516ede0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.965+0000 7f7e13fff700 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 0x55d516edabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7e12ffd700 1 -- 172.21.15.62:0/72538695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 msgr2=0x55d516ede0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:59.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 0x55d516ede0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7e12ffd700 1 -- 172.21.15.62:0/72538695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 msgr2=0x55d516edabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:59.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 0x55d516edabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7e12ffd700 1 -- 172.21.15.62:0/72538695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d516c7c2d0 con 0x55d516ed6ea0 2023-07-14T16:21:59.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee7720 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7e08097df0 tx=0x7f7e080d3fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ed28285c1b62e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:59.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7dfa7fc700 1 -- 172.21.15.62:0/72538695 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0809af10 con 0x55d516ed6ea0 2023-07-14T16:21:59.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7dfa7fc700 1 -- 172.21.15.62:0/72538695 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e0809b1a0 con 0x55d516ed6ea0 2023-07-14T16:21:59.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.966+0000 7f7dfa7fc700 1 -- 172.21.15.62:0/72538695 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0809f490 con 0x55d516ed6ea0 2023-07-14T16:21:59.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 -- 172.21.15.62:0/72538695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 msgr2=0x55d516ee7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee7720 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7e08097df0 tx=0x7f7e080d3fa0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 -- 172.21.15.62:0/72538695 shutdown_connections 2023-07-14T16:21:59.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 0x55d516edabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 0x55d516ede0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/72538695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee7720 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 -- 172.21.15.62:0/72538695 >> 172.21.15.62:0/72538695 conn(0x55d516ecbe20 msgr2=0x55d516eedbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:59.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 -- 172.21.15.62:0/72538695 shutdown_connections 2023-07-14T16:21:59.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.967+0000 7f7e2d811b80 1 -- 172.21.15.62:0/72538695 wait complete. 2023-07-14T16:21:59.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.968+0000 7f7e2d811b80 1 -- start start 2023-07-14T16:21:59.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.968+0000 7f7e2d811b80 1 Processor -- start 2023-07-14T16:21:59.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 0x55d516ee73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e2d811b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 0x55d516ee4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516ee48d0 con 0x55d516ed6ea0 2023-07-14T16:21:59.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516ee4a40 con 0x55d516ed7cf0 2023-07-14T16:21:59.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e2d811b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d516f3fa20 con 0x55d516ed6550 2023-07-14T16:21:59.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e2d811b80 10 client.0 _ll_register_callbacks cb 0x55d516dc57e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:21:59.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49372/0 (socket says 172.21.15.62:49372) 2023-07-14T16:21:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2543078181 learned_addr learned my addr 172.21.15.62:0/2543078181 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:21:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e13fff700 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 0x55d516ee4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.969+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 0x55d516ee73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2543078181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 msgr2=0x55d516ee73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 0x55d516ee73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2543078181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 msgr2=0x55d516ee4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:21:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 0x55d516ee4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7e12ffd700 1 -- 172.21.15.62:0/2543078181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d516c3ece0 con 0x55d516ed6ea0 2023-07-14T16:21:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee3e50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7e0809b360 tx=0x7f7e08097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6f94744f3c4eea server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:21:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7df2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:21:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2543078181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080d1290 con 0x55d516ed6ea0 2023-07-14T16:21:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2543078181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e080860b0 con 0x55d516ed6ea0 2023-07-14T16:21:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.970+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2543078181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e04004470 con 0x55d516ed6ea0 2023-07-14T16:21:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2543078181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0809f470 con 0x55d516ed6ea0 2023-07-14T16:21:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7e2d811b80 10 client.17292 fetch_fsmap learned FSMap version 25 2023-07-14T16:21:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2543078181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d516c54330 con 0x55d516ed6ea0 2023-07-14T16:21:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2543078181 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e04004470 con 0x55d516ed6ea0 2023-07-14T16:21:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2543078181 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7e080b3d20 con 0x55d516ed6ea0 2023-07-14T16:21:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7e2d811b80 10 client.17292 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:21:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7e2d811b80 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:21:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7e2d811b80 20 client.17292 populate_metadata read hostname 'smithi062' 2023-07-14T16:21:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d516f40d60 0x55d516f43170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:21:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7e2d811b80 4 client.17292 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:21:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.971+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2543078181 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d516de30d0 con 0x55d516f40d60 2023-07-14T16:21:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.972+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d516f40d60 0x55d516f43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:21:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.972+0000 7f7df2ffd700 10 client.17292 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:59.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.972+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d516f40d60 0x55d516f43170 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570077e1637c6b86 server_cookie=3a3b6f16bf07a0ae in_seq=0 out_seq=0 2023-07-14T16:21:59.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7df2ffd700 1 -- 172.21.15.62:0/2543078181 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d516de30d0 con 0x55d516f40d60 2023-07-14T16:21:59.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7df2ffd700 10 client.17292 handle_command_reply: tid=0 2023-07-14T16:21:59.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 client.17292 shutdown 2023-07-14T16:21:59.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2543078181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 msgr2=0x55d516ee3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee3e50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7e0809b360 tx=0x7f7e08097040 comp rx=0 tx=0).stop 2023-07-14T16:21:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2543078181 shutdown_connections 2023-07-14T16:21:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d516f40d60 0x55d516f43170 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d516ed7cf0 0x55d516ee4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d516ed6550 0x55d516ee73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 --2- 172.21.15.62:0/2543078181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d516ed6ea0 0x55d516ee3e50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2543078181 >> 172.21.15.62:0/2543078181 conn(0x55d516ecbe20 msgr2=0x55d516ede0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7df2ffd700 0 client.17292 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:21:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2543078181 shutdown_connections 2023-07-14T16:21:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.974+0000 7f7e2d811b80 1 -- 172.21.15.62:0/2543078181 wait complete. 2023-07-14T16:21:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e2d811b80 20 client.17292 trim_cache size 0 max 16384 2023-07-14T16:21:59.956 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:21:59.956 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:21:59.956 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "c5f06f3a-f8a3-4064-a986-246c1a4feef0", 2023-07-14T16:21:59.957 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:21:59.957 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:21:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e19dff700 1 -- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1150 msgr2=0x7f7dfc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e19dff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0076a0 tx=0x7f7e0c009040 comp rx=0 tx=0).stop 2023-07-14T16:21:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e19dff700 1 -- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 msgr2=0x7f7e14197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:21:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e19dff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 0x7f7e14197ef0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098e0 tx=0x7f7e08004730 comp rx=0 tx=0).stop 2023-07-14T16:21:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e19dff700 1 -- 172.21.15.62:0/2922080043 shutdown_connections 2023-07-14T16:21:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e19dff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e19dff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e14101660 0x7f7e1419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.975+0000 7f7e19dff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e140ffae0 0x7f7e141979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.976+0000 7f7e19dff700 1 --2- 172.21.15.62:0/2922080043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14100cd0 0x7f7e14197ef0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:21:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.976+0000 7f7e19dff700 1 -- 172.21.15.62:0/2922080043 >> 172.21.15.62:0/2922080043 conn(0x7f7e140fb0e0 msgr2=0x7f7e14106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:21:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.976+0000 7f7e19dff700 1 -- 172.21.15.62:0/2922080043 shutdown_connections 2023-07-14T16:21:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:21:59.976+0000 7f7e19dff700 1 -- 172.21.15.62:0/2922080043 wait complete. 2023-07-14T16:21:59.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:22:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 -- 172.21.15.62:0/2016238786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8102f40 msgr2=0x7efff8105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 --2- 172.21.15.62:0/2016238786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8102f40 0x7efff8105320 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7effec009990 tx=0x7effec009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 -- 172.21.15.62:0/2016238786 shutdown_connections 2023-07-14T16:22:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 --2- 172.21.15.62:0/2016238786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efff8105860 0x7efff8107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 --2- 172.21.15.62:0/2016238786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efff8100620 0x7efff8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 --2- 172.21.15.62:0/2016238786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8102f40 0x7efff8105320 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 -- 172.21.15.62:0/2016238786 >> 172.21.15.62:0/2016238786 conn(0x7efff80fa000 msgr2=0x7efff80fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 -- 172.21.15.62:0/2016238786 shutdown_connections 2023-07-14T16:22:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.076+0000 7efffe3ff700 1 -- 172.21.15.62:0/2016238786 wait complete. 2023-07-14T16:22:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.078+0000 7efffe3ff700 1 Processor -- start 2023-07-14T16:22:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.078+0000 7efffe3ff700 1 -- start start 2023-07-14T16:22:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.078+0000 7efffe3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efff8100620 0x7efff80fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.078+0000 7efffe3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efff8102f40 0x7efff80feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.078+0000 7efffe3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8105860 0x7efff80ff050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.078+0000 7efffe3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff819f7e0 con 0x7efff8105860 2023-07-14T16:22:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.078+0000 7efffe3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff819f920 con 0x7efff8100620 2023-07-14T16:22:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.078+0000 7efffe3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff819fa60 con 0x7efff8102f40 2023-07-14T16:22:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efff8100620 0x7efff80fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efff8100620 0x7efff80fe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57314/0 (socket says 172.21.15.62:57314) 2023-07-14T16:22:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff7fff700 1 -- 172.21.15.62:0/257797338 learned_addr learned my addr 172.21.15.62:0/257797338 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff77fe700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efff8102f40 0x7efff80feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efffc99c700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8105860 0x7efff80ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff77fe700 1 -- 172.21.15.62:0/257797338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efff8100620 msgr2=0x7efff80fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff77fe700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efff8100620 0x7efff80fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff77fe700 1 -- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8105860 msgr2=0x7efff80ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff77fe700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8105860 0x7efff80ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.079+0000 7efff77fe700 1 -- 172.21.15.62:0/257797338 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec0095f0 con 0x7efff8102f40 2023-07-14T16:22:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.080+0000 7efff77fe700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efff8102f40 0x7efff80feb10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7effec003f60 tx=0x7effec005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.080+0000 7efff57fa700 1 -- 172.21.15.62:0/257797338 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec01d070 con 0x7efff8102f40 2023-07-14T16:22:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.080+0000 7efffe3ff700 1 -- 172.21.15.62:0/257797338 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff819fc90 con 0x7efff8102f40 2023-07-14T16:22:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.080+0000 7efff57fa700 1 -- 172.21.15.62:0/257797338 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7effec00be30 con 0x7efff8102f40 2023-07-14T16:22:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.080+0000 7efff57fa700 1 -- 172.21.15.62:0/257797338 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec022db0 con 0x7efff8102f40 2023-07-14T16:22:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.080+0000 7efffe3ff700 1 -- 172.21.15.62:0/257797338 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff81a0110 con 0x7efff8102f40 2023-07-14T16:22:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.085+0000 7efff57fa700 1 -- 172.21.15.62:0/257797338 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7effec022570 con 0x7efff8102f40 2023-07-14T16:22:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.086+0000 7f0011d9fb80 1 Processor -- start 2023-07-14T16:22:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.086+0000 7f0011d9fb80 1 -- start start 2023-07-14T16:22:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441facb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441eb0c50 con 0x560441fab840 2023-07-14T16:22:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441f039e0 con 0x560441fac6e0 2023-07-14T16:22:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441efce70 con 0x560441e076b0 2023-07-14T16:22:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7efff7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7efff7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49410/0 (socket says 172.21.15.62:49410) 2023-07-14T16:22:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7efff7fff700 1 -- 172.21.15.62:0/849485021 learned_addr learned my addr 172.21.15.62:0/849485021 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7efffc99c700 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.087+0000 7efff77fe700 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441facb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.088+0000 7efffc99c700 1 -- 172.21.15.62:0/849485021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 msgr2=0x560441facb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.088+0000 7efffc99c700 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441facb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.088+0000 7efffc99c700 1 -- 172.21.15.62:0/849485021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 msgr2=0x560441fac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.088+0000 7efffc99c700 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.088+0000 7efffc99c700 1 -- 172.21.15.62:0/849485021 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560441d2ad80 con 0x560441e076b0 2023-07-14T16:22:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.088+0000 7efffc99c700 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fab300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efff0009c40 tx=0x7efff0009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f88b9ef14acb653 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.088+0000 7efff57fa700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7effe00a11d0 0x7effe00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.089+0000 7effde7fc700 1 -- 172.21.15.62:0/849485021 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0019070 con 0x560441e076b0 2023-07-14T16:22:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.089+0000 7effde7fc700 1 -- 172.21.15.62:0/849485021 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff000f040 con 0x560441e076b0 2023-07-14T16:22:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.089+0000 7effde7fc700 1 -- 172.21.15.62:0/849485021 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0014640 con 0x560441e076b0 2023-07-14T16:22:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.089+0000 7efff57fa700 1 -- 172.21.15.62:0/257797338 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7effec0cdd70 con 0x7efff8102f40 2023-07-14T16:22:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.089+0000 7efff7fff700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7effe00a11d0 0x7effe00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.089+0000 7f0011d9fb80 1 -- 172.21.15.62:0/849485021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 msgr2=0x560441fab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.089+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fab300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efff0009c40 tx=0x7efff0009f50 comp rx=0 tx=0).stop 2023-07-14T16:22:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7efff7fff700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7effe00a11d0 0x7effe00a3680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7effe8009690 tx=0x7effe8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7f0011d9fb80 1 -- 172.21.15.62:0/849485021 shutdown_connections 2023-07-14T16:22:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441facb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fab300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/849485021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7f0011d9fb80 1 -- 172.21.15.62:0/849485021 >> 172.21.15.62:0/849485021 conn(0x560441fa0be0 msgr2=0x560441fa5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7f0011d9fb80 1 -- 172.21.15.62:0/849485021 shutdown_connections 2023-07-14T16:22:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7f0011d9fb80 1 -- 172.21.15.62:0/849485021 wait complete. 2023-07-14T16:22:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.090+0000 7f0011d9fb80 1 -- start start 2023-07-14T16:22:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.091+0000 7f0011d9fb80 1 Processor -- start 2023-07-14T16:22:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.091+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441e076b0 0x560441fb48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.091+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441fbbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441e9ca60 con 0x560441e076b0 2023-07-14T16:22:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441e9c760 con 0x560441fac6e0 2023-07-14T16:22:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441d808c0 con 0x560441fab840 2023-07-14T16:22:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7f0011d9fb80 10 client.0 _ll_register_callbacks cb 0x560441e9a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efff7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441e076b0 0x560441fb48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efff7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441e076b0 0x560441fb48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49418/0 (socket says 172.21.15.62:49418) 2023-07-14T16:22:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efff7fff700 1 -- 172.21.15.62:0/1414074475 learned_addr learned my addr 172.21.15.62:0/1414074475 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efffc99c700 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441fbbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efff7fff700 1 -- 172.21.15.62:0/1414074475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 msgr2=0x560441fb4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efff7fff700 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efff7fff700 1 -- 172.21.15.62:0/1414074475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 msgr2=0x560441fbbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efff7fff700 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441fbbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.092+0000 7efff7fff700 1 -- 172.21.15.62:0/1414074475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff0005e70 con 0x560441e076b0 2023-07-14T16:22:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.093+0000 7efff7fff700 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441e076b0 0x560441fb48f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7effe80099a0 tx=0x7effe80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3348e66f20fe20fd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.093+0000 7effd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.093+0000 7effd77fe700 1 -- 172.21.15.62:0/1414074475 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8022070 con 0x560441e076b0 2023-07-14T16:22:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.093+0000 7effd77fe700 1 -- 172.21.15.62:0/1414074475 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7effe8012d80 con 0x560441e076b0 2023-07-14T16:22:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.093+0000 7f0011d9fb80 1 -- 172.21.15.62:0/1414074475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560441e56490 con 0x560441e076b0 2023-07-14T16:22:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.093+0000 7effd77fe700 1 -- 172.21.15.62:0/1414074475 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7effe8017070 con 0x560441e076b0 2023-07-14T16:22:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7f0011d9fb80 10 client.17304 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7f0011d9fb80 1 -- 172.21.15.62:0/1414074475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560441d13c00 con 0x560441e076b0 2023-07-14T16:22:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7effd77fe700 1 -- 172.21.15.62:0/1414074475 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560441e56490 con 0x560441e076b0 2023-07-14T16:22:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7effd77fe700 1 -- 172.21.15.62:0/1414074475 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7effe8033650 con 0x560441e076b0 2023-07-14T16:22:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7f0011d9fb80 10 client.17304 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7f0011d9fb80 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7f0011d9fb80 20 client.17304 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560441fc07a0 0x560441fc2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7f0011d9fb80 4 client.17304 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.094+0000 7f0011d9fb80 1 -- 172.21.15.62:0/1414074475 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560441ef5e50 con 0x560441fc07a0 2023-07-14T16:22:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.095+0000 7efff77fe700 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560441fc07a0 0x560441fc2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.096+0000 7efff77fe700 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560441fc07a0 0x560441fc2bb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41628488505044d server_cookie=e12d6e3ed68f92ad in_seq=0 out_seq=0 2023-07-14T16:22:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.096+0000 7effd77fe700 10 client.17304 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.096+0000 7effd77fe700 1 -- 172.21.15.62:0/1414074475 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560441ef5e50 con 0x560441fc07a0 2023-07-14T16:22:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.096+0000 7effd77fe700 10 client.17304 handle_command_reply: tid=0 2023-07-14T16:22:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 client.17304 shutdown 2023-07-14T16:22:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 -- 172.21.15.62:0/1414074475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441e076b0 msgr2=0x560441fb48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441e076b0 0x560441fb48f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7effe80099a0 tx=0x7effe80047c0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 -- 172.21.15.62:0/1414074475 shutdown_connections 2023-07-14T16:22:00.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560441fc07a0 0x560441fc2bb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441fbbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441e076b0 0x560441fb48f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/1414074475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7effd77fe700 0 client.17304 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 -- 172.21.15.62:0/1414074475 >> 172.21.15.62:0/1414074475 conn(0x560441fa0be0 msgr2=0x560441fa54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 -- 172.21.15.62:0/1414074475 shutdown_connections 2023-07-14T16:22:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.097+0000 7f0011d9fb80 1 -- 172.21.15.62:0/1414074475 wait complete. 2023-07-14T16:22:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.098+0000 7f0011d9fb80 20 client.17304 trim_cache size 0 max 16384 2023-07-14T16:22:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.118+0000 7f0011d9fb80 1 Processor -- start 2023-07-14T16:22:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.118+0000 7f0011d9fb80 1 -- start start 2023-07-14T16:22:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.118+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fb35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.118+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fbbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.118+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441faf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.118+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441fabc10 con 0x560441fab840 2023-07-14T16:22:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.118+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441fb3da0 con 0x560441fac6e0 2023-07-14T16:22:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.118+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441fa2d00 con 0x560441e076b0 2023-07-14T16:22:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efff77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fbbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efffc99c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441faf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efff7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fb35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efff77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fbbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49434/0 (socket says 172.21.15.62:49434) 2023-07-14T16:22:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efff77fe700 1 -- 172.21.15.62:0/3675789676 learned_addr learned my addr 172.21.15.62:0/3675789676 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efffc99c700 1 -- 172.21.15.62:0/3675789676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 msgr2=0x560441fb35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efffc99c700 1 --2- 172.21.15.62:0/3675789676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fb35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efffc99c700 1 -- 172.21.15.62:0/3675789676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 msgr2=0x560441fbbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efffc99c700 1 --2- 172.21.15.62:0/3675789676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fbbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efffc99c700 1 -- 172.21.15.62:0/3675789676 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560441d511f0 con 0x560441fac6e0 2023-07-14T16:22:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.119+0000 7efff77fe700 1 --2- 172.21.15.62:0/3675789676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fbbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.120+0000 7efffc99c700 1 --2- 172.21.15.62:0/3675789676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441faf590 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efff0009c10 tx=0x7efff0007e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f144a1d3dca9480 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.120+0000 7effdeffd700 1 -- 172.21.15.62:0/3675789676 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0019040 con 0x560441fac6e0 2023-07-14T16:22:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.120+0000 7effdeffd700 1 -- 172.21.15.62:0/3675789676 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff0015b10 con 0x560441fac6e0 2023-07-14T16:22:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.120+0000 7effdeffd700 1 -- 172.21.15.62:0/3675789676 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff001fa70 con 0x560441fac6e0 2023-07-14T16:22:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 -- 172.21.15.62:0/3675789676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 msgr2=0x560441faf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/3675789676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441faf590 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efff0009c10 tx=0x7efff0007e50 comp rx=0 tx=0).stop 2023-07-14T16:22:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 -- 172.21.15.62:0/3675789676 shutdown_connections 2023-07-14T16:22:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/3675789676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441fac6e0 0x560441faf590 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/3675789676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441e076b0 0x560441fb35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/3675789676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fab840 0x560441fbbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 -- 172.21.15.62:0/3675789676 >> 172.21.15.62:0/3675789676 conn(0x560441fa0be0 msgr2=0x560441fc7510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 -- 172.21.15.62:0/3675789676 shutdown_connections 2023-07-14T16:22:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.121+0000 7f0011d9fb80 1 -- 172.21.15.62:0/3675789676 wait complete. 2023-07-14T16:22:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 1 -- start start 2023-07-14T16:22:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 1 Processor -- start 2023-07-14T16:22:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441e076b0 0x560441fb8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fac6e0 0x560441fbb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441fbb6a0 con 0x560441fac6e0 2023-07-14T16:22:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441fbb810 con 0x560441e076b0 2023-07-14T16:22:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560441fbb9d0 con 0x560441fab840 2023-07-14T16:22:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.122+0000 7f0011d9fb80 10 client.0 _ll_register_callbacks cb 0x7effe801d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efffc99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fac6e0 0x560441fbb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efff77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efffc99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fac6e0 0x560441fbb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49446/0 (socket says 172.21.15.62:49446) 2023-07-14T16:22:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efff77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44314/0 (socket says 172.21.15.62:44314) 2023-07-14T16:22:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efffc99c700 1 -- 172.21.15.62:0/832993727 learned_addr learned my addr 172.21.15.62:0/832993727 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efff7fff700 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441e076b0 0x560441fb8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efff77fe700 1 -- 172.21.15.62:0/832993727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441e076b0 msgr2=0x560441fb8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efff77fe700 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441e076b0 0x560441fb8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efff77fe700 1 -- 172.21.15.62:0/832993727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fac6e0 msgr2=0x560441fbb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efff77fe700 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fac6e0 0x560441fbb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.123+0000 7efff77fe700 1 -- 172.21.15.62:0/832993727 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff81a0110 con 0x560441fab840 2023-07-14T16:22:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7efff77fe700 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb85a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effec000c00 tx=0x7effec005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e94398b784b3ccf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7effd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:22:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7effd77fe700 1 -- 172.21.15.62:0/832993727 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec09aff0 con 0x560441fab840 2023-07-14T16:22:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7f0011d9fb80 1 -- 172.21.15.62:0/832993727 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efff000f040 con 0x560441fab840 2023-07-14T16:22:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7effd77fe700 1 -- 172.21.15.62:0/832993727 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7effec0a10d0 con 0x560441fab840 2023-07-14T16:22:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7effd77fe700 1 -- 172.21.15.62:0/832993727 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7effec09ad60 con 0x560441fab840 2023-07-14T16:22:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7f0011d9fb80 10 client.26881 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7f0011d9fb80 1 -- 172.21.15.62:0/832993727 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560441d2ad80 con 0x560441fab840 2023-07-14T16:22:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.124+0000 7effd77fe700 1 -- 172.21.15.62:0/832993727 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff000f040 con 0x560441fab840 2023-07-14T16:22:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.125+0000 7effd77fe700 1 -- 172.21.15.62:0/832993727 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7effec09a710 con 0x560441fab840 2023-07-14T16:22:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.125+0000 7f0011d9fb80 10 client.26881 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.125+0000 7f0011d9fb80 10 client.26881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.125+0000 7f0011d9fb80 20 client.26881 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.125+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604420152e0 0x5604420176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.125+0000 7f0011d9fb80 4 client.26881 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:22:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.125+0000 7f0011d9fb80 1 -- 172.21.15.62:0/832993727 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560441eb7ff0 con 0x5604420152e0 2023-07-14T16:22:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.125+0000 7efff7fff700 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604420152e0 0x5604420176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.126+0000 7efff7fff700 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604420152e0 0x5604420176f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a59bc4c394b9a213 server_cookie=6259595ba68f5413 in_seq=0 out_seq=0 2023-07-14T16:22:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.126+0000 7effd77fe700 10 client.26881 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7effd77fe700 1 -- 172.21.15.62:0/832993727 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560441eb7ff0 con 0x5604420152e0 2023-07-14T16:22:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7effd77fe700 10 client.26881 handle_command_reply: tid=0 2023-07-14T16:22:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 client.26881 shutdown 2023-07-14T16:22:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 -- 172.21.15.62:0/832993727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 msgr2=0x560441fb85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb85a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effec000c00 tx=0x7effec005c40 comp rx=0 tx=0).stop 2023-07-14T16:22:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 -- 172.21.15.62:0/832993727 shutdown_connections 2023-07-14T16:22:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560441fac6e0 0x560441fbb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604420152e0 0x5604420176f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560441e076b0 0x560441fb8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 --2- 172.21.15.62:0/832993727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560441fab840 0x560441fb85a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7effd77fe700 0 client.26881 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 -- 172.21.15.62:0/832993727 >> 172.21.15.62:0/832993727 conn(0x560441fa0be0 msgr2=0x560441fb35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 -- 172.21.15.62:0/832993727 shutdown_connections 2023-07-14T16:22:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 1 -- 172.21.15.62:0/832993727 wait complete. 2023-07-14T16:22:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.135+0000 7f0011d9fb80 20 client.26881 trim_cache size 0 max 16384 2023-07-14T16:22:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:22:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:22:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:22:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:22:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 -- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7effe00a11d0 msgr2=0x7effe00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7effe00a11d0 0x7effe00a3680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7effe8009690 tx=0x7effe8009260 comp rx=0 tx=0).stop 2023-07-14T16:22:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 -- 172.21.15.62:0/257797338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efff8102f40 msgr2=0x7efff80feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efff8102f40 0x7efff80feb10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7effec003f60 tx=0x7effec005740 comp rx=0 tx=0).stop 2023-07-14T16:22:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 -- 172.21.15.62:0/257797338 shutdown_connections 2023-07-14T16:22:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efff8105860 0x7efff80ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7effe00a11d0 0x7effe00a3680 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efff8100620 0x7efff80fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 --2- 172.21.15.62:0/257797338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efff8102f40 0x7efff80feb10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 -- 172.21.15.62:0/257797338 >> 172.21.15.62:0/257797338 conn(0x7efff80fa000 msgr2=0x7efff8101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.136+0000 7efffe3ff700 1 -- 172.21.15.62:0/257797338 shutdown_connections 2023-07-14T16:22:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:00.137+0000 7efffe3ff700 1 -- 172.21.15.62:0/257797338 wait complete. 2023-07-14T16:22:00.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.259+0000 7f1a737ff700 1 -- 172.21.15.62:0/345872036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 msgr2=0x7f1a6c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.259+0000 7f1a737ff700 1 --2- 172.21.15.62:0/345872036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 0x7f1a6c102940 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009990 tx=0x7f1a68009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.260+0000 7f1a737ff700 1 -- 172.21.15.62:0/345872036 shutdown_connections 2023-07-14T16:22:00.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.260+0000 7f1a737ff700 1 --2- 172.21.15.62:0/345872036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a6c1057a0 0x7f1a6c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.260+0000 7f1a737ff700 1 --2- 172.21.15.62:0/345872036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 0x7f1a6c102940 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.260+0000 7f1a737ff700 1 --2- 172.21.15.62:0/345872036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a6c102e80 0x7f1a6c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.260+0000 7f1a737ff700 1 -- 172.21.15.62:0/345872036 >> 172.21.15.62:0/345872036 conn(0x7f1a6c0fa000 msgr2=0x7f1a6c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.260+0000 7f1a737ff700 1 -- 172.21.15.62:0/345872036 shutdown_connections 2023-07-14T16:22:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.260+0000 7f1a737ff700 1 -- 172.21.15.62:0/345872036 wait complete. 2023-07-14T16:22:00.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.261+0000 7f1a737ff700 1 Processor -- start 2023-07-14T16:22:00.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.261+0000 7f1a737ff700 1 -- start start 2023-07-14T16:22:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.261+0000 7f1a737ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 0x7f1a6c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.261+0000 7f1a737ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a6c102e80 0x7f1a6c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.261+0000 7f1a737ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a6c1057a0 0x7f1a6c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.261+0000 7f1a737ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c19f4c0 con 0x7f1a6c100560 2023-07-14T16:22:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.261+0000 7f1a737ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c19f600 con 0x7f1a6c1057a0 2023-07-14T16:22:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.261+0000 7f1a737ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c19f790 con 0x7f1a6c102e80 2023-07-14T16:22:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 0x7f1a6c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 0x7f1a6c197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49454/0 (socket says 172.21.15.62:49454) 2023-07-14T16:22:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 -- 172.21.15.62:0/3530417830 learned_addr learned my addr 172.21.15.62:0/3530417830 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a6c102e80 0x7f1a6c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a6c1057a0 0x7f1a6c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 -- 172.21.15.62:0/3530417830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a6c102e80 msgr2=0x7f1a6c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a6c102e80 0x7f1a6c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 -- 172.21.15.62:0/3530417830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a6c1057a0 msgr2=0x7f1a6c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a6c1057a0 0x7f1a6c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 -- 172.21.15.62:0/3530417830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680095f0 con 0x7f1a6c100560 2023-07-14T16:22:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.262+0000 7f1a7159b700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 0x7f1a6c197940 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1a68006010 tx=0x7f1a68003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.263+0000 7f1a627fc700 1 -- 172.21.15.62:0/3530417830 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6801d070 con 0x7f1a6c100560 2023-07-14T16:22:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.263+0000 7f1a627fc700 1 -- 172.21.15.62:0/3530417830 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a6800bd70 con 0x7f1a6c100560 2023-07-14T16:22:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.263+0000 7f1a737ff700 1 -- 172.21.15.62:0/3530417830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c19fa10 con 0x7f1a6c100560 2023-07-14T16:22:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.263+0000 7f1a627fc700 1 -- 172.21.15.62:0/3530417830 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68022c40 con 0x7f1a6c100560 2023-07-14T16:22:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.263+0000 7f1a737ff700 1 -- 172.21.15.62:0/3530417830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c19fe90 con 0x7f1a6c100560 2023-07-14T16:22:00.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.267+0000 7f1a627fc700 1 -- 172.21.15.62:0/3530417830 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a68022470 con 0x7f1a6c100560 2023-07-14T16:22:00.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.268+0000 7f1a871b3b80 1 Processor -- start 2023-07-14T16:22:00.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.268+0000 7f1a871b3b80 1 -- start start 2023-07-14T16:22:00.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa14d0 0x560b51fa18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa1e20 0x560b51fa26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51fa30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51ea6cb0 con 0x560b51fa2c70 2023-07-14T16:22:00.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51ef9a40 con 0x560b51fa14d0 2023-07-14T16:22:00.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51ef2ed0 con 0x560b51fa1e20 2023-07-14T16:22:00.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a70d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51fa30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a71d9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa14d0 0x560b51fa18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a71d9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa14d0 0x560b51fa18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57378/0 (socket says 172.21.15.62:57378) 2023-07-14T16:22:00.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a70d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51fa30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49460/0 (socket says 172.21.15.62:49460) 2023-07-14T16:22:00.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a71d9c700 1 -- 172.21.15.62:0/3622004564 learned_addr learned my addr 172.21.15.62:0/3622004564 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.269+0000 7f1a7159b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa1e20 0x560b51fa26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a627fc700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1a580a11d0 0x7f1a580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a7159b700 1 -- 172.21.15.62:0/3622004564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa14d0 msgr2=0x560b51fa18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a7159b700 1 --2- 172.21.15.62:0/3622004564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa14d0 0x560b51fa18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a7159b700 1 -- 172.21.15.62:0/3622004564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 msgr2=0x560b51fa30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a7159b700 1 --2- 172.21.15.62:0/3622004564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51fa30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a7159b700 1 -- 172.21.15.62:0/3622004564 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b51d20de0 con 0x560b51fa1e20 2023-07-14T16:22:00.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/3622004564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51fa30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:00.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a627fc700 1 -- 172.21.15.62:0/3530417830 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1a680cdeb0 con 0x7f1a6c100560 2023-07-14T16:22:00.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1a580a11d0 0x7f1a580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.270+0000 7f1a7159b700 1 --2- 172.21.15.62:0/3622004564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa1e20 0x560b51fa26e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a680cb900 tx=0x7f1a68009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd620dfceb835195 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.271+0000 7f1a577fe700 1 -- 172.21.15.62:0/3622004564 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6809ae40 con 0x560b51fa1e20 2023-07-14T16:22:00.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.271+0000 7f1a577fe700 1 -- 172.21.15.62:0/3622004564 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a6809ac60 con 0x560b51fa1e20 2023-07-14T16:22:00.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.271+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1a580a11d0 0x7f1a580a3680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009a60 tx=0x7f1a64004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.271+0000 7f1a577fe700 1 -- 172.21.15.62:0/3622004564 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680d1420 con 0x560b51fa1e20 2023-07-14T16:22:00.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.271+0000 7f1a871b3b80 1 -- 172.21.15.62:0/3622004564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa1e20 msgr2=0x560b51fa26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.271+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/3622004564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa1e20 0x560b51fa26e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a680cb900 tx=0x7f1a68009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.272+0000 7f1a871b3b80 1 -- 172.21.15.62:0/3622004564 shutdown_connections 2023-07-14T16:22:00.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.272+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/3622004564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51fa30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.272+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/3622004564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa14d0 0x560b51fa18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.272+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/3622004564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa1e20 0x560b51fa26e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.272+0000 7f1a871b3b80 1 -- 172.21.15.62:0/3622004564 >> 172.21.15.62:0/3622004564 conn(0x560b51f96da0 msgr2=0x560b51f9c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.272+0000 7f1a871b3b80 1 -- 172.21.15.62:0/3622004564 shutdown_connections 2023-07-14T16:22:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.272+0000 7f1a871b3b80 1 -- 172.21.15.62:0/3622004564 wait complete. 2023-07-14T16:22:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.272+0000 7f1a871b3b80 1 -- start start 2023-07-14T16:22:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.273+0000 7f1a871b3b80 1 Processor -- start 2023-07-14T16:22:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.273+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa14d0 0x560b51faad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.273+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.273+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa2c70 0x560b51fb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.273+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51e92ac0 con 0x560b51fa14d0 2023-07-14T16:22:00.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.273+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51e927c0 con 0x560b51fa1e20 2023-07-14T16:22:00.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.273+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51d76920 con 0x560b51fa2c70 2023-07-14T16:22:00.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.273+0000 7f1a871b3b80 10 client.0 _ll_register_callbacks cb 0x560b51e90760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:00.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a7159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa14d0 0x560b51faad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a7159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa14d0 0x560b51faad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49468/0 (socket says 172.21.15.62:49468) 2023-07-14T16:22:00.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a7159b700 1 -- 172.21.15.62:0/513293728 learned_addr learned my addr 172.21.15.62:0/513293728 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa2c70 0x560b51fb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a7159b700 1 -- 172.21.15.62:0/513293728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa2c70 msgr2=0x560b51fb22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a7159b700 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa2c70 0x560b51fb22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a7159b700 1 -- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 msgr2=0x560b51fab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a7159b700 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a7159b700 1 -- 172.21.15.62:0/513293728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c00b8e0 con 0x560b51fa14d0 2023-07-14T16:22:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.274+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a7159b700 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa14d0 0x560b51faad60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1a68014c00 tx=0x7f1a68009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e05621f4ec6d8d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a54ff9700 1 -- 172.21.15.62:0/513293728 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680142a0 con 0x560b51fa14d0 2023-07-14T16:22:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a54ff9700 1 -- 172.21.15.62:0/513293728 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1a6809b300 con 0x560b51fa14d0 2023-07-14T16:22:00.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a871b3b80 1 -- 172.21.15.62:0/513293728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b51e4c4f0 con 0x560b51fa14d0 2023-07-14T16:22:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a54ff9700 1 -- 172.21.15.62:0/513293728 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a680996a0 con 0x560b51fa14d0 2023-07-14T16:22:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a871b3b80 10 client.17322 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a871b3b80 1 -- 172.21.15.62:0/513293728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f1a6c19fe90 con 0x560b51fa14d0 2023-07-14T16:22:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a54ff9700 1 -- 172.21.15.62:0/513293728 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a680c9080 con 0x560b51fa14d0 2023-07-14T16:22:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.275+0000 7f1a54ff9700 1 -- 172.21.15.62:0/513293728 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a680d0040 con 0x560b51fa14d0 2023-07-14T16:22:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.276+0000 7f1a871b3b80 10 client.17322 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.276+0000 7f1a871b3b80 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.276+0000 7f1a871b3b80 20 client.17322 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:00.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.276+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560b51fb6610 0x560b51fb8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.276+0000 7f1a871b3b80 4 client.17322 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:00.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.276+0000 7f1a871b3b80 1 -- 172.21.15.62:0/513293728 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b51eebeb0 con 0x560b51fb6610 2023-07-14T16:22:00.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.276+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560b51fb6610 0x560b51fb8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.277+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560b51fb6610 0x560b51fb8a20 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a1855305eccd91 server_cookie=826c00ba7799f6bc in_seq=0 out_seq=0 2023-07-14T16:22:00.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.277+0000 7f1a54ff9700 10 client.17322 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:00.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a54ff9700 1 -- 172.21.15.62:0/513293728 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560b51eebeb0 con 0x560b51fb6610 2023-07-14T16:22:00.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a54ff9700 10 client.17322 handle_command_reply: tid=0 2023-07-14T16:22:00.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 client.17322 shutdown 2023-07-14T16:22:00.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 -- 172.21.15.62:0/513293728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa14d0 msgr2=0x560b51faad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa14d0 0x560b51faad60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1a68014c00 tx=0x7f1a68009310 comp rx=0 tx=0).stop 2023-07-14T16:22:00.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 -- 172.21.15.62:0/513293728 shutdown_connections 2023-07-14T16:22:00.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa2c70 0x560b51fb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa14d0 0x560b51faad60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560b51fb6610 0x560b51fb8a20 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/513293728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a54ff9700 0 client.17322 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:00.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 -- 172.21.15.62:0/513293728 >> 172.21.15.62:0/513293728 conn(0x560b51f96da0 msgr2=0x560b51f9b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 -- 172.21.15.62:0/513293728 shutdown_connections 2023-07-14T16:22:00.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.278+0000 7f1a871b3b80 1 -- 172.21.15.62:0/513293728 wait complete. 2023-07-14T16:22:00.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.279+0000 7f1a871b3b80 20 client.17322 trim_cache size 0 max 16384 2023-07-14T16:22:00.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a871b3b80 1 Processor -- start 2023-07-14T16:22:00.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a871b3b80 1 -- start start 2023-07-14T16:22:00.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 0x560b51fb2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fb22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51faab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51faa930 con 0x560b51fa2c70 2023-07-14T16:22:00.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51faa210 con 0x560b51fa1e20 2023-07-14T16:22:00.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51f9cf00 con 0x560b51fa14d0 2023-07-14T16:22:00.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.297+0000 7f1a71d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51faab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a71d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51faab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49482/0 (socket says 172.21.15.62:49482) 2023-07-14T16:22:00.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a71d9c700 1 -- 172.21.15.62:0/1987886642 learned_addr learned my addr 172.21.15.62:0/1987886642 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a7159b700 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 0x560b51fb2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fb22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a71d9c700 1 -- 172.21.15.62:0/1987886642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 msgr2=0x560b51fb2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 0x560b51fb2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a71d9c700 1 -- 172.21.15.62:0/1987886642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 msgr2=0x560b51fb22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fb22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a71d9c700 1 -- 172.21.15.62:0/1987886642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b51d09c60 con 0x560b51fa2c70 2023-07-14T16:22:00.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.298+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51faab10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1a640048e0 tx=0x7f1a64016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6465f2a330e108a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.299+0000 7f1a57fff700 1 -- 172.21.15.62:0/1987886642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64004910 con 0x560b51fa2c70 2023-07-14T16:22:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.299+0000 7f1a57fff700 1 -- 172.21.15.62:0/1987886642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a6400f040 con 0x560b51fa2c70 2023-07-14T16:22:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.299+0000 7f1a57fff700 1 -- 172.21.15.62:0/1987886642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64024750 con 0x560b51fa2c70 2023-07-14T16:22:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.299+0000 7f1a871b3b80 1 -- 172.21.15.62:0/1987886642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 msgr2=0x560b51faab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.299+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51faab10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1a640048e0 tx=0x7f1a64016000 comp rx=0 tx=0).stop 2023-07-14T16:22:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 -- 172.21.15.62:0/1987886642 shutdown_connections 2023-07-14T16:22:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa2c70 0x560b51faab10 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 0x560b51fb2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/1987886642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa1e20 0x560b51fb22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 -- 172.21.15.62:0/1987886642 >> 172.21.15.62:0/1987886642 conn(0x560b51f96da0 msgr2=0x560b51fbd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 -- 172.21.15.62:0/1987886642 shutdown_connections 2023-07-14T16:22:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 -- 172.21.15.62:0/1987886642 wait complete. 2023-07-14T16:22:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 -- start start 2023-07-14T16:22:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.300+0000 7f1a871b3b80 1 Processor -- start 2023-07-14T16:22:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 0x560b51fae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa1e20 0x560b51faea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a871b3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa2c70 0x560b51fb1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51fb1fb0 con 0x560b51fa1e20 2023-07-14T16:22:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51fb2120 con 0x560b51fa2c70 2023-07-14T16:22:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a871b3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b51fb22e0 con 0x560b51fa14d0 2023-07-14T16:22:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a871b3b80 10 client.0 _ll_register_callbacks cb 0x7f1a5c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a70d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa1e20 0x560b51faea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a70d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa1e20 0x560b51faea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49490/0 (socket says 172.21.15.62:49490) 2023-07-14T16:22:00.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a70d9a700 1 -- 172.21.15.62:0/2618258179 learned_addr learned my addr 172.21.15.62:0/2618258179 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa2c70 0x560b51fb1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.301+0000 7f1a7159b700 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 0x560b51fae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a70d9a700 1 -- 172.21.15.62:0/2618258179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 msgr2=0x560b51fae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 0x560b51fae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a70d9a700 1 -- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa2c70 msgr2=0x560b51fb1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa2c70 0x560b51fb1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a70d9a700 1 -- 172.21.15.62:0/2618258179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a64004590 con 0x560b51fa1e20 2023-07-14T16:22:00.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a71d9c700 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa2c70 0x560b51fb1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:00.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a70d9a700 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa1e20 0x560b51faea60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000c00 tx=0x7f1a5c004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5b408e706b4a08 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:00.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:00.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a54ff9700 1 -- 172.21.15.62:0/2618258179 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c016070 con 0x560b51fa1e20 2023-07-14T16:22:00.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a871b3b80 1 -- 172.21.15.62:0/2618258179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a6809ac60 con 0x560b51fa1e20 2023-07-14T16:22:00.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.302+0000 7f1a54ff9700 1 -- 172.21.15.62:0/2618258179 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1a5c003c60 con 0x560b51fa1e20 2023-07-14T16:22:00.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a54ff9700 1 -- 172.21.15.62:0/2618258179 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a5c003e20 con 0x560b51fa1e20 2023-07-14T16:22:00.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a871b3b80 10 client.17328 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:00.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a871b3b80 1 -- 172.21.15.62:0/2618258179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560b51d47250 con 0x560b51fa1e20 2023-07-14T16:22:00.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a54ff9700 1 -- 172.21.15.62:0/2618258179 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a6809ac60 con 0x560b51fa1e20 2023-07-14T16:22:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a54ff9700 1 -- 172.21.15.62:0/2618258179 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a5c011930 con 0x560b51fa1e20 2023-07-14T16:22:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a871b3b80 10 client.17328 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a871b3b80 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a871b3b80 20 client.17328 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560b5200b960 0x560b5200dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a871b3b80 4 client.17328 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:22:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.303+0000 7f1a871b3b80 1 -- 172.21.15.62:0/2618258179 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b51eae050 con 0x560b5200b960 2023-07-14T16:22:00.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.304+0000 7f1a7159b700 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560b5200b960 0x560b5200dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.304+0000 7f1a7159b700 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560b5200b960 0x560b5200dd70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2692dae64c77a359 server_cookie=486a7021e62720b5 in_seq=0 out_seq=0 2023-07-14T16:22:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.304+0000 7f1a54ff9700 10 client.17328 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a54ff9700 1 -- 172.21.15.62:0/2618258179 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560b51eae050 con 0x560b5200b960 2023-07-14T16:22:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a54ff9700 10 client.17328 handle_command_reply: tid=0 2023-07-14T16:22:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 client.17328 shutdown 2023-07-14T16:22:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 -- 172.21.15.62:0/2618258179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa1e20 msgr2=0x560b51faea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa1e20 0x560b51faea60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000c00 tx=0x7f1a5c004420 comp rx=0 tx=0).stop 2023-07-14T16:22:00.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 -- 172.21.15.62:0/2618258179 shutdown_connections 2023-07-14T16:22:00.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560b5200b960 0x560b5200dd70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560b51fa2c70 0x560b51fb1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560b51fa14d0 0x560b51fae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 --2- 172.21.15.62:0/2618258179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b51fa1e20 0x560b51faea60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 -- 172.21.15.62:0/2618258179 >> 172.21.15.62:0/2618258179 conn(0x560b51f96da0 msgr2=0x560b51fb2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a54ff9700 0 client.17328 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:00.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 -- 172.21.15.62:0/2618258179 shutdown_connections 2023-07-14T16:22:00.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 1 -- 172.21.15.62:0/2618258179 wait complete. 2023-07-14T16:22:00.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.305+0000 7f1a871b3b80 20 client.17328 trim_cache size 0 max 16384 2023-07-14T16:22:00.289 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:22:00.289 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:22:00.290 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:22:00.290 INFO:teuthology.orchestra.run.smithi062.stdout: "c5f06f3a-f8a3-4064-a986-246c1a4feef0": { 2023-07-14T16:22:00.290 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:22:00.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "c5f06f3a-f8a3-4064-a986-246c1a4feef0", 2023-07-14T16:22:00.290 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:22:00.290 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:22:00.290 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:22:00.291 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:22:00.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 -- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1a580a11d0 msgr2=0x7f1a580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1a580a11d0 0x7f1a580a3680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009a60 tx=0x7f1a64004ad0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 -- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 msgr2=0x7f1a6c197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:00.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 0x7f1a6c197940 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1a68006010 tx=0x7f1a68003a50 comp rx=0 tx=0).stop 2023-07-14T16:22:00.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 -- 172.21.15.62:0/3530417830 shutdown_connections 2023-07-14T16:22:00.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1a580a11d0 0x7f1a580a3680 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a6c1057a0 0x7f1a6c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a6c100560 0x7f1a6c197940 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 --2- 172.21.15.62:0/3530417830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a6c102e80 0x7f1a6c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:00.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 -- 172.21.15.62:0/3530417830 >> 172.21.15.62:0/3530417830 conn(0x7f1a6c0fa000 msgr2=0x7f1a6c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:00.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 -- 172.21.15.62:0/3530417830 shutdown_connections 2023-07-14T16:22:00.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:00.306+0000 7f1a737ff700 1 -- 172.21.15.62:0/3530417830 wait complete. 2023-07-14T16:22:00.307 INFO:tasks.cephfs.filesystem:scrub status for tag:c5f06f3a-f8a3-4064-a986-246c1a4feef0 - {'path': '/', 'tag': 'c5f06f3a-f8a3-4064-a986-246c1a4feef0', 'options': 'recursive,force'} 2023-07-14T16:22:00.402 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:21:59 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:21:59.982+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v149>,<0x10000000003/client.0 v150>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v162>,<0x10000000001/sv_0 v176>,<0x10000000000/_nogroup v170>,<0x1/volumes v160>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:22:01.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: pgmap v548: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:01.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:22:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:22:01.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:22:01.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:22:01.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:22:01.027 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:22:01.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.028 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:00 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: pgmap v548: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:22:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:22:01.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:22:01.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:22:01.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:22:01.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:22:01.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:00 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: pgmap v548: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:22:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:22:01.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:22:01.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:22:01.154 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:22:01.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:22:01.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:01.155 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:00 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:03.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:02 smithi164 ceph-mon[97427]: pgmap v549: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 21 op/s 2023-07-14T16:22:03.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:02 smithi062 ceph-mon[95543]: pgmap v549: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 21 op/s 2023-07-14T16:22:03.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:02 smithi081 ceph-mon[127795]: pgmap v549: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 21 op/s 2023-07-14T16:22:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:04 smithi164 ceph-mon[97427]: pgmap v550: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:22:05.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:04 smithi062 ceph-mon[95543]: pgmap v550: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:22:05.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:04 smithi081 ceph-mon[127795]: pgmap v550: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:22:07.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:06 smithi164 ceph-mon[97427]: pgmap v551: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:22:07.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:06 smithi062 ceph-mon[95543]: pgmap v551: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:22:07.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:06 smithi081 ceph-mon[127795]: pgmap v551: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:22:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:08 smithi164 ceph-mon[97427]: pgmap v552: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:22:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:08 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:22:09.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:08 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:22:09.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:08 smithi062 ceph-mon[95543]: pgmap v552: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:22:09.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:08 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:22:09.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:08 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:22:09.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:08 smithi081 ceph-mon[127795]: pgmap v552: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:22:09.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:08 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:22:09.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:08 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:22:11.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:10 smithi164 ceph-mon[97427]: pgmap v553: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:22:11.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:10 smithi062 ceph-mon[95543]: pgmap v553: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:22:11.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:10 smithi081 ceph-mon[127795]: pgmap v553: 97 pgs: 97 active+clean; 4.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:22:13.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:12 smithi164 ceph-mon[97427]: pgmap v554: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:22:13.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:12 smithi062 ceph-mon[95543]: pgmap v554: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:22:13.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:12 smithi081 ceph-mon[127795]: pgmap v554: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:22:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.295+0000 7f0608d9d700 1 -- 172.21.15.62:0/1215620873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0604101b60 msgr2=0x7f0604101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.295+0000 7f0608d9d700 1 --2- 172.21.15.62:0/1215620873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0604101b60 0x7f0604101f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009990 tx=0x7f05f4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.295+0000 7f0608d9d700 1 -- 172.21.15.62:0/1215620873 shutdown_connections 2023-07-14T16:22:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.295+0000 7f0608d9d700 1 --2- 172.21.15.62:0/1215620873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f06041036e0 0x7f0604109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.295+0000 7f0608d9d700 1 --2- 172.21.15.62:0/1215620873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0604101b60 0x7f0604101f70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.295+0000 7f0608d9d700 1 --2- 172.21.15.62:0/1215620873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0604102d50 0x7f06041031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.295+0000 7f0608d9d700 1 -- 172.21.15.62:0/1215620873 >> 172.21.15.62:0/1215620873 conn(0x7f06040fd180 msgr2=0x7f06040ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.295+0000 7f0608d9d700 1 -- 172.21.15.62:0/1215620873 shutdown_connections 2023-07-14T16:22:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.296+0000 7f0608d9d700 1 -- 172.21.15.62:0/1215620873 wait complete. 2023-07-14T16:22:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f0608d9d700 1 Processor -- start 2023-07-14T16:22:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f0608d9d700 1 -- start start 2023-07-14T16:22:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f0608d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0604101b60 0x7f0604197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f0608d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0604102d50 0x7f0604197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f0608d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06041036e0 0x7f060419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f0608d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060419f2d0 con 0x7f0604101b60 2023-07-14T16:22:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f0608d9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060419f410 con 0x7f0604102d50 2023-07-14T16:22:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f0608d9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060419f5a0 con 0x7f06041036e0 2023-07-14T16:22:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f06037fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0604101b60 0x7f0604197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f06037fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0604101b60 0x7f0604197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40482/0 (socket says 172.21.15.62:40482) 2023-07-14T16:22:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.297+0000 7f06037fe700 1 -- 172.21.15.62:0/3505006767 learned_addr learned my addr 172.21.15.62:0/3505006767 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.298+0000 7f0602ffd700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0604102d50 0x7f0604197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.298+0000 7f0603fff700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06041036e0 0x7f060419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.298+0000 7f06037fe700 1 -- 172.21.15.62:0/3505006767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06041036e0 msgr2=0x7f060419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.298+0000 7f06037fe700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06041036e0 0x7f060419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.298+0000 7f06037fe700 1 -- 172.21.15.62:0/3505006767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0604102d50 msgr2=0x7f0604197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.298+0000 7f06037fe700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0604102d50 0x7f0604197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.298+0000 7f06037fe700 1 -- 172.21.15.62:0/3505006767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40095f0 con 0x7f0604101b60 2023-07-14T16:22:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.298+0000 7f06037fe700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0604101b60 0x7f0604197750 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f05f4003ec0 tx=0x7f05f4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.299+0000 7f0600ff9700 1 -- 172.21.15.62:0/3505006767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f401d070 con 0x7f0604101b60 2023-07-14T16:22:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.299+0000 7f0608d9d700 1 -- 172.21.15.62:0/3505006767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060419f820 con 0x7f0604101b60 2023-07-14T16:22:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.299+0000 7f0600ff9700 1 -- 172.21.15.62:0/3505006767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05f400be70 con 0x7f0604101b60 2023-07-14T16:22:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.299+0000 7f0600ff9700 1 -- 172.21.15.62:0/3505006767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4022ea0 con 0x7f0604101b60 2023-07-14T16:22:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.299+0000 7f0608d9d700 1 -- 172.21.15.62:0/3505006767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060419fca0 con 0x7f0604101b60 2023-07-14T16:22:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.303+0000 7f0600ff9700 1 -- 172.21.15.62:0/3505006767 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f05f4022670 con 0x7f0604101b60 2023-07-14T16:22:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.304+0000 7f061d9d1b80 1 Processor -- start 2023-07-14T16:22:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.304+0000 7f061d9d1b80 1 -- start start 2023-07-14T16:22:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.304+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc0e986b0 0x55ebc103c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.304+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103ca90 0x55ebc103d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc103dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc0f41c50 con 0x55ebc103ca90 2023-07-14T16:22:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc0f949e0 con 0x55ebc0e986b0 2023-07-14T16:22:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc0f8de70 con 0x55ebc103d7e0 2023-07-14T16:22:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f0600ff9700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05ec0a1150 0x7f05ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f0600ff9700 1 -- 172.21.15.62:0/3505006767 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05f40cdc30 con 0x7f0604101b60 2023-07-14T16:22:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f0603fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc0e986b0 0x55ebc103c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f06037fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103ca90 0x55ebc103d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f0603fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc0e986b0 0x55ebc103c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:38730/0 (socket says 172.21.15.62:38730) 2023-07-14T16:22:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f0603fff700 1 -- 172.21.15.62:0/1072476718 learned_addr learned my addr 172.21.15.62:0/1072476718 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f0603fff700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05ec0a1150 0x7f05ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.305+0000 7f0602ffd700 1 --2- 172.21.15.62:0/1072476718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc103dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f0603fff700 1 -- 172.21.15.62:0/1072476718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 msgr2=0x55ebc103dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f0603fff700 1 --2- 172.21.15.62:0/1072476718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc103dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f0603fff700 1 -- 172.21.15.62:0/1072476718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103ca90 msgr2=0x55ebc103d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f0603fff700 1 --2- 172.21.15.62:0/1072476718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103ca90 0x55ebc103d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f0603fff700 1 -- 172.21.15.62:0/1072476718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebc0dbbd80 con 0x55ebc0e986b0 2023-07-14T16:22:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f0603fff700 1 --2- 172.21.15.62:0/1072476718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc0e986b0 0x55ebc103c550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05fc019500 tx=0x7f05fc019810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4531597bc10f5dc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f0603fff700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05ec0a1150 0x7f05ec0a3600 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f05fc019b60 tx=0x7f05fc01d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f05e9ffb700 1 -- 172.21.15.62:0/1072476718 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc025070 con 0x55ebc0e986b0 2023-07-14T16:22:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f05e9ffb700 1 -- 172.21.15.62:0/1072476718 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05fc019e90 con 0x55ebc0e986b0 2023-07-14T16:22:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.306+0000 7f05e9ffb700 1 -- 172.21.15.62:0/1072476718 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00e040 con 0x55ebc0e986b0 2023-07-14T16:22:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1072476718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc0e986b0 msgr2=0x55ebc103c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1072476718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc0e986b0 0x55ebc103c550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05fc019500 tx=0x7f05fc019810 comp rx=0 tx=0).stop 2023-07-14T16:22:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1072476718 shutdown_connections 2023-07-14T16:22:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1072476718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc103dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1072476718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc0e986b0 0x55ebc103c550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1072476718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103ca90 0x55ebc103d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1072476718 >> 172.21.15.62:0/1072476718 conn(0x55ebc1031d40 msgr2=0x55ebc1036e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1072476718 shutdown_connections 2023-07-14T16:22:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.307+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1072476718 wait complete. 2023-07-14T16:22:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.308+0000 7f061d9d1b80 1 -- start start 2023-07-14T16:22:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.308+0000 7f061d9d1b80 1 Processor -- start 2023-07-14T16:22:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 0x55ebc1045a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc1045fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc104cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc0f2da60 con 0x55ebc103d7e0 2023-07-14T16:22:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc0f2d760 con 0x55ebc103ca90 2023-07-14T16:22:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc0e118c0 con 0x55ebc0e986b0 2023-07-14T16:22:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f061d9d1b80 10 client.0 _ll_register_callbacks cb 0x55ebc0f2b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0603fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc104cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0602ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc1045fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0603fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc104cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40494/0 (socket says 172.21.15.62:40494) 2023-07-14T16:22:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0603fff700 1 -- 172.21.15.62:0/1539707036 learned_addr learned my addr 172.21.15.62:0/1539707036 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f06037fe700 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 0x55ebc1045a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0603fff700 1 -- 172.21.15.62:0/1539707036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 msgr2=0x55ebc1045a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0603fff700 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 0x55ebc1045a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0603fff700 1 -- 172.21.15.62:0/1539707036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 msgr2=0x55ebc1045fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0603fff700 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc1045fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.309+0000 7f0603fff700 1 -- 172.21.15.62:0/1539707036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc012cc0 con 0x55ebc103d7e0 2023-07-14T16:22:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.310+0000 7f0603fff700 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc104cfe0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f05fc012c90 tx=0x7f05fc004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36da49328e8ebba9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.310+0000 7f05e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.310+0000 7f05e2ffd700 1 -- 172.21.15.62:0/1539707036 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc025070 con 0x55ebc103d7e0 2023-07-14T16:22:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.310+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1539707036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebc0ee7490 con 0x55ebc103d7e0 2023-07-14T16:22:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.310+0000 7f05e2ffd700 1 -- 172.21.15.62:0/1539707036 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05fc0123b0 con 0x55ebc103d7e0 2023-07-14T16:22:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.310+0000 7f05e2ffd700 1 -- 172.21.15.62:0/1539707036 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05fc0098a0 con 0x55ebc103d7e0 2023-07-14T16:22:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f061d9d1b80 10 client.17340 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1539707036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ebc0da4c00 con 0x55ebc103d7e0 2023-07-14T16:22:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f05e2ffd700 1 -- 172.21.15.62:0/1539707036 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05fc039020 con 0x55ebc103d7e0 2023-07-14T16:22:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f05e2ffd700 1 -- 172.21.15.62:0/1539707036 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f05fc035480 con 0x55ebc103d7e0 2023-07-14T16:22:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f061d9d1b80 10 client.17340 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f061d9d1b80 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f061d9d1b80 20 client.17340 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1051880 0x55ebc1053c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f061d9d1b80 4 client.17340 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.311+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1539707036 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebc0f86e50 con 0x55ebc1051880 2023-07-14T16:22:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.312+0000 7f06037fe700 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1051880 0x55ebc1053c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.312+0000 7f05e2ffd700 10 client.17340 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.312+0000 7f06037fe700 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1051880 0x55ebc1053c90 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedae99490d1016 server_cookie=4604fc934ab189e9 in_seq=0 out_seq=0 2023-07-14T16:22:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f05e2ffd700 1 -- 172.21.15.62:0/1539707036 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ebc0f86e50 con 0x55ebc1051880 2023-07-14T16:22:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f05e2ffd700 10 client.17340 handle_command_reply: tid=0 2023-07-14T16:22:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 client.17340 shutdown 2023-07-14T16:22:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1539707036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 msgr2=0x55ebc104cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc104cfe0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f05fc012c90 tx=0x7f05fc004d20 comp rx=0 tx=0).stop 2023-07-14T16:22:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1539707036 shutdown_connections 2023-07-14T16:22:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc104cfe0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 0x55ebc1045a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc1051880 0x55ebc1053c90 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/1539707036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc1045fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f05e2ffd700 0 client.17340 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.313+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1539707036 >> 172.21.15.62:0/1539707036 conn(0x55ebc1031d40 msgr2=0x55ebc1036640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.314+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1539707036 shutdown_connections 2023-07-14T16:22:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.314+0000 7f061d9d1b80 1 -- 172.21.15.62:0/1539707036 wait complete. 2023-07-14T16:22:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.314+0000 7f061d9d1b80 20 client.17340 trim_cache size 0 max 16384 2023-07-14T16:22:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.329+0000 7f061d9d1b80 1 Processor -- start 2023-07-14T16:22:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.329+0000 7f061d9d1b80 1 -- start start 2023-07-14T16:22:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 0x55ebc103c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc1044700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc10389a0 con 0x55ebc103d7e0 2023-07-14T16:22:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc10455d0 con 0x55ebc103ca90 2023-07-14T16:22:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc10466e0 con 0x55ebc0e986b0 2023-07-14T16:22:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0602ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0603fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc1044700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0602ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:38748/0 (socket says 172.21.15.62:38748) 2023-07-14T16:22:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0602ffd700 1 -- 172.21.15.62:0/525517196 learned_addr learned my addr 172.21.15.62:0/525517196 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f06037fe700 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 0x55ebc103c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0603fff700 1 -- 172.21.15.62:0/525517196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 msgr2=0x55ebc103c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0603fff700 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 0x55ebc103c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0603fff700 1 -- 172.21.15.62:0/525517196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 msgr2=0x55ebc104cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0603fff700 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.330+0000 7f0603fff700 1 -- 172.21.15.62:0/525517196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebc0de21f0 con 0x55ebc103d7e0 2023-07-14T16:22:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.331+0000 7f0602ffd700 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.331+0000 7f0603fff700 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc1044700 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0357b0 tx=0x7f05fc012240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675c0969f96f748 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.331+0000 7f05ea7fc700 1 -- 172.21.15.62:0/525517196 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc025020 con 0x55ebc103d7e0 2023-07-14T16:22:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.331+0000 7f05ea7fc700 1 -- 172.21.15.62:0/525517196 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05fc039040 con 0x55ebc103d7e0 2023-07-14T16:22:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.331+0000 7f05ea7fc700 1 -- 172.21.15.62:0/525517196 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00e070 con 0x55ebc103d7e0 2023-07-14T16:22:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 -- 172.21.15.62:0/525517196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 msgr2=0x55ebc1044700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc1044700 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0357b0 tx=0x7f05fc012240 comp rx=0 tx=0).stop 2023-07-14T16:22:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 -- 172.21.15.62:0/525517196 shutdown_connections 2023-07-14T16:22:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc103d7e0 0x55ebc1044700 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc0e986b0 0x55ebc103c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/525517196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 -- 172.21.15.62:0/525517196 >> 172.21.15.62:0/525517196 conn(0x55ebc1031d40 msgr2=0x55ebc10585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 -- 172.21.15.62:0/525517196 shutdown_connections 2023-07-14T16:22:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 -- 172.21.15.62:0/525517196 wait complete. 2023-07-14T16:22:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.332+0000 7f061d9d1b80 1 -- start start 2023-07-14T16:22:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f061d9d1b80 1 Processor -- start 2023-07-14T16:22:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc0e986b0 0x55ebc104b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f061d9d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc104bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc104c3c0 con 0x55ebc0e986b0 2023-07-14T16:22:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc104c530 con 0x55ebc103ca90 2023-07-14T16:22:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f061d9d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebc104c6f0 con 0x55ebc103d7e0 2023-07-14T16:22:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f061d9d1b80 10 client.0 _ll_register_callbacks cb 0x7f05f40094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f06037fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc0e986b0 0x55ebc104b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.333+0000 7f0602ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0603fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc104bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0602ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:38760/0 (socket says 172.21.15.62:38760) 2023-07-14T16:22:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0603fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc104bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41558/0 (socket says 172.21.15.62:41558) 2023-07-14T16:22:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0602ffd700 1 -- 172.21.15.62:0/3632253389 learned_addr learned my addr 172.21.15.62:0/3632253389 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0603fff700 1 -- 172.21.15.62:0/3632253389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 msgr2=0x55ebc104b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0603fff700 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0603fff700 1 -- 172.21.15.62:0/3632253389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc0e986b0 msgr2=0x55ebc104b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0603fff700 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc0e986b0 0x55ebc104b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0603fff700 1 -- 172.21.15.62:0/3632253389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f8009be0 con 0x55ebc103d7e0 2023-07-14T16:22:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f0603fff700 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc104bc60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f05fc01aab0 tx=0x7f05fc0123b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf326a1b71181e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.334+0000 7f05e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:22:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.335+0000 7f05e2ffd700 1 -- 172.21.15.62:0/3632253389 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc025070 con 0x55ebc103d7e0 2023-07-14T16:22:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.335+0000 7f061d9d1b80 1 -- 172.21.15.62:0/3632253389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05fc019e90 con 0x55ebc103d7e0 2023-07-14T16:22:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.335+0000 7f05e2ffd700 1 -- 172.21.15.62:0/3632253389 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05fc03a080 con 0x55ebc103d7e0 2023-07-14T16:22:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.335+0000 7f05e2ffd700 1 -- 172.21.15.62:0/3632253389 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05fc00e040 con 0x55ebc103d7e0 2023-07-14T16:22:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.335+0000 7f061d9d1b80 10 client.26917 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.335+0000 7f061d9d1b80 1 -- 172.21.15.62:0/3632253389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ebc0da4c00 con 0x55ebc103d7e0 2023-07-14T16:22:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f05e2ffd700 1 -- 172.21.15.62:0/3632253389 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05fc019e90 con 0x55ebc103d7e0 2023-07-14T16:22:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f05e2ffd700 1 -- 172.21.15.62:0/3632253389 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f05fc003fc0 con 0x55ebc103d7e0 2023-07-14T16:22:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f061d9d1b80 10 client.26917 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f061d9d1b80 10 client.26917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f061d9d1b80 20 client.26917 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc10a5e60 0x55ebc10a8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f061d9d1b80 4 client.26917 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:22:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f061d9d1b80 1 -- 172.21.15.62:0/3632253389 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ebc0f48ff0 con 0x55ebc10a5e60 2023-07-14T16:22:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.336+0000 7f06037fe700 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc10a5e60 0x55ebc10a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.337+0000 7f06037fe700 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc10a5e60 0x55ebc10a8240 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361a8dcdba3e2258 server_cookie=17596d6b02d7bb88 in_seq=0 out_seq=0 2023-07-14T16:22:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.337+0000 7f05e2ffd700 10 client.26917 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f05e2ffd700 1 -- 172.21.15.62:0/3632253389 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ebc0f48ff0 con 0x55ebc10a5e60 2023-07-14T16:22:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f05e2ffd700 10 client.26917 handle_command_reply: tid=0 2023-07-14T16:22:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 client.26917 shutdown 2023-07-14T16:22:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 -- 172.21.15.62:0/3632253389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 msgr2=0x55ebc104bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc104bc60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f05fc01aab0 tx=0x7f05fc0123b0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 -- 172.21.15.62:0/3632253389 shutdown_connections 2023-07-14T16:22:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ebc103d7e0 0x55ebc104bc60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ebc0e986b0 0x55ebc104b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ebc10a5e60 0x55ebc10a8240 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 --2- 172.21.15.62:0/3632253389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ebc103ca90 0x55ebc104b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f05e2ffd700 0 client.26917 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.346+0000 7f061d9d1b80 1 -- 172.21.15.62:0/3632253389 >> 172.21.15.62:0/3632253389 conn(0x55ebc1031d40 msgr2=0x55ebc1045a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.347+0000 7f061d9d1b80 1 -- 172.21.15.62:0/3632253389 shutdown_connections 2023-07-14T16:22:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.347+0000 7f061d9d1b80 1 -- 172.21.15.62:0/3632253389 wait complete. 2023-07-14T16:22:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.347+0000 7f061d9d1b80 20 client.26917 trim_cache size 0 max 16384 2023-07-14T16:22:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:22:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:22:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:22:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:22:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.347+0000 7f0608d9d700 1 -- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05ec0a1150 msgr2=0x7f05ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.347+0000 7f0608d9d700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05ec0a1150 0x7f05ec0a3600 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f05fc019b60 tx=0x7f05fc01d000 comp rx=0 tx=0).stop 2023-07-14T16:22:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.347+0000 7f0608d9d700 1 -- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0604101b60 msgr2=0x7f0604197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.347+0000 7f0608d9d700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0604101b60 0x7f0604197750 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f05f4003ec0 tx=0x7f05f4005740 comp rx=0 tx=0).stop 2023-07-14T16:22:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.348+0000 7f0608d9d700 1 -- 172.21.15.62:0/3505006767 shutdown_connections 2023-07-14T16:22:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.348+0000 7f0608d9d700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06041036e0 0x7f060419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.348+0000 7f0608d9d700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0604101b60 0x7f0604197750 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.348+0000 7f0608d9d700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f05ec0a1150 0x7f05ec0a3600 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.348+0000 7f0608d9d700 1 --2- 172.21.15.62:0/3505006767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0604102d50 0x7f0604197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.348+0000 7f0608d9d700 1 -- 172.21.15.62:0/3505006767 >> 172.21.15.62:0/3505006767 conn(0x7f06040fd180 msgr2=0x7f06041062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.348+0000 7f0608d9d700 1 -- 172.21.15.62:0/3505006767 shutdown_connections 2023-07-14T16:22:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:13.348+0000 7f0608d9d700 1 -- 172.21.15.62:0/3505006767 wait complete. 2023-07-14T16:22:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:14 smithi164 ceph-mon[97427]: pgmap v555: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 30 op/s 2023-07-14T16:22:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:14 smithi062 ceph-mon[95543]: pgmap v555: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 30 op/s 2023-07-14T16:22:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:14 smithi081 ceph-mon[127795]: pgmap v555: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 30 op/s 2023-07-14T16:22:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:16 smithi062 ceph-mon[95543]: pgmap v556: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 19 op/s 2023-07-14T16:22:17.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:16 smithi081 ceph-mon[127795]: pgmap v556: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 19 op/s 2023-07-14T16:22:17.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:16 smithi164 ceph-mon[97427]: pgmap v556: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 19 op/s 2023-07-14T16:22:19.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:18 smithi062 ceph-mon[95543]: pgmap v557: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:22:19.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:18 smithi081 ceph-mon[127795]: pgmap v557: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:22:19.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:18 smithi164 ceph-mon[97427]: pgmap v557: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:22:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:20 smithi062 ceph-mon[95543]: pgmap v558: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:21.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:20 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:22:21.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:20 smithi081 ceph-mon[127795]: pgmap v558: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:21.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:20 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:22:21.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:20 smithi164 ceph-mon[97427]: pgmap v558: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:21.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:20 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:22:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:22:22.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:22:22.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:22:22.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:22:22.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:22:22.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:22:22.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:22:22.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:22:22.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:22:22.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:22:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:22 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi062 to 220.9M 2023-07-14T16:22:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:22 smithi062 ceph-mon[95543]: Unable to set osd_memory_target on smithi062 to 231684812: error parsing value: Value '231684812' is below minimum 939524096 2023-07-14T16:22:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:22 smithi062 ceph-mon[95543]: pgmap v559: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:22:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:22 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi062 to 220.9M 2023-07-14T16:22:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:22 smithi081 ceph-mon[127795]: Unable to set osd_memory_target on smithi062 to 231684812: error parsing value: Value '231684812' is below minimum 939524096 2023-07-14T16:22:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:22 smithi081 ceph-mon[127795]: pgmap v559: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:22:23.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:22 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi062 to 220.9M 2023-07-14T16:22:23.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:22 smithi164 ceph-mon[97427]: Unable to set osd_memory_target on smithi062 to 231684812: error parsing value: Value '231684812' is below minimum 939524096 2023-07-14T16:22:23.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:22 smithi164 ceph-mon[97427]: pgmap v559: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:22:25.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:24 smithi062 ceph-mon[95543]: pgmap v560: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:25.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:24 smithi081 ceph-mon[127795]: pgmap v560: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:25.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:24 smithi164 ceph-mon[97427]: pgmap v560: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:22:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 -- 172.21.15.62:0/584118005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4072f10 msgr2=0x7f6bd4073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 --2- 172.21.15.62:0/584118005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4072f10 0x7f6bd4073380 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00098f0 tx=0x7f6bc0009c00 comp rx=0 tx=0).stop 2023-07-14T16:22:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 -- 172.21.15.62:0/584118005 shutdown_connections 2023-07-14T16:22:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 --2- 172.21.15.62:0/584118005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd41079b0 0x7f6bd4109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 --2- 172.21.15.62:0/584118005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd40744f0 0x7f6bd4072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 --2- 172.21.15.62:0/584118005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4072f10 0x7f6bd4073380 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 -- 172.21.15.62:0/584118005 >> 172.21.15.62:0/584118005 conn(0x7f6bd40fb120 msgr2=0x7f6bd40fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 -- 172.21.15.62:0/584118005 shutdown_connections 2023-07-14T16:22:26.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.521+0000 7f6bdadff700 1 -- 172.21.15.62:0/584118005 wait complete. 2023-07-14T16:22:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.522+0000 7f6bdadff700 1 Processor -- start 2023-07-14T16:22:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.522+0000 7f6bdadff700 1 -- start start 2023-07-14T16:22:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.522+0000 7f6bdadff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd4072f10 0x7f6bd4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bdadff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd40744f0 0x7f6bd4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bdadff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd41079b0 0x7f6bd4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bdadff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd41994f0 con 0x7f6bd40744f0 2023-07-14T16:22:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bdadff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd4199660 con 0x7f6bd4072f10 2023-07-14T16:22:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bdadff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd4199820 con 0x7f6bd41079b0 2023-07-14T16:22:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd40744f0 0x7f6bd4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd40744f0 0x7f6bd4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38838/0 (socket says 172.21.15.62:38838) 2023-07-14T16:22:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd8b9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd4072f10 0x7f6bd4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd3fff700 1 -- 172.21.15.62:0/3462473163 learned_addr learned my addr 172.21.15.62:0/3462473163 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd939c700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd41079b0 0x7f6bd4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd8b9b700 1 -- 172.21.15.62:0/3462473163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd41079b0 msgr2=0x7f6bd4198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd41079b0 0x7f6bd4198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd8b9b700 1 -- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd40744f0 msgr2=0x7f6bd4198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd40744f0 0x7f6bd4198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd8b9b700 1 -- 172.21.15.62:0/3462473163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc0009520 con 0x7f6bd4072f10 2023-07-14T16:22:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.523+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd40744f0 0x7f6bd4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.524+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd4072f10 0x7f6bd4197ad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800e9c0 tx=0x7f6bc800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.524+0000 7f6bd1ffb700 1 -- 172.21.15.62:0/3462473163 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800c9c0 con 0x7f6bd4072f10 2023-07-14T16:22:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.524+0000 7f6bdadff700 1 -- 172.21.15.62:0/3462473163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd4199b00 con 0x7f6bd4072f10 2023-07-14T16:22:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.524+0000 7f6bdadff700 1 -- 172.21.15.62:0/3462473163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd41a02c0 con 0x7f6bd4072f10 2023-07-14T16:22:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.524+0000 7f6bd1ffb700 1 -- 172.21.15.62:0/3462473163 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bc8010430 con 0x7f6bd4072f10 2023-07-14T16:22:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.524+0000 7f6bd1ffb700 1 -- 172.21.15.62:0/3462473163 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8018680 con 0x7f6bd4072f10 2023-07-14T16:22:26.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.528+0000 7f6bd1ffb700 1 -- 172.21.15.62:0/3462473163 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6bc800cb20 con 0x7f6bd4072f10 2023-07-14T16:22:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.529+0000 7f6bd1ffb700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6bc40a1150 0x7f6bc40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.529+0000 7f6bd1ffb700 1 -- 172.21.15.62:0/3462473163 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6bc8014070 con 0x7f6bd4072f10 2023-07-14T16:22:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.530+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6bc40a1150 0x7f6bc40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.530+0000 7f6bee6d8b80 1 Processor -- start 2023-07-14T16:22:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.530+0000 7f6bee6d8b80 1 -- start start 2023-07-14T16:22:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.530+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6bc40a1150 0x7f6bc40a3600 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000f9d0 tx=0x7f6bc0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2c586b0 0x55a3c2dfc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2dfd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2dfdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2d01c50 con 0x55a3c2c586b0 2023-07-14T16:22:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2d549e0 con 0x55a3c2dfc840 2023-07-14T16:22:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2d4de70 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd8b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2c586b0 0x55a3c2dfc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd8b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2c586b0 0x55a3c2dfc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38854/0 (socket says 172.21.15.62:38854) 2023-07-14T16:22:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd8b9b700 1 -- 172.21.15.62:0/4190534059 learned_addr learned my addr 172.21.15.62:0/4190534059 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2dfd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd939c700 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2dfdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd8b9b700 1 -- 172.21.15.62:0/4190534059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfd6e0 msgr2=0x55a3c2dfdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2dfdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd8b9b700 1 -- 172.21.15.62:0/4190534059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 msgr2=0x55a3c2dfd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.531+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2dfd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.532+0000 7f6bd8b9b700 1 -- 172.21.15.62:0/4190534059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c2b7bd80 con 0x55a3c2c586b0 2023-07-14T16:22:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.532+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2c586b0 0x55a3c2dfc300 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80999b0 tx=0x7f6bc80998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439cc5fd44a2fe15 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.532+0000 7f6bbeffd700 1 -- 172.21.15.62:0/4190534059 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8099cb0 con 0x55a3c2c586b0 2023-07-14T16:22:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.532+0000 7f6bbeffd700 1 -- 172.21.15.62:0/4190534059 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bc809dce0 con 0x55a3c2c586b0 2023-07-14T16:22:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.532+0000 7f6bbeffd700 1 -- 172.21.15.62:0/4190534059 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80a1c70 con 0x55a3c2c586b0 2023-07-14T16:22:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/4190534059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2c586b0 msgr2=0x55a3c2dfc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2c586b0 0x55a3c2dfc300 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80999b0 tx=0x7f6bc80998c0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/4190534059 shutdown_connections 2023-07-14T16:22:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2dfdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2c586b0 0x55a3c2dfc300 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/4190534059 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2dfd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/4190534059 >> 172.21.15.62:0/4190534059 conn(0x55a3c2df1be0 msgr2=0x55a3c2df6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/4190534059 shutdown_connections 2023-07-14T16:22:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.533+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/4190534059 wait complete. 2023-07-14T16:22:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.534+0000 7f6bee6d8b80 1 -- start start 2023-07-14T16:22:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.534+0000 7f6bee6d8b80 1 Processor -- start 2023-07-14T16:22:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2c586b0 0x55a3c2e05810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfc840 0x55a3c2e05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2ceda60 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2ced760 con 0x55a3c2c586b0 2023-07-14T16:22:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2bd18c0 con 0x55a3c2dfc840 2023-07-14T16:22:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bee6d8b80 10 client.0 _ll_register_callbacks cb 0x55a3c2ceb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38862/0 (socket says 172.21.15.62:38862) 2023-07-14T16:22:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfc840 0x55a3c2e05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd8b9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2c586b0 0x55a3c2e05810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd939c700 1 -- 172.21.15.62:0/3335341497 learned_addr learned my addr 172.21.15.62:0/3335341497 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd939c700 1 -- 172.21.15.62:0/3335341497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfc840 msgr2=0x55a3c2e05d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd939c700 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfc840 0x55a3c2e05d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd939c700 1 -- 172.21.15.62:0/3335341497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2c586b0 msgr2=0x55a3c2e05810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.535+0000 7f6bd939c700 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2c586b0 0x55a3c2e05810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bd939c700 1 -- 172.21.15.62:0/3335341497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd41a02c0 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2c586b0 0x55a3c2e05810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bd939c700 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0cd90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc011e80 tx=0x7f6bcc014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b41fca1455b1fcd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bb3fff700 1 -- 172.21.15.62:0/3335341497 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00e040 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/3335341497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3c2ca7490 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bb3fff700 1 -- 172.21.15.62:0/3335341497 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6bcc01a070 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bb3fff700 1 -- 172.21.15.62:0/3335341497 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bcc01e420 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bee6d8b80 10 client.17358 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.536+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/3335341497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a3c2b64c00 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bb3fff700 1 -- 172.21.15.62:0/3335341497 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a3c2ca7490 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bb3fff700 1 -- 172.21.15.62:0/3335341497 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6bcc016500 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bee6d8b80 10 client.17358 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bee6d8b80 10 client.17358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bee6d8b80 20 client.17358 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a3c2e116d0 0x55a3c2e13ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bee6d8b80 4 client.17358 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/3335341497 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3c2d46e50 con 0x55a3c2e116d0 2023-07-14T16:22:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.537+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a3c2e116d0 0x55a3c2e13ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.538+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a3c2e116d0 0x55a3c2e13ae0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ddbf6a9310964f server_cookie=f9cf0d8aa9cbdd8f in_seq=0 out_seq=0 2023-07-14T16:22:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.538+0000 7f6bb3fff700 10 client.17358 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bb3fff700 1 -- 172.21.15.62:0/3335341497 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a3c2d46e50 con 0x55a3c2e116d0 2023-07-14T16:22:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bb3fff700 10 client.17358 handle_command_reply: tid=0 2023-07-14T16:22:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 client.17358 shutdown 2023-07-14T16:22:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/3335341497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 msgr2=0x55a3c2e0cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0cd90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc011e80 tx=0x7f6bcc014600 comp rx=0 tx=0).stop 2023-07-14T16:22:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/3335341497 shutdown_connections 2023-07-14T16:22:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0cd90 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a3c2e116d0 0x55a3c2e13ae0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2c586b0 0x55a3c2e05810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/3335341497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2dfc840 0x55a3c2e05d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/3335341497 >> 172.21.15.62:0/3335341497 conn(0x55a3c2df1be0 msgr2=0x55a3c2df64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bb3fff700 0 client.17358 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/3335341497 shutdown_connections 2023-07-14T16:22:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.539+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/3335341497 wait complete. 2023-07-14T16:22:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.540+0000 7f6bee6d8b80 20 client.17358 trim_cache size 0 max 16384 2023-07-14T16:22:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.555+0000 7f6bee6d8b80 1 Processor -- start 2023-07-14T16:22:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.555+0000 7f6bee6d8b80 1 -- start start 2023-07-14T16:22:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.555+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e00590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.555+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.555+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e00ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.555+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2e066b0 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.555+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2df7b80 con 0x55a3c2dfc840 2023-07-14T16:22:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.555+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2df3d00 con 0x55a3c2c586b0 2023-07-14T16:22:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e00ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e0cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:42204/0 (socket says 172.21.15.62:42204) 2023-07-14T16:22:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 -- 172.21.15.62:0/390653469 learned_addr learned my addr 172.21.15.62:0/390653469 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e00ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38872/0 (socket says 172.21.15.62:38872) 2023-07-14T16:22:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e00590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 -- 172.21.15.62:0/390653469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 msgr2=0x55a3c2e00590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e00590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 -- 172.21.15.62:0/390653469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 msgr2=0x55a3c2e00ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e00ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 -- 172.21.15.62:0/390653469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c2ba21f0 con 0x55a3c2dfc840 2023-07-14T16:22:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e00590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.556+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e0cd50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009230 tx=0x7f6bc001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=703defeea3b021d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.557+0000 7f6bbf7fe700 1 -- 172.21.15.62:0/390653469 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc002c050 con 0x55a3c2dfc840 2023-07-14T16:22:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.557+0000 7f6bbf7fe700 1 -- 172.21.15.62:0/390653469 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bc002d040 con 0x55a3c2dfc840 2023-07-14T16:22:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.557+0000 7f6bbf7fe700 1 -- 172.21.15.62:0/390653469 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0003c80 con 0x55a3c2dfc840 2023-07-14T16:22:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.557+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/390653469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 msgr2=0x55a3c2e0cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.557+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e0cd50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009230 tx=0x7f6bc001f000 comp rx=0 tx=0).stop 2023-07-14T16:22:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/390653469 shutdown_connections 2023-07-14T16:22:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e00ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e00590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/390653469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e0cd50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/390653469 >> 172.21.15.62:0/390653469 conn(0x55a3c2df1be0 msgr2=0x55a3c2e03a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/390653469 shutdown_connections 2023-07-14T16:22:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/390653469 wait complete. 2023-07-14T16:22:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 -- start start 2023-07-14T16:22:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.558+0000 7f6bee6d8b80 1 Processor -- start 2023-07-14T16:22:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e08300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e08840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bee6d8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2e0bda0 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2e0bf10 con 0x55a3c2dfc840 2023-07-14T16:22:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bee6d8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c2e0c0d0 con 0x55a3c2c586b0 2023-07-14T16:22:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bee6d8b80 10 client.0 _ll_register_callbacks cb 0x7f6bcc0117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bd939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bd939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38874/0 (socket says 172.21.15.62:38874) 2023-07-14T16:22:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bd8b9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e08300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bd939c700 1 -- 172.21.15.62:0/1667994567 learned_addr learned my addr 172.21.15.62:0/1667994567 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.559+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e08840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bd939c700 1 -- 172.21.15.62:0/1667994567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 msgr2=0x55a3c2e08300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bd939c700 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e08300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bd939c700 1 -- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 msgr2=0x55a3c2e08840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bd939c700 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e08840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bd939c700 1 -- 172.21.15.62:0/1667994567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd410aa50 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bd3fff700 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e08840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bd939c700 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0b860 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc012010 tx=0x7f6bcc006080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b5bfe13dbda8d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bb3fff700 1 -- 172.21.15.62:0/1667994567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00e040 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.560+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/1667994567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bc809dce0 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bb3fff700 1 -- 172.21.15.62:0/1667994567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6bcc01a040 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bb3fff700 1 -- 172.21.15.62:0/1667994567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bcc003a00 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bee6d8b80 10 client.17364 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/1667994567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a3c2b7a250 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bb3fff700 1 -- 172.21.15.62:0/1667994567 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bc809dce0 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bb3fff700 1 -- 172.21.15.62:0/1667994567 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6bcc005860 con 0x55a3c2dfd6e0 2023-07-14T16:22:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bee6d8b80 10 client.17364 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bee6d8b80 10 client.17364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bee6d8b80 20 client.17364 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a3c2e65ce0 0x55a3c2e680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bee6d8b80 4 client.17364 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:22:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.561+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/1667994567 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a3c2d08ff0 con 0x55a3c2e65ce0 2023-07-14T16:22:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.562+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a3c2e65ce0 0x55a3c2e680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.562+0000 7f6bd8b9b700 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a3c2e65ce0 0x55a3c2e680f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa17f7798e94884a server_cookie=7cacd701fbfdf4b5 in_seq=0 out_seq=0 2023-07-14T16:22:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.562+0000 7f6bb3fff700 10 client.17364 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bb3fff700 1 -- 172.21.15.62:0/1667994567 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a3c2d08ff0 con 0x55a3c2e65ce0 2023-07-14T16:22:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bb3fff700 10 client.17364 handle_command_reply: tid=0 2023-07-14T16:22:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 client.17364 shutdown 2023-07-14T16:22:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/1667994567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 msgr2=0x55a3c2e0b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0b860 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc012010 tx=0x7f6bcc006080 comp rx=0 tx=0).stop 2023-07-14T16:22:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/1667994567 shutdown_connections 2023-07-14T16:22:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a3c2dfd6e0 0x55a3c2e0b860 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3c2c586b0 0x55a3c2e08300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a3c2e65ce0 0x55a3c2e680f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 --2- 172.21.15.62:0/1667994567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a3c2dfc840 0x55a3c2e08840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bb3fff700 0 client.17364 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/1667994567 >> 172.21.15.62:0/1667994567 conn(0x55a3c2df1be0 msgr2=0x55a3c2e00590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/1667994567 shutdown_connections 2023-07-14T16:22:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.571+0000 7f6bee6d8b80 1 -- 172.21.15.62:0/1667994567 wait complete. 2023-07-14T16:22:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bee6d8b80 20 client.17364 trim_cache size 0 max 16384 2023-07-14T16:22:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:22:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:22:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:22:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:22:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 -- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6bc40a1150 msgr2=0x7f6bc40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6bc40a1150 0x7f6bc40a3600 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000f9d0 tx=0x7f6bc0019040 comp rx=0 tx=0).stop 2023-07-14T16:22:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 -- 172.21.15.62:0/3462473163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd4072f10 msgr2=0x7f6bd4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd4072f10 0x7f6bd4197ad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800e9c0 tx=0x7f6bc800ecd0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 -- 172.21.15.62:0/3462473163 shutdown_connections 2023-07-14T16:22:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bd41079b0 0x7f6bd4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6bc40a1150 0x7f6bc40a3600 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd4072f10 0x7f6bd4197ad0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 --2- 172.21.15.62:0/3462473163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd40744f0 0x7f6bd4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 -- 172.21.15.62:0/3462473163 >> 172.21.15.62:0/3462473163 conn(0x7f6bd40fb120 msgr2=0x7f6bd40fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.572+0000 7f6bdadff700 1 -- 172.21.15.62:0/3462473163 shutdown_connections 2023-07-14T16:22:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:26.573+0000 7f6bdadff700 1 -- 172.21.15.62:0/3462473163 wait complete. 2023-07-14T16:22:27.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:26 smithi062 ceph-mon[95543]: pgmap v561: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 12 op/s 2023-07-14T16:22:27.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:26 smithi081 ceph-mon[127795]: pgmap v561: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 12 op/s 2023-07-14T16:22:27.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:26 smithi164 ceph-mon[97427]: pgmap v561: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 12 op/s 2023-07-14T16:22:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:28 smithi062 ceph-mon[95543]: pgmap v562: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 11 op/s 2023-07-14T16:22:29.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:28 smithi081 ceph-mon[127795]: pgmap v562: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 11 op/s 2023-07-14T16:22:29.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:28 smithi164 ceph-mon[97427]: pgmap v562: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 11 op/s 2023-07-14T16:22:30.308 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:22:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 -- 172.21.15.62:0/1024512965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 msgr2=0x7fb38c0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 --2- 172.21.15.62:0/1024512965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 0x7fb38c0ffe00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009990 tx=0x7fb37c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 -- 172.21.15.62:0/1024512965 shutdown_connections 2023-07-14T16:22:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 --2- 172.21.15.62:0/1024512965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb38c101630 0x7fb38c107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 --2- 172.21.15.62:0/1024512965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 0x7fb38c0ffe00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 --2- 172.21.15.62:0/1024512965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb38c100c80 0x7fb38c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 -- 172.21.15.62:0/1024512965 >> 172.21.15.62:0/1024512965 conn(0x7fb38c0fb010 msgr2=0x7fb38c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 -- 172.21.15.62:0/1024512965 shutdown_connections 2023-07-14T16:22:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.562+0000 7fb38bfff700 1 -- 172.21.15.62:0/1024512965 wait complete. 2023-07-14T16:22:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.563+0000 7fb38bfff700 1 Processor -- start 2023-07-14T16:22:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.563+0000 7fb38bfff700 1 -- start start 2023-07-14T16:22:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 0x7fb38c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb38c100c80 0x7fb38c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb38c101630 0x7fb38c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38bfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c19f290 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38bfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c19f3d0 con 0x7fb38c101630 2023-07-14T16:22:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38bfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c19f560 con 0x7fb38c100c80 2023-07-14T16:22:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 0x7fb38c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 0x7fb38c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38890/0 (socket says 172.21.15.62:38890) 2023-07-14T16:22:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38affd700 1 -- 172.21.15.62:0/3747354314 learned_addr learned my addr 172.21.15.62:0/3747354314 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38b7fe700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb38c101630 0x7fb38c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.564+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb38c100c80 0x7fb38c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb38affd700 1 -- 172.21.15.62:0/3747354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb38c100c80 msgr2=0x7fb38c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb38affd700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb38c100c80 0x7fb38c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb38affd700 1 -- 172.21.15.62:0/3747354314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb38c101630 msgr2=0x7fb38c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb38affd700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb38c101630 0x7fb38c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb38affd700 1 -- 172.21.15.62:0/3747354314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0095f0 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb38affd700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 0x7fb38c197710 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0098e0 tx=0x7fb37c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:30.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb373fff700 1 -- 172.21.15.62:0/3747354314 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c01d070 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb373fff700 1 -- 172.21.15.62:0/3747354314 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb37c00be30 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb38bfff700 1 -- 172.21.15.62:0/3747354314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c19f7e0 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb373fff700 1 -- 172.21.15.62:0/3747354314 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c022e20 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.565+0000 7fb38bfff700 1 -- 172.21.15.62:0/3747354314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c19fc60 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.570+0000 7fb373fff700 1 -- 172.21.15.62:0/3747354314 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb37c022470 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.570+0000 7fb3a4f03b80 1 Processor -- start 2023-07-14T16:22:30.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.571+0000 7fb3a4f03b80 1 -- start start 2023-07-14T16:22:30.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.571+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c1ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.571+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c1b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.571+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c1c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.571+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575b1fcb0 con 0x559575c1b070 2023-07-14T16:22:30.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.571+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575b72a40 con 0x559575c1be30 2023-07-14T16:22:30.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.571+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575b6bed0 con 0x559575c1a720 2023-07-14T16:22:30.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c1b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c1c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c1ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c1ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33964/0 (socket says 172.21.15.62:33964) 2023-07-14T16:22:30.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38b7fe700 1 -- 172.21.15.62:0/578237660 learned_addr learned my addr 172.21.15.62:0/578237660 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:30.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb373fff700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3740a1270 0x7fb3740a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb373fff700 1 -- 172.21.15.62:0/3747354314 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb37c0cde90 con 0x7fb38c0ff9f0 2023-07-14T16:22:30.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38a7fc700 1 -- 172.21.15.62:0/578237660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 msgr2=0x559575c1ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:30.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/578237660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c1ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38a7fc700 1 -- 172.21.15.62:0/578237660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 msgr2=0x559575c1b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:30.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/578237660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c1b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38a7fc700 1 -- 172.21.15.62:0/578237660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559575999de0 con 0x559575c1be30 2023-07-14T16:22:30.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.572+0000 7fb38b7fe700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3740a1270 0x7fb3740a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.573+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/578237660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c1c2a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb380009c00 tx=0x7fb380009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b6f481d001365d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:30.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.573+0000 7fb3717fa700 1 -- 172.21.15.62:0/578237660 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380011b70 con 0x559575c1be30 2023-07-14T16:22:30.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.573+0000 7fb3717fa700 1 -- 172.21.15.62:0/578237660 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb380011cd0 con 0x559575c1be30 2023-07-14T16:22:30.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.573+0000 7fb3717fa700 1 -- 172.21.15.62:0/578237660 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38000f6a0 con 0x559575c1be30 2023-07-14T16:22:30.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/578237660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 msgr2=0x559575c1c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/578237660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c1c2a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb380009c00 tx=0x7fb380009f10 comp rx=0 tx=0).stop 2023-07-14T16:22:30.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb38b7fe700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3740a1270 0x7fb3740a3720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb38400fb80 tx=0x7fb384004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:30.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/578237660 shutdown_connections 2023-07-14T16:22:30.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/578237660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c1c2a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/578237660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c1ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/578237660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c1b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/578237660 >> 172.21.15.62:0/578237660 conn(0x559575c0fff0 msgr2=0x559575c15460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:30.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/578237660 shutdown_connections 2023-07-14T16:22:30.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.574+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/578237660 wait complete. 2023-07-14T16:22:30.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.575+0000 7fb3a4f03b80 1 -- start start 2023-07-14T16:22:30.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.575+0000 7fb3a4f03b80 1 Processor -- start 2023-07-14T16:22:30.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.575+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1a720 0x559575c24310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.575+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1b070 0x559575c24850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.575+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1be30 0x559575c2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.575+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575b0bac0 con 0x559575c1be30 2023-07-14T16:22:30.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.575+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575b0b7c0 con 0x559575c1a720 2023-07-14T16:22:30.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.575+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595759ef920 con 0x559575c1b070 2023-07-14T16:22:30.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb3a4f03b80 10 client.0 _ll_register_callbacks cb 0x559575b09760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:30.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1be30 0x559575c2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1be30 0x559575c2b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38902/0 (socket says 172.21.15.62:38902) 2023-07-14T16:22:30.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1a720 0x559575c24310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1b070 0x559575c24850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38b7fe700 1 -- 172.21.15.62:0/3937516198 learned_addr learned my addr 172.21.15.62:0/3937516198 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:30.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38a7fc700 1 -- 172.21.15.62:0/3937516198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1a720 msgr2=0x559575c24310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:30.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1a720 0x559575c24310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38a7fc700 1 -- 172.21.15.62:0/3937516198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1be30 msgr2=0x559575c2b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:30.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1be30 0x559575c2b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38a7fc700 1 -- 172.21.15.62:0/3937516198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb380009830 con 0x559575c1b070 2023-07-14T16:22:30.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.576+0000 7fb38affd700 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1a720 0x559575c24310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:30.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.577+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1b070 0x559575c24850 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb38001c040 tx=0x7fb380012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c9e443a62aa597a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:30.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.577+0000 7fb36eff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:22:30.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.577+0000 7fb36eff5700 1 -- 172.21.15.62:0/3937516198 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380011e40 con 0x559575c1b070 2023-07-14T16:22:30.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.577+0000 7fb36eff5700 1 -- 172.21.15.62:0/3937516198 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb380005050 con 0x559575c1b070 2023-07-14T16:22:30.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.577+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/3937516198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559575ac54f0 con 0x559575c1b070 2023-07-14T16:22:30.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.577+0000 7fb36eff5700 1 -- 172.21.15.62:0/3937516198 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb380005230 con 0x559575c1b070 2023-07-14T16:22:30.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.577+0000 7fb3a4f03b80 10 client.26947 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:30.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.577+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/3937516198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559575982c60 con 0x559575c1b070 2023-07-14T16:22:30.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.578+0000 7fb36eff5700 1 -- 172.21.15.62:0/3937516198 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb380033020 con 0x559575c1b070 2023-07-14T16:22:30.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.578+0000 7fb36eff5700 1 -- 172.21.15.62:0/3937516198 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb380022070 con 0x559575c1b070 2023-07-14T16:22:30.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.578+0000 7fb3a4f03b80 10 client.26947 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:30.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.578+0000 7fb3a4f03b80 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:30.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.578+0000 7fb3a4f03b80 20 client.26947 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:30.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.578+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559575c2ffe0 0x559575c323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.578+0000 7fb3a4f03b80 4 client.26947 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.578+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/3937516198 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559575b64eb0 con 0x559575c2ffe0 2023-07-14T16:22:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.579+0000 7fb38affd700 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559575c2ffe0 0x559575c323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.579+0000 7fb38affd700 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559575c2ffe0 0x559575c323f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad0d53f5d7d3e5cf server_cookie=91655186e00406b5 in_seq=0 out_seq=0 2023-07-14T16:22:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.579+0000 7fb36eff5700 10 client.26947 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.580+0000 7fb36eff5700 1 -- 172.21.15.62:0/3937516198 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559575b64eb0 con 0x559575c2ffe0 2023-07-14T16:22:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.580+0000 7fb36eff5700 10 client.26947 handle_command_reply: tid=0 2023-07-14T16:22:30.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.580+0000 7fb3a4f03b80 1 client.26947 shutdown 2023-07-14T16:22:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/3937516198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1b070 msgr2=0x559575c24850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1b070 0x559575c24850 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb38001c040 tx=0x7fb380012680 comp rx=0 tx=0).stop 2023-07-14T16:22:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/3937516198 shutdown_connections 2023-07-14T16:22:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1be30 0x559575c2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559575c2ffe0 0x559575c323f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1a720 0x559575c24310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb36eff5700 0 client.26947 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/3937516198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1b070 0x559575c24850 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/3937516198 >> 172.21.15.62:0/3937516198 conn(0x559575c0fff0 msgr2=0x559575c14b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/3937516198 shutdown_connections 2023-07-14T16:22:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/3937516198 wait complete. 2023-07-14T16:22:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.581+0000 7fb3a4f03b80 20 client.26947 trim_cache size 0 max 16384 2023-07-14T16:22:30.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.600+0000 7fb3a4f03b80 1 Processor -- start 2023-07-14T16:22:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.601+0000 7fb3a4f03b80 1 -- start start 2023-07-14T16:22:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.601+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1a720 0x559575c221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.601+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1b070 0x559575c2b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.601+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1be30 0x559575c1ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.601+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575c24f90 con 0x559575c1a720 2023-07-14T16:22:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.601+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575c21f70 con 0x559575c1b070 2023-07-14T16:22:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.601+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575c25210 con 0x559575c1be30 2023-07-14T16:22:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1a720 0x559575c221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1a720 0x559575c221c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38906/0 (socket says 172.21.15.62:38906) 2023-07-14T16:22:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 -- 172.21.15.62:0/2895381542 learned_addr learned my addr 172.21.15.62:0/2895381542 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38b7fe700 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1be30 0x559575c1ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1b070 0x559575c2b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 -- 172.21.15.62:0/2895381542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1be30 msgr2=0x559575c1ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1be30 0x559575c1ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 -- 172.21.15.62:0/2895381542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1b070 msgr2=0x559575c2b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1b070 0x559575c2b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 -- 172.21.15.62:0/2895381542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595759c0250 con 0x559575c1a720 2023-07-14T16:22:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.602+0000 7fb38affd700 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1a720 0x559575c221c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0cb940 tx=0x7fb37c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb49b7cbdc6058dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.603+0000 7fb371ffb700 1 -- 172.21.15.62:0/2895381542 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c09b2e0 con 0x559575c1a720 2023-07-14T16:22:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.603+0000 7fb371ffb700 1 -- 172.21.15.62:0/2895381542 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb37c09f310 con 0x559575c1a720 2023-07-14T16:22:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.603+0000 7fb371ffb700 1 -- 172.21.15.62:0/2895381542 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c09f030 con 0x559575c1a720 2023-07-14T16:22:30.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.603+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/2895381542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1a720 msgr2=0x559575c221c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.603+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1a720 0x559575c221c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0cb940 tx=0x7fb37c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.604+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/2895381542 shutdown_connections 2023-07-14T16:22:30.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.604+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1be30 0x559575c1ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.604+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1a720 0x559575c221c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.604+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/2895381542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1b070 0x559575c2b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.604+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/2895381542 >> 172.21.15.62:0/2895381542 conn(0x559575c0fff0 msgr2=0x559575c23010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:30.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.604+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/2895381542 shutdown_connections 2023-07-14T16:22:30.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.604+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/2895381542 wait complete. 2023-07-14T16:22:30.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.604+0000 7fb3a4f03b80 1 -- start start 2023-07-14T16:22:30.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.605+0000 7fb3a4f03b80 1 Processor -- start 2023-07-14T16:22:30.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.605+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.605+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c27ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.605+0000 7fb3a4f03b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c2aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.605+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575c2b020 con 0x559575c1b070 2023-07-14T16:22:30.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.605+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575c2b190 con 0x559575c1be30 2023-07-14T16:22:30.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.605+0000 7fb3a4f03b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559575c2b350 con 0x559575c1a720 2023-07-14T16:22:30.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.606+0000 7fb3a4f03b80 10 client.0 _ll_register_callbacks cb 0x7fb37c09b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:30.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.606+0000 7fb38a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c27ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.606+0000 7fb38b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c2aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.606+0000 7fb38affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.606+0000 7fb38b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c2aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:42296/0 (socket says 172.21.15.62:42296) 2023-07-14T16:22:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.606+0000 7fb38b7fe700 1 -- 172.21.15.62:0/782562173 learned_addr learned my addr 172.21.15.62:0/782562173 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.606+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c27ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38916/0 (socket says 172.21.15.62:38916) 2023-07-14T16:22:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.607+0000 7fb38affd700 1 -- 172.21.15.62:0/782562173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 msgr2=0x559575c2aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.607+0000 7fb38affd700 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c2aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.607+0000 7fb38affd700 1 -- 172.21.15.62:0/782562173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 msgr2=0x559575c27ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.607+0000 7fb38affd700 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c27ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.607+0000 7fb38affd700 1 -- 172.21.15.62:0/782562173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c19fc60 con 0x559575c1a720 2023-07-14T16:22:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.607+0000 7fb38affd700 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c27580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x559575c25510 tx=0x7fb37c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c80e97d2cbebf80b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.607+0000 7fb36eff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:22:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.608+0000 7fb36eff5700 1 -- 172.21.15.62:0/782562173 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0cba50 con 0x559575c1a720 2023-07-14T16:22:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.608+0000 7fb36eff5700 1 -- 172.21.15.62:0/782562173 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb37c0a10d0 con 0x559575c1a720 2023-07-14T16:22:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.608+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/782562173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb380011cd0 con 0x559575c1a720 2023-07-14T16:22:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.608+0000 7fb36eff5700 1 -- 172.21.15.62:0/782562173 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb37c0a3210 con 0x559575c1a720 2023-07-14T16:22:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.608+0000 7fb3a4f03b80 10 client.26953 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.608+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/782562173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5595759982b0 con 0x559575c1a720 2023-07-14T16:22:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb36eff5700 1 -- 172.21.15.62:0/782562173 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb37c0b7080 con 0x559575c1a720 2023-07-14T16:22:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb36eff5700 1 -- 172.21.15.62:0/782562173 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb37c0970d0 con 0x559575c1a720 2023-07-14T16:22:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb3a4f03b80 10 client.26953 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb3a4f03b80 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb3a4f03b80 20 client.26953 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559575c8d030 0x559575c8f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb3a4f03b80 4 client.26953 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:22:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/782562173 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559575b27050 con 0x559575c8d030 2023-07-14T16:22:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.609+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559575c8d030 0x559575c8f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.610+0000 7fb38a7fc700 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559575c8d030 0x559575c8f440 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8354dd1263dcf44 server_cookie=b5b6365d4480571c in_seq=0 out_seq=0 2023-07-14T16:22:30.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.610+0000 7fb36eff5700 10 client.26953 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:30.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.610+0000 7fb36eff5700 1 -- 172.21.15.62:0/782562173 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559575b27050 con 0x559575c8d030 2023-07-14T16:22:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.610+0000 7fb36eff5700 10 client.26953 handle_command_reply: tid=0 2023-07-14T16:22:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 client.26953 shutdown 2023-07-14T16:22:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/782562173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 msgr2=0x559575c27580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c27580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x559575c25510 tx=0x7fb37c005fd0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/782562173 shutdown_connections 2023-07-14T16:22:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559575c8d030 0x559575c8f440 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559575c1be30 0x559575c2aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559575c1a720 0x559575c27580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb36eff5700 0 client.26953 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 --2- 172.21.15.62:0/782562173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559575c1b070 0x559575c27ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/782562173 >> 172.21.15.62:0/782562173 conn(0x559575c0fff0 msgr2=0x559575c221c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/782562173 shutdown_connections 2023-07-14T16:22:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 1 -- 172.21.15.62:0/782562173 wait complete. 2023-07-14T16:22:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.611+0000 7fb3a4f03b80 20 client.26953 trim_cache size 0 max 16384 2023-07-14T16:22:30.595 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:22:30.595 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:22:30.595 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:22:30.595 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:22:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 -- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3740a1270 msgr2=0x7fb3740a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3740a1270 0x7fb3740a3720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb38400fb80 tx=0x7fb384004b40 comp rx=0 tx=0).stop 2023-07-14T16:22:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 -- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 msgr2=0x7fb38c197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 0x7fb38c197710 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0098e0 tx=0x7fb37c004730 comp rx=0 tx=0).stop 2023-07-14T16:22:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 -- 172.21.15.62:0/3747354314 shutdown_connections 2023-07-14T16:22:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3740a1270 0x7fb3740a3720 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb38c101630 0x7fb38c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb38c0ff9f0 0x7fb38c197710 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 --2- 172.21.15.62:0/3747354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb38c100c80 0x7fb38c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.612+0000 7fb38bfff700 1 -- 172.21.15.62:0/3747354314 >> 172.21.15.62:0/3747354314 conn(0x7fb38c0fb010 msgr2=0x7fb38c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.613+0000 7fb38bfff700 1 -- 172.21.15.62:0/3747354314 shutdown_connections 2023-07-14T16:22:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.613+0000 7fb38bfff700 1 -- 172.21.15.62:0/3747354314 wait complete. 2023-07-14T16:22:30.610 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:22:30.610 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:22:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 -- 172.21.15.62:0/869891085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c100cf0 msgr2=0x7f7a3c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 --2- 172.21.15.62:0/869891085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c100cf0 0x7f7a3c101140 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009990 tx=0x7f7a2c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 -- 172.21.15.62:0/869891085 shutdown_connections 2023-07-14T16:22:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 --2- 172.21.15.62:0/869891085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3c101680 0x7f7a3c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 --2- 172.21.15.62:0/869891085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a3c0ffb00 0x7f7a3c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 --2- 172.21.15.62:0/869891085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c100cf0 0x7f7a3c101140 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 -- 172.21.15.62:0/869891085 >> 172.21.15.62:0/869891085 conn(0x7f7a3c0fb0e0 msgr2=0x7f7a3c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 -- 172.21.15.62:0/869891085 shutdown_connections 2023-07-14T16:22:30.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.864+0000 7f7a435ff700 1 -- 172.21.15.62:0/869891085 wait complete. 2023-07-14T16:22:30.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.865+0000 7f7a435ff700 1 Processor -- start 2023-07-14T16:22:30.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.865+0000 7f7a435ff700 1 -- start start 2023-07-14T16:22:30.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a435ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c0ffb00 0x7f7a3c195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a435ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3c100cf0 0x7f7a3c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a435ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a3c101680 0x7f7a3c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a435ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c19d300 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a435ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c19d440 con 0x7f7a3c101680 2023-07-14T16:22:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a435ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c19d5d0 con 0x7f7a3c100cf0 2023-07-14T16:22:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a4139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c0ffb00 0x7f7a3c195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a4139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c0ffb00 0x7f7a3c195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38946/0 (socket says 172.21.15.62:38946) 2023-07-14T16:22:30.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a4139b700 1 -- 172.21.15.62:0/96210268 learned_addr learned my addr 172.21.15.62:0/96210268 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a40b9a700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3c100cf0 0x7f7a3c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a41b9c700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a3c101680 0x7f7a3c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a4139b700 1 -- 172.21.15.62:0/96210268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3c100cf0 msgr2=0x7f7a3c195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a4139b700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3c100cf0 0x7f7a3c195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a4139b700 1 -- 172.21.15.62:0/96210268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a3c101680 msgr2=0x7f7a3c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a4139b700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a3c101680 0x7f7a3c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a4139b700 1 -- 172.21.15.62:0/96210268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c0095f0 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a41b9c700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a3c101680 0x7f7a3c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:22:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.866+0000 7f7a40b9a700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3c100cf0 0x7f7a3c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.867+0000 7f7a4139b700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c0ffb00 0x7f7a3c195780 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e970 tx=0x7f7a3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.867+0000 7f7a327fc700 1 -- 172.21.15.62:0/96210268 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38009790 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.867+0000 7f7a327fc700 1 -- 172.21.15.62:0/96210268 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a380044b0 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.867+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c19d8b0 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.867+0000 7f7a327fc700 1 -- 172.21.15.62:0/96210268 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38003dd0 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.867+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c19de10 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.868+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c10b1d0 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.871+0000 7f7a327fc700 1 -- 172.21.15.62:0/96210268 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7a38004000 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.872+0000 7f7a327fc700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a280a1220 0x7f7a280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:30.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.873+0000 7f7a327fc700 1 -- 172.21.15.62:0/96210268 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a38014070 con 0x7f7a3c0ffb00 2023-07-14T16:22:30.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.873+0000 7f7a40b9a700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a280a1220 0x7f7a280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:30.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.874+0000 7f7a40b9a700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a280a1220 0x7f7a280a36d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006010 tx=0x7f7a2c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:30.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:30.878+0000 7f7a327fc700 1 -- 172.21.15.62:0/96210268 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7a38096340 con 0x7f7a3c0ffb00 2023-07-14T16:22:31.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:30 smithi062 ceph-mon[95543]: pgmap v563: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:22:31.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:30 smithi081 ceph-mon[127795]: pgmap v563: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:22:31.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.187+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a3c106a30 con 0x7f7a3c0ffb00 2023-07-14T16:22:31.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.188+0000 7f7a327fc700 1 -- 172.21.15.62:0/96210268 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f7a38099960 con 0x7f7a3c0ffb00 2023-07-14T16:22:31.165 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:22:31.165 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:22:31.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a280a1220 msgr2=0x7f7a280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a280a1220 0x7f7a280a36d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006010 tx=0x7f7a2c00b260 comp rx=0 tx=0).stop 2023-07-14T16:22:31.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c0ffb00 msgr2=0x7f7a3c195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c0ffb00 0x7f7a3c195780 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e970 tx=0x7f7a3800ed30 comp rx=0 tx=0).stop 2023-07-14T16:22:31.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 shutdown_connections 2023-07-14T16:22:31.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a280a1220 0x7f7a280a36d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a3c101680 0x7f7a3c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a3c0ffb00 0x7f7a3c195780 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 --2- 172.21.15.62:0/96210268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a3c100cf0 0x7f7a3c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 >> 172.21.15.62:0/96210268 conn(0x7f7a3c0fb0e0 msgr2=0x7f7a3c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 shutdown_connections 2023-07-14T16:22:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.190+0000 7f7a435ff700 1 -- 172.21.15.62:0/96210268 wait complete. 2023-07-14T16:22:31.169 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:22:31.187 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:22:31.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:30 smithi164 ceph-mon[97427]: pgmap v563: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:22:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 -- 172.21.15.62:0/119107961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc0fe9e0 msgr2=0x7fecdc0fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 --2- 172.21.15.62:0/119107961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc0fe9e0 0x7fecdc0fedf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009990 tx=0x7fecd8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 -- 172.21.15.62:0/119107961 shutdown_connections 2023-07-14T16:22:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 --2- 172.21.15.62:0/119107961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc100560 0x7fecdc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 --2- 172.21.15.62:0/119107961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc0fe9e0 0x7fecdc0fedf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 --2- 172.21.15.62:0/119107961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecdc0ffbd0 0x7fecdc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 -- 172.21.15.62:0/119107961 >> 172.21.15.62:0/119107961 conn(0x7fecdc0fa000 msgr2=0x7fecdc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 -- 172.21.15.62:0/119107961 shutdown_connections 2023-07-14T16:22:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.447+0000 7fece47ff700 1 -- 172.21.15.62:0/119107961 wait complete. 2023-07-14T16:22:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.448+0000 7fece47ff700 1 Processor -- start 2023-07-14T16:22:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece47ff700 1 -- start start 2023-07-14T16:22:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece47ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc0fe9e0 0x7fecdc102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece47ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0ffbd0 0x7fecdc100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece47ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecdc100560 0x7fecdc1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece47ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc101c90 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece47ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc101e00 con 0x7fecdc100560 2023-07-14T16:22:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece47ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc19f810 con 0x7fecdc0fe9e0 2023-07-14T16:22:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece1d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0ffbd0 0x7fecdc100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece1d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0ffbd0 0x7fecdc100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38958/0 (socket says 172.21.15.62:38958) 2023-07-14T16:22:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece1d9a700 1 -- 172.21.15.62:0/1484652583 learned_addr learned my addr 172.21.15.62:0/1484652583 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece2d9c700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecdc100560 0x7fecdc1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.449+0000 7fece259b700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc0fe9e0 0x7fecdc102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.450+0000 7fece1d9a700 1 -- 172.21.15.62:0/1484652583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc0fe9e0 msgr2=0x7fecdc102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.450+0000 7fece1d9a700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc0fe9e0 0x7fecdc102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.450+0000 7fece1d9a700 1 -- 172.21.15.62:0/1484652583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecdc100560 msgr2=0x7fecdc1014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.450+0000 7fece1d9a700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecdc100560 0x7fecdc1014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.450+0000 7fece1d9a700 1 -- 172.21.15.62:0/1484652583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd80095f0 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.450+0000 7fece2d9c700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecdc100560 0x7fecdc1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.450+0000 7fece1d9a700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0ffbd0 0x7fecdc100fb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7feccc00e9a0 tx=0x7feccc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.450+0000 7fecd37fe700 1 -- 172.21.15.62:0/1484652583 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc009790 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.451+0000 7fece47ff700 1 -- 172.21.15.62:0/1484652583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc19faf0 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.451+0000 7fecd37fe700 1 -- 172.21.15.62:0/1484652583 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feccc010430 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.451+0000 7fecd37fe700 1 -- 172.21.15.62:0/1484652583 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc00cea0 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.451+0000 7fece47ff700 1 -- 172.21.15.62:0/1484652583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc1a0000 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.456+0000 7fecd37fe700 1 -- 172.21.15.62:0/1484652583 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feccc0105a0 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.456+0000 7fecf8110b80 1 Processor -- start 2023-07-14T16:22:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fecf8110b80 1 -- start start 2023-07-14T16:22:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb12e20 0x5588adb136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb13c70 0x5588adb140e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fecf8110b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ada17cb0 con 0x5588adb12e20 2023-07-14T16:22:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fecf8110b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ada6aa40 con 0x5588adb124d0 2023-07-14T16:22:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fecf8110b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ada63ed0 con 0x5588adb13c70 2023-07-14T16:22:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fecd37fe700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fecc80a1150 0x7fecc80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.457+0000 7fece259b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb12e20 0x5588adb136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece259b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb12e20 0x5588adb136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38966/0 (socket says 172.21.15.62:38966) 2023-07-14T16:22:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece259b700 1 -- 172.21.15.62:0/3553161215 learned_addr learned my addr 172.21.15.62:0/3553161215 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece1d9a700 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb13c70 0x5588adb140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece2d9c700 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fecd37fe700 1 -- 172.21.15.62:0/1484652583 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feccc014070 con 0x7fecdc0ffbd0 2023-07-14T16:22:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece2d9c700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fecc80a1150 0x7fecc80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece259b700 1 -- 172.21.15.62:0/3553161215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb13c70 msgr2=0x5588adb140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece259b700 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb13c70 0x5588adb140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece259b700 1 -- 172.21.15.62:0/3553161215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 msgr2=0x5588adb128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece259b700 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece259b700 1 -- 172.21.15.62:0/3553161215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588ad891de0 con 0x5588adb12e20 2023-07-14T16:22:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.458+0000 7fece259b700 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb12e20 0x5588adb136e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009990 tx=0x7fecd8005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ff9b3b4656233f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.459+0000 7fece2d9c700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fecc80a1150 0x7fecc80a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009ec0 tx=0x7fecd400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.459+0000 7fecd0ff9700 1 -- 172.21.15.62:0/3553161215 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd801d070 con 0x5588adb12e20 2023-07-14T16:22:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.459+0000 7fecd0ff9700 1 -- 172.21.15.62:0/3553161215 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecd801e040 con 0x5588adb12e20 2023-07-14T16:22:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.459+0000 7fecd0ff9700 1 -- 172.21.15.62:0/3553161215 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd800b710 con 0x5588adb12e20 2023-07-14T16:22:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 -- 172.21.15.62:0/3553161215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb12e20 msgr2=0x5588adb136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb12e20 0x5588adb136e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009990 tx=0x7fecd8005f00 comp rx=0 tx=0).stop 2023-07-14T16:22:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 -- 172.21.15.62:0/3553161215 shutdown_connections 2023-07-14T16:22:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb13c70 0x5588adb140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 --2- 172.21.15.62:0/3553161215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb12e20 0x5588adb136e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 -- 172.21.15.62:0/3553161215 >> 172.21.15.62:0/3553161215 conn(0x5588adb07da0 msgr2=0x5588adb0d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 -- 172.21.15.62:0/3553161215 shutdown_connections 2023-07-14T16:22:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 -- 172.21.15.62:0/3553161215 wait complete. 2023-07-14T16:22:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.460+0000 7fecf8110b80 1 -- start start 2023-07-14T16:22:31.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.461+0000 7fecf8110b80 1 Processor -- start 2023-07-14T16:22:31.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.461+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb1c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.461+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb1c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.461+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.461+0000 7fecf8110b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ada03ac0 con 0x5588adb13c70 2023-07-14T16:22:31.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.461+0000 7fecf8110b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ada037c0 con 0x5588adb124d0 2023-07-14T16:22:31.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.461+0000 7fecf8110b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ad8e7920 con 0x5588adb12e20 2023-07-14T16:22:31.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.461+0000 7fecf8110b80 10 client.0 _ll_register_callbacks cb 0x5588ada01760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:31.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece2d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece2d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb23600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38982/0 (socket says 172.21.15.62:38982) 2023-07-14T16:22:31.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece2d9c700 1 -- 172.21.15.62:0/819766397 learned_addr learned my addr 172.21.15.62:0/819766397 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece259b700 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb1c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece1d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb1c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece1d9a700 1 -- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 msgr2=0x5588adb1c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece1d9a700 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb1c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece1d9a700 1 -- 172.21.15.62:0/819766397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 msgr2=0x5588adb23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece1d9a700 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb23600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece1d9a700 1 -- 172.21.15.62:0/819766397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc107750 con 0x5588adb12e20 2023-07-14T16:22:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece259b700 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb1c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.462+0000 7fece1d9a700 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb1c5c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feccc0ca650 tx=0x7feccc00ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=838361adbef85718 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:22:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecc5ffb700 1 -- 172.21.15.62:0/819766397 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc09a370 con 0x5588adb12e20 2023-07-14T16:22:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecf8110b80 1 -- 172.21.15.62:0/819766397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588ad9bd4f0 con 0x5588adb12e20 2023-07-14T16:22:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecc5ffb700 1 -- 172.21.15.62:0/819766397 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feccc010880 con 0x5588adb12e20 2023-07-14T16:22:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecc5ffb700 1 -- 172.21.15.62:0/819766397 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feccc099f10 con 0x5588adb12e20 2023-07-14T16:22:31.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecf8110b80 10 client.26971 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:31.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecf8110b80 1 -- 172.21.15.62:0/819766397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5588ad87ac60 con 0x5588adb12e20 2023-07-14T16:22:31.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecc5ffb700 1 -- 172.21.15.62:0/819766397 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5588ad9bd4f0 con 0x5588adb12e20 2023-07-14T16:22:31.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.463+0000 7fecc5ffb700 1 -- 172.21.15.62:0/819766397 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feccc0a1ed0 con 0x5588adb12e20 2023-07-14T16:22:31.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.464+0000 7fecf8110b80 10 client.26971 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:31.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.464+0000 7fecf8110b80 10 client.26971 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:22:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.464+0000 7fecf8110b80 20 client.26971 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.464+0000 7fecf8110b80 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5588adb27ef0 0x5588adb2a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.464+0000 7fecf8110b80 4 client.26971 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.464+0000 7fecf8110b80 1 -- 172.21.15.62:0/819766397 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588ada5ceb0 con 0x5588adb27ef0 2023-07-14T16:22:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.464+0000 7fece259b700 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5588adb27ef0 0x5588adb2a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.465+0000 7fece259b700 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5588adb27ef0 0x5588adb2a300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7f1e44b53c21ebb server_cookie=7d3691209e0b9f39 in_seq=0 out_seq=0 2023-07-14T16:22:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.465+0000 7fecc5ffb700 10 client.26971 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:22:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecc5ffb700 1 -- 172.21.15.62:0/819766397 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5588ada5ceb0 con 0x5588adb27ef0 2023-07-14T16:22:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecc5ffb700 10 client.26971 handle_command_reply: tid=0 2023-07-14T16:22:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 client.26971 shutdown 2023-07-14T16:22:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 -- 172.21.15.62:0/819766397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 msgr2=0x5588adb1c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb1c5c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feccc0ca650 tx=0x7feccc00ac80 comp rx=0 tx=0).stop 2023-07-14T16:22:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 -- 172.21.15.62:0/819766397 shutdown_connections 2023-07-14T16:22:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5588adb27ef0 0x5588adb2a300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecc5ffb700 0 client.26971 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:22:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb1c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 --2- 172.21.15.62:0/819766397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb1c5c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.466+0000 7fecf8110b80 1 -- 172.21.15.62:0/819766397 >> 172.21.15.62:0/819766397 conn(0x5588adb07da0 msgr2=0x5588adb0c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.467+0000 7fecf8110b80 1 -- 172.21.15.62:0/819766397 shutdown_connections 2023-07-14T16:22:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.467+0000 7fecf8110b80 1 -- 172.21.15.62:0/819766397 wait complete. 2023-07-14T16:22:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.467+0000 7fecf8110b80 20 client.26971 trim_cache size 0 max 16384 2023-07-14T16:22:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.486+0000 7fecf8110b80 1 Processor -- start 2023-07-14T16:22:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.486+0000 7fecf8110b80 1 -- start start 2023-07-14T16:22:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.486+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.486+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.486+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.486+0000 7fecf8110b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588adb1b360 con 0x5588adb13c70 2023-07-14T16:22:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.486+0000 7fecf8110b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588adb27cc0 con 0x5588adb124d0 2023-07-14T16:22:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.486+0000 7fecf8110b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588adb19d10 con 0x5588adb12e20 2023-07-14T16:22:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece2d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece2d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38996/0 (socket says 172.21.15.62:38996) 2023-07-14T16:22:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece2d9c700 1 -- 172.21.15.62:0/2139354548 learned_addr learned my addr 172.21.15.62:0/2139354548 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece259b700 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece1d9a700 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece2d9c700 1 -- 172.21.15.62:0/2139354548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 msgr2=0x5588adb235c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece2d9c700 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb235c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece2d9c700 1 -- 172.21.15.62:0/2139354548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 msgr2=0x5588adb128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece2d9c700 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece2d9c700 1 -- 172.21.15.62:0/2139354548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588ad8b8250 con 0x5588adb13c70 2023-07-14T16:22:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.487+0000 7fece259b700 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.488+0000 7fece2d9c700 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1ae20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fecd400e540 tx=0x7fecd4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4b3a4a8093bc3b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.488+0000 7fecd17fa700 1 -- 172.21.15.62:0/2139354548 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd402a070 con 0x5588adb13c70 2023-07-14T16:22:31.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.488+0000 7fecd17fa700 1 -- 172.21.15.62:0/2139354548 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecd400f040 con 0x5588adb13c70 2023-07-14T16:22:31.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.488+0000 7fecd17fa700 1 -- 172.21.15.62:0/2139354548 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd401ac60 con 0x5588adb13c70 2023-07-14T16:22:31.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 -- 172.21.15.62:0/2139354548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 msgr2=0x5588adb1ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1ae20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fecd400e540 tx=0x7fecd4009920 comp rx=0 tx=0).stop 2023-07-14T16:22:31.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 -- 172.21.15.62:0/2139354548 shutdown_connections 2023-07-14T16:22:31.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1ae20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2139354548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb235c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 -- 172.21.15.62:0/2139354548 >> 172.21.15.62:0/2139354548 conn(0x5588adb07da0 msgr2=0x5588adb0d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 -- 172.21.15.62:0/2139354548 shutdown_connections 2023-07-14T16:22:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 -- 172.21.15.62:0/2139354548 wait complete. 2023-07-14T16:22:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 -- start start 2023-07-14T16:22:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.489+0000 7fecf8110b80 1 Processor -- start 2023-07-14T16:22:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb22df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb23330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fecf8110b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fecf8110b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588adb1fdd0 con 0x5588adb13c70 2023-07-14T16:22:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fecf8110b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588adb1ff10 con 0x5588adb124d0 2023-07-14T16:22:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fecf8110b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588adb200d0 con 0x5588adb12e20 2023-07-14T16:22:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fecf8110b80 10 client.0 _ll_register_callbacks cb 0x5588ada01760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fece2d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fece2d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39000/0 (socket says 172.21.15.62:39000) 2023-07-14T16:22:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fece2d9c700 1 -- 172.21.15.62:0/2045424855 learned_addr learned my addr 172.21.15.62:0/2045424855 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fece259b700 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb22df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.490+0000 7fece1d9a700 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb23330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.491+0000 7fece2d9c700 1 -- 172.21.15.62:0/2045424855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 msgr2=0x5588adb23330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.491+0000 7fece2d9c700 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb23330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.491+0000 7fece2d9c700 1 -- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 msgr2=0x5588adb22df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.491+0000 7fece2d9c700 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb22df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.491+0000 7fece2d9c700 1 -- 172.21.15.62:0/2045424855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588ad891de0 con 0x5588adb13c70 2023-07-14T16:22:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.491+0000 7fece259b700 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb22df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.491+0000 7fece2d9c700 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1f890 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fecd401ae30 tx=0x7fecd401ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d2a0bbcb2314d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.491+0000 7fecc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.492+0000 7fecc5ffb700 1 -- 172.21.15.62:0/2045424855 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd402a040 con 0x5588adb13c70 2023-07-14T16:22:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.492+0000 7fecf8110b80 1 -- 172.21.15.62:0/2045424855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecd801e040 con 0x5588adb13c70 2023-07-14T16:22:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.492+0000 7fecc5ffb700 1 -- 172.21.15.62:0/2045424855 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fecd401a680 con 0x5588adb13c70 2023-07-14T16:22:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.492+0000 7fecc5ffb700 1 -- 172.21.15.62:0/2045424855 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd401a840 con 0x5588adb13c70 2023-07-14T16:22:31.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.492+0000 7fecf8110b80 10 client.17406 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:31.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.492+0000 7fecf8110b80 1 -- 172.21.15.62:0/2045424855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fecd40083e0 con 0x5588adb13c70 2023-07-14T16:22:31.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.492+0000 7fecc5ffb700 1 -- 172.21.15.62:0/2045424855 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecd403e020 con 0x5588adb13c70 2023-07-14T16:22:31.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.493+0000 7fecc5ffb700 1 -- 172.21.15.62:0/2045424855 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fecd4025070 con 0x5588adb13c70 2023-07-14T16:22:31.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.493+0000 7fecf8110b80 10 client.17406 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:31.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.493+0000 7fecf8110b80 10 client.17406 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:22:31.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.493+0000 7fecf8110b80 20 client.17406 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:31.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.493+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5588adb7d220 0x5588adb7f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.493+0000 7fecf8110b80 4 client.17406 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:22:31.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.493+0000 7fecf8110b80 1 -- 172.21.15.62:0/2045424855 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588ada1f050 con 0x5588adb7d220 2023-07-14T16:22:31.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.493+0000 7fece259b700 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5588adb7d220 0x5588adb7f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fece259b700 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5588adb7d220 0x5588adb7f630 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e3238d2ad992917 server_cookie=7bc1fb088989b125 in_seq=0 out_seq=0 2023-07-14T16:22:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecc5ffb700 10 client.17406 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:22:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecc5ffb700 1 -- 172.21.15.62:0/2045424855 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588ada1f050 con 0x5588adb7d220 2023-07-14T16:22:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecc5ffb700 10 client.17406 handle_command_reply: tid=0 2023-07-14T16:22:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 client.17406 shutdown 2023-07-14T16:22:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 -- 172.21.15.62:0/2045424855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 msgr2=0x5588adb1f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1f890 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fecd401ae30 tx=0x7fecd401ae60 comp rx=0 tx=0).stop 2023-07-14T16:22:31.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 -- 172.21.15.62:0/2045424855 shutdown_connections 2023-07-14T16:22:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5588adb7d220 0x5588adb7f630 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588adb13c70 0x5588adb1f890 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588adb124d0 0x5588adb22df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 --2- 172.21.15.62:0/2045424855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5588adb12e20 0x5588adb23330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecf8110b80 1 -- 172.21.15.62:0/2045424855 >> 172.21.15.62:0/2045424855 conn(0x5588adb07da0 msgr2=0x5588adb0d160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.494+0000 7fecc5ffb700 0 client.17406 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:22:31.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fecf8110b80 1 -- 172.21.15.62:0/2045424855 shutdown_connections 2023-07-14T16:22:31.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fecf8110b80 1 -- 172.21.15.62:0/2045424855 wait complete. 2023-07-14T16:22:31.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fecf8110b80 20 client.17406 trim_cache size 0 max 16384 2023-07-14T16:22:31.480 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:22:31.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 -- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fecc80a1150 msgr2=0x7fecc80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fecc80a1150 0x7fecc80a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009ec0 tx=0x7fecd400b040 comp rx=0 tx=0).stop 2023-07-14T16:22:31.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 -- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0ffbd0 msgr2=0x7fecdc100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0ffbd0 0x7fecdc100fb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7feccc00e9a0 tx=0x7feccc00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 -- 172.21.15.62:0/1484652583 shutdown_connections 2023-07-14T16:22:31.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fecc80a1150 0x7fecc80a3600 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecdc100560 0x7fecdc1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc0fe9e0 0x7fecdc102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 --2- 172.21.15.62:0/1484652583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecdc0ffbd0 0x7fecdc100fb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.495+0000 7fece47ff700 1 -- 172.21.15.62:0/1484652583 >> 172.21.15.62:0/1484652583 conn(0x7fecdc0fa000 msgr2=0x7fecdc0fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.496+0000 7fece47ff700 1 -- 172.21.15.62:0/1484652583 shutdown_connections 2023-07-14T16:22:31.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.496+0000 7fece47ff700 1 -- 172.21.15.62:0/1484652583 wait complete. 2023-07-14T16:22:31.494 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:22:31.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 -- 172.21.15.62:0/3696637388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 msgr2=0x7fc858102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/3696637388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 0x7fc858102940 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc848009990 tx=0x7fc848009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 -- 172.21.15.62:0/3696637388 shutdown_connections 2023-07-14T16:22:31.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/3696637388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8581057a0 0x7fc858107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/3696637388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 0x7fc858102940 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/3696637388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc858102e80 0x7fc858105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 -- 172.21.15.62:0/3696637388 >> 172.21.15.62:0/3696637388 conn(0x7fc8580fa000 msgr2=0x7fc8580fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 -- 172.21.15.62:0/3696637388 shutdown_connections 2023-07-14T16:22:31.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.762+0000 7fc85e3ff700 1 -- 172.21.15.62:0/3696637388 wait complete. 2023-07-14T16:22:31.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.763+0000 7fc85e3ff700 1 Processor -- start 2023-07-14T16:22:31.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.763+0000 7fc85e3ff700 1 -- start start 2023-07-14T16:22:31.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc85e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 0x7fc8581979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc85e3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc858102e80 0x7fc858197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc85e3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8581057a0 0x7fc85819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc85e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85819f550 con 0x7fc858100560 2023-07-14T16:22:31.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc85e3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85819f690 con 0x7fc858102e80 2023-07-14T16:22:31.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc85e3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85819f820 con 0x7fc8581057a0 2023-07-14T16:22:31.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc857fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 0x7fc8581979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc857fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 0x7fc8581979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39010/0 (socket says 172.21.15.62:39010) 2023-07-14T16:22:31.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc857fff700 1 -- 172.21.15.62:0/2810430753 learned_addr learned my addr 172.21.15.62:0/2810430753 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc8577fe700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc858102e80 0x7fc858197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.764+0000 7fc85c99c700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8581057a0 0x7fc85819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc857fff700 1 -- 172.21.15.62:0/2810430753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8581057a0 msgr2=0x7fc85819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc857fff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8581057a0 0x7fc85819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc857fff700 1 -- 172.21.15.62:0/2810430753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc858102e80 msgr2=0x7fc858197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc857fff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc858102e80 0x7fc858197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc857fff700 1 -- 172.21.15.62:0/2810430753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8480095f0 con 0x7fc858100560 2023-07-14T16:22:31.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc857fff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 0x7fc8581979d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc848005910 tx=0x7fc848005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc8557fa700 1 -- 172.21.15.62:0/2810430753 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84801d070 con 0x7fc858100560 2023-07-14T16:22:31.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc8557fa700 1 -- 172.21.15.62:0/2810430753 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc848022470 con 0x7fc858100560 2023-07-14T16:22:31.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc85e3ff700 1 -- 172.21.15.62:0/2810430753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85819faa0 con 0x7fc858100560 2023-07-14T16:22:31.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.765+0000 7fc8557fa700 1 -- 172.21.15.62:0/2810430753 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848022ea0 con 0x7fc858100560 2023-07-14T16:22:31.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.766+0000 7fc85e3ff700 1 -- 172.21.15.62:0/2810430753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85819fec0 con 0x7fc858100560 2023-07-14T16:22:31.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.770+0000 7fc8557fa700 1 -- 172.21.15.62:0/2810430753 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc84800ba60 con 0x7fc858100560 2023-07-14T16:22:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.772+0000 7fc871c40b80 1 Processor -- start 2023-07-14T16:22:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.772+0000 7fc871c40b80 1 -- start start 2023-07-14T16:22:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.772+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc6d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.772+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6de20 0x56402fc6e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.772+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6ec70 0x56402fc6f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.772+0000 7fc871c40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fb72cb0 con 0x56402fc6ec70 2023-07-14T16:22:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.772+0000 7fc871c40b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fbc5a40 con 0x56402fc6de20 2023-07-14T16:22:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.772+0000 7fc871c40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fbbeed0 con 0x56402fc6d4d0 2023-07-14T16:22:31.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6ec70 0x56402fc6f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc857fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6de20 0x56402fc6e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8557fa700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc8400a1150 0x7fc8400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6ec70 0x56402fc6f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39014/0 (socket says 172.21.15.62:39014) 2023-07-14T16:22:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8577fe700 1 -- 172.21.15.62:0/2403560193 learned_addr learned my addr 172.21.15.62:0/2403560193 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc85c99c700 1 --2- 172.21.15.62:0/2403560193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc6d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8557fa700 1 -- 172.21.15.62:0/2810430753 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc8480cdce0 con 0x7fc858100560 2023-07-14T16:22:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8577fe700 1 -- 172.21.15.62:0/2403560193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 msgr2=0x56402fc6d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8577fe700 1 --2- 172.21.15.62:0/2403560193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc6d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8577fe700 1 -- 172.21.15.62:0/2403560193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6de20 msgr2=0x56402fc6e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8577fe700 1 --2- 172.21.15.62:0/2403560193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6de20 0x56402fc6e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc8577fe700 1 -- 172.21.15.62:0/2403560193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56402f9ecde0 con 0x56402fc6ec70 2023-07-14T16:22:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.773+0000 7fc85c99c700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc8400a1150 0x7fc8400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.774+0000 7fc8577fe700 1 --2- 172.21.15.62:0/2403560193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6ec70 0x56402fc6f0e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007d70 tx=0x7fc84c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d283e9505b0a8c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.774+0000 7fc83e7fc700 1 -- 172.21.15.62:0/2403560193 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c010e10 con 0x56402fc6ec70 2023-07-14T16:22:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.774+0000 7fc83e7fc700 1 -- 172.21.15.62:0/2403560193 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc84c015070 con 0x56402fc6ec70 2023-07-14T16:22:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.774+0000 7fc83e7fc700 1 -- 172.21.15.62:0/2403560193 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c003dc0 con 0x56402fc6ec70 2023-07-14T16:22:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc85c99c700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc8400a1150 0x7fc8400a3600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc850009ec0 tx=0x7fc850004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 -- 172.21.15.62:0/2403560193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6ec70 msgr2=0x56402fc6f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 --2- 172.21.15.62:0/2403560193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6ec70 0x56402fc6f0e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007d70 tx=0x7fc84c0106b0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 -- 172.21.15.62:0/2403560193 shutdown_connections 2023-07-14T16:22:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 --2- 172.21.15.62:0/2403560193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6ec70 0x56402fc6f0e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 --2- 172.21.15.62:0/2403560193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc6d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 --2- 172.21.15.62:0/2403560193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6de20 0x56402fc6e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 -- 172.21.15.62:0/2403560193 >> 172.21.15.62:0/2403560193 conn(0x56402fc62da0 msgr2=0x56402fc681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 -- 172.21.15.62:0/2403560193 shutdown_connections 2023-07-14T16:22:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.775+0000 7fc871c40b80 1 -- 172.21.15.62:0/2403560193 wait complete. 2023-07-14T16:22:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 1 -- start start 2023-07-14T16:22:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 1 Processor -- start 2023-07-14T16:22:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc77130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc77670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6ec70 0x56402fc7e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fb5eac0 con 0x56402fc6de20 2023-07-14T16:22:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fb5e7c0 con 0x56402fc6ec70 2023-07-14T16:22:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fa42920 con 0x56402fc6d4d0 2023-07-14T16:22:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.776+0000 7fc871c40b80 10 client.0 _ll_register_callbacks cb 0x56402fb5c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc857fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc77130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc77670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc857fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc77130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34102/0 (socket says 172.21.15.62:34102) 2023-07-14T16:22:31.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc857fff700 1 -- 172.21.15.62:0/1960514707 learned_addr learned my addr 172.21.15.62:0/1960514707 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc857fff700 1 -- 172.21.15.62:0/1960514707 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6ec70 msgr2=0x56402fc7e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc857fff700 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6ec70 0x56402fc7e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc857fff700 1 -- 172.21.15.62:0/1960514707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 msgr2=0x56402fc77670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc857fff700 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc77670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc857fff700 1 -- 172.21.15.62:0/1960514707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c0079a0 con 0x56402fc6d4d0 2023-07-14T16:22:31.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.777+0000 7fc8577fe700 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc77670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.778+0000 7fc857fff700 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc77130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc84809b120 tx=0x7fc84809b070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a4dc9f548d612a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.778+0000 7fc8377fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:22:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.778+0000 7fc8377fe700 1 -- 172.21.15.62:0/1960514707 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84809aaf0 con 0x56402fc6d4d0 2023-07-14T16:22:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.778+0000 7fc8377fe700 1 -- 172.21.15.62:0/1960514707 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc848022800 con 0x56402fc6d4d0 2023-07-14T16:22:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.778+0000 7fc871c40b80 1 -- 172.21.15.62:0/1960514707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56402fb184f0 con 0x56402fc6d4d0 2023-07-14T16:22:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.778+0000 7fc8377fe700 1 -- 172.21.15.62:0/1960514707 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc84809b190 con 0x56402fc6d4d0 2023-07-14T16:22:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.778+0000 7fc871c40b80 10 client.26989 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.778+0000 7fc871c40b80 1 -- 172.21.15.62:0/1960514707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56402f9d5c60 con 0x56402fc6d4d0 2023-07-14T16:22:31.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc8377fe700 1 -- 172.21.15.62:0/1960514707 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56402fb184f0 con 0x56402fc6d4d0 2023-07-14T16:22:31.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc8377fe700 1 -- 172.21.15.62:0/1960514707 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc84809a680 con 0x56402fc6d4d0 2023-07-14T16:22:31.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc871c40b80 10 client.26989 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:31.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc871c40b80 10 client.26989 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:22:31.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc871c40b80 20 client.26989 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:31.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc871c40b80 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56402fc82b40 0x56402fc84f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc871c40b80 4 client.26989 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:31.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc871c40b80 1 -- 172.21.15.62:0/1960514707 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56402fbb7eb0 con 0x56402fc82b40 2023-07-14T16:22:31.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.779+0000 7fc8577fe700 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56402fc82b40 0x56402fc84f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.780+0000 7fc8577fe700 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56402fc82b40 0x56402fc84f50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e3862ac16cd3934 server_cookie=25e41aa32d91c648 in_seq=0 out_seq=0 2023-07-14T16:22:31.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.780+0000 7fc8377fe700 10 client.26989 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:22:31.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc8377fe700 1 -- 172.21.15.62:0/1960514707 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56402fbb7eb0 con 0x56402fc82b40 2023-07-14T16:22:31.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc8377fe700 10 client.26989 handle_command_reply: tid=0 2023-07-14T16:22:31.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 client.26989 shutdown 2023-07-14T16:22:31.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 -- 172.21.15.62:0/1960514707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 msgr2=0x56402fc77130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc77130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc84809b120 tx=0x7fc84809b070 comp rx=0 tx=0).stop 2023-07-14T16:22:31.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 -- 172.21.15.62:0/1960514707 shutdown_connections 2023-07-14T16:22:31.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56402fc82b40 0x56402fc84f50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6ec70 0x56402fc7e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc77130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc8377fe700 0 client.26989 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:22:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 --2- 172.21.15.62:0/1960514707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc77670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 -- 172.21.15.62:0/1960514707 >> 172.21.15.62:0/1960514707 conn(0x56402fc62da0 msgr2=0x56402fc678b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 -- 172.21.15.62:0/1960514707 shutdown_connections 2023-07-14T16:22:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 1 -- 172.21.15.62:0/1960514707 wait complete. 2023-07-14T16:22:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.781+0000 7fc871c40b80 20 client.26989 trim_cache size 0 max 16384 2023-07-14T16:22:31.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.799+0000 7fc871c40b80 1 Processor -- start 2023-07-14T16:22:31.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.799+0000 7fc871c40b80 1 -- start start 2023-07-14T16:22:31.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6d4d0 0x56402fc7efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6ec70 0x56402fc71c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc871c40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fc76d00 con 0x56402fc6de20 2023-07-14T16:22:31.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc871c40b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fb184f0 con 0x56402fc6d4d0 2023-07-14T16:22:31.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc871c40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fc78320 con 0x56402fc6ec70 2023-07-14T16:22:31.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39036/0 (socket says 172.21.15.62:39036) 2023-07-14T16:22:31.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.800+0000 7fc8577fe700 1 -- 172.21.15.62:0/3208080236 learned_addr learned my addr 172.21.15.62:0/3208080236 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc857fff700 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6d4d0 0x56402fc7efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc85c99c700 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6ec70 0x56402fc71c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc8577fe700 1 -- 172.21.15.62:0/3208080236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6ec70 msgr2=0x56402fc71c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc8577fe700 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6ec70 0x56402fc71c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc8577fe700 1 -- 172.21.15.62:0/3208080236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6d4d0 msgr2=0x56402fc7efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc8577fe700 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6d4d0 0x56402fc7efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc8577fe700 1 -- 172.21.15.62:0/3208080236 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56402fa13250 con 0x56402fc6de20 2023-07-14T16:22:31.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc85c99c700 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6ec70 0x56402fc71c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:22:31.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.801+0000 7fc8577fe700 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7e670 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc84c000c00 tx=0x7fc84c0073d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92dbc825eaf5af2b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.802+0000 7fc83effd700 1 -- 172.21.15.62:0/3208080236 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c027a50 con 0x56402fc6de20 2023-07-14T16:22:31.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.802+0000 7fc83effd700 1 -- 172.21.15.62:0/3208080236 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc84c01f040 con 0x56402fc6de20 2023-07-14T16:22:31.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.802+0000 7fc83effd700 1 -- 172.21.15.62:0/3208080236 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c005400 con 0x56402fc6de20 2023-07-14T16:22:31.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 -- 172.21.15.62:0/3208080236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 msgr2=0x56402fc7e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7e670 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc84c000c00 tx=0x7fc84c0073d0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 -- 172.21.15.62:0/3208080236 shutdown_connections 2023-07-14T16:22:31.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6ec70 0x56402fc71c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6d4d0 0x56402fc7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 --2- 172.21.15.62:0/3208080236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7e670 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 -- 172.21.15.62:0/3208080236 >> 172.21.15.62:0/3208080236 conn(0x56402fc62da0 msgr2=0x56402fc75e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 -- 172.21.15.62:0/3208080236 shutdown_connections 2023-07-14T16:22:31.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.803+0000 7fc871c40b80 1 -- 172.21.15.62:0/3208080236 wait complete. 2023-07-14T16:22:31.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.804+0000 7fc871c40b80 1 -- start start 2023-07-14T16:22:31.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.804+0000 7fc871c40b80 1 Processor -- start 2023-07-14T16:22:31.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.804+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc7a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc871c40b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6ec70 0x56402fc7d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc871c40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fc7df00 con 0x56402fc6de20 2023-07-14T16:22:31.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc871c40b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fc7e070 con 0x56402fc6ec70 2023-07-14T16:22:31.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc871c40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56402fc7e230 con 0x56402fc6d4d0 2023-07-14T16:22:31.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc871c40b80 10 client.0 _ll_register_callbacks cb 0x7fc848005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:31.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39042/0 (socket says 172.21.15.62:39042) 2023-07-14T16:22:31.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc8577fe700 1 -- 172.21.15.62:0/34828764 learned_addr learned my addr 172.21.15.62:0/34828764 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:31.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.805+0000 7fc857fff700 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc7a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.806+0000 7fc8577fe700 1 -- 172.21.15.62:0/34828764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 msgr2=0x56402fc7a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.806+0000 7fc8577fe700 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc7a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.806+0000 7fc8577fe700 1 -- 172.21.15.62:0/34828764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6ec70 msgr2=0x56402fc7d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:31.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.806+0000 7fc8577fe700 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6ec70 0x56402fc7d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.806+0000 7fc8577fe700 1 -- 172.21.15.62:0/34828764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56402f9ecde0 con 0x56402fc6de20 2023-07-14T16:22:31.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.806+0000 7fc8577fe700 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7a9a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007d40 tx=0x7fc84c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f48a882f93d89ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:31.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.806+0000 7fc8377fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:31.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.807+0000 7fc8377fe700 1 -- 172.21.15.62:0/34828764 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0272f0 con 0x56402fc6de20 2023-07-14T16:22:31.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.807+0000 7fc8377fe700 1 -- 172.21.15.62:0/34828764 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc84c027c80 con 0x56402fc6de20 2023-07-14T16:22:31.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.807+0000 7fc871c40b80 1 -- 172.21.15.62:0/34828764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc84c015070 con 0x56402fc6de20 2023-07-14T16:22:31.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.807+0000 7fc8377fe700 1 -- 172.21.15.62:0/34828764 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc84c027450 con 0x56402fc6de20 2023-07-14T16:22:31.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.807+0000 7fc871c40b80 10 client.17424 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:31.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.807+0000 7fc871c40b80 1 -- 172.21.15.62:0/34828764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56402f9eb2b0 con 0x56402fc6de20 2023-07-14T16:22:31.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.808+0000 7fc8377fe700 1 -- 172.21.15.62:0/34828764 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc84c015070 con 0x56402fc6de20 2023-07-14T16:22:31.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.808+0000 7fc8377fe700 1 -- 172.21.15.62:0/34828764 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc84c003b10 con 0x56402fc6de20 2023-07-14T16:22:31.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.808+0000 7fc871c40b80 10 client.17424 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:31.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.808+0000 7fc871c40b80 10 client.17424 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:22:31.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.808+0000 7fc871c40b80 20 client.17424 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:31.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.808+0000 7fc871c40b80 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56402fcd7560 0x56402fcd9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:31.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.808+0000 7fc871c40b80 4 client.17424 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:22:31.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.808+0000 7fc871c40b80 1 -- 172.21.15.62:0/34828764 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56402fb7a050 con 0x56402fcd7560 2023-07-14T16:22:31.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.809+0000 7fc857fff700 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56402fcd7560 0x56402fcd9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:31.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc8377fe700 10 client.17424 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:22:31.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc857fff700 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56402fcd7560 0x56402fcd9970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ad11dcb42b21c7c server_cookie=881e3843cf89252d in_seq=0 out_seq=0 2023-07-14T16:22:31.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc8377fe700 1 -- 172.21.15.62:0/34828764 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56402fb7a050 con 0x56402fcd7560 2023-07-14T16:22:31.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc8377fe700 10 client.17424 handle_command_reply: tid=0 2023-07-14T16:22:31.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 client.17424 shutdown 2023-07-14T16:22:31.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 -- 172.21.15.62:0/34828764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 msgr2=0x56402fc7a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7a9a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007d40 tx=0x7fc84c010f40 comp rx=0 tx=0).stop 2023-07-14T16:22:31.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 -- 172.21.15.62:0/34828764 shutdown_connections 2023-07-14T16:22:31.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56402fcd7560 0x56402fcd9970 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56402fc6ec70 0x56402fc7d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56402fc6d4d0 0x56402fc7a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 --2- 172.21.15.62:0/34828764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56402fc6de20 0x56402fc7a9a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.810+0000 7fc871c40b80 1 -- 172.21.15.62:0/34828764 >> 172.21.15.62:0/34828764 conn(0x56402fc62da0 msgr2=0x56402fc7efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.811+0000 7fc8377fe700 0 client.17424 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:22:31.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.811+0000 7fc871c40b80 1 -- 172.21.15.62:0/34828764 shutdown_connections 2023-07-14T16:22:31.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.811+0000 7fc871c40b80 1 -- 172.21.15.62:0/34828764 wait complete. 2023-07-14T16:22:31.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.811+0000 7fc871c40b80 20 client.17424 trim_cache size 0 max 16384 2023-07-14T16:22:31.794 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:22:31.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 -- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc8400a1150 msgr2=0x7fc8400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc8400a1150 0x7fc8400a3600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc850009ec0 tx=0x7fc850004b40 comp rx=0 tx=0).stop 2023-07-14T16:22:31.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 -- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 msgr2=0x7fc8581979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:31.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 0x7fc8581979d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc848005910 tx=0x7fc848005720 comp rx=0 tx=0).stop 2023-07-14T16:22:31.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 -- 172.21.15.62:0/2810430753 shutdown_connections 2023-07-14T16:22:31.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8581057a0 0x7fc85819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc858100560 0x7fc8581979d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc8400a1150 0x7fc8400a3600 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 --2- 172.21.15.62:0/2810430753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc858102e80 0x7fc858197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:31.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 -- 172.21.15.62:0/2810430753 >> 172.21.15.62:0/2810430753 conn(0x7fc8580fa000 msgr2=0x7fc8580ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:31.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 -- 172.21.15.62:0/2810430753 shutdown_connections 2023-07-14T16:22:31.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:31.812+0000 7fc85e3ff700 1 -- 172.21.15.62:0/2810430753 wait complete. 2023-07-14T16:22:31.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:22:32.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 -- 172.21.15.62:0/3983694715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac8100cf0 msgr2=0x7f0ac8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/3983694715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac8100cf0 0x7f0ac8101140 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009990 tx=0x7f0ab8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 -- 172.21.15.62:0/3983694715 shutdown_connections 2023-07-14T16:22:32.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/3983694715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac8101680 0x7f0ac8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/3983694715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac80ffb00 0x7f0ac80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/3983694715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac8100cf0 0x7f0ac8101140 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 -- 172.21.15.62:0/3983694715 >> 172.21.15.62:0/3983694715 conn(0x7f0ac80fb120 msgr2=0x7f0ac80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:32.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 -- 172.21.15.62:0/3983694715 shutdown_connections 2023-07-14T16:22:32.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.080+0000 7f0ad01ff700 1 -- 172.21.15.62:0/3983694715 wait complete. 2023-07-14T16:22:32.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.082+0000 7f0ad01ff700 1 Processor -- start 2023-07-14T16:22:32.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.082+0000 7f0ad01ff700 1 -- start start 2023-07-14T16:22:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.082+0000 7f0ad01ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac80ffb00 0x7f0ac8195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0ad01ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac8100cf0 0x7f0ac8195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0ad01ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8101680 0x7f0ac819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0ad01ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819d2a0 con 0x7f0ac80ffb00 2023-07-14T16:22:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0ad01ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819d3e0 con 0x7f0ac8101680 2023-07-14T16:22:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0ad01ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819d570 con 0x7f0ac8100cf0 2023-07-14T16:22:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acdf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac80ffb00 0x7f0ac8195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acdf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac80ffb00 0x7f0ac8195720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39056/0 (socket says 172.21.15.62:39056) 2023-07-14T16:22:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acdf9b700 1 -- 172.21.15.62:0/735658158 learned_addr learned my addr 172.21.15.62:0/735658158 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acd79a700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac8100cf0 0x7f0ac8195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0ace79c700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8101680 0x7f0ac819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acdf9b700 1 -- 172.21.15.62:0/735658158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac8100cf0 msgr2=0x7f0ac8195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac8100cf0 0x7f0ac8195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acdf9b700 1 -- 172.21.15.62:0/735658158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8101680 msgr2=0x7f0ac819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8101680 0x7f0ac819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acdf9b700 1 -- 172.21.15.62:0/735658158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80095f0 con 0x7f0ac80ffb00 2023-07-14T16:22:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.083+0000 7f0acd79a700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac8100cf0 0x7f0ac8195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.084+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac80ffb00 0x7f0ac8195720 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400e970 tx=0x7f0ac400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.084+0000 7f0abeffd700 1 -- 172.21.15.62:0/735658158 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4009790 con 0x7f0ac80ffb00 2023-07-14T16:22:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.084+0000 7f0abeffd700 1 -- 172.21.15.62:0/735658158 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ac40044b0 con 0x7f0ac80ffb00 2023-07-14T16:22:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.084+0000 7f0ad01ff700 1 -- 172.21.15.62:0/735658158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac819d850 con 0x7f0ac80ffb00 2023-07-14T16:22:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.084+0000 7f0abeffd700 1 -- 172.21.15.62:0/735658158 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4003dd0 con 0x7f0ac80ffb00 2023-07-14T16:22:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.084+0000 7f0ad01ff700 1 -- 172.21.15.62:0/735658158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac819dd60 con 0x7f0ac80ffb00 2023-07-14T16:22:32.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.088+0000 7f0abeffd700 1 -- 172.21.15.62:0/735658158 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0ac4004000 con 0x7f0ac80ffb00 2023-07-14T16:22:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.089+0000 7f0ae3be2b80 1 Processor -- start 2023-07-14T16:22:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.089+0000 7f0ae3be2b80 1 -- start start 2023-07-14T16:22:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b598e0 0x55e795b59cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5a230 0x55e795b5aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 0x55e795b5b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795a5acb0 con 0x55e795b598e0 2023-07-14T16:22:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795aada40 con 0x55e795b5a230 2023-07-14T16:22:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795aa6ed0 con 0x55e795b5b080 2023-07-14T16:22:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ace79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b598e0 0x55e795b59cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ace79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b598e0 0x55e795b59cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39064/0 (socket says 172.21.15.62:39064) 2023-07-14T16:22:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0ace79c700 1 -- 172.21.15.62:0/4048165351 learned_addr learned my addr 172.21.15.62:0/4048165351 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0acd79a700 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 0x55e795b5b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.090+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5a230 0x55e795b5aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0ace79c700 1 -- 172.21.15.62:0/4048165351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 msgr2=0x55e795b5b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0ace79c700 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 0x55e795b5b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0ace79c700 1 -- 172.21.15.62:0/4048165351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5a230 msgr2=0x55e795b5aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0ace79c700 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5a230 0x55e795b5aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0abeffd700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0ab40a1150 0x7f0ab40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0ace79c700 1 -- 172.21.15.62:0/4048165351 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7958d4de0 con 0x55e795b598e0 2023-07-14T16:22:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0abeffd700 1 -- 172.21.15.62:0/735658158 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0ac4014070 con 0x7f0ac80ffb00 2023-07-14T16:22:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0ace79c700 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b598e0 0x55e795b59cf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009dc0 tx=0x7f0ac00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc882f14ef78cdc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.091+0000 7f0ace79c700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0ab40a1150 0x7f0ab40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.092+0000 7f0ab3fff700 1 -- 172.21.15.62:0/4048165351 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0014a10 con 0x55e795b598e0 2023-07-14T16:22:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.092+0000 7f0ab3fff700 1 -- 172.21.15.62:0/4048165351 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ac001b070 con 0x55e795b598e0 2023-07-14T16:22:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.092+0000 7f0ab3fff700 1 -- 172.21.15.62:0/4048165351 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac001f340 con 0x55e795b598e0 2023-07-14T16:22:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/4048165351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b598e0 msgr2=0x55e795b59cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ace79c700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0ab40a1150 0x7f0ab40a3600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0ac001e000 tx=0x7f0ac001f910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b598e0 0x55e795b59cf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009dc0 tx=0x7f0ac00077e0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/4048165351 shutdown_connections 2023-07-14T16:22:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 0x55e795b5b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b598e0 0x55e795b59cf0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/4048165351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5a230 0x55e795b5aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/4048165351 >> 172.21.15.62:0/4048165351 conn(0x55e795b4ada0 msgr2=0x55e795b501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/4048165351 shutdown_connections 2023-07-14T16:22:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.093+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/4048165351 wait complete. 2023-07-14T16:22:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.094+0000 7f0ae3be2b80 1 -- start start 2023-07-14T16:22:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.095+0000 7f0ae3be2b80 1 Processor -- start 2023-07-14T16:22:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.095+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b63520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.095+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b63a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.095+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b6aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.095+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795a46ac0 con 0x55e795b5a230 2023-07-14T16:22:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.095+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795a467c0 con 0x55e795b5b080 2023-07-14T16:22:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.095+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79592a920 con 0x55e795b598e0 2023-07-14T16:22:32.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.095+0000 7f0ae3be2b80 10 client.0 _ll_register_callbacks cb 0x55e795a44760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b63520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b63520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:34150/0 (socket says 172.21.15.62:34150) 2023-07-14T16:22:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 -- 172.21.15.62:0/811311690 learned_addr learned my addr 172.21.15.62:0/811311690 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acd79a700 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b63a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0ace79c700 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b6aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 -- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 msgr2=0x55e795b6aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b6aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 -- 172.21.15.62:0/811311690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 msgr2=0x55e795b63a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b63a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 -- 172.21.15.62:0/811311690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac0009a20 con 0x55e795b598e0 2023-07-14T16:22:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0ace79c700 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b6aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b63520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40a1fa0 tx=0x7f0ac40d00f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=737ef8675bd28322 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.096+0000 7f0ab17fa700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:22:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.097+0000 7f0ab17fa700 1 -- 172.21.15.62:0/811311690 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40d5040 con 0x55e795b598e0 2023-07-14T16:22:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.097+0000 7f0ab17fa700 1 -- 172.21.15.62:0/811311690 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0ac400b2c0 con 0x55e795b598e0 2023-07-14T16:22:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.097+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/811311690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e795a004f0 con 0x55e795b598e0 2023-07-14T16:22:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.097+0000 7f0ab17fa700 1 -- 172.21.15.62:0/811311690 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ac4099bc0 con 0x55e795b598e0 2023-07-14T16:22:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.097+0000 7f0ae3be2b80 10 client.27004 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.097+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/811311690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e7958bdc60 con 0x55e795b598e0 2023-07-14T16:22:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.097+0000 7f0ab17fa700 1 -- 172.21.15.62:0/811311690 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ac40b6020 con 0x55e795b598e0 2023-07-14T16:22:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.098+0000 7f0ab17fa700 1 -- 172.21.15.62:0/811311690 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ac4099570 con 0x55e795b598e0 2023-07-14T16:22:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.098+0000 7f0ae3be2b80 10 client.27004 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.098+0000 7f0ae3be2b80 10 client.27004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.098+0000 7f0ae3be2b80 20 client.27004 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.098+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e795b6f330 0x55e795b71740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.098+0000 7f0ae3be2b80 4 client.27004 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.098+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/811311690 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e795a9feb0 con 0x55e795b6f330 2023-07-14T16:22:32.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.098+0000 7f0acd79a700 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e795b6f330 0x55e795b71740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.099+0000 7f0acd79a700 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e795b6f330 0x55e795b71740 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b824cca20f73680 server_cookie=5da8498f304b18a6 in_seq=0 out_seq=0 2023-07-14T16:22:32.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.099+0000 7f0ab17fa700 10 client.27004 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:32.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ab17fa700 1 -- 172.21.15.62:0/811311690 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e795a9feb0 con 0x55e795b6f330 2023-07-14T16:22:32.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ab17fa700 10 client.27004 handle_command_reply: tid=0 2023-07-14T16:22:32.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 client.27004 shutdown 2023-07-14T16:22:32.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/811311690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 msgr2=0x55e795b63520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b63520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40a1fa0 tx=0x7f0ac40d00f0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/811311690 shutdown_connections 2023-07-14T16:22:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e795b6f330 0x55e795b71740 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b6aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ab17fa700 0 client.27004 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b63520 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/811311690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b63a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/811311690 >> 172.21.15.62:0/811311690 conn(0x55e795b4ada0 msgr2=0x55e795b4f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/811311690 shutdown_connections 2023-07-14T16:22:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.100+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/811311690 wait complete. 2023-07-14T16:22:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.101+0000 7f0ae3be2b80 20 client.27004 trim_cache size 0 max 16384 2023-07-14T16:22:32.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0ae3be2b80 1 Processor -- start 2023-07-14T16:22:32.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0ae3be2b80 1 -- start start 2023-07-14T16:22:32.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b5dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795b6c790 con 0x55e795b5a230 2023-07-14T16:22:32.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795b629d0 con 0x55e795b5b080 2023-07-14T16:22:32.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795b4a970 con 0x55e795b598e0 2023-07-14T16:22:32.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.119+0000 7f0acd79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acd79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39080/0 (socket says 172.21.15.62:39080) 2023-07-14T16:22:32.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acd79a700 1 -- 172.21.15.62:0/884411672 learned_addr learned my addr 172.21.15.62:0/884411672 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:32.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0ace79c700 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b5dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acd79a700 1 -- 172.21.15.62:0/884411672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 msgr2=0x55e795b614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:32.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acd79a700 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acd79a700 1 -- 172.21.15.62:0/884411672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 msgr2=0x55e795b5dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:32.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acd79a700 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b5dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acd79a700 1 -- 172.21.15.62:0/884411672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7958fb250 con 0x55e795b5a230 2023-07-14T16:22:32.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.120+0000 7f0acd79a700 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6aa60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80059d0 tx=0x7f0ab8005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72068c11cf937069 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:32.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0abcff9700 1 -- 172.21.15.62:0/884411672 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8005470 con 0x55e795b5a230 2023-07-14T16:22:32.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0abcff9700 1 -- 172.21.15.62:0/884411672 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ab801c070 con 0x55e795b5a230 2023-07-14T16:22:32.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0abcff9700 1 -- 172.21.15.62:0/884411672 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800bb70 con 0x55e795b5a230 2023-07-14T16:22:32.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/884411672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 msgr2=0x55e795b6aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6aa60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80059d0 tx=0x7f0ab8005940 comp rx=0 tx=0).stop 2023-07-14T16:22:32.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/884411672 shutdown_connections 2023-07-14T16:22:32.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b5b080 0x55e795b5dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b598e0 0x55e795b614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/884411672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6aa60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.121+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/884411672 >> 172.21.15.62:0/884411672 conn(0x55e795b4ada0 msgr2=0x55e795b621f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:32.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.122+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/884411672 shutdown_connections 2023-07-14T16:22:32.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.122+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/884411672 wait complete. 2023-07-14T16:22:32.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.122+0000 7f0ae3be2b80 1 -- start start 2023-07-14T16:22:32.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.122+0000 7f0ae3be2b80 1 Processor -- start 2023-07-14T16:22:32.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b598e0 0x55e795b6a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0ae3be2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 0x55e795b66e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795b67500 con 0x55e795b5a230 2023-07-14T16:22:32.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795b67670 con 0x55e795b598e0 2023-07-14T16:22:32.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0ae3be2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e795b67830 con 0x55e795b5b080 2023-07-14T16:22:32.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0ae3be2b80 10 client.0 _ll_register_callbacks cb 0x7f0ac4004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:32.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0acd79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0acd79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39094/0 (socket says 172.21.15.62:39094) 2023-07-14T16:22:32.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0acd79a700 1 -- 172.21.15.62:0/135183032 learned_addr learned my addr 172.21.15.62:0/135183032 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:32.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b598e0 0x55e795b6a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.123+0000 7f0ace79c700 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 0x55e795b66e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0acd79a700 1 -- 172.21.15.62:0/135183032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 msgr2=0x55e795b66e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:32.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0acd79a700 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 0x55e795b66e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0acd79a700 1 -- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b598e0 msgr2=0x55e795b6a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:32.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0acd79a700 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b598e0 0x55e795b6a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0acd79a700 1 -- 172.21.15.62:0/135183032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac8108870 con 0x55e795b5a230 2023-07-14T16:22:32.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b598e0 0x55e795b6a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:32.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0acd79a700 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6a8c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8003f60 tx=0x7f0ab8005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d96eda0894e950 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:32.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0ab17fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:32.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0ab17fa700 1 -- 172.21.15.62:0/135183032 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800bd00 con 0x55e795b5a230 2023-07-14T16:22:32.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/135183032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ac001b070 con 0x55e795b5a230 2023-07-14T16:22:32.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.124+0000 7f0ab17fa700 1 -- 172.21.15.62:0/135183032 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0ab800be60 con 0x55e795b5a230 2023-07-14T16:22:32.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ab17fa700 1 -- 172.21.15.62:0/135183032 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab8005690 con 0x55e795b5a230 2023-07-14T16:22:32.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ae3be2b80 10 client.17442 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:32.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/135183032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e7958d32b0 con 0x55e795b5a230 2023-07-14T16:22:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ab17fa700 1 -- 172.21.15.62:0/135183032 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ac001b070 con 0x55e795b5a230 2023-07-14T16:22:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ab17fa700 1 -- 172.21.15.62:0/135183032 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ab8004a30 con 0x55e795b5a230 2023-07-14T16:22:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ae3be2b80 10 client.17442 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ae3be2b80 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ae3be2b80 20 client.17442 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e795bc4310 0x55e795bc6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ae3be2b80 4 client.17442 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:22:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.125+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/135183032 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e795a62050 con 0x55e795bc4310 2023-07-14T16:22:32.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.126+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e795bc4310 0x55e795bc6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:32.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.126+0000 7f0acdf9b700 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e795bc4310 0x55e795bc6720 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e936501a37f97e server_cookie=e2db3a8e425951e1 in_seq=0 out_seq=0 2023-07-14T16:22:32.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.126+0000 7f0ab17fa700 10 client.17442 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:32.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ab17fa700 1 -- 172.21.15.62:0/135183032 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e795a62050 con 0x55e795bc4310 2023-07-14T16:22:32.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ab17fa700 10 client.17442 handle_command_reply: tid=0 2023-07-14T16:22:32.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 client.17442 shutdown 2023-07-14T16:22:32.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/135183032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 msgr2=0x55e795b6a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6a8c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8003f60 tx=0x7f0ab8005360 comp rx=0 tx=0).stop 2023-07-14T16:22:32.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/135183032 shutdown_connections 2023-07-14T16:22:32.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e795b5b080 0x55e795b66e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e795bc4310 0x55e795bc6720 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e795b598e0 0x55e795b6a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 --2- 172.21.15.62:0/135183032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e795b5a230 0x55e795b6a8c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/135183032 >> 172.21.15.62:0/135183032 conn(0x55e795b4ada0 msgr2=0x55e795b614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:32.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ab17fa700 0 client.17442 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:32.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/135183032 shutdown_connections 2023-07-14T16:22:32.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 1 -- 172.21.15.62:0/135183032 wait complete. 2023-07-14T16:22:32.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.127+0000 7f0ae3be2b80 20 client.17442 trim_cache size 0 max 16384 2023-07-14T16:22:32.111 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:22:32.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 -- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0ab40a1150 msgr2=0x7f0ab40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0ab40a1150 0x7f0ab40a3600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0ac001e000 tx=0x7f0ac001f910 comp rx=0 tx=0).stop 2023-07-14T16:22:32.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 -- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac80ffb00 msgr2=0x7f0ac8195720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:32.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac80ffb00 0x7f0ac8195720 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400e970 tx=0x7f0ac400ed30 comp rx=0 tx=0).stop 2023-07-14T16:22:32.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 -- 172.21.15.62:0/735658158 shutdown_connections 2023-07-14T16:22:32.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0ab40a1150 0x7f0ab40a3600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ac8101680 0x7f0ac819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ac80ffb00 0x7f0ac8195720 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 --2- 172.21.15.62:0/735658158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ac8100cf0 0x7f0ac8195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:32.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 -- 172.21.15.62:0/735658158 >> 172.21.15.62:0/735658158 conn(0x7f0ac80fb120 msgr2=0x7f0ac80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:32.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 -- 172.21.15.62:0/735658158 shutdown_connections 2023-07-14T16:22:32.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:32.128+0000 7f0ad01ff700 1 -- 172.21.15.62:0/735658158 wait complete. 2023-07-14T16:22:32.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:31 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/96210268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:22:32.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:31 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/96210268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:22:32.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:31 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/96210268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:22:33.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:32 smithi062 ceph-mon[95543]: pgmap v564: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:22:33.125 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:22:33.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:22:33.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:32 smithi081 ceph-mon[127795]: pgmap v564: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:22:33.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:32 smithi164 ceph-mon[97427]: pgmap v564: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:22:33.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 -- 172.21.15.62:0/321054221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc101660 msgr2=0x7fd8fc107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 --2- 172.21.15.62:0/321054221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc101660 0x7fd8fc107f20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400b0a0 tx=0x7fd8f400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 -- 172.21.15.62:0/321054221 shutdown_connections 2023-07-14T16:22:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 --2- 172.21.15.62:0/321054221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc101660 0x7fd8fc107f20 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 --2- 172.21.15.62:0/321054221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8fc0ffae0 0x7fd8fc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 --2- 172.21.15.62:0/321054221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8fc100cd0 0x7fd8fc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 -- 172.21.15.62:0/321054221 >> 172.21.15.62:0/321054221 conn(0x7fd8fc0fb0e0 msgr2=0x7fd8fc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 -- 172.21.15.62:0/321054221 shutdown_connections 2023-07-14T16:22:33.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.391+0000 7fd9027ff700 1 -- 172.21.15.62:0/321054221 wait complete. 2023-07-14T16:22:33.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.392+0000 7fd9027ff700 1 Processor -- start 2023-07-14T16:22:33.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd9027ff700 1 -- start start 2023-07-14T16:22:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd9027ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc0ffae0 0x7fd8fc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd9027ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8fc100cd0 0x7fd8fc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd9027ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8fc101660 0x7fd8fc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd9027ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc19f610 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd9027ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc19f750 con 0x7fd8fc100cd0 2023-07-14T16:22:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd9027ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc19f8e0 con 0x7fd8fc101660 2023-07-14T16:22:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd8fbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc0ffae0 0x7fd8fc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd8fbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc0ffae0 0x7fd8fc197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53010/0 (socket says 172.21.15.62:53010) 2023-07-14T16:22:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.393+0000 7fd8fbfff700 1 -- 172.21.15.62:0/1266091416 learned_addr learned my addr 172.21.15.62:0/1266091416 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd900d9c700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8fc101660 0x7fd8fc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8fc100cd0 0x7fd8fc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8fbfff700 1 -- 172.21.15.62:0/1266091416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8fc101660 msgr2=0x7fd8fc19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8fbfff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8fc101660 0x7fd8fc19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8fbfff700 1 -- 172.21.15.62:0/1266091416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8fc100cd0 msgr2=0x7fd8fc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8fbfff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8fc100cd0 0x7fd8fc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8fbfff700 1 -- 172.21.15.62:0/1266091416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f4009d00 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8fc100cd0 0x7fd8fc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:33.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8fbfff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc0ffae0 0x7fd8fc197a90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e950 tx=0x7fd8ec00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.394+0000 7fd8f97fa700 1 -- 172.21.15.62:0/1266091416 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec009770 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.395+0000 7fd9027ff700 1 -- 172.21.15.62:0/1266091416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc19fbc0 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.395+0000 7fd8f97fa700 1 -- 172.21.15.62:0/1266091416 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8ec00eea0 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.395+0000 7fd8f97fa700 1 -- 172.21.15.62:0/1266091416 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec003cc0 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.395+0000 7fd9027ff700 1 -- 172.21.15.62:0/1266091416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc1a00d0 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.399+0000 7fd8f97fa700 1 -- 172.21.15.62:0/1266091416 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd8ec003e20 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.399+0000 7fd91615bb80 1 Processor -- start 2023-07-14T16:22:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd91615bb80 1 -- start start 2023-07-14T16:22:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031b8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 0x55cb031b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b9cf0 0x55cb031ba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd91615bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb030bdd30 con 0x55cb031b9cf0 2023-07-14T16:22:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd91615bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb03110ac0 con 0x55cb031b8ea0 2023-07-14T16:22:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd91615bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb03109f50 con 0x55cb031b8550 2023-07-14T16:22:33.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b9cf0 0x55cb031ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b9cf0 0x55cb031ba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53020/0 (socket says 172.21.15.62:53020) 2023-07-14T16:22:33.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/3039896235 learned_addr learned my addr 172.21.15.62:0/3039896235 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.400+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3039896235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.401+0000 7fd900d9c700 1 -- 172.21.15.62:0/3039896235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 msgr2=0x55cb031b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.401+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3039896235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 0x55cb031b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.401+0000 7fd900d9c700 1 -- 172.21.15.62:0/3039896235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b9cf0 msgr2=0x55cb031ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.401+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3039896235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b9cf0 0x55cb031ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.401+0000 7fd900d9c700 1 -- 172.21.15.62:0/3039896235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb02f37e60 con 0x55cb031b8550 2023-07-14T16:22:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.401+0000 7fd8f97fa700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8e40a1150 0x7fd8e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.401+0000 7fd8f97fa700 1 -- 172.21.15.62:0/1266091416 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8ec014070 con 0x7fd8fc0ffae0 2023-07-14T16:22:33.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.402+0000 7fd8fbfff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8e40a1150 0x7fd8e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.402+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3039896235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031b8960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009fd0 tx=0x7fd8f4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfbea8e49da1eab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.402+0000 7fd8e27fc700 1 -- 172.21.15.62:0/3039896235 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400e040 con 0x55cb031b8550 2023-07-14T16:22:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.402+0000 7fd8fbfff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8e40a1150 0x7fd8e40a3600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0964e0 tx=0x7fd8ec0036f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.402+0000 7fd8e27fc700 1 -- 172.21.15.62:0/3039896235 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8f400ba50 con 0x55cb031b8550 2023-07-14T16:22:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.402+0000 7fd8e27fc700 1 -- 172.21.15.62:0/3039896235 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4003dc0 con 0x55cb031b8550 2023-07-14T16:22:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.403+0000 7fd91615bb80 1 -- 172.21.15.62:0/3039896235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 msgr2=0x55cb031b8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.403+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3039896235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031b8960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009fd0 tx=0x7fd8f4009c30 comp rx=0 tx=0).stop 2023-07-14T16:22:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.404+0000 7fd91615bb80 1 -- 172.21.15.62:0/3039896235 shutdown_connections 2023-07-14T16:22:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.404+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3039896235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b9cf0 0x55cb031ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.404+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3039896235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031b8960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.404+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3039896235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 0x55cb031b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.404+0000 7fd91615bb80 1 -- 172.21.15.62:0/3039896235 >> 172.21.15.62:0/3039896235 conn(0x55cb031ade20 msgr2=0x55cb031b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.404+0000 7fd91615bb80 1 -- 172.21.15.62:0/3039896235 shutdown_connections 2023-07-14T16:22:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.404+0000 7fd91615bb80 1 -- 172.21.15.62:0/3039896235 wait complete. 2023-07-14T16:22:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.404+0000 7fd91615bb80 1 -- start start 2023-07-14T16:22:33.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.405+0000 7fd91615bb80 1 Processor -- start 2023-07-14T16:22:33.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.405+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8550 0x55cb031c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.405+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 0x55cb031c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd91615bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb030a9b40 con 0x55cb031b8ea0 2023-07-14T16:22:33.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd91615bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb030a9840 con 0x55cb031b8550 2023-07-14T16:22:33.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd91615bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb02f8d9a0 con 0x55cb031b9cf0 2023-07-14T16:22:33.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd91615bb80 10 client.0 _ll_register_callbacks cb 0x55cb030a77e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:33.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53028/0 (socket says 172.21.15.62:53028) 2023-07-14T16:22:33.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/3811751050 learned_addr learned my addr 172.21.15.62:0/3811751050 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8550 0x55cb031c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 0x55cb031c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/3811751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 msgr2=0x55cb031c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 0x55cb031c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/3811751050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8550 msgr2=0x55cb031c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.406+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8550 0x55cb031c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/3811751050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc108850 con 0x55cb031b8ea0 2023-07-14T16:22:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 0x55cb031c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c2650 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000b960 tx=0x7fd8f000bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa02c1448dba8cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd8db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd8db7fe700 1 -- 172.21.15.62:0/3811751050 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0004040 con 0x55cb031b8ea0 2023-07-14T16:22:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd8db7fe700 1 -- 172.21.15.62:0/3811751050 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8f001b440 con 0x55cb031b8ea0 2023-07-14T16:22:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd91615bb80 1 -- 172.21.15.62:0/3811751050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb03063570 con 0x55cb031b8ea0 2023-07-14T16:22:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd8db7fe700 1 -- 172.21.15.62:0/3811751050 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8f0021030 con 0x55cb031b8ea0 2023-07-14T16:22:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.407+0000 7fd91615bb80 10 client.17454 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd91615bb80 1 -- 172.21.15.62:0/3811751050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cb02f20ce0 con 0x55cb031b8ea0 2023-07-14T16:22:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd8db7fe700 1 -- 172.21.15.62:0/3811751050 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cb03063570 con 0x55cb031b8ea0 2023-07-14T16:22:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd8db7fe700 1 -- 172.21.15.62:0/3811751050 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd8f00116a0 con 0x55cb031b8ea0 2023-07-14T16:22:33.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd91615bb80 10 client.17454 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:33.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd91615bb80 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:33.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd91615bb80 20 client.17454 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:33.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb031cdf50 0x55cb031d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd91615bb80 4 client.17454 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:33.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.408+0000 7fd91615bb80 1 -- 172.21.15.62:0/3811751050 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb03102f30 con 0x55cb031cdf50 2023-07-14T16:22:33.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.409+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb031cdf50 0x55cb031d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.409+0000 7fd8db7fe700 10 client.17454 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:33.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.409+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb031cdf50 0x55cb031d0360 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d63f0a883434a server_cookie=320a34bb2fefa5ae in_seq=0 out_seq=0 2023-07-14T16:22:33.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd8db7fe700 1 -- 172.21.15.62:0/3811751050 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb03102f30 con 0x55cb031cdf50 2023-07-14T16:22:33.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd8db7fe700 10 client.17454 handle_command_reply: tid=0 2023-07-14T16:22:33.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 client.17454 shutdown 2023-07-14T16:22:33.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 -- 172.21.15.62:0/3811751050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 msgr2=0x55cb031c2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c2650 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000b960 tx=0x7fd8f000bc70 comp rx=0 tx=0).stop 2023-07-14T16:22:33.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 -- 172.21.15.62:0/3811751050 shutdown_connections 2023-07-14T16:22:33.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 0x55cb031c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb031cdf50 0x55cb031d0360 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8550 0x55cb031c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3811751050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c2650 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd8db7fe700 0 client.17454 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:33.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.410+0000 7fd91615bb80 1 -- 172.21.15.62:0/3811751050 >> 172.21.15.62:0/3811751050 conn(0x55cb031ade20 msgr2=0x55cb031b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.411+0000 7fd91615bb80 1 -- 172.21.15.62:0/3811751050 shutdown_connections 2023-07-14T16:22:33.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.411+0000 7fd91615bb80 1 -- 172.21.15.62:0/3811751050 wait complete. 2023-07-14T16:22:33.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.411+0000 7fd91615bb80 20 client.17454 trim_cache size 0 max 16384 2023-07-14T16:22:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.431+0000 7fd91615bb80 1 Processor -- start 2023-07-14T16:22:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.431+0000 7fd91615bb80 1 -- start start 2023-07-14T16:22:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.431+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8550 0x55cb031c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 0x55cb031c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 0x55cb031c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd91615bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb031c15f0 con 0x55cb031b8550 2023-07-14T16:22:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd91615bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb031c0960 con 0x55cb031b8ea0 2023-07-14T16:22:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd91615bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb031b2510 con 0x55cb031b9cf0 2023-07-14T16:22:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8550 0x55cb031c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8550 0x55cb031c1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53030/0 (socket says 172.21.15.62:53030) 2023-07-14T16:22:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 -- 172.21.15.62:0/3712729237 learned_addr learned my addr 172.21.15.62:0/3712729237 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/3712729237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 0x55cb031c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 -- 172.21.15.62:0/3712729237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 msgr2=0x55cb031c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3712729237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 0x55cb031c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 -- 172.21.15.62:0/3712729237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 msgr2=0x55cb031c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3712729237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 0x55cb031c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 -- 172.21.15.62:0/3712729237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb031c1140 con 0x55cb031b8550 2023-07-14T16:22:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.432+0000 7fd900d9c700 1 --2- 172.21.15.62:0/3712729237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8550 0x55cb031c1ec0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400b0a0 tx=0x7fd8f40077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23ba4a7ca82f8b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.433+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/3712729237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4009360 con 0x55cb031b8550 2023-07-14T16:22:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.433+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/3712729237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8f4032460 con 0x55cb031b8550 2023-07-14T16:22:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.433+0000 7fd8e2ffd700 1 -- 172.21.15.62:0/3712729237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f401c5c0 con 0x55cb031b8550 2023-07-14T16:22:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.433+0000 7fd91615bb80 1 -- 172.21.15.62:0/3712729237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8550 msgr2=0x55cb031c1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.433+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3712729237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8550 0x55cb031c1ec0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400b0a0 tx=0x7fd8f40077c0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 -- 172.21.15.62:0/3712729237 shutdown_connections 2023-07-14T16:22:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3712729237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b9cf0 0x55cb031c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3712729237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8550 0x55cb031c1ec0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 --2- 172.21.15.62:0/3712729237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b8ea0 0x55cb031c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 -- 172.21.15.62:0/3712729237 >> 172.21.15.62:0/3712729237 conn(0x55cb031ade20 msgr2=0x55cb031bcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 -- 172.21.15.62:0/3712729237 shutdown_connections 2023-07-14T16:22:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 -- 172.21.15.62:0/3712729237 wait complete. 2023-07-14T16:22:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 -- start start 2023-07-14T16:22:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.434+0000 7fd91615bb80 1 Processor -- start 2023-07-14T16:22:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd91615bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b9cf0 0x55cb03221e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd91615bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb031c6f20 con 0x55cb031b8ea0 2023-07-14T16:22:33.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd91615bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb031c7090 con 0x55cb031b9cf0 2023-07-14T16:22:33.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd91615bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb032223c0 con 0x55cb031b8550 2023-07-14T16:22:33.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd91615bb80 10 client.0 _ll_register_callbacks cb 0x55cb030a77e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:33.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53040/0 (socket says 172.21.15.62:53040) 2023-07-14T16:22:33.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/2318184789 learned_addr learned my addr 172.21.15.62:0/2318184789 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd900d9c700 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b9cf0 0x55cb03221e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.435+0000 7fd900d9c700 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/2318184789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 msgr2=0x55cb031c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/2318184789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b9cf0 msgr2=0x55cb03221e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b9cf0 0x55cb03221e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8fb7fe700 1 -- 172.21.15.62:0/2318184789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f402d040 con 0x55cb031b8ea0 2023-07-14T16:22:33.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd900d9c700 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:33.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8fb7fe700 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c6950 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0011b00 tx=0x7fd8f0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbe38897beb646a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:33.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8db7fe700 1 -- 172.21.15.62:0/2318184789 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0021040 con 0x55cb031b8ea0 2023-07-14T16:22:33.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd8db7fe700 1 -- 172.21.15.62:0/2318184789 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8f0022070 con 0x55cb031b8ea0 2023-07-14T16:22:33.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.436+0000 7fd91615bb80 1 -- 172.21.15.62:0/2318184789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8f400ba50 con 0x55cb031b8ea0 2023-07-14T16:22:33.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd8db7fe700 1 -- 172.21.15.62:0/2318184789 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8f0011640 con 0x55cb031b8ea0 2023-07-14T16:22:33.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd91615bb80 10 client.17460 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:33.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd91615bb80 1 -- 172.21.15.62:0/2318184789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cb02f5e2d0 con 0x55cb031b8ea0 2023-07-14T16:22:33.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd8db7fe700 1 -- 172.21.15.62:0/2318184789 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8f003e020 con 0x55cb031b8ea0 2023-07-14T16:22:33.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd8db7fe700 1 -- 172.21.15.62:0/2318184789 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd8f0004710 con 0x55cb031b8ea0 2023-07-14T16:22:33.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd91615bb80 10 client.17460 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:33.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd91615bb80 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:33.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd91615bb80 20 client.17460 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:33.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd91615bb80 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb03223d90 0x55cb032261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd91615bb80 4 client.17460 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:22:33.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.437+0000 7fd91615bb80 1 -- 172.21.15.62:0/2318184789 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cb030c50d0 con 0x55cb03223d90 2023-07-14T16:22:33.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.438+0000 7fd900d9c700 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb03223d90 0x55cb032261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.438+0000 7fd900d9c700 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb03223d90 0x55cb032261a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eea474c12368c0b server_cookie=1e48791256e7d790 in_seq=0 out_seq=0 2023-07-14T16:22:33.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.438+0000 7fd8db7fe700 10 client.17460 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:33.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.439+0000 7fd8db7fe700 1 -- 172.21.15.62:0/2318184789 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cb030c50d0 con 0x55cb03223d90 2023-07-14T16:22:33.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.439+0000 7fd8db7fe700 10 client.17460 handle_command_reply: tid=0 2023-07-14T16:22:33.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.439+0000 7fd91615bb80 1 client.17460 shutdown 2023-07-14T16:22:33.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.439+0000 7fd91615bb80 1 -- 172.21.15.62:0/2318184789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 msgr2=0x55cb031c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.439+0000 7fd91615bb80 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c6950 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0011b00 tx=0x7fd8f0003f90 comp rx=0 tx=0).stop 2023-07-14T16:22:33.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.439+0000 7fd91615bb80 1 -- 172.21.15.62:0/2318184789 shutdown_connections 2023-07-14T16:22:33.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.439+0000 7fd91615bb80 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb03223d90 0x55cb032261a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.440+0000 7fd91615bb80 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb031b9cf0 0x55cb03221e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.440+0000 7fd91615bb80 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb031b8550 0x55cb031c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.440+0000 7fd91615bb80 1 --2- 172.21.15.62:0/2318184789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb031b8ea0 0x55cb031c6950 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.440+0000 7fd91615bb80 1 -- 172.21.15.62:0/2318184789 >> 172.21.15.62:0/2318184789 conn(0x55cb031ade20 msgr2=0x55cb031c1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.440+0000 7fd8db7fe700 0 client.17460 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:33.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.440+0000 7fd91615bb80 1 -- 172.21.15.62:0/2318184789 shutdown_connections 2023-07-14T16:22:33.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.440+0000 7fd91615bb80 1 -- 172.21.15.62:0/2318184789 wait complete. 2023-07-14T16:22:33.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.440+0000 7fd91615bb80 20 client.17460 trim_cache size 0 max 16384 2023-07-14T16:22:33.425 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:22:33.425 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:22:33.425 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "018bc4b4-1a0c-412c-8a2f-a0860ec1bbdf", 2023-07-14T16:22:33.425 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:22:33.426 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:22:33.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 -- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8e40a1150 msgr2=0x7fd8e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8e40a1150 0x7fd8e40a3600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0964e0 tx=0x7fd8ec0036f0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 -- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc0ffae0 msgr2=0x7fd8fc197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc0ffae0 0x7fd8fc197a90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e950 tx=0x7fd8ec00ec60 comp rx=0 tx=0).stop 2023-07-14T16:22:33.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 -- 172.21.15.62:0/1266091416 shutdown_connections 2023-07-14T16:22:33.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8fc101660 0x7fd8fc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc0ffae0 0x7fd8fc197a90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8e40a1150 0x7fd8e40a3600 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 --2- 172.21.15.62:0/1266091416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8fc100cd0 0x7fd8fc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 -- 172.21.15.62:0/1266091416 >> 172.21.15.62:0/1266091416 conn(0x7fd8fc0fb0e0 msgr2=0x7fd8fc1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 -- 172.21.15.62:0/1266091416 shutdown_connections 2023-07-14T16:22:33.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.441+0000 7fd9027ff700 1 -- 172.21.15.62:0/1266091416 wait complete. 2023-07-14T16:22:33.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:22:33.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.711+0000 7f5f4afff700 1 -- 172.21.15.62:0/1879593761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44072f70 msgr2=0x7f5f440733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.711+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1879593761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44072f70 0x7f5f440733e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009990 tx=0x7f5f30009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.712+0000 7f5f4afff700 1 -- 172.21.15.62:0/1879593761 shutdown_connections 2023-07-14T16:22:33.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.712+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1879593761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4410c0c0 0x7f5f4410e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.712+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1879593761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f44074550 0x7f5f440729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.712+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1879593761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44072f70 0x7f5f440733e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.712+0000 7f5f4afff700 1 -- 172.21.15.62:0/1879593761 >> 172.21.15.62:0/1879593761 conn(0x7f5f440fb130 msgr2=0x7f5f440fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.712+0000 7f5f4afff700 1 -- 172.21.15.62:0/1879593761 shutdown_connections 2023-07-14T16:22:33.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.712+0000 7f5f4afff700 1 -- 172.21.15.62:0/1879593761 wait complete. 2023-07-14T16:22:33.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.713+0000 7f5f4afff700 1 Processor -- start 2023-07-14T16:22:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.713+0000 7f5f4afff700 1 -- start start 2023-07-14T16:22:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.713+0000 7f5f4afff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f44072f70 0x7f5f4419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.713+0000 7f5f4afff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44074550 0x7f5f4419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.713+0000 7f5f4afff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4410c0c0 0x7f5f441a3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.713+0000 7f5f4afff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441a38d0 con 0x7f5f44074550 2023-07-14T16:22:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.713+0000 7f5f4afff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441a3a10 con 0x7f5f44072f70 2023-07-14T16:22:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.713+0000 7f5f4afff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441a3ba0 con 0x7f5f4410c0c0 2023-07-14T16:22:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f43fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44074550 0x7f5f4419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f48d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f44072f70 0x7f5f4419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f43fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44074550 0x7f5f4419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53062/0 (socket says 172.21.15.62:53062) 2023-07-14T16:22:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f43fff700 1 -- 172.21.15.62:0/1002365776 learned_addr learned my addr 172.21.15.62:0/1002365776 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f4959c700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4410c0c0 0x7f5f441a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f43fff700 1 -- 172.21.15.62:0/1002365776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4410c0c0 msgr2=0x7f5f441a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f43fff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4410c0c0 0x7f5f441a3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f43fff700 1 -- 172.21.15.62:0/1002365776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f44072f70 msgr2=0x7f5f4419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f43fff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f44072f70 0x7f5f4419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f43fff700 1 -- 172.21.15.62:0/1002365776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f300095f0 con 0x7f5f44074550 2023-07-14T16:22:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.714+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f44072f70 0x7f5f4419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.715+0000 7f5f43fff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44074550 0x7f5f4419c290 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5f300098e0 tx=0x7f5f30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.715+0000 7f5f41ffb700 1 -- 172.21.15.62:0/1002365776 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3001d070 con 0x7f5f44074550 2023-07-14T16:22:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.715+0000 7f5f41ffb700 1 -- 172.21.15.62:0/1002365776 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f3000be30 con 0x7f5f44074550 2023-07-14T16:22:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.715+0000 7f5f4afff700 1 -- 172.21.15.62:0/1002365776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f441a3e20 con 0x7f5f44074550 2023-07-14T16:22:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.715+0000 7f5f41ffb700 1 -- 172.21.15.62:0/1002365776 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30022dd0 con 0x7f5f44074550 2023-07-14T16:22:33.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.715+0000 7f5f4afff700 1 -- 172.21.15.62:0/1002365776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f441a4190 con 0x7f5f44074550 2023-07-14T16:22:33.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.720+0000 7f5f41ffb700 1 -- 172.21.15.62:0/1002365776 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5f30022570 con 0x7f5f44074550 2023-07-14T16:22:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.721+0000 7f5f5e969b80 1 Processor -- start 2023-07-14T16:22:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.721+0000 7f5f5e969b80 1 -- start start 2023-07-14T16:22:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c1e20 0x55715b4c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b3c6cb0 con 0x55715b4c14d0 2023-07-14T16:22:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f41ffb700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5f340a1150 0x7f5f340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b419a40 con 0x55715b4c2c70 2023-07-14T16:22:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b412ed0 con 0x55715b4c1e20 2023-07-14T16:22:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f41ffb700 1 -- 172.21.15.62:0/1002365776 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f300cdf00 con 0x7f5f44074550 2023-07-14T16:22:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f4959c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f43fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f43fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40520/0 (socket says 172.21.15.62:40520) 2023-07-14T16:22:33.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f43fff700 1 -- 172.21.15.62:0/935222719 learned_addr learned my addr 172.21.15.62:0/935222719 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f4959c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4c18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53072/0 (socket says 172.21.15.62:53072) 2023-07-14T16:22:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/935222719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c1e20 0x55715b4c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.722+0000 7f5f4959c700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5f340a1150 0x7f5f340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.723+0000 7f5f4959c700 1 -- 172.21.15.62:0/935222719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c1e20 msgr2=0x55715b4c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.723+0000 7f5f4959c700 1 --2- 172.21.15.62:0/935222719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c1e20 0x55715b4c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.723+0000 7f5f4959c700 1 -- 172.21.15.62:0/935222719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 msgr2=0x55715b4c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.723+0000 7f5f4959c700 1 --2- 172.21.15.62:0/935222719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.723+0000 7f5f4959c700 1 -- 172.21.15.62:0/935222719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55715b240de0 con 0x55715b4c14d0 2023-07-14T16:22:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.723+0000 7f5f4959c700 1 --2- 172.21.15.62:0/935222719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4c18e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c013e40 tx=0x7f5f3c017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f9ae4cea5fda87 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.723+0000 7f5f4959c700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5f340a1150 0x7f5f340a3600 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0179a0 tx=0x7f5f3c00a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.724+0000 7f5f2effd700 1 -- 172.21.15.62:0/935222719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c013920 con 0x55715b4c14d0 2023-07-14T16:22:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.724+0000 7f5f2effd700 1 -- 172.21.15.62:0/935222719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f3c00d040 con 0x55715b4c14d0 2023-07-14T16:22:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.724+0000 7f5f2effd700 1 -- 172.21.15.62:0/935222719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c021650 con 0x55715b4c14d0 2023-07-14T16:22:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.724+0000 7f5f5e969b80 1 -- 172.21.15.62:0/935222719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 msgr2=0x55715b4c18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.724+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/935222719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4c18e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c013e40 tx=0x7f5f3c017650 comp rx=0 tx=0).stop 2023-07-14T16:22:33.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.725+0000 7f5f5e969b80 1 -- 172.21.15.62:0/935222719 shutdown_connections 2023-07-14T16:22:33.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.725+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/935222719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.725+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/935222719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4c18e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.725+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/935222719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c1e20 0x55715b4c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.725+0000 7f5f5e969b80 1 -- 172.21.15.62:0/935222719 >> 172.21.15.62:0/935222719 conn(0x55715b4b6da0 msgr2=0x55715b4bc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.725+0000 7f5f5e969b80 1 -- 172.21.15.62:0/935222719 shutdown_connections 2023-07-14T16:22:33.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.725+0000 7f5f5e969b80 1 -- 172.21.15.62:0/935222719 wait complete. 2023-07-14T16:22:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.725+0000 7f5f5e969b80 1 -- start start 2023-07-14T16:22:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f5e969b80 1 Processor -- start 2023-07-14T16:22:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 0x55715b4cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b3b2ac0 con 0x55715b4c14d0 2023-07-14T16:22:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b3b27c0 con 0x55715b4c1e20 2023-07-14T16:22:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b296920 con 0x55715b4c2c70 2023-07-14T16:22:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f5e969b80 10 client.0 _ll_register_callbacks cb 0x55715b3b0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f48d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f48d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53078/0 (socket says 172.21.15.62:53078) 2023-07-14T16:22:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.726+0000 7f5f48d9b700 1 -- 172.21.15.62:0/3393089007 learned_addr learned my addr 172.21.15.62:0/3393089007 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f4959c700 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f43fff700 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 0x55715b4cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f48d9b700 1 -- 172.21.15.62:0/3393089007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 msgr2=0x55715b4d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f48d9b700 1 -- 172.21.15.62:0/3393089007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 msgr2=0x55715b4cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 0x55715b4cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f48d9b700 1 -- 172.21.15.62:0/3393089007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c00f040 con 0x55715b4c14d0 2023-07-14T16:22:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4cb100 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5f38011d10 tx=0x7f5f3800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624aaf34a1402fbe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.727+0000 7f5f23fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.728+0000 7f5f23fff700 1 -- 172.21.15.62:0/3393089007 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38016070 con 0x55715b4c14d0 2023-07-14T16:22:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.728+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3393089007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55715b36c4f0 con 0x55715b4c14d0 2023-07-14T16:22:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.728+0000 7f5f23fff700 1 -- 172.21.15.62:0/3393089007 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f38004050 con 0x55715b4c14d0 2023-07-14T16:22:33.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.728+0000 7f5f23fff700 1 -- 172.21.15.62:0/3393089007 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f3800fe50 con 0x55715b4c14d0 2023-07-14T16:22:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.728+0000 7f5f5e969b80 10 client.17472 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.728+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3393089007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55715b229c60 con 0x55715b4c14d0 2023-07-14T16:22:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.728+0000 7f5f23fff700 1 -- 172.21.15.62:0/3393089007 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55715b36c4f0 con 0x55715b4c14d0 2023-07-14T16:22:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.728+0000 7f5f23fff700 1 -- 172.21.15.62:0/3393089007 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5f3801a410 con 0x55715b4c14d0 2023-07-14T16:22:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.729+0000 7f5f5e969b80 10 client.17472 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.729+0000 7f5f5e969b80 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:33.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.729+0000 7f5f5e969b80 20 client.17472 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:33.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.729+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715b4d6eb0 0x55715b4d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.729+0000 7f5f5e969b80 4 client.17472 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:33.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.729+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3393089007 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55715b40beb0 con 0x55715b4d6eb0 2023-07-14T16:22:33.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.729+0000 7f5f43fff700 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715b4d6eb0 0x55715b4d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.730+0000 7f5f43fff700 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715b4d6eb0 0x55715b4d92c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ef9ca79c6faa02 server_cookie=603e346826ce0907 in_seq=0 out_seq=0 2023-07-14T16:22:33.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.730+0000 7f5f23fff700 10 client.17472 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:33.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.730+0000 7f5f23fff700 1 -- 172.21.15.62:0/3393089007 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55715b40beb0 con 0x55715b4d6eb0 2023-07-14T16:22:33.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.730+0000 7f5f23fff700 10 client.17472 handle_command_reply: tid=0 2023-07-14T16:22:33.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.730+0000 7f5f5e969b80 1 client.17472 shutdown 2023-07-14T16:22:33.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3393089007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 msgr2=0x55715b4cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4cb100 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5f38011d10 tx=0x7f5f3800f6a0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3393089007 shutdown_connections 2023-07-14T16:22:33.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4cb100 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715b4d6eb0 0x55715b4d92c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3393089007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 0x55715b4cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3393089007 >> 172.21.15.62:0/3393089007 conn(0x55715b4b6da0 msgr2=0x55715b4bb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f23fff700 0 client.17472 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:33.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3393089007 shutdown_connections 2023-07-14T16:22:33.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3393089007 wait complete. 2023-07-14T16:22:33.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.731+0000 7f5f5e969b80 20 client.17472 trim_cache size 0 max 16384 2023-07-14T16:22:33.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.753+0000 7f5f5e969b80 1 Processor -- start 2023-07-14T16:22:33.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f5e969b80 1 -- start start 2023-07-14T16:22:33.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c14d0 0x55715b4d8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c1e20 0x55715b4d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b4d43b0 con 0x55715b4c1e20 2023-07-14T16:22:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b4c2340 con 0x55715b4c2c70 2023-07-14T16:22:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b4d6c00 con 0x55715b4c14d0 2023-07-14T16:22:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f43fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c1e20 0x55715b4d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f4959c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f43fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c1e20 0x55715b4d2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53082/0 (socket says 172.21.15.62:53082) 2023-07-14T16:22:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f43fff700 1 -- 172.21.15.62:0/2799022652 learned_addr learned my addr 172.21.15.62:0/2799022652 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.754+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c14d0 0x55715b4d8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f43fff700 1 -- 172.21.15.62:0/2799022652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c14d0 msgr2=0x55715b4d8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f43fff700 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c14d0 0x55715b4d8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f43fff700 1 -- 172.21.15.62:0/2799022652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 msgr2=0x55715b4c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f43fff700 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f43fff700 1 -- 172.21.15.62:0/2799022652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55715b267250 con 0x55715b4c1e20 2023-07-14T16:22:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f4959c700 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f43fff700 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c1e20 0x55715b4d2640 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000c00 tx=0x7f5f30097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3296998bb053a51 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f2f7fe700 1 -- 172.21.15.62:0/2799022652 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300a3710 con 0x55715b4c1e20 2023-07-14T16:22:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f2f7fe700 1 -- 172.21.15.62:0/2799022652 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f300996f0 con 0x55715b4c1e20 2023-07-14T16:22:33.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.755+0000 7f5f2f7fe700 1 -- 172.21.15.62:0/2799022652 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300a3550 con 0x55715b4c1e20 2023-07-14T16:22:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 -- 172.21.15.62:0/2799022652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c1e20 msgr2=0x55715b4d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c1e20 0x55715b4d2640 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000c00 tx=0x7f5f30097040 comp rx=0 tx=0).stop 2023-07-14T16:22:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 -- 172.21.15.62:0/2799022652 shutdown_connections 2023-07-14T16:22:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c2c70 0x55715b4c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c14d0 0x55715b4d8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/2799022652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c1e20 0x55715b4d2640 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 -- 172.21.15.62:0/2799022652 >> 172.21.15.62:0/2799022652 conn(0x55715b4b6da0 msgr2=0x55715b4d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 -- 172.21.15.62:0/2799022652 shutdown_connections 2023-07-14T16:22:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.756+0000 7f5f5e969b80 1 -- 172.21.15.62:0/2799022652 wait complete. 2023-07-14T16:22:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.757+0000 7f5f5e969b80 1 -- start start 2023-07-14T16:22:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.757+0000 7f5f5e969b80 1 Processor -- start 2023-07-14T16:22:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.757+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.757+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 0x55715b4d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.757+0000 7f5f5e969b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.757+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b4d0d50 con 0x55715b4c14d0 2023-07-14T16:22:33.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b4d0ec0 con 0x55715b4c1e20 2023-07-14T16:22:33.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f5e969b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b4d1080 con 0x55715b4c2c70 2023-07-14T16:22:33.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f5e969b80 10 client.0 _ll_register_callbacks cb 0x7f5f380117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:33.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f48d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f48d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53086/0 (socket says 172.21.15.62:53086) 2023-07-14T16:22:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f48d9b700 1 -- 172.21.15.62:0/3186861067 learned_addr learned my addr 172.21.15.62:0/3186861067 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f4959c700 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f43fff700 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 0x55715b4d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f48d9b700 1 -- 172.21.15.62:0/3186861067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 msgr2=0x55715b4d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f48d9b700 1 -- 172.21.15.62:0/3186861067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 msgr2=0x55715b4d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 0x55715b4d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f48d9b700 1 -- 172.21.15.62:0/3186861067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55715b240de0 con 0x55715b4c14d0 2023-07-14T16:22:33.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.758+0000 7f5f4959c700 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:33.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f48d9b700 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4d2060 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5f38005a80 tx=0x7f5f3800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb0e02c57da6221 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:33.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f23fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:33.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f23fff700 1 -- 172.21.15.62:0/3186861067 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38016040 con 0x55715b4c14d0 2023-07-14T16:22:33.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3186861067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f3c00d040 con 0x55715b4c14d0 2023-07-14T16:22:33.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f23fff700 1 -- 172.21.15.62:0/3186861067 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f38000e40 con 0x55715b4c14d0 2023-07-14T16:22:33.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f23fff700 1 -- 172.21.15.62:0/3186861067 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f38005250 con 0x55715b4c14d0 2023-07-14T16:22:33.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f5e969b80 10 client.17478 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:33.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3186861067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f5f300cbe70 con 0x55715b4c14d0 2023-07-14T16:22:33.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.759+0000 7f5f23fff700 1 -- 172.21.15.62:0/3186861067 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f3c00d040 con 0x55715b4c14d0 2023-07-14T16:22:33.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.760+0000 7f5f23fff700 1 -- 172.21.15.62:0/3186861067 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5f3802c030 con 0x55715b4c14d0 2023-07-14T16:22:33.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.760+0000 7f5f5e969b80 10 client.17478 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:33.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.760+0000 7f5f5e969b80 10 client.17478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:33.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.760+0000 7f5f5e969b80 20 client.17478 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:33.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.760+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715b52c100 0x55715b52e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:33.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.760+0000 7f5f5e969b80 4 client.17478 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:22:33.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.760+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3186861067 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55715b3ce050 con 0x55715b52c100 2023-07-14T16:22:33.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.760+0000 7f5f43fff700 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715b52c100 0x55715b52e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:33.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f23fff700 10 client.17478 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:33.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f43fff700 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715b52c100 0x55715b52e510 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f090c0791c5956df server_cookie=5cb7ad35cabc1950 in_seq=0 out_seq=0 2023-07-14T16:22:33.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f23fff700 1 -- 172.21.15.62:0/3186861067 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55715b3ce050 con 0x55715b52c100 2023-07-14T16:22:33.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f23fff700 10 client.17478 handle_command_reply: tid=0 2023-07-14T16:22:33.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 client.17478 shutdown 2023-07-14T16:22:33.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3186861067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 msgr2=0x55715b4d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4d2060 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5f38005a80 tx=0x7f5f3800fa80 comp rx=0 tx=0).stop 2023-07-14T16:22:33.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3186861067 shutdown_connections 2023-07-14T16:22:33.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715b4c2c70 0x55715b4d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715b4c14d0 0x55715b4d2060 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715b52c100 0x55715b52e510 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 --2- 172.21.15.62:0/3186861067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715b4c1e20 0x55715b4d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3186861067 >> 172.21.15.62:0/3186861067 conn(0x55715b4b6da0 msgr2=0x55715b4d8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f23fff700 0 client.17478 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:33.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.761+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3186861067 shutdown_connections 2023-07-14T16:22:33.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f5e969b80 1 -- 172.21.15.62:0/3186861067 wait complete. 2023-07-14T16:22:33.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f5e969b80 20 client.17478 trim_cache size 0 max 16384 2023-07-14T16:22:33.748 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:22:33.748 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:22:33.748 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:22:33.748 INFO:teuthology.orchestra.run.smithi062.stdout: "018bc4b4-1a0c-412c-8a2f-a0860ec1bbdf": { 2023-07-14T16:22:33.748 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:22:33.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "018bc4b4-1a0c-412c-8a2f-a0860ec1bbdf", 2023-07-14T16:22:33.749 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:22:33.749 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:22:33.749 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:22:33.749 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:22:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 -- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5f340a1150 msgr2=0x7f5f340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5f340a1150 0x7f5f340a3600 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0179a0 tx=0x7f5f3c00a000 comp rx=0 tx=0).stop 2023-07-14T16:22:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 -- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44074550 msgr2=0x7f5f4419c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44074550 0x7f5f4419c290 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5f300098e0 tx=0x7f5f30004730 comp rx=0 tx=0).stop 2023-07-14T16:22:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 -- 172.21.15.62:0/1002365776 shutdown_connections 2023-07-14T16:22:33.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4410c0c0 0x7f5f441a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5f340a1150 0x7f5f340a3600 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f44072f70 0x7f5f4419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 --2- 172.21.15.62:0/1002365776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44074550 0x7f5f4419c290 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:33.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.762+0000 7f5f4afff700 1 -- 172.21.15.62:0/1002365776 >> 172.21.15.62:0/1002365776 conn(0x7f5f440fb130 msgr2=0x7f5f440fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:33.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.763+0000 7f5f4afff700 1 -- 172.21.15.62:0/1002365776 shutdown_connections 2023-07-14T16:22:33.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:22:33.763+0000 7f5f4afff700 1 -- 172.21.15.62:0/1002365776 wait complete. 2023-07-14T16:22:33.758 INFO:tasks.cephfs.filesystem:scrub status for tag:018bc4b4-1a0c-412c-8a2f-a0860ec1bbdf - {'path': '/', 'tag': '018bc4b4-1a0c-412c-8a2f-a0860ec1bbdf', 'options': 'recursive,force'} 2023-07-14T16:22:33.813 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:22:33 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:22:33.449+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v157>,<0x10000000003/client.0 v158>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v170>,<0x10000000001/sv_0 v184>,<0x10000000000/_nogroup v178>,<0x1/volumes v168>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:22:34.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:33 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:22:34.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:33 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:34.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:33 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:22:34.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:33 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:34.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:33 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:22:34.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:33 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:34.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:33 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:22:34.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:33 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:34.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:33 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:22:34.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:33 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:34.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:33 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:22:34.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:33 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:34 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:22:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:34 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:34 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:22:35.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:34 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:22:35.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:34 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:35.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:34 smithi062 ceph-mon[95543]: pgmap v565: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 20 op/s 2023-07-14T16:22:35.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:34 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:22:35.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:34 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:34 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:22:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:34 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:22:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:34 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:35.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:34 smithi081 ceph-mon[127795]: pgmap v565: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 20 op/s 2023-07-14T16:22:35.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:34 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:22:35.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:34 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:35.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:34 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:22:35.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:34 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:22:35.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:34 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:22:35.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:34 smithi164 ceph-mon[97427]: pgmap v565: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 20 op/s 2023-07-14T16:22:36.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:35 smithi062 ceph-mon[95543]: pgmap v566: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 23 op/s 2023-07-14T16:22:36.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:35 smithi081 ceph-mon[127795]: pgmap v566: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 23 op/s 2023-07-14T16:22:36.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:35 smithi164 ceph-mon[97427]: pgmap v566: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 23 op/s 2023-07-14T16:22:38.902 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:38 smithi081 ceph-mon[127795]: pgmap v567: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:22:39.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:38 smithi164 ceph-mon[97427]: pgmap v567: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:22:39.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:38 smithi062 ceph-mon[95543]: pgmap v567: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:22:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 -- 172.21.15.62:0/2224275130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 msgr2=0x7fb508100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 --2- 172.21.15.62:0/2224275130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 0x7fb508100f80 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009990 tx=0x7fb4fc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 -- 172.21.15.62:0/2224275130 shutdown_connections 2023-07-14T16:22:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 --2- 172.21.15.62:0/2224275130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5081014c0 0x7fb508107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 --2- 172.21.15.62:0/2224275130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5080ff940 0x7fb5080ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 --2- 172.21.15.62:0/2224275130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 0x7fb508100f80 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 -- 172.21.15.62:0/2224275130 >> 172.21.15.62:0/2224275130 conn(0x7fb5080faf60 msgr2=0x7fb5080fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 -- 172.21.15.62:0/2224275130 shutdown_connections 2023-07-14T16:22:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.753+0000 7fb50759e700 1 -- 172.21.15.62:0/2224275130 wait complete. 2023-07-14T16:22:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.754+0000 7fb50759e700 1 Processor -- start 2023-07-14T16:22:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.754+0000 7fb50759e700 1 -- start start 2023-07-14T16:22:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.754+0000 7fb50759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5080ff940 0x7fb5080742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.754+0000 7fb50759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 0x7fb508072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb50759e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5081014c0 0x7fb508072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb50759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080735e0 con 0x7fb508100b30 2023-07-14T16:22:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb50759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb508073750 con 0x7fb5080ff940 2023-07-14T16:22:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb50759e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50819f620 con 0x7fb5081014c0 2023-07-14T16:22:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb505d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 0x7fb508072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb505d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 0x7fb508072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53108/0 (socket says 172.21.15.62:53108) 2023-07-14T16:22:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb505d9b700 1 -- 172.21.15.62:0/3557074407 learned_addr learned my addr 172.21.15.62:0/3557074407 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb50659c700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5080ff940 0x7fb5080742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb506d9d700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5081014c0 0x7fb508072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb505d9b700 1 -- 172.21.15.62:0/3557074407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5081014c0 msgr2=0x7fb508072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5081014c0 0x7fb508072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb505d9b700 1 -- 172.21.15.62:0/3557074407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5080ff940 msgr2=0x7fb5080742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5080ff940 0x7fb5080742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.755+0000 7fb505d9b700 1 -- 172.21.15.62:0/3557074407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0095f0 con 0x7fb508100b30 2023-07-14T16:22:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.756+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 0x7fb508072900 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0098e0 tx=0x7fb4fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.756+0000 7fb4f77fe700 1 -- 172.21.15.62:0/3557074407 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc01d070 con 0x7fb508100b30 2023-07-14T16:22:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.756+0000 7fb50759e700 1 -- 172.21.15.62:0/3557074407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50819f8a0 con 0x7fb508100b30 2023-07-14T16:22:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.756+0000 7fb4f77fe700 1 -- 172.21.15.62:0/3557074407 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4fc00bdd0 con 0x7fb508100b30 2023-07-14T16:22:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.756+0000 7fb4f77fe700 1 -- 172.21.15.62:0/3557074407 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc022d50 con 0x7fb508100b30 2023-07-14T16:22:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.756+0000 7fb50759e700 1 -- 172.21.15.62:0/3557074407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50819fcc0 con 0x7fb508100b30 2023-07-14T16:22:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.760+0000 7fb4f77fe700 1 -- 172.21.15.62:0/3557074407 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb4fc00b940 con 0x7fb508100b30 2023-07-14T16:22:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb52035ab80 1 Processor -- start 2023-07-14T16:22:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb52035ab80 1 -- start start 2023-07-14T16:22:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb4f77fe700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4f00a1150 0x7fb4f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfe0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfe12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfe1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb52035ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bee5c50 con 0x561c5be3c6b0 2023-07-14T16:22:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb52035ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bf389e0 con 0x561c5bfe0a90 2023-07-14T16:22:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.762+0000 7fb52035ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bf31e70 con 0x561c5bfe17e0 2023-07-14T16:22:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb4f77fe700 1 -- 172.21.15.62:0/3557074407 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb4fc0cdff0 con 0x7fb508100b30 2023-07-14T16:22:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb506d9d700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4f00a1150 0x7fb4f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb506d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfe1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb50659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfe0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb505d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfe12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb506d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfe1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45002/0 (socket says 172.21.15.62:45002) 2023-07-14T16:22:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb506d9d700 1 -- 172.21.15.62:0/3742335015 learned_addr learned my addr 172.21.15.62:0/3742335015 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb505d9b700 1 -- 172.21.15.62:0/3742335015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 msgr2=0x561c5bfe1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3742335015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfe1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb505d9b700 1 -- 172.21.15.62:0/3742335015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 msgr2=0x561c5bfe0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3742335015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfe0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.763+0000 7fb505d9b700 1 -- 172.21.15.62:0/3742335015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c5bd5fd80 con 0x561c5bfe0a90 2023-07-14T16:22:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.764+0000 7fb506d9d700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4f00a1150 0x7fb4f00a3600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb500013e00 tx=0x7fb500013480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.764+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3742335015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfe12a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc09b140 tx=0x7fb4fc005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf96a7d01992bb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.764+0000 7fb4f4ff9700 1 -- 172.21.15.62:0/3742335015 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc09ada0 con 0x561c5bfe0a90 2023-07-14T16:22:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.764+0000 7fb4f4ff9700 1 -- 172.21.15.62:0/3742335015 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4fc022470 con 0x561c5bfe0a90 2023-07-14T16:22:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.764+0000 7fb4f4ff9700 1 -- 172.21.15.62:0/3742335015 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0cbbb0 con 0x561c5bfe0a90 2023-07-14T16:22:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 -- 172.21.15.62:0/3742335015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 msgr2=0x561c5bfe12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3742335015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfe12a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc09b140 tx=0x7fb4fc005a00 comp rx=0 tx=0).stop 2023-07-14T16:22:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 -- 172.21.15.62:0/3742335015 shutdown_connections 2023-07-14T16:22:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3742335015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfe1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3742335015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfe0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3742335015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfe12a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 -- 172.21.15.62:0/3742335015 >> 172.21.15.62:0/3742335015 conn(0x561c5bfd5d40 msgr2=0x561c5bfdae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 -- 172.21.15.62:0/3742335015 shutdown_connections 2023-07-14T16:22:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.765+0000 7fb52035ab80 1 -- 172.21.15.62:0/3742335015 wait complete. 2023-07-14T16:22:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 1 -- start start 2023-07-14T16:22:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 1 Processor -- start 2023-07-14T16:22:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5be3c6b0 0x561c5bfe96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bfe9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bff0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bed1a60 con 0x561c5bfe0a90 2023-07-14T16:22:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bed1760 con 0x561c5be3c6b0 2023-07-14T16:22:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bdb58c0 con 0x561c5bfe17e0 2023-07-14T16:22:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.766+0000 7fb52035ab80 10 client.0 _ll_register_callbacks cb 0x561c5becf700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5be3c6b0 0x561c5bfe96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb505d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bfe9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5be3c6b0 0x561c5bfe96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40586/0 (socket says 172.21.15.62:40586) 2023-07-14T16:22:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 -- 172.21.15.62:0/1082360838 learned_addr learned my addr 172.21.15.62:0/1082360838 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb506d9d700 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bff0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 -- 172.21.15.62:0/1082360838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 msgr2=0x561c5bff0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bff0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 -- 172.21.15.62:0/1082360838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 msgr2=0x561c5bfe9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bfe9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 -- 172.21.15.62:0/1082360838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50819fcc0 con 0x561c5be3c6b0 2023-07-14T16:22:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb505d9b700 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bfe9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb50659c700 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5be3c6b0 0x561c5bfe96a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800bee0 tx=0x7fb4f8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f8ad7db0abb2b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.767+0000 7fb4edffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:22:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.768+0000 7fb4edffb700 1 -- 172.21.15.62:0/1082360838 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8016070 con 0x561c5be3c6b0 2023-07-14T16:22:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.768+0000 7fb52035ab80 1 -- 172.21.15.62:0/1082360838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c5be8b490 con 0x561c5be3c6b0 2023-07-14T16:22:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.768+0000 7fb4edffb700 1 -- 172.21.15.62:0/1082360838 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb4f8004020 con 0x561c5be3c6b0 2023-07-14T16:22:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.768+0000 7fb4edffb700 1 -- 172.21.15.62:0/1082360838 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f80041e0 con 0x561c5be3c6b0 2023-07-14T16:22:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.768+0000 7fb52035ab80 10 client.27167 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.768+0000 7fb52035ab80 1 -- 172.21.15.62:0/1082360838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561c5bd48c00 con 0x561c5be3c6b0 2023-07-14T16:22:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.768+0000 7fb4edffb700 1 -- 172.21.15.62:0/1082360838 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561c5be8b490 con 0x561c5be3c6b0 2023-07-14T16:22:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.769+0000 7fb4edffb700 1 -- 172.21.15.62:0/1082360838 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb4f8003680 con 0x561c5be3c6b0 2023-07-14T16:22:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.769+0000 7fb52035ab80 10 client.27167 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.769+0000 7fb52035ab80 10 client.27167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.769+0000 7fb52035ab80 20 client.27167 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.769+0000 7fb52035ab80 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561c5bff5450 0x561c5bff7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.769+0000 7fb52035ab80 4 client.27167 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.769+0000 7fb52035ab80 1 -- 172.21.15.62:0/1082360838 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c5bf2ae50 con 0x561c5bff5450 2023-07-14T16:22:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.769+0000 7fb505d9b700 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561c5bff5450 0x561c5bff7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.770+0000 7fb4edffb700 10 client.27167 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.770+0000 7fb505d9b700 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561c5bff5450 0x561c5bff7860 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ee98a9f919e7f9 server_cookie=9b3695ae4c61fab2 in_seq=0 out_seq=0 2023-07-14T16:22:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb4edffb700 1 -- 172.21.15.62:0/1082360838 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c5bf2ae50 con 0x561c5bff5450 2023-07-14T16:22:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb4edffb700 10 client.27167 handle_command_reply: tid=0 2023-07-14T16:22:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 client.27167 shutdown 2023-07-14T16:22:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 -- 172.21.15.62:0/1082360838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5be3c6b0 msgr2=0x561c5bfe96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5be3c6b0 0x561c5bfe96a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800bee0 tx=0x7fb4f8011720 comp rx=0 tx=0).stop 2023-07-14T16:22:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 -- 172.21.15.62:0/1082360838 shutdown_connections 2023-07-14T16:22:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bff0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561c5bff5450 0x561c5bff7860 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5be3c6b0 0x561c5bfe96a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 --2- 172.21.15.62:0/1082360838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bfe9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb4edffb700 0 client.27167 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 -- 172.21.15.62:0/1082360838 >> 172.21.15.62:0/1082360838 conn(0x561c5bfd5d40 msgr2=0x561c5bfda640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 -- 172.21.15.62:0/1082360838 shutdown_connections 2023-07-14T16:22:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.771+0000 7fb52035ab80 1 -- 172.21.15.62:0/1082360838 wait complete. 2023-07-14T16:22:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.772+0000 7fb52035ab80 20 client.27167 trim_cache size 0 max 16384 2023-07-14T16:22:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.791+0000 7fb52035ab80 1 Processor -- start 2023-07-14T16:22:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.791+0000 7fb52035ab80 1 -- start start 2023-07-14T16:22:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.791+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5be3c6b0 0x561c5bfe8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.791+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bff0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.791+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe17e0 0x561c5bfd9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.791+0000 7fb52035ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bf2ae50 con 0x561c5bfe0a90 2023-07-14T16:22:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.791+0000 7fb52035ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5be8b490 con 0x561c5bfe17e0 2023-07-14T16:22:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.791+0000 7fb52035ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bff2910 con 0x561c5be3c6b0 2023-07-14T16:22:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bff0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bff0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53126/0 (socket says 172.21.15.62:53126) 2023-07-14T16:22:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 -- 172.21.15.62:0/2894329085 learned_addr learned my addr 172.21.15.62:0/2894329085 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb506d9d700 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe17e0 0x561c5bfd9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb50659c700 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5be3c6b0 0x561c5bfe8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 -- 172.21.15.62:0/2894329085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5be3c6b0 msgr2=0x561c5bfe8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5be3c6b0 0x561c5bfe8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 -- 172.21.15.62:0/2894329085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe17e0 msgr2=0x561c5bfd9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe17e0 0x561c5bfd9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 -- 172.21.15.62:0/2894329085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c5bd861f0 con 0x561c5bfe0a90 2023-07-14T16:22:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb50659c700 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5be3c6b0 0x561c5bfe8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb505d9b700 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bff0be0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc097940 tx=0x7fb4fc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8d7130a1cf2a1b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.792+0000 7fb4f57fa700 1 -- 172.21.15.62:0/2894329085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc09f470 con 0x561c5bfe0a90 2023-07-14T16:22:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb4f57fa700 1 -- 172.21.15.62:0/2894329085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4fc0a3800 con 0x561c5bfe0a90 2023-07-14T16:22:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb4f57fa700 1 -- 172.21.15.62:0/2894329085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc099870 con 0x561c5bfe0a90 2023-07-14T16:22:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 -- 172.21.15.62:0/2894329085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 msgr2=0x561c5bff0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bff0be0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc097940 tx=0x7fb4fc009310 comp rx=0 tx=0).stop 2023-07-14T16:22:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 -- 172.21.15.62:0/2894329085 shutdown_connections 2023-07-14T16:22:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe17e0 0x561c5bfd9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5be3c6b0 0x561c5bfe8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 --2- 172.21.15.62:0/2894329085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5bfe0a90 0x561c5bff0be0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 -- 172.21.15.62:0/2894329085 >> 172.21.15.62:0/2894329085 conn(0x561c5bfd5d40 msgr2=0x561c5bfe7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 -- 172.21.15.62:0/2894329085 shutdown_connections 2023-07-14T16:22:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.793+0000 7fb52035ab80 1 -- 172.21.15.62:0/2894329085 wait complete. 2023-07-14T16:22:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.794+0000 7fb52035ab80 1 -- start start 2023-07-14T16:22:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.794+0000 7fb52035ab80 1 Processor -- start 2023-07-14T16:22:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfeca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb52035ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfefaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb52035ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bfeffe0 con 0x561c5be3c6b0 2023-07-14T16:22:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb52035ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bff0150 con 0x561c5bfe0a90 2023-07-14T16:22:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb52035ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5bff0310 con 0x561c5bfe17e0 2023-07-14T16:22:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb52035ab80 10 client.0 _ll_register_callbacks cb 0x7fb4f800b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb50659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb50659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfec540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53142/0 (socket says 172.21.15.62:53142) 2023-07-14T16:22:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb50659c700 1 -- 172.21.15.62:0/3790390542 learned_addr learned my addr 172.21.15.62:0/3790390542 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfeca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.795+0000 7fb506d9d700 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfefaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb50659c700 1 -- 172.21.15.62:0/3790390542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 msgr2=0x561c5bfefaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb50659c700 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfefaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb50659c700 1 -- 172.21.15.62:0/3790390542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 msgr2=0x561c5bfeca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb50659c700 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfeca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb50659c700 1 -- 172.21.15.62:0/3790390542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c5bd5fd80 con 0x561c5be3c6b0 2023-07-14T16:22:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb50659c700 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfec540 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8005e10 tx=0x7fb4f800b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302aebaa6301137e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb4edffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb4edffb700 1 -- 172.21.15.62:0/3790390542 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8016030 con 0x561c5be3c6b0 2023-07-14T16:22:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb4edffb700 1 -- 172.21.15.62:0/3790390542 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb4f8020020 con 0x561c5be3c6b0 2023-07-14T16:22:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.796+0000 7fb52035ab80 1 -- 172.21.15.62:0/3790390542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4fc022470 con 0x561c5be3c6b0 2023-07-14T16:22:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb4edffb700 1 -- 172.21.15.62:0/3790390542 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f80050c0 con 0x561c5be3c6b0 2023-07-14T16:22:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb52035ab80 10 client.17496 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb52035ab80 1 -- 172.21.15.62:0/3790390542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561c5bd5e250 con 0x561c5be3c6b0 2023-07-14T16:22:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb4edffb700 1 -- 172.21.15.62:0/3790390542 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4fc022470 con 0x561c5be3c6b0 2023-07-14T16:22:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb4edffb700 1 -- 172.21.15.62:0/3790390542 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb4f802c070 con 0x561c5be3c6b0 2023-07-14T16:22:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb52035ab80 10 client.17496 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb52035ab80 10 client.17496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb52035ab80 20 client.17496 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561c5c049e30 0x561c5c04c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb52035ab80 4 client.17496 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:22:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.797+0000 7fb52035ab80 1 -- 172.21.15.62:0/3790390542 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561c5beecff0 con 0x561c5c049e30 2023-07-14T16:22:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.798+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561c5c049e30 0x561c5c04c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.798+0000 7fb505d9b700 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561c5c049e30 0x561c5c04c240 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2faabeb22724b6b server_cookie=755e25c79a8dc2cd in_seq=0 out_seq=0 2023-07-14T16:22:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.798+0000 7fb4edffb700 10 client.17496 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.808+0000 7fb4edffb700 1 -- 172.21.15.62:0/3790390542 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561c5beecff0 con 0x561c5c049e30 2023-07-14T16:22:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.808+0000 7fb4edffb700 10 client.17496 handle_command_reply: tid=0 2023-07-14T16:22:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.808+0000 7fb52035ab80 1 client.17496 shutdown 2023-07-14T16:22:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.808+0000 7fb52035ab80 1 -- 172.21.15.62:0/3790390542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 msgr2=0x561c5bfec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.808+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfec540 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8005e10 tx=0x7fb4f800b500 comp rx=0 tx=0).stop 2023-07-14T16:22:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.808+0000 7fb52035ab80 1 -- 172.21.15.62:0/3790390542 shutdown_connections 2023-07-14T16:22:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.808+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561c5bfe17e0 0x561c5bfefaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.808+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561c5be3c6b0 0x561c5bfec540 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.809+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561c5c049e30 0x561c5c04c240 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.809+0000 7fb52035ab80 1 --2- 172.21.15.62:0/3790390542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561c5bfe0a90 0x561c5bfeca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.809+0000 7fb52035ab80 1 -- 172.21.15.62:0/3790390542 >> 172.21.15.62:0/3790390542 conn(0x561c5bfd5d40 msgr2=0x561c5bfe9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.809+0000 7fb4edffb700 0 client.17496 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.809+0000 7fb52035ab80 1 -- 172.21.15.62:0/3790390542 shutdown_connections 2023-07-14T16:22:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.809+0000 7fb52035ab80 1 -- 172.21.15.62:0/3790390542 wait complete. 2023-07-14T16:22:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.809+0000 7fb52035ab80 20 client.17496 trim_cache size 0 max 16384 2023-07-14T16:22:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:22:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:22:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:22:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:22:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 -- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4f00a1150 msgr2=0x7fb4f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4f00a1150 0x7fb4f00a3600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb500013e00 tx=0x7fb500013480 comp rx=0 tx=0).stop 2023-07-14T16:22:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 -- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 msgr2=0x7fb508072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 0x7fb508072900 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0098e0 tx=0x7fb4fc004730 comp rx=0 tx=0).stop 2023-07-14T16:22:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 -- 172.21.15.62:0/3557074407 shutdown_connections 2023-07-14T16:22:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5081014c0 0x7fb508072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4f00a1150 0x7fb4f00a3600 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5080ff940 0x7fb5080742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 --2- 172.21.15.62:0/3557074407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb508100b30 0x7fb508072900 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 -- 172.21.15.62:0/3557074407 >> 172.21.15.62:0/3557074407 conn(0x7fb5080faf60 msgr2=0x7fb5080fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 -- 172.21.15.62:0/3557074407 shutdown_connections 2023-07-14T16:22:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:39.810+0000 7fb50759e700 1 -- 172.21.15.62:0/3557074407 wait complete. 2023-07-14T16:22:41.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:40 smithi164 ceph-mon[97427]: pgmap v568: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:22:41.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:40 smithi062 ceph-mon[95543]: pgmap v568: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:22:41.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:40 smithi081 ceph-mon[127795]: pgmap v568: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:22:43.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:42 smithi164 ceph-mon[97427]: pgmap v569: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:22:43.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:42 smithi062 ceph-mon[95543]: pgmap v569: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:22:43.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:42 smithi081 ceph-mon[127795]: pgmap v569: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:22:44.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:43 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:22:44.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:43 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:22:44.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:43 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:22:44.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:43 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:22:44.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:43 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:22:44.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:43 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:22:45.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:44 smithi164 ceph-mon[97427]: pgmap v570: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:22:45.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:44 smithi062 ceph-mon[95543]: pgmap v570: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:22:45.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:44 smithi081 ceph-mon[127795]: pgmap v570: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 37 op/s 2023-07-14T16:22:47.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:46 smithi164 ceph-mon[97427]: pgmap v571: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 31 op/s 2023-07-14T16:22:47.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:46 smithi062 ceph-mon[95543]: pgmap v571: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 31 op/s 2023-07-14T16:22:47.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:46 smithi081 ceph-mon[127795]: pgmap v571: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 31 op/s 2023-07-14T16:22:49.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:48 smithi164 ceph-mon[97427]: pgmap v572: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:22:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:48 smithi062 ceph-mon[95543]: pgmap v572: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:22:49.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:48 smithi081 ceph-mon[127795]: pgmap v572: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:22:51.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:50 smithi164 ceph-mon[97427]: pgmap v573: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:22:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:50 smithi062 ceph-mon[95543]: pgmap v573: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:22:51.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:50 smithi081 ceph-mon[127795]: pgmap v573: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:22:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.973+0000 7f194abff700 1 -- 172.21.15.62:0/2319965831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 msgr2=0x7f1944107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.973+0000 7f194abff700 1 --2- 172.21.15.62:0/2319965831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 0x7f1944107c40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f193c009ec0 tx=0x7f193c00b330 comp rx=0 tx=0).stop 2023-07-14T16:22:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.973+0000 7f194abff700 1 -- 172.21.15.62:0/2319965831 shutdown_connections 2023-07-14T16:22:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.973+0000 7f194abff700 1 --2- 172.21.15.62:0/2319965831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 0x7f1944107c40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.973+0000 7f194abff700 1 --2- 172.21.15.62:0/2319965831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1944100620 0x7f1944102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.973+0000 7f194abff700 1 --2- 172.21.15.62:0/2319965831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1944102f40 0x7f1944105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.973+0000 7f194abff700 1 -- 172.21.15.62:0/2319965831 >> 172.21.15.62:0/2319965831 conn(0x7f19440fa000 msgr2=0x7f19440fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.973+0000 7f194abff700 1 -- 172.21.15.62:0/2319965831 shutdown_connections 2023-07-14T16:22:52.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.974+0000 7f194abff700 1 -- 172.21.15.62:0/2319965831 wait complete. 2023-07-14T16:22:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.974+0000 7f194abff700 1 Processor -- start 2023-07-14T16:22:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194abff700 1 -- start start 2023-07-14T16:22:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194abff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1944100620 0x7f19441959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194abff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1944102f40 0x7f1944195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194abff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 0x7f1944196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194abff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1944197410 con 0x7f1944105860 2023-07-14T16:22:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194abff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1944197580 con 0x7f1944100620 2023-07-14T16:22:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194abff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1944197740 con 0x7f1944102f40 2023-07-14T16:22:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194899b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1944100620 0x7f19441959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194899b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1944100620 0x7f19441959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:55638/0 (socket says 172.21.15.62:55638) 2023-07-14T16:22:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194919c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 0x7f1944196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.975+0000 7f194899b700 1 -- 172.21.15.62:0/4133875421 learned_addr learned my addr 172.21.15.62:0/4133875421 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.976+0000 7f1943fff700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1944102f40 0x7f1944195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.976+0000 7f194919c700 1 -- 172.21.15.62:0/4133875421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1944102f40 msgr2=0x7f1944195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.976+0000 7f194919c700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1944102f40 0x7f1944195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.976+0000 7f194919c700 1 -- 172.21.15.62:0/4133875421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1944100620 msgr2=0x7f19441959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.976+0000 7f194919c700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1944100620 0x7f19441959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.976+0000 7f194919c700 1 -- 172.21.15.62:0/4133875421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c005e70 con 0x7f1944105860 2023-07-14T16:22:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.976+0000 7f194899b700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1944100620 0x7f19441959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.976+0000 7f194919c700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 0x7f1944196db0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f193c015000 tx=0x7f193c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.977+0000 7f1941ffb700 1 -- 172.21.15.62:0/4133875421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00e040 con 0x7f1944105860 2023-07-14T16:22:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.977+0000 7f194abff700 1 -- 172.21.15.62:0/4133875421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19441979c0 con 0x7f1944105860 2023-07-14T16:22:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.977+0000 7f1941ffb700 1 -- 172.21.15.62:0/4133875421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f193c01d910 con 0x7f1944105860 2023-07-14T16:22:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.977+0000 7f1941ffb700 1 -- 172.21.15.62:0/4133875421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c01b960 con 0x7f1944105860 2023-07-14T16:22:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.977+0000 7f194abff700 1 -- 172.21.15.62:0/4133875421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194419e1e0 con 0x7f1944105860 2023-07-14T16:22:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.980+0000 7f1941ffb700 1 -- 172.21.15.62:0/4133875421 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f193c01da80 con 0x7f1944105860 2023-07-14T16:22:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.982+0000 7f195e4beb80 1 Processor -- start 2023-07-14T16:22:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.982+0000 7f195e4beb80 1 -- start start 2023-07-14T16:22:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.982+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8514776b0 0x55a85161b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.982+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85161c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.982+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161c6e0 0x55a85161cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.982+0000 7f195e4beb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a851520c50 con 0x55a85161c6e0 2023-07-14T16:22:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.982+0000 7f195e4beb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8515739e0 con 0x55a85161b840 2023-07-14T16:22:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.982+0000 7f195e4beb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85156ce70 con 0x55a8514776b0 2023-07-14T16:22:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1941ffb700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19340a1150 0x7f19340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161c6e0 0x55a85161cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161c6e0 0x55a85161cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57420/0 (socket says 172.21.15.62:57420) 2023-07-14T16:22:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 -- 172.21.15.62:0/3573874911 learned_addr learned my addr 172.21.15.62:0/3573874911 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1941ffb700 1 -- 172.21.15.62:0/4133875421 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f193c0ce4d0 con 0x7f1944105860 2023-07-14T16:22:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8514776b0 0x55a85161b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f194919c700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19340a1150 0x7f19340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f194899b700 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85161c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 -- 172.21.15.62:0/3573874911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8514776b0 msgr2=0x55a85161b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8514776b0 0x55a85161b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 -- 172.21.15.62:0/3573874911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 msgr2=0x55a85161c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85161c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.983+0000 7f1943fff700 1 -- 172.21.15.62:0/3573874911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a85139ad80 con 0x55a85161c6e0 2023-07-14T16:22:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.984+0000 7f1943fff700 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161c6e0 0x55a85161cb80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f19300118b0 tx=0x7f1930009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713e09da7afb6510 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.984+0000 7f194919c700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19340a1150 0x7f19340a3600 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f193c020040 tx=0x7f193c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.984+0000 7f192effd700 1 -- 172.21.15.62:0/3573874911 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930004030 con 0x55a85161c6e0 2023-07-14T16:22:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.984+0000 7f192effd700 1 -- 172.21.15.62:0/3573874911 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1930004830 con 0x55a85161c6e0 2023-07-14T16:22:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.984+0000 7f192effd700 1 -- 172.21.15.62:0/3573874911 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930005280 con 0x55a85161c6e0 2023-07-14T16:22:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 -- 172.21.15.62:0/3573874911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161c6e0 msgr2=0x55a85161cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161c6e0 0x55a85161cb80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f19300118b0 tx=0x7f1930009880 comp rx=0 tx=0).stop 2023-07-14T16:22:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 -- 172.21.15.62:0/3573874911 shutdown_connections 2023-07-14T16:22:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161c6e0 0x55a85161cb80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8514776b0 0x55a85161b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 --2- 172.21.15.62:0/3573874911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85161c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 -- 172.21.15.62:0/3573874911 >> 172.21.15.62:0/3573874911 conn(0x55a851610be0 msgr2=0x55a851615cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 -- 172.21.15.62:0/3573874911 shutdown_connections 2023-07-14T16:22:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 -- 172.21.15.62:0/3573874911 wait complete. 2023-07-14T16:22:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.985+0000 7f195e4beb80 1 -- start start 2023-07-14T16:22:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f195e4beb80 1 Processor -- start 2023-07-14T16:22:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851624890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851624dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a85162be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f195e4beb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85150ca60 con 0x55a85161b840 2023-07-14T16:22:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f195e4beb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85150c760 con 0x55a8514776b0 2023-07-14T16:22:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f195e4beb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8513f08c0 con 0x55a85161c6e0 2023-07-14T16:22:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f195e4beb80 10 client.0 _ll_register_callbacks cb 0x55a85150a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.986+0000 7f194899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851624dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f194899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851624dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57436/0 (socket says 172.21.15.62:57436) 2023-07-14T16:22:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f194899b700 1 -- 172.21.15.62:0/1215600561 learned_addr learned my addr 172.21.15.62:0/1215600561 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f1943fff700 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851624890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f1943fff700 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a85162be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f194899b700 1 -- 172.21.15.62:0/1215600561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 msgr2=0x55a85162be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f194899b700 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a85162be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f194899b700 1 -- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 msgr2=0x55a851624890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f194899b700 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851624890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f194899b700 1 -- 172.21.15.62:0/1215600561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1930019040 con 0x55a85161b840 2023-07-14T16:22:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f1943fff700 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851624890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:22:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f194899b700 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851624dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1938012d20 tx=0x7f193800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a448f870d81283 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.987+0000 7f1923fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.988+0000 7f1923fff700 1 -- 172.21.15.62:0/1215600561 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938017070 con 0x55a85161b840 2023-07-14T16:22:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.988+0000 7f1923fff700 1 -- 172.21.15.62:0/1215600561 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f193800fc30 con 0x55a85161b840 2023-07-14T16:22:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.988+0000 7f195e4beb80 1 -- 172.21.15.62:0/1215600561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8514c6490 con 0x55a85161b840 2023-07-14T16:22:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.988+0000 7f1923fff700 1 -- 172.21.15.62:0/1215600561 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1938022030 con 0x55a85161b840 2023-07-14T16:22:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.988+0000 7f195e4beb80 10 client.17508 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.988+0000 7f195e4beb80 1 -- 172.21.15.62:0/1215600561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a851383c00 con 0x55a85161b840 2023-07-14T16:22:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.988+0000 7f1923fff700 1 -- 172.21.15.62:0/1215600561 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a8514c6490 con 0x55a85161b840 2023-07-14T16:22:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.988+0000 7f1923fff700 1 -- 172.21.15.62:0/1215600561 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1938010430 con 0x55a85161b840 2023-07-14T16:22:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.989+0000 7f195e4beb80 10 client.17508 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.989+0000 7f195e4beb80 10 client.17508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.989+0000 7f195e4beb80 20 client.17508 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.989+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a8516306a0 0x55a851632ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.989+0000 7f195e4beb80 4 client.17508 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:22:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.989+0000 7f195e4beb80 1 -- 172.21.15.62:0/1215600561 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a851565e50 con 0x55a8516306a0 2023-07-14T16:22:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.989+0000 7f1943fff700 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a8516306a0 0x55a851632ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.990+0000 7f1943fff700 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a8516306a0 0x55a851632ab0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec2df57bc9f090e server_cookie=5a46c7a6d0a18e79 in_seq=0 out_seq=0 2023-07-14T16:22:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.990+0000 7f1923fff700 10 client.17508 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.990+0000 7f1923fff700 1 -- 172.21.15.62:0/1215600561 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a851565e50 con 0x55a8516306a0 2023-07-14T16:22:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.990+0000 7f1923fff700 10 client.17508 handle_command_reply: tid=0 2023-07-14T16:22:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.990+0000 7f195e4beb80 1 client.17508 shutdown 2023-07-14T16:22:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 -- 172.21.15.62:0/1215600561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 msgr2=0x55a851624dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851624dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1938012d20 tx=0x7f193800f6a0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 -- 172.21.15.62:0/1215600561 shutdown_connections 2023-07-14T16:22:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a85162be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a8516306a0 0x55a851632ab0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851624890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1215600561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851624dd0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f1923fff700 0 client.17508 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 -- 172.21.15.62:0/1215600561 >> 172.21.15.62:0/1215600561 conn(0x55a851610be0 msgr2=0x55a8516154e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 -- 172.21.15.62:0/1215600561 shutdown_connections 2023-07-14T16:22:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 1 -- 172.21.15.62:0/1215600561 wait complete. 2023-07-14T16:22:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:52.991+0000 7f195e4beb80 20 client.17508 trim_cache size 0 max 16384 2023-07-14T16:22:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.006+0000 7f195e4beb80 1 Processor -- start 2023-07-14T16:22:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f195e4beb80 1 -- start start 2023-07-14T16:22:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8514776b0 0x55a851624640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85162bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a851624b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f195e4beb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85162db00 con 0x55a8514776b0 2023-07-14T16:22:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f195e4beb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a851623d40 con 0x55a85161b840 2023-07-14T16:22:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f195e4beb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a851617140 con 0x55a85161c6e0 2023-07-14T16:22:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f1943fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8514776b0 0x55a851624640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f1943fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8514776b0 0x55a851624640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57450/0 (socket says 172.21.15.62:57450) 2023-07-14T16:22:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f1943fff700 1 -- 172.21.15.62:0/3863887490 learned_addr learned my addr 172.21.15.62:0/3863887490 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.007+0000 7f194899b700 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85162bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.008+0000 7f1943fff700 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a851624b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.008+0000 7f194899b700 1 -- 172.21.15.62:0/3863887490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 msgr2=0x55a851624b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.008+0000 7f194899b700 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a851624b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.008+0000 7f194899b700 1 -- 172.21.15.62:0/3863887490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8514776b0 msgr2=0x55a851624640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.008+0000 7f194899b700 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8514776b0 0x55a851624640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.008+0000 7f194899b700 1 -- 172.21.15.62:0/3863887490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8513c11f0 con 0x55a85161b840 2023-07-14T16:22:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.008+0000 7f1943fff700 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8514776b0 0x55a851624640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:22:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.008+0000 7f194899b700 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85162bdd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1938012cf0 tx=0x7f1938000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae69310c3eba6f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.009+0000 7f192f7fe700 1 -- 172.21.15.62:0/3863887490 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938017040 con 0x55a85161b840 2023-07-14T16:22:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.009+0000 7f192f7fe700 1 -- 172.21.15.62:0/3863887490 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19380092d0 con 0x55a85161b840 2023-07-14T16:22:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.009+0000 7f192f7fe700 1 -- 172.21.15.62:0/3863887490 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938022040 con 0x55a85161b840 2023-07-14T16:22:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.009+0000 7f195e4beb80 1 -- 172.21.15.62:0/3863887490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 msgr2=0x55a85162bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.009+0000 7f195e4beb80 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85162bdd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1938012cf0 tx=0x7f1938000bc0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 -- 172.21.15.62:0/3863887490 shutdown_connections 2023-07-14T16:22:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a851624b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a8514776b0 0x55a851624640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 --2- 172.21.15.62:0/3863887490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a85161b840 0x55a85162bdd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 -- 172.21.15.62:0/3863887490 >> 172.21.15.62:0/3863887490 conn(0x55a851610be0 msgr2=0x55a85161f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 -- 172.21.15.62:0/3863887490 shutdown_connections 2023-07-14T16:22:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 -- 172.21.15.62:0/3863887490 wait complete. 2023-07-14T16:22:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 -- start start 2023-07-14T16:22:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.010+0000 7f195e4beb80 1 Processor -- start 2023-07-14T16:22:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851627af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851628030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f195e4beb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a85162b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f195e4beb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85162b590 con 0x55a85161b840 2023-07-14T16:22:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f195e4beb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85162b700 con 0x55a8514776b0 2023-07-14T16:22:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f195e4beb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85162b8c0 con 0x55a85161c6e0 2023-07-14T16:22:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f195e4beb80 10 client.0 _ll_register_callbacks cb 0x7f19380127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:22:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f194899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851628030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f194899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851628030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57460/0 (socket says 172.21.15.62:57460) 2023-07-14T16:22:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f194899b700 1 -- 172.21.15.62:0/1111475206 learned_addr learned my addr 172.21.15.62:0/1111475206 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:22:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.011+0000 7f1943fff700 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851627af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f1943fff700 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a85162b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f194899b700 1 -- 172.21.15.62:0/1111475206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 msgr2=0x55a85162b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f194899b700 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a85162b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f194899b700 1 -- 172.21.15.62:0/1111475206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 msgr2=0x55a851627af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:22:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f194899b700 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851627af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f194899b700 1 -- 172.21.15.62:0/1111475206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1930036040 con 0x55a85161b840 2023-07-14T16:22:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f194899b700 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851628030 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1938012a40 tx=0x7f1938012010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d5ce4b6c620444 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:22:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f1923fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:22:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.012+0000 7f1923fff700 1 -- 172.21.15.62:0/1111475206 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938017070 con 0x55a85161b840 2023-07-14T16:22:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 1 -- 172.21.15.62:0/1111475206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1930004830 con 0x55a85161b840 2023-07-14T16:22:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f1923fff700 1 -- 172.21.15.62:0/1111475206 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1938023070 con 0x55a85161b840 2023-07-14T16:22:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f1923fff700 1 -- 172.21.15.62:0/1111475206 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1938022040 con 0x55a85161b840 2023-07-14T16:22:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 10 client.17514 fetch_fsmap learned FSMap version 25 2023-07-14T16:22:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 1 -- 172.21.15.62:0/1111475206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a851383c00 con 0x55a85161b840 2023-07-14T16:22:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f1923fff700 1 -- 172.21.15.62:0/1111475206 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1930004830 con 0x55a85161b840 2023-07-14T16:22:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f1923fff700 1 -- 172.21.15.62:0/1111475206 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f193802d030 con 0x55a85161b840 2023-07-14T16:22:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 10 client.17514 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:22:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 10 client.17514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:22:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 20 client.17514 populate_metadata read hostname 'smithi062' 2023-07-14T16:22:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a8516850d0 0x55a8516874e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:22:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 4 client.17514 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:22:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.013+0000 7f195e4beb80 1 -- 172.21.15.62:0/1111475206 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a851527ff0 con 0x55a8516850d0 2023-07-14T16:22:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.014+0000 7f1943fff700 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a8516850d0 0x55a8516874e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:22:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.014+0000 7f1943fff700 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a8516850d0 0x55a8516874e0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb34c458bdbd6c02 server_cookie=322a6b18b5113556 in_seq=0 out_seq=0 2023-07-14T16:22:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.014+0000 7f1923fff700 10 client.17514 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f1923fff700 1 -- 172.21.15.62:0/1111475206 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a851527ff0 con 0x55a8516850d0 2023-07-14T16:22:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f1923fff700 10 client.17514 handle_command_reply: tid=0 2023-07-14T16:22:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 client.17514 shutdown 2023-07-14T16:22:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 -- 172.21.15.62:0/1111475206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 msgr2=0x55a851628030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851628030 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1938012a40 tx=0x7f1938012010 comp rx=0 tx=0).stop 2023-07-14T16:22:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 -- 172.21.15.62:0/1111475206 shutdown_connections 2023-07-14T16:22:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a85161c6e0 0x55a85162b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a8516850d0 0x55a8516874e0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a8514776b0 0x55a851627af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 --2- 172.21.15.62:0/1111475206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a85161b840 0x55a851628030 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 -- 172.21.15.62:0/1111475206 >> 172.21.15.62:0/1111475206 conn(0x55a851610be0 msgr2=0x55a851624640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f1923fff700 0 client.17514 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:22:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 -- 172.21.15.62:0/1111475206 shutdown_connections 2023-07-14T16:22:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 1 -- 172.21.15.62:0/1111475206 wait complete. 2023-07-14T16:22:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.024+0000 7f195e4beb80 20 client.17514 trim_cache size 0 max 16384 2023-07-14T16:22:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:22:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:22:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:22:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:22:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 -- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19340a1150 msgr2=0x7f19340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19340a1150 0x7f19340a3600 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f193c020040 tx=0x7f193c009040 comp rx=0 tx=0).stop 2023-07-14T16:22:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 -- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 msgr2=0x7f1944196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:22:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 0x7f1944196db0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f193c015000 tx=0x7f193c009580 comp rx=0 tx=0).stop 2023-07-14T16:22:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 -- 172.21.15.62:0/4133875421 shutdown_connections 2023-07-14T16:22:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1944105860 0x7f1944196db0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19340a1150 0x7f19340a3600 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1944100620 0x7f19441959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 --2- 172.21.15.62:0/4133875421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1944102f40 0x7f1944195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:22:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 -- 172.21.15.62:0/4133875421 >> 172.21.15.62:0/4133875421 conn(0x7f19440fa000 msgr2=0x7f1944101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:22:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 -- 172.21.15.62:0/4133875421 shutdown_connections 2023-07-14T16:22:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:22:53.025+0000 7f194abff700 1 -- 172.21.15.62:0/4133875421 wait complete. 2023-07-14T16:22:53.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:52 smithi164 ceph-mon[97427]: pgmap v574: 97 pgs: 97 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:22:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:52 smithi062 ceph-mon[95543]: pgmap v574: 97 pgs: 97 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:22:53.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:52 smithi081 ceph-mon[127795]: pgmap v574: 97 pgs: 97 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:22:55.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:54 smithi164 ceph-mon[97427]: pgmap v575: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:22:55.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:54 smithi062 ceph-mon[95543]: pgmap v575: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:22:55.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:54 smithi081 ceph-mon[127795]: pgmap v575: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:22:57.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:56 smithi164 ceph-mon[97427]: pgmap v576: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:22:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:56 smithi062 ceph-mon[95543]: pgmap v576: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:22:57.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:56 smithi081 ceph-mon[127795]: pgmap v576: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:22:59.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:22:58 smithi164 ceph-mon[97427]: pgmap v577: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:22:59.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:22:58 smithi062 ceph-mon[95543]: pgmap v577: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:22:59.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:22:58 smithi081 ceph-mon[127795]: pgmap v577: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:23:01.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:00 smithi164 ceph-mon[97427]: pgmap v578: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:01.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:00 smithi062 ceph-mon[95543]: pgmap v578: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:01.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:00 smithi081 ceph-mon[127795]: pgmap v578: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:03.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:02 smithi164 ceph-mon[97427]: pgmap v579: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:23:03.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:02 smithi062 ceph-mon[95543]: pgmap v579: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:23:03.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:02 smithi081 ceph-mon[127795]: pgmap v579: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:23:03.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:23:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.022+0000 7ff931fff700 1 -- 172.21.15.62:0/106852380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c100560 msgr2=0x7ff92c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.022+0000 7ff931fff700 1 --2- 172.21.15.62:0/106852380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c100560 0x7ff92c102940 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009990 tx=0x7ff91c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.023+0000 7ff931fff700 1 -- 172.21.15.62:0/106852380 shutdown_connections 2023-07-14T16:23:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.023+0000 7ff931fff700 1 --2- 172.21.15.62:0/106852380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff92c1057a0 0x7ff92c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.023+0000 7ff931fff700 1 --2- 172.21.15.62:0/106852380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c100560 0x7ff92c102940 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.023+0000 7ff931fff700 1 --2- 172.21.15.62:0/106852380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c102e80 0x7ff92c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.023+0000 7ff931fff700 1 -- 172.21.15.62:0/106852380 >> 172.21.15.62:0/106852380 conn(0x7ff92c0fa000 msgr2=0x7ff92c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.023+0000 7ff931fff700 1 -- 172.21.15.62:0/106852380 shutdown_connections 2023-07-14T16:23:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.023+0000 7ff931fff700 1 -- 172.21.15.62:0/106852380 wait complete. 2023-07-14T16:23:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.024+0000 7ff931fff700 1 Processor -- start 2023-07-14T16:23:04.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.024+0000 7ff931fff700 1 -- start start 2023-07-14T16:23:04.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.024+0000 7ff931fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff92c100560 0x7ff92c0ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.024+0000 7ff931fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c102e80 0x7ff92c0fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.024+0000 7ff931fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c1057a0 0x7ff92c0fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.024+0000 7ff931fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0ff1f0 con 0x7ff92c102e80 2023-07-14T16:23:04.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.024+0000 7ff931fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0ff360 con 0x7ff92c100560 2023-07-14T16:23:04.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.024+0000 7ff931fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c19f7d0 con 0x7ff92c1057a0 2023-07-14T16:23:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c102e80 0x7ff92c0fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff92c100560 0x7ff92c0ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c102e80 0x7ff92c0fe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34472/0 (socket says 172.21.15.62:34472) 2023-07-14T16:23:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92affd700 1 -- 172.21.15.62:0/2564819053 learned_addr learned my addr 172.21.15.62:0/2564819053 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92bfff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c1057a0 0x7ff92c0fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92affd700 1 -- 172.21.15.62:0/2564819053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c1057a0 msgr2=0x7ff92c0fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92affd700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c1057a0 0x7ff92c0fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92affd700 1 -- 172.21.15.62:0/2564819053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff92c100560 msgr2=0x7ff92c0ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92affd700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff92c100560 0x7ff92c0ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.025+0000 7ff92affd700 1 -- 172.21.15.62:0/2564819053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0095f0 con 0x7ff92c102e80 2023-07-14T16:23:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.026+0000 7ff92affd700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c102e80 0x7ff92c0fe510 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff92000d950 tx=0x7ff92000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.026+0000 7ff928ff9700 1 -- 172.21.15.62:0/2564819053 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000f890 con 0x7ff92c102e80 2023-07-14T16:23:04.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.026+0000 7ff928ff9700 1 -- 172.21.15.62:0/2564819053 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff92000fe80 con 0x7ff92c102e80 2023-07-14T16:23:04.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.026+0000 7ff931fff700 1 -- 172.21.15.62:0/2564819053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c19fab0 con 0x7ff92c102e80 2023-07-14T16:23:04.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.026+0000 7ff928ff9700 1 -- 172.21.15.62:0/2564819053 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000e3b0 con 0x7ff92c102e80 2023-07-14T16:23:04.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.026+0000 7ff931fff700 1 -- 172.21.15.62:0/2564819053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c19ffc0 con 0x7ff92c102e80 2023-07-14T16:23:04.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.030+0000 7ff928ff9700 1 -- 172.21.15.62:0/2564819053 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff920010460 con 0x7ff92c102e80 2023-07-14T16:23:04.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.031+0000 7ff94594eb80 1 Processor -- start 2023-07-14T16:23:04.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.031+0000 7ff94594eb80 1 -- start start 2023-07-14T16:23:04.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff928ff9700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9140a1150 0x7ff9140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe54d0 0x55d58ffe58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 0x55d58ffe66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff928ff9700 1 -- 172.21.15.62:0/2564819053 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff9200cc700 con 0x7ff92c102e80 2023-07-14T16:23:04.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe6c70 0x55d58ffe70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff94594eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58feeacb0 con 0x55d58ffe6c70 2023-07-14T16:23:04.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff94594eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58ff3da40 con 0x55d58ffe54d0 2023-07-14T16:23:04.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff94594eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58ff36ed0 con 0x55d58ffe5e20 2023-07-14T16:23:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff92bfff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9140a1150 0x7ff9140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe6c70 0x55d58ffe70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe6c70 0x55d58ffe70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34474/0 (socket says 172.21.15.62:34474) 2023-07-14T16:23:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff92bfff700 1 -- 172.21.15.62:0/2216606064 learned_addr learned my addr 172.21.15.62:0/2216606064 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff92affd700 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 0x55d58ffe66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.032+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe54d0 0x55d58ffe58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff92bfff700 1 -- 172.21.15.62:0/2216606064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 msgr2=0x55d58ffe66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff92bfff700 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 0x55d58ffe66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff92bfff700 1 -- 172.21.15.62:0/2216606064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe54d0 msgr2=0x55d58ffe58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff92bfff700 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe54d0 0x55d58ffe58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff92bfff700 1 -- 172.21.15.62:0/2216606064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d58fd64de0 con 0x55d58ffe6c70 2023-07-14T16:23:04.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe54d0 0x55d58ffe58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:04.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff92bfff700 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe6c70 0x55d58ffe70e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff924007ea0 tx=0x7ff924019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb14634202f0117 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff92bfff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9140a1150 0x7ff9140a3600 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff9240199a0 tx=0x7ff92400f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff911ffb700 1 -- 172.21.15.62:0/2216606064 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924007590 con 0x55d58ffe6c70 2023-07-14T16:23:04.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff911ffb700 1 -- 172.21.15.62:0/2216606064 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff92400f070 con 0x55d58ffe6c70 2023-07-14T16:23:04.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.033+0000 7ff911ffb700 1 -- 172.21.15.62:0/2216606064 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924003c60 con 0x55d58ffe6c70 2023-07-14T16:23:04.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 -- 172.21.15.62:0/2216606064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe6c70 msgr2=0x55d58ffe70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe6c70 0x55d58ffe70e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff924007ea0 tx=0x7ff924019650 comp rx=0 tx=0).stop 2023-07-14T16:23:04.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 -- 172.21.15.62:0/2216606064 shutdown_connections 2023-07-14T16:23:04.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe6c70 0x55d58ffe70e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe54d0 0x55d58ffe58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 --2- 172.21.15.62:0/2216606064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 0x55d58ffe66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 -- 172.21.15.62:0/2216606064 >> 172.21.15.62:0/2216606064 conn(0x55d58ffdada0 msgr2=0x55d58ffe01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 -- 172.21.15.62:0/2216606064 shutdown_connections 2023-07-14T16:23:04.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.034+0000 7ff94594eb80 1 -- 172.21.15.62:0/2216606064 wait complete. 2023-07-14T16:23:04.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.035+0000 7ff94594eb80 1 -- start start 2023-07-14T16:23:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.035+0000 7ff94594eb80 1 Processor -- start 2023-07-14T16:23:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58ffef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 0x55d58fff66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff94594eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58fed6ac0 con 0x55d58ffe54d0 2023-07-14T16:23:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff94594eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58fed67c0 con 0x55d58ffe5e20 2023-07-14T16:23:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff94594eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58fdba920 con 0x55d58ffe6c70 2023-07-14T16:23:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff94594eb80 10 client.0 _ll_register_callbacks cb 0x55d58fed4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58ffef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffef160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34482/0 (socket says 172.21.15.62:34482) 2023-07-14T16:23:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92b7fe700 1 -- 172.21.15.62:0/1935674064 learned_addr learned my addr 172.21.15.62:0/1935674064 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92bfff700 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 0x55d58fff66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92affd700 1 -- 172.21.15.62:0/1935674064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 msgr2=0x55d58fff66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92affd700 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 0x55d58fff66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92affd700 1 -- 172.21.15.62:0/1935674064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 msgr2=0x55d58ffef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.036+0000 7ff92affd700 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff92affd700 1 -- 172.21.15.62:0/1935674064 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff924007a80 con 0x55d58ffe5e20 2023-07-14T16:23:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff92affd700 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58ffef6a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9200997c0 tx=0x7ff920099b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80343f9d4baa096e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff90affd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:23:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff90affd700 1 -- 172.21.15.62:0/1935674064 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200991b0 con 0x55d58ffe5e20 2023-07-14T16:23:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff90affd700 1 -- 172.21.15.62:0/1935674064 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff9200a1a80 con 0x55d58ffe5e20 2023-07-14T16:23:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff94594eb80 1 -- 172.21.15.62:0/1935674064 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d58fe904f0 con 0x55d58ffe5e20 2023-07-14T16:23:04.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff90affd700 1 -- 172.21.15.62:0/1935674064 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9200b2200 con 0x55d58ffe5e20 2023-07-14T16:23:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff94594eb80 10 client.27203 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.037+0000 7ff94594eb80 1 -- 172.21.15.62:0/1935674064 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d58fd4dc60 con 0x55d58ffe5e20 2023-07-14T16:23:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.038+0000 7ff90affd700 1 -- 172.21.15.62:0/1935674064 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d58fe904f0 con 0x55d58ffe5e20 2023-07-14T16:23:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.038+0000 7ff90affd700 1 -- 172.21.15.62:0/1935674064 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff920095ab0 con 0x55d58ffe5e20 2023-07-14T16:23:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.038+0000 7ff94594eb80 10 client.27203 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.038+0000 7ff94594eb80 10 client.27203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:04.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.038+0000 7ff94594eb80 20 client.27203 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.038+0000 7ff94594eb80 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d58fffae10 0x55d58fffd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.038+0000 7ff94594eb80 4 client.27203 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:04.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.038+0000 7ff94594eb80 1 -- 172.21.15.62:0/1935674064 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d58ff2feb0 con 0x55d58fffae10 2023-07-14T16:23:04.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.039+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d58fffae10 0x55d58fffd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.039+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d58fffae10 0x55d58fffd220 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f88ab93cd570c6 server_cookie=6cb5227e1d042a2c in_seq=0 out_seq=0 2023-07-14T16:23:04.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.039+0000 7ff90affd700 10 client.27203 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:04.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.040+0000 7ff90affd700 1 -- 172.21.15.62:0/1935674064 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d58ff2feb0 con 0x55d58fffae10 2023-07-14T16:23:04.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.040+0000 7ff90affd700 10 client.27203 handle_command_reply: tid=0 2023-07-14T16:23:04.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.040+0000 7ff94594eb80 1 client.27203 shutdown 2023-07-14T16:23:04.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.040+0000 7ff94594eb80 1 -- 172.21.15.62:0/1935674064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 msgr2=0x55d58ffef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.040+0000 7ff94594eb80 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58ffef6a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9200997c0 tx=0x7ff920099b70 comp rx=0 tx=0).stop 2023-07-14T16:23:04.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.040+0000 7ff94594eb80 1 -- 172.21.15.62:0/1935674064 shutdown_connections 2023-07-14T16:23:04.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.040+0000 7ff94594eb80 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 0x55d58fff66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.040+0000 7ff94594eb80 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.041+0000 7ff94594eb80 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d58fffae10 0x55d58fffd220 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.041+0000 7ff94594eb80 1 --2- 172.21.15.62:0/1935674064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58ffef6a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.041+0000 7ff94594eb80 1 -- 172.21.15.62:0/1935674064 >> 172.21.15.62:0/1935674064 conn(0x55d58ffdada0 msgr2=0x55d58ffdf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.041+0000 7ff90affd700 0 client.27203 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:04.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.041+0000 7ff94594eb80 1 -- 172.21.15.62:0/1935674064 shutdown_connections 2023-07-14T16:23:04.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.041+0000 7ff94594eb80 1 -- 172.21.15.62:0/1935674064 wait complete. 2023-07-14T16:23:04.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.041+0000 7ff94594eb80 20 client.27203 trim_cache size 0 max 16384 2023-07-14T16:23:04.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.062+0000 7ff94594eb80 1 Processor -- start 2023-07-14T16:23:04.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.062+0000 7ff94594eb80 1 -- start start 2023-07-14T16:23:04.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.062+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffed100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.062+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 0x55d58fff66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.062+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe6c70 0x55d58ffe9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.062+0000 7ff94594eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58ffdcec0 con 0x55d58ffe54d0 2023-07-14T16:23:04.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.062+0000 7ff94594eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58ffee610 con 0x55d58ffe6c70 2023-07-14T16:23:04.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.062+0000 7ff94594eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58ffeecd0 con 0x55d58ffe5e20 2023-07-14T16:23:04.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffed100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34496/0 (socket says 172.21.15.62:34496) 2023-07-14T16:23:04.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 -- 172.21.15.62:0/3278076682 learned_addr learned my addr 172.21.15.62:0/3278076682 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92affd700 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 0x55d58fff66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92bfff700 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe6c70 0x55d58ffe9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 -- 172.21.15.62:0/3278076682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 msgr2=0x55d58fff66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 0x55d58fff66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 -- 172.21.15.62:0/3278076682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe6c70 msgr2=0x55d58ffe9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe6c70 0x55d58ffe9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 -- 172.21.15.62:0/3278076682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d58fd8b250 con 0x55d58ffe54d0 2023-07-14T16:23:04.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92bfff700 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe6c70 0x55d58ffe9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:04.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.063+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffed100 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x55d58fffdb30 tx=0x7ff91c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b47a0ceaa1900 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.064+0000 7ff9127fc700 1 -- 172.21.15.62:0/3278076682 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c01d040 con 0x55d58ffe54d0 2023-07-14T16:23:04.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.064+0000 7ff9127fc700 1 -- 172.21.15.62:0/3278076682 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff91c01e040 con 0x55d58ffe54d0 2023-07-14T16:23:04.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.064+0000 7ff9127fc700 1 -- 172.21.15.62:0/3278076682 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c005040 con 0x55d58ffe54d0 2023-07-14T16:23:04.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 -- 172.21.15.62:0/3278076682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 msgr2=0x55d58ffed100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffed100 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x55d58fffdb30 tx=0x7ff91c005fb0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 -- 172.21.15.62:0/3278076682 shutdown_connections 2023-07-14T16:23:04.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe6c70 0x55d58ffe9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58ffed100 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 --2- 172.21.15.62:0/3278076682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe5e20 0x55d58fff66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 -- 172.21.15.62:0/3278076682 >> 172.21.15.62:0/3278076682 conn(0x55d58ffdada0 msgr2=0x55d58ffedd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 -- 172.21.15.62:0/3278076682 shutdown_connections 2023-07-14T16:23:04.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 -- 172.21.15.62:0/3278076682 wait complete. 2023-07-14T16:23:04.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.065+0000 7ff94594eb80 1 -- start start 2023-07-14T16:23:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff94594eb80 1 Processor -- start 2023-07-14T16:23:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58fff6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58fff4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff94594eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 0x55d58fff52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff94594eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58fff59f0 con 0x55d58ffe54d0 2023-07-14T16:23:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff94594eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59004ecc0 con 0x55d58ffe5e20 2023-07-14T16:23:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff94594eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59004ee00 con 0x55d58ffe6c70 2023-07-14T16:23:04.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff94594eb80 10 client.0 _ll_register_callbacks cb 0x7ff920099420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:04.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58fff6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58fff6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34504/0 (socket says 172.21.15.62:34504) 2023-07-14T16:23:04.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff92b7fe700 1 -- 172.21.15.62:0/843854399 learned_addr learned my addr 172.21.15.62:0/843854399 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.066+0000 7ff92affd700 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58fff4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff92bfff700 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 0x55d58fff52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff92b7fe700 1 -- 172.21.15.62:0/843854399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 msgr2=0x55d58fff52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 0x55d58fff52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff92b7fe700 1 -- 172.21.15.62:0/843854399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 msgr2=0x55d58fff4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58fff4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff92b7fe700 1 -- 172.21.15.62:0/843854399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c1087f0 con 0x55d58ffe54d0 2023-07-14T16:23:04.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff92b7fe700 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58fff6670 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009ce0 tx=0x7ff91c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e49ae33d593e5f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff90affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:04.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.067+0000 7ff90affd700 1 -- 172.21.15.62:0/843854399 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c01d040 con 0x55d58ffe54d0 2023-07-14T16:23:04.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff90affd700 1 -- 172.21.15.62:0/843854399 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff91c00b9f0 con 0x55d58ffe54d0 2023-07-14T16:23:04.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 1 -- 172.21.15.62:0/843854399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff92400f070 con 0x55d58ffe54d0 2023-07-14T16:23:04.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff90affd700 1 -- 172.21.15.62:0/843854399 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff91c00bbb0 con 0x55d58ffe54d0 2023-07-14T16:23:04.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 10 client.17532 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:04.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 1 -- 172.21.15.62:0/843854399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d58fd632b0 con 0x55d58ffe54d0 2023-07-14T16:23:04.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff90affd700 1 -- 172.21.15.62:0/843854399 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff92400f070 con 0x55d58ffe54d0 2023-07-14T16:23:04.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff90affd700 1 -- 172.21.15.62:0/843854399 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff91c004620 con 0x55d58ffe54d0 2023-07-14T16:23:04.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 10 client.17532 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:04.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:04.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 20 client.17532 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:04.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d590050280 0x55d590052690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 4 client.17532 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:23:04.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.068+0000 7ff94594eb80 1 -- 172.21.15.62:0/843854399 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d58fef2050 con 0x55d590050280 2023-07-14T16:23:04.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.069+0000 7ff92affd700 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d590050280 0x55d590052690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.069+0000 7ff90affd700 10 client.17532 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:04.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.069+0000 7ff92affd700 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d590050280 0x55d590052690 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3120165d9fefa server_cookie=44b7ec8dfb385822 in_seq=0 out_seq=0 2023-07-14T16:23:04.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff90affd700 1 -- 172.21.15.62:0/843854399 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d58fef2050 con 0x55d590050280 2023-07-14T16:23:04.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff90affd700 10 client.17532 handle_command_reply: tid=0 2023-07-14T16:23:04.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 client.17532 shutdown 2023-07-14T16:23:04.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 -- 172.21.15.62:0/843854399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 msgr2=0x55d58fff6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58fff6670 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009ce0 tx=0x7ff91c005a40 comp rx=0 tx=0).stop 2023-07-14T16:23:04.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 -- 172.21.15.62:0/843854399 shutdown_connections 2023-07-14T16:23:04.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d58ffe6c70 0x55d58fff52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d58ffe54d0 0x55d58fff6670 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d590050280 0x55d590052690 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 --2- 172.21.15.62:0/843854399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d58ffe5e20 0x55d58fff4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 -- 172.21.15.62:0/843854399 >> 172.21.15.62:0/843854399 conn(0x55d58ffdada0 msgr2=0x55d58ffed100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff90affd700 0 client.17532 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:04.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 -- 172.21.15.62:0/843854399 shutdown_connections 2023-07-14T16:23:04.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.070+0000 7ff94594eb80 1 -- 172.21.15.62:0/843854399 wait complete. 2023-07-14T16:23:04.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff94594eb80 20 client.17532 trim_cache size 0 max 16384 2023-07-14T16:23:04.055 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:23:04.056 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:23:04.056 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:23:04.056 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:23:04.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 -- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9140a1150 msgr2=0x7ff9140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9140a1150 0x7ff9140a3600 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff9240199a0 tx=0x7ff92400f000 comp rx=0 tx=0).stop 2023-07-14T16:23:04.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 -- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c102e80 msgr2=0x7ff92c0fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c102e80 0x7ff92c0fe510 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff92000d950 tx=0x7ff92000dc60 comp rx=0 tx=0).stop 2023-07-14T16:23:04.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 -- 172.21.15.62:0/2564819053 shutdown_connections 2023-07-14T16:23:04.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff92c1057a0 0x7ff92c0fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff9140a1150 0x7ff9140a3600 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff92c100560 0x7ff92c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 --2- 172.21.15.62:0/2564819053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff92c102e80 0x7ff92c0fe510 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 -- 172.21.15.62:0/2564819053 >> 172.21.15.62:0/2564819053 conn(0x7ff92c0fa000 msgr2=0x7ff92c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 -- 172.21.15.62:0/2564819053 shutdown_connections 2023-07-14T16:23:04.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.071+0000 7ff931fff700 1 -- 172.21.15.62:0/2564819053 wait complete. 2023-07-14T16:23:04.067 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:23:04.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:23:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2477280868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 msgr2=0x7f5eb8102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2477280868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 0x7f5eb8102930 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009990 tx=0x7f5eb4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2477280868 shutdown_connections 2023-07-14T16:23:04.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2477280868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8105790 0x7f5eb8107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2477280868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 0x7f5eb8102930 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2477280868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8102e70 0x7f5eb8105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2477280868 >> 172.21.15.62:0/2477280868 conn(0x7f5eb80f9fd0 msgr2=0x7f5eb80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2477280868 shutdown_connections 2023-07-14T16:23:04.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.328+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2477280868 wait complete. 2023-07-14T16:23:04.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.329+0000 7f5ebf9ff700 1 Processor -- start 2023-07-14T16:23:04.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebf9ff700 1 -- start start 2023-07-14T16:23:04.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebf9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 0x7f5eb80ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebf9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e70 0x7f5eb80fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebf9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8105790 0x7f5eb80fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebf9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80ff1e0 con 0x7f5eb8100550 2023-07-14T16:23:04.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebf9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80ff350 con 0x7f5eb8102e70 2023-07-14T16:23:04.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebf9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb819f780 con 0x7f5eb8105790 2023-07-14T16:23:04.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 0x7f5eb80ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 0x7f5eb80ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34532/0 (socket says 172.21.15.62:34532) 2023-07-14T16:23:04.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebd79b700 1 -- 172.21.15.62:0/2407486014 learned_addr learned my addr 172.21.15.62:0/2407486014 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.330+0000 7f5ebcf9a700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e70 0x7f5eb80fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5ebdf9c700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8105790 0x7f5eb80fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5ebd79b700 1 -- 172.21.15.62:0/2407486014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8105790 msgr2=0x7f5eb80fea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5ebd79b700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8105790 0x7f5eb80fea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5ebd79b700 1 -- 172.21.15.62:0/2407486014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e70 msgr2=0x7f5eb80fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5ebd79b700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e70 0x7f5eb80fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5ebd79b700 1 -- 172.21.15.62:0/2407486014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40095f0 con 0x7f5eb8100550 2023-07-14T16:23:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5ebdf9c700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8105790 0x7f5eb80fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5ebd79b700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 0x7f5eb80ffec0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4005910 tx=0x7f5eb4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.331+0000 7f5eae7fc700 1 -- 172.21.15.62:0/2407486014 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb401d070 con 0x7f5eb8100550 2023-07-14T16:23:04.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.332+0000 7f5eae7fc700 1 -- 172.21.15.62:0/2407486014 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5eb400be70 con 0x7f5eb8100550 2023-07-14T16:23:04.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.332+0000 7f5eae7fc700 1 -- 172.21.15.62:0/2407486014 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4022ea0 con 0x7f5eb8100550 2023-07-14T16:23:04.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.332+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb819fa00 con 0x7f5eb8100550 2023-07-14T16:23:04.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.332+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb819fdd0 con 0x7f5eb8100550 2023-07-14T16:23:04.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.333+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb80fbb40 con 0x7f5eb8100550 2023-07-14T16:23:04.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.335+0000 7f5eae7fc700 1 -- 172.21.15.62:0/2407486014 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5eb4022670 con 0x7f5eb8100550 2023-07-14T16:23:04.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.337+0000 7f5eae7fc700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5ea40a1220 0x7f5ea40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.337+0000 7f5eae7fc700 1 -- 172.21.15.62:0/2407486014 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5eb40cdcd0 con 0x7f5eb8100550 2023-07-14T16:23:04.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.337+0000 7f5ebcf9a700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5ea40a1220 0x7f5ea40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.338+0000 7f5ebcf9a700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5ea40a1220 0x7f5ea40a36d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009c60 tx=0x7f5ea8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.341+0000 7f5eae7fc700 1 -- 172.21.15.62:0/2407486014 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5eb405c790 con 0x7f5eb8100550 2023-07-14T16:23:04.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.606+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5eb802c850 con 0x7f5eb8100550 2023-07-14T16:23:04.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.607+0000 7f5eae7fc700 1 -- 172.21.15.62:0/2407486014 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f5eb4027090 con 0x7f5eb8100550 2023-07-14T16:23:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:23:04.584 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:23:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5ea40a1220 msgr2=0x7f5ea40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5ea40a1220 0x7f5ea40a36d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009c60 tx=0x7f5ea8009260 comp rx=0 tx=0).stop 2023-07-14T16:23:04.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 msgr2=0x7f5eb80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 0x7f5eb80ffec0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4005910 tx=0x7f5eb4005740 comp rx=0 tx=0).stop 2023-07-14T16:23:04.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 shutdown_connections 2023-07-14T16:23:04.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb8105790 0x7f5eb80fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb8100550 0x7f5eb80ffec0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5ea40a1220 0x7f5ea40a36d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 --2- 172.21.15.62:0/2407486014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e70 0x7f5eb80fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 >> 172.21.15.62:0/2407486014 conn(0x7f5eb80f9fd0 msgr2=0x7f5eb81011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 shutdown_connections 2023-07-14T16:23:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.609+0000 7f5ebf9ff700 1 -- 172.21.15.62:0/2407486014 wait complete. 2023-07-14T16:23:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:23:04.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:23:04.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 -- 172.21.15.62:0/3898333501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100cd0 msgr2=0x7f9584101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 --2- 172.21.15.62:0/3898333501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100cd0 0x7f9584101120 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9578009990 tx=0x7f9578009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 -- 172.21.15.62:0/3898333501 shutdown_connections 2023-07-14T16:23:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 --2- 172.21.15.62:0/3898333501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9584101660 0x7f9584107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 --2- 172.21.15.62:0/3898333501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95840ffae0 0x7f95840ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 --2- 172.21.15.62:0/3898333501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9584100cd0 0x7f9584101120 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 -- 172.21.15.62:0/3898333501 >> 172.21.15.62:0/3898333501 conn(0x7f95840fb0e0 msgr2=0x7f95840fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 -- 172.21.15.62:0/3898333501 shutdown_connections 2023-07-14T16:23:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.858+0000 7f95889ff700 1 -- 172.21.15.62:0/3898333501 wait complete. 2023-07-14T16:23:04.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.860+0000 7f95889ff700 1 Processor -- start 2023-07-14T16:23:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.860+0000 7f95889ff700 1 -- start start 2023-07-14T16:23:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.860+0000 7f95889ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffae0 0x7f9584074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.860+0000 7f95889ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9584100cd0 0x7f9584072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.860+0000 7f95889ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9584101660 0x7f9584072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.860+0000 7f95889ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840735e0 con 0x7f95840ffae0 2023-07-14T16:23:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.860+0000 7f95889ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9584073750 con 0x7f9584100cd0 2023-07-14T16:23:04.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f95889ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958419f840 con 0x7f9584101660 2023-07-14T16:23:04.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f958259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffae0 0x7f9584074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f958259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffae0 0x7f9584074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34544/0 (socket says 172.21.15.62:34544) 2023-07-14T16:23:04.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f958259c700 1 -- 172.21.15.62:0/3249497005 learned_addr learned my addr 172.21.15.62:0/3249497005 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9584100cd0 0x7f9584072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f9582d9d700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9584101660 0x7f9584072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f958259c700 1 -- 172.21.15.62:0/3249497005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9584101660 msgr2=0x7f9584072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f958259c700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9584101660 0x7f9584072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f958259c700 1 -- 172.21.15.62:0/3249497005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9584100cd0 msgr2=0x7f9584072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f958259c700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9584100cd0 0x7f9584072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f958259c700 1 -- 172.21.15.62:0/3249497005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780095f0 con 0x7f95840ffae0 2023-07-14T16:23:04.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9584100cd0 0x7f9584072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:04.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.861+0000 7f9582d9d700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9584101660 0x7f9584072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:04.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.862+0000 7f958259c700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffae0 0x7f9584074270 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f957400e970 tx=0x7f957400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.862+0000 7f95737fe700 1 -- 172.21.15.62:0/3249497005 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574009790 con 0x7f95840ffae0 2023-07-14T16:23:04.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.862+0000 7f95889ff700 1 -- 172.21.15.62:0/3249497005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958419fad0 con 0x7f95840ffae0 2023-07-14T16:23:04.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.862+0000 7f95737fe700 1 -- 172.21.15.62:0/3249497005 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f957400eec0 con 0x7f95840ffae0 2023-07-14T16:23:04.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.862+0000 7f95737fe700 1 -- 172.21.15.62:0/3249497005 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574003c10 con 0x7f95840ffae0 2023-07-14T16:23:04.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.862+0000 7f95889ff700 1 -- 172.21.15.62:0/3249497005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958419ffe0 con 0x7f95840ffae0 2023-07-14T16:23:04.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.866+0000 7f95737fe700 1 -- 172.21.15.62:0/3249497005 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f957400cb70 con 0x7f95840ffae0 2023-07-14T16:23:04.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.867+0000 7f959c2d0b80 1 Processor -- start 2023-07-14T16:23:04.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.867+0000 7f959c2d0b80 1 -- start start 2023-07-14T16:23:04.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066be18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066be26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066be30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066ae6cb0 con 0x557066be14d0 2023-07-14T16:23:04.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066b39a40 con 0x557066be2c70 2023-07-14T16:23:04.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066b32ed0 con 0x557066be1e20 2023-07-14T16:23:04.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f95737fe700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f9582d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066be18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066be30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f95737fe700 1 -- 172.21.15.62:0/3249497005 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9574014070 con 0x7f95840ffae0 2023-07-14T16:23:04.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066be30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51722/0 (socket says 172.21.15.62:51722) 2023-07-14T16:23:04.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.868+0000 7f9581d9b700 1 -- 172.21.15.62:0/3918012867 learned_addr learned my addr 172.21.15.62:0/3918012867 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.869+0000 7f958259c700 1 --2- 172.21.15.62:0/3918012867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066be26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.869+0000 7f9582d9d700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.869+0000 7f9581d9b700 1 -- 172.21.15.62:0/3918012867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 msgr2=0x557066be26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.869+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3918012867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066be26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.869+0000 7f9581d9b700 1 -- 172.21.15.62:0/3918012867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 msgr2=0x557066be18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.869+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3918012867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066be18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.869+0000 7f9581d9b700 1 -- 172.21.15.62:0/3918012867 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557066960de0 con 0x557066be2c70 2023-07-14T16:23:04.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.869+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3918012867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066be30e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9578009990 tx=0x7f9578005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f1c80fdd4410e0d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.870+0000 7f9582d9d700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f956c0a1150 0x7f956c0a3600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f957c009fd0 tx=0x7f957c009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.870+0000 7f9570ff9700 1 -- 172.21.15.62:0/3918012867 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957801d050 con 0x557066be2c70 2023-07-14T16:23:04.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.870+0000 7f9570ff9700 1 -- 172.21.15.62:0/3918012867 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f957801e040 con 0x557066be2c70 2023-07-14T16:23:04.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.870+0000 7f9570ff9700 1 -- 172.21.15.62:0/3918012867 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578005290 con 0x557066be2c70 2023-07-14T16:23:04.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.870+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3918012867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 msgr2=0x557066be30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.870+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3918012867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066be30e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9578009990 tx=0x7f9578005c00 comp rx=0 tx=0).stop 2023-07-14T16:23:04.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.871+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3918012867 shutdown_connections 2023-07-14T16:23:04.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.871+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3918012867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066be30e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.871+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3918012867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066be18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.871+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3918012867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066be26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.871+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3918012867 >> 172.21.15.62:0/3918012867 conn(0x557066bd6da0 msgr2=0x557066bdc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.871+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3918012867 shutdown_connections 2023-07-14T16:23:04.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.871+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3918012867 wait complete. 2023-07-14T16:23:04.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.871+0000 7f959c2d0b80 1 -- start start 2023-07-14T16:23:04.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f959c2d0b80 1 Processor -- start 2023-07-14T16:23:04.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be14d0 0x557066beb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be1e20 0x557066beb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be2c70 0x557066bf2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066ad2ac0 con 0x557066be2c70 2023-07-14T16:23:04.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066ad27c0 con 0x557066be1e20 2023-07-14T16:23:04.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570669b6920 con 0x557066be14d0 2023-07-14T16:23:04.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f959c2d0b80 10 client.0 _ll_register_callbacks cb 0x557066ad0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:04.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.872+0000 7f958259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be14d0 0x557066beb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f958259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be14d0 0x557066beb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55208/0 (socket says 172.21.15.62:55208) 2023-07-14T16:23:04.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9582d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be2c70 0x557066bf2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be1e20 0x557066beb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f958259c700 1 -- 172.21.15.62:0/3150369050 learned_addr learned my addr 172.21.15.62:0/3150369050 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9581d9b700 1 -- 172.21.15.62:0/3150369050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be14d0 msgr2=0x557066beb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be14d0 0x557066beb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9581d9b700 1 -- 172.21.15.62:0/3150369050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be2c70 msgr2=0x557066bf2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be2c70 0x557066bf2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9581d9b700 1 -- 172.21.15.62:0/3150369050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9584108850 con 0x557066be1e20 2023-07-14T16:23:04.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be1e20 0x557066beb600 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9578006010 tx=0x7f9578005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f740352b53f40a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.873+0000 7f9569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:23:04.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.874+0000 7f9569ffb700 1 -- 172.21.15.62:0/3150369050 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800b8b0 con 0x557066be1e20 2023-07-14T16:23:04.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.874+0000 7f9569ffb700 1 -- 172.21.15.62:0/3150369050 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f957800ba10 con 0x557066be1e20 2023-07-14T16:23:04.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.874+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3150369050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557066a8c4f0 con 0x557066be1e20 2023-07-14T16:23:04.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.874+0000 7f9569ffb700 1 -- 172.21.15.62:0/3150369050 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f957801d020 con 0x557066be1e20 2023-07-14T16:23:04.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.874+0000 7f959c2d0b80 10 client.27227 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:04.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.874+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3150369050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557066949c60 con 0x557066be1e20 2023-07-14T16:23:04.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.874+0000 7f9569ffb700 1 -- 172.21.15.62:0/3150369050 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557066a8c4f0 con 0x557066be1e20 2023-07-14T16:23:04.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.875+0000 7f9569ffb700 1 -- 172.21.15.62:0/3150369050 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f957800f460 con 0x557066be1e20 2023-07-14T16:23:04.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.875+0000 7f959c2d0b80 10 client.27227 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:04.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.875+0000 7f959c2d0b80 10 client.27227 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:23:04.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.875+0000 7f959c2d0b80 20 client.27227 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:04.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.875+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557066bf6a10 0x557066bf8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.875+0000 7f959c2d0b80 4 client.27227 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:04.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.875+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3150369050 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557066b2beb0 con 0x557066bf6a10 2023-07-14T16:23:04.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.875+0000 7f958259c700 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557066bf6a10 0x557066bf8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.876+0000 7f958259c700 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557066bf6a10 0x557066bf8df0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f1e273fecd1948 server_cookie=6e79430bfc5e3f08 in_seq=0 out_seq=0 2023-07-14T16:23:04.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.876+0000 7f9569ffb700 10 client.27227 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:23:04.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.876+0000 7f9569ffb700 1 -- 172.21.15.62:0/3150369050 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557066b2beb0 con 0x557066bf6a10 2023-07-14T16:23:04.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.876+0000 7f9569ffb700 10 client.27227 handle_command_reply: tid=0 2023-07-14T16:23:04.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 client.27227 shutdown 2023-07-14T16:23:04.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3150369050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be1e20 msgr2=0x557066beb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be1e20 0x557066beb600 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9578006010 tx=0x7f9578005a40 comp rx=0 tx=0).stop 2023-07-14T16:23:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3150369050 shutdown_connections 2023-07-14T16:23:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557066bf6a10 0x557066bf8df0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be2c70 0x557066bf2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be14d0 0x557066beb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f9569ffb700 0 client.27227 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:23:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3150369050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be1e20 0x557066beb600 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3150369050 >> 172.21.15.62:0/3150369050 conn(0x557066bd6da0 msgr2=0x557066bdb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3150369050 shutdown_connections 2023-07-14T16:23:04.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3150369050 wait complete. 2023-07-14T16:23:04.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.877+0000 7f959c2d0b80 20 client.27227 trim_cache size 0 max 16384 2023-07-14T16:23:04.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.893+0000 7f959c2d0b80 1 Processor -- start 2023-07-14T16:23:04.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.893+0000 7f959c2d0b80 1 -- start start 2023-07-14T16:23:04.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be14d0 0x557066be18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be1e20 0x557066bf2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be2c70 0x557066be9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066bd69a0 con 0x557066be1e20 2023-07-14T16:23:04.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066bebd40 con 0x557066be14d0 2023-07-14T16:23:04.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066be8e30 con 0x557066be2c70 2023-07-14T16:23:04.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be1e20 0x557066bf2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be1e20 0x557066bf2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34570/0 (socket says 172.21.15.62:34570) 2023-07-14T16:23:04.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9581d9b700 1 -- 172.21.15.62:0/1555703538 learned_addr learned my addr 172.21.15.62:0/1555703538 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9582d9d700 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be2c70 0x557066be9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f958259c700 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be14d0 0x557066be18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9581d9b700 1 -- 172.21.15.62:0/1555703538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be2c70 msgr2=0x557066be9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9581d9b700 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be2c70 0x557066be9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9581d9b700 1 -- 172.21.15.62:0/1555703538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be14d0 msgr2=0x557066be18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9581d9b700 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be14d0 0x557066be18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.894+0000 7f9581d9b700 1 -- 172.21.15.62:0/1555703538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557066987250 con 0x557066be1e20 2023-07-14T16:23:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.895+0000 7f9581d9b700 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be1e20 0x557066bf2600 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9578001eb0 tx=0x7f957800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f70535791225a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.895+0000 7f95717fa700 1 -- 172.21.15.62:0/1555703538 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957801d070 con 0x557066be1e20 2023-07-14T16:23:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.895+0000 7f95717fa700 1 -- 172.21.15.62:0/1555703538 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9578003a30 con 0x557066be1e20 2023-07-14T16:23:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.895+0000 7f95717fa700 1 -- 172.21.15.62:0/1555703538 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578004f80 con 0x557066be1e20 2023-07-14T16:23:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 -- 172.21.15.62:0/1555703538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be1e20 msgr2=0x557066bf2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be1e20 0x557066bf2600 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9578001eb0 tx=0x7f957800b260 comp rx=0 tx=0).stop 2023-07-14T16:23:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 -- 172.21.15.62:0/1555703538 shutdown_connections 2023-07-14T16:23:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be2c70 0x557066be9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be14d0 0x557066be18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/1555703538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be1e20 0x557066bf2600 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 -- 172.21.15.62:0/1555703538 >> 172.21.15.62:0/1555703538 conn(0x557066bd6da0 msgr2=0x557066bfd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 -- 172.21.15.62:0/1555703538 shutdown_connections 2023-07-14T16:23:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 -- 172.21.15.62:0/1555703538 wait complete. 2023-07-14T16:23:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.896+0000 7f959c2d0b80 1 -- start start 2023-07-14T16:23:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f959c2d0b80 1 Processor -- start 2023-07-14T16:23:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066bee790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066beecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f959c2d0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066bf18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066bf1e30 con 0x557066be14d0 2023-07-14T16:23:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066bf1fa0 con 0x557066be2c70 2023-07-14T16:23:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f959c2d0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557066bf2160 con 0x557066be1e20 2023-07-14T16:23:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f959c2d0b80 10 client.0 _ll_register_callbacks cb 0x7f957c009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.897+0000 7f958259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066bee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066beecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:04 smithi062 ceph-mon[95543]: pgmap v580: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:04.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:04 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2407486014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:23:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066beecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55220/0 (socket says 172.21.15.62:55220) 2023-07-14T16:23:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 -- 172.21.15.62:0/3300276890 learned_addr learned my addr 172.21.15.62:0/3300276890 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9582d9d700 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066bf18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 -- 172.21.15.62:0/3300276890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 msgr2=0x557066bf18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066bf18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 -- 172.21.15.62:0/3300276890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 msgr2=0x557066bee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066bee790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 -- 172.21.15.62:0/3300276890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557066949c60 con 0x557066be1e20 2023-07-14T16:23:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f958259c700 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066bee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9581d9b700 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066beecd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9578009ce0 tx=0x7f957800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1e323c6a733e4c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.898+0000 7f9569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:23:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.899+0000 7f9569ffb700 1 -- 172.21.15.62:0/3300276890 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957801d070 con 0x557066be1e20 2023-07-14T16:23:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.899+0000 7f9569ffb700 1 -- 172.21.15.62:0/3300276890 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9578000e40 con 0x557066be1e20 2023-07-14T16:23:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.899+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3300276890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557066aee050 con 0x557066be1e20 2023-07-14T16:23:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.899+0000 7f9569ffb700 1 -- 172.21.15.62:0/3300276890 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9578003a30 con 0x557066be1e20 2023-07-14T16:23:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.899+0000 7f959c2d0b80 10 client.27115 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.899+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3300276890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55706695f2b0 con 0x557066be1e20 2023-07-14T16:23:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f9569ffb700 1 -- 172.21.15.62:0/3300276890 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557066aee050 con 0x557066be1e20 2023-07-14T16:23:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f9569ffb700 1 -- 172.21.15.62:0/3300276890 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9578005560 con 0x557066be1e20 2023-07-14T16:23:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f959c2d0b80 10 client.27115 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f959c2d0b80 10 client.27115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:23:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f959c2d0b80 20 client.27115 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557066c4b220 0x557066c4d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f959c2d0b80 4 client.27115 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:23:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3300276890 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557066981f70 con 0x557066c4b220 2023-07-14T16:23:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.900+0000 7f958259c700 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557066c4b220 0x557066c4d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f958259c700 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557066c4b220 0x557066c4d600 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60346f702430314d server_cookie=64fd9a62f750f974 in_seq=0 out_seq=0 2023-07-14T16:23:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f9569ffb700 10 client.27115 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:23:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f9569ffb700 1 -- 172.21.15.62:0/3300276890 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557066981f70 con 0x557066c4b220 2023-07-14T16:23:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f9569ffb700 10 client.27115 handle_command_reply: tid=0 2023-07-14T16:23:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 client.27115 shutdown 2023-07-14T16:23:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3300276890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 msgr2=0x557066beecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066beecd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9578009ce0 tx=0x7f957800f690 comp rx=0 tx=0).stop 2023-07-14T16:23:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3300276890 shutdown_connections 2023-07-14T16:23:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557066c4b220 0x557066c4d600 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557066be2c70 0x557066bf18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557066be14d0 0x557066bee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 --2- 172.21.15.62:0/3300276890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557066be1e20 0x557066beecd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f9569ffb700 0 client.27115 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:23:04.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.901+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3300276890 >> 172.21.15.62:0/3300276890 conn(0x557066bd6da0 msgr2=0x557066be9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.902+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3300276890 shutdown_connections 2023-07-14T16:23:04.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.902+0000 7f959c2d0b80 1 -- 172.21.15.62:0/3300276890 wait complete. 2023-07-14T16:23:04.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.902+0000 7f959c2d0b80 20 client.27115 trim_cache size 0 max 16384 2023-07-14T16:23:04.890 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:23:04.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.902+0000 7f95889ff700 1 -- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f956c0a1150 msgr2=0x7f956c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.902+0000 7f95889ff700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f956c0a1150 0x7f956c0a3600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f957c009fd0 tx=0x7f957c009620 comp rx=0 tx=0).stop 2023-07-14T16:23:04.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.902+0000 7f95889ff700 1 -- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffae0 msgr2=0x7f9584074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:04.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.902+0000 7f95889ff700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffae0 0x7f9584074270 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f957400e970 tx=0x7f957400ec80 comp rx=0 tx=0).stop 2023-07-14T16:23:04.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.903+0000 7f95889ff700 1 -- 172.21.15.62:0/3249497005 shutdown_connections 2023-07-14T16:23:04.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.903+0000 7f95889ff700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9584101660 0x7f9584072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.903+0000 7f95889ff700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95840ffae0 0x7f9584074270 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.903+0000 7f95889ff700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.903+0000 7f95889ff700 1 --2- 172.21.15.62:0/3249497005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9584100cd0 0x7f9584072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:04.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.903+0000 7f95889ff700 1 -- 172.21.15.62:0/3249497005 >> 172.21.15.62:0/3249497005 conn(0x7f95840fb0e0 msgr2=0x7f95841042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:04.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.903+0000 7f95889ff700 1 -- 172.21.15.62:0/3249497005 shutdown_connections 2023-07-14T16:23:04.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:04.903+0000 7f95889ff700 1 -- 172.21.15.62:0/3249497005 wait complete. 2023-07-14T16:23:04.901 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:23:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:04 smithi164 ceph-mon[97427]: pgmap v580: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:05.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:04 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2407486014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:23:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 -- 172.21.15.62:0/3030784260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac072f10 msgr2=0x7fd7ac073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/3030784260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac072f10 0x7fd7ac073380 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009990 tx=0x7fd7a0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 -- 172.21.15.62:0/3030784260 shutdown_connections 2023-07-14T16:23:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/3030784260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7ac107980 0x7fd7ac109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/3030784260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7ac0744f0 0x7fd7ac072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/3030784260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac072f10 0x7fd7ac073380 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 -- 172.21.15.62:0/3030784260 >> 172.21.15.62:0/3030784260 conn(0x7fd7ac0fb130 msgr2=0x7fd7ac0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 -- 172.21.15.62:0/3030784260 shutdown_connections 2023-07-14T16:23:05.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.161+0000 7fd7b1dff700 1 -- 172.21.15.62:0/3030784260 wait complete. 2023-07-14T16:23:05.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.163+0000 7fd7b1dff700 1 Processor -- start 2023-07-14T16:23:05.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.163+0000 7fd7b1dff700 1 -- start start 2023-07-14T16:23:05.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.163+0000 7fd7b1dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7ac072f10 0x7fd7ac197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.163+0000 7fd7b1dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac0744f0 0x7fd7ac197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.163+0000 7fd7b1dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7ac107980 0x7fd7ac19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.163+0000 7fd7b1dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac19f510 con 0x7fd7ac0744f0 2023-07-14T16:23:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.163+0000 7fd7b1dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac19f650 con 0x7fd7ac072f10 2023-07-14T16:23:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.163+0000 7fd7b1dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac19f7e0 con 0x7fd7ac107980 2023-07-14T16:23:05.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac0744f0 0x7fd7ac197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7ab7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7ac072f10 0x7fd7ac197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac0744f0 0x7fd7ac197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34600/0 (socket says 172.21.15.62:34600) 2023-07-14T16:23:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7aaffd700 1 -- 172.21.15.62:0/4180840404 learned_addr learned my addr 172.21.15.62:0/4180840404 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7ab7fe700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7ac072f10 0x7fd7ac197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51776/0 (socket says 172.21.15.62:51776) 2023-07-14T16:23:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7abfff700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7ac107980 0x7fd7ac19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7ab7fe700 1 -- 172.21.15.62:0/4180840404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7ac107980 msgr2=0x7fd7ac19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7ab7fe700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7ac107980 0x7fd7ac19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7ab7fe700 1 -- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac0744f0 msgr2=0x7fd7ac197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7ab7fe700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac0744f0 0x7fd7ac197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.164+0000 7fd7ab7fe700 1 -- 172.21.15.62:0/4180840404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00095f0 con 0x7fd7ac072f10 2023-07-14T16:23:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.165+0000 7fd7ab7fe700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7ac072f10 0x7fd7ac197990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00e9f0 tx=0x7fd79c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.165+0000 7fd7a8ff9700 1 -- 172.21.15.62:0/4180840404 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00c9c0 con 0x7fd7ac072f10 2023-07-14T16:23:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.165+0000 7fd7a8ff9700 1 -- 172.21.15.62:0/4180840404 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd79c010430 con 0x7fd7ac072f10 2023-07-14T16:23:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.165+0000 7fd7b1dff700 1 -- 172.21.15.62:0/4180840404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac19fac0 con 0x7fd7ac072f10 2023-07-14T16:23:05.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.165+0000 7fd7a8ff9700 1 -- 172.21.15.62:0/4180840404 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c018680 con 0x7fd7ac072f10 2023-07-14T16:23:05.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.165+0000 7fd7b1dff700 1 -- 172.21.15.62:0/4180840404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac1a0020 con 0x7fd7ac072f10 2023-07-14T16:23:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7a8ff9700 1 -- 172.21.15.62:0/4180840404 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd79c0105a0 con 0x7fd7ac072f10 2023-07-14T16:23:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7c56b0b80 1 Processor -- start 2023-07-14T16:23:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7c56b0b80 1 -- start start 2023-07-14T16:23:05.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5da4d0 0x56341b5da8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5db6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dbc70 0x56341b5dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b4dfcb0 con 0x56341b5da4d0 2023-07-14T16:23:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b532a40 con 0x56341b5dae20 2023-07-14T16:23:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.171+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b52bed0 con 0x56341b5dbc70 2023-07-14T16:23:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5da4d0 0x56341b5da8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5da4d0 0x56341b5da8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34604/0 (socket says 172.21.15.62:34604) 2023-07-14T16:23:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7abfff700 1 -- 172.21.15.62:0/1213874016 learned_addr learned my addr 172.21.15.62:0/1213874016 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dbc70 0x56341b5dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7ab7fe700 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5db6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7abfff700 1 -- 172.21.15.62:0/1213874016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dbc70 msgr2=0x56341b5dc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7abfff700 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dbc70 0x56341b5dc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7abfff700 1 -- 172.21.15.62:0/1213874016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 msgr2=0x56341b5db6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7abfff700 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5db6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7abfff700 1 -- 172.21.15.62:0/1213874016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56341b359de0 con 0x56341b5da4d0 2023-07-14T16:23:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.172+0000 7fd7ab7fe700 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5db6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.173+0000 7fd7abfff700 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5da4d0 0x56341b5da8e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4003f90 tx=0x7fd7a4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8922e4207efdf2e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.173+0000 7fd791ffb700 1 -- 172.21.15.62:0/1213874016 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4009e60 con 0x56341b5da4d0 2023-07-14T16:23:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.173+0000 7fd791ffb700 1 -- 172.21.15.62:0/1213874016 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a400f040 con 0x56341b5da4d0 2023-07-14T16:23:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.173+0000 7fd791ffb700 1 -- 172.21.15.62:0/1213874016 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40156e0 con 0x56341b5da4d0 2023-07-14T16:23:05.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.173+0000 7fd7a8ff9700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7940a11d0 0x7fd7940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.173+0000 7fd7a8ff9700 1 -- 172.21.15.62:0/4180840404 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd79c014070 con 0x7fd7ac072f10 2023-07-14T16:23:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.173+0000 7fd7ab7fe700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7940a11d0 0x7fd7940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/1213874016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5da4d0 msgr2=0x56341b5da8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5da4d0 0x56341b5da8e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4003f90 tx=0x7fd7a4009c20 comp rx=0 tx=0).stop 2023-07-14T16:23:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/1213874016 shutdown_connections 2023-07-14T16:23:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dbc70 0x56341b5dc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5da4d0 0x56341b5da8e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/1213874016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5db6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7ab7fe700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7940a11d0 0x7fd7940a3680 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0cca90 tx=0x7fd79c01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/1213874016 >> 172.21.15.62:0/1213874016 conn(0x56341b5cfda0 msgr2=0x56341b5d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/1213874016 shutdown_connections 2023-07-14T16:23:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.174+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/1213874016 wait complete. 2023-07-14T16:23:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.175+0000 7fd7c56b0b80 1 -- start start 2023-07-14T16:23:05.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.175+0000 7fd7c56b0b80 1 Processor -- start 2023-07-14T16:23:05.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.175+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.175+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b4cbac0 con 0x56341b5dbc70 2023-07-14T16:23:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b4cb7c0 con 0x56341b5da4d0 2023-07-14T16:23:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b3af920 con 0x56341b5dae20 2023-07-14T16:23:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7c56b0b80 10 client.0 _ll_register_callbacks cb 0x56341b4c9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5e4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51794/0 (socket says 172.21.15.62:51794) 2023-07-14T16:23:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 -- 172.21.15.62:0/2498406149 learned_addr learned my addr 172.21.15.62:0/2498406149 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 -- 172.21.15.62:0/2498406149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 msgr2=0x56341b5e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5e4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 -- 172.21.15.62:0/2498406149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 msgr2=0x56341b5eb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5eb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.176+0000 7fd7abfff700 1 -- 172.21.15.62:0/2498406149 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a4003bf0 con 0x56341b5da4d0 2023-07-14T16:23:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.177+0000 7fd7abfff700 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5e4120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009f90 tx=0x7fd7a400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eec15c85f67ee6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.177+0000 7fd78affd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:23:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.177+0000 7fd78affd700 1 -- 172.21.15.62:0/2498406149 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4015a40 con 0x56341b5da4d0 2023-07-14T16:23:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.177+0000 7fd78affd700 1 -- 172.21.15.62:0/2498406149 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7a4015ba0 con 0x56341b5da4d0 2023-07-14T16:23:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.177+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/2498406149 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56341b4854f0 con 0x56341b5da4d0 2023-07-14T16:23:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.177+0000 7fd78affd700 1 -- 172.21.15.62:0/2498406149 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a4015d60 con 0x56341b5da4d0 2023-07-14T16:23:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.177+0000 7fd7c56b0b80 10 client.27239 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.177+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/2498406149 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56341b342c60 con 0x56341b5da4d0 2023-07-14T16:23:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.178+0000 7fd78affd700 1 -- 172.21.15.62:0/2498406149 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56341b4854f0 con 0x56341b5da4d0 2023-07-14T16:23:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.178+0000 7fd78affd700 1 -- 172.21.15.62:0/2498406149 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7a401e450 con 0x56341b5da4d0 2023-07-14T16:23:05.164 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:04 smithi081 ceph-mon[127795]: pgmap v580: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:05.164 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:04 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2407486014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:23:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.178+0000 7fd7c56b0b80 10 client.27239 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.178+0000 7fd7c56b0b80 10 client.27239 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:23:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.178+0000 7fd7c56b0b80 20 client.27239 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.178+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56341b5efe50 0x56341b5f2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.178+0000 7fd7c56b0b80 4 client.27239 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.178+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/2498406149 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56341b524eb0 con 0x56341b5efe50 2023-07-14T16:23:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.179+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56341b5efe50 0x56341b5f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.179+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56341b5efe50 0x56341b5f2260 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91c67fcdb6c97af9 server_cookie=310865677851fc03 in_seq=0 out_seq=0 2023-07-14T16:23:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.179+0000 7fd78affd700 10 client.27239 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:23:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.180+0000 7fd78affd700 1 -- 172.21.15.62:0/2498406149 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56341b524eb0 con 0x56341b5efe50 2023-07-14T16:23:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.180+0000 7fd78affd700 10 client.27239 handle_command_reply: tid=0 2023-07-14T16:23:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.180+0000 7fd7c56b0b80 1 client.27239 shutdown 2023-07-14T16:23:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.180+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/2498406149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 msgr2=0x56341b5e4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5e4120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009f90 tx=0x7fd7a400b040 comp rx=0 tx=0).stop 2023-07-14T16:23:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/2498406149 shutdown_connections 2023-07-14T16:23:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56341b5efe50 0x56341b5f2260 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5eb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5e4120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd78affd700 0 client.27239 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:23:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/2498406149 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/2498406149 >> 172.21.15.62:0/2498406149 conn(0x56341b5cfda0 msgr2=0x56341b5d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/2498406149 shutdown_connections 2023-07-14T16:23:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/2498406149 wait complete. 2023-07-14T16:23:05.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.181+0000 7fd7c56b0b80 20 client.27239 trim_cache size 0 max 16384 2023-07-14T16:23:05.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.196+0000 7fd7c56b0b80 1 Processor -- start 2023-07-14T16:23:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.196+0000 7fd7c56b0b80 1 -- start start 2023-07-14T16:23:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5dc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b5d60c0 con 0x56341b5dbc70 2023-07-14T16:23:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b5e35d0 con 0x56341b5da4d0 2023-07-14T16:23:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b5d6a00 con 0x56341b5dae20 2023-07-14T16:23:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5f6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34614/0 (socket says 172.21.15.62:34614) 2023-07-14T16:23:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7abfff700 1 -- 172.21.15.62:0/4031983546 learned_addr learned my addr 172.21.15.62:0/4031983546 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.197+0000 7fd7abfff700 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5dc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7abfff700 1 -- 172.21.15.62:0/4031983546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 msgr2=0x56341b5eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7abfff700 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7abfff700 1 -- 172.21.15.62:0/4031983546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 msgr2=0x56341b5dc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7abfff700 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5dc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7abfff700 1 -- 172.21.15.62:0/4031983546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56341b380250 con 0x56341b5dbc70 2023-07-14T16:23:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7abfff700 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5f6b80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x56341b5e5380 tx=0x7fd7a40095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2541aa98df480318 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7927fc700 1 -- 172.21.15.62:0/4031983546 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a401ea50 con 0x56341b5dbc70 2023-07-14T16:23:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7927fc700 1 -- 172.21.15.62:0/4031983546 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a4029070 con 0x56341b5dbc70 2023-07-14T16:23:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.198+0000 7fd7927fc700 1 -- 172.21.15.62:0/4031983546 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4015aa0 con 0x56341b5dbc70 2023-07-14T16:23:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/4031983546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 msgr2=0x56341b5f6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5f6b80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x56341b5e5380 tx=0x7fd7a40095d0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/4031983546 shutdown_connections 2023-07-14T16:23:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5f6b80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5da4d0 0x56341b5dc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/4031983546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5dae20 0x56341b5eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/4031983546 >> 172.21.15.62:0/4031983546 conn(0x56341b5cfda0 msgr2=0x56341b5d51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/4031983546 shutdown_connections 2023-07-14T16:23:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.199+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/4031983546 wait complete. 2023-07-14T16:23:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 1 -- start start 2023-07-14T16:23:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 1 Processor -- start 2023-07-14T16:23:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5da4d0 0x56341b5e7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b5ead10 con 0x56341b5dbc70 2023-07-14T16:23:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b5eae80 con 0x56341b5dae20 2023-07-14T16:23:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56341b5eb040 con 0x56341b5da4d0 2023-07-14T16:23:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.200+0000 7fd7c56b0b80 10 client.0 _ll_register_callbacks cb 0x7fd7a4015790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5ea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34624/0 (socket says 172.21.15.62:34624) 2023-07-14T16:23:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 -- 172.21.15.62:0/200975717 learned_addr learned my addr 172.21.15.62:0/200975717 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5da4d0 0x56341b5e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 -- 172.21.15.62:0/200975717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5da4d0 msgr2=0x56341b5e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5da4d0 0x56341b5e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 -- 172.21.15.62:0/200975717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 msgr2=0x56341b5e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7abfff700 1 -- 172.21.15.62:0/200975717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac10aa20 con 0x56341b5dbc70 2023-07-14T16:23:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.201+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:05.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd7abfff700 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5ea7d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009660 tx=0x7fd7a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28853bb5e4121179 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd78affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd78affd700 1 -- 172.21.15.62:0/200975717 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a401e450 con 0x56341b5dbc70 2023-07-14T16:23:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/200975717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a400f040 con 0x56341b5dbc70 2023-07-14T16:23:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd78affd700 1 -- 172.21.15.62:0/200975717 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7a401e5b0 con 0x56341b5dbc70 2023-07-14T16:23:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd78affd700 1 -- 172.21.15.62:0/200975717 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a4015840 con 0x56341b5dbc70 2023-07-14T16:23:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd7c56b0b80 10 client.17574 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:05.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/200975717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56341b342c60 con 0x56341b5dbc70 2023-07-14T16:23:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.202+0000 7fd78affd700 1 -- 172.21.15.62:0/200975717 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a400f040 con 0x56341b5dbc70 2023-07-14T16:23:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.203+0000 7fd78affd700 1 -- 172.21.15.62:0/200975717 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7a4004040 con 0x56341b5dbc70 2023-07-14T16:23:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.203+0000 7fd7c56b0b80 10 client.17574 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.203+0000 7fd7c56b0b80 10 client.17574 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:23:05.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.203+0000 7fd7c56b0b80 20 client.17574 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.203+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56341b6447f0 0x56341b646c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.203+0000 7fd7c56b0b80 4 client.17574 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:23:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.203+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/200975717 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56341b4e7050 con 0x56341b6447f0 2023-07-14T16:23:05.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.203+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56341b6447f0 0x56341b646c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.204+0000 7fd7aaffd700 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56341b6447f0 0x56341b646c00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e9db4ec0d2bb2ec server_cookie=870053d6310113e1 in_seq=0 out_seq=0 2023-07-14T16:23:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.204+0000 7fd78affd700 10 client.17574 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:23:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.204+0000 7fd78affd700 1 -- 172.21.15.62:0/200975717 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56341b4e7050 con 0x56341b6447f0 2023-07-14T16:23:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.204+0000 7fd78affd700 10 client.17574 handle_command_reply: tid=0 2023-07-14T16:23:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.204+0000 7fd7c56b0b80 1 client.17574 shutdown 2023-07-14T16:23:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.204+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/200975717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 msgr2=0x56341b5ea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.204+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5ea7d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009660 tx=0x7fd7a400b040 comp rx=0 tx=0).stop 2023-07-14T16:23:05.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/200975717 shutdown_connections 2023-07-14T16:23:05.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56341b6447f0 0x56341b646c00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56341b5dbc70 0x56341b5ea7d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56341b5da4d0 0x56341b5e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 1 --2- 172.21.15.62:0/200975717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56341b5dae20 0x56341b5e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/200975717 >> 172.21.15.62:0/200975717 conn(0x56341b5cfda0 msgr2=0x56341b5dead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd78affd700 0 client.17574 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:23:05.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/200975717 shutdown_connections 2023-07-14T16:23:05.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 1 -- 172.21.15.62:0/200975717 wait complete. 2023-07-14T16:23:05.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7c56b0b80 20 client.17574 trim_cache size 0 max 16384 2023-07-14T16:23:05.191 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:23:05.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7b1dff700 1 -- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7940a11d0 msgr2=0x7fd7940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.205+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7940a11d0 0x7fd7940a3680 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0cca90 tx=0x7fd79c01f000 comp rx=0 tx=0).stop 2023-07-14T16:23:05.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 -- 172.21.15.62:0/4180840404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7ac072f10 msgr2=0x7fd7ac197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7ac072f10 0x7fd7ac197990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00e9f0 tx=0x7fd79c00ed00 comp rx=0 tx=0).stop 2023-07-14T16:23:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 -- 172.21.15.62:0/4180840404 shutdown_connections 2023-07-14T16:23:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7ac107980 0x7fd7ac19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7940a11d0 0x7fd7940a3680 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7ac072f10 0x7fd7ac197990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 --2- 172.21.15.62:0/4180840404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ac0744f0 0x7fd7ac197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 -- 172.21.15.62:0/4180840404 >> 172.21.15.62:0/4180840404 conn(0x7fd7ac0fb130 msgr2=0x7fd7ac0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 -- 172.21.15.62:0/4180840404 shutdown_connections 2023-07-14T16:23:05.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.206+0000 7fd7b1dff700 1 -- 172.21.15.62:0/4180840404 wait complete. 2023-07-14T16:23:05.200 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:23:05.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 -- 172.21.15.62:0/1713235969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 msgr2=0x7f936c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 --2- 172.21.15.62:0/1713235969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 0x7f936c107c00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f936400b0a0 tx=0x7f936400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 -- 172.21.15.62:0/1713235969 shutdown_connections 2023-07-14T16:23:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 --2- 172.21.15.62:0/1713235969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 0x7f936c107c00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 --2- 172.21.15.62:0/1713235969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f936c100560 0x7f936c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 --2- 172.21.15.62:0/1713235969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f936c102e80 0x7f936c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 -- 172.21.15.62:0/1713235969 >> 172.21.15.62:0/1713235969 conn(0x7f936c0fa000 msgr2=0x7f936c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 -- 172.21.15.62:0/1713235969 shutdown_connections 2023-07-14T16:23:05.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.474+0000 7f93735ff700 1 -- 172.21.15.62:0/1713235969 wait complete. 2023-07-14T16:23:05.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.475+0000 7f93735ff700 1 Processor -- start 2023-07-14T16:23:05.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f93735ff700 1 -- start start 2023-07-14T16:23:05.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f93735ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f936c100560 0x7f936c195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f93735ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f936c102e80 0x7f936c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f93735ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 0x7f936c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f93735ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c19d3d0 con 0x7f936c1057a0 2023-07-14T16:23:05.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f93735ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c19d510 con 0x7f936c102e80 2023-07-14T16:23:05.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f93735ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c19d6a0 con 0x7f936c100560 2023-07-14T16:23:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f9371b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 0x7f936c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f9371b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 0x7f936c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34652/0 (socket says 172.21.15.62:34652) 2023-07-14T16:23:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.476+0000 7f9371b9c700 1 -- 172.21.15.62:0/39267020 learned_addr learned my addr 172.21.15.62:0/39267020 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f9370b9a700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f936c102e80 0x7f936c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f937139b700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f936c100560 0x7f936c195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f9371b9c700 1 -- 172.21.15.62:0/39267020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f936c100560 msgr2=0x7f936c195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f9371b9c700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f936c100560 0x7f936c195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f9371b9c700 1 -- 172.21.15.62:0/39267020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f936c102e80 msgr2=0x7f936c195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f9371b9c700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f936c102e80 0x7f936c195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f9371b9c700 1 -- 172.21.15.62:0/39267020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009520 con 0x7f936c1057a0 2023-07-14T16:23:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f9371b9c700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 0x7f936c19ce00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f936400b780 tx=0x7f936400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f93627fc700 1 -- 172.21.15.62:0/39267020 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400e040 con 0x7f936c1057a0 2023-07-14T16:23:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f93627fc700 1 -- 172.21.15.62:0/39267020 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9364004e60 con 0x7f936c1057a0 2023-07-14T16:23:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.477+0000 7f93735ff700 1 -- 172.21.15.62:0/39267020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9364009d00 con 0x7f936c1057a0 2023-07-14T16:23:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.478+0000 7f93627fc700 1 -- 172.21.15.62:0/39267020 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640052a0 con 0x7f936c1057a0 2023-07-14T16:23:05.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.478+0000 7f93735ff700 1 -- 172.21.15.62:0/39267020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c19dc10 con 0x7f936c1057a0 2023-07-14T16:23:05.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.482+0000 7f9386e35b80 1 Processor -- start 2023-07-14T16:23:05.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.482+0000 7f93627fc700 1 -- 172.21.15.62:0/39267020 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f936401d990 con 0x7f936c1057a0 2023-07-14T16:23:05.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.483+0000 7f9386e35b80 1 -- start start 2023-07-14T16:23:05.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.483+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 0x56098f64ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.483+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f64faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.483+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f650080 0x56098f6504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.483+0000 7f9386e35b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f54fcb0 con 0x56098f650080 2023-07-14T16:23:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.483+0000 7f9386e35b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f5a2a40 con 0x56098f64e8e0 2023-07-14T16:23:05.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.483+0000 7f9386e35b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f59bed0 con 0x56098f64f230 2023-07-14T16:23:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f937139b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f64faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f937139b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f64faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55302/0 (socket says 172.21.15.62:55302) 2023-07-14T16:23:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f9370b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f650080 0x56098f6504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f937139b700 1 -- 172.21.15.62:0/2856454623 learned_addr learned my addr 172.21.15.62:0/2856454623 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f9370b9a700 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 0x56098f64ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f937139b700 1 -- 172.21.15.62:0/2856454623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 msgr2=0x56098f64ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f937139b700 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 0x56098f64ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f937139b700 1 -- 172.21.15.62:0/2856454623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f650080 msgr2=0x56098f6504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f937139b700 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f650080 0x56098f6504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f937139b700 1 -- 172.21.15.62:0/2856454623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56098f3c9de0 con 0x56098f64f230 2023-07-14T16:23:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.484+0000 7f9370b9a700 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f650080 0x56098f6504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.485+0000 7f93627fc700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93580a11d0 0x7f93580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.485+0000 7f937139b700 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f64faf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9368009eb0 tx=0x7f93680116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e31be144ea6221c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.485+0000 7f93627fc700 1 -- 172.21.15.62:0/39267020 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f93640ce100 con 0x7f936c1057a0 2023-07-14T16:23:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.485+0000 7f9370b9a700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93580a11d0 0x7f93580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.485+0000 7f93577fe700 1 -- 172.21.15.62:0/2856454623 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368011e20 con 0x56098f64f230 2023-07-14T16:23:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.485+0000 7f93577fe700 1 -- 172.21.15.62:0/2856454623 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93680128e0 con 0x56098f64f230 2023-07-14T16:23:05.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.485+0000 7f93577fe700 1 -- 172.21.15.62:0/2856454623 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800f770 con 0x56098f64f230 2023-07-14T16:23:05.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 -- 172.21.15.62:0/2856454623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 msgr2=0x56098f64faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f64faf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9368009eb0 tx=0x7f93680116c0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9370b9a700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93580a11d0 0x7f93580a3680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f935c014a40 tx=0x7f935c00c450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 -- 172.21.15.62:0/2856454623 shutdown_connections 2023-07-14T16:23:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f650080 0x56098f6504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 0x56098f64ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2856454623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f64faf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 -- 172.21.15.62:0/2856454623 >> 172.21.15.62:0/2856454623 conn(0x56098f63fda0 msgr2=0x56098f6451d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 -- 172.21.15.62:0/2856454623 shutdown_connections 2023-07-14T16:23:05.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.486+0000 7f9386e35b80 1 -- 172.21.15.62:0/2856454623 wait complete. 2023-07-14T16:23:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.487+0000 7f9386e35b80 1 -- start start 2023-07-14T16:23:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.487+0000 7f9386e35b80 1 Processor -- start 2023-07-14T16:23:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.488+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64e8e0 0x56098f658530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.488+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f658a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.488+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f65fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.488+0000 7f9386e35b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f53bac0 con 0x56098f64e8e0 2023-07-14T16:23:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.488+0000 7f9386e35b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f53b7c0 con 0x56098f650080 2023-07-14T16:23:05.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.488+0000 7f9386e35b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f41f920 con 0x56098f64f230 2023-07-14T16:23:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.488+0000 7f9386e35b80 10 client.0 _ll_register_callbacks cb 0x56098f539760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.488+0000 7f937139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64e8e0 0x56098f658530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f937139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64e8e0 0x56098f658530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34674/0 (socket says 172.21.15.62:34674) 2023-07-14T16:23:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f937139b700 1 -- 172.21.15.62:0/983912407 learned_addr learned my addr 172.21.15.62:0/983912407 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f9370b9a700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f658a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f9371b9c700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f65fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f937139b700 1 -- 172.21.15.62:0/983912407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 msgr2=0x56098f658a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f937139b700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f658a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f937139b700 1 -- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 msgr2=0x56098f65fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f937139b700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f65fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f937139b700 1 -- 172.21.15.62:0/983912407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c014600 con 0x56098f64e8e0 2023-07-14T16:23:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f9370b9a700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f658a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f9371b9c700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f65fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f937139b700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64e8e0 0x56098f658530 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f936801c040 tx=0x7f9368012850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a0ef23a4e50618 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.489+0000 7f9354ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.490+0000 7f9354ff9700 1 -- 172.21.15.62:0/983912407 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368005040 con 0x56098f64e8e0 2023-07-14T16:23:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.490+0000 7f9354ff9700 1 -- 172.21.15.62:0/983912407 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f93680051a0 con 0x56098f64e8e0 2023-07-14T16:23:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.490+0000 7f9386e35b80 1 -- 172.21.15.62:0/983912407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56098f4f54f0 con 0x56098f64e8e0 2023-07-14T16:23:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.490+0000 7f9354ff9700 1 -- 172.21.15.62:0/983912407 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9368005360 con 0x56098f64e8e0 2023-07-14T16:23:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.490+0000 7f9386e35b80 10 client.17586 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.490+0000 7f9386e35b80 1 -- 172.21.15.62:0/983912407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f9368009ae0 con 0x56098f64e8e0 2023-07-14T16:23:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.490+0000 7f9354ff9700 1 -- 172.21.15.62:0/983912407 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56098f4f54f0 con 0x56098f64e8e0 2023-07-14T16:23:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.491+0000 7f9354ff9700 1 -- 172.21.15.62:0/983912407 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9368022070 con 0x56098f64e8e0 2023-07-14T16:23:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.491+0000 7f9386e35b80 10 client.17586 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.491+0000 7f9386e35b80 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.491+0000 7f9386e35b80 20 client.17586 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.491+0000 7f9386e35b80 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56098f6641e0 0x56098f6665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.491+0000 7f9386e35b80 4 client.17586 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.491+0000 7f9386e35b80 1 -- 172.21.15.62:0/983912407 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56098f594eb0 con 0x56098f6641e0 2023-07-14T16:23:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.491+0000 7f9370b9a700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56098f6641e0 0x56098f6665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.492+0000 7f9354ff9700 10 client.17586 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.492+0000 7f9370b9a700 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56098f6641e0 0x56098f6665f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb017ba3d0b67d03 server_cookie=a1b4d337465ce5f5 in_seq=0 out_seq=0 2023-07-14T16:23:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.492+0000 7f9354ff9700 1 -- 172.21.15.62:0/983912407 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56098f594eb0 con 0x56098f6641e0 2023-07-14T16:23:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.492+0000 7f9354ff9700 10 client.17586 handle_command_reply: tid=0 2023-07-14T16:23:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 client.17586 shutdown 2023-07-14T16:23:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 -- 172.21.15.62:0/983912407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64e8e0 msgr2=0x56098f658530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64e8e0 0x56098f658530 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f936801c040 tx=0x7f9368012850 comp rx=0 tx=0).stop 2023-07-14T16:23:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 -- 172.21.15.62:0/983912407 shutdown_connections 2023-07-14T16:23:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56098f6641e0 0x56098f6665f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f65fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64e8e0 0x56098f658530 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9354ff9700 0 client.17586 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 --2- 172.21.15.62:0/983912407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64f230 0x56098f658a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 -- 172.21.15.62:0/983912407 >> 172.21.15.62:0/983912407 conn(0x56098f63fda0 msgr2=0x56098f6448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 -- 172.21.15.62:0/983912407 shutdown_connections 2023-07-14T16:23:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 1 -- 172.21.15.62:0/983912407 wait complete. 2023-07-14T16:23:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.493+0000 7f9386e35b80 20 client.17586 trim_cache size 0 max 16384 2023-07-14T16:23:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.507+0000 7f9386e35b80 1 Processor -- start 2023-07-14T16:23:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.507+0000 7f9386e35b80 1 -- start start 2023-07-14T16:23:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.507+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64e8e0 0x56098f657be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.507+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.507+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f657260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.507+0000 7f9386e35b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f656f90 con 0x56098f64f230 2023-07-14T16:23:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.507+0000 7f9386e35b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f646c60 con 0x56098f650080 2023-07-14T16:23:05.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.507+0000 7f9386e35b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f64ef00 con 0x56098f64e8e0 2023-07-14T16:23:05.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34684/0 (socket says 172.21.15.62:34684) 2023-07-14T16:23:05.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 -- 172.21.15.62:0/3226546444 learned_addr learned my addr 172.21.15.62:0/3226546444 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f937139b700 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64e8e0 0x56098f657be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9371b9c700 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f657260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 -- 172.21.15.62:0/3226546444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64e8e0 msgr2=0x56098f657be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64e8e0 0x56098f657be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 -- 172.21.15.62:0/3226546444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 msgr2=0x56098f657260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f657260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 -- 172.21.15.62:0/3226546444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56098f3b2c60 con 0x56098f64f230 2023-07-14T16:23:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.508+0000 7f9370b9a700 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65fa70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f935c00d010 tx=0x7f935c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d74b031d9e8cd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9357fff700 1 -- 172.21.15.62:0/3226546444 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c02e070 con 0x56098f64f230 2023-07-14T16:23:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9357fff700 1 -- 172.21.15.62:0/3226546444 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f935c01a070 con 0x56098f64f230 2023-07-14T16:23:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9357fff700 1 -- 172.21.15.62:0/3226546444 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c027070 con 0x56098f64f230 2023-07-14T16:23:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9386e35b80 1 -- 172.21.15.62:0/3226546444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 msgr2=0x56098f65fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9386e35b80 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65fa70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f935c00d010 tx=0x7f935c004750 comp rx=0 tx=0).stop 2023-07-14T16:23:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9386e35b80 1 -- 172.21.15.62:0/3226546444 shutdown_connections 2023-07-14T16:23:05.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9386e35b80 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f650080 0x56098f657260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9386e35b80 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f64e8e0 0x56098f657be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9386e35b80 1 --2- 172.21.15.62:0/3226546444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65fa70 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.509+0000 7f9386e35b80 1 -- 172.21.15.62:0/3226546444 >> 172.21.15.62:0/3226546444 conn(0x56098f63fda0 msgr2=0x56098f66af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.510+0000 7f9386e35b80 1 -- 172.21.15.62:0/3226546444 shutdown_connections 2023-07-14T16:23:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.510+0000 7f9386e35b80 1 -- 172.21.15.62:0/3226546444 wait complete. 2023-07-14T16:23:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.510+0000 7f9386e35b80 1 -- start start 2023-07-14T16:23:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.510+0000 7f9386e35b80 1 Processor -- start 2023-07-14T16:23:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 0x56098f65bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9386e35b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f650080 0x56098f65f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9386e35b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f65f830 con 0x56098f64f230 2023-07-14T16:23:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9386e35b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f65f9a0 con 0x56098f64e8e0 2023-07-14T16:23:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9386e35b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098f65fb60 con 0x56098f650080 2023-07-14T16:23:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9386e35b80 10 client.0 _ll_register_callbacks cb 0x7f935c00ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9370b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9370b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34690/0 (socket says 172.21.15.62:34690) 2023-07-14T16:23:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9370b9a700 1 -- 172.21.15.62:0/2328302192 learned_addr learned my addr 172.21.15.62:0/2328302192 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f937139b700 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 0x56098f65bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.511+0000 7f9371b9c700 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f650080 0x56098f65f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.512+0000 7f9370b9a700 1 -- 172.21.15.62:0/2328302192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f650080 msgr2=0x56098f65f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.512+0000 7f9370b9a700 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f650080 0x56098f65f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.512+0000 7f9370b9a700 1 -- 172.21.15.62:0/2328302192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 msgr2=0x56098f65bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.512+0000 7f9370b9a700 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 0x56098f65bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.512+0000 7f9370b9a700 1 -- 172.21.15.62:0/2328302192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c0297f0 con 0x56098f64f230 2023-07-14T16:23:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.512+0000 7f9370b9a700 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65c2d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f935c00c870 tx=0x7f935c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3648b06ee34c8811 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.512+0000 7f9354ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.512+0000 7f9354ff9700 1 -- 172.21.15.62:0/2328302192 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c02e040 con 0x56098f64f230 2023-07-14T16:23:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9354ff9700 1 -- 172.21.15.62:0/2328302192 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f935c025070 con 0x56098f64f230 2023-07-14T16:23:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 1 -- 172.21.15.62:0/2328302192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93680128e0 con 0x56098f64f230 2023-07-14T16:23:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9354ff9700 1 -- 172.21.15.62:0/2328302192 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f935c027040 con 0x56098f64f230 2023-07-14T16:23:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 10 client.17592 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:05.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 1 -- 172.21.15.62:0/2328302192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56098f3f0250 con 0x56098f64f230 2023-07-14T16:23:05.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9354ff9700 1 -- 172.21.15.62:0/2328302192 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93680128e0 con 0x56098f64f230 2023-07-14T16:23:05.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9354ff9700 1 -- 172.21.15.62:0/2328302192 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f935c03eba0 con 0x56098f64f230 2023-07-14T16:23:05.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 10 client.17592 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:05.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:05.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 20 client.17592 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:05.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56098f6b9480 0x56098f6bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:05.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 4 client.17592 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:23:05.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.513+0000 7f9386e35b80 1 -- 172.21.15.62:0/2328302192 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56098f557050 con 0x56098f6b9480 2023-07-14T16:23:05.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.514+0000 7f937139b700 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56098f6b9480 0x56098f6bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:05.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9354ff9700 10 client.17592 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:05.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f937139b700 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56098f6b9480 0x56098f6bb890 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc7b464210d6b0e server_cookie=298f5a51dea8bfa2 in_seq=0 out_seq=0 2023-07-14T16:23:05.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9354ff9700 1 -- 172.21.15.62:0/2328302192 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56098f557050 con 0x56098f6b9480 2023-07-14T16:23:05.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9354ff9700 10 client.17592 handle_command_reply: tid=0 2023-07-14T16:23:05.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 client.17592 shutdown 2023-07-14T16:23:05.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 -- 172.21.15.62:0/2328302192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 msgr2=0x56098f65c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65c2d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f935c00c870 tx=0x7f935c00c8a0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 -- 172.21.15.62:0/2328302192 shutdown_connections 2023-07-14T16:23:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56098f650080 0x56098f65f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56098f6b9480 0x56098f6bb890 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56098f64e8e0 0x56098f65bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 --2- 172.21.15.62:0/2328302192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56098f64f230 0x56098f65c2d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9354ff9700 0 client.17592 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:05.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 -- 172.21.15.62:0/2328302192 >> 172.21.15.62:0/2328302192 conn(0x56098f63fda0 msgr2=0x56098f657920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.515+0000 7f9386e35b80 1 -- 172.21.15.62:0/2328302192 shutdown_connections 2023-07-14T16:23:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f9386e35b80 1 -- 172.21.15.62:0/2328302192 wait complete. 2023-07-14T16:23:05.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f9386e35b80 20 client.17592 trim_cache size 0 max 16384 2023-07-14T16:23:05.498 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:23:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 -- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93580a11d0 msgr2=0x7f93580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93580a11d0 0x7f93580a3680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f935c014a40 tx=0x7f935c00c450 comp rx=0 tx=0).stop 2023-07-14T16:23:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 -- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 msgr2=0x7f936c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 0x7f936c19ce00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f936400b780 tx=0x7f936400bdf0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 -- 172.21.15.62:0/39267020 shutdown_connections 2023-07-14T16:23:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f936c1057a0 0x7f936c19ce00 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f936c100560 0x7f936c195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f93580a11d0 0x7f93580a3680 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 --2- 172.21.15.62:0/39267020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f936c102e80 0x7f936c195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.516+0000 7f93735ff700 1 -- 172.21.15.62:0/39267020 >> 172.21.15.62:0/39267020 conn(0x7f936c0fa000 msgr2=0x7f936c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.517+0000 7f93735ff700 1 -- 172.21.15.62:0/39267020 shutdown_connections 2023-07-14T16:23:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:05.517+0000 7f93735ff700 1 -- 172.21.15.62:0/39267020 wait complete. 2023-07-14T16:23:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 -- 172.21.15.62:0/1146664701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0105860 msgr2=0x7ffbb0107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/1146664701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0105860 0x7ffbb0107c40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009e00 tx=0x7ffba800b330 comp rx=0 tx=0).stop 2023-07-14T16:23:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 -- 172.21.15.62:0/1146664701 shutdown_connections 2023-07-14T16:23:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/1146664701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0105860 0x7ffbb0107c40 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/1146664701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100620 0x7ffbb0102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/1146664701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb0102f40 0x7ffbb0105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 -- 172.21.15.62:0/1146664701 >> 172.21.15.62:0/1146664701 conn(0x7ffbb00fa000 msgr2=0x7ffbb00fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 -- 172.21.15.62:0/1146664701 shutdown_connections 2023-07-14T16:23:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.185+0000 7ffbb59ff700 1 -- 172.21.15.62:0/1146664701 wait complete. 2023-07-14T16:23:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.186+0000 7ffbb59ff700 1 Processor -- start 2023-07-14T16:23:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.186+0000 7ffbb59ff700 1 -- start start 2023-07-14T16:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.186+0000 7ffbb59ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100620 0x7ffbb00fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.186+0000 7ffbb59ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb0102f40 0x7ffbb00feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.186+0000 7ffbb59ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0105860 0x7ffbb00ff050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.186+0000 7ffbb59ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019f940 con 0x7ffbb0105860 2023-07-14T16:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.186+0000 7ffbb59ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019fa80 con 0x7ffbb0100620 2023-07-14T16:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.186+0000 7ffbb59ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019fbc0 con 0x7ffbb0102f40 2023-07-14T16:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100620 0x7ffbb00fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0105860 0x7ffbb00ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100620 0x7ffbb00fe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51886/0 (socket says 172.21.15.62:51886) 2023-07-14T16:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaeffd700 1 -- 172.21.15.62:0/3300935553 learned_addr learned my addr 172.21.15.62:0/3300935553 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb0102f40 0x7ffbb00feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaeffd700 1 -- 172.21.15.62:0/3300935553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb0102f40 msgr2=0x7ffbb00feb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb0102f40 0x7ffbb00feb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaeffd700 1 -- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0105860 msgr2=0x7ffbb00ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0105860 0x7ffbb00ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.187+0000 7ffbaeffd700 1 -- 172.21.15.62:0/3300935553 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00095f0 con 0x7ffbb0100620 2023-07-14T16:23:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.188+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100620 0x7ffbb00fe5d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffba000ec70 tx=0x7ffba000ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.188+0000 7ffbb49fd700 1 -- 172.21.15.62:0/3300935553 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba00099d0 con 0x7ffbb0100620 2023-07-14T16:23:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.188+0000 7ffbb59ff700 1 -- 172.21.15.62:0/3300935553 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba8005e70 con 0x7ffbb0100620 2023-07-14T16:23:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.188+0000 7ffbb49fd700 1 -- 172.21.15.62:0/3300935553 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba00044b0 con 0x7ffbb0100620 2023-07-14T16:23:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.188+0000 7ffbb49fd700 1 -- 172.21.15.62:0/3300935553 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0003e30 con 0x7ffbb0100620 2023-07-14T16:23:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.188+0000 7ffbb59ff700 1 -- 172.21.15.62:0/3300935553 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb01a01d0 con 0x7ffbb0100620 2023-07-14T16:23:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.191+0000 7ffbb49fd700 1 -- 172.21.15.62:0/3300935553 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffba0004000 con 0x7ffbb0100620 2023-07-14T16:23:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.193+0000 7ffbb49fd700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a1150 0x7ffb980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.193+0000 7ffbb49fd700 1 -- 172.21.15.62:0/3300935553 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffba0014070 con 0x7ffbb0100620 2023-07-14T16:23:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.193+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a1150 0x7ffb980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.193+0000 7ffbc93aab80 1 Processor -- start 2023-07-14T16:23:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbc93aab80 1 -- start start 2023-07-14T16:23:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c841656b0 0x560c84309300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a1150 0x7ffb980a3600 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00ff8d0 tx=0x7ffba4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c8430a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 0x560c8430ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8420ec50 con 0x560c841656b0 2023-07-14T16:23:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c842619e0 con 0x560c8430a6e0 2023-07-14T16:23:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c8425ae70 con 0x560c84309840 2023-07-14T16:23:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbaeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c841656b0 0x560c84309300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbaeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c841656b0 0x560c84309300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34722/0 (socket says 172.21.15.62:34722) 2023-07-14T16:23:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.194+0000 7ffbaeffd700 1 -- 172.21.15.62:0/1030390705 learned_addr learned my addr 172.21.15.62:0/1030390705 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbaf7fe700 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 0x560c8430ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c8430a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbaeffd700 1 -- 172.21.15.62:0/1030390705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 msgr2=0x560c8430a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c8430a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbaeffd700 1 -- 172.21.15.62:0/1030390705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 msgr2=0x560c8430ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 0x560c8430ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbaeffd700 1 -- 172.21.15.62:0/1030390705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c84088d80 con 0x560c841656b0 2023-07-14T16:23:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c8430a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c841656b0 0x560c84309300 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffba00a20b0 tx=0x7ffba009a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48b97beff0456c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffb95ffb700 1 -- 172.21.15.62:0/1030390705 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba009a190 con 0x560c841656b0 2023-07-14T16:23:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffb95ffb700 1 -- 172.21.15.62:0/1030390705 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba00ade90 con 0x560c841656b0 2023-07-14T16:23:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.195+0000 7ffb95ffb700 1 -- 172.21.15.62:0/1030390705 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0098530 con 0x560c841656b0 2023-07-14T16:23:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.196+0000 7ffbc93aab80 1 -- 172.21.15.62:0/1030390705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c841656b0 msgr2=0x560c84309300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.196+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c841656b0 0x560c84309300 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffba00a20b0 tx=0x7ffba009a090 comp rx=0 tx=0).stop 2023-07-14T16:23:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.196+0000 7ffbc93aab80 1 -- 172.21.15.62:0/1030390705 shutdown_connections 2023-07-14T16:23:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.196+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 0x560c8430ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.196+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c841656b0 0x560c84309300 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.196+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/1030390705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c8430a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.196+0000 7ffbc93aab80 1 -- 172.21.15.62:0/1030390705 >> 172.21.15.62:0/1030390705 conn(0x560c842febe0 msgr2=0x560c84303cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.197+0000 7ffbc93aab80 1 -- 172.21.15.62:0/1030390705 shutdown_connections 2023-07-14T16:23:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.197+0000 7ffbc93aab80 1 -- 172.21.15.62:0/1030390705 wait complete. 2023-07-14T16:23:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.197+0000 7ffbc93aab80 1 -- start start 2023-07-14T16:23:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.198+0000 7ffbc93aab80 1 Processor -- start 2023-07-14T16:23:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.198+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c84312880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.198+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c84309840 0x560c84312dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.198+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 0x560c84319e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.198+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c841faa60 con 0x560c84309840 2023-07-14T16:23:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.198+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c841fa760 con 0x560c8430a6e0 2023-07-14T16:23:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.198+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c840de8c0 con 0x560c841656b0 2023-07-14T16:23:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.198+0000 7ffbc93aab80 10 client.0 _ll_register_callbacks cb 0x560c841f8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbae7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c84309840 0x560c84312dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbaeffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c84312880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbaeffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c84312880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55376/0 (socket says 172.21.15.62:55376) 2023-07-14T16:23:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbaeffd700 1 -- 172.21.15.62:0/653291592 learned_addr learned my addr 172.21.15.62:0/653291592 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbaf7fe700 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 0x560c84319e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbae7fc700 1 -- 172.21.15.62:0/653291592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 msgr2=0x560c84312880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c84312880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbae7fc700 1 -- 172.21.15.62:0/653291592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 msgr2=0x560c84319e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 0x560c84319e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbae7fc700 1 -- 172.21.15.62:0/653291592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb01a01d0 con 0x560c84309840 2023-07-14T16:23:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c84312880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c84309840 0x560c84312dc0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ffba4004490 tx=0x7ffba40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b952b210e65acf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.199+0000 7ffb8effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.200+0000 7ffb8effd700 1 -- 172.21.15.62:0/653291592 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4027440 con 0x560c84309840 2023-07-14T16:23:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.200+0000 7ffb8effd700 1 -- 172.21.15.62:0/653291592 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffba40275a0 con 0x560c84309840 2023-07-14T16:23:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.200+0000 7ffbc93aab80 1 -- 172.21.15.62:0/653291592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c841b4490 con 0x560c84309840 2023-07-14T16:23:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.200+0000 7ffb8effd700 1 -- 172.21.15.62:0/653291592 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffba402d030 con 0x560c84309840 2023-07-14T16:23:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.200+0000 7ffbc93aab80 10 client.17604 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.200+0000 7ffbc93aab80 1 -- 172.21.15.62:0/653291592 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560c84071c00 con 0x560c84309840 2023-07-14T16:23:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.200+0000 7ffb8effd700 1 -- 172.21.15.62:0/653291592 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c841b4490 con 0x560c84309840 2023-07-14T16:23:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.200+0000 7ffb8effd700 1 -- 172.21.15.62:0/653291592 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffba401c530 con 0x560c84309840 2023-07-14T16:23:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.201+0000 7ffbc93aab80 10 client.17604 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.201+0000 7ffbc93aab80 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.201+0000 7ffbc93aab80 20 client.17604 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.201+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560c8431e690 0x560c84320aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.201+0000 7ffbc93aab80 4 client.17604 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.201+0000 7ffbc93aab80 1 -- 172.21.15.62:0/653291592 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c84253e50 con 0x560c8431e690 2023-07-14T16:23:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.201+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560c8431e690 0x560c84320aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.202+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560c8431e690 0x560c84320aa0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af1ace215555545 server_cookie=5bef7b289f40f9c1 in_seq=0 out_seq=0 2023-07-14T16:23:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.202+0000 7ffb8effd700 10 client.17604 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffb8effd700 1 -- 172.21.15.62:0/653291592 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c84253e50 con 0x560c8431e690 2023-07-14T16:23:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffb8effd700 10 client.17604 handle_command_reply: tid=0 2023-07-14T16:23:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 client.17604 shutdown 2023-07-14T16:23:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 -- 172.21.15.62:0/653291592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c84309840 msgr2=0x560c84312dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c84309840 0x560c84312dc0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ffba4004490 tx=0x7ffba40047c0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 -- 172.21.15.62:0/653291592 shutdown_connections 2023-07-14T16:23:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560c8431e690 0x560c84320aa0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c8430a6e0 0x560c84319e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c84312880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/653291592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c84309840 0x560c84312dc0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffbc93aab80 1 -- 172.21.15.62:0/653291592 >> 172.21.15.62:0/653291592 conn(0x560c842febe0 msgr2=0x560c843034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.203+0000 7ffb8effd700 0 client.17604 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.204+0000 7ffbc93aab80 1 -- 172.21.15.62:0/653291592 shutdown_connections 2023-07-14T16:23:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.204+0000 7ffbc93aab80 1 -- 172.21.15.62:0/653291592 wait complete. 2023-07-14T16:23:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.204+0000 7ffbc93aab80 20 client.17604 trim_cache size 0 max 16384 2023-07-14T16:23:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.219+0000 7ffbc93aab80 1 Processor -- start 2023-07-14T16:23:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.219+0000 7ffbc93aab80 1 -- start start 2023-07-14T16:23:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c8430d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c84309840 0x560c84319dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c8430dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c841b4490 con 0x560c8430a6e0 2023-07-14T16:23:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c843049f0 con 0x560c84309840 2023-07-14T16:23:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c84300d00 con 0x560c841656b0 2023-07-14T16:23:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbaf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c8430dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbaf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c8430dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34738/0 (socket says 172.21.15.62:34738) 2023-07-14T16:23:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbaf7fe700 1 -- 172.21.15.62:0/177037665 learned_addr learned my addr 172.21.15.62:0/177037665 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c8430d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.220+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c84309840 0x560c84319dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffbaf7fe700 1 -- 172.21.15.62:0/177037665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 msgr2=0x560c8430d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffbaf7fe700 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c8430d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffbaf7fe700 1 -- 172.21.15.62:0/177037665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c84309840 msgr2=0x560c84319dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffbaf7fe700 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c84309840 0x560c84319dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffbaf7fe700 1 -- 172.21.15.62:0/177037665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c840af1f0 con 0x560c8430a6e0 2023-07-14T16:23:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c84309840 0x560c84319dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffbaf7fe700 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c8430dad0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009430 tx=0x7ffba8009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3325d46924ccee1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffb967fc700 1 -- 172.21.15.62:0/177037665 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800e040 con 0x560c8430a6e0 2023-07-14T16:23:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffb967fc700 1 -- 172.21.15.62:0/177037665 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba8021070 con 0x560c8430a6e0 2023-07-14T16:23:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.221+0000 7ffb967fc700 1 -- 172.21.15.62:0/177037665 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba801d7a0 con 0x560c8430a6e0 2023-07-14T16:23:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 -- 172.21.15.62:0/177037665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 msgr2=0x560c8430dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c8430dad0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009430 tx=0x7ffba8009460 comp rx=0 tx=0).stop 2023-07-14T16:23:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 -- 172.21.15.62:0/177037665 shutdown_connections 2023-07-14T16:23:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c8430dad0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c841656b0 0x560c8430d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/177037665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c84309840 0x560c84319dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 -- 172.21.15.62:0/177037665 >> 172.21.15.62:0/177037665 conn(0x560c842febe0 msgr2=0x560c84310a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 -- 172.21.15.62:0/177037665 shutdown_connections 2023-07-14T16:23:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.222+0000 7ffbc93aab80 1 -- 172.21.15.62:0/177037665 wait complete. 2023-07-14T16:23:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.223+0000 7ffbc93aab80 1 -- start start 2023-07-14T16:23:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.223+0000 7ffbc93aab80 1 Processor -- start 2023-07-14T16:23:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.223+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c841656b0 0x560c84315fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.223+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c84316510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.223+0000 7ffbc93aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c84319530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c84319a70 con 0x560c8430a6e0 2023-07-14T16:23:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c84319be0 con 0x560c841656b0 2023-07-14T16:23:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbc93aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c84319da0 con 0x560c84309840 2023-07-14T16:23:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbc93aab80 10 client.0 _ll_register_callbacks cb 0x7ffba4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c84319530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c84319530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34746/0 (socket says 172.21.15.62:34746) 2023-07-14T16:23:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaf7fe700 1 -- 172.21.15.62:0/696873152 learned_addr learned my addr 172.21.15.62:0/696873152 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c841656b0 0x560c84315fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbae7fc700 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c84316510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaf7fe700 1 -- 172.21.15.62:0/696873152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 msgr2=0x560c84316510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaf7fe700 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c84316510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaf7fe700 1 -- 172.21.15.62:0/696873152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c841656b0 msgr2=0x560c84315fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaf7fe700 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c841656b0 0x560c84315fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.224+0000 7ffbaf7fe700 1 -- 172.21.15.62:0/696873152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb0108860 con 0x560c8430a6e0 2023-07-14T16:23:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.225+0000 7ffbaf7fe700 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c84319530 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009320 tx=0x7ffba80079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc5f041f9299b52 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.225+0000 7ffb8effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.225+0000 7ffb8effd700 1 -- 172.21.15.62:0/696873152 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800e040 con 0x560c8430a6e0 2023-07-14T16:23:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.225+0000 7ffbc93aab80 1 -- 172.21.15.62:0/696873152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffba00ade90 con 0x560c8430a6e0 2023-07-14T16:23:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.225+0000 7ffb8effd700 1 -- 172.21.15.62:0/696873152 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffba801d520 con 0x560c8430a6e0 2023-07-14T16:23:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.225+0000 7ffb8effd700 1 -- 172.21.15.62:0/696873152 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffba8007c10 con 0x560c8430a6e0 2023-07-14T16:23:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.225+0000 7ffbc93aab80 10 client.17610 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.225+0000 7ffbc93aab80 1 -- 172.21.15.62:0/696873152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560c84087250 con 0x560c8430a6e0 2023-07-14T16:23:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.226+0000 7ffb8effd700 1 -- 172.21.15.62:0/696873152 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba00ade90 con 0x560c8430a6e0 2023-07-14T16:23:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.226+0000 7ffb8effd700 1 -- 172.21.15.62:0/696873152 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffba800b050 con 0x560c8430a6e0 2023-07-14T16:23:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.226+0000 7ffbc93aab80 10 client.17610 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.226+0000 7ffbc93aab80 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.226+0000 7ffbc93aab80 20 client.17610 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.226+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560c8437bed0 0x560c8437e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.226+0000 7ffbc93aab80 4 client.17610 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:23:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.226+0000 7ffbc93aab80 1 -- 172.21.15.62:0/696873152 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c84215ff0 con 0x560c8437bed0 2023-07-14T16:23:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.227+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560c8437bed0 0x560c8437e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.227+0000 7ffbaeffd700 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560c8437bed0 0x560c8437e2e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=448941c00c538e66 server_cookie=5f4d45b01703b11 in_seq=0 out_seq=0 2023-07-14T16:23:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.227+0000 7ffb8effd700 10 client.17610 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.236+0000 7ffb8effd700 1 -- 172.21.15.62:0/696873152 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c84215ff0 con 0x560c8437bed0 2023-07-14T16:23:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.236+0000 7ffb8effd700 10 client.17610 handle_command_reply: tid=0 2023-07-14T16:23:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 client.17610 shutdown 2023-07-14T16:23:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 -- 172.21.15.62:0/696873152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 msgr2=0x560c84319530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c84319530 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009320 tx=0x7ffba80079e0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 -- 172.21.15.62:0/696873152 shutdown_connections 2023-07-14T16:23:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c8430a6e0 0x560c84319530 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560c8437bed0 0x560c8437e2e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c841656b0 0x560c84315fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 --2- 172.21.15.62:0/696873152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c84309840 0x560c84316510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 -- 172.21.15.62:0/696873152 >> 172.21.15.62:0/696873152 conn(0x560c842febe0 msgr2=0x560c84310a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffb8effd700 0 client.17610 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 -- 172.21.15.62:0/696873152 shutdown_connections 2023-07-14T16:23:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 1 -- 172.21.15.62:0/696873152 wait complete. 2023-07-14T16:23:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.237+0000 7ffbc93aab80 20 client.17610 trim_cache size 0 max 16384 2023-07-14T16:23:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:23:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:23:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:23:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:23:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 -- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a1150 msgr2=0x7ffb980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a1150 0x7ffb980a3600 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00ff8d0 tx=0x7ffba4009260 comp rx=0 tx=0).stop 2023-07-14T16:23:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 -- 172.21.15.62:0/3300935553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100620 msgr2=0x7ffbb00fe5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100620 0x7ffbb00fe5d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffba000ec70 tx=0x7ffba000ef80 comp rx=0 tx=0).stop 2023-07-14T16:23:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 -- 172.21.15.62:0/3300935553 shutdown_connections 2023-07-14T16:23:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0105860 0x7ffbb00ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a1150 0x7ffb980a3600 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100620 0x7ffbb00fe5d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 --2- 172.21.15.62:0/3300935553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb0102f40 0x7ffbb00feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 -- 172.21.15.62:0/3300935553 >> 172.21.15.62:0/3300935553 conn(0x7ffbb00fa000 msgr2=0x7ffbb0101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 -- 172.21.15.62:0/3300935553 shutdown_connections 2023-07-14T16:23:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:06.238+0000 7ffbb59ff700 1 -- 172.21.15.62:0/3300935553 wait complete. 2023-07-14T16:23:06.511 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:23:06.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:23:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 -- 172.21.15.62:0/1205962527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4105690 msgr2=0x7f86d4107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 --2- 172.21.15.62:0/1205962527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4105690 0x7f86d4107af0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00b0a0 tx=0x7f86cc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 -- 172.21.15.62:0/1205962527 shutdown_connections 2023-07-14T16:23:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 --2- 172.21.15.62:0/1205962527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4105690 0x7f86d4107af0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 --2- 172.21.15.62:0/1205962527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d4100450 0x7f86d4102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 --2- 172.21.15.62:0/1205962527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d4102d70 0x7f86d4105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 -- 172.21.15.62:0/1205962527 >> 172.21.15.62:0/1205962527 conn(0x7f86d40f9ed0 msgr2=0x7f86d40fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 -- 172.21.15.62:0/1205962527 shutdown_connections 2023-07-14T16:23:06.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.780+0000 7f86d919d700 1 -- 172.21.15.62:0/1205962527 wait complete. 2023-07-14T16:23:06.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.781+0000 7f86d919d700 1 Processor -- start 2023-07-14T16:23:06.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d919d700 1 -- start start 2023-07-14T16:23:06.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d919d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d4100450 0x7f86d40ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d919d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4102d70 0x7f86d40fe400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d919d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d4105690 0x7f86d40fe940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d919d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40ff0e0 con 0x7f86d4102d70 2023-07-14T16:23:06.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d919d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40ff250 con 0x7f86d4105690 2023-07-14T16:23:06.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d919d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d419f5b0 con 0x7f86d4100450 2023-07-14T16:23:06.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4102d70 0x7f86d40fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4102d70 0x7f86d40fe400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34754/0 (socket says 172.21.15.62:34754) 2023-07-14T16:23:06.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.782+0000 7f86d37fe700 1 -- 172.21.15.62:0/553537133 learned_addr learned my addr 172.21.15.62:0/553537133 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d899c700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d4105690 0x7f86d40fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d3fff700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d4100450 0x7f86d40ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d37fe700 1 -- 172.21.15.62:0/553537133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d4100450 msgr2=0x7f86d40ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d37fe700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d4100450 0x7f86d40ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d37fe700 1 -- 172.21.15.62:0/553537133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d4105690 msgr2=0x7f86d40fe940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d37fe700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d4105690 0x7f86d40fe940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d37fe700 1 -- 172.21.15.62:0/553537133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc009d00 con 0x7f86d4102d70 2023-07-14T16:23:06.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d3fff700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d4100450 0x7f86d40ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:06.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.783+0000 7f86d37fe700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4102d70 0x7f86d40fe400 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9a0 tx=0x7f86c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.784+0000 7f86d17fa700 1 -- 172.21.15.62:0/553537133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8009790 con 0x7f86d4102d70 2023-07-14T16:23:06.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.784+0000 7f86d919d700 1 -- 172.21.15.62:0/553537133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d419f890 con 0x7f86d4102d70 2023-07-14T16:23:06.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.784+0000 7f86d17fa700 1 -- 172.21.15.62:0/553537133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c800cbd0 con 0x7f86d4102d70 2023-07-14T16:23:06.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.784+0000 7f86d17fa700 1 -- 172.21.15.62:0/553537133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8018680 con 0x7f86d4102d70 2023-07-14T16:23:06.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.784+0000 7f86d919d700 1 -- 172.21.15.62:0/553537133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d419fdf0 con 0x7f86d4102d70 2023-07-14T16:23:06.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.787+0000 7f86d17fa700 1 -- 172.21.15.62:0/553537133 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f86c800cd40 con 0x7f86d4102d70 2023-07-14T16:23:06.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.789+0000 7f86d17fa700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86bc0a11f0 0x7f86bc0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.789+0000 7f86edda1b80 1 Processor -- start 2023-07-14T16:23:06.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.789+0000 7f86d17fa700 1 -- 172.21.15.62:0/553537133 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86c8014070 con 0x7f86d4102d70 2023-07-14T16:23:06.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.789+0000 7f86edda1b80 1 -- start start 2023-07-14T16:23:06.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.789+0000 7f86d3fff700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86bc0a11f0 0x7f86bc0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.789+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb00bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb01920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb02320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86edda1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceba05d30 con 0x556cebb01eb0 2023-07-14T16:23:06.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86edda1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceba58ac0 con 0x556cebb007a0 2023-07-14T16:23:06.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86edda1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceba51f50 con 0x556cebb010f0 2023-07-14T16:23:06.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb00bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb00bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51950/0 (socket says 172.21.15.62:51950) 2023-07-14T16:23:06.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 -- 172.21.15.62:0/2259961272 learned_addr learned my addr 172.21.15.62:0/2259961272 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d899c700 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb02320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d37fe700 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb01920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86bc0a11f0 0x7f86bc0a36a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009d00 tx=0x7f86c40116c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 -- 172.21.15.62:0/2259961272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 msgr2=0x556cebb01920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb01920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 -- 172.21.15.62:0/2259961272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 msgr2=0x556cebb02320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb02320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.790+0000 7f86d3fff700 1 -- 172.21.15.62:0/2259961272 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ceb87fe60 con 0x556cebb007a0 2023-07-14T16:23:06.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.791+0000 7f86d3fff700 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb00bb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f86c4011f70 tx=0x7f86c401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd33867a4eb9024c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.791+0000 7f86ba7fc700 1 -- 172.21.15.62:0/2259961272 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40142d0 con 0x556cebb007a0 2023-07-14T16:23:06.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.791+0000 7f86ba7fc700 1 -- 172.21.15.62:0/2259961272 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c40044c0 con 0x556cebb007a0 2023-07-14T16:23:06.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.791+0000 7f86ba7fc700 1 -- 172.21.15.62:0/2259961272 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4003cb0 con 0x556cebb007a0 2023-07-14T16:23:06.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 -- 172.21.15.62:0/2259961272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 msgr2=0x556cebb00bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb00bb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f86c4011f70 tx=0x7f86c401c000 comp rx=0 tx=0).stop 2023-07-14T16:23:06.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 -- 172.21.15.62:0/2259961272 shutdown_connections 2023-07-14T16:23:06.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb02320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb00bb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 --2- 172.21.15.62:0/2259961272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb01920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 -- 172.21.15.62:0/2259961272 >> 172.21.15.62:0/2259961272 conn(0x556cebaf6070 msgr2=0x556cebafb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 -- 172.21.15.62:0/2259961272 shutdown_connections 2023-07-14T16:23:06.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.792+0000 7f86edda1b80 1 -- 172.21.15.62:0/2259961272 wait complete. 2023-07-14T16:23:06.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.793+0000 7f86edda1b80 1 -- start start 2023-07-14T16:23:06.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.793+0000 7f86edda1b80 1 Processor -- start 2023-07-14T16:23:06.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.794+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb007a0 0x556cebb0a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.794+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb0a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.794+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb01eb0 0x556cebb11940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.794+0000 7f86edda1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceb9f1b40 con 0x556cebb007a0 2023-07-14T16:23:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.794+0000 7f86edda1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceb9f1840 con 0x556cebb01eb0 2023-07-14T16:23:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.794+0000 7f86edda1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceb8d59a0 con 0x556cebb010f0 2023-07-14T16:23:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.794+0000 7f86edda1b80 10 client.0 _ll_register_callbacks cb 0x556ceb9ef7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.794+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb007a0 0x556cebb0a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb007a0 0x556cebb0a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34772/0 (socket says 172.21.15.62:34772) 2023-07-14T16:23:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d3fff700 1 -- 172.21.15.62:0/3219899646 learned_addr learned my addr 172.21.15.62:0/3219899646 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d37fe700 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb0a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d3fff700 1 -- 172.21.15.62:0/3219899646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 msgr2=0x556cebb0a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d3fff700 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb0a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d899c700 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb01eb0 0x556cebb11940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d3fff700 1 -- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb01eb0 msgr2=0x556cebb11940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d3fff700 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb01eb0 0x556cebb11940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d3fff700 1 -- 172.21.15.62:0/3219899646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c4011c70 con 0x556cebb007a0 2023-07-14T16:23:06.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d899c700 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb01eb0 0x556cebb11940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:06.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86d3fff700 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb007a0 0x556cebb0a3c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f86c4011ac0 tx=0x7f86c4009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8411dbced0a68834 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.795+0000 7f86b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:06.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.796+0000 7f86b37fe700 1 -- 172.21.15.62:0/3219899646 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4005440 con 0x556cebb007a0 2023-07-14T16:23:06.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.796+0000 7f86edda1b80 1 -- 172.21.15.62:0/3219899646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ceb9ab570 con 0x556cebb007a0 2023-07-14T16:23:06.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.796+0000 7f86b37fe700 1 -- 172.21.15.62:0/3219899646 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86c401a050 con 0x556cebb007a0 2023-07-14T16:23:06.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.796+0000 7f86b37fe700 1 -- 172.21.15.62:0/3219899646 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c40055a0 con 0x556cebb007a0 2023-07-14T16:23:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.796+0000 7f86edda1b80 10 client.17622 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.796+0000 7f86edda1b80 1 -- 172.21.15.62:0/3219899646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ceb868ce0 con 0x556cebb007a0 2023-07-14T16:23:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.796+0000 7f86b37fe700 1 -- 172.21.15.62:0/3219899646 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ceb9ab570 con 0x556cebb007a0 2023-07-14T16:23:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.796+0000 7f86b37fe700 1 -- 172.21.15.62:0/3219899646 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86c4003c90 con 0x556cebb007a0 2023-07-14T16:23:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.797+0000 7f86edda1b80 10 client.17622 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.797+0000 7f86edda1b80 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.797+0000 7f86edda1b80 20 client.17622 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.797+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556cebb16070 0x556cebb18480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.797+0000 7f86edda1b80 4 client.17622 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.797+0000 7f86edda1b80 1 -- 172.21.15.62:0/3219899646 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ceba4af30 con 0x556cebb16070 2023-07-14T16:23:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.797+0000 7f86d37fe700 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556cebb16070 0x556cebb18480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.798+0000 7f86d37fe700 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556cebb16070 0x556cebb18480 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964880188a83f647 server_cookie=6d1e26e3789f606f in_seq=0 out_seq=0 2023-07-14T16:23:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.798+0000 7f86b37fe700 10 client.17622 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86b37fe700 1 -- 172.21.15.62:0/3219899646 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ceba4af30 con 0x556cebb16070 2023-07-14T16:23:06.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86b37fe700 10 client.17622 handle_command_reply: tid=0 2023-07-14T16:23:06.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 client.17622 shutdown 2023-07-14T16:23:06.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 -- 172.21.15.62:0/3219899646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb007a0 msgr2=0x556cebb0a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb007a0 0x556cebb0a3c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f86c4011ac0 tx=0x7f86c4009710 comp rx=0 tx=0).stop 2023-07-14T16:23:06.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 -- 172.21.15.62:0/3219899646 shutdown_connections 2023-07-14T16:23:06.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556cebb16070 0x556cebb18480 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb01eb0 0x556cebb11940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb007a0 0x556cebb0a3c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3219899646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb0a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 -- 172.21.15.62:0/3219899646 >> 172.21.15.62:0/3219899646 conn(0x556cebaf6070 msgr2=0x556cebafab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86b37fe700 0 client.17622 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:06.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 -- 172.21.15.62:0/3219899646 shutdown_connections 2023-07-14T16:23:06.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.799+0000 7f86edda1b80 1 -- 172.21.15.62:0/3219899646 wait complete. 2023-07-14T16:23:06.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.800+0000 7f86edda1b80 20 client.17622 trim_cache size 0 max 16384 2023-07-14T16:23:06.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.826+0000 7f86edda1b80 1 Processor -- start 2023-07-14T16:23:06.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.826+0000 7f86edda1b80 1 -- start start 2023-07-14T16:23:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.826+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb04d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.826+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.826+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.826+0000 7f86edda1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cebafc330 con 0x556cebb01eb0 2023-07-14T16:23:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.826+0000 7f86edda1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cebb15bb0 con 0x556cebb007a0 2023-07-14T16:23:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.826+0000 7f86edda1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cebb09840 con 0x556cebb010f0 2023-07-14T16:23:06.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d899c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d899c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34780/0 (socket says 172.21.15.62:34780) 2023-07-14T16:23:06.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d899c700 1 -- 172.21.15.62:0/2544096959 learned_addr learned my addr 172.21.15.62:0/2544096959 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d37fe700 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d3fff700 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb04d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d37fe700 1 -- 172.21.15.62:0/2544096959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 msgr2=0x556cebb04d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d37fe700 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb04d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d37fe700 1 -- 172.21.15.62:0/2544096959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 msgr2=0x556cebb052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d37fe700 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d37fe700 1 -- 172.21.15.62:0/2544096959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ceb8a62d0 con 0x556cebb010f0 2023-07-14T16:23:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.827+0000 7f86d899c700 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.828+0000 7f86d37fe700 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb11900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ca740 tx=0x7f86c80ca610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d74bd6240a83bfbc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.828+0000 7f86baffd700 1 -- 172.21.15.62:0/2544096959 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80a24a0 con 0x556cebb010f0 2023-07-14T16:23:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.828+0000 7f86baffd700 1 -- 172.21.15.62:0/2544096959 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c80ca850 con 0x556cebb010f0 2023-07-14T16:23:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.828+0000 7f86baffd700 1 -- 172.21.15.62:0/2544096959 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80a2270 con 0x556cebb010f0 2023-07-14T16:23:06.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 -- 172.21.15.62:0/2544096959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 msgr2=0x556cebb11900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb11900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ca740 tx=0x7f86c80ca610 comp rx=0 tx=0).stop 2023-07-14T16:23:06.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 -- 172.21.15.62:0/2544096959 shutdown_connections 2023-07-14T16:23:06.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb01eb0 0x556cebb052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb04d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 --2- 172.21.15.62:0/2544096959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb010f0 0x556cebb11900 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 -- 172.21.15.62:0/2544096959 >> 172.21.15.62:0/2544096959 conn(0x556cebaf6070 msgr2=0x556cebb08240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 -- 172.21.15.62:0/2544096959 shutdown_connections 2023-07-14T16:23:06.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 -- 172.21.15.62:0/2544096959 wait complete. 2023-07-14T16:23:06.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.829+0000 7f86edda1b80 1 -- start start 2023-07-14T16:23:06.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86edda1b80 1 Processor -- start 2023-07-14T16:23:06.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb0e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb010f0 0x556cebb0ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86edda1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb01eb0 0x556cebb69ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86edda1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cebb11a70 con 0x556cebb010f0 2023-07-14T16:23:06.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86edda1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cebb11be0 con 0x556cebb007a0 2023-07-14T16:23:06.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86edda1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cebb6a420 con 0x556cebb01eb0 2023-07-14T16:23:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86edda1b80 10 client.0 _ll_register_callbacks cb 0x7f86c8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb010f0 0x556cebb0ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb0e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb010f0 0x556cebb0ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34790/0 (socket says 172.21.15.62:34790) 2023-07-14T16:23:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86d37fe700 1 -- 172.21.15.62:0/3198883635 learned_addr learned my addr 172.21.15.62:0/3198883635 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.830+0000 7f86d899c700 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb01eb0 0x556cebb69ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86d3fff700 1 -- 172.21.15.62:0/3198883635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb01eb0 msgr2=0x556cebb69ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86d3fff700 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb01eb0 0x556cebb69ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86d3fff700 1 -- 172.21.15.62:0/3198883635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb010f0 msgr2=0x556cebb0ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86d3fff700 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb010f0 0x556cebb0ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86d3fff700 1 -- 172.21.15.62:0/3198883635 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d41086e0 con 0x556cebb007a0 2023-07-14T16:23:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86d37fe700 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb010f0 0x556cebb0ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86d3fff700 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb0e510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f86c4005cc0 tx=0x7f86c4004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd15b542043c5e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:23:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.831+0000 7f86b37fe700 1 -- 172.21.15.62:0/3198883635 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4003890 con 0x556cebb007a0 2023-07-14T16:23:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.832+0000 7f86edda1b80 1 -- 172.21.15.62:0/3198883635 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c40044c0 con 0x556cebb007a0 2023-07-14T16:23:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.832+0000 7f86b37fe700 1 -- 172.21.15.62:0/3198883635 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86c401a040 con 0x556cebb007a0 2023-07-14T16:23:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.832+0000 7f86b37fe700 1 -- 172.21.15.62:0/3198883635 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c40039f0 con 0x556cebb007a0 2023-07-14T16:23:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.832+0000 7f86edda1b80 10 client.27296 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.832+0000 7f86edda1b80 1 -- 172.21.15.62:0/3198883635 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f86d419fdf0 con 0x556cebb007a0 2023-07-14T16:23:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.832+0000 7f86b37fe700 1 -- 172.21.15.62:0/3198883635 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86c40044c0 con 0x556cebb007a0 2023-07-14T16:23:06.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.833+0000 7f86b37fe700 1 -- 172.21.15.62:0/3198883635 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86c4004b60 con 0x556cebb007a0 2023-07-14T16:23:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.833+0000 7f86edda1b80 10 client.27296 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.833+0000 7f86edda1b80 10 client.27296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.833+0000 7f86edda1b80 20 client.27296 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.833+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556cebb6b8e0 0x556cebb6dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.833+0000 7f86edda1b80 4 client.27296 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:23:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.833+0000 7f86edda1b80 1 -- 172.21.15.62:0/3198883635 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556ceba0d0d0 con 0x556cebb6b8e0 2023-07-14T16:23:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.833+0000 7f86d37fe700 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556cebb6b8e0 0x556cebb6dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.834+0000 7f86d37fe700 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556cebb6b8e0 0x556cebb6dcf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b8586770894e0f server_cookie=218eb29738a38a5e in_seq=0 out_seq=0 2023-07-14T16:23:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.834+0000 7f86b37fe700 10 client.27296 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86b37fe700 1 -- 172.21.15.62:0/3198883635 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556ceba0d0d0 con 0x556cebb6b8e0 2023-07-14T16:23:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86b37fe700 10 client.27296 handle_command_reply: tid=0 2023-07-14T16:23:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 client.27296 shutdown 2023-07-14T16:23:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 -- 172.21.15.62:0/3198883635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 msgr2=0x556cebb0e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb0e510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f86c4005cc0 tx=0x7f86c4004170 comp rx=0 tx=0).stop 2023-07-14T16:23:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 -- 172.21.15.62:0/3198883635 shutdown_connections 2023-07-14T16:23:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cebb01eb0 0x556cebb69ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556cebb6b8e0 0x556cebb6dcf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cebb007a0 0x556cebb0e510 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 --2- 172.21.15.62:0/3198883635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cebb010f0 0x556cebb0ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86b37fe700 0 client.27296 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.835+0000 7f86edda1b80 1 -- 172.21.15.62:0/3198883635 >> 172.21.15.62:0/3198883635 conn(0x556cebaf6070 msgr2=0x556cebb08240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.836+0000 7f86edda1b80 1 -- 172.21.15.62:0/3198883635 shutdown_connections 2023-07-14T16:23:06.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.836+0000 7f86edda1b80 1 -- 172.21.15.62:0/3198883635 wait complete. 2023-07-14T16:23:06.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.836+0000 7f86edda1b80 20 client.27296 trim_cache size 0 max 16384 2023-07-14T16:23:06.818 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:23:06.818 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:23:06.818 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "fc4caebe-b23a-4d7a-b04b-dd469bbfe5c2", 2023-07-14T16:23:06.819 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:23:06.819 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:23:06.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.836+0000 7f86d919d700 1 -- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86bc0a11f0 msgr2=0x7f86bc0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.836+0000 7f86d919d700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86bc0a11f0 0x7f86bc0a36a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009d00 tx=0x7f86c40116c0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.836+0000 7f86d919d700 1 -- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4102d70 msgr2=0x7f86d40fe400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:06.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.836+0000 7f86d919d700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4102d70 0x7f86d40fe400 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9a0 tx=0x7f86c800ecb0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.837+0000 7f86d919d700 1 -- 172.21.15.62:0/553537133 shutdown_connections 2023-07-14T16:23:06.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.837+0000 7f86d919d700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86bc0a11f0 0x7f86bc0a36a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.837+0000 7f86d919d700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d4105690 0x7f86d40fe940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.837+0000 7f86d919d700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d4100450 0x7f86d40ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.837+0000 7f86d919d700 1 --2- 172.21.15.62:0/553537133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86d4102d70 0x7f86d40fe400 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:06.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.837+0000 7f86d919d700 1 -- 172.21.15.62:0/553537133 >> 172.21.15.62:0/553537133 conn(0x7f86d40f9ed0 msgr2=0x7f86d41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:06.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.837+0000 7f86d919d700 1 -- 172.21.15.62:0/553537133 shutdown_connections 2023-07-14T16:23:06.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:06.837+0000 7f86d919d700 1 -- 172.21.15.62:0/553537133 wait complete. 2023-07-14T16:23:06.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:06 smithi062 ceph-mon[95543]: pgmap v581: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:23:06.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:23:07.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:06 smithi164 ceph-mon[97427]: pgmap v581: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:23:07.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.099+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1638240639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8103890 msgr2=0x7f4ea810a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.099+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1638240639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8103890 0x7f4ea810a150 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000b0a0 tx=0x7f4ea000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.100+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1638240639 shutdown_connections 2023-07-14T16:23:07.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.100+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1638240639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8103890 0x7f4ea810a150 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.100+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1638240639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ea8101d10 0x7f4ea8102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.100+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1638240639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ea8102f00 0x7f4ea8103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.100+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1638240639 >> 172.21.15.62:0/1638240639 conn(0x7f4ea80fd350 msgr2=0x7f4ea80ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:07.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.100+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1638240639 shutdown_connections 2023-07-14T16:23:07.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.100+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1638240639 wait complete. 2023-07-14T16:23:07.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.101+0000 7f4ead1ff700 1 Processor -- start 2023-07-14T16:23:07.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.101+0000 7f4ead1ff700 1 -- start start 2023-07-14T16:23:07.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.101+0000 7f4ead1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ea8101d10 0x7f4ea8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.101+0000 7f4ead1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8102f00 0x7f4ea8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.101+0000 7f4ead1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ea8103890 0x7f4ea819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.101+0000 7f4ead1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea819f5e0 con 0x7f4ea8102f00 2023-07-14T16:23:07.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.101+0000 7f4ead1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea819f720 con 0x7f4ea8101d10 2023-07-14T16:23:07.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.101+0000 7f4ead1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea819f8b0 con 0x7f4ea8103890 2023-07-14T16:23:07.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8102f00 0x7f4ea8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8102f00 0x7f4ea8197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34818/0 (socket says 172.21.15.62:34818) 2023-07-14T16:23:07.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ea8101d10 0x7f4ea8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea659c700 1 -- 172.21.15.62:0/1235565170 learned_addr learned my addr 172.21.15.62:0/1235565170 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:07.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea759e700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ea8103890 0x7f4ea819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea6d9d700 1 -- 172.21.15.62:0/1235565170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ea8103890 msgr2=0x7f4ea819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ea8103890 0x7f4ea819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea6d9d700 1 -- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8102f00 msgr2=0x7f4ea8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8102f00 0x7f4ea8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea6d9d700 1 -- 172.21.15.62:0/1235565170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0009d00 con 0x7f4ea8101d10 2023-07-14T16:23:07.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.102+0000 7f4ea659c700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8102f00 0x7f4ea8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:07.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.103+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ea8101d10 0x7f4ea8197a60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800d720 tx=0x7f4e9800da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:07.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.103+0000 7f4e8ffff700 1 -- 172.21.15.62:0/1235565170 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800f690 con 0x7f4ea8101d10 2023-07-14T16:23:07.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.103+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1235565170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea819fb90 con 0x7f4ea8101d10 2023-07-14T16:23:07.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.103+0000 7f4e8ffff700 1 -- 172.21.15.62:0/1235565170 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e9800fc80 con 0x7f4ea8101d10 2023-07-14T16:23:07.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.103+0000 7f4e8ffff700 1 -- 172.21.15.62:0/1235565170 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98010d90 con 0x7f4ea8101d10 2023-07-14T16:23:07.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.103+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1235565170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea81a00f0 con 0x7f4ea8101d10 2023-07-14T16:23:07.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.107+0000 7f4e8ffff700 1 -- 172.21.15.62:0/1235565170 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4e980104f0 con 0x7f4ea8101d10 2023-07-14T16:23:07.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.108+0000 7f4ec0b08b80 1 Processor -- start 2023-07-14T16:23:07.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4ec0b08b80 1 -- start start 2023-07-14T16:23:07.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4e8ffff700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e900a1150 0x7f4e900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0a4d0 0x56225ee0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0ae20 0x56225ee0b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ed0fcb0 con 0x56225ee0bc70 2023-07-14T16:23:07.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ed62a40 con 0x56225ee0a4d0 2023-07-14T16:23:07.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ed5bed0 con 0x56225ee0ae20 2023-07-14T16:23:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4e8ffff700 1 -- 172.21.15.62:0/1235565170 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4e980cc550 con 0x7f4ea8101d10 2023-07-14T16:23:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.109+0000 7f4ea759e700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e900a1150 0x7f4e900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34832/0 (socket says 172.21.15.62:34832) 2023-07-14T16:23:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea759e700 1 -- 172.21.15.62:0/473657053 learned_addr learned my addr 172.21.15.62:0/473657053 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea659c700 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0ae20 0x56225ee0b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0a4d0 0x56225ee0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea659c700 1 -- 172.21.15.62:0/473657053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0a4d0 msgr2=0x56225ee0a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea659c700 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0a4d0 0x56225ee0a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea659c700 1 -- 172.21.15.62:0/473657053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 msgr2=0x56225ee0c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:07.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea659c700 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.110+0000 7f4ea659c700 1 -- 172.21.15.62:0/473657053 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56225eb89de0 con 0x56225ee0ae20 2023-07-14T16:23:07.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.111+0000 7f4ea659c700 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0ae20 0x56225ee0b6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c010550 tx=0x7f4e9c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb6bf0d8e97742c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:07.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.111+0000 7f4ea759e700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e900a1150 0x7f4e900a3600 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000d010 tx=0x7f4ea000cf00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:07.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.111+0000 7f4e8d7fa700 1 -- 172.21.15.62:0/473657053 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c011640 con 0x56225ee0ae20 2023-07-14T16:23:07.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.111+0000 7f4e8d7fa700 1 -- 172.21.15.62:0/473657053 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e9c015070 con 0x56225ee0ae20 2023-07-14T16:23:07.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.111+0000 7f4e8d7fa700 1 -- 172.21.15.62:0/473657053 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0198f0 con 0x56225ee0ae20 2023-07-14T16:23:07.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/473657053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0ae20 msgr2=0x56225ee0b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0ae20 0x56225ee0b6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c010550 tx=0x7f4e9c010860 comp rx=0 tx=0).stop 2023-07-14T16:23:07.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/473657053 shutdown_connections 2023-07-14T16:23:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0a4d0 0x56225ee0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/473657053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0ae20 0x56225ee0b6e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/473657053 >> 172.21.15.62:0/473657053 conn(0x56225edffda0 msgr2=0x56225ee051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/473657053 shutdown_connections 2023-07-14T16:23:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/473657053 wait complete. 2023-07-14T16:23:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.112+0000 7f4ec0b08b80 1 -- start start 2023-07-14T16:23:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ec0b08b80 1 Processor -- start 2023-07-14T16:23:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0ae20 0x56225ee145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0bc70 0x56225ee1b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ecfbac0 con 0x56225ee0ae20 2023-07-14T16:23:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ecfb7c0 con 0x56225ee0bc70 2023-07-14T16:23:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ebdf920 con 0x56225ee0a4d0 2023-07-14T16:23:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ec0b08b80 10 client.0 _ll_register_callbacks cb 0x56225ecf9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ea659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0ae20 0x56225ee145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ea659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0ae20 0x56225ee145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34846/0 (socket says 172.21.15.62:34846) 2023-07-14T16:23:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.113+0000 7f4ea659c700 1 -- 172.21.15.62:0/2832843913 learned_addr learned my addr 172.21.15.62:0/2832843913 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea759e700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0bc70 0x56225ee1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea659c700 1 -- 172.21.15.62:0/2832843913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 msgr2=0x56225ee140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea659c700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea659c700 1 -- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0bc70 msgr2=0x56225ee1b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea659c700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0bc70 0x56225ee1b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea659c700 1 -- 172.21.15.62:0/2832843913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c00bda0 con 0x56225ee0ae20 2023-07-14T16:23:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea759e700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0bc70 0x56225ee1b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4ea659c700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0ae20 0x56225ee145f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005c50 tx=0x7f4e9c005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3c0020b36c6e52 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.114+0000 7f4e8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.115+0000 7f4e8aff5700 1 -- 172.21.15.62:0/2832843913 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c005630 con 0x56225ee0ae20 2023-07-14T16:23:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.115+0000 7f4e8aff5700 1 -- 172.21.15.62:0/2832843913 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4e9c004030 con 0x56225ee0ae20 2023-07-14T16:23:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.115+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/2832843913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56225ecb54f0 con 0x56225ee0ae20 2023-07-14T16:23:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.115+0000 7f4e8aff5700 1 -- 172.21.15.62:0/2832843913 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e9c0041f0 con 0x56225ee0ae20 2023-07-14T16:23:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.115+0000 7f4ec0b08b80 10 client.17640 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.115+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/2832843913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56225eb72c60 con 0x56225ee0ae20 2023-07-14T16:23:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4e8aff5700 1 -- 172.21.15.62:0/2832843913 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e9c031020 con 0x56225ee0ae20 2023-07-14T16:23:07.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4e8aff5700 1 -- 172.21.15.62:0/2832843913 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e9c029060 con 0x56225ee0ae20 2023-07-14T16:23:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4ec0b08b80 10 client.17640 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4ec0b08b80 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4ec0b08b80 20 client.17640 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56225ee1fda0 0x56225ee221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4ec0b08b80 4 client.17640 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/2832843913 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56225ed54eb0 con 0x56225ee1fda0 2023-07-14T16:23:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.116+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56225ee1fda0 0x56225ee221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.117+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56225ee1fda0 0x56225ee221b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd6e61123dbf15e server_cookie=460325a2d92c56e2 in_seq=0 out_seq=0 2023-07-14T16:23:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.117+0000 7f4e8aff5700 10 client.17640 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4e8aff5700 1 -- 172.21.15.62:0/2832843913 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56225ed54eb0 con 0x56225ee1fda0 2023-07-14T16:23:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4e8aff5700 10 client.17640 handle_command_reply: tid=0 2023-07-14T16:23:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 client.17640 shutdown 2023-07-14T16:23:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/2832843913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0ae20 msgr2=0x56225ee145f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0ae20 0x56225ee145f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005c50 tx=0x7f4e9c005150 comp rx=0 tx=0).stop 2023-07-14T16:23:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/2832843913 shutdown_connections 2023-07-14T16:23:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56225ee1fda0 0x56225ee221b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0bc70 0x56225ee1b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/2832843913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0ae20 0x56225ee145f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/2832843913 >> 172.21.15.62:0/2832843913 conn(0x56225edffda0 msgr2=0x56225ee048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4e8aff5700 0 client.17640 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/2832843913 shutdown_connections 2023-07-14T16:23:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.118+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/2832843913 wait complete. 2023-07-14T16:23:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.119+0000 7f4ec0b08b80 20 client.17640 trim_cache size 0 max 16384 2023-07-14T16:23:07.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.139+0000 7f4ec0b08b80 1 Processor -- start 2023-07-14T16:23:07.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.139+0000 7f4ec0b08b80 1 -- start start 2023-07-14T16:23:07.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.139+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee0ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.139+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 0x56225ee1b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.139+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.139+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ee14d30 con 0x56225ee0bc70 2023-07-14T16:23:07.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.139+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ee11cb0 con 0x56225ee0ae20 2023-07-14T16:23:07.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.139+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ee1faa0 con 0x56225ee0a4d0 2023-07-14T16:23:07.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34862/0 (socket says 172.21.15.62:34862) 2023-07-14T16:23:07.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 -- 172.21.15.62:0/756528489 learned_addr learned my addr 172.21.15.62:0/756528489 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee0ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea659c700 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 0x56225ee1b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 -- 172.21.15.62:0/756528489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 msgr2=0x56225ee0ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee0ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 -- 172.21.15.62:0/756528489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 msgr2=0x56225ee1b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 0x56225ee1b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 -- 172.21.15.62:0/756528489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56225ebb0250 con 0x56225ee0bc70 2023-07-14T16:23:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.140+0000 7f4ea759e700 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0ef70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009610 tx=0x7f4ea0028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0691f6749b1c12 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.141+0000 7f4e8dffb700 1 -- 172.21.15.62:0/756528489 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0025070 con 0x56225ee0bc70 2023-07-14T16:23:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.141+0000 7f4e8dffb700 1 -- 172.21.15.62:0/756528489 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ea00209a0 con 0x56225ee0bc70 2023-07-14T16:23:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.141+0000 7f4e8dffb700 1 -- 172.21.15.62:0/756528489 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea002d8b0 con 0x56225ee0bc70 2023-07-14T16:23:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/756528489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 msgr2=0x56225ee0ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0ef70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009610 tx=0x7f4ea0028000 comp rx=0 tx=0).stop 2023-07-14T16:23:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/756528489 shutdown_connections 2023-07-14T16:23:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee0ef70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee0ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/756528489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 0x56225ee1b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/756528489 >> 172.21.15.62:0/756528489 conn(0x56225edffda0 msgr2=0x56225ee21db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/756528489 shutdown_connections 2023-07-14T16:23:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/756528489 wait complete. 2023-07-14T16:23:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 -- start start 2023-07-14T16:23:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.142+0000 7f4ec0b08b80 1 Processor -- start 2023-07-14T16:23:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee1b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 0x56225ee17c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ec0b08b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ee1ba10 con 0x56225ee0bc70 2023-07-14T16:23:07.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ee18740 con 0x56225ee0ae20 2023-07-14T16:23:07.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ec0b08b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225ee18900 con 0x56225ee0a4d0 2023-07-14T16:23:07.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ec0b08b80 10 client.0 _ll_register_callbacks cb 0x7f4ea0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:07.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ea759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ea759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee181d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34874/0 (socket says 172.21.15.62:34874) 2023-07-14T16:23:07.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ea759e700 1 -- 172.21.15.62:0/1841891276 learned_addr learned my addr 172.21.15.62:0/1841891276 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:07.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.143+0000 7f4ea659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 0x56225ee17c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee1b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4ea759e700 1 -- 172.21.15.62:0/1841891276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 msgr2=0x56225ee1b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:07.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4ea759e700 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee1b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4ea759e700 1 -- 172.21.15.62:0/1841891276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 msgr2=0x56225ee17c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:07.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4ea759e700 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 0x56225ee17c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4ea759e700 1 -- 172.21.15.62:0/1841891276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea810aa80 con 0x56225ee0bc70 2023-07-14T16:23:07.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4ea759e700 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee181d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00060b0 tx=0x7f4ea00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8456d2842ed4b972 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:07.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4e8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:07.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.144+0000 7f4e8aff5700 1 -- 172.21.15.62:0/1841891276 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0025040 con 0x56225ee0bc70 2023-07-14T16:23:07.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/1841891276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e9c015070 con 0x56225ee0bc70 2023-07-14T16:23:07.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4e8aff5700 1 -- 172.21.15.62:0/1841891276 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ea002d7f0 con 0x56225ee0bc70 2023-07-14T16:23:07.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4e8aff5700 1 -- 172.21.15.62:0/1841891276 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ea0020e60 con 0x56225ee0bc70 2023-07-14T16:23:07.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4ec0b08b80 10 client.17646 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:07.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/1841891276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56225eb882b0 con 0x56225ee0bc70 2023-07-14T16:23:07.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4e8aff5700 1 -- 172.21.15.62:0/1841891276 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e9c015070 con 0x56225ee0bc70 2023-07-14T16:23:07.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4e8aff5700 1 -- 172.21.15.62:0/1841891276 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4ea0038b30 con 0x56225ee0bc70 2023-07-14T16:23:07.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4ec0b08b80 10 client.17646 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:07.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4ec0b08b80 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:07.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4ec0b08b80 20 client.17646 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:07.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.145+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56225ee75320 0x56225ee77730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.146+0000 7f4ec0b08b80 4 client.17646 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:23:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.146+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/1841891276 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56225ed17050 con 0x56225ee75320 2023-07-14T16:23:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.146+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56225ee75320 0x56225ee77730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.146+0000 7f4ea6d9d700 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56225ee75320 0x56225ee77730 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95efd88193213f46 server_cookie=c373c1b0bb96a77e in_seq=0 out_seq=0 2023-07-14T16:23:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.146+0000 7f4e8aff5700 10 client.17646 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4e8aff5700 1 -- 172.21.15.62:0/1841891276 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56225ed17050 con 0x56225ee75320 2023-07-14T16:23:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4e8aff5700 10 client.17646 handle_command_reply: tid=0 2023-07-14T16:23:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 client.17646 shutdown 2023-07-14T16:23:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/1841891276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 msgr2=0x56225ee181d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee181d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00060b0 tx=0x7f4ea00099e0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/1841891276 shutdown_connections 2023-07-14T16:23:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56225ee0bc70 0x56225ee181d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56225ee0a4d0 0x56225ee1b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56225ee75320 0x56225ee77730 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 --2- 172.21.15.62:0/1841891276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56225ee0ae20 0x56225ee17c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4e8aff5700 0 client.17646 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.147+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/1841891276 >> 172.21.15.62:0/1841891276 conn(0x56225edffda0 msgr2=0x56225ee0ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.148+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/1841891276 shutdown_connections 2023-07-14T16:23:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.148+0000 7f4ec0b08b80 1 -- 172.21.15.62:0/1841891276 wait complete. 2023-07-14T16:23:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.148+0000 7f4ec0b08b80 20 client.17646 trim_cache size 0 max 16384 2023-07-14T16:23:07.130 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:23:07.131 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:23:07.131 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:23:07.131 INFO:teuthology.orchestra.run.smithi062.stdout: "fc4caebe-b23a-4d7a-b04b-dd469bbfe5c2": { 2023-07-14T16:23:07.131 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:23:07.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "fc4caebe-b23a-4d7a-b04b-dd469bbfe5c2", 2023-07-14T16:23:07.131 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:23:07.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:23:07.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:23:07.132 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:23:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e900a1150 msgr2=0x7f4e900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e900a1150 0x7f4e900a3600 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000d010 tx=0x7f4ea000cf00 comp rx=0 tx=0).stop 2023-07-14T16:23:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1235565170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ea8101d10 msgr2=0x7f4ea8197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ea8101d10 0x7f4ea8197a60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800d720 tx=0x7f4e9800da30 comp rx=0 tx=0).stop 2023-07-14T16:23:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1235565170 shutdown_connections 2023-07-14T16:23:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ea8103890 0x7f4ea819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e900a1150 0x7f4e900a3600 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ea8101d10 0x7f4ea8197a60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 --2- 172.21.15.62:0/1235565170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea8102f00 0x7f4ea8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1235565170 >> 172.21.15.62:0/1235565170 conn(0x7f4ea80fd350 msgr2=0x7f4ea81063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:07.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1235565170 shutdown_connections 2023-07-14T16:23:07.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:07.149+0000 7f4ead1ff700 1 -- 172.21.15.62:0/1235565170 wait complete. 2023-07-14T16:23:07.144 INFO:tasks.cephfs.filesystem:scrub status for tag:fc4caebe-b23a-4d7a-b04b-dd469bbfe5c2 - {'path': '/', 'tag': 'fc4caebe-b23a-4d7a-b04b-dd469bbfe5c2', 'options': 'recursive,force'} 2023-07-14T16:23:07.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:06 smithi081 ceph-mon[127795]: pgmap v581: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:23:07.152 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:23:06 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:23:06.844+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v165>,<0x10000000003/client.0 v166>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v178>,<0x10000000001/sv_0 v192>,<0x10000000000/_nogroup v186>,<0x1/volumes v176>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:23:08.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:23:08.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:23:08.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:23:08.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:23:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:23:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:07 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:23:08.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:23:08.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:23:08.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:23:08.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:23:08.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:07 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:23:08.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:23:08.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:23:08.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:23:08.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:23:08.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:08.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:07 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:09.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:08 smithi164 ceph-mon[97427]: pgmap v582: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:23:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:08 smithi062 ceph-mon[95543]: pgmap v582: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:23:09.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:08 smithi081 ceph-mon[127795]: pgmap v582: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:23:11.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:10 smithi164 ceph-mon[97427]: pgmap v583: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:23:11.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:10 smithi062 ceph-mon[95543]: pgmap v583: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:23:11.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:10 smithi081 ceph-mon[127795]: pgmap v583: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:23:13.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:12 smithi164 ceph-mon[97427]: pgmap v584: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:13.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:12 smithi062 ceph-mon[95543]: pgmap v584: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:13.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:12 smithi081 ceph-mon[127795]: pgmap v584: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:14.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:13 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:23:14.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:13 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:23:14.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:13 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:23:14.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:13 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:23:14.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:13 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:23:14.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:13 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:23:15.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:14 smithi164 ceph-mon[97427]: pgmap v585: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:23:15.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:14 smithi062 ceph-mon[95543]: pgmap v585: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:23:15.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:14 smithi081 ceph-mon[127795]: pgmap v585: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:23:17.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:16 smithi164 ceph-mon[97427]: pgmap v586: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:23:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:16 smithi062 ceph-mon[95543]: pgmap v586: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:23:17.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:16 smithi081 ceph-mon[127795]: pgmap v586: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:23:19.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:18 smithi164 ceph-mon[97427]: pgmap v587: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:23:19.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:18 smithi062 ceph-mon[95543]: pgmap v587: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:23:19.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:18 smithi081 ceph-mon[127795]: pgmap v587: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 -- 172.21.15.62:0/3432608819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1380744f0 msgr2=0x7ff138072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 --2- 172.21.15.62:0/3432608819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1380744f0 0x7ff138072940 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff128009990 tx=0x7ff128009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 -- 172.21.15.62:0/3432608819 shutdown_connections 2023-07-14T16:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 --2- 172.21.15.62:0/3432608819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1381077d0 0x7ff138109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 --2- 172.21.15.62:0/3432608819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1380744f0 0x7ff138072940 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 --2- 172.21.15.62:0/3432608819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff138072f10 0x7ff138073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 -- 172.21.15.62:0/3432608819 >> 172.21.15.62:0/3432608819 conn(0x7ff1380faf80 msgr2=0x7ff1380fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 -- 172.21.15.62:0/3432608819 shutdown_connections 2023-07-14T16:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.401+0000 7ff13d59d700 1 -- 172.21.15.62:0/3432608819 wait complete. 2023-07-14T16:23:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.402+0000 7ff13d59d700 1 Processor -- start 2023-07-14T16:23:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.402+0000 7ff13d59d700 1 -- start start 2023-07-14T16:23:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff13d59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138072f10 0x7ff138197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff13d59d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1380744f0 0x7ff138197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff13d59d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1381077d0 0x7ff13819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff13d59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13819f310 con 0x7ff138072f10 2023-07-14T16:23:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff13d59d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13819f450 con 0x7ff1380744f0 2023-07-14T16:23:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff13d59d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13819f5e0 con 0x7ff1381077d0 2023-07-14T16:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff137fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138072f10 0x7ff138197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff137fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138072f10 0x7ff138197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56016/0 (socket says 172.21.15.62:56016) 2023-07-14T16:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff137fff700 1 -- 172.21.15.62:0/2905847978 learned_addr learned my addr 172.21.15.62:0/2905847978 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff1377fe700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1380744f0 0x7ff138197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1381077d0 0x7ff13819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff137fff700 1 -- 172.21.15.62:0/2905847978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1381077d0 msgr2=0x7ff13819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff137fff700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1381077d0 0x7ff13819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff137fff700 1 -- 172.21.15.62:0/2905847978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1380744f0 msgr2=0x7ff138197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff137fff700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1380744f0 0x7ff138197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff137fff700 1 -- 172.21.15.62:0/2905847978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280095f0 con 0x7ff138072f10 2023-07-14T16:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.403+0000 7ff1377fe700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1380744f0 0x7ff138197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.404+0000 7ff137fff700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138072f10 0x7ff138197790 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff1280059d0 tx=0x7ff1280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.404+0000 7ff1357fa700 1 -- 172.21.15.62:0/2905847978 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12801d070 con 0x7ff138072f10 2023-07-14T16:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.404+0000 7ff13d59d700 1 -- 172.21.15.62:0/2905847978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13819f860 con 0x7ff138072f10 2023-07-14T16:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.404+0000 7ff1357fa700 1 -- 172.21.15.62:0/2905847978 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff12800f460 con 0x7ff138072f10 2023-07-14T16:23:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.404+0000 7ff1357fa700 1 -- 172.21.15.62:0/2905847978 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128021670 con 0x7ff138072f10 2023-07-14T16:23:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.404+0000 7ff13d59d700 1 -- 172.21.15.62:0/2905847978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13819fd60 con 0x7ff138072f10 2023-07-14T16:23:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.409+0000 7ff1357fa700 1 -- 172.21.15.62:0/2905847978 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff12800f5d0 con 0x7ff138072f10 2023-07-14T16:23:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.409+0000 7ff152106b80 1 Processor -- start 2023-07-14T16:23:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.410+0000 7ff152106b80 1 -- start start 2023-07-14T16:23:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.410+0000 7ff152106b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.410+0000 7ff152106b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.410+0000 7ff152106b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.410+0000 7ff152106b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a4a5c50 con 0x55995a3fc6b0 2023-07-14T16:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.410+0000 7ff152106b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a4f89e0 con 0x55995a5a17e0 2023-07-14T16:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.410+0000 7ff152106b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a4f1e70 con 0x55995a5a0a90 2023-07-14T16:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.410+0000 7ff13cd9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff1357fa700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1200a11d0 0x7ff1200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff137fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff137fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:58644/0 (socket says 172.21.15.62:58644) 2023-07-14T16:23:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff137fff700 1 -- 172.21.15.62:0/2419818285 learned_addr learned my addr 172.21.15.62:0/2419818285 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff1377fe700 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff1357fa700 1 -- 172.21.15.62:0/2905847978 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff1280cdd30 con 0x7ff138072f10 2023-07-14T16:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1200a11d0 0x7ff1200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff13cd9c700 1 -- 172.21.15.62:0/2419818285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 msgr2=0x55995a5a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff13cd9c700 1 -- 172.21.15.62:0/2419818285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 msgr2=0x55995a5a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff13cd9c700 1 -- 172.21.15.62:0/2419818285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55995a31fd80 con 0x55995a3fc6b0 2023-07-14T16:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.411+0000 7ff137fff700 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.412+0000 7ff1377fe700 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.412+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a0550 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff130007c60 tx=0x7ff130007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc8d8c977003630 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.412+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1200a11d0 0x7ff1200a3680 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff130009510 tx=0x7ff13001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.412+0000 7ff11e7fc700 1 -- 172.21.15.62:0/2419818285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13000e070 con 0x55995a3fc6b0 2023-07-14T16:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.412+0000 7ff11e7fc700 1 -- 172.21.15.62:0/2419818285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff13000f040 con 0x55995a3fc6b0 2023-07-14T16:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.412+0000 7ff11e7fc700 1 -- 172.21.15.62:0/2419818285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13001b990 con 0x55995a3fc6b0 2023-07-14T16:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 -- 172.21.15.62:0/2419818285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 msgr2=0x55995a5a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a0550 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff130007c60 tx=0x7ff130007f70 comp rx=0 tx=0).stop 2023-07-14T16:23:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 -- 172.21.15.62:0/2419818285 shutdown_connections 2023-07-14T16:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a0550 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 --2- 172.21.15.62:0/2419818285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 -- 172.21.15.62:0/2419818285 >> 172.21.15.62:0/2419818285 conn(0x55995a595d40 msgr2=0x55995a59ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 -- 172.21.15.62:0/2419818285 shutdown_connections 2023-07-14T16:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.413+0000 7ff152106b80 1 -- 172.21.15.62:0/2419818285 wait complete. 2023-07-14T16:23:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.414+0000 7ff152106b80 1 -- start start 2023-07-14T16:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.414+0000 7ff152106b80 1 Processor -- start 2023-07-14T16:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff152106b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff152106b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff152106b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff152106b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a491a60 con 0x55995a3fc6b0 2023-07-14T16:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff152106b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a491760 con 0x55995a5a17e0 2023-07-14T16:23:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff152106b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a3758c0 con 0x55995a5a0a90 2023-07-14T16:23:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff152106b80 10 client.0 _ll_register_callbacks cb 0x55995a48f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff137fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff137fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56026/0 (socket says 172.21.15.62:56026) 2023-07-14T16:23:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff137fff700 1 -- 172.21.15.62:0/767136584 learned_addr learned my addr 172.21.15.62:0/767136584 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff137fff700 1 -- 172.21.15.62:0/767136584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 msgr2=0x55995a5a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff137fff700 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff137fff700 1 -- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 msgr2=0x55995a5b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff137fff700 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff137fff700 1 -- 172.21.15.62:0/767136584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13819fd60 con 0x55995a3fc6b0 2023-07-14T16:23:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.415+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff137fff700 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a9a70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff128005930 tx=0x7ff12809b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab4553bffbfbafb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff1177fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff1177fe700 1 -- 172.21.15.62:0/767136584 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280a10b0 con 0x55995a3fc6b0 2023-07-14T16:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff1177fe700 1 -- 172.21.15.62:0/767136584 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff12809b1e0 con 0x55995a3fc6b0 2023-07-14T16:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff152106b80 1 -- 172.21.15.62:0/767136584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55995a44b490 con 0x55995a3fc6b0 2023-07-14T16:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff1177fe700 1 -- 172.21.15.62:0/767136584 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff12809f210 con 0x55995a3fc6b0 2023-07-14T16:23:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff152106b80 10 client.17658 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff152106b80 1 -- 172.21.15.62:0/767136584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7ff130011040 con 0x55995a3fc6b0 2023-07-14T16:23:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.416+0000 7ff1177fe700 1 -- 172.21.15.62:0/767136584 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55995a44b490 con 0x55995a3fc6b0 2023-07-14T16:23:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.417+0000 7ff1177fe700 1 -- 172.21.15.62:0/767136584 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff12809b4d0 con 0x55995a3fc6b0 2023-07-14T16:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.417+0000 7ff152106b80 10 client.17658 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.417+0000 7ff152106b80 10 client.17658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.417+0000 7ff152106b80 20 client.17658 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.417+0000 7ff152106b80 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55995a5b54b0 0x55995a5b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.417+0000 7ff152106b80 4 client.17658 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.417+0000 7ff152106b80 1 -- 172.21.15.62:0/767136584 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55995a4eae50 con 0x55995a5b54b0 2023-07-14T16:23:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.417+0000 7ff1377fe700 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55995a5b54b0 0x55995a5b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.418+0000 7ff1377fe700 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55995a5b54b0 0x55995a5b7890 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7d00223f37327d server_cookie=4fb2ed8e1949ad65 in_seq=0 out_seq=0 2023-07-14T16:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.418+0000 7ff1177fe700 10 client.17658 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff1177fe700 1 -- 172.21.15.62:0/767136584 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55995a4eae50 con 0x55995a5b54b0 2023-07-14T16:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff1177fe700 10 client.17658 handle_command_reply: tid=0 2023-07-14T16:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 client.17658 shutdown 2023-07-14T16:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 -- 172.21.15.62:0/767136584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 msgr2=0x55995a5a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a9a70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff128005930 tx=0x7ff12809b0e0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 -- 172.21.15.62:0/767136584 shutdown_connections 2023-07-14T16:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55995a5b54b0 0x55995a5b7890 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a3fc6b0 0x55995a5a9a70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 --2- 172.21.15.62:0/767136584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a0a90 0x55995a5a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 -- 172.21.15.62:0/767136584 >> 172.21.15.62:0/767136584 conn(0x55995a595d40 msgr2=0x55995a59a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff1177fe700 0 client.17658 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 -- 172.21.15.62:0/767136584 shutdown_connections 2023-07-14T16:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.419+0000 7ff152106b80 1 -- 172.21.15.62:0/767136584 wait complete. 2023-07-14T16:23:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.420+0000 7ff152106b80 20 client.17658 trim_cache size 0 max 16384 2023-07-14T16:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.437+0000 7ff152106b80 1 Processor -- start 2023-07-14T16:23:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.437+0000 7ff152106b80 1 -- start start 2023-07-14T16:23:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.437+0000 7ff152106b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a3fc6b0 0x55995a5bc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.437+0000 7ff152106b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.437+0000 7ff152106b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.437+0000 7ff152106b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a5660c0 con 0x55995a5a0a90 2023-07-14T16:23:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.437+0000 7ff152106b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a5a84f0 con 0x55995a5a17e0 2023-07-14T16:23:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.437+0000 7ff152106b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a4eae50 con 0x55995a3fc6b0 2023-07-14T16:23:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff1377fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff137fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a3fc6b0 0x55995a5bc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff1377fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56034/0 (socket says 172.21.15.62:56034) 2023-07-14T16:23:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff1377fe700 1 -- 172.21.15.62:0/3074509211 learned_addr learned my addr 172.21.15.62:0/3074509211 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff1377fe700 1 -- 172.21.15.62:0/3074509211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a3fc6b0 msgr2=0x55995a5bc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff1377fe700 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a3fc6b0 0x55995a5bc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff1377fe700 1 -- 172.21.15.62:0/3074509211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 msgr2=0x55995a5a86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff1377fe700 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff1377fe700 1 -- 172.21.15.62:0/3074509211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55995a308c00 con 0x55995a5a0a90 2023-07-14T16:23:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.438+0000 7ff137fff700 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a3fc6b0 0x55995a5bc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.439+0000 7ff1377fe700 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5b0fb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x55995a5b8190 tx=0x7ff12c009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0ea885287c87aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.439+0000 7ff11effd700 1 -- 172.21.15.62:0/3074509211 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c01a520 con 0x55995a5a0a90 2023-07-14T16:23:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.439+0000 7ff11effd700 1 -- 172.21.15.62:0/3074509211 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff12c01ad60 con 0x55995a5a0a90 2023-07-14T16:23:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.439+0000 7ff11effd700 1 -- 172.21.15.62:0/3074509211 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c012450 con 0x55995a5a0a90 2023-07-14T16:23:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.439+0000 7ff152106b80 1 -- 172.21.15.62:0/3074509211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 msgr2=0x55995a5b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.439+0000 7ff152106b80 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5b0fb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x55995a5b8190 tx=0x7ff12c009770 comp rx=0 tx=0).stop 2023-07-14T16:23:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 -- 172.21.15.62:0/3074509211 shutdown_connections 2023-07-14T16:23:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a5a17e0 0x55995a5a86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a3fc6b0 0x55995a5bc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 --2- 172.21.15.62:0/3074509211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5b0fb0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 -- 172.21.15.62:0/3074509211 >> 172.21.15.62:0/3074509211 conn(0x55995a595d40 msgr2=0x55995a59ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 -- 172.21.15.62:0/3074509211 shutdown_connections 2023-07-14T16:23:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 -- 172.21.15.62:0/3074509211 wait complete. 2023-07-14T16:23:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 -- start start 2023-07-14T16:23:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.440+0000 7ff152106b80 1 Processor -- start 2023-07-14T16:23:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff152106b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a3fc6b0 0x55995a5ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff152106b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff152106b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a17e0 0x55995a5b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff152106b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a5b0bf0 con 0x55995a5a0a90 2023-07-14T16:23:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff152106b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a5b0d60 con 0x55995a3fc6b0 2023-07-14T16:23:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff152106b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995a5b0f20 con 0x55995a5a17e0 2023-07-14T16:23:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff152106b80 10 client.0 _ll_register_callbacks cb 0x7ff12800fef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff1377fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff1377fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56040/0 (socket says 172.21.15.62:56040) 2023-07-14T16:23:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff1377fe700 1 -- 172.21.15.62:0/3732892878 learned_addr learned my addr 172.21.15.62:0/3732892878 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff137fff700 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a3fc6b0 0x55995a5ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.441+0000 7ff13cd9c700 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a17e0 0x55995a5b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.442+0000 7ff1377fe700 1 -- 172.21.15.62:0/3732892878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a17e0 msgr2=0x55995a5b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.442+0000 7ff1377fe700 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a17e0 0x55995a5b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.442+0000 7ff1377fe700 1 -- 172.21.15.62:0/3732892878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a3fc6b0 msgr2=0x55995a5ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.442+0000 7ff1377fe700 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a3fc6b0 0x55995a5ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.442+0000 7ff1377fe700 1 -- 172.21.15.62:0/3732892878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff130011040 con 0x55995a5a0a90 2023-07-14T16:23:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.442+0000 7ff1377fe700 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5ad690 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009b80 tx=0x7ff12c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3785acc574357d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.442+0000 7ff1177fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff1177fe700 1 -- 172.21.15.62:0/3732892878 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c01a520 con 0x55995a5a0a90 2023-07-14T16:23:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff1177fe700 1 -- 172.21.15.62:0/3732892878 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff12c017070 con 0x55995a5a0a90 2023-07-14T16:23:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 1 -- 172.21.15.62:0/3732892878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff13000f040 con 0x55995a5a0a90 2023-07-14T16:23:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff1177fe700 1 -- 172.21.15.62:0/3732892878 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff12c003910 con 0x55995a5a0a90 2023-07-14T16:23:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 10 client.17664 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 1 -- 172.21.15.62:0/3732892878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7ff12c009c90 con 0x55995a5a0a90 2023-07-14T16:23:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff1177fe700 1 -- 172.21.15.62:0/3732892878 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff13000f040 con 0x55995a5a0a90 2023-07-14T16:23:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff1177fe700 1 -- 172.21.15.62:0/3732892878 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff12c012c40 con 0x55995a5a0a90 2023-07-14T16:23:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 10 client.17664 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 10 client.17664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 20 client.17664 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55995a60a600 0x55995a60ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 4 client.17664 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:23:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.443+0000 7ff152106b80 1 -- 172.21.15.62:0/3732892878 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55995a4acff0 con 0x55995a60a600 2023-07-14T16:23:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.444+0000 7ff137fff700 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55995a60a600 0x55995a60ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.444+0000 7ff137fff700 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55995a60a600 0x55995a60ca10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4502e50e6e440af server_cookie=d470aea511f45518 in_seq=0 out_seq=0 2023-07-14T16:23:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.444+0000 7ff1177fe700 10 client.17664 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.454+0000 7ff1177fe700 1 -- 172.21.15.62:0/3732892878 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55995a4acff0 con 0x55995a60a600 2023-07-14T16:23:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.454+0000 7ff1177fe700 10 client.17664 handle_command_reply: tid=0 2023-07-14T16:23:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.454+0000 7ff152106b80 1 client.17664 shutdown 2023-07-14T16:23:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 -- 172.21.15.62:0/3732892878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 msgr2=0x55995a5ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5ad690 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009b80 tx=0x7ff12c003e30 comp rx=0 tx=0).stop 2023-07-14T16:23:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 -- 172.21.15.62:0/3732892878 shutdown_connections 2023-07-14T16:23:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55995a5a17e0 0x55995a5b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55995a60a600 0x55995a60ca10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55995a3fc6b0 0x55995a5ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 --2- 172.21.15.62:0/3732892878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55995a5a0a90 0x55995a5ad690 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 -- 172.21.15.62:0/3732892878 >> 172.21.15.62:0/3732892878 conn(0x55995a595d40 msgr2=0x55995a5a86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff1177fe700 0 client.17664 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 -- 172.21.15.62:0/3732892878 shutdown_connections 2023-07-14T16:23:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 1 -- 172.21.15.62:0/3732892878 wait complete. 2023-07-14T16:23:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.455+0000 7ff152106b80 20 client.17664 trim_cache size 0 max 16384 2023-07-14T16:23:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:23:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:23:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:23:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:23:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 -- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1200a11d0 msgr2=0x7ff1200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1200a11d0 0x7ff1200a3680 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff130009510 tx=0x7ff13001d000 comp rx=0 tx=0).stop 2023-07-14T16:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 -- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138072f10 msgr2=0x7ff138197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138072f10 0x7ff138197790 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff1280059d0 tx=0x7ff1280047b0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 -- 172.21.15.62:0/2905847978 shutdown_connections 2023-07-14T16:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1381077d0 0x7ff13819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138072f10 0x7ff138197790 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1200a11d0 0x7ff1200a3680 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 --2- 172.21.15.62:0/2905847978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1380744f0 0x7ff138197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 -- 172.21.15.62:0/2905847978 >> 172.21.15.62:0/2905847978 conn(0x7ff1380faf80 msgr2=0x7ff1380fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 -- 172.21.15.62:0/2905847978 shutdown_connections 2023-07-14T16:23:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:19.456+0000 7ff13d59d700 1 -- 172.21.15.62:0/2905847978 wait complete. 2023-07-14T16:23:21.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:20 smithi164 ceph-mon[97427]: pgmap v588: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:21.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:20 smithi062 ceph-mon[95543]: pgmap v588: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:21.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:20 smithi081 ceph-mon[127795]: pgmap v588: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:22.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:21 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:23:22.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:21 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:23:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:21 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:23:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: pgmap v589: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 24 op/s 2023-07-14T16:23:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:23:23.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:23:23.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:22 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:23:23.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: pgmap v589: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 24 op/s 2023-07-14T16:23:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:23:23.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:23:23.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:22 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:23:23.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: pgmap v589: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 24 op/s 2023-07-14T16:23:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:23:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:23:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:23:23.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:22 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:23:24.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:23 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi081 to 220.9M 2023-07-14T16:23:24.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:23 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi164 to 220.9M 2023-07-14T16:23:24.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:23 smithi164 ceph-mon[97427]: Unable to set osd_memory_target on smithi081 to 231674777: error parsing value: Value '231674777' is below minimum 939524096 2023-07-14T16:23:24.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:23 smithi164 ceph-mon[97427]: Unable to set osd_memory_target on smithi164 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T16:23:24.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:23 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi081 to 220.9M 2023-07-14T16:23:24.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:23 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi164 to 220.9M 2023-07-14T16:23:24.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:23 smithi062 ceph-mon[95543]: Unable to set osd_memory_target on smithi081 to 231674777: error parsing value: Value '231674777' is below minimum 939524096 2023-07-14T16:23:24.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:23 smithi062 ceph-mon[95543]: Unable to set osd_memory_target on smithi164 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T16:23:24.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:23 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi081 to 220.9M 2023-07-14T16:23:24.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:23 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi164 to 220.9M 2023-07-14T16:23:24.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:23 smithi081 ceph-mon[127795]: Unable to set osd_memory_target on smithi081 to 231674777: error parsing value: Value '231674777' is below minimum 939524096 2023-07-14T16:23:24.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:23 smithi081 ceph-mon[127795]: Unable to set osd_memory_target on smithi164 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T16:23:25.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:24 smithi164 ceph-mon[97427]: pgmap v590: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:25.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:24 smithi062 ceph-mon[95543]: pgmap v590: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:25.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:24 smithi081 ceph-mon[127795]: pgmap v590: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:23:27.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:26 smithi164 ceph-mon[97427]: pgmap v591: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:23:27.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:26 smithi062 ceph-mon[95543]: pgmap v591: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:23:27.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:26 smithi081 ceph-mon[127795]: pgmap v591: 97 pgs: 97 active+clean; 5.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:23:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:28 smithi062 ceph-mon[95543]: pgmap v592: 97 pgs: 97 active+clean; 5.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:23:29.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:28 smithi081 ceph-mon[127795]: pgmap v592: 97 pgs: 97 active+clean; 5.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:23:29.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:28 smithi164 ceph-mon[97427]: pgmap v592: 97 pgs: 97 active+clean; 5.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:23:31.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:30 smithi062 ceph-mon[95543]: pgmap v593: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:23:31.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:30 smithi081 ceph-mon[127795]: pgmap v593: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:23:31.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:30 smithi164 ceph-mon[97427]: pgmap v593: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 -- 172.21.15.62:0/2744585650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 msgr2=0x7fd340101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 --2- 172.21.15.62:0/2744585650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 0x7fd340101150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009990 tx=0x7fd32c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 -- 172.21.15.62:0/2744585650 shutdown_connections 2023-07-14T16:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 --2- 172.21.15.62:0/2744585650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd340101690 0x7fd340107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 --2- 172.21.15.62:0/2744585650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3400ffb10 0x7fd3400fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 --2- 172.21.15.62:0/2744585650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 0x7fd340101150 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 -- 172.21.15.62:0/2744585650 >> 172.21.15.62:0/2744585650 conn(0x7fd3400fb130 msgr2=0x7fd3400fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 -- 172.21.15.62:0/2744585650 shutdown_connections 2023-07-14T16:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.629+0000 7fd346fff700 1 -- 172.21.15.62:0/2744585650 wait complete. 2023-07-14T16:23:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.630+0000 7fd346fff700 1 Processor -- start 2023-07-14T16:23:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.630+0000 7fd346fff700 1 -- start start 2023-07-14T16:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd346fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3400ffb10 0x7fd340197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd346fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 0x7fd340197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd346fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd340101690 0x7fd340198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd346fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340199470 con 0x7fd340100d00 2023-07-14T16:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd346fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401995e0 con 0x7fd3400ffb10 2023-07-14T16:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd346fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401997a0 con 0x7fd340101690 2023-07-14T16:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd33ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 0x7fd340197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd33ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 0x7fd340197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52858/0 (socket says 172.21.15.62:52858) 2023-07-14T16:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd33ffff700 1 -- 172.21.15.62:0/2299691795 learned_addr learned my addr 172.21.15.62:0/2299691795 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd344d9b700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3400ffb10 0x7fd340197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd34559c700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd340101690 0x7fd340198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd33ffff700 1 -- 172.21.15.62:0/2299691795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd340101690 msgr2=0x7fd340198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd33ffff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd340101690 0x7fd340198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd33ffff700 1 -- 172.21.15.62:0/2299691795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3400ffb10 msgr2=0x7fd340197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd33ffff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3400ffb10 0x7fd340197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.631+0000 7fd33ffff700 1 -- 172.21.15.62:0/2299691795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0095f0 con 0x7fd340100d00 2023-07-14T16:23:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.632+0000 7fd33ffff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 0x7fd340197f90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0095c0 tx=0x7fd32c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.632+0000 7fd33dffb700 1 -- 172.21.15.62:0/2299691795 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c01d070 con 0x7fd340100d00 2023-07-14T16:23:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.632+0000 7fd33dffb700 1 -- 172.21.15.62:0/2299691795 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd32c003ec0 con 0x7fd340100d00 2023-07-14T16:23:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.632+0000 7fd346fff700 1 -- 172.21.15.62:0/2299691795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd340199a20 con 0x7fd340100d00 2023-07-14T16:23:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.632+0000 7fd33dffb700 1 -- 172.21.15.62:0/2299691795 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c022c00 con 0x7fd340100d00 2023-07-14T16:23:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.632+0000 7fd346fff700 1 -- 172.21.15.62:0/2299691795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3401a0240 con 0x7fd340100d00 2023-07-14T16:23:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.636+0000 7fd33dffb700 1 -- 172.21.15.62:0/2299691795 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd32c004030 con 0x7fd340100d00 2023-07-14T16:23:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.637+0000 7fd33dffb700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd3300a1150 0x7fd3300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.638+0000 7fd33dffb700 1 -- 172.21.15.62:0/2299691795 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd32c0cdef0 con 0x7fd340100d00 2023-07-14T16:23:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.638+0000 7fd344d9b700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd3300a1150 0x7fd3300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.638+0000 7fd35a937b80 1 Processor -- start 2023-07-14T16:23:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.638+0000 7fd35a937b80 1 -- start start 2023-07-14T16:23:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.638+0000 7fd344d9b700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd3300a1150 0x7fd3300a3600 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd340100b60 tx=0x7fd334007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b33300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 0x5589d3b34b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd35a937b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3a38c50 con 0x5589d3b33840 2023-07-14T16:23:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd35a937b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3a8b9e0 con 0x5589d3b346e0 2023-07-14T16:23:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd35a937b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3a84e70 con 0x5589d398f6b0 2023-07-14T16:23:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd33ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd33ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b341a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52862/0 (socket says 172.21.15.62:52862) 2023-07-14T16:23:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd33ffff700 1 -- 172.21.15.62:0/3073310985 learned_addr learned my addr 172.21.15.62:0/3073310985 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd344d9b700 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b33300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd34559c700 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 0x5589d3b34b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd33ffff700 1 -- 172.21.15.62:0/3073310985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 msgr2=0x5589d3b33300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.639+0000 7fd33ffff700 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b33300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.640+0000 7fd33ffff700 1 -- 172.21.15.62:0/3073310985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 msgr2=0x5589d3b34b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.640+0000 7fd33ffff700 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 0x5589d3b34b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.640+0000 7fd33ffff700 1 -- 172.21.15.62:0/3073310985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589d38b2d80 con 0x5589d3b33840 2023-07-14T16:23:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.640+0000 7fd344d9b700 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b33300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.640+0000 7fd33ffff700 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b341a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0a31d0 tx=0x7fd32c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9a12a38616d39e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.640+0000 7fd32affd700 1 -- 172.21.15.62:0/3073310985 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c09f160 con 0x5589d3b33840 2023-07-14T16:23:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.640+0000 7fd32affd700 1 -- 172.21.15.62:0/3073310985 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd32c09b340 con 0x5589d3b33840 2023-07-14T16:23:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.640+0000 7fd32affd700 1 -- 172.21.15.62:0/3073310985 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c09a830 con 0x5589d3b33840 2023-07-14T16:23:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 -- 172.21.15.62:0/3073310985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 msgr2=0x5589d3b341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b341a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0a31d0 tx=0x7fd32c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 -- 172.21.15.62:0/3073310985 shutdown_connections 2023-07-14T16:23:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 0x5589d3b34b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b33300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3073310985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b341a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 -- 172.21.15.62:0/3073310985 >> 172.21.15.62:0/3073310985 conn(0x5589d3b28be0 msgr2=0x5589d3b2dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 -- 172.21.15.62:0/3073310985 shutdown_connections 2023-07-14T16:23:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.641+0000 7fd35a937b80 1 -- 172.21.15.62:0/3073310985 wait complete. 2023-07-14T16:23:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.642+0000 7fd35a937b80 1 -- start start 2023-07-14T16:23:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.642+0000 7fd35a937b80 1 Processor -- start 2023-07-14T16:23:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.642+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b3ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd35a937b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3a24a60 con 0x5589d398f6b0 2023-07-14T16:23:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd35a937b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3a24760 con 0x5589d3b33840 2023-07-14T16:23:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd35a937b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d39088c0 con 0x5589d3b346e0 2023-07-14T16:23:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd35a937b80 10 client.0 _ll_register_callbacks cb 0x5589d3a22700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd344d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd344d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52868/0 (socket says 172.21.15.62:52868) 2023-07-14T16:23:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd344d9b700 1 -- 172.21.15.62:0/3157674366 learned_addr learned my addr 172.21.15.62:0/3157674366 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd33ffff700 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b3ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd34559c700 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd344d9b700 1 -- 172.21.15.62:0/3157674366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 msgr2=0x5589d3b43a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd344d9b700 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b43a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd344d9b700 1 -- 172.21.15.62:0/3157674366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 msgr2=0x5589d3b3ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd344d9b700 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b3ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd344d9b700 1 -- 172.21.15.62:0/3157674366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3401a0240 con 0x5589d398f6b0 2023-07-14T16:23:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.643+0000 7fd34559c700 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b43a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.644+0000 7fd344d9b700 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c4e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd334011540 tx=0x7fd3340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feeff1c87a195518 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.644+0000 7fd31ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.644+0000 7fd31ffff700 1 -- 172.21.15.62:0/3157674366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334004250 con 0x5589d398f6b0 2023-07-14T16:23:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.644+0000 7fd35a937b80 1 -- 172.21.15.62:0/3157674366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589d39de490 con 0x5589d398f6b0 2023-07-14T16:23:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.644+0000 7fd31ffff700 1 -- 172.21.15.62:0/3157674366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd334017070 con 0x5589d398f6b0 2023-07-14T16:23:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.644+0000 7fd31ffff700 1 -- 172.21.15.62:0/3157674366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3340043b0 con 0x5589d398f6b0 2023-07-14T16:23:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.644+0000 7fd35a937b80 10 client.17676 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.644+0000 7fd35a937b80 1 -- 172.21.15.62:0/3157674366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5589d389bc00 con 0x5589d398f6b0 2023-07-14T16:23:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.645+0000 7fd31ffff700 1 -- 172.21.15.62:0/3157674366 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5589d39de490 con 0x5589d398f6b0 2023-07-14T16:23:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.645+0000 7fd31ffff700 1 -- 172.21.15.62:0/3157674366 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd334031070 con 0x5589d398f6b0 2023-07-14T16:23:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.645+0000 7fd35a937b80 10 client.17676 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.645+0000 7fd35a937b80 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.645+0000 7fd35a937b80 20 client.17676 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.645+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589d3b48390 0x5589d3b4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.645+0000 7fd35a937b80 4 client.17676 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.645+0000 7fd35a937b80 1 -- 172.21.15.62:0/3157674366 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589d3a7de50 con 0x5589d3b48390 2023-07-14T16:23:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.646+0000 7fd33ffff700 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589d3b48390 0x5589d3b4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.646+0000 7fd33ffff700 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589d3b48390 0x5589d3b4a7a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=343eeb724cac5f7d server_cookie=4e9a11378f5d0d4 in_seq=0 out_seq=0 2023-07-14T16:23:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.646+0000 7fd31ffff700 10 client.17676 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd31ffff700 1 -- 172.21.15.62:0/3157674366 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5589d3a7de50 con 0x5589d3b48390 2023-07-14T16:23:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd31ffff700 10 client.17676 handle_command_reply: tid=0 2023-07-14T16:23:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 client.17676 shutdown 2023-07-14T16:23:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 -- 172.21.15.62:0/3157674366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 msgr2=0x5589d3b3c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c4e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd334011540 tx=0x7fd3340047c0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 -- 172.21.15.62:0/3157674366 shutdown_connections 2023-07-14T16:23:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b43a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c4e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589d3b48390 0x5589d3b4a7a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3157674366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b3ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd35a937b80 1 -- 172.21.15.62:0/3157674366 >> 172.21.15.62:0/3157674366 conn(0x5589d3b28be0 msgr2=0x5589d3b2d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.647+0000 7fd31ffff700 0 client.17676 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.648+0000 7fd35a937b80 1 -- 172.21.15.62:0/3157674366 shutdown_connections 2023-07-14T16:23:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.648+0000 7fd35a937b80 1 -- 172.21.15.62:0/3157674366 wait complete. 2023-07-14T16:23:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.648+0000 7fd35a937b80 20 client.17676 trim_cache size 0 max 16384 2023-07-14T16:23:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.668+0000 7fd35a937b80 1 Processor -- start 2023-07-14T16:23:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.668+0000 7fd35a937b80 1 -- start start 2023-07-14T16:23:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b43a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b3c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd35a937b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3b3c0b0 con 0x5589d398f6b0 2023-07-14T16:23:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd35a937b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3a7de80 con 0x5589d3b33840 2023-07-14T16:23:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd35a937b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3af9060 con 0x5589d3b346e0 2023-07-14T16:23:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd344d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd344d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52878/0 (socket says 172.21.15.62:52878) 2023-07-14T16:23:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd344d9b700 1 -- 172.21.15.62:0/3047687369 learned_addr learned my addr 172.21.15.62:0/3047687369 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd34559c700 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b3c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd344d9b700 1 -- 172.21.15.62:0/3047687369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 msgr2=0x5589d3b3c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.669+0000 7fd344d9b700 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b3c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd344d9b700 1 -- 172.21.15.62:0/3047687369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 msgr2=0x5589d3b43a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd33ffff700 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd344d9b700 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b43a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd344d9b700 1 -- 172.21.15.62:0/3047687369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589d38d91f0 con 0x5589d398f6b0 2023-07-14T16:23:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd34559c700 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b3c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd33ffff700 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b43a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd344d9b700 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c290 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd334000c00 tx=0x7fd334010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0478d26f1f52bf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd32b7fe700 1 -- 172.21.15.62:0/3047687369 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33401fae0 con 0x5589d398f6b0 2023-07-14T16:23:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd32b7fe700 1 -- 172.21.15.62:0/3047687369 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd33402e070 con 0x5589d398f6b0 2023-07-14T16:23:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.670+0000 7fd32b7fe700 1 -- 172.21.15.62:0/3047687369 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33403cb30 con 0x5589d398f6b0 2023-07-14T16:23:32.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 -- 172.21.15.62:0/3047687369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 msgr2=0x5589d3b3c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c290 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd334000c00 tx=0x7fd334010810 comp rx=0 tx=0).stop 2023-07-14T16:23:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 -- 172.21.15.62:0/3047687369 shutdown_connections 2023-07-14T16:23:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d3b346e0 0x5589d3b3c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d398f6b0 0x5589d3b3c290 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 --2- 172.21.15.62:0/3047687369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b33840 0x5589d3b43a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 -- 172.21.15.62:0/3047687369 >> 172.21.15.62:0/3047687369 conn(0x5589d3b28be0 msgr2=0x5589d3b4f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 -- 172.21.15.62:0/3047687369 shutdown_connections 2023-07-14T16:23:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.671+0000 7fd35a937b80 1 -- 172.21.15.62:0/3047687369 wait complete. 2023-07-14T16:23:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.672+0000 7fd35a937b80 1 -- start start 2023-07-14T16:23:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.672+0000 7fd35a937b80 1 Processor -- start 2023-07-14T16:23:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.672+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b3f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.672+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b3fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.672+0000 7fd35a937b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 0x5589d3b42ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd35a937b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3b431e0 con 0x5589d3b33840 2023-07-14T16:23:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd35a937b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3b43350 con 0x5589d3b346e0 2023-07-14T16:23:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd35a937b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d3b43510 con 0x5589d398f6b0 2023-07-14T16:23:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd35a937b80 10 client.0 _ll_register_callbacks cb 0x7fd334010c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd33ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b3fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd33ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b3fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52886/0 (socket says 172.21.15.62:52886) 2023-07-14T16:23:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd33ffff700 1 -- 172.21.15.62:0/720940886 learned_addr learned my addr 172.21.15.62:0/720940886 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd344d9b700 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b3f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd34559c700 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 0x5589d3b42ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd33ffff700 1 -- 172.21.15.62:0/720940886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 msgr2=0x5589d3b3f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd33ffff700 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b3f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd33ffff700 1 -- 172.21.15.62:0/720940886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 msgr2=0x5589d3b42ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd33ffff700 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 0x5589d3b42ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.673+0000 7fd33ffff700 1 -- 172.21.15.62:0/720940886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589d389bc00 con 0x5589d3b33840 2023-07-14T16:23:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.674+0000 7fd33ffff700 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b3fc80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0042e0 tx=0x7fd32c09b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cddcf839977097 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.674+0000 7fd31ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.674+0000 7fd31ffff700 1 -- 172.21.15.62:0/720940886 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c09f370 con 0x5589d3b33840 2023-07-14T16:23:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.674+0000 7fd31ffff700 1 -- 172.21.15.62:0/720940886 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd32c0a3700 con 0x5589d3b33840 2023-07-14T16:23:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.674+0000 7fd35a937b80 1 -- 172.21.15.62:0/720940886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd32c09b340 con 0x5589d3b33840 2023-07-14T16:23:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.674+0000 7fd31ffff700 1 -- 172.21.15.62:0/720940886 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd32c0a10f0 con 0x5589d3b33840 2023-07-14T16:23:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.674+0000 7fd35a937b80 10 client.17682 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.674+0000 7fd35a937b80 1 -- 172.21.15.62:0/720940886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5589d38b1250 con 0x5589d3b33840 2023-07-14T16:23:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd31ffff700 1 -- 172.21.15.62:0/720940886 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd32c09b340 con 0x5589d3b33840 2023-07-14T16:23:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd31ffff700 1 -- 172.21.15.62:0/720940886 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd32c0a32e0 con 0x5589d3b33840 2023-07-14T16:23:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd35a937b80 10 client.17682 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd35a937b80 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd35a937b80 20 client.17682 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd35a937b80 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589d3b9cc50 0x5589d3b9f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd35a937b80 4 client.17682 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:23:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd35a937b80 1 -- 172.21.15.62:0/720940886 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5589d3a3fff0 con 0x5589d3b9cc50 2023-07-14T16:23:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.675+0000 7fd344d9b700 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589d3b9cc50 0x5589d3b9f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.676+0000 7fd344d9b700 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589d3b9cc50 0x5589d3b9f060 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28abaa9f9a2ada7 server_cookie=e83b1e2de18c03a5 in_seq=0 out_seq=0 2023-07-14T16:23:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.676+0000 7fd31ffff700 10 client.17682 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.686+0000 7fd31ffff700 1 -- 172.21.15.62:0/720940886 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5589d3a3fff0 con 0x5589d3b9cc50 2023-07-14T16:23:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.686+0000 7fd31ffff700 10 client.17682 handle_command_reply: tid=0 2023-07-14T16:23:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.686+0000 7fd35a937b80 1 client.17682 shutdown 2023-07-14T16:23:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 -- 172.21.15.62:0/720940886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 msgr2=0x5589d3b3fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b3fc80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0042e0 tx=0x7fd32c09b8c0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 -- 172.21.15.62:0/720940886 shutdown_connections 2023-07-14T16:23:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589d3b9cc50 0x5589d3b9f060 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589d3b346e0 0x5589d3b42ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589d398f6b0 0x5589d3b3f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 --2- 172.21.15.62:0/720940886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589d3b33840 0x5589d3b3fc80 secure :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0042e0 tx=0x7fd32c09b8c0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd31ffff700 0 client.17682 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 -- 172.21.15.62:0/720940886 >> 172.21.15.62:0/720940886 conn(0x5589d3b28be0 msgr2=0x5589d3b3c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 -- 172.21.15.62:0/720940886 shutdown_connections 2023-07-14T16:23:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 1 -- 172.21.15.62:0/720940886 wait complete. 2023-07-14T16:23:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.687+0000 7fd35a937b80 20 client.17682 trim_cache size 0 max 16384 2023-07-14T16:23:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:23:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:23:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:23:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:23:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.688+0000 7fd346fff700 1 -- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd3300a1150 msgr2=0x7fd3300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.688+0000 7fd346fff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd3300a1150 0x7fd3300a3600 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd340100b60 tx=0x7fd334007400 comp rx=0 tx=0).stop 2023-07-14T16:23:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.688+0000 7fd346fff700 1 -- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 msgr2=0x7fd340197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.688+0000 7fd346fff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 0x7fd340197f90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0095c0 tx=0x7fd32c005740 comp rx=0 tx=0).stop 2023-07-14T16:23:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.689+0000 7fd346fff700 1 -- 172.21.15.62:0/2299691795 shutdown_connections 2023-07-14T16:23:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.689+0000 7fd346fff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd340101690 0x7fd340198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.689+0000 7fd346fff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd3300a1150 0x7fd3300a3600 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.689+0000 7fd346fff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3400ffb10 0x7fd340197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.690+0000 7fd346fff700 1 --2- 172.21.15.62:0/2299691795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd340100d00 0x7fd340197f90 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.690+0000 7fd346fff700 1 -- 172.21.15.62:0/2299691795 >> 172.21.15.62:0/2299691795 conn(0x7fd3400fb130 msgr2=0x7fd340106680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.690+0000 7fd346fff700 1 -- 172.21.15.62:0/2299691795 shutdown_connections 2023-07-14T16:23:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:32.690+0000 7fd346fff700 1 -- 172.21.15.62:0/2299691795 wait complete. 2023-07-14T16:23:33.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:32 smithi062 ceph-mon[95543]: pgmap v594: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:23:33.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:32 smithi081 ceph-mon[127795]: pgmap v594: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:23:33.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:32 smithi164 ceph-mon[97427]: pgmap v594: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:23:35.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:34 smithi062 ceph-mon[95543]: pgmap v595: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:23:35.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:34 smithi081 ceph-mon[127795]: pgmap v595: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:23:35.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:34 smithi164 ceph-mon[97427]: pgmap v595: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:23:37.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:36 smithi062 ceph-mon[95543]: pgmap v596: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:23:37.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:23:37.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:36 smithi081 ceph-mon[127795]: pgmap v596: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:23:37.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:36 smithi164 ceph-mon[97427]: pgmap v596: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:23:37.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/1163087911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 msgr2=0x7fcbb810a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/1163087911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 0x7fcbb810a040 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000b0a0 tx=0x7fcbb000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/1163087911 shutdown_connections 2023-07-14T16:23:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/1163087911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 0x7fcbb810a040 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/1163087911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbb8101c00 0x7fcbb8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/1163087911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb8102df0 0x7fcbb8103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/1163087911 >> 172.21.15.62:0/1163087911 conn(0x7fcbb80fd220 msgr2=0x7fcbb80ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/1163087911 shutdown_connections 2023-07-14T16:23:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.393+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/1163087911 wait complete. 2023-07-14T16:23:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.395+0000 7fcbbc9fe700 1 Processor -- start 2023-07-14T16:23:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.395+0000 7fcbbc9fe700 1 -- start start 2023-07-14T16:23:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.395+0000 7fcbbc9fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbb8101c00 0x7fcbb8197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.395+0000 7fcbbc9fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb8102df0 0x7fcbb8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.395+0000 7fcbbc9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 0x7fcbb819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.395+0000 7fcbbc9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb819f390 con 0x7fcbb8103780 2023-07-14T16:23:37.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.395+0000 7fcbbc9fe700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb819f4d0 con 0x7fcbb8102df0 2023-07-14T16:23:37.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.395+0000 7fcbbc9fe700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb819f660 con 0x7fcbb8101c00 2023-07-14T16:23:37.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 0x7fcbb819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb8102df0 0x7fcbb8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 0x7fcbb819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57988/0 (socket says 172.21.15.62:57988) 2023-07-14T16:23:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 -- 172.21.15.62:0/999626074 learned_addr learned my addr 172.21.15.62:0/999626074 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbb8101c00 0x7fcbb8197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 -- 172.21.15.62:0/999626074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbb8101c00 msgr2=0x7fcbb8197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbb8101c00 0x7fcbb8197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 -- 172.21.15.62:0/999626074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb8102df0 msgr2=0x7fcbb8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb8102df0 0x7fcbb8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 -- 172.21.15.62:0/999626074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb0009d00 con 0x7fcbb8103780 2023-07-14T16:23:37.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb659c700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb8102df0 0x7fcbb8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:37.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.396+0000 7fcbb759e700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 0x7fcbb819edc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009fd0 tx=0x7fcbb000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:37.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.397+0000 7fcb9ffff700 1 -- 172.21.15.62:0/999626074 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000e040 con 0x7fcbb8103780 2023-07-14T16:23:37.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.397+0000 7fcb9ffff700 1 -- 172.21.15.62:0/999626074 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbb000be90 con 0x7fcbb8103780 2023-07-14T16:23:37.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.397+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/999626074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb819f8e0 con 0x7fcbb8103780 2023-07-14T16:23:37.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.397+0000 7fcb9ffff700 1 -- 172.21.15.62:0/999626074 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb001b460 con 0x7fcbb8103780 2023-07-14T16:23:37.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.397+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/999626074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb819fd60 con 0x7fcbb8103780 2023-07-14T16:23:37.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.401+0000 7fcb9ffff700 1 -- 172.21.15.62:0/999626074 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcbb001d470 con 0x7fcbb8103780 2023-07-14T16:23:37.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.402+0000 7fcbd0b9bb80 1 Processor -- start 2023-07-14T16:23:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcbd0b9bb80 1 -- start start 2023-07-14T16:23:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcb9ffff700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcba00a1270 0x7fcba00a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcb9ffff700 1 -- 172.21.15.62:0/999626074 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcbb00cd840 con 0x7fcbb8103780 2023-07-14T16:23:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d42ebcb0 con 0x55b4d43e7070 2023-07-14T16:23:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d433ea40 con 0x55b4d43e7e30 2023-07-14T16:23:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.403+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d4337ed0 con 0x55b4d43e6720 2023-07-14T16:23:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb659c700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcba00a1270 0x7fcba00a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:57092/0 (socket says 172.21.15.62:57092) 2023-07-14T16:23:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 -- 172.21.15.62:0/1704583392 learned_addr learned my addr 172.21.15.62:0/1704583392 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb659c700 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb759e700 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 -- 172.21.15.62:0/1704583392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 msgr2=0x55b4d43e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 -- 172.21.15.62:0/1704583392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 msgr2=0x55b4d43e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 -- 172.21.15.62:0/1704583392 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4d4165de0 con 0x55b4d43e6720 2023-07-14T16:23:37.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb759e700 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:37.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb659c700 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:37.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb659c700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcba00a1270 0x7fcba00a3720 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0097c0 tx=0x7fcbac009750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:37.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.404+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e6b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d30 tx=0x7fcba8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d9cc02745a62f22 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:37.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.405+0000 7fcb9d7fa700 1 -- 172.21.15.62:0/1704583392 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8004220 con 0x55b4d43e6720 2023-07-14T16:23:37.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.405+0000 7fcb9d7fa700 1 -- 172.21.15.62:0/1704583392 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcba8004380 con 0x55b4d43e6720 2023-07-14T16:23:37.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.405+0000 7fcb9d7fa700 1 -- 172.21.15.62:0/1704583392 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80197a0 con 0x55b4d43e6720 2023-07-14T16:23:37.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1704583392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 msgr2=0x55b4d43e6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e6b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d30 tx=0x7fcba8011720 comp rx=0 tx=0).stop 2023-07-14T16:23:37.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1704583392 shutdown_connections 2023-07-14T16:23:37.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e6b30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1704583392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1704583392 >> 172.21.15.62:0/1704583392 conn(0x55b4d43dbff0 msgr2=0x55b4d43e1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:37.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1704583392 shutdown_connections 2023-07-14T16:23:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1704583392 wait complete. 2023-07-14T16:23:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.406+0000 7fcbd0b9bb80 1 -- start start 2023-07-14T16:23:37.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.407+0000 7fcbd0b9bb80 1 Processor -- start 2023-07-14T16:23:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.407+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e6720 0x55b4d43f0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.407+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7070 0x55b4d43f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.407+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.407+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d42d7ac0 con 0x55b4d43e6720 2023-07-14T16:23:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.407+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d42d77c0 con 0x55b4d43e7e30 2023-07-14T16:23:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.407+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d41bb920 con 0x55b4d43e7070 2023-07-14T16:23:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.407+0000 7fcbd0b9bb80 10 client.0 _ll_register_callbacks cb 0x55b4d42d5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e6720 0x55b4d43f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7070 0x55b4d43f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e6720 0x55b4d43f0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58012/0 (socket says 172.21.15.62:58012) 2023-07-14T16:23:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb6d9d700 1 -- 172.21.15.62:0/3932526536 learned_addr learned my addr 172.21.15.62:0/3932526536 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb759e700 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb6d9d700 1 -- 172.21.15.62:0/3932526536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7070 msgr2=0x55b4d43f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7070 0x55b4d43f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb6d9d700 1 -- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 msgr2=0x55b4d43f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb6d9d700 1 -- 172.21.15.62:0/3932526536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba8009960 con 0x55b4d43e6720 2023-07-14T16:23:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.408+0000 7fcbb759e700 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.409+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e6720 0x55b4d43f0290 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fcba8005920 tx=0x7fcba8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59f7a5a9bb87526 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.409+0000 7fcb9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.409+0000 7fcb9aff5700 1 -- 172.21.15.62:0/3932526536 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8005240 con 0x55b4d43e6720 2023-07-14T16:23:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.409+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/3932526536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4d42914f0 con 0x55b4d43e6720 2023-07-14T16:23:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.409+0000 7fcb9aff5700 1 -- 172.21.15.62:0/3932526536 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcba80053a0 con 0x55b4d43e6720 2023-07-14T16:23:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.409+0000 7fcb9aff5700 1 -- 172.21.15.62:0/3932526536 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba8005560 con 0x55b4d43e6720 2023-07-14T16:23:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.409+0000 7fcbd0b9bb80 10 client.17694 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.409+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/3932526536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b4d414ec60 con 0x55b4d43e6720 2023-07-14T16:23:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.410+0000 7fcb9aff5700 1 -- 172.21.15.62:0/3932526536 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b4d42914f0 con 0x55b4d43e6720 2023-07-14T16:23:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.410+0000 7fcb9aff5700 1 -- 172.21.15.62:0/3932526536 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcba8021040 con 0x55b4d43e6720 2023-07-14T16:23:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.410+0000 7fcbd0b9bb80 10 client.17694 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.410+0000 7fcbd0b9bb80 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.410+0000 7fcbd0b9bb80 20 client.17694 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.410+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b4d43fbf40 0x55b4d43fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.410+0000 7fcbd0b9bb80 4 client.17694 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.410+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/3932526536 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4d4330eb0 con 0x55b4d43fbf40 2023-07-14T16:23:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.411+0000 7fcbb659c700 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b4d43fbf40 0x55b4d43fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.411+0000 7fcbb659c700 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b4d43fbf40 0x55b4d43fe350 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b219682a7bbff5b server_cookie=a8b148687b4f21ef in_seq=0 out_seq=0 2023-07-14T16:23:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.411+0000 7fcb9aff5700 10 client.17694 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcb9aff5700 1 -- 172.21.15.62:0/3932526536 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b4d4330eb0 con 0x55b4d43fbf40 2023-07-14T16:23:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcb9aff5700 10 client.17694 handle_command_reply: tid=0 2023-07-14T16:23:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 client.17694 shutdown 2023-07-14T16:23:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/3932526536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e6720 msgr2=0x55b4d43f0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e6720 0x55b4d43f0290 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fcba8005920 tx=0x7fcba8003680 comp rx=0 tx=0).stop 2023-07-14T16:23:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/3932526536 shutdown_connections 2023-07-14T16:23:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b4d43fbf40 0x55b4d43fe350 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e6720 0x55b4d43f0290 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/3932526536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7070 0x55b4d43f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcb9aff5700 0 client.17694 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/3932526536 >> 172.21.15.62:0/3932526536 conn(0x55b4d43dbff0 msgr2=0x55b4d43e0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.412+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/3932526536 shutdown_connections 2023-07-14T16:23:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.413+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/3932526536 wait complete. 2023-07-14T16:23:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.413+0000 7fcbd0b9bb80 20 client.17694 trim_cache size 0 max 16384 2023-07-14T16:23:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.435+0000 7fcbd0b9bb80 1 Processor -- start 2023-07-14T16:23:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.435+0000 7fcbd0b9bb80 1 -- start start 2023-07-14T16:23:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d43e2490 con 0x55b4d43e7070 2023-07-14T16:23:37.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d43fba80 con 0x55b4d43e7e30 2023-07-14T16:23:37.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d43eb800 con 0x55b4d43e6720 2023-07-14T16:23:37.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58026/0 (socket says 172.21.15.62:58026) 2023-07-14T16:23:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb659c700 1 -- 172.21.15.62:0/2164624800 learned_addr learned my addr 172.21.15.62:0/2164624800 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb759e700 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb659c700 1 -- 172.21.15.62:0/2164624800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 msgr2=0x55b4d43e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:37.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb659c700 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb659c700 1 -- 172.21.15.62:0/2164624800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 msgr2=0x55b4d43f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:37.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb659c700 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb659c700 1 -- 172.21.15.62:0/2164624800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4d418c250 con 0x55b4d43e7070 2023-07-14T16:23:37.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.436+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:37.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.437+0000 7fcbb659c700 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f77d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcbac011670 tx=0x7fcbac013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fdef8356f84238 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.437+0000 7fcb9dffb700 1 -- 172.21.15.62:0/2164624800 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac022030 con 0x55b4d43e7070 2023-07-14T16:23:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.437+0000 7fcb9dffb700 1 -- 172.21.15.62:0/2164624800 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbac023040 con 0x55b4d43e7070 2023-07-14T16:23:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.437+0000 7fcb9dffb700 1 -- 172.21.15.62:0/2164624800 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac02c290 con 0x55b4d43e7070 2023-07-14T16:23:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/2164624800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 msgr2=0x55b4d43f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f77d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcbac011670 tx=0x7fcbac013070 comp rx=0 tx=0).stop 2023-07-14T16:23:37.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/2164624800 shutdown_connections 2023-07-14T16:23:37.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e7e30 0x55b4d43f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e6720 0x55b4d43e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/2164624800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f77d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/2164624800 >> 172.21.15.62:0/2164624800 conn(0x55b4d43dbff0 msgr2=0x55b4d4402c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:37.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/2164624800 shutdown_connections 2023-07-14T16:23:37.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.438+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/2164624800 wait complete. 2023-07-14T16:23:37.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 1 -- start start 2023-07-14T16:23:37.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 1 Processor -- start 2023-07-14T16:23:37.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e6720 0x55b4d43f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7e30 0x55b4d43f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d43f4540 con 0x55b4d43e7070 2023-07-14T16:23:37.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d43f46b0 con 0x55b4d43e6720 2023-07-14T16:23:37.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d43f4870 con 0x55b4d43e7e30 2023-07-14T16:23:37.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.439+0000 7fcbd0b9bb80 10 client.0 _ll_register_callbacks cb 0x7fcba8009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:37.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e6720 0x55b4d43f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58034/0 (socket says 172.21.15.62:58034) 2023-07-14T16:23:37.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 -- 172.21.15.62:0/1888295567 learned_addr learned my addr 172.21.15.62:0/1888295567 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:37.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb759e700 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7e30 0x55b4d43f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 -- 172.21.15.62:0/1888295567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7e30 msgr2=0x55b4d43f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:37.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7e30 0x55b4d43f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 -- 172.21.15.62:0/1888295567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e6720 msgr2=0x55b4d43f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:37.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e6720 0x55b4d43f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 -- 172.21.15.62:0/1888295567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac019040 con 0x55b4d43e7070 2023-07-14T16:23:37.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e6720 0x55b4d43f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:37.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcbb659c700 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f7780 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0092a0 tx=0x7fcbac005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7f72c9aa96f770 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:37.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.440+0000 7fcb9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:37.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.441+0000 7fcb9aff5700 1 -- 172.21.15.62:0/1888295567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac022040 con 0x55b4d43e7070 2023-07-14T16:23:37.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.441+0000 7fcb9aff5700 1 -- 172.21.15.62:0/1888295567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcbac02cb10 con 0x55b4d43e7070 2023-07-14T16:23:37.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.441+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1888295567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcba8004380 con 0x55b4d43e7070 2023-07-14T16:23:37.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.441+0000 7fcb9aff5700 1 -- 172.21.15.62:0/1888295567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbac02ccd0 con 0x55b4d43e7070 2023-07-14T16:23:37.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.441+0000 7fcbd0b9bb80 10 client.17700 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:37.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.441+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1888295567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b4d41642b0 con 0x55b4d43e7070 2023-07-14T16:23:37.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.441+0000 7fcb9aff5700 1 -- 172.21.15.62:0/1888295567 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcba8004380 con 0x55b4d43e7070 2023-07-14T16:23:37.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.442+0000 7fcb9aff5700 1 -- 172.21.15.62:0/1888295567 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcbac02b6f0 con 0x55b4d43e7070 2023-07-14T16:23:37.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.442+0000 7fcbd0b9bb80 10 client.17700 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:37.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.442+0000 7fcbd0b9bb80 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:37.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.442+0000 7fcbd0b9bb80 20 client.17700 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:37.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.442+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b4d4451380 0x55b4d4453790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.442+0000 7fcbd0b9bb80 4 client.17700 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:23:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.442+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1888295567 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b4d42f3050 con 0x55b4d4451380 2023-07-14T16:23:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.442+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b4d4451380 0x55b4d4453790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbb6d9d700 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b4d4451380 0x55b4d4453790 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ec7af7d748850c7 server_cookie=8ff5d187e7498886 in_seq=0 out_seq=0 2023-07-14T16:23:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcb9aff5700 10 client.17700 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcb9aff5700 1 -- 172.21.15.62:0/1888295567 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b4d42f3050 con 0x55b4d4451380 2023-07-14T16:23:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcb9aff5700 10 client.17700 handle_command_reply: tid=0 2023-07-14T16:23:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 client.17700 shutdown 2023-07-14T16:23:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1888295567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 msgr2=0x55b4d43f7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f7780 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0092a0 tx=0x7fcbac005910 comp rx=0 tx=0).stop 2023-07-14T16:23:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1888295567 shutdown_connections 2023-07-14T16:23:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b4d43e7e30 0x55b4d43f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b4d4451380 0x55b4d4453790 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b4d43e6720 0x55b4d43f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 --2- 172.21.15.62:0/1888295567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b4d43e7070 0x55b4d43f7780 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1888295567 >> 172.21.15.62:0/1888295567 conn(0x55b4d43dbff0 msgr2=0x55b4d43f8100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.443+0000 7fcb9aff5700 0 client.17700 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1888295567 shutdown_connections 2023-07-14T16:23:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbd0b9bb80 1 -- 172.21.15.62:0/1888295567 wait complete. 2023-07-14T16:23:37.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbd0b9bb80 20 client.17700 trim_cache size 0 max 16384 2023-07-14T16:23:37.432 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:23:37.432 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:23:37.432 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:23:37.432 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:23:37.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcba00a1270 msgr2=0x7fcba00a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcba00a1270 0x7fcba00a3720 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0097c0 tx=0x7fcbac009750 comp rx=0 tx=0).stop 2023-07-14T16:23:37.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 msgr2=0x7fcbb819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 0x7fcbb819edc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009fd0 tx=0x7fcbb000ba50 comp rx=0 tx=0).stop 2023-07-14T16:23:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/999626074 shutdown_connections 2023-07-14T16:23:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbb8103780 0x7fcbb819edc0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbb8101c00 0x7fcbb8197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcba00a1270 0x7fcba00a3720 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 --2- 172.21.15.62:0/999626074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbb8102df0 0x7fcbb8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.444+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/999626074 >> 172.21.15.62:0/999626074 conn(0x7fcbb80fd220 msgr2=0x7fcbb8106290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.445+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/999626074 shutdown_connections 2023-07-14T16:23:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.445+0000 7fcbbc9fe700 1 -- 172.21.15.62:0/999626074 wait complete. 2023-07-14T16:23:37.441 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:23:37.441 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:23:37.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.709+0000 7fb2489ff700 1 -- 172.21.15.62:0/3184275995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244102f10 msgr2=0x7fb244103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.709+0000 7fb2489ff700 1 --2- 172.21.15.62:0/3184275995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244102f10 0x7fb244103360 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fb238009990 tx=0x7fb238009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.710+0000 7fb2489ff700 1 -- 172.21.15.62:0/3184275995 shutdown_connections 2023-07-14T16:23:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.710+0000 7fb2489ff700 1 --2- 172.21.15.62:0/3184275995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2441038a0 0x7fb24410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.710+0000 7fb2489ff700 1 --2- 172.21.15.62:0/3184275995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb244101d20 0x7fb244102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.710+0000 7fb2489ff700 1 --2- 172.21.15.62:0/3184275995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244102f10 0x7fb244103360 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.710+0000 7fb2489ff700 1 -- 172.21.15.62:0/3184275995 >> 172.21.15.62:0/3184275995 conn(0x7fb2440fd360 msgr2=0x7fb2440ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.710+0000 7fb2489ff700 1 -- 172.21.15.62:0/3184275995 shutdown_connections 2023-07-14T16:23:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.710+0000 7fb2489ff700 1 -- 172.21.15.62:0/3184275995 wait complete. 2023-07-14T16:23:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.711+0000 7fb2489ff700 1 Processor -- start 2023-07-14T16:23:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.711+0000 7fb2489ff700 1 -- start start 2023-07-14T16:23:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.711+0000 7fb2489ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101d20 0x7fb244197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.711+0000 7fb2489ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb244102f10 0x7fb244197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.711+0000 7fb2489ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2441038a0 0x7fb24419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.711+0000 7fb2489ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24419f4d0 con 0x7fb244101d20 2023-07-14T16:23:37.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.711+0000 7fb2489ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24419f610 con 0x7fb244102f10 2023-07-14T16:23:37.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.711+0000 7fb2489ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24419f7a0 con 0x7fb2441038a0 2023-07-14T16:23:37.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101d20 0x7fb244197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101d20 0x7fb244197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58058/0 (socket says 172.21.15.62:58058) 2023-07-14T16:23:37.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 -- 172.21.15.62:0/2394535404 learned_addr learned my addr 172.21.15.62:0/2394535404 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb241d9b700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb244102f10 0x7fb244197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb242d9d700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2441038a0 0x7fb24419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 -- 172.21.15.62:0/2394535404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2441038a0 msgr2=0x7fb24419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2441038a0 0x7fb24419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 -- 172.21.15.62:0/2394535404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb244102f10 msgr2=0x7fb244197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb244102f10 0x7fb244197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 -- 172.21.15.62:0/2394535404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb234009520 con 0x7fb244101d20 2023-07-14T16:23:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.712+0000 7fb24259c700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101d20 0x7fb244197950 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb23400ea50 tx=0x7fb23400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.713+0000 7fb2337fe700 1 -- 172.21.15.62:0/2394535404 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23400cbd0 con 0x7fb244101d20 2023-07-14T16:23:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.713+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2380095f0 con 0x7fb244101d20 2023-07-14T16:23:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.713+0000 7fb2337fe700 1 -- 172.21.15.62:0/2394535404 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb23400cd30 con 0x7fb244101d20 2023-07-14T16:23:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.713+0000 7fb2337fe700 1 -- 172.21.15.62:0/2394535404 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234018680 con 0x7fb244101d20 2023-07-14T16:23:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.713+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24419fe50 con 0x7fb244101d20 2023-07-14T16:23:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.714+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2441092b0 con 0x7fb244101d20 2023-07-14T16:23:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.716+0000 7fb2337fe700 1 -- 172.21.15.62:0/2394535404 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb234010430 con 0x7fb244101d20 2023-07-14T16:23:37.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.718+0000 7fb2337fe700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb22c0a11a0 0x7fb22c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:37.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.718+0000 7fb2337fe700 1 -- 172.21.15.62:0/2394535404 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb234014070 con 0x7fb244101d20 2023-07-14T16:23:37.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.718+0000 7fb241d9b700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb22c0a11a0 0x7fb22c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:37.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.719+0000 7fb241d9b700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb22c0a11a0 0x7fb22c0a3650 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb238006010 tx=0x7fb23800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:37.722+0000 7fb2337fe700 1 -- 172.21.15.62:0/2394535404 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb2340966b0 con 0x7fb244101d20 2023-07-14T16:23:38.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.032+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb24402c850 con 0x7fb244101d20 2023-07-14T16:23:38.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.033+0000 7fb2337fe700 1 -- 172.21.15.62:0/2394535404 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fb234099cd0 con 0x7fb244101d20 2023-07-14T16:23:38.010 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:23:38.011 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:23:38.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.035+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb22c0a11a0 msgr2=0x7fb22c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.035+0000 7fb2489ff700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb22c0a11a0 0x7fb22c0a3650 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb238006010 tx=0x7fb23800b260 comp rx=0 tx=0).stop 2023-07-14T16:23:38.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.035+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101d20 msgr2=0x7fb244197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.035+0000 7fb2489ff700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101d20 0x7fb244197950 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb23400ea50 tx=0x7fb23400ed60 comp rx=0 tx=0).stop 2023-07-14T16:23:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.036+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 shutdown_connections 2023-07-14T16:23:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.036+0000 7fb2489ff700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2441038a0 0x7fb24419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.036+0000 7fb2489ff700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb244101d20 0x7fb244197950 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.036+0000 7fb2489ff700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb22c0a11a0 0x7fb22c0a3650 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.036+0000 7fb2489ff700 1 --2- 172.21.15.62:0/2394535404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb244102f10 0x7fb244197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.036+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 >> 172.21.15.62:0/2394535404 conn(0x7fb2440fd360 msgr2=0x7fb2441063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.036+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 shutdown_connections 2023-07-14T16:23:38.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.036+0000 7fb2489ff700 1 -- 172.21.15.62:0/2394535404 wait complete. 2023-07-14T16:23:38.014 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:23:38.032 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:23:38.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.291+0000 7fb0c9fff700 1 -- 172.21.15.62:0/2905386886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4102f00 msgr2=0x7fb0c4103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.291+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/2905386886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4102f00 0x7fb0c4103350 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009990 tx=0x7fb0b8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.292+0000 7fb0c9fff700 1 -- 172.21.15.62:0/2905386886 shutdown_connections 2023-07-14T16:23:38.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.292+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/2905386886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c4103890 0x7fb0c410a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.292+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/2905386886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c4101d10 0x7fb0c4102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.292+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/2905386886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4102f00 0x7fb0c4103350 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.292+0000 7fb0c9fff700 1 -- 172.21.15.62:0/2905386886 >> 172.21.15.62:0/2905386886 conn(0x7fb0c40fd350 msgr2=0x7fb0c40ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.292+0000 7fb0c9fff700 1 -- 172.21.15.62:0/2905386886 shutdown_connections 2023-07-14T16:23:38.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.292+0000 7fb0c9fff700 1 -- 172.21.15.62:0/2905386886 wait complete. 2023-07-14T16:23:38.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.293+0000 7fb0c9fff700 1 Processor -- start 2023-07-14T16:23:38.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.293+0000 7fb0c9fff700 1 -- start start 2023-07-14T16:23:38.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c9fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4101d10 0x7fb0c4197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c9fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c4102f00 0x7fb0c4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c9fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c4103890 0x7fb0c419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c9fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c419f510 con 0x7fb0c4101d10 2023-07-14T16:23:38.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c9fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c419f650 con 0x7fb0c4103890 2023-07-14T16:23:38.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c9fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c419f7e0 con 0x7fb0c4102f00 2023-07-14T16:23:38.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4101d10 0x7fb0c4197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4101d10 0x7fb0c4197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58080/0 (socket says 172.21.15.62:58080) 2023-07-14T16:23:38.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c37fe700 1 -- 172.21.15.62:0/3882686985 learned_addr learned my addr 172.21.15.62:0/3882686985 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c4102f00 0x7fb0c4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c4103890 0x7fb0c419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c37fe700 1 -- 172.21.15.62:0/3882686985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c4102f00 msgr2=0x7fb0c4197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c4102f00 0x7fb0c4197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c37fe700 1 -- 172.21.15.62:0/3882686985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c4103890 msgr2=0x7fb0c419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c4103890 0x7fb0c419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c37fe700 1 -- 172.21.15.62:0/3882686985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80095f0 con 0x7fb0c4101d10 2023-07-14T16:23:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c4102f00 0x7fb0c4197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:38.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.294+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c4103890 0x7fb0c419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:38.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.295+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4101d10 0x7fb0c4197990 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400e970 tx=0x7fb0b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.295+0000 7fb0c0ff9700 1 -- 172.21.15.62:0/3882686985 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4009790 con 0x7fb0c4101d10 2023-07-14T16:23:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.295+0000 7fb0c0ff9700 1 -- 172.21.15.62:0/3882686985 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0b40044b0 con 0x7fb0c4101d10 2023-07-14T16:23:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.295+0000 7fb0c9fff700 1 -- 172.21.15.62:0/3882686985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c419fac0 con 0x7fb0c4101d10 2023-07-14T16:23:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.295+0000 7fb0c0ff9700 1 -- 172.21.15.62:0/3882686985 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4003dd0 con 0x7fb0c4101d10 2023-07-14T16:23:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.295+0000 7fb0c9fff700 1 -- 172.21.15.62:0/3882686985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c41a0020 con 0x7fb0c4101d10 2023-07-14T16:23:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.298+0000 7fb0c0ff9700 1 -- 172.21.15.62:0/3882686985 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb0b4003f30 con 0x7fb0c4101d10 2023-07-14T16:23:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.300+0000 7fb0dd9aab80 1 Processor -- start 2023-07-14T16:23:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0c0ff9700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ac0a11d0 0x7fb0ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0dd9aab80 1 -- start start 2023-07-14T16:23:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0c0ff9700 1 -- 172.21.15.62:0/3882686985 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0b4014070 con 0x7fb0c4101d10 2023-07-14T16:23:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ac0a11d0 0x7fb0ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 0x55c17ec098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec0a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec0b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17eb0ecb0 con 0x55c17ec09e20 2023-07-14T16:23:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17eb61a40 con 0x55c17ec0ac70 2023-07-14T16:23:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.301+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17eb5aed0 con 0x55c17ec094d0 2023-07-14T16:23:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec0a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec0a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58090/0 (socket says 172.21.15.62:58090) 2023-07-14T16:23:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 -- 172.21.15.62:0/14683018 learned_addr learned my addr 172.21.15.62:0/14683018 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 0x55c17ec098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec0b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 -- 172.21.15.62:0/14683018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 msgr2=0x55c17ec098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ac0a11d0 0x7fb0ac0a3680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009fd0 tx=0x7fb0bc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 0x55c17ec098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 -- 172.21.15.62:0/14683018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 msgr2=0x55c17ec0b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec0b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 -- 172.21.15.62:0/14683018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c17e988de0 con 0x55c17ec09e20 2023-07-14T16:23:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.302+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec0a6e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8000c00 tx=0x7fb0b8005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c3c3578d8e9185 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.303+0000 7fb0a9ffb700 1 -- 172.21.15.62:0/14683018 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b801e070 con 0x55c17ec09e20 2023-07-14T16:23:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.303+0000 7fb0a9ffb700 1 -- 172.21.15.62:0/14683018 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0b8004080 con 0x55c17ec09e20 2023-07-14T16:23:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.303+0000 7fb0a9ffb700 1 -- 172.21.15.62:0/14683018 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80056d0 con 0x55c17ec09e20 2023-07-14T16:23:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/14683018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 msgr2=0x55c17ec0a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec0a6e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8000c00 tx=0x7fb0b8005ee0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/14683018 shutdown_connections 2023-07-14T16:23:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec0b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 0x55c17ec098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/14683018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec0a6e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/14683018 >> 172.21.15.62:0/14683018 conn(0x55c17ebfeda0 msgr2=0x55c17ec041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/14683018 shutdown_connections 2023-07-14T16:23:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/14683018 wait complete. 2023-07-14T16:23:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.304+0000 7fb0dd9aab80 1 -- start start 2023-07-14T16:23:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0dd9aab80 1 Processor -- start 2023-07-14T16:23:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 0x55c17ec13040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec13580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec1a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17eafaac0 con 0x55c17ec09e20 2023-07-14T16:23:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17eafa7c0 con 0x55c17ec0ac70 2023-07-14T16:23:38.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17e9de920 con 0x55c17ec094d0 2023-07-14T16:23:38.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0dd9aab80 10 client.0 _ll_register_callbacks cb 0x55c17eaf8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:38.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.305+0000 7fb0c2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec13580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec13580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58104/0 (socket says 172.21.15.62:58104) 2023-07-14T16:23:38.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c2ffd700 1 -- 172.21.15.62:0/4226700866 learned_addr learned my addr 172.21.15.62:0/4226700866 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 0x55c17ec13040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec1a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c2ffd700 1 -- 172.21.15.62:0/4226700866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 msgr2=0x55c17ec13040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 0x55c17ec13040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c2ffd700 1 -- 172.21.15.62:0/4226700866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 msgr2=0x55c17ec1a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec1a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c2ffd700 1 -- 172.21.15.62:0/4226700866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c410aa80 con 0x55c17ec09e20 2023-07-14T16:23:38.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec13580 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800b2e0 tx=0x7fb0b8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a21f21f4ccb6ee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.306+0000 7fb0a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:38.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.307+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/4226700866 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b801e040 con 0x55c17ec09e20 2023-07-14T16:23:38.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.307+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/4226700866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c17eab44f0 con 0x55c17ec09e20 2023-07-14T16:23:38.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.307+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/4226700866 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0b800b6b0 con 0x55c17ec09e20 2023-07-14T16:23:38.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.307+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/4226700866 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0b80041f0 con 0x55c17ec09e20 2023-07-14T16:23:38.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.307+0000 7fb0dd9aab80 10 client.17718 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:38.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.307+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/4226700866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c17e971c60 con 0x55c17ec09e20 2023-07-14T16:23:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.307+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/4226700866 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c17eab44f0 con 0x55c17ec09e20 2023-07-14T16:23:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.308+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/4226700866 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb0b8004d70 con 0x55c17ec09e20 2023-07-14T16:23:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.308+0000 7fb0dd9aab80 10 client.17718 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.308+0000 7fb0dd9aab80 10 client.17718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:23:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.308+0000 7fb0dd9aab80 20 client.17718 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.308+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c17ec1e9f0 0x55c17ec20e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.308+0000 7fb0dd9aab80 4 client.17718 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.308+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/4226700866 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c17eb53eb0 con 0x55c17ec1e9f0 2023-07-14T16:23:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.308+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c17ec1e9f0 0x55c17ec20e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.309+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c17ec1e9f0 0x55c17ec20e00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1577edfa93c2d8 server_cookie=b6269ef26a8396a in_seq=0 out_seq=0 2023-07-14T16:23:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.309+0000 7fb0a2ffd700 10 client.17718 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:23:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/4226700866 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c17eb53eb0 con 0x55c17ec1e9f0 2023-07-14T16:23:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0a2ffd700 10 client.17718 handle_command_reply: tid=0 2023-07-14T16:23:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 client.17718 shutdown 2023-07-14T16:23:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/4226700866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 msgr2=0x55c17ec13580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec13580 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800b2e0 tx=0x7fb0b8005810 comp rx=0 tx=0).stop 2023-07-14T16:23:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/4226700866 shutdown_connections 2023-07-14T16:23:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c17ec1e9f0 0x55c17ec20e00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec1a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec094d0 0x55c17ec13040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/4226700866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec09e20 0x55c17ec13580 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0a2ffd700 0 client.17718 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:23:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/4226700866 >> 172.21.15.62:0/4226700866 conn(0x55c17ebfeda0 msgr2=0x55c17ec038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/4226700866 shutdown_connections 2023-07-14T16:23:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.310+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/4226700866 wait complete. 2023-07-14T16:23:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.311+0000 7fb0dd9aab80 20 client.17718 trim_cache size 0 max 16384 2023-07-14T16:23:38.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0dd9aab80 1 Processor -- start 2023-07-14T16:23:38.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0dd9aab80 1 -- start start 2023-07-14T16:23:38.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec0b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec09e20 0x55c17ec1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec25720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17ec05100 con 0x55c17ec094d0 2023-07-14T16:23:38.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17ec13cf0 con 0x55c17ec0ac70 2023-07-14T16:23:38.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17ec10d90 con 0x55c17ec09e20 2023-07-14T16:23:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec0b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec0b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58120/0 (socket says 172.21.15.62:58120) 2023-07-14T16:23:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0c37fe700 1 -- 172.21.15.62:0/2906053039 learned_addr learned my addr 172.21.15.62:0/2906053039 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.326+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec09e20 0x55c17ec1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec25720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0c37fe700 1 -- 172.21.15.62:0/2906053039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec09e20 msgr2=0x55c17ec1a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec09e20 0x55c17ec1a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0c37fe700 1 -- 172.21.15.62:0/2906053039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 msgr2=0x55c17ec25720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec25720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0c37fe700 1 -- 172.21.15.62:0/2906053039 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c17e9af250 con 0x55c17ec094d0 2023-07-14T16:23:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec09e20 0x55c17ec1a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec0b040 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4099f80 tx=0x7fb0b40ca5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612323360f2e7b7e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0aa7fc700 1 -- 172.21.15.62:0/2906053039 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400cdf0 con 0x55c17ec094d0 2023-07-14T16:23:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0aa7fc700 1 -- 172.21.15.62:0/2906053039 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0b4098130 con 0x55c17ec094d0 2023-07-14T16:23:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.327+0000 7fb0aa7fc700 1 -- 172.21.15.62:0/2906053039 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40041e0 con 0x55c17ec094d0 2023-07-14T16:23:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/2906053039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 msgr2=0x55c17ec0b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec0b040 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4099f80 tx=0x7fb0b40ca5b0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/2906053039 shutdown_connections 2023-07-14T16:23:38.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec0ac70 0x55c17ec25720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec0b040 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/2906053039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec09e20 0x55c17ec1a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/2906053039 >> 172.21.15.62:0/2906053039 conn(0x55c17ebfeda0 msgr2=0x55c17ec041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/2906053039 shutdown_connections 2023-07-14T16:23:38.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.328+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/2906053039 wait complete. 2023-07-14T16:23:38.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 1 -- start start 2023-07-14T16:23:38.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 1 Processor -- start 2023-07-14T16:23:38.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec16890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec09e20 0x55c17ec16dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec0ac70 0x55c17ec19df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17ec1a330 con 0x55c17ec094d0 2023-07-14T16:23:38.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17ec1a4a0 con 0x55c17ec09e20 2023-07-14T16:23:38.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17ec1a660 con 0x55c17ec0ac70 2023-07-14T16:23:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.329+0000 7fb0dd9aab80 10 client.0 _ll_register_callbacks cb 0x7fb0bc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec16890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec16890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58134/0 (socket says 172.21.15.62:58134) 2023-07-14T16:23:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 -- 172.21.15.62:0/1075522624 learned_addr learned my addr 172.21.15.62:0/1075522624 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec09e20 0x55c17ec16dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec0ac70 0x55c17ec19df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 -- 172.21.15.62:0/1075522624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec0ac70 msgr2=0x55c17ec19df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec0ac70 0x55c17ec19df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 -- 172.21.15.62:0/1075522624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec09e20 msgr2=0x55c17ec16dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec09e20 0x55c17ec16dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 -- 172.21.15.62:0/1075522624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c41a0020 con 0x55c17ec094d0 2023-07-14T16:23:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c3fff700 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec0ac70 0x55c17ec19df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0c37fe700 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec16890 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40a1f10 tx=0x7fb0b4098390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6098480dd90c625 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.330+0000 7fb0a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/1075522624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400cc50 con 0x55c17ec094d0 2023-07-14T16:23:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/1075522624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0b40041e0 con 0x55c17ec094d0 2023-07-14T16:23:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/1075522624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0b8004080 con 0x55c17ec094d0 2023-07-14T16:23:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/1075522624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0b409a250 con 0x55c17ec094d0 2023-07-14T16:23:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0dd9aab80 10 client.17724 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/1075522624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c17e9872b0 con 0x55c17ec094d0 2023-07-14T16:23:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/1075522624 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0b8004080 con 0x55c17ec094d0 2023-07-14T16:23:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/1075522624 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb0b40a1fb0 con 0x55c17ec094d0 2023-07-14T16:23:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0dd9aab80 10 client.17724 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0dd9aab80 10 client.17724 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:23:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.331+0000 7fb0dd9aab80 20 client.17724 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.332+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c17ec73900 0x55c17ec75d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.332+0000 7fb0dd9aab80 4 client.17724 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:23:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.332+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/1075522624 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c17eb16050 con 0x55c17ec73900 2023-07-14T16:23:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.332+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c17ec73900 0x55c17ec75d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.332+0000 7fb0c2ffd700 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c17ec73900 0x55c17ec75d10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7ec56a8baaca3c4 server_cookie=18675b8971b82cee in_seq=0 out_seq=0 2023-07-14T16:23:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.332+0000 7fb0a2ffd700 10 client.17724 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:23:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0a2ffd700 1 -- 172.21.15.62:0/1075522624 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c17eb16050 con 0x55c17ec73900 2023-07-14T16:23:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0a2ffd700 10 client.17724 handle_command_reply: tid=0 2023-07-14T16:23:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 client.17724 shutdown 2023-07-14T16:23:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/1075522624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 msgr2=0x55c17ec16890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec16890 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40a1f10 tx=0x7fb0b4098390 comp rx=0 tx=0).stop 2023-07-14T16:23:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/1075522624 shutdown_connections 2023-07-14T16:23:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c17ec73900 0x55c17ec75d10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c17ec0ac70 0x55c17ec19df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0a2ffd700 0 client.17724 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:23:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c17ec094d0 0x55c17ec16890 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 --2- 172.21.15.62:0/1075522624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c17ec09e20 0x55c17ec16dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/1075522624 >> 172.21.15.62:0/1075522624 conn(0x55c17ebfeda0 msgr2=0x55c17ec1aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/1075522624 shutdown_connections 2023-07-14T16:23:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 1 -- 172.21.15.62:0/1075522624 wait complete. 2023-07-14T16:23:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.333+0000 7fb0dd9aab80 20 client.17724 trim_cache size 0 max 16384 2023-07-14T16:23:38.319 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:23:38.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 -- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ac0a11d0 msgr2=0x7fb0ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ac0a11d0 0x7fb0ac0a3680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009fd0 tx=0x7fb0bc00b040 comp rx=0 tx=0).stop 2023-07-14T16:23:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 -- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4101d10 msgr2=0x7fb0c4197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4101d10 0x7fb0c4197990 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400e970 tx=0x7fb0b400ed30 comp rx=0 tx=0).stop 2023-07-14T16:23:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 -- 172.21.15.62:0/3882686985 shutdown_connections 2023-07-14T16:23:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0ac0a11d0 0x7fb0ac0a3680 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c4103890 0x7fb0c419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0c4101d10 0x7fb0c4197990 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 --2- 172.21.15.62:0/3882686985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0c4102f00 0x7fb0c4197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 -- 172.21.15.62:0/3882686985 >> 172.21.15.62:0/3882686985 conn(0x7fb0c40fd350 msgr2=0x7fb0c40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 -- 172.21.15.62:0/3882686985 shutdown_connections 2023-07-14T16:23:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.334+0000 7fb0c9fff700 1 -- 172.21.15.62:0/3882686985 wait complete. 2023-07-14T16:23:38.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:23:38.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 -- 172.21.15.62:0/1122134117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070103830 msgr2=0x7f707010a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 --2- 172.21.15.62:0/1122134117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070103830 0x7f707010a0d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f706800b0a0 tx=0x7f706800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 -- 172.21.15.62:0/1122134117 shutdown_connections 2023-07-14T16:23:38.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 --2- 172.21.15.62:0/1122134117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070103830 0x7f707010a0d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 --2- 172.21.15.62:0/1122134117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7070101bf0 0x7f7070102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 --2- 172.21.15.62:0/1122134117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7070102e80 0x7f70701032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 -- 172.21.15.62:0/1122134117 >> 172.21.15.62:0/1122134117 conn(0x7f70700fd210 msgr2=0x7f70700ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 -- 172.21.15.62:0/1122134117 shutdown_connections 2023-07-14T16:23:38.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.593+0000 7f707499d700 1 -- 172.21.15.62:0/1122134117 wait complete. 2023-07-14T16:23:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.594+0000 7f707499d700 1 Processor -- start 2023-07-14T16:23:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f707499d700 1 -- start start 2023-07-14T16:23:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f707499d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070101bf0 0x7f7070197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f707499d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7070102e80 0x7f7070197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f707499d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7070103830 0x7f707019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f707499d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707019f3b0 con 0x7f7070101bf0 2023-07-14T16:23:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f707499d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707019f4f0 con 0x7f7070103830 2023-07-14T16:23:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f707499d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707019f680 con 0x7f7070102e80 2023-07-14T16:23:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f706f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070101bf0 0x7f7070197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f706f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070101bf0 0x7f7070197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58160/0 (socket says 172.21.15.62:58160) 2023-07-14T16:23:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.595+0000 7f706f7fe700 1 -- 172.21.15.62:0/1096534087 learned_addr learned my addr 172.21.15.62:0/1096534087 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706effd700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7070102e80 0x7f7070197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706ffff700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7070103830 0x7f707019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706f7fe700 1 -- 172.21.15.62:0/1096534087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7070102e80 msgr2=0x7f7070197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706f7fe700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7070102e80 0x7f7070197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706f7fe700 1 -- 172.21.15.62:0/1096534087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7070103830 msgr2=0x7f707019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706f7fe700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7070103830 0x7f707019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706f7fe700 1 -- 172.21.15.62:0/1096534087 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068009d00 con 0x7f7070101bf0 2023-07-14T16:23:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706f7fe700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070101bf0 0x7f7070197830 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f706000e9a0 tx=0x7f706000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.596+0000 7f706cff9700 1 -- 172.21.15.62:0/1096534087 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060009790 con 0x7f7070101bf0 2023-07-14T16:23:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.597+0000 7f706cff9700 1 -- 172.21.15.62:0/1096534087 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7060010430 con 0x7f7070101bf0 2023-07-14T16:23:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.597+0000 7f707499d700 1 -- 172.21.15.62:0/1096534087 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707019f960 con 0x7f7070101bf0 2023-07-14T16:23:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.597+0000 7f706cff9700 1 -- 172.21.15.62:0/1096534087 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706000cea0 con 0x7f7070101bf0 2023-07-14T16:23:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.597+0000 7f707499d700 1 -- 172.21.15.62:0/1096534087 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707019fe70 con 0x7f7070101bf0 2023-07-14T16:23:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.602+0000 7f706cff9700 1 -- 172.21.15.62:0/1096534087 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f70600105a0 con 0x7f7070101bf0 2023-07-14T16:23:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.602+0000 7f7089515b80 1 Processor -- start 2023-07-14T16:23:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f7089515b80 1 -- start start 2023-07-14T16:23:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f7089515b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c6f30 0x55565b1c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f7089515b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c78a0 0x55565b1c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f7089515b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c8670 0x55565b1c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f7089515b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b0c3cb0 con 0x55565b1c6f30 2023-07-14T16:23:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f7089515b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b116a40 con 0x55565b1c8670 2023-07-14T16:23:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f7089515b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b10fed0 con 0x55565b1c78a0 2023-07-14T16:23:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f706ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c6f30 0x55565b1c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f706ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c6f30 0x55565b1c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58164/0 (socket says 172.21.15.62:58164) 2023-07-14T16:23:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.603+0000 7f706ffff700 1 -- 172.21.15.62:0/2636189105 learned_addr learned my addr 172.21.15.62:0/2636189105 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706f7fe700 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c78a0 0x55565b1c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706effd700 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c8670 0x55565b1c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706ffff700 1 -- 172.21.15.62:0/2636189105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c78a0 msgr2=0x55565b1c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706ffff700 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c78a0 0x55565b1c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706ffff700 1 -- 172.21.15.62:0/2636189105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c8670 msgr2=0x55565b1c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706ffff700 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c8670 0x55565b1c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706ffff700 1 -- 172.21.15.62:0/2636189105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55565af3dde0 con 0x55565b1c6f30 2023-07-14T16:23:38.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706cff9700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70580a11f0 0x7f70580a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706cff9700 1 -- 172.21.15.62:0/1096534087 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7060014070 con 0x7f7070101bf0 2023-07-14T16:23:38.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706ffff700 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c6f30 0x55565b1c7360 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f70680092e0 tx=0x7f7068009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de3e56242442379 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.604+0000 7f706f7fe700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70580a11f0 0x7f70580a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.605+0000 7f7055ffb700 1 -- 172.21.15.62:0/2636189105 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706800e040 con 0x55565b1c6f30 2023-07-14T16:23:38.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.605+0000 7f7055ffb700 1 -- 172.21.15.62:0/2636189105 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7068007d90 con 0x55565b1c6f30 2023-07-14T16:23:38.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.605+0000 7f7055ffb700 1 -- 172.21.15.62:0/2636189105 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068003e60 con 0x55565b1c6f30 2023-07-14T16:23:38.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.605+0000 7f7089515b80 1 -- 172.21.15.62:0/2636189105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c6f30 msgr2=0x55565b1c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.605+0000 7f706f7fe700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70580a11f0 0x7f70580a36a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7060099f00 tx=0x7f70600d0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.605+0000 7f7089515b80 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c6f30 0x55565b1c7360 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f70680092e0 tx=0x7f7068009310 comp rx=0 tx=0).stop 2023-07-14T16:23:38.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 -- 172.21.15.62:0/2636189105 shutdown_connections 2023-07-14T16:23:38.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c8670 0x55565b1c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c6f30 0x55565b1c7360 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 --2- 172.21.15.62:0/2636189105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c78a0 0x55565b1c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 -- 172.21.15.62:0/2636189105 >> 172.21.15.62:0/2636189105 conn(0x55565b1b3ff0 msgr2=0x55565b1b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 -- 172.21.15.62:0/2636189105 shutdown_connections 2023-07-14T16:23:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 -- 172.21.15.62:0/2636189105 wait complete. 2023-07-14T16:23:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 -- start start 2023-07-14T16:23:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.606+0000 7f7089515b80 1 Processor -- start 2023-07-14T16:23:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f7089515b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c6f30 0x55565b1d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f7089515b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c78a0 0x55565b1d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f7089515b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c8670 0x55565b1d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f7089515b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b0afac0 con 0x55565b1c78a0 2023-07-14T16:23:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f7089515b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b0af7c0 con 0x55565b1c6f30 2023-07-14T16:23:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f7089515b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565af93920 con 0x55565b1c8670 2023-07-14T16:23:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f7089515b80 10 client.0 _ll_register_callbacks cb 0x55565b0ad760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f706effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c78a0 0x55565b1d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f706effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c78a0 0x55565b1d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58176/0 (socket says 172.21.15.62:58176) 2023-07-14T16:23:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f706effd700 1 -- 172.21.15.62:0/3734265197 learned_addr learned my addr 172.21.15.62:0/3734265197 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.607+0000 7f706ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c8670 0x55565b1d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f706ffff700 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c6f30 0x55565b1d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f706effd700 1 -- 172.21.15.62:0/3734265197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c8670 msgr2=0x55565b1d80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f706effd700 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c8670 0x55565b1d80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f706effd700 1 -- 172.21.15.62:0/3734265197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c6f30 msgr2=0x55565b1d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f706effd700 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c6f30 0x55565b1d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f706effd700 1 -- 172.21.15.62:0/3734265197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707010a940 con 0x55565b1c78a0 2023-07-14T16:23:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f706ffff700 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c6f30 0x55565b1d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f706effd700 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c78a0 0x55565b1d1060 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7064007db0 tx=0x7f7064012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d717db09e1fedc03 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.608+0000 7f704effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:38.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f704effd700 1 -- 172.21.15.62:0/3734265197 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064004a30 con 0x55565b1c78a0 2023-07-14T16:23:38.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f7089515b80 1 -- 172.21.15.62:0/3734265197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55565b0694f0 con 0x55565b1c78a0 2023-07-14T16:23:38.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f704effd700 1 -- 172.21.15.62:0/3734265197 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7064004b90 con 0x55565b1c78a0 2023-07-14T16:23:38.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f704effd700 1 -- 172.21.15.62:0/3734265197 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7064004d50 con 0x55565b1c78a0 2023-07-14T16:23:38.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f7089515b80 10 client.17736 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:38.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f7089515b80 1 -- 172.21.15.62:0/3734265197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55565af26c60 con 0x55565b1c78a0 2023-07-14T16:23:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f704effd700 1 -- 172.21.15.62:0/3734265197 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55565b0694f0 con 0x55565b1c78a0 2023-07-14T16:23:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f704effd700 1 -- 172.21.15.62:0/3734265197 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7064004eb0 con 0x55565b1c78a0 2023-07-14T16:23:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f7089515b80 10 client.17736 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.609+0000 7f7089515b80 10 client.17736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:23:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.610+0000 7f7089515b80 20 client.17736 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.610+0000 7f7089515b80 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55565b1dc900 0x55565b1ded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.610+0000 7f7089515b80 4 client.17736 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.610+0000 7f7089515b80 1 -- 172.21.15.62:0/3734265197 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55565b108eb0 con 0x55565b1dc900 2023-07-14T16:23:38.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.610+0000 7f706ffff700 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55565b1dc900 0x55565b1ded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.611+0000 7f704effd700 10 client.17736 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:23:38.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.611+0000 7f706ffff700 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55565b1dc900 0x55565b1ded10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9bd560773c6008 server_cookie=ce19ad0036f665e5 in_seq=0 out_seq=0 2023-07-14T16:23:38.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.611+0000 7f704effd700 1 -- 172.21.15.62:0/3734265197 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55565b108eb0 con 0x55565b1dc900 2023-07-14T16:23:38.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.611+0000 7f704effd700 10 client.17736 handle_command_reply: tid=0 2023-07-14T16:23:38.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.611+0000 7f7089515b80 1 client.17736 shutdown 2023-07-14T16:23:38.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 -- 172.21.15.62:0/3734265197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c78a0 msgr2=0x55565b1d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c78a0 0x55565b1d1060 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7064007db0 tx=0x7f7064012730 comp rx=0 tx=0).stop 2023-07-14T16:23:38.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 -- 172.21.15.62:0/3734265197 shutdown_connections 2023-07-14T16:23:38.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55565b1dc900 0x55565b1ded10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c8670 0x55565b1d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c6f30 0x55565b1d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 --2- 172.21.15.62:0/3734265197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c78a0 0x55565b1d1060 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f704effd700 0 client.17736 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:23:38.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 -- 172.21.15.62:0/3734265197 >> 172.21.15.62:0/3734265197 conn(0x55565b1b3ff0 msgr2=0x55565b1b8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 -- 172.21.15.62:0/3734265197 shutdown_connections 2023-07-14T16:23:38.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 1 -- 172.21.15.62:0/3734265197 wait complete. 2023-07-14T16:23:38.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.612+0000 7f7089515b80 20 client.17736 trim_cache size 0 max 16384 2023-07-14T16:23:38.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.627+0000 7f7089515b80 1 Processor -- start 2023-07-14T16:23:38.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.627+0000 7f7089515b80 1 -- start start 2023-07-14T16:23:38.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.627+0000 7f7089515b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 0x55565b1dece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.627+0000 7f7089515b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.627+0000 7f7089515b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.627+0000 7f7089515b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b1b1140 con 0x55565b1c8670 2023-07-14T16:23:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.627+0000 7f7089515b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b1ba540 con 0x55565b1c78a0 2023-07-14T16:23:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.627+0000 7f7089515b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b1ce7f0 con 0x55565b1c6f30 2023-07-14T16:23:38.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58184/0 (socket says 172.21.15.62:58184) 2023-07-14T16:23:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706ffff700 1 -- 172.21.15.62:0/2210245336 learned_addr learned my addr 172.21.15.62:0/2210245336 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706effd700 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706ffff700 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 0x55565b1dece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706effd700 1 -- 172.21.15.62:0/2210245336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 msgr2=0x55565b1dece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706effd700 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 0x55565b1dece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706effd700 1 -- 172.21.15.62:0/2210245336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 msgr2=0x55565b1cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706effd700 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706effd700 1 -- 172.21.15.62:0/2210245336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55565af64250 con 0x55565b1c78a0 2023-07-14T16:23:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.628+0000 7f706ffff700 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.629+0000 7f706effd700 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d8060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7064005360 tx=0x7f7064005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a0a66203969e39 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.629+0000 7f70567fc700 1 -- 172.21.15.62:0/2210245336 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064005680 con 0x55565b1c78a0 2023-07-14T16:23:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.629+0000 7f70567fc700 1 -- 172.21.15.62:0/2210245336 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7064012b30 con 0x55565b1c78a0 2023-07-14T16:23:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.629+0000 7f70567fc700 1 -- 172.21.15.62:0/2210245336 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70640109f0 con 0x55565b1c78a0 2023-07-14T16:23:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 -- 172.21.15.62:0/2210245336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 msgr2=0x55565b1d8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d8060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7064005360 tx=0x7f7064005c90 comp rx=0 tx=0).stop 2023-07-14T16:23:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 -- 172.21.15.62:0/2210245336 shutdown_connections 2023-07-14T16:23:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 0x55565b1dece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 --2- 172.21.15.62:0/2210245336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d8060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 -- 172.21.15.62:0/2210245336 >> 172.21.15.62:0/2210245336 conn(0x55565b1b3ff0 msgr2=0x55565b1cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 -- 172.21.15.62:0/2210245336 shutdown_connections 2023-07-14T16:23:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 -- 172.21.15.62:0/2210245336 wait complete. 2023-07-14T16:23:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 -- start start 2023-07-14T16:23:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.630+0000 7f7089515b80 1 Processor -- start 2023-07-14T16:23:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f7089515b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 0x55565b1d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f7089515b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f7089515b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1d4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f7089515b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b1d4b50 con 0x55565b1c8670 2023-07-14T16:23:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f7089515b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b1d4cc0 con 0x55565b1c78a0 2023-07-14T16:23:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f7089515b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565b1d4e80 con 0x55565b1c6f30 2023-07-14T16:23:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f7089515b80 10 client.0 _ll_register_callbacks cb 0x7f7064007810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:38.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f706ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1d4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f706ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1d4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58194/0 (socket says 172.21.15.62:58194) 2023-07-14T16:23:38.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f706ffff700 1 -- 172.21.15.62:0/440471247 learned_addr learned my addr 172.21.15.62:0/440471247 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f706ffff700 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 0x55565b1d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.631+0000 7f706effd700 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.632+0000 7f706effd700 1 -- 172.21.15.62:0/440471247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 msgr2=0x55565b1d79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.632+0000 7f706effd700 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 0x55565b1d79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.632+0000 7f706effd700 1 -- 172.21.15.62:0/440471247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 msgr2=0x55565b1d4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.632+0000 7f706effd700 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1d4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.632+0000 7f706effd700 1 -- 172.21.15.62:0/440471247 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706802d040 con 0x55565b1c78a0 2023-07-14T16:23:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.632+0000 7f706effd700 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d7f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70640053b0 tx=0x7f7064010980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4051686a42be8f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.632+0000 7f704effd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:23:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.633+0000 7f704effd700 1 -- 172.21.15.62:0/440471247 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064003680 con 0x55565b1c78a0 2023-07-14T16:23:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.633+0000 7f704effd700 1 -- 172.21.15.62:0/440471247 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f70640037e0 con 0x55565b1c78a0 2023-07-14T16:23:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.633+0000 7f7089515b80 1 -- 172.21.15.62:0/440471247 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7068007d90 con 0x55565b1c78a0 2023-07-14T16:23:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.633+0000 7f704effd700 1 -- 172.21.15.62:0/440471247 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70640039a0 con 0x55565b1c78a0 2023-07-14T16:23:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.633+0000 7f7089515b80 10 client.27398 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.633+0000 7f7089515b80 1 -- 172.21.15.62:0/440471247 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55565af26c60 con 0x55565b1c78a0 2023-07-14T16:23:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.633+0000 7f704effd700 1 -- 172.21.15.62:0/440471247 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7068007d90 con 0x55565b1c78a0 2023-07-14T16:23:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.634+0000 7f704effd700 1 -- 172.21.15.62:0/440471247 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7064025a60 con 0x55565b1c78a0 2023-07-14T16:23:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.634+0000 7f7089515b80 10 client.27398 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.634+0000 7f7089515b80 10 client.27398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:23:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.634+0000 7f7089515b80 20 client.27398 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.634+0000 7f7089515b80 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55565b231ab0 0x55565b233ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.634+0000 7f7089515b80 4 client.27398 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:23:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.634+0000 7f7089515b80 1 -- 172.21.15.62:0/440471247 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55565b0cb050 con 0x55565b231ab0 2023-07-14T16:23:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.634+0000 7f706ffff700 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55565b231ab0 0x55565b233ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f704effd700 10 client.27398 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:23:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f706ffff700 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55565b231ab0 0x55565b233ec0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a47ceec3fa4e0692 server_cookie=9e51060239817229 in_seq=0 out_seq=0 2023-07-14T16:23:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f704effd700 1 -- 172.21.15.62:0/440471247 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55565b0cb050 con 0x55565b231ab0 2023-07-14T16:23:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f704effd700 10 client.27398 handle_command_reply: tid=0 2023-07-14T16:23:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 client.27398 shutdown 2023-07-14T16:23:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 -- 172.21.15.62:0/440471247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 msgr2=0x55565b1d7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d7f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70640053b0 tx=0x7f7064010980 comp rx=0 tx=0).stop 2023-07-14T16:23:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 -- 172.21.15.62:0/440471247 shutdown_connections 2023-07-14T16:23:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55565b231ab0 0x55565b233ec0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55565b1c8670 0x55565b1d4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55565b1c6f30 0x55565b1d79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 --2- 172.21.15.62:0/440471247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55565b1c78a0 0x55565b1d7f10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f704effd700 0 client.27398 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:23:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.635+0000 7f7089515b80 1 -- 172.21.15.62:0/440471247 >> 172.21.15.62:0/440471247 conn(0x55565b1b3ff0 msgr2=0x55565b1cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.636+0000 7f7089515b80 1 -- 172.21.15.62:0/440471247 shutdown_connections 2023-07-14T16:23:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.636+0000 7f7089515b80 1 -- 172.21.15.62:0/440471247 wait complete. 2023-07-14T16:23:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.636+0000 7f7089515b80 20 client.27398 trim_cache size 0 max 16384 2023-07-14T16:23:38.626 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:23:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.636+0000 7f707499d700 1 -- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70580a11f0 msgr2=0x7f70580a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.636+0000 7f707499d700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70580a11f0 0x7f70580a36a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7060099f00 tx=0x7f70600d0040 comp rx=0 tx=0).stop 2023-07-14T16:23:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.636+0000 7f707499d700 1 -- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070101bf0 msgr2=0x7f7070197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.636+0000 7f707499d700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070101bf0 0x7f7070197830 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f706000e9a0 tx=0x7f706000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.637+0000 7f707499d700 1 -- 172.21.15.62:0/1096534087 shutdown_connections 2023-07-14T16:23:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.637+0000 7f707499d700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f70580a11f0 0x7f70580a36a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.637+0000 7f707499d700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7070103830 0x7f707019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.637+0000 7f707499d700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7070101bf0 0x7f7070197830 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.637+0000 7f707499d700 1 --2- 172.21.15.62:0/1096534087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7070102e80 0x7f7070197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.637+0000 7f707499d700 1 -- 172.21.15.62:0/1096534087 >> 172.21.15.62:0/1096534087 conn(0x7f70700fd210 msgr2=0x7f7070106360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.637+0000 7f707499d700 1 -- 172.21.15.62:0/1096534087 shutdown_connections 2023-07-14T16:23:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.637+0000 7f707499d700 1 -- 172.21.15.62:0/1096534087 wait complete. 2023-07-14T16:23:38.632 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:23:38.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.902+0000 7f8243bff700 1 -- 172.21.15.62:0/2161956736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 msgr2=0x7f823c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.902+0000 7f8243bff700 1 --2- 172.21.15.62:0/2161956736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 0x7f823c101150 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f822c009990 tx=0x7f822c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.902+0000 7f8243bff700 1 -- 172.21.15.62:0/2161956736 shutdown_connections 2023-07-14T16:23:38.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.902+0000 7f8243bff700 1 --2- 172.21.15.62:0/2161956736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c101690 0x7f823c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.902+0000 7f8243bff700 1 --2- 172.21.15.62:0/2161956736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c0ffb10 0x7f823c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.902+0000 7f8243bff700 1 --2- 172.21.15.62:0/2161956736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 0x7f823c101150 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.902+0000 7f8243bff700 1 -- 172.21.15.62:0/2161956736 >> 172.21.15.62:0/2161956736 conn(0x7f823c0fb130 msgr2=0x7f823c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.903+0000 7f8243bff700 1 -- 172.21.15.62:0/2161956736 shutdown_connections 2023-07-14T16:23:38.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.903+0000 7f8243bff700 1 -- 172.21.15.62:0/2161956736 wait complete. 2023-07-14T16:23:38.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.904+0000 7f8243bff700 1 Processor -- start 2023-07-14T16:23:38.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.904+0000 7f8243bff700 1 -- start start 2023-07-14T16:23:38.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.904+0000 7f8243bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c0ffb10 0x7f823c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.904+0000 7f8243bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 0x7f823c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.904+0000 7f8243bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c101690 0x7f823c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.904+0000 7f8243bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c19f4a0 con 0x7f823c100d00 2023-07-14T16:23:38.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.904+0000 7f8243bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c19f5e0 con 0x7f823c0ffb10 2023-07-14T16:23:38.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.904+0000 7f8243bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c19f770 con 0x7f823c101690 2023-07-14T16:23:38.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 0x7f823c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 0x7f823c197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58204/0 (socket says 172.21.15.62:58204) 2023-07-14T16:23:38.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824119a700 1 -- 172.21.15.62:0/3791047047 learned_addr learned my addr 172.21.15.62:0/3791047047 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824219c700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c101690 0x7f823c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824199b700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c0ffb10 0x7f823c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824119a700 1 -- 172.21.15.62:0/3791047047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c101690 msgr2=0x7f823c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824119a700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c101690 0x7f823c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824119a700 1 -- 172.21.15.62:0/3791047047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c0ffb10 msgr2=0x7f823c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824119a700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c0ffb10 0x7f823c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.905+0000 7f824119a700 1 -- 172.21.15.62:0/3791047047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c0095f0 con 0x7f823c100d00 2023-07-14T16:23:38.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.906+0000 7f824119a700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 0x7f823c197e60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f822c005740 tx=0x7f822c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.906+0000 7f8232ffd700 1 -- 172.21.15.62:0/3791047047 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c01d070 con 0x7f823c100d00 2023-07-14T16:23:38.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.906+0000 7f8232ffd700 1 -- 172.21.15.62:0/3791047047 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f822c00bd50 con 0x7f823c100d00 2023-07-14T16:23:38.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.906+0000 7f8243bff700 1 -- 172.21.15.62:0/3791047047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c19f9f0 con 0x7f823c100d00 2023-07-14T16:23:38.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.906+0000 7f8232ffd700 1 -- 172.21.15.62:0/3791047047 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c022db0 con 0x7f823c100d00 2023-07-14T16:23:38.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.906+0000 7f8243bff700 1 -- 172.21.15.62:0/3791047047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c19fe70 con 0x7f823c100d00 2023-07-14T16:23:38.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.910+0000 7f8232ffd700 1 -- 172.21.15.62:0/3791047047 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f822c022470 con 0x7f823c100d00 2023-07-14T16:23:38.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.911+0000 7f82575e1b80 1 Processor -- start 2023-07-14T16:23:38.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.911+0000 7f82575e1b80 1 -- start start 2023-07-14T16:23:38.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.911+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208d4d0 0x559f3208d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.911+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f3208e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.911+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208ec70 0x559f3208f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.911+0000 7f82575e1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f31f92cb0 con 0x559f3208d4d0 2023-07-14T16:23:38.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.911+0000 7f82575e1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f31fe5a40 con 0x559f3208de20 2023-07-14T16:23:38.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.911+0000 7f82575e1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f31fdeed0 con 0x559f3208ec70 2023-07-14T16:23:38.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208d4d0 0x559f3208d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208d4d0 0x559f3208d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58206/0 (socket says 172.21.15.62:58206) 2023-07-14T16:23:38.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 -- 172.21.15.62:0/1164183212 learned_addr learned my addr 172.21.15.62:0/1164183212 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f8232ffd700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82280a11d0 0x7f82280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824199b700 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f3208e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824119a700 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208ec70 0x559f3208f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f8232ffd700 1 -- 172.21.15.62:0/3791047047 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f822c0cde50 con 0x7f823c100d00 2023-07-14T16:23:38.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82280a11d0 0x7f82280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 -- 172.21.15.62:0/1164183212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208ec70 msgr2=0x559f3208f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208ec70 0x559f3208f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 -- 172.21.15.62:0/1164183212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 msgr2=0x559f3208e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f3208e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824219c700 1 -- 172.21.15.62:0/1164183212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f31e0cde0 con 0x559f3208d4d0 2023-07-14T16:23:38.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.912+0000 7f824199b700 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f3208e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:38.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.913+0000 7f824219c700 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208d4d0 0x559f3208d8e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8234013c80 tx=0x7f8234013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795253e0b2ba26bb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.913+0000 7f824219c700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82280a11d0 0x7f82280a3680 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8234009000 tx=0x7f823401f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.913+0000 7f8227fff700 1 -- 172.21.15.62:0/1164183212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400e040 con 0x559f3208d4d0 2023-07-14T16:23:38.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.913+0000 7f8227fff700 1 -- 172.21.15.62:0/1164183212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f823400f040 con 0x559f3208d4d0 2023-07-14T16:23:38.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.913+0000 7f8227fff700 1 -- 172.21.15.62:0/1164183212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823401a6a0 con 0x559f3208d4d0 2023-07-14T16:23:38.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 -- 172.21.15.62:0/1164183212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208d4d0 msgr2=0x559f3208d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208d4d0 0x559f3208d8e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8234013c80 tx=0x7f8234013f90 comp rx=0 tx=0).stop 2023-07-14T16:23:38.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 -- 172.21.15.62:0/1164183212 shutdown_connections 2023-07-14T16:23:38.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208ec70 0x559f3208f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208d4d0 0x559f3208d8e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1164183212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f3208e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 -- 172.21.15.62:0/1164183212 >> 172.21.15.62:0/1164183212 conn(0x559f32082da0 msgr2=0x559f320881d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 -- 172.21.15.62:0/1164183212 shutdown_connections 2023-07-14T16:23:38.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.914+0000 7f82575e1b80 1 -- 172.21.15.62:0/1164183212 wait complete. 2023-07-14T16:23:38.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.915+0000 7f82575e1b80 1 -- start start 2023-07-14T16:23:38.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.915+0000 7f82575e1b80 1 Processor -- start 2023-07-14T16:23:38.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 0x559f32097150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f32097690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f82575e1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f31f7eac0 con 0x559f3208ec70 2023-07-14T16:23:38.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f82575e1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f31f7e7c0 con 0x559f3208de20 2023-07-14T16:23:38.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f82575e1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f31e62920 con 0x559f3208d4d0 2023-07-14T16:23:38.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f82575e1b80 10 client.0 _ll_register_callbacks cb 0x559f31f7c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:38.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f824119a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f32097690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f824119a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f32097690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:55062/0 (socket says 172.21.15.62:55062) 2023-07-14T16:23:38.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f824119a700 1 -- 172.21.15.62:0/3143221878 learned_addr learned my addr 172.21.15.62:0/3143221878 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f824199b700 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 0x559f32097150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.916+0000 7f824219c700 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f824119a700 1 -- 172.21.15.62:0/3143221878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 msgr2=0x559f32097150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f824119a700 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 0x559f32097150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f824119a700 1 -- 172.21.15.62:0/3143221878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 msgr2=0x559f3209e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f824119a700 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f824119a700 1 -- 172.21.15.62:0/3143221878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82380098e0 con 0x559f3208de20 2023-07-14T16:23:38.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f824119a700 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f32097690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f822c003ec0 tx=0x7f822c02af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3856a5dd9fed4943 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f82257fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:23:38.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f82257fa700 1 -- 172.21.15.62:0/3143221878 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c09b300 con 0x559f3208de20 2023-07-14T16:23:38.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f82257fa700 1 -- 172.21.15.62:0/3143221878 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f822c09f330 con 0x559f3208de20 2023-07-14T16:23:38.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.917+0000 7f82575e1b80 1 -- 172.21.15.62:0/3143221878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f31f384f0 con 0x559f3208de20 2023-07-14T16:23:38.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82257fa700 1 -- 172.21.15.62:0/3143221878 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f822c09f150 con 0x559f3208de20 2023-07-14T16:23:38.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82575e1b80 10 client.27410 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:38.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82575e1b80 1 -- 172.21.15.62:0/3143221878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f8234013830 con 0x559f3208de20 2023-07-14T16:23:38.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82257fa700 1 -- 172.21.15.62:0/3143221878 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559f31f384f0 con 0x559f3208de20 2023-07-14T16:23:38.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82257fa700 1 -- 172.21.15.62:0/3143221878 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f822c0a3230 con 0x559f3208de20 2023-07-14T16:23:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82575e1b80 10 client.27410 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82575e1b80 10 client.27410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82575e1b80 20 client.27410 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559f320a2a50 0x559f320a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82575e1b80 4 client.27410 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.918+0000 7f82575e1b80 1 -- 172.21.15.62:0/3143221878 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f31fd7eb0 con 0x559f320a2a50 2023-07-14T16:23:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.919+0000 7f824199b700 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559f320a2a50 0x559f320a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.919+0000 7f82257fa700 10 client.27410 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.919+0000 7f824199b700 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559f320a2a50 0x559f320a4e60 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162d2b79064dd460 server_cookie=f7a7ca3972ffb14a in_seq=0 out_seq=0 2023-07-14T16:23:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82257fa700 1 -- 172.21.15.62:0/3143221878 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559f31fd7eb0 con 0x559f320a2a50 2023-07-14T16:23:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82257fa700 10 client.27410 handle_command_reply: tid=0 2023-07-14T16:23:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 client.27410 shutdown 2023-07-14T16:23:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 -- 172.21.15.62:0/3143221878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 msgr2=0x559f32097690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f32097690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f822c003ec0 tx=0x7f822c02af90 comp rx=0 tx=0).stop 2023-07-14T16:23:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 -- 172.21.15.62:0/3143221878 shutdown_connections 2023-07-14T16:23:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 0x559f32097150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559f320a2a50 0x559f320a4e60 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3143221878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208de20 0x559f32097690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82575e1b80 1 -- 172.21.15.62:0/3143221878 >> 172.21.15.62:0/3143221878 conn(0x559f32082da0 msgr2=0x559f320878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.920+0000 7f82257fa700 0 client.27410 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.921+0000 7f82575e1b80 1 -- 172.21.15.62:0/3143221878 shutdown_connections 2023-07-14T16:23:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.921+0000 7f82575e1b80 1 -- 172.21.15.62:0/3143221878 wait complete. 2023-07-14T16:23:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.921+0000 7f82575e1b80 20 client.27410 trim_cache size 0 max 16384 2023-07-14T16:23:38.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f82575e1b80 1 Processor -- start 2023-07-14T16:23:38.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f82575e1b80 1 -- start start 2023-07-14T16:23:38.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 0x559f3208d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208de20 0x559f3209e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208ec70 0x559f32095dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f82575e1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f32098340 con 0x559f3208de20 2023-07-14T16:23:38.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f82575e1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f32098050 con 0x559f3208ec70 2023-07-14T16:23:38.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f82575e1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f32096cc0 con 0x559f3208d4d0 2023-07-14T16:23:38.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f824119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208de20 0x559f3209e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f824119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208de20 0x559f3209e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58222/0 (socket says 172.21.15.62:58222) 2023-07-14T16:23:38.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f824119a700 1 -- 172.21.15.62:0/3269731965 learned_addr learned my addr 172.21.15.62:0/3269731965 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f824199b700 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 0x559f3208d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.939+0000 7f824219c700 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208ec70 0x559f32095dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f824119a700 1 -- 172.21.15.62:0/3269731965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 msgr2=0x559f3208d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f824119a700 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 0x559f3208d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f824119a700 1 -- 172.21.15.62:0/3269731965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208ec70 msgr2=0x559f32095dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f824119a700 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208ec70 0x559f32095dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f824119a700 1 -- 172.21.15.62:0/3269731965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f31df5c60 con 0x559f3208de20 2023-07-14T16:23:38.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f824119a700 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208de20 0x559f3209e690 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f822c09b290 tx=0x7f822c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08174d724387cfa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f8230bff700 1 -- 172.21.15.62:0/3269731965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c09f150 con 0x559f3208de20 2023-07-14T16:23:38.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f8230bff700 1 -- 172.21.15.62:0/3269731965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f822c09f330 con 0x559f3208de20 2023-07-14T16:23:38.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.940+0000 7f8230bff700 1 -- 172.21.15.62:0/3269731965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c0a33a0 con 0x559f3208de20 2023-07-14T16:23:38.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 -- 172.21.15.62:0/3269731965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208de20 msgr2=0x559f3209e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208de20 0x559f3209e690 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f822c09b290 tx=0x7f822c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 -- 172.21.15.62:0/3269731965 shutdown_connections 2023-07-14T16:23:38.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208ec70 0x559f32095dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208d4d0 0x559f3208d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 --2- 172.21.15.62:0/3269731965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208de20 0x559f3209e690 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 -- 172.21.15.62:0/3269731965 >> 172.21.15.62:0/3269731965 conn(0x559f32082da0 msgr2=0x559f320a9780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 -- 172.21.15.62:0/3269731965 shutdown_connections 2023-07-14T16:23:38.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.941+0000 7f82575e1b80 1 -- 172.21.15.62:0/3269731965 wait complete. 2023-07-14T16:23:38.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.942+0000 7f82575e1b80 1 -- start start 2023-07-14T16:23:38.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.942+0000 7f82575e1b80 1 Processor -- start 2023-07-14T16:23:38.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208d4d0 0x559f3209df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208de20 0x559f3209e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f82575e1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f82575e1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3209ca90 con 0x559f3208ec70 2023-07-14T16:23:38.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f82575e1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3209cc00 con 0x559f3208d4d0 2023-07-14T16:23:38.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f82575e1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3209cdc0 con 0x559f3208de20 2023-07-14T16:23:38.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f82575e1b80 10 client.0 _ll_register_callbacks cb 0x7f822c0976e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:38.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824219c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824219c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58232/0 (socket says 172.21.15.62:58232) 2023-07-14T16:23:38.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824219c700 1 -- 172.21.15.62:0/1226522991 learned_addr learned my addr 172.21.15.62:0/1226522991 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:38.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824199b700 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208d4d0 0x559f3209df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824119a700 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208de20 0x559f3209e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824219c700 1 -- 172.21.15.62:0/1226522991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208de20 msgr2=0x559f3209e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824219c700 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208de20 0x559f3209e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824219c700 1 -- 172.21.15.62:0/1226522991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208d4d0 msgr2=0x559f3209df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:38.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.943+0000 7f824219c700 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208d4d0 0x559f3209df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.944+0000 7f824219c700 1 -- 172.21.15.62:0/1226522991 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8234013830 con 0x559f3208ec70 2023-07-14T16:23:38.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.944+0000 7f824219c700 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209c550 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8234013c80 tx=0x7f8234003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3560b6f7ee05ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:38.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.944+0000 7f82257fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:38.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.944+0000 7f82257fa700 1 -- 172.21.15.62:0/1226522991 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400e040 con 0x559f3208ec70 2023-07-14T16:23:38.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.944+0000 7f82575e1b80 1 -- 172.21.15.62:0/1226522991 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f823400f040 con 0x559f3208ec70 2023-07-14T16:23:38.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.944+0000 7f82257fa700 1 -- 172.21.15.62:0/1226522991 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f823401e070 con 0x559f3208ec70 2023-07-14T16:23:38.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82257fa700 1 -- 172.21.15.62:0/1226522991 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8234007710 con 0x559f3208ec70 2023-07-14T16:23:38.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82575e1b80 10 client.17760 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:38.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82575e1b80 1 -- 172.21.15.62:0/1226522991 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559f31e33250 con 0x559f3208ec70 2023-07-14T16:23:38.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82257fa700 1 -- 172.21.15.62:0/1226522991 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f823400f040 con 0x559f3208ec70 2023-07-14T16:23:38.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82257fa700 1 -- 172.21.15.62:0/1226522991 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8234004380 con 0x559f3208ec70 2023-07-14T16:23:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82575e1b80 10 client.17760 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82575e1b80 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82575e1b80 20 client.17760 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559f320f7ae0 0x559f320f9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82575e1b80 4 client.17760 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:23:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.945+0000 7f82575e1b80 1 -- 172.21.15.62:0/1226522991 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559f31f9a050 con 0x559f320f7ae0 2023-07-14T16:23:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.946+0000 7f824199b700 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559f320f7ae0 0x559f320f9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:38.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.946+0000 7f824199b700 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559f320f7ae0 0x559f320f9ef0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd32aa6d1dde1b6a server_cookie=1746fb678d2afbf2 in_seq=0 out_seq=0 2023-07-14T16:23:38.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.946+0000 7f82257fa700 10 client.17760 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:38.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82257fa700 1 -- 172.21.15.62:0/1226522991 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559f31f9a050 con 0x559f320f7ae0 2023-07-14T16:23:38.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82257fa700 10 client.17760 handle_command_reply: tid=0 2023-07-14T16:23:38.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 client.17760 shutdown 2023-07-14T16:23:38.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 -- 172.21.15.62:0/1226522991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 msgr2=0x559f3209c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209c550 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8234013c80 tx=0x7f8234003f90 comp rx=0 tx=0).stop 2023-07-14T16:23:38.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 -- 172.21.15.62:0/1226522991 shutdown_connections 2023-07-14T16:23:38.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f3208ec70 0x559f3209c550 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559f320f7ae0 0x559f320f9ef0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559f3208d4d0 0x559f3209df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 --2- 172.21.15.62:0/1226522991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f3208de20 0x559f3209e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 -- 172.21.15.62:0/1226522991 >> 172.21.15.62:0/1226522991 conn(0x559f32082da0 msgr2=0x559f32095dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82257fa700 0 client.17760 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:38.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 -- 172.21.15.62:0/1226522991 shutdown_connections 2023-07-14T16:23:38.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 1 -- 172.21.15.62:0/1226522991 wait complete. 2023-07-14T16:23:38.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.947+0000 7f82575e1b80 20 client.17760 trim_cache size 0 max 16384 2023-07-14T16:23:38.930 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:23:38.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 -- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82280a11d0 msgr2=0x7f82280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82280a11d0 0x7f82280a3680 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8234009000 tx=0x7f823401f000 comp rx=0 tx=0).stop 2023-07-14T16:23:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 -- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 msgr2=0x7f823c197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 0x7f823c197e60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f822c005740 tx=0x7f822c005770 comp rx=0 tx=0).stop 2023-07-14T16:23:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 -- 172.21.15.62:0/3791047047 shutdown_connections 2023-07-14T16:23:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f823c101690 0x7f823c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82280a11d0 0x7f82280a3680 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f823c0ffb10 0x7f823c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 --2- 172.21.15.62:0/3791047047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f823c100d00 0x7f823c197e60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:38.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 -- 172.21.15.62:0/3791047047 >> 172.21.15.62:0/3791047047 conn(0x7f823c0fb130 msgr2=0x7f823c106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:38.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 -- 172.21.15.62:0/3791047047 shutdown_connections 2023-07-14T16:23:38.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:38.948+0000 7f8243bff700 1 -- 172.21.15.62:0/3791047047 wait complete. 2023-07-14T16:23:38.939 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:38 smithi062 ceph-mon[95543]: pgmap v597: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:23:38.939 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:38 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2394535404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:23:39.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:38 smithi081 ceph-mon[127795]: pgmap v597: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:23:39.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:38 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2394535404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:23:39.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:38 smithi164 ceph-mon[97427]: pgmap v597: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:23:39.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:38 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2394535404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:23:39.942 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:23:39.942 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:23:40.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/2372417383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd54072ef0 msgr2=0x7fdd54073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/2372417383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd54072ef0 0x7fdd54073360 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009990 tx=0x7fdd48009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/2372417383 shutdown_connections 2023-07-14T16:23:40.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/2372417383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd541079b0 0x7fdd54109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/2372417383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd540744d0 0x7fdd54072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/2372417383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd54072ef0 0x7fdd54073360 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/2372417383 >> 172.21.15.62:0/2372417383 conn(0x7fdd540fb120 msgr2=0x7fdd540fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/2372417383 shutdown_connections 2023-07-14T16:23:40.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.214+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/2372417383 wait complete. 2023-07-14T16:23:40.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.215+0000 7fdd5a9ff700 1 Processor -- start 2023-07-14T16:23:40.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.215+0000 7fdd5a9ff700 1 -- start start 2023-07-14T16:23:40.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd5a9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd54072ef0 0x7fdd541978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd5a9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd540744d0 0x7fdd54197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd5a9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd541079b0 0x7fdd5419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd5a9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419f420 con 0x7fdd540744d0 2023-07-14T16:23:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd5a9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419f560 con 0x7fdd54072ef0 2023-07-14T16:23:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd5a9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419f6f0 con 0x7fdd541079b0 2023-07-14T16:23:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd540744d0 0x7fdd54197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd540744d0 0x7fdd54197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58258/0 (socket says 172.21.15.62:58258) 2023-07-14T16:23:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd537fe700 1 -- 172.21.15.62:0/3187591662 learned_addr learned my addr 172.21.15.62:0/3187591662 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd53fff700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd54072ef0 0x7fdd541978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd541079b0 0x7fdd5419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd537fe700 1 -- 172.21.15.62:0/3187591662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd541079b0 msgr2=0x7fdd5419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd537fe700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd541079b0 0x7fdd5419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd537fe700 1 -- 172.21.15.62:0/3187591662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd54072ef0 msgr2=0x7fdd541978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd537fe700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd54072ef0 0x7fdd541978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd537fe700 1 -- 172.21.15.62:0/3187591662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd480095f0 con 0x7fdd540744d0 2023-07-14T16:23:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd53fff700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd54072ef0 0x7fdd541978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.216+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd541079b0 0x7fdd5419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.217+0000 7fdd537fe700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd540744d0 0x7fdd54197de0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000c00 tx=0x7fdd48003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.217+0000 7fdd517fa700 1 -- 172.21.15.62:0/3187591662 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4801d070 con 0x7fdd540744d0 2023-07-14T16:23:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.217+0000 7fdd517fa700 1 -- 172.21.15.62:0/3187591662 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd4800bd70 con 0x7fdd540744d0 2023-07-14T16:23:40.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.217+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/3187591662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5419f970 con 0x7fdd540744d0 2023-07-14T16:23:40.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.217+0000 7fdd517fa700 1 -- 172.21.15.62:0/3187591662 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48022e30 con 0x7fdd540744d0 2023-07-14T16:23:40.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.217+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/3187591662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5419fdf0 con 0x7fdd540744d0 2023-07-14T16:23:40.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.221+0000 7fdd517fa700 1 -- 172.21.15.62:0/3187591662 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdd48022470 con 0x7fdd540744d0 2023-07-14T16:23:40.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd6e39ab80 1 Processor -- start 2023-07-14T16:23:40.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd6e39ab80 1 -- start start 2023-07-14T16:23:40.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6550 0x55cdaf2e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf1ebd30 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf23eac0 con 0x55cdaf2e6550 2023-07-14T16:23:40.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf237f50 con 0x55cdaf2e6ea0 2023-07-14T16:23:40.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.223+0000 7fdd517fa700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd3c0a11d0 0x7fdd3c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd517fa700 1 -- 172.21.15.62:0/3187591662 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd480cde10 con 0x7fdd540744d0 2023-07-14T16:23:40.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd53fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd53fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:57340/0 (socket says 172.21.15.62:57340) 2023-07-14T16:23:40.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd53fff700 1 -- 172.21.15.62:0/2098716884 learned_addr learned my addr 172.21.15.62:0/2098716884 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6550 0x55cdaf2e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd3c0a11d0 0x7fdd3c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd58f9c700 1 -- 172.21.15.62:0/2098716884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 msgr2=0x55cdaf2e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd58f9c700 1 -- 172.21.15.62:0/2098716884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 msgr2=0x55cdaf2e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.224+0000 7fdd58f9c700 1 -- 172.21.15.62:0/2098716884 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdaf065e60 con 0x55cdaf2e6550 2023-07-14T16:23:40.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.225+0000 7fdd537fe700 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:40.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.225+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6550 0x55cdaf2e6960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0129f0 tx=0x7fdd4c012d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391ccd003f3b4187 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.225+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd3c0a11d0 0x7fdd3c0a3680 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00e000 tx=0x7fdd4c01a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.226+0000 7fdd3a7fc700 1 -- 172.21.15.62:0/2098716884 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c015e90 con 0x55cdaf2e6550 2023-07-14T16:23:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.226+0000 7fdd3a7fc700 1 -- 172.21.15.62:0/2098716884 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd4c00d040 con 0x55cdaf2e6550 2023-07-14T16:23:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.226+0000 7fdd3a7fc700 1 -- 172.21.15.62:0/2098716884 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004720 con 0x55cdaf2e6550 2023-07-14T16:23:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.226+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/2098716884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6550 msgr2=0x55cdaf2e6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.226+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6550 0x55cdaf2e6960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0129f0 tx=0x7fdd4c012d00 comp rx=0 tx=0).stop 2023-07-14T16:23:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.227+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/2098716884 shutdown_connections 2023-07-14T16:23:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.227+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.227+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6550 0x55cdaf2e6960 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.227+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/2098716884 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.227+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/2098716884 >> 172.21.15.62:0/2098716884 conn(0x55cdaf2dbe20 msgr2=0x55cdaf2e1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.227+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/2098716884 shutdown_connections 2023-07-14T16:23:40.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.227+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/2098716884 wait complete. 2023-07-14T16:23:40.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.228+0000 7fdd6e39ab80 1 -- start start 2023-07-14T16:23:40.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.228+0000 7fdd6e39ab80 1 Processor -- start 2023-07-14T16:23:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf2f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf1d7b40 con 0x55cdaf2e6ea0 2023-07-14T16:23:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf1d7840 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf0bb9a0 con 0x55cdaf2e6550 2023-07-14T16:23:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd6e39ab80 10 client.0 _ll_register_callbacks cb 0x55cdaf1d57e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58276/0 (socket says 172.21.15.62:58276) 2023-07-14T16:23:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd537fe700 1 -- 172.21.15.62:0/89633648 learned_addr learned my addr 172.21.15.62:0/89633648 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.229+0000 7fdd53fff700 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf2f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd53fff700 1 -- 172.21.15.62:0/89633648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 msgr2=0x55cdaf2f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd53fff700 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd53fff700 1 -- 172.21.15.62:0/89633648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6ea0 msgr2=0x55cdaf2f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd53fff700 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd53fff700 1 -- 172.21.15.62:0/89633648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5419fdf0 con 0x55cdaf2e6550 2023-07-14T16:23:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd537fe700 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd53fff700 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf2f0200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd44011a30 tx=0x7fdd44011df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bddd9e674a3448c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.230+0000 7fdd337fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:23:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.231+0000 7fdd337fe700 1 -- 172.21.15.62:0/89633648 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44008ee0 con 0x55cdaf2e6550 2023-07-14T16:23:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.231+0000 7fdd337fe700 1 -- 172.21.15.62:0/89633648 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd44004440 con 0x55cdaf2e6550 2023-07-14T16:23:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.231+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/89633648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdaf191570 con 0x55cdaf2e6550 2023-07-14T16:23:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.231+0000 7fdd337fe700 1 -- 172.21.15.62:0/89633648 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd4401f030 con 0x55cdaf2e6550 2023-07-14T16:23:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.231+0000 7fdd6e39ab80 10 client.27319 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.231+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/89633648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fdd4c00f040 con 0x55cdaf2e6550 2023-07-14T16:23:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.231+0000 7fdd337fe700 1 -- 172.21.15.62:0/89633648 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cdaf191570 con 0x55cdaf2e6550 2023-07-14T16:23:40.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.232+0000 7fdd337fe700 1 -- 172.21.15.62:0/89633648 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdd44003af0 con 0x55cdaf2e6550 2023-07-14T16:23:40.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.232+0000 7fdd6e39ab80 10 client.27319 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:40.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.232+0000 7fdd6e39ab80 10 client.27319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:40.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.232+0000 7fdd6e39ab80 20 client.27319 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:40.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.232+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdaf2fbf90 0x55cdaf2fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.232+0000 7fdd6e39ab80 4 client.27319 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:40.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.232+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/89633648 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdaf230f30 con 0x55cdaf2fbf90 2023-07-14T16:23:40.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.232+0000 7fdd537fe700 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdaf2fbf90 0x55cdaf2fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.233+0000 7fdd537fe700 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdaf2fbf90 0x55cdaf2fe3a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1d98251b699938 server_cookie=f5039e98ad37f0b0 in_seq=0 out_seq=0 2023-07-14T16:23:40.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.233+0000 7fdd337fe700 10 client.27319 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:40.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.234+0000 7fdd337fe700 1 -- 172.21.15.62:0/89633648 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cdaf230f30 con 0x55cdaf2fbf90 2023-07-14T16:23:40.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.234+0000 7fdd337fe700 10 client.27319 handle_command_reply: tid=0 2023-07-14T16:23:40.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.234+0000 7fdd6e39ab80 1 client.27319 shutdown 2023-07-14T16:23:40.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/89633648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 msgr2=0x55cdaf2f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf2f0200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd44011a30 tx=0x7fdd44011df0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/89633648 shutdown_connections 2023-07-14T16:23:40.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdaf2fbf90 0x55cdaf2fe3a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf2f0200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/89633648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd337fe700 0 client.27319 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/89633648 >> 172.21.15.62:0/89633648 conn(0x55cdaf2dbe20 msgr2=0x55cdaf2e0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/89633648 shutdown_connections 2023-07-14T16:23:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/89633648 wait complete. 2023-07-14T16:23:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.235+0000 7fdd6e39ab80 20 client.27319 trim_cache size 0 max 16384 2023-07-14T16:23:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.258+0000 7fdd6e39ab80 1 Processor -- start 2023-07-14T16:23:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.258+0000 7fdd6e39ab80 1 -- start start 2023-07-14T16:23:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6550 0x55cdaf2effb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf2f13c0 con 0x55cdaf2e6550 2023-07-14T16:23:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf2e23e0 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf2e2dd0 con 0x55cdaf2e6ea0 2023-07-14T16:23:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd53fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6550 0x55cdaf2effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd53fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6550 0x55cdaf2effb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58282/0 (socket says 172.21.15.62:58282) 2023-07-14T16:23:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd53fff700 1 -- 172.21.15.62:0/1548230225 learned_addr learned my addr 172.21.15.62:0/1548230225 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd537fe700 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.259+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd53fff700 1 -- 172.21.15.62:0/1548230225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 msgr2=0x55cdaf2f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd53fff700 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd53fff700 1 -- 172.21.15.62:0/1548230225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 msgr2=0x55cdaf2f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd53fff700 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd53fff700 1 -- 172.21.15.62:0/1548230225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdaf04ece0 con 0x55cdaf2e6550 2023-07-14T16:23:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd53fff700 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6550 0x55cdaf2effb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdd440086a0 tx=0x7fdd440086d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42b06b3f8e1035 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd3affd700 1 -- 172.21.15.62:0/1548230225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4401f030 con 0x55cdaf2e6550 2023-07-14T16:23:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd3affd700 1 -- 172.21.15.62:0/1548230225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd44005750 con 0x55cdaf2e6550 2023-07-14T16:23:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.260+0000 7fdd3affd700 1 -- 172.21.15.62:0/1548230225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400f8b0 con 0x55cdaf2e6550 2023-07-14T16:23:40.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/1548230225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6550 msgr2=0x55cdaf2effb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6550 0x55cdaf2effb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdd440086a0 tx=0x7fdd440086d0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/1548230225 shutdown_connections 2023-07-14T16:23:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf2f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e6550 0x55cdaf2effb0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/1548230225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf2f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/1548230225 >> 172.21.15.62:0/1548230225 conn(0x55cdaf2dbe20 msgr2=0x55cdaf2eac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/1548230225 shutdown_connections 2023-07-14T16:23:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.261+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/1548230225 wait complete. 2023-07-14T16:23:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.262+0000 7fdd6e39ab80 1 -- start start 2023-07-14T16:23:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.262+0000 7fdd6e39ab80 1 Processor -- start 2023-07-14T16:23:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf34be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf34e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd6e39ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf3508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf2f7b30 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf350e30 con 0x55cdaf2e6ea0 2023-07-14T16:23:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd6e39ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdaf350ff0 con 0x55cdaf2e6550 2023-07-14T16:23:40.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd6e39ab80 10 client.0 _ll_register_callbacks cb 0x7fdd44008d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:40.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd58f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf3508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd58f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf3508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58296/0 (socket says 172.21.15.62:58296) 2023-07-14T16:23:40.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd58f9c700 1 -- 172.21.15.62:0/3038129472 learned_addr learned my addr 172.21.15.62:0/3038129472 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd53fff700 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf34be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.263+0000 7fdd537fe700 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf34e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd58f9c700 1 -- 172.21.15.62:0/3038129472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 msgr2=0x55cdaf34be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf34be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd58f9c700 1 -- 172.21.15.62:0/3038129472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6ea0 msgr2=0x55cdaf34e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf34e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd58f9c700 1 -- 172.21.15.62:0/3038129472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c00f040 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd58f9c700 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf3508c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0129c0 tx=0x7fdd4c004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699dcbf946ff158e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd337fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:40.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd337fe700 1 -- 172.21.15.62:0/3038129472 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c007840 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/3038129472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdaf2ef560 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.264+0000 7fdd337fe700 1 -- 172.21.15.62:0/3038129472 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd4c004700 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd337fe700 1 -- 172.21.15.62:0/3038129472 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd4c007c00 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd6e39ab80 10 client.17778 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:40.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/3038129472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cdaf08c2d0 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd337fe700 1 -- 172.21.15.62:0/3038129472 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cdaf2ef560 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd337fe700 1 -- 172.21.15.62:0/3038129472 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdd4c03e070 con 0x55cdaf2e7cf0 2023-07-14T16:23:40.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd6e39ab80 10 client.17778 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:40.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd6e39ab80 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:40.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd6e39ab80 20 client.17778 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:40.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdaf352a80 0x55cdaf354e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd6e39ab80 4 client.17778 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:23:40.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.265+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/3038129472 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7fdd4c00d040 con 0x55cdaf352a80 2023-07-14T16:23:40.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.266+0000 7fdd53fff700 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdaf352a80 0x55cdaf354e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.266+0000 7fdd53fff700 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdaf352a80 0x55cdaf354e90 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f830443270f2e95 server_cookie=ab13bf568ea14f2d in_seq=0 out_seq=0 2023-07-14T16:23:40.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.266+0000 7fdd337fe700 10 client.17778 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:40.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.267+0000 7fdd337fe700 1 -- 172.21.15.62:0/3038129472 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7fdd4c00d040 con 0x55cdaf352a80 2023-07-14T16:23:40.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.267+0000 7fdd337fe700 10 client.17778 handle_command_reply: tid=0 2023-07-14T16:23:40.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.267+0000 7fdd6e39ab80 1 client.17778 shutdown 2023-07-14T16:23:40.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/3038129472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 msgr2=0x55cdaf3508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf3508c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0129c0 tx=0x7fdd4c004300 comp rx=0 tx=0).stop 2023-07-14T16:23:40.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/3038129472 shutdown_connections 2023-07-14T16:23:40.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cdaf2e7cf0 0x55cdaf3508c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cdaf2e6550 0x55cdaf34be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cdaf352a80 0x55cdaf354e90 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 --2- 172.21.15.62:0/3038129472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cdaf2e6ea0 0x55cdaf34e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/3038129472 >> 172.21.15.62:0/3038129472 conn(0x55cdaf2dbe20 msgr2=0x55cdaf2effb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd337fe700 0 client.17778 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/3038129472 shutdown_connections 2023-07-14T16:23:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 1 -- 172.21.15.62:0/3038129472 wait complete. 2023-07-14T16:23:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.268+0000 7fdd6e39ab80 20 client.17778 trim_cache size 0 max 16384 2023-07-14T16:23:40.252 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:23:40.252 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:23:40.252 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "66b72714-ef71-4e69-9ace-daec0789f017", 2023-07-14T16:23:40.252 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:23:40.253 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:23:40.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd3c0a11d0 msgr2=0x7fdd3c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd3c0a11d0 0x7fdd3c0a3680 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00e000 tx=0x7fdd4c01a000 comp rx=0 tx=0).stop 2023-07-14T16:23:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd540744d0 msgr2=0x7fdd54197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd540744d0 0x7fdd54197de0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000c00 tx=0x7fdd48003b00 comp rx=0 tx=0).stop 2023-07-14T16:23:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/3187591662 shutdown_connections 2023-07-14T16:23:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd541079b0 0x7fdd5419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdd3c0a11d0 0x7fdd3c0a3680 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd54072ef0 0x7fdd541978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 --2- 172.21.15.62:0/3187591662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd540744d0 0x7fdd54197de0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/3187591662 >> 172.21.15.62:0/3187591662 conn(0x7fdd540fb120 msgr2=0x7fdd54103940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/3187591662 shutdown_connections 2023-07-14T16:23:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.269+0000 7fdd5a9ff700 1 -- 172.21.15.62:0/3187591662 wait complete. 2023-07-14T16:23:40.268 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:23:40.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.517+0000 7fda26bff700 1 -- 172.21.15.62:0/1248924304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20100ce0 msgr2=0x7fda20101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.517+0000 7fda26bff700 1 --2- 172.21.15.62:0/1248924304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20100ce0 0x7fda20101130 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009990 tx=0x7fda0c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.518+0000 7fda26bff700 1 -- 172.21.15.62:0/1248924304 shutdown_connections 2023-07-14T16:23:40.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.518+0000 7fda26bff700 1 --2- 172.21.15.62:0/1248924304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda20101670 0x7fda20107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.518+0000 7fda26bff700 1 --2- 172.21.15.62:0/1248924304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda200ffaf0 0x7fda200fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.518+0000 7fda26bff700 1 --2- 172.21.15.62:0/1248924304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20100ce0 0x7fda20101130 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.518+0000 7fda26bff700 1 -- 172.21.15.62:0/1248924304 >> 172.21.15.62:0/1248924304 conn(0x7fda200fb130 msgr2=0x7fda200fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.518+0000 7fda26bff700 1 -- 172.21.15.62:0/1248924304 shutdown_connections 2023-07-14T16:23:40.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.518+0000 7fda26bff700 1 -- 172.21.15.62:0/1248924304 wait complete. 2023-07-14T16:23:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.519+0000 7fda26bff700 1 Processor -- start 2023-07-14T16:23:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.519+0000 7fda26bff700 1 -- start start 2023-07-14T16:23:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda26bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda200ffaf0 0x7fda200742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda26bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda20100ce0 0x7fda20072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda26bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20101670 0x7fda20072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda26bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda200735e0 con 0x7fda20101670 2023-07-14T16:23:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda26bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda20073750 con 0x7fda200ffaf0 2023-07-14T16:23:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda26bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2019f810 con 0x7fda20100ce0 2023-07-14T16:23:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda2519c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20101670 0x7fda20072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda20100ce0 0x7fda20072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda2519c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20101670 0x7fda20072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58312/0 (socket says 172.21.15.62:58312) 2023-07-14T16:23:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda2519c700 1 -- 172.21.15.62:0/1261637398 learned_addr learned my addr 172.21.15.62:0/1261637398 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda20100ce0 0x7fda20072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:57382/0 (socket says 172.21.15.62:57382) 2023-07-14T16:23:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.520+0000 7fda2499b700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda200ffaf0 0x7fda200742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda2519c700 1 -- 172.21.15.62:0/1261637398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda20100ce0 msgr2=0x7fda20072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda2519c700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda20100ce0 0x7fda20072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda2519c700 1 -- 172.21.15.62:0/1261637398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda200ffaf0 msgr2=0x7fda200742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda2519c700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda200ffaf0 0x7fda200742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda2519c700 1 -- 172.21.15.62:0/1261637398 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c0095f0 con 0x7fda20101670 2023-07-14T16:23:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda2519c700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20101670 0x7fda20072e40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ef30 tx=0x7fda1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda1dffb700 1 -- 172.21.15.62:0/1261637398 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18018070 con 0x7fda20101670 2023-07-14T16:23:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda1dffb700 1 -- 172.21.15.62:0/1261637398 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda1800f040 con 0x7fda20101670 2023-07-14T16:23:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.521+0000 7fda1dffb700 1 -- 172.21.15.62:0/1261637398 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800cbc0 con 0x7fda20101670 2023-07-14T16:23:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.522+0000 7fda26bff700 1 -- 172.21.15.62:0/1261637398 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2019fa90 con 0x7fda20101670 2023-07-14T16:23:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.522+0000 7fda26bff700 1 -- 172.21.15.62:0/1261637398 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2019ff50 con 0x7fda20101670 2023-07-14T16:23:40.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.527+0000 7fda3a555b80 1 Processor -- start 2023-07-14T16:23:40.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.527+0000 7fda1dffb700 1 -- 172.21.15.62:0/1261637398 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fda1800cd80 con 0x7fda20101670 2023-07-14T16:23:40.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.527+0000 7fda3a555b80 1 -- start start 2023-07-14T16:23:40.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be554a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be554b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be554c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda3a555b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be544fcb0 con 0x564be554a4d0 2023-07-14T16:23:40.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda3a555b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be54a2a40 con 0x564be554bc70 2023-07-14T16:23:40.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda3a555b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be549bed0 con 0x564be554ae20 2023-07-14T16:23:40.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be554a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be554a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58318/0 (socket says 172.21.15.62:58318) 2023-07-14T16:23:40.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.528+0000 7fda1ffff700 1 -- 172.21.15.62:0/2041703465 learned_addr learned my addr 172.21.15.62:0/2041703465 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda2499b700 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be554b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be554c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda1dffb700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda100a11d0 0x7fda100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda1ffff700 1 -- 172.21.15.62:0/2041703465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 msgr2=0x564be554b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be554b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda1ffff700 1 -- 172.21.15.62:0/2041703465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 msgr2=0x564be554c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be554c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda1ffff700 1 -- 172.21.15.62:0/2041703465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564be52c9de0 con 0x564be554a4d0 2023-07-14T16:23:40.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.529+0000 7fda1dffb700 1 -- 172.21.15.62:0/1261637398 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fda180cc420 con 0x7fda20101670 2023-07-14T16:23:40.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.530+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be554a8e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009990 tx=0x7fda0c005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d8e411ed67cd53 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.530+0000 7fda2519c700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda100a11d0 0x7fda100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.530+0000 7fda0affd700 1 -- 172.21.15.62:0/2041703465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c01d070 con 0x564be554a4d0 2023-07-14T16:23:40.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.530+0000 7fda0affd700 1 -- 172.21.15.62:0/2041703465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda0c004160 con 0x564be554a4d0 2023-07-14T16:23:40.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.530+0000 7fda0affd700 1 -- 172.21.15.62:0/2041703465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00b6b0 con 0x564be554a4d0 2023-07-14T16:23:40.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.531+0000 7fda3a555b80 1 -- 172.21.15.62:0/2041703465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 msgr2=0x564be554a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.531+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be554a8e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009990 tx=0x7fda0c005c20 comp rx=0 tx=0).stop 2023-07-14T16:23:40.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.531+0000 7fda2519c700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda100a11d0 0x7fda100a3680 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fda18000f80 tx=0x7fda180cc630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.531+0000 7fda3a555b80 1 -- 172.21.15.62:0/2041703465 shutdown_connections 2023-07-14T16:23:40.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.531+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be554c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.531+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be554a8e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.531+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2041703465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be554b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.531+0000 7fda3a555b80 1 -- 172.21.15.62:0/2041703465 >> 172.21.15.62:0/2041703465 conn(0x564be553fda0 msgr2=0x564be55451d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.532+0000 7fda3a555b80 1 -- 172.21.15.62:0/2041703465 shutdown_connections 2023-07-14T16:23:40.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.532+0000 7fda3a555b80 1 -- 172.21.15.62:0/2041703465 wait complete. 2023-07-14T16:23:40.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.532+0000 7fda3a555b80 1 -- start start 2023-07-14T16:23:40.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.533+0000 7fda3a555b80 1 Processor -- start 2023-07-14T16:23:40.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.533+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 0x564be5554110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.533+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554ae20 0x564be5554650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.533+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554bc70 0x564be555b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.533+0000 7fda3a555b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be543bac0 con 0x564be554bc70 2023-07-14T16:23:40.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.533+0000 7fda3a555b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be543b7c0 con 0x564be554ae20 2023-07-14T16:23:40.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.533+0000 7fda3a555b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be531f920 con 0x564be554a4d0 2023-07-14T16:23:40.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.533+0000 7fda3a555b80 10 client.0 _ll_register_callbacks cb 0x564be5439760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:40.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554bc70 0x564be555b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554bc70 0x564be555b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58328/0 (socket says 172.21.15.62:58328) 2023-07-14T16:23:40.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 -- 172.21.15.62:0/400252745 learned_addr learned my addr 172.21.15.62:0/400252745 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 0x564be5554110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 -- 172.21.15.62:0/400252745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 msgr2=0x564be5554110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 0x564be5554110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 -- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554ae20 msgr2=0x564be5554650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda2499b700 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554ae20 0x564be5554650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554ae20 0x564be5554650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 -- 172.21.15.62:0/400252745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c020040 con 0x564be554bc70 2023-07-14T16:23:40.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda2499b700 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554ae20 0x564be5554650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:40.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fda1ffff700 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554bc70 0x564be555b690 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fda0c003be0 tx=0x7fda0c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3865d595e9bc3c82 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.534+0000 7fd9fffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:40.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.535+0000 7fd9fffff700 1 -- 172.21.15.62:0/400252745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c01d030 con 0x564be554bc70 2023-07-14T16:23:40.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.535+0000 7fd9fffff700 1 -- 172.21.15.62:0/400252745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fda0c00b9b0 con 0x564be554bc70 2023-07-14T16:23:40.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.535+0000 7fda3a555b80 1 -- 172.21.15.62:0/400252745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564be53f54f0 con 0x564be554bc70 2023-07-14T16:23:40.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.535+0000 7fd9fffff700 1 -- 172.21.15.62:0/400252745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda0c00bb70 con 0x564be554bc70 2023-07-14T16:23:40.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.535+0000 7fda3a555b80 10 client.17790 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:40.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.535+0000 7fda3a555b80 1 -- 172.21.15.62:0/400252745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564be52b2c60 con 0x564be554bc70 2023-07-14T16:23:40.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.536+0000 7fd9fffff700 1 -- 172.21.15.62:0/400252745 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564be53f54f0 con 0x564be554bc70 2023-07-14T16:23:40.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.536+0000 7fd9fffff700 1 -- 172.21.15.62:0/400252745 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fda0c03b020 con 0x564be554bc70 2023-07-14T16:23:40.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.536+0000 7fda3a555b80 10 client.17790 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:40.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.536+0000 7fda3a555b80 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:40.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.536+0000 7fda3a555b80 20 client.17790 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:40.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.536+0000 7fda3a555b80 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564be555f950 0x564be5561d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.536+0000 7fda3a555b80 4 client.17790 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:40.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.536+0000 7fda3a555b80 1 -- 172.21.15.62:0/400252745 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564be5494eb0 con 0x564be555f950 2023-07-14T16:23:40.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.537+0000 7fda2499b700 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564be555f950 0x564be5561d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.537+0000 7fd9fffff700 10 client.17790 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:40.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.537+0000 7fda2499b700 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564be555f950 0x564be5561d30 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da0d1b89bd01396 server_cookie=43484deef2c8de0d in_seq=0 out_seq=0 2023-07-14T16:23:40.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.538+0000 7fd9fffff700 1 -- 172.21.15.62:0/400252745 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564be5494eb0 con 0x564be555f950 2023-07-14T16:23:40.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.538+0000 7fd9fffff700 10 client.17790 handle_command_reply: tid=0 2023-07-14T16:23:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.538+0000 7fda3a555b80 1 client.17790 shutdown 2023-07-14T16:23:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.538+0000 7fda3a555b80 1 -- 172.21.15.62:0/400252745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554bc70 msgr2=0x564be555b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.538+0000 7fda3a555b80 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554bc70 0x564be555b690 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fda0c003be0 tx=0x7fda0c00b260 comp rx=0 tx=0).stop 2023-07-14T16:23:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 1 -- 172.21.15.62:0/400252745 shutdown_connections 2023-07-14T16:23:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554bc70 0x564be555b690 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 0x564be5554110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564be555f950 0x564be5561d30 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 1 --2- 172.21.15.62:0/400252745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554ae20 0x564be5554650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 1 -- 172.21.15.62:0/400252745 >> 172.21.15.62:0/400252745 conn(0x564be553fda0 msgr2=0x564be55448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fd9fffff700 0 client.17790 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:40.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 1 -- 172.21.15.62:0/400252745 shutdown_connections 2023-07-14T16:23:40.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 1 -- 172.21.15.62:0/400252745 wait complete. 2023-07-14T16:23:40.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.539+0000 7fda3a555b80 20 client.17790 trim_cache size 0 max 16384 2023-07-14T16:23:40.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.565+0000 7fda3a555b80 1 Processor -- start 2023-07-14T16:23:40.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda3a555b80 1 -- start start 2023-07-14T16:23:40.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be55448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be555b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be554eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda3a555b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be55460c0 con 0x564be554a4d0 2023-07-14T16:23:40.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda3a555b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be55464c0 con 0x564be554bc70 2023-07-14T16:23:40.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda3a555b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be553cfd0 con 0x564be554ae20 2023-07-14T16:23:40.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be55448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be55448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58336/0 (socket says 172.21.15.62:58336) 2023-07-14T16:23:40.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda1ffff700 1 -- 172.21.15.62:0/2171302780 learned_addr learned my addr 172.21.15.62:0/2171302780 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.566+0000 7fda2499b700 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be555b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda1ffff700 1 -- 172.21.15.62:0/2171302780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 msgr2=0x564be555b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be555b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda1ffff700 1 -- 172.21.15.62:0/2171302780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 msgr2=0x564be554eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be554eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda1ffff700 1 -- 172.21.15.62:0/2171302780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564be554c560 con 0x564be554a4d0 2023-07-14T16:23:40.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda2499b700 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be555b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:40.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be55448b0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009990 tx=0x7fda0c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1954bfdbb0ad653b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda0b7fe700 1 -- 172.21.15.62:0/2171302780 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c01d040 con 0x564be554a4d0 2023-07-14T16:23:40.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda0b7fe700 1 -- 172.21.15.62:0/2171302780 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda0c038070 con 0x564be554a4d0 2023-07-14T16:23:40.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.567+0000 7fda0b7fe700 1 -- 172.21.15.62:0/2171302780 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00ecb0 con 0x564be554a4d0 2023-07-14T16:23:40.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 -- 172.21.15.62:0/2171302780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 msgr2=0x564be55448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be55448b0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009990 tx=0x7fda0c004020 comp rx=0 tx=0).stop 2023-07-14T16:23:40.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 -- 172.21.15.62:0/2171302780 shutdown_connections 2023-07-14T16:23:40.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be554eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554a4d0 0x564be55448b0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2171302780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554ae20 0x564be555b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 -- 172.21.15.62:0/2171302780 >> 172.21.15.62:0/2171302780 conn(0x564be553fda0 msgr2=0x564be5552e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 -- 172.21.15.62:0/2171302780 shutdown_connections 2023-07-14T16:23:40.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.568+0000 7fda3a555b80 1 -- 172.21.15.62:0/2171302780 wait complete. 2023-07-14T16:23:40.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 1 -- start start 2023-07-14T16:23:40.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 1 Processor -- start 2023-07-14T16:23:40.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 0x564be5557510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554ae20 0x564be5557a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be555aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be555afb0 con 0x564be554ae20 2023-07-14T16:23:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be555b120 con 0x564be554bc70 2023-07-14T16:23:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be555b2e0 con 0x564be554a4d0 2023-07-14T16:23:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.569+0000 7fda3a555b80 10 client.0 _ll_register_callbacks cb 0x7fda1400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda2499b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554ae20 0x564be5557a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda2499b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554ae20 0x564be5557a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58338/0 (socket says 172.21.15.62:58338) 2023-07-14T16:23:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda2499b700 1 -- 172.21.15.62:0/2333019453 learned_addr learned my addr 172.21.15.62:0/2333019453 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 0x564be5557510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be555aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda2499b700 1 -- 172.21.15.62:0/2333019453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 msgr2=0x564be5557510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda2499b700 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 0x564be5557510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda2499b700 1 -- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 msgr2=0x564be555aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:40.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda2499b700 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be555aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda2499b700 1 -- 172.21.15.62:0/2333019453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564be52b2c60 con 0x564be554ae20 2023-07-14T16:23:40.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.570+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be555aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:40.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fda2499b700 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554ae20 0x564be5557a50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x564be55625a0 tx=0x7fda1401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b080a5c0db2f72 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:40.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fd9fffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fd9fffff700 1 -- 172.21.15.62:0/2333019453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14017040 con 0x564be554ae20 2023-07-14T16:23:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fd9fffff700 1 -- 172.21.15.62:0/2333019453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fda1401c310 con 0x564be554ae20 2023-07-14T16:23:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fda3a555b80 1 -- 172.21.15.62:0/2333019453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda0c004160 con 0x564be554ae20 2023-07-14T16:23:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fd9fffff700 1 -- 172.21.15.62:0/2333019453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda1402a030 con 0x564be554ae20 2023-07-14T16:23:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fda3a555b80 10 client.17796 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fda3a555b80 1 -- 172.21.15.62:0/2333019453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564be52f0250 con 0x564be554ae20 2023-07-14T16:23:40.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.571+0000 7fd9fffff700 1 -- 172.21.15.62:0/2333019453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda0c004160 con 0x564be554ae20 2023-07-14T16:23:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.572+0000 7fd9fffff700 1 -- 172.21.15.62:0/2333019453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fda14004b60 con 0x564be554ae20 2023-07-14T16:23:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.572+0000 7fda3a555b80 10 client.17796 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.572+0000 7fda3a555b80 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.572+0000 7fda3a555b80 20 client.17796 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.572+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564be55b4940 0x564be55b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.572+0000 7fda3a555b80 4 client.17796 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:23:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.572+0000 7fda3a555b80 1 -- 172.21.15.62:0/2333019453 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564be5457050 con 0x564be55b4940 2023-07-14T16:23:40.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.572+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564be55b4940 0x564be55b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:40.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.573+0000 7fd9fffff700 10 client.17796 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:40.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.573+0000 7fda1ffff700 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564be55b4940 0x564be55b6d50 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6145e1c5e0bcff9c server_cookie=f669ed846f2d44b2 in_seq=0 out_seq=0 2023-07-14T16:23:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.573+0000 7fd9fffff700 1 -- 172.21.15.62:0/2333019453 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564be5457050 con 0x564be55b4940 2023-07-14T16:23:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.573+0000 7fd9fffff700 10 client.17796 handle_command_reply: tid=0 2023-07-14T16:23:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.573+0000 7fda3a555b80 1 client.17796 shutdown 2023-07-14T16:23:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 -- 172.21.15.62:0/2333019453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554ae20 msgr2=0x564be5557a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554ae20 0x564be5557a50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x564be55625a0 tx=0x7fda1401cc30 comp rx=0 tx=0).stop 2023-07-14T16:23:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 -- 172.21.15.62:0/2333019453 shutdown_connections 2023-07-14T16:23:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564be55b4940 0x564be55b6d50 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564be554bc70 0x564be555aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564be554a4d0 0x564be5557510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 --2- 172.21.15.62:0/2333019453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564be554ae20 0x564be5557a50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 -- 172.21.15.62:0/2333019453 >> 172.21.15.62:0/2333019453 conn(0x564be553fda0 msgr2=0x564be55448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fd9fffff700 0 client.17796 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:40.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 -- 172.21.15.62:0/2333019453 shutdown_connections 2023-07-14T16:23:40.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 1 -- 172.21.15.62:0/2333019453 wait complete. 2023-07-14T16:23:40.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.574+0000 7fda3a555b80 20 client.17796 trim_cache size 0 max 16384 2023-07-14T16:23:40.557 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:23:40.557 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:23:40.557 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:23:40.557 INFO:teuthology.orchestra.run.smithi062.stdout: "66b72714-ef71-4e69-9ace-daec0789f017": { 2023-07-14T16:23:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:23:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "66b72714-ef71-4e69-9ace-daec0789f017", 2023-07-14T16:23:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:23:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:23:40.558 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:23:40.558 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:23:40.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 -- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda100a11d0 msgr2=0x7fda100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda100a11d0 0x7fda100a3680 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fda18000f80 tx=0x7fda180cc630 comp rx=0 tx=0).stop 2023-07-14T16:23:40.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 -- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20101670 msgr2=0x7fda20072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:40.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20101670 0x7fda20072e40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ef30 tx=0x7fda1800c3a0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 -- 172.21.15.62:0/1261637398 shutdown_connections 2023-07-14T16:23:40.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda20101670 0x7fda20072e40 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda100a11d0 0x7fda100a3680 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda200ffaf0 0x7fda200742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 --2- 172.21.15.62:0/1261637398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda20100ce0 0x7fda20072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 -- 172.21.15.62:0/1261637398 >> 172.21.15.62:0/1261637398 conn(0x7fda200fb130 msgr2=0x7fda200fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 -- 172.21.15.62:0/1261637398 shutdown_connections 2023-07-14T16:23:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:23:40.575+0000 7fda26bff700 1 -- 172.21.15.62:0/1261637398 wait complete. 2023-07-14T16:23:40.571 INFO:tasks.cephfs.filesystem:scrub status for tag:66b72714-ef71-4e69-9ace-daec0789f017 - {'path': '/', 'tag': '66b72714-ef71-4e69-9ace-daec0789f017', 'options': 'recursive,force'} 2023-07-14T16:23:40.651 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:23:40 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:23:40.277+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v173>,<0x10000000003/client.0 v174>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v186>,<0x10000000001/sv_0 v200>,<0x10000000000/_nogroup v194>,<0x1/volumes v184>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:23:40.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:40 smithi062 ceph-mon[95543]: pgmap v598: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:23:40.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:23:40.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:40.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:23:40.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:40.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:23:40.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:41.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:40 smithi081 ceph-mon[127795]: pgmap v598: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:23:41.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:23:41.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:41.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:23:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:23:41.153 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:41.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:40 smithi164 ceph-mon[97427]: pgmap v598: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:23:41.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:23:41.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:41.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:23:41.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:41.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:23:41.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:42.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:41 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:23:42.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:41 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:42.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:41 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:23:42.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:41 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:23:42.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:41 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:42.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:41 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:23:42.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:41 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:42.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:41 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:23:42.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:41 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:23:42.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:41 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:42.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:41 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:23:42.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:41 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:42.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:41 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:23:42.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:41 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:23:42.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:41 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:23:43.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:42 smithi062 ceph-mon[95543]: pgmap v599: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:23:43.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:42 smithi081 ceph-mon[127795]: pgmap v599: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:23:43.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:42 smithi164 ceph-mon[97427]: pgmap v599: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:23:45.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:44 smithi062 ceph-mon[95543]: pgmap v600: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:23:45.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:44 smithi081 ceph-mon[127795]: pgmap v600: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:23:45.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:44 smithi164 ceph-mon[97427]: pgmap v600: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:23:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.859+0000 7fb94259d700 1 -- 172.21.15.62:0/2334908538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 msgr2=0x7fb93c0ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.859+0000 7fb94259d700 1 --2- 172.21.15.62:0/2334908538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 0x7fb93c0ffd50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb938009990 tx=0x7fb938009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.859+0000 7fb94259d700 1 -- 172.21.15.62:0/2334908538 shutdown_connections 2023-07-14T16:23:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.859+0000 7fb94259d700 1 --2- 172.21.15.62:0/2334908538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb93c1014c0 0x7fb93c107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.860+0000 7fb94259d700 1 --2- 172.21.15.62:0/2334908538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 0x7fb93c0ffd50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.860+0000 7fb94259d700 1 --2- 172.21.15.62:0/2334908538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb93c100b30 0x7fb93c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.860+0000 7fb94259d700 1 -- 172.21.15.62:0/2334908538 >> 172.21.15.62:0/2334908538 conn(0x7fb93c0faf60 msgr2=0x7fb93c0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.860+0000 7fb94259d700 1 -- 172.21.15.62:0/2334908538 shutdown_connections 2023-07-14T16:23:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.860+0000 7fb94259d700 1 -- 172.21.15.62:0/2334908538 wait complete. 2023-07-14T16:23:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94259d700 1 Processor -- start 2023-07-14T16:23:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94259d700 1 -- start start 2023-07-14T16:23:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94259d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 0x7fb93c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94259d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb93c100b30 0x7fb93c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94259d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb93c1014c0 0x7fb93c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94259d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19f370 con 0x7fb93c0ff940 2023-07-14T16:23:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94259d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19f4b0 con 0x7fb93c100b30 2023-07-14T16:23:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94259d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19f640 con 0x7fb93c1014c0 2023-07-14T16:23:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.861+0000 7fb94159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 0x7fb93c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb941d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb93c1014c0 0x7fb93c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb94159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 0x7fb93c1977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35038/0 (socket says 172.21.15.62:35038) 2023-07-14T16:23:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb94159b700 1 -- 172.21.15.62:0/1755233210 learned_addr learned my addr 172.21.15.62:0/1755233210 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb940d9a700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb93c100b30 0x7fb93c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb94159b700 1 -- 172.21.15.62:0/1755233210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb93c1014c0 msgr2=0x7fb93c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb94159b700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb93c1014c0 0x7fb93c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb94159b700 1 -- 172.21.15.62:0/1755233210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb93c100b30 msgr2=0x7fb93c197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb94159b700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb93c100b30 0x7fb93c197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb94159b700 1 -- 172.21.15.62:0/1755233210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9380095f0 con 0x7fb93c0ff940 2023-07-14T16:23:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.862+0000 7fb94159b700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 0x7fb93c1977f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb938005910 tx=0x7fb938005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.863+0000 7fb9327fc700 1 -- 172.21.15.62:0/1755233210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93801d070 con 0x7fb93c0ff940 2023-07-14T16:23:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.863+0000 7fb9327fc700 1 -- 172.21.15.62:0/1755233210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb93800bd50 con 0x7fb93c0ff940 2023-07-14T16:23:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.863+0000 7fb9327fc700 1 -- 172.21.15.62:0/1755233210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938022d00 con 0x7fb93c0ff940 2023-07-14T16:23:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.863+0000 7fb94259d700 1 -- 172.21.15.62:0/1755233210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c19f8c0 con 0x7fb93c0ff940 2023-07-14T16:23:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.863+0000 7fb94259d700 1 -- 172.21.15.62:0/1755233210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c19fc90 con 0x7fb93c0ff940 2023-07-14T16:23:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.866+0000 7fb9327fc700 1 -- 172.21.15.62:0/1755233210 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb93800b8c0 con 0x7fb93c0ff940 2023-07-14T16:23:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.868+0000 7fb9327fc700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9280a1150 0x7fb9280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.868+0000 7fb9327fc700 1 -- 172.21.15.62:0/1755233210 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9380cdda0 con 0x7fb93c0ff940 2023-07-14T16:23:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.868+0000 7fb957220b80 1 Processor -- start 2023-07-14T16:23:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.868+0000 7fb940d9a700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9280a1150 0x7fb9280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb957220b80 1 -- start start 2023-07-14T16:23:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb957220b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bacd76b0 0x5642bae7b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb957220b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 0x5642bae7c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb957220b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae7cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb957220b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bad80c50 con 0x5642bacd76b0 2023-07-14T16:23:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb957220b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642badd39e0 con 0x5642bae7ba90 2023-07-14T16:23:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb957220b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642badcce70 con 0x5642bae7c7e0 2023-07-14T16:23:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb940d9a700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9280a1150 0x7fb9280a3600 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009c60 tx=0x7fb92c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb94159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bacd76b0 0x5642bae7b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb94159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bacd76b0 0x5642bae7b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35044/0 (socket says 172.21.15.62:35044) 2023-07-14T16:23:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb94159b700 1 -- 172.21.15.62:0/3407681012 learned_addr learned my addr 172.21.15.62:0/3407681012 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.869+0000 7fb940d9a700 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 0x5642bae7c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb941d9c700 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae7cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb94159b700 1 -- 172.21.15.62:0/3407681012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 msgr2=0x5642bae7cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb94159b700 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae7cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb94159b700 1 -- 172.21.15.62:0/3407681012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 msgr2=0x5642bae7c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb94159b700 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 0x5642bae7c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb94159b700 1 -- 172.21.15.62:0/3407681012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642babfad80 con 0x5642bacd76b0 2023-07-14T16:23:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb941d9c700 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae7cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:23:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb94159b700 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bacd76b0 0x5642bae7b550 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c010 tx=0x7fb938009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2dda40c7f46e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb9277fe700 1 -- 172.21.15.62:0/3407681012 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93800bb70 con 0x5642bacd76b0 2023-07-14T16:23:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb9277fe700 1 -- 172.21.15.62:0/3407681012 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb93809b1f0 con 0x5642bacd76b0 2023-07-14T16:23:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.870+0000 7fb9277fe700 1 -- 172.21.15.62:0/3407681012 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9380993a0 con 0x5642bacd76b0 2023-07-14T16:23:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 -- 172.21.15.62:0/3407681012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bacd76b0 msgr2=0x5642bae7b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bacd76b0 0x5642bae7b550 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c010 tx=0x7fb938009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 -- 172.21.15.62:0/3407681012 shutdown_connections 2023-07-14T16:23:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae7cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bacd76b0 0x5642bae7b550 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 --2- 172.21.15.62:0/3407681012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 0x5642bae7c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 -- 172.21.15.62:0/3407681012 >> 172.21.15.62:0/3407681012 conn(0x5642bae70d40 msgr2=0x5642bae75e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 -- 172.21.15.62:0/3407681012 shutdown_connections 2023-07-14T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.871+0000 7fb957220b80 1 -- 172.21.15.62:0/3407681012 wait complete. 2023-07-14T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.872+0000 7fb957220b80 1 -- start start 2023-07-14T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.872+0000 7fb957220b80 1 Processor -- start 2023-07-14T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb957220b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bacd76b0 0x5642bae84700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb957220b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae84c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb957220b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae8bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb957220b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bad6ca60 con 0x5642bae7ba90 2023-07-14T16:23:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb957220b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bad6c760 con 0x5642bacd76b0 2023-07-14T16:23:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb957220b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bac508c0 con 0x5642bae7c7e0 2023-07-14T16:23:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb957220b80 10 client.0 _ll_register_callbacks cb 0x5642bad6a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb940d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae84c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb940d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae84c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35054/0 (socket says 172.21.15.62:35054) 2023-07-14T16:23:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb940d9a700 1 -- 172.21.15.62:0/3218962552 learned_addr learned my addr 172.21.15.62:0/3218962552 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb94159b700 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bacd76b0 0x5642bae84700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.873+0000 7fb941d9c700 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae8bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb940d9a700 1 -- 172.21.15.62:0/3218962552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 msgr2=0x5642bae8bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb940d9a700 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae8bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb940d9a700 1 -- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bacd76b0 msgr2=0x5642bae84700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb940d9a700 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bacd76b0 0x5642bae84700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb940d9a700 1 -- 172.21.15.62:0/3218962552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c19fc90 con 0x5642bae7ba90 2023-07-14T16:23:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb94159b700 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bacd76b0 0x5642bae84700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:23:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb940d9a700 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae84c40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005910 tx=0x7fb92c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6106ad0476afdd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb924ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb924ff9700 1 -- 172.21.15.62:0/3218962552 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c022070 con 0x5642bae7ba90 2023-07-14T16:23:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb957220b80 1 -- 172.21.15.62:0/3218962552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642bad26490 con 0x5642bae7ba90 2023-07-14T16:23:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.874+0000 7fb924ff9700 1 -- 172.21.15.62:0/3218962552 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb92c01d620 con 0x5642bae7ba90 2023-07-14T16:23:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb924ff9700 1 -- 172.21.15.62:0/3218962552 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92c01d7e0 con 0x5642bae7ba90 2023-07-14T16:23:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb957220b80 10 client.17808 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb957220b80 1 -- 172.21.15.62:0/3218962552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5642babe3c00 con 0x5642bae7ba90 2023-07-14T16:23:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb924ff9700 1 -- 172.21.15.62:0/3218962552 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5642bad26490 con 0x5642bae7ba90 2023-07-14T16:23:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb924ff9700 1 -- 172.21.15.62:0/3218962552 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb92c027980 con 0x5642bae7ba90 2023-07-14T16:23:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb957220b80 10 client.17808 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb957220b80 10 client.17808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb957220b80 20 client.17808 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb957220b80 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5642bae904d0 0x5642bae928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb957220b80 4 client.17808 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.875+0000 7fb957220b80 1 -- 172.21.15.62:0/3218962552 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642badc5e50 con 0x5642bae904d0 2023-07-14T16:23:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.876+0000 7fb94159b700 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5642bae904d0 0x5642bae928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.876+0000 7fb94159b700 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5642bae904d0 0x5642bae928e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98246a909d45773 server_cookie=133a1c53cb4b81a8 in_seq=0 out_seq=0 2023-07-14T16:23:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.876+0000 7fb924ff9700 10 client.17808 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.877+0000 7fb924ff9700 1 -- 172.21.15.62:0/3218962552 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5642badc5e50 con 0x5642bae904d0 2023-07-14T16:23:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.877+0000 7fb924ff9700 10 client.17808 handle_command_reply: tid=0 2023-07-14T16:23:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.877+0000 7fb957220b80 1 client.17808 shutdown 2023-07-14T16:23:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.877+0000 7fb957220b80 1 -- 172.21.15.62:0/3218962552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 msgr2=0x5642bae84c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.877+0000 7fb957220b80 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae84c40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005910 tx=0x7fb92c0047c0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.877+0000 7fb957220b80 1 -- 172.21.15.62:0/3218962552 shutdown_connections 2023-07-14T16:23:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.877+0000 7fb957220b80 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bae7c7e0 0x5642bae8bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.877+0000 7fb957220b80 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5642bae904d0 0x5642bae928e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.878+0000 7fb957220b80 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bacd76b0 0x5642bae84700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.878+0000 7fb957220b80 1 --2- 172.21.15.62:0/3218962552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae84c40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.878+0000 7fb957220b80 1 -- 172.21.15.62:0/3218962552 >> 172.21.15.62:0/3218962552 conn(0x5642bae70d40 msgr2=0x5642bae75640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.878+0000 7fb924ff9700 0 client.17808 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.878+0000 7fb957220b80 1 -- 172.21.15.62:0/3218962552 shutdown_connections 2023-07-14T16:23:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.878+0000 7fb957220b80 1 -- 172.21.15.62:0/3218962552 wait complete. 2023-07-14T16:23:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.878+0000 7fb957220b80 20 client.17808 trim_cache size 0 max 16384 2023-07-14T16:23:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.894+0000 7fb957220b80 1 Processor -- start 2023-07-14T16:23:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.894+0000 7fb957220b80 1 -- start start 2023-07-14T16:23:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb957220b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae7cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb957220b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae8bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb957220b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7c7e0 0x5642bae844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb957220b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bae858f0 con 0x5642bae7ba90 2023-07-14T16:23:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb957220b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bae776a0 con 0x5642bae7c7e0 2023-07-14T16:23:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb957220b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bae84270 con 0x5642bacd76b0 2023-07-14T16:23:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb940d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae8bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb940d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae8bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35064/0 (socket says 172.21.15.62:35064) 2023-07-14T16:23:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb940d9a700 1 -- 172.21.15.62:0/462321263 learned_addr learned my addr 172.21.15.62:0/462321263 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb941d9c700 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7c7e0 0x5642bae844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb94159b700 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae7cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb940d9a700 1 -- 172.21.15.62:0/462321263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 msgr2=0x5642bae7cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb940d9a700 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae7cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.895+0000 7fb940d9a700 1 -- 172.21.15.62:0/462321263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7c7e0 msgr2=0x5642bae844b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.896+0000 7fb940d9a700 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7c7e0 0x5642bae844b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.896+0000 7fb940d9a700 1 -- 172.21.15.62:0/462321263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642bac211f0 con 0x5642bae7ba90 2023-07-14T16:23:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.896+0000 7fb940d9a700 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae8bc40 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb92c003f70 tx=0x7fb92c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad53d22dfef413 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.896+0000 7fb927fff700 1 -- 172.21.15.62:0/462321263 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c022070 con 0x5642bae7ba90 2023-07-14T16:23:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.896+0000 7fb927fff700 1 -- 172.21.15.62:0/462321263 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb92c004580 con 0x5642bae7ba90 2023-07-14T16:23:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.896+0000 7fb927fff700 1 -- 172.21.15.62:0/462321263 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c01da70 con 0x5642bae7ba90 2023-07-14T16:23:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 -- 172.21.15.62:0/462321263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 msgr2=0x5642bae8bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae8bc40 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb92c003f70 tx=0x7fb92c01c410 comp rx=0 tx=0).stop 2023-07-14T16:23:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 -- 172.21.15.62:0/462321263 shutdown_connections 2023-07-14T16:23:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7c7e0 0x5642bae844b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae7cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 --2- 172.21.15.62:0/462321263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7ba90 0x5642bae8bc40 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 -- 172.21.15.62:0/462321263 >> 172.21.15.62:0/462321263 conn(0x5642bae70d40 msgr2=0x5642bae97240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 -- 172.21.15.62:0/462321263 shutdown_connections 2023-07-14T16:23:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 -- 172.21.15.62:0/462321263 wait complete. 2023-07-14T16:23:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.897+0000 7fb957220b80 1 -- start start 2023-07-14T16:23:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.898+0000 7fb957220b80 1 Processor -- start 2023-07-14T16:23:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.898+0000 7fb957220b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae87740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.898+0000 7fb957220b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 0x5642bae87c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.898+0000 7fb957220b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7c7e0 0x5642bae8aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.898+0000 7fb957220b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bae8b1e0 con 0x5642bae7c7e0 2023-07-14T16:23:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.898+0000 7fb957220b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bae8b350 con 0x5642bae7ba90 2023-07-14T16:23:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.898+0000 7fb957220b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642bae8b510 con 0x5642bacd76b0 2023-07-14T16:23:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.898+0000 7fb957220b80 10 client.0 _ll_register_callbacks cb 0x7fb92c0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae87740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae87740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:48196/0 (socket says 172.21.15.62:48196) 2023-07-14T16:23:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 -- 172.21.15.62:0/1627239744 learned_addr learned my addr 172.21.15.62:0/1627239744 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb941d9c700 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7c7e0 0x5642bae8aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb940d9a700 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 0x5642bae87c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 -- 172.21.15.62:0/1627239744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 msgr2=0x5642bae87c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 0x5642bae87c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 -- 172.21.15.62:0/1627239744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7c7e0 msgr2=0x5642bae8aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7c7e0 0x5642bae8aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 -- 172.21.15.62:0/1627239744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642babe3c00 con 0x5642bacd76b0 2023-07-14T16:23:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb94159b700 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae87740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb938097c60 tx=0x7fb938097660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dab89459ad542cb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.899+0000 7fb924ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:23:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.900+0000 7fb924ff9700 1 -- 172.21.15.62:0/1627239744 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93800bbf0 con 0x5642bacd76b0 2023-07-14T16:23:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.900+0000 7fb924ff9700 1 -- 172.21.15.62:0/1627239744 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb93809f220 con 0x5642bacd76b0 2023-07-14T16:23:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.900+0000 7fb957220b80 1 -- 172.21.15.62:0/1627239744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb93809b1f0 con 0x5642bacd76b0 2023-07-14T16:23:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.900+0000 7fb924ff9700 1 -- 172.21.15.62:0/1627239744 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9380b8050 con 0x5642bacd76b0 2023-07-14T16:23:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.900+0000 7fb957220b80 10 client.27358 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.900+0000 7fb957220b80 1 -- 172.21.15.62:0/1627239744 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5642babf9250 con 0x5642bacd76b0 2023-07-14T16:23:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.900+0000 7fb924ff9700 1 -- 172.21.15.62:0/1627239744 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb93809b1f0 con 0x5642bacd76b0 2023-07-14T16:23:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.901+0000 7fb924ff9700 1 -- 172.21.15.62:0/1627239744 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb9380a3120 con 0x5642bacd76b0 2023-07-14T16:23:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.901+0000 7fb957220b80 10 client.27358 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.901+0000 7fb957220b80 10 client.27358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.901+0000 7fb957220b80 20 client.27358 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.901+0000 7fb957220b80 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5642baee4cb0 0x5642baee70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.901+0000 7fb957220b80 4 client.27358 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:23:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.901+0000 7fb957220b80 1 -- 172.21.15.62:0/1627239744 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5642bad87ff0 con 0x5642baee4cb0 2023-07-14T16:23:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.901+0000 7fb940d9a700 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5642baee4cb0 0x5642baee70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.902+0000 7fb924ff9700 10 client.27358 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.902+0000 7fb940d9a700 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5642baee4cb0 0x5642baee70c0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55b85e041f95b9d server_cookie=762da14202b7248b in_seq=0 out_seq=0 2023-07-14T16:23:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb924ff9700 1 -- 172.21.15.62:0/1627239744 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5642bad87ff0 con 0x5642baee4cb0 2023-07-14T16:23:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb924ff9700 10 client.27358 handle_command_reply: tid=0 2023-07-14T16:23:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 client.27358 shutdown 2023-07-14T16:23:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 -- 172.21.15.62:0/1627239744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 msgr2=0x5642bae87740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae87740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb938097c60 tx=0x7fb938097660 comp rx=0 tx=0).stop 2023-07-14T16:23:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 -- 172.21.15.62:0/1627239744 shutdown_connections 2023-07-14T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642bae7c7e0 0x5642bae8aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5642bacd76b0 0x5642bae87740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5642baee4cb0 0x5642baee70c0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 --2- 172.21.15.62:0/1627239744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5642bae7ba90 0x5642bae87c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb924ff9700 0 client.27358 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 -- 172.21.15.62:0/1627239744 >> 172.21.15.62:0/1627239744 conn(0x5642bae70d40 msgr2=0x5642bae844b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 -- 172.21.15.62:0/1627239744 shutdown_connections 2023-07-14T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.911+0000 7fb957220b80 1 -- 172.21.15.62:0/1627239744 wait complete. 2023-07-14T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb957220b80 20 client.27358 trim_cache size 0 max 16384 2023-07-14T16:23:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:23:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:23:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:23:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 -- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9280a1150 msgr2=0x7fb9280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9280a1150 0x7fb9280a3600 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009c60 tx=0x7fb92c009260 comp rx=0 tx=0).stop 2023-07-14T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 -- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 msgr2=0x7fb93c1977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 0x7fb93c1977f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb938005910 tx=0x7fb938005740 comp rx=0 tx=0).stop 2023-07-14T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 -- 172.21.15.62:0/1755233210 shutdown_connections 2023-07-14T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb93c1014c0 0x7fb93c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb93c0ff940 0x7fb93c1977f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9280a1150 0x7fb9280a3600 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 --2- 172.21.15.62:0/1755233210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb93c100b30 0x7fb93c197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.912+0000 7fb94259d700 1 -- 172.21.15.62:0/1755233210 >> 172.21.15.62:0/1755233210 conn(0x7fb93c0faf60 msgr2=0x7fb93c1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.913+0000 7fb94259d700 1 -- 172.21.15.62:0/1755233210 shutdown_connections 2023-07-14T16:23:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:45.913+0000 7fb94259d700 1 -- 172.21.15.62:0/1755233210 wait complete. 2023-07-14T16:23:47.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:46 smithi062 ceph-mon[95543]: pgmap v601: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:23:47.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:46 smithi081 ceph-mon[127795]: pgmap v601: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:23:47.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:46 smithi164 ceph-mon[97427]: pgmap v601: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:23:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:48 smithi062 ceph-mon[95543]: pgmap v602: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 33 op/s 2023-07-14T16:23:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:48 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:23:49.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:48 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:23:49.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:48 smithi081 ceph-mon[127795]: pgmap v602: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 33 op/s 2023-07-14T16:23:49.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:48 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:23:49.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:48 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:23:49.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:48 smithi164 ceph-mon[97427]: pgmap v602: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 33 op/s 2023-07-14T16:23:49.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:48 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:23:49.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:48 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:23:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:50 smithi062 ceph-mon[95543]: pgmap v603: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:23:51.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:50 smithi081 ceph-mon[127795]: pgmap v603: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:23:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:50 smithi164 ceph-mon[97427]: pgmap v603: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:23:52.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:51 smithi062 ceph-mon[95543]: pgmap v604: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:52.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:51 smithi081 ceph-mon[127795]: pgmap v604: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:52.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:51 smithi164 ceph-mon[97427]: pgmap v604: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:23:55.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:54 smithi164 ceph-mon[97427]: pgmap v605: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 30 op/s 2023-07-14T16:23:55.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:54 smithi062 ceph-mon[95543]: pgmap v605: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 30 op/s 2023-07-14T16:23:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:54 smithi081 ceph-mon[127795]: pgmap v605: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 30 op/s 2023-07-14T16:23:57.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:56 smithi164 ceph-mon[97427]: pgmap v606: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:23:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:56 smithi062 ceph-mon[95543]: pgmap v606: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:23:57.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:56 smithi081 ceph-mon[127795]: pgmap v606: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:23:59.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:23:58 smithi164 ceph-mon[97427]: pgmap v607: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 -- 172.21.15.62:0/121007690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4072f70 msgr2=0x7f3fe40733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/121007690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4072f70 0x7f3fe40733e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009990 tx=0x7f3fd8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 -- 172.21.15.62:0/121007690 shutdown_connections 2023-07-14T16:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/121007690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe410c0e0 0x7f3fe410e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/121007690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fe4074550 0x7f3fe40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/121007690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4072f70 0x7f3fe40733e0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 -- 172.21.15.62:0/121007690 >> 172.21.15.62:0/121007690 conn(0x7f3fe40fb130 msgr2=0x7f3fe40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 -- 172.21.15.62:0/121007690 shutdown_connections 2023-07-14T16:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.065+0000 7f3fe8dff700 1 -- 172.21.15.62:0/121007690 wait complete. 2023-07-14T16:23:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.066+0000 7f3fe8dff700 1 Processor -- start 2023-07-14T16:23:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.066+0000 7f3fe8dff700 1 -- start start 2023-07-14T16:23:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe8dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fe4072f70 0x7f3fe419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe8dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4074550 0x7f3fe419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe8dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe410c0e0 0x7f3fe419d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe8dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe419d920 con 0x7f3fe4074550 2023-07-14T16:23:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe8dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe419da90 con 0x7f3fe410c0e0 2023-07-14T16:23:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe8dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe419dc50 con 0x7f3fe4072f70 2023-07-14T16:23:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4074550 0x7f3fe419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4074550 0x7f3fe419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57820/0 (socket says 172.21.15.62:57820) 2023-07-14T16:23:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/1792980017 learned_addr learned my addr 172.21.15.62:0/1792980017 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe2d9d700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe410c0e0 0x7f3fe419d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe259c700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fe4072f70 0x7f3fe419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/1792980017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fe4072f70 msgr2=0x7f3fe419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fe4072f70 0x7f3fe419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/1792980017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe410c0e0 msgr2=0x7f3fe419d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.067+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe410c0e0 0x7f3fe419d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.068+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/1792980017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80095f0 con 0x7f3fe4074550 2023-07-14T16:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.068+0000 7f3fe259c700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fe4072f70 0x7f3fe419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.068+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4074550 0x7f3fe419c440 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80098e0 tx=0x7f3fd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.068+0000 7f3fd37fe700 1 -- 172.21.15.62:0/1792980017 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd801d070 con 0x7f3fe4074550 2023-07-14T16:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.068+0000 7f3fe8dff700 1 -- 172.21.15.62:0/1792980017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe419ded0 con 0x7f3fe4074550 2023-07-14T16:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.068+0000 7f3fd37fe700 1 -- 172.21.15.62:0/1792980017 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd800be30 con 0x7f3fe4074550 2023-07-14T16:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.068+0000 7f3fd37fe700 1 -- 172.21.15.62:0/1792980017 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8022dd0 con 0x7f3fe4074550 2023-07-14T16:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.068+0000 7f3fe8dff700 1 -- 172.21.15.62:0/1792980017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe41a46f0 con 0x7f3fe4074550 2023-07-14T16:23:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.073+0000 7f3fd37fe700 1 -- 172.21.15.62:0/1792980017 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3fd8022570 con 0x7f3fe4074550 2023-07-14T16:23:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.074+0000 7f3ffc786b80 1 Processor -- start 2023-07-14T16:23:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3ffc786b80 1 -- start start 2023-07-14T16:23:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3fd37fe700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3fcc0a1150 0x7f3fcc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 0x5607adb21300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb21840 0x5607adb221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb226e0 0x5607adb22b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ada26c50 con 0x5607adb226e0 2023-07-14T16:23:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ada799e0 con 0x5607ad97d6b0 2023-07-14T16:23:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ada72e70 con 0x5607adb21840 2023-07-14T16:23:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3fd37fe700 1 -- 172.21.15.62:0/1792980017 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3fd80cde90 con 0x7f3fe4074550 2023-07-14T16:23:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.075+0000 7f3fe2d9d700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3fcc0a1150 0x7f3fcc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb226e0 0x5607adb22b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb21840 0x5607adb221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb21840 0x5607adb221a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:49724/0 (socket says 172.21.15.62:49724) 2023-07-14T16:23:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/3021020132 learned_addr learned my addr 172.21.15.62:0/3021020132 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe259c700 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 0x5607adb21300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/3021020132 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 msgr2=0x5607adb21300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 0x5607adb21300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/3021020132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb226e0 msgr2=0x5607adb22b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb226e0 0x5607adb22b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/3021020132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607ad8a0d80 con 0x5607adb21840 2023-07-14T16:23:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe2d9d700 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb226e0 0x5607adb22b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe2d9d700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3fcc0a1150 0x7f3fcc0a3600 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc007770 tx=0x7f3fdc009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.076+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb21840 0x5607adb221a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8097840 tx=0x7f3fd809b190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=100aeb8ab9b047a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.077+0000 7f3fd0ff9700 1 -- 172.21.15.62:0/3021020132 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8078d60 con 0x5607adb21840 2023-07-14T16:23:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.077+0000 7f3fd0ff9700 1 -- 172.21.15.62:0/3021020132 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd8078b80 con 0x5607adb21840 2023-07-14T16:23:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.077+0000 7f3fd0ff9700 1 -- 172.21.15.62:0/3021020132 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80a3700 con 0x5607adb21840 2023-07-14T16:23:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.077+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3021020132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb21840 msgr2=0x5607adb221a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.077+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb21840 0x5607adb221a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8097840 tx=0x7f3fd809b190 comp rx=0 tx=0).stop 2023-07-14T16:23:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.078+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3021020132 shutdown_connections 2023-07-14T16:23:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.078+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb226e0 0x5607adb22b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.078+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 0x5607adb21300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.078+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/3021020132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb21840 0x5607adb221a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.078+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3021020132 >> 172.21.15.62:0/3021020132 conn(0x5607adb16be0 msgr2=0x5607adb1bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.078+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3021020132 shutdown_connections 2023-07-14T16:23:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.078+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3021020132 wait complete. 2023-07-14T16:23:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.078+0000 7f3ffc786b80 1 -- start start 2023-07-14T16:23:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3ffc786b80 1 Processor -- start 2023-07-14T16:23:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607ad97d6b0 0x5607adb2a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb226e0 0x5607adb31a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ada12a60 con 0x5607adb21840 2023-07-14T16:23:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ada12760 con 0x5607adb226e0 2023-07-14T16:23:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ad8f68c0 con 0x5607ad97d6b0 2023-07-14T16:23:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3ffc786b80 10 client.0 _ll_register_callbacks cb 0x5607ada10700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3fe1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3fe1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57840/0 (socket says 172.21.15.62:57840) 2023-07-14T16:23:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.079+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/1854270489 learned_addr learned my addr 172.21.15.62:0/1854270489 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fe259c700 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607ad97d6b0 0x5607adb2a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fe2d9d700 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb226e0 0x5607adb31a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/1854270489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607ad97d6b0 msgr2=0x5607adb2a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607ad97d6b0 0x5607adb2a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/1854270489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb226e0 msgr2=0x5607adb31a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb226e0 0x5607adb31a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/1854270489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe41a46f0 con 0x5607adb21840 2023-07-14T16:23:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2aa20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80cb9f0 tx=0x7f3fd8078960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1001ba5182d7e221 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.080+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/1854270489 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd809f170 con 0x5607adb21840 2023-07-14T16:23:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 1 -- 172.21.15.62:0/1854270489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607ad9cc490 con 0x5607adb21840 2023-07-14T16:23:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/1854270489 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3fd80a1110 con 0x5607adb21840 2023-07-14T16:23:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/1854270489 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd8078510 con 0x5607adb21840 2023-07-14T16:23:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 10 client.17826 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 1 -- 172.21.15.62:0/1854270489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5607ad889c00 con 0x5607adb21840 2023-07-14T16:23:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/1854270489 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd80c9080 con 0x5607adb21840 2023-07-14T16:23:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/1854270489 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3fd809a830 con 0x5607adb21840 2023-07-14T16:23:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 10 client.17826 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 20 client.17826 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5607adb35de0 0x5607adb381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 4 client.17826 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:23:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.081+0000 7f3ffc786b80 1 -- 172.21.15.62:0/1854270489 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607ada6be50 con 0x5607adb35de0 2023-07-14T16:23:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.082+0000 7f3fe259c700 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5607adb35de0 0x5607adb381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.082+0000 7f3fe259c700 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5607adb35de0 0x5607adb381c0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a4b685b304b749 server_cookie=308680d95ad14072 in_seq=0 out_seq=0 2023-07-14T16:23:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.082+0000 7f3fc9ffb700 10 client.17826 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.083+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/1854270489 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5607ada6be50 con 0x5607adb35de0 2023-07-14T16:23:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.083+0000 7f3fc9ffb700 10 client.17826 handle_command_reply: tid=0 2023-07-14T16:23:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.083+0000 7f3ffc786b80 1 client.17826 shutdown 2023-07-14T16:23:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.083+0000 7f3ffc786b80 1 -- 172.21.15.62:0/1854270489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 msgr2=0x5607adb2aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.083+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2aa20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80cb9f0 tx=0x7f3fd8078960 comp rx=0 tx=0).stop 2023-07-14T16:23:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.083+0000 7f3ffc786b80 1 -- 172.21.15.62:0/1854270489 shutdown_connections 2023-07-14T16:23:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.083+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5607adb35de0 0x5607adb381c0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.084+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb226e0 0x5607adb31a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.084+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607ad97d6b0 0x5607adb2a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.084+0000 7f3fc9ffb700 0 client.17826 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.084+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/1854270489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2aa20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.084+0000 7f3ffc786b80 1 -- 172.21.15.62:0/1854270489 >> 172.21.15.62:0/1854270489 conn(0x5607adb16be0 msgr2=0x5607adb1b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.084+0000 7f3ffc786b80 1 -- 172.21.15.62:0/1854270489 shutdown_connections 2023-07-14T16:23:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.084+0000 7f3ffc786b80 1 -- 172.21.15.62:0/1854270489 wait complete. 2023-07-14T16:23:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.084+0000 7f3ffc786b80 20 client.17826 trim_cache size 0 max 16384 2023-07-14T16:23:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.102+0000 7f3ffc786b80 1 Processor -- start 2023-07-14T16:23:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.102+0000 7f3ffc786b80 1 -- start start 2023-07-14T16:23:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.102+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607ad97d6b0 0x5607adb253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.102+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb21840 0x5607adb31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.102+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb25910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607adb2a0b0 con 0x5607ad97d6b0 2023-07-14T16:23:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607adae7060 con 0x5607adb21840 2023-07-14T16:23:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607adb29990 con 0x5607adb226e0 2023-07-14T16:23:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607ad97d6b0 0x5607adb253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607ad97d6b0 0x5607adb253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57844/0 (socket says 172.21.15.62:57844) 2023-07-14T16:23:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe259c700 1 -- 172.21.15.62:0/3648815891 learned_addr learned my addr 172.21.15.62:0/3648815891 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe2d9d700 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb25910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb21840 0x5607adb31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe259c700 1 -- 172.21.15.62:0/3648815891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 msgr2=0x5607adb25910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe259c700 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb25910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe259c700 1 -- 172.21.15.62:0/3648815891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb21840 msgr2=0x5607adb31a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe259c700 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb21840 0x5607adb31a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe259c700 1 -- 172.21.15.62:0/3648815891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607ad8c71f0 con 0x5607ad97d6b0 2023-07-14T16:23:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.103+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb21840 0x5607adb31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.104+0000 7f3fe259c700 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607ad97d6b0 0x5607adb253d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009db0 tx=0x7f3fd4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef3f1e6e752315 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.104+0000 7f3fd17fa700 1 -- 172.21.15.62:0/3648815891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4017070 con 0x5607ad97d6b0 2023-07-14T16:23:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.104+0000 7f3fd17fa700 1 -- 172.21.15.62:0/3648815891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd4018040 con 0x5607ad97d6b0 2023-07-14T16:23:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.104+0000 7f3fd17fa700 1 -- 172.21.15.62:0/3648815891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4011860 con 0x5607ad97d6b0 2023-07-14T16:23:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3648815891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607ad97d6b0 msgr2=0x5607adb253d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607ad97d6b0 0x5607adb253d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009db0 tx=0x7f3fd4009260 comp rx=0 tx=0).stop 2023-07-14T16:23:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3648815891 shutdown_connections 2023-07-14T16:23:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb25910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607ad97d6b0 0x5607adb253d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/3648815891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607adb21840 0x5607adb31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3648815891 >> 172.21.15.62:0/3648815891 conn(0x5607adb16be0 msgr2=0x5607adb3cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3648815891 shutdown_connections 2023-07-14T16:23:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 -- 172.21.15.62:0/3648815891 wait complete. 2023-07-14T16:23:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.105+0000 7f3ffc786b80 1 -- start start 2023-07-14T16:23:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.106+0000 7f3ffc786b80 1 Processor -- start 2023-07-14T16:23:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.106+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 0x5607adb2d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.106+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.106+0000 7f3ffc786b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb30e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.106+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607adb31340 con 0x5607adb21840 2023-07-14T16:23:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.106+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607adb314b0 con 0x5607ad97d6b0 2023-07-14T16:23:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.106+0000 7f3ffc786b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607adb31670 con 0x5607adb226e0 2023-07-14T16:23:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.106+0000 7f3ffc786b80 10 client.0 _ll_register_callbacks cb 0x7f3fd4009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:23:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57854/0 (socket says 172.21.15.62:57854) 2023-07-14T16:23:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/941030771 learned_addr learned my addr 172.21.15.62:0/941030771 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:23:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe259c700 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 0x5607adb2d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe2d9d700 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb30e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/941030771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 msgr2=0x5607adb30e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb30e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/941030771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 msgr2=0x5607adb2d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:23:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 0x5607adb2d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 -- 172.21.15.62:0/941030771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd4009c20 con 0x5607adb21840 2023-07-14T16:23:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe2d9d700 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb30e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:23:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fe1d9b700 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2dde0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8097840 tx=0x7f3fd80998f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5370a03fc0906eb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:23:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.107+0000 7f3fc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:23:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/941030771 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80789d0 con 0x5607adb21840 2023-07-14T16:23:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/941030771 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3fd809b5a0 con 0x5607adb21840 2023-07-14T16:23:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3ffc786b80 1 -- 172.21.15.62:0/941030771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fd8078b80 con 0x5607adb21840 2023-07-14T16:23:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/941030771 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd809f170 con 0x5607adb21840 2023-07-14T16:23:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3ffc786b80 10 client.17832 fetch_fsmap learned FSMap version 25 2023-07-14T16:23:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3ffc786b80 1 -- 172.21.15.62:0/941030771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5607ad89f250 con 0x5607adb21840 2023-07-14T16:23:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/941030771 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd8078b80 con 0x5607adb21840 2023-07-14T16:23:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/941030771 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3fd809a830 con 0x5607adb21840 2023-07-14T16:23:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3ffc786b80 10 client.17832 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:23:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3ffc786b80 10 client.17832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:23:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3ffc786b80 20 client.17832 populate_metadata read hostname 'smithi062' 2023-07-14T16:23:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.108+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5607adb8a700 0x5607adb8cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:23:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.109+0000 7f3ffc786b80 4 client.17832 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:23:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.109+0000 7f3ffc786b80 1 -- 172.21.15.62:0/941030771 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5607ada2dff0 con 0x5607adb8a700 2023-07-14T16:23:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.109+0000 7f3fe259c700 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5607adb8a700 0x5607adb8cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:23:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.110+0000 7f3fc9ffb700 10 client.17832 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.110+0000 7f3fe259c700 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5607adb8a700 0x5607adb8cae0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e86b136dfc4410 server_cookie=717209aaa5eac765 in_seq=0 out_seq=0 2023-07-14T16:23:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3fc9ffb700 1 -- 172.21.15.62:0/941030771 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5607ada2dff0 con 0x5607adb8a700 2023-07-14T16:23:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3fc9ffb700 10 client.17832 handle_command_reply: tid=0 2023-07-14T16:23:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 client.17832 shutdown 2023-07-14T16:23:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 -- 172.21.15.62:0/941030771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 msgr2=0x5607adb2dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2dde0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8097840 tx=0x7f3fd80998f0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 -- 172.21.15.62:0/941030771 shutdown_connections 2023-07-14T16:23:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5607adb226e0 0x5607adb30e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5607adb8a700 0x5607adb8cae0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607ad97d6b0 0x5607adb2d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 --2- 172.21.15.62:0/941030771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607adb21840 0x5607adb2dde0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3fc9ffb700 0 client.17832 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:23:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 -- 172.21.15.62:0/941030771 >> 172.21.15.62:0/941030771 conn(0x5607adb16be0 msgr2=0x5607adb2a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 -- 172.21.15.62:0/941030771 shutdown_connections 2023-07-14T16:23:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.120+0000 7f3ffc786b80 1 -- 172.21.15.62:0/941030771 wait complete. 2023-07-14T16:23:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.121+0000 7f3ffc786b80 20 client.17832 trim_cache size 0 max 16384 2023-07-14T16:23:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:23:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:23:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:23:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:23:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.121+0000 7f3fe8dff700 1 -- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3fcc0a1150 msgr2=0x7f3fcc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.121+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3fcc0a1150 0x7f3fcc0a3600 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc007770 tx=0x7f3fdc009940 comp rx=0 tx=0).stop 2023-07-14T16:23:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.121+0000 7f3fe8dff700 1 -- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4074550 msgr2=0x7f3fe419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:23:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.121+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4074550 0x7f3fe419c440 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80098e0 tx=0x7f3fd8004730 comp rx=0 tx=0).stop 2023-07-14T16:23:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.122+0000 7f3fe8dff700 1 -- 172.21.15.62:0/1792980017 shutdown_connections 2023-07-14T16:23:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.122+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3fcc0a1150 0x7f3fcc0a3600 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.122+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe410c0e0 0x7f3fe419d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.122+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3fe4072f70 0x7f3fe419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.122+0000 7f3fe8dff700 1 --2- 172.21.15.62:0/1792980017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fe4074550 0x7f3fe419c440 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:23:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.122+0000 7f3fe8dff700 1 -- 172.21.15.62:0/1792980017 >> 172.21.15.62:0/1792980017 conn(0x7f3fe40fb130 msgr2=0x7f3fe40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:23:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.122+0000 7f3fe8dff700 1 -- 172.21.15.62:0/1792980017 shutdown_connections 2023-07-14T16:23:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:23:59.122+0000 7f3fe8dff700 1 -- 172.21.15.62:0/1792980017 wait complete. 2023-07-14T16:23:59.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:23:58 smithi062 ceph-mon[95543]: pgmap v607: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:23:59.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:23:58 smithi081 ceph-mon[127795]: pgmap v607: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:24:01.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:00 smithi164 ceph-mon[97427]: pgmap v608: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:24:01.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:00 smithi062 ceph-mon[95543]: pgmap v608: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:24:01.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:00 smithi081 ceph-mon[127795]: pgmap v608: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:24:03.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:02 smithi164 ceph-mon[97427]: pgmap v609: 97 pgs: 97 active+clean; 5.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:03.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:02 smithi062 ceph-mon[95543]: pgmap v609: 97 pgs: 97 active+clean; 5.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:03.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:02 smithi081 ceph-mon[127795]: pgmap v609: 97 pgs: 97 active+clean; 5.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:04 smithi164 ceph-mon[97427]: pgmap v610: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:05.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:04 smithi062 ceph-mon[95543]: pgmap v610: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:05.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:04 smithi081 ceph-mon[127795]: pgmap v610: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:07.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:06 smithi164 ceph-mon[97427]: pgmap v611: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:24:07.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:06 smithi062 ceph-mon[95543]: pgmap v611: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:24:07.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:06 smithi081 ceph-mon[127795]: pgmap v611: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:24:09.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:08 smithi164 ceph-mon[97427]: pgmap v612: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:08 smithi062 ceph-mon[95543]: pgmap v612: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:09.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:08 smithi081 ceph-mon[127795]: pgmap v612: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:10.573 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:24:10.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.823+0000 7fd2de7ff700 1 -- 172.21.15.62:0/2546523319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d8072ef0 msgr2=0x7fd2d8073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.823+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/2546523319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d8072ef0 0x7fd2d8073360 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009990 tx=0x7fd2cc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.823+0000 7fd2de7ff700 1 -- 172.21.15.62:0/2546523319 shutdown_connections 2023-07-14T16:24:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.823+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/2546523319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2d81079b0 0x7fd2d8109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.823+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/2546523319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2d80744d0 0x7fd2d8072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.823+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/2546523319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d8072ef0 0x7fd2d8073360 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.823+0000 7fd2de7ff700 1 -- 172.21.15.62:0/2546523319 >> 172.21.15.62:0/2546523319 conn(0x7fd2d80fb120 msgr2=0x7fd2d80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.824+0000 7fd2de7ff700 1 -- 172.21.15.62:0/2546523319 shutdown_connections 2023-07-14T16:24:10.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.824+0000 7fd2de7ff700 1 -- 172.21.15.62:0/2546523319 wait complete. 2023-07-14T16:24:10.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.825+0000 7fd2de7ff700 1 Processor -- start 2023-07-14T16:24:10.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.825+0000 7fd2de7ff700 1 -- start start 2023-07-14T16:24:10.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.825+0000 7fd2de7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2d8072ef0 0x7fd2d8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.825+0000 7fd2de7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d80744d0 0x7fd2d8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.825+0000 7fd2de7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2d81079b0 0x7fd2d819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.825+0000 7fd2de7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819f510 con 0x7fd2d80744d0 2023-07-14T16:24:10.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.825+0000 7fd2de7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819f650 con 0x7fd2d8072ef0 2023-07-14T16:24:10.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.825+0000 7fd2de7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819f7e0 con 0x7fd2d81079b0 2023-07-14T16:24:10.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d80744d0 0x7fd2d8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2d8072ef0 0x7fd2d8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d80744d0 0x7fd2d8197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44826/0 (socket says 172.21.15.62:44826) 2023-07-14T16:24:10.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 -- 172.21.15.62:0/755239657 learned_addr learned my addr 172.21.15.62:0/755239657 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:10.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2dcd9c700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2d81079b0 0x7fd2d819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 -- 172.21.15.62:0/755239657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2d81079b0 msgr2=0x7fd2d819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2d81079b0 0x7fd2d819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 -- 172.21.15.62:0/755239657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2d8072ef0 msgr2=0x7fd2d8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2d8072ef0 0x7fd2d8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 -- 172.21.15.62:0/755239657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0095f0 con 0x7fd2d80744d0 2023-07-14T16:24:10.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.826+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d80744d0 0x7fd2d8197ed0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc005910 tx=0x7fd2cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:10.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.827+0000 7fd2d57fa700 1 -- 172.21.15.62:0/755239657 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc01d070 con 0x7fd2d80744d0 2023-07-14T16:24:10.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.827+0000 7fd2d57fa700 1 -- 172.21.15.62:0/755239657 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2cc004c70 con 0x7fd2d80744d0 2023-07-14T16:24:10.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.827+0000 7fd2de7ff700 1 -- 172.21.15.62:0/755239657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d819fa60 con 0x7fd2d80744d0 2023-07-14T16:24:10.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.827+0000 7fd2d57fa700 1 -- 172.21.15.62:0/755239657 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc003a30 con 0x7fd2d80744d0 2023-07-14T16:24:10.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.827+0000 7fd2de7ff700 1 -- 172.21.15.62:0/755239657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d819fe80 con 0x7fd2d80744d0 2023-07-14T16:24:10.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.830+0000 7fd2d57fa700 1 -- 172.21.15.62:0/755239657 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd2cc005000 con 0x7fd2d80744d0 2023-07-14T16:24:10.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.832+0000 7fd2d57fa700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2c00a1150 0x7fd2c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.832+0000 7fd2f2157b80 1 Processor -- start 2023-07-14T16:24:10.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.832+0000 7fd2d57fa700 1 -- 172.21.15.62:0/755239657 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd2cc0cd850 con 0x7fd2d80744d0 2023-07-14T16:24:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.832+0000 7fd2f2157b80 1 -- start start 2023-07-14T16:24:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.832+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2c00a1150 0x7fd2c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f67704d0 0x55a2f67708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f67716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f6771c70 0x55a2f67720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6675cb0 con 0x55a2f6770e20 2023-07-14T16:24:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f66c8a40 con 0x55a2f6771c70 2023-07-14T16:24:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f66c1ed0 con 0x55a2f67704d0 2023-07-14T16:24:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f67716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f67704d0 0x55a2f67708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f67716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44832/0 (socket says 172.21.15.62:44832) 2023-07-14T16:24:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2d77fe700 1 -- 172.21.15.62:0/1152951740 learned_addr learned my addr 172.21.15.62:0/1152951740 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2c00a1150 0x7fd2c00a3600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8005950 tx=0x7fd2c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:10.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.833+0000 7fd2dcd9c700 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f6771c70 0x55a2f67720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2d77fe700 1 -- 172.21.15.62:0/1152951740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f67704d0 msgr2=0x55a2f67708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:10.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f67704d0 0x55a2f67708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2d77fe700 1 -- 172.21.15.62:0/1152951740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f6771c70 msgr2=0x55a2f67720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:10.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f6771c70 0x55a2f67720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2d77fe700 1 -- 172.21.15.62:0/1152951740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f64efde0 con 0x55a2f6770e20 2023-07-14T16:24:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f67704d0 0x55a2f67708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2dcd9c700 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f6771c70 0x55a2f67720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f67716e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0a2b90 tx=0x7fd2cc005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10f3a48d93ca4da server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2be7fc700 1 -- 172.21.15.62:0/1152951740 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc09ad00 con 0x55a2f6770e20 2023-07-14T16:24:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2be7fc700 1 -- 172.21.15.62:0/1152951740 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2cc09ed30 con 0x55a2f6770e20 2023-07-14T16:24:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.834+0000 7fd2be7fc700 1 -- 172.21.15.62:0/1152951740 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0a2e30 con 0x55a2f6770e20 2023-07-14T16:24:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1152951740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 msgr2=0x55a2f67716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f67716e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0a2b90 tx=0x7fd2cc005ac0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1152951740 shutdown_connections 2023-07-14T16:24:10.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f6771c70 0x55a2f67720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f67704d0 0x55a2f67708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1152951740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f67716e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1152951740 >> 172.21.15.62:0/1152951740 conn(0x55a2f6765da0 msgr2=0x55a2f676b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:10.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1152951740 shutdown_connections 2023-07-14T16:24:10.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.835+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1152951740 wait complete. 2023-07-14T16:24:10.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.836+0000 7fd2f2157b80 1 -- start start 2023-07-14T16:24:10.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.836+0000 7fd2f2157b80 1 Processor -- start 2023-07-14T16:24:10.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.836+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f6779e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.836+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.836+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f67813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.836+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6661ac0 con 0x55a2f6770e20 2023-07-14T16:24:10.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.836+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f66617c0 con 0x55a2f67704d0 2023-07-14T16:24:10.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.836+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6545920 con 0x55a2f6771c70 2023-07-14T16:24:10.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2f2157b80 10 client.0 _ll_register_callbacks cb 0x55a2f665f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:10.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44846/0 (socket says 172.21.15.62:44846) 2023-07-14T16:24:10.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d77fe700 1 -- 172.21.15.62:0/2480676458 learned_addr learned my addr 172.21.15.62:0/2480676458 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:10.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f6779e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2dcd9c700 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f67813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d77fe700 1 -- 172.21.15.62:0/2480676458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 msgr2=0x55a2f67813c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:10.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f67813c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d77fe700 1 -- 172.21.15.62:0/2480676458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 msgr2=0x55a2f6779e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:10.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f6779e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.837+0000 7fd2d77fe700 1 -- 172.21.15.62:0/2480676458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d819fe80 con 0x55a2f6770e20 2023-07-14T16:24:10.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677a380 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0cb3b0 tx=0x7fd2cc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ced75615fb6389a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:10.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:10.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2b77fe700 1 -- 172.21.15.62:0/2480676458 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0049e0 con 0x55a2f6770e20 2023-07-14T16:24:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2b77fe700 1 -- 172.21.15.62:0/2480676458 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd2cc09ad00 con 0x55a2f6770e20 2023-07-14T16:24:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2f2157b80 1 -- 172.21.15.62:0/2480676458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2f661b4f0 con 0x55a2f6770e20 2023-07-14T16:24:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2b77fe700 1 -- 172.21.15.62:0/2480676458 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2cc0b3120 con 0x55a2f6770e20 2023-07-14T16:24:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2f2157b80 10 client.17844 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2f2157b80 1 -- 172.21.15.62:0/2480676458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a2f64d8c60 con 0x55a2f6770e20 2023-07-14T16:24:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.838+0000 7fd2b77fe700 1 -- 172.21.15.62:0/2480676458 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a2f661b4f0 con 0x55a2f6770e20 2023-07-14T16:24:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.839+0000 7fd2b77fe700 1 -- 172.21.15.62:0/2480676458 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2cc099c60 con 0x55a2f6770e20 2023-07-14T16:24:10.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.839+0000 7fd2f2157b80 10 client.17844 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:10.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.839+0000 7fd2f2157b80 10 client.17844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:10.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.839+0000 7fd2f2157b80 20 client.17844 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:10.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.839+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a2f6785880 0x55a2f6787c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.839+0000 7fd2f2157b80 4 client.17844 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.839+0000 7fd2f2157b80 1 -- 172.21.15.62:0/2480676458 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2f66baeb0 con 0x55a2f6785880 2023-07-14T16:24:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.839+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a2f6785880 0x55a2f6787c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.840+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a2f6785880 0x55a2f6787c90 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88980cb1ca74d776 server_cookie=688dc11fc2cea518 in_seq=0 out_seq=0 2023-07-14T16:24:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.840+0000 7fd2b77fe700 10 client.17844 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2b77fe700 1 -- 172.21.15.62:0/2480676458 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a2f66baeb0 con 0x55a2f6785880 2023-07-14T16:24:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2b77fe700 10 client.17844 handle_command_reply: tid=0 2023-07-14T16:24:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 client.17844 shutdown 2023-07-14T16:24:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 -- 172.21.15.62:0/2480676458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 msgr2=0x55a2f677a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677a380 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0cb3b0 tx=0x7fd2cc005bc0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 -- 172.21.15.62:0/2480676458 shutdown_connections 2023-07-14T16:24:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f67813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a2f6785880 0x55a2f6787c90 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f6779e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/2480676458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677a380 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 -- 172.21.15.62:0/2480676458 >> 172.21.15.62:0/2480676458 conn(0x55a2f6765da0 msgr2=0x55a2f676a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2b77fe700 0 client.17844 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 -- 172.21.15.62:0/2480676458 shutdown_connections 2023-07-14T16:24:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.841+0000 7fd2f2157b80 1 -- 172.21.15.62:0/2480676458 wait complete. 2023-07-14T16:24:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.842+0000 7fd2f2157b80 20 client.17844 trim_cache size 0 max 16384 2023-07-14T16:24:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.861+0000 7fd2f2157b80 1 Processor -- start 2023-07-14T16:24:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2f2157b80 1 -- start start 2023-07-14T16:24:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f6772040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f6781380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f676b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f676ccd0 con 0x55a2f6770e20 2023-07-14T16:24:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f67659a0 con 0x55a2f67704d0 2023-07-14T16:24:10.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f676b710 con 0x55a2f6771c70 2023-07-14T16:24:10.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f6781380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f6781380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44858/0 (socket says 172.21.15.62:44858) 2023-07-14T16:24:10.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2d77fe700 1 -- 172.21.15.62:0/3677983150 learned_addr learned my addr 172.21.15.62:0/3677983150 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:10.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f6772040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.862+0000 7fd2dcd9c700 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f676b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2d77fe700 1 -- 172.21.15.62:0/3677983150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 msgr2=0x55a2f676b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:10.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f676b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2d77fe700 1 -- 172.21.15.62:0/3677983150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 msgr2=0x55a2f6772040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f6772040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2d77fe700 1 -- 172.21.15.62:0/3677983150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f6516250 con 0x55a2f6770e20 2023-07-14T16:24:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f6781380 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc09aa00 tx=0x7fd2cc0992b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6285c606ef05d5d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2beffd700 1 -- 172.21.15.62:0/3677983150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc099e60 con 0x55a2f6770e20 2023-07-14T16:24:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2beffd700 1 -- 172.21.15.62:0/3677983150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2cc09a2f0 con 0x55a2f6770e20 2023-07-14T16:24:10.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.863+0000 7fd2beffd700 1 -- 172.21.15.62:0/3677983150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc060bc0 con 0x55a2f6770e20 2023-07-14T16:24:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 -- 172.21.15.62:0/3677983150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 msgr2=0x55a2f6781380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f6781380 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc09aa00 tx=0x7fd2cc0992b0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 -- 172.21.15.62:0/3677983150 shutdown_connections 2023-07-14T16:24:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f676b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f6772040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/3677983150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f6781380 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 -- 172.21.15.62:0/3677983150 >> 172.21.15.62:0/3677983150 conn(0x55a2f6765da0 msgr2=0x55a2f6778b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 -- 172.21.15.62:0/3677983150 shutdown_connections 2023-07-14T16:24:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.864+0000 7fd2f2157b80 1 -- 172.21.15.62:0/3677983150 wait complete. 2023-07-14T16:24:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.865+0000 7fd2f2157b80 1 -- start start 2023-07-14T16:24:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.865+0000 7fd2f2157b80 1 Processor -- start 2023-07-14T16:24:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.865+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f677d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2f2157b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f6780860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6780da0 con 0x55a2f6770e20 2023-07-14T16:24:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6780f10 con 0x55a2f67704d0 2023-07-14T16:24:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2f2157b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f67810d0 con 0x55a2f6771c70 2023-07-14T16:24:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2f2157b80 10 client.0 _ll_register_callbacks cb 0x7fd2c800b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f677d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44868/0 (socket says 172.21.15.62:44868) 2023-07-14T16:24:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d77fe700 1 -- 172.21.15.62:0/1181161739 learned_addr learned my addr 172.21.15.62:0/1181161739 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:10.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f677d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:58556/0 (socket says 172.21.15.62:58556) 2023-07-14T16:24:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2dcd9c700 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f6780860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d77fe700 1 -- 172.21.15.62:0/1181161739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 msgr2=0x55a2f6780860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f6780860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d77fe700 1 -- 172.21.15.62:0/1181161739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 msgr2=0x55a2f677d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f677d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.866+0000 7fd2d77fe700 1 -- 172.21.15.62:0/1181161739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2f64d8c60 con 0x55a2f6770e20 2023-07-14T16:24:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2d77fe700 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677db90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc09a020 tx=0x7fd2cc09a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c31d350e7b433 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:10.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2b77fe700 1 -- 172.21.15.62:0/1181161739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc09aad0 con 0x55a2f6770e20 2023-07-14T16:24:10.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1181161739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2cc09ed30 con 0x55a2f6770e20 2023-07-14T16:24:10.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2b77fe700 1 -- 172.21.15.62:0/1181161739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd2cc09ad00 con 0x55a2f6770e20 2023-07-14T16:24:10.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2b77fe700 1 -- 172.21.15.62:0/1181161739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2cc0ba4c0 con 0x55a2f6770e20 2023-07-14T16:24:10.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2f2157b80 10 client.17850 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:10.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1181161739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a2f64ee2b0 con 0x55a2f6770e20 2023-07-14T16:24:10.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.867+0000 7fd2b77fe700 1 -- 172.21.15.62:0/1181161739 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2cc09ed30 con 0x55a2f6770e20 2023-07-14T16:24:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.868+0000 7fd2b77fe700 1 -- 172.21.15.62:0/1181161739 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2cc09bae0 con 0x55a2f6770e20 2023-07-14T16:24:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.868+0000 7fd2f2157b80 10 client.17850 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.868+0000 7fd2f2157b80 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.868+0000 7fd2f2157b80 20 client.17850 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.868+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a2f67da190 0x55a2f67dc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.868+0000 7fd2f2157b80 4 client.17850 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:24:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.868+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1181161739 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a2f667d050 con 0x55a2f67da190 2023-07-14T16:24:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.868+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a2f67da190 0x55a2f67dc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2d7fff700 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a2f67da190 0x55a2f67dc570 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4b939f9e6392b5e server_cookie=fc39e67e89b330ec in_seq=0 out_seq=0 2023-07-14T16:24:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2b77fe700 10 client.17850 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2b77fe700 1 -- 172.21.15.62:0/1181161739 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a2f667d050 con 0x55a2f67da190 2023-07-14T16:24:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2b77fe700 10 client.17850 handle_command_reply: tid=0 2023-07-14T16:24:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 client.17850 shutdown 2023-07-14T16:24:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1181161739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 msgr2=0x55a2f677db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677db90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc09a020 tx=0x7fd2cc09a710 comp rx=0 tx=0).stop 2023-07-14T16:24:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1181161739 shutdown_connections 2023-07-14T16:24:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a2f6771c70 0x55a2f6780860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a2f67da190 0x55a2f67dc570 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a2f67704d0 0x55a2f677d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 --2- 172.21.15.62:0/1181161739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a2f6770e20 0x55a2f677db90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1181161739 >> 172.21.15.62:0/1181161739 conn(0x55a2f6765da0 msgr2=0x55a2f6781bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:10.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.869+0000 7fd2b77fe700 0 client.17850 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:10.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1181161739 shutdown_connections 2023-07-14T16:24:10.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2f2157b80 1 -- 172.21.15.62:0/1181161739 wait complete. 2023-07-14T16:24:10.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2f2157b80 20 client.17850 trim_cache size 0 max 16384 2023-07-14T16:24:10.853 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:24:10.853 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:24:10.854 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:24:10.854 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:24:10.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 -- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2c00a1150 msgr2=0x7fd2c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2c00a1150 0x7fd2c00a3600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8005950 tx=0x7fd2c800b410 comp rx=0 tx=0).stop 2023-07-14T16:24:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 -- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d80744d0 msgr2=0x7fd2d8197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d80744d0 0x7fd2d8197ed0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc005910 tx=0x7fd2cc004730 comp rx=0 tx=0).stop 2023-07-14T16:24:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 -- 172.21.15.62:0/755239657 shutdown_connections 2023-07-14T16:24:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2d81079b0 0x7fd2d819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd2c00a1150 0x7fd2c00a3600 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2d8072ef0 0x7fd2d8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.870+0000 7fd2de7ff700 1 --2- 172.21.15.62:0/755239657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd2d80744d0 0x7fd2d8197ed0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:10.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.871+0000 7fd2de7ff700 1 -- 172.21.15.62:0/755239657 >> 172.21.15.62:0/755239657 conn(0x7fd2d80fb120 msgr2=0x7fd2d80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:10.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.871+0000 7fd2de7ff700 1 -- 172.21.15.62:0/755239657 shutdown_connections 2023-07-14T16:24:10.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:10.871+0000 7fd2de7ff700 1 -- 172.21.15.62:0/755239657 wait complete. 2023-07-14T16:24:10.865 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:24:10.865 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:24:10.931 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:10 smithi062 ceph-mon[95543]: pgmap v613: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:11.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:10 smithi164 ceph-mon[97427]: pgmap v613: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:11.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 -- 172.21.15.62:0/2444009974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe470100460 msgr2=0x7fe470102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 --2- 172.21.15.62:0/2444009974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe470100460 0x7fe470102840 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe460009990 tx=0x7fe460009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 -- 172.21.15.62:0/2444009974 shutdown_connections 2023-07-14T16:24:11.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 --2- 172.21.15.62:0/2444009974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4701056a0 0x7fe470107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 --2- 172.21.15.62:0/2444009974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe470100460 0x7fe470102840 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 --2- 172.21.15.62:0/2444009974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe470102d80 0x7fe470105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 -- 172.21.15.62:0/2444009974 >> 172.21.15.62:0/2444009974 conn(0x7fe4700f9ee0 msgr2=0x7fe4700fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 -- 172.21.15.62:0/2444009974 shutdown_connections 2023-07-14T16:24:11.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.126+0000 7fe474f9d700 1 -- 172.21.15.62:0/2444009974 wait complete. 2023-07-14T16:24:11.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.127+0000 7fe474f9d700 1 Processor -- start 2023-07-14T16:24:11.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe474f9d700 1 -- start start 2023-07-14T16:24:11.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe474f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe470100460 0x7fe470197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe474f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe470102d80 0x7fe470197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe474f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4701056a0 0x7fe47019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe474f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47019f210 con 0x7fe4701056a0 2023-07-14T16:24:11.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe474f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47019f350 con 0x7fe470100460 2023-07-14T16:24:11.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe474f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47019f4e0 con 0x7fe470102d80 2023-07-14T16:24:11.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe46ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4701056a0 0x7fe47019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe46ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4701056a0 0x7fe47019ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44882/0 (socket says 172.21.15.62:44882) 2023-07-14T16:24:11.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe46ffff700 1 -- 172.21.15.62:0/587702431 learned_addr learned my addr 172.21.15.62:0/587702431 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.128+0000 7fe46effd700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe470102d80 0x7fe470197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46f7fe700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe470100460 0x7fe470197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46ffff700 1 -- 172.21.15.62:0/587702431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe470102d80 msgr2=0x7fe470197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46ffff700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe470102d80 0x7fe470197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46ffff700 1 -- 172.21.15.62:0/587702431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe470100460 msgr2=0x7fe470197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46ffff700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe470100460 0x7fe470197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46ffff700 1 -- 172.21.15.62:0/587702431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4600095f0 con 0x7fe4701056a0 2023-07-14T16:24:11.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46f7fe700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe470100460 0x7fe470197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:11.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46ffff700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4701056a0 0x7fe47019ec40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe468007ee0 tx=0x7fe46800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46cff9700 1 -- 172.21.15.62:0/587702431 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468017070 con 0x7fe4701056a0 2023-07-14T16:24:11.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe46cff9700 1 -- 172.21.15.62:0/587702431 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe46800f040 con 0x7fe4701056a0 2023-07-14T16:24:11.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.129+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47019f7c0 con 0x7fe4701056a0 2023-07-14T16:24:11.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.130+0000 7fe46cff9700 1 -- 172.21.15.62:0/587702431 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468005710 con 0x7fe4701056a0 2023-07-14T16:24:11.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.130+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47019fcd0 con 0x7fe4701056a0 2023-07-14T16:24:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.131+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe470107260 con 0x7fe4701056a0 2023-07-14T16:24:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.133+0000 7fe46cff9700 1 -- 172.21.15.62:0/587702431 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe468005870 con 0x7fe4701056a0 2023-07-14T16:24:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.134+0000 7fe46cff9700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4580a1240 0x7fe4580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.134+0000 7fe46cff9700 1 -- 172.21.15.62:0/587702431 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe4680cc160 con 0x7fe4701056a0 2023-07-14T16:24:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.135+0000 7fe46f7fe700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4580a1240 0x7fe4580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.135+0000 7fe46f7fe700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4580a1240 0x7fe4580a36f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe460006010 tx=0x7fe46000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.140+0000 7fe46cff9700 1 -- 172.21.15.62:0/587702431 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe468095860 con 0x7fe4701056a0 2023-07-14T16:24:11.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:10 smithi081 ceph-mon[127795]: pgmap v613: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:11.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.434+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe47002c850 con 0x7fe4701056a0 2023-07-14T16:24:11.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.435+0000 7fe46cff9700 1 -- 172.21.15.62:0/587702431 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fe468098e80 con 0x7fe4701056a0 2023-07-14T16:24:11.411 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:24:11.411 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:24:11.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4580a1240 msgr2=0x7fe4580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4580a1240 0x7fe4580a36f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe460006010 tx=0x7fe46000b260 comp rx=0 tx=0).stop 2023-07-14T16:24:11.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4701056a0 msgr2=0x7fe47019ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4701056a0 0x7fe47019ec40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe468007ee0 tx=0x7fe46800d3b0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 shutdown_connections 2023-07-14T16:24:11.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4701056a0 0x7fe47019ec40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4580a1240 0x7fe4580a36f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe470100460 0x7fe470197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 --2- 172.21.15.62:0/587702431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe470102d80 0x7fe470197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 >> 172.21.15.62:0/587702431 conn(0x7fe4700f9ee0 msgr2=0x7fe4700feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 shutdown_connections 2023-07-14T16:24:11.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.438+0000 7fe474f9d700 1 -- 172.21.15.62:0/587702431 wait complete. 2023-07-14T16:24:11.416 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:24:11.436 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:24:11.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 -- 172.21.15.62:0/1204165764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c074550 msgr2=0x7f7d9c0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 --2- 172.21.15.62:0/1204165764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c074550 0x7f7d9c0729a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009990 tx=0x7f7d98009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 -- 172.21.15.62:0/1204165764 shutdown_connections 2023-07-14T16:24:11.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 --2- 172.21.15.62:0/1204165764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9c1079a0 0x7f7d9c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 --2- 172.21.15.62:0/1204165764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c074550 0x7f7d9c0729a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 --2- 172.21.15.62:0/1204165764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d9c072f70 0x7f7d9c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 -- 172.21.15.62:0/1204165764 >> 172.21.15.62:0/1204165764 conn(0x7f7d9c0fb0e0 msgr2=0x7f7d9c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 -- 172.21.15.62:0/1204165764 shutdown_connections 2023-07-14T16:24:11.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.700+0000 7f7da3fff700 1 -- 172.21.15.62:0/1204165764 wait complete. 2023-07-14T16:24:11.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.701+0000 7f7da3fff700 1 Processor -- start 2023-07-14T16:24:11.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.701+0000 7f7da3fff700 1 -- start start 2023-07-14T16:24:11.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c072f70 0x7f7d9c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d9c074550 0x7f7d9c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9c1079a0 0x7f7d9c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19f4f0 con 0x7f7d9c072f70 2023-07-14T16:24:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da3fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19f630 con 0x7f7d9c1079a0 2023-07-14T16:24:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da3fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19f7c0 con 0x7f7d9c074550 2023-07-14T16:24:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c072f70 0x7f7d9c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c072f70 0x7f7d9c197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44906/0 (socket says 172.21.15.62:44906) 2023-07-14T16:24:11.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da1d9b700 1 -- 172.21.15.62:0/3280056060 learned_addr learned my addr 172.21.15.62:0/3280056060 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da159a700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d9c074550 0x7f7d9c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da1d9b700 1 -- 172.21.15.62:0/3280056060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d9c074550 msgr2=0x7f7d9c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d9c074550 0x7f7d9c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da1d9b700 1 -- 172.21.15.62:0/3280056060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9c1079a0 msgr2=0x7f7d9c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9c1079a0 0x7f7d9c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.702+0000 7f7da1d9b700 1 -- 172.21.15.62:0/3280056060 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d980095f0 con 0x7f7d9c072f70 2023-07-14T16:24:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.703+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c072f70 0x7f7d9c197970 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7d98004730 tx=0x7f7d98005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.703+0000 7f7d92ffd700 1 -- 172.21.15.62:0/3280056060 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9801d070 con 0x7f7d9c072f70 2023-07-14T16:24:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.703+0000 7f7d92ffd700 1 -- 172.21.15.62:0/3280056060 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d98004ca0 con 0x7f7d9c072f70 2023-07-14T16:24:11.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.703+0000 7f7d92ffd700 1 -- 172.21.15.62:0/3280056060 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98003d90 con 0x7f7d9c072f70 2023-07-14T16:24:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.703+0000 7f7da3fff700 1 -- 172.21.15.62:0/3280056060 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c19fa40 con 0x7f7d9c072f70 2023-07-14T16:24:11.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.703+0000 7f7da3fff700 1 -- 172.21.15.62:0/3280056060 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c19fe10 con 0x7f7d9c072f70 2023-07-14T16:24:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.708+0000 7f7d92ffd700 1 -- 172.21.15.62:0/3280056060 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7d98004810 con 0x7f7d9c072f70 2023-07-14T16:24:11.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.709+0000 7f7db7929b80 1 Processor -- start 2023-07-14T16:24:11.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7db7929b80 1 -- start start 2023-07-14T16:24:11.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7d92ffd700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7d880a11d0 0x7f7d880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba3294d0 0x560fba3298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba32a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba32ac70 0x560fba32b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7db7929b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba22ecb0 con 0x560fba329e20 2023-07-14T16:24:11.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7db7929b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba281a40 con 0x560fba32ac70 2023-07-14T16:24:11.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7db7929b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba27aed0 con 0x560fba3294d0 2023-07-14T16:24:11.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7d92ffd700 1 -- 172.21.15.62:0/3280056060 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d980cd9c0 con 0x7f7d9c072f70 2023-07-14T16:24:11.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7da159a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba32a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7da159a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba32a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44914/0 (socket says 172.21.15.62:44914) 2023-07-14T16:24:11.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.710+0000 7f7da159a700 1 -- 172.21.15.62:0/2110070700 learned_addr learned my addr 172.21.15.62:0/2110070700 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da259c700 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba32ac70 0x560fba32b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba3294d0 0x560fba3298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da259c700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7d880a11d0 0x7f7d880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da159a700 1 -- 172.21.15.62:0/2110070700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba3294d0 msgr2=0x560fba3298e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da159a700 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba3294d0 0x560fba3298e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da159a700 1 -- 172.21.15.62:0/2110070700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba32ac70 msgr2=0x560fba32b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da159a700 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba32ac70 0x560fba32b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da159a700 1 -- 172.21.15.62:0/2110070700 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fba0a8de0 con 0x560fba329e20 2023-07-14T16:24:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da259c700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7d880a11d0 0x7f7d880a3680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009690 tx=0x7f7d94009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.711+0000 7f7da159a700 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba32a6e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c008c20 tx=0x7f7d8c008f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da1d80b825707ee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.712+0000 7f7d87fff700 1 -- 172.21.15.62:0/2110070700 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00ed10 con 0x560fba329e20 2023-07-14T16:24:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.712+0000 7f7d87fff700 1 -- 172.21.15.62:0/2110070700 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d8c00ee70 con 0x560fba329e20 2023-07-14T16:24:11.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.712+0000 7f7d87fff700 1 -- 172.21.15.62:0/2110070700 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c005280 con 0x560fba329e20 2023-07-14T16:24:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.712+0000 7f7db7929b80 1 -- 172.21.15.62:0/2110070700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 msgr2=0x560fba32a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba32a6e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c008c20 tx=0x7f7d8c008f30 comp rx=0 tx=0).stop 2023-07-14T16:24:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 -- 172.21.15.62:0/2110070700 shutdown_connections 2023-07-14T16:24:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba32ac70 0x560fba32b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba3294d0 0x560fba3298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2110070700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba32a6e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 -- 172.21.15.62:0/2110070700 >> 172.21.15.62:0/2110070700 conn(0x560fba31eda0 msgr2=0x560fba3241d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 -- 172.21.15.62:0/2110070700 shutdown_connections 2023-07-14T16:24:11.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 -- 172.21.15.62:0/2110070700 wait complete. 2023-07-14T16:24:11.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.713+0000 7f7db7929b80 1 -- start start 2023-07-14T16:24:11.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7db7929b80 1 Processor -- start 2023-07-14T16:24:11.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba3330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba333620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 0x560fba33a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7db7929b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba21aac0 con 0x560fba329e20 2023-07-14T16:24:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7db7929b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba21a7c0 con 0x560fba3294d0 2023-07-14T16:24:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7db7929b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba0fe920 con 0x560fba32ac70 2023-07-14T16:24:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7db7929b80 10 client.0 _ll_register_callbacks cb 0x560fba218760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7da159a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba333620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7da159a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba333620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44922/0 (socket says 172.21.15.62:44922) 2023-07-14T16:24:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.714+0000 7f7da159a700 1 -- 172.21.15.62:0/3996431275 learned_addr learned my addr 172.21.15.62:0/3996431275 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7da259c700 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 0x560fba33a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba3330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7da159a700 1 -- 172.21.15.62:0/3996431275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 msgr2=0x560fba33a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7da159a700 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 0x560fba33a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7da159a700 1 -- 172.21.15.62:0/3996431275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 msgr2=0x560fba3330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7da159a700 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba3330e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7da159a700 1 -- 172.21.15.62:0/3996431275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8c008850 con 0x560fba329e20 2023-07-14T16:24:11.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7da159a700 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba333620 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c008bf0 tx=0x7f7d8c004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b4479654c17773 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.715+0000 7f7d857fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:11.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.716+0000 7f7d857fa700 1 -- 172.21.15.62:0/3996431275 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c01f070 con 0x560fba329e20 2023-07-14T16:24:11.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.716+0000 7f7db7929b80 1 -- 172.21.15.62:0/3996431275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fba1d44f0 con 0x560fba329e20 2023-07-14T16:24:11.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.716+0000 7f7d857fa700 1 -- 172.21.15.62:0/3996431275 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d8c004300 con 0x560fba329e20 2023-07-14T16:24:11.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.716+0000 7f7d857fa700 1 -- 172.21.15.62:0/3996431275 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d8c0044c0 con 0x560fba329e20 2023-07-14T16:24:11.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.716+0000 7f7db7929b80 10 client.17868 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:11.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.716+0000 7f7db7929b80 1 -- 172.21.15.62:0/3996431275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560fba091c60 con 0x560fba329e20 2023-07-14T16:24:11.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7d857fa700 1 -- 172.21.15.62:0/3996431275 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560fba1d44f0 con 0x560fba329e20 2023-07-14T16:24:11.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7d857fa700 1 -- 172.21.15.62:0/3996431275 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7d8c005150 con 0x560fba329e20 2023-07-14T16:24:11.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7db7929b80 10 client.17868 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:11.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7db7929b80 10 client.17868 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:24:11.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7db7929b80 20 client.17868 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:11.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7db7929b80 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x560fba33ee20 0x560fba341230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7db7929b80 4 client.17868 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:11.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7db7929b80 1 -- 172.21.15.62:0/3996431275 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fba273eb0 con 0x560fba33ee20 2023-07-14T16:24:11.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.717+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x560fba33ee20 0x560fba341230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.718+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x560fba33ee20 0x560fba341230 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de664c8aa2c0b5cc server_cookie=2333d959b53647c7 in_seq=0 out_seq=0 2023-07-14T16:24:11.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.718+0000 7f7d857fa700 10 client.17868 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:24:11.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7d857fa700 1 -- 172.21.15.62:0/3996431275 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560fba273eb0 con 0x560fba33ee20 2023-07-14T16:24:11.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7d857fa700 10 client.17868 handle_command_reply: tid=0 2023-07-14T16:24:11.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 client.17868 shutdown 2023-07-14T16:24:11.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 -- 172.21.15.62:0/3996431275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 msgr2=0x560fba333620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba333620 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c008bf0 tx=0x7f7d8c004980 comp rx=0 tx=0).stop 2023-07-14T16:24:11.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 -- 172.21.15.62:0/3996431275 shutdown_connections 2023-07-14T16:24:11.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x560fba33ee20 0x560fba341230 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 0x560fba33a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba3330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 --2- 172.21.15.62:0/3996431275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba333620 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7d857fa700 0 client.17868 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:24:11.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 -- 172.21.15.62:0/3996431275 >> 172.21.15.62:0/3996431275 conn(0x560fba31eda0 msgr2=0x560fba3238b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.719+0000 7f7db7929b80 1 -- 172.21.15.62:0/3996431275 shutdown_connections 2023-07-14T16:24:11.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.720+0000 7f7db7929b80 1 -- 172.21.15.62:0/3996431275 wait complete. 2023-07-14T16:24:11.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.720+0000 7f7db7929b80 20 client.17868 trim_cache size 0 max 16384 2023-07-14T16:24:11.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.736+0000 7f7db7929b80 1 Processor -- start 2023-07-14T16:24:11.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7db7929b80 1 -- start start 2023-07-14T16:24:11.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba3330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba329e20 0x560fba33a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba32ac70 0x560fba333620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7db7929b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba333d60 con 0x560fba32ac70 2023-07-14T16:24:11.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7db7929b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba330d30 con 0x560fba3294d0 2023-07-14T16:24:11.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7db7929b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba334230 con 0x560fba329e20 2023-07-14T16:24:11.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7da259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba32ac70 0x560fba333620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7da259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba32ac70 0x560fba333620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44926/0 (socket says 172.21.15.62:44926) 2023-07-14T16:24:11.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7da259c700 1 -- 172.21.15.62:0/1953923379 learned_addr learned my addr 172.21.15.62:0/1953923379 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7da159a700 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba329e20 0x560fba33a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.737+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba3330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.738+0000 7f7da259c700 1 -- 172.21.15.62:0/1953923379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba329e20 msgr2=0x560fba33a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.738+0000 7f7da259c700 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba329e20 0x560fba33a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.738+0000 7f7da259c700 1 -- 172.21.15.62:0/1953923379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 msgr2=0x560fba3330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.738+0000 7f7da259c700 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba3330e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.738+0000 7f7da259c700 1 -- 172.21.15.62:0/1953923379 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fba0cf250 con 0x560fba32ac70 2023-07-14T16:24:11.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.738+0000 7f7da259c700 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba32ac70 0x560fba333620 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009890 tx=0x7f7d94009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3521be3f201bf502 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.738+0000 7f7d90ff9700 1 -- 172.21.15.62:0/1953923379 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94020040 con 0x560fba32ac70 2023-07-14T16:24:11.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7d90ff9700 1 -- 172.21.15.62:0/1953923379 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d9400f030 con 0x560fba32ac70 2023-07-14T16:24:11.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7d90ff9700 1 -- 172.21.15.62:0/1953923379 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94009ea0 con 0x560fba32ac70 2023-07-14T16:24:11.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7db7929b80 1 -- 172.21.15.62:0/1953923379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba32ac70 msgr2=0x560fba333620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7db7929b80 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba32ac70 0x560fba333620 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009890 tx=0x7f7d94009000 comp rx=0 tx=0).stop 2023-07-14T16:24:11.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7db7929b80 1 -- 172.21.15.62:0/1953923379 shutdown_connections 2023-07-14T16:24:11.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7db7929b80 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba32ac70 0x560fba333620 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7db7929b80 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba3330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7db7929b80 1 --2- 172.21.15.62:0/1953923379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba329e20 0x560fba33a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.739+0000 7f7db7929b80 1 -- 172.21.15.62:0/1953923379 >> 172.21.15.62:0/1953923379 conn(0x560fba31eda0 msgr2=0x560fba345b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.740+0000 7f7db7929b80 1 -- 172.21.15.62:0/1953923379 shutdown_connections 2023-07-14T16:24:11.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.740+0000 7f7db7929b80 1 -- 172.21.15.62:0/1953923379 wait complete. 2023-07-14T16:24:11.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.740+0000 7f7db7929b80 1 -- start start 2023-07-14T16:24:11.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.740+0000 7f7db7929b80 1 Processor -- start 2023-07-14T16:24:11.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba339e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba33a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7db7929b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 0x560fba3368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7db7929b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba336e60 con 0x560fba329e20 2023-07-14T16:24:11.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7db7929b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba336fd0 con 0x560fba3294d0 2023-07-14T16:24:11.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7db7929b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fba337190 con 0x560fba32ac70 2023-07-14T16:24:11.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7db7929b80 10 client.0 _ll_register_callbacks cb 0x7f7d94012260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:11.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7da159a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba33a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7da159a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba33a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44930/0 (socket says 172.21.15.62:44930) 2023-07-14T16:24:11.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7da159a700 1 -- 172.21.15.62:0/2263802912 learned_addr learned my addr 172.21.15.62:0/2263802912 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba339e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.741+0000 7f7da259c700 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 0x560fba3368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7da1d9b700 1 -- 172.21.15.62:0/2263802912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 msgr2=0x560fba3368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 0x560fba3368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7da1d9b700 1 -- 172.21.15.62:0/2263802912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 msgr2=0x560fba33a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba33a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7da1d9b700 1 -- 172.21.15.62:0/2263802912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d94012d90 con 0x560fba3294d0 2023-07-14T16:24:11.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7da159a700 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba33a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:11.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7da1d9b700 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba339e40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800b5c0 tx=0x7f7d9809eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d930ba26375af1e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7d857fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:24:11.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7d857fa700 1 -- 172.21.15.62:0/2263802912 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98099210 con 0x560fba3294d0 2023-07-14T16:24:11.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7d857fa700 1 -- 172.21.15.62:0/2263802912 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d9801ac30 con 0x560fba3294d0 2023-07-14T16:24:11.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.742+0000 7f7db7929b80 1 -- 172.21.15.62:0/2263802912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d8c00ee70 con 0x560fba3294d0 2023-07-14T16:24:11.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.746+0000 7f7d857fa700 1 -- 172.21.15.62:0/2263802912 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d9801adf0 con 0x560fba3294d0 2023-07-14T16:24:11.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.746+0000 7f7db7929b80 10 client.27512 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:11.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.746+0000 7f7db7929b80 1 -- 172.21.15.62:0/2263802912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560fba0a72b0 con 0x560fba3294d0 2023-07-14T16:24:11.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.746+0000 7f7d857fa700 1 -- 172.21.15.62:0/2263802912 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d980b7080 con 0x560fba3294d0 2023-07-14T16:24:11.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.746+0000 7f7d857fa700 1 -- 172.21.15.62:0/2263802912 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7d980b8f80 con 0x560fba3294d0 2023-07-14T16:24:11.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.746+0000 7f7db7929b80 10 client.27512 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:11.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.746+0000 7f7db7929b80 10 client.27512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:24:11.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.747+0000 7f7db7929b80 20 client.27512 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:11.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.747+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x560fba393df0 0x560fba396200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.747+0000 7f7db7929b80 4 client.27512 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:24:11.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.747+0000 7f7db7929b80 1 -- 172.21.15.62:0/2263802912 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560fba236050 con 0x560fba393df0 2023-07-14T16:24:11.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.747+0000 7f7da159a700 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x560fba393df0 0x560fba396200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.748+0000 7f7d857fa700 10 client.27512 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:24:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.748+0000 7f7da159a700 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x560fba393df0 0x560fba396200 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=731974c64c1f1330 server_cookie=1746f264088b966f in_seq=0 out_seq=0 2023-07-14T16:24:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.748+0000 7f7d857fa700 1 -- 172.21.15.62:0/2263802912 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560fba236050 con 0x560fba393df0 2023-07-14T16:24:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.748+0000 7f7d857fa700 10 client.27512 handle_command_reply: tid=0 2023-07-14T16:24:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.748+0000 7f7db7929b80 1 client.27512 shutdown 2023-07-14T16:24:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 -- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 msgr2=0x560fba339e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba339e40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800b5c0 tx=0x7f7d9809eea0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 -- 172.21.15.62:0/2263802912 shutdown_connections 2023-07-14T16:24:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x560fba393df0 0x560fba396200 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fba32ac70 0x560fba3368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560fba3294d0 0x560fba339e40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7d857fa700 0 client.27512 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:24:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 --2- 172.21.15.62:0/2263802912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560fba329e20 0x560fba33a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 -- 172.21.15.62:0/2263802912 >> 172.21.15.62:0/2263802912 conn(0x560fba31eda0 msgr2=0x560fba32da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 -- 172.21.15.62:0/2263802912 shutdown_connections 2023-07-14T16:24:11.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 1 -- 172.21.15.62:0/2263802912 wait complete. 2023-07-14T16:24:11.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.749+0000 7f7db7929b80 20 client.27512 trim_cache size 0 max 16384 2023-07-14T16:24:11.729 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:24:11.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 -- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7d880a11d0 msgr2=0x7f7d880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7d880a11d0 0x7f7d880a3680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009690 tx=0x7f7d94009600 comp rx=0 tx=0).stop 2023-07-14T16:24:11.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 -- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c072f70 msgr2=0x7f7d9c197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c072f70 0x7f7d9c197970 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7d98004730 tx=0x7f7d98005e80 comp rx=0 tx=0).stop 2023-07-14T16:24:11.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 -- 172.21.15.62:0/3280056060 shutdown_connections 2023-07-14T16:24:11.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7d880a11d0 0x7f7d880a3680 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9c1079a0 0x7f7d9c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d9c072f70 0x7f7d9c197970 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 --2- 172.21.15.62:0/3280056060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d9c074550 0x7f7d9c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 -- 172.21.15.62:0/3280056060 >> 172.21.15.62:0/3280056060 conn(0x7f7d9c0fb0e0 msgr2=0x7f7d9c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 -- 172.21.15.62:0/3280056060 shutdown_connections 2023-07-14T16:24:11.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:11.750+0000 7f7da3fff700 1 -- 172.21.15.62:0/3280056060 wait complete. 2023-07-14T16:24:11.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:24:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 -- 172.21.15.62:0/1009127722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418101670 msgr2=0x7f5418107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 --2- 172.21.15.62:0/1009127722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418101670 0x7f5418107f30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f541000b0a0 tx=0x7f541000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 -- 172.21.15.62:0/1009127722 shutdown_connections 2023-07-14T16:24:11.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 --2- 172.21.15.62:0/1009127722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5418101670 0x7f5418107f30 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 --2- 172.21.15.62:0/1009127722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54180ffaf0 0x7f54180fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 --2- 172.21.15.62:0/1009127722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418100ce0 0x7f5418101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 -- 172.21.15.62:0/1009127722 >> 172.21.15.62:0/1009127722 conn(0x7f54180fb130 msgr2=0x7f54180fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 -- 172.21.15.62:0/1009127722 shutdown_connections 2023-07-14T16:24:11.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.001+0000 7f541fbff700 1 -- 172.21.15.62:0/1009127722 wait complete. 2023-07-14T16:24:11.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.002+0000 7f541fbff700 1 Processor -- start 2023-07-14T16:24:11.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.003+0000 7f541fbff700 1 -- start start 2023-07-14T16:24:11.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.003+0000 7f541fbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54180ffaf0 0x7f5418197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.003+0000 7f541fbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5418100ce0 0x7f5418197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.003+0000 7f541fbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418101670 0x7f541819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.003+0000 7f541fbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541819f5f0 con 0x7f54180ffaf0 2023-07-14T16:24:11.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.003+0000 7f541fbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541819f730 con 0x7f5418101670 2023-07-14T16:24:11.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.003+0000 7f541fbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541819f8c0 con 0x7f5418100ce0 2023-07-14T16:24:11.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.003+0000 7f541d99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54180ffaf0 0x7f5418197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d19a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5418100ce0 0x7f5418197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54180ffaf0 0x7f5418197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44940/0 (socket says 172.21.15.62:44940) 2023-07-14T16:24:11.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d99b700 1 -- 172.21.15.62:0/3687878017 learned_addr learned my addr 172.21.15.62:0/3687878017 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541e19c700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418101670 0x7f541819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d99b700 1 -- 172.21.15.62:0/3687878017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5418100ce0 msgr2=0x7f5418197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d99b700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5418100ce0 0x7f5418197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d99b700 1 -- 172.21.15.62:0/3687878017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418101670 msgr2=0x7f541819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d99b700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418101670 0x7f541819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d99b700 1 -- 172.21.15.62:0/3687878017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5410009d00 con 0x7f54180ffaf0 2023-07-14T16:24:11.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.004+0000 7f541d99b700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54180ffaf0 0x7f5418197a70 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f541400e970 tx=0x7f541400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.005+0000 7f540effd700 1 -- 172.21.15.62:0/3687878017 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414009790 con 0x7f54180ffaf0 2023-07-14T16:24:11.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.005+0000 7f541fbff700 1 -- 172.21.15.62:0/3687878017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541819fba0 con 0x7f54180ffaf0 2023-07-14T16:24:11.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.005+0000 7f540effd700 1 -- 172.21.15.62:0/3687878017 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f541400c7e0 con 0x7f54180ffaf0 2023-07-14T16:24:11.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.005+0000 7f540effd700 1 -- 172.21.15.62:0/3687878017 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414003bd0 con 0x7f54180ffaf0 2023-07-14T16:24:11.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.005+0000 7f541fbff700 1 -- 172.21.15.62:0/3687878017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54181a00b0 con 0x7f54180ffaf0 2023-07-14T16:24:11.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.009+0000 7f540effd700 1 -- 172.21.15.62:0/3687878017 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f541400c950 con 0x7f54180ffaf0 2023-07-14T16:24:11.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.010+0000 7f540effd700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f54040a1150 0x7f54040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.010+0000 7f540effd700 1 -- 172.21.15.62:0/3687878017 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5414014070 con 0x7f54180ffaf0 2023-07-14T16:24:11.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.010+0000 7f5433442b80 1 Processor -- start 2023-07-14T16:24:11.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.010+0000 7f541d19a700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f54040a1150 0x7f54040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f5433442b80 1 -- start start 2023-07-14T16:24:11.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f5433442b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f5433442b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f5433442b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f5433442b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8d0ccb0 con 0x5622e8e074d0 2023-07-14T16:24:11.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f5433442b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8d5fa40 con 0x5622e8e07e20 2023-07-14T16:24:11.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f541d19a700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f54040a1150 0x7f54040a3600 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5408009c60 tx=0x7f5408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f5433442b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8d58ed0 con 0x5622e8e08c70 2023-07-14T16:24:11.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f541d99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f541d99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44954/0 (socket says 172.21.15.62:44954) 2023-07-14T16:24:11.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.011+0000 7f541d99b700 1 -- 172.21.15.62:0/4120582843 learned_addr learned my addr 172.21.15.62:0/4120582843 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f541d19a700 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f541e19c700 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f541d99b700 1 -- 172.21.15.62:0/4120582843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 msgr2=0x5622e8e090e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f541d99b700 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e090e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f541d99b700 1 -- 172.21.15.62:0/4120582843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 msgr2=0x5622e8e086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f541d99b700 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f541d99b700 1 -- 172.21.15.62:0/4120582843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622e8b86de0 con 0x5622e8e074d0 2023-07-14T16:24:11.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f541d99b700 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e078e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5414096980 tx=0x7f5414005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1cc05d8cc1de2d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.012+0000 7f5403fff700 1 -- 172.21.15.62:0/4120582843 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54140b1070 con 0x5622e8e074d0 2023-07-14T16:24:11.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5403fff700 1 -- 172.21.15.62:0/4120582843 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f541409df40 con 0x5622e8e074d0 2023-07-14T16:24:11.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5403fff700 1 -- 172.21.15.62:0/4120582843 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541409a170 con 0x5622e8e074d0 2023-07-14T16:24:11.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5433442b80 1 -- 172.21.15.62:0/4120582843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 msgr2=0x5622e8e078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:11.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5433442b80 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e078e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5414096980 tx=0x7f5414005910 comp rx=0 tx=0).stop 2023-07-14T16:24:11.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5433442b80 1 -- 172.21.15.62:0/4120582843 shutdown_connections 2023-07-14T16:24:11.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5433442b80 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5433442b80 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e078e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5433442b80 1 --2- 172.21.15.62:0/4120582843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5433442b80 1 -- 172.21.15.62:0/4120582843 >> 172.21.15.62:0/4120582843 conn(0x5622e8dfcda0 msgr2=0x5622e8e021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:11.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.013+0000 7f5433442b80 1 -- 172.21.15.62:0/4120582843 shutdown_connections 2023-07-14T16:24:11.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.014+0000 7f5433442b80 1 -- 172.21.15.62:0/4120582843 wait complete. 2023-07-14T16:24:11.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.014+0000 7f5433442b80 1 -- start start 2023-07-14T16:24:11.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.014+0000 7f5433442b80 1 Processor -- start 2023-07-14T16:24:11.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f5433442b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e074d0 0x5622e8e111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f5433442b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e07e20 0x5622e8e116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f5433442b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e18730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:11.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f5433442b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8cf8ac0 con 0x5622e8e08c70 2023-07-14T16:24:11.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f5433442b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8cf87c0 con 0x5622e8e074d0 2023-07-14T16:24:11.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f5433442b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8bdc920 con 0x5622e8e07e20 2023-07-14T16:24:11.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f5433442b80 10 client.0 _ll_register_callbacks cb 0x5622e8cf6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:11.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f541e19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e18730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f541e19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e18730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44962/0 (socket says 172.21.15.62:44962) 2023-07-14T16:24:11.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f541e19c700 1 -- 172.21.15.62:0/3112831350 learned_addr learned my addr 172.21.15.62:0/3112831350 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:11.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.015+0000 7f541d99b700 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e074d0 0x5622e8e111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f541d19a700 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e07e20 0x5622e8e116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:11.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f541e19c700 1 -- 172.21.15.62:0/3112831350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e07e20 msgr2=0x5622e8e116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f541e19c700 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e07e20 0x5622e8e116f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f541e19c700 1 -- 172.21.15.62:0/3112831350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e074d0 msgr2=0x5622e8e111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:11.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f541e19c700 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e074d0 0x5622e8e111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:11.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f541e19c700 1 -- 172.21.15.62:0/3112831350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54181a00b0 con 0x5622e8e08c70 2023-07-14T16:24:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f541e19c700 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e18730 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f541000b0a0 tx=0x7f5410007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60853b9e1443045 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f54017fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f54017fa700 1 -- 172.21.15.62:0/3112831350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000e040 con 0x5622e8e08c70 2023-07-14T16:24:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f5433442b80 1 -- 172.21.15.62:0/3112831350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622e8cb24f0 con 0x5622e8e08c70 2023-07-14T16:24:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.016+0000 7f54017fa700 1 -- 172.21.15.62:0/3112831350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5410003a00 con 0x5622e8e08c70 2023-07-14T16:24:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f54017fa700 1 -- 172.21.15.62:0/3112831350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5410007780 con 0x5622e8e08c70 2023-07-14T16:24:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f5433442b80 10 client.17886 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f5433442b80 1 -- 172.21.15.62:0/3112831350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5622e8b6fc60 con 0x5622e8e08c70 2023-07-14T16:24:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f54017fa700 1 -- 172.21.15.62:0/3112831350 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5622e8cb24f0 con 0x5622e8e08c70 2023-07-14T16:24:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f54017fa700 1 -- 172.21.15.62:0/3112831350 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f541002e070 con 0x5622e8e08c70 2023-07-14T16:24:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f5433442b80 10 client.17886 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f5433442b80 10 client.17886 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:24:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f5433442b80 20 client.17886 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f5433442b80 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5622e8e1cf40 0x5622e8e1f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f5433442b80 4 client.17886 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:12.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.017+0000 7f5433442b80 1 -- 172.21.15.62:0/3112831350 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622e8d51eb0 con 0x5622e8e1cf40 2023-07-14T16:24:12.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.018+0000 7f541d99b700 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5622e8e1cf40 0x5622e8e1f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.018+0000 7f54017fa700 10 client.17886 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:24:12.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.018+0000 7f541d99b700 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5622e8e1cf40 0x5622e8e1f350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf5c28067921b6db server_cookie=a8a41536364b7dc8 in_seq=0 out_seq=0 2023-07-14T16:24:12.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.019+0000 7f54017fa700 1 -- 172.21.15.62:0/3112831350 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5622e8d51eb0 con 0x5622e8e1cf40 2023-07-14T16:24:12.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.019+0000 7f54017fa700 10 client.17886 handle_command_reply: tid=0 2023-07-14T16:24:12.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.019+0000 7f5433442b80 1 client.17886 shutdown 2023-07-14T16:24:12.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 -- 172.21.15.62:0/3112831350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 msgr2=0x5622e8e18730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e18730 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f541000b0a0 tx=0x7f5410007d50 comp rx=0 tx=0).stop 2023-07-14T16:24:12.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 -- 172.21.15.62:0/3112831350 shutdown_connections 2023-07-14T16:24:12.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5622e8e1cf40 0x5622e8e1f350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e18730 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e074d0 0x5622e8e111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 --2- 172.21.15.62:0/3112831350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e07e20 0x5622e8e116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 -- 172.21.15.62:0/3112831350 >> 172.21.15.62:0/3112831350 conn(0x5622e8dfcda0 msgr2=0x5622e8e018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f54017fa700 0 client.17886 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:24:12.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 -- 172.21.15.62:0/3112831350 shutdown_connections 2023-07-14T16:24:12.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 1 -- 172.21.15.62:0/3112831350 wait complete. 2023-07-14T16:24:12.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.020+0000 7f5433442b80 20 client.17886 trim_cache size 0 max 16384 2023-07-14T16:24:12.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.036+0000 7f5433442b80 1 Processor -- start 2023-07-14T16:24:12.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.036+0000 7f5433442b80 1 -- start start 2023-07-14T16:24:12.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.036+0000 7f5433442b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.036+0000 7f5433442b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.036+0000 7f5433442b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e0bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.036+0000 7f5433442b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8e10d20 con 0x5622e8e074d0 2023-07-14T16:24:12.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.036+0000 7f5433442b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8e0f800 con 0x5622e8e07e20 2023-07-14T16:24:12.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.036+0000 7f5433442b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8e03070 con 0x5622e8e08c70 2023-07-14T16:24:12.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e1ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e1ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44968/0 (socket says 172.21.15.62:44968) 2023-07-14T16:24:12.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 -- 172.21.15.62:0/3959727280 learned_addr learned my addr 172.21.15.62:0/3959727280 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d19a700 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541e19c700 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e0bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 -- 172.21.15.62:0/3959727280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 msgr2=0x5622e8e0bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e0bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 -- 172.21.15.62:0/3959727280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 msgr2=0x5622e8e18650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e18650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 -- 172.21.15.62:0/3959727280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622e8bad250 con 0x5622e8e074d0 2023-07-14T16:24:12.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541e19c700 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e0bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:24:12.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.037+0000 7f541d99b700 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e1ef50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5414099c10 tx=0x7f541400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3da154df62f464 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.038+0000 7f540cbff700 1 -- 172.21.15.62:0/3959727280 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54140b1040 con 0x5622e8e074d0 2023-07-14T16:24:12.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.038+0000 7f540cbff700 1 -- 172.21.15.62:0/3959727280 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54140980d0 con 0x5622e8e074d0 2023-07-14T16:24:12.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.038+0000 7f540cbff700 1 -- 172.21.15.62:0/3959727280 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541409a170 con 0x5622e8e074d0 2023-07-14T16:24:12.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.038+0000 7f5433442b80 1 -- 172.21.15.62:0/3959727280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 msgr2=0x5622e8e1ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.038+0000 7f5433442b80 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e1ef50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5414099c10 tx=0x7f541400cf50 comp rx=0 tx=0).stop 2023-07-14T16:24:12.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 -- 172.21.15.62:0/3959727280 shutdown_connections 2023-07-14T16:24:12.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e08c70 0x5622e8e0bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e074d0 0x5622e8e1ef50 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 --2- 172.21.15.62:0/3959727280 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e18650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 -- 172.21.15.62:0/3959727280 >> 172.21.15.62:0/3959727280 conn(0x5622e8dfcda0 msgr2=0x5622e8e0fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 -- 172.21.15.62:0/3959727280 shutdown_connections 2023-07-14T16:24:12.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 -- 172.21.15.62:0/3959727280 wait complete. 2023-07-14T16:24:12.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 -- start start 2023-07-14T16:24:12.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.039+0000 7f5433442b80 1 Processor -- start 2023-07-14T16:24:12.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f5433442b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e074d0 0x5622e8e18010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f5433442b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e18550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f5433442b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e16620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f5433442b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8e16d00 con 0x5622e8e08c70 2023-07-14T16:24:12.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f5433442b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8e16e70 con 0x5622e8e07e20 2023-07-14T16:24:12.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f5433442b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8e17030 con 0x5622e8e074d0 2023-07-14T16:24:12.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f5433442b80 10 client.0 _ll_register_callbacks cb 0x7f5410009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:12.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f541e19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e16620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f541e19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e16620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44970/0 (socket says 172.21.15.62:44970) 2023-07-14T16:24:12.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f541e19c700 1 -- 172.21.15.62:0/352230223 learned_addr learned my addr 172.21.15.62:0/352230223 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.040+0000 7f541d19a700 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e18550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f541d99b700 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e074d0 0x5622e8e18010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f541e19c700 1 -- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e074d0 msgr2=0x5622e8e18010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f541e19c700 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e074d0 0x5622e8e18010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f541e19c700 1 -- 172.21.15.62:0/352230223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 msgr2=0x5622e8e18550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f541e19c700 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e18550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f541e19c700 1 -- 172.21.15.62:0/352230223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622e8b86de0 con 0x5622e8e08c70 2023-07-14T16:24:12.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f541d99b700 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e074d0 0x5622e8e18010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f541e19c700 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e16620 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f54100075d0 tx=0x7f54100097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2e7fd590dceb6d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f54017fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f54017fa700 1 -- 172.21.15.62:0/352230223 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000e030 con 0x5622e8e08c70 2023-07-14T16:24:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f5433442b80 1 -- 172.21.15.62:0/352230223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f541409df40 con 0x5622e8e08c70 2023-07-14T16:24:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.041+0000 7f54017fa700 1 -- 172.21.15.62:0/352230223 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f54100076f0 con 0x5622e8e08c70 2023-07-14T16:24:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f54017fa700 1 -- 172.21.15.62:0/352230223 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5410003c00 con 0x5622e8e08c70 2023-07-14T16:24:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f5433442b80 10 client.17892 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f5433442b80 1 -- 172.21.15.62:0/352230223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5622e8b852b0 con 0x5622e8e08c70 2023-07-14T16:24:12.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f54017fa700 1 -- 172.21.15.62:0/352230223 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f541409df40 con 0x5622e8e08c70 2023-07-14T16:24:12.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f54017fa700 1 -- 172.21.15.62:0/352230223 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5410019040 con 0x5622e8e08c70 2023-07-14T16:24:12.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f5433442b80 10 client.17892 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:12.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f5433442b80 10 client.17892 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:24:12.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f5433442b80 20 client.17892 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:12.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f5433442b80 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5622e8e71e30 0x5622e8e74240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f5433442b80 4 client.17892 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:24:12.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.042+0000 7f5433442b80 1 -- 172.21.15.62:0/352230223 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5622e8d14050 con 0x5622e8e71e30 2023-07-14T16:24:12.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.043+0000 7f541d99b700 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5622e8e71e30 0x5622e8e74240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.043+0000 7f541d99b700 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5622e8e71e30 0x5622e8e74240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a50a9892212586af server_cookie=9d3c0462299a18cd in_seq=0 out_seq=0 2023-07-14T16:24:12.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.043+0000 7f54017fa700 10 client.17892 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:24:12.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f54017fa700 1 -- 172.21.15.62:0/352230223 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622e8d14050 con 0x5622e8e71e30 2023-07-14T16:24:12.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f54017fa700 10 client.17892 handle_command_reply: tid=0 2023-07-14T16:24:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 client.17892 shutdown 2023-07-14T16:24:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 -- 172.21.15.62:0/352230223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 msgr2=0x5622e8e16620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e16620 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f54100075d0 tx=0x7f54100097f0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 -- 172.21.15.62:0/352230223 shutdown_connections 2023-07-14T16:24:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5622e8e71e30 0x5622e8e74240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e8e08c70 0x5622e8e16620 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f54017fa700 0 client.17892 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:24:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e8e074d0 0x5622e8e18010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 --2- 172.21.15.62:0/352230223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e8e07e20 0x5622e8e18550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 -- 172.21.15.62:0/352230223 >> 172.21.15.62:0/352230223 conn(0x5622e8dfcda0 msgr2=0x5622e8e1ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 -- 172.21.15.62:0/352230223 shutdown_connections 2023-07-14T16:24:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 1 -- 172.21.15.62:0/352230223 wait complete. 2023-07-14T16:24:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.044+0000 7f5433442b80 20 client.17892 trim_cache size 0 max 16384 2023-07-14T16:24:12.028 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:24:12.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 -- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f54040a1150 msgr2=0x7f54040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f54040a1150 0x7f54040a3600 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5408009c60 tx=0x7f5408009260 comp rx=0 tx=0).stop 2023-07-14T16:24:12.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 -- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54180ffaf0 msgr2=0x7f5418197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54180ffaf0 0x7f5418197a70 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f541400e970 tx=0x7f541400ec80 comp rx=0 tx=0).stop 2023-07-14T16:24:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 -- 172.21.15.62:0/3687878017 shutdown_connections 2023-07-14T16:24:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f54040a1150 0x7f54040a3600 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5418101670 0x7f541819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54180ffaf0 0x7f5418197a70 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 --2- 172.21.15.62:0/3687878017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5418100ce0 0x7f5418197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 -- 172.21.15.62:0/3687878017 >> 172.21.15.62:0/3687878017 conn(0x7f54180fb130 msgr2=0x7f5418106560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 -- 172.21.15.62:0/3687878017 shutdown_connections 2023-07-14T16:24:12.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.045+0000 7f541fbff700 1 -- 172.21.15.62:0/3687878017 wait complete. 2023-07-14T16:24:12.031 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:11 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/587702431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:24:12.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:11 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/587702431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:24:12.042 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:24:12.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:11 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/587702431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:24:12.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.293+0000 7fc1c27ff700 1 -- 172.21.15.62:0/3888201649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc0744f0 msgr2=0x7fc1bc072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.293+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/3888201649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc0744f0 0x7fc1bc072940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009990 tx=0x7fc1ac009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.293+0000 7fc1c27ff700 1 -- 172.21.15.62:0/3888201649 shutdown_connections 2023-07-14T16:24:12.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.293+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/3888201649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1bc1079a0 0x7fc1bc109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.293+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/3888201649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc0744f0 0x7fc1bc072940 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.293+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/3888201649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1bc072f10 0x7fc1bc073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.294+0000 7fc1c27ff700 1 -- 172.21.15.62:0/3888201649 >> 172.21.15.62:0/3888201649 conn(0x7fc1bc0fb130 msgr2=0x7fc1bc0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.294+0000 7fc1c27ff700 1 -- 172.21.15.62:0/3888201649 shutdown_connections 2023-07-14T16:24:12.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.294+0000 7fc1c27ff700 1 -- 172.21.15.62:0/3888201649 wait complete. 2023-07-14T16:24:12.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.295+0000 7fc1c27ff700 1 Processor -- start 2023-07-14T16:24:12.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.295+0000 7fc1c27ff700 1 -- start start 2023-07-14T16:24:12.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.295+0000 7fc1c27ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc072f10 0x7fc1bc197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.295+0000 7fc1c27ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1bc0744f0 0x7fc1bc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.295+0000 7fc1c27ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1bc1079a0 0x7fc1bc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.295+0000 7fc1c27ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc19f4e0 con 0x7fc1bc072f10 2023-07-14T16:24:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.295+0000 7fc1c27ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc19f620 con 0x7fc1bc1079a0 2023-07-14T16:24:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.295+0000 7fc1c27ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc19f7b0 con 0x7fc1bc0744f0 2023-07-14T16:24:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc072f10 0x7fc1bc197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc072f10 0x7fc1bc197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44988/0 (socket says 172.21.15.62:44988) 2023-07-14T16:24:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bbfff700 1 -- 172.21.15.62:0/516886115 learned_addr learned my addr 172.21.15.62:0/516886115 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bb7fe700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1bc0744f0 0x7fc1bc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1bc1079a0 0x7fc1bc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bbfff700 1 -- 172.21.15.62:0/516886115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1bc0744f0 msgr2=0x7fc1bc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1bc0744f0 0x7fc1bc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bbfff700 1 -- 172.21.15.62:0/516886115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1bc1079a0 msgr2=0x7fc1bc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1bc1079a0 0x7fc1bc19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1bbfff700 1 -- 172.21.15.62:0/516886115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac0095f0 con 0x7fc1bc072f10 2023-07-14T16:24:12.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.296+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1bc1079a0 0x7fc1bc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:12.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.297+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc072f10 0x7fc1bc197960 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0098e0 tx=0x7fc1ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.297+0000 7fc1b97fa700 1 -- 172.21.15.62:0/516886115 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac01d070 con 0x7fc1bc072f10 2023-07-14T16:24:12.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.297+0000 7fc1b97fa700 1 -- 172.21.15.62:0/516886115 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1ac00be30 con 0x7fc1bc072f10 2023-07-14T16:24:12.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.297+0000 7fc1c27ff700 1 -- 172.21.15.62:0/516886115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc19fa30 con 0x7fc1bc072f10 2023-07-14T16:24:12.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.297+0000 7fc1b97fa700 1 -- 172.21.15.62:0/516886115 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac022e00 con 0x7fc1bc072f10 2023-07-14T16:24:12.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.297+0000 7fc1c27ff700 1 -- 172.21.15.62:0/516886115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc19fe50 con 0x7fc1bc072f10 2023-07-14T16:24:12.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.301+0000 7fc1b97fa700 1 -- 172.21.15.62:0/516886115 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc1ac00b9a0 con 0x7fc1bc072f10 2023-07-14T16:24:12.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.303+0000 7fc1d6218b80 1 Processor -- start 2023-07-14T16:24:12.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.303+0000 7fc1d6218b80 1 -- start start 2023-07-14T16:24:12.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e359228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 0x561e359236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e359240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e35827cb0 con 0x561e35923c70 2023-07-14T16:24:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3587aa40 con 0x561e35922e20 2023-07-14T16:24:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e35873ed0 con 0x561e359224d0 2023-07-14T16:24:12.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1b97fa700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1a40a1150 0x7fc1a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e359240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e359240e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44992/0 (socket says 172.21.15.62:44992) 2023-07-14T16:24:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1bb7fe700 1 -- 172.21.15.62:0/1969463421 learned_addr learned my addr 172.21.15.62:0/1969463421 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e359228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.304+0000 7fc1b97fa700 1 -- 172.21.15.62:0/516886115 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc1ac0cddc0 con 0x7fc1bc072f10 2023-07-14T16:24:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.305+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 0x561e359236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.305+0000 7fc1bb7fe700 1 -- 172.21.15.62:0/1969463421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 msgr2=0x561e359228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.305+0000 7fc1bb7fe700 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e359228e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.305+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1a40a1150 0x7fc1a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.305+0000 7fc1bb7fe700 1 -- 172.21.15.62:0/1969463421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 msgr2=0x561e359236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.305+0000 7fc1bb7fe700 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 0x561e359236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.305+0000 7fc1bb7fe700 1 -- 172.21.15.62:0/1969463421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e356a1de0 con 0x561e35923c70 2023-07-14T16:24:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.306+0000 7fc1bb7fe700 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e359240e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000bd70 tx=0x7fc1b00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507eeb4bb718ff7d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.306+0000 7fc1a27fc700 1 -- 172.21.15.62:0/1969463421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0010e10 con 0x561e35923c70 2023-07-14T16:24:12.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.306+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1a40a1150 0x7fc1a40a3600 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40079a0 tx=0x7fc1b4004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.306+0000 7fc1a27fc700 1 -- 172.21.15.62:0/1969463421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b0015070 con 0x561e35923c70 2023-07-14T16:24:12.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.306+0000 7fc1a27fc700 1 -- 172.21.15.62:0/1969463421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00116a0 con 0x561e35923c70 2023-07-14T16:24:12.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 -- 172.21.15.62:0/1969463421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 msgr2=0x561e359240e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e359240e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000bd70 tx=0x7fc1b00106b0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 -- 172.21.15.62:0/1969463421 shutdown_connections 2023-07-14T16:24:12.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e359240e0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e359228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/1969463421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 0x561e359236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 -- 172.21.15.62:0/1969463421 >> 172.21.15.62:0/1969463421 conn(0x561e35917da0 msgr2=0x561e3591d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 -- 172.21.15.62:0/1969463421 shutdown_connections 2023-07-14T16:24:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.307+0000 7fc1d6218b80 1 -- 172.21.15.62:0/1969463421 wait complete. 2023-07-14T16:24:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.308+0000 7fc1d6218b80 1 -- start start 2023-07-14T16:24:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.308+0000 7fc1d6218b80 1 Processor -- start 2023-07-14T16:24:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 0x561e3592c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e35933680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e35813ac0 con 0x561e35923c70 2023-07-14T16:24:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e358137c0 con 0x561e35922e20 2023-07-14T16:24:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e356f7920 con 0x561e359224d0 2023-07-14T16:24:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1d6218b80 10 client.0 _ll_register_callbacks cb 0x561e35811760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1c0d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e35933680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1bbfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1bbfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54800/0 (socket says 172.21.15.62:54800) 2023-07-14T16:24:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1bbfff700 1 -- 172.21.15.62:0/2571746670 learned_addr learned my addr 172.21.15.62:0/2571746670 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.309+0000 7fc1bb7fe700 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 0x561e3592c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc1c0d9c700 1 -- 172.21.15.62:0/2571746670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 msgr2=0x561e3592c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc1c0d9c700 1 -- 172.21.15.62:0/2571746670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 msgr2=0x561e3592c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 0x561e3592c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc1c0d9c700 1 -- 172.21.15.62:0/2571746670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b000b9a0 con 0x561e35923c70 2023-07-14T16:24:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e35933680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40175d0 tx=0x7fc1b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100af96b40ce6daf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc19b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc19b7fe700 1 -- 172.21.15.62:0/2571746670 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400e040 con 0x561e35923c70 2023-07-14T16:24:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2571746670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e357cd4f0 con 0x561e35923c70 2023-07-14T16:24:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.310+0000 7fc19b7fe700 1 -- 172.21.15.62:0/2571746670 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc1b401e040 con 0x561e35923c70 2023-07-14T16:24:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc19b7fe700 1 -- 172.21.15.62:0/2571746670 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1b4007e10 con 0x561e35923c70 2023-07-14T16:24:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc1d6218b80 10 client.17904 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2571746670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e3568ac60 con 0x561e35923c70 2023-07-14T16:24:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc19b7fe700 1 -- 172.21.15.62:0/2571746670 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b4031020 con 0x561e35923c70 2023-07-14T16:24:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc19b7fe700 1 -- 172.21.15.62:0/2571746670 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc1b4003d00 con 0x561e35923c70 2023-07-14T16:24:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc1d6218b80 10 client.17904 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc1d6218b80 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc1d6218b80 20 client.17904 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561e35937f80 0x561e3593a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc1d6218b80 4 client.17904 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.311+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2571746670 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e3586ceb0 con 0x561e35937f80 2023-07-14T16:24:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.312+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561e35937f80 0x561e3593a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.313+0000 7fc19b7fe700 10 client.17904 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.313+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561e35937f80 0x561e3593a390 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0039e9f5b649ac server_cookie=bae67223f308374f in_seq=0 out_seq=0 2023-07-14T16:24:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.313+0000 7fc19b7fe700 1 -- 172.21.15.62:0/2571746670 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561e3586ceb0 con 0x561e35937f80 2023-07-14T16:24:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.313+0000 7fc19b7fe700 10 client.17904 handle_command_reply: tid=0 2023-07-14T16:24:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 client.17904 shutdown 2023-07-14T16:24:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2571746670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 msgr2=0x561e35933680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e35933680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40175d0 tx=0x7fc1b4004ad0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2571746670 shutdown_connections 2023-07-14T16:24:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e35933680 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561e35937f80 0x561e3593a390 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2571746670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35922e20 0x561e3592c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc19b7fe700 0 client.17904 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2571746670 >> 172.21.15.62:0/2571746670 conn(0x561e35917da0 msgr2=0x561e3591c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2571746670 shutdown_connections 2023-07-14T16:24:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.314+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2571746670 wait complete. 2023-07-14T16:24:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.315+0000 7fc1d6218b80 20 client.17904 trim_cache size 0 max 16384 2023-07-14T16:24:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.315+0000 7f884d1ff700 1 -- 172.21.15.62:0/3401298877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 msgr2=0x7f88480fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.315+0000 7f884d1ff700 1 --2- 172.21.15.62:0/3401298877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 0x7f88480fff50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8838009990 tx=0x7f8838009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.315+0000 7f884d1ff700 1 -- 172.21.15.62:0/3401298877 shutdown_connections 2023-07-14T16:24:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.315+0000 7f884d1ff700 1 --2- 172.21.15.62:0/3401298877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88481016c0 0x7f8848107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.315+0000 7f884d1ff700 1 --2- 172.21.15.62:0/3401298877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 0x7f88480fff50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.315+0000 7f884d1ff700 1 --2- 172.21.15.62:0/3401298877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8848100d30 0x7f8848101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.315+0000 7f884d1ff700 1 -- 172.21.15.62:0/3401298877 >> 172.21.15.62:0/3401298877 conn(0x7f88480fb160 msgr2=0x7f88480fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.316+0000 7f884d1ff700 1 -- 172.21.15.62:0/3401298877 shutdown_connections 2023-07-14T16:24:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.316+0000 7f884d1ff700 1 -- 172.21.15.62:0/3401298877 wait complete. 2023-07-14T16:24:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.317+0000 7f884d1ff700 1 Processor -- start 2023-07-14T16:24:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.317+0000 7f884d1ff700 1 -- start start 2023-07-14T16:24:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.317+0000 7f884d1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 0x7f8848195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.317+0000 7f884d1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8848100d30 0x7f8848195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.317+0000 7f884d1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88481016c0 0x7f8848196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.317+0000 7f884d1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88481972b0 con 0x7f88481016c0 2023-07-14T16:24:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.317+0000 7f884d1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8848197420 con 0x7f88480ffb40 2023-07-14T16:24:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.317+0000 7f884d1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88481975e0 con 0x7f8848100d30 2023-07-14T16:24:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f8846d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 0x7f8848195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f884659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8848100d30 0x7f8848195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f8846d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 0x7f8848195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:58748/0 (socket says 172.21.15.62:58748) 2023-07-14T16:24:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f8846d9d700 1 -- 172.21.15.62:0/1176718896 learned_addr learned my addr 172.21.15.62:0/1176718896 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f884759e700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88481016c0 0x7f8848196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f884659c700 1 -- 172.21.15.62:0/1176718896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 msgr2=0x7f8848195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f884659c700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 0x7f8848195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f884659c700 1 -- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88481016c0 msgr2=0x7f8848196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f884659c700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88481016c0 0x7f8848196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.318+0000 7f884659c700 1 -- 172.21.15.62:0/1176718896 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88380095f0 con 0x7f8848100d30 2023-07-14T16:24:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.319+0000 7f884659c700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8848100d30 0x7f8848195dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e9f0 tx=0x7f883c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.319+0000 7f882ffff700 1 -- 172.21.15.62:0/1176718896 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00c9c0 con 0x7f8848100d30 2023-07-14T16:24:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.319+0000 7f884d1ff700 1 -- 172.21.15.62:0/1176718896 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88481978c0 con 0x7f8848100d30 2023-07-14T16:24:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.319+0000 7f882ffff700 1 -- 172.21.15.62:0/1176718896 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f883c010430 con 0x7f8848100d30 2023-07-14T16:24:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.319+0000 7f882ffff700 1 -- 172.21.15.62:0/1176718896 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c018680 con 0x7f8848100d30 2023-07-14T16:24:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.319+0000 7f884d1ff700 1 -- 172.21.15.62:0/1176718896 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884819e080 con 0x7f8848100d30 2023-07-14T16:24:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.323+0000 7f882ffff700 1 -- 172.21.15.62:0/1176718896 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f883c0105a0 con 0x7f8848100d30 2023-07-14T16:24:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.325+0000 7f882ffff700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88300a1150 0x7f88300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.325+0000 7f882ffff700 1 -- 172.21.15.62:0/1176718896 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f883c014070 con 0x7f8848100d30 2023-07-14T16:24:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.325+0000 7f8846d9d700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88300a1150 0x7f88300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.326+0000 7f8860c00b80 1 Processor -- start 2023-07-14T16:24:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.326+0000 7f8846d9d700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88300a1150 0x7f88300a3600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f883800b2e0 tx=0x7f8838004000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.326+0000 7f8860c00b80 1 -- start start 2023-07-14T16:24:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f8860c00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be0b8c50 con 0x55e8be1b8af0 2023-07-14T16:24:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f8860c00b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be10b9e0 con 0x55e8be00f6b0 2023-07-14T16:24:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f8860c00b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be104e70 con 0x55e8be1b7c50 2023-07-14T16:24:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f884759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f884759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1b8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45032/0 (socket says 172.21.15.62:45032) 2023-07-14T16:24:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f884759e700 1 -- 172.21.15.62:0/4208714157 learned_addr learned my addr 172.21.15.62:0/4208714157 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f884659c700 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.327+0000 7f8846d9d700 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f884759e700 1 -- 172.21.15.62:0/4208714157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 msgr2=0x55e8be1b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f884759e700 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f884759e700 1 -- 172.21.15.62:0/4208714157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 msgr2=0x55e8be1b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f884759e700 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f884759e700 1 -- 172.21.15.62:0/4208714157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bdf32d80 con 0x55e8be1b8af0 2023-07-14T16:24:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f884759e700 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1b8f90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8840011680 tx=0x7f8840009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26d0b582056b7c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f882d7fa700 1 -- 172.21.15.62:0/4208714157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884000e040 con 0x55e8be1b8af0 2023-07-14T16:24:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f882d7fa700 1 -- 172.21.15.62:0/4208714157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f884000f040 con 0x55e8be1b8af0 2023-07-14T16:24:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.328+0000 7f882d7fa700 1 -- 172.21.15.62:0/4208714157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840004730 con 0x55e8be1b8af0 2023-07-14T16:24:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.329+0000 7f8860c00b80 1 -- 172.21.15.62:0/4208714157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 msgr2=0x55e8be1b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.329+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1b8f90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8840011680 tx=0x7f8840009920 comp rx=0 tx=0).stop 2023-07-14T16:24:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.329+0000 7f8860c00b80 1 -- 172.21.15.62:0/4208714157 shutdown_connections 2023-07-14T16:24:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.329+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1b8f90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.329+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.329+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4208714157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.329+0000 7f8860c00b80 1 -- 172.21.15.62:0/4208714157 >> 172.21.15.62:0/4208714157 conn(0x55e8be1a8be0 msgr2=0x55e8be1adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.330+0000 7f8860c00b80 1 -- 172.21.15.62:0/4208714157 shutdown_connections 2023-07-14T16:24:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.330+0000 7f8860c00b80 1 -- 172.21.15.62:0/4208714157 wait complete. 2023-07-14T16:24:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.330+0000 7f8860c00b80 1 -- start start 2023-07-14T16:24:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.330+0000 7f8860c00b80 1 Processor -- start 2023-07-14T16:24:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f8860c00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be0a4a60 con 0x55e8be00f6b0 2023-07-14T16:24:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f8860c00b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be0a4760 con 0x55e8be1b8af0 2023-07-14T16:24:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f8860c00b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bdf888c0 con 0x55e8be1b7c50 2023-07-14T16:24:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f8860c00b80 10 client.0 _ll_register_callbacks cb 0x55e8be0a2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f8846d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.331+0000 7f884659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f8846d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45040/0 (socket says 172.21.15.62:45040) 2023-07-14T16:24:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f8846d9d700 1 -- 172.21.15.62:0/3349041110 learned_addr learned my addr 172.21.15.62:0/3349041110 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f884759e700 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f884659c700 1 -- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 msgr2=0x55e8be1c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f884659c700 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f884659c700 1 -- 172.21.15.62:0/3349041110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be00f6b0 msgr2=0x55e8be1c0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f884659c700 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f884659c700 1 -- 172.21.15.62:0/3349041110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8840011330 con 0x55e8be1b7c50 2023-07-14T16:24:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f8846d9d700 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f884759e700 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f884659c700 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f883c0ca590 tx=0x7f883c09a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf3759c13929848 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.332+0000 7f882aff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:24:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.333+0000 7f882aff5700 1 -- 172.21.15.62:0/3349041110 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0a2010 con 0x55e8be1b7c50 2023-07-14T16:24:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.333+0000 7f882aff5700 1 -- 172.21.15.62:0/3349041110 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f883c010880 con 0x55e8be1b7c50 2023-07-14T16:24:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.333+0000 7f8860c00b80 1 -- 172.21.15.62:0/3349041110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8be05e490 con 0x55e8be1b7c50 2023-07-14T16:24:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.333+0000 7f882aff5700 1 -- 172.21.15.62:0/3349041110 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f883c099e50 con 0x55e8be1b7c50 2023-07-14T16:24:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.333+0000 7f8860c00b80 10 client.27454 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.333+0000 7f8860c00b80 1 -- 172.21.15.62:0/3349041110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e8bdf1bc00 con 0x55e8be1b7c50 2023-07-14T16:24:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.333+0000 7f882aff5700 1 -- 172.21.15.62:0/3349041110 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f883c0c8020 con 0x55e8be1b7c50 2023-07-14T16:24:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.334+0000 7f882aff5700 1 -- 172.21.15.62:0/3349041110 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f883c09c070 con 0x55e8be1b7c50 2023-07-14T16:24:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.334+0000 7f8860c00b80 10 client.27454 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.334+0000 7f8860c00b80 10 client.27454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.334+0000 7f8860c00b80 20 client.27454 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.334+0000 7f8860c00b80 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8be1b62d0 0x55e8be1b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.334+0000 7f8860c00b80 4 client.27454 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.334+0000 7f8860c00b80 1 -- 172.21.15.62:0/3349041110 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8be0fde50 con 0x55e8be1b62d0 2023-07-14T16:24:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.334+0000 7f8846d9d700 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8be1b62d0 0x55e8be1b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.335+0000 7fc1d6218b80 1 Processor -- start 2023-07-14T16:24:12.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.335+0000 7fc1d6218b80 1 -- start start 2023-07-14T16:24:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.335+0000 7f882aff5700 10 client.27454 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.335+0000 7f8846d9d700 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8be1b62d0 0x55e8be1b66d0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e4b9940b14752fd server_cookie=e9daab597ee81662 in_seq=0 out_seq=0 2023-07-14T16:24:12.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.335+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e359224d0 0x561e359228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.335+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e35922e20 0x561e35933610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e3592ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e35929d50 con 0x561e35923c70 2023-07-14T16:24:12.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e35924400 con 0x561e359224d0 2023-07-14T16:24:12.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3593ecf0 con 0x561e35922e20 2023-07-14T16:24:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.336+0000 7f882aff5700 1 -- 172.21.15.62:0/3349041110 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e8be0fde50 con 0x55e8be1b62d0 2023-07-14T16:24:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.336+0000 7f882aff5700 10 client.27454 handle_command_reply: tid=0 2023-07-14T16:24:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.336+0000 7f8860c00b80 1 client.27454 shutdown 2023-07-14T16:24:12.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e359224d0 0x561e359228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e359224d0 0x561e359228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:58784/0 (socket says 172.21.15.62:58784) 2023-07-14T16:24:12.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1c0d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e3592ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bbfff700 1 -- 172.21.15.62:0/2361520431 learned_addr learned my addr 172.21.15.62:0/2361520431 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bb7fe700 1 --2- 172.21.15.62:0/2361520431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e35922e20 0x561e35933610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bbfff700 1 -- 172.21.15.62:0/2361520431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e35922e20 msgr2=0x561e35933610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/2361520431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e35922e20 0x561e35933610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bbfff700 1 -- 172.21.15.62:0/2361520431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 msgr2=0x561e3592ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/2361520431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e3592ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.336+0000 7fc1bbfff700 1 -- 172.21.15.62:0/2361520431 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e356c8250 con 0x561e359224d0 2023-07-14T16:24:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.336+0000 7f8860c00b80 1 -- 172.21.15.62:0/3349041110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 msgr2=0x55e8be1c11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.336+0000 7f8860c00b80 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f883c0ca590 tx=0x7f883c09a390 comp rx=0 tx=0).stop 2023-07-14T16:24:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 1 -- 172.21.15.62:0/3349041110 shutdown_connections 2023-07-14T16:24:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8be1b62d0 0x55e8be1b66d0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 1 --2- 172.21.15.62:0/3349041110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 1 -- 172.21.15.62:0/3349041110 >> 172.21.15.62:0/3349041110 conn(0x55e8be1a8be0 msgr2=0x55e8be1ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f882aff5700 0 client.27454 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:12.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.337+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/2361520431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e359224d0 0x561e359228a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000c00 tx=0x7fc1ac0a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e400bf284a1f2df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 1 -- 172.21.15.62:0/3349041110 shutdown_connections 2023-07-14T16:24:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 1 -- 172.21.15.62:0/3349041110 wait complete. 2023-07-14T16:24:12.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.337+0000 7fc1a2ffd700 1 -- 172.21.15.62:0/2361520431 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0a10b0 con 0x561e359224d0 2023-07-14T16:24:12.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.337+0000 7fc1a2ffd700 1 -- 172.21.15.62:0/2361520431 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1ac09b210 con 0x561e359224d0 2023-07-14T16:24:12.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.337+0000 7fc1a2ffd700 1 -- 172.21.15.62:0/2361520431 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0cb980 con 0x561e359224d0 2023-07-14T16:24:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.337+0000 7f8860c00b80 20 client.27454 trim_cache size 0 max 16384 2023-07-14T16:24:12.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2361520431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e359224d0 msgr2=0x561e359228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2361520431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e359224d0 0x561e359228a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000c00 tx=0x7fc1ac0a1040 comp rx=0 tx=0).stop 2023-07-14T16:24:12.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2361520431 shutdown_connections 2023-07-14T16:24:12.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2361520431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35923c70 0x561e3592ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2361520431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e359224d0 0x561e359228a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/2361520431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e35922e20 0x561e35933610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2361520431 >> 172.21.15.62:0/2361520431 conn(0x561e35917da0 msgr2=0x561e3591d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2361520431 shutdown_connections 2023-07-14T16:24:12.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.338+0000 7fc1d6218b80 1 -- 172.21.15.62:0/2361520431 wait complete. 2023-07-14T16:24:12.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.339+0000 7fc1d6218b80 1 -- start start 2023-07-14T16:24:12.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.339+0000 7fc1d6218b80 1 Processor -- start 2023-07-14T16:24:12.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.339+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.339+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35922e20 0x561e3592fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.339+0000 7fc1d6218b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35923c70 0x561e35932b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.339+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e35933050 con 0x561e35922e20 2023-07-14T16:24:12.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.339+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e359331c0 con 0x561e35923c70 2023-07-14T16:24:12.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.339+0000 7fc1d6218b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e35933380 con 0x561e359224d0 2023-07-14T16:24:12.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1d6218b80 10 client.0 _ll_register_callbacks cb 0x561e35811760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:12.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35922e20 0x561e3592fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35922e20 0x561e3592fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45058/0 (socket says 172.21.15.62:45058) 2023-07-14T16:24:12.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1bb7fe700 1 -- 172.21.15.62:0/3869398112 learned_addr learned my addr 172.21.15.62:0/3869398112 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35923c70 0x561e35932b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1c0d9c700 1 -- 172.21.15.62:0/3869398112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 msgr2=0x561e3592f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1c0d9c700 1 -- 172.21.15.62:0/3869398112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35922e20 msgr2=0x561e3592fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35922e20 0x561e3592fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1c0d9c700 1 -- 172.21.15.62:0/3869398112 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc19fe50 con 0x561e35923c70 2023-07-14T16:24:12.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.340+0000 7fc1bb7fe700 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35922e20 0x561e3592fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:12.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.341+0000 7fc1c0d9c700 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35923c70 0x561e35932b10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4004170 tx=0x7fc1b4003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0b409cfd51b4dc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.341+0000 7fc19b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:24:12.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.341+0000 7fc19b7fe700 1 -- 172.21.15.62:0/3869398112 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400e070 con 0x561e35923c70 2023-07-14T16:24:12.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.341+0000 7fc1d6218b80 1 -- 172.21.15.62:0/3869398112 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1b0015070 con 0x561e35923c70 2023-07-14T16:24:12.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.341+0000 7fc19b7fe700 1 -- 172.21.15.62:0/3869398112 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc1b4032080 con 0x561e35923c70 2023-07-14T16:24:12.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.341+0000 7fc19b7fe700 1 -- 172.21.15.62:0/3869398112 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1b4004bb0 con 0x561e35923c70 2023-07-14T16:24:12.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.341+0000 7fc1d6218b80 10 client.27560 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:12.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.341+0000 7fc1d6218b80 1 -- 172.21.15.62:0/3869398112 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e356a02b0 con 0x561e35923c70 2023-07-14T16:24:12.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc19b7fe700 1 -- 172.21.15.62:0/3869398112 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b0015070 con 0x561e35923c70 2023-07-14T16:24:12.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc19b7fe700 1 -- 172.21.15.62:0/3869398112 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc1b4012070 con 0x561e35923c70 2023-07-14T16:24:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc1d6218b80 10 client.27560 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc1d6218b80 10 client.27560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc1d6218b80 20 client.27560 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561e3598cdf0 0x561e3598f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc1d6218b80 4 client.27560 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:24:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc1d6218b80 1 -- 172.21.15.62:0/3869398112 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e3582f050 con 0x561e3598cdf0 2023-07-14T16:24:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.342+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561e3598cdf0 0x561e3598f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc1bbfff700 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561e3598cdf0 0x561e3598f200 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46959ab7f56c3030 server_cookie=371baa79963c2f72 in_seq=0 out_seq=0 2023-07-14T16:24:12.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc19b7fe700 10 client.27560 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:12.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc19b7fe700 1 -- 172.21.15.62:0/3869398112 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e3582f050 con 0x561e3598cdf0 2023-07-14T16:24:12.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc19b7fe700 10 client.27560 handle_command_reply: tid=0 2023-07-14T16:24:12.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc1d6218b80 1 client.27560 shutdown 2023-07-14T16:24:12.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc1d6218b80 1 -- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35923c70 msgr2=0x561e35932b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35923c70 0x561e35932b10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4004170 tx=0x7fc1b4003f80 comp rx=0 tx=0).stop 2023-07-14T16:24:12.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc1d6218b80 1 -- 172.21.15.62:0/3869398112 shutdown_connections 2023-07-14T16:24:12.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561e3598cdf0 0x561e3598f200 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e35923c70 0x561e35932b10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561e359224d0 0x561e3592f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.343+0000 7fc19b7fe700 0 client.27560 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:12.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.344+0000 7fc1d6218b80 1 --2- 172.21.15.62:0/3869398112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e35922e20 0x561e3592fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.344+0000 7fc1d6218b80 1 -- 172.21.15.62:0/3869398112 >> 172.21.15.62:0/3869398112 conn(0x561e35917da0 msgr2=0x561e35933eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.344+0000 7fc1d6218b80 1 -- 172.21.15.62:0/3869398112 shutdown_connections 2023-07-14T16:24:12.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.344+0000 7fc1d6218b80 1 -- 172.21.15.62:0/3869398112 wait complete. 2023-07-14T16:24:12.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.344+0000 7fc1d6218b80 20 client.27560 trim_cache size 0 max 16384 2023-07-14T16:24:12.337 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:24:12.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 -- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1a40a1150 msgr2=0x7fc1a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1a40a1150 0x7fc1a40a3600 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40079a0 tx=0x7fc1b4004b40 comp rx=0 tx=0).stop 2023-07-14T16:24:12.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 -- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc072f10 msgr2=0x7fc1bc197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc072f10 0x7fc1bc197960 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0098e0 tx=0x7fc1ac004730 comp rx=0 tx=0).stop 2023-07-14T16:24:12.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 -- 172.21.15.62:0/516886115 shutdown_connections 2023-07-14T16:24:12.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1a40a1150 0x7fc1a40a3600 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1bc1079a0 0x7fc1bc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1bc072f10 0x7fc1bc197960 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 --2- 172.21.15.62:0/516886115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1bc0744f0 0x7fc1bc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 -- 172.21.15.62:0/516886115 >> 172.21.15.62:0/516886115 conn(0x7fc1bc0fb130 msgr2=0x7fc1bc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 -- 172.21.15.62:0/516886115 shutdown_connections 2023-07-14T16:24:12.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:12.345+0000 7fc1c27ff700 1 -- 172.21.15.62:0/516886115 wait complete. 2023-07-14T16:24:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.358+0000 7f8860c00b80 1 Processor -- start 2023-07-14T16:24:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.358+0000 7f8860c00b80 1 -- start start 2023-07-14T16:24:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.358+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.358+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.358+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.358+0000 7f8860c00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be1c07f0 con 0x55e8be1b7c50 2023-07-14T16:24:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.358+0000 7f8860c00b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be1ae7a0 con 0x55e8be1b8af0 2023-07-14T16:24:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.358+0000 7f8860c00b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be1aad00 con 0x55e8be00f6b0 2023-07-14T16:24:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f8846d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f884659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f884759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f8846d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54876/0 (socket says 172.21.15.62:54876) 2023-07-14T16:24:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f8846d9d700 1 -- 172.21.15.62:0/1453254718 learned_addr learned my addr 172.21.15.62:0/1453254718 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f8846d9d700 1 -- 172.21.15.62:0/1453254718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 msgr2=0x55e8be1c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f8846d9d700 1 --2- 172.21.15.62:0/1453254718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f8846d9d700 1 -- 172.21.15.62:0/1453254718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b7c50 msgr2=0x55e8be1c11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f8846d9d700 1 --2- 172.21.15.62:0/1453254718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.359+0000 7f8846d9d700 1 -- 172.21.15.62:0/1453254718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bdf591f0 con 0x55e8be00f6b0 2023-07-14T16:24:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.360+0000 7f8846d9d700 1 --2- 172.21.15.62:0/1453254718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8838004360 tx=0x7f8838020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c1cf1475011a50 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.360+0000 7f882dffb700 1 -- 172.21.15.62:0/1453254718 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838003680 con 0x55e8be00f6b0 2023-07-14T16:24:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.360+0000 7f882dffb700 1 -- 172.21.15.62:0/1453254718 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f883801e040 con 0x55e8be00f6b0 2023-07-14T16:24:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.360+0000 7f882dffb700 1 -- 172.21.15.62:0/1453254718 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883800bbb0 con 0x55e8be00f6b0 2023-07-14T16:24:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 -- 172.21.15.62:0/1453254718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be00f6b0 msgr2=0x55e8be1c0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 --2- 172.21.15.62:0/1453254718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8838004360 tx=0x7f8838020000 comp rx=0 tx=0).stop 2023-07-14T16:24:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 -- 172.21.15.62:0/1453254718 shutdown_connections 2023-07-14T16:24:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 --2- 172.21.15.62:0/1453254718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be1b8af0 0x55e8be1c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 --2- 172.21.15.62:0/1453254718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be00f6b0 0x55e8be1c0c80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 --2- 172.21.15.62:0/1453254718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b7c50 0x55e8be1c11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 -- 172.21.15.62:0/1453254718 >> 172.21.15.62:0/1453254718 conn(0x55e8be1a8be0 msgr2=0x55e8be1adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 -- 172.21.15.62:0/1453254718 shutdown_connections 2023-07-14T16:24:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 -- 172.21.15.62:0/1453254718 wait complete. 2023-07-14T16:24:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.361+0000 7f8860c00b80 1 -- start start 2023-07-14T16:24:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.362+0000 7f8860c00b80 1 Processor -- start 2023-07-14T16:24:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.362+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.362+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.362+0000 7f8860c00b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.362+0000 7f8860c00b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be1c7450 con 0x55e8be1b8af0 2023-07-14T16:24:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.362+0000 7f8860c00b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be1c75c0 con 0x55e8be00f6b0 2023-07-14T16:24:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.362+0000 7f8860c00b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8be1c7780 con 0x55e8be1b7c50 2023-07-14T16:24:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f8860c00b80 10 client.0 _ll_register_callbacks cb 0x7f883c096280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54884/0 (socket says 172.21.15.62:54884) 2023-07-14T16:24:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884659c700 1 -- 172.21.15.62:0/4063490509 learned_addr learned my addr 172.21.15.62:0/4063490509 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f8846d9d700 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884659c700 1 -- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 msgr2=0x55e8be1c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884659c700 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884659c700 1 -- 172.21.15.62:0/4063490509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 msgr2=0x55e8be1c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884659c700 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884659c700 1 -- 172.21.15.62:0/4063490509 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883802c040 con 0x55e8be1b7c50 2023-07-14T16:24:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.363+0000 7f884759e700 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f8846d9d700 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f884659c700 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c3ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f883c0ca8b0 tx=0x7f883c099620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b07cbab6fbba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f882aff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:24:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f882aff5700 1 -- 172.21.15.62:0/4063490509 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c099990 con 0x55e8be1b7c50 2023-07-14T16:24:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f882aff5700 1 -- 172.21.15.62:0/4063490509 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f883c0c9080 con 0x55e8be1b7c50 2023-07-14T16:24:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f8860c00b80 1 -- 172.21.15.62:0/4063490509 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f884000f040 con 0x55e8be1b7c50 2023-07-14T16:24:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f882aff5700 1 -- 172.21.15.62:0/4063490509 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f883c010880 con 0x55e8be1b7c50 2023-07-14T16:24:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f8860c00b80 10 client.27466 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.364+0000 7f8860c00b80 1 -- 172.21.15.62:0/4063490509 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e8bdf31250 con 0x55e8be1b7c50 2023-07-14T16:24:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.365+0000 7f882aff5700 1 -- 172.21.15.62:0/4063490509 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f884000f040 con 0x55e8be1b7c50 2023-07-14T16:24:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.365+0000 7f882aff5700 1 -- 172.21.15.62:0/4063490509 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f883c09c040 con 0x55e8be1b7c50 2023-07-14T16:24:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.365+0000 7f8860c00b80 10 client.27466 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.365+0000 7f8860c00b80 10 client.27466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.365+0000 7f8860c00b80 20 client.27466 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.365+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8be2257e0 0x55e8be227bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.365+0000 7f8860c00b80 4 client.27466 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:24:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.365+0000 7f8860c00b80 1 -- 172.21.15.62:0/4063490509 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e8be0bfff0 con 0x55e8be2257e0 2023-07-14T16:24:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.366+0000 7f8846d9d700 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8be2257e0 0x55e8be227bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.366+0000 7f8846d9d700 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8be2257e0 0x55e8be227bf0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758cc89105583ca server_cookie=443323a8e208d4c2 in_seq=0 out_seq=0 2023-07-14T16:24:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.366+0000 7f882aff5700 10 client.27466 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.375+0000 7f882aff5700 1 -- 172.21.15.62:0/4063490509 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e8be0bfff0 con 0x55e8be2257e0 2023-07-14T16:24:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.375+0000 7f882aff5700 10 client.27466 handle_command_reply: tid=0 2023-07-14T16:24:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.375+0000 7f8860c00b80 1 client.27466 shutdown 2023-07-14T16:24:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 -- 172.21.15.62:0/4063490509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 msgr2=0x55e8be1c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c3ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f883c0ca8b0 tx=0x7f883c099620 comp rx=0 tx=0).stop 2023-07-14T16:24:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 -- 172.21.15.62:0/4063490509 shutdown_connections 2023-07-14T16:24:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8be1b8af0 0x55e8be1c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8be2257e0 0x55e8be227bf0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8be00f6b0 0x55e8be1c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 --2- 172.21.15.62:0/4063490509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8be1b7c50 0x55e8be1c3ef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f882aff5700 0 client.27466 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 -- 172.21.15.62:0/4063490509 >> 172.21.15.62:0/4063490509 conn(0x55e8be1a8be0 msgr2=0x55e8be1bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 -- 172.21.15.62:0/4063490509 shutdown_connections 2023-07-14T16:24:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 1 -- 172.21.15.62:0/4063490509 wait complete. 2023-07-14T16:24:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.376+0000 7f8860c00b80 20 client.27466 trim_cache size 0 max 16384 2023-07-14T16:24:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:24:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:24:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:24:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:24:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 -- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88300a1150 msgr2=0x7f88300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88300a1150 0x7f88300a3600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f883800b2e0 tx=0x7f8838004000 comp rx=0 tx=0).stop 2023-07-14T16:24:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 -- 172.21.15.62:0/1176718896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8848100d30 msgr2=0x7f8848195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8848100d30 0x7f8848195dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e9f0 tx=0x7f883c00ed00 comp rx=0 tx=0).stop 2023-07-14T16:24:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 -- 172.21.15.62:0/1176718896 shutdown_connections 2023-07-14T16:24:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88481016c0 0x7f8848196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88300a1150 0x7f88300a3600 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88480ffb40 0x7f8848195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 --2- 172.21.15.62:0/1176718896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8848100d30 0x7f8848195dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 -- 172.21.15.62:0/1176718896 >> 172.21.15.62:0/1176718896 conn(0x7f88480fb160 msgr2=0x7f88481042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 -- 172.21.15.62:0/1176718896 shutdown_connections 2023-07-14T16:24:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:12.377+0000 7f884d1ff700 1 -- 172.21.15.62:0/1176718896 wait complete. 2023-07-14T16:24:13.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:12 smithi164 ceph-mon[97427]: pgmap v614: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:24:13.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:12 smithi062 ceph-mon[95543]: pgmap v614: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:24:13.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:12 smithi081 ceph-mon[127795]: pgmap v614: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:24:13.351 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:24:13.352 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:24:13.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.611+0000 7f91bf1ff700 1 -- 172.21.15.62:0/3624960896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b8100560 msgr2=0x7f91b8102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.611+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/3624960896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b8100560 0x7f91b8102940 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009990 tx=0x7f91ac009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.612+0000 7f91bf1ff700 1 -- 172.21.15.62:0/3624960896 shutdown_connections 2023-07-14T16:24:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.612+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/3624960896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91b81057a0 0x7f91b8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.612+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/3624960896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b8100560 0x7f91b8102940 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.612+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/3624960896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b8102e80 0x7f91b8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.612+0000 7f91bf1ff700 1 -- 172.21.15.62:0/3624960896 >> 172.21.15.62:0/3624960896 conn(0x7f91b80fa000 msgr2=0x7f91b80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.612+0000 7f91bf1ff700 1 -- 172.21.15.62:0/3624960896 shutdown_connections 2023-07-14T16:24:13.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.612+0000 7f91bf1ff700 1 -- 172.21.15.62:0/3624960896 wait complete. 2023-07-14T16:24:13.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.613+0000 7f91bf1ff700 1 Processor -- start 2023-07-14T16:24:13.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.614+0000 7f91bf1ff700 1 -- start start 2023-07-14T16:24:13.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.614+0000 7f91bf1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b8100560 0x7f91b8197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.614+0000 7f91bf1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91b8102e80 0x7f91b8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.614+0000 7f91bf1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b81057a0 0x7f91b819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.614+0000 7f91bf1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819f4a0 con 0x7f91b81057a0 2023-07-14T16:24:13.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.614+0000 7f91bf1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819f5e0 con 0x7f91b8102e80 2023-07-14T16:24:13.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.614+0000 7f91bf1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819f770 con 0x7f91b8100560 2023-07-14T16:24:13.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bd79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b81057a0 0x7f91b819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bcf9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b8100560 0x7f91b8197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91b8102e80 0x7f91b8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bd79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b81057a0 0x7f91b819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57408/0 (socket says 172.21.15.62:57408) 2023-07-14T16:24:13.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bd79c700 1 -- 172.21.15.62:0/2599363846 learned_addr learned my addr 172.21.15.62:0/2599363846 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bd79c700 1 -- 172.21.15.62:0/2599363846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b8100560 msgr2=0x7f91b8197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bd79c700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b8100560 0x7f91b8197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bd79c700 1 -- 172.21.15.62:0/2599363846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91b8102e80 msgr2=0x7f91b8197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bd79c700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91b8102e80 0x7f91b8197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91bd79c700 1 -- 172.21.15.62:0/2599363846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0095f0 con 0x7f91b81057a0 2023-07-14T16:24:13.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.615+0000 7f91b7fff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91b8102e80 0x7f91b8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:13.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.616+0000 7f91bd79c700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b81057a0 0x7f91b819eed0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007ee0 tx=0x7f91b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.616+0000 7f91b5ffb700 1 -- 172.21.15.62:0/2599363846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0017070 con 0x7f91b81057a0 2023-07-14T16:24:13.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.616+0000 7f91b5ffb700 1 -- 172.21.15.62:0/2599363846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91b000f040 con 0x7f91b81057a0 2023-07-14T16:24:13.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.616+0000 7f91bf1ff700 1 -- 172.21.15.62:0/2599363846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b819fa50 con 0x7f91b81057a0 2023-07-14T16:24:13.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.616+0000 7f91b5ffb700 1 -- 172.21.15.62:0/2599363846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0005570 con 0x7f91b81057a0 2023-07-14T16:24:13.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.616+0000 7f91bf1ff700 1 -- 172.21.15.62:0/2599363846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b819ff10 con 0x7f91b81057a0 2023-07-14T16:24:13.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.621+0000 7f91d2b87b80 1 Processor -- start 2023-07-14T16:24:13.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.621+0000 7f91b5ffb700 1 -- 172.21.15.62:0/2599363846 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f91b000db20 con 0x7f91b81057a0 2023-07-14T16:24:13.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.622+0000 7f91d2b87b80 1 -- start start 2023-07-14T16:24:13.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.622+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354db550 0x55fd354db960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.622+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.622+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dccf0 0x55fd354dd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.622+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd353e0d30 con 0x55fd354dccf0 2023-07-14T16:24:13.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.622+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd35433ac0 con 0x55fd354dbea0 2023-07-14T16:24:13.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.622+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3542cf50 con 0x55fd354db550 2023-07-14T16:24:13.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.623+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354db550 0x55fd354db960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.623+0000 7f91bcf9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.623+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354db550 0x55fd354db960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37868/0 (socket says 172.21.15.62:37868) 2023-07-14T16:24:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.623+0000 7f91b7fff700 1 -- 172.21.15.62:0/261926402 learned_addr learned my addr 172.21.15.62:0/261926402 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.623+0000 7f91bcf9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354dc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51884/0 (socket says 172.21.15.62:51884) 2023-07-14T16:24:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.623+0000 7f91b7fff700 1 --2- 172.21.15.62:0/261926402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dccf0 0x55fd354dd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.624+0000 7f91bcf9b700 1 -- 172.21.15.62:0/261926402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354db550 msgr2=0x55fd354db960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.624+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/261926402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354db550 0x55fd354db960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.624+0000 7f91bcf9b700 1 -- 172.21.15.62:0/261926402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dccf0 msgr2=0x55fd354dd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.624+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/261926402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dccf0 0x55fd354dd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.624+0000 7f91bcf9b700 1 -- 172.21.15.62:0/261926402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd3525ae60 con 0x55fd354dbea0 2023-07-14T16:24:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.624+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/261926402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354dc760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f91ac000c00 tx=0x7f91ac009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d936dbbbdc6b566 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.625+0000 7f91a2ffd700 1 -- 172.21.15.62:0/261926402 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac01d070 con 0x55fd354dbea0 2023-07-14T16:24:13.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.625+0000 7f91a2ffd700 1 -- 172.21.15.62:0/261926402 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91ac01e040 con 0x55fd354dbea0 2023-07-14T16:24:13.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.625+0000 7f91a2ffd700 1 -- 172.21.15.62:0/261926402 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac004500 con 0x55fd354dbea0 2023-07-14T16:24:13.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.625+0000 7f91b5ffb700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f91a80a1150 0x7f91a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.625+0000 7f91b5ffb700 1 -- 172.21.15.62:0/2599363846 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f91b00cbfb0 con 0x7f91b81057a0 2023-07-14T16:24:13.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.625+0000 7f91b7fff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f91a80a1150 0x7f91a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 -- 172.21.15.62:0/261926402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 msgr2=0x55fd354dc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/261926402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354dc760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f91ac000c00 tx=0x7f91ac009f90 comp rx=0 tx=0).stop 2023-07-14T16:24:13.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 -- 172.21.15.62:0/261926402 shutdown_connections 2023-07-14T16:24:13.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/261926402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dccf0 0x55fd354dd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/261926402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354db550 0x55fd354db960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/261926402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354dc760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 -- 172.21.15.62:0/261926402 >> 172.21.15.62:0/261926402 conn(0x55fd354d0e20 msgr2=0x55fd354d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 -- 172.21.15.62:0/261926402 shutdown_connections 2023-07-14T16:24:13.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91b7fff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f91a80a1150 0x7f91a80a3600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f91a400f5e0 tx=0x7f91a400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.626+0000 7f91d2b87b80 1 -- 172.21.15.62:0/261926402 wait complete. 2023-07-14T16:24:13.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.627+0000 7f91d2b87b80 1 -- start start 2023-07-14T16:24:13.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.628+0000 7f91d2b87b80 1 Processor -- start 2023-07-14T16:24:13.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.628+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354db550 0x55fd354e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.628+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dbea0 0x55fd354e5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.628+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dccf0 0x55fd354ec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.628+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd353ccb40 con 0x55fd354dbea0 2023-07-14T16:24:13.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.628+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd353cc840 con 0x55fd354db550 2023-07-14T16:24:13.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.628+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd352b09a0 con 0x55fd354dccf0 2023-07-14T16:24:13.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.628+0000 7f91d2b87b80 10 client.0 _ll_register_callbacks cb 0x55fd353ca7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:13.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bcf9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354db550 0x55fd354e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bcf9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354db550 0x55fd354e5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51896/0 (socket says 172.21.15.62:51896) 2023-07-14T16:24:13.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bcf9b700 1 -- 172.21.15.62:0/1854105496 learned_addr learned my addr 172.21.15.62:0/1854105496 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91b7fff700 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dbea0 0x55fd354e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bd79c700 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dccf0 0x55fd354ec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bcf9b700 1 -- 172.21.15.62:0/1854105496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dccf0 msgr2=0x55fd354ec6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dccf0 0x55fd354ec6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bcf9b700 1 -- 172.21.15.62:0/1854105496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dbea0 msgr2=0x55fd354e5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dbea0 0x55fd354e5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.629+0000 7f91bcf9b700 1 -- 172.21.15.62:0/1854105496 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b81087f0 con 0x55fd354db550 2023-07-14T16:24:13.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.630+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354db550 0x55fd354e5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00beb0 tx=0x7f91ac0092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a97ef610f6a8407 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.630+0000 7f9197fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:24:13.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.630+0000 7f9197fff700 1 -- 172.21.15.62:0/1854105496 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac01d070 con 0x55fd354db550 2023-07-14T16:24:13.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.630+0000 7f9197fff700 1 -- 172.21.15.62:0/1854105496 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f91ac003c30 con 0x55fd354db550 2023-07-14T16:24:13.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.630+0000 7f91d2b87b80 1 -- 172.21.15.62:0/1854105496 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd35386570 con 0x55fd354db550 2023-07-14T16:24:13.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.631+0000 7f9197fff700 1 -- 172.21.15.62:0/1854105496 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91ac003df0 con 0x55fd354db550 2023-07-14T16:24:13.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.631+0000 7f91d2b87b80 10 client.27578 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:13.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.631+0000 7f91d2b87b80 1 -- 172.21.15.62:0/1854105496 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fd35243ce0 con 0x55fd354db550 2023-07-14T16:24:13.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.632+0000 7f9197fff700 1 -- 172.21.15.62:0/1854105496 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fd35386570 con 0x55fd354db550 2023-07-14T16:24:13.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.632+0000 7f9197fff700 1 -- 172.21.15.62:0/1854105496 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f91ac00fbe0 con 0x55fd354db550 2023-07-14T16:24:13.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.632+0000 7f91d2b87b80 10 client.27578 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:13.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.632+0000 7f91d2b87b80 10 client.27578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:13.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.632+0000 7f91d2b87b80 20 client.27578 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:13.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.632+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fd354f0a50 0x55fd354f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.632+0000 7f91d2b87b80 4 client.27578 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:13.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.632+0000 7f91d2b87b80 1 -- 172.21.15.62:0/1854105496 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd35425f30 con 0x55fd354f0a50 2023-07-14T16:24:13.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.633+0000 7f91b7fff700 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fd354f0a50 0x55fd354f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.634+0000 7f91b7fff700 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fd354f0a50 0x55fd354f2e60 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a7d0c9d65a22b8 server_cookie=db9817b64e1efee in_seq=0 out_seq=0 2023-07-14T16:24:13.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.634+0000 7f9197fff700 10 client.27578 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:13.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.634+0000 7f9197fff700 1 -- 172.21.15.62:0/1854105496 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd35425f30 con 0x55fd354f0a50 2023-07-14T16:24:13.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.634+0000 7f9197fff700 10 client.27578 handle_command_reply: tid=0 2023-07-14T16:24:13.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 client.27578 shutdown 2023-07-14T16:24:13.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 -- 172.21.15.62:0/1854105496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354db550 msgr2=0x55fd354e5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354db550 0x55fd354e5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00beb0 tx=0x7f91ac0092f0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 -- 172.21.15.62:0/1854105496 shutdown_connections 2023-07-14T16:24:13.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dccf0 0x55fd354ec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fd354f0a50 0x55fd354f2e60 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354db550 0x55fd354e5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/1854105496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354dbea0 0x55fd354e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f9197fff700 0 client.27578 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:13.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 -- 172.21.15.62:0/1854105496 >> 172.21.15.62:0/1854105496 conn(0x55fd354d0e20 msgr2=0x55fd354d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 -- 172.21.15.62:0/1854105496 shutdown_connections 2023-07-14T16:24:13.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 1 -- 172.21.15.62:0/1854105496 wait complete. 2023-07-14T16:24:13.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.635+0000 7f91d2b87b80 20 client.27578 trim_cache size 0 max 16384 2023-07-14T16:24:13.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.653+0000 7f91d2b87b80 1 Processor -- start 2023-07-14T16:24:13.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.653+0000 7f91d2b87b80 1 -- start start 2023-07-14T16:24:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354d0e20 0x55fd354dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dbea0 0x55fd354ec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd35386570 con 0x55fd354db550 2023-07-14T16:24:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd354d7540 con 0x55fd354d0e20 2023-07-14T16:24:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd354e4600 con 0x55fd354dbea0 2023-07-14T16:24:13.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57450/0 (socket says 172.21.15.62:57450) 2023-07-14T16:24:13.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91b7fff700 1 -- 172.21.15.62:0/808179824 learned_addr learned my addr 172.21.15.62:0/808179824 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354d0e20 0x55fd354dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91bd79c700 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dbea0 0x55fd354ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91bcf9b700 1 -- 172.21.15.62:0/808179824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dbea0 msgr2=0x55fd354ec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dbea0 0x55fd354ec690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91bcf9b700 1 -- 172.21.15.62:0/808179824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 msgr2=0x55fd354e00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91bcf9b700 1 -- 172.21.15.62:0/808179824 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd352812d0 con 0x55fd354d0e20 2023-07-14T16:24:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.654+0000 7f91b7fff700 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.655+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354d0e20 0x55fd354dfbb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009ce0 tx=0x7f91ac010f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62bfcd3cb1aab66 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.655+0000 7f91a37fe700 1 -- 172.21.15.62:0/808179824 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac01d040 con 0x55fd354d0e20 2023-07-14T16:24:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.655+0000 7f91a37fe700 1 -- 172.21.15.62:0/808179824 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91ac0054d0 con 0x55fd354d0e20 2023-07-14T16:24:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.655+0000 7f91a37fe700 1 -- 172.21.15.62:0/808179824 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac02ce00 con 0x55fd354d0e20 2023-07-14T16:24:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 -- 172.21.15.62:0/808179824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354d0e20 msgr2=0x55fd354dfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354d0e20 0x55fd354dfbb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009ce0 tx=0x7f91ac010f80 comp rx=0 tx=0).stop 2023-07-14T16:24:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 -- 172.21.15.62:0/808179824 shutdown_connections 2023-07-14T16:24:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354dbea0 0x55fd354ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354d0e20 0x55fd354dfbb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/808179824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 -- 172.21.15.62:0/808179824 >> 172.21.15.62:0/808179824 conn(0x55fd354e4200 msgr2=0x55fd354e3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 -- 172.21.15.62:0/808179824 shutdown_connections 2023-07-14T16:24:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 -- 172.21.15.62:0/808179824 wait complete. 2023-07-14T16:24:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.656+0000 7f91d2b87b80 1 -- start start 2023-07-14T16:24:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91d2b87b80 1 Processor -- start 2023-07-14T16:24:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354d0e20 0x55fd354ec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91d2b87b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd354e99c0 con 0x55fd354db550 2023-07-14T16:24:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd354e9b30 con 0x55fd354dbea0 2023-07-14T16:24:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91d2b87b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd355448f0 con 0x55fd354d0e20 2023-07-14T16:24:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91d2b87b80 10 client.0 _ll_register_callbacks cb 0x7f91a4011710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57452/0 (socket says 172.21.15.62:57452) 2023-07-14T16:24:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.657+0000 7f91b7fff700 1 -- 172.21.15.62:0/3447412047 learned_addr learned my addr 172.21.15.62:0/3447412047 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354d0e20 0x55fd354ec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f91bd79c700 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f91b7fff700 1 -- 172.21.15.62:0/3447412047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354d0e20 msgr2=0x55fd354ec3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f91b7fff700 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354d0e20 0x55fd354ec3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f91b7fff700 1 -- 172.21.15.62:0/3447412047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 msgr2=0x55fd354e9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f91b7fff700 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354e9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f91b7fff700 1 -- 172.21.15.62:0/3447412047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a4019040 con 0x55fd354db550 2023-07-14T16:24:13.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f91b7fff700 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e8f00 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f91a402b660 tx=0x7f91a400abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826869fe206deebb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f9197fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:13.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.658+0000 7f9197fff700 1 -- 172.21.15.62:0/3447412047 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a402bbe0 con 0x55fd354db550 2023-07-14T16:24:13.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 1 -- 172.21.15.62:0/3447412047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91ac01e040 con 0x55fd354db550 2023-07-14T16:24:13.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f9197fff700 1 -- 172.21.15.62:0/3447412047 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f91a402bd40 con 0x55fd354db550 2023-07-14T16:24:13.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f9197fff700 1 -- 172.21.15.62:0/3447412047 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91a401e760 con 0x55fd354db550 2023-07-14T16:24:13.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 10 client.17946 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:13.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 1 -- 172.21.15.62:0/3447412047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fd35243ce0 con 0x55fd354db550 2023-07-14T16:24:13.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f9197fff700 1 -- 172.21.15.62:0/3447412047 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91ac01e040 con 0x55fd354db550 2023-07-14T16:24:13.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f9197fff700 1 -- 172.21.15.62:0/3447412047 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f91a403e760 con 0x55fd354db550 2023-07-14T16:24:13.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 10 client.17946 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:13.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 20 client.17946 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fd354dccf0 0x55fd355483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 4 client.17946 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:24:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.659+0000 7f91d2b87b80 1 -- 172.21.15.62:0/3447412047 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fd353e80d0 con 0x55fd354dccf0 2023-07-14T16:24:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.660+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fd354dccf0 0x55fd355483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.660+0000 7f91bcf9b700 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fd354dccf0 0x55fd355483f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daba5a5c96223a9 server_cookie=df40e4f1bded7972 in_seq=0 out_seq=0 2023-07-14T16:24:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.660+0000 7f9197fff700 10 client.17946 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:13.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.661+0000 7f9197fff700 1 -- 172.21.15.62:0/3447412047 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fd353e80d0 con 0x55fd354dccf0 2023-07-14T16:24:13.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.661+0000 7f9197fff700 10 client.17946 handle_command_reply: tid=0 2023-07-14T16:24:13.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 client.17946 shutdown 2023-07-14T16:24:13.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 -- 172.21.15.62:0/3447412047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 msgr2=0x55fd354e8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e8f00 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f91a402b660 tx=0x7f91a400abc0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 -- 172.21.15.62:0/3447412047 shutdown_connections 2023-07-14T16:24:13.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fd354dccf0 0x55fd355483f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fd354dbea0 0x55fd354e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fd354d0e20 0x55fd354ec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 --2- 172.21.15.62:0/3447412047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fd354db550 0x55fd354e8f00 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f9197fff700 0 client.17946 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:13.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 -- 172.21.15.62:0/3447412047 >> 172.21.15.62:0/3447412047 conn(0x55fd354e4200 msgr2=0x55fd354dfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 -- 172.21.15.62:0/3447412047 shutdown_connections 2023-07-14T16:24:13.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 1 -- 172.21.15.62:0/3447412047 wait complete. 2023-07-14T16:24:13.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.662+0000 7f91d2b87b80 20 client.17946 trim_cache size 0 max 16384 2023-07-14T16:24:13.645 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:24:13.646 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:24:13.646 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "2e28e95e-9c7f-43bb-bd78-c0c925b09b34", 2023-07-14T16:24:13.646 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:24:13.646 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:24:13.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 -- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f91a80a1150 msgr2=0x7f91a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f91a80a1150 0x7f91a80a3600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f91a400f5e0 tx=0x7f91a400a380 comp rx=0 tx=0).stop 2023-07-14T16:24:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 -- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b81057a0 msgr2=0x7f91b819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b81057a0 0x7f91b819eed0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007ee0 tx=0x7f91b000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 -- 172.21.15.62:0/2599363846 shutdown_connections 2023-07-14T16:24:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b81057a0 0x7f91b819eed0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91b8100560 0x7f91b8197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f91a80a1150 0x7f91a80a3600 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 --2- 172.21.15.62:0/2599363846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91b8102e80 0x7f91b8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 -- 172.21.15.62:0/2599363846 >> 172.21.15.62:0/2599363846 conn(0x7f91b80fa000 msgr2=0x7f91b80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.663+0000 7f91bf1ff700 1 -- 172.21.15.62:0/2599363846 shutdown_connections 2023-07-14T16:24:13.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.664+0000 7f91bf1ff700 1 -- 172.21.15.62:0/2599363846 wait complete. 2023-07-14T16:24:13.664 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:24:13.901 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:24:13 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:24:13.671+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v181>,<0x10000000003/client.0 v182>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v194>,<0x10000000001/sv_0 v208>,<0x10000000000/_nogroup v202>,<0x1/volumes v192>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:24:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 -- 172.21.15.62:0/990273917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c1057a0 msgr2=0x7f363c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 --2- 172.21.15.62:0/990273917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c1057a0 0x7f363c107c00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f363400b0a0 tx=0x7f363400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 -- 172.21.15.62:0/990273917 shutdown_connections 2023-07-14T16:24:13.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 --2- 172.21.15.62:0/990273917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c1057a0 0x7f363c107c00 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 --2- 172.21.15.62:0/990273917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c100560 0x7f363c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 --2- 172.21.15.62:0/990273917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f363c102e80 0x7f363c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 -- 172.21.15.62:0/990273917 >> 172.21.15.62:0/990273917 conn(0x7f363c0fa000 msgr2=0x7f363c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 -- 172.21.15.62:0/990273917 shutdown_connections 2023-07-14T16:24:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.930+0000 7f36427ff700 1 -- 172.21.15.62:0/990273917 wait complete. 2023-07-14T16:24:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.932+0000 7f36427ff700 1 Processor -- start 2023-07-14T16:24:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.932+0000 7f36427ff700 1 -- start start 2023-07-14T16:24:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f36427ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c100560 0x7f363c0ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f36427ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102e80 0x7f363c0fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f36427ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f363c1057a0 0x7f363c0fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f36427ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0ff1f0 con 0x7f363c100560 2023-07-14T16:24:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f36427ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0ff360 con 0x7f363c1057a0 2023-07-14T16:24:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f36427ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c19f900 con 0x7f363c102e80 2023-07-14T16:24:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f363bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c100560 0x7f363c0ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f363bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c100560 0x7f363c0ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57472/0 (socket says 172.21.15.62:57472) 2023-07-14T16:24:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f363bfff700 1 -- 172.21.15.62:0/2777240545 learned_addr learned my addr 172.21.15.62:0/2777240545 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.933+0000 7f363b7fe700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102e80 0x7f363c0fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f3640d9c700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f363c1057a0 0x7f363c0fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f363bfff700 1 -- 172.21.15.62:0/2777240545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102e80 msgr2=0x7f363c0fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f363bfff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102e80 0x7f363c0fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f363bfff700 1 -- 172.21.15.62:0/2777240545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f363c1057a0 msgr2=0x7f363c0fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f363bfff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f363c1057a0 0x7f363c0fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f363bfff700 1 -- 172.21.15.62:0/2777240545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3634009d00 con 0x7f363c100560 2023-07-14T16:24:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f363b7fe700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102e80 0x7f363c0fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f3640d9c700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f363c1057a0 0x7f363c0fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.934+0000 7f363bfff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c100560 0x7f363c0ffed0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e9a0 tx=0x7f362c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.935+0000 7f36397fa700 1 -- 172.21.15.62:0/2777240545 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c009790 con 0x7f363c100560 2023-07-14T16:24:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.935+0000 7f36397fa700 1 -- 172.21.15.62:0/2777240545 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f362c00cbd0 con 0x7f363c100560 2023-07-14T16:24:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.935+0000 7f36427ff700 1 -- 172.21.15.62:0/2777240545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c19fbe0 con 0x7f363c100560 2023-07-14T16:24:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.935+0000 7f36397fa700 1 -- 172.21.15.62:0/2777240545 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c018680 con 0x7f363c100560 2023-07-14T16:24:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.935+0000 7f36427ff700 1 -- 172.21.15.62:0/2777240545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c1a00f0 con 0x7f363c100560 2023-07-14T16:24:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.939+0000 7f36397fa700 1 -- 172.21.15.62:0/2777240545 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f362c00cd40 con 0x7f363c100560 2023-07-14T16:24:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.940+0000 7f36560adb80 1 Processor -- start 2023-07-14T16:24:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.940+0000 7f36397fa700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36240a1150 0x7f36240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f36560adb80 1 -- start start 2023-07-14T16:24:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f36397fa700 1 -- 172.21.15.62:0/2777240545 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f362c014070 con 0x7f363c100560 2023-07-14T16:24:13.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f3640d9c700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36240a1150 0x7f36240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f36560adb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a6c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f36560adb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 0x560ed2a6d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f36560adb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6dc70 0x560ed2a6e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f36560adb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed2971cb0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f36560adb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed29c4a40 con 0x560ed2a6dc70 2023-07-14T16:24:13.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f36560adb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed29bded0 con 0x560ed2a6ce20 2023-07-14T16:24:13.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f363bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a6c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f363bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a6c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57482/0 (socket says 172.21.15.62:57482) 2023-07-14T16:24:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.941+0000 7f363bfff700 1 -- 172.21.15.62:0/210486267 learned_addr learned my addr 172.21.15.62:0/210486267 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f3640d9c700 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6dc70 0x560ed2a6e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f363b7fe700 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 0x560ed2a6d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f363bfff700 1 -- 172.21.15.62:0/210486267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 msgr2=0x560ed2a6d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f363bfff700 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 0x560ed2a6d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f363bfff700 1 -- 172.21.15.62:0/210486267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6dc70 msgr2=0x560ed2a6e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f363bfff700 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6dc70 0x560ed2a6e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f363bfff700 1 -- 172.21.15.62:0/210486267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ed27ebde0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f3640d9c700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36240a1150 0x7f36240a3600 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f363c100530 tx=0x7f3634006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f363bfff700 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a6c8e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f362c0a20c0 tx=0x7f362c099db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1db5c6129c34fc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f36227fc700 1 -- 172.21.15.62:0/210486267 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0adea0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f36227fc700 1 -- 172.21.15.62:0/210486267 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f362c0a25f0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.942+0000 7f36227fc700 1 -- 172.21.15.62:0/210486267 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0a23c0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 -- 172.21.15.62:0/210486267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 msgr2=0x560ed2a6c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a6c8e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f362c0a20c0 tx=0x7f362c099db0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 -- 172.21.15.62:0/210486267 shutdown_connections 2023-07-14T16:24:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6dc70 0x560ed2a6e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a6c8e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 --2- 172.21.15.62:0/210486267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 0x560ed2a6d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 -- 172.21.15.62:0/210486267 >> 172.21.15.62:0/210486267 conn(0x560ed2a61da0 msgr2=0x560ed2a671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 -- 172.21.15.62:0/210486267 shutdown_connections 2023-07-14T16:24:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.943+0000 7f36560adb80 1 -- 172.21.15.62:0/210486267 wait complete. 2023-07-14T16:24:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.944+0000 7f36560adb80 1 -- start start 2023-07-14T16:24:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.944+0000 7f36560adb80 1 Processor -- start 2023-07-14T16:24:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f36560adb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a76070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f36560adb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 0x560ed2a765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f36560adb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f36560adb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed295dac0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f36560adb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed295d7c0 con 0x560ed2a6ce20 2023-07-14T16:24:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f36560adb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed2841920 con 0x560ed2a6dc70 2023-07-14T16:24:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f36560adb80 10 client.0 _ll_register_callbacks cb 0x560ed295b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f363bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a76070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f363bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a76070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57494/0 (socket says 172.21.15.62:57494) 2023-07-14T16:24:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f363bfff700 1 -- 172.21.15.62:0/178142672 learned_addr learned my addr 172.21.15.62:0/178142672 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f363b7fe700 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 0x560ed2a765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.945+0000 7f3640d9c700 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f363bfff700 1 -- 172.21.15.62:0/178142672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 msgr2=0x560ed2a7d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f363bfff700 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f363bfff700 1 -- 172.21.15.62:0/178142672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 msgr2=0x560ed2a765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f363bfff700 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 0x560ed2a765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f363bfff700 1 -- 172.21.15.62:0/178142672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c1a00f0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f3640d9c700 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f363bfff700 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a76070 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f362c0109c0 tx=0x7f362c010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d9ccdc62ccd80a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f361b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f361b7fe700 1 -- 172.21.15.62:0/178142672 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0b8070 con 0x560ed2a6c4d0 2023-07-14T16:24:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f361b7fe700 1 -- 172.21.15.62:0/178142672 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f362c098600 con 0x560ed2a6c4d0 2023-07-14T16:24:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f36560adb80 1 -- 172.21.15.62:0/178142672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ed29174f0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f361b7fe700 1 -- 172.21.15.62:0/178142672 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f362c0adea0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f36560adb80 10 client.17958 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.946+0000 7f36560adb80 1 -- 172.21.15.62:0/178142672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560ed27d4c60 con 0x560ed2a6c4d0 2023-07-14T16:24:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.947+0000 7f361b7fe700 1 -- 172.21.15.62:0/178142672 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f362c07e080 con 0x560ed2a6c4d0 2023-07-14T16:24:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.947+0000 7f361b7fe700 1 -- 172.21.15.62:0/178142672 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f362c099290 con 0x560ed2a6c4d0 2023-07-14T16:24:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.947+0000 7f36560adb80 10 client.17958 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.947+0000 7f36560adb80 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.947+0000 7f36560adb80 20 client.17958 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.947+0000 7f36560adb80 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560ed2a81b80 0x560ed2a83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.947+0000 7f36560adb80 4 client.17958 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.947+0000 7f36560adb80 1 -- 172.21.15.62:0/178142672 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ed29b6eb0 con 0x560ed2a81b80 2023-07-14T16:24:13.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.948+0000 7f363b7fe700 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560ed2a81b80 0x560ed2a83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.949+0000 7f363b7fe700 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560ed2a81b80 0x560ed2a83f90 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac31a3ad2d59969 server_cookie=4d1a1158206a611f in_seq=0 out_seq=0 2023-07-14T16:24:13.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.949+0000 7f361b7fe700 10 client.17958 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:13.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.949+0000 7f361b7fe700 1 -- 172.21.15.62:0/178142672 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560ed29b6eb0 con 0x560ed2a81b80 2023-07-14T16:24:13.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.949+0000 7f361b7fe700 10 client.17958 handle_command_reply: tid=0 2023-07-14T16:24:13.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.949+0000 7f36560adb80 1 client.17958 shutdown 2023-07-14T16:24:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 -- 172.21.15.62:0/178142672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 msgr2=0x560ed2a76070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a76070 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f362c0109c0 tx=0x7f362c010a70 comp rx=0 tx=0).stop 2023-07-14T16:24:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 -- 172.21.15.62:0/178142672 shutdown_connections 2023-07-14T16:24:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a76070 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560ed2a81b80 0x560ed2a83f90 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 --2- 172.21.15.62:0/178142672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 0x560ed2a765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f361b7fe700 0 client.17958 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 -- 172.21.15.62:0/178142672 >> 172.21.15.62:0/178142672 conn(0x560ed2a61da0 msgr2=0x560ed2a668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 -- 172.21.15.62:0/178142672 shutdown_connections 2023-07-14T16:24:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 1 -- 172.21.15.62:0/178142672 wait complete. 2023-07-14T16:24:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.950+0000 7f36560adb80 20 client.17958 trim_cache size 0 max 16384 2023-07-14T16:24:13.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.973+0000 7f36560adb80 1 Processor -- start 2023-07-14T16:24:13.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.973+0000 7f36560adb80 1 -- start start 2023-07-14T16:24:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f36560adb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6c4d0 0x560ed2a74d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f36560adb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 0x560ed2a7d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f36560adb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6dc70 0x560ed2a668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f36560adb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed2a68d70 con 0x560ed2a6dc70 2023-07-14T16:24:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f36560adb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed2a68310 con 0x560ed2a6c4d0 2023-07-14T16:24:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f36560adb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed2a73e40 con 0x560ed2a6ce20 2023-07-14T16:24:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f3640d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6dc70 0x560ed2a668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f363bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6c4d0 0x560ed2a74d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f363b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 0x560ed2a7d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f363bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6c4d0 0x560ed2a74d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51956/0 (socket says 172.21.15.62:51956) 2023-07-14T16:24:13.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.974+0000 7f363bfff700 1 -- 172.21.15.62:0/682124146 learned_addr learned my addr 172.21.15.62:0/682124146 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3640d9c700 1 -- 172.21.15.62:0/682124146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 msgr2=0x560ed2a7d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3640d9c700 1 --2- 172.21.15.62:0/682124146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 0x560ed2a7d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3640d9c700 1 -- 172.21.15.62:0/682124146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6c4d0 msgr2=0x560ed2a74d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3640d9c700 1 --2- 172.21.15.62:0/682124146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6c4d0 0x560ed2a74d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3640d9c700 1 -- 172.21.15.62:0/682124146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ed2812250 con 0x560ed2a6dc70 2023-07-14T16:24:13.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f363bfff700 1 --2- 172.21.15.62:0/682124146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6c4d0 0x560ed2a74d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3640d9c700 1 --2- 172.21.15.62:0/682124146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6dc70 0x560ed2a668b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f363400b070 tx=0x7f3634023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346953540c16cb57 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3622ffd700 1 -- 172.21.15.62:0/682124146 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400e040 con 0x560ed2a6dc70 2023-07-14T16:24:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3622ffd700 1 -- 172.21.15.62:0/682124146 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3634003e90 con 0x560ed2a6dc70 2023-07-14T16:24:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.975+0000 7f3622ffd700 1 -- 172.21.15.62:0/682124146 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340055a0 con 0x560ed2a6dc70 2023-07-14T16:24:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 -- 172.21.15.62:0/682124146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6dc70 msgr2=0x560ed2a668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 --2- 172.21.15.62:0/682124146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6dc70 0x560ed2a668b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f363400b070 tx=0x7f3634023000 comp rx=0 tx=0).stop 2023-07-14T16:24:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 -- 172.21.15.62:0/682124146 shutdown_connections 2023-07-14T16:24:13.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 --2- 172.21.15.62:0/682124146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6dc70 0x560ed2a668b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 --2- 172.21.15.62:0/682124146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6c4d0 0x560ed2a74d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 --2- 172.21.15.62:0/682124146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6ce20 0x560ed2a7d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 -- 172.21.15.62:0/682124146 >> 172.21.15.62:0/682124146 conn(0x560ed2a61da0 msgr2=0x560ed2a888b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 -- 172.21.15.62:0/682124146 shutdown_connections 2023-07-14T16:24:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.976+0000 7f36560adb80 1 -- 172.21.15.62:0/682124146 wait complete. 2023-07-14T16:24:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.977+0000 7f36560adb80 1 -- start start 2023-07-14T16:24:13.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.977+0000 7f36560adb80 1 Processor -- start 2023-07-14T16:24:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f36560adb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f36560adb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 0x560ed2a79ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f36560adb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f36560adb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed2a7ab00 con 0x560ed2a6c4d0 2023-07-14T16:24:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f36560adb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed2ad59f0 con 0x560ed2a6ce20 2023-07-14T16:24:13.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f36560adb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed2ad5b60 con 0x560ed2a6dc70 2023-07-14T16:24:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f36560adb80 10 client.0 _ll_register_callbacks cb 0x7f362c09a060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a7d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57510/0 (socket says 172.21.15.62:57510) 2023-07-14T16:24:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363bfff700 1 -- 172.21.15.62:0/2775563992 learned_addr learned my addr 172.21.15.62:0/2775563992 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363b7fe700 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 0x560ed2a79ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f3640d9c700 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363bfff700 1 -- 172.21.15.62:0/2775563992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 msgr2=0x560ed2a7a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363bfff700 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363bfff700 1 -- 172.21.15.62:0/2775563992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 msgr2=0x560ed2a79ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363bfff700 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 0x560ed2a79ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f363bfff700 1 -- 172.21.15.62:0/2775563992 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ed27d4c60 con 0x560ed2a6c4d0 2023-07-14T16:24:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.978+0000 7f3640d9c700 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:24:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f363bfff700 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a7d440 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f362c0b81a0 tx=0x7f362c0993e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da5f3869d96ea80 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f361b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:13.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f361b7fe700 1 -- 172.21.15.62:0/2775563992 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0adea0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f36560adb80 1 -- 172.21.15.62:0/2775563992 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f362c0a25f0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f361b7fe700 1 -- 172.21.15.62:0/2775563992 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f362c018e40 con 0x560ed2a6c4d0 2023-07-14T16:24:13.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f361b7fe700 1 -- 172.21.15.62:0/2775563992 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f362c0b8040 con 0x560ed2a6c4d0 2023-07-14T16:24:13.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f36560adb80 10 client.17964 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:13.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f36560adb80 1 -- 172.21.15.62:0/2775563992 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f363c1087f0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f361b7fe700 1 -- 172.21.15.62:0/2775563992 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f362c0bd020 con 0x560ed2a6c4d0 2023-07-14T16:24:13.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.979+0000 7f361b7fe700 1 -- 172.21.15.62:0/2775563992 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f362c0839f0 con 0x560ed2a6c4d0 2023-07-14T16:24:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f36560adb80 10 client.17964 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f36560adb80 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f36560adb80 20 client.17964 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f36560adb80 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560ed2ad70b0 0x560ed2ad9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f36560adb80 4 client.17964 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:24:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f36560adb80 1 -- 172.21.15.62:0/2775563992 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ed2979050 con 0x560ed2ad70b0 2023-07-14T16:24:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f363b7fe700 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560ed2ad70b0 0x560ed2ad9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:13.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f363b7fe700 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560ed2ad70b0 0x560ed2ad9490 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a3d6cbd84a4912 server_cookie=c0595daa54c666ee in_seq=0 out_seq=0 2023-07-14T16:24:13.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.980+0000 7f361b7fe700 10 client.17964 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:13.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f361b7fe700 1 -- 172.21.15.62:0/2775563992 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560ed2979050 con 0x560ed2ad70b0 2023-07-14T16:24:13.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f361b7fe700 10 client.17964 handle_command_reply: tid=0 2023-07-14T16:24:13.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 client.17964 shutdown 2023-07-14T16:24:13.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 -- 172.21.15.62:0/2775563992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 msgr2=0x560ed2a7d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a7d440 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f362c0b81a0 tx=0x7f362c0993e0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 -- 172.21.15.62:0/2775563992 shutdown_connections 2023-07-14T16:24:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560ed2a6dc70 0x560ed2a7a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ed2a6c4d0 0x560ed2a7d440 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560ed2ad70b0 0x560ed2ad9490 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 --2- 172.21.15.62:0/2775563992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ed2a6ce20 0x560ed2a79ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 -- 172.21.15.62:0/2775563992 >> 172.21.15.62:0/2775563992 conn(0x560ed2a61da0 msgr2=0x560ed2a668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f361b7fe700 0 client.17964 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 -- 172.21.15.62:0/2775563992 shutdown_connections 2023-07-14T16:24:13.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 1 -- 172.21.15.62:0/2775563992 wait complete. 2023-07-14T16:24:13.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.981+0000 7f36560adb80 20 client.17964 trim_cache size 0 max 16384 2023-07-14T16:24:13.966 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:24:13.966 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:24:13.966 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:24:13.966 INFO:teuthology.orchestra.run.smithi062.stdout: "2e28e95e-9c7f-43bb-bd78-c0c925b09b34": { 2023-07-14T16:24:13.966 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:24:13.967 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "2e28e95e-9c7f-43bb-bd78-c0c925b09b34", 2023-07-14T16:24:13.967 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:24:13.967 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:24:13.967 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:24:13.967 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:24:13.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 -- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36240a1150 msgr2=0x7f36240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36240a1150 0x7f36240a3600 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f363c100530 tx=0x7f3634006210 comp rx=0 tx=0).stop 2023-07-14T16:24:13.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 -- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c100560 msgr2=0x7f363c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:13.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c100560 0x7f363c0ffed0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e9a0 tx=0x7f362c00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 -- 172.21.15.62:0/2777240545 shutdown_connections 2023-07-14T16:24:13.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36240a1150 0x7f36240a3600 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f363c1057a0 0x7f363c0fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c100560 0x7f363c0ffed0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 --2- 172.21.15.62:0/2777240545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f363c102e80 0x7f363c0fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:13.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 -- 172.21.15.62:0/2777240545 >> 172.21.15.62:0/2777240545 conn(0x7f363c0fa000 msgr2=0x7f363c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:13.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 -- 172.21.15.62:0/2777240545 shutdown_connections 2023-07-14T16:24:13.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:13.982+0000 7f36427ff700 1 -- 172.21.15.62:0/2777240545 wait complete. 2023-07-14T16:24:13.982 INFO:tasks.cephfs.filesystem:scrub status for tag:2e28e95e-9c7f-43bb-bd78-c0c925b09b34 - {'path': '/', 'tag': '2e28e95e-9c7f-43bb-bd78-c0c925b09b34', 'options': 'recursive,force'} 2023-07-14T16:24:15.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: pgmap v615: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:15.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:24:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:24:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:24:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:24:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:24:15.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.026 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:14 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: pgmap v615: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:15.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:24:15.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:24:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:24:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:24:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:24:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:14 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: pgmap v615: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:15.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:24:15.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:24:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:24:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:24:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:24:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:14 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:17.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:16 smithi164 ceph-mon[97427]: pgmap v616: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 18 op/s 2023-07-14T16:24:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:16 smithi062 ceph-mon[95543]: pgmap v616: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 18 op/s 2023-07-14T16:24:17.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:16 smithi081 ceph-mon[127795]: pgmap v616: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 18 op/s 2023-07-14T16:24:19.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:18 smithi164 ceph-mon[97427]: pgmap v617: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 28 op/s 2023-07-14T16:24:19.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:18 smithi062 ceph-mon[95543]: pgmap v617: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 28 op/s 2023-07-14T16:24:19.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:18 smithi081 ceph-mon[127795]: pgmap v617: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 28 op/s 2023-07-14T16:24:21.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:20 smithi164 ceph-mon[97427]: pgmap v618: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:24:21.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:20 smithi062 ceph-mon[95543]: pgmap v618: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:24:21.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:20 smithi081 ceph-mon[127795]: pgmap v618: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:24:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:22 smithi164 ceph-mon[97427]: pgmap v619: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:24:23.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:22 smithi062 ceph-mon[95543]: pgmap v619: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:24:23.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:22 smithi081 ceph-mon[127795]: pgmap v619: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:24:24.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:23 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:24:24.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:23 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:24:24.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:23 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:24:24.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:23 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:24:24.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:23 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:24:24.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:23 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:24:24.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:23 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:24:24.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:23 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:24:24.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:23 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:24:25.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:24 smithi164 ceph-mon[97427]: pgmap v620: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:24:25.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:24:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:24:25.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:24:25.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:24 smithi062 ceph-mon[95543]: pgmap v620: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:24:25.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:24:25.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:24:25.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:24:25.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:24 smithi081 ceph-mon[127795]: pgmap v620: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:24:25.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:24:25.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:24:25.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:24:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.558+0000 7fc28cfff700 1 -- 172.21.15.62:0/2464983163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288105860 msgr2=0x7fc288107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.558+0000 7fc28cfff700 1 --2- 172.21.15.62:0/2464983163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288105860 0x7fc288107c40 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc280009ec0 tx=0x7fc28000b330 comp rx=0 tx=0).stop 2023-07-14T16:24:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.559+0000 7fc28cfff700 1 -- 172.21.15.62:0/2464983163 shutdown_connections 2023-07-14T16:24:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.559+0000 7fc28cfff700 1 --2- 172.21.15.62:0/2464983163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288105860 0x7fc288107c40 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.559+0000 7fc28cfff700 1 --2- 172.21.15.62:0/2464983163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc288100620 0x7fc288102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.559+0000 7fc28cfff700 1 --2- 172.21.15.62:0/2464983163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc288102f40 0x7fc288105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.559+0000 7fc28cfff700 1 -- 172.21.15.62:0/2464983163 >> 172.21.15.62:0/2464983163 conn(0x7fc2880fa000 msgr2=0x7fc2880fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.559+0000 7fc28cfff700 1 -- 172.21.15.62:0/2464983163 shutdown_connections 2023-07-14T16:24:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.559+0000 7fc28cfff700 1 -- 172.21.15.62:0/2464983163 wait complete. 2023-07-14T16:24:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.560+0000 7fc28cfff700 1 Processor -- start 2023-07-14T16:24:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.560+0000 7fc28cfff700 1 -- start start 2023-07-14T16:24:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.560+0000 7fc28cfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288100620 0x7fc288197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.560+0000 7fc28cfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc288102f40 0x7fc288198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28cfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc288105860 0x7fc288198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28cfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2881995e0 con 0x7fc288100620 2023-07-14T16:24:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28cfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288199750 con 0x7fc288102f40 2023-07-14T16:24:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28cfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288199910 con 0x7fc288105860 2023-07-14T16:24:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288100620 0x7fc288197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288100620 0x7fc288197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35860/0 (socket says 172.21.15.62:35860) 2023-07-14T16:24:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28659c700 1 -- 172.21.15.62:0/371162175 learned_addr learned my addr 172.21.15.62:0/371162175 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc285d9b700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc288102f40 0x7fc288198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc286d9d700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc288105860 0x7fc288198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28659c700 1 -- 172.21.15.62:0/371162175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc288105860 msgr2=0x7fc288198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28659c700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc288105860 0x7fc288198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28659c700 1 -- 172.21.15.62:0/371162175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc288102f40 msgr2=0x7fc288198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28659c700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc288102f40 0x7fc288198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.561+0000 7fc28659c700 1 -- 172.21.15.62:0/371162175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280005e70 con 0x7fc288100620 2023-07-14T16:24:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.562+0000 7fc28659c700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288100620 0x7fc288197bc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc27800e970 tx=0x7fc27800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.562+0000 7fc2777fe700 1 -- 172.21.15.62:0/371162175 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278009790 con 0x7fc288100620 2023-07-14T16:24:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.562+0000 7fc2777fe700 1 -- 172.21.15.62:0/371162175 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc27800c7e0 con 0x7fc288100620 2023-07-14T16:24:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.562+0000 7fc28cfff700 1 -- 172.21.15.62:0/371162175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc288199bf0 con 0x7fc288100620 2023-07-14T16:24:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.562+0000 7fc2777fe700 1 -- 172.21.15.62:0/371162175 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278005440 con 0x7fc288100620 2023-07-14T16:24:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.562+0000 7fc28cfff700 1 -- 172.21.15.62:0/371162175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2881a03b0 con 0x7fc288100620 2023-07-14T16:24:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.566+0000 7fc2777fe700 1 -- 172.21.15.62:0/371162175 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc27800c950 con 0x7fc288100620 2023-07-14T16:24:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.567+0000 7fc2a096fb80 1 Processor -- start 2023-07-14T16:24:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.567+0000 7fc2a096fb80 1 -- start start 2023-07-14T16:24:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031fb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fb840 0x558a031fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 0x558a031fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03100c50 con 0x558a030576b0 2023-07-14T16:24:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a031539e0 con 0x558a031fb840 2023-07-14T16:24:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0314ce70 con 0x558a031fc6e0 2023-07-14T16:24:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc28659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fb840 0x558a031fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc28659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fb840 0x558a031fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53624/0 (socket says 172.21.15.62:53624) 2023-07-14T16:24:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc28659c700 1 -- 172.21.15.62:0/941788021 learned_addr learned my addr 172.21.15.62:0/941788021 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc286d9d700 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031fb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.568+0000 7fc285d9b700 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 0x558a031fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc2777fe700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc2700a1150 0x7fc2700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc2777fe700 1 -- 172.21.15.62:0/371162175 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc278014070 con 0x7fc288100620 2023-07-14T16:24:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc28659c700 1 -- 172.21.15.62:0/941788021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 msgr2=0x558a031fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc28659c700 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 0x558a031fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc28659c700 1 -- 172.21.15.62:0/941788021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 msgr2=0x558a031fb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc28659c700 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031fb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc28659c700 1 -- 172.21.15.62:0/941788021 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a02f7ad80 con 0x558a031fb840 2023-07-14T16:24:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc286d9d700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc2700a1150 0x7fc2700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.569+0000 7fc28659c700 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fb840 0x558a031fc1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc27800cef0 tx=0x7fc278005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c553e99cfb11e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.570+0000 7fc274ff9700 1 -- 172.21.15.62:0/941788021 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278099a10 con 0x558a031fb840 2023-07-14T16:24:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.570+0000 7fc274ff9700 1 -- 172.21.15.62:0/941788021 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc278099830 con 0x558a031fb840 2023-07-14T16:24:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.570+0000 7fc274ff9700 1 -- 172.21.15.62:0/941788021 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780055a0 con 0x558a031fb840 2023-07-14T16:24:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 -- 172.21.15.62:0/941788021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fb840 msgr2=0x558a031fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc286d9d700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc2700a1150 0x7fc2700a3600 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x558a031fb060 tx=0x7fc2800099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fb840 0x558a031fc1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc27800cef0 tx=0x7fc278005910 comp rx=0 tx=0).stop 2023-07-14T16:24:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 -- 172.21.15.62:0/941788021 shutdown_connections 2023-07-14T16:24:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 0x558a031fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/941788021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fb840 0x558a031fc1a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 -- 172.21.15.62:0/941788021 >> 172.21.15.62:0/941788021 conn(0x558a031f0be0 msgr2=0x558a031f5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 -- 172.21.15.62:0/941788021 shutdown_connections 2023-07-14T16:24:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 -- 172.21.15.62:0/941788021 wait complete. 2023-07-14T16:24:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.571+0000 7fc2a096fb80 1 -- start start 2023-07-14T16:24:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.572+0000 7fc2a096fb80 1 Processor -- start 2023-07-14T16:24:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.572+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 0x558a03204840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.572+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a03204d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.572+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fc6e0 0x558a0320bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.572+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a030eca60 con 0x558a031fc6e0 2023-07-14T16:24:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.572+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a030ec760 con 0x558a030576b0 2023-07-14T16:24:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.572+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a02fd08c0 con 0x558a031fb840 2023-07-14T16:24:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.572+0000 7fc2a096fb80 10 client.0 _ll_register_callbacks cb 0x558a030ea700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc286d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fc6e0 0x558a0320bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc285d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a03204d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc285d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a03204d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60230/0 (socket says 172.21.15.62:60230) 2023-07-14T16:24:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc286d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fc6e0 0x558a0320bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35878/0 (socket says 172.21.15.62:35878) 2023-07-14T16:24:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc285d9b700 1 -- 172.21.15.62:0/1006425110 learned_addr learned my addr 172.21.15.62:0/1006425110 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc28659c700 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 0x558a03204840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc285d9b700 1 -- 172.21.15.62:0/1006425110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 msgr2=0x558a03204840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc285d9b700 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 0x558a03204840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc285d9b700 1 -- 172.21.15.62:0/1006425110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fc6e0 msgr2=0x558a0320bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc285d9b700 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fc6e0 0x558a0320bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.573+0000 7fc285d9b700 1 -- 172.21.15.62:0/1006425110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2881a03b0 con 0x558a031fb840 2023-07-14T16:24:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.574+0000 7fc285d9b700 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a03204d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b960 tx=0x7fc27c00bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbbbad84ce0e0534 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.574+0000 7fc26dffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:24:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.574+0000 7fc26dffb700 1 -- 172.21.15.62:0/1006425110 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c016070 con 0x558a031fb840 2023-07-14T16:24:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.574+0000 7fc26dffb700 1 -- 172.21.15.62:0/1006425110 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc27c003b40 con 0x558a031fb840 2023-07-14T16:24:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.574+0000 7fc2a096fb80 1 -- 172.21.15.62:0/1006425110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a030a6490 con 0x558a031fb840 2023-07-14T16:24:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.574+0000 7fc26dffb700 1 -- 172.21.15.62:0/1006425110 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc27c003d00 con 0x558a031fb840 2023-07-14T16:24:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.574+0000 7fc2a096fb80 10 client.27508 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.574+0000 7fc2a096fb80 1 -- 172.21.15.62:0/1006425110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558a02f63c00 con 0x558a031fb840 2023-07-14T16:24:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.575+0000 7fc26dffb700 1 -- 172.21.15.62:0/1006425110 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558a030a6490 con 0x558a031fb840 2023-07-14T16:24:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.575+0000 7fc26dffb700 1 -- 172.21.15.62:0/1006425110 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc27c01b420 con 0x558a031fb840 2023-07-14T16:24:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.575+0000 7fc2a096fb80 10 client.27508 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.575+0000 7fc2a096fb80 10 client.27508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.575+0000 7fc2a096fb80 20 client.27508 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.575+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a032106a0 0x558a03212ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.575+0000 7fc2a096fb80 4 client.27508 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.575+0000 7fc2a096fb80 1 -- 172.21.15.62:0/1006425110 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a03145e50 con 0x558a032106a0 2023-07-14T16:24:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.576+0000 7fc28659c700 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a032106a0 0x558a03212ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.576+0000 7fc28659c700 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a032106a0 0x558a03212ab0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1edcf637ff13bc server_cookie=c0220717737a2737 in_seq=0 out_seq=0 2023-07-14T16:24:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.576+0000 7fc26dffb700 10 client.27508 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc26dffb700 1 -- 172.21.15.62:0/1006425110 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a03145e50 con 0x558a032106a0 2023-07-14T16:24:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc26dffb700 10 client.27508 handle_command_reply: tid=0 2023-07-14T16:24:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 client.27508 shutdown 2023-07-14T16:24:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 -- 172.21.15.62:0/1006425110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 msgr2=0x558a03204d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a03204d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00b960 tx=0x7fc27c00bc70 comp rx=0 tx=0).stop 2023-07-14T16:24:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 -- 172.21.15.62:0/1006425110 shutdown_connections 2023-07-14T16:24:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fc6e0 0x558a0320bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a032106a0 0x558a03212ab0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 0x558a03204840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/1006425110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a03204d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc2a096fb80 1 -- 172.21.15.62:0/1006425110 >> 172.21.15.62:0/1006425110 conn(0x558a031f0be0 msgr2=0x558a031f54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.577+0000 7fc26dffb700 0 client.27508 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.578+0000 7fc2a096fb80 1 -- 172.21.15.62:0/1006425110 shutdown_connections 2023-07-14T16:24:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.578+0000 7fc2a096fb80 1 -- 172.21.15.62:0/1006425110 wait complete. 2023-07-14T16:24:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.578+0000 7fc2a096fb80 20 client.27508 trim_cache size 0 max 16384 2023-07-14T16:24:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc2a096fb80 1 Processor -- start 2023-07-14T16:24:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc2a096fb80 1 -- start start 2023-07-14T16:24:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a031ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fc6e0 0x558a0320bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a031f2d00 con 0x558a030576b0 2023-07-14T16:24:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0320dab0 con 0x558a031fc6e0 2023-07-14T16:24:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a032043b0 con 0x558a031fb840 2023-07-14T16:24:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc28659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc28659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35888/0 (socket says 172.21.15.62:35888) 2023-07-14T16:24:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc28659c700 1 -- 172.21.15.62:0/3664195213 learned_addr learned my addr 172.21.15.62:0/3664195213 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.596+0000 7fc286d9d700 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fc6e0 0x558a0320bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc285d9b700 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a031ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc28659c700 1 -- 172.21.15.62:0/3664195213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 msgr2=0x558a031ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc28659c700 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a031ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc28659c700 1 -- 172.21.15.62:0/3664195213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fc6e0 msgr2=0x558a0320bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc28659c700 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fc6e0 0x558a0320bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc28659c700 1 -- 172.21.15.62:0/3664195213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a02fa11f0 con 0x558a030576b0 2023-07-14T16:24:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc28659c700 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031ff5f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc2780999a0 tx=0x7fc27800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e528a3860972e2d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc2757fa700 1 -- 172.21.15.62:0/3664195213 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278099f50 con 0x558a030576b0 2023-07-14T16:24:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc2757fa700 1 -- 172.21.15.62:0/3664195213 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc27809df00 con 0x558a030576b0 2023-07-14T16:24:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.597+0000 7fc2757fa700 1 -- 172.21.15.62:0/3664195213 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27809a130 con 0x558a030576b0 2023-07-14T16:24:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 -- 172.21.15.62:0/3664195213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 msgr2=0x558a031ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031ff5f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc2780999a0 tx=0x7fc27800cf50 comp rx=0 tx=0).stop 2023-07-14T16:24:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 -- 172.21.15.62:0/3664195213 shutdown_connections 2023-07-14T16:24:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a031fc6e0 0x558a0320bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a030576b0 0x558a031ff5f0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/3664195213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fb840 0x558a031ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 -- 172.21.15.62:0/3664195213 >> 172.21.15.62:0/3664195213 conn(0x558a031f0be0 msgr2=0x558a03204840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 -- 172.21.15.62:0/3664195213 shutdown_connections 2023-07-14T16:24:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.598+0000 7fc2a096fb80 1 -- 172.21.15.62:0/3664195213 wait complete. 2023-07-14T16:24:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.599+0000 7fc2a096fb80 1 -- start start 2023-07-14T16:24:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.599+0000 7fc2a096fb80 1 Processor -- start 2023-07-14T16:24:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 0x558a03207ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fb840 0x558a03208410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc2a096fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 0x558a0320b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0320b970 con 0x558a031fb840 2023-07-14T16:24:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0320bae0 con 0x558a030576b0 2023-07-14T16:24:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc2a096fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0320bca0 con 0x558a031fc6e0 2023-07-14T16:24:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc2a096fb80 10 client.0 _ll_register_callbacks cb 0x7fc27c004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc285d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fb840 0x558a03208410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc285d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fb840 0x558a03208410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35894/0 (socket says 172.21.15.62:35894) 2023-07-14T16:24:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc285d9b700 1 -- 172.21.15.62:0/2609463543 learned_addr learned my addr 172.21.15.62:0/2609463543 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc286d9d700 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 0x558a0320b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc28659c700 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 0x558a03207ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc285d9b700 1 -- 172.21.15.62:0/2609463543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 msgr2=0x558a0320b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc285d9b700 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 0x558a0320b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc285d9b700 1 -- 172.21.15.62:0/2609463543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 msgr2=0x558a03207ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc285d9b700 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 0x558a03207ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.600+0000 7fc285d9b700 1 -- 172.21.15.62:0/2609463543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a02f7ad80 con 0x558a031fb840 2023-07-14T16:24:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.601+0000 7fc285d9b700 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fb840 0x558a03208410 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc27c000c00 tx=0x7fc27c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4f03ee31dc9c5f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.601+0000 7fc26dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.601+0000 7fc26dffb700 1 -- 172.21.15.62:0/2609463543 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c016040 con 0x558a031fb840 2023-07-14T16:24:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.601+0000 7fc2a096fb80 1 -- 172.21.15.62:0/2609463543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc278099830 con 0x558a031fb840 2023-07-14T16:24:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.601+0000 7fc26dffb700 1 -- 172.21.15.62:0/2609463543 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc27c000e40 con 0x558a031fb840 2023-07-14T16:24:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.601+0000 7fc26dffb700 1 -- 172.21.15.62:0/2609463543 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc27c036030 con 0x558a031fb840 2023-07-14T16:24:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.601+0000 7fc2a096fb80 10 client.17982 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.601+0000 7fc2a096fb80 1 -- 172.21.15.62:0/2609463543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558a02f79250 con 0x558a031fb840 2023-07-14T16:24:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc26dffb700 1 -- 172.21.15.62:0/2609463543 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc278099830 con 0x558a031fb840 2023-07-14T16:24:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc26dffb700 1 -- 172.21.15.62:0/2609463543 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc27c010c40 con 0x558a031fb840 2023-07-14T16:24:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc2a096fb80 10 client.17982 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc2a096fb80 10 client.17982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc2a096fb80 20 client.17982 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a03265820 0x558a03267c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc2a096fb80 4 client.17982 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:24:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc2a096fb80 1 -- 172.21.15.62:0/2609463543 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558a03107ff0 con 0x558a03265820 2023-07-14T16:24:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.602+0000 7fc28659c700 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a03265820 0x558a03267c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.603+0000 7fc26dffb700 10 client.17982 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.603+0000 7fc28659c700 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a03265820 0x558a03267c30 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a95916758e1327 server_cookie=9fd01026aa6b0fbf in_seq=0 out_seq=0 2023-07-14T16:24:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc26dffb700 1 -- 172.21.15.62:0/2609463543 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558a03107ff0 con 0x558a03265820 2023-07-14T16:24:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc26dffb700 10 client.17982 handle_command_reply: tid=0 2023-07-14T16:24:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 client.17982 shutdown 2023-07-14T16:24:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 -- 172.21.15.62:0/2609463543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fb840 msgr2=0x558a03208410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fb840 0x558a03208410 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc27c000c00 tx=0x7fc27c0057a0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 -- 172.21.15.62:0/2609463543 shutdown_connections 2023-07-14T16:24:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558a031fc6e0 0x558a0320b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558a03265820 0x558a03267c30 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558a030576b0 0x558a03207ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 --2- 172.21.15.62:0/2609463543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a031fb840 0x558a03208410 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc26dffb700 0 client.17982 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 -- 172.21.15.62:0/2609463543 >> 172.21.15.62:0/2609463543 conn(0x558a031f0be0 msgr2=0x558a032047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 -- 172.21.15.62:0/2609463543 shutdown_connections 2023-07-14T16:24:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.613+0000 7fc2a096fb80 1 -- 172.21.15.62:0/2609463543 wait complete. 2023-07-14T16:24:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc2a096fb80 20 client.17982 trim_cache size 0 max 16384 2023-07-14T16:24:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:24:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:24:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:24:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:24:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 -- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc2700a1150 msgr2=0x7fc2700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc2700a1150 0x7fc2700a3600 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x558a031fb060 tx=0x7fc2800099e0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 -- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288100620 msgr2=0x7fc288197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288100620 0x7fc288197bc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc27800e970 tx=0x7fc27800ec80 comp rx=0 tx=0).stop 2023-07-14T16:24:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 -- 172.21.15.62:0/371162175 shutdown_connections 2023-07-14T16:24:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc288105860 0x7fc288198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc288100620 0x7fc288197bc0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc2700a1150 0x7fc2700a3600 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 --2- 172.21.15.62:0/371162175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc288102f40 0x7fc288198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.614+0000 7fc28cfff700 1 -- 172.21.15.62:0/371162175 >> 172.21.15.62:0/371162175 conn(0x7fc2880fa000 msgr2=0x7fc2880feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.615+0000 7fc28cfff700 1 -- 172.21.15.62:0/371162175 shutdown_connections 2023-07-14T16:24:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:25.615+0000 7fc28cfff700 1 -- 172.21.15.62:0/371162175 wait complete. 2023-07-14T16:24:27.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:26 smithi164 ceph-mon[97427]: pgmap v621: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 34 op/s 2023-07-14T16:24:27.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:26 smithi062 ceph-mon[95543]: pgmap v621: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 34 op/s 2023-07-14T16:24:27.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:26 smithi081 ceph-mon[127795]: pgmap v621: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 34 op/s 2023-07-14T16:24:29.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:28 smithi164 ceph-mon[97427]: pgmap v622: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 28 op/s 2023-07-14T16:24:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:28 smithi062 ceph-mon[95543]: pgmap v622: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 28 op/s 2023-07-14T16:24:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:28 smithi081 ceph-mon[127795]: pgmap v622: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 28 op/s 2023-07-14T16:24:31.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:30 smithi164 ceph-mon[97427]: pgmap v623: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:24:31.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:30 smithi062 ceph-mon[95543]: pgmap v623: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:24:31.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:30 smithi081 ceph-mon[127795]: pgmap v623: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:24:33.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:32 smithi164 ceph-mon[97427]: pgmap v624: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:24:33.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:32 smithi062 ceph-mon[95543]: pgmap v624: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:24:33.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:32 smithi081 ceph-mon[127795]: pgmap v624: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:24:35.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:34 smithi164 ceph-mon[97427]: pgmap v625: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:24:35.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:34 smithi062 ceph-mon[95543]: pgmap v625: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:24:35.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:34 smithi081 ceph-mon[127795]: pgmap v625: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:24:37.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:36 smithi164 ceph-mon[97427]: pgmap v626: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:37.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:36 smithi062 ceph-mon[95543]: pgmap v626: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:37.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:36 smithi081 ceph-mon[127795]: pgmap v626: 97 pgs: 97 active+clean; 6.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:24:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.785+0000 7f7948fff700 1 -- 172.21.15.62:0/1172184357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 msgr2=0x7f7944107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.785+0000 7f7948fff700 1 --2- 172.21.15.62:0/1172184357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 0x7f7944107c40 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f793c009ec0 tx=0x7f793c00b330 comp rx=0 tx=0).stop 2023-07-14T16:24:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.786+0000 7f7948fff700 1 -- 172.21.15.62:0/1172184357 shutdown_connections 2023-07-14T16:24:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.786+0000 7f7948fff700 1 --2- 172.21.15.62:0/1172184357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 0x7f7944107c40 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.786+0000 7f7948fff700 1 --2- 172.21.15.62:0/1172184357 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7944100620 0x7f7944102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.786+0000 7f7948fff700 1 --2- 172.21.15.62:0/1172184357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7944102f40 0x7f7944105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.786+0000 7f7948fff700 1 -- 172.21.15.62:0/1172184357 >> 172.21.15.62:0/1172184357 conn(0x7f79440fa000 msgr2=0x7f79440fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.786+0000 7f7948fff700 1 -- 172.21.15.62:0/1172184357 shutdown_connections 2023-07-14T16:24:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.786+0000 7f7948fff700 1 -- 172.21.15.62:0/1172184357 wait complete. 2023-07-14T16:24:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.787+0000 7f7948fff700 1 Processor -- start 2023-07-14T16:24:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.787+0000 7f7948fff700 1 -- start start 2023-07-14T16:24:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7948fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7944100620 0x7f79441959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7948fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7944102f40 0x7f7944195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7948fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 0x7f7944196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7948fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441973e0 con 0x7f7944105860 2023-07-14T16:24:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7948fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944197550 con 0x7f7944100620 2023-07-14T16:24:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7948fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944197710 con 0x7f7944102f40 2023-07-14T16:24:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7942d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 0x7f7944196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7942d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 0x7f7944196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48166/0 (socket says 172.21.15.62:48166) 2023-07-14T16:24:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f794259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7944100620 0x7f79441959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7942d9d700 1 -- 172.21.15.62:0/3912167137 learned_addr learned my addr 172.21.15.62:0/3912167137 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.788+0000 7f7941d9b700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7944102f40 0x7f7944195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f7942d9d700 1 -- 172.21.15.62:0/3912167137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7944102f40 msgr2=0x7f7944195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f7942d9d700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7944102f40 0x7f7944195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f7942d9d700 1 -- 172.21.15.62:0/3912167137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7944100620 msgr2=0x7f79441959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f7942d9d700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7944100620 0x7f79441959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f7942d9d700 1 -- 172.21.15.62:0/3912167137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c005e70 con 0x7f7944105860 2023-07-14T16:24:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f794259c700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7944100620 0x7f79441959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f7942d9d700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 0x7f7944196d80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f793c0095d0 tx=0x7f793c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f79337fe700 1 -- 172.21.15.62:0/3912167137 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c00e040 con 0x7f7944105860 2023-07-14T16:24:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f79337fe700 1 -- 172.21.15.62:0/3912167137 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f793c01d910 con 0x7f7944105860 2023-07-14T16:24:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f7948fff700 1 -- 172.21.15.62:0/3912167137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7944197990 con 0x7f7944105860 2023-07-14T16:24:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f79337fe700 1 -- 172.21.15.62:0/3912167137 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c01b9e0 con 0x7f7944105860 2023-07-14T16:24:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.789+0000 7f7948fff700 1 -- 172.21.15.62:0/3912167137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79440fe5f0 con 0x7f7944105860 2023-07-14T16:24:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.793+0000 7f79337fe700 1 -- 172.21.15.62:0/3912167137 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f793c01da80 con 0x7f7944105860 2023-07-14T16:24:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.794+0000 7f795c969b80 1 Processor -- start 2023-07-14T16:24:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.795+0000 7f795c969b80 1 -- start start 2023-07-14T16:24:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.795+0000 7f795c969b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa762300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.795+0000 7f795c969b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa762840 0x55befa7631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.795+0000 7f795c969b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 0x55befa763b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.795+0000 7f795c969b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa667c50 con 0x55befa5be6b0 2023-07-14T16:24:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.795+0000 7f795c969b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa6ba9e0 con 0x55befa7636e0 2023-07-14T16:24:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.795+0000 7f795c969b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa6b3e70 con 0x55befa762840 2023-07-14T16:24:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.795+0000 7f79337fe700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f792c0a1150 0x7f792c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa762300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f79337fe700 1 -- 172.21.15.62:0/3912167137 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f793c0ce620 con 0x7f7944105860 2023-07-14T16:24:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa762300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48178/0 (socket says 172.21.15.62:48178) 2023-07-14T16:24:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 -- 172.21.15.62:0/2771334947 learned_addr learned my addr 172.21.15.62:0/2771334947 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 0x55befa763b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f794259c700 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa762840 0x55befa7631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7942d9d700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f792c0a1150 0x7f792c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 -- 172.21.15.62:0/2771334947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa762840 msgr2=0x55befa7631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa762840 0x55befa7631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 -- 172.21.15.62:0/2771334947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 msgr2=0x55befa763b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 0x55befa763b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.796+0000 7f7941d9b700 1 -- 172.21.15.62:0/2771334947 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55befa4e1d80 con 0x55befa5be6b0 2023-07-14T16:24:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.797+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa762300 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f79380138f0 tx=0x7f7938009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dddb92a37fd624 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.797+0000 7f7942d9d700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f792c0a1150 0x7f792c0a3600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f793c09b770 tx=0x7f793c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.797+0000 7f7930ff9700 1 -- 172.21.15.62:0/2771334947 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938004830 con 0x55befa5be6b0 2023-07-14T16:24:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.797+0000 7f7930ff9700 1 -- 172.21.15.62:0/2771334947 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7938004e20 con 0x55befa5be6b0 2023-07-14T16:24:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.797+0000 7f7930ff9700 1 -- 172.21.15.62:0/2771334947 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938010460 con 0x55befa5be6b0 2023-07-14T16:24:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 -- 172.21.15.62:0/2771334947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 msgr2=0x55befa762300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa762300 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f79380138f0 tx=0x7f7938009850 comp rx=0 tx=0).stop 2023-07-14T16:24:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 -- 172.21.15.62:0/2771334947 shutdown_connections 2023-07-14T16:24:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 0x55befa763b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa762300 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 --2- 172.21.15.62:0/2771334947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa762840 0x55befa7631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 -- 172.21.15.62:0/2771334947 >> 172.21.15.62:0/2771334947 conn(0x55befa757be0 msgr2=0x55befa75ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 -- 172.21.15.62:0/2771334947 shutdown_connections 2023-07-14T16:24:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.798+0000 7f795c969b80 1 -- 172.21.15.62:0/2771334947 wait complete. 2023-07-14T16:24:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.799+0000 7f795c969b80 1 -- start start 2023-07-14T16:24:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.799+0000 7f795c969b80 1 Processor -- start 2023-07-14T16:24:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f795c969b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa5be6b0 0x55befa76b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f795c969b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa762840 0x55befa76bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f795c969b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 0x55befa772d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f795c969b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa653a60 con 0x55befa762840 2023-07-14T16:24:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f795c969b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa653760 con 0x55befa7636e0 2023-07-14T16:24:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f795c969b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa5378c0 con 0x55befa5be6b0 2023-07-14T16:24:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f795c969b80 10 client.0 _ll_register_callbacks cb 0x55befa651700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f794259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa762840 0x55befa76bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.800+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa5be6b0 0x55befa76b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f794259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa762840 0x55befa76bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48192/0 (socket says 172.21.15.62:48192) 2023-07-14T16:24:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa5be6b0 0x55befa76b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55512/0 (socket says 172.21.15.62:55512) 2023-07-14T16:24:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f794259c700 1 -- 172.21.15.62:0/2634171709 learned_addr learned my addr 172.21.15.62:0/2634171709 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 0x55befa772d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f7941d9b700 1 -- 172.21.15.62:0/2634171709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 msgr2=0x55befa772d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 0x55befa772d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f7941d9b700 1 -- 172.21.15.62:0/2634171709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa762840 msgr2=0x55befa76bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa762840 0x55befa76bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f7941d9b700 1 -- 172.21.15.62:0/2634171709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793801b040 con 0x55befa5be6b0 2023-07-14T16:24:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f794259c700 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa762840 0x55befa76bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.801+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa5be6b0 0x55befa76b800 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55befa762060 tx=0x7f7938015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cdb011b68dfc3b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.802+0000 7f7929ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:24:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.802+0000 7f7929ffb700 1 -- 172.21.15.62:0/2634171709 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938004260 con 0x55befa5be6b0 2023-07-14T16:24:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.802+0000 7f7929ffb700 1 -- 172.21.15.62:0/2634171709 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f79380043c0 con 0x55befa5be6b0 2023-07-14T16:24:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.802+0000 7f795c969b80 1 -- 172.21.15.62:0/2634171709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55befa60d490 con 0x55befa5be6b0 2023-07-14T16:24:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.802+0000 7f7929ffb700 1 -- 172.21.15.62:0/2634171709 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7938004580 con 0x55befa5be6b0 2023-07-14T16:24:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.802+0000 7f795c969b80 10 client.27526 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.802+0000 7f795c969b80 1 -- 172.21.15.62:0/2634171709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55befa4cac00 con 0x55befa5be6b0 2023-07-14T16:24:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.803+0000 7f7929ffb700 1 -- 172.21.15.62:0/2634171709 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55befa60d490 con 0x55befa5be6b0 2023-07-14T16:24:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.803+0000 7f7929ffb700 1 -- 172.21.15.62:0/2634171709 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7938003a40 con 0x55befa5be6b0 2023-07-14T16:24:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.803+0000 7f795c969b80 10 client.27526 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.803+0000 7f795c969b80 10 client.27526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.803+0000 7f795c969b80 20 client.27526 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.803+0000 7f795c969b80 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55befa777480 0x55befa779890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.803+0000 7f795c969b80 4 client.27526 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.803+0000 7f795c969b80 1 -- 172.21.15.62:0/2634171709 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55befa6ace50 con 0x55befa777480 2023-07-14T16:24:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.804+0000 7f794259c700 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55befa777480 0x55befa779890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.804+0000 7f794259c700 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55befa777480 0x55befa779890 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d8aeb37c0bc741 server_cookie=795d21693d0669bb in_seq=0 out_seq=0 2023-07-14T16:24:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.804+0000 7f7929ffb700 10 client.27526 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f7929ffb700 1 -- 172.21.15.62:0/2634171709 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55befa6ace50 con 0x55befa777480 2023-07-14T16:24:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f7929ffb700 10 client.27526 handle_command_reply: tid=0 2023-07-14T16:24:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 client.27526 shutdown 2023-07-14T16:24:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 -- 172.21.15.62:0/2634171709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa5be6b0 msgr2=0x55befa76b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa5be6b0 0x55befa76b800 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55befa762060 tx=0x7f7938015040 comp rx=0 tx=0).stop 2023-07-14T16:24:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 -- 172.21.15.62:0/2634171709 shutdown_connections 2023-07-14T16:24:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55befa777480 0x55befa779890 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa7636e0 0x55befa772d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa5be6b0 0x55befa76b800 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 --2- 172.21.15.62:0/2634171709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa762840 0x55befa76bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f7929ffb700 0 client.27526 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 -- 172.21.15.62:0/2634171709 >> 172.21.15.62:0/2634171709 conn(0x55befa757be0 msgr2=0x55befa75c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 -- 172.21.15.62:0/2634171709 shutdown_connections 2023-07-14T16:24:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.805+0000 7f795c969b80 1 -- 172.21.15.62:0/2634171709 wait complete. 2023-07-14T16:24:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.806+0000 7f795c969b80 20 client.27526 trim_cache size 0 max 16384 2023-07-14T16:24:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.825+0000 7f795c969b80 1 Processor -- start 2023-07-14T16:24:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.825+0000 7f795c969b80 1 -- start start 2023-07-14T16:24:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.825+0000 7f795c969b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa76aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.825+0000 7f795c969b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa772d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f795c969b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 0x55befa75cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f795c969b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa75da20 con 0x55befa5be6b0 2023-07-14T16:24:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f795c969b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa76c980 con 0x55befa762840 2023-07-14T16:24:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f795c969b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa75d1c0 con 0x55befa7636e0 2023-07-14T16:24:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa76aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa76aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48200/0 (socket says 172.21.15.62:48200) 2023-07-14T16:24:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 -- 172.21.15.62:0/3498616677 learned_addr learned my addr 172.21.15.62:0/3498616677 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f794259c700 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa772d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 0x55befa75cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 -- 172.21.15.62:0/3498616677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 msgr2=0x55befa75cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 0x55befa75cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 -- 172.21.15.62:0/3498616677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 msgr2=0x55befa772d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa772d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f7941d9b700 1 -- 172.21.15.62:0/3498616677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55befa5081f0 con 0x55befa5be6b0 2023-07-14T16:24:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.826+0000 7f794259c700 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa772d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.827+0000 7f7941d9b700 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa76aeb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f79380138f0 tx=0x7f7938005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7dc96df8758a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.827+0000 7f79317fa700 1 -- 172.21.15.62:0/3498616677 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938013580 con 0x55befa5be6b0 2023-07-14T16:24:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.827+0000 7f79317fa700 1 -- 172.21.15.62:0/3498616677 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7938017020 con 0x55befa5be6b0 2023-07-14T16:24:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.827+0000 7f79317fa700 1 -- 172.21.15.62:0/3498616677 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938003c00 con 0x55befa5be6b0 2023-07-14T16:24:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 -- 172.21.15.62:0/3498616677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 msgr2=0x55befa76aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa76aeb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f79380138f0 tx=0x7f7938005850 comp rx=0 tx=0).stop 2023-07-14T16:24:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 -- 172.21.15.62:0/3498616677 shutdown_connections 2023-07-14T16:24:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 0x55befa75cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa76aeb0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 --2- 172.21.15.62:0/3498616677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa772d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 -- 172.21.15.62:0/3498616677 >> 172.21.15.62:0/3498616677 conn(0x55befa757be0 msgr2=0x55befa76a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 -- 172.21.15.62:0/3498616677 shutdown_connections 2023-07-14T16:24:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 -- 172.21.15.62:0/3498616677 wait complete. 2023-07-14T16:24:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.828+0000 7f795c969b80 1 -- start start 2023-07-14T16:24:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f795c969b80 1 Processor -- start 2023-07-14T16:24:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f795c969b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa770d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f795c969b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa7c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f795c969b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 0x55befa7c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f795c969b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa7c9eb0 con 0x55befa5be6b0 2023-07-14T16:24:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f795c969b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa7ca020 con 0x55befa762840 2023-07-14T16:24:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f795c969b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55befa7ca1e0 con 0x55befa7636e0 2023-07-14T16:24:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f795c969b80 10 client.0 _ll_register_callbacks cb 0x7f7938010460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.829+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa770d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f7941d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa770d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48212/0 (socket says 172.21.15.62:48212) 2023-07-14T16:24:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f794259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa7c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f7941d9b700 1 -- 172.21.15.62:0/2994102519 learned_addr learned my addr 172.21.15.62:0/2994102519 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 0x55befa7c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f794259c700 1 -- 172.21.15.62:0/2994102519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 msgr2=0x55befa7c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f794259c700 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 0x55befa7c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f794259c700 1 -- 172.21.15.62:0/2994102519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 msgr2=0x55befa770d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f794259c700 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa770d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f794259c700 1 -- 172.21.15.62:0/2994102519 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55befa4cac00 con 0x55befa762840 2023-07-14T16:24:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f794259c700 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa7c7310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55befa779e70 tx=0x7f7934012210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936f84b92f296b40 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.830+0000 7f7929ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:24:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f7929ffb700 1 -- 172.21.15.62:0/2994102519 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934017030 con 0x55befa762840 2023-07-14T16:24:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f7929ffb700 1 -- 172.21.15.62:0/2994102519 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7934021050 con 0x55befa762840 2023-07-14T16:24:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f795c969b80 1 -- 172.21.15.62:0/2994102519 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7938004e20 con 0x55befa762840 2023-07-14T16:24:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f7929ffb700 1 -- 172.21.15.62:0/2994102519 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7934021e60 con 0x55befa762840 2023-07-14T16:24:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f795c969b80 10 client.27635 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f795c969b80 1 -- 172.21.15.62:0/2994102519 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55befa4e0250 con 0x55befa762840 2023-07-14T16:24:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f7929ffb700 1 -- 172.21.15.62:0/2994102519 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7938004e20 con 0x55befa762840 2023-07-14T16:24:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f7929ffb700 1 -- 172.21.15.62:0/2994102519 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7934027020 con 0x55befa762840 2023-07-14T16:24:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f795c969b80 10 client.27635 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f795c969b80 10 client.27635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.831+0000 7f795c969b80 20 client.27635 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.832+0000 7f795c969b80 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55befa7cbbf0 0x55befa7ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.832+0000 7f795c969b80 4 client.27635 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:24:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.832+0000 7f795c969b80 1 -- 172.21.15.62:0/2994102519 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55befa66eff0 con 0x55befa7cbbf0 2023-07-14T16:24:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.832+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55befa7cbbf0 0x55befa7ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.832+0000 7f7941d9b700 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55befa7cbbf0 0x55befa7ce000 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7baa1e0dcc8fa124 server_cookie=b24de4a0e6853da0 in_seq=0 out_seq=0 2023-07-14T16:24:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.832+0000 7f7929ffb700 10 client.27635 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.841+0000 7f7929ffb700 1 -- 172.21.15.62:0/2994102519 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55befa66eff0 con 0x55befa7cbbf0 2023-07-14T16:24:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.841+0000 7f7929ffb700 10 client.27635 handle_command_reply: tid=0 2023-07-14T16:24:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.841+0000 7f795c969b80 1 client.27635 shutdown 2023-07-14T16:24:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 -- 172.21.15.62:0/2994102519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 msgr2=0x55befa7c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa7c7310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55befa779e70 tx=0x7f7934012210 comp rx=0 tx=0).stop 2023-07-14T16:24:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 -- 172.21.15.62:0/2994102519 shutdown_connections 2023-07-14T16:24:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55befa7636e0 0x55befa7c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befa5be6b0 0x55befa770d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55befa7cbbf0 0x55befa7ce000 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 --2- 172.21.15.62:0/2994102519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55befa762840 0x55befa7c7310 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f7929ffb700 0 client.27635 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 -- 172.21.15.62:0/2994102519 >> 172.21.15.62:0/2994102519 conn(0x55befa757be0 msgr2=0x55befa76a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 -- 172.21.15.62:0/2994102519 shutdown_connections 2023-07-14T16:24:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 1 -- 172.21.15.62:0/2994102519 wait complete. 2023-07-14T16:24:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.842+0000 7f795c969b80 20 client.27635 trim_cache size 0 max 16384 2023-07-14T16:24:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:24:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:24:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:24:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:24:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 -- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f792c0a1150 msgr2=0x7f792c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f792c0a1150 0x7f792c0a3600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f793c09b770 tx=0x7f793c009040 comp rx=0 tx=0).stop 2023-07-14T16:24:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 -- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 msgr2=0x7f7944196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 0x7f7944196d80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f793c0095d0 tx=0x7f793c00c9e0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 -- 172.21.15.62:0/3912167137 shutdown_connections 2023-07-14T16:24:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7944105860 0x7f7944196d80 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f792c0a1150 0x7f792c0a3600 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7944100620 0x7f79441959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 --2- 172.21.15.62:0/3912167137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7944102f40 0x7f7944195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 -- 172.21.15.62:0/3912167137 >> 172.21.15.62:0/3912167137 conn(0x7f79440fa000 msgr2=0x7f7944101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 -- 172.21.15.62:0/3912167137 shutdown_connections 2023-07-14T16:24:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:38.843+0000 7f7948fff700 1 -- 172.21.15.62:0/3912167137 wait complete. 2023-07-14T16:24:39.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:38 smithi062 ceph-mon[95543]: pgmap v627: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:24:39.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:38 smithi081 ceph-mon[127795]: pgmap v627: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:24:39.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:38 smithi164 ceph-mon[97427]: pgmap v627: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:24:41.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:40 smithi062 ceph-mon[95543]: pgmap v628: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:41.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:40 smithi081 ceph-mon[127795]: pgmap v628: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:41.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:40 smithi164 ceph-mon[97427]: pgmap v628: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:43.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:42 smithi062 ceph-mon[95543]: pgmap v629: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:24:43.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:42 smithi081 ceph-mon[127795]: pgmap v629: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:24:43.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:42 smithi164 ceph-mon[97427]: pgmap v629: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:24:43.982 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:24:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 -- 172.21.15.62:0/2102334296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 msgr2=0x7f3918107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 --2- 172.21.15.62:0/2102334296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 0x7f3918107f20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f391000b0a0 tx=0x7f391000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 -- 172.21.15.62:0/2102334296 shutdown_connections 2023-07-14T16:24:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 --2- 172.21.15.62:0/2102334296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 0x7f3918107f20 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 --2- 172.21.15.62:0/2102334296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39180ffae0 0x7f39180ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 --2- 172.21.15.62:0/2102334296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3918100cd0 0x7f3918101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 -- 172.21.15.62:0/2102334296 >> 172.21.15.62:0/2102334296 conn(0x7f39180fb0e0 msgr2=0x7f39180fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 -- 172.21.15.62:0/2102334296 shutdown_connections 2023-07-14T16:24:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.262+0000 7f391ddff700 1 -- 172.21.15.62:0/2102334296 wait complete. 2023-07-14T16:24:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.264+0000 7f391ddff700 1 Processor -- start 2023-07-14T16:24:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.264+0000 7f391ddff700 1 -- start start 2023-07-14T16:24:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.264+0000 7f391ddff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39180ffae0 0x7f39180742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.264+0000 7f391ddff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3918100cd0 0x7f3918072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.264+0000 7f391ddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 0x7f3918072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.264+0000 7f391ddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39180735e0 con 0x7f3918101660 2023-07-14T16:24:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.264+0000 7f391ddff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3918073750 con 0x7f3918100cd0 2023-07-14T16:24:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.264+0000 7f391ddff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391819f940 con 0x7f39180ffae0 2023-07-14T16:24:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f39177fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39180ffae0 0x7f39180742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f3917fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 0x7f3918072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f39177fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39180ffae0 0x7f39180742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46542/0 (socket says 172.21.15.62:46542) 2023-07-14T16:24:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f39177fe700 1 -- 172.21.15.62:0/2721735767 learned_addr learned my addr 172.21.15.62:0/2721735767 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f3916ffd700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3918100cd0 0x7f3918072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f3917fff700 1 -- 172.21.15.62:0/2721735767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39180ffae0 msgr2=0x7f39180742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f3917fff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39180ffae0 0x7f39180742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f3917fff700 1 -- 172.21.15.62:0/2721735767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3918100cd0 msgr2=0x7f3918072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f3917fff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3918100cd0 0x7f3918072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f3917fff700 1 -- 172.21.15.62:0/2721735767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910009d00 con 0x7f3918101660 2023-07-14T16:24:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.265+0000 7f3917fff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 0x7f3918072e40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3910009fd0 tx=0x7f3910004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:44.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.266+0000 7f3914ff9700 1 -- 172.21.15.62:0/2721735767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391000e040 con 0x7f3918101660 2023-07-14T16:24:44.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.266+0000 7f391ddff700 1 -- 172.21.15.62:0/2721735767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391819fbc0 con 0x7f3918101660 2023-07-14T16:24:44.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.266+0000 7f3914ff9700 1 -- 172.21.15.62:0/2721735767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39100056e0 con 0x7f3918101660 2023-07-14T16:24:44.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.266+0000 7f3914ff9700 1 -- 172.21.15.62:0/2721735767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391001da70 con 0x7f3918101660 2023-07-14T16:24:44.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.266+0000 7f391ddff700 1 -- 172.21.15.62:0/2721735767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391819ff90 con 0x7f3918101660 2023-07-14T16:24:44.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.270+0000 7f3914ff9700 1 -- 172.21.15.62:0/2721735767 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3910005850 con 0x7f3918101660 2023-07-14T16:24:44.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.271+0000 7f39316acb80 1 Processor -- start 2023-07-14T16:24:44.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.271+0000 7f39316acb80 1 -- start start 2023-07-14T16:24:44.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.271+0000 7f39316acb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 0x557c0df848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.271+0000 7f39316acb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df84e20 0x557c0df856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.271+0000 7f39316acb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.271+0000 7f39316acb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0de89cb0 con 0x557c0df85c70 2023-07-14T16:24:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.271+0000 7f39316acb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0dedca40 con 0x557c0df84e20 2023-07-14T16:24:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.271+0000 7f39316acb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0ded5ed0 con 0x557c0df844d0 2023-07-14T16:24:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f39177fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df84e20 0x557c0df856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 0x557c0df848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f39177fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df84e20 0x557c0df856e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52524/0 (socket says 172.21.15.62:52524) 2023-07-14T16:24:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f39177fe700 1 -- 172.21.15.62:0/1100865655 learned_addr learned my addr 172.21.15.62:0/1100865655 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f3916ffd700 1 --2- 172.21.15.62:0/1100865655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f3914ff9700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f39000a1150 0x7f39000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f39177fe700 1 -- 172.21.15.62:0/1100865655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 msgr2=0x557c0df848e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:44.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f39177fe700 1 --2- 172.21.15.62:0/1100865655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 0x557c0df848e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f3914ff9700 1 -- 172.21.15.62:0/2721735767 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f39100ce350 con 0x7f3918101660 2023-07-14T16:24:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f39177fe700 1 -- 172.21.15.62:0/1100865655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 msgr2=0x557c0df860e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f39177fe700 1 --2- 172.21.15.62:0/1100865655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df860e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.272+0000 7f39177fe700 1 -- 172.21.15.62:0/1100865655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c0dd03de0 con 0x557c0df84e20 2023-07-14T16:24:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.273+0000 7f3917fff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f39000a1150 0x7f39000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.273+0000 7f39177fe700 1 --2- 172.21.15.62:0/1100865655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df84e20 0x557c0df856e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f390800bea0 tx=0x7f39080106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7bc9af29fd1ab41 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.273+0000 7f38fdffb700 1 -- 172.21.15.62:0/1100865655 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908010e10 con 0x557c0df84e20 2023-07-14T16:24:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.273+0000 7f38fdffb700 1 -- 172.21.15.62:0/1100865655 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3908015070 con 0x557c0df84e20 2023-07-14T16:24:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.273+0000 7f38fdffb700 1 -- 172.21.15.62:0/1100865655 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908003da0 con 0x557c0df84e20 2023-07-14T16:24:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 -- 172.21.15.62:0/1100865655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df84e20 msgr2=0x557c0df856e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 --2- 172.21.15.62:0/1100865655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df84e20 0x557c0df856e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f390800bea0 tx=0x7f39080106b0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f3917fff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f39000a1150 0x7f39000a3600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3910098270 tx=0x7f3910009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 -- 172.21.15.62:0/1100865655 shutdown_connections 2023-07-14T16:24:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 --2- 172.21.15.62:0/1100865655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 --2- 172.21.15.62:0/1100865655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 0x557c0df848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 --2- 172.21.15.62:0/1100865655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df84e20 0x557c0df856e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 -- 172.21.15.62:0/1100865655 >> 172.21.15.62:0/1100865655 conn(0x557c0df79da0 msgr2=0x557c0df7f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:44.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 -- 172.21.15.62:0/1100865655 shutdown_connections 2023-07-14T16:24:44.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.274+0000 7f39316acb80 1 -- 172.21.15.62:0/1100865655 wait complete. 2023-07-14T16:24:44.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 1 -- start start 2023-07-14T16:24:44.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 1 Processor -- start 2023-07-14T16:24:44.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df8e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 0x557c0df8e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0de75ac0 con 0x557c0df85c70 2023-07-14T16:24:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0de757c0 con 0x557c0df844d0 2023-07-14T16:24:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0dd59920 con 0x557c0df84e20 2023-07-14T16:24:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.275+0000 7f39316acb80 10 client.0 _ll_register_callbacks cb 0x557c0de73760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df8e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df956e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55040/0 (socket says 172.21.15.62:55040) 2023-07-14T16:24:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 -- 172.21.15.62:0/2971404559 learned_addr learned my addr 172.21.15.62:0/2971404559 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f39177fe700 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 0x557c0df8e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 -- 172.21.15.62:0/2971404559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 msgr2=0x557c0df8e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 0x557c0df8e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 -- 172.21.15.62:0/2971404559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 msgr2=0x557c0df8e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df8e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 -- 172.21.15.62:0/2971404559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390800bad0 con 0x557c0df85c70 2023-07-14T16:24:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f3916ffd700 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df956e0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f390c012f40 tx=0x7f390c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e929236575d68c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.276+0000 7f38f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.277+0000 7f38f6ffd700 1 -- 172.21.15.62:0/2971404559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c010ce0 con 0x557c0df85c70 2023-07-14T16:24:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.277+0000 7f39316acb80 1 -- 172.21.15.62:0/2971404559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c0de2f4f0 con 0x557c0df85c70 2023-07-14T16:24:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.277+0000 7f38f6ffd700 1 -- 172.21.15.62:0/2971404559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f390c010e40 con 0x557c0df85c70 2023-07-14T16:24:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.277+0000 7f38f6ffd700 1 -- 172.21.15.62:0/2971404559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f390c027030 con 0x557c0df85c70 2023-07-14T16:24:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.277+0000 7f39316acb80 10 client.18012 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.277+0000 7f39316acb80 1 -- 172.21.15.62:0/2971404559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c0dcecc60 con 0x557c0df85c70 2023-07-14T16:24:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.277+0000 7f38f6ffd700 1 -- 172.21.15.62:0/2971404559 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c0de2f4f0 con 0x557c0df85c70 2023-07-14T16:24:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.278+0000 7f38f6ffd700 1 -- 172.21.15.62:0/2971404559 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f390c003ae0 con 0x557c0df85c70 2023-07-14T16:24:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.278+0000 7f39316acb80 10 client.18012 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.278+0000 7f39316acb80 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.278+0000 7f39316acb80 20 client.18012 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.278+0000 7f39316acb80 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557c0df99f50 0x557c0df9c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.278+0000 7f39316acb80 4 client.18012 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.278+0000 7f39316acb80 1 -- 172.21.15.62:0/2971404559 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c0deceeb0 con 0x557c0df99f50 2023-07-14T16:24:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.278+0000 7f39177fe700 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557c0df99f50 0x557c0df9c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.279+0000 7f39177fe700 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557c0df99f50 0x557c0df9c360 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8ccca1436d9f59 server_cookie=1252f37cbe93ad95 in_seq=0 out_seq=0 2023-07-14T16:24:44.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.279+0000 7f38f6ffd700 10 client.18012 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:44.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f38f6ffd700 1 -- 172.21.15.62:0/2971404559 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c0deceeb0 con 0x557c0df99f50 2023-07-14T16:24:44.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f38f6ffd700 10 client.18012 handle_command_reply: tid=0 2023-07-14T16:24:44.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 client.18012 shutdown 2023-07-14T16:24:44.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 -- 172.21.15.62:0/2971404559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 msgr2=0x557c0df956e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df956e0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f390c012f40 tx=0x7f390c0047c0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 -- 172.21.15.62:0/2971404559 shutdown_connections 2023-07-14T16:24:44.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df85c70 0x557c0df956e0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557c0df99f50 0x557c0df9c360 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df8e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 --2- 172.21.15.62:0/2971404559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 0x557c0df8e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f38f6ffd700 0 client.18012 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.280+0000 7f39316acb80 1 -- 172.21.15.62:0/2971404559 >> 172.21.15.62:0/2971404559 conn(0x557c0df79da0 msgr2=0x557c0df7e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.281+0000 7f39316acb80 1 -- 172.21.15.62:0/2971404559 shutdown_connections 2023-07-14T16:24:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.281+0000 7f39316acb80 1 -- 172.21.15.62:0/2971404559 wait complete. 2023-07-14T16:24:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.281+0000 7f39316acb80 20 client.18012 trim_cache size 0 max 16384 2023-07-14T16:24:44.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.298+0000 7f39316acb80 1 Processor -- start 2023-07-14T16:24:44.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.298+0000 7f39316acb80 1 -- start start 2023-07-14T16:24:44.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f39316acb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df79da0 0x557c0df8ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f39316acb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 0x557c0df95600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f39316acb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df84e20 0x557c0df7e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f39316acb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0df8d610 con 0x557c0df84e20 2023-07-14T16:24:44.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f39316acb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0df80a00 con 0x557c0df79da0 2023-07-14T16:24:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f39316acb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0df8ede0 con 0x557c0df844d0 2023-07-14T16:24:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df84e20 0x557c0df7e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df84e20 0x557c0df7e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55048/0 (socket says 172.21.15.62:55048) 2023-07-14T16:24:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f3916ffd700 1 -- 172.21.15.62:0/4049613779 learned_addr learned my addr 172.21.15.62:0/4049613779 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.299+0000 7f39177fe700 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 0x557c0df95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f3916ffd700 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df79da0 0x557c0df8ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f3916ffd700 1 -- 172.21.15.62:0/4049613779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 msgr2=0x557c0df95600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:44.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f3916ffd700 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 0x557c0df95600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f3916ffd700 1 -- 172.21.15.62:0/4049613779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df79da0 msgr2=0x557c0df8ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f3916ffd700 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df79da0 0x557c0df8ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f3916ffd700 1 -- 172.21.15.62:0/4049613779 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c0dd2a250 con 0x557c0df84e20 2023-07-14T16:24:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f3916ffd700 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df84e20 0x557c0df7e8b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b790 tx=0x7f390c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf588b739efc9e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f38fe7fc700 1 -- 172.21.15.62:0/4049613779 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c027070 con 0x557c0df84e20 2023-07-14T16:24:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f38fe7fc700 1 -- 172.21.15.62:0/4049613779 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f390c016070 con 0x557c0df84e20 2023-07-14T16:24:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.300+0000 7f38fe7fc700 1 -- 172.21.15.62:0/4049613779 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c010ce0 con 0x557c0df84e20 2023-07-14T16:24:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 -- 172.21.15.62:0/4049613779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df84e20 msgr2=0x557c0df7e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df84e20 0x557c0df7e8b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b790 tx=0x7f390c004750 comp rx=0 tx=0).stop 2023-07-14T16:24:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 -- 172.21.15.62:0/4049613779 shutdown_connections 2023-07-14T16:24:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df84e20 0x557c0df7e8b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df79da0 0x557c0df8ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 --2- 172.21.15.62:0/4049613779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df844d0 0x557c0df95600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 -- 172.21.15.62:0/4049613779 >> 172.21.15.62:0/4049613779 conn(0x557c0df7b600 msgr2=0x557c0dfa0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 -- 172.21.15.62:0/4049613779 shutdown_connections 2023-07-14T16:24:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.301+0000 7f39316acb80 1 -- 172.21.15.62:0/4049613779 wait complete. 2023-07-14T16:24:44.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.302+0000 7f39316acb80 1 -- start start 2023-07-14T16:24:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.302+0000 7f39316acb80 1 Processor -- start 2023-07-14T16:24:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f39316acb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df79da0 0x557c0df95500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f39316acb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df91fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f39316acb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 0x557c0df92500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f39316acb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0df92c10 con 0x557c0df79da0 2023-07-14T16:24:44.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f39316acb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0dfede30 con 0x557c0df844d0 2023-07-14T16:24:44.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f39316acb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0dfedff0 con 0x557c0df84e20 2023-07-14T16:24:44.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f39316acb80 10 client.0 _ll_register_callbacks cb 0x7f390c012a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:44.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df79da0 0x557c0df95500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df79da0 0x557c0df95500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55060/0 (socket says 172.21.15.62:55060) 2023-07-14T16:24:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f3916ffd700 1 -- 172.21.15.62:0/4203707465 learned_addr learned my addr 172.21.15.62:0/4203707465 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f39177fe700 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df91fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f3916ffd700 1 -- 172.21.15.62:0/4203707465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 msgr2=0x557c0df92500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.303+0000 7f3916ffd700 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 0x557c0df92500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f3916ffd700 1 -- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 msgr2=0x557c0df91fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f3916ffd700 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df91fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f3916ffd700 1 -- 172.21.15.62:0/4203707465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c0dcecc60 con 0x557c0df79da0 2023-07-14T16:24:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f39177fe700 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df91fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f3916ffd700 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df79da0 0x557c0df95500 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f390c003ef0 tx=0x7f390c012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aeb4f1e919751b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f38f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f38f6ffd700 1 -- 172.21.15.62:0/4203707465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c027070 con 0x557c0df79da0 2023-07-14T16:24:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f38f6ffd700 1 -- 172.21.15.62:0/4203707465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f390c000e40 con 0x557c0df79da0 2023-07-14T16:24:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.304+0000 7f39316acb80 1 -- 172.21.15.62:0/4203707465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3908015070 con 0x557c0df79da0 2023-07-14T16:24:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f38f6ffd700 1 -- 172.21.15.62:0/4203707465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f390c0127a0 con 0x557c0df79da0 2023-07-14T16:24:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f39316acb80 10 client.18018 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f39316acb80 1 -- 172.21.15.62:0/4203707465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c0dd022b0 con 0x557c0df79da0 2023-07-14T16:24:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f38f6ffd700 1 -- 172.21.15.62:0/4203707465 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f390c046020 con 0x557c0df79da0 2023-07-14T16:24:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f38f6ffd700 1 -- 172.21.15.62:0/4203707465 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f390c0048e0 con 0x557c0df79da0 2023-07-14T16:24:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f39316acb80 10 client.18018 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f39316acb80 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f39316acb80 20 client.18018 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f39316acb80 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557c0df85c70 0x557c0dffa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f39316acb80 4 client.18018 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:24:44.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.305+0000 7f39316acb80 1 -- 172.21.15.62:0/4203707465 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c0de91050 con 0x557c0df85c70 2023-07-14T16:24:44.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.306+0000 7f39177fe700 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557c0df85c70 0x557c0dffa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.306+0000 7f39177fe700 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557c0df85c70 0x557c0dffa0a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3445dc3cdc23253 server_cookie=f077b94ec672c6a2 in_seq=0 out_seq=0 2023-07-14T16:24:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.306+0000 7f38f6ffd700 10 client.18018 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f38f6ffd700 1 -- 172.21.15.62:0/4203707465 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557c0de91050 con 0x557c0df85c70 2023-07-14T16:24:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f38f6ffd700 10 client.18018 handle_command_reply: tid=0 2023-07-14T16:24:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 client.18018 shutdown 2023-07-14T16:24:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 -- 172.21.15.62:0/4203707465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df79da0 msgr2=0x557c0df95500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df79da0 0x557c0df95500 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f390c003ef0 tx=0x7f390c012f50 comp rx=0 tx=0).stop 2023-07-14T16:24:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 -- 172.21.15.62:0/4203707465 shutdown_connections 2023-07-14T16:24:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557c0df84e20 0x557c0df92500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c0df79da0 0x557c0df95500 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557c0df85c70 0x557c0dffa0a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 --2- 172.21.15.62:0/4203707465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c0df844d0 0x557c0df91fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 -- 172.21.15.62:0/4203707465 >> 172.21.15.62:0/4203707465 conn(0x557c0df7b600 msgr2=0x557c0df7e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f38f6ffd700 0 client.18018 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:44.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 -- 172.21.15.62:0/4203707465 shutdown_connections 2023-07-14T16:24:44.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 1 -- 172.21.15.62:0/4203707465 wait complete. 2023-07-14T16:24:44.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.307+0000 7f39316acb80 20 client.18018 trim_cache size 0 max 16384 2023-07-14T16:24:44.289 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:24:44.289 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:24:44.289 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:24:44.290 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:24:44.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 -- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f39000a1150 msgr2=0x7f39000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f39000a1150 0x7f39000a3600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3910098270 tx=0x7f3910009040 comp rx=0 tx=0).stop 2023-07-14T16:24:44.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 -- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 msgr2=0x7f3918072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 0x7f3918072e40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3910009fd0 tx=0x7f3910004c80 comp rx=0 tx=0).stop 2023-07-14T16:24:44.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 -- 172.21.15.62:0/2721735767 shutdown_connections 2023-07-14T16:24:44.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3918101660 0x7f3918072e40 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f39180ffae0 0x7f39180742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f39000a1150 0x7f39000a3600 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 --2- 172.21.15.62:0/2721735767 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3918100cd0 0x7f3918072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 -- 172.21.15.62:0/2721735767 >> 172.21.15.62:0/2721735767 conn(0x7f39180fb0e0 msgr2=0x7f39181041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:44.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 -- 172.21.15.62:0/2721735767 shutdown_connections 2023-07-14T16:24:44.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.308+0000 7f391ddff700 1 -- 172.21.15.62:0/2721735767 wait complete. 2023-07-14T16:24:44.301 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:24:44.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:24:44.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 -- 172.21.15.62:0/1188948349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb4074550 msgr2=0x7f0fb40729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/1188948349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb4074550 0x7f0fb40729a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009990 tx=0x7f0fa4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 -- 172.21.15.62:0/1188948349 shutdown_connections 2023-07-14T16:24:44.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/1188948349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fb410c0d0 0x7f0fb410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/1188948349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb4074550 0x7f0fb40729a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/1188948349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb4072f70 0x7f0fb40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 -- 172.21.15.62:0/1188948349 >> 172.21.15.62:0/1188948349 conn(0x7f0fb40fb0e0 msgr2=0x7f0fb40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:44.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 -- 172.21.15.62:0/1188948349 shutdown_connections 2023-07-14T16:24:44.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.565+0000 7f0fb9bff700 1 -- 172.21.15.62:0/1188948349 wait complete. 2023-07-14T16:24:44.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.567+0000 7f0fb9bff700 1 Processor -- start 2023-07-14T16:24:44.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.567+0000 7f0fb9bff700 1 -- start start 2023-07-14T16:24:44.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.567+0000 7f0fb9bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fb4072f70 0x7f0fb419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.567+0000 7f0fb9bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb4074550 0x7f0fb419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.567+0000 7f0fb9bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb410c0d0 0x7f0fb41a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.567+0000 7f0fb9bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41a3960 con 0x7f0fb4074550 2023-07-14T16:24:44.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.567+0000 7f0fb9bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41a3aa0 con 0x7f0fb4072f70 2023-07-14T16:24:44.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.567+0000 7f0fb9bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41a3c30 con 0x7f0fb410c0d0 2023-07-14T16:24:44.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fb4072f70 0x7f0fb419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fb4072f70 0x7f0fb419bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52590/0 (socket says 172.21.15.62:52590) 2023-07-14T16:24:44.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb37fe700 1 -- 172.21.15.62:0/4263437904 learned_addr learned my addr 172.21.15.62:0/4263437904 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:44.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb3fff700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb410c0d0 0x7f0fb41a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb2ffd700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb4074550 0x7f0fb419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb37fe700 1 -- 172.21.15.62:0/4263437904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb410c0d0 msgr2=0x7f0fb41a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb37fe700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb410c0d0 0x7f0fb41a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb37fe700 1 -- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb4074550 msgr2=0x7f0fb419c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb37fe700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb4074550 0x7f0fb419c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.568+0000 7f0fb37fe700 1 -- 172.21.15.62:0/4263437904 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40095f0 con 0x7f0fb4072f70 2023-07-14T16:24:44.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.569+0000 7f0fb37fe700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fb4072f70 0x7f0fb419bde0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40098e0 tx=0x7f0fa4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:44.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.569+0000 7f0fb0ff9700 1 -- 172.21.15.62:0/4263437904 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa401d070 con 0x7f0fb4072f70 2023-07-14T16:24:44.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.569+0000 7f0fb0ff9700 1 -- 172.21.15.62:0/4263437904 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fa400be30 con 0x7f0fb4072f70 2023-07-14T16:24:44.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.569+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb41a3eb0 con 0x7f0fb4072f70 2023-07-14T16:24:44.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.569+0000 7f0fb0ff9700 1 -- 172.21.15.62:0/4263437904 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4022d80 con 0x7f0fb4072f70 2023-07-14T16:24:44.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.570+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb41a42f0 con 0x7f0fb4072f70 2023-07-14T16:24:44.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.571+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb4101c20 con 0x7f0fb4072f70 2023-07-14T16:24:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.573+0000 7f0fb0ff9700 1 -- 172.21.15.62:0/4263437904 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0fa4022470 con 0x7f0fb4072f70 2023-07-14T16:24:44.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.574+0000 7f0fb0ff9700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f9c0a1220 0x7f0f9c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:44.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.575+0000 7f0fb0ff9700 1 -- 172.21.15.62:0/4263437904 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0fa40cdd70 con 0x7f0fb4072f70 2023-07-14T16:24:44.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.575+0000 7f0fb2ffd700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f9c0a1220 0x7f0f9c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:44.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.575+0000 7f0fb2ffd700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f9c0a1220 0x7f0f9c0a36d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4074270 tx=0x7f0fa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:44.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.578+0000 7f0fb0ff9700 1 -- 172.21.15.62:0/4263437904 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0fa4097500 con 0x7f0fb4072f70 2023-07-14T16:24:44.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.859+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fb402c850 con 0x7f0fb4072f70 2023-07-14T16:24:44.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.860+0000 7f0fb0ff9700 1 -- 172.21.15.62:0/4263437904 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f0fa409ab20 con 0x7f0fb4072f70 2023-07-14T16:24:44.835 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:24:44.836 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:24:44.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f9c0a1220 msgr2=0x7f0f9c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f9c0a1220 0x7f0f9c0a36d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4074270 tx=0x7f0fa8009260 comp rx=0 tx=0).stop 2023-07-14T16:24:44.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fb4072f70 msgr2=0x7f0fb419bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:44.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fb4072f70 0x7f0fb419bde0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40098e0 tx=0x7f0fa4004730 comp rx=0 tx=0).stop 2023-07-14T16:24:44.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 shutdown_connections 2023-07-14T16:24:44.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fb410c0d0 0x7f0fb41a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f9c0a1220 0x7f0f9c0a36d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0fb4072f70 0x7f0fb419bde0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 --2- 172.21.15.62:0/4263437904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb4074550 0x7f0fb419c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:44.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 >> 172.21.15.62:0/4263437904 conn(0x7f0fb40fb0e0 msgr2=0x7f0fb40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:44.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 shutdown_connections 2023-07-14T16:24:44.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:44.862+0000 7f0fb9bff700 1 -- 172.21.15.62:0/4263437904 wait complete. 2023-07-14T16:24:44.839 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:24:44.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:24:45.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.104+0000 7f7b003ff700 1 -- 172.21.15.62:0/2409307023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af8102e80 msgr2=0x7f7af8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.104+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2409307023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af8102e80 0x7f7af8105260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098f0 tx=0x7f7ae8009c00 comp rx=0 tx=0).stop 2023-07-14T16:24:45.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.104+0000 7f7b003ff700 1 -- 172.21.15.62:0/2409307023 shutdown_connections 2023-07-14T16:24:45.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.104+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2409307023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af81057a0 0x7f7af8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.104+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2409307023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af8100560 0x7f7af8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.104+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2409307023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af8102e80 0x7f7af8105260 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.104+0000 7f7b003ff700 1 -- 172.21.15.62:0/2409307023 >> 172.21.15.62:0/2409307023 conn(0x7f7af80fa000 msgr2=0x7f7af80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.105+0000 7f7b003ff700 1 -- 172.21.15.62:0/2409307023 shutdown_connections 2023-07-14T16:24:45.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.105+0000 7f7b003ff700 1 -- 172.21.15.62:0/2409307023 wait complete. 2023-07-14T16:24:45.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.106+0000 7f7b003ff700 1 Processor -- start 2023-07-14T16:24:45.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.106+0000 7f7b003ff700 1 -- start start 2023-07-14T16:24:45.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.106+0000 7f7b003ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8100560 0x7f7af80ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.106+0000 7f7b003ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af8102e80 0x7f7af80fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.106+0000 7f7b003ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af81057a0 0x7f7af80fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.106+0000 7f7b003ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80ff1f0 con 0x7f7af8100560 2023-07-14T16:24:45.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.106+0000 7f7b003ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80ff360 con 0x7f7af8102e80 2023-07-14T16:24:45.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.106+0000 7f7b003ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af819f840 con 0x7f7af81057a0 2023-07-14T16:24:45.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8100560 0x7f7af80ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8100560 0x7f7af80ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55086/0 (socket says 172.21.15.62:55086) 2023-07-14T16:24:45.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 -- 172.21.15.62:0/2993109613 learned_addr learned my addr 172.21.15.62:0/2993109613 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe99c700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af81057a0 0x7f7af80fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afd99a700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af8102e80 0x7f7af80fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 -- 172.21.15.62:0/2993109613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af81057a0 msgr2=0x7f7af80fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af81057a0 0x7f7af80fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 -- 172.21.15.62:0/2993109613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af8102e80 msgr2=0x7f7af80fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af8102e80 0x7f7af80fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 -- 172.21.15.62:0/2993109613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8009520 con 0x7f7af8100560 2023-07-14T16:24:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe99c700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af81057a0 0x7f7af80fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.107+0000 7f7afe19b700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8100560 0x7f7af80ffed0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7af400d980 tx=0x7f7af400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.108+0000 7f7aef7fe700 1 -- 172.21.15.62:0/2993109613 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4004220 con 0x7f7af8100560 2023-07-14T16:24:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.108+0000 7f7b003ff700 1 -- 172.21.15.62:0/2993109613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af819fb20 con 0x7f7af8100560 2023-07-14T16:24:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.108+0000 7f7aef7fe700 1 -- 172.21.15.62:0/2993109613 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af4007640 con 0x7f7af8100560 2023-07-14T16:24:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.108+0000 7f7aef7fe700 1 -- 172.21.15.62:0/2993109613 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40055e0 con 0x7f7af8100560 2023-07-14T16:24:45.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.108+0000 7f7b003ff700 1 -- 172.21.15.62:0/2993109613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af81a0080 con 0x7f7af8100560 2023-07-14T16:24:45.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.112+0000 7f7aef7fe700 1 -- 172.21.15.62:0/2993109613 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7af4004750 con 0x7f7af8100560 2023-07-14T16:24:45.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.113+0000 7f7b13cd2b80 1 Processor -- start 2023-07-14T16:24:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.113+0000 7f7b13cd2b80 1 -- start start 2023-07-14T16:24:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fbc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fbd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fbe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886ec1cb0 con 0x563886fbdc70 2023-07-14T16:24:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886f14a40 con 0x563886fbce20 2023-07-14T16:24:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886f0ded0 con 0x563886fbc4d0 2023-07-14T16:24:45.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7afe19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fbd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7afe19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fbd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52608/0 (socket says 172.21.15.62:52608) 2023-07-14T16:24:45.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7afe19b700 1 -- 172.21.15.62:0/1875133014 learned_addr learned my addr 172.21.15.62:0/1875133014 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7afe99c700 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fbc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.114+0000 7f7afd99a700 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fbe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7afe19b700 1 -- 172.21.15.62:0/1875133014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 msgr2=0x563886fbc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7afe19b700 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fbc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7afe19b700 1 -- 172.21.15.62:0/1875133014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 msgr2=0x563886fbe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7afe19b700 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fbe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7afe19b700 1 -- 172.21.15.62:0/1875133014 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563886d3bde0 con 0x563886fbce20 2023-07-14T16:24:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7aef7fe700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7ae40a1150 0x7f7ae40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7aef7fe700 1 -- 172.21.15.62:0/2993109613 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7af40cc790 con 0x7f7af8100560 2023-07-14T16:24:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7afe19b700 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fbd6e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7af40d2a80 tx=0x7f7af40d2d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84be890e5f381bc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.115+0000 7f7afe99c700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7ae40a1150 0x7f7ae40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7aecbfe700 1 -- 172.21.15.62:0/1875133014 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4004fa0 con 0x563886fbce20 2023-07-14T16:24:45.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7aecbfe700 1 -- 172.21.15.62:0/1875133014 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af40d1030 con 0x563886fbce20 2023-07-14T16:24:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7aecbfe700 1 -- 172.21.15.62:0/1875133014 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4099510 con 0x563886fbce20 2023-07-14T16:24:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/1875133014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 msgr2=0x563886fbd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fbd6e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7af40d2a80 tx=0x7f7af40d2d90 comp rx=0 tx=0).stop 2023-07-14T16:24:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7afe99c700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7ae40a1150 0x7f7ae40a3600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009fd0 tx=0x7f7af0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/1875133014 shutdown_connections 2023-07-14T16:24:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fbe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fbc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/1875133014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fbd6e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.116+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/1875133014 >> 172.21.15.62:0/1875133014 conn(0x563886fb1da0 msgr2=0x563886fb71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.117+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/1875133014 shutdown_connections 2023-07-14T16:24:45.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.117+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/1875133014 wait complete. 2023-07-14T16:24:45.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.117+0000 7f7b13cd2b80 1 -- start start 2023-07-14T16:24:45.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.117+0000 7f7b13cd2b80 1 Processor -- start 2023-07-14T16:24:45.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 0x563886fc6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 0x563886fcd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886eadac0 con 0x563886fbce20 2023-07-14T16:24:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886ead7c0 con 0x563886fbc4d0 2023-07-14T16:24:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886d91920 con 0x563886fbdc70 2023-07-14T16:24:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7b13cd2b80 10 client.0 _ll_register_callbacks cb 0x563886eab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7afd99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7afd99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55096/0 (socket says 172.21.15.62:55096) 2023-07-14T16:24:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7afd99a700 1 -- 172.21.15.62:0/3841192890 learned_addr learned my addr 172.21.15.62:0/3841192890 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.118+0000 7f7afe99c700 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 0x563886fcd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.119+0000 7f7afe19b700 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 0x563886fc6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.119+0000 7f7afd99a700 1 -- 172.21.15.62:0/3841192890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 msgr2=0x563886fcd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.119+0000 7f7afd99a700 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 0x563886fcd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.119+0000 7f7afd99a700 1 -- 172.21.15.62:0/3841192890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 msgr2=0x563886fc6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.119+0000 7f7afd99a700 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 0x563886fc6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.119+0000 7f7afd99a700 1 -- 172.21.15.62:0/3841192890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af81a0080 con 0x563886fbce20 2023-07-14T16:24:45.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.119+0000 7f7afd99a700 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc6670 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098c0 tx=0x7f7ae800f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468353511599d757 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.119+0000 7f7ae1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:45.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/3841192890 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800b970 con 0x563886fbce20 2023-07-14T16:24:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/3841192890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563886e674f0 con 0x563886fbce20 2023-07-14T16:24:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/3841192890 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7ae800bad0 con 0x563886fbce20 2023-07-14T16:24:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/3841192890 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae8005040 con 0x563886fbce20 2023-07-14T16:24:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7b13cd2b80 10 client.18036 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/3841192890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563886d24c60 con 0x563886fbce20 2023-07-14T16:24:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/3841192890 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563886e674f0 con 0x563886fbce20 2023-07-14T16:24:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/3841192890 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ae8015020 con 0x563886fbce20 2023-07-14T16:24:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7b13cd2b80 10 client.18036 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:45.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.120+0000 7f7b13cd2b80 10 client.18036 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:24:45.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.121+0000 7f7b13cd2b80 20 client.18036 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:45.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.121+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563886fd1fb0 0x563886fd43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.121+0000 7f7b13cd2b80 4 client.18036 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:45.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.121+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/3841192890 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563886f06eb0 con 0x563886fd1fb0 2023-07-14T16:24:45.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.121+0000 7f7afe19b700 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563886fd1fb0 0x563886fd43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.122+0000 7f7afe19b700 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563886fd1fb0 0x563886fd43c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=330dc678ce8c01a5 server_cookie=ed6d19d1f5c3b261 in_seq=0 out_seq=0 2023-07-14T16:24:45.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.122+0000 7f7ae1ffb700 10 client.18036 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:24:45.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.122+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/3841192890 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563886f06eb0 con 0x563886fd1fb0 2023-07-14T16:24:45.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.122+0000 7f7ae1ffb700 10 client.18036 handle_command_reply: tid=0 2023-07-14T16:24:45.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.122+0000 7f7b13cd2b80 1 client.18036 shutdown 2023-07-14T16:24:45.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/3841192890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 msgr2=0x563886fc6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc6670 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098c0 tx=0x7f7ae800f7d0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/3841192890 shutdown_connections 2023-07-14T16:24:45.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563886fd1fb0 0x563886fd43c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 0x563886fcd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 0x563886fc6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/3841192890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc6670 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/3841192890 >> 172.21.15.62:0/3841192890 conn(0x563886fb1da0 msgr2=0x563886fb68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7ae1ffb700 0 client.18036 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:24:45.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/3841192890 shutdown_connections 2023-07-14T16:24:45.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.123+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/3841192890 wait complete. 2023-07-14T16:24:45.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.124+0000 7f7b13cd2b80 20 client.18036 trim_cache size 0 max 16384 2023-07-14T16:24:45.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.142+0000 7f7b13cd2b80 1 Processor -- start 2023-07-14T16:24:45.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7b13cd2b80 1 -- start start 2023-07-14T16:24:45.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fc0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fc1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fcd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886fc55e0 con 0x563886fbdc70 2023-07-14T16:24:45.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886fc5ca0 con 0x563886fbce20 2023-07-14T16:24:45.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886fb7ed0 con 0x563886fbc4d0 2023-07-14T16:24:45.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7afe99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fcd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7afe19b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fc0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7afe19b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fc0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46638/0 (socket says 172.21.15.62:46638) 2023-07-14T16:24:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7afe99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fcd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55098/0 (socket says 172.21.15.62:55098) 2023-07-14T16:24:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7afe19b700 1 -- 172.21.15.62:0/369443794 learned_addr learned my addr 172.21.15.62:0/369443794 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.143+0000 7f7afd99a700 1 --2- 172.21.15.62:0/369443794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fc1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7afe19b700 1 -- 172.21.15.62:0/369443794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 msgr2=0x563886fc1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7afe19b700 1 --2- 172.21.15.62:0/369443794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fc1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7afe19b700 1 -- 172.21.15.62:0/369443794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 msgr2=0x563886fcd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7afe19b700 1 --2- 172.21.15.62:0/369443794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fcd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7afe19b700 1 -- 172.21.15.62:0/369443794 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563886fbe560 con 0x563886fbc4d0 2023-07-14T16:24:45.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7afe19b700 1 --2- 172.21.15.62:0/369443794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fc0af0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7af4099b40 tx=0x7f7af40d2550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de1ed48e7bb1d105 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7aed3ff700 1 -- 172.21.15.62:0/369443794 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40b1070 con 0x563886fbc4d0 2023-07-14T16:24:45.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7aed3ff700 1 -- 172.21.15.62:0/369443794 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af4005740 con 0x563886fbc4d0 2023-07-14T16:24:45.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.144+0000 7f7aed3ff700 1 -- 172.21.15.62:0/369443794 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af400f3c0 con 0x563886fbc4d0 2023-07-14T16:24:45.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/369443794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 msgr2=0x563886fc0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/369443794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fc0af0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7af4099b40 tx=0x7f7af40d2550 comp rx=0 tx=0).stop 2023-07-14T16:24:45.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/369443794 shutdown_connections 2023-07-14T16:24:45.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/369443794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbdc70 0x563886fcd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/369443794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbc4d0 0x563886fc0af0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/369443794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbce20 0x563886fc1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/369443794 >> 172.21.15.62:0/369443794 conn(0x563886fb1da0 msgr2=0x563886fc3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/369443794 shutdown_connections 2023-07-14T16:24:45.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.145+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/369443794 wait complete. 2023-07-14T16:24:45.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.146+0000 7f7b13cd2b80 1 -- start start 2023-07-14T16:24:45.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.146+0000 7f7b13cd2b80 1 Processor -- start 2023-07-14T16:24:45.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 0x563886fc98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7b13cd2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 0x563886fccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886fcd090 con 0x563886fbce20 2023-07-14T16:24:45.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886fcd200 con 0x563886fbc4d0 2023-07-14T16:24:45.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7b13cd2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563886fcd370 con 0x563886fbdc70 2023-07-14T16:24:45.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7b13cd2b80 10 client.0 _ll_register_callbacks cb 0x7f7ae800feb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:45.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7afd99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7afd99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55110/0 (socket says 172.21.15.62:55110) 2023-07-14T16:24:45.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7afd99a700 1 -- 172.21.15.62:0/473088235 learned_addr learned my addr 172.21.15.62:0/473088235 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.147+0000 7f7afe19b700 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 0x563886fc98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:44 smithi062 ceph-mon[95543]: pgmap v630: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:45.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7afd99a700 1 -- 172.21.15.62:0/473088235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 msgr2=0x563886fccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7afd99a700 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 0x563886fccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7afd99a700 1 -- 172.21.15.62:0/473088235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 msgr2=0x563886fc98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7afd99a700 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 0x563886fc98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7afd99a700 1 -- 172.21.15.62:0/473088235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563886d3bde0 con 0x563886fbce20 2023-07-14T16:24:45.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7afd99a700 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc9e40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8003c20 tx=0x7f7ae800b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5a3f070c5a9ff5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7ae1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:45.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/473088235 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800b970 con 0x563886fbce20 2023-07-14T16:24:45.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/473088235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7af40d1030 con 0x563886fbce20 2023-07-14T16:24:45.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.148+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/473088235 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7ae800bad0 con 0x563886fbce20 2023-07-14T16:24:45.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/473088235 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae8005b00 con 0x563886fbce20 2023-07-14T16:24:45.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7b13cd2b80 10 client.18042 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:45.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/473088235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563886d62250 con 0x563886fbce20 2023-07-14T16:24:45.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/473088235 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af40d1030 con 0x563886fbce20 2023-07-14T16:24:45.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/473088235 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ae8009c70 con 0x563886fbce20 2023-07-14T16:24:45.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7b13cd2b80 10 client.18042 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:45.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7b13cd2b80 10 client.18042 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:24:45.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7b13cd2b80 20 client.18042 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:45.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563887026a90 0x563887028ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7b13cd2b80 4 client.18042 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:24:45.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.149+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/473088235 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563886ec9050 con 0x563887026a90 2023-07-14T16:24:45.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.150+0000 7f7afe19b700 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563887026a90 0x563887028ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.150+0000 7f7ae1ffb700 10 client.18042 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:24:45.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.150+0000 7f7afe19b700 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563887026a90 0x563887028ea0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d73351cf77b641b6 server_cookie=cb4ab79fff935544 in_seq=0 out_seq=0 2023-07-14T16:24:45.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7ae1ffb700 1 -- 172.21.15.62:0/473088235 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563886ec9050 con 0x563887026a90 2023-07-14T16:24:45.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7ae1ffb700 10 client.18042 handle_command_reply: tid=0 2023-07-14T16:24:45.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 client.18042 shutdown 2023-07-14T16:24:45.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/473088235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 msgr2=0x563886fc9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc9e40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8003c20 tx=0x7f7ae800b2c0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/473088235 shutdown_connections 2023-07-14T16:24:45.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563887026a90 0x563887028ea0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563886fbdc70 0x563886fccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563886fbc4d0 0x563886fc98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 --2- 172.21.15.62:0/473088235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563886fbce20 0x563886fc9e40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/473088235 >> 172.21.15.62:0/473088235 conn(0x563886fb1da0 msgr2=0x563886fc3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7ae1ffb700 0 client.18042 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:24:45.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/473088235 shutdown_connections 2023-07-14T16:24:45.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 1 -- 172.21.15.62:0/473088235 wait complete. 2023-07-14T16:24:45.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.151+0000 7f7b13cd2b80 20 client.18042 trim_cache size 0 max 16384 2023-07-14T16:24:45.133 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:24:45.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 -- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7ae40a1150 msgr2=0x7f7ae40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7ae40a1150 0x7f7ae40a3600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009fd0 tx=0x7f7af0009040 comp rx=0 tx=0).stop 2023-07-14T16:24:45.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 -- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8100560 msgr2=0x7f7af80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8100560 0x7f7af80ffed0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7af400d980 tx=0x7f7af400dc90 comp rx=0 tx=0).stop 2023-07-14T16:24:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 -- 172.21.15.62:0/2993109613 shutdown_connections 2023-07-14T16:24:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7af81057a0 0x7f7af80fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af8100560 0x7f7af80ffed0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7ae40a1150 0x7f7ae40a3600 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 --2- 172.21.15.62:0/2993109613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af8102e80 0x7f7af80fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 -- 172.21.15.62:0/2993109613 >> 172.21.15.62:0/2993109613 conn(0x7f7af80fa000 msgr2=0x7f7af81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 -- 172.21.15.62:0/2993109613 shutdown_connections 2023-07-14T16:24:45.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.152+0000 7f7b003ff700 1 -- 172.21.15.62:0/2993109613 wait complete. 2023-07-14T16:24:45.150 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:24:45.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:44 smithi081 ceph-mon[127795]: pgmap v630: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:45.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:44 smithi164 ceph-mon[97427]: pgmap v630: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.404+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/2378215579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb24102ec0 msgr2=0x7fdb241052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.404+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/2378215579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb24102ec0 0x7fdb241052a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009990 tx=0x7fdb14009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.405+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/2378215579 shutdown_connections 2023-07-14T16:24:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.405+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/2378215579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb241057e0 0x7fdb24107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.405+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/2378215579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb241005a0 0x7fdb24102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.405+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/2378215579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb24102ec0 0x7fdb241052a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.405+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/2378215579 >> 172.21.15.62:0/2378215579 conn(0x7fdb240fa020 msgr2=0x7fdb240fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.405+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/2378215579 shutdown_connections 2023-07-14T16:24:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.405+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/2378215579 wait complete. 2023-07-14T16:24:45.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.406+0000 7fdb2c5ff700 1 Processor -- start 2023-07-14T16:24:45.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.406+0000 7fdb2c5ff700 1 -- start start 2023-07-14T16:24:45.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.406+0000 7fdb2c5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb241005a0 0x7fdb24197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.406+0000 7fdb2c5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb24102ec0 0x7fdb24197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.406+0000 7fdb2c5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb241057e0 0x7fdb2419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.406+0000 7fdb2c5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2419f4c0 con 0x7fdb241005a0 2023-07-14T16:24:45.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.406+0000 7fdb2c5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2419f600 con 0x7fdb24102ec0 2023-07-14T16:24:45.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.406+0000 7fdb2c5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2419f790 con 0x7fdb241057e0 2023-07-14T16:24:45.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb241005a0 0x7fdb24197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb241005a0 0x7fdb24197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55122/0 (socket says 172.21.15.62:55122) 2023-07-14T16:24:45.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 -- 172.21.15.62:0/662137622 learned_addr learned my addr 172.21.15.62:0/662137622 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb24102ec0 0x7fdb24197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb241057e0 0x7fdb2419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 -- 172.21.15.62:0/662137622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb241057e0 msgr2=0x7fdb2419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb241057e0 0x7fdb2419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 -- 172.21.15.62:0/662137622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb24102ec0 msgr2=0x7fdb24197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb24102ec0 0x7fdb24197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 -- 172.21.15.62:0/662137622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb140095f0 con 0x7fdb241005a0 2023-07-14T16:24:45.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.407+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb241005a0 0x7fdb24197940 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000e9a0 tx=0x7fdb2000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.408+0000 7fdb1b7fe700 1 -- 172.21.15.62:0/662137622 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20009790 con 0x7fdb241005a0 2023-07-14T16:24:45.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.408+0000 7fdb1b7fe700 1 -- 172.21.15.62:0/662137622 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb2000cbd0 con 0x7fdb241005a0 2023-07-14T16:24:45.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.408+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/662137622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2419fa70 con 0x7fdb241005a0 2023-07-14T16:24:45.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.408+0000 7fdb1b7fe700 1 -- 172.21.15.62:0/662137622 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20018680 con 0x7fdb241005a0 2023-07-14T16:24:45.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.408+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/662137622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2419ffd0 con 0x7fdb241005a0 2023-07-14T16:24:45.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.412+0000 7fdb1b7fe700 1 -- 172.21.15.62:0/662137622 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdb2000cd40 con 0x7fdb241005a0 2023-07-14T16:24:45.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.413+0000 7fdb3fe46b80 1 Processor -- start 2023-07-14T16:24:45.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.413+0000 7fdb3fe46b80 1 -- start start 2023-07-14T16:24:45.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.413+0000 7fdb1b7fe700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb100a11d0 0x7fdb100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2950434d0 0x55c2950438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295044c70 0x55c2950450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c294f48cb0 con 0x55c295043e20 2023-07-14T16:24:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c294f9ba40 con 0x55c2950434d0 2023-07-14T16:24:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c294f94ed0 con 0x55c295044c70 2023-07-14T16:24:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb1b7fe700 1 -- 172.21.15.62:0/662137622 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb20014070 con 0x7fdb241005a0 2023-07-14T16:24:45.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb29b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb100a11d0 0x7fdb100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb29b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55128/0 (socket says 172.21.15.62:55128) 2023-07-14T16:24:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb29b9a700 1 -- 172.21.15.62:0/2093987782 learned_addr learned my addr 172.21.15.62:0/2093987782 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2950434d0 0x55c2950438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb29b9a700 1 -- 172.21.15.62:0/2093987782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295044c70 msgr2=0x55c2950450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295044c70 0x55c2950450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295044c70 0x55c2950450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb29b9a700 1 -- 172.21.15.62:0/2093987782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2950434d0 msgr2=0x55c2950438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2950434d0 0x55c2950438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.414+0000 7fdb29b9a700 1 -- 172.21.15.62:0/2093987782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c294dc2de0 con 0x55c295043e20 2023-07-14T16:24:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.415+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2950434d0 0x55c2950438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:45.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.415+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295044c70 0x55c2950450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:24:45.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.415+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950446e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdb140059d0 tx=0x7fdb14005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08ed6d6813c8f63 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.415+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb100a11d0 0x7fdb100a3680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0060b0 tx=0x7fdb1c009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.415+0000 7fdb18bfe700 1 -- 172.21.15.62:0/2093987782 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1401d070 con 0x55c295043e20 2023-07-14T16:24:45.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.415+0000 7fdb18bfe700 1 -- 172.21.15.62:0/2093987782 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb1401e040 con 0x55c295043e20 2023-07-14T16:24:45.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.415+0000 7fdb18bfe700 1 -- 172.21.15.62:0/2093987782 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400b970 con 0x55c295043e20 2023-07-14T16:24:45.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2093987782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 msgr2=0x55c2950446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950446e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdb140059d0 tx=0x7fdb14005c70 comp rx=0 tx=0).stop 2023-07-14T16:24:45.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2093987782 shutdown_connections 2023-07-14T16:24:45.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295044c70 0x55c2950450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2950434d0 0x55c2950438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/2093987782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950446e0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2093987782 >> 172.21.15.62:0/2093987782 conn(0x55c295038da0 msgr2=0x55c29503e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2093987782 shutdown_connections 2023-07-14T16:24:45.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.416+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2093987782 wait complete. 2023-07-14T16:24:45.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.417+0000 7fdb3fe46b80 1 -- start start 2023-07-14T16:24:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.417+0000 7fdb3fe46b80 1 Processor -- start 2023-07-14T16:24:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2950434d0 0x55c29504d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295043e20 0x55c29504d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c295054600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c294f34ac0 con 0x55c2950434d0 2023-07-14T16:24:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c294f347c0 con 0x55c295044c70 2023-07-14T16:24:45.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c294e18920 con 0x55c295043e20 2023-07-14T16:24:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb3fe46b80 10 client.0 _ll_register_callbacks cb 0x55c294f32760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb2a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2950434d0 0x55c29504d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb2a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2950434d0 0x55c29504d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55132/0 (socket says 172.21.15.62:55132) 2023-07-14T16:24:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb2a39b700 1 -- 172.21.15.62:0/3009345596 learned_addr learned my addr 172.21.15.62:0/3009345596 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.418+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c295054600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295043e20 0x55c29504d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb2a39b700 1 -- 172.21.15.62:0/3009345596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295043e20 msgr2=0x55c29504d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295043e20 0x55c29504d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb2a39b700 1 -- 172.21.15.62:0/3009345596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 msgr2=0x55c295054600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c295054600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb2a39b700 1 -- 172.21.15.62:0/3009345596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2419ffd0 con 0x55c2950434d0 2023-07-14T16:24:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c295054600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2950434d0 0x55c29504d080 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fdb20099ed0 tx=0x7fdb200999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a954d840dd3280ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:45.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3009345596 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2009df70 con 0x55c2950434d0 2023-07-14T16:24:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3009345596 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb200a2390 con 0x55c2950434d0 2023-07-14T16:24:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.419+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3009345596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c294eee4f0 con 0x55c2950434d0 2023-07-14T16:24:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3009345596 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb2009a1a0 con 0x55c2950434d0 2023-07-14T16:24:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb3fe46b80 10 client.18054 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3009345596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fdb24108830 con 0x55c2950434d0 2023-07-14T16:24:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3009345596 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c294eee4f0 con 0x55c2950434d0 2023-07-14T16:24:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3009345596 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb200a1f00 con 0x55c2950434d0 2023-07-14T16:24:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb3fe46b80 10 client.18054 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb3fe46b80 10 client.18054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:24:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb3fe46b80 20 client.18054 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c295058d40 0x55c29505b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb3fe46b80 4 client.18054 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.420+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3009345596 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c294f8deb0 con 0x55c295058d40 2023-07-14T16:24:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.421+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c295058d40 0x55c29505b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.421+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c295058d40 0x55c29505b150 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90f1f35e357bd364 server_cookie=368ce5e35e5e9984 in_seq=0 out_seq=0 2023-07-14T16:24:45.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.421+0000 7fdb0dffb700 10 client.18054 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:24:45.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.422+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3009345596 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c294f8deb0 con 0x55c295058d40 2023-07-14T16:24:45.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.422+0000 7fdb0dffb700 10 client.18054 handle_command_reply: tid=0 2023-07-14T16:24:45.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.422+0000 7fdb3fe46b80 1 client.18054 shutdown 2023-07-14T16:24:45.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.422+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3009345596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2950434d0 msgr2=0x55c29504d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.422+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2950434d0 0x55c29504d080 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fdb20099ed0 tx=0x7fdb200999d0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.422+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3009345596 shutdown_connections 2023-07-14T16:24:45.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.422+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c295058d40 0x55c29505b150 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.423+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c295054600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.423+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2950434d0 0x55c29504d080 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.423+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3009345596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c295043e20 0x55c29504d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.423+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3009345596 >> 172.21.15.62:0/3009345596 conn(0x55c295038da0 msgr2=0x55c29503d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.423+0000 7fdb0dffb700 0 client.18054 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:24:45.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.423+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3009345596 shutdown_connections 2023-07-14T16:24:45.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.423+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3009345596 wait complete. 2023-07-14T16:24:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.423+0000 7fdb3fe46b80 20 client.18054 trim_cache size 0 max 16384 2023-07-14T16:24:45.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.445+0000 7fdb3fe46b80 1 Processor -- start 2023-07-14T16:24:45.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.445+0000 7fdb3fe46b80 1 -- start start 2023-07-14T16:24:45.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 0x55c2950450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c29504be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29504c390 con 0x55c295043e20 2023-07-14T16:24:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c295044530 con 0x55c295044c70 2023-07-14T16:24:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29503f070 con 0x55c2950434d0 2023-07-14T16:24:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb29b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb29b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55134/0 (socket says 172.21.15.62:55134) 2023-07-14T16:24:45.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb29b9a700 1 -- 172.21.15.62:0/2995671689 learned_addr learned my addr 172.21.15.62:0/2995671689 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.446+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/2995671689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c29504be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb29b9a700 1 -- 172.21.15.62:0/2995671689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 msgr2=0x55c2950450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/2995671689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 0x55c2950450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb29b9a700 1 -- 172.21.15.62:0/2995671689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 msgr2=0x55c29504be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/2995671689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c29504be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb29b9a700 1 -- 172.21.15.62:0/2995671689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c294dabc60 con 0x55c295043e20 2023-07-14T16:24:45.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/2995671689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950545c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009990 tx=0x7fdb1401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444cdb7d1289b5ca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb193ff700 1 -- 172.21.15.62:0/2995671689 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14031070 con 0x55c295043e20 2023-07-14T16:24:45.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb193ff700 1 -- 172.21.15.62:0/2995671689 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb140055d0 con 0x55c295043e20 2023-07-14T16:24:45.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.447+0000 7fdb193ff700 1 -- 172.21.15.62:0/2995671689 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1401d040 con 0x55c295043e20 2023-07-14T16:24:45.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.448+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2995671689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 msgr2=0x55c2950545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.448+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/2995671689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950545c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009990 tx=0x7fdb1401a040 comp rx=0 tx=0).stop 2023-07-14T16:24:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.448+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2995671689 shutdown_connections 2023-07-14T16:24:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.448+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/2995671689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c29504be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.448+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/2995671689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 0x55c2950450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.448+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/2995671689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950545c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.448+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2995671689 >> 172.21.15.62:0/2995671689 conn(0x55c295038da0 msgr2=0x55c29503e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.448+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2995671689 shutdown_connections 2023-07-14T16:24:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.449+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/2995671689 wait complete. 2023-07-14T16:24:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.449+0000 7fdb3fe46b80 1 -- start start 2023-07-14T16:24:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.449+0000 7fdb3fe46b80 1 Processor -- start 2023-07-14T16:24:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 0x55c295050260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb3fe46b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c2950537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c295053d00 con 0x55c295043e20 2023-07-14T16:24:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c295053e70 con 0x55c295044c70 2023-07-14T16:24:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb3fe46b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c295054030 con 0x55c2950434d0 2023-07-14T16:24:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb3fe46b80 10 client.0 _ll_register_callbacks cb 0x7fdb20099da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb2a39b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 0x55c295050260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb29b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb2ab9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c2950537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb2a39b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 0x55c295050260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46696/0 (socket says 172.21.15.62:46696) 2023-07-14T16:24:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.450+0000 7fdb2a39b700 1 -- 172.21.15.62:0/3056523241 learned_addr learned my addr 172.21.15.62:0/3056523241 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.451+0000 7fdb29b9a700 1 -- 172.21.15.62:0/3056523241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 msgr2=0x55c295050260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.451+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 0x55c295050260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.451+0000 7fdb29b9a700 1 -- 172.21.15.62:0/3056523241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 msgr2=0x55c2950537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.451+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c2950537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.451+0000 7fdb29b9a700 1 -- 172.21.15.62:0/3056523241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c294dc2de0 con 0x55c295043e20 2023-07-14T16:24:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.451+0000 7fdb2ab9c700 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c2950537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.451+0000 7fdb29b9a700 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950507a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400b2e0 tx=0x7fdb14005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e419927706070a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.451+0000 7fdb0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.452+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3056523241 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14031040 con 0x55c295043e20 2023-07-14T16:24:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.452+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3056523241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb1401e040 con 0x55c295043e20 2023-07-14T16:24:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.452+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3056523241 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb1400b710 con 0x55c295043e20 2023-07-14T16:24:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.452+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3056523241 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb1401d040 con 0x55c295043e20 2023-07-14T16:24:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.452+0000 7fdb3fe46b80 10 client.18060 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.452+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3056523241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c294de9250 con 0x55c295043e20 2023-07-14T16:24:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.452+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3056523241 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb1401e040 con 0x55c295043e20 2023-07-14T16:24:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.452+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3056523241 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb14033b50 con 0x55c295043e20 2023-07-14T16:24:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.453+0000 7fdb3fe46b80 10 client.18060 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.453+0000 7fdb3fe46b80 10 client.18060 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:24:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.453+0000 7fdb3fe46b80 20 client.18060 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.453+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c2950ad560 0x55c2950af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.453+0000 7fdb3fe46b80 4 client.18060 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:24:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.453+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3056523241 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c294f50050 con 0x55c2950ad560 2023-07-14T16:24:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.453+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c2950ad560 0x55c2950af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb0dffb700 10 client.18060 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:24:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb2a39b700 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c2950ad560 0x55c2950af940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=547601c54d05f0d3 server_cookie=1773d8360f207b51 in_seq=0 out_seq=0 2023-07-14T16:24:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb0dffb700 1 -- 172.21.15.62:0/3056523241 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c294f50050 con 0x55c2950ad560 2023-07-14T16:24:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb0dffb700 10 client.18060 handle_command_reply: tid=0 2023-07-14T16:24:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 client.18060 shutdown 2023-07-14T16:24:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3056523241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 msgr2=0x55c2950507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950507a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400b2e0 tx=0x7fdb14005810 comp rx=0 tx=0).stop 2023-07-14T16:24:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3056523241 shutdown_connections 2023-07-14T16:24:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55c2950ad560 0x55c2950af940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c295044c70 0x55c2950537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2950434d0 0x55c295050260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb0dffb700 0 client.18060 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:24:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 --2- 172.21.15.62:0/3056523241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c295043e20 0x55c2950507a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.454+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3056523241 >> 172.21.15.62:0/3056523241 conn(0x55c295038da0 msgr2=0x55c29503e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.455+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3056523241 shutdown_connections 2023-07-14T16:24:45.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.455+0000 7fdb3fe46b80 1 -- 172.21.15.62:0/3056523241 wait complete. 2023-07-14T16:24:45.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.455+0000 7fdb3fe46b80 20 client.18060 trim_cache size 0 max 16384 2023-07-14T16:24:45.439 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:24:45.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.455+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb100a11d0 msgr2=0x7fdb100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.455+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb100a11d0 0x7fdb100a3680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0060b0 tx=0x7fdb1c009c00 comp rx=0 tx=0).stop 2023-07-14T16:24:45.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.455+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb241005a0 msgr2=0x7fdb24197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.455+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb241005a0 0x7fdb24197940 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000e9a0 tx=0x7fdb2000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.456+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/662137622 shutdown_connections 2023-07-14T16:24:45.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.456+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb241057e0 0x7fdb2419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.456+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb241005a0 0x7fdb24197940 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.456+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb100a11d0 0x7fdb100a3680 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.456+0000 7fdb2c5ff700 1 --2- 172.21.15.62:0/662137622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb24102ec0 0x7fdb24197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.456+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/662137622 >> 172.21.15.62:0/662137622 conn(0x7fdb240fa020 msgr2=0x7fdb240fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.456+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/662137622 shutdown_connections 2023-07-14T16:24:45.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.456+0000 7fdb2c5ff700 1 -- 172.21.15.62:0/662137622 wait complete. 2023-07-14T16:24:45.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:24:45.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 -- 172.21.15.62:0/895748182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 msgr2=0x7fd768102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/895748182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 0x7fd768102870 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd758009990 tx=0x7fd758009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 -- 172.21.15.62:0/895748182 shutdown_connections 2023-07-14T16:24:45.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/895748182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7681056d0 0x7fd768107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/895748182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 0x7fd768102870 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/895748182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd768102db0 0x7fd768105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 -- 172.21.15.62:0/895748182 >> 172.21.15.62:0/895748182 conn(0x7fd7680f9ed0 msgr2=0x7fd7680fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 -- 172.21.15.62:0/895748182 shutdown_connections 2023-07-14T16:24:45.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.697+0000 7fd76cb9d700 1 -- 172.21.15.62:0/895748182 wait complete. 2023-07-14T16:24:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.698+0000 7fd76cb9d700 1 Processor -- start 2023-07-14T16:24:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd76cb9d700 1 -- start start 2023-07-14T16:24:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd76cb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 0x7fd768197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd76cb9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd768102db0 0x7fd768197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd76cb9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7681056d0 0x7fd76819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd76cb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819f2a0 con 0x7fd768100490 2023-07-14T16:24:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd76cb9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819f3e0 con 0x7fd768102db0 2023-07-14T16:24:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd76cb9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819f570 con 0x7fd7681056d0 2023-07-14T16:24:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 0x7fd768197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 0x7fd768197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55158/0 (socket says 172.21.15.62:55158) 2023-07-14T16:24:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.699+0000 7fd7677fe700 1 -- 172.21.15.62:0/2907836137 learned_addr learned my addr 172.21.15.62:0/2907836137 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd766ffd700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd768102db0 0x7fd768197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd767fff700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7681056d0 0x7fd76819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd7677fe700 1 -- 172.21.15.62:0/2907836137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7681056d0 msgr2=0x7fd76819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd7677fe700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7681056d0 0x7fd76819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd7677fe700 1 -- 172.21.15.62:0/2907836137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd768102db0 msgr2=0x7fd768197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd7677fe700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd768102db0 0x7fd768197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd7677fe700 1 -- 172.21.15.62:0/2907836137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580095f0 con 0x7fd768100490 2023-07-14T16:24:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd766ffd700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd768102db0 0x7fd768197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd767fff700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7681056d0 0x7fd76819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd7677fe700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 0x7fd768197720 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd758005910 tx=0x7fd7580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd764ff9700 1 -- 172.21.15.62:0/2907836137 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75801d070 con 0x7fd768100490 2023-07-14T16:24:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd764ff9700 1 -- 172.21.15.62:0/2907836137 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd758022470 con 0x7fd768100490 2023-07-14T16:24:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.700+0000 7fd76cb9d700 1 -- 172.21.15.62:0/2907836137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76819f7f0 con 0x7fd768100490 2023-07-14T16:24:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.701+0000 7fd764ff9700 1 -- 172.21.15.62:0/2907836137 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75800f6c0 con 0x7fd768100490 2023-07-14T16:24:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.701+0000 7fd76cb9d700 1 -- 172.21.15.62:0/2907836137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76819fc70 con 0x7fd768100490 2023-07-14T16:24:45.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.704+0000 7fd764ff9700 1 -- 172.21.15.62:0/2907836137 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd7580225e0 con 0x7fd768100490 2023-07-14T16:24:45.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd781707b80 1 Processor -- start 2023-07-14T16:24:45.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd781707b80 1 -- start start 2023-07-14T16:24:45.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd764ff9700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7500a1270 0x7fd7500a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd781707b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdb720 0x560bc4cdbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd764ff9700 1 -- 172.21.15.62:0/2907836137 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7580cdde0 con 0x7fd768100490 2023-07-14T16:24:45.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd781707b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4cdc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd781707b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdce30 0x560bc4cdd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd781707b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4be0cb0 con 0x560bc4cdc070 2023-07-14T16:24:45.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd781707b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4c33a40 con 0x560bc4cdce30 2023-07-14T16:24:45.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.706+0000 7fd781707b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4c2ced0 con 0x560bc4cdb720 2023-07-14T16:24:45.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4cdc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdb720 0x560bc4cdbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd767fff700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7500a1270 0x7fd7500a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4cdc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55164/0 (socket says 172.21.15.62:55164) 2023-07-14T16:24:45.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdb720 0x560bc4cdbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46728/0 (socket says 172.21.15.62:46728) 2023-07-14T16:24:45.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd766ffd700 1 -- 172.21.15.62:0/4104851470 learned_addr learned my addr 172.21.15.62:0/4104851470 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd767fff700 1 --2- 172.21.15.62:0/4104851470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdce30 0x560bc4cdd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd766ffd700 1 -- 172.21.15.62:0/4104851470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdb720 msgr2=0x560bc4cdbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd766ffd700 1 --2- 172.21.15.62:0/4104851470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdb720 0x560bc4cdbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd766ffd700 1 -- 172.21.15.62:0/4104851470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdce30 msgr2=0x560bc4cdd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd766ffd700 1 --2- 172.21.15.62:0/4104851470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdce30 0x560bc4cdd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.707+0000 7fd766ffd700 1 -- 172.21.15.62:0/4104851470 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bc4a5ade0 con 0x560bc4cdc070 2023-07-14T16:24:45.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.708+0000 7fd766ffd700 1 --2- 172.21.15.62:0/4104851470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4cdc8a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009de0 tx=0x7fd75c0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930589b5d51452f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.708+0000 7fd767fff700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7500a1270 0x7fd7500a3720 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd760015e20 tx=0x7fd760015420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.708+0000 7fd74dffb700 1 -- 172.21.15.62:0/4104851470 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c011e20 con 0x560bc4cdc070 2023-07-14T16:24:45.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.708+0000 7fd74dffb700 1 -- 172.21.15.62:0/4104851470 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd75c0128e0 con 0x560bc4cdc070 2023-07-14T16:24:45.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.708+0000 7fd74dffb700 1 -- 172.21.15.62:0/4104851470 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c00f6a0 con 0x560bc4cdc070 2023-07-14T16:24:45.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 -- 172.21.15.62:0/4104851470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 msgr2=0x560bc4cdc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 --2- 172.21.15.62:0/4104851470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4cdc8a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009de0 tx=0x7fd75c0116c0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 -- 172.21.15.62:0/4104851470 shutdown_connections 2023-07-14T16:24:45.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 --2- 172.21.15.62:0/4104851470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdce30 0x560bc4cdd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 --2- 172.21.15.62:0/4104851470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdb720 0x560bc4cdbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 --2- 172.21.15.62:0/4104851470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4cdc8a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 -- 172.21.15.62:0/4104851470 >> 172.21.15.62:0/4104851470 conn(0x560bc4cd0ff0 msgr2=0x560bc4cd6460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 -- 172.21.15.62:0/4104851470 shutdown_connections 2023-07-14T16:24:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.709+0000 7fd781707b80 1 -- 172.21.15.62:0/4104851470 wait complete. 2023-07-14T16:24:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.710+0000 7fd781707b80 1 -- start start 2023-07-14T16:24:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.710+0000 7fd781707b80 1 Processor -- start 2023-07-14T16:24:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd781707b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4ce5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd781707b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 0x560bc4ce57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd781707b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd781707b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4bccac0 con 0x560bc4cdb720 2023-07-14T16:24:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd781707b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4bcc7c0 con 0x560bc4cdc070 2023-07-14T16:24:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd781707b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4ab0920 con 0x560bc4cdce30 2023-07-14T16:24:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd781707b80 10 client.0 _ll_register_callbacks cb 0x560bc4bca760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4ce5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4ce5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55178/0 (socket says 172.21.15.62:55178) 2023-07-14T16:24:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd7677fe700 1 -- 172.21.15.62:0/612655199 learned_addr learned my addr 172.21.15.62:0/612655199 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd766ffd700 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 0x560bc4ce57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.711+0000 7fd767fff700 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd7677fe700 1 -- 172.21.15.62:0/612655199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 msgr2=0x560bc4cec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd7677fe700 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd7677fe700 1 -- 172.21.15.62:0/612655199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 msgr2=0x560bc4ce57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd7677fe700 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 0x560bc4ce57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd7677fe700 1 -- 172.21.15.62:0/612655199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75c009a10 con 0x560bc4cdb720 2023-07-14T16:24:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd7677fe700 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4ce5270 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd758000c00 tx=0x7fd758022ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0b6a7ba8cdad05 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd746ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd746ffd700 1 -- 172.21.15.62:0/612655199 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7580aeea0 con 0x560bc4cdb720 2023-07-14T16:24:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd746ffd700 1 -- 172.21.15.62:0/612655199 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd75809b230 con 0x560bc4cdb720 2023-07-14T16:24:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd781707b80 1 -- 172.21.15.62:0/612655199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bc4b864f0 con 0x560bc4cdb720 2023-07-14T16:24:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd746ffd700 1 -- 172.21.15.62:0/612655199 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd75809f260 con 0x560bc4cdb720 2023-07-14T16:24:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd781707b80 10 client.18072 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.712+0000 7fd781707b80 1 -- 172.21.15.62:0/612655199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560bc4a43c60 con 0x560bc4cdb720 2023-07-14T16:24:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.713+0000 7fd746ffd700 1 -- 172.21.15.62:0/612655199 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7580c9020 con 0x560bc4cdb720 2023-07-14T16:24:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.713+0000 7fd746ffd700 1 -- 172.21.15.62:0/612655199 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7580a3160 con 0x560bc4cdb720 2023-07-14T16:24:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.714+0000 7fd781707b80 10 client.18072 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:45.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.714+0000 7fd781707b80 10 client.18072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:45.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.714+0000 7fd781707b80 20 client.18072 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:45.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.714+0000 7fd781707b80 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560bc4cf0cd0 0x560bc4cf30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.714+0000 7fd781707b80 4 client.18072 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:45.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.714+0000 7fd781707b80 1 -- 172.21.15.62:0/612655199 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bc4c25eb0 con 0x560bc4cf0cd0 2023-07-14T16:24:45.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.714+0000 7fd766ffd700 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560bc4cf0cd0 0x560bc4cf30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.715+0000 7fd766ffd700 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560bc4cf0cd0 0x560bc4cf30e0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9147771790d6cf0 server_cookie=671b3981bc4d95f0 in_seq=0 out_seq=0 2023-07-14T16:24:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.715+0000 7fd746ffd700 10 client.18072 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.715+0000 7fd746ffd700 1 -- 172.21.15.62:0/612655199 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560bc4c25eb0 con 0x560bc4cf0cd0 2023-07-14T16:24:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.715+0000 7fd746ffd700 10 client.18072 handle_command_reply: tid=0 2023-07-14T16:24:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.715+0000 7fd781707b80 1 client.18072 shutdown 2023-07-14T16:24:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 -- 172.21.15.62:0/612655199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 msgr2=0x560bc4ce5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4ce5270 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd758000c00 tx=0x7fd758022ba0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 -- 172.21.15.62:0/612655199 shutdown_connections 2023-07-14T16:24:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4ce5270 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560bc4cf0cd0 0x560bc4cf30e0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 --2- 172.21.15.62:0/612655199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 0x560bc4ce57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 -- 172.21.15.62:0/612655199 >> 172.21.15.62:0/612655199 conn(0x560bc4cd0ff0 msgr2=0x560bc4cd5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd746ffd700 0 client.18072 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 -- 172.21.15.62:0/612655199 shutdown_connections 2023-07-14T16:24:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 1 -- 172.21.15.62:0/612655199 wait complete. 2023-07-14T16:24:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.716+0000 7fd781707b80 20 client.18072 trim_cache size 0 max 16384 2023-07-14T16:24:45.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.737+0000 7fd781707b80 1 Processor -- start 2023-07-14T16:24:45.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.737+0000 7fd781707b80 1 -- start start 2023-07-14T16:24:45.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd781707b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4cf7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd781707b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 0x560bc4cec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd781707b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4ced020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd781707b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4ce4e40 con 0x560bc4cdb720 2023-07-14T16:24:45.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd781707b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4cdc5d0 con 0x560bc4cdc070 2023-07-14T16:24:45.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd781707b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4ce6280 con 0x560bc4cdce30 2023-07-14T16:24:45.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4cf7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4cf7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55190/0 (socket says 172.21.15.62:55190) 2023-07-14T16:24:45.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd7677fe700 1 -- 172.21.15.62:0/455240794 learned_addr learned my addr 172.21.15.62:0/455240794 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd766ffd700 1 --2- 172.21.15.62:0/455240794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 0x560bc4cec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.738+0000 7fd7677fe700 1 -- 172.21.15.62:0/455240794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 msgr2=0x560bc4ced020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.739+0000 7fd7677fe700 1 --2- 172.21.15.62:0/455240794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4ced020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.739+0000 7fd7677fe700 1 -- 172.21.15.62:0/455240794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 msgr2=0x560bc4cec7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.739+0000 7fd7677fe700 1 --2- 172.21.15.62:0/455240794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 0x560bc4cec7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.739+0000 7fd7677fe700 1 -- 172.21.15.62:0/455240794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bc4a81250 con 0x560bc4cdb720 2023-07-14T16:24:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.739+0000 7fd7677fe700 1 --2- 172.21.15.62:0/455240794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4cf7a40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd758005f50 tx=0x7fd7580997e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2499490204abc96b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.739+0000 7fd74e7fc700 1 -- 172.21.15.62:0/455240794 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7580aeea0 con 0x560bc4cdb720 2023-07-14T16:24:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.739+0000 7fd74e7fc700 1 -- 172.21.15.62:0/455240794 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7580c9070 con 0x560bc4cdb720 2023-07-14T16:24:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.739+0000 7fd74e7fc700 1 -- 172.21.15.62:0/455240794 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75809a720 con 0x560bc4cdb720 2023-07-14T16:24:45.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 -- 172.21.15.62:0/455240794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 msgr2=0x560bc4cf7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 --2- 172.21.15.62:0/455240794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4cf7a40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd758005f50 tx=0x7fd7580997e0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 -- 172.21.15.62:0/455240794 shutdown_connections 2023-07-14T16:24:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 --2- 172.21.15.62:0/455240794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4ced020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 --2- 172.21.15.62:0/455240794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdb720 0x560bc4cf7a40 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 --2- 172.21.15.62:0/455240794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdc070 0x560bc4cec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 -- 172.21.15.62:0/455240794 >> 172.21.15.62:0/455240794 conn(0x560bc4cd0ff0 msgr2=0x560bc4cd2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 -- 172.21.15.62:0/455240794 shutdown_connections 2023-07-14T16:24:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.740+0000 7fd781707b80 1 -- 172.21.15.62:0/455240794 wait complete. 2023-07-14T16:24:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.741+0000 7fd781707b80 1 -- start start 2023-07-14T16:24:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.741+0000 7fd781707b80 1 Processor -- start 2023-07-14T16:24:45.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd781707b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdb720 0x560bc4ce8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd781707b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4ce8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd781707b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd781707b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4cec4d0 con 0x560bc4cdc070 2023-07-14T16:24:45.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd781707b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4cec640 con 0x560bc4cdb720 2023-07-14T16:24:45.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd781707b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc4cec800 con 0x560bc4cdce30 2023-07-14T16:24:45.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd781707b80 10 client.0 _ll_register_callbacks cb 0x7fd7580975d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:45.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4ce8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4ce8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55202/0 (socket says 172.21.15.62:55202) 2023-07-14T16:24:45.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd766ffd700 1 -- 172.21.15.62:0/56360387 learned_addr learned my addr 172.21.15.62:0/56360387 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:45.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd7677fe700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdb720 0x560bc4ce8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.742+0000 7fd767fff700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd766ffd700 1 -- 172.21.15.62:0/56360387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 msgr2=0x560bc4cebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd766ffd700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd766ffd700 1 -- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdb720 msgr2=0x560bc4ce8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:45.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd766ffd700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdb720 0x560bc4ce8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd766ffd700 1 -- 172.21.15.62:0/56360387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bc4a43c60 con 0x560bc4cdc070 2023-07-14T16:24:45.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd767fff700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:45.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd7677fe700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdb720 0x560bc4ce8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:45.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd766ffd700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4ce8f70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd75c01c040 tx=0x7fd75c0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0c7bc7a8f53a64 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:45.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd746ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:45.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.743+0000 7fd746ffd700 1 -- 172.21.15.62:0/56360387 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c012690 con 0x560bc4cdc070 2023-07-14T16:24:45.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 1 -- 172.21.15.62:0/56360387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd75c0128e0 con 0x560bc4cdc070 2023-07-14T16:24:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd746ffd700 1 -- 172.21.15.62:0/56360387 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd75c029b50 con 0x560bc4cdc070 2023-07-14T16:24:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd746ffd700 1 -- 172.21.15.62:0/56360387 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd75c038030 con 0x560bc4cdc070 2023-07-14T16:24:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 10 client.18078 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 1 -- 172.21.15.62:0/56360387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560bc4a592b0 con 0x560bc4cdc070 2023-07-14T16:24:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd746ffd700 1 -- 172.21.15.62:0/56360387 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd75c0128e0 con 0x560bc4cdc070 2023-07-14T16:24:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd746ffd700 1 -- 172.21.15.62:0/56360387 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd75c0113e0 con 0x560bc4cdc070 2023-07-14T16:24:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 10 client.18078 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 10 client.18078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:45.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 20 client.18078 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:45.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560bc4d45ed0 0x560bc4d482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:45.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 4 client.18078 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:24:45.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.744+0000 7fd781707b80 1 -- 172.21.15.62:0/56360387 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560bc4be8050 con 0x560bc4d45ed0 2023-07-14T16:24:45.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.745+0000 7fd7677fe700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560bc4d45ed0 0x560bc4d482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:45.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd7677fe700 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560bc4d45ed0 0x560bc4d482e0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3203160d092dbd5b server_cookie=9bfcd02a759c5755 in_seq=0 out_seq=0 2023-07-14T16:24:45.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd746ffd700 10 client.18078 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:45.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd746ffd700 1 -- 172.21.15.62:0/56360387 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560bc4be8050 con 0x560bc4d45ed0 2023-07-14T16:24:45.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd746ffd700 10 client.18078 handle_command_reply: tid=0 2023-07-14T16:24:45.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 client.18078 shutdown 2023-07-14T16:24:45.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 -- 172.21.15.62:0/56360387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 msgr2=0x560bc4ce8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4ce8f70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd75c01c040 tx=0x7fd75c0092f0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 -- 172.21.15.62:0/56360387 shutdown_connections 2023-07-14T16:24:45.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560bc4cdce30 0x560bc4cebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560bc4d45ed0 0x560bc4d482e0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560bc4cdb720 0x560bc4ce8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 --2- 172.21.15.62:0/56360387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bc4cdc070 0x560bc4ce8f70 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd746ffd700 0 client.18078 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:45.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 -- 172.21.15.62:0/56360387 >> 172.21.15.62:0/56360387 conn(0x560bc4cd0ff0 msgr2=0x560bc4ced020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.746+0000 7fd781707b80 1 -- 172.21.15.62:0/56360387 shutdown_connections 2023-07-14T16:24:45.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd781707b80 1 -- 172.21.15.62:0/56360387 wait complete. 2023-07-14T16:24:45.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd781707b80 20 client.18078 trim_cache size 0 max 16384 2023-07-14T16:24:45.729 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:24:45.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 -- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7500a1270 msgr2=0x7fd7500a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7500a1270 0x7fd7500a3720 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd760015e20 tx=0x7fd760015420 comp rx=0 tx=0).stop 2023-07-14T16:24:45.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 -- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 msgr2=0x7fd768197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:45.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 0x7fd768197720 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd758005910 tx=0x7fd7580057f0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 -- 172.21.15.62:0/2907836137 shutdown_connections 2023-07-14T16:24:45.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7681056d0 0x7fd76819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd768100490 0x7fd768197720 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7500a1270 0x7fd7500a3720 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 --2- 172.21.15.62:0/2907836137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd768102db0 0x7fd768197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:45.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.747+0000 7fd76cb9d700 1 -- 172.21.15.62:0/2907836137 >> 172.21.15.62:0/2907836137 conn(0x7fd7680f9ed0 msgr2=0x7fd7680feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:45.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.748+0000 7fd76cb9d700 1 -- 172.21.15.62:0/2907836137 shutdown_connections 2023-07-14T16:24:45.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:45.748+0000 7fd76cb9d700 1 -- 172.21.15.62:0/2907836137 wait complete. 2023-07-14T16:24:46.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:45 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/4263437904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:24:46.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:45 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/4263437904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:24:46.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:45 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/4263437904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:24:46.743 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:24:46.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:24:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 -- 172.21.15.62:0/3038404877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 msgr2=0x7fdb48101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 --2- 172.21.15.62:0/3038404877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 0x7fdb48101130 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009990 tx=0x7fdb38009ca0 comp rx=0 tx=0).stop 2023-07-14T16:24:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 -- 172.21.15.62:0/3038404877 shutdown_connections 2023-07-14T16:24:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 --2- 172.21.15.62:0/3038404877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb48101670 0x7fdb48107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 --2- 172.21.15.62:0/3038404877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb480ffaf0 0x7fdb480fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 --2- 172.21.15.62:0/3038404877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 0x7fdb48101130 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 -- 172.21.15.62:0/3038404877 >> 172.21.15.62:0/3038404877 conn(0x7fdb480fb130 msgr2=0x7fdb480fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 -- 172.21.15.62:0/3038404877 shutdown_connections 2023-07-14T16:24:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.998+0000 7fdb501ff700 1 -- 172.21.15.62:0/3038404877 wait complete. 2023-07-14T16:24:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.999+0000 7fdb501ff700 1 Processor -- start 2023-07-14T16:24:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.999+0000 7fdb501ff700 1 -- start start 2023-07-14T16:24:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.999+0000 7fdb501ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb480ffaf0 0x7fdb480742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.999+0000 7fdb501ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 0x7fdb48072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.999+0000 7fdb501ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb48101670 0x7fdb48072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:46.999+0000 7fdb501ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480735e0 con 0x7fdb48100ce0 2023-07-14T16:24:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb501ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48073750 con 0x7fdb48101670 2023-07-14T16:24:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb501ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4819f7d0 con 0x7fdb480ffaf0 2023-07-14T16:24:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4df9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb480ffaf0 0x7fdb480742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4df9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb480ffaf0 0x7fdb480742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46772/0 (socket says 172.21.15.62:46772) 2023-07-14T16:24:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4df9b700 1 -- 172.21.15.62:0/2007201471 learned_addr learned my addr 172.21.15.62:0/2007201471 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4d79a700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 0x7fdb48072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:46.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb48101670 0x7fdb48072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:46.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4df9b700 1 -- 172.21.15.62:0/2007201471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb48101670 msgr2=0x7fdb48072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:46.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb48101670 0x7fdb48072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:46.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4df9b700 1 -- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 msgr2=0x7fdb48072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:46.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 0x7fdb48072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:46.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4df9b700 1 -- 172.21.15.62:0/2007201471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb380095f0 con 0x7fdb480ffaf0 2023-07-14T16:24:46.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.000+0000 7fdb4d79a700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 0x7fdb48072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:46.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.001+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb480ffaf0 0x7fdb480742c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400e970 tx=0x7fdb4400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:46.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.001+0000 7fdb3effd700 1 -- 172.21.15.62:0/2007201471 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44009790 con 0x7fdb480ffaf0 2023-07-14T16:24:46.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.001+0000 7fdb501ff700 1 -- 172.21.15.62:0/2007201471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4819fab0 con 0x7fdb480ffaf0 2023-07-14T16:24:46.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.001+0000 7fdb3effd700 1 -- 172.21.15.62:0/2007201471 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb4400eec0 con 0x7fdb480ffaf0 2023-07-14T16:24:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.001+0000 7fdb3effd700 1 -- 172.21.15.62:0/2007201471 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44003c10 con 0x7fdb480ffaf0 2023-07-14T16:24:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.001+0000 7fdb501ff700 1 -- 172.21.15.62:0/2007201471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4819ffc0 con 0x7fdb480ffaf0 2023-07-14T16:24:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.005+0000 7fdb3effd700 1 -- 172.21.15.62:0/2007201471 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdb44003db0 con 0x7fdb480ffaf0 2023-07-14T16:24:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.006+0000 7fdb63ad5b80 1 Processor -- start 2023-07-14T16:24:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.006+0000 7fdb63ad5b80 1 -- start start 2023-07-14T16:24:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.006+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a94960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.006+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94ea0 0x55f575a95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.006+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 0x55f575a96160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.006+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575999d30 con 0x55f575a94550 2023-07-14T16:24:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.006+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5759ecac0 con 0x55f575a94ea0 2023-07-14T16:24:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.006+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5759e5f50 con 0x55f575a95cf0 2023-07-14T16:24:46.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb3effd700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb340a1150 0x7fdb340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:46.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a94960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:46.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb3effd700 1 -- 172.21.15.62:0/2007201471 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb44014070 con 0x7fdb480ffaf0 2023-07-14T16:24:46.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a94960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55234/0 (socket says 172.21.15.62:55234) 2023-07-14T16:24:46.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 -- 172.21.15.62:0/423916152 learned_addr learned my addr 172.21.15.62:0/423916152 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:46.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb340a1150 0x7fdb340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:46.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94ea0 0x55f575a95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4d79a700 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 0x55f575a96160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 -- 172.21.15.62:0/423916152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 msgr2=0x55f575a96160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 0x55f575a96160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:46.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 -- 172.21.15.62:0/423916152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94ea0 msgr2=0x55f575a95760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94ea0 0x55f575a95760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.007+0000 7fdb4e79c700 1 -- 172.21.15.62:0/423916152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f575813e60 con 0x55f575a94550 2023-07-14T16:24:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.008+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a94960 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fdb400194d0 tx=0x7fdb400197e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2578b2be985bd04d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.008+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb340a1150 0x7fdb340a3600 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdb40019b30 tx=0x7fdb4001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.008+0000 7fdb33fff700 1 -- 172.21.15.62:0/423916152 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40025070 con 0x55f575a94550 2023-07-14T16:24:47.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.008+0000 7fdb33fff700 1 -- 172.21.15.62:0/423916152 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb40012380 con 0x55f575a94550 2023-07-14T16:24:47.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.008+0000 7fdb33fff700 1 -- 172.21.15.62:0/423916152 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000e040 con 0x55f575a94550 2023-07-14T16:24:47.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/423916152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 msgr2=0x55f575a94960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a94960 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fdb400194d0 tx=0x7fdb400197e0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/423916152 shutdown_connections 2023-07-14T16:24:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 0x55f575a96160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a94960 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/423916152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94ea0 0x55f575a95760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/423916152 >> 172.21.15.62:0/423916152 conn(0x55f575a89e20 msgr2=0x55f575a8f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/423916152 shutdown_connections 2023-07-14T16:24:47.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.009+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/423916152 wait complete. 2023-07-14T16:24:47.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.010+0000 7fdb63ad5b80 1 -- start start 2023-07-14T16:24:47.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.010+0000 7fdb63ad5b80 1 Processor -- start 2023-07-14T16:24:47.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.010+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a9e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.010+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 0x55f575a9e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.010+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575aa57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575985b40 con 0x55f575a94550 2023-07-14T16:24:47.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575985840 con 0x55f575a95cf0 2023-07-14T16:24:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5758699a0 con 0x55f575a94ea0 2023-07-14T16:24:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb63ad5b80 10 client.0 _ll_register_callbacks cb 0x55f5759837e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a9e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a9e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55242/0 (socket says 172.21.15.62:55242) 2023-07-14T16:24:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4df9b700 1 -- 172.21.15.62:0/1846907573 learned_addr learned my addr 172.21.15.62:0/1846907573 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4d79a700 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 0x55f575a9e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575aa57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4df9b700 1 -- 172.21.15.62:0/1846907573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 msgr2=0x55f575a9e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 0x55f575a9e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4df9b700 1 -- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 msgr2=0x55f575aa57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575aa57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4df9b700 1 -- 172.21.15.62:0/1846907573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb40012c90 con 0x55f575a94550 2023-07-14T16:24:47.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.011+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575aa57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:24:47.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a9e220 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fdb44004800 tx=0x7fdb4400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389916ebb2fc2193 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb317fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:24:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb317fa700 1 -- 172.21.15.62:0/1846907573 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44099660 con 0x55f575a94550 2023-07-14T16:24:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb317fa700 1 -- 172.21.15.62:0/1846907573 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb4409dfa0 con 0x55f575a94550 2023-07-14T16:24:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1846907573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f57593f570 con 0x55f575a94550 2023-07-14T16:24:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb317fa700 1 -- 172.21.15.62:0/1846907573 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb440a23c0 con 0x55f575a94550 2023-07-14T16:24:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb63ad5b80 10 client.18090 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:47.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1846907573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f5757fcce0 con 0x55f575a94550 2023-07-14T16:24:47.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.012+0000 7fdb317fa700 1 -- 172.21.15.62:0/1846907573 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb440c8080 con 0x55f575a94550 2023-07-14T16:24:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.013+0000 7fdb317fa700 1 -- 172.21.15.62:0/1846907573 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb44094b80 con 0x55f575a94550 2023-07-14T16:24:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.013+0000 7fdb63ad5b80 10 client.18090 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:47.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.013+0000 7fdb63ad5b80 10 client.18090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.013+0000 7fdb63ad5b80 20 client.18090 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.013+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f575aa9fa0 0x55f575aac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.013+0000 7fdb63ad5b80 4 client.18090 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.013+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1846907573 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5759def30 con 0x55f575aa9fa0 2023-07-14T16:24:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.013+0000 7fdb4d79a700 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f575aa9fa0 0x55f575aac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.014+0000 7fdb4d79a700 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f575aa9fa0 0x55f575aac3b0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0201b67f7636d1 server_cookie=9bead1323aefbb93 in_seq=0 out_seq=0 2023-07-14T16:24:47.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.014+0000 7fdb317fa700 10 client.18090 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:47.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb317fa700 1 -- 172.21.15.62:0/1846907573 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f5759def30 con 0x55f575aa9fa0 2023-07-14T16:24:47.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb317fa700 10 client.18090 handle_command_reply: tid=0 2023-07-14T16:24:47.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 client.18090 shutdown 2023-07-14T16:24:47.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1846907573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 msgr2=0x55f575a9e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a9e220 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fdb44004800 tx=0x7fdb4400cd50 comp rx=0 tx=0).stop 2023-07-14T16:24:47.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1846907573 shutdown_connections 2023-07-14T16:24:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f575aa9fa0 0x55f575aac3b0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575aa57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575a9e220 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1846907573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 0x55f575a9e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1846907573 >> 172.21.15.62:0/1846907573 conn(0x55f575a89e20 msgr2=0x55f575a8e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.015+0000 7fdb317fa700 0 client.18090 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.016+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1846907573 shutdown_connections 2023-07-14T16:24:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.016+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1846907573 wait complete. 2023-07-14T16:24:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.016+0000 7fdb63ad5b80 20 client.18090 trim_cache size 0 max 16384 2023-07-14T16:24:47.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.040+0000 7fdb63ad5b80 1 Processor -- start 2023-07-14T16:24:47.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.040+0000 7fdb63ad5b80 1 -- start start 2023-07-14T16:24:47.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94550 0x55f575a9c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94ea0 0x55f575aa5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 0x55f575aabfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575a95430 con 0x55f575a94ea0 2023-07-14T16:24:47.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575a955f0 con 0x55f575a94550 2023-07-14T16:24:47.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575a89a40 con 0x55f575a95cf0 2023-07-14T16:24:47.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb4df9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94550 0x55f575a9c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb4d79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94ea0 0x55f575aa5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb4df9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94550 0x55f575a9c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52808/0 (socket says 172.21.15.62:52808) 2023-07-14T16:24:47.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb4df9b700 1 -- 172.21.15.62:0/3918132337 learned_addr learned my addr 172.21.15.62:0/3918132337 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:47.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 0x55f575aabfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.041+0000 7fdb4df9b700 1 -- 172.21.15.62:0/3918132337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 msgr2=0x55f575aabfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 0x55f575aabfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb4df9b700 1 -- 172.21.15.62:0/3918132337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94ea0 msgr2=0x55f575aa5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94ea0 0x55f575aa5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb4df9b700 1 -- 172.21.15.62:0/3918132337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f575a9d1a0 con 0x55f575a94550 2023-07-14T16:24:47.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb4d79a700 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94ea0 0x55f575aa5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:47.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94550 0x55f575a9c110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb440969e0 tx=0x7fdb44000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa22729e6ea01007 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb3cff9700 1 -- 172.21.15.62:0/3918132337 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4409dfa0 con 0x55f575a94550 2023-07-14T16:24:47.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb3cff9700 1 -- 172.21.15.62:0/3918132337 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f57593f570 con 0x55f575a94550 2023-07-14T16:24:47.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.042+0000 7fdb3cff9700 1 -- 172.21.15.62:0/3918132337 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44099460 con 0x55f575a94550 2023-07-14T16:24:47.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/3918132337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94550 msgr2=0x55f575a9c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94550 0x55f575a9c110 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb440969e0 tx=0x7fdb44000bc0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/3918132337 shutdown_connections 2023-07-14T16:24:47.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a95cf0 0x55f575aabfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a94550 0x55f575a9c110 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/3918132337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94ea0 0x55f575aa5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/3918132337 >> 172.21.15.62:0/3918132337 conn(0x55f575a89e20 msgr2=0x55f575a8e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/3918132337 shutdown_connections 2023-07-14T16:24:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/3918132337 wait complete. 2023-07-14T16:24:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.043+0000 7fdb63ad5b80 1 -- start start 2023-07-14T16:24:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.044+0000 7fdb63ad5b80 1 Processor -- start 2023-07-14T16:24:47.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.044+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575aa2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.044+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 0x55f575aa2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.044+0000 7fdb63ad5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575afde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.044+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575aa58b0 con 0x55f575a94550 2023-07-14T16:24:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.044+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575aa5a20 con 0x55f575a95cf0 2023-07-14T16:24:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.044+0000 7fdb63ad5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f575afe390 con 0x55f575a94ea0 2023-07-14T16:24:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.044+0000 7fdb63ad5b80 10 client.0 _ll_register_callbacks cb 0x7fdb44099ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575aa2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575aa2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55258/0 (socket says 172.21.15.62:55258) 2023-07-14T16:24:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4df9b700 1 -- 172.21.15.62:0/1267026018 learned_addr learned my addr 172.21.15.62:0/1267026018 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575afde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4d79a700 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 0x55f575aa2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4e79c700 1 -- 172.21.15.62:0/1267026018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 msgr2=0x55f575aa2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 0x55f575aa2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4e79c700 1 -- 172.21.15.62:0/1267026018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 msgr2=0x55f575aa2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575aa2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4e79c700 1 -- 172.21.15.62:0/1267026018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb48108860 con 0x55f575a95cf0 2023-07-14T16:24:47.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.045+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575aa2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:47.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.046+0000 7fdb4e79c700 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575afde50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009650 tx=0x7fdb4001ad60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc7ba72a426898f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.046+0000 7fdb317fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:24:47.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.046+0000 7fdb317fa700 1 -- 172.21.15.62:0/1267026018 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40025040 con 0x55f575a95cf0 2023-07-14T16:24:47.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.046+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1267026018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb40012380 con 0x55f575a95cf0 2023-07-14T16:24:47.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.046+0000 7fdb317fa700 1 -- 172.21.15.62:0/1267026018 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb40007500 con 0x55f575a95cf0 2023-07-14T16:24:47.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.046+0000 7fdb317fa700 1 -- 172.21.15.62:0/1267026018 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4000e040 con 0x55f575a95cf0 2023-07-14T16:24:47.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.046+0000 7fdb63ad5b80 10 client.27725 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:47.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.046+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1267026018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f5757fcce0 con 0x55f575a95cf0 2023-07-14T16:24:47.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb317fa700 1 -- 172.21.15.62:0/1267026018 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb40012380 con 0x55f575a95cf0 2023-07-14T16:24:47.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb317fa700 1 -- 172.21.15.62:0/1267026018 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb40007d20 con 0x55f575a95cf0 2023-07-14T16:24:47.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb63ad5b80 10 client.27725 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:47.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb63ad5b80 10 client.27725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:47.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb63ad5b80 20 client.27725 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:47.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f575affcb0 0x55f575b020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb63ad5b80 4 client.27725 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:24:47.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1267026018 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f5759a10d0 con 0x55f575affcb0 2023-07-14T16:24:47.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.047+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f575affcb0 0x55f575b020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.048+0000 7fdb317fa700 10 client.27725 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:47.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.048+0000 7fdb4df9b700 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f575affcb0 0x55f575b020c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b83ce5f06c03cf server_cookie=720577eec1f5b54a in_seq=0 out_seq=0 2023-07-14T16:24:47.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb317fa700 1 -- 172.21.15.62:0/1267026018 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f5759a10d0 con 0x55f575affcb0 2023-07-14T16:24:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb317fa700 10 client.27725 handle_command_reply: tid=0 2023-07-14T16:24:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 client.27725 shutdown 2023-07-14T16:24:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 msgr2=0x55f575afde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575afde50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009650 tx=0x7fdb4001ad60 comp rx=0 tx=0).stop 2023-07-14T16:24:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1267026018 shutdown_connections 2023-07-14T16:24:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f575affcb0 0x55f575b020c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f575a95cf0 0x55f575afde50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f575a94550 0x55f575aa2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 --2- 172.21.15.62:0/1267026018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f575a94ea0 0x55f575aa2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1267026018 >> 172.21.15.62:0/1267026018 conn(0x55f575a89e20 msgr2=0x55f575a9c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.049+0000 7fdb317fa700 0 client.27725 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.050+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1267026018 shutdown_connections 2023-07-14T16:24:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.050+0000 7fdb63ad5b80 1 -- 172.21.15.62:0/1267026018 wait complete. 2023-07-14T16:24:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.050+0000 7fdb63ad5b80 20 client.27725 trim_cache size 0 max 16384 2023-07-14T16:24:47.047 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:24:47.047 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:24:47.047 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "19dd4fd5-3742-4089-9695-d6882e32b3bd", 2023-07-14T16:24:47.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:24:47.047 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:24:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.050+0000 7fdb501ff700 1 -- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb340a1150 msgr2=0x7fdb340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.050+0000 7fdb501ff700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb340a1150 0x7fdb340a3600 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdb40019b30 tx=0x7fdb4001d000 comp rx=0 tx=0).stop 2023-07-14T16:24:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.050+0000 7fdb501ff700 1 -- 172.21.15.62:0/2007201471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb480ffaf0 msgr2=0x7fdb480742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.050+0000 7fdb501ff700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb480ffaf0 0x7fdb480742c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400e970 tx=0x7fdb4400ec80 comp rx=0 tx=0).stop 2023-07-14T16:24:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.051+0000 7fdb501ff700 1 -- 172.21.15.62:0/2007201471 shutdown_connections 2023-07-14T16:24:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.051+0000 7fdb501ff700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb340a1150 0x7fdb340a3600 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.051+0000 7fdb501ff700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb48101670 0x7fdb48072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.051+0000 7fdb501ff700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb480ffaf0 0x7fdb480742c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.051+0000 7fdb501ff700 1 --2- 172.21.15.62:0/2007201471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb48100ce0 0x7fdb48072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.051+0000 7fdb501ff700 1 -- 172.21.15.62:0/2007201471 >> 172.21.15.62:0/2007201471 conn(0x7fdb480fb130 msgr2=0x7fdb480fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.051+0000 7fdb501ff700 1 -- 172.21.15.62:0/2007201471 shutdown_connections 2023-07-14T16:24:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.051+0000 7fdb501ff700 1 -- 172.21.15.62:0/2007201471 wait complete. 2023-07-14T16:24:47.050 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:46 smithi062 ceph-mon[95543]: pgmap v631: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:24:47.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:24:47.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:46 smithi081 ceph-mon[127795]: pgmap v631: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:24:47.151 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:24:47.060+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v189>,<0x10000000003/client.0 v190>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v202>,<0x10000000001/sv_0 v216>,<0x10000000000/_nogroup v210>,<0x1/volumes v200>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:24:47.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:46 smithi164 ceph-mon[97427]: pgmap v631: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:24:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.309+0000 7f35789ff700 1 -- 172.21.15.62:0/3563243271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357406d6c0 msgr2=0x7f3574116d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.309+0000 7f35789ff700 1 --2- 172.21.15.62:0/3563243271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357406d6c0 0x7f3574116d60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f356c009920 tx=0x7f356c009c30 comp rx=0 tx=0).stop 2023-07-14T16:24:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.310+0000 7f35789ff700 1 -- 172.21.15.62:0/3563243271 shutdown_connections 2023-07-14T16:24:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.310+0000 7f35789ff700 1 --2- 172.21.15.62:0/3563243271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357406d6c0 0x7f3574116d60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.310+0000 7f35789ff700 1 --2- 172.21.15.62:0/3563243271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357406baf0 0x7f357406bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.310+0000 7f35789ff700 1 --2- 172.21.15.62:0/3563243271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f357406cd30 0x7f357406d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.310+0000 7f35789ff700 1 -- 172.21.15.62:0/3563243271 >> 172.21.15.62:0/3563243271 conn(0x7f357406a180 msgr2=0x7f357406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.310+0000 7f35789ff700 1 -- 172.21.15.62:0/3563243271 shutdown_connections 2023-07-14T16:24:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.310+0000 7f35789ff700 1 -- 172.21.15.62:0/3563243271 wait complete. 2023-07-14T16:24:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.311+0000 7f35789ff700 1 Processor -- start 2023-07-14T16:24:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.312+0000 7f35789ff700 1 -- start start 2023-07-14T16:24:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.312+0000 7f35789ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f357406baf0 0x7f35741a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.312+0000 7f35789ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357406cd30 0x7f35741a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.312+0000 7f35789ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357406d6c0 0x7f35741abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.312+0000 7f35789ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35741ac130 con 0x7f357406baf0 2023-07-14T16:24:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.312+0000 7f35789ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35741ac270 con 0x7f357406d6c0 2023-07-14T16:24:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.312+0000 7f35789ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35741ac400 con 0x7f357406cd30 2023-07-14T16:24:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.312+0000 7f357259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f357406baf0 0x7f35741a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357406d6c0 0x7f35741abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f357259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f357406baf0 0x7f35741a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55270/0 (socket says 172.21.15.62:55270) 2023-07-14T16:24:47.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f357259c700 1 -- 172.21.15.62:0/1093411194 learned_addr learned my addr 172.21.15.62:0/1093411194 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:47.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f3572d9d700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357406d6c0 0x7f35741abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52830/0 (socket says 172.21.15.62:52830) 2023-07-14T16:24:47.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f3571d9b700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357406cd30 0x7f35741a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f357259c700 1 -- 172.21.15.62:0/1093411194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357406cd30 msgr2=0x7f35741a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f357259c700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357406cd30 0x7f35741a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f357259c700 1 -- 172.21.15.62:0/1093411194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357406d6c0 msgr2=0x7f35741abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f357259c700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357406d6c0 0x7f35741abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f357259c700 1 -- 172.21.15.62:0/1093411194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c009520 con 0x7f357406baf0 2023-07-14T16:24:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.313+0000 7f357259c700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f357406baf0 0x7f35741a45b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f356400efb0 tx=0x7f356400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.314+0000 7f35637fe700 1 -- 172.21.15.62:0/1093411194 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564018070 con 0x7f357406baf0 2023-07-14T16:24:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.314+0000 7f35789ff700 1 -- 172.21.15.62:0/1093411194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35741ac680 con 0x7f357406baf0 2023-07-14T16:24:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.314+0000 7f35789ff700 1 -- 172.21.15.62:0/1093411194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35741acbe0 con 0x7f357406baf0 2023-07-14T16:24:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.314+0000 7f35637fe700 1 -- 172.21.15.62:0/1093411194 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f356400f040 con 0x7f357406baf0 2023-07-14T16:24:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.314+0000 7f35637fe700 1 -- 172.21.15.62:0/1093411194 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564004280 con 0x7f357406baf0 2023-07-14T16:24:47.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.319+0000 7f358c2d3b80 1 Processor -- start 2023-07-14T16:24:47.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.319+0000 7f35637fe700 1 -- 172.21.15.62:0/1093411194 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f356400ca20 con 0x7f357406baf0 2023-07-14T16:24:47.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.319+0000 7f358c2d3b80 1 -- start start 2023-07-14T16:24:47.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.319+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.319+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460068e20 0x5594600696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x55946006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945ff6dcb0 con 0x559460068e20 2023-07-14T16:24:47.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945ffc0a40 con 0x559460069c70 2023-07-14T16:24:47.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945ffb9ed0 con 0x5594600684d0 2023-07-14T16:24:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f357259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460068e20 0x5594600696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600688e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46832/0 (socket says 172.21.15.62:46832) 2023-07-14T16:24:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f3572d9d700 1 -- 172.21.15.62:0/1601461604 learned_addr learned my addr 172.21.15.62:0/1601461604 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f3571d9b700 1 --2- 172.21.15.62:0/1601461604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x55946006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f357259c700 1 -- 172.21.15.62:0/1601461604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 msgr2=0x5594600688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f357259c700 1 --2- 172.21.15.62:0/1601461604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.320+0000 7f357259c700 1 -- 172.21.15.62:0/1601461604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 msgr2=0x55946006a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f357259c700 1 --2- 172.21.15.62:0/1601461604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x55946006a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f357259c700 1 -- 172.21.15.62:0/1601461604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55945fde7de0 con 0x559460068e20 2023-07-14T16:24:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f35637fe700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f355c0a11d0 0x7f355c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f357259c700 1 --2- 172.21.15.62:0/1601461604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460068e20 0x5594600696e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f35640ca190 tx=0x7f35640d2d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816ab5008379e3f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f35637fe700 1 -- 172.21.15.62:0/1093411194 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f35640cc740 con 0x7f357406baf0 2023-07-14T16:24:47.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f3572d9d700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f355c0a11d0 0x7f355c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f3560ff9700 1 -- 172.21.15.62:0/1601461604 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35640b9070 con 0x559460068e20 2023-07-14T16:24:47.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f3560ff9700 1 -- 172.21.15.62:0/1601461604 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3564003a00 con 0x559460068e20 2023-07-14T16:24:47.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.321+0000 7f3560ff9700 1 -- 172.21.15.62:0/1601461604 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35640d26d0 con 0x559460068e20 2023-07-14T16:24:47.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.322+0000 7f3572d9d700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f355c0a11d0 0x7f355c0a3680 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f356c009920 tx=0x7f356c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.322+0000 7f358c2d3b80 1 -- 172.21.15.62:0/1601461604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460068e20 msgr2=0x5594600696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.322+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/1601461604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460068e20 0x5594600696e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f35640ca190 tx=0x7f35640d2d40 comp rx=0 tx=0).stop 2023-07-14T16:24:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.322+0000 7f358c2d3b80 1 -- 172.21.15.62:0/1601461604 shutdown_connections 2023-07-14T16:24:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.322+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/1601461604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x55946006a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.322+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/1601461604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.322+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/1601461604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460068e20 0x5594600696e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.322+0000 7f358c2d3b80 1 -- 172.21.15.62:0/1601461604 >> 172.21.15.62:0/1601461604 conn(0x55946005dda0 msgr2=0x5594600631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.323+0000 7f358c2d3b80 1 -- 172.21.15.62:0/1601461604 shutdown_connections 2023-07-14T16:24:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.323+0000 7f358c2d3b80 1 -- 172.21.15.62:0/1601461604 wait complete. 2023-07-14T16:24:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.323+0000 7f358c2d3b80 1 -- start start 2023-07-14T16:24:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.324+0000 7f358c2d3b80 1 Processor -- start 2023-07-14T16:24:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.324+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x559460072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.324+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 0x559460072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.324+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x5594600796d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.324+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945ff59ac0 con 0x559460069c70 2023-07-14T16:24:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.324+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945ff597c0 con 0x559460068e20 2023-07-14T16:24:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.324+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55945fe3d920 con 0x5594600684d0 2023-07-14T16:24:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.324+0000 7f358c2d3b80 10 client.0 _ll_register_callbacks cb 0x55945ff57760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x5594600796d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f357259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x559460072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x5594600796d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55284/0 (socket says 172.21.15.62:55284) 2023-07-14T16:24:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f3572d9d700 1 -- 172.21.15.62:0/14127763 learned_addr learned my addr 172.21.15.62:0/14127763 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f357259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x559460072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46842/0 (socket says 172.21.15.62:46842) 2023-07-14T16:24:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f3571d9b700 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 0x559460072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f357259c700 1 -- 172.21.15.62:0/14127763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 msgr2=0x559460072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f357259c700 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 0x559460072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f357259c700 1 -- 172.21.15.62:0/14127763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 msgr2=0x5594600796d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f357259c700 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x5594600796d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f357259c700 1 -- 172.21.15.62:0/14127763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35741acbe0 con 0x5594600684d0 2023-07-14T16:24:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f3572d9d700 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x5594600796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f357259c700 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x559460072150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3564003ff0 tx=0x7f3564004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac02995f0d51ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.325+0000 7f3559ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:24:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.326+0000 7f3559ffb700 1 -- 172.21.15.62:0/14127763 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35640b9070 con 0x5594600684d0 2023-07-14T16:24:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.326+0000 7f3559ffb700 1 -- 172.21.15.62:0/14127763 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3564099b90 con 0x5594600684d0 2023-07-14T16:24:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.326+0000 7f358c2d3b80 1 -- 172.21.15.62:0/14127763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55945ffb2eb0 con 0x5594600684d0 2023-07-14T16:24:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.326+0000 7f3559ffb700 1 -- 172.21.15.62:0/14127763 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35640b2020 con 0x5594600684d0 2023-07-14T16:24:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.326+0000 7f358c2d3b80 10 client.27625 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.326+0000 7f358c2d3b80 1 -- 172.21.15.62:0/14127763 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55945fdd0c60 con 0x5594600684d0 2023-07-14T16:24:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.326+0000 7f3559ffb700 1 -- 172.21.15.62:0/14127763 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55945ffb2eb0 con 0x5594600684d0 2023-07-14T16:24:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.326+0000 7f3559ffb700 1 -- 172.21.15.62:0/14127763 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f35640b2180 con 0x5594600684d0 2023-07-14T16:24:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.327+0000 7f358c2d3b80 10 client.27625 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.327+0000 7f358c2d3b80 10 client.27625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.327+0000 7f358c2d3b80 20 client.27625 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.327+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55946007da00 0x55946007fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.327+0000 7f358c2d3b80 4 client.27625 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.327+0000 7f358c2d3b80 1 -- 172.21.15.62:0/14127763 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55945ff75050 con 0x55946007da00 2023-07-14T16:24:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.327+0000 7f3571d9b700 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55946007da00 0x55946007fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.328+0000 7f3571d9b700 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55946007da00 0x55946007fde0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cee595414b3e62 server_cookie=e77d3c64001e607e in_seq=0 out_seq=0 2023-07-14T16:24:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.328+0000 7f3559ffb700 10 client.27625 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.328+0000 7f3559ffb700 1 -- 172.21.15.62:0/14127763 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55945ff75050 con 0x55946007da00 2023-07-14T16:24:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.328+0000 7f3559ffb700 10 client.27625 handle_command_reply: tid=0 2023-07-14T16:24:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.328+0000 7f358c2d3b80 1 client.27625 shutdown 2023-07-14T16:24:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 -- 172.21.15.62:0/14127763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 msgr2=0x559460072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x559460072150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3564003ff0 tx=0x7f3564004ad0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 -- 172.21.15.62:0/14127763 shutdown_connections 2023-07-14T16:24:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x5594600796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x559460072150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55946007da00 0x55946007fde0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/14127763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 0x559460072690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f3559ffb700 0 client.27625 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 -- 172.21.15.62:0/14127763 >> 172.21.15.62:0/14127763 conn(0x55946005dda0 msgr2=0x5594600628b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 -- 172.21.15.62:0/14127763 shutdown_connections 2023-07-14T16:24:47.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 1 -- 172.21.15.62:0/14127763 wait complete. 2023-07-14T16:24:47.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.329+0000 7f358c2d3b80 20 client.27625 trim_cache size 0 max 16384 2023-07-14T16:24:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.349+0000 7f358c2d3b80 1 Processor -- start 2023-07-14T16:24:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.349+0000 7f358c2d3b80 1 -- start start 2023-07-14T16:24:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 0x559460079690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x55946006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559460071d20 con 0x559460069c70 2023-07-14T16:24:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559460064090 con 0x559460068e20 2023-07-14T16:24:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559460072f10 con 0x5594600684d0 2023-07-14T16:24:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x55946006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3572d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x55946006cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55288/0 (socket says 172.21.15.62:55288) 2023-07-14T16:24:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3572d9d700 1 -- 172.21.15.62:0/194649810 learned_addr learned my addr 172.21.15.62:0/194649810 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f357259c700 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3571d9b700 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 0x559460079690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3572d9d700 1 -- 172.21.15.62:0/194649810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 msgr2=0x5594600628b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3572d9d700 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600628b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3572d9d700 1 -- 172.21.15.62:0/194649810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 msgr2=0x559460079690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3572d9d700 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 0x559460079690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.350+0000 7f3572d9d700 1 -- 172.21.15.62:0/194649810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55945fe0e250 con 0x559460069c70 2023-07-14T16:24:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.351+0000 7f3572d9d700 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x55946006cb30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f356c00ff80 tx=0x7f356c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f19687cfdd6f5f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.351+0000 7f35617fa700 1 -- 172.21.15.62:0/194649810 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00bb70 con 0x559460069c70 2023-07-14T16:24:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.351+0000 7f35617fa700 1 -- 172.21.15.62:0/194649810 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f356c01d040 con 0x559460069c70 2023-07-14T16:24:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.351+0000 7f35617fa700 1 -- 172.21.15.62:0/194649810 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c003bc0 con 0x559460069c70 2023-07-14T16:24:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 -- 172.21.15.62:0/194649810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 msgr2=0x55946006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x55946006cb30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f356c00ff80 tx=0x7f356c021000 comp rx=0 tx=0).stop 2023-07-14T16:24:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 -- 172.21.15.62:0/194649810 shutdown_connections 2023-07-14T16:24:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559460069c70 0x55946006cb30 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5594600684d0 0x5594600628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/194649810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460068e20 0x559460079690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 -- 172.21.15.62:0/194649810 >> 172.21.15.62:0/194649810 conn(0x55946005dda0 msgr2=0x559460079f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 -- 172.21.15.62:0/194649810 shutdown_connections 2023-07-14T16:24:47.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 -- 172.21.15.62:0/194649810 wait complete. 2023-07-14T16:24:47.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.352+0000 7f358c2d3b80 1 -- start start 2023-07-14T16:24:47.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.353+0000 7f358c2d3b80 1 Processor -- start 2023-07-14T16:24:47.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.353+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594600684d0 0x559460078e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.353+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559460068e20 0x5594600793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.353+0000 7f358c2d3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x559460075900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.353+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559460075e40 con 0x5594600684d0 2023-07-14T16:24:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.353+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559460075fb0 con 0x559460069c70 2023-07-14T16:24:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.353+0000 7f358c2d3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559460076170 con 0x559460068e20 2023-07-14T16:24:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.353+0000 7f358c2d3b80 10 client.0 _ll_register_callbacks cb 0x7f3568009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f357259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594600684d0 0x559460078e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559460068e20 0x5594600793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559460068e20 0x5594600793a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46846/0 (socket says 172.21.15.62:46846) 2023-07-14T16:24:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 -- 172.21.15.62:0/3531851858 learned_addr learned my addr 172.21.15.62:0/3531851858 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3572d9d700 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x559460075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 -- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 msgr2=0x559460075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x559460075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 -- 172.21.15.62:0/3531851858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594600684d0 msgr2=0x559460078e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:47.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594600684d0 0x559460078e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 -- 172.21.15.62:0/3531851858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3574117620 con 0x559460068e20 2023-07-14T16:24:47.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3572d9d700 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x559460075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:47.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3571d9b700 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559460068e20 0x5594600793a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x559460080690 tx=0x7f356801fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d4ae5040ad94991 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:47.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.354+0000 7f3559ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:24:47.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.355+0000 7f3559ffb700 1 -- 172.21.15.62:0/3531851858 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568004460 con 0x559460068e20 2023-07-14T16:24:47.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.355+0000 7f358c2d3b80 1 -- 172.21.15.62:0/3531851858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55945ff134f0 con 0x559460068e20 2023-07-14T16:24:47.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.355+0000 7f3559ffb700 1 -- 172.21.15.62:0/3531851858 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3568010770 con 0x559460068e20 2023-07-14T16:24:47.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.355+0000 7f3559ffb700 1 -- 172.21.15.62:0/3531851858 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f356801f5e0 con 0x559460068e20 2023-07-14T16:24:47.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.355+0000 7f358c2d3b80 10 client.27631 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:47.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.355+0000 7f358c2d3b80 1 -- 172.21.15.62:0/3531851858 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55945fde62b0 con 0x559460068e20 2023-07-14T16:24:47.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f3559ffb700 1 -- 172.21.15.62:0/3531851858 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55945ff134f0 con 0x559460068e20 2023-07-14T16:24:47.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f3559ffb700 1 -- 172.21.15.62:0/3531851858 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3568031070 con 0x559460068e20 2023-07-14T16:24:47.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f358c2d3b80 10 client.27631 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:47.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f358c2d3b80 10 client.27631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f358c2d3b80 20 client.27631 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5594600d2df0 0x5594600d5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f358c2d3b80 4 client.27631 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:24:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f358c2d3b80 1 -- 172.21.15.62:0/3531851858 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55945fe08f70 con 0x5594600d2df0 2023-07-14T16:24:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.356+0000 7f357259c700 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5594600d2df0 0x5594600d5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f357259c700 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5594600d2df0 0x5594600d5200 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=595fceaf5c76ef81 server_cookie=c13b26291fa4603 in_seq=0 out_seq=0 2023-07-14T16:24:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f3559ffb700 10 client.27631 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f3559ffb700 1 -- 172.21.15.62:0/3531851858 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55945fe08f70 con 0x5594600d2df0 2023-07-14T16:24:47.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f3559ffb700 10 client.27631 handle_command_reply: tid=0 2023-07-14T16:24:47.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 client.27631 shutdown 2023-07-14T16:24:47.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 -- 172.21.15.62:0/3531851858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559460068e20 msgr2=0x5594600793a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559460068e20 0x5594600793a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x559460080690 tx=0x7f356801fe60 comp rx=0 tx=0).stop 2023-07-14T16:24:47.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 -- 172.21.15.62:0/3531851858 shutdown_connections 2023-07-14T16:24:47.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5594600d2df0 0x5594600d5200 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559460069c70 0x559460075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5594600684d0 0x559460078e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f3559ffb700 0 client.27631 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:47.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 --2- 172.21.15.62:0/3531851858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559460068e20 0x5594600793a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.357+0000 7f358c2d3b80 1 -- 172.21.15.62:0/3531851858 >> 172.21.15.62:0/3531851858 conn(0x55946005dda0 msgr2=0x55946006cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.358+0000 7f358c2d3b80 1 -- 172.21.15.62:0/3531851858 shutdown_connections 2023-07-14T16:24:47.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.358+0000 7f358c2d3b80 1 -- 172.21.15.62:0/3531851858 wait complete. 2023-07-14T16:24:47.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.358+0000 7f358c2d3b80 20 client.27631 trim_cache size 0 max 16384 2023-07-14T16:24:47.342 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:24:47.342 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:24:47.342 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:24:47.342 INFO:teuthology.orchestra.run.smithi062.stdout: "19dd4fd5-3742-4089-9695-d6882e32b3bd": { 2023-07-14T16:24:47.342 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:24:47.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "19dd4fd5-3742-4089-9695-d6882e32b3bd", 2023-07-14T16:24:47.343 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:24:47.343 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:24:47.343 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:24:47.343 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:24:47.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.358+0000 7f35789ff700 1 -- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f355c0a11d0 msgr2=0x7f355c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.358+0000 7f35789ff700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f355c0a11d0 0x7f355c0a3680 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f356c009920 tx=0x7f356c019040 comp rx=0 tx=0).stop 2023-07-14T16:24:47.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.358+0000 7f35789ff700 1 -- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f357406baf0 msgr2=0x7f35741a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:47.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.358+0000 7f35789ff700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f357406baf0 0x7f35741a45b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f356400efb0 tx=0x7f356400c3a0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.359+0000 7f35789ff700 1 -- 172.21.15.62:0/1093411194 shutdown_connections 2023-07-14T16:24:47.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.359+0000 7f35789ff700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f355c0a11d0 0x7f355c0a3680 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.359+0000 7f35789ff700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357406d6c0 0x7f35741abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.359+0000 7f35789ff700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f357406baf0 0x7f35741a45b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.359+0000 7f35789ff700 1 --2- 172.21.15.62:0/1093411194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357406cd30 0x7f35741a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:47.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.359+0000 7f35789ff700 1 -- 172.21.15.62:0/1093411194 >> 172.21.15.62:0/1093411194 conn(0x7f357406a180 msgr2=0x7f3574107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:47.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.359+0000 7f35789ff700 1 -- 172.21.15.62:0/1093411194 shutdown_connections 2023-07-14T16:24:47.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:24:47.359+0000 7f35789ff700 1 -- 172.21.15.62:0/1093411194 wait complete. 2023-07-14T16:24:47.354 INFO:tasks.cephfs.filesystem:scrub status for tag:19dd4fd5-3742-4089-9695-d6882e32b3bd - {'path': '/', 'tag': '19dd4fd5-3742-4089-9695-d6882e32b3bd', 'options': 'recursive,force'} 2023-07-14T16:24:48.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:24:48.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:24:48.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:24:48.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:24:48.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:24:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:47 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:24:48.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:24:48.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:24:48.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:24:48.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:24:48.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:47 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:24:48.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:24:48.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:24:48.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:24:48.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:24:48.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:48.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:47 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:24:49.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:48 smithi062 ceph-mon[95543]: pgmap v632: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:49.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:48 smithi081 ceph-mon[127795]: pgmap v632: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:49.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:48 smithi164 ceph-mon[97427]: pgmap v632: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:24:51.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:50 smithi062 ceph-mon[95543]: pgmap v633: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:24:51.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:50 smithi081 ceph-mon[127795]: pgmap v633: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:24:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:50 smithi164 ceph-mon[97427]: pgmap v633: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:24:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 -- 172.21.15.62:0/2964792990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 msgr2=0x7fe848107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/2964792990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 0x7fe848107f50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe84000b0a0 tx=0x7fe84000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 -- 172.21.15.62:0/2964792990 shutdown_connections 2023-07-14T16:24:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/2964792990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 0x7fe848107f50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/2964792990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8480ffb10 0x7fe8480fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/2964792990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe848100d00 0x7fe848101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 -- 172.21.15.62:0/2964792990 >> 172.21.15.62:0/2964792990 conn(0x7fe8480fb130 msgr2=0x7fe8480fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 -- 172.21.15.62:0/2964792990 shutdown_connections 2023-07-14T16:24:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.005+0000 7fe84f9ff700 1 -- 172.21.15.62:0/2964792990 wait complete. 2023-07-14T16:24:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.007+0000 7fe84f9ff700 1 Processor -- start 2023-07-14T16:24:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.007+0000 7fe84f9ff700 1 -- start start 2023-07-14T16:24:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.007+0000 7fe84f9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8480ffb10 0x7fe848072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.007+0000 7fe84f9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe848100d00 0x7fe848072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.007+0000 7fe84f9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 0x7fe848073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.007+0000 7fe84f9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84819f910 con 0x7fe848101690 2023-07-14T16:24:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.007+0000 7fe84f9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84819fa50 con 0x7fe8480ffb10 2023-07-14T16:24:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.007+0000 7fe84f9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84819fb90 con 0x7fe848100d00 2023-07-14T16:24:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8480ffb10 0x7fe848072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 0x7fe848073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84cf9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe848100d00 0x7fe848072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8480ffb10 0x7fe848072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52876/0 (socket says 172.21.15.62:52876) 2023-07-14T16:24:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 -- 172.21.15.62:0/392910772 learned_addr learned my addr 172.21.15.62:0/392910772 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 -- 172.21.15.62:0/392910772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe848100d00 msgr2=0x7fe848072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe848100d00 0x7fe848072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 -- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 msgr2=0x7fe848073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 0x7fe848073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 -- 172.21.15.62:0/392910772 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe840009d00 con 0x7fe8480ffb10 2023-07-14T16:24:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84df9c700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 0x7fe848073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:24:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.008+0000 7fe84d79b700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8480ffb10 0x7fe848072900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe84400d980 tx=0x7fe84400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.009+0000 7fe83e7fc700 1 -- 172.21.15.62:0/392910772 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400f890 con 0x7fe8480ffb10 2023-07-14T16:24:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.009+0000 7fe84f9ff700 1 -- 172.21.15.62:0/392910772 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84819fe20 con 0x7fe8480ffb10 2023-07-14T16:24:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.009+0000 7fe83e7fc700 1 -- 172.21.15.62:0/392910772 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe84400fe80 con 0x7fe8480ffb10 2023-07-14T16:24:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.009+0000 7fe83e7fc700 1 -- 172.21.15.62:0/392910772 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844019360 con 0x7fe8480ffb10 2023-07-14T16:24:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.009+0000 7fe84f9ff700 1 -- 172.21.15.62:0/392910772 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8481a0250 con 0x7fe8480ffb10 2023-07-14T16:24:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.013+0000 7fe83e7fc700 1 -- 172.21.15.62:0/392910772 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe84400f9f0 con 0x7fe8480ffb10 2023-07-14T16:24:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.013+0000 7fe86341db80 1 Processor -- start 2023-07-14T16:24:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe86341db80 1 -- start start 2023-07-14T16:24:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe86341db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb878796b0 0x55cb87a1d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe86341db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a1e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe86341db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb87a1e6e0 0x55cb87a1eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe86341db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87922c50 con 0x55cb878796b0 2023-07-14T16:24:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe86341db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb879759e0 con 0x55cb87a1e6e0 2023-07-14T16:24:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe86341db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8796ee70 con 0x55cb87a1d840 2023-07-14T16:24:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe84df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb878796b0 0x55cb87a1d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.014+0000 7fe84df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb878796b0 0x55cb87a1d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55340/0 (socket says 172.21.15.62:55340) 2023-07-14T16:24:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84df9c700 1 -- 172.21.15.62:0/3032618973 learned_addr learned my addr 172.21.15.62:0/3032618973 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84d79b700 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a1e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb87a1e6e0 0x55cb87a1eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84df9c700 1 -- 172.21.15.62:0/3032618973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 msgr2=0x55cb87a1e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84df9c700 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a1e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84df9c700 1 -- 172.21.15.62:0/3032618973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb87a1e6e0 msgr2=0x55cb87a1eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84df9c700 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb87a1e6e0 0x55cb87a1eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84df9c700 1 -- 172.21.15.62:0/3032618973 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb8779cd80 con 0x55cb878796b0 2023-07-14T16:24:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe83e7fc700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe8340a11d0 0x7fe8340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe83e7fc700 1 -- 172.21.15.62:0/392910772 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe8440cc2f0 con 0x7fe8480ffb10 2023-07-14T16:24:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84d79b700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe8340a11d0 0x7fe8340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.015+0000 7fe84df9c700 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb878796b0 0x55cb87a1d300 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe84000b0a0 tx=0x7fe840009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaeb82866bfaefc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.016+0000 7fe8337fe700 1 -- 172.21.15.62:0/3032618973 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84000e040 con 0x55cb878796b0 2023-07-14T16:24:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.016+0000 7fe8337fe700 1 -- 172.21.15.62:0/3032618973 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8400092e0 con 0x55cb878796b0 2023-07-14T16:24:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.016+0000 7fe8337fe700 1 -- 172.21.15.62:0/3032618973 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8400052a0 con 0x55cb878796b0 2023-07-14T16:24:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 -- 172.21.15.62:0/3032618973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb878796b0 msgr2=0x55cb87a1d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb878796b0 0x55cb87a1d300 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe84000b0a0 tx=0x7fe840009c50 comp rx=0 tx=0).stop 2023-07-14T16:24:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe84d79b700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe8340a11d0 0x7fe8340a3680 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe8440a15d0 tx=0x7fe8440cfcf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 -- 172.21.15.62:0/3032618973 shutdown_connections 2023-07-14T16:24:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb87a1e6e0 0x55cb87a1eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb878796b0 0x55cb87a1d300 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 --2- 172.21.15.62:0/3032618973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a1e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 -- 172.21.15.62:0/3032618973 >> 172.21.15.62:0/3032618973 conn(0x55cb87a12be0 msgr2=0x55cb87a17cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 -- 172.21.15.62:0/3032618973 shutdown_connections 2023-07-14T16:24:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 -- 172.21.15.62:0/3032618973 wait complete. 2023-07-14T16:24:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.017+0000 7fe86341db80 1 -- start start 2023-07-14T16:24:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.018+0000 7fe86341db80 1 Processor -- start 2023-07-14T16:24:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.018+0000 7fe86341db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.018+0000 7fe86341db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.018+0000 7fe86341db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.018+0000 7fe86341db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8790ea60 con 0x55cb87a1e6e0 2023-07-14T16:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.018+0000 7fe86341db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8790e760 con 0x55cb878796b0 2023-07-14T16:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.018+0000 7fe86341db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb877f28c0 con 0x55cb87a1d840 2023-07-14T16:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.018+0000 7fe86341db80 10 client.0 _ll_register_callbacks cb 0x55cb8790c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84cf9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55348/0 (socket says 172.21.15.62:55348) 2023-07-14T16:24:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84df9c700 1 -- 172.21.15.62:0/4030348411 learned_addr learned my addr 172.21.15.62:0/4030348411 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84df9c700 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84cf9a700 1 -- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 msgr2=0x55cb87a26880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84cf9a700 1 -- 172.21.15.62:0/4030348411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 msgr2=0x55cb87a2de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84cf9a700 1 -- 172.21.15.62:0/4030348411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe848108880 con 0x55cb87a1d840 2023-07-14T16:24:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84df9c700 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe838012680 tx=0x7fe838012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a776491cef9db259 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.019+0000 7fe830ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:24:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.020+0000 7fe830ff9700 1 -- 172.21.15.62:0/4030348411 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838017070 con 0x55cb87a1d840 2023-07-14T16:24:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.020+0000 7fe86341db80 1 -- 172.21.15.62:0/4030348411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb878c8490 con 0x55cb87a1d840 2023-07-14T16:24:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.020+0000 7fe830ff9700 1 -- 172.21.15.62:0/4030348411 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe83800f480 con 0x55cb87a1d840 2023-07-14T16:24:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.020+0000 7fe830ff9700 1 -- 172.21.15.62:0/4030348411 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe83800f640 con 0x55cb87a1d840 2023-07-14T16:24:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.020+0000 7fe86341db80 10 client.27643 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.020+0000 7fe86341db80 1 -- 172.21.15.62:0/4030348411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cb87785c00 con 0x55cb87a1d840 2023-07-14T16:24:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.020+0000 7fe830ff9700 1 -- 172.21.15.62:0/4030348411 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cb878c8490 con 0x55cb87a1d840 2023-07-14T16:24:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.021+0000 7fe830ff9700 1 -- 172.21.15.62:0/4030348411 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe838010890 con 0x55cb87a1d840 2023-07-14T16:24:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.021+0000 7fe86341db80 10 client.27643 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.021+0000 7fe86341db80 10 client.27643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.021+0000 7fe86341db80 20 client.27643 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.021+0000 7fe86341db80 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb87a32730 0x55cb87a34b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.021+0000 7fe86341db80 4 client.27643 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:24:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.021+0000 7fe86341db80 1 -- 172.21.15.62:0/4030348411 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb87967e50 con 0x55cb87a32730 2023-07-14T16:24:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.021+0000 7fe84df9c700 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb87a32730 0x55cb87a34b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.022+0000 7fe84df9c700 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb87a32730 0x55cb87a34b40 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16eae996d5acea8 server_cookie=a671d4285c7a837b in_seq=0 out_seq=0 2023-07-14T16:24:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.022+0000 7fe830ff9700 10 client.27643 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe830ff9700 1 -- 172.21.15.62:0/4030348411 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb87967e50 con 0x55cb87a32730 2023-07-14T16:24:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe830ff9700 10 client.27643 handle_command_reply: tid=0 2023-07-14T16:24:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 client.27643 shutdown 2023-07-14T16:24:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 -- 172.21.15.62:0/4030348411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 msgr2=0x55cb87a26dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe838012680 tx=0x7fe838012990 comp rx=0 tx=0).stop 2023-07-14T16:24:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 -- 172.21.15.62:0/4030348411 shutdown_connections 2023-07-14T16:24:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb87a32730 0x55cb87a34b40 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe830ff9700 0 client.27643 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 --2- 172.21.15.62:0/4030348411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 -- 172.21.15.62:0/4030348411 >> 172.21.15.62:0/4030348411 conn(0x55cb87a12be0 msgr2=0x55cb87a174e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 -- 172.21.15.62:0/4030348411 shutdown_connections 2023-07-14T16:24:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.023+0000 7fe86341db80 1 -- 172.21.15.62:0/4030348411 wait complete. 2023-07-14T16:24:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.024+0000 7fe86341db80 20 client.27643 trim_cache size 0 max 16384 2023-07-14T16:24:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe86341db80 1 Processor -- start 2023-07-14T16:24:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe86341db80 1 -- start start 2023-07-14T16:24:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe86341db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe86341db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe86341db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe86341db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87a2e260 con 0x55cb87a1d840 2023-07-14T16:24:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe86341db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87a2faf0 con 0x55cb878796b0 2023-07-14T16:24:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe86341db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87a27870 con 0x55cb87a1e6e0 2023-07-14T16:24:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe84cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe84cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55354/0 (socket says 172.21.15.62:55354) 2023-07-14T16:24:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.040+0000 7fe84cf9a700 1 -- 172.21.15.62:0/2163858879 learned_addr learned my addr 172.21.15.62:0/2163858879 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84df9c700 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84df9c700 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84cf9a700 1 -- 172.21.15.62:0/2163858879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1e6e0 msgr2=0x55cb87a2dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84cf9a700 1 -- 172.21.15.62:0/2163858879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 msgr2=0x55cb87a26880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84cf9a700 1 -- 172.21.15.62:0/2163858879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb877c31f0 con 0x55cb87a1d840 2023-07-14T16:24:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84df9c700 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:24:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe838000c00 tx=0x7fe838011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18618b2f766be4e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe833fff700 1 -- 172.21.15.62:0/2163858879 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838017040 con 0x55cb87a1d840 2023-07-14T16:24:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe833fff700 1 -- 172.21.15.62:0/2163858879 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe83801cd40 con 0x55cb87a1d840 2023-07-14T16:24:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.041+0000 7fe833fff700 1 -- 172.21.15.62:0/2163858879 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838010a30 con 0x55cb87a1d840 2023-07-14T16:24:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 -- 172.21.15.62:0/2163858879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1d840 msgr2=0x55cb87a26dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe838000c00 tx=0x7fe838011e20 comp rx=0 tx=0).stop 2023-07-14T16:24:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 -- 172.21.15.62:0/2163858879 shutdown_connections 2023-07-14T16:24:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a26880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 --2- 172.21.15.62:0/2163858879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1d840 0x55cb87a26dc0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 -- 172.21.15.62:0/2163858879 >> 172.21.15.62:0/2163858879 conn(0x55cb87a12be0 msgr2=0x55cb87a255a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 -- 172.21.15.62:0/2163858879 shutdown_connections 2023-07-14T16:24:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.042+0000 7fe86341db80 1 -- 172.21.15.62:0/2163858879 wait complete. 2023-07-14T16:24:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.043+0000 7fe86341db80 1 -- start start 2023-07-14T16:24:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.043+0000 7fe86341db80 1 Processor -- start 2023-07-14T16:24:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.043+0000 7fe86341db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a29f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.043+0000 7fe86341db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a2a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.043+0000 7fe86341db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.043+0000 7fe86341db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87a2d320 con 0x55cb87a1e6e0 2023-07-14T16:24:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.043+0000 7fe86341db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87a2d490 con 0x55cb878796b0 2023-07-14T16:24:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe86341db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87a2d650 con 0x55cb87a1d840 2023-07-14T16:24:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe86341db80 10 client.0 _ll_register_callbacks cb 0x7fe838009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:24:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84df9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a29f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84cf9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a2a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84df9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a29f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52912/0 (socket says 172.21.15.62:52912) 2023-07-14T16:24:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84cf9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a2a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46898/0 (socket says 172.21.15.62:46898) 2023-07-14T16:24:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84df9c700 1 -- 172.21.15.62:0/827323863 learned_addr learned my addr 172.21.15.62:0/827323863 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84df9c700 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84cf9a700 1 -- 172.21.15.62:0/827323863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 msgr2=0x55cb87a29f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a29f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84cf9a700 1 -- 172.21.15.62:0/827323863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 msgr2=0x55cb87a2cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.044+0000 7fe84cf9a700 1 -- 172.21.15.62:0/827323863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb87785c00 con 0x55cb87a1d840 2023-07-14T16:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.045+0000 7fe84cf9a700 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a2a4d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe838012680 tx=0x7fe838005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6532422df8665b58 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:24:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.045+0000 7fe830ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:24:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.045+0000 7fe830ff9700 1 -- 172.21.15.62:0/827323863 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838017040 con 0x55cb87a1d840 2023-07-14T16:24:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.045+0000 7fe86341db80 1 -- 172.21.15.62:0/827323863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8400092e0 con 0x55cb87a1d840 2023-07-14T16:24:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.045+0000 7fe830ff9700 1 -- 172.21.15.62:0/827323863 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe838000e40 con 0x55cb87a1d840 2023-07-14T16:24:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.045+0000 7fe830ff9700 1 -- 172.21.15.62:0/827323863 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe83801c890 con 0x55cb87a1d840 2023-07-14T16:24:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.045+0000 7fe86341db80 10 client.27649 fetch_fsmap learned FSMap version 25 2023-07-14T16:24:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.045+0000 7fe86341db80 1 -- 172.21.15.62:0/827323863 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cb8779b250 con 0x55cb87a1d840 2023-07-14T16:24:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.046+0000 7fe830ff9700 1 -- 172.21.15.62:0/827323863 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8400092e0 con 0x55cb87a1d840 2023-07-14T16:24:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.046+0000 7fe830ff9700 1 -- 172.21.15.62:0/827323863 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe838003680 con 0x55cb87a1d840 2023-07-14T16:24:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.046+0000 7fe86341db80 10 client.27649 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:24:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.046+0000 7fe86341db80 10 client.27649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:24:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.046+0000 7fe86341db80 20 client.27649 populate_metadata read hostname 'smithi062' 2023-07-14T16:24:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.046+0000 7fe86341db80 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb87a86fa0 0x55cb87a893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:24:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.046+0000 7fe86341db80 4 client.27649 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:24:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.046+0000 7fe86341db80 1 -- 172.21.15.62:0/827323863 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb87929ff0 con 0x55cb87a86fa0 2023-07-14T16:24:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.047+0000 7fe84df9c700 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb87a86fa0 0x55cb87a893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:24:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.047+0000 7fe84df9c700 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb87a86fa0 0x55cb87a893b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f4f32ebf42bf6a server_cookie=fa429fe6ef19409d in_seq=0 out_seq=0 2023-07-14T16:24:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.047+0000 7fe830ff9700 10 client.27649 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.057+0000 7fe830ff9700 1 -- 172.21.15.62:0/827323863 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb87929ff0 con 0x55cb87a86fa0 2023-07-14T16:24:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.057+0000 7fe830ff9700 10 client.27649 handle_command_reply: tid=0 2023-07-14T16:24:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.057+0000 7fe86341db80 1 client.27649 shutdown 2023-07-14T16:24:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.057+0000 7fe86341db80 1 -- 172.21.15.62:0/827323863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 msgr2=0x55cb87a2a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.057+0000 7fe86341db80 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a2a4d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe838012680 tx=0x7fe838005ce0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 1 -- 172.21.15.62:0/827323863 shutdown_connections 2023-07-14T16:24:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cb87a1e6e0 0x55cb87a2cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cb87a86fa0 0x55cb87a893b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cb878796b0 0x55cb87a29f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 1 --2- 172.21.15.62:0/827323863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cb87a1d840 0x55cb87a2a4d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe830ff9700 0 client.27649 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:24:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 1 -- 172.21.15.62:0/827323863 >> 172.21.15.62:0/827323863 conn(0x55cb87a12be0 msgr2=0x55cb87a21630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 1 -- 172.21.15.62:0/827323863 shutdown_connections 2023-07-14T16:24:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 1 -- 172.21.15.62:0/827323863 wait complete. 2023-07-14T16:24:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe86341db80 20 client.27649 trim_cache size 0 max 16384 2023-07-14T16:24:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:24:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:24:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:24:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:24:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.058+0000 7fe84f9ff700 1 -- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe8340a11d0 msgr2=0x7fe8340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe8340a11d0 0x7fe8340a3680 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe8440a15d0 tx=0x7fe8440cfcf0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 -- 172.21.15.62:0/392910772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8480ffb10 msgr2=0x7fe848072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:24:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8480ffb10 0x7fe848072900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe84400d980 tx=0x7fe84400dc90 comp rx=0 tx=0).stop 2023-07-14T16:24:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 -- 172.21.15.62:0/392910772 shutdown_connections 2023-07-14T16:24:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe848101690 0x7fe848073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe8340a11d0 0x7fe8340a3680 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8480ffb10 0x7fe848072900 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 --2- 172.21.15.62:0/392910772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe848100d00 0x7fe848072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:24:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 -- 172.21.15.62:0/392910772 >> 172.21.15.62:0/392910772 conn(0x7fe8480fb130 msgr2=0x7fe848104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:24:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 -- 172.21.15.62:0/392910772 shutdown_connections 2023-07-14T16:24:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:24:52.059+0000 7fe84f9ff700 1 -- 172.21.15.62:0/392910772 wait complete. 2023-07-14T16:24:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:52 smithi062 ceph-mon[95543]: pgmap v634: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:24:53.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:52 smithi081 ceph-mon[127795]: pgmap v634: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:24:53.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:52 smithi164 ceph-mon[97427]: pgmap v634: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:24:54.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:53 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:24:54.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:53 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:24:54.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:53 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:24:54.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:53 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:24:54.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:53 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:24:54.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:53 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:24:55.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:54 smithi062 ceph-mon[95543]: pgmap v635: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:24:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:54 smithi081 ceph-mon[127795]: pgmap v635: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:24:55.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:54 smithi164 ceph-mon[97427]: pgmap v635: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:24:57.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:56 smithi062 ceph-mon[95543]: pgmap v636: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:24:57.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:56 smithi081 ceph-mon[127795]: pgmap v636: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:24:57.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:56 smithi164 ceph-mon[97427]: pgmap v636: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:24:59.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:58 smithi062 ceph-mon[95543]: pgmap v637: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:24:59.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:58 smithi081 ceph-mon[127795]: pgmap v637: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:24:59.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:58 smithi164 ceph-mon[97427]: pgmap v637: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:25:00.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:24:59 smithi062 ceph-mon[95543]: pgmap v638: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:25:00.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:24:59 smithi081 ceph-mon[127795]: pgmap v638: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:25:00.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:24:59 smithi164 ceph-mon[97427]: pgmap v638: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:25:02.813 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:02 smithi164 ceph-mon[97427]: pgmap v639: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:25:03.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:02 smithi062 ceph-mon[95543]: pgmap v639: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:25:03.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:02 smithi081 ceph-mon[127795]: pgmap v639: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 25 op/s 2023-07-14T16:25:05.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:04 smithi164 ceph-mon[97427]: pgmap v640: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:25:05.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:04 smithi062 ceph-mon[95543]: pgmap v640: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:25:05.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:04 smithi081 ceph-mon[127795]: pgmap v640: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:25:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/2268299963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e981079a0 msgr2=0x7f9e98109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/2268299963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e981079a0 0x7f9e98109e20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000b0a0 tx=0x7f9e9000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/2268299963 shutdown_connections 2023-07-14T16:25:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/2268299963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e981079a0 0x7f9e98109e20 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/2268299963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e980744f0 0x7f9e98072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/2268299963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e98072f10 0x7f9e98073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/2268299963 >> 172.21.15.62:0/2268299963 conn(0x7f9e980fb130 msgr2=0x7f9e980fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/2268299963 shutdown_connections 2023-07-14T16:25:05.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.217+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/2268299963 wait complete. 2023-07-14T16:25:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9f9ff700 1 Processor -- start 2023-07-14T16:25:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9f9ff700 1 -- start start 2023-07-14T16:25:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9f9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e98072f10 0x7f9e98197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9f9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e980744f0 0x7f9e981980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9f9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e981079a0 0x7f9e98198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9f9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e981995a0 con 0x7f9e98072f10 2023-07-14T16:25:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9f9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e98199710 con 0x7f9e981079a0 2023-07-14T16:25:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9f9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e981998d0 con 0x7f9e980744f0 2023-07-14T16:25:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.219+0000 7f9e9d79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e98072f10 0x7f9e98197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9d79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e98072f10 0x7f9e98197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54616/0 (socket says 172.21.15.62:54616) 2023-07-14T16:25:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9d79b700 1 -- 172.21.15.62:0/3911735370 learned_addr learned my addr 172.21.15.62:0/3911735370 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:05.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9df9c700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e981079a0 0x7f9e98198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e980744f0 0x7f9e981980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9d79b700 1 -- 172.21.15.62:0/3911735370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e980744f0 msgr2=0x7f9e981980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e980744f0 0x7f9e981980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9d79b700 1 -- 172.21.15.62:0/3911735370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e981079a0 msgr2=0x7f9e98198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e981079a0 0x7f9e98198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9d79b700 1 -- 172.21.15.62:0/3911735370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e90009d00 con 0x7f9e98072f10 2023-07-14T16:25:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.220+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e98072f10 0x7f9e98197b80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400d980 tx=0x7f9e9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.221+0000 7f9e8e7fc700 1 -- 172.21.15.62:0/3911735370 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94004220 con 0x7f9e98072f10 2023-07-14T16:25:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.221+0000 7f9e8e7fc700 1 -- 172.21.15.62:0/3911735370 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e94007640 con 0x7f9e98072f10 2023-07-14T16:25:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.221+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/3911735370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e98199bb0 con 0x7f9e98072f10 2023-07-14T16:25:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.221+0000 7f9e8e7fc700 1 -- 172.21.15.62:0/3911735370 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94005660 con 0x7f9e98072f10 2023-07-14T16:25:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.221+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/3911735370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e981a0370 con 0x7f9e98072f10 2023-07-14T16:25:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.224+0000 7f9e8e7fc700 1 -- 172.21.15.62:0/3911735370 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9e94004750 con 0x7f9e98072f10 2023-07-14T16:25:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.226+0000 7f9e8e7fc700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e840a1150 0x7f9e840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.226+0000 7f9e8e7fc700 1 -- 172.21.15.62:0/3911735370 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e940cc6a0 con 0x7f9e98072f10 2023-07-14T16:25:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.226+0000 7f9eb3404b80 1 Processor -- start 2023-07-14T16:25:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.226+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e840a1150 0x7f9e840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9eb3404b80 1 -- start start 2023-07-14T16:25:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x564300903300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x5643009041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300904b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564300808c50 con 0x564300903840 2023-07-14T16:25:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430085b9e0 con 0x56430075f6b0 2023-07-14T16:25:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564300854e70 con 0x5643009046e0 2023-07-14T16:25:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9e9cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x5643009041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9e9cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x5643009041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54620/0 (socket says 172.21.15.62:54620) 2023-07-14T16:25:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.227+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/320418002 learned_addr learned my addr 172.21.15.62:0/320418002 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e840a1150 0x7f9e840a3600 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9e88005950 tx=0x7f9e8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x564300903300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9df9c700 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300904b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/320418002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 msgr2=0x564300904b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300904b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/320418002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 msgr2=0x564300903300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x564300903300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/320418002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564300682d80 con 0x564300903840 2023-07-14T16:25:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.228+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x5643009041a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9e88010fa0 tx=0x7f9e88013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5ca757a66b29ae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.229+0000 7f9e837fe700 1 -- 172.21.15.62:0/320418002 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88011db0 con 0x564300903840 2023-07-14T16:25:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.229+0000 7f9e837fe700 1 -- 172.21.15.62:0/320418002 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e88015070 con 0x564300903840 2023-07-14T16:25:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.229+0000 7f9e837fe700 1 -- 172.21.15.62:0/320418002 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88026660 con 0x564300903840 2023-07-14T16:25:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.229+0000 7f9eb3404b80 1 -- 172.21.15.62:0/320418002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 msgr2=0x5643009041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.229+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x5643009041a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9e88010fa0 tx=0x7f9e88013000 comp rx=0 tx=0).stop 2023-07-14T16:25:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.230+0000 7f9eb3404b80 1 -- 172.21.15.62:0/320418002 shutdown_connections 2023-07-14T16:25:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.230+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300904b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.230+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x564300903300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.230+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/320418002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x5643009041a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.230+0000 7f9eb3404b80 1 -- 172.21.15.62:0/320418002 >> 172.21.15.62:0/320418002 conn(0x5643008f8be0 msgr2=0x5643008fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.230+0000 7f9eb3404b80 1 -- 172.21.15.62:0/320418002 shutdown_connections 2023-07-14T16:25:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.230+0000 7f9eb3404b80 1 -- 172.21.15.62:0/320418002 wait complete. 2023-07-14T16:25:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.230+0000 7f9eb3404b80 1 -- start start 2023-07-14T16:25:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.231+0000 7f9eb3404b80 1 Processor -- start 2023-07-14T16:25:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.231+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.231+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x56430090cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.231+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x564300913dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.231+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643007f4a60 con 0x5643009046e0 2023-07-14T16:25:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.231+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643007f4760 con 0x564300903840 2023-07-14T16:25:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.231+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643006d88c0 con 0x56430075f6b0 2023-07-14T16:25:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9eb3404b80 10 client.0 _ll_register_callbacks cb 0x5643007f2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9cf9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x56430090cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x564300913dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9cf9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x56430090cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:39370/0 (socket says 172.21.15.62:39370) 2023-07-14T16:25:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/4043053469 learned_addr learned my addr 172.21.15.62:0/4043053469 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/4043053469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 msgr2=0x56430090c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/4043053469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 msgr2=0x564300913dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x564300913dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/4043053469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e88010c50 con 0x564300903840 2023-07-14T16:25:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.232+0000 7f9e9df9c700 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x564300913dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.233+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x56430090cd80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9e8801ec20 tx=0x7f9e88011e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2615384addf10f65 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.233+0000 7f9e80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:25:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.233+0000 7f9e80ff9700 1 -- 172.21.15.62:0/4043053469 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8802a070 con 0x564300903840 2023-07-14T16:25:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.233+0000 7f9eb3404b80 1 -- 172.21.15.62:0/4043053469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643007ae490 con 0x564300903840 2023-07-14T16:25:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.233+0000 7f9e80ff9700 1 -- 172.21.15.62:0/4043053469 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e88017070 con 0x564300903840 2023-07-14T16:25:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.233+0000 7f9e80ff9700 1 -- 172.21.15.62:0/4043053469 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e8800be60 con 0x564300903840 2023-07-14T16:25:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9eb3404b80 10 client.27773 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9eb3404b80 1 -- 172.21.15.62:0/4043053469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56430066bc00 con 0x564300903840 2023-07-14T16:25:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9e80ff9700 1 -- 172.21.15.62:0/4043053469 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5643007ae490 con 0x564300903840 2023-07-14T16:25:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9e80ff9700 1 -- 172.21.15.62:0/4043053469 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9e88024020 con 0x564300903840 2023-07-14T16:25:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9eb3404b80 10 client.27773 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9eb3404b80 10 client.27773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9eb3404b80 20 client.27773 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564300918610 0x56430091aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9eb3404b80 4 client.27773 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.234+0000 7f9eb3404b80 1 -- 172.21.15.62:0/4043053469 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56430084de50 con 0x564300918610 2023-07-14T16:25:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.235+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564300918610 0x56430091aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.235+0000 7f9e80ff9700 10 client.27773 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.235+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564300918610 0x56430091aa20 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7313b26e5837b7f server_cookie=25150a9c7c83d013 in_seq=0 out_seq=0 2023-07-14T16:25:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.236+0000 7f9e80ff9700 1 -- 172.21.15.62:0/4043053469 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56430084de50 con 0x564300918610 2023-07-14T16:25:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.236+0000 7f9e80ff9700 10 client.27773 handle_command_reply: tid=0 2023-07-14T16:25:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.236+0000 7f9eb3404b80 1 client.27773 shutdown 2023-07-14T16:25:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.236+0000 7f9eb3404b80 1 -- 172.21.15.62:0/4043053469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 msgr2=0x56430090cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.236+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x56430090cd80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9e8801ec20 tx=0x7f9e88011e60 comp rx=0 tx=0).stop 2023-07-14T16:25:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 1 -- 172.21.15.62:0/4043053469 shutdown_connections 2023-07-14T16:25:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x564300913dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564300918610 0x56430091aa20 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/4043053469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x56430090cd80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9e80ff9700 0 client.27773 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 1 -- 172.21.15.62:0/4043053469 >> 172.21.15.62:0/4043053469 conn(0x5643008f8be0 msgr2=0x5643008fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 1 -- 172.21.15.62:0/4043053469 shutdown_connections 2023-07-14T16:25:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 1 -- 172.21.15.62:0/4043053469 wait complete. 2023-07-14T16:25:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.237+0000 7f9eb3404b80 20 client.27773 trim_cache size 0 max 16384 2023-07-14T16:25:05.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.256+0000 7f9eb3404b80 1 Processor -- start 2023-07-14T16:25:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.256+0000 7f9eb3404b80 1 -- start start 2023-07-14T16:25:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x564300913d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x56430090c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643008ff810 con 0x5643009046e0 2023-07-14T16:25:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643008c9090 con 0x564300903840 2023-07-14T16:25:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643008fe7a0 con 0x56430075f6b0 2023-07-14T16:25:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9e9df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x56430090c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9e9df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x56430090c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54636/0 (socket says 172.21.15.62:54636) 2023-07-14T16:25:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9e9df9c700 1 -- 172.21.15.62:0/3830752714 learned_addr learned my addr 172.21.15.62:0/3830752714 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x564300913d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.257+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e9df9c700 1 -- 172.21.15.62:0/3830752714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 msgr2=0x56430090b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e9df9c700 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e9df9c700 1 -- 172.21.15.62:0/3830752714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 msgr2=0x564300913d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e9df9c700 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x564300913d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e9df9c700 1 -- 172.21.15.62:0/3830752714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643006a91f0 con 0x5643009046e0 2023-07-14T16:25:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x564300913d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e9df9c700 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x56430090c840 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000b0a0 tx=0x7f9e9000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cd40e981900c36 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e83fff700 1 -- 172.21.15.62:0/3830752714 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000e040 con 0x5643009046e0 2023-07-14T16:25:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e83fff700 1 -- 172.21.15.62:0/3830752714 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e90022070 con 0x5643009046e0 2023-07-14T16:25:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.258+0000 7f9e83fff700 1 -- 172.21.15.62:0/3830752714 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e90005700 con 0x5643009046e0 2023-07-14T16:25:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3830752714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 msgr2=0x56430090c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x56430090c840 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000b0a0 tx=0x7f9e9000cef0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3830752714 shutdown_connections 2023-07-14T16:25:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5643009046e0 0x56430090c840 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56430075f6b0 0x56430090b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3830752714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564300903840 0x564300913d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3830752714 >> 172.21.15.62:0/3830752714 conn(0x5643008f8be0 msgr2=0x56430091f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3830752714 shutdown_connections 2023-07-14T16:25:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.259+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3830752714 wait complete. 2023-07-14T16:25:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.260+0000 7f9eb3404b80 1 -- start start 2023-07-14T16:25:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.260+0000 7f9eb3404b80 1 Processor -- start 2023-07-14T16:25:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.260+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x56430090fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x564300910230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9eb3404b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300913250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564300913790 con 0x564300903840 2023-07-14T16:25:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564300913900 con 0x56430075f6b0 2023-07-14T16:25:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9eb3404b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564300913ac0 con 0x5643009046e0 2023-07-14T16:25:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9eb3404b80 10 client.0 _ll_register_callbacks cb 0x7f9e94004fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x564300910230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9cf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x564300910230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54640/0 (socket says 172.21.15.62:54640) 2023-07-14T16:25:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/3102366564 learned_addr learned my addr 172.21.15.62:0/3102366564 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x56430090fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9df9c700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300913250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/3102366564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 msgr2=0x564300913250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300913250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 msgr2=0x56430090fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x56430090fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9cf9a700 1 -- 172.21.15.62:0/3102366564 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56430066bc00 con 0x564300903840 2023-07-14T16:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9df9c700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300913250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.261+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x56430090fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.262+0000 7f9e9cf9a700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x564300910230 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f9e88038b90 tx=0x7f9e8800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a2dbd14b933d26 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.262+0000 7f9e80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.262+0000 7f9e80ff9700 1 -- 172.21.15.62:0/3102366564 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8802a070 con 0x564300903840 2023-07-14T16:25:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.262+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3102366564 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e88015070 con 0x564300903840 2023-07-14T16:25:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.262+0000 7f9e80ff9700 1 -- 172.21.15.62:0/3102366564 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e88017040 con 0x564300903840 2023-07-14T16:25:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.262+0000 7f9e80ff9700 1 -- 172.21.15.62:0/3102366564 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e8801e630 con 0x564300903840 2023-07-14T16:25:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.262+0000 7f9eb3404b80 10 client.18150 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.262+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3102366564 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f9e9810aa40 con 0x564300903840 2023-07-14T16:25:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9e80ff9700 1 -- 172.21.15.62:0/3102366564 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e88015070 con 0x564300903840 2023-07-14T16:25:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9e80ff9700 1 -- 172.21.15.62:0/3102366564 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9e88024050 con 0x564300903840 2023-07-14T16:25:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9eb3404b80 10 client.18150 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9eb3404b80 10 client.18150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9eb3404b80 20 client.18150 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56430096d0a0 0x56430096f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9eb3404b80 4 client.18150 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:25:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3102366564 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56430080fff0 con 0x56430096d0a0 2023-07-14T16:25:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.263+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56430096d0a0 0x56430096f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.264+0000 7f9e80ff9700 10 client.18150 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.264+0000 7f9e9d79b700 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56430096d0a0 0x56430096f4b0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b17b30c558125a server_cookie=d4934f995453f969 in_seq=0 out_seq=0 2023-07-14T16:25:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9e80ff9700 1 -- 172.21.15.62:0/3102366564 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56430080fff0 con 0x56430096d0a0 2023-07-14T16:25:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9e80ff9700 10 client.18150 handle_command_reply: tid=0 2023-07-14T16:25:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 client.18150 shutdown 2023-07-14T16:25:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3102366564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 msgr2=0x564300910230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x564300910230 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f9e88038b90 tx=0x7f9e8800be60 comp rx=0 tx=0).stop 2023-07-14T16:25:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3102366564 shutdown_connections 2023-07-14T16:25:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5643009046e0 0x564300913250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56430096d0a0 0x56430096f4b0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56430075f6b0 0x56430090fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 --2- 172.21.15.62:0/3102366564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564300903840 0x564300910230 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9e80ff9700 0 client.18150 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3102366564 >> 172.21.15.62:0/3102366564 conn(0x5643008f8be0 msgr2=0x56430090b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3102366564 shutdown_connections 2023-07-14T16:25:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 1 -- 172.21.15.62:0/3102366564 wait complete. 2023-07-14T16:25:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.274+0000 7f9eb3404b80 20 client.18150 trim_cache size 0 max 16384 2023-07-14T16:25:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:25:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:25:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:25:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:25:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e840a1150 msgr2=0x7f9e840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e840a1150 0x7f9e840a3600 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9e88005950 tx=0x7f9e8800b410 comp rx=0 tx=0).stop 2023-07-14T16:25:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e98072f10 msgr2=0x7f9e98197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e98072f10 0x7f9e98197b80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400d980 tx=0x7f9e9400dc90 comp rx=0 tx=0).stop 2023-07-14T16:25:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/3911735370 shutdown_connections 2023-07-14T16:25:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e840a1150 0x7f9e840a3600 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e981079a0 0x7f9e98198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e98072f10 0x7f9e98197b80 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 --2- 172.21.15.62:0/3911735370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e980744f0 0x7f9e981980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/3911735370 >> 172.21.15.62:0/3911735370 conn(0x7f9e980fb130 msgr2=0x7f9e980fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/3911735370 shutdown_connections 2023-07-14T16:25:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:05.275+0000 7f9e9f9ff700 1 -- 172.21.15.62:0/3911735370 wait complete. 2023-07-14T16:25:06.983 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:06 smithi164 ceph-mon[97427]: pgmap v641: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 MiB/s wr, 11 op/s 2023-07-14T16:25:07.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:06 smithi062 ceph-mon[95543]: pgmap v641: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 MiB/s wr, 11 op/s 2023-07-14T16:25:07.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:06 smithi081 ceph-mon[127795]: pgmap v641: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 MiB/s wr, 11 op/s 2023-07-14T16:25:09.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:08 smithi164 ceph-mon[97427]: pgmap v642: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 MiB/s wr, 11 op/s 2023-07-14T16:25:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:08 smithi062 ceph-mon[95543]: pgmap v642: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 MiB/s wr, 11 op/s 2023-07-14T16:25:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:08 smithi081 ceph-mon[127795]: pgmap v642: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 MiB/s wr, 11 op/s 2023-07-14T16:25:11.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:10 smithi164 ceph-mon[97427]: pgmap v643: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:25:11.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:10 smithi062 ceph-mon[95543]: pgmap v643: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:25:11.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:10 smithi081 ceph-mon[127795]: pgmap v643: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:25:13.005 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:12 smithi164 ceph-mon[97427]: pgmap v644: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:25:13.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:12 smithi062 ceph-mon[95543]: pgmap v644: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:25:13.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:12 smithi081 ceph-mon[127795]: pgmap v644: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:25:15.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:14 smithi164 ceph-mon[97427]: pgmap v645: 97 pgs: 97 active+clean; 6.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:25:15.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:14 smithi062 ceph-mon[95543]: pgmap v645: 97 pgs: 97 active+clean; 6.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:25:15.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:14 smithi081 ceph-mon[127795]: pgmap v645: 97 pgs: 97 active+clean; 6.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:25:17.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:16 smithi164 ceph-mon[97427]: pgmap v646: 97 pgs: 97 active+clean; 6.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:25:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:16 smithi062 ceph-mon[95543]: pgmap v646: 97 pgs: 97 active+clean; 6.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:25:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:16 smithi081 ceph-mon[127795]: pgmap v646: 97 pgs: 97 active+clean; 6.7 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:25:17.355 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:25:17.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 -- 172.21.15.62:0/3872957786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb104101660 msgr2=0x7fb104107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/3872957786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb104101660 0x7fb104107f20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00b0a0 tx=0x7fb0fc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 -- 172.21.15.62:0/3872957786 shutdown_connections 2023-07-14T16:25:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/3872957786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb104101660 0x7fb104107f20 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/3872957786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1040ffae0 0x7fb1040ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/3872957786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104100cd0 0x7fb104101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 -- 172.21.15.62:0/3872957786 >> 172.21.15.62:0/3872957786 conn(0x7fb1040fb0e0 msgr2=0x7fb1040fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 -- 172.21.15.62:0/3872957786 shutdown_connections 2023-07-14T16:25:17.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.613+0000 7fb10c1ff700 1 -- 172.21.15.62:0/3872957786 wait complete. 2023-07-14T16:25:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.614+0000 7fb10c1ff700 1 Processor -- start 2023-07-14T16:25:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.614+0000 7fb10c1ff700 1 -- start start 2023-07-14T16:25:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb10c1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1040ffae0 0x7fb1041958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb10c1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb104100cd0 0x7fb104195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb10c1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101660 0x7fb10419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb10c1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10419d420 con 0x7fb1040ffae0 2023-07-14T16:25:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb10c1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10419d560 con 0x7fb104100cd0 2023-07-14T16:25:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb10c1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10419d6f0 con 0x7fb104101660 2023-07-14T16:25:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb109f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1040ffae0 0x7fb1041958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb109f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1040ffae0 0x7fb1041958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51828/0 (socket says 172.21.15.62:51828) 2023-07-14T16:25:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb109f9b700 1 -- 172.21.15.62:0/1012488258 learned_addr learned my addr 172.21.15.62:0/1012488258 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb10a79c700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101660 0x7fb10419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.615+0000 7fb10979a700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb104100cd0 0x7fb104195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb109f9b700 1 -- 172.21.15.62:0/1012488258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101660 msgr2=0x7fb10419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb109f9b700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101660 0x7fb10419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb109f9b700 1 -- 172.21.15.62:0/1012488258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb104100cd0 msgr2=0x7fb104195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb109f9b700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb104100cd0 0x7fb104195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb109f9b700 1 -- 172.21.15.62:0/1012488258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc009d00 con 0x7fb1040ffae0 2023-07-14T16:25:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb10a79c700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101660 0x7fb10419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb109f9b700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1040ffae0 0x7fb1041958a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb10000e970 tx=0x7fb10000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb0faffd700 1 -- 172.21.15.62:0/1012488258 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100009790 con 0x7fb1040ffae0 2023-07-14T16:25:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb10c1ff700 1 -- 172.21.15.62:0/1012488258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1040728e0 con 0x7fb1040ffae0 2023-07-14T16:25:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb0faffd700 1 -- 172.21.15.62:0/1012488258 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb10000c7e0 con 0x7fb1040ffae0 2023-07-14T16:25:17.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.616+0000 7fb0faffd700 1 -- 172.21.15.62:0/1012488258 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100003bd0 con 0x7fb1040ffae0 2023-07-14T16:25:17.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.617+0000 7fb10c1ff700 1 -- 172.21.15.62:0/1012488258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb104072e40 con 0x7fb1040ffae0 2023-07-14T16:25:17.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.620+0000 7fb0faffd700 1 -- 172.21.15.62:0/1012488258 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb10000c950 con 0x7fb1040ffae0 2023-07-14T16:25:17.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.622+0000 7fb11fb57b80 1 Processor -- start 2023-07-14T16:25:17.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.622+0000 7fb11fb57b80 1 -- start start 2023-07-14T16:25:17.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403bb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637402bfcb0 con 0x5637403bae20 2023-07-14T16:25:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563740312a40 con 0x5637403ba4d0 2023-07-14T16:25:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374030bed0 con 0x5637403bbc70 2023-07-14T16:25:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb0faffd700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0f00a11d0 0x7fb0f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb109f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403bb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb0faffd700 1 -- 172.21.15.62:0/1012488258 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb100014070 con 0x7fb1040ffae0 2023-07-14T16:25:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb109f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403bb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51840/0 (socket says 172.21.15.62:51840) 2023-07-14T16:25:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb109f9b700 1 -- 172.21.15.62:0/3872931408 learned_addr learned my addr 172.21.15.62:0/3872931408 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb10a79c700 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb10979a700 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.623+0000 7fb10a79c700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0f00a11d0 0x7fb0f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb109f9b700 1 -- 172.21.15.62:0/3872931408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 msgr2=0x5637403bc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb109f9b700 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403bc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb109f9b700 1 -- 172.21.15.62:0/3872931408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 msgr2=0x5637403ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb109f9b700 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb109f9b700 1 -- 172.21.15.62:0/3872931408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563740139de0 con 0x5637403bae20 2023-07-14T16:25:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb10979a700 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403bc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb109f9b700 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403bb6e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb1000968a0 tx=0x7fb100069300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b382fc77eb239ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb10a79c700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0f00a11d0 0x7fb0f00a3680 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00b780 tx=0x7fb0fc006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.624+0000 7fb0effff700 1 -- 172.21.15.62:0/3872931408 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000cc50 con 0x5637403bae20 2023-07-14T16:25:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb0effff700 1 -- 172.21.15.62:0/3872931408 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb100099e30 con 0x5637403bae20 2023-07-14T16:25:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb0effff700 1 -- 172.21.15.62:0/3872931408 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10001dc10 con 0x5637403bae20 2023-07-14T16:25:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3872931408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 msgr2=0x5637403bb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403bb6e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb1000968a0 tx=0x7fb100069300 comp rx=0 tx=0).stop 2023-07-14T16:25:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3872931408 shutdown_connections 2023-07-14T16:25:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403bc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/3872931408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403bb6e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3872931408 >> 172.21.15.62:0/3872931408 conn(0x5637403afda0 msgr2=0x5637403b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3872931408 shutdown_connections 2023-07-14T16:25:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.625+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3872931408 wait complete. 2023-07-14T16:25:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.626+0000 7fb11fb57b80 1 -- start start 2023-07-14T16:25:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.626+0000 7fb11fb57b80 1 Processor -- start 2023-07-14T16:25:17.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403ba4d0 0x5637403c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403bae20 0x5637403c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637402abac0 con 0x5637403bbc70 2023-07-14T16:25:17.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637402ab7c0 con 0x5637403bae20 2023-07-14T16:25:17.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374018f920 con 0x5637403ba4d0 2023-07-14T16:25:17.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb11fb57b80 10 client.0 _ll_register_callbacks cb 0x5637402a9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:17.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb10a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb10a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51844/0 (socket says 172.21.15.62:51844) 2023-07-14T16:25:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.627+0000 7fb10a79c700 1 -- 172.21.15.62:0/2220217527 learned_addr learned my addr 172.21.15.62:0/2220217527 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb109f9b700 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403ba4d0 0x5637403c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb10979a700 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403bae20 0x5637403c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb10a79c700 1 -- 172.21.15.62:0/2220217527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403ba4d0 msgr2=0x5637403c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb10a79c700 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403ba4d0 0x5637403c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb10a79c700 1 -- 172.21.15.62:0/2220217527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403bae20 msgr2=0x5637403c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb10a79c700 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403bae20 0x5637403c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb10a79c700 1 -- 172.21.15.62:0/2220217527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f40098e0 con 0x5637403bbc70 2023-07-14T16:25:17.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb109f9b700 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403ba4d0 0x5637403c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:17.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb10a79c700 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403cb720 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00b0a0 tx=0x7fb0fc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f88452638edfcd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:17.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb0ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:17.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/2220217527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00e040 con 0x5637403bbc70 2023-07-14T16:25:17.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/2220217527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0fc0207b0 con 0x5637403bbc70 2023-07-14T16:25:17.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.628+0000 7fb11fb57b80 1 -- 172.21.15.62:0/2220217527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637402654f0 con 0x5637403bbc70 2023-07-14T16:25:17.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/2220217527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0fc020970 con 0x5637403bbc70 2023-07-14T16:25:17.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb11fb57b80 10 client.18162 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb11fb57b80 1 -- 172.21.15.62:0/2220217527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb104072e40 con 0x5637403bbc70 2023-07-14T16:25:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/2220217527 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5637402654f0 con 0x5637403bbc70 2023-07-14T16:25:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/2220217527 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb0fc038ad0 con 0x5637403bbc70 2023-07-14T16:25:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb11fb57b80 10 client.18162 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb11fb57b80 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb11fb57b80 20 client.18162 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637403cfea0 0x5637403d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb11fb57b80 4 client.18162 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:17.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.629+0000 7fb11fb57b80 1 -- 172.21.15.62:0/2220217527 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563740304eb0 con 0x5637403cfea0 2023-07-14T16:25:17.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.630+0000 7fb109f9b700 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637403cfea0 0x5637403d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.630+0000 7fb0ed7fa700 10 client.18162 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:17.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.630+0000 7fb109f9b700 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637403cfea0 0x5637403d22b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b49c7786c667687a server_cookie=654a00b9a5c4d243 in_seq=0 out_seq=0 2023-07-14T16:25:17.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/2220217527 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563740304eb0 con 0x5637403cfea0 2023-07-14T16:25:17.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb0ed7fa700 10 client.18162 handle_command_reply: tid=0 2023-07-14T16:25:17.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 client.18162 shutdown 2023-07-14T16:25:17.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 -- 172.21.15.62:0/2220217527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 msgr2=0x5637403cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403cb720 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00b0a0 tx=0x7fb0fc0099e0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 -- 172.21.15.62:0/2220217527 shutdown_connections 2023-07-14T16:25:17.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403cb720 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403ba4d0 0x5637403c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637403cfea0 0x5637403d22b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/2220217527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403bae20 0x5637403c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb0ed7fa700 0 client.18162 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:17.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.631+0000 7fb11fb57b80 1 -- 172.21.15.62:0/2220217527 >> 172.21.15.62:0/2220217527 conn(0x5637403afda0 msgr2=0x5637403b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:17.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.632+0000 7fb11fb57b80 1 -- 172.21.15.62:0/2220217527 shutdown_connections 2023-07-14T16:25:17.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.632+0000 7fb11fb57b80 1 -- 172.21.15.62:0/2220217527 wait complete. 2023-07-14T16:25:17.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.632+0000 7fb11fb57b80 20 client.18162 trim_cache size 0 max 16384 2023-07-14T16:25:17.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.652+0000 7fb11fb57b80 1 Processor -- start 2023-07-14T16:25:17.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.652+0000 7fb11fb57b80 1 -- start start 2023-07-14T16:25:17.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.652+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.652+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637403c51e0 con 0x5637403bae20 2023-07-14T16:25:17.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637403c3670 con 0x5637403ba4d0 2023-07-14T16:25:17.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637403c3d10 con 0x5637403bbc70 2023-07-14T16:25:17.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10979a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10979a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51860/0 (socket says 172.21.15.62:51860) 2023-07-14T16:25:17.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10979a700 1 -- 172.21.15.62:0/3216533334 learned_addr learned my addr 172.21.15.62:0/3216533334 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10a79c700 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb109f9b700 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10979a700 1 -- 172.21.15.62:0/3216533334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 msgr2=0x5637403cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10979a700 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10979a700 1 -- 172.21.15.62:0/3216533334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 msgr2=0x5637403c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10979a700 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10979a700 1 -- 172.21.15.62:0/3216533334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563740122c60 con 0x5637403bae20 2023-07-14T16:25:17.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.653+0000 7fb10a79c700 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:17.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.654+0000 7fb10979a700 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403cb6e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4010430 tx=0x7fb0f4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa74c96728d60176 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:17.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.654+0000 7fb0f8ff9700 1 -- 172.21.15.62:0/3216533334 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f401c5f0 con 0x5637403bae20 2023-07-14T16:25:17.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.654+0000 7fb0f8ff9700 1 -- 172.21.15.62:0/3216533334 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0f4018040 con 0x5637403bae20 2023-07-14T16:25:17.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.654+0000 7fb0f8ff9700 1 -- 172.21.15.62:0/3216533334 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f400fb00 con 0x5637403bae20 2023-07-14T16:25:17.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3216533334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 msgr2=0x5637403cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403cb6e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4010430 tx=0x7fb0f4010740 comp rx=0 tx=0).stop 2023-07-14T16:25:17.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3216533334 shutdown_connections 2023-07-14T16:25:17.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bbc70 0x5637403cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/3216533334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bae20 0x5637403cb6e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3216533334 >> 172.21.15.62:0/3216533334 conn(0x5637403afda0 msgr2=0x5637403d6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:17.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3216533334 shutdown_connections 2023-07-14T16:25:17.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 -- 172.21.15.62:0/3216533334 wait complete. 2023-07-14T16:25:17.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.655+0000 7fb11fb57b80 1 -- start start 2023-07-14T16:25:17.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb11fb57b80 1 Processor -- start 2023-07-14T16:25:17.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bae20 0x5637403c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb11fb57b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637403c8950 con 0x5637403bbc70 2023-07-14T16:25:17.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637403c8ac0 con 0x5637403ba4d0 2023-07-14T16:25:17.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb11fb57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563740423d50 con 0x5637403bae20 2023-07-14T16:25:17.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb11fb57b80 10 client.0 _ll_register_callbacks cb 0x7fb0fc009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:17.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb10a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb10a79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51872/0 (socket says 172.21.15.62:51872) 2023-07-14T16:25:17.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.656+0000 7fb10a79c700 1 -- 172.21.15.62:0/4077850629 learned_addr learned my addr 172.21.15.62:0/4077850629 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:17.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb109f9b700 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb10979a700 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bae20 0x5637403c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb10a79c700 1 -- 172.21.15.62:0/4077850629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bae20 msgr2=0x5637403c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:17.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb10a79c700 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bae20 0x5637403c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb10a79c700 1 -- 172.21.15.62:0/4077850629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 msgr2=0x5637403cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb10a79c700 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb10a79c700 1 -- 172.21.15.62:0/4077850629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb104072e40 con 0x5637403bbc70 2023-07-14T16:25:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb10979a700 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bae20 0x5637403c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb10a79c700 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403c8410 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00b070 tx=0x7fb0fc009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e205ac9e40522a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb0ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/4077850629 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc01fa60 con 0x5637403bbc70 2023-07-14T16:25:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/4077850629 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0fc01fbc0 con 0x5637403bbc70 2023-07-14T16:25:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.657+0000 7fb11fb57b80 1 -- 172.21.15.62:0/4077850629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb100099e30 con 0x5637403bbc70 2023-07-14T16:25:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/4077850629 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0fc01fd80 con 0x5637403bbc70 2023-07-14T16:25:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb11fb57b80 10 client.18168 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb11fb57b80 1 -- 172.21.15.62:0/4077850629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb0f4009ce0 con 0x5637403bbc70 2023-07-14T16:25:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/4077850629 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb100099e30 con 0x5637403bbc70 2023-07-14T16:25:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/4077850629 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb0fc0386e0 con 0x5637403bbc70 2023-07-14T16:25:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb11fb57b80 10 client.18168 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb11fb57b80 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb11fb57b80 20 client.18168 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637404252c0 0x5637404276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb11fb57b80 4 client.18168 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:25:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.658+0000 7fb11fb57b80 1 -- 172.21.15.62:0/4077850629 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5637402c7050 con 0x5637404252c0 2023-07-14T16:25:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.659+0000 7fb109f9b700 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637404252c0 0x5637404276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.659+0000 7fb109f9b700 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637404252c0 0x5637404276d0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a174a7c55db5508 server_cookie=2725c87ca68454f1 in_seq=0 out_seq=0 2023-07-14T16:25:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.659+0000 7fb0ed7fa700 10 client.18168 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb0ed7fa700 1 -- 172.21.15.62:0/4077850629 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5637402c7050 con 0x5637404252c0 2023-07-14T16:25:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb0ed7fa700 10 client.18168 handle_command_reply: tid=0 2023-07-14T16:25:17.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 client.18168 shutdown 2023-07-14T16:25:17.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 -- 172.21.15.62:0/4077850629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 msgr2=0x5637403c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403c8410 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00b070 tx=0x7fb0fc009630 comp rx=0 tx=0).stop 2023-07-14T16:25:17.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 -- 172.21.15.62:0/4077850629 shutdown_connections 2023-07-14T16:25:17.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5637403bbc70 0x5637403c8410 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5637404252c0 0x5637404276d0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5637403ba4d0 0x5637403cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 --2- 172.21.15.62:0/4077850629 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5637403bae20 0x5637403c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 -- 172.21.15.62:0/4077850629 >> 172.21.15.62:0/4077850629 conn(0x5637403afda0 msgr2=0x5637403c2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:17.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb0ed7fa700 0 client.18168 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 -- 172.21.15.62:0/4077850629 shutdown_connections 2023-07-14T16:25:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 1 -- 172.21.15.62:0/4077850629 wait complete. 2023-07-14T16:25:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.660+0000 7fb11fb57b80 20 client.18168 trim_cache size 0 max 16384 2023-07-14T16:25:17.649 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:25:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:25:17.650 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:25:17.650 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:25:17.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 -- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0f00a11d0 msgr2=0x7fb0f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0f00a11d0 0x7fb0f00a3680 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00b780 tx=0x7fb0fc006230 comp rx=0 tx=0).stop 2023-07-14T16:25:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 -- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1040ffae0 msgr2=0x7fb1041958a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1040ffae0 0x7fb1041958a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb10000e970 tx=0x7fb10000ec80 comp rx=0 tx=0).stop 2023-07-14T16:25:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 -- 172.21.15.62:0/1012488258 shutdown_connections 2023-07-14T16:25:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb104101660 0x7fb10419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1040ffae0 0x7fb1041958a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0f00a11d0 0x7fb0f00a3680 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 --2- 172.21.15.62:0/1012488258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb104100cd0 0x7fb104195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 -- 172.21.15.62:0/1012488258 >> 172.21.15.62:0/1012488258 conn(0x7fb1040fb0e0 msgr2=0x7fb1040fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:17.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 -- 172.21.15.62:0/1012488258 shutdown_connections 2023-07-14T16:25:17.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.661+0000 7fb10c1ff700 1 -- 172.21.15.62:0/1012488258 wait complete. 2023-07-14T16:25:17.656 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:25:17.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:25:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 -- 172.21.15.62:0/154052556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 msgr2=0x7ff1d80fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/154052556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 0x7ff1d80fede0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009990 tx=0x7ff1d4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 -- 172.21.15.62:0/154052556 shutdown_connections 2023-07-14T16:25:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/154052556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d8100550 0x7ff1d8106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/154052556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 0x7ff1d80fede0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/154052556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1d80ffbc0 0x7ff1d8100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 -- 172.21.15.62:0/154052556 >> 172.21.15.62:0/154052556 conn(0x7ff1d80f9fd0 msgr2=0x7ff1d80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 -- 172.21.15.62:0/154052556 shutdown_connections 2023-07-14T16:25:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.915+0000 7ff1e07ff700 1 -- 172.21.15.62:0/154052556 wait complete. 2023-07-14T16:25:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.916+0000 7ff1e07ff700 1 Processor -- start 2023-07-14T16:25:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1e07ff700 1 -- start start 2023-07-14T16:25:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1e07ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 0x7ff1d8102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1e07ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1d80ffbc0 0x7ff1d8100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1e07ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d8100550 0x7ff1d81014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1e07ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8199590 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1e07ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8101a20 con 0x7ff1d8100550 2023-07-14T16:25:17.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1e07ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8101be0 con 0x7ff1d80ffbc0 2023-07-14T16:25:17.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1de59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 0x7ff1d8102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.917+0000 7ff1de59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 0x7ff1d8102910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51892/0 (socket says 172.21.15.62:51892) 2023-07-14T16:25:17.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1de59b700 1 -- 172.21.15.62:0/1455385041 learned_addr learned my addr 172.21.15.62:0/1455385041 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1ddd9a700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1d80ffbc0 0x7ff1d8100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1ded9c700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d8100550 0x7ff1d81014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1de59b700 1 -- 172.21.15.62:0/1455385041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1d80ffbc0 msgr2=0x7ff1d8100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1de59b700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1d80ffbc0 0x7ff1d8100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1de59b700 1 -- 172.21.15.62:0/1455385041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d8100550 msgr2=0x7ff1d81014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1de59b700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d8100550 0x7ff1d81014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:17.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1de59b700 1 -- 172.21.15.62:0/1455385041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40095f0 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.918+0000 7ff1de59b700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 0x7ff1d8102910 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4004730 tx=0x7ff1d4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:17.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.919+0000 7ff1cf7fe700 1 -- 172.21.15.62:0/1455385041 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d401d070 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.919+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d8101e00 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.919+0000 7ff1cf7fe700 1 -- 172.21.15.62:0/1455385041 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d4004c60 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.919+0000 7ff1cf7fe700 1 -- 172.21.15.62:0/1455385041 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4003af0 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.919+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d81a4180 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.920+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d810a0a0 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.923+0000 7ff1cf7fe700 1 -- 172.21.15.62:0/1455385041 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff1d40047d0 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.924+0000 7ff1cf7fe700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1c40a11a0 0x7ff1c40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:17.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.925+0000 7ff1cf7fe700 1 -- 172.21.15.62:0/1455385041 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff1d40cdc20 con 0x7ff1d80fe9d0 2023-07-14T16:25:17.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.925+0000 7ff1ddd9a700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1c40a11a0 0x7ff1c40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.926+0000 7ff1ddd9a700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1c40a11a0 0x7ff1c40a3650 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8005950 tx=0x7ff1c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:17.927+0000 7ff1cf7fe700 1 -- 172.21.15.62:0/1455385041 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff1d40973b0 con 0x7ff1d80fe9d0 2023-07-14T16:25:18.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.219+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1d81a4790 con 0x7ff1d80fe9d0 2023-07-14T16:25:18.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.220+0000 7ff1cf7fe700 1 -- 172.21.15.62:0/1455385041 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7ff1d409a9d0 con 0x7ff1d80fe9d0 2023-07-14T16:25:18.196 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:25:18.197 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:25:18.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.222+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1c40a11a0 msgr2=0x7ff1c40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.222+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1c40a11a0 0x7ff1c40a3650 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8005950 tx=0x7ff1c800b410 comp rx=0 tx=0).stop 2023-07-14T16:25:18.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.222+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 msgr2=0x7ff1d8102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.222+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 0x7ff1d8102910 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4004730 tx=0x7ff1d4009580 comp rx=0 tx=0).stop 2023-07-14T16:25:18.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.223+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 shutdown_connections 2023-07-14T16:25:18.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.223+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1c40a11a0 0x7ff1c40a3650 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.223+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d8100550 0x7ff1d81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.223+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1d80fe9d0 0x7ff1d8102910 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.223+0000 7ff1e07ff700 1 --2- 172.21.15.62:0/1455385041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1d80ffbc0 0x7ff1d8100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.223+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 >> 172.21.15.62:0/1455385041 conn(0x7ff1d80f9fd0 msgr2=0x7ff1d8067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.223+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 shutdown_connections 2023-07-14T16:25:18.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.223+0000 7ff1e07ff700 1 -- 172.21.15.62:0/1455385041 wait complete. 2023-07-14T16:25:18.201 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:25:18.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:25:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.455+0000 7f6a27dff700 1 -- 172.21.15.62:0/2736833525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 msgr2=0x7f6a20107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.455+0000 7f6a27dff700 1 --2- 172.21.15.62:0/2736833525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 0x7f6a20107f40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b0a0 tx=0x7f6a1800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.456+0000 7f6a27dff700 1 -- 172.21.15.62:0/2736833525 shutdown_connections 2023-07-14T16:25:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.456+0000 7f6a27dff700 1 --2- 172.21.15.62:0/2736833525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 0x7f6a20107f40 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.456+0000 7f6a27dff700 1 --2- 172.21.15.62:0/2736833525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a200ffb00 0x7f6a200fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.456+0000 7f6a27dff700 1 --2- 172.21.15.62:0/2736833525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a20100cf0 0x7f6a20101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.456+0000 7f6a27dff700 1 -- 172.21.15.62:0/2736833525 >> 172.21.15.62:0/2736833525 conn(0x7f6a200fb0e0 msgr2=0x7f6a200fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.456+0000 7f6a27dff700 1 -- 172.21.15.62:0/2736833525 shutdown_connections 2023-07-14T16:25:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.456+0000 7f6a27dff700 1 -- 172.21.15.62:0/2736833525 wait complete. 2023-07-14T16:25:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.457+0000 7f6a27dff700 1 Processor -- start 2023-07-14T16:25:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.457+0000 7f6a27dff700 1 -- start start 2023-07-14T16:25:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.457+0000 7f6a27dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a200ffb00 0x7f6a20197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.457+0000 7f6a27dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a20100cf0 0x7f6a201980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.457+0000 7f6a27dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 0x7f6a20198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.457+0000 7f6a27dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a201995b0 con 0x7f6a20101680 2023-07-14T16:25:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.457+0000 7f6a27dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a20199720 con 0x7f6a20100cf0 2023-07-14T16:25:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.457+0000 7f6a27dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a201998e0 con 0x7f6a200ffb00 2023-07-14T16:25:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a25b9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a200ffb00 0x7f6a20197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a25b9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a200ffb00 0x7f6a20197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42406/0 (socket says 172.21.15.62:42406) 2023-07-14T16:25:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a25b9b700 1 -- 172.21.15.62:0/252788237 learned_addr learned my addr 172.21.15.62:0/252788237 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a2639c700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 0x7f6a20198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a2539a700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a20100cf0 0x7f6a201980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a2639c700 1 -- 172.21.15.62:0/252788237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a200ffb00 msgr2=0x7f6a20197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a2639c700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a200ffb00 0x7f6a20197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a2639c700 1 -- 172.21.15.62:0/252788237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a20100cf0 msgr2=0x7f6a201980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a2639c700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a20100cf0 0x7f6a201980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.458+0000 7f6a2639c700 1 -- 172.21.15.62:0/252788237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a18009d00 con 0x7f6a20101680 2023-07-14T16:25:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.459+0000 7f6a2639c700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 0x7f6a20198f50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009fd0 tx=0x7f6a1800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.459+0000 7f6a16ffd700 1 -- 172.21.15.62:0/252788237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800e040 con 0x7f6a20101680 2023-07-14T16:25:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.459+0000 7f6a16ffd700 1 -- 172.21.15.62:0/252788237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a18004c50 con 0x7f6a20101680 2023-07-14T16:25:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.459+0000 7f6a27dff700 1 -- 172.21.15.62:0/252788237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a20199b60 con 0x7f6a20101680 2023-07-14T16:25:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.459+0000 7f6a16ffd700 1 -- 172.21.15.62:0/252788237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18005040 con 0x7f6a20101680 2023-07-14T16:25:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.459+0000 7f6a27dff700 1 -- 172.21.15.62:0/252788237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a201a0380 con 0x7f6a20101680 2023-07-14T16:25:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.463+0000 7f6a16ffd700 1 -- 172.21.15.62:0/252788237 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6a1801d470 con 0x7f6a20101680 2023-07-14T16:25:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.464+0000 7f6a3b74fb80 1 Processor -- start 2023-07-14T16:25:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.464+0000 7f6a3b74fb80 1 -- start start 2023-07-14T16:25:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559216e5d6b0 0x559217001300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x5592170021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592170026e0 0x559217002b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559216f06c50 con 0x559216e5d6b0 2023-07-14T16:25:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559216f599e0 con 0x5592170026e0 2023-07-14T16:25:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559216f52e70 con 0x559217001840 2023-07-14T16:25:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a25b9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x5592170021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a25b9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x5592170021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42420/0 (socket says 172.21.15.62:42420) 2023-07-14T16:25:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a25b9b700 1 -- 172.21.15.62:0/2447039231 learned_addr learned my addr 172.21.15.62:0/2447039231 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a2539a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559216e5d6b0 0x559217001300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a16ffd700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6a0c0a1150 0x7f6a0c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.465+0000 7f6a2539a700 1 --2- 172.21.15.62:0/2447039231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592170026e0 0x559217002b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.466+0000 7f6a16ffd700 1 -- 172.21.15.62:0/252788237 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6a180cdf60 con 0x7f6a20101680 2023-07-14T16:25:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.466+0000 7f6a25b9b700 1 -- 172.21.15.62:0/2447039231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592170026e0 msgr2=0x559217002b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.466+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/2447039231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592170026e0 0x559217002b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.466+0000 7f6a25b9b700 1 -- 172.21.15.62:0/2447039231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559216e5d6b0 msgr2=0x559217001300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.466+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/2447039231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559216e5d6b0 0x559217001300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.466+0000 7f6a25b9b700 1 -- 172.21.15.62:0/2447039231 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559216d80d80 con 0x559217001840 2023-07-14T16:25:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.466+0000 7f6a2639c700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6a0c0a1150 0x7f6a0c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.466+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/2447039231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x5592170021a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c007e20 tx=0x7f6a1c0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee073a5881023c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.467+0000 7f6a2639c700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6a0c0a1150 0x7f6a0c0a3600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6a180062a0 tx=0x7f6a1800bf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.467+0000 7f6a0bfff700 1 -- 172.21.15.62:0/2447039231 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c010e10 con 0x559217001840 2023-07-14T16:25:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.467+0000 7f6a0bfff700 1 -- 172.21.15.62:0/2447039231 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a1c015070 con 0x559217001840 2023-07-14T16:25:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.467+0000 7f6a0bfff700 1 -- 172.21.15.62:0/2447039231 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c019630 con 0x559217001840 2023-07-14T16:25:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2447039231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 msgr2=0x5592170021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2447039231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x5592170021a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c007e20 tx=0x7f6a1c0106b0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2447039231 shutdown_connections 2023-07-14T16:25:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2447039231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592170026e0 0x559217002b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2447039231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559216e5d6b0 0x559217001300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2447039231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x5592170021a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2447039231 >> 172.21.15.62:0/2447039231 conn(0x559216ff6be0 msgr2=0x559216ffbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2447039231 shutdown_connections 2023-07-14T16:25:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2447039231 wait complete. 2023-07-14T16:25:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.468+0000 7f6a3b74fb80 1 -- start start 2023-07-14T16:25:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.469+0000 7f6a3b74fb80 1 Processor -- start 2023-07-14T16:25:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.469+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.469+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559217001840 0x55921700ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.469+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592170026e0 0x559217011dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.469+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559216ef2a60 con 0x559217001840 2023-07-14T16:25:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.469+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559216ef2760 con 0x559216e5d6b0 2023-07-14T16:25:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.469+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559216dd68c0 con 0x5592170026e0 2023-07-14T16:25:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.469+0000 7f6a3b74fb80 10 client.0 _ll_register_callbacks cb 0x559216ef0700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a25b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559217001840 0x55921700ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a25b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559217001840 0x55921700ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51914/0 (socket says 172.21.15.62:51914) 2023-07-14T16:25:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a25b9b700 1 -- 172.21.15.62:0/2115629663 learned_addr learned my addr 172.21.15.62:0/2115629663 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a2539a700 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592170026e0 0x559217011dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a2539a700 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a25b9b700 1 -- 172.21.15.62:0/2115629663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592170026e0 msgr2=0x559217011dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592170026e0 0x559217011dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a25b9b700 1 -- 172.21.15.62:0/2115629663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 msgr2=0x55921700a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.470+0000 7f6a25b9b700 1 -- 172.21.15.62:0/2115629663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c007a50 con 0x559217001840 2023-07-14T16:25:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.471+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559217001840 0x55921700ad90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c007730 tx=0x7f6a1c011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98507c2536ccf7e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.471+0000 7f6a097fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.471+0000 7f6a097fa700 1 -- 172.21.15.62:0/2115629663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c011440 con 0x559217001840 2023-07-14T16:25:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.471+0000 7f6a097fa700 1 -- 172.21.15.62:0/2115629663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6a1c0115a0 con 0x559217001840 2023-07-14T16:25:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.471+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2115629663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559216eac490 con 0x559217001840 2023-07-14T16:25:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a097fa700 1 -- 172.21.15.62:0/2115629663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a1c011760 con 0x559217001840 2023-07-14T16:25:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a3b74fb80 10 client.18186 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2115629663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559216d69c00 con 0x559217001840 2023-07-14T16:25:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a097fa700 1 -- 172.21.15.62:0/2115629663 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559216eac490 con 0x559217001840 2023-07-14T16:25:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a097fa700 1 -- 172.21.15.62:0/2115629663 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6a1c004830 con 0x559217001840 2023-07-14T16:25:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a3b74fb80 10 client.18186 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a3b74fb80 10 client.18186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a3b74fb80 20 client.18186 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5592170165a0 0x5592170189b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a3b74fb80 4 client.18186 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.472+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2115629663 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559216f4be50 con 0x5592170165a0 2023-07-14T16:25:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.473+0000 7f6a2539a700 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5592170165a0 0x5592170189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.473+0000 7f6a2539a700 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5592170165a0 0x5592170189b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5666ce019558422 server_cookie=9106479d22786aaf in_seq=0 out_seq=0 2023-07-14T16:25:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.473+0000 7f6a097fa700 10 client.18186 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.474+0000 7f6a097fa700 1 -- 172.21.15.62:0/2115629663 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559216f4be50 con 0x5592170165a0 2023-07-14T16:25:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.474+0000 7f6a097fa700 10 client.18186 handle_command_reply: tid=0 2023-07-14T16:25:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.474+0000 7f6a3b74fb80 1 client.18186 shutdown 2023-07-14T16:25:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.474+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2115629663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559217001840 msgr2=0x55921700ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.474+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559217001840 0x55921700ad90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c007730 tx=0x7f6a1c011cc0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2115629663 shutdown_connections 2023-07-14T16:25:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592170026e0 0x559217011dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5592170165a0 0x5592170189b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/2115629663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559217001840 0x55921700ad90 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2115629663 >> 172.21.15.62:0/2115629663 conn(0x559216ff6be0 msgr2=0x559216ffb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a097fa700 0 client.18186 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2115629663 shutdown_connections 2023-07-14T16:25:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/2115629663 wait complete. 2023-07-14T16:25:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.475+0000 7f6a3b74fb80 20 client.18186 trim_cache size 0 max 16384 2023-07-14T16:25:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 -- 172.21.15.62:0/886745412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8220102e80 msgr2=0x7f8220105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 --2- 172.21.15.62:0/886745412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8220102e80 0x7f8220105260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f82140098f0 tx=0x7f8214009c00 comp rx=0 tx=0).stop 2023-07-14T16:25:18.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 -- 172.21.15.62:0/886745412 shutdown_connections 2023-07-14T16:25:18.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 --2- 172.21.15.62:0/886745412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82201057a0 0x7f8220107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 --2- 172.21.15.62:0/886745412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8220100560 0x7f8220102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 --2- 172.21.15.62:0/886745412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8220102e80 0x7f8220105260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 -- 172.21.15.62:0/886745412 >> 172.21.15.62:0/886745412 conn(0x7f82200fa000 msgr2=0x7f82200fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 -- 172.21.15.62:0/886745412 shutdown_connections 2023-07-14T16:25:18.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.487+0000 7f82269ff700 1 -- 172.21.15.62:0/886745412 wait complete. 2023-07-14T16:25:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f82269ff700 1 Processor -- start 2023-07-14T16:25:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f82269ff700 1 -- start start 2023-07-14T16:25:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f82269ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8220100560 0x7f82201979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f82269ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8220102e80 0x7f8220197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f82269ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82201057a0 0x7f822019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f82269ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019f570 con 0x7f8220100560 2023-07-14T16:25:18.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f82269ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019f6b0 con 0x7f8220102e80 2023-07-14T16:25:18.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f82269ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822019f840 con 0x7f82201057a0 2023-07-14T16:25:18.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f821ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8220100560 0x7f82201979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f821ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8220100560 0x7f82201979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51934/0 (socket says 172.21.15.62:51934) 2023-07-14T16:25:18.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.489+0000 7f821ffff700 1 -- 172.21.15.62:0/3017531604 learned_addr learned my addr 172.21.15.62:0/3017531604 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821f7fe700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8220102e80 0x7f8220197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f8224f9c700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82201057a0 0x7f822019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821ffff700 1 -- 172.21.15.62:0/3017531604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82201057a0 msgr2=0x7f822019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821ffff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82201057a0 0x7f822019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821ffff700 1 -- 172.21.15.62:0/3017531604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8220102e80 msgr2=0x7f8220197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821ffff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8220102e80 0x7f8220197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821ffff700 1 -- 172.21.15.62:0/3017531604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82100095f0 con 0x7f8220100560 2023-07-14T16:25:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821f7fe700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8220102e80 0x7f8220197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:18.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821ffff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8220100560 0x7f82201979f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f821000ec10 tx=0x7f821000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f821d7fa700 1 -- 172.21.15.62:0/3017531604 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210009a10 con 0x7f8220100560 2023-07-14T16:25:18.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.490+0000 7f82269ff700 1 -- 172.21.15.62:0/3017531604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8214009520 con 0x7f8220100560 2023-07-14T16:25:18.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.491+0000 7f821d7fa700 1 -- 172.21.15.62:0/3017531604 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f821000cde0 con 0x7f8220100560 2023-07-14T16:25:18.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.491+0000 7f82269ff700 1 -- 172.21.15.62:0/3017531604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822019fef0 con 0x7f8220100560 2023-07-14T16:25:18.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.491+0000 7f821d7fa700 1 -- 172.21.15.62:0/3017531604 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210010430 con 0x7f8220100560 2023-07-14T16:25:18.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.494+0000 7f821d7fa700 1 -- 172.21.15.62:0/3017531604 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8210003680 con 0x7f8220100560 2023-07-14T16:25:18.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.495+0000 7f823a30eb80 1 Processor -- start 2023-07-14T16:25:18.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.495+0000 7f823a30eb80 1 -- start start 2023-07-14T16:25:18.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e7417196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e74171a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f823a30eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74161dcb0 con 0x55e7417184d0 2023-07-14T16:25:18.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f823a30eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741670a40 con 0x55e741719c70 2023-07-14T16:25:18.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f823a30eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741669ed0 con 0x55e741718e20 2023-07-14T16:25:18.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f8224f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f8224f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417188e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51948/0 (socket says 172.21.15.62:51948) 2023-07-14T16:25:18.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f8224f9c700 1 -- 172.21.15.62:0/3175313209 learned_addr learned my addr 172.21.15.62:0/3175313209 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.496+0000 7f821f7fe700 1 --2- 172.21.15.62:0/3175313209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e74171a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f821d7fa700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82080a11d0 0x7f82080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f821f7fe700 1 -- 172.21.15.62:0/3175313209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 msgr2=0x55e7417196e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:25:18.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f821f7fe700 1 --2- 172.21.15.62:0/3175313209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e7417196e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f821f7fe700 1 -- 172.21.15.62:0/3175313209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 msgr2=0x55e7417188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f821f7fe700 1 --2- 172.21.15.62:0/3175313209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f821f7fe700 1 -- 172.21.15.62:0/3175313209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e741497de0 con 0x55e741719c70 2023-07-14T16:25:18.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f821d7fa700 1 -- 172.21.15.62:0/3017531604 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8210014070 con 0x7f8220100560 2023-07-14T16:25:18.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f821f7fe700 1 --2- 172.21.15.62:0/3175313209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e74171a0e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8214009230 tx=0x7f8214009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b169986dd1891cf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.497+0000 7f8224f9c700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82080a11d0 0x7f82080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f6a3b74fb80 1 Processor -- start 2023-07-14T16:25:18.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f82067fc700 1 -- 172.21.15.62:0/3175313209 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821401c070 con 0x55e741719c70 2023-07-14T16:25:18.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f82067fc700 1 -- 172.21.15.62:0/3175313209 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f821400fba0 con 0x55e741719c70 2023-07-14T16:25:18.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f82067fc700 1 -- 172.21.15.62:0/3175313209 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214005040 con 0x55e741719c70 2023-07-14T16:25:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f6a3b74fb80 1 -- start start 2023-07-14T16:25:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559216e5d6b0 0x5592170012c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559217001840 0x559217011cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 0x559217009600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559216ff8d00 con 0x5592170026e0 2023-07-14T16:25:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592170087b0 con 0x559217001840 2023-07-14T16:25:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.498+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55921700b9a0 con 0x559216e5d6b0 2023-07-14T16:25:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559216e5d6b0 0x5592170012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559216e5d6b0 0x5592170012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42460/0 (socket says 172.21.15.62:42460) 2023-07-14T16:25:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 -- 172.21.15.62:0/460373045 learned_addr learned my addr 172.21.15.62:0/460373045 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 -- 172.21.15.62:0/3175313209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 msgr2=0x55e74171a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 --2- 172.21.15.62:0/3175313209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e74171a0e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8214009230 tx=0x7f8214009460 comp rx=0 tx=0).stop 2023-07-14T16:25:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f8224f9c700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82080a11d0 0x7f82080a3680 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8218009fd0 tx=0x7f8218009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 0x559217009600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559217001840 0x559217011cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 -- 172.21.15.62:0/3175313209 shutdown_connections 2023-07-14T16:25:18.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 --2- 172.21.15.62:0/3175313209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e74171a0e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 --2- 172.21.15.62:0/3175313209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 --2- 172.21.15.62:0/3175313209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e7417196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 -- 172.21.15.62:0/3175313209 >> 172.21.15.62:0/3175313209 conn(0x55e74170dda0 msgr2=0x55e7417131d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 -- 172.21.15.62:0/3175313209 shutdown_connections 2023-07-14T16:25:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 -- 172.21.15.62:0/460373045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559217001840 msgr2=0x559217011cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559217001840 0x559217011cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 -- 172.21.15.62:0/460373045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 msgr2=0x559217009600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 0x559217009600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f6a2539a700 1 -- 172.21.15.62:0/460373045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559216da71f0 con 0x559216e5d6b0 2023-07-14T16:25:18.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.499+0000 7f823a30eb80 1 -- 172.21.15.62:0/3175313209 wait complete. 2023-07-14T16:25:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.500+0000 7f6a2539a700 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559216e5d6b0 0x5592170012c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005950 tx=0x7f6a10004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fd81a988a18158 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.500+0000 7f6a14bff700 1 -- 172.21.15.62:0/460373045 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a10004410 con 0x559216e5d6b0 2023-07-14T16:25:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.500+0000 7f6a14bff700 1 -- 172.21.15.62:0/460373045 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a10004570 con 0x559216e5d6b0 2023-07-14T16:25:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.500+0000 7f6a14bff700 1 -- 172.21.15.62:0/460373045 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a10026040 con 0x559216e5d6b0 2023-07-14T16:25:18.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.500+0000 7f823a30eb80 1 -- start start 2023-07-14T16:25:18.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.500+0000 7f823a30eb80 1 Processor -- start 2023-07-14T16:25:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/460373045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559216e5d6b0 msgr2=0x5592170012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559216e5d6b0 0x5592170012c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005950 tx=0x7f6a10004750 comp rx=0 tx=0).stop 2023-07-14T16:25:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/460373045 shutdown_connections 2023-07-14T16:25:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 0x559217009600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559216e5d6b0 0x5592170012c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/460373045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559217001840 0x559217011cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/460373045 >> 172.21.15.62:0/460373045 conn(0x559216ff6be0 msgr2=0x55921701d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741722620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741729660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f823a30eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741609ac0 con 0x55e7417184d0 2023-07-14T16:25:18.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f823a30eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7416097c0 con 0x55e741719c70 2023-07-14T16:25:18.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f823a30eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7414ed920 con 0x55e741718e20 2023-07-14T16:25:18.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f823a30eb80 10 client.0 _ll_register_callbacks cb 0x55e741607760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/460373045 shutdown_connections 2023-07-14T16:25:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.501+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/460373045 wait complete. 2023-07-14T16:25:18.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741722620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f6a3b74fb80 1 -- start start 2023-07-14T16:25:18.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f8224f9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741729660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741722620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42476/0 (socket says 172.21.15.62:42476) 2023-07-14T16:25:18.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51962/0 (socket says 172.21.15.62:51962) 2023-07-14T16:25:18.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 -- 172.21.15.62:0/2340271661 learned_addr learned my addr 172.21.15.62:0/2340271661 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f6a3b74fb80 1 Processor -- start 2023-07-14T16:25:18.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 -- 172.21.15.62:0/2340271661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 msgr2=0x55e741729660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741729660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 -- 172.21.15.62:0/2340271661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 msgr2=0x55e7417220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 -- 172.21.15.62:0/2340271661 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82201087f0 con 0x55e741718e20 2023-07-14T16:25:18.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f821f7fe700 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741722620 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8214009c40 tx=0x7f8214010f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7d4c6920521ec8d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.502+0000 7f81ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:25:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x55921700d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a3b74fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 0x5592170109a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559217010ee0 con 0x5592170026e0 2023-07-14T16:25:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559217011050 con 0x559216e5d6b0 2023-07-14T16:25:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a3b74fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559217011210 con 0x559217001840 2023-07-14T16:25:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a3b74fb80 10 client.0 _ll_register_callbacks cb 0x7f6a10009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:18.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f81ff7fe700 1 -- 172.21.15.62:0/2340271661 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821401c050 con 0x55e741718e20 2023-07-14T16:25:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f823a30eb80 1 -- 172.21.15.62:0/2340271661 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7415c34f0 con 0x55e741718e20 2023-07-14T16:25:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f81ff7fe700 1 -- 172.21.15.62:0/2340271661 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8214005040 con 0x55e741718e20 2023-07-14T16:25:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a2539a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f81ff7fe700 1 -- 172.21.15.62:0/2340271661 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8214005200 con 0x55e741718e20 2023-07-14T16:25:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a2539a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 0x5592170109a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a2539a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46616/0 (socket says 172.21.15.62:46616) 2023-07-14T16:25:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a2539a700 1 -- 172.21.15.62:0/3046547049 learned_addr learned my addr 172.21.15.62:0/3046547049 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f823a30eb80 10 client.27712 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f823a30eb80 1 -- 172.21.15.62:0/2340271661 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e741480c60 con 0x55e741718e20 2023-07-14T16:25:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.503+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x55921700d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f81ff7fe700 1 -- 172.21.15.62:0/2340271661 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e7415c34f0 con 0x55e741718e20 2023-07-14T16:25:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f6a2539a700 1 -- 172.21.15.62:0/3046547049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 msgr2=0x55921700d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f6a2539a700 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x55921700d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f6a2539a700 1 -- 172.21.15.62:0/3046547049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 msgr2=0x5592170109a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f6a2539a700 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 0x5592170109a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f6a2539a700 1 -- 172.21.15.62:0/3046547049 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1001b040 con 0x559216e5d6b0 2023-07-14T16:25:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f81ff7fe700 1 -- 172.21.15.62:0/2340271661 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8214027030 con 0x55e741718e20 2023-07-14T16:25:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f823a30eb80 10 client.27712 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f823a30eb80 10 client.27712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:25:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f823a30eb80 20 client.27712 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f823a30eb80 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e74172da60 0x55e74172fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f823a30eb80 4 client.27712 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f823a30eb80 1 -- 172.21.15.62:0/2340271661 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e741662eb0 con 0x55e74172da60 2023-07-14T16:25:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.504+0000 7f6a2539a700 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700d440 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6a10013bb0 tx=0x7f6a10014fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d5cd076bb8c6d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f6a097fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:25:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f6a097fa700 1 -- 172.21.15.62:0/3046547049 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a10013040 con 0x559216e5d6b0 2023-07-14T16:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f6a097fa700 1 -- 172.21.15.62:0/3046547049 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6a100131a0 con 0x559216e5d6b0 2023-07-14T16:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/3046547049 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a1c015070 con 0x559216e5d6b0 2023-07-14T16:25:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f821ffff700 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e74172da60 0x55e74172fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f6a097fa700 1 -- 172.21.15.62:0/3046547049 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a10013360 con 0x559216e5d6b0 2023-07-14T16:25:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f6a3b74fb80 10 client.27827 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.505+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/3046547049 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559216d7f250 con 0x559216e5d6b0 2023-07-14T16:25:18.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f821ffff700 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e74172da60 0x55e74172fe70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0d60cf32683e06a server_cookie=cc1bdc61c1d6fa89 in_seq=0 out_seq=0 2023-07-14T16:25:18.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f81ff7fe700 10 client.27712 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:25:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a097fa700 1 -- 172.21.15.62:0/3046547049 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a1c015070 con 0x559216e5d6b0 2023-07-14T16:25:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a097fa700 1 -- 172.21.15.62:0/3046547049 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6a10003960 con 0x559216e5d6b0 2023-07-14T16:25:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a3b74fb80 10 client.27827 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a3b74fb80 10 client.27827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a3b74fb80 20 client.27827 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55921706ace0 0x55921706d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a3b74fb80 4 client.27827 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:25:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/3046547049 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559216f0dff0 con 0x55921706ace0 2023-07-14T16:25:18.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f81ff7fe700 1 -- 172.21.15.62:0/2340271661 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e741662eb0 con 0x55e74172da60 2023-07-14T16:25:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f81ff7fe700 10 client.27712 handle_command_reply: tid=0 2023-07-14T16:25:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.506+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55921706ace0 0x55921706d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 client.27712 shutdown 2023-07-14T16:25:18.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 -- 172.21.15.62:0/2340271661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 msgr2=0x55e741722620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741722620 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8214009c40 tx=0x7f8214010f50 comp rx=0 tx=0).stop 2023-07-14T16:25:18.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 -- 172.21.15.62:0/2340271661 shutdown_connections 2023-07-14T16:25:18.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e74172da60 0x55e74172fe70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741729660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f81ff7fe700 0 client.27712 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:25:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 --2- 172.21.15.62:0/2340271661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741722620 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 -- 172.21.15.62:0/2340271661 >> 172.21.15.62:0/2340271661 conn(0x55e74170dda0 msgr2=0x55e7417128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f6a25b9b700 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55921706ace0 0x55921706d0f0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c75b2a654733340 server_cookie=f7c2dd8ed5b8089d in_seq=0 out_seq=0 2023-07-14T16:25:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f6a097fa700 10 client.27827 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:18.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 -- 172.21.15.62:0/2340271661 shutdown_connections 2023-07-14T16:25:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 1 -- 172.21.15.62:0/2340271661 wait complete. 2023-07-14T16:25:18.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.507+0000 7f823a30eb80 20 client.27712 trim_cache size 0 max 16384 2023-07-14T16:25:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a097fa700 1 -- 172.21.15.62:0/3046547049 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559216f0dff0 con 0x55921706ace0 2023-07-14T16:25:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a097fa700 10 client.27827 handle_command_reply: tid=0 2023-07-14T16:25:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 client.27827 shutdown 2023-07-14T16:25:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/3046547049 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 msgr2=0x55921700d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700d440 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6a10013bb0 tx=0x7f6a10014fd0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/3046547049 shutdown_connections 2023-07-14T16:25:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5592170026e0 0x5592170109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55921706ace0 0x55921706d0f0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559216e5d6b0 0x55921700d440 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 --2- 172.21.15.62:0/3046547049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559217001840 0x55921700d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/3046547049 >> 172.21.15.62:0/3046547049 conn(0x559216ff6be0 msgr2=0x559217005530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.518+0000 7f6a097fa700 0 client.27827 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.519+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/3046547049 shutdown_connections 2023-07-14T16:25:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.519+0000 7f6a3b74fb80 1 -- 172.21.15.62:0/3046547049 wait complete. 2023-07-14T16:25:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.519+0000 7f6a3b74fb80 20 client.27827 trim_cache size 0 max 16384 2023-07-14T16:25:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:25:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:25:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:25:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:25:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 -- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6a0c0a1150 msgr2=0x7f6a0c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6a0c0a1150 0x7f6a0c0a3600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6a180062a0 tx=0x7f6a1800bf50 comp rx=0 tx=0).stop 2023-07-14T16:25:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 -- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 msgr2=0x7f6a20198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 0x7f6a20198f50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009fd0 tx=0x7f6a1800ba50 comp rx=0 tx=0).stop 2023-07-14T16:25:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 -- 172.21.15.62:0/252788237 shutdown_connections 2023-07-14T16:25:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a20101680 0x7f6a20198f50 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6a200ffb00 0x7f6a20197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6a0c0a1150 0x7f6a0c0a3600 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 --2- 172.21.15.62:0/252788237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6a20100cf0 0x7f6a201980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 -- 172.21.15.62:0/252788237 >> 172.21.15.62:0/252788237 conn(0x7f6a200fb0e0 msgr2=0x7f6a200fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 -- 172.21.15.62:0/252788237 shutdown_connections 2023-07-14T16:25:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:18.520+0000 7f6a27dff700 1 -- 172.21.15.62:0/252788237 wait complete. 2023-07-14T16:25:18.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.526+0000 7f823a30eb80 1 Processor -- start 2023-07-14T16:25:18.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.527+0000 7f823a30eb80 1 -- start start 2023-07-14T16:25:18.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.527+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e741729f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.527+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741729580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.527+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e7417220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.527+0000 7f823a30eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741729ac0 con 0x55e7417184d0 2023-07-14T16:25:18.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.527+0000 7f823a30eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741714070 con 0x55e741719c70 2023-07-14T16:25:18.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.527+0000 7f823a30eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741714610 con 0x55e741718e20 2023-07-14T16:25:18.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.527+0000 7f821ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e741729f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741729580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e741729f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51972/0 (socket says 172.21.15.62:51972) 2023-07-14T16:25:18.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741729580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42488/0 (socket says 172.21.15.62:42488) 2023-07-14T16:25:18.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821ffff700 1 -- 172.21.15.62:0/1981268278 learned_addr learned my addr 172.21.15.62:0/1981268278 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f8224f9c700 1 --2- 172.21.15.62:0/1981268278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e7417220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821ffff700 1 -- 172.21.15.62:0/1981268278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 msgr2=0x55e741729580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821ffff700 1 --2- 172.21.15.62:0/1981268278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741729580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821ffff700 1 -- 172.21.15.62:0/1981268278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 msgr2=0x55e7417220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821ffff700 1 --2- 172.21.15.62:0/1981268278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e7417220c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821ffff700 1 -- 172.21.15.62:0/1981268278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7414be250 con 0x55e7417184d0 2023-07-14T16:25:18.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.528+0000 7f821ffff700 1 --2- 172.21.15.62:0/1981268278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e741729f50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f821009a230 tx=0x7f821000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739edd83b7485300 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.529+0000 7f8206ffd700 1 -- 172.21.15.62:0/1981268278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821009a2a0 con 0x55e7417184d0 2023-07-14T16:25:18.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.529+0000 7f8206ffd700 1 -- 172.21.15.62:0/1981268278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f821009e2d0 con 0x55e7417184d0 2023-07-14T16:25:18.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.529+0000 7f8206ffd700 1 -- 172.21.15.62:0/1981268278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821009a500 con 0x55e7417184d0 2023-07-14T16:25:18.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 -- 172.21.15.62:0/1981268278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 msgr2=0x55e741729f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 --2- 172.21.15.62:0/1981268278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e741729f50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f821009a230 tx=0x7f821000cf50 comp rx=0 tx=0).stop 2023-07-14T16:25:18.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 -- 172.21.15.62:0/1981268278 shutdown_connections 2023-07-14T16:25:18.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 --2- 172.21.15.62:0/1981268278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e7417220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 --2- 172.21.15.62:0/1981268278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e741729f50 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 --2- 172.21.15.62:0/1981268278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741729580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 -- 172.21.15.62:0/1981268278 >> 172.21.15.62:0/1981268278 conn(0x55e74170dda0 msgr2=0x55e7417347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 -- 172.21.15.62:0/1981268278 shutdown_connections 2023-07-14T16:25:18.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.530+0000 7f823a30eb80 1 -- 172.21.15.62:0/1981268278 wait complete. 2023-07-14T16:25:18.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.531+0000 7f823a30eb80 1 -- start start 2023-07-14T16:25:18.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.531+0000 7f823a30eb80 1 Processor -- start 2023-07-14T16:25:18.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741725cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f823a30eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741728cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f823a30eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7417291f0 con 0x55e7417184d0 2023-07-14T16:25:18.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f823a30eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741729360 con 0x55e741719c70 2023-07-14T16:25:18.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f823a30eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741729520 con 0x55e741718e20 2023-07-14T16:25:18.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f823a30eb80 10 client.0 _ll_register_callbacks cb 0x7f8218009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:18.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f821ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f821ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417257b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51982/0 (socket says 172.21.15.62:51982) 2023-07-14T16:25:18.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f821ffff700 1 -- 172.21.15.62:0/518252484 learned_addr learned my addr 172.21.15.62:0/518252484 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.532+0000 7f821f7fe700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741725cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f8224f9c700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741728cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f821f7fe700 1 -- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 msgr2=0x55e741728cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f821f7fe700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741728cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f821f7fe700 1 -- 172.21.15.62:0/518252484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 msgr2=0x55e7417257b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f821f7fe700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417257b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f821f7fe700 1 -- 172.21.15.62:0/518252484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822019fef0 con 0x55e741718e20 2023-07-14T16:25:18.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f821f7fe700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741725cf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55e74170dd00 tx=0x7f82140058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd011f8a35b1f5a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f81ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:25:18.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f8224f9c700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741728cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:18.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.533+0000 7f821ffff700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:18.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.534+0000 7f81ff7fe700 1 -- 172.21.15.62:0/518252484 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821401c070 con 0x55e741718e20 2023-07-14T16:25:18.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.534+0000 7f823a30eb80 1 -- 172.21.15.62:0/518252484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f821400fba0 con 0x55e741718e20 2023-07-14T16:25:18.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.534+0000 7f81ff7fe700 1 -- 172.21.15.62:0/518252484 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8214000e40 con 0x55e741718e20 2023-07-14T16:25:18.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.534+0000 7f81ff7fe700 1 -- 172.21.15.62:0/518252484 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82140295a0 con 0x55e741718e20 2023-07-14T16:25:18.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.534+0000 7f823a30eb80 10 client.27721 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:18.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.534+0000 7f823a30eb80 1 -- 172.21.15.62:0/518252484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e7414962b0 con 0x55e741718e20 2023-07-14T16:25:18.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.535+0000 7f81ff7fe700 1 -- 172.21.15.62:0/518252484 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f821400fba0 con 0x55e741718e20 2023-07-14T16:25:18.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.535+0000 7f81ff7fe700 1 -- 172.21.15.62:0/518252484 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8214009920 con 0x55e741718e20 2023-07-14T16:25:18.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.535+0000 7f823a30eb80 10 client.27721 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:18.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.535+0000 7f823a30eb80 10 client.27721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:25:18.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.535+0000 7f823a30eb80 20 client.27721 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:18.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.535+0000 7f823a30eb80 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e74172da60 0x55e74178d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.535+0000 7f823a30eb80 4 client.27721 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:25:18.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.535+0000 7f823a30eb80 1 -- 172.21.15.62:0/518252484 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e741625050 con 0x55e74172da60 2023-07-14T16:25:18.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.536+0000 7f821ffff700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e74172da60 0x55e74178d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.536+0000 7f81ff7fe700 10 client.27721 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:25:18.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.536+0000 7f821ffff700 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e74172da60 0x55e74178d040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ddfa94cfe498e84f server_cookie=b7b380a574baa9f6 in_seq=0 out_seq=0 2023-07-14T16:25:18.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f81ff7fe700 1 -- 172.21.15.62:0/518252484 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e741625050 con 0x55e74172da60 2023-07-14T16:25:18.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f81ff7fe700 10 client.27721 handle_command_reply: tid=0 2023-07-14T16:25:18.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 client.27721 shutdown 2023-07-14T16:25:18.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 -- 172.21.15.62:0/518252484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 msgr2=0x55e741725cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741725cf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55e74170dd00 tx=0x7f82140058e0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 -- 172.21.15.62:0/518252484 shutdown_connections 2023-07-14T16:25:18.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e74172da60 0x55e74178d040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e741719c70 0x55e741728cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7417184d0 0x55e7417257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 --2- 172.21.15.62:0/518252484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e741718e20 0x55e741725cf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f81ff7fe700 0 client.27721 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:25:18.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 -- 172.21.15.62:0/518252484 >> 172.21.15.62:0/518252484 conn(0x55e74170dda0 msgr2=0x55e741729f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 -- 172.21.15.62:0/518252484 shutdown_connections 2023-07-14T16:25:18.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.537+0000 7f823a30eb80 1 -- 172.21.15.62:0/518252484 wait complete. 2023-07-14T16:25:18.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.538+0000 7f823a30eb80 20 client.27721 trim_cache size 0 max 16384 2023-07-14T16:25:18.542 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:25:18.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.538+0000 7f82269ff700 1 -- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82080a11d0 msgr2=0x7f82080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.538+0000 7f82269ff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82080a11d0 0x7f82080a3680 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8218009fd0 tx=0x7f8218009040 comp rx=0 tx=0).stop 2023-07-14T16:25:18.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.538+0000 7f82269ff700 1 -- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8220100560 msgr2=0x7f82201979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.538+0000 7f82269ff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8220100560 0x7f82201979f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f821000ec10 tx=0x7f821000c2d0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.539+0000 7f82269ff700 1 -- 172.21.15.62:0/3017531604 shutdown_connections 2023-07-14T16:25:18.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.539+0000 7f82269ff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82201057a0 0x7f822019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.539+0000 7f82269ff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8220100560 0x7f82201979f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.539+0000 7f82269ff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82080a11d0 0x7f82080a3680 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.539+0000 7f82269ff700 1 --2- 172.21.15.62:0/3017531604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8220102e80 0x7f8220197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.539+0000 7f82269ff700 1 -- 172.21.15.62:0/3017531604 >> 172.21.15.62:0/3017531604 conn(0x7f82200fa000 msgr2=0x7f82201011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.539+0000 7f82269ff700 1 -- 172.21.15.62:0/3017531604 shutdown_connections 2023-07-14T16:25:18.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.539+0000 7f82269ff700 1 -- 172.21.15.62:0/3017531604 wait complete. 2023-07-14T16:25:18.546 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:25:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.837+0000 7f38fe3ff700 1 -- 172.21.15.62:0/2797068299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 msgr2=0x7f38f80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.837+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/2797068299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 0x7f38f80fff10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009990 tx=0x7f38e8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.838+0000 7f38fe3ff700 1 -- 172.21.15.62:0/2797068299 shutdown_connections 2023-07-14T16:25:18.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.838+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/2797068299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f8101680 0x7f38f8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.838+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/2797068299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 0x7f38f80fff10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.838+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/2797068299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f8100cf0 0x7f38f8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.838+0000 7f38fe3ff700 1 -- 172.21.15.62:0/2797068299 >> 172.21.15.62:0/2797068299 conn(0x7f38f80fb120 msgr2=0x7f38f80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.838+0000 7f38fe3ff700 1 -- 172.21.15.62:0/2797068299 shutdown_connections 2023-07-14T16:25:18.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.838+0000 7f38fe3ff700 1 -- 172.21.15.62:0/2797068299 wait complete. 2023-07-14T16:25:18.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.839+0000 7f38fe3ff700 1 Processor -- start 2023-07-14T16:25:18.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.839+0000 7f38fe3ff700 1 -- start start 2023-07-14T16:25:18.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.839+0000 7f38fe3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 0x7f38f80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38fe3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f8100cf0 0x7f38f8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38fe3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f8101680 0x7f38f8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38fe3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80735e0 con 0x7f38f80ffb00 2023-07-14T16:25:18.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38fe3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8073750 con 0x7f38f8101680 2023-07-14T16:25:18.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38fe3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f819f7f0 con 0x7f38f8100cf0 2023-07-14T16:25:18.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 0x7f38f80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 0x7f38f80742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51998/0 (socket says 172.21.15.62:51998) 2023-07-14T16:25:18.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f7fff700 1 -- 172.21.15.62:0/4199941138 learned_addr learned my addr 172.21.15.62:0/4199941138 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f77fe700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f8100cf0 0x7f38f8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38fc99c700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f8101680 0x7f38f8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f7fff700 1 -- 172.21.15.62:0/4199941138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f8100cf0 msgr2=0x7f38f8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f7fff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f8100cf0 0x7f38f8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f7fff700 1 -- 172.21.15.62:0/4199941138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f8101680 msgr2=0x7f38f8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f7fff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f8101680 0x7f38f8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.840+0000 7f38f7fff700 1 -- 172.21.15.62:0/4199941138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80095f0 con 0x7f38f80ffb00 2023-07-14T16:25:18.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.841+0000 7f38f7fff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 0x7f38f80742c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f38e8005910 tx=0x7f38e8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.841+0000 7f38f57fa700 1 -- 172.21.15.62:0/4199941138 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e801d070 con 0x7f38f80ffb00 2023-07-14T16:25:18.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.841+0000 7f38f57fa700 1 -- 172.21.15.62:0/4199941138 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38e800bd50 con 0x7f38f80ffb00 2023-07-14T16:25:18.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.841+0000 7f38fe3ff700 1 -- 172.21.15.62:0/4199941138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f819fa70 con 0x7f38f80ffb00 2023-07-14T16:25:18.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.841+0000 7f38f57fa700 1 -- 172.21.15.62:0/4199941138 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8022d00 con 0x7f38f80ffb00 2023-07-14T16:25:18.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.841+0000 7f38fe3ff700 1 -- 172.21.15.62:0/4199941138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f819fef0 con 0x7f38f80ffb00 2023-07-14T16:25:18.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.846+0000 7f38f57fa700 1 -- 172.21.15.62:0/4199941138 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f38e800b8c0 con 0x7f38f80ffb00 2023-07-14T16:25:18.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.846+0000 7f3911c2bb80 1 Processor -- start 2023-07-14T16:25:18.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.846+0000 7f3911c2bb80 1 -- start start 2023-07-14T16:25:18.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8a4d0 0x562a96d8a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d8b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8bc70 0x562a96d8c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96c8fcb0 con 0x562a96d8a4d0 2023-07-14T16:25:18.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96ce2a40 con 0x562a96d8ae20 2023-07-14T16:25:18.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96cdbed0 con 0x562a96d8bc70 2023-07-14T16:25:18.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f38fc99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8a4d0 0x562a96d8a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f38fc99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8a4d0 0x562a96d8a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52004/0 (socket says 172.21.15.62:52004) 2023-07-14T16:25:18.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f38fc99c700 1 -- 172.21.15.62:0/2568672986 learned_addr learned my addr 172.21.15.62:0/2568672986 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.847+0000 7f38f7fff700 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d8b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38f77fe700 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8bc70 0x562a96d8c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38fc99c700 1 -- 172.21.15.62:0/2568672986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8bc70 msgr2=0x562a96d8c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38fc99c700 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8bc70 0x562a96d8c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38fc99c700 1 -- 172.21.15.62:0/2568672986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 msgr2=0x562a96d8b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38fc99c700 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d8b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38fc99c700 1 -- 172.21.15.62:0/2568672986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a96b09de0 con 0x562a96d8a4d0 2023-07-14T16:25:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38fc99c700 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8a4d0 0x562a96d8a8e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f38f0011600 tx=0x7f38f0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646548495945ef53 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38f57fa700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38e00a11d0 0x7f38e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38de7fc700 1 -- 172.21.15.62:0/2568672986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f000e040 con 0x562a96d8a4d0 2023-07-14T16:25:18.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38de7fc700 1 -- 172.21.15.62:0/2568672986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38f000f040 con 0x562a96d8a4d0 2023-07-14T16:25:18.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38de7fc700 1 -- 172.21.15.62:0/2568672986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0004730 con 0x562a96d8a4d0 2023-07-14T16:25:18.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.848+0000 7f38f7fff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38e00a11d0 0x7f38e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f38f57fa700 1 -- 172.21.15.62:0/4199941138 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f38e80cdd20 con 0x7f38f80ffb00 2023-07-14T16:25:18.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f3911c2bb80 1 -- 172.21.15.62:0/2568672986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8a4d0 msgr2=0x562a96d8a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8a4d0 0x562a96d8a8e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f38f0011600 tx=0x7f38f0009920 comp rx=0 tx=0).stop 2023-07-14T16:25:18.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f38f7fff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38e00a11d0 0x7f38e00a3680 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f38e8097670 tx=0x7f38e80d1720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f3911c2bb80 1 -- 172.21.15.62:0/2568672986 shutdown_connections 2023-07-14T16:25:18.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8bc70 0x562a96d8c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8a4d0 0x562a96d8a8e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/2568672986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d8b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f3911c2bb80 1 -- 172.21.15.62:0/2568672986 >> 172.21.15.62:0/2568672986 conn(0x562a96d7fda0 msgr2=0x562a96d851d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.849+0000 7f3911c2bb80 1 -- 172.21.15.62:0/2568672986 shutdown_connections 2023-07-14T16:25:18.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.850+0000 7f3911c2bb80 1 -- 172.21.15.62:0/2568672986 wait complete. 2023-07-14T16:25:18.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.850+0000 7f3911c2bb80 1 -- start start 2023-07-14T16:25:18.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.850+0000 7f3911c2bb80 1 Processor -- start 2023-07-14T16:25:18.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d94130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d94670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 0x562a96d9b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96c7bac0 con 0x562a96d8ae20 2023-07-14T16:25:18.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96c7b7c0 con 0x562a96d8bc70 2023-07-14T16:25:18.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96b5f920 con 0x562a96d8a4d0 2023-07-14T16:25:18.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f3911c2bb80 10 client.0 _ll_register_callbacks cb 0x562a96c79760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:18.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f38f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d94670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f38f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d94670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52010/0 (socket says 172.21.15.62:52010) 2023-07-14T16:25:18.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f38f77fe700 1 -- 172.21.15.62:0/231383426 learned_addr learned my addr 172.21.15.62:0/231383426 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f38fc99c700 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d94130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.851+0000 7f38fc99c700 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 0x562a96d9b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38f77fe700 1 -- 172.21.15.62:0/231383426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 msgr2=0x562a96d94130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38f77fe700 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d94130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38f77fe700 1 -- 172.21.15.62:0/231383426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 msgr2=0x562a96d9b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38f77fe700 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 0x562a96d9b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38f77fe700 1 -- 172.21.15.62:0/231383426 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f00112b0 con 0x562a96d8ae20 2023-07-14T16:25:18.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38f77fe700 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d94670 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f38ec011800 tx=0x7f38ec011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d7fa40957e12ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:18.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38d77fe700 1 -- 172.21.15.62:0/231383426 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec016070 con 0x562a96d8ae20 2023-07-14T16:25:18.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f3911c2bb80 1 -- 172.21.15.62:0/231383426 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a96c354f0 con 0x562a96d8ae20 2023-07-14T16:25:18.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.852+0000 7f38d77fe700 1 -- 172.21.15.62:0/231383426 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f38ec00f690 con 0x562a96d8ae20 2023-07-14T16:25:18.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f38d77fe700 1 -- 172.21.15.62:0/231383426 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38ec00f850 con 0x562a96d8ae20 2023-07-14T16:25:18.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f3911c2bb80 10 client.18222 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:18.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f3911c2bb80 1 -- 172.21.15.62:0/231383426 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562a96af2c60 con 0x562a96d8ae20 2023-07-14T16:25:18.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f38d77fe700 1 -- 172.21.15.62:0/231383426 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562a96c354f0 con 0x562a96d8ae20 2023-07-14T16:25:18.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f38d77fe700 1 -- 172.21.15.62:0/231383426 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38ec025a70 con 0x562a96d8ae20 2023-07-14T16:25:18.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f3911c2bb80 10 client.18222 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:18.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f3911c2bb80 10 client.18222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:25:18.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f3911c2bb80 20 client.18222 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:18.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562a96d9ffc0 0x562a96da23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f3911c2bb80 4 client.18222 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:18.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.853+0000 7f3911c2bb80 1 -- 172.21.15.62:0/231383426 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a96cd4eb0 con 0x562a96d9ffc0 2023-07-14T16:25:18.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.854+0000 7f38fc99c700 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562a96d9ffc0 0x562a96da23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.854+0000 7f38fc99c700 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562a96d9ffc0 0x562a96da23d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f8d3f96465fbd34 server_cookie=214fe80104d238c2 in_seq=0 out_seq=0 2023-07-14T16:25:18.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.854+0000 7f38d77fe700 10 client.18222 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:25:18.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.855+0000 7f38d77fe700 1 -- 172.21.15.62:0/231383426 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562a96cd4eb0 con 0x562a96d9ffc0 2023-07-14T16:25:18.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.855+0000 7f38d77fe700 10 client.18222 handle_command_reply: tid=0 2023-07-14T16:25:18.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.855+0000 7f3911c2bb80 1 client.18222 shutdown 2023-07-14T16:25:18.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.855+0000 7f3911c2bb80 1 -- 172.21.15.62:0/231383426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 msgr2=0x562a96d94670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.855+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d94670 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f38ec011800 tx=0x7f38ec011b10 comp rx=0 tx=0).stop 2023-07-14T16:25:18.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 1 -- 172.21.15.62:0/231383426 shutdown_connections 2023-07-14T16:25:18.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562a96d9ffc0 0x562a96da23d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 0x562a96d9b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d94130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/231383426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d94670 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 1 -- 172.21.15.62:0/231383426 >> 172.21.15.62:0/231383426 conn(0x562a96d7fda0 msgr2=0x562a96d848b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f38d77fe700 0 client.18222 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:25:18.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 1 -- 172.21.15.62:0/231383426 shutdown_connections 2023-07-14T16:25:18.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 1 -- 172.21.15.62:0/231383426 wait complete. 2023-07-14T16:25:18.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.856+0000 7f3911c2bb80 20 client.18222 trim_cache size 0 max 16384 2023-07-14T16:25:18.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.872+0000 7f3911c2bb80 1 Processor -- start 2023-07-14T16:25:18.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.872+0000 7f3911c2bb80 1 -- start start 2023-07-14T16:25:18.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.872+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d9be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.872+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d9b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.872+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8bc70 0x562a96d920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.872+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96d86070 con 0x562a96d8bc70 2023-07-14T16:25:18.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.872+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96d95080 con 0x562a96d8ae20 2023-07-14T16:25:18.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.872+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96d91e60 con 0x562a96d8a4d0 2023-07-14T16:25:18.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8bc70 0x562a96d920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8bc70 0x562a96d920b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52020/0 (socket says 172.21.15.62:52020) 2023-07-14T16:25:18.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 -- 172.21.15.62:0/1855604915 learned_addr learned my addr 172.21.15.62:0/1855604915 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38f77fe700 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d9b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d9be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 -- 172.21.15.62:0/1855604915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 msgr2=0x562a96d9be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d9be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 -- 172.21.15.62:0/1855604915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 msgr2=0x562a96d9b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d9b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 -- 172.21.15.62:0/1855604915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a96b30250 con 0x562a96d8bc70 2023-07-14T16:25:18.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38f77fe700 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d9b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:18.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.873+0000 7f38fc99c700 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8bc70 0x562a96d920b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f38f0011600 tx=0x7f38f0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbd6bfe9b061d84 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.874+0000 7f38deffd700 1 -- 172.21.15.62:0/1855604915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f000e030 con 0x562a96d8bc70 2023-07-14T16:25:18.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.874+0000 7f38deffd700 1 -- 172.21.15.62:0/1855604915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38f002e5d0 con 0x562a96d8bc70 2023-07-14T16:25:18.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.874+0000 7f38deffd700 1 -- 172.21.15.62:0/1855604915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0003ad0 con 0x562a96d8bc70 2023-07-14T16:25:18.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 -- 172.21.15.62:0/1855604915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8bc70 msgr2=0x562a96d920b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8bc70 0x562a96d920b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f38f0011600 tx=0x7f38f0009920 comp rx=0 tx=0).stop 2023-07-14T16:25:18.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 -- 172.21.15.62:0/1855604915 shutdown_connections 2023-07-14T16:25:18.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8bc70 0x562a96d920b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d9be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/1855604915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8ae20 0x562a96d9b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 -- 172.21.15.62:0/1855604915 >> 172.21.15.62:0/1855604915 conn(0x562a96d7fda0 msgr2=0x562a96d92e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 -- 172.21.15.62:0/1855604915 shutdown_connections 2023-07-14T16:25:18.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 -- 172.21.15.62:0/1855604915 wait complete. 2023-07-14T16:25:18.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.875+0000 7f3911c2bb80 1 -- start start 2023-07-14T16:25:18.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f3911c2bb80 1 Processor -- start 2023-07-14T16:25:18.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d97de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f3911c2bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 0x562a96d9aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96d9af70 con 0x562a96d8ae20 2023-07-14T16:25:18.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96d9b0b0 con 0x562a96d8bc70 2023-07-14T16:25:18.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f3911c2bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a96d9b270 con 0x562a96d8a4d0 2023-07-14T16:25:18.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f3911c2bb80 10 client.0 _ll_register_callbacks cb 0x7f38ec009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:18.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f38f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d97de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.876+0000 7f38f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d97de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52024/0 (socket says 172.21.15.62:52024) 2023-07-14T16:25:18.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38f77fe700 1 -- 172.21.15.62:0/330823298 learned_addr learned my addr 172.21.15.62:0/330823298 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:18.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38fc99c700 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 0x562a96d9aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38fc99c700 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38f77fe700 1 -- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 msgr2=0x562a96d978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38f77fe700 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38f77fe700 1 -- 172.21.15.62:0/330823298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 msgr2=0x562a96d9aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38f77fe700 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 0x562a96d9aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38f77fe700 1 -- 172.21.15.62:0/330823298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a96af2c60 con 0x562a96d8ae20 2023-07-14T16:25:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38fc99c700 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38f77fe700 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d97de0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f38ec025ed0 tx=0x7f38ec019a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30386dadd070730f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:18.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:18.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38d77fe700 1 -- 172.21.15.62:0/330823298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec016070 con 0x562a96d8ae20 2023-07-14T16:25:18.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f3911c2bb80 1 -- 172.21.15.62:0/330823298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38f000f040 con 0x562a96d8ae20 2023-07-14T16:25:18.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.877+0000 7f38d77fe700 1 -- 172.21.15.62:0/330823298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f38ec0194e0 con 0x562a96d8ae20 2023-07-14T16:25:18.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f38d77fe700 1 -- 172.21.15.62:0/330823298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38ec019c70 con 0x562a96d8ae20 2023-07-14T16:25:18.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f3911c2bb80 10 client.18228 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:18.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f3911c2bb80 1 -- 172.21.15.62:0/330823298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f38f0027040 con 0x562a96d8ae20 2023-07-14T16:25:18.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f38d77fe700 1 -- 172.21.15.62:0/330823298 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38f000f040 con 0x562a96d8ae20 2023-07-14T16:25:18.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f38d77fe700 1 -- 172.21.15.62:0/330823298 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38ec00f900 con 0x562a96d8ae20 2023-07-14T16:25:18.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f3911c2bb80 10 client.18228 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:18.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f3911c2bb80 10 client.18228 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:25:18.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f3911c2bb80 20 client.18228 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:18.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562a96df4ee0 0x562a96df72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:18.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f3911c2bb80 4 client.18228 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:25:18.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.878+0000 7f3911c2bb80 1 -- 172.21.15.62:0/330823298 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a96c97050 con 0x562a96df4ee0 2023-07-14T16:25:18.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.879+0000 7f38fc99c700 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562a96df4ee0 0x562a96df72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:18.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.879+0000 7f38fc99c700 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562a96df4ee0 0x562a96df72f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c92fabed0cc5585d server_cookie=5f36bcea0880038d in_seq=0 out_seq=0 2023-07-14T16:25:18.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.879+0000 7f38d77fe700 10 client.18228 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:25:18.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.879+0000 7f38d77fe700 1 -- 172.21.15.62:0/330823298 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a96c97050 con 0x562a96df4ee0 2023-07-14T16:25:18.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.879+0000 7f38d77fe700 10 client.18228 handle_command_reply: tid=0 2023-07-14T16:25:18.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.879+0000 7f3911c2bb80 1 client.18228 shutdown 2023-07-14T16:25:18.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 -- 172.21.15.62:0/330823298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 msgr2=0x562a96d97de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d97de0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f38ec025ed0 tx=0x7f38ec019a00 comp rx=0 tx=0).stop 2023-07-14T16:25:18.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 -- 172.21.15.62:0/330823298 shutdown_connections 2023-07-14T16:25:18.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562a96df4ee0 0x562a96df72f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a96d8bc70 0x562a96d9aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562a96d8a4d0 0x562a96d978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 --2- 172.21.15.62:0/330823298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562a96d8ae20 0x562a96d97de0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f38d77fe700 0 client.18228 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:25:18.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 -- 172.21.15.62:0/330823298 >> 172.21.15.62:0/330823298 conn(0x562a96d7fda0 msgr2=0x562a96d920b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 -- 172.21.15.62:0/330823298 shutdown_connections 2023-07-14T16:25:18.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 1 -- 172.21.15.62:0/330823298 wait complete. 2023-07-14T16:25:18.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.880+0000 7f3911c2bb80 20 client.18228 trim_cache size 0 max 16384 2023-07-14T16:25:18.960 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:25:18.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 -- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38e00a11d0 msgr2=0x7f38e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38e00a11d0 0x7f38e00a3680 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f38e8097670 tx=0x7f38e80d1720 comp rx=0 tx=0).stop 2023-07-14T16:25:18.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 -- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 msgr2=0x7f38f80742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:18.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 0x7f38f80742c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f38e8005910 tx=0x7f38e8005740 comp rx=0 tx=0).stop 2023-07-14T16:25:18.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 -- 172.21.15.62:0/4199941138 shutdown_connections 2023-07-14T16:25:18.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38e00a11d0 0x7f38e00a3680 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f8101680 0x7f38f8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f80ffb00 0x7f38f80742c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 --2- 172.21.15.62:0/4199941138 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f8100cf0 0x7f38f8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:18.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 -- 172.21.15.62:0/4199941138 >> 172.21.15.62:0/4199941138 conn(0x7f38f80fb120 msgr2=0x7f38f80fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:18.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 -- 172.21.15.62:0/4199941138 shutdown_connections 2023-07-14T16:25:18.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:18.881+0000 7f38fe3ff700 1 -- 172.21.15.62:0/4199941138 wait complete. 2023-07-14T16:25:18.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:18 smithi062 ceph-mon[95543]: pgmap v647: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 MiB/s wr, 13 op/s 2023-07-14T16:25:18.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:18 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1455385041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:25:18.964 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:25:19.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:18 smithi164 ceph-mon[97427]: pgmap v647: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 MiB/s wr, 13 op/s 2023-07-14T16:25:19.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:18 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1455385041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:25:19.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:18 smithi081 ceph-mon[127795]: pgmap v647: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 MiB/s wr, 13 op/s 2023-07-14T16:25:19.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:18 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1455385041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:25:19.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 -- 172.21.15.62:0/3486345058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 msgr2=0x7f9e24107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/3486345058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 0x7f9e24107ed0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00b0a0 tx=0x7f9e1c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 -- 172.21.15.62:0/3486345058 shutdown_connections 2023-07-14T16:25:19.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/3486345058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 0x7f9e24107ed0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/3486345058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e240ff9f0 0x7f9e240ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/3486345058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e24100c80 0x7f9e241010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 -- 172.21.15.62:0/3486345058 >> 172.21.15.62:0/3486345058 conn(0x7f9e240fb010 msgr2=0x7f9e240fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:19.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 -- 172.21.15.62:0/3486345058 shutdown_connections 2023-07-14T16:25:19.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.222+0000 7f9e28f9d700 1 -- 172.21.15.62:0/3486345058 wait complete. 2023-07-14T16:25:19.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e28f9d700 1 Processor -- start 2023-07-14T16:25:19.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e28f9d700 1 -- start start 2023-07-14T16:25:19.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e28f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e240ff9f0 0x7f9e241977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e28f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e24100c80 0x7f9e24197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e28f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 0x7f9e2419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e28f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2419f320 con 0x7f9e24101630 2023-07-14T16:25:19.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e28f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2419f460 con 0x7f9e240ff9f0 2023-07-14T16:25:19.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e28f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2419f5f0 con 0x7f9e24100c80 2023-07-14T16:25:19.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 0x7f9e2419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.224+0000 7f9e23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 0x7f9e2419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52044/0 (socket says 172.21.15.62:52044) 2023-07-14T16:25:19.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e23fff700 1 -- 172.21.15.62:0/1741030533 learned_addr learned my addr 172.21.15.62:0/1741030533 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:19.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e24100c80 0x7f9e24197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e237fe700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e240ff9f0 0x7f9e241977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e23fff700 1 -- 172.21.15.62:0/1741030533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e24100c80 msgr2=0x7f9e24197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e23fff700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e24100c80 0x7f9e24197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e23fff700 1 -- 172.21.15.62:0/1741030533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e240ff9f0 msgr2=0x7f9e241977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e23fff700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e240ff9f0 0x7f9e241977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e23fff700 1 -- 172.21.15.62:0/1741030533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c009d00 con 0x7f9e24101630 2023-07-14T16:25:19.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e24100c80 0x7f9e24197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:19.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e23fff700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 0x7f9e2419ed50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00d010 tx=0x7f9e1c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:19.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.225+0000 7f9e20ff9700 1 -- 172.21.15.62:0/1741030533 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00e040 con 0x7f9e24101630 2023-07-14T16:25:19.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.226+0000 7f9e28f9d700 1 -- 172.21.15.62:0/1741030533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2419f870 con 0x7f9e24101630 2023-07-14T16:25:19.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.226+0000 7f9e20ff9700 1 -- 172.21.15.62:0/1741030533 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e1c003e10 con 0x7f9e24101630 2023-07-14T16:25:19.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.226+0000 7f9e20ff9700 1 -- 172.21.15.62:0/1741030533 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0053c0 con 0x7f9e24101630 2023-07-14T16:25:19.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.226+0000 7f9e28f9d700 1 -- 172.21.15.62:0/1741030533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2419fcf0 con 0x7f9e24101630 2023-07-14T16:25:19.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.229+0000 7f9e20ff9700 1 -- 172.21.15.62:0/1741030533 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9e1c01db10 con 0x7f9e24101630 2023-07-14T16:25:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.231+0000 7f9e20ff9700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e0c0a11f0 0x7f9e0c0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.231+0000 7f9e3da59b80 1 Processor -- start 2023-07-14T16:25:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e20ff9700 1 -- 172.21.15.62:0/1741030533 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e1c0cdbc0 con 0x7f9e24101630 2023-07-14T16:25:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e3da59b80 1 -- start start 2023-07-14T16:25:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e237fe700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e0c0a11f0 0x7f9e0c0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 0x557e53f19b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f1a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53e1ecb0 con 0x557e53f1a070 2023-07-14T16:25:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53e71a40 con 0x557e53f1ae30 2023-07-14T16:25:19.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.232+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53e6aed0 con 0x557e53f19720 2023-07-14T16:25:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f1a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 0x557e53f19b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f1a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52058/0 (socket says 172.21.15.62:52058) 2023-07-14T16:25:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 -- 172.21.15.62:0/277706285 learned_addr learned my addr 172.21.15.62:0/277706285 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:19.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e23fff700 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e237fe700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e0c0a11f0 0x7f9e0c0a36a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400bc20 tx=0x7f9e1400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:19.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 -- 172.21.15.62:0/277706285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 msgr2=0x557e53f19b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:19.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 0x557e53f19b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 -- 172.21.15.62:0/277706285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 msgr2=0x557e53f1b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:19.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 -- 172.21.15.62:0/277706285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e53c98de0 con 0x557e53f1a070 2023-07-14T16:25:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e23fff700 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.233+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f1a8a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009dc0 tx=0x7f9e180106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94cad25ccc74a3f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.234+0000 7f9e09ffb700 1 -- 172.21.15.62:0/277706285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18010e10 con 0x557e53f1a070 2023-07-14T16:25:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.234+0000 7f9e09ffb700 1 -- 172.21.15.62:0/277706285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e18015070 con 0x557e53f1a070 2023-07-14T16:25:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.234+0000 7f9e09ffb700 1 -- 172.21.15.62:0/277706285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1801a5e0 con 0x557e53f1a070 2023-07-14T16:25:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 -- 172.21.15.62:0/277706285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 msgr2=0x557e53f1a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f1a8a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009dc0 tx=0x7f9e180106b0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 -- 172.21.15.62:0/277706285 shutdown_connections 2023-07-14T16:25:19.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 0x557e53f19b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/277706285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f1a8a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 -- 172.21.15.62:0/277706285 >> 172.21.15.62:0/277706285 conn(0x557e53f0eff0 msgr2=0x557e53f14460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:19.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 -- 172.21.15.62:0/277706285 shutdown_connections 2023-07-14T16:25:19.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 -- 172.21.15.62:0/277706285 wait complete. 2023-07-14T16:25:19.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.235+0000 7f9e3da59b80 1 -- start start 2023-07-14T16:25:19.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.236+0000 7f9e3da59b80 1 Processor -- start 2023-07-14T16:25:19.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.236+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f19720 0x557e53f23330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.236+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 0x557e53f23870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.236+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1ae30 0x557e53f2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.236+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53e0aac0 con 0x557e53f1ae30 2023-07-14T16:25:19.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.236+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53e0a7c0 con 0x557e53f19720 2023-07-14T16:25:19.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.236+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53cee920 con 0x557e53f1a070 2023-07-14T16:25:19.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.236+0000 7f9e3da59b80 10 client.0 _ll_register_callbacks cb 0x557e53e08760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:19.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1ae30 0x557e53f2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 0x557e53f23870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1ae30 0x557e53f2a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52066/0 (socket says 172.21.15.62:52066) 2023-07-14T16:25:19.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 -- 172.21.15.62:0/327626169 learned_addr learned my addr 172.21.15.62:0/327626169 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:19.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e237fe700 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f19720 0x557e53f23330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 -- 172.21.15.62:0/327626169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 msgr2=0x557e53f23870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 0x557e53f23870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 -- 172.21.15.62:0/327626169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f19720 msgr2=0x557e53f23330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f19720 0x557e53f23330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 -- 172.21.15.62:0/327626169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e180099f0 con 0x557e53f1ae30 2023-07-14T16:25:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.237+0000 7f9e23fff700 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1ae30 0x557e53f2a8b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c09afa0 tx=0x7f9e1c09a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241f20835ea31737 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e02ffd700 1 -- 172.21.15.62:0/327626169 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c09a700 con 0x557e53f1ae30 2023-07-14T16:25:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e02ffd700 1 -- 172.21.15.62:0/327626169 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e1c09f040 con 0x557e53f1ae30 2023-07-14T16:25:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e3da59b80 1 -- 172.21.15.62:0/327626169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e53dc44f0 con 0x557e53f1ae30 2023-07-14T16:25:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e02ffd700 1 -- 172.21.15.62:0/327626169 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e1c0a33d0 con 0x557e53f1ae30 2023-07-14T16:25:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e3da59b80 10 client.18240 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e3da59b80 1 -- 172.21.15.62:0/327626169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557e53c81c60 con 0x557e53f1ae30 2023-07-14T16:25:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e02ffd700 1 -- 172.21.15.62:0/327626169 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e1c0c9080 con 0x557e53f1ae30 2023-07-14T16:25:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e02ffd700 1 -- 172.21.15.62:0/327626169 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9e1c096e00 con 0x557e53f1ae30 2023-07-14T16:25:19.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.238+0000 7f9e3da59b80 10 client.18240 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:19.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.239+0000 7f9e3da59b80 10 client.18240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:19.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.239+0000 7f9e3da59b80 20 client.18240 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:19.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.239+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e53f2edd0 0x557e53f311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.239+0000 7f9e3da59b80 4 client.18240 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:19.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.239+0000 7f9e3da59b80 1 -- 172.21.15.62:0/327626169 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e53e63eb0 con 0x557e53f2edd0 2023-07-14T16:25:19.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.239+0000 7f9e237fe700 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e53f2edd0 0x557e53f311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.240+0000 7f9e237fe700 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e53f2edd0 0x557e53f311e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c962de79d12710 server_cookie=16ac63b8c4e02ce in_seq=0 out_seq=0 2023-07-14T16:25:19.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.240+0000 7f9e02ffd700 10 client.18240 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:19.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.240+0000 7f9e02ffd700 1 -- 172.21.15.62:0/327626169 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557e53e63eb0 con 0x557e53f2edd0 2023-07-14T16:25:19.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.240+0000 7f9e02ffd700 10 client.18240 handle_command_reply: tid=0 2023-07-14T16:25:19.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.240+0000 7f9e3da59b80 1 client.18240 shutdown 2023-07-14T16:25:19.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 -- 172.21.15.62:0/327626169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1ae30 msgr2=0x557e53f2a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1ae30 0x557e53f2a8b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c09afa0 tx=0x7f9e1c09a970 comp rx=0 tx=0).stop 2023-07-14T16:25:19.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 -- 172.21.15.62:0/327626169 shutdown_connections 2023-07-14T16:25:19.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1ae30 0x557e53f2a8b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e53f2edd0 0x557e53f311e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f19720 0x557e53f23330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/327626169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 0x557e53f23870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 -- 172.21.15.62:0/327626169 >> 172.21.15.62:0/327626169 conn(0x557e53f0eff0 msgr2=0x557e53f13b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:19.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e02ffd700 0 client.18240 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:19.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 -- 172.21.15.62:0/327626169 shutdown_connections 2023-07-14T16:25:19.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 1 -- 172.21.15.62:0/327626169 wait complete. 2023-07-14T16:25:19.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.241+0000 7f9e3da59b80 20 client.18240 trim_cache size 0 max 16384 2023-07-14T16:25:19.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.262+0000 7f9e3da59b80 1 Processor -- start 2023-07-14T16:25:19.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.262+0000 7f9e3da59b80 1 -- start start 2023-07-14T16:25:19.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.262+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f19720 0x557e53f21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.262+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 0x557e53f2a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.262+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.262+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53f22ea0 con 0x557e53f19720 2023-07-14T16:25:19.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.262+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53f152b0 con 0x557e53f1ae30 2023-07-14T16:25:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.262+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53f0c140 con 0x557e53f1a070 2023-07-14T16:25:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f19720 0x557e53f21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f19720 0x557e53f21260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52080/0 (socket says 172.21.15.62:52080) 2023-07-14T16:25:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 -- 172.21.15.62:0/496421106 learned_addr learned my addr 172.21.15.62:0/496421106 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e23fff700 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 0x557e53f2a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 -- 172.21.15.62:0/496421106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 msgr2=0x557e53f2a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 0x557e53f2a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 -- 172.21.15.62:0/496421106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 msgr2=0x557e53f1dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:19.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 -- 172.21.15.62:0/496421106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e53cbf250 con 0x557e53f19720 2023-07-14T16:25:19.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.263+0000 7f9e237fe700 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f19720 0x557e53f21260 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x557e53f31af0 tx=0x7f9e1401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f261060bb6429b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:19.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.264+0000 7f9e0a7fc700 1 -- 172.21.15.62:0/496421106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14023040 con 0x557e53f19720 2023-07-14T16:25:19.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.264+0000 7f9e0a7fc700 1 -- 172.21.15.62:0/496421106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e14028460 con 0x557e53f19720 2023-07-14T16:25:19.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.264+0000 7f9e0a7fc700 1 -- 172.21.15.62:0/496421106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14017040 con 0x557e53f19720 2023-07-14T16:25:19.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.264+0000 7f9e3da59b80 1 -- 172.21.15.62:0/496421106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f19720 msgr2=0x557e53f21260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.264+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f19720 0x557e53f21260 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x557e53f31af0 tx=0x7f9e1401a000 comp rx=0 tx=0).stop 2023-07-14T16:25:19.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 -- 172.21.15.62:0/496421106 shutdown_connections 2023-07-14T16:25:19.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f1dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f19720 0x557e53f21260 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/496421106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f1a070 0x557e53f2a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 -- 172.21.15.62:0/496421106 >> 172.21.15.62:0/496421106 conn(0x557e53f0eff0 msgr2=0x557e53f22010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:19.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 -- 172.21.15.62:0/496421106 shutdown_connections 2023-07-14T16:25:19.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 -- 172.21.15.62:0/496421106 wait complete. 2023-07-14T16:25:19.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 -- start start 2023-07-14T16:25:19.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.265+0000 7f9e3da59b80 1 Processor -- start 2023-07-14T16:25:19.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 0x557e53f26ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f26ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e3da59b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f2a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53f2a550 con 0x557e53f1a070 2023-07-14T16:25:19.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53f2a6c0 con 0x557e53f1ae30 2023-07-14T16:25:19.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e3da59b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53f2a880 con 0x557e53f19720 2023-07-14T16:25:19.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e3da59b80 10 client.0 _ll_register_callbacks cb 0x7f9e1c09ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:19.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f26ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f26ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52096/0 (socket says 172.21.15.62:52096) 2023-07-14T16:25:19.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.266+0000 7f9e22ffd700 1 -- 172.21.15.62:0/354485625 learned_addr learned my addr 172.21.15.62:0/354485625 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:19.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e237fe700 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 0x557e53f26ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e23fff700 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f2a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e22ffd700 1 -- 172.21.15.62:0/354485625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 msgr2=0x557e53f26ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:19.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 0x557e53f26ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e22ffd700 1 -- 172.21.15.62:0/354485625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 msgr2=0x557e53f2a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:19.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f2a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e22ffd700 1 -- 172.21.15.62:0/354485625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1400f9c0 con 0x557e53f1a070 2023-07-14T16:25:19.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e22ffd700 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f26ff0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009d90 tx=0x7f9e18005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99985273588ce6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:19.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.267+0000 7f9e02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:19.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e02ffd700 1 -- 172.21.15.62:0/354485625 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18022070 con 0x557e53f1a070 2023-07-14T16:25:19.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 1 -- 172.21.15.62:0/354485625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e18015070 con 0x557e53f1a070 2023-07-14T16:25:19.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e02ffd700 1 -- 172.21.15.62:0/354485625 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e18017070 con 0x557e53f1a070 2023-07-14T16:25:19.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e02ffd700 1 -- 172.21.15.62:0/354485625 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e18005570 con 0x557e53f1a070 2023-07-14T16:25:19.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 10 client.18246 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:19.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 1 -- 172.21.15.62:0/354485625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557e53c972b0 con 0x557e53f1a070 2023-07-14T16:25:19.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e02ffd700 1 -- 172.21.15.62:0/354485625 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e18015070 con 0x557e53f1a070 2023-07-14T16:25:19.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e02ffd700 1 -- 172.21.15.62:0/354485625 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9e18011560 con 0x557e53f1a070 2023-07-14T16:25:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 10 client.18246 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 10 client.18246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 20 client.18246 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e53f8c760 0x557e53f8eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 4 client.18246 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:25:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.268+0000 7f9e3da59b80 1 -- 172.21.15.62:0/354485625 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557e53e26050 con 0x557e53f8c760 2023-07-14T16:25:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.269+0000 7f9e237fe700 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e53f8c760 0x557e53f8eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.269+0000 7f9e237fe700 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e53f8c760 0x557e53f8eb70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fddab6924ccf886 server_cookie=8d41541a42edf9f3 in_seq=0 out_seq=0 2023-07-14T16:25:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.269+0000 7f9e02ffd700 10 client.18246 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e02ffd700 1 -- 172.21.15.62:0/354485625 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557e53e26050 con 0x557e53f8c760 2023-07-14T16:25:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e02ffd700 10 client.18246 handle_command_reply: tid=0 2023-07-14T16:25:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 client.18246 shutdown 2023-07-14T16:25:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 -- 172.21.15.62:0/354485625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 msgr2=0x557e53f26ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f26ff0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009d90 tx=0x7f9e18005270 comp rx=0 tx=0).stop 2023-07-14T16:25:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 -- 172.21.15.62:0/354485625 shutdown_connections 2023-07-14T16:25:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557e53f8c760 0x557e53f8eb70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557e53f1ae30 0x557e53f2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557e53f19720 0x557e53f26ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 --2- 172.21.15.62:0/354485625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557e53f1a070 0x557e53f26ff0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e02ffd700 0 client.18246 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 -- 172.21.15.62:0/354485625 >> 172.21.15.62:0/354485625 conn(0x557e53f0eff0 msgr2=0x557e53f21260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 -- 172.21.15.62:0/354485625 shutdown_connections 2023-07-14T16:25:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 1 -- 172.21.15.62:0/354485625 wait complete. 2023-07-14T16:25:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.270+0000 7f9e3da59b80 20 client.18246 trim_cache size 0 max 16384 2023-07-14T16:25:19.256 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:25:19.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 -- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e0c0a11f0 msgr2=0x7f9e0c0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e0c0a11f0 0x7f9e0c0a36a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400bc20 tx=0x7f9e1400b5b0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 -- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 msgr2=0x7f9e2419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:19.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 0x7f9e2419ed50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00d010 tx=0x7f9e1c00bd70 comp rx=0 tx=0).stop 2023-07-14T16:25:19.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 -- 172.21.15.62:0/1741030533 shutdown_connections 2023-07-14T16:25:19.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e24101630 0x7f9e2419ed50 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e0c0a11f0 0x7f9e0c0a36a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e240ff9f0 0x7f9e241977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 --2- 172.21.15.62:0/1741030533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e24100c80 0x7f9e24197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 -- 172.21.15.62:0/1741030533 >> 172.21.15.62:0/1741030533 conn(0x7f9e240fb010 msgr2=0x7f9e240fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 -- 172.21.15.62:0/1741030533 shutdown_connections 2023-07-14T16:25:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:19.271+0000 7f9e28f9d700 1 -- 172.21.15.62:0/1741030533 wait complete. 2023-07-14T16:25:20.267 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:25:20.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:25:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.520+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/2002476341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94103890 msgr2=0x7f8c9410a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.520+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/2002476341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94103890 0x7f8c9410a150 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00b0a0 tx=0x7f8c8c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.520+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/2002476341 shutdown_connections 2023-07-14T16:25:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.520+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/2002476341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94103890 0x7f8c9410a150 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.520+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/2002476341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c94101d10 0x7f8c94102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.520+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/2002476341 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c94102f00 0x7f8c94103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.520+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/2002476341 >> 172.21.15.62:0/2002476341 conn(0x7f8c940fd350 msgr2=0x7f8c940ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.521+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/2002476341 shutdown_connections 2023-07-14T16:25:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.521+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/2002476341 wait complete. 2023-07-14T16:25:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.522+0000 7f8c9c3ff700 1 Processor -- start 2023-07-14T16:25:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.522+0000 7f8c9c3ff700 1 -- start start 2023-07-14T16:25:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.522+0000 7f8c9c3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94101d10 0x7f8c94197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.522+0000 7f8c9c3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c94102f00 0x7f8c94197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.522+0000 7f8c9c3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c94103890 0x7f8c9419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9c3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9419f5c0 con 0x7f8c94101d10 2023-07-14T16:25:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9c3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9419f700 con 0x7f8c94102f00 2023-07-14T16:25:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9c3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9419f890 con 0x7f8c94103890 2023-07-14T16:25:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9a19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94101d10 0x7f8c94197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9999a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c94102f00 0x7f8c94197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9999a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c94102f00 0x7f8c94197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46738/0 (socket says 172.21.15.62:46738) 2023-07-14T16:25:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9999a700 1 -- 172.21.15.62:0/3258973259 learned_addr learned my addr 172.21.15.62:0/3258973259 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c94103890 0x7f8c9419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9a19b700 1 -- 172.21.15.62:0/3258973259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c94103890 msgr2=0x7f8c9419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c94103890 0x7f8c9419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9a19b700 1 -- 172.21.15.62:0/3258973259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c94102f00 msgr2=0x7f8c94197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c94102f00 0x7f8c94197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9a19b700 1 -- 172.21.15.62:0/3258973259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c009d00 con 0x7f8c94101d10 2023-07-14T16:25:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.523+0000 7f8c9999a700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c94102f00 0x7f8c94197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.524+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94101d10 0x7f8c94197a40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000e9c0 tx=0x7f8c9000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.524+0000 7f8c8b7fe700 1 -- 172.21.15.62:0/3258973259 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90004220 con 0x7f8c94101d10 2023-07-14T16:25:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.524+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/3258973259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9419fb70 con 0x7f8c94101d10 2023-07-14T16:25:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.524+0000 7f8c8b7fe700 1 -- 172.21.15.62:0/3258973259 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c90004380 con 0x7f8c94101d10 2023-07-14T16:25:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.524+0000 7f8c8b7fe700 1 -- 172.21.15.62:0/3258973259 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90003b40 con 0x7f8c94101d10 2023-07-14T16:25:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.524+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/3258973259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c941a00d0 con 0x7f8c94101d10 2023-07-14T16:25:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.528+0000 7f8c8b7fe700 1 -- 172.21.15.62:0/3258973259 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8c90003ca0 con 0x7f8c94101d10 2023-07-14T16:25:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.529+0000 7f8cafd43b80 1 Processor -- start 2023-07-14T16:25:20.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.529+0000 7f8cafd43b80 1 -- start start 2023-07-14T16:25:20.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ebb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ebc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04dc0d30 con 0x561b04ebbea0 2023-07-14T16:25:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04e13ac0 con 0x561b04ebb550 2023-07-14T16:25:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04e0cf50 con 0x561b04ebccf0 2023-07-14T16:25:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8c9a19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ebc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8c8b7fe700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8c800a11d0 0x7f8c800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8c9a19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ebc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52132/0 (socket says 172.21.15.62:52132) 2023-07-14T16:25:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8c9a19b700 1 -- 172.21.15.62:0/3211818507 learned_addr learned my addr 172.21.15.62:0/3211818507 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.530+0000 7f8c9999a700 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ebb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c8b7fe700 1 -- 172.21.15.62:0/3258973259 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8c90014070 con 0x7f8c94101d10 2023-07-14T16:25:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c9999a700 1 -- 172.21.15.62:0/3211818507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 msgr2=0x561b04ebb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c9999a700 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ebb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c9999a700 1 -- 172.21.15.62:0/3211818507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 msgr2=0x561b04ebc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c9999a700 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ebc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c9999a700 1 -- 172.21.15.62:0/3211818507 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b04c3ae60 con 0x561b04ebccf0 2023-07-14T16:25:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8c800a11d0 0x7f8c800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c9999a700 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebd160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c84011570 tx=0x7f8c84011880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d32e8adddd425b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.531+0000 7f8c88bfe700 1 -- 172.21.15.62:0/3211818507 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84004230 con 0x561b04ebccf0 2023-07-14T16:25:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.532+0000 7f8c88bfe700 1 -- 172.21.15.62:0/3211818507 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c84004390 con 0x561b04ebccf0 2023-07-14T16:25:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.532+0000 7f8c88bfe700 1 -- 172.21.15.62:0/3211818507 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840197c0 con 0x561b04ebccf0 2023-07-14T16:25:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.532+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3211818507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 msgr2=0x561b04ebd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.532+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebd160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c84011570 tx=0x7f8c84011880 comp rx=0 tx=0).stop 2023-07-14T16:25:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.532+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8c800a11d0 0x7f8c800a3680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009cd0 tx=0x7f8c8c00b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.532+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3211818507 shutdown_connections 2023-07-14T16:25:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.533+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebd160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.533+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ebb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.533+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/3211818507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ebc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.533+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3211818507 >> 172.21.15.62:0/3211818507 conn(0x561b04eb0e20 msgr2=0x561b04eb6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.533+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3211818507 shutdown_connections 2023-07-14T16:25:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.533+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3211818507 wait complete. 2023-07-14T16:25:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.533+0000 7f8cafd43b80 1 -- start start 2023-07-14T16:25:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.533+0000 7f8cafd43b80 1 Processor -- start 2023-07-14T16:25:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebb550 0x561b04ec51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebbea0 0x561b04ec56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04dacb40 con 0x561b04ebb550 2023-07-14T16:25:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04dac840 con 0x561b04ebbea0 2023-07-14T16:25:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04c909a0 con 0x561b04ebccf0 2023-07-14T16:25:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8cafd43b80 10 client.0 _ll_register_callbacks cb 0x561b04daa7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8c9a19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebb550 0x561b04ec51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8c9a19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebb550 0x561b04ec51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52136/0 (socket says 172.21.15.62:52136) 2023-07-14T16:25:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8c9a19b700 1 -- 172.21.15.62:0/4248009163 learned_addr learned my addr 172.21.15.62:0/4248009163 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.534+0000 7f8c9999a700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebbea0 0x561b04ec56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c9a99c700 1 -- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebbea0 msgr2=0x561b04ec56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebbea0 0x561b04ec56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c9a99c700 1 -- 172.21.15.62:0/4248009163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebb550 msgr2=0x561b04ec51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebb550 0x561b04ec51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c9a99c700 1 -- 172.21.15.62:0/4248009163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c84009d60 con 0x561b04ebccf0 2023-07-14T16:25:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebb550 0x561b04ec51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c9999a700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebbea0 0x561b04ec56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecc720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c007950 tx=0x7f8c8c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39c9a7f4e5750e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.535+0000 7f8c7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:25:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.536+0000 7f8c7dffb700 1 -- 172.21.15.62:0/4248009163 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00e040 con 0x561b04ebccf0 2023-07-14T16:25:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.536+0000 7f8cafd43b80 1 -- 172.21.15.62:0/4248009163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b04d66570 con 0x561b04ebccf0 2023-07-14T16:25:20.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.536+0000 7f8c7dffb700 1 -- 172.21.15.62:0/4248009163 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8c8c02b470 con 0x561b04ebccf0 2023-07-14T16:25:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.536+0000 7f8c7dffb700 1 -- 172.21.15.62:0/4248009163 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c8c02b630 con 0x561b04ebccf0 2023-07-14T16:25:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.536+0000 7f8cafd43b80 10 client.27766 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.536+0000 7f8cafd43b80 1 -- 172.21.15.62:0/4248009163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561b04c23ce0 con 0x561b04ebccf0 2023-07-14T16:25:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8c7dffb700 1 -- 172.21.15.62:0/4248009163 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561b04d66570 con 0x561b04ebccf0 2023-07-14T16:25:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8c7dffb700 1 -- 172.21.15.62:0/4248009163 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8c8c0206b0 con 0x561b04ebccf0 2023-07-14T16:25:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8cafd43b80 10 client.27766 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8cafd43b80 10 client.27766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8cafd43b80 20 client.27766 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561b04ed0f10 0x561b04ed3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8cafd43b80 4 client.27766 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8cafd43b80 1 -- 172.21.15.62:0/4248009163 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b04e05f30 con 0x561b04ed0f10 2023-07-14T16:25:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.537+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561b04ed0f10 0x561b04ed3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.538+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561b04ed0f10 0x561b04ed3320 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3e8255dac46198 server_cookie=3cb6503c57c2e890 in_seq=0 out_seq=0 2023-07-14T16:25:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.538+0000 7f8c7dffb700 10 client.27766 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8c7dffb700 1 -- 172.21.15.62:0/4248009163 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561b04e05f30 con 0x561b04ed0f10 2023-07-14T16:25:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8c7dffb700 10 client.27766 handle_command_reply: tid=0 2023-07-14T16:25:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 client.27766 shutdown 2023-07-14T16:25:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 -- 172.21.15.62:0/4248009163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 msgr2=0x561b04ecc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecc720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c007950 tx=0x7f8c8c0099e0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 -- 172.21.15.62:0/4248009163 shutdown_connections 2023-07-14T16:25:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecc720 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebb550 0x561b04ec51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561b04ed0f10 0x561b04ed3320 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/4248009163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebbea0 0x561b04ec56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 -- 172.21.15.62:0/4248009163 >> 172.21.15.62:0/4248009163 conn(0x561b04eb0e20 msgr2=0x561b04eb5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8c7dffb700 0 client.27766 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:20.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.539+0000 7f8cafd43b80 1 -- 172.21.15.62:0/4248009163 shutdown_connections 2023-07-14T16:25:20.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.540+0000 7f8cafd43b80 1 -- 172.21.15.62:0/4248009163 wait complete. 2023-07-14T16:25:20.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.540+0000 7f8cafd43b80 20 client.27766 trim_cache size 0 max 16384 2023-07-14T16:25:20.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8cafd43b80 1 Processor -- start 2023-07-14T16:25:20.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8cafd43b80 1 -- start start 2023-07-14T16:25:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ecc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04eb09f0 con 0x561b04ebbea0 2023-07-14T16:25:20.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04ed0c60 con 0x561b04ebb550 2023-07-14T16:25:20.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04ec3c00 con 0x561b04ebccf0 2023-07-14T16:25:20.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8c9999a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ecc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.566+0000 7f8c9a19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46752/0 (socket says 172.21.15.62:46752) 2023-07-14T16:25:20.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a19b700 1 -- 172.21.15.62:0/3596583175 learned_addr learned my addr 172.21.15.62:0/3596583175 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/3596583175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a19b700 1 -- 172.21.15.62:0/3596583175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 msgr2=0x561b04ebfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/3596583175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a19b700 1 -- 172.21.15.62:0/3596583175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 msgr2=0x561b04ecc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/3596583175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ecc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a19b700 1 -- 172.21.15.62:0/3596583175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b04c612d0 con 0x561b04ebb550 2023-07-14T16:25:20.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/3596583175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec3050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c900968f0 tx=0x7f8c90095b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77fe107031dc1559 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.567+0000 7f8c893ff700 1 -- 172.21.15.62:0/3596583175 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900ad040 con 0x561b04ebb550 2023-07-14T16:25:20.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8c893ff700 1 -- 172.21.15.62:0/3596583175 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c900044f0 con 0x561b04ebb550 2023-07-14T16:25:20.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8c893ff700 1 -- 172.21.15.62:0/3596583175 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900a2360 con 0x561b04ebb550 2023-07-14T16:25:20.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3596583175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 msgr2=0x561b04ec3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/3596583175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec3050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c900968f0 tx=0x7f8c90095b40 comp rx=0 tx=0).stop 2023-07-14T16:25:20.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3596583175 shutdown_connections 2023-07-14T16:25:20.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/3596583175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ebfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/3596583175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec3050 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/3596583175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04ecc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3596583175 >> 172.21.15.62:0/3596583175 conn(0x561b04eb0e20 msgr2=0x561b04ecd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.568+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3596583175 shutdown_connections 2023-07-14T16:25:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.569+0000 7f8cafd43b80 1 -- 172.21.15.62:0/3596583175 wait complete. 2023-07-14T16:25:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.569+0000 7f8cafd43b80 1 -- start start 2023-07-14T16:25:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.569+0000 7f8cafd43b80 1 Processor -- start 2023-07-14T16:25:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04eca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8cafd43b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04ecb720 con 0x561b04ebbea0 2023-07-14T16:25:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04ecb890 con 0x561b04ebb550 2023-07-14T16:25:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8cafd43b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b04ecba50 con 0x561b04ebccf0 2023-07-14T16:25:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8cafd43b80 10 client.0 _ll_register_callbacks cb 0x7f8c8c007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9999a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04eca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9999a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04eca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52160/0 (socket says 172.21.15.62:52160) 2023-07-14T16:25:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9a19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9999a700 1 -- 172.21.15.62:0/1411810340 learned_addr learned my addr 172.21.15.62:0/1411810340 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9a19b700 1 -- 172.21.15.62:0/1411810340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 msgr2=0x561b04ecb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9a19b700 1 -- 172.21.15.62:0/1411810340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 msgr2=0x561b04eca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04eca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9a19b700 1 -- 172.21.15.62:0/1411810340 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c941a00d0 con 0x561b04ebb550 2023-07-14T16:25:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9a99c700 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.570+0000 7f8c9999a700 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04eca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:20.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.571+0000 7f8c9a19b700 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec9c90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009230 tx=0x7f8c90099af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be80b36a6d062ce2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.571+0000 7f8c7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:25:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.571+0000 7f8c7dffb700 1 -- 172.21.15.62:0/1411810340 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900b82c0 con 0x561b04ebb550 2023-07-14T16:25:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.571+0000 7f8c7dffb700 1 -- 172.21.15.62:0/1411810340 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8c90016330 con 0x561b04ebb550 2023-07-14T16:25:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.571+0000 7f8cafd43b80 1 -- 172.21.15.62:0/1411810340 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c84004390 con 0x561b04ebb550 2023-07-14T16:25:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.571+0000 7f8c7dffb700 1 -- 172.21.15.62:0/1411810340 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c900ca1e0 con 0x561b04ebb550 2023-07-14T16:25:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.571+0000 7f8cafd43b80 10 client.27884 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.571+0000 7f8cafd43b80 1 -- 172.21.15.62:0/1411810340 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561b04c39330 con 0x561b04ebb550 2023-07-14T16:25:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8c7dffb700 1 -- 172.21.15.62:0/1411810340 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c900bd080 con 0x561b04ebb550 2023-07-14T16:25:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8c7dffb700 1 -- 172.21.15.62:0/1411810340 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8c900a1e40 con 0x561b04ebb550 2023-07-14T16:25:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8cafd43b80 10 client.27884 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8cafd43b80 10 client.27884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8cafd43b80 20 client.27884 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561b04f27130 0x561b04f29540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8cafd43b80 4 client.27884 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:25:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8cafd43b80 1 -- 172.21.15.62:0/1411810340 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561b04dc80d0 con 0x561b04f27130 2023-07-14T16:25:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.572+0000 7f8c9999a700 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561b04f27130 0x561b04f29540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.573+0000 7f8c9999a700 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561b04f27130 0x561b04f29540 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ba2d568107bbdd server_cookie=ce49b0ebd6d94c4f in_seq=0 out_seq=0 2023-07-14T16:25:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.573+0000 7f8c7dffb700 10 client.27884 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.574+0000 7f8c7dffb700 1 -- 172.21.15.62:0/1411810340 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561b04dc80d0 con 0x561b04f27130 2023-07-14T16:25:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.574+0000 7f8c7dffb700 10 client.27884 handle_command_reply: tid=0 2023-07-14T16:25:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.574+0000 7f8cafd43b80 1 client.27884 shutdown 2023-07-14T16:25:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 -- 172.21.15.62:0/1411810340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 msgr2=0x561b04ec9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec9c90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009230 tx=0x7f8c90099af0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 -- 172.21.15.62:0/1411810340 shutdown_connections 2023-07-14T16:25:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561b04ebccf0 0x561b04ecb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561b04f27130 0x561b04f29540 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561b04ebb550 0x561b04ec9c90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 --2- 172.21.15.62:0/1411810340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b04ebbea0 0x561b04eca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8c7dffb700 0 client.27884 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 -- 172.21.15.62:0/1411810340 >> 172.21.15.62:0/1411810340 conn(0x561b04eb0e20 msgr2=0x561b04ec3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 -- 172.21.15.62:0/1411810340 shutdown_connections 2023-07-14T16:25:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 1 -- 172.21.15.62:0/1411810340 wait complete. 2023-07-14T16:25:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.575+0000 7f8cafd43b80 20 client.27884 trim_cache size 0 max 16384 2023-07-14T16:25:20.558 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:25:20.558 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:25:20.558 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "a2b537db-05dd-4af2-bc90-722465c01394", 2023-07-14T16:25:20.558 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:25:20.558 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:25:20.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8c800a11d0 msgr2=0x7f8c800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8c800a11d0 0x7f8c800a3680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009cd0 tx=0x7f8c8c00b740 comp rx=0 tx=0).stop 2023-07-14T16:25:20.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94101d10 msgr2=0x7f8c94197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94101d10 0x7f8c94197a40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000e9c0 tx=0x7f8c9000ecd0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/3258973259 shutdown_connections 2023-07-14T16:25:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c94103890 0x7f8c9419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c94101d10 0x7f8c94197a40 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8c800a11d0 0x7f8c800a3680 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 --2- 172.21.15.62:0/3258973259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c94102f00 0x7f8c94197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/3258973259 >> 172.21.15.62:0/3258973259 conn(0x7f8c940fd350 msgr2=0x7f8c941063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/3258973259 shutdown_connections 2023-07-14T16:25:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.576+0000 7f8c9c3ff700 1 -- 172.21.15.62:0/3258973259 wait complete. 2023-07-14T16:25:20.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:25:20.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.843+0000 7fbed5f9d700 1 -- 172.21.15.62:0/1387601707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0101610 msgr2=0x7fbed0107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.843+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/1387601707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0101610 0x7fbed0107eb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b0a0 tx=0x7fbec800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.844+0000 7fbed5f9d700 1 -- 172.21.15.62:0/1387601707 shutdown_connections 2023-07-14T16:25:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.844+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/1387601707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0101610 0x7fbed0107eb0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.844+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/1387601707 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbed00ff9d0 0x7fbed00ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.844+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/1387601707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed0100c60 0x7fbed01010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.844+0000 7fbed5f9d700 1 -- 172.21.15.62:0/1387601707 >> 172.21.15.62:0/1387601707 conn(0x7fbed00fb010 msgr2=0x7fbed00fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.844+0000 7fbed5f9d700 1 -- 172.21.15.62:0/1387601707 shutdown_connections 2023-07-14T16:25:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.844+0000 7fbed5f9d700 1 -- 172.21.15.62:0/1387601707 wait complete. 2023-07-14T16:25:20.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbed5f9d700 1 Processor -- start 2023-07-14T16:25:20.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbed5f9d700 1 -- start start 2023-07-14T16:25:20.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbed5f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00ff9d0 0x7fbed0197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbed5f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0100c60 0x7fbed0197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbed5f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbed0101610 0x7fbed019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbed5f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed019f390 con 0x7fbed0100c60 2023-07-14T16:25:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbed5f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed019f4d0 con 0x7fbed0101610 2023-07-14T16:25:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbed5f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed019f660 con 0x7fbed00ff9d0 2023-07-14T16:25:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.845+0000 7fbecffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0100c60 0x7fbed0197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbecffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0100c60 0x7fbed0197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52178/0 (socket says 172.21.15.62:52178) 2023-07-14T16:25:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbecffff700 1 -- 172.21.15.62:0/3354440898 learned_addr learned my addr 172.21.15.62:0/3354440898 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbed579c700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbed0101610 0x7fbed019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00ff9d0 0x7fbed0197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbecffff700 1 -- 172.21.15.62:0/3354440898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00ff9d0 msgr2=0x7fbed0197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbecffff700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00ff9d0 0x7fbed0197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbecffff700 1 -- 172.21.15.62:0/3354440898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbed0101610 msgr2=0x7fbed019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbecffff700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbed0101610 0x7fbed019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbecffff700 1 -- 172.21.15.62:0/3354440898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009d00 con 0x7fbed0100c60 2023-07-14T16:25:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.846+0000 7fbecffff700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0100c60 0x7fbed0197d50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00d950 tx=0x7fbebc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.847+0000 7fbecdffb700 1 -- 172.21.15.62:0/3354440898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00f890 con 0x7fbed0100c60 2023-07-14T16:25:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.847+0000 7fbecdffb700 1 -- 172.21.15.62:0/3354440898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbebc00fe80 con 0x7fbed0100c60 2023-07-14T16:25:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.847+0000 7fbed5f9d700 1 -- 172.21.15.62:0/3354440898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed019f940 con 0x7fbed0100c60 2023-07-14T16:25:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.847+0000 7fbecdffb700 1 -- 172.21.15.62:0/3354440898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00e3b0 con 0x7fbed0100c60 2023-07-14T16:25:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.847+0000 7fbed5f9d700 1 -- 172.21.15.62:0/3354440898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed019fea0 con 0x7fbed0100c60 2023-07-14T16:25:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.851+0000 7fbecdffb700 1 -- 172.21.15.62:0/3354440898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbebc010460 con 0x7fbed0100c60 2023-07-14T16:25:20.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.852+0000 7fbeeab3db80 1 Processor -- start 2023-07-14T16:25:20.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbecdffb700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbec00a1270 0x7fbec00a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbeeab3db80 1 -- start start 2023-07-14T16:25:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbecdffb700 1 -- 172.21.15.62:0/3354440898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbebc0cc660 con 0x7fbed0100c60 2023-07-14T16:25:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbed579c700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbec00a1270 0x7fbec00a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399de070 0x5630399de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dee30 0x5630399df2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630398e2cb0 con 0x5630399dd720 2023-07-14T16:25:20.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563039935a40 con 0x5630399dee30 2023-07-14T16:25:20.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56303992eed0 con 0x5630399de070 2023-07-14T16:25:20.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbed4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbed4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399ddb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52188/0 (socket says 172.21.15.62:52188) 2023-07-14T16:25:20.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.853+0000 7fbed4f9b700 1 -- 172.21.15.62:0/3691983554 learned_addr learned my addr 172.21.15.62:0/3691983554 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbecffff700 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399de070 0x5630399de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbed579c700 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dee30 0x5630399df2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbed4f9b700 1 -- 172.21.15.62:0/3691983554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399de070 msgr2=0x5630399de8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399de070 0x5630399de8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbed579c700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbec00a1270 0x7fbec00a3720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b830 tx=0x7fbec8006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbed4f9b700 1 -- 172.21.15.62:0/3691983554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dee30 msgr2=0x5630399df2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dee30 0x5630399df2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbed4f9b700 1 -- 172.21.15.62:0/3691983554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56303975cde0 con 0x5630399dd720 2023-07-14T16:25:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399ddb30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbec400bd70 tx=0x7fbec40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dcf66c0bf8db66 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbebaffd700 1 -- 172.21.15.62:0/3691983554 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4010e10 con 0x5630399dd720 2023-07-14T16:25:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.854+0000 7fbebaffd700 1 -- 172.21.15.62:0/3691983554 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbec4015070 con 0x5630399dd720 2023-07-14T16:25:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbebaffd700 1 -- 172.21.15.62:0/3691983554 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec40116a0 con 0x5630399dd720 2023-07-14T16:25:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3691983554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 msgr2=0x5630399ddb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399ddb30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbec400bd70 tx=0x7fbec40106b0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3691983554 shutdown_connections 2023-07-14T16:25:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dee30 0x5630399df2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399ddb30 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3691983554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399de070 0x5630399de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3691983554 >> 172.21.15.62:0/3691983554 conn(0x5630399d2ff0 msgr2=0x5630399d8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.855+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3691983554 shutdown_connections 2023-07-14T16:25:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.856+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3691983554 wait complete. 2023-07-14T16:25:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.856+0000 7fbeeab3db80 1 -- start start 2023-07-14T16:25:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.856+0000 7fbeeab3db80 1 Processor -- start 2023-07-14T16:25:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dd720 0x5630399e7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399de070 0x5630399e7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630398ceac0 con 0x5630399de070 2023-07-14T16:25:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630398ce7c0 con 0x5630399dd720 2023-07-14T16:25:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630397b2920 con 0x5630399dee30 2023-07-14T16:25:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbeeab3db80 10 client.0 _ll_register_callbacks cb 0x5630398cc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbecffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399de070 0x5630399e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbecffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399de070 0x5630399e7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52198/0 (socket says 172.21.15.62:52198) 2023-07-14T16:25:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbecffff700 1 -- 172.21.15.62:0/1127115440 learned_addr learned my addr 172.21.15.62:0/1127115440 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbed579c700 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.857+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dd720 0x5630399e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbecffff700 1 -- 172.21.15.62:0/1127115440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 msgr2=0x5630399ee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbecffff700 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399ee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbecffff700 1 -- 172.21.15.62:0/1127115440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dd720 msgr2=0x5630399e7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbecffff700 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dd720 0x5630399e7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbecffff700 1 -- 172.21.15.62:0/1127115440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec400b9a0 con 0x5630399de070 2023-07-14T16:25:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbecffff700 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399de070 0x5630399e7860 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0ca170 tx=0x7fbebc099a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107f6df07a182b98 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbeaffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbeaffff700 1 -- 172.21.15.62:0/1127115440 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc099170 con 0x5630399de070 2023-07-14T16:25:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1127115440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630398884f0 con 0x5630399de070 2023-07-14T16:25:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbeaffff700 1 -- 172.21.15.62:0/1127115440 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbebc0a1ed0 con 0x5630399de070 2023-07-14T16:25:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbeaffff700 1 -- 172.21.15.62:0/1127115440 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbebc0b2200 con 0x5630399de070 2023-07-14T16:25:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbeeab3db80 10 client.18276 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.858+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1127115440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563039745c60 con 0x5630399de070 2023-07-14T16:25:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbeaffff700 1 -- 172.21.15.62:0/1127115440 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5630398884f0 con 0x5630399de070 2023-07-14T16:25:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbeaffff700 1 -- 172.21.15.62:0/1127115440 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbebc0959e0 con 0x5630399de070 2023-07-14T16:25:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbeeab3db80 10 client.18276 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbeeab3db80 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbeeab3db80 20 client.18276 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630399f2f80 0x5630399f5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbeeab3db80 4 client.18276 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1127115440 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563039927eb0 con 0x5630399f2f80 2023-07-14T16:25:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.859+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630399f2f80 0x5630399f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.860+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630399f2f80 0x5630399f5390 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ada237dc35a248 server_cookie=bef06751b65fa4c0 in_seq=0 out_seq=0 2023-07-14T16:25:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.860+0000 7fbeaffff700 10 client.18276 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeaffff700 1 -- 172.21.15.62:0/1127115440 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563039927eb0 con 0x5630399f2f80 2023-07-14T16:25:20.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeaffff700 10 client.18276 handle_command_reply: tid=0 2023-07-14T16:25:20.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 client.18276 shutdown 2023-07-14T16:25:20.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1127115440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399de070 msgr2=0x5630399e7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399de070 0x5630399e7860 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0ca170 tx=0x7fbebc099a80 comp rx=0 tx=0).stop 2023-07-14T16:25:20.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1127115440 shutdown_connections 2023-07-14T16:25:20.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399ee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5630399f2f80 0x5630399f5390 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399dd720 0x5630399e7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeaffff700 0 client.18276 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1127115440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399de070 0x5630399e7860 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1127115440 >> 172.21.15.62:0/1127115440 conn(0x5630399d2ff0 msgr2=0x5630399d7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1127115440 shutdown_connections 2023-07-14T16:25:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.861+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1127115440 wait complete. 2023-07-14T16:25:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.862+0000 7fbeeab3db80 20 client.18276 trim_cache size 0 max 16384 2023-07-14T16:25:20.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbeeab3db80 1 Processor -- start 2023-07-14T16:25:20.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbeeab3db80 1 -- start start 2023-07-14T16:25:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399ee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630399e6ee0 con 0x5630399dd720 2023-07-14T16:25:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630399d9540 con 0x5630399de070 2023-07-14T16:25:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630399ddaf0 con 0x5630399dee30 2023-07-14T16:25:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbed4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbed4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399e51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52204/0 (socket says 172.21.15.62:52204) 2023-07-14T16:25:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.883+0000 7fbed4f9b700 1 -- 172.21.15.62:0/1366107596 learned_addr learned my addr 172.21.15.62:0/1366107596 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbecffff700 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbed4f9b700 1 -- 172.21.15.62:0/1366107596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 msgr2=0x5630399e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbed579c700 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbed4f9b700 1 -- 172.21.15.62:0/1366107596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 msgr2=0x5630399ee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399ee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbed4f9b700 1 -- 172.21.15.62:0/1366107596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563039783250 con 0x5630399dd720 2023-07-14T16:25:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbed579c700 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbecffff700 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399e51e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbec4010a00 tx=0x7fbec4005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c4955dfb4380e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbebb7fe700 1 -- 172.21.15.62:0/1366107596 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4022040 con 0x5630399dd720 2023-07-14T16:25:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.884+0000 7fbebb7fe700 1 -- 172.21.15.62:0/1366107596 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbec4027de0 con 0x5630399dd720 2023-07-14T16:25:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbebb7fe700 1 -- 172.21.15.62:0/1366107596 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec40054a0 con 0x5630399dd720 2023-07-14T16:25:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1366107596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 msgr2=0x5630399e51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399e51e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbec4010a00 tx=0x7fbec4005950 comp rx=0 tx=0).stop 2023-07-14T16:25:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1366107596 shutdown_connections 2023-07-14T16:25:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399e51e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/1366107596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1366107596 >> 172.21.15.62:0/1366107596 conn(0x5630399d2ff0 msgr2=0x5630399e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1366107596 shutdown_connections 2023-07-14T16:25:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.885+0000 7fbeeab3db80 1 -- 172.21.15.62:0/1366107596 wait complete. 2023-07-14T16:25:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.886+0000 7fbeeab3db80 1 -- start start 2023-07-14T16:25:20.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.886+0000 7fbeeab3db80 1 Processor -- start 2023-07-14T16:25:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.886+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.886+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbeeab3db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630399ee4d0 con 0x5630399dd720 2023-07-14T16:25:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630399ee640 con 0x5630399de070 2023-07-14T16:25:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbeeab3db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630399ee800 con 0x5630399dee30 2023-07-14T16:25:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbeeab3db80 10 client.0 _ll_register_callbacks cb 0x7fbebc0993e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399eaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52214/0 (socket says 172.21.15.62:52214) 2023-07-14T16:25:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed4f9b700 1 -- 172.21.15.62:0/3115216390 learned_addr learned my addr 172.21.15.62:0/3115216390 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbecffff700 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed579c700 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed4f9b700 1 -- 172.21.15.62:0/3115216390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 msgr2=0x5630399edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed4f9b700 1 -- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 msgr2=0x5630399eafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399eafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbed4f9b700 1 -- 172.21.15.62:0/3115216390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563039745c60 con 0x5630399dd720 2023-07-14T16:25:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.887+0000 7fbecffff700 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399eafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.888+0000 7fbed4f9b700 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399eaa90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbec400bc90 tx=0x7fbec4010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53ee1d7f1c28db9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.888+0000 7fbeaffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.888+0000 7fbeaffff700 1 -- 172.21.15.62:0/3115216390 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4022070 con 0x5630399dd720 2023-07-14T16:25:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.888+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3115216390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbec4015070 con 0x5630399dd720 2023-07-14T16:25:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.888+0000 7fbeaffff700 1 -- 172.21.15.62:0/3115216390 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbec4017070 con 0x5630399dd720 2023-07-14T16:25:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.888+0000 7fbeaffff700 1 -- 172.21.15.62:0/3115216390 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbec4005220 con 0x5630399dd720 2023-07-14T16:25:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.888+0000 7fbeeab3db80 10 client.18282 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.888+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3115216390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56303975cde0 con 0x5630399dd720 2023-07-14T16:25:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbeaffff700 1 -- 172.21.15.62:0/3115216390 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbec4015070 con 0x5630399dd720 2023-07-14T16:25:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbeaffff700 1 -- 172.21.15.62:0/3115216390 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbec4042070 con 0x5630399dd720 2023-07-14T16:25:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbeeab3db80 10 client.18282 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbeeab3db80 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbeeab3db80 20 client.18282 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563039a47fd0 0x563039a4a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbeeab3db80 4 client.18282 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:25:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3115216390 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5630398ea050 con 0x563039a47fd0 2023-07-14T16:25:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.889+0000 7fbecffff700 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563039a47fd0 0x563039a4a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbecffff700 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563039a47fd0 0x563039a4a3e0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6fe07de438c1c9 server_cookie=4df5aaefbf7d4e4b in_seq=0 out_seq=0 2023-07-14T16:25:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeaffff700 10 client.18282 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:20.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeaffff700 1 -- 172.21.15.62:0/3115216390 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5630398ea050 con 0x563039a47fd0 2023-07-14T16:25:20.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeaffff700 10 client.18282 handle_command_reply: tid=0 2023-07-14T16:25:20.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 client.18282 shutdown 2023-07-14T16:25:20.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3115216390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 msgr2=0x5630399eaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399eaa90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbec400bc90 tx=0x7fbec4010f20 comp rx=0 tx=0).stop 2023-07-14T16:25:20.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3115216390 shutdown_connections 2023-07-14T16:25:20.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5630399dee30 0x5630399edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5630399dd720 0x5630399eaa90 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563039a47fd0 0x563039a4a3e0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 --2- 172.21.15.62:0/3115216390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5630399de070 0x5630399eafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeaffff700 0 client.18282 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:20.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.890+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3115216390 >> 172.21.15.62:0/3115216390 conn(0x5630399d2ff0 msgr2=0x5630399e51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.891+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3115216390 shutdown_connections 2023-07-14T16:25:20.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.891+0000 7fbeeab3db80 1 -- 172.21.15.62:0/3115216390 wait complete. 2023-07-14T16:25:20.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.891+0000 7fbeeab3db80 20 client.18282 trim_cache size 0 max 16384 2023-07-14T16:25:20.875 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:25:20.875 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:25:20.875 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:25:20.875 INFO:teuthology.orchestra.run.smithi062.stdout: "a2b537db-05dd-4af2-bc90-722465c01394": { 2023-07-14T16:25:20.875 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:25:20.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "a2b537db-05dd-4af2-bc90-722465c01394", 2023-07-14T16:25:20.876 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:25:20.876 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:25:20.876 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:25:20.876 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:25:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.891+0000 7fbed5f9d700 1 -- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbec00a1270 msgr2=0x7fbec00a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.891+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbec00a1270 0x7fbec00a3720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b830 tx=0x7fbec8006210 comp rx=0 tx=0).stop 2023-07-14T16:25:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.891+0000 7fbed5f9d700 1 -- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0100c60 msgr2=0x7fbed0197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.891+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0100c60 0x7fbed0197d50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00d950 tx=0x7fbebc00dc60 comp rx=0 tx=0).stop 2023-07-14T16:25:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.891+0000 7fbed5f9d700 1 -- 172.21.15.62:0/3354440898 shutdown_connections 2023-07-14T16:25:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.892+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbec00a1270 0x7fbec00a3720 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.892+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbed0101610 0x7fbed019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.892+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed00ff9d0 0x7fbed0197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.892+0000 7fbed5f9d700 1 --2- 172.21.15.62:0/3354440898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbed0100c60 0x7fbed0197d50 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:20.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.892+0000 7fbed5f9d700 1 -- 172.21.15.62:0/3354440898 >> 172.21.15.62:0/3354440898 conn(0x7fbed00fb010 msgr2=0x7fbed00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:20.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.892+0000 7fbed5f9d700 1 -- 172.21.15.62:0/3354440898 shutdown_connections 2023-07-14T16:25:20.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:20.892+0000 7fbed5f9d700 1 -- 172.21.15.62:0/3354440898 wait complete. 2023-07-14T16:25:20.890 INFO:tasks.cephfs.filesystem:scrub status for tag:a2b537db-05dd-4af2-bc90-722465c01394 - {'path': '/', 'tag': 'a2b537db-05dd-4af2-bc90-722465c01394', 'options': 'recursive,force'} 2023-07-14T16:25:20.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:20 smithi081 ceph-mon[127795]: pgmap v648: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:25:20.901 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:25:20 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:25:20.584+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v197>,<0x10000000003/client.0 v198>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v210>,<0x10000000001/sv_0 v224>,<0x10000000000/_nogroup v218>,<0x1/volumes v208>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:25:21.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:20 smithi164 ceph-mon[97427]: pgmap v648: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:25:21.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:20 smithi062 ceph-mon[95543]: pgmap v648: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:25:22.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:25:22.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:25:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:25:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:25:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:25:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:21 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:25:22.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:25:22.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:25:22.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:25:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:25:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:21 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:25:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:25:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:25:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:25:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:25:22.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:22.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:21 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:23.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:22 smithi164 ceph-mon[97427]: pgmap v649: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 21 op/s 2023-07-14T16:25:23.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:22 smithi062 ceph-mon[95543]: pgmap v649: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 21 op/s 2023-07-14T16:25:23.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:22 smithi081 ceph-mon[127795]: pgmap v649: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 21 op/s 2023-07-14T16:25:25.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:24 smithi164 ceph-mon[97427]: pgmap v650: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:25:25.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:24 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:25:25.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:24 smithi062 ceph-mon[95543]: pgmap v650: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:25:25.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:24 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:25:25.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:24 smithi081 ceph-mon[127795]: pgmap v650: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:25:25.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:24 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:25:26.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:25:26.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:25:26.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:25:26.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:25:26.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:25:26.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:25:26.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:25:26.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:25:26.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:25:27.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:26 smithi164 ceph-mon[97427]: pgmap v651: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:25:27.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:26 smithi062 ceph-mon[95543]: pgmap v651: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:25:27.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:26 smithi081 ceph-mon[127795]: pgmap v651: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:25:29.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:28 smithi164 ceph-mon[97427]: pgmap v652: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:25:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:25:29.025 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:25:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:28 smithi062 ceph-mon[95543]: pgmap v652: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:25:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:25:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:25:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:28 smithi081 ceph-mon[127795]: pgmap v652: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:25:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:25:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:25:31.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:30 smithi164 ceph-mon[97427]: pgmap v653: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:25:31.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:30 smithi062 ceph-mon[95543]: pgmap v653: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:25:31.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:30 smithi081 ceph-mon[127795]: pgmap v653: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:25:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.680+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2230893701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c981079a0 msgr2=0x7f6c98109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.680+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2230893701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c981079a0 0x7f6c98109e20 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000b0a0 tx=0x7f6c9000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.681+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2230893701 shutdown_connections 2023-07-14T16:25:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.681+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2230893701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c981079a0 0x7f6c98109e20 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.681+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2230893701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c980744f0 0x7f6c98072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.681+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2230893701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c98072f10 0x7f6c98073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.681+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2230893701 >> 172.21.15.62:0/2230893701 conn(0x7f6c980fb130 msgr2=0x7f6c980fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.681+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2230893701 shutdown_connections 2023-07-14T16:25:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.681+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2230893701 wait complete. 2023-07-14T16:25:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.682+0000 7f6c9ddff700 1 Processor -- start 2023-07-14T16:25:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.682+0000 7f6c9ddff700 1 -- start start 2023-07-14T16:25:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.682+0000 7f6c9ddff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c98072f10 0x7f6c98197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.682+0000 7f6c9ddff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c980744f0 0x7f6c981980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.682+0000 7f6c9ddff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c981079a0 0x7f6c98198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.682+0000 7f6c9ddff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98199590 con 0x7f6c98072f10 2023-07-14T16:25:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.682+0000 7f6c9ddff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98199700 con 0x7f6c981079a0 2023-07-14T16:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.682+0000 7f6c9ddff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981998c0 con 0x7f6c980744f0 2023-07-14T16:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c98072f10 0x7f6c98197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c98072f10 0x7f6c98197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49082/0 (socket says 172.21.15.62:49082) 2023-07-14T16:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c977fe700 1 -- 172.21.15.62:0/2818806784 learned_addr learned my addr 172.21.15.62:0/2818806784 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c97fff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c981079a0 0x7f6c98198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c980744f0 0x7f6c981980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c977fe700 1 -- 172.21.15.62:0/2818806784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c980744f0 msgr2=0x7f6c981980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c977fe700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c980744f0 0x7f6c981980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c977fe700 1 -- 172.21.15.62:0/2818806784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c981079a0 msgr2=0x7f6c98198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c977fe700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c981079a0 0x7f6c98198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.683+0000 7f6c977fe700 1 -- 172.21.15.62:0/2818806784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90009d00 con 0x7f6c98072f10 2023-07-14T16:25:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.684+0000 7f6c977fe700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c98072f10 0x7f6c98197b70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800d950 tx=0x7f6c8800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.684+0000 7f6c94ff9700 1 -- 172.21.15.62:0/2818806784 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800f890 con 0x7f6c98072f10 2023-07-14T16:25:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.684+0000 7f6c94ff9700 1 -- 172.21.15.62:0/2818806784 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c8800fe80 con 0x7f6c98072f10 2023-07-14T16:25:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.684+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2818806784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c98199ba0 con 0x7f6c98072f10 2023-07-14T16:25:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.684+0000 7f6c94ff9700 1 -- 172.21.15.62:0/2818806784 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800e3b0 con 0x7f6c98072f10 2023-07-14T16:25:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.684+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2818806784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c981a0360 con 0x7f6c98072f10 2023-07-14T16:25:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.688+0000 7f6c94ff9700 1 -- 172.21.15.62:0/2818806784 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6c88010460 con 0x7f6c98072f10 2023-07-14T16:25:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.689+0000 7f6cb163db80 1 Processor -- start 2023-07-14T16:25:31.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.689+0000 7f6cb163db80 1 -- start start 2023-07-14T16:25:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c51300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c52b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6cb163db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8b56c50 con 0x5627e8c51840 2023-07-14T16:25:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6cb163db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8ba99e0 con 0x5627e8aad6b0 2023-07-14T16:25:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6cb163db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8ba2e70 con 0x5627e8c526e0 2023-07-14T16:25:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6c94ff9700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c800a11d0 0x7f6c800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c52b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c51300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c52b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46566/0 (socket says 172.21.15.62:46566) 2023-07-14T16:25:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6c96ffd700 1 -- 172.21.15.62:0/2964620588 learned_addr learned my addr 172.21.15.62:0/2964620588 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6c94ff9700 1 -- 172.21.15.62:0/2818806784 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c880cc6c0 con 0x7f6c98072f10 2023-07-14T16:25:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6c977fe700 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.690+0000 7f6c97fff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c800a11d0 0x7f6c800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c96ffd700 1 -- 172.21.15.62:0/2964620588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 msgr2=0x5627e8c51300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c51300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c96ffd700 1 -- 172.21.15.62:0/2964620588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 msgr2=0x5627e8c521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c96ffd700 1 -- 172.21.15.62:0/2964620588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e89d0d80 con 0x5627e8c526e0 2023-07-14T16:25:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c977fe700 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c97fff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c800a11d0 0x7f6c800a3680 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000b830 tx=0x7f6c90006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:31.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c52b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bd70 tx=0x7f6c8c0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d7d64acd102204 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c7dffb700 1 -- 172.21.15.62:0/2964620588 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c010e10 con 0x5627e8c526e0 2023-07-14T16:25:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.691+0000 7f6c7dffb700 1 -- 172.21.15.62:0/2964620588 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c8c015070 con 0x5627e8c526e0 2023-07-14T16:25:31.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6c7dffb700 1 -- 172.21.15.62:0/2964620588 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0116a0 con 0x5627e8c526e0 2023-07-14T16:25:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6cb163db80 1 -- 172.21.15.62:0/2964620588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 msgr2=0x5627e8c52b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c52b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bd70 tx=0x7f6c8c0106b0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6cb163db80 1 -- 172.21.15.62:0/2964620588 shutdown_connections 2023-07-14T16:25:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c52b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c51300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2964620588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6cb163db80 1 -- 172.21.15.62:0/2964620588 >> 172.21.15.62:0/2964620588 conn(0x5627e8c46be0 msgr2=0x5627e8c4bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.692+0000 7f6cb163db80 1 -- 172.21.15.62:0/2964620588 shutdown_connections 2023-07-14T16:25:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.693+0000 7f6cb163db80 1 -- 172.21.15.62:0/2964620588 wait complete. 2023-07-14T16:25:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.693+0000 7f6cb163db80 1 -- start start 2023-07-14T16:25:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.693+0000 7f6cb163db80 1 Processor -- start 2023-07-14T16:25:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.694+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c5a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.694+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c5ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.694+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c61dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.694+0000 7f6cb163db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8b42a60 con 0x5627e8c51840 2023-07-14T16:25:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.694+0000 7f6cb163db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8b42760 con 0x5627e8aad6b0 2023-07-14T16:25:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.694+0000 7f6cb163db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8a268c0 con 0x5627e8c526e0 2023-07-14T16:25:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.694+0000 7f6cb163db80 10 client.0 _ll_register_callbacks cb 0x5627e8b40700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.694+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c5ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c5ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49094/0 (socket says 172.21.15.62:49094) 2023-07-14T16:25:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c96ffd700 1 -- 172.21.15.62:0/2154558211 learned_addr learned my addr 172.21.15.62:0/2154558211 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c977fe700 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c5a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c97fff700 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c61dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c96ffd700 1 -- 172.21.15.62:0/2154558211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 msgr2=0x5627e8c61dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c61dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c96ffd700 1 -- 172.21.15.62:0/2154558211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 msgr2=0x5627e8c5a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c5a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c96ffd700 1 -- 172.21.15.62:0/2154558211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c00b9a0 con 0x5627e8c51840 2023-07-14T16:25:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c97fff700 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c61dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c5ad80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00b6c0 tx=0x7f6c8c010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0f6ac9fe1bf68e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.695+0000 7f6c76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.696+0000 7f6c76ffd700 1 -- 172.21.15.62:0/2154558211 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c020040 con 0x5627e8c51840 2023-07-14T16:25:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.696+0000 7f6c76ffd700 1 -- 172.21.15.62:0/2154558211 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c8c017070 con 0x5627e8c51840 2023-07-14T16:25:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.696+0000 7f6cb163db80 1 -- 172.21.15.62:0/2154558211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627e8afc490 con 0x5627e8c51840 2023-07-14T16:25:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.696+0000 7f6c76ffd700 1 -- 172.21.15.62:0/2154558211 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c8c00bec0 con 0x5627e8c51840 2023-07-14T16:25:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.696+0000 7f6cb163db80 10 client.18291 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.696+0000 7f6cb163db80 1 -- 172.21.15.62:0/2154558211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5627e89b9c00 con 0x5627e8c51840 2023-07-14T16:25:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.697+0000 7f6c76ffd700 1 -- 172.21.15.62:0/2154558211 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5627e8afc490 con 0x5627e8c51840 2023-07-14T16:25:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.697+0000 7f6c76ffd700 1 -- 172.21.15.62:0/2154558211 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6c8c003ad0 con 0x5627e8c51840 2023-07-14T16:25:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.697+0000 7f6cb163db80 10 client.18291 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.697+0000 7f6cb163db80 10 client.18291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.697+0000 7f6cb163db80 20 client.18291 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.697+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5627e8c66500 0x5627e8c68910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.697+0000 7f6cb163db80 4 client.18291 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.697+0000 7f6cb163db80 1 -- 172.21.15.62:0/2154558211 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627e8b9be50 con 0x5627e8c66500 2023-07-14T16:25:31.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.698+0000 7f6c977fe700 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5627e8c66500 0x5627e8c68910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.698+0000 7f6c977fe700 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5627e8c66500 0x5627e8c68910 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863013f0e41c2a34 server_cookie=61a2c7688519ca1a in_seq=0 out_seq=0 2023-07-14T16:25:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.698+0000 7f6c76ffd700 10 client.18291 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6c76ffd700 1 -- 172.21.15.62:0/2154558211 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5627e8b9be50 con 0x5627e8c66500 2023-07-14T16:25:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6c76ffd700 10 client.18291 handle_command_reply: tid=0 2023-07-14T16:25:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 client.18291 shutdown 2023-07-14T16:25:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 -- 172.21.15.62:0/2154558211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 msgr2=0x5627e8c5ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c5ad80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00b6c0 tx=0x7f6c8c010e40 comp rx=0 tx=0).stop 2023-07-14T16:25:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 -- 172.21.15.62:0/2154558211 shutdown_connections 2023-07-14T16:25:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c61dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5627e8c66500 0x5627e8c68910 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c5a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6c76ffd700 0 client.18291 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 --2- 172.21.15.62:0/2154558211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c5ad80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.699+0000 7f6cb163db80 1 -- 172.21.15.62:0/2154558211 >> 172.21.15.62:0/2154558211 conn(0x5627e8c46be0 msgr2=0x5627e8c4b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.700+0000 7f6cb163db80 1 -- 172.21.15.62:0/2154558211 shutdown_connections 2023-07-14T16:25:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.700+0000 7f6cb163db80 1 -- 172.21.15.62:0/2154558211 wait complete. 2023-07-14T16:25:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.700+0000 7f6cb163db80 20 client.18291 trim_cache size 0 max 16384 2023-07-14T16:25:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.720+0000 7f6cb163db80 1 Processor -- start 2023-07-14T16:25:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.720+0000 7f6cb163db80 1 -- start start 2023-07-14T16:25:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.720+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c6d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c61d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c5a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6cb163db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8c48d00 con 0x5627e8c51840 2023-07-14T16:25:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6cb163db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8c43bb0 con 0x5627e8aad6b0 2023-07-14T16:25:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6cb163db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8c5b4c0 con 0x5627e8c526e0 2023-07-14T16:25:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c61d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c61d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49104/0 (socket says 172.21.15.62:49104) 2023-07-14T16:25:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c96ffd700 1 -- 172.21.15.62:0/3553132315 learned_addr learned my addr 172.21.15.62:0/3553132315 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c97fff700 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c5a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c977fe700 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c6d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c96ffd700 1 -- 172.21.15.62:0/3553132315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 msgr2=0x5627e8c5a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c5a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c96ffd700 1 -- 172.21.15.62:0/3553132315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 msgr2=0x5627e8c6d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c6d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c96ffd700 1 -- 172.21.15.62:0/3553132315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e89f71f0 con 0x5627e8c51840 2023-07-14T16:25:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c977fe700 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c6d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.721+0000 7f6c97fff700 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c5a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.722+0000 7f6c96ffd700 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c61d80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00b450 tx=0x7f6c8c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc19e6a6e0879388 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.722+0000 7f6c7e7fc700 1 -- 172.21.15.62:0/3553132315 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c020070 con 0x5627e8c51840 2023-07-14T16:25:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.722+0000 7f6c7e7fc700 1 -- 172.21.15.62:0/3553132315 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c8c00bec0 con 0x5627e8c51840 2023-07-14T16:25:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.722+0000 7f6c7e7fc700 1 -- 172.21.15.62:0/3553132315 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c011d20 con 0x5627e8c51840 2023-07-14T16:25:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 -- 172.21.15.62:0/3553132315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 msgr2=0x5627e8c61d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c61d80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00b450 tx=0x7f6c8c00b560 comp rx=0 tx=0).stop 2023-07-14T16:25:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 -- 172.21.15.62:0/3553132315 shutdown_connections 2023-07-14T16:25:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8c526e0 0x5627e8c5a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8aad6b0 0x5627e8c6d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3553132315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c51840 0x5627e8c61d80 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 -- 172.21.15.62:0/3553132315 >> 172.21.15.62:0/3553132315 conn(0x5627e8c46be0 msgr2=0x5627e8c595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 -- 172.21.15.62:0/3553132315 shutdown_connections 2023-07-14T16:25:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 -- 172.21.15.62:0/3553132315 wait complete. 2023-07-14T16:25:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.723+0000 7f6cb163db80 1 -- start start 2023-07-14T16:25:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6cb163db80 1 Processor -- start 2023-07-14T16:25:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8aad6b0 0x5627e8c5df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8c51840 0x5627e8c5e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6cb163db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c526e0 0x5627e8c60d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6cb163db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8c61260 con 0x5627e8c526e0 2023-07-14T16:25:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6cb163db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8c613d0 con 0x5627e8c51840 2023-07-14T16:25:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6cb163db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e8c61590 con 0x5627e8aad6b0 2023-07-14T16:25:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6cb163db80 10 client.0 _ll_register_callbacks cb 0x7f6c90009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.724+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c526e0 0x5627e8c60d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8c51840 0x5627e8c5e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c526e0 0x5627e8c60d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49114/0 (socket says 172.21.15.62:49114) 2023-07-14T16:25:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c97fff700 1 -- 172.21.15.62:0/3409074760 learned_addr learned my addr 172.21.15.62:0/3409074760 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c977fe700 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8aad6b0 0x5627e8c5df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c97fff700 1 -- 172.21.15.62:0/3409074760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8aad6b0 msgr2=0x5627e8c5df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c97fff700 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8aad6b0 0x5627e8c5df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c97fff700 1 -- 172.21.15.62:0/3409074760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8c51840 msgr2=0x5627e8c5e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c97fff700 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8c51840 0x5627e8c5e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c97fff700 1 -- 172.21.15.62:0/3409074760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627e89b9c00 con 0x5627e8c526e0 2023-07-14T16:25:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c97fff700 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c526e0 0x5627e8c60d20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6c90007a30 tx=0x7f6c900099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed28e7587c39f52 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.725+0000 7f6c76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.726+0000 7f6c76ffd700 1 -- 172.21.15.62:0/3409074760 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90025040 con 0x5627e8c526e0 2023-07-14T16:25:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.726+0000 7f6c76ffd700 1 -- 172.21.15.62:0/3409074760 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c9001fcb0 con 0x5627e8c526e0 2023-07-14T16:25:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.726+0000 7f6cb163db80 1 -- 172.21.15.62:0/3409074760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627e8b5dff0 con 0x5627e8c526e0 2023-07-14T16:25:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.726+0000 7f6c76ffd700 1 -- 172.21.15.62:0/3409074760 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c9000e070 con 0x5627e8c526e0 2023-07-14T16:25:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.726+0000 7f6cb163db80 10 client.18297 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.726+0000 7f6cb163db80 1 -- 172.21.15.62:0/3409074760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5627e89cf250 con 0x5627e8c526e0 2023-07-14T16:25:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.726+0000 7f6c76ffd700 1 -- 172.21.15.62:0/3409074760 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c9003c020 con 0x5627e8c526e0 2023-07-14T16:25:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.727+0000 7f6c76ffd700 1 -- 172.21.15.62:0/3409074760 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6c90005040 con 0x5627e8c526e0 2023-07-14T16:25:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.727+0000 7f6cb163db80 10 client.18297 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.727+0000 7f6cb163db80 10 client.18297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.727+0000 7f6cb163db80 20 client.18297 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.727+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5627e8cbaf30 0x5627e8cbd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.727+0000 7f6cb163db80 4 client.18297 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:25:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.727+0000 7f6cb163db80 1 -- 172.21.15.62:0/3409074760 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5627e89f1f10 con 0x5627e8cbaf30 2023-07-14T16:25:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.727+0000 7f6c977fe700 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5627e8cbaf30 0x5627e8cbd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.728+0000 7f6c76ffd700 10 client.18297 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.728+0000 7f6c977fe700 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5627e8cbaf30 0x5627e8cbd340 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c9fde8f55e36 server_cookie=7cb6278dc2e6e137 in_seq=0 out_seq=0 2023-07-14T16:25:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.737+0000 7f6c76ffd700 1 -- 172.21.15.62:0/3409074760 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5627e89f1f10 con 0x5627e8cbaf30 2023-07-14T16:25:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.737+0000 7f6c76ffd700 10 client.18297 handle_command_reply: tid=0 2023-07-14T16:25:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.737+0000 7f6cb163db80 1 client.18297 shutdown 2023-07-14T16:25:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 -- 172.21.15.62:0/3409074760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c526e0 msgr2=0x5627e8c60d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c526e0 0x5627e8c60d20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6c90007a30 tx=0x7f6c900099e0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 -- 172.21.15.62:0/3409074760 shutdown_connections 2023-07-14T16:25:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5627e8c526e0 0x5627e8c60d20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5627e8aad6b0 0x5627e8c5df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5627e8cbaf30 0x5627e8cbd340 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 --2- 172.21.15.62:0/3409074760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e8c51840 0x5627e8c5e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6c76ffd700 0 client.18297 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 -- 172.21.15.62:0/3409074760 >> 172.21.15.62:0/3409074760 conn(0x5627e8c46be0 msgr2=0x5627e8c6d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 -- 172.21.15.62:0/3409074760 shutdown_connections 2023-07-14T16:25:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 1 -- 172.21.15.62:0/3409074760 wait complete. 2023-07-14T16:25:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.738+0000 7f6cb163db80 20 client.18297 trim_cache size 0 max 16384 2023-07-14T16:25:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:25:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:25:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:25:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:25:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c800a11d0 msgr2=0x7f6c800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c800a11d0 0x7f6c800a3680 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000b830 tx=0x7f6c90006210 comp rx=0 tx=0).stop 2023-07-14T16:25:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c98072f10 msgr2=0x7f6c98197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c98072f10 0x7f6c98197b70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800d950 tx=0x7f6c8800dc60 comp rx=0 tx=0).stop 2023-07-14T16:25:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2818806784 shutdown_connections 2023-07-14T16:25:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c800a11d0 0x7f6c800a3680 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c981079a0 0x7f6c98198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c98072f10 0x7f6c98197b70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 --2- 172.21.15.62:0/2818806784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c980744f0 0x7f6c981980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2818806784 >> 172.21.15.62:0/2818806784 conn(0x7f6c980fb130 msgr2=0x7f6c980fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2818806784 shutdown_connections 2023-07-14T16:25:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:31.739+0000 7f6c9ddff700 1 -- 172.21.15.62:0/2818806784 wait complete. 2023-07-14T16:25:33.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:32 smithi164 ceph-mon[97427]: pgmap v654: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:25:33.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:32 smithi062 ceph-mon[95543]: pgmap v654: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:25:33.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:32 smithi081 ceph-mon[127795]: pgmap v654: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:25:35.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:34 smithi164 ceph-mon[97427]: pgmap v655: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 29 op/s 2023-07-14T16:25:35.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:34 smithi062 ceph-mon[95543]: pgmap v655: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 29 op/s 2023-07-14T16:25:35.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:34 smithi081 ceph-mon[127795]: pgmap v655: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 29 op/s 2023-07-14T16:25:37.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:36 smithi062 ceph-mon[95543]: pgmap v656: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 18 op/s 2023-07-14T16:25:37.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:36 smithi081 ceph-mon[127795]: pgmap v656: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 18 op/s 2023-07-14T16:25:37.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:36 smithi164 ceph-mon[97427]: pgmap v656: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 18 op/s 2023-07-14T16:25:39.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:38 smithi062 ceph-mon[95543]: pgmap v657: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:25:39.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:38 smithi081 ceph-mon[127795]: pgmap v657: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:25:39.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:38 smithi164 ceph-mon[97427]: pgmap v657: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:25:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:40 smithi062 ceph-mon[95543]: pgmap v658: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:25:41.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:40 smithi081 ceph-mon[127795]: pgmap v658: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:25:41.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:40 smithi164 ceph-mon[97427]: pgmap v658: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:25:43.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:42 smithi062 ceph-mon[95543]: pgmap v659: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:25:43.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:42 smithi081 ceph-mon[127795]: pgmap v659: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:25:43.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:42 smithi164 ceph-mon[97427]: pgmap v659: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:25:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 -- 172.21.15.62:0/873253062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c0ffb00 msgr2=0x7f652c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 --2- 172.21.15.62:0/873253062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c0ffb00 0x7f652c0fff10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6520009990 tx=0x7f6520009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 -- 172.21.15.62:0/873253062 shutdown_connections 2023-07-14T16:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 --2- 172.21.15.62:0/873253062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f652c101680 0x7f652c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 --2- 172.21.15.62:0/873253062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c0ffb00 0x7f652c0fff10 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 --2- 172.21.15.62:0/873253062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f652c100cf0 0x7f652c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 -- 172.21.15.62:0/873253062 >> 172.21.15.62:0/873253062 conn(0x7f652c0fb0e0 msgr2=0x7f652c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 -- 172.21.15.62:0/873253062 shutdown_connections 2023-07-14T16:25:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.902+0000 7f6532bff700 1 -- 172.21.15.62:0/873253062 wait complete. 2023-07-14T16:25:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.903+0000 7f6532bff700 1 Processor -- start 2023-07-14T16:25:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.903+0000 7f6532bff700 1 -- start start 2023-07-14T16:25:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f6532bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f652c0ffb00 0x7f652c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f6532bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f652c100cf0 0x7f652c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f6532bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c101680 0x7f652c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f6532bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1994e0 con 0x7f652c101680 2023-07-14T16:25:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f6532bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c199650 con 0x7f652c100cf0 2023-07-14T16:25:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f6532bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c199810 con 0x7f652c0ffb00 2023-07-14T16:25:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653119c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c101680 0x7f652c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653119c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c101680 0x7f652c198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34122/0 (socket says 172.21.15.62:34122) 2023-07-14T16:25:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653119c700 1 -- 172.21.15.62:0/1630152013 learned_addr learned my addr 172.21.15.62:0/1630152013 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f652bfff700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f652c100cf0 0x7f652c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653099b700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f652c0ffb00 0x7f652c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653119c700 1 -- 172.21.15.62:0/1630152013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f652c0ffb00 msgr2=0x7f652c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653119c700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f652c0ffb00 0x7f652c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653119c700 1 -- 172.21.15.62:0/1630152013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f652c100cf0 msgr2=0x7f652c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653119c700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f652c100cf0 0x7f652c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.904+0000 7f653119c700 1 -- 172.21.15.62:0/1630152013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200095f0 con 0x7f652c101680 2023-07-14T16:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.905+0000 7f653119c700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c101680 0x7f652c198e80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6524007ee0 tx=0x7f652400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.905+0000 7f6529ffb700 1 -- 172.21.15.62:0/1630152013 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524017070 con 0x7f652c101680 2023-07-14T16:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.905+0000 7f6532bff700 1 -- 172.21.15.62:0/1630152013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c199af0 con 0x7f652c101680 2023-07-14T16:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.905+0000 7f6529ffb700 1 -- 172.21.15.62:0/1630152013 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f652400f040 con 0x7f652c101680 2023-07-14T16:25:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.905+0000 7f6529ffb700 1 -- 172.21.15.62:0/1630152013 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524005710 con 0x7f652c101680 2023-07-14T16:25:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.905+0000 7f6532bff700 1 -- 172.21.15.62:0/1630152013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c1a02b0 con 0x7f652c101680 2023-07-14T16:25:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.909+0000 7f6529ffb700 1 -- 172.21.15.62:0/1630152013 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6524005870 con 0x7f652c101680 2023-07-14T16:25:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.911+0000 7f6546487b80 1 Processor -- start 2023-07-14T16:25:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.911+0000 7f6546487b80 1 -- start start 2023-07-14T16:25:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.911+0000 7f6546487b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892da300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.911+0000 7f6546487b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892da840 0x55e1892db1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.911+0000 7f6546487b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892db6e0 0x55e1892dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f6546487b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1891dfc50 con 0x55e1891366b0 2023-07-14T16:25:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f6546487b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1892329e0 con 0x55e1892db6e0 2023-07-14T16:25:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f6546487b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18922be70 con 0x55e1892da840 2023-07-14T16:25:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f6529ffb700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f651c0a1150 0x7f651c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f6529ffb700 1 -- 172.21.15.62:0/1630152013 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65240cc160 con 0x7f652c101680 2023-07-14T16:25:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f652bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892da300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f652bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892da300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34130/0 (socket says 172.21.15.62:34130) 2023-07-14T16:25:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f652bfff700 1 -- 172.21.15.62:0/1461103902 learned_addr learned my addr 172.21.15.62:0/1461103902 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f652bfff700 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892db6e0 0x55e1892dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f653099b700 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892da840 0x55e1892db1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.912+0000 7f653119c700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f651c0a1150 0x7f651c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f652bfff700 1 -- 172.21.15.62:0/1461103902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892da840 msgr2=0x55e1892db1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f652bfff700 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892da840 0x55e1892db1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f652bfff700 1 -- 172.21.15.62:0/1461103902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892db6e0 msgr2=0x55e1892dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f652bfff700 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892db6e0 0x55e1892dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f652bfff700 1 -- 172.21.15.62:0/1461103902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e189059d80 con 0x55e1891366b0 2023-07-14T16:25:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f653119c700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f651c0a1150 0x7f651c0a3600 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6524007fd0 tx=0x7f652400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f652bfff700 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892da300 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f65180118f0 tx=0x7f6518009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9118e921b083728 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f6516ffd700 1 -- 172.21.15.62:0/1461103902 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651801f440 con 0x55e1891366b0 2023-07-14T16:25:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f6516ffd700 1 -- 172.21.15.62:0/1461103902 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f651801fa30 con 0x55e1891366b0 2023-07-14T16:25:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.913+0000 7f6516ffd700 1 -- 172.21.15.62:0/1461103902 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800e6f0 con 0x55e1891366b0 2023-07-14T16:25:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.914+0000 7f6546487b80 1 -- 172.21.15.62:0/1461103902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 msgr2=0x55e1892da300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.914+0000 7f6546487b80 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892da300 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f65180118f0 tx=0x7f6518009850 comp rx=0 tx=0).stop 2023-07-14T16:25:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.914+0000 7f6546487b80 1 -- 172.21.15.62:0/1461103902 shutdown_connections 2023-07-14T16:25:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.914+0000 7f6546487b80 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892db6e0 0x55e1892dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.914+0000 7f6546487b80 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892da300 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.914+0000 7f6546487b80 1 --2- 172.21.15.62:0/1461103902 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892da840 0x55e1892db1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.914+0000 7f6546487b80 1 -- 172.21.15.62:0/1461103902 >> 172.21.15.62:0/1461103902 conn(0x55e1892cfbe0 msgr2=0x55e1892d4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.915+0000 7f6546487b80 1 -- 172.21.15.62:0/1461103902 shutdown_connections 2023-07-14T16:25:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.915+0000 7f6546487b80 1 -- 172.21.15.62:0/1461103902 wait complete. 2023-07-14T16:25:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.915+0000 7f6546487b80 1 -- start start 2023-07-14T16:25:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.915+0000 7f6546487b80 1 Processor -- start 2023-07-14T16:25:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f6546487b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892e3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f6546487b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f6546487b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 0x55e1892eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f6546487b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1891cba60 con 0x55e1891366b0 2023-07-14T16:25:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f6546487b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1891cb760 con 0x55e1892da840 2023-07-14T16:25:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f6546487b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1890af8c0 con 0x55e1892db6e0 2023-07-14T16:25:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f6546487b80 10 client.0 _ll_register_callbacks cb 0x55e1891c9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f652bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f652bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892e3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34140/0 (socket says 172.21.15.62:34140) 2023-07-14T16:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f652bfff700 1 -- 172.21.15.62:0/3956479036 learned_addr learned my addr 172.21.15.62:0/3956479036 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.916+0000 7f653099b700 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f652bfff700 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 0x55e1892eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f652bfff700 1 -- 172.21.15.62:0/3956479036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 msgr2=0x55e1892eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f652bfff700 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 0x55e1892eadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f652bfff700 1 -- 172.21.15.62:0/3956479036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 msgr2=0x55e1892e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f652bfff700 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f652bfff700 1 -- 172.21.15.62:0/3956479036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6518019040 con 0x55e1891366b0 2023-07-14T16:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f652bfff700 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892e3850 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f651801ff80 tx=0x7f6518005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88b76f19630b062 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f650bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f650bfff700 1 -- 172.21.15.62:0/3956479036 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518023030 con 0x55e1891366b0 2023-07-14T16:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f650bfff700 1 -- 172.21.15.62:0/3956479036 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6518024070 con 0x55e1891366b0 2023-07-14T16:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.917+0000 7f6546487b80 1 -- 172.21.15.62:0/3956479036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e189185490 con 0x55e1891366b0 2023-07-14T16:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f650bfff700 1 -- 172.21.15.62:0/3956479036 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6518026040 con 0x55e1891366b0 2023-07-14T16:25:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f6546487b80 10 client.18309 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f6546487b80 1 -- 172.21.15.62:0/3956479036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e189042c00 con 0x55e1891366b0 2023-07-14T16:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f650bfff700 1 -- 172.21.15.62:0/3956479036 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e189185490 con 0x55e1891366b0 2023-07-14T16:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f650bfff700 1 -- 172.21.15.62:0/3956479036 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6518003c80 con 0x55e1891366b0 2023-07-14T16:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f6546487b80 10 client.18309 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f6546487b80 10 client.18309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f6546487b80 20 client.18309 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.918+0000 7f6546487b80 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e1892ef440 0x55e1892f1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.919+0000 7f6546487b80 4 client.18309 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.919+0000 7f6546487b80 1 -- 172.21.15.62:0/3956479036 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e189224e50 con 0x55e1892ef440 2023-07-14T16:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.919+0000 7f653099b700 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e1892ef440 0x55e1892f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.920+0000 7f653099b700 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e1892ef440 0x55e1892f1850 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a630aae2f29dcc server_cookie=6f83b1fb20269e2f in_seq=0 out_seq=0 2023-07-14T16:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.920+0000 7f650bfff700 10 client.18309 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.920+0000 7f650bfff700 1 -- 172.21.15.62:0/3956479036 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e189224e50 con 0x55e1892ef440 2023-07-14T16:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.920+0000 7f650bfff700 10 client.18309 handle_command_reply: tid=0 2023-07-14T16:25:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.920+0000 7f6546487b80 1 client.18309 shutdown 2023-07-14T16:25:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 -- 172.21.15.62:0/3956479036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 msgr2=0x55e1892e3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892e3850 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f651801ff80 tx=0x7f6518005760 comp rx=0 tx=0).stop 2023-07-14T16:25:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 -- 172.21.15.62:0/3956479036 shutdown_connections 2023-07-14T16:25:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 0x55e1892eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1891366b0 0x55e1892e3850 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e1892ef440 0x55e1892f1850 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 --2- 172.21.15.62:0/3956479036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 -- 172.21.15.62:0/3956479036 >> 172.21.15.62:0/3956479036 conn(0x55e1892cfbe0 msgr2=0x55e1892d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f650bfff700 0 client.18309 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 -- 172.21.15.62:0/3956479036 shutdown_connections 2023-07-14T16:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 1 -- 172.21.15.62:0/3956479036 wait complete. 2023-07-14T16:25:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.921+0000 7f6546487b80 20 client.18309 trim_cache size 0 max 16384 2023-07-14T16:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f6546487b80 1 Processor -- start 2023-07-14T16:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f6546487b80 1 -- start start 2023-07-14T16:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f6546487b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1891366b0 0x55e1892e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f6546487b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892da840 0x55e1892ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f6546487b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 0x55e1892de630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f6546487b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1892a0060 con 0x55e1892da840 2023-07-14T16:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f6546487b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1892e2d00 con 0x55e1891366b0 2023-07-14T16:25:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f6546487b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1892d1d00 con 0x55e1892db6e0 2023-07-14T16:25:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f653099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892da840 0x55e1892ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f653099b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892da840 0x55e1892ead90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34148/0 (socket says 172.21.15.62:34148) 2023-07-14T16:25:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.941+0000 7f653099b700 1 -- 172.21.15.62:0/3671836032 learned_addr learned my addr 172.21.15.62:0/3671836032 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f652bfff700 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 0x55e1892de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f652bfff700 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1891366b0 0x55e1892e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f653099b700 1 -- 172.21.15.62:0/3671836032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 msgr2=0x55e1892de630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f653099b700 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 0x55e1892de630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f653099b700 1 -- 172.21.15.62:0/3671836032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1891366b0 msgr2=0x55e1892e1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f653099b700 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1891366b0 0x55e1892e1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f653099b700 1 -- 172.21.15.62:0/3671836032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1890801f0 con 0x55e1892da840 2023-07-14T16:25:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f652bfff700 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1891366b0 0x55e1892e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f653099b700 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892da840 0x55e1892ead90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f652c100b50 tx=0x7f6520009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85bf382d8343dc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f65177fe700 1 -- 172.21.15.62:0/3671836032 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200052c0 con 0x55e1892da840 2023-07-14T16:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f65177fe700 1 -- 172.21.15.62:0/3671836032 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f652001e040 con 0x55e1892da840 2023-07-14T16:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.942+0000 7f65177fe700 1 -- 172.21.15.62:0/3671836032 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000b910 con 0x55e1892da840 2023-07-14T16:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 -- 172.21.15.62:0/3671836032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892da840 msgr2=0x55e1892ead90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892da840 0x55e1892ead90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f652c100b50 tx=0x7f6520009310 comp rx=0 tx=0).stop 2023-07-14T16:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 -- 172.21.15.62:0/3671836032 shutdown_connections 2023-07-14T16:25:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1892db6e0 0x55e1892de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1891366b0 0x55e1892e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 --2- 172.21.15.62:0/3671836032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892da840 0x55e1892ead90 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 -- 172.21.15.62:0/3671836032 >> 172.21.15.62:0/3671836032 conn(0x55e1892cfbe0 msgr2=0x55e1892e24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 -- 172.21.15.62:0/3671836032 shutdown_connections 2023-07-14T16:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.943+0000 7f6546487b80 1 -- 172.21.15.62:0/3671836032 wait complete. 2023-07-14T16:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 1 -- start start 2023-07-14T16:25:44.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 1 Processor -- start 2023-07-14T16:25:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1891366b0 0x55e1892e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892db6e0 0x55e1892ea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1892ea5a0 con 0x55e1892db6e0 2023-07-14T16:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1892ea710 con 0x55e1892da840 2023-07-14T16:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1892ea8d0 con 0x55e1891366b0 2023-07-14T16:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.944+0000 7f6546487b80 10 client.0 _ll_register_callbacks cb 0x7f6518011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892db6e0 0x55e1892ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f653099b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1891366b0 0x55e1892e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f653099b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:33288/0 (socket says 172.21.15.62:33288) 2023-07-14T16:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1891366b0 0x55e1892e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:43680/0 (socket says 172.21.15.62:43680) 2023-07-14T16:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f653099b700 1 -- 172.21.15.62:0/2445257533 learned_addr learned my addr 172.21.15.62:0/2445257533 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 -- 172.21.15.62:0/2445257533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 msgr2=0x55e1892e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 -- 172.21.15.62:0/2445257533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892db6e0 msgr2=0x55e1892ea060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892db6e0 0x55e1892ea060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 -- 172.21.15.62:0/2445257533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c108870 con 0x55e1891366b0 2023-07-14T16:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f653099b700 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f652bfff700 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1891366b0 0x55e1892e6b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6518011fd0 tx=0x7f651801feb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ac8a31ec98ea02 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.945+0000 7f650bfff700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.946+0000 7f650bfff700 1 -- 172.21.15.62:0/2445257533 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518026030 con 0x55e1891366b0 2023-07-14T16:25:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.946+0000 7f6546487b80 1 -- 172.21.15.62:0/2445257533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f651801fa30 con 0x55e1891366b0 2023-07-14T16:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.946+0000 7f650bfff700 1 -- 172.21.15.62:0/2445257533 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6518005020 con 0x55e1891366b0 2023-07-14T16:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.946+0000 7f650bfff700 1 -- 172.21.15.62:0/2445257533 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6518023040 con 0x55e1891366b0 2023-07-14T16:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.946+0000 7f6546487b80 10 client.27823 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.946+0000 7f6546487b80 1 -- 172.21.15.62:0/2445257533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e189058250 con 0x55e1891366b0 2023-07-14T16:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f650bfff700 1 -- 172.21.15.62:0/2445257533 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f651801fa30 con 0x55e1891366b0 2023-07-14T16:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f650bfff700 1 -- 172.21.15.62:0/2445257533 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f651800ead0 con 0x55e1891366b0 2023-07-14T16:25:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f6546487b80 10 client.27823 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f6546487b80 10 client.27823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f6546487b80 20 client.27823 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f6546487b80 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e189343ed0 0x55e1893462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f6546487b80 4 client.27823 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f6546487b80 1 -- 172.21.15.62:0/2445257533 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e1891e6ff0 con 0x55e189343ed0 2023-07-14T16:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.947+0000 7f653099b700 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e189343ed0 0x55e1893462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.948+0000 7f653099b700 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e189343ed0 0x55e1893462e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43801d72921b666f server_cookie=22b17c3dd9e20929 in_seq=0 out_seq=0 2023-07-14T16:25:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.948+0000 7f650bfff700 10 client.27823 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f650bfff700 1 -- 172.21.15.62:0/2445257533 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e1891e6ff0 con 0x55e189343ed0 2023-07-14T16:25:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f650bfff700 10 client.27823 handle_command_reply: tid=0 2023-07-14T16:25:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 client.27823 shutdown 2023-07-14T16:25:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 -- 172.21.15.62:0/2445257533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1891366b0 msgr2=0x55e1892e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1891366b0 0x55e1892e6b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6518011fd0 tx=0x7f651801feb0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 -- 172.21.15.62:0/2445257533 shutdown_connections 2023-07-14T16:25:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1892db6e0 0x55e1892ea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e1891366b0 0x55e1892e6b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e189343ed0 0x55e1893462e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 --2- 172.21.15.62:0/2445257533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1892da840 0x55e1892e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 -- 172.21.15.62:0/2445257533 >> 172.21.15.62:0/2445257533 conn(0x55e1892cfbe0 msgr2=0x55e1892e1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f650bfff700 0 client.27823 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 -- 172.21.15.62:0/2445257533 shutdown_connections 2023-07-14T16:25:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.965+0000 7f6546487b80 1 -- 172.21.15.62:0/2445257533 wait complete. 2023-07-14T16:25:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.966+0000 7f6546487b80 20 client.27823 trim_cache size 0 max 16384 2023-07-14T16:25:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:25:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:25:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:25:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:25:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.966+0000 7f6532bff700 1 -- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f651c0a1150 msgr2=0x7f651c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.966+0000 7f6532bff700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f651c0a1150 0x7f651c0a3600 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6524007fd0 tx=0x7f652400b040 comp rx=0 tx=0).stop 2023-07-14T16:25:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.966+0000 7f6532bff700 1 -- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c101680 msgr2=0x7f652c198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.966+0000 7f6532bff700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c101680 0x7f652c198e80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6524007ee0 tx=0x7f652400d3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.967+0000 7f6532bff700 1 -- 172.21.15.62:0/1630152013 shutdown_connections 2023-07-14T16:25:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.967+0000 7f6532bff700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f652c101680 0x7f652c198e80 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.967+0000 7f6532bff700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f652c0ffb00 0x7f652c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.967+0000 7f6532bff700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f651c0a1150 0x7f651c0a3600 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.967+0000 7f6532bff700 1 --2- 172.21.15.62:0/1630152013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f652c100cf0 0x7f652c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.967+0000 7f6532bff700 1 -- 172.21.15.62:0/1630152013 >> 172.21.15.62:0/1630152013 conn(0x7f652c0fb0e0 msgr2=0x7f652c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.967+0000 7f6532bff700 1 -- 172.21.15.62:0/1630152013 shutdown_connections 2023-07-14T16:25:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:44.967+0000 7f6532bff700 1 -- 172.21.15.62:0/1630152013 wait complete. 2023-07-14T16:25:45.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:44 smithi062 ceph-mon[95543]: pgmap v660: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:25:45.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:44 smithi081 ceph-mon[127795]: pgmap v660: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:25:45.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:44 smithi164 ceph-mon[97427]: pgmap v660: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:46 smithi062 ceph-mon[95543]: pgmap v661: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:25:47.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:46 smithi081 ceph-mon[127795]: pgmap v661: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:25:47.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:46 smithi164 ceph-mon[97427]: pgmap v661: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:25:49.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:48 smithi062 ceph-mon[95543]: pgmap v662: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:25:49.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:48 smithi081 ceph-mon[127795]: pgmap v662: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:25:49.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:48 smithi164 ceph-mon[97427]: pgmap v662: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:25:50.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:25:51.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:50 smithi062 ceph-mon[95543]: pgmap v663: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:25:51.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 -- 172.21.15.62:0/922458784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c072f70 msgr2=0x7f4c1c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 --2- 172.21.15.62:0/922458784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c072f70 0x7f4c1c0733e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009990 tx=0x7f4c0c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 -- 172.21.15.62:0/922458784 shutdown_connections 2023-07-14T16:25:51.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 --2- 172.21.15.62:0/922458784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c1c10c0c0 0x7f4c1c10e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 --2- 172.21.15.62:0/922458784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1c074550 0x7f4c1c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 --2- 172.21.15.62:0/922458784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c072f70 0x7f4c1c0733e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 -- 172.21.15.62:0/922458784 >> 172.21.15.62:0/922458784 conn(0x7f4c1c0fb130 msgr2=0x7f4c1c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:51.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 -- 172.21.15.62:0/922458784 shutdown_connections 2023-07-14T16:25:51.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.148+0000 7f4c235ff700 1 -- 172.21.15.62:0/922458784 wait complete. 2023-07-14T16:25:51.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c235ff700 1 Processor -- start 2023-07-14T16:25:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c235ff700 1 -- start start 2023-07-14T16:25:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c235ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c1c072f70 0x7f4c1c19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c235ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1c074550 0x7f4c1c19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c235ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c10c0c0 0x7f4c1c1a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c235ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1a3990 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c235ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1a3ad0 con 0x7f4c1c072f70 2023-07-14T16:25:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c235ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1a3c60 con 0x7f4c1c074550 2023-07-14T16:25:51.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.150+0000 7f4c21b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c10c0c0 0x7f4c1c1a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c21b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c10c0c0 0x7f4c1c1a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34172/0 (socket says 172.21.15.62:34172) 2023-07-14T16:25:51.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c21b9c700 1 -- 172.21.15.62:0/1923094505 learned_addr learned my addr 172.21.15.62:0/1923094505 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:51.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c2139b700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c1c072f70 0x7f4c1c19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1c074550 0x7f4c1c19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c21b9c700 1 -- 172.21.15.62:0/1923094505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1c074550 msgr2=0x7f4c1c19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1c074550 0x7f4c1c19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c21b9c700 1 -- 172.21.15.62:0/1923094505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c1c072f70 msgr2=0x7f4c1c19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c1c072f70 0x7f4c1c19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c21b9c700 1 -- 172.21.15.62:0/1923094505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c0095f0 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c10c0c0 0x7f4c1c1a33c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007ee0 tx=0x7f4c1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.151+0000 7f4c127fc700 1 -- 172.21.15.62:0/1923094505 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400dca0 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.152+0000 7f4c127fc700 1 -- 172.21.15.62:0/1923094505 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c1400f040 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.152+0000 7f4c235ff700 1 -- 172.21.15.62:0/1923094505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c1a3f40 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.152+0000 7f4c127fc700 1 -- 172.21.15.62:0/1923094505 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14004710 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.152+0000 7f4c235ff700 1 -- 172.21.15.62:0/1923094505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c1a4370 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.155+0000 7f4c127fc700 1 -- 172.21.15.62:0/1923094505 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4c14003a00 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.156+0000 7f4c127fc700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c080a11a0 0x7f4c080a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.157+0000 7f4c127fc700 1 -- 172.21.15.62:0/1923094505 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c140cc080 con 0x7f4c1c10c0c0 2023-07-14T16:25:51.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.157+0000 7f4c2139b700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c080a11a0 0x7f4c080a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.157+0000 7f4c36f30b80 1 Processor -- start 2023-07-14T16:25:51.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c2139b700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c080a11a0 0x7f4c080a3650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c074920 tx=0x7f4c180073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:51.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c36f30b80 1 -- start start 2023-07-14T16:25:51.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d3d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3de20 0x55c1e7d3e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d3f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7c42cb0 con 0x55c1e7d3de20 2023-07-14T16:25:51.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7c95a40 con 0x55c1e7d3ec70 2023-07-14T16:25:51.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7c8eed0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c20b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3de20 0x55c1e7d3e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c20b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3de20 0x55c1e7d3e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34184/0 (socket says 172.21.15.62:34184) 2023-07-14T16:25:51.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c20b9a700 1 -- 172.21.15.62:0/95613572 learned_addr learned my addr 172.21.15.62:0/95613572 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:51.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.158+0000 7f4c2139b700 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d3d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.159+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d3f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.159+0000 7f4c20b9a700 1 -- 172.21.15.62:0/95613572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3d4d0 msgr2=0x55c1e7d3d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:51.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.159+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d3d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.159+0000 7f4c20b9a700 1 -- 172.21.15.62:0/95613572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3ec70 msgr2=0x55c1e7d3f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:51.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.159+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d3f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.159+0000 7f4c20b9a700 1 -- 172.21.15.62:0/95613572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1e7abcde0 con 0x55c1e7d3de20 2023-07-14T16:25:51.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.159+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3de20 0x55c1e7d3e6e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c006010 tx=0x7f4c0c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7609d6bca7bc494f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:51.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.159+0000 7f4c077fe700 1 -- 172.21.15.62:0/95613572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c01d070 con 0x55c1e7d3de20 2023-07-14T16:25:51.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c077fe700 1 -- 172.21.15.62:0/95613572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c0c01e040 con 0x55c1e7d3de20 2023-07-14T16:25:51.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c077fe700 1 -- 172.21.15.62:0/95613572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c004500 con 0x55c1e7d3de20 2023-07-14T16:25:51.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c36f30b80 1 -- 172.21.15.62:0/95613572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3de20 msgr2=0x55c1e7d3e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3de20 0x55c1e7d3e6e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c006010 tx=0x7f4c0c005bc0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c36f30b80 1 -- 172.21.15.62:0/95613572 shutdown_connections 2023-07-14T16:25:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d3f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d3d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/95613572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3de20 0x55c1e7d3e6e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c36f30b80 1 -- 172.21.15.62:0/95613572 >> 172.21.15.62:0/95613572 conn(0x55c1e7d32da0 msgr2=0x55c1e7d381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.160+0000 7f4c36f30b80 1 -- 172.21.15.62:0/95613572 shutdown_connections 2023-07-14T16:25:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.161+0000 7f4c36f30b80 1 -- 172.21.15.62:0/95613572 wait complete. 2023-07-14T16:25:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.161+0000 7f4c36f30b80 1 -- start start 2023-07-14T16:25:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.161+0000 7f4c36f30b80 1 Processor -- start 2023-07-14T16:25:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d47630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7c2eac0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7c2e7c0 con 0x55c1e7d3de20 2023-07-14T16:25:51.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7b12920 con 0x55c1e7d3ec70 2023-07-14T16:25:51.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c36f30b80 10 client.0 _ll_register_callbacks cb 0x55c1e7c2c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:51.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c2139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c2139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34188/0 (socket says 172.21.15.62:34188) 2023-07-14T16:25:51.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c2139b700 1 -- 172.21.15.62:0/2933607339 learned_addr learned my addr 172.21.15.62:0/2933607339 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:51.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c2139b700 1 -- 172.21.15.62:0/2933607339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 msgr2=0x55c1e7d4e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:51.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c2139b700 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c2139b700 1 -- 172.21.15.62:0/2933607339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 msgr2=0x55c1e7d47630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:51.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c2139b700 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d47630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.162+0000 7f4c2139b700 1 -- 172.21.15.62:0/2933607339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c10ee50 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.163+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:51.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.163+0000 7f4c2139b700 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d470f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4c180116f0 tx=0x7f4c180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaadcd5ac666d81 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.163+0000 7f4c04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.163+0000 7f4c04ff9700 1 -- 172.21.15.62:0/2933607339 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18004250 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.163+0000 7f4c04ff9700 1 -- 172.21.15.62:0/2933607339 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c18017070 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.163+0000 7f4c36f30b80 1 -- 172.21.15.62:0/2933607339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1e7be84f0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.163+0000 7f4c04ff9700 1 -- 172.21.15.62:0/2933607339 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c18010ec0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.163+0000 7f4c36f30b80 10 client.18327 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c36f30b80 1 -- 172.21.15.62:0/2933607339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c1e7aa5c60 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c04ff9700 1 -- 172.21.15.62:0/2933607339 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c1e7be84f0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c04ff9700 1 -- 172.21.15.62:0/2933607339 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4c18031070 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c36f30b80 10 client.18327 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c36f30b80 10 client.18327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c36f30b80 20 client.18327 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c1e7d53020 0x55c1e7d55430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c36f30b80 4 client.18327 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.164+0000 7f4c36f30b80 1 -- 172.21.15.62:0/2933607339 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1e7c87eb0 con 0x55c1e7d53020 2023-07-14T16:25:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.165+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c1e7d53020 0x55c1e7d55430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.165+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c1e7d53020 0x55c1e7d55430 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6591e683054b9e2 server_cookie=c2d0b4bc1cd3caaa in_seq=0 out_seq=0 2023-07-14T16:25:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.165+0000 7f4c04ff9700 10 client.18327 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.166+0000 7f4c04ff9700 1 -- 172.21.15.62:0/2933607339 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c1e7c87eb0 con 0x55c1e7d53020 2023-07-14T16:25:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.166+0000 7f4c04ff9700 10 client.18327 handle_command_reply: tid=0 2023-07-14T16:25:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.166+0000 7f4c36f30b80 1 client.18327 shutdown 2023-07-14T16:25:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 -- 172.21.15.62:0/2933607339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 msgr2=0x55c1e7d470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d470f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4c180116f0 tx=0x7f4c180047c0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 -- 172.21.15.62:0/2933607339 shutdown_connections 2023-07-14T16:25:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d470f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c1e7d53020 0x55c1e7d55430 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/2933607339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 -- 172.21.15.62:0/2933607339 >> 172.21.15.62:0/2933607339 conn(0x55c1e7d32da0 msgr2=0x55c1e7d378b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c04ff9700 0 client.18327 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 -- 172.21.15.62:0/2933607339 shutdown_connections 2023-07-14T16:25:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 1 -- 172.21.15.62:0/2933607339 wait complete. 2023-07-14T16:25:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.167+0000 7f4c36f30b80 20 client.18327 trim_cache size 0 max 16384 2023-07-14T16:25:51.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:50 smithi081 ceph-mon[127795]: pgmap v663: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:25:51.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c36f30b80 1 Processor -- start 2023-07-14T16:25:51.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c36f30b80 1 -- start start 2023-07-14T16:25:51.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d45e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7d34ec0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7d394c0 con 0x55c1e7d3de20 2023-07-14T16:25:51.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7d3e1f0 con 0x55c1e7d3ec70 2023-07-14T16:25:51.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c2139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d45e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c2139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d45e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34204/0 (socket says 172.21.15.62:34204) 2023-07-14T16:25:51.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.194+0000 7f4c2139b700 1 -- 172.21.15.62:0/3197869260 learned_addr learned my addr 172.21.15.62:0/3197869260 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:51.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c2139b700 1 -- 172.21.15.62:0/3197869260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 msgr2=0x55c1e7d450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:51.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c2139b700 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c2139b700 1 -- 172.21.15.62:0/3197869260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 msgr2=0x55c1e7d4e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c2139b700 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d4e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c2139b700 1 -- 172.21.15.62:0/3197869260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1e7ae3250 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c2139b700 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d45e20 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005b80 tx=0x7f4c18005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2a222f7db0ebe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.195+0000 7f4c07fff700 1 -- 172.21.15.62:0/3197869260 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18004240 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c07fff700 1 -- 172.21.15.62:0/3197869260 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c1802e070 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c07fff700 1 -- 172.21.15.62:0/3197869260 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1801fea0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3197869260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 msgr2=0x55c1e7d45e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d45e20 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005b80 tx=0x7f4c18005bb0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3197869260 shutdown_connections 2023-07-14T16:25:51.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d45e20 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3197869260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d4e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.196+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3197869260 >> 172.21.15.62:0/3197869260 conn(0x55c1e7d32da0 msgr2=0x55c1e7d59d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:51.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.197+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3197869260 shutdown_connections 2023-07-14T16:25:51.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.197+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3197869260 wait complete. 2023-07-14T16:25:51.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.197+0000 7f4c36f30b80 1 -- start start 2023-07-14T16:25:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.197+0000 7f4c36f30b80 1 Processor -- start 2023-07-14T16:25:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d4e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d4e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c36f30b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7d4b1f0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7d4b360 con 0x55c1e7d3de20 2023-07-14T16:25:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c36f30b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e7d4b520 con 0x55c1e7d3ec70 2023-07-14T16:25:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c36f30b80 10 client.0 _ll_register_callbacks cb 0x55c1e7c2c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c2139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d4e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c2139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d4e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34208/0 (socket says 172.21.15.62:34208) 2023-07-14T16:25:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.198+0000 7f4c2139b700 1 -- 172.21.15.62:0/3717771157 learned_addr learned my addr 172.21.15.62:0/3717771157 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d4e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c2139b700 1 -- 172.21.15.62:0/3717771157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 msgr2=0x55c1e7d4aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c2139b700 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c2139b700 1 -- 172.21.15.62:0/3717771157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 msgr2=0x55c1e7d4e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:51.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c2139b700 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d4e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c2139b700 1 -- 172.21.15.62:0/3717771157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1e7aa5c60 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c21b9c700 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:51.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c2139b700 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d4e000 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4c18010c50 tx=0x7f4c18005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f929b7ac5f196a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:51.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:51.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.199+0000 7f4c04ff9700 1 -- 172.21.15.62:0/3717771157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18005300 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c04ff9700 1 -- 172.21.15.62:0/3717771157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c18017070 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3717771157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c0c01e040 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c04ff9700 1 -- 172.21.15.62:0/3717771157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c18005460 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 10 client.18333 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3717771157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c1e7abb2b0 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c04ff9700 1 -- 172.21.15.62:0/3717771157 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c0c01e040 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c04ff9700 1 -- 172.21.15.62:0/3717771157 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4c1804f070 con 0x55c1e7d3d4d0 2023-07-14T16:25:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 10 client.18333 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:51.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 20 client.18333 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:51.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c1e7da81b0 0x55c1e7daa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 4 client.18333 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:25:51.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.200+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3717771157 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1e7c4a050 con 0x55c1e7da81b0 2023-07-14T16:25:51.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.201+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c1e7da81b0 0x55c1e7daa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.201+0000 7f4c20b9a700 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c1e7da81b0 0x55c1e7daa5c0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6cb54b7634dbde server_cookie=c585f899a6f3c3f8 in_seq=0 out_seq=0 2023-07-14T16:25:51.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.201+0000 7f4c04ff9700 10 client.18333 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c04ff9700 1 -- 172.21.15.62:0/3717771157 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c1e7c4a050 con 0x55c1e7da81b0 2023-07-14T16:25:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c04ff9700 10 client.18333 handle_command_reply: tid=0 2023-07-14T16:25:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 client.18333 shutdown 2023-07-14T16:25:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3717771157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 msgr2=0x55c1e7d4e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d4e000 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4c18010c50 tx=0x7f4c18005f60 comp rx=0 tx=0).stop 2023-07-14T16:25:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3717771157 shutdown_connections 2023-07-14T16:25:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1e7d3ec70 0x55c1e7d4aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c1e7d3d4d0 0x55c1e7d4e000 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c1e7da81b0 0x55c1e7daa5c0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 --2- 172.21.15.62:0/3717771157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1e7d3de20 0x55c1e7d4e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3717771157 >> 172.21.15.62:0/3717771157 conn(0x55c1e7d32da0 msgr2=0x55c1e7d45e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:51.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c04ff9700 0 client.18333 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:51.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3717771157 shutdown_connections 2023-07-14T16:25:51.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 1 -- 172.21.15.62:0/3717771157 wait complete. 2023-07-14T16:25:51.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.202+0000 7f4c36f30b80 20 client.18333 trim_cache size 0 max 16384 2023-07-14T16:25:51.185 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:25:51.185 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:25:51.185 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:25:51.186 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:25:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 -- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c080a11a0 msgr2=0x7f4c080a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c080a11a0 0x7f4c080a3650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c074920 tx=0x7f4c180073d0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 -- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c10c0c0 msgr2=0x7f4c1c1a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c10c0c0 0x7f4c1c1a33c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007ee0 tx=0x7f4c1400d3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 -- 172.21.15.62:0/1923094505 shutdown_connections 2023-07-14T16:25:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c1c10c0c0 0x7f4c1c1a33c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4c080a11a0 0x7f4c080a3650 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c1c072f70 0x7f4c1c19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 --2- 172.21.15.62:0/1923094505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1c074550 0x7f4c1c19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 -- 172.21.15.62:0/1923094505 >> 172.21.15.62:0/1923094505 conn(0x7f4c1c0fb130 msgr2=0x7f4c1c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 -- 172.21.15.62:0/1923094505 shutdown_connections 2023-07-14T16:25:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.203+0000 7f4c235ff700 1 -- 172.21.15.62:0/1923094505 wait complete. 2023-07-14T16:25:51.198 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:25:51.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:25:51.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:50 smithi164 ceph-mon[97427]: pgmap v663: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:25:51.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 -- 172.21.15.62:0/2798533191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 msgr2=0x7f40800fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 --2- 172.21.15.62:0/2798533191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 0x7f40800fff10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4070009990 tx=0x7f4070009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 -- 172.21.15.62:0/2798533191 shutdown_connections 2023-07-14T16:25:51.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 --2- 172.21.15.62:0/2798533191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4080101680 0x7f4080107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 --2- 172.21.15.62:0/2798533191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 0x7f40800fff10 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 --2- 172.21.15.62:0/2798533191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4080100cf0 0x7f4080101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 -- 172.21.15.62:0/2798533191 >> 172.21.15.62:0/2798533191 conn(0x7f40800fb0e0 msgr2=0x7f40800fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:51.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 -- 172.21.15.62:0/2798533191 shutdown_connections 2023-07-14T16:25:51.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.466+0000 7f40867ff700 1 -- 172.21.15.62:0/2798533191 wait complete. 2023-07-14T16:25:51.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.467+0000 7f40867ff700 1 Processor -- start 2023-07-14T16:25:51.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f40867ff700 1 -- start start 2023-07-14T16:25:51.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f40867ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 0x7f40800742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f40867ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4080100cf0 0x7f4080072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f40867ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4080101680 0x7f4080072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f40867ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40800735e0 con 0x7f40800ffb00 2023-07-14T16:25:51.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f40867ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4080073750 con 0x7f4080100cf0 2023-07-14T16:25:51.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f40867ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408019f7d0 con 0x7f4080101680 2023-07-14T16:25:51.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f407ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 0x7f40800742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f407ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 0x7f40800742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34228/0 (socket says 172.21.15.62:34228) 2023-07-14T16:25:51.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f407ffff700 1 -- 172.21.15.62:0/1939169661 learned_addr learned my addr 172.21.15.62:0/1939169661 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:51.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f407f7fe700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4080100cf0 0x7f4080072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.468+0000 7f4084d9c700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4080101680 0x7f4080072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407ffff700 1 -- 172.21.15.62:0/1939169661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4080101680 msgr2=0x7f4080072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407ffff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4080101680 0x7f4080072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407ffff700 1 -- 172.21.15.62:0/1939169661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4080100cf0 msgr2=0x7f4080072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407ffff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4080100cf0 0x7f4080072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407ffff700 1 -- 172.21.15.62:0/1939169661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40700095f0 con 0x7f40800ffb00 2023-07-14T16:25:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407ffff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 0x7f40800742c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f407000b5c0 tx=0x7f4070005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:51.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407d7fa700 1 -- 172.21.15.62:0/1939169661 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407001d070 con 0x7f40800ffb00 2023-07-14T16:25:51.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407d7fa700 1 -- 172.21.15.62:0/1939169661 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4070003ec0 con 0x7f40800ffb00 2023-07-14T16:25:51.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408019fa50 con 0x7f40800ffb00 2023-07-14T16:25:51.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.469+0000 7f407d7fa700 1 -- 172.21.15.62:0/1939169661 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070022be0 con 0x7f40800ffb00 2023-07-14T16:25:51.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.470+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408019fe70 con 0x7f40800ffb00 2023-07-14T16:25:51.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.471+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408010b1d0 con 0x7f40800ffb00 2023-07-14T16:25:51.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.473+0000 7f407d7fa700 1 -- 172.21.15.62:0/1939169661 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4070004030 con 0x7f40800ffb00 2023-07-14T16:25:51.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.475+0000 7f407d7fa700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40680a11a0 0x7f40680a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:51.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.475+0000 7f407d7fa700 1 -- 172.21.15.62:0/1939169661 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40700cdf70 con 0x7f40800ffb00 2023-07-14T16:25:51.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.475+0000 7f407f7fe700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40680a11a0 0x7f40680a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:51.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.476+0000 7f407f7fe700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40680a11a0 0x7f40680a3650 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4074005950 tx=0x7f407400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:51.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.480+0000 7f407d7fa700 1 -- 172.21.15.62:0/1939169661 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4070097760 con 0x7f40800ffb00 2023-07-14T16:25:51.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.824+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4080106a70 con 0x7f40800ffb00 2023-07-14T16:25:51.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.826+0000 7f407d7fa700 1 -- 172.21.15.62:0/1939169661 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f407009ad80 con 0x7f40800ffb00 2023-07-14T16:25:51.801 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:25:51.802 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:25:51.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40680a11a0 msgr2=0x7f40680a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40680a11a0 0x7f40680a3650 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4074005950 tx=0x7f407400b410 comp rx=0 tx=0).stop 2023-07-14T16:25:51.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 msgr2=0x7f40800742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:51.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 0x7f40800742c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f407000b5c0 tx=0x7f4070005740 comp rx=0 tx=0).stop 2023-07-14T16:25:51.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 shutdown_connections 2023-07-14T16:25:51.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4080101680 0x7f4080072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40800ffb00 0x7f40800742c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40680a11a0 0x7f40680a3650 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 --2- 172.21.15.62:0/1939169661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4080100cf0 0x7f4080072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:51.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 >> 172.21.15.62:0/1939169661 conn(0x7f40800fb0e0 msgr2=0x7f40800fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:51.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 shutdown_connections 2023-07-14T16:25:51.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:51.829+0000 7f40867ff700 1 -- 172.21.15.62:0/1939169661 wait complete. 2023-07-14T16:25:51.808 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:25:51.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:25:52.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 -- 172.21.15.62:0/632578959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 msgr2=0x7f5254107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 --2- 172.21.15.62:0/632578959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 0x7f5254107f20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f524c00b0a0 tx=0x7f524c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 -- 172.21.15.62:0/632578959 shutdown_connections 2023-07-14T16:25:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 --2- 172.21.15.62:0/632578959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 0x7f5254107f20 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 --2- 172.21.15.62:0/632578959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52540ffae0 0x7f52540ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 --2- 172.21.15.62:0/632578959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5254100cd0 0x7f5254101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 -- 172.21.15.62:0/632578959 >> 172.21.15.62:0/632578959 conn(0x7f52540fb0e0 msgr2=0x7f52540fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 -- 172.21.15.62:0/632578959 shutdown_connections 2023-07-14T16:25:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.090+0000 7f525b7ff700 1 -- 172.21.15.62:0/632578959 wait complete. 2023-07-14T16:25:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.091+0000 7f525b7ff700 1 Processor -- start 2023-07-14T16:25:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.091+0000 7f525b7ff700 1 -- start start 2023-07-14T16:25:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f525b7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52540ffae0 0x7f52540742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f525b7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5254100cd0 0x7f5254072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f525b7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 0x7f5254072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f525b7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540735e0 con 0x7f5254101660 2023-07-14T16:25:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f525b7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254073750 con 0x7f52540ffae0 2023-07-14T16:25:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f525b7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525419f940 con 0x7f5254100cd0 2023-07-14T16:25:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f5259d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 0x7f5254072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f5259d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 0x7f5254072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34246/0 (socket says 172.21.15.62:34246) 2023-07-14T16:25:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f5259d9c700 1 -- 172.21.15.62:0/1230380348 learned_addr learned my addr 172.21.15.62:0/1230380348 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f5258d9a700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5254100cd0 0x7f5254072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.092+0000 7f525959b700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52540ffae0 0x7f52540742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f5259d9c700 1 -- 172.21.15.62:0/1230380348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5254100cd0 msgr2=0x7f5254072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f5259d9c700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5254100cd0 0x7f5254072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f5259d9c700 1 -- 172.21.15.62:0/1230380348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52540ffae0 msgr2=0x7f52540742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f5259d9c700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52540ffae0 0x7f52540742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f5259d9c700 1 -- 172.21.15.62:0/1230380348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c009d00 con 0x7f5254101660 2023-07-14T16:25:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f5259d9c700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 0x7f5254072e40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f524c009fd0 tx=0x7f524c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f524a7fc700 1 -- 172.21.15.62:0/1230380348 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00e040 con 0x7f5254101660 2023-07-14T16:25:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f524a7fc700 1 -- 172.21.15.62:0/1230380348 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f524c007760 con 0x7f5254101660 2023-07-14T16:25:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f525b7ff700 1 -- 172.21.15.62:0/1230380348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525419fbc0 con 0x7f5254101660 2023-07-14T16:25:52.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f524a7fc700 1 -- 172.21.15.62:0/1230380348 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c01b6c0 con 0x7f5254101660 2023-07-14T16:25:52.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.093+0000 7f525b7ff700 1 -- 172.21.15.62:0/1230380348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525419ffe0 con 0x7f5254101660 2023-07-14T16:25:52.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.097+0000 7f524a7fc700 1 -- 172.21.15.62:0/1230380348 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f524c01d470 con 0x7f5254101660 2023-07-14T16:25:52.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.099+0000 7f526f063b80 1 Processor -- start 2023-07-14T16:25:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.099+0000 7f526f063b80 1 -- start start 2023-07-14T16:25:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.099+0000 7f524a7fc700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f52400a11d0 0x7f52400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.099+0000 7f526f063b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 0x55c2bf73c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f524a7fc700 1 -- 172.21.15.62:0/1230380348 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f524c0cdbe0 con 0x7f5254101660 2023-07-14T16:25:52.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f526f063b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf73d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f526f063b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73dc70 0x55c2bf73e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f526f063b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf641cb0 con 0x55c2bf73ce20 2023-07-14T16:25:52.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f526f063b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf694a40 con 0x55c2bf73dc70 2023-07-14T16:25:52.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f526f063b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf68ded0 con 0x55c2bf73c4d0 2023-07-14T16:25:52.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f5258d9a700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f52400a11d0 0x7f52400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f5258d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf73d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f5258d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf73d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34260/0 (socket says 172.21.15.62:34260) 2023-07-14T16:25:52.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f5258d9a700 1 -- 172.21.15.62:0/282509587 learned_addr learned my addr 172.21.15.62:0/282509587 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f5259d9c700 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73dc70 0x55c2bf73e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.100+0000 7f525959b700 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 0x55c2bf73c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f5259d9c700 1 -- 172.21.15.62:0/282509587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 msgr2=0x55c2bf73c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f5259d9c700 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 0x55c2bf73c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f5259d9c700 1 -- 172.21.15.62:0/282509587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 msgr2=0x55c2bf73d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f5259d9c700 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf73d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f5259d9c700 1 -- 172.21.15.62:0/282509587 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2bf4bbde0 con 0x55c2bf73dc70 2023-07-14T16:25:52.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f5258d9a700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f52400a11d0 0x7f52400a3680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5244013a70 tx=0x7f5244009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f5259d9c700 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73dc70 0x55c2bf73e0e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f524c097530 tx=0x7f524c003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee47afb00fa39f6d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f523f7fe700 1 -- 172.21.15.62:0/282509587 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c01a3c0 con 0x55c2bf73dc70 2023-07-14T16:25:52.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f523f7fe700 1 -- 172.21.15.62:0/282509587 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f524c09b030 con 0x55c2bf73dc70 2023-07-14T16:25:52.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.101+0000 7f523f7fe700 1 -- 172.21.15.62:0/282509587 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c09b4f0 con 0x55c2bf73dc70 2023-07-14T16:25:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 -- 172.21.15.62:0/282509587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73dc70 msgr2=0x55c2bf73e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73dc70 0x55c2bf73e0e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f524c097530 tx=0x7f524c003f10 comp rx=0 tx=0).stop 2023-07-14T16:25:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 -- 172.21.15.62:0/282509587 shutdown_connections 2023-07-14T16:25:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73dc70 0x55c2bf73e0e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 0x55c2bf73c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 --2- 172.21.15.62:0/282509587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf73d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 -- 172.21.15.62:0/282509587 >> 172.21.15.62:0/282509587 conn(0x55c2bf731da0 msgr2=0x55c2bf7371d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 -- 172.21.15.62:0/282509587 shutdown_connections 2023-07-14T16:25:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.102+0000 7f526f063b80 1 -- 172.21.15.62:0/282509587 wait complete. 2023-07-14T16:25:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.103+0000 7f526f063b80 1 -- start start 2023-07-14T16:25:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.103+0000 7f526f063b80 1 Processor -- start 2023-07-14T16:25:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f526f063b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73c4d0 0x55c2bf745ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f526f063b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 0x55c2bf746400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f526f063b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 0x55c2bf74d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f526f063b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf62dac0 con 0x55c2bf73c4d0 2023-07-14T16:25:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f526f063b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf62d7c0 con 0x55c2bf73ce20 2023-07-14T16:25:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f526f063b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf511920 con 0x55c2bf73dc70 2023-07-14T16:25:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f526f063b80 10 client.0 _ll_register_callbacks cb 0x55c2bf62b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f525959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73c4d0 0x55c2bf745ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f525959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73c4d0 0x55c2bf745ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34276/0 (socket says 172.21.15.62:34276) 2023-07-14T16:25:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f525959b700 1 -- 172.21.15.62:0/3763927774 learned_addr learned my addr 172.21.15.62:0/3763927774 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f5258d9a700 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 0x55c2bf746400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f5259d9c700 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 0x55c2bf74d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f525959b700 1 -- 172.21.15.62:0/3763927774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 msgr2=0x55c2bf74d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f525959b700 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 0x55c2bf74d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f525959b700 1 -- 172.21.15.62:0/3763927774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 msgr2=0x55c2bf746400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f525959b700 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 0x55c2bf746400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.104+0000 7f525959b700 1 -- 172.21.15.62:0/3763927774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525419ffe0 con 0x55c2bf73c4d0 2023-07-14T16:25:52.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.105+0000 7f525959b700 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73c4d0 0x55c2bf745ec0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f52500114c0 tx=0x7f52500117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643fc7481d6b21e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.105+0000 7f523cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:52.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.105+0000 7f523cff9700 1 -- 172.21.15.62:0/3763927774 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250016070 con 0x55c2bf73c4d0 2023-07-14T16:25:52.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.105+0000 7f526f063b80 1 -- 172.21.15.62:0/3763927774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2bf5e74f0 con 0x55c2bf73c4d0 2023-07-14T16:25:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.105+0000 7f523cff9700 1 -- 172.21.15.62:0/3763927774 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5250004020 con 0x55c2bf73c4d0 2023-07-14T16:25:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.105+0000 7f523cff9700 1 -- 172.21.15.62:0/3763927774 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5250004220 con 0x55c2bf73c4d0 2023-07-14T16:25:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.105+0000 7f526f063b80 10 client.18351 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.105+0000 7f526f063b80 1 -- 172.21.15.62:0/3763927774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c2bf4a4c60 con 0x55c2bf73c4d0 2023-07-14T16:25:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.106+0000 7f523cff9700 1 -- 172.21.15.62:0/3763927774 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c2bf5e74f0 con 0x55c2bf73c4d0 2023-07-14T16:25:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.106+0000 7f523cff9700 1 -- 172.21.15.62:0/3763927774 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f52500106d0 con 0x55c2bf73c4d0 2023-07-14T16:25:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.106+0000 7f526f063b80 10 client.18351 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.106+0000 7f526f063b80 10 client.18351 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:25:52.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.106+0000 7f526f063b80 20 client.18351 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:52.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.106+0000 7f526f063b80 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c2bf751c90 0x55c2bf7540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.106+0000 7f526f063b80 4 client.18351 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:52.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.106+0000 7f526f063b80 1 -- 172.21.15.62:0/3763927774 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2bf686eb0 con 0x55c2bf751c90 2023-07-14T16:25:52.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.107+0000 7f5258d9a700 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c2bf751c90 0x55c2bf7540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.107+0000 7f5258d9a700 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c2bf751c90 0x55c2bf7540a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3db15f0298e4c273 server_cookie=6592acdceadcbf51 in_seq=0 out_seq=0 2023-07-14T16:25:52.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.107+0000 7f523cff9700 10 client.18351 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:25:52.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f523cff9700 1 -- 172.21.15.62:0/3763927774 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c2bf686eb0 con 0x55c2bf751c90 2023-07-14T16:25:52.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f523cff9700 10 client.18351 handle_command_reply: tid=0 2023-07-14T16:25:52.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 client.18351 shutdown 2023-07-14T16:25:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 -- 172.21.15.62:0/3763927774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73c4d0 msgr2=0x55c2bf745ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73c4d0 0x55c2bf745ec0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f52500114c0 tx=0x7f52500117d0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 -- 172.21.15.62:0/3763927774 shutdown_connections 2023-07-14T16:25:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c2bf751c90 0x55c2bf7540a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 0x55c2bf74d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73c4d0 0x55c2bf745ec0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f523cff9700 0 client.18351 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:25:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 --2- 172.21.15.62:0/3763927774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 0x55c2bf746400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 -- 172.21.15.62:0/3763927774 >> 172.21.15.62:0/3763927774 conn(0x55c2bf731da0 msgr2=0x55c2bf7368b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 -- 172.21.15.62:0/3763927774 shutdown_connections 2023-07-14T16:25:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.108+0000 7f526f063b80 1 -- 172.21.15.62:0/3763927774 wait complete. 2023-07-14T16:25:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.109+0000 7f526f063b80 20 client.18351 trim_cache size 0 max 16384 2023-07-14T16:25:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.122+0000 7f526f063b80 1 Processor -- start 2023-07-14T16:25:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.122+0000 7f526f063b80 1 -- start start 2023-07-14T16:25:52.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f526f063b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 0x55c2bf744c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f526f063b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f526f063b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73dc70 0x55c2bf740af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f526f063b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf74f130 con 0x55c2bf73dc70 2023-07-14T16:25:52.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f526f063b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf73d570 con 0x55c2bf73ce20 2023-07-14T16:25:52.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f526f063b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf72eef0 con 0x55c2bf73c4d0 2023-07-14T16:25:52.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f5259d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73dc70 0x55c2bf740af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f5259d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73dc70 0x55c2bf740af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34292/0 (socket says 172.21.15.62:34292) 2023-07-14T16:25:52.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f5259d9c700 1 -- 172.21.15.62:0/2320239399 learned_addr learned my addr 172.21.15.62:0/2320239399 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f525959b700 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 0x55c2bf744c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.123+0000 7f5258d9a700 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f5259d9c700 1 -- 172.21.15.62:0/2320239399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 msgr2=0x55c2bf744c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f5259d9c700 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 0x55c2bf744c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f5259d9c700 1 -- 172.21.15.62:0/2320239399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 msgr2=0x55c2bf74d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f5259d9c700 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f5259d9c700 1 -- 172.21.15.62:0/2320239399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2bf4e2250 con 0x55c2bf73dc70 2023-07-14T16:25:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f5259d9c700 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73dc70 0x55c2bf740af0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f524c0a2ec0 tx=0x7f524c00b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5e9966a5d9d59e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f523ffff700 1 -- 172.21.15.62:0/2320239399 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c09f060 con 0x55c2bf73dc70 2023-07-14T16:25:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f523ffff700 1 -- 172.21.15.62:0/2320239399 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f524c01a3c0 con 0x55c2bf73dc70 2023-07-14T16:25:52.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.124+0000 7f523ffff700 1 -- 172.21.15.62:0/2320239399 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c09b290 con 0x55c2bf73dc70 2023-07-14T16:25:52.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 -- 172.21.15.62:0/2320239399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73dc70 msgr2=0x55c2bf740af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73dc70 0x55c2bf740af0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f524c0a2ec0 tx=0x7f524c00b7a0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 -- 172.21.15.62:0/2320239399 shutdown_connections 2023-07-14T16:25:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73dc70 0x55c2bf740af0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73c4d0 0x55c2bf744c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 --2- 172.21.15.62:0/2320239399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 -- 172.21.15.62:0/2320239399 >> 172.21.15.62:0/2320239399 conn(0x55c2bf731da0 msgr2=0x55c2bf74dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 -- 172.21.15.62:0/2320239399 shutdown_connections 2023-07-14T16:25:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.125+0000 7f526f063b80 1 -- 172.21.15.62:0/2320239399 wait complete. 2023-07-14T16:25:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.126+0000 7f526f063b80 1 -- start start 2023-07-14T16:25:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.126+0000 7f526f063b80 1 Processor -- start 2023-07-14T16:25:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f526f063b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73c4d0 0x55c2bf74cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f526f063b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f526f063b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 0x55c2bf7497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f526f063b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf749d50 con 0x55c2bf73ce20 2023-07-14T16:25:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f526f063b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf749ec0 con 0x55c2bf73c4d0 2023-07-14T16:25:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f526f063b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf74a080 con 0x55c2bf73dc70 2023-07-14T16:25:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f526f063b80 10 client.0 _ll_register_callbacks cb 0x7f525000bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5258d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5258d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34300/0 (socket says 172.21.15.62:34300) 2023-07-14T16:25:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5258d9a700 1 -- 172.21.15.62:0/825040222 learned_addr learned my addr 172.21.15.62:0/825040222 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f525959b700 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73c4d0 0x55c2bf74cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5259d9c700 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 0x55c2bf7497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5258d9a700 1 -- 172.21.15.62:0/825040222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 msgr2=0x55c2bf7497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5258d9a700 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 0x55c2bf7497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5258d9a700 1 -- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73c4d0 msgr2=0x55c2bf74cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5258d9a700 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73c4d0 0x55c2bf74cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f5258d9a700 1 -- 172.21.15.62:0/825040222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2bf4bbde0 con 0x55c2bf73ce20 2023-07-14T16:25:52.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.127+0000 7f525959b700 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73c4d0 0x55c2bf74cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:52.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.128+0000 7f5258d9a700 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d270 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x55c2bf7549b0 tx=0x7f5244020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bb94c6995b5158 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.128+0000 7f523cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:52.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.128+0000 7f523cff9700 1 -- 172.21.15.62:0/825040222 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440043b0 con 0x55c2bf73ce20 2023-07-14T16:25:52.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.128+0000 7f523cff9700 1 -- 172.21.15.62:0/825040222 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5244004510 con 0x55c2bf73ce20 2023-07-14T16:25:52.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.128+0000 7f526f063b80 1 -- 172.21.15.62:0/825040222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f524c09b030 con 0x55c2bf73ce20 2023-07-14T16:25:52.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.128+0000 7f523cff9700 1 -- 172.21.15.62:0/825040222 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5244011850 con 0x55c2bf73ce20 2023-07-14T16:25:52.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.128+0000 7f526f063b80 10 client.18357 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:52.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.128+0000 7f526f063b80 1 -- 172.21.15.62:0/825040222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c2bf4ba2b0 con 0x55c2bf73ce20 2023-07-14T16:25:52.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f523cff9700 1 -- 172.21.15.62:0/825040222 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f524c09b030 con 0x55c2bf73ce20 2023-07-14T16:25:52.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f523cff9700 1 -- 172.21.15.62:0/825040222 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f524403b070 con 0x55c2bf73ce20 2023-07-14T16:25:52.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f526f063b80 10 client.18357 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:52.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f526f063b80 10 client.18357 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:25:52.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f526f063b80 20 client.18357 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:52.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f526f063b80 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c2bf7a7040 0x55c2bf7a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f526f063b80 4 client.18357 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:25:52.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f526f063b80 1 -- 172.21.15.62:0/825040222 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c2bf649050 con 0x55c2bf7a7040 2023-07-14T16:25:52.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.129+0000 7f525959b700 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c2bf7a7040 0x55c2bf7a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.130+0000 7f523cff9700 10 client.18357 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:25:52.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.130+0000 7f525959b700 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c2bf7a7040 0x55c2bf7a9450 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=893b751e9820cea0 server_cookie=95fef5f35391184a in_seq=0 out_seq=0 2023-07-14T16:25:52.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.130+0000 7f523cff9700 1 -- 172.21.15.62:0/825040222 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c2bf649050 con 0x55c2bf7a7040 2023-07-14T16:25:52.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.130+0000 7f523cff9700 10 client.18357 handle_command_reply: tid=0 2023-07-14T16:25:52.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.130+0000 7f526f063b80 1 client.18357 shutdown 2023-07-14T16:25:52.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.130+0000 7f526f063b80 1 -- 172.21.15.62:0/825040222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 msgr2=0x55c2bf74d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.130+0000 7f526f063b80 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d270 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x55c2bf7549b0 tx=0x7f5244020000 comp rx=0 tx=0).stop 2023-07-14T16:25:52.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 1 -- 172.21.15.62:0/825040222 shutdown_connections 2023-07-14T16:25:52.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55c2bf7a7040 0x55c2bf7a9450 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c2bf73dc70 0x55c2bf7497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c2bf73c4d0 0x55c2bf74cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 1 --2- 172.21.15.62:0/825040222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2bf73ce20 0x55c2bf74d270 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f523cff9700 0 client.18357 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:25:52.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 1 -- 172.21.15.62:0/825040222 >> 172.21.15.62:0/825040222 conn(0x55c2bf731da0 msgr2=0x55c2bf744c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 1 -- 172.21.15.62:0/825040222 shutdown_connections 2023-07-14T16:25:52.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 1 -- 172.21.15.62:0/825040222 wait complete. 2023-07-14T16:25:52.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.131+0000 7f526f063b80 20 client.18357 trim_cache size 0 max 16384 2023-07-14T16:25:52.125 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:25:52.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 -- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f52400a11d0 msgr2=0x7f52400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f52400a11d0 0x7f52400a3680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5244013a70 tx=0x7f5244009c00 comp rx=0 tx=0).stop 2023-07-14T16:25:52.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 -- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 msgr2=0x7f5254072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 0x7f5254072e40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f524c009fd0 tx=0x7f524c00bc80 comp rx=0 tx=0).stop 2023-07-14T16:25:52.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 -- 172.21.15.62:0/1230380348 shutdown_connections 2023-07-14T16:25:52.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5254101660 0x7f5254072e40 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f52400a11d0 0x7f52400a3680 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52540ffae0 0x7f52540742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 --2- 172.21.15.62:0/1230380348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5254100cd0 0x7f5254072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 -- 172.21.15.62:0/1230380348 >> 172.21.15.62:0/1230380348 conn(0x7f52540fb0e0 msgr2=0x7f52541041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 -- 172.21.15.62:0/1230380348 shutdown_connections 2023-07-14T16:25:52.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.132+0000 7f525b7ff700 1 -- 172.21.15.62:0/1230380348 wait complete. 2023-07-14T16:25:52.128 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:25:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 -- 172.21.15.62:0/579407557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2dc102e80 msgr2=0x7ff2dc105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/579407557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2dc102e80 0x7ff2dc105260 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009990 tx=0x7ff2cc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 -- 172.21.15.62:0/579407557 shutdown_connections 2023-07-14T16:25:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/579407557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2dc1057a0 0x7ff2dc107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/579407557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2dc100560 0x7ff2dc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/579407557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2dc102e80 0x7ff2dc105260 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 -- 172.21.15.62:0/579407557 >> 172.21.15.62:0/579407557 conn(0x7ff2dc0fa000 msgr2=0x7ff2dc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 -- 172.21.15.62:0/579407557 shutdown_connections 2023-07-14T16:25:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.369+0000 7ff2e3dff700 1 -- 172.21.15.62:0/579407557 wait complete. 2023-07-14T16:25:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.371+0000 7ff2e3dff700 1 Processor -- start 2023-07-14T16:25:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.371+0000 7ff2e3dff700 1 -- start start 2023-07-14T16:25:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.371+0000 7ff2e3dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2dc100560 0x7ff2dc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.371+0000 7ff2e3dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2dc102e80 0x7ff2dc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.371+0000 7ff2e3dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2dc1057a0 0x7ff2dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.371+0000 7ff2e3dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc19f510 con 0x7ff2dc1057a0 2023-07-14T16:25:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.371+0000 7ff2e3dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc19f650 con 0x7ff2dc100560 2023-07-14T16:25:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.371+0000 7ff2e3dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc19f7e0 con 0x7ff2dc102e80 2023-07-14T16:25:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2dc100560 0x7ff2dc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e239c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2dc1057a0 0x7ff2dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e139a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2dc102e80 0x7ff2dc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2dc100560 0x7ff2dc197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:33452/0 (socket says 172.21.15.62:33452) 2023-07-14T16:25:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 -- 172.21.15.62:0/24042096 learned_addr learned my addr 172.21.15.62:0/24042096 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 -- 172.21.15.62:0/24042096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2dc102e80 msgr2=0x7ff2dc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2dc102e80 0x7ff2dc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 -- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2dc1057a0 msgr2=0x7ff2dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2dc1057a0 0x7ff2dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 -- 172.21.15.62:0/24042096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc0095f0 con 0x7ff2dc100560 2023-07-14T16:25:52.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.372+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2dc100560 0x7ff2dc197990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800d980 tx=0x7ff2d800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.373+0000 7ff2d2ffd700 1 -- 172.21.15.62:0/24042096 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8004220 con 0x7ff2dc100560 2023-07-14T16:25:52.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.373+0000 7ff2e3dff700 1 -- 172.21.15.62:0/24042096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc19fac0 con 0x7ff2dc100560 2023-07-14T16:25:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.373+0000 7ff2d2ffd700 1 -- 172.21.15.62:0/24042096 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d8007640 con 0x7ff2dc100560 2023-07-14T16:25:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.373+0000 7ff2d2ffd700 1 -- 172.21.15.62:0/24042096 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8005660 con 0x7ff2dc100560 2023-07-14T16:25:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.373+0000 7ff2e3dff700 1 -- 172.21.15.62:0/24042096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc1a0020 con 0x7ff2dc100560 2023-07-14T16:25:52.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.377+0000 7ff2d2ffd700 1 -- 172.21.15.62:0/24042096 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff2d8004750 con 0x7ff2dc100560 2023-07-14T16:25:52.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.378+0000 7ff2f7721b80 1 Processor -- start 2023-07-14T16:25:52.353 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2f7721b80 1 -- start start 2023-07-14T16:25:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2d2ffd700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2c80a11d0 0x7ff2c80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.354 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae8134d0 0x556cae8138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae813e20 0x556cae8146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae814c70 0x556cae8150e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae718cb0 con 0x556cae8134d0 2023-07-14T16:25:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae76ba40 con 0x556cae813e20 2023-07-14T16:25:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae764ed0 con 0x556cae814c70 2023-07-14T16:25:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2d2ffd700 1 -- 172.21.15.62:0/24042096 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff2d80cc610 con 0x7ff2dc100560 2023-07-14T16:25:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2e1b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae8134d0 0x556cae8138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2e239c700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2c80a11d0 0x7ff2c80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2e1b9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae8134d0 0x556cae8138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34314/0 (socket says 172.21.15.62:34314) 2023-07-14T16:25:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.379+0000 7ff2e1b9b700 1 -- 172.21.15.62:0/2639613320 learned_addr learned my addr 172.21.15.62:0/2639613320 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e239c700 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae814c70 0x556cae8150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e139a700 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae813e20 0x556cae8146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e1b9b700 1 -- 172.21.15.62:0/2639613320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae814c70 msgr2=0x556cae8150e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae814c70 0x556cae8150e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e1b9b700 1 -- 172.21.15.62:0/2639613320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae813e20 msgr2=0x556cae8146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae813e20 0x556cae8146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e1b9b700 1 -- 172.21.15.62:0/2639613320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cae592de0 con 0x556cae8134d0 2023-07-14T16:25:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e239c700 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae814c70 0x556cae8150e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae8134d0 0x556cae8138e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8007fd0 tx=0x7ff2d8099390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adc1690632ad479 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2e239c700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2c80a11d0 0x7ff2c80a3680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009ec0 tx=0x7ff2d4009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.380+0000 7ff2c7fff700 1 -- 172.21.15.62:0/2639613320 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8099a30 con 0x556cae8134d0 2023-07-14T16:25:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.381+0000 7ff2c7fff700 1 -- 172.21.15.62:0/2639613320 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d809ae90 con 0x556cae8134d0 2023-07-14T16:25:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.381+0000 7ff2c7fff700 1 -- 172.21.15.62:0/2639613320 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8082e90 con 0x556cae8134d0 2023-07-14T16:25:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.381+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2639613320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae8134d0 msgr2=0x556cae8138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.381+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae8134d0 0x556cae8138e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8007fd0 tx=0x7ff2d8099390 comp rx=0 tx=0).stop 2023-07-14T16:25:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.381+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2639613320 shutdown_connections 2023-07-14T16:25:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.381+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae814c70 0x556cae8150e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.381+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae8134d0 0x556cae8138e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.381+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/2639613320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae813e20 0x556cae8146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.382+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2639613320 >> 172.21.15.62:0/2639613320 conn(0x556cae808da0 msgr2=0x556cae80e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.382+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2639613320 shutdown_connections 2023-07-14T16:25:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.382+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2639613320 wait complete. 2023-07-14T16:25:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.382+0000 7ff2f7721b80 1 -- start start 2023-07-14T16:25:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.382+0000 7ff2f7721b80 1 Processor -- start 2023-07-14T16:25:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae8134d0 0x556cae81d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae813e20 0x556cae81d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae814c70 0x556cae8245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae704ac0 con 0x556cae813e20 2023-07-14T16:25:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae7047c0 con 0x556cae814c70 2023-07-14T16:25:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae5e8920 con 0x556cae8134d0 2023-07-14T16:25:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2f7721b80 10 client.0 _ll_register_callbacks cb 0x556cae702760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2e139a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae813e20 0x556cae81d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2e139a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae813e20 0x556cae81d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34318/0 (socket says 172.21.15.62:34318) 2023-07-14T16:25:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.383+0000 7ff2e139a700 1 -- 172.21.15.62:0/3473964340 learned_addr learned my addr 172.21.15.62:0/3473964340 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2e239c700 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae814c70 0x556cae8245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae8134d0 0x556cae81d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2e139a700 1 -- 172.21.15.62:0/3473964340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae8134d0 msgr2=0x556cae81d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2e139a700 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae8134d0 0x556cae81d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2e139a700 1 -- 172.21.15.62:0/3473964340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae814c70 msgr2=0x556cae8245b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2e139a700 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae814c70 0x556cae8245b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2e139a700 1 -- 172.21.15.62:0/3473964340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc1a0020 con 0x556cae813e20 2023-07-14T16:25:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2e139a700 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae813e20 0x556cae81d570 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009990 tx=0x7ff2cc005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83ea12ace96619c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:52.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3473964340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc01d070 con 0x556cae813e20 2023-07-14T16:25:52.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3473964340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cae6be4f0 con 0x556cae813e20 2023-07-14T16:25:52.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.384+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3473964340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff2cc003b10 con 0x556cae813e20 2023-07-14T16:25:52.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3473964340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2cc003cd0 con 0x556cae813e20 2023-07-14T16:25:52.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2f7721b80 10 client.18369 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:52.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3473964340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556cae57bc60 con 0x556cae813e20 2023-07-14T16:25:52.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3473964340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556cae6be4f0 con 0x556cae813e20 2023-07-14T16:25:52.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3473964340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff2cc0047f0 con 0x556cae813e20 2023-07-14T16:25:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2f7721b80 10 client.18369 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2f7721b80 10 client.18369 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:25:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2f7721b80 20 client.18369 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556cae828e10 0x556cae82b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2f7721b80 4 client.18369 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.385+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3473964340 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cae75deb0 con 0x556cae828e10 2023-07-14T16:25:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.386+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556cae828e10 0x556cae82b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.386+0000 7ff2c57fa700 10 client.18369 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:25:52.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.386+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556cae828e10 0x556cae82b220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79ab28bfc155cc8c server_cookie=15638ff021739bc in_seq=0 out_seq=0 2023-07-14T16:25:52.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.387+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3473964340 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556cae75deb0 con 0x556cae828e10 2023-07-14T16:25:52.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.387+0000 7ff2c57fa700 10 client.18369 handle_command_reply: tid=0 2023-07-14T16:25:52.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.387+0000 7ff2f7721b80 1 client.18369 shutdown 2023-07-14T16:25:52.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.387+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3473964340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae813e20 msgr2=0x556cae81d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae813e20 0x556cae81d570 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009990 tx=0x7ff2cc005460 comp rx=0 tx=0).stop 2023-07-14T16:25:52.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3473964340 shutdown_connections 2023-07-14T16:25:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556cae828e10 0x556cae82b220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae814c70 0x556cae8245b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae8134d0 0x556cae81d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2c57fa700 0 client.18369 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:25:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3473964340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae813e20 0x556cae81d570 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3473964340 >> 172.21.15.62:0/3473964340 conn(0x556cae808da0 msgr2=0x556cae80d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3473964340 shutdown_connections 2023-07-14T16:25:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3473964340 wait complete. 2023-07-14T16:25:52.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.388+0000 7ff2f7721b80 20 client.18369 trim_cache size 0 max 16384 2023-07-14T16:25:52.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2f7721b80 1 Processor -- start 2023-07-14T16:25:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2f7721b80 1 -- start start 2023-07-14T16:25:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae81c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae824570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae824ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae80f070 con 0x556cae814c70 2023-07-14T16:25:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae81dcb0 con 0x556cae8134d0 2023-07-14T16:25:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae81ad90 con 0x556cae813e20 2023-07-14T16:25:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2e239c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae824ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2e239c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae824ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34320/0 (socket says 172.21.15.62:34320) 2023-07-14T16:25:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2e239c700 1 -- 172.21.15.62:0/2856305305 learned_addr learned my addr 172.21.15.62:0/2856305305 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.405+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae81c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e139a700 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae824570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e239c700 1 -- 172.21.15.62:0/2856305305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 msgr2=0x556cae824570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e239c700 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae824570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e239c700 1 -- 172.21.15.62:0/2856305305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 msgr2=0x556cae81c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e239c700 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae81c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e239c700 1 -- 172.21.15.62:0/2856305305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cae5b9250 con 0x556cae814c70 2023-07-14T16:25:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae81c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e139a700 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae824570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.406+0000 7ff2e239c700 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae824ea0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009e20 tx=0x7ff2d4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a6ad025ab6ff6d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.407+0000 7ff2d0bff700 1 -- 172.21.15.62:0/2856305305 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400e040 con 0x556cae814c70 2023-07-14T16:25:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.407+0000 7ff2d0bff700 1 -- 172.21.15.62:0/2856305305 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d400f040 con 0x556cae814c70 2023-07-14T16:25:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.407+0000 7ff2d0bff700 1 -- 172.21.15.62:0/2856305305 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d401a7b0 con 0x556cae814c70 2023-07-14T16:25:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.407+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2856305305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 msgr2=0x556cae824ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.407+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae824ea0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009e20 tx=0x7ff2d4004b40 comp rx=0 tx=0).stop 2023-07-14T16:25:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2856305305 shutdown_connections 2023-07-14T16:25:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae824ea0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae81c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/2856305305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae824570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2856305305 >> 172.21.15.62:0/2856305305 conn(0x556cae808da0 msgr2=0x556cae82fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2856305305 shutdown_connections 2023-07-14T16:25:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 -- 172.21.15.62:0/2856305305 wait complete. 2023-07-14T16:25:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 -- start start 2023-07-14T16:25:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.408+0000 7ff2f7721b80 1 Processor -- start 2023-07-14T16:25:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae8207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae820ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2f7721b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae8239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae823f20 con 0x556cae814c70 2023-07-14T16:25:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae824090 con 0x556cae8134d0 2023-07-14T16:25:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2f7721b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cae824250 con 0x556cae813e20 2023-07-14T16:25:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2f7721b80 10 client.0 _ll_register_callbacks cb 0x7ff2cc005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2e239c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae8239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2e139a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae820ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2e239c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae8239e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34330/0 (socket says 172.21.15.62:34330) 2023-07-14T16:25:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2e239c700 1 -- 172.21.15.62:0/3684769340 learned_addr learned my addr 172.21.15.62:0/3684769340 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.409+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae8207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2e239c700 1 -- 172.21.15.62:0/3684769340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 msgr2=0x556cae820ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2e239c700 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae820ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2e239c700 1 -- 172.21.15.62:0/3684769340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 msgr2=0x556cae8207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2e239c700 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae8207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2e239c700 1 -- 172.21.15.62:0/3684769340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cae592de0 con 0x556cae814c70 2023-07-14T16:25:52.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae8207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2e239c700 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae8239e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4019010 tx=0x7ff2d4007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10d64126816d597 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3684769340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4036070 con 0x556cae814c70 2023-07-14T16:25:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3684769340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2d809ae90 con 0x556cae814c70 2023-07-14T16:25:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.410+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3684769340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff2d4007b40 con 0x556cae814c70 2023-07-14T16:25:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3684769340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d400e040 con 0x556cae814c70 2023-07-14T16:25:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2f7721b80 10 client.18375 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3684769340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7ff2d40090f0 con 0x556cae814c70 2023-07-14T16:25:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3684769340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d4041020 con 0x556cae814c70 2023-07-14T16:25:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3684769340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff2d4039020 con 0x556cae814c70 2023-07-14T16:25:52.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2f7721b80 10 client.18375 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:52.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2f7721b80 10 client.18375 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:25:52.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2f7721b80 20 client.18375 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:52.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556cae87dd40 0x556cae880150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2f7721b80 4 client.18375 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:25:52.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.411+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3684769340 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556cae720050 con 0x556cae87dd40 2023-07-14T16:25:52.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.412+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556cae87dd40 0x556cae880150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.412+0000 7ff2c57fa700 10 client.18375 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:25:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.412+0000 7ff2e1b9b700 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556cae87dd40 0x556cae880150 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3a06dd564967399 server_cookie=e009105fe603520b in_seq=0 out_seq=0 2023-07-14T16:25:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.412+0000 7ff2c57fa700 1 -- 172.21.15.62:0/3684769340 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556cae720050 con 0x556cae87dd40 2023-07-14T16:25:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.412+0000 7ff2c57fa700 10 client.18375 handle_command_reply: tid=0 2023-07-14T16:25:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.412+0000 7ff2f7721b80 1 client.18375 shutdown 2023-07-14T16:25:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3684769340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 msgr2=0x556cae8239e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae8239e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4019010 tx=0x7ff2d4007830 comp rx=0 tx=0).stop 2023-07-14T16:25:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3684769340 shutdown_connections 2023-07-14T16:25:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556cae87dd40 0x556cae880150 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556cae814c70 0x556cae8239e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556cae8134d0 0x556cae8207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 --2- 172.21.15.62:0/3684769340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556cae813e20 0x556cae820ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3684769340 >> 172.21.15.62:0/3684769340 conn(0x556cae808da0 msgr2=0x556cae817b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2c57fa700 0 client.18375 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:25:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3684769340 shutdown_connections 2023-07-14T16:25:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 1 -- 172.21.15.62:0/3684769340 wait complete. 2023-07-14T16:25:52.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.413+0000 7ff2f7721b80 20 client.18375 trim_cache size 0 max 16384 2023-07-14T16:25:52.397 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:25:52.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 -- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2c80a11d0 msgr2=0x7ff2c80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2c80a11d0 0x7ff2c80a3680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009ec0 tx=0x7ff2d4009920 comp rx=0 tx=0).stop 2023-07-14T16:25:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 -- 172.21.15.62:0/24042096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2dc100560 msgr2=0x7ff2dc197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2dc100560 0x7ff2dc197990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800d980 tx=0x7ff2d800dc90 comp rx=0 tx=0).stop 2023-07-14T16:25:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 -- 172.21.15.62:0/24042096 shutdown_connections 2023-07-14T16:25:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2dc1057a0 0x7ff2dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2c80a11d0 0x7ff2c80a3680 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2dc100560 0x7ff2dc197990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 --2- 172.21.15.62:0/24042096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff2dc102e80 0x7ff2dc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 -- 172.21.15.62:0/24042096 >> 172.21.15.62:0/24042096 conn(0x7ff2dc0fa000 msgr2=0x7ff2dc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 -- 172.21.15.62:0/24042096 shutdown_connections 2023-07-14T16:25:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.414+0000 7ff2e3dff700 1 -- 172.21.15.62:0/24042096 wait complete. 2023-07-14T16:25:52.410 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:25:52.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.665+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2010918780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 msgr2=0x7fdff4107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.665+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2010918780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 0x7fdff4107f60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00b0a0 tx=0x7fdfec00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.666+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2010918780 shutdown_connections 2023-07-14T16:25:52.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.666+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2010918780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 0x7fdff4107f60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.666+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2010918780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdff40ffb20 0x7fdff40fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.666+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2010918780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdff4100d10 0x7fdff4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.666+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2010918780 >> 172.21.15.62:0/2010918780 conn(0x7fdff40fb160 msgr2=0x7fdff40fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.666+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2010918780 shutdown_connections 2023-07-14T16:25:52.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.666+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2010918780 wait complete. 2023-07-14T16:25:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.667+0000 7fdffc5ff700 1 Processor -- start 2023-07-14T16:25:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.667+0000 7fdffc5ff700 1 -- start start 2023-07-14T16:25:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.667+0000 7fdffc5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdff40ffb20 0x7fdff40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.667+0000 7fdffc5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdff4100d10 0x7fdff4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffc5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 0x7fdff4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffc5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40735e0 con 0x7fdff41016a0 2023-07-14T16:25:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffc5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4073750 con 0x7fdff40ffb20 2023-07-14T16:25:52.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffc5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff419f900 con 0x7fdff4100d10 2023-07-14T16:25:52.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 0x7fdff4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffa39b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdff40ffb20 0x7fdff40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 0x7fdff4072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34352/0 (socket says 172.21.15.62:34352) 2023-07-14T16:25:52.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffab9c700 1 -- 172.21.15.62:0/2620810692 learned_addr learned my addr 172.21.15.62:0/2620810692 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdff4100d10 0x7fdff4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffab9c700 1 -- 172.21.15.62:0/2620810692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdff4100d10 msgr2=0x7fdff4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffab9c700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdff4100d10 0x7fdff4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffab9c700 1 -- 172.21.15.62:0/2620810692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdff40ffb20 msgr2=0x7fdff40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffab9c700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdff40ffb20 0x7fdff40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.668+0000 7fdffab9c700 1 -- 172.21.15.62:0/2620810692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec009d00 con 0x7fdff41016a0 2023-07-14T16:25:52.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.669+0000 7fdffab9c700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 0x7fdff4072e40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009fd0 tx=0x7fdfec00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.669+0000 7fdfeb7fe700 1 -- 172.21.15.62:0/2620810692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00e040 con 0x7fdff41016a0 2023-07-14T16:25:52.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.669+0000 7fdfeb7fe700 1 -- 172.21.15.62:0/2620810692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfec00be90 con 0x7fdff41016a0 2023-07-14T16:25:52.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.669+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2620810692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff419fb80 con 0x7fdff41016a0 2023-07-14T16:25:52.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.669+0000 7fdfeb7fe700 1 -- 172.21.15.62:0/2620810692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec01b460 con 0x7fdff41016a0 2023-07-14T16:25:52.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.669+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2620810692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff419ffa0 con 0x7fdff41016a0 2023-07-14T16:25:52.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.673+0000 7fdfeb7fe700 1 -- 172.21.15.62:0/2620810692 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdfec01d470 con 0x7fdff41016a0 2023-07-14T16:25:52.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.674+0000 7fe00ff96b80 1 Processor -- start 2023-07-14T16:25:52.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.674+0000 7fdfeb7fe700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdfe00a11d0 0x7fdfe00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.674+0000 7fe00ff96b80 1 -- start start 2023-07-14T16:25:52.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fdfeb7fe700 1 -- 172.21.15.62:0/2620810692 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdfec0cda30 con 0x7fdff41016a0 2023-07-14T16:25:52.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdfe00a11d0 0x7fdfe00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8e26cb0 con 0x5605d8f22c70 2023-07-14T16:25:52.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8e79a40 con 0x5605d8f214d0 2023-07-14T16:25:52.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8e72ed0 con 0x5605d8f21e20 2023-07-14T16:25:52.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fdffab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fdffab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f230e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34354/0 (socket says 172.21.15.62:34354) 2023-07-14T16:25:52.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.675+0000 7fdffab9c700 1 -- 172.21.15.62:0/2862793746 learned_addr learned my addr 172.21.15.62:0/2862793746 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdffa39b700 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdffab9c700 1 -- 172.21.15.62:0/2862793746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 msgr2=0x5605d8f226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdffab9c700 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdffab9c700 1 -- 172.21.15.62:0/2862793746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 msgr2=0x5605d8f218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdffab9c700 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdffab9c700 1 -- 172.21.15.62:0/2862793746 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605d8ca0de0 con 0x5605d8f22c70 2023-07-14T16:25:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdfe00a11d0 0x7fdfe00a3680 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009c70 tx=0x7fdfe4011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdffab9c700 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f230e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0078b0 tx=0x7fdfec09aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2cb6f88490ce35 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdfe8bfe700 1 -- 172.21.15.62:0/2862793746 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec09aee0 con 0x5605d8f22c70 2023-07-14T16:25:52.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdfe8bfe700 1 -- 172.21.15.62:0/2862793746 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfec09ef10 con 0x5605d8f22c70 2023-07-14T16:25:52.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.676+0000 7fdfe8bfe700 1 -- 172.21.15.62:0/2862793746 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec01d990 con 0x5605d8f22c70 2023-07-14T16:25:52.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2862793746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 msgr2=0x5605d8f230e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f230e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0078b0 tx=0x7fdfec09aaa0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2862793746 shutdown_connections 2023-07-14T16:25:52.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f230e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2862793746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2862793746 >> 172.21.15.62:0/2862793746 conn(0x5605d8f16da0 msgr2=0x5605d8f1c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2862793746 shutdown_connections 2023-07-14T16:25:52.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.677+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2862793746 wait complete. 2023-07-14T16:25:52.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.678+0000 7fe00ff96b80 1 -- start start 2023-07-14T16:25:52.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.678+0000 7fe00ff96b80 1 Processor -- start 2023-07-14T16:25:52.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f2b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f32460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8e12ac0 con 0x5605d8f22c70 2023-07-14T16:25:52.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8e127c0 con 0x5605d8f214d0 2023-07-14T16:25:52.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8cf6920 con 0x5605d8f21e20 2023-07-14T16:25:52.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fe00ff96b80 10 client.0 _ll_register_callbacks cb 0x5605d8e10760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:52.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fdffab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f32460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fdffab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f32460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34368/0 (socket says 172.21.15.62:34368) 2023-07-14T16:25:52.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fdffab9c700 1 -- 172.21.15.62:0/2771594336 learned_addr learned my addr 172.21.15.62:0/2771594336 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.679+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f2b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdff9b9a700 1 -- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 msgr2=0x5605d8f2aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdffa39b700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdff9b9a700 1 -- 172.21.15.62:0/2771594336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 msgr2=0x5605d8f32460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f32460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdff9b9a700 1 -- 172.21.15.62:0/2771594336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff419ffa0 con 0x5605d8f21e20 2023-07-14T16:25:52.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdffa39b700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:52.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdffab9c700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f32460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:52.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f2b420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401e9b0 tx=0x7fdfe40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05eeaf24dad67d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.680+0000 7fdfddffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:25:52.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.681+0000 7fdfddffb700 1 -- 172.21.15.62:0/2771594336 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe401e220 con 0x5605d8f21e20 2023-07-14T16:25:52.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.681+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2771594336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605d8dcc4f0 con 0x5605d8f21e20 2023-07-14T16:25:52.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.681+0000 7fdfddffb700 1 -- 172.21.15.62:0/2771594336 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdfe401ee30 con 0x5605d8f21e20 2023-07-14T16:25:52.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.681+0000 7fdfddffb700 1 -- 172.21.15.62:0/2771594336 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe402d030 con 0x5605d8f21e20 2023-07-14T16:25:52.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.681+0000 7fe00ff96b80 10 client.27892 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:52.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.681+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2771594336 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5605d8c89c60 con 0x5605d8f21e20 2023-07-14T16:25:52.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.681+0000 7fdfddffb700 1 -- 172.21.15.62:0/2771594336 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5605d8dcc4f0 con 0x5605d8f21e20 2023-07-14T16:25:52.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.681+0000 7fdfddffb700 1 -- 172.21.15.62:0/2771594336 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdfe4005020 con 0x5605d8f21e20 2023-07-14T16:25:52.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.682+0000 7fe00ff96b80 10 client.27892 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:52.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.682+0000 7fe00ff96b80 10 client.27892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:52.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.682+0000 7fe00ff96b80 20 client.27892 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:52.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.682+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605d8f36de0 0x5605d8f391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.682+0000 7fe00ff96b80 4 client.27892 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:52.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.682+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2771594336 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605d8e6beb0 con 0x5605d8f36de0 2023-07-14T16:25:52.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.682+0000 7fdffa39b700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605d8f36de0 0x5605d8f391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.683+0000 7fdffa39b700 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605d8f36de0 0x5605d8f391f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d7b3535d215ce5 server_cookie=b027a711b5f70170 in_seq=0 out_seq=0 2023-07-14T16:25:52.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.683+0000 7fdfddffb700 10 client.27892 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:52.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fdfddffb700 1 -- 172.21.15.62:0/2771594336 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5605d8e6beb0 con 0x5605d8f36de0 2023-07-14T16:25:52.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fdfddffb700 10 client.27892 handle_command_reply: tid=0 2023-07-14T16:25:52.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 client.27892 shutdown 2023-07-14T16:25:52.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2771594336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 msgr2=0x5605d8f2b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f2b420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401e9b0 tx=0x7fdfe40047c0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2771594336 shutdown_connections 2023-07-14T16:25:52.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f22c70 0x5605d8f32460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605d8f36de0 0x5605d8f391f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2771594336 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f21e20 0x5605d8f2b420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2771594336 >> 172.21.15.62:0/2771594336 conn(0x5605d8f16da0 msgr2=0x5605d8f1b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fdfddffb700 0 client.27892 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:52.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2771594336 shutdown_connections 2023-07-14T16:25:52.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2771594336 wait complete. 2023-07-14T16:25:52.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.684+0000 7fe00ff96b80 20 client.27892 trim_cache size 0 max 16384 2023-07-14T16:25:52.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fe00ff96b80 1 Processor -- start 2023-07-14T16:25:52.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fe00ff96b80 1 -- start start 2023-07-14T16:25:52.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f3db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f32420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 0x5605d8f32c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8f2aa50 con 0x5605d8f21e20 2023-07-14T16:25:52.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8f221f0 con 0x5605d8f214d0 2023-07-14T16:25:52.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8f1ced0 con 0x5605d8f22c70 2023-07-14T16:25:52.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.700+0000 7fdff9b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f32420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdff9b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f32420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34384/0 (socket says 172.21.15.62:34384) 2023-07-14T16:25:52.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdff9b9a700 1 -- 172.21.15.62:0/4156563353 learned_addr learned my addr 172.21.15.62:0/4156563353 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdffab9c700 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 0x5605d8f32c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdffa39b700 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f3db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdff9b9a700 1 -- 172.21.15.62:0/4156563353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 msgr2=0x5605d8f32c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 0x5605d8f32c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdff9b9a700 1 -- 172.21.15.62:0/4156563353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 msgr2=0x5605d8f3db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f3db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdff9b9a700 1 -- 172.21.15.62:0/4156563353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605d8cc7250 con 0x5605d8f21e20 2023-07-14T16:25:52.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdffa39b700 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f3db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:52.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.701+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f32420 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40053d0 tx=0x7fdfe4009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fab4059c97010bf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.702+0000 7fdfe93ff700 1 -- 172.21.15.62:0/4156563353 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40092e0 con 0x5605d8f21e20 2023-07-14T16:25:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.702+0000 7fdfe93ff700 1 -- 172.21.15.62:0/4156563353 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfe400fe70 con 0x5605d8f21e20 2023-07-14T16:25:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.702+0000 7fdfe93ff700 1 -- 172.21.15.62:0/4156563353 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4004e90 con 0x5605d8f21e20 2023-07-14T16:25:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.702+0000 7fe00ff96b80 1 -- 172.21.15.62:0/4156563353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 msgr2=0x5605d8f32420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.702+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f32420 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40053d0 tx=0x7fdfe4009dd0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 -- 172.21.15.62:0/4156563353 shutdown_connections 2023-07-14T16:25:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 0x5605d8f32c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f3db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/4156563353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f32420 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 -- 172.21.15.62:0/4156563353 >> 172.21.15.62:0/4156563353 conn(0x5605d8f16da0 msgr2=0x5605d8f1c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 -- 172.21.15.62:0/4156563353 shutdown_connections 2023-07-14T16:25:52.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 -- 172.21.15.62:0/4156563353 wait complete. 2023-07-14T16:25:52.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 -- start start 2023-07-14T16:25:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.703+0000 7fe00ff96b80 1 Processor -- start 2023-07-14T16:25:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f2ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fe00ff96b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 0x5605d8f31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8f31f60 con 0x5605d8f21e20 2023-07-14T16:25:52.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8f320d0 con 0x5605d8f214d0 2023-07-14T16:25:52.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fe00ff96b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d8f32290 con 0x5605d8f22c70 2023-07-14T16:25:52.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fe00ff96b80 10 client.0 _ll_register_callbacks cb 0x7fdfe401e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:52.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fdff9b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f2ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fdff9b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f2ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34390/0 (socket says 172.21.15.62:34390) 2023-07-14T16:25:52.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fdff9b9a700 1 -- 172.21.15.62:0/2649034467 learned_addr learned my addr 172.21.15.62:0/2649034467 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.704+0000 7fdffab9c700 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 0x5605d8f31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdffa39b700 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdff9b9a700 1 -- 172.21.15.62:0/2649034467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 msgr2=0x5605d8f31a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 0x5605d8f31a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdff9b9a700 1 -- 172.21.15.62:0/2649034467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 msgr2=0x5605d8f2e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:52.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdff9b9a700 1 -- 172.21.15.62:0/2649034467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605d8c89c60 con 0x5605d8f21e20 2023-07-14T16:25:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdff9b9a700 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f2ea00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401e9b0 tx=0x7fdfe4005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3316b85565bce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdfddffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.705+0000 7fdfddffb700 1 -- 172.21.15.62:0/2649034467 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4004c10 con 0x5605d8f21e20 2023-07-14T16:25:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2649034467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfec09ef10 con 0x5605d8f21e20 2023-07-14T16:25:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fdfddffb700 1 -- 172.21.15.62:0/2649034467 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdfe4040070 con 0x5605d8f21e20 2023-07-14T16:25:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fdfddffb700 1 -- 172.21.15.62:0/2649034467 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe404a030 con 0x5605d8f21e20 2023-07-14T16:25:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 10 client.18393 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2649034467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5605d8c9f2b0 con 0x5605d8f21e20 2023-07-14T16:25:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fdfddffb700 1 -- 172.21.15.62:0/2649034467 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfec09ef10 con 0x5605d8f21e20 2023-07-14T16:25:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fdfddffb700 1 -- 172.21.15.62:0/2649034467 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdfe40448c0 con 0x5605d8f21e20 2023-07-14T16:25:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 10 client.18393 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 10 client.18393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 20 client.18393 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:52.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605d8f8ba70 0x5605d8f8de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 4 client.18393 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:25:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.706+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2649034467 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5605d8e2e050 con 0x5605d8f8ba70 2023-07-14T16:25:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.707+0000 7fdffa39b700 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605d8f8ba70 0x5605d8f8de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.707+0000 7fdfddffb700 10 client.18393 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.707+0000 7fdffa39b700 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605d8f8ba70 0x5605d8f8de80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b35809d5e4875a6 server_cookie=fb581713bb49bbbc in_seq=0 out_seq=0 2023-07-14T16:25:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fdfddffb700 1 -- 172.21.15.62:0/2649034467 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5605d8e2e050 con 0x5605d8f8ba70 2023-07-14T16:25:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fdfddffb700 10 client.18393 handle_command_reply: tid=0 2023-07-14T16:25:52.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 client.18393 shutdown 2023-07-14T16:25:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2649034467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 msgr2=0x5605d8f2ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f2ea00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401e9b0 tx=0x7fdfe4005de0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2649034467 shutdown_connections 2023-07-14T16:25:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605d8f22c70 0x5605d8f31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605d8f8ba70 0x5605d8f8de80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605d8f214d0 0x5605d8f2e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 --2- 172.21.15.62:0/2649034467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605d8f21e20 0x5605d8f2ea00 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fdfddffb700 0 client.18393 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2649034467 >> 172.21.15.62:0/2649034467 conn(0x5605d8f16da0 msgr2=0x5605d8f1c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2649034467 shutdown_connections 2023-07-14T16:25:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 1 -- 172.21.15.62:0/2649034467 wait complete. 2023-07-14T16:25:52.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.708+0000 7fe00ff96b80 20 client.18393 trim_cache size 0 max 16384 2023-07-14T16:25:52.698 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:25:52.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdfe00a11d0 msgr2=0x7fdfe00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdfe00a11d0 0x7fdfe00a3680 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009c70 tx=0x7fdfe4011720 comp rx=0 tx=0).stop 2023-07-14T16:25:52.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 msgr2=0x7fdff4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:52.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 0x7fdff4072e40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009fd0 tx=0x7fdfec00ba50 comp rx=0 tx=0).stop 2023-07-14T16:25:52.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2620810692 shutdown_connections 2023-07-14T16:25:52.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff41016a0 0x7fdff4072e40 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdfe00a11d0 0x7fdfe00a3680 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdff40ffb20 0x7fdff40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 --2- 172.21.15.62:0/2620810692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdff4100d10 0x7fdff4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:52.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2620810692 >> 172.21.15.62:0/2620810692 conn(0x7fdff40fb160 msgr2=0x7fdff40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:52.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2620810692 shutdown_connections 2023-07-14T16:25:52.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:52.709+0000 7fdffc5ff700 1 -- 172.21.15.62:0/2620810692 wait complete. 2023-07-14T16:25:53.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:52 smithi062 ceph-mon[95543]: pgmap v664: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:25:53.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:52 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1939169661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:25:53.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:52 smithi081 ceph-mon[127795]: pgmap v664: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:25:53.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:52 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1939169661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:25:53.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:52 smithi164 ceph-mon[97427]: pgmap v664: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 14 op/s 2023-07-14T16:25:53.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:52 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1939169661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:25:53.704 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:25:53.704 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:25:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.969+0000 7f3b28b9d700 1 -- 172.21.15.62:0/3123398909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b24101c00 msgr2=0x7f3b24102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.969+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/3123398909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b24101c00 0x7f3b24102010 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009990 tx=0x7f3b14009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.970+0000 7f3b28b9d700 1 -- 172.21.15.62:0/3123398909 shutdown_connections 2023-07-14T16:25:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.970+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/3123398909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b24103780 0x7f3b2410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.970+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/3123398909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b24101c00 0x7f3b24102010 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.970+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/3123398909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b24102df0 0x7f3b24103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.970+0000 7f3b28b9d700 1 -- 172.21.15.62:0/3123398909 >> 172.21.15.62:0/3123398909 conn(0x7f3b240fd220 msgr2=0x7f3b240ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.970+0000 7f3b28b9d700 1 -- 172.21.15.62:0/3123398909 shutdown_connections 2023-07-14T16:25:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.970+0000 7f3b28b9d700 1 -- 172.21.15.62:0/3123398909 wait complete. 2023-07-14T16:25:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.971+0000 7f3b28b9d700 1 Processor -- start 2023-07-14T16:25:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.971+0000 7f3b28b9d700 1 -- start start 2023-07-14T16:25:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.971+0000 7f3b28b9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b24101c00 0x7f3b241977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.971+0000 7f3b28b9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b24102df0 0x7f3b24197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b28b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b24103780 0x7f3b2419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b28b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2419f360 con 0x7f3b24103780 2023-07-14T16:25:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b28b9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2419f4a0 con 0x7f3b24101c00 2023-07-14T16:25:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b28b9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2419f630 con 0x7f3b24102df0 2023-07-14T16:25:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b237fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b24101c00 0x7f3b241977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b237fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b24101c00 0x7f3b241977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52970/0 (socket says 172.21.15.62:52970) 2023-07-14T16:25:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b237fe700 1 -- 172.21.15.62:0/1432160555 learned_addr learned my addr 172.21.15.62:0/1432160555 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b23fff700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b24103780 0x7f3b2419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b24102df0 0x7f3b24197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b237fe700 1 -- 172.21.15.62:0/1432160555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b24102df0 msgr2=0x7f3b24197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b237fe700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b24102df0 0x7f3b24197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b237fe700 1 -- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b24103780 msgr2=0x7f3b2419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:53.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b237fe700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b24103780 0x7f3b2419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.972+0000 7f3b237fe700 1 -- 172.21.15.62:0/1432160555 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b140095f0 con 0x7f3b24101c00 2023-07-14T16:25:53.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.973+0000 7f3b237fe700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b24101c00 0x7f3b241977e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005910 tx=0x7f3b14005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:53.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.973+0000 7f3b20ff9700 1 -- 172.21.15.62:0/1432160555 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1401d070 con 0x7f3b24101c00 2023-07-14T16:25:53.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.973+0000 7f3b20ff9700 1 -- 172.21.15.62:0/1432160555 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b1400be50 con 0x7f3b24101c00 2023-07-14T16:25:53.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.973+0000 7f3b28b9d700 1 -- 172.21.15.62:0/1432160555 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2419f8b0 con 0x7f3b24101c00 2023-07-14T16:25:53.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.973+0000 7f3b20ff9700 1 -- 172.21.15.62:0/1432160555 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14022da0 con 0x7f3b24101c00 2023-07-14T16:25:53.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.973+0000 7f3b28b9d700 1 -- 172.21.15.62:0/1432160555 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2419fcd0 con 0x7f3b24101c00 2023-07-14T16:25:53.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.977+0000 7f3b20ff9700 1 -- 172.21.15.62:0/1432160555 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3b14022570 con 0x7f3b24101c00 2023-07-14T16:25:53.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.978+0000 7f3b3d6a9b80 1 Processor -- start 2023-07-14T16:25:53.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.978+0000 7f3b3d6a9b80 1 -- start start 2023-07-14T16:25:53.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.978+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559ee4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.978+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ee5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.978+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee5eb0 0x55b559ee6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.978+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559de9d30 con 0x55b559ee47a0 2023-07-14T16:25:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.978+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559e3cac0 con 0x55b559ee50f0 2023-07-14T16:25:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.978+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559e35f50 con 0x55b559ee5eb0 2023-07-14T16:25:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559ee4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559ee4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48610/0 (socket says 172.21.15.62:48610) 2023-07-14T16:25:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b23fff700 1 -- 172.21.15.62:0/2997797572 learned_addr learned my addr 172.21.15.62:0/2997797572 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:53.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b20ff9700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3b0c0a11f0 0x7f3b0c0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee5eb0 0x55b559ee6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b237fe700 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ee5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b20ff9700 1 -- 172.21.15.62:0/1432160555 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3b140cdf60 con 0x7f3b24101c00 2023-07-14T16:25:53.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b23fff700 1 -- 172.21.15.62:0/2997797572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee5eb0 msgr2=0x55b559ee6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:53.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b23fff700 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee5eb0 0x55b559ee6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b23fff700 1 -- 172.21.15.62:0/2997797572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 msgr2=0x55b559ee5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:53.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b23fff700 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ee5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b23fff700 1 -- 172.21.15.62:0/2997797572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b559c63e60 con 0x55b559ee47a0 2023-07-14T16:25:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.979+0000 7f3b237fe700 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ee5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.980+0000 7f3b23fff700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3b0c0a11f0 0x7f3b0c0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.980+0000 7f3b23fff700 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559ee4bb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0099f0 tx=0x7f3b1c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a912af45743e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.980+0000 7f3b09ffb700 1 -- 172.21.15.62:0/2997797572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c004570 con 0x55b559ee47a0 2023-07-14T16:25:53.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.980+0000 7f3b09ffb700 1 -- 172.21.15.62:0/2997797572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b1c026070 con 0x55b559ee47a0 2023-07-14T16:25:53.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.980+0000 7f3b09ffb700 1 -- 172.21.15.62:0/2997797572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c022610 con 0x55b559ee47a0 2023-07-14T16:25:53.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.981+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2997797572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 msgr2=0x55b559ee4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:53.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.981+0000 7f3b23fff700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3b0c0a11f0 0x7f3b0c0a36a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c021000 tx=0x7f3b1c01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:53.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.981+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559ee4bb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0099f0 tx=0x7f3b1c009d00 comp rx=0 tx=0).stop 2023-07-14T16:25:53.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.981+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2997797572 shutdown_connections 2023-07-14T16:25:53.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.981+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee5eb0 0x55b559ee6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.966 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.981+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559ee4bb0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.981+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2997797572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ee5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.981+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2997797572 >> 172.21.15.62:0/2997797572 conn(0x55b559eda070 msgr2=0x55b559edf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.982+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2997797572 shutdown_connections 2023-07-14T16:25:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.982+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2997797572 wait complete. 2023-07-14T16:25:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.982+0000 7f3b3d6a9b80 1 -- start start 2023-07-14T16:25:53.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.982+0000 7f3b3d6a9b80 1 Processor -- start 2023-07-14T16:25:53.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559eee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee50f0 0x55b559eee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee5eb0 0x55b559ef5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559dd5b40 con 0x55b559ee47a0 2023-07-14T16:25:53.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559dd5840 con 0x55b559ee5eb0 2023-07-14T16:25:53.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559cb99a0 con 0x55b559ee50f0 2023-07-14T16:25:53.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b3d6a9b80 10 client.0 _ll_register_callbacks cb 0x55b559dd37e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:53.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b237fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559eee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b237fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559eee380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48612/0 (socket says 172.21.15.62:48612) 2023-07-14T16:25:53.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.983+0000 7f3b237fe700 1 -- 172.21.15.62:0/427960105 learned_addr learned my addr 172.21.15.62:0/427960105 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:53.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee50f0 0x55b559eee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.970 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b23fff700 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee5eb0 0x55b559ef5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b237fe700 1 -- 172.21.15.62:0/427960105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee50f0 msgr2=0x55b559eee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:53.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b237fe700 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee50f0 0x55b559eee8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b237fe700 1 -- 172.21.15.62:0/427960105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee5eb0 msgr2=0x55b559ef5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:53.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b237fe700 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee5eb0 0x55b559ef5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b237fe700 1 -- 172.21.15.62:0/427960105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c003bb0 con 0x55b559ee47a0 2023-07-14T16:25:53.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b237fe700 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559eee380 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005f50 tx=0x7f3b14009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911c3c10e4e8704e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:53.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:53.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b02ffd700 1 -- 172.21.15.62:0/427960105 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1409b1e0 con 0x55b559ee47a0 2023-07-14T16:25:53.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b02ffd700 1 -- 172.21.15.62:0/427960105 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3b1409b410 con 0x55b559ee47a0 2023-07-14T16:25:53.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.984+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/427960105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b559d8f570 con 0x55b559ee47a0 2023-07-14T16:25:53.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b02ffd700 1 -- 172.21.15.62:0/427960105 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b1409f440 con 0x55b559ee47a0 2023-07-14T16:25:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b3d6a9b80 10 client.18405 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/427960105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b559c4cce0 con 0x55b559ee47a0 2023-07-14T16:25:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b02ffd700 1 -- 172.21.15.62:0/427960105 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b559d8f570 con 0x55b559ee47a0 2023-07-14T16:25:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b02ffd700 1 -- 172.21.15.62:0/427960105 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b140a10d0 con 0x55b559ee47a0 2023-07-14T16:25:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b3d6a9b80 10 client.18405 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b3d6a9b80 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b3d6a9b80 20 client.18405 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b559ef9dd0 0x55b559efc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b3d6a9b80 4 client.18405 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:53.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.985+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/427960105 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b559e2ef30 con 0x55b559ef9dd0 2023-07-14T16:25:53.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.986+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b559ef9dd0 0x55b559efc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.986+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b559ef9dd0 0x55b559efc1b0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6518485aa7d6ea server_cookie=8e55f34720580511 in_seq=0 out_seq=0 2023-07-14T16:25:53.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.986+0000 7f3b02ffd700 10 client.18405 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:53.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.987+0000 7f3b02ffd700 1 -- 172.21.15.62:0/427960105 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b559e2ef30 con 0x55b559ef9dd0 2023-07-14T16:25:53.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.987+0000 7f3b02ffd700 10 client.18405 handle_command_reply: tid=0 2023-07-14T16:25:53.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.987+0000 7f3b3d6a9b80 1 client.18405 shutdown 2023-07-14T16:25:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.987+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/427960105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 msgr2=0x55b559eee380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.987+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559eee380 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005f50 tx=0x7f3b14009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/427960105 shutdown_connections 2023-07-14T16:25:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b559ef9dd0 0x55b559efc1b0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee5eb0 0x55b559ef5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee47a0 0x55b559eee380 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b02ffd700 0 client.18405 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/427960105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee50f0 0x55b559eee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/427960105 >> 172.21.15.62:0/427960105 conn(0x55b559eda070 msgr2=0x55b559edeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/427960105 shutdown_connections 2023-07-14T16:25:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/427960105 wait complete. 2023-07-14T16:25:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:53.988+0000 7f3b3d6a9b80 20 client.18405 trim_cache size 0 max 16384 2023-07-14T16:25:53.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.012+0000 7f3b3d6a9b80 1 Processor -- start 2023-07-14T16:25:53.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.012+0000 7f3b3d6a9b80 1 -- start start 2023-07-14T16:25:53.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.012+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 0x55b559eec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.012+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.012+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559ee8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.012+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559ee0570 con 0x55b559ee5eb0 2023-07-14T16:25:53.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.012+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559ee54c0 con 0x55b559ee50f0 2023-07-14T16:25:53.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.012+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559edb5e0 con 0x55b559ee47a0 2023-07-14T16:25:53.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559ee8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559ee8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48624/0 (socket says 172.21.15.62:48624) 2023-07-14T16:25:53.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b23fff700 1 -- 172.21.15.62:0/3251149824 learned_addr learned my addr 172.21.15.62:0/3251149824 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:53.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b237fe700 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 0x55b559eec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b23fff700 1 -- 172.21.15.62:0/3251149824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 msgr2=0x55b559eec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:53.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b23fff700 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 0x55b559eec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b23fff700 1 -- 172.21.15.62:0/3251149824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 msgr2=0x55b559ef58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:53.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b23fff700 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b23fff700 1 -- 172.21.15.62:0/3251149824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b559edb8b0 con 0x55b559ee5eb0 2023-07-14T16:25:53.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.013+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:53.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.014+0000 7f3b23fff700 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559ee8d70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006030 tx=0x7f3b1c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2734553f9cc0ce5e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:53.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.014+0000 7f3b0a7fc700 1 -- 172.21.15.62:0/3251149824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c018640 con 0x55b559ee5eb0 2023-07-14T16:25:53.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.014+0000 7f3b0a7fc700 1 -- 172.21.15.62:0/3251149824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b1c00f040 con 0x55b559ee5eb0 2023-07-14T16:25:53.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.014+0000 7f3b0a7fc700 1 -- 172.21.15.62:0/3251149824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c02cbd0 con 0x55b559ee5eb0 2023-07-14T16:25:53.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/3251149824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 msgr2=0x55b559ee8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559ee8d70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006030 tx=0x7f3b1c013000 comp rx=0 tx=0).stop 2023-07-14T16:25:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/3251149824 shutdown_connections 2023-07-14T16:25:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559ee8d70 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 0x55b559eec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/3251149824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/3251149824 >> 172.21.15.62:0/3251149824 conn(0x55b559eda070 msgr2=0x55b559f00b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/3251149824 shutdown_connections 2023-07-14T16:25:53.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/3251149824 wait complete. 2023-07-14T16:25:53.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.015+0000 7f3b3d6a9b80 1 -- start start 2023-07-14T16:25:53.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.016+0000 7f3b3d6a9b80 1 Processor -- start 2023-07-14T16:25:53.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.016+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 0x55b559ef2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.016+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.016+0000 7f3b3d6a9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559f4dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:53.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.016+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559f4e190 con 0x55b559ee5eb0 2023-07-14T16:25:53.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.016+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559f4e2d0 con 0x55b559ee50f0 2023-07-14T16:25:53.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.016+0000 7f3b3d6a9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b559f4e440 con 0x55b559ee47a0 2023-07-14T16:25:53.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.016+0000 7f3b3d6a9b80 10 client.0 _ll_register_callbacks cb 0x55b559dd37e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:53.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53014/0 (socket says 172.21.15.62:53014) 2023-07-14T16:25:53.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 -- 172.21.15.62:0/2005246654 learned_addr learned my addr 172.21.15.62:0/2005246654 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:53.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b23fff700 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559f4dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b237fe700 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 0x55b559ef2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:53.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 -- 172.21.15.62:0/2005246654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 msgr2=0x55b559ef2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:53.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 0x55b559ef2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 -- 172.21.15.62:0/2005246654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 msgr2=0x55b559f4dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:53.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559f4dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:53.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 -- 172.21.15.62:0/2005246654 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b18009c70 con 0x55b559ee50f0 2023-07-14T16:25:53.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b22ffd700 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef2d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009e00 tx=0x7f3b18004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6be340ac7bbba9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:53.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:25:53.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.017+0000 7f3b02ffd700 1 -- 172.21.15.62:0/2005246654 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18016040 con 0x55b559ee50f0 2023-07-14T16:25:53.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b02ffd700 1 -- 172.21.15.62:0/2005246654 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3b18004a30 con 0x55b559ee50f0 2023-07-14T16:25:53.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2005246654 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b1c026070 con 0x55b559ee50f0 2023-07-14T16:25:53.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b02ffd700 1 -- 172.21.15.62:0/2005246654 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b18004bf0 con 0x55b559ee50f0 2023-07-14T16:25:53.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 10 client.28019 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:53.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2005246654 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b559c63e60 con 0x55b559ee50f0 2023-07-14T16:25:53.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b02ffd700 1 -- 172.21.15.62:0/2005246654 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b1c026070 con 0x55b559ee50f0 2023-07-14T16:25:53.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b02ffd700 1 -- 172.21.15.62:0/2005246654 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b180037a0 con 0x55b559ee50f0 2023-07-14T16:25:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 10 client.28019 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 10 client.28019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 20 client.28019 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:54.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b559f4ff20 0x55b559f52330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 4 client.28019 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:25:54.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.018+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2005246654 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b559df10d0 con 0x55b559f4ff20 2023-07-14T16:25:54.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.019+0000 7f3b237fe700 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b559f4ff20 0x55b559f52330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.019+0000 7f3b237fe700 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b559f4ff20 0x55b559f52330 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af79577cd0db0251 server_cookie=6c1f7db25e6e6440 in_seq=0 out_seq=0 2023-07-14T16:25:54.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.019+0000 7f3b02ffd700 10 client.28019 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:54.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.020+0000 7f3b02ffd700 1 -- 172.21.15.62:0/2005246654 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b559df10d0 con 0x55b559f4ff20 2023-07-14T16:25:54.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.020+0000 7f3b02ffd700 10 client.28019 handle_command_reply: tid=0 2023-07-14T16:25:54.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.020+0000 7f3b3d6a9b80 1 client.28019 shutdown 2023-07-14T16:25:54.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2005246654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 msgr2=0x55b559ef2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef2d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009e00 tx=0x7f3b18004040 comp rx=0 tx=0).stop 2023-07-14T16:25:54.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2005246654 shutdown_connections 2023-07-14T16:25:54.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b559ee5eb0 0x55b559f4dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b559ee47a0 0x55b559ef2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b559f4ff20 0x55b559f52330 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 --2- 172.21.15.62:0/2005246654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b559ee50f0 0x55b559ef2d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2005246654 >> 172.21.15.62:0/2005246654 conn(0x55b559eda070 msgr2=0x55b559ee8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:54.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b02ffd700 0 client.28019 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:54.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2005246654 shutdown_connections 2023-07-14T16:25:54.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 1 -- 172.21.15.62:0/2005246654 wait complete. 2023-07-14T16:25:54.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.021+0000 7f3b3d6a9b80 20 client.28019 trim_cache size 0 max 16384 2023-07-14T16:25:54.006 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:25:54.007 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:25:54.007 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "1eca2f9a-b18b-4d61-b84d-7cc506bd1a9b", 2023-07-14T16:25:54.007 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:25:54.007 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:25:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 -- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3b0c0a11f0 msgr2=0x7f3b0c0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3b0c0a11f0 0x7f3b0c0a36a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c021000 tx=0x7f3b1c01e000 comp rx=0 tx=0).stop 2023-07-14T16:25:54.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 -- 172.21.15.62:0/1432160555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b24101c00 msgr2=0x7f3b241977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b24101c00 0x7f3b241977e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3b14005910 tx=0x7f3b14005740 comp rx=0 tx=0).stop 2023-07-14T16:25:54.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 -- 172.21.15.62:0/1432160555 shutdown_connections 2023-07-14T16:25:54.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b24103780 0x7f3b2419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3b0c0a11f0 0x7f3b0c0a36a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b24101c00 0x7f3b241977e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 --2- 172.21.15.62:0/1432160555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b24102df0 0x7f3b24197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 -- 172.21.15.62:0/1432160555 >> 172.21.15.62:0/1432160555 conn(0x7f3b240fd220 msgr2=0x7f3b241062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:54.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 -- 172.21.15.62:0/1432160555 shutdown_connections 2023-07-14T16:25:54.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.022+0000 7f3b28b9d700 1 -- 172.21.15.62:0/1432160555 wait complete. 2023-07-14T16:25:54.021 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:25:54.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 -- 172.21.15.62:0/3965456101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44780ffae0 msgr2=0x7f44780ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 --2- 172.21.15.62:0/3965456101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44780ffae0 0x7f44780ffef0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4474009990 tx=0x7f4474009ca0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 -- 172.21.15.62:0/3965456101 shutdown_connections 2023-07-14T16:25:54.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 --2- 172.21.15.62:0/3965456101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4478101660 0x7f4478107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 --2- 172.21.15.62:0/3965456101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44780ffae0 0x7f44780ffef0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 --2- 172.21.15.62:0/3965456101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4478100cd0 0x7f4478101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 -- 172.21.15.62:0/3965456101 >> 172.21.15.62:0/3965456101 conn(0x7f44780fb0e0 msgr2=0x7f44780fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:54.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 -- 172.21.15.62:0/3965456101 shutdown_connections 2023-07-14T16:25:54.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.290+0000 7f447fdff700 1 -- 172.21.15.62:0/3965456101 wait complete. 2023-07-14T16:25:54.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.292+0000 7f447fdff700 1 Processor -- start 2023-07-14T16:25:54.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.292+0000 7f447fdff700 1 -- start start 2023-07-14T16:25:54.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.292+0000 7f447fdff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44780ffae0 0x7f44781979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.292+0000 7f447fdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4478100cd0 0x7f4478197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.292+0000 7f447fdff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4478101660 0x7f447819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.292+0000 7f447fdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819f550 con 0x7f4478100cd0 2023-07-14T16:25:54.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.292+0000 7f447fdff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819f690 con 0x7f4478101660 2023-07-14T16:25:54.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.292+0000 7f447fdff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819f820 con 0x7f44780ffae0 2023-07-14T16:25:54.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4478100cd0 0x7f4478197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4478100cd0 0x7f4478197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48648/0 (socket says 172.21.15.62:48648) 2023-07-14T16:25:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 -- 172.21.15.62:0/3469893433 learned_addr learned my addr 172.21.15.62:0/3469893433 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447e39c700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4478101660 0x7f447819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447db9b700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44780ffae0 0x7f44781979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 -- 172.21.15.62:0/3469893433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44780ffae0 msgr2=0x7f44781979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44780ffae0 0x7f44781979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 -- 172.21.15.62:0/3469893433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4478101660 msgr2=0x7f447819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4478101660 0x7f447819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 -- 172.21.15.62:0/3469893433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740095f0 con 0x7f4478100cd0 2023-07-14T16:25:54.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447db9b700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44780ffae0 0x7f44781979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:25:54.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.293+0000 7f447d39a700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4478100cd0 0x7f4478197f10 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f446800d950 tx=0x7f446800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:54.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.294+0000 7f446effd700 1 -- 172.21.15.62:0/3469893433 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468009940 con 0x7f4478100cd0 2023-07-14T16:25:54.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.294+0000 7f446effd700 1 -- 172.21.15.62:0/3469893433 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4468010460 con 0x7f4478100cd0 2023-07-14T16:25:54.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.294+0000 7f447fdff700 1 -- 172.21.15.62:0/3469893433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447819fb00 con 0x7f4478100cd0 2023-07-14T16:25:54.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.294+0000 7f446effd700 1 -- 172.21.15.62:0/3469893433 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800f620 con 0x7f4478100cd0 2023-07-14T16:25:54.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.294+0000 7f447fdff700 1 -- 172.21.15.62:0/3469893433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44781a0060 con 0x7f4478100cd0 2023-07-14T16:25:54.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.298+0000 7f446effd700 1 -- 172.21.15.62:0/3469893433 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f44680105d0 con 0x7f4478100cd0 2023-07-14T16:25:54.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f446effd700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44640a11d0 0x7f44640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f4493811b80 1 Processor -- start 2023-07-14T16:25:54.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f446effd700 1 -- 172.21.15.62:0/3469893433 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44680cc710 con 0x7f4478100cd0 2023-07-14T16:25:54.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f4493811b80 1 -- start start 2023-07-14T16:25:54.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f447db9b700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44640a11d0 0x7f44640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f4493811b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da234d0 0x560f0da238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f4493811b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da246e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f4493811b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 0x560f0da250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f4493811b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0d928cb0 con 0x560f0da23e20 2023-07-14T16:25:54.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f4493811b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0d97ba40 con 0x560f0da234d0 2023-07-14T16:25:54.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.300+0000 7f4493811b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0d974ed0 con 0x560f0da24c70 2023-07-14T16:25:54.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447d39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da246e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447d39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da246e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48664/0 (socket says 172.21.15.62:48664) 2023-07-14T16:25:54.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447d39a700 1 -- 172.21.15.62:0/1632551376 learned_addr learned my addr 172.21.15.62:0/1632551376 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:54.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447e39c700 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 0x560f0da250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447db9b700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44640a11d0 0x7f44640a3680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f44740098e0 tx=0x7f4474005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:54.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447db9b700 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da234d0 0x560f0da238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447d39a700 1 -- 172.21.15.62:0/1632551376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 msgr2=0x560f0da250e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:54.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447d39a700 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 0x560f0da250e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447d39a700 1 -- 172.21.15.62:0/1632551376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da234d0 msgr2=0x560f0da238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:54.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447d39a700 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da234d0 0x560f0da238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447d39a700 1 -- 172.21.15.62:0/1632551376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f0d7a2de0 con 0x560f0da23e20 2023-07-14T16:25:54.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.301+0000 7f447e39c700 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 0x560f0da250e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:54.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.302+0000 7f447d39a700 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da246e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4468099a60 tx=0x7f4468099430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5df53e9ba827bec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:54.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.302+0000 7f4463fff700 1 -- 172.21.15.62:0/1632551376 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468099ad0 con 0x560f0da23e20 2023-07-14T16:25:54.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.302+0000 7f4463fff700 1 -- 172.21.15.62:0/1632551376 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f446809db00 con 0x560f0da23e20 2023-07-14T16:25:54.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.302+0000 7f4463fff700 1 -- 172.21.15.62:0/1632551376 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44680a1a90 con 0x560f0da23e20 2023-07-14T16:25:54.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.302+0000 7f4493811b80 1 -- 172.21.15.62:0/1632551376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 msgr2=0x560f0da246e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.302+0000 7f4493811b80 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da246e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4468099a60 tx=0x7f4468099430 comp rx=0 tx=0).stop 2023-07-14T16:25:54.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.303+0000 7f4493811b80 1 -- 172.21.15.62:0/1632551376 shutdown_connections 2023-07-14T16:25:54.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.303+0000 7f4493811b80 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 0x560f0da250e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.303+0000 7f4493811b80 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da234d0 0x560f0da238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.303+0000 7f4493811b80 1 --2- 172.21.15.62:0/1632551376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da246e0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.303+0000 7f4493811b80 1 -- 172.21.15.62:0/1632551376 >> 172.21.15.62:0/1632551376 conn(0x560f0da18da0 msgr2=0x560f0da1e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:54.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.303+0000 7f4493811b80 1 -- 172.21.15.62:0/1632551376 shutdown_connections 2023-07-14T16:25:54.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.303+0000 7f4493811b80 1 -- 172.21.15.62:0/1632551376 wait complete. 2023-07-14T16:25:54.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.303+0000 7f4493811b80 1 -- start start 2023-07-14T16:25:54.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f4493811b80 1 Processor -- start 2023-07-14T16:25:54.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f4493811b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f4493811b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da2d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f4493811b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da24c70 0x560f0da344c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f4493811b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0d914ac0 con 0x560f0da23e20 2023-07-14T16:25:54.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f4493811b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0d9147c0 con 0x560f0da24c70 2023-07-14T16:25:54.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f4493811b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0d7f8920 con 0x560f0da234d0 2023-07-14T16:25:54.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f4493811b80 10 client.0 _ll_register_callbacks cb 0x560f0d912760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:54.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f447d39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da2d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f447d39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da2d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48668/0 (socket says 172.21.15.62:48668) 2023-07-14T16:25:54.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.304+0000 7f447d39a700 1 -- 172.21.15.62:0/323768150 learned_addr learned my addr 172.21.15.62:0/323768150 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:54.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f447db9b700 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f447e39c700 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da24c70 0x560f0da344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f447d39a700 1 -- 172.21.15.62:0/323768150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 msgr2=0x560f0da2cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:54.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f447d39a700 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f447d39a700 1 -- 172.21.15.62:0/323768150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da24c70 msgr2=0x560f0da344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:54.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f447d39a700 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da24c70 0x560f0da344c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f447d39a700 1 -- 172.21.15.62:0/323768150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4470011230 con 0x560f0da23e20 2023-07-14T16:25:54.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f447d39a700 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da2d480 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f44680997d0 tx=0x7f4468010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8610e00d63b078c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:54.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f44617fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:54.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f44617fa700 1 -- 172.21.15.62:0/323768150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44680a1f20 con 0x560f0da23e20 2023-07-14T16:25:54.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f44617fa700 1 -- 172.21.15.62:0/323768150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44680a1a90 con 0x560f0da23e20 2023-07-14T16:25:54.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.305+0000 7f4493811b80 1 -- 172.21.15.62:0/323768150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f0d8ce4f0 con 0x560f0da23e20 2023-07-14T16:25:54.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f44617fa700 1 -- 172.21.15.62:0/323768150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44680a1c50 con 0x560f0da23e20 2023-07-14T16:25:54.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f4493811b80 10 client.18423 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:54.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f4493811b80 1 -- 172.21.15.62:0/323768150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f44781a0060 con 0x560f0da23e20 2023-07-14T16:25:54.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f44617fa700 1 -- 172.21.15.62:0/323768150 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f446807c080 con 0x560f0da23e20 2023-07-14T16:25:54.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f44617fa700 1 -- 172.21.15.62:0/323768150 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4468009aa0 con 0x560f0da23e20 2023-07-14T16:25:54.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f4493811b80 10 client.18423 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:54.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f4493811b80 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:54.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f4493811b80 20 client.18423 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:54.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f4493811b80 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560f0da38a00 0x560f0da3ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f4493811b80 4 client.18423 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:54.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.306+0000 7f4493811b80 1 -- 172.21.15.62:0/323768150 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f0d96deb0 con 0x560f0da38a00 2023-07-14T16:25:54.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.307+0000 7f447db9b700 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560f0da38a00 0x560f0da3ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.307+0000 7f447db9b700 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560f0da38a00 0x560f0da3ae10 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995620df65ba47ef server_cookie=9d3cefb7a24ae005 in_seq=0 out_seq=0 2023-07-14T16:25:54.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.307+0000 7f44617fa700 10 client.18423 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:54.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.308+0000 7f44617fa700 1 -- 172.21.15.62:0/323768150 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560f0d96deb0 con 0x560f0da38a00 2023-07-14T16:25:54.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.308+0000 7f44617fa700 10 client.18423 handle_command_reply: tid=0 2023-07-14T16:25:54.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.308+0000 7f4493811b80 1 client.18423 shutdown 2023-07-14T16:25:54.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.308+0000 7f4493811b80 1 -- 172.21.15.62:0/323768150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 msgr2=0x560f0da2d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.308+0000 7f4493811b80 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da2d480 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f44680997d0 tx=0x7f4468010a00 comp rx=0 tx=0).stop 2023-07-14T16:25:54.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 1 -- 172.21.15.62:0/323768150 shutdown_connections 2023-07-14T16:25:54.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560f0da38a00 0x560f0da3ae10 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da24c70 0x560f0da344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 1 --2- 172.21.15.62:0/323768150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da23e20 0x560f0da2d480 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f44617fa700 0 client.18423 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:54.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 1 -- 172.21.15.62:0/323768150 >> 172.21.15.62:0/323768150 conn(0x560f0da18da0 msgr2=0x560f0da1d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:54.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 1 -- 172.21.15.62:0/323768150 shutdown_connections 2023-07-14T16:25:54.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 1 -- 172.21.15.62:0/323768150 wait complete. 2023-07-14T16:25:54.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.309+0000 7f4493811b80 20 client.18423 trim_cache size 0 max 16384 2023-07-14T16:25:54.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.333+0000 7f4493811b80 1 Processor -- start 2023-07-14T16:25:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.333+0000 7f4493811b80 1 -- start start 2023-07-14T16:25:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f4493811b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f4493811b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 0x560f0da34480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f4493811b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da24c70 0x560f0da2d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f4493811b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0da28280 con 0x560f0da24c70 2023-07-14T16:25:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f4493811b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0da1f4d0 con 0x560f0da23e20 2023-07-14T16:25:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f4493811b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0da2c3c0 con 0x560f0da234d0 2023-07-14T16:25:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f447e39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da24c70 0x560f0da2d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f447e39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da24c70 0x560f0da2d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48680/0 (socket says 172.21.15.62:48680) 2023-07-14T16:25:54.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f447e39c700 1 -- 172.21.15.62:0/3262485592 learned_addr learned my addr 172.21.15.62:0/3262485592 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:54.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f447db9b700 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.334+0000 7f447d39a700 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 0x560f0da34480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f447db9b700 1 -- 172.21.15.62:0/3262485592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 msgr2=0x560f0da34480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:54.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f447db9b700 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 0x560f0da34480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f447db9b700 1 -- 172.21.15.62:0/3262485592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da24c70 msgr2=0x560f0da2d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f447db9b700 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da24c70 0x560f0da2d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f447db9b700 1 -- 172.21.15.62:0/3262485592 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f0d78bc60 con 0x560f0da234d0 2023-07-14T16:25:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f447e39c700 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da24c70 0x560f0da2d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:54.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f447db9b700 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4474003f60 tx=0x7f447401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e300ca6d05e847c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:54.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f446cbff700 1 -- 172.21.15.62:0/3262485592 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447402b040 con 0x560f0da234d0 2023-07-14T16:25:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.335+0000 7f446cbff700 1 -- 172.21.15.62:0/3262485592 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f447402c040 con 0x560f0da234d0 2023-07-14T16:25:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f446cbff700 1 -- 172.21.15.62:0/3262485592 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474005660 con 0x560f0da234d0 2023-07-14T16:25:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 -- 172.21.15.62:0/3262485592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 msgr2=0x560f0da2cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4474003f60 tx=0x7f447401f000 comp rx=0 tx=0).stop 2023-07-14T16:25:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 -- 172.21.15.62:0/3262485592 shutdown_connections 2023-07-14T16:25:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da24c70 0x560f0da2d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da234d0 0x560f0da2cf40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 --2- 172.21.15.62:0/3262485592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 0x560f0da34480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 -- 172.21.15.62:0/3262485592 >> 172.21.15.62:0/3262485592 conn(0x560f0da18da0 msgr2=0x560f0da27b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 -- 172.21.15.62:0/3262485592 shutdown_connections 2023-07-14T16:25:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.336+0000 7f4493811b80 1 -- 172.21.15.62:0/3262485592 wait complete. 2023-07-14T16:25:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 1 -- start start 2023-07-14T16:25:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 1 Processor -- start 2023-07-14T16:25:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da234d0 0x560f0da31920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 0x560f0da31e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 0x560f0da32e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0da333b0 con 0x560f0da234d0 2023-07-14T16:25:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0da33520 con 0x560f0da23e20 2023-07-14T16:25:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0da336e0 con 0x560f0da24c70 2023-07-14T16:25:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.337+0000 7f4493811b80 10 client.0 _ll_register_callbacks cb 0x7f4474005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da234d0 0x560f0da31920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da234d0 0x560f0da31920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48682/0 (socket says 172.21.15.62:48682) 2023-07-14T16:25:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 -- 172.21.15.62:0/3410537279 learned_addr learned my addr 172.21.15.62:0/3410537279 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447d39a700 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 0x560f0da31e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 -- 172.21.15.62:0/3410537279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 msgr2=0x560f0da32e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 0x560f0da32e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 -- 172.21.15.62:0/3410537279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 msgr2=0x560f0da31e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 0x560f0da31e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 -- 172.21.15.62:0/3410537279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44700093b0 con 0x560f0da234d0 2023-07-14T16:25:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f447db9b700 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da234d0 0x560f0da31920 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4474009350 tx=0x7f4474003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3c527d11a7d97c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:54.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.338+0000 7f44617fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:54.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.339+0000 7f44617fa700 1 -- 172.21.15.62:0/3410537279 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740329d0 con 0x560f0da234d0 2023-07-14T16:25:54.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.339+0000 7f44617fa700 1 -- 172.21.15.62:0/3410537279 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4474032b30 con 0x560f0da234d0 2023-07-14T16:25:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.339+0000 7f4493811b80 1 -- 172.21.15.62:0/3410537279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f446809db00 con 0x560f0da234d0 2023-07-14T16:25:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.339+0000 7f44617fa700 1 -- 172.21.15.62:0/3410537279 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4474032cf0 con 0x560f0da234d0 2023-07-14T16:25:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.339+0000 7f4493811b80 10 client.18429 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.339+0000 7f4493811b80 1 -- 172.21.15.62:0/3410537279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f4478108850 con 0x560f0da234d0 2023-07-14T16:25:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.339+0000 7f44617fa700 1 -- 172.21.15.62:0/3410537279 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f446809db00 con 0x560f0da234d0 2023-07-14T16:25:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.339+0000 7f44617fa700 1 -- 172.21.15.62:0/3410537279 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4474045070 con 0x560f0da234d0 2023-07-14T16:25:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f4493811b80 10 client.18429 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f4493811b80 10 client.18429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f4493811b80 20 client.18429 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f4493811b80 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560f0da8eb00 0x560f0da90f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f4493811b80 4 client.18429 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:25:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f4493811b80 1 -- 172.21.15.62:0/3410537279 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f0d930050 con 0x560f0da8eb00 2023-07-14T16:25:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f447d39a700 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560f0da8eb00 0x560f0da90f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f447d39a700 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560f0da8eb00 0x560f0da90f10 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b68df70b6a8de70 server_cookie=1f91052f55933043 in_seq=0 out_seq=0 2023-07-14T16:25:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.340+0000 7f44617fa700 10 client.18429 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:54.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f44617fa700 1 -- 172.21.15.62:0/3410537279 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560f0d930050 con 0x560f0da8eb00 2023-07-14T16:25:54.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f44617fa700 10 client.18429 handle_command_reply: tid=0 2023-07-14T16:25:54.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 client.18429 shutdown 2023-07-14T16:25:54.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 -- 172.21.15.62:0/3410537279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da234d0 msgr2=0x560f0da31920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da234d0 0x560f0da31920 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4474009350 tx=0x7f4474003910 comp rx=0 tx=0).stop 2023-07-14T16:25:54.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 -- 172.21.15.62:0/3410537279 shutdown_connections 2023-07-14T16:25:54.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0da24c70 0x560f0da32e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560f0da234d0 0x560f0da31920 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560f0da8eb00 0x560f0da90f10 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 --2- 172.21.15.62:0/3410537279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560f0da23e20 0x560f0da31e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f44617fa700 0 client.18429 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:54.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 -- 172.21.15.62:0/3410537279 >> 172.21.15.62:0/3410537279 conn(0x560f0da18da0 msgr2=0x560f0da2cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:54.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 -- 172.21.15.62:0/3410537279 shutdown_connections 2023-07-14T16:25:54.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 1 -- 172.21.15.62:0/3410537279 wait complete. 2023-07-14T16:25:54.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.341+0000 7f4493811b80 20 client.18429 trim_cache size 0 max 16384 2023-07-14T16:25:54.324 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:25:54.324 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:25:54.324 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:25:54.324 INFO:teuthology.orchestra.run.smithi062.stdout: "1eca2f9a-b18b-4d61-b84d-7cc506bd1a9b": { 2023-07-14T16:25:54.324 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:25:54.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "1eca2f9a-b18b-4d61-b84d-7cc506bd1a9b", 2023-07-14T16:25:54.325 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:25:54.325 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:25:54.325 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:25:54.325 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:25:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 -- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44640a11d0 msgr2=0x7f44640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44640a11d0 0x7f44640a3680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f44740098e0 tx=0x7f4474005fd0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 -- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4478100cd0 msgr2=0x7f4478197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4478100cd0 0x7f4478197f10 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f446800d950 tx=0x7f446800dc60 comp rx=0 tx=0).stop 2023-07-14T16:25:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 -- 172.21.15.62:0/3469893433 shutdown_connections 2023-07-14T16:25:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44640a11d0 0x7f44640a3680 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4478101660 0x7f447819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44780ffae0 0x7f44781979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 --2- 172.21.15.62:0/3469893433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4478100cd0 0x7f4478197f10 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 -- 172.21.15.62:0/3469893433 >> 172.21.15.62:0/3469893433 conn(0x7f44780fb0e0 msgr2=0x7f44780fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 -- 172.21.15.62:0/3469893433 shutdown_connections 2023-07-14T16:25:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:25:54.342+0000 7f447fdff700 1 -- 172.21.15.62:0/3469893433 wait complete. 2023-07-14T16:25:54.336 INFO:tasks.cephfs.filesystem:scrub status for tag:1eca2f9a-b18b-4d61-b84d-7cc506bd1a9b - {'path': '/', 'tag': '1eca2f9a-b18b-4d61-b84d-7cc506bd1a9b', 'options': 'recursive,force'} 2023-07-14T16:25:54.400 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:25:54.029+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v205>,<0x10000000003/client.0 v206>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v218>,<0x10000000001/sv_0 v232>,<0x10000000000/_nogroup v226>,<0x1/volumes v216>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:25:55.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: pgmap v665: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 17 MiB/s wr, 16 op/s 2023-07-14T16:25:55.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:25:55.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:25:55.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:25:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:25:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:25:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:54 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: pgmap v665: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 17 MiB/s wr, 16 op/s 2023-07-14T16:25:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:25:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:25:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:25:55.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:25:55.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:25:55.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:54 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: pgmap v665: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 17 MiB/s wr, 16 op/s 2023-07-14T16:25:55.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:25:55.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:25:55.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:25:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:25:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:25:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:55.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:54 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:25:57.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:56 smithi062 ceph-mon[95543]: pgmap v666: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:25:57.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:56 smithi081 ceph-mon[127795]: pgmap v666: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:25:57.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:56 smithi164 ceph-mon[97427]: pgmap v666: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:25:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 -- 172.21.15.62:0/1318501376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6401078f0 msgr2=0x7ff640109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 --2- 172.21.15.62:0/1318501376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6401078f0 0x7ff640109da0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff638009ec0 tx=0x7ff63800b330 comp rx=0 tx=0).stop 2023-07-14T16:25:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 -- 172.21.15.62:0/1318501376 shutdown_connections 2023-07-14T16:25:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 --2- 172.21.15.62:0/1318501376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6401078f0 0x7ff640109da0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 --2- 172.21.15.62:0/1318501376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff640068640 0x7ff640104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 --2- 172.21.15.62:0/1318501376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff640104fd0 0x7ff6401073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 -- 172.21.15.62:0/1318501376 >> 172.21.15.62:0/1318501376 conn(0x7ff6400fa000 msgr2=0x7ff6400fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 -- 172.21.15.62:0/1318501376 shutdown_connections 2023-07-14T16:25:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.156+0000 7ff6479ff700 1 -- 172.21.15.62:0/1318501376 wait complete. 2023-07-14T16:25:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff6479ff700 1 Processor -- start 2023-07-14T16:25:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff6479ff700 1 -- start start 2023-07-14T16:25:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff6479ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff640068640 0x7ff640197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff6479ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640104fd0 0x7ff640198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff6479ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6401078f0 0x7ff640198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff6479ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401995f0 con 0x7ff640104fd0 2023-07-14T16:25:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff6479ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640199760 con 0x7ff6401078f0 2023-07-14T16:25:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff6479ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640199920 con 0x7ff640068640 2023-07-14T16:25:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff644f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640104fd0 0x7ff640198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.158+0000 7ff644f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640104fd0 0x7ff640198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48708/0 (socket says 172.21.15.62:48708) 2023-07-14T16:25:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff644f9a700 1 -- 172.21.15.62:0/963361465 learned_addr learned my addr 172.21.15.62:0/963361465 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff64579b700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff640068640 0x7ff640197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff645f9c700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6401078f0 0x7ff640198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff644f9a700 1 -- 172.21.15.62:0/963361465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff640068640 msgr2=0x7ff640197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff644f9a700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff640068640 0x7ff640197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff644f9a700 1 -- 172.21.15.62:0/963361465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6401078f0 msgr2=0x7ff640198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff644f9a700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6401078f0 0x7ff640198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff644f9a700 1 -- 172.21.15.62:0/963361465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638005e70 con 0x7ff640104fd0 2023-07-14T16:25:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff645f9c700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6401078f0 0x7ff640198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:25:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.159+0000 7ff644f9a700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640104fd0 0x7ff640198110 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff63000d950 tx=0x7ff63000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.160+0000 7ff6367fc700 1 -- 172.21.15.62:0/963361465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630009940 con 0x7ff640104fd0 2023-07-14T16:25:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.160+0000 7ff6367fc700 1 -- 172.21.15.62:0/963361465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff630010460 con 0x7ff640104fd0 2023-07-14T16:25:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.160+0000 7ff6479ff700 1 -- 172.21.15.62:0/963361465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff640199c00 con 0x7ff640104fd0 2023-07-14T16:25:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.160+0000 7ff6367fc700 1 -- 172.21.15.62:0/963361465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63000f620 con 0x7ff640104fd0 2023-07-14T16:25:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.160+0000 7ff6479ff700 1 -- 172.21.15.62:0/963361465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401a03c0 con 0x7ff640104fd0 2023-07-14T16:25:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.164+0000 7ff6367fc700 1 -- 172.21.15.62:0/963361465 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff6300105d0 con 0x7ff640104fd0 2023-07-14T16:25:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.165+0000 7ff65b3ccb80 1 Processor -- start 2023-07-14T16:25:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.166+0000 7ff65b3ccb80 1 -- start start 2023-07-14T16:25:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.166+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e6c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.166+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e6d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.166+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e6db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.166+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9d71c50 con 0x55bca9e6d6e0 2023-07-14T16:25:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.166+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9dc49e0 con 0x55bca9e6c840 2023-07-14T16:25:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.166+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9dbde70 con 0x55bca9cc86b0 2023-07-14T16:25:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.166+0000 7ff6367fc700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff62c0a11d0 0x7ff62c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff6367fc700 1 -- 172.21.15.62:0/963361465 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff6300cc190 con 0x7ff640104fd0 2023-07-14T16:25:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e6c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff644f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e6db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e6c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:57588/0 (socket says 172.21.15.62:57588) 2023-07-14T16:25:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 -- 172.21.15.62:0/3879336645 learned_addr learned my addr 172.21.15.62:0/3879336645 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff64579b700 1 --2- 172.21.15.62:0/3879336645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e6d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff62c0a11d0 0x7ff62c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 -- 172.21.15.62:0/3879336645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 msgr2=0x55bca9e6d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 --2- 172.21.15.62:0/3879336645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e6d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 -- 172.21.15.62:0/3879336645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 msgr2=0x55bca9e6db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 --2- 172.21.15.62:0/3879336645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e6db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.167+0000 7ff645f9c700 1 -- 172.21.15.62:0/3879336645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bca9bebd80 con 0x55bca9cc86b0 2023-07-14T16:25:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.168+0000 7ff645f9c700 1 --2- 172.21.15.62:0/3879336645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e6c300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff638009fd0 tx=0x7ff638009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4343c640200b5a18 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.168+0000 7ff645f9c700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff62c0a11d0 0x7ff62c0a3680 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e000 tx=0x7ff638024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.168+0000 7ff62b7fe700 1 -- 172.21.15.62:0/3879336645 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800e030 con 0x55bca9cc86b0 2023-07-14T16:25:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.168+0000 7ff62b7fe700 1 -- 172.21.15.62:0/3879336645 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff638007c10 con 0x55bca9cc86b0 2023-07-14T16:25:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.168+0000 7ff62b7fe700 1 -- 172.21.15.62:0/3879336645 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63801e430 con 0x55bca9cc86b0 2023-07-14T16:25:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/3879336645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 msgr2=0x55bca9e6c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/3879336645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e6c300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff638009fd0 tx=0x7ff638009480 comp rx=0 tx=0).stop 2023-07-14T16:25:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/3879336645 shutdown_connections 2023-07-14T16:25:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/3879336645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e6db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/3879336645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e6c300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/3879336645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e6d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/3879336645 >> 172.21.15.62:0/3879336645 conn(0x55bca9e61be0 msgr2=0x55bca9e66cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/3879336645 shutdown_connections 2023-07-14T16:25:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.169+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/3879336645 wait complete. 2023-07-14T16:25:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.170+0000 7ff65b3ccb80 1 -- start start 2023-07-14T16:25:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.170+0000 7ff65b3ccb80 1 Processor -- start 2023-07-14T16:25:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e75e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9d5da60 con 0x55bca9e6d6e0 2023-07-14T16:25:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9d5d760 con 0x55bca9e6c840 2023-07-14T16:25:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9c418c0 con 0x55bca9cc86b0 2023-07-14T16:25:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff65b3ccb80 10 client.0 _ll_register_callbacks cb 0x55bca9d5b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff645f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff644f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e75e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff645f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48732/0 (socket says 172.21.15.62:48732) 2023-07-14T16:25:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff645f9c700 1 -- 172.21.15.62:0/2751803312 learned_addr learned my addr 172.21.15.62:0/2751803312 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.171+0000 7ff64579b700 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff644f9a700 1 -- 172.21.15.62:0/2751803312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 msgr2=0x55bca9e758d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff644f9a700 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e758d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff644f9a700 1 -- 172.21.15.62:0/2751803312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 msgr2=0x55bca9e7ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff644f9a700 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff644f9a700 1 -- 172.21.15.62:0/2751803312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64010aab0 con 0x55bca9e6c840 2023-07-14T16:25:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff645f9c700 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff644f9a700 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e75e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff6300a1470 tx=0x7ff630098f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9858bf6d739ec5f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff628ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:25:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff628ff9700 1 -- 172.21.15.62:0/2751803312 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63009d350 con 0x55bca9e6c840 2023-07-14T16:25:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff628ff9700 1 -- 172.21.15.62:0/2751803312 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff630099550 con 0x55bca9e6c840 2023-07-14T16:25:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.172+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2751803312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bca9d17490 con 0x55bca9e6c840 2023-07-14T16:25:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff628ff9700 1 -- 172.21.15.62:0/2751803312 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff63009d580 con 0x55bca9e6c840 2023-07-14T16:25:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff65b3ccb80 10 client.28046 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2751803312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bca9bd4c00 con 0x55bca9e6c840 2023-07-14T16:25:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff628ff9700 1 -- 172.21.15.62:0/2751803312 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bca9d17490 con 0x55bca9e6c840 2023-07-14T16:25:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff628ff9700 1 -- 172.21.15.62:0/2751803312 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6300af070 con 0x55bca9e6c840 2023-07-14T16:25:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff65b3ccb80 10 client.28046 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff65b3ccb80 10 client.28046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff65b3ccb80 20 client.28046 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bca9e81670 0x55bca9e83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff65b3ccb80 4 client.28046 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:25:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.173+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2751803312 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bca9db6e50 con 0x55bca9e81670 2023-07-14T16:25:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.174+0000 7ff64579b700 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bca9e81670 0x55bca9e83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.174+0000 7ff64579b700 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bca9e81670 0x55bca9e83a80 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0786dcc0322a5d server_cookie=fe93b5822c98167e in_seq=0 out_seq=0 2023-07-14T16:25:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.174+0000 7ff628ff9700 10 client.28046 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.175+0000 7ff628ff9700 1 -- 172.21.15.62:0/2751803312 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bca9db6e50 con 0x55bca9e81670 2023-07-14T16:25:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.175+0000 7ff628ff9700 10 client.28046 handle_command_reply: tid=0 2023-07-14T16:25:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.175+0000 7ff65b3ccb80 1 client.28046 shutdown 2023-07-14T16:25:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.175+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2751803312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 msgr2=0x55bca9e75e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.175+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e75e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff6300a1470 tx=0x7ff630098f30 comp rx=0 tx=0).stop 2023-07-14T16:25:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2751803312 shutdown_connections 2023-07-14T16:25:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9cc86b0 0x55bca9e758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bca9e81670 0x55bca9e83a80 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2751803312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6c840 0x55bca9e75e10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff628ff9700 0 client.28046 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2751803312 >> 172.21.15.62:0/2751803312 conn(0x55bca9e61be0 msgr2=0x55bca9e664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2751803312 shutdown_connections 2023-07-14T16:25:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2751803312 wait complete. 2023-07-14T16:25:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.176+0000 7ff65b3ccb80 20 client.28046 trim_cache size 0 max 16384 2023-07-14T16:25:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.200+0000 7ff65b3ccb80 1 Processor -- start 2023-07-14T16:25:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.200+0000 7ff65b3ccb80 1 -- start start 2023-07-14T16:25:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9cc86b0 0x55bca9e74560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6c840 0x55bca9e7ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6d6e0 0x55bca9e758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9e767d0 con 0x55bca9e6c840 2023-07-14T16:25:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9e74d80 con 0x55bca9cc86b0 2023-07-14T16:25:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9e75440 con 0x55bca9e6d6e0 2023-07-14T16:25:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff644f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6c840 0x55bca9e7ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff644f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6c840 0x55bca9e7ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48738/0 (socket says 172.21.15.62:48738) 2023-07-14T16:25:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff644f9a700 1 -- 172.21.15.62:0/1342319709 learned_addr learned my addr 172.21.15.62:0/1342319709 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff64579b700 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9cc86b0 0x55bca9e74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff645f9c700 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6d6e0 0x55bca9e758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff644f9a700 1 -- 172.21.15.62:0/1342319709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6d6e0 msgr2=0x55bca9e758d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff644f9a700 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6d6e0 0x55bca9e758d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff644f9a700 1 -- 172.21.15.62:0/1342319709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9cc86b0 msgr2=0x55bca9e74560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff644f9a700 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9cc86b0 0x55bca9e74560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.201+0000 7ff644f9a700 1 -- 172.21.15.62:0/1342319709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bca9c121f0 con 0x55bca9e6c840 2023-07-14T16:25:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.202+0000 7ff644f9a700 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6c840 0x55bca9e7ce10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff630010b70 tx=0x7ff630095240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44a26626f173c47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.202+0000 7ff62bfff700 1 -- 172.21.15.62:0/1342319709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630099550 con 0x55bca9e6c840 2023-07-14T16:25:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.202+0000 7ff62bfff700 1 -- 172.21.15.62:0/1342319709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff63009d580 con 0x55bca9e6c840 2023-07-14T16:25:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.202+0000 7ff62bfff700 1 -- 172.21.15.62:0/1342319709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63009d390 con 0x55bca9e6c840 2023-07-14T16:25:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/1342319709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6c840 msgr2=0x55bca9e7ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6c840 0x55bca9e7ce10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff630010b70 tx=0x7ff630095240 comp rx=0 tx=0).stop 2023-07-14T16:25:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/1342319709 shutdown_connections 2023-07-14T16:25:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6d6e0 0x55bca9e758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9cc86b0 0x55bca9e74560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/1342319709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9e6c840 0x55bca9e7ce10 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/1342319709 >> 172.21.15.62:0/1342319709 conn(0x55bca9e61be0 msgr2=0x55bca9e883e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/1342319709 shutdown_connections 2023-07-14T16:25:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/1342319709 wait complete. 2023-07-14T16:25:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.203+0000 7ff65b3ccb80 1 -- start start 2023-07-14T16:25:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff65b3ccb80 1 Processor -- start 2023-07-14T16:25:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9cc86b0 0x55bca9e79010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6c840 0x55bca9e79550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff65b3ccb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9e7c3b0 con 0x55bca9cc86b0 2023-07-14T16:25:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9e7c520 con 0x55bca9e6d6e0 2023-07-14T16:25:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff65b3ccb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9e7c6e0 con 0x55bca9e6c840 2023-07-14T16:25:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff65b3ccb80 10 client.0 _ll_register_callbacks cb 0x7ff630099320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:25:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.204+0000 7ff64579b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9cc86b0 0x55bca9e79010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff64579b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9cc86b0 0x55bca9e79010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48744/0 (socket says 172.21.15.62:48744) 2023-07-14T16:25:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff64579b700 1 -- 172.21.15.62:0/2020507837 learned_addr learned my addr 172.21.15.62:0/2020507837 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:25:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff644f9a700 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6c840 0x55bca9e79550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff645f9c700 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff64579b700 1 -- 172.21.15.62:0/2020507837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6c840 msgr2=0x55bca9e79550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff64579b700 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6c840 0x55bca9e79550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff64579b700 1 -- 172.21.15.62:0/2020507837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6d6e0 msgr2=0x55bca9e7be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:25:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff64579b700 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff64579b700 1 -- 172.21.15.62:0/2020507837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bca9bd4c00 con 0x55bca9cc86b0 2023-07-14T16:25:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff644f9a700 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6c840 0x55bca9e79550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:25:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff64579b700 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9cc86b0 0x55bca9e79010 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff63c000c00 tx=0x7ff63c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b49bd1b5f167c90 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:25:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.205+0000 7ff628ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:25:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff628ff9700 1 -- 172.21.15.62:0/2020507837 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c026c70 con 0x55bca9cc86b0 2023-07-14T16:25:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2020507837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff638007c10 con 0x55bca9cc86b0 2023-07-14T16:25:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff628ff9700 1 -- 172.21.15.62:0/2020507837 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff63c026dd0 con 0x55bca9cc86b0 2023-07-14T16:25:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff628ff9700 1 -- 172.21.15.62:0/2020507837 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff63c004040 con 0x55bca9cc86b0 2023-07-14T16:25:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff65b3ccb80 10 client.18447 fetch_fsmap learned FSMap version 25 2023-07-14T16:25:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2020507837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bca9bea250 con 0x55bca9cc86b0 2023-07-14T16:25:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff628ff9700 1 -- 172.21.15.62:0/2020507837 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff638007c10 con 0x55bca9cc86b0 2023-07-14T16:25:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff628ff9700 1 -- 172.21.15.62:0/2020507837 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff63c038070 con 0x55bca9cc86b0 2023-07-14T16:25:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff65b3ccb80 10 client.18447 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:25:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff65b3ccb80 10 client.18447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:25:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff65b3ccb80 20 client.18447 populate_metadata read hostname 'smithi062' 2023-07-14T16:25:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.206+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bca9ed61b0 0x55bca9ed85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:25:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.207+0000 7ff65b3ccb80 4 client.18447 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:25:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.207+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2020507837 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bca9d78ff0 con 0x55bca9ed61b0 2023-07-14T16:25:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.207+0000 7ff644f9a700 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bca9ed61b0 0x55bca9ed85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:25:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.208+0000 7ff644f9a700 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bca9ed61b0 0x55bca9ed85c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70f034e990ae927 server_cookie=dc057626650f9bb in_seq=0 out_seq=0 2023-07-14T16:25:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.208+0000 7ff628ff9700 10 client.18447 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff628ff9700 1 -- 172.21.15.62:0/2020507837 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bca9d78ff0 con 0x55bca9ed61b0 2023-07-14T16:25:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff628ff9700 10 client.18447 handle_command_reply: tid=0 2023-07-14T16:25:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 client.18447 shutdown 2023-07-14T16:25:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2020507837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9cc86b0 msgr2=0x55bca9e79010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9cc86b0 0x55bca9e79010 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff63c000c00 tx=0x7ff63c009f60 comp rx=0 tx=0).stop 2023-07-14T16:25:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2020507837 shutdown_connections 2023-07-14T16:25:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bca9ed61b0 0x55bca9ed85c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bca9e6d6e0 0x55bca9e7be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bca9cc86b0 0x55bca9e79010 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 --2- 172.21.15.62:0/2020507837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bca9e6c840 0x55bca9e79550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2020507837 >> 172.21.15.62:0/2020507837 conn(0x55bca9e61be0 msgr2=0x55bca9e74560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.217+0000 7ff628ff9700 0 client.18447 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:25:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.218+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2020507837 shutdown_connections 2023-07-14T16:25:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.218+0000 7ff65b3ccb80 1 -- 172.21.15.62:0/2020507837 wait complete. 2023-07-14T16:25:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.218+0000 7ff65b3ccb80 20 client.18447 trim_cache size 0 max 16384 2023-07-14T16:25:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:25:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:25:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:25:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:25:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.218+0000 7ff6479ff700 1 -- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff62c0a11d0 msgr2=0x7ff62c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.218+0000 7ff6479ff700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff62c0a11d0 0x7ff62c0a3680 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e000 tx=0x7ff638024000 comp rx=0 tx=0).stop 2023-07-14T16:25:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.218+0000 7ff6479ff700 1 -- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640104fd0 msgr2=0x7ff640198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:25:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.218+0000 7ff6479ff700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640104fd0 0x7ff640198110 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff63000d950 tx=0x7ff63000dc60 comp rx=0 tx=0).stop 2023-07-14T16:25:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.219+0000 7ff6479ff700 1 -- 172.21.15.62:0/963361465 shutdown_connections 2023-07-14T16:25:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.219+0000 7ff6479ff700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff62c0a11d0 0x7ff62c0a3680 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.219+0000 7ff6479ff700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6401078f0 0x7ff640198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.219+0000 7ff6479ff700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff640068640 0x7ff640197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.219+0000 7ff6479ff700 1 --2- 172.21.15.62:0/963361465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff640104fd0 0x7ff640198110 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:25:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.219+0000 7ff6479ff700 1 -- 172.21.15.62:0/963361465 >> 172.21.15.62:0/963361465 conn(0x7ff6400fa000 msgr2=0x7ff640105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:25:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.219+0000 7ff6479ff700 1 -- 172.21.15.62:0/963361465 shutdown_connections 2023-07-14T16:25:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:25:58.219+0000 7ff6479ff700 1 -- 172.21.15.62:0/963361465 wait complete. 2023-07-14T16:25:59.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:25:58 smithi062 ceph-mon[95543]: pgmap v667: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 29 op/s 2023-07-14T16:25:59.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:25:58 smithi081 ceph-mon[127795]: pgmap v667: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 29 op/s 2023-07-14T16:25:59.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:25:58 smithi164 ceph-mon[97427]: pgmap v667: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 29 op/s 2023-07-14T16:26:01.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:00 smithi062 ceph-mon[95543]: pgmap v668: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:26:01.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:00 smithi081 ceph-mon[127795]: pgmap v668: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:26:01.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:00 smithi164 ceph-mon[97427]: pgmap v668: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:26:03.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:02 smithi062 ceph-mon[95543]: pgmap v669: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:26:03.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:02 smithi081 ceph-mon[127795]: pgmap v669: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:26:03.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:02 smithi164 ceph-mon[97427]: pgmap v669: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:26:04.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:26:04.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:26:04.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:26:04.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:26:04.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:26:04.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:26:05.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:04 smithi062 ceph-mon[95543]: pgmap v670: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:26:05.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:04 smithi081 ceph-mon[127795]: pgmap v670: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:26:05.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:04 smithi164 ceph-mon[97427]: pgmap v670: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:26:07.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:06 smithi062 ceph-mon[95543]: pgmap v671: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 34 op/s 2023-07-14T16:26:07.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:06 smithi081 ceph-mon[127795]: pgmap v671: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 34 op/s 2023-07-14T16:26:07.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:06 smithi164 ceph-mon[97427]: pgmap v671: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 34 op/s 2023-07-14T16:26:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:08 smithi062 ceph-mon[95543]: pgmap v672: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 29 op/s 2023-07-14T16:26:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:08 smithi081 ceph-mon[127795]: pgmap v672: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 29 op/s 2023-07-14T16:26:09.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:08 smithi164 ceph-mon[97427]: pgmap v672: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 29 op/s 2023-07-14T16:26:10.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:09 smithi081 ceph-mon[127795]: pgmap v673: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:26:10.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:09 smithi164 ceph-mon[97427]: pgmap v673: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:26:10.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:09 smithi062 ceph-mon[95543]: pgmap v673: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:26:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 -- 172.21.15.62:0/75551326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700100430 msgr2=0x7f8700102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 --2- 172.21.15.62:0/75551326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700100430 0x7f8700102810 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009990 tx=0x7f86f0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 -- 172.21.15.62:0/75551326 shutdown_connections 2023-07-14T16:26:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 --2- 172.21.15.62:0/75551326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8700105670 0x7f8700107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 --2- 172.21.15.62:0/75551326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700100430 0x7f8700102810 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 --2- 172.21.15.62:0/75551326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8700102d50 0x7f8700105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 -- 172.21.15.62:0/75551326 >> 172.21.15.62:0/75551326 conn(0x7f87000f9e30 msgr2=0x7f87000fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 -- 172.21.15.62:0/75551326 shutdown_connections 2023-07-14T16:26:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.385+0000 7f870539d700 1 -- 172.21.15.62:0/75551326 wait complete. 2023-07-14T16:26:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.386+0000 7f870539d700 1 Processor -- start 2023-07-14T16:26:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.386+0000 7f870539d700 1 -- start start 2023-07-14T16:26:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f870539d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8700100430 0x7f87000ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f870539d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8700102d50 0x7f87000fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f870539d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700105670 0x7f87000fe920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f870539d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000ff0c0 con 0x7f8700105670 2023-07-14T16:26:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f870539d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000ff230 con 0x7f8700102d50 2023-07-14T16:26:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f870539d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870019f930 con 0x7f8700100430 2023-07-14T16:26:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f8704b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700105670 0x7f87000fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8700102d50 0x7f87000fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f8704b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700105670 0x7f87000fe920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39758/0 (socket says 172.21.15.62:39758) 2023-07-14T16:26:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8700102d50 0x7f87000fe3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46640/0 (socket says 172.21.15.62:46640) 2023-07-14T16:26:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f8704b9c700 1 -- 172.21.15.62:0/1228382141 learned_addr learned my addr 172.21.15.62:0/1228382141 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.387+0000 7f86fffff700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8700100430 0x7f87000ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f8704b9c700 1 -- 172.21.15.62:0/1228382141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8700100430 msgr2=0x7f87000ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f8704b9c700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8700100430 0x7f87000ffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f8704b9c700 1 -- 172.21.15.62:0/1228382141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8700102d50 msgr2=0x7f87000fe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f8704b9c700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8700102d50 0x7f87000fe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f8704b9c700 1 -- 172.21.15.62:0/1228382141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00095f0 con 0x7f8700105670 2023-07-14T16:26:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f8704b9c700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700105670 0x7f87000fe920 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f86f8007f10 tx=0x7f86f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f86fd7fa700 1 -- 172.21.15.62:0/1228382141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8017070 con 0x7f8700105670 2023-07-14T16:26:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f86fd7fa700 1 -- 172.21.15.62:0/1228382141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86f800f040 con 0x7f8700105670 2023-07-14T16:26:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f870539d700 1 -- 172.21.15.62:0/1228382141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870019fc10 con 0x7f8700105670 2023-07-14T16:26:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f86fd7fa700 1 -- 172.21.15.62:0/1228382141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8005570 con 0x7f8700105670 2023-07-14T16:26:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.388+0000 7f870539d700 1 -- 172.21.15.62:0/1228382141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001a0120 con 0x7f8700105670 2023-07-14T16:26:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.392+0000 7f86fd7fa700 1 -- 172.21.15.62:0/1228382141 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f86f8004ad0 con 0x7f8700105670 2023-07-14T16:26:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f8719eb4b80 1 Processor -- start 2023-07-14T16:26:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f8719eb4b80 1 -- start start 2023-07-14T16:26:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052deca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f86fd7fa700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86e80a11d0 0x7f86e80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 0x56052decb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052decbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052ddcfc50 con 0x56052decb7e0 2023-07-14T16:26:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052de229e0 con 0x56052dd266b0 2023-07-14T16:26:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052de1be70 con 0x56052decaa90 2023-07-14T16:26:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.394+0000 7f86fd7fa700 1 -- 172.21.15.62:0/1228382141 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86f80cbfc0 con 0x7f8700105670 2023-07-14T16:26:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052deca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052deca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46656/0 (socket says 172.21.15.62:46656) 2023-07-14T16:26:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 -- 172.21.15.62:0/1566404399 learned_addr learned my addr 172.21.15.62:0/1566404399 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 0x56052decb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f8704b9c700 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052decbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86fffff700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86e80a11d0 0x7f86e80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 -- 172.21.15.62:0/1566404399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 msgr2=0x56052decb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 0x56052decb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 -- 172.21.15.62:0/1566404399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 msgr2=0x56052decbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052decbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f86ff7fe700 1 -- 172.21.15.62:0/1566404399 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56052dc49d80 con 0x56052dd266b0 2023-07-14T16:26:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.395+0000 7f8704b9c700 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052decbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.396+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052deca550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86f4012810 tx=0x7f86f4007840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=301d5ee5f00dd523 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.396+0000 7f86fffff700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86e80a11d0 0x7f86e80a3680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f86f000b5c0 tx=0x7f86f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.396+0000 7f86e67fc700 1 -- 172.21.15.62:0/1566404399 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4012e30 con 0x56052dd266b0 2023-07-14T16:26:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.396+0000 7f86e67fc700 1 -- 172.21.15.62:0/1566404399 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86f40044c0 con 0x56052dd266b0 2023-07-14T16:26:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.396+0000 7f86e67fc700 1 -- 172.21.15.62:0/1566404399 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4005220 con 0x56052dd266b0 2023-07-14T16:26:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 -- 172.21.15.62:0/1566404399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 msgr2=0x56052deca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052deca550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86f4012810 tx=0x7f86f4007840 comp rx=0 tx=0).stop 2023-07-14T16:26:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 -- 172.21.15.62:0/1566404399 shutdown_connections 2023-07-14T16:26:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052decbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052deca550 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/1566404399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 0x56052decb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 -- 172.21.15.62:0/1566404399 >> 172.21.15.62:0/1566404399 conn(0x56052debfd40 msgr2=0x56052dec4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 -- 172.21.15.62:0/1566404399 shutdown_connections 2023-07-14T16:26:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.397+0000 7f8719eb4b80 1 -- 172.21.15.62:0/1566404399 wait complete. 2023-07-14T16:26:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.398+0000 7f8719eb4b80 1 -- start start 2023-07-14T16:26:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.398+0000 7f8719eb4b80 1 Processor -- start 2023-07-14T16:26:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052ded39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decaa90 0x56052ded3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decb7e0 0x56052dedaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052ddbba60 con 0x56052decaa90 2023-07-14T16:26:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052ddbb760 con 0x56052dd266b0 2023-07-14T16:26:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052dc9f8c0 con 0x56052decb7e0 2023-07-14T16:26:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f8719eb4b80 10 client.0 _ll_register_callbacks cb 0x56052ddb9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decaa90 0x56052ded3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decaa90 0x56052ded3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39768/0 (socket says 172.21.15.62:39768) 2023-07-14T16:26:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f86ff7fe700 1 -- 172.21.15.62:0/2850347365 learned_addr learned my addr 172.21.15.62:0/2850347365 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.399+0000 7f86fffff700 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052ded39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f8704b9c700 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decb7e0 0x56052dedaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86ff7fe700 1 -- 172.21.15.62:0/2850347365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decb7e0 msgr2=0x56052dedaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decb7e0 0x56052dedaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86ff7fe700 1 -- 172.21.15.62:0/2850347365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 msgr2=0x56052ded39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052ded39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86ff7fe700 1 -- 172.21.15.62:0/2850347365 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001a0120 con 0x56052decaa90 2023-07-14T16:26:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decaa90 0x56052ded3f20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f86f40049b0 tx=0x7f86f4004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c71c3b85045881 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86df7fe700 1 -- 172.21.15.62:0/2850347365 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4004e70 con 0x56052decaa90 2023-07-14T16:26:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.400+0000 7f86df7fe700 1 -- 172.21.15.62:0/2850347365 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86f4012e30 con 0x56052decaa90 2023-07-14T16:26:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.401+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2850347365 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56052dd75490 con 0x56052decaa90 2023-07-14T16:26:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.401+0000 7f86df7fe700 1 -- 172.21.15.62:0/2850347365 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86f4005230 con 0x56052decaa90 2023-07-14T16:26:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.401+0000 7f8719eb4b80 10 client.18459 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.401+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2850347365 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f86f401a040 con 0x56052decaa90 2023-07-14T16:26:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.401+0000 7f86df7fe700 1 -- 172.21.15.62:0/2850347365 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56052dd75490 con 0x56052decaa90 2023-07-14T16:26:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.401+0000 7f86df7fe700 1 -- 172.21.15.62:0/2850347365 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86f4020b60 con 0x56052decaa90 2023-07-14T16:26:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.401+0000 7f8719eb4b80 10 client.18459 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.401+0000 7f8719eb4b80 10 client.18459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.402+0000 7f8719eb4b80 20 client.18459 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.402+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56052dedf6c0 0x56052dee1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.402+0000 7f8719eb4b80 4 client.18459 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.402+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2850347365 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56052de14e50 con 0x56052dedf6c0 2023-07-14T16:26:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.402+0000 7f86fffff700 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56052dedf6c0 0x56052dee1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.403+0000 7f86df7fe700 10 client.18459 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.403+0000 7f86fffff700 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56052dedf6c0 0x56052dee1ad0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5838118232ef4c server_cookie=4e994037242da34 in_seq=0 out_seq=0 2023-07-14T16:26:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.403+0000 7f86df7fe700 1 -- 172.21.15.62:0/2850347365 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56052de14e50 con 0x56052dedf6c0 2023-07-14T16:26:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.403+0000 7f86df7fe700 10 client.18459 handle_command_reply: tid=0 2023-07-14T16:26:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.403+0000 7f8719eb4b80 1 client.18459 shutdown 2023-07-14T16:26:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2850347365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decaa90 msgr2=0x56052ded3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decaa90 0x56052ded3f20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f86f40049b0 tx=0x7f86f4004be0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2850347365 shutdown_connections 2023-07-14T16:26:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decb7e0 0x56052dedaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56052dedf6c0 0x56052dee1ad0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052ded39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2850347365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decaa90 0x56052ded3f20 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2850347365 >> 172.21.15.62:0/2850347365 conn(0x56052debfd40 msgr2=0x56052dec4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f86df7fe700 0 client.18459 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2850347365 shutdown_connections 2023-07-14T16:26:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2850347365 wait complete. 2023-07-14T16:26:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.404+0000 7f8719eb4b80 20 client.18459 trim_cache size 0 max 16384 2023-07-14T16:26:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.426+0000 7f8719eb4b80 1 Processor -- start 2023-07-14T16:26:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.426+0000 7f8719eb4b80 1 -- start start 2023-07-14T16:26:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.426+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052decbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.426+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 0x56052dedaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.426+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052dece690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.426+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052decb0d0 con 0x56052decb7e0 2023-07-14T16:26:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.426+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052ded4660 con 0x56052dd266b0 2023-07-14T16:26:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.426+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052decae60 con 0x56052decaa90 2023-07-14T16:26:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052decbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f8704b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052dece690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052decbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46678/0 (socket says 172.21.15.62:46678) 2023-07-14T16:26:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 -- 172.21.15.62:0/19120810 learned_addr learned my addr 172.21.15.62:0/19120810 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/19120810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 0x56052dedaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 -- 172.21.15.62:0/19120810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 msgr2=0x56052dedaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 --2- 172.21.15.62:0/19120810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 0x56052dedaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 -- 172.21.15.62:0/19120810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 msgr2=0x56052dece690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 --2- 172.21.15.62:0/19120810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052dece690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 -- 172.21.15.62:0/19120810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56052dc32c00 con 0x56052dd266b0 2023-07-14T16:26:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.427+0000 7f86fffff700 1 --2- 172.21.15.62:0/19120810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052decbbb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009990 tx=0x7f86f0020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dab8aaadb8e03b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.428+0000 7f86e6ffd700 1 -- 172.21.15.62:0/19120810 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f002b040 con 0x56052dd266b0 2023-07-14T16:26:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.428+0000 7f86e6ffd700 1 -- 172.21.15.62:0/19120810 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86f002c040 con 0x56052dd266b0 2023-07-14T16:26:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.428+0000 7f86e6ffd700 1 -- 172.21.15.62:0/19120810 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0004760 con 0x56052dd266b0 2023-07-14T16:26:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.428+0000 7f8719eb4b80 1 -- 172.21.15.62:0/19120810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 msgr2=0x56052decbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.428+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/19120810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052decbbb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009990 tx=0x7f86f0020000 comp rx=0 tx=0).stop 2023-07-14T16:26:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.429+0000 7f8719eb4b80 1 -- 172.21.15.62:0/19120810 shutdown_connections 2023-07-14T16:26:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.429+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/19120810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052dece690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.429+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/19120810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052dd266b0 0x56052decbbb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.429+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/19120810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052decaa90 0x56052dedaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.429+0000 7f8719eb4b80 1 -- 172.21.15.62:0/19120810 >> 172.21.15.62:0/19120810 conn(0x56052debfd40 msgr2=0x56052dee63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.429+0000 7f8719eb4b80 1 -- 172.21.15.62:0/19120810 shutdown_connections 2023-07-14T16:26:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.429+0000 7f8719eb4b80 1 -- 172.21.15.62:0/19120810 wait complete. 2023-07-14T16:26:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.429+0000 7f8719eb4b80 1 -- start start 2023-07-14T16:26:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.430+0000 7f8719eb4b80 1 Processor -- start 2023-07-14T16:26:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.430+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052dd266b0 0x56052ded6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.430+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052decaa90 0x56052ded6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.430+0000 7f8719eb4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052ded9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.430+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052deda410 con 0x56052decb7e0 2023-07-14T16:26:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.430+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052deda580 con 0x56052decaa90 2023-07-14T16:26:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.430+0000 7f8719eb4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052deda740 con 0x56052dd266b0 2023-07-14T16:26:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.430+0000 7f8719eb4b80 10 client.0 _ll_register_callbacks cb 0x7f86f8004e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f86fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052dd266b0 0x56052ded6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f86ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052decaa90 0x56052ded6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f8704b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052ded9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f86fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052dd266b0 0x56052ded6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:45688/0 (socket says 172.21.15.62:45688) 2023-07-14T16:26:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f86fffff700 1 -- 172.21.15.62:0/2769114390 learned_addr learned my addr 172.21.15.62:0/2769114390 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f8704b9c700 1 -- 172.21.15.62:0/2769114390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052dd266b0 msgr2=0x56052ded6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f8704b9c700 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052dd266b0 0x56052ded6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f8704b9c700 1 -- 172.21.15.62:0/2769114390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052decaa90 msgr2=0x56052ded6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f8704b9c700 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052decaa90 0x56052ded6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f8704b9c700 1 -- 172.21.15.62:0/2769114390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8700108670 con 0x56052decb7e0 2023-07-14T16:26:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f86ff7fe700 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052decaa90 0x56052ded6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f86fffff700 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052dd266b0 0x56052ded6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f8704b9c700 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052ded9ed0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f86f8007ee0 tx=0x7f86f8004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3805ffa946372c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.431+0000 7f86df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f86df7fe700 1 -- 172.21.15.62:0/2769114390 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8099380 con 0x56052decb7e0 2023-07-14T16:26:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f86df7fe700 1 -- 172.21.15.62:0/2769114390 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86f809d3b0 con 0x56052decb7e0 2023-07-14T16:26:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2769114390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86f40044c0 con 0x56052decb7e0 2023-07-14T16:26:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f86df7fe700 1 -- 172.21.15.62:0/2769114390 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86f80a17d0 con 0x56052decb7e0 2023-07-14T16:26:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f8719eb4b80 10 client.18465 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2769114390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56052dc701f0 con 0x56052decb7e0 2023-07-14T16:26:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f86df7fe700 1 -- 172.21.15.62:0/2769114390 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86f8081020 con 0x56052decb7e0 2023-07-14T16:26:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f86df7fe700 1 -- 172.21.15.62:0/2769114390 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f86f8082bd0 con 0x56052decb7e0 2023-07-14T16:26:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f8719eb4b80 10 client.18465 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f8719eb4b80 10 client.18465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f8719eb4b80 20 client.18465 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.432+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56052df341f0 0x56052df36600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.433+0000 7f8719eb4b80 4 client.18465 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:26:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.433+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2769114390 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56052ddd6ff0 con 0x56052df341f0 2023-07-14T16:26:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.433+0000 7f86fffff700 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56052df341f0 0x56052df36600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.434+0000 7f86fffff700 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56052df341f0 0x56052df36600 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10359e059f5cd05e server_cookie=2c0362f974a0f911 in_seq=0 out_seq=0 2023-07-14T16:26:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.434+0000 7f86df7fe700 10 client.18465 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f86df7fe700 1 -- 172.21.15.62:0/2769114390 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56052ddd6ff0 con 0x56052df341f0 2023-07-14T16:26:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f86df7fe700 10 client.18465 handle_command_reply: tid=0 2023-07-14T16:26:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 client.18465 shutdown 2023-07-14T16:26:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2769114390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 msgr2=0x56052ded9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052ded9ed0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f86f8007ee0 tx=0x7f86f8004f90 comp rx=0 tx=0).stop 2023-07-14T16:26:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2769114390 shutdown_connections 2023-07-14T16:26:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56052decb7e0 0x56052ded9ed0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56052dd266b0 0x56052ded6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56052df341f0 0x56052df36600 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 --2- 172.21.15.62:0/2769114390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56052decaa90 0x56052ded6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f86df7fe700 0 client.18465 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2769114390 >> 172.21.15.62:0/2769114390 conn(0x56052debfd40 msgr2=0x56052dece690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2769114390 shutdown_connections 2023-07-14T16:26:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 1 -- 172.21.15.62:0/2769114390 wait complete. 2023-07-14T16:26:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.443+0000 7f8719eb4b80 20 client.18465 trim_cache size 0 max 16384 2023-07-14T16:26:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:26:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:26:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:26:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:26:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 -- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86e80a11d0 msgr2=0x7f86e80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86e80a11d0 0x7f86e80a3680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f86f000b5c0 tx=0x7f86f0005fb0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 -- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700105670 msgr2=0x7f87000fe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700105670 0x7f87000fe920 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f86f8007f10 tx=0x7f86f800d3b0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 -- 172.21.15.62:0/1228382141 shutdown_connections 2023-07-14T16:26:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8700105670 0x7f87000fe920 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8700100430 0x7f87000ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86e80a11d0 0x7f86e80a3680 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 --2- 172.21.15.62:0/1228382141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8700102d50 0x7f87000fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 -- 172.21.15.62:0/1228382141 >> 172.21.15.62:0/1228382141 conn(0x7f87000f9e30 msgr2=0x7f8700101090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 -- 172.21.15.62:0/1228382141 shutdown_connections 2023-07-14T16:26:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:11.444+0000 7f870539d700 1 -- 172.21.15.62:0/1228382141 wait complete. 2023-07-14T16:26:13.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:12 smithi164 ceph-mon[97427]: pgmap v674: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:13.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:12 smithi062 ceph-mon[95543]: pgmap v674: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:13.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:12 smithi081 ceph-mon[127795]: pgmap v674: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:15.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:14 smithi164 ceph-mon[97427]: pgmap v675: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:26:15.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:14 smithi062 ceph-mon[95543]: pgmap v675: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:26:15.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:14 smithi081 ceph-mon[127795]: pgmap v675: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:26:17.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:16 smithi164 ceph-mon[97427]: pgmap v676: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:26:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:16 smithi062 ceph-mon[95543]: pgmap v676: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:26:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:16 smithi081 ceph-mon[127795]: pgmap v676: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:26:19.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:18 smithi164 ceph-mon[97427]: pgmap v677: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:26:19.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:18 smithi062 ceph-mon[95543]: pgmap v677: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:26:19.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:18 smithi081 ceph-mon[127795]: pgmap v677: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 17 MiB/s wr, 14 op/s 2023-07-14T16:26:21.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:20 smithi164 ceph-mon[97427]: pgmap v678: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:26:21.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:20 smithi062 ceph-mon[95543]: pgmap v678: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:26:21.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:20 smithi081 ceph-mon[127795]: pgmap v678: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:26:23.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:22 smithi164 ceph-mon[97427]: pgmap v679: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:26:23.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:22 smithi062 ceph-mon[95543]: pgmap v679: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:26:23.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:22 smithi081 ceph-mon[127795]: pgmap v679: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:26:24.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:26:24.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.596+0000 7efeaf3ff700 1 -- 172.21.15.62:0/2494932322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 msgr2=0x7efea81052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.596+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/2494932322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 0x7efea81052e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7efe98009990 tx=0x7efe98009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.596+0000 7efeaf3ff700 1 -- 172.21.15.62:0/2494932322 shutdown_connections 2023-07-14T16:26:24.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.596+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/2494932322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea8105820 0x7efea8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.596+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/2494932322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea81005e0 0x7efea81029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.596+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/2494932322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 0x7efea81052e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.596+0000 7efeaf3ff700 1 -- 172.21.15.62:0/2494932322 >> 172.21.15.62:0/2494932322 conn(0x7efea80fa000 msgr2=0x7efea80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.596+0000 7efeaf3ff700 1 -- 172.21.15.62:0/2494932322 shutdown_connections 2023-07-14T16:26:24.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.597+0000 7efeaf3ff700 1 -- 172.21.15.62:0/2494932322 wait complete. 2023-07-14T16:26:24.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeaf3ff700 1 Processor -- start 2023-07-14T16:26:24.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeaf3ff700 1 -- start start 2023-07-14T16:26:24.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeaf3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea81005e0 0x7efea81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeaf3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 0x7efea8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeaf3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea8105820 0x7efea819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeaf3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea819f530 con 0x7efea8102f00 2023-07-14T16:26:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeaf3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea819f670 con 0x7efea81005e0 2023-07-14T16:26:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeaf3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea819f800 con 0x7efea8105820 2023-07-14T16:26:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.598+0000 7efeac99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 0x7efea8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efeac99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 0x7efea8197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59144/0 (socket says 172.21.15.62:59144) 2023-07-14T16:26:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efeac99a700 1 -- 172.21.15.62:0/3619653603 learned_addr learned my addr 172.21.15.62:0/3619653603 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efead19b700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea81005e0 0x7efea81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efead99c700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea8105820 0x7efea819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efeac99a700 1 -- 172.21.15.62:0/3619653603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea8105820 msgr2=0x7efea819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efeac99a700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea8105820 0x7efea819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efeac99a700 1 -- 172.21.15.62:0/3619653603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea81005e0 msgr2=0x7efea81979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efeac99a700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea81005e0 0x7efea81979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efeac99a700 1 -- 172.21.15.62:0/3619653603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe980095f0 con 0x7efea8102f00 2023-07-14T16:26:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.599+0000 7efeac99a700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 0x7efea8197ef0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7efe98005910 tx=0x7efe98005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.600+0000 7efe9e7fc700 1 -- 172.21.15.62:0/3619653603 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9801d070 con 0x7efea8102f00 2023-07-14T16:26:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.600+0000 7efe9e7fc700 1 -- 172.21.15.62:0/3619653603 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe98022470 con 0x7efea8102f00 2023-07-14T16:26:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.600+0000 7efe9e7fc700 1 -- 172.21.15.62:0/3619653603 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98022ea0 con 0x7efea8102f00 2023-07-14T16:26:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.600+0000 7efeaf3ff700 1 -- 172.21.15.62:0/3619653603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea819fa80 con 0x7efea8102f00 2023-07-14T16:26:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.600+0000 7efeaf3ff700 1 -- 172.21.15.62:0/3619653603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea819fe50 con 0x7efea8102f00 2023-07-14T16:26:24.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.604+0000 7efe9e7fc700 1 -- 172.21.15.62:0/3619653603 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efe9800ba60 con 0x7efea8102f00 2023-07-14T16:26:24.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efec2c39b80 1 Processor -- start 2023-07-14T16:26:24.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efec2c39b80 1 -- start start 2023-07-14T16:26:24.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efe9e7fc700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe940a1150 0x7efe940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efe9e7fc700 1 -- 172.21.15.62:0/3619653603 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe980cdc70 con 0x7efea8102f00 2023-07-14T16:26:24.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c76304d0 0x5604c76308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c76316e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 0x5604c76320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efec2c39b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c7535cb0 con 0x5604c76304d0 2023-07-14T16:26:24.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efec2c39b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c7588a40 con 0x5604c7631c70 2023-07-14T16:26:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.606+0000 7efec2c39b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c7581ed0 con 0x5604c7630e20 2023-07-14T16:26:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead99c700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe940a1150 0x7efe940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c76304d0 0x5604c76308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead99c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 0x5604c76320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efeac99a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c76316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c76304d0 0x5604c76308e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59150/0 (socket says 172.21.15.62:59150) 2023-07-14T16:26:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead19b700 1 -- 172.21.15.62:0/3599160691 learned_addr learned my addr 172.21.15.62:0/3599160691 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead19b700 1 -- 172.21.15.62:0/3599160691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 msgr2=0x5604c76316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead19b700 1 --2- 172.21.15.62:0/3599160691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c76316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead19b700 1 -- 172.21.15.62:0/3599160691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 msgr2=0x5604c76320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead19b700 1 --2- 172.21.15.62:0/3599160691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 0x5604c76320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.607+0000 7efead19b700 1 -- 172.21.15.62:0/3599160691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604c73afde0 con 0x5604c76304d0 2023-07-14T16:26:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.608+0000 7efead99c700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe940a1150 0x7efe940a3600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efea0009ec0 tx=0x7efea00092f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.608+0000 7efead19b700 1 --2- 172.21.15.62:0/3599160691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c76304d0 0x5604c76308e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7efea400bd70 tx=0x7efea40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc7c43f412c9d99 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.608+0000 7efe937fe700 1 -- 172.21.15.62:0/3599160691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4010d10 con 0x5604c76304d0 2023-07-14T16:26:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.608+0000 7efe937fe700 1 -- 172.21.15.62:0/3599160691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efea4015070 con 0x5604c76304d0 2023-07-14T16:26:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.608+0000 7efe937fe700 1 -- 172.21.15.62:0/3599160691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4011640 con 0x5604c76304d0 2023-07-14T16:26:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 -- 172.21.15.62:0/3599160691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c76304d0 msgr2=0x5604c76308e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3599160691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c76304d0 0x5604c76308e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7efea400bd70 tx=0x7efea40106b0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 -- 172.21.15.62:0/3599160691 shutdown_connections 2023-07-14T16:26:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3599160691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 0x5604c76320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3599160691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c76304d0 0x5604c76308e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3599160691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c76316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 -- 172.21.15.62:0/3599160691 >> 172.21.15.62:0/3599160691 conn(0x5604c7625da0 msgr2=0x5604c762b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 -- 172.21.15.62:0/3599160691 shutdown_connections 2023-07-14T16:26:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.609+0000 7efec2c39b80 1 -- 172.21.15.62:0/3599160691 wait complete. 2023-07-14T16:26:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.610+0000 7efec2c39b80 1 -- start start 2023-07-14T16:26:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.610+0000 7efec2c39b80 1 Processor -- start 2023-07-14T16:26:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 0x5604c763a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7630e20 0x5604c763a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 0x5604c76416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efec2c39b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c7521ac0 con 0x5604c7630e20 2023-07-14T16:26:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efec2c39b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c75217c0 con 0x5604c7631c70 2023-07-14T16:26:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efec2c39b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c7405920 con 0x5604c76304d0 2023-07-14T16:26:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efec2c39b80 10 client.0 _ll_register_callbacks cb 0x5604c751f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efeac99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7630e20 0x5604c763a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efeac99a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7630e20 0x5604c763a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59152/0 (socket says 172.21.15.62:59152) 2023-07-14T16:26:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efeac99a700 1 -- 172.21.15.62:0/3194359508 learned_addr learned my addr 172.21.15.62:0/3194359508 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.611+0000 7efead19b700 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 0x5604c763a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efead99c700 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 0x5604c76416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efeac99a700 1 -- 172.21.15.62:0/3194359508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 msgr2=0x5604c763a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efeac99a700 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 0x5604c763a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efeac99a700 1 -- 172.21.15.62:0/3194359508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 msgr2=0x5604c76416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efeac99a700 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 0x5604c76416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efeac99a700 1 -- 172.21.15.62:0/3194359508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea400b9a0 con 0x5604c7630e20 2023-07-14T16:26:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efeac99a700 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7630e20 0x5604c763a660 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7efe980cbaf0 tx=0x7efe9809ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad0ecb2e985caf7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efe90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efe90ff9700 1 -- 172.21.15.62:0/3194359508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9809b120 con 0x5604c7630e20 2023-07-14T16:26:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efe90ff9700 1 -- 172.21.15.62:0/3194359508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe9809f150 con 0x5604c7630e20 2023-07-14T16:26:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.612+0000 7efec2c39b80 1 -- 172.21.15.62:0/3194359508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604c74db4f0 con 0x5604c7630e20 2023-07-14T16:26:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efe90ff9700 1 -- 172.21.15.62:0/3194359508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe980b9030 con 0x5604c7630e20 2023-07-14T16:26:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efec2c39b80 10 client.18477 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efec2c39b80 1 -- 172.21.15.62:0/3194359508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5604c7398c60 con 0x5604c7630e20 2023-07-14T16:26:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efe90ff9700 1 -- 172.21.15.62:0/3194359508 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe980c9080 con 0x5604c7630e20 2023-07-14T16:26:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efe90ff9700 1 -- 172.21.15.62:0/3194359508 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efe980a3050 con 0x5604c7630e20 2023-07-14T16:26:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efec2c39b80 10 client.18477 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efec2c39b80 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efec2c39b80 20 client.18477 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604c7645b80 0x5604c7647f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efec2c39b80 4 client.18477 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.613+0000 7efec2c39b80 1 -- 172.21.15.62:0/3194359508 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604c757aeb0 con 0x5604c7645b80 2023-07-14T16:26:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.614+0000 7efead19b700 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604c7645b80 0x5604c7647f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.615+0000 7efe90ff9700 10 client.18477 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.615+0000 7efead19b700 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604c7645b80 0x5604c7647f60 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d2b75420baae0c server_cookie=a7b266d44432ee7c in_seq=0 out_seq=0 2023-07-14T16:26:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.615+0000 7efe90ff9700 1 -- 172.21.15.62:0/3194359508 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5604c757aeb0 con 0x5604c7645b80 2023-07-14T16:26:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.615+0000 7efe90ff9700 10 client.18477 handle_command_reply: tid=0 2023-07-14T16:26:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.615+0000 7efec2c39b80 1 client.18477 shutdown 2023-07-14T16:26:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 -- 172.21.15.62:0/3194359508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7630e20 msgr2=0x5604c763a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7630e20 0x5604c763a660 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7efe980cbaf0 tx=0x7efe9809ad30 comp rx=0 tx=0).stop 2023-07-14T16:26:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 -- 172.21.15.62:0/3194359508 shutdown_connections 2023-07-14T16:26:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604c7645b80 0x5604c7647f60 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7631c70 0x5604c76416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 0x5604c763a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3194359508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7630e20 0x5604c763a660 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 -- 172.21.15.62:0/3194359508 >> 172.21.15.62:0/3194359508 conn(0x5604c7625da0 msgr2=0x5604c762a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efe90ff9700 0 client.18477 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 -- 172.21.15.62:0/3194359508 shutdown_connections 2023-07-14T16:26:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 1 -- 172.21.15.62:0/3194359508 wait complete. 2023-07-14T16:26:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.616+0000 7efec2c39b80 20 client.18477 trim_cache size 0 max 16384 2023-07-14T16:26:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 -- 172.21.15.62:0/2645894775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f123c105860 msgr2=0x7f123c107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 --2- 172.21.15.62:0/2645894775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f123c105860 0x7f123c107c40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1234009e00 tx=0x7f123400b330 comp rx=0 tx=0).stop 2023-07-14T16:26:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 -- 172.21.15.62:0/2645894775 shutdown_connections 2023-07-14T16:26:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 --2- 172.21.15.62:0/2645894775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f123c105860 0x7f123c107c40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 --2- 172.21.15.62:0/2645894775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f123c100620 0x7f123c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 --2- 172.21.15.62:0/2645894775 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c102f40 0x7f123c105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 -- 172.21.15.62:0/2645894775 >> 172.21.15.62:0/2645894775 conn(0x7f123c0fa000 msgr2=0x7f123c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 -- 172.21.15.62:0/2645894775 shutdown_connections 2023-07-14T16:26:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.635+0000 7f12423ff700 1 -- 172.21.15.62:0/2645894775 wait complete. 2023-07-14T16:26:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.637+0000 7f12423ff700 1 Processor -- start 2023-07-14T16:26:24.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.637+0000 7f12423ff700 1 -- start start 2023-07-14T16:26:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.637+0000 7f12423ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c100620 0x7f123c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.637+0000 7f12423ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f123c102f40 0x7f123c195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.637+0000 7f12423ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f123c105860 0x7f123c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.637+0000 7f12423ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c197410 con 0x7f123c102f40 2023-07-14T16:26:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.637+0000 7f12423ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c197580 con 0x7f123c105860 2023-07-14T16:26:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.637+0000 7f12423ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c197740 con 0x7f123c100620 2023-07-14T16:26:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f123c102f40 0x7f123c195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f123c102f40 0x7f123c195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59166/0 (socket says 172.21.15.62:59166) 2023-07-14T16:26:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c100620 0x7f123c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123b7fe700 1 -- 172.21.15.62:0/1436221371 learned_addr learned my addr 172.21.15.62:0/1436221371 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f124099c700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f123c105860 0x7f123c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123bfff700 1 -- 172.21.15.62:0/1436221371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f123c105860 msgr2=0x7f123c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123bfff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f123c105860 0x7f123c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123bfff700 1 -- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f123c102f40 msgr2=0x7f123c195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123bfff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f123c102f40 0x7f123c195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.638+0000 7f123bfff700 1 -- 172.21.15.62:0/1436221371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1234005e70 con 0x7f123c100620 2023-07-14T16:26:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.639+0000 7f123bfff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c100620 0x7f123c1959f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f122c005fd0 tx=0x7f122c00eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.639+0000 7f12397fa700 1 -- 172.21.15.62:0/1436221371 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c010900 con 0x7f123c100620 2023-07-14T16:26:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.639+0000 7f12397fa700 1 -- 172.21.15.62:0/1436221371 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f122c018420 con 0x7f123c100620 2023-07-14T16:26:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.639+0000 7f12423ff700 1 -- 172.21.15.62:0/1436221371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c197a20 con 0x7f123c100620 2023-07-14T16:26:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.639+0000 7f12397fa700 1 -- 172.21.15.62:0/1436221371 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c00f680 con 0x7f123c100620 2023-07-14T16:26:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.639+0000 7f12423ff700 1 -- 172.21.15.62:0/1436221371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c19e1e0 con 0x7f123c100620 2023-07-14T16:26:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.641+0000 7efec2c39b80 1 Processor -- start 2023-07-14T16:26:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.641+0000 7efec2c39b80 1 -- start start 2023-07-14T16:26:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.641+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c76304d0 0x5604c7638e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.641+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c7641660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.641+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c7641f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.641+0000 7efec2c39b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c762cd00 con 0x5604c7631c70 2023-07-14T16:26:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.641+0000 7efec2c39b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c763ada0 con 0x5604c76304d0 2023-07-14T16:26:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.641+0000 7efec2c39b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c7637dd0 con 0x5604c7630e20 2023-07-14T16:26:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c7641f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c76304d0 0x5604c7638e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c7641f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59168/0 (socket says 172.21.15.62:59168) 2023-07-14T16:26:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead99c700 1 -- 172.21.15.62:0/3732105779 learned_addr learned my addr 172.21.15.62:0/3732105779 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead19b700 1 -- 172.21.15.62:0/3732105779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 msgr2=0x5604c7641660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efeac99a700 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c7641660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead19b700 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c7641660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead19b700 1 -- 172.21.15.62:0/3732105779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 msgr2=0x5604c7641f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead19b700 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c7641f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead19b700 1 -- 172.21.15.62:0/3732105779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604c73d6250 con 0x5604c76304d0 2023-07-14T16:26:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efeac99a700 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c7641660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:26:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.642+0000 7efead19b700 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c76304d0 0x5604c7638e50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efea40059d0 tx=0x7efea4005d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb999b53150ccb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.643+0000 7efe93fff700 1 -- 172.21.15.62:0/3732105779 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4022040 con 0x5604c76304d0 2023-07-14T16:26:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.643+0000 7efe93fff700 1 -- 172.21.15.62:0/3732105779 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efea40275b0 con 0x5604c76304d0 2023-07-14T16:26:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.643+0000 7efe93fff700 1 -- 172.21.15.62:0/3732105779 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40052a0 con 0x5604c76304d0 2023-07-14T16:26:24.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 -- 172.21.15.62:0/3732105779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c76304d0 msgr2=0x5604c7638e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c76304d0 0x5604c7638e50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efea40059d0 tx=0x7efea4005d00 comp rx=0 tx=0).stop 2023-07-14T16:26:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 -- 172.21.15.62:0/3732105779 shutdown_connections 2023-07-14T16:26:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c7641f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c76304d0 0x5604c7638e50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 --2- 172.21.15.62:0/3732105779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c7630e20 0x5604c7641660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 -- 172.21.15.62:0/3732105779 >> 172.21.15.62:0/3732105779 conn(0x5604c7625da0 msgr2=0x5604c764c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 -- 172.21.15.62:0/3732105779 shutdown_connections 2023-07-14T16:26:24.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 -- 172.21.15.62:0/3732105779 wait complete. 2023-07-14T16:26:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.644+0000 7efec2c39b80 1 -- start start 2023-07-14T16:26:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.645+0000 7f12397fa700 1 -- 172.21.15.62:0/1436221371 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f122c018590 con 0x7f123c100620 2023-07-14T16:26:24.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.645+0000 7efec2c39b80 1 Processor -- start 2023-07-14T16:26:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.645+0000 7f1255c85b80 1 Processor -- start 2023-07-14T16:26:24.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.645+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 0x5604c76411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.645+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7630e20 0x5604c763de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.645+0000 7efec2c39b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c763e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.645+0000 7efec2c39b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c763ea70 con 0x5604c7631c70 2023-07-14T16:26:24.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.645+0000 7efec2c39b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c7641870 con 0x5604c7630e20 2023-07-14T16:26:24.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.645+0000 7efec2c39b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c7641a00 con 0x5604c76304d0 2023-07-14T16:26:24.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.645+0000 7efec2c39b80 10 client.0 _ll_register_callbacks cb 0x7efe98097550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.645+0000 7f1255c85b80 1 -- start start 2023-07-14T16:26:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c763e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c763e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59182/0 (socket says 172.21.15.62:59182) 2023-07-14T16:26:24.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead99c700 1 -- 172.21.15.62:0/586608020 learned_addr learned my addr 172.21.15.62:0/586608020 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfdb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead19b700 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 0x5604c76411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdc6e0 0x560a5dfdcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f1255c85b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5dee0c50 con 0x560a5dfdb840 2023-07-14T16:26:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f1255c85b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5df339e0 con 0x560a5dfdc6e0 2023-07-14T16:26:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f1255c85b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5df2ce70 con 0x560a5de376b0 2023-07-14T16:26:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f124099c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfdb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f124099c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfdb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52288/0 (socket says 172.21.15.62:52288) 2023-07-14T16:26:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f124099c700 1 -- 172.21.15.62:0/3538029545 learned_addr learned my addr 172.21.15.62:0/3538029545 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.646+0000 7f123b7fe700 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdc6e0 0x560a5dfdcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead99c700 1 -- 172.21.15.62:0/586608020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 msgr2=0x5604c76411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead99c700 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 0x5604c76411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead99c700 1 -- 172.21.15.62:0/586608020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7630e20 msgr2=0x5604c763de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead99c700 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7630e20 0x5604c763de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.646+0000 7efead99c700 1 -- 172.21.15.62:0/586608020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604c73afde0 con 0x5604c7631c70 2023-07-14T16:26:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.647+0000 7f123bfff700 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.647+0000 7f124099c700 1 -- 172.21.15.62:0/3538029545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdc6e0 msgr2=0x560a5dfdcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.647+0000 7f124099c700 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdc6e0 0x560a5dfdcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.647+0000 7f124099c700 1 -- 172.21.15.62:0/3538029545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 msgr2=0x560a5dfdc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.647+0000 7f124099c700 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.647+0000 7f124099c700 1 -- 172.21.15.62:0/3538029545 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a5dd5ad80 con 0x560a5de376b0 2023-07-14T16:26:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.647+0000 7efead99c700 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c763e390 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7efea001ae80 tx=0x7efea000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835a6d0fbdf4c742 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.647+0000 7efe90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.647+0000 7f124099c700 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfdb300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1234009b50 tx=0x7f1234009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adeaaba45bad8aa3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.647+0000 7efe90ff9700 1 -- 172.21.15.62:0/586608020 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000e070 con 0x5604c7631c70 2023-07-14T16:26:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.647+0000 7efec2c39b80 1 -- 172.21.15.62:0/586608020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea4015070 con 0x5604c7631c70 2023-07-14T16:26:24.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.647+0000 7efe90ff9700 1 -- 172.21.15.62:0/586608020 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efea0021040 con 0x5604c7631c70 2023-07-14T16:26:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.647+0000 7f12397fa700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12240a11d0 0x7f12240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.648+0000 7efe90ff9700 1 -- 172.21.15.62:0/586608020 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea001be80 con 0x5604c7631c70 2023-07-14T16:26:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.648+0000 7f12397fa700 1 -- 172.21.15.62:0/1436221371 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f122c014070 con 0x7f123c100620 2023-07-14T16:26:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.648+0000 7f12227fc700 1 -- 172.21.15.62:0/3538029545 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123400e040 con 0x560a5de376b0 2023-07-14T16:26:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.648+0000 7f12227fc700 1 -- 172.21.15.62:0/3538029545 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f123400bda0 con 0x560a5de376b0 2023-07-14T16:26:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.648+0000 7f12227fc700 1 -- 172.21.15.62:0/3538029545 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123401c7c0 con 0x560a5de376b0 2023-07-14T16:26:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.648+0000 7efec2c39b80 10 client.18489 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:24.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.648+0000 7efec2c39b80 1 -- 172.21.15.62:0/586608020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5604c73ae2b0 con 0x5604c7631c70 2023-07-14T16:26:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.648+0000 7f123bfff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12240a11d0 0x7f12240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 -- 172.21.15.62:0/3538029545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 msgr2=0x560a5dfdb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfdb300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1234009b50 tx=0x7f1234009b80 comp rx=0 tx=0).stop 2023-07-14T16:26:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.648+0000 7efe90ff9700 1 -- 172.21.15.62:0/586608020 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efea4015070 con 0x5604c7631c70 2023-07-14T16:26:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.649+0000 7efe90ff9700 1 -- 172.21.15.62:0/586608020 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efea00273e0 con 0x5604c7631c70 2023-07-14T16:26:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.649+0000 7efec2c39b80 10 client.18489 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:24.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.649+0000 7efec2c39b80 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.649+0000 7efec2c39b80 20 client.18489 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.649+0000 7efec2c39b80 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604c769b1e0 0x5604c769d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.649+0000 7efec2c39b80 4 client.18489 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:26:24.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.649+0000 7efec2c39b80 1 -- 172.21.15.62:0/586608020 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604c753d050 con 0x5604c769b1e0 2023-07-14T16:26:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 -- 172.21.15.62:0/3538029545 shutdown_connections 2023-07-14T16:26:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdc6e0 0x560a5dfdcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfdb300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3538029545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 -- 172.21.15.62:0/3538029545 >> 172.21.15.62:0/3538029545 conn(0x560a5dfd0be0 msgr2=0x560a5dfd5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 -- 172.21.15.62:0/3538029545 shutdown_connections 2023-07-14T16:26:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f1255c85b80 1 -- 172.21.15.62:0/3538029545 wait complete. 2023-07-14T16:26:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.649+0000 7f123bfff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12240a11d0 0x7f12240a3680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f122c096730 tx=0x7f122c018a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.650+0000 7efead19b700 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604c769b1e0 0x5604c769d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.650+0000 7f1255c85b80 1 -- start start 2023-07-14T16:26:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.650+0000 7f1255c85b80 1 Processor -- start 2023-07-14T16:26:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.650+0000 7efe90ff9700 10 client.18489 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:24.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.650+0000 7efead19b700 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604c769b1e0 0x5604c769d5f0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1f170cc9c911ba server_cookie=4670bdbb8daf198d in_seq=0 out_seq=0 2023-07-14T16:26:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5de376b0 0x560a5dfe8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 0x560a5dff01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f1255c85b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5decca60 con 0x560a5de376b0 2023-07-14T16:26:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f1255c85b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5decc760 con 0x560a5dfdb840 2023-07-14T16:26:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f1255c85b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5ddb08c0 con 0x560a5dfdc6e0 2023-07-14T16:26:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f1255c85b80 10 client.0 _ll_register_callbacks cb 0x560a5deca700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:24.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efe90ff9700 1 -- 172.21.15.62:0/586608020 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5604c753d050 con 0x5604c769b1e0 2023-07-14T16:26:24.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efe90ff9700 10 client.18489 handle_command_reply: tid=0 2023-07-14T16:26:24.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 client.18489 shutdown 2023-07-14T16:26:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f124099c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5de376b0 0x560a5dfe8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f124099c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5de376b0 0x560a5dfe8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59210/0 (socket says 172.21.15.62:59210) 2023-07-14T16:26:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f124099c700 1 -- 172.21.15.62:0/468168496 learned_addr learned my addr 172.21.15.62:0/468168496 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f123b7fe700 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 -- 172.21.15.62:0/586608020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 msgr2=0x5604c763e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c763e390 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7efea001ae80 tx=0x7efea000b040 comp rx=0 tx=0).stop 2023-07-14T16:26:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.651+0000 7f124099c700 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 0x560a5dff01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 -- 172.21.15.62:0/586608020 shutdown_connections 2023-07-14T16:26:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604c7631c70 0x5604c763e390 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604c76304d0 0x5604c76411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604c769b1e0 0x5604c769d5f0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 --2- 172.21.15.62:0/586608020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604c7630e20 0x5604c763de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efec2c39b80 1 -- 172.21.15.62:0/586608020 >> 172.21.15.62:0/586608020 conn(0x5604c7625da0 msgr2=0x5604c7638e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.651+0000 7efe90ff9700 0 client.18489 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:24.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.652+0000 7efec2c39b80 1 -- 172.21.15.62:0/586608020 shutdown_connections 2023-07-14T16:26:24.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.652+0000 7efec2c39b80 1 -- 172.21.15.62:0/586608020 wait complete. 2023-07-14T16:26:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.652+0000 7f123b7fe700 1 -- 172.21.15.62:0/468168496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 msgr2=0x560a5dff01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.652+0000 7f123b7fe700 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 0x560a5dff01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.652+0000 7f123b7fe700 1 -- 172.21.15.62:0/468168496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5de376b0 msgr2=0x560a5dfe8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.652+0000 7efec2c39b80 20 client.18489 trim_cache size 0 max 16384 2023-07-14T16:26:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.652+0000 7f123b7fe700 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5de376b0 0x560a5dfe8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.652+0000 7f123b7fe700 1 -- 172.21.15.62:0/468168496 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c108860 con 0x560a5dfdb840 2023-07-14T16:26:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.652+0000 7f124099c700 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5de376b0 0x560a5dfe8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:24.646 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:26:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:26:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:26:24.647 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:26:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.652+0000 7f123b7fe700 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe91b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1230011440 tx=0x7f1230011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d52db15e80fc0bf9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.652+0000 7f121b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:26:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.653+0000 7f121b7fe700 1 -- 172.21.15.62:0/468168496 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230016070 con 0x560a5dfdb840 2023-07-14T16:26:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.653+0000 7f121b7fe700 1 -- 172.21.15.62:0/468168496 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1230004020 con 0x560a5dfdb840 2023-07-14T16:26:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.653+0000 7f1255c85b80 1 -- 172.21.15.62:0/468168496 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a5de86490 con 0x560a5dfdb840 2023-07-14T16:26:24.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 -- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe940a1150 msgr2=0x7efe940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe940a1150 0x7efe940a3600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efea0009ec0 tx=0x7efea00092f0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 -- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 msgr2=0x7efea8197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 0x7efea8197ef0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7efe98005910 tx=0x7efe98005720 comp rx=0 tx=0).stop 2023-07-14T16:26:24.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 -- 172.21.15.62:0/3619653603 shutdown_connections 2023-07-14T16:26:24.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efea8105820 0x7efea819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efe940a1150 0x7efe940a3600 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efea81005e0 0x7efea81979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 --2- 172.21.15.62:0/3619653603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efea8102f00 0x7efea8197ef0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 -- 172.21.15.62:0/3619653603 >> 172.21.15.62:0/3619653603 conn(0x7efea80fa000 msgr2=0x7efea80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.653+0000 7f121b7fe700 1 -- 172.21.15.62:0/468168496 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1230021030 con 0x560a5dfdb840 2023-07-14T16:26:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.653+0000 7f1255c85b80 10 client.28097 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.653+0000 7f1255c85b80 1 -- 172.21.15.62:0/468168496 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560a5dd43c00 con 0x560a5dfdb840 2023-07-14T16:26:24.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 -- 172.21.15.62:0/3619653603 shutdown_connections 2023-07-14T16:26:24.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.653+0000 7efeaf3ff700 1 -- 172.21.15.62:0/3619653603 wait complete. 2023-07-14T16:26:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f121b7fe700 1 -- 172.21.15.62:0/468168496 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560a5de86490 con 0x560a5dfdb840 2023-07-14T16:26:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f121b7fe700 1 -- 172.21.15.62:0/468168496 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1230010640 con 0x560a5dfdb840 2023-07-14T16:26:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f1255c85b80 10 client.28097 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f1255c85b80 10 client.28097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f1255c85b80 20 client.28097 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f1255c85b80 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a5dff4a30 0x560a5dff6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f1255c85b80 4 client.28097 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f1255c85b80 1 -- 172.21.15.62:0/468168496 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a5df25e50 con 0x560a5dff4a30 2023-07-14T16:26:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.654+0000 7f124099c700 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a5dff4a30 0x560a5dff6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.655+0000 7f124099c700 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a5dff4a30 0x560a5dff6e40 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0473a1e078c500 server_cookie=5cbc4a3b286b4d6c in_seq=0 out_seq=0 2023-07-14T16:26:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.655+0000 7f121b7fe700 10 client.28097 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f121b7fe700 1 -- 172.21.15.62:0/468168496 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a5df25e50 con 0x560a5dff4a30 2023-07-14T16:26:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f121b7fe700 10 client.28097 handle_command_reply: tid=0 2023-07-14T16:26:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 client.28097 shutdown 2023-07-14T16:26:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 -- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 msgr2=0x560a5dfe91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe91b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1230011440 tx=0x7f1230011750 comp rx=0 tx=0).stop 2023-07-14T16:26:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 -- 172.21.15.62:0/468168496 shutdown_connections 2023-07-14T16:26:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 0x560a5dff01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5de376b0 0x560a5dfe8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a5dff4a30 0x560a5dff6e40 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 --2- 172.21.15.62:0/468168496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe91b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 -- 172.21.15.62:0/468168496 >> 172.21.15.62:0/468168496 conn(0x560a5dfd0be0 msgr2=0x560a5dfd54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f121b7fe700 0 client.28097 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.656+0000 7f1255c85b80 1 -- 172.21.15.62:0/468168496 shutdown_connections 2023-07-14T16:26:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.657+0000 7f1255c85b80 1 -- 172.21.15.62:0/468168496 wait complete. 2023-07-14T16:26:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.657+0000 7f1255c85b80 20 client.28097 trim_cache size 0 max 16384 2023-07-14T16:26:24.656 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:26:24.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:26:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.679+0000 7f1255c85b80 1 Processor -- start 2023-07-14T16:26:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.679+0000 7f1255c85b80 1 -- start start 2023-07-14T16:26:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5de376b0 0x560a5dfdf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 0x560a5dfe8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f1255c85b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5dfe9e90 con 0x560a5dfdb840 2023-07-14T16:26:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f1255c85b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5dfdbf10 con 0x560a5de376b0 2023-07-14T16:26:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f1255c85b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5dff1ee0 con 0x560a5dfdc6e0 2023-07-14T16:26:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f123b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f124099c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5de376b0 0x560a5dfdf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f123b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59214/0 (socket says 172.21.15.62:59214) 2023-07-14T16:26:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.680+0000 7f123b7fe700 1 -- 172.21.15.62:0/3759027285 learned_addr learned my addr 172.21.15.62:0/3759027285 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f124099c700 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 0x560a5dfe8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f123b7fe700 1 -- 172.21.15.62:0/3759027285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 msgr2=0x560a5dfe8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f123b7fe700 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 0x560a5dfe8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f123b7fe700 1 -- 172.21.15.62:0/3759027285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5de376b0 msgr2=0x560a5dfdf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f123b7fe700 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5de376b0 0x560a5dfdf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f123b7fe700 1 -- 172.21.15.62:0/3759027285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a5dd811f0 con 0x560a5dfdb840 2023-07-14T16:26:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f124099c700 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5de376b0 0x560a5dfdf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f123b7fe700 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdfb70 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1230005950 tx=0x7f1230011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4c27c9f2613133 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f1222ffd700 1 -- 172.21.15.62:0/3759027285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230016040 con 0x560a5dfdb840 2023-07-14T16:26:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f1222ffd700 1 -- 172.21.15.62:0/3759027285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1230010bc0 con 0x560a5dfdb840 2023-07-14T16:26:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.681+0000 7f1222ffd700 1 -- 172.21.15.62:0/3759027285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230021040 con 0x560a5dfdb840 2023-07-14T16:26:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 -- 172.21.15.62:0/3759027285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 msgr2=0x560a5dfdfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdfb70 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1230005950 tx=0x7f1230011b20 comp rx=0 tx=0).stop 2023-07-14T16:26:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 -- 172.21.15.62:0/3759027285 shutdown_connections 2023-07-14T16:26:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5dfdc6e0 0x560a5dfe8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5de376b0 0x560a5dfdf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3759027285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdb840 0x560a5dfdfb70 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 -- 172.21.15.62:0/3759027285 >> 172.21.15.62:0/3759027285 conn(0x560a5dfd0be0 msgr2=0x560a5dff6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 -- 172.21.15.62:0/3759027285 shutdown_connections 2023-07-14T16:26:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.682+0000 7f1255c85b80 1 -- 172.21.15.62:0/3759027285 wait complete. 2023-07-14T16:26:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.683+0000 7f1255c85b80 1 -- start start 2023-07-14T16:26:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.683+0000 7f1255c85b80 1 Processor -- start 2023-07-14T16:26:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.683+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfe4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.683+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.683+0000 7f1255c85b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdc6e0 0x560a5dff0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f1255c85b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5dfe5760 con 0x560a5dfdc6e0 2023-07-14T16:26:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f1255c85b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5dfe60f0 con 0x560a5dfdb840 2023-07-14T16:26:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f1255c85b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5dfe62b0 con 0x560a5de376b0 2023-07-14T16:26:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f1255c85b80 10 client.0 _ll_register_callbacks cb 0x7f123000bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f124099c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfe4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f123b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f123b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43050/0 (socket says 172.21.15.62:43050) 2023-07-14T16:26:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f123b7fe700 1 -- 172.21.15.62:0/3244176594 learned_addr learned my addr 172.21.15.62:0/3244176594 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f124099c700 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdc6e0 0x560a5dff0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f123b7fe700 1 -- 172.21.15.62:0/3244176594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 msgr2=0x560a5dfe4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f123b7fe700 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfe4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f123b7fe700 1 -- 172.21.15.62:0/3244176594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdc6e0 msgr2=0x560a5dff0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f123b7fe700 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdc6e0 0x560a5dff0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f123b7fe700 1 -- 172.21.15.62:0/3244176594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123402e040 con 0x560a5dfdb840 2023-07-14T16:26:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.684+0000 7f124099c700 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfe4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.685+0000 7f123b7fe700 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe50e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1230003950 tx=0x7f1230010870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46f729eafe1453c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.685+0000 7f121b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:26:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.685+0000 7f121b7fe700 1 -- 172.21.15.62:0/3244176594 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230021070 con 0x560a5dfdb840 2023-07-14T16:26:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.685+0000 7f121b7fe700 1 -- 172.21.15.62:0/3244176594 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1230000e40 con 0x560a5dfdb840 2023-07-14T16:26:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.685+0000 7f1255c85b80 1 -- 172.21.15.62:0/3244176594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f123400bda0 con 0x560a5dfdb840 2023-07-14T16:26:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f121b7fe700 1 -- 172.21.15.62:0/3244176594 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1230010d30 con 0x560a5dfdb840 2023-07-14T16:26:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f1255c85b80 10 client.28103 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f1255c85b80 1 -- 172.21.15.62:0/3244176594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560a5dd43c00 con 0x560a5dfdb840 2023-07-14T16:26:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f121b7fe700 1 -- 172.21.15.62:0/3244176594 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f123400bda0 con 0x560a5dfdb840 2023-07-14T16:26:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f121b7fe700 1 -- 172.21.15.62:0/3244176594 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1230030750 con 0x560a5dfdb840 2023-07-14T16:26:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f1255c85b80 10 client.28103 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f1255c85b80 10 client.28103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f1255c85b80 20 client.28103 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a5e04d120 0x560a5e04f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f1255c85b80 4 client.28103 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:26:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.686+0000 7f1255c85b80 1 -- 172.21.15.62:0/3244176594 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a5dee7ff0 con 0x560a5e04d120 2023-07-14T16:26:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.687+0000 7f124099c700 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a5e04d120 0x560a5e04f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.687+0000 7f124099c700 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a5e04d120 0x560a5e04f500 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc162190f19c80c server_cookie=c5850a362cb1f0b4 in_seq=0 out_seq=0 2023-07-14T16:26:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.687+0000 7f121b7fe700 10 client.28103 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.696+0000 7f121b7fe700 1 -- 172.21.15.62:0/3244176594 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a5dee7ff0 con 0x560a5e04d120 2023-07-14T16:26:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.696+0000 7f121b7fe700 10 client.28103 handle_command_reply: tid=0 2023-07-14T16:26:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.696+0000 7f1255c85b80 1 client.28103 shutdown 2023-07-14T16:26:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 -- 172.21.15.62:0/3244176594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 msgr2=0x560a5dfe50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe50e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1230003950 tx=0x7f1230010870 comp rx=0 tx=0).stop 2023-07-14T16:26:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 -- 172.21.15.62:0/3244176594 shutdown_connections 2023-07-14T16:26:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a5dfdc6e0 0x560a5dff0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a5de376b0 0x560a5dfe4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a5e04d120 0x560a5e04f500 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 --2- 172.21.15.62:0/3244176594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a5dfdb840 0x560a5dfe50e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 -- 172.21.15.62:0/3244176594 >> 172.21.15.62:0/3244176594 conn(0x560a5dfd0be0 msgr2=0x560a5dfe8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f121b7fe700 0 client.28103 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 -- 172.21.15.62:0/3244176594 shutdown_connections 2023-07-14T16:26:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 1 -- 172.21.15.62:0/3244176594 wait complete. 2023-07-14T16:26:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.697+0000 7f1255c85b80 20 client.28103 trim_cache size 0 max 16384 2023-07-14T16:26:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:26:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:26:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:26:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:26:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 -- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12240a11d0 msgr2=0x7f12240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12240a11d0 0x7f12240a3680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f122c096730 tx=0x7f122c018a60 comp rx=0 tx=0).stop 2023-07-14T16:26:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 -- 172.21.15.62:0/1436221371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c100620 msgr2=0x7f123c1959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c100620 0x7f123c1959f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f122c005fd0 tx=0x7f122c00eca0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 -- 172.21.15.62:0/1436221371 shutdown_connections 2023-07-14T16:26:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12240a11d0 0x7f12240a3680 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f123c105860 0x7f123c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f123c100620 0x7f123c1959f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 --2- 172.21.15.62:0/1436221371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f123c102f40 0x7f123c195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 -- 172.21.15.62:0/1436221371 >> 172.21.15.62:0/1436221371 conn(0x7f123c0fa000 msgr2=0x7f123c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 -- 172.21.15.62:0/1436221371 shutdown_connections 2023-07-14T16:26:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:24.698+0000 7f12423ff700 1 -- 172.21.15.62:0/1436221371 wait complete. 2023-07-14T16:26:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 -- 172.21.15.62:0/669473352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 msgr2=0x7fbb0c101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/669473352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 0x7fbb0c101020 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009990 tx=0x7fbb00009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 -- 172.21.15.62:0/669473352 shutdown_connections 2023-07-14T16:26:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/669473352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c101560 0x7fbb0c107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/669473352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb0c0ff9e0 0x7fbb0c0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/669473352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 0x7fbb0c101020 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 -- 172.21.15.62:0/669473352 >> 172.21.15.62:0/669473352 conn(0x7fbb0c0fafe0 msgr2=0x7fbb0c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 -- 172.21.15.62:0/669473352 shutdown_connections 2023-07-14T16:26:24.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.919+0000 7fbb10f9d700 1 -- 172.21.15.62:0/669473352 wait complete. 2023-07-14T16:26:24.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.920+0000 7fbb10f9d700 1 Processor -- start 2023-07-14T16:26:24.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb10f9d700 1 -- start start 2023-07-14T16:26:24.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb10f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb0c0ff9e0 0x7fbb0c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb10f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 0x7fbb0c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb10f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c101560 0x7fbb0c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb10f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c0735e0 con 0x7fbb0c100bd0 2023-07-14T16:26:24.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb10f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c073750 con 0x7fbb0c0ff9e0 2023-07-14T16:26:24.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb10f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c19f4e0 con 0x7fbb0c101560 2023-07-14T16:26:24.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 0x7fbb0c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.921+0000 7fbb0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 0x7fbb0c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59240/0 (socket says 172.21.15.62:59240) 2023-07-14T16:26:24.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0affd700 1 -- 172.21.15.62:0/2182014558 learned_addr learned my addr 172.21.15.62:0/2182014558 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:24.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0bfff700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c101560 0x7fbb0c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0b7fe700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb0c0ff9e0 0x7fbb0c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0affd700 1 -- 172.21.15.62:0/2182014558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c101560 msgr2=0x7fbb0c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0affd700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c101560 0x7fbb0c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0affd700 1 -- 172.21.15.62:0/2182014558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb0c0ff9e0 msgr2=0x7fbb0c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0affd700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb0c0ff9e0 0x7fbb0c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:24.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0affd700 1 -- 172.21.15.62:0/2182014558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000095f0 con 0x7fbb0c100bd0 2023-07-14T16:26:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.922+0000 7fbb0affd700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 0x7fbb0c072900 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005910 tx=0x7fbb00005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.923+0000 7fbb08ff9700 1 -- 172.21.15.62:0/2182014558 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0001d070 con 0x7fbb0c100bd0 2023-07-14T16:26:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.923+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0c19f760 con 0x7fbb0c100bd0 2023-07-14T16:26:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.923+0000 7fbb08ff9700 1 -- 172.21.15.62:0/2182014558 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb0000be90 con 0x7fbb0c100bd0 2023-07-14T16:26:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.923+0000 7fbb08ff9700 1 -- 172.21.15.62:0/2182014558 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb00022dd0 con 0x7fbb0c100bd0 2023-07-14T16:26:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.923+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c19fb90 con 0x7fbb0c100bd0 2023-07-14T16:26:24.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.924+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0c10b0b0 con 0x7fbb0c100bd0 2023-07-14T16:26:24.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.926+0000 7fbb08ff9700 1 -- 172.21.15.62:0/2182014558 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbb0000ba00 con 0x7fbb0c100bd0 2023-07-14T16:26:24.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.928+0000 7fbb08ff9700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbaf40a1240 0x7fbaf40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.929+0000 7fbb08ff9700 1 -- 172.21.15.62:0/2182014558 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbb000ce630 con 0x7fbb0c100bd0 2023-07-14T16:26:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.929+0000 7fbb0b7fe700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbaf40a1240 0x7fbaf40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:24.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.929+0000 7fbb0b7fe700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbaf40a1240 0x7fbaf40a36f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c100a30 tx=0x7fbafc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:24.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:24.931+0000 7fbb08ff9700 1 -- 172.21.15.62:0/2182014558 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbb00097dc0 con 0x7fbb0c100bd0 2023-07-14T16:26:25.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:24 smithi164 ceph-mon[97427]: pgmap v680: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:26:25.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:24 smithi062 ceph-mon[95543]: pgmap v680: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:26:25.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:24 smithi081 ceph-mon[127795]: pgmap v680: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:26:25.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.205+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb0c106820 con 0x7fbb0c100bd0 2023-07-14T16:26:25.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.206+0000 7fbb08ff9700 1 -- 172.21.15.62:0/2182014558 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fbb0009b3e0 con 0x7fbb0c100bd0 2023-07-14T16:26:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:26:25.182 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:26:25.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbaf40a1240 msgr2=0x7fbaf40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbaf40a1240 0x7fbaf40a36f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c100a30 tx=0x7fbafc00b410 comp rx=0 tx=0).stop 2023-07-14T16:26:25.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 msgr2=0x7fbb0c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 0x7fbb0c072900 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005910 tx=0x7fbb00005720 comp rx=0 tx=0).stop 2023-07-14T16:26:25.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 shutdown_connections 2023-07-14T16:26:25.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c101560 0x7fbb0c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbaf40a1240 0x7fbaf40a36f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb0c0ff9e0 0x7fbb0c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 --2- 172.21.15.62:0/2182014558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb0c100bd0 0x7fbb0c072900 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 >> 172.21.15.62:0/2182014558 conn(0x7fbb0c0fafe0 msgr2=0x7fbb0c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 shutdown_connections 2023-07-14T16:26:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.209+0000 7fbb10f9d700 1 -- 172.21.15.62:0/2182014558 wait complete. 2023-07-14T16:26:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:26:25.203 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:26:25.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.498+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3007066142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab48100560 msgr2=0x7fab48106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.498+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3007066142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab48100560 0x7fab48106e20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fab4000b0a0 tx=0x7fab4000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.499+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3007066142 shutdown_connections 2023-07-14T16:26:25.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.499+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3007066142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab48100560 0x7fab48106e20 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.499+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3007066142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab480fe9e0 0x7fab480fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.499+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3007066142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab480ffbd0 0x7fab48100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.499+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3007066142 >> 172.21.15.62:0/3007066142 conn(0x7fab480fa000 msgr2=0x7fab480fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.499+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3007066142 shutdown_connections 2023-07-14T16:26:25.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.499+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3007066142 wait complete. 2023-07-14T16:26:25.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab4e5ff700 1 Processor -- start 2023-07-14T16:26:25.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab4e5ff700 1 -- start start 2023-07-14T16:26:25.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab4e5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab480fe9e0 0x7fab4810a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab4e5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab480ffbd0 0x7fab48108fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab4e5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab48100560 0x7fab48109520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab4e5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48199760 con 0x7fab480ffbd0 2023-07-14T16:26:25.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab4e5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48109a60 con 0x7fab480fe9e0 2023-07-14T16:26:25.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab4e5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48109c20 con 0x7fab48100560 2023-07-14T16:26:25.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.500+0000 7fab477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab480ffbd0 0x7fab48108fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab480ffbd0 0x7fab48108fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59264/0 (socket says 172.21.15.62:59264) 2023-07-14T16:26:25.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab477fe700 1 -- 172.21.15.62:0/3289155364 learned_addr learned my addr 172.21.15.62:0/3289155364 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab47fff700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab480fe9e0 0x7fab4810a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab4cb9c700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab48100560 0x7fab48109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab477fe700 1 -- 172.21.15.62:0/3289155364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab48100560 msgr2=0x7fab48109520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab477fe700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab48100560 0x7fab48109520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab477fe700 1 -- 172.21.15.62:0/3289155364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab480fe9e0 msgr2=0x7fab4810a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab477fe700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab480fe9e0 0x7fab4810a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab477fe700 1 -- 172.21.15.62:0/3289155364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40009d00 con 0x7fab480ffbd0 2023-07-14T16:26:25.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.501+0000 7fab477fe700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab480ffbd0 0x7fab48108fe0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00d950 tx=0x7fab3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.502+0000 7fab457fa700 1 -- 172.21.15.62:0/3289155364 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00f890 con 0x7fab480ffbd0 2023-07-14T16:26:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.502+0000 7fab457fa700 1 -- 172.21.15.62:0/3289155364 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab3c00fe80 con 0x7fab480ffbd0 2023-07-14T16:26:25.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.502+0000 7fab457fa700 1 -- 172.21.15.62:0/3289155364 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00e3b0 con 0x7fab480ffbd0 2023-07-14T16:26:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.502+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3289155364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab481a4110 con 0x7fab480ffbd0 2023-07-14T16:26:25.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.502+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3289155364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab481a44e0 con 0x7fab480ffbd0 2023-07-14T16:26:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.506+0000 7fab457fa700 1 -- 172.21.15.62:0/3289155364 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fab3c010460 con 0x7fab480ffbd0 2023-07-14T16:26:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.507+0000 7fab61ecfb80 1 Processor -- start 2023-07-14T16:26:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.507+0000 7fab61ecfb80 1 -- start start 2023-07-14T16:26:25.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.507+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf774d0 0x55f05cf778e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.507+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.507+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 0x55f05cf790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.507+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05ce7ccb0 con 0x55f05cf77e20 2023-07-14T16:26:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.507+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cecfa40 con 0x55f05cf774d0 2023-07-14T16:26:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.507+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cec8ed0 con 0x55f05cf78c70 2023-07-14T16:26:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab4cb9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf774d0 0x55f05cf778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab4cb9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf774d0 0x55f05cf778e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43110/0 (socket says 172.21.15.62:43110) 2023-07-14T16:26:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab4cb9c700 1 -- 172.21.15.62:0/103157835 learned_addr learned my addr 172.21.15.62:0/103157835 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab47fff700 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab477fe700 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 0x55f05cf790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab457fa700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab300a1150 0x7fab300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab47fff700 1 -- 172.21.15.62:0/103157835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 msgr2=0x55f05cf790e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab47fff700 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 0x55f05cf790e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab47fff700 1 -- 172.21.15.62:0/103157835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf774d0 msgr2=0x55f05cf778e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab47fff700 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf774d0 0x55f05cf778e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab47fff700 1 -- 172.21.15.62:0/103157835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f05ccf6de0 con 0x55f05cf77e20 2023-07-14T16:26:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.508+0000 7fab457fa700 1 -- 172.21.15.62:0/3289155364 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fab3c0cc6a0 con 0x7fab480ffbd0 2023-07-14T16:26:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.509+0000 7fab4cb9c700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab300a1150 0x7fab300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.509+0000 7fab47fff700 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf786e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fab3800bc60 tx=0x7fab3800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c890aec30343ab96 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.509+0000 7fab2e7fc700 1 -- 172.21.15.62:0/103157835 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38010ca0 con 0x55f05cf77e20 2023-07-14T16:26:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.509+0000 7fab2e7fc700 1 -- 172.21.15.62:0/103157835 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab38015070 con 0x55f05cf77e20 2023-07-14T16:26:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.509+0000 7fab2e7fc700 1 -- 172.21.15.62:0/103157835 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38003d60 con 0x55f05cf77e20 2023-07-14T16:26:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab4cb9c700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab300a1150 0x7fab300a3600 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fab40007a30 tx=0x7fab400099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 -- 172.21.15.62:0/103157835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 msgr2=0x55f05cf786e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf786e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fab3800bc60 tx=0x7fab3800bf70 comp rx=0 tx=0).stop 2023-07-14T16:26:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 -- 172.21.15.62:0/103157835 shutdown_connections 2023-07-14T16:26:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 0x55f05cf790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf774d0 0x55f05cf778e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/103157835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf786e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 -- 172.21.15.62:0/103157835 >> 172.21.15.62:0/103157835 conn(0x55f05cf6cda0 msgr2=0x55f05cf721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 -- 172.21.15.62:0/103157835 shutdown_connections 2023-07-14T16:26:25.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.510+0000 7fab61ecfb80 1 -- 172.21.15.62:0/103157835 wait complete. 2023-07-14T16:26:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.511+0000 7fab61ecfb80 1 -- start start 2023-07-14T16:26:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.511+0000 7fab61ecfb80 1 Processor -- start 2023-07-14T16:26:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf81110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf81650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf78c70 0x55f05cf88690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05ce68ac0 con 0x55f05cf78c70 2023-07-14T16:26:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05ce687c0 con 0x55f05cf77e20 2023-07-14T16:26:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cd4c920 con 0x55f05cf774d0 2023-07-14T16:26:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab61ecfb80 10 client.0 _ll_register_callbacks cb 0x55f05ce66760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab4cb9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf78c70 0x55f05cf88690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab47fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf81110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab47fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf81110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52392/0 (socket says 172.21.15.62:52392) 2023-07-14T16:26:25.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab47fff700 1 -- 172.21.15.62:0/1376336089 learned_addr learned my addr 172.21.15.62:0/1376336089 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab477fe700 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf81650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab47fff700 1 -- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 msgr2=0x55f05cf81650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab47fff700 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf81650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab47fff700 1 -- 172.21.15.62:0/1376336089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf78c70 msgr2=0x55f05cf88690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.512+0000 7fab47fff700 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf78c70 0x55f05cf88690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.513+0000 7fab47fff700 1 -- 172.21.15.62:0/1376336089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3800b8c0 con 0x55f05cf774d0 2023-07-14T16:26:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.513+0000 7fab477fe700 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf81650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:25.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.513+0000 7fab4cb9c700 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf78c70 0x55f05cf88690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.513+0000 7fab47fff700 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf81110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab38010cb0 tx=0x7fab38005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79401392d6cc88 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.513+0000 7fab277fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:26:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.513+0000 7fab277fe700 1 -- 172.21.15.62:0/1376336089 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38005450 con 0x55f05cf774d0 2023-07-14T16:26:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.513+0000 7fab277fe700 1 -- 172.21.15.62:0/1376336089 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fab38017070 con 0x55f05cf774d0 2023-07-14T16:26:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.513+0000 7fab61ecfb80 1 -- 172.21.15.62:0/1376336089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f05ce224f0 con 0x55f05cf774d0 2023-07-14T16:26:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab277fe700 1 -- 172.21.15.62:0/1376336089 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab380055b0 con 0x55f05cf774d0 2023-07-14T16:26:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab61ecfb80 10 client.28018 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:25.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab61ecfb80 1 -- 172.21.15.62:0/1376336089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f05ccdfc60 con 0x55f05cf774d0 2023-07-14T16:26:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab277fe700 1 -- 172.21.15.62:0/1376336089 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f05ce224f0 con 0x55f05cf774d0 2023-07-14T16:26:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab277fe700 1 -- 172.21.15.62:0/1376336089 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fab380116a0 con 0x55f05cf774d0 2023-07-14T16:26:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab61ecfb80 10 client.28018 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab61ecfb80 10 client.28018 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:26:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab61ecfb80 20 client.28018 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f05cf8cdf0 0x55f05cf8f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab61ecfb80 4 client.28018 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.514+0000 7fab61ecfb80 1 -- 172.21.15.62:0/1376336089 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f05cec1eb0 con 0x55f05cf8cdf0 2023-07-14T16:26:25.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.515+0000 7fab477fe700 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f05cf8cdf0 0x55f05cf8f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.515+0000 7fab477fe700 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f05cf8cdf0 0x55f05cf8f200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa6d5d2d5a4f0aaa server_cookie=5747e7e706759de6 in_seq=0 out_seq=0 2023-07-14T16:26:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.515+0000 7fab277fe700 10 client.28018 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:26:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.516+0000 7fab277fe700 1 -- 172.21.15.62:0/1376336089 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f05cec1eb0 con 0x55f05cf8cdf0 2023-07-14T16:26:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.516+0000 7fab277fe700 10 client.28018 handle_command_reply: tid=0 2023-07-14T16:26:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.516+0000 7fab61ecfb80 1 client.28018 shutdown 2023-07-14T16:26:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.516+0000 7fab61ecfb80 1 -- 172.21.15.62:0/1376336089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 msgr2=0x55f05cf81110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.516+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf81110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab38010cb0 tx=0x7fab38005d50 comp rx=0 tx=0).stop 2023-07-14T16:26:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 1 -- 172.21.15.62:0/1376336089 shutdown_connections 2023-07-14T16:26:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f05cf8cdf0 0x55f05cf8f200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf78c70 0x55f05cf88690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf81110 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab277fe700 0 client.28018 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:26:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/1376336089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf81650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 1 -- 172.21.15.62:0/1376336089 >> 172.21.15.62:0/1376336089 conn(0x55f05cf6cda0 msgr2=0x55f05cf718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 1 -- 172.21.15.62:0/1376336089 shutdown_connections 2023-07-14T16:26:25.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 1 -- 172.21.15.62:0/1376336089 wait complete. 2023-07-14T16:26:25.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.517+0000 7fab61ecfb80 20 client.28018 trim_cache size 0 max 16384 2023-07-14T16:26:25.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.532+0000 7fab61ecfb80 1 Processor -- start 2023-07-14T16:26:25.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.532+0000 7fab61ecfb80 1 -- start start 2023-07-14T16:26:25.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf88650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf78c70 0x55f05cf93b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cf94080 con 0x55f05cf77e20 2023-07-14T16:26:25.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cf8cac0 con 0x55f05cf78c70 2023-07-14T16:26:25.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cf73a00 con 0x55f05cf774d0 2023-07-14T16:26:25.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf88650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf88650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59290/0 (socket says 172.21.15.62:59290) 2023-07-14T16:26:25.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab477fe700 1 -- 172.21.15.62:0/380209901 learned_addr learned my addr 172.21.15.62:0/380209901 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab4cb9c700 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf78c70 0x55f05cf93b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab47fff700 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab477fe700 1 -- 172.21.15.62:0/380209901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 msgr2=0x55f05cf807c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab477fe700 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf807c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab477fe700 1 -- 172.21.15.62:0/380209901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf78c70 msgr2=0x55f05cf93b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab477fe700 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf78c70 0x55f05cf93b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab477fe700 1 -- 172.21.15.62:0/380209901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f05cd1d250 con 0x55f05cf77e20 2023-07-14T16:26:25.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.533+0000 7fab47fff700 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:25.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.534+0000 7fab477fe700 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf88650 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fab3c005950 tx=0x7fab3c0d2550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5936508298cb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.534+0000 7fab2effd700 1 -- 172.21.15.62:0/380209901 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c014070 con 0x55f05cf77e20 2023-07-14T16:26:25.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.534+0000 7fab2effd700 1 -- 172.21.15.62:0/380209901 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab3c099420 con 0x55f05cf77e20 2023-07-14T16:26:25.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.534+0000 7fab2effd700 1 -- 172.21.15.62:0/380209901 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c097e60 con 0x55f05cf77e20 2023-07-14T16:26:25.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 -- 172.21.15.62:0/380209901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 msgr2=0x55f05cf88650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf88650 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fab3c005950 tx=0x7fab3c0d2550 comp rx=0 tx=0).stop 2023-07-14T16:26:25.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 -- 172.21.15.62:0/380209901 shutdown_connections 2023-07-14T16:26:25.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf78c70 0x55f05cf93b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf774d0 0x55f05cf807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/380209901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf77e20 0x55f05cf88650 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 -- 172.21.15.62:0/380209901 >> 172.21.15.62:0/380209901 conn(0x55f05cf6cda0 msgr2=0x55f05cf7fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 -- 172.21.15.62:0/380209901 shutdown_connections 2023-07-14T16:26:25.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 -- 172.21.15.62:0/380209901 wait complete. 2023-07-14T16:26:25.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.535+0000 7fab61ecfb80 1 -- start start 2023-07-14T16:26:25.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.536+0000 7fab61ecfb80 1 Processor -- start 2023-07-14T16:26:25.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.536+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf774d0 0x55f05cf84510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.536+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf84a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.536+0000 7fab61ecfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 0x55f05cf87a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.536+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cf87fb0 con 0x55f05cf774d0 2023-07-14T16:26:25.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.536+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cf88120 con 0x55f05cf77e20 2023-07-14T16:26:25.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.536+0000 7fab61ecfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05cf882e0 con 0x55f05cf78c70 2023-07-14T16:26:25.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab61ecfb80 10 client.0 _ll_register_callbacks cb 0x7fab40009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:25.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf774d0 0x55f05cf84510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab47fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf774d0 0x55f05cf84510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59300/0 (socket says 172.21.15.62:59300) 2023-07-14T16:26:25.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab47fff700 1 -- 172.21.15.62:0/2557209498 learned_addr learned my addr 172.21.15.62:0/2557209498 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab477fe700 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf84a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab47fff700 1 -- 172.21.15.62:0/2557209498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 msgr2=0x55f05cf87a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab47fff700 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 0x55f05cf87a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab47fff700 1 -- 172.21.15.62:0/2557209498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 msgr2=0x55f05cf84a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab47fff700 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf84a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.537+0000 7fab47fff700 1 -- 172.21.15.62:0/2557209498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab481a44e0 con 0x55f05cf774d0 2023-07-14T16:26:25.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab47fff700 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf774d0 0x55f05cf84510 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fab38011a30 tx=0x7fab38011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620e2d37193de3b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab277fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:25.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab277fe700 1 -- 172.21.15.62:0/2557209498 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38011640 con 0x55f05cf774d0 2023-07-14T16:26:25.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab277fe700 1 -- 172.21.15.62:0/2557209498 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fab38017050 con 0x55f05cf774d0 2023-07-14T16:26:25.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab61ecfb80 1 -- 172.21.15.62:0/2557209498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab38015070 con 0x55f05cf774d0 2023-07-14T16:26:25.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab277fe700 1 -- 172.21.15.62:0/2557209498 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab38011d20 con 0x55f05cf774d0 2023-07-14T16:26:25.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab61ecfb80 10 client.18525 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:25.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab61ecfb80 1 -- 172.21.15.62:0/2557209498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f05ccf52b0 con 0x55f05cf774d0 2023-07-14T16:26:25.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.538+0000 7fab277fe700 1 -- 172.21.15.62:0/2557209498 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab38015070 con 0x55f05cf774d0 2023-07-14T16:26:25.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.539+0000 7fab277fe700 1 -- 172.21.15.62:0/2557209498 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fab38003ad0 con 0x55f05cf774d0 2023-07-14T16:26:25.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.539+0000 7fab61ecfb80 10 client.18525 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:25.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.539+0000 7fab61ecfb80 10 client.18525 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:26:25.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.539+0000 7fab61ecfb80 20 client.18525 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:25.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.539+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f05cfe1970 0x55f05cfe3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.539+0000 7fab61ecfb80 4 client.18525 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:26:25.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.539+0000 7fab61ecfb80 1 -- 172.21.15.62:0/2557209498 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f05ce84050 con 0x55f05cfe1970 2023-07-14T16:26:25.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.539+0000 7fab477fe700 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f05cfe1970 0x55f05cfe3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab477fe700 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f05cfe1970 0x55f05cfe3d80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9734d613bf0602a server_cookie=f5464d12dfc6ce38 in_seq=0 out_seq=0 2023-07-14T16:26:25.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab277fe700 10 client.18525 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:26:25.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab277fe700 1 -- 172.21.15.62:0/2557209498 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f05ce84050 con 0x55f05cfe1970 2023-07-14T16:26:25.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab277fe700 10 client.18525 handle_command_reply: tid=0 2023-07-14T16:26:25.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 client.18525 shutdown 2023-07-14T16:26:25.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 -- 172.21.15.62:0/2557209498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf774d0 msgr2=0x55f05cf84510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf774d0 0x55f05cf84510 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fab38011a30 tx=0x7fab38011c40 comp rx=0 tx=0).stop 2023-07-14T16:26:25.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 -- 172.21.15.62:0/2557209498 shutdown_connections 2023-07-14T16:26:25.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55f05cfe1970 0x55f05cfe3d80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f05cf78c70 0x55f05cf87a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f05cf774d0 0x55f05cf84510 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 --2- 172.21.15.62:0/2557209498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f05cf77e20 0x55f05cf84a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab61ecfb80 1 -- 172.21.15.62:0/2557209498 >> 172.21.15.62:0/2557209498 conn(0x55f05cf6cda0 msgr2=0x55f05cf7bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.540+0000 7fab277fe700 0 client.18525 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:26:25.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab61ecfb80 1 -- 172.21.15.62:0/2557209498 shutdown_connections 2023-07-14T16:26:25.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab61ecfb80 1 -- 172.21.15.62:0/2557209498 wait complete. 2023-07-14T16:26:25.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab61ecfb80 20 client.18525 trim_cache size 0 max 16384 2023-07-14T16:26:25.559 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:26:25.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab300a1150 msgr2=0x7fab300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab300a1150 0x7fab300a3600 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fab40007a30 tx=0x7fab400099e0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab480ffbd0 msgr2=0x7fab48108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab480ffbd0 0x7fab48108fe0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00d950 tx=0x7fab3c00dc60 comp rx=0 tx=0).stop 2023-07-14T16:26:25.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3289155364 shutdown_connections 2023-07-14T16:26:25.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.541+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab48100560 0x7fab48109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.542+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fab300a1150 0x7fab300a3600 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.542+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab480fe9e0 0x7fab4810a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.542+0000 7fab4e5ff700 1 --2- 172.21.15.62:0/3289155364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab480ffbd0 0x7fab48108fe0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.542+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3289155364 >> 172.21.15.62:0/3289155364 conn(0x7fab480fa000 msgr2=0x7fab48103110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.542+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3289155364 shutdown_connections 2023-07-14T16:26:25.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.542+0000 7fab4e5ff700 1 -- 172.21.15.62:0/3289155364 wait complete. 2023-07-14T16:26:25.562 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:26:25.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.843+0000 7fb0301ff700 1 -- 172.21.15.62:0/2225454839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb028100560 msgr2=0x7fb028102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.843+0000 7fb0301ff700 1 --2- 172.21.15.62:0/2225454839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb028100560 0x7fb028102940 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb024009990 tx=0x7fb024009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.844+0000 7fb0301ff700 1 -- 172.21.15.62:0/2225454839 shutdown_connections 2023-07-14T16:26:25.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.844+0000 7fb0301ff700 1 --2- 172.21.15.62:0/2225454839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0281057a0 0x7fb028107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.844+0000 7fb0301ff700 1 --2- 172.21.15.62:0/2225454839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb028100560 0x7fb028102940 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.844+0000 7fb0301ff700 1 --2- 172.21.15.62:0/2225454839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb028102e80 0x7fb028105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.844+0000 7fb0301ff700 1 -- 172.21.15.62:0/2225454839 >> 172.21.15.62:0/2225454839 conn(0x7fb0280fa000 msgr2=0x7fb0280fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.844+0000 7fb0301ff700 1 -- 172.21.15.62:0/2225454839 shutdown_connections 2023-07-14T16:26:25.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.844+0000 7fb0301ff700 1 -- 172.21.15.62:0/2225454839 wait complete. 2023-07-14T16:26:25.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.845+0000 7fb0301ff700 1 Processor -- start 2023-07-14T16:26:25.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb0301ff700 1 -- start start 2023-07-14T16:26:25.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb0301ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb028100560 0x7fb0280ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb0301ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb028102e80 0x7fb0280fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb0301ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0281057a0 0x7fb0280fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb0301ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280ff1f0 con 0x7fb0281057a0 2023-07-14T16:26:25.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb0301ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280ff360 con 0x7fb028100560 2023-07-14T16:26:25.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb0301ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02819f820 con 0x7fb028102e80 2023-07-14T16:26:25.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb028100560 0x7fb0280ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb02d79a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb028102e80 0x7fb0280fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb028100560 0x7fb0280ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43154/0 (socket says 172.21.15.62:43154) 2023-07-14T16:26:25.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb02e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0281057a0 0x7fb0280fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.846+0000 7fb02df9b700 1 -- 172.21.15.62:0/829518883 learned_addr learned my addr 172.21.15.62:0/829518883 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.847+0000 7fb02df9b700 1 -- 172.21.15.62:0/829518883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb028102e80 msgr2=0x7fb0280fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.847+0000 7fb02df9b700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb028102e80 0x7fb0280fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.847+0000 7fb02df9b700 1 -- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0281057a0 msgr2=0x7fb0280fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.847+0000 7fb02df9b700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0281057a0 0x7fb0280fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.847+0000 7fb02df9b700 1 -- 172.21.15.62:0/829518883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0240095f0 con 0x7fb028100560 2023-07-14T16:26:25.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.847+0000 7fb02d79a700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb028102e80 0x7fb0280fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:25.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.847+0000 7fb02df9b700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb028100560 0x7fb0280ffed0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb0240098e0 tx=0x7fb024004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.848+0000 7fb01effd700 1 -- 172.21.15.62:0/829518883 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02401d070 con 0x7fb028100560 2023-07-14T16:26:25.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.848+0000 7fb01effd700 1 -- 172.21.15.62:0/829518883 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02400bdb0 con 0x7fb028100560 2023-07-14T16:26:25.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.848+0000 7fb0301ff700 1 -- 172.21.15.62:0/829518883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02819faa0 con 0x7fb028100560 2023-07-14T16:26:25.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.848+0000 7fb01effd700 1 -- 172.21.15.62:0/829518883 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024022d20 con 0x7fb028100560 2023-07-14T16:26:25.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.848+0000 7fb0301ff700 1 -- 172.21.15.62:0/829518883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02819fec0 con 0x7fb028100560 2023-07-14T16:26:25.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.851+0000 7fb01effd700 1 -- 172.21.15.62:0/829518883 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb02400b920 con 0x7fb028100560 2023-07-14T16:26:25.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.853+0000 7fb01effd700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0140a1150 0x7fb0140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.853+0000 7fb01effd700 1 -- 172.21.15.62:0/829518883 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0240cdf50 con 0x7fb028100560 2023-07-14T16:26:25.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.853+0000 7fb02d79a700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0140a1150 0x7fb0140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.853+0000 7fb043a32b80 1 Processor -- start 2023-07-14T16:26:25.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb043a32b80 1 -- start start 2023-07-14T16:26:25.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820988e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x56288209a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb043a32b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562881f9dcb0 con 0x5628820984d0 2023-07-14T16:26:25.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb043a32b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562881ff0a40 con 0x562882099c70 2023-07-14T16:26:25.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb043a32b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562881fe9ed0 con 0x562882098e20 2023-07-14T16:26:25.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb02d79a700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0140a1150 0x7fb0140a3600 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb018011490 tx=0x7fb018009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.854+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02d79a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x56288209a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x56288209a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43164/0 (socket says 172.21.15.62:43164) 2023-07-14T16:26:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02d79a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52446/0 (socket says 172.21.15.62:52446) 2023-07-14T16:26:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 -- 172.21.15.62:0/4163492004 learned_addr learned my addr 172.21.15.62:0/4163492004 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 -- 172.21.15.62:0/4163492004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 msgr2=0x5628820996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 --2- 172.21.15.62:0/4163492004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 -- 172.21.15.62:0/4163492004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 msgr2=0x5628820988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 --2- 172.21.15.62:0/4163492004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 -- 172.21.15.62:0/4163492004 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562881e17de0 con 0x562882099c70 2023-07-14T16:26:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02d79a700 1 --2- 172.21.15.62:0/4163492004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:25.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.855+0000 7fb02e79c700 1 --2- 172.21.15.62:0/4163492004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x56288209a0e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb020011600 tx=0x7fb020009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8edeb54543560df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.856+0000 7fb013fff700 1 -- 172.21.15.62:0/4163492004 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000e040 con 0x562882099c70 2023-07-14T16:26:25.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.856+0000 7fb013fff700 1 -- 172.21.15.62:0/4163492004 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02000f040 con 0x562882099c70 2023-07-14T16:26:25.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.856+0000 7fb013fff700 1 -- 172.21.15.62:0/4163492004 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200045e0 con 0x562882099c70 2023-07-14T16:26:25.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.856+0000 7fb043a32b80 1 -- 172.21.15.62:0/4163492004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 msgr2=0x56288209a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.856+0000 7fb043a32b80 1 --2- 172.21.15.62:0/4163492004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x56288209a0e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb020011600 tx=0x7fb020009920 comp rx=0 tx=0).stop 2023-07-14T16:26:25.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.857+0000 7fb043a32b80 1 -- 172.21.15.62:0/4163492004 shutdown_connections 2023-07-14T16:26:25.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.857+0000 7fb043a32b80 1 --2- 172.21.15.62:0/4163492004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x56288209a0e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.857+0000 7fb043a32b80 1 --2- 172.21.15.62:0/4163492004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.857+0000 7fb043a32b80 1 --2- 172.21.15.62:0/4163492004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.857+0000 7fb043a32b80 1 -- 172.21.15.62:0/4163492004 >> 172.21.15.62:0/4163492004 conn(0x56288208dda0 msgr2=0x5628820931d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.857+0000 7fb043a32b80 1 -- 172.21.15.62:0/4163492004 shutdown_connections 2023-07-14T16:26:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.857+0000 7fb043a32b80 1 -- 172.21.15.62:0/4163492004 wait complete. 2023-07-14T16:26:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.857+0000 7fb043a32b80 1 -- start start 2023-07-14T16:26:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.858+0000 7fb043a32b80 1 Processor -- start 2023-07-14T16:26:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.858+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628820984d0 0x5628820a2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.858+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882098e20 0x5628820a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.858+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x5628820a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.858+0000 7fb043a32b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562881f89ac0 con 0x562882098e20 2023-07-14T16:26:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.858+0000 7fb043a32b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562881f897c0 con 0x562882099c70 2023-07-14T16:26:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.858+0000 7fb043a32b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562881e6d920 con 0x5628820984d0 2023-07-14T16:26:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.858+0000 7fb043a32b80 10 client.0 _ll_register_callbacks cb 0x562881f87760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628820984d0 0x5628820a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628820984d0 0x5628820a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52458/0 (socket says 172.21.15.62:52458) 2023-07-14T16:26:25.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 -- 172.21.15.62:0/1776411424 learned_addr learned my addr 172.21.15.62:0/1776411424 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02d79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882098e20 0x5628820a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02e79c700 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x5628820a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 -- 172.21.15.62:0/1776411424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 msgr2=0x5628820a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x5628820a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 -- 172.21.15.62:0/1776411424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882098e20 msgr2=0x5628820a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882098e20 0x5628820a26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 -- 172.21.15.62:0/1776411424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018017070 con 0x5628820984d0 2023-07-14T16:26:25.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02d79a700 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882098e20 0x5628820a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:25.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02e79c700 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x5628820a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:25.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb02df9b700 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628820984d0 0x5628820a2190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0240cbdd0 tx=0x7fb024009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f75813139397f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.859+0000 7fb0117fa700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:26:25.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb0117fa700 1 -- 172.21.15.62:0/1776411424 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02409b400 con 0x5628820984d0 2023-07-14T16:26:25.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb0117fa700 1 -- 172.21.15.62:0/1776411424 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb02409f430 con 0x5628820984d0 2023-07-14T16:26:25.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb043a32b80 1 -- 172.21.15.62:0/1776411424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562881f434f0 con 0x5628820984d0 2023-07-14T16:26:25.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb0117fa700 1 -- 172.21.15.62:0/1776411424 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0240a37c0 con 0x5628820984d0 2023-07-14T16:26:25.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb043a32b80 10 client.28033 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:25.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb043a32b80 1 -- 172.21.15.62:0/1776411424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb0200112b0 con 0x5628820984d0 2023-07-14T16:26:25.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb0117fa700 1 -- 172.21.15.62:0/1776411424 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0240c9020 con 0x5628820984d0 2023-07-14T16:26:25.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb0117fa700 1 -- 172.21.15.62:0/1776411424 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb02409a9f0 con 0x5628820984d0 2023-07-14T16:26:25.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb043a32b80 10 client.28033 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:25.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb043a32b80 10 client.28033 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:26:25.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.860+0000 7fb043a32b80 20 client.28033 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:25.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.861+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5628820adbe0 0x5628820affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.861+0000 7fb043a32b80 4 client.28033 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:25.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.861+0000 7fb043a32b80 1 -- 172.21.15.62:0/1776411424 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562881fe2eb0 con 0x5628820adbe0 2023-07-14T16:26:25.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.861+0000 7fb02d79a700 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5628820adbe0 0x5628820affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.862+0000 7fb02d79a700 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5628820adbe0 0x5628820affc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e994939e8c99275 server_cookie=402e09651c1eaea0 in_seq=0 out_seq=0 2023-07-14T16:26:25.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.862+0000 7fb0117fa700 10 client.28033 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:26:25.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.862+0000 7fb0117fa700 1 -- 172.21.15.62:0/1776411424 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562881fe2eb0 con 0x5628820adbe0 2023-07-14T16:26:25.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.862+0000 7fb0117fa700 10 client.28033 handle_command_reply: tid=0 2023-07-14T16:26:25.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.862+0000 7fb043a32b80 1 client.28033 shutdown 2023-07-14T16:26:25.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 -- 172.21.15.62:0/1776411424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628820984d0 msgr2=0x5628820a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628820984d0 0x5628820a2190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0240cbdd0 tx=0x7fb024009310 comp rx=0 tx=0).stop 2023-07-14T16:26:25.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 -- 172.21.15.62:0/1776411424 shutdown_connections 2023-07-14T16:26:25.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5628820adbe0 0x5628820affc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882099c70 0x5628820a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5628820984d0 0x5628820a2190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb0117fa700 0 client.28033 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:26:25.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1776411424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882098e20 0x5628820a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 -- 172.21.15.62:0/1776411424 >> 172.21.15.62:0/1776411424 conn(0x56288208dda0 msgr2=0x5628820928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 -- 172.21.15.62:0/1776411424 shutdown_connections 2023-07-14T16:26:25.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 1 -- 172.21.15.62:0/1776411424 wait complete. 2023-07-14T16:26:25.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.863+0000 7fb043a32b80 20 client.28033 trim_cache size 0 max 16384 2023-07-14T16:26:25.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.878+0000 7fb043a32b80 1 Processor -- start 2023-07-14T16:26:25.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.878+0000 7fb043a32b80 1 -- start start 2023-07-14T16:26:25.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5628820984d0 0x56288209a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882099c70 0x5628820b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb043a32b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562882094d30 con 0x562882099c70 2023-07-14T16:26:25.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb043a32b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628820b4e60 con 0x5628820984d0 2023-07-14T16:26:25.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb043a32b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628820a3090 con 0x562882098e20 2023-07-14T16:26:25.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5628820984d0 0x56288209a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb02d79a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5628820984d0 0x56288209a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43170/0 (socket says 172.21.15.62:43170) 2023-07-14T16:26:25.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb02df9b700 1 -- 172.21.15.62:0/2299523860 learned_addr learned my addr 172.21.15.62:0/2299523860 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.879+0000 7fb02e79c700 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882099c70 0x5628820b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb02df9b700 1 -- 172.21.15.62:0/2299523860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 msgr2=0x5628820a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb02df9b700 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb02df9b700 1 -- 172.21.15.62:0/2299523860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882099c70 msgr2=0x5628820b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb02df9b700 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882099c70 0x5628820b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb02df9b700 1 -- 172.21.15.62:0/2299523860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562881e00c60 con 0x5628820984d0 2023-07-14T16:26:25.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb02d79a700 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:25.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb02df9b700 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5628820984d0 0x56288209a040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb0240977f0 tx=0x7fb0240a1410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d27d54947a85d8d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb01cff9700 1 -- 172.21.15.62:0/2299523860 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240a35c0 con 0x5628820984d0 2023-07-14T16:26:25.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb01cff9700 1 -- 172.21.15.62:0/2299523860 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0240c9020 con 0x5628820984d0 2023-07-14T16:26:25.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.880+0000 7fb01cff9700 1 -- 172.21.15.62:0/2299523860 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024022590 con 0x5628820984d0 2023-07-14T16:26:25.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 -- 172.21.15.62:0/2299523860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5628820984d0 msgr2=0x56288209a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5628820984d0 0x56288209a040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb0240977f0 tx=0x7fb0240a1410 comp rx=0 tx=0).stop 2023-07-14T16:26:25.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 -- 172.21.15.62:0/2299523860 shutdown_connections 2023-07-14T16:26:25.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562882099c70 0x5628820b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5628820984d0 0x56288209a040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 --2- 172.21.15.62:0/2299523860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882098e20 0x5628820a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 -- 172.21.15.62:0/2299523860 >> 172.21.15.62:0/2299523860 conn(0x56288208dda0 msgr2=0x5628820931a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 -- 172.21.15.62:0/2299523860 shutdown_connections 2023-07-14T16:26:25.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.881+0000 7fb043a32b80 1 -- 172.21.15.62:0/2299523860 wait complete. 2023-07-14T16:26:25.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.882+0000 7fb043a32b80 1 -- start start 2023-07-14T16:26:25.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.882+0000 7fb043a32b80 1 Processor -- start 2023-07-14T16:26:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.882+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.882+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882098e20 0x5628820a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.882+0000 7fb043a32b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882099c70 0x5628820a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.882+0000 7fb043a32b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628820a8c00 con 0x5628820984d0 2023-07-14T16:26:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb043a32b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628820a8d70 con 0x562882098e20 2023-07-14T16:26:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb043a32b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628820a8f30 con 0x562882099c70 2023-07-14T16:26:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb043a32b80 10 client.0 _ll_register_callbacks cb 0x562881f87760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02d79a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882098e20 0x5628820a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02d79a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882098e20 0x5628820a5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43174/0 (socket says 172.21.15.62:43174) 2023-07-14T16:26:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59350/0 (socket says 172.21.15.62:59350) 2023-07-14T16:26:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02d79a700 1 -- 172.21.15.62:0/1233543078 learned_addr learned my addr 172.21.15.62:0/1233543078 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02e79c700 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882099c70 0x5628820a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02d79a700 1 -- 172.21.15.62:0/1233543078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882099c70 msgr2=0x5628820a86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02d79a700 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882099c70 0x5628820a86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02d79a700 1 -- 172.21.15.62:0/1233543078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 msgr2=0x5628820a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:25.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02d79a700 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.883+0000 7fb02d79a700 1 -- 172.21.15.62:0/1233543078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018029950 con 0x562882098e20 2023-07-14T16:26:25.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.884+0000 7fb02d79a700 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882098e20 0x5628820a5da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5628820b08a0 tx=0x7fb0180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391ee3605148d949 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:25.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.884+0000 7fb0117fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:26:25.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.884+0000 7fb0117fa700 1 -- 172.21.15.62:0/1233543078 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018024040 con 0x562882098e20 2023-07-14T16:26:25.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.884+0000 7fb043a32b80 1 -- 172.21.15.62:0/1233543078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb02000f040 con 0x562882098e20 2023-07-14T16:26:25.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.884+0000 7fb0117fa700 1 -- 172.21.15.62:0/1233543078 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb018029510 con 0x562882098e20 2023-07-14T16:26:25.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.884+0000 7fb0117fa700 1 -- 172.21.15.62:0/1233543078 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb018037030 con 0x562882098e20 2023-07-14T16:26:25.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.884+0000 7fb043a32b80 10 client.28145 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:25.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.884+0000 7fb043a32b80 1 -- 172.21.15.62:0/1233543078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fb0200112b0 con 0x562882098e20 2023-07-14T16:26:25.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb0117fa700 1 -- 172.21.15.62:0/1233543078 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02000f040 con 0x562882098e20 2023-07-14T16:26:25.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb0117fa700 1 -- 172.21.15.62:0/1233543078 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb01800e3b0 con 0x562882098e20 2023-07-14T16:26:25.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb043a32b80 10 client.28145 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:25.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb043a32b80 10 client.28145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:26:25.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb043a32b80 20 client.28145 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:25.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562882102790 0x562882104ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:25.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb043a32b80 4 client.28145 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:26:25.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb043a32b80 1 -- 172.21.15.62:0/1233543078 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562881fa5050 con 0x562882102790 2023-07-14T16:26:25.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.885+0000 7fb02df9b700 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562882102790 0x562882104ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:25.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb02df9b700 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562882102790 0x562882104ba0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd87af4657ed2131 server_cookie=de76cbd198a6d563 in_seq=0 out_seq=0 2023-07-14T16:26:25.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb0117fa700 10 client.28145 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:26:25.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb0117fa700 1 -- 172.21.15.62:0/1233543078 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562881fa5050 con 0x562882102790 2023-07-14T16:26:25.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb0117fa700 10 client.28145 handle_command_reply: tid=0 2023-07-14T16:26:25.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb043a32b80 1 client.28145 shutdown 2023-07-14T16:26:25.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb043a32b80 1 -- 172.21.15.62:0/1233543078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882098e20 msgr2=0x5628820a5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882098e20 0x5628820a5da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5628820b08a0 tx=0x7fb0180047c0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb043a32b80 1 -- 172.21.15.62:0/1233543078 shutdown_connections 2023-07-14T16:26:25.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.886+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x562882102790 0x562882104ba0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.887+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562882099c70 0x5628820a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.887+0000 7fb0117fa700 0 client.28145 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:26:25.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.887+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5628820984d0 0x5628820a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.887+0000 7fb043a32b80 1 --2- 172.21.15.62:0/1233543078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562882098e20 0x5628820a5da0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.887+0000 7fb043a32b80 1 -- 172.21.15.62:0/1233543078 >> 172.21.15.62:0/1233543078 conn(0x56288208dda0 msgr2=0x562882093160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.887+0000 7fb043a32b80 1 -- 172.21.15.62:0/1233543078 shutdown_connections 2023-07-14T16:26:25.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.887+0000 7fb043a32b80 1 -- 172.21.15.62:0/1233543078 wait complete. 2023-07-14T16:26:25.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.887+0000 7fb043a32b80 20 client.28145 trim_cache size 0 max 16384 2023-07-14T16:26:25.870 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:26:25.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 -- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0140a1150 msgr2=0x7fb0140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0140a1150 0x7fb0140a3600 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb018011490 tx=0x7fb018009260 comp rx=0 tx=0).stop 2023-07-14T16:26:25.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 -- 172.21.15.62:0/829518883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb028100560 msgr2=0x7fb0280ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:25.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb028100560 0x7fb0280ffed0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb0240098e0 tx=0x7fb024004730 comp rx=0 tx=0).stop 2023-07-14T16:26:25.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 -- 172.21.15.62:0/829518883 shutdown_connections 2023-07-14T16:26:25.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0281057a0 0x7fb0280fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0140a1150 0x7fb0140a3600 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb028100560 0x7fb0280ffed0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 --2- 172.21.15.62:0/829518883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb028102e80 0x7fb0280fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:25.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 -- 172.21.15.62:0/829518883 >> 172.21.15.62:0/829518883 conn(0x7fb0280fa000 msgr2=0x7fb0281011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:25.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 -- 172.21.15.62:0/829518883 shutdown_connections 2023-07-14T16:26:25.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:25.888+0000 7fb0301ff700 1 -- 172.21.15.62:0/829518883 wait complete. 2023-07-14T16:26:25.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:25 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2182014558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:26:25.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:25 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:26:25.885 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:26:26.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:25 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2182014558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:26:26.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:25 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:26:26.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 -- 172.21.15.62:0/2405127331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5818100ce0 msgr2=0x7f5818101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2405127331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5818100ce0 0x7f5818101130 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f580c009990 tx=0x7f580c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 -- 172.21.15.62:0/2405127331 shutdown_connections 2023-07-14T16:26:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2405127331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5818101670 0x7f5818107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2405127331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58180ffaf0 0x7f58180fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2405127331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5818100ce0 0x7f5818101130 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 -- 172.21.15.62:0/2405127331 >> 172.21.15.62:0/2405127331 conn(0x7f58180fb130 msgr2=0x7f58180fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 -- 172.21.15.62:0/2405127331 shutdown_connections 2023-07-14T16:26:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.155+0000 7f581e9ff700 1 -- 172.21.15.62:0/2405127331 wait complete. 2023-07-14T16:26:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.157+0000 7f581e9ff700 1 Processor -- start 2023-07-14T16:26:26.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.157+0000 7f581e9ff700 1 -- start start 2023-07-14T16:26:26.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.157+0000 7f581e9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58180ffaf0 0x7f58181957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.157+0000 7f581e9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5818100ce0 0x7f5818195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.157+0000 7f581e9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5818101670 0x7f581819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.157+0000 7f581e9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581819d330 con 0x7f5818100ce0 2023-07-14T16:26:26.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.157+0000 7f581e9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581819d470 con 0x7f58180ffaf0 2023-07-14T16:26:26.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.157+0000 7f581e9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581819d600 con 0x7f5818101670 2023-07-14T16:26:26.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58180ffaf0 0x7f58181957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58180ffaf0 0x7f58181957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43200/0 (socket says 172.21.15.62:43200) 2023-07-14T16:26:26.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 -- 172.21.15.62:0/2497678515 learned_addr learned my addr 172.21.15.62:0/2497678515 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:26.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f58177fe700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5818100ce0 0x7f5818195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f581cf9c700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5818101670 0x7f581819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 -- 172.21.15.62:0/2497678515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5818101670 msgr2=0x7f581819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5818101670 0x7f581819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 -- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5818100ce0 msgr2=0x7f5818195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5818100ce0 0x7f5818195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 -- 172.21.15.62:0/2497678515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0095f0 con 0x7f58180ffaf0 2023-07-14T16:26:26.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f58177fe700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5818100ce0 0x7f5818195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:26.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.158+0000 7f5817fff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58180ffaf0 0x7f58181957b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f580800e970 tx=0x7f580800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:26.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.159+0000 7f58157fa700 1 -- 172.21.15.62:0/2497678515 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808009790 con 0x7f58180ffaf0 2023-07-14T16:26:26.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.159+0000 7f58157fa700 1 -- 172.21.15.62:0/2497678515 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f580800eec0 con 0x7f58180ffaf0 2023-07-14T16:26:26.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.159+0000 7f581e9ff700 1 -- 172.21.15.62:0/2497678515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581819d8e0 con 0x7f58180ffaf0 2023-07-14T16:26:26.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.159+0000 7f58157fa700 1 -- 172.21.15.62:0/2497678515 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808003c10 con 0x7f58180ffaf0 2023-07-14T16:26:26.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.159+0000 7f581e9ff700 1 -- 172.21.15.62:0/2497678515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581819de40 con 0x7f58180ffaf0 2023-07-14T16:26:26.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.163+0000 7f58157fa700 1 -- 172.21.15.62:0/2497678515 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5808003db0 con 0x7f58180ffaf0 2023-07-14T16:26:26.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.164+0000 7f58157fa700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58000a11d0 0x7f58000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.164+0000 7f5832373b80 1 Processor -- start 2023-07-14T16:26:26.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f58157fa700 1 -- 172.21.15.62:0/2497678515 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5808014070 con 0x7f58180ffaf0 2023-07-14T16:26:26.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5832373b80 1 -- start start 2023-07-14T16:26:26.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f58177fe700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58000a11d0 0x7f58000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5832373b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665dc8e0 0x55ed665dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5832373b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5832373b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665de080 0x55ed665de4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5832373b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed664ddcb0 con 0x55ed665de080 2023-07-14T16:26:26.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5832373b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed66530a40 con 0x55ed665dd230 2023-07-14T16:26:26.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5832373b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed66529ed0 con 0x55ed665dc8e0 2023-07-14T16:26:26.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f58177fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5817fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665dc8e0 0x55ed665dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f581cf9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665de080 0x55ed665de4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f58177fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665ddaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43212/0 (socket says 172.21.15.62:43212) 2023-07-14T16:26:26.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f58177fe700 1 -- 172.21.15.62:0/2391073636 learned_addr learned my addr 172.21.15.62:0/2391073636 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:26.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.165+0000 7f5817fff700 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665dc8e0 0x55ed665dccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52510/0 (socket says 172.21.15.62:52510) 2023-07-14T16:26:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f5817fff700 1 -- 172.21.15.62:0/2391073636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 msgr2=0x55ed665ddaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f5817fff700 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665ddaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f5817fff700 1 -- 172.21.15.62:0/2391073636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665de080 msgr2=0x55ed665de4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f5817fff700 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665de080 0x55ed665de4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f5817fff700 1 -- 172.21.15.62:0/2391073636 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed66357de0 con 0x55ed665dc8e0 2023-07-14T16:26:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f58177fe700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58000a11d0 0x7f58000a3680 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f580c003f60 tx=0x7f580c0051f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f581cf9c700 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665de080 0x55ed665de4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f5817fff700 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665dc8e0 0x55ed665dccf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5808005950 tx=0x7f5808099a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f796874898c1048e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.166+0000 7f57fe7fc700 1 -- 172.21.15.62:0/2391073636 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808099f20 con 0x55ed665dc8e0 2023-07-14T16:26:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.167+0000 7f57fe7fc700 1 -- 172.21.15.62:0/2391073636 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f580809df50 con 0x55ed665dc8e0 2023-07-14T16:26:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.167+0000 7f57fe7fc700 1 -- 172.21.15.62:0/2391073636 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808099410 con 0x55ed665dc8e0 2023-07-14T16:26:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.167+0000 7f5832373b80 1 -- 172.21.15.62:0/2391073636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665dc8e0 msgr2=0x55ed665dccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.167+0000 7f5832373b80 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665dc8e0 0x55ed665dccf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5808005950 tx=0x7f5808099a60 comp rx=0 tx=0).stop 2023-07-14T16:26:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.168+0000 7f5832373b80 1 -- 172.21.15.62:0/2391073636 shutdown_connections 2023-07-14T16:26:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.168+0000 7f5832373b80 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665de080 0x55ed665de4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.168+0000 7f5832373b80 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665dc8e0 0x55ed665dccf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.168+0000 7f5832373b80 1 --2- 172.21.15.62:0/2391073636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665ddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.168+0000 7f5832373b80 1 -- 172.21.15.62:0/2391073636 >> 172.21.15.62:0/2391073636 conn(0x55ed665cdda0 msgr2=0x55ed665d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.168+0000 7f5832373b80 1 -- 172.21.15.62:0/2391073636 shutdown_connections 2023-07-14T16:26:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.168+0000 7f5832373b80 1 -- 172.21.15.62:0/2391073636 wait complete. 2023-07-14T16:26:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.168+0000 7f5832373b80 1 -- start start 2023-07-14T16:26:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.169+0000 7f5832373b80 1 Processor -- start 2023-07-14T16:26:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.169+0000 7f5832373b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dc8e0 0x55ed665e6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.169+0000 7f5832373b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.169+0000 7f5832373b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.169+0000 7f5832373b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed664c9ac0 con 0x55ed665dc8e0 2023-07-14T16:26:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5832373b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed664c97c0 con 0x55ed665dd230 2023-07-14T16:26:26.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5832373b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed663ad920 con 0x55ed665de080 2023-07-14T16:26:26.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5832373b80 10 client.0 _ll_register_callbacks cb 0x55ed664c7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:26.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5817fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dc8e0 0x55ed665e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5817fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dc8e0 0x55ed665e6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59390/0 (socket says 172.21.15.62:59390) 2023-07-14T16:26:26.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5817fff700 1 -- 172.21.15.62:0/3839163393 learned_addr learned my addr 172.21.15.62:0/3839163393 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:26.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f58177fe700 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f581cf9c700 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:25 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2182014558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:26:26.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:25 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:26:26.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5817fff700 1 -- 172.21.15.62:0/3839163393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 msgr2=0x55ed665edaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:26.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5817fff700 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665edaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5817fff700 1 -- 172.21.15.62:0/3839163393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 msgr2=0x55ed665e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:26.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5817fff700 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.170+0000 7f5817fff700 1 -- 172.21.15.62:0/3839163393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5810011230 con 0x55ed665dc8e0 2023-07-14T16:26:26.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.171+0000 7f5817fff700 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dc8e0 0x55ed665e6520 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5808096420 tx=0x7f580800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f626e91913e756 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:26.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.171+0000 7f57f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:26.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.171+0000 7f57f77fe700 1 -- 172.21.15.62:0/3839163393 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808099f20 con 0x55ed665dc8e0 2023-07-14T16:26:26.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.171+0000 7f57f77fe700 1 -- 172.21.15.62:0/3839163393 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5808078e40 con 0x55ed665dc8e0 2023-07-14T16:26:26.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.171+0000 7f5832373b80 1 -- 172.21.15.62:0/3839163393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed664834f0 con 0x55ed665dc8e0 2023-07-14T16:26:26.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.171+0000 7f57f77fe700 1 -- 172.21.15.62:0/3839163393 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58080a2370 con 0x55ed665dc8e0 2023-07-14T16:26:26.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.171+0000 7f5832373b80 10 client.18555 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:26.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.171+0000 7f5832373b80 1 -- 172.21.15.62:0/3839163393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f581819de40 con 0x55ed665dc8e0 2023-07-14T16:26:26.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f57f77fe700 1 -- 172.21.15.62:0/3839163393 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ed664834f0 con 0x55ed665dc8e0 2023-07-14T16:26:26.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f57f77fe700 1 -- 172.21.15.62:0/3839163393 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5808081070 con 0x55ed665dc8e0 2023-07-14T16:26:26.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f5832373b80 10 client.18555 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:26.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f5832373b80 10 client.18555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:26.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f5832373b80 20 client.18555 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:26.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f5832373b80 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ed665f1ed0 0x55ed665f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f5832373b80 4 client.18555 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:26.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f5832373b80 1 -- 172.21.15.62:0/3839163393 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed66522eb0 con 0x55ed665f1ed0 2023-07-14T16:26:26.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.172+0000 7f58177fe700 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ed665f1ed0 0x55ed665f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.173+0000 7f58177fe700 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ed665f1ed0 0x55ed665f42b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ea4dcfbd8dfe4f server_cookie=d6118697d8f41c02 in_seq=0 out_seq=0 2023-07-14T16:26:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.173+0000 7f57f77fe700 10 client.18555 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f57f77fe700 1 -- 172.21.15.62:0/3839163393 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ed66522eb0 con 0x55ed665f1ed0 2023-07-14T16:26:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f57f77fe700 10 client.18555 handle_command_reply: tid=0 2023-07-14T16:26:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 client.18555 shutdown 2023-07-14T16:26:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 -- 172.21.15.62:0/3839163393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dc8e0 msgr2=0x55ed665e6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dc8e0 0x55ed665e6520 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5808096420 tx=0x7f580800cf30 comp rx=0 tx=0).stop 2023-07-14T16:26:26.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 -- 172.21.15.62:0/3839163393 shutdown_connections 2023-07-14T16:26:26.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dc8e0 0x55ed665e6520 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ed665f1ed0 0x55ed665f42b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 --2- 172.21.15.62:0/3839163393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dd230 0x55ed665e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 -- 172.21.15.62:0/3839163393 >> 172.21.15.62:0/3839163393 conn(0x55ed665cdda0 msgr2=0x55ed665d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:26.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f57f77fe700 0 client.18555 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:26.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 -- 172.21.15.62:0/3839163393 shutdown_connections 2023-07-14T16:26:26.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.174+0000 7f5832373b80 1 -- 172.21.15.62:0/3839163393 wait complete. 2023-07-14T16:26:26.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.175+0000 7f5832373b80 20 client.18555 trim_cache size 0 max 16384 2023-07-14T16:26:26.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.190+0000 7f5832373b80 1 Processor -- start 2023-07-14T16:26:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.190+0000 7f5832373b80 1 -- start start 2023-07-14T16:26:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5832373b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5832373b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5832373b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5832373b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed665caff0 con 0x55ed665dd230 2023-07-14T16:26:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5832373b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed665e7710 con 0x55ed665dc8e0 2023-07-14T16:26:26.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5832373b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed665e59d0 con 0x55ed665de080 2023-07-14T16:26:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5817fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f58177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5817fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43222/0 (socket says 172.21.15.62:43222) 2023-07-14T16:26:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f5817fff700 1 -- 172.21.15.62:0/1083879420 learned_addr learned my addr 172.21.15.62:0/1083879420 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.191+0000 7f581cf9c700 1 --2- 172.21.15.62:0/1083879420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f5817fff700 1 -- 172.21.15.62:0/1083879420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 msgr2=0x55ed665eda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:26.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f5817fff700 1 --2- 172.21.15.62:0/1083879420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665eda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f5817fff700 1 -- 172.21.15.62:0/1083879420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 msgr2=0x55ed665e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:26.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f5817fff700 1 --2- 172.21.15.62:0/1083879420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f5817fff700 1 -- 172.21.15.62:0/1083879420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed66340c60 con 0x55ed665dc8e0 2023-07-14T16:26:26.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f5817fff700 1 --2- 172.21.15.62:0/1083879420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e43d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5808005950 tx=0x7f58080b8400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d0576394f4067 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:26.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f57feffd700 1 -- 172.21.15.62:0/1083879420 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080a2370 con 0x55ed665dc8e0 2023-07-14T16:26:26.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f57feffd700 1 -- 172.21.15.62:0/1083879420 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f580807e070 con 0x55ed665dc8e0 2023-07-14T16:26:26.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.192+0000 7f57feffd700 1 -- 172.21.15.62:0/1083879420 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080cad80 con 0x55ed665dc8e0 2023-07-14T16:26:26.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 -- 172.21.15.62:0/1083879420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 msgr2=0x55ed665e43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 --2- 172.21.15.62:0/1083879420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e43d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5808005950 tx=0x7f58080b8400 comp rx=0 tx=0).stop 2023-07-14T16:26:26.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 -- 172.21.15.62:0/1083879420 shutdown_connections 2023-07-14T16:26:26.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 --2- 172.21.15.62:0/1083879420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 --2- 172.21.15.62:0/1083879420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e43d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 --2- 172.21.15.62:0/1083879420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 -- 172.21.15.62:0/1083879420 >> 172.21.15.62:0/1083879420 conn(0x55ed665cdda0 msgr2=0x55ed665f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 -- 172.21.15.62:0/1083879420 shutdown_connections 2023-07-14T16:26:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.193+0000 7f5832373b80 1 -- 172.21.15.62:0/1083879420 wait complete. 2023-07-14T16:26:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.194+0000 7f5832373b80 1 -- start start 2023-07-14T16:26:26.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.194+0000 7f5832373b80 1 Processor -- start 2023-07-14T16:26:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f5832373b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f5832373b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665ea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f5832373b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665ed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f5832373b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed665ed650 con 0x55ed665dd230 2023-07-14T16:26:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f5832373b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed665ed7c0 con 0x55ed665dc8e0 2023-07-14T16:26:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f5832373b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed665ed980 con 0x55ed665de080 2023-07-14T16:26:26.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f5832373b80 10 client.0 _ll_register_callbacks cb 0x7f5810009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f58177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665ea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f58177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665ea130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59416/0 (socket says 172.21.15.62:59416) 2023-07-14T16:26:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f58177fe700 1 -- 172.21.15.62:0/3942852582 learned_addr learned my addr 172.21.15.62:0/3942852582 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f581cf9c700 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.195+0000 7f5817fff700 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f58177fe700 1 -- 172.21.15.62:0/3942852582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 msgr2=0x55ed665ed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f58177fe700 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665ed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f58177fe700 1 -- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 msgr2=0x55ed665e9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f58177fe700 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f58177fe700 1 -- 172.21.15.62:0/3942852582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581819de40 con 0x55ed665dd230 2023-07-14T16:26:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f5817fff700 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f58177fe700 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665ea130 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f580c009990 tx=0x7f580c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eebe97108dc373 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:26.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f57f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:26.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.196+0000 7f57f77fe700 1 -- 172.21.15.62:0/3942852582 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c032620 con 0x55ed665dd230 2023-07-14T16:26:26.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f5832373b80 1 -- 172.21.15.62:0/3942852582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f580809df50 con 0x55ed665dd230 2023-07-14T16:26:26.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f57f77fe700 1 -- 172.21.15.62:0/3942852582 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f580c003700 con 0x55ed665dd230 2023-07-14T16:26:26.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f57f77fe700 1 -- 172.21.15.62:0/3942852582 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f580c0038c0 con 0x55ed665dd230 2023-07-14T16:26:26.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f5832373b80 10 client.18561 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:26.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f5832373b80 1 -- 172.21.15.62:0/3942852582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ed6637e250 con 0x55ed665dd230 2023-07-14T16:26:26.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f57f77fe700 1 -- 172.21.15.62:0/3942852582 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f580809df50 con 0x55ed665dd230 2023-07-14T16:26:26.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f57f77fe700 1 -- 172.21.15.62:0/3942852582 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f580c044070 con 0x55ed665dd230 2023-07-14T16:26:26.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f5832373b80 10 client.18561 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:26.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f5832373b80 10 client.18561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:26.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f5832373b80 20 client.18561 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:26.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f5832373b80 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ed66647020 0x55ed66649430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:26.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.197+0000 7f5832373b80 4 client.18561 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:26:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.198+0000 7f5832373b80 1 -- 172.21.15.62:0/3942852582 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed664e5050 con 0x55ed66647020 2023-07-14T16:26:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.198+0000 7f5817fff700 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ed66647020 0x55ed66649430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5817fff700 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ed66647020 0x55ed66649430 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4727a48ddaab8ab server_cookie=f747ea25479eaf20 in_seq=0 out_seq=0 2023-07-14T16:26:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f57f77fe700 10 client.18561 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f57f77fe700 1 -- 172.21.15.62:0/3942852582 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed664e5050 con 0x55ed66647020 2023-07-14T16:26:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f57f77fe700 10 client.18561 handle_command_reply: tid=0 2023-07-14T16:26:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 client.18561 shutdown 2023-07-14T16:26:26.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 -- 172.21.15.62:0/3942852582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 msgr2=0x55ed665ea130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665ea130 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f580c009990 tx=0x7f580c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 -- 172.21.15.62:0/3942852582 shutdown_connections 2023-07-14T16:26:26.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed665de080 0x55ed665ed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ed66647020 0x55ed66649430 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed665dc8e0 0x55ed665e9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 --2- 172.21.15.62:0/3942852582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ed665dd230 0x55ed665ea130 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 -- 172.21.15.62:0/3942852582 >> 172.21.15.62:0/3942852582 conn(0x55ed665cdda0 msgr2=0x55ed665e43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:26.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f57f77fe700 0 client.18561 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:26.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.199+0000 7f5832373b80 1 -- 172.21.15.62:0/3942852582 shutdown_connections 2023-07-14T16:26:26.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f5832373b80 1 -- 172.21.15.62:0/3942852582 wait complete. 2023-07-14T16:26:26.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f5832373b80 20 client.18561 trim_cache size 0 max 16384 2023-07-14T16:26:26.182 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:26:26.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f581e9ff700 1 -- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58000a11d0 msgr2=0x7f58000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58000a11d0 0x7f58000a3680 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f580c003f60 tx=0x7f580c0051f0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f581e9ff700 1 -- 172.21.15.62:0/2497678515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58180ffaf0 msgr2=0x7f58181957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:26.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58180ffaf0 0x7f58181957b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f580800e970 tx=0x7f580800ec80 comp rx=0 tx=0).stop 2023-07-14T16:26:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f581e9ff700 1 -- 172.21.15.62:0/2497678515 shutdown_connections 2023-07-14T16:26:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5818101670 0x7f581819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f58000a11d0 0x7f58000a3680 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.200+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58180ffaf0 0x7f58181957b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.201+0000 7f581e9ff700 1 --2- 172.21.15.62:0/2497678515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5818100ce0 0x7f5818195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.201+0000 7f581e9ff700 1 -- 172.21.15.62:0/2497678515 >> 172.21.15.62:0/2497678515 conn(0x7f58180fb130 msgr2=0x7f5818104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.201+0000 7f581e9ff700 1 -- 172.21.15.62:0/2497678515 shutdown_connections 2023-07-14T16:26:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:26.201+0000 7f581e9ff700 1 -- 172.21.15.62:0/2497678515 wait complete. 2023-07-14T16:26:27.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:26 smithi164 ceph-mon[97427]: pgmap v681: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:26:27.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:27.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:26 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:27.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:26 smithi062 ceph-mon[95543]: pgmap v681: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:26:27.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:27.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:26 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:27.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:26 smithi081 ceph-mon[127795]: pgmap v681: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:26:27.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:27.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:26 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:27.199 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:26:27.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:26:27.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 -- 172.21.15.62:0/1781833680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3080105800 msgr2=0x7f3080107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 --2- 172.21.15.62:0/1781833680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3080105800 0x7f3080107c00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f307800b0a0 tx=0x7f307800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 -- 172.21.15.62:0/1781833680 shutdown_connections 2023-07-14T16:26:27.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 --2- 172.21.15.62:0/1781833680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3080105800 0x7f3080107c00 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 --2- 172.21.15.62:0/1781833680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30801005c0 0x7f30801029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 --2- 172.21.15.62:0/1781833680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3080102ee0 0x7f30801052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 -- 172.21.15.62:0/1781833680 >> 172.21.15.62:0/1781833680 conn(0x7f30800fa000 msgr2=0x7f30800fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 -- 172.21.15.62:0/1781833680 shutdown_connections 2023-07-14T16:26:27.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.475+0000 7f30887ff700 1 -- 172.21.15.62:0/1781833680 wait complete. 2023-07-14T16:26:27.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.477+0000 7f30887ff700 1 Processor -- start 2023-07-14T16:26:27.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.477+0000 7f30887ff700 1 -- start start 2023-07-14T16:26:27.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f30887ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30801005c0 0x7f30801958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f30887ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3080102ee0 0x7f3080195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f30887ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3080105800 0x7f308019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f30887ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308019d450 con 0x7f3080105800 2023-07-14T16:26:27.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f30887ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308019d590 con 0x7f3080102ee0 2023-07-14T16:26:27.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f30887ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308019d720 con 0x7f30801005c0 2023-07-14T16:26:27.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f308659b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30801005c0 0x7f30801958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f3085d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3080102ee0 0x7f3080195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f3086d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3080105800 0x7f308019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f308659b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30801005c0 0x7f30801958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52552/0 (socket says 172.21.15.62:52552) 2023-07-14T16:26:27.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.478+0000 7f308659b700 1 -- 172.21.15.62:0/3307730207 learned_addr learned my addr 172.21.15.62:0/3307730207 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.479+0000 7f3085d9a700 1 -- 172.21.15.62:0/3307730207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30801005c0 msgr2=0x7f30801958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.479+0000 7f3085d9a700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30801005c0 0x7f30801958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.479+0000 7f3085d9a700 1 -- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3080105800 msgr2=0x7f308019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.479+0000 7f3085d9a700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3080105800 0x7f308019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.479+0000 7f3085d9a700 1 -- 172.21.15.62:0/3307730207 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078009d00 con 0x7f3080102ee0 2023-07-14T16:26:27.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.479+0000 7f3085d9a700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3080102ee0 0x7f3080195e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f307000d950 tx=0x7f307000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.479+0000 7f30777fe700 1 -- 172.21.15.62:0/3307730207 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000f890 con 0x7f3080102ee0 2023-07-14T16:26:27.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.479+0000 7f30777fe700 1 -- 172.21.15.62:0/3307730207 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f307000fe80 con 0x7f3080102ee0 2023-07-14T16:26:27.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.480+0000 7f30777fe700 1 -- 172.21.15.62:0/3307730207 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000e3b0 con 0x7f3080102ee0 2023-07-14T16:26:27.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.480+0000 7f30887ff700 1 -- 172.21.15.62:0/3307730207 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308019da00 con 0x7f3080102ee0 2023-07-14T16:26:27.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.480+0000 7f30887ff700 1 -- 172.21.15.62:0/3307730207 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308019df10 con 0x7f3080102ee0 2023-07-14T16:26:27.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.485+0000 7f309c0fbb80 1 Processor -- start 2023-07-14T16:26:27.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.485+0000 7f30777fe700 1 -- 172.21.15.62:0/3307730207 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f307000f9f0 con 0x7f3080102ee0 2023-07-14T16:26:27.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.485+0000 7f309c0fbb80 1 -- start start 2023-07-14T16:26:27.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 0x563bf29d6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf28d5d30 con 0x563bf29d52b0 2023-07-14T16:26:27.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf2928ac0 con 0x563bf29d4960 2023-07-14T16:26:27.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf2921f50 con 0x563bf29d6100 2023-07-14T16:26:27.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f308659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f3085d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 0x563bf29d6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f308659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59450/0 (socket says 172.21.15.62:59450) 2023-07-14T16:26:27.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f308659b700 1 -- 172.21.15.62:0/1670270278 learned_addr learned my addr 172.21.15.62:0/1670270278 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.486+0000 7f3086d9c700 1 --2- 172.21.15.62:0/1670270278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3086d9c700 1 -- 172.21.15.62:0/1670270278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 msgr2=0x563bf29d6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3086d9c700 1 --2- 172.21.15.62:0/1670270278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 0x563bf29d6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3086d9c700 1 -- 172.21.15.62:0/1670270278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 msgr2=0x563bf29d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3086d9c700 1 --2- 172.21.15.62:0/1670270278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3086d9c700 1 -- 172.21.15.62:0/1670270278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bf274fe60 con 0x563bf29d4960 2023-07-14T16:26:27.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3086d9c700 1 --2- 172.21.15.62:0/1670270278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d4d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3078009fd0 tx=0x7f307800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58c350bf9afe501 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f30777fe700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f306c0a11d0 0x7f306c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f30777fe700 1 -- 172.21.15.62:0/3307730207 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f30700cc560 con 0x7f3080102ee0 2023-07-14T16:26:27.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3074ff9700 1 -- 172.21.15.62:0/1670270278 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307800e040 con 0x563bf29d4960 2023-07-14T16:26:27.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3074ff9700 1 -- 172.21.15.62:0/1670270278 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30780076e0 con 0x563bf29d4960 2023-07-14T16:26:27.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.487+0000 7f3074ff9700 1 -- 172.21.15.62:0/1670270278 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307801c6a0 con 0x563bf29d4960 2023-07-14T16:26:27.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.488+0000 7f308659b700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f306c0a11d0 0x7f306c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.488+0000 7f309c0fbb80 1 -- 172.21.15.62:0/1670270278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 msgr2=0x563bf29d4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.488+0000 7f308659b700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f306c0a11d0 0x7f306c0a3680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f307c00c010 tx=0x7f307c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.488+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/1670270278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d4d70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3078009fd0 tx=0x7f307800cde0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.489+0000 7f309c0fbb80 1 -- 172.21.15.62:0/1670270278 shutdown_connections 2023-07-14T16:26:27.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.489+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/1670270278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 0x563bf29d6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.489+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/1670270278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d4d70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.489+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/1670270278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.489+0000 7f309c0fbb80 1 -- 172.21.15.62:0/1670270278 >> 172.21.15.62:0/1670270278 conn(0x563bf29c5e20 msgr2=0x563bf29cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.489+0000 7f309c0fbb80 1 -- 172.21.15.62:0/1670270278 shutdown_connections 2023-07-14T16:26:27.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.489+0000 7f309c0fbb80 1 -- 172.21.15.62:0/1670270278 wait complete. 2023-07-14T16:26:27.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.489+0000 7f309c0fbb80 1 -- start start 2023-07-14T16:26:27.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.490+0000 7f309c0fbb80 1 Processor -- start 2023-07-14T16:26:27.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d4960 0x563bf29d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d52b0 0x563bf29d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf28c1b40 con 0x563bf29d6100 2023-07-14T16:26:27.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf28c1840 con 0x563bf29d52b0 2023-07-14T16:26:27.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf27a59a0 con 0x563bf29d4960 2023-07-14T16:26:27.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f309c0fbb80 10 client.0 _ll_register_callbacks cb 0x563bf28bf7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:27.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f308659b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d4960 0x563bf29d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f308659b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d4960 0x563bf29d0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52572/0 (socket says 172.21.15.62:52572) 2023-07-14T16:26:27.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f308659b700 1 -- 172.21.15.62:0/487259500 learned_addr learned my addr 172.21.15.62:0/487259500 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.492+0000 7f3085d9a700 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d52b0 0x563bf29d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f3086d9c700 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f308659b700 1 -- 172.21.15.62:0/487259500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d52b0 msgr2=0x563bf29d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f308659b700 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d52b0 0x563bf29d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f308659b700 1 -- 172.21.15.62:0/487259500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 msgr2=0x563bf29cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f308659b700 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f308659b700 1 -- 172.21.15.62:0/487259500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30801087f0 con 0x563bf29d4960 2023-07-14T16:26:27.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f308659b700 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d4960 0x563bf29d0b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f307c00bdf0 tx=0x7f307c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3441674892fd66e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f3069ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:26:27.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.493+0000 7f3069ffb700 1 -- 172.21.15.62:0/487259500 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c016070 con 0x563bf29d4960 2023-07-14T16:26:27.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.494+0000 7f3069ffb700 1 -- 172.21.15.62:0/487259500 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f307c027410 con 0x563bf29d4960 2023-07-14T16:26:27.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.494+0000 7f309c0fbb80 1 -- 172.21.15.62:0/487259500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bf287b570 con 0x563bf29d4960 2023-07-14T16:26:27.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.494+0000 7f3069ffb700 1 -- 172.21.15.62:0/487259500 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f307c01fe90 con 0x563bf29d4960 2023-07-14T16:26:27.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.494+0000 7f309c0fbb80 10 client.28069 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:27.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.494+0000 7f309c0fbb80 1 -- 172.21.15.62:0/487259500 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563bf2738ce0 con 0x563bf29d4960 2023-07-14T16:26:27.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.494+0000 7f3069ffb700 1 -- 172.21.15.62:0/487259500 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563bf287b570 con 0x563bf29d4960 2023-07-14T16:26:27.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.495+0000 7f3069ffb700 1 -- 172.21.15.62:0/487259500 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f307c01c560 con 0x563bf29d4960 2023-07-14T16:26:27.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.495+0000 7f309c0fbb80 10 client.28069 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:27.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.495+0000 7f309c0fbb80 10 client.28069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:27.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.495+0000 7f309c0fbb80 20 client.28069 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:27.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.495+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563bf29e1e40 0x563bf29e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.495+0000 7f309c0fbb80 4 client.28069 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:27.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.495+0000 7f309c0fbb80 1 -- 172.21.15.62:0/487259500 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bf291af30 con 0x563bf29e1e40 2023-07-14T16:26:27.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.495+0000 7f3085d9a700 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563bf29e1e40 0x563bf29e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.496+0000 7f3069ffb700 10 client.28069 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:27.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.496+0000 7f3085d9a700 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563bf29e1e40 0x563bf29e4250 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0deb61117b69b0 server_cookie=ace028cc2daeed7c in_seq=0 out_seq=0 2023-07-14T16:26:27.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.496+0000 7f3069ffb700 1 -- 172.21.15.62:0/487259500 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563bf291af30 con 0x563bf29e1e40 2023-07-14T16:26:27.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.496+0000 7f3069ffb700 10 client.28069 handle_command_reply: tid=0 2023-07-14T16:26:27.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.496+0000 7f309c0fbb80 1 client.28069 shutdown 2023-07-14T16:26:27.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 -- 172.21.15.62:0/487259500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d4960 msgr2=0x563bf29d0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d4960 0x563bf29d0b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f307c00bdf0 tx=0x7f307c0047c0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 -- 172.21.15.62:0/487259500 shutdown_connections 2023-07-14T16:26:27.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d4960 0x563bf29d0b90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563bf29e1e40 0x563bf29e4250 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/487259500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d52b0 0x563bf29d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 -- 172.21.15.62:0/487259500 >> 172.21.15.62:0/487259500 conn(0x563bf29c5e20 msgr2=0x563bf29ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f3069ffb700 0 client.28069 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:27.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 -- 172.21.15.62:0/487259500 shutdown_connections 2023-07-14T16:26:27.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 1 -- 172.21.15.62:0/487259500 wait complete. 2023-07-14T16:26:27.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.497+0000 7f309c0fbb80 20 client.28069 trim_cache size 0 max 16384 2023-07-14T16:26:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.526+0000 7f309c0fbb80 1 Processor -- start 2023-07-14T16:26:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.526+0000 7f309c0fbb80 1 -- start start 2023-07-14T16:26:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.526+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.526+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d52b0 0x563bf29d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.526+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29ddc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.526+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf29dda00 con 0x563bf29d6100 2023-07-14T16:26:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.526+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf29c59f0 con 0x563bf29d4960 2023-07-14T16:26:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.526+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf29dc1e0 con 0x563bf29d52b0 2023-07-14T16:26:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f308659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f308659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43262/0 (socket says 172.21.15.62:43262) 2023-07-14T16:26:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f308659b700 1 -- 172.21.15.62:0/397429718 learned_addr learned my addr 172.21.15.62:0/397429718 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f3086d9c700 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29ddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f3085d9a700 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d52b0 0x563bf29d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f308659b700 1 -- 172.21.15.62:0/397429718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d52b0 msgr2=0x563bf29d94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f308659b700 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d52b0 0x563bf29d94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f308659b700 1 -- 172.21.15.62:0/397429718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 msgr2=0x563bf29ddc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f308659b700 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29ddc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f308659b700 1 -- 172.21.15.62:0/397429718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bf29dd580 con 0x563bf29d4960 2023-07-14T16:26:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f3086d9c700 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29ddc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.527+0000 7f3085d9a700 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d52b0 0x563bf29d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:26:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.528+0000 7f308659b700 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d8f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f307c0058b0 tx=0x7f307c01fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f483de56295511 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.528+0000 7f30757fa700 1 -- 172.21.15.62:0/397429718 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c038070 con 0x563bf29d4960 2023-07-14T16:26:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.528+0000 7f30757fa700 1 -- 172.21.15.62:0/397429718 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f307c01fe90 con 0x563bf29d4960 2023-07-14T16:26:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.528+0000 7f30757fa700 1 -- 172.21.15.62:0/397429718 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c033b10 con 0x563bf29d4960 2023-07-14T16:26:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 -- 172.21.15.62:0/397429718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 msgr2=0x563bf29d8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d8f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f307c0058b0 tx=0x7f307c01fb50 comp rx=0 tx=0).stop 2023-07-14T16:26:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 -- 172.21.15.62:0/397429718 shutdown_connections 2023-07-14T16:26:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d6100 0x563bf29ddc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d8f60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/397429718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d52b0 0x563bf29d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 -- 172.21.15.62:0/397429718 >> 172.21.15.62:0/397429718 conn(0x563bf29c5e20 msgr2=0x563bf29dc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 -- 172.21.15.62:0/397429718 shutdown_connections 2023-07-14T16:26:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 -- 172.21.15.62:0/397429718 wait complete. 2023-07-14T16:26:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.529+0000 7f309c0fbb80 1 -- start start 2023-07-14T16:26:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.530+0000 7f309c0fbb80 1 Processor -- start 2023-07-14T16:26:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.530+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.530+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.530+0000 7f309c0fbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 0x563bf29d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.530+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf29d2d40 con 0x563bf29d52b0 2023-07-14T16:26:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.530+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf29d2eb0 con 0x563bf29d4960 2023-07-14T16:26:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.530+0000 7f309c0fbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bf29d3070 con 0x563bf29d6100 2023-07-14T16:26:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f309c0fbb80 10 client.0 _ll_register_callbacks cb 0x7f307c011e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f308659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f3085d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f308659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43276/0 (socket says 172.21.15.62:43276) 2023-07-14T16:26:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f308659b700 1 -- 172.21.15.62:0/3929967973 learned_addr learned my addr 172.21.15.62:0/3929967973 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f3086d9c700 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 0x563bf29d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f308659b700 1 -- 172.21.15.62:0/3929967973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 msgr2=0x563bf29d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f308659b700 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 0x563bf29d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f308659b700 1 -- 172.21.15.62:0/3929967973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 msgr2=0x563bf29d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f308659b700 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.531+0000 7f308659b700 1 -- 172.21.15.62:0/3929967973 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bf274fe60 con 0x563bf29d4960 2023-07-14T16:26:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.532+0000 7f308659b700 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d12b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f307c018040 tx=0x7f307c033660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eca72c95f43a22a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.532+0000 7f3069ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:26:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.532+0000 7f3069ffb700 1 -- 172.21.15.62:0/3929967973 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c038040 con 0x563bf29d4960 2023-07-14T16:26:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.532+0000 7f309c0fbb80 1 -- 172.21.15.62:0/3929967973 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30780076e0 con 0x563bf29d4960 2023-07-14T16:26:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.532+0000 7f3069ffb700 1 -- 172.21.15.62:0/3929967973 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f307c01ce40 con 0x563bf29d4960 2023-07-14T16:26:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.532+0000 7f3069ffb700 1 -- 172.21.15.62:0/3929967973 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f307c033a90 con 0x563bf29d4960 2023-07-14T16:26:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.532+0000 7f309c0fbb80 10 client.28181 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.532+0000 7f309c0fbb80 1 -- 172.21.15.62:0/3929967973 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563bf2738ce0 con 0x563bf29d4960 2023-07-14T16:26:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f3069ffb700 1 -- 172.21.15.62:0/3929967973 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30780076e0 con 0x563bf29d4960 2023-07-14T16:26:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f3069ffb700 1 -- 172.21.15.62:0/3929967973 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f307c004750 con 0x563bf29d4960 2023-07-14T16:26:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f309c0fbb80 10 client.28181 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f309c0fbb80 10 client.28181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f309c0fbb80 20 client.28181 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563bf2a4cb20 0x563bf2a4ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f309c0fbb80 4 client.28181 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:26:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f309c0fbb80 1 -- 172.21.15.62:0/3929967973 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563bf28dd0d0 con 0x563bf2a4cb20 2023-07-14T16:26:27.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.533+0000 7f3085d9a700 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563bf2a4cb20 0x563bf2a4ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.534+0000 7f3085d9a700 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563bf2a4cb20 0x563bf2a4ef00 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73261d68ca12f9b6 server_cookie=f2eba4ef4d88070b in_seq=0 out_seq=0 2023-07-14T16:26:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.534+0000 7f3069ffb700 10 client.28181 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f3069ffb700 1 -- 172.21.15.62:0/3929967973 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563bf28dd0d0 con 0x563bf2a4cb20 2023-07-14T16:26:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f3069ffb700 10 client.28181 handle_command_reply: tid=0 2023-07-14T16:26:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 client.28181 shutdown 2023-07-14T16:26:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 -- 172.21.15.62:0/3929967973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 msgr2=0x563bf29d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d12b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f307c018040 tx=0x7f307c033660 comp rx=0 tx=0).stop 2023-07-14T16:26:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 -- 172.21.15.62:0/3929967973 shutdown_connections 2023-07-14T16:26:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bf29d6100 0x563bf29d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563bf2a4cb20 0x563bf2a4ef00 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563bf29d4960 0x563bf29d12b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 --2- 172.21.15.62:0/3929967973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bf29d52b0 0x563bf29d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f3069ffb700 0 client.28181 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 -- 172.21.15.62:0/3929967973 >> 172.21.15.62:0/3929967973 conn(0x563bf29c5e20 msgr2=0x563bf29dc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.535+0000 7f309c0fbb80 1 -- 172.21.15.62:0/3929967973 shutdown_connections 2023-07-14T16:26:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f309c0fbb80 1 -- 172.21.15.62:0/3929967973 wait complete. 2023-07-14T16:26:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f309c0fbb80 20 client.28181 trim_cache size 0 max 16384 2023-07-14T16:26:27.518 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:26:27.518 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:26:27.518 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "ed5463de-103e-4d28-a596-e3792d83c001", 2023-07-14T16:26:27.518 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:26:27.518 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:26:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 -- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f306c0a11d0 msgr2=0x7f306c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f306c0a11d0 0x7f306c0a3680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f307c00c010 tx=0x7f307c00b500 comp rx=0 tx=0).stop 2023-07-14T16:26:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 -- 172.21.15.62:0/3307730207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3080102ee0 msgr2=0x7f3080195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3080102ee0 0x7f3080195e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f307000d950 tx=0x7f307000dc60 comp rx=0 tx=0).stop 2023-07-14T16:26:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 -- 172.21.15.62:0/3307730207 shutdown_connections 2023-07-14T16:26:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3080105800 0x7f308019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30801005c0 0x7f30801958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f306c0a11d0 0x7f306c0a3680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.536+0000 7f30887ff700 1 --2- 172.21.15.62:0/3307730207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3080102ee0 0x7f3080195e10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.537+0000 7f30887ff700 1 -- 172.21.15.62:0/3307730207 >> 172.21.15.62:0/3307730207 conn(0x7f30800fa000 msgr2=0x7f3080101240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.537+0000 7f30887ff700 1 -- 172.21.15.62:0/3307730207 shutdown_connections 2023-07-14T16:26:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.537+0000 7f30887ff700 1 -- 172.21.15.62:0/3307730207 wait complete. 2023-07-14T16:26:27.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:26:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.803+0000 7f03b9fff700 1 -- 172.21.15.62:0/3515481639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 msgr2=0x7f03b4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.803+0000 7f03b9fff700 1 --2- 172.21.15.62:0/3515481639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 0x7f03b4102940 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f039c009990 tx=0x7f039c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.803+0000 7f03b9fff700 1 -- 172.21.15.62:0/3515481639 shutdown_connections 2023-07-14T16:26:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.803+0000 7f03b9fff700 1 --2- 172.21.15.62:0/3515481639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b41057a0 0x7f03b4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.803+0000 7f03b9fff700 1 --2- 172.21.15.62:0/3515481639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 0x7f03b4102940 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.803+0000 7f03b9fff700 1 --2- 172.21.15.62:0/3515481639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03b4102e80 0x7f03b4105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.803+0000 7f03b9fff700 1 -- 172.21.15.62:0/3515481639 >> 172.21.15.62:0/3515481639 conn(0x7f03b40fa000 msgr2=0x7f03b40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.804+0000 7f03b9fff700 1 -- 172.21.15.62:0/3515481639 shutdown_connections 2023-07-14T16:26:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.804+0000 7f03b9fff700 1 -- 172.21.15.62:0/3515481639 wait complete. 2023-07-14T16:26:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.805+0000 7f03b9fff700 1 Processor -- start 2023-07-14T16:26:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.805+0000 7f03b9fff700 1 -- start start 2023-07-14T16:26:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.805+0000 7f03b9fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 0x7f03b41979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b9fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03b4102e80 0x7f03b4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b9fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b41057a0 0x7f03b419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b9fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b419f520 con 0x7f03b4100560 2023-07-14T16:26:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b9fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b419f660 con 0x7f03b41057a0 2023-07-14T16:26:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b9fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b419f7f0 con 0x7f03b4102e80 2023-07-14T16:26:27.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 0x7f03b41979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 0x7f03b41979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59508/0 (socket says 172.21.15.62:59508) 2023-07-14T16:26:27.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b37fe700 1 -- 172.21.15.62:0/2078455905 learned_addr learned my addr 172.21.15.62:0/2078455905 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03b4102e80 0x7f03b4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.806+0000 7f03b3fff700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b41057a0 0x7f03b419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b37fe700 1 -- 172.21.15.62:0/2078455905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03b4102e80 msgr2=0x7f03b4197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b37fe700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03b4102e80 0x7f03b4197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b37fe700 1 -- 172.21.15.62:0/2078455905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b41057a0 msgr2=0x7f03b419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b37fe700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b41057a0 0x7f03b419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b37fe700 1 -- 172.21.15.62:0/2078455905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0095f0 con 0x7f03b4100560 2023-07-14T16:26:27.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b3fff700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b41057a0 0x7f03b419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:27.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b37fe700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 0x7f03b41979a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f039c0098e0 tx=0x7f039c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b0ff9700 1 -- 172.21.15.62:0/2078455905 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c01d070 con 0x7f03b4100560 2023-07-14T16:26:27.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b0ff9700 1 -- 172.21.15.62:0/2078455905 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f039c00be30 con 0x7f03b4100560 2023-07-14T16:26:27.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.807+0000 7f03b9fff700 1 -- 172.21.15.62:0/2078455905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b419fa70 con 0x7f03b4100560 2023-07-14T16:26:27.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.808+0000 7f03b0ff9700 1 -- 172.21.15.62:0/2078455905 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c022e20 con 0x7f03b4100560 2023-07-14T16:26:27.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.808+0000 7f03b9fff700 1 -- 172.21.15.62:0/2078455905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b419fef0 con 0x7f03b4100560 2023-07-14T16:26:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.813+0000 7f03b0ff9700 1 -- 172.21.15.62:0/2078455905 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f039c022470 con 0x7f03b4100560 2023-07-14T16:26:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.814+0000 7f03cd9a3b80 1 Processor -- start 2023-07-14T16:26:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.814+0000 7f03cd9a3b80 1 -- start start 2023-07-14T16:26:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.814+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.814+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f48856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.814+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4885c70 0x55d2f48860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.814+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4789cb0 con 0x55d2f4885c70 2023-07-14T16:26:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.814+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f47dca40 con 0x55d2f48844d0 2023-07-14T16:26:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.814+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f47d5ed0 con 0x55d2f4884e20 2023-07-14T16:26:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48848e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43310/0 (socket says 172.21.15.62:43310) 2023-07-14T16:26:27.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f48856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b3fff700 1 -- 172.21.15.62:0/4166647797 learned_addr learned my addr 172.21.15.62:0/4166647797 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4885c70 0x55d2f48860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b3fff700 1 -- 172.21.15.62:0/4166647797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 msgr2=0x55d2f48856e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b3fff700 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f48856e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b3fff700 1 -- 172.21.15.62:0/4166647797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4885c70 msgr2=0x55d2f48860e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b3fff700 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4885c70 0x55d2f48860e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b3fff700 1 -- 172.21.15.62:0/4166647797 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2f4603de0 con 0x55d2f48844d0 2023-07-14T16:26:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4885c70 0x55d2f48860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:26:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.815+0000 7f03b0ff9700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03a00a11d0 0x7f03a00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.816+0000 7f03b0ff9700 1 -- 172.21.15.62:0/2078455905 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f039c0cde80 con 0x7f03b4100560 2023-07-14T16:26:27.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.816+0000 7f03b3fff700 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48848e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009440 tx=0x7f03ac009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e515ef8471ebc6ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.816+0000 7f03b37fe700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03a00a11d0 0x7f03a00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.816+0000 7f03a9ffb700 1 -- 172.21.15.62:0/4166647797 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00e070 con 0x55d2f48844d0 2023-07-14T16:26:27.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.816+0000 7f03a9ffb700 1 -- 172.21.15.62:0/4166647797 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03ac00f040 con 0x55d2f48844d0 2023-07-14T16:26:27.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.816+0000 7f03a9ffb700 1 -- 172.21.15.62:0/4166647797 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac015b90 con 0x55d2f48844d0 2023-07-14T16:26:27.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.817+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/4166647797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 msgr2=0x55d2f48848e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.817+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48848e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009440 tx=0x7f03ac009920 comp rx=0 tx=0).stop 2023-07-14T16:26:27.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.817+0000 7f03b37fe700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03a00a11d0 0x7f03a00a3680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f039c0a3160 tx=0x7f039c0d1880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.817+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/4166647797 shutdown_connections 2023-07-14T16:26:27.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.817+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4885c70 0x55d2f48860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.817+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48848e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.817+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/4166647797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f48856e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.817+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/4166647797 >> 172.21.15.62:0/4166647797 conn(0x55d2f4879da0 msgr2=0x55d2f487f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.818+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/4166647797 shutdown_connections 2023-07-14T16:26:27.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.818+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/4166647797 wait complete. 2023-07-14T16:26:27.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.818+0000 7f03cd9a3b80 1 -- start start 2023-07-14T16:26:27.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.819+0000 7f03cd9a3b80 1 Processor -- start 2023-07-14T16:26:27.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.819+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 0x55d2f48969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.819+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f4896f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.819+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 0x55d2f489df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.819+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4775ac0 con 0x55d2f48844d0 2023-07-14T16:26:27.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.819+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f47757c0 con 0x55d2f4885c70 2023-07-14T16:26:27.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.819+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4659920 con 0x55d2f4884e20 2023-07-14T16:26:27.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.819+0000 7f03cd9a3b80 10 client.0 _ll_register_callbacks cb 0x55d2f4773760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:27.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.820+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f4896f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.820+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f4896f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52612/0 (socket says 172.21.15.62:52612) 2023-07-14T16:26:27.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.820+0000 7f03b2ffd700 1 -- 172.21.15.62:0/1699369460 learned_addr learned my addr 172.21.15.62:0/1699369460 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.820+0000 7f03b2ffd700 1 -- 172.21.15.62:0/1699369460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 msgr2=0x55d2f489df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.820+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 0x55d2f489df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.820+0000 7f03b2ffd700 1 -- 172.21.15.62:0/1699369460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 msgr2=0x55d2f48969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.820+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 0x55d2f48969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.820+0000 7f03b2ffd700 1 -- 172.21.15.62:0/1699369460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac0090f0 con 0x55d2f4884e20 2023-07-14T16:26:27.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.821+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f4896f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03a4012840 tx=0x7f03a4012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8060b8ab9a77734d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.821+0000 7f0392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:26:27.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.821+0000 7f0392ffd700 1 -- 172.21.15.62:0/1699369460 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4010870 con 0x55d2f4884e20 2023-07-14T16:26:27.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.821+0000 7f0392ffd700 1 -- 172.21.15.62:0/1699369460 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03a40109d0 con 0x55d2f4884e20 2023-07-14T16:26:27.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.822+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/1699369460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2f472f4f0 con 0x55d2f4884e20 2023-07-14T16:26:27.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.822+0000 7f0392ffd700 1 -- 172.21.15.62:0/1699369460 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03a4010b90 con 0x55d2f4884e20 2023-07-14T16:26:27.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.822+0000 7f03cd9a3b80 10 client.28084 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:27.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.822+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/1699369460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d2f45ecc60 con 0x55d2f4884e20 2023-07-14T16:26:27.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.822+0000 7f0392ffd700 1 -- 172.21.15.62:0/1699369460 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d2f472f4f0 con 0x55d2f4884e20 2023-07-14T16:26:27.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.823+0000 7f0392ffd700 1 -- 172.21.15.62:0/1699369460 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f03a400e3b0 con 0x55d2f4884e20 2023-07-14T16:26:27.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.823+0000 7f03cd9a3b80 10 client.28084 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:27.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.823+0000 7f03cd9a3b80 10 client.28084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:27.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.823+0000 7f03cd9a3b80 20 client.28084 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:27.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.823+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2f48a27e0 0x55d2f48a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.823+0000 7f03cd9a3b80 4 client.28084 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:27.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.823+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/1699369460 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2f47ceeb0 con 0x55d2f48a27e0 2023-07-14T16:26:27.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.824+0000 7f03b3fff700 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2f48a27e0 0x55d2f48a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.824+0000 7f03b3fff700 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2f48a27e0 0x55d2f48a4bf0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9547f25477cfae98 server_cookie=ff43c04cf500d53b in_seq=0 out_seq=0 2023-07-14T16:26:27.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.824+0000 7f0392ffd700 10 client.28084 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:27.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.825+0000 7f0392ffd700 1 -- 172.21.15.62:0/1699369460 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d2f47ceeb0 con 0x55d2f48a27e0 2023-07-14T16:26:27.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.825+0000 7f0392ffd700 10 client.28084 handle_command_reply: tid=0 2023-07-14T16:26:27.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.825+0000 7f03cd9a3b80 1 client.28084 shutdown 2023-07-14T16:26:27.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.825+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/1699369460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 msgr2=0x55d2f4896f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.825+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f4896f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03a4012840 tx=0x7f03a4012c00 comp rx=0 tx=0).stop 2023-07-14T16:26:27.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/1699369460 shutdown_connections 2023-07-14T16:26:27.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2f48a27e0 0x55d2f48a4bf0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 0x55d2f489df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 0x55d2f48969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/1699369460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f4896f20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/1699369460 >> 172.21.15.62:0/1699369460 conn(0x55d2f4879da0 msgr2=0x55d2f487e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f0392ffd700 0 client.28084 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:27.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/1699369460 shutdown_connections 2023-07-14T16:26:27.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.826+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/1699369460 wait complete. 2023-07-14T16:26:27.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.827+0000 7f03cd9a3b80 20 client.28084 trim_cache size 0 max 16384 2023-07-14T16:26:27.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.846+0000 7f03cd9a3b80 1 Processor -- start 2023-07-14T16:26:27.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.846+0000 7f03cd9a3b80 1 -- start start 2023-07-14T16:26:27.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4884e20 0x55d2f489df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4885c70 0x55d2f489e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4880ca0 con 0x55d2f4884e20 2023-07-14T16:26:27.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f488cb80 con 0x55d2f48844d0 2023-07-14T16:26:27.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f48a4bc0 con 0x55d2f4885c70 2023-07-14T16:26:27.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4884e20 0x55d2f489df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4884e20 0x55d2f489df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59540/0 (socket says 172.21.15.62:59540) 2023-07-14T16:26:27.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03b2ffd700 1 -- 172.21.15.62:0/2281620590 learned_addr learned my addr 172.21.15.62:0/2281620590 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03b3fff700 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.847+0000 7f03b3fff700 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4885c70 0x55d2f489e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03b2ffd700 1 -- 172.21.15.62:0/2281620590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4885c70 msgr2=0x55d2f489e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4885c70 0x55d2f489e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03b2ffd700 1 -- 172.21.15.62:0/2281620590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 msgr2=0x55d2f48956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03b2ffd700 1 -- 172.21.15.62:0/2281620590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2f462a250 con 0x55d2f4884e20 2023-07-14T16:26:27.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03b3fff700 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4885c70 0x55d2f489e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:26:27.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4884e20 0x55d2f489df20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009340 tx=0x7f03a40104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0b570cf2d6a674 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03aa7fc700 1 -- 172.21.15.62:0/2281620590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a400e530 con 0x55d2f4884e20 2023-07-14T16:26:27.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03aa7fc700 1 -- 172.21.15.62:0/2281620590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03a40050b0 con 0x55d2f4884e20 2023-07-14T16:26:27.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.848+0000 7f03aa7fc700 1 -- 172.21.15.62:0/2281620590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40038c0 con 0x55d2f4884e20 2023-07-14T16:26:27.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/2281620590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4884e20 msgr2=0x55d2f489df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4884e20 0x55d2f489df20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009340 tx=0x7f03a40104d0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/2281620590 shutdown_connections 2023-07-14T16:26:27.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4885c70 0x55d2f489e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f48844d0 0x55d2f48956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/2281620590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f4884e20 0x55d2f489df20 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/2281620590 >> 172.21.15.62:0/2281620590 conn(0x55d2f4879da0 msgr2=0x55d2f48a9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/2281620590 shutdown_connections 2023-07-14T16:26:27.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.849+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/2281620590 wait complete. 2023-07-14T16:26:27.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.850+0000 7f03cd9a3b80 1 -- start start 2023-07-14T16:26:27.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.850+0000 7f03cd9a3b80 1 Processor -- start 2023-07-14T16:26:27.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 0x55d2f489dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f489a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03cd9a3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 0x55d2f489ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f489b260 con 0x55d2f48844d0 2023-07-14T16:26:27.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f489b3d0 con 0x55d2f4885c70 2023-07-14T16:26:27.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03cd9a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f48f6710 con 0x55d2f4884e20 2023-07-14T16:26:27.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03cd9a3b80 10 client.0 _ll_register_callbacks cb 0x7f03a4009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:27.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f489a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03b3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 0x55d2f489dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f489a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52642/0 (socket says 172.21.15.62:52642) 2023-07-14T16:26:27.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.851+0000 7f03b2ffd700 1 -- 172.21.15.62:0/3650060723 learned_addr learned my addr 172.21.15.62:0/3650060723 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:27.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.852+0000 7f03b3fff700 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 0x55d2f489ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.852+0000 7f03b2ffd700 1 -- 172.21.15.62:0/3650060723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 msgr2=0x55d2f489ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.852+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 0x55d2f489ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.852+0000 7f03b2ffd700 1 -- 172.21.15.62:0/3650060723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 msgr2=0x55d2f489dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:27.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.852+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 0x55d2f489dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.852+0000 7f03b2ffd700 1 -- 172.21.15.62:0/3650060723 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2f45ecc60 con 0x55d2f4884e20 2023-07-14T16:26:27.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.852+0000 7f03b2ffd700 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f489a750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03a4012560 tx=0x7f03a4005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a4448ae49051ad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:27.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.852+0000 7f0392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:26:27.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.853+0000 7f0392ffd700 1 -- 172.21.15.62:0/3650060723 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a400ecc0 con 0x55d2f4884e20 2023-07-14T16:26:27.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.853+0000 7f0392ffd700 1 -- 172.21.15.62:0/3650060723 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03a4000e40 con 0x55d2f4884e20 2023-07-14T16:26:27.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.853+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/3650060723 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03ac00f040 con 0x55d2f4884e20 2023-07-14T16:26:27.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.853+0000 7f0392ffd700 1 -- 172.21.15.62:0/3650060723 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03a4003680 con 0x55d2f4884e20 2023-07-14T16:26:27.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.853+0000 7f03cd9a3b80 10 client.28087 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:27.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.853+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/3650060723 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d2f46022b0 con 0x55d2f4884e20 2023-07-14T16:26:27.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f0392ffd700 1 -- 172.21.15.62:0/3650060723 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03ac00f040 con 0x55d2f4884e20 2023-07-14T16:26:27.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f0392ffd700 1 -- 172.21.15.62:0/3650060723 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f03a4010830 con 0x55d2f4884e20 2023-07-14T16:26:27.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f03cd9a3b80 10 client.28087 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:27.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f03cd9a3b80 10 client.28087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:27.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f03cd9a3b80 20 client.28087 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:27.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2f48f7cb0 0x55d2f48fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:27.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f03cd9a3b80 4 client.28087 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:26:27.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/3650060723 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d2f4791050 con 0x55d2f48f7cb0 2023-07-14T16:26:27.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.854+0000 7f03b3fff700 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2f48f7cb0 0x55d2f48fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:27.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.855+0000 7f0392ffd700 10 client.28087 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:27.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.855+0000 7f03b3fff700 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2f48f7cb0 0x55d2f48fa0c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6866d8b64ea179f1 server_cookie=d0ac4770fa5ee033 in_seq=0 out_seq=0 2023-07-14T16:26:27.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.855+0000 7f0392ffd700 1 -- 172.21.15.62:0/3650060723 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d2f4791050 con 0x55d2f48f7cb0 2023-07-14T16:26:27.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.855+0000 7f0392ffd700 10 client.28087 handle_command_reply: tid=0 2023-07-14T16:26:27.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.855+0000 7f03cd9a3b80 1 client.28087 shutdown 2023-07-14T16:26:27.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.855+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/3650060723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 msgr2=0x55d2f489a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.855+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f489a750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03a4012560 tx=0x7f03a4005450 comp rx=0 tx=0).stop 2023-07-14T16:26:27.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/3650060723 shutdown_connections 2023-07-14T16:26:27.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2f48f7cb0 0x55d2f48fa0c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2f4885c70 0x55d2f489ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2f48844d0 0x55d2f489dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 1 --2- 172.21.15.62:0/3650060723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2f4884e20 0x55d2f489a750 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f0392ffd700 0 client.28087 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/3650060723 >> 172.21.15.62:0/3650060723 conn(0x55d2f4879da0 msgr2=0x55d2f488c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/3650060723 shutdown_connections 2023-07-14T16:26:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 1 -- 172.21.15.62:0/3650060723 wait complete. 2023-07-14T16:26:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.856+0000 7f03cd9a3b80 20 client.28087 trim_cache size 0 max 16384 2023-07-14T16:26:27.838 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:26:27.838 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:26:27.838 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:26:27.838 INFO:teuthology.orchestra.run.smithi062.stdout: "ed5463de-103e-4d28-a596-e3792d83c001": { 2023-07-14T16:26:27.838 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:26:27.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "ed5463de-103e-4d28-a596-e3792d83c001", 2023-07-14T16:26:27.839 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:26:27.839 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:26:27.839 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:26:27.839 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:26:27.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 -- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03a00a11d0 msgr2=0x7f03a00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03a00a11d0 0x7f03a00a3680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f039c0a3160 tx=0x7f039c0d1880 comp rx=0 tx=0).stop 2023-07-14T16:26:27.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 -- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 msgr2=0x7f03b41979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:27.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 0x7f03b41979a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f039c0098e0 tx=0x7f039c004730 comp rx=0 tx=0).stop 2023-07-14T16:26:27.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 -- 172.21.15.62:0/2078455905 shutdown_connections 2023-07-14T16:26:27.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f03a00a11d0 0x7f03a00a3680 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03b41057a0 0x7f03b419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b4100560 0x7f03b41979a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 --2- 172.21.15.62:0/2078455905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03b4102e80 0x7f03b4197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:27.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 -- 172.21.15.62:0/2078455905 >> 172.21.15.62:0/2078455905 conn(0x7f03b40fa000 msgr2=0x7f03b41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:27.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 -- 172.21.15.62:0/2078455905 shutdown_connections 2023-07-14T16:26:27.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:27.857+0000 7f03b9fff700 1 -- 172.21.15.62:0/2078455905 wait complete. 2023-07-14T16:26:27.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:27 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:26:27.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:27 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:27.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:27 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:26:27.855 INFO:tasks.cephfs.filesystem:scrub status for tag:ed5463de-103e-4d28-a596-e3792d83c001 - {'path': '/', 'tag': 'ed5463de-103e-4d28-a596-e3792d83c001', 'options': 'recursive,force'} 2023-07-14T16:26:27.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:27 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:26:27.900 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:27 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:27.901 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:27 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:26:27.901 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:26:27 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:26:27.548+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v213>,<0x10000000003/client.0 v214>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v226>,<0x10000000001/sv_0 v240>,<0x10000000000/_nogroup v234>,<0x1/volumes v224>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:26:28.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:27 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:26:28.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:27 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:28.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:27 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:26:29.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:28 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:26:29.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:28 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:28 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:26:29.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:28 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:26:29.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:28 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:28 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:28 smithi164 ceph-mon[97427]: pgmap v682: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:29.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:28 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:26:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:28 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:28 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:26:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:28 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:26:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:28 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:28 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:28 smithi062 ceph-mon[95543]: pgmap v682: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:29.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:28 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:26:29.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:28 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:28 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:26:29.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:28 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:26:29.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:28 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:28 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:26:29.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:28 smithi081 ceph-mon[127795]: pgmap v682: 97 pgs: 97 active+clean; 7.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:30.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:30 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:30 smithi062 ceph-mon[95543]: pgmap v683: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:26:30.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:30 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:30 smithi081 ceph-mon[127795]: pgmap v683: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:26:30.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:30 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:30.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:30 smithi164 ceph-mon[97427]: pgmap v683: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:26:31.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:31.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:31.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:26:31.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:26:31.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:26:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:26:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:31.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:31.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:31.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:26:31.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:26:31.651 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:26:32.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:32 smithi164 ceph-mon[97427]: pgmap v684: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:26:32.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:32 smithi062 ceph-mon[95543]: pgmap v684: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:26:32.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:32 smithi081 ceph-mon[127795]: pgmap v684: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:26:33.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:33 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:26:33.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:33 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:26:33.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:33 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:26:34.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:34 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:26:34.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:34 smithi164 ceph-mon[97427]: pgmap v685: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:26:34.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:34 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:26:34.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:34 smithi062 ceph-mon[95543]: pgmap v685: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:26:34.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:34 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:26:34.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:34 smithi081 ceph-mon[127795]: pgmap v685: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:26:37.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:36 smithi164 ceph-mon[97427]: pgmap v686: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:26:37.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:36 smithi062 ceph-mon[95543]: pgmap v686: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:26:37.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:36 smithi081 ceph-mon[127795]: pgmap v686: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:26:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.858+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1434566710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 msgr2=0x7f7a2410e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.858+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1434566710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 0x7f7a2410e2f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b0a0 tx=0x7f7a1c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.858+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1434566710 shutdown_connections 2023-07-14T16:26:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.858+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1434566710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 0x7f7a2410e2f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.858+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1434566710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a24074550 0x7f7a240729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.858+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1434566710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a24072f70 0x7f7a240733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.858+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1434566710 >> 172.21.15.62:0/1434566710 conn(0x7f7a240faf80 msgr2=0x7f7a240fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.858+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1434566710 shutdown_connections 2023-07-14T16:26:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.859+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1434566710 wait complete. 2023-07-14T16:26:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.860+0000 7f7a29b9d700 1 Processor -- start 2023-07-14T16:26:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.860+0000 7f7a29b9d700 1 -- start start 2023-07-14T16:26:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.860+0000 7f7a29b9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a24072f70 0x7f7a2419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.860+0000 7f7a29b9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a24074550 0x7f7a2419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.860+0000 7f7a29b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 0x7f7a241a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.860+0000 7f7a29b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a241a3960 con 0x7f7a2410bf10 2023-07-14T16:26:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.860+0000 7f7a29b9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a241a3aa0 con 0x7f7a24072f70 2023-07-14T16:26:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.860+0000 7f7a29b9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a241a3c30 con 0x7f7a24074550 2023-07-14T16:26:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 0x7f7a241a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 0x7f7a241a3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52412/0 (socket says 172.21.15.62:52412) 2023-07-14T16:26:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 -- 172.21.15.62:0/1162528000 learned_addr learned my addr 172.21.15.62:0/1162528000 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a23fff700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a24074550 0x7f7a2419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a24072f70 0x7f7a2419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 -- 172.21.15.62:0/1162528000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a24074550 msgr2=0x7f7a2419c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a24074550 0x7f7a2419c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 -- 172.21.15.62:0/1162528000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a24072f70 msgr2=0x7f7a2419bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a24072f70 0x7f7a2419bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 -- 172.21.15.62:0/1162528000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c009d00 con 0x7f7a2410bf10 2023-07-14T16:26:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.861+0000 7f7a2939c700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 0x7f7a241a3390 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00d010 tx=0x7f7a1c00bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.862+0000 7f7a21ffb700 1 -- 172.21.15.62:0/1162528000 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00e040 con 0x7f7a2410bf10 2023-07-14T16:26:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.862+0000 7f7a21ffb700 1 -- 172.21.15.62:0/1162528000 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a1c009480 con 0x7f7a2410bf10 2023-07-14T16:26:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.862+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1162528000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a241a3eb0 con 0x7f7a2410bf10 2023-07-14T16:26:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.862+0000 7f7a21ffb700 1 -- 172.21.15.62:0/1162528000 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0052a0 con 0x7f7a2410bf10 2023-07-14T16:26:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.862+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1162528000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a241a4270 con 0x7f7a2410bf10 2023-07-14T16:26:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.866+0000 7f7a21ffb700 1 -- 172.21.15.62:0/1162528000 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7a1c005400 con 0x7f7a2410bf10 2023-07-14T16:26:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.867+0000 7f7a3e7a0b80 1 Processor -- start 2023-07-14T16:26:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a3e7a0b80 1 -- start start 2023-07-14T16:26:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a21ffb700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a140a11d0 0x7f7a140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623737176b0 0x5623738bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623738bc7e0 0x5623738bcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a21ffb700 1 -- 172.21.15.62:0/1162528000 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a1c0cdb80 con 0x7f7a2410bf10 2023-07-14T16:26:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623737c0c50 con 0x5623738bba90 2023-07-14T16:26:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623738139e0 con 0x5623737176b0 2023-07-14T16:26:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56237380ce70 con 0x5623738bc7e0 2023-07-14T16:26:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a23fff700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a140a11d0 0x7f7a140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738bc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52418/0 (socket says 172.21.15.62:52418) 2023-07-14T16:26:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a23fff700 1 -- 172.21.15.62:0/4149243424 learned_addr learned my addr 172.21.15.62:0/4149243424 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.868+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623737176b0 0x5623738bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.869+0000 7f7a2939c700 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623738bc7e0 0x5623738bcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.869+0000 7f7a28b9b700 1 -- 172.21.15.62:0/4149243424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623738bc7e0 msgr2=0x5623738bcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.869+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623738bc7e0 0x5623738bcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.869+0000 7f7a28b9b700 1 -- 172.21.15.62:0/4149243424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 msgr2=0x5623738bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.869+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738bc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.869+0000 7f7a28b9b700 1 -- 172.21.15.62:0/4149243424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56237363ad80 con 0x5623737176b0 2023-07-14T16:26:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.869+0000 7f7a23fff700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a140a11d0 0x7f7a140a3680 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7a10011a70 tx=0x7f7a10009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.869+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623737176b0 0x5623738bb550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009c80 tx=0x7f7a18009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea48694c46d4166b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a0effd700 1 -- 172.21.15.62:0/4149243424 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18011cb0 con 0x5623737176b0 2023-07-14T16:26:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a0effd700 1 -- 172.21.15.62:0/4149243424 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a18011e10 con 0x5623737176b0 2023-07-14T16:26:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a0effd700 1 -- 172.21.15.62:0/4149243424 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18005020 con 0x5623737176b0 2023-07-14T16:26:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/4149243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623737176b0 msgr2=0x5623738bb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623737176b0 0x5623738bb550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009c80 tx=0x7f7a18009f90 comp rx=0 tx=0).stop 2023-07-14T16:26:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/4149243424 shutdown_connections 2023-07-14T16:26:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623738bc7e0 0x5623738bcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623737176b0 0x5623738bb550 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/4149243424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738bc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.870+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/4149243424 >> 172.21.15.62:0/4149243424 conn(0x5623738b0d40 msgr2=0x5623738b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.871+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/4149243424 shutdown_connections 2023-07-14T16:26:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.871+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/4149243424 wait complete. 2023-07-14T16:26:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.871+0000 7f7a3e7a0b80 1 -- start start 2023-07-14T16:26:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.871+0000 7f7a3e7a0b80 1 Processor -- start 2023-07-14T16:26:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623737aca60 con 0x5623738bc7e0 2023-07-14T16:26:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623737ac760 con 0x5623738bba90 2023-07-14T16:26:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623736908c0 con 0x5623737176b0 2023-07-14T16:26:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a3e7a0b80 10 client.0 _ll_register_callbacks cb 0x5623737aa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a2939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a2939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52420/0 (socket says 172.21.15.62:52420) 2023-07-14T16:26:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a2939c700 1 -- 172.21.15.62:0/2651544392 learned_addr learned my addr 172.21.15.62:0/2651544392 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.872+0000 7f7a23fff700 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a2939c700 1 -- 172.21.15.62:0/2651544392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 msgr2=0x5623738c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a2939c700 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a2939c700 1 -- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 msgr2=0x5623738c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a2939c700 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a2939c700 1 -- 172.21.15.62:0/2651544392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a180098e0 con 0x5623738bc7e0 2023-07-14T16:26:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a23fff700 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a2939c700 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738cc030 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009fd0 tx=0x7f7a1c09a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8907ac2c8991678 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a03fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a03fff700 1 -- 172.21.15.62:0/2651544392 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c09b430 con 0x5623738bc7e0 2023-07-14T16:26:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a03fff700 1 -- 172.21.15.62:0/2651544392 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a1c09f000 con 0x5623738bc7e0 2023-07-14T16:26:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2651544392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562373766490 con 0x5623738bc7e0 2023-07-14T16:26:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a03fff700 1 -- 172.21.15.62:0/2651544392 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a1c0b9030 con 0x5623738bc7e0 2023-07-14T16:26:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a3e7a0b80 10 client.18603 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.873+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2651544392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562373623c00 con 0x5623738bc7e0 2023-07-14T16:26:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.874+0000 7f7a03fff700 1 -- 172.21.15.62:0/2651544392 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a1c0c9020 con 0x5623738bc7e0 2023-07-14T16:26:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.874+0000 7f7a03fff700 1 -- 172.21.15.62:0/2651544392 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a1c09a4c0 con 0x5623738bc7e0 2023-07-14T16:26:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.874+0000 7f7a3e7a0b80 10 client.18603 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.874+0000 7f7a3e7a0b80 10 client.18603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.874+0000 7f7a3e7a0b80 20 client.18603 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.874+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5623738d0650 0x5623738d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.874+0000 7f7a3e7a0b80 4 client.18603 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.874+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2651544392 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562373805e50 con 0x5623738d0650 2023-07-14T16:26:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.875+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5623738d0650 0x5623738d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.875+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5623738d0650 0x5623738d2a60 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57fbaf1d398a751 server_cookie=d51280b08f36ef91 in_seq=0 out_seq=0 2023-07-14T16:26:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.875+0000 7f7a03fff700 10 client.18603 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.876+0000 7f7a03fff700 1 -- 172.21.15.62:0/2651544392 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562373805e50 con 0x5623738d0650 2023-07-14T16:26:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.876+0000 7f7a03fff700 10 client.18603 handle_command_reply: tid=0 2023-07-14T16:26:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.876+0000 7f7a3e7a0b80 1 client.18603 shutdown 2023-07-14T16:26:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.876+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2651544392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 msgr2=0x5623738cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.876+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738cc030 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009fd0 tx=0x7f7a1c09a930 comp rx=0 tx=0).stop 2023-07-14T16:26:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2651544392 shutdown_connections 2023-07-14T16:26:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738cc030 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5623738d0650 0x5623738d2a60 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2651544392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2651544392 >> 172.21.15.62:0/2651544392 conn(0x5623738b0d40 msgr2=0x5623738b5640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a03fff700 0 client.18603 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2651544392 shutdown_connections 2023-07-14T16:26:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2651544392 wait complete. 2023-07-14T16:26:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.877+0000 7f7a3e7a0b80 20 client.18603 trim_cache size 0 max 16384 2023-07-14T16:26:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a3e7a0b80 1 Processor -- start 2023-07-14T16:26:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a3e7a0b80 1 -- start start 2023-07-14T16:26:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623738c5ca0 con 0x5623738bc7e0 2023-07-14T16:26:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623738b6f10 con 0x5623738bba90 2023-07-14T16:26:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623738c3f60 con 0x5623737176b0 2023-07-14T16:26:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a2939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a2939c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52430/0 (socket says 172.21.15.62:52430) 2023-07-14T16:26:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a2939c700 1 -- 172.21.15.62:0/258520188 learned_addr learned my addr 172.21.15.62:0/258520188 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.896+0000 7f7a23fff700 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a2939c700 1 -- 172.21.15.62:0/258520188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 msgr2=0x5623738bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a2939c700 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a2939c700 1 -- 172.21.15.62:0/258520188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 msgr2=0x5623738cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a2939c700 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a2939c700 1 -- 172.21.15.62:0/258520188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623736611f0 con 0x5623738bc7e0 2023-07-14T16:26:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a2939c700 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738bfb70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0979f0 tx=0x7f7a1c099580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8e941e8b03716e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a0f7fe700 1 -- 172.21.15.62:0/258520188 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0b9040 con 0x5623738bc7e0 2023-07-14T16:26:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a0f7fe700 1 -- 172.21.15.62:0/258520188 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a1c0c9030 con 0x5623738bc7e0 2023-07-14T16:26:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.897+0000 7f7a0f7fe700 1 -- 172.21.15.62:0/258520188 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0cb2b0 con 0x5623738bc7e0 2023-07-14T16:26:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/258520188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 msgr2=0x5623738bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738bfb70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0979f0 tx=0x7f7a1c099580 comp rx=0 tx=0).stop 2023-07-14T16:26:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/258520188 shutdown_connections 2023-07-14T16:26:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bc7e0 0x5623738bfb70 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/258520188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bba90 0x5623738cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/258520188 >> 172.21.15.62:0/258520188 conn(0x5623738b0d40 msgr2=0x5623738c37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/258520188 shutdown_connections 2023-07-14T16:26:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.898+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/258520188 wait complete. 2023-07-14T16:26:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.899+0000 7f7a3e7a0b80 1 -- start start 2023-07-14T16:26:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.899+0000 7f7a3e7a0b80 1 Processor -- start 2023-07-14T16:26:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a3e7a0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bc7e0 0x5623738cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623738cb2f0 con 0x5623738bba90 2023-07-14T16:26:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623738cb460 con 0x5623738bc7e0 2023-07-14T16:26:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a3e7a0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623738cb620 con 0x5623737176b0 2023-07-14T16:26:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a3e7a0b80 10 client.0 _ll_register_callbacks cb 0x5623737aa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a23fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52440/0 (socket says 172.21.15.62:52440) 2023-07-14T16:26:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a23fff700 1 -- 172.21.15.62:0/2463620436 learned_addr learned my addr 172.21.15.62:0/2463620436 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a2939c700 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bc7e0 0x5623738cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a23fff700 1 -- 172.21.15.62:0/2463620436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 msgr2=0x5623738c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a23fff700 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a23fff700 1 -- 172.21.15.62:0/2463620436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bc7e0 msgr2=0x5623738cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a23fff700 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bc7e0 0x5623738cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.900+0000 7f7a23fff700 1 -- 172.21.15.62:0/2463620436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562373623c00 con 0x5623738bba90 2023-07-14T16:26:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.901+0000 7f7a23fff700 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738c7d90 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7a10004540 tx=0x7f7a10009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f3f57ac7239d18 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.901+0000 7f7a03fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.901+0000 7f7a03fff700 1 -- 172.21.15.62:0/2463620436 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10011440 con 0x5623738bba90 2023-07-14T16:26:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.901+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2463620436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a18011e10 con 0x5623738bba90 2023-07-14T16:26:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.901+0000 7f7a03fff700 1 -- 172.21.15.62:0/2463620436 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a10003890 con 0x5623738bba90 2023-07-14T16:26:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.901+0000 7f7a03fff700 1 -- 172.21.15.62:0/2463620436 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a10004570 con 0x5623738bba90 2023-07-14T16:26:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.901+0000 7f7a3e7a0b80 10 client.18609 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.901+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2463620436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562373639250 con 0x5623738bba90 2023-07-14T16:26:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.902+0000 7f7a03fff700 1 -- 172.21.15.62:0/2463620436 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a18011e10 con 0x5623738bba90 2023-07-14T16:26:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.902+0000 7f7a03fff700 1 -- 172.21.15.62:0/2463620436 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a1002ab00 con 0x5623738bba90 2023-07-14T16:26:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.902+0000 7f7a3e7a0b80 10 client.18609 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.902+0000 7f7a3e7a0b80 10 client.18609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.902+0000 7f7a3e7a0b80 20 client.18609 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.902+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562373925030 0x562373927440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.902+0000 7f7a3e7a0b80 4 client.18609 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:26:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.902+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2463620436 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623737c7ff0 con 0x562373925030 2023-07-14T16:26:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.903+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562373925030 0x562373927440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.903+0000 7f7a03fff700 10 client.18609 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.903+0000 7f7a28b9b700 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562373925030 0x562373927440 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879801f17af17af0 server_cookie=7de72c8d30a112cb in_seq=0 out_seq=0 2023-07-14T16:26:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a03fff700 1 -- 172.21.15.62:0/2463620436 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623737c7ff0 con 0x562373925030 2023-07-14T16:26:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a03fff700 10 client.18609 handle_command_reply: tid=0 2023-07-14T16:26:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 client.18609 shutdown 2023-07-14T16:26:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2463620436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 msgr2=0x5623738c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738c7d90 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7a10004540 tx=0x7f7a10009260 comp rx=0 tx=0).stop 2023-07-14T16:26:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2463620436 shutdown_connections 2023-07-14T16:26:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562373925030 0x562373927440 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623738bc7e0 0x5623738cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5623737176b0 0x5623738c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 --2- 172.21.15.62:0/2463620436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5623738bba90 0x5623738c7d90 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2463620436 >> 172.21.15.62:0/2463620436 conn(0x5623738b0d40 msgr2=0x5623738bf630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a03fff700 0 client.18609 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2463620436 shutdown_connections 2023-07-14T16:26:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.913+0000 7f7a3e7a0b80 1 -- 172.21.15.62:0/2463620436 wait complete. 2023-07-14T16:26:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a3e7a0b80 20 client.18609 trim_cache size 0 max 16384 2023-07-14T16:26:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:26:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:26:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:26:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:26:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a140a11d0 msgr2=0x7f7a140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a140a11d0 0x7f7a140a3680 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7a10011a70 tx=0x7f7a10009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 msgr2=0x7f7a241a3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 0x7f7a241a3390 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00d010 tx=0x7f7a1c00bcc0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1162528000 shutdown_connections 2023-07-14T16:26:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2410bf10 0x7f7a241a3390 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7a140a11d0 0x7f7a140a3680 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7a24072f70 0x7f7a2419bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 --2- 172.21.15.62:0/1162528000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a24074550 0x7f7a2419c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.914+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1162528000 >> 172.21.15.62:0/1162528000 conn(0x7f7a240faf80 msgr2=0x7f7a2410dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.915+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1162528000 shutdown_connections 2023-07-14T16:26:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:37.915+0000 7f7a29b9d700 1 -- 172.21.15.62:0/1162528000 wait complete. 2023-07-14T16:26:39.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:38 smithi164 ceph-mon[97427]: pgmap v687: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-14T16:26:39.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:38 smithi062 ceph-mon[95543]: pgmap v687: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-14T16:26:39.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:38 smithi081 ceph-mon[127795]: pgmap v687: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-14T16:26:41.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:40 smithi164 ceph-mon[97427]: pgmap v688: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 40 op/s 2023-07-14T16:26:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:40 smithi062 ceph-mon[95543]: pgmap v688: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 40 op/s 2023-07-14T16:26:41.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:40 smithi081 ceph-mon[127795]: pgmap v688: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 40 op/s 2023-07-14T16:26:43.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:42 smithi164 ceph-mon[97427]: pgmap v689: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 24 op/s 2023-07-14T16:26:43.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:42 smithi062 ceph-mon[95543]: pgmap v689: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 24 op/s 2023-07-14T16:26:43.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:42 smithi081 ceph-mon[127795]: pgmap v689: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 24 op/s 2023-07-14T16:26:45.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:44 smithi164 ceph-mon[97427]: pgmap v690: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:26:45.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:44 smithi062 ceph-mon[95543]: pgmap v690: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:26:45.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:44 smithi081 ceph-mon[127795]: pgmap v690: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:26:47.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:46 smithi164 ceph-mon[97427]: pgmap v691: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:26:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:46 smithi062 ceph-mon[95543]: pgmap v691: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:26:47.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:46 smithi081 ceph-mon[127795]: pgmap v691: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:26:49.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:48 smithi164 ceph-mon[97427]: pgmap v692: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:26:49.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:48 smithi062 ceph-mon[95543]: pgmap v692: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:26:49.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:48 smithi081 ceph-mon[127795]: pgmap v692: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:26:51.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:50 smithi164 ceph-mon[97427]: pgmap v693: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 -- 172.21.15.62:0/2350939016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8102d50 msgr2=0x7efcc81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2350939016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8102d50 0x7efcc81031a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009990 tx=0x7efcb4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 -- 172.21.15.62:0/2350939016 shutdown_connections 2023-07-14T16:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2350939016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc81036e0 0x7efcc8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2350939016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcc8101b60 0x7efcc8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2350939016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8102d50 0x7efcc81031a0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 -- 172.21.15.62:0/2350939016 >> 172.21.15.62:0/2350939016 conn(0x7efcc80fd180 msgr2=0x7efcc80ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 -- 172.21.15.62:0/2350939016 shutdown_connections 2023-07-14T16:26:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.085+0000 7efccdd9d700 1 -- 172.21.15.62:0/2350939016 wait complete. 2023-07-14T16:26:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.086+0000 7efccdd9d700 1 Processor -- start 2023-07-14T16:26:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.086+0000 7efccdd9d700 1 -- start start 2023-07-14T16:26:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efccdd9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8101b60 0x7efcc81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efccdd9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc8102d50 0x7efcc8197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efccdd9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcc81036e0 0x7efcc819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efccdd9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819f350 con 0x7efcc8101b60 2023-07-14T16:26:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efccdd9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819f490 con 0x7efcc81036e0 2023-07-14T16:26:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efccdd9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819f620 con 0x7efcc8102d50 2023-07-14T16:26:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcccd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8101b60 0x7efcc81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc8102d50 0x7efcc8197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcccd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8101b60 0x7efcc81977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47488/0 (socket says 172.21.15.62:47488) 2023-07-14T16:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcccd9b700 1 -- 172.21.15.62:0/2960818193 learned_addr learned my addr 172.21.15.62:0/2960818193 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efccd59c700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcc81036e0 0x7efcc819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcccd9b700 1 -- 172.21.15.62:0/2960818193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc8102d50 msgr2=0x7efcc8197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcccd9b700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc8102d50 0x7efcc8197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcccd9b700 1 -- 172.21.15.62:0/2960818193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcc81036e0 msgr2=0x7efcc819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcccd9b700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcc81036e0 0x7efcc819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.087+0000 7efcccd9b700 1 -- 172.21.15.62:0/2960818193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40095f0 con 0x7efcc8101b60 2023-07-14T16:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.088+0000 7efcccd9b700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8101b60 0x7efcc81977d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00e970 tx=0x7efcbc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.088+0000 7efcc5ffb700 1 -- 172.21.15.62:0/2960818193 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc009790 con 0x7efcc8101b60 2023-07-14T16:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.088+0000 7efcc5ffb700 1 -- 172.21.15.62:0/2960818193 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcbc00c7e0 con 0x7efcc8101b60 2023-07-14T16:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.088+0000 7efccdd9d700 1 -- 172.21.15.62:0/2960818193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc819f900 con 0x7efcc8101b60 2023-07-14T16:26:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.088+0000 7efcc5ffb700 1 -- 172.21.15.62:0/2960818193 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc003bd0 con 0x7efcc8101b60 2023-07-14T16:26:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.088+0000 7efccdd9d700 1 -- 172.21.15.62:0/2960818193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc819fe60 con 0x7efcc8101b60 2023-07-14T16:26:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.092+0000 7efcc5ffb700 1 -- 172.21.15.62:0/2960818193 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efcbc00c950 con 0x7efcc8101b60 2023-07-14T16:26:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.093+0000 7efce299eb80 1 Processor -- start 2023-07-14T16:26:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efce299eb80 1 -- start start 2023-07-14T16:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efce299eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160b346b0 0x560160cd8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efcc5ffb700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efcb80a11d0 0x7efcb80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efce299eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160cd92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efce299eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cd9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efce299eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160bddc50 con 0x560160cd8a90 2023-07-14T16:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efce299eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160c309e0 con 0x560160cd97e0 2023-07-14T16:26:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efce299eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160c29e70 con 0x560160b346b0 2023-07-14T16:26:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efcc5ffb700 1 -- 172.21.15.62:0/2960818193 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efcbc014070 con 0x7efcc8101b60 2023-07-14T16:26:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160cd92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efcccd9b700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efcb80a11d0 0x7efcb80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160cd92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47494/0 (socket says 172.21.15.62:47494) 2023-07-14T16:26:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.094+0000 7efcc7fff700 1 -- 172.21.15.62:0/821047824 learned_addr learned my addr 172.21.15.62:0/821047824 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efccd59c700 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160b346b0 0x560160cd8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efccd59c700 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cd9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efcc7fff700 1 -- 172.21.15.62:0/821047824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160b346b0 msgr2=0x560160cd8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efcc7fff700 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160b346b0 0x560160cd8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efcc7fff700 1 -- 172.21.15.62:0/821047824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 msgr2=0x560160cd9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efcc7fff700 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cd9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efcc7fff700 1 -- 172.21.15.62:0/821047824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560160a57d80 con 0x560160cd8a90 2023-07-14T16:26:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efccd59c700 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cd9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efcccd9b700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efcb80a11d0 0x7efcb80a3680 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efcbc099b30 tx=0x7efcbc00cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efcc7fff700 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160cd92a0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009990 tx=0x7efcb4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc383791ccb2e686 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.095+0000 7efcb2ffd700 1 -- 172.21.15.62:0/821047824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb401d070 con 0x560160cd8a90 2023-07-14T16:26:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.096+0000 7efcb2ffd700 1 -- 172.21.15.62:0/821047824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcb401e040 con 0x560160cd8a90 2023-07-14T16:26:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.096+0000 7efcb2ffd700 1 -- 172.21.15.62:0/821047824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400b6b0 con 0x560160cd8a90 2023-07-14T16:26:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.096+0000 7efce299eb80 1 -- 172.21.15.62:0/821047824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 msgr2=0x560160cd92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.096+0000 7efce299eb80 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160cd92a0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009990 tx=0x7efcb4005f00 comp rx=0 tx=0).stop 2023-07-14T16:26:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 -- 172.21.15.62:0/821047824 shutdown_connections 2023-07-14T16:26:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cd9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160b346b0 0x560160cd8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 --2- 172.21.15.62:0/821047824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160cd92a0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 -- 172.21.15.62:0/821047824 >> 172.21.15.62:0/821047824 conn(0x560160ccdd40 msgr2=0x560160cd2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 -- 172.21.15.62:0/821047824 shutdown_connections 2023-07-14T16:26:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 -- 172.21.15.62:0/821047824 wait complete. 2023-07-14T16:26:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 -- start start 2023-07-14T16:26:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.097+0000 7efce299eb80 1 Processor -- start 2023-07-14T16:26:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efce299eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160ce19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efce299eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 0x560160ce1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efce299eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160ce8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efce299eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160bc9a60 con 0x560160b346b0 2023-07-14T16:26:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efce299eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160bc9760 con 0x560160cd97e0 2023-07-14T16:26:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efce299eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160aad8c0 con 0x560160cd8a90 2023-07-14T16:26:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efce299eb80 10 client.0 _ll_register_callbacks cb 0x560160bc7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efccd59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160ce19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efccd59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160ce19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47500/0 (socket says 172.21.15.62:47500) 2023-07-14T16:26:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.098+0000 7efccd59c700 1 -- 172.21.15.62:0/2174491323 learned_addr learned my addr 172.21.15.62:0/2174491323 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efccd59c700 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160ce8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efcc7fff700 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 0x560160ce1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efccd59c700 1 -- 172.21.15.62:0/2174491323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 msgr2=0x560160ce1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efccd59c700 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 0x560160ce1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efccd59c700 1 -- 172.21.15.62:0/2174491323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 msgr2=0x560160ce8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efccd59c700 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160ce8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efccd59c700 1 -- 172.21.15.62:0/2174491323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc810a8d0 con 0x560160b346b0 2023-07-14T16:26:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efccd59c700 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160ce19d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7efcc0008fc0 tx=0x7efcc0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c067cc9cbbc8353 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.099+0000 7efca7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efca7fff700 1 -- 172.21.15.62:0/2174491323 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0003a00 con 0x560160b346b0 2023-07-14T16:26:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 1 -- 172.21.15.62:0/2174491323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560160b83490 con 0x560160b346b0 2023-07-14T16:26:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efca7fff700 1 -- 172.21.15.62:0/2174491323 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efcc0003b60 con 0x560160b346b0 2023-07-14T16:26:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efca7fff700 1 -- 172.21.15.62:0/2174491323 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcc0003d60 con 0x560160b346b0 2023-07-14T16:26:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 10 client.18621 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 1 -- 172.21.15.62:0/2174491323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560160a40c00 con 0x560160b346b0 2023-07-14T16:26:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efca7fff700 1 -- 172.21.15.62:0/2174491323 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560160b83490 con 0x560160b346b0 2023-07-14T16:26:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efca7fff700 1 -- 172.21.15.62:0/2174491323 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efcc0003ec0 con 0x560160b346b0 2023-07-14T16:26:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 10 client.18621 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 10 client.18621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 20 client.18621 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560160ced880 0x560160cefc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 4 client.18621 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.100+0000 7efce299eb80 1 -- 172.21.15.62:0/2174491323 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560160c22e50 con 0x560160ced880 2023-07-14T16:26:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.101+0000 7efcc7fff700 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560160ced880 0x560160cefc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.102+0000 7efcc7fff700 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560160ced880 0x560160cefc90 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee47fbb7b6da5e1 server_cookie=7ae347c8d3706166 in_seq=0 out_seq=0 2023-07-14T16:26:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.102+0000 7efca7fff700 10 client.18621 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.102+0000 7efca7fff700 1 -- 172.21.15.62:0/2174491323 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560160c22e50 con 0x560160ced880 2023-07-14T16:26:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.102+0000 7efca7fff700 10 client.18621 handle_command_reply: tid=0 2023-07-14T16:26:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.102+0000 7efce299eb80 1 client.18621 shutdown 2023-07-14T16:26:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 -- 172.21.15.62:0/2174491323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 msgr2=0x560160ce19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160ce19d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7efcc0008fc0 tx=0x7efcc0004ad0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 -- 172.21.15.62:0/2174491323 shutdown_connections 2023-07-14T16:26:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560160ced880 0x560160cefc90 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160ce8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160ce19d0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 --2- 172.21.15.62:0/2174491323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 0x560160ce1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 -- 172.21.15.62:0/2174491323 >> 172.21.15.62:0/2174491323 conn(0x560160ccdd40 msgr2=0x560160cd2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efca7fff700 0 client.18621 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 -- 172.21.15.62:0/2174491323 shutdown_connections 2023-07-14T16:26:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 1 -- 172.21.15.62:0/2174491323 wait complete. 2023-07-14T16:26:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.103+0000 7efce299eb80 20 client.18621 trim_cache size 0 max 16384 2023-07-14T16:26:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.120+0000 7efce299eb80 1 Processor -- start 2023-07-14T16:26:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.120+0000 7efce299eb80 1 -- start start 2023-07-14T16:26:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efce299eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160cdc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efce299eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 0x560160ce8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efce299eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cdcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efce299eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160cd81e0 con 0x560160b346b0 2023-07-14T16:26:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efce299eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160cd3960 con 0x560160cd97e0 2023-07-14T16:26:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efce299eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160ccfe90 con 0x560160cd8a90 2023-07-14T16:26:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efccd59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160cdc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efccd59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160cdc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47516/0 (socket says 172.21.15.62:47516) 2023-07-14T16:26:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efccd59c700 1 -- 172.21.15.62:0/889188210 learned_addr learned my addr 172.21.15.62:0/889188210 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efcc7fff700 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 0x560160ce8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.121+0000 7efccd59c700 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cdcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.122+0000 7efccd59c700 1 -- 172.21.15.62:0/889188210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 msgr2=0x560160ce8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.122+0000 7efccd59c700 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 0x560160ce8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.122+0000 7efccd59c700 1 -- 172.21.15.62:0/889188210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 msgr2=0x560160cdcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.122+0000 7efccd59c700 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cdcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.122+0000 7efccd59c700 1 -- 172.21.15.62:0/889188210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560160a7e1f0 con 0x560160b346b0 2023-07-14T16:26:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.122+0000 7efccd59c700 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160cdc5b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7efcc0000f80 tx=0x7efcc0009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac1478543797131 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.122+0000 7efcb37fe700 1 -- 172.21.15.62:0/889188210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00047d0 con 0x560160b346b0 2023-07-14T16:26:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efcb37fe700 1 -- 172.21.15.62:0/889188210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcc000f070 con 0x560160b346b0 2023-07-14T16:26:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efcb37fe700 1 -- 172.21.15.62:0/889188210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0003c00 con 0x560160b346b0 2023-07-14T16:26:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efce299eb80 1 -- 172.21.15.62:0/889188210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 msgr2=0x560160cdc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efce299eb80 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160cdc5b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7efcc0000f80 tx=0x7efcc0009340 comp rx=0 tx=0).stop 2023-07-14T16:26:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efce299eb80 1 -- 172.21.15.62:0/889188210 shutdown_connections 2023-07-14T16:26:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efce299eb80 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160cd97e0 0x560160cdcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efce299eb80 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160b346b0 0x560160cdc5b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efce299eb80 1 --2- 172.21.15.62:0/889188210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd8a90 0x560160ce8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efce299eb80 1 -- 172.21.15.62:0/889188210 >> 172.21.15.62:0/889188210 conn(0x560160ccdd40 msgr2=0x560160cef890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.123+0000 7efce299eb80 1 -- 172.21.15.62:0/889188210 shutdown_connections 2023-07-14T16:26:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.124+0000 7efce299eb80 1 -- 172.21.15.62:0/889188210 wait complete. 2023-07-14T16:26:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.124+0000 7efce299eb80 1 -- start start 2023-07-14T16:26:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.124+0000 7efce299eb80 1 Processor -- start 2023-07-14T16:26:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efce299eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160b346b0 0x560160ce4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efce299eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160ce4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efce299eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd97e0 0x560160ce7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efce299eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160ce83b0 con 0x560160cd8a90 2023-07-14T16:26:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efce299eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160ce8520 con 0x560160b346b0 2023-07-14T16:26:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efce299eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560160ce86e0 con 0x560160cd97e0 2023-07-14T16:26:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efce299eb80 10 client.0 _ll_register_callbacks cb 0x7efcc00089f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160ce4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efcc7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160ce4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47532/0 (socket says 172.21.15.62:47532) 2023-07-14T16:26:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efcc7fff700 1 -- 172.21.15.62:0/758108219 learned_addr learned my addr 172.21.15.62:0/758108219 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efccd59c700 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160b346b0 0x560160ce4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efccd59c700 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd97e0 0x560160ce7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efcc7fff700 1 -- 172.21.15.62:0/758108219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd97e0 msgr2=0x560160ce7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efcc7fff700 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd97e0 0x560160ce7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efcc7fff700 1 -- 172.21.15.62:0/758108219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160b346b0 msgr2=0x560160ce4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efcc7fff700 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160b346b0 0x560160ce4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efcc7fff700 1 -- 172.21.15.62:0/758108219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560160a40c00 con 0x560160cd8a90 2023-07-14T16:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.125+0000 7efccd59c700 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd97e0 0x560160ce7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.126+0000 7efcc7fff700 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160ce4e80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x560160cd3cb0 tx=0x7efcb4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bc4255de5eefdf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.126+0000 7efca7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.126+0000 7efca7fff700 1 -- 172.21.15.62:0/758108219 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4033070 con 0x560160cd8a90 2023-07-14T16:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.126+0000 7efca7fff700 1 -- 172.21.15.62:0/758108219 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efcb40333d0 con 0x560160cd8a90 2023-07-14T16:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.126+0000 7efce299eb80 1 -- 172.21.15.62:0/758108219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcb401e040 con 0x560160cd8a90 2023-07-14T16:26:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.126+0000 7efca7fff700 1 -- 172.21.15.62:0/758108219 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcb401d040 con 0x560160cd8a90 2023-07-14T16:26:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.126+0000 7efce299eb80 10 client.18627 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.126+0000 7efce299eb80 1 -- 172.21.15.62:0/758108219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560160a56250 con 0x560160cd8a90 2023-07-14T16:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efca7fff700 1 -- 172.21.15.62:0/758108219 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcb401e040 con 0x560160cd8a90 2023-07-14T16:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efca7fff700 1 -- 172.21.15.62:0/758108219 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efcb4005160 con 0x560160cd8a90 2023-07-14T16:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efce299eb80 10 client.18627 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efce299eb80 10 client.18627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efce299eb80 20 client.18627 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efce299eb80 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560160d41e10 0x560160d44220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efce299eb80 4 client.18627 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:26:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efce299eb80 1 -- 172.21.15.62:0/758108219 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560160be4ff0 con 0x560160d41e10 2023-07-14T16:26:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.127+0000 7efccd59c700 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560160d41e10 0x560160d44220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.128+0000 7efccd59c700 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560160d41e10 0x560160d44220 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=147644086f005a99 server_cookie=ed28e7237c1dc02a in_seq=0 out_seq=0 2023-07-14T16:26:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.128+0000 7efca7fff700 10 client.18627 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efca7fff700 1 -- 172.21.15.62:0/758108219 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560160be4ff0 con 0x560160d41e10 2023-07-14T16:26:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efca7fff700 10 client.18627 handle_command_reply: tid=0 2023-07-14T16:26:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 client.18627 shutdown 2023-07-14T16:26:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 -- 172.21.15.62:0/758108219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 msgr2=0x560160ce4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160ce4e80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x560160cd3cb0 tx=0x7efcb4009310 comp rx=0 tx=0).stop 2023-07-14T16:26:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 -- 172.21.15.62:0/758108219 shutdown_connections 2023-07-14T16:26:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560160cd97e0 0x560160ce7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560160d41e10 0x560160d44220 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560160b346b0 0x560160ce4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 --2- 172.21.15.62:0/758108219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560160cd8a90 0x560160ce4e80 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efca7fff700 0 client.18627 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.139+0000 7efce299eb80 1 -- 172.21.15.62:0/758108219 >> 172.21.15.62:0/758108219 conn(0x560160ccdd40 msgr2=0x560160cdc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.140+0000 7efce299eb80 1 -- 172.21.15.62:0/758108219 shutdown_connections 2023-07-14T16:26:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.140+0000 7efce299eb80 1 -- 172.21.15.62:0/758108219 wait complete. 2023-07-14T16:26:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.140+0000 7efce299eb80 20 client.18627 trim_cache size 0 max 16384 2023-07-14T16:26:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:26:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:26:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:26:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:26:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 -- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efcb80a11d0 msgr2=0x7efcb80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efcb80a11d0 0x7efcb80a3680 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efcbc099b30 tx=0x7efcbc00cf50 comp rx=0 tx=0).stop 2023-07-14T16:26:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 -- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8101b60 msgr2=0x7efcc81977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8101b60 0x7efcc81977d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00e970 tx=0x7efcbc00ec80 comp rx=0 tx=0).stop 2023-07-14T16:26:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 -- 172.21.15.62:0/2960818193 shutdown_connections 2023-07-14T16:26:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efcb80a11d0 0x7efcb80a3680 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcc81036e0 0x7efcc819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcc8101b60 0x7efcc81977d0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 --2- 172.21.15.62:0/2960818193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcc8102d50 0x7efcc8197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 -- 172.21.15.62:0/2960818193 >> 172.21.15.62:0/2960818193 conn(0x7efcc80fd180 msgr2=0x7efcc81062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 -- 172.21.15.62:0/2960818193 shutdown_connections 2023-07-14T16:26:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:26:51.141+0000 7efccdd9d700 1 -- 172.21.15.62:0/2960818193 wait complete. 2023-07-14T16:26:51.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:50 smithi062 ceph-mon[95543]: pgmap v693: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:26:51.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:50 smithi081 ceph-mon[127795]: pgmap v693: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:26:53.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:52 smithi164 ceph-mon[97427]: pgmap v694: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:53.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:52 smithi062 ceph-mon[95543]: pgmap v694: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:53.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:52 smithi081 ceph-mon[127795]: pgmap v694: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:26:55.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:54 smithi164 ceph-mon[97427]: pgmap v695: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:26:55.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:54 smithi062 ceph-mon[95543]: pgmap v695: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:26:55.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:54 smithi081 ceph-mon[127795]: pgmap v695: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:26:57.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:56 smithi164 ceph-mon[97427]: pgmap v696: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:26:57.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:56 smithi062 ceph-mon[95543]: pgmap v696: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:26:57.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:56 smithi081 ceph-mon[127795]: pgmap v696: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:26:57.856 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:26:58.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.114+0000 7fbd59fff700 1 -- 172.21.15.62:0/1912080365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 msgr2=0x7fbd54101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.114+0000 7fbd59fff700 1 --2- 172.21.15.62:0/1912080365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 0x7fbd54101130 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009990 tx=0x7fbd48009ca0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.115+0000 7fbd59fff700 1 -- 172.21.15.62:0/1912080365 shutdown_connections 2023-07-14T16:26:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.115+0000 7fbd59fff700 1 --2- 172.21.15.62:0/1912080365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd54101670 0x7fbd54107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.115+0000 7fbd59fff700 1 --2- 172.21.15.62:0/1912080365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd540ffaf0 0x7fbd540fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.115+0000 7fbd59fff700 1 --2- 172.21.15.62:0/1912080365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 0x7fbd54101130 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.115+0000 7fbd59fff700 1 -- 172.21.15.62:0/1912080365 >> 172.21.15.62:0/1912080365 conn(0x7fbd540fb130 msgr2=0x7fbd540fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.115+0000 7fbd59fff700 1 -- 172.21.15.62:0/1912080365 shutdown_connections 2023-07-14T16:26:58.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.115+0000 7fbd59fff700 1 -- 172.21.15.62:0/1912080365 wait complete. 2023-07-14T16:26:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.116+0000 7fbd59fff700 1 Processor -- start 2023-07-14T16:26:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.116+0000 7fbd59fff700 1 -- start start 2023-07-14T16:26:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd59fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd540ffaf0 0x7fbd540742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd59fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 0x7fbd54072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd59fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54101670 0x7fbd54072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd59fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540735e0 con 0x7fbd54100ce0 2023-07-14T16:26:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd59fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54073750 con 0x7fbd54101670 2023-07-14T16:26:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd59fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5419f870 con 0x7fbd540ffaf0 2023-07-14T16:26:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 0x7fbd54072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 0x7fbd54072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40364/0 (socket says 172.21.15.62:40364) 2023-07-14T16:26:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd52ffd700 1 -- 172.21.15.62:0/3704387397 learned_addr learned my addr 172.21.15.62:0/3704387397 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd537fe700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd540ffaf0 0x7fbd540742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.117+0000 7fbd53fff700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54101670 0x7fbd54072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd52ffd700 1 -- 172.21.15.62:0/3704387397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd540ffaf0 msgr2=0x7fbd540742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd540ffaf0 0x7fbd540742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd52ffd700 1 -- 172.21.15.62:0/3704387397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54101670 msgr2=0x7fbd54072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54101670 0x7fbd54072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd52ffd700 1 -- 172.21.15.62:0/3704387397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480095f0 con 0x7fbd54100ce0 2023-07-14T16:26:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd53fff700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54101670 0x7fbd54072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 0x7fbd54072900 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fbd48006010 tx=0x7fbd48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd50ff9700 1 -- 172.21.15.62:0/3704387397 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4801d070 con 0x7fbd54100ce0 2023-07-14T16:26:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd59fff700 1 -- 172.21.15.62:0/3704387397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5419faf0 con 0x7fbd54100ce0 2023-07-14T16:26:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd50ff9700 1 -- 172.21.15.62:0/3704387397 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd480054c0 con 0x7fbd54100ce0 2023-07-14T16:26:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.118+0000 7fbd50ff9700 1 -- 172.21.15.62:0/3704387397 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48003ab0 con 0x7fbd54100ce0 2023-07-14T16:26:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.119+0000 7fbd59fff700 1 -- 172.21.15.62:0/3704387397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5419ff10 con 0x7fbd54100ce0 2023-07-14T16:26:58.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.122+0000 7fbd50ff9700 1 -- 172.21.15.62:0/3704387397 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbd480047a0 con 0x7fbd54100ce0 2023-07-14T16:26:58.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.123+0000 7fbd50ff9700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd3c0a1150 0x7fbd3c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.123+0000 7fbd50ff9700 1 -- 172.21.15.62:0/3704387397 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbd480cd910 con 0x7fbd54100ce0 2023-07-14T16:26:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.123+0000 7fbd537fe700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd3c0a1150 0x7fbd3c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.124+0000 7fbd6d8b7b80 1 Processor -- start 2023-07-14T16:26:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.124+0000 7fbd6d8b7b80 1 -- start start 2023-07-14T16:26:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.124+0000 7fbd537fe700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd3c0a1150 0x7fbd3c0a3600 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbd54100b40 tx=0x7fbd4400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.124+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.124+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31df630 0x5569c31dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.124+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31e0480 0x5569c31e08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c30dbcb0 con 0x5569c31e0480 2023-07-14T16:26:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c312ea40 con 0x5569c31df630 2023-07-14T16:26:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c3127ed0 con 0x5569c31dece0 2023-07-14T16:26:58.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd537fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31df630 0x5569c31dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd537fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31df0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42286/0 (socket says 172.21.15.62:42286) 2023-07-14T16:26:58.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd537fe700 1 -- 172.21.15.62:0/1436705796 learned_addr learned my addr 172.21.15.62:0/1436705796 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:58.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd53fff700 1 --2- 172.21.15.62:0/1436705796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31e0480 0x5569c31e08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd537fe700 1 -- 172.21.15.62:0/1436705796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31df630 msgr2=0x5569c31dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:58.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd537fe700 1 --2- 172.21.15.62:0/1436705796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31df630 0x5569c31dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd537fe700 1 -- 172.21.15.62:0/1436705796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31e0480 msgr2=0x5569c31e08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd537fe700 1 --2- 172.21.15.62:0/1436705796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31e0480 0x5569c31e08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.125+0000 7fbd537fe700 1 -- 172.21.15.62:0/1436705796 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569c2f55de0 con 0x5569c31dece0 2023-07-14T16:26:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.126+0000 7fbd537fe700 1 --2- 172.21.15.62:0/1436705796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31df0f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd44010f70 tx=0x7fbd44013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a513a630c5de30a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.126+0000 7fbd39ffb700 1 -- 172.21.15.62:0/1436705796 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44011db0 con 0x5569c31dece0 2023-07-14T16:26:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.126+0000 7fbd39ffb700 1 -- 172.21.15.62:0/1436705796 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd44015070 con 0x5569c31dece0 2023-07-14T16:26:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.126+0000 7fbd39ffb700 1 -- 172.21.15.62:0/1436705796 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4401c610 con 0x5569c31dece0 2023-07-14T16:26:58.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.127+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/1436705796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 msgr2=0x5569c31df0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.127+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/1436705796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31df0f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd44010f70 tx=0x7fbd44013000 comp rx=0 tx=0).stop 2023-07-14T16:26:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.127+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/1436705796 shutdown_connections 2023-07-14T16:26:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.127+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/1436705796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31e0480 0x5569c31e08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.127+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/1436705796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31df0f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.127+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/1436705796 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31df630 0x5569c31dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.127+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/1436705796 >> 172.21.15.62:0/1436705796 conn(0x5569c31cbda0 msgr2=0x5569c31d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.128+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/1436705796 shutdown_connections 2023-07-14T16:26:58.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.128+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/1436705796 wait complete. 2023-07-14T16:26:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.128+0000 7fbd6d8b7b80 1 -- start start 2023-07-14T16:26:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.128+0000 7fbd6d8b7b80 1 Processor -- start 2023-07-14T16:26:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 0x5569c31efed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c30c7ac0 con 0x5569c31df630 2023-07-14T16:26:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c30c77c0 con 0x5569c31e0480 2023-07-14T16:26:58.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c2fab920 con 0x5569c31dece0 2023-07-14T16:26:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd6d8b7b80 10 client.0 _ll_register_callbacks cb 0x5569c30c5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31e8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40380/0 (socket says 172.21.15.62:40380) 2023-07-14T16:26:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd52ffd700 1 -- 172.21.15.62:0/966862075 learned_addr learned my addr 172.21.15.62:0/966862075 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.129+0000 7fbd53fff700 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 0x5569c31efed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.130+0000 7fbd537fe700 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.130+0000 7fbd52ffd700 1 -- 172.21.15.62:0/966862075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 msgr2=0x5569c31e8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:58.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.130+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31e8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.130+0000 7fbd52ffd700 1 -- 172.21.15.62:0/966862075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 msgr2=0x5569c31efed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.130+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 0x5569c31efed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.130+0000 7fbd52ffd700 1 -- 172.21.15.62:0/966862075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd44010c50 con 0x5569c31df630 2023-07-14T16:26:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.130+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31e8e90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fbd48097260 tx=0x7fbd4809b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cf988ef5139c4f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.130+0000 7fbd32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd32ffd700 1 -- 172.21.15.62:0/966862075 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd480b3420 con 0x5569c31df630 2023-07-14T16:26:58.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd32ffd700 1 -- 172.21.15.62:0/966862075 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbd480a3120 con 0x5569c31df630 2023-07-14T16:26:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/966862075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569c30814f0 con 0x5569c31df630 2023-07-14T16:26:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd32ffd700 1 -- 172.21.15.62:0/966862075 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd480a2c90 con 0x5569c31df630 2023-07-14T16:26:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd6d8b7b80 10 client.18639 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/966862075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5569c2f3ec60 con 0x5569c31df630 2023-07-14T16:26:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd32ffd700 1 -- 172.21.15.62:0/966862075 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd480c9080 con 0x5569c31df630 2023-07-14T16:26:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd32ffd700 1 -- 172.21.15.62:0/966862075 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbd480caf80 con 0x5569c31df630 2023-07-14T16:26:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd6d8b7b80 10 client.18639 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:58.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.131+0000 7fbd6d8b7b80 10 client.18639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.132+0000 7fbd6d8b7b80 20 client.18639 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.132+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5569c31f43f0 0x5569c31f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.132+0000 7fbd6d8b7b80 4 client.18639 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:58.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.132+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/966862075 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569c3120eb0 con 0x5569c31f43f0 2023-07-14T16:26:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.132+0000 7fbd537fe700 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5569c31f43f0 0x5569c31f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.133+0000 7fbd32ffd700 10 client.18639 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.133+0000 7fbd537fe700 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5569c31f43f0 0x5569c31f67d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623c1f8c29876f7a server_cookie=e72a153cdb0bd83d in_seq=0 out_seq=0 2023-07-14T16:26:58.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.134+0000 7fbd32ffd700 1 -- 172.21.15.62:0/966862075 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5569c3120eb0 con 0x5569c31f43f0 2023-07-14T16:26:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.134+0000 7fbd32ffd700 10 client.18639 handle_command_reply: tid=0 2023-07-14T16:26:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.134+0000 7fbd6d8b7b80 1 client.18639 shutdown 2023-07-14T16:26:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/966862075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 msgr2=0x5569c31e8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31e8e90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fbd48097260 tx=0x7fbd4809b2a0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/966862075 shutdown_connections 2023-07-14T16:26:58.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5569c31f43f0 0x5569c31f67d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 0x5569c31efed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/966862075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31e8e90 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd32ffd700 0 client.18639 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:58.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/966862075 >> 172.21.15.62:0/966862075 conn(0x5569c31cbda0 msgr2=0x5569c31d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/966862075 shutdown_connections 2023-07-14T16:26:58.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/966862075 wait complete. 2023-07-14T16:26:58.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.135+0000 7fbd6d8b7b80 20 client.18639 trim_cache size 0 max 16384 2023-07-14T16:26:58.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.159+0000 7fbd6d8b7b80 1 Processor -- start 2023-07-14T16:26:58.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.159+0000 7fbd6d8b7b80 1 -- start start 2023-07-14T16:26:58.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.159+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.159+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31efe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.159+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 0x5569c31e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.159+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c31e84c0 con 0x5569c31df630 2023-07-14T16:26:58.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.159+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c31cb970 con 0x5569c31e0480 2023-07-14T16:26:58.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.159+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c31f41d0 con 0x5569c31dece0 2023-07-14T16:26:58.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31efe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40396/0 (socket says 172.21.15.62:40396) 2023-07-14T16:26:58.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd52ffd700 1 -- 172.21.15.62:0/4124256890 learned_addr learned my addr 172.21.15.62:0/4124256890 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:58.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd53fff700 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 0x5569c31e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd537fe700 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd52ffd700 1 -- 172.21.15.62:0/4124256890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 msgr2=0x5569c31e8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:58.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31e8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd52ffd700 1 -- 172.21.15.62:0/4124256890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 msgr2=0x5569c31e8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:58.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 0x5569c31e8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.160+0000 7fbd52ffd700 1 -- 172.21.15.62:0/4124256890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569c2f7c250 con 0x5569c31df630 2023-07-14T16:26:58.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.161+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31efe90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fbd480977d0 tx=0x7fbd48099310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bccdc10bdd5629e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.161+0000 7fbd3a7fc700 1 -- 172.21.15.62:0/4124256890 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48096ef0 con 0x5569c31df630 2023-07-14T16:26:58.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.161+0000 7fbd3a7fc700 1 -- 172.21.15.62:0/4124256890 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5569c30814f0 con 0x5569c31df630 2023-07-14T16:26:58.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.161+0000 7fbd3a7fc700 1 -- 172.21.15.62:0/4124256890 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd480b3480 con 0x5569c31df630 2023-07-14T16:26:58.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/4124256890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 msgr2=0x5569c31efe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31efe90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fbd480977d0 tx=0x7fbd48099310 comp rx=0 tx=0).stop 2023-07-14T16:26:58.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/4124256890 shutdown_connections 2023-07-14T16:26:58.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31e0480 0x5569c31e8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31dece0 0x5569c31e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/4124256890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31efe90 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/4124256890 >> 172.21.15.62:0/4124256890 conn(0x5569c31cbda0 msgr2=0x5569c31e67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/4124256890 shutdown_connections 2023-07-14T16:26:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/4124256890 wait complete. 2023-07-14T16:26:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.162+0000 7fbd6d8b7b80 1 -- start start 2023-07-14T16:26:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd6d8b7b80 1 Processor -- start 2023-07-14T16:26:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31dece0 0x5569c31efa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31ec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd6d8b7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31e0480 0x5569c31ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c31f0270 con 0x5569c31df630 2023-07-14T16:26:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c31ed130 con 0x5569c31dece0 2023-07-14T16:26:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd6d8b7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569c31ed2f0 con 0x5569c31e0480 2023-07-14T16:26:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd6d8b7b80 10 client.0 _ll_register_callbacks cb 0x7fbd48005650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd52ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40400/0 (socket says 172.21.15.62:40400) 2023-07-14T16:26:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd52ffd700 1 -- 172.21.15.62:0/297841242 learned_addr learned my addr 172.21.15.62:0/297841242 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.163+0000 7fbd537fe700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31dece0 0x5569c31efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd53fff700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31e0480 0x5569c31ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd52ffd700 1 -- 172.21.15.62:0/297841242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31e0480 msgr2=0x5569c31ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31e0480 0x5569c31ecbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd52ffd700 1 -- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31dece0 msgr2=0x5569c31efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31dece0 0x5569c31efa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd52ffd700 1 -- 172.21.15.62:0/297841242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569c2f3ec60 con 0x5569c31df630 2023-07-14T16:26:58.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd53fff700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31e0480 0x5569c31ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:26:58.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd537fe700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31dece0 0x5569c31efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:58.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd52ffd700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31ec680 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fbd48099cc0 tx=0x7fbd48099f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7b8ebc81c4740d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:58.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd32ffd700 1 -- 172.21.15.62:0/297841242 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48096ef0 con 0x5569c31df630 2023-07-14T16:26:58.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/297841242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd44015070 con 0x5569c31df630 2023-07-14T16:26:58.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.164+0000 7fbd32ffd700 1 -- 172.21.15.62:0/297841242 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbd480a3120 con 0x5569c31df630 2023-07-14T16:26:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd32ffd700 1 -- 172.21.15.62:0/297841242 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd48066030 con 0x5569c31df630 2023-07-14T16:26:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd6d8b7b80 10 client.18645 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/297841242 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5569c2f542b0 con 0x5569c31df630 2023-07-14T16:26:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd32ffd700 1 -- 172.21.15.62:0/297841242 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd4807d070 con 0x5569c31df630 2023-07-14T16:26:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd32ffd700 1 -- 172.21.15.62:0/297841242 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbd4809bb40 con 0x5569c31df630 2023-07-14T16:26:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd6d8b7b80 10 client.18645 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd6d8b7b80 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd6d8b7b80 20 client.18645 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5569c3249620 0x5569c324ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd6d8b7b80 4 client.18645 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:26:58.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.165+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/297841242 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5569c30e3050 con 0x5569c3249620 2023-07-14T16:26:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.166+0000 7fbd537fe700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5569c3249620 0x5569c324ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.166+0000 7fbd537fe700 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5569c3249620 0x5569c324ba30 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7efef3f93063aa server_cookie=d9cda330aff6af11 in_seq=0 out_seq=0 2023-07-14T16:26:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.166+0000 7fbd32ffd700 10 client.18645 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:58.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd32ffd700 1 -- 172.21.15.62:0/297841242 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5569c30e3050 con 0x5569c3249620 2023-07-14T16:26:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd32ffd700 10 client.18645 handle_command_reply: tid=0 2023-07-14T16:26:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 client.18645 shutdown 2023-07-14T16:26:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/297841242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 msgr2=0x5569c31ec680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31ec680 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fbd48099cc0 tx=0x7fbd48099f70 comp rx=0 tx=0).stop 2023-07-14T16:26:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/297841242 shutdown_connections 2023-07-14T16:26:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569c31e0480 0x5569c31ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5569c3249620 0x5569c324ba30 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569c31dece0 0x5569c31efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 --2- 172.21.15.62:0/297841242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569c31df630 0x5569c31ec680 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/297841242 >> 172.21.15.62:0/297841242 conn(0x5569c31cbda0 msgr2=0x5569c31e88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd32ffd700 0 client.18645 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/297841242 shutdown_connections 2023-07-14T16:26:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 1 -- 172.21.15.62:0/297841242 wait complete. 2023-07-14T16:26:58.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.167+0000 7fbd6d8b7b80 20 client.18645 trim_cache size 0 max 16384 2023-07-14T16:26:58.151 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:26:58.151 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:26:58.151 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:26:58.151 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:26:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 -- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd3c0a1150 msgr2=0x7fbd3c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd3c0a1150 0x7fbd3c0a3600 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbd54100b40 tx=0x7fbd4400b410 comp rx=0 tx=0).stop 2023-07-14T16:26:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 -- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 msgr2=0x7fbd54072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 0x7fbd54072900 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fbd48006010 tx=0x7fbd48004730 comp rx=0 tx=0).stop 2023-07-14T16:26:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 -- 172.21.15.62:0/3704387397 shutdown_connections 2023-07-14T16:26:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbd3c0a1150 0x7fbd3c0a3600 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54101670 0x7fbd54072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd540ffaf0 0x7fbd540742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 --2- 172.21.15.62:0/3704387397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd54100ce0 0x7fbd54072900 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 -- 172.21.15.62:0/3704387397 >> 172.21.15.62:0/3704387397 conn(0x7fbd540fb130 msgr2=0x7fbd541042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 -- 172.21.15.62:0/3704387397 shutdown_connections 2023-07-14T16:26:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.168+0000 7fbd59fff700 1 -- 172.21.15.62:0/3704387397 wait complete. 2023-07-14T16:26:58.162 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:26:58.163 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:26:58.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.420+0000 7fd9c5bff700 1 -- 172.21.15.62:0/2443119609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 msgr2=0x7fd9c0107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.420+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/2443119609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 0x7fd9c0107f40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800b0a0 tx=0x7fd9b800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.420+0000 7fd9c5bff700 1 -- 172.21.15.62:0/2443119609 shutdown_connections 2023-07-14T16:26:58.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.420+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/2443119609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 0x7fd9c0107f40 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.420+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/2443119609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9c00ffb00 0x7fd9c00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.420+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/2443119609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c0100cf0 0x7fd9c0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.420+0000 7fd9c5bff700 1 -- 172.21.15.62:0/2443119609 >> 172.21.15.62:0/2443119609 conn(0x7fd9c00fb0e0 msgr2=0x7fd9c00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.420+0000 7fd9c5bff700 1 -- 172.21.15.62:0/2443119609 shutdown_connections 2023-07-14T16:26:58.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.421+0000 7fd9c5bff700 1 -- 172.21.15.62:0/2443119609 wait complete. 2023-07-14T16:26:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9c5bff700 1 Processor -- start 2023-07-14T16:26:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9c5bff700 1 -- start start 2023-07-14T16:26:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9c5bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9c00ffb00 0x7fd9c01979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9c5bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c0100cf0 0x7fd9c0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9c5bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 0x7fd9c019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9c5bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c019f520 con 0x7fd9c0101680 2023-07-14T16:26:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9c5bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c019f660 con 0x7fd9c0100cf0 2023-07-14T16:26:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9c5bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c019f7f0 con 0x7fd9c00ffb00 2023-07-14T16:26:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 0x7fd9c019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 0x7fd9c019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40418/0 (socket says 172.21.15.62:40418) 2023-07-14T16:26:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.422+0000 7fd9bffff700 1 -- 172.21.15.62:0/113027720 learned_addr learned my addr 172.21.15.62:0/113027720 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.423+0000 7fd9beffd700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c0100cf0 0x7fd9c0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.423+0000 7fd9bf7fe700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9c00ffb00 0x7fd9c01979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.423+0000 7fd9bffff700 1 -- 172.21.15.62:0/113027720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9c00ffb00 msgr2=0x7fd9c01979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.423+0000 7fd9bffff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9c00ffb00 0x7fd9c01979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.423+0000 7fd9bffff700 1 -- 172.21.15.62:0/113027720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c0100cf0 msgr2=0x7fd9c0197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.423+0000 7fd9bffff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c0100cf0 0x7fd9c0197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.423+0000 7fd9bffff700 1 -- 172.21.15.62:0/113027720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8009d00 con 0x7fd9c0101680 2023-07-14T16:26:58.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.423+0000 7fd9bffff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 0x7fd9c019ef50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80095e0 tx=0x7fd9b800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.424+0000 7fd9bcff9700 1 -- 172.21.15.62:0/113027720 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800e040 con 0x7fd9c0101680 2023-07-14T16:26:58.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.424+0000 7fd9bcff9700 1 -- 172.21.15.62:0/113027720 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9b800be90 con 0x7fd9c0101680 2023-07-14T16:26:58.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.424+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c019fa70 con 0x7fd9c0101680 2023-07-14T16:26:58.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.424+0000 7fd9bcff9700 1 -- 172.21.15.62:0/113027720 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8005040 con 0x7fd9c0101680 2023-07-14T16:26:58.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.424+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c019ff40 con 0x7fd9c0101680 2023-07-14T16:26:58.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.425+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c010b190 con 0x7fd9c0101680 2023-07-14T16:26:58.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.428+0000 7fd9bcff9700 1 -- 172.21.15.62:0/113027720 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd9b8003c00 con 0x7fd9c0101680 2023-07-14T16:26:58.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.430+0000 7fd9bcff9700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9a80a1220 0x7fd9a80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.430+0000 7fd9bcff9700 1 -- 172.21.15.62:0/113027720 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9b80cdb30 con 0x7fd9c0101680 2023-07-14T16:26:58.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.430+0000 7fd9bf7fe700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9a80a1220 0x7fd9a80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.431+0000 7fd9bf7fe700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9a80a1220 0x7fd9a80a36d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0100b50 tx=0x7fd9b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.434+0000 7fd9bcff9700 1 -- 172.21.15.62:0/113027720 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd9b8097320 con 0x7fd9c0101680 2023-07-14T16:26:58.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.742+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9c010b3b0 con 0x7fd9c0101680 2023-07-14T16:26:58.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.743+0000 7fd9bcff9700 1 -- 172.21.15.62:0/113027720 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fd9b809a940 con 0x7fd9c0101680 2023-07-14T16:26:58.719 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:26:58.719 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:26:58.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9a80a1220 msgr2=0x7fd9a80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9a80a1220 0x7fd9a80a36d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0100b50 tx=0x7fd9b0009260 comp rx=0 tx=0).stop 2023-07-14T16:26:58.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 msgr2=0x7fd9c019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 0x7fd9c019ef50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80095e0 tx=0x7fd9b800ba50 comp rx=0 tx=0).stop 2023-07-14T16:26:58.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 shutdown_connections 2023-07-14T16:26:58.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c0101680 0x7fd9c019ef50 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9c00ffb00 0x7fd9c01979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd9a80a1220 0x7fd9a80a36d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 --2- 172.21.15.62:0/113027720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9c0100cf0 0x7fd9c0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.745+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 >> 172.21.15.62:0/113027720 conn(0x7fd9c00fb0e0 msgr2=0x7fd9c0106540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.746+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 shutdown_connections 2023-07-14T16:26:58.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:58.746+0000 7fd9c5bff700 1 -- 172.21.15.62:0/113027720 wait complete. 2023-07-14T16:26:58.723 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:26:58.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:26:58.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 -- 172.21.15.62:0/1366318814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3858105800 msgr2=0x7f3858107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 --2- 172.21.15.62:0/1366318814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3858105800 0x7f3858107c00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f385000b0a0 tx=0x7f385000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 -- 172.21.15.62:0/1366318814 shutdown_connections 2023-07-14T16:26:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 --2- 172.21.15.62:0/1366318814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3858105800 0x7f3858107c00 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 --2- 172.21.15.62:0/1366318814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38581005c0 0x7f38581029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 --2- 172.21.15.62:0/1366318814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3858102ee0 0x7f38581052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 -- 172.21.15.62:0/1366318814 >> 172.21.15.62:0/1366318814 conn(0x7f38580fa000 msgr2=0x7f38580fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 -- 172.21.15.62:0/1366318814 shutdown_connections 2023-07-14T16:26:58.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.009+0000 7f385e7ff700 1 -- 172.21.15.62:0/1366318814 wait complete. 2023-07-14T16:26:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.011+0000 7f385e7ff700 1 Processor -- start 2023-07-14T16:26:58.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.011+0000 7f385e7ff700 1 -- start start 2023-07-14T16:26:58.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.011+0000 7f385e7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38581005c0 0x7f3858195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.011+0000 7f385e7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3858102ee0 0x7f3858195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.011+0000 7f385e7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3858105800 0x7f385819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.011+0000 7f385e7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385819d3a0 con 0x7f3858105800 2023-07-14T16:26:58.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.011+0000 7f385e7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385819d4e0 con 0x7f3858102ee0 2023-07-14T16:26:58.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.011+0000 7f385e7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385819d670 con 0x7f38581005c0 2023-07-14T16:26:58.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f3857fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38581005c0 0x7f3858195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f3857fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38581005c0 0x7f3858195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42384/0 (socket says 172.21.15.62:42384) 2023-07-14T16:26:58.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f3857fff700 1 -- 172.21.15.62:0/348740679 learned_addr learned my addr 172.21.15.62:0/348740679 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:58.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f385cd9c700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3858105800 0x7f385819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f38577fe700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3858102ee0 0x7f3858195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f3857fff700 1 -- 172.21.15.62:0/348740679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3858102ee0 msgr2=0x7f3858195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f3857fff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3858102ee0 0x7f3858195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f3857fff700 1 -- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3858105800 msgr2=0x7f385819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:58.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f3857fff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3858105800 0x7f385819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.012+0000 7f3857fff700 1 -- 172.21.15.62:0/348740679 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3850009d00 con 0x7f38581005c0 2023-07-14T16:26:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.013+0000 7f3857fff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38581005c0 0x7f3858195820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f384800e9a0 tx=0x7f384800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.013+0000 7f38557fa700 1 -- 172.21.15.62:0/348740679 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848009790 con 0x7f38581005c0 2023-07-14T16:26:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.013+0000 7f385e7ff700 1 -- 172.21.15.62:0/348740679 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385819d950 con 0x7f38581005c0 2023-07-14T16:26:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.013+0000 7f38557fa700 1 -- 172.21.15.62:0/348740679 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3848010430 con 0x7f38581005c0 2023-07-14T16:26:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.013+0000 7f38557fa700 1 -- 172.21.15.62:0/348740679 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800cea0 con 0x7f38581005c0 2023-07-14T16:26:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.013+0000 7f385e7ff700 1 -- 172.21.15.62:0/348740679 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385819deb0 con 0x7f38581005c0 2023-07-14T16:26:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.018+0000 7f38557fa700 1 -- 172.21.15.62:0/348740679 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f38480105a0 con 0x7f38581005c0 2023-07-14T16:26:58.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.020+0000 7f38557fa700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38400a1150 0x7f38400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.020+0000 7f38557fa700 1 -- 172.21.15.62:0/348740679 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3848014070 con 0x7f38581005c0 2023-07-14T16:26:58.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.020+0000 7f387205fb80 1 Processor -- start 2023-07-14T16:26:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.021+0000 7f38577fe700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38400a1150 0x7f38400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.021+0000 7f387205fb80 1 -- start start 2023-07-14T16:26:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.021+0000 7f38577fe700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38400a1150 0x7f38400a3600 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f384c009c60 tx=0x7f384c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f387205fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f387205fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 0x55bcdff0eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f387205fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff0f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:58.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f387205fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdfe0acb0 con 0x55bcdff054d0 2023-07-14T16:26:58.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f387205fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdfe5da40 con 0x55bcdff05e20 2023-07-14T16:26:58.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f387205fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdfe56ed0 con 0x55bcdff0f480 2023-07-14T16:26:58.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f3857fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:58.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f3857fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40440/0 (socket says 172.21.15.62:40440) 2023-07-14T16:26:58.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f3857fff700 1 -- 172.21.15.62:0/1498316861 learned_addr learned my addr 172.21.15.62:0/1498316861 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:58.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f38577fe700 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 0x55bcdff0eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f385cd9c700 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff0f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f3857fff700 1 -- 172.21.15.62:0/1498316861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 msgr2=0x55bcdff0f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f3857fff700 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff0f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f3857fff700 1 -- 172.21.15.62:0/1498316861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 msgr2=0x55bcdff0eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f3857fff700 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 0x55bcdff0eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.022+0000 7f3857fff700 1 -- 172.21.15.62:0/1498316861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdfc84de0 con 0x55bcdff054d0 2023-07-14T16:26:59.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.023+0000 7f3857fff700 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff058e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3848096470 tx=0x7f38480999a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e93ddc641670aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.023+0000 7f383e7fc700 1 -- 172.21.15.62:0/1498316861 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384809dfa0 con 0x55bcdff054d0 2023-07-14T16:26:59.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.023+0000 7f383e7fc700 1 -- 172.21.15.62:0/1498316861 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38480a23c0 con 0x55bcdff054d0 2023-07-14T16:26:59.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.023+0000 7f383e7fc700 1 -- 172.21.15.62:0/1498316861 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480a2170 con 0x55bcdff054d0 2023-07-14T16:26:59.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.024+0000 7f387205fb80 1 -- 172.21.15.62:0/1498316861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 msgr2=0x55bcdff058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.024+0000 7f387205fb80 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff058e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3848096470 tx=0x7f38480999a0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.024+0000 7f387205fb80 1 -- 172.21.15.62:0/1498316861 shutdown_connections 2023-07-14T16:26:59.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.024+0000 7f387205fb80 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff0f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.025+0000 7f387205fb80 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff058e0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.025+0000 7f387205fb80 1 --2- 172.21.15.62:0/1498316861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 0x55bcdff0eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.025+0000 7f387205fb80 1 -- 172.21.15.62:0/1498316861 >> 172.21.15.62:0/1498316861 conn(0x55bcdfefada0 msgr2=0x55bcdff001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.025+0000 7f387205fb80 1 -- 172.21.15.62:0/1498316861 shutdown_connections 2023-07-14T16:26:59.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.025+0000 7f387205fb80 1 -- 172.21.15.62:0/1498316861 wait complete. 2023-07-14T16:26:59.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.025+0000 7f387205fb80 1 -- start start 2023-07-14T16:26:59.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.026+0000 7f387205fb80 1 Processor -- start 2023-07-14T16:26:59.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.026+0000 7f387205fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff17960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.026+0000 7f387205fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 0x55bcdff17ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.026+0000 7f387205fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff1eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.026+0000 7f387205fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdfdf6ac0 con 0x55bcdff054d0 2023-07-14T16:26:59.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.026+0000 7f387205fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdfdf67c0 con 0x55bcdff05e20 2023-07-14T16:26:59.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.026+0000 7f387205fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdfcda920 con 0x55bcdff0f480 2023-07-14T16:26:59.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.026+0000 7f387205fb80 10 client.0 _ll_register_callbacks cb 0x55bcdfdf4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:59.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff17960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff17960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40456/0 (socket says 172.21.15.62:40456) 2023-07-14T16:26:59.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 -- 172.21.15.62:0/298944690 learned_addr learned my addr 172.21.15.62:0/298944690 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f385cd9c700 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff1eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f38577fe700 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 0x55bcdff17ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 -- 172.21.15.62:0/298944690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 msgr2=0x55bcdff1eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff1eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 -- 172.21.15.62:0/298944690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 msgr2=0x55bcdff17ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 0x55bcdff17ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 -- 172.21.15.62:0/298944690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385819deb0 con 0x55bcdff054d0 2023-07-14T16:26:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f3857fff700 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff17960 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3848096420 tx=0x7f384809a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f087d84f0ccc3a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.027+0000 7f38377fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f38377fe700 1 -- 172.21.15.62:0/298944690 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384809e020 con 0x55bcdff054d0 2023-07-14T16:26:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f387205fb80 1 -- 172.21.15.62:0/298944690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcdfdb04f0 con 0x55bcdff054d0 2023-07-14T16:26:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f38377fe700 1 -- 172.21.15.62:0/298944690 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f38480b82c0 con 0x55bcdff054d0 2023-07-14T16:26:59.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f38377fe700 1 -- 172.21.15.62:0/298944690 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38480ca2b0 con 0x55bcdff054d0 2023-07-14T16:26:59.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f387205fb80 10 client.18663 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:59.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f387205fb80 1 -- 172.21.15.62:0/298944690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bcdfc6dc60 con 0x55bcdff054d0 2023-07-14T16:26:59.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f38377fe700 1 -- 172.21.15.62:0/298944690 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bcdfdb04f0 con 0x55bcdff054d0 2023-07-14T16:26:59.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f38377fe700 1 -- 172.21.15.62:0/298944690 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38480cab20 con 0x55bcdff054d0 2023-07-14T16:26:59.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f387205fb80 10 client.18663 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:59.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f387205fb80 10 client.18663 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:26:59.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.028+0000 7f387205fb80 20 client.18663 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:59.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.029+0000 7f387205fb80 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bcdff23440 0x55bcdff25850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.029+0000 7f387205fb80 4 client.18663 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:59.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.029+0000 7f387205fb80 1 -- 172.21.15.62:0/298944690 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcdfe4feb0 con 0x55bcdff23440 2023-07-14T16:26:59.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.029+0000 7f38577fe700 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bcdff23440 0x55bcdff25850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.030+0000 7f38377fe700 10 client.18663 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:26:59.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.030+0000 7f38577fe700 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bcdff23440 0x55bcdff25850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5fb7acaaa89d57b server_cookie=ea3bff4968fe2a26 in_seq=0 out_seq=0 2023-07-14T16:26:59.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f38377fe700 1 -- 172.21.15.62:0/298944690 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bcdfe4feb0 con 0x55bcdff23440 2023-07-14T16:26:59.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f38377fe700 10 client.18663 handle_command_reply: tid=0 2023-07-14T16:26:59.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 client.18663 shutdown 2023-07-14T16:26:59.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 -- 172.21.15.62:0/298944690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 msgr2=0x55bcdff17960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff17960 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3848096420 tx=0x7f384809a030 comp rx=0 tx=0).stop 2023-07-14T16:26:59.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 -- 172.21.15.62:0/298944690 shutdown_connections 2023-07-14T16:26:59.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bcdff23440 0x55bcdff25850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff1eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff054d0 0x55bcdff17960 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 --2- 172.21.15.62:0/298944690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff05e20 0x55bcdff17ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f38377fe700 0 client.18663 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:26:59.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 -- 172.21.15.62:0/298944690 >> 172.21.15.62:0/298944690 conn(0x55bcdfefada0 msgr2=0x55bcdfeff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 -- 172.21.15.62:0/298944690 shutdown_connections 2023-07-14T16:26:59.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.031+0000 7f387205fb80 1 -- 172.21.15.62:0/298944690 wait complete. 2023-07-14T16:26:59.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.032+0000 7f387205fb80 20 client.18663 trim_cache size 0 max 16384 2023-07-14T16:26:59.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.045+0000 7f387205fb80 1 Processor -- start 2023-07-14T16:26:59.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.045+0000 7f387205fb80 1 -- start start 2023-07-14T16:26:59.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.045+0000 7f387205fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff17010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.045+0000 7f387205fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.045+0000 7f387205fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff2a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.045+0000 7f387205fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdfefcec0 con 0x55bcdff05e20 2023-07-14T16:26:59.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f387205fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdff16e10 con 0x55bcdff054d0 2023-07-14T16:26:59.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f387205fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdff012e0 con 0x55bcdff0f480 2023-07-14T16:26:59.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f38577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f3857fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff17010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f3857fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff17010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59092/0 (socket says 172.21.15.62:59092) 2023-07-14T16:26:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f3857fff700 1 -- 172.21.15.62:0/2635971369 learned_addr learned my addr 172.21.15.62:0/2635971369 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f385cd9c700 1 --2- 172.21.15.62:0/2635971369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff2a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f3857fff700 1 -- 172.21.15.62:0/2635971369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 msgr2=0x55bcdff2a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f3857fff700 1 --2- 172.21.15.62:0/2635971369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff2a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f3857fff700 1 -- 172.21.15.62:0/2635971369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 msgr2=0x55bcdff1eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f3857fff700 1 --2- 172.21.15.62:0/2635971369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.046+0000 7f3857fff700 1 -- 172.21.15.62:0/2635971369 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdfcab250 con 0x55bcdff054d0 2023-07-14T16:26:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.047+0000 7f3857fff700 1 --2- 172.21.15.62:0/2635971369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff17010 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f38480982c0 tx=0x7f3848099dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55332f3551f563f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.047+0000 7f383effd700 1 -- 172.21.15.62:0/2635971369 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480960c0 con 0x55bcdff054d0 2023-07-14T16:26:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.047+0000 7f383effd700 1 -- 172.21.15.62:0/2635971369 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3848095e80 con 0x55bcdff054d0 2023-07-14T16:26:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.047+0000 7f383effd700 1 -- 172.21.15.62:0/2635971369 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480ca530 con 0x55bcdff054d0 2023-07-14T16:26:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 -- 172.21.15.62:0/2635971369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 msgr2=0x55bcdff17010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 --2- 172.21.15.62:0/2635971369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff17010 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f38480982c0 tx=0x7f3848099dc0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 -- 172.21.15.62:0/2635971369 shutdown_connections 2023-07-14T16:26:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 --2- 172.21.15.62:0/2635971369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff2a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 --2- 172.21.15.62:0/2635971369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff17010 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 --2- 172.21.15.62:0/2635971369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 -- 172.21.15.62:0/2635971369 >> 172.21.15.62:0/2635971369 conn(0x55bcdfefada0 msgr2=0x55bcdff001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 -- 172.21.15.62:0/2635971369 shutdown_connections 2023-07-14T16:26:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 -- 172.21.15.62:0/2635971369 wait complete. 2023-07-14T16:26:59.031 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:58 smithi164 ceph-mon[97427]: pgmap v697: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:26:59.031 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:26:58 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/113027720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:26:59.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 -- start start 2023-07-14T16:26:59.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.048+0000 7f387205fb80 1 Processor -- start 2023-07-14T16:26:59.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f387205fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff1b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f387205fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f387205fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff1e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f387205fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdff1e590 con 0x55bcdff05e20 2023-07-14T16:26:59.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f387205fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdff1e700 con 0x55bcdff054d0 2023-07-14T16:26:59.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f387205fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdff1e8c0 con 0x55bcdff0f480 2023-07-14T16:26:59.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f387205fb80 10 client.0 _ll_register_callbacks cb 0x7f384c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:59.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f38577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f38577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40474/0 (socket says 172.21.15.62:40474) 2023-07-14T16:26:59.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f38577fe700 1 -- 172.21.15.62:0/30224985 learned_addr learned my addr 172.21.15.62:0/30224985 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f385cd9c700 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff1e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.049+0000 7f3857fff700 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff1b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38577fe700 1 -- 172.21.15.62:0/30224985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 msgr2=0x55bcdff1e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38577fe700 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff1e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38577fe700 1 -- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 msgr2=0x55bcdff1b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38577fe700 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff1b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38577fe700 1 -- 172.21.15.62:0/30224985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcdfc6dc60 con 0x55bcdff05e20 2023-07-14T16:26:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f3857fff700 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff1b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:26:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38577fe700 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1b570 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x55bcdff26130 tx=0x7f384c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe20dcbbc677d927 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38377fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38377fe700 1 -- 172.21.15.62:0/30224985 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c003a90 con 0x55bcdff05e20 2023-07-14T16:26:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f387205fb80 1 -- 172.21.15.62:0/30224985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38480a23c0 con 0x55bcdff05e20 2023-07-14T16:26:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.050+0000 7f38377fe700 1 -- 172.21.15.62:0/30224985 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f384c030070 con 0x55bcdff05e20 2023-07-14T16:26:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f38377fe700 1 -- 172.21.15.62:0/30224985 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f384c03b030 con 0x55bcdff05e20 2023-07-14T16:26:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f387205fb80 10 client.18669 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f387205fb80 1 -- 172.21.15.62:0/30224985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bcdfc832b0 con 0x55bcdff05e20 2023-07-14T16:26:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f38377fe700 1 -- 172.21.15.62:0/30224985 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38480a23c0 con 0x55bcdff05e20 2023-07-14T16:26:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f38377fe700 1 -- 172.21.15.62:0/30224985 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f384c01d8c0 con 0x55bcdff05e20 2023-07-14T16:26:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f387205fb80 10 client.18669 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f387205fb80 10 client.18669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:26:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f387205fb80 20 client.18669 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f387205fb80 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bcdff781d0 0x55bcdff7a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f387205fb80 4 client.18669 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:26:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.051+0000 7f387205fb80 1 -- 172.21.15.62:0/30224985 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bcdfe12050 con 0x55bcdff781d0 2023-07-14T16:26:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.052+0000 7f3857fff700 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bcdff781d0 0x55bcdff7a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.052+0000 7f38377fe700 10 client.18669 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:26:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.052+0000 7f3857fff700 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bcdff781d0 0x55bcdff7a5e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d2975af45354335 server_cookie=6eda9b84e8c7bd66 in_seq=0 out_seq=0 2023-07-14T16:26:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f38377fe700 1 -- 172.21.15.62:0/30224985 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bcdfe12050 con 0x55bcdff781d0 2023-07-14T16:26:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f38377fe700 10 client.18669 handle_command_reply: tid=0 2023-07-14T16:26:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 client.18669 shutdown 2023-07-14T16:26:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 -- 172.21.15.62:0/30224985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 msgr2=0x55bcdff1b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1b570 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x55bcdff26130 tx=0x7f384c0047c0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 -- 172.21.15.62:0/30224985 shutdown_connections 2023-07-14T16:26:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55bcdff781d0 0x55bcdff7a5e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bcdff0f480 0x55bcdff1e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bcdff054d0 0x55bcdff1b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 --2- 172.21.15.62:0/30224985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bcdff05e20 0x55bcdff1b570 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 -- 172.21.15.62:0/30224985 >> 172.21.15.62:0/30224985 conn(0x55bcdfefada0 msgr2=0x55bcdff00160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f38377fe700 0 client.18669 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:26:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 -- 172.21.15.62:0/30224985 shutdown_connections 2023-07-14T16:26:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.053+0000 7f387205fb80 1 -- 172.21.15.62:0/30224985 wait complete. 2023-07-14T16:26:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f387205fb80 20 client.18669 trim_cache size 0 max 16384 2023-07-14T16:26:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:26:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 -- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38400a1150 msgr2=0x7f38400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38400a1150 0x7f38400a3600 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f384c009c60 tx=0x7f384c009260 comp rx=0 tx=0).stop 2023-07-14T16:26:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 -- 172.21.15.62:0/348740679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38581005c0 msgr2=0x7f3858195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38581005c0 0x7f3858195820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f384800e9a0 tx=0x7f384800ecb0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 -- 172.21.15.62:0/348740679 shutdown_connections 2023-07-14T16:26:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3858105800 0x7f385819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38581005c0 0x7f3858195820 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38400a1150 0x7f38400a3600 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 --2- 172.21.15.62:0/348740679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3858102ee0 0x7f3858195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.054+0000 7f385e7ff700 1 -- 172.21.15.62:0/348740679 >> 172.21.15.62:0/348740679 conn(0x7f38580fa000 msgr2=0x7f38581012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.055+0000 7f385e7ff700 1 -- 172.21.15.62:0/348740679 shutdown_connections 2023-07-14T16:26:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.055+0000 7f385e7ff700 1 -- 172.21.15.62:0/348740679 wait complete. 2023-07-14T16:26:59.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:58 smithi062 ceph-mon[95543]: pgmap v697: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:26:59.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:26:58 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/113027720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:26:59.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:26:59.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:58 smithi081 ceph-mon[127795]: pgmap v697: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:26:59.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:26:58 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/113027720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:26:59.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.312+0000 7f894df9d700 1 -- 172.21.15.62:0/224817996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 msgr2=0x7f89480ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.312+0000 7f894df9d700 1 --2- 172.21.15.62:0/224817996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 0x7f89480ff9b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f894000b0a0 tx=0x7f894000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.312+0000 7f894df9d700 1 -- 172.21.15.62:0/224817996 shutdown_connections 2023-07-14T16:26:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.312+0000 7f894df9d700 1 --2- 172.21.15.62:0/224817996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 0x7f89480ff9b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.312+0000 7f894df9d700 1 --2- 172.21.15.62:0/224817996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89480fe240 0x7f89480fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.312+0000 7f894df9d700 1 --2- 172.21.15.62:0/224817996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89480feb90 0x7f89480ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.312+0000 7f894df9d700 1 -- 172.21.15.62:0/224817996 >> 172.21.15.62:0/224817996 conn(0x7f89480f9ed0 msgr2=0x7f89480fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.312+0000 7f894df9d700 1 -- 172.21.15.62:0/224817996 shutdown_connections 2023-07-14T16:26:59.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.313+0000 7f894df9d700 1 -- 172.21.15.62:0/224817996 wait complete. 2023-07-14T16:26:59.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.314+0000 7f894df9d700 1 Processor -- start 2023-07-14T16:26:59.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.314+0000 7f894df9d700 1 -- start start 2023-07-14T16:26:59.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.314+0000 7f894df9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89480fe240 0x7f8948197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.314+0000 7f894df9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89480feb90 0x7f8948197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.314+0000 7f894df9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 0x7f894819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.314+0000 7f894df9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894819f2e0 con 0x7f89480ff540 2023-07-14T16:26:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.314+0000 7f894df9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894819f420 con 0x7f89480fe240 2023-07-14T16:26:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.314+0000 7f894df9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894819f5b0 con 0x7f89480feb90 2023-07-14T16:26:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 0x7f894819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 0x7f894819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40494/0 (socket says 172.21.15.62:40494) 2023-07-14T16:26:59.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 -- 172.21.15.62:0/1651822438 learned_addr learned my addr 172.21.15.62:0/1651822438 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f8947fff700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89480feb90 0x7f8948197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894cf9b700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89480fe240 0x7f8948197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 -- 172.21.15.62:0/1651822438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89480feb90 msgr2=0x7f8948197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89480feb90 0x7f8948197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 -- 172.21.15.62:0/1651822438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89480fe240 msgr2=0x7f8948197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89480fe240 0x7f8948197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 -- 172.21.15.62:0/1651822438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8940009d00 con 0x7f89480ff540 2023-07-14T16:26:59.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894cf9b700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89480fe240 0x7f8948197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:59.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.315+0000 7f894d79c700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 0x7f894819ed10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f894000b780 tx=0x7f89400078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.316+0000 7f8945ffb700 1 -- 172.21.15.62:0/1651822438 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894000e040 con 0x7f89480ff540 2023-07-14T16:26:59.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.316+0000 7f8945ffb700 1 -- 172.21.15.62:0/1651822438 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8940004c50 con 0x7f89480ff540 2023-07-14T16:26:59.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.316+0000 7f894df9d700 1 -- 172.21.15.62:0/1651822438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894819f830 con 0x7f89480ff540 2023-07-14T16:26:59.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.316+0000 7f8945ffb700 1 -- 172.21.15.62:0/1651822438 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89400052a0 con 0x7f89480ff540 2023-07-14T16:26:59.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.316+0000 7f894df9d700 1 -- 172.21.15.62:0/1651822438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894819fcb0 con 0x7f89480ff540 2023-07-14T16:26:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.320+0000 7f8945ffb700 1 -- 172.21.15.62:0/1651822438 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8940005400 con 0x7f89480ff540 2023-07-14T16:26:59.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.321+0000 7f8962aadb80 1 Processor -- start 2023-07-14T16:26:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.321+0000 7f8962aadb80 1 -- start start 2023-07-14T16:26:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.321+0000 7f8945ffb700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f89380a1270 0x7f89380a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f8945ffb700 1 -- 172.21.15.62:0/1651822438 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f89400cded0 con 0x7f89480ff540 2023-07-14T16:26:59.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba1720 0x56393aba1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393aba28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393aba32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f8962aadb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393aaa6cb0 con 0x56393aba2e30 2023-07-14T16:26:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f8962aadb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393aaf9a40 con 0x56393aba1720 2023-07-14T16:26:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f8962aadb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393aaf2ed0 con 0x56393aba2070 2023-07-14T16:26:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f8947fff700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f89380a1270 0x7f89380a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f894d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393aba32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f894d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393aba32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40500/0 (socket says 172.21.15.62:40500) 2023-07-14T16:26:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f894d79c700 1 -- 172.21.15.62:0/2776047951 learned_addr learned my addr 172.21.15.62:0/2776047951 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.322+0000 7f894cf9b700 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba1720 0x56393aba1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f8947fff700 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393aba28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f894d79c700 1 -- 172.21.15.62:0/2776047951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 msgr2=0x56393aba28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f894d79c700 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393aba28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f894d79c700 1 -- 172.21.15.62:0/2776047951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba1720 msgr2=0x56393aba1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f894d79c700 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba1720 0x56393aba1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f894d79c700 1 -- 172.21.15.62:0/2776047951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393a920de0 con 0x56393aba2e30 2023-07-14T16:26:59.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f894cf9b700 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba1720 0x56393aba1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:59.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f8947fff700 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393aba28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:26:59.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f8947fff700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f89380a1270 0x7f89380a3720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8934009c70 tx=0x7f8934011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f894d79c700 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393aba32a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f894009b310 tx=0x7f894001dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5cf7e4c5ba35f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f8932ffd700 1 -- 172.21.15.62:0/2776047951 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89400cbea0 con 0x56393aba2e30 2023-07-14T16:26:59.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f8932ffd700 1 -- 172.21.15.62:0/2776047951 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f894009b380 con 0x56393aba2e30 2023-07-14T16:26:59.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.323+0000 7f8932ffd700 1 -- 172.21.15.62:0/2776047951 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940099720 con 0x56393aba2e30 2023-07-14T16:26:59.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 -- 172.21.15.62:0/2776047951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 msgr2=0x56393aba32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393aba32a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f894009b310 tx=0x7f894001dc10 comp rx=0 tx=0).stop 2023-07-14T16:26:59.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 -- 172.21.15.62:0/2776047951 shutdown_connections 2023-07-14T16:26:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393aba32a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba1720 0x56393aba1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 --2- 172.21.15.62:0/2776047951 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393aba28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 -- 172.21.15.62:0/2776047951 >> 172.21.15.62:0/2776047951 conn(0x56393ab96ff0 msgr2=0x56393ab9c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 -- 172.21.15.62:0/2776047951 shutdown_connections 2023-07-14T16:26:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.324+0000 7f8962aadb80 1 -- 172.21.15.62:0/2776047951 wait complete. 2023-07-14T16:26:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.325+0000 7f8962aadb80 1 -- start start 2023-07-14T16:26:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.325+0000 7f8962aadb80 1 Processor -- start 2023-07-14T16:26:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba1720 0x56393abab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 0x56393abab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393abb26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f8962aadb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393aa92ac0 con 0x56393aba2e30 2023-07-14T16:26:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f8962aadb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393aa927c0 con 0x56393aba2070 2023-07-14T16:26:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f8962aadb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393a976920 con 0x56393aba1720 2023-07-14T16:26:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f8962aadb80 10 client.0 _ll_register_callbacks cb 0x56393aa90760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f894d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393abb26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f894cf9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba1720 0x56393abab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f894cf9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba1720 0x56393abab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42440/0 (socket says 172.21.15.62:42440) 2023-07-14T16:26:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f894d79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393abb26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40512/0 (socket says 172.21.15.62:40512) 2023-07-14T16:26:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f894cf9b700 1 -- 172.21.15.62:0/332816067 learned_addr learned my addr 172.21.15.62:0/332816067 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.326+0000 7f8947fff700 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 0x56393abab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f894d79c700 1 -- 172.21.15.62:0/332816067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba1720 msgr2=0x56393abab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f894d79c700 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba1720 0x56393abab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f894d79c700 1 -- 172.21.15.62:0/332816067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 msgr2=0x56393abab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f894d79c700 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 0x56393abab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f894d79c700 1 -- 172.21.15.62:0/332816067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894819fcb0 con 0x56393aba2e30 2023-07-14T16:26:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f894cf9b700 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba1720 0x56393abab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f894d79c700 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393abb26a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f894009b310 tx=0x7f8940009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4351f55a663f45aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f8927fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f8927fff700 1 -- 172.21.15.62:0/332816067 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89400cbea0 con 0x56393aba2e30 2023-07-14T16:26:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f8962aadb80 1 -- 172.21.15.62:0/332816067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56393aa4c4f0 con 0x56393aba2e30 2023-07-14T16:26:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.327+0000 7f8927fff700 1 -- 172.21.15.62:0/332816067 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f894009f3b0 con 0x56393aba2e30 2023-07-14T16:26:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8927fff700 1 -- 172.21.15.62:0/332816067 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89400860b0 con 0x56393aba2e30 2023-07-14T16:26:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8962aadb80 10 client.18681 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8962aadb80 1 -- 172.21.15.62:0/332816067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56393a909c60 con 0x56393aba2e30 2023-07-14T16:26:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8927fff700 1 -- 172.21.15.62:0/332816067 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56393aa4c4f0 con 0x56393aba2e30 2023-07-14T16:26:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8927fff700 1 -- 172.21.15.62:0/332816067 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f89400b5d90 con 0x56393aba2e30 2023-07-14T16:26:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8962aadb80 10 client.18681 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8962aadb80 10 client.18681 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:26:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8962aadb80 20 client.18681 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8962aadb80 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56393abb6940 0x56393abb8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8962aadb80 4 client.18681 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.328+0000 7f8962aadb80 1 -- 172.21.15.62:0/332816067 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56393aaebeb0 con 0x56393abb6940 2023-07-14T16:26:59.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.329+0000 7f894cf9b700 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56393abb6940 0x56393abb8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.329+0000 7f894cf9b700 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56393abb6940 0x56393abb8d50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d3239f7a42f094a server_cookie=227090608e6bc405 in_seq=0 out_seq=0 2023-07-14T16:26:59.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.329+0000 7f8927fff700 10 client.18681 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:26:59.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8927fff700 1 -- 172.21.15.62:0/332816067 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56393aaebeb0 con 0x56393abb6940 2023-07-14T16:26:59.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8927fff700 10 client.18681 handle_command_reply: tid=0 2023-07-14T16:26:59.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 client.18681 shutdown 2023-07-14T16:26:59.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 -- 172.21.15.62:0/332816067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 msgr2=0x56393abb26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393abb26a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f894009b310 tx=0x7f8940009960 comp rx=0 tx=0).stop 2023-07-14T16:26:59.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 -- 172.21.15.62:0/332816067 shutdown_connections 2023-07-14T16:26:59.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56393abb6940 0x56393abb8d50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba2e30 0x56393abb26a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba1720 0x56393abab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 --2- 172.21.15.62:0/332816067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 0x56393abab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8927fff700 0 client.18681 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:26:59.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 -- 172.21.15.62:0/332816067 >> 172.21.15.62:0/332816067 conn(0x56393ab96ff0 msgr2=0x56393ab9bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 -- 172.21.15.62:0/332816067 shutdown_connections 2023-07-14T16:26:59.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.330+0000 7f8962aadb80 1 -- 172.21.15.62:0/332816067 wait complete. 2023-07-14T16:26:59.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.331+0000 7f8962aadb80 20 client.18681 trim_cache size 0 max 16384 2023-07-14T16:26:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.348+0000 7f8962aadb80 1 Processor -- start 2023-07-14T16:26:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.348+0000 7f8962aadb80 1 -- start start 2023-07-14T16:26:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393aba5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 0x56393aba6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2e30 0x56393abb2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f8962aadb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393ab9d6a0 con 0x56393aba1720 2023-07-14T16:26:59.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f8962aadb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393abaa5d0 con 0x56393aba2070 2023-07-14T16:26:59.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f8962aadb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393ab96e00 con 0x56393aba2e30 2023-07-14T16:26:59.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894cf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393aba5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894cf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393aba5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40518/0 (socket says 172.21.15.62:40518) 2023-07-14T16:26:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894cf9b700 1 -- 172.21.15.62:0/3256527840 learned_addr learned my addr 172.21.15.62:0/3256527840 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894d79c700 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2e30 0x56393abb2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f8947fff700 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 0x56393aba6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894cf9b700 1 -- 172.21.15.62:0/3256527840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2e30 msgr2=0x56393abb2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894cf9b700 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2e30 0x56393abb2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894cf9b700 1 -- 172.21.15.62:0/3256527840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 msgr2=0x56393aba6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894cf9b700 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 0x56393aba6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894cf9b700 1 -- 172.21.15.62:0/3256527840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393a947250 con 0x56393aba1720 2023-07-14T16:26:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.349+0000 7f894d79c700 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2e30 0x56393abb2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:59.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.350+0000 7f894cf9b700 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393aba5d30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x56393abb95f0 tx=0x7f893c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d2740f563bef97 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.350+0000 7f89337fe700 1 -- 172.21.15.62:0/3256527840 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c018040 con 0x56393aba1720 2023-07-14T16:26:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.350+0000 7f89337fe700 1 -- 172.21.15.62:0/3256527840 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f893c0133d0 con 0x56393aba1720 2023-07-14T16:26:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.350+0000 7f89337fe700 1 -- 172.21.15.62:0/3256527840 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c013d20 con 0x56393aba1720 2023-07-14T16:26:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 -- 172.21.15.62:0/3256527840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 msgr2=0x56393aba5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393aba5d30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x56393abb95f0 tx=0x7f893c000bc0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 -- 172.21.15.62:0/3256527840 shutdown_connections 2023-07-14T16:26:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2e30 0x56393abb2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393aba5d30 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3256527840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2070 0x56393aba6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 -- 172.21.15.62:0/3256527840 >> 172.21.15.62:0/3256527840 conn(0x56393ab96ff0 msgr2=0x56393aba9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 -- 172.21.15.62:0/3256527840 shutdown_connections 2023-07-14T16:26:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 -- 172.21.15.62:0/3256527840 wait complete. 2023-07-14T16:26:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.351+0000 7f8962aadb80 1 -- start start 2023-07-14T16:26:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.352+0000 7f8962aadb80 1 Processor -- start 2023-07-14T16:26:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.352+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393abb1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.352+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393abb24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8962aadb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2e30 0x56393abaea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8962aadb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393abaf0e0 con 0x56393aba1720 2023-07-14T16:26:59.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8962aadb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393abaf250 con 0x56393aba2e30 2023-07-14T16:26:59.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8962aadb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393abaf410 con 0x56393aba2070 2023-07-14T16:26:59.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8962aadb80 10 client.0 _ll_register_callbacks cb 0x7f894009b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:59.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8947fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393abb24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8947fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393abb24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42454/0 (socket says 172.21.15.62:42454) 2023-07-14T16:26:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8947fff700 1 -- 172.21.15.62:0/3726857024 learned_addr learned my addr 172.21.15.62:0/3726857024 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f894cf9b700 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393abb1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f894d79c700 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2e30 0x56393abaea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8947fff700 1 -- 172.21.15.62:0/3726857024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2e30 msgr2=0x56393abaea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8947fff700 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2e30 0x56393abaea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8947fff700 1 -- 172.21.15.62:0/3726857024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 msgr2=0x56393abb1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8947fff700 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393abb1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.353+0000 7f8947fff700 1 -- 172.21.15.62:0/3726857024 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c013040 con 0x56393aba2070 2023-07-14T16:26:59.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.354+0000 7f8947fff700 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393abb24e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f893400fb30 tx=0x7f89340047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f094c9d9ae50f1a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.354+0000 7f8927fff700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:26:59.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.354+0000 7f8927fff700 1 -- 172.21.15.62:0/3726857024 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934025070 con 0x56393aba2070 2023-07-14T16:26:59.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.354+0000 7f8962aadb80 1 -- 172.21.15.62:0/3726857024 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f894009b380 con 0x56393aba2070 2023-07-14T16:26:59.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.354+0000 7f8927fff700 1 -- 172.21.15.62:0/3726857024 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8934004830 con 0x56393aba2070 2023-07-14T16:26:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.354+0000 7f8927fff700 1 -- 172.21.15.62:0/3726857024 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89340049f0 con 0x56393aba2070 2023-07-14T16:26:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.354+0000 7f8962aadb80 10 client.28174 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.354+0000 7f8962aadb80 1 -- 172.21.15.62:0/3726857024 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56393a91f2b0 con 0x56393aba2070 2023-07-14T16:26:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f8927fff700 1 -- 172.21.15.62:0/3726857024 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f894009b380 con 0x56393aba2070 2023-07-14T16:26:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f8927fff700 1 -- 172.21.15.62:0/3726857024 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f893401f6e0 con 0x56393aba2070 2023-07-14T16:26:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f8962aadb80 10 client.28174 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:59.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f8962aadb80 10 client.28174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:26:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f8962aadb80 20 client.28174 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56393ac0bea0 0x56393ac0e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f8962aadb80 4 client.28174 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:26:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f8962aadb80 1 -- 172.21.15.62:0/3726857024 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56393aaae050 con 0x56393ac0bea0 2023-07-14T16:26:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.355+0000 7f894cf9b700 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56393ac0bea0 0x56393ac0e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.356+0000 7f8927fff700 10 client.28174 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:26:59.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.356+0000 7f894cf9b700 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56393ac0bea0 0x56393ac0e2b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15c7293835aa2bd3 server_cookie=cf61b1276fe7d121 in_seq=0 out_seq=0 2023-07-14T16:26:59.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.356+0000 7f8927fff700 1 -- 172.21.15.62:0/3726857024 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56393aaae050 con 0x56393ac0bea0 2023-07-14T16:26:59.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.356+0000 7f8927fff700 10 client.28174 handle_command_reply: tid=0 2023-07-14T16:26:59.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.356+0000 7f8962aadb80 1 client.28174 shutdown 2023-07-14T16:26:59.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 -- 172.21.15.62:0/3726857024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 msgr2=0x56393abb24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393abb24e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f893400fb30 tx=0x7f89340047c0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 -- 172.21.15.62:0/3726857024 shutdown_connections 2023-07-14T16:26:59.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56393ac0bea0 0x56393ac0e2b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56393aba2e30 0x56393abaea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56393aba1720 0x56393abb1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8927fff700 0 client.28174 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:26:59.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 --2- 172.21.15.62:0/3726857024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56393aba2070 0x56393abb24e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 -- 172.21.15.62:0/3726857024 >> 172.21.15.62:0/3726857024 conn(0x56393ab96ff0 msgr2=0x56393aba5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 -- 172.21.15.62:0/3726857024 shutdown_connections 2023-07-14T16:26:59.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 1 -- 172.21.15.62:0/3726857024 wait complete. 2023-07-14T16:26:59.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.357+0000 7f8962aadb80 20 client.28174 trim_cache size 0 max 16384 2023-07-14T16:26:59.339 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:26:59.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 -- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f89380a1270 msgr2=0x7f89380a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f89380a1270 0x7f89380a3720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8934009c70 tx=0x7f8934011720 comp rx=0 tx=0).stop 2023-07-14T16:26:59.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 -- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 msgr2=0x7f894819ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 0x7f894819ed10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f894000b780 tx=0x7f89400078e0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 -- 172.21.15.62:0/1651822438 shutdown_connections 2023-07-14T16:26:59.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89480ff540 0x7f894819ed10 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f89380a1270 0x7f89380a3720 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89480fe240 0x7f8948197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 --2- 172.21.15.62:0/1651822438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89480feb90 0x7f8948197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 -- 172.21.15.62:0/1651822438 >> 172.21.15.62:0/1651822438 conn(0x7f89480f9ed0 msgr2=0x7f89480fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 -- 172.21.15.62:0/1651822438 shutdown_connections 2023-07-14T16:26:59.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.358+0000 7f894df9d700 1 -- 172.21.15.62:0/1651822438 wait complete. 2023-07-14T16:26:59.353 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:26:59.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.608+0000 7f46ca79d700 1 -- 172.21.15.62:0/2090262666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 msgr2=0x7f46c4107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.608+0000 7f46ca79d700 1 --2- 172.21.15.62:0/2090262666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 0x7f46c4107af0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b0a0 tx=0x7f46bc00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.609+0000 7f46ca79d700 1 -- 172.21.15.62:0/2090262666 shutdown_connections 2023-07-14T16:26:59.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.609+0000 7f46ca79d700 1 --2- 172.21.15.62:0/2090262666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 0x7f46c4107af0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.609+0000 7f46ca79d700 1 --2- 172.21.15.62:0/2090262666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46c4100490 0x7f46c4102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.609+0000 7f46ca79d700 1 --2- 172.21.15.62:0/2090262666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46c4102db0 0x7f46c4105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.609+0000 7f46ca79d700 1 -- 172.21.15.62:0/2090262666 >> 172.21.15.62:0/2090262666 conn(0x7f46c40f9ed0 msgr2=0x7f46c40fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.609+0000 7f46ca79d700 1 -- 172.21.15.62:0/2090262666 shutdown_connections 2023-07-14T16:26:59.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.609+0000 7f46ca79d700 1 -- 172.21.15.62:0/2090262666 wait complete. 2023-07-14T16:26:59.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.610+0000 7f46ca79d700 1 Processor -- start 2023-07-14T16:26:59.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.610+0000 7f46ca79d700 1 -- start start 2023-07-14T16:26:59.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46ca79d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46c4100490 0x7f46c4195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46ca79d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46c4102db0 0x7f46c4195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46ca79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 0x7f46c419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46ca79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c419d100 con 0x7f46c41056d0 2023-07-14T16:26:59.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46ca79d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c419d240 con 0x7f46c4100490 2023-07-14T16:26:59.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46ca79d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c419d3d0 con 0x7f46c4102db0 2023-07-14T16:26:59.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46c9f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 0x7f46c419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46c9f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 0x7f46c419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40546/0 (socket says 172.21.15.62:40546) 2023-07-14T16:26:59.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46c979b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46c4100490 0x7f46c4195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46c9f9c700 1 -- 172.21.15.62:0/1463611457 learned_addr learned my addr 172.21.15.62:0/1463611457 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.611+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46c4102db0 0x7f46c4195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46c9f9c700 1 -- 172.21.15.62:0/1463611457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46c4102db0 msgr2=0x7f46c4195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46c4102db0 0x7f46c4195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46c9f9c700 1 -- 172.21.15.62:0/1463611457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46c4100490 msgr2=0x7f46c4195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46c4100490 0x7f46c4195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46c9f9c700 1 -- 172.21.15.62:0/1463611457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b4009520 con 0x7f46c41056d0 2023-07-14T16:26:59.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46c979b700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46c4100490 0x7f46c4195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:59.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 0x7f46c419cb30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00d010 tx=0x7f46bc00bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46ba7fc700 1 -- 172.21.15.62:0/1463611457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00e040 con 0x7f46c41056d0 2023-07-14T16:26:59.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46ca79d700 1 -- 172.21.15.62:0/1463611457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc009d00 con 0x7f46c41056d0 2023-07-14T16:26:59.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46ba7fc700 1 -- 172.21.15.62:0/1463611457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46bc009480 con 0x7f46c41056d0 2023-07-14T16:26:59.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.612+0000 7f46ba7fc700 1 -- 172.21.15.62:0/1463611457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0052a0 con 0x7f46c41056d0 2023-07-14T16:26:59.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.613+0000 7f46ca79d700 1 -- 172.21.15.62:0/1463611457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c40fe510 con 0x7f46c41056d0 2023-07-14T16:26:59.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.618+0000 7f46df338b80 1 Processor -- start 2023-07-14T16:26:59.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.618+0000 7f46ba7fc700 1 -- 172.21.15.62:0/1463611457 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f46bc005400 con 0x7f46c41056d0 2023-07-14T16:26:59.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.618+0000 7f46df338b80 1 -- start start 2023-07-14T16:26:59.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.618+0000 7f46df338b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.618+0000 7f46df338b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ac830 0x556ea43ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.618+0000 7f46df338b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46df338b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea42a8cb0 con 0x556ea43ac830 2023-07-14T16:26:59.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46df338b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea42fba40 con 0x556ea43abee0 2023-07-14T16:26:59.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46df338b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea42f4ed0 con 0x556ea43ad660 2023-07-14T16:26:59.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46c979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ac830 0x556ea43ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46c979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ac830 0x556ea43ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40550/0 (socket says 172.21.15.62:40550) 2023-07-14T16:26:59.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46c979b700 1 -- 172.21.15.62:0/2776872521 learned_addr learned my addr 172.21.15.62:0/2776872521 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46c8f9a700 1 -- 172.21.15.62:0/2776872521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 msgr2=0x556ea43adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.619+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46c8f9a700 1 -- 172.21.15.62:0/2776872521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ac830 msgr2=0x556ea43ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ac830 0x556ea43ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46c8f9a700 1 -- 172.21.15.62:0/2776872521 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea4122de0 con 0x556ea43abee0 2023-07-14T16:26:59.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46c979b700 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ac830 0x556ea43ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:59.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43ac2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f46b4011970 tx=0x7f46b4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d79b1422ae2528d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46af7fe700 1 -- 172.21.15.62:0/2776872521 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4004050 con 0x556ea43abee0 2023-07-14T16:26:59.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46ba7fc700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46b00a1270 0x7f46b00a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46af7fe700 1 -- 172.21.15.62:0/2776872521 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46b4004830 con 0x556ea43abee0 2023-07-14T16:26:59.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46af7fe700 1 -- 172.21.15.62:0/2776872521 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4005280 con 0x556ea43abee0 2023-07-14T16:26:59.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.620+0000 7f46ba7fc700 1 -- 172.21.15.62:0/1463611457 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f46bc0cdc40 con 0x7f46c41056d0 2023-07-14T16:26:59.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46c979b700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46b00a1270 0x7f46b00a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46df338b80 1 -- 172.21.15.62:0/2776872521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 msgr2=0x556ea43ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46df338b80 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43ac2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f46b4011970 tx=0x7f46b4009800 comp rx=0 tx=0).stop 2023-07-14T16:26:59.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46c979b700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46b00a1270 0x7f46b00a3720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x556ea43acc40 tx=0x7f46c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46df338b80 1 -- 172.21.15.62:0/2776872521 shutdown_connections 2023-07-14T16:26:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46df338b80 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46df338b80 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43ac2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46df338b80 1 --2- 172.21.15.62:0/2776872521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ac830 0x556ea43ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.621+0000 7f46df338b80 1 -- 172.21.15.62:0/2776872521 >> 172.21.15.62:0/2776872521 conn(0x556ea4398ff0 msgr2=0x556ea439e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.622+0000 7f46df338b80 1 -- 172.21.15.62:0/2776872521 shutdown_connections 2023-07-14T16:26:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.622+0000 7f46df338b80 1 -- 172.21.15.62:0/2776872521 wait complete. 2023-07-14T16:26:59.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.622+0000 7f46df338b80 1 -- start start 2023-07-14T16:26:59.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.623+0000 7f46df338b80 1 Processor -- start 2023-07-14T16:26:59.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46df338b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43abee0 0x556ea43b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46df338b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 0x556ea43b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46df338b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46df338b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea4294ac0 con 0x556ea43abee0 2023-07-14T16:26:59.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46df338b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea42947c0 con 0x556ea43ac830 2023-07-14T16:26:59.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46df338b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea4178920 con 0x556ea43ad660 2023-07-14T16:26:59.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46df338b80 10 client.0 _ll_register_callbacks cb 0x556ea4292760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:59.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46c979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43abee0 0x556ea43b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.624+0000 7f46c979b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43abee0 0x556ea43b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40554/0 (socket says 172.21.15.62:40554) 2023-07-14T16:26:59.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.625+0000 7f46c979b700 1 -- 172.21.15.62:0/1996226072 learned_addr learned my addr 172.21.15.62:0/1996226072 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.625+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.625+0000 7f46c979b700 1 -- 172.21.15.62:0/1996226072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 msgr2=0x556ea43bd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.625+0000 7f46c979b700 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43bd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.625+0000 7f46c979b700 1 -- 172.21.15.62:0/1996226072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 msgr2=0x556ea43b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.625+0000 7f46c979b700 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 0x556ea43b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.625+0000 7f46c979b700 1 -- 172.21.15.62:0/1996226072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c0009dd0 con 0x556ea43abee0 2023-07-14T16:26:59.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.625+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:59.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.626+0000 7f46c979b700 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43abee0 0x556ea43b5bf0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f46c000f830 tx=0x7f46c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db48f5e7b8f2cf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.626+0000 7f46acff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:26:59.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.626+0000 7f46acff9700 1 -- 172.21.15.62:0/1996226072 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0017070 con 0x556ea43abee0 2023-07-14T16:26:59.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.626+0000 7f46df338b80 1 -- 172.21.15.62:0/1996226072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ea424e4f0 con 0x556ea43abee0 2023-07-14T16:26:59.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.626+0000 7f46acff9700 1 -- 172.21.15.62:0/1996226072 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f46c0004240 con 0x556ea43abee0 2023-07-14T16:26:59.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.626+0000 7f46acff9700 1 -- 172.21.15.62:0/1996226072 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c002f030 con 0x556ea43abee0 2023-07-14T16:26:59.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.626+0000 7f46df338b80 10 client.18699 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:59.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.626+0000 7f46df338b80 1 -- 172.21.15.62:0/1996226072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f46b4019040 con 0x556ea43abee0 2023-07-14T16:26:59.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.627+0000 7f46acff9700 1 -- 172.21.15.62:0/1996226072 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ea424e4f0 con 0x556ea43abee0 2023-07-14T16:26:59.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.627+0000 7f46acff9700 1 -- 172.21.15.62:0/1996226072 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f46c0011660 con 0x556ea43abee0 2023-07-14T16:26:59.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.627+0000 7f46df338b80 10 client.18699 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:59.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.627+0000 7f46df338b80 10 client.18699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:59.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.627+0000 7f46df338b80 20 client.18699 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:59.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.627+0000 7f46df338b80 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556ea43c1ad0 0x556ea43c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.627+0000 7f46df338b80 4 client.18699 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:26:59.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.627+0000 7f46df338b80 1 -- 172.21.15.62:0/1996226072 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ea42edeb0 con 0x556ea43c1ad0 2023-07-14T16:26:59.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.629+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556ea43c1ad0 0x556ea43c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.631+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556ea43c1ad0 0x556ea43c3ee0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff80a7b2b11a33d server_cookie=59246549e45b876d in_seq=0 out_seq=0 2023-07-14T16:26:59.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.631+0000 7f46acff9700 10 client.18699 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:59.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46acff9700 1 -- 172.21.15.62:0/1996226072 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ea42edeb0 con 0x556ea43c1ad0 2023-07-14T16:26:59.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46acff9700 10 client.18699 handle_command_reply: tid=0 2023-07-14T16:26:59.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 client.18699 shutdown 2023-07-14T16:26:59.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 -- 172.21.15.62:0/1996226072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43abee0 msgr2=0x556ea43b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43abee0 0x556ea43b5bf0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f46c000f830 tx=0x7f46c00047c0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 -- 172.21.15.62:0/1996226072 shutdown_connections 2023-07-14T16:26:59.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ad660 0x556ea43bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43abee0 0x556ea43b5bf0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556ea43c1ad0 0x556ea43c3ee0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 --2- 172.21.15.62:0/1996226072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 0x556ea43b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46acff9700 0 client.18699 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 -- 172.21.15.62:0/1996226072 >> 172.21.15.62:0/1996226072 conn(0x556ea4398ff0 msgr2=0x556ea439db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 -- 172.21.15.62:0/1996226072 shutdown_connections 2023-07-14T16:26:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.632+0000 7f46df338b80 1 -- 172.21.15.62:0/1996226072 wait complete. 2023-07-14T16:26:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.633+0000 7f46df338b80 20 client.18699 trim_cache size 0 max 16384 2023-07-14T16:26:59.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.650+0000 7f46df338b80 1 Processor -- start 2023-07-14T16:26:59.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46df338b80 1 -- start start 2023-07-14T16:26:59.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46df338b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43abee0 0x556ea43b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46df338b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 0x556ea43bd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46df338b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46df338b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea43961d0 con 0x556ea43ad660 2023-07-14T16:26:59.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46df338b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea439f790 con 0x556ea43ac830 2023-07-14T16:26:59.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46df338b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea43b4580 con 0x556ea43abee0 2023-07-14T16:26:59.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46c9f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46c9f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40558/0 (socket says 172.21.15.62:40558) 2023-07-14T16:26:59.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46c9f9c700 1 -- 172.21.15.62:0/1969458050 learned_addr learned my addr 172.21.15.62:0/1969458050 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 0x556ea43bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.651+0000 7f46c979b700 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43abee0 0x556ea43b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46c9f9c700 1 -- 172.21.15.62:0/1969458050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43abee0 msgr2=0x556ea43b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43abee0 0x556ea43b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46c9f9c700 1 -- 172.21.15.62:0/1969458050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 msgr2=0x556ea43bd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 0x556ea43bd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46c9f9c700 1 -- 172.21.15.62:0/1969458050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea410bc60 con 0x556ea43ad660 2023-07-14T16:26:59.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46c979b700 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43abee0 0x556ea43b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:26:59.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43b3ab0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009fd0 tx=0x7f46bc09ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3239cacb977b7824 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46affff700 1 -- 172.21.15.62:0/1969458050 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc09b090 con 0x556ea43ad660 2023-07-14T16:26:59.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46affff700 1 -- 172.21.15.62:0/1969458050 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46bc09f0c0 con 0x556ea43ad660 2023-07-14T16:26:59.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.652+0000 7f46affff700 1 -- 172.21.15.62:0/1969458050 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc09b5a0 con 0x556ea43ad660 2023-07-14T16:26:59.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 -- 172.21.15.62:0/1969458050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 msgr2=0x556ea43b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43b3ab0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009fd0 tx=0x7f46bc09ac50 comp rx=0 tx=0).stop 2023-07-14T16:26:59.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 -- 172.21.15.62:0/1969458050 shutdown_connections 2023-07-14T16:26:59.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43b3ab0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43abee0 0x556ea43b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 --2- 172.21.15.62:0/1969458050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43ac830 0x556ea43bd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 -- 172.21.15.62:0/1969458050 >> 172.21.15.62:0/1969458050 conn(0x556ea4398ff0 msgr2=0x556ea43c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 -- 172.21.15.62:0/1969458050 shutdown_connections 2023-07-14T16:26:59.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.653+0000 7f46df338b80 1 -- 172.21.15.62:0/1969458050 wait complete. 2023-07-14T16:26:59.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.654+0000 7f46df338b80 1 -- start start 2023-07-14T16:26:59.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.654+0000 7f46df338b80 1 Processor -- start 2023-07-14T16:26:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46df338b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43bace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46df338b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ac830 0x556ea43bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46df338b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43bc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46df338b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea43bc770 con 0x556ea43ad660 2023-07-14T16:26:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46df338b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea43bc8e0 con 0x556ea43abee0 2023-07-14T16:26:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46df338b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea43bcaa0 con 0x556ea43ac830 2023-07-14T16:26:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46df338b80 10 client.0 _ll_register_callbacks cb 0x7f46c00123d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:26:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46c9f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43bc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46c8f9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ac830 0x556ea43bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46c979b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43bace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46c8f9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ac830 0x556ea43bb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42510/0 (socket says 172.21.15.62:42510) 2023-07-14T16:26:59.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.655+0000 7f46c8f9a700 1 -- 172.21.15.62:0/927977869 learned_addr learned my addr 172.21.15.62:0/927977869 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:26:59.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.656+0000 7f46c979b700 1 -- 172.21.15.62:0/927977869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ac830 msgr2=0x556ea43bb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.656+0000 7f46c979b700 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ac830 0x556ea43bb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.656+0000 7f46c979b700 1 -- 172.21.15.62:0/927977869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 msgr2=0x556ea43bc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:26:59.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.656+0000 7f46c979b700 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43bc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.656+0000 7f46c979b700 1 -- 172.21.15.62:0/927977869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c40fe510 con 0x556ea43abee0 2023-07-14T16:26:59.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.656+0000 7f46c9f9c700 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43bc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:26:59.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.656+0000 7f46c979b700 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43bace0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46c00119c0 tx=0x7f46c00296e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde1fecb88b8daef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:26:59.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.656+0000 7f46acff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:26:59.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.657+0000 7f46acff9700 1 -- 172.21.15.62:0/927977869 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c002f070 con 0x556ea43abee0 2023-07-14T16:26:59.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.657+0000 7f46acff9700 1 -- 172.21.15.62:0/927977869 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f46c0030070 con 0x556ea43abee0 2023-07-14T16:26:59.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.657+0000 7f46df338b80 1 -- 172.21.15.62:0/927977869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46b4004830 con 0x556ea43abee0 2023-07-14T16:26:59.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.657+0000 7f46acff9700 1 -- 172.21.15.62:0/927977869 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c0011640 con 0x556ea43abee0 2023-07-14T16:26:59.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.657+0000 7f46df338b80 10 client.28304 fetch_fsmap learned FSMap version 25 2023-07-14T16:26:59.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.657+0000 7f46df338b80 1 -- 172.21.15.62:0/927977869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ea4149250 con 0x556ea43abee0 2023-07-14T16:26:59.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.657+0000 7f46acff9700 1 -- 172.21.15.62:0/927977869 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46c0043020 con 0x556ea43abee0 2023-07-14T16:26:59.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.658+0000 7f46acff9700 1 -- 172.21.15.62:0/927977869 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f46c0003710 con 0x556ea43abee0 2023-07-14T16:26:59.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.658+0000 7f46df338b80 10 client.28304 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:26:59.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.658+0000 7f46df338b80 10 client.28304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:26:59.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.658+0000 7f46df338b80 20 client.28304 populate_metadata read hostname 'smithi062' 2023-07-14T16:26:59.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.658+0000 7f46df338b80 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556ea44166d0 0x556ea4418ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:26:59.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.658+0000 7f46df338b80 4 client.28304 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:26:59.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.658+0000 7f46df338b80 1 -- 172.21.15.62:0/927977869 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ea42b0050 con 0x556ea44166d0 2023-07-14T16:26:59.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.658+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556ea44166d0 0x556ea4418ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:26:59.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46c8f9a700 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556ea44166d0 0x556ea4418ae0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b9406d9586f9c9 server_cookie=b09823c8e64cf56f in_seq=0 out_seq=0 2023-07-14T16:26:59.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46acff9700 10 client.28304 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:59.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46acff9700 1 -- 172.21.15.62:0/927977869 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ea42b0050 con 0x556ea44166d0 2023-07-14T16:26:59.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46acff9700 10 client.28304 handle_command_reply: tid=0 2023-07-14T16:26:59.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 client.28304 shutdown 2023-07-14T16:26:59.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 -- 172.21.15.62:0/927977869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 msgr2=0x556ea43bace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43bace0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46c00119c0 tx=0x7f46c00296e0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 -- 172.21.15.62:0/927977869 shutdown_connections 2023-07-14T16:26:59.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ea43ad660 0x556ea43bc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556ea44166d0 0x556ea4418ae0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ea43abee0 0x556ea43bace0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 --2- 172.21.15.62:0/927977869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556ea43ac830 0x556ea43bb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46acff9700 0 client.28304 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:26:59.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 -- 172.21.15.62:0/927977869 >> 172.21.15.62:0/927977869 conn(0x556ea4398ff0 msgr2=0x556ea43b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 -- 172.21.15.62:0/927977869 shutdown_connections 2023-07-14T16:26:59.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.659+0000 7f46df338b80 1 -- 172.21.15.62:0/927977869 wait complete. 2023-07-14T16:26:59.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46df338b80 20 client.28304 trim_cache size 0 max 16384 2023-07-14T16:26:59.642 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:26:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 -- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46b00a1270 msgr2=0x7f46b00a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46b00a1270 0x7f46b00a3720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x556ea43acc40 tx=0x7f46c0009260 comp rx=0 tx=0).stop 2023-07-14T16:26:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 -- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 msgr2=0x7f46c419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:26:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 0x7f46c419cb30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00d010 tx=0x7f46bc00bcc0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 -- 172.21.15.62:0/1463611457 shutdown_connections 2023-07-14T16:26:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c41056d0 0x7f46c419cb30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f46b00a1270 0x7f46b00a3720 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46c4100490 0x7f46c4195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 --2- 172.21.15.62:0/1463611457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46c4102db0 0x7f46c4195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:26:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.660+0000 7f46ca79d700 1 -- 172.21.15.62:0/1463611457 >> 172.21.15.62:0/1463611457 conn(0x7f46c40f9ed0 msgr2=0x7f46c41010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:26:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.661+0000 7f46ca79d700 1 -- 172.21.15.62:0/1463611457 shutdown_connections 2023-07-14T16:26:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:26:59.661+0000 7f46ca79d700 1 -- 172.21.15.62:0/1463611457 wait complete. 2023-07-14T16:27:00.658 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:27:00.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:27:00.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.912+0000 7fda3a1ff700 1 -- 172.21.15.62:0/3158488587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 msgr2=0x7fda341011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.912+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/3158488587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 0x7fda341011d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fda28009990 tx=0x7fda28009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.912+0000 7fda3a1ff700 1 -- 172.21.15.62:0/3158488587 shutdown_connections 2023-07-14T16:27:00.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.912+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/3158488587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda34101710 0x7fda34107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.912+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/3158488587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda340ffad0 0x7fda340ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.912+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/3158488587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 0x7fda341011d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.912+0000 7fda3a1ff700 1 -- 172.21.15.62:0/3158488587 >> 172.21.15.62:0/3158488587 conn(0x7fda340fb0f0 msgr2=0x7fda340fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:00.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.912+0000 7fda3a1ff700 1 -- 172.21.15.62:0/3158488587 shutdown_connections 2023-07-14T16:27:00.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.913+0000 7fda3a1ff700 1 -- 172.21.15.62:0/3158488587 wait complete. 2023-07-14T16:27:00.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.914+0000 7fda3a1ff700 1 Processor -- start 2023-07-14T16:27:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.914+0000 7fda3a1ff700 1 -- start start 2023-07-14T16:27:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.914+0000 7fda3a1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda340ffad0 0x7fda34197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.914+0000 7fda3a1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 0x7fda34197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.914+0000 7fda3a1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda34101710 0x7fda3419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.914+0000 7fda3a1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419f4e0 con 0x7fda34100d60 2023-07-14T16:27:00.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.914+0000 7fda3a1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419f620 con 0x7fda34101710 2023-07-14T16:27:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.914+0000 7fda3a1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419f7b0 con 0x7fda340ffad0 2023-07-14T16:27:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 0x7fda34197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 0x7fda34197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40592/0 (socket says 172.21.15.62:40592) 2023-07-14T16:27:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 -- 172.21.15.62:0/2352332993 learned_addr learned my addr 172.21.15.62:0/2352332993 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda337fe700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda340ffad0 0x7fda34197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda33fff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda34101710 0x7fda3419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 -- 172.21.15.62:0/2352332993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda340ffad0 msgr2=0x7fda34197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda340ffad0 0x7fda34197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 -- 172.21.15.62:0/2352332993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda34101710 msgr2=0x7fda3419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda34101710 0x7fda3419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 -- 172.21.15.62:0/2352332993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280095f0 con 0x7fda34100d60 2023-07-14T16:27:00.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.915+0000 7fda32ffd700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 0x7fda34197ea0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fda28006010 tx=0x7fda28003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:00.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.916+0000 7fda30ff9700 1 -- 172.21.15.62:0/2352332993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2801d070 con 0x7fda34100d60 2023-07-14T16:27:00.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.916+0000 7fda3a1ff700 1 -- 172.21.15.62:0/2352332993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3419fa30 con 0x7fda34100d60 2023-07-14T16:27:00.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.916+0000 7fda30ff9700 1 -- 172.21.15.62:0/2352332993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda2800bd70 con 0x7fda34100d60 2023-07-14T16:27:00.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.916+0000 7fda30ff9700 1 -- 172.21.15.62:0/2352332993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28022c90 con 0x7fda34100d60 2023-07-14T16:27:00.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.916+0000 7fda3a1ff700 1 -- 172.21.15.62:0/2352332993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3419fe50 con 0x7fda34100d60 2023-07-14T16:27:00.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.919+0000 7fda30ff9700 1 -- 172.21.15.62:0/2352332993 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fda28003dd0 con 0x7fda34100d60 2023-07-14T16:27:00.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.921+0000 7fda4dbbeb80 1 Processor -- start 2023-07-14T16:27:00.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.921+0000 7fda4dbbeb80 1 -- start start 2023-07-14T16:27:00.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x559393403960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393404760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x559393405160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559393308d30 con 0x559393403ea0 2023-07-14T16:27:00.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55939335bac0 con 0x559393403550 2023-07-14T16:27:00.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559393354f50 con 0x559393404cf0 2023-07-14T16:27:00.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393404760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda33fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x559393403960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda30ff9700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda1c0a1150 0x7fda1c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda33fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x559393403960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59194/0 (socket says 172.21.15.62:59194) 2023-07-14T16:27:00.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x559393405160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393404760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40596/0 (socket says 172.21.15.62:40596) 2023-07-14T16:27:00.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda33fff700 1 -- 172.21.15.62:0/405343107 learned_addr learned my addr 172.21.15.62:0/405343107 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:00.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.922+0000 7fda30ff9700 1 -- 172.21.15.62:0/2352332993 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fda280cde30 con 0x7fda34100d60 2023-07-14T16:27:00.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda33fff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda1c0a1150 0x7fda1c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda32ffd700 1 -- 172.21.15.62:0/405343107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 msgr2=0x559393403960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:00.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda32ffd700 1 --2- 172.21.15.62:0/405343107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x559393403960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda32ffd700 1 -- 172.21.15.62:0/405343107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 msgr2=0x559393404760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:00.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda32ffd700 1 --2- 172.21.15.62:0/405343107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393404760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda32ffd700 1 -- 172.21.15.62:0/405343107 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559393182e60 con 0x559393404cf0 2023-07-14T16:27:00.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda32ffd700 1 --2- 172.21.15.62:0/405343107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x559393405160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda28097cf0 tx=0x7fda28009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a08754e8a794368f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:00.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda19ffb700 1 -- 172.21.15.62:0/405343107 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2809abe0 con 0x559393404cf0 2023-07-14T16:27:00.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda33fff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda1c0a1150 0x7fda1c0a3600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fda2c003fd0 tx=0x7fda2c004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:00.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda19ffb700 1 -- 172.21.15.62:0/405343107 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda2809b4e0 con 0x559393404cf0 2023-07-14T16:27:00.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.923+0000 7fda19ffb700 1 -- 172.21.15.62:0/405343107 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28022470 con 0x559393404cf0 2023-07-14T16:27:00.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/405343107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 msgr2=0x559393405160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/405343107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x559393405160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda28097cf0 tx=0x7fda28009310 comp rx=0 tx=0).stop 2023-07-14T16:27:00.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/405343107 shutdown_connections 2023-07-14T16:27:00.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/405343107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x559393405160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/405343107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x559393403960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/405343107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393404760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/405343107 >> 172.21.15.62:0/405343107 conn(0x5593933f8e20 msgr2=0x5593933fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/405343107 shutdown_connections 2023-07-14T16:27:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.924+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/405343107 wait complete. 2023-07-14T16:27:00.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.925+0000 7fda4dbbeb80 1 -- start start 2023-07-14T16:27:00.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.925+0000 7fda4dbbeb80 1 Processor -- start 2023-07-14T16:27:00.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x55939340ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x55939340d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934143c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593932f4b40 con 0x559393403ea0 2023-07-14T16:27:00.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593932f4840 con 0x559393403550 2023-07-14T16:27:00.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593931d89a0 con 0x559393404cf0 2023-07-14T16:27:00.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda4dbbeb80 10 client.0 _ll_register_callbacks cb 0x5593932f27e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:00.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x55939340d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x55939340d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40612/0 (socket says 172.21.15.62:40612) 2023-07-14T16:27:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda32ffd700 1 -- 172.21.15.62:0/3024124284 learned_addr learned my addr 172.21.15.62:0/3024124284 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda33fff700 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda337fe700 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x55939340ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda32ffd700 1 -- 172.21.15.62:0/3024124284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 msgr2=0x5593934143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda32ffd700 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda32ffd700 1 -- 172.21.15.62:0/3024124284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 msgr2=0x55939340ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:00.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda32ffd700 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x55939340ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.926+0000 7fda32ffd700 1 -- 172.21.15.62:0/3024124284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3419fe50 con 0x559393403ea0 2023-07-14T16:27:00.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.927+0000 7fda32ffd700 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x55939340d380 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fda280098e0 tx=0x7fda280cb880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bcf10a61f989b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:00.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.927+0000 7fda12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:00.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.927+0000 7fda12ffd700 1 -- 172.21.15.62:0/3024124284 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2809abe0 con 0x559393403ea0 2023-07-14T16:27:00.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.927+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/3024124284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593932ae570 con 0x559393403ea0 2023-07-14T16:27:00.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.927+0000 7fda12ffd700 1 -- 172.21.15.62:0/3024124284 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fda2809b280 con 0x559393403ea0 2023-07-14T16:27:00.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.927+0000 7fda12ffd700 1 -- 172.21.15.62:0/3024124284 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda280a10b0 con 0x559393403ea0 2023-07-14T16:27:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.927+0000 7fda4dbbeb80 10 client.18717 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.927+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/3024124284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55939316bce0 con 0x559393403ea0 2023-07-14T16:27:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda12ffd700 1 -- 172.21.15.62:0/3024124284 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5593932ae570 con 0x559393403ea0 2023-07-14T16:27:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda12ffd700 1 -- 172.21.15.62:0/3024124284 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fda280cb960 con 0x559393403ea0 2023-07-14T16:27:00.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda4dbbeb80 10 client.18717 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:00.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda4dbbeb80 10 client.18717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:00.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda4dbbeb80 20 client.18717 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:00.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5593934186f0 0x55939341ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda4dbbeb80 4 client.18717 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:00.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/3024124284 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55939334df30 con 0x5593934186f0 2023-07-14T16:27:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.928+0000 7fda337fe700 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5593934186f0 0x55939341ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.929+0000 7fda337fe700 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5593934186f0 0x55939341ab00 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212d7838dbaa4f29 server_cookie=25fe9799c32c0924 in_seq=0 out_seq=0 2023-07-14T16:27:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.929+0000 7fda12ffd700 10 client.18717 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:00.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda12ffd700 1 -- 172.21.15.62:0/3024124284 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55939334df30 con 0x5593934186f0 2023-07-14T16:27:00.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda12ffd700 10 client.18717 handle_command_reply: tid=0 2023-07-14T16:27:00.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 client.18717 shutdown 2023-07-14T16:27:00.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/3024124284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 msgr2=0x55939340d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x55939340d380 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fda280098e0 tx=0x7fda280cb880 comp rx=0 tx=0).stop 2023-07-14T16:27:00.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/3024124284 shutdown_connections 2023-07-14T16:27:00.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5593934186f0 0x55939341ab00 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x55939340ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/3024124284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x55939340d380 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/3024124284 >> 172.21.15.62:0/3024124284 conn(0x5593933f8e20 msgr2=0x5593933fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:00.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.930+0000 7fda12ffd700 0 client.18717 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:00.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.931+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/3024124284 shutdown_connections 2023-07-14T16:27:00.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.931+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/3024124284 wait complete. 2023-07-14T16:27:00.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.931+0000 7fda4dbbeb80 20 client.18717 trim_cache size 0 max 16384 2023-07-14T16:27:00.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.955+0000 7fda4dbbeb80 1 Processor -- start 2023-07-14T16:27:00.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.955+0000 7fda4dbbeb80 1 -- start start 2023-07-14T16:27:00.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.955+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393403550 0x559393407b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.955+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393408090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.955+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393404cf0 0x559393414380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.955+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55939340ca10 con 0x559393403ea0 2023-07-14T16:27:00.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.955+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55939334df30 con 0x559393404cf0 2023-07-14T16:27:00.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.955+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593932ae570 con 0x559393403550 2023-07-14T16:27:00.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393408090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393408090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40616/0 (socket says 172.21.15.62:40616) 2023-07-14T16:27:00.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 -- 172.21.15.62:0/1509312069 learned_addr learned my addr 172.21.15.62:0/1509312069 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:00.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda337fe700 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393403550 0x559393407b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda33fff700 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393404cf0 0x559393414380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 -- 172.21.15.62:0/1509312069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393403550 msgr2=0x559393407b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:00.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393403550 0x559393407b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 -- 172.21.15.62:0/1509312069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393404cf0 msgr2=0x559393414380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393404cf0 0x559393414380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 -- 172.21.15.62:0/1509312069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593933fa660 con 0x559393403ea0 2023-07-14T16:27:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.956+0000 7fda32ffd700 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393408090 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fda2809af80 tx=0x7fda28005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88720adecc895751 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.957+0000 7fda1a7fc700 1 -- 172.21.15.62:0/1509312069 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2809abe0 con 0x559393403ea0 2023-07-14T16:27:00.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.957+0000 7fda1a7fc700 1 -- 172.21.15.62:0/1509312069 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda28095e90 con 0x559393403ea0 2023-07-14T16:27:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.957+0000 7fda1a7fc700 1 -- 172.21.15.62:0/1509312069 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280993c0 con 0x559393403ea0 2023-07-14T16:27:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.957+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1509312069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 msgr2=0x559393408090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.957+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393408090 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fda2809af80 tx=0x7fda28005c90 comp rx=0 tx=0).stop 2023-07-14T16:27:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1509312069 shutdown_connections 2023-07-14T16:27:00.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393404cf0 0x559393414380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393403550 0x559393407b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1509312069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393408090 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1509312069 >> 172.21.15.62:0/1509312069 conn(0x5593933f8e20 msgr2=0x559393414cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1509312069 shutdown_connections 2023-07-14T16:27:00.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1509312069 wait complete. 2023-07-14T16:27:00.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 -- start start 2023-07-14T16:27:00.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.958+0000 7fda4dbbeb80 1 Processor -- start 2023-07-14T16:27:00.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x5593934141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393410c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda4dbbeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593934118a0 con 0x559393403ea0 2023-07-14T16:27:00.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55939346c580 con 0x559393403550 2023-07-14T16:27:00.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda4dbbeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55939346c6c0 con 0x559393404cf0 2023-07-14T16:27:00.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda4dbbeb80 10 client.0 _ll_register_callbacks cb 0x7fda240077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:00.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393410c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393410c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40628/0 (socket says 172.21.15.62:40628) 2023-07-14T16:27:00.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.959+0000 7fda32ffd700 1 -- 172.21.15.62:0/1708696102 learned_addr learned my addr 172.21.15.62:0/1708696102 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:00.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda337fe700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x5593934141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda33fff700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda32ffd700 1 -- 172.21.15.62:0/1708696102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 msgr2=0x5593934111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:00.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda32ffd700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda32ffd700 1 -- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 msgr2=0x5593934141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:00.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda32ffd700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x5593934141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda32ffd700 1 -- 172.21.15.62:0/1708696102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55939316bce0 con 0x559393403ea0 2023-07-14T16:27:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda33fff700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda337fe700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x5593934141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:00.945 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda32ffd700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393410c80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fda28000c00 tx=0x7fda280a36b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e853843364ce51d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:00.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:00.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda12ffd700 1 -- 172.21.15.62:0/1708696102 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2809b280 con 0x559393403ea0 2023-07-14T16:27:00.946 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda12ffd700 1 -- 172.21.15.62:0/1708696102 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fda2809f2b0 con 0x559393403ea0 2023-07-14T16:27:00.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.960+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1708696102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda2809b4e0 con 0x559393403ea0 2023-07-14T16:27:00.947 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda12ffd700 1 -- 172.21.15.62:0/1708696102 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda2809b650 con 0x559393403ea0 2023-07-14T16:27:00.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda4dbbeb80 10 client.18723 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:00.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1708696102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559393405390 con 0x559393403ea0 2023-07-14T16:27:00.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda12ffd700 1 -- 172.21.15.62:0/1708696102 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda2809b4e0 con 0x559393403ea0 2023-07-14T16:27:00.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda12ffd700 1 -- 172.21.15.62:0/1708696102 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fda2809a1e0 con 0x559393403ea0 2023-07-14T16:27:00.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda4dbbeb80 10 client.18723 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:00.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda4dbbeb80 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:00.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda4dbbeb80 20 client.18723 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:00.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55939346da10 0x55939346fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:00.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda4dbbeb80 4 client.18723 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:27:00.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.961+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1708696102 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5593933100d0 con 0x55939346da10 2023-07-14T16:27:00.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.962+0000 7fda337fe700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55939346da10 0x55939346fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:00.951 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.962+0000 7fda337fe700 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55939346da10 0x55939346fe20 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74cb93c3efb50cb server_cookie=aed146f09fce5387 in_seq=0 out_seq=0 2023-07-14T16:27:00.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.962+0000 7fda12ffd700 10 client.18723 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:00.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.963+0000 7fda12ffd700 1 -- 172.21.15.62:0/1708696102 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5593933100d0 con 0x55939346da10 2023-07-14T16:27:00.952 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.963+0000 7fda12ffd700 10 client.18723 handle_command_reply: tid=0 2023-07-14T16:27:00.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.963+0000 7fda4dbbeb80 1 client.18723 shutdown 2023-07-14T16:27:00.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1708696102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 msgr2=0x559393410c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.953 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393410c80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fda28000c00 tx=0x7fda280a36b0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1708696102 shutdown_connections 2023-07-14T16:27:00.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559393404cf0 0x5593934111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55939346da10 0x55939346fe20 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.954 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559393403550 0x5593934141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 --2- 172.21.15.62:0/1708696102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559393403ea0 0x559393410c80 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1708696102 >> 172.21.15.62:0/1708696102 conn(0x5593933f8e20 msgr2=0x559393407b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:00.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda12ffd700 0 client.18723 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:00.955 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1708696102 shutdown_connections 2023-07-14T16:27:00.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 1 -- 172.21.15.62:0/1708696102 wait complete. 2023-07-14T16:27:00.956 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.964+0000 7fda4dbbeb80 20 client.18723 trim_cache size 0 max 16384 2023-07-14T16:27:00.957 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:27:00.957 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:27:00.957 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "079352be-fb27-496d-97fa-b8e4929a4977", 2023-07-14T16:27:00.957 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:27:00.958 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:27:00.958 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 -- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda1c0a1150 msgr2=0x7fda1c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda1c0a1150 0x7fda1c0a3600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fda2c003fd0 tx=0x7fda2c004b40 comp rx=0 tx=0).stop 2023-07-14T16:27:00.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 -- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 msgr2=0x7fda34197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:00.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 0x7fda34197ea0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fda28006010 tx=0x7fda28003a50 comp rx=0 tx=0).stop 2023-07-14T16:27:00.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 -- 172.21.15.62:0/2352332993 shutdown_connections 2023-07-14T16:27:00.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda1c0a1150 0x7fda1c0a3600 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda34101710 0x7fda3419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda340ffad0 0x7fda34197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 --2- 172.21.15.62:0/2352332993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda34100d60 0x7fda34197ea0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:00.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 -- 172.21.15.62:0/2352332993 >> 172.21.15.62:0/2352332993 conn(0x7fda340fb0f0 msgr2=0x7fda340fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:00.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 -- 172.21.15.62:0/2352332993 shutdown_connections 2023-07-14T16:27:00.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:00.965+0000 7fda3a1ff700 1 -- 172.21.15.62:0/2352332993 wait complete. 2023-07-14T16:27:00.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:27:01.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:00 smithi164 ceph-mon[97427]: pgmap v698: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:27:01.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:00 smithi062 ceph-mon[95543]: pgmap v698: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:27:01.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:00 smithi081 ceph-mon[127795]: pgmap v698: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:27:01.150 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:27:00 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:27:00.973+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v221>,<0x10000000003/client.0 v222>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v234>,<0x10000000001/sv_0 v248>,<0x10000000000/_nogroup v242>,<0x1/volumes v232>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:27:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 -- 172.21.15.62:0/3759034922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918101ba0 msgr2=0x7f5918101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 --2- 172.21.15.62:0/3759034922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918101ba0 0x7f5918101fb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59080098f0 tx=0x7f5908009c00 comp rx=0 tx=0).stop 2023-07-14T16:27:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 -- 172.21.15.62:0/3759034922 shutdown_connections 2023-07-14T16:27:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 --2- 172.21.15.62:0/3759034922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5918103720 0x7f5918109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 --2- 172.21.15.62:0/3759034922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918101ba0 0x7f5918101fb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 --2- 172.21.15.62:0/3759034922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918102d90 0x7f59181031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 -- 172.21.15.62:0/3759034922 >> 172.21.15.62:0/3759034922 conn(0x7f59180fd1e0 msgr2=0x7f59180ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 -- 172.21.15.62:0/3759034922 shutdown_connections 2023-07-14T16:27:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.222+0000 7f591c9fe700 1 -- 172.21.15.62:0/3759034922 wait complete. 2023-07-14T16:27:01.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.223+0000 7f591c9fe700 1 Processor -- start 2023-07-14T16:27:01.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.223+0000 7f591c9fe700 1 -- start start 2023-07-14T16:27:01.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.223+0000 7f591c9fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5918101ba0 0x7f5918197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.223+0000 7f591c9fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918102d90 0x7f5918197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591c9fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918103720 0x7f591819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591c9fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591819f2e0 con 0x7f5918103720 2023-07-14T16:27:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591c9fe700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591819f420 con 0x7f5918102d90 2023-07-14T16:27:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591c9fe700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591819f5b0 con 0x7f5918101ba0 2023-07-14T16:27:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918103720 0x7f591819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918102d90 0x7f5918197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918102d90 0x7f5918197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:59250/0 (socket says 172.21.15.62:59250) 2023-07-14T16:27:01.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591659c700 1 -- 172.21.15.62:0/2294657266 learned_addr learned my addr 172.21.15.62:0/2294657266 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:01.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f5916d9d700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5918101ba0 0x7f5918197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591659c700 1 -- 172.21.15.62:0/2294657266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5918101ba0 msgr2=0x7f5918197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591659c700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5918101ba0 0x7f5918197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591659c700 1 -- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918103720 msgr2=0x7f591819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591659c700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918103720 0x7f591819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591659c700 1 -- 172.21.15.62:0/2294657266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c0095f0 con 0x7f5918102d90 2023-07-14T16:27:01.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.224+0000 7f591759e700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918103720 0x7f591819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:01.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.225+0000 7f591659c700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918102d90 0x7f5918197ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f590c00ec10 tx=0x7f590c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.225+0000 7f58fffff700 1 -- 172.21.15.62:0/2294657266 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c00cc90 con 0x7f5918102d90 2023-07-14T16:27:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.225+0000 7f58fffff700 1 -- 172.21.15.62:0/2294657266 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f590c00cdf0 con 0x7f5918102d90 2023-07-14T16:27:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.225+0000 7f591c9fe700 1 -- 172.21.15.62:0/2294657266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5908009520 con 0x7f5918102d90 2023-07-14T16:27:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.225+0000 7f58fffff700 1 -- 172.21.15.62:0/2294657266 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c003e80 con 0x7f5918102d90 2023-07-14T16:27:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.225+0000 7f591c9fe700 1 -- 172.21.15.62:0/2294657266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591819fc10 con 0x7f5918102d90 2023-07-14T16:27:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.230+0000 7f58fffff700 1 -- 172.21.15.62:0/2294657266 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f590c004000 con 0x7f5918102d90 2023-07-14T16:27:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.231+0000 7f5930b83b80 1 Processor -- start 2023-07-14T16:27:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.231+0000 7f5930b83b80 1 -- start start 2023-07-14T16:27:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.231+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.231+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.231+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.231+0000 7f5930b83b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fefa6cb0 con 0x55f8ff0a2070 2023-07-14T16:27:01.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.231+0000 7f5930b83b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8feff9a40 con 0x55f8ff0a1720 2023-07-14T16:27:01.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.231+0000 7f5930b83b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8feff2ed0 con 0x55f8ff0a2e30 2023-07-14T16:27:01.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f5916d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f591759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f5916d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40660/0 (socket says 172.21.15.62:40660) 2023-07-14T16:27:01.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f5916d9d700 1 -- 172.21.15.62:0/369572294 learned_addr learned my addr 172.21.15.62:0/369572294 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:01.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f591659c700 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f58fffff700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f59000a11f0 0x7f59000a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f58fffff700 1 -- 172.21.15.62:0/2294657266 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f590c014070 con 0x7f5918102d90 2023-07-14T16:27:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f5916d9d700 1 -- 172.21.15.62:0/369572294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 msgr2=0x55f8ff0a32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f5916d9d700 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0a32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f5916d9d700 1 -- 172.21.15.62:0/369572294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 msgr2=0x55f8ff0a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:01.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f5916d9d700 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f5916d9d700 1 -- 172.21.15.62:0/369572294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8fee20de0 con 0x55f8ff0a2070 2023-07-14T16:27:01.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f591659c700 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0a32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:01.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.232+0000 7f591759e700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f59000a11f0 0x7f59000a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.233+0000 7f5916d9d700 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a28a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f59080098c0 tx=0x7f5908005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe5fd28290abfd5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.233+0000 7f58fd7fa700 1 -- 172.21.15.62:0/369572294 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800f660 con 0x55f8ff0a2070 2023-07-14T16:27:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.233+0000 7f58fd7fa700 1 -- 172.21.15.62:0/369572294 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f590801d040 con 0x55f8ff0a2070 2023-07-14T16:27:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.233+0000 7f58fd7fa700 1 -- 172.21.15.62:0/369572294 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59080052a0 con 0x55f8ff0a2070 2023-07-14T16:27:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 -- 172.21.15.62:0/369572294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2070 msgr2=0x55f8ff0a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f591759e700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f59000a11f0 0x7f59000a36a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5910009110 tx=0x7f5910004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a28a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f59080098c0 tx=0x7f5908005dc0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 -- 172.21.15.62:0/369572294 shutdown_connections 2023-07-14T16:27:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0a32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 --2- 172.21.15.62:0/369572294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a28a0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 -- 172.21.15.62:0/369572294 >> 172.21.15.62:0/369572294 conn(0x55f8ff096ff0 msgr2=0x55f8ff09c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 -- 172.21.15.62:0/369572294 shutdown_connections 2023-07-14T16:27:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.234+0000 7f5930b83b80 1 -- 172.21.15.62:0/369572294 wait complete. 2023-07-14T16:27:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.235+0000 7f5930b83b80 1 -- start start 2023-07-14T16:27:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.235+0000 7f5930b83b80 1 Processor -- start 2023-07-14T16:27:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a2070 0x55f8ff0ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5930b83b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fef92ac0 con 0x55f8ff0a1720 2023-07-14T16:27:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5930b83b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fef927c0 con 0x55f8ff0a2070 2023-07-14T16:27:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5930b83b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fee76920 con 0x55f8ff0a2e30 2023-07-14T16:27:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5930b83b80 10 client.0 _ll_register_callbacks cb 0x55f8fef90760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5916d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5916d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ab2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40662/0 (socket says 172.21.15.62:40662) 2023-07-14T16:27:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5916d9d700 1 -- 172.21.15.62:0/1130912613 learned_addr learned my addr 172.21.15.62:0/1130912613 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f591659c700 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a2070 0x55f8ff0ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5916d9d700 1 -- 172.21.15.62:0/1130912613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 msgr2=0x55f8ff0b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5916d9d700 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5916d9d700 1 -- 172.21.15.62:0/1130912613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a2070 msgr2=0x55f8ff0ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5916d9d700 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a2070 0x55f8ff0ab7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.236+0000 7f5916d9d700 1 -- 172.21.15.62:0/1130912613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591810a910 con 0x55f8ff0a1720 2023-07-14T16:27:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.237+0000 7f5916d9d700 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ab2a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5908000c00 tx=0x7f590800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7462fb7a4b4e208a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.237+0000 7f58faff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.237+0000 7f58faff5700 1 -- 172.21.15.62:0/1130912613 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908005070 con 0x55f8ff0a1720 2023-07-14T16:27:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.237+0000 7f5930b83b80 1 -- 172.21.15.62:0/1130912613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8fef4c4f0 con 0x55f8ff0a1720 2023-07-14T16:27:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.237+0000 7f58faff5700 1 -- 172.21.15.62:0/1130912613 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f59080051d0 con 0x55f8ff0a1720 2023-07-14T16:27:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.237+0000 7f58faff5700 1 -- 172.21.15.62:0/1130912613 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5908005390 con 0x55f8ff0a1720 2023-07-14T16:27:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.237+0000 7f5930b83b80 10 client.18735 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.237+0000 7f5930b83b80 1 -- 172.21.15.62:0/1130912613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8fee09c60 con 0x55f8ff0a1720 2023-07-14T16:27:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.238+0000 7f58faff5700 1 -- 172.21.15.62:0/1130912613 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f590803b020 con 0x55f8ff0a1720 2023-07-14T16:27:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.238+0000 7f58faff5700 1 -- 172.21.15.62:0/1130912613 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5908009c70 con 0x55f8ff0a1720 2023-07-14T16:27:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.238+0000 7f5930b83b80 10 client.18735 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.238+0000 7f5930b83b80 10 client.18735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.238+0000 7f5930b83b80 20 client.18735 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.238+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f8ff0b6b60 0x55f8ff0b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.238+0000 7f5930b83b80 4 client.18735 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.238+0000 7f5930b83b80 1 -- 172.21.15.62:0/1130912613 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8fefebeb0 con 0x55f8ff0b6b60 2023-07-14T16:27:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.239+0000 7f591659c700 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f8ff0b6b60 0x55f8ff0b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.239+0000 7f58faff5700 10 client.18735 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.239+0000 7f591659c700 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f8ff0b6b60 0x55f8ff0b8f70 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb8a8e8b5d28c17 server_cookie=f7f0bbb65ba876b7 in_seq=0 out_seq=0 2023-07-14T16:27:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f58faff5700 1 -- 172.21.15.62:0/1130912613 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f8fefebeb0 con 0x55f8ff0b6b60 2023-07-14T16:27:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f58faff5700 10 client.18735 handle_command_reply: tid=0 2023-07-14T16:27:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 client.18735 shutdown 2023-07-14T16:27:01.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 -- 172.21.15.62:0/1130912613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a1720 msgr2=0x55f8ff0ab2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ab2a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5908000c00 tx=0x7f590800f510 comp rx=0 tx=0).stop 2023-07-14T16:27:01.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 -- 172.21.15.62:0/1130912613 shutdown_connections 2023-07-14T16:27:01.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ab2a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f8ff0b6b60 0x55f8ff0b8f70 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1130912613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a2070 0x55f8ff0ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 -- 172.21.15.62:0/1130912613 >> 172.21.15.62:0/1130912613 conn(0x55f8ff096ff0 msgr2=0x55f8ff09bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f58faff5700 0 client.18735 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 -- 172.21.15.62:0/1130912613 shutdown_connections 2023-07-14T16:27:01.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.240+0000 7f5930b83b80 1 -- 172.21.15.62:0/1130912613 wait complete. 2023-07-14T16:27:01.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.241+0000 7f5930b83b80 20 client.18735 trim_cache size 0 max 16384 2023-07-14T16:27:01.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5930b83b80 1 Processor -- start 2023-07-14T16:27:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5930b83b80 1 -- start start 2023-07-14T16:27:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5930b83b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ff0aae70 con 0x55f8ff0a2e30 2023-07-14T16:27:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5930b83b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ff0a2690 con 0x55f8ff0a1720 2023-07-14T16:27:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5930b83b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ff09dbd0 con 0x55f8ff0a2070 2023-07-14T16:27:01.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f591759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f591759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40668/0 (socket says 172.21.15.62:40668) 2023-07-14T16:27:01.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f591759e700 1 -- 172.21.15.62:0/1212655250 learned_addr learned my addr 172.21.15.62:0/1212655250 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:01.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.262+0000 7f5916d9d700 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.263+0000 7f591659c700 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.263+0000 7f5916d9d700 1 -- 172.21.15.62:0/1212655250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 msgr2=0x55f8ff0a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.263+0000 7f5916d9d700 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.263+0000 7f5916d9d700 1 -- 172.21.15.62:0/1212655250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 msgr2=0x55f8ff0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.263+0000 7f5916d9d700 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.263+0000 7f5916d9d700 1 -- 172.21.15.62:0/1212655250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8fee47250 con 0x55f8ff0a1720 2023-07-14T16:27:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.263+0000 7f591759e700 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.263+0000 7f5916d9d700 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a9180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5908009810 tx=0x7f5908009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1e6ce36e841391b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.264+0000 7f58fdffb700 1 -- 172.21.15.62:0/1212655250 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800b880 con 0x55f8ff0a1720 2023-07-14T16:27:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.264+0000 7f58fdffb700 1 -- 172.21.15.62:0/1212655250 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f590803b030 con 0x55f8ff0a1720 2023-07-14T16:27:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.264+0000 7f58fdffb700 1 -- 172.21.15.62:0/1212655250 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908005300 con 0x55f8ff0a1720 2023-07-14T16:27:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.264+0000 7f5930b83b80 1 -- 172.21.15.62:0/1212655250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 msgr2=0x55f8ff0a9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.264+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a9180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5908009810 tx=0x7f5908009fd0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 -- 172.21.15.62:0/1212655250 shutdown_connections 2023-07-14T16:27:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0a9180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 --2- 172.21.15.62:0/1212655250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 -- 172.21.15.62:0/1212655250 >> 172.21.15.62:0/1212655250 conn(0x55f8ff096ff0 msgr2=0x55f8ff0b8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 -- 172.21.15.62:0/1212655250 shutdown_connections 2023-07-14T16:27:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 -- 172.21.15.62:0/1212655250 wait complete. 2023-07-14T16:27:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 -- start start 2023-07-14T16:27:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.265+0000 7f5930b83b80 1 Processor -- start 2023-07-14T16:27:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.266+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.266+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.266+0000 7f5930b83b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.266+0000 7f5930b83b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ff0b2280 con 0x55f8ff0a2e30 2023-07-14T16:27:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.266+0000 7f5930b83b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ff0b23f0 con 0x55f8ff0a1720 2023-07-14T16:27:01.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.266+0000 7f5930b83b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ff0b25b0 con 0x55f8ff0a2070 2023-07-14T16:27:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.266+0000 7f5930b83b80 10 client.0 _ll_register_callbacks cb 0x7f5910009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.266+0000 7f591759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40670/0 (socket says 172.21.15.62:40670) 2023-07-14T16:27:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591759e700 1 -- 172.21.15.62:0/712310335 learned_addr learned my addr 172.21.15.62:0/712310335 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f5916d9d700 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591659c700 1 -- 172.21.15.62:0/712310335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 msgr2=0x55f8ff0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591659c700 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591659c700 1 -- 172.21.15.62:0/712310335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 msgr2=0x55f8ff0b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591659c700 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591659c700 1 -- 172.21.15.62:0/712310335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5910004570 con 0x55f8ff0a2070 2023-07-14T16:27:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591759e700 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f591659c700 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0aed20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f590c0cce10 tx=0x7f590c00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9822bf8cc212ed5b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.267+0000 7f58faff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:27:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.268+0000 7f58faff5700 1 -- 172.21.15.62:0/712310335 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c09e160 con 0x55f8ff0a2070 2023-07-14T16:27:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.268+0000 7f58faff5700 1 -- 172.21.15.62:0/712310335 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f590c0a2580 con 0x55f8ff0a2070 2023-07-14T16:27:01.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.268+0000 7f5930b83b80 1 -- 172.21.15.62:0/712310335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f590801d040 con 0x55f8ff0a2070 2023-07-14T16:27:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.268+0000 7f58faff5700 1 -- 172.21.15.62:0/712310335 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f590c0984d0 con 0x55f8ff0a2070 2023-07-14T16:27:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.268+0000 7f5930b83b80 10 client.28225 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.268+0000 7f5930b83b80 1 -- 172.21.15.62:0/712310335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8fee09c60 con 0x55f8ff0a2070 2023-07-14T16:27:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.268+0000 7f58faff5700 1 -- 172.21.15.62:0/712310335 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f590801d040 con 0x55f8ff0a2070 2023-07-14T16:27:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.269+0000 7f58faff5700 1 -- 172.21.15.62:0/712310335 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f590c09a390 con 0x55f8ff0a2070 2023-07-14T16:27:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.269+0000 7f5930b83b80 10 client.28225 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.269+0000 7f5930b83b80 10 client.28225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.269+0000 7f5930b83b80 20 client.28225 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.269+0000 7f5930b83b80 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f8ff10bae0 0x55f8ff10def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.269+0000 7f5930b83b80 4 client.28225 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:27:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.269+0000 7f5930b83b80 1 -- 172.21.15.62:0/712310335 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8fefae050 con 0x55f8ff10bae0 2023-07-14T16:27:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.269+0000 7f5916d9d700 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f8ff10bae0 0x55f8ff10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5916d9d700 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f8ff10bae0 0x55f8ff10def0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4806a3123a651c server_cookie=17edc8261827a72e in_seq=0 out_seq=0 2023-07-14T16:27:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f58faff5700 10 client.28225 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f58faff5700 1 -- 172.21.15.62:0/712310335 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f8fefae050 con 0x55f8ff10bae0 2023-07-14T16:27:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f58faff5700 10 client.28225 handle_command_reply: tid=0 2023-07-14T16:27:01.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 client.28225 shutdown 2023-07-14T16:27:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 -- 172.21.15.62:0/712310335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 msgr2=0x55f8ff0aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0aed20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f590c0cce10 tx=0x7f590c00cf60 comp rx=0 tx=0).stop 2023-07-14T16:27:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 -- 172.21.15.62:0/712310335 shutdown_connections 2023-07-14T16:27:01.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8ff0a2e30 0x55f8ff0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f8ff10bae0 0x55f8ff10def0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f8ff0a1720 0x55f8ff0ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 --2- 172.21.15.62:0/712310335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f8ff0a2070 0x55f8ff0aed20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f58faff5700 0 client.28225 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:01.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 -- 172.21.15.62:0/712310335 >> 172.21.15.62:0/712310335 conn(0x55f8ff096ff0 msgr2=0x55f8ff0a9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:01.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.270+0000 7f5930b83b80 1 -- 172.21.15.62:0/712310335 shutdown_connections 2023-07-14T16:27:01.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f5930b83b80 1 -- 172.21.15.62:0/712310335 wait complete. 2023-07-14T16:27:01.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f5930b83b80 20 client.28225 trim_cache size 0 max 16384 2023-07-14T16:27:01.260 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:27:01.260 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:27:01.260 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:27:01.260 INFO:teuthology.orchestra.run.smithi062.stdout: "079352be-fb27-496d-97fa-b8e4929a4977": { 2023-07-14T16:27:01.260 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:27:01.261 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "079352be-fb27-496d-97fa-b8e4929a4977", 2023-07-14T16:27:01.261 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:27:01.261 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:27:01.261 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:27:01.261 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:27:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 -- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f59000a11f0 msgr2=0x7f59000a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f59000a11f0 0x7f59000a36a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5910009110 tx=0x7f5910004ad0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 -- 172.21.15.62:0/2294657266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918102d90 msgr2=0x7f5918197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:01.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918102d90 0x7f5918197ca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f590c00ec10 tx=0x7f590c00ef20 comp rx=0 tx=0).stop 2023-07-14T16:27:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 -- 172.21.15.62:0/2294657266 shutdown_connections 2023-07-14T16:27:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5918103720 0x7f591819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5918101ba0 0x7f5918197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f59000a11f0 0x7f59000a36a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 --2- 172.21.15.62:0/2294657266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5918102d90 0x7f5918197ca0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 -- 172.21.15.62:0/2294657266 >> 172.21.15.62:0/2294657266 conn(0x7f59180fd1e0 msgr2=0x7f59180ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.271+0000 7f591c9fe700 1 -- 172.21.15.62:0/2294657266 shutdown_connections 2023-07-14T16:27:01.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:01.272+0000 7f591c9fe700 1 -- 172.21.15.62:0/2294657266 wait complete. 2023-07-14T16:27:01.265 INFO:tasks.cephfs.filesystem:scrub status for tag:079352be-fb27-496d-97fa-b8e4929a4977 - {'path': '/', 'tag': '079352be-fb27-496d-97fa-b8e4929a4977', 'options': 'recursive,force'} 2023-07-14T16:27:02.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.023 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:27:02.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:27:02.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.024 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:01 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:27:02.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:27:02.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:27:02.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:27:02.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:27:02.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:01 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:27:02.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:27:02.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:27:02.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:27:02.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:27:02.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:02.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:01 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:03.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:02 smithi062 ceph-mon[95543]: pgmap v699: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:27:03.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:02 smithi081 ceph-mon[127795]: pgmap v699: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:27:03.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:02 smithi164 ceph-mon[97427]: pgmap v699: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:27:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/2612030863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8100cf0 msgr2=0x7f5bd8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/2612030863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8100cf0 0x7f5bd8101140 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009990 tx=0x7f5bcc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/2612030863 shutdown_connections 2023-07-14T16:27:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/2612030863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd8101680 0x7f5bd8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/2612030863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bd80ffb00 0x7f5bd80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/2612030863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8100cf0 0x7f5bd8101140 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/2612030863 >> 172.21.15.62:0/2612030863 conn(0x7f5bd80fb0e0 msgr2=0x7f5bd80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/2612030863 shutdown_connections 2023-07-14T16:27:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.315+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/2612030863 wait complete. 2023-07-14T16:27:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.316+0000 7f5bdd3ff700 1 Processor -- start 2023-07-14T16:27:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.316+0000 7f5bdd3ff700 1 -- start start 2023-07-14T16:27:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bdd3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bd80ffb00 0x7f5bd8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bdd3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd8100cf0 0x7f5bd8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bdd3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8101680 0x7f5bd8073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bdd3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80744a0 con 0x7f5bd8101680 2023-07-14T16:27:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bdd3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd8074610 con 0x7f5bd80ffb00 2023-07-14T16:27:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bdd3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80747d0 con 0x7f5bd8100cf0 2023-07-14T16:27:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bd77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8101680 0x7f5bd8073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bd77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8101680 0x7f5bd8073380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41538/0 (socket says 172.21.15.62:41538) 2023-07-14T16:27:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bd77fe700 1 -- 172.21.15.62:0/1092697986 learned_addr learned my addr 172.21.15.62:0/1092697986 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd8100cf0 0x7f5bd8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.317+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bd80ffb00 0x7f5bd8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bd77fe700 1 -- 172.21.15.62:0/1092697986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd8100cf0 msgr2=0x7f5bd8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bd77fe700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd8100cf0 0x7f5bd8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bd77fe700 1 -- 172.21.15.62:0/1092697986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bd80ffb00 msgr2=0x7f5bd8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bd77fe700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bd80ffb00 0x7f5bd8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bd77fe700 1 -- 172.21.15.62:0/1092697986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc0095f0 con 0x7f5bd8101680 2023-07-14T16:27:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bd80ffb00 0x7f5bd8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bd77fe700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8101680 0x7f5bd8073380 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000ef60 tx=0x7f5bd000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bbffff700 1 -- 172.21.15.62:0/1092697986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0018070 con 0x7f5bd8101680 2023-07-14T16:27:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bbffff700 1 -- 172.21.15.62:0/1092697986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bd000f040 con 0x7f5bd8101680 2023-07-14T16:27:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bbffff700 1 -- 172.21.15.62:0/1092697986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000cd10 con 0x7f5bd8101680 2023-07-14T16:27:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/1092697986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd81a4110 con 0x7f5bd8101680 2023-07-14T16:27:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.318+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/1092697986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd81a45a0 con 0x7f5bd8101680 2023-07-14T16:27:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.322+0000 7f5bbffff700 1 -- 172.21.15.62:0/1092697986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5bd0004ad0 con 0x7f5bd8101680 2023-07-14T16:27:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.324+0000 7f5bf0c4bb80 1 Processor -- start 2023-07-14T16:27:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bf0c4bb80 1 -- start start 2023-07-14T16:27:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019259300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557019259840 0x55701925a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 0x55701925ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701915ec50 con 0x5570190b56b0 2023-07-14T16:27:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bbffff700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bc00a1150 0x7f5bc00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570191b19e0 con 0x55701925a6e0 2023-07-14T16:27:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570191aae70 con 0x557019259840 2023-07-14T16:27:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bbffff700 1 -- 172.21.15.62:0/1092697986 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5bd00cc4d0 con 0x7f5bd8101680 2023-07-14T16:27:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bd67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019259300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.325+0000 7f5bd67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019259300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41540/0 (socket says 172.21.15.62:41540) 2023-07-14T16:27:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd67fc700 1 -- 172.21.15.62:0/1947840008 learned_addr learned my addr 172.21.15.62:0/1947840008 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557019259840 0x55701925a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd77fe700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bc00a1150 0x7f5bc00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 0x55701925ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd67fc700 1 -- 172.21.15.62:0/1947840008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557019259840 msgr2=0x55701925a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557019259840 0x55701925a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd67fc700 1 -- 172.21.15.62:0/1947840008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 msgr2=0x55701925ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 0x55701925ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd67fc700 1 -- 172.21.15.62:0/1947840008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557018fd8d80 con 0x5570190b56b0 2023-07-14T16:27:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019259300 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005910 tx=0x7f5bcc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e492f90547ed470a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.326+0000 7f5bd77fe700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bc00a1150 0x7f5bc00a3600 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0099620 tx=0x7f5bd001c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.327+0000 7f5bbd7fa700 1 -- 172.21.15.62:0/1947840008 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc01d070 con 0x5570190b56b0 2023-07-14T16:27:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.327+0000 7f5bbd7fa700 1 -- 172.21.15.62:0/1947840008 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bcc004cc0 con 0x5570190b56b0 2023-07-14T16:27:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.327+0000 7f5bbd7fa700 1 -- 172.21.15.62:0/1947840008 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc005560 con 0x5570190b56b0 2023-07-14T16:27:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/1947840008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 msgr2=0x557019259300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019259300 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005910 tx=0x7f5bcc005c00 comp rx=0 tx=0).stop 2023-07-14T16:27:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/1947840008 shutdown_connections 2023-07-14T16:27:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 0x55701925ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019259300 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/1947840008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557019259840 0x55701925a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/1947840008 >> 172.21.15.62:0/1947840008 conn(0x55701924ebe0 msgr2=0x557019253cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/1947840008 shutdown_connections 2023-07-14T16:27:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/1947840008 wait complete. 2023-07-14T16:27:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.328+0000 7f5bf0c4bb80 1 -- start start 2023-07-14T16:27:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.329+0000 7f5bf0c4bb80 1 Processor -- start 2023-07-14T16:27:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.329+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5570190b56b0 0x5570192628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.329+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019262df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.329+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 0x557019269e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.329+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701914aa60 con 0x557019259840 2023-07-14T16:27:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.329+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701914a760 con 0x5570190b56b0 2023-07-14T16:27:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.329+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701902e8c0 con 0x55701925a6e0 2023-07-14T16:27:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.329+0000 7f5bf0c4bb80 10 client.0 _ll_register_callbacks cb 0x557019148700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019262df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5570190b56b0 0x5570192628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5570190b56b0 0x5570192628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:39042/0 (socket says 172.21.15.62:39042) 2023-07-14T16:27:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd67fc700 1 -- 172.21.15.62:0/2685947122 learned_addr learned my addr 172.21.15.62:0/2685947122 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 0x557019269e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd6ffd700 1 -- 172.21.15.62:0/2685947122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 msgr2=0x557019269e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 0x557019269e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd6ffd700 1 -- 172.21.15.62:0/2685947122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5570190b56b0 msgr2=0x5570192628b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5570190b56b0 0x5570192628b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.330+0000 7f5bd6ffd700 1 -- 172.21.15.62:0/2685947122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc020040 con 0x557019259840 2023-07-14T16:27:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019262df0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8100b50 tx=0x7f5bc8012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb141af1dd9c6d0a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bbaff5700 1 -- 172.21.15.62:0/2685947122 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8017070 con 0x557019259840 2023-07-14T16:27:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bbaff5700 1 -- 172.21.15.62:0/2685947122 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5bc8004260 con 0x557019259840 2023-07-14T16:27:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2685947122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557019104490 con 0x557019259840 2023-07-14T16:27:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bbaff5700 1 -- 172.21.15.62:0/2685947122 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bc8022030 con 0x557019259840 2023-07-14T16:27:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bf0c4bb80 10 client.18753 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2685947122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557018fc1c00 con 0x557019259840 2023-07-14T16:27:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.331+0000 7f5bbaff5700 1 -- 172.21.15.62:0/2685947122 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557019104490 con 0x557019259840 2023-07-14T16:27:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.332+0000 7f5bbaff5700 1 -- 172.21.15.62:0/2685947122 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5bc80116c0 con 0x557019259840 2023-07-14T16:27:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.332+0000 7f5bf0c4bb80 10 client.18753 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.332+0000 7f5bf0c4bb80 10 client.18753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.332+0000 7f5bf0c4bb80 20 client.18753 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.332+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55701926e710 0x557019270b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.332+0000 7f5bf0c4bb80 4 client.18753 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.332+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2685947122 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570191a3e50 con 0x55701926e710 2023-07-14T16:27:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.332+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55701926e710 0x557019270b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.333+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55701926e710 0x557019270b20 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6930e53e6d72f3b6 server_cookie=a0a2b2feb0a2ab07 in_seq=0 out_seq=0 2023-07-14T16:27:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.333+0000 7f5bbaff5700 10 client.18753 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bbaff5700 1 -- 172.21.15.62:0/2685947122 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5570191a3e50 con 0x55701926e710 2023-07-14T16:27:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bbaff5700 10 client.18753 handle_command_reply: tid=0 2023-07-14T16:27:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 client.18753 shutdown 2023-07-14T16:27:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2685947122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 msgr2=0x557019262df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019262df0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8100b50 tx=0x7f5bc8012a00 comp rx=0 tx=0).stop 2023-07-14T16:27:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2685947122 shutdown_connections 2023-07-14T16:27:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 0x557019269e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55701926e710 0x557019270b20 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5570190b56b0 0x5570192628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2685947122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019262df0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2685947122 >> 172.21.15.62:0/2685947122 conn(0x55701924ebe0 msgr2=0x5570192534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bbaff5700 0 client.18753 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2685947122 shutdown_connections 2023-07-14T16:27:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2685947122 wait complete. 2023-07-14T16:27:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.334+0000 7f5bf0c4bb80 20 client.18753 trim_cache size 0 max 16384 2023-07-14T16:27:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bf0c4bb80 1 Processor -- start 2023-07-14T16:27:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bf0c4bb80 1 -- start start 2023-07-14T16:27:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570190b56b0 0x5570192615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019269df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 0x55701925d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557019262420 con 0x557019259840 2023-07-14T16:27:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557019261370 con 0x55701925a6e0 2023-07-14T16:27:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557019250d00 con 0x5570190b56b0 2023-07-14T16:27:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019269df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019269df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41568/0 (socket says 172.21.15.62:41568) 2023-07-14T16:27:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bd6ffd700 1 -- 172.21.15.62:0/2541659410 learned_addr learned my addr 172.21.15.62:0/2541659410 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.351+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570190b56b0 0x5570192615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 0x55701925d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bd6ffd700 1 -- 172.21.15.62:0/2541659410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570190b56b0 msgr2=0x5570192615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570190b56b0 0x5570192615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bd6ffd700 1 -- 172.21.15.62:0/2541659410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 msgr2=0x55701925d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 0x55701925d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bd6ffd700 1 -- 172.21.15.62:0/2541659410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557018fff1f0 con 0x557019259840 2023-07-14T16:27:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570190b56b0 0x5570192615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019269df0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8012010 tx=0x7f5bc8012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3964f6c84178ada1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bbdffb700 1 -- 172.21.15.62:0/2541659410 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8017070 con 0x557019259840 2023-07-14T16:27:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bbdffb700 1 -- 172.21.15.62:0/2541659410 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bc8018040 con 0x557019259840 2023-07-14T16:27:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.352+0000 7f5bbdffb700 1 -- 172.21.15.62:0/2541659410 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8022040 con 0x557019259840 2023-07-14T16:27:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2541659410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 msgr2=0x557019269df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019269df0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8012010 tx=0x7f5bc8012e00 comp rx=0 tx=0).stop 2023-07-14T16:27:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2541659410 shutdown_connections 2023-07-14T16:27:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55701925a6e0 0x55701925d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570190b56b0 0x5570192615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/2541659410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557019259840 0x557019269df0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2541659410 >> 172.21.15.62:0/2541659410 conn(0x55701924ebe0 msgr2=0x557019275440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2541659410 shutdown_connections 2023-07-14T16:27:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.353+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/2541659410 wait complete. 2023-07-14T16:27:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 1 -- start start 2023-07-14T16:27:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 1 Processor -- start 2023-07-14T16:27:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019266020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557019259840 0x557019266560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 0x5570192690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570192695f0 con 0x5570190b56b0 2023-07-14T16:27:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557019269760 con 0x557019259840 2023-07-14T16:27:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557019269920 con 0x55701925a6e0 2023-07-14T16:27:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.354+0000 7f5bf0c4bb80 10 client.0 _ll_register_callbacks cb 0x557019148700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019266020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019266020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41574/0 (socket says 172.21.15.62:41574) 2023-07-14T16:27:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 -- 172.21.15.62:0/148194046 learned_addr learned my addr 172.21.15.62:0/148194046 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557019259840 0x557019266560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 0x5570192690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 -- 172.21.15.62:0/148194046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 msgr2=0x5570192690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 0x5570192690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 -- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557019259840 msgr2=0x557019266560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557019259840 0x557019266560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 -- 172.21.15.62:0/148194046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557018fc1c00 con 0x5570190b56b0 2023-07-14T16:27:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557019259840 0x557019266560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bd67fc700 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019266020 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009990 tx=0x7f5bc8009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8720d3325b91f89a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.355+0000 7f5bbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.356+0000 7f5bbaff5700 1 -- 172.21.15.62:0/148194046 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc02e040 con 0x5570190b56b0 2023-07-14T16:27:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.356+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/148194046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bcc004cc0 con 0x5570190b56b0 2023-07-14T16:27:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.356+0000 7f5bbaff5700 1 -- 172.21.15.62:0/148194046 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5bcc005030 con 0x5570190b56b0 2023-07-14T16:27:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.356+0000 7f5bbaff5700 1 -- 172.21.15.62:0/148194046 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bcc01d040 con 0x5570190b56b0 2023-07-14T16:27:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.356+0000 7f5bf0c4bb80 10 client.18759 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.356+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/148194046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557018fd7250 con 0x5570190b56b0 2023-07-14T16:27:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.356+0000 7f5bbaff5700 1 -- 172.21.15.62:0/148194046 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bcc004cc0 con 0x5570190b56b0 2023-07-14T16:27:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.357+0000 7f5bbaff5700 1 -- 172.21.15.62:0/148194046 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5bcc036a80 con 0x5570190b56b0 2023-07-14T16:27:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.357+0000 7f5bf0c4bb80 10 client.18759 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.357+0000 7f5bf0c4bb80 10 client.18759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.357+0000 7f5bf0c4bb80 20 client.18759 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.357+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5570192c2ff0 0x5570192c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.357+0000 7f5bf0c4bb80 4 client.18759 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:27:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.357+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/148194046 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557019165ff0 con 0x5570192c2ff0 2023-07-14T16:27:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.357+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5570192c2ff0 0x5570192c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.358+0000 7f5bbaff5700 10 client.18759 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.358+0000 7f5bd6ffd700 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5570192c2ff0 0x5570192c53d0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0528d3636233c41 server_cookie=e3a5046c3efa46dc in_seq=0 out_seq=0 2023-07-14T16:27:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bbaff5700 1 -- 172.21.15.62:0/148194046 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557019165ff0 con 0x5570192c2ff0 2023-07-14T16:27:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bbaff5700 10 client.18759 handle_command_reply: tid=0 2023-07-14T16:27:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 client.18759 shutdown 2023-07-14T16:27:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/148194046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 msgr2=0x557019266020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019266020 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009990 tx=0x7f5bc8009e80 comp rx=0 tx=0).stop 2023-07-14T16:27:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/148194046 shutdown_connections 2023-07-14T16:27:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55701925a6e0 0x5570192690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570190b56b0 0x557019266020 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5570192c2ff0 0x5570192c53d0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 --2- 172.21.15.62:0/148194046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557019259840 0x557019266560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bbaff5700 0 client.18759 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.367+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/148194046 >> 172.21.15.62:0/148194046 conn(0x55701924ebe0 msgr2=0x5570192615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.368+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/148194046 shutdown_connections 2023-07-14T16:27:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.368+0000 7f5bf0c4bb80 1 -- 172.21.15.62:0/148194046 wait complete. 2023-07-14T16:27:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.368+0000 7f5bf0c4bb80 20 client.18759 trim_cache size 0 max 16384 2023-07-14T16:27:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:27:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:27:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:27:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:27:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.368+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bc00a1150 msgr2=0x7f5bc00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.368+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bc00a1150 0x7f5bc00a3600 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0099620 tx=0x7f5bd001c000 comp rx=0 tx=0).stop 2023-07-14T16:27:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.368+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8101680 msgr2=0x7f5bd8073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.368+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8101680 0x7f5bd8073380 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000ef60 tx=0x7f5bd000ef90 comp rx=0 tx=0).stop 2023-07-14T16:27:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.369+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/1092697986 shutdown_connections 2023-07-14T16:27:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.369+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bd8101680 0x7f5bd8073380 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.369+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5bc00a1150 0x7f5bc00a3600 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.369+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5bd80ffb00 0x7f5bd8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.369+0000 7f5bdd3ff700 1 --2- 172.21.15.62:0/1092697986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd8100cf0 0x7f5bd8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.369+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/1092697986 >> 172.21.15.62:0/1092697986 conn(0x7f5bd80fb0e0 msgr2=0x7f5bd80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.369+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/1092697986 shutdown_connections 2023-07-14T16:27:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:04.369+0000 7f5bdd3ff700 1 -- 172.21.15.62:0/1092697986 wait complete. 2023-07-14T16:27:05.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:04 smithi062 ceph-mon[95543]: pgmap v700: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 28 op/s 2023-07-14T16:27:05.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:04 smithi081 ceph-mon[127795]: pgmap v700: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 28 op/s 2023-07-14T16:27:05.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:04 smithi164 ceph-mon[97427]: pgmap v700: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 28 op/s 2023-07-14T16:27:07.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:06 smithi062 ceph-mon[95543]: pgmap v701: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 30 op/s 2023-07-14T16:27:07.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:06 smithi081 ceph-mon[127795]: pgmap v701: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 30 op/s 2023-07-14T16:27:07.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:06 smithi164 ceph-mon[97427]: pgmap v701: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 30 op/s 2023-07-14T16:27:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:08 smithi062 ceph-mon[95543]: pgmap v702: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 34 op/s 2023-07-14T16:27:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:08 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:27:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:08 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:27:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:08 smithi081 ceph-mon[127795]: pgmap v702: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 34 op/s 2023-07-14T16:27:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:08 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:27:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:08 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:27:09.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:08 smithi164 ceph-mon[97427]: pgmap v702: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 34 op/s 2023-07-14T16:27:09.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:08 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:27:09.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:08 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:27:11.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:10 smithi062 ceph-mon[95543]: pgmap v703: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:27:11.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:10 smithi081 ceph-mon[127795]: pgmap v703: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:27:11.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:10 smithi164 ceph-mon[97427]: pgmap v703: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:27:13.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:12 smithi062 ceph-mon[95543]: pgmap v704: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:27:13.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:12 smithi081 ceph-mon[127795]: pgmap v704: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:27:13.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:12 smithi164 ceph-mon[97427]: pgmap v704: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:27:15.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:14 smithi062 ceph-mon[95543]: pgmap v705: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:27:15.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:14 smithi081 ceph-mon[127795]: pgmap v705: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:27:15.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:14 smithi164 ceph-mon[97427]: pgmap v705: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:27:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:16 smithi062 ceph-mon[95543]: pgmap v706: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 23 op/s 2023-07-14T16:27:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:16 smithi081 ceph-mon[127795]: pgmap v706: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 23 op/s 2023-07-14T16:27:17.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:16 smithi164 ceph-mon[97427]: pgmap v706: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 23 op/s 2023-07-14T16:27:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 -- 172.21.15.62:0/1012654778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b80ff940 msgr2=0x7fe0b80ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/1012654778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b80ff940 0x7fe0b80ffd50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009990 tx=0x7fe0b4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 -- 172.21.15.62:0/1012654778 shutdown_connections 2023-07-14T16:27:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/1012654778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0b81014c0 0x7fe0b8107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/1012654778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b80ff940 0x7fe0b80ffd50 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/1012654778 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0b8100b30 0x7fe0b8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 -- 172.21.15.62:0/1012654778 >> 172.21.15.62:0/1012654778 conn(0x7fe0b80faf60 msgr2=0x7fe0b80fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 -- 172.21.15.62:0/1012654778 shutdown_connections 2023-07-14T16:27:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.534+0000 7fe0beb9d700 1 -- 172.21.15.62:0/1012654778 wait complete. 2023-07-14T16:27:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.535+0000 7fe0beb9d700 1 Processor -- start 2023-07-14T16:27:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0beb9d700 1 -- start start 2023-07-14T16:27:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0beb9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0b80ff940 0x7fe0b80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0beb9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0b8100b30 0x7fe0b8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0beb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b81014c0 0x7fe0b8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0beb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80735e0 con 0x7fe0b81014c0 2023-07-14T16:27:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0beb9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b8073750 con 0x7fe0b80ff940 2023-07-14T16:27:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0beb9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b819f570 con 0x7fe0b8100b30 2023-07-14T16:27:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0be39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b81014c0 0x7fe0b8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0be39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b81014c0 0x7fe0b8072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59322/0 (socket says 172.21.15.62:59322) 2023-07-14T16:27:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0be39c700 1 -- 172.21.15.62:0/3298994191 learned_addr learned my addr 172.21.15.62:0/3298994191 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.536+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0b8100b30 0x7fe0b8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0b80ff940 0x7fe0b80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0be39c700 1 -- 172.21.15.62:0/3298994191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0b8100b30 msgr2=0x7fe0b8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0be39c700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0b8100b30 0x7fe0b8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0be39c700 1 -- 172.21.15.62:0/3298994191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0b80ff940 msgr2=0x7fe0b80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0be39c700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0b80ff940 0x7fe0b80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0be39c700 1 -- 172.21.15.62:0/3298994191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40095f0 con 0x7fe0b81014c0 2023-07-14T16:27:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0be39c700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b81014c0 0x7fe0b8072e40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0007d70 tx=0x7fe0b000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0aeffd700 1 -- 172.21.15.62:0/3298994191 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000ac90 con 0x7fe0b81014c0 2023-07-14T16:27:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.537+0000 7fe0aeffd700 1 -- 172.21.15.62:0/3298994191 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0b000adf0 con 0x7fe0b81014c0 2023-07-14T16:27:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.538+0000 7fe0beb9d700 1 -- 172.21.15.62:0/3298994191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b819f850 con 0x7fe0b81014c0 2023-07-14T16:27:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.538+0000 7fe0aeffd700 1 -- 172.21.15.62:0/3298994191 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00115e0 con 0x7fe0b81014c0 2023-07-14T16:27:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.538+0000 7fe0beb9d700 1 -- 172.21.15.62:0/3298994191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b819fdb0 con 0x7fe0b81014c0 2023-07-14T16:27:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.543+0000 7fe0aeffd700 1 -- 172.21.15.62:0/3298994191 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe0b0012420 con 0x7fe0b81014c0 2023-07-14T16:27:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.544+0000 7fe0d37aab80 1 Processor -- start 2023-07-14T16:27:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.544+0000 7fe0d37aab80 1 -- start start 2023-07-14T16:27:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.544+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x56350943a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.544+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943aa90 0x56350943b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.544+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943b7e0 0x56350943bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.544+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350933fc50 con 0x56350943b7e0 2023-07-14T16:27:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.544+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635093929e0 con 0x56350943aa90 2023-07-14T16:27:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.544+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350938be70 con 0x5635092966b0 2023-07-14T16:27:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bdb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943aa90 0x56350943b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bd39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943b7e0 0x56350943bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bdb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943aa90 0x56350943b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53266/0 (socket says 172.21.15.62:53266) 2023-07-14T16:27:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/2854195500 learned_addr learned my addr 172.21.15.62:0/2854195500 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0aeffd700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe0a40a11d0 0x7fe0a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x56350943a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0aeffd700 1 -- 172.21.15.62:0/3298994191 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe0b00cc710 con 0x7fe0b81014c0 2023-07-14T16:27:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0be39c700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe0a40a11d0 0x7fe0a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/2854195500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 msgr2=0x56350943a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x56350943a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/2854195500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943b7e0 msgr2=0x56350943bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943b7e0 0x56350943bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/2854195500 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635091b9d80 con 0x56350943aa90 2023-07-14T16:27:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.545+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943b7e0 0x56350943bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.546+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943aa90 0x56350943b2a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8100990 tx=0x7fe0b4005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0b35c58dc1e05c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.546+0000 7fe0be39c700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe0a40a11d0 0x7fe0a40a3680 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00a1a50 tx=0x7fe0b000af60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.546+0000 7fe0a3fff700 1 -- 172.21.15.62:0/2854195500 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b401d070 con 0x56350943aa90 2023-07-14T16:27:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.546+0000 7fe0a3fff700 1 -- 172.21.15.62:0/2854195500 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0b401e040 con 0x56350943aa90 2023-07-14T16:27:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.546+0000 7fe0a3fff700 1 -- 172.21.15.62:0/2854195500 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4004760 con 0x56350943aa90 2023-07-14T16:27:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 -- 172.21.15.62:0/2854195500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943aa90 msgr2=0x56350943b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943aa90 0x56350943b2a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8100990 tx=0x7fe0b4005bc0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 -- 172.21.15.62:0/2854195500 shutdown_connections 2023-07-14T16:27:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943b7e0 0x56350943bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x56350943a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/2854195500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943aa90 0x56350943b2a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 -- 172.21.15.62:0/2854195500 >> 172.21.15.62:0/2854195500 conn(0x56350942fd40 msgr2=0x563509434e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 -- 172.21.15.62:0/2854195500 shutdown_connections 2023-07-14T16:27:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.547+0000 7fe0d37aab80 1 -- 172.21.15.62:0/2854195500 wait complete. 2023-07-14T16:27:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.548+0000 7fe0d37aab80 1 -- start start 2023-07-14T16:27:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.548+0000 7fe0d37aab80 1 Processor -- start 2023-07-14T16:27:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.548+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x5635094439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.548+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509443f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x56350944af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350932ba60 con 0x56350943aa90 2023-07-14T16:27:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350932b760 con 0x56350943b7e0 2023-07-14T16:27:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350920f8c0 con 0x5635092966b0 2023-07-14T16:27:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0d37aab80 10 client.0 _ll_register_callbacks cb 0x563509329700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bdb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509443f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bdb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509443f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59344/0 (socket says 172.21.15.62:59344) 2023-07-14T16:27:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/254043008 learned_addr learned my addr 172.21.15.62:0/254043008 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x56350944af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x5635094439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/254043008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 msgr2=0x5635094439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x5635094439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/254043008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 msgr2=0x56350944af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x56350944af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.549+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/254043008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a8019040 con 0x56350943aa90 2023-07-14T16:27:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.550+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509443f20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005d10 tx=0x7fe0b40092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a324adfc6da4f360 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.550+0000 7fe0a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.550+0000 7fe0a17fa700 1 -- 172.21.15.62:0/254043008 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b401d030 con 0x56350943aa90 2023-07-14T16:27:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.550+0000 7fe0a17fa700 1 -- 172.21.15.62:0/254043008 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe0b4003c30 con 0x56350943aa90 2023-07-14T16:27:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.550+0000 7fe0d37aab80 1 -- 172.21.15.62:0/254043008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635092e5490 con 0x56350943aa90 2023-07-14T16:27:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.550+0000 7fe0a17fa700 1 -- 172.21.15.62:0/254043008 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b40058e0 con 0x56350943aa90 2023-07-14T16:27:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.550+0000 7fe0d37aab80 10 client.18771 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.550+0000 7fe0d37aab80 1 -- 172.21.15.62:0/254043008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe0b81086b0 con 0x56350943aa90 2023-07-14T16:27:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.551+0000 7fe0a17fa700 1 -- 172.21.15.62:0/254043008 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5635092e5490 con 0x56350943aa90 2023-07-14T16:27:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.551+0000 7fe0a17fa700 1 -- 172.21.15.62:0/254043008 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe0b400fa00 con 0x56350943aa90 2023-07-14T16:27:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.551+0000 7fe0d37aab80 10 client.18771 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.551+0000 7fe0d37aab80 10 client.18771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.551+0000 7fe0d37aab80 20 client.18771 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.551+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56350944f350 0x563509451760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.551+0000 7fe0d37aab80 4 client.18771 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.551+0000 7fe0d37aab80 1 -- 172.21.15.62:0/254043008 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563509384e50 con 0x56350944f350 2023-07-14T16:27:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.552+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56350944f350 0x563509451760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.552+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56350944f350 0x563509451760 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbc6b43e35d2a96 server_cookie=bf01f2da35d8d3c7 in_seq=0 out_seq=0 2023-07-14T16:27:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.552+0000 7fe0a17fa700 10 client.18771 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0a17fa700 1 -- 172.21.15.62:0/254043008 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563509384e50 con 0x56350944f350 2023-07-14T16:27:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0a17fa700 10 client.18771 handle_command_reply: tid=0 2023-07-14T16:27:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 client.18771 shutdown 2023-07-14T16:27:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 -- 172.21.15.62:0/254043008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 msgr2=0x563509443f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509443f20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005d10 tx=0x7fe0b40092f0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 -- 172.21.15.62:0/254043008 shutdown_connections 2023-07-14T16:27:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56350944f350 0x563509451760 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x56350944af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x5635094439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0a17fa700 0 client.18771 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/254043008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509443f20 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 -- 172.21.15.62:0/254043008 >> 172.21.15.62:0/254043008 conn(0x56350942fd40 msgr2=0x563509434640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:17.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 -- 172.21.15.62:0/254043008 shutdown_connections 2023-07-14T16:27:17.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.553+0000 7fe0d37aab80 1 -- 172.21.15.62:0/254043008 wait complete. 2023-07-14T16:27:17.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.554+0000 7fe0d37aab80 20 client.18771 trim_cache size 0 max 16384 2023-07-14T16:27:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0d37aab80 1 Processor -- start 2023-07-14T16:27:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0d37aab80 1 -- start start 2023-07-14T16:27:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x563509434e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x56350944af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x5635094560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635094366a0 con 0x56350943aa90 2023-07-14T16:27:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350943b0d0 con 0x56350943b7e0 2023-07-14T16:27:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563509435380 con 0x5635092966b0 2023-07-14T16:27:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.572+0000 7fe0bdb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x56350944af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bdb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x56350944af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59348/0 (socket says 172.21.15.62:59348) 2023-07-14T16:27:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/1383880209 learned_addr learned my addr 172.21.15.62:0/1383880209 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x5635094560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x563509434e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/1383880209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 msgr2=0x563509434e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x563509434e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/1383880209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 msgr2=0x5635094560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x5635094560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/1383880209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635091a2c00 con 0x56350943aa90 2023-07-14T16:27:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x5635094560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.573+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x56350944af20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005c00 tx=0x7fe0b4010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd43353c99af708e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0acbff700 1 -- 172.21.15.62:0/1383880209 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b401d040 con 0x56350943aa90 2023-07-14T16:27:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0acbff700 1 -- 172.21.15.62:0/1383880209 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0b402c450 con 0x56350943aa90 2023-07-14T16:27:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0acbff700 1 -- 172.21.15.62:0/1383880209 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b402ca60 con 0x56350943aa90 2023-07-14T16:27:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1383880209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 msgr2=0x56350944af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x56350944af20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005c00 tx=0x7fe0b4010fc0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1383880209 shutdown_connections 2023-07-14T16:27:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x5635094560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x563509434e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1383880209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x56350944af20 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.574+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1383880209 >> 172.21.15.62:0/1383880209 conn(0x56350942fd40 msgr2=0x563509442730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.575+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1383880209 shutdown_connections 2023-07-14T16:27:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.575+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1383880209 wait complete. 2023-07-14T16:27:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.575+0000 7fe0d37aab80 1 -- start start 2023-07-14T16:27:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.575+0000 7fe0d37aab80 1 Processor -- start 2023-07-14T16:27:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x563509446830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509446d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0d37aab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x563509449d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350944a2d0 con 0x56350943aa90 2023-07-14T16:27:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350944a440 con 0x56350943b7e0 2023-07-14T16:27:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0d37aab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350944a600 con 0x5635092966b0 2023-07-14T16:27:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0d37aab80 10 client.0 _ll_register_callbacks cb 0x7fe0a800aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0bdb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509446d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0bdb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509446d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59360/0 (socket says 172.21.15.62:59360) 2023-07-14T16:27:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.576+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/1432678663 learned_addr learned my addr 172.21.15.62:0/1432678663 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x563509449d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x563509446830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/1432678663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 msgr2=0x563509446830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x563509446830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 msgr2=0x563509449d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x563509449d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bdb9b700 1 -- 172.21.15.62:0/1432678663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b81086b0 con 0x56350943aa90 2023-07-14T16:27:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x563509449d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0bdb9b700 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509446d70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005d10 tx=0x7fe0b402cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf904432d2be083 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0a17fa700 1 -- 172.21.15.62:0/1432678663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b401d040 con 0x56350943aa90 2023-07-14T16:27:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0a17fa700 1 -- 172.21.15.62:0/1432678663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe0b4000e40 con 0x56350943aa90 2023-07-14T16:27:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.577+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1432678663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0b401e040 con 0x56350943aa90 2023-07-14T16:27:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0a17fa700 1 -- 172.21.15.62:0/1432678663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b4047070 con 0x56350943aa90 2023-07-14T16:27:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0d37aab80 10 client.18777 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1432678663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5635091e01f0 con 0x56350943aa90 2023-07-14T16:27:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0a17fa700 1 -- 172.21.15.62:0/1432678663 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0b401e040 con 0x56350943aa90 2023-07-14T16:27:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0a17fa700 1 -- 172.21.15.62:0/1432678663 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe0b4004080 con 0x56350943aa90 2023-07-14T16:27:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0d37aab80 10 client.18777 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0d37aab80 10 client.18777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0d37aab80 20 client.18777 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5635094a3940 0x5635094a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0d37aab80 4 client.18777 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:27:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.578+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1432678663 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563509346ff0 con 0x5635094a3940 2023-07-14T16:27:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.579+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5635094a3940 0x5635094a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.579+0000 7fe0a17fa700 10 client.18777 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.579+0000 7fe0bd39a700 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5635094a3940 0x5635094a5d20 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68002cd5a6ffe923 server_cookie=15356979766a0107 in_seq=0 out_seq=0 2023-07-14T16:27:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.588+0000 7fe0a17fa700 1 -- 172.21.15.62:0/1432678663 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563509346ff0 con 0x5635094a3940 2023-07-14T16:27:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.588+0000 7fe0a17fa700 10 client.18777 handle_command_reply: tid=0 2023-07-14T16:27:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.588+0000 7fe0d37aab80 1 client.18777 shutdown 2023-07-14T16:27:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1432678663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 msgr2=0x563509446d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509446d70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005d10 tx=0x7fe0b402cb60 comp rx=0 tx=0).stop 2023-07-14T16:27:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1432678663 shutdown_connections 2023-07-14T16:27:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5635094a3940 0x5635094a5d20 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56350943b7e0 0x563509449d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5635092966b0 0x563509446830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 --2- 172.21.15.62:0/1432678663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56350943aa90 0x563509446d70 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0a17fa700 0 client.18777 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1432678663 >> 172.21.15.62:0/1432678663 conn(0x56350942fd40 msgr2=0x5635094560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1432678663 shutdown_connections 2023-07-14T16:27:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 1 -- 172.21.15.62:0/1432678663 wait complete. 2023-07-14T16:27:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.589+0000 7fe0d37aab80 20 client.18777 trim_cache size 0 max 16384 2023-07-14T16:27:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:27:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:27:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:27:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:27:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 -- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe0a40a11d0 msgr2=0x7fe0a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe0a40a11d0 0x7fe0a40a3680 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00a1a50 tx=0x7fe0b000af60 comp rx=0 tx=0).stop 2023-07-14T16:27:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 -- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b81014c0 msgr2=0x7fe0b8072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b81014c0 0x7fe0b8072e40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0007d70 tx=0x7fe0b000a340 comp rx=0 tx=0).stop 2023-07-14T16:27:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 -- 172.21.15.62:0/3298994191 shutdown_connections 2023-07-14T16:27:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b81014c0 0x7fe0b8072e40 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe0a40a11d0 0x7fe0a40a3680 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0b80ff940 0x7fe0b80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 --2- 172.21.15.62:0/3298994191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0b8100b30 0x7fe0b8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 -- 172.21.15.62:0/3298994191 >> 172.21.15.62:0/3298994191 conn(0x7fe0b80faf60 msgr2=0x7fe0b8103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 -- 172.21.15.62:0/3298994191 shutdown_connections 2023-07-14T16:27:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:17.590+0000 7fe0beb9d700 1 -- 172.21.15.62:0/3298994191 wait complete. 2023-07-14T16:27:19.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:18 smithi062 ceph-mon[95543]: pgmap v707: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:27:19.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:18 smithi081 ceph-mon[127795]: pgmap v707: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:27:19.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:18 smithi164 ceph-mon[97427]: pgmap v707: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:27:21.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:20 smithi062 ceph-mon[95543]: pgmap v708: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:27:21.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:20 smithi081 ceph-mon[127795]: pgmap v708: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:27:21.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:20 smithi164 ceph-mon[97427]: pgmap v708: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:27:23.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:22 smithi062 ceph-mon[95543]: pgmap v709: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:27:23.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:22 smithi081 ceph-mon[127795]: pgmap v709: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:27:23.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:22 smithi164 ceph-mon[97427]: pgmap v709: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:27:25.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:24 smithi062 ceph-mon[95543]: pgmap v710: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:27:25.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:24 smithi081 ceph-mon[127795]: pgmap v710: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:27:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:24 smithi164 ceph-mon[97427]: pgmap v710: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:27:27.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:26 smithi062 ceph-mon[95543]: pgmap v711: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:27:27.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:26 smithi081 ceph-mon[127795]: pgmap v711: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:27:27.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:26 smithi164 ceph-mon[97427]: pgmap v711: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:27:29.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:28 smithi062 ceph-mon[95543]: pgmap v712: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:27:29.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:28 smithi081 ceph-mon[127795]: pgmap v712: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:27:29.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:28 smithi164 ceph-mon[97427]: pgmap v712: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:27:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 -- 172.21.15.62:0/4241324310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 msgr2=0x7f1f70107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 --2- 172.21.15.62:0/4241324310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 0x7f1f70107c40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009ec0 tx=0x7f1f6800b330 comp rx=0 tx=0).stop 2023-07-14T16:27:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 -- 172.21.15.62:0/4241324310 shutdown_connections 2023-07-14T16:27:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 --2- 172.21.15.62:0/4241324310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 0x7f1f70107c40 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 --2- 172.21.15.62:0/4241324310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f70100620 0x7f1f70102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 --2- 172.21.15.62:0/4241324310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f70102f40 0x7f1f70105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 -- 172.21.15.62:0/4241324310 >> 172.21.15.62:0/4241324310 conn(0x7f1f700fa000 msgr2=0x7f1f700fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 -- 172.21.15.62:0/4241324310 shutdown_connections 2023-07-14T16:27:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.756+0000 7f1f783ff700 1 -- 172.21.15.62:0/4241324310 wait complete. 2023-07-14T16:27:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.757+0000 7f1f783ff700 1 Processor -- start 2023-07-14T16:27:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f783ff700 1 -- start start 2023-07-14T16:27:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f783ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f70100620 0x7f1f70197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f783ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f70102f40 0x7f1f70198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f783ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 0x7f1f70198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f783ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f70199570 con 0x7f1f70105860 2023-07-14T16:27:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f783ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f701996e0 con 0x7f1f70102f40 2023-07-14T16:27:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f783ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f701998a0 con 0x7f1f70100620 2023-07-14T16:27:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f7699c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 0x7f1f70198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f7699c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 0x7f1f70198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51450/0 (socket says 172.21.15.62:51450) 2023-07-14T16:27:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f7699c700 1 -- 172.21.15.62:0/3225391018 learned_addr learned my addr 172.21.15.62:0/3225391018 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f7619b700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f70100620 0x7f1f70197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.758+0000 7f1f7599a700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f70102f40 0x7f1f70198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.759+0000 7f1f7699c700 1 -- 172.21.15.62:0/3225391018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f70100620 msgr2=0x7f1f70197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.759+0000 7f1f7699c700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f70100620 0x7f1f70197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.759+0000 7f1f7699c700 1 -- 172.21.15.62:0/3225391018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f70102f40 msgr2=0x7f1f70198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.759+0000 7f1f7699c700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f70102f40 0x7f1f70198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.759+0000 7f1f7699c700 1 -- 172.21.15.62:0/3225391018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f68005e70 con 0x7f1f70105860 2023-07-14T16:27:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.759+0000 7f1f7699c700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 0x7f1f70198f10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f1f68006070 tx=0x7f1f6800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.759+0000 7f1f677fe700 1 -- 172.21.15.62:0/3225391018 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6800e040 con 0x7f1f70105860 2023-07-14T16:27:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.760+0000 7f1f783ff700 1 -- 172.21.15.62:0/3225391018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f70199b20 con 0x7f1f70105860 2023-07-14T16:27:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.760+0000 7f1f677fe700 1 -- 172.21.15.62:0/3225391018 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f6800ced0 con 0x7f1f70105860 2023-07-14T16:27:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.760+0000 7f1f677fe700 1 -- 172.21.15.62:0/3225391018 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6801b980 con 0x7f1f70105860 2023-07-14T16:27:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.760+0000 7f1f783ff700 1 -- 172.21.15.62:0/3225391018 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f701a0340 con 0x7f1f70105860 2023-07-14T16:27:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.763+0000 7f1f677fe700 1 -- 172.21.15.62:0/3225391018 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1f6801bb80 con 0x7f1f70105860 2023-07-14T16:27:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.765+0000 7f1f8bd72b80 1 Processor -- start 2023-07-14T16:27:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.765+0000 7f1f8bd72b80 1 -- start start 2023-07-14T16:27:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4198f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f677fe700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1f5c0a1150 0x7f1f5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 0x559a419901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 0x559a41990b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a41894c50 con 0x559a417eb6b0 2023-07-14T16:27:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a418e79e0 con 0x559a419906e0 2023-07-14T16:27:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a418e0e70 con 0x559a4198f840 2023-07-14T16:27:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f677fe700 1 -- 172.21.15.62:0/3225391018 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1f680ce6f0 con 0x7f1f70105860 2023-07-14T16:27:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f7599a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 0x559a419901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f7599a700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1f5c0a1150 0x7f1f5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f7619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4198f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f7619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4198f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51456/0 (socket says 172.21.15.62:51456) 2023-07-14T16:27:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.766+0000 7f1f7619b700 1 -- 172.21.15.62:0/2198521803 learned_addr learned my addr 172.21.15.62:0/2198521803 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f7699c700 1 --2- 172.21.15.62:0/2198521803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 0x559a41990b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f7619b700 1 -- 172.21.15.62:0/2198521803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 msgr2=0x559a419901a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f7619b700 1 --2- 172.21.15.62:0/2198521803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 0x559a419901a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f7619b700 1 -- 172.21.15.62:0/2198521803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 msgr2=0x559a41990b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f7619b700 1 --2- 172.21.15.62:0/2198521803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 0x559a41990b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f7619b700 1 -- 172.21.15.62:0/2198521803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a4170ed80 con 0x559a417eb6b0 2023-07-14T16:27:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f7599a700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1f5c0a1150 0x7f1f5c0a3600 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1f60007fd0 tx=0x7f1f600073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f7619b700 1 --2- 172.21.15.62:0/2198521803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4198f300 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009d20 tx=0x7f1f6c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dbf33f690f9105 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f64bfe700 1 -- 172.21.15.62:0/2198521803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c019630 con 0x559a417eb6b0 2023-07-14T16:27:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f64bfe700 1 -- 172.21.15.62:0/2198521803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f6c019c20 con 0x559a417eb6b0 2023-07-14T16:27:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.767+0000 7f1f64bfe700 1 -- 172.21.15.62:0/2198521803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00eb60 con 0x559a417eb6b0 2023-07-14T16:27:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2198521803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 msgr2=0x559a4198f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2198521803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4198f300 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009d20 tx=0x7f1f6c011720 comp rx=0 tx=0).stop 2023-07-14T16:27:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2198521803 shutdown_connections 2023-07-14T16:27:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2198521803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 0x559a41990b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2198521803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4198f300 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2198521803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 0x559a419901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2198521803 >> 172.21.15.62:0/2198521803 conn(0x559a41984be0 msgr2=0x559a41989cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2198521803 shutdown_connections 2023-07-14T16:27:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.768+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2198521803 wait complete. 2023-07-14T16:27:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.769+0000 7f1f8bd72b80 1 -- start start 2023-07-14T16:27:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.769+0000 7f1f8bd72b80 1 Processor -- start 2023-07-14T16:27:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a41998870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 0x559a41998db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 0x559a4199fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a41880a60 con 0x559a417eb6b0 2023-07-14T16:27:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a41880760 con 0x559a419906e0 2023-07-14T16:27:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a417648c0 con 0x559a4198f840 2023-07-14T16:27:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f8bd72b80 10 client.0 _ll_register_callbacks cb 0x559a4187e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f7619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a41998870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f7619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a41998870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51466/0 (socket says 172.21.15.62:51466) 2023-07-14T16:27:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f7619b700 1 -- 172.21.15.62:0/3763609294 learned_addr learned my addr 172.21.15.62:0/3763609294 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.770+0000 7f1f7599a700 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 0x559a41998db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f7699c700 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 0x559a4199fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f7619b700 1 -- 172.21.15.62:0/3763609294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 msgr2=0x559a41998db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f7619b700 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 0x559a41998db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f7619b700 1 -- 172.21.15.62:0/3763609294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 msgr2=0x559a4199fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f7619b700 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 0x559a4199fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f7619b700 1 -- 172.21.15.62:0/3763609294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c009980 con 0x559a417eb6b0 2023-07-14T16:27:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f7619b700 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a41998870 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00ee80 tx=0x7f1f6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8eba36fe8fbfbc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f59ffb700 1 -- 172.21.15.62:0/3763609294 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c019840 con 0x559a417eb6b0 2023-07-14T16:27:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/3763609294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a4183a490 con 0x559a417eb6b0 2023-07-14T16:27:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.771+0000 7f1f59ffb700 1 -- 172.21.15.62:0/3763609294 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1f6c005ce0 con 0x559a417eb6b0 2023-07-14T16:27:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f59ffb700 1 -- 172.21.15.62:0/3763609294 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f6c0199a0 con 0x559a417eb6b0 2023-07-14T16:27:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f8bd72b80 10 client.18789 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/3763609294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559a416f7c00 con 0x559a417eb6b0 2023-07-14T16:27:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f59ffb700 1 -- 172.21.15.62:0/3763609294 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559a4183a490 con 0x559a417eb6b0 2023-07-14T16:27:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f59ffb700 1 -- 172.21.15.62:0/3763609294 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1f6c033070 con 0x559a417eb6b0 2023-07-14T16:27:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f8bd72b80 10 client.18789 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f8bd72b80 10 client.18789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f8bd72b80 20 client.18789 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a419a44d0 0x559a419a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f8bd72b80 4 client.18789 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.772+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/3763609294 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a418d9e50 con 0x559a419a44d0 2023-07-14T16:27:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.773+0000 7f1f7599a700 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a419a44d0 0x559a419a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.774+0000 7f1f59ffb700 10 client.18789 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.774+0000 7f1f7599a700 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a419a44d0 0x559a419a68e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4636e0f86161c5 server_cookie=9360de81abdef8a6 in_seq=0 out_seq=0 2023-07-14T16:27:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.774+0000 7f1f59ffb700 1 -- 172.21.15.62:0/3763609294 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a418d9e50 con 0x559a419a44d0 2023-07-14T16:27:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.774+0000 7f1f59ffb700 10 client.18789 handle_command_reply: tid=0 2023-07-14T16:27:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.774+0000 7f1f8bd72b80 1 client.18789 shutdown 2023-07-14T16:27:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/3763609294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 msgr2=0x559a41998870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a41998870 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00ee80 tx=0x7f1f6c00eeb0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/3763609294 shutdown_connections 2023-07-14T16:27:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a419a44d0 0x559a419a68e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a419906e0 0x559a4199fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a41998870 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f59ffb700 0 client.18789 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/3763609294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a4198f840 0x559a41998db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/3763609294 >> 172.21.15.62:0/3763609294 conn(0x559a41984be0 msgr2=0x559a419894e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/3763609294 shutdown_connections 2023-07-14T16:27:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/3763609294 wait complete. 2023-07-14T16:27:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.775+0000 7f1f8bd72b80 20 client.18789 trim_cache size 0 max 16384 2023-07-14T16:27:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.791+0000 7f1f8bd72b80 1 Processor -- start 2023-07-14T16:27:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.791+0000 7f1f8bd72b80 1 -- start start 2023-07-14T16:27:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.791+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a419ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.791+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 0x559a4199fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.791+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a419935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.791+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a41998440 con 0x559a417eb6b0 2023-07-14T16:27:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.791+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a41996870 con 0x559a4198f840 2023-07-14T16:27:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.791+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a41994160 con 0x559a419906e0 2023-07-14T16:27:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a419ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a419ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51480/0 (socket says 172.21.15.62:51480) 2023-07-14T16:27:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7619b700 1 -- 172.21.15.62:0/4122855677 learned_addr learned my addr 172.21.15.62:0/4122855677 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7599a700 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 0x559a4199fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7699c700 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a419935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7619b700 1 -- 172.21.15.62:0/4122855677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 msgr2=0x559a419935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7619b700 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a419935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7619b700 1 -- 172.21.15.62:0/4122855677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 msgr2=0x559a4199fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7619b700 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 0x559a4199fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.792+0000 7f1f7619b700 1 -- 172.21.15.62:0/4122855677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a417351f0 con 0x559a417eb6b0 2023-07-14T16:27:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.793+0000 7f1f7619b700 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a419ab220 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009c70 tx=0x7f1f6c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c0036d33446f68 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.793+0000 7f1f653ff700 1 -- 172.21.15.62:0/4122855677 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c005c20 con 0x559a417eb6b0 2023-07-14T16:27:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.793+0000 7f1f653ff700 1 -- 172.21.15.62:0/4122855677 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f6c030070 con 0x559a417eb6b0 2023-07-14T16:27:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.793+0000 7f1f653ff700 1 -- 172.21.15.62:0/4122855677 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c022b60 con 0x559a417eb6b0 2023-07-14T16:27:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.793+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/4122855677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 msgr2=0x559a419ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.793+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a419ab220 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009c70 tx=0x7f1f6c011720 comp rx=0 tx=0).stop 2023-07-14T16:27:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/4122855677 shutdown_connections 2023-07-14T16:27:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a419935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a419ab220 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/4122855677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 0x559a4199fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/4122855677 >> 172.21.15.62:0/4122855677 conn(0x559a41984be0 msgr2=0x559a419975a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/4122855677 shutdown_connections 2023-07-14T16:27:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/4122855677 wait complete. 2023-07-14T16:27:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 -- start start 2023-07-14T16:27:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.794+0000 7f1f8bd72b80 1 Processor -- start 2023-07-14T16:27:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4199b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 0x559a4199bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f8bd72b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a4199ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4199f1d0 con 0x559a417eb6b0 2023-07-14T16:27:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4199f340 con 0x559a4198f840 2023-07-14T16:27:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f8bd72b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4199f500 con 0x559a419906e0 2023-07-14T16:27:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f8bd72b80 10 client.0 _ll_register_callbacks cb 0x7f1f68009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:30.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f7619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4199b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f7619b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4199b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51496/0 (socket says 172.21.15.62:51496) 2023-07-14T16:27:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f7619b700 1 -- 172.21.15.62:0/2444618650 learned_addr learned my addr 172.21.15.62:0/2444618650 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.795+0000 7f1f7599a700 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 0x559a4199bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f7699c700 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a4199ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f7619b700 1 -- 172.21.15.62:0/2444618650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 msgr2=0x559a4199ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f7619b700 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a4199ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f7619b700 1 -- 172.21.15.62:0/2444618650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 msgr2=0x559a4199bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f7619b700 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 0x559a4199bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f7619b700 1 -- 172.21.15.62:0/2444618650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a416f7c00 con 0x559a417eb6b0 2023-07-14T16:27:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f7699c700 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a4199ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:27:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f7619b700 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4199b730 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c005a80 tx=0x7f1f6c0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734126d127b3e9e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f59ffb700 1 -- 172.21.15.62:0/2444618650 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c019840 con 0x559a417eb6b0 2023-07-14T16:27:30.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2444618650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f6c019c20 con 0x559a417eb6b0 2023-07-14T16:27:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.796+0000 7f1f59ffb700 1 -- 172.21.15.62:0/2444618650 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1f6c000e40 con 0x559a417eb6b0 2023-07-14T16:27:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f59ffb700 1 -- 172.21.15.62:0/2444618650 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f6c0199a0 con 0x559a417eb6b0 2023-07-14T16:27:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f8bd72b80 10 client.18795 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2444618650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559a4170d250 con 0x559a417eb6b0 2023-07-14T16:27:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f59ffb700 1 -- 172.21.15.62:0/2444618650 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f6c04d020 con 0x559a417eb6b0 2023-07-14T16:27:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f59ffb700 1 -- 172.21.15.62:0/2444618650 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1f6c011790 con 0x559a417eb6b0 2023-07-14T16:27:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f8bd72b80 10 client.18795 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f8bd72b80 10 client.18795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f8bd72b80 20 client.18795 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a419f8ab0 0x559a419faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f8bd72b80 4 client.18795 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:27:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.797+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2444618650 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559a4189bff0 con 0x559a419f8ab0 2023-07-14T16:27:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.798+0000 7f1f7599a700 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a419f8ab0 0x559a419faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.798+0000 7f1f7599a700 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a419f8ab0 0x559a419faec0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b8efeb163b5348 server_cookie=630d1d20d7b3babd in_seq=0 out_seq=0 2023-07-14T16:27:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.798+0000 7f1f59ffb700 10 client.18795 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f59ffb700 1 -- 172.21.15.62:0/2444618650 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559a4189bff0 con 0x559a419f8ab0 2023-07-14T16:27:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f59ffb700 10 client.18795 handle_command_reply: tid=0 2023-07-14T16:27:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 client.18795 shutdown 2023-07-14T16:27:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2444618650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 msgr2=0x559a4199b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4199b730 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c005a80 tx=0x7f1f6c0096d0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2444618650 shutdown_connections 2023-07-14T16:27:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a419906e0 0x559a4199ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a417eb6b0 0x559a4199b730 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a419f8ab0 0x559a419faec0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 --2- 172.21.15.62:0/2444618650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a4198f840 0x559a4199bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f59ffb700 0 client.18795 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2444618650 >> 172.21.15.62:0/2444618650 conn(0x559a41984be0 msgr2=0x559a419ab220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2444618650 shutdown_connections 2023-07-14T16:27:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 1 -- 172.21.15.62:0/2444618650 wait complete. 2023-07-14T16:27:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.808+0000 7f1f8bd72b80 20 client.18795 trim_cache size 0 max 16384 2023-07-14T16:27:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:27:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:27:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:27:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:27:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 -- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1f5c0a1150 msgr2=0x7f1f5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1f5c0a1150 0x7f1f5c0a3600 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1f60007fd0 tx=0x7f1f600073d0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 -- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 msgr2=0x7f1f70198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 0x7f1f70198f10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f1f68006070 tx=0x7f1f6800ca60 comp rx=0 tx=0).stop 2023-07-14T16:27:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 -- 172.21.15.62:0/3225391018 shutdown_connections 2023-07-14T16:27:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f70105860 0x7f1f70198f10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f70100620 0x7f1f70197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1f5c0a1150 0x7f1f5c0a3600 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 --2- 172.21.15.62:0/3225391018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f70102f40 0x7f1f70198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 -- 172.21.15.62:0/3225391018 >> 172.21.15.62:0/3225391018 conn(0x7f1f700fa000 msgr2=0x7f1f700feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 -- 172.21.15.62:0/3225391018 shutdown_connections 2023-07-14T16:27:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:30.809+0000 7f1f783ff700 1 -- 172.21.15.62:0/3225391018 wait complete. 2023-07-14T16:27:31.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:30 smithi062 ceph-mon[95543]: pgmap v713: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:27:31.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:30 smithi081 ceph-mon[127795]: pgmap v713: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:27:31.266 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:27:31.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:30 smithi164 ceph-mon[97427]: pgmap v713: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:27:31.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.545+0000 7ffbb4fff700 1 -- 172.21.15.62:0/1322361784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb00ffae0 msgr2=0x7ffbb00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.545+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/1322361784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb00ffae0 0x7ffbb00ffef0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009990 tx=0x7ffba0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.546+0000 7ffbb4fff700 1 -- 172.21.15.62:0/1322361784 shutdown_connections 2023-07-14T16:27:31.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.546+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/1322361784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0101660 0x7ffbb0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.546+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/1322361784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb00ffae0 0x7ffbb00ffef0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.546+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/1322361784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb0100cd0 0x7ffbb0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.546+0000 7ffbb4fff700 1 -- 172.21.15.62:0/1322361784 >> 172.21.15.62:0/1322361784 conn(0x7ffbb00fb0e0 msgr2=0x7ffbb00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:31.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.546+0000 7ffbb4fff700 1 -- 172.21.15.62:0/1322361784 shutdown_connections 2023-07-14T16:27:31.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.546+0000 7ffbb4fff700 1 -- 172.21.15.62:0/1322361784 wait complete. 2023-07-14T16:27:31.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.547+0000 7ffbb4fff700 1 Processor -- start 2023-07-14T16:27:31.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.547+0000 7ffbb4fff700 1 -- start start 2023-07-14T16:27:31.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbb4fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb00ffae0 0x7ffbb0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbb4fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100cd0 0x7ffbb0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbb4fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0101660 0x7ffbb019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbb4fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019f4c0 con 0x7ffbb0101660 2023-07-14T16:27:31.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbb4fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019f600 con 0x7ffbb0100cd0 2023-07-14T16:27:31.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbb4fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019f790 con 0x7ffbb00ffae0 2023-07-14T16:27:31.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbaed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0101660 0x7ffbb019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbae59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb00ffae0 0x7ffbb0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbaed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0101660 0x7ffbb019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51518/0 (socket says 172.21.15.62:51518) 2023-07-14T16:27:31.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbaed9d700 1 -- 172.21.15.62:0/3271990070 learned_addr learned my addr 172.21.15.62:0/3271990070 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:31.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbae59c700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb00ffae0 0x7ffbb0197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41188/0 (socket says 172.21.15.62:41188) 2023-07-14T16:27:31.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.548+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100cd0 0x7ffbb0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.549+0000 7ffbaed9d700 1 -- 172.21.15.62:0/3271990070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb00ffae0 msgr2=0x7ffbb0197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.549+0000 7ffbaed9d700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb00ffae0 0x7ffbb0197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.549+0000 7ffbaed9d700 1 -- 172.21.15.62:0/3271990070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100cd0 msgr2=0x7ffbb0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.549+0000 7ffbaed9d700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100cd0 0x7ffbb0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.549+0000 7ffbaed9d700 1 -- 172.21.15.62:0/3271990070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00095f0 con 0x7ffbb0101660 2023-07-14T16:27:31.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.549+0000 7ffbaed9d700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0101660 0x7ffbb019eef0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ffba8007ee0 tx=0x7ffba800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:31.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.550+0000 7ffb9f7fe700 1 -- 172.21.15.62:0/3271990070 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8017070 con 0x7ffbb0101660 2023-07-14T16:27:31.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.550+0000 7ffb9f7fe700 1 -- 172.21.15.62:0/3271990070 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba800f040 con 0x7ffbb0101660 2023-07-14T16:27:31.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.550+0000 7ffbb4fff700 1 -- 172.21.15.62:0/3271990070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb019fa70 con 0x7ffbb0101660 2023-07-14T16:27:31.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.550+0000 7ffb9f7fe700 1 -- 172.21.15.62:0/3271990070 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8005570 con 0x7ffbb0101660 2023-07-14T16:27:31.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.550+0000 7ffbb4fff700 1 -- 172.21.15.62:0/3271990070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb019ffd0 con 0x7ffbb0101660 2023-07-14T16:27:31.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.554+0000 7ffb9f7fe700 1 -- 172.21.15.62:0/3271990070 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffba800db20 con 0x7ffbb0101660 2023-07-14T16:27:31.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.555+0000 7ffbc8847b80 1 Processor -- start 2023-07-14T16:27:31.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.555+0000 7ffbc8847b80 1 -- start start 2023-07-14T16:27:31.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be785e8cb0 con 0x55be786e3e20 2023-07-14T16:27:31.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7863ba40 con 0x55be786e4c70 2023-07-14T16:27:31.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be78634ed0 con 0x55be786e34d0 2023-07-14T16:27:31.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbadd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbadd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786e38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41190/0 (socket says 172.21.15.62:41190) 2023-07-14T16:27:31.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbadd9b700 1 -- 172.21.15.62:0/177631463 learned_addr learned my addr 172.21.15.62:0/177631463 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:31.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbae59c700 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffb9f7fe700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a11d0 0x7ffb980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbadd9b700 1 -- 172.21.15.62:0/177631463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 msgr2=0x55be786e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:31.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.556+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffbadd9b700 1 -- 172.21.15.62:0/177631463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 msgr2=0x55be786e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:31.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786e46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffbadd9b700 1 -- 172.21.15.62:0/177631463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be78462de0 con 0x55be786e34d0 2023-07-14T16:27:31.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffb9f7fe700 1 -- 172.21.15.62:0/3271990070 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffba80cbf80 con 0x7ffbb0101660 2023-07-14T16:27:31.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffbaed9d700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a11d0 0x7ffb980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786e38e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffba4011860 tx=0x7ffba400a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e39a650dcdfde0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:31.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffb9cff9700 1 -- 172.21.15.62:0/177631463 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4011e80 con 0x55be786e34d0 2023-07-14T16:27:31.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffb9cff9700 1 -- 172.21.15.62:0/177631463 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba4004830 con 0x55be786e34d0 2023-07-14T16:27:31.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.557+0000 7ffb9cff9700 1 -- 172.21.15.62:0/177631463 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4005410 con 0x55be786e34d0 2023-07-14T16:27:31.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.558+0000 7ffbaed9d700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a11d0 0x7ffb980a3680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffba80990a0 tx=0x7ffba8095040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:31.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.558+0000 7ffbc8847b80 1 -- 172.21.15.62:0/177631463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 msgr2=0x55be786e38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.558+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786e38e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffba4011860 tx=0x7ffba400a7f0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.558+0000 7ffbc8847b80 1 -- 172.21.15.62:0/177631463 shutdown_connections 2023-07-14T16:27:31.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.558+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.558+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786e38e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.558+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/177631463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.559+0000 7ffbc8847b80 1 -- 172.21.15.62:0/177631463 >> 172.21.15.62:0/177631463 conn(0x55be786d8da0 msgr2=0x55be786de1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:31.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.559+0000 7ffbc8847b80 1 -- 172.21.15.62:0/177631463 shutdown_connections 2023-07-14T16:27:31.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.559+0000 7ffbc8847b80 1 -- 172.21.15.62:0/177631463 wait complete. 2023-07-14T16:27:31.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.559+0000 7ffbc8847b80 1 -- start start 2023-07-14T16:27:31.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.560+0000 7ffbc8847b80 1 Processor -- start 2023-07-14T16:27:31.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.560+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786ed100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e3e20 0x55be786ed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e4c70 0x55be786f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be785d4ac0 con 0x55be786e34d0 2023-07-14T16:27:31.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be785d47c0 con 0x55be786e3e20 2023-07-14T16:27:31.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be784b8920 con 0x55be786e4c70 2023-07-14T16:27:31.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbc8847b80 10 client.0 _ll_register_callbacks cb 0x55be785d2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:31.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbadd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbadd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786ed100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51526/0 (socket says 172.21.15.62:51526) 2023-07-14T16:27:31.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbadd9b700 1 -- 172.21.15.62:0/2311733513 learned_addr learned my addr 172.21.15.62:0/2311733513 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:31.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbae59c700 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e3e20 0x55be786ed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e4c70 0x55be786f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbae59c700 1 -- 172.21.15.62:0/2311733513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e4c70 msgr2=0x55be786f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbae59c700 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e4c70 0x55be786f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbae59c700 1 -- 172.21.15.62:0/2311733513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 msgr2=0x55be786ed100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbae59c700 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786ed100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.561+0000 7ffbae59c700 1 -- 172.21.15.62:0/2311733513 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba4019040 con 0x55be786e3e20 2023-07-14T16:27:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.562+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e4c70 0x55be786f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:27:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.562+0000 7ffbae59c700 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e3e20 0x55be786ed640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffba0000c00 tx=0x7ffba0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e595abdc743940df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:31.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.562+0000 7ffb95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:27:31.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.562+0000 7ffb95ffb700 1 -- 172.21.15.62:0/2311733513 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba001d070 con 0x55be786e3e20 2023-07-14T16:27:31.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.562+0000 7ffbc8847b80 1 -- 172.21.15.62:0/2311733513 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be7858e4f0 con 0x55be786e3e20 2023-07-14T16:27:31.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.562+0000 7ffb95ffb700 1 -- 172.21.15.62:0/2311733513 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffba0005620 con 0x55be786e3e20 2023-07-14T16:27:31.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffb95ffb700 1 -- 172.21.15.62:0/2311733513 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffba0004500 con 0x55be786e3e20 2023-07-14T16:27:31.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffbc8847b80 10 client.28400 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:31.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffbc8847b80 1 -- 172.21.15.62:0/2311733513 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be7844bc60 con 0x55be786e3e20 2023-07-14T16:27:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffb95ffb700 1 -- 172.21.15.62:0/2311733513 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55be7858e4f0 con 0x55be786e3e20 2023-07-14T16:27:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffb95ffb700 1 -- 172.21.15.62:0/2311733513 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffba0003a20 con 0x55be786e3e20 2023-07-14T16:27:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffbc8847b80 10 client.28400 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffbc8847b80 10 client.28400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffbc8847b80 20 client.28400 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55be786f8e50 0x55be786fb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffbc8847b80 4 client.28400 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.563+0000 7ffbc8847b80 1 -- 172.21.15.62:0/2311733513 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be7862deb0 con 0x55be786f8e50 2023-07-14T16:27:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.564+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55be786f8e50 0x55be786fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.564+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55be786f8e50 0x55be786fb260 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d960d94d6534e85e server_cookie=448e2f04932a31b8 in_seq=0 out_seq=0 2023-07-14T16:27:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.564+0000 7ffb95ffb700 10 client.28400 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.565+0000 7ffb95ffb700 1 -- 172.21.15.62:0/2311733513 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55be7862deb0 con 0x55be786f8e50 2023-07-14T16:27:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.565+0000 7ffb95ffb700 10 client.28400 handle_command_reply: tid=0 2023-07-14T16:27:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.565+0000 7ffbc8847b80 1 client.28400 shutdown 2023-07-14T16:27:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.565+0000 7ffbc8847b80 1 -- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e3e20 msgr2=0x55be786ed640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.565+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e3e20 0x55be786ed640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffba0000c00 tx=0x7ffba0005270 comp rx=0 tx=0).stop 2023-07-14T16:27:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 1 -- 172.21.15.62:0/2311733513 shutdown_connections 2023-07-14T16:27:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e4c70 0x55be786f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786ed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55be786f8e50 0x55be786fb260 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/2311733513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e3e20 0x55be786ed640 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 1 -- 172.21.15.62:0/2311733513 >> 172.21.15.62:0/2311733513 conn(0x55be786d8da0 msgr2=0x55be786dd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffb95ffb700 0 client.28400 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 1 -- 172.21.15.62:0/2311733513 shutdown_connections 2023-07-14T16:27:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 1 -- 172.21.15.62:0/2311733513 wait complete. 2023-07-14T16:27:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.566+0000 7ffbc8847b80 20 client.28400 trim_cache size 0 max 16384 2023-07-14T16:27:31.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.585+0000 7ffbc8847b80 1 Processor -- start 2023-07-14T16:27:31.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.585+0000 7ffbc8847b80 1 -- start start 2023-07-14T16:27:31.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786fae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e3e20 0x55be786f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be786def20 con 0x55be786e34d0 2023-07-14T16:27:31.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be786eadd0 con 0x55be786e4c70 2023-07-14T16:27:31.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be786f8be0 con 0x55be786e3e20 2023-07-14T16:27:31.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbadd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbadd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786fae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51532/0 (socket says 172.21.15.62:51532) 2023-07-14T16:27:31.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbadd9b700 1 -- 172.21.15.62:0/828475798 learned_addr learned my addr 172.21.15.62:0/828475798 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:31.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbae59c700 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e3e20 0x55be786f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.586+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffbae59c700 1 -- 172.21.15.62:0/828475798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 msgr2=0x55be786eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffbae59c700 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786eb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffbae59c700 1 -- 172.21.15.62:0/828475798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 msgr2=0x55be786fae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffbae59c700 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786fae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffbae59c700 1 -- 172.21.15.62:0/828475798 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be78489250 con 0x55be786e3e20 2023-07-14T16:27:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffbae59c700 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e3e20 0x55be786f4640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffba00098e0 tx=0x7ffba0005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d690db63a627037d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:31.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffb9d7fa700 1 -- 172.21.15.62:0/828475798 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba001d040 con 0x55be786e3e20 2023-07-14T16:27:31.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffb9d7fa700 1 -- 172.21.15.62:0/828475798 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba001e040 con 0x55be786e3e20 2023-07-14T16:27:31.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.587+0000 7ffb9d7fa700 1 -- 172.21.15.62:0/828475798 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0004720 con 0x55be786e3e20 2023-07-14T16:27:31.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 -- 172.21.15.62:0/828475798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e3e20 msgr2=0x55be786f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e3e20 0x55be786f4640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffba00098e0 tx=0x7ffba0005bc0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 -- 172.21.15.62:0/828475798 shutdown_connections 2023-07-14T16:27:31.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e34d0 0x55be786fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/828475798 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e3e20 0x55be786f4640 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 -- 172.21.15.62:0/828475798 >> 172.21.15.62:0/828475798 conn(0x55be786d8da0 msgr2=0x55be786ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:31.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 -- 172.21.15.62:0/828475798 shutdown_connections 2023-07-14T16:27:31.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.588+0000 7ffbc8847b80 1 -- 172.21.15.62:0/828475798 wait complete. 2023-07-14T16:27:31.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.589+0000 7ffbc8847b80 1 -- start start 2023-07-14T16:27:31.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.589+0000 7ffbc8847b80 1 Processor -- start 2023-07-14T16:27:31.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.589+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786f2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.589+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbc8847b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786f3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be786f39a0 con 0x55be786e3e20 2023-07-14T16:27:31.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be786f3b10 con 0x55be786e4c70 2023-07-14T16:27:31.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbc8847b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be786f3cd0 con 0x55be786e34d0 2023-07-14T16:27:31.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbc8847b80 10 client.0 _ll_register_callbacks cb 0x55be785d2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:31.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbadd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbadd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41220/0 (socket says 172.21.15.62:41220) 2023-07-14T16:27:31.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbadd9b700 1 -- 172.21.15.62:0/3650942873 learned_addr learned my addr 172.21.15.62:0/3650942873 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:31.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbae59c700 1 -- 172.21.15.62:0/3650942873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 msgr2=0x55be786f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:31.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbae59c700 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbae59c700 1 -- 172.21.15.62:0/3650942873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 msgr2=0x55be786f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:31.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbae59c700 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786f3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.590+0000 7ffbae59c700 1 -- 172.21.15.62:0/3650942873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be7844bc60 con 0x55be786e3e20 2023-07-14T16:27:31.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.591+0000 7ffbae59c700 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786f2dc0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x55be786d8af0 tx=0x7ffba0005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b74018d2c54edc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:31.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.591+0000 7ffb95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:31.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.591+0000 7ffb95ffb700 1 -- 172.21.15.62:0/3650942873 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba001d040 con 0x55be786e3e20 2023-07-14T16:27:31.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.591+0000 7ffbc8847b80 1 -- 172.21.15.62:0/3650942873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffba4004830 con 0x55be786e3e20 2023-07-14T16:27:31.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.591+0000 7ffb95ffb700 1 -- 172.21.15.62:0/3650942873 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffba0000e40 con 0x55be786e3e20 2023-07-14T16:27:31.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.591+0000 7ffb95ffb700 1 -- 172.21.15.62:0/3650942873 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffba0009290 con 0x55be786e3e20 2023-07-14T16:27:31.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.591+0000 7ffbc8847b80 10 client.18813 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:31.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffbc8847b80 1 -- 172.21.15.62:0/3650942873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be784612b0 con 0x55be786e3e20 2023-07-14T16:27:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffb95ffb700 1 -- 172.21.15.62:0/3650942873 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba4004830 con 0x55be786e3e20 2023-07-14T16:27:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffb95ffb700 1 -- 172.21.15.62:0/3650942873 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffba0038670 con 0x55be786e3e20 2023-07-14T16:27:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffbc8847b80 10 client.18813 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffbc8847b80 10 client.18813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffbc8847b80 20 client.18813 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55be7874d330 0x55be7874f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffbc8847b80 4 client.18813 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:27:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.592+0000 7ffbc8847b80 1 -- 172.21.15.62:0/3650942873 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be785f0050 con 0x55be7874d330 2023-07-14T16:27:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.593+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55be7874d330 0x55be7874f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.593+0000 7ffbadd9b700 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55be7874d330 0x55be7874f710 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3198dac38e21fd2f server_cookie=2590bda0ece0dd5f in_seq=0 out_seq=0 2023-07-14T16:27:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.593+0000 7ffb95ffb700 10 client.18813 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.593+0000 7ffb95ffb700 1 -- 172.21.15.62:0/3650942873 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55be785f0050 con 0x55be7874d330 2023-07-14T16:27:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.593+0000 7ffb95ffb700 10 client.18813 handle_command_reply: tid=0 2023-07-14T16:27:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.593+0000 7ffbc8847b80 1 client.18813 shutdown 2023-07-14T16:27:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 -- 172.21.15.62:0/3650942873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 msgr2=0x55be786f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786f2dc0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x55be786d8af0 tx=0x7ffba0005a80 comp rx=0 tx=0).stop 2023-07-14T16:27:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 -- 172.21.15.62:0/3650942873 shutdown_connections 2023-07-14T16:27:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55be7874d330 0x55be7874f710 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55be786e4c70 0x55be786f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55be786e34d0 0x55be786f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 --2- 172.21.15.62:0/3650942873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be786e3e20 0x55be786f2dc0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 -- 172.21.15.62:0/3650942873 >> 172.21.15.62:0/3650942873 conn(0x55be786d8da0 msgr2=0x55be786fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:31.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffb95ffb700 0 client.18813 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:31.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 -- 172.21.15.62:0/3650942873 shutdown_connections 2023-07-14T16:27:31.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 1 -- 172.21.15.62:0/3650942873 wait complete. 2023-07-14T16:27:31.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.594+0000 7ffbc8847b80 20 client.18813 trim_cache size 0 max 16384 2023-07-14T16:27:31.576 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:27:31.576 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:27:31.577 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:27:31.577 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:27:31.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 -- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a11d0 msgr2=0x7ffb980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a11d0 0x7ffb980a3680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffba80990a0 tx=0x7ffba8095040 comp rx=0 tx=0).stop 2023-07-14T16:27:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 -- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0101660 msgr2=0x7ffbb019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0101660 0x7ffbb019eef0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ffba8007ee0 tx=0x7ffba800d3b0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 -- 172.21.15.62:0/3271990070 shutdown_connections 2023-07-14T16:27:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb0101660 0x7ffbb019eef0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb00ffae0 0x7ffbb0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ffb980a11d0 0x7ffb980a3680 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 --2- 172.21.15.62:0/3271990070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbb0100cd0 0x7ffbb0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 -- 172.21.15.62:0/3271990070 >> 172.21.15.62:0/3271990070 conn(0x7ffbb00fb0e0 msgr2=0x7ffbb00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 -- 172.21.15.62:0/3271990070 shutdown_connections 2023-07-14T16:27:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.595+0000 7ffbb4fff700 1 -- 172.21.15.62:0/3271990070 wait complete. 2023-07-14T16:27:31.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:27:31.592 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:27:31.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.877+0000 7fd7413ff700 1 -- 172.21.15.62:0/3118529962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c0ffb00 msgr2=0x7fd73c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.877+0000 7fd7413ff700 1 --2- 172.21.15.62:0/3118529962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c0ffb00 0x7fd73c0fff10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009990 tx=0x7fd72c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.877+0000 7fd7413ff700 1 -- 172.21.15.62:0/3118529962 shutdown_connections 2023-07-14T16:27:31.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.877+0000 7fd7413ff700 1 --2- 172.21.15.62:0/3118529962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd73c101680 0x7fd73c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.877+0000 7fd7413ff700 1 --2- 172.21.15.62:0/3118529962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c0ffb00 0x7fd73c0fff10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.877+0000 7fd7413ff700 1 --2- 172.21.15.62:0/3118529962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd73c100cf0 0x7fd73c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.877+0000 7fd7413ff700 1 -- 172.21.15.62:0/3118529962 >> 172.21.15.62:0/3118529962 conn(0x7fd73c0fb0e0 msgr2=0x7fd73c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:31.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.877+0000 7fd7413ff700 1 -- 172.21.15.62:0/3118529962 shutdown_connections 2023-07-14T16:27:31.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.878+0000 7fd7413ff700 1 -- 172.21.15.62:0/3118529962 wait complete. 2023-07-14T16:27:31.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd7413ff700 1 Processor -- start 2023-07-14T16:27:31.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd7413ff700 1 -- start start 2023-07-14T16:27:31.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd7413ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd73c0ffb00 0x7fd73c1957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd7413ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c100cf0 0x7fd73c195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd7413ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd73c101680 0x7fd73c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd7413ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c19d330 con 0x7fd73c0ffb00 2023-07-14T16:27:31.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd7413ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c19d470 con 0x7fd73c100cf0 2023-07-14T16:27:31.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd7413ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c19d600 con 0x7fd73c101680 2023-07-14T16:27:31.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd73affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd73c0ffb00 0x7fd73c1957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.879+0000 7fd73affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd73c0ffb00 0x7fd73c1957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51568/0 (socket says 172.21.15.62:51568) 2023-07-14T16:27:31.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73affd700 1 -- 172.21.15.62:0/2958792014 learned_addr learned my addr 172.21.15.62:0/2958792014 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:31.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73a7fc700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c100cf0 0x7fd73c195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73b7fe700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd73c101680 0x7fd73c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73affd700 1 -- 172.21.15.62:0/2958792014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd73c101680 msgr2=0x7fd73c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73affd700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd73c101680 0x7fd73c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73affd700 1 -- 172.21.15.62:0/2958792014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c100cf0 msgr2=0x7fd73c195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:31.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73affd700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c100cf0 0x7fd73c195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:31.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73affd700 1 -- 172.21.15.62:0/2958792014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0095f0 con 0x7fd73c0ffb00 2023-07-14T16:27:31.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73a7fc700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c100cf0 0x7fd73c195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:31.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.880+0000 7fd73affd700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd73c0ffb00 0x7fd73c1957b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd72c005910 tx=0x7fd72c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:31.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.881+0000 7fd723fff700 1 -- 172.21.15.62:0/2958792014 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c01d070 con 0x7fd73c0ffb00 2023-07-14T16:27:31.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.881+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c19d880 con 0x7fd73c0ffb00 2023-07-14T16:27:31.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.881+0000 7fd723fff700 1 -- 172.21.15.62:0/2958792014 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd72c00bdd0 con 0x7fd73c0ffb00 2023-07-14T16:27:31.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.881+0000 7fd723fff700 1 -- 172.21.15.62:0/2958792014 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c022d60 con 0x7fd73c0ffb00 2023-07-14T16:27:31.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.881+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c19dca0 con 0x7fd73c0ffb00 2023-07-14T16:27:31.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.882+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c1070e0 con 0x7fd73c0ffb00 2023-07-14T16:27:31.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.885+0000 7fd723fff700 1 -- 172.21.15.62:0/2958792014 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd72c022500 con 0x7fd73c0ffb00 2023-07-14T16:27:31.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.887+0000 7fd723fff700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7240a11a0 0x7fd7240a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:31.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.887+0000 7fd723fff700 1 -- 172.21.15.62:0/2958792014 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd72c0ce570 con 0x7fd73c0ffb00 2023-07-14T16:27:31.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.888+0000 7fd73a7fc700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7240a11a0 0x7fd7240a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:31.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.888+0000 7fd73a7fc700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7240a11a0 0x7fd7240a3650 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd730009cb0 tx=0x7fd730009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:31.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:31.893+0000 7fd723fff700 1 -- 172.21.15.62:0/2958792014 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd72c097d00 con 0x7fd73c0ffb00 2023-07-14T16:27:31.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:31 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:27:32.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:31 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:27:32.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.252+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd73c02c850 con 0x7fd73c0ffb00 2023-07-14T16:27:32.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.253+0000 7fd723fff700 1 -- 172.21.15.62:0/2958792014 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fd72c09b320 con 0x7fd73c0ffb00 2023-07-14T16:27:32.228 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:27:32.228 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:27:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7240a11a0 msgr2=0x7fd7240a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7240a11a0 0x7fd7240a3650 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd730009cb0 tx=0x7fd730009260 comp rx=0 tx=0).stop 2023-07-14T16:27:32.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd73c0ffb00 msgr2=0x7fd73c1957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd73c0ffb00 0x7fd73c1957b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd72c005910 tx=0x7fd72c005740 comp rx=0 tx=0).stop 2023-07-14T16:27:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 shutdown_connections 2023-07-14T16:27:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd73c101680 0x7fd73c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd73c0ffb00 0x7fd73c1957b0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7240a11a0 0x7fd7240a3650 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 --2- 172.21.15.62:0/2958792014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c100cf0 0x7fd73c195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 >> 172.21.15.62:0/2958792014 conn(0x7fd73c0fb0e0 msgr2=0x7fd73c104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 shutdown_connections 2023-07-14T16:27:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.256+0000 7fd7413ff700 1 -- 172.21.15.62:0/2958792014 wait complete. 2023-07-14T16:27:32.233 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:27:32.254 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:27:32.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:31 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:27:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.516+0000 7f2d513ff700 1 -- 172.21.15.62:0/3979952165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 msgr2=0x7f2d4c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.516+0000 7f2d513ff700 1 --2- 172.21.15.62:0/3979952165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c101120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2d400098f0 tx=0x7f2d40009c00 comp rx=0 tx=0).stop 2023-07-14T16:27:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.516+0000 7f2d513ff700 1 -- 172.21.15.62:0/3979952165 shutdown_connections 2023-07-14T16:27:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.516+0000 7f2d513ff700 1 --2- 172.21.15.62:0/3979952165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c101660 0x7f2d4c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.516+0000 7f2d513ff700 1 --2- 172.21.15.62:0/3979952165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.516+0000 7f2d513ff700 1 --2- 172.21.15.62:0/3979952165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c101120 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.516+0000 7f2d513ff700 1 -- 172.21.15.62:0/3979952165 >> 172.21.15.62:0/3979952165 conn(0x7f2d4c0fb0e0 msgr2=0x7f2d4c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.517+0000 7f2d513ff700 1 -- 172.21.15.62:0/3979952165 shutdown_connections 2023-07-14T16:27:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.517+0000 7f2d513ff700 1 -- 172.21.15.62:0/3979952165 wait complete. 2023-07-14T16:27:32.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.518+0000 7f2d513ff700 1 Processor -- start 2023-07-14T16:27:32.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.518+0000 7f2d513ff700 1 -- start start 2023-07-14T16:27:32.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d513ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d513ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d513ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d4c101660 0x7f2d4c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d513ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0735e0 con 0x7f2d4c0ffae0 2023-07-14T16:27:32.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d513ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c073750 con 0x7f2d4c100cd0 2023-07-14T16:27:32.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d513ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c19f860 con 0x7f2d4c101660 2023-07-14T16:27:32.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d4affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d4b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d4c101660 0x7f2d4c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d4affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51584/0 (socket says 172.21.15.62:51584) 2023-07-14T16:27:32.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d4affd700 1 -- 172.21.15.62:0/4269462337 learned_addr learned my addr 172.21.15.62:0/4269462337 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.519+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d4a7fc700 1 -- 172.21.15.62:0/4269462337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d4c101660 msgr2=0x7f2d4c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d4c101660 0x7f2d4c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d4a7fc700 1 -- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0ffae0 msgr2=0x7f2d4c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d4a7fc700 1 -- 172.21.15.62:0/4269462337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d40009520 con 0x7f2d4c100cd0 2023-07-14T16:27:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d4affd700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c072900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009230 tx=0x7f2d4000bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d33fff700 1 -- 172.21.15.62:0/4269462337 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4001c070 con 0x7f2d4c100cd0 2023-07-14T16:27:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d513ff700 1 -- 172.21.15.62:0/4269462337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c195420 con 0x7f2d4c100cd0 2023-07-14T16:27:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d33fff700 1 -- 172.21.15.62:0/4269462337 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d4000bb10 con 0x7f2d4c100cd0 2023-07-14T16:27:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.520+0000 7f2d33fff700 1 -- 172.21.15.62:0/4269462337 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400038e0 con 0x7f2d4c100cd0 2023-07-14T16:27:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.521+0000 7f2d513ff700 1 -- 172.21.15.62:0/4269462337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c1958a0 con 0x7f2d4c100cd0 2023-07-14T16:27:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.525+0000 7f2d33fff700 1 -- 172.21.15.62:0/4269462337 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2d40020030 con 0x7f2d4c100cd0 2023-07-14T16:27:32.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.526+0000 7f2d64c57b80 1 Processor -- start 2023-07-14T16:27:32.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.527+0000 7f2d64c57b80 1 -- start start 2023-07-14T16:27:32.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.527+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b4a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.527+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b4b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.527+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4bc70 0x55a064b4c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.527+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064a4fcb0 con 0x55a064b4a4d0 2023-07-14T16:27:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.527+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064aa2a40 con 0x55a064b4ae20 2023-07-14T16:27:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.527+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064a9bed0 con 0x55a064b4bc70 2023-07-14T16:27:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.527+0000 7f2d4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b4a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b4a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51586/0 (socket says 172.21.15.62:51586) 2023-07-14T16:27:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4b7fe700 1 -- 172.21.15.62:0/2961378165 learned_addr learned my addr 172.21.15.62:0/2961378165 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d33fff700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d340a1150 0x7f2d340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b4b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4bc70 0x55a064b4c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4b7fe700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d340a1150 0x7f2d340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d33fff700 1 -- 172.21.15.62:0/4269462337 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d400cd760 con 0x7f2d4c100cd0 2023-07-14T16:27:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4affd700 1 -- 172.21.15.62:0/2961378165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4bc70 msgr2=0x55a064b4c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4bc70 0x55a064b4c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4affd700 1 -- 172.21.15.62:0/2961378165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 msgr2=0x55a064b4a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b4a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4affd700 1 -- 172.21.15.62:0/2961378165 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0648c9de0 con 0x55a064b4ae20 2023-07-14T16:27:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4bc70 0x55a064b4c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.528+0000 7f2d4b7fe700 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b4a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.529+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b4b6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c011540 tx=0x7f2d3c011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f0c253930b816 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.529+0000 7f2d4b7fe700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d340a1150 0x7f2d340a3600 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2d440060b0 tx=0x7f2d44004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.529+0000 7f2d317fa700 1 -- 172.21.15.62:0/2961378165 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c004250 con 0x55a064b4ae20 2023-07-14T16:27:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.529+0000 7f2d317fa700 1 -- 172.21.15.62:0/2961378165 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d3c004cc0 con 0x55a064b4ae20 2023-07-14T16:27:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.529+0000 7f2d317fa700 1 -- 172.21.15.62:0/2961378165 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c0056a0 con 0x55a064b4ae20 2023-07-14T16:27:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2961378165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 msgr2=0x55a064b4b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b4b6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c011540 tx=0x7f2d3c011850 comp rx=0 tx=0).stop 2023-07-14T16:27:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2961378165 shutdown_connections 2023-07-14T16:27:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4bc70 0x55a064b4c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b4a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2961378165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b4b6e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2961378165 >> 172.21.15.62:0/2961378165 conn(0x55a064b3fda0 msgr2=0x55a064b451d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2961378165 shutdown_connections 2023-07-14T16:27:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.530+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2961378165 wait complete. 2023-07-14T16:27:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.531+0000 7f2d64c57b80 1 -- start start 2023-07-14T16:27:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.531+0000 7f2d64c57b80 1 Processor -- start 2023-07-14T16:27:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b54110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4ae20 0x55a064b54650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 0x55a064b5b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064a3bac0 con 0x55a064b4a4d0 2023-07-14T16:27:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064a3b7c0 con 0x55a064b4bc70 2023-07-14T16:27:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a06491f920 con 0x55a064b4ae20 2023-07-14T16:27:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d64c57b80 10 client.0 _ll_register_callbacks cb 0x55a064a39760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b54110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b54110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51592/0 (socket says 172.21.15.62:51592) 2023-07-14T16:27:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4affd700 1 -- 172.21.15.62:0/3938988726 learned_addr learned my addr 172.21.15.62:0/3938988726 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4ae20 0x55a064b54650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4b7fe700 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 0x55a064b5b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4affd700 1 -- 172.21.15.62:0/3938988726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4ae20 msgr2=0x55a064b54650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4affd700 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4ae20 0x55a064b54650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4affd700 1 -- 172.21.15.62:0/3938988726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 msgr2=0x55a064b5b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4affd700 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 0x55a064b5b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.532+0000 7f2d4affd700 1 -- 172.21.15.62:0/3938988726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d44013850 con 0x55a064b4a4d0 2023-07-14T16:27:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.533+0000 7f2d4affd700 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b54110 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009770 tx=0x7f2d3c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707d764bb7ce8ff4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.533+0000 7f2d2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.534+0000 7f2d2eff5700 1 -- 172.21.15.62:0/3938988726 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c004050 con 0x55a064b4a4d0 2023-07-14T16:27:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.534+0000 7f2d2eff5700 1 -- 172.21.15.62:0/3938988726 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d3c0041b0 con 0x55a064b4a4d0 2023-07-14T16:27:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.534+0000 7f2d64c57b80 1 -- 172.21.15.62:0/3938988726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a0649f54f0 con 0x55a064b4a4d0 2023-07-14T16:27:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.534+0000 7f2d2eff5700 1 -- 172.21.15.62:0/3938988726 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d3c004370 con 0x55a064b4a4d0 2023-07-14T16:27:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.534+0000 7f2d64c57b80 10 client.18831 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.534+0000 7f2d64c57b80 1 -- 172.21.15.62:0/3938988726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f2d3c009d30 con 0x55a064b4a4d0 2023-07-14T16:27:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.534+0000 7f2d2eff5700 1 -- 172.21.15.62:0/3938988726 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a0649f54f0 con 0x55a064b4a4d0 2023-07-14T16:27:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.535+0000 7f2d2eff5700 1 -- 172.21.15.62:0/3938988726 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d3c022400 con 0x55a064b4a4d0 2023-07-14T16:27:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.535+0000 7f2d64c57b80 10 client.18831 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.535+0000 7f2d64c57b80 10 client.18831 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:27:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.535+0000 7f2d64c57b80 20 client.18831 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.535+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55a064b5fca0 0x55a064b620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.535+0000 7f2d64c57b80 4 client.18831 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.535+0000 7f2d64c57b80 1 -- 172.21.15.62:0/3938988726 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a064a94eb0 con 0x55a064b5fca0 2023-07-14T16:27:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.535+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55a064b5fca0 0x55a064b620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.536+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55a064b5fca0 0x55a064b620b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fa65f926901529e server_cookie=e96c66b671e8d5d3 in_seq=0 out_seq=0 2023-07-14T16:27:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.536+0000 7f2d2eff5700 10 client.18831 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:27:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.536+0000 7f2d2eff5700 1 -- 172.21.15.62:0/3938988726 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a064a94eb0 con 0x55a064b5fca0 2023-07-14T16:27:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.536+0000 7f2d2eff5700 10 client.18831 handle_command_reply: tid=0 2023-07-14T16:27:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.536+0000 7f2d64c57b80 1 client.18831 shutdown 2023-07-14T16:27:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 -- 172.21.15.62:0/3938988726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 msgr2=0x55a064b54110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b54110 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009770 tx=0x7f2d3c004ee0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 -- 172.21.15.62:0/3938988726 shutdown_connections 2023-07-14T16:27:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55a064b5fca0 0x55a064b620b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 0x55a064b5b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4a4d0 0x55a064b54110 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/3938988726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4ae20 0x55a064b54650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 -- 172.21.15.62:0/3938988726 >> 172.21.15.62:0/3938988726 conn(0x55a064b3fda0 msgr2=0x55a064b448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d2eff5700 0 client.18831 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:27:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 -- 172.21.15.62:0/3938988726 shutdown_connections 2023-07-14T16:27:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 1 -- 172.21.15.62:0/3938988726 wait complete. 2023-07-14T16:27:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.537+0000 7f2d64c57b80 20 client.18831 trim_cache size 0 max 16384 2023-07-14T16:27:32.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.553+0000 7f2d64c57b80 1 Processor -- start 2023-07-14T16:27:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d64c57b80 1 -- start start 2023-07-14T16:27:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b5b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4bc70 0x55a064b5bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064b52870 con 0x55a064b4bc70 2023-07-14T16:27:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064b51d70 con 0x55a064b4ae20 2023-07-14T16:27:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064b4a8a0 con 0x55a064b4a4d0 2023-07-14T16:27:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4bc70 0x55a064b5bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4bc70 0x55a064b5bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51608/0 (socket says 172.21.15.62:51608) 2023-07-14T16:27:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d4b7fe700 1 -- 172.21.15.62:0/2470461460 learned_addr learned my addr 172.21.15.62:0/2470461460 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.554+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b5b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4a7fc700 1 -- 172.21.15.62:0/2470461460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 msgr2=0x55a064b537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4a7fc700 1 -- 172.21.15.62:0/2470461460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4bc70 msgr2=0x55a064b5bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4bc70 0x55a064b5bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4a7fc700 1 -- 172.21.15.62:0/2470461460 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0648b2c60 con 0x55a064b4ae20 2023-07-14T16:27:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4b7fe700 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4bc70 0x55a064b5bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.555+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b5b650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2d400cb2c0 tx=0x7f2d40009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba83a36746adfeed server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d31ffb700 1 -- 172.21.15.62:0/2470461460 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4009eea0 con 0x55a064b4ae20 2023-07-14T16:27:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d31ffb700 1 -- 172.21.15.62:0/2470461460 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d4009ec40 con 0x55a064b4ae20 2023-07-14T16:27:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d31ffb700 1 -- 172.21.15.62:0/2470461460 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40005240 con 0x55a064b4ae20 2023-07-14T16:27:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2470461460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 msgr2=0x55a064b5b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b5b650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2d400cb2c0 tx=0x7f2d40009c00 comp rx=0 tx=0).stop 2023-07-14T16:27:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2470461460 shutdown_connections 2023-07-14T16:27:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4bc70 0x55a064b5bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2470461460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4ae20 0x55a064b5b650 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.556+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2470461460 >> 172.21.15.62:0/2470461460 conn(0x55a064b3fda0 msgr2=0x55a064b66a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.557+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2470461460 shutdown_connections 2023-07-14T16:27:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.557+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2470461460 wait complete. 2023-07-14T16:27:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.557+0000 7f2d64c57b80 1 -- start start 2023-07-14T16:27:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.557+0000 7f2d64c57b80 1 Processor -- start 2023-07-14T16:27:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4ae20 0x55a064b57bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d64c57b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 0x55a064b5ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064b5b150 con 0x55a064b4ae20 2023-07-14T16:27:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064b5b2c0 con 0x55a064b4bc70 2023-07-14T16:27:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d64c57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a064b5b480 con 0x55a064b4a4d0 2023-07-14T16:27:32.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d64c57b80 10 client.0 _ll_register_callbacks cb 0x55a064a39760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:32.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d4affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d4a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4ae20 0x55a064b57bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d4affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41282/0 (socket says 172.21.15.62:41282) 2023-07-14T16:27:32.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d4affd700 1 -- 172.21.15.62:0/2548840240 learned_addr learned my addr 172.21.15.62:0/2548840240 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d4b7fe700 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 0x55a064b5ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d4affd700 1 -- 172.21.15.62:0/2548840240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 msgr2=0x55a064b5ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.558+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 0x55a064b5ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.559+0000 7f2d4affd700 1 -- 172.21.15.62:0/2548840240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4ae20 msgr2=0x55a064b57bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.559+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4ae20 0x55a064b57bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.559+0000 7f2d4affd700 1 -- 172.21.15.62:0/2548840240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c009d30 con 0x55a064b4a4d0 2023-07-14T16:27:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.559+0000 7f2d4affd700 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b576b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c005770 tx=0x7f2d3c011850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86024b99e95bfd85 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.559+0000 7f2d2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:27:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.559+0000 7f2d2eff5700 1 -- 172.21.15.62:0/2548840240 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c009350 con 0x55a064b4a4d0 2023-07-14T16:27:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.559+0000 7f2d2eff5700 1 -- 172.21.15.62:0/2548840240 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d3c022ad0 con 0x55a064b4a4d0 2023-07-14T16:27:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.559+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2548840240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d3c004cc0 con 0x55a064b4a4d0 2023-07-14T16:27:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d2eff5700 1 -- 172.21.15.62:0/2548840240 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d3c004800 con 0x55a064b4a4d0 2023-07-14T16:27:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d64c57b80 10 client.28309 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2548840240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f2d44023040 con 0x55a064b4a4d0 2023-07-14T16:27:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d2eff5700 1 -- 172.21.15.62:0/2548840240 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d3c004cc0 con 0x55a064b4a4d0 2023-07-14T16:27:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d2eff5700 1 -- 172.21.15.62:0/2548840240 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d3c003c10 con 0x55a064b4a4d0 2023-07-14T16:27:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d64c57b80 10 client.28309 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d64c57b80 10 client.28309 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:27:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d64c57b80 20 client.28309 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55a064bb4a90 0x55a064bb6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d64c57b80 4 client.28309 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:27:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.560+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2548840240 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a064a57050 con 0x55a064bb4a90 2023-07-14T16:27:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.561+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55a064bb4a90 0x55a064bb6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d2eff5700 10 client.28309 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:27:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d4a7fc700 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55a064bb4a90 0x55a064bb6ea0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5cffb287ba9282 server_cookie=52ada697c2734b58 in_seq=0 out_seq=0 2023-07-14T16:27:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d2eff5700 1 -- 172.21.15.62:0/2548840240 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a064a57050 con 0x55a064bb4a90 2023-07-14T16:27:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d2eff5700 10 client.28309 handle_command_reply: tid=0 2023-07-14T16:27:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 client.28309 shutdown 2023-07-14T16:27:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2548840240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 msgr2=0x55a064b576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b576b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c005770 tx=0x7f2d3c011850 comp rx=0 tx=0).stop 2023-07-14T16:27:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2548840240 shutdown_connections 2023-07-14T16:27:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55a064bb4a90 0x55a064bb6ea0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a064b4bc70 0x55a064b5ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a064b4a4d0 0x55a064b576b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d2eff5700 0 client.28309 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:27:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 --2- 172.21.15.62:0/2548840240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a064b4ae20 0x55a064b57bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2548840240 >> 172.21.15.62:0/2548840240 conn(0x55a064b3fda0 msgr2=0x55a064b5bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2548840240 shutdown_connections 2023-07-14T16:27:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.562+0000 7f2d64c57b80 1 -- 172.21.15.62:0/2548840240 wait complete. 2023-07-14T16:27:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d64c57b80 20 client.28309 trim_cache size 0 max 16384 2023-07-14T16:27:32.546 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:27:32.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 -- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d340a1150 msgr2=0x7f2d340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d340a1150 0x7f2d340a3600 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2d440060b0 tx=0x7f2d44004b40 comp rx=0 tx=0).stop 2023-07-14T16:27:32.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 -- 172.21.15.62:0/4269462337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 msgr2=0x7f2d4c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c072900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009230 tx=0x7f2d4000bf80 comp rx=0 tx=0).stop 2023-07-14T16:27:32.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 -- 172.21.15.62:0/4269462337 shutdown_connections 2023-07-14T16:27:32.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d4c101660 0x7f2d4c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d340a1150 0x7f2d340a3600 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 --2- 172.21.15.62:0/4269462337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c072900 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 -- 172.21.15.62:0/4269462337 >> 172.21.15.62:0/4269462337 conn(0x7f2d4c0fb0e0 msgr2=0x7f2d4c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.563+0000 7f2d513ff700 1 -- 172.21.15.62:0/4269462337 shutdown_connections 2023-07-14T16:27:32.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.564+0000 7f2d513ff700 1 -- 172.21.15.62:0/4269462337 wait complete. 2023-07-14T16:27:32.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:27:32.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.802+0000 7f9c11fff700 1 -- 172.21.15.62:0/2207813136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c1005c0 msgr2=0x7f9c0c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.802+0000 7f9c11fff700 1 --2- 172.21.15.62:0/2207813136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c1005c0 0x7f9c0c1029a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009990 tx=0x7f9bfc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.802+0000 7f9c11fff700 1 -- 172.21.15.62:0/2207813136 shutdown_connections 2023-07-14T16:27:32.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.802+0000 7f9c11fff700 1 --2- 172.21.15.62:0/2207813136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c105800 0x7f9c0c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.802+0000 7f9c11fff700 1 --2- 172.21.15.62:0/2207813136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c1005c0 0x7f9c0c1029a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.802+0000 7f9c11fff700 1 --2- 172.21.15.62:0/2207813136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c102ee0 0x7f9c0c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.802+0000 7f9c11fff700 1 -- 172.21.15.62:0/2207813136 >> 172.21.15.62:0/2207813136 conn(0x7f9c0c0fa000 msgr2=0x7f9c0c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.803+0000 7f9c11fff700 1 -- 172.21.15.62:0/2207813136 shutdown_connections 2023-07-14T16:27:32.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.803+0000 7f9c11fff700 1 -- 172.21.15.62:0/2207813136 wait complete. 2023-07-14T16:27:32.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.804+0000 7f9c11fff700 1 Processor -- start 2023-07-14T16:27:32.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.804+0000 7f9c11fff700 1 -- start start 2023-07-14T16:27:32.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.804+0000 7f9c11fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c1005c0 0x7f9c0c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c11fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c102ee0 0x7f9c0c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c11fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c105800 0x7f9c0c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c11fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c19f4e0 con 0x7f9c0c105800 2023-07-14T16:27:32.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c11fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c19f620 con 0x7f9c0c102ee0 2023-07-14T16:27:32.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c11fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c19f7b0 con 0x7f9c0c1005c0 2023-07-14T16:27:32.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c1005c0 0x7f9c0c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c102ee0 0x7f9c0c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c1005c0 0x7f9c0c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:51950/0 (socket says 172.21.15.62:51950) 2023-07-14T16:27:32.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0b7fe700 1 -- 172.21.15.62:0/3140686399 learned_addr learned my addr 172.21.15.62:0/3140686399 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c105800 0x7f9c0c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0affd700 1 -- 172.21.15.62:0/3140686399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c1005c0 msgr2=0x7f9c0c197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0affd700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c1005c0 0x7f9c0c197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0affd700 1 -- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c105800 msgr2=0x7f9c0c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0affd700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c105800 0x7f9c0c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0affd700 1 -- 172.21.15.62:0/3140686399 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0095f0 con 0x7f9c0c102ee0 2023-07-14T16:27:32.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c1005c0 0x7f9c0c197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:32.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.805+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c105800 0x7f9c0c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:32.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.806+0000 7f9c0affd700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c102ee0 0x7f9c0c197ea0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e9f0 tx=0x7f9c0000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.806+0000 7f9c08ff9700 1 -- 172.21.15.62:0/3140686399 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000c9c0 con 0x7f9c0c102ee0 2023-07-14T16:27:32.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.806+0000 7f9c08ff9700 1 -- 172.21.15.62:0/3140686399 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c00010430 con 0x7f9c0c102ee0 2023-07-14T16:27:32.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.806+0000 7f9c11fff700 1 -- 172.21.15.62:0/3140686399 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0c19fa90 con 0x7f9c0c102ee0 2023-07-14T16:27:32.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.806+0000 7f9c08ff9700 1 -- 172.21.15.62:0/3140686399 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00018680 con 0x7f9c0c102ee0 2023-07-14T16:27:32.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.806+0000 7f9c11fff700 1 -- 172.21.15.62:0/3140686399 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c19fff0 con 0x7f9c0c102ee0 2023-07-14T16:27:32.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.811+0000 7f9c08ff9700 1 -- 172.21.15.62:0/3140686399 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9c000105a0 con 0x7f9c0c102ee0 2023-07-14T16:27:32.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.812+0000 7f9c25912b80 1 Processor -- start 2023-07-14T16:27:32.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.812+0000 7f9c25912b80 1 -- start start 2023-07-14T16:27:32.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c25912b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac945cb0 con 0x55cbaca40e20 2023-07-14T16:27:32.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c25912b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac998a40 con 0x55cbaca404d0 2023-07-14T16:27:32.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c25912b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac991ed0 con 0x55cbaca41c70 2023-07-14T16:27:32.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:39940/0 (socket says 172.21.15.62:39940) 2023-07-14T16:27:32.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0bfff700 1 -- 172.21.15.62:0/3700109717 learned_addr learned my addr 172.21.15.62:0/3700109717 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0affd700 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0bfff700 1 -- 172.21.15.62:0/3700109717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 msgr2=0x55cbaca420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0bfff700 1 -- 172.21.15.62:0/3700109717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 msgr2=0x55cbaca416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c0bfff700 1 -- 172.21.15.62:0/3700109717 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbac7bfde0 con 0x55cbaca404d0 2023-07-14T16:27:32.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.813+0000 7f9c08ff9700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bf40a11d0 0x7f9bf40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.814+0000 7f9c0affd700 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:32.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.814+0000 7f9c08ff9700 1 -- 172.21.15.62:0/3140686399 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9c00014070 con 0x7f9c0c102ee0 2023-07-14T16:27:32.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.814+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca408e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c040078d0 tx=0x7f9c04007be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0cd2ccf793f92f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.814+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bf40a11d0 0x7f9bf40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.814+0000 7f9bf1ffb700 1 -- 172.21.15.62:0/3700109717 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04004bb0 con 0x55cbaca404d0 2023-07-14T16:27:32.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.814+0000 7f9bf1ffb700 1 -- 172.21.15.62:0/3700109717 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c0400f040 con 0x55cbaca404d0 2023-07-14T16:27:32.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.814+0000 7f9bf1ffb700 1 -- 172.21.15.62:0/3700109717 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c040052a0 con 0x55cbaca404d0 2023-07-14T16:27:32.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 -- 172.21.15.62:0/3700109717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 msgr2=0x55cbaca408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bf40a11d0 0x7f9bf40a3680 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc006010 tx=0x7f9bfc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca408e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c040078d0 tx=0x7f9c04007be0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 -- 172.21.15.62:0/3700109717 shutdown_connections 2023-07-14T16:27:32.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca408e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3700109717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 -- 172.21.15.62:0/3700109717 >> 172.21.15.62:0/3700109717 conn(0x55cbaca35da0 msgr2=0x55cbaca3b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 -- 172.21.15.62:0/3700109717 shutdown_connections 2023-07-14T16:27:32.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.815+0000 7f9c25912b80 1 -- 172.21.15.62:0/3700109717 wait complete. 2023-07-14T16:27:32.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.816+0000 7f9c25912b80 1 -- start start 2023-07-14T16:27:32.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.816+0000 7f9c25912b80 1 Processor -- start 2023-07-14T16:27:32.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.816+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca404d0 0x55cbaca4a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.816+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca4a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.816+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca51660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.816+0000 7f9c25912b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac931ac0 con 0x55cbaca40e20 2023-07-14T16:27:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.816+0000 7f9c25912b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac9317c0 con 0x55cbaca41c70 2023-07-14T16:27:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.816+0000 7f9c25912b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbac815920 con 0x55cbaca404d0 2023-07-14T16:27:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c25912b80 10 client.0 _ll_register_callbacks cb 0x55cbac92f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca4a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca404d0 0x55cbaca4a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca51660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca404d0 0x55cbaca4a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:51960/0 (socket says 172.21.15.62:51960) 2023-07-14T16:27:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca4a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54512/0 (socket says 172.21.15.62:54512) 2023-07-14T16:27:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0b7fe700 1 -- 172.21.15.62:0/3969273330 learned_addr learned my addr 172.21.15.62:0/3969273330 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0bfff700 1 -- 172.21.15.62:0/3969273330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca404d0 msgr2=0x55cbaca4a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca404d0 0x55cbaca4a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0bfff700 1 -- 172.21.15.62:0/3969273330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 msgr2=0x55cbaca4a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca4a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0bfff700 1 -- 172.21.15.62:0/3969273330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c04007500 con 0x55cbaca41c70 2023-07-14T16:27:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.817+0000 7f9c0affd700 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca4a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca51660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009e20 tx=0x7f9c04015900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c9e311c38dc26bf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9beaffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:27:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9beaffd700 1 -- 172.21.15.62:0/3969273330 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04015c00 con 0x55cbaca41c70 2023-07-14T16:27:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9beaffd700 1 -- 172.21.15.62:0/3969273330 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9c04003c10 con 0x55cbaca41c70 2023-07-14T16:27:32.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9c25912b80 1 -- 172.21.15.62:0/3969273330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbac8eb4f0 con 0x55cbaca41c70 2023-07-14T16:27:32.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9beaffd700 1 -- 172.21.15.62:0/3969273330 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c04003dd0 con 0x55cbaca41c70 2023-07-14T16:27:32.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9c25912b80 10 client.28442 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:32.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9c25912b80 1 -- 172.21.15.62:0/3969273330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cbac7a8c60 con 0x55cbaca41c70 2023-07-14T16:27:32.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.818+0000 7f9beaffd700 1 -- 172.21.15.62:0/3969273330 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cbac8eb4f0 con 0x55cbaca41c70 2023-07-14T16:27:32.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.819+0000 7f9beaffd700 1 -- 172.21.15.62:0/3969273330 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c0401f650 con 0x55cbaca41c70 2023-07-14T16:27:32.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.819+0000 7f9c25912b80 10 client.28442 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:32.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.819+0000 7f9c25912b80 10 client.28442 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:27:32.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.819+0000 7f9c25912b80 20 client.28442 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:32.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.819+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55cbaca55e50 0x55cbaca58260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.819+0000 7f9c25912b80 4 client.28442 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:32.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.819+0000 7f9c25912b80 1 -- 172.21.15.62:0/3969273330 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbac98aeb0 con 0x55cbaca55e50 2023-07-14T16:27:32.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.819+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55cbaca55e50 0x55cbaca58260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.820+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55cbaca55e50 0x55cbaca58260 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53ec186b12bf2c9c server_cookie=a05b6f4c2464c1a5 in_seq=0 out_seq=0 2023-07-14T16:27:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.820+0000 7f9beaffd700 10 client.28442 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:27:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.820+0000 7f9beaffd700 1 -- 172.21.15.62:0/3969273330 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cbac98aeb0 con 0x55cbaca55e50 2023-07-14T16:27:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.820+0000 7f9beaffd700 10 client.28442 handle_command_reply: tid=0 2023-07-14T16:27:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.820+0000 7f9c25912b80 1 client.28442 shutdown 2023-07-14T16:27:32.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 -- 172.21.15.62:0/3969273330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 msgr2=0x55cbaca51660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca51660 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009e20 tx=0x7f9c04015900 comp rx=0 tx=0).stop 2023-07-14T16:27:32.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 -- 172.21.15.62:0/3969273330 shutdown_connections 2023-07-14T16:27:32.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55cbaca55e50 0x55cbaca58260 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca51660 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca404d0 0x55cbaca4a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9beaffd700 0 client.28442 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:27:32.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 --2- 172.21.15.62:0/3969273330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca4a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 -- 172.21.15.62:0/3969273330 >> 172.21.15.62:0/3969273330 conn(0x55cbaca35da0 msgr2=0x55cbaca3a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 -- 172.21.15.62:0/3969273330 shutdown_connections 2023-07-14T16:27:32.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 1 -- 172.21.15.62:0/3969273330 wait complete. 2023-07-14T16:27:32.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.821+0000 7f9c25912b80 20 client.28442 trim_cache size 0 max 16384 2023-07-14T16:27:32.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.838+0000 7f9c25912b80 1 Processor -- start 2023-07-14T16:27:32.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.838+0000 7f9c25912b80 1 -- start start 2023-07-14T16:27:32.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.838+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca3c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.838+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca51620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.838+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca5cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.838+0000 7f9c25912b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaca58980 con 0x55cbaca40e20 2023-07-14T16:27:32.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.838+0000 7f9c25912b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaca47d80 con 0x55cbaca404d0 2023-07-14T16:27:32.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.838+0000 7f9c25912b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaca3ca00 con 0x55cbaca41c70 2023-07-14T16:27:32.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca51620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca51620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54522/0 (socket says 172.21.15.62:54522) 2023-07-14T16:27:32.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 -- 172.21.15.62:0/178550178 learned_addr learned my addr 172.21.15.62:0/178550178 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca3c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca5cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 -- 172.21.15.62:0/178550178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 msgr2=0x55cbaca5cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca5cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 -- 172.21.15.62:0/178550178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 msgr2=0x55cbaca3c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca3c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 -- 172.21.15.62:0/178550178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbac7e6250 con 0x55cbaca40e20 2023-07-14T16:27:32.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.839+0000 7f9c0affd700 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca51620 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9c000962b0 tx=0x7f9c000ccec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c6a5d3344aa2be server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.840+0000 7f9bf27fc700 1 -- 172.21.15.62:0/178550178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000998f0 con 0x55cbaca40e20 2023-07-14T16:27:32.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.840+0000 7f9bf27fc700 1 -- 172.21.15.62:0/178550178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c00099710 con 0x55cbaca40e20 2023-07-14T16:27:32.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.840+0000 7f9bf27fc700 1 -- 172.21.15.62:0/178550178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000992a0 con 0x55cbaca40e20 2023-07-14T16:27:32.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 -- 172.21.15.62:0/178550178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 msgr2=0x55cbaca51620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca51620 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9c000962b0 tx=0x7f9c000ccec0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 -- 172.21.15.62:0/178550178 shutdown_connections 2023-07-14T16:27:32.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca41c70 0x55cbaca5cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca404d0 0x55cbaca3c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 --2- 172.21.15.62:0/178550178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca40e20 0x55cbaca51620 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 -- 172.21.15.62:0/178550178 >> 172.21.15.62:0/178550178 conn(0x55cbaca35da0 msgr2=0x55cbaca37310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 -- 172.21.15.62:0/178550178 shutdown_connections 2023-07-14T16:27:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.841+0000 7f9c25912b80 1 -- 172.21.15.62:0/178550178 wait complete. 2023-07-14T16:27:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.842+0000 7f9c25912b80 1 -- start start 2023-07-14T16:27:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.842+0000 7f9c25912b80 1 Processor -- start 2023-07-14T16:27:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.842+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca404d0 0x55cbaca50f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.842+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca40e20 0x55cbaca51480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.842+0000 7f9c25912b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.842+0000 7f9c25912b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaca4df60 con 0x55cbaca404d0 2023-07-14T16:27:32.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.842+0000 7f9c25912b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaca4e0d0 con 0x55cbaca41c70 2023-07-14T16:27:32.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c25912b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbaca4e290 con 0x55cbaca40e20 2023-07-14T16:27:32.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c25912b80 10 client.0 _ll_register_callbacks cb 0x7f9c00005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:32.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca404d0 0x55cbaca50f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca404d0 0x55cbaca50f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54536/0 (socket says 172.21.15.62:54536) 2023-07-14T16:27:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca40e20 0x55cbaca51480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0b7fe700 1 -- 172.21.15.62:0/2667257594 learned_addr learned my addr 172.21.15.62:0/2667257594 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0b7fe700 1 -- 172.21.15.62:0/2667257594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca40e20 msgr2=0x55cbaca51480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca40e20 0x55cbaca51480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0b7fe700 1 -- 172.21.15.62:0/2667257594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 msgr2=0x55cbaca4d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca4d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0b7fe700 1 -- 172.21.15.62:0/2667257594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c19fff0 con 0x55cbaca404d0 2023-07-14T16:27:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0affd700 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca40e20 0x55cbaca51480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.843+0000 7f9c0bfff700 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.844+0000 7f9c0b7fe700 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca404d0 0x55cbaca50f40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x55cbaca58b00 tx=0x7f9bfc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bf7922c4564514 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.844+0000 7f9beaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.844+0000 7f9beaffd700 1 -- 172.21.15.62:0/2667257594 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00bc70 con 0x55cbaca404d0 2023-07-14T16:27:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.844+0000 7f9beaffd700 1 -- 172.21.15.62:0/2667257594 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9bfc034e10 con 0x55cbaca404d0 2023-07-14T16:27:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.844+0000 7f9c25912b80 1 -- 172.21.15.62:0/2667257594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c0400f040 con 0x55cbaca404d0 2023-07-14T16:27:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.844+0000 7f9beaffd700 1 -- 172.21.15.62:0/2667257594 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bfc043030 con 0x55cbaca404d0 2023-07-14T16:27:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.844+0000 7f9c25912b80 10 client.18855 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9c25912b80 1 -- 172.21.15.62:0/2667257594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cbac7be2b0 con 0x55cbaca404d0 2023-07-14T16:27:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9beaffd700 1 -- 172.21.15.62:0/2667257594 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c0400f040 con 0x55cbaca404d0 2023-07-14T16:27:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9beaffd700 1 -- 172.21.15.62:0/2667257594 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9bfc004620 con 0x55cbaca404d0 2023-07-14T16:27:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9c25912b80 10 client.18855 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9c25912b80 10 client.18855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:27:32.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9c25912b80 20 client.18855 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:32.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9c25912b80 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55cbacaab1d0 0x55cbacaad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:32.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9c25912b80 4 client.18855 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:27:32.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.845+0000 7f9c25912b80 1 -- 172.21.15.62:0/2667257594 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cbac94d050 con 0x55cbacaab1d0 2023-07-14T16:27:32.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.846+0000 7f9c0affd700 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55cbacaab1d0 0x55cbacaad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:32.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.846+0000 7f9beaffd700 10 client.18855 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:27:32.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.846+0000 7f9c0affd700 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55cbacaab1d0 0x55cbacaad5e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9726875368653f39 server_cookie=67a5c98dcc115df6 in_seq=0 out_seq=0 2023-07-14T16:27:32.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9beaffd700 1 -- 172.21.15.62:0/2667257594 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cbac94d050 con 0x55cbacaab1d0 2023-07-14T16:27:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9beaffd700 10 client.18855 handle_command_reply: tid=0 2023-07-14T16:27:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 client.18855 shutdown 2023-07-14T16:27:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 -- 172.21.15.62:0/2667257594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca404d0 msgr2=0x55cbaca50f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca404d0 0x55cbaca50f40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x55cbaca58b00 tx=0x7f9bfc009310 comp rx=0 tx=0).stop 2023-07-14T16:27:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 -- 172.21.15.62:0/2667257594 shutdown_connections 2023-07-14T16:27:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55cbacaab1d0 0x55cbacaad5e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cbaca41c70 0x55cbaca4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaca404d0 0x55cbaca50f40 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 --2- 172.21.15.62:0/2667257594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cbaca40e20 0x55cbaca51480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9beaffd700 0 client.18855 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:27:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 -- 172.21.15.62:0/2667257594 >> 172.21.15.62:0/2667257594 conn(0x55cbaca35da0 msgr2=0x55cbaca3a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 -- 172.21.15.62:0/2667257594 shutdown_connections 2023-07-14T16:27:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 1 -- 172.21.15.62:0/2667257594 wait complete. 2023-07-14T16:27:32.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.847+0000 7f9c25912b80 20 client.18855 trim_cache size 0 max 16384 2023-07-14T16:27:32.830 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:27:32.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.848+0000 7f9c11fff700 1 -- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bf40a11d0 msgr2=0x7f9bf40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.848+0000 7f9c11fff700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bf40a11d0 0x7f9bf40a3680 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc006010 tx=0x7f9bfc00b260 comp rx=0 tx=0).stop 2023-07-14T16:27:32.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.848+0000 7f9c11fff700 1 -- 172.21.15.62:0/3140686399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c102ee0 msgr2=0x7f9c0c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:32.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.848+0000 7f9c11fff700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c102ee0 0x7f9c0c197ea0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e9f0 tx=0x7f9c0000ed00 comp rx=0 tx=0).stop 2023-07-14T16:27:32.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.848+0000 7f9c11fff700 1 -- 172.21.15.62:0/3140686399 shutdown_connections 2023-07-14T16:27:32.831 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.848+0000 7f9c11fff700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c105800 0x7f9c0c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.849+0000 7f9c11fff700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c1005c0 0x7f9c0c197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.849+0000 7f9c11fff700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9bf40a11d0 0x7f9bf40a3680 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.849+0000 7f9c11fff700 1 --2- 172.21.15.62:0/3140686399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c0c102ee0 0x7f9c0c197ea0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.849+0000 7f9c11fff700 1 -- 172.21.15.62:0/3140686399 >> 172.21.15.62:0/3140686399 conn(0x7f9c0c0fa000 msgr2=0x7f9c0c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.849+0000 7f9c11fff700 1 -- 172.21.15.62:0/3140686399 shutdown_connections 2023-07-14T16:27:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:32.849+0000 7f9c11fff700 1 -- 172.21.15.62:0/3140686399 wait complete. 2023-07-14T16:27:32.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:27:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 -- 172.21.15.62:0/158570062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c100560 msgr2=0x7f622c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 --2- 172.21.15.62:0/158570062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c100560 0x7f622c106e20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6224009990 tx=0x7f6224009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 -- 172.21.15.62:0/158570062 shutdown_connections 2023-07-14T16:27:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 --2- 172.21.15.62:0/158570062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c100560 0x7f622c106e20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 --2- 172.21.15.62:0/158570062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c0fe9e0 0x7f622c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 --2- 172.21.15.62:0/158570062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c0ffbd0 0x7f622c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 -- 172.21.15.62:0/158570062 >> 172.21.15.62:0/158570062 conn(0x7f622c0fa000 msgr2=0x7f622c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:33.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 -- 172.21.15.62:0/158570062 shutdown_connections 2023-07-14T16:27:33.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.118+0000 7f62335ff700 1 -- 172.21.15.62:0/158570062 wait complete. 2023-07-14T16:27:33.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.119+0000 7f62335ff700 1 Processor -- start 2023-07-14T16:27:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.119+0000 7f62335ff700 1 -- start start 2023-07-14T16:27:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f62335ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c0fe9e0 0x7f622c102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f62335ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c0ffbd0 0x7f622c100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f62335ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c100560 0x7f622c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f62335ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c101c90 con 0x7f622c0ffbd0 2023-07-14T16:27:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f62335ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c101e00 con 0x7f622c100560 2023-07-14T16:27:33.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f62335ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c19f790 con 0x7f622c0fe9e0 2023-07-14T16:27:33.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6230b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c0ffbd0 0x7f622c100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6230b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c0ffbd0 0x7f622c100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54552/0 (socket says 172.21.15.62:54552) 2023-07-14T16:27:33.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6230b9a700 1 -- 172.21.15.62:0/3870800446 learned_addr learned my addr 172.21.15.62:0/3870800446 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:33.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6231b9c700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c100560 0x7f622c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f623139b700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c0fe9e0 0x7f622c102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6230b9a700 1 -- 172.21.15.62:0/3870800446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c0fe9e0 msgr2=0x7f622c102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6230b9a700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c0fe9e0 0x7f622c102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6230b9a700 1 -- 172.21.15.62:0/3870800446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c100560 msgr2=0x7f622c1014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6230b9a700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c100560 0x7f622c1014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.120+0000 7f6230b9a700 1 -- 172.21.15.62:0/3870800446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62240095f0 con 0x7f622c0ffbd0 2023-07-14T16:27:33.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.121+0000 7f6230b9a700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c0ffbd0 0x7f622c100fb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f621c00e9a0 tx=0x7f621c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:33.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.121+0000 7f62227fc700 1 -- 172.21.15.62:0/3870800446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c009790 con 0x7f622c0ffbd0 2023-07-14T16:27:33.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.121+0000 7f62335ff700 1 -- 172.21.15.62:0/3870800446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c19fa70 con 0x7f622c0ffbd0 2023-07-14T16:27:33.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.121+0000 7f62227fc700 1 -- 172.21.15.62:0/3870800446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f621c010430 con 0x7f622c0ffbd0 2023-07-14T16:27:33.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.121+0000 7f62227fc700 1 -- 172.21.15.62:0/3870800446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00cea0 con 0x7f622c0ffbd0 2023-07-14T16:27:33.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.121+0000 7f62335ff700 1 -- 172.21.15.62:0/3870800446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c19ffd0 con 0x7f622c0ffbd0 2023-07-14T16:27:33.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.125+0000 7f62227fc700 1 -- 172.21.15.62:0/3870800446 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f621c0105a0 con 0x7f622c0ffbd0 2023-07-14T16:27:33.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.126+0000 7f6246e42b80 1 Processor -- start 2023-07-14T16:27:33.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.126+0000 7f6246e42b80 1 -- start start 2023-07-14T16:27:33.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.127+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9794a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.127+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9794aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.127+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 0x560a9794b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.127+0000 7f6246e42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a97846cb0 con 0x560a97949ce0 2023-07-14T16:27:33.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.127+0000 7f6246e42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a97899a40 con 0x560a9794a630 2023-07-14T16:27:33.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.127+0000 7f6246e42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a97892ed0 con 0x560a9794b480 2023-07-14T16:27:33.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.128+0000 7f6231b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9794a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.128+0000 7f6231b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9794a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54564/0 (socket says 172.21.15.62:54564) 2023-07-14T16:27:33.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.128+0000 7f6231b9c700 1 -- 172.21.15.62:0/16076593 learned_addr learned my addr 172.21.15.62:0/16076593 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:33.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.128+0000 7f6230b9a700 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 0x560a9794b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.128+0000 7f623139b700 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9794aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f6231b9c700 1 -- 172.21.15.62:0/16076593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 msgr2=0x560a9794b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:33.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f6231b9c700 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 0x560a9794b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f6231b9c700 1 -- 172.21.15.62:0/16076593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 msgr2=0x560a9794aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:33.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f62227fc700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a1150 0x7f62180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f6231b9c700 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9794aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f6231b9c700 1 -- 172.21.15.62:0/16076593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a976c0de0 con 0x560a97949ce0 2023-07-14T16:27:33.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f623139b700 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9794aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:33.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f62227fc700 1 -- 172.21.15.62:0/3870800446 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f621c014070 con 0x7f622c0ffbd0 2023-07-14T16:27:33.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f6231b9c700 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9794a0f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6224015000 tx=0x7f6224005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd176336e0c4b77 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:33.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.129+0000 7f6231b9c700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a1150 0x7f62180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.130+0000 7f62177fe700 1 -- 172.21.15.62:0/16076593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622401d070 con 0x560a97949ce0 2023-07-14T16:27:33.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.130+0000 7f62177fe700 1 -- 172.21.15.62:0/16076593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6224029070 con 0x560a97949ce0 2023-07-14T16:27:33.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.130+0000 7f62177fe700 1 -- 172.21.15.62:0/16076593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224004760 con 0x560a97949ce0 2023-07-14T16:27:33.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 -- 172.21.15.62:0/16076593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 msgr2=0x560a9794a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9794a0f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6224015000 tx=0x7f6224005e80 comp rx=0 tx=0).stop 2023-07-14T16:27:33.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6231b9c700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a1150 0x7f62180a3600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f622402c000 tx=0x7f622402d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:33.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 -- 172.21.15.62:0/16076593 shutdown_connections 2023-07-14T16:27:33.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 0x560a9794b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9794a0f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 --2- 172.21.15.62:0/16076593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9794aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 -- 172.21.15.62:0/16076593 >> 172.21.15.62:0/16076593 conn(0x560a97936da0 msgr2=0x560a9793c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:33.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 -- 172.21.15.62:0/16076593 shutdown_connections 2023-07-14T16:27:33.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.131+0000 7f6246e42b80 1 -- 172.21.15.62:0/16076593 wait complete. 2023-07-14T16:27:33.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.132+0000 7f6246e42b80 1 -- start start 2023-07-14T16:27:33.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.132+0000 7f6246e42b80 1 Processor -- start 2023-07-14T16:27:33.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a97949ce0 0x560a97953940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97953e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a9794b480 0x560a9795aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f6246e42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a97832ac0 con 0x560a9794b480 2023-07-14T16:27:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f6246e42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a978327c0 con 0x560a97949ce0 2023-07-14T16:27:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f6246e42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a97716920 con 0x560a9794a630 2023-07-14T16:27:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f6246e42b80 10 client.0 _ll_register_callbacks cb 0x560a97830760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f623139b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a97949ce0 0x560a97953940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f623139b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a97949ce0 0x560a97953940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40004/0 (socket says 172.21.15.62:40004) 2023-07-14T16:27:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f623139b700 1 -- 172.21.15.62:0/115174419 learned_addr learned my addr 172.21.15.62:0/115174419 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.133+0000 7f6230b9a700 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97953e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f6231b9c700 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a9794b480 0x560a9795aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f623139b700 1 -- 172.21.15.62:0/115174419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 msgr2=0x560a97953e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f623139b700 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97953e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f623139b700 1 -- 172.21.15.62:0/115174419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a9794b480 msgr2=0x560a9795aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f623139b700 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a9794b480 0x560a9795aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f623139b700 1 -- 172.21.15.62:0/115174419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c107750 con 0x560a97949ce0 2023-07-14T16:27:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f6230b9a700 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97953e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f623139b700 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a97949ce0 0x560a97953940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6228007f00 tx=0x7f6228007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=819cc67bb0e4e2b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.134+0000 7f6214ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:27:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.135+0000 7f6214ff9700 1 -- 172.21.15.62:0/115174419 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800e040 con 0x560a97949ce0 2023-07-14T16:27:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.135+0000 7f6246e42b80 1 -- 172.21.15.62:0/115174419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a977ec4f0 con 0x560a97949ce0 2023-07-14T16:27:33.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.135+0000 7f6214ff9700 1 -- 172.21.15.62:0/115174419 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6228009320 con 0x560a97949ce0 2023-07-14T16:27:33.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.135+0000 7f6214ff9700 1 -- 172.21.15.62:0/115174419 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6228013850 con 0x560a97949ce0 2023-07-14T16:27:33.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.135+0000 7f6246e42b80 10 client.28460 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:33.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.135+0000 7f6246e42b80 1 -- 172.21.15.62:0/115174419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560a976a9c60 con 0x560a97949ce0 2023-07-14T16:27:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.135+0000 7f6214ff9700 1 -- 172.21.15.62:0/115174419 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560a977ec4f0 con 0x560a97949ce0 2023-07-14T16:27:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.136+0000 7f6214ff9700 1 -- 172.21.15.62:0/115174419 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6228028070 con 0x560a97949ce0 2023-07-14T16:27:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.136+0000 7f6246e42b80 10 client.28460 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.136+0000 7f6246e42b80 10 client.28460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.136+0000 7f6246e42b80 20 client.28460 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.136+0000 7f6246e42b80 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a9795f770 0x560a97961b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.136+0000 7f6246e42b80 4 client.28460 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.136+0000 7f6246e42b80 1 -- 172.21.15.62:0/115174419 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a9788beb0 con 0x560a9795f770 2023-07-14T16:27:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.137+0000 7f6230b9a700 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a9795f770 0x560a97961b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.137+0000 7f6230b9a700 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a9795f770 0x560a97961b80 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74227ea9b8cb5a83 server_cookie=424a0c328da6a8b1 in_seq=0 out_seq=0 2023-07-14T16:27:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.137+0000 7f6214ff9700 10 client.28460 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6214ff9700 1 -- 172.21.15.62:0/115174419 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a9788beb0 con 0x560a9795f770 2023-07-14T16:27:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6214ff9700 10 client.28460 handle_command_reply: tid=0 2023-07-14T16:27:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 client.28460 shutdown 2023-07-14T16:27:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 -- 172.21.15.62:0/115174419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a97949ce0 msgr2=0x560a97953940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a97949ce0 0x560a97953940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6228007f00 tx=0x7f6228007f30 comp rx=0 tx=0).stop 2023-07-14T16:27:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 -- 172.21.15.62:0/115174419 shutdown_connections 2023-07-14T16:27:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a9794b480 0x560a9795aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a9795f770 0x560a97961b80 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a97949ce0 0x560a97953940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 --2- 172.21.15.62:0/115174419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97953e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6214ff9700 0 client.28460 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.138+0000 7f6246e42b80 1 -- 172.21.15.62:0/115174419 >> 172.21.15.62:0/115174419 conn(0x560a97936da0 msgr2=0x560a9793b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.139+0000 7f6246e42b80 1 -- 172.21.15.62:0/115174419 shutdown_connections 2023-07-14T16:27:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.139+0000 7f6246e42b80 1 -- 172.21.15.62:0/115174419 wait complete. 2023-07-14T16:27:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.139+0000 7f6246e42b80 20 client.28460 trim_cache size 0 max 16384 2023-07-14T16:27:33.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:32 smithi062 ceph-mon[95543]: pgmap v714: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:27:33.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:32 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:32 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2958792014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:27:33.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.159+0000 7f6246e42b80 1 Processor -- start 2023-07-14T16:27:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.159+0000 7f6246e42b80 1 -- start start 2023-07-14T16:27:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.159+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9795b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.159+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9795ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.159+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 0x560a9794e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.159+0000 7f6246e42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a977ec4f0 con 0x560a97949ce0 2023-07-14T16:27:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.159+0000 7f6246e42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9794ad80 con 0x560a9794a630 2023-07-14T16:27:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.159+0000 7f6246e42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a97933fd0 con 0x560a9794b480 2023-07-14T16:27:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9795ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9795ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40020/0 (socket says 172.21.15.62:40020) 2023-07-14T16:27:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 -- 172.21.15.62:0/3003569957 learned_addr learned my addr 172.21.15.62:0/3003569957 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6231b9c700 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 0x560a9794e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f623139b700 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9795b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 -- 172.21.15.62:0/3003569957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 msgr2=0x560a9794e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 0x560a9794e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 -- 172.21.15.62:0/3003569957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 msgr2=0x560a9795b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9795b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 -- 172.21.15.62:0/3003569957 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a976e7250 con 0x560a9794a630 2023-07-14T16:27:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.160+0000 7f6230b9a700 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9795ade0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f621c099d20 tx=0x7f621c0d2530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca4a9f91a93cf48c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.161+0000 7f6217fff700 1 -- 172.21.15.62:0/3003569957 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0996f0 con 0x560a9794a630 2023-07-14T16:27:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.161+0000 7f6217fff700 1 -- 172.21.15.62:0/3003569957 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f621c010880 con 0x560a9794a630 2023-07-14T16:27:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.161+0000 7f6217fff700 1 -- 172.21.15.62:0/3003569957 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c098250 con 0x560a9794a630 2023-07-14T16:27:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 -- 172.21.15.62:0/3003569957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 msgr2=0x560a9795ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9795ade0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f621c099d20 tx=0x7f621c0d2530 comp rx=0 tx=0).stop 2023-07-14T16:27:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 -- 172.21.15.62:0/3003569957 shutdown_connections 2023-07-14T16:27:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794b480 0x560a9794e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a9795b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 --2- 172.21.15.62:0/3003569957 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794a630 0x560a9795ade0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 -- 172.21.15.62:0/3003569957 >> 172.21.15.62:0/3003569957 conn(0x560a97936da0 msgr2=0x560a979664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 -- 172.21.15.62:0/3003569957 shutdown_connections 2023-07-14T16:27:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 -- 172.21.15.62:0/3003569957 wait complete. 2023-07-14T16:27:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.162+0000 7f6246e42b80 1 -- start start 2023-07-14T16:27:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.163+0000 7f6246e42b80 1 Processor -- start 2023-07-14T16:27:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.163+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a97956f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.163+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97957490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.163+0000 7f6246e42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794b480 0x560a9795a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.163+0000 7f6246e42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9795a9f0 con 0x560a97949ce0 2023-07-14T16:27:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.163+0000 7f6246e42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9795ab60 con 0x560a9794b480 2023-07-14T16:27:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.163+0000 7f6246e42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9795ad20 con 0x560a9794a630 2023-07-14T16:27:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.163+0000 7f6246e42b80 10 client.0 _ll_register_callbacks cb 0x7f6228007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f623139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a97956f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6230b9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97957490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f623139b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a97956f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54590/0 (socket says 172.21.15.62:54590) 2023-07-14T16:27:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f623139b700 1 -- 172.21.15.62:0/2157409192 learned_addr learned my addr 172.21.15.62:0/2157409192 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6231b9c700 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794b480 0x560a9795a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6230b9a700 1 -- 172.21.15.62:0/2157409192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794b480 msgr2=0x560a9795a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6230b9a700 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794b480 0x560a9795a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6230b9a700 1 -- 172.21.15.62:0/2157409192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 msgr2=0x560a97956f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6230b9a700 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a97956f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6230b9a700 1 -- 172.21.15.62:0/2157409192 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c19ffd0 con 0x560a9794a630 2023-07-14T16:27:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f623139b700 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a97956f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6230b9a700 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97957490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f621c099d20 tx=0x7f621c005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3604febe2bcdd947 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.164+0000 7f6214ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:27:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6214ff9700 1 -- 172.21.15.62:0/2157409192 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0a1f50 con 0x560a9794a630 2023-07-14T16:27:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6214ff9700 1 -- 172.21.15.62:0/2157409192 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f621c099d90 con 0x560a9794a630 2023-07-14T16:27:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6246e42b80 1 -- 172.21.15.62:0/2157409192 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6224029070 con 0x560a9794a630 2023-07-14T16:27:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6214ff9700 1 -- 172.21.15.62:0/2157409192 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f621c09ddc0 con 0x560a9794a630 2023-07-14T16:27:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6246e42b80 10 client.28345 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6246e42b80 1 -- 172.21.15.62:0/2157409192 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560a976bf2b0 con 0x560a9794a630 2023-07-14T16:27:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6214ff9700 1 -- 172.21.15.62:0/2157409192 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6224029070 con 0x560a9794a630 2023-07-14T16:27:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6214ff9700 1 -- 172.21.15.62:0/2157409192 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f621c095d80 con 0x560a9794a630 2023-07-14T16:27:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6246e42b80 10 client.28345 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6246e42b80 10 client.28345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.165+0000 7f6246e42b80 20 client.28345 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.166+0000 7f6246e42b80 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a979b46a0 0x560a979b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.166+0000 7f6246e42b80 4 client.28345 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:27:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.166+0000 7f6246e42b80 1 -- 172.21.15.62:0/2157409192 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a9784e050 con 0x560a979b46a0 2023-07-14T16:27:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.166+0000 7f623139b700 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a979b46a0 0x560a979b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.166+0000 7f623139b700 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a979b46a0 0x560a979b6ab0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5888e7f7a67ca77 server_cookie=cc95c70348257235 in_seq=0 out_seq=0 2023-07-14T16:27:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.166+0000 7f6214ff9700 10 client.28345 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6214ff9700 1 -- 172.21.15.62:0/2157409192 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a9784e050 con 0x560a979b46a0 2023-07-14T16:27:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6214ff9700 10 client.28345 handle_command_reply: tid=0 2023-07-14T16:27:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 client.28345 shutdown 2023-07-14T16:27:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 -- 172.21.15.62:0/2157409192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 msgr2=0x560a97957490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97957490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f621c099d20 tx=0x7f621c005cf0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 -- 172.21.15.62:0/2157409192 shutdown_connections 2023-07-14T16:27:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560a979b46a0 0x560a979b6ab0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560a9794b480 0x560a9795a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560a97949ce0 0x560a97956f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 --2- 172.21.15.62:0/2157409192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a9794a630 0x560a97957490 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6214ff9700 0 client.28345 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 -- 172.21.15.62:0/2157409192 >> 172.21.15.62:0/2157409192 conn(0x560a97936da0 msgr2=0x560a9795b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 -- 172.21.15.62:0/2157409192 shutdown_connections 2023-07-14T16:27:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 1 -- 172.21.15.62:0/2157409192 wait complete. 2023-07-14T16:27:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.167+0000 7f6246e42b80 20 client.28345 trim_cache size 0 max 16384 2023-07-14T16:27:33.154 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:27:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 -- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a1150 msgr2=0x7f62180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a1150 0x7f62180a3600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f622402c000 tx=0x7f622402d000 comp rx=0 tx=0).stop 2023-07-14T16:27:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 -- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c0ffbd0 msgr2=0x7f622c100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c0ffbd0 0x7f622c100fb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f621c00e9a0 tx=0x7f621c00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 -- 172.21.15.62:0/3870800446 shutdown_connections 2023-07-14T16:27:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f62180a1150 0x7f62180a3600 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f622c100560 0x7f622c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f622c0fe9e0 0x7f622c102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 --2- 172.21.15.62:0/3870800446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f622c0ffbd0 0x7f622c100fb0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 -- 172.21.15.62:0/3870800446 >> 172.21.15.62:0/3870800446 conn(0x7f622c0fa000 msgr2=0x7f622c067750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.168+0000 7f62335ff700 1 -- 172.21.15.62:0/3870800446 shutdown_connections 2023-07-14T16:27:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:33.169+0000 7f62335ff700 1 -- 172.21.15.62:0/3870800446 wait complete. 2023-07-14T16:27:33.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:32 smithi081 ceph-mon[127795]: pgmap v714: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:27:33.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.157 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.158 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:32 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.158 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:32 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2958792014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:27:33.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:32 smithi164 ceph-mon[97427]: pgmap v714: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:27:33.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:32 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:33.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:32 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2958792014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:27:34.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:27:34.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:27:34.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:34.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:27:34.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:27:34.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:34.163 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:27:34.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:27:34.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:27:34.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:27:34.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:27:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 -- 172.21.15.62:0/3905807511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 msgr2=0x7f2118101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 --2- 172.21.15.62:0/3905807511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 0x7f2118101130 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f2104009990 tx=0x7f2104009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 -- 172.21.15.62:0/3905807511 shutdown_connections 2023-07-14T16:27:34.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 --2- 172.21.15.62:0/3905807511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2118101670 0x7f2118107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 --2- 172.21.15.62:0/3905807511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21180ffaf0 0x7f21180fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 --2- 172.21.15.62:0/3905807511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 0x7f2118101130 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 -- 172.21.15.62:0/3905807511 >> 172.21.15.62:0/3905807511 conn(0x7f21180fb130 msgr2=0x7f21180fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 -- 172.21.15.62:0/3905807511 shutdown_connections 2023-07-14T16:27:34.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.418+0000 7f211edff700 1 -- 172.21.15.62:0/3905807511 wait complete. 2023-07-14T16:27:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.420+0000 7f211edff700 1 Processor -- start 2023-07-14T16:27:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.420+0000 7f211edff700 1 -- start start 2023-07-14T16:27:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.420+0000 7f211edff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21180ffaf0 0x7f2118195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.420+0000 7f211edff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 0x7f2118195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.420+0000 7f211edff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2118101670 0x7f211819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.420+0000 7f211edff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211819d2b0 con 0x7f2118100ce0 2023-07-14T16:27:34.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.420+0000 7f211edff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211819d3f0 con 0x7f2118101670 2023-07-14T16:27:34.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.420+0000 7f211edff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211819d580 con 0x7f21180ffaf0 2023-07-14T16:27:34.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 0x7f2118195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 0x7f2118195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54612/0 (socket says 172.21.15.62:54612) 2023-07-14T16:27:34.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 -- 172.21.15.62:0/959615852 learned_addr learned my addr 172.21.15.62:0/959615852 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f211d39c700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2118101670 0x7f211819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f211cb9b700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21180ffaf0 0x7f2118195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 -- 172.21.15.62:0/959615852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21180ffaf0 msgr2=0x7f2118195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21180ffaf0 0x7f2118195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 -- 172.21.15.62:0/959615852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2118101670 msgr2=0x7f211819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2118101670 0x7f211819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 -- 172.21.15.62:0/959615852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21040095f0 con 0x7f2118100ce0 2023-07-14T16:27:34.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.421+0000 7f2117fff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 0x7f2118195c70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f21040092e0 tx=0x7f2104003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.422+0000 7f2115ffb700 1 -- 172.21.15.62:0/959615852 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210401d070 con 0x7f2118100ce0 2023-07-14T16:27:34.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.422+0000 7f2115ffb700 1 -- 172.21.15.62:0/959615852 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f210400bdf0 con 0x7f2118100ce0 2023-07-14T16:27:34.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.422+0000 7f211edff700 1 -- 172.21.15.62:0/959615852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211819d800 con 0x7f2118100ce0 2023-07-14T16:27:34.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.422+0000 7f2115ffb700 1 -- 172.21.15.62:0/959615852 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104005290 con 0x7f2118100ce0 2023-07-14T16:27:34.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.422+0000 7f211edff700 1 -- 172.21.15.62:0/959615852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211819dc20 con 0x7f2118100ce0 2023-07-14T16:27:34.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.425+0000 7f2115ffb700 1 -- 172.21.15.62:0/959615852 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2104004800 con 0x7f2118100ce0 2023-07-14T16:27:34.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.427+0000 7f2115ffb700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21080a11d0 0x7f21080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.427+0000 7f2115ffb700 1 -- 172.21.15.62:0/959615852 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f21040cda90 con 0x7f2118100ce0 2023-07-14T16:27:34.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.427+0000 7f211cb9b700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21080a11d0 0x7f21080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.428+0000 7f2132639b80 1 Processor -- start 2023-07-14T16:27:34.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.428+0000 7f211cb9b700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21080a11d0 0x7f21080a3680 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2118100b40 tx=0x7f210c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.428+0000 7f2132639b80 1 -- start start 2023-07-14T16:27:34.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f2132639b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9abd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f2132639b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f2132639b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f2132639b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f8acd30 con 0x56293f9ad100 2023-07-14T16:27:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f2132639b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f8ffac0 con 0x56293f9ab960 2023-07-14T16:27:34.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f2132639b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f8f8f50 con 0x56293f9ac2b0 2023-07-14T16:27:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f211cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f211d39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f211cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9abd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40054/0 (socket says 172.21.15.62:40054) 2023-07-14T16:27:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f211cb9b700 1 -- 172.21.15.62:0/364624269 learned_addr learned my addr 172.21.15.62:0/364624269 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f211d39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54618/0 (socket says 172.21.15.62:54618) 2023-07-14T16:27:34.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.429+0000 7f2117fff700 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f211d39c700 1 -- 172.21.15.62:0/364624269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 msgr2=0x56293f9acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f211d39c700 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f211d39c700 1 -- 172.21.15.62:0/364624269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 msgr2=0x56293f9abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f211d39c700 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9abd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f211d39c700 1 -- 172.21.15.62:0/364624269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293f726e60 con 0x56293f9ad100 2023-07-14T16:27:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f2117fff700 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f211d39c700 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9ad570 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2110009c10 tx=0x7f2110009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aa4ea11d79d6b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f2102ffd700 1 -- 172.21.15.62:0/364624269 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110011cb0 con 0x56293f9ad100 2023-07-14T16:27:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f2102ffd700 1 -- 172.21.15.62:0/364624269 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2110011e10 con 0x56293f9ad100 2023-07-14T16:27:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.430+0000 7f2102ffd700 1 -- 172.21.15.62:0/364624269 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110005020 con 0x56293f9ad100 2023-07-14T16:27:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 -- 172.21.15.62:0/364624269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 msgr2=0x56293f9ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9ad570 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2110009c10 tx=0x7f2110009f20 comp rx=0 tx=0).stop 2023-07-14T16:27:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 -- 172.21.15.62:0/364624269 shutdown_connections 2023-07-14T16:27:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9ad570 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 --2- 172.21.15.62:0/364624269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 -- 172.21.15.62:0/364624269 >> 172.21.15.62:0/364624269 conn(0x56293f99ce20 msgr2=0x56293f9a2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 -- 172.21.15.62:0/364624269 shutdown_connections 2023-07-14T16:27:34.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.431+0000 7f2132639b80 1 -- 172.21.15.62:0/364624269 wait complete. 2023-07-14T16:27:34.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.432+0000 7f2132639b80 1 -- start start 2023-07-14T16:27:34.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.432+0000 7f2132639b80 1 Processor -- start 2023-07-14T16:27:34.413 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f2132639b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f2132639b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f2132639b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f2132639b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f898b40 con 0x56293f9ad100 2023-07-14T16:27:34.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f2132639b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f898840 con 0x56293f9ab960 2023-07-14T16:27:34.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f2132639b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f77c9a0 con 0x56293f9ac2b0 2023-07-14T16:27:34.414 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f2132639b80 10 client.0 _ll_register_callbacks cb 0x56293f8967e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:34.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211d39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211d39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9bcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54632/0 (socket says 172.21.15.62:54632) 2023-07-14T16:27:34.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40066/0 (socket says 172.21.15.62:40066) 2023-07-14T16:27:34.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211d39c700 1 -- 172.21.15.62:0/1603446210 learned_addr learned my addr 172.21.15.62:0/1603446210 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f2117fff700 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211cb9b700 1 -- 172.21.15.62:0/1603446210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 msgr2=0x56293f9b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211cb9b700 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211cb9b700 1 -- 172.21.15.62:0/1603446210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 msgr2=0x56293f9bcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211cb9b700 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9bcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211cb9b700 1 -- 172.21.15.62:0/1603446210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211819dc20 con 0x56293f9ab960 2023-07-14T16:27:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f211cb9b700 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b5580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f210c009fa0 tx=0x7f210c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6fe639e7286a665 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f20f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:27:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.433+0000 7f20f7fff700 1 -- 172.21.15.62:0/1603446210 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c00e040 con 0x56293f9ab960 2023-07-14T16:27:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.434+0000 7f20f7fff700 1 -- 172.21.15.62:0/1603446210 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f210c01c040 con 0x56293f9ab960 2023-07-14T16:27:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.434+0000 7f2132639b80 1 -- 172.21.15.62:0/1603446210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56293f852570 con 0x56293f9ab960 2023-07-14T16:27:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.434+0000 7f20f7fff700 1 -- 172.21.15.62:0/1603446210 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f210c015e40 con 0x56293f9ab960 2023-07-14T16:27:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.434+0000 7f2132639b80 10 client.28478 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.434+0000 7f2132639b80 1 -- 172.21.15.62:0/1603446210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f21100098c0 con 0x56293f9ab960 2023-07-14T16:27:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.434+0000 7f20f7fff700 1 -- 172.21.15.62:0/1603446210 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56293f852570 con 0x56293f9ab960 2023-07-14T16:27:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.434+0000 7f20f7fff700 1 -- 172.21.15.62:0/1603446210 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f210c004600 con 0x56293f9ab960 2023-07-14T16:27:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.435+0000 7f2132639b80 10 client.28478 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.435+0000 7f2132639b80 10 client.28478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.435+0000 7f2132639b80 20 client.28478 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.435+0000 7f2132639b80 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56293f9c1350 0x56293f9c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.435+0000 7f2132639b80 4 client.28478 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.435+0000 7f2132639b80 1 -- 172.21.15.62:0/1603446210 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56293f8f1f30 con 0x56293f9c1350 2023-07-14T16:27:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.435+0000 7f2117fff700 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56293f9c1350 0x56293f9c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.436+0000 7f2117fff700 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56293f9c1350 0x56293f9c3760 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e06cf4a2a45601 server_cookie=2dfcfd74c0d493a5 in_seq=0 out_seq=0 2023-07-14T16:27:34.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.436+0000 7f20f7fff700 10 client.28478 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.436+0000 7f20f7fff700 1 -- 172.21.15.62:0/1603446210 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56293f8f1f30 con 0x56293f9c1350 2023-07-14T16:27:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f20f7fff700 10 client.28478 handle_command_reply: tid=0 2023-07-14T16:27:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 client.28478 shutdown 2023-07-14T16:27:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 -- 172.21.15.62:0/1603446210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 msgr2=0x56293f9b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b5580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f210c009fa0 tx=0x7f210c004ad0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 -- 172.21.15.62:0/1603446210 shutdown_connections 2023-07-14T16:27:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293f9bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56293f9c1350 0x56293f9c3760 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b5580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 --2- 172.21.15.62:0/1603446210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f20f7fff700 0 client.28478 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:34.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 -- 172.21.15.62:0/1603446210 >> 172.21.15.62:0/1603446210 conn(0x56293f99ce20 msgr2=0x56293f9a1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 -- 172.21.15.62:0/1603446210 shutdown_connections 2023-07-14T16:27:34.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 1 -- 172.21.15.62:0/1603446210 wait complete. 2023-07-14T16:27:34.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.437+0000 7f2132639b80 20 client.28478 trim_cache size 0 max 16384 2023-07-14T16:27:34.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.455+0000 7f2132639b80 1 Processor -- start 2023-07-14T16:27:34.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.455+0000 7f2132639b80 1 -- start start 2023-07-14T16:27:34.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2132639b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2132639b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ac2b0 0x56293f9b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2132639b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ad100 0x56293f9bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2132639b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f9b6770 con 0x56293f9ac2b0 2023-07-14T16:27:34.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2132639b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f9b4a30 con 0x56293f9ab960 2023-07-14T16:27:34.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2132639b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f9c0ec0 con 0x56293f9ad100 2023-07-14T16:27:34.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2117fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ac2b0 0x56293f9b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2117fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ac2b0 0x56293f9b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54636/0 (socket says 172.21.15.62:54636) 2023-07-14T16:27:34.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2117fff700 1 -- 172.21.15.62:0/3903687973 learned_addr learned my addr 172.21.15.62:0/3903687973 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f211cb9b700 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f211d39c700 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ad100 0x56293f9bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2117fff700 1 -- 172.21.15.62:0/3903687973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ad100 msgr2=0x56293f9bcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2117fff700 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ad100 0x56293f9bcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2117fff700 1 -- 172.21.15.62:0/3903687973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 msgr2=0x56293f9aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.456+0000 7f2117fff700 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.457+0000 7f2117fff700 1 -- 172.21.15.62:0/3903687973 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293f9b45b0 con 0x56293f9ac2b0 2023-07-14T16:27:34.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.457+0000 7f211cb9b700 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:34.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.457+0000 7f211d39c700 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ad100 0x56293f9bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:34.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.457+0000 7f2117fff700 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ac2b0 0x56293f9b04a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f21040cb540 tx=0x7f21040cb410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7337ba04d506d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.457+0000 7f21037fe700 1 -- 172.21.15.62:0/3903687973 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210409ef10 con 0x56293f9ac2b0 2023-07-14T16:27:34.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.457+0000 7f21037fe700 1 -- 172.21.15.62:0/3903687973 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21040a32a0 con 0x56293f9ac2b0 2023-07-14T16:27:34.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.457+0000 7f21037fe700 1 -- 172.21.15.62:0/3903687973 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21040a2e10 con 0x56293f9ac2b0 2023-07-14T16:27:34.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 -- 172.21.15.62:0/3903687973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ac2b0 msgr2=0x56293f9b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ac2b0 0x56293f9b04a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f21040cb540 tx=0x7f21040cb410 comp rx=0 tx=0).stop 2023-07-14T16:27:34.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 -- 172.21.15.62:0/3903687973 shutdown_connections 2023-07-14T16:27:34.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ad100 0x56293f9bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 --2- 172.21.15.62:0/3903687973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ac2b0 0x56293f9b04a0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 -- 172.21.15.62:0/3903687973 >> 172.21.15.62:0/3903687973 conn(0x56293f99ce20 msgr2=0x56293f9bd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 -- 172.21.15.62:0/3903687973 shutdown_connections 2023-07-14T16:27:34.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 -- 172.21.15.62:0/3903687973 wait complete. 2023-07-14T16:27:34.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.458+0000 7f2132639b80 1 -- start start 2023-07-14T16:27:34.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f2132639b80 1 Processor -- start 2023-07-14T16:27:34.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f2132639b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f2132639b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f2132639b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293fa15200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f2132639b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f9bcba0 con 0x56293f9ad100 2023-07-14T16:27:34.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f2132639b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293f9bcd10 con 0x56293f9ab960 2023-07-14T16:27:34.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f2132639b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293fa15740 con 0x56293f9ac2b0 2023-07-14T16:27:34.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f2132639b80 10 client.0 _ll_register_callbacks cb 0x7f210c017880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:34.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f211d39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293fa15200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.459+0000 7f211cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f211d39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293fa15200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54640/0 (socket says 172.21.15.62:54640) 2023-07-14T16:27:34.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f211d39c700 1 -- 172.21.15.62:0/115769216 learned_addr learned my addr 172.21.15.62:0/115769216 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f2117fff700 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f211cb9b700 1 -- 172.21.15.62:0/115769216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 msgr2=0x56293f9b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f211cb9b700 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f211cb9b700 1 -- 172.21.15.62:0/115769216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 msgr2=0x56293fa15200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f211cb9b700 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293fa15200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f211cb9b700 1 -- 172.21.15.62:0/115769216 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21100098c0 con 0x56293f9ab960 2023-07-14T16:27:34.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f211cb9b700 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b9640 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f210c009f70 tx=0x7f210c0239c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bee2b2992eeefddd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.460+0000 7f20f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:27:34.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f20f7fff700 1 -- 172.21.15.62:0/115769216 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c00e040 con 0x56293f9ab960 2023-07-14T16:27:34.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f20f7fff700 1 -- 172.21.15.62:0/115769216 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f210c015e40 con 0x56293f9ab960 2023-07-14T16:27:34.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f2132639b80 1 -- 172.21.15.62:0/115769216 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2110011e10 con 0x56293f9ab960 2023-07-14T16:27:34.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f20f7fff700 1 -- 172.21.15.62:0/115769216 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f210c015a80 con 0x56293f9ab960 2023-07-14T16:27:34.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f2132639b80 10 client.28484 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:34.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f2132639b80 1 -- 172.21.15.62:0/115769216 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f211819d800 con 0x56293f9ab960 2023-07-14T16:27:34.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f20f7fff700 1 -- 172.21.15.62:0/115769216 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2110011e10 con 0x56293f9ab960 2023-07-14T16:27:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f20f7fff700 1 -- 172.21.15.62:0/115769216 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f210c0052b0 con 0x56293f9ab960 2023-07-14T16:27:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f2132639b80 10 client.28484 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f2132639b80 10 client.28484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.461+0000 7f2132639b80 20 client.28484 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.462+0000 7f2132639b80 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56293fa16fb0 0x56293fa193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.462+0000 7f2132639b80 4 client.28484 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:27:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.462+0000 7f2132639b80 1 -- 172.21.15.62:0/115769216 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56293f8b40d0 con 0x56293fa16fb0 2023-07-14T16:27:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.462+0000 7f2117fff700 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56293fa16fb0 0x56293fa193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.462+0000 7f2117fff700 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56293fa16fb0 0x56293fa193c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=587c5819b50826e1 server_cookie=977c0679412c728d in_seq=0 out_seq=0 2023-07-14T16:27:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.462+0000 7f20f7fff700 10 client.28484 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f20f7fff700 1 -- 172.21.15.62:0/115769216 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56293f8b40d0 con 0x56293fa16fb0 2023-07-14T16:27:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f20f7fff700 10 client.28484 handle_command_reply: tid=0 2023-07-14T16:27:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 client.28484 shutdown 2023-07-14T16:27:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 -- 172.21.15.62:0/115769216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 msgr2=0x56293f9b9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b9640 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f210c009f70 tx=0x7f210c0239c0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 -- 172.21.15.62:0/115769216 shutdown_connections 2023-07-14T16:27:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56293f9ad100 0x56293fa15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56293fa16fb0 0x56293fa193c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56293f9ab960 0x56293f9b9640 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 --2- 172.21.15.62:0/115769216 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56293f9ac2b0 0x56293f9b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f20f7fff700 0 client.28484 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 -- 172.21.15.62:0/115769216 >> 172.21.15.62:0/115769216 conn(0x56293f99ce20 msgr2=0x56293f9aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 -- 172.21.15.62:0/115769216 shutdown_connections 2023-07-14T16:27:34.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.464+0000 7f2132639b80 1 -- 172.21.15.62:0/115769216 wait complete. 2023-07-14T16:27:34.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f2132639b80 20 client.28484 trim_cache size 0 max 16384 2023-07-14T16:27:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:27:34.448 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:27:34.449 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "e3eda34a-3820-4b63-953b-0da98330630e", 2023-07-14T16:27:34.449 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:27:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:27:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 -- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21080a11d0 msgr2=0x7f21080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21080a11d0 0x7f21080a3680 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2118100b40 tx=0x7f210c00b040 comp rx=0 tx=0).stop 2023-07-14T16:27:34.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 -- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 msgr2=0x7f2118195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 0x7f2118195c70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f21040092e0 tx=0x7f2104003920 comp rx=0 tx=0).stop 2023-07-14T16:27:34.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 -- 172.21.15.62:0/959615852 shutdown_connections 2023-07-14T16:27:34.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21080a11d0 0x7f21080a3680 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2118101670 0x7f211819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21180ffaf0 0x7f2118195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 --2- 172.21.15.62:0/959615852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2118100ce0 0x7f2118195c70 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 -- 172.21.15.62:0/959615852 >> 172.21.15.62:0/959615852 conn(0x7f21180fb130 msgr2=0x7f2118104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 -- 172.21.15.62:0/959615852 shutdown_connections 2023-07-14T16:27:34.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.465+0000 7f211edff700 1 -- 172.21.15.62:0/959615852 wait complete. 2023-07-14T16:27:34.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:27:34.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.717+0000 7fdb60fff700 1 -- 172.21.15.62:0/3252984368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb5c0744d0 msgr2=0x7fdb5c072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.717+0000 7fdb60fff700 1 --2- 172.21.15.62:0/3252984368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb5c0744d0 0x7fdb5c072920 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009990 tx=0x7fdb4c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.717+0000 7fdb60fff700 1 -- 172.21.15.62:0/3252984368 shutdown_connections 2023-07-14T16:27:34.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.717+0000 7fdb60fff700 1 --2- 172.21.15.62:0/3252984368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c1079c0 0x7fdb5c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.717+0000 7fdb60fff700 1 --2- 172.21.15.62:0/3252984368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb5c0744d0 0x7fdb5c072920 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.717+0000 7fdb60fff700 1 --2- 172.21.15.62:0/3252984368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5c072ef0 0x7fdb5c073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.718+0000 7fdb60fff700 1 -- 172.21.15.62:0/3252984368 >> 172.21.15.62:0/3252984368 conn(0x7fdb5c0fb130 msgr2=0x7fdb5c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.718+0000 7fdb60fff700 1 -- 172.21.15.62:0/3252984368 shutdown_connections 2023-07-14T16:27:34.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.718+0000 7fdb60fff700 1 -- 172.21.15.62:0/3252984368 wait complete. 2023-07-14T16:27:34.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.719+0000 7fdb60fff700 1 Processor -- start 2023-07-14T16:27:34.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.719+0000 7fdb60fff700 1 -- start start 2023-07-14T16:27:34.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.719+0000 7fdb60fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c072ef0 0x7fdb5c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.719+0000 7fdb60fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5c0744d0 0x7fdb5c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.719+0000 7fdb60fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb5c1079c0 0x7fdb5c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.719+0000 7fdb60fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19f530 con 0x7fdb5c1079c0 2023-07-14T16:27:34.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.719+0000 7fdb60fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19f670 con 0x7fdb5c072ef0 2023-07-14T16:27:34.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.719+0000 7fdb60fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19f800 con 0x7fdb5c0744d0 2023-07-14T16:27:34.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb5c1079c0 0x7fdb5c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c072ef0 0x7fdb5c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c072ef0 0x7fdb5c1979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40092/0 (socket says 172.21.15.62:40092) 2023-07-14T16:27:34.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5a59c700 1 -- 172.21.15.62:0/4223747916 learned_addr learned my addr 172.21.15.62:0/4223747916 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5c0744d0 0x7fdb5c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5a59c700 1 -- 172.21.15.62:0/4223747916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5c0744d0 msgr2=0x7fdb5c197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.698 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5c0744d0 0x7fdb5c197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5a59c700 1 -- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb5c1079c0 msgr2=0x7fdb5c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb5c1079c0 0x7fdb5c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.720+0000 7fdb5a59c700 1 -- 172.21.15.62:0/4223747916 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c0095f0 con 0x7fdb5c072ef0 2023-07-14T16:27:34.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.721+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c072ef0 0x7fdb5c1979b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0098e0 tx=0x7fdb4c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.721+0000 7fdb4b7fe700 1 -- 172.21.15.62:0/4223747916 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c01d070 con 0x7fdb5c072ef0 2023-07-14T16:27:34.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.721+0000 7fdb4b7fe700 1 -- 172.21.15.62:0/4223747916 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb4c00be30 con 0x7fdb5c072ef0 2023-07-14T16:27:34.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.721+0000 7fdb4b7fe700 1 -- 172.21.15.62:0/4223747916 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c022db0 con 0x7fdb5c072ef0 2023-07-14T16:27:34.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.721+0000 7fdb60fff700 1 -- 172.21.15.62:0/4223747916 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c19fa80 con 0x7fdb5c072ef0 2023-07-14T16:27:34.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.721+0000 7fdb60fff700 1 -- 172.21.15.62:0/4223747916 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c19fe50 con 0x7fdb5c072ef0 2023-07-14T16:27:34.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.725+0000 7fdb4b7fe700 1 -- 172.21.15.62:0/4223747916 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdb4c022470 con 0x7fdb5c072ef0 2023-07-14T16:27:34.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.725+0000 7fdb74988b80 1 Processor -- start 2023-07-14T16:27:34.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb74988b80 1 -- start start 2023-07-14T16:27:34.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae4648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae464e20 0x5577ae4656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae465c70 0x5577ae4660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb74988b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae369cb0 con 0x5577ae465c70 2023-07-14T16:27:34.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb74988b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae3bca40 con 0x5577ae464e20 2023-07-14T16:27:34.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb74988b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae3b5ed0 con 0x5577ae4644d0 2023-07-14T16:27:34.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb59d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae465c70 0x5577ae4660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.726+0000 7fdb5ad9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae4648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb59d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae465c70 0x5577ae4660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54664/0 (socket says 172.21.15.62:54664) 2023-07-14T16:27:34.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb59d9b700 1 -- 172.21.15.62:0/4241175014 learned_addr learned my addr 172.21.15.62:0/4241175014 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae464e20 0x5577ae4656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb4b7fe700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb440a1150 0x7fdb440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb59d9b700 1 -- 172.21.15.62:0/4241175014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 msgr2=0x5577ae4648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae4648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb59d9b700 1 -- 172.21.15.62:0/4241175014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae464e20 msgr2=0x5577ae4656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae464e20 0x5577ae4656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb59d9b700 1 -- 172.21.15.62:0/4241175014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ae1e3de0 con 0x5577ae465c70 2023-07-14T16:27:34.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb5ad9d700 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae4648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:34.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb4b7fe700 1 -- 172.21.15.62:0/4223747916 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb4c0cdf20 con 0x7fdb5c072ef0 2023-07-14T16:27:34.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb5ad9d700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb440a1150 0x7fdb440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.727+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae465c70 0x5577ae4660e0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009d80 tx=0x7fdb50011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6bf06a7d3ac096 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.728+0000 7fdb48ff9700 1 -- 172.21.15.62:0/4241175014 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50004230 con 0x5577ae465c70 2023-07-14T16:27:34.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.728+0000 7fdb48ff9700 1 -- 172.21.15.62:0/4241175014 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb50004390 con 0x5577ae465c70 2023-07-14T16:27:34.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.728+0000 7fdb48ff9700 1 -- 172.21.15.62:0/4241175014 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500197e0 con 0x5577ae465c70 2023-07-14T16:27:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 -- 172.21.15.62:0/4241175014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae465c70 msgr2=0x5577ae4660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb5ad9d700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb440a1150 0x7fdb440a3600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009b60 tx=0x7fdb54004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae465c70 0x5577ae4660e0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009d80 tx=0x7fdb50011720 comp rx=0 tx=0).stop 2023-07-14T16:27:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 -- 172.21.15.62:0/4241175014 shutdown_connections 2023-07-14T16:27:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae465c70 0x5577ae4660e0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae4648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 --2- 172.21.15.62:0/4241175014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae464e20 0x5577ae4656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 -- 172.21.15.62:0/4241175014 >> 172.21.15.62:0/4241175014 conn(0x5577ae459da0 msgr2=0x5577ae45f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 -- 172.21.15.62:0/4241175014 shutdown_connections 2023-07-14T16:27:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 -- 172.21.15.62:0/4241175014 wait complete. 2023-07-14T16:27:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.729+0000 7fdb74988b80 1 -- start start 2023-07-14T16:27:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.730+0000 7fdb74988b80 1 Processor -- start 2023-07-14T16:27:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.730+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae46e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.730+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae46e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.730+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae475640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.730+0000 7fdb74988b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae355ac0 con 0x5577ae464e20 2023-07-14T16:27:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.730+0000 7fdb74988b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae3557c0 con 0x5577ae465c70 2023-07-14T16:27:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.730+0000 7fdb74988b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae239920 con 0x5577ae4644d0 2023-07-14T16:27:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.730+0000 7fdb74988b80 10 client.0 _ll_register_callbacks cb 0x5577ae353760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb59d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae46e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae46e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae46e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:52132/0 (socket says 172.21.15.62:52132) 2023-07-14T16:27:34.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 -- 172.21.15.62:0/3446528133 learned_addr learned my addr 172.21.15.62:0/3446528133 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5ad9d700 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae475640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 -- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 msgr2=0x5577ae475640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae475640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 -- 172.21.15.62:0/3446528133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 msgr2=0x5577ae46e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae46e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 -- 172.21.15.62:0/3446528133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb54003b10 con 0x5577ae4644d0 2023-07-14T16:27:34.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5ad9d700 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae475640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:34.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae46e0c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c09b2b0 tx=0x7fdb4c004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92f5af2933a8ff23 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.731+0000 7fdb41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:27:34.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.732+0000 7fdb41ffb700 1 -- 172.21.15.62:0/3446528133 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0860b0 con 0x5577ae4644d0 2023-07-14T16:27:34.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.732+0000 7fdb74988b80 1 -- 172.21.15.62:0/3446528133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577ae30f4f0 con 0x5577ae4644d0 2023-07-14T16:27:34.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.732+0000 7fdb41ffb700 1 -- 172.21.15.62:0/3446528133 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb4c00bc70 con 0x5577ae4644d0 2023-07-14T16:27:34.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.732+0000 7fdb41ffb700 1 -- 172.21.15.62:0/3446528133 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4c09aeb0 con 0x5577ae4644d0 2023-07-14T16:27:34.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.732+0000 7fdb74988b80 10 client.28375 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:34.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.732+0000 7fdb74988b80 1 -- 172.21.15.62:0/3446528133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fdb500099e0 con 0x5577ae4644d0 2023-07-14T16:27:34.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.732+0000 7fdb41ffb700 1 -- 172.21.15.62:0/3446528133 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5577ae30f4f0 con 0x5577ae4644d0 2023-07-14T16:27:34.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.733+0000 7fdb41ffb700 1 -- 172.21.15.62:0/3446528133 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb4c0a10d0 con 0x5577ae4644d0 2023-07-14T16:27:34.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.733+0000 7fdb74988b80 10 client.28375 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:34.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.733+0000 7fdb74988b80 10 client.28375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:34.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.733+0000 7fdb74988b80 20 client.28375 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:34.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.733+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577ae479a30 0x5577ae47be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.733+0000 7fdb74988b80 4 client.28375 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:34.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.733+0000 7fdb74988b80 1 -- 172.21.15.62:0/3446528133 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577ae3aeeb0 con 0x5577ae479a30 2023-07-14T16:27:34.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.733+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577ae479a30 0x5577ae47be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.734+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577ae479a30 0x5577ae47be10 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5907459bf4ba5503 server_cookie=324ee0b2e7c943a5 in_seq=0 out_seq=0 2023-07-14T16:27:34.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.734+0000 7fdb41ffb700 10 client.28375 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:34.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb41ffb700 1 -- 172.21.15.62:0/3446528133 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5577ae3aeeb0 con 0x5577ae479a30 2023-07-14T16:27:34.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb41ffb700 10 client.28375 handle_command_reply: tid=0 2023-07-14T16:27:34.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 client.28375 shutdown 2023-07-14T16:27:34.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 -- 172.21.15.62:0/3446528133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 msgr2=0x5577ae46e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae46e0c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c09b2b0 tx=0x7fdb4c004320 comp rx=0 tx=0).stop 2023-07-14T16:27:34.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 -- 172.21.15.62:0/3446528133 shutdown_connections 2023-07-14T16:27:34.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577ae479a30 0x5577ae47be10 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae475640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae4644d0 0x5577ae46e0c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb41ffb700 0 client.28375 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3446528133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae46e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 -- 172.21.15.62:0/3446528133 >> 172.21.15.62:0/3446528133 conn(0x5577ae459da0 msgr2=0x5577ae45e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 -- 172.21.15.62:0/3446528133 shutdown_connections 2023-07-14T16:27:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 1 -- 172.21.15.62:0/3446528133 wait complete. 2023-07-14T16:27:34.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.735+0000 7fdb74988b80 20 client.28375 trim_cache size 0 max 16384 2023-07-14T16:27:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.756+0000 7fdb74988b80 1 Processor -- start 2023-07-14T16:27:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.756+0000 7fdb74988b80 1 -- start start 2023-07-14T16:27:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.756+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae4644d0 0x5577ae45e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae464e20 0x5577ae475600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae46bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb74988b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae46dc90 con 0x5577ae4644d0 2023-07-14T16:27:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb74988b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae45ff20 con 0x5577ae465c70 2023-07-14T16:27:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb74988b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae460a30 con 0x5577ae464e20 2023-07-14T16:27:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae4644d0 0x5577ae45e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae4644d0 0x5577ae45e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54682/0 (socket says 172.21.15.62:54682) 2023-07-14T16:27:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5a59c700 1 -- 172.21.15.62:0/3985900165 learned_addr learned my addr 172.21.15.62:0/3985900165 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae464e20 0x5577ae475600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5ad9d700 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae46bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5a59c700 1 -- 172.21.15.62:0/3985900165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae464e20 msgr2=0x5577ae475600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae464e20 0x5577ae475600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5a59c700 1 -- 172.21.15.62:0/3985900165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 msgr2=0x5577ae46bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae46bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.757+0000 7fdb5a59c700 1 -- 172.21.15.62:0/3985900165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ae1ccc60 con 0x5577ae4644d0 2023-07-14T16:27:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.758+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae4644d0 0x5577ae45e8b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c022860 tx=0x7fdb4c0cb990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0555cc30728519d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.758+0000 7fdb497fa700 1 -- 172.21.15.62:0/3985900165 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0860b0 con 0x5577ae4644d0 2023-07-14T16:27:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.758+0000 7fdb497fa700 1 -- 172.21.15.62:0/3985900165 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb4c0cbe90 con 0x5577ae4644d0 2023-07-14T16:27:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.758+0000 7fdb497fa700 1 -- 172.21.15.62:0/3985900165 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c09f3a0 con 0x5577ae4644d0 2023-07-14T16:27:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.758+0000 7fdb74988b80 1 -- 172.21.15.62:0/3985900165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae4644d0 msgr2=0x5577ae45e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.758+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae4644d0 0x5577ae45e8b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c022860 tx=0x7fdb4c0cb990 comp rx=0 tx=0).stop 2023-07-14T16:27:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.759+0000 7fdb74988b80 1 -- 172.21.15.62:0/3985900165 shutdown_connections 2023-07-14T16:27:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.759+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae465c70 0x5577ae46bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.759+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae4644d0 0x5577ae45e8b0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.759+0000 7fdb74988b80 1 --2- 172.21.15.62:0/3985900165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae464e20 0x5577ae475600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.759+0000 7fdb74988b80 1 -- 172.21.15.62:0/3985900165 >> 172.21.15.62:0/3985900165 conn(0x5577ae459da0 msgr2=0x5577ae480740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.759+0000 7fdb74988b80 1 -- 172.21.15.62:0/3985900165 shutdown_connections 2023-07-14T16:27:34.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.759+0000 7fdb74988b80 1 -- 172.21.15.62:0/3985900165 wait complete. 2023-07-14T16:27:34.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.759+0000 7fdb74988b80 1 -- start start 2023-07-14T16:27:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb74988b80 1 Processor -- start 2023-07-14T16:27:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae4644d0 0x5577ae475080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae4755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb74988b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae465c70 0x5577ae471cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb74988b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae4723c0 con 0x5577ae464e20 2023-07-14T16:27:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb74988b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae472530 con 0x5577ae4644d0 2023-07-14T16:27:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb74988b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ae4726f0 con 0x5577ae465c70 2023-07-14T16:27:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb74988b80 10 client.0 _ll_register_callbacks cb 0x5577ae353760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb59d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae4755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb59d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae4755c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54684/0 (socket says 172.21.15.62:54684) 2023-07-14T16:27:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae4644d0 0x5577ae475080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb59d9b700 1 -- 172.21.15.62:0/2887930083 learned_addr learned my addr 172.21.15.62:0/2887930083 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.760+0000 7fdb5ad9d700 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae465c70 0x5577ae471cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb5a59c700 1 -- 172.21.15.62:0/2887930083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae465c70 msgr2=0x5577ae471cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae465c70 0x5577ae471cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb5a59c700 1 -- 172.21.15.62:0/2887930083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 msgr2=0x5577ae4755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae4755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb5a59c700 1 -- 172.21.15.62:0/2887930083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb500099e0 con 0x5577ae4644d0 2023-07-14T16:27:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae4755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb5a59c700 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae4644d0 0x5577ae475080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5577ae30f580 tx=0x7fdb4c00bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ba8cd6f40018a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:27:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb41ffb700 1 -- 172.21.15.62:0/2887930083 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c09aeb0 con 0x5577ae4644d0 2023-07-14T16:27:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb41ffb700 1 -- 172.21.15.62:0/2887930083 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdb4c09b370 con 0x5577ae4644d0 2023-07-14T16:27:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.761+0000 7fdb74988b80 1 -- 172.21.15.62:0/2887930083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb50004390 con 0x5577ae4644d0 2023-07-14T16:27:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb41ffb700 1 -- 172.21.15.62:0/2887930083 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4c09f3a0 con 0x5577ae4644d0 2023-07-14T16:27:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb74988b80 10 client.28502 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb74988b80 1 -- 172.21.15.62:0/2887930083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5577ae20a250 con 0x5577ae4644d0 2023-07-14T16:27:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb41ffb700 1 -- 172.21.15.62:0/2887930083 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb50004390 con 0x5577ae4644d0 2023-07-14T16:27:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb41ffb700 1 -- 172.21.15.62:0/2887930083 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb4c09a3f0 con 0x5577ae4644d0 2023-07-14T16:27:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb74988b80 10 client.28502 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb74988b80 10 client.28502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb74988b80 20 client.28502 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb74988b80 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577ae4cea40 0x5577ae4d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb74988b80 4 client.28502 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:27:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.762+0000 7fdb74988b80 1 -- 172.21.15.62:0/2887930083 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577ae371050 con 0x5577ae4cea40 2023-07-14T16:27:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.763+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577ae4cea40 0x5577ae4d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.763+0000 7fdb59d9b700 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577ae4cea40 0x5577ae4d0e20 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82341c764dd77f3 server_cookie=6a84c1ccb98d601e in_seq=0 out_seq=0 2023-07-14T16:27:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.763+0000 7fdb41ffb700 10 client.28502 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:34.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.763+0000 7fdb41ffb700 1 -- 172.21.15.62:0/2887930083 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5577ae371050 con 0x5577ae4cea40 2023-07-14T16:27:34.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.763+0000 7fdb41ffb700 10 client.28502 handle_command_reply: tid=0 2023-07-14T16:27:34.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.763+0000 7fdb74988b80 1 client.28502 shutdown 2023-07-14T16:27:34.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.763+0000 7fdb74988b80 1 -- 172.21.15.62:0/2887930083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae4644d0 msgr2=0x5577ae475080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.763+0000 7fdb74988b80 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae4644d0 0x5577ae475080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5577ae30f580 tx=0x7fdb4c00bfa0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 1 -- 172.21.15.62:0/2887930083 shutdown_connections 2023-07-14T16:27:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577ae465c70 0x5577ae471cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577ae4cea40 0x5577ae4d0e20 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5577ae4644d0 0x5577ae475080 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 1 --2- 172.21.15.62:0/2887930083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577ae464e20 0x5577ae4755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb41ffb700 0 client.28502 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 1 -- 172.21.15.62:0/2887930083 >> 172.21.15.62:0/2887930083 conn(0x5577ae459da0 msgr2=0x5577ae45e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 1 -- 172.21.15.62:0/2887930083 shutdown_connections 2023-07-14T16:27:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 1 -- 172.21.15.62:0/2887930083 wait complete. 2023-07-14T16:27:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb74988b80 20 client.28502 trim_cache size 0 max 16384 2023-07-14T16:27:34.750 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:27:34.750 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:27:34.751 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:27:34.751 INFO:teuthology.orchestra.run.smithi062.stdout: "e3eda34a-3820-4b63-953b-0da98330630e": { 2023-07-14T16:27:34.751 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:27:34.751 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "e3eda34a-3820-4b63-953b-0da98330630e", 2023-07-14T16:27:34.752 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:27:34.752 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:27:34.752 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:27:34.752 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:27:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb60fff700 1 -- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb440a1150 msgr2=0x7fdb440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb60fff700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb440a1150 0x7fdb440a3600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009b60 tx=0x7fdb54004ad0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb60fff700 1 -- 172.21.15.62:0/4223747916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c072ef0 msgr2=0x7fdb5c1979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.764+0000 7fdb60fff700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c072ef0 0x7fdb5c1979b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0098e0 tx=0x7fdb4c004730 comp rx=0 tx=0).stop 2023-07-14T16:27:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.765+0000 7fdb60fff700 1 -- 172.21.15.62:0/4223747916 shutdown_connections 2023-07-14T16:27:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.765+0000 7fdb60fff700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb5c1079c0 0x7fdb5c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.765+0000 7fdb60fff700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdb440a1150 0x7fdb440a3600 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.765+0000 7fdb60fff700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb5c072ef0 0x7fdb5c1979b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.765+0000 7fdb60fff700 1 --2- 172.21.15.62:0/4223747916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb5c0744d0 0x7fdb5c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.765+0000 7fdb60fff700 1 -- 172.21.15.62:0/4223747916 >> 172.21.15.62:0/4223747916 conn(0x7fdb5c0fb130 msgr2=0x7fdb5c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.765+0000 7fdb60fff700 1 -- 172.21.15.62:0/4223747916 shutdown_connections 2023-07-14T16:27:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:27:34.765+0000 7fdb60fff700 1 -- 172.21.15.62:0/4223747916 wait complete. 2023-07-14T16:27:34.759 INFO:tasks.cephfs.filesystem:scrub status for tag:e3eda34a-3820-4b63-953b-0da98330630e - {'path': '/', 'tag': 'e3eda34a-3820-4b63-953b-0da98330630e', 'options': 'recursive,force'} 2023-07-14T16:27:34.828 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:27:34 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:27:34.474+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v229>,<0x10000000003/client.0 v230>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v242>,<0x10000000001/sv_0 v256>,<0x10000000000/_nogroup v250>,<0x1/volumes v240>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:27:35.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:34 smithi062 ceph-mon[95543]: pgmap v715: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:27:35.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:34 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:27:35.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:34 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:35.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:34 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:27:35.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:34 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:35.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:34 smithi081 ceph-mon[127795]: pgmap v715: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:27:35.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:34 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:27:35.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:34 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:35.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:34 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:27:35.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:34 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:35.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:34 smithi164 ceph-mon[97427]: pgmap v715: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:27:35.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:34 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:27:35.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:34 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:35.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:34 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:27:35.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:34 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:36.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:35 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:27:36.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:35 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:36.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:35 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:27:36.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:35 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:27:36.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:35 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:36.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:35 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:27:36.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:35 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:36.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:35 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:27:36.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:35 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:27:36.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:35 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:36.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:35 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:27:36.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:35 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:36.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:35 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:27:36.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:35 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:27:36.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:35 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:27:37.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:36 smithi062 ceph-mon[95543]: pgmap v716: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:27:37.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:36 smithi081 ceph-mon[127795]: pgmap v716: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:27:37.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:36 smithi164 ceph-mon[97427]: pgmap v716: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:27:39.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:38 smithi062 ceph-mon[95543]: pgmap v717: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 27 op/s 2023-07-14T16:27:39.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:38 smithi081 ceph-mon[127795]: pgmap v717: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 27 op/s 2023-07-14T16:27:39.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:38 smithi164 ceph-mon[97427]: pgmap v717: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 MiB/s wr, 27 op/s 2023-07-14T16:27:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:40 smithi062 ceph-mon[95543]: pgmap v718: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:27:41.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:40 smithi081 ceph-mon[127795]: pgmap v718: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:27:41.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:40 smithi164 ceph-mon[97427]: pgmap v718: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:27:42.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:41 smithi062 ceph-mon[95543]: pgmap v719: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:27:42.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:41 smithi081 ceph-mon[127795]: pgmap v719: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:27:42.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:41 smithi164 ceph-mon[97427]: pgmap v719: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:27:43.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:43 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:27:43.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:43 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:27:43.649 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:43 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:27:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 -- 172.21.15.62:0/3864159996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff974101d40 msgr2=0x7ff974102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 --2- 172.21.15.62:0/3864159996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff974101d40 0x7ff974102150 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff964009990 tx=0x7ff964009ca0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 -- 172.21.15.62:0/3864159996 shutdown_connections 2023-07-14T16:27:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 --2- 172.21.15.62:0/3864159996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9741038c0 0x7ff97410a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 --2- 172.21.15.62:0/3864159996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff974101d40 0x7ff974102150 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 --2- 172.21.15.62:0/3864159996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff974102f30 0x7ff974103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 -- 172.21.15.62:0/3864159996 >> 172.21.15.62:0/3864159996 conn(0x7ff9740fd360 msgr2=0x7ff9740ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 -- 172.21.15.62:0/3864159996 shutdown_connections 2023-07-14T16:27:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.971+0000 7ff978dff700 1 -- 172.21.15.62:0/3864159996 wait complete. 2023-07-14T16:27:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.972+0000 7ff978dff700 1 Processor -- start 2023-07-14T16:27:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff978dff700 1 -- start start 2023-07-14T16:27:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff978dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974101d40 0x7ff974197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff978dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff974102f30 0x7ff974197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff978dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9741038c0 0x7ff974198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff978dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9741994a0 con 0x7ff9741038c0 2023-07-14T16:27:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff978dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974199610 con 0x7ff974101d40 2023-07-14T16:27:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff978dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9741997d0 con 0x7ff974102f30 2023-07-14T16:27:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff972d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9741038c0 0x7ff974198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff97259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974101d40 0x7ff974197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff971d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff974102f30 0x7ff974197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff971d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff974102f30 0x7ff974197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39490/0 (socket says 172.21.15.62:39490) 2023-07-14T16:27:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff972d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9741038c0 0x7ff974198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58804/0 (socket says 172.21.15.62:58804) 2023-07-14T16:27:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff971d9b700 1 -- 172.21.15.62:0/2699615381 learned_addr learned my addr 172.21.15.62:0/2699615381 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.973+0000 7ff97259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974101d40 0x7ff974197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:42268/0 (socket says 172.21.15.62:42268) 2023-07-14T16:27:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff97259c700 1 -- 172.21.15.62:0/2699615381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff974102f30 msgr2=0x7ff974197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff97259c700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff974102f30 0x7ff974197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff97259c700 1 -- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9741038c0 msgr2=0x7ff974198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff97259c700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9741038c0 0x7ff974198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff97259c700 1 -- 172.21.15.62:0/2699615381 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9640095f0 con 0x7ff974101d40 2023-07-14T16:27:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff972d9d700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9741038c0 0x7ff974198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff97259c700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974101d40 0x7ff974197a80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff964000c00 tx=0x7ff9640057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff9637fe700 1 -- 172.21.15.62:0/2699615381 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96401d070 con 0x7ff974101d40 2023-07-14T16:27:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff9637fe700 1 -- 172.21.15.62:0/2699615381 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff96400be90 con 0x7ff974101d40 2023-07-14T16:27:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.974+0000 7ff9637fe700 1 -- 172.21.15.62:0/2699615381 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96400f460 con 0x7ff974101d40 2023-07-14T16:27:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.975+0000 7ff978dff700 1 -- 172.21.15.62:0/2699615381 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff974199a50 con 0x7ff974101d40 2023-07-14T16:27:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.975+0000 7ff978dff700 1 -- 172.21.15.62:0/2699615381 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9741a0270 con 0x7ff974101d40 2023-07-14T16:27:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.979+0000 7ff9637fe700 1 -- 172.21.15.62:0/2699615381 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff964022700 con 0x7ff974101d40 2023-07-14T16:27:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.980+0000 7ff98c761b80 1 Processor -- start 2023-07-14T16:27:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.980+0000 7ff98c761b80 1 -- start start 2023-07-14T16:27:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.980+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.980+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.980+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a76e0 0x55e75d7a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.980+0000 7ff98c761b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d6abc50 con 0x55e75d7a76e0 2023-07-14T16:27:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.980+0000 7ff98c761b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d6fe9e0 con 0x55e75d7a6840 2023-07-14T16:27:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.980+0000 7ff98c761b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d6f7e70 con 0x55e75d6026b0 2023-07-14T16:27:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff971d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a76e0 0x55e75d7a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff971d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a76e0 0x55e75d7a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58812/0 (socket says 172.21.15.62:58812) 2023-07-14T16:27:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff971d9b700 1 -- 172.21.15.62:0/1544723230 learned_addr learned my addr 172.21.15.62:0/1544723230 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff9637fe700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff95c0a11d0 0x7ff95c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff97259c700 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff9637fe700 1 -- 172.21.15.62:0/2699615381 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff9640cdd10 con 0x7ff974101d40 2023-07-14T16:27:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff972d9d700 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff972d9d700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff95c0a11d0 0x7ff95c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff971d9b700 1 -- 172.21.15.62:0/1544723230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 msgr2=0x55e75d7a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff971d9b700 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff971d9b700 1 -- 172.21.15.62:0/1544723230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 msgr2=0x55e75d7a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff971d9b700 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.981+0000 7ff971d9b700 1 -- 172.21.15.62:0/1544723230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e75d525d80 con 0x55e75d7a76e0 2023-07-14T16:27:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.982+0000 7ff971d9b700 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a76e0 0x55e75d7a7b80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff968009e50 tx=0x7ff968011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c41ead7e670e4f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.982+0000 7ff960ff9700 1 -- 172.21.15.62:0/1544723230 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968004a30 con 0x55e75d7a76e0 2023-07-14T16:27:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.982+0000 7ff972d9d700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff95c0a11d0 0x7ff95c0a3680 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55e75d7a6060 tx=0x7ff96c009130 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.982+0000 7ff960ff9700 1 -- 172.21.15.62:0/1544723230 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9680054b0 con 0x55e75d7a76e0 2023-07-14T16:27:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.982+0000 7ff960ff9700 1 -- 172.21.15.62:0/1544723230 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968019720 con 0x55e75d7a76e0 2023-07-14T16:27:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 -- 172.21.15.62:0/1544723230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a76e0 msgr2=0x55e75d7a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a76e0 0x55e75d7a7b80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff968009e50 tx=0x7ff968011720 comp rx=0 tx=0).stop 2023-07-14T16:27:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 -- 172.21.15.62:0/1544723230 shutdown_connections 2023-07-14T16:27:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a76e0 0x55e75d7a7b80 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 --2- 172.21.15.62:0/1544723230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 -- 172.21.15.62:0/1544723230 >> 172.21.15.62:0/1544723230 conn(0x55e75d79bbe0 msgr2=0x55e75d7a0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 -- 172.21.15.62:0/1544723230 shutdown_connections 2023-07-14T16:27:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.983+0000 7ff98c761b80 1 -- 172.21.15.62:0/1544723230 wait complete. 2023-07-14T16:27:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.984+0000 7ff98c761b80 1 -- start start 2023-07-14T16:27:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff98c761b80 1 Processor -- start 2023-07-14T16:27:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 0x55e75d7b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff98c761b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d697a60 con 0x55e75d7a6840 2023-07-14T16:27:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff98c761b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d697760 con 0x55e75d6026b0 2023-07-14T16:27:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff98c761b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d57b8c0 con 0x55e75d7a76e0 2023-07-14T16:27:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff98c761b80 10 client.0 _ll_register_callbacks cb 0x55e75d695700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.985+0000 7ff971d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff971d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58828/0 (socket says 172.21.15.62:58828) 2023-07-14T16:27:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff971d9b700 1 -- 172.21.15.62:0/2851488721 learned_addr learned my addr 172.21.15.62:0/2851488721 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff97259c700 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff972d9d700 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 0x55e75d7b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff971d9b700 1 -- 172.21.15.62:0/2851488721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 msgr2=0x55e75d7b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff971d9b700 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 0x55e75d7b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff971d9b700 1 -- 172.21.15.62:0/2851488721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d6026b0 msgr2=0x55e75d7af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff971d9b700 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff971d9b700 1 -- 172.21.15.62:0/2851488721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff968009a80 con 0x55e75d7a6840 2023-07-14T16:27:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff971d9b700 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7afd80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff968006010 tx=0x7ff9680052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf791edff3d2f20e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.986+0000 7ff959ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.987+0000 7ff959ffb700 1 -- 172.21.15.62:0/2851488721 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968004450 con 0x55e75d7a6840 2023-07-14T16:27:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.987+0000 7ff98c761b80 1 -- 172.21.15.62:0/2851488721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e75d651490 con 0x55e75d7a6840 2023-07-14T16:27:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.987+0000 7ff959ffb700 1 -- 172.21.15.62:0/2851488721 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff968005ce0 con 0x55e75d7a6840 2023-07-14T16:27:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.987+0000 7ff959ffb700 1 -- 172.21.15.62:0/2851488721 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9680045b0 con 0x55e75d7a6840 2023-07-14T16:27:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.987+0000 7ff98c761b80 10 client.18921 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.987+0000 7ff98c761b80 1 -- 172.21.15.62:0/2851488721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e75d50ec00 con 0x55e75d7a6840 2023-07-14T16:27:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.987+0000 7ff959ffb700 1 -- 172.21.15.62:0/2851488721 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e75d651490 con 0x55e75d7a6840 2023-07-14T16:27:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.987+0000 7ff959ffb700 1 -- 172.21.15.62:0/2851488721 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff968021040 con 0x55e75d7a6840 2023-07-14T16:27:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.988+0000 7ff98c761b80 10 client.18921 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.988+0000 7ff98c761b80 10 client.18921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.988+0000 7ff98c761b80 20 client.18921 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.988+0000 7ff98c761b80 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e75d7bb5f0 0x55e75d7bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.988+0000 7ff98c761b80 4 client.18921 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.988+0000 7ff98c761b80 1 -- 172.21.15.62:0/2851488721 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e75d6f0e50 con 0x55e75d7bb5f0 2023-07-14T16:27:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.988+0000 7ff97259c700 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e75d7bb5f0 0x55e75d7bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.989+0000 7ff97259c700 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e75d7bb5f0 0x55e75d7bda00 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d1f0dfd4f60aef server_cookie=b834c77141538573 in_seq=0 out_seq=0 2023-07-14T16:27:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.989+0000 7ff959ffb700 10 client.18921 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.989+0000 7ff959ffb700 1 -- 172.21.15.62:0/2851488721 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e75d6f0e50 con 0x55e75d7bb5f0 2023-07-14T16:27:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.989+0000 7ff959ffb700 10 client.18921 handle_command_reply: tid=0 2023-07-14T16:27:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.989+0000 7ff98c761b80 1 client.18921 shutdown 2023-07-14T16:27:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 -- 172.21.15.62:0/2851488721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 msgr2=0x55e75d7afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7afd80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff968006010 tx=0x7ff9680052b0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 -- 172.21.15.62:0/2851488721 shutdown_connections 2023-07-14T16:27:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 0x55e75d7b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e75d7bb5f0 0x55e75d7bda00 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 --2- 172.21.15.62:0/2851488721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7afd80 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 -- 172.21.15.62:0/2851488721 >> 172.21.15.62:0/2851488721 conn(0x55e75d79bbe0 msgr2=0x55e75d7a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff959ffb700 0 client.18921 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 -- 172.21.15.62:0/2851488721 shutdown_connections 2023-07-14T16:27:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 1 -- 172.21.15.62:0/2851488721 wait complete. 2023-07-14T16:27:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:43.990+0000 7ff98c761b80 20 client.18921 trim_cache size 0 max 16384 2023-07-14T16:27:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff98c761b80 1 Processor -- start 2023-07-14T16:27:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff98c761b80 1 -- start start 2023-07-14T16:27:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a76e0 0x55e75d7afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff98c761b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d7a17a0 con 0x55e75d7a6840 2023-07-14T16:27:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff98c761b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d7aecf0 con 0x55e75d7a76e0 2023-07-14T16:27:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff98c761b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d79dd00 con 0x55e75d6026b0 2023-07-14T16:27:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.006+0000 7ff971d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff97259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff971d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58832/0 (socket says 172.21.15.62:58832) 2023-07-14T16:27:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff971d9b700 1 -- 172.21.15.62:0/1025361081 learned_addr learned my addr 172.21.15.62:0/1025361081 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff972d9d700 1 --2- 172.21.15.62:0/1025361081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a76e0 0x55e75d7afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff97259c700 1 -- 172.21.15.62:0/1025361081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a76e0 msgr2=0x55e75d7afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff97259c700 1 --2- 172.21.15.62:0/1025361081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a76e0 0x55e75d7afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff97259c700 1 -- 172.21.15.62:0/1025361081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 msgr2=0x55e75d7b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff97259c700 1 --2- 172.21.15.62:0/1025361081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff97259c700 1 -- 172.21.15.62:0/1025361081 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e75d54c1f0 con 0x55e75d6026b0 2023-07-14T16:27:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.007+0000 7ff97259c700 1 --2- 172.21.15.62:0/1025361081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff9640cb870 tx=0x7ff96401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77b82221d4ec368 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.008+0000 7ff9617fa700 1 -- 172.21.15.62:0/1025361081 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96409ab20 con 0x55e75d6026b0 2023-07-14T16:27:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.008+0000 7ff9617fa700 1 -- 172.21.15.62:0/1025361081 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff96409b1c0 con 0x55e75d6026b0 2023-07-14T16:27:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.008+0000 7ff9617fa700 1 -- 172.21.15.62:0/1025361081 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9640cb930 con 0x55e75d6026b0 2023-07-14T16:27:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.008+0000 7ff98c761b80 1 -- 172.21.15.62:0/1025361081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 msgr2=0x55e75d7af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.008+0000 7ff98c761b80 1 --2- 172.21.15.62:0/1025361081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff9640cb870 tx=0x7ff96401a040 comp rx=0 tx=0).stop 2023-07-14T16:27:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 -- 172.21.15.62:0/1025361081 shutdown_connections 2023-07-14T16:27:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 --2- 172.21.15.62:0/1025361081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a76e0 0x55e75d7afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 --2- 172.21.15.62:0/1025361081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d6026b0 0x55e75d7af840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 --2- 172.21.15.62:0/1025361081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d7a6840 0x55e75d7b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 -- 172.21.15.62:0/1025361081 >> 172.21.15.62:0/1025361081 conn(0x55e75d79bbe0 msgr2=0x55e75d7ada40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 -- 172.21.15.62:0/1025361081 shutdown_connections 2023-07-14T16:27:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 -- 172.21.15.62:0/1025361081 wait complete. 2023-07-14T16:27:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 -- start start 2023-07-14T16:27:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.009+0000 7ff98c761b80 1 Processor -- start 2023-07-14T16:27:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d6026b0 0x55e75d7b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff98c761b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 0x55e75d7b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff98c761b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d7b6260 con 0x55e75d6026b0 2023-07-14T16:27:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff98c761b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d7b63d0 con 0x55e75d7a6840 2023-07-14T16:27:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff98c761b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75d7b6590 con 0x55e75d7a76e0 2023-07-14T16:27:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff98c761b80 10 client.0 _ll_register_callbacks cb 0x7ff968009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff97259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d6026b0 0x55e75d7b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff97259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d6026b0 0x55e75d7b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58848/0 (socket says 172.21.15.62:58848) 2023-07-14T16:27:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff97259c700 1 -- 172.21.15.62:0/3524391603 learned_addr learned my addr 172.21.15.62:0/3524391603 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff972d9d700 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 0x55e75d7b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.010+0000 7ff971d9b700 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff97259c700 1 -- 172.21.15.62:0/3524391603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 msgr2=0x55e75d7b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff97259c700 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 0x55e75d7b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff97259c700 1 -- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 msgr2=0x55e75d7b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff97259c700 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff97259c700 1 -- 172.21.15.62:0/3524391603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9741a0270 con 0x55e75d6026b0 2023-07-14T16:27:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff971d9b700 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff97259c700 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d6026b0 0x55e75d7b2f10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff964005910 tx=0x7ff96409f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5399372b11864fc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff959ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff959ffb700 1 -- 172.21.15.62:0/3524391603 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9640a3580 con 0x55e75d6026b0 2023-07-14T16:27:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff98c761b80 1 -- 172.21.15.62:0/3524391603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9680054b0 con 0x55e75d6026b0 2023-07-14T16:27:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff959ffb700 1 -- 172.21.15.62:0/3524391603 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff9640b0e40 con 0x55e75d6026b0 2023-07-14T16:27:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff959ffb700 1 -- 172.21.15.62:0/3524391603 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9640b9310 con 0x55e75d6026b0 2023-07-14T16:27:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff98c761b80 10 client.18927 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.011+0000 7ff98c761b80 1 -- 172.21.15.62:0/3524391603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e75d524250 con 0x55e75d6026b0 2023-07-14T16:27:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.012+0000 7ff959ffb700 1 -- 172.21.15.62:0/3524391603 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9680054b0 con 0x55e75d6026b0 2023-07-14T16:27:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.012+0000 7ff959ffb700 1 -- 172.21.15.62:0/3524391603 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff9640a3180 con 0x55e75d6026b0 2023-07-14T16:27:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.012+0000 7ff98c761b80 10 client.18927 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.012+0000 7ff98c761b80 10 client.18927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.012+0000 7ff98c761b80 20 client.18927 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.012+0000 7ff98c761b80 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e75d80fb90 0x55e75d811fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.012+0000 7ff98c761b80 4 client.18927 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:27:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.012+0000 7ff98c761b80 1 -- 172.21.15.62:0/3524391603 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e75d6b2ff0 con 0x55e75d80fb90 2023-07-14T16:27:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.013+0000 7ff971d9b700 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e75d80fb90 0x55e75d811fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.013+0000 7ff959ffb700 10 client.18927 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.013+0000 7ff971d9b700 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e75d80fb90 0x55e75d811fa0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbabff148368e0e server_cookie=aee6e66ab6aa9fe in_seq=0 out_seq=0 2023-07-14T16:27:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff959ffb700 1 -- 172.21.15.62:0/3524391603 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e75d6b2ff0 con 0x55e75d80fb90 2023-07-14T16:27:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff959ffb700 10 client.18927 handle_command_reply: tid=0 2023-07-14T16:27:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 client.18927 shutdown 2023-07-14T16:27:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 -- 172.21.15.62:0/3524391603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d6026b0 msgr2=0x55e75d7b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d6026b0 0x55e75d7b2f10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff964005910 tx=0x7ff96409f120 comp rx=0 tx=0).stop 2023-07-14T16:27:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 -- 172.21.15.62:0/3524391603 shutdown_connections 2023-07-14T16:27:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e75d7a76e0 0x55e75d7b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e75d6026b0 0x55e75d7b2f10 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e75d80fb90 0x55e75d811fa0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 --2- 172.21.15.62:0/3524391603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e75d7a6840 0x55e75d7b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 -- 172.21.15.62:0/3524391603 >> 172.21.15.62:0/3524391603 conn(0x55e75d79bbe0 msgr2=0x55e75d7af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff959ffb700 0 client.18927 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 -- 172.21.15.62:0/3524391603 shutdown_connections 2023-07-14T16:27:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 1 -- 172.21.15.62:0/3524391603 wait complete. 2023-07-14T16:27:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.023+0000 7ff98c761b80 20 client.18927 trim_cache size 0 max 16384 2023-07-14T16:27:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:27:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:27:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:27:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:27:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 -- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff95c0a11d0 msgr2=0x7ff95c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff95c0a11d0 0x7ff95c0a3680 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55e75d7a6060 tx=0x7ff96c009130 comp rx=0 tx=0).stop 2023-07-14T16:27:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 -- 172.21.15.62:0/2699615381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974101d40 msgr2=0x7ff974197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974101d40 0x7ff974197a80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff964000c00 tx=0x7ff9640057b0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 -- 172.21.15.62:0/2699615381 shutdown_connections 2023-07-14T16:27:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9741038c0 0x7ff974198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff95c0a11d0 0x7ff95c0a3680 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974101d40 0x7ff974197a80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 --2- 172.21.15.62:0/2699615381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff974102f30 0x7ff974197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 -- 172.21.15.62:0/2699615381 >> 172.21.15.62:0/2699615381 conn(0x7ff9740fd360 msgr2=0x7ff974106460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 -- 172.21.15.62:0/2699615381 shutdown_connections 2023-07-14T16:27:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:44.024+0000 7ff978dff700 1 -- 172.21.15.62:0/2699615381 wait complete. 2023-07-14T16:27:44.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:44 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:27:44.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:44 smithi164 ceph-mon[97427]: pgmap v720: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:27:44.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:44 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:27:44.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:44 smithi062 ceph-mon[95543]: pgmap v720: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:27:44.649 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:44 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:27:44.650 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:44 smithi081 ceph-mon[127795]: pgmap v720: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:27:47.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:46 smithi164 ceph-mon[97427]: pgmap v721: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:27:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:46 smithi062 ceph-mon[95543]: pgmap v721: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:27:47.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:46 smithi081 ceph-mon[127795]: pgmap v721: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-14T16:27:49.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:48 smithi164 ceph-mon[97427]: pgmap v722: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 31 op/s 2023-07-14T16:27:49.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:48 smithi062 ceph-mon[95543]: pgmap v722: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 31 op/s 2023-07-14T16:27:49.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:48 smithi081 ceph-mon[127795]: pgmap v722: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 31 op/s 2023-07-14T16:27:51.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:50 smithi164 ceph-mon[97427]: pgmap v723: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 25 op/s 2023-07-14T16:27:51.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:50 smithi062 ceph-mon[95543]: pgmap v723: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 25 op/s 2023-07-14T16:27:51.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:50 smithi081 ceph-mon[127795]: pgmap v723: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 25 op/s 2023-07-14T16:27:53.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:52 smithi164 ceph-mon[97427]: pgmap v724: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:27:53.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:52 smithi062 ceph-mon[95543]: pgmap v724: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:27:53.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:52 smithi081 ceph-mon[127795]: pgmap v724: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:27:55.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:54 smithi164 ceph-mon[97427]: pgmap v725: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:27:55.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:54 smithi062 ceph-mon[95543]: pgmap v725: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:27:55.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:54 smithi081 ceph-mon[127795]: pgmap v725: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:27:57.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:56 smithi164 ceph-mon[97427]: pgmap v726: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:27:57.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:56 smithi062 ceph-mon[95543]: pgmap v726: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:27:57.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:56 smithi081 ceph-mon[127795]: pgmap v726: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:27:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.208+0000 7f0748999700 1 -- 172.21.15.62:0/1563101301 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c007c90 con 0x7f0744105860 2023-07-14T16:27:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.208+0000 7f074bbff700 1 -- 172.21.15.62:0/1563101301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744105860 msgr2=0x7f0744107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.208+0000 7f074bbff700 1 --2- 172.21.15.62:0/1563101301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744105860 0x7f0744107c40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f073c009ec0 tx=0x7f073c00b330 comp rx=0 tx=0).stop 2023-07-14T16:27:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.208+0000 7f074bbff700 1 -- 172.21.15.62:0/1563101301 shutdown_connections 2023-07-14T16:27:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.208+0000 7f074bbff700 1 --2- 172.21.15.62:0/1563101301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744105860 0x7f0744107c40 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.208+0000 7f074bbff700 1 --2- 172.21.15.62:0/1563101301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0744100620 0x7f0744102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.208+0000 7f074bbff700 1 --2- 172.21.15.62:0/1563101301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0744102f40 0x7f0744105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.208+0000 7f074bbff700 1 -- 172.21.15.62:0/1563101301 >> 172.21.15.62:0/1563101301 conn(0x7f07440fa000 msgr2=0x7f07440fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.209+0000 7f074bbff700 1 -- 172.21.15.62:0/1563101301 shutdown_connections 2023-07-14T16:27:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.209+0000 7f074bbff700 1 -- 172.21.15.62:0/1563101301 wait complete. 2023-07-14T16:27:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.210+0000 7f074bbff700 1 Processor -- start 2023-07-14T16:27:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.210+0000 7f074bbff700 1 -- start start 2023-07-14T16:27:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074bbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0744100620 0x7f074419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074bbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744102f40 0x7f074419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074bbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0744105860 0x7f0744193490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074bbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07441939d0 con 0x7f0744102f40 2023-07-14T16:27:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074bbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744193b40 con 0x7f0744105860 2023-07-14T16:27:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074bbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744193d00 con 0x7f0744100620 2023-07-14T16:27:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074919a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744102f40 0x7f074419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074919a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744102f40 0x7f074419dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48196/0 (socket says 172.21.15.62:48196) 2023-07-14T16:27:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.211+0000 7f074919a700 1 -- 172.21.15.62:0/2472364277 learned_addr learned my addr 172.21.15.62:0/2472364277 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074a19c700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0744105860 0x7f0744193490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074999b700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0744100620 0x7f074419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074919a700 1 -- 172.21.15.62:0/2472364277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0744100620 msgr2=0x7f074419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074919a700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0744100620 0x7f074419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074919a700 1 -- 172.21.15.62:0/2472364277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0744105860 msgr2=0x7f0744193490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074919a700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0744105860 0x7f0744193490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074919a700 1 -- 172.21.15.62:0/2472364277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c005e70 con 0x7f0744102f40 2023-07-14T16:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074999b700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0744100620 0x7f074419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:27:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074919a700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744102f40 0x7f074419dc50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9a0 tx=0x7f073400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f073affd700 1 -- 172.21.15.62:0/2472364277 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734009790 con 0x7f0744102f40 2023-07-14T16:27:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f074bbff700 1 -- 172.21.15.62:0/2472364277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0744193fe0 con 0x7f0744102f40 2023-07-14T16:27:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f073affd700 1 -- 172.21.15.62:0/2472364277 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f073400cbd0 con 0x7f0744102f40 2023-07-14T16:27:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.212+0000 7f073affd700 1 -- 172.21.15.62:0/2472364277 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734018680 con 0x7f0744102f40 2023-07-14T16:27:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.213+0000 7f074bbff700 1 -- 172.21.15.62:0/2472364277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07441a4920 con 0x7f0744102f40 2023-07-14T16:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.217+0000 7f073affd700 1 -- 172.21.15.62:0/2472364277 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f073400cd40 con 0x7f0744102f40 2023-07-14T16:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.217+0000 7f075f49cb80 1 Processor -- start 2023-07-14T16:27:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f075f49cb80 1 -- start start 2023-07-14T16:27:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d03300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d04b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f075f49cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1c08c50 con 0x557aa1b5f6b0 2023-07-14T16:27:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f075f49cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1c5b9e0 con 0x557aa1d03840 2023-07-14T16:27:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f075f49cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1c54e70 con 0x557aa1d046e0 2023-07-14T16:27:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f074a19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d03300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f074a19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d03300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48200/0 (socket says 172.21.15.62:48200) 2023-07-14T16:27:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.218+0000 7f074a19c700 1 -- 172.21.15.62:0/1707241662 learned_addr learned my addr 172.21.15.62:0/1707241662 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074999b700 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074919a700 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d04b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074a19c700 1 -- 172.21.15.62:0/1707241662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 msgr2=0x557aa1d04b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074a19c700 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d04b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074a19c700 1 -- 172.21.15.62:0/1707241662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 msgr2=0x557aa1d041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074a19c700 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074a19c700 1 -- 172.21.15.62:0/1707241662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa1a82d80 con 0x557aa1b5f6b0 2023-07-14T16:27:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f073affd700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07300a1150 0x7f07300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f073affd700 1 -- 172.21.15.62:0/2472364277 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0734014070 con 0x7f0744102f40 2023-07-14T16:27:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074a19c700 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d03300 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f073c006110 tx=0x7f073c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773cf58e2948a93f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.219+0000 7f074999b700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07300a1150 0x7f07300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.220+0000 7f072ffff700 1 -- 172.21.15.62:0/1707241662 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00e040 con 0x557aa1b5f6b0 2023-07-14T16:27:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.220+0000 7f072ffff700 1 -- 172.21.15.62:0/1707241662 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f073c00be60 con 0x557aa1b5f6b0 2023-07-14T16:27:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.220+0000 7f072ffff700 1 -- 172.21.15.62:0/1707241662 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c01c820 con 0x557aa1b5f6b0 2023-07-14T16:27:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 -- 172.21.15.62:0/1707241662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 msgr2=0x557aa1d03300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f074999b700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07300a1150 0x7f07300a3600 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f07440683c0 tx=0x7f0740009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d03300 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f073c006110 tx=0x7f073c009b80 comp rx=0 tx=0).stop 2023-07-14T16:27:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 -- 172.21.15.62:0/1707241662 shutdown_connections 2023-07-14T16:27:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d04b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d03300 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 --2- 172.21.15.62:0/1707241662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 -- 172.21.15.62:0/1707241662 >> 172.21.15.62:0/1707241662 conn(0x557aa1cf8be0 msgr2=0x557aa1cfdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 -- 172.21.15.62:0/1707241662 shutdown_connections 2023-07-14T16:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.221+0000 7f075f49cb80 1 -- 172.21.15.62:0/1707241662 wait complete. 2023-07-14T16:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.222+0000 7f075f49cb80 1 -- start start 2023-07-14T16:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.222+0000 7f075f49cb80 1 Processor -- start 2023-07-14T16:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.222+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.222+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d03840 0x557aa1d0cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.222+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d046e0 0x557aa1d13de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.222+0000 7f075f49cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1bf4a60 con 0x557aa1d046e0 2023-07-14T16:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.222+0000 7f075f49cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1bf4760 con 0x557aa1b5f6b0 2023-07-14T16:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.222+0000 7f075f49cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1ad88c0 con 0x557aa1d03840 2023-07-14T16:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f075f49cb80 10 client.0 _ll_register_callbacks cb 0x557aa1bf2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074a19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d046e0 0x557aa1d13de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074a19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d046e0 0x557aa1d13de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48204/0 (socket says 172.21.15.62:48204) 2023-07-14T16:27:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074a19c700 1 -- 172.21.15.62:0/3321431535 learned_addr learned my addr 172.21.15.62:0/3321431535 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074999b700 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074919a700 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d03840 0x557aa1d0cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074a19c700 1 -- 172.21.15.62:0/3321431535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d03840 msgr2=0x557aa1d0cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074a19c700 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d03840 0x557aa1d0cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074a19c700 1 -- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 msgr2=0x557aa1d0c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074a19c700 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074a19c700 1 -- 172.21.15.62:0/3321431535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0744108860 con 0x557aa1d046e0 2023-07-14T16:27:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.223+0000 7f074999b700 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.224+0000 7f074a19c700 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d046e0 0x557aa1d13de0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f073c00d010 tx=0x7f073c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c51034f8448486 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.224+0000 7f072d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.224+0000 7f072d7fa700 1 -- 172.21.15.62:0/3321431535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00e050 con 0x557aa1d046e0 2023-07-14T16:27:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.224+0000 7f075f49cb80 1 -- 172.21.15.62:0/3321431535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557aa1bae490 con 0x557aa1d046e0 2023-07-14T16:27:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.224+0000 7f072d7fa700 1 -- 172.21.15.62:0/3321431535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f073c01c930 con 0x557aa1d046e0 2023-07-14T16:27:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.224+0000 7f072d7fa700 1 -- 172.21.15.62:0/3321431535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f073c0075e0 con 0x557aa1d046e0 2023-07-14T16:27:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.224+0000 7f075f49cb80 10 client.18939 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.224+0000 7f075f49cb80 1 -- 172.21.15.62:0/3321431535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557aa1a6bc00 con 0x557aa1d046e0 2023-07-14T16:27:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.225+0000 7f072d7fa700 1 -- 172.21.15.62:0/3321431535 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557aa1bae490 con 0x557aa1d046e0 2023-07-14T16:27:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.225+0000 7f072d7fa700 1 -- 172.21.15.62:0/3321431535 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f073c019050 con 0x557aa1d046e0 2023-07-14T16:27:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.225+0000 7f075f49cb80 10 client.18939 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.225+0000 7f075f49cb80 10 client.18939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.225+0000 7f075f49cb80 20 client.18939 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.225+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557aa1d18170 0x557aa1d1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.225+0000 7f075f49cb80 4 client.18939 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:27:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.225+0000 7f075f49cb80 1 -- 172.21.15.62:0/3321431535 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557aa1c4de50 con 0x557aa1d18170 2023-07-14T16:27:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.226+0000 7f074999b700 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557aa1d18170 0x557aa1d1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.226+0000 7f074999b700 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557aa1d18170 0x557aa1d1a580 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773bed63a1ce4bcf server_cookie=a2c4bc09b91b33b in_seq=0 out_seq=0 2023-07-14T16:27:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.226+0000 7f072d7fa700 10 client.18939 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f072d7fa700 1 -- 172.21.15.62:0/3321431535 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557aa1c4de50 con 0x557aa1d18170 2023-07-14T16:27:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f072d7fa700 10 client.18939 handle_command_reply: tid=0 2023-07-14T16:27:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 client.18939 shutdown 2023-07-14T16:27:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 -- 172.21.15.62:0/3321431535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d046e0 msgr2=0x557aa1d13de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d046e0 0x557aa1d13de0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f073c00d010 tx=0x7f073c00bc80 comp rx=0 tx=0).stop 2023-07-14T16:27:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 -- 172.21.15.62:0/3321431535 shutdown_connections 2023-07-14T16:27:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d046e0 0x557aa1d13de0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557aa1d18170 0x557aa1d1a580 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3321431535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d03840 0x557aa1d0cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 -- 172.21.15.62:0/3321431535 >> 172.21.15.62:0/3321431535 conn(0x557aa1cf8be0 msgr2=0x557aa1cfd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f072d7fa700 0 client.18939 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.227+0000 7f075f49cb80 1 -- 172.21.15.62:0/3321431535 shutdown_connections 2023-07-14T16:27:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.228+0000 7f075f49cb80 1 -- 172.21.15.62:0/3321431535 wait complete. 2023-07-14T16:27:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.228+0000 7f075f49cb80 20 client.18939 trim_cache size 0 max 16384 2023-07-14T16:27:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.244+0000 7f075f49cb80 1 Processor -- start 2023-07-14T16:27:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.244+0000 7f075f49cb80 1 -- start start 2023-07-14T16:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d03840 0x557aa1d13da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d0ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f075f49cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1d0c3d0 con 0x557aa1d03840 2023-07-14T16:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f075f49cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1cfe9f0 con 0x557aa1b5f6b0 2023-07-14T16:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f075f49cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1cff780 con 0x557aa1d046e0 2023-07-14T16:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074919a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d03840 0x557aa1d13da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074919a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d03840 0x557aa1d13da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48206/0 (socket says 172.21.15.62:48206) 2023-07-14T16:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074919a700 1 -- 172.21.15.62:0/1853668225 learned_addr learned my addr 172.21.15.62:0/1853668225 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074999b700 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074a19c700 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d0ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074919a700 1 -- 172.21.15.62:0/1853668225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 msgr2=0x557aa1d0ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074919a700 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d0ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074919a700 1 -- 172.21.15.62:0/1853668225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 msgr2=0x557aa1d0b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074919a700 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.245+0000 7f074919a700 1 -- 172.21.15.62:0/1853668225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa1aa91f0 con 0x557aa1d03840 2023-07-14T16:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.246+0000 7f074919a700 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d03840 0x557aa1d13da0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f07340ca6b0 tx=0x7f073400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f818c3c8635c0214 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.246+0000 7f0738bff700 1 -- 172.21.15.62:0/1853668225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734099d90 con 0x557aa1d03840 2023-07-14T16:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.246+0000 7f0738bff700 1 -- 172.21.15.62:0/1853668225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0734099b00 con 0x557aa1d03840 2023-07-14T16:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.246+0000 7f0738bff700 1 -- 172.21.15.62:0/1853668225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734098480 con 0x557aa1d03840 2023-07-14T16:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 -- 172.21.15.62:0/1853668225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d03840 msgr2=0x557aa1d13da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d03840 0x557aa1d13da0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f07340ca6b0 tx=0x7f073400eef0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 -- 172.21.15.62:0/1853668225 shutdown_connections 2023-07-14T16:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d0ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 --2- 172.21.15.62:0/1853668225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1d03840 0x557aa1d13da0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 -- 172.21.15.62:0/1853668225 >> 172.21.15.62:0/1853668225 conn(0x557aa1cf8be0 msgr2=0x557aa1d1eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 -- 172.21.15.62:0/1853668225 shutdown_connections 2023-07-14T16:27:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 -- 172.21.15.62:0/1853668225 wait complete. 2023-07-14T16:27:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.247+0000 7f075f49cb80 1 -- start start 2023-07-14T16:27:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.248+0000 7f075f49cb80 1 Processor -- start 2023-07-14T16:27:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.248+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.248+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d0f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.248+0000 7f075f49cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.248+0000 7f075f49cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1d12e30 con 0x557aa1b5f6b0 2023-07-14T16:27:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.248+0000 7f075f49cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1d12fa0 con 0x557aa1d03840 2023-07-14T16:27:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.248+0000 7f075f49cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa1d13160 con 0x557aa1d046e0 2023-07-14T16:27:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.248+0000 7f075f49cb80 10 client.0 _ll_register_callbacks cb 0x7f0740009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:27:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074919a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d0f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48220/0 (socket says 172.21.15.62:48220) 2023-07-14T16:27:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 -- 172.21.15.62:0/3898425511 learned_addr learned my addr 172.21.15.62:0/3898425511 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:27:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074a19c700 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 -- 172.21.15.62:0/3898425511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 msgr2=0x557aa1d128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 -- 172.21.15.62:0/3898425511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 msgr2=0x557aa1d0f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:27:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d0f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 -- 172.21.15.62:0/3898425511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07441a4920 con 0x557aa1b5f6b0 2023-07-14T16:27:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074919a700 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d0f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:27:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f074999b700 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0f390 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0740009c20 tx=0x7f07400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c899532bf1e4ecf9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:27:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.249+0000 7f072d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:27:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f072d7fa700 1 -- 172.21.15.62:0/3898425511 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740025040 con 0x557aa1b5f6b0 2023-07-14T16:27:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f075f49cb80 1 -- 172.21.15.62:0/3898425511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f073c00be60 con 0x557aa1b5f6b0 2023-07-14T16:27:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f072d7fa700 1 -- 172.21.15.62:0/3898425511 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f074002be20 con 0x557aa1b5f6b0 2023-07-14T16:27:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f072d7fa700 1 -- 172.21.15.62:0/3898425511 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f074003a030 con 0x557aa1b5f6b0 2023-07-14T16:27:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f075f49cb80 10 client.18945 fetch_fsmap learned FSMap version 25 2023-07-14T16:27:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f075f49cb80 1 -- 172.21.15.62:0/3898425511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557aa1a81250 con 0x557aa1b5f6b0 2023-07-14T16:27:57.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f072d7fa700 1 -- 172.21.15.62:0/3898425511 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f073c00be60 con 0x557aa1b5f6b0 2023-07-14T16:27:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f072d7fa700 1 -- 172.21.15.62:0/3898425511 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f07400345a0 con 0x557aa1b5f6b0 2023-07-14T16:27:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f075f49cb80 10 client.18945 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:27:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f075f49cb80 10 client.18945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:27:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.250+0000 7f075f49cb80 20 client.18945 populate_metadata read hostname 'smithi062' 2023-07-14T16:27:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.251+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557aa1d6c8b0 0x557aa1d6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:27:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.251+0000 7f075f49cb80 4 client.18945 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:27:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.251+0000 7f075f49cb80 1 -- 172.21.15.62:0/3898425511 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557aa1c0fff0 con 0x557aa1d6c8b0 2023-07-14T16:27:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.251+0000 7f074919a700 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557aa1d6c8b0 0x557aa1d6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:27:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.251+0000 7f074919a700 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557aa1d6c8b0 0x557aa1d6ecc0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3d18f48b726dc5 server_cookie=3118b479557f0d48 in_seq=0 out_seq=0 2023-07-14T16:27:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.251+0000 7f072d7fa700 10 client.18945 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f072d7fa700 1 -- 172.21.15.62:0/3898425511 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557aa1c0fff0 con 0x557aa1d6c8b0 2023-07-14T16:27:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f072d7fa700 10 client.18945 handle_command_reply: tid=0 2023-07-14T16:27:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 client.18945 shutdown 2023-07-14T16:27:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 -- 172.21.15.62:0/3898425511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 msgr2=0x557aa1d0f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0f390 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0740009c20 tx=0x7f07400047c0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 -- 172.21.15.62:0/3898425511 shutdown_connections 2023-07-14T16:27:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557aa1d046e0 0x557aa1d128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557aa1b5f6b0 0x557aa1d0f390 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557aa1d6c8b0 0x557aa1d6ecc0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 --2- 172.21.15.62:0/3898425511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557aa1d03840 0x557aa1d0f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 -- 172.21.15.62:0/3898425511 >> 172.21.15.62:0/3898425511 conn(0x557aa1cf8be0 msgr2=0x557aa1d0b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f072d7fa700 0 client.18945 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:27:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 -- 172.21.15.62:0/3898425511 shutdown_connections 2023-07-14T16:27:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 1 -- 172.21.15.62:0/3898425511 wait complete. 2023-07-14T16:27:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.262+0000 7f075f49cb80 20 client.18945 trim_cache size 0 max 16384 2023-07-14T16:27:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:27:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:27:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:27:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:27:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 -- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07300a1150 msgr2=0x7f07300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07300a1150 0x7f07300a3600 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f07440683c0 tx=0x7f0740009260 comp rx=0 tx=0).stop 2023-07-14T16:27:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 -- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744102f40 msgr2=0x7f074419dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:27:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744102f40 0x7f074419dc50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9a0 tx=0x7f073400ecb0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 -- 172.21.15.62:0/2472364277 shutdown_connections 2023-07-14T16:27:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07300a1150 0x7f07300a3600 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0744105860 0x7f0744193490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0744100620 0x7f074419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 --2- 172.21.15.62:0/2472364277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0744102f40 0x7f074419dc50 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:27:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 -- 172.21.15.62:0/2472364277 >> 172.21.15.62:0/2472364277 conn(0x7f07440fa000 msgr2=0x7f0744101300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:27:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 -- 172.21.15.62:0/2472364277 shutdown_connections 2023-07-14T16:27:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:27:57.263+0000 7f074bbff700 1 -- 172.21.15.62:0/2472364277 wait complete. 2023-07-14T16:27:59.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:27:58 smithi164 ceph-mon[97427]: pgmap v727: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 17 MiB/s wr, 13 op/s 2023-07-14T16:27:59.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:27:58 smithi062 ceph-mon[95543]: pgmap v727: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 17 MiB/s wr, 13 op/s 2023-07-14T16:27:59.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:27:58 smithi081 ceph-mon[127795]: pgmap v727: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 17 MiB/s wr, 13 op/s 2023-07-14T16:28:01.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:00 smithi164 ceph-mon[97427]: pgmap v728: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:28:01.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:00 smithi062 ceph-mon[95543]: pgmap v728: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:28:01.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:00 smithi081 ceph-mon[127795]: pgmap v728: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:28:03.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:02 smithi164 ceph-mon[97427]: pgmap v729: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:28:03.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:02 smithi062 ceph-mon[95543]: pgmap v729: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:28:03.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:02 smithi081 ceph-mon[127795]: pgmap v729: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:28:04.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:28:05.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.025+0000 7fa93c9ff700 1 -- 172.21.15.62:0/2833933523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa938101670 msgr2=0x7fa938107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.025+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/2833933523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa938101670 0x7fa938107f30 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa93000b0a0 tx=0x7fa93000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.025+0000 7fa93c9ff700 1 -- 172.21.15.62:0/2833933523 shutdown_connections 2023-07-14T16:28:05.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.025+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/2833933523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa938101670 0x7fa938107f30 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.025+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/2833933523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9380ffaf0 0x7fa9380fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.025+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/2833933523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa938100ce0 0x7fa938101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.025+0000 7fa93c9ff700 1 -- 172.21.15.62:0/2833933523 >> 172.21.15.62:0/2833933523 conn(0x7fa9380fb130 msgr2=0x7fa9380fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.026+0000 7fa93c9ff700 1 -- 172.21.15.62:0/2833933523 shutdown_connections 2023-07-14T16:28:05.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.026+0000 7fa93c9ff700 1 -- 172.21.15.62:0/2833933523 wait complete. 2023-07-14T16:28:05.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.027+0000 7fa93c9ff700 1 Processor -- start 2023-07-14T16:28:05.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.027+0000 7fa93c9ff700 1 -- start start 2023-07-14T16:28:05.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93c9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9380ffaf0 0x7fa9381958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93c9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa938100ce0 0x7fa938195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93c9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa938101670 0x7fa93819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93c9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93819d430 con 0x7fa9380ffaf0 2023-07-14T16:28:05.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93c9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93819d570 con 0x7fa938100ce0 2023-07-14T16:28:05.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93c9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93819d700 con 0x7fa938101670 2023-07-14T16:28:05.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9380ffaf0 0x7fa9381958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9380ffaf0 0x7fa9381958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46074/0 (socket says 172.21.15.62:46074) 2023-07-14T16:28:05.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93659c700 1 -- 172.21.15.62:0/460656642 learned_addr learned my addr 172.21.15.62:0/460656642 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa936d9d700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa938101670 0x7fa93819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa935d9b700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa938100ce0 0x7fa938195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93659c700 1 -- 172.21.15.62:0/460656642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa938101670 msgr2=0x7fa93819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93659c700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa938101670 0x7fa93819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93659c700 1 -- 172.21.15.62:0/460656642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa938100ce0 msgr2=0x7fa938195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93659c700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa938100ce0 0x7fa938195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa93659c700 1 -- 172.21.15.62:0/460656642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa930009d00 con 0x7fa9380ffaf0 2023-07-14T16:28:05.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.028+0000 7fa936d9d700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa938101670 0x7fa93819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:05.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.029+0000 7fa93659c700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9380ffaf0 0x7fa9381958b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa92800e970 tx=0x7fa92800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.029+0000 7fa9277fe700 1 -- 172.21.15.62:0/460656642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928009790 con 0x7fa9380ffaf0 2023-07-14T16:28:05.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.029+0000 7fa9277fe700 1 -- 172.21.15.62:0/460656642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9280044b0 con 0x7fa9380ffaf0 2023-07-14T16:28:05.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.029+0000 7fa93c9ff700 1 -- 172.21.15.62:0/460656642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93819d9e0 con 0x7fa9380ffaf0 2023-07-14T16:28:05.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.029+0000 7fa9277fe700 1 -- 172.21.15.62:0/460656642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928003dd0 con 0x7fa9380ffaf0 2023-07-14T16:28:05.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.029+0000 7fa93c9ff700 1 -- 172.21.15.62:0/460656642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93819def0 con 0x7fa9380ffaf0 2023-07-14T16:28:05.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.032+0000 7fa9277fe700 1 -- 172.21.15.62:0/460656642 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa928003f30 con 0x7fa9380ffaf0 2023-07-14T16:28:05.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.034+0000 7fa9503a5b80 1 Processor -- start 2023-07-14T16:28:05.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.034+0000 7fa9503a5b80 1 -- start start 2023-07-14T16:28:05.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa9277fe700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9200a1150 0x7fa9200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa9277fe700 1 -- 172.21.15.62:0/460656642 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa928014070 con 0x7fa9380ffaf0 2023-07-14T16:28:05.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a18e20 0x55c243a196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa936d9d700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9200a1150 0x7fa9200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a19c70 0x55c243a1a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24391dcb0 con 0x55c243a19c70 2023-07-14T16:28:05.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243970a40 con 0x55c243a184d0 2023-07-14T16:28:05.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243969ed0 con 0x55c243a18e20 2023-07-14T16:28:05.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa936d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a19c70 0x55c243a1a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa93659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa936d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a19c70 0x55c243a1a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46084/0 (socket says 172.21.15.62:46084) 2023-07-14T16:28:05.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa935d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a18e20 0x55c243a196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa936d9d700 1 -- 172.21.15.62:0/3258630072 learned_addr learned my addr 172.21.15.62:0/3258630072 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.035+0000 7fa93659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a188e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52784/0 (socket says 172.21.15.62:52784) 2023-07-14T16:28:05.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa93659c700 1 -- 172.21.15.62:0/3258630072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a18e20 msgr2=0x55c243a196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa93659c700 1 --2- 172.21.15.62:0/3258630072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a18e20 0x55c243a196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa93659c700 1 -- 172.21.15.62:0/3258630072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a19c70 msgr2=0x55c243a1a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa93659c700 1 --2- 172.21.15.62:0/3258630072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a19c70 0x55c243a1a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa93659c700 1 -- 172.21.15.62:0/3258630072 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c243797de0 con 0x55c243a184d0 2023-07-14T16:28:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa936d9d700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9200a1150 0x7fa9200a3600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa930007950 tx=0x7fa9300077e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa93659c700 1 --2- 172.21.15.62:0/3258630072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a188e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9280ca7a0 tx=0x7fa928099c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f81ff0d611866d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa924ff9700 1 -- 172.21.15.62:0/3258630072 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92809a060 con 0x55c243a184d0 2023-07-14T16:28:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa924ff9700 1 -- 172.21.15.62:0/3258630072 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa92809e090 con 0x55c243a184d0 2023-07-14T16:28:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.036+0000 7fa924ff9700 1 -- 172.21.15.62:0/3258630072 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92809a2c0 con 0x55c243a184d0 2023-07-14T16:28:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3258630072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 msgr2=0x55c243a188e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/3258630072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a188e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9280ca7a0 tx=0x7fa928099c20 comp rx=0 tx=0).stop 2023-07-14T16:28:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3258630072 shutdown_connections 2023-07-14T16:28:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/3258630072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a19c70 0x55c243a1a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/3258630072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a188e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/3258630072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a18e20 0x55c243a196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3258630072 >> 172.21.15.62:0/3258630072 conn(0x55c243a0dda0 msgr2=0x55c243a131d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3258630072 shutdown_connections 2023-07-14T16:28:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.037+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3258630072 wait complete. 2023-07-14T16:28:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.038+0000 7fa9503a5b80 1 -- start start 2023-07-14T16:28:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.038+0000 7fa9503a5b80 1 Processor -- start 2023-07-14T16:28:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a26530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a2dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243909ac0 con 0x55c243a184d0 2023-07-14T16:28:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2439097c0 con 0x55c243a18e20 2023-07-14T16:28:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2437ed920 con 0x55c243a19c70 2023-07-14T16:28:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa9503a5b80 10 client.0 _ll_register_callbacks cb 0x55c243907760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa93659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a26530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa93659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a26530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46086/0 (socket says 172.21.15.62:46086) 2023-07-14T16:28:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa93659c700 1 -- 172.21.15.62:0/2611661209 learned_addr learned my addr 172.21.15.62:0/2611661209 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa936d9d700 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a2dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa935d9b700 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa93659c700 1 -- 172.21.15.62:0/2611661209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 msgr2=0x55c243a2dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa93659c700 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a2dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa93659c700 1 -- 172.21.15.62:0/2611661209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 msgr2=0x55c243a26a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa93659c700 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a26a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.039+0000 7fa93659c700 1 -- 172.21.15.62:0/2611661209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93819def0 con 0x55c243a184d0 2023-07-14T16:28:05.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.040+0000 7fa93659c700 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a26530 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa928000c00 tx=0x7fa928004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7080dc827619cad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.040+0000 7fa91dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:05.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.040+0000 7fa91dffb700 1 -- 172.21.15.62:0/2611661209 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280981a0 con 0x55c243a184d0 2023-07-14T16:28:05.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.040+0000 7fa91dffb700 1 -- 172.21.15.62:0/2611661209 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa92809a060 con 0x55c243a184d0 2023-07-14T16:28:05.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.040+0000 7fa9503a5b80 1 -- 172.21.15.62:0/2611661209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2438c34f0 con 0x55c243a184d0 2023-07-14T16:28:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.040+0000 7fa91dffb700 1 -- 172.21.15.62:0/2611661209 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa92800cbd0 con 0x55c243a184d0 2023-07-14T16:28:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.040+0000 7fa9503a5b80 10 client.18957 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.040+0000 7fa9503a5b80 1 -- 172.21.15.62:0/2611661209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c243780c60 con 0x55c243a184d0 2023-07-14T16:28:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa91dffb700 1 -- 172.21.15.62:0/2611661209 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c2438c34f0 con 0x55c243a184d0 2023-07-14T16:28:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa91dffb700 1 -- 172.21.15.62:0/2611661209 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa92809a350 con 0x55c243a184d0 2023-07-14T16:28:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa9503a5b80 10 client.18957 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa9503a5b80 10 client.18957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa9503a5b80 20 client.18957 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c243a31dc0 0x55c243a341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa9503a5b80 4 client.18957 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa9503a5b80 1 -- 172.21.15.62:0/2611661209 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c243962eb0 con 0x55c243a31dc0 2023-07-14T16:28:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.041+0000 7fa935d9b700 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c243a31dc0 0x55c243a341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.042+0000 7fa935d9b700 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c243a31dc0 0x55c243a341a0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc345f35b41ab5f server_cookie=8fddc4cc91d1e7ba in_seq=0 out_seq=0 2023-07-14T16:28:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.042+0000 7fa91dffb700 10 client.18957 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:05.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa91dffb700 1 -- 172.21.15.62:0/2611661209 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c243962eb0 con 0x55c243a31dc0 2023-07-14T16:28:05.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa91dffb700 10 client.18957 handle_command_reply: tid=0 2023-07-14T16:28:05.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 client.18957 shutdown 2023-07-14T16:28:05.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 -- 172.21.15.62:0/2611661209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 msgr2=0x55c243a26530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a26530 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa928000c00 tx=0x7fa928004750 comp rx=0 tx=0).stop 2023-07-14T16:28:05.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 -- 172.21.15.62:0/2611661209 shutdown_connections 2023-07-14T16:28:05.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a2dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a26530 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c243a31dc0 0x55c243a341a0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/2611661209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 -- 172.21.15.62:0/2611661209 >> 172.21.15.62:0/2611661209 conn(0x55c243a0dda0 msgr2=0x55c243a128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa91dffb700 0 client.18957 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:05.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 -- 172.21.15.62:0/2611661209 shutdown_connections 2023-07-14T16:28:05.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.043+0000 7fa9503a5b80 1 -- 172.21.15.62:0/2611661209 wait complete. 2023-07-14T16:28:05.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.044+0000 7fa9503a5b80 20 client.18957 trim_cache size 0 max 16384 2023-07-14T16:28:05.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.064+0000 7fa9503a5b80 1 Processor -- start 2023-07-14T16:28:05.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.064+0000 7fa9503a5b80 1 -- start start 2023-07-14T16:28:05.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.064+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a1cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.064+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a18e20 0x55c243a2da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.064+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a1d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.064+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243a27580 con 0x55c243a18e20 2023-07-14T16:28:05.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.064+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243a140c0 con 0x55c243a184d0 2023-07-14T16:28:05.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.064+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243a215d0 con 0x55c243a19c70 2023-07-14T16:28:05.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a18e20 0x55c243a2da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a18e20 0x55c243a2da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46088/0 (socket says 172.21.15.62:46088) 2023-07-14T16:28:05.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa93659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a1cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 -- 172.21.15.62:0/3816409559 learned_addr learned my addr 172.21.15.62:0/3816409559 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa936d9d700 1 --2- 172.21.15.62:0/3816409559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a1d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 -- 172.21.15.62:0/3816409559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 msgr2=0x55c243a1d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 --2- 172.21.15.62:0/3816409559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a1d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 -- 172.21.15.62:0/3816409559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 msgr2=0x55c243a1cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 --2- 172.21.15.62:0/3816409559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a1cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 -- 172.21.15.62:0/3816409559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2437be250 con 0x55c243a18e20 2023-07-14T16:28:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.065+0000 7fa935d9b700 1 --2- 172.21.15.62:0/3816409559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a18e20 0x55c243a2da70 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa92c005970 tx=0x7fa92c01d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6735bc4655bf3f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.066+0000 7fa9257fa700 1 -- 172.21.15.62:0/3816409559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c021030 con 0x55c243a18e20 2023-07-14T16:28:05.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.066+0000 7fa9257fa700 1 -- 172.21.15.62:0/3816409559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa92c01d5d0 con 0x55c243a18e20 2023-07-14T16:28:05.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.066+0000 7fa9257fa700 1 -- 172.21.15.62:0/3816409559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c018040 con 0x55c243a18e20 2023-07-14T16:28:05.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3816409559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a18e20 msgr2=0x55c243a2da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/3816409559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a18e20 0x55c243a2da70 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa92c005970 tx=0x7fa92c01d950 comp rx=0 tx=0).stop 2023-07-14T16:28:05.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3816409559 shutdown_connections 2023-07-14T16:28:05.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/3816409559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a1d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/3816409559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a184d0 0x55c243a1cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/3816409559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a18e20 0x55c243a2da70 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3816409559 >> 172.21.15.62:0/3816409559 conn(0x55c243a0dda0 msgr2=0x55c243a38b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3816409559 shutdown_connections 2023-07-14T16:28:05.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 -- 172.21.15.62:0/3816409559 wait complete. 2023-07-14T16:28:05.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.067+0000 7fa9503a5b80 1 -- start start 2023-07-14T16:28:05.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa9503a5b80 1 Processor -- start 2023-07-14T16:28:05.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a2d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a2d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa9503a5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a29f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243a2a520 con 0x55c243a184d0 2023-07-14T16:28:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243a2a690 con 0x55c243a18e20 2023-07-14T16:28:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa9503a5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c243a2a850 con 0x55c243a19c70 2023-07-14T16:28:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa9503a5b80 10 client.0 _ll_register_callbacks cb 0x7fa930009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:05.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa93659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a2d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa93659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a2d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46090/0 (socket says 172.21.15.62:46090) 2023-07-14T16:28:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.068+0000 7fa93659c700 1 -- 172.21.15.62:0/358978567 learned_addr learned my addr 172.21.15.62:0/358978567 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa936d9d700 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a29f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa935d9b700 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a2d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa93659c700 1 -- 172.21.15.62:0/358978567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 msgr2=0x55c243a29f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa93659c700 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a29f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa93659c700 1 -- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 msgr2=0x55c243a2d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa93659c700 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a2d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa93659c700 1 -- 172.21.15.62:0/358978567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2437962b0 con 0x55c243a184d0 2023-07-14T16:28:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa935d9b700 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a2d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa93659c700 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a2d430 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa928003840 tx=0x7fa928095d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7648e1697c6df3d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa91dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa91dffb700 1 -- 172.21.15.62:0/358978567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92809dea0 con 0x55c243a184d0 2023-07-14T16:28:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa9503a5b80 1 -- 172.21.15.62:0/358978567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa92809e090 con 0x55c243a184d0 2023-07-14T16:28:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.069+0000 7fa91dffb700 1 -- 172.21.15.62:0/358978567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa92809a060 con 0x55c243a184d0 2023-07-14T16:28:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa91dffb700 1 -- 172.21.15.62:0/358978567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa92800cbd0 con 0x55c243a184d0 2023-07-14T16:28:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa9503a5b80 10 client.18963 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa9503a5b80 1 -- 172.21.15.62:0/358978567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c243864a80 con 0x55c243a184d0 2023-07-14T16:28:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa91dffb700 1 -- 172.21.15.62:0/358978567 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa92809e090 con 0x55c243a184d0 2023-07-14T16:28:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa91dffb700 1 -- 172.21.15.62:0/358978567 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa9280ca490 con 0x55c243a184d0 2023-07-14T16:28:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa9503a5b80 10 client.18963 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa9503a5b80 10 client.18963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:05.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa9503a5b80 20 client.18963 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:05.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c243a86bd0 0x55c243a88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa9503a5b80 4 client.18963 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:28:05.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.070+0000 7fa9503a5b80 1 -- 172.21.15.62:0/358978567 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c243925050 con 0x55c243a86bd0 2023-07-14T16:28:05.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.071+0000 7fa935d9b700 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c243a86bd0 0x55c243a88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.071+0000 7fa91dffb700 10 client.18963 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:05.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.071+0000 7fa935d9b700 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c243a86bd0 0x55c243a88fb0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b2750e437cfa05 server_cookie=109c59414edea420 in_seq=0 out_seq=0 2023-07-14T16:28:05.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa91dffb700 1 -- 172.21.15.62:0/358978567 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c243925050 con 0x55c243a86bd0 2023-07-14T16:28:05.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa91dffb700 10 client.18963 handle_command_reply: tid=0 2023-07-14T16:28:05.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 client.18963 shutdown 2023-07-14T16:28:05.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 -- 172.21.15.62:0/358978567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 msgr2=0x55c243a2d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a2d430 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa928003840 tx=0x7fa928095d00 comp rx=0 tx=0).stop 2023-07-14T16:28:05.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 -- 172.21.15.62:0/358978567 shutdown_connections 2023-07-14T16:28:05.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c243a19c70 0x55c243a29f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c243a184d0 0x55c243a2d430 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c243a86bd0 0x55c243a88fb0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 --2- 172.21.15.62:0/358978567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c243a18e20 0x55c243a2d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 -- 172.21.15.62:0/358978567 >> 172.21.15.62:0/358978567 conn(0x55c243a0dda0 msgr2=0x55c243a1cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa91dffb700 0 client.18963 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:05.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 -- 172.21.15.62:0/358978567 shutdown_connections 2023-07-14T16:28:05.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 1 -- 172.21.15.62:0/358978567 wait complete. 2023-07-14T16:28:05.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.072+0000 7fa9503a5b80 20 client.18963 trim_cache size 0 max 16384 2023-07-14T16:28:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:28:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:28:05.057 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:28:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:28:05.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 -- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9200a1150 msgr2=0x7fa9200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9200a1150 0x7fa9200a3600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa930007950 tx=0x7fa9300077e0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 -- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9380ffaf0 msgr2=0x7fa9381958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9380ffaf0 0x7fa9381958b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa92800e970 tx=0x7fa92800ed30 comp rx=0 tx=0).stop 2023-07-14T16:28:05.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 -- 172.21.15.62:0/460656642 shutdown_connections 2023-07-14T16:28:05.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa938101670 0x7fa93819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9380ffaf0 0x7fa9381958b0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa9200a1150 0x7fa9200a3600 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 --2- 172.21.15.62:0/460656642 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa938100ce0 0x7fa938195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 -- 172.21.15.62:0/460656642 >> 172.21.15.62:0/460656642 conn(0x7fa9380fb130 msgr2=0x7fa9380fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 -- 172.21.15.62:0/460656642 shutdown_connections 2023-07-14T16:28:05.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.073+0000 7fa93c9ff700 1 -- 172.21.15.62:0/460656642 wait complete. 2023-07-14T16:28:05.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:04 smithi062 ceph-mon[95543]: pgmap v730: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:28:05.067 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:28:05.067 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:28:05.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:04 smithi081 ceph-mon[127795]: pgmap v730: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:28:05.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:04 smithi164 ceph-mon[97427]: pgmap v730: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:28:05.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.322+0000 7f2fb519d700 1 -- 172.21.15.62:0/2300606004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01004c0 msgr2=0x7f2fb01028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.322+0000 7f2fb519d700 1 --2- 172.21.15.62:0/2300606004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01004c0 0x7f2fb01028a0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009990 tx=0x7f2fa0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.322+0000 7f2fb519d700 1 -- 172.21.15.62:0/2300606004 shutdown_connections 2023-07-14T16:28:05.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.322+0000 7f2fb519d700 1 --2- 172.21.15.62:0/2300606004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb0105700 0x7f2fb0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.322+0000 7f2fb519d700 1 --2- 172.21.15.62:0/2300606004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01004c0 0x7f2fb01028a0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.322+0000 7f2fb519d700 1 --2- 172.21.15.62:0/2300606004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb0102de0 0x7f2fb01051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.322+0000 7f2fb519d700 1 -- 172.21.15.62:0/2300606004 >> 172.21.15.62:0/2300606004 conn(0x7f2fb00f9ee0 msgr2=0x7f2fb00fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.322+0000 7f2fb519d700 1 -- 172.21.15.62:0/2300606004 shutdown_connections 2023-07-14T16:28:05.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.323+0000 7f2fb519d700 1 -- 172.21.15.62:0/2300606004 wait complete. 2023-07-14T16:28:05.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.324+0000 7f2fb519d700 1 Processor -- start 2023-07-14T16:28:05.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.324+0000 7f2fb519d700 1 -- start start 2023-07-14T16:28:05.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.324+0000 7f2fb519d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb01004c0 0x7f2fb0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.324+0000 7f2fb519d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb0102de0 0x7f2fb0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.324+0000 7f2fb519d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0105700 0x7f2fb019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.324+0000 7f2fb519d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019f1d0 con 0x7f2fb0105700 2023-07-14T16:28:05.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.324+0000 7f2fb519d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019f310 con 0x7f2fb01004c0 2023-07-14T16:28:05.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.324+0000 7f2fb519d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019f4a0 con 0x7f2fb0102de0 2023-07-14T16:28:05.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0105700 0x7f2fb019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb0102de0 0x7f2fb0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0105700 0x7f2fb019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46110/0 (socket says 172.21.15.62:46110) 2023-07-14T16:28:05.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 -- 172.21.15.62:0/1255396100 learned_addr learned my addr 172.21.15.62:0/1255396100 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2faffff700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb01004c0 0x7f2fb0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 -- 172.21.15.62:0/1255396100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb0102de0 msgr2=0x7f2fb0197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb0102de0 0x7f2fb0197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 -- 172.21.15.62:0/1255396100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb01004c0 msgr2=0x7f2fb0197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb01004c0 0x7f2fb0197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 -- 172.21.15.62:0/1255396100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00095f0 con 0x7f2fb0105700 2023-07-14T16:28:05.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.325+0000 7f2fb499c700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0105700 0x7f2fb019ec00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8007ee0 tx=0x7f2fa800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.326+0000 7f2fad7fa700 1 -- 172.21.15.62:0/1255396100 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8017070 con 0x7f2fb0105700 2023-07-14T16:28:05.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.326+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb019f780 con 0x7f2fb0105700 2023-07-14T16:28:05.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.326+0000 7f2fad7fa700 1 -- 172.21.15.62:0/1255396100 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fa800f040 con 0x7f2fb0105700 2023-07-14T16:28:05.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.326+0000 7f2fad7fa700 1 -- 172.21.15.62:0/1255396100 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8005570 con 0x7f2fb0105700 2023-07-14T16:28:05.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.326+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb019fce0 con 0x7f2fb0105700 2023-07-14T16:28:05.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.327+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb0107270 con 0x7f2fb0105700 2023-07-14T16:28:05.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.329+0000 7f2fad7fa700 1 -- 172.21.15.62:0/1255396100 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2fa800db20 con 0x7f2fb0105700 2023-07-14T16:28:05.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.331+0000 7f2fad7fa700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f980a1240 0x7f2f980a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.331+0000 7f2fad7fa700 1 -- 172.21.15.62:0/1255396100 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2fa80cc710 con 0x7f2fb0105700 2023-07-14T16:28:05.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.331+0000 7f2faffff700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f980a1240 0x7f2f980a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.332+0000 7f2faffff700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f980a1240 0x7f2f980a36f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000b5c0 tx=0x7f2fa0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.333+0000 7f2fad7fa700 1 -- 172.21.15.62:0/1255396100 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2fa8095e10 con 0x7f2fb0105700 2023-07-14T16:28:05.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.619+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fb002caa0 con 0x7f2fb0105700 2023-07-14T16:28:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.620+0000 7f2fad7fa700 1 -- 172.21.15.62:0/1255396100 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f2fa8099430 con 0x7f2fb0105700 2023-07-14T16:28:05.596 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:28:05.596 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:28:05.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f980a1240 msgr2=0x7f2f980a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f980a1240 0x7f2f980a36f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000b5c0 tx=0x7f2fa0005fb0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0105700 msgr2=0x7f2fb019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0105700 0x7f2fb019ec00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8007ee0 tx=0x7f2fa800d3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 shutdown_connections 2023-07-14T16:28:05.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0105700 0x7f2fb019ec00 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f980a1240 0x7f2f980a36f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fb01004c0 0x7f2fb0197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 --2- 172.21.15.62:0/1255396100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb0102de0 0x7f2fb0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.623+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 >> 172.21.15.62:0/1255396100 conn(0x7f2fb00f9ee0 msgr2=0x7f2fb00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.624+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 shutdown_connections 2023-07-14T16:28:05.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.624+0000 7f2fb519d700 1 -- 172.21.15.62:0/1255396100 wait complete. 2023-07-14T16:28:05.601 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:28:05.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:28:05.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 -- 172.21.15.62:0/2081560942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8101660 msgr2=0x7fb7e8107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 --2- 172.21.15.62:0/2081560942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8101660 0x7fb7e8107f20 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b0a0 tx=0x7fb7e000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 -- 172.21.15.62:0/2081560942 shutdown_connections 2023-07-14T16:28:05.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 --2- 172.21.15.62:0/2081560942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8101660 0x7fb7e8107f20 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 --2- 172.21.15.62:0/2081560942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7e80ffae0 0x7fb7e80ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 --2- 172.21.15.62:0/2081560942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7e8100cd0 0x7fb7e8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 -- 172.21.15.62:0/2081560942 >> 172.21.15.62:0/2081560942 conn(0x7fb7e80fb0e0 msgr2=0x7fb7e80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 -- 172.21.15.62:0/2081560942 shutdown_connections 2023-07-14T16:28:05.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.881+0000 7fb7edfff700 1 -- 172.21.15.62:0/2081560942 wait complete. 2023-07-14T16:28:05.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.882+0000 7fb7edfff700 1 Processor -- start 2023-07-14T16:28:05.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7edfff700 1 -- start start 2023-07-14T16:28:05.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7edfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7e80ffae0 0x7fb7e8197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7edfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8100cd0 0x7fb7e8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7edfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7e8101660 0x7fb7e819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7edfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e819f690 con 0x7fb7e8100cd0 2023-07-14T16:28:05.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7edfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e819f7d0 con 0x7fb7e80ffae0 2023-07-14T16:28:05.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7edfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e819f960 con 0x7fb7e8101660 2023-07-14T16:28:05.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8100cd0 0x7fb7e8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8100cd0 0x7fb7e8198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46138/0 (socket says 172.21.15.62:46138) 2023-07-14T16:28:05.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/4207499473 learned_addr learned my addr 172.21.15.62:0/4207499473 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.883+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7e80ffae0 0x7fb7e8197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e7fff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7e8101660 0x7fb7e819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/4207499473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7e8101660 msgr2=0x7fb7e819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7e8101660 0x7fb7e819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/4207499473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7e80ffae0 msgr2=0x7fb7e8197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7e80ffae0 0x7fb7e8197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/4207499473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009d00 con 0x7fb7e8100cd0 2023-07-14T16:28:05.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7e80ffae0 0x7fb7e8197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:05.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8100cd0 0x7fb7e8198050 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00d950 tx=0x7fb7dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.884+0000 7fb7e4ff9700 1 -- 172.21.15.62:0/4207499473 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc009940 con 0x7fb7e8100cd0 2023-07-14T16:28:05.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.885+0000 7fb7edfff700 1 -- 172.21.15.62:0/4207499473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e819fc40 con 0x7fb7e8100cd0 2023-07-14T16:28:05.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.885+0000 7fb7e4ff9700 1 -- 172.21.15.62:0/4207499473 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7dc010460 con 0x7fb7e8100cd0 2023-07-14T16:28:05.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.885+0000 7fb7e4ff9700 1 -- 172.21.15.62:0/4207499473 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00f620 con 0x7fb7e8100cd0 2023-07-14T16:28:05.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.885+0000 7fb7edfff700 1 -- 172.21.15.62:0/4207499473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e81a0150 con 0x7fb7e8100cd0 2023-07-14T16:28:05.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.888+0000 7fb7e4ff9700 1 -- 172.21.15.62:0/4207499473 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb7dc0105d0 con 0x7fb7e8100cd0 2023-07-14T16:28:05.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.890+0000 7fb801942b80 1 Processor -- start 2023-07-14T16:28:05.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.890+0000 7fb801942b80 1 -- start start 2023-07-14T16:28:05.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb801942b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fa18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb801942b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa1e20 0x55fc63fa26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb801942b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 0x55fc63fa30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb801942b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63ea6cb0 con 0x55fc63fa14d0 2023-07-14T16:28:05.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb801942b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63ef9a40 con 0x55fc63fa1e20 2023-07-14T16:28:05.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb801942b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63ef2ed0 con 0x55fc63fa2c70 2023-07-14T16:28:05.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb7e4ff9700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7d00a1150 0x7fb7d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb7e7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fa18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb7e4ff9700 1 -- 172.21.15.62:0/4207499473 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7dc0cc730 con 0x7fb7e8100cd0 2023-07-14T16:28:05.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb7e77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa1e20 0x55fc63fa26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb7e7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fa18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46154/0 (socket says 172.21.15.62:46154) 2023-07-14T16:28:05.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb7e7fff700 1 -- 172.21.15.62:0/2802584784 learned_addr learned my addr 172.21.15.62:0/2802584784 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/2802584784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 0x55fc63fa30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.891+0000 7fb7e7fff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7d00a1150 0x7fb7d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7e77fe700 1 -- 172.21.15.62:0/2802584784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 msgr2=0x55fc63fa30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/2802584784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 0x55fc63fa30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7e77fe700 1 -- 172.21.15.62:0/2802584784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 msgr2=0x55fc63fa18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/2802584784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fa18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7e77fe700 1 -- 172.21.15.62:0/2802584784 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc63d20de0 con 0x55fc63fa1e20 2023-07-14T16:28:05.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/2802584784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa1e20 0x55fc63fa26e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80114c0 tx=0x7fb7d80117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d86369796bca469 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7cdffb700 1 -- 172.21.15.62:0/2802584784 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8004310 con 0x55fc63fa1e20 2023-07-14T16:28:05.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7cdffb700 1 -- 172.21.15.62:0/2802584784 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7d8004470 con 0x55fc63fa1e20 2023-07-14T16:28:05.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7e7fff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7d00a1150 0x7fb7d00a3600 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b0a0 tx=0x7fb7e00099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.892+0000 7fb7cdffb700 1 -- 172.21.15.62:0/2802584784 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8005660 con 0x55fc63fa1e20 2023-07-14T16:28:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 -- 172.21.15.62:0/2802584784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa1e20 msgr2=0x55fc63fa26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 --2- 172.21.15.62:0/2802584784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa1e20 0x55fc63fa26e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80114c0 tx=0x7fb7d80117d0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 -- 172.21.15.62:0/2802584784 shutdown_connections 2023-07-14T16:28:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 --2- 172.21.15.62:0/2802584784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 0x55fc63fa30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 --2- 172.21.15.62:0/2802584784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fa18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 --2- 172.21.15.62:0/2802584784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa1e20 0x55fc63fa26e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 -- 172.21.15.62:0/2802584784 >> 172.21.15.62:0/2802584784 conn(0x55fc63f96da0 msgr2=0x55fc63f9c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 -- 172.21.15.62:0/2802584784 shutdown_connections 2023-07-14T16:28:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.893+0000 7fb801942b80 1 -- 172.21.15.62:0/2802584784 wait complete. 2023-07-14T16:28:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.894+0000 7fb801942b80 1 -- start start 2023-07-14T16:28:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.894+0000 7fb801942b80 1 Processor -- start 2023-07-14T16:28:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb801942b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb801942b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63fab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb801942b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa2c70 0x55fc63fb2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb801942b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63e92ac0 con 0x55fc63fa2c70 2023-07-14T16:28:05.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb801942b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63e927c0 con 0x55fc63fa14d0 2023-07-14T16:28:05.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb801942b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63d76920 con 0x55fc63fa1e20 2023-07-14T16:28:05.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb801942b80 10 client.0 _ll_register_callbacks cb 0x55fc63e90760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:05.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb7e7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa2c70 0x55fc63fb2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb7e6ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63fab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb7e7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa2c70 0x55fc63fb2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46166/0 (socket says 172.21.15.62:46166) 2023-07-14T16:28:05.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb7e7fff700 1 -- 172.21.15.62:0/3063174203 learned_addr learned my addr 172.21.15.62:0/3063174203 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.895+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 msgr2=0x55fc63fab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/3063174203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa2c70 msgr2=0x55fc63fb2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa2c70 0x55fc63fb2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/3063174203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d8009c90 con 0x55fc63fa1e20 2023-07-14T16:28:05.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:05.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63fab630 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0a1a10 tx=0x7fb7dc099450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d50db75d9ad31d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:28:05.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:05 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1255396100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:28:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/3063174203 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc099af0 con 0x55fc63fa1e20 2023-07-14T16:28:05.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/3063174203 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7dc09db20 con 0x55fc63fa1e20 2023-07-14T16:28:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.896+0000 7fb801942b80 1 -- 172.21.15.62:0/3063174203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc63e4c4f0 con 0x55fc63fa1e20 2023-07-14T16:28:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/3063174203 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7dc0a1f40 con 0x55fc63fa1e20 2023-07-14T16:28:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb801942b80 10 client.28453 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:05.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb801942b80 1 -- 172.21.15.62:0/3063174203 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fc63d09c60 con 0x55fc63fa1e20 2023-07-14T16:28:05.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/3063174203 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fc63e4c4f0 con 0x55fc63fa1e20 2023-07-14T16:28:05.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/3063174203 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7dc0a1ab0 con 0x55fc63fa1e20 2023-07-14T16:28:05.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb801942b80 10 client.28453 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:05.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb801942b80 10 client.28453 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:28:05.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb801942b80 20 client.28453 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:05.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb801942b80 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55fc63fb6cb0 0x55fc63fb90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb801942b80 4 client.28453 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:05.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.897+0000 7fb801942b80 1 -- 172.21.15.62:0/3063174203 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc63eebeb0 con 0x55fc63fb6cb0 2023-07-14T16:28:05.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.898+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55fc63fb6cb0 0x55fc63fb90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.898+0000 7fb7c6ffd700 10 client.28453 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:28:05.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.898+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55fc63fb6cb0 0x55fc63fb90c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5c1d19b384cf7ac server_cookie=97b09162b37216c2 in_seq=0 out_seq=0 2023-07-14T16:28:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.899+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/3063174203 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fc63eebeb0 con 0x55fc63fb6cb0 2023-07-14T16:28:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.899+0000 7fb7c6ffd700 10 client.28453 handle_command_reply: tid=0 2023-07-14T16:28:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.899+0000 7fb801942b80 1 client.28453 shutdown 2023-07-14T16:28:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.899+0000 7fb801942b80 1 -- 172.21.15.62:0/3063174203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 msgr2=0x55fc63fab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.899+0000 7fb801942b80 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63fab630 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0a1a10 tx=0x7fb7dc099450 comp rx=0 tx=0).stop 2023-07-14T16:28:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 1 -- 172.21.15.62:0/3063174203 shutdown_connections 2023-07-14T16:28:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55fc63fb6cb0 0x55fc63fb90c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa2c70 0x55fc63fb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb7c6ffd700 0 client.28453 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:28:05.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 1 --2- 172.21.15.62:0/3063174203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63fab630 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 1 -- 172.21.15.62:0/3063174203 >> 172.21.15.62:0/3063174203 conn(0x55fc63f96da0 msgr2=0x55fc63f9b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 1 -- 172.21.15.62:0/3063174203 shutdown_connections 2023-07-14T16:28:05.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 1 -- 172.21.15.62:0/3063174203 wait complete. 2023-07-14T16:28:05.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.900+0000 7fb801942b80 20 client.28453 trim_cache size 0 max 16384 2023-07-14T16:28:05.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.918+0000 7fb801942b80 1 Processor -- start 2023-07-14T16:28:05.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.918+0000 7fb801942b80 1 -- start start 2023-07-14T16:28:05.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.918+0000 7fb801942b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fbd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.918+0000 7fb801942b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa1e20 0x55fc63fb2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.918+0000 7fb801942b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 0x55fc63f9b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.918+0000 7fb801942b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63f9ced0 con 0x55fc63fa1e20 2023-07-14T16:28:05.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.918+0000 7fb801942b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63f96970 con 0x55fc63fa14d0 2023-07-14T16:28:05.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.918+0000 7fb801942b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63fac270 con 0x55fc63fa2c70 2023-07-14T16:28:05.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa1e20 0x55fc63fb2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa1e20 0x55fc63fb2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46172/0 (socket says 172.21.15.62:46172) 2023-07-14T16:28:05.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/565323541 learned_addr learned my addr 172.21.15.62:0/565323541 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fbd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e7fff700 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 0x55fc63f9b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/565323541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 msgr2=0x55fc63f9b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 0x55fc63f9b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/565323541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 msgr2=0x55fc63fbd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fbd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e6ffd700 1 -- 172.21.15.62:0/565323541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc63d47250 con 0x55fc63fa1e20 2023-07-14T16:28:05.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.919+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fbd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:05.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.920+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa1e20 0x55fc63fb2630 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc097e40 tx=0x7fb7dc0cc970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d832726a52a4bf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.920+0000 7fb7ce7fc700 1 -- 172.21.15.62:0/565323541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc014070 con 0x55fc63fa1e20 2023-07-14T16:28:05.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.920+0000 7fb7ce7fc700 1 -- 172.21.15.62:0/565323541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7dc09db20 con 0x55fc63fa1e20 2023-07-14T16:28:05.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.920+0000 7fb7ce7fc700 1 -- 172.21.15.62:0/565323541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc098b20 con 0x55fc63fa1e20 2023-07-14T16:28:05.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 -- 172.21.15.62:0/565323541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa1e20 msgr2=0x55fc63fb2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa1e20 0x55fc63fb2630 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc097e40 tx=0x7fb7dc0cc970 comp rx=0 tx=0).stop 2023-07-14T16:28:05.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 -- 172.21.15.62:0/565323541 shutdown_connections 2023-07-14T16:28:05.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa2c70 0x55fc63f9b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa14d0 0x55fc63fbd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 --2- 172.21.15.62:0/565323541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa1e20 0x55fc63fb2630 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 -- 172.21.15.62:0/565323541 >> 172.21.15.62:0/565323541 conn(0x55fc63f96da0 msgr2=0x55fc63f9c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 -- 172.21.15.62:0/565323541 shutdown_connections 2023-07-14T16:28:05.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 -- 172.21.15.62:0/565323541 wait complete. 2023-07-14T16:28:05.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.921+0000 7fb801942b80 1 -- start start 2023-07-14T16:28:05.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb801942b80 1 Processor -- start 2023-07-14T16:28:05.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb801942b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb801942b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63faeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb801942b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa2c70 0x55fc63fb1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb801942b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63fb2020 con 0x55fc63fa14d0 2023-07-14T16:28:05.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb801942b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63fb2190 con 0x55fc63fa2c70 2023-07-14T16:28:05.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb801942b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc63fb2350 con 0x55fc63fa1e20 2023-07-14T16:28:05.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb801942b80 10 client.0 _ll_register_callbacks cb 0x55fc63e90760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:05.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb7e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb7e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46182/0 (socket says 172.21.15.62:46182) 2023-07-14T16:28:05.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.922+0000 7fb7e77fe700 1 -- 172.21.15.62:0/1687253973 learned_addr learned my addr 172.21.15.62:0/1687253973 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:05.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63faeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e7fff700 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa2c70 0x55fc63fb1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e77fe700 1 -- 172.21.15.62:0/1687253973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 msgr2=0x55fc63faeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63faeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e77fe700 1 -- 172.21.15.62:0/1687253973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa2c70 msgr2=0x55fc63fb1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:05.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa2c70 0x55fc63fb1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e77fe700 1 -- 172.21.15.62:0/1687253973 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d801d040 con 0x55fc63fa14d0 2023-07-14T16:28:05.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63faeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:05.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7e77fe700 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fae580 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8013010 tx=0x7fb7d80041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffa8da774ed1f27 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:05.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:05.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.923+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/1687253973 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80055a0 con 0x55fc63fa14d0 2023-07-14T16:28:05.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.924+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/1687253973 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb7d8028b00 con 0x55fc63fa14d0 2023-07-14T16:28:05.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.924+0000 7fb801942b80 1 -- 172.21.15.62:0/1687253973 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7d8004470 con 0x55fc63fa14d0 2023-07-14T16:28:05.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.924+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/1687253973 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d8036030 con 0x55fc63fa14d0 2023-07-14T16:28:05.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.924+0000 7fb801942b80 10 client.18987 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:05.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.924+0000 7fb801942b80 1 -- 172.21.15.62:0/1687253973 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fc63d09c60 con 0x55fc63fa14d0 2023-07-14T16:28:05.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.924+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/1687253973 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7d8004470 con 0x55fc63fa14d0 2023-07-14T16:28:05.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.924+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/1687253973 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7d802e020 con 0x55fc63fa14d0 2023-07-14T16:28:05.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.924+0000 7fb801942b80 10 client.18987 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:05.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.925+0000 7fb801942b80 10 client.18987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:28:05.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.925+0000 7fb801942b80 20 client.18987 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:05.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.925+0000 7fb801942b80 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55fc6400bc40 0x55fc6400e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:05.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.925+0000 7fb801942b80 4 client.18987 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:28:05.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.925+0000 7fb801942b80 1 -- 172.21.15.62:0/1687253973 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc63eae050 con 0x55fc6400bc40 2023-07-14T16:28:05.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.925+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55fc6400bc40 0x55fc6400e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:05.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb7e6ffd700 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55fc6400bc40 0x55fc6400e050 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=803dcb33b0da6679 server_cookie=493fb63446a898c3 in_seq=0 out_seq=0 2023-07-14T16:28:05.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb7c6ffd700 10 client.18987 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:28:05.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb7c6ffd700 1 -- 172.21.15.62:0/1687253973 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc63eae050 con 0x55fc6400bc40 2023-07-14T16:28:05.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb7c6ffd700 10 client.18987 handle_command_reply: tid=0 2023-07-14T16:28:05.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 client.18987 shutdown 2023-07-14T16:28:05.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 -- 172.21.15.62:0/1687253973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 msgr2=0x55fc63fae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fae580 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8013010 tx=0x7fb7d80041c0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 -- 172.21.15.62:0/1687253973 shutdown_connections 2023-07-14T16:28:05.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55fc6400bc40 0x55fc6400e050 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fc63fa2c70 0x55fc63fb1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fc63fa14d0 0x55fc63fae580 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb7c6ffd700 0 client.18987 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:28:05.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 --2- 172.21.15.62:0/1687253973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fc63fa1e20 0x55fc63faeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 -- 172.21.15.62:0/1687253973 >> 172.21.15.62:0/1687253973 conn(0x55fc63f96da0 msgr2=0x55fc63fa8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 -- 172.21.15.62:0/1687253973 shutdown_connections 2023-07-14T16:28:05.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.926+0000 7fb801942b80 1 -- 172.21.15.62:0/1687253973 wait complete. 2023-07-14T16:28:05.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb801942b80 20 client.18987 trim_cache size 0 max 16384 2023-07-14T16:28:05.923 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:28:05.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 -- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7d00a1150 msgr2=0x7fb7d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7d00a1150 0x7fb7d00a3600 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b0a0 tx=0x7fb7e00099e0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 -- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8100cd0 msgr2=0x7fb7e8198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8100cd0 0x7fb7e8198050 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00d950 tx=0x7fb7dc00dc60 comp rx=0 tx=0).stop 2023-07-14T16:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 -- 172.21.15.62:0/4207499473 shutdown_connections 2023-07-14T16:28:05.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7e8101660 0x7fb7e819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb7d00a1150 0x7fb7d00a3600 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7e80ffae0 0x7fb7e8197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 --2- 172.21.15.62:0/4207499473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e8100cd0 0x7fb7e8198050 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:05.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 -- 172.21.15.62:0/4207499473 >> 172.21.15.62:0/4207499473 conn(0x7fb7e80fb0e0 msgr2=0x7fb7e80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:05.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 -- 172.21.15.62:0/4207499473 shutdown_connections 2023-07-14T16:28:05.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:05.927+0000 7fb7edfff700 1 -- 172.21.15.62:0/4207499473 wait complete. 2023-07-14T16:28:05.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:28:06.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:05 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1255396100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:28:06.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 -- 172.21.15.62:0/3346976686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 msgr2=0x7fe3e4102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 --2- 172.21.15.62:0/3346976686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 0x7fe3e4102a00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009990 tx=0x7fe3d8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 -- 172.21.15.62:0/3346976686 shutdown_connections 2023-07-14T16:28:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 --2- 172.21.15.62:0/3346976686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4105860 0x7fe3e4107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 --2- 172.21.15.62:0/3346976686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 0x7fe3e4102a00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 --2- 172.21.15.62:0/3346976686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3e4102f40 0x7fe3e4105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 -- 172.21.15.62:0/3346976686 >> 172.21.15.62:0/3346976686 conn(0x7fe3e40fa000 msgr2=0x7fe3e40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 -- 172.21.15.62:0/3346976686 shutdown_connections 2023-07-14T16:28:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.178+0000 7fe3eabff700 1 -- 172.21.15.62:0/3346976686 wait complete. 2023-07-14T16:28:06.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3eabff700 1 Processor -- start 2023-07-14T16:28:06.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3eabff700 1 -- start start 2023-07-14T16:28:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3eabff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 0x7fe3e41957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3eabff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3e4102f40 0x7fe3e4195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3eabff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4105860 0x7fe3e419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3eabff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419d320 con 0x7fe3e4100620 2023-07-14T16:28:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3eabff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419d460 con 0x7fe3e4102f40 2023-07-14T16:28:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3eabff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419d5f0 con 0x7fe3e4105860 2023-07-14T16:28:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.180+0000 7fe3e899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 0x7fe3e41957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 0x7fe3e41957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46196/0 (socket says 172.21.15.62:46196) 2023-07-14T16:28:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e899b700 1 -- 172.21.15.62:0/1766772901 learned_addr learned my addr 172.21.15.62:0/1766772901 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e3fff700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3e4102f40 0x7fe3e4195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e919c700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4105860 0x7fe3e419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e899b700 1 -- 172.21.15.62:0/1766772901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4105860 msgr2=0x7fe3e419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e899b700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4105860 0x7fe3e419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e899b700 1 -- 172.21.15.62:0/1766772901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3e4102f40 msgr2=0x7fe3e4195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e899b700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3e4102f40 0x7fe3e4195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e899b700 1 -- 172.21.15.62:0/1766772901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d80095f0 con 0x7fe3e4100620 2023-07-14T16:28:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e919c700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4105860 0x7fe3e419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.181+0000 7fe3e899b700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 0x7fe3e41957a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8005910 tx=0x7fe3d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.182+0000 7fe3e1ffb700 1 -- 172.21.15.62:0/1766772901 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d801d070 con 0x7fe3e4100620 2023-07-14T16:28:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.182+0000 7fe3e1ffb700 1 -- 172.21.15.62:0/1766772901 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d80054c0 con 0x7fe3e4100620 2023-07-14T16:28:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.182+0000 7fe3eabff700 1 -- 172.21.15.62:0/1766772901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e419d870 con 0x7fe3e4100620 2023-07-14T16:28:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.182+0000 7fe3e1ffb700 1 -- 172.21.15.62:0/1766772901 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8003ab0 con 0x7fe3e4100620 2023-07-14T16:28:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.182+0000 7fe3eabff700 1 -- 172.21.15.62:0/1766772901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e419dcf0 con 0x7fe3e4100620 2023-07-14T16:28:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.185+0000 7fe3e1ffb700 1 -- 172.21.15.62:0/1766772901 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe3d80047a0 con 0x7fe3e4100620 2023-07-14T16:28:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3fe542b80 1 Processor -- start 2023-07-14T16:28:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3e1ffb700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe3d40a11d0 0x7fe3d40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3fe542b80 1 -- start start 2023-07-14T16:28:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3e1ffb700 1 -- 172.21.15.62:0/1766772901 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe3d80cd8a0 con 0x7fe3e4100620 2023-07-14T16:28:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3e919c700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe3d40a11d0 0x7fe3d40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d3ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40c3fcb0 con 0x55de40d40080 2023-07-14T16:28:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40c92a40 con 0x55de40d3e8e0 2023-07-14T16:28:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.187+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40c8bed0 con 0x55de40d3f230 2023-07-14T16:28:06.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e899b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d3ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e899b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d3ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52928/0 (socket says 172.21.15.62:52928) 2023-07-14T16:28:06.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e899b700 1 -- 172.21.15.62:0/3393269782 learned_addr learned my addr 172.21.15.62:0/3393269782 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d3faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55430/0 (socket says 172.21.15.62:55430) 2023-07-14T16:28:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e919c700 1 --2- 172.21.15.62:0/3393269782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e899b700 1 -- 172.21.15.62:0/3393269782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 msgr2=0x55de40d3faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e899b700 1 --2- 172.21.15.62:0/3393269782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d3faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e899b700 1 -- 172.21.15.62:0/3393269782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 msgr2=0x55de40d404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e899b700 1 --2- 172.21.15.62:0/3393269782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.188+0000 7fe3e899b700 1 -- 172.21.15.62:0/3393269782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de40ab9de0 con 0x55de40d3e8e0 2023-07-14T16:28:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.189+0000 7fe3e919c700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe3d40a11d0 0x7fe3d40a3680 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc007610 tx=0x7fe3dc0079d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.189+0000 7fe3e899b700 1 --2- 172.21.15.62:0/3393269782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d3ecf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80a2be0 tx=0x7fe3d8005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04159414393c7f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.189+0000 7fe3ceffd700 1 -- 172.21.15.62:0/3393269782 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80a3110 con 0x55de40d3e8e0 2023-07-14T16:28:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.189+0000 7fe3ceffd700 1 -- 172.21.15.62:0/3393269782 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d80990f0 con 0x55de40d3e8e0 2023-07-14T16:28:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.189+0000 7fe3ceffd700 1 -- 172.21.15.62:0/3393269782 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80a2c80 con 0x55de40d3e8e0 2023-07-14T16:28:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3393269782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 msgr2=0x55de40d3ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3393269782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d3ecf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80a2be0 tx=0x7fe3d8005ac0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3393269782 shutdown_connections 2023-07-14T16:28:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3393269782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3393269782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d3ecf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3393269782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d3faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3393269782 >> 172.21.15.62:0/3393269782 conn(0x55de40d2fda0 msgr2=0x55de40d351d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3393269782 shutdown_connections 2023-07-14T16:28:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.190+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3393269782 wait complete. 2023-07-14T16:28:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 1 -- start start 2023-07-14T16:28:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 1 Processor -- start 2023-07-14T16:28:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3e8e0 0x55de40d48170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3f230 0x55de40d486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d4f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40c2bac0 con 0x55de40d40080 2023-07-14T16:28:06.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40c2b7c0 con 0x55de40d3f230 2023-07-14T16:28:06.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40b0f920 con 0x55de40d3e8e0 2023-07-14T16:28:06.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.191+0000 7fe3fe542b80 10 client.0 _ll_register_callbacks cb 0x55de40c29760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:06.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e899b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3e8e0 0x55de40d48170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e899b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3e8e0 0x55de40d48170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55438/0 (socket says 172.21.15.62:55438) 2023-07-14T16:28:06.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e899b700 1 -- 172.21.15.62:0/3411707980 learned_addr learned my addr 172.21.15.62:0/3411707980 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e919c700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d4f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e3fff700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3f230 0x55de40d486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e899b700 1 -- 172.21.15.62:0/3411707980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3f230 msgr2=0x55de40d486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e899b700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3f230 0x55de40d486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e899b700 1 -- 172.21.15.62:0/3411707980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 msgr2=0x55de40d4f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e899b700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d4f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e899b700 1 -- 172.21.15.62:0/3411707980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e419dcf0 con 0x55de40d3e8e0 2023-07-14T16:28:06.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e3fff700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3f230 0x55de40d486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:06.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.192+0000 7fe3e919c700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d4f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:06.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3e899b700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3e8e0 0x55de40d48170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809aa00 tx=0x7fe3d80098f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d515f4edd71f1b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:28:06.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3c3fff700 1 -- 172.21.15.62:0/3411707980 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80b2070 con 0x55de40d3e8e0 2023-07-14T16:28:06.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3411707980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de40be54f0 con 0x55de40d3e8e0 2023-07-14T16:28:06.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3c3fff700 1 -- 172.21.15.62:0/3411707980 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe3d8098e90 con 0x55de40d3e8e0 2023-07-14T16:28:06.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3c3fff700 1 -- 172.21.15.62:0/3411707980 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3d80b3260 con 0x55de40d3e8e0 2023-07-14T16:28:06.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3fe542b80 10 client.28468 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:06.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3411707980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de40aa2c60 con 0x55de40d3e8e0 2023-07-14T16:28:06.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.193+0000 7fe3c3fff700 1 -- 172.21.15.62:0/3411707980 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d80c9080 con 0x55de40d3e8e0 2023-07-14T16:28:06.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.194+0000 7fe3c3fff700 1 -- 172.21.15.62:0/3411707980 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3d80caf80 con 0x55de40d3e8e0 2023-07-14T16:28:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.194+0000 7fe3fe542b80 10 client.28468 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.194+0000 7fe3fe542b80 10 client.28468 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:28:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.194+0000 7fe3fe542b80 20 client.28468 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.194+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de40d539b0 0x55de40d55d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.194+0000 7fe3fe542b80 4 client.28468 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.194+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3411707980 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de40c84eb0 con 0x55de40d539b0 2023-07-14T16:28:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.194+0000 7fe3e3fff700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de40d539b0 0x55de40d55d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.195+0000 7fe3e3fff700 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de40d539b0 0x55de40d55d90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33ce74d61f78004c server_cookie=e335ad32252861a6 in_seq=0 out_seq=0 2023-07-14T16:28:06.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.195+0000 7fe3c3fff700 10 client.28468 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:28:06.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3c3fff700 1 -- 172.21.15.62:0/3411707980 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de40c84eb0 con 0x55de40d539b0 2023-07-14T16:28:06.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3c3fff700 10 client.28468 handle_command_reply: tid=0 2023-07-14T16:28:06.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 client.28468 shutdown 2023-07-14T16:28:06.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3411707980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3e8e0 msgr2=0x55de40d48170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3e8e0 0x55de40d48170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809aa00 tx=0x7fe3d80098f0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3411707980 shutdown_connections 2023-07-14T16:28:06.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de40d539b0 0x55de40d55d90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d40080 0x55de40d4f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3e8e0 0x55de40d48170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/3411707980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3f230 0x55de40d486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3411707980 >> 172.21.15.62:0/3411707980 conn(0x55de40d2fda0 msgr2=0x55de40d348b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3c3fff700 0 client.28468 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:28:06.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3411707980 shutdown_connections 2023-07-14T16:28:06.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.196+0000 7fe3fe542b80 1 -- 172.21.15.62:0/3411707980 wait complete. 2023-07-14T16:28:06.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.197+0000 7fe3fe542b80 20 client.28468 trim_cache size 0 max 16384 2023-07-14T16:28:06.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3fe542b80 1 Processor -- start 2023-07-14T16:28:06.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3fe542b80 1 -- start start 2023-07-14T16:28:06.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3e8e0 0x55de40d4ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d4f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d40080 0x55de40d42ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40d47620 con 0x55de40d3e8e0 2023-07-14T16:28:06.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40d537a0 con 0x55de40d40080 2023-07-14T16:28:06.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40d362e0 con 0x55de40d3f230 2023-07-14T16:28:06.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.214+0000 7fe3e899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3e8e0 0x55de40d4ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d4f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e919c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d40080 0x55de40d42ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e899b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3e8e0 0x55de40d4ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46216/0 (socket says 172.21.15.62:46216) 2023-07-14T16:28:06.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e899b700 1 -- 172.21.15.62:0/2521207498 learned_addr learned my addr 172.21.15.62:0/2521207498 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e899b700 1 -- 172.21.15.62:0/2521207498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 msgr2=0x55de40d4f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e899b700 1 --2- 172.21.15.62:0/2521207498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d4f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e899b700 1 -- 172.21.15.62:0/2521207498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d40080 msgr2=0x55de40d42ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e899b700 1 --2- 172.21.15.62:0/2521207498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d40080 0x55de40d42ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e899b700 1 -- 172.21.15.62:0/2521207498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de40ae0250 con 0x55de40d3e8e0 2023-07-14T16:28:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e919c700 1 --2- 172.21.15.62:0/2521207498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d40080 0x55de40d42ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.215+0000 7fe3e899b700 1 --2- 172.21.15.62:0/2521207498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3e8e0 0x55de40d4ffe0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80cb720 tx=0x7fe3d8005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8ee2e2ff697637 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3cf7fe700 1 -- 172.21.15.62:0/2521207498 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80b2030 con 0x55de40d3e8e0 2023-07-14T16:28:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3cf7fe700 1 -- 172.21.15.62:0/2521207498 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de40be54f0 con 0x55de40d3e8e0 2023-07-14T16:28:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3cf7fe700 1 -- 172.21.15.62:0/2521207498 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8096b40 con 0x55de40d3e8e0 2023-07-14T16:28:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2521207498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3e8e0 msgr2=0x55de40d4ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2521207498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3e8e0 0x55de40d4ffe0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80cb720 tx=0x7fe3d8005ac0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2521207498 shutdown_connections 2023-07-14T16:28:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2521207498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d40080 0x55de40d42ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2521207498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3e8e0 0x55de40d4ffe0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2521207498 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d3f230 0x55de40d4f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.216+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2521207498 >> 172.21.15.62:0/2521207498 conn(0x55de40d2fda0 msgr2=0x55de40d5a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.217+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2521207498 shutdown_connections 2023-07-14T16:28:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.217+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2521207498 wait complete. 2023-07-14T16:28:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.217+0000 7fe3fe542b80 1 -- start start 2023-07-14T16:28:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.217+0000 7fe3fe542b80 1 Processor -- start 2023-07-14T16:28:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d4b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3f230 0x55de40d4b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3fe542b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d40080 0x55de40d4e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40d4eeb0 con 0x55de40d3f230 2023-07-14T16:28:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40d4f020 con 0x55de40d3e8e0 2023-07-14T16:28:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3fe542b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40d4f1e0 con 0x55de40d40080 2023-07-14T16:28:06.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3fe542b80 10 client.0 _ll_register_callbacks cb 0x7fe3d00077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:06.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3f230 0x55de40d4b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3f230 0x55de40d4b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46230/0 (socket says 172.21.15.62:46230) 2023-07-14T16:28:06.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e3fff700 1 -- 172.21.15.62:0/2980236876 learned_addr learned my addr 172.21.15.62:0/2980236876 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e899b700 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d4b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e919c700 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d40080 0x55de40d4e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e3fff700 1 -- 172.21.15.62:0/2980236876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d40080 msgr2=0x55de40d4e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e3fff700 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d40080 0x55de40d4e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e3fff700 1 -- 172.21.15.62:0/2980236876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 msgr2=0x55de40d4b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e3fff700 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d4b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.218+0000 7fe3e3fff700 1 -- 172.21.15.62:0/2980236876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc01c810 con 0x55de40d3f230 2023-07-14T16:28:06.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3e3fff700 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3f230 0x55de40d4b950 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x55de40d56630 tx=0x7fe3d001fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4d35a07ac3019a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:06.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3c3fff700 1 -- 172.21.15.62:0/2980236876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0016070 con 0x55de40d3f230 2023-07-14T16:28:06.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2980236876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3d80990f0 con 0x55de40d3f230 2023-07-14T16:28:06.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3c3fff700 1 -- 172.21.15.62:0/2980236876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe3d001f510 con 0x55de40d3f230 2023-07-14T16:28:06.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3c3fff700 1 -- 172.21.15.62:0/2980236876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3d0004050 con 0x55de40d3f230 2023-07-14T16:28:06.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3fe542b80 10 client.19005 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:06.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2980236876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de40aa2c60 con 0x55de40d3f230 2023-07-14T16:28:06.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.219+0000 7fe3c3fff700 1 -- 172.21.15.62:0/2980236876 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d80990f0 con 0x55de40d3f230 2023-07-14T16:28:06.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.220+0000 7fe3c3fff700 1 -- 172.21.15.62:0/2980236876 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3d00036f0 con 0x55de40d3f230 2023-07-14T16:28:06.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.220+0000 7fe3fe542b80 10 client.19005 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:06.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.220+0000 7fe3fe542b80 10 client.19005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:28:06.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.220+0000 7fe3fe542b80 20 client.19005 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.220+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de40da8840 0x55de40daac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.220+0000 7fe3fe542b80 4 client.19005 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:28:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.220+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2980236876 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de40c47050 con 0x55de40da8840 2023-07-14T16:28:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.220+0000 7fe3e899b700 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de40da8840 0x55de40daac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3e899b700 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de40da8840 0x55de40daac50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bffec4285e083f56 server_cookie=f73be97e1fd0c828 in_seq=0 out_seq=0 2023-07-14T16:28:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3c3fff700 10 client.19005 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:28:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3c3fff700 1 -- 172.21.15.62:0/2980236876 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de40c47050 con 0x55de40da8840 2023-07-14T16:28:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3c3fff700 10 client.19005 handle_command_reply: tid=0 2023-07-14T16:28:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 client.19005 shutdown 2023-07-14T16:28:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2980236876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3f230 msgr2=0x55de40d4b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3f230 0x55de40d4b950 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x55de40d56630 tx=0x7fe3d001fef0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2980236876 shutdown_connections 2023-07-14T16:28:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55de40da8840 0x55de40daac50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55de40d40080 0x55de40d4e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55de40d3e8e0 0x55de40d4b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 --2- 172.21.15.62:0/2980236876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55de40d3f230 0x55de40d4b950 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2980236876 >> 172.21.15.62:0/2980236876 conn(0x55de40d2fda0 msgr2=0x55de40d4ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3c3fff700 0 client.19005 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:28:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.221+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2980236876 shutdown_connections 2023-07-14T16:28:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3fe542b80 1 -- 172.21.15.62:0/2980236876 wait complete. 2023-07-14T16:28:06.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3fe542b80 20 client.19005 trim_cache size 0 max 16384 2023-07-14T16:28:06.207 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:28:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 -- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe3d40a11d0 msgr2=0x7fe3d40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe3d40a11d0 0x7fe3d40a3680 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc007610 tx=0x7fe3dc0079d0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 -- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 msgr2=0x7fe3e41957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 0x7fe3e41957a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8005910 tx=0x7fe3d8004730 comp rx=0 tx=0).stop 2023-07-14T16:28:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 -- 172.21.15.62:0/1766772901 shutdown_connections 2023-07-14T16:28:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e4105860 0x7fe3e419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e4100620 0x7fe3e41957a0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe3d40a11d0 0x7fe3d40a3680 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 --2- 172.21.15.62:0/1766772901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3e4102f40 0x7fe3e4195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.222+0000 7fe3eabff700 1 -- 172.21.15.62:0/1766772901 >> 172.21.15.62:0/1766772901 conn(0x7fe3e40fa000 msgr2=0x7fe3e4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.223+0000 7fe3eabff700 1 -- 172.21.15.62:0/1766772901 shutdown_connections 2023-07-14T16:28:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.223+0000 7fe3eabff700 1 -- 172.21.15.62:0/1766772901 wait complete. 2023-07-14T16:28:06.216 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:28:06.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:05 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1255396100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:28:06.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.486+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1753132715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 msgr2=0x7fd0f4102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.486+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1753132715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 0x7fd0f4102890 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009990 tx=0x7fd0e4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.487+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1753132715 shutdown_connections 2023-07-14T16:28:06.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.487+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1753132715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f41056f0 0x7fd0f4107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.487+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1753132715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 0x7fd0f4102890 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.487+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1753132715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f4102dd0 0x7fd0f41051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.487+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1753132715 >> 172.21.15.62:0/1753132715 conn(0x7fd0f40f9ed0 msgr2=0x7fd0f40fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.487+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1753132715 shutdown_connections 2023-07-14T16:28:06.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.487+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1753132715 wait complete. 2023-07-14T16:28:06.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.488+0000 7fd0f3fff700 1 Processor -- start 2023-07-14T16:28:06.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.488+0000 7fd0f3fff700 1 -- start start 2023-07-14T16:28:06.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.488+0000 7fd0f3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 0x7fd0f41976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.488+0000 7fd0f3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f4102dd0 0x7fd0f4197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.488+0000 7fd0f3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f41056f0 0x7fd0f419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.488+0000 7fd0f3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f419f240 con 0x7fd0f41004b0 2023-07-14T16:28:06.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.488+0000 7fd0f3fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f419f380 con 0x7fd0f41056f0 2023-07-14T16:28:06.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.488+0000 7fd0f3fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f419f510 con 0x7fd0f4102dd0 2023-07-14T16:28:06.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 0x7fd0f41976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 0x7fd0f41976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46254/0 (socket says 172.21.15.62:46254) 2023-07-14T16:28:06.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/1994377686 learned_addr learned my addr 172.21.15.62:0/1994377686 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f4102dd0 0x7fd0f4197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f37fe700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f41056f0 0x7fd0f419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/1994377686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f4102dd0 msgr2=0x7fd0f4197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f4102dd0 0x7fd0f4197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/1994377686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f41056f0 msgr2=0x7fd0f419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f41056f0 0x7fd0f419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/1994377686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40095f0 con 0x7fd0f41004b0 2023-07-14T16:28:06.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.489+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 0x7fd0f41976c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4000c00 tx=0x7fd0e4003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.490+0000 7fd0dbfff700 1 -- 172.21.15.62:0/1994377686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401d070 con 0x7fd0f41004b0 2023-07-14T16:28:06.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.490+0000 7fd0dbfff700 1 -- 172.21.15.62:0/1994377686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e400bd70 con 0x7fd0f41004b0 2023-07-14T16:28:06.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.490+0000 7fd0dbfff700 1 -- 172.21.15.62:0/1994377686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4022c20 con 0x7fd0f41004b0 2023-07-14T16:28:06.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.490+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1994377686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f419f790 con 0x7fd0f41004b0 2023-07-14T16:28:06.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.490+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1994377686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f419fbb0 con 0x7fd0f41004b0 2023-07-14T16:28:06.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.493+0000 7fd0dbfff700 1 -- 172.21.15.62:0/1994377686 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd0e4003e90 con 0x7fd0f41004b0 2023-07-14T16:28:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.495+0000 7fd0dbfff700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd0dc0a1270 0x7fd0dc0a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.495+0000 7fd0dbfff700 1 -- 172.21.15.62:0/1994377686 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd0e40cdf00 con 0x7fd0f41004b0 2023-07-14T16:28:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.495+0000 7fd10ce8cb80 1 Processor -- start 2023-07-14T16:28:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd0dc0a1270 0x7fd0dc0a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd10ce8cb80 1 -- start start 2023-07-14T16:28:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x5579554ecb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ed070 0x5579554ed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579553f1cb0 con 0x5579554ec720 2023-07-14T16:28:06.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557955444a40 con 0x5579554ed070 2023-07-14T16:28:06.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55795543ded0 con 0x5579554ede30 2023-07-14T16:28:06.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd0dc0a1270 0x7fd0dc0a3720 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005950 tx=0x7fd0e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.496+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x5579554ecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x5579554ecb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46270/0 (socket says 172.21.15.62:46270) 2023-07-14T16:28:06.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/538278019 learned_addr learned my addr 172.21.15.62:0/538278019 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ed070 0x5579554ed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f37fe700 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/538278019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 msgr2=0x5579554ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554ee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/538278019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ed070 msgr2=0x5579554ed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ed070 0x5579554ed8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/538278019 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55795526bde0 con 0x5579554ec720 2023-07-14T16:28:06.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x5579554ecb30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4097850 tx=0x7fd0e4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aee5df263253ef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.497+0000 7fd0d97fa700 1 -- 172.21.15.62:0/538278019 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4022470 con 0x5579554ec720 2023-07-14T16:28:06.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd0d97fa700 1 -- 172.21.15.62:0/538278019 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e409b350 con 0x5579554ec720 2023-07-14T16:28:06.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd0d97fa700 1 -- 172.21.15.62:0/538278019 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40a10d0 con 0x5579554ec720 2023-07-14T16:28:06.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/538278019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 msgr2=0x5579554ecb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x5579554ecb30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4097850 tx=0x7fd0e4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/538278019 shutdown_connections 2023-07-14T16:28:06.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x5579554ecb30 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/538278019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ed070 0x5579554ed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.498+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/538278019 >> 172.21.15.62:0/538278019 conn(0x5579554e1ff0 msgr2=0x5579554e7460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.499+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/538278019 shutdown_connections 2023-07-14T16:28:06.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.499+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/538278019 wait complete. 2023-07-14T16:28:06.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.499+0000 7fd10ce8cb80 1 -- start start 2023-07-14T16:28:06.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.499+0000 7fd10ce8cb80 1 Processor -- start 2023-07-14T16:28:06.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554fd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579553ddac0 con 0x5579554ed070 2023-07-14T16:28:06.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579553dd7c0 con 0x5579554ec720 2023-07-14T16:28:06.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579552c1920 con 0x5579554ede30 2023-07-14T16:28:06.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd10ce8cb80 10 client.0 _ll_register_callbacks cb 0x5579553db760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:06.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46278/0 (socket says 172.21.15.62:46278) 2023-07-14T16:28:06.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.500+0000 7fd0f27fc700 1 -- 172.21.15.62:0/1749453487 learned_addr learned my addr 172.21.15.62:0/1749453487 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0f37fe700 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0f27fc700 1 -- 172.21.15.62:0/1749453487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 msgr2=0x5579554fd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554fd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0f27fc700 1 -- 172.21.15.62:0/1749453487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 msgr2=0x5579554f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0f27fc700 1 -- 172.21.15.62:0/1749453487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f419fbb0 con 0x5579554ed070 2023-07-14T16:28:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f64f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80116f0 tx=0x7fd0e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27b7ff92a321cd5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.501+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/1749453487 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e801e830 con 0x5579554ed070 2023-07-14T16:28:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.502+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/1749453487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579553974f0 con 0x5579554ed070 2023-07-14T16:28:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.502+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/1749453487 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd0e8017070 con 0x5579554ed070 2023-07-14T16:28:06.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.502+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/1749453487 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0e802e030 con 0x5579554ed070 2023-07-14T16:28:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.502+0000 7fd10ce8cb80 10 client.19017 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.502+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/1749453487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557955254c60 con 0x5579554ed070 2023-07-14T16:28:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.502+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/1749453487 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5579553974f0 con 0x5579554ed070 2023-07-14T16:28:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.502+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/1749453487 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd0e80289f0 con 0x5579554ed070 2023-07-14T16:28:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.503+0000 7fd10ce8cb80 10 client.19017 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.503+0000 7fd10ce8cb80 10 client.19017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.503+0000 7fd10ce8cb80 20 client.19017 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:06.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.503+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557955501df0 0x557955504200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.503+0000 7fd10ce8cb80 4 client.19017 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.503+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/1749453487 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557955436eb0 con 0x557955501df0 2023-07-14T16:28:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.503+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557955501df0 0x557955504200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.504+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557955501df0 0x557955504200 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d8bb36a276c02e server_cookie=7b3b19d5dd4fc082 in_seq=0 out_seq=0 2023-07-14T16:28:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.504+0000 7fd0d6ff5700 10 client.19017 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.504+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/1749453487 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557955436eb0 con 0x557955501df0 2023-07-14T16:28:06.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.504+0000 7fd0d6ff5700 10 client.19017 handle_command_reply: tid=0 2023-07-14T16:28:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.504+0000 7fd10ce8cb80 1 client.19017 shutdown 2023-07-14T16:28:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/1749453487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 msgr2=0x5579554f64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f64f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80116f0 tx=0x7fd0e80047c0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/1749453487 shutdown_connections 2023-07-14T16:28:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554fd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557955501df0 0x557955504200 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/1749453487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f64f0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/1749453487 >> 172.21.15.62:0/1749453487 conn(0x5579554e1ff0 msgr2=0x5579554e6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd0d6ff5700 0 client.19017 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/1749453487 shutdown_connections 2023-07-14T16:28:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/1749453487 wait complete. 2023-07-14T16:28:06.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.505+0000 7fd10ce8cb80 20 client.19017 trim_cache size 0 max 16384 2023-07-14T16:28:06.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.522+0000 7fd10ce8cb80 1 Processor -- start 2023-07-14T16:28:06.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.522+0000 7fd10ce8cb80 1 -- start start 2023-07-14T16:28:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.522+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x557955508b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.522+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ed070 0x5579554fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.522+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ede30 0x5579554f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.522+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579554ff220 con 0x5579554ec720 2023-07-14T16:28:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.522+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579554f6c30 con 0x5579554ede30 2023-07-14T16:28:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.522+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579554f4000 con 0x5579554ed070 2023-07-14T16:28:06.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x557955508b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x557955508b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46286/0 (socket says 172.21.15.62:46286) 2023-07-14T16:28:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/378445552 learned_addr learned my addr 172.21.15.62:0/378445552 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f37fe700 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ede30 0x5579554f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ed070 0x5579554fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/378445552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ed070 msgr2=0x5579554fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ed070 0x5579554fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/378445552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ede30 msgr2=0x5579554f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ede30 0x5579554f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 -- 172.21.15.62:0/378445552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557955292250 con 0x5579554ec720 2023-07-14T16:28:06.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ed070 0x5579554fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:06.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.523+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x557955508b60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40970d0 tx=0x7fd0e4086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb7248978a650c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.524+0000 7fd0d9ffb700 1 -- 172.21.15.62:0/378445552 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40a3780 con 0x5579554ec720 2023-07-14T16:28:06.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.524+0000 7fd0d9ffb700 1 -- 172.21.15.62:0/378445552 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e4022470 con 0x5579554ec720 2023-07-14T16:28:06.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.524+0000 7fd0d9ffb700 1 -- 172.21.15.62:0/378445552 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e409f380 con 0x5579554ec720 2023-07-14T16:28:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.524+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/378445552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 msgr2=0x557955508b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x557955508b60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40970d0 tx=0x7fd0e4086040 comp rx=0 tx=0).stop 2023-07-14T16:28:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/378445552 shutdown_connections 2023-07-14T16:28:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ede30 0x5579554f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ec720 0x557955508b60 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/378445552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ed070 0x5579554fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/378445552 >> 172.21.15.62:0/378445552 conn(0x5579554e1ff0 msgr2=0x5579554e7430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/378445552 shutdown_connections 2023-07-14T16:28:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/378445552 wait complete. 2023-07-14T16:28:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.525+0000 7fd10ce8cb80 1 -- start start 2023-07-14T16:28:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd10ce8cb80 1 Processor -- start 2023-07-14T16:28:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd10ce8cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579554fc920 con 0x5579554ed070 2023-07-14T16:28:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579554fca90 con 0x5579554ec720 2023-07-14T16:28:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd10ce8cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579554fcc50 con 0x5579554ede30 2023-07-14T16:28:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd10ce8cb80 10 client.0 _ll_register_callbacks cb 0x7fd0e8010e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd0f2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46300/0 (socket says 172.21.15.62:46300) 2023-07-14T16:28:06.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd0f27fc700 1 -- 172.21.15.62:0/3690676721 learned_addr learned my addr 172.21.15.62:0/3690676721 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:06.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.526+0000 7fd0f37fe700 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0f27fc700 1 -- 172.21.15.62:0/3690676721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 msgr2=0x5579554fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0f27fc700 1 -- 172.21.15.62:0/3690676721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 msgr2=0x5579554f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:06.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0f27fc700 1 -- 172.21.15.62:0/3690676721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55795526bde0 con 0x5579554ed070 2023-07-14T16:28:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0f27fc700 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f93c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8028d50 tx=0x7fd0e8028bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afb272671a3ba9c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.527+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/3690676721 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e802e070 con 0x5579554ed070 2023-07-14T16:28:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/3690676721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0e409b350 con 0x5579554ed070 2023-07-14T16:28:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/3690676721 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd0e8017040 con 0x5579554ed070 2023-07-14T16:28:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/3690676721 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0e8010bf0 con 0x5579554ed070 2023-07-14T16:28:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 10 client.19023 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:06.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/3690676721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55795526a2b0 con 0x5579554ed070 2023-07-14T16:28:06.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/3690676721 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e409b350 con 0x5579554ed070 2023-07-14T16:28:06.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/3690676721 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd0e8044070 con 0x5579554ed070 2023-07-14T16:28:06.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 10 client.19023 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:06.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:06.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 20 client.19023 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:06.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557955556780 0x557955558b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:06.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 4 client.19023 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:28:06.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.528+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/3690676721 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5579553f9050 con 0x557955556780 2023-07-14T16:28:06.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.529+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557955556780 0x557955558b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:06.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.529+0000 7fd0f2ffd700 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557955556780 0x557955558b90 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d00a35655fbcb2 server_cookie=b15706c94ba3d5d8 in_seq=0 out_seq=0 2023-07-14T16:28:06.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.529+0000 7fd0d6ff5700 10 client.19023 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:06.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd0d6ff5700 1 -- 172.21.15.62:0/3690676721 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5579553f9050 con 0x557955556780 2023-07-14T16:28:06.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd0d6ff5700 10 client.19023 handle_command_reply: tid=0 2023-07-14T16:28:06.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 client.19023 shutdown 2023-07-14T16:28:06.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/3690676721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 msgr2=0x5579554f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f93c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8028d50 tx=0x7fd0e8028bd0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/3690676721 shutdown_connections 2023-07-14T16:28:06.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5579554ede30 0x5579554fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557955556780 0x557955558b90 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5579554ec720 0x5579554f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 --2- 172.21.15.62:0/3690676721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579554ed070 0x5579554f93c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd0d6ff5700 0 client.19023 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:06.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/3690676721 >> 172.21.15.62:0/3690676721 conn(0x5579554e1ff0 msgr2=0x5579554f4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/3690676721 shutdown_connections 2023-07-14T16:28:06.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 1 -- 172.21.15.62:0/3690676721 wait complete. 2023-07-14T16:28:06.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.530+0000 7fd10ce8cb80 20 client.19023 trim_cache size 0 max 16384 2023-07-14T16:28:06.514 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:28:06.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd0dc0a1270 msgr2=0x7fd0dc0a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd0dc0a1270 0x7fd0dc0a3720 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005950 tx=0x7fd0e800b410 comp rx=0 tx=0).stop 2023-07-14T16:28:06.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 msgr2=0x7fd0f41976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:06.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 0x7fd0f41976c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4000c00 tx=0x7fd0e4003a50 comp rx=0 tx=0).stop 2023-07-14T16:28:06.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1994377686 shutdown_connections 2023-07-14T16:28:06.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd0dc0a1270 0x7fd0dc0a3720 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f41056f0 0x7fd0f419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f41004b0 0x7fd0f41976c0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 --2- 172.21.15.62:0/1994377686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0f4102dd0 0x7fd0f4197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:06.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1994377686 >> 172.21.15.62:0/1994377686 conn(0x7fd0f40f9ed0 msgr2=0x7fd0f40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:06.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1994377686 shutdown_connections 2023-07-14T16:28:06.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:06.531+0000 7fd0f3fff700 1 -- 172.21.15.62:0/1994377686 wait complete. 2023-07-14T16:28:07.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:06 smithi062 ceph-mon[95543]: pgmap v731: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:28:07.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:06 smithi081 ceph-mon[127795]: pgmap v731: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:28:07.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:06 smithi164 ceph-mon[97427]: pgmap v731: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:28:07.525 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:28:07.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:28:07.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.787+0000 7f23f77ff700 1 -- 172.21.15.62:0/831233753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f0101660 msgr2=0x7f23f0107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.787+0000 7f23f77ff700 1 --2- 172.21.15.62:0/831233753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f0101660 0x7f23f0107f20 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f23e800b0a0 tx=0x7f23e800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.788+0000 7f23f77ff700 1 -- 172.21.15.62:0/831233753 shutdown_connections 2023-07-14T16:28:07.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.788+0000 7f23f77ff700 1 --2- 172.21.15.62:0/831233753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f0101660 0x7f23f0107f20 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.788+0000 7f23f77ff700 1 --2- 172.21.15.62:0/831233753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23f00ffae0 0x7f23f00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.788+0000 7f23f77ff700 1 --2- 172.21.15.62:0/831233753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f0100cd0 0x7f23f0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.788+0000 7f23f77ff700 1 -- 172.21.15.62:0/831233753 >> 172.21.15.62:0/831233753 conn(0x7f23f00fb0e0 msgr2=0x7f23f00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.788+0000 7f23f77ff700 1 -- 172.21.15.62:0/831233753 shutdown_connections 2023-07-14T16:28:07.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.788+0000 7f23f77ff700 1 -- 172.21.15.62:0/831233753 wait complete. 2023-07-14T16:28:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f77ff700 1 Processor -- start 2023-07-14T16:28:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f77ff700 1 -- start start 2023-07-14T16:28:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f77ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f00ffae0 0x7f23f0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f77ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23f0100cd0 0x7f23f0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f77ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f0101660 0x7f23f019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f77ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f019f690 con 0x7f23f00ffae0 2023-07-14T16:28:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f77ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f019f7d0 con 0x7f23f0100cd0 2023-07-14T16:28:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f77ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f019f960 con 0x7f23f0101660 2023-07-14T16:28:07.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.789+0000 7f23f559b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f00ffae0 0x7f23f0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f559b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f00ffae0 0x7f23f0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46314/0 (socket says 172.21.15.62:46314) 2023-07-14T16:28:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f559b700 1 -- 172.21.15.62:0/2218256131 learned_addr learned my addr 172.21.15.62:0/2218256131 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23f0100cd0 0x7f23f0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f0101660 0x7f23f019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f559b700 1 -- 172.21.15.62:0/2218256131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f0101660 msgr2=0x7f23f019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f559b700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f0101660 0x7f23f019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f559b700 1 -- 172.21.15.62:0/2218256131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23f0100cd0 msgr2=0x7f23f0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f559b700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23f0100cd0 0x7f23f0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f559b700 1 -- 172.21.15.62:0/2218256131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e8009d00 con 0x7f23f00ffae0 2023-07-14T16:28:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.790+0000 7f23f559b700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f00ffae0 0x7f23f0197b10 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00e970 tx=0x7f23ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.791+0000 7f23e67fc700 1 -- 172.21.15.62:0/2218256131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec009790 con 0x7f23f00ffae0 2023-07-14T16:28:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.791+0000 7f23e67fc700 1 -- 172.21.15.62:0/2218256131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23ec00c7e0 con 0x7f23f00ffae0 2023-07-14T16:28:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.791+0000 7f23f77ff700 1 -- 172.21.15.62:0/2218256131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f019fc40 con 0x7f23f00ffae0 2023-07-14T16:28:07.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.791+0000 7f23e67fc700 1 -- 172.21.15.62:0/2218256131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec003bd0 con 0x7f23f00ffae0 2023-07-14T16:28:07.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.791+0000 7f23f77ff700 1 -- 172.21.15.62:0/2218256131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f01a0150 con 0x7f23f00ffae0 2023-07-14T16:28:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.795+0000 7f23e67fc700 1 -- 172.21.15.62:0/2218256131 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f23ec00c950 con 0x7f23f00ffae0 2023-07-14T16:28:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.796+0000 7f23e67fc700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f23dc0a1150 0x7f23dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.796+0000 7f23e67fc700 1 -- 172.21.15.62:0/2218256131 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23ec014070 con 0x7f23f00ffae0 2023-07-14T16:28:07.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f23dc0a1150 0x7f23dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f240b0ebb80 1 Processor -- start 2023-07-14T16:28:07.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f240b0ebb80 1 -- start start 2023-07-14T16:28:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e4335170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4335f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e4336970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e4231d30 con 0x5646e4336500 2023-07-14T16:28:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e4284ac0 con 0x5646e4334d60 2023-07-14T16:28:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.797+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e427df50 con 0x5646e43356b0 2023-07-14T16:28:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f23dc0a1150 0x7f23dc0a3600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f23e0005950 tx=0x7f23e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e4336970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f4d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4335f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e4336970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46330/0 (socket says 172.21.15.62:46330) 2023-07-14T16:28:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 -- 172.21.15.62:0/1229092066 learned_addr learned my addr 172.21.15.62:0/1229092066 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f559b700 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e4335170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 -- 172.21.15.62:0/1229092066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 msgr2=0x5646e4335f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4335f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 -- 172.21.15.62:0/1229092066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 msgr2=0x5646e4335170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e4335170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 -- 172.21.15.62:0/1229092066 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e40abe60 con 0x5646e4336500 2023-07-14T16:28:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4335f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:07.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f559b700 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e4335170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:07.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e4336970 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f23e800b0a0 tx=0x7f23e8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c739942a10ca80 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:07.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23db7fe700 1 -- 172.21.15.62:0/1229092066 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e800e040 con 0x5646e4336500 2023-07-14T16:28:07.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23db7fe700 1 -- 172.21.15.62:0/1229092066 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23e800ba50 con 0x5646e4336500 2023-07-14T16:28:07.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.799+0000 7f23db7fe700 1 -- 172.21.15.62:0/1229092066 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e8003dc0 con 0x5646e4336500 2023-07-14T16:28:07.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1229092066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 msgr2=0x5646e4336970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e4336970 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f23e800b0a0 tx=0x7f23e8009c30 comp rx=0 tx=0).stop 2023-07-14T16:28:07.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1229092066 shutdown_connections 2023-07-14T16:28:07.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e4336970 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e4335170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1229092066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4335f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1229092066 >> 172.21.15.62:0/1229092066 conn(0x5646e4321e20 msgr2=0x5646e4327250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1229092066 shutdown_connections 2023-07-14T16:28:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1229092066 wait complete. 2023-07-14T16:28:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.800+0000 7f240b0ebb80 1 -- start start 2023-07-14T16:28:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f240b0ebb80 1 Processor -- start 2023-07-14T16:28:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e433e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e43356b0 0x5646e433ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4336500 0x5646e4345f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e421db40 con 0x5646e43356b0 2023-07-14T16:28:07.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e421d840 con 0x5646e4334d60 2023-07-14T16:28:07.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e41019a0 con 0x5646e4336500 2023-07-14T16:28:07.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f240b0ebb80 10 client.0 _ll_register_callbacks cb 0x5646e421b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:07.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f23f4d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e43356b0 0x5646e433ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f23f559b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e433e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.801+0000 7f23f4d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e43356b0 0x5646e433ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46346/0 (socket says 172.21.15.62:46346) 2023-07-14T16:28:07.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f4d9a700 1 -- 172.21.15.62:0/1582354314 learned_addr learned my addr 172.21.15.62:0/1582354314 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:07.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4336500 0x5646e4345f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f559b700 1 -- 172.21.15.62:0/1582354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4336500 msgr2=0x5646e4345f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f559b700 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4336500 0x5646e4345f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f559b700 1 -- 172.21.15.62:0/1582354314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e43356b0 msgr2=0x5646e433ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f559b700 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e43356b0 0x5646e433ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f559b700 1 -- 172.21.15.62:0/1582354314 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f01a0150 con 0x5646e4334d60 2023-07-14T16:28:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4336500 0x5646e4345f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.802+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e43356b0 0x5646e433ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f23f559b700 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e433e9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f23ec099c10 tx=0x7f23ec00cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9512d39fbd6216 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f23d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:28:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f23d8ff9700 1 -- 172.21.15.62:0/1582354314 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0a2360 con 0x5646e4334d60 2023-07-14T16:28:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f23d8ff9700 1 -- 172.21.15.62:0/1582354314 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23ec0b1070 con 0x5646e4334d60 2023-07-14T16:28:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1582354314 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646e41d7570 con 0x5646e4334d60 2023-07-14T16:28:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f23d8ff9700 1 -- 172.21.15.62:0/1582354314 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ec0ca170 con 0x5646e4334d60 2023-07-14T16:28:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f240b0ebb80 10 client.28625 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1582354314 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f23e0010a50 con 0x5646e4334d60 2023-07-14T16:28:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.803+0000 7f23d8ff9700 1 -- 172.21.15.62:0/1582354314 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23ec063080 con 0x5646e4334d60 2023-07-14T16:28:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.804+0000 7f23d8ff9700 1 -- 172.21.15.62:0/1582354314 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f23ec099470 con 0x5646e4334d60 2023-07-14T16:28:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.804+0000 7f240b0ebb80 10 client.28625 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.804+0000 7f240b0ebb80 10 client.28625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.804+0000 7f240b0ebb80 20 client.28625 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.804+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5646e434a6e0 0x5646e434caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.804+0000 7f240b0ebb80 4 client.28625 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:07.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.804+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1582354314 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646e4276f30 con 0x5646e434a6e0 2023-07-14T16:28:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.804+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5646e434a6e0 0x5646e434caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.805+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5646e434a6e0 0x5646e434caf0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41fa413e21bae04 server_cookie=3e3d15303f80b4a9 in_seq=0 out_seq=0 2023-07-14T16:28:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.805+0000 7f23d8ff9700 10 client.28625 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f23d8ff9700 1 -- 172.21.15.62:0/1582354314 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5646e4276f30 con 0x5646e434a6e0 2023-07-14T16:28:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f23d8ff9700 10 client.28625 handle_command_reply: tid=0 2023-07-14T16:28:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 client.28625 shutdown 2023-07-14T16:28:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1582354314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 msgr2=0x5646e433e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e433e9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f23ec099c10 tx=0x7f23ec00cd20 comp rx=0 tx=0).stop 2023-07-14T16:28:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1582354314 shutdown_connections 2023-07-14T16:28:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4336500 0x5646e4345f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5646e434a6e0 0x5646e434caf0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4334d60 0x5646e433e9c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/1582354314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e43356b0 0x5646e433ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f23d8ff9700 0 client.28625 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:07.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.806+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1582354314 >> 172.21.15.62:0/1582354314 conn(0x5646e4321e20 msgr2=0x5646e4326930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:07.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.807+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1582354314 shutdown_connections 2023-07-14T16:28:07.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.807+0000 7f240b0ebb80 1 -- 172.21.15.62:0/1582354314 wait complete. 2023-07-14T16:28:07.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.807+0000 7f240b0ebb80 20 client.28625 trim_cache size 0 max 16384 2023-07-14T16:28:07.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.825+0000 7f240b0ebb80 1 Processor -- start 2023-07-14T16:28:07.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.825+0000 7f240b0ebb80 1 -- start start 2023-07-14T16:28:07.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.825+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4334d60 0x5646e4346830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.825+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4345f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.825+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4336500 0x5646e433c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.825+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e434a530 con 0x5646e4334d60 2023-07-14T16:28:07.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.825+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e4335ab0 con 0x5646e4336500 2023-07-14T16:28:07.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.825+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e4321a40 con 0x5646e43356b0 2023-07-14T16:28:07.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f559b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4334d60 0x5646e4346830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f559b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4334d60 0x5646e4346830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46358/0 (socket says 172.21.15.62:46358) 2023-07-14T16:28:07.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f559b700 1 -- 172.21.15.62:0/3700197741 learned_addr learned my addr 172.21.15.62:0/3700197741 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:07.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4336500 0x5646e433c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4345f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f559b700 1 -- 172.21.15.62:0/3700197741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 msgr2=0x5646e4345f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:07.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f559b700 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4345f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f559b700 1 -- 172.21.15.62:0/3700197741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4336500 msgr2=0x5646e433c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:07.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f559b700 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4336500 0x5646e433c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f559b700 1 -- 172.21.15.62:0/3700197741 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f0108850 con 0x5646e4334d60 2023-07-14T16:28:07.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.826+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4345f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:07.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.827+0000 7f23f559b700 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4334d60 0x5646e4346830 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009230 tx=0x7f23ec096380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931dd46aca554f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:07.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.827+0000 7f23dbfff700 1 -- 172.21.15.62:0/3700197741 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0a2040 con 0x5646e4334d60 2023-07-14T16:28:07.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.827+0000 7f23dbfff700 1 -- 172.21.15.62:0/3700197741 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5646e41d7570 con 0x5646e4334d60 2023-07-14T16:28:07.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.827+0000 7f23dbfff700 1 -- 172.21.15.62:0/3700197741 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec09a3d0 con 0x5646e4334d60 2023-07-14T16:28:07.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3700197741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4334d60 msgr2=0x5646e4346830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4334d60 0x5646e4346830 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009230 tx=0x7f23ec096380 comp rx=0 tx=0).stop 2023-07-14T16:28:07.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3700197741 shutdown_connections 2023-07-14T16:28:07.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e4336500 0x5646e433c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4334d60 0x5646e4346830 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3700197741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e43356b0 0x5646e4345f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3700197741 >> 172.21.15.62:0/3700197741 conn(0x5646e4321e20 msgr2=0x5646e433d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:07.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3700197741 shutdown_connections 2023-07-14T16:28:07.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3700197741 wait complete. 2023-07-14T16:28:07.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.828+0000 7f240b0ebb80 1 -- start start 2023-07-14T16:28:07.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f240b0ebb80 1 Processor -- start 2023-07-14T16:28:07.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4334d60 0x5646e4342d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e43356b0 0x5646e43432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f240b0ebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e439e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e43439b0 con 0x5646e4336500 2023-07-14T16:28:07.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e439ead0 con 0x5646e43356b0 2023-07-14T16:28:07.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f240b0ebb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e439ec60 con 0x5646e4334d60 2023-07-14T16:28:07.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f240b0ebb80 10 client.0 _ll_register_callbacks cb 0x5646e421b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:07.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.829+0000 7f23f5d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e439e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f5d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e439e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46362/0 (socket says 172.21.15.62:46362) 2023-07-14T16:28:07.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f5d9c700 1 -- 172.21.15.62:0/3784786322 learned_addr learned my addr 172.21.15.62:0/3784786322 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:07.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f4d9a700 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e43356b0 0x5646e43432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f559b700 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4334d60 0x5646e4342d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f5d9c700 1 -- 172.21.15.62:0/3784786322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4334d60 msgr2=0x5646e4342d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:07.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4334d60 0x5646e4342d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f5d9c700 1 -- 172.21.15.62:0/3784786322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e43356b0 msgr2=0x5646e43432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:07.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e43356b0 0x5646e43432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f5d9c700 1 -- 172.21.15.62:0/3784786322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e0010a50 con 0x5646e4336500 2023-07-14T16:28:07.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23f5d9c700 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e439e590 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f23e800b0a0 tx=0x7f23e8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ce5f232a591d4e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:07.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.830+0000 7f23d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:07.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.831+0000 7f23d8ff9700 1 -- 172.21.15.62:0/3784786322 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e8029070 con 0x5646e4336500 2023-07-14T16:28:07.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.831+0000 7f23d8ff9700 1 -- 172.21.15.62:0/3784786322 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23e800b700 con 0x5646e4336500 2023-07-14T16:28:07.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.831+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3784786322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23e800ba50 con 0x5646e4336500 2023-07-14T16:28:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.831+0000 7f23d8ff9700 1 -- 172.21.15.62:0/3784786322 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23e800e040 con 0x5646e4336500 2023-07-14T16:28:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.831+0000 7f240b0ebb80 10 client.19041 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.831+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3784786322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5646e4094ce0 con 0x5646e4336500 2023-07-14T16:28:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.831+0000 7f23d8ff9700 1 -- 172.21.15.62:0/3784786322 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23e800ba50 con 0x5646e4336500 2023-07-14T16:28:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.831+0000 7f23d8ff9700 1 -- 172.21.15.62:0/3784786322 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f23e8036070 con 0x5646e4336500 2023-07-14T16:28:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.832+0000 7f240b0ebb80 10 client.19041 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.832+0000 7f240b0ebb80 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:07.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.832+0000 7f240b0ebb80 20 client.19041 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:07.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.832+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5646e43a01e0 0x5646e43a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:07.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.832+0000 7f240b0ebb80 4 client.19041 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:28:07.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.832+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3784786322 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5646e42390d0 con 0x5646e43a01e0 2023-07-14T16:28:07.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.832+0000 7f23f559b700 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5646e43a01e0 0x5646e43a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:07.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.833+0000 7f23d8ff9700 10 client.19041 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:07.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.833+0000 7f23f559b700 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5646e43a01e0 0x5646e43a25f0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b10bbec6300b547 server_cookie=316dd8397f66a31f in_seq=0 out_seq=0 2023-07-14T16:28:07.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f23d8ff9700 1 -- 172.21.15.62:0/3784786322 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5646e42390d0 con 0x5646e43a01e0 2023-07-14T16:28:07.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f23d8ff9700 10 client.19041 handle_command_reply: tid=0 2023-07-14T16:28:07.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 client.19041 shutdown 2023-07-14T16:28:07.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3784786322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 msgr2=0x5646e439e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e439e590 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f23e800b0a0 tx=0x7f23e8009510 comp rx=0 tx=0).stop 2023-07-14T16:28:07.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3784786322 shutdown_connections 2023-07-14T16:28:07.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646e4336500 0x5646e439e590 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646e4334d60 0x5646e4342d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5646e43a01e0 0x5646e43a25f0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 --2- 172.21.15.62:0/3784786322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5646e43356b0 0x5646e43432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3784786322 >> 172.21.15.62:0/3784786322 conn(0x5646e4321e20 msgr2=0x5646e4346830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:07.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.834+0000 7f23d8ff9700 0 client.19041 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:07.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.835+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3784786322 shutdown_connections 2023-07-14T16:28:07.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.835+0000 7f240b0ebb80 1 -- 172.21.15.62:0/3784786322 wait complete. 2023-07-14T16:28:07.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.835+0000 7f240b0ebb80 20 client.19041 trim_cache size 0 max 16384 2023-07-14T16:28:07.816 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:28:07.816 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:28:07.816 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "09800069-6bd5-43d7-a700-0e415084587a", 2023-07-14T16:28:07.816 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:28:07.816 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:28:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.835+0000 7f23f77ff700 1 -- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f23dc0a1150 msgr2=0x7f23dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.835+0000 7f23f77ff700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f23dc0a1150 0x7f23dc0a3600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f23e0005950 tx=0x7f23e000b410 comp rx=0 tx=0).stop 2023-07-14T16:28:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.835+0000 7f23f77ff700 1 -- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f00ffae0 msgr2=0x7f23f0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.835+0000 7f23f77ff700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f00ffae0 0x7f23f0197b10 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00e970 tx=0x7f23ec00ec80 comp rx=0 tx=0).stop 2023-07-14T16:28:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.836+0000 7f23f77ff700 1 -- 172.21.15.62:0/2218256131 shutdown_connections 2023-07-14T16:28:07.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.836+0000 7f23f77ff700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f0101660 0x7f23f019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.836+0000 7f23f77ff700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23f00ffae0 0x7f23f0197b10 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.836+0000 7f23f77ff700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f23dc0a1150 0x7f23dc0a3600 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.836+0000 7f23f77ff700 1 --2- 172.21.15.62:0/2218256131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23f0100cd0 0x7f23f0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:07.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.836+0000 7f23f77ff700 1 -- 172.21.15.62:0/2218256131 >> 172.21.15.62:0/2218256131 conn(0x7f23f00fb0e0 msgr2=0x7f23f01065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:07.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.836+0000 7f23f77ff700 1 -- 172.21.15.62:0/2218256131 shutdown_connections 2023-07-14T16:28:07.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:07.836+0000 7f23f77ff700 1 -- 172.21.15.62:0/2218256131 wait complete. 2023-07-14T16:28:07.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:28:08.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 -- 172.21.15.62:0/3494640464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 msgr2=0x7f88d81010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 --2- 172.21.15.62:0/3494640464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 0x7f88d81010f0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009990 tx=0x7f88c8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 -- 172.21.15.62:0/3494640464 shutdown_connections 2023-07-14T16:28:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 --2- 172.21.15.62:0/3494640464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88d8101630 0x7f88d8107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 --2- 172.21.15.62:0/3494640464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88d80ff9f0 0x7f88d80ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 --2- 172.21.15.62:0/3494640464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 0x7f88d81010f0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 -- 172.21.15.62:0/3494640464 >> 172.21.15.62:0/3494640464 conn(0x7f88d80fb010 msgr2=0x7f88d80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 -- 172.21.15.62:0/3494640464 shutdown_connections 2023-07-14T16:28:08.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.099+0000 7f88de79d700 1 -- 172.21.15.62:0/3494640464 wait complete. 2023-07-14T16:28:08.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88de79d700 1 Processor -- start 2023-07-14T16:28:08.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88de79d700 1 -- start start 2023-07-14T16:28:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88de79d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88d80ff9f0 0x7f88d81976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88de79d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 0x7f88d8197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88de79d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88d8101630 0x7f88d819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88de79d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d819f230 con 0x7f88d8100c80 2023-07-14T16:28:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88de79d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d819f370 con 0x7f88d80ff9f0 2023-07-14T16:28:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88de79d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d819f500 con 0x7f88d8101630 2023-07-14T16:28:08.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dcf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 0x7f88d8197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dcf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 0x7f88d8197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46380/0 (socket says 172.21.15.62:46380) 2023-07-14T16:28:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dd79b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88d80ff9f0 0x7f88d81976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dcf9a700 1 -- 172.21.15.62:0/3199115949 learned_addr learned my addr 172.21.15.62:0/3199115949 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88ddf9c700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88d8101630 0x7f88d819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dd79b700 1 -- 172.21.15.62:0/3199115949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88d8101630 msgr2=0x7f88d819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dd79b700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88d8101630 0x7f88d819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dd79b700 1 -- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 msgr2=0x7f88d8197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dd79b700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 0x7f88d8197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88dd79b700 1 -- 172.21.15.62:0/3199115949 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80095f0 con 0x7f88d80ff9f0 2023-07-14T16:28:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.101+0000 7f88ddf9c700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88d8101630 0x7f88d819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.103+0000 7f88dd79b700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88d80ff9f0 0x7f88d81976b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9f0 tx=0x7f88d400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.103+0000 7f88ce7fc700 1 -- 172.21.15.62:0/3199115949 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400eee0 con 0x7f88d80ff9f0 2023-07-14T16:28:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.103+0000 7f88ce7fc700 1 -- 172.21.15.62:0/3199115949 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88d400cbd0 con 0x7f88d80ff9f0 2023-07-14T16:28:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.103+0000 7f88de79d700 1 -- 172.21.15.62:0/3199115949 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d819f7e0 con 0x7f88d80ff9f0 2023-07-14T16:28:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.103+0000 7f88ce7fc700 1 -- 172.21.15.62:0/3199115949 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4018680 con 0x7f88d80ff9f0 2023-07-14T16:28:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.103+0000 7f88de79d700 1 -- 172.21.15.62:0/3199115949 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d819fcf0 con 0x7f88d80ff9f0 2023-07-14T16:28:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.106+0000 7f88ce7fc700 1 -- 172.21.15.62:0/3199115949 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f88d400cd40 con 0x7f88d80ff9f0 2023-07-14T16:28:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.107+0000 7f88f3372b80 1 Processor -- start 2023-07-14T16:28:08.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.107+0000 7f88f3372b80 1 -- start start 2023-07-14T16:28:08.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.107+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c42fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.107+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c4308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.107+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c4312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.107+0000 7f88f3372b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c334cb0 con 0x56087c430070 2023-07-14T16:28:08.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.107+0000 7f88f3372b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c387a40 con 0x56087c42f720 2023-07-14T16:28:08.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.107+0000 7f88f3372b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c380ed0 con 0x56087c430e30 2023-07-14T16:28:08.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88ce7fc700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88c40a11f0 0x7f88c40a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c4308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c4308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46390/0 (socket says 172.21.15.62:46390) 2023-07-14T16:28:08.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dd79b700 1 -- 172.21.15.62:0/372025417 learned_addr learned my addr 172.21.15.62:0/372025417 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88ddf9c700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88c40a11f0 0x7f88c40a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88ddf9c700 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c42fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88ce7fc700 1 -- 172.21.15.62:0/3199115949 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f88d4014070 con 0x7f88d80ff9f0 2023-07-14T16:28:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c4312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dd79b700 1 -- 172.21.15.62:0/372025417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 msgr2=0x56087c4312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dd79b700 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c4312a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dd79b700 1 -- 172.21.15.62:0/372025417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 msgr2=0x56087c42fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dd79b700 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c42fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dd79b700 1 -- 172.21.15.62:0/372025417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56087c1aede0 con 0x56087c430070 2023-07-14T16:28:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.108+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c4312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.109+0000 7f88dd79b700 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c4308a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f88d4000c00 tx=0x7f88d4024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45961a54ecde698c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:08.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.109+0000 7f88ddf9c700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88c40a11f0 0x7f88c40a36a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f88d0015e20 tx=0x7f88d0015420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.109+0000 7f88c37fe700 1 -- 172.21.15.62:0/372025417 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4099f40 con 0x56087c430070 2023-07-14T16:28:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.109+0000 7f88c37fe700 1 -- 172.21.15.62:0/372025417 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88d409df70 con 0x56087c430070 2023-07-14T16:28:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.109+0000 7f88c37fe700 1 -- 172.21.15.62:0/372025417 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d40ca0e0 con 0x56087c430070 2023-07-14T16:28:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 -- 172.21.15.62:0/372025417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 msgr2=0x56087c4308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c4308a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f88d4000c00 tx=0x7f88d4024000 comp rx=0 tx=0).stop 2023-07-14T16:28:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 -- 172.21.15.62:0/372025417 shutdown_connections 2023-07-14T16:28:08.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c4312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c42fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 --2- 172.21.15.62:0/372025417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c4308a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 -- 172.21.15.62:0/372025417 >> 172.21.15.62:0/372025417 conn(0x56087c424ff0 msgr2=0x56087c42a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 -- 172.21.15.62:0/372025417 shutdown_connections 2023-07-14T16:28:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.111+0000 7f88f3372b80 1 -- 172.21.15.62:0/372025417 wait complete. 2023-07-14T16:28:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.112+0000 7f88f3372b80 1 -- start start 2023-07-14T16:28:08.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.112+0000 7f88f3372b80 1 Processor -- start 2023-07-14T16:28:08.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c42f720 0x56087c439390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430070 0x56087c4398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c440910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88f3372b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c320ac0 con 0x56087c42f720 2023-07-14T16:28:08.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88f3372b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c3207c0 con 0x56087c430070 2023-07-14T16:28:08.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88f3372b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c204920 con 0x56087c430e30 2023-07-14T16:28:08.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88f3372b80 10 client.0 _ll_register_callbacks cb 0x56087c31e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:08.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c42f720 0x56087c439390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dd79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c42f720 0x56087c439390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46400/0 (socket says 172.21.15.62:46400) 2023-07-14T16:28:08.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dd79b700 1 -- 172.21.15.62:0/3362564157 learned_addr learned my addr 172.21.15.62:0/3362564157 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:08.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88ddf9c700 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c440910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430070 0x56087c4398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dd79b700 1 -- 172.21.15.62:0/3362564157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 msgr2=0x56087c440910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:08.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dd79b700 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c440910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dd79b700 1 -- 172.21.15.62:0/3362564157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430070 msgr2=0x56087c4398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:08.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dd79b700 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430070 0x56087c4398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.113+0000 7f88dd79b700 1 -- 172.21.15.62:0/3362564157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d40c8e70 con 0x56087c42f720 2023-07-14T16:28:08.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88dd79b700 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c42f720 0x56087c439390 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f88d4096440 tx=0x7f88d409a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f939ad90922d8e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:08.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:08.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88c0ff9700 1 -- 172.21.15.62:0/3362564157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4098080 con 0x56087c42f720 2023-07-14T16:28:08.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 1 -- 172.21.15.62:0/3362564157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56087c2da4f0 con 0x56087c42f720 2023-07-14T16:28:08.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88c0ff9700 1 -- 172.21.15.62:0/3362564157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f88d4099f40 con 0x56087c42f720 2023-07-14T16:28:08.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88c0ff9700 1 -- 172.21.15.62:0/3362564157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88d40a1f00 con 0x56087c42f720 2023-07-14T16:28:08.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 10 client.19053 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:08.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 1 -- 172.21.15.62:0/3362564157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56087c197c60 con 0x56087c42f720 2023-07-14T16:28:08.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88c0ff9700 1 -- 172.21.15.62:0/3362564157 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56087c2da4f0 con 0x56087c42f720 2023-07-14T16:28:08.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88c0ff9700 1 -- 172.21.15.62:0/3362564157 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f88d4095e50 con 0x56087c42f720 2023-07-14T16:28:08.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 10 client.19053 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:08.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 10 client.19053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:08.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 20 client.19053 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:08.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56087c444c20 0x56087c447030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 4 client.19053 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:08.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88f3372b80 1 -- 172.21.15.62:0/3362564157 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56087c379eb0 con 0x56087c444c20 2023-07-14T16:28:08.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.114+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56087c444c20 0x56087c447030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.116+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56087c444c20 0x56087c447030 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42ffe55450b712 server_cookie=9a8e67888c273109 in_seq=0 out_seq=0 2023-07-14T16:28:08.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.116+0000 7f88c0ff9700 10 client.19053 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:08.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.116+0000 7f88c0ff9700 1 -- 172.21.15.62:0/3362564157 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56087c379eb0 con 0x56087c444c20 2023-07-14T16:28:08.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.116+0000 7f88c0ff9700 10 client.19053 handle_command_reply: tid=0 2023-07-14T16:28:08.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.116+0000 7f88f3372b80 1 client.19053 shutdown 2023-07-14T16:28:08.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 -- 172.21.15.62:0/3362564157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c42f720 msgr2=0x56087c439390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c42f720 0x56087c439390 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f88d4096440 tx=0x7f88d409a450 comp rx=0 tx=0).stop 2023-07-14T16:28:08.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 -- 172.21.15.62:0/3362564157 shutdown_connections 2023-07-14T16:28:08.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430e30 0x56087c440910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c42f720 0x56087c439390 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56087c444c20 0x56087c447030 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 --2- 172.21.15.62:0/3362564157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430070 0x56087c4398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 -- 172.21.15.62:0/3362564157 >> 172.21.15.62:0/3362564157 conn(0x56087c424ff0 msgr2=0x56087c429b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:08.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88c0ff9700 0 client.19053 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:08.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 -- 172.21.15.62:0/3362564157 shutdown_connections 2023-07-14T16:28:08.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 1 -- 172.21.15.62:0/3362564157 wait complete. 2023-07-14T16:28:08.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.117+0000 7f88f3372b80 20 client.19053 trim_cache size 0 max 16384 2023-07-14T16:28:08.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.135+0000 7f88f3372b80 1 Processor -- start 2023-07-14T16:28:08.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.135+0000 7f88f3372b80 1 -- start start 2023-07-14T16:28:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c446c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430070 0x56087c4408d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430e30 0x56087c433d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88f3372b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c424bc0 con 0x56087c430e30 2023-07-14T16:28:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88f3372b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c42b320 con 0x56087c42f720 2023-07-14T16:28:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88f3372b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c424e00 con 0x56087c430070 2023-07-14T16:28:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88ddf9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430e30 0x56087c433d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88ddf9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430e30 0x56087c433d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46412/0 (socket says 172.21.15.62:46412) 2023-07-14T16:28:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88ddf9c700 1 -- 172.21.15.62:0/46801955 learned_addr learned my addr 172.21.15.62:0/46801955 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88dd79b700 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c446c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.136+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430070 0x56087c4408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88dd79b700 1 -- 172.21.15.62:0/46801955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430070 msgr2=0x56087c4408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88dd79b700 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430070 0x56087c4408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88dd79b700 1 -- 172.21.15.62:0/46801955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430e30 msgr2=0x56087c433d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88dd79b700 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430e30 0x56087c433d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88dd79b700 1 -- 172.21.15.62:0/46801955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56087c438310 con 0x56087c42f720 2023-07-14T16:28:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88dd79b700 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c446c30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f88d4099a10 tx=0x7f88d40ca6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2278cc9fc7d7872 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88c3fff700 1 -- 172.21.15.62:0/46801955 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4098fc0 con 0x56087c42f720 2023-07-14T16:28:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88c3fff700 1 -- 172.21.15.62:0/46801955 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88d40caaf0 con 0x56087c42f720 2023-07-14T16:28:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.137+0000 7f88c3fff700 1 -- 172.21.15.62:0/46801955 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4099260 con 0x56087c42f720 2023-07-14T16:28:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 -- 172.21.15.62:0/46801955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 msgr2=0x56087c446c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c446c30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f88d4099a10 tx=0x7f88d40ca6d0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 -- 172.21.15.62:0/46801955 shutdown_connections 2023-07-14T16:28:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430e30 0x56087c433d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c42f720 0x56087c446c30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 --2- 172.21.15.62:0/46801955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c430070 0x56087c4408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 -- 172.21.15.62:0/46801955 >> 172.21.15.62:0/46801955 conn(0x56087c424ff0 msgr2=0x56087c441200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 -- 172.21.15.62:0/46801955 shutdown_connections 2023-07-14T16:28:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.138+0000 7f88f3372b80 1 -- 172.21.15.62:0/46801955 wait complete. 2023-07-14T16:28:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 1 -- start start 2023-07-14T16:28:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 1 Processor -- start 2023-07-14T16:28:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c42f720 0x56087c440140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c440680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430e30 0x56087c43cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c43d1b0 con 0x56087c430070 2023-07-14T16:28:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c43d320 con 0x56087c430e30 2023-07-14T16:28:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56087c43d4e0 con 0x56087c42f720 2023-07-14T16:28:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.139+0000 7f88f3372b80 10 client.0 _ll_register_callbacks cb 0x7f88d0015970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c440680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c440680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46426/0 (socket says 172.21.15.62:46426) 2023-07-14T16:28:08.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 -- 172.21.15.62:0/1141445817 learned_addr learned my addr 172.21.15.62:0/1141445817 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88ddf9c700 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430e30 0x56087c43cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dd79b700 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c42f720 0x56087c440140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 -- 172.21.15.62:0/1141445817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c42f720 msgr2=0x56087c440140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c42f720 0x56087c440140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 -- 172.21.15.62:0/1141445817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430e30 msgr2=0x56087c43cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430e30 0x56087c43cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 -- 172.21.15.62:0/1141445817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56087c197c60 con 0x56087c430070 2023-07-14T16:28:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88dcf9a700 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c440680 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f88c80098e0 tx=0x7f88c8004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983050a3373d616a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.140+0000 7f88c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.141+0000 7f88c0ff9700 1 -- 172.21.15.62:0/1141445817 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8026a50 con 0x56087c430070 2023-07-14T16:28:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.141+0000 7f88f3372b80 1 -- 172.21.15.62:0/1141445817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88d409df70 con 0x56087c430070 2023-07-14T16:28:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.141+0000 7f88c0ff9700 1 -- 172.21.15.62:0/1141445817 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f88c8026bb0 con 0x56087c430070 2023-07-14T16:28:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.141+0000 7f88c0ff9700 1 -- 172.21.15.62:0/1141445817 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c8026d70 con 0x56087c430070 2023-07-14T16:28:08.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.141+0000 7f88f3372b80 10 client.19059 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:08.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.141+0000 7f88f3372b80 1 -- 172.21.15.62:0/1141445817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56087c1d5250 con 0x56087c430070 2023-07-14T16:28:08.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.141+0000 7f88c0ff9700 1 -- 172.21.15.62:0/1141445817 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88d409df70 con 0x56087c430070 2023-07-14T16:28:08.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.141+0000 7f88c0ff9700 1 -- 172.21.15.62:0/1141445817 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f88c8003b70 con 0x56087c430070 2023-07-14T16:28:08.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.142+0000 7f88f3372b80 10 client.19059 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:08.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.142+0000 7f88f3372b80 10 client.19059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:08.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.142+0000 7f88f3372b80 20 client.19059 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:08.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.142+0000 7f88f3372b80 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56087c499fa0 0x56087c49c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:08.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.142+0000 7f88f3372b80 4 client.19059 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:28:08.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.142+0000 7f88f3372b80 1 -- 172.21.15.62:0/1141445817 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56087c33c050 con 0x56087c499fa0 2023-07-14T16:28:08.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.142+0000 7f88dd79b700 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56087c499fa0 0x56087c49c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:08.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88dd79b700 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56087c499fa0 0x56087c49c3b0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbac2f8e5c1e7863 server_cookie=ea246b9c519bb166 in_seq=0 out_seq=0 2023-07-14T16:28:08.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88c0ff9700 10 client.19059 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:08.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88c0ff9700 1 -- 172.21.15.62:0/1141445817 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56087c33c050 con 0x56087c499fa0 2023-07-14T16:28:08.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88c0ff9700 10 client.19059 handle_command_reply: tid=0 2023-07-14T16:28:08.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 client.19059 shutdown 2023-07-14T16:28:08.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 -- 172.21.15.62:0/1141445817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 msgr2=0x56087c440680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c440680 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f88c80098e0 tx=0x7f88c8004190 comp rx=0 tx=0).stop 2023-07-14T16:28:08.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 -- 172.21.15.62:0/1141445817 shutdown_connections 2023-07-14T16:28:08.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56087c499fa0 0x56087c49c3b0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56087c430e30 0x56087c43cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56087c42f720 0x56087c440140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88c0ff9700 0 client.19059 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:08.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 --2- 172.21.15.62:0/1141445817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56087c430070 0x56087c440680 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 -- 172.21.15.62:0/1141445817 >> 172.21.15.62:0/1141445817 conn(0x56087c424ff0 msgr2=0x56087c446c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:08.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.143+0000 7f88f3372b80 1 -- 172.21.15.62:0/1141445817 shutdown_connections 2023-07-14T16:28:08.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88f3372b80 1 -- 172.21.15.62:0/1141445817 wait complete. 2023-07-14T16:28:08.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88f3372b80 20 client.19059 trim_cache size 0 max 16384 2023-07-14T16:28:08.126 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:28:08.126 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:28:08.126 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "09800069-6bd5-43d7-a700-0e415084587a": { 2023-07-14T16:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "09800069-6bd5-43d7-a700-0e415084587a", 2023-07-14T16:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:28:08.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 -- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88c40a11f0 msgr2=0x7f88c40a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88c40a11f0 0x7f88c40a36a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f88d0015e20 tx=0x7f88d0015420 comp rx=0 tx=0).stop 2023-07-14T16:28:08.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 -- 172.21.15.62:0/3199115949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88d80ff9f0 msgr2=0x7f88d81976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:08.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88d80ff9f0 0x7f88d81976b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9f0 tx=0x7f88d400ed00 comp rx=0 tx=0).stop 2023-07-14T16:28:08.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 -- 172.21.15.62:0/3199115949 shutdown_connections 2023-07-14T16:28:08.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88d8101630 0x7f88d819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f88c40a11f0 0x7f88c40a36a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88d80ff9f0 0x7f88d81976b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 --2- 172.21.15.62:0/3199115949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88d8100c80 0x7f88d8197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:08.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.144+0000 7f88de79d700 1 -- 172.21.15.62:0/3199115949 >> 172.21.15.62:0/3199115949 conn(0x7f88d80fb010 msgr2=0x7f88d80fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:08.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.145+0000 7f88de79d700 1 -- 172.21.15.62:0/3199115949 shutdown_connections 2023-07-14T16:28:08.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:08.145+0000 7f88de79d700 1 -- 172.21.15.62:0/3199115949 wait complete. 2023-07-14T16:28:08.140 INFO:tasks.cephfs.filesystem:scrub status for tag:09800069-6bd5-43d7-a700-0e415084587a - {'path': '/', 'tag': '09800069-6bd5-43d7-a700-0e415084587a', 'options': 'recursive,force'} 2023-07-14T16:28:08.149 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:28:07 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:28:07.846+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v237>,<0x10000000003/client.0 v238>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v250>,<0x10000000001/sv_0 v264>,<0x10000000000/_nogroup v258>,<0x1/volumes v248>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:28:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: pgmap v732: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:28:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:28:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:28:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:28:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:28:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:28:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:08 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: pgmap v732: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:28:09.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:28:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:28:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:28:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:28:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:28:09.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:08 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: pgmap v732: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:28:09.402 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:28:09.402 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.402 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:28:09.402 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:28:09.403 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.403 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:28:09.403 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:28:09.403 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:09.403 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:08 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 -- 172.21.15.62:0/2289772386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 msgr2=0x7f7d08107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/2289772386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 0x7f7d08107f40 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000b0a0 tx=0x7f7d0000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 -- 172.21.15.62:0/2289772386 shutdown_connections 2023-07-14T16:28:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/2289772386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 0x7f7d08107f40 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/2289772386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d080ffb00 0x7f7d080fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/2289772386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d08100cf0 0x7f7d08101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 -- 172.21.15.62:0/2289772386 >> 172.21.15.62:0/2289772386 conn(0x7f7d080fb0e0 msgr2=0x7f7d080fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 -- 172.21.15.62:0/2289772386 shutdown_connections 2023-07-14T16:28:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.435+0000 7f7d0ebff700 1 -- 172.21.15.62:0/2289772386 wait complete. 2023-07-14T16:28:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.436+0000 7f7d0ebff700 1 Processor -- start 2023-07-14T16:28:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0ebff700 1 -- start start 2023-07-14T16:28:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0ebff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d080ffb00 0x7f7d081959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0ebff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d08100cf0 0x7f7d08195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0ebff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 0x7f7d08196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0ebff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d081973d0 con 0x7f7d08101680 2023-07-14T16:28:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0ebff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08197540 con 0x7f7d08100cf0 2023-07-14T16:28:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0ebff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08197700 con 0x7f7d080ffb00 2023-07-14T16:28:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0d19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 0x7f7d08196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0d19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 0x7f7d08196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46436/0 (socket says 172.21.15.62:46436) 2023-07-14T16:28:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0d19c700 1 -- 172.21.15.62:0/1929182233 learned_addr learned my addr 172.21.15.62:0/1929182233 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d07fff700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d08100cf0 0x7f7d08195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.437+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d080ffb00 0x7f7d081959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d0d19c700 1 -- 172.21.15.62:0/1929182233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d080ffb00 msgr2=0x7f7d081959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d080ffb00 0x7f7d081959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d0d19c700 1 -- 172.21.15.62:0/1929182233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d08100cf0 msgr2=0x7f7d08195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d08100cf0 0x7f7d08195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d0d19c700 1 -- 172.21.15.62:0/1929182233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00009d00 con 0x7f7d08101680 2023-07-14T16:28:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 0x7f7d08196d70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009fd0 tx=0x7f7d0000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d05ffb700 1 -- 172.21.15.62:0/1929182233 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000e040 con 0x7f7d08101680 2023-07-14T16:28:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d05ffb700 1 -- 172.21.15.62:0/1929182233 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d00003c00 con 0x7f7d08101680 2023-07-14T16:28:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d0ebff700 1 -- 172.21.15.62:0/1929182233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d08197980 con 0x7f7d08101680 2023-07-14T16:28:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.438+0000 7f7d05ffb700 1 -- 172.21.15.62:0/1929182233 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0001b6c0 con 0x7f7d08101680 2023-07-14T16:28:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.439+0000 7f7d0ebff700 1 -- 172.21.15.62:0/1929182233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0819e1a0 con 0x7f7d08101680 2023-07-14T16:28:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.443+0000 7f7d05ffb700 1 -- 172.21.15.62:0/1929182233 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7d0001d470 con 0x7f7d08101680 2023-07-14T16:28:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.443+0000 7f7d22481b80 1 Processor -- start 2023-07-14T16:28:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.444+0000 7f7d22481b80 1 -- start start 2023-07-14T16:28:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.444+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f0a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.444+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f0b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.444+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f0bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.444+0000 7f7d22481b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864e0fc50 con 0x563864f0a840 2023-07-14T16:28:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.444+0000 7f7d22481b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864e629e0 con 0x563864f0b6e0 2023-07-14T16:28:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.444+0000 7f7d22481b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864e5be70 con 0x563864d666b0 2023-07-14T16:28:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.444+0000 7f7d0c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f0b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f0bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f0bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53126/0 (socket says 172.21.15.62:53126) 2023-07-14T16:28:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 -- 172.21.15.62:0/1387528464 learned_addr learned my addr 172.21.15.62:0/1387528464 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 --2- 172.21.15.62:0/1387528464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f0a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 -- 172.21.15.62:0/1387528464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 msgr2=0x563864f0a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 --2- 172.21.15.62:0/1387528464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f0a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 -- 172.21.15.62:0/1387528464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 msgr2=0x563864f0b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 --2- 172.21.15.62:0/1387528464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f0b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 -- 172.21.15.62:0/1387528464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563864c89d80 con 0x563864f0b6e0 2023-07-14T16:28:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d05ffb700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7cf80a1150 0x7f7cf80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d05ffb700 1 -- 172.21.15.62:0/1929182233 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d000cdb50 con 0x7f7d08101680 2023-07-14T16:28:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.445+0000 7f7d07fff700 1 --2- 172.21.15.62:0/1387528464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f0bb80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40118b0 tx=0x7f7cf4009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=340a46d84d39fcc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.446+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7cf80a1150 0x7f7cf80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.446+0000 7f7cf2ffd700 1 -- 172.21.15.62:0/1387528464 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4004030 con 0x563864f0b6e0 2023-07-14T16:28:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.446+0000 7f7cf2ffd700 1 -- 172.21.15.62:0/1387528464 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cf4004830 con 0x563864f0b6e0 2023-07-14T16:28:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.446+0000 7f7cf2ffd700 1 -- 172.21.15.62:0/1387528464 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4005280 con 0x563864f0b6e0 2023-07-14T16:28:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.446+0000 7f7d22481b80 1 -- 172.21.15.62:0/1387528464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 msgr2=0x563864f0bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.446+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7cf80a1150 0x7f7cf80a3600 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7d08100b50 tx=0x7f7cfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.446+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1387528464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f0bb80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40118b0 tx=0x7f7cf4009880 comp rx=0 tx=0).stop 2023-07-14T16:28:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.447+0000 7f7d22481b80 1 -- 172.21.15.62:0/1387528464 shutdown_connections 2023-07-14T16:28:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.447+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1387528464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f0bb80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.447+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1387528464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.447+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1387528464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f0b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.447+0000 7f7d22481b80 1 -- 172.21.15.62:0/1387528464 >> 172.21.15.62:0/1387528464 conn(0x563864effbe0 msgr2=0x563864f04cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.447+0000 7f7d22481b80 1 -- 172.21.15.62:0/1387528464 shutdown_connections 2023-07-14T16:28:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.447+0000 7f7d22481b80 1 -- 172.21.15.62:0/1387528464 wait complete. 2023-07-14T16:28:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.447+0000 7f7d22481b80 1 -- start start 2023-07-14T16:28:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.448+0000 7f7d22481b80 1 Processor -- start 2023-07-14T16:28:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.448+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f17cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.448+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0a840 0x563864f18230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.448+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0b6e0 0x563864f1f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.448+0000 7f7d22481b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864dfba60 con 0x563864f0b6e0 2023-07-14T16:28:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.448+0000 7f7d22481b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864dfb760 con 0x563864f0a840 2023-07-14T16:28:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.448+0000 7f7d22481b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864cdf8c0 con 0x563864d666b0 2023-07-14T16:28:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.448+0000 7f7d22481b80 10 client.0 _ll_register_callbacks cb 0x563864df9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0b6e0 0x563864f1f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0c99b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f17cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0b6e0 0x563864f1f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46450/0 (socket says 172.21.15.62:46450) 2023-07-14T16:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 -- 172.21.15.62:0/3878918776 learned_addr learned my addr 172.21.15.62:0/3878918776 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d07fff700 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0a840 0x563864f18230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 -- 172.21.15.62:0/3878918776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 msgr2=0x563864f17cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f17cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 -- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0a840 msgr2=0x563864f18230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0a840 0x563864f18230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 -- 172.21.15.62:0/3878918776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf4019040 con 0x563864f0b6e0 2023-07-14T16:28:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d07fff700 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0a840 0x563864f18230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0b6e0 0x563864f1f270 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7d0009af90 tx=0x7f7d0009aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa2a267de20995a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.449+0000 7f7ce7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7ce7fff700 1 -- 172.21.15.62:0/3878918776 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0009b460 con 0x563864f0b6e0 2023-07-14T16:28:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7ce7fff700 1 -- 172.21.15.62:0/3878918776 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7d000a33c0 con 0x563864f0b6e0 2023-07-14T16:28:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7d22481b80 1 -- 172.21.15.62:0/3878918776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563864db5490 con 0x563864f0b6e0 2023-07-14T16:28:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7ce7fff700 1 -- 172.21.15.62:0/3878918776 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d0009f090 con 0x563864f0b6e0 2023-07-14T16:28:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7d22481b80 10 client.19071 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7d22481b80 1 -- 172.21.15.62:0/3878918776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563864c72c00 con 0x563864f0b6e0 2023-07-14T16:28:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7ce7fff700 1 -- 172.21.15.62:0/3878918776 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d000b9030 con 0x563864f0b6e0 2023-07-14T16:28:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7ce7fff700 1 -- 172.21.15.62:0/3878918776 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7d000a2f30 con 0x563864f0b6e0 2023-07-14T16:28:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7d22481b80 10 client.19071 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.450+0000 7f7d22481b80 10 client.19071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.451+0000 7f7d22481b80 20 client.19071 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.451+0000 7f7d22481b80 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563864f23890 0x563864f25ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.451+0000 7f7d22481b80 4 client.19071 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.451+0000 7f7d22481b80 1 -- 172.21.15.62:0/3878918776 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563864e54e50 con 0x563864f23890 2023-07-14T16:28:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.451+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563864f23890 0x563864f25ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.452+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563864f23890 0x563864f25ca0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01a2629891a6f7 server_cookie=a28d8adaeafaa696 in_seq=0 out_seq=0 2023-07-14T16:28:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.452+0000 7f7ce7fff700 10 client.19071 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7ce7fff700 1 -- 172.21.15.62:0/3878918776 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563864e54e50 con 0x563864f23890 2023-07-14T16:28:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7ce7fff700 10 client.19071 handle_command_reply: tid=0 2023-07-14T16:28:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 client.19071 shutdown 2023-07-14T16:28:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 -- 172.21.15.62:0/3878918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0b6e0 msgr2=0x563864f1f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0b6e0 0x563864f1f270 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7d0009af90 tx=0x7f7d0009aa10 comp rx=0 tx=0).stop 2023-07-14T16:28:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 -- 172.21.15.62:0/3878918776 shutdown_connections 2023-07-14T16:28:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0b6e0 0x563864f1f270 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f17cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563864f23890 0x563864f25ca0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 --2- 172.21.15.62:0/3878918776 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0a840 0x563864f18230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 -- 172.21.15.62:0/3878918776 >> 172.21.15.62:0/3878918776 conn(0x563864effbe0 msgr2=0x563864f044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7ce7fff700 0 client.19071 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 -- 172.21.15.62:0/3878918776 shutdown_connections 2023-07-14T16:28:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 1 -- 172.21.15.62:0/3878918776 wait complete. 2023-07-14T16:28:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.453+0000 7f7d22481b80 20 client.19071 trim_cache size 0 max 16384 2023-07-14T16:28:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.471+0000 7f7d22481b80 1 Processor -- start 2023-07-14T16:28:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.471+0000 7f7d22481b80 1 -- start start 2023-07-14T16:28:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.471+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.471+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f1f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.471+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f11a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.471+0000 7f7d22481b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864e54e50 con 0x563864f0a840 2023-07-14T16:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.471+0000 7f7d22481b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864ed0060 con 0x563864f0b6e0 2023-07-14T16:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.471+0000 7f7d22481b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864f12d90 con 0x563864d666b0 2023-07-14T16:28:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f1f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d07fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f1f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46466/0 (socket says 172.21.15.62:46466) 2023-07-14T16:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d07fff700 1 -- 172.21.15.62:0/2043954967 learned_addr learned my addr 172.21.15.62:0/2043954967 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f11a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d07fff700 1 -- 172.21.15.62:0/2043954967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 msgr2=0x563864f125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d07fff700 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d07fff700 1 -- 172.21.15.62:0/2043954967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 msgr2=0x563864f11a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d07fff700 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f11a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d07fff700 1 -- 172.21.15.62:0/2043954967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563864cb01f0 con 0x563864f0a840 2023-07-14T16:28:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.472+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f11a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.473+0000 7f7d07fff700 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f1f230 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40046b0 tx=0x7f7cf40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939377559eb3a222 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.473+0000 7f7cf37fe700 1 -- 172.21.15.62:0/2043954967 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4025040 con 0x563864f0a840 2023-07-14T16:28:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.473+0000 7f7cf37fe700 1 -- 172.21.15.62:0/2043954967 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cf4009c90 con 0x563864f0a840 2023-07-14T16:28:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.473+0000 7f7cf37fe700 1 -- 172.21.15.62:0/2043954967 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400e620 con 0x563864f0a840 2023-07-14T16:28:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 -- 172.21.15.62:0/2043954967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 msgr2=0x563864f1f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f1f230 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40046b0 tx=0x7f7cf40092e0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 -- 172.21.15.62:0/2043954967 shutdown_connections 2023-07-14T16:28:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f11a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864d666b0 0x563864f125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 --2- 172.21.15.62:0/2043954967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864f0a840 0x563864f1f230 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 -- 172.21.15.62:0/2043954967 >> 172.21.15.62:0/2043954967 conn(0x563864effbe0 msgr2=0x563864f2a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 -- 172.21.15.62:0/2043954967 shutdown_connections 2023-07-14T16:28:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 -- 172.21.15.62:0/2043954967 wait complete. 2023-07-14T16:28:10.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 -- start start 2023-07-14T16:28:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.474+0000 7f7d22481b80 1 Processor -- start 2023-07-14T16:28:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864d666b0 0x563864f1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864f0a840 0x563864f1b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d22481b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f1e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d22481b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864f1e6e0 con 0x563864d666b0 2023-07-14T16:28:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d22481b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864f1e850 con 0x563864f0b6e0 2023-07-14T16:28:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d22481b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563864f1ea10 con 0x563864f0a840 2023-07-14T16:28:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d22481b80 10 client.0 _ll_register_callbacks cb 0x7f7d0009add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d0c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864d666b0 0x563864f1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d0c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864d666b0 0x563864f1ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46476/0 (socket says 172.21.15.62:46476) 2023-07-14T16:28:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d0c99b700 1 -- 172.21.15.62:0/1794864979 learned_addr learned my addr 172.21.15.62:0/1794864979 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.475+0000 7f7d07fff700 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864f0a840 0x563864f1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f1e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d0c99b700 1 -- 172.21.15.62:0/1794864979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864f0a840 msgr2=0x563864f1b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864f0a840 0x563864f1b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d0c99b700 1 -- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 msgr2=0x563864f1e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f1e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d0c99b700 1 -- 172.21.15.62:0/1794864979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563864c72c00 con 0x563864d666b0 2023-07-14T16:28:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d0d19c700 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f1e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d0c99b700 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864d666b0 0x563864f1ac40 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x563864f26550 tx=0x7f7cfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230c7bea050536e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7ce7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7ce7fff700 1 -- 172.21.15.62:0/1794864979 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc025040 con 0x563864d666b0 2023-07-14T16:28:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7d22481b80 1 -- 172.21.15.62:0/1794864979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cf4004830 con 0x563864d666b0 2023-07-14T16:28:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.476+0000 7f7ce7fff700 1 -- 172.21.15.62:0/1794864979 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7cfc02a950 con 0x563864d666b0 2023-07-14T16:28:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7ce7fff700 1 -- 172.21.15.62:0/1794864979 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cfc017070 con 0x563864d666b0 2023-07-14T16:28:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7d22481b80 10 client.19077 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7d22481b80 1 -- 172.21.15.62:0/1794864979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563864c89d80 con 0x563864d666b0 2023-07-14T16:28:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7ce7fff700 1 -- 172.21.15.62:0/1794864979 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cf4004830 con 0x563864d666b0 2023-07-14T16:28:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7ce7fff700 1 -- 172.21.15.62:0/1794864979 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7cfc003ce0 con 0x563864d666b0 2023-07-14T16:28:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7d22481b80 10 client.19077 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7d22481b80 10 client.19077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7d22481b80 20 client.19077 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563864f783f0 0x563864f7a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7d22481b80 4 client.19077 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.477+0000 7f7d22481b80 1 -- 172.21.15.62:0/1794864979 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563864e16ff0 con 0x563864f783f0 2023-07-14T16:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.478+0000 7f7d07fff700 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563864f783f0 0x563864f7a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.478+0000 7f7d07fff700 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563864f783f0 0x563864f7a800 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284eb0d5eb3d4c81 server_cookie=d3291e9a148f0f9a in_seq=0 out_seq=0 2023-07-14T16:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.478+0000 7f7ce7fff700 10 client.19077 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.488+0000 7f7ce7fff700 1 -- 172.21.15.62:0/1794864979 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563864e16ff0 con 0x563864f783f0 2023-07-14T16:28:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.488+0000 7f7ce7fff700 10 client.19077 handle_command_reply: tid=0 2023-07-14T16:28:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.488+0000 7f7d22481b80 1 client.19077 shutdown 2023-07-14T16:28:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 -- 172.21.15.62:0/1794864979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864d666b0 msgr2=0x563864f1ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864d666b0 0x563864f1ac40 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x563864f26550 tx=0x7f7cfc004750 comp rx=0 tx=0).stop 2023-07-14T16:28:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 -- 172.21.15.62:0/1794864979 shutdown_connections 2023-07-14T16:28:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563864f783f0 0x563864f7a800 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563864f0b6e0 0x563864f1e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563864d666b0 0x563864f1ac40 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 --2- 172.21.15.62:0/1794864979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563864f0a840 0x563864f1b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 -- 172.21.15.62:0/1794864979 >> 172.21.15.62:0/1794864979 conn(0x563864effbe0 msgr2=0x563864f125d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7ce7fff700 0 client.19077 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 -- 172.21.15.62:0/1794864979 shutdown_connections 2023-07-14T16:28:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 1 -- 172.21.15.62:0/1794864979 wait complete. 2023-07-14T16:28:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.489+0000 7f7d22481b80 20 client.19077 trim_cache size 0 max 16384 2023-07-14T16:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:28:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 -- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7cf80a1150 msgr2=0x7f7cf80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7cf80a1150 0x7f7cf80a3600 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7d08100b50 tx=0x7f7cfc009260 comp rx=0 tx=0).stop 2023-07-14T16:28:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 -- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 msgr2=0x7f7d08196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 0x7f7d08196d70 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009fd0 tx=0x7f7d0000ba50 comp rx=0 tx=0).stop 2023-07-14T16:28:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 -- 172.21.15.62:0/1929182233 shutdown_connections 2023-07-14T16:28:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d08101680 0x7f7d08196d70 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d080ffb00 0x7f7d081959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7cf80a1150 0x7f7cf80a3600 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 --2- 172.21.15.62:0/1929182233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d08100cf0 0x7f7d08195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 -- 172.21.15.62:0/1929182233 >> 172.21.15.62:0/1929182233 conn(0x7f7d080fb0e0 msgr2=0x7f7d08104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 -- 172.21.15.62:0/1929182233 shutdown_connections 2023-07-14T16:28:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:10.490+0000 7f7d0ebff700 1 -- 172.21.15.62:0/1929182233 wait complete. 2023-07-14T16:28:11.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:10 smithi062 ceph-mon[95543]: pgmap v733: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:28:11.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:10 smithi081 ceph-mon[127795]: pgmap v733: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:28:11.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:10 smithi164 ceph-mon[97427]: pgmap v733: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:28:13.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:12 smithi062 ceph-mon[95543]: pgmap v734: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:28:13.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:12 smithi081 ceph-mon[127795]: pgmap v734: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:28:13.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:12 smithi164 ceph-mon[97427]: pgmap v734: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:28:14.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:13 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:28:14.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:13 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:28:14.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:13 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:28:14.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:13 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:28:14.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:13 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:28:14.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:13 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:28:15.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:14 smithi062 ceph-mon[95543]: pgmap v735: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:28:15.152 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:14 smithi081 ceph-mon[127795]: pgmap v735: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:28:15.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:14 smithi164 ceph-mon[97427]: pgmap v735: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 37 op/s 2023-07-14T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:16 smithi062 ceph-mon[95543]: pgmap v736: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:28:17.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:16 smithi081 ceph-mon[127795]: pgmap v736: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:28:17.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:16 smithi164 ceph-mon[97427]: pgmap v736: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:28:19.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:18 smithi062 ceph-mon[95543]: pgmap v737: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:28:19.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:18 smithi081 ceph-mon[127795]: pgmap v737: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:28:19.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:18 smithi164 ceph-mon[97427]: pgmap v737: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 35 op/s 2023-07-14T16:28:21.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:20 smithi062 ceph-mon[95543]: pgmap v738: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:28:21.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:20 smithi081 ceph-mon[127795]: pgmap v738: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:28:21.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:20 smithi164 ceph-mon[97427]: pgmap v738: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:28:23.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:22 smithi062 ceph-mon[95543]: pgmap v739: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 24 op/s 2023-07-14T16:28:23.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:22 smithi081 ceph-mon[127795]: pgmap v739: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 24 op/s 2023-07-14T16:28:23.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:22 smithi164 ceph-mon[97427]: pgmap v739: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 24 op/s 2023-07-14T16:28:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2669377850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f01038c0 msgr2=0x7fa8f010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2669377850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f01038c0 0x7fa8f010a180 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800b0a0 tx=0x7fa8e800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2669377850 shutdown_connections 2023-07-14T16:28:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2669377850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f01038c0 0x7fa8f010a180 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2669377850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8f0101d40 0x7fa8f0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2669377850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8f0102f30 0x7fa8f0103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2669377850 >> 172.21.15.62:0/2669377850 conn(0x7fa8f00fd360 msgr2=0x7fa8f00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2669377850 shutdown_connections 2023-07-14T16:28:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.660+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2669377850 wait complete. 2023-07-14T16:28:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.661+0000 7fa8f55ff700 1 Processor -- start 2023-07-14T16:28:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8f55ff700 1 -- start start 2023-07-14T16:28:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8f55ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f0101d40 0x7fa8f0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8f55ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8f0102f30 0x7fa8f0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8f55ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8f01038c0 0x7fa8f0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8f55ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0199560 con 0x7fa8f0101d40 2023-07-14T16:28:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8f55ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01996d0 con 0x7fa8f01038c0 2023-07-14T16:28:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8f55ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0199890 con 0x7fa8f0102f30 2023-07-14T16:28:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8eeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f0101d40 0x7fa8f0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8eeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f0101d40 0x7fa8f0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38162/0 (socket says 172.21.15.62:38162) 2023-07-14T16:28:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.662+0000 7fa8eeffd700 1 -- 172.21.15.62:0/2453970613 learned_addr learned my addr 172.21.15.62:0/2453970613 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8ef7fe700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8f01038c0 0x7fa8f0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8f0102f30 0x7fa8f0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8eeffd700 1 -- 172.21.15.62:0/2453970613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8f0102f30 msgr2=0x7fa8f0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8f0102f30 0x7fa8f0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8eeffd700 1 -- 172.21.15.62:0/2453970613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8f01038c0 msgr2=0x7fa8f0198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8f01038c0 0x7fa8f0198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8eeffd700 1 -- 172.21.15.62:0/2453970613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e8009d00 con 0x7fa8f0101d40 2023-07-14T16:28:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f0101d40 0x7fa8f0197b40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000e970 tx=0x7fa8e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.663+0000 7fa8d7fff700 1 -- 172.21.15.62:0/2453970613 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0009790 con 0x7fa8f0101d40 2023-07-14T16:28:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.664+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2453970613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f0199b70 con 0x7fa8f0101d40 2023-07-14T16:28:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.664+0000 7fa8d7fff700 1 -- 172.21.15.62:0/2453970613 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8e000c7e0 con 0x7fa8f0101d40 2023-07-14T16:28:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.664+0000 7fa8d7fff700 1 -- 172.21.15.62:0/2453970613 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0003bd0 con 0x7fa8f0101d40 2023-07-14T16:28:23.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.664+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2453970613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f01a0330 con 0x7fa8f0101d40 2023-07-14T16:28:23.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.668+0000 7fa8d7fff700 1 -- 172.21.15.62:0/2453970613 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa8e000c950 con 0x7fa8f0101d40 2023-07-14T16:28:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.669+0000 7fa90900ab80 1 Processor -- start 2023-07-14T16:28:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.669+0000 7fa90900ab80 1 -- start start 2023-07-14T16:28:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.669+0000 7fa8d7fff700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a1150 0x7fa8d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.669+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ec3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.669+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec3840 0x55c4c0ec41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ec4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa90900ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0dc8c50 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa90900ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0e1b9e0 con 0x55c4c0ec46e0 2023-07-14T16:28:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8d7fff700 1 -- 172.21.15.62:0/2453970613 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8e0014070 con 0x7fa8f0101d40 2023-07-14T16:28:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa90900ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0e14e70 con 0x55c4c0ec3840 2023-07-14T16:28:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8ef7fe700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a1150 0x7fa8d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8eeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ec3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8eeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ec3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38170/0 (socket says 172.21.15.62:38170) 2023-07-14T16:28:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8ef7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ec4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8eeffd700 1 -- 172.21.15.62:0/1006790195 learned_addr learned my addr 172.21.15.62:0/1006790195 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec3840 0x55c4c0ec41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8eeffd700 1 -- 172.21.15.62:0/1006790195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec3840 msgr2=0x55c4c0ec41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec3840 0x55c4c0ec41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8eeffd700 1 -- 172.21.15.62:0/1006790195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 msgr2=0x55c4c0ec4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ec4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.670+0000 7fa8eeffd700 1 -- 172.21.15.62:0/1006790195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4c0c42d80 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.671+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec3840 0x55c4c0ec41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.671+0000 7fa8ef7fe700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a1150 0x7fa8d80a3600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800b780 tx=0x7fa8e80076e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.671+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ec3300 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00a1e30 tx=0x7fa8e0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4546e089cb6491e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.671+0000 7fa8d57fa700 1 -- 172.21.15.62:0/1006790195 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0051500 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.671+0000 7fa8d57fa700 1 -- 172.21.15.62:0/1006790195 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8e000cc50 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.671+0000 7fa8d57fa700 1 -- 172.21.15.62:0/1006790195 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00ca7e0 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 -- 172.21.15.62:0/1006790195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0d1f6b0 msgr2=0x55c4c0ec3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ec3300 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00a1e30 tx=0x7fa8e0003680 comp rx=0 tx=0).stop 2023-07-14T16:28:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 -- 172.21.15.62:0/1006790195 shutdown_connections 2023-07-14T16:28:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ec4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ec3300 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1006790195 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec3840 0x55c4c0ec41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 -- 172.21.15.62:0/1006790195 >> 172.21.15.62:0/1006790195 conn(0x55c4c0eb8be0 msgr2=0x55c4c0ebdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 -- 172.21.15.62:0/1006790195 shutdown_connections 2023-07-14T16:28:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.672+0000 7fa90900ab80 1 -- 172.21.15.62:0/1006790195 wait complete. 2023-07-14T16:28:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 1 -- start start 2023-07-14T16:28:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 1 Processor -- start 2023-07-14T16:28:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0eccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ed3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0db4a60 con 0x55c4c0ec3840 2023-07-14T16:28:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0db4760 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0c988c0 con 0x55c4c0ec46e0 2023-07-14T16:28:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.673+0000 7fa90900ab80 10 client.0 _ll_register_callbacks cb 0x55c4c0db2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8eeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8ee7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0eccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8eeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:39780/0 (socket says 172.21.15.62:39780) 2023-07-14T16:28:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8eeffd700 1 -- 172.21.15.62:0/3537347355 learned_addr learned my addr 172.21.15.62:0/3537347355 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8ef7fe700 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ed3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8eeffd700 1 -- 172.21.15.62:0/3537347355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 msgr2=0x55c4c0ed3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ed3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8eeffd700 1 -- 172.21.15.62:0/3537347355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 msgr2=0x55c4c0eccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0eccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8eeffd700 1 -- 172.21.15.62:0/3537347355 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f01a0330 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.674+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0eccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecc890 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009230 tx=0x7fa8e0099ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e308c6f61e5d2020 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa8d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:28:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/3537347355 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00518f0 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/3537347355 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8e009a170 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa90900ab80 1 -- 172.21.15.62:0/3537347355 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4c0d6e490 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/3537347355 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8e009a370 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa90900ab80 10 client.28679 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa90900ab80 1 -- 172.21.15.62:0/3537347355 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c4c0c2bc00 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.675+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/3537347355 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c4c0d6e490 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.676+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/3537347355 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa8e004ef40 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.676+0000 7fa90900ab80 10 client.28679 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.676+0000 7fa90900ab80 10 client.28679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.676+0000 7fa90900ab80 20 client.28679 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.676+0000 7fa90900ab80 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c4c0ed81f0 0x55c4c0eda600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.676+0000 7fa90900ab80 4 client.28679 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.676+0000 7fa90900ab80 1 -- 172.21.15.62:0/3537347355 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4c0e0de50 con 0x55c4c0ed81f0 2023-07-14T16:28:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.676+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c4c0ed81f0 0x55c4c0eda600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.677+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c4c0ed81f0 0x55c4c0eda600 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9aed15801ac4d46 server_cookie=a8b41276e982f5a3 in_seq=0 out_seq=0 2023-07-14T16:28:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.677+0000 7fa8d2ff5700 10 client.28679 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/3537347355 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c4c0e0de50 con 0x55c4c0ed81f0 2023-07-14T16:28:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa8d2ff5700 10 client.28679 handle_command_reply: tid=0 2023-07-14T16:28:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 client.28679 shutdown 2023-07-14T16:28:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 -- 172.21.15.62:0/3537347355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 msgr2=0x55c4c0ecc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecc890 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009230 tx=0x7fa8e0099ce0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 -- 172.21.15.62:0/3537347355 shutdown_connections 2023-07-14T16:28:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ed3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c4c0ed81f0 0x55c4c0eda600 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecc890 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 --2- 172.21.15.62:0/3537347355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0eccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 -- 172.21.15.62:0/3537347355 >> 172.21.15.62:0/3537347355 conn(0x55c4c0eb8be0 msgr2=0x55c4c0ebd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa8d2ff5700 0 client.28679 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 -- 172.21.15.62:0/3537347355 shutdown_connections 2023-07-14T16:28:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.678+0000 7fa90900ab80 1 -- 172.21.15.62:0/3537347355 wait complete. 2023-07-14T16:28:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.679+0000 7fa90900ab80 20 client.28679 trim_cache size 0 max 16384 2023-07-14T16:28:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.703+0000 7fa90900ab80 1 Processor -- start 2023-07-14T16:28:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.703+0000 7fa90900ab80 1 -- start start 2023-07-14T16:28:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.703+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.703+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.703+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ecab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.703+0000 7fa90900ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0ecc460 con 0x55c4c0ec3840 2023-07-14T16:28:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.703+0000 7fa90900ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0ecda80 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.703+0000 7fa90900ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0ecd8e0 con 0x55c4c0ec46e0 2023-07-14T16:28:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ee7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ee7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38194/0 (socket says 172.21.15.62:38194) 2023-07-14T16:28:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ee7fc700 1 -- 172.21.15.62:0/775720872 learned_addr learned my addr 172.21.15.62:0/775720872 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ef7fe700 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ecab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ee7fc700 1 -- 172.21.15.62:0/775720872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 msgr2=0x55c4c0ecab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ecab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ee7fc700 1 -- 172.21.15.62:0/775720872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 msgr2=0x55c4c0ecb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.704+0000 7fa8ee7fc700 1 -- 172.21.15.62:0/775720872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4c0c691f0 con 0x55c4c0ec3840 2023-07-14T16:28:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.705+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed3dd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x55c4c0edaeb0 tx=0x7fa8e401dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438f1755c04f0cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.705+0000 7fa8d5ffb700 1 -- 172.21.15.62:0/775720872 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4017040 con 0x55c4c0ec3840 2023-07-14T16:28:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.705+0000 7fa8d5ffb700 1 -- 172.21.15.62:0/775720872 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8e401d610 con 0x55c4c0ec3840 2023-07-14T16:28:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.705+0000 7fa8d5ffb700 1 -- 172.21.15.62:0/775720872 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4005310 con 0x55c4c0ec3840 2023-07-14T16:28:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 -- 172.21.15.62:0/775720872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 msgr2=0x55c4c0ed3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed3dd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x55c4c0edaeb0 tx=0x7fa8e401dd50 comp rx=0 tx=0).stop 2023-07-14T16:28:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 -- 172.21.15.62:0/775720872 shutdown_connections 2023-07-14T16:28:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ecab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ecb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 --2- 172.21.15.62:0/775720872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed3dd0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 -- 172.21.15.62:0/775720872 >> 172.21.15.62:0/775720872 conn(0x55c4c0eb8be0 msgr2=0x55c4c0edef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 -- 172.21.15.62:0/775720872 shutdown_connections 2023-07-14T16:28:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 -- 172.21.15.62:0/775720872 wait complete. 2023-07-14T16:28:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.706+0000 7fa90900ab80 1 -- start start 2023-07-14T16:28:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.707+0000 7fa90900ab80 1 Processor -- start 2023-07-14T16:28:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.707+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ed2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.707+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.707+0000 7fa90900ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ed2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.707+0000 7fa90900ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0ed3290 con 0x55c4c0ec3840 2023-07-14T16:28:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.707+0000 7fa90900ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0ed3400 con 0x55c4c0ec46e0 2023-07-14T16:28:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.707+0000 7fa90900ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4c0ed35c0 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.707+0000 7fa90900ab80 10 client.0 _ll_register_callbacks cb 0x7fa8e0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8eeffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ed2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8eeffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ed2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:59670/0 (socket says 172.21.15.62:59670) 2023-07-14T16:28:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8eeffd700 1 -- 172.21.15.62:0/1857399224 learned_addr learned my addr 172.21.15.62:0/1857399224 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8ef7fe700 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ed2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8eeffd700 1 -- 172.21.15.62:0/1857399224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 msgr2=0x55c4c0ed2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ed2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8eeffd700 1 -- 172.21.15.62:0/1857399224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 msgr2=0x55c4c0ed2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.708+0000 7fa8eeffd700 1 -- 172.21.15.62:0/1857399224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e40044e0 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa8eeffd700 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ed2010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0096410 tx=0x7fa8e0003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc7fc7ac605c50f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa8d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:28:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/1857399224 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0099600 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/1857399224 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8e00518f0 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa90900ab80 1 -- 172.21.15.62:0/1857399224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8e000cc50 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/1857399224 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8e0096070 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa90900ab80 10 client.28558 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa90900ab80 1 -- 172.21.15.62:0/1857399224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c4c0c41250 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.709+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/1857399224 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8e000cc50 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.710+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/1857399224 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa8e009a170 con 0x55c4c0d1f6b0 2023-07-14T16:28:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.710+0000 7fa90900ab80 10 client.28558 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.710+0000 7fa90900ab80 10 client.28558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.710+0000 7fa90900ab80 20 client.28558 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:23.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.710+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c4c0f2c780 0x55c4c0f2eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.710+0000 7fa90900ab80 4 client.28558 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:28:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.710+0000 7fa90900ab80 1 -- 172.21.15.62:0/1857399224 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c4c0dcfff0 con 0x55c4c0f2c780 2023-07-14T16:28:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.710+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c4c0f2c780 0x55c4c0f2eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.711+0000 7fa8ee7fc700 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c4c0f2c780 0x55c4c0f2eb60 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44aee2cc589d4004 server_cookie=5274715febae40ed in_seq=0 out_seq=0 2023-07-14T16:28:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.711+0000 7fa8d2ff5700 10 client.28558 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa8d2ff5700 1 -- 172.21.15.62:0/1857399224 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c4c0dcfff0 con 0x55c4c0f2c780 2023-07-14T16:28:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa8d2ff5700 10 client.28558 handle_command_reply: tid=0 2023-07-14T16:28:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 client.28558 shutdown 2023-07-14T16:28:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 -- 172.21.15.62:0/1857399224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0d1f6b0 msgr2=0x55c4c0ed2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ed2010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0096410 tx=0x7fa8e0003680 comp rx=0 tx=0).stop 2023-07-14T16:28:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 -- 172.21.15.62:0/1857399224 shutdown_connections 2023-07-14T16:28:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c4c0f2c780 0x55c4c0f2eb60 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4c0ec46e0 0x55c4c0ed2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4c0d1f6b0 0x55c4c0ed2010 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 --2- 172.21.15.62:0/1857399224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4c0ec3840 0x55c4c0ed2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa8d2ff5700 0 client.28558 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 -- 172.21.15.62:0/1857399224 >> 172.21.15.62:0/1857399224 conn(0x55c4c0eb8be0 msgr2=0x55c4c0ecb530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.720+0000 7fa90900ab80 1 -- 172.21.15.62:0/1857399224 shutdown_connections 2023-07-14T16:28:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa90900ab80 1 -- 172.21.15.62:0/1857399224 wait complete. 2023-07-14T16:28:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa90900ab80 20 client.28558 trim_cache size 0 max 16384 2023-07-14T16:28:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:28:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:28:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:28:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:28:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a1150 msgr2=0x7fa8d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a1150 0x7fa8d80a3600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800b780 tx=0x7fa8e80076e0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f0101d40 msgr2=0x7fa8f0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f0101d40 0x7fa8f0197b40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000e970 tx=0x7fa8e000ec80 comp rx=0 tx=0).stop 2023-07-14T16:28:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2453970613 shutdown_connections 2023-07-14T16:28:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa8d80a1150 0x7fa8d80a3600 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8f01038c0 0x7fa8f0198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8f0101d40 0x7fa8f0197b40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 --2- 172.21.15.62:0/2453970613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa8f0102f30 0x7fa8f0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.721+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2453970613 >> 172.21.15.62:0/2453970613 conn(0x7fa8f00fd360 msgr2=0x7fa8f01063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.722+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2453970613 shutdown_connections 2023-07-14T16:28:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:23.722+0000 7fa8f55ff700 1 -- 172.21.15.62:0/2453970613 wait complete. 2023-07-14T16:28:25.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:24 smithi062 ceph-mon[95543]: pgmap v740: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:28:25.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:24 smithi081 ceph-mon[127795]: pgmap v740: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:28:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:24 smithi164 ceph-mon[97427]: pgmap v740: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:28:27.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:26 smithi062 ceph-mon[95543]: pgmap v741: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:28:27.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:26 smithi081 ceph-mon[127795]: pgmap v741: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:28:27.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:26 smithi164 ceph-mon[97427]: pgmap v741: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:28:29.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:28 smithi062 ceph-mon[95543]: pgmap v742: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:28:29.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:28 smithi081 ceph-mon[127795]: pgmap v742: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:28:29.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:28 smithi164 ceph-mon[97427]: pgmap v742: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:28:31.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:30 smithi062 ceph-mon[95543]: pgmap v743: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:28:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:30 smithi081 ceph-mon[127795]: pgmap v743: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:28:31.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:30 smithi164 ceph-mon[97427]: pgmap v743: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:28:33.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:32 smithi062 ceph-mon[95543]: pgmap v744: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:28:33.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:32 smithi081 ceph-mon[127795]: pgmap v744: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:28:33.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:32 smithi164 ceph-mon[97427]: pgmap v744: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:28:34.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:33 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:28:34.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:33 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:28:34.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:33 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:28:35.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:34 smithi062 ceph-mon[95543]: pgmap v745: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:28:35.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:28:35.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:28:35.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:28:35.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:34 smithi081 ceph-mon[127795]: pgmap v745: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:28:35.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:28:35.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:28:35.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:28:35.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:34 smithi164 ceph-mon[97427]: pgmap v745: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:28:35.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:28:35.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:28:35.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:28:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.885+0000 7f64261ff700 1 -- 172.21.15.62:0/3392344364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420074ed0 msgr2=0x7f64200752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.885+0000 7f64261ff700 1 --2- 172.21.15.62:0/3392344364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420074ed0 0x7f64200752e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f6410009990 tx=0x7f6410009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.885+0000 7f64261ff700 1 -- 172.21.15.62:0/3392344364 shutdown_connections 2023-07-14T16:28:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.885+0000 7f64261ff700 1 --2- 172.21.15.62:0/3392344364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6420076ac0 0x7f6420077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.885+0000 7f64261ff700 1 --2- 172.21.15.62:0/3392344364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420074ed0 0x7f64200752e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.885+0000 7f64261ff700 1 --2- 172.21.15.62:0/3392344364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420076110 0x7f6420076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.885+0000 7f64261ff700 1 -- 172.21.15.62:0/3392344364 >> 172.21.15.62:0/3392344364 conn(0x7f64200fd320 msgr2=0x7f64200ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.886+0000 7f64261ff700 1 -- 172.21.15.62:0/3392344364 shutdown_connections 2023-07-14T16:28:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.886+0000 7f64261ff700 1 -- 172.21.15.62:0/3392344364 wait complete. 2023-07-14T16:28:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.887+0000 7f64261ff700 1 Processor -- start 2023-07-14T16:28:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.887+0000 7f64261ff700 1 -- start start 2023-07-14T16:28:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.887+0000 7f64261ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420074ed0 0x7f642019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.887+0000 7f64261ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6420076110 0x7f642019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.887+0000 7f64261ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420076ac0 0x7f642019d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.887+0000 7f64261ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642019d840 con 0x7f6420076ac0 2023-07-14T16:28:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.887+0000 7f64261ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642019d9b0 con 0x7f6420074ed0 2023-07-14T16:28:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f64261ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642019db70 con 0x7f6420076110 2023-07-14T16:28:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420076ac0 0x7f642019d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6420076110 0x7f642019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6420076110 0x7f642019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55266/0 (socket says 172.21.15.62:55266) 2023-07-14T16:28:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641effd700 1 -- 172.21.15.62:0/3437416643 learned_addr learned my addr 172.21.15.62:0/3437416643 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420074ed0 0x7f642019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641ffff700 1 -- 172.21.15.62:0/3437416643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6420076110 msgr2=0x7f642019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641ffff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6420076110 0x7f642019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641ffff700 1 -- 172.21.15.62:0/3437416643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420074ed0 msgr2=0x7f642019bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641ffff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420074ed0 0x7f642019bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.888+0000 7f641ffff700 1 -- 172.21.15.62:0/3437416643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64100095f0 con 0x7f6420076ac0 2023-07-14T16:28:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.889+0000 7f641ffff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420076ac0 0x7f642019d140 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f6418007f10 tx=0x7f641800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.889+0000 7f641cff9700 1 -- 172.21.15.62:0/3437416643 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418009ea0 con 0x7f6420076ac0 2023-07-14T16:28:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.889+0000 7f64261ff700 1 -- 172.21.15.62:0/3437416643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642019de50 con 0x7f6420076ac0 2023-07-14T16:28:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.889+0000 7f641cff9700 1 -- 172.21.15.62:0/3437416643 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f641800f040 con 0x7f6420076ac0 2023-07-14T16:28:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.889+0000 7f641cff9700 1 -- 172.21.15.62:0/3437416643 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418004280 con 0x7f6420076ac0 2023-07-14T16:28:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.889+0000 7f64261ff700 1 -- 172.21.15.62:0/3437416643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64201a45c0 con 0x7f6420076ac0 2023-07-14T16:28:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.893+0000 7f641cff9700 1 -- 172.21.15.62:0/3437416643 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f641800db20 con 0x7f6420076ac0 2023-07-14T16:28:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.894+0000 7f641cff9700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64080a11d0 0x7f64080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.894+0000 7f641cff9700 1 -- 172.21.15.62:0/3437416643 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f64180cc160 con 0x7f6420076ac0 2023-07-14T16:28:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f6439a6bb80 1 Processor -- start 2023-07-14T16:28:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64080a11d0 0x7f64080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f6439a6bb80 1 -- start start 2023-07-14T16:28:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9dae4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 0x559a9dae51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae56e0 0x559a9dae5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9d9e9c50 con 0x559a9d9406b0 2023-07-14T16:28:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9da3c9e0 con 0x559a9dae56e0 2023-07-14T16:28:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.895+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9da35e70 con 0x559a9dae4840 2023-07-14T16:28:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9dae4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64080a11d0 0x7f64080a3680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f641000b5c0 tx=0x7f6410005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9dae4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35496/0 (socket says 172.21.15.62:35496) 2023-07-14T16:28:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 -- 172.21.15.62:0/3800262834 learned_addr learned my addr 172.21.15.62:0/3800262834 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641effd700 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 0x559a9dae51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641ffff700 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae56e0 0x559a9dae5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 -- 172.21.15.62:0/3800262834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 msgr2=0x559a9dae51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 0x559a9dae51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 -- 172.21.15.62:0/3800262834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae56e0 msgr2=0x559a9dae5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae56e0 0x559a9dae5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 -- 172.21.15.62:0/3800262834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a9d863d80 con 0x559a9d9406b0 2023-07-14T16:28:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.896+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9dae4300 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f641000bb40 tx=0x7f6410020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3bbe9027c963d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.897+0000 7f6405ffb700 1 -- 172.21.15.62:0/3800262834 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64100056d0 con 0x559a9d9406b0 2023-07-14T16:28:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.897+0000 7f6405ffb700 1 -- 172.21.15.62:0/3800262834 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f641001e040 con 0x559a9d9406b0 2023-07-14T16:28:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.897+0000 7f6405ffb700 1 -- 172.21.15.62:0/3800262834 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410003b90 con 0x559a9d9406b0 2023-07-14T16:28:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3800262834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 msgr2=0x559a9dae4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9dae4300 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f641000bb40 tx=0x7f6410020000 comp rx=0 tx=0).stop 2023-07-14T16:28:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3800262834 shutdown_connections 2023-07-14T16:28:36.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae56e0 0x559a9dae5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9dae4300 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3800262834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 0x559a9dae51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3800262834 >> 172.21.15.62:0/3800262834 conn(0x559a9dad9be0 msgr2=0x559a9dadecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3800262834 shutdown_connections 2023-07-14T16:28:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.898+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3800262834 wait complete. 2023-07-14T16:28:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.899+0000 7f6439a6bb80 1 -- start start 2023-07-14T16:28:36.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.899+0000 7f6439a6bb80 1 Processor -- start 2023-07-14T16:28:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.899+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9d9406b0 0x559a9daed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.899+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 0x559a9daeddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.899+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daf4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9d9d5a60 con 0x559a9dae56e0 2023-07-14T16:28:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9d9d5760 con 0x559a9d9406b0 2023-07-14T16:28:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9d8b98c0 con 0x559a9dae4840 2023-07-14T16:28:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f6439a6bb80 10 client.0 _ll_register_callbacks cb 0x559a9d9d3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daf4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 0x559a9daeddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daf4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35498/0 (socket says 172.21.15.62:35498) 2023-07-14T16:28:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641ffff700 1 -- 172.21.15.62:0/2191925654 learned_addr learned my addr 172.21.15.62:0/2191925654 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641f7fe700 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9d9406b0 0x559a9daed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641ffff700 1 -- 172.21.15.62:0/2191925654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 msgr2=0x559a9daeddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641ffff700 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 0x559a9daeddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641ffff700 1 -- 172.21.15.62:0/2191925654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9d9406b0 msgr2=0x559a9daed870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641ffff700 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9d9406b0 0x559a9daed870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.900+0000 7f641ffff700 1 -- 172.21.15.62:0/2191925654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6420077c40 con 0x559a9dae56e0 2023-07-14T16:28:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f641ffff700 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daf4df0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6418003ab0 tx=0x7f6418099430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cf38db60022172 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f63feffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f63feffd700 1 -- 172.21.15.62:0/2191925654 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180cb040 con 0x559a9dae56e0 2023-07-14T16:28:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f63feffd700 1 -- 172.21.15.62:0/2191925654 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6418095040 con 0x559a9dae56e0 2023-07-14T16:28:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f6439a6bb80 1 -- 172.21.15.62:0/2191925654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a9d98f490 con 0x559a9dae56e0 2023-07-14T16:28:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f63feffd700 1 -- 172.21.15.62:0/2191925654 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64180b9070 con 0x559a9dae56e0 2023-07-14T16:28:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f6439a6bb80 10 client.19107 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f6439a6bb80 1 -- 172.21.15.62:0/2191925654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559a9d84cc00 con 0x559a9dae56e0 2023-07-14T16:28:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.901+0000 7f63feffd700 1 -- 172.21.15.62:0/2191925654 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559a9d98f490 con 0x559a9dae56e0 2023-07-14T16:28:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.902+0000 7f63feffd700 1 -- 172.21.15.62:0/2191925654 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6418098a70 con 0x559a9dae56e0 2023-07-14T16:28:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.902+0000 7f6439a6bb80 10 client.19107 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.902+0000 7f6439a6bb80 10 client.19107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.902+0000 7f6439a6bb80 20 client.19107 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:36.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.902+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a9daf9580 0x559a9dafb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.902+0000 7f6439a6bb80 4 client.19107 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.902+0000 7f6439a6bb80 1 -- 172.21.15.62:0/2191925654 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a9da2ee50 con 0x559a9daf9580 2023-07-14T16:28:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.902+0000 7f641f7fe700 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a9daf9580 0x559a9dafb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.903+0000 7f641f7fe700 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a9daf9580 0x559a9dafb990 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7818d497ebf8ff5f server_cookie=3de84c97c45c45e in_seq=0 out_seq=0 2023-07-14T16:28:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.903+0000 7f63feffd700 10 client.19107 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f63feffd700 1 -- 172.21.15.62:0/2191925654 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a9da2ee50 con 0x559a9daf9580 2023-07-14T16:28:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f63feffd700 10 client.19107 handle_command_reply: tid=0 2023-07-14T16:28:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 client.19107 shutdown 2023-07-14T16:28:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 -- 172.21.15.62:0/2191925654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 msgr2=0x559a9daf4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daf4df0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6418003ab0 tx=0x7f6418099430 comp rx=0 tx=0).stop 2023-07-14T16:28:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 -- 172.21.15.62:0/2191925654 shutdown_connections 2023-07-14T16:28:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daf4df0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a9daf9580 0x559a9dafb990 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9d9406b0 0x559a9daed870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/2191925654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae4840 0x559a9daeddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f63feffd700 0 client.19107 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 -- 172.21.15.62:0/2191925654 >> 172.21.15.62:0/2191925654 conn(0x559a9dad9be0 msgr2=0x559a9dade4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 -- 172.21.15.62:0/2191925654 shutdown_connections 2023-07-14T16:28:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.904+0000 7f6439a6bb80 1 -- 172.21.15.62:0/2191925654 wait complete. 2023-07-14T16:28:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.905+0000 7f6439a6bb80 20 client.19107 trim_cache size 0 max 16384 2023-07-14T16:28:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.922+0000 7f6439a6bb80 1 Processor -- start 2023-07-14T16:28:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.922+0000 7f6439a6bb80 1 -- start start 2023-07-14T16:28:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.922+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9d9406b0 0x559a9dae5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.922+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9daec350 con 0x559a9dae56e0 2023-07-14T16:28:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9dad6c00 con 0x559a9dae4840 2023-07-14T16:28:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9daed3e0 con 0x559a9d9406b0 2023-07-14T16:28:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daec560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35510/0 (socket says 172.21.15.62:35510) 2023-07-14T16:28:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641ffff700 1 -- 172.21.15.62:0/3139998412 learned_addr learned my addr 172.21.15.62:0/3139998412 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9d9406b0 0x559a9dae5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641effd700 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641ffff700 1 -- 172.21.15.62:0/3139998412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9d9406b0 msgr2=0x559a9dae5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641ffff700 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9d9406b0 0x559a9dae5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641ffff700 1 -- 172.21.15.62:0/3139998412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 msgr2=0x559a9daf4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641ffff700 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.923+0000 7f641ffff700 1 -- 172.21.15.62:0/3139998412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a9d88a1f0 con 0x559a9dae56e0 2023-07-14T16:28:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.924+0000 7f641ffff700 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daec560 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6418004b80 tx=0x7f6418007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc6bf79036ffee0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.924+0000 7f64067fc700 1 -- 172.21.15.62:0/3139998412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180cb070 con 0x559a9dae56e0 2023-07-14T16:28:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.924+0000 7f64067fc700 1 -- 172.21.15.62:0/3139998412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6418099580 con 0x559a9dae56e0 2023-07-14T16:28:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.924+0000 7f64067fc700 1 -- 172.21.15.62:0/3139998412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641809d150 con 0x559a9dae56e0 2023-07-14T16:28:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3139998412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 msgr2=0x559a9daec560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daec560 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6418004b80 tx=0x7f6418007ea0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3139998412 shutdown_connections 2023-07-14T16:28:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9dae56e0 0x559a9daec560 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9d9406b0 0x559a9dae5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3139998412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3139998412 >> 172.21.15.62:0/3139998412 conn(0x559a9dad9be0 msgr2=0x559a9db002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3139998412 shutdown_connections 2023-07-14T16:28:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3139998412 wait complete. 2023-07-14T16:28:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.925+0000 7f6439a6bb80 1 -- start start 2023-07-14T16:28:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f6439a6bb80 1 Processor -- start 2023-07-14T16:28:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9daf0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f6439a6bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae56e0 0x559a9daf3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9daf41b0 con 0x559a9d9406b0 2023-07-14T16:28:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9daf4320 con 0x559a9dae4840 2023-07-14T16:28:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f6439a6bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9daf44e0 con 0x559a9dae56e0 2023-07-14T16:28:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f6439a6bb80 10 client.0 _ll_register_callbacks cb 0x7f64180958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f641f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9daf0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f641f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9daf0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35512/0 (socket says 172.21.15.62:35512) 2023-07-14T16:28:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.926+0000 7f641f7fe700 1 -- 172.21.15.62:0/3673201386 learned_addr learned my addr 172.21.15.62:0/3673201386 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641ffff700 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae56e0 0x559a9daf3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641effd700 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641f7fe700 1 -- 172.21.15.62:0/3673201386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae56e0 msgr2=0x559a9daf3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae56e0 0x559a9daf3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641f7fe700 1 -- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 msgr2=0x559a9daf0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641f7fe700 1 -- 172.21.15.62:0/3673201386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a9d84cc00 con 0x559a9d9406b0 2023-07-14T16:28:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641effd700 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f641f7fe700 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9daf0740 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6410005f50 tx=0x7f6410004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57384ffb0737799 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f63feffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.927+0000 7f63feffd700 1 -- 172.21.15.62:0/3673201386 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410035040 con 0x559a9d9406b0 2023-07-14T16:28:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f63feffd700 1 -- 172.21.15.62:0/3673201386 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f641003b580 con 0x559a9d9406b0 2023-07-14T16:28:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3673201386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f641001e040 con 0x559a9d9406b0 2023-07-14T16:28:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f63feffd700 1 -- 172.21.15.62:0/3673201386 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f641004a030 con 0x559a9d9406b0 2023-07-14T16:28:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 10 client.19113 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3673201386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559a9d862250 con 0x559a9d9406b0 2023-07-14T16:28:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f63feffd700 1 -- 172.21.15.62:0/3673201386 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f641001e040 con 0x559a9d9406b0 2023-07-14T16:28:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f63feffd700 1 -- 172.21.15.62:0/3673201386 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f64100049c0 con 0x559a9d9406b0 2023-07-14T16:28:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 10 client.19113 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 10 client.19113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 20 client.19113 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a9db4db40 0x559a9db4ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 4 client.19113 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:28:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.928+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3673201386 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559a9d9f0ff0 con 0x559a9db4db40 2023-07-14T16:28:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.929+0000 7f641effd700 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a9db4db40 0x559a9db4ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.929+0000 7f63feffd700 10 client.19113 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.929+0000 7f641effd700 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a9db4db40 0x559a9db4ff50 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d276697aee37d6ed server_cookie=f828b82592043c06 in_seq=0 out_seq=0 2023-07-14T16:28:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f63feffd700 1 -- 172.21.15.62:0/3673201386 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559a9d9f0ff0 con 0x559a9db4db40 2023-07-14T16:28:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f63feffd700 10 client.19113 handle_command_reply: tid=0 2023-07-14T16:28:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 client.19113 shutdown 2023-07-14T16:28:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3673201386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 msgr2=0x559a9daf0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9daf0740 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6410005f50 tx=0x7f6410004dc0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3673201386 shutdown_connections 2023-07-14T16:28:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559a9dae56e0 0x559a9daf3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a9d9406b0 0x559a9daf0740 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559a9db4db40 0x559a9db4ff50 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 --2- 172.21.15.62:0/3673201386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559a9dae4840 0x559a9daf0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f63feffd700 0 client.19113 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3673201386 >> 172.21.15.62:0/3673201386 conn(0x559a9dad9be0 msgr2=0x559a9daec560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3673201386 shutdown_connections 2023-07-14T16:28:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 1 -- 172.21.15.62:0/3673201386 wait complete. 2023-07-14T16:28:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.940+0000 7f6439a6bb80 20 client.19113 trim_cache size 0 max 16384 2023-07-14T16:28:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:28:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:28:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:28:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:28:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 -- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64080a11d0 msgr2=0x7f64080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64080a11d0 0x7f64080a3680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f641000b5c0 tx=0x7f6410005fb0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 -- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420076ac0 msgr2=0x7f642019d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420076ac0 0x7f642019d140 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f6418007f10 tx=0x7f641800d3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 -- 172.21.15.62:0/3437416643 shutdown_connections 2023-07-14T16:28:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6420076ac0 0x7f642019d140 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64080a11d0 0x7f64080a3680 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420074ed0 0x7f642019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 --2- 172.21.15.62:0/3437416643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6420076110 0x7f642019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 -- 172.21.15.62:0/3437416643 >> 172.21.15.62:0/3437416643 conn(0x7f64200fd320 msgr2=0x7f64200ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 -- 172.21.15.62:0/3437416643 shutdown_connections 2023-07-14T16:28:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:36.941+0000 7f64261ff700 1 -- 172.21.15.62:0/3437416643 wait complete. 2023-07-14T16:28:37.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:36 smithi062 ceph-mon[95543]: pgmap v746: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:28:37.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:36 smithi081 ceph-mon[127795]: pgmap v746: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:28:37.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:36 smithi164 ceph-mon[97427]: pgmap v746: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:28:38.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:28:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.406+0000 7f9e3ffff700 1 -- 172.21.15.62:0/2941837264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e38101660 msgr2=0x7f9e38107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.406+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/2941837264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e38101660 0x7f9e38107f20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000b0a0 tx=0x7f9e3000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.406+0000 7f9e3ffff700 1 -- 172.21.15.62:0/2941837264 shutdown_connections 2023-07-14T16:28:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.406+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/2941837264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e38101660 0x7f9e38107f20 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.406+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/2941837264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e380ffae0 0x7f9e380ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.406+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/2941837264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e38100cd0 0x7f9e38101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.406+0000 7f9e3ffff700 1 -- 172.21.15.62:0/2941837264 >> 172.21.15.62:0/2941837264 conn(0x7f9e380fb0e0 msgr2=0x7f9e380fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.407+0000 7f9e3ffff700 1 -- 172.21.15.62:0/2941837264 shutdown_connections 2023-07-14T16:28:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.407+0000 7f9e3ffff700 1 -- 172.21.15.62:0/2941837264 wait complete. 2023-07-14T16:28:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.408+0000 7f9e3ffff700 1 Processor -- start 2023-07-14T16:28:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.408+0000 7f9e3ffff700 1 -- start start 2023-07-14T16:28:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e380ffae0 0x7f9e38195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e38100cd0 0x7f9e38195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e38101660 0x7f9e3819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3819d3d0 con 0x7f9e380ffae0 2023-07-14T16:28:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3819d510 con 0x7f9e38101660 2023-07-14T16:28:38.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3ffff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3819d6a0 con 0x7f9e38100cd0 2023-07-14T16:28:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e380ffae0 0x7f9e38195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e380ffae0 0x7f9e38195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35526/0 (socket says 172.21.15.62:35526) 2023-07-14T16:28:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/3576059458 learned_addr learned my addr 172.21.15.62:0/3576059458 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e38100cd0 0x7f9e38195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.409+0000 7f9e3e59c700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e38101660 0x7f9e3819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/3576059458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e38100cd0 msgr2=0x7f9e38195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e38100cd0 0x7f9e38195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/3576059458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e38101660 msgr2=0x7f9e3819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e38101660 0x7f9e3819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/3576059458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009d00 con 0x7f9e380ffae0 2023-07-14T16:28:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e380ffae0 0x7f9e38195850 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400e970 tx=0x7f9e3400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e2effd700 1 -- 172.21.15.62:0/3576059458 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34009790 con 0x7f9e380ffae0 2023-07-14T16:28:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e2effd700 1 -- 172.21.15.62:0/3576059458 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e3400c7e0 con 0x7f9e380ffae0 2023-07-14T16:28:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e3ffff700 1 -- 172.21.15.62:0/3576059458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3819d980 con 0x7f9e380ffae0 2023-07-14T16:28:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.410+0000 7f9e2effd700 1 -- 172.21.15.62:0/3576059458 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34003bd0 con 0x7f9e380ffae0 2023-07-14T16:28:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.411+0000 7f9e3ffff700 1 -- 172.21.15.62:0/3576059458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3819dee0 con 0x7f9e380ffae0 2023-07-14T16:28:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.414+0000 7f9e2effd700 1 -- 172.21.15.62:0/3576059458 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9e3400c950 con 0x7f9e380ffae0 2023-07-14T16:28:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.416+0000 7f9e2effd700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e240a11d0 0x7f9e240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.416+0000 7f9e2effd700 1 -- 172.21.15.62:0/3576059458 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e34014070 con 0x7f9e380ffae0 2023-07-14T16:28:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e53923b80 1 Processor -- start 2023-07-14T16:28:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e240a11d0 0x7f9e240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e53923b80 1 -- start start 2023-07-14T16:28:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb94d0 0x561664fb98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb9e20 0x561664fba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 0x561664fbb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e53923b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664ebecb0 con 0x561664fb9e20 2023-07-14T16:28:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e53923b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664f11a40 con 0x561664fbac70 2023-07-14T16:28:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.417+0000 7f9e53923b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664f0aed0 con 0x561664fb94d0 2023-07-14T16:28:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e240a11d0 0x7f9e240a3680 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009c60 tx=0x7f9e28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb94d0 0x561664fb98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 0x561664fbb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb94d0 0x561664fb98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55336/0 (socket says 172.21.15.62:55336) 2023-07-14T16:28:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/1890022390 learned_addr learned my addr 172.21.15.62:0/1890022390 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/1890022390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb9e20 0x561664fba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/1890022390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 msgr2=0x561664fbb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/1890022390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 0x561664fbb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/1890022390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb9e20 msgr2=0x561664fba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/1890022390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb9e20 0x561664fba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.418+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/1890022390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561664d38de0 con 0x561664fb94d0 2023-07-14T16:28:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.419+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/1890022390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb94d0 0x561664fb98e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e34099dc0 tx=0x7f9e3400cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54e4dba5a6c631c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.419+0000 7f9e23fff700 1 -- 172.21.15.62:0/1890022390 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340b1070 con 0x561664fb94d0 2023-07-14T16:28:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.419+0000 7f9e23fff700 1 -- 172.21.15.62:0/1890022390 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e34099e30 con 0x561664fb94d0 2023-07-14T16:28:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.419+0000 7f9e23fff700 1 -- 172.21.15.62:0/1890022390 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340982c0 con 0x561664fb94d0 2023-07-14T16:28:38.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 -- 172.21.15.62:0/1890022390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb94d0 msgr2=0x561664fb98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1890022390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb94d0 0x561664fb98e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e34099dc0 tx=0x7f9e3400cf50 comp rx=0 tx=0).stop 2023-07-14T16:28:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 -- 172.21.15.62:0/1890022390 shutdown_connections 2023-07-14T16:28:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1890022390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 0x561664fbb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1890022390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb94d0 0x561664fb98e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1890022390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb9e20 0x561664fba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 -- 172.21.15.62:0/1890022390 >> 172.21.15.62:0/1890022390 conn(0x561664faeda0 msgr2=0x561664fb41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 -- 172.21.15.62:0/1890022390 shutdown_connections 2023-07-14T16:28:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.420+0000 7f9e53923b80 1 -- 172.21.15.62:0/1890022390 wait complete. 2023-07-14T16:28:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.421+0000 7f9e53923b80 1 -- start start 2023-07-14T16:28:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.421+0000 7f9e53923b80 1 Processor -- start 2023-07-14T16:28:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb9e20 0x561664fc7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 0x561664fceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e53923b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664eaaac0 con 0x561664fb94d0 2023-07-14T16:28:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e53923b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664eaa7c0 con 0x561664fbac70 2023-07-14T16:28:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e53923b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664d8e920 con 0x561664fb9e20 2023-07-14T16:28:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e53923b80 10 client.0 _ll_register_callbacks cb 0x561664ea8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35546/0 (socket says 172.21.15.62:35546) 2023-07-14T16:28:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/1374643373 learned_addr learned my addr 172.21.15.62:0/1374643373 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb9e20 0x561664fc7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.422+0000 7f9e3e59c700 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 0x561664fceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/1374643373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb9e20 msgr2=0x561664fc7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb9e20 0x561664fc7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/1374643373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 msgr2=0x561664fceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 0x561664fceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/1374643373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3819dee0 con 0x561664fb94d0 2023-07-14T16:28:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc7560 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9e340ca570 tx=0x7f9e34095b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1439354321017f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e217fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e217fa700 1 -- 172.21.15.62:0/1374643373 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340b1070 con 0x561664fb94d0 2023-07-14T16:28:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e217fa700 1 -- 172.21.15.62:0/1374643373 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e340a2280 con 0x561664fb94d0 2023-07-14T16:28:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e53923b80 1 -- 172.21.15.62:0/1374643373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561664e644f0 con 0x561664fb94d0 2023-07-14T16:28:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e217fa700 1 -- 172.21.15.62:0/1374643373 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e34099530 con 0x561664fb94d0 2023-07-14T16:28:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e53923b80 10 client.19125 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.423+0000 7f9e53923b80 1 -- 172.21.15.62:0/1374643373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561664d21c60 con 0x561664fb94d0 2023-07-14T16:28:38.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e217fa700 1 -- 172.21.15.62:0/1374643373 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e3407c020 con 0x561664fb94d0 2023-07-14T16:28:38.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e217fa700 1 -- 172.21.15.62:0/1374643373 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9e340a1df0 con 0x561664fb94d0 2023-07-14T16:28:38.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e53923b80 10 client.19125 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:38.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e53923b80 10 client.19125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:38.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e53923b80 20 client.19125 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:38.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561664fd2f80 0x561664fd5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e53923b80 4 client.19125 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:38.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e53923b80 1 -- 172.21.15.62:0/1374643373 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561664f03eb0 con 0x561664fd2f80 2023-07-14T16:28:38.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.424+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561664fd2f80 0x561664fd5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.425+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561664fd2f80 0x561664fd5390 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ff03a7389726ce server_cookie=b880a3e1a90ecfc6 in_seq=0 out_seq=0 2023-07-14T16:28:38.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.425+0000 7f9e217fa700 10 client.19125 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:38.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e217fa700 1 -- 172.21.15.62:0/1374643373 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561664f03eb0 con 0x561664fd2f80 2023-07-14T16:28:38.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e217fa700 10 client.19125 handle_command_reply: tid=0 2023-07-14T16:28:38.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 client.19125 shutdown 2023-07-14T16:28:38.404 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 -- 172.21.15.62:0/1374643373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 msgr2=0x561664fc7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc7560 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9e340ca570 tx=0x7f9e34095b00 comp rx=0 tx=0).stop 2023-07-14T16:28:38.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 -- 172.21.15.62:0/1374643373 shutdown_connections 2023-07-14T16:28:38.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561664fd2f80 0x561664fd5390 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fbac70 0x561664fceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc7560 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 --2- 172.21.15.62:0/1374643373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fb9e20 0x561664fc7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e217fa700 0 client.19125 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:38.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.426+0000 7f9e53923b80 1 -- 172.21.15.62:0/1374643373 >> 172.21.15.62:0/1374643373 conn(0x561664faeda0 msgr2=0x561664fb38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:38.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.427+0000 7f9e53923b80 1 -- 172.21.15.62:0/1374643373 shutdown_connections 2023-07-14T16:28:38.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.427+0000 7f9e53923b80 1 -- 172.21.15.62:0/1374643373 wait complete. 2023-07-14T16:28:38.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.427+0000 7f9e53923b80 20 client.19125 trim_cache size 0 max 16384 2023-07-14T16:28:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.451+0000 7f9e53923b80 1 Processor -- start 2023-07-14T16:28:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.451+0000 7f9e53923b80 1 -- start start 2023-07-14T16:28:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.451+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.451+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 0x561664fbdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e53923b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664fb98a0 con 0x561664fb94d0 2023-07-14T16:28:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e53923b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664fb9a10 con 0x561664fb9e20 2023-07-14T16:28:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e53923b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664fb5a00 con 0x561664fbac70 2023-07-14T16:28:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3d59a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35554/0 (socket says 172.21.15.62:35554) 2023-07-14T16:28:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/2679461724 learned_addr learned my addr 172.21.15.62:0/2679461724 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3e59c700 1 --2- 172.21.15.62:0/2679461724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 0x561664fbdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3d59a700 1 -- 172.21.15.62:0/2679461724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 msgr2=0x561664fbdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/2679461724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 0x561664fbdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3d59a700 1 -- 172.21.15.62:0/2679461724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 msgr2=0x561664fc0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/2679461724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.452+0000 7f9e3d59a700 1 -- 172.21.15.62:0/2679461724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561664d5f250 con 0x561664fb9e20 2023-07-14T16:28:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.453+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/2679461724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcea70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x561664fd5c40 tx=0x7f9e28014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3cec045a629d9ce server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.453+0000 7f9e2cff9700 1 -- 172.21.15.62:0/2679461724 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28022030 con 0x561664fb9e20 2023-07-14T16:28:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.453+0000 7f9e2cff9700 1 -- 172.21.15.62:0/2679461724 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e28023040 con 0x561664fb9e20 2023-07-14T16:28:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.453+0000 7f9e2cff9700 1 -- 172.21.15.62:0/2679461724 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2802b510 con 0x561664fb9e20 2023-07-14T16:28:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 -- 172.21.15.62:0/2679461724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 msgr2=0x561664fcea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2679461724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcea70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x561664fd5c40 tx=0x7f9e28014000 comp rx=0 tx=0).stop 2023-07-14T16:28:38.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 -- 172.21.15.62:0/2679461724 shutdown_connections 2023-07-14T16:28:38.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2679461724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 0x561664fbdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2679461724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fc0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2679461724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcea70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 -- 172.21.15.62:0/2679461724 >> 172.21.15.62:0/2679461724 conn(0x561664faeda0 msgr2=0x561664fd9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:38.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 -- 172.21.15.62:0/2679461724 shutdown_connections 2023-07-14T16:28:38.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 -- 172.21.15.62:0/2679461724 wait complete. 2023-07-14T16:28:38.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.454+0000 7f9e53923b80 1 -- start start 2023-07-14T16:28:38.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.455+0000 7f9e53923b80 1 Processor -- start 2023-07-14T16:28:38.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fcb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e53923b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 0x561665026e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e53923b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664fceba0 con 0x561664fb94d0 2023-07-14T16:28:38.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e53923b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561664fced10 con 0x561664fb9e20 2023-07-14T16:28:38.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e53923b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561665027370 con 0x561664fbac70 2023-07-14T16:28:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e53923b80 10 client.0 _ll_register_callbacks cb 0x7f9e28009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fcb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fcb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35566/0 (socket says 172.21.15.62:35566) 2023-07-14T16:28:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/2159215855 learned_addr learned my addr 172.21.15.62:0/2159215855 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3e59c700 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 0x561665026e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/2159215855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 msgr2=0x561665026e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 0x561665026e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/2159215855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 msgr2=0x561664fcbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.456+0000 7f9e3dd9b700 1 -- 172.21.15.62:0/2159215855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e28017070 con 0x561664fb94d0 2023-07-14T16:28:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.457+0000 7f9e3dd9b700 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fcb640 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400cde0 tx=0x7f9e340981d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4aa3889342d1bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.457+0000 7f9e217fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.457+0000 7f9e217fa700 1 -- 172.21.15.62:0/2159215855 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340b1020 con 0x561664fb94d0 2023-07-14T16:28:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.457+0000 7f9e217fa700 1 -- 172.21.15.62:0/2159215855 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9e3407d080 con 0x561664fb94d0 2023-07-14T16:28:38.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.457+0000 7f9e53923b80 1 -- 172.21.15.62:0/2159215855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e34099e30 con 0x561664fb94d0 2023-07-14T16:28:38.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.457+0000 7f9e217fa700 1 -- 172.21.15.62:0/2159215855 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e34095f90 con 0x561664fb94d0 2023-07-14T16:28:38.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.457+0000 7f9e53923b80 10 client.19131 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:38.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.457+0000 7f9e53923b80 1 -- 172.21.15.62:0/2159215855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561664d372b0 con 0x561664fb94d0 2023-07-14T16:28:38.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e217fa700 1 -- 172.21.15.62:0/2159215855 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e34099e30 con 0x561664fb94d0 2023-07-14T16:28:38.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e217fa700 1 -- 172.21.15.62:0/2159215855 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9e340a1df0 con 0x561664fb94d0 2023-07-14T16:28:38.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e53923b80 10 client.19131 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:38.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e53923b80 10 client.19131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:38.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e53923b80 20 client.19131 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:38.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561665028650 0x56166502aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e53923b80 4 client.19131 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:28:38.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e53923b80 1 -- 172.21.15.62:0/2159215855 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561664ec6050 con 0x561665028650 2023-07-14T16:28:38.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.458+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561665028650 0x56166502aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e3d59a700 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561665028650 0x56166502aa60 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8466ab09cf35a099 server_cookie=47fcb1f11e1b131a in_seq=0 out_seq=0 2023-07-14T16:28:38.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e217fa700 10 client.19131 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:38.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e217fa700 1 -- 172.21.15.62:0/2159215855 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561664ec6050 con 0x561665028650 2023-07-14T16:28:38.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e217fa700 10 client.19131 handle_command_reply: tid=0 2023-07-14T16:28:38.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 client.19131 shutdown 2023-07-14T16:28:38.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 -- 172.21.15.62:0/2159215855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 msgr2=0x561664fcb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fcb640 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400cde0 tx=0x7f9e340981d0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 -- 172.21.15.62:0/2159215855 shutdown_connections 2023-07-14T16:28:38.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561664fbac70 0x561665026e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561664fb94d0 0x561664fcb640 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561665028650 0x56166502aa60 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 --2- 172.21.15.62:0/2159215855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561664fb9e20 0x561664fcbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.459+0000 7f9e53923b80 1 -- 172.21.15.62:0/2159215855 >> 172.21.15.62:0/2159215855 conn(0x561664faeda0 msgr2=0x561664fc0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:38.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.460+0000 7f9e217fa700 0 client.19131 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:38.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.460+0000 7f9e53923b80 1 -- 172.21.15.62:0/2159215855 shutdown_connections 2023-07-14T16:28:38.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.460+0000 7f9e53923b80 1 -- 172.21.15.62:0/2159215855 wait complete. 2023-07-14T16:28:38.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.460+0000 7f9e53923b80 20 client.19131 trim_cache size 0 max 16384 2023-07-14T16:28:38.441 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:28:38.441 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:28:38.441 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:28:38.441 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:28:38.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.460+0000 7f9e3ffff700 1 -- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e240a11d0 msgr2=0x7f9e240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.460+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e240a11d0 0x7f9e240a3680 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009c60 tx=0x7f9e28009260 comp rx=0 tx=0).stop 2023-07-14T16:28:38.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 -- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e380ffae0 msgr2=0x7f9e38195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e380ffae0 0x7f9e38195850 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400e970 tx=0x7f9e3400ec80 comp rx=0 tx=0).stop 2023-07-14T16:28:38.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 -- 172.21.15.62:0/3576059458 shutdown_connections 2023-07-14T16:28:38.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9e240a11d0 0x7f9e240a3680 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e38101660 0x7f9e3819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e380ffae0 0x7f9e38195850 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 --2- 172.21.15.62:0/3576059458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e38100cd0 0x7f9e38195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 -- 172.21.15.62:0/3576059458 >> 172.21.15.62:0/3576059458 conn(0x7f9e380fb0e0 msgr2=0x7f9e380fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 -- 172.21.15.62:0/3576059458 shutdown_connections 2023-07-14T16:28:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.461+0000 7f9e3ffff700 1 -- 172.21.15.62:0/3576059458 wait complete. 2023-07-14T16:28:38.453 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:28:38.454 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:28:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.707+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2559315985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc101680 msgr2=0x7fd7cc107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.707+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2559315985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc101680 0x7fd7cc107f40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400b0a0 tx=0x7fd7c400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.707+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2559315985 shutdown_connections 2023-07-14T16:28:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.707+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2559315985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc101680 0x7fd7cc107f40 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.707+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2559315985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7cc0ffb00 0x7fd7cc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.707+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2559315985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7cc100cf0 0x7fd7cc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.707+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2559315985 >> 172.21.15.62:0/2559315985 conn(0x7fd7cc0fb0e0 msgr2=0x7fd7cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.708+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2559315985 shutdown_connections 2023-07-14T16:28:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.708+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2559315985 wait complete. 2023-07-14T16:28:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.709+0000 7fd7d1fff700 1 Processor -- start 2023-07-14T16:28:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.709+0000 7fd7d1fff700 1 -- start start 2023-07-14T16:28:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.709+0000 7fd7d1fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7cc0ffb00 0x7fd7cc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.709+0000 7fd7d1fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc100cf0 0x7fd7cc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.709+0000 7fd7d1fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7cc101680 0x7fd7cc19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.709+0000 7fd7d1fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc19f5f0 con 0x7fd7cc100cf0 2023-07-14T16:28:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.709+0000 7fd7d1fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc19f730 con 0x7fd7cc0ffb00 2023-07-14T16:28:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.709+0000 7fd7d1fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc19f8c0 con 0x7fd7cc101680 2023-07-14T16:28:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc100cf0 0x7fd7cc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc100cf0 0x7fd7cc197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35582/0 (socket says 172.21.15.62:35582) 2023-07-14T16:28:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 -- 172.21.15.62:0/2529752624 learned_addr learned my addr 172.21.15.62:0/2529752624 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7cb7fe700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7cc0ffb00 0x7fd7cc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7cbfff700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7cc101680 0x7fd7cc19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 -- 172.21.15.62:0/2529752624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7cc101680 msgr2=0x7fd7cc19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7cc101680 0x7fd7cc19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 -- 172.21.15.62:0/2529752624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7cc0ffb00 msgr2=0x7fd7cc197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7cc0ffb00 0x7fd7cc197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 -- 172.21.15.62:0/2529752624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4009d00 con 0x7fd7cc100cf0 2023-07-14T16:28:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7cb7fe700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7cc0ffb00 0x7fd7cc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:38.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.710+0000 7fd7caffd700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc100cf0 0x7fd7cc197fb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000d950 tx=0x7fd7c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:38.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.711+0000 7fd7c8ff9700 1 -- 172.21.15.62:0/2529752624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000f890 con 0x7fd7cc100cf0 2023-07-14T16:28:38.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.711+0000 7fd7c8ff9700 1 -- 172.21.15.62:0/2529752624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7c000fe80 con 0x7fd7cc100cf0 2023-07-14T16:28:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.711+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc19fba0 con 0x7fd7cc100cf0 2023-07-14T16:28:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.711+0000 7fd7c8ff9700 1 -- 172.21.15.62:0/2529752624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000e3b0 con 0x7fd7cc100cf0 2023-07-14T16:28:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.711+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc1a0100 con 0x7fd7cc100cf0 2023-07-14T16:28:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.712+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc10b1e0 con 0x7fd7cc100cf0 2023-07-14T16:28:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.714+0000 7fd7c8ff9700 1 -- 172.21.15.62:0/2529752624 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd7c0010460 con 0x7fd7cc100cf0 2023-07-14T16:28:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.716+0000 7fd7c8ff9700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7b40a1220 0x7fd7b40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.716+0000 7fd7c8ff9700 1 -- 172.21.15.62:0/2529752624 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c00cc640 con 0x7fd7cc100cf0 2023-07-14T16:28:38.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.716+0000 7fd7cb7fe700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7b40a1220 0x7fd7b40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:38.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.717+0000 7fd7cb7fe700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7b40a1220 0x7fd7b40a36d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc100b50 tx=0x7fd7bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:38.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:38.720+0000 7fd7c8ff9700 1 -- 172.21.15.62:0/2529752624 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd7c0095da0 con 0x7fd7cc100cf0 2023-07-14T16:28:39.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.044+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7cc10b400 con 0x7fd7cc100cf0 2023-07-14T16:28:39.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.044+0000 7fd7c8ff9700 1 -- 172.21.15.62:0/2529752624 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fd7c00993c0 con 0x7fd7cc100cf0 2023-07-14T16:28:39.019 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:28:39.019 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:20:03.692896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:28:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7b40a1220 msgr2=0x7fd7b40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7b40a1220 0x7fd7b40a36d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc100b50 tx=0x7fd7bc009260 comp rx=0 tx=0).stop 2023-07-14T16:28:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc100cf0 msgr2=0x7fd7cc197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc100cf0 0x7fd7cc197fb0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000d950 tx=0x7fd7c000dc60 comp rx=0 tx=0).stop 2023-07-14T16:28:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 shutdown_connections 2023-07-14T16:28:39.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7cc101680 0x7fd7cc19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7b40a1220 0x7fd7b40a36d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7cc0ffb00 0x7fd7cc197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 --2- 172.21.15.62:0/2529752624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7cc100cf0 0x7fd7cc197fb0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 >> 172.21.15.62:0/2529752624 conn(0x7fd7cc0fb0e0 msgr2=0x7fd7cc0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 shutdown_connections 2023-07-14T16:28:39.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.047+0000 7fd7d1fff700 1 -- 172.21.15.62:0/2529752624 wait complete. 2023-07-14T16:28:39.023 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 25 2023-07-14T16:28:39.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:28:39.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:38 smithi062 ceph-mon[95543]: pgmap v747: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:28:39.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:38 smithi081 ceph-mon[127795]: pgmap v747: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:28:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.295+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/715546411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 msgr2=0x7f2e981052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.295+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/715546411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 0x7f2e981052c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009990 tx=0x7f2e8c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.296+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/715546411 shutdown_connections 2023-07-14T16:28:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.296+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/715546411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e98105800 0x7f2e98107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.296+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/715546411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e981005c0 0x7f2e981029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.296+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/715546411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 0x7f2e981052c0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.296+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/715546411 >> 172.21.15.62:0/715546411 conn(0x7f2e980fa000 msgr2=0x7f2e980fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.296+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/715546411 shutdown_connections 2023-07-14T16:28:39.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.296+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/715546411 wait complete. 2023-07-14T16:28:39.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:38 smithi164 ceph-mon[97427]: pgmap v747: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 16 MiB/s wr, 12 op/s 2023-07-14T16:28:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.297+0000 7f2e9e3ff700 1 Processor -- start 2023-07-14T16:28:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.297+0000 7f2e9e3ff700 1 -- start start 2023-07-14T16:28:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e9e3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e981005c0 0x7f2e981979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e9e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 0x7f2e98197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e9e3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e98105800 0x7f2e9819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e9e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819f520 con 0x7f2e98102ee0 2023-07-14T16:28:39.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e9e3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819f660 con 0x7f2e98105800 2023-07-14T16:28:39.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e9e3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819f7f0 con 0x7f2e981005c0 2023-07-14T16:28:39.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 0x7f2e98197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 0x7f2e98197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35602/0 (socket says 172.21.15.62:35602) 2023-07-14T16:28:39.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e977fe700 1 -- 172.21.15.62:0/1398286867 learned_addr learned my addr 172.21.15.62:0/1398286867 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e97fff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e981005c0 0x7f2e981979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.298+0000 7f2e9c99c700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e98105800 0x7f2e9819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e97fff700 1 -- 172.21.15.62:0/1398286867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e98105800 msgr2=0x7f2e9819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e97fff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e98105800 0x7f2e9819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e97fff700 1 -- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 msgr2=0x7f2e98197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e97fff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 0x7f2e98197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e97fff700 1 -- 172.21.15.62:0/1398286867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0095f0 con 0x7f2e981005c0 2023-07-14T16:28:39.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e977fe700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 0x7f2e98197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:39.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e97fff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e981005c0 0x7f2e981979a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800e9f0 tx=0x7f2e8800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e957fa700 1 -- 172.21.15.62:0/1398286867 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800c9c0 con 0x7f2e981005c0 2023-07-14T16:28:39.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e957fa700 1 -- 172.21.15.62:0/1398286867 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e88010430 con 0x7f2e981005c0 2023-07-14T16:28:39.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/1398286867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9819fad0 con 0x7f2e981005c0 2023-07-14T16:28:39.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.299+0000 7f2e957fa700 1 -- 172.21.15.62:0/1398286867 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88018680 con 0x7f2e981005c0 2023-07-14T16:28:39.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.300+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/1398286867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9819ffe0 con 0x7f2e981005c0 2023-07-14T16:28:39.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.305+0000 7f2e957fa700 1 -- 172.21.15.62:0/1398286867 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2e880105a0 con 0x7f2e981005c0 2023-07-14T16:28:39.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.305+0000 7f2eb1d52b80 1 Processor -- start 2023-07-14T16:28:39.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.306+0000 7f2eb1d52b80 1 -- start start 2023-07-14T16:28:39.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.306+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.306+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fce20 0x5569165fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.306+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fdc70 0x5569165fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.306+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556916501cb0 con 0x5569165fc4d0 2023-07-14T16:28:39.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.306+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556916554a40 con 0x5569165fce20 2023-07-14T16:28:39.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.306+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691654ded0 con 0x5569165fdc70 2023-07-14T16:28:39.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.306+0000 7f2e9c99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fce20 0x5569165fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e9c99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165fc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35612/0 (socket says 172.21.15.62:35612) 2023-07-14T16:28:39.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e9c99c700 1 -- 172.21.15.62:0/2127010094 learned_addr learned my addr 172.21.15.62:0/2127010094 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e977fe700 1 --2- 172.21.15.62:0/2127010094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fdc70 0x5569165fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e9c99c700 1 -- 172.21.15.62:0/2127010094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fdc70 msgr2=0x5569165fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e9c99c700 1 --2- 172.21.15.62:0/2127010094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fdc70 0x5569165fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e9c99c700 1 -- 172.21.15.62:0/2127010094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fce20 msgr2=0x5569165fd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e9c99c700 1 --2- 172.21.15.62:0/2127010094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fce20 0x5569165fd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e957fa700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e800a1150 0x7f2e800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e9c99c700 1 -- 172.21.15.62:0/2127010094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691637bde0 con 0x5569165fc4d0 2023-07-14T16:28:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.307+0000 7f2e957fa700 1 -- 172.21.15.62:0/1398286867 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2e88014070 con 0x7f2e981005c0 2023-07-14T16:28:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.308+0000 7f2e977fe700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e800a1150 0x7f2e800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.308+0000 7f2e9c99c700 1 --2- 172.21.15.62:0/2127010094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165fc8e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2e90011680 tx=0x7f2e90009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd025f2219619aaa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.308+0000 7f2e7e7fc700 1 -- 172.21.15.62:0/2127010094 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000e040 con 0x5569165fc4d0 2023-07-14T16:28:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.308+0000 7f2e7e7fc700 1 -- 172.21.15.62:0/2127010094 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e9000f040 con 0x5569165fc4d0 2023-07-14T16:28:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.308+0000 7f2e7e7fc700 1 -- 172.21.15.62:0/2127010094 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90004770 con 0x5569165fc4d0 2023-07-14T16:28:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2127010094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 msgr2=0x5569165fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2e977fe700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e800a1150 0x7f2e800a3600 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006010 tx=0x7f2e8c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2127010094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165fc8e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2e90011680 tx=0x7f2e90009920 comp rx=0 tx=0).stop 2023-07-14T16:28:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2127010094 shutdown_connections 2023-07-14T16:28:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2127010094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fdc70 0x5569165fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2127010094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165fc8e0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2127010094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fce20 0x5569165fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2127010094 >> 172.21.15.62:0/2127010094 conn(0x5569165f1da0 msgr2=0x5569165f71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2127010094 shutdown_connections 2023-07-14T16:28:39.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.309+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2127010094 wait complete. 2023-07-14T16:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 1 -- start start 2023-07-14T16:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 1 Processor -- start 2023-07-14T16:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fc4d0 0x556916606100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x556916606640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fdc70 0x55691660d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569164edac0 con 0x5569165fdc70 2023-07-14T16:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569164ed7c0 con 0x5569165fc4d0 2023-07-14T16:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569163d1920 con 0x5569165fce20 2023-07-14T16:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.310+0000 7f2eb1d52b80 10 client.0 _ll_register_callbacks cb 0x5569164eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fc4d0 0x556916606100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e9c99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fdc70 0x55691660d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fc4d0 0x556916606100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:60736/0 (socket says 172.21.15.62:60736) 2023-07-14T16:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e97fff700 1 -- 172.21.15.62:0/2313482784 learned_addr learned my addr 172.21.15.62:0/2313482784 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e977fe700 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x556916606640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e97fff700 1 -- 172.21.15.62:0/2313482784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 msgr2=0x556916606640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e97fff700 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x556916606640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e97fff700 1 -- 172.21.15.62:0/2313482784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fdc70 msgr2=0x55691660d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e97fff700 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fdc70 0x55691660d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.311+0000 7f2e97fff700 1 -- 172.21.15.62:0/2313482784 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e90011330 con 0x5569165fc4d0 2023-07-14T16:28:39.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2e97fff700 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fc4d0 0x556916606100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2e880963a0 tx=0x7f2e8809a3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c851d2f9c6a00696 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2e777fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:28:39.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2e777fe700 1 -- 172.21.15.62:0/2313482784 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880a2060 con 0x5569165fc4d0 2023-07-14T16:28:39.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2313482784 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569164a74f0 con 0x5569165fc4d0 2023-07-14T16:28:39.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2e777fe700 1 -- 172.21.15.62:0/2313482784 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2e88010880 con 0x5569165fc4d0 2023-07-14T16:28:39.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2e777fe700 1 -- 172.21.15.62:0/2313482784 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e880999e0 con 0x5569165fc4d0 2023-07-14T16:28:39.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2eb1d52b80 10 client.28739 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:39.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2313482784 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556916364c60 con 0x5569165fc4d0 2023-07-14T16:28:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.312+0000 7f2e777fe700 1 -- 172.21.15.62:0/2313482784 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5569164a74f0 con 0x5569165fc4d0 2023-07-14T16:28:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.313+0000 7f2e777fe700 1 -- 172.21.15.62:0/2313482784 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2e880d4030 con 0x5569165fc4d0 2023-07-14T16:28:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.313+0000 7f2eb1d52b80 10 client.28739 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.313+0000 7f2eb1d52b80 10 client.28739 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:28:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.313+0000 7f2eb1d52b80 20 client.28739 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.313+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556916611e30 0x556916614240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.313+0000 7f2eb1d52b80 4 client.28739 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.313+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2313482784 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556916546eb0 con 0x556916611e30 2023-07-14T16:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.313+0000 7f2e977fe700 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556916611e30 0x556916614240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.314+0000 7f2e777fe700 10 client.28739 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.314+0000 7f2e977fe700 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556916611e30 0x556916614240 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e4775a831de5174 server_cookie=9d60a06a70074b42 in_seq=0 out_seq=0 2023-07-14T16:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2e777fe700 1 -- 172.21.15.62:0/2313482784 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556916546eb0 con 0x556916611e30 2023-07-14T16:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2e777fe700 10 client.28739 handle_command_reply: tid=0 2023-07-14T16:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 client.28739 shutdown 2023-07-14T16:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2313482784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fc4d0 msgr2=0x556916606100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fc4d0 0x556916606100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2e880963a0 tx=0x7f2e8809a3e0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2313482784 shutdown_connections 2023-07-14T16:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556916611e30 0x556916614240 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fdc70 0x55691660d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fc4d0 0x556916606100 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/2313482784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x556916606640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2313482784 >> 172.21.15.62:0/2313482784 conn(0x5569165f1da0 msgr2=0x5569165f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2e777fe700 0 client.28739 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:28:39.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2313482784 shutdown_connections 2023-07-14T16:28:39.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.315+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/2313482784 wait complete. 2023-07-14T16:28:39.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.316+0000 7f2eb1d52b80 20 client.28739 trim_cache size 0 max 16384 2023-07-14T16:28:39.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.337+0000 7f2eb1d52b80 1 Processor -- start 2023-07-14T16:28:39.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.337+0000 7f2eb1d52b80 1 -- start start 2023-07-14T16:28:39.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.337+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.337+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.337+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 0x556916613e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.337+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569165f8a00 con 0x5569165fc4d0 2023-07-14T16:28:39.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.337+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691660f370 con 0x5569165fdc70 2023-07-14T16:28:39.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.337+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556916611b30 con 0x5569165fce20 2023-07-14T16:28:39.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165f68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35628/0 (socket says 172.21.15.62:35628) 2023-07-14T16:28:39.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 -- 172.21.15.62:0/183231727 learned_addr learned my addr 172.21.15.62:0/183231727 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e9c99c700 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 0x556916613e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e977fe700 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 -- 172.21.15.62:0/183231727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 msgr2=0x55691660d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 -- 172.21.15.62:0/183231727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 msgr2=0x556916613e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 0x556916613e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 -- 172.21.15.62:0/183231727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569165fe560 con 0x5569165fc4d0 2023-07-14T16:28:39.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e977fe700 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:39.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.338+0000 7f2e97fff700 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165f68b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800c5b0 tx=0x7f2e880d25c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16d3031093bb3f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.339+0000 7f2e7effd700 1 -- 172.21.15.62:0/183231727 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88099800 con 0x5569165fc4d0 2023-07-14T16:28:39.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.339+0000 7f2e7effd700 1 -- 172.21.15.62:0/183231727 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e88010880 con 0x5569165fc4d0 2023-07-14T16:28:39.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.339+0000 7f2e7effd700 1 -- 172.21.15.62:0/183231727 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8809a6c0 con 0x5569165fc4d0 2023-07-14T16:28:39.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.339+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/183231727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 msgr2=0x5569165f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.339+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165f68b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800c5b0 tx=0x7f2e880d25c0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.339+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/183231727 shutdown_connections 2023-07-14T16:28:39.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.339+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 0x556916613e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.339+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x5569165f68b0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.340+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/183231727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.340+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/183231727 >> 172.21.15.62:0/183231727 conn(0x5569165f1da0 msgr2=0x556916618ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.340+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/183231727 shutdown_connections 2023-07-14T16:28:39.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.340+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/183231727 wait complete. 2023-07-14T16:28:39.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.340+0000 7f2eb1d52b80 1 -- start start 2023-07-14T16:28:39.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.340+0000 7f2eb1d52b80 1 Processor -- start 2023-07-14T16:28:39.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x55691660cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2eb1d52b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 0x556916609990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556916609ed0 con 0x5569165fc4d0 2023-07-14T16:28:39.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691660a040 con 0x5569165fdc70 2023-07-14T16:28:39.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2eb1d52b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691660a200 con 0x5569165fce20 2023-07-14T16:28:39.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2eb1d52b80 10 client.0 _ll_register_callbacks cb 0x7f2e88010a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:39.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x55691660cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x55691660cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35638/0 (socket says 172.21.15.62:35638) 2023-07-14T16:28:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.341+0000 7f2e97fff700 1 -- 172.21.15.62:0/3591405288 learned_addr learned my addr 172.21.15.62:0/3591405288 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e9c99c700 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 0x556916609990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e977fe700 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e97fff700 1 -- 172.21.15.62:0/3591405288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 msgr2=0x55691660d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e97fff700 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e97fff700 1 -- 172.21.15.62:0/3591405288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 msgr2=0x556916609990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e97fff700 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 0x556916609990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e97fff700 1 -- 172.21.15.62:0/3591405288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556916364c60 con 0x5569165fc4d0 2023-07-14T16:28:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e97fff700 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x55691660cef0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800e9c0 tx=0x7f2e8800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f152f553391ae5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e777fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e777fe700 1 -- 172.21.15.62:0/3591405288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880a22f0 con 0x5569165fc4d0 2023-07-14T16:28:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2e777fe700 1 -- 172.21.15.62:0/3591405288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2e88099ea0 con 0x5569165fc4d0 2023-07-14T16:28:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.342+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/3591405288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e9000f040 con 0x5569165fc4d0 2023-07-14T16:28:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2e777fe700 1 -- 172.21.15.62:0/3591405288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e88099800 con 0x5569165fc4d0 2023-07-14T16:28:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2eb1d52b80 10 client.19155 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/3591405288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5569163a2250 con 0x5569165fc4d0 2023-07-14T16:28:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2e777fe700 1 -- 172.21.15.62:0/3591405288 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e9000f040 con 0x5569165fc4d0 2023-07-14T16:28:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2e777fe700 1 -- 172.21.15.62:0/3591405288 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2e880d4070 con 0x5569165fc4d0 2023-07-14T16:28:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2eb1d52b80 10 client.19155 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2eb1d52b80 10 client.19155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:28:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2eb1d52b80 20 client.19155 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5569166669d0 0x556916668db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2eb1d52b80 4 client.19155 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:28:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.343+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/3591405288 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556916509050 con 0x5569166669d0 2023-07-14T16:28:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.344+0000 7f2e977fe700 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5569166669d0 0x556916668db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.344+0000 7f2e777fe700 10 client.19155 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:28:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.344+0000 7f2e977fe700 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5569166669d0 0x556916668db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f54ce1313f793a90 server_cookie=b83e323615aeefc9 in_seq=0 out_seq=0 2023-07-14T16:28:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.344+0000 7f2e777fe700 1 -- 172.21.15.62:0/3591405288 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556916509050 con 0x5569166669d0 2023-07-14T16:28:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.344+0000 7f2e777fe700 10 client.19155 handle_command_reply: tid=0 2023-07-14T16:28:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.344+0000 7f2eb1d52b80 1 client.19155 shutdown 2023-07-14T16:28:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/3591405288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 msgr2=0x55691660cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x55691660cef0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800e9c0 tx=0x7f2e8800cdd0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/3591405288 shutdown_connections 2023-07-14T16:28:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5569166669d0 0x556916668db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5569165fdc70 0x556916609990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569165fc4d0 0x55691660cef0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2e777fe700 0 client.19155 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:28:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 --2- 172.21.15.62:0/3591405288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569165fce20 0x55691660d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/3591405288 >> 172.21.15.62:0/3591405288 conn(0x5569165f1da0 msgr2=0x5569165f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/3591405288 shutdown_connections 2023-07-14T16:28:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 1 -- 172.21.15.62:0/3591405288 wait complete. 2023-07-14T16:28:39.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.345+0000 7f2eb1d52b80 20 client.19155 trim_cache size 0 max 16384 2023-07-14T16:28:39.329 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:28:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e800a1150 msgr2=0x7f2e800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e800a1150 0x7f2e800a3600 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006010 tx=0x7f2e8c00b260 comp rx=0 tx=0).stop 2023-07-14T16:28:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/1398286867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e981005c0 msgr2=0x7f2e981979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e981005c0 0x7f2e981979a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800e9f0 tx=0x7f2e8800ed00 comp rx=0 tx=0).stop 2023-07-14T16:28:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/1398286867 shutdown_connections 2023-07-14T16:28:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e800a1150 0x7f2e800a3600 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e98105800 0x7f2e9819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e981005c0 0x7f2e981979a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 --2- 172.21.15.62:0/1398286867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e98102ee0 0x7f2e98197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/1398286867 >> 172.21.15.62:0/1398286867 conn(0x7f2e980fa000 msgr2=0x7f2e98101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/1398286867 shutdown_connections 2023-07-14T16:28:39.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.346+0000 7f2e9e3ff700 1 -- 172.21.15.62:0/1398286867 wait complete. 2023-07-14T16:28:39.339 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:28:39.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 -- 172.21.15.62:0/430793863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0100c80 msgr2=0x7f5ea01010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/430793863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0100c80 0x7f5ea01010f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009990 tx=0x7f5e94009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 -- 172.21.15.62:0/430793863 shutdown_connections 2023-07-14T16:28:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/430793863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ea0101630 0x7f5ea0107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/430793863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea00ff9f0 0x7f5ea00ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/430793863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea0100c80 0x7f5ea01010f0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 -- 172.21.15.62:0/430793863 >> 172.21.15.62:0/430793863 conn(0x7f5ea00fb010 msgr2=0x7f5ea00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 -- 172.21.15.62:0/430793863 shutdown_connections 2023-07-14T16:28:39.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.608+0000 7f5e9ffff700 1 -- 172.21.15.62:0/430793863 wait complete. 2023-07-14T16:28:39.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.609+0000 7f5e9ffff700 1 Processor -- start 2023-07-14T16:28:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.609+0000 7f5e9ffff700 1 -- start start 2023-07-14T16:28:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea00ff9f0 0x7f5ea01954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ea0100c80 0x7f5ea01959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea0101630 0x7f5ea019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9ffff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea019d030 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea019d170 con 0x7f5ea0100c80 2023-07-14T16:28:39.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9ffff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea019d300 con 0x7f5ea0101630 2023-07-14T16:28:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea00ff9f0 0x7f5ea01954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea00ff9f0 0x7f5ea01954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35654/0 (socket says 172.21.15.62:35654) 2023-07-14T16:28:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.610+0000 7f5e9effd700 1 -- 172.21.15.62:0/205524466 learned_addr learned my addr 172.21.15.62:0/205524466 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ea0100c80 0x7f5ea01959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea0101630 0x7f5ea019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9effd700 1 -- 172.21.15.62:0/205524466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea0101630 msgr2=0x7f5ea019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9effd700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea0101630 0x7f5ea019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9effd700 1 -- 172.21.15.62:0/205524466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ea0100c80 msgr2=0x7f5ea01959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9effd700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ea0100c80 0x7f5ea01959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9effd700 1 -- 172.21.15.62:0/205524466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940095f0 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ea0100c80 0x7f5ea01959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:39.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea0101630 0x7f5ea019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:39.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9effd700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea00ff9f0 0x7f5ea01954b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000d950 tx=0x7f5e9000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e87fff700 1 -- 172.21.15.62:0/205524466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90009b40 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e87fff700 1 -- 172.21.15.62:0/205524466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e90009ca0 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e87fff700 1 -- 172.21.15.62:0/205524466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000f620 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9ffff700 1 -- 172.21.15.62:0/205524466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea019d5e0 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.611+0000 7f5e9ffff700 1 -- 172.21.15.62:0/205524466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea019daf0 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.616+0000 7f5e87fff700 1 -- 172.21.15.62:0/205524466 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5e90010460 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.616+0000 7f5eb916eb80 1 Processor -- start 2023-07-14T16:28:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.616+0000 7f5eb916eb80 1 -- start start 2023-07-14T16:28:39.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 0x55dd44773cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd447746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44673cb0 con 0x55dd44772b30 2023-07-14T16:28:39.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd446c6a40 con 0x55dd44773480 2023-07-14T16:28:39.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd446bfed0 con 0x55dd44774240 2023-07-14T16:28:39.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35660/0 (socket says 172.21.15.62:35660) 2023-07-14T16:28:39.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5e9f7fe700 1 -- 172.21.15.62:0/1801854180 learned_addr learned my addr 172.21.15.62:0/1801854180 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd447746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.617+0000 7f5e9effd700 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 0x55dd44773cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e9f7fe700 1 -- 172.21.15.62:0/1801854180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 msgr2=0x55dd447746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd447746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e9f7fe700 1 -- 172.21.15.62:0/1801854180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 msgr2=0x55dd44773cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 0x55dd44773cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e9f7fe700 1 -- 172.21.15.62:0/1801854180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd444edde0 con 0x55dd44772b30 2023-07-14T16:28:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800d300 tx=0x7f5e9800d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ef52654259d72c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e87fff700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e880a1270 0x7f5e880a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e857fa700 1 -- 172.21.15.62:0/1801854180 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980125f0 con 0x55dd44772b30 2023-07-14T16:28:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e857fa700 1 -- 172.21.15.62:0/1801854180 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e98012be0 con 0x55dd44772b30 2023-07-14T16:28:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e857fa700 1 -- 172.21.15.62:0/1801854180 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98011b90 con 0x55dd44772b30 2023-07-14T16:28:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.618+0000 7f5e87fff700 1 -- 172.21.15.62:0/205524466 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5e900ccc90 con 0x7f5ea00ff9f0 2023-07-14T16:28:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5e9effd700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e880a1270 0x7f5e880a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1801854180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 msgr2=0x55dd44772f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5e9effd700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e880a1270 0x7f5e880a3720 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5e90099d50 tx=0x7f5e90010b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800d300 tx=0x7f5e9800d610 comp rx=0 tx=0).stop 2023-07-14T16:28:39.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1801854180 shutdown_connections 2023-07-14T16:28:39.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd447746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f40 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/1801854180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 0x55dd44773cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1801854180 >> 172.21.15.62:0/1801854180 conn(0x55dd44763ff0 msgr2=0x55dd447693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.619+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1801854180 shutdown_connections 2023-07-14T16:28:39.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.620+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1801854180 wait complete. 2023-07-14T16:28:39.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.620+0000 7f5eb916eb80 1 -- start start 2023-07-14T16:28:39.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.620+0000 7f5eb916eb80 1 Processor -- start 2023-07-14T16:28:39.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44772b30 0x55dd4476ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd4476f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd4476b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4465fac0 con 0x55dd44773480 2023-07-14T16:28:39.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4465f7c0 con 0x55dd44772b30 2023-07-14T16:28:39.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44543920 con 0x55dd44774240 2023-07-14T16:28:39.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5eb916eb80 10 client.0 _ll_register_callbacks cb 0x55dd4465d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5e9e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd4476f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5e9e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd4476f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35664/0 (socket says 172.21.15.62:35664) 2023-07-14T16:28:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5e9e7fc700 1 -- 172.21.15.62:0/4087229084 learned_addr learned my addr 172.21.15.62:0/4087229084 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.621+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44772b30 0x55dd4476ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd4476b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e9e7fc700 1 -- 172.21.15.62:0/4087229084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 msgr2=0x55dd4476b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd4476b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e9e7fc700 1 -- 172.21.15.62:0/4087229084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44772b30 msgr2=0x55dd4476ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44772b30 0x55dd4476ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e9e7fc700 1 -- 172.21.15.62:0/4087229084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e98007de0 con 0x55dd44773480 2023-07-14T16:28:39.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44772b30 0x55dd4476ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:39.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd4476f2a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400bec0 tx=0x7f5e9400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc78098e87be396 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.622+0000 7f5e82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:39.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.623+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4087229084 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94005450 con 0x55dd44773480 2023-07-14T16:28:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.623+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4087229084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd446194f0 con 0x55dd44773480 2023-07-14T16:28:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.623+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4087229084 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5e940055b0 con 0x55dd44773480 2023-07-14T16:28:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.623+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4087229084 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e94004500 con 0x55dd44773480 2023-07-14T16:28:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.623+0000 7f5eb916eb80 10 client.19167 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.623+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4087229084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dd444d6c60 con 0x55dd44773480 2023-07-14T16:28:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.623+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4087229084 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dd446194f0 con 0x55dd44773480 2023-07-14T16:28:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.624+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4087229084 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e94003a40 con 0x55dd44773480 2023-07-14T16:28:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.624+0000 7f5eb916eb80 10 client.19167 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.624+0000 7f5eb916eb80 10 client.19167 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:28:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.624+0000 7f5eb916eb80 20 client.19167 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.624+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55dd4478c8b0 0x55dd4478ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.624+0000 7f5eb916eb80 4 client.19167 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.624+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4087229084 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd446b8eb0 con 0x55dd4478c8b0 2023-07-14T16:28:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.624+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55dd4478c8b0 0x55dd4478ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.625+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55dd4478c8b0 0x55dd4478ecc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f95c003328b63b4e server_cookie=8a7d7f1ac053a6c2 in_seq=0 out_seq=0 2023-07-14T16:28:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.625+0000 7f5e82ff5700 10 client.19167 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:28:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4087229084 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd446b8eb0 con 0x55dd4478c8b0 2023-07-14T16:28:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5e82ff5700 10 client.19167 handle_command_reply: tid=0 2023-07-14T16:28:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 client.19167 shutdown 2023-07-14T16:28:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4087229084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 msgr2=0x55dd4476f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd4476f2a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400bec0 tx=0x7f5e9400bef0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4087229084 shutdown_connections 2023-07-14T16:28:39.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55dd4478c8b0 0x55dd4478ecc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd4476b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5e82ff5700 0 client.19167 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:28:39.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44772b30 0x55dd4476ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4087229084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd4476f2a0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4087229084 >> 172.21.15.62:0/4087229084 conn(0x55dd44763ff0 msgr2=0x55dd44768b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.626+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4087229084 shutdown_connections 2023-07-14T16:28:39.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.627+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4087229084 wait complete. 2023-07-14T16:28:39.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.627+0000 7f5eb916eb80 20 client.19167 trim_cache size 0 max 16384 2023-07-14T16:28:39.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5eb916eb80 1 Processor -- start 2023-07-14T16:28:39.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5eb916eb80 1 -- start start 2023-07-14T16:28:39.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 0x55dd44774610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd44793620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4476bed0 con 0x55dd44772b30 2023-07-14T16:28:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4476a810 con 0x55dd44773480 2023-07-14T16:28:39.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4476ae90 con 0x55dd44774240 2023-07-14T16:28:39.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.644+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35672/0 (socket says 172.21.15.62:35672) 2023-07-14T16:28:39.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9f7fe700 1 -- 172.21.15.62:0/1361321157 learned_addr learned my addr 172.21.15.62:0/1361321157 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/1361321157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 0x55dd44774610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9f7fe700 1 -- 172.21.15.62:0/1361321157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 msgr2=0x55dd44793620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/1361321157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd44793620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9f7fe700 1 -- 172.21.15.62:0/1361321157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 msgr2=0x55dd44774610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/1361321157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 0x55dd44774610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9f7fe700 1 -- 172.21.15.62:0/1361321157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd44514250 con 0x55dd44772b30 2023-07-14T16:28:39.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/1361321157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800d220 tx=0x7f5e98011760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd44039583825fa3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.645+0000 7f5e85ffb700 1 -- 172.21.15.62:0/1361321157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98011350 con 0x55dd44772b30 2023-07-14T16:28:39.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5e85ffb700 1 -- 172.21.15.62:0/1361321157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e98023040 con 0x55dd44772b30 2023-07-14T16:28:39.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5e85ffb700 1 -- 172.21.15.62:0/1361321157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98011e90 con 0x55dd44772b30 2023-07-14T16:28:39.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1361321157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 msgr2=0x55dd44772f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/1361321157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800d220 tx=0x7f5e98011760 comp rx=0 tx=0).stop 2023-07-14T16:28:39.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1361321157 shutdown_connections 2023-07-14T16:28:39.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/1361321157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44774240 0x55dd44793620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/1361321157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44772b30 0x55dd44772f00 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/1361321157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44773480 0x55dd44774610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1361321157 >> 172.21.15.62:0/1361321157 conn(0x55dd44763ff0 msgr2=0x55dd44765560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1361321157 shutdown_connections 2023-07-14T16:28:39.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.646+0000 7f5eb916eb80 1 -- 172.21.15.62:0/1361321157 wait complete. 2023-07-14T16:28:39.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.647+0000 7f5eb916eb80 1 -- start start 2023-07-14T16:28:39.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.647+0000 7f5eb916eb80 1 Processor -- start 2023-07-14T16:28:39.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.647+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44772b30 0x55dd44770120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd44770660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5eb916eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44774240 0x55dd44771670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44771bb0 con 0x55dd44773480 2023-07-14T16:28:39.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44771d20 con 0x55dd44774240 2023-07-14T16:28:39.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5eb916eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44771ee0 con 0x55dd44772b30 2023-07-14T16:28:39.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5eb916eb80 10 client.0 _ll_register_callbacks cb 0x7f5e94009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:39.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd44770660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd44770660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35680/0 (socket says 172.21.15.62:35680) 2023-07-14T16:28:39.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9e7fc700 1 -- 172.21.15.62:0/4106565299 learned_addr learned my addr 172.21.15.62:0/4106565299 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44774240 0x55dd44771670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9e7fc700 1 -- 172.21.15.62:0/4106565299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44772b30 msgr2=0x55dd44770120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44772b30 0x55dd44770120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9e7fc700 1 -- 172.21.15.62:0/4106565299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44774240 msgr2=0x55dd44771670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44774240 0x55dd44771670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9e7fc700 1 -- 172.21.15.62:0/4106565299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e98025040 con 0x55dd44773480 2023-07-14T16:28:39.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.648+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44774240 0x55dd44771670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:39.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.649+0000 7f5e9e7fc700 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd44770660 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009350 tx=0x7f5e940055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c470cc2e811c79d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.649+0000 7f5e82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:39.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.649+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4106565299 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9400f870 con 0x55dd44773480 2023-07-14T16:28:39.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.649+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4106565299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e98012be0 con 0x55dd44773480 2023-07-14T16:28:39.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.649+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4106565299 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5e940040c0 con 0x55dd44773480 2023-07-14T16:28:39.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.649+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4106565299 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e9400f9d0 con 0x55dd44773480 2023-07-14T16:28:39.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.649+0000 7f5eb916eb80 10 client.19173 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:39.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.649+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4106565299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dd444ec2b0 con 0x55dd44773480 2023-07-14T16:28:39.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.650+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4106565299 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e94047020 con 0x55dd44773480 2023-07-14T16:28:39.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.650+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4106565299 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e94004990 con 0x55dd44773480 2023-07-14T16:28:39.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.650+0000 7f5eb916eb80 10 client.19173 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:39.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.650+0000 7f5eb916eb80 10 client.19173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:28:39.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.650+0000 7f5eb916eb80 20 client.19173 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:39.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.650+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55dd447e1320 0x55dd447e3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.650+0000 7f5eb916eb80 4 client.19173 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:28:39.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.650+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4106565299 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd4467b050 con 0x55dd447e1320 2023-07-14T16:28:39.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.651+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55dd447e1320 0x55dd447e3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.651+0000 7f5e9f7fe700 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55dd447e1320 0x55dd447e3730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d43ea24fe72d4553 server_cookie=3cdfc232990efd7c in_seq=0 out_seq=0 2023-07-14T16:28:39.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.651+0000 7f5e82ff5700 10 client.19173 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:28:39.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.651+0000 7f5e82ff5700 1 -- 172.21.15.62:0/4106565299 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd4467b050 con 0x55dd447e1320 2023-07-14T16:28:39.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.651+0000 7f5e82ff5700 10 client.19173 handle_command_reply: tid=0 2023-07-14T16:28:39.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.651+0000 7f5eb916eb80 1 client.19173 shutdown 2023-07-14T16:28:39.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4106565299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 msgr2=0x55dd44770660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd44770660 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009350 tx=0x7f5e940055b0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4106565299 shutdown_connections 2023-07-14T16:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55dd447e1320 0x55dd447e3730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55dd44774240 0x55dd44771670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dd44772b30 0x55dd44770120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5e82ff5700 0 client.19173 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 --2- 172.21.15.62:0/4106565299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dd44773480 0x55dd44770660 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4106565299 >> 172.21.15.62:0/4106565299 conn(0x55dd44763ff0 msgr2=0x55dd4477b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4106565299 shutdown_connections 2023-07-14T16:28:39.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 1 -- 172.21.15.62:0/4106565299 wait complete. 2023-07-14T16:28:39.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.652+0000 7f5eb916eb80 20 client.19173 trim_cache size 0 max 16384 2023-07-14T16:28:39.633 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 -- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e880a1270 msgr2=0x7f5e880a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e880a1270 0x7f5e880a3720 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5e90099d50 tx=0x7f5e90010b10 comp rx=0 tx=0).stop 2023-07-14T16:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 -- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea00ff9f0 msgr2=0x7f5ea01954b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea00ff9f0 0x7f5ea01954b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000d950 tx=0x7f5e9000dd10 comp rx=0 tx=0).stop 2023-07-14T16:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 -- 172.21.15.62:0/205524466 shutdown_connections 2023-07-14T16:28:39.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea0101630 0x7f5ea019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea00ff9f0 0x7f5ea01954b0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5e880a1270 0x7f5e880a3720 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 --2- 172.21.15.62:0/205524466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ea0100c80 0x7f5ea01959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 -- 172.21.15.62:0/205524466 >> 172.21.15.62:0/205524466 conn(0x7f5ea00fb010 msgr2=0x7f5ea00fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 -- 172.21.15.62:0/205524466 shutdown_connections 2023-07-14T16:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.653+0000 7f5e9ffff700 1 -- 172.21.15.62:0/205524466 wait complete. 2023-07-14T16:28:39.646 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:28:39.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.913+0000 7f9d331ff700 1 -- 172.21.15.62:0/1117192543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c1005c0 msgr2=0x7f9d2c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.913+0000 7f9d331ff700 1 --2- 172.21.15.62:0/1117192543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c1005c0 0x7f9d2c1029a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009990 tx=0x7f9d20009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.913+0000 7f9d331ff700 1 -- 172.21.15.62:0/1117192543 shutdown_connections 2023-07-14T16:28:39.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.913+0000 7f9d331ff700 1 --2- 172.21.15.62:0/1117192543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d2c105800 0x7f9d2c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.913+0000 7f9d331ff700 1 --2- 172.21.15.62:0/1117192543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c1005c0 0x7f9d2c1029a0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.913+0000 7f9d331ff700 1 --2- 172.21.15.62:0/1117192543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c102ee0 0x7f9d2c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.913+0000 7f9d331ff700 1 -- 172.21.15.62:0/1117192543 >> 172.21.15.62:0/1117192543 conn(0x7f9d2c0fa000 msgr2=0x7f9d2c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.914+0000 7f9d331ff700 1 -- 172.21.15.62:0/1117192543 shutdown_connections 2023-07-14T16:28:39.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.914+0000 7f9d331ff700 1 -- 172.21.15.62:0/1117192543 wait complete. 2023-07-14T16:28:39.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.915+0000 7f9d331ff700 1 Processor -- start 2023-07-14T16:28:39.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.915+0000 7f9d331ff700 1 -- start start 2023-07-14T16:28:39.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d331ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d2c1005c0 0x7f9d2c0fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d331ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c102ee0 0x7f9d2c0fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d331ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c105800 0x7f9d2c0feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d331ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0ff250 con 0x7f9d2c102ee0 2023-07-14T16:28:39.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d331ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0ff3c0 con 0x7f9d2c1005c0 2023-07-14T16:28:39.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d331ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c19f7d0 con 0x7f9d2c105800 2023-07-14T16:28:39.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c102ee0 0x7f9d2c0fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c102ee0 0x7f9d2c0fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35696/0 (socket says 172.21.15.62:35696) 2023-07-14T16:28:39.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d2bfff700 1 -- 172.21.15.62:0/4006371244 learned_addr learned my addr 172.21.15.62:0/4006371244 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d3179c700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c105800 0x7f9d2c0feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.916+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d2c1005c0 0x7f9d2c0fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d2bfff700 1 -- 172.21.15.62:0/4006371244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c105800 msgr2=0x7f9d2c0feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c105800 0x7f9d2c0feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d2bfff700 1 -- 172.21.15.62:0/4006371244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d2c1005c0 msgr2=0x7f9d2c0fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d2c1005c0 0x7f9d2c0fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d2bfff700 1 -- 172.21.15.62:0/4006371244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d200095f0 con 0x7f9d2c102ee0 2023-07-14T16:28:39.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c102ee0 0x7f9d2c0fe570 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800d980 tx=0x7f9d1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d29ffb700 1 -- 172.21.15.62:0/4006371244 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18004220 con 0x7f9d2c102ee0 2023-07-14T16:28:39.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d29ffb700 1 -- 172.21.15.62:0/4006371244 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d18007640 con 0x7f9d2c102ee0 2023-07-14T16:28:39.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d331ff700 1 -- 172.21.15.62:0/4006371244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c19fab0 con 0x7f9d2c102ee0 2023-07-14T16:28:39.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.917+0000 7f9d29ffb700 1 -- 172.21.15.62:0/4006371244 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18005660 con 0x7f9d2c102ee0 2023-07-14T16:28:39.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.918+0000 7f9d331ff700 1 -- 172.21.15.62:0/4006371244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c1a0010 con 0x7f9d2c102ee0 2023-07-14T16:28:39.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.922+0000 7f9d29ffb700 1 -- 172.21.15.62:0/4006371244 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9d18004750 con 0x7f9d2c102ee0 2023-07-14T16:28:39.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.923+0000 7f9d46af4b80 1 Processor -- start 2023-07-14T16:28:39.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.923+0000 7f9d46af4b80 1 -- start start 2023-07-14T16:28:39.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 0x55715bb438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715ba48cb0 con 0x55715bb44c70 2023-07-14T16:28:39.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715ba9ba40 con 0x55715bb43e20 2023-07-14T16:28:39.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715ba94ed0 con 0x55715bb434d0 2023-07-14T16:28:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d29ffb700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d1c0a11d0 0x7f9d1c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d29ffb700 1 -- 172.21.15.62:0/4006371244 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d180cc650 con 0x7f9d2c102ee0 2023-07-14T16:28:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d30f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d3179c700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d1c0a11d0 0x7f9d1c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d30f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:60860/0 (socket says 172.21.15.62:60860) 2023-07-14T16:28:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d30f9b700 1 -- 172.21.15.62:0/685404968 learned_addr learned my addr 172.21.15.62:0/685404968 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35706/0 (socket says 172.21.15.62:35706) 2023-07-14T16:28:39.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.924+0000 7f9d3179c700 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 0x55715bb438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d30f9b700 1 -- 172.21.15.62:0/685404968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 msgr2=0x55715bb438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 0x55715bb438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d30f9b700 1 -- 172.21.15.62:0/685404968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 msgr2=0x55715bb450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d30f9b700 1 -- 172.21.15.62:0/685404968 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55715b8c2de0 con 0x55715bb43e20 2023-07-14T16:28:39.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d3179c700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d1c0a11d0 0x7f9d1c0a3680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009550 tx=0x7f9d24009c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb446e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9d20015000 tx=0x7f9d20009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97ff6a9fa36719b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d16ffd700 1 -- 172.21.15.62:0/685404968 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2001d070 con 0x55715bb43e20 2023-07-14T16:28:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.925+0000 7f9d16ffd700 1 -- 172.21.15.62:0/685404968 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d2001e040 con 0x55715bb43e20 2023-07-14T16:28:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.926+0000 7f9d16ffd700 1 -- 172.21.15.62:0/685404968 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20004500 con 0x55715bb43e20 2023-07-14T16:28:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.926+0000 7f9d46af4b80 1 -- 172.21.15.62:0/685404968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 msgr2=0x55715bb446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.926+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb446e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9d20015000 tx=0x7f9d20009ee0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.926+0000 7f9d46af4b80 1 -- 172.21.15.62:0/685404968 shutdown_connections 2023-07-14T16:28:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.926+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.926+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 0x55715bb438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.926+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/685404968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb446e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.926+0000 7f9d46af4b80 1 -- 172.21.15.62:0/685404968 >> 172.21.15.62:0/685404968 conn(0x55715bb38da0 msgr2=0x55715bb3e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.927+0000 7f9d46af4b80 1 -- 172.21.15.62:0/685404968 shutdown_connections 2023-07-14T16:28:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.927+0000 7f9d46af4b80 1 -- 172.21.15.62:0/685404968 wait complete. 2023-07-14T16:28:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.927+0000 7f9d46af4b80 1 -- start start 2023-07-14T16:28:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.927+0000 7f9d46af4b80 1 Processor -- start 2023-07-14T16:28:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 0x55715bb4d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb4d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715ba34ac0 con 0x55715bb44c70 2023-07-14T16:28:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715ba347c0 con 0x55715bb43e20 2023-07-14T16:28:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715b918920 con 0x55715bb434d0 2023-07-14T16:28:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d46af4b80 10 client.0 _ll_register_callbacks cb 0x55715ba32760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d3179c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d3179c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35722/0 (socket says 172.21.15.62:35722) 2023-07-14T16:28:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d3179c700 1 -- 172.21.15.62:0/4131997166 learned_addr learned my addr 172.21.15.62:0/4131997166 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 0x55715bb4d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.928+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb4d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d3179c700 1 -- 172.21.15.62:0/4131997166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 msgr2=0x55715bb4d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d3179c700 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 0x55715bb4d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d3179c700 1 -- 172.21.15.62:0/4131997166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 msgr2=0x55715bb4d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d3179c700 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb4d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d3179c700 1 -- 172.21.15.62:0/4131997166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1809ae70 con 0x55715bb44c70 2023-07-14T16:28:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d3179c700 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb545c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9d24019f00 tx=0x7f9d24009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cfcb8a813b1dd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d0bfff700 1 -- 172.21.15.62:0/4131997166 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240095e0 con 0x55715bb44c70 2023-07-14T16:28:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d0bfff700 1 -- 172.21.15.62:0/4131997166 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d2401bdd0 con 0x55715bb44c70 2023-07-14T16:28:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.929+0000 7f9d46af4b80 1 -- 172.21.15.62:0/4131997166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55715b9ee4f0 con 0x55715bb44c70 2023-07-14T16:28:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d0bfff700 1 -- 172.21.15.62:0/4131997166 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d24032480 con 0x55715bb44c70 2023-07-14T16:28:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d46af4b80 10 client.19185 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d46af4b80 1 -- 172.21.15.62:0/4131997166 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f9d2c1087f0 con 0x55715bb44c70 2023-07-14T16:28:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d0bfff700 1 -- 172.21.15.62:0/4131997166 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55715b9ee4f0 con 0x55715bb44c70 2023-07-14T16:28:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d0bfff700 1 -- 172.21.15.62:0/4131997166 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9d2402f400 con 0x55715bb44c70 2023-07-14T16:28:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d46af4b80 10 client.19185 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d46af4b80 10 client.19185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d46af4b80 20 client.19185 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:39.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715bb58f10 0x55715bb5b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d46af4b80 4 client.19185 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:39.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.930+0000 7f9d46af4b80 1 -- 172.21.15.62:0/4131997166 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55715ba8deb0 con 0x55715bb58f10 2023-07-14T16:28:39.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.931+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715bb58f10 0x55715bb5b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.931+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715bb58f10 0x55715bb5b320 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3778dcf2e2343d3 server_cookie=f36446d8ab410fe0 in_seq=0 out_seq=0 2023-07-14T16:28:39.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.931+0000 7f9d0bfff700 10 client.19185 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:39.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d0bfff700 1 -- 172.21.15.62:0/4131997166 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55715ba8deb0 con 0x55715bb58f10 2023-07-14T16:28:39.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d0bfff700 10 client.19185 handle_command_reply: tid=0 2023-07-14T16:28:39.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 client.19185 shutdown 2023-07-14T16:28:39.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 -- 172.21.15.62:0/4131997166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 msgr2=0x55715bb545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb545c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9d24019f00 tx=0x7f9d24009100 comp rx=0 tx=0).stop 2023-07-14T16:28:39.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 -- 172.21.15.62:0/4131997166 shutdown_connections 2023-07-14T16:28:39.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb44c70 0x55715bb545c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb434d0 0x55715bb4d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715bb58f10 0x55715bb5b320 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/4131997166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb4d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.932+0000 7f9d46af4b80 1 -- 172.21.15.62:0/4131997166 >> 172.21.15.62:0/4131997166 conn(0x55715bb38da0 msgr2=0x55715bb3d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.933+0000 7f9d0bfff700 0 client.19185 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:39.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.933+0000 7f9d46af4b80 1 -- 172.21.15.62:0/4131997166 shutdown_connections 2023-07-14T16:28:39.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.933+0000 7f9d46af4b80 1 -- 172.21.15.62:0/4131997166 wait complete. 2023-07-14T16:28:39.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.933+0000 7f9d46af4b80 20 client.19185 trim_cache size 0 max 16384 2023-07-14T16:28:39.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d46af4b80 1 Processor -- start 2023-07-14T16:28:39.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d46af4b80 1 -- start start 2023-07-14T16:28:39.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb3a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb44c70 0x55715bb5fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715bb3a890 con 0x55715bb434d0 2023-07-14T16:28:39.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715bb601b0 con 0x55715bb43e20 2023-07-14T16:28:39.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715bb4cdf0 con 0x55715bb44c70 2023-07-14T16:28:39.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d30f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb3a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d30f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb3a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35724/0 (socket says 172.21.15.62:35724) 2023-07-14T16:28:39.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.952+0000 7f9d30f9b700 1 -- 172.21.15.62:0/1888822727 learned_addr learned my addr 172.21.15.62:0/1888822727 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d3179c700 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb44c70 0x55715bb5fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d30f9b700 1 -- 172.21.15.62:0/1888822727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb44c70 msgr2=0x55715bb5fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb44c70 0x55715bb5fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d30f9b700 1 -- 172.21.15.62:0/1888822727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 msgr2=0x55715bb544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d30f9b700 1 -- 172.21.15.62:0/1888822727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55715b8abc60 con 0x55715bb434d0 2023-07-14T16:28:39.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb3a310 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000c010 tx=0x7f9d20005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f1f456bffae346 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.953+0000 7f9d177fe700 1 -- 172.21.15.62:0/1888822727 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20032040 con 0x55715bb434d0 2023-07-14T16:28:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d177fe700 1 -- 172.21.15.62:0/1888822727 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d2000bde0 con 0x55715bb434d0 2023-07-14T16:28:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d177fe700 1 -- 172.21.15.62:0/1888822727 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2001d040 con 0x55715bb434d0 2023-07-14T16:28:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d46af4b80 1 -- 172.21.15.62:0/1888822727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 msgr2=0x55715bb3a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb3a310 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000c010 tx=0x7f9d20005fb0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d46af4b80 1 -- 172.21.15.62:0/1888822727 shutdown_connections 2023-07-14T16:28:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb44c70 0x55715bb5fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb3a310 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/1888822727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb43e20 0x55715bb544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d46af4b80 1 -- 172.21.15.62:0/1888822727 >> 172.21.15.62:0/1888822727 conn(0x55715bb38da0 msgr2=0x55715bb3e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.954+0000 7f9d46af4b80 1 -- 172.21.15.62:0/1888822727 shutdown_connections 2023-07-14T16:28:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.955+0000 7f9d46af4b80 1 -- 172.21.15.62:0/1888822727 wait complete. 2023-07-14T16:28:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.955+0000 7f9d46af4b80 1 -- start start 2023-07-14T16:28:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.955+0000 7f9d46af4b80 1 Processor -- start 2023-07-14T16:28:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb50150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb43e20 0x55715bb50690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d46af4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb44c70 0x55715bb536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715bb53bf0 con 0x55715bb434d0 2023-07-14T16:28:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715bb53d60 con 0x55715bb44c70 2023-07-14T16:28:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d46af4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55715bb53f20 con 0x55715bb43e20 2023-07-14T16:28:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d46af4b80 10 client.0 _ll_register_callbacks cb 0x7f9d24019ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:39.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d30f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb50150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d30f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb50150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35728/0 (socket says 172.21.15.62:35728) 2023-07-14T16:28:39.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d30f9b700 1 -- 172.21.15.62:0/3501019997 learned_addr learned my addr 172.21.15.62:0/3501019997 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:39.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb43e20 0x55715bb50690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d3179c700 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb44c70 0x55715bb536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d30f9b700 1 -- 172.21.15.62:0/3501019997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb43e20 msgr2=0x55715bb50690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb43e20 0x55715bb50690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d30f9b700 1 -- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb44c70 msgr2=0x55715bb536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:39.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb44c70 0x55715bb536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.956+0000 7f9d30f9b700 1 -- 172.21.15.62:0/3501019997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55715b8c2de0 con 0x55715bb434d0 2023-07-14T16:28:39.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d3179c700 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb44c70 0x55715bb536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:39.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d30f9b700 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb50150 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009990 tx=0x7f9d2000b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beed7e9cb888c8a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:39.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:39.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d0bfff700 1 -- 172.21.15.62:0/3501019997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2001d070 con 0x55715bb434d0 2023-07-14T16:28:39.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d46af4b80 1 -- 172.21.15.62:0/3501019997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55715ba50050 con 0x55715bb434d0 2023-07-14T16:28:39.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d0bfff700 1 -- 172.21.15.62:0/3501019997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d20035a70 con 0x55715bb434d0 2023-07-14T16:28:39.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d0bfff700 1 -- 172.21.15.62:0/3501019997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d20046030 con 0x55715bb434d0 2023-07-14T16:28:39.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d46af4b80 10 client.19191 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:39.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.957+0000 7f9d46af4b80 1 -- 172.21.15.62:0/3501019997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55715b8e9250 con 0x55715bb434d0 2023-07-14T16:28:39.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d0bfff700 1 -- 172.21.15.62:0/3501019997 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55715ba50050 con 0x55715bb434d0 2023-07-14T16:28:39.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d0bfff700 1 -- 172.21.15.62:0/3501019997 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9d2003e070 con 0x55715bb434d0 2023-07-14T16:28:39.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d46af4b80 10 client.19191 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:39.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d46af4b80 10 client.19191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:39.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d46af4b80 20 client.19191 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:39.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715bbad7a0 0x55715bbafbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:39.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d46af4b80 4 client.19191 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:28:39.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d46af4b80 1 -- 172.21.15.62:0/3501019997 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55715b8e3f70 con 0x55715bbad7a0 2023-07-14T16:28:39.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.958+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715bbad7a0 0x55715bbafbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:39.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.959+0000 7f9d2bfff700 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715bbad7a0 0x55715bbafbb0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb437b444998cea4 server_cookie=5037b15c2fdcb179 in_seq=0 out_seq=0 2023-07-14T16:28:39.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.959+0000 7f9d0bfff700 10 client.19191 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:39.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.959+0000 7f9d0bfff700 1 -- 172.21.15.62:0/3501019997 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55715b8e3f70 con 0x55715bbad7a0 2023-07-14T16:28:39.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.959+0000 7f9d0bfff700 10 client.19191 handle_command_reply: tid=0 2023-07-14T16:28:39.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.959+0000 7f9d46af4b80 1 client.19191 shutdown 2023-07-14T16:28:39.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 -- 172.21.15.62:0/3501019997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 msgr2=0x55715bb50150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb50150 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009990 tx=0x7f9d2000b5c0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 -- 172.21.15.62:0/3501019997 shutdown_connections 2023-07-14T16:28:39.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55715bbad7a0 0x55715bbafbb0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55715bb44c70 0x55715bb536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55715bb434d0 0x55715bb50150 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d0bfff700 0 client.19191 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:39.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 --2- 172.21.15.62:0/3501019997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55715bb43e20 0x55715bb50690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 -- 172.21.15.62:0/3501019997 >> 172.21.15.62:0/3501019997 conn(0x55715bb38da0 msgr2=0x55715bb47b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 -- 172.21.15.62:0/3501019997 shutdown_connections 2023-07-14T16:28:39.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 1 -- 172.21.15.62:0/3501019997 wait complete. 2023-07-14T16:28:39.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.960+0000 7f9d46af4b80 20 client.19191 trim_cache size 0 max 16384 2023-07-14T16:28:39.997 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:28:39.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 -- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d1c0a11d0 msgr2=0x7f9d1c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d1c0a11d0 0x7f9d1c0a3680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009550 tx=0x7f9d24009c10 comp rx=0 tx=0).stop 2023-07-14T16:28:39.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 -- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c102ee0 msgr2=0x7f9d2c0fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:39.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c102ee0 0x7f9d2c0fe570 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800d980 tx=0x7f9d1800dc90 comp rx=0 tx=0).stop 2023-07-14T16:28:39.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 -- 172.21.15.62:0/4006371244 shutdown_connections 2023-07-14T16:28:39.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d2c105800 0x7f9d2c0feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d1c0a11d0 0x7f9d1c0a3680 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9d2c1005c0 0x7f9d2c0fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 --2- 172.21.15.62:0/4006371244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d2c102ee0 0x7f9d2c0fe570 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:39.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 -- 172.21.15.62:0/4006371244 >> 172.21.15.62:0/4006371244 conn(0x7f9d2c0fa000 msgr2=0x7f9d2c101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:39.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 -- 172.21.15.62:0/4006371244 shutdown_connections 2023-07-14T16:28:39.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:39.961+0000 7f9d331ff700 1 -- 172.21.15.62:0/4006371244 wait complete. 2023-07-14T16:28:40.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:39 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2529752624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:28:40.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:39 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2529752624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:28:40.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:39 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2529752624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:28:41.000 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:28:41.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:28:41.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:40 smithi062 ceph-mon[95543]: pgmap v748: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:28:41.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:40 smithi081 ceph-mon[127795]: pgmap v748: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:28:41.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 -- 172.21.15.62:0/3556196479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a0100560 msgr2=0x7fb5a0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/3556196479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a0100560 0x7fb5a0102940 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb594009990 tx=0x7fb594009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 -- 172.21.15.62:0/3556196479 shutdown_connections 2023-07-14T16:28:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/3556196479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a01057a0 0x7fb5a0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/3556196479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a0100560 0x7fb5a0102940 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/3556196479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5a0102e80 0x7fb5a0105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 -- 172.21.15.62:0/3556196479 >> 172.21.15.62:0/3556196479 conn(0x7fb5a00fa000 msgr2=0x7fb5a00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 -- 172.21.15.62:0/3556196479 shutdown_connections 2023-07-14T16:28:41.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.260+0000 7fb5a71ff700 1 -- 172.21.15.62:0/3556196479 wait complete. 2023-07-14T16:28:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.261+0000 7fb5a71ff700 1 Processor -- start 2023-07-14T16:28:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.261+0000 7fb5a71ff700 1 -- start start 2023-07-14T16:28:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a71ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0100560 0x7fb5a0197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a71ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5a0102e80 0x7fb5a0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a71ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a01057a0 0x7fb5a019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a71ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019f500 con 0x7fb5a01057a0 2023-07-14T16:28:41.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a71ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019f640 con 0x7fb5a0102e80 2023-07-14T16:28:41.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a71ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019f7d0 con 0x7fb5a0100560 2023-07-14T16:28:41.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a01057a0 0x7fb5a019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5a0102e80 0x7fb5a0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5a0102e80 0x7fb5a0197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:60910/0 (socket says 172.21.15.62:60910) 2023-07-14T16:28:41.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a01057a0 0x7fb5a019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35736/0 (socket says 172.21.15.62:35736) 2023-07-14T16:28:41.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb59ffff700 1 -- 172.21.15.62:0/1653574836 learned_addr learned my addr 172.21.15.62:0/1653574836 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.262+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0100560 0x7fb5a0197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb5a579c700 1 -- 172.21.15.62:0/1653574836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0100560 msgr2=0x7fb5a0197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb5a579c700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0100560 0x7fb5a0197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb5a579c700 1 -- 172.21.15.62:0/1653574836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5a0102e80 msgr2=0x7fb5a0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb5a579c700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5a0102e80 0x7fb5a0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb5a579c700 1 -- 172.21.15.62:0/1653574836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940095f0 con 0x7fb5a01057a0 2023-07-14T16:28:41.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb5a579c700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a01057a0 0x7fb5a019ef30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb59800efc0 tx=0x7fb59800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb59dffb700 1 -- 172.21.15.62:0/1653574836 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598018070 con 0x7fb5a01057a0 2023-07-14T16:28:41.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb59dffb700 1 -- 172.21.15.62:0/1653574836 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb59800f040 con 0x7fb5a01057a0 2023-07-14T16:28:41.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb5a71ff700 1 -- 172.21.15.62:0/1653574836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a019fa50 con 0x7fb5a01057a0 2023-07-14T16:28:41.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb59dffb700 1 -- 172.21.15.62:0/1653574836 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59800cba0 con 0x7fb5a01057a0 2023-07-14T16:28:41.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.263+0000 7fb5a71ff700 1 -- 172.21.15.62:0/1653574836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a019ff60 con 0x7fb5a01057a0 2023-07-14T16:28:41.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.267+0000 7fb59dffb700 1 -- 172.21.15.62:0/1653574836 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb598003a00 con 0x7fb5a01057a0 2023-07-14T16:28:41.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.268+0000 7fb5bac0ab80 1 Processor -- start 2023-07-14T16:28:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb5bac0ab80 1 -- start start 2023-07-14T16:28:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb59dffb700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5900a1150 0x7fb5900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x55890456a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x55890456b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x55890456c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890446fd30 con 0x55890456bcf0 2023-07-14T16:28:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589044c2ac0 con 0x55890456aea0 2023-07-14T16:28:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589044bbf50 con 0x55890456a550 2023-07-14T16:28:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.269+0000 7fb59dffb700 1 -- 172.21.15.62:0/1653574836 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb5980cc370 con 0x7fb5a01057a0 2023-07-14T16:28:41.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb59ffff700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5900a1150 0x7fb5900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a4f9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x55890456a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a4f9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x55890456a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55536/0 (socket says 172.21.15.62:55536) 2023-07-14T16:28:41.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/2107179866 learned_addr learned my addr 172.21.15.62:0/2107179866 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a579c700 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x55890456c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb59ffff700 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x55890456b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/2107179866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 msgr2=0x55890456b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x55890456b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/2107179866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 msgr2=0x55890456c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x55890456c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/2107179866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589042e9e60 con 0x55890456a550 2023-07-14T16:28:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb59ffff700 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x55890456b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.270+0000 7fb59ffff700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5900a1150 0x7fb5900a3600 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb58c007e40 tx=0x7fb58c0078a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.271+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x55890456a960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb594009990 tx=0x7fb594005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b0f07a3e374768 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.271+0000 7fb58affd700 1 -- 172.21.15.62:0/2107179866 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59401d070 con 0x55890456a550 2023-07-14T16:28:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.271+0000 7fb58affd700 1 -- 172.21.15.62:0/2107179866 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb59401e040 con 0x55890456a550 2023-07-14T16:28:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.271+0000 7fb58affd700 1 -- 172.21.15.62:0/2107179866 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400b710 con 0x55890456a550 2023-07-14T16:28:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/2107179866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 msgr2=0x55890456a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x55890456a960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb594009990 tx=0x7fb594005f00 comp rx=0 tx=0).stop 2023-07-14T16:28:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/2107179866 shutdown_connections 2023-07-14T16:28:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x55890456c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x55890456a960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/2107179866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x55890456b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/2107179866 >> 172.21.15.62:0/2107179866 conn(0x55890455fe20 msgr2=0x558904565250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/2107179866 shutdown_connections 2023-07-14T16:28:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.272+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/2107179866 wait complete. 2023-07-14T16:28:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 1 -- start start 2023-07-14T16:28:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 1 Processor -- start 2023-07-14T16:28:41.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x5589045740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456aea0 0x5589045745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456bcf0 0x55890457b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890445bb40 con 0x55890456aea0 2023-07-14T16:28:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890445b840 con 0x55890456bcf0 2023-07-14T16:28:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890433f9a0 con 0x55890456a550 2023-07-14T16:28:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.273+0000 7fb5bac0ab80 10 client.0 _ll_register_callbacks cb 0x5589044597e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456aea0 0x5589045745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456aea0 0x5589045745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35754/0 (socket says 172.21.15.62:35754) 2023-07-14T16:28:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb59ffff700 1 -- 172.21.15.62:0/82001691 learned_addr learned my addr 172.21.15.62:0/82001691 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb5a579c700 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456bcf0 0x55890457b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x5589045740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb59ffff700 1 -- 172.21.15.62:0/82001691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 msgr2=0x5589045740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb59ffff700 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x5589045740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb59ffff700 1 -- 172.21.15.62:0/82001691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456bcf0 msgr2=0x55890457b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb59ffff700 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456bcf0 0x55890457b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.274+0000 7fb59ffff700 1 -- 172.21.15.62:0/82001691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a01087f0 con 0x55890456aea0 2023-07-14T16:28:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb59ffff700 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456aea0 0x5589045745f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00f950 tx=0x7fb58c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a64222fde71d531 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb57ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb57ffff700 1 -- 172.21.15.62:0/82001691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c00e860 con 0x55890456aea0 2023-07-14T16:28:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/82001691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558904415570 con 0x55890456aea0 2023-07-14T16:28:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb57ffff700 1 -- 172.21.15.62:0/82001691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb58c00e9c0 con 0x55890456aea0 2023-07-14T16:28:41.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb57ffff700 1 -- 172.21.15.62:0/82001691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb58c02d030 con 0x55890456aea0 2023-07-14T16:28:41.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb5bac0ab80 10 client.19203 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:41.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/82001691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5589042d2ce0 con 0x55890456aea0 2023-07-14T16:28:41.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.275+0000 7fb57ffff700 1 -- 172.21.15.62:0/82001691 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558904415570 con 0x55890456aea0 2023-07-14T16:28:41.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.276+0000 7fb57ffff700 1 -- 172.21.15.62:0/82001691 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb58c025020 con 0x55890456aea0 2023-07-14T16:28:41.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.276+0000 7fb5bac0ab80 10 client.19203 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:41.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.276+0000 7fb5bac0ab80 10 client.19203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:41.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.276+0000 7fb5bac0ab80 20 client.19203 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:41.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.276+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55890457ff00 0x558904582310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.276+0000 7fb5bac0ab80 4 client.19203 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:41.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.276+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/82001691 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589044b4f30 con 0x55890457ff00 2023-07-14T16:28:41.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.276+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55890457ff00 0x558904582310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.277+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55890457ff00 0x558904582310 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28330f78bfadcedd server_cookie=712f61a3f6aae1c0 in_seq=0 out_seq=0 2023-07-14T16:28:41.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.277+0000 7fb57ffff700 10 client.19203 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb57ffff700 1 -- 172.21.15.62:0/82001691 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5589044b4f30 con 0x55890457ff00 2023-07-14T16:28:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb57ffff700 10 client.19203 handle_command_reply: tid=0 2023-07-14T16:28:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 client.19203 shutdown 2023-07-14T16:28:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/82001691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456aea0 msgr2=0x5589045745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456aea0 0x5589045745f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00f950 tx=0x7fb58c0047c0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/82001691 shutdown_connections 2023-07-14T16:28:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55890457ff00 0x558904582310 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456bcf0 0x55890457b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456a550 0x5589045740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/82001691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456aea0 0x5589045745f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/82001691 >> 172.21.15.62:0/82001691 conn(0x55890455fe20 msgr2=0x558904564930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb57ffff700 0 client.19203 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:41.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/82001691 shutdown_connections 2023-07-14T16:28:41.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/82001691 wait complete. 2023-07-14T16:28:41.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.278+0000 7fb5bac0ab80 20 client.19203 trim_cache size 0 max 16384 2023-07-14T16:28:41.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:40 smithi164 ceph-mon[97427]: pgmap v748: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:28:41.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.300+0000 7fb5bac0ab80 1 Processor -- start 2023-07-14T16:28:41.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.300+0000 7fb5bac0ab80 1 -- start start 2023-07-14T16:28:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.300+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456a550 0x558904581f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.300+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456aea0 0x55890457b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.300+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x5589045720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.300+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558904573c20 con 0x55890456bcf0 2023-07-14T16:28:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.300+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558904566d40 con 0x55890456a550 2023-07-14T16:28:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.300+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558904566360 con 0x55890456aea0 2023-07-14T16:28:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456a550 0x558904581f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x5589045720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456a550 0x558904581f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:60936/0 (socket says 172.21.15.62:60936) 2023-07-14T16:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/3419952185 learned_addr learned my addr 172.21.15.62:0/3419952185 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb59ffff700 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456aea0 0x55890457b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/3419952185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456aea0 msgr2=0x55890457b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456aea0 0x55890457b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/3419952185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 msgr2=0x5589045720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x5589045720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/3419952185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558904573170 con 0x55890456a550 2023-07-14T16:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a579c700 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x5589045720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:41.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.301+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456a550 0x558904581f10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb594009990 tx=0x7fb59401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22e5cea3af59f3f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.302+0000 7fb58b7fe700 1 -- 172.21.15.62:0/3419952185 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59401d040 con 0x55890456a550 2023-07-14T16:28:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.302+0000 7fb58b7fe700 1 -- 172.21.15.62:0/3419952185 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5940355e0 con 0x55890456a550 2023-07-14T16:28:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.302+0000 7fb58b7fe700 1 -- 172.21.15.62:0/3419952185 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594035cb0 con 0x55890456a550 2023-07-14T16:28:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/3419952185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456a550 msgr2=0x558904581f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456a550 0x558904581f10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb594009990 tx=0x7fb59401a040 comp rx=0 tx=0).stop 2023-07-14T16:28:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/3419952185 shutdown_connections 2023-07-14T16:28:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456bcf0 0x5589045720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456a550 0x558904581f10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/3419952185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456aea0 0x55890457b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/3419952185 >> 172.21.15.62:0/3419952185 conn(0x55890455fe20 msgr2=0x55890457be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/3419952185 shutdown_connections 2023-07-14T16:28:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/3419952185 wait complete. 2023-07-14T16:28:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.303+0000 7fb5bac0ab80 1 -- start start 2023-07-14T16:28:41.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5bac0ab80 1 Processor -- start 2023-07-14T16:28:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456a550 0x55890457b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x558904577c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5bac0ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456bcf0 0x5589045781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890457b9f0 con 0x55890456a550 2023-07-14T16:28:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589045786f0 con 0x55890456aea0 2023-07-14T16:28:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5bac0ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589045788b0 con 0x55890456bcf0 2023-07-14T16:28:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5bac0ab80 10 client.0 _ll_register_callbacks cb 0x7fb58c00f530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:41.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.304+0000 7fb5a4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456a550 0x55890457b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456a550 0x55890457b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35780/0 (socket says 172.21.15.62:35780) 2023-07-14T16:28:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/750106646 learned_addr learned my addr 172.21.15.62:0/750106646 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb59ffff700 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x558904577c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a579c700 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456bcf0 0x5589045781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/750106646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456bcf0 msgr2=0x5589045781b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456bcf0 0x5589045781b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/750106646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 msgr2=0x558904577c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x558904577c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a4f9b700 1 -- 172.21.15.62:0/750106646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a019ff60 con 0x55890456a550 2023-07-14T16:28:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb5a4f9b700 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456a550 0x55890457b1d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb594003f60 tx=0x7fb594003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f6625d9590b0ff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb57ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.305+0000 7fb57ffff700 1 -- 172.21.15.62:0/750106646 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59401d030 con 0x55890456a550 2023-07-14T16:28:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/750106646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb59401e040 con 0x55890456a550 2023-07-14T16:28:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb57ffff700 1 -- 172.21.15.62:0/750106646 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb594032040 con 0x55890456a550 2023-07-14T16:28:41.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb57ffff700 1 -- 172.21.15.62:0/750106646 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb594035460 con 0x55890456a550 2023-07-14T16:28:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 10 client.19209 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/750106646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5589042e9e60 con 0x55890456a550 2023-07-14T16:28:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb57ffff700 1 -- 172.21.15.62:0/750106646 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb59401e040 con 0x55890456a550 2023-07-14T16:28:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb57ffff700 1 -- 172.21.15.62:0/750106646 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb59403d020 con 0x55890456a550 2023-07-14T16:28:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 10 client.19209 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 10 client.19209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 20 client.19209 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589045d5170 0x5589045d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 4 client.19209 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:28:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.306+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/750106646 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5589044770d0 con 0x5589045d5170 2023-07-14T16:28:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.307+0000 7fb59ffff700 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589045d5170 0x5589045d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.307+0000 7fb59ffff700 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589045d5170 0x5589045d7580 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a4b4c2c01f73fa server_cookie=9a5117f01ab366db in_seq=0 out_seq=0 2023-07-14T16:28:41.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.307+0000 7fb57ffff700 10 client.19209 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:41.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb57ffff700 1 -- 172.21.15.62:0/750106646 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5589044770d0 con 0x5589045d5170 2023-07-14T16:28:41.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb57ffff700 10 client.19209 handle_command_reply: tid=0 2023-07-14T16:28:41.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 client.19209 shutdown 2023-07-14T16:28:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/750106646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456a550 msgr2=0x55890457b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456a550 0x55890457b1d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb594003f60 tx=0x7fb594003e60 comp rx=0 tx=0).stop 2023-07-14T16:28:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/750106646 shutdown_connections 2023-07-14T16:28:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55890456bcf0 0x5589045781b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55890456a550 0x55890457b1d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5589045d5170 0x5589045d7580 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 --2- 172.21.15.62:0/750106646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55890456aea0 0x558904577c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/750106646 >> 172.21.15.62:0/750106646 conn(0x55890455fe20 msgr2=0x55890457be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb57ffff700 0 client.19209 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/750106646 shutdown_connections 2023-07-14T16:28:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.309+0000 7fb5bac0ab80 1 -- 172.21.15.62:0/750106646 wait complete. 2023-07-14T16:28:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5bac0ab80 20 client.19209 trim_cache size 0 max 16384 2023-07-14T16:28:41.291 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:28:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:28:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "fc51b5fd-1b0d-4079-9682-a08e1f6e6c22", 2023-07-14T16:28:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:28:41.292 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:28:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 -- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5900a1150 msgr2=0x7fb5900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5900a1150 0x7fb5900a3600 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb58c007e40 tx=0x7fb58c0078a0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 -- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a01057a0 msgr2=0x7fb5a019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a01057a0 0x7fb5a019ef30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb59800efc0 tx=0x7fb59800c3a0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 -- 172.21.15.62:0/1653574836 shutdown_connections 2023-07-14T16:28:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5a01057a0 0x7fb5a019ef30 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5a0100560 0x7fb5a0197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb5900a1150 0x7fb5900a3600 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 --2- 172.21.15.62:0/1653574836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5a0102e80 0x7fb5a0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 -- 172.21.15.62:0/1653574836 >> 172.21.15.62:0/1653574836 conn(0x7fb5a00fa000 msgr2=0x7fb5a00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 -- 172.21.15.62:0/1653574836 shutdown_connections 2023-07-14T16:28:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.310+0000 7fb5a71ff700 1 -- 172.21.15.62:0/1653574836 wait complete. 2023-07-14T16:28:41.307 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:28:41.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 -- 172.21.15.62:0/1877865753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 msgr2=0x7f7334073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 --2- 172.21.15.62:0/1877865753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 0x7f7334073360 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7328009990 tx=0x7f7328009ca0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 -- 172.21.15.62:0/1877865753 shutdown_connections 2023-07-14T16:28:41.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 --2- 172.21.15.62:0/1877865753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73341079a0 0x7f7334109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 --2- 172.21.15.62:0/1877865753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73340744d0 0x7f7334072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 --2- 172.21.15.62:0/1877865753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 0x7f7334073360 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 -- 172.21.15.62:0/1877865753 >> 172.21.15.62:0/1877865753 conn(0x7f73340fb130 msgr2=0x7f73340fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 -- 172.21.15.62:0/1877865753 shutdown_connections 2023-07-14T16:28:41.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.570+0000 7f73393ff700 1 -- 172.21.15.62:0/1877865753 wait complete. 2023-07-14T16:28:41.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.571+0000 7f73393ff700 1 Processor -- start 2023-07-14T16:28:41.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.571+0000 7f73393ff700 1 -- start start 2023-07-14T16:28:41.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f73393ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 0x7f7334197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f73393ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73340744d0 0x7f7334198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f73393ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73341079a0 0x7f733419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f73393ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733419f670 con 0x7f7334072ef0 2023-07-14T16:28:41.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f73393ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733419f7b0 con 0x7f73341079a0 2023-07-14T16:28:41.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f73393ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733419f940 con 0x7f73340744d0 2023-07-14T16:28:41.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f7332ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 0x7f7334197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f7332ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 0x7f7334197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35802/0 (socket says 172.21.15.62:35802) 2023-07-14T16:28:41.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f7332ffd700 1 -- 172.21.15.62:0/1671881443 learned_addr learned my addr 172.21.15.62:0/1671881443 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f73337fe700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73341079a0 0x7f733419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.572+0000 7f73327fc700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73340744d0 0x7f7334198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f7332ffd700 1 -- 172.21.15.62:0/1671881443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73340744d0 msgr2=0x7f7334198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f7332ffd700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73340744d0 0x7f7334198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f7332ffd700 1 -- 172.21.15.62:0/1671881443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73341079a0 msgr2=0x7f733419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f7332ffd700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73341079a0 0x7f733419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f7332ffd700 1 -- 172.21.15.62:0/1671881443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280095f0 con 0x7f7334072ef0 2023-07-14T16:28:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f7332ffd700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 0x7f7334197af0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f732400b530 tx=0x7f732400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f731bfff700 1 -- 172.21.15.62:0/1671881443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324011830 con 0x7f7334072ef0 2023-07-14T16:28:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f731bfff700 1 -- 172.21.15.62:0/1671881443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7324011e20 con 0x7f7334072ef0 2023-07-14T16:28:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f731bfff700 1 -- 172.21.15.62:0/1671881443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732400f2e0 con 0x7f7334072ef0 2023-07-14T16:28:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.573+0000 7f73393ff700 1 -- 172.21.15.62:0/1671881443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733419fc20 con 0x7f7334072ef0 2023-07-14T16:28:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.574+0000 7f73393ff700 1 -- 172.21.15.62:0/1671881443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73341a0130 con 0x7f7334072ef0 2023-07-14T16:28:41.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.578+0000 7f731bfff700 1 -- 172.21.15.62:0/1671881443 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7324011990 con 0x7f7334072ef0 2023-07-14T16:28:41.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.578+0000 7f734ce15b80 1 Processor -- start 2023-07-14T16:28:41.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f734ce15b80 1 -- start start 2023-07-14T16:28:41.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa475504d0 0x55aa475508e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa475516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa475520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f734ce15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa47455cb0 con 0x55aa47551c70 2023-07-14T16:28:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f734ce15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa474a8a40 con 0x55aa47550e20 2023-07-14T16:28:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f734ce15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa474a1ed0 con 0x55aa475504d0 2023-07-14T16:28:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f73337fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa475504d0 0x55aa475508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f73337fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa475504d0 0x55aa475508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:55592/0 (socket says 172.21.15.62:55592) 2023-07-14T16:28:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.579+0000 7f73337fe700 1 -- 172.21.15.62:0/2678075151 learned_addr learned my addr 172.21.15.62:0/2678075151 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f73327fc700 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa475520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f7332ffd700 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa475516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f73337fe700 1 -- 172.21.15.62:0/2678075151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 msgr2=0x55aa475516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f73337fe700 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa475516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f73337fe700 1 -- 172.21.15.62:0/2678075151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 msgr2=0x55aa475520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f73337fe700 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa475520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f73337fe700 1 -- 172.21.15.62:0/2678075151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa472cfde0 con 0x55aa475504d0 2023-07-14T16:28:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f7332ffd700 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa475516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:28:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f731bfff700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f731c0a11d0 0x7f731c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f73337fe700 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa475504d0 0x55aa475508e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f732c00a430 tx=0x7f732c009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b248a99337ca4334 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f731bfff700 1 -- 172.21.15.62:0/1671881443 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f73240cc3f0 con 0x7f7334072ef0 2023-07-14T16:28:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.580+0000 7f7332ffd700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f731c0a11d0 0x7f731c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.581+0000 7f73197fa700 1 -- 172.21.15.62:0/2678075151 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c010040 con 0x55aa475504d0 2023-07-14T16:28:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.581+0000 7f73197fa700 1 -- 172.21.15.62:0/2678075151 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f732c014ab0 con 0x55aa475504d0 2023-07-14T16:28:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.581+0000 7f73197fa700 1 -- 172.21.15.62:0/2678075151 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c013950 con 0x55aa475504d0 2023-07-14T16:28:41.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 -- 172.21.15.62:0/2678075151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa475504d0 msgr2=0x55aa475508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f7332ffd700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f731c0a11d0 0x7f731c0a3680 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f73240994b0 tx=0x7f7324011f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa475504d0 0x55aa475508e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f732c00a430 tx=0x7f732c009650 comp rx=0 tx=0).stop 2023-07-14T16:28:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 -- 172.21.15.62:0/2678075151 shutdown_connections 2023-07-14T16:28:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa475520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa475504d0 0x55aa475508e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 --2- 172.21.15.62:0/2678075151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa475516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 -- 172.21.15.62:0/2678075151 >> 172.21.15.62:0/2678075151 conn(0x55aa47545da0 msgr2=0x55aa4754b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 -- 172.21.15.62:0/2678075151 shutdown_connections 2023-07-14T16:28:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.582+0000 7f734ce15b80 1 -- 172.21.15.62:0/2678075151 wait complete. 2023-07-14T16:28:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 1 -- start start 2023-07-14T16:28:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 1 Processor -- start 2023-07-14T16:28:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa475504d0 0x55aa4755a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa4755a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa47561690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa47441ac0 con 0x55aa47551c70 2023-07-14T16:28:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa474417c0 con 0x55aa475504d0 2023-07-14T16:28:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa47325920 con 0x55aa47550e20 2023-07-14T16:28:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.583+0000 7f734ce15b80 10 client.0 _ll_register_callbacks cb 0x55aa4743f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa47561690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa47561690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35810/0 (socket says 172.21.15.62:35810) 2023-07-14T16:28:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73327fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa4755a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73337fe700 1 -- 172.21.15.62:0/972328123 learned_addr learned my addr 172.21.15.62:0/972328123 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73337fe700 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa475504d0 0x55aa4755a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73327fc700 1 -- 172.21.15.62:0/972328123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa475504d0 msgr2=0x55aa4755a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73327fc700 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa475504d0 0x55aa4755a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73327fc700 1 -- 172.21.15.62:0/972328123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 msgr2=0x55aa47561690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73327fc700 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa47561690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73327fc700 1 -- 172.21.15.62:0/972328123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c009c60 con 0x55aa47550e20 2023-07-14T16:28:41.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.584+0000 7f73337fe700 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa47561690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:41.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.585+0000 7f73327fc700 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa4755a650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7328000c00 tx=0x7f7328005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27aae8016c2080ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.585+0000 7f7316ff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:28:41.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.585+0000 7f7316ff5700 1 -- 172.21.15.62:0/972328123 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328005570 con 0x55aa47550e20 2023-07-14T16:28:41.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.585+0000 7f7316ff5700 1 -- 172.21.15.62:0/972328123 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7328004500 con 0x55aa47550e20 2023-07-14T16:28:41.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.585+0000 7f734ce15b80 1 -- 172.21.15.62:0/972328123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa473fb4f0 con 0x55aa47550e20 2023-07-14T16:28:41.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.585+0000 7f7316ff5700 1 -- 172.21.15.62:0/972328123 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73280056d0 con 0x55aa47550e20 2023-07-14T16:28:41.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.585+0000 7f734ce15b80 10 client.28669 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:41.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.585+0000 7f734ce15b80 1 -- 172.21.15.62:0/972328123 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aa472b8c60 con 0x55aa47550e20 2023-07-14T16:28:41.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.586+0000 7f7316ff5700 1 -- 172.21.15.62:0/972328123 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa473fb4f0 con 0x55aa47550e20 2023-07-14T16:28:41.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.586+0000 7f7316ff5700 1 -- 172.21.15.62:0/972328123 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f73280039e0 con 0x55aa47550e20 2023-07-14T16:28:41.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.586+0000 7f734ce15b80 10 client.28669 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:41.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.586+0000 7f734ce15b80 10 client.28669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:41.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.586+0000 7f734ce15b80 20 client.28669 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:41.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.586+0000 7f734ce15b80 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55aa47565fb0 0x55aa475683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.586+0000 7f734ce15b80 4 client.28669 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:41.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.586+0000 7f734ce15b80 1 -- 172.21.15.62:0/972328123 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa4749aeb0 con 0x55aa47565fb0 2023-07-14T16:28:41.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.587+0000 7f73337fe700 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55aa47565fb0 0x55aa475683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.587+0000 7f73337fe700 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55aa47565fb0 0x55aa475683c0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19113cde9121cf2e server_cookie=1edb801dbe02c3fa in_seq=0 out_seq=0 2023-07-14T16:28:41.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.587+0000 7f7316ff5700 10 client.28669 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:41.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f7316ff5700 1 -- 172.21.15.62:0/972328123 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x7f732c02b780 con 0x55aa47565fb0 2023-07-14T16:28:41.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f7316ff5700 10 client.28669 handle_command_reply: tid=0 2023-07-14T16:28:41.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 client.28669 shutdown 2023-07-14T16:28:41.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 -- 172.21.15.62:0/972328123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 msgr2=0x55aa4755a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa4755a650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7328000c00 tx=0x7f7328005270 comp rx=0 tx=0).stop 2023-07-14T16:28:41.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 -- 172.21.15.62:0/972328123 shutdown_connections 2023-07-14T16:28:41.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa47551c70 0x55aa47561690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55aa47565fb0 0x55aa475683c0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa475504d0 0x55aa4755a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 --2- 172.21.15.62:0/972328123 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa4755a650 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f7316ff5700 0 client.28669 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:41.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 -- 172.21.15.62:0/972328123 >> 172.21.15.62:0/972328123 conn(0x55aa47545da0 msgr2=0x55aa4754a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.588+0000 7f734ce15b80 1 -- 172.21.15.62:0/972328123 shutdown_connections 2023-07-14T16:28:41.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.589+0000 7f734ce15b80 1 -- 172.21.15.62:0/972328123 wait complete. 2023-07-14T16:28:41.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.589+0000 7f734ce15b80 20 client.28669 trim_cache size 0 max 16384 2023-07-14T16:28:41.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.616+0000 7f734ce15b80 1 Processor -- start 2023-07-14T16:28:41.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.616+0000 7f734ce15b80 1 -- start start 2023-07-14T16:28:41.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa47558020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa47558560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47551c70 0x55aa47561650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f734ce15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa47557dd0 con 0x55aa475504d0 2023-07-14T16:28:41.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f734ce15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa475511f0 con 0x55aa47551c70 2023-07-14T16:28:41.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f734ce15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa475514a0 con 0x55aa47550e20 2023-07-14T16:28:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f73337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa47558020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f73337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa47558020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35826/0 (socket says 172.21.15.62:35826) 2023-07-14T16:28:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f73337fe700 1 -- 172.21.15.62:0/2892764314 learned_addr learned my addr 172.21.15.62:0/2892764314 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f73327fc700 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa47558560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.617+0000 7f73337fe700 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47551c70 0x55aa47561650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f73337fe700 1 -- 172.21.15.62:0/2892764314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 msgr2=0x55aa47558560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f73337fe700 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa47558560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f73337fe700 1 -- 172.21.15.62:0/2892764314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47551c70 msgr2=0x55aa47561650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f73337fe700 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47551c70 0x55aa47561650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f73337fe700 1 -- 172.21.15.62:0/2892764314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa472f6250 con 0x55aa475504d0 2023-07-14T16:28:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f73337fe700 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa47558020 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x55aa4755b410 tx=0x7f732c00af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bdabeed5d178b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f7319ffb700 1 -- 172.21.15.62:0/2892764314 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c013580 con 0x55aa475504d0 2023-07-14T16:28:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f7319ffb700 1 -- 172.21.15.62:0/2892764314 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f732c027040 con 0x55aa475504d0 2023-07-14T16:28:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.618+0000 7f7319ffb700 1 -- 172.21.15.62:0/2892764314 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c01f740 con 0x55aa475504d0 2023-07-14T16:28:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 -- 172.21.15.62:0/2892764314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 msgr2=0x55aa47558020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa47558020 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x55aa4755b410 tx=0x7f732c00af60 comp rx=0 tx=0).stop 2023-07-14T16:28:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 -- 172.21.15.62:0/2892764314 shutdown_connections 2023-07-14T16:28:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47551c70 0x55aa47561650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa47558020 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 --2- 172.21.15.62:0/2892764314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47550e20 0x55aa47558560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 -- 172.21.15.62:0/2892764314 >> 172.21.15.62:0/2892764314 conn(0x55aa47545da0 msgr2=0x55aa47558e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 -- 172.21.15.62:0/2892764314 shutdown_connections 2023-07-14T16:28:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.619+0000 7f734ce15b80 1 -- 172.21.15.62:0/2892764314 wait complete. 2023-07-14T16:28:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 1 -- start start 2023-07-14T16:28:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 1 Processor -- start 2023-07-14T16:28:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa4755d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa4755dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47551c70 0x55aa47560d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa475612b0 con 0x55aa475504d0 2023-07-14T16:28:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa47561420 con 0x55aa47550e20 2023-07-14T16:28:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa475615e0 con 0x55aa47551c70 2023-07-14T16:28:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.620+0000 7f734ce15b80 10 client.0 _ll_register_callbacks cb 0x7f732800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa4755d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa4755d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35834/0 (socket says 172.21.15.62:35834) 2023-07-14T16:28:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 -- 172.21.15.62:0/531391189 learned_addr learned my addr 172.21.15.62:0/531391189 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73327fc700 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa4755dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47551c70 0x55aa47560d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 -- 172.21.15.62:0/531391189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47551c70 msgr2=0x55aa47560d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47551c70 0x55aa47560d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 -- 172.21.15.62:0/531391189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 msgr2=0x55aa4755dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa4755dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 -- 172.21.15.62:0/531391189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa472cfde0 con 0x55aa475504d0 2023-07-14T16:28:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f73337fe700 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa4755d850 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f732c00a400 tx=0x7f732c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b6011df732133f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.621+0000 7f7316ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.622+0000 7f7316ff5700 1 -- 172.21.15.62:0/531391189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c03a070 con 0x55aa475504d0 2023-07-14T16:28:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.622+0000 7f7316ff5700 1 -- 172.21.15.62:0/531391189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f732c006e40 con 0x55aa475504d0 2023-07-14T16:28:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.622+0000 7f734ce15b80 1 -- 172.21.15.62:0/531391189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f732c014ab0 con 0x55aa475504d0 2023-07-14T16:28:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.622+0000 7f7316ff5700 1 -- 172.21.15.62:0/531391189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f732c014630 con 0x55aa475504d0 2023-07-14T16:28:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.622+0000 7f734ce15b80 10 client.19227 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.622+0000 7f734ce15b80 1 -- 172.21.15.62:0/531391189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aa472ce2b0 con 0x55aa475504d0 2023-07-14T16:28:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.622+0000 7f7316ff5700 1 -- 172.21.15.62:0/531391189 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f732c045080 con 0x55aa475504d0 2023-07-14T16:28:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.623+0000 7f7316ff5700 1 -- 172.21.15.62:0/531391189 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f732c03d020 con 0x55aa475504d0 2023-07-14T16:28:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.623+0000 7f734ce15b80 10 client.19227 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.623+0000 7f734ce15b80 10 client.19227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.623+0000 7f734ce15b80 20 client.19227 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.623+0000 7f734ce15b80 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55aa475bae10 0x55aa475bd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.623+0000 7f734ce15b80 4 client.19227 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:28:41.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.623+0000 7f734ce15b80 1 -- 172.21.15.62:0/531391189 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aa4745d050 con 0x55aa475bae10 2023-07-14T16:28:41.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.623+0000 7f73327fc700 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55aa475bae10 0x55aa475bd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f73327fc700 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55aa475bae10 0x55aa475bd220 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128aaf6a53dd198 server_cookie=7f427f6a506e1f11 in_seq=0 out_seq=0 2023-07-14T16:28:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f7316ff5700 10 client.19227 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f7316ff5700 1 -- 172.21.15.62:0/531391189 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aa4745d050 con 0x55aa475bae10 2023-07-14T16:28:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f7316ff5700 10 client.19227 handle_command_reply: tid=0 2023-07-14T16:28:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 client.19227 shutdown 2023-07-14T16:28:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 -- 172.21.15.62:0/531391189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 msgr2=0x55aa4755d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa4755d850 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f732c00a400 tx=0x7f732c004be0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 -- 172.21.15.62:0/531391189 shutdown_connections 2023-07-14T16:28:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa47551c70 0x55aa47560d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55aa475504d0 0x55aa4755d850 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55aa475bae10 0x55aa475bd220 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 --2- 172.21.15.62:0/531391189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aa47550e20 0x55aa4755dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f7316ff5700 0 client.19227 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.624+0000 7f734ce15b80 1 -- 172.21.15.62:0/531391189 >> 172.21.15.62:0/531391189 conn(0x55aa47545da0 msgr2=0x55aa47558020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.625+0000 7f734ce15b80 1 -- 172.21.15.62:0/531391189 shutdown_connections 2023-07-14T16:28:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.625+0000 7f734ce15b80 1 -- 172.21.15.62:0/531391189 wait complete. 2023-07-14T16:28:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.625+0000 7f734ce15b80 20 client.19227 trim_cache size 0 max 16384 2023-07-14T16:28:41.607 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:28:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:28:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:28:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "fc51b5fd-1b0d-4079-9682-a08e1f6e6c22": { 2023-07-14T16:28:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:28:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "fc51b5fd-1b0d-4079-9682-a08e1f6e6c22", 2023-07-14T16:28:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:28:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:28:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:28:41.608 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:28:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.625+0000 7f73393ff700 1 -- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f731c0a11d0 msgr2=0x7f731c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.625+0000 7f73393ff700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f731c0a11d0 0x7f731c0a3680 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f73240994b0 tx=0x7f7324011f90 comp rx=0 tx=0).stop 2023-07-14T16:28:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.625+0000 7f73393ff700 1 -- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 msgr2=0x7f7334197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.625+0000 7f73393ff700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 0x7f7334197af0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f732400b530 tx=0x7f732400b840 comp rx=0 tx=0).stop 2023-07-14T16:28:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.626+0000 7f73393ff700 1 -- 172.21.15.62:0/1671881443 shutdown_connections 2023-07-14T16:28:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.626+0000 7f73393ff700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f731c0a11d0 0x7f731c0a3680 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.626+0000 7f73393ff700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73341079a0 0x7f733419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.626+0000 7f73393ff700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7334072ef0 0x7f7334197af0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.626+0000 7f73393ff700 1 --2- 172.21.15.62:0/1671881443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73340744d0 0x7f7334198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:41.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.626+0000 7f73393ff700 1 -- 172.21.15.62:0/1671881443 >> 172.21.15.62:0/1671881443 conn(0x7f73340fb130 msgr2=0x7f73340fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:41.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.626+0000 7f73393ff700 1 -- 172.21.15.62:0/1671881443 shutdown_connections 2023-07-14T16:28:41.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:28:41.626+0000 7f73393ff700 1 -- 172.21.15.62:0/1671881443 wait complete. 2023-07-14T16:28:41.618 INFO:tasks.cephfs.filesystem:scrub status for tag:fc51b5fd-1b0d-4079-9682-a08e1f6e6c22 - {'path': '/', 'tag': 'fc51b5fd-1b0d-4079-9682-a08e1f6e6c22', 'options': 'recursive,force'} 2023-07-14T16:28:41.649 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:28:41 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:28:41.318+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v245>,<0x10000000003/client.0 v246>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v258>,<0x10000000001/sv_0 v272>,<0x10000000000/_nogroup v266>,<0x1/volumes v256>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:28:41.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:41 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:28:41.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:41 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:41.872 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:41 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:28:41.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:41 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:41.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:41 smithi062 ceph-mon[95543]: pgmap v749: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:28:42.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:41 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:28:42.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:41 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:42.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:41 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:28:42.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:41 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:42.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:41 smithi081 ceph-mon[127795]: pgmap v749: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:28:42.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:41 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:28:42.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:41 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:42.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:41 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:28:42.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:41 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:42.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:41 smithi164 ceph-mon[97427]: pgmap v749: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:28:43.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:42 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:28:43.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:42 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:43.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:42 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:28:43.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:42 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:28:43.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:42 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:43.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:42 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:28:43.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:42 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:43.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:42 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:28:43.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:42 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:28:43.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:42 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:43.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:42 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:28:43.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:42 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:43.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:42 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:28:43.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:42 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:28:43.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:42 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:28:45.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:44 smithi164 ceph-mon[97427]: pgmap v750: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:28:45.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:44 smithi062 ceph-mon[95543]: pgmap v750: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:28:45.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:44 smithi081 ceph-mon[127795]: pgmap v750: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-14T16:28:47.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:46 smithi164 ceph-mon[97427]: pgmap v751: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 30 op/s 2023-07-14T16:28:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:46 smithi062 ceph-mon[95543]: pgmap v751: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 30 op/s 2023-07-14T16:28:47.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:46 smithi081 ceph-mon[127795]: pgmap v751: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 30 op/s 2023-07-14T16:28:49.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:48 smithi164 ceph-mon[97427]: pgmap v752: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:28:49.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:48 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:28:49.022 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:48 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:28:49.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:48 smithi062 ceph-mon[95543]: pgmap v752: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:28:49.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:48 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:28:49.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:48 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:28:49.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:48 smithi081 ceph-mon[127795]: pgmap v752: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:28:49.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:48 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:28:49.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:48 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:28:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 -- 172.21.15.62:0/3724702625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c1077d0 msgr2=0x7f7b8c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3724702625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c1077d0 0x7f7b8c109c50 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400b0a0 tx=0x7f7b8400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 -- 172.21.15.62:0/3724702625 shutdown_connections 2023-07-14T16:28:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3724702625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c1077d0 0x7f7b8c109c50 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3724702625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c0744f0 0x7f7b8c072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3724702625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b8c072f10 0x7f7b8c073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 -- 172.21.15.62:0/3724702625 >> 172.21.15.62:0/3724702625 conn(0x7f7b8c0faf80 msgr2=0x7f7b8c0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 -- 172.21.15.62:0/3724702625 shutdown_connections 2023-07-14T16:28:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.114+0000 7f7b9099d700 1 -- 172.21.15.62:0/3724702625 wait complete. 2023-07-14T16:28:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b9099d700 1 Processor -- start 2023-07-14T16:28:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b9099d700 1 -- start start 2023-07-14T16:28:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b9099d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c072f10 0x7f7b8c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b9099d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c0744f0 0x7f7b8c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b9099d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b8c1077d0 0x7f7b8c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b9099d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19f3f0 con 0x7f7b8c072f10 2023-07-14T16:28:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b9099d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19f530 con 0x7f7b8c0744f0 2023-07-14T16:28:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b9099d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19f6c0 con 0x7f7b8c1077d0 2023-07-14T16:28:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c072f10 0x7f7b8c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c072f10 0x7f7b8c197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36166/0 (socket says 172.21.15.62:36166) 2023-07-14T16:28:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.116+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/3375921513 learned_addr learned my addr 172.21.15.62:0/3375921513 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8affd700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c0744f0 0x7f7b8c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8bfff700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b8c1077d0 0x7f7b8c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/3375921513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b8c1077d0 msgr2=0x7f7b8c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b8c1077d0 0x7f7b8c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/3375921513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c0744f0 msgr2=0x7f7b8c197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c0744f0 0x7f7b8c197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/3375921513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84009d00 con 0x7f7b8c072f10 2023-07-14T16:28:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8bfff700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b8c1077d0 0x7f7b8c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:28:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c072f10 0x7f7b8c197870 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00e9a0 tx=0x7f7b7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.117+0000 7f7b88ff9700 1 -- 172.21.15.62:0/3375921513 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c009790 con 0x7f7b8c072f10 2023-07-14T16:28:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.118+0000 7f7b9099d700 1 -- 172.21.15.62:0/3375921513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8c19f9a0 con 0x7f7b8c072f10 2023-07-14T16:28:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.118+0000 7f7b88ff9700 1 -- 172.21.15.62:0/3375921513 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b7c00cbd0 con 0x7f7b8c072f10 2023-07-14T16:28:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.118+0000 7f7b88ff9700 1 -- 172.21.15.62:0/3375921513 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c018680 con 0x7f7b8c072f10 2023-07-14T16:28:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.118+0000 7f7b9099d700 1 -- 172.21.15.62:0/3375921513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c19feb0 con 0x7f7b8c072f10 2023-07-14T16:28:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.121+0000 7f7b88ff9700 1 -- 172.21.15.62:0/3375921513 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7b7c00cd40 con 0x7f7b8c072f10 2023-07-14T16:28:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.123+0000 7f7b88ff9700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b740a1150 0x7f7b740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.123+0000 7f7ba546bb80 1 Processor -- start 2023-07-14T16:28:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7b88ff9700 1 -- 172.21.15.62:0/3375921513 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b7c014070 con 0x7f7b8c072f10 2023-07-14T16:28:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7b8affd700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b740a1150 0x7f7b740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7ba546bb80 1 -- start start 2023-07-14T16:28:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dcb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dcbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4ccfc50 con 0x55ecc4c266b0 2023-07-14T16:28:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4d229e0 con 0x55ecc4dcb7e0 2023-07-14T16:28:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.124+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4d1be70 con 0x55ecc4dcaa90 2023-07-14T16:28:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8affd700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b740a1150 0x7f7b740a3600 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005950 tx=0x7f7b8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dcb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dcbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dcb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:48608/0 (socket says 172.21.15.62:48608) 2023-07-14T16:28:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8affd700 1 -- 172.21.15.62:0/1456269674 learned_addr learned my addr 172.21.15.62:0/1456269674 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36174/0 (socket says 172.21.15.62:36174) 2023-07-14T16:28:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/1456269674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 msgr2=0x55ecc4dcb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/1456269674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dcb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/1456269674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 msgr2=0x55ecc4dcbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/1456269674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dcbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/1456269674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecc4b49d80 con 0x55ecc4c266b0 2023-07-14T16:28:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.125+0000 7f7b8affd700 1 --2- 172.21.15.62:0/1456269674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dcb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:28:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.126+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/1456269674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dca550 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00c5b0 tx=0x7f7b7c099b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce779f7ce548f95 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.126+0000 7f7b71ffb700 1 -- 172.21.15.62:0/1456269674 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c099fc0 con 0x55ecc4c266b0 2023-07-14T16:28:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.126+0000 7f7b71ffb700 1 -- 172.21.15.62:0/1456269674 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b7c09dff0 con 0x55ecc4c266b0 2023-07-14T16:28:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.126+0000 7f7b71ffb700 1 -- 172.21.15.62:0/1456269674 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0a21e0 con 0x55ecc4c266b0 2023-07-14T16:28:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1456269674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 msgr2=0x55ecc4dca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1456269674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dca550 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00c5b0 tx=0x7f7b7c099b80 comp rx=0 tx=0).stop 2023-07-14T16:28:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1456269674 shutdown_connections 2023-07-14T16:28:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1456269674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dcbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1456269674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dca550 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1456269674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dcb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1456269674 >> 172.21.15.62:0/1456269674 conn(0x55ecc4dbfd40 msgr2=0x55ecc4dc4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1456269674 shutdown_connections 2023-07-14T16:28:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1456269674 wait complete. 2023-07-14T16:28:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.127+0000 7f7ba546bb80 1 -- start start 2023-07-14T16:28:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.128+0000 7f7ba546bb80 1 Processor -- start 2023-07-14T16:28:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.128+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.128+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dd3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.128+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4ddafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.128+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4cbba60 con 0x55ecc4dcb7e0 2023-07-14T16:28:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.128+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4cbb760 con 0x55ecc4c266b0 2023-07-14T16:28:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.128+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4b9f8c0 con 0x55ecc4dcaa90 2023-07-14T16:28:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.128+0000 7f7ba546bb80 10 client.0 _ll_register_callbacks cb 0x55ecc4cb9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:37318/0 (socket says 172.21.15.62:37318) 2023-07-14T16:28:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/4158363183 learned_addr learned my addr 172.21.15.62:0/4158363183 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8bfff700 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4ddafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8affd700 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dd3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/4158363183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 msgr2=0x55ecc4dd3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dd3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/4158363183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4dcb7e0 msgr2=0x55ecc4ddafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4ddafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.129+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/4158363183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b80010c50 con 0x55ecc4c266b0 2023-07-14T16:28:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd3a60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c099cc0 tx=0x7f7b7c094f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae9cba02ac9c438 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7b6affd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:28:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7b6affd700 1 -- 172.21.15.62:0/4158363183 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c099750 con 0x55ecc4c266b0 2023-07-14T16:28:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7ba546bb80 1 -- 172.21.15.62:0/4158363183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecc4c75490 con 0x55ecc4c266b0 2023-07-14T16:28:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7b6affd700 1 -- 172.21.15.62:0/4158363183 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b7c0a2410 con 0x55ecc4c266b0 2023-07-14T16:28:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7b6affd700 1 -- 172.21.15.62:0/4158363183 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7c099fc0 con 0x55ecc4c266b0 2023-07-14T16:28:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7ba546bb80 10 client.28811 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7ba546bb80 1 -- 172.21.15.62:0/4158363183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f7b8c19feb0 con 0x55ecc4c266b0 2023-07-14T16:28:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.130+0000 7f7b6affd700 1 -- 172.21.15.62:0/4158363183 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b7c063080 con 0x55ecc4c266b0 2023-07-14T16:28:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.131+0000 7f7b6affd700 1 -- 172.21.15.62:0/4158363183 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b7c010430 con 0x55ecc4c266b0 2023-07-14T16:28:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.131+0000 7f7ba546bb80 10 client.28811 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.131+0000 7f7ba546bb80 10 client.28811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.131+0000 7f7ba546bb80 20 client.28811 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.131+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ecc4ddf530 0x55ecc4de1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.131+0000 7f7ba546bb80 4 client.28811 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:28:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.131+0000 7f7ba546bb80 1 -- 172.21.15.62:0/4158363183 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecc4d14e50 con 0x55ecc4ddf530 2023-07-14T16:28:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.131+0000 7f7b8affd700 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ecc4ddf530 0x55ecc4de1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.132+0000 7f7b6affd700 10 client.28811 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.132+0000 7f7b8affd700 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ecc4ddf530 0x55ecc4de1940 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3adfb8ad8054a9e server_cookie=240ebbe176675bf8 in_seq=0 out_seq=0 2023-07-14T16:28:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7b6affd700 1 -- 172.21.15.62:0/4158363183 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ecc4d14e50 con 0x55ecc4ddf530 2023-07-14T16:28:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7b6affd700 10 client.28811 handle_command_reply: tid=0 2023-07-14T16:28:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 client.28811 shutdown 2023-07-14T16:28:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 -- 172.21.15.62:0/4158363183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4c266b0 msgr2=0x55ecc4dd3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd3a60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c099cc0 tx=0x7f7b7c094f90 comp rx=0 tx=0).stop 2023-07-14T16:28:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 -- 172.21.15.62:0/4158363183 shutdown_connections 2023-07-14T16:28:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4ddafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ecc4ddf530 0x55ecc4de1940 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd3a60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/4158363183 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dd3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 -- 172.21.15.62:0/4158363183 >> 172.21.15.62:0/4158363183 conn(0x55ecc4dbfd40 msgr2=0x55ecc4dc4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7b6affd700 0 client.28811 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 -- 172.21.15.62:0/4158363183 shutdown_connections 2023-07-14T16:28:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 1 -- 172.21.15.62:0/4158363183 wait complete. 2023-07-14T16:28:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.133+0000 7f7ba546bb80 20 client.28811 trim_cache size 0 max 16384 2023-07-14T16:28:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.149+0000 7f7ba546bb80 1 Processor -- start 2023-07-14T16:28:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.149+0000 7f7ba546bb80 1 -- start start 2023-07-14T16:28:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.149+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.149+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcaa90 0x55ecc4ddafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.149+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dd1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.149+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4dd35d0 con 0x55ecc4c266b0 2023-07-14T16:28:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.149+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4dc1e60 con 0x55ecc4dcaa90 2023-07-14T16:28:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.149+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4dbcd80 con 0x55ecc4dcb7e0 2023-07-14T16:28:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36188/0 (socket says 172.21.15.62:36188) 2023-07-14T16:28:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dd1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/774380651 learned_addr learned my addr 172.21.15.62:0/774380651 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8affd700 1 --2- 172.21.15.62:0/774380651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcaa90 0x55ecc4ddafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/774380651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcb7e0 msgr2=0x55ecc4dd1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/774380651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dd1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/774380651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcaa90 msgr2=0x55ecc4ddafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/774380651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcaa90 0x55ecc4ddafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.150+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/774380651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecc4b32c00 con 0x55ecc4c266b0 2023-07-14T16:28:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.151+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/774380651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd1980 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c098310 tx=0x7f7b7c098570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cb9eed21309308 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.151+0000 7f7b727fc700 1 -- 172.21.15.62:0/774380651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0a2410 con 0x55ecc4c266b0 2023-07-14T16:28:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.151+0000 7f7b727fc700 1 -- 172.21.15.62:0/774380651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ecc4c75490 con 0x55ecc4c266b0 2023-07-14T16:28:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.151+0000 7f7b727fc700 1 -- 172.21.15.62:0/774380651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c095ed0 con 0x55ecc4c266b0 2023-07-14T16:28:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.151+0000 7f7ba546bb80 1 -- 172.21.15.62:0/774380651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 msgr2=0x55ecc4dd1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.151+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/774380651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd1980 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c098310 tx=0x7f7b7c098570 comp rx=0 tx=0).stop 2023-07-14T16:28:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.151+0000 7f7ba546bb80 1 -- 172.21.15.62:0/774380651 shutdown_connections 2023-07-14T16:28:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.152+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/774380651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dd1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.152+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/774380651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd1980 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.152+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/774380651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcaa90 0x55ecc4ddafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.152+0000 7f7ba546bb80 1 -- 172.21.15.62:0/774380651 >> 172.21.15.62:0/774380651 conn(0x55ecc4dbfd40 msgr2=0x55ecc4dc4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.152+0000 7f7ba546bb80 1 -- 172.21.15.62:0/774380651 shutdown_connections 2023-07-14T16:28:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.152+0000 7f7ba546bb80 1 -- 172.21.15.62:0/774380651 wait complete. 2023-07-14T16:28:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.152+0000 7f7ba546bb80 1 -- start start 2023-07-14T16:28:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.152+0000 7f7ba546bb80 1 Processor -- start 2023-07-14T16:28:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dd7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7ba546bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dda160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4dda6a0 con 0x55ecc4c266b0 2023-07-14T16:28:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4dda810 con 0x55ecc4dcb7e0 2023-07-14T16:28:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7ba546bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecc4dda9d0 con 0x55ecc4dcaa90 2023-07-14T16:28:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7ba546bb80 10 client.0 _ll_register_callbacks cb 0x7f7b84009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:28:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7b8b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36194/0 (socket says 172.21.15.62:36194) 2023-07-14T16:28:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/1550440398 learned_addr learned my addr 172.21.15.62:0/1550440398 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:28:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.153+0000 7f7b8bfff700 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dda160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b8affd700 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dd7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/1550440398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 msgr2=0x55ecc4dd7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dd7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/1550440398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 msgr2=0x55ecc4dda160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:28:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dda160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b8b7fe700 1 -- 172.21.15.62:0/1550440398 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c19feb0 con 0x55ecc4c266b0 2023-07-14T16:28:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b8b7fe700 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd7130 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0839a0 tx=0x7f7b7c094f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c426aa857c72f4c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:28:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b6affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:28:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b6affd700 1 -- 172.21.15.62:0/1550440398 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c099560 con 0x55ecc4c266b0 2023-07-14T16:28:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7b6affd700 1 -- 172.21.15.62:0/1550440398 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7b7c099fc0 con 0x55ecc4c266b0 2023-07-14T16:28:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.154+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1550440398 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b7c09dff0 con 0x55ecc4c266b0 2023-07-14T16:28:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7b6affd700 1 -- 172.21.15.62:0/1550440398 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7c0a2410 con 0x55ecc4c266b0 2023-07-14T16:28:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7ba546bb80 10 client.19245 fetch_fsmap learned FSMap version 25 2023-07-14T16:28:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1550440398 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ecc4b701f0 con 0x55ecc4c266b0 2023-07-14T16:28:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7b6affd700 1 -- 172.21.15.62:0/1550440398 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b7c063030 con 0x55ecc4c266b0 2023-07-14T16:28:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7b6affd700 1 -- 172.21.15.62:0/1550440398 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b7c098f20 con 0x55ecc4c266b0 2023-07-14T16:28:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7ba546bb80 10 client.19245 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T16:28:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7ba546bb80 10 client.19245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:28:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7ba546bb80 20 client.19245 populate_metadata read hostname 'smithi062' 2023-07-14T16:28:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ecc4e33bc0 0x55ecc4e35fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:28:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7ba546bb80 4 client.19245 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:28:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.155+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1550440398 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecc4cd6ff0 con 0x55ecc4e33bc0 2023-07-14T16:28:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.156+0000 7f7b8affd700 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ecc4e33bc0 0x55ecc4e35fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:28:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.157+0000 7f7b6affd700 10 client.19245 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.157+0000 7f7b8affd700 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ecc4e33bc0 0x55ecc4e35fd0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2b5d8869674a26 server_cookie=10e1bba964484e77 in_seq=0 out_seq=0 2023-07-14T16:28:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.166+0000 7f7b6affd700 1 -- 172.21.15.62:0/1550440398 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecc4cd6ff0 con 0x55ecc4e33bc0 2023-07-14T16:28:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.166+0000 7f7b6affd700 10 client.19245 handle_command_reply: tid=0 2023-07-14T16:28:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.166+0000 7f7ba546bb80 1 client.19245 shutdown 2023-07-14T16:28:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.166+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1550440398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 msgr2=0x55ecc4dd7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.166+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd7130 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0839a0 tx=0x7f7b7c094f90 comp rx=0 tx=0).stop 2023-07-14T16:28:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1550440398 shutdown_connections 2023-07-14T16:28:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ecc4e33bc0 0x55ecc4e35fd0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc4dcb7e0 0x55ecc4dda160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ecc4c266b0 0x55ecc4dd7130 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 1 --2- 172.21.15.62:0/1550440398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecc4dcaa90 0x55ecc4dd7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1550440398 >> 172.21.15.62:0/1550440398 conn(0x55ecc4dbfd40 msgr2=0x55ecc4dd3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7b6affd700 0 client.19245 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:28:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1550440398 shutdown_connections 2023-07-14T16:28:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 1 -- 172.21.15.62:0/1550440398 wait complete. 2023-07-14T16:28:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.167+0000 7f7ba546bb80 20 client.19245 trim_cache size 0 max 16384 2023-07-14T16:28:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:28:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:28:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:28:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:28:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 -- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b740a1150 msgr2=0x7f7b740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b740a1150 0x7f7b740a3600 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005950 tx=0x7f7b8000b410 comp rx=0 tx=0).stop 2023-07-14T16:28:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 -- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c072f10 msgr2=0x7f7b8c197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:28:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c072f10 0x7f7b8c197870 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00e9a0 tx=0x7f7b7c00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 -- 172.21.15.62:0/3375921513 shutdown_connections 2023-07-14T16:28:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b8c1077d0 0x7f7b8c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b8c072f10 0x7f7b8c197870 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7b740a1150 0x7f7b740a3600 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 --2- 172.21.15.62:0/3375921513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b8c0744f0 0x7f7b8c197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:28:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 -- 172.21.15.62:0/3375921513 >> 172.21.15.62:0/3375921513 conn(0x7f7b8c0faf80 msgr2=0x7f7b8c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:28:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 -- 172.21.15.62:0/3375921513 shutdown_connections 2023-07-14T16:28:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:28:50.168+0000 7f7b9099d700 1 -- 172.21.15.62:0/3375921513 wait complete. 2023-07-14T16:28:51.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:50 smithi164 ceph-mon[97427]: pgmap v753: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:28:51.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:50 smithi062 ceph-mon[95543]: pgmap v753: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:28:51.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:50 smithi081 ceph-mon[127795]: pgmap v753: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:28:53.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:52 smithi062 ceph-mon[95543]: pgmap v754: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:28:53.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:52 smithi081 ceph-mon[127795]: pgmap v754: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:28:53.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:52 smithi164 ceph-mon[97427]: pgmap v754: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 34 op/s 2023-07-14T16:28:55.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:54 smithi062 ceph-mon[95543]: pgmap v755: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:28:55.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:54 smithi081 ceph-mon[127795]: pgmap v755: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:28:55.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:54 smithi164 ceph-mon[97427]: pgmap v755: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:28:57.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:56 smithi062 ceph-mon[95543]: pgmap v756: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 26 op/s 2023-07-14T16:28:57.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:56 smithi081 ceph-mon[127795]: pgmap v756: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 26 op/s 2023-07-14T16:28:57.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:56 smithi164 ceph-mon[97427]: pgmap v756: 97 pgs: 97 active+clean; 9.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 26 op/s 2023-07-14T16:28:59.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:58 smithi062 ceph-mon[95543]: pgmap v757: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 21 op/s 2023-07-14T16:28:59.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:58 smithi081 ceph-mon[127795]: pgmap v757: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 21 op/s 2023-07-14T16:28:59.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:58 smithi164 ceph-mon[97427]: pgmap v757: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 21 op/s 2023-07-14T16:29:00.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:28:59 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:29:00.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:28:59 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:29:00.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:28:59 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:29:01.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:00 smithi062 ceph-mon[95543]: pgmap v758: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 16 MiB/s wr, 18 op/s 2023-07-14T16:29:01.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:00 smithi081 ceph-mon[127795]: pgmap v758: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 16 MiB/s wr, 18 op/s 2023-07-14T16:29:01.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:00 smithi164 ceph-mon[97427]: pgmap v758: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 16 MiB/s wr, 18 op/s 2023-07-14T16:29:03.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:02 smithi062 ceph-mon[95543]: pgmap v759: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:29:03.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:02 smithi081 ceph-mon[127795]: pgmap v759: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:29:03.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:02 smithi164 ceph-mon[97427]: pgmap v759: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 MiB/s wr, 16 op/s 2023-07-14T16:29:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 -- 172.21.15.62:0/1099862468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c840ffb00 msgr2=0x7f6c840fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 --2- 172.21.15.62:0/1099862468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c840ffb00 0x7f6c840fff10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009990 tx=0x7f6c74009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 -- 172.21.15.62:0/1099862468 shutdown_connections 2023-07-14T16:29:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 --2- 172.21.15.62:0/1099862468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c84101680 0x7f6c84107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 --2- 172.21.15.62:0/1099862468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c840ffb00 0x7f6c840fff10 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 --2- 172.21.15.62:0/1099862468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c84100cf0 0x7f6c84101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 -- 172.21.15.62:0/1099862468 >> 172.21.15.62:0/1099862468 conn(0x7f6c840fb0e0 msgr2=0x7f6c840fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 -- 172.21.15.62:0/1099862468 shutdown_connections 2023-07-14T16:29:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.408+0000 7f6c89bff700 1 -- 172.21.15.62:0/1099862468 wait complete. 2023-07-14T16:29:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.409+0000 7f6c89bff700 1 Processor -- start 2023-07-14T16:29:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.409+0000 7f6c89bff700 1 -- start start 2023-07-14T16:29:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c89bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c840ffb00 0x7f6c84197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c89bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c84100cf0 0x7f6c84198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c89bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c84101680 0x7f6c84198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c89bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84199510 con 0x7f6c84100cf0 2023-07-14T16:29:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c89bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84199680 con 0x7f6c840ffb00 2023-07-14T16:29:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c89bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84199840 con 0x7f6c84101680 2023-07-14T16:29:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c82ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c84100cf0 0x7f6c84198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c82ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c84100cf0 0x7f6c84198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39250/0 (socket says 172.21.15.62:39250) 2023-07-14T16:29:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c82ffd700 1 -- 172.21.15.62:0/2511019538 learned_addr learned my addr 172.21.15.62:0/2511019538 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c83fff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c84101680 0x7f6c84198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.410+0000 7f6c837fe700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c840ffb00 0x7f6c84197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c82ffd700 1 -- 172.21.15.62:0/2511019538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c84101680 msgr2=0x7f6c84198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c84101680 0x7f6c84198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c82ffd700 1 -- 172.21.15.62:0/2511019538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c840ffb00 msgr2=0x7f6c84197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c840ffb00 0x7f6c84197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c82ffd700 1 -- 172.21.15.62:0/2511019538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c740095f0 con 0x7f6c84100cf0 2023-07-14T16:29:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c83fff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c84101680 0x7f6c84198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c837fe700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c840ffb00 0x7f6c84197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c84100cf0 0x7f6c84198030 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800d950 tx=0x7f6c7800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c80ff9700 1 -- 172.21.15.62:0/2511019538 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78009b40 con 0x7f6c84100cf0 2023-07-14T16:29:03.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c80ff9700 1 -- 172.21.15.62:0/2511019538 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c78009ca0 con 0x7f6c84100cf0 2023-07-14T16:29:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c89bff700 1 -- 172.21.15.62:0/2511019538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c84199b20 con 0x7f6c84100cf0 2023-07-14T16:29:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c80ff9700 1 -- 172.21.15.62:0/2511019538 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7800f620 con 0x7f6c84100cf0 2023-07-14T16:29:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.411+0000 7f6c89bff700 1 -- 172.21.15.62:0/2511019538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c841a02e0 con 0x7f6c84100cf0 2023-07-14T16:29:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.416+0000 7f6c80ff9700 1 -- 172.21.15.62:0/2511019538 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6c78010460 con 0x7f6c84100cf0 2023-07-14T16:29:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.416+0000 7f6c9d4b4b80 1 Processor -- start 2023-07-14T16:29:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.417+0000 7f6c9d4b4b80 1 -- start start 2023-07-14T16:29:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.417+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d32516b0 0x55d4d33f5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.417+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d33f5840 0x55d4d33f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.417+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 0x55d4d33f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.417+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d32fac50 con 0x55d4d33f5840 2023-07-14T16:29:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.417+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d334d9e0 con 0x55d4d32516b0 2023-07-14T16:29:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.417+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d3346e70 con 0x55d4d33f66e0 2023-07-14T16:29:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d33f5840 0x55d4d33f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c83fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d32516b0 0x55d4d33f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c83fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d32516b0 0x55d4d33f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:46642/0 (socket says 172.21.15.62:46642) 2023-07-14T16:29:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c83fff700 1 -- 172.21.15.62:0/3167826486 learned_addr learned my addr 172.21.15.62:0/3167826486 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/3167826486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 0x55d4d33f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c80ff9700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c6c0a12a0 0x7f6c6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c80ff9700 1 -- 172.21.15.62:0/2511019538 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c780cc640 con 0x7f6c84100cf0 2023-07-14T16:29:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c83fff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c6c0a12a0 0x7f6c6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c837fe700 1 -- 172.21.15.62:0/3167826486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 msgr2=0x55d4d33f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c837fe700 1 --2- 172.21.15.62:0/3167826486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 0x55d4d33f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c837fe700 1 -- 172.21.15.62:0/3167826486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d32516b0 msgr2=0x55d4d33f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c837fe700 1 --2- 172.21.15.62:0/3167826486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d32516b0 0x55d4d33f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.418+0000 7f6c837fe700 1 -- 172.21.15.62:0/3167826486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4d3174d80 con 0x55d4d33f5840 2023-07-14T16:29:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.419+0000 7f6c837fe700 1 --2- 172.21.15.62:0/3167826486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d33f5840 0x55d4d33f61a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000c00 tx=0x7f6c74005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e7530c60602cd7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.419+0000 7f6c69ffb700 1 -- 172.21.15.62:0/3167826486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400b970 con 0x55d4d33f5840 2023-07-14T16:29:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.419+0000 7f6c69ffb700 1 -- 172.21.15.62:0/3167826486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7401e040 con 0x55d4d33f5840 2023-07-14T16:29:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.419+0000 7f6c83fff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c6c0a12a0 0x7f6c6c0a3750 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55d4d33f5060 tx=0x7f6c7c004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.419+0000 7f6c69ffb700 1 -- 172.21.15.62:0/3167826486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74004760 con 0x55d4d33f5840 2023-07-14T16:29:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3167826486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d33f5840 msgr2=0x55d4d33f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3167826486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d33f5840 0x55d4d33f61a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000c00 tx=0x7f6c74005e80 comp rx=0 tx=0).stop 2023-07-14T16:29:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3167826486 shutdown_connections 2023-07-14T16:29:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3167826486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 0x55d4d33f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3167826486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d32516b0 0x55d4d33f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3167826486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d33f5840 0x55d4d33f61a0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3167826486 >> 172.21.15.62:0/3167826486 conn(0x55d4d33eabe0 msgr2=0x55d4d33efcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3167826486 shutdown_connections 2023-07-14T16:29:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.420+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3167826486 wait complete. 2023-07-14T16:29:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.421+0000 7f6c9d4b4b80 1 -- start start 2023-07-14T16:29:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.421+0000 7f6c9d4b4b80 1 Processor -- start 2023-07-14T16:29:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 0x55d4d33fed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 0x55d4d3405dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d32e6a60 con 0x55d4d32516b0 2023-07-14T16:29:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d32e6760 con 0x55d4d33f66e0 2023-07-14T16:29:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d31ca8c0 con 0x55d4d33f5840 2023-07-14T16:29:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c9d4b4b80 10 client.0 _ll_register_callbacks cb 0x55d4d32e4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39274/0 (socket says 172.21.15.62:39274) 2023-07-14T16:29:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c837fe700 1 -- 172.21.15.62:0/216421846 learned_addr learned my addr 172.21.15.62:0/216421846 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 0x55d4d33fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.422+0000 7f6c83fff700 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 0x55d4d3405dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c837fe700 1 -- 172.21.15.62:0/216421846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 msgr2=0x55d4d33fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c837fe700 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 0x55d4d33fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c837fe700 1 -- 172.21.15.62:0/216421846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 msgr2=0x55d4d3405dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c837fe700 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 0x55d4d3405dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c837fe700 1 -- 172.21.15.62:0/216421846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c84108870 con 0x55d4d32516b0 2023-07-14T16:29:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c837fe700 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fe850 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400c010 tx=0x7f6c740052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13c2e6187240fac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c62ffd700 1 -- 172.21.15.62:0/216421846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74004700 con 0x55d4d32516b0 2023-07-14T16:29:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c62ffd700 1 -- 172.21.15.62:0/216421846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c74004860 con 0x55d4d32516b0 2023-07-14T16:29:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.423+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/216421846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4d32a0490 con 0x55d4d32516b0 2023-07-14T16:29:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c62ffd700 1 -- 172.21.15.62:0/216421846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c74004a20 con 0x55d4d32516b0 2023-07-14T16:29:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c9d4b4b80 10 client.19257 fetch_fsmap learned FSMap version 26 2023-07-14T16:29:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/216421846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d4d315dc00 con 0x55d4d32516b0 2023-07-14T16:29:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c62ffd700 1 -- 172.21.15.62:0/216421846 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7403b020 con 0x55d4d32516b0 2023-07-14T16:29:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c62ffd700 1 -- 172.21.15.62:0/216421846 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6c74003cb0 con 0x55d4d32516b0 2023-07-14T16:29:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c9d4b4b80 10 client.19257 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T16:29:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c9d4b4b80 10 client.19257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c9d4b4b80 20 client.19257 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d4d340a1d0 0x55d4d340c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c9d4b4b80 4 client.19257 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.424+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/216421846 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4d333fe50 con 0x55d4d340a1d0 2023-07-14T16:29:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.425+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d4d340a1d0 0x55d4d340c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.425+0000 7f6c62ffd700 10 client.19257 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.425+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d4d340a1d0 0x55d4d340c5e0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870e28e3490715b server_cookie=2cb5d15e76b291fa in_seq=0 out_seq=0 2023-07-14T16:29:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c62ffd700 1 -- 172.21.15.62:0/216421846 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4d333fe50 con 0x55d4d340a1d0 2023-07-14T16:29:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c62ffd700 10 client.19257 handle_command_reply: tid=0 2023-07-14T16:29:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 client.19257 shutdown 2023-07-14T16:29:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/216421846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 msgr2=0x55d4d33fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fe850 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400c010 tx=0x7f6c740052f0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/216421846 shutdown_connections 2023-07-14T16:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d4d340a1d0 0x55d4d340c5e0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 0x55d4d3405dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fe850 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c62ffd700 0 client.19257 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/216421846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 0x55d4d33fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.426+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/216421846 >> 172.21.15.62:0/216421846 conn(0x55d4d33eabe0 msgr2=0x55d4d33ef4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.427+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/216421846 shutdown_connections 2023-07-14T16:29:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.427+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/216421846 wait complete. 2023-07-14T16:29:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.427+0000 7f6c9d4b4b80 20 client.19257 trim_cache size 0 max 16384 2023-07-14T16:29:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.443+0000 7f6c9d4b4b80 1 Processor -- start 2023-07-14T16:29:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.443+0000 7f6c9d4b4b80 1 -- start start 2023-07-14T16:29:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.443+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.443+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 0x55d4d3405d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.443+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 0x55d4d3410f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d33f5f90 con 0x55d4d32516b0 2023-07-14T16:29:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d33fa1a0 con 0x55d4d33f66e0 2023-07-14T16:29:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d3411440 con 0x55d4d33f5840 2023-07-14T16:29:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39286/0 (socket says 172.21.15.62:39286) 2023-07-14T16:29:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c837fe700 1 -- 172.21.15.62:0/2466647639 learned_addr learned my addr 172.21.15.62:0/2466647639 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c83fff700 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 0x55d4d3410f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 0x55d4d3405d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c837fe700 1 -- 172.21.15.62:0/2466647639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 msgr2=0x55d4d3405d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c837fe700 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 0x55d4d3405d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c837fe700 1 -- 172.21.15.62:0/2466647639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 msgr2=0x55d4d3410f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c837fe700 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 0x55d4d3410f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.444+0000 7f6c837fe700 1 -- 172.21.15.62:0/2466647639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4d319b1f0 con 0x55d4d32516b0 2023-07-14T16:29:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.445+0000 7f6c837fe700 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fdf00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6c74022430 tx=0x7f6c74005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0298ba96e7e468 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.445+0000 7f6c6a7fc700 1 -- 172.21.15.62:0/2466647639 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400bda0 con 0x55d4d32516b0 2023-07-14T16:29:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.445+0000 7f6c6a7fc700 1 -- 172.21.15.62:0/2466647639 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7403b030 con 0x55d4d32516b0 2023-07-14T16:29:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.445+0000 7f6c6a7fc700 1 -- 172.21.15.62:0/2466647639 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400f8b0 con 0x55d4d32516b0 2023-07-14T16:29:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/2466647639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 msgr2=0x55d4d33fdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fdf00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6c74022430 tx=0x7f6c74005fb0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/2466647639 shutdown_connections 2023-07-14T16:29:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f66e0 0x55d4d3410f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d33fdf00 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/2466647639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f5840 0x55d4d3405d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/2466647639 >> 172.21.15.62:0/2466647639 conn(0x55d4d33eabe0 msgr2=0x55d4d33efc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/2466647639 shutdown_connections 2023-07-14T16:29:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.446+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/2466647639 wait complete. 2023-07-14T16:29:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 1 -- start start 2023-07-14T16:29:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 1 Processor -- start 2023-07-14T16:29:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d3401750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f5840 0x55d4d3401c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 0x55d4d3404cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d34051f0 con 0x55d4d32516b0 2023-07-14T16:29:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d3405360 con 0x55d4d33f5840 2023-07-14T16:29:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4d3405520 con 0x55d4d33f66e0 2023-07-14T16:29:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.447+0000 7f6c9d4b4b80 10 client.0 _ll_register_callbacks cb 0x7f6c7c009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d3401750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c837fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d3401750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39294/0 (socket says 172.21.15.62:39294) 2023-07-14T16:29:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c82ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f5840 0x55d4d3401c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c837fe700 1 -- 172.21.15.62:0/3739621676 learned_addr learned my addr 172.21.15.62:0/3739621676 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c83fff700 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 0x55d4d3404cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c837fe700 1 -- 172.21.15.62:0/3739621676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 msgr2=0x55d4d3404cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c837fe700 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 0x55d4d3404cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c837fe700 1 -- 172.21.15.62:0/3739621676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f5840 msgr2=0x55d4d3401c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c837fe700 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f5840 0x55d4d3401c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c837fe700 1 -- 172.21.15.62:0/3739621676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4d315dc00 con 0x55d4d32516b0 2023-07-14T16:29:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.448+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f5840 0x55d4d3401c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.449+0000 7f6c837fe700 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d3401750 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000c00 tx=0x7f6c7400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0e22cf83e6fd54 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.449+0000 7f6c62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.449+0000 7f6c62ffd700 1 -- 172.21.15.62:0/3739621676 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400f460 con 0x55d4d32516b0 2023-07-14T16:29:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.449+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3739621676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c7401e040 con 0x55d4d32516b0 2023-07-14T16:29:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.449+0000 7f6c62ffd700 1 -- 172.21.15.62:0/3739621676 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c74000e40 con 0x55d4d32516b0 2023-07-14T16:29:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.449+0000 7f6c62ffd700 1 -- 172.21.15.62:0/3739621676 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c740052f0 con 0x55d4d32516b0 2023-07-14T16:29:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.449+0000 7f6c9d4b4b80 10 client.19263 fetch_fsmap learned FSMap version 26 2023-07-14T16:29:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.449+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3739621676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d4d3173250 con 0x55d4d32516b0 2023-07-14T16:29:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.450+0000 7f6c62ffd700 1 -- 172.21.15.62:0/3739621676 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7401e040 con 0x55d4d32516b0 2023-07-14T16:29:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.450+0000 7f6c62ffd700 1 -- 172.21.15.62:0/3739621676 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6c74052070 con 0x55d4d32516b0 2023-07-14T16:29:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.450+0000 7f6c9d4b4b80 10 client.19263 fetch_fsmap finished waiting for FSMap version 26 2023-07-14T16:29:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.450+0000 7f6c9d4b4b80 10 client.19263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.450+0000 7f6c9d4b4b80 20 client.19263 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.450+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d4d345e700 0x55d4d3460ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.450+0000 7f6c9d4b4b80 4 client.19263 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:29:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.450+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3739621676 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4d3301ff0 con 0x55d4d345e700 2023-07-14T16:29:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.451+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d4d345e700 0x55d4d3460ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.451+0000 7f6c62ffd700 10 client.19263 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.451+0000 7f6c82ffd700 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d4d345e700 0x55d4d3460ae0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262b3eb7a998b1fe server_cookie=2bd7a94b87f94298 in_seq=0 out_seq=0 2023-07-14T16:29:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c62ffd700 1 -- 172.21.15.62:0/3739621676 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4d3301ff0 con 0x55d4d345e700 2023-07-14T16:29:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c62ffd700 10 client.19263 handle_command_reply: tid=0 2023-07-14T16:29:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 client.19263 shutdown 2023-07-14T16:29:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3739621676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 msgr2=0x55d4d3401750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d3401750 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000c00 tx=0x7f6c7400bda0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3739621676 shutdown_connections 2023-07-14T16:29:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d4d33f66e0 0x55d4d3404cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d4d32516b0 0x55d4d3401750 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d4d345e700 0x55d4d3460ae0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 --2- 172.21.15.62:0/3739621676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4d33f5840 0x55d4d3401c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3739621676 >> 172.21.15.62:0/3739621676 conn(0x55d4d33eabe0 msgr2=0x55d4d33f9630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.461+0000 7f6c62ffd700 0 client.19263 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.462+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3739621676 shutdown_connections 2023-07-14T16:29:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.462+0000 7f6c9d4b4b80 1 -- 172.21.15.62:0/3739621676 wait complete. 2023-07-14T16:29:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:29:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:29:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:29:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:29:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.462+0000 7f6c9d4b4b80 20 client.19263 trim_cache size 0 max 16384 2023-07-14T16:29:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 -- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c6c0a12a0 msgr2=0x7f6c6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c6c0a12a0 0x7f6c6c0a3750 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55d4d33f5060 tx=0x7f6c7c004b40 comp rx=0 tx=0).stop 2023-07-14T16:29:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 -- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c84100cf0 msgr2=0x7f6c84198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c84100cf0 0x7f6c84198030 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800d950 tx=0x7f6c7800dd10 comp rx=0 tx=0).stop 2023-07-14T16:29:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 -- 172.21.15.62:0/2511019538 shutdown_connections 2023-07-14T16:29:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c84101680 0x7f6c84198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6c6c0a12a0 0x7f6c6c0a3750 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c840ffb00 0x7f6c84197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 --2- 172.21.15.62:0/2511019538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c84100cf0 0x7f6c84198030 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 -- 172.21.15.62:0/2511019538 >> 172.21.15.62:0/2511019538 conn(0x7f6c840fb0e0 msgr2=0x7f6c841066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 -- 172.21.15.62:0/2511019538 shutdown_connections 2023-07-14T16:29:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:03.463+0000 7f6c89bff700 1 -- 172.21.15.62:0/2511019538 wait complete. 2023-07-14T16:29:05.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:04 smithi062 ceph-mon[95543]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:29:05.135 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:04 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:29:05.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:04 smithi081 ceph-mon[127795]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:29:05.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:04 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:29:05.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:04 smithi164 ceph-mon[97427]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:29:05.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:04 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:29:07.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:06 smithi062 ceph-mon[95543]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:29:07.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:06 smithi081 ceph-mon[127795]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:29:07.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:06 smithi164 ceph-mon[97427]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:29:09.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:08 smithi062 ceph-mon[95543]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:29:09.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:08 smithi081 ceph-mon[127795]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:29:09.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:08 smithi164 ceph-mon[97427]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:29:11.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:10 smithi062 ceph-mon[95543]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:29:11.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:10 smithi081 ceph-mon[127795]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:29:11.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:10 smithi164 ceph-mon[97427]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:29:11.619 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:29:12.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 -- 172.21.15.62:0/49808260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9994101660 msgr2=0x7f9994107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 --2- 172.21.15.62:0/49808260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9994101660 0x7f9994107f20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f998c00b0a0 tx=0x7f998c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 -- 172.21.15.62:0/49808260 shutdown_connections 2023-07-14T16:29:12.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 --2- 172.21.15.62:0/49808260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9994101660 0x7f9994107f20 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 --2- 172.21.15.62:0/49808260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99940ffae0 0x7f99940ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 --2- 172.21.15.62:0/49808260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9994100cd0 0x7f9994101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 -- 172.21.15.62:0/49808260 >> 172.21.15.62:0/49808260 conn(0x7f99940fb0e0 msgr2=0x7f99940fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:12.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 -- 172.21.15.62:0/49808260 shutdown_connections 2023-07-14T16:29:12.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.843+0000 7f999c5ff700 1 -- 172.21.15.62:0/49808260 wait complete. 2023-07-14T16:29:12.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999c5ff700 1 Processor -- start 2023-07-14T16:29:12.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999c5ff700 1 -- start start 2023-07-14T16:29:12.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999c5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99940ffae0 0x7f9994195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999c5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9994100cd0 0x7f9994195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999c5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9994101660 0x7f999419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999c5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999419d3d0 con 0x7f99940ffae0 2023-07-14T16:29:12.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999c5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999419d510 con 0x7f9994101660 2023-07-14T16:29:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999c5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999419d6a0 con 0x7f9994100cd0 2023-07-14T16:29:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99940ffae0 0x7f9994195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99940ffae0 0x7f9994195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52796/0 (socket says 172.21.15.62:52796) 2023-07-14T16:29:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f999a39b700 1 -- 172.21.15.62:0/1646689338 learned_addr learned my addr 172.21.15.62:0/1646689338 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.845+0000 7f9999b9a700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9994100cd0 0x7f9994195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f999ab9c700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9994101660 0x7f999419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f999a39b700 1 -- 172.21.15.62:0/1646689338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9994100cd0 msgr2=0x7f9994195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f999a39b700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9994100cd0 0x7f9994195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f999a39b700 1 -- 172.21.15.62:0/1646689338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9994101660 msgr2=0x7f999419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f999a39b700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9994101660 0x7f999419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f999a39b700 1 -- 172.21.15.62:0/1646689338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c009d00 con 0x7f99940ffae0 2023-07-14T16:29:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f9999b9a700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9994100cd0 0x7f9994195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f999a39b700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99940ffae0 0x7f9994195850 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f999000e970 tx=0x7f999000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.846+0000 7f998b7fe700 1 -- 172.21.15.62:0/1646689338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990009790 con 0x7f99940ffae0 2023-07-14T16:29:12.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.847+0000 7f999c5ff700 1 -- 172.21.15.62:0/1646689338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999419d980 con 0x7f99940ffae0 2023-07-14T16:29:12.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.847+0000 7f998b7fe700 1 -- 172.21.15.62:0/1646689338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f999000c7e0 con 0x7f99940ffae0 2023-07-14T16:29:12.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.847+0000 7f998b7fe700 1 -- 172.21.15.62:0/1646689338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990003bd0 con 0x7f99940ffae0 2023-07-14T16:29:12.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.847+0000 7f999c5ff700 1 -- 172.21.15.62:0/1646689338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999419dee0 con 0x7f99940ffae0 2023-07-14T16:29:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.850+0000 7f998b7fe700 1 -- 172.21.15.62:0/1646689338 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f999000c950 con 0x7f99940ffae0 2023-07-14T16:29:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.852+0000 7f998b7fe700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f99800a1150 0x7f99800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.852+0000 7f998b7fe700 1 -- 172.21.15.62:0/1646689338 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9990014070 con 0x7f99940ffae0 2023-07-14T16:29:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.852+0000 7f9999b9a700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f99800a1150 0x7f99800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.853+0000 7f9999b9a700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f99800a1150 0x7f99800a3600 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9984007fd0 tx=0x7f99840073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:12.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.879+0000 7f99aff40b80 1 Processor -- start 2023-07-14T16:29:12.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.879+0000 7f99aff40b80 1 -- start start 2023-07-14T16:29:12.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.879+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a164d0 0x5605a9a168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.879+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.879+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a17c70 0x5605a9a180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f99aff40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a991bcb0 con 0x5605a9a16e20 2023-07-14T16:29:12.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f99aff40b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a996ea40 con 0x5605a9a164d0 2023-07-14T16:29:12.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f99aff40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9967ed0 con 0x5605a9a17c70 2023-07-14T16:29:12.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a176e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52812/0 (socket says 172.21.15.62:52812) 2023-07-14T16:29:12.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 -- 172.21.15.62:0/1499351487 learned_addr learned my addr 172.21.15.62:0/1499351487 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:12.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f999a39b700 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a164d0 0x5605a9a168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f999ab9c700 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a17c70 0x5605a9a180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 -- 172.21.15.62:0/1499351487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a17c70 msgr2=0x5605a9a180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:12.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a17c70 0x5605a9a180e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 -- 172.21.15.62:0/1499351487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a164d0 msgr2=0x5605a9a168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:12.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a164d0 0x5605a9a168e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 -- 172.21.15.62:0/1499351487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605a9795de0 con 0x5605a9a16e20 2023-07-14T16:29:12.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f999a39b700 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a164d0 0x5605a9a168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:12.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9999b9a700 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a176e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9984004350 tx=0x7f9984013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2905e90d61f7f3c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:12.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9988bfe700 1 -- 172.21.15.62:0/1499351487 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984003900 con 0x5605a9a16e20 2023-07-14T16:29:12.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9988bfe700 1 -- 172.21.15.62:0/1499351487 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9984011440 con 0x5605a9a16e20 2023-07-14T16:29:12.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.881+0000 7f9988bfe700 1 -- 172.21.15.62:0/1499351487 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998401f690 con 0x5605a9a16e20 2023-07-14T16:29:12.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 -- 172.21.15.62:0/1499351487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 msgr2=0x5605a9a176e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a176e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9984004350 tx=0x7f9984013000 comp rx=0 tx=0).stop 2023-07-14T16:29:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 -- 172.21.15.62:0/1499351487 shutdown_connections 2023-07-14T16:29:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a17c70 0x5605a9a180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a164d0 0x5605a9a168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 --2- 172.21.15.62:0/1499351487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a176e0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 -- 172.21.15.62:0/1499351487 >> 172.21.15.62:0/1499351487 conn(0x5605a9a0bda0 msgr2=0x5605a9a111d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 -- 172.21.15.62:0/1499351487 shutdown_connections 2023-07-14T16:29:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.882+0000 7f99aff40b80 1 -- 172.21.15.62:0/1499351487 wait complete. 2023-07-14T16:29:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.883+0000 7f99aff40b80 1 -- start start 2023-07-14T16:29:12.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.883+0000 7f99aff40b80 1 Processor -- start 2023-07-14T16:29:12.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.883+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.883+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a16e20 0x5605a9a24af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a17c70 0x5605a9a2bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f99aff40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9907ac0 con 0x5605a9a17c70 2023-07-14T16:29:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f99aff40b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a99077c0 con 0x5605a9a16e20 2023-07-14T16:29:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f99aff40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a97eb920 con 0x5605a9a164d0 2023-07-14T16:29:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f99aff40b80 10 client.0 _ll_register_callbacks cb 0x5605a9905760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999a39b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f9999b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a16e20 0x5605a9a24af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999a39b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a245b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50994/0 (socket says 172.21.15.62:50994) 2023-07-14T16:29:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999a39b700 1 -- 172.21.15.62:0/4047158945 learned_addr learned my addr 172.21.15.62:0/4047158945 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999ab9c700 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a17c70 0x5605a9a2bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999a39b700 1 -- 172.21.15.62:0/4047158945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a16e20 msgr2=0x5605a9a24af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999a39b700 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a16e20 0x5605a9a24af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999a39b700 1 -- 172.21.15.62:0/4047158945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a17c70 msgr2=0x5605a9a2bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999a39b700 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a17c70 0x5605a9a2bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999a39b700 1 -- 172.21.15.62:0/4047158945 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9984004030 con 0x5605a9a164d0 2023-07-14T16:29:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f999ab9c700 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a17c70 0x5605a9a2bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.884+0000 7f9999b9a700 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a16e20 0x5605a9a24af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.885+0000 7f999a39b700 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a245b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f99900ca600 tx=0x7f999000cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3145c784f377b47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.885+0000 7f997dffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:29:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.885+0000 7f997dffb700 1 -- 172.21.15.62:0/4047158945 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999009a370 con 0x5605a9a164d0 2023-07-14T16:29:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.885+0000 7f997dffb700 1 -- 172.21.15.62:0/4047158945 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f99900a2360 con 0x5605a9a164d0 2023-07-14T16:29:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.885+0000 7f99aff40b80 1 -- 172.21.15.62:0/4047158945 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605a98c14f0 con 0x5605a9a164d0 2023-07-14T16:29:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.885+0000 7f997dffb700 1 -- 172.21.15.62:0/4047158945 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9990099400 con 0x5605a9a164d0 2023-07-14T16:29:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.885+0000 7f99aff40b80 10 client.28720 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.885+0000 7f99aff40b80 1 -- 172.21.15.62:0/4047158945 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5605a977ec60 con 0x5605a9a164d0 2023-07-14T16:29:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.886+0000 7f997dffb700 1 -- 172.21.15.62:0/4047158945 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5605a98c14f0 con 0x5605a9a164d0 2023-07-14T16:29:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.886+0000 7f997dffb700 1 -- 172.21.15.62:0/4047158945 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9990094b20 con 0x5605a9a164d0 2023-07-14T16:29:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.886+0000 7f99aff40b80 10 client.28720 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.886+0000 7f99aff40b80 10 client.28720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.886+0000 7f99aff40b80 20 client.28720 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.886+0000 7f99aff40b80 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605a9a302e0 0x5605a9a326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.886+0000 7f99aff40b80 4 client.28720 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:12.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.886+0000 7f99aff40b80 1 -- 172.21.15.62:0/4047158945 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605a9960eb0 con 0x5605a9a302e0 2023-07-14T16:29:12.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.887+0000 7f9999b9a700 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605a9a302e0 0x5605a9a326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.887+0000 7f9999b9a700 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605a9a302e0 0x5605a9a326f0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de09f3e946ba583 server_cookie=fe89c1ab91f0d6f3 in_seq=0 out_seq=0 2023-07-14T16:29:12.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.887+0000 7f997dffb700 10 client.28720 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:12.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f997dffb700 1 -- 172.21.15.62:0/4047158945 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5605a9960eb0 con 0x5605a9a302e0 2023-07-14T16:29:12.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f997dffb700 10 client.28720 handle_command_reply: tid=0 2023-07-14T16:29:12.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 client.28720 shutdown 2023-07-14T16:29:12.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 -- 172.21.15.62:0/4047158945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 msgr2=0x5605a9a245b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a245b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f99900ca600 tx=0x7f999000cda0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 -- 172.21.15.62:0/4047158945 shutdown_connections 2023-07-14T16:29:12.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a17c70 0x5605a9a2bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a245b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605a9a302e0 0x5605a9a326f0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 --2- 172.21.15.62:0/4047158945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a16e20 0x5605a9a24af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f997dffb700 0 client.28720 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:12.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 -- 172.21.15.62:0/4047158945 >> 172.21.15.62:0/4047158945 conn(0x5605a9a0bda0 msgr2=0x5605a9a108b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:12.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 -- 172.21.15.62:0/4047158945 shutdown_connections 2023-07-14T16:29:12.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.888+0000 7f99aff40b80 1 -- 172.21.15.62:0/4047158945 wait complete. 2023-07-14T16:29:12.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.889+0000 7f99aff40b80 20 client.28720 trim_cache size 0 max 16384 2023-07-14T16:29:12.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.913+0000 7f99aff40b80 1 Processor -- start 2023-07-14T16:29:12.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.913+0000 7f99aff40b80 1 -- start start 2023-07-14T16:29:12.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.913+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a1e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.913+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a2baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a1abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f99aff40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a98c14f0 con 0x5605a9a16e20 2023-07-14T16:29:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f99aff40b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9a2d820 con 0x5605a9a17c70 2023-07-14T16:29:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f99aff40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9a1de30 con 0x5605a9a164d0 2023-07-14T16:29:12.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f9999b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a2baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f9999b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a2baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52818/0 (socket says 172.21.15.62:52818) 2023-07-14T16:29:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f9999b9a700 1 -- 172.21.15.62:0/643337174 learned_addr learned my addr 172.21.15.62:0/643337174 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f999a39b700 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a1e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f999ab9c700 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a1abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f999a39b700 1 -- 172.21.15.62:0/643337174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 msgr2=0x5605a9a1abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f999a39b700 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a1abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f999a39b700 1 -- 172.21.15.62:0/643337174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 msgr2=0x5605a9a2baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f999a39b700 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a2baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f999a39b700 1 -- 172.21.15.62:0/643337174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605a97bc250 con 0x5605a9a164d0 2023-07-14T16:29:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f999ab9c700 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a1abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.914+0000 7f9999b9a700 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a2baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.915+0000 7f999a39b700 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a1e080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9990099bc0 tx=0x7f999000c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b3e8cca22549d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:12.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.915+0000 7f99893ff700 1 -- 172.21.15.62:0/643337174 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99900a2360 con 0x5605a9a164d0 2023-07-14T16:29:12.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.915+0000 7f99893ff700 1 -- 172.21.15.62:0/643337174 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9990099f10 con 0x5605a9a164d0 2023-07-14T16:29:12.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.915+0000 7f99893ff700 1 -- 172.21.15.62:0/643337174 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99900ca180 con 0x5605a9a164d0 2023-07-14T16:29:12.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 -- 172.21.15.62:0/643337174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 msgr2=0x5605a9a1e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a1e080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9990099bc0 tx=0x7f999000c450 comp rx=0 tx=0).stop 2023-07-14T16:29:12.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 -- 172.21.15.62:0/643337174 shutdown_connections 2023-07-14T16:29:12.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a1abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a1e080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 --2- 172.21.15.62:0/643337174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a2baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 -- 172.21.15.62:0/643337174 >> 172.21.15.62:0/643337174 conn(0x5605a9a0bda0 msgr2=0x5605a9a232c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 -- 172.21.15.62:0/643337174 shutdown_connections 2023-07-14T16:29:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 -- 172.21.15.62:0/643337174 wait complete. 2023-07-14T16:29:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 -- start start 2023-07-14T16:29:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.916+0000 7f99aff40b80 1 Processor -- start 2023-07-14T16:29:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a2b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f99aff40b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a28820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f99aff40b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9a28d60 con 0x5605a9a16e20 2023-07-14T16:29:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f99aff40b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9a28ed0 con 0x5605a9a17c70 2023-07-14T16:29:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f99aff40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a9a84190 con 0x5605a9a164d0 2023-07-14T16:29:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f99aff40b80 10 client.0 _ll_register_callbacks cb 0x7f9990099870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f9999b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f9999b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a282e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52824/0 (socket says 172.21.15.62:52824) 2023-07-14T16:29:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.917+0000 7f9999b9a700 1 -- 172.21.15.62:0/3974986961 learned_addr learned my addr 172.21.15.62:0/3974986961 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f999a39b700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a2b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f999ab9c700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a28820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f9999b9a700 1 -- 172.21.15.62:0/3974986961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 msgr2=0x5605a9a2b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f9999b9a700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a2b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f9999b9a700 1 -- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 msgr2=0x5605a9a28820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f9999b9a700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a28820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f9999b9a700 1 -- 172.21.15.62:0/3974986961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605a9795de0 con 0x5605a9a16e20 2023-07-14T16:29:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f999a39b700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a2b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f999ab9c700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a28820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f9999b9a700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a282e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f9984005e40 tx=0x7f99840115b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e198848cc1edefa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f997dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f997dffb700 1 -- 172.21.15.62:0/3974986961 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984031350 con 0x5605a9a16e20 2023-07-14T16:29:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f99aff40b80 1 -- 172.21.15.62:0/3974986961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9984011440 con 0x5605a9a16e20 2023-07-14T16:29:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.918+0000 7f997dffb700 1 -- 172.21.15.62:0/3974986961 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9984011e40 con 0x5605a9a16e20 2023-07-14T16:29:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f997dffb700 1 -- 172.21.15.62:0/3974986961 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99840314b0 con 0x5605a9a16e20 2023-07-14T16:29:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f99aff40b80 10 client.19281 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f99aff40b80 1 -- 172.21.15.62:0/3974986961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5605a977ec60 con 0x5605a9a16e20 2023-07-14T16:29:12.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f997dffb700 1 -- 172.21.15.62:0/3974986961 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9984011440 con 0x5605a9a16e20 2023-07-14T16:29:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f997dffb700 1 -- 172.21.15.62:0/3974986961 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9984031910 con 0x5605a9a16e20 2023-07-14T16:29:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f99aff40b80 10 client.19281 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f99aff40b80 10 client.19281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f99aff40b80 20 client.19281 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:12.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f99aff40b80 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605a9a85a80 0x5605a9a87e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f99aff40b80 4 client.19281 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:29:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.919+0000 7f99aff40b80 1 -- 172.21.15.62:0/3974986961 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5605a9923050 con 0x5605a9a85a80 2023-07-14T16:29:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.920+0000 7f999a39b700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605a9a85a80 0x5605a9a87e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.920+0000 7f999a39b700 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605a9a85a80 0x5605a9a87e90 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=558f5dad98623b03 server_cookie=7bffcb2e3ceb4e61 in_seq=0 out_seq=0 2023-07-14T16:29:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.920+0000 7f997dffb700 10 client.19281 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.920+0000 7f997dffb700 1 -- 172.21.15.62:0/3974986961 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5605a9923050 con 0x5605a9a85a80 2023-07-14T16:29:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.920+0000 7f997dffb700 10 client.19281 handle_command_reply: tid=0 2023-07-14T16:29:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.920+0000 7f99aff40b80 1 client.19281 shutdown 2023-07-14T16:29:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 -- 172.21.15.62:0/3974986961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 msgr2=0x5605a9a282e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a282e0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f9984005e40 tx=0x7f99840115b0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 -- 172.21.15.62:0/3974986961 shutdown_connections 2023-07-14T16:29:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5605a9a85a80 0x5605a9a87e90 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5605a9a17c70 0x5605a9a28820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5605a9a164d0 0x5605a9a2b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 --2- 172.21.15.62:0/3974986961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a9a16e20 0x5605a9a282e0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 -- 172.21.15.62:0/3974986961 >> 172.21.15.62:0/3974986961 conn(0x5605a9a0bda0 msgr2=0x5605a9a1e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f997dffb700 0 client.19281 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 -- 172.21.15.62:0/3974986961 shutdown_connections 2023-07-14T16:29:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 1 -- 172.21.15.62:0/3974986961 wait complete. 2023-07-14T16:29:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.921+0000 7f99aff40b80 20 client.19281 trim_cache size 0 max 16384 2023-07-14T16:29:12.911 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:29:12.911 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:29:12.911 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:29:12.911 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:29:12.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 -- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f99800a1150 msgr2=0x7f99800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f99800a1150 0x7f99800a3600 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9984007fd0 tx=0x7f99840073d0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 -- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99940ffae0 msgr2=0x7f9994195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:12.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99940ffae0 0x7f9994195850 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f999000e970 tx=0x7f999000ec80 comp rx=0 tx=0).stop 2023-07-14T16:29:12.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 -- 172.21.15.62:0/1646689338 shutdown_connections 2023-07-14T16:29:12.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f99800a1150 0x7f99800a3600 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9994101660 0x7f999419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99940ffae0 0x7f9994195850 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 --2- 172.21.15.62:0/1646689338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9994100cd0 0x7f9994195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:12.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 -- 172.21.15.62:0/1646689338 >> 172.21.15.62:0/1646689338 conn(0x7f99940fb0e0 msgr2=0x7f9994104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:12.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 -- 172.21.15.62:0/1646689338 shutdown_connections 2023-07-14T16:29:12.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:12.922+0000 7f999c5ff700 1 -- 172.21.15.62:0/1646689338 wait complete. 2023-07-14T16:29:12.915 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:29:12.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:29:13.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:12 smithi062 ceph-mon[95543]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:29:13.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:12 smithi081 ceph-mon[127795]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:29:13.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.287+0000 7f6ac85ff700 1 -- 172.21.15.62:0/512444640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 msgr2=0x7f6ac0105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.287+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/512444640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 0x7f6ac0105250 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009990 tx=0x7f6ab0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.288+0000 7f6ac85ff700 1 -- 172.21.15.62:0/512444640 shutdown_connections 2023-07-14T16:29:13.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.288+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/512444640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ac0105790 0x7f6ac0107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.288+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/512444640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ac0100550 0x7f6ac0102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.288+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/512444640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 0x7f6ac0105250 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.288+0000 7f6ac85ff700 1 -- 172.21.15.62:0/512444640 >> 172.21.15.62:0/512444640 conn(0x7f6ac00f9fd0 msgr2=0x7f6ac00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.288+0000 7f6ac85ff700 1 -- 172.21.15.62:0/512444640 shutdown_connections 2023-07-14T16:29:13.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.288+0000 7f6ac85ff700 1 -- 172.21.15.62:0/512444640 wait complete. 2023-07-14T16:29:13.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.289+0000 7f6ac85ff700 1 Processor -- start 2023-07-14T16:29:13.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.289+0000 7f6ac85ff700 1 -- start start 2023-07-14T16:29:13.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac85ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ac0100550 0x7f6ac0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac85ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 0x7f6ac0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac85ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ac0105790 0x7f6ac019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac85ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac019f4e0 con 0x7f6ac0102e70 2023-07-14T16:29:13.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac85ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac019f620 con 0x7f6ac0100550 2023-07-14T16:29:13.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac85ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac019f7b0 con 0x7f6ac0105790 2023-07-14T16:29:13.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac5b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 0x7f6ac0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac5b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 0x7f6ac0197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52836/0 (socket says 172.21.15.62:52836) 2023-07-14T16:29:13.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac639b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ac0100550 0x7f6ac0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac5b9a700 1 -- 172.21.15.62:0/3251010891 learned_addr learned my addr 172.21.15.62:0/3251010891 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:13.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac6b9c700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ac0105790 0x7f6ac019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac5b9a700 1 -- 172.21.15.62:0/3251010891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ac0105790 msgr2=0x7f6ac019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac5b9a700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ac0105790 0x7f6ac019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac5b9a700 1 -- 172.21.15.62:0/3251010891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ac0100550 msgr2=0x7f6ac0197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac5b9a700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ac0100550 0x7f6ac0197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.290+0000 7f6ac5b9a700 1 -- 172.21.15.62:0/3251010891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab00095f0 con 0x7f6ac0102e70 2023-07-14T16:29:13.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.291+0000 7f6ac5b9a700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 0x7f6ac0197ea0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0005910 tx=0x7f6ab0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:13.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.291+0000 7f6ab77fe700 1 -- 172.21.15.62:0/3251010891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab001d070 con 0x7f6ac0102e70 2023-07-14T16:29:13.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.291+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac019fa30 con 0x7f6ac0102e70 2023-07-14T16:29:13.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.291+0000 7f6ab77fe700 1 -- 172.21.15.62:0/3251010891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ab0022470 con 0x7f6ac0102e70 2023-07-14T16:29:13.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.291+0000 7f6ab77fe700 1 -- 172.21.15.62:0/3251010891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0022ea0 con 0x7f6ac0102e70 2023-07-14T16:29:13.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:12 smithi164 ceph-mon[97427]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:29:13.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.291+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac019fe60 con 0x7f6ac0102e70 2023-07-14T16:29:13.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.293+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac0107350 con 0x7f6ac0102e70 2023-07-14T16:29:13.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.295+0000 7f6ab77fe700 1 -- 172.21.15.62:0/3251010891 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6ab000ba60 con 0x7f6ac0102e70 2023-07-14T16:29:13.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.297+0000 7f6ab77fe700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6aac0a11a0 0x7f6aac0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.297+0000 7f6ab77fe700 1 -- 172.21.15.62:0/3251010891 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6ab00cdd40 con 0x7f6ac0102e70 2023-07-14T16:29:13.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.297+0000 7f6ac639b700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6aac0a11a0 0x7f6aac0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.297+0000 7f6ac639b700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6aac0a11a0 0x7f6aac0a3650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009c60 tx=0x7f6abc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:13.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.300+0000 7f6ab77fe700 1 -- 172.21.15.62:0/3251010891 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6ab005c790 con 0x7f6ac0102e70 2023-07-14T16:29:13.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.610+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ac002cce0 con 0x7f6ac0102e70 2023-07-14T16:29:13.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.611+0000 7f6ab77fe700 1 -- 172.21.15.62:0/3251010891 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10828 (secure 0 0 0) 0x7f6ab00974d0 con 0x7f6ac0102e70 2023-07-14T16:29:13.585 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:29:13.585 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:29:03.831862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:29:13.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.613+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6aac0a11a0 msgr2=0x7f6aac0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.613+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6aac0a11a0 0x7f6aac0a3650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009c60 tx=0x7f6abc009260 comp rx=0 tx=0).stop 2023-07-14T16:29:13.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.613+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 msgr2=0x7f6ac0197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.613+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 0x7f6ac0197ea0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0005910 tx=0x7f6ab0005720 comp rx=0 tx=0).stop 2023-07-14T16:29:13.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.614+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 shutdown_connections 2023-07-14T16:29:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.614+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ac0105790 0x7f6ac019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.614+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6aac0a11a0 0x7f6aac0a3650 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.614+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ac0100550 0x7f6ac0197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.614+0000 7f6ac85ff700 1 --2- 172.21.15.62:0/3251010891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac0102e70 0x7f6ac0197ea0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.614+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 >> 172.21.15.62:0/3251010891 conn(0x7f6ac00f9fd0 msgr2=0x7f6ac00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:13.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.614+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 shutdown_connections 2023-07-14T16:29:13.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.614+0000 7f6ac85ff700 1 -- 172.21.15.62:0/3251010891 wait complete. 2023-07-14T16:29:13.591 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2023-07-14T16:29:13.610 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:29:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 -- 172.21.15.62:0/3279234752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 msgr2=0x7f75bc107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 --2- 172.21.15.62:0/3279234752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 0x7f75bc107c00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b0a0 tx=0x7f75b400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 -- 172.21.15.62:0/3279234752 shutdown_connections 2023-07-14T16:29:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 --2- 172.21.15.62:0/3279234752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 0x7f75bc107c00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 --2- 172.21.15.62:0/3279234752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75bc100560 0x7f75bc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 --2- 172.21.15.62:0/3279234752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75bc102e80 0x7f75bc105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 -- 172.21.15.62:0/3279234752 >> 172.21.15.62:0/3279234752 conn(0x7f75bc0fa000 msgr2=0x7f75bc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:13.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 -- 172.21.15.62:0/3279234752 shutdown_connections 2023-07-14T16:29:13.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.869+0000 7f75c0fff700 1 -- 172.21.15.62:0/3279234752 wait complete. 2023-07-14T16:29:13.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.870+0000 7f75c0fff700 1 Processor -- start 2023-07-14T16:29:13.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.870+0000 7f75c0fff700 1 -- start start 2023-07-14T16:29:13.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.870+0000 7f75c0fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75bc100560 0x7f75bc1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.870+0000 7f75c0fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75bc102e80 0x7f75bc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75c0fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 0x7f75bc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75c0fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc19f570 con 0x7f75bc100560 2023-07-14T16:29:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75c0fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc19f6b0 con 0x7f75bc1057a0 2023-07-14T16:29:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75c0fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc19f840 con 0x7f75bc102e80 2023-07-14T16:29:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75ba59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75bc100560 0x7f75bc1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75ba59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75bc100560 0x7f75bc1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52852/0 (socket says 172.21.15.62:52852) 2023-07-14T16:29:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75ba59c700 1 -- 172.21.15.62:0/581637401 learned_addr learned my addr 172.21.15.62:0/581637401 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75bc102e80 0x7f75bc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75bad9d700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 0x7f75bc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75ba59c700 1 -- 172.21.15.62:0/581637401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75bc102e80 msgr2=0x7f75bc197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75ba59c700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75bc102e80 0x7f75bc197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75ba59c700 1 -- 172.21.15.62:0/581637401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 msgr2=0x7f75bc19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75ba59c700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 0x7f75bc19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75ba59c700 1 -- 172.21.15.62:0/581637401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b4009d00 con 0x7f75bc100560 2023-07-14T16:29:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.871+0000 7f75bad9d700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 0x7f75bc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.872+0000 7f75ba59c700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75bc100560 0x7f75bc1979f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00e9a0 tx=0x7f75ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.872+0000 7f75ab7fe700 1 -- 172.21.15.62:0/581637401 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac009790 con 0x7f75bc100560 2023-07-14T16:29:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.872+0000 7f75c0fff700 1 -- 172.21.15.62:0/581637401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75bc19fb20 con 0x7f75bc100560 2023-07-14T16:29:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.872+0000 7f75ab7fe700 1 -- 172.21.15.62:0/581637401 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75ac00cbd0 con 0x7f75bc100560 2023-07-14T16:29:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.872+0000 7f75ab7fe700 1 -- 172.21.15.62:0/581637401 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac018680 con 0x7f75bc100560 2023-07-14T16:29:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.872+0000 7f75c0fff700 1 -- 172.21.15.62:0/581637401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc1a0080 con 0x7f75bc100560 2023-07-14T16:29:13.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.878+0000 7f75ab7fe700 1 -- 172.21.15.62:0/581637401 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f75ac00cd40 con 0x7f75bc100560 2023-07-14T16:29:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.879+0000 7f75d48bab80 1 Processor -- start 2023-07-14T16:29:13.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75d48bab80 1 -- start start 2023-07-14T16:29:13.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378707ce0 0x5573787080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75ab7fe700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f75a40a1150 0x7f75a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378708ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x5573787098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75d48bab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378604cb0 con 0x557378709480 2023-07-14T16:29:13.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75d48bab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378657a40 con 0x557378708630 2023-07-14T16:29:13.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75d48bab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378650ed0 con 0x557378707ce0 2023-07-14T16:29:13.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.880+0000 7f75ab7fe700 1 -- 172.21.15.62:0/581637401 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f75ac014070 con 0x7f75bc100560 2023-07-14T16:29:13.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.881+0000 7f75bad9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378707ce0 0x5573787080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.881+0000 7f75bad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x5573787098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.881+0000 7f75ba59c700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f75a40a1150 0x7f75a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.881+0000 7f75bad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x5573787098f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52854/0 (socket says 172.21.15.62:52854) 2023-07-14T16:29:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.881+0000 7f75bad9d700 1 -- 172.21.15.62:0/2668685925 learned_addr learned my addr 172.21.15.62:0/2668685925 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.881+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/2668685925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378708ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75bad9d700 1 -- 172.21.15.62:0/2668685925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378707ce0 msgr2=0x5573787080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75bad9d700 1 --2- 172.21.15.62:0/2668685925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378707ce0 0x5573787080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75bad9d700 1 -- 172.21.15.62:0/2668685925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 msgr2=0x557378708ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75bad9d700 1 --2- 172.21.15.62:0/2668685925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378708ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75bad9d700 1 -- 172.21.15.62:0/2668685925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737847ede0 con 0x557378709480 2023-07-14T16:29:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75ba59c700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f75a40a1150 0x7f75a40a3600 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75ac099f50 tx=0x7f75ac084f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75bad9d700 1 --2- 172.21.15.62:0/2668685925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x5573787098f0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f75b4015000 tx=0x7f75b400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4bf40eb5d94d93 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75a8ff9700 1 -- 172.21.15.62:0/2668685925 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b400e040 con 0x557378709480 2023-07-14T16:29:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75a8ff9700 1 -- 172.21.15.62:0/2668685925 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75b4007b90 con 0x557378709480 2023-07-14T16:29:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.882+0000 7f75a8ff9700 1 -- 172.21.15.62:0/2668685925 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b401d460 con 0x557378709480 2023-07-14T16:29:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.883+0000 7f75d48bab80 1 -- 172.21.15.62:0/2668685925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 msgr2=0x5573787098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.883+0000 7f75d48bab80 1 --2- 172.21.15.62:0/2668685925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x5573787098f0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f75b4015000 tx=0x7f75b400b700 comp rx=0 tx=0).stop 2023-07-14T16:29:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.884+0000 7f75d48bab80 1 -- 172.21.15.62:0/2668685925 shutdown_connections 2023-07-14T16:29:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.884+0000 7f75d48bab80 1 --2- 172.21.15.62:0/2668685925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x5573787098f0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.884+0000 7f75d48bab80 1 --2- 172.21.15.62:0/2668685925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378707ce0 0x5573787080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.884+0000 7f75d48bab80 1 --2- 172.21.15.62:0/2668685925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378708ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.884+0000 7f75d48bab80 1 -- 172.21.15.62:0/2668685925 >> 172.21.15.62:0/2668685925 conn(0x5573786f4da0 msgr2=0x5573786fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.884+0000 7f75d48bab80 1 -- 172.21.15.62:0/2668685925 shutdown_connections 2023-07-14T16:29:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.884+0000 7f75d48bab80 1 -- 172.21.15.62:0/2668685925 wait complete. 2023-07-14T16:29:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.884+0000 7f75d48bab80 1 -- start start 2023-07-14T16:29:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75d48bab80 1 Processor -- start 2023-07-14T16:29:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x5573787118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378711e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 0x557378718e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75d48bab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573785f0ac0 con 0x557378707ce0 2023-07-14T16:29:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75d48bab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573785f07c0 con 0x557378708630 2023-07-14T16:29:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75d48bab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573784d4920 con 0x557378709480 2023-07-14T16:29:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75d48bab80 10 client.0 _ll_register_callbacks cb 0x5573785ee760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75bad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x5573787118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75bad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x5573787118e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52868/0 (socket says 172.21.15.62:52868) 2023-07-14T16:29:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75bad9d700 1 -- 172.21.15.62:0/1328220295 learned_addr learned my addr 172.21.15.62:0/1328220295 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.885+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378711e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.886+0000 7f75bad9d700 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 0x557378718e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.886+0000 7f75bad9d700 1 -- 172.21.15.62:0/1328220295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 msgr2=0x557378718e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.886+0000 7f75bad9d700 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 0x557378718e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.886+0000 7f75bad9d700 1 -- 172.21.15.62:0/1328220295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 msgr2=0x557378711e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.886+0000 7f75bad9d700 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378711e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.886+0000 7f75bad9d700 1 -- 172.21.15.62:0/1328220295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc1087f0 con 0x557378707ce0 2023-07-14T16:29:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.886+0000 7f75bad9d700 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x5573787118e0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009ca0 tx=0x7f75b40061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687c7af1e6e71360 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.886+0000 7f75a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.887+0000 7f75a1ffb700 1 -- 172.21.15.62:0/1328220295 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b400e040 con 0x557378707ce0 2023-07-14T16:29:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.887+0000 7f75d48bab80 1 -- 172.21.15.62:0/1328220295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573785aa4f0 con 0x557378707ce0 2023-07-14T16:29:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.887+0000 7f75a1ffb700 1 -- 172.21.15.62:0/1328220295 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f75b4007750 con 0x557378707ce0 2023-07-14T16:29:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.887+0000 7f75a1ffb700 1 -- 172.21.15.62:0/1328220295 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b4005300 con 0x557378707ce0 2023-07-14T16:29:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.887+0000 7f75d48bab80 10 client.19299 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.887+0000 7f75d48bab80 1 -- 172.21.15.62:0/1328220295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557378467c60 con 0x557378707ce0 2023-07-14T16:29:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.887+0000 7f75a1ffb700 1 -- 172.21.15.62:0/1328220295 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5573785aa4f0 con 0x557378707ce0 2023-07-14T16:29:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.887+0000 7f75a1ffb700 1 -- 172.21.15.62:0/1328220295 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f75b4019070 con 0x557378707ce0 2023-07-14T16:29:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.888+0000 7f75d48bab80 10 client.19299 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.888+0000 7f75d48bab80 10 client.19299 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:29:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.888+0000 7f75d48bab80 20 client.19299 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.888+0000 7f75d48bab80 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55737871d210 0x55737871f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.888+0000 7f75d48bab80 4 client.19299 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.888+0000 7f75d48bab80 1 -- 172.21.15.62:0/1328220295 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557378649eb0 con 0x55737871d210 2023-07-14T16:29:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.888+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55737871d210 0x55737871f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.889+0000 7f75a1ffb700 10 client.19299 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:29:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.889+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55737871d210 0x55737871f5f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcc00127d01a9c5d server_cookie=21546022cf9616f5 in_seq=0 out_seq=0 2023-07-14T16:29:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75a1ffb700 1 -- 172.21.15.62:0/1328220295 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557378649eb0 con 0x55737871d210 2023-07-14T16:29:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75a1ffb700 10 client.19299 handle_command_reply: tid=0 2023-07-14T16:29:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 client.19299 shutdown 2023-07-14T16:29:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 -- 172.21.15.62:0/1328220295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 msgr2=0x5573787118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x5573787118e0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009ca0 tx=0x7f75b40061f0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 -- 172.21.15.62:0/1328220295 shutdown_connections 2023-07-14T16:29:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55737871d210 0x55737871f5f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 0x557378718e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x5573787118e0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 --2- 172.21.15.62:0/1328220295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378711e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 -- 172.21.15.62:0/1328220295 >> 172.21.15.62:0/1328220295 conn(0x5573786f4da0 msgr2=0x5573786f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75a1ffb700 0 client.19299 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:29:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 -- 172.21.15.62:0/1328220295 shutdown_connections 2023-07-14T16:29:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 1 -- 172.21.15.62:0/1328220295 wait complete. 2023-07-14T16:29:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.890+0000 7f75d48bab80 20 client.19299 trim_cache size 0 max 16384 2023-07-14T16:29:13.871 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:13 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3251010891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:29:13.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.908+0000 7f75d48bab80 1 Processor -- start 2023-07-14T16:29:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.908+0000 7f75d48bab80 1 -- start start 2023-07-14T16:29:13.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.908+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x557378709850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.908+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378718e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.908+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 0x557378723f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.908+0000 7f75d48bab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737870f620 con 0x557378707ce0 2023-07-14T16:29:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.908+0000 7f75d48bab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573786fbca0 con 0x557378708630 2023-07-14T16:29:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.908+0000 7f75d48bab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378724450 con 0x557378709480 2023-07-14T16:29:13.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378718e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75bad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x557378709850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378718e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36338/0 (socket says 172.21.15.62:36338) 2023-07-14T16:29:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 -- 172.21.15.62:0/3517145589 learned_addr learned my addr 172.21.15.62:0/3517145589 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75bad9d700 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x557378709850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52874/0 (socket says 172.21.15.62:52874) 2023-07-14T16:29:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75bad9d700 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 0x557378723f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 -- 172.21.15.62:0/3517145589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 msgr2=0x557378723f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 0x557378723f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 -- 172.21.15.62:0/3517145589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 msgr2=0x557378709850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:13.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x557378709850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 -- 172.21.15.62:0/3517145589 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573784a5250 con 0x557378708630 2023-07-14T16:29:13.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75bad9d700 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x557378709850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:13.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.909+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378718e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75b00097c0 tx=0x7f75b0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75c7e38555735b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:13.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.910+0000 7f75a97fa700 1 -- 172.21.15.62:0/3517145589 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0017030 con 0x557378708630 2023-07-14T16:29:13.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.910+0000 7f75a97fa700 1 -- 172.21.15.62:0/3517145589 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75b0018040 con 0x557378708630 2023-07-14T16:29:13.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.910+0000 7f75a97fa700 1 -- 172.21.15.62:0/3517145589 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0003bd0 con 0x557378708630 2023-07-14T16:29:13.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 -- 172.21.15.62:0/3517145589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 msgr2=0x557378718e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378718e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75b00097c0 tx=0x7f75b0009260 comp rx=0 tx=0).stop 2023-07-14T16:29:13.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 -- 172.21.15.62:0/3517145589 shutdown_connections 2023-07-14T16:29:13.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378709480 0x557378723f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378707ce0 0x557378709850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3517145589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378708630 0x557378718e20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 -- 172.21.15.62:0/3517145589 >> 172.21.15.62:0/3517145589 conn(0x5573786f4da0 msgr2=0x5573786fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:13.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 -- 172.21.15.62:0/3517145589 shutdown_connections 2023-07-14T16:29:13.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 -- 172.21.15.62:0/3517145589 wait complete. 2023-07-14T16:29:13.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.911+0000 7f75d48bab80 1 -- start start 2023-07-14T16:29:13.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75d48bab80 1 Processor -- start 2023-07-14T16:29:13.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378707ce0 0x557378714790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378708630 0x557378714cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75d48bab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x557378717c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75d48bab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573787181c0 con 0x557378709480 2023-07-14T16:29:13.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75d48bab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378718330 con 0x557378707ce0 2023-07-14T16:29:13.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75d48bab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573787184f0 con 0x557378708630 2023-07-14T16:29:13.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75d48bab80 10 client.0 _ll_register_callbacks cb 0x7f75b0009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:13.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75bad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x557378717c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75bad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x557378717c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52878/0 (socket says 172.21.15.62:52878) 2023-07-14T16:29:13.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.912+0000 7f75bad9d700 1 -- 172.21.15.62:0/3541426194 learned_addr learned my addr 172.21.15.62:0/3541426194 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:13.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75bad9d700 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378707ce0 0x557378714790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378708630 0x557378714cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75bad9d700 1 -- 172.21.15.62:0/3541426194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378708630 msgr2=0x557378714cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:13.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75bad9d700 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378708630 0x557378714cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75bad9d700 1 -- 172.21.15.62:0/3541426194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378707ce0 msgr2=0x557378714790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:13.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75bad9d700 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378707ce0 0x557378714790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75bad9d700 1 -- 172.21.15.62:0/3541426194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557378467c60 con 0x557378709480 2023-07-14T16:29:13.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378708630 0x557378714cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:13.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75bad9d700 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x557378717c80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009610 tx=0x7f75b40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd675b0923a1813 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:13.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:13.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.913+0000 7f75a1ffb700 1 -- 172.21.15.62:0/3541426194 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b400e070 con 0x557378709480 2023-07-14T16:29:13.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 1 -- 172.21.15.62:0/3541426194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75b4007b90 con 0x557378709480 2023-07-14T16:29:13.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75a1ffb700 1 -- 172.21.15.62:0/3541426194 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f75b401d660 con 0x557378709480 2023-07-14T16:29:13.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75a1ffb700 1 -- 172.21.15.62:0/3541426194 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b401d820 con 0x557378709480 2023-07-14T16:29:13.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 10 client.19305 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:13.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 1 -- 172.21.15.62:0/3541426194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f75b0005ce0 con 0x557378709480 2023-07-14T16:29:13.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75a1ffb700 1 -- 172.21.15.62:0/3541426194 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75b4007b90 con 0x557378709480 2023-07-14T16:29:13.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75a1ffb700 1 -- 172.21.15.62:0/3541426194 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f75b4041070 con 0x557378709480 2023-07-14T16:29:13.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 10 client.19305 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:13.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 10 client.19305 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:29:13.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 20 client.19305 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:13.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557378771710 0x557378773af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:13.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 4 client.19305 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:29:13.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.914+0000 7f75d48bab80 1 -- 172.21.15.62:0/3541426194 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55737860c050 con 0x557378771710 2023-07-14T16:29:13.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.915+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557378771710 0x557378773af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:13.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75a1ffb700 10 client.19305 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:29:13.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75b9d9b700 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557378771710 0x557378773af0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b72fd7da6f6d95c server_cookie=53acf3d330c4d06b in_seq=0 out_seq=0 2023-07-14T16:29:13.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75a1ffb700 1 -- 172.21.15.62:0/3541426194 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55737860c050 con 0x557378771710 2023-07-14T16:29:13.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75a1ffb700 10 client.19305 handle_command_reply: tid=0 2023-07-14T16:29:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 client.19305 shutdown 2023-07-14T16:29:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 -- 172.21.15.62:0/3541426194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 msgr2=0x557378717c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x557378717c80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009610 tx=0x7f75b40099e0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 -- 172.21.15.62:0/3541426194 shutdown_connections 2023-07-14T16:29:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x557378771710 0x557378773af0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557378709480 0x557378717c80 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557378707ce0 0x557378714790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 --2- 172.21.15.62:0/3541426194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557378708630 0x557378714cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 -- 172.21.15.62:0/3541426194 >> 172.21.15.62:0/3541426194 conn(0x5573786f4da0 msgr2=0x5573787105d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75a1ffb700 0 client.19305 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:29:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 -- 172.21.15.62:0/3541426194 shutdown_connections 2023-07-14T16:29:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.916+0000 7f75d48bab80 1 -- 172.21.15.62:0/3541426194 wait complete. 2023-07-14T16:29:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75d48bab80 20 client.19305 trim_cache size 0 max 16384 2023-07-14T16:29:13.897 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:29:13.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 -- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f75a40a1150 msgr2=0x7f75a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f75a40a1150 0x7f75a40a3600 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75ac099f50 tx=0x7f75ac084f90 comp rx=0 tx=0).stop 2023-07-14T16:29:13.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 -- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75bc100560 msgr2=0x7f75bc1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75bc100560 0x7f75bc1979f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00e9a0 tx=0x7f75ac00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 -- 172.21.15.62:0/581637401 shutdown_connections 2023-07-14T16:29:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f75a40a1150 0x7f75a40a3600 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1057a0 0x7f75bc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75bc100560 0x7f75bc1979f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 --2- 172.21.15.62:0/581637401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75bc102e80 0x7f75bc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 -- 172.21.15.62:0/581637401 >> 172.21.15.62:0/581637401 conn(0x7f75bc0fa000 msgr2=0x7f75bc1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 -- 172.21.15.62:0/581637401 shutdown_connections 2023-07-14T16:29:13.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:13.917+0000 7f75c0fff700 1 -- 172.21.15.62:0/581637401 wait complete. 2023-07-14T16:29:13.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:29:14.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:13 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3251010891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:29:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.281+0000 7f44981ff700 1 -- 172.21.15.62:0/1056488744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44901079a0 msgr2=0x7f4490109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.281+0000 7f44981ff700 1 --2- 172.21.15.62:0/1056488744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44901079a0 0x7f4490109e20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f448800b0a0 tx=0x7f448800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.281+0000 7f44981ff700 1 -- 172.21.15.62:0/1056488744 shutdown_connections 2023-07-14T16:29:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.281+0000 7f44981ff700 1 --2- 172.21.15.62:0/1056488744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44901079a0 0x7f4490109e20 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.281+0000 7f44981ff700 1 --2- 172.21.15.62:0/1056488744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44900744f0 0x7f4490072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.281+0000 7f44981ff700 1 --2- 172.21.15.62:0/1056488744 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4490072f10 0x7f4490073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.281+0000 7f44981ff700 1 -- 172.21.15.62:0/1056488744 >> 172.21.15.62:0/1056488744 conn(0x7f44900fb130 msgr2=0x7f44900fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.281+0000 7f44981ff700 1 -- 172.21.15.62:0/1056488744 shutdown_connections 2023-07-14T16:29:14.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.282+0000 7f44981ff700 1 -- 172.21.15.62:0/1056488744 wait complete. 2023-07-14T16:29:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.283+0000 7f44981ff700 1 Processor -- start 2023-07-14T16:29:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.283+0000 7f44981ff700 1 -- start start 2023-07-14T16:29:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f44981ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4490072f10 0x7f4490197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f44981ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44900744f0 0x7f4490197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f44981ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44901079a0 0x7f449019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f44981ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019f5a0 con 0x7f4490072f10 2023-07-14T16:29:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f44981ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019f6e0 con 0x7f44901079a0 2023-07-14T16:29:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f44981ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019f870 con 0x7f44900744f0 2023-07-14T16:29:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f4495f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4490072f10 0x7f4490197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f4495f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4490072f10 0x7f4490197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52890/0 (socket says 172.21.15.62:52890) 2023-07-14T16:29:14.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f4495f9b700 1 -- 172.21.15.62:0/487632262 learned_addr learned my addr 172.21.15.62:0/487632262 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f449579a700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44900744f0 0x7f4490197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f449679c700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44901079a0 0x7f449019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f4495f9b700 1 -- 172.21.15.62:0/487632262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44900744f0 msgr2=0x7f4490197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f4495f9b700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44900744f0 0x7f4490197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f4495f9b700 1 -- 172.21.15.62:0/487632262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44901079a0 msgr2=0x7f449019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f4495f9b700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44901079a0 0x7f449019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f4495f9b700 1 -- 172.21.15.62:0/487632262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488009d00 con 0x7f4490072f10 2023-07-14T16:29:14.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f449579a700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44900744f0 0x7f4490197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:14.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.284+0000 7f449679c700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44901079a0 0x7f449019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:29:14.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.285+0000 7f4495f9b700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4490072f10 0x7f4490197a20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f448c00e9a0 tx=0x7f448c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.285+0000 7f4486ffd700 1 -- 172.21.15.62:0/487632262 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c009790 con 0x7f4490072f10 2023-07-14T16:29:14.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.285+0000 7f44981ff700 1 -- 172.21.15.62:0/487632262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449019fb50 con 0x7f4490072f10 2023-07-14T16:29:14.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.285+0000 7f4486ffd700 1 -- 172.21.15.62:0/487632262 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f448c00cbd0 con 0x7f4490072f10 2023-07-14T16:29:14.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.285+0000 7f4486ffd700 1 -- 172.21.15.62:0/487632262 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c018680 con 0x7f4490072f10 2023-07-14T16:29:14.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.285+0000 7f44981ff700 1 -- 172.21.15.62:0/487632262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44901a0060 con 0x7f4490072f10 2023-07-14T16:29:14.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.289+0000 7f4486ffd700 1 -- 172.21.15.62:0/487632262 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f448c00cd40 con 0x7f4490072f10 2023-07-14T16:29:14.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f44abb1bb80 1 Processor -- start 2023-07-14T16:29:14.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f44abb1bb80 1 -- start start 2023-07-14T16:29:14.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f4486ffd700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f447c0a1150 0x7f447c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 0x556988f7d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7de20 0x556988f7e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7ec70 0x556988f7f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988e82cb0 con 0x556988f7ec70 2023-07-14T16:29:14.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988ed5a40 con 0x556988f7d4d0 2023-07-14T16:29:14.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988eceed0 con 0x556988f7de20 2023-07-14T16:29:14.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.291+0000 7f4486ffd700 1 -- 172.21.15.62:0/487632262 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f448c014070 con 0x7f4490072f10 2023-07-14T16:29:14.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449679c700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f447c0a1150 0x7f447c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449579a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7de20 0x556988f7e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7ec70 0x556988f7f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449579a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7de20 0x556988f7e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:51110/0 (socket says 172.21.15.62:51110) 2023-07-14T16:29:14.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449579a700 1 -- 172.21.15.62:0/4207789243 learned_addr learned my addr 172.21.15.62:0/4207789243 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f4495f9b700 1 --2- 172.21.15.62:0/4207789243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 0x556988f7d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449579a700 1 -- 172.21.15.62:0/4207789243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 msgr2=0x556988f7d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449579a700 1 --2- 172.21.15.62:0/4207789243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 0x556988f7d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449579a700 1 -- 172.21.15.62:0/4207789243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7ec70 msgr2=0x556988f7f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449579a700 1 --2- 172.21.15.62:0/4207789243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7ec70 0x556988f7f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.292+0000 7f449579a700 1 -- 172.21.15.62:0/4207789243 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556988cfcde0 con 0x556988f7de20 2023-07-14T16:29:14.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.293+0000 7f449579a700 1 --2- 172.21.15.62:0/4207789243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7de20 0x556988f7e6e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4480009de0 tx=0x7f44800116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22a03f2402da3aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.293+0000 7f449679c700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f447c0a1150 0x7f447c0a3600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f448800b780 tx=0x7f4488009420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.293+0000 7f447bfff700 1 -- 172.21.15.62:0/4207789243 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480011e20 con 0x556988f7de20 2023-07-14T16:29:14.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.293+0000 7f447bfff700 1 -- 172.21.15.62:0/4207789243 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44800128e0 con 0x556988f7de20 2023-07-14T16:29:14.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.293+0000 7f447bfff700 1 -- 172.21.15.62:0/4207789243 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000f660 con 0x556988f7de20 2023-07-14T16:29:14.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 -- 172.21.15.62:0/4207789243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7de20 msgr2=0x556988f7e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/4207789243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7de20 0x556988f7e6e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4480009de0 tx=0x7f44800116c0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 -- 172.21.15.62:0/4207789243 shutdown_connections 2023-07-14T16:29:14.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/4207789243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7ec70 0x556988f7f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/4207789243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 0x556988f7d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/4207789243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7de20 0x556988f7e6e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 -- 172.21.15.62:0/4207789243 >> 172.21.15.62:0/4207789243 conn(0x556988f72da0 msgr2=0x556988f781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 -- 172.21.15.62:0/4207789243 shutdown_connections 2023-07-14T16:29:14.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.294+0000 7f44abb1bb80 1 -- 172.21.15.62:0/4207789243 wait complete. 2023-07-14T16:29:14.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.295+0000 7f44abb1bb80 1 -- start start 2023-07-14T16:29:14.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.295+0000 7f44abb1bb80 1 Processor -- start 2023-07-14T16:29:14.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f87140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f8e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988e6eac0 con 0x556988f7d4d0 2023-07-14T16:29:14.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988e6e7c0 con 0x556988f7de20 2023-07-14T16:29:14.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988d52920 con 0x556988f7ec70 2023-07-14T16:29:14.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f44abb1bb80 10 client.0 _ll_register_callbacks cb 0x556988e6c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:14.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f4495f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f87140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f4495f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f87140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52906/0 (socket says 172.21.15.62:52906) 2023-07-14T16:29:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f4495f9b700 1 -- 172.21.15.62:0/755334708 learned_addr learned my addr 172.21.15.62:0/755334708 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f449579a700 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f449679c700 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f8e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f4495f9b700 1 -- 172.21.15.62:0/755334708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 msgr2=0x556988f8e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f4495f9b700 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f8e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f4495f9b700 1 -- 172.21.15.62:0/755334708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 msgr2=0x556988f87680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f4495f9b700 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f87680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.296+0000 7f4495f9b700 1 -- 172.21.15.62:0/755334708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4480009a10 con 0x556988f7d4d0 2023-07-14T16:29:14.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f4495f9b700 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f87140 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f448c0ca750 tx=0x7f448c06a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c68fca54fc3e06 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f44797fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:14.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f44797fa700 1 -- 172.21.15.62:0/755334708 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c099750 con 0x556988f7d4d0 2023-07-14T16:29:14.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f44797fa700 1 -- 172.21.15.62:0/755334708 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f448c0a24b0 con 0x556988f7d4d0 2023-07-14T16:29:14.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f44abb1bb80 1 -- 172.21.15.62:0/755334708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556988e284f0 con 0x556988f7d4d0 2023-07-14T16:29:14.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f44797fa700 1 -- 172.21.15.62:0/755334708 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448c0a2020 con 0x556988f7d4d0 2023-07-14T16:29:14.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f44abb1bb80 10 client.19317 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f44abb1bb80 1 -- 172.21.15.62:0/755334708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556988ce5c60 con 0x556988f7d4d0 2023-07-14T16:29:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.297+0000 7f44797fa700 1 -- 172.21.15.62:0/755334708 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f448c0b6020 con 0x556988f7d4d0 2023-07-14T16:29:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.298+0000 7f44797fa700 1 -- 172.21.15.62:0/755334708 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f448c095e50 con 0x556988f7d4d0 2023-07-14T16:29:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.298+0000 7f44abb1bb80 10 client.19317 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.298+0000 7f44abb1bb80 10 client.19317 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:29:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.298+0000 7f44abb1bb80 20 client.19317 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.298+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556988f92ed0 0x556988f952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.298+0000 7f44abb1bb80 4 client.19317 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.298+0000 7f44abb1bb80 1 -- 172.21.15.62:0/755334708 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556988ec7eb0 con 0x556988f92ed0 2023-07-14T16:29:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.298+0000 7f449579a700 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556988f92ed0 0x556988f952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.299+0000 7f449579a700 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556988f92ed0 0x556988f952e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0054e63d8a93ac6 server_cookie=b151b9d193b8ae22 in_seq=0 out_seq=0 2023-07-14T16:29:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.299+0000 7f44797fa700 10 client.19317 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:29:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.299+0000 7f44797fa700 1 -- 172.21.15.62:0/755334708 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556988ec7eb0 con 0x556988f92ed0 2023-07-14T16:29:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.299+0000 7f44797fa700 10 client.19317 handle_command_reply: tid=0 2023-07-14T16:29:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 client.19317 shutdown 2023-07-14T16:29:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 -- 172.21.15.62:0/755334708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 msgr2=0x556988f87140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f87140 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f448c0ca750 tx=0x7f448c06a480 comp rx=0 tx=0).stop 2023-07-14T16:29:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 -- 172.21.15.62:0/755334708 shutdown_connections 2023-07-14T16:29:14.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556988f92ed0 0x556988f952e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f8e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f87140 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/755334708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f87680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 -- 172.21.15.62:0/755334708 >> 172.21.15.62:0/755334708 conn(0x556988f72da0 msgr2=0x556988f778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44797fa700 0 client.19317 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:29:14.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 -- 172.21.15.62:0/755334708 shutdown_connections 2023-07-14T16:29:14.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 1 -- 172.21.15.62:0/755334708 wait complete. 2023-07-14T16:29:14.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.300+0000 7f44abb1bb80 20 client.19317 trim_cache size 0 max 16384 2023-07-14T16:29:14.294 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:13 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3251010891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:29:14.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.317+0000 7f44abb1bb80 1 Processor -- start 2023-07-14T16:29:14.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.317+0000 7f44abb1bb80 1 -- start start 2023-07-14T16:29:14.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.317+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 0x556988f8efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.317+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7de20 0x556988f8e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.317+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f81b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.317+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f85890 con 0x556988f7de20 2023-07-14T16:29:14.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.317+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f85b80 con 0x556988f7d4d0 2023-07-14T16:29:14.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.317+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f79c60 con 0x556988f7ec70 2023-07-14T16:29:14.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7de20 0x556988f8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7de20 0x556988f8e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52908/0 (socket says 172.21.15.62:52908) 2023-07-14T16:29:14.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449579a700 1 -- 172.21.15.62:0/2650809377 learned_addr learned my addr 172.21.15.62:0/2650809377 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f4495f9b700 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 0x556988f8efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449679c700 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f81b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449579a700 1 -- 172.21.15.62:0/2650809377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 msgr2=0x556988f81b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449579a700 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f81b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449579a700 1 -- 172.21.15.62:0/2650809377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 msgr2=0x556988f8efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449579a700 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 0x556988f8efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.318+0000 7f449579a700 1 -- 172.21.15.62:0/2650809377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556988d23250 con 0x556988f7de20 2023-07-14T16:29:14.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.319+0000 7f449579a700 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7de20 0x556988f8e680 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4480009de0 tx=0x7f44800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f385b63e50ec757f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.319+0000 7f4484ff9700 1 -- 172.21.15.62:0/2650809377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448002e070 con 0x556988f7de20 2023-07-14T16:29:14.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.319+0000 7f4484ff9700 1 -- 172.21.15.62:0/2650809377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4480011d40 con 0x556988f7de20 2023-07-14T16:29:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.319+0000 7f4484ff9700 1 -- 172.21.15.62:0/2650809377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480005580 con 0x556988f7de20 2023-07-14T16:29:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2650809377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7de20 msgr2=0x556988f8e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7de20 0x556988f8e680 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4480009de0 tx=0x7f44800095c0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2650809377 shutdown_connections 2023-07-14T16:29:14.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f81b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7d4d0 0x556988f8efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2650809377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7de20 0x556988f8e680 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2650809377 >> 172.21.15.62:0/2650809377 conn(0x556988f72da0 msgr2=0x556988f85e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2650809377 shutdown_connections 2023-07-14T16:29:14.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.320+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2650809377 wait complete. 2023-07-14T16:29:14.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.321+0000 7f44abb1bb80 1 -- start start 2023-07-14T16:29:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.321+0000 7f44abb1bb80 1 Processor -- start 2023-07-14T16:29:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.321+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f8e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.321+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f8ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.321+0000 7f44abb1bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f8b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.321+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f8b940 con 0x556988f7d4d0 2023-07-14T16:29:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f8e8a0 con 0x556988f7de20 2023-07-14T16:29:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f44abb1bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f8ea30 con 0x556988f7ec70 2023-07-14T16:29:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f44abb1bb80 10 client.0 _ll_register_callbacks cb 0x7f448c0999c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f4495f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f8e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f4495f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f8e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52922/0 (socket says 172.21.15.62:52922) 2023-07-14T16:29:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f4495f9b700 1 -- 172.21.15.62:0/2347321666 learned_addr learned my addr 172.21.15.62:0/2347321666 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f449679c700 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f8b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f449579a700 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f8ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f4495f9b700 1 -- 172.21.15.62:0/2347321666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 msgr2=0x556988f8b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f4495f9b700 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f8b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f4495f9b700 1 -- 172.21.15.62:0/2347321666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 msgr2=0x556988f8ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f4495f9b700 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f8ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f4495f9b700 1 -- 172.21.15.62:0/2347321666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556988cfcde0 con 0x556988f7d4d0 2023-07-14T16:29:14.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.322+0000 7f449579a700 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f8ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f4495f9b700 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f8e220 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f448c096ad0 tx=0x7f448c0a22d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25ca95f59784cc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f44797fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f44797fa700 1 -- 172.21.15.62:0/2347321666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c095e50 con 0x556988f7d4d0 2023-07-14T16:29:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f44797fa700 1 -- 172.21.15.62:0/2347321666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f448c0b7080 con 0x556988f7d4d0 2023-07-14T16:29:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2347321666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556988e8a050 con 0x556988f7d4d0 2023-07-14T16:29:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f44797fa700 1 -- 172.21.15.62:0/2347321666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448c095fb0 con 0x556988f7d4d0 2023-07-14T16:29:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f44abb1bb80 10 client.19323 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2347321666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556988cfb2b0 con 0x556988f7d4d0 2023-07-14T16:29:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.323+0000 7f44797fa700 1 -- 172.21.15.62:0/2347321666 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f448c0ca020 con 0x556988f7d4d0 2023-07-14T16:29:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.324+0000 7f44797fa700 1 -- 172.21.15.62:0/2347321666 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f448c0ca420 con 0x556988f7d4d0 2023-07-14T16:29:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.324+0000 7f44abb1bb80 10 client.19323 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.324+0000 7f44abb1bb80 10 client.19323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:29:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.324+0000 7f44abb1bb80 20 client.19323 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.324+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556988fe8360 0x556988fea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.324+0000 7f44abb1bb80 4 client.19323 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:29:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.324+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2347321666 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556988d1df70 con 0x556988fe8360 2023-07-14T16:29:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.324+0000 7f449579a700 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556988fe8360 0x556988fea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44797fa700 10 client.19323 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:29:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f449579a700 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556988fe8360 0x556988fea770 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56aea5b46d49683 server_cookie=1dadc72481239e25 in_seq=0 out_seq=0 2023-07-14T16:29:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44797fa700 1 -- 172.21.15.62:0/2347321666 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556988d1df70 con 0x556988fe8360 2023-07-14T16:29:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44797fa700 10 client.19323 handle_command_reply: tid=0 2023-07-14T16:29:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 client.19323 shutdown 2023-07-14T16:29:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2347321666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 msgr2=0x556988f8e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f8e220 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f448c096ad0 tx=0x7f448c0a22d0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2347321666 shutdown_connections 2023-07-14T16:29:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x556988fe8360 0x556988fea770 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556988f7ec70 0x556988f8b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556988f7d4d0 0x556988f8e220 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 --2- 172.21.15.62:0/2347321666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556988f7de20 0x556988f8ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2347321666 >> 172.21.15.62:0/2347321666 conn(0x556988f72da0 msgr2=0x556988f85e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44797fa700 0 client.19323 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:29:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.325+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2347321666 shutdown_connections 2023-07-14T16:29:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44abb1bb80 1 -- 172.21.15.62:0/2347321666 wait complete. 2023-07-14T16:29:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44abb1bb80 20 client.19323 trim_cache size 0 max 16384 2023-07-14T16:29:14.321 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:29:14.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 -- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f447c0a1150 msgr2=0x7f447c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f447c0a1150 0x7f447c0a3600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f448800b780 tx=0x7f4488009420 comp rx=0 tx=0).stop 2023-07-14T16:29:14.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 -- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4490072f10 msgr2=0x7f4490197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4490072f10 0x7f4490197a20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f448c00e9a0 tx=0x7f448c00ed60 comp rx=0 tx=0).stop 2023-07-14T16:29:14.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 -- 172.21.15.62:0/487632262 shutdown_connections 2023-07-14T16:29:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f447c0a1150 0x7f447c0a3600 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44901079a0 0x7f449019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4490072f10 0x7f4490197a20 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 --2- 172.21.15.62:0/487632262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44900744f0 0x7f4490197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.326+0000 7f44981ff700 1 -- 172.21.15.62:0/487632262 >> 172.21.15.62:0/487632262 conn(0x7f44900fb130 msgr2=0x7f44900fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.327+0000 7f44981ff700 1 -- 172.21.15.62:0/487632262 shutdown_connections 2023-07-14T16:29:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.327+0000 7f44981ff700 1 -- 172.21.15.62:0/487632262 wait complete. 2023-07-14T16:29:14.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:29:14.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.589+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3785936312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8102e80 msgr2=0x7fb9f8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.589+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3785936312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8102e80 0x7fb9f8105260 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009990 tx=0x7fb9ec009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.589+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3785936312 shutdown_connections 2023-07-14T16:29:14.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.589+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3785936312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9f81057a0 0x7fb9f8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.589+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3785936312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9f8100560 0x7fb9f8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.589+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3785936312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8102e80 0x7fb9f8105260 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.589+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3785936312 >> 172.21.15.62:0/3785936312 conn(0x7fb9f80fa000 msgr2=0x7fb9f80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.590+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3785936312 shutdown_connections 2023-07-14T16:29:14.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.590+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3785936312 wait complete. 2023-07-14T16:29:14.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9fe9ff700 1 Processor -- start 2023-07-14T16:29:14.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9fe9ff700 1 -- start start 2023-07-14T16:29:14.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9fe9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8100560 0x7fb9f8195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9fe9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9f8102e80 0x7fb9f8195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9fe9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9f81057a0 0x7fb9f819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9fe9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f819d2c0 con 0x7fb9f8100560 2023-07-14T16:29:14.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9fe9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f819d400 con 0x7fb9f8102e80 2023-07-14T16:29:14.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9fe9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f819d590 con 0x7fb9f81057a0 2023-07-14T16:29:14.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8100560 0x7fb9f8195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8100560 0x7fb9f8195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52938/0 (socket says 172.21.15.62:52938) 2023-07-14T16:29:14.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.591+0000 7fb9f7fff700 1 -- 172.21.15.62:0/3375457483 learned_addr learned my addr 172.21.15.62:0/3375457483 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9f8102e80 0x7fb9f8195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9f81057a0 0x7fb9f819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9f7fff700 1 -- 172.21.15.62:0/3375457483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9f81057a0 msgr2=0x7fb9f819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9f7fff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9f81057a0 0x7fb9f819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9f7fff700 1 -- 172.21.15.62:0/3375457483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9f8102e80 msgr2=0x7fb9f8195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9f7fff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9f8102e80 0x7fb9f8195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9f7fff700 1 -- 172.21.15.62:0/3375457483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e8009520 con 0x7fb9f8100560 2023-07-14T16:29:14.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9f7fff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8100560 0x7fb9f8195740 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800eb30 tx=0x7fb9e800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.592+0000 7fb9f57fa700 1 -- 172.21.15.62:0/3375457483 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8009880 con 0x7fb9f8100560 2023-07-14T16:29:14.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.593+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3375457483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec0095f0 con 0x7fb9f8100560 2023-07-14T16:29:14.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.593+0000 7fb9f57fa700 1 -- 172.21.15.62:0/3375457483 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9e80044b0 con 0x7fb9f8100560 2023-07-14T16:29:14.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.593+0000 7fb9f57fa700 1 -- 172.21.15.62:0/3375457483 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8003df0 con 0x7fb9f8100560 2023-07-14T16:29:14.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.593+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3375457483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f819dbf0 con 0x7fb9f8100560 2023-07-14T16:29:14.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.597+0000 7fb9f57fa700 1 -- 172.21.15.62:0/3375457483 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb9e8003f50 con 0x7fb9f8100560 2023-07-14T16:29:14.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.598+0000 7fba1241bb80 1 Processor -- start 2023-07-14T16:29:14.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.598+0000 7fba1241bb80 1 -- start start 2023-07-14T16:29:14.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb88e0 0x560218cb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fb9f57fa700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e00a1150 0x7fb9e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb9230 0x560218cb9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fba1241bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218bb9cb0 con 0x560218cb88e0 2023-07-14T16:29:14.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fba1241bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218c0ca40 con 0x560218cb9230 2023-07-14T16:29:14.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fba1241bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218c05ed0 con 0x560218cba080 2023-07-14T16:29:14.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fb9f57fa700 1 -- 172.21.15.62:0/3375457483 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9e8014070 con 0x7fb9f8100560 2023-07-14T16:29:14.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fb9fcf9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb88e0 0x560218cb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fb9fcf9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb88e0 0x560218cb8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52952/0 (socket says 172.21.15.62:52952) 2023-07-14T16:29:14.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fb9fcf9c700 1 -- 172.21.15.62:0/2112740583 learned_addr learned my addr 172.21.15.62:0/2112740583 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fb9f7fff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e00a1150 0x7fb9e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.599+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb9230 0x560218cb9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.600+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.600+0000 7fb9fcf9c700 1 -- 172.21.15.62:0/2112740583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 msgr2=0x560218cba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.600+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.600+0000 7fb9fcf9c700 1 -- 172.21.15.62:0/2112740583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb9230 msgr2=0x560218cb9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.600+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb9230 0x560218cb9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.600+0000 7fb9fcf9c700 1 -- 172.21.15.62:0/2112740583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560218a33de0 con 0x560218cb88e0 2023-07-14T16:29:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.600+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb88e0 0x560218cb8cf0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00078b0 tx=0x7fb9f0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f36d1901351e8b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.600+0000 7fb9f7fff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e00a1150 0x7fb9e00a3600 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80965f0 tx=0x7fb9e8003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.601+0000 7fb9de7fc700 1 -- 172.21.15.62:0/2112740583 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000e070 con 0x560218cb88e0 2023-07-14T16:29:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.601+0000 7fb9de7fc700 1 -- 172.21.15.62:0/2112740583 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9f000f040 con 0x560218cb88e0 2023-07-14T16:29:14.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.601+0000 7fb9de7fc700 1 -- 172.21.15.62:0/2112740583 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f00176c0 con 0x560218cb88e0 2023-07-14T16:29:14.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 -- 172.21.15.62:0/2112740583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb88e0 msgr2=0x560218cb8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb88e0 0x560218cb8cf0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00078b0 tx=0x7fb9f0009920 comp rx=0 tx=0).stop 2023-07-14T16:29:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 -- 172.21.15.62:0/2112740583 shutdown_connections 2023-07-14T16:29:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb88e0 0x560218cb8cf0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 --2- 172.21.15.62:0/2112740583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb9230 0x560218cb9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 -- 172.21.15.62:0/2112740583 >> 172.21.15.62:0/2112740583 conn(0x560218ca9da0 msgr2=0x560218caf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 -- 172.21.15.62:0/2112740583 shutdown_connections 2023-07-14T16:29:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.602+0000 7fba1241bb80 1 -- 172.21.15.62:0/2112740583 wait complete. 2023-07-14T16:29:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.603+0000 7fba1241bb80 1 -- start start 2023-07-14T16:29:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.603+0000 7fba1241bb80 1 Processor -- start 2023-07-14T16:29:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fba1241bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218ba5ac0 con 0x560218cb9230 2023-07-14T16:29:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fba1241bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218ba57c0 con 0x560218cb88e0 2023-07-14T16:29:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fba1241bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218a89920 con 0x560218cba080 2023-07-14T16:29:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fba1241bb80 10 client.0 _ll_register_callbacks cb 0x560218ba3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fb9f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fb9f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52960/0 (socket says 172.21.15.62:52960) 2023-07-14T16:29:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fb9f77fe700 1 -- 172.21.15.62:0/759699893 learned_addr learned my addr 172.21.15.62:0/759699893 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.604+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.605+0000 7fb9f77fe700 1 -- 172.21.15.62:0/759699893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 msgr2=0x560218cc9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.605+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.605+0000 7fb9f77fe700 1 -- 172.21.15.62:0/759699893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 msgr2=0x560218cc2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.605+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.605+0000 7fb9f77fe700 1 -- 172.21.15.62:0/759699893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f0007500 con 0x560218cb9230 2023-07-14T16:29:14.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.605+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc2a80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0059d0 tx=0x7fb9ec0053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc33eb1af85f60e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.605+0000 7fb9d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.606+0000 7fb9d77fe700 1 -- 172.21.15.62:0/759699893 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec01d070 con 0x560218cb9230 2023-07-14T16:29:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.606+0000 7fba1241bb80 1 -- 172.21.15.62:0/759699893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560218b5f4f0 con 0x560218cb9230 2023-07-14T16:29:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.606+0000 7fb9d77fe700 1 -- 172.21.15.62:0/759699893 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9ec003bd0 con 0x560218cb9230 2023-07-14T16:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.606+0000 7fb9d77fe700 1 -- 172.21.15.62:0/759699893 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9ec00b710 con 0x560218cb9230 2023-07-14T16:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.606+0000 7fba1241bb80 10 client.19335 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.606+0000 7fba1241bb80 1 -- 172.21.15.62:0/759699893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560218a1cc60 con 0x560218cb9230 2023-07-14T16:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.606+0000 7fb9d77fe700 1 -- 172.21.15.62:0/759699893 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560218b5f4f0 con 0x560218cb9230 2023-07-14T16:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.606+0000 7fb9d77fe700 1 -- 172.21.15.62:0/759699893 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb9ec004830 con 0x560218cb9230 2023-07-14T16:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.607+0000 7fba1241bb80 10 client.19335 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.607+0000 7fba1241bb80 10 client.19335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.607+0000 7fba1241bb80 20 client.19335 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.607+0000 7fba1241bb80 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560218cce240 0x560218cd0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.607+0000 7fba1241bb80 4 client.19335 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.607+0000 7fba1241bb80 1 -- 172.21.15.62:0/759699893 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560218bfeeb0 con 0x560218cce240 2023-07-14T16:29:14.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.607+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560218cce240 0x560218cd0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.608+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560218cce240 0x560218cd0650 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaca6da374a11824 server_cookie=f610f00d4bc01af1 in_seq=0 out_seq=0 2023-07-14T16:29:14.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.608+0000 7fb9d77fe700 10 client.19335 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:14.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fb9d77fe700 1 -- 172.21.15.62:0/759699893 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560218bfeeb0 con 0x560218cce240 2023-07-14T16:29:14.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fb9d77fe700 10 client.19335 handle_command_reply: tid=0 2023-07-14T16:29:14.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 client.19335 shutdown 2023-07-14T16:29:14.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 -- 172.21.15.62:0/759699893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 msgr2=0x560218cc2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc2a80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0059d0 tx=0x7fb9ec0053b0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 -- 172.21.15.62:0/759699893 shutdown_connections 2023-07-14T16:29:14.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560218cce240 0x560218cd0650 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 --2- 172.21.15.62:0/759699893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc2a80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 -- 172.21.15.62:0/759699893 >> 172.21.15.62:0/759699893 conn(0x560218ca9da0 msgr2=0x560218cae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fb9d77fe700 0 client.19335 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 -- 172.21.15.62:0/759699893 shutdown_connections 2023-07-14T16:29:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.609+0000 7fba1241bb80 1 -- 172.21.15.62:0/759699893 wait complete. 2023-07-14T16:29:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.610+0000 7fba1241bb80 20 client.19335 trim_cache size 0 max 16384 2023-07-14T16:29:14.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fba1241bb80 1 Processor -- start 2023-07-14T16:29:14.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fba1241bb80 1 -- start start 2023-07-14T16:29:14.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cd0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fba1241bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218cb9800 con 0x560218cb9230 2023-07-14T16:29:14.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fba1241bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218cc01e0 con 0x560218cb88e0 2023-07-14T16:29:14.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fba1241bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218cc3690 con 0x560218cba080 2023-07-14T16:29:14.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.632+0000 7fb9f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9f77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52974/0 (socket says 172.21.15.62:52974) 2023-07-14T16:29:14.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9f77fe700 1 -- 172.21.15.62:0/4102953541 learned_addr learned my addr 172.21.15.62:0/4102953541 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cd0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9f77fe700 1 -- 172.21.15.62:0/4102953541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 msgr2=0x560218cc0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9f77fe700 1 -- 172.21.15.62:0/4102953541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 msgr2=0x560218cd0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cd0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9f77fe700 1 -- 172.21.15.62:0/4102953541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560218a5a250 con 0x560218cb9230 2023-07-14T16:29:14.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:29:14.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc9a80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec005930 tx=0x7fb9ec009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3552d99ad0952a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.633+0000 7fb9deffd700 1 -- 172.21.15.62:0/4102953541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec01d070 con 0x560218cb9230 2023-07-14T16:29:14.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fb9deffd700 1 -- 172.21.15.62:0/4102953541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9ec01e040 con 0x560218cb9230 2023-07-14T16:29:14.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fb9deffd700 1 -- 172.21.15.62:0/4102953541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00b970 con 0x560218cb9230 2023-07-14T16:29:14.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fba1241bb80 1 -- 172.21.15.62:0/4102953541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 msgr2=0x560218cc9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc9a80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec005930 tx=0x7fb9ec009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fba1241bb80 1 -- 172.21.15.62:0/4102953541 shutdown_connections 2023-07-14T16:29:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cba080 0x560218cc0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cd0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4102953541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cb9230 0x560218cc9a80 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fba1241bb80 1 -- 172.21.15.62:0/4102953541 >> 172.21.15.62:0/4102953541 conn(0x560218ca9da0 msgr2=0x560218cc1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.634+0000 7fba1241bb80 1 -- 172.21.15.62:0/4102953541 shutdown_connections 2023-07-14T16:29:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.635+0000 7fba1241bb80 1 -- 172.21.15.62:0/4102953541 wait complete. 2023-07-14T16:29:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.635+0000 7fba1241bb80 1 -- start start 2023-07-14T16:29:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.635+0000 7fba1241bb80 1 Processor -- start 2023-07-14T16:29:14.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.635+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cb9230 0x560218cc8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fba1241bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cba080 0x560218cc86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fba1241bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218cc8db0 con 0x560218cba080 2023-07-14T16:29:14.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fba1241bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218cc8f20 con 0x560218cb88e0 2023-07-14T16:29:14.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fba1241bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218cc90e0 con 0x560218cb9230 2023-07-14T16:29:14.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fba1241bb80 10 client.0 _ll_register_callbacks cb 0x7fb9ec005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:14.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36468/0 (socket says 172.21.15.62:36468) 2023-07-14T16:29:14.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 -- 172.21.15.62:0/4266832967 learned_addr learned my addr 172.21.15.62:0/4266832967 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:14.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cba080 0x560218cc86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cb9230 0x560218cc8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 -- 172.21.15.62:0/4266832967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cb9230 msgr2=0x560218cc8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cb9230 0x560218cc8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 -- 172.21.15.62:0/4266832967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cba080 msgr2=0x560218cc86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cba080 0x560218cc86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.636+0000 7fb9fcf9c700 1 -- 172.21.15.62:0/4266832967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560218a1cc60 con 0x560218cb88e0 2023-07-14T16:29:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.637+0000 7fb9fcf9c700 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc7c20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0008fe0 tx=0x7fb9f0029040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1f705a57978768 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.637+0000 7fb9d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:29:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.637+0000 7fb9d77fe700 1 -- 172.21.15.62:0/4266832967 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000e040 con 0x560218cb88e0 2023-07-14T16:29:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.637+0000 7fba1241bb80 1 -- 172.21.15.62:0/4266832967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9f000f040 con 0x560218cb88e0 2023-07-14T16:29:14.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.637+0000 7fb9d77fe700 1 -- 172.21.15.62:0/4266832967 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9f002d560 con 0x560218cb88e0 2023-07-14T16:29:14.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.637+0000 7fb9d77fe700 1 -- 172.21.15.62:0/4266832967 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9f0003a00 con 0x560218cb88e0 2023-07-14T16:29:14.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.637+0000 7fba1241bb80 10 client.28910 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:14.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.637+0000 7fba1241bb80 1 -- 172.21.15.62:0/4266832967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560218a322b0 con 0x560218cb88e0 2023-07-14T16:29:14.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fb9d77fe700 1 -- 172.21.15.62:0/4266832967 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9f000f040 con 0x560218cb88e0 2023-07-14T16:29:14.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fb9d77fe700 1 -- 172.21.15.62:0/4266832967 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb9f0019630 con 0x560218cb88e0 2023-07-14T16:29:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fba1241bb80 10 client.28910 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fba1241bb80 10 client.28910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fba1241bb80 20 client.28910 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560218d22bc0 0x560218d24fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fba1241bb80 4 client.28910 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:29:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fba1241bb80 1 -- 172.21.15.62:0/4266832967 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560218bc1050 con 0x560218d22bc0 2023-07-14T16:29:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.638+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560218d22bc0 0x560218d24fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.639+0000 7fb9f77fe700 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560218d22bc0 0x560218d24fd0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1dfeb3100185cc server_cookie=92f3116e5c30f7b8 in_seq=0 out_seq=0 2023-07-14T16:29:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.639+0000 7fb9d77fe700 10 client.28910 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.639+0000 7fb9d77fe700 1 -- 172.21.15.62:0/4266832967 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560218bc1050 con 0x560218d22bc0 2023-07-14T16:29:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.639+0000 7fb9d77fe700 10 client.28910 handle_command_reply: tid=0 2023-07-14T16:29:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.639+0000 7fba1241bb80 1 client.28910 shutdown 2023-07-14T16:29:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.639+0000 7fba1241bb80 1 -- 172.21.15.62:0/4266832967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 msgr2=0x560218cc7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.639+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc7c20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0008fe0 tx=0x7fb9f0029040 comp rx=0 tx=0).stop 2023-07-14T16:29:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 1 -- 172.21.15.62:0/4266832967 shutdown_connections 2023-07-14T16:29:14.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560218cba080 0x560218cc86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560218d22bc0 0x560218d24fd0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560218cb88e0 0x560218cc7c20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fb9d77fe700 0 client.28910 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:14.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 1 --2- 172.21.15.62:0/4266832967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560218cb9230 0x560218cc8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 1 -- 172.21.15.62:0/4266832967 >> 172.21.15.62:0/4266832967 conn(0x560218ca9da0 msgr2=0x560218cd0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 1 -- 172.21.15.62:0/4266832967 shutdown_connections 2023-07-14T16:29:14.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 1 -- 172.21.15.62:0/4266832967 wait complete. 2023-07-14T16:29:14.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.640+0000 7fba1241bb80 20 client.28910 trim_cache size 0 max 16384 2023-07-14T16:29:14.636 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:29:14.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e00a1150 msgr2=0x7fb9e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e00a1150 0x7fb9e00a3600 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80965f0 tx=0x7fb9e8003680 comp rx=0 tx=0).stop 2023-07-14T16:29:14.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8100560 msgr2=0x7fb9f8195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:14.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8100560 0x7fb9f8195740 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800eb30 tx=0x7fb9e800ee40 comp rx=0 tx=0).stop 2023-07-14T16:29:14.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3375457483 shutdown_connections 2023-07-14T16:29:14.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9f81057a0 0x7fb9f819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9f8100560 0x7fb9f8195740 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9e00a1150 0x7fb9e00a3600 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 --2- 172.21.15.62:0/3375457483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9f8102e80 0x7fb9f8195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:14.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3375457483 >> 172.21.15.62:0/3375457483 conn(0x7fb9f80fa000 msgr2=0x7fb9f81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:14.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3375457483 shutdown_connections 2023-07-14T16:29:14.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:14.641+0000 7fb9fe9ff700 1 -- 172.21.15.62:0/3375457483 wait complete. 2023-07-14T16:29:15.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:14 smithi062 ceph-mon[95543]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:29:15.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:14 smithi081 ceph-mon[127795]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:29:15.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:14 smithi164 ceph-mon[97427]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:29:15.643 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:29:15.643 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:29:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 -- 172.21.15.62:0/1459457832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c80ffae0 msgr2=0x7f27c80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/1459457832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c80ffae0 0x7f27c80ffef0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009990 tx=0x7f27c4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 -- 172.21.15.62:0/1459457832 shutdown_connections 2023-07-14T16:29:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/1459457832 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c8101660 0x7f27c8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/1459457832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c80ffae0 0x7f27c80ffef0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/1459457832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27c8100cd0 0x7f27c8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 -- 172.21.15.62:0/1459457832 >> 172.21.15.62:0/1459457832 conn(0x7f27c80fb0e0 msgr2=0x7f27c80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:15.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 -- 172.21.15.62:0/1459457832 shutdown_connections 2023-07-14T16:29:15.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.944+0000 7f27cf5ff700 1 -- 172.21.15.62:0/1459457832 wait complete. 2023-07-14T16:29:15.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.945+0000 7f27cf5ff700 1 Processor -- start 2023-07-14T16:29:15.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.945+0000 7f27cf5ff700 1 -- start start 2023-07-14T16:29:15.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cf5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27c80ffae0 0x7f27c80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cf5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c8100cd0 0x7f27c8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cf5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8101660 0x7f27c8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cf5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80735e0 con 0x7f27c8101660 2023-07-14T16:29:15.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cf5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8073750 con 0x7f27c8100cd0 2023-07-14T16:29:15.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cf5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819f820 con 0x7f27c80ffae0 2023-07-14T16:29:15.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cdb9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8101660 0x7f27c8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cdb9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8101660 0x7f27c8072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53000/0 (socket says 172.21.15.62:53000) 2023-07-14T16:29:15.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cdb9c700 1 -- 172.21.15.62:0/4009811603 learned_addr learned my addr 172.21.15.62:0/4009811603 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:15.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c8100cd0 0x7f27c8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.946+0000 7f27cd39b700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27c80ffae0 0x7f27c80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cdb9c700 1 -- 172.21.15.62:0/4009811603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27c80ffae0 msgr2=0x7f27c80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cdb9c700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27c80ffae0 0x7f27c80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cdb9c700 1 -- 172.21.15.62:0/4009811603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c8100cd0 msgr2=0x7f27c8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cdb9c700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c8100cd0 0x7f27c8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cdb9c700 1 -- 172.21.15.62:0/4009811603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40095f0 con 0x7f27c8101660 2023-07-14T16:29:15.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cd39b700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27c80ffae0 0x7f27c80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:15.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c8100cd0 0x7f27c8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:15.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cdb9c700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8101660 0x7f27c8072e40 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f27c0007ee0 tx=0x7f27c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:15.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27be7fc700 1 -- 172.21.15.62:0/4009811603 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000ddd0 con 0x7f27c8101660 2023-07-14T16:29:15.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27be7fc700 1 -- 172.21.15.62:0/4009811603 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c000f040 con 0x7f27c8101660 2023-07-14T16:29:15.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cf5ff700 1 -- 172.21.15.62:0/4009811603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c819fb00 con 0x7f27c8101660 2023-07-14T16:29:15.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27be7fc700 1 -- 172.21.15.62:0/4009811603 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00047d0 con 0x7f27c8101660 2023-07-14T16:29:15.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.947+0000 7f27cf5ff700 1 -- 172.21.15.62:0/4009811603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c819ffc0 con 0x7f27c8101660 2023-07-14T16:29:15.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.951+0000 7f27be7fc700 1 -- 172.21.15.62:0/4009811603 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f27c0003a00 con 0x7f27c8101660 2023-07-14T16:29:15.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.953+0000 7f27e2ec4b80 1 Processor -- start 2023-07-14T16:29:15.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.953+0000 7f27e2ec4b80 1 -- start start 2023-07-14T16:29:15.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.953+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4d550 0x556a4ca4d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.953+0000 7f27be7fc700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f27b40a1150 0x7f27b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4ecf0 0x556a4ca4f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4c952d30 con 0x556a4ca4dea0 2023-07-14T16:29:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4c9a5ac0 con 0x556a4ca4d550 2023-07-14T16:29:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4c99ef50 con 0x556a4ca4ecf0 2023-07-14T16:29:15.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27be7fc700 1 -- 172.21.15.62:0/4009811603 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f27c00cc210 con 0x7f27c8101660 2023-07-14T16:29:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f27b40a1150 0x7f27b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27ccb9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27cd39b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4d550 0x556a4ca4d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27ccb9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca4e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53008/0 (socket says 172.21.15.62:53008) 2023-07-14T16:29:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27ccb9a700 1 -- 172.21.15.62:0/2289919251 learned_addr learned my addr 172.21.15.62:0/2289919251 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.954+0000 7f27cdb9c700 1 --2- 172.21.15.62:0/2289919251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4ecf0 0x556a4ca4f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.955+0000 7f27cd39b700 1 -- 172.21.15.62:0/2289919251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4ecf0 msgr2=0x556a4ca4f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:15.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.955+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2289919251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4ecf0 0x556a4ca4f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.955+0000 7f27cd39b700 1 -- 172.21.15.62:0/2289919251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 msgr2=0x556a4ca4e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.955+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2289919251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca4e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.955+0000 7f27cd39b700 1 -- 172.21.15.62:0/2289919251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a4c7cce60 con 0x556a4ca4d550 2023-07-14T16:29:15.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.955+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f27b40a1150 0x7f27b40a3600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f27b800afd0 tx=0x7f27b8011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:15.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.955+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2289919251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4d550 0x556a4ca4d960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f27c4000c00 tx=0x7f27c4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4024c2315a007d8e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:15.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.955+0000 7f27b37fe700 1 -- 172.21.15.62:0/2289919251 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c400b970 con 0x556a4ca4d550 2023-07-14T16:29:15.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27b37fe700 1 -- 172.21.15.62:0/2289919251 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c401e040 con 0x556a4ca4d550 2023-07-14T16:29:15.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27b37fe700 1 -- 172.21.15.62:0/2289919251 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4004760 con 0x556a4ca4d550 2023-07-14T16:29:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2289919251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4d550 msgr2=0x556a4ca4d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2289919251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4d550 0x556a4ca4d960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f27c4000c00 tx=0x7f27c4005c00 comp rx=0 tx=0).stop 2023-07-14T16:29:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2289919251 shutdown_connections 2023-07-14T16:29:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2289919251 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4ecf0 0x556a4ca4f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2289919251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4d550 0x556a4ca4d960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2289919251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca4e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2289919251 >> 172.21.15.62:0/2289919251 conn(0x556a4ca42e20 msgr2=0x556a4ca48250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.956+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2289919251 shutdown_connections 2023-07-14T16:29:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.957+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2289919251 wait complete. 2023-07-14T16:29:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.957+0000 7f27e2ec4b80 1 -- start start 2023-07-14T16:29:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.957+0000 7f27e2ec4b80 1 Processor -- start 2023-07-14T16:29:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca57210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 0x556a4ca57750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca5e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.935 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4c93eb40 con 0x556a4ca4d550 2023-07-14T16:29:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4c93e840 con 0x556a4ca4ecf0 2023-07-14T16:29:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4c8229a0 con 0x556a4ca4dea0 2023-07-14T16:29:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27e2ec4b80 10 client.0 _ll_register_callbacks cb 0x556a4c93c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27cd39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca57210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27cd39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca57210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53014/0 (socket says 172.21.15.62:53014) 2023-07-14T16:29:15.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27cd39b700 1 -- 172.21.15.62:0/2401712064 learned_addr learned my addr 172.21.15.62:0/2401712064 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27cdb9c700 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca5e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 0x556a4ca57750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27cd39b700 1 -- 172.21.15.62:0/2401712064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 msgr2=0x556a4ca57750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 0x556a4ca57750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.958+0000 7f27cd39b700 1 -- 172.21.15.62:0/2401712064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 msgr2=0x556a4ca5e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.959+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca5e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.959+0000 7f27cd39b700 1 -- 172.21.15.62:0/2401712064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c8108850 con 0x556a4ca4d550 2023-07-14T16:29:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.959+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca57210 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f27c4006010 tx=0x7f27c4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3476f2d888a7b532 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.959+0000 7f27b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.959+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2401712064 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c402f070 con 0x556a4ca4d550 2023-07-14T16:29:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.959+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2401712064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a4c8f8570 con 0x556a4ca4d550 2023-07-14T16:29:15.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.959+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2401712064 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f27c4005670 con 0x556a4ca4d550 2023-07-14T16:29:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2401712064 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c4004500 con 0x556a4ca4d550 2023-07-14T16:29:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27e2ec4b80 10 client.19353 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:15.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2401712064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556a4c7b5ce0 con 0x556a4ca4d550 2023-07-14T16:29:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2401712064 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c403a080 con 0x556a4ca4d550 2023-07-14T16:29:15.940 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2401712064 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27c4032030 con 0x556a4ca4d550 2023-07-14T16:29:15.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27e2ec4b80 10 client.19353 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:15.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27e2ec4b80 10 client.19353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:15.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27e2ec4b80 20 client.19353 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:15.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556a4ca62aa0 0x556a4ca64e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27e2ec4b80 4 client.19353 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:15.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.960+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2401712064 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a4c997f30 con 0x556a4ca62aa0 2023-07-14T16:29:15.941 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.961+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556a4ca62aa0 0x556a4ca64e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.961+0000 7f27b0ff9700 10 client.19353 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.961+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556a4ca62aa0 0x556a4ca64e80 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc384205d20f89b4 server_cookie=c7498ca4be3e31f5 in_seq=0 out_seq=0 2023-07-14T16:29:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.962+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2401712064 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556a4c997f30 con 0x556a4ca62aa0 2023-07-14T16:29:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.962+0000 7f27b0ff9700 10 client.19353 handle_command_reply: tid=0 2023-07-14T16:29:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.962+0000 7f27e2ec4b80 1 client.19353 shutdown 2023-07-14T16:29:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.962+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2401712064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 msgr2=0x556a4ca57210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.962+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca57210 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f27c4006010 tx=0x7f27c4005270 comp rx=0 tx=0).stop 2023-07-14T16:29:15.942 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.962+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2401712064 shutdown_connections 2023-07-14T16:29:15.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556a4ca62aa0 0x556a4ca64e80 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca5e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca57210 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2401712064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 0x556a4ca57750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2401712064 >> 172.21.15.62:0/2401712064 conn(0x556a4ca42e20 msgr2=0x556a4ca47930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:15.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27b0ff9700 0 client.19353 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:15.943 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2401712064 shutdown_connections 2023-07-14T16:29:15.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2401712064 wait complete. 2023-07-14T16:29:15.944 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.963+0000 7f27e2ec4b80 20 client.19353 trim_cache size 0 max 16384 2023-07-14T16:29:15.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.985+0000 7f27e2ec4b80 1 Processor -- start 2023-07-14T16:29:15.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.985+0000 7f27e2ec4b80 1 -- start start 2023-07-14T16:29:15.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca55160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 0x556a4ca5e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4ca566c0 con 0x556a4ca4d550 2023-07-14T16:29:15.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4ca55be0 con 0x556a4ca4ecf0 2023-07-14T16:29:15.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4ca62840 con 0x556a4ca4dea0 2023-07-14T16:29:15.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27cd39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca55160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27cd39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca55160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53022/0 (socket says 172.21.15.62:53022) 2023-07-14T16:29:15.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27cd39b700 1 -- 172.21.15.62:0/3862157536 learned_addr learned my addr 172.21.15.62:0/3862157536 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:15.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27cdb9c700 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 0x556a4ca5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27cd39b700 1 -- 172.21.15.62:0/3862157536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 msgr2=0x556a4ca5e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:15.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27cd39b700 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 0x556a4ca5e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27cd39b700 1 -- 172.21.15.62:0/3862157536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 msgr2=0x556a4ca556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:15.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.986+0000 7f27cd39b700 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.987+0000 7f27cd39b700 1 -- 172.21.15.62:0/3862157536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a4ca4f390 con 0x556a4ca4d550 2023-07-14T16:29:15.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.987+0000 7f27cd39b700 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca55160 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c010 tx=0x7f27c4005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbbe55adeaf961c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:15.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.987+0000 7f27b3fff700 1 -- 172.21.15.62:0/3862157536 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c403d070 con 0x556a4ca4d550 2023-07-14T16:29:15.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.987+0000 7f27b3fff700 1 -- 172.21.15.62:0/3862157536 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556a4c8f8570 con 0x556a4ca4d550 2023-07-14T16:29:15.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.987+0000 7f27b3fff700 1 -- 172.21.15.62:0/3862157536 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c402f030 con 0x556a4ca4d550 2023-07-14T16:29:15.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/3862157536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 msgr2=0x556a4ca55160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.971 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca55160 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c010 tx=0x7f27c4005360 comp rx=0 tx=0).stop 2023-07-14T16:29:15.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/3862157536 shutdown_connections 2023-07-14T16:29:15.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4d550 0x556a4ca55160 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/3862157536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4dea0 0x556a4ca5e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/3862157536 >> 172.21.15.62:0/3862157536 conn(0x556a4ca42e20 msgr2=0x556a4ca51bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/3862157536 shutdown_connections 2023-07-14T16:29:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/3862157536 wait complete. 2023-07-14T16:29:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.988+0000 7f27e2ec4b80 1 -- start start 2023-07-14T16:29:15.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.989+0000 7f27e2ec4b80 1 Processor -- start 2023-07-14T16:29:15.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.989+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4d550 0x556a4ca5e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.989+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca5b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.989+0000 7f27e2ec4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.989+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4ca5bc90 con 0x556a4ca4dea0 2023-07-14T16:29:15.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.989+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4cab68b0 con 0x556a4ca4ecf0 2023-07-14T16:29:15.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.989+0000 7f27e2ec4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4cab69f0 con 0x556a4ca4d550 2023-07-14T16:29:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.989+0000 7f27e2ec4b80 10 client.0 _ll_register_callbacks cb 0x7f27c4025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27ccb9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca5b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27ccb9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca5b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53030/0 (socket says 172.21.15.62:53030) 2023-07-14T16:29:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27ccb9a700 1 -- 172.21.15.62:0/2770236241 learned_addr learned my addr 172.21.15.62:0/2770236241 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:15.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4d550 0x556a4ca5e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27cdb9c700 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27ccb9a700 1 -- 172.21.15.62:0/2770236241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4d550 msgr2=0x556a4ca5e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:15.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4d550 0x556a4ca5e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27ccb9a700 1 -- 172.21.15.62:0/2770236241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 msgr2=0x556a4ca5b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:15.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca5b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27ccb9a700 1 -- 172.21.15.62:0/2770236241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a4c7b5ce0 con 0x556a4ca4dea0 2023-07-14T16:29:15.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.990+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4d550 0x556a4ca5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:15.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27ccb9a700 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca5b070 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005950 tx=0x7f27b800a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9e22dcbd8fa570 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:15.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:15.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2770236241 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8028600 con 0x556a4ca4dea0 2023-07-14T16:29:15.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2770236241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27c401e040 con 0x556a4ca4dea0 2023-07-14T16:29:15.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2770236241 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f27b8024040 con 0x556a4ca4dea0 2023-07-14T16:29:15.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2770236241 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27b8004020 con 0x556a4ca4dea0 2023-07-14T16:29:15.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27e2ec4b80 10 client.19359 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2770236241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556a4c7f32d0 con 0x556a4ca4dea0 2023-07-14T16:29:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.991+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2770236241 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c401e040 con 0x556a4ca4dea0 2023-07-14T16:29:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.992+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2770236241 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27b801f7f0 con 0x556a4ca4dea0 2023-07-14T16:29:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.992+0000 7f27e2ec4b80 10 client.19359 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:15.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.992+0000 7f27e2ec4b80 10 client.19359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:15.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.992+0000 7f27e2ec4b80 20 client.19359 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:15.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.992+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556a4cab8400 0x556a4caba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.992+0000 7f27e2ec4b80 4 client.19359 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:29:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.992+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2770236241 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556a4c95a0d0 con 0x556a4cab8400 2023-07-14T16:29:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.992+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556a4cab8400 0x556a4caba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.993+0000 7f27b0ff9700 10 client.19359 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.993+0000 7f27cd39b700 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556a4cab8400 0x556a4caba810 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e54ca95482dc28a server_cookie=4b99bf6f0768773b in_seq=0 out_seq=0 2023-07-14T16:29:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27b0ff9700 1 -- 172.21.15.62:0/2770236241 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556a4c95a0d0 con 0x556a4cab8400 2023-07-14T16:29:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27b0ff9700 10 client.19359 handle_command_reply: tid=0 2023-07-14T16:29:15.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 client.19359 shutdown 2023-07-14T16:29:15.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2770236241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 msgr2=0x556a4ca5b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca5b070 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005950 tx=0x7f27b800a380 comp rx=0 tx=0).stop 2023-07-14T16:29:15.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2770236241 shutdown_connections 2023-07-14T16:29:15.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556a4cab8400 0x556a4caba810 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a4ca4ecf0 0x556a4ca5b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556a4ca4d550 0x556a4ca5e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 --2- 172.21.15.62:0/2770236241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556a4ca4dea0 0x556a4ca5b070 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27b0ff9700 0 client.19359 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:15.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2770236241 >> 172.21.15.62:0/2770236241 conn(0x556a4ca42e20 msgr2=0x556a4ca51bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:15.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.994+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2770236241 shutdown_connections 2023-07-14T16:29:15.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27e2ec4b80 1 -- 172.21.15.62:0/2770236241 wait complete. 2023-07-14T16:29:15.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27e2ec4b80 20 client.19359 trim_cache size 0 max 16384 2023-07-14T16:29:15.989 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:29:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:29:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "5388d5d3-d6b1-4f02-a378-c6b9bd138083", 2023-07-14T16:29:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:29:15.991 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:29:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 -- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f27b40a1150 msgr2=0x7f27b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f27b40a1150 0x7f27b40a3600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f27b800afd0 tx=0x7f27b8011440 comp rx=0 tx=0).stop 2023-07-14T16:29:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 -- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8101660 msgr2=0x7f27c8072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8101660 0x7f27c8072e40 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f27c0007ee0 tx=0x7f27c000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 -- 172.21.15.62:0/4009811603 shutdown_connections 2023-07-14T16:29:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27c8101660 0x7f27c8072e40 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27c80ffae0 0x7f27c80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f27b40a1150 0x7f27b40a3600 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 --2- 172.21.15.62:0/4009811603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c8100cd0 0x7f27c8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.995+0000 7f27cf5ff700 1 -- 172.21.15.62:0/4009811603 >> 172.21.15.62:0/4009811603 conn(0x7f27c80fb0e0 msgr2=0x7f27c81041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.996+0000 7f27cf5ff700 1 -- 172.21.15.62:0/4009811603 shutdown_connections 2023-07-14T16:29:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:15.996+0000 7f27cf5ff700 1 -- 172.21.15.62:0/4009811603 wait complete. 2023-07-14T16:29:15.994 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:29:16.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.257+0000 7f81a59ff700 1 -- 172.21.15.62:0/1498271631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0100560 msgr2=0x7f81a0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.257+0000 7f81a59ff700 1 --2- 172.21.15.62:0/1498271631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0100560 0x7f81a0102940 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f8190009990 tx=0x7f8190009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.258+0000 7f81a59ff700 1 -- 172.21.15.62:0/1498271631 shutdown_connections 2023-07-14T16:29:16.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.258+0000 7f81a59ff700 1 --2- 172.21.15.62:0/1498271631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81a01057a0 0x7f81a0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.258+0000 7f81a59ff700 1 --2- 172.21.15.62:0/1498271631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0100560 0x7f81a0102940 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.258+0000 7f81a59ff700 1 --2- 172.21.15.62:0/1498271631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0102e80 0x7f81a0105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.258+0000 7f81a59ff700 1 -- 172.21.15.62:0/1498271631 >> 172.21.15.62:0/1498271631 conn(0x7f81a00fa000 msgr2=0x7f81a00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.258+0000 7f81a59ff700 1 -- 172.21.15.62:0/1498271631 shutdown_connections 2023-07-14T16:29:16.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.258+0000 7f81a59ff700 1 -- 172.21.15.62:0/1498271631 wait complete. 2023-07-14T16:29:16.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.259+0000 7f81a59ff700 1 Processor -- start 2023-07-14T16:29:16.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.259+0000 7f81a59ff700 1 -- start start 2023-07-14T16:29:16.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.259+0000 7f81a59ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0100560 0x7f81a00ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.259+0000 7f81a59ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0102e80 0x7f81a00fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.259+0000 7f81a59ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81a01057a0 0x7f81a00fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.259+0000 7f81a59ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00ff1f0 con 0x7f81a0102e80 2023-07-14T16:29:16.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.259+0000 7f81a59ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00ff360 con 0x7f81a01057a0 2023-07-14T16:29:16.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.259+0000 7f81a59ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a019f7d0 con 0x7f81a0100560 2023-07-14T16:29:16.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0102e80 0x7f81a00fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0102e80 0x7f81a00fe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53036/0 (socket says 172.21.15.62:53036) 2023-07-14T16:29:16.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 -- 172.21.15.62:0/2074946579 learned_addr learned my addr 172.21.15.62:0/2074946579 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819f7fe700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81a01057a0 0x7f81a00fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819effd700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0100560 0x7f81a00ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 -- 172.21.15.62:0/2074946579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0100560 msgr2=0x7f81a00ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0100560 0x7f81a00ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 -- 172.21.15.62:0/2074946579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81a01057a0 msgr2=0x7f81a00fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81a01057a0 0x7f81a00fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 -- 172.21.15.62:0/2074946579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81900095f0 con 0x7f81a0102e80 2023-07-14T16:29:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.260+0000 7f819e7fc700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0102e80 0x7f81a00fe510 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f819400d950 tx=0x7f819400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.261+0000 7f81a49fd700 1 -- 172.21.15.62:0/2074946579 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194009940 con 0x7f81a0102e80 2023-07-14T16:29:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.261+0000 7f81a59ff700 1 -- 172.21.15.62:0/2074946579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a019fab0 con 0x7f81a0102e80 2023-07-14T16:29:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.261+0000 7f81a49fd700 1 -- 172.21.15.62:0/2074946579 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8194010460 con 0x7f81a0102e80 2023-07-14T16:29:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.261+0000 7f81a49fd700 1 -- 172.21.15.62:0/2074946579 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819400f620 con 0x7f81a0102e80 2023-07-14T16:29:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.261+0000 7f81a59ff700 1 -- 172.21.15.62:0/2074946579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a019ffc0 con 0x7f81a0102e80 2023-07-14T16:29:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.265+0000 7f81a49fd700 1 -- 172.21.15.62:0/2074946579 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f81940105d0 con 0x7f81a0102e80 2023-07-14T16:29:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.266+0000 7f81a49fd700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81880a1150 0x7f81880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.266+0000 7f81a49fd700 1 -- 172.21.15.62:0/2074946579 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f81940cc7b0 con 0x7f81a0102e80 2023-07-14T16:29:16.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.266+0000 7f819effd700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81880a1150 0x7f81880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f81b93cab80 1 Processor -- start 2023-07-14T16:29:16.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f81b93cab80 1 -- start start 2023-07-14T16:29:16.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb78e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb78eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 0x5587fb78f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f81b93cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb68acb0 con 0x5587fb78e630 2023-07-14T16:29:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f81b93cab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb6dda40 con 0x5587fb78dce0 2023-07-14T16:29:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f81b93cab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb6d6ed0 con 0x5587fb78f480 2023-07-14T16:29:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.267+0000 7f819effd700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81880a1150 0x7f81880a3600 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f819000b5c0 tx=0x7f8190005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb78eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb78eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53040/0 (socket says 172.21.15.62:53040) 2023-07-14T16:29:16.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 -- 172.21.15.62:0/3295615355 learned_addr learned my addr 172.21.15.62:0/3295615355 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819effd700 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb78e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819f7fe700 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 0x5587fb78f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 -- 172.21.15.62:0/3295615355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 msgr2=0x5587fb78f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 0x5587fb78f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 -- 172.21.15.62:0/3295615355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 msgr2=0x5587fb78e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb78e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 -- 172.21.15.62:0/3295615355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587fb504de0 con 0x5587fb78e630 2023-07-14T16:29:16.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.268+0000 7f819e7fc700 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb78eef0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8194096070 tx=0x7f81940994d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaa5360dde71a0b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.269+0000 7f8185ffb700 1 -- 172.21.15.62:0/3295615355 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194099b70 con 0x5587fb78e630 2023-07-14T16:29:16.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.269+0000 7f8185ffb700 1 -- 172.21.15.62:0/3295615355 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f819409dba0 con 0x5587fb78e630 2023-07-14T16:29:16.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.269+0000 7f8185ffb700 1 -- 172.21.15.62:0/3295615355 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940a1b30 con 0x5587fb78e630 2023-07-14T16:29:16.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.269+0000 7f81b93cab80 1 -- 172.21.15.62:0/3295615355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 msgr2=0x5587fb78eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.269+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb78eef0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8194096070 tx=0x7f81940994d0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 -- 172.21.15.62:0/3295615355 shutdown_connections 2023-07-14T16:29:16.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 0x5587fb78f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb78e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3295615355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb78eef0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 -- 172.21.15.62:0/3295615355 >> 172.21.15.62:0/3295615355 conn(0x5587fb77ada0 msgr2=0x5587fb7801d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 -- 172.21.15.62:0/3295615355 shutdown_connections 2023-07-14T16:29:16.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 -- 172.21.15.62:0/3295615355 wait complete. 2023-07-14T16:29:16.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 -- start start 2023-07-14T16:29:16.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.270+0000 7f81b93cab80 1 Processor -- start 2023-07-14T16:29:16.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb797820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb797d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 0x5587fb79eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f81b93cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb676ac0 con 0x5587fb78e630 2023-07-14T16:29:16.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f81b93cab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb6767c0 con 0x5587fb78f480 2023-07-14T16:29:16.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f81b93cab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb55a920 con 0x5587fb78dce0 2023-07-14T16:29:16.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f81b93cab80 10 client.0 _ll_register_callbacks cb 0x5587fb674760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:16.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb797d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb797d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53042/0 (socket says 172.21.15.62:53042) 2023-07-14T16:29:16.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f819e7fc700 1 -- 172.21.15.62:0/3890368448 learned_addr learned my addr 172.21.15.62:0/3890368448 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f819f7fe700 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 0x5587fb79eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.271+0000 7f819effd700 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb797820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f819e7fc700 1 -- 172.21.15.62:0/3890368448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 msgr2=0x5587fb797820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f819e7fc700 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb797820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f819e7fc700 1 -- 172.21.15.62:0/3890368448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 msgr2=0x5587fb79eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f819e7fc700 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 0x5587fb79eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f819e7fc700 1 -- 172.21.15.62:0/3890368448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a019ffc0 con 0x5587fb78e630 2023-07-14T16:29:16.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f819effd700 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb797820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:29:16.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f819e7fc700 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb797d60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f8194099640 tx=0x7f81940a2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3ecf0c041d6841 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f817effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:16.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f817effd700 1 -- 172.21.15.62:0/3890368448 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940a1b30 con 0x5587fb78e630 2023-07-14T16:29:16.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f81b93cab80 1 -- 172.21.15.62:0/3890368448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587fb6304f0 con 0x5587fb78e630 2023-07-14T16:29:16.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f817effd700 1 -- 172.21.15.62:0/3890368448 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f81940a1c90 con 0x5587fb78e630 2023-07-14T16:29:16.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f817effd700 1 -- 172.21.15.62:0/3890368448 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81940ca120 con 0x5587fb78e630 2023-07-14T16:29:16.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f81b93cab80 10 client.19371 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:16.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.272+0000 7f81b93cab80 1 -- 172.21.15.62:0/3890368448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5587fb4edc60 con 0x5587fb78e630 2023-07-14T16:29:16.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.273+0000 7f817effd700 1 -- 172.21.15.62:0/3890368448 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5587fb6304f0 con 0x5587fb78e630 2023-07-14T16:29:16.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.273+0000 7f817effd700 1 -- 172.21.15.62:0/3890368448 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8194009aa0 con 0x5587fb78e630 2023-07-14T16:29:16.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.273+0000 7f81b93cab80 10 client.19371 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:16.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.273+0000 7f81b93cab80 10 client.19371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:16.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.273+0000 7f81b93cab80 20 client.19371 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:16.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.273+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fb7a3170 0x5587fb7a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.273+0000 7f81b93cab80 4 client.19371 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:16.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.273+0000 7f81b93cab80 1 -- 172.21.15.62:0/3890368448 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587fb6cfeb0 con 0x5587fb7a3170 2023-07-14T16:29:16.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.274+0000 7f819effd700 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fb7a3170 0x5587fb7a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.274+0000 7f819effd700 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fb7a3170 0x5587fb7a5580 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d9dd02971389e8 server_cookie=94d9fe401420a2ba in_seq=0 out_seq=0 2023-07-14T16:29:16.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.274+0000 7f817effd700 10 client.19371 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:16.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f817effd700 1 -- 172.21.15.62:0/3890368448 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5587fb6cfeb0 con 0x5587fb7a3170 2023-07-14T16:29:16.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f817effd700 10 client.19371 handle_command_reply: tid=0 2023-07-14T16:29:16.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 client.19371 shutdown 2023-07-14T16:29:16.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 -- 172.21.15.62:0/3890368448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 msgr2=0x5587fb797d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb797d60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f8194099640 tx=0x7f81940a2040 comp rx=0 tx=0).stop 2023-07-14T16:29:16.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 -- 172.21.15.62:0/3890368448 shutdown_connections 2023-07-14T16:29:16.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fb7a3170 0x5587fb7a5580 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 0x5587fb79eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb797820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 --2- 172.21.15.62:0/3890368448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb797d60 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f817effd700 0 client.19371 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:16.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 -- 172.21.15.62:0/3890368448 >> 172.21.15.62:0/3890368448 conn(0x5587fb77ada0 msgr2=0x5587fb77f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 -- 172.21.15.62:0/3890368448 shutdown_connections 2023-07-14T16:29:16.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.275+0000 7f81b93cab80 1 -- 172.21.15.62:0/3890368448 wait complete. 2023-07-14T16:29:16.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.276+0000 7f81b93cab80 20 client.19371 trim_cache size 0 max 16384 2023-07-14T16:29:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.293+0000 7f81b93cab80 1 Processor -- start 2023-07-14T16:29:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.293+0000 7f81b93cab80 1 -- start start 2023-07-14T16:29:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.293+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb796490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.293+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.293+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 0x5587fb79f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.293+0000 7f81b93cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb77a9c0 con 0x5587fb78e630 2023-07-14T16:29:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.293+0000 7f81b93cab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb798720 con 0x5587fb78f480 2023-07-14T16:29:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.293+0000 7f81b93cab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb796d00 con 0x5587fb78dce0 2023-07-14T16:29:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53056/0 (socket says 172.21.15.62:53056) 2023-07-14T16:29:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819e7fc700 1 -- 172.21.15.62:0/637314791 learned_addr learned my addr 172.21.15.62:0/637314791 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819f7fe700 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 0x5587fb79f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819effd700 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb796490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819e7fc700 1 -- 172.21.15.62:0/637314791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 msgr2=0x5587fb796490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819e7fc700 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb796490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819e7fc700 1 -- 172.21.15.62:0/637314791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 msgr2=0x5587fb79f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819e7fc700 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 0x5587fb79f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819e7fc700 1 -- 172.21.15.62:0/637314791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587fb52b250 con 0x5587fb78e630 2023-07-14T16:29:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.294+0000 7f819effd700 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb796490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:16.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.295+0000 7f819e7fc700 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79ed60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8194096070 tx=0x7f8194095eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c004af37e8e835 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.295+0000 7f81867fc700 1 -- 172.21.15.62:0/637314791 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940ca120 con 0x5587fb78e630 2023-07-14T16:29:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.295+0000 7f81867fc700 1 -- 172.21.15.62:0/637314791 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f81940b2270 con 0x5587fb78e630 2023-07-14T16:29:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.295+0000 7f81867fc700 1 -- 172.21.15.62:0/637314791 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940a1b30 con 0x5587fb78e630 2023-07-14T16:29:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 -- 172.21.15.62:0/637314791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 msgr2=0x5587fb79ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79ed60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8194096070 tx=0x7f8194095eb0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 -- 172.21.15.62:0/637314791 shutdown_connections 2023-07-14T16:29:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78f480 0x5587fb79f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78dce0 0x5587fb796490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 --2- 172.21.15.62:0/637314791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79ed60 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 -- 172.21.15.62:0/637314791 >> 172.21.15.62:0/637314791 conn(0x5587fb77ada0 msgr2=0x5587fb7a9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 -- 172.21.15.62:0/637314791 shutdown_connections 2023-07-14T16:29:16.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 -- 172.21.15.62:0/637314791 wait complete. 2023-07-14T16:29:16.273 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.296+0000 7f81b93cab80 1 -- start start 2023-07-14T16:29:16.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.297+0000 7f81b93cab80 1 Processor -- start 2023-07-14T16:29:16.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.297+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb79ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.297+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.297+0000 7f81b93cab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 0x5587fb79d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.297+0000 7f81b93cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb79db10 con 0x5587fb78e630 2023-07-14T16:29:16.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.297+0000 7f81b93cab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb79dc80 con 0x5587fb78dce0 2023-07-14T16:29:16.274 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.297+0000 7f81b93cab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fb79de40 con 0x5587fb78f480 2023-07-14T16:29:16.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.297+0000 7f81b93cab80 10 client.0 _ll_register_callbacks cb 0x7f8190005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:16.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb79ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 0x5587fb79d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb79ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36566/0 (socket says 172.21.15.62:36566) 2023-07-14T16:29:16.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53068/0 (socket says 172.21.15.62:53068) 2023-07-14T16:29:16.275 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819effd700 1 -- 172.21.15.62:0/1035097050 learned_addr learned my addr 172.21.15.62:0/1035097050 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819e7fc700 1 -- 172.21.15.62:0/1035097050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 msgr2=0x5587fb79d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819e7fc700 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 0x5587fb79d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819e7fc700 1 -- 172.21.15.62:0/1035097050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 msgr2=0x5587fb79ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819e7fc700 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb79ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819e7fc700 1 -- 172.21.15.62:0/1035097050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587fb4edc60 con 0x5587fb78e630 2023-07-14T16:29:16.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f819e7fc700 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79d090 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8194010c80 tx=0x7f81940a2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0d07cc50c4b5b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.298+0000 7f817effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:16.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f817effd700 1 -- 172.21.15.62:0/1035097050 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940ca120 con 0x5587fb78e630 2023-07-14T16:29:16.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f817effd700 1 -- 172.21.15.62:0/1035097050 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8194095960 con 0x5587fb78e630 2023-07-14T16:29:16.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 1 -- 172.21.15.62:0/1035097050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f819409dba0 con 0x5587fb78e630 2023-07-14T16:29:16.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f817effd700 1 -- 172.21.15.62:0/1035097050 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8194099060 con 0x5587fb78e630 2023-07-14T16:29:16.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 10 client.19377 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:16.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 1 -- 172.21.15.62:0/1035097050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5587fb5032b0 con 0x5587fb78e630 2023-07-14T16:29:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f817effd700 1 -- 172.21.15.62:0/1035097050 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f819409dba0 con 0x5587fb78e630 2023-07-14T16:29:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f817effd700 1 -- 172.21.15.62:0/1035097050 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8194078c90 con 0x5587fb78e630 2023-07-14T16:29:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 10 client.19377 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 10 client.19377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 20 client.19377 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:16.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fb7f82c0 0x5587fb7fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 4 client.19377 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:29:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.299+0000 7f81b93cab80 1 -- 172.21.15.62:0/1035097050 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5587fb692050 con 0x5587fb7f82c0 2023-07-14T16:29:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.300+0000 7f819effd700 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fb7f82c0 0x5587fb7fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f819effd700 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fb7f82c0 0x5587fb7fa6a0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47eabc00e8102ba8 server_cookie=a6577890d7fd8315 in_seq=0 out_seq=0 2023-07-14T16:29:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f817effd700 10 client.19377 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:16.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f817effd700 1 -- 172.21.15.62:0/1035097050 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5587fb692050 con 0x5587fb7f82c0 2023-07-14T16:29:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f817effd700 10 client.19377 handle_command_reply: tid=0 2023-07-14T16:29:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 client.19377 shutdown 2023-07-14T16:29:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 -- 172.21.15.62:0/1035097050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 msgr2=0x5587fb79d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79d090 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8194010c80 tx=0x7f81940a2040 comp rx=0 tx=0).stop 2023-07-14T16:29:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 -- 172.21.15.62:0/1035097050 shutdown_connections 2023-07-14T16:29:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fb78f480 0x5587fb79d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fb7f82c0 0x5587fb7fa6a0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fb78dce0 0x5587fb79ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 --2- 172.21.15.62:0/1035097050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fb78e630 0x5587fb79d090 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f817effd700 0 client.19377 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:16.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 -- 172.21.15.62:0/1035097050 >> 172.21.15.62:0/1035097050 conn(0x5587fb77ada0 msgr2=0x5587fb79f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 -- 172.21.15.62:0/1035097050 shutdown_connections 2023-07-14T16:29:16.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.301+0000 7f81b93cab80 1 -- 172.21.15.62:0/1035097050 wait complete. 2023-07-14T16:29:16.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81b93cab80 20 client.19377 trim_cache size 0 max 16384 2023-07-14T16:29:16.284 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:29:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:29:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:29:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "5388d5d3-d6b1-4f02-a378-c6b9bd138083": { 2023-07-14T16:29:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:29:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "5388d5d3-d6b1-4f02-a378-c6b9bd138083", 2023-07-14T16:29:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:29:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:29:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:29:16.285 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:29:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 -- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81880a1150 msgr2=0x7f81880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81880a1150 0x7f81880a3600 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f819000b5c0 tx=0x7f8190005fb0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 -- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0102e80 msgr2=0x7f81a00fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0102e80 0x7f81a00fe510 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f819400d950 tx=0x7f819400dc60 comp rx=0 tx=0).stop 2023-07-14T16:29:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 -- 172.21.15.62:0/2074946579 shutdown_connections 2023-07-14T16:29:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81880a1150 0x7f81880a3600 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81a01057a0 0x7f81a00fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81a0100560 0x7f81a00ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 --2- 172.21.15.62:0/2074946579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81a0102e80 0x7f81a00fe510 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.302+0000 7f81a59ff700 1 -- 172.21.15.62:0/2074946579 >> 172.21.15.62:0/2074946579 conn(0x7f81a00fa000 msgr2=0x7f81a01011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.303+0000 7f81a59ff700 1 -- 172.21.15.62:0/2074946579 shutdown_connections 2023-07-14T16:29:16.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:16.303+0000 7f81a59ff700 1 -- 172.21.15.62:0/2074946579 wait complete. 2023-07-14T16:29:16.294 INFO:tasks.cephfs.filesystem:scrub status for tag:5388d5d3-d6b1-4f02-a378-c6b9bd138083 - {'path': '/', 'tag': '5388d5d3-d6b1-4f02-a378-c6b9bd138083', 'options': 'recursive,force'} 2023-07-14T16:29:16.399 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:29:16.007+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v253>,<0x10000000003/client.0 v254>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v266>,<0x10000000001/sv_0 v280>,<0x10000000000/_nogroup v274>,<0x1/volumes v264>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:29:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 -- 172.21.15.62:0/615194107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 msgr2=0x7faeb0073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 --2- 172.21.15.62:0/615194107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 0x7faeb0073380 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7faea4009990 tx=0x7faea4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 -- 172.21.15.62:0/615194107 shutdown_connections 2023-07-14T16:29:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 --2- 172.21.15.62:0/615194107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeb01079c0 0x7faeb0109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 --2- 172.21.15.62:0/615194107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb00744f0 0x7faeb0072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 --2- 172.21.15.62:0/615194107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 0x7faeb0073380 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 -- 172.21.15.62:0/615194107 >> 172.21.15.62:0/615194107 conn(0x7faeb00fb130 msgr2=0x7faeb00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 -- 172.21.15.62:0/615194107 shutdown_connections 2023-07-14T16:29:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.630+0000 7faeb5bff700 1 -- 172.21.15.62:0/615194107 wait complete. 2023-07-14T16:29:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.631+0000 7faeb5bff700 1 Processor -- start 2023-07-14T16:29:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.631+0000 7faeb5bff700 1 -- start start 2023-07-14T16:29:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeb5bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 0x7faeb0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeb5bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeb00744f0 0x7faeb0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeb5bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb01079c0 0x7faeb0198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeb5bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb0199480 con 0x7faeb0072f10 2023-07-14T16:29:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeb5bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb01995f0 con 0x7faeb01079c0 2023-07-14T16:29:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeb5bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb01997b0 con 0x7faeb00744f0 2023-07-14T16:29:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 0x7faeb0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 0x7faeb0197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53094/0 (socket says 172.21.15.62:53094) 2023-07-14T16:29:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaf7fe700 1 -- 172.21.15.62:0/3980889729 learned_addr learned my addr 172.21.15.62:0/3980889729 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaeffd700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeb00744f0 0x7faeb0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaffff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb01079c0 0x7faeb0198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaf7fe700 1 -- 172.21.15.62:0/3980889729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeb00744f0 msgr2=0x7faeb0197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeb00744f0 0x7faeb0197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaf7fe700 1 -- 172.21.15.62:0/3980889729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb01079c0 msgr2=0x7faeb0198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb01079c0 0x7faeb0198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.632+0000 7faeaf7fe700 1 -- 172.21.15.62:0/3980889729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea40095f0 con 0x7faeb0072f10 2023-07-14T16:29:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.633+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 0x7faeb0197a60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7faea000e9a0 tx=0x7faea000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.633+0000 7faeacff9700 1 -- 172.21.15.62:0/3980889729 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0009790 con 0x7faeb0072f10 2023-07-14T16:29:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.633+0000 7faeacff9700 1 -- 172.21.15.62:0/3980889729 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faea000cbd0 con 0x7faeb0072f10 2023-07-14T16:29:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.633+0000 7faeb5bff700 1 -- 172.21.15.62:0/3980889729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb0199a90 con 0x7faeb0072f10 2023-07-14T16:29:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.633+0000 7faeacff9700 1 -- 172.21.15.62:0/3980889729 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0018680 con 0x7faeb0072f10 2023-07-14T16:29:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.633+0000 7faeb5bff700 1 -- 172.21.15.62:0/3980889729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb01a0250 con 0x7faeb0072f10 2023-07-14T16:29:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.637+0000 7faeacff9700 1 -- 172.21.15.62:0/3980889729 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faea000cd40 con 0x7faeb0072f10 2023-07-14T16:29:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.638+0000 7faec95b5b80 1 Processor -- start 2023-07-14T16:29:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.638+0000 7faec95b5b80 1 -- start start 2023-07-14T16:29:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58fdc66b0 0x55b58ff6a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff6b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faec95b5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58fe6fc50 con 0x55b58fdc66b0 2023-07-14T16:29:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faec95b5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58fec29e0 con 0x55b58ff6a840 2023-07-14T16:29:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faec95b5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58febbe70 con 0x55b58ff6b6e0 2023-07-14T16:29:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faeaffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58fdc66b0 0x55b58ff6a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faeaeffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faeaffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58fdc66b0 0x55b58ff6a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53096/0 (socket says 172.21.15.62:53096) 2023-07-14T16:29:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faeaffff700 1 -- 172.21.15.62:0/3842366298 learned_addr learned my addr 172.21.15.62:0/3842366298 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.639+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff6b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeacff9700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fae980a11d0 0x7fae980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeaeffd700 1 -- 172.21.15.62:0/3842366298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 msgr2=0x55b58ff6b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeaeffd700 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff6b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeaeffd700 1 -- 172.21.15.62:0/3842366298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58fdc66b0 msgr2=0x55b58ff6a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeaeffd700 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58fdc66b0 0x55b58ff6a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeaeffd700 1 -- 172.21.15.62:0/3842366298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58fce9d80 con 0x55b58ff6b6e0 2023-07-14T16:29:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeaffff700 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58fdc66b0 0x55b58ff6a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeacff9700 1 -- 172.21.15.62:0/3980889729 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faea0014070 con 0x7faeb0072f10 2023-07-14T16:29:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeaffff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fae980a11d0 0x7fae980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.640+0000 7faeaeffd700 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6bb80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faea4000c00 tx=0x7faea4005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17a18177f9f02e94 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.641+0000 7fae95ffb700 1 -- 172.21.15.62:0/3842366298 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4005240 con 0x55b58ff6b6e0 2023-07-14T16:29:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.641+0000 7faeaffff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fae980a11d0 0x7fae980a3680 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faea8009fd0 tx=0x7faea8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.641+0000 7fae95ffb700 1 -- 172.21.15.62:0/3842366298 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faea401e040 con 0x55b58ff6b6e0 2023-07-14T16:29:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.641+0000 7fae95ffb700 1 -- 172.21.15.62:0/3842366298 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400b970 con 0x55b58ff6b6e0 2023-07-14T16:29:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 -- 172.21.15.62:0/3842366298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 msgr2=0x55b58ff6bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6bb80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faea4000c00 tx=0x7faea4005ee0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 -- 172.21.15.62:0/3842366298 shutdown_connections 2023-07-14T16:29:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6bb80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58fdc66b0 0x55b58ff6a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 --2- 172.21.15.62:0/3842366298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff6b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 -- 172.21.15.62:0/3842366298 >> 172.21.15.62:0/3842366298 conn(0x55b58ff5fbe0 msgr2=0x55b58ff64cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 -- 172.21.15.62:0/3842366298 shutdown_connections 2023-07-14T16:29:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 -- 172.21.15.62:0/3842366298 wait complete. 2023-07-14T16:29:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.642+0000 7faec95b5b80 1 -- start start 2023-07-14T16:29:16.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.643+0000 7faec95b5b80 1 Processor -- start 2023-07-14T16:29:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.643+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58fdc66b0 0x55b58ff73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.643+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6a840 0x55b58ff73d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.643+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff7ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.643+0000 7faec95b5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58fe5ba60 con 0x55b58ff6a840 2023-07-14T16:29:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.643+0000 7faec95b5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58fe5b760 con 0x55b58fdc66b0 2023-07-14T16:29:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.643+0000 7faec95b5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58fd3f8c0 con 0x55b58ff6b6e0 2023-07-14T16:29:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.643+0000 7faec95b5b80 10 client.0 _ll_register_callbacks cb 0x55b58fe59700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6a840 0x55b58ff73d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6a840 0x55b58ff73d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53100/0 (socket says 172.21.15.62:53100) 2023-07-14T16:29:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaeffd700 1 -- 172.21.15.62:0/2272027233 learned_addr learned my addr 172.21.15.62:0/2272027233 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58fdc66b0 0x55b58ff73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaffff700 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff7ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaf7fe700 1 -- 172.21.15.62:0/2272027233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 msgr2=0x55b58ff7ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff7ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaf7fe700 1 -- 172.21.15.62:0/2272027233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6a840 msgr2=0x55b58ff73d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6a840 0x55b58ff73d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaf7fe700 1 -- 172.21.15.62:0/2272027233 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea80090f0 con 0x55b58fdc66b0 2023-07-14T16:29:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaffff700 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff7ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58fdc66b0 0x55b58ff73800 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faea00ca7b0 tx=0x7faea009a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93031eeb17bb9345 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.644+0000 7fae8effd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:29:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.645+0000 7fae8effd700 1 -- 172.21.15.62:0/2272027233 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea00a2230 con 0x55b58fdc66b0 2023-07-14T16:29:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.645+0000 7fae8effd700 1 -- 172.21.15.62:0/2272027233 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faea009a070 con 0x55b58fdc66b0 2023-07-14T16:29:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.645+0000 7faec95b5b80 1 -- 172.21.15.62:0/2272027233 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b58fe15490 con 0x55b58fdc66b0 2023-07-14T16:29:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.645+0000 7fae8effd700 1 -- 172.21.15.62:0/2272027233 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea0099bb0 con 0x55b58fdc66b0 2023-07-14T16:29:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.645+0000 7faec95b5b80 10 client.28958 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.645+0000 7faec95b5b80 1 -- 172.21.15.62:0/2272027233 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7faeb010aa60 con 0x55b58fdc66b0 2023-07-14T16:29:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.645+0000 7fae8effd700 1 -- 172.21.15.62:0/2272027233 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b58fe15490 con 0x55b58fdc66b0 2023-07-14T16:29:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.645+0000 7fae8effd700 1 -- 172.21.15.62:0/2272027233 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faea00b2a50 con 0x55b58fdc66b0 2023-07-14T16:29:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.646+0000 7faec95b5b80 10 client.28958 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:16.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.646+0000 7faec95b5b80 10 client.28958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.646+0000 7faec95b5b80 20 client.28958 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.646+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b58ff7f5a0 0x55b58ff819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.646+0000 7faec95b5b80 4 client.28958 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.646+0000 7faec95b5b80 1 -- 172.21.15.62:0/2272027233 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b58feb4e50 con 0x55b58ff7f5a0 2023-07-14T16:29:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.646+0000 7faeaeffd700 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b58ff7f5a0 0x55b58ff819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.647+0000 7faeaeffd700 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b58ff7f5a0 0x55b58ff819b0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6d5b6ad964ed29 server_cookie=6500c7fb9891310c in_seq=0 out_seq=0 2023-07-14T16:29:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.647+0000 7fae8effd700 10 client.28958 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.647+0000 7fae8effd700 1 -- 172.21.15.62:0/2272027233 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b58feb4e50 con 0x55b58ff7f5a0 2023-07-14T16:29:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.647+0000 7fae8effd700 10 client.28958 handle_command_reply: tid=0 2023-07-14T16:29:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 client.28958 shutdown 2023-07-14T16:29:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 -- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58fdc66b0 msgr2=0x55b58ff73800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58fdc66b0 0x55b58ff73800 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faea00ca7b0 tx=0x7faea009a5b0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 -- 172.21.15.62:0/2272027233 shutdown_connections 2023-07-14T16:29:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58ff6b6e0 0x55b58ff7ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b58ff7f5a0 0x55b58ff819b0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58fdc66b0 0x55b58ff73800 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2272027233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6a840 0x55b58ff73d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7fae8effd700 0 client.28958 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 -- 172.21.15.62:0/2272027233 >> 172.21.15.62:0/2272027233 conn(0x55b58ff5fbe0 msgr2=0x55b58ff644e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 -- 172.21.15.62:0/2272027233 shutdown_connections 2023-07-14T16:29:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 1 -- 172.21.15.62:0/2272027233 wait complete. 2023-07-14T16:29:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.648+0000 7faec95b5b80 20 client.28958 trim_cache size 0 max 16384 2023-07-14T16:29:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.664+0000 7faec95b5b80 1 Processor -- start 2023-07-14T16:29:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.664+0000 7faec95b5b80 1 -- start start 2023-07-14T16:29:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.664+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff86310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.664+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff7ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.664+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.664+0000 7faec95b5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ff72cf0 con 0x55b58ff6b6e0 2023-07-14T16:29:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.664+0000 7faec95b5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ff733a0 con 0x55b58ff6a840 2023-07-14T16:29:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.664+0000 7faec95b5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ff657f0 con 0x55b58fdc66b0 2023-07-14T16:29:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaf7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff86310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53108/0 (socket says 172.21.15.62:53108) 2023-07-14T16:29:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaffff700 1 -- 172.21.15.62:0/4017515870 learned_addr learned my addr 172.21.15.62:0/4017515870 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaeffd700 1 --2- 172.21.15.62:0/4017515870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff7ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaf7fe700 1 -- 172.21.15.62:0/4017515870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 msgr2=0x55b58ff7ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/4017515870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff7ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaf7fe700 1 -- 172.21.15.62:0/4017515870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 msgr2=0x55b58ff6e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/4017515870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.665+0000 7faeaf7fe700 1 -- 172.21.15.62:0/4017515870 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58fcd2c00 con 0x55b58fdc66b0 2023-07-14T16:29:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.666+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/4017515870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff86310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faea0096520 tx=0x7faea000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc1f4f67e6b300d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.666+0000 7fae967fc700 1 -- 172.21.15.62:0/4017515870 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0099bb0 con 0x55b58fdc66b0 2023-07-14T16:29:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.666+0000 7fae967fc700 1 -- 172.21.15.62:0/4017515870 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faea00d5050 con 0x55b58fdc66b0 2023-07-14T16:29:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.666+0000 7fae967fc700 1 -- 172.21.15.62:0/4017515870 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea00a2290 con 0x55b58fdc66b0 2023-07-14T16:29:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 -- 172.21.15.62:0/4017515870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 msgr2=0x55b58ff86310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 --2- 172.21.15.62:0/4017515870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff86310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faea0096520 tx=0x7faea000c6e0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 -- 172.21.15.62:0/4017515870 shutdown_connections 2023-07-14T16:29:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 --2- 172.21.15.62:0/4017515870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff6e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 --2- 172.21.15.62:0/4017515870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff86310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 --2- 172.21.15.62:0/4017515870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff7ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 -- 172.21.15.62:0/4017515870 >> 172.21.15.62:0/4017515870 conn(0x55b58ff5fbe0 msgr2=0x55b58ff72560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 -- 172.21.15.62:0/4017515870 shutdown_connections 2023-07-14T16:29:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 -- 172.21.15.62:0/4017515870 wait complete. 2023-07-14T16:29:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 -- start start 2023-07-14T16:29:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.667+0000 7faec95b5b80 1 Processor -- start 2023-07-14T16:29:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff76f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faec95b5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff79ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faec95b5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ff7a420 con 0x55b58ff6b6e0 2023-07-14T16:29:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faec95b5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ff7a590 con 0x55b58ff6a840 2023-07-14T16:29:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faec95b5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58ff7a750 con 0x55b58fdc66b0 2023-07-14T16:29:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faec95b5b80 10 client.0 _ll_register_callbacks cb 0x55b58fe59700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faeaffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff79ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faeaffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff79ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53114/0 (socket says 172.21.15.62:53114) 2023-07-14T16:29:16.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faeaffff700 1 -- 172.21.15.62:0/2283528210 learned_addr learned my addr 172.21.15.62:0/2283528210 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.668+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff76f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faeaeffd700 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faeaffff700 1 -- 172.21.15.62:0/2283528210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 msgr2=0x55b58ff76f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faeaffff700 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff76f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faeaffff700 1 -- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 msgr2=0x55b58ff774b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faeaffff700 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff774b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faeaffff700 1 -- 172.21.15.62:0/2283528210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb010aa60 con 0x55b58ff6b6e0 2023-07-14T16:29:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faeaeffd700 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:29:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faeaffff700 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff79ee0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7faea8024400 tx=0x7faea8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4f903819ce1195 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7fae8effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7fae8effd700 1 -- 172.21.15.62:0/2283528210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8025440 con 0x55b58ff6b6e0 2023-07-14T16:29:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7faec95b5b80 1 -- 172.21.15.62:0/2283528210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faea401e040 con 0x55b58ff6b6e0 2023-07-14T16:29:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.669+0000 7fae8effd700 1 -- 172.21.15.62:0/2283528210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faea80255a0 con 0x55b58ff6b6e0 2023-07-14T16:29:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7fae8effd700 1 -- 172.21.15.62:0/2283528210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea80257a0 con 0x55b58ff6b6e0 2023-07-14T16:29:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7faec95b5b80 10 client.19395 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7faec95b5b80 1 -- 172.21.15.62:0/2283528210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b58fd101f0 con 0x55b58ff6b6e0 2023-07-14T16:29:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7fae8effd700 1 -- 172.21.15.62:0/2283528210 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faea401e040 con 0x55b58ff6b6e0 2023-07-14T16:29:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7fae8effd700 1 -- 172.21.15.62:0/2283528210 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faea802e990 con 0x55b58ff6b6e0 2023-07-14T16:29:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7faec95b5b80 10 client.19395 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7faec95b5b80 10 client.19395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7faec95b5b80 20 client.19395 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b58ffd4320 0x55b58ffd6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7faec95b5b80 4 client.19395 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:29:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.670+0000 7faec95b5b80 1 -- 172.21.15.62:0/2283528210 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b58fe76ff0 con 0x55b58ffd4320 2023-07-14T16:29:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.671+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b58ffd4320 0x55b58ffd6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.671+0000 7faeaf7fe700 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b58ffd4320 0x55b58ffd6730 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7553622198747 server_cookie=a6ebb3ca5d250ecc in_seq=0 out_seq=0 2023-07-14T16:29:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.671+0000 7fae8effd700 10 client.19395 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.680+0000 7fae8effd700 1 -- 172.21.15.62:0/2283528210 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b58fe76ff0 con 0x55b58ffd4320 2023-07-14T16:29:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.680+0000 7fae8effd700 10 client.19395 handle_command_reply: tid=0 2023-07-14T16:29:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.680+0000 7faec95b5b80 1 client.19395 shutdown 2023-07-14T16:29:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 -- 172.21.15.62:0/2283528210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 msgr2=0x55b58ff79ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff79ee0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7faea8024400 tx=0x7faea8004ad0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 -- 172.21.15.62:0/2283528210 shutdown_connections 2023-07-14T16:29:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b58ff6b6e0 0x55b58ff79ee0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b58fdc66b0 0x55b58ff76f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b58ffd4320 0x55b58ffd6730 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 --2- 172.21.15.62:0/2283528210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58ff6a840 0x55b58ff774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 -- 172.21.15.62:0/2283528210 >> 172.21.15.62:0/2283528210 conn(0x55b58ff5fbe0 msgr2=0x55b58ff72560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7fae8effd700 0 client.19395 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 -- 172.21.15.62:0/2283528210 shutdown_connections 2023-07-14T16:29:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 1 -- 172.21.15.62:0/2283528210 wait complete. 2023-07-14T16:29:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.681+0000 7faec95b5b80 20 client.19395 trim_cache size 0 max 16384 2023-07-14T16:29:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:29:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:29:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:29:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:29:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 -- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fae980a11d0 msgr2=0x7fae980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fae980a11d0 0x7fae980a3680 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faea8009fd0 tx=0x7faea8009040 comp rx=0 tx=0).stop 2023-07-14T16:29:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 -- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 msgr2=0x7faeb0197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 0x7faeb0197a60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7faea000e9a0 tx=0x7faea000ed60 comp rx=0 tx=0).stop 2023-07-14T16:29:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 -- 172.21.15.62:0/3980889729 shutdown_connections 2023-07-14T16:29:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fae980a11d0 0x7fae980a3680 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faeb01079c0 0x7faeb0198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeb0072f10 0x7faeb0197a60 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 --2- 172.21.15.62:0/3980889729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeb00744f0 0x7faeb0197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 -- 172.21.15.62:0/3980889729 >> 172.21.15.62:0/3980889729 conn(0x7faeb00fb130 msgr2=0x7faeb00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 -- 172.21.15.62:0/3980889729 shutdown_connections 2023-07-14T16:29:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:16.682+0000 7faeb5bff700 1 -- 172.21.15.62:0/3980889729 wait complete. 2023-07-14T16:29:17.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:29:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:29:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:29:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:29:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:29:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:29:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:29:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:16 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:29:17.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:29:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:29:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:29:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:29:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:29:17.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:29:17.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:16 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:29:17.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:29:17.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:29:17.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:29:17.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:29:17.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:29:17.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:29:17.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:17.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:16 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:19.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:18 smithi062 ceph-mon[95543]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:29:19.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:18 smithi081 ceph-mon[127795]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:29:19.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:18 smithi164 ceph-mon[97427]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:29:21.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:20 smithi062 ceph-mon[95543]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:29:21.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:20 smithi081 ceph-mon[127795]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:29:21.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:20 smithi164 ceph-mon[97427]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 33 op/s 2023-07-14T16:29:23.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:22 smithi062 ceph-mon[95543]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:29:23.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:22 smithi081 ceph-mon[127795]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:29:23.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:22 smithi164 ceph-mon[97427]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:29:25.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:24 smithi062 ceph-mon[95543]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:29:25.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:24 smithi081 ceph-mon[127795]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:29:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:24 smithi164 ceph-mon[97427]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 38 op/s 2023-07-14T16:29:27.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:26 smithi062 ceph-mon[95543]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:29:27.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:26 smithi081 ceph-mon[127795]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:29:27.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:26 smithi164 ceph-mon[97427]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:29:29.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:28 smithi062 ceph-mon[95543]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:29:29.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:29:29.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:29:29.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:29:29.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:28 smithi081 ceph-mon[127795]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:29:29.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:29:29.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:29:29.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:29:29.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:28 smithi164 ceph-mon[97427]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:29:29.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:29:29.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:29:29.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:29:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.885+0000 7f5aab59e700 1 -- 172.21.15.62:0/2804453114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 msgr2=0x7f5aac100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.885+0000 7f5aab59e700 1 --2- 172.21.15.62:0/2804453114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 0x7f5aac100f80 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009990 tx=0x7f5aa0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.886+0000 7f5aab59e700 1 -- 172.21.15.62:0/2804453114 shutdown_connections 2023-07-14T16:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.886+0000 7f5aab59e700 1 --2- 172.21.15.62:0/2804453114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aac1014c0 0x7f5aac107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.886+0000 7f5aab59e700 1 --2- 172.21.15.62:0/2804453114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aac0ff940 0x7f5aac0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.886+0000 7f5aab59e700 1 --2- 172.21.15.62:0/2804453114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 0x7f5aac100f80 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.886+0000 7f5aab59e700 1 -- 172.21.15.62:0/2804453114 >> 172.21.15.62:0/2804453114 conn(0x7f5aac0faf80 msgr2=0x7f5aac0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.886+0000 7f5aab59e700 1 -- 172.21.15.62:0/2804453114 shutdown_connections 2023-07-14T16:29:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.886+0000 7f5aab59e700 1 -- 172.21.15.62:0/2804453114 wait complete. 2023-07-14T16:29:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.887+0000 7f5aab59e700 1 Processor -- start 2023-07-14T16:29:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.887+0000 7f5aab59e700 1 -- start start 2023-07-14T16:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aab59e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aac0ff940 0x7f5aac0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aab59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 0x7f5aac072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aab59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aac1014c0 0x7f5aac072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aab59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0735e0 con 0x7f5aac100b30 2023-07-14T16:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aab59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac073750 con 0x7f5aac1014c0 2023-07-14T16:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aab59e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac19f610 con 0x7f5aac0ff940 2023-07-14T16:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aa9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 0x7f5aac072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aa9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 0x7f5aac072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39136/0 (socket says 172.21.15.62:39136) 2023-07-14T16:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/587602218 learned_addr learned my addr 172.21.15.62:0/587602218 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aac0ff940 0x7f5aac0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aac1014c0 0x7f5aac072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/587602218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aac0ff940 msgr2=0x7f5aac0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aac0ff940 0x7f5aac0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/587602218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aac1014c0 msgr2=0x7f5aac072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aac1014c0 0x7f5aac072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/587602218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00095f0 con 0x7f5aac100b30 2023-07-14T16:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.888+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aac1014c0 0x7f5aac072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.889+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 0x7f5aac072900 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0005910 tx=0x7f5aa0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.889+0000 7f5a9b7fe700 1 -- 172.21.15.62:0/587602218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa001d070 con 0x7f5aac100b30 2023-07-14T16:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.889+0000 7f5a9b7fe700 1 -- 172.21.15.62:0/587602218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aa000be90 con 0x7f5aac100b30 2023-07-14T16:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.889+0000 7f5a9b7fe700 1 -- 172.21.15.62:0/587602218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0022e60 con 0x7f5aac100b30 2023-07-14T16:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.889+0000 7f5aab59e700 1 -- 172.21.15.62:0/587602218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac19f890 con 0x7f5aac100b30 2023-07-14T16:29:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.889+0000 7f5aab59e700 1 -- 172.21.15.62:0/587602218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac19fc60 con 0x7f5aac100b30 2023-07-14T16:29:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.894+0000 7f5a9b7fe700 1 -- 172.21.15.62:0/587602218 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5aa0022600 con 0x7f5aac100b30 2023-07-14T16:29:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.895+0000 7f5ac494cb80 1 Processor -- start 2023-07-14T16:29:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5ac494cb80 1 -- start start 2023-07-14T16:29:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5a9b7fe700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a940a11d0 0x7f5a940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b263c3c50 con 0x555b2631a6b0 2023-07-14T16:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b264169e0 con 0x555b264bf7e0 2023-07-14T16:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b2640fe70 con 0x555b264bea90 2023-07-14T16:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5a9b7fe700 1 -- 172.21.15.62:0/587602218 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5aa00cdcc0 con 0x7f5aac100b30 2023-07-14T16:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.896+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a940a11d0 0x7f5a940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264be550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39138/0 (socket says 172.21.15.62:39138) 2023-07-14T16:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaa59c700 1 -- 172.21.15.62:0/2781349029 learned_addr learned my addr 172.21.15.62:0/2781349029 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaa59c700 1 -- 172.21.15.62:0/2781349029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 msgr2=0x555b264bf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264bf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaa59c700 1 -- 172.21.15.62:0/2781349029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 msgr2=0x555b264bfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264bfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaa59c700 1 -- 172.21.15.62:0/2781349029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b2623dd80 con 0x555b2631a6b0 2023-07-14T16:29:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264bfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:29:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264bf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.897+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a940a11d0 0x7f5a940a3680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009550 tx=0x7f5aa40094c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.898+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264be550 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009d30 tx=0x7f5a9c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485b60c22cee8ee5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.898+0000 7f5a98ff9700 1 -- 172.21.15.62:0/2781349029 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c004220 con 0x555b2631a6b0 2023-07-14T16:29:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.898+0000 7f5a98ff9700 1 -- 172.21.15.62:0/2781349029 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a9c004380 con 0x555b2631a6b0 2023-07-14T16:29:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.898+0000 7f5a98ff9700 1 -- 172.21.15.62:0/2781349029 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0197a0 con 0x555b2631a6b0 2023-07-14T16:29:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 -- 172.21.15.62:0/2781349029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 msgr2=0x555b264be550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264be550 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009d30 tx=0x7f5a9c011720 comp rx=0 tx=0).stop 2023-07-14T16:29:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 -- 172.21.15.62:0/2781349029 shutdown_connections 2023-07-14T16:29:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264bfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264be550 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/2781349029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264bf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 -- 172.21.15.62:0/2781349029 >> 172.21.15.62:0/2781349029 conn(0x555b264b3d40 msgr2=0x555b264b8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 -- 172.21.15.62:0/2781349029 shutdown_connections 2023-07-14T16:29:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 -- 172.21.15.62:0/2781349029 wait complete. 2023-07-14T16:29:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.899+0000 7f5ac494cb80 1 -- start start 2023-07-14T16:29:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.900+0000 7f5ac494cb80 1 Processor -- start 2023-07-14T16:29:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.900+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.900+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.900+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.900+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b263afa60 con 0x555b2631a6b0 2023-07-14T16:29:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.900+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b263af760 con 0x555b264bf7e0 2023-07-14T16:29:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.900+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b262938c0 con 0x555b264bea90 2023-07-14T16:29:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5ac494cb80 10 client.0 _ll_register_callbacks cb 0x555b263ad700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aaa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aa9d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aa9d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:36054/0 (socket says 172.21.15.62:36054) 2023-07-14T16:29:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/4139429030 learned_addr learned my addr 172.21.15.62:0/4139429030 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 msgr2=0x555b264cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/4139429030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 msgr2=0x555b264c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/4139429030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac19fc60 con 0x555b264bea90 2023-07-14T16:29:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.901+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264c7ef0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00cb7d0 tx=0x7f5aa009b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4981539e6f2811 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5a91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:29:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5a91ffb700 1 -- 172.21.15.62:0/4139429030 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00a32a0 con 0x555b264bea90 2023-07-14T16:29:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5a91ffb700 1 -- 172.21.15.62:0/4139429030 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5aa009b170 con 0x555b264bea90 2023-07-14T16:29:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5ac494cb80 1 -- 172.21.15.62:0/4139429030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b26369490 con 0x555b264bea90 2023-07-14T16:29:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5a91ffb700 1 -- 172.21.15.62:0/4139429030 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aa009f1a0 con 0x555b264bea90 2023-07-14T16:29:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5ac494cb80 10 client.28843 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5ac494cb80 1 -- 172.21.15.62:0/4139429030 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f5a9c009960 con 0x555b264bea90 2023-07-14T16:29:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5a91ffb700 1 -- 172.21.15.62:0/4139429030 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aa00b9030 con 0x555b264bea90 2023-07-14T16:29:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5a91ffb700 1 -- 172.21.15.62:0/4139429030 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5aa0096f60 con 0x555b264bea90 2023-07-14T16:29:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.902+0000 7f5ac494cb80 10 client.28843 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.903+0000 7f5ac494cb80 10 client.28843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.903+0000 7f5ac494cb80 20 client.28843 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.903+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b264d3400 0x555b264d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.903+0000 7f5ac494cb80 4 client.28843 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.903+0000 7f5ac494cb80 1 -- 172.21.15.62:0/4139429030 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b26408e50 con 0x555b264d3400 2023-07-14T16:29:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.903+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b264d3400 0x555b264d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.904+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b264d3400 0x555b264d5810 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e30d66897d7d11a server_cookie=60fc1ed90caba174 in_seq=0 out_seq=0 2023-07-14T16:29:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.904+0000 7f5a91ffb700 10 client.28843 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5a91ffb700 1 -- 172.21.15.62:0/4139429030 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555b26408e50 con 0x555b264d3400 2023-07-14T16:29:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5a91ffb700 10 client.28843 handle_command_reply: tid=0 2023-07-14T16:29:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 client.28843 shutdown 2023-07-14T16:29:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 -- 172.21.15.62:0/4139429030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 msgr2=0x555b264c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264c7ef0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00cb7d0 tx=0x7f5aa009b6b0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 -- 172.21.15.62:0/4139429030 shutdown_connections 2023-07-14T16:29:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b264d3400 0x555b264d5810 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5a91ffb700 0 client.28843 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/4139429030 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bea90 0x555b264c7ef0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 -- 172.21.15.62:0/4139429030 >> 172.21.15.62:0/4139429030 conn(0x555b264b3d40 msgr2=0x555b264b8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 -- 172.21.15.62:0/4139429030 shutdown_connections 2023-07-14T16:29:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.905+0000 7f5ac494cb80 1 -- 172.21.15.62:0/4139429030 wait complete. 2023-07-14T16:29:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.906+0000 7f5ac494cb80 20 client.28843 trim_cache size 0 max 16384 2023-07-14T16:29:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5ac494cb80 1 Processor -- start 2023-07-14T16:29:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5ac494cb80 1 -- start start 2023-07-14T16:29:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b2631a6b0 0x555b264da170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b264bea90 0x555b264ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b26408e50 con 0x555b264bea90 2023-07-14T16:29:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b264bf130 con 0x555b264bf7e0 2023-07-14T16:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b264bee60 con 0x555b2631a6b0 2023-07-14T16:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.927+0000 7f5aa9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b264bea90 0x555b264ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aa9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b264bea90 0x555b264ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39146/0 (socket says 172.21.15.62:39146) 2023-07-14T16:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/1347302493 learned_addr learned my addr 172.21.15.62:0/1347302493 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b2631a6b0 0x555b264da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/1347302493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b2631a6b0 msgr2=0x555b264da170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b2631a6b0 0x555b264da170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/1347302493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 msgr2=0x555b264c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aa9d9b700 1 -- 172.21.15.62:0/1347302493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b26226c00 con 0x555b264bea90 2023-07-14T16:29:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b2631a6b0 0x555b264da170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b264bea90 0x555b264ceef0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00cbb40 tx=0x7f5aa009abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3e8c24e1cee313 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.928+0000 7f5a997fa700 1 -- 172.21.15.62:0/1347302493 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa009b170 con 0x555b264bea90 2023-07-14T16:29:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5a997fa700 1 -- 172.21.15.62:0/1347302493 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aa00b9040 con 0x555b264bea90 2023-07-14T16:29:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5a997fa700 1 -- 172.21.15.62:0/1347302493 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00a32a0 con 0x555b264bea90 2023-07-14T16:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1347302493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b264bea90 msgr2=0x555b264ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b264bea90 0x555b264ceef0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00cbb40 tx=0x7f5aa009abb0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1347302493 shutdown_connections 2023-07-14T16:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bf7e0 0x555b264c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b2631a6b0 0x555b264da170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1347302493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b264bea90 0x555b264ceef0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1347302493 >> 172.21.15.62:0/1347302493 conn(0x555b264b3d40 msgr2=0x555b264c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1347302493 shutdown_connections 2023-07-14T16:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.929+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1347302493 wait complete. 2023-07-14T16:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.930+0000 7f5ac494cb80 1 -- start start 2023-07-14T16:29:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.930+0000 7f5ac494cb80 1 Processor -- start 2023-07-14T16:29:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bea90 0x555b264caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5ac494cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bf7e0 0x555b264cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b264ce400 con 0x555b2631a6b0 2023-07-14T16:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b264ce570 con 0x555b264bea90 2023-07-14T16:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5ac494cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b264ce730 con 0x555b264bf7e0 2023-07-14T16:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5ac494cb80 10 client.0 _ll_register_callbacks cb 0x7f5aa009af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39152/0 (socket says 172.21.15.62:39152) 2023-07-14T16:29:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaa59c700 1 -- 172.21.15.62:0/1586908800 learned_addr learned my addr 172.21.15.62:0/1586908800 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bea90 0x555b264caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaad9d700 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bf7e0 0x555b264cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaa59c700 1 -- 172.21.15.62:0/1586908800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bf7e0 msgr2=0x555b264cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bf7e0 0x555b264cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaa59c700 1 -- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bea90 msgr2=0x555b264caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bea90 0x555b264caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aaa59c700 1 -- 172.21.15.62:0/1586908800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c009960 con 0x555b2631a6b0 2023-07-14T16:29:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.931+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bea90 0x555b264caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.932+0000 7f5aaa59c700 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264ca960 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009d30 tx=0x7f5a9c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4321c9b031faa855 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.932+0000 7f5a91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.932+0000 7f5a91ffb700 1 -- 172.21.15.62:0/1586908800 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c003680 con 0x555b2631a6b0 2023-07-14T16:29:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.932+0000 7f5a91ffb700 1 -- 172.21.15.62:0/1586908800 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5a9c029040 con 0x555b2631a6b0 2023-07-14T16:29:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.932+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1586908800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a9c004380 con 0x555b2631a6b0 2023-07-14T16:29:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.932+0000 7f5a91ffb700 1 -- 172.21.15.62:0/1586908800 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a9c0037e0 con 0x555b2631a6b0 2023-07-14T16:29:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.932+0000 7f5ac494cb80 10 client.19413 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.932+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1586908800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555b262641f0 con 0x555b2631a6b0 2023-07-14T16:29:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5a91ffb700 1 -- 172.21.15.62:0/1586908800 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a9c004380 con 0x555b2631a6b0 2023-07-14T16:29:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5a91ffb700 1 -- 172.21.15.62:0/1586908800 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a9c011440 con 0x555b2631a6b0 2023-07-14T16:29:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5ac494cb80 10 client.19413 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5ac494cb80 10 client.19413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5ac494cb80 20 client.19413 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b26527ed0 0x555b2652a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5ac494cb80 4 client.19413 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:29:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1586908800 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555b263caff0 con 0x555b26527ed0 2023-07-14T16:29:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.933+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b26527ed0 0x555b2652a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.934+0000 7f5aa9d9b700 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b26527ed0 0x555b2652a2e0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fccd960747370a1e server_cookie=31b151b99a144264 in_seq=0 out_seq=0 2023-07-14T16:29:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.934+0000 7f5a91ffb700 10 client.19413 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5a91ffb700 1 -- 172.21.15.62:0/1586908800 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555b263caff0 con 0x555b26527ed0 2023-07-14T16:29:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5a91ffb700 10 client.19413 handle_command_reply: tid=0 2023-07-14T16:29:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5ac494cb80 1 client.19413 shutdown 2023-07-14T16:29:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1586908800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 msgr2=0x555b264ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264ca960 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009d30 tx=0x7f5a9c004310 comp rx=0 tx=0).stop 2023-07-14T16:29:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1586908800 shutdown_connections 2023-07-14T16:29:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555b264bf7e0 0x555b264cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b2631a6b0 0x555b264ca960 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555b26527ed0 0x555b2652a2e0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.944+0000 7f5ac494cb80 1 --2- 172.21.15.62:0/1586908800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555b264bea90 0x555b264caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.945+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1586908800 >> 172.21.15.62:0/1586908800 conn(0x555b264b3d40 msgr2=0x555b264c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.945+0000 7f5a91ffb700 0 client.19413 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.945+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1586908800 shutdown_connections 2023-07-14T16:29:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.945+0000 7f5ac494cb80 1 -- 172.21.15.62:0/1586908800 wait complete. 2023-07-14T16:29:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.945+0000 7f5ac494cb80 20 client.19413 trim_cache size 0 max 16384 2023-07-14T16:29:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:29:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:29:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:29:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:29:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 -- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a940a11d0 msgr2=0x7f5a940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a940a11d0 0x7f5a940a3680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009550 tx=0x7f5aa40094c0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 -- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 msgr2=0x7f5aac072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 0x7f5aac072900 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0005910 tx=0x7f5aa0005720 comp rx=0 tx=0).stop 2023-07-14T16:29:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 -- 172.21.15.62:0/587602218 shutdown_connections 2023-07-14T16:29:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5a940a11d0 0x7f5a940a3680 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5aac1014c0 0x7f5aac072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aac0ff940 0x7f5aac0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 --2- 172.21.15.62:0/587602218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aac100b30 0x7f5aac072900 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.946+0000 7f5aab59e700 1 -- 172.21.15.62:0/587602218 >> 172.21.15.62:0/587602218 conn(0x7f5aac0faf80 msgr2=0x7f5aac0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.947+0000 7f5aab59e700 1 -- 172.21.15.62:0/587602218 shutdown_connections 2023-07-14T16:29:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:29.947+0000 7f5aab59e700 1 -- 172.21.15.62:0/587602218 wait complete. 2023-07-14T16:29:31.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:30 smithi062 ceph-mon[95543]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:29:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:30 smithi081 ceph-mon[127795]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:29:31.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:30 smithi164 ceph-mon[97427]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:29:33.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:32 smithi062 ceph-mon[95543]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:29:33.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:32 smithi081 ceph-mon[127795]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:29:33.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:32 smithi164 ceph-mon[97427]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 17 op/s 2023-07-14T16:29:35.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:34 smithi062 ceph-mon[95543]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:29:35.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:34 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:29:35.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:34 smithi081 ceph-mon[127795]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:29:35.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:34 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:29:35.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:34 smithi164 ceph-mon[97427]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 16 MiB/s wr, 15 op/s 2023-07-14T16:29:35.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:34 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:29:37.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:36 smithi062 ceph-mon[95543]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:29:37.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:29:37.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:29:37.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:29:37.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:36 smithi081 ceph-mon[127795]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:29:37.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:29:37.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:29:37.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:29:37.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:36 smithi164 ceph-mon[97427]: pgmap v776: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:29:37.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:29:37.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:29:37.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:29:39.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:38 smithi062 ceph-mon[95543]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:29:39.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:38 smithi081 ceph-mon[127795]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:29:39.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:38 smithi164 ceph-mon[97427]: pgmap v777: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:29:41.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:40 smithi062 ceph-mon[95543]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:29:41.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:40 smithi081 ceph-mon[127795]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:29:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:40 smithi164 ceph-mon[97427]: pgmap v778: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 16 op/s 2023-07-14T16:29:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.123+0000 7f043c7ff700 1 -- 172.21.15.62:0/3677263025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 msgr2=0x7f04340fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.123+0000 7f043c7ff700 1 --2- 172.21.15.62:0/3677263025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 0x7f04340fff10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0430009990 tx=0x7f0430009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.123+0000 7f043c7ff700 1 -- 172.21.15.62:0/3677263025 shutdown_connections 2023-07-14T16:29:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.123+0000 7f043c7ff700 1 --2- 172.21.15.62:0/3677263025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0434101680 0x7f0434107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.123+0000 7f043c7ff700 1 --2- 172.21.15.62:0/3677263025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 0x7f04340fff10 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.123+0000 7f043c7ff700 1 --2- 172.21.15.62:0/3677263025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0434100cf0 0x7f0434101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.123+0000 7f043c7ff700 1 -- 172.21.15.62:0/3677263025 >> 172.21.15.62:0/3677263025 conn(0x7f04340fb0e0 msgr2=0x7f04340fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.124+0000 7f043c7ff700 1 -- 172.21.15.62:0/3677263025 shutdown_connections 2023-07-14T16:29:43.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.124+0000 7f043c7ff700 1 -- 172.21.15.62:0/3677263025 wait complete. 2023-07-14T16:29:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043c7ff700 1 Processor -- start 2023-07-14T16:29:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043c7ff700 1 -- start start 2023-07-14T16:29:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043c7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 0x7f0434197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043c7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0434100cf0 0x7f0434197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043c7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0434101680 0x7f0434198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043c7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0434199490 con 0x7f04340ffb00 2023-07-14T16:29:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043c7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0434199600 con 0x7f0434101680 2023-07-14T16:29:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043c7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04341997c0 con 0x7f0434100cf0 2023-07-14T16:29:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043a59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 0x7f0434197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043a59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 0x7f0434197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46268/0 (socket says 172.21.15.62:46268) 2023-07-14T16:29:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.125+0000 7f043a59b700 1 -- 172.21.15.62:0/2769817715 learned_addr learned my addr 172.21.15.62:0/2769817715 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f0439d9a700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0434100cf0 0x7f0434197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f043ad9c700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0434101680 0x7f0434198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f043a59b700 1 -- 172.21.15.62:0/2769817715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0434100cf0 msgr2=0x7f0434197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f043a59b700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0434100cf0 0x7f0434197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f043a59b700 1 -- 172.21.15.62:0/2769817715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0434101680 msgr2=0x7f0434198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f043a59b700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0434101680 0x7f0434198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f043a59b700 1 -- 172.21.15.62:0/2769817715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300095f0 con 0x7f04340ffb00 2023-07-14T16:29:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f043ad9c700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0434101680 0x7f0434198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f043a59b700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 0x7f0434197a70 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f04300098e0 tx=0x7f0430004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.126+0000 7f042b7fe700 1 -- 172.21.15.62:0/2769817715 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043001d070 con 0x7f04340ffb00 2023-07-14T16:29:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.127+0000 7f043c7ff700 1 -- 172.21.15.62:0/2769817715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0434199a40 con 0x7f04340ffb00 2023-07-14T16:29:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.127+0000 7f042b7fe700 1 -- 172.21.15.62:0/2769817715 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f043000bdd0 con 0x7f04340ffb00 2023-07-14T16:29:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.127+0000 7f042b7fe700 1 -- 172.21.15.62:0/2769817715 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430022d70 con 0x7f04340ffb00 2023-07-14T16:29:43.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.127+0000 7f043c7ff700 1 -- 172.21.15.62:0/2769817715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04341a0260 con 0x7f04340ffb00 2023-07-14T16:29:43.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.130+0000 7f042b7fe700 1 -- 172.21.15.62:0/2769817715 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0430022470 con 0x7f04340ffb00 2023-07-14T16:29:43.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.132+0000 7f042b7fe700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04200a1150 0x7f04200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.132+0000 7f042b7fe700 1 -- 172.21.15.62:0/2769817715 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f04300cdea0 con 0x7f04340ffb00 2023-07-14T16:29:43.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.132+0000 7f04500a3b80 1 Processor -- start 2023-07-14T16:29:43.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.132+0000 7f0439d9a700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04200a1150 0x7f04200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.132+0000 7f04500a3b80 1 -- start start 2023-07-14T16:29:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319d840 0x55fcd319e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319e6e0 0x55fcd319eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f04500a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd30a2c50 con 0x55fcd319e6e0 2023-07-14T16:29:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f04500a3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd30f59e0 con 0x55fcd319d840 2023-07-14T16:29:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f04500a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd30eee70 con 0x55fcd2ff96b0 2023-07-14T16:29:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f0439d9a700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04200a1150 0x7f04200a3600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0424009c60 tx=0x7f0424009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f043ad9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319e6e0 0x55fcd319eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f0439d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319d840 0x55fcd319e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f0439d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319d840 0x55fcd319e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35916/0 (socket says 172.21.15.62:35916) 2023-07-14T16:29:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f0439d9a700 1 -- 172.21.15.62:0/3478421299 learned_addr learned my addr 172.21.15.62:0/3478421299 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.133+0000 7f043a59b700 1 --2- 172.21.15.62:0/3478421299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.134+0000 7f0439d9a700 1 -- 172.21.15.62:0/3478421299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 msgr2=0x55fcd319d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.134+0000 7f0439d9a700 1 --2- 172.21.15.62:0/3478421299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.134+0000 7f0439d9a700 1 -- 172.21.15.62:0/3478421299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319e6e0 msgr2=0x55fcd319eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.134+0000 7f0439d9a700 1 --2- 172.21.15.62:0/3478421299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319e6e0 0x55fcd319eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.134+0000 7f0439d9a700 1 -- 172.21.15.62:0/3478421299 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd2f1cd80 con 0x55fcd319d840 2023-07-14T16:29:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.134+0000 7f0439d9a700 1 --2- 172.21.15.62:0/3478421299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319d840 0x55fcd319e1a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f042400e430 tx=0x7f0424014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edadd73ea98ce05 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.134+0000 7f0428ff9700 1 -- 172.21.15.62:0/3478421299 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042400edd0 con 0x55fcd319d840 2023-07-14T16:29:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f0428ff9700 1 -- 172.21.15.62:0/3478421299 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0424021070 con 0x55fcd319d840 2023-07-14T16:29:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f0428ff9700 1 -- 172.21.15.62:0/3478421299 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424025610 con 0x55fcd319d840 2023-07-14T16:29:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f04500a3b80 1 -- 172.21.15.62:0/3478421299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319d840 msgr2=0x55fcd319e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f04500a3b80 1 --2- 172.21.15.62:0/3478421299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319d840 0x55fcd319e1a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f042400e430 tx=0x7f0424014000 comp rx=0 tx=0).stop 2023-07-14T16:29:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f04500a3b80 1 -- 172.21.15.62:0/3478421299 shutdown_connections 2023-07-14T16:29:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f04500a3b80 1 --2- 172.21.15.62:0/3478421299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319e6e0 0x55fcd319eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f04500a3b80 1 --2- 172.21.15.62:0/3478421299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f04500a3b80 1 --2- 172.21.15.62:0/3478421299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319d840 0x55fcd319e1a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.135+0000 7f04500a3b80 1 -- 172.21.15.62:0/3478421299 >> 172.21.15.62:0/3478421299 conn(0x55fcd3192be0 msgr2=0x55fcd3197cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.136+0000 7f04500a3b80 1 -- 172.21.15.62:0/3478421299 shutdown_connections 2023-07-14T16:29:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.136+0000 7f04500a3b80 1 -- 172.21.15.62:0/3478421299 wait complete. 2023-07-14T16:29:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.136+0000 7f04500a3b80 1 -- start start 2023-07-14T16:29:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.136+0000 7f04500a3b80 1 Processor -- start 2023-07-14T16:29:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd31a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f04500a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd308ea60 con 0x55fcd319d840 2023-07-14T16:29:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f04500a3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd308e760 con 0x55fcd319e6e0 2023-07-14T16:29:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f04500a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd2f728c0 con 0x55fcd2ff96b0 2023-07-14T16:29:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f04500a3b80 10 client.0 _ll_register_callbacks cb 0x55fcd308c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f0439d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f043ad9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f0439d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46272/0 (socket says 172.21.15.62:46272) 2023-07-14T16:29:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.137+0000 7f0439d9a700 1 -- 172.21.15.62:0/2253359900 learned_addr learned my addr 172.21.15.62:0/2253359900 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f043a59b700 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd31a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f043ad9c700 1 -- 172.21.15.62:0/2253359900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 msgr2=0x55fcd31a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f043ad9c700 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd31a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f043ad9c700 1 -- 172.21.15.62:0/2253359900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 msgr2=0x55fcd31a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f043ad9c700 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f043ad9c700 1 -- 172.21.15.62:0/2253359900 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0424017070 con 0x55fcd319e6e0 2023-07-14T16:29:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f0439d9a700 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f043ad9c700 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31add60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f042c011d30 tx=0x7f042c00f4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dfebb57e58499e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.138+0000 7f041dffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:29:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.139+0000 7f041dffb700 1 -- 172.21.15.62:0/2253359900 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c004bb0 con 0x55fcd319e6e0 2023-07-14T16:29:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.139+0000 7f04500a3b80 1 -- 172.21.15.62:0/2253359900 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fcd3048490 con 0x55fcd319e6e0 2023-07-14T16:29:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.139+0000 7f041dffb700 1 -- 172.21.15.62:0/2253359900 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f042c004d10 con 0x55fcd319e6e0 2023-07-14T16:29:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.139+0000 7f041dffb700 1 -- 172.21.15.62:0/2253359900 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f042c003a00 con 0x55fcd319e6e0 2023-07-14T16:29:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.139+0000 7f04500a3b80 10 client.28985 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.139+0000 7f04500a3b80 1 -- 172.21.15.62:0/2253359900 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fcd2f05c00 con 0x55fcd319e6e0 2023-07-14T16:29:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.139+0000 7f041dffb700 1 -- 172.21.15.62:0/2253359900 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f042c024020 con 0x55fcd319e6e0 2023-07-14T16:29:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.140+0000 7f041dffb700 1 -- 172.21.15.62:0/2253359900 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f042c004470 con 0x55fcd319e6e0 2023-07-14T16:29:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.140+0000 7f04500a3b80 10 client.28985 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.140+0000 7f04500a3b80 10 client.28985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.140+0000 7f04500a3b80 20 client.28985 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.140+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fcd31b26c0 0x55fcd31b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.140+0000 7f04500a3b80 4 client.28985 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.140+0000 7f04500a3b80 1 -- 172.21.15.62:0/2253359900 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcd30e7e50 con 0x55fcd31b26c0 2023-07-14T16:29:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.140+0000 7f043a59b700 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fcd31b26c0 0x55fcd31b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.141+0000 7f041dffb700 10 client.28985 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.141+0000 7f043a59b700 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fcd31b26c0 0x55fcd31b4ad0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f722637408f3d84 server_cookie=46a3d2bd4032f0b0 in_seq=0 out_seq=0 2023-07-14T16:29:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.141+0000 7f041dffb700 1 -- 172.21.15.62:0/2253359900 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fcd30e7e50 con 0x55fcd31b26c0 2023-07-14T16:29:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f041dffb700 10 client.28985 handle_command_reply: tid=0 2023-07-14T16:29:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 client.28985 shutdown 2023-07-14T16:29:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 -- 172.21.15.62:0/2253359900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 msgr2=0x55fcd31add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31add60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f042c011d30 tx=0x7f042c00f4b0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 -- 172.21.15.62:0/2253359900 shutdown_connections 2023-07-14T16:29:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fcd31b26c0 0x55fcd31b4ad0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31add60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f041dffb700 0 client.28985 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd31a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2253359900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 -- 172.21.15.62:0/2253359900 >> 172.21.15.62:0/2253359900 conn(0x55fcd3192be0 msgr2=0x55fcd31974e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 -- 172.21.15.62:0/2253359900 shutdown_connections 2023-07-14T16:29:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 1 -- 172.21.15.62:0/2253359900 wait complete. 2023-07-14T16:29:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.142+0000 7f04500a3b80 20 client.28985 trim_cache size 0 max 16384 2023-07-14T16:29:43.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:42 smithi062 ceph-mon[95543]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:29:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.158+0000 7f04500a3b80 1 Processor -- start 2023-07-14T16:29:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.158+0000 7f04500a3b80 1 -- start start 2023-07-14T16:29:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.158+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31b9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f04500a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd31997a0 con 0x55fcd319d840 2023-07-14T16:29:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f04500a3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd319dc10 con 0x55fcd319e6e0 2023-07-14T16:29:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f04500a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd31a4850 con 0x55fcd2ff96b0 2023-07-14T16:29:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f0439d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043a59b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043a59b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54460/0 (socket says 172.21.15.62:54460) 2023-07-14T16:29:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043a59b700 1 -- 172.21.15.62:0/1990262120 learned_addr learned my addr 172.21.15.62:0/1990262120 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043ad9c700 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043a59b700 1 -- 172.21.15.62:0/1990262120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 msgr2=0x55fcd31b9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043a59b700 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31b9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043a59b700 1 -- 172.21.15.62:0/1990262120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 msgr2=0x55fcd31add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043a59b700 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043a59b700 1 -- 172.21.15.62:0/1990262120 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd2f431f0 con 0x55fcd2ff96b0 2023-07-14T16:29:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.159+0000 7f043ad9c700 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31b9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.160+0000 7f043a59b700 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f043009b2e0 tx=0x7f043001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c835d57f6bd35336 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.160+0000 7f04297fa700 1 -- 172.21.15.62:0/1990262120 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043009f380 con 0x55fcd2ff96b0 2023-07-14T16:29:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.160+0000 7f04297fa700 1 -- 172.21.15.62:0/1990262120 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04300a3710 con 0x55fcd2ff96b0 2023-07-14T16:29:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.160+0000 7f04297fa700 1 -- 172.21.15.62:0/1990262120 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043009b810 con 0x55fcd2ff96b0 2023-07-14T16:29:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 -- 172.21.15.62:0/1990262120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 msgr2=0x55fcd319d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f043009b2e0 tx=0x7f043001a040 comp rx=0 tx=0).stop 2023-07-14T16:29:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 -- 172.21.15.62:0/1990262120 shutdown_connections 2023-07-14T16:29:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd31b9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd2ff96b0 0x55fcd319d2c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 --2- 172.21.15.62:0/1990262120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd319d840 0x55fcd31add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 -- 172.21.15.62:0/1990262120 >> 172.21.15.62:0/1990262120 conn(0x55fcd3192be0 msgr2=0x55fcd31a5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 -- 172.21.15.62:0/1990262120 shutdown_connections 2023-07-14T16:29:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 -- 172.21.15.62:0/1990262120 wait complete. 2023-07-14T16:29:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.161+0000 7f04500a3b80 1 -- start start 2023-07-14T16:29:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f04500a3b80 1 Processor -- start 2023-07-14T16:29:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd2ff96b0 0x55fcd31abd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd319d840 0x55fcd3202550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f04500a3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd3204aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f04500a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd32051b0 con 0x55fcd2ff96b0 2023-07-14T16:29:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f04500a3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd3205320 con 0x55fcd319e6e0 2023-07-14T16:29:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f04500a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd32054e0 con 0x55fcd319d840 2023-07-14T16:29:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f04500a3b80 10 client.0 _ll_register_callbacks cb 0x7f042c0115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f043a59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd2ff96b0 0x55fcd31abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f043a59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd2ff96b0 0x55fcd31abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46280/0 (socket says 172.21.15.62:46280) 2023-07-14T16:29:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.162+0000 7f043a59b700 1 -- 172.21.15.62:0/2862494244 learned_addr learned my addr 172.21.15.62:0/2862494244 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f0439d9a700 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd319d840 0x55fcd3202550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f043ad9c700 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd3204aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f043a59b700 1 -- 172.21.15.62:0/2862494244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd319d840 msgr2=0x55fcd3202550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f043a59b700 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd319d840 0x55fcd3202550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f043a59b700 1 -- 172.21.15.62:0/2862494244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 msgr2=0x55fcd3204aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f043a59b700 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd3204aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f043a59b700 1 -- 172.21.15.62:0/2862494244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04341a0260 con 0x55fcd2ff96b0 2023-07-14T16:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f0439d9a700 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd319d840 0x55fcd3202550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f043a59b700 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd2ff96b0 0x55fcd31abd10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f04300a1080 tx=0x7f0430009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd323ee806e2ce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f041dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f041dffb700 1 -- 172.21.15.62:0/2862494244 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430099550 con 0x55fcd2ff96b0 2023-07-14T16:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f04500a3b80 1 -- 172.21.15.62:0/2862494244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0424021070 con 0x55fcd2ff96b0 2023-07-14T16:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.163+0000 7f041dffb700 1 -- 172.21.15.62:0/2862494244 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f043009b5b0 con 0x55fcd2ff96b0 2023-07-14T16:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f041dffb700 1 -- 172.21.15.62:0/2862494244 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043009b770 con 0x55fcd2ff96b0 2023-07-14T16:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f04500a3b80 10 client.19431 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f04500a3b80 1 -- 172.21.15.62:0/2862494244 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fcd2f1b250 con 0x55fcd2ff96b0 2023-07-14T16:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f041dffb700 1 -- 172.21.15.62:0/2862494244 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0424021070 con 0x55fcd2ff96b0 2023-07-14T16:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f041dffb700 1 -- 172.21.15.62:0/2862494244 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04300c5af0 con 0x55fcd2ff96b0 2023-07-14T16:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f04500a3b80 10 client.19431 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f04500a3b80 10 client.19431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f04500a3b80 20 client.19431 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fcd3206a80 0x55fcd3208e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f04500a3b80 4 client.19431 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.164+0000 7f04500a3b80 1 -- 172.21.15.62:0/2862494244 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fcd30a9ff0 con 0x55fcd3206a80 2023-07-14T16:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.165+0000 7f0439d9a700 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fcd3206a80 0x55fcd3208e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.165+0000 7f041dffb700 10 client.19431 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.165+0000 7f0439d9a700 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fcd3206a80 0x55fcd3208e60 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6adc5c3ba7245d3 server_cookie=2e5f33909ee88c22 in_seq=0 out_seq=0 2023-07-14T16:29:43.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:42 smithi081 ceph-mon[127795]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f041dffb700 1 -- 172.21.15.62:0/2862494244 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fcd30a9ff0 con 0x55fcd3206a80 2023-07-14T16:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f041dffb700 10 client.19431 handle_command_reply: tid=0 2023-07-14T16:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 client.19431 shutdown 2023-07-14T16:29:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 -- 172.21.15.62:0/2862494244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd2ff96b0 msgr2=0x55fcd31abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd2ff96b0 0x55fcd31abd10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f04300a1080 tx=0x7f0430009310 comp rx=0 tx=0).stop 2023-07-14T16:29:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 -- 172.21.15.62:0/2862494244 shutdown_connections 2023-07-14T16:29:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fcd3206a80 0x55fcd3208e60 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fcd319e6e0 0x55fcd3204aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fcd2ff96b0 0x55fcd31abd10 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 --2- 172.21.15.62:0/2862494244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fcd319d840 0x55fcd3202550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f041dffb700 0 client.19431 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 -- 172.21.15.62:0/2862494244 >> 172.21.15.62:0/2862494244 conn(0x55fcd3192be0 msgr2=0x55fcd31b9430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 -- 172.21.15.62:0/2862494244 shutdown_connections 2023-07-14T16:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 1 -- 172.21.15.62:0/2862494244 wait complete. 2023-07-14T16:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.176+0000 7f04500a3b80 20 client.19431 trim_cache size 0 max 16384 2023-07-14T16:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:29:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 -- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04200a1150 msgr2=0x7f04200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04200a1150 0x7f04200a3600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0424009c60 tx=0x7f0424009260 comp rx=0 tx=0).stop 2023-07-14T16:29:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 -- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 msgr2=0x7f0434197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 0x7f0434197a70 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f04300098e0 tx=0x7f0430004730 comp rx=0 tx=0).stop 2023-07-14T16:29:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 -- 172.21.15.62:0/2769817715 shutdown_connections 2023-07-14T16:29:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04200a1150 0x7f04200a3600 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0434101680 0x7f0434198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04340ffb00 0x7f0434197a70 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 --2- 172.21.15.62:0/2769817715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0434100cf0 0x7f0434197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 -- 172.21.15.62:0/2769817715 >> 172.21.15.62:0/2769817715 conn(0x7f04340fb0e0 msgr2=0x7f04340fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 -- 172.21.15.62:0/2769817715 shutdown_connections 2023-07-14T16:29:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:43.177+0000 7f043c7ff700 1 -- 172.21.15.62:0/2769817715 wait complete. 2023-07-14T16:29:43.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:42 smithi164 ceph-mon[97427]: pgmap v779: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:29:45.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:44 smithi062 ceph-mon[95543]: pgmap v780: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:29:45.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:44 smithi081 ceph-mon[127795]: pgmap v780: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:29:45.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:44 smithi164 ceph-mon[97427]: pgmap v780: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:29:46.295 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:29:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.620+0000 7ff31499d700 1 -- 172.21.15.62:0/4202525003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 msgr2=0x7ff310107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.620+0000 7ff31499d700 1 --2- 172.21.15.62:0/4202525003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 0x7ff310107af0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b0a0 tx=0x7ff30800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.621+0000 7ff31499d700 1 -- 172.21.15.62:0/4202525003 shutdown_connections 2023-07-14T16:29:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.621+0000 7ff31499d700 1 --2- 172.21.15.62:0/4202525003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 0x7ff310107af0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.621+0000 7ff31499d700 1 --2- 172.21.15.62:0/4202525003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff310100450 0x7ff310102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.621+0000 7ff31499d700 1 --2- 172.21.15.62:0/4202525003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff310102d70 0x7ff310105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.621+0000 7ff31499d700 1 -- 172.21.15.62:0/4202525003 >> 172.21.15.62:0/4202525003 conn(0x7ff3100f9ed0 msgr2=0x7ff3100fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.621+0000 7ff31499d700 1 -- 172.21.15.62:0/4202525003 shutdown_connections 2023-07-14T16:29:46.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.621+0000 7ff31499d700 1 -- 172.21.15.62:0/4202525003 wait complete. 2023-07-14T16:29:46.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.622+0000 7ff31499d700 1 Processor -- start 2023-07-14T16:29:46.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.622+0000 7ff31499d700 1 -- start start 2023-07-14T16:29:46.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.622+0000 7ff31499d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff310100450 0x7ff310197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.622+0000 7ff31499d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff310102d70 0x7ff310197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff31499d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 0x7ff31019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff31499d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31019f2e0 con 0x7ff310105690 2023-07-14T16:29:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff31499d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31019f420 con 0x7ff310102d70 2023-07-14T16:29:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff31499d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31019f5b0 con 0x7ff310100450 2023-07-14T16:29:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 0x7ff31019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 0x7ff31019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46290/0 (socket says 172.21.15.62:46290) 2023-07-14T16:29:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30ffff700 1 -- 172.21.15.62:0/3018452451 learned_addr learned my addr 172.21.15.62:0/3018452451 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:46.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff310100450 0x7ff310197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30effd700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff310102d70 0x7ff310197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30ffff700 1 -- 172.21.15.62:0/3018452451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff310100450 msgr2=0x7ff310197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30ffff700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff310100450 0x7ff310197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30ffff700 1 -- 172.21.15.62:0/3018452451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff310102d70 msgr2=0x7ff310197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30ffff700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff310102d70 0x7ff310197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.623+0000 7ff30ffff700 1 -- 172.21.15.62:0/3018452451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff308009d00 con 0x7ff310105690 2023-07-14T16:29:46.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.624+0000 7ff30ffff700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 0x7ff31019ed10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ff30800d010 tx=0x7ff30800bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:46.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.624+0000 7ff30cff9700 1 -- 172.21.15.62:0/3018452451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30800e040 con 0x7ff310105690 2023-07-14T16:29:46.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.624+0000 7ff31499d700 1 -- 172.21.15.62:0/3018452451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31019f830 con 0x7ff310105690 2023-07-14T16:29:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.624+0000 7ff30cff9700 1 -- 172.21.15.62:0/3018452451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff308009480 con 0x7ff310105690 2023-07-14T16:29:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.624+0000 7ff30cff9700 1 -- 172.21.15.62:0/3018452451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080052a0 con 0x7ff310105690 2023-07-14T16:29:46.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.624+0000 7ff31499d700 1 -- 172.21.15.62:0/3018452451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31019fc00 con 0x7ff310105690 2023-07-14T16:29:46.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.629+0000 7ff30cff9700 1 -- 172.21.15.62:0/3018452451 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff308005400 con 0x7ff310105690 2023-07-14T16:29:46.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.630+0000 7ff3295a7b80 1 Processor -- start 2023-07-14T16:29:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.630+0000 7ff3295a7b80 1 -- start start 2023-07-14T16:29:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.630+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c3589db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.630+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589e070 0x558c3589e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.630+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589ee30 0x558c3589f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.630+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c357a2cb0 con 0x558c3589d720 2023-07-14T16:29:46.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.630+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c357f5a40 con 0x558c3589ee30 2023-07-14T16:29:46.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.630+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c357eeed0 con 0x558c3589e070 2023-07-14T16:29:46.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c3589db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c3589db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46294/0 (socket says 172.21.15.62:46294) 2023-07-14T16:29:46.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 -- 172.21.15.62:0/3259090550 learned_addr learned my addr 172.21.15.62:0/3259090550 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:46.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589e070 0x558c3589e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589ee30 0x558c3589f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 -- 172.21.15.62:0/3259090550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589e070 msgr2=0x558c3589e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589e070 0x558c3589e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 -- 172.21.15.62:0/3259090550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589ee30 msgr2=0x558c3589f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589ee30 0x558c3589f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.631+0000 7ff30effd700 1 -- 172.21.15.62:0/3259090550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c3561cde0 con 0x558c3589d720 2023-07-14T16:29:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.632+0000 7ff30effd700 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c3589db30 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff304013630 tx=0x7ff304009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48019c31a39f7e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:46.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.632+0000 7ff30cff9700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2f80a11f0 0x7ff2f80a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.632+0000 7ff30cff9700 1 -- 172.21.15.62:0/3018452451 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3080cdcd0 con 0x7ff310105690 2023-07-14T16:29:46.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.632+0000 7ff2f5ffb700 1 -- 172.21.15.62:0/3259090550 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304011470 con 0x558c3589d720 2023-07-14T16:29:46.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.632+0000 7ff2f5ffb700 1 -- 172.21.15.62:0/3259090550 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff304011a60 con 0x558c3589d720 2023-07-14T16:29:46.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.632+0000 7ff2f5ffb700 1 -- 172.21.15.62:0/3259090550 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400f880 con 0x558c3589d720 2023-07-14T16:29:46.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.632+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2f80a11f0 0x7ff2f80a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 -- 172.21.15.62:0/3259090550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 msgr2=0x558c3589db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2f80a11f0 0x7ff2f80a36a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff300009e30 tx=0x7ff300009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:46.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c3589db30 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff304013630 tx=0x7ff304009870 comp rx=0 tx=0).stop 2023-07-14T16:29:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 -- 172.21.15.62:0/3259090550 shutdown_connections 2023-07-14T16:29:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589ee30 0x558c3589f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c3589db30 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/3259090550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589e070 0x558c3589e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 -- 172.21.15.62:0/3259090550 >> 172.21.15.62:0/3259090550 conn(0x558c35892ff0 msgr2=0x558c35898460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:46.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 -- 172.21.15.62:0/3259090550 shutdown_connections 2023-07-14T16:29:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.633+0000 7ff3295a7b80 1 -- 172.21.15.62:0/3259090550 wait complete. 2023-07-14T16:29:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 1 -- start start 2023-07-14T16:29:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 1 Processor -- start 2023-07-14T16:29:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 0x558c358a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3578eac0 con 0x558c3589d720 2023-07-14T16:29:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3578e7c0 con 0x558c3589e070 2023-07-14T16:29:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c35672920 con 0x558c3589ee30 2023-07-14T16:29:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.634+0000 7ff3295a7b80 10 client.0 _ll_register_callbacks cb 0x558c3578c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 0x558c358a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358a73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46296/0 (socket says 172.21.15.62:46296) 2023-07-14T16:29:46.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 -- 172.21.15.62:0/1779667131 learned_addr learned my addr 172.21.15.62:0/1779667131 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30ffff700 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 -- 172.21.15.62:0/1779667131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 msgr2=0x558c358ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 -- 172.21.15.62:0/1779667131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 msgr2=0x558c358a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 0x558c358a78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 -- 172.21.15.62:0/1779667131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff304009db0 con 0x558c3589d720 2023-07-14T16:29:46.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358a73b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff300009c10 tx=0x7ff3000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d358812b61b2025a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.635+0000 7ff2eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.636+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1779667131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300017070 con 0x558c3589d720 2023-07-14T16:29:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.636+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1779667131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c357484f0 con 0x558c3589d720 2023-07-14T16:29:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.636+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1779667131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff300012d00 con 0x558c3589d720 2023-07-14T16:29:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.636+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1779667131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff300012ec0 con 0x558c3589d720 2023-07-14T16:29:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.636+0000 7ff3295a7b80 10 client.19443 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:46.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.636+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1779667131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558c35605c60 con 0x558c3589d720 2023-07-14T16:29:46.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.636+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1779667131 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558c357484f0 con 0x558c3589d720 2023-07-14T16:29:46.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.637+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1779667131 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff30001d520 con 0x558c3589d720 2023-07-14T16:29:46.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.637+0000 7ff3295a7b80 10 client.19443 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:46.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.637+0000 7ff3295a7b80 10 client.19443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:46.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.637+0000 7ff3295a7b80 20 client.19443 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:46.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.637+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558c358b3160 0x558c358b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.637+0000 7ff3295a7b80 4 client.19443 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:46.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.637+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1779667131 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c357e7eb0 con 0x558c358b3160 2023-07-14T16:29:46.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.638+0000 7ff30effd700 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558c358b3160 0x558c358b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.639+0000 7ff30effd700 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558c358b3160 0x558c358b5570 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c423d7df0a3b64 server_cookie=d564e9b652c75da2 in_seq=0 out_seq=0 2023-07-14T16:29:46.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.639+0000 7ff2eeffd700 10 client.19443 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.639+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1779667131 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558c357e7eb0 con 0x558c358b3160 2023-07-14T16:29:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.639+0000 7ff2eeffd700 10 client.19443 handle_command_reply: tid=0 2023-07-14T16:29:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.639+0000 7ff3295a7b80 1 client.19443 shutdown 2023-07-14T16:29:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1779667131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 msgr2=0x558c358a73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358a73b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff300009c10 tx=0x7ff3000047c0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1779667131 shutdown_connections 2023-07-14T16:29:46.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358a73b0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558c358b3160 0x558c358b5570 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1779667131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 0x558c358a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1779667131 >> 172.21.15.62:0/1779667131 conn(0x558c35892ff0 msgr2=0x558c35897b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff2eeffd700 0 client.19443 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1779667131 shutdown_connections 2023-07-14T16:29:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1779667131 wait complete. 2023-07-14T16:29:46.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.640+0000 7ff3295a7b80 20 client.19443 trim_cache size 0 max 16384 2023-07-14T16:29:46.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.658+0000 7ff3295a7b80 1 Processor -- start 2023-07-14T16:29:46.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.658+0000 7ff3295a7b80 1 -- start start 2023-07-14T16:29:46.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589d720 0x558c358a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589e070 0x558c358ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3589e690 con 0x558c3589e070 2023-07-14T16:29:46.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c358b2e60 con 0x558c3589d720 2023-07-14T16:29:46.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c358a5d40 con 0x558c3589ee30 2023-07-14T16:29:46.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff30effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589e070 0x558c358ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff30effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589e070 0x558c358ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46298/0 (socket says 172.21.15.62:46298) 2023-07-14T16:29:46.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff30effd700 1 -- 172.21.15.62:0/731628127 learned_addr learned my addr 172.21.15.62:0/731628127 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:46.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589d720 0x558c358a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.659+0000 7ff30ffff700 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff30effd700 1 -- 172.21.15.62:0/731628127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 msgr2=0x558c358a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:46.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff30effd700 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff30effd700 1 -- 172.21.15.62:0/731628127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589d720 msgr2=0x558c358a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:46.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff30effd700 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589d720 0x558c358a5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff30effd700 1 -- 172.21.15.62:0/731628127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c35643250 con 0x558c3589e070 2023-07-14T16:29:46.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff30effd700 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589e070 0x558c358ae8f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff3040092a0 tx=0x7ff304022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a19de89c8742e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:46.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff2f67fc700 1 -- 172.21.15.62:0/731628127 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304025040 con 0x558c3589e070 2023-07-14T16:29:46.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff2f67fc700 1 -- 172.21.15.62:0/731628127 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff30402d5f0 con 0x558c3589e070 2023-07-14T16:29:46.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.660+0000 7ff2f67fc700 1 -- 172.21.15.62:0/731628127 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040052a0 con 0x558c3589e070 2023-07-14T16:29:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 -- 172.21.15.62:0/731628127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589e070 msgr2=0x558c358ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589e070 0x558c358ae8f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff3040092a0 tx=0x7ff304022040 comp rx=0 tx=0).stop 2023-07-14T16:29:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 -- 172.21.15.62:0/731628127 shutdown_connections 2023-07-14T16:29:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589d720 0x558c358a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/731628127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589e070 0x558c358ae8f0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 -- 172.21.15.62:0/731628127 >> 172.21.15.62:0/731628127 conn(0x558c35892ff0 msgr2=0x558c358a6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:46.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 -- 172.21.15.62:0/731628127 shutdown_connections 2023-07-14T16:29:46.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.661+0000 7ff3295a7b80 1 -- 172.21.15.62:0/731628127 wait complete. 2023-07-14T16:29:46.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.662+0000 7ff3295a7b80 1 -- start start 2023-07-14T16:29:46.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.662+0000 7ff3295a7b80 1 Processor -- start 2023-07-14T16:29:46.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.662+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.662+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 0x558c358aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff3295a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c358aec60 con 0x558c3589d720 2023-07-14T16:29:46.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c358ad4d0 con 0x558c3589e070 2023-07-14T16:29:46.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff3295a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c358ad690 con 0x558c3589ee30 2023-07-14T16:29:46.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff3295a7b80 10 client.0 _ll_register_callbacks cb 0x7ff3000043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:46.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46300/0 (socket says 172.21.15.62:46300) 2023-07-14T16:29:46.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30f7fe700 1 -- 172.21.15.62:0/1875136273 learned_addr learned my addr 172.21.15.62:0/1875136273 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30effd700 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 0x558c358aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30ffff700 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30f7fe700 1 -- 172.21.15.62:0/1875136273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 msgr2=0x558c358acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30f7fe700 1 -- 172.21.15.62:0/1875136273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 msgr2=0x558c358aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 0x558c358aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.663+0000 7ff30f7fe700 1 -- 172.21.15.62:0/1875136273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c3561cde0 con 0x558c3589d720 2023-07-14T16:29:46.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.664+0000 7ff30f7fe700 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358ae440 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff300019040 tx=0x7ff300012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf9b169e5f7487c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:46.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.664+0000 7ff2eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:46.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.664+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1875136273 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300017050 con 0x558c3589d720 2023-07-14T16:29:46.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.664+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1875136273 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff300004830 con 0x558c3589d720 2023-07-14T16:29:46.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.664+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1875136273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff304011a60 con 0x558c3589d720 2023-07-14T16:29:46.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.664+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1875136273 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3000049f0 con 0x558c3589d720 2023-07-14T16:29:46.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.664+0000 7ff3295a7b80 10 client.19449 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:46.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.664+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1875136273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558c3561b2b0 con 0x558c3589d720 2023-07-14T16:29:46.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1875136273 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff304011a60 con 0x558c3589d720 2023-07-14T16:29:46.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1875136273 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff300044070 con 0x558c3589d720 2023-07-14T16:29:46.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff3295a7b80 10 client.19449 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:46.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff3295a7b80 10 client.19449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:46.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff3295a7b80 20 client.19449 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:46.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558c35910db0 0x558c359131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff3295a7b80 4 client.19449 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:29:46.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1875136273 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c357aa050 con 0x558c35910db0 2023-07-14T16:29:46.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.665+0000 7ff30effd700 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558c35910db0 0x558c359131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff30effd700 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558c35910db0 0x558c359131c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef7ea63b09d0974 server_cookie=bce7b86e2e5eaa67 in_seq=0 out_seq=0 2023-07-14T16:29:46.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff2eeffd700 10 client.19449 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:46.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff2eeffd700 1 -- 172.21.15.62:0/1875136273 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558c357aa050 con 0x558c35910db0 2023-07-14T16:29:46.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff2eeffd700 10 client.19449 handle_command_reply: tid=0 2023-07-14T16:29:46.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 client.19449 shutdown 2023-07-14T16:29:46.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1875136273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 msgr2=0x558c358ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358ae440 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7ff300019040 tx=0x7ff300012d00 comp rx=0 tx=0).stop 2023-07-14T16:29:46.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1875136273 shutdown_connections 2023-07-14T16:29:46.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558c3589ee30 0x558c358acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c3589d720 0x558c358ae440 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558c35910db0 0x558c359131c0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 --2- 172.21.15.62:0/1875136273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558c3589e070 0x558c358aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff2eeffd700 0 client.19449 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:46.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.666+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1875136273 >> 172.21.15.62:0/1875136273 conn(0x558c35892ff0 msgr2=0x558c358a5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:46.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.667+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1875136273 shutdown_connections 2023-07-14T16:29:46.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.667+0000 7ff3295a7b80 1 -- 172.21.15.62:0/1875136273 wait complete. 2023-07-14T16:29:46.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.667+0000 7ff3295a7b80 20 client.19449 trim_cache size 0 max 16384 2023-07-14T16:29:46.649 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:29:46.649 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:29:46.650 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:29:46.650 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:29:46.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.667+0000 7ff31499d700 1 -- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2f80a11f0 msgr2=0x7ff2f80a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.667+0000 7ff31499d700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2f80a11f0 0x7ff2f80a36a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff300009e30 tx=0x7ff300009260 comp rx=0 tx=0).stop 2023-07-14T16:29:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.667+0000 7ff31499d700 1 -- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 msgr2=0x7ff31019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.667+0000 7ff31499d700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 0x7ff31019ed10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ff30800d010 tx=0x7ff30800bcc0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.668+0000 7ff31499d700 1 -- 172.21.15.62:0/3018452451 shutdown_connections 2023-07-14T16:29:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.668+0000 7ff31499d700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff310105690 0x7ff31019ed10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.668+0000 7ff31499d700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff310100450 0x7ff310197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.668+0000 7ff31499d700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff2f80a11f0 0x7ff2f80a36a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.668+0000 7ff31499d700 1 --2- 172.21.15.62:0/3018452451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff310102d70 0x7ff310197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.668+0000 7ff31499d700 1 -- 172.21.15.62:0/3018452451 >> 172.21.15.62:0/3018452451 conn(0x7ff3100f9ed0 msgr2=0x7ff3100fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.668+0000 7ff31499d700 1 -- 172.21.15.62:0/3018452451 shutdown_connections 2023-07-14T16:29:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.668+0000 7ff31499d700 1 -- 172.21.15.62:0/3018452451 wait complete. 2023-07-14T16:29:46.661 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:29:46.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:29:46.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3519835486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff081016a0 msgr2=0x7eff08107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3519835486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff081016a0 0x7eff08107f60 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7eff0000b0a0 tx=0x7eff0000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3519835486 shutdown_connections 2023-07-14T16:29:46.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3519835486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff081016a0 0x7eff08107f60 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3519835486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff080ffb20 0x7eff080fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3519835486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff08100d10 0x7eff08101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3519835486 >> 172.21.15.62:0/3519835486 conn(0x7eff080fb160 msgr2=0x7eff080fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:46.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3519835486 shutdown_connections 2023-07-14T16:29:46.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.913+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3519835486 wait complete. 2023-07-14T16:29:46.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.915+0000 7eff0f5ff700 1 Processor -- start 2023-07-14T16:29:46.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.915+0000 7eff0f5ff700 1 -- start start 2023-07-14T16:29:46.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.915+0000 7eff0f5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff080ffb20 0x7eff08195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.915+0000 7eff0f5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff08100d10 0x7eff08195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.915+0000 7eff0f5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff081016a0 0x7eff0819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.915+0000 7eff0f5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0819d390 con 0x7eff08100d10 2023-07-14T16:29:46.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.915+0000 7eff0f5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0819d4d0 con 0x7eff080ffb20 2023-07-14T16:29:46.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.915+0000 7eff0f5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0819d660 con 0x7eff081016a0 2023-07-14T16:29:46.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0d39b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff080ffb20 0x7eff08195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0d39b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff080ffb20 0x7eff08195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36012/0 (socket says 172.21.15.62:36012) 2023-07-14T16:29:46.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0d39b700 1 -- 172.21.15.62:0/3065959807 learned_addr learned my addr 172.21.15.62:0/3065959807 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:46.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0db9c700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff081016a0 0x7eff0819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0cb9a700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff08100d10 0x7eff08195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0d39b700 1 -- 172.21.15.62:0/3065959807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff081016a0 msgr2=0x7eff0819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0d39b700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff081016a0 0x7eff0819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0d39b700 1 -- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff08100d10 msgr2=0x7eff08195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:46.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0d39b700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff08100d10 0x7eff08195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:46.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.916+0000 7eff0d39b700 1 -- 172.21.15.62:0/3065959807 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff00009d00 con 0x7eff080ffb20 2023-07-14T16:29:46.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.917+0000 7eff0d39b700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff080ffb20 0x7eff08195810 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff0400e970 tx=0x7eff0400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.917+0000 7efefe7fc700 1 -- 172.21.15.62:0/3065959807 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04009790 con 0x7eff080ffb20 2023-07-14T16:29:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.917+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0819d940 con 0x7eff080ffb20 2023-07-14T16:29:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.917+0000 7efefe7fc700 1 -- 172.21.15.62:0/3065959807 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff0400c7e0 con 0x7eff080ffb20 2023-07-14T16:29:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.917+0000 7efefe7fc700 1 -- 172.21.15.62:0/3065959807 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04003bd0 con 0x7eff080ffb20 2023-07-14T16:29:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.917+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0819dea0 con 0x7eff080ffb20 2023-07-14T16:29:46.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.918+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff08106960 con 0x7eff080ffb20 2023-07-14T16:29:46.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.920+0000 7efefe7fc700 1 -- 172.21.15.62:0/3065959807 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7eff0400c950 con 0x7eff080ffb20 2023-07-14T16:29:46.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.921+0000 7efefe7fc700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efef40a1220 0x7efef40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:46.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.922+0000 7efefe7fc700 1 -- 172.21.15.62:0/3065959807 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7eff04014070 con 0x7eff080ffb20 2023-07-14T16:29:46.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.922+0000 7eff0cb9a700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efef40a1220 0x7efef40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:46.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.922+0000 7eff0cb9a700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efef40a1220 0x7efef40a36d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efef8009c60 tx=0x7efef8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:46.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:46.924+0000 7efefe7fc700 1 -- 172.21.15.62:0/3065959807 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7eff04096730 con 0x7eff080ffb20 2023-07-14T16:29:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:46 smithi062 ceph-mon[95543]: pgmap v781: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 MiB/s wr, 14 op/s 2023-07-14T16:29:47.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:46 smithi081 ceph-mon[127795]: pgmap v781: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 MiB/s wr, 14 op/s 2023-07-14T16:29:47.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.214+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff0802c850 con 0x7eff080ffb20 2023-07-14T16:29:47.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.215+0000 7efefe7fc700 1 -- 172.21.15.62:0/3065959807 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10828 (secure 0 0 0) 0x7eff04099d50 con 0x7eff080ffb20 2023-07-14T16:29:47.190 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:29:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:29:03.831862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:29:47.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efef40a1220 msgr2=0x7efef40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efef40a1220 0x7efef40a36d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efef8009c60 tx=0x7efef8009260 comp rx=0 tx=0).stop 2023-07-14T16:29:47.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff080ffb20 msgr2=0x7eff08195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff080ffb20 0x7eff08195810 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff0400e970 tx=0x7eff0400ec80 comp rx=0 tx=0).stop 2023-07-14T16:29:47.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 shutdown_connections 2023-07-14T16:29:47.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff081016a0 0x7eff0819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7efef40a1220 0x7efef40a36d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff080ffb20 0x7eff08195810 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 --2- 172.21.15.62:0/3065959807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff08100d10 0x7eff08195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 >> 172.21.15.62:0/3065959807 conn(0x7eff080fb160 msgr2=0x7eff080fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 shutdown_connections 2023-07-14T16:29:47.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.219+0000 7eff0f5ff700 1 -- 172.21.15.62:0/3065959807 wait complete. 2023-07-14T16:29:47.197 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2023-07-14T16:29:47.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:29:47.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:46 smithi164 ceph-mon[97427]: pgmap v781: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 MiB/s wr, 14 op/s 2023-07-14T16:29:47.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.464+0000 7f958e1ff700 1 -- 172.21.15.62:0/3859789737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9588105800 msgr2=0x7f9588107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.464+0000 7f958e1ff700 1 --2- 172.21.15.62:0/3859789737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9588105800 0x7f9588107c00 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f958000b0a0 tx=0x7f958000b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.464+0000 7f958e1ff700 1 -- 172.21.15.62:0/3859789737 shutdown_connections 2023-07-14T16:29:47.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.464+0000 7f958e1ff700 1 --2- 172.21.15.62:0/3859789737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9588105800 0x7f9588107c00 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.464+0000 7f958e1ff700 1 --2- 172.21.15.62:0/3859789737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95881005c0 0x7f95881029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.464+0000 7f958e1ff700 1 --2- 172.21.15.62:0/3859789737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9588102ee0 0x7f95881052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.464+0000 7f958e1ff700 1 -- 172.21.15.62:0/3859789737 >> 172.21.15.62:0/3859789737 conn(0x7f95880fa000 msgr2=0x7f95880fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.464+0000 7f958e1ff700 1 -- 172.21.15.62:0/3859789737 shutdown_connections 2023-07-14T16:29:47.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.465+0000 7f958e1ff700 1 -- 172.21.15.62:0/3859789737 wait complete. 2023-07-14T16:29:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.466+0000 7f958e1ff700 1 Processor -- start 2023-07-14T16:29:47.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.466+0000 7f958e1ff700 1 -- start start 2023-07-14T16:29:47.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f958e1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95881005c0 0x7f9588197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f958e1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9588102ee0 0x7f9588197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f958e1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9588105800 0x7f958819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f958e1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819f5e0 con 0x7f95881005c0 2023-07-14T16:29:47.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f958e1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819f720 con 0x7f9588102ee0 2023-07-14T16:29:47.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f958e1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958819f8b0 con 0x7f9588105800 2023-07-14T16:29:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f95877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95881005c0 0x7f9588197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f95877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95881005c0 0x7f9588197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46322/0 (socket says 172.21.15.62:46322) 2023-07-14T16:29:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f95877fe700 1 -- 172.21.15.62:0/1688190561 learned_addr learned my addr 172.21.15.62:0/1688190561 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f9587fff700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9588105800 0x7f958819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f9586ffd700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9588102ee0 0x7f9588197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f95877fe700 1 -- 172.21.15.62:0/1688190561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9588105800 msgr2=0x7f958819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.467+0000 7f95877fe700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9588105800 0x7f958819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f95877fe700 1 -- 172.21.15.62:0/1688190561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9588102ee0 msgr2=0x7f9588197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f95877fe700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9588102ee0 0x7f9588197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f95877fe700 1 -- 172.21.15.62:0/1688190561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580009d00 con 0x7f95881005c0 2023-07-14T16:29:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f9587fff700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9588105800 0x7f958819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f9586ffd700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9588102ee0 0x7f9588197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f95877fe700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95881005c0 0x7f9588197a60 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f957800d950 tx=0x7f957800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f9584ff9700 1 -- 172.21.15.62:0/1688190561 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578009b40 con 0x7f95881005c0 2023-07-14T16:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f9584ff9700 1 -- 172.21.15.62:0/1688190561 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9578009ca0 con 0x7f95881005c0 2023-07-14T16:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f9584ff9700 1 -- 172.21.15.62:0/1688190561 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800f620 con 0x7f95881005c0 2023-07-14T16:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f958e1ff700 1 -- 172.21.15.62:0/1688190561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958819fb90 con 0x7f95881005c0 2023-07-14T16:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.468+0000 7f958e1ff700 1 -- 172.21.15.62:0/1688190561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95881a00a0 con 0x7f95881005c0 2023-07-14T16:29:47.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.472+0000 7f9584ff9700 1 -- 172.21.15.62:0/1688190561 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9578010460 con 0x7f95881005c0 2023-07-14T16:29:47.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.473+0000 7f9584ff9700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f95700a11d0 0x7f95700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.473+0000 7f9584ff9700 1 -- 172.21.15.62:0/1688190561 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f95780ccc90 con 0x7f95881005c0 2023-07-14T16:29:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.473+0000 7f9586ffd700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f95700a11d0 0x7f95700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.474+0000 7f95a1b26b80 1 Processor -- start 2023-07-14T16:29:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.474+0000 7f9586ffd700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f95700a11d0 0x7f95700a3680 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f957c009cb0 tx=0x7f957c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.474+0000 7f95a1b26b80 1 -- start start 2023-07-14T16:29:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9ae48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9ae56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9ae60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def99e9cb0 con 0x55def9ae44d0 2023-07-14T16:29:47.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def9a3ca40 con 0x55def9ae4e20 2023-07-14T16:29:47.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def9a35ed0 con 0x55def9ae5c70 2023-07-14T16:29:47.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9ae48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9ae48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46332/0 (socket says 172.21.15.62:46332) 2023-07-14T16:29:47.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95877fe700 1 -- 172.21.15.62:0/121352002 learned_addr learned my addr 172.21.15.62:0/121352002 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f9586ffd700 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9ae56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95877fe700 1 -- 172.21.15.62:0/121352002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 msgr2=0x55def9ae60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f9587fff700 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9ae60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95877fe700 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9ae60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95877fe700 1 -- 172.21.15.62:0/121352002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 msgr2=0x55def9ae56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95877fe700 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9ae56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f95877fe700 1 -- 172.21.15.62:0/121352002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def9863de0 con 0x55def9ae44d0 2023-07-14T16:29:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f9586ffd700 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9ae56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:47.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.475+0000 7f9587fff700 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9ae60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:29:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.476+0000 7f95877fe700 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9ae48e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9578096ac0 tx=0x7f9578099f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325cce1c0c4c9f0d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.476+0000 7f956dffb700 1 -- 172.21.15.62:0/121352002 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957809e080 con 0x55def9ae44d0 2023-07-14T16:29:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.476+0000 7f956dffb700 1 -- 172.21.15.62:0/121352002 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95780a24a0 con 0x55def9ae44d0 2023-07-14T16:29:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.476+0000 7f956dffb700 1 -- 172.21.15.62:0/121352002 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957809a510 con 0x55def9ae44d0 2023-07-14T16:29:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 -- 172.21.15.62:0/121352002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 msgr2=0x55def9ae48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9ae48e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9578096ac0 tx=0x7f9578099f50 comp rx=0 tx=0).stop 2023-07-14T16:29:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 -- 172.21.15.62:0/121352002 shutdown_connections 2023-07-14T16:29:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9ae60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9ae48e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/121352002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9ae56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 -- 172.21.15.62:0/121352002 >> 172.21.15.62:0/121352002 conn(0x55def9ad9da0 msgr2=0x55def9adf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 -- 172.21.15.62:0/121352002 shutdown_connections 2023-07-14T16:29:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.477+0000 7f95a1b26b80 1 -- 172.21.15.62:0/121352002 wait complete. 2023-07-14T16:29:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.478+0000 7f95a1b26b80 1 -- start start 2023-07-14T16:29:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.478+0000 7f95a1b26b80 1 Processor -- start 2023-07-14T16:29:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.478+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae44d0 0x55def9aee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.478+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9aee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9af56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def99d5ac0 con 0x55def9ae4e20 2023-07-14T16:29:47.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def99d57c0 con 0x55def9ae44d0 2023-07-14T16:29:47.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def98b9920 con 0x55def9ae5c70 2023-07-14T16:29:47.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f95a1b26b80 10 client.0 _ll_register_callbacks cb 0x55def99d3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:47.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9586ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9aee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9586ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9aee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46348/0 (socket says 172.21.15.62:46348) 2023-07-14T16:29:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9586ffd700 1 -- 172.21.15.62:0/2226060911 learned_addr learned my addr 172.21.15.62:0/2226060911 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f95877fe700 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae44d0 0x55def9aee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9587fff700 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9af56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9586ffd700 1 -- 172.21.15.62:0/2226060911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 msgr2=0x55def9af56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9586ffd700 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9af56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9586ffd700 1 -- 172.21.15.62:0/2226060911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae44d0 msgr2=0x55def9aee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9586ffd700 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae44d0 0x55def9aee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.479+0000 7f9586ffd700 1 -- 172.21.15.62:0/2226060911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95881a00a0 con 0x55def9ae4e20 2023-07-14T16:29:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.480+0000 7f9586ffd700 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9aee660 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f957c012870 tx=0x7f957c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b109dfbb3ad5bb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.480+0000 7f9566ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.480+0000 7f9566ffd700 1 -- 172.21.15.62:0/2226060911 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c029630 con 0x55def9ae4e20 2023-07-14T16:29:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.480+0000 7f95a1b26b80 1 -- 172.21.15.62:0/2226060911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55def998f4f0 con 0x55def9ae4e20 2023-07-14T16:29:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.480+0000 7f9566ffd700 1 -- 172.21.15.62:0/2226060911 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f957c018070 con 0x55def9ae4e20 2023-07-14T16:29:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.480+0000 7f9566ffd700 1 -- 172.21.15.62:0/2226060911 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f957c029790 con 0x55def9ae4e20 2023-07-14T16:29:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.480+0000 7f95a1b26b80 10 client.19467 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.480+0000 7f95a1b26b80 1 -- 172.21.15.62:0/2226060911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55def984cc60 con 0x55def9ae4e20 2023-07-14T16:29:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.481+0000 7f9566ffd700 1 -- 172.21.15.62:0/2226060911 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55def998f4f0 con 0x55def9ae4e20 2023-07-14T16:29:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.481+0000 7f9566ffd700 1 -- 172.21.15.62:0/2226060911 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f957c0208f0 con 0x55def9ae4e20 2023-07-14T16:29:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.481+0000 7f95a1b26b80 10 client.19467 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.481+0000 7f95a1b26b80 10 client.19467 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:29:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.481+0000 7f95a1b26b80 20 client.19467 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.481+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55def9af9f10 0x55def9afc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.481+0000 7f95a1b26b80 4 client.19467 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.481+0000 7f95a1b26b80 1 -- 172.21.15.62:0/2226060911 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55def9a2eeb0 con 0x55def9af9f10 2023-07-14T16:29:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.482+0000 7f95877fe700 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55def9af9f10 0x55def9afc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.482+0000 7f95877fe700 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55def9af9f10 0x55def9afc320 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebbab825b0382f46 server_cookie=acbf92117721ebf2 in_seq=0 out_seq=0 2023-07-14T16:29:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.482+0000 7f9566ffd700 10 client.19467 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:29:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f9566ffd700 1 -- 172.21.15.62:0/2226060911 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55def9a2eeb0 con 0x55def9af9f10 2023-07-14T16:29:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f9566ffd700 10 client.19467 handle_command_reply: tid=0 2023-07-14T16:29:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 client.19467 shutdown 2023-07-14T16:29:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 -- 172.21.15.62:0/2226060911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 msgr2=0x55def9aee660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9aee660 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f957c012870 tx=0x7f957c0047c0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 -- 172.21.15.62:0/2226060911 shutdown_connections 2023-07-14T16:29:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55def9af9f10 0x55def9afc320 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9af56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae44d0 0x55def9aee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/2226060911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9aee660 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 -- 172.21.15.62:0/2226060911 >> 172.21.15.62:0/2226060911 conn(0x55def9ad9da0 msgr2=0x55def9ade8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f9566ffd700 0 client.19467 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:29:47.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.483+0000 7f95a1b26b80 1 -- 172.21.15.62:0/2226060911 shutdown_connections 2023-07-14T16:29:47.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.484+0000 7f95a1b26b80 1 -- 172.21.15.62:0/2226060911 wait complete. 2023-07-14T16:29:47.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.484+0000 7f95a1b26b80 20 client.19467 trim_cache size 0 max 16384 2023-07-14T16:29:47.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.500+0000 7f95a1b26b80 1 Processor -- start 2023-07-14T16:29:47.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95a1b26b80 1 -- start start 2023-07-14T16:29:47.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9aecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9af5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9aec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def9aeefc0 con 0x55def9ae44d0 2023-07-14T16:29:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def9aebe80 con 0x55def9ae4e20 2023-07-14T16:29:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def9af9c10 con 0x55def9ae5c70 2023-07-14T16:29:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9aecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9aecd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46358/0 (socket says 172.21.15.62:46358) 2023-07-14T16:29:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.501+0000 7f95877fe700 1 -- 172.21.15.62:0/651653224 learned_addr learned my addr 172.21.15.62:0/651653224 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f9586ffd700 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9af5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f9587fff700 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9aec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f95877fe700 1 -- 172.21.15.62:0/651653224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 msgr2=0x55def9aec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f95877fe700 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9aec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f95877fe700 1 -- 172.21.15.62:0/651653224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 msgr2=0x55def9af5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f95877fe700 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9af5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f95877fe700 1 -- 172.21.15.62:0/651653224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def988a250 con 0x55def9ae44d0 2023-07-14T16:29:47.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f9587fff700 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9aec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:47.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f95877fe700 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9aecd90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9578010bc0 tx=0x7f9578010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843a7fd82856cf9d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f956e7fc700 1 -- 172.21.15.62:0/651653224 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578010710 con 0x55def9ae44d0 2023-07-14T16:29:47.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f956e7fc700 1 -- 172.21.15.62:0/651653224 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9578099540 con 0x55def9ae44d0 2023-07-14T16:29:47.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.502+0000 7f956e7fc700 1 -- 172.21.15.62:0/651653224 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780a2010 con 0x55def9ae44d0 2023-07-14T16:29:47.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 -- 172.21.15.62:0/651653224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 msgr2=0x55def9aecd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9aecd90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9578010bc0 tx=0x7f9578010e70 comp rx=0 tx=0).stop 2023-07-14T16:29:47.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 -- 172.21.15.62:0/651653224 shutdown_connections 2023-07-14T16:29:47.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae5c70 0x55def9aec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae44d0 0x55def9aecd90 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/651653224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae4e20 0x55def9af5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 -- 172.21.15.62:0/651653224 >> 172.21.15.62:0/651653224 conn(0x55def9ad9da0 msgr2=0x55def9b00c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 -- 172.21.15.62:0/651653224 shutdown_connections 2023-07-14T16:29:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.503+0000 7f95a1b26b80 1 -- 172.21.15.62:0/651653224 wait complete. 2023-07-14T16:29:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 1 -- start start 2023-07-14T16:29:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 1 Processor -- start 2023-07-14T16:29:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae44d0 0x55def9af3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9af39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae5c70 0x55def9af49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def9af4f20 con 0x55def9ae4e20 2023-07-14T16:29:47.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def9af5090 con 0x55def9ae5c70 2023-07-14T16:29:47.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def9af5250 con 0x55def9ae44d0 2023-07-14T16:29:47.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.504+0000 7f95a1b26b80 10 client.0 _ll_register_callbacks cb 0x7f957c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:47.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9586ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9af39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f95877fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae44d0 0x55def9af3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f95877fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae44d0 0x55def9af3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54612/0 (socket says 172.21.15.62:54612) 2023-07-14T16:29:47.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9586ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9af39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46360/0 (socket says 172.21.15.62:46360) 2023-07-14T16:29:47.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f95877fe700 1 -- 172.21.15.62:0/1554561145 learned_addr learned my addr 172.21.15.62:0/1554561145 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9587fff700 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae5c70 0x55def9af49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9586ffd700 1 -- 172.21.15.62:0/1554561145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae44d0 msgr2=0x55def9af3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9586ffd700 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae44d0 0x55def9af3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9586ffd700 1 -- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae5c70 msgr2=0x55def9af49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9586ffd700 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae5c70 0x55def9af49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9586ffd700 1 -- 172.21.15.62:0/1554561145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55def9863de0 con 0x55def9ae4e20 2023-07-14T16:29:47.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f95877fe700 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae44d0 0x55def9af3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.505+0000 7f9587fff700 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae5c70 0x55def9af49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f9586ffd700 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9af39d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f957c009890 tx=0x7f957c009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1305291aa386c652 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f9566ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f9566ffd700 1 -- 172.21.15.62:0/1554561145 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c020900 con 0x55def9ae4e20 2023-07-14T16:29:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f95a1b26b80 1 -- 172.21.15.62:0/1554561145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95780a24a0 con 0x55def9ae4e20 2023-07-14T16:29:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f9566ffd700 1 -- 172.21.15.62:0/1554561145 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f957c038070 con 0x55def9ae4e20 2023-07-14T16:29:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f9566ffd700 1 -- 172.21.15.62:0/1554561145 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f957c020a60 con 0x55def9ae4e20 2023-07-14T16:29:47.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f95a1b26b80 10 client.19473 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f95a1b26b80 1 -- 172.21.15.62:0/1554561145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55def98622b0 con 0x55def9ae4e20 2023-07-14T16:29:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.506+0000 7f9566ffd700 1 -- 172.21.15.62:0/1554561145 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95780a24a0 con 0x55def9ae4e20 2023-07-14T16:29:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.507+0000 7f9566ffd700 1 -- 172.21.15.62:0/1554561145 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f957c029430 con 0x55def9ae4e20 2023-07-14T16:29:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.507+0000 7f95a1b26b80 10 client.19473 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.507+0000 7f95a1b26b80 10 client.19473 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:29:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.507+0000 7f95a1b26b80 20 client.19473 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:47.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.507+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55def9b4edd0 0x55def9b511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.507+0000 7f95a1b26b80 4 client.19473 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:29:47.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.507+0000 7f95a1b26b80 1 -- 172.21.15.62:0/1554561145 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55def99f1050 con 0x55def9b4edd0 2023-07-14T16:29:47.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.507+0000 7f95877fe700 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55def9b4edd0 0x55def9b511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95877fe700 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55def9b4edd0 0x55def9b511e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cadaf32eadc7f48 server_cookie=646f56f081ade249 in_seq=0 out_seq=0 2023-07-14T16:29:47.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f9566ffd700 10 client.19473 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:29:47.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f9566ffd700 1 -- 172.21.15.62:0/1554561145 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55def99f1050 con 0x55def9b4edd0 2023-07-14T16:29:47.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f9566ffd700 10 client.19473 handle_command_reply: tid=0 2023-07-14T16:29:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 client.19473 shutdown 2023-07-14T16:29:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 -- 172.21.15.62:0/1554561145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 msgr2=0x55def9af39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9af39d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f957c009890 tx=0x7f957c009af0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 -- 172.21.15.62:0/1554561145 shutdown_connections 2023-07-14T16:29:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55def9b4edd0 0x55def9b511e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def9ae5c70 0x55def9af49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55def9ae44d0 0x55def9af3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f9566ffd700 0 client.19473 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:29:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 --2- 172.21.15.62:0/1554561145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55def9ae4e20 0x55def9af39d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.508+0000 7f95a1b26b80 1 -- 172.21.15.62:0/1554561145 >> 172.21.15.62:0/1554561145 conn(0x55def9ad9da0 msgr2=0x55def9aecd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.509+0000 7f95a1b26b80 1 -- 172.21.15.62:0/1554561145 shutdown_connections 2023-07-14T16:29:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.509+0000 7f95a1b26b80 1 -- 172.21.15.62:0/1554561145 wait complete. 2023-07-14T16:29:47.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.509+0000 7f95a1b26b80 20 client.19473 trim_cache size 0 max 16384 2023-07-14T16:29:47.497 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:29:47.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.509+0000 7f958e1ff700 1 -- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f95700a11d0 msgr2=0x7f95700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.509+0000 7f958e1ff700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f95700a11d0 0x7f95700a3680 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f957c009cb0 tx=0x7f957c009260 comp rx=0 tx=0).stop 2023-07-14T16:29:47.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.509+0000 7f958e1ff700 1 -- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95881005c0 msgr2=0x7f9588197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.509+0000 7f958e1ff700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95881005c0 0x7f9588197a60 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f957800d950 tx=0x7f957800dd10 comp rx=0 tx=0).stop 2023-07-14T16:29:47.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.510+0000 7f958e1ff700 1 -- 172.21.15.62:0/1688190561 shutdown_connections 2023-07-14T16:29:47.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.510+0000 7f958e1ff700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9588105800 0x7f958819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.510+0000 7f958e1ff700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95881005c0 0x7f9588197a60 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.510+0000 7f958e1ff700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f95700a11d0 0x7f95700a3680 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.510+0000 7f958e1ff700 1 --2- 172.21.15.62:0/1688190561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9588102ee0 0x7f9588197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.510+0000 7f958e1ff700 1 -- 172.21.15.62:0/1688190561 >> 172.21.15.62:0/1688190561 conn(0x7f95880fa000 msgr2=0x7f95880fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.510+0000 7f958e1ff700 1 -- 172.21.15.62:0/1688190561 shutdown_connections 2023-07-14T16:29:47.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.510+0000 7f958e1ff700 1 -- 172.21.15.62:0/1688190561 wait complete. 2023-07-14T16:29:47.504 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:29:47.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.774+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3203598630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d30101bf0 msgr2=0x7f6d30102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.774+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3203598630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d30101bf0 0x7f6d30102000 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009990 tx=0x7f6d20009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.774+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3203598630 shutdown_connections 2023-07-14T16:29:47.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.774+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3203598630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d30103770 0x7f6d3010a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.774+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3203598630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d30101bf0 0x7f6d30102000 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.774+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3203598630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d30102de0 0x7f6d30103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.775+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3203598630 >> 172.21.15.62:0/3203598630 conn(0x7f6d300fd210 msgr2=0x7f6d300ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.775+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3203598630 shutdown_connections 2023-07-14T16:29:47.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.775+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3203598630 wait complete. 2023-07-14T16:29:47.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.776+0000 7f6d34f9d700 1 Processor -- start 2023-07-14T16:29:47.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.776+0000 7f6d34f9d700 1 -- start start 2023-07-14T16:29:47.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.776+0000 7f6d34f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d30101bf0 0x7f6d30197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.776+0000 7f6d34f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d30102de0 0x7f6d30197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d34f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d30103770 0x7f6d3019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d34f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3019f290 con 0x7f6d30103770 2023-07-14T16:29:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d34f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3019f3d0 con 0x7f6d30101bf0 2023-07-14T16:29:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d34f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3019f560 con 0x7f6d30102de0 2023-07-14T16:29:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d30103770 0x7f6d3019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d30101bf0 0x7f6d30197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d30102de0 0x7f6d30197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d30101bf0 0x7f6d30197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36058/0 (socket says 172.21.15.62:36058) 2023-07-14T16:29:47.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d30102de0 0x7f6d30197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54642/0 (socket says 172.21.15.62:54642) 2023-07-14T16:29:47.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/3062832515 learned_addr learned my addr 172.21.15.62:0/3062832515 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2effd700 1 -- 172.21.15.62:0/3062832515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d30101bf0 msgr2=0x7f6d30197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2effd700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d30101bf0 0x7f6d30197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2effd700 1 -- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d30103770 msgr2=0x7f6d3019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2effd700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d30103770 0x7f6d3019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2effd700 1 -- 172.21.15.62:0/3062832515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d200095f0 con 0x7f6d30102de0 2023-07-14T16:29:47.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.777+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d30101bf0 0x7f6d30197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:47.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.778+0000 7f6d2effd700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d30102de0 0x7f6d30197c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400e9f0 tx=0x7f6d2400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.778+0000 7f6d2cff9700 1 -- 172.21.15.62:0/3062832515 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400c9c0 con 0x7f6d30102de0 2023-07-14T16:29:47.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.778+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3062832515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3019f840 con 0x7f6d30102de0 2023-07-14T16:29:47.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.778+0000 7f6d2cff9700 1 -- 172.21.15.62:0/3062832515 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d24010430 con 0x7f6d30102de0 2023-07-14T16:29:47.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.778+0000 7f6d2cff9700 1 -- 172.21.15.62:0/3062832515 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24018680 con 0x7f6d30102de0 2023-07-14T16:29:47.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.778+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3062832515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3019fda0 con 0x7f6d30102de0 2023-07-14T16:29:47.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.782+0000 7f6d2cff9700 1 -- 172.21.15.62:0/3062832515 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6d240105a0 con 0x7f6d30102de0 2023-07-14T16:29:47.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.784+0000 7f6d49ab8b80 1 Processor -- start 2023-07-14T16:29:47.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.784+0000 7f6d2cff9700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d180a1270 0x7f6d180a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.784+0000 7f6d2cff9700 1 -- 172.21.15.62:0/3062832515 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d24014070 con 0x7f6d30102de0 2023-07-14T16:29:47.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.784+0000 7f6d49ab8b80 1 -- start start 2023-07-14T16:29:47.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.784+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d180a1270 0x7f6d180a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.784+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af83360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.784+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 0x56437af84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.784+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af84670 0x56437af84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ae7fcb0 con 0x56437af82f30 2023-07-14T16:29:47.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437aed2a40 con 0x56437af84670 2023-07-14T16:29:47.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437aecbed0 con 0x56437af838a0 2023-07-14T16:29:47.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af83360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af83360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46376/0 (socket says 172.21.15.62:46376) 2023-07-14T16:29:47.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/2631282943 learned_addr learned my addr 172.21.15.62:0/2631282943 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2effd700 1 --2- 172.21.15.62:0/2631282943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 0x56437af84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/2631282943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 msgr2=0x56437af84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d180a1270 0x7f6d180a3720 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800f540 tx=0x7f6d2800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/2631282943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 0x56437af84090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/2631282943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af84670 msgr2=0x56437af84ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/2631282943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af84670 0x56437af84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/2631282943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437acf9de0 con 0x56437af82f30 2023-07-14T16:29:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.785+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/2631282943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af83360 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6d20005f30 tx=0x7f6d20005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3774ec005c9135d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.786+0000 7f6d15ffb700 1 -- 172.21.15.62:0/2631282943 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2001d070 con 0x56437af82f30 2023-07-14T16:29:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.786+0000 7f6d15ffb700 1 -- 172.21.15.62:0/2631282943 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d2001e040 con 0x56437af82f30 2023-07-14T16:29:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.786+0000 7f6d15ffb700 1 -- 172.21.15.62:0/2631282943 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2000bc30 con 0x56437af82f30 2023-07-14T16:29:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2631282943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 msgr2=0x56437af83360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2631282943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af83360 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6d20005f30 tx=0x7f6d20005c00 comp rx=0 tx=0).stop 2023-07-14T16:29:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2631282943 shutdown_connections 2023-07-14T16:29:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2631282943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af84670 0x56437af84ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2631282943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af83360 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2631282943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 0x56437af84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2631282943 >> 172.21.15.62:0/2631282943 conn(0x56437af6fff0 msgr2=0x56437af75460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2631282943 shutdown_connections 2023-07-14T16:29:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2631282943 wait complete. 2023-07-14T16:29:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.787+0000 7f6d49ab8b80 1 -- start start 2023-07-14T16:29:47.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.788+0000 7f6d49ab8b80 1 Processor -- start 2023-07-14T16:29:47.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.788+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.788+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 0x56437af8d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.788+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af94050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.788+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ae6bac0 con 0x56437af82f30 2023-07-14T16:29:47.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.788+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ae6b7c0 con 0x56437af838a0 2023-07-14T16:29:47.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.788+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437ad4f920 con 0x56437af84670 2023-07-14T16:29:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d49ab8b80 10 client.0 _ll_register_callbacks cb 0x56437ae69760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46392/0 (socket says 172.21.15.62:46392) 2023-07-14T16:29:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/806932079 learned_addr learned my addr 172.21.15.62:0/806932079 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2effd700 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 0x56437af8d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af94050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/806932079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 msgr2=0x56437af94050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af94050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/806932079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 msgr2=0x56437af8d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 0x56437af8d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.789+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/806932079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3010a960 con 0x56437af82f30 2023-07-14T16:29:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.790+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8cad0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6d20000c00 tx=0x7f6d2000f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27f340a1069e35b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.790+0000 7f6d0effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.790+0000 7f6d0effd700 1 -- 172.21.15.62:0/806932079 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2001d040 con 0x56437af82f30 2023-07-14T16:29:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.790+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/806932079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56437ae254f0 con 0x56437af82f30 2023-07-14T16:29:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.790+0000 7f6d0effd700 1 -- 172.21.15.62:0/806932079 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d2000be70 con 0x56437af82f30 2023-07-14T16:29:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.790+0000 7f6d0effd700 1 -- 172.21.15.62:0/806932079 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d20005960 con 0x56437af82f30 2023-07-14T16:29:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.790+0000 7f6d49ab8b80 10 client.19485 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.790+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/806932079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56437ace2c60 con 0x56437af82f30 2023-07-14T16:29:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.791+0000 7f6d0effd700 1 -- 172.21.15.62:0/806932079 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56437ae254f0 con 0x56437af82f30 2023-07-14T16:29:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.791+0000 7f6d0effd700 1 -- 172.21.15.62:0/806932079 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d20004620 con 0x56437af82f30 2023-07-14T16:29:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.791+0000 7f6d49ab8b80 10 client.19485 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.791+0000 7f6d49ab8b80 10 client.19485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:29:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.791+0000 7f6d49ab8b80 20 client.19485 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.791+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56437af983d0 0x56437af9a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.791+0000 7f6d49ab8b80 4 client.19485 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.791+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/806932079 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56437aec4eb0 con 0x56437af983d0 2023-07-14T16:29:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.792+0000 7f6d2effd700 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56437af983d0 0x56437af9a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.792+0000 7f6d2effd700 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56437af983d0 0x56437af9a7e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=593217caeaa923c5 server_cookie=bbfa96399151e8c7 in_seq=0 out_seq=0 2023-07-14T16:29:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.792+0000 7f6d0effd700 10 client.19485 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:29:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d0effd700 1 -- 172.21.15.62:0/806932079 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56437aec4eb0 con 0x56437af983d0 2023-07-14T16:29:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d0effd700 10 client.19485 handle_command_reply: tid=0 2023-07-14T16:29:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 client.19485 shutdown 2023-07-14T16:29:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/806932079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 msgr2=0x56437af8cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8cad0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6d20000c00 tx=0x7f6d2000f7c0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/806932079 shutdown_connections 2023-07-14T16:29:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56437af983d0 0x56437af9a7e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af94050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8cad0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/806932079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 0x56437af8d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/806932079 >> 172.21.15.62:0/806932079 conn(0x56437af6fff0 msgr2=0x56437af74b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d0effd700 0 client.19485 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:29:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/806932079 shutdown_connections 2023-07-14T16:29:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.793+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/806932079 wait complete. 2023-07-14T16:29:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.794+0000 7f6d49ab8b80 20 client.19485 trim_cache size 0 max 16384 2023-07-14T16:29:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.811+0000 7f6d49ab8b80 1 Processor -- start 2023-07-14T16:29:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.811+0000 7f6d49ab8b80 1 -- start start 2023-07-14T16:29:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af82f30 0x56437af84a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 0x56437af94010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af84670 0x56437af9f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437af8a760 con 0x56437af84670 2023-07-14T16:29:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437af9f680 con 0x56437af82f30 2023-07-14T16:29:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437af83300 con 0x56437af838a0 2023-07-14T16:29:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af84670 0x56437af9f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af84670 0x56437af9f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46394/0 (socket says 172.21.15.62:46394) 2023-07-14T16:29:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2ffff700 1 -- 172.21.15.62:0/668908663 learned_addr learned my addr 172.21.15.62:0/668908663 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af82f30 0x56437af84a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2effd700 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 0x56437af94010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2ffff700 1 -- 172.21.15.62:0/668908663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 msgr2=0x56437af94010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 0x56437af94010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2ffff700 1 -- 172.21.15.62:0/668908663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af82f30 msgr2=0x56437af84a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.812+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af82f30 0x56437af84a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.813+0000 7f6d2ffff700 1 -- 172.21.15.62:0/668908663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437ad20250 con 0x56437af84670 2023-07-14T16:29:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.813+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af82f30 0x56437af84a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.813+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af84670 0x56437af9f140 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009390 tx=0x7f6d28004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49531b62b90c6f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.813+0000 7f6d167fc700 1 -- 172.21.15.62:0/668908663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801c040 con 0x56437af84670 2023-07-14T16:29:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.813+0000 7f6d167fc700 1 -- 172.21.15.62:0/668908663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d2801b070 con 0x56437af84670 2023-07-14T16:29:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.813+0000 7f6d167fc700 1 -- 172.21.15.62:0/668908663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800e040 con 0x56437af84670 2023-07-14T16:29:47.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/668908663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af84670 msgr2=0x56437af9f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af84670 0x56437af9f140 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009390 tx=0x7f6d28004b40 comp rx=0 tx=0).stop 2023-07-14T16:29:47.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/668908663 shutdown_connections 2023-07-14T16:29:47.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af84670 0x56437af9f140 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af82f30 0x56437af84a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/668908663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af838a0 0x56437af94010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/668908663 >> 172.21.15.62:0/668908663 conn(0x56437af6fff0 msgr2=0x56437af8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/668908663 shutdown_connections 2023-07-14T16:29:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.814+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/668908663 wait complete. 2023-07-14T16:29:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.815+0000 7f6d49ab8b80 1 -- start start 2023-07-14T16:29:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.815+0000 7f6d49ab8b80 1 Processor -- start 2023-07-14T16:29:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.815+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.815+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 0x56437af900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.815+0000 7f6d49ab8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.815+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437af93620 con 0x56437af82f30 2023-07-14T16:29:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.815+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437af93790 con 0x56437af838a0 2023-07-14T16:29:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d49ab8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437af93950 con 0x56437af84670 2023-07-14T16:29:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d49ab8b80 10 client.0 _ll_register_callbacks cb 0x7f6d20009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46396/0 (socket says 172.21.15.62:46396) 2023-07-14T16:29:47.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2f7fe700 1 -- 172.21.15.62:0/2725394433 learned_addr learned my addr 172.21.15.62:0/2725394433 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:47.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2effd700 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 0x56437af900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2ffff700 1 -- 172.21.15.62:0/2725394433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 msgr2=0x56437af900c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 0x56437af900c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2ffff700 1 -- 172.21.15.62:0/2725394433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 msgr2=0x56437af8fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2ffff700 1 -- 172.21.15.62:0/2725394433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56437ace2c60 con 0x56437af84670 2023-07-14T16:29:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.816+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.817+0000 7f6d2ffff700 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af930e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d28022cb0 tx=0x7f6d28022910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9523ae3b5e7b3439 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.817+0000 7f6d0effd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:29:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.817+0000 7f6d0effd700 1 -- 172.21.15.62:0/2725394433 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800e040 con 0x56437af84670 2023-07-14T16:29:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.817+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2725394433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d2001e040 con 0x56437af84670 2023-07-14T16:29:47.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.817+0000 7f6d0effd700 1 -- 172.21.15.62:0/2725394433 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d280093c0 con 0x56437af84670 2023-07-14T16:29:47.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.817+0000 7f6d0effd700 1 -- 172.21.15.62:0/2725394433 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d28008e10 con 0x56437af84670 2023-07-14T16:29:47.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.817+0000 7f6d49ab8b80 10 client.28921 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:47.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2725394433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f6d28009040 con 0x56437af84670 2023-07-14T16:29:47.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d0effd700 1 -- 172.21.15.62:0/2725394433 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d2001e040 con 0x56437af84670 2023-07-14T16:29:47.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d0effd700 1 -- 172.21.15.62:0/2725394433 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d2802b580 con 0x56437af84670 2023-07-14T16:29:47.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d49ab8b80 10 client.28921 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:47.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d49ab8b80 10 client.28921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:29:47.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d49ab8b80 20 client.28921 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:47.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56437afecfe0 0x56437afef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:47.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d49ab8b80 4 client.28921 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:29:47.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.818+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2725394433 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56437ae87050 con 0x56437afecfe0 2023-07-14T16:29:47.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.819+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56437afecfe0 0x56437afef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:47.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.819+0000 7f6d0effd700 10 client.28921 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:29:47.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.819+0000 7f6d2f7fe700 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56437afecfe0 0x56437afef3f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b6bc0a590bbc4b6 server_cookie=68b6b154c18f3394 in_seq=0 out_seq=0 2023-07-14T16:29:47.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.819+0000 7f6d0effd700 1 -- 172.21.15.62:0/2725394433 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56437ae87050 con 0x56437afecfe0 2023-07-14T16:29:47.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.819+0000 7f6d0effd700 10 client.28921 handle_command_reply: tid=0 2023-07-14T16:29:47.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.819+0000 7f6d49ab8b80 1 client.28921 shutdown 2023-07-14T16:29:47.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.819+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 msgr2=0x56437af930e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.819+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af930e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d28022cb0 tx=0x7f6d28022910 comp rx=0 tx=0).stop 2023-07-14T16:29:47.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2725394433 shutdown_connections 2023-07-14T16:29:47.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x56437afecfe0 0x56437afef3f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56437af84670 0x56437af930e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56437af82f30 0x56437af8fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d0effd700 0 client.28921 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:29:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 1 --2- 172.21.15.62:0/2725394433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56437af838a0 0x56437af900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2725394433 >> 172.21.15.62:0/2725394433 conn(0x56437af6fff0 msgr2=0x56437af9a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2725394433 shutdown_connections 2023-07-14T16:29:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 1 -- 172.21.15.62:0/2725394433 wait complete. 2023-07-14T16:29:47.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.820+0000 7f6d49ab8b80 20 client.28921 trim_cache size 0 max 16384 2023-07-14T16:29:47.804 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:29:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d180a1270 msgr2=0x7f6d180a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d180a1270 0x7f6d180a3720 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800f540 tx=0x7f6d2800b040 comp rx=0 tx=0).stop 2023-07-14T16:29:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3062832515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d30102de0 msgr2=0x7f6d30197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d30102de0 0x7f6d30197c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400e9f0 tx=0x7f6d2400ed00 comp rx=0 tx=0).stop 2023-07-14T16:29:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3062832515 shutdown_connections 2023-07-14T16:29:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d30103770 0x7f6d3019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d180a1270 0x7f6d180a3720 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d30101bf0 0x7f6d30197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 --2- 172.21.15.62:0/3062832515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d30102de0 0x7f6d30197c50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3062832515 >> 172.21.15.62:0/3062832515 conn(0x7f6d300fd210 msgr2=0x7f6d300ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3062832515 shutdown_connections 2023-07-14T16:29:47.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:47.821+0000 7f6d34f9d700 1 -- 172.21.15.62:0/3062832515 wait complete. 2023-07-14T16:29:47.815 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:29:48.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 -- 172.21.15.62:0/260433904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 msgr2=0x7fdc4c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 --2- 172.21.15.62:0/260433904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 0x7fdc4c107c00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b0a0 tx=0x7fdc4400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 -- 172.21.15.62:0/260433904 shutdown_connections 2023-07-14T16:29:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 --2- 172.21.15.62:0/260433904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 0x7fdc4c107c00 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 --2- 172.21.15.62:0/260433904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c100560 0x7fdc4c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 --2- 172.21.15.62:0/260433904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc4c102e80 0x7fdc4c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 -- 172.21.15.62:0/260433904 >> 172.21.15.62:0/260433904 conn(0x7fdc4c0fa000 msgr2=0x7fdc4c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 -- 172.21.15.62:0/260433904 shutdown_connections 2023-07-14T16:29:48.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.090+0000 7fdc515ff700 1 -- 172.21.15.62:0/260433904 wait complete. 2023-07-14T16:29:48.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.091+0000 7fdc515ff700 1 Processor -- start 2023-07-14T16:29:48.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.091+0000 7fdc515ff700 1 -- start start 2023-07-14T16:29:48.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.091+0000 7fdc515ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc4c100560 0x7fdc4c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.091+0000 7fdc515ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c102e80 0x7fdc4c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.091+0000 7fdc515ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 0x7fdc4c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.091+0000 7fdc515ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c19f5a0 con 0x7fdc4c1057a0 2023-07-14T16:29:48.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.091+0000 7fdc515ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c19f6e0 con 0x7fdc4c102e80 2023-07-14T16:29:48.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.091+0000 7fdc515ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c19f870 con 0x7fdc4c100560 2023-07-14T16:29:48.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 0x7fdc4c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4affd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc4c100560 0x7fdc4c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c102e80 0x7fdc4c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 0x7fdc4c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46416/0 (socket says 172.21.15.62:46416) 2023-07-14T16:29:48.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/785855897 learned_addr learned my addr 172.21.15.62:0/785855897 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:48.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/785855897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc4c100560 msgr2=0x7fdc4c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc4c100560 0x7fdc4c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/785855897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c102e80 msgr2=0x7fdc4c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c102e80 0x7fdc4c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/785855897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc44009d00 con 0x7fdc4c1057a0 2023-07-14T16:29:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.092+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 0x7fdc4c19efd0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400d010 tx=0x7fdc4400bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.093+0000 7fdc33fff700 1 -- 172.21.15.62:0/785855897 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400e040 con 0x7fdc4c1057a0 2023-07-14T16:29:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.093+0000 7fdc515ff700 1 -- 172.21.15.62:0/785855897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c19faf0 con 0x7fdc4c1057a0 2023-07-14T16:29:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.093+0000 7fdc33fff700 1 -- 172.21.15.62:0/785855897 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc44009480 con 0x7fdc4c1057a0 2023-07-14T16:29:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.093+0000 7fdc33fff700 1 -- 172.21.15.62:0/785855897 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc440052a0 con 0x7fdc4c1057a0 2023-07-14T16:29:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.093+0000 7fdc515ff700 1 -- 172.21.15.62:0/785855897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c19ff10 con 0x7fdc4c1057a0 2023-07-14T16:29:48.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.097+0000 7fdc33fff700 1 -- 172.21.15.62:0/785855897 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdc44005400 con 0x7fdc4c1057a0 2023-07-14T16:29:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.098+0000 7fdc64ee9b80 1 Processor -- start 2023-07-14T16:29:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.098+0000 7fdc64ee9b80 1 -- start start 2023-07-14T16:29:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.098+0000 7fdc33fff700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdc340a1150 0x7fdc340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc33fff700 1 -- 172.21.15.62:0/785855897 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc440cd940 con 0x7fdc4c1057a0 2023-07-14T16:29:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd64d0 0x55f169cd68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd6e20 0x55f169cd76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cd80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169bdbcb0 con 0x55f169cd7c70 2023-07-14T16:29:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169c2ea40 con 0x55f169cd64d0 2023-07-14T16:29:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169c27ed0 con 0x55f169cd6e20 2023-07-14T16:29:48.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdc340a1150 0x7fdc340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cd80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cd80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46418/0 (socket says 172.21.15.62:46418) 2023-07-14T16:29:48.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/1993085425 learned_addr learned my addr 172.21.15.62:0/1993085425 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:48.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.099+0000 7fdc4affd700 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd64d0 0x55f169cd68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd6e20 0x55f169cd76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/1993085425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd6e20 msgr2=0x55f169cd76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:48.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd6e20 0x55f169cd76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/1993085425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd64d0 msgr2=0x55f169cd68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:48.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd64d0 0x55f169cd68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/1993085425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f169a55de0 con 0x55f169cd7c70 2023-07-14T16:29:48.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd6e20 0x55f169cd76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:29:48.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4affd700 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd64d0 0x55f169cd68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:48.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdc340a1150 0x7fdc340a3600 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009c70 tx=0x7fdc40011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:48.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cd80e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fdc440a2c80 tx=0x7fdc4409ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aba6407bb127663 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:48.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc317fa700 1 -- 172.21.15.62:0/1993085425 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4409adf0 con 0x55f169cd7c70 2023-07-14T16:29:48.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc317fa700 1 -- 172.21.15.62:0/1993085425 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc4409ee20 con 0x55f169cd7c70 2023-07-14T16:29:48.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.100+0000 7fdc317fa700 1 -- 172.21.15.62:0/1993085425 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4409a2e0 con 0x55f169cd7c70 2023-07-14T16:29:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/1993085425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 msgr2=0x55f169cd80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cd80e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fdc440a2c80 tx=0x7fdc4409ac90 comp rx=0 tx=0).stop 2023-07-14T16:29:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/1993085425 shutdown_connections 2023-07-14T16:29:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cd80e0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd64d0 0x55f169cd68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/1993085425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd6e20 0x55f169cd76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/1993085425 >> 172.21.15.62:0/1993085425 conn(0x55f169ccbda0 msgr2=0x55f169cd11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/1993085425 shutdown_connections 2023-07-14T16:29:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.101+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/1993085425 wait complete. 2023-07-14T16:29:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.102+0000 7fdc64ee9b80 1 -- start start 2023-07-14T16:29:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.102+0000 7fdc64ee9b80 1 Processor -- start 2023-07-14T16:29:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169cdfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd6e20 0x55f169ce03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd7c70 0x55f169ce7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169bc7ac0 con 0x55f169cd6e20 2023-07-14T16:29:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169bc77c0 con 0x55f169cd7c70 2023-07-14T16:29:48.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169aab920 con 0x55f169cd64d0 2023-07-14T16:29:48.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc64ee9b80 10 client.0 _ll_register_callbacks cb 0x55f169bc5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:48.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc4a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd6e20 0x55f169ce03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd7c70 0x55f169ce7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc4a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd6e20 0x55f169ce03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46424/0 (socket says 172.21.15.62:46424) 2023-07-14T16:29:48.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc4a7fc700 1 -- 172.21.15.62:0/2888296505 learned_addr learned my addr 172.21.15.62:0/2888296505 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:48.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd7c70 0x55f169ce7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36122/0 (socket says 172.21.15.62:36122) 2023-07-14T16:29:48.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.103+0000 7fdc4affd700 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169cdfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc4a7fc700 1 -- 172.21.15.62:0/2888296505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 msgr2=0x55f169cdfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:48.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169cdfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc4a7fc700 1 -- 172.21.15.62:0/2888296505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd7c70 msgr2=0x55f169ce7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:48.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd7c70 0x55f169ce7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc4a7fc700 1 -- 172.21.15.62:0/2888296505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c0098e0 con 0x55f169cd6e20 2023-07-14T16:29:48.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd6e20 0x55f169ce03f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000f9b0 tx=0x7fdc400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e22a54b7099cae3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:48.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:48.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc2eff5700 1 -- 172.21.15.62:0/2888296505 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4001f820 con 0x55f169cd6e20 2023-07-14T16:29:48.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc2eff5700 1 -- 172.21.15.62:0/2888296505 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc4001f980 con 0x55f169cd6e20 2023-07-14T16:29:48.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.104+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2888296505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f169b814f0 con 0x55f169cd6e20 2023-07-14T16:29:48.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc2eff5700 1 -- 172.21.15.62:0/2888296505 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc4000fe70 con 0x55f169cd6e20 2023-07-14T16:29:48.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc64ee9b80 10 client.19503 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:48.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2888296505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fdc4c19ff10 con 0x55f169cd6e20 2023-07-14T16:29:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc2eff5700 1 -- 172.21.15.62:0/2888296505 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f169b814f0 con 0x55f169cd6e20 2023-07-14T16:29:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc2eff5700 1 -- 172.21.15.62:0/2888296505 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc40031070 con 0x55f169cd6e20 2023-07-14T16:29:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc64ee9b80 10 client.19503 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc64ee9b80 10 client.19503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc64ee9b80 20 client.19503 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169cebd00 0x55f169cee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc64ee9b80 4 client.19503 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:48.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.105+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2888296505 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f169c20eb0 con 0x55f169cebd00 2023-07-14T16:29:48.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.106+0000 7fdc4affd700 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169cebd00 0x55f169cee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.107+0000 7fdc4affd700 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169cebd00 0x55f169cee110 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bfce01bbdc7dced server_cookie=28eea3c572ca14e0 in_seq=0 out_seq=0 2023-07-14T16:29:48.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.107+0000 7fdc2eff5700 10 client.19503 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.107+0000 7fdc2eff5700 1 -- 172.21.15.62:0/2888296505 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f169c20eb0 con 0x55f169cebd00 2023-07-14T16:29:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.107+0000 7fdc2eff5700 10 client.19503 handle_command_reply: tid=0 2023-07-14T16:29:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.107+0000 7fdc64ee9b80 1 client.19503 shutdown 2023-07-14T16:29:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2888296505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd6e20 msgr2=0x55f169ce03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd6e20 0x55f169ce03f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000f9b0 tx=0x7fdc400047c0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2888296505 shutdown_connections 2023-07-14T16:29:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169cebd00 0x55f169cee110 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd7c70 0x55f169ce7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169cdfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2888296505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd6e20 0x55f169ce03f0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2888296505 >> 172.21.15.62:0/2888296505 conn(0x55f169ccbda0 msgr2=0x55f169cd08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc2eff5700 0 client.19503 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2888296505 shutdown_connections 2023-07-14T16:29:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2888296505 wait complete. 2023-07-14T16:29:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.108+0000 7fdc64ee9b80 20 client.19503 trim_cache size 0 max 16384 2023-07-14T16:29:48.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.122+0000 7fdc64ee9b80 1 Processor -- start 2023-07-14T16:29:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.122+0000 7fdc64ee9b80 1 -- start start 2023-07-14T16:29:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.122+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169cd2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.122+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.122+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cf2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.122+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169cdfa20 con 0x55f169cd7c70 2023-07-14T16:29:48.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.122+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169cd71f0 con 0x55f169cd6e20 2023-07-14T16:29:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.122+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169ce0d80 con 0x55f169cd64d0 2023-07-14T16:29:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cf2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cf2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46426/0 (socket says 172.21.15.62:46426) 2023-07-14T16:29:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/2647615652 learned_addr learned my addr 172.21.15.62:0/2647615652 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:48.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4affd700 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169cd2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/2647615652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 msgr2=0x55f169cd2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:48.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169cd2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/2647615652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 msgr2=0x55f169ce73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:48.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/2647615652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f169a3ec60 con 0x55f169cd7c70 2023-07-14T16:29:48.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.123+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cf2a30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fdc440972f0 tx=0x7fdc4409b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5db877be7cc6593 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc31ffb700 1 -- 172.21.15.62:0/2647615652 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc440b9170 con 0x55f169cd7c70 2023-07-14T16:29:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc31ffb700 1 -- 172.21.15.62:0/2647615652 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc4409adf0 con 0x55f169cd7c70 2023-07-14T16:29:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc31ffb700 1 -- 172.21.15.62:0/2647615652 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4409a540 con 0x55f169cd7c70 2023-07-14T16:29:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2647615652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 msgr2=0x55f169cf2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cf2a30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fdc440972f0 tx=0x7fdc4409b050 comp rx=0 tx=0).stop 2023-07-14T16:29:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2647615652 shutdown_connections 2023-07-14T16:29:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169cf2a30 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169cd2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/2647615652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2647615652 >> 172.21.15.62:0/2647615652 conn(0x55f169ccbda0 msgr2=0x55f169cd11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2647615652 shutdown_connections 2023-07-14T16:29:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.124+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/2647615652 wait complete. 2023-07-14T16:29:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.125+0000 7fdc64ee9b80 1 -- start start 2023-07-14T16:29:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.125+0000 7fdc64ee9b80 1 Processor -- start 2023-07-14T16:29:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169ce5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc64ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169ce6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169ce6820 con 0x55f169cd7c70 2023-07-14T16:29:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169ce6990 con 0x55f169cd6e20 2023-07-14T16:29:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc64ee9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f169ce6b50 con 0x55f169cd64d0 2023-07-14T16:29:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc64ee9b80 10 client.0 _ll_register_callbacks cb 0x7fdc4000f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169ce6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc4b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169ce6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46430/0 (socket says 172.21.15.62:46430) 2023-07-14T16:29:48.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/3722083934 learned_addr learned my addr 172.21.15.62:0/3722083934 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:48.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.126+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4affd700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169ce5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/3722083934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 msgr2=0x55f169ce5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:48.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169ce5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 msgr2=0x55f169ce5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:48.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4b7fe700 1 -- 172.21.15.62:0/3722083934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f169a55de0 con 0x55f169cd7c70 2023-07-14T16:29:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4affd700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169ce5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4a7fc700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc4b7fe700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169ce6110 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdc440cb7c0 tx=0x7fdc4409a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b69ec1e2088cb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc2eff5700 1 -- 172.21.15.62:0/3722083934 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4409abc0 con 0x55f169cd7c70 2023-07-14T16:29:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/3722083934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc4409ee20 con 0x55f169cd7c70 2023-07-14T16:29:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.127+0000 7fdc2eff5700 1 -- 172.21.15.62:0/3722083934 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc440a31b0 con 0x55f169cd7c70 2023-07-14T16:29:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc2eff5700 1 -- 172.21.15.62:0/3722083934 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc440a2d20 con 0x55f169cd7c70 2023-07-14T16:29:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc64ee9b80 10 client.19509 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/3722083934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f169a7c250 con 0x55f169cd7c70 2023-07-14T16:29:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc2eff5700 1 -- 172.21.15.62:0/3722083934 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc4407d080 con 0x55f169cd7c70 2023-07-14T16:29:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc2eff5700 1 -- 172.21.15.62:0/3722083934 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc44099d50 con 0x55f169cd7c70 2023-07-14T16:29:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc64ee9b80 10 client.19509 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:48.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc64ee9b80 10 client.19509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:48.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc64ee9b80 20 client.19509 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:48.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169d401c0 0x55f169d425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:48.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc64ee9b80 4 client.19509 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:29:48.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.128+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/3722083934 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f169be3050 con 0x55f169d401c0 2023-07-14T16:29:48.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.129+0000 7fdc4affd700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169d401c0 0x55f169d425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:48.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.129+0000 7fdc4affd700 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169d401c0 0x55f169d425d0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9719f6adf6f350d server_cookie=ea256884d34fc61c in_seq=0 out_seq=0 2023-07-14T16:29:48.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.129+0000 7fdc2eff5700 10 client.19509 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:48.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc2eff5700 1 -- 172.21.15.62:0/3722083934 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f169be3050 con 0x55f169d401c0 2023-07-14T16:29:48.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc2eff5700 10 client.19509 handle_command_reply: tid=0 2023-07-14T16:29:48.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 client.19509 shutdown 2023-07-14T16:29:48.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/3722083934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 msgr2=0x55f169ce6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169ce6110 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdc440cb7c0 tx=0x7fdc4409a610 comp rx=0 tx=0).stop 2023-07-14T16:29:48.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/3722083934 shutdown_connections 2023-07-14T16:29:48.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f169cd7c70 0x55f169ce6110 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f169cd64d0 0x55f169ce5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f169d401c0 0x55f169d425d0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 --2- 172.21.15.62:0/3722083934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f169cd6e20 0x55f169ce5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/3722083934 >> 172.21.15.62:0/3722083934 conn(0x55f169ccbda0 msgr2=0x55f169cd1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:48.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc2eff5700 0 client.19509 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:48.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/3722083934 shutdown_connections 2023-07-14T16:29:48.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 1 -- 172.21.15.62:0/3722083934 wait complete. 2023-07-14T16:29:48.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.130+0000 7fdc64ee9b80 20 client.19509 trim_cache size 0 max 16384 2023-07-14T16:29:48.115 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:29:48.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 -- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdc340a1150 msgr2=0x7fdc340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdc340a1150 0x7fdc340a3600 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009c70 tx=0x7fdc40011720 comp rx=0 tx=0).stop 2023-07-14T16:29:48.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 -- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 msgr2=0x7fdc4c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:48.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 0x7fdc4c19efd0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400d010 tx=0x7fdc4400bcc0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 -- 172.21.15.62:0/785855897 shutdown_connections 2023-07-14T16:29:48.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc4c1057a0 0x7fdc4c19efd0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc4c100560 0x7fdc4c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdc340a1150 0x7fdc340a3600 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 --2- 172.21.15.62:0/785855897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc4c102e80 0x7fdc4c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:48.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 -- 172.21.15.62:0/785855897 >> 172.21.15.62:0/785855897 conn(0x7fdc4c0fa000 msgr2=0x7fdc4c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:48.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 -- 172.21.15.62:0/785855897 shutdown_connections 2023-07-14T16:29:48.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:48.131+0000 7fdc515ff700 1 -- 172.21.15.62:0/785855897 wait complete. 2023-07-14T16:29:48.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:47 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3065959807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:29:48.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:47 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3065959807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:29:48.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:47 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3065959807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:29:49.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:48 smithi062 ceph-mon[95543]: pgmap v782: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:29:49.124 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:29:49.125 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:29:49.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:48 smithi081 ceph-mon[127795]: pgmap v782: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:29:49.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:48 smithi164 ceph-mon[97427]: pgmap v782: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:29:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 -- 172.21.15.62:0/1895790011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 msgr2=0x7f4e70107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1895790011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 0x7f4e70107c00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b0a0 tx=0x7f4e6800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 -- 172.21.15.62:0/1895790011 shutdown_connections 2023-07-14T16:29:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1895790011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 0x7f4e70107c00 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1895790011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70100560 0x7f4e70102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1895790011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e70102e80 0x7f4e70105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 -- 172.21.15.62:0/1895790011 >> 172.21.15.62:0/1895790011 conn(0x7f4e700fa000 msgr2=0x7f4e700fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 -- 172.21.15.62:0/1895790011 shutdown_connections 2023-07-14T16:29:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.400+0000 7f4e74fff700 1 -- 172.21.15.62:0/1895790011 wait complete. 2023-07-14T16:29:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.401+0000 7f4e74fff700 1 Processor -- start 2023-07-14T16:29:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.402+0000 7f4e74fff700 1 -- start start 2023-07-14T16:29:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.402+0000 7f4e74fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70100560 0x7f4e700ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.402+0000 7f4e74fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e70102e80 0x7f4e700fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.402+0000 7f4e74fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 0x7f4e700fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.402+0000 7f4e74fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700ff1f0 con 0x7f4e701057a0 2023-07-14T16:29:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.402+0000 7f4e74fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700ff360 con 0x7f4e70100560 2023-07-14T16:29:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.402+0000 7f4e74fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7019f880 con 0x7f4e70102e80 2023-07-14T16:29:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 0x7f4e700fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70100560 0x7f4e700ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 0x7f4e700fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46446/0 (socket says 172.21.15.62:46446) 2023-07-14T16:29:49.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 -- 172.21.15.62:0/1164255373 learned_addr learned my addr 172.21.15.62:0/1164255373 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e70102e80 0x7f4e700fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 -- 172.21.15.62:0/1164255373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e70102e80 msgr2=0x7f4e700fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.383 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e70102e80 0x7f4e700fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 -- 172.21.15.62:0/1164255373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70100560 msgr2=0x7f4e700ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70100560 0x7f4e700ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 -- 172.21.15.62:0/1164255373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68009d00 con 0x7f4e701057a0 2023-07-14T16:29:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70100560 0x7f4e700ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.403+0000 7f4e6ed9d700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 0x7f4e700fea50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800d010 tx=0x7f4e6800bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.404+0000 7f4e5f7fe700 1 -- 172.21.15.62:0/1164255373 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800e040 con 0x7f4e701057a0 2023-07-14T16:29:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.404+0000 7f4e5f7fe700 1 -- 172.21.15.62:0/1164255373 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e68009480 con 0x7f4e701057a0 2023-07-14T16:29:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.404+0000 7f4e74fff700 1 -- 172.21.15.62:0/1164255373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7019fb00 con 0x7f4e701057a0 2023-07-14T16:29:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.404+0000 7f4e5f7fe700 1 -- 172.21.15.62:0/1164255373 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680052a0 con 0x7f4e701057a0 2023-07-14T16:29:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.404+0000 7f4e74fff700 1 -- 172.21.15.62:0/1164255373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7019ff80 con 0x7f4e701057a0 2023-07-14T16:29:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.408+0000 7f4e5f7fe700 1 -- 172.21.15.62:0/1164255373 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4e68005400 con 0x7f4e701057a0 2023-07-14T16:29:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.410+0000 7f4e5f7fe700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e580a11d0 0x7f4e580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.410+0000 7f4e5f7fe700 1 -- 172.21.15.62:0/1164255373 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4e680cda30 con 0x7f4e701057a0 2023-07-14T16:29:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.410+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e580a11d0 0x7f4e580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.410+0000 7f4e888eab80 1 Processor -- start 2023-07-14T16:29:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.411+0000 7f4e888eab80 1 -- start start 2023-07-14T16:29:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.411+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e580a11d0 0x7f4e580a3680 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4e60011490 tx=0x7f4e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.411+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e66170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.411+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e666b0 0x55e2e7e66f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.411+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e67500 0x55e2e7e67970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.411+0000 7f4e888eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7d62d30 con 0x55e2e7e67500 2023-07-14T16:29:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.411+0000 7f4e888eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7db5ac0 con 0x55e2e7e65d60 2023-07-14T16:29:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.411+0000 7f4e888eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7daef50 con 0x55e2e7e666b0 2023-07-14T16:29:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e67500 0x55e2e7e67970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e66170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6dd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e666b0 0x55e2e7e66f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e66170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36174/0 (socket says 172.21.15.62:36174) 2023-07-14T16:29:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6e59c700 1 -- 172.21.15.62:0/3179842941 learned_addr learned my addr 172.21.15.62:0/3179842941 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6e59c700 1 -- 172.21.15.62:0/3179842941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e666b0 msgr2=0x55e2e7e66f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/3179842941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e666b0 0x55e2e7e66f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6e59c700 1 -- 172.21.15.62:0/3179842941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e67500 msgr2=0x55e2e7e67970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/3179842941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e67500 0x55e2e7e67970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6e59c700 1 -- 172.21.15.62:0/3179842941 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e7bdce60 con 0x55e2e7e65d60 2023-07-14T16:29:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.412+0000 7f4e6ed9d700 1 --2- 172.21.15.62:0/3179842941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e67500 0x55e2e7e67970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.413+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/3179842941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e66170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e600043c0 tx=0x7f4e60014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af62d4514539d527 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.413+0000 7f4e5cff9700 1 -- 172.21.15.62:0/3179842941 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60003d10 con 0x55e2e7e65d60 2023-07-14T16:29:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.413+0000 7f4e5cff9700 1 -- 172.21.15.62:0/3179842941 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e60003e70 con 0x55e2e7e65d60 2023-07-14T16:29:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.413+0000 7f4e5cff9700 1 -- 172.21.15.62:0/3179842941 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6000e730 con 0x55e2e7e65d60 2023-07-14T16:29:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 -- 172.21.15.62:0/3179842941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 msgr2=0x55e2e7e66170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 --2- 172.21.15.62:0/3179842941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e66170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e600043c0 tx=0x7f4e60014000 comp rx=0 tx=0).stop 2023-07-14T16:29:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 -- 172.21.15.62:0/3179842941 shutdown_connections 2023-07-14T16:29:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 --2- 172.21.15.62:0/3179842941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e67500 0x55e2e7e67970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 --2- 172.21.15.62:0/3179842941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e66170 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.392 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 --2- 172.21.15.62:0/3179842941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e666b0 0x55e2e7e66f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 -- 172.21.15.62:0/3179842941 >> 172.21.15.62:0/3179842941 conn(0x55e2e7e52e20 msgr2=0x55e2e7e58250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 -- 172.21.15.62:0/3179842941 shutdown_connections 2023-07-14T16:29:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.414+0000 7f4e888eab80 1 -- 172.21.15.62:0/3179842941 wait complete. 2023-07-14T16:29:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.415+0000 7f4e888eab80 1 -- start start 2023-07-14T16:29:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.415+0000 7f4e888eab80 1 Processor -- start 2023-07-14T16:29:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e6ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e67500 0x55e2e7e76f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e888eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7d4eb40 con 0x55e2e7e666b0 2023-07-14T16:29:49.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e888eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7d4e840 con 0x55e2e7e67500 2023-07-14T16:29:49.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e888eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7c329a0 con 0x55e2e7e65d60 2023-07-14T16:29:49.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e888eab80 10 client.0 _ll_register_callbacks cb 0x55e2e7d4c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:49.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e6dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e6ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e6dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e6ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46456/0 (socket says 172.21.15.62:46456) 2023-07-14T16:29:49.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e6dd9b700 1 -- 172.21.15.62:0/875859394 learned_addr learned my addr 172.21.15.62:0/875859394 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.416+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6ed9d700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e67500 0x55e2e7e76f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6dd9b700 1 -- 172.21.15.62:0/875859394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e65d60 msgr2=0x55e2e7e6fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6dd9b700 1 -- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e67500 msgr2=0x55e2e7e76f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e67500 0x55e2e7e76f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6dd9b700 1 -- 172.21.15.62:0/875859394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7019ff80 con 0x55e2e7e666b0 2023-07-14T16:29:49.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:49.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6ed9d700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e67500 0x55e2e7e76f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:49.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e6ff40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f4e64012810 tx=0x7f4e64012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14ad5e86b1a3fb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.417+0000 7f4e55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:49.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.418+0000 7f4e55ffb700 1 -- 172.21.15.62:0/875859394 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64017070 con 0x55e2e7e666b0 2023-07-14T16:29:49.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.418+0000 7f4e888eab80 1 -- 172.21.15.62:0/875859394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2e7d08570 con 0x55e2e7e666b0 2023-07-14T16:29:49.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.418+0000 7f4e55ffb700 1 -- 172.21.15.62:0/875859394 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4e64011710 con 0x55e2e7e666b0 2023-07-14T16:29:49.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.418+0000 7f4e55ffb700 1 -- 172.21.15.62:0/875859394 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e640118d0 con 0x55e2e7e666b0 2023-07-14T16:29:49.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.418+0000 7f4e888eab80 10 client.19521 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:49.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.418+0000 7f4e888eab80 1 -- 172.21.15.62:0/875859394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4e60017070 con 0x55e2e7e666b0 2023-07-14T16:29:49.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.418+0000 7f4e55ffb700 1 -- 172.21.15.62:0/875859394 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e2e7d08570 con 0x55e2e7e666b0 2023-07-14T16:29:49.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.419+0000 7f4e55ffb700 1 -- 172.21.15.62:0/875859394 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4e6400fbb0 con 0x55e2e7e666b0 2023-07-14T16:29:49.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.419+0000 7f4e888eab80 10 client.19521 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:49.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.419+0000 7f4e888eab80 10 client.19521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:49.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.419+0000 7f4e888eab80 20 client.19521 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:49.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.419+0000 7f4e888eab80 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e2e7e7b790 0x55e2e7e7dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.419+0000 7f4e888eab80 4 client.19521 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:49.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.419+0000 7f4e888eab80 1 -- 172.21.15.62:0/875859394 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2e7da7f30 con 0x55e2e7e7b790 2023-07-14T16:29:49.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.419+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e2e7e7b790 0x55e2e7e7dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.420+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e2e7e7b790 0x55e2e7e7dba0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86eb4a05671d6cd server_cookie=784610ce9edfe1f in_seq=0 out_seq=0 2023-07-14T16:29:49.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.420+0000 7f4e55ffb700 10 client.19521 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:49.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e55ffb700 1 -- 172.21.15.62:0/875859394 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e2e7da7f30 con 0x55e2e7e7b790 2023-07-14T16:29:49.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e55ffb700 10 client.19521 handle_command_reply: tid=0 2023-07-14T16:29:49.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 client.19521 shutdown 2023-07-14T16:29:49.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 -- 172.21.15.62:0/875859394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 msgr2=0x55e2e7e6ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e6ff40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f4e64012810 tx=0x7f4e64012b20 comp rx=0 tx=0).stop 2023-07-14T16:29:49.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 -- 172.21.15.62:0/875859394 shutdown_connections 2023-07-14T16:29:49.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e2e7e7b790 0x55e2e7e7dba0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e67500 0x55e2e7e76f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 --2- 172.21.15.62:0/875859394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e6ff40 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 -- 172.21.15.62:0/875859394 >> 172.21.15.62:0/875859394 conn(0x55e2e7e52e20 msgr2=0x55e2e7e57930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e55ffb700 0 client.19521 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:49.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 -- 172.21.15.62:0/875859394 shutdown_connections 2023-07-14T16:29:49.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.421+0000 7f4e888eab80 1 -- 172.21.15.62:0/875859394 wait complete. 2023-07-14T16:29:49.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.422+0000 7f4e888eab80 20 client.19521 trim_cache size 0 max 16384 2023-07-14T16:29:49.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.446+0000 7f4e888eab80 1 Processor -- start 2023-07-14T16:29:49.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.446+0000 7f4e888eab80 1 -- start start 2023-07-14T16:29:49.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.446+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.446+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e76f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.446+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 0x55e2e7e6ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.446+0000 7f4e888eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7e6f570 con 0x55e2e7e666b0 2023-07-14T16:29:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.446+0000 7f4e888eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7e590f0 con 0x55e2e7e65d60 2023-07-14T16:29:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.446+0000 7f4e888eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7e592f0 con 0x55e2e7e67500 2023-07-14T16:29:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e76f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e76f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46460/0 (socket says 172.21.15.62:46460) 2023-07-14T16:29:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 -- 172.21.15.62:0/3914626355 learned_addr learned my addr 172.21.15.62:0/3914626355 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6ed9d700 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 0x55e2e7e6ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 -- 172.21.15.62:0/3914626355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 msgr2=0x55e2e7e6ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 0x55e2e7e6ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 -- 172.21.15.62:0/3914626355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 msgr2=0x55e2e7e6d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 -- 172.21.15.62:0/3914626355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e7bc5ce0 con 0x55e2e7e666b0 2023-07-14T16:29:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.447+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e76f40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e640092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de354137fbc7f69b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e5d7fa700 1 -- 172.21.15.62:0/3914626355 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64017070 con 0x55e2e7e666b0 2023-07-14T16:29:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e5d7fa700 1 -- 172.21.15.62:0/3914626355 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e64018040 con 0x55e2e7e666b0 2023-07-14T16:29:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e5d7fa700 1 -- 172.21.15.62:0/3914626355 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6401cc70 con 0x55e2e7e666b0 2023-07-14T16:29:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e888eab80 1 -- 172.21.15.62:0/3914626355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 msgr2=0x55e2e7e76f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e888eab80 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e76f40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e640092d0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e888eab80 1 -- 172.21.15.62:0/3914626355 shutdown_connections 2023-07-14T16:29:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e888eab80 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 0x55e2e7e6ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e888eab80 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e65d60 0x55e2e7e6d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e888eab80 1 --2- 172.21.15.62:0/3914626355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e666b0 0x55e2e7e76f40 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.448+0000 7f4e888eab80 1 -- 172.21.15.62:0/3914626355 >> 172.21.15.62:0/3914626355 conn(0x55e2e7e52e20 msgr2=0x55e2e7e77870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.449+0000 7f4e888eab80 1 -- 172.21.15.62:0/3914626355 shutdown_connections 2023-07-14T16:29:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.449+0000 7f4e888eab80 1 -- 172.21.15.62:0/3914626355 wait complete. 2023-07-14T16:29:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.449+0000 7f4e888eab80 1 -- start start 2023-07-14T16:29:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.449+0000 7f4e888eab80 1 Processor -- start 2023-07-14T16:29:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e65d60 0x55e2e7e72f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e666b0 0x55e2e7e73460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e888eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 0x55e2e7e76480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e888eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7e769c0 con 0x55e2e7e65d60 2023-07-14T16:29:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e888eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7e76b30 con 0x55e2e7e666b0 2023-07-14T16:29:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e888eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e7e76cf0 con 0x55e2e7e67500 2023-07-14T16:29:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e888eab80 10 client.0 _ll_register_callbacks cb 0x7f4e64009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e65d60 0x55e2e7e72f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e6e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e65d60 0x55e2e7e72f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46464/0 (socket says 172.21.15.62:46464) 2023-07-14T16:29:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e6e59c700 1 -- 172.21.15.62:0/2426535911 learned_addr learned my addr 172.21.15.62:0/2426535911 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e6ed9d700 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 0x55e2e7e76480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.450+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e666b0 0x55e2e7e73460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e6e59c700 1 -- 172.21.15.62:0/2426535911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 msgr2=0x55e2e7e76480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 0x55e2e7e76480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e6e59c700 1 -- 172.21.15.62:0/2426535911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e666b0 msgr2=0x55e2e7e73460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e666b0 0x55e2e7e73460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e6e59c700 1 -- 172.21.15.62:0/2426535911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e64012e70 con 0x55e2e7e65d60 2023-07-14T16:29:49.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e6e59c700 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e65d60 0x55e2e7e72f20 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4e60000c00 tx=0x7f4e60005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5820232b6b4ace11 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:49.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e55ffb700 1 -- 172.21.15.62:0/2426535911 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6002c040 con 0x55e2e7e65d60 2023-07-14T16:29:49.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e55ffb700 1 -- 172.21.15.62:0/2426535911 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4e60033bd0 con 0x55e2e7e65d60 2023-07-14T16:29:49.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.451+0000 7f4e888eab80 1 -- 172.21.15.62:0/2426535911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e60003e70 con 0x55e2e7e65d60 2023-07-14T16:29:49.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e55ffb700 1 -- 172.21.15.62:0/2426535911 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e60033d90 con 0x55e2e7e65d60 2023-07-14T16:29:49.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e888eab80 10 client.19527 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:49.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e888eab80 1 -- 172.21.15.62:0/2426535911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e2e7c032d0 con 0x55e2e7e65d60 2023-07-14T16:29:49.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e55ffb700 1 -- 172.21.15.62:0/2426535911 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e60003e70 con 0x55e2e7e65d60 2023-07-14T16:29:49.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e55ffb700 1 -- 172.21.15.62:0/2426535911 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4e60024020 con 0x55e2e7e65d60 2023-07-14T16:29:49.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e888eab80 10 client.19527 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:49.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e888eab80 10 client.19527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:49.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e888eab80 20 client.19527 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:49.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e888eab80 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e2e7ed0740 0x55e2e7ed2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e888eab80 4 client.19527 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:29:49.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.452+0000 7f4e888eab80 1 -- 172.21.15.62:0/2426535911 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e2e7d6a0d0 con 0x55e2e7ed0740 2023-07-14T16:29:49.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.453+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e2e7ed0740 0x55e2e7ed2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.453+0000 7f4e6dd9b700 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e2e7ed0740 0x55e2e7ed2b50 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbd55976f879617 server_cookie=6bfe8a4a785b5a0 in_seq=0 out_seq=0 2023-07-14T16:29:49.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.453+0000 7f4e55ffb700 10 client.19527 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:49.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.454+0000 7f4e55ffb700 1 -- 172.21.15.62:0/2426535911 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e2e7d6a0d0 con 0x55e2e7ed0740 2023-07-14T16:29:49.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.454+0000 7f4e55ffb700 10 client.19527 handle_command_reply: tid=0 2023-07-14T16:29:49.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.454+0000 7f4e888eab80 1 client.19527 shutdown 2023-07-14T16:29:49.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 -- 172.21.15.62:0/2426535911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e65d60 msgr2=0x55e2e7e72f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e65d60 0x55e2e7e72f20 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4e60000c00 tx=0x7f4e60005c80 comp rx=0 tx=0).stop 2023-07-14T16:29:49.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 -- 172.21.15.62:0/2426535911 shutdown_connections 2023-07-14T16:29:49.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e2e7e67500 0x55e2e7e76480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2e7e65d60 0x55e2e7e72f20 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e2e7ed0740 0x55e2e7ed2b50 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 --2- 172.21.15.62:0/2426535911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2e7e666b0 0x55e2e7e73460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 -- 172.21.15.62:0/2426535911 >> 172.21.15.62:0/2426535911 conn(0x55e2e7e52e20 msgr2=0x55e2e7e6d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e55ffb700 0 client.19527 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:49.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 -- 172.21.15.62:0/2426535911 shutdown_connections 2023-07-14T16:29:49.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 1 -- 172.21.15.62:0/2426535911 wait complete. 2023-07-14T16:29:49.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.455+0000 7f4e888eab80 20 client.19527 trim_cache size 0 max 16384 2023-07-14T16:29:49.443 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:29:49.444 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:29:49.444 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "cbd9fbce-74b5-4539-8234-575dbcfd80a7", 2023-07-14T16:29:49.444 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:29:49.444 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:29:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 -- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e580a11d0 msgr2=0x7f4e580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e580a11d0 0x7f4e580a3680 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4e60011490 tx=0x7f4e60009260 comp rx=0 tx=0).stop 2023-07-14T16:29:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 -- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 msgr2=0x7f4e700fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 0x7f4e700fea50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800d010 tx=0x7f4e6800bcc0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 -- 172.21.15.62:0/1164255373 shutdown_connections 2023-07-14T16:29:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e701057a0 0x7f4e700fea50 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4e580a11d0 0x7f4e580a3680 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e70100560 0x7f4e700ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 --2- 172.21.15.62:0/1164255373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e70102e80 0x7f4e700fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 -- 172.21.15.62:0/1164255373 >> 172.21.15.62:0/1164255373 conn(0x7f4e700fa000 msgr2=0x7f4e701011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 -- 172.21.15.62:0/1164255373 shutdown_connections 2023-07-14T16:29:49.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.456+0000 7f4e74fff700 1 -- 172.21.15.62:0/1164255373 wait complete. 2023-07-14T16:29:49.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:29:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.725+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/2228299723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea40744b0 msgr2=0x7f2ea40729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.725+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/2228299723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea40744b0 0x7f2ea40729a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009990 tx=0x7f2e94009ca0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.725+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/2228299723 shutdown_connections 2023-07-14T16:29:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.725+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/2228299723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ea4107850 0x7f2ea4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.725+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/2228299723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea40744b0 0x7f2ea40729a0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.725+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/2228299723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea4072ee0 0x7f2ea4073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.725+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/2228299723 >> 172.21.15.62:0/2228299723 conn(0x7f2ea40fb010 msgr2=0x7f2ea40fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.726+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/2228299723 shutdown_connections 2023-07-14T16:29:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.726+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/2228299723 wait complete. 2023-07-14T16:29:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea8b9d700 1 Processor -- start 2023-07-14T16:29:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea8b9d700 1 -- start start 2023-07-14T16:29:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea8b9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ea4072ee0 0x7f2ea41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea8b9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea40744b0 0x7f2ea4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea8b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea4107850 0x7f2ea419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea8b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea419f270 con 0x7f2ea4107850 2023-07-14T16:29:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea8b9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea419f3b0 con 0x7f2ea4072ee0 2023-07-14T16:29:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea8b9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea419f540 con 0x7f2ea40744b0 2023-07-14T16:29:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.727+0000 7f2ea3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea4107850 0x7f2ea419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ea4072ee0 0x7f2ea41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea4107850 0x7f2ea419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46490/0 (socket says 172.21.15.62:46490) 2023-07-14T16:29:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea40744b0 0x7f2ea4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea3fff700 1 -- 172.21.15.62:0/1360017830 learned_addr learned my addr 172.21.15.62:0/1360017830 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea2ffd700 1 -- 172.21.15.62:0/1360017830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ea4072ee0 msgr2=0x7f2ea41976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ea4072ee0 0x7f2ea41976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea2ffd700 1 -- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea4107850 msgr2=0x7f2ea419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea4107850 0x7f2ea419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea2ffd700 1 -- 172.21.15.62:0/1360017830 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e940095f0 con 0x7f2ea40744b0 2023-07-14T16:29:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.728+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea40744b0 0x7f2ea4197c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800e9f0 tx=0x7f2e9800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.729+0000 7f2ea0ff9700 1 -- 172.21.15.62:0/1360017830 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800c9c0 con 0x7f2ea40744b0 2023-07-14T16:29:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.729+0000 7f2ea0ff9700 1 -- 172.21.15.62:0/1360017830 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e98010430 con 0x7f2ea40744b0 2023-07-14T16:29:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.729+0000 7f2ea0ff9700 1 -- 172.21.15.62:0/1360017830 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98018680 con 0x7f2ea40744b0 2023-07-14T16:29:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.729+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/1360017830 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea419f820 con 0x7f2ea40744b0 2023-07-14T16:29:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.729+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/1360017830 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea419fd30 con 0x7f2ea40744b0 2023-07-14T16:29:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.732+0000 7f2ea0ff9700 1 -- 172.21.15.62:0/1360017830 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2e980105a0 con 0x7f2ea40744b0 2023-07-14T16:29:49.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.734+0000 7f2ea0ff9700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e8c0a1270 0x7f2e8c0a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.734+0000 7f2ea0ff9700 1 -- 172.21.15.62:0/1360017830 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2e98014070 con 0x7f2ea40744b0 2023-07-14T16:29:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.734+0000 7f2ebd7a7b80 1 Processor -- start 2023-07-14T16:29:49.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.734+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e8c0a1270 0x7f2e8c0a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ebd7a7b80 1 -- start start 2023-07-14T16:29:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 0x5614a7b21b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22070 0x5614a7b228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7a26cb0 con 0x5614a7b22070 2023-07-14T16:29:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7a79a40 con 0x5614a7b21720 2023-07-14T16:29:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7a72ed0 con 0x5614a7b22e30 2023-07-14T16:29:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e8c0a1270 0x7f2e8c0a3720 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2e94006010 tx=0x7f2e94005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.735+0000 7f2ea2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22070 0x5614a7b228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22070 0x5614a7b228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46500/0 (socket says 172.21.15.62:46500) 2023-07-14T16:29:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea2ffd700 1 -- 172.21.15.62:0/984037588 learned_addr learned my addr 172.21.15.62:0/984037588 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 0x5614a7b21b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea2ffd700 1 -- 172.21.15.62:0/984037588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 msgr2=0x5614a7b232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea2ffd700 1 -- 172.21.15.62:0/984037588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 msgr2=0x5614a7b21b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 0x5614a7b21b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea2ffd700 1 -- 172.21.15.62:0/984037588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614a78a0de0 con 0x5614a7b22070 2023-07-14T16:29:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22070 0x5614a7b228a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2e98099d50 tx=0x7f2e98099c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354a2d3c5c990b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.736+0000 7f2e89ffb700 1 -- 172.21.15.62:0/984037588 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98010880 con 0x5614a7b22070 2023-07-14T16:29:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2e89ffb700 1 -- 172.21.15.62:0/984037588 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e98099dc0 con 0x5614a7b22070 2023-07-14T16:29:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2e89ffb700 1 -- 172.21.15.62:0/984037588 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98099510 con 0x5614a7b22070 2023-07-14T16:29:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/984037588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22070 msgr2=0x5614a7b228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22070 0x5614a7b228a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2e98099d50 tx=0x7f2e98099c10 comp rx=0 tx=0).stop 2023-07-14T16:29:49.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/984037588 shutdown_connections 2023-07-14T16:29:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 0x5614a7b21b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/984037588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22070 0x5614a7b228a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.737+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/984037588 >> 172.21.15.62:0/984037588 conn(0x5614a7b16ff0 msgr2=0x5614a7b1c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.738+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/984037588 shutdown_connections 2023-07-14T16:29:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.738+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/984037588 wait complete. 2023-07-14T16:29:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.738+0000 7f2ebd7a7b80 1 -- start start 2023-07-14T16:29:49.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.738+0000 7f2ebd7a7b80 1 Processor -- start 2023-07-14T16:29:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.739+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 0x5614a7b2b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.739+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22070 0x5614a7b2b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.739+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b32880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.739+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7a12ac0 con 0x5614a7b22e30 2023-07-14T16:29:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.739+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7a127c0 con 0x5614a7b21720 2023-07-14T16:29:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.739+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a78f6920 con 0x5614a7b22070 2023-07-14T16:29:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.739+0000 7f2ebd7a7b80 10 client.0 _ll_register_callbacks cb 0x5614a7a10760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.739+0000 7f2ea3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b32880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22070 0x5614a7b2b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b32880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46502/0 (socket says 172.21.15.62:46502) 2023-07-14T16:29:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3003650011 learned_addr learned my addr 172.21.15.62:0/3003650011 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 0x5614a7b2b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3003650011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22070 msgr2=0x5614a7b2b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22070 0x5614a7b2b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3003650011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 msgr2=0x5614a7b2b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 0x5614a7b2b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea3fff700 1 -- 172.21.15.62:0/3003650011 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea419fd30 con 0x5614a7b22e30 2023-07-14T16:29:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b32880 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c007f70 tx=0x7f2e9c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77357142ce9c4db server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.740+0000 7f2e82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:29:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2e82ffd700 1 -- 172.21.15.62:0/3003650011 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c003a00 con 0x5614a7b22e30 2023-07-14T16:29:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/3003650011 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614a79cc4f0 con 0x5614a7b22e30 2023-07-14T16:29:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2e82ffd700 1 -- 172.21.15.62:0/3003650011 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2e9c003b60 con 0x5614a7b22e30 2023-07-14T16:29:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2e82ffd700 1 -- 172.21.15.62:0/3003650011 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e9c003d20 con 0x5614a7b22e30 2023-07-14T16:29:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2ebd7a7b80 10 client.19539 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/3003650011 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5614a7889c60 con 0x5614a7b22e30 2023-07-14T16:29:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2e82ffd700 1 -- 172.21.15.62:0/3003650011 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5614a79cc4f0 con 0x5614a7b22e30 2023-07-14T16:29:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2e82ffd700 1 -- 172.21.15.62:0/3003650011 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2e9c004370 con 0x5614a7b22e30 2023-07-14T16:29:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2ebd7a7b80 10 client.19539 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2ebd7a7b80 10 client.19539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.741+0000 7f2ebd7a7b80 20 client.19539 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.742+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5614a7b37180 0x5614a7b39590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.742+0000 7f2ebd7a7b80 4 client.19539 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.742+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/3003650011 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614a7a6beb0 con 0x5614a7b37180 2023-07-14T16:29:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.742+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5614a7b37180 0x5614a7b39590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.743+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5614a7b37180 0x5614a7b39590 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd460acf0dcb246a server_cookie=9c07add0082c19b7 in_seq=0 out_seq=0 2023-07-14T16:29:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.743+0000 7f2e82ffd700 10 client.19539 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2e82ffd700 1 -- 172.21.15.62:0/3003650011 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5614a7a6beb0 con 0x5614a7b37180 2023-07-14T16:29:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2e82ffd700 10 client.19539 handle_command_reply: tid=0 2023-07-14T16:29:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 client.19539 shutdown 2023-07-14T16:29:49.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/3003650011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 msgr2=0x5614a7b32880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b32880 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c007f70 tx=0x7f2e9c009d00 comp rx=0 tx=0).stop 2023-07-14T16:29:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/3003650011 shutdown_connections 2023-07-14T16:29:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b32880 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5614a7b37180 0x5614a7b39590 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b21720 0x5614a7b2b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/3003650011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22070 0x5614a7b2b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/3003650011 >> 172.21.15.62:0/3003650011 conn(0x5614a7b16ff0 msgr2=0x5614a7b1bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2e82ffd700 0 client.19539 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/3003650011 shutdown_connections 2023-07-14T16:29:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/3003650011 wait complete. 2023-07-14T16:29:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.744+0000 7f2ebd7a7b80 20 client.19539 trim_cache size 0 max 16384 2023-07-14T16:29:49.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.769+0000 7f2ebd7a7b80 1 Processor -- start 2023-07-14T16:29:49.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.769+0000 7f2ebd7a7b80 1 -- start start 2023-07-14T16:29:49.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b21720 0x5614a7b29180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b327a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7b1d160 con 0x5614a7b21720 2023-07-14T16:29:49.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7b32ce0 con 0x5614a7b22070 2023-07-14T16:29:49.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7b22460 con 0x5614a7b22e30 2023-07-14T16:29:49.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ea37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b21720 0x5614a7b29180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ea37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b21720 0x5614a7b29180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46516/0 (socket says 172.21.15.62:46516) 2023-07-14T16:29:49.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ea37fe700 1 -- 172.21.15.62:0/2792477808 learned_addr learned my addr 172.21.15.62:0/2792477808 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.770+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2ea37fe700 1 -- 172.21.15.62:0/2792477808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 msgr2=0x5614a7b327a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b327a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2ea37fe700 1 -- 172.21.15.62:0/2792477808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 msgr2=0x5614a7b296c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b296c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2ea37fe700 1 -- 172.21.15.62:0/2792477808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614a78c7250 con 0x5614a7b21720 2023-07-14T16:29:49.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b327a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:49.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:49.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b21720 0x5614a7b29180 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009990 tx=0x7f2e94020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8864baf1f8ca3767 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2e8a7fc700 1 -- 172.21.15.62:0/2792477808 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94035ae0 con 0x5614a7b21720 2023-07-14T16:29:49.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2e8a7fc700 1 -- 172.21.15.62:0/2792477808 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e9401e040 con 0x5614a7b21720 2023-07-14T16:29:49.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.771+0000 7f2e8a7fc700 1 -- 172.21.15.62:0/2792477808 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400bee0 con 0x5614a7b21720 2023-07-14T16:29:49.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/2792477808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b21720 msgr2=0x5614a7b29180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b21720 0x5614a7b29180 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009990 tx=0x7f2e94020000 comp rx=0 tx=0).stop 2023-07-14T16:29:49.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/2792477808 shutdown_connections 2023-07-14T16:29:49.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b22e30 0x5614a7b327a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b21720 0x5614a7b29180 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/2792477808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/2792477808 >> 172.21.15.62:0/2792477808 conn(0x5614a7b16ff0 msgr2=0x5614a7b39190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/2792477808 shutdown_connections 2023-07-14T16:29:49.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.772+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/2792477808 wait complete. 2023-07-14T16:29:49.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 1 -- start start 2023-07-14T16:29:49.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 1 Processor -- start 2023-07-14T16:29:49.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b21720 0x5614a7b321a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b2eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7b2f3b0 con 0x5614a7b22e30 2023-07-14T16:29:49.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7b2f520 con 0x5614a7b22070 2023-07-14T16:29:49.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a7b2f6e0 con 0x5614a7b21720 2023-07-14T16:29:49.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.773+0000 7f2ebd7a7b80 10 client.0 _ll_register_callbacks cb 0x7f2e9c0078a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:49.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b21720 0x5614a7b321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b21720 0x5614a7b321a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:54848/0 (socket says 172.21.15.62:54848) 2023-07-14T16:29:49.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea37fe700 1 -- 172.21.15.62:0/935265983 learned_addr learned my addr 172.21.15.62:0/935265983 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:49.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b2eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea37fe700 1 -- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 msgr2=0x5614a7b326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b326e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea37fe700 1 -- 172.21.15.62:0/935265983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 msgr2=0x5614a7b2eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:49.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b2eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea37fe700 1 -- 172.21.15.62:0/935265983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614a78a0de0 con 0x5614a7b21720 2023-07-14T16:29:49.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.774+0000 7f2ea3fff700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b2eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2ea37fe700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b21720 0x5614a7b321a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e94005460 tx=0x7f2e94005a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f85758a07ba4bef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2e82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:29:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2e82ffd700 1 -- 172.21.15.62:0/935265983 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400be40 con 0x5614a7b21720 2023-07-14T16:29:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2e82ffd700 1 -- 172.21.15.62:0/935265983 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2e94035cf0 con 0x5614a7b21720 2023-07-14T16:29:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/935265983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e98099dc0 con 0x5614a7b21720 2023-07-14T16:29:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2e82ffd700 1 -- 172.21.15.62:0/935265983 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e9400b710 con 0x5614a7b21720 2023-07-14T16:29:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2ebd7a7b80 10 client.28972 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/935265983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5614a7889c60 con 0x5614a7b21720 2023-07-14T16:29:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.775+0000 7f2e82ffd700 1 -- 172.21.15.62:0/935265983 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e98099dc0 con 0x5614a7b21720 2023-07-14T16:29:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.776+0000 7f2e82ffd700 1 -- 172.21.15.62:0/935265983 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2e940047a0 con 0x5614a7b21720 2023-07-14T16:29:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.776+0000 7f2ebd7a7b80 10 client.28972 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.776+0000 7f2ebd7a7b80 10 client.28972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.776+0000 7f2ebd7a7b80 20 client.28972 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.776+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5614a7b8c210 0x5614a7b8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.776+0000 7f2ebd7a7b80 4 client.28972 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:29:49.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.776+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/935265983 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614a7a2e050 con 0x5614a7b8c210 2023-07-14T16:29:49.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.776+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5614a7b8c210 0x5614a7b8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:49.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ea2ffd700 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5614a7b8c210 0x5614a7b8e5f0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28abac319af904a server_cookie=7fc4a4de0f374f1f in_seq=0 out_seq=0 2023-07-14T16:29:49.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2e82ffd700 10 client.28972 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:49.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2e82ffd700 1 -- 172.21.15.62:0/935265983 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5614a7a2e050 con 0x5614a7b8c210 2023-07-14T16:29:49.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2e82ffd700 10 client.28972 handle_command_reply: tid=0 2023-07-14T16:29:49.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 client.28972 shutdown 2023-07-14T16:29:49.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/935265983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b21720 msgr2=0x5614a7b321a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b21720 0x5614a7b321a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2e94005460 tx=0x7f2e94005a50 comp rx=0 tx=0).stop 2023-07-14T16:29:49.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/935265983 shutdown_connections 2023-07-14T16:29:49.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5614a7b22e30 0x5614a7b2eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5614a7b21720 0x5614a7b321a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5614a7b8c210 0x5614a7b8e5f0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 --2- 172.21.15.62:0/935265983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614a7b22070 0x5614a7b326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/935265983 >> 172.21.15.62:0/935265983 conn(0x5614a7b16ff0 msgr2=0x5614a7b29180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2e82ffd700 0 client.28972 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:49.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/935265983 shutdown_connections 2023-07-14T16:29:49.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.777+0000 7f2ebd7a7b80 1 -- 172.21.15.62:0/935265983 wait complete. 2023-07-14T16:29:49.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ebd7a7b80 20 client.28972 trim_cache size 0 max 16384 2023-07-14T16:29:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:29:49.760 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:29:49.760 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:29:49.760 INFO:teuthology.orchestra.run.smithi062.stdout: "cbd9fbce-74b5-4539-8234-575dbcfd80a7": { 2023-07-14T16:29:49.760 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:29:49.760 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "cbd9fbce-74b5-4539-8234-575dbcfd80a7", 2023-07-14T16:29:49.760 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:29:49.761 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:29:49.761 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:29:49.761 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:29:49.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e8c0a1270 msgr2=0x7f2e8c0a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e8c0a1270 0x7f2e8c0a3720 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2e94006010 tx=0x7f2e94005960 comp rx=0 tx=0).stop 2023-07-14T16:29:49.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/1360017830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea40744b0 msgr2=0x7f2ea4197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:49.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea40744b0 0x7f2ea4197c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800e9f0 tx=0x7f2e9800ed00 comp rx=0 tx=0).stop 2023-07-14T16:29:49.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/1360017830 shutdown_connections 2023-07-14T16:29:49.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea4107850 0x7f2ea419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2e8c0a1270 0x7f2e8c0a3720 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ea4072ee0 0x7f2ea41976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 --2- 172.21.15.62:0/1360017830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ea40744b0 0x7f2ea4197c30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:49.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.778+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/1360017830 >> 172.21.15.62:0/1360017830 conn(0x7f2ea40fb010 msgr2=0x7f2ea40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:49.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.779+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/1360017830 shutdown_connections 2023-07-14T16:29:49.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:29:49.779+0000 7f2ea8b9d700 1 -- 172.21.15.62:0/1360017830 wait complete. 2023-07-14T16:29:49.774 INFO:tasks.cephfs.filesystem:scrub status for tag:cbd9fbce-74b5-4539-8234-575dbcfd80a7 - {'path': '/', 'tag': 'cbd9fbce-74b5-4539-8234-575dbcfd80a7', 'options': 'recursive,force'} 2023-07-14T16:29:49.832 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:29:49 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:29:49.467+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v261>,<0x10000000003/client.0 v262>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v274>,<0x10000000001/sv_0 v288>,<0x10000000000/_nogroup v282>,<0x1/volumes v272>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:29:50.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:49 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:29:50.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:49 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:50.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:49 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:29:50.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:49 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:29:50.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:49 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:50.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:49 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:50.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:49 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:29:50.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:49 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:50.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:49 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:29:50.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:49 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:29:50.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:49 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:50.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:49 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:50.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:49 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:29:50.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:49 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:50.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:49 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:29:50.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:49 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:29:50.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:49 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:50.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:49 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:51.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:50 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:29:51.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:50 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:51.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:50 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:29:51.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:50 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:29:51.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:50 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:51.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:50 smithi062 ceph-mon[95543]: pgmap v783: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:29:51.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:50 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:29:51.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:50 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:51.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:50 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:29:51.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:50 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:29:51.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:50 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:51.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:50 smithi081 ceph-mon[127795]: pgmap v783: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:29:51.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:50 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:29:51.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:50 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:51.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:50 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:29:51.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:50 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:29:51.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:50 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:29:51.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:50 smithi164 ceph-mon[97427]: pgmap v783: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 17 op/s 2023-07-14T16:29:53.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:52 smithi062 ceph-mon[95543]: pgmap v784: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:29:53.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:52 smithi081 ceph-mon[127795]: pgmap v784: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:29:53.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:52 smithi164 ceph-mon[97427]: pgmap v784: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 25 op/s 2023-07-14T16:29:55.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:54 smithi062 ceph-mon[95543]: pgmap v785: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:29:55.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:54 smithi081 ceph-mon[127795]: pgmap v785: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:29:55.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:54 smithi164 ceph-mon[97427]: pgmap v785: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:29:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.340+0000 7f20087ff700 1 -- 172.21.15.62:0/3572500331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000101690 msgr2=0x7f2000107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.340+0000 7f20087ff700 1 --2- 172.21.15.62:0/3572500331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000101690 0x7f2000107f50 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800b0a0 tx=0x7f1ff800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.341+0000 7f20087ff700 1 -- 172.21.15.62:0/3572500331 shutdown_connections 2023-07-14T16:29:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.341+0000 7f20087ff700 1 --2- 172.21.15.62:0/3572500331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000101690 0x7f2000107f50 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.341+0000 7f20087ff700 1 --2- 172.21.15.62:0/3572500331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20000ffb10 0x7f20000fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.341+0000 7f20087ff700 1 --2- 172.21.15.62:0/3572500331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2000100d00 0x7f2000101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.341+0000 7f20087ff700 1 -- 172.21.15.62:0/3572500331 >> 172.21.15.62:0/3572500331 conn(0x7f20000fb130 msgr2=0x7f20000fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.341+0000 7f20087ff700 1 -- 172.21.15.62:0/3572500331 shutdown_connections 2023-07-14T16:29:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.341+0000 7f20087ff700 1 -- 172.21.15.62:0/3572500331 wait complete. 2023-07-14T16:29:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.342+0000 7f20087ff700 1 Processor -- start 2023-07-14T16:29:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.342+0000 7f20087ff700 1 -- start start 2023-07-14T16:29:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.342+0000 7f20087ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20000ffb10 0x7f2000072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.342+0000 7f20087ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2000100d00 0x7f2000072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.342+0000 7f20087ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2000101690 0x7f2000073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.342+0000 7f20087ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200019f8e0 con 0x7f20000ffb10 2023-07-14T16:29:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.342+0000 7f20087ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200019fa20 con 0x7f2000101690 2023-07-14T16:29:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.342+0000 7f20087ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200019fb60 con 0x7f2000100d00 2023-07-14T16:29:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20000ffb10 0x7f2000072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20000ffb10 0x7f2000072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34554/0 (socket says 172.21.15.62:34554) 2023-07-14T16:29:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 -- 172.21.15.62:0/2916026377 learned_addr learned my addr 172.21.15.62:0/2916026377 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f2005d9a700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2000100d00 0x7f2000072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f2006d9c700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2000101690 0x7f2000073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 -- 172.21.15.62:0/2916026377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2000100d00 msgr2=0x7f2000072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2000100d00 0x7f2000072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 -- 172.21.15.62:0/2916026377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2000101690 msgr2=0x7f2000073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2000101690 0x7f2000073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 -- 172.21.15.62:0/2916026377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff8009d00 con 0x7f20000ffb10 2023-07-14T16:29:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f2006d9c700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2000101690 0x7f2000073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.343+0000 7f200659b700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20000ffb10 0x7f2000072900 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00e970 tx=0x7f1ffc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.344+0000 7f1ff77fe700 1 -- 172.21.15.62:0/2916026377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc009790 con 0x7f20000ffb10 2023-07-14T16:29:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.344+0000 7f1ff77fe700 1 -- 172.21.15.62:0/2916026377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ffc00eec0 con 0x7f20000ffb10 2023-07-14T16:29:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.344+0000 7f1ff77fe700 1 -- 172.21.15.62:0/2916026377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc003c10 con 0x7f20000ffb10 2023-07-14T16:29:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.344+0000 7f20087ff700 1 -- 172.21.15.62:0/2916026377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200019fdf0 con 0x7f20000ffb10 2023-07-14T16:29:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.344+0000 7f20087ff700 1 -- 172.21.15.62:0/2916026377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20001a01c0 con 0x7f20000ffb10 2023-07-14T16:29:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.348+0000 7f1ff77fe700 1 -- 172.21.15.62:0/2916026377 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1ffc003db0 con 0x7f20000ffb10 2023-07-14T16:29:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.349+0000 7f201c1c4b80 1 Processor -- start 2023-07-14T16:29:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.349+0000 7f201c1c4b80 1 -- start start 2023-07-14T16:29:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.349+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc8fce6b0 0x55bbc9172300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.349+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc91731a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.349+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc91736e0 0x55bbc9173b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.349+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc9077c50 con 0x55bbc9172840 2023-07-14T16:29:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.349+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc90ca9e0 con 0x55bbc91736e0 2023-07-14T16:29:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.349+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc90c3e70 con 0x55bbc8fce6b0 2023-07-14T16:29:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f200659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc91731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2005d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc91736e0 0x55bbc9173b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2005d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc91736e0 0x55bbc9173b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:39738/0 (socket says 172.21.15.62:39738) 2023-07-14T16:29:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2005d9a700 1 -- 172.21.15.62:0/3957092619 learned_addr learned my addr 172.21.15.62:0/3957092619 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2006d9c700 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc8fce6b0 0x55bbc9172300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2006d9c700 1 -- 172.21.15.62:0/3957092619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc91736e0 msgr2=0x55bbc9173b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2006d9c700 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc91736e0 0x55bbc9173b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2006d9c700 1 -- 172.21.15.62:0/3957092619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 msgr2=0x55bbc91731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2006d9c700 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc91731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f1ff77fe700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec0a1150 0x7f1fec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f2006d9c700 1 -- 172.21.15.62:0/3957092619 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbc8ef1d80 con 0x55bbc8fce6b0 2023-07-14T16:29:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.350+0000 7f200659b700 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc91731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.351+0000 7f1ff77fe700 1 -- 172.21.15.62:0/2916026377 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ffc014070 con 0x7f20000ffb10 2023-07-14T16:29:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.351+0000 7f2006d9c700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec0a1150 0x7f1fec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.351+0000 7f2006d9c700 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc8fce6b0 0x55bbc9172300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009fd0 tx=0x7f1ff80092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc19a5fa7f5bf80b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.351+0000 7f1ff4ff9700 1 -- 172.21.15.62:0/3957092619 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800e050 con 0x55bbc8fce6b0 2023-07-14T16:29:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f1ff4ff9700 1 -- 172.21.15.62:0/3957092619 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ff80094d0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f1ff4ff9700 1 -- 172.21.15.62:0/3957092619 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8028730 con 0x55bbc8fce6b0 2023-07-14T16:29:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f2006d9c700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec0a1150 0x7f1fec0a3600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800b780 tx=0x7f1ff802a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3957092619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc8fce6b0 msgr2=0x55bbc9172300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc8fce6b0 0x55bbc9172300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009fd0 tx=0x7f1ff80092e0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3957092619 shutdown_connections 2023-07-14T16:29:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc91736e0 0x55bbc9173b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc8fce6b0 0x55bbc9172300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/3957092619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc91731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3957092619 >> 172.21.15.62:0/3957092619 conn(0x55bbc9167be0 msgr2=0x55bbc916ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.352+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3957092619 shutdown_connections 2023-07-14T16:29:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.353+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3957092619 wait complete. 2023-07-14T16:29:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.353+0000 7f201c1c4b80 1 -- start start 2023-07-14T16:29:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.354+0000 7f201c1c4b80 1 Processor -- start 2023-07-14T16:29:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.354+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.354+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc917bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.354+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc9182e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.354+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc9063a60 con 0x55bbc9172840 2023-07-14T16:29:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.354+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc9063760 con 0x55bbc8fce6b0 2023-07-14T16:29:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.354+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc8f478c0 con 0x55bbc91736e0 2023-07-14T16:29:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.354+0000 7f201c1c4b80 10 client.0 _ll_register_callbacks cb 0x55bbc9061700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:56.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f2005d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc917bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:39742/0 (socket says 172.21.15.62:39742) 2023-07-14T16:29:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 -- 172.21.15.62:0/98452700 learned_addr learned my addr 172.21.15.62:0/98452700 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f2005d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc917bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34576/0 (socket says 172.21.15.62:34576) 2023-07-14T16:29:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f2006d9c700 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc9182e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 -- 172.21.15.62:0/98452700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 msgr2=0x55bbc9182e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc9182e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 -- 172.21.15.62:0/98452700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 msgr2=0x55bbc917bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc917bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 -- 172.21.15.62:0/98452700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20001a01c0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f2006d9c700 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc9182e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:29:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f2005d9a700 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc917bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:29:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f200659b700 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b890 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0d22d0 tx=0x7f1ffc0963a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d27007de2a8d7a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.355+0000 7f1fe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:29:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.356+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/98452700 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc099660 con 0x55bbc8fce6b0 2023-07-14T16:29:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.356+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/98452700 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ffc00ccd0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.356+0000 7f201c1c4b80 1 -- 172.21.15.62:0/98452700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbc901d490 con 0x55bbc8fce6b0 2023-07-14T16:29:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.356+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/98452700 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ffc099f70 con 0x55bbc8fce6b0 2023-07-14T16:29:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.356+0000 7f201c1c4b80 10 client.29114 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.356+0000 7f201c1c4b80 1 -- 172.21.15.62:0/98452700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f2000108880 con 0x55bbc8fce6b0 2023-07-14T16:29:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.356+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/98452700 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bbc901d490 con 0x55bbc8fce6b0 2023-07-14T16:29:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.357+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/98452700 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ffc094b80 con 0x55bbc8fce6b0 2023-07-14T16:29:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.357+0000 7f201c1c4b80 10 client.29114 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.357+0000 7f201c1c4b80 10 client.29114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.357+0000 7f201c1c4b80 20 client.29114 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.357+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bbc9187650 0x55bbc9189a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.357+0000 7f201c1c4b80 4 client.29114 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:29:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.357+0000 7f201c1c4b80 1 -- 172.21.15.62:0/98452700 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbc90bce50 con 0x55bbc9187650 2023-07-14T16:29:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.357+0000 7f2005d9a700 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bbc9187650 0x55bbc9189a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.358+0000 7f1fe9ffb700 10 client.29114 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.358+0000 7f2005d9a700 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bbc9187650 0x55bbc9189a60 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c894f77c35c2215 server_cookie=9d8c20ec3f5cf71 in_seq=0 out_seq=0 2023-07-14T16:29:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/98452700 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bbc90bce50 con 0x55bbc9187650 2023-07-14T16:29:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f1fe9ffb700 10 client.29114 handle_command_reply: tid=0 2023-07-14T16:29:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 client.29114 shutdown 2023-07-14T16:29:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 -- 172.21.15.62:0/98452700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 msgr2=0x55bbc917b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b890 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0d22d0 tx=0x7f1ffc0963a0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 -- 172.21.15.62:0/98452700 shutdown_connections 2023-07-14T16:29:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc9182e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bbc9187650 0x55bbc9189a60 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b890 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f1fe9ffb700 0 client.29114 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/98452700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc9172840 0x55bbc917bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 -- 172.21.15.62:0/98452700 >> 172.21.15.62:0/98452700 conn(0x55bbc9167be0 msgr2=0x55bbc916c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 -- 172.21.15.62:0/98452700 shutdown_connections 2023-07-14T16:29:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 1 -- 172.21.15.62:0/98452700 wait complete. 2023-07-14T16:29:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.359+0000 7f201c1c4b80 20 client.29114 trim_cache size 0 max 16384 2023-07-14T16:29:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f201c1c4b80 1 Processor -- start 2023-07-14T16:29:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f201c1c4b80 1 -- start start 2023-07-14T16:29:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc9172840 0x55bbc9182da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc917bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc90bce50 con 0x55bbc8fce6b0 2023-07-14T16:29:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc901d490 con 0x55bbc9172840 2023-07-14T16:29:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc916d880 con 0x55bbc91736e0 2023-07-14T16:29:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f200659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f200659b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34588/0 (socket says 172.21.15.62:34588) 2023-07-14T16:29:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f200659b700 1 -- 172.21.15.62:0/3325201752 learned_addr learned my addr 172.21.15.62:0/3325201752 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.375+0000 7f2005d9a700 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc9172840 0x55bbc9182da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f2006d9c700 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc917bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f200659b700 1 -- 172.21.15.62:0/3325201752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 msgr2=0x55bbc917bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f200659b700 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc917bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f200659b700 1 -- 172.21.15.62:0/3325201752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc9172840 msgr2=0x55bbc9182da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f200659b700 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc9172840 0x55bbc9182da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f200659b700 1 -- 172.21.15.62:0/3325201752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbc8edac00 con 0x55bbc8fce6b0 2023-07-14T16:29:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f200659b700 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b640 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c5b0 tx=0x7f1ffc0ccc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316019b1f2ee3966 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f1ff57fa700 1 -- 172.21.15.62:0/3325201752 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc00ccd0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f1ff57fa700 1 -- 172.21.15.62:0/3325201752 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ffc0d1070 con 0x55bbc8fce6b0 2023-07-14T16:29:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.376+0000 7f1ff57fa700 1 -- 172.21.15.62:0/3325201752 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc099460 con 0x55bbc8fce6b0 2023-07-14T16:29:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3325201752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc8fce6b0 msgr2=0x55bbc917b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b640 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c5b0 tx=0x7f1ffc0ccc80 comp rx=0 tx=0).stop 2023-07-14T16:29:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3325201752 shutdown_connections 2023-07-14T16:29:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc91736e0 0x55bbc917bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc8fce6b0 0x55bbc917b640 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/3325201752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc9172840 0x55bbc9182da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3325201752 >> 172.21.15.62:0/3325201752 conn(0x55bbc9167be0 msgr2=0x55bbc9176590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3325201752 shutdown_connections 2023-07-14T16:29:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.377+0000 7f201c1c4b80 1 -- 172.21.15.62:0/3325201752 wait complete. 2023-07-14T16:29:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.378+0000 7f201c1c4b80 1 -- start start 2023-07-14T16:29:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.378+0000 7f201c1c4b80 1 Processor -- start 2023-07-14T16:29:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc9180f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc9172840 0x55bbc91814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f201c1c4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc91736e0 0x55bbc9181b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc9182290 con 0x55bbc91736e0 2023-07-14T16:29:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc9182400 con 0x55bbc8fce6b0 2023-07-14T16:29:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f201c1c4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc91825c0 con 0x55bbc9172840 2023-07-14T16:29:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f201c1c4b80 10 client.0 _ll_register_callbacks cb 0x7f1ffc00cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:29:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f200659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc9180f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f200659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc9180f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:39762/0 (socket says 172.21.15.62:39762) 2023-07-14T16:29:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f200659b700 1 -- 172.21.15.62:0/1920305046 learned_addr learned my addr 172.21.15.62:0/1920305046 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:29:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f2005d9a700 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc9172840 0x55bbc91814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f2006d9c700 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc91736e0 0x55bbc9181b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f200659b700 1 -- 172.21.15.62:0/1920305046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc9172840 msgr2=0x55bbc91814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f200659b700 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc9172840 0x55bbc91814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f200659b700 1 -- 172.21.15.62:0/1920305046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc91736e0 msgr2=0x55bbc9181b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:29:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.379+0000 7f200659b700 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc91736e0 0x55bbc9181b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f200659b700 1 -- 172.21.15.62:0/1920305046 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2000108880 con 0x55bbc8fce6b0 2023-07-14T16:29:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f200659b700 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc9180f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009230 tx=0x7f1ffc09dfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee4b5a077111704 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:29:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f1fe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:29:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/1920305046 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0962b0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/1920305046 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ffc00ccd0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f201c1c4b80 1 -- 172.21.15.62:0/1920305046 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ff80094d0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/1920305046 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ffc0b8350 con 0x55bbc8fce6b0 2023-07-14T16:29:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f201c1c4b80 10 client.29120 fetch_fsmap learned FSMap version 27 2023-07-14T16:29:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.380+0000 7f201c1c4b80 1 -- 172.21.15.62:0/1920305046 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bbc8f181f0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.381+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/1920305046 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ff80094d0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.381+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/1920305046 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ffc098ed0 con 0x55bbc8fce6b0 2023-07-14T16:29:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.381+0000 7f201c1c4b80 10 client.29120 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:29:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.381+0000 7f201c1c4b80 10 client.29120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:29:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.381+0000 7f201c1c4b80 20 client.29120 populate_metadata read hostname 'smithi062' 2023-07-14T16:29:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.381+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bbc91dba90 0x55bbc91dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:29:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.381+0000 7f201c1c4b80 4 client.29120 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:29:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.381+0000 7f201c1c4b80 1 -- 172.21.15.62:0/1920305046 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bbc907eff0 con 0x55bbc91dba90 2023-07-14T16:29:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.382+0000 7f2005d9a700 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bbc91dba90 0x55bbc91dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:29:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.382+0000 7f1fe9ffb700 10 client.29120 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.382+0000 7f2005d9a700 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bbc91dba90 0x55bbc91dde70 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8148174542ea20 server_cookie=1b484572de6e49e1 in_seq=0 out_seq=0 2023-07-14T16:29:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f1fe9ffb700 1 -- 172.21.15.62:0/1920305046 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bbc907eff0 con 0x55bbc91dba90 2023-07-14T16:29:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f1fe9ffb700 10 client.29120 handle_command_reply: tid=0 2023-07-14T16:29:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 client.29120 shutdown 2023-07-14T16:29:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 -- 172.21.15.62:0/1920305046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 msgr2=0x55bbc9180f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc9180f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009230 tx=0x7f1ffc09dfa0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 -- 172.21.15.62:0/1920305046 shutdown_connections 2023-07-14T16:29:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bbc91736e0 0x55bbc9181b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55bbc91dba90 0x55bbc91dde70 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bbc8fce6b0 0x55bbc9180f70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 --2- 172.21.15.62:0/1920305046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bbc9172840 0x55bbc91814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 -- 172.21.15.62:0/1920305046 >> 172.21.15.62:0/1920305046 conn(0x55bbc9167be0 msgr2=0x55bbc917b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f1fe9ffb700 0 client.29120 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:29:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 -- 172.21.15.62:0/1920305046 shutdown_connections 2023-07-14T16:29:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.392+0000 7f201c1c4b80 1 -- 172.21.15.62:0/1920305046 wait complete. 2023-07-14T16:29:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f201c1c4b80 20 client.29120 trim_cache size 0 max 16384 2023-07-14T16:29:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:29:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:29:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:29:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:29:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 -- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec0a1150 msgr2=0x7f1fec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec0a1150 0x7f1fec0a3600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800b780 tx=0x7f1ff802a000 comp rx=0 tx=0).stop 2023-07-14T16:29:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 -- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20000ffb10 msgr2=0x7f2000072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:29:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20000ffb10 0x7f2000072900 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00e970 tx=0x7f1ffc00ec80 comp rx=0 tx=0).stop 2023-07-14T16:29:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 -- 172.21.15.62:0/2916026377 shutdown_connections 2023-07-14T16:29:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1fec0a1150 0x7f1fec0a3600 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2000101690 0x7f2000073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20000ffb10 0x7f2000072900 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 --2- 172.21.15.62:0/2916026377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2000100d00 0x7f2000072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:29:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.393+0000 7f20087ff700 1 -- 172.21.15.62:0/2916026377 >> 172.21.15.62:0/2916026377 conn(0x7f20000fb130 msgr2=0x7f20001041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:29:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.394+0000 7f20087ff700 1 -- 172.21.15.62:0/2916026377 shutdown_connections 2023-07-14T16:29:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:29:56.394+0000 7f20087ff700 1 -- 172.21.15.62:0/2916026377 wait complete. 2023-07-14T16:29:57.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:56 smithi062 ceph-mon[95543]: pgmap v786: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:29:57.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:56 smithi081 ceph-mon[127795]: pgmap v786: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:29:57.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:56 smithi164 ceph-mon[97427]: pgmap v786: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 36 op/s 2023-07-14T16:29:59.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:29:58 smithi062 ceph-mon[95543]: pgmap v787: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 34 op/s 2023-07-14T16:29:59.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:29:58 smithi081 ceph-mon[127795]: pgmap v787: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 34 op/s 2023-07-14T16:29:59.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:29:58 smithi164 ceph-mon[97427]: pgmap v787: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 34 op/s 2023-07-14T16:30:01.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:00 smithi062 ceph-mon[95543]: pgmap v788: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:01.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:00 smithi062 ceph-mon[95543]: overall HEALTH_OK 2023-07-14T16:30:01.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:00 smithi081 ceph-mon[127795]: pgmap v788: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:01.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:00 smithi081 ceph-mon[127795]: overall HEALTH_OK 2023-07-14T16:30:01.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:00 smithi164 ceph-mon[97427]: pgmap v788: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:01.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:00 smithi164 ceph-mon[97427]: overall HEALTH_OK 2023-07-14T16:30:03.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:02 smithi062 ceph-mon[95543]: pgmap v789: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 35 op/s 2023-07-14T16:30:03.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:02 smithi081 ceph-mon[127795]: pgmap v789: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 35 op/s 2023-07-14T16:30:03.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:02 smithi164 ceph-mon[97427]: pgmap v789: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 35 op/s 2023-07-14T16:30:04.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:30:04.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:30:04.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:30:04.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:30:04.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:30:04.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:30:04.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:30:04.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:30:04.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:30:05.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:04 smithi062 ceph-mon[95543]: pgmap v790: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:30:05.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:04 smithi081 ceph-mon[127795]: pgmap v790: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:30:05.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:04 smithi164 ceph-mon[97427]: pgmap v790: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 27 op/s 2023-07-14T16:30:06.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:05 smithi062 ceph-mon[95543]: pgmap v791: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:06.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:05 smithi081 ceph-mon[127795]: pgmap v791: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:06.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:05 smithi164 ceph-mon[97427]: pgmap v791: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:09.021 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:08 smithi164 ceph-mon[97427]: pgmap v792: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:30:09.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:08 smithi062 ceph-mon[95543]: pgmap v792: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:30:09.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:08 smithi081 ceph-mon[127795]: pgmap v792: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2023-07-14T16:30:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 -- 172.21.15.62:0/1355425075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4180ffb00 msgr2=0x7fb4180fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 --2- 172.21.15.62:0/1355425075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4180ffb00 0x7fb4180fff10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009990 tx=0x7fb40c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 -- 172.21.15.62:0/1355425075 shutdown_connections 2023-07-14T16:30:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 --2- 172.21.15.62:0/1355425075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb418101680 0x7fb418107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 --2- 172.21.15.62:0/1355425075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4180ffb00 0x7fb4180fff10 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 --2- 172.21.15.62:0/1355425075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb418100cf0 0x7fb418101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 -- 172.21.15.62:0/1355425075 >> 172.21.15.62:0/1355425075 conn(0x7fb4180fb0e0 msgr2=0x7fb4180fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 -- 172.21.15.62:0/1355425075 shutdown_connections 2023-07-14T16:30:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.570+0000 7fb41ebff700 1 -- 172.21.15.62:0/1355425075 wait complete. 2023-07-14T16:30:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.571+0000 7fb41ebff700 1 Processor -- start 2023-07-14T16:30:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41ebff700 1 -- start start 2023-07-14T16:30:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41ebff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180ffb00 0x7fb418197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41ebff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb418100cf0 0x7fb418198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41ebff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb418101680 0x7fb418198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41ebff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4181994e0 con 0x7fb418101680 2023-07-14T16:30:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41ebff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418199650 con 0x7fb418100cf0 2023-07-14T16:30:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41ebff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418199810 con 0x7fb4180ffb00 2023-07-14T16:30:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41d19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb418101680 0x7fb418198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb417fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb418100cf0 0x7fb418198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41d19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb418101680 0x7fb418198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45050/0 (socket says 172.21.15.62:45050) 2023-07-14T16:30:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.572+0000 7fb41d19c700 1 -- 172.21.15.62:0/4140622429 learned_addr learned my addr 172.21.15.62:0/4140622429 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb41c99b700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180ffb00 0x7fb418197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb41d19c700 1 -- 172.21.15.62:0/4140622429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180ffb00 msgr2=0x7fb418197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb41d19c700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180ffb00 0x7fb418197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb41d19c700 1 -- 172.21.15.62:0/4140622429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb418100cf0 msgr2=0x7fb418198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb41d19c700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb418100cf0 0x7fb418198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb41d19c700 1 -- 172.21.15.62:0/4140622429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0095f0 con 0x7fb418101680 2023-07-14T16:30:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb417fff700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb418100cf0 0x7fb418198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb41d19c700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb418101680 0x7fb418198e80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb41000ef30 tx=0x7fb41000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.573+0000 7fb415ffb700 1 -- 172.21.15.62:0/4140622429 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410018070 con 0x7fb418101680 2023-07-14T16:30:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.574+0000 7fb41ebff700 1 -- 172.21.15.62:0/4140622429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb418199a90 con 0x7fb418101680 2023-07-14T16:30:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.574+0000 7fb415ffb700 1 -- 172.21.15.62:0/4140622429 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb41000f040 con 0x7fb418101680 2023-07-14T16:30:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.574+0000 7fb415ffb700 1 -- 172.21.15.62:0/4140622429 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000cbc0 con 0x7fb418101680 2023-07-14T16:30:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.574+0000 7fb41ebff700 1 -- 172.21.15.62:0/4140622429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4181a02b0 con 0x7fb418101680 2023-07-14T16:30:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.577+0000 7fb415ffb700 1 -- 172.21.15.62:0/4140622429 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb410007500 con 0x7fb418101680 2023-07-14T16:30:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.579+0000 7fb415ffb700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4080a11d0 0x7fb4080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.579+0000 7fb415ffb700 1 -- 172.21.15.62:0/4140622429 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb4100cc3c0 con 0x7fb418101680 2023-07-14T16:30:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.579+0000 7fb41c99b700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4080a11d0 0x7fb4080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.579+0000 7fb4324edb80 1 Processor -- start 2023-07-14T16:30:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb4324edb80 1 -- start start 2023-07-14T16:30:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a71448e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a71448f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a71448fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb4324edb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a714393c50 con 0x55a7142ea6b0 2023-07-14T16:30:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb4324edb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7143e69e0 con 0x55a71448e840 2023-07-14T16:30:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb4324edb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7143dfe70 con 0x55a71448f6e0 2023-07-14T16:30:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb41c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a71448e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb41c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a71448e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45058/0 (socket says 172.21.15.62:45058) 2023-07-14T16:30:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb41c99b700 1 -- 172.21.15.62:0/4069204481 learned_addr learned my addr 172.21.15.62:0/4069204481 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb41c99b700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4080a11d0 0x7fb4080a3680 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0098e0 tx=0x7fb40c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.580+0000 7fb417fff700 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a71448f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.581+0000 7fb41d19c700 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a71448fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.581+0000 7fb41c99b700 1 -- 172.21.15.62:0/4069204481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 msgr2=0x55a71448fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.581+0000 7fb41c99b700 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a71448fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.581+0000 7fb41c99b700 1 -- 172.21.15.62:0/4069204481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 msgr2=0x55a71448f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.581+0000 7fb41c99b700 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a71448f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.581+0000 7fb41c99b700 1 -- 172.21.15.62:0/4069204481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a71420dd80 con 0x55a7142ea6b0 2023-07-14T16:30:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.581+0000 7fb41c99b700 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a71448e300 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb418100b50 tx=0x7fb40c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd0b4ab4ac9527e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb402ffd700 1 -- 172.21.15.62:0/4069204481 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c003ce0 con 0x55a7142ea6b0 2023-07-14T16:30:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb402ffd700 1 -- 172.21.15.62:0/4069204481 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb40c01e040 con 0x55a7142ea6b0 2023-07-14T16:30:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb402ffd700 1 -- 172.21.15.62:0/4069204481 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c004790 con 0x55a7142ea6b0 2023-07-14T16:30:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb4324edb80 1 -- 172.21.15.62:0/4069204481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 msgr2=0x55a71448e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb4324edb80 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a71448e300 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb418100b50 tx=0x7fb40c020000 comp rx=0 tx=0).stop 2023-07-14T16:30:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb4324edb80 1 -- 172.21.15.62:0/4069204481 shutdown_connections 2023-07-14T16:30:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb4324edb80 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a71448fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb4324edb80 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a71448e300 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb4324edb80 1 --2- 172.21.15.62:0/4069204481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a71448f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.582+0000 7fb4324edb80 1 -- 172.21.15.62:0/4069204481 >> 172.21.15.62:0/4069204481 conn(0x55a714483be0 msgr2=0x55a714488cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.583+0000 7fb4324edb80 1 -- 172.21.15.62:0/4069204481 shutdown_connections 2023-07-14T16:30:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.583+0000 7fb4324edb80 1 -- 172.21.15.62:0/4069204481 wait complete. 2023-07-14T16:30:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.583+0000 7fb4324edb80 1 -- start start 2023-07-14T16:30:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.583+0000 7fb4324edb80 1 Processor -- start 2023-07-14T16:30:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a714497850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a714497d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a71449edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb4324edb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71437fa60 con 0x55a7142ea6b0 2023-07-14T16:30:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb4324edb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71437f760 con 0x55a71448e840 2023-07-14T16:30:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb4324edb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7142638c0 con 0x55a71448f6e0 2023-07-14T16:30:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb4324edb80 10 client.0 _ll_register_callbacks cb 0x55a71437d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb41c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a714497850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb41c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a714497850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45072/0 (socket says 172.21.15.62:45072) 2023-07-14T16:30:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb41c99b700 1 -- 172.21.15.62:0/3504305535 learned_addr learned my addr 172.21.15.62:0/3504305535 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.584+0000 7fb417fff700 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a714497d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb41c99b700 1 -- 172.21.15.62:0/3504305535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 msgr2=0x55a71449edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb41c99b700 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a71449edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb41c99b700 1 -- 172.21.15.62:0/3504305535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 msgr2=0x55a714497d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb41c99b700 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a714497d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb41c99b700 1 -- 172.21.15.62:0/3504305535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb418108870 con 0x55a7142ea6b0 2023-07-14T16:30:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb41c99b700 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a714497850 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009ce0 tx=0x7fb40c004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8facc278efaa8f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb3f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb3f7fff700 1 -- 172.21.15.62:0/3504305535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00b6b0 con 0x55a7142ea6b0 2023-07-14T16:30:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb4324edb80 1 -- 172.21.15.62:0/3504305535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a714339490 con 0x55a7142ea6b0 2023-07-14T16:30:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.585+0000 7fb3f7fff700 1 -- 172.21.15.62:0/3504305535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb40c00b810 con 0x55a7142ea6b0 2023-07-14T16:30:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb3f7fff700 1 -- 172.21.15.62:0/3504305535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb40c00b9d0 con 0x55a7142ea6b0 2023-07-14T16:30:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb4324edb80 10 client.19575 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb4324edb80 1 -- 172.21.15.62:0/3504305535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a7141f6c00 con 0x55a7142ea6b0 2023-07-14T16:30:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb3f7fff700 1 -- 172.21.15.62:0/3504305535 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a714339490 con 0x55a7142ea6b0 2023-07-14T16:30:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb3f7fff700 1 -- 172.21.15.62:0/3504305535 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb40c00fc60 con 0x55a7142ea6b0 2023-07-14T16:30:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb4324edb80 10 client.19575 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb4324edb80 10 client.19575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb4324edb80 20 client.19575 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb4324edb80 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a7144a31c0 0x55a7144a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb4324edb80 4 client.19575 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.586+0000 7fb4324edb80 1 -- 172.21.15.62:0/3504305535 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7143d8e50 con 0x55a7144a31c0 2023-07-14T16:30:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.587+0000 7fb417fff700 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a7144a31c0 0x55a7144a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.587+0000 7fb3f7fff700 10 client.19575 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.587+0000 7fb417fff700 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a7144a31c0 0x55a7144a55d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa67903640f3648 server_cookie=791a53257a29fdce in_seq=0 out_seq=0 2023-07-14T16:30:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.588+0000 7fb3f7fff700 1 -- 172.21.15.62:0/3504305535 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a7143d8e50 con 0x55a7144a31c0 2023-07-14T16:30:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.588+0000 7fb3f7fff700 10 client.19575 handle_command_reply: tid=0 2023-07-14T16:30:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.588+0000 7fb4324edb80 1 client.19575 shutdown 2023-07-14T16:30:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.588+0000 7fb4324edb80 1 -- 172.21.15.62:0/3504305535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 msgr2=0x55a714497850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.588+0000 7fb4324edb80 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a714497850 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009ce0 tx=0x7fb40c004820 comp rx=0 tx=0).stop 2023-07-14T16:30:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 1 -- 172.21.15.62:0/3504305535 shutdown_connections 2023-07-14T16:30:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a71449edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a714497850 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a7144a31c0 0x55a7144a55d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 1 --2- 172.21.15.62:0/3504305535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a714497d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 1 -- 172.21.15.62:0/3504305535 >> 172.21.15.62:0/3504305535 conn(0x55a714483be0 msgr2=0x55a7144884e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb3f7fff700 0 client.19575 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 1 -- 172.21.15.62:0/3504305535 shutdown_connections 2023-07-14T16:30:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 1 -- 172.21.15.62:0/3504305535 wait complete. 2023-07-14T16:30:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.589+0000 7fb4324edb80 20 client.19575 trim_cache size 0 max 16384 2023-07-14T16:30:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.609+0000 7fb4324edb80 1 Processor -- start 2023-07-14T16:30:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.609+0000 7fb4324edb80 1 -- start start 2023-07-14T16:30:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a7144925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a71449ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a714492b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb4324edb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a714496d00 con 0x55a7142ea6b0 2023-07-14T16:30:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb4324edb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a714485d00 con 0x55a71448e840 2023-07-14T16:30:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb4324edb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71448a120 con 0x55a71448f6e0 2023-07-14T16:30:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb41c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a7144925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb41c99b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a7144925d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45082/0 (socket says 172.21.15.62:45082) 2023-07-14T16:30:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb41c99b700 1 -- 172.21.15.62:0/1618833593 learned_addr learned my addr 172.21.15.62:0/1618833593 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb417fff700 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a71449ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb41d19c700 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a714492b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb41c99b700 1 -- 172.21.15.62:0/1618833593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 msgr2=0x55a714492b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb41c99b700 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a714492b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb41c99b700 1 -- 172.21.15.62:0/1618833593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 msgr2=0x55a71449ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.610+0000 7fb41c99b700 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a71449ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.611+0000 7fb41c99b700 1 -- 172.21.15.62:0/1618833593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7142341f0 con 0x55a7142ea6b0 2023-07-14T16:30:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.611+0000 7fb41d19c700 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a714492b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.611+0000 7fb41c99b700 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a7144925d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb40c000c00 tx=0x7fb40c030570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5ae30a5475b7ed server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.611+0000 7fb4037fe700 1 -- 172.21.15.62:0/1618833593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0306c0 con 0x55a7142ea6b0 2023-07-14T16:30:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.611+0000 7fb4037fe700 1 -- 172.21.15.62:0/1618833593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb40c029050 con 0x55a7142ea6b0 2023-07-14T16:30:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.611+0000 7fb4037fe700 1 -- 172.21.15.62:0/1618833593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00b9a0 con 0x55a7142ea6b0 2023-07-14T16:30:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 -- 172.21.15.62:0/1618833593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 msgr2=0x55a7144925d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a7144925d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb40c000c00 tx=0x7fb40c030570 comp rx=0 tx=0).stop 2023-07-14T16:30:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 -- 172.21.15.62:0/1618833593 shutdown_connections 2023-07-14T16:30:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448f6e0 0x55a714492b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a7142ea6b0 0x55a7144925d0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 --2- 172.21.15.62:0/1618833593 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a71448e840 0x55a71449ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 -- 172.21.15.62:0/1618833593 >> 172.21.15.62:0/1618833593 conn(0x55a714483be0 msgr2=0x55a7144964f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 -- 172.21.15.62:0/1618833593 shutdown_connections 2023-07-14T16:30:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.612+0000 7fb4324edb80 1 -- 172.21.15.62:0/1618833593 wait complete. 2023-07-14T16:30:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.613+0000 7fb4324edb80 1 -- start start 2023-07-14T16:30:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.613+0000 7fb4324edb80 1 Processor -- start 2023-07-14T16:30:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7142ea6b0 0x55a71449a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448e840 0x55a71449aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb4324edb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a71448f6e0 0x55a71449ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb4324edb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71449e410 con 0x55a71448f6e0 2023-07-14T16:30:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb4324edb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71449e580 con 0x55a7142ea6b0 2023-07-14T16:30:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb4324edb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a71449e740 con 0x55a71448e840 2023-07-14T16:30:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb4324edb80 10 client.0 _ll_register_callbacks cb 0x7fb404009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb41d19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a71448f6e0 0x55a71449ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb41d19c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a71448f6e0 0x55a71449ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45094/0 (socket says 172.21.15.62:45094) 2023-07-14T16:30:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb41d19c700 1 -- 172.21.15.62:0/2330613231 learned_addr learned my addr 172.21.15.62:0/2330613231 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb417fff700 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448e840 0x55a71449aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.614+0000 7fb41c99b700 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7142ea6b0 0x55a71449a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb41d19c700 1 -- 172.21.15.62:0/2330613231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448e840 msgr2=0x55a71449aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb41d19c700 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448e840 0x55a71449aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb41d19c700 1 -- 172.21.15.62:0/2330613231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7142ea6b0 msgr2=0x55a71449a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb41d19c700 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7142ea6b0 0x55a71449a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb41d19c700 1 -- 172.21.15.62:0/2330613231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7141f6c00 con 0x55a71448f6e0 2023-07-14T16:30:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb41d19c700 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a71448f6e0 0x55a71449ded0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fb4100c9e70 tx=0x7fb4100b9390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5694dcd535d53b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb3f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb3f7fff700 1 -- 172.21.15.62:0/2330613231 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41009d840 con 0x55a71448f6e0 2023-07-14T16:30:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb3f7fff700 1 -- 172.21.15.62:0/2330613231 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb4100a1bd0 con 0x55a71448f6e0 2023-07-14T16:30:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.615+0000 7fb4324edb80 1 -- 172.21.15.62:0/2330613231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb40c01e040 con 0x55a71448f6e0 2023-07-14T16:30:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb3f7fff700 1 -- 172.21.15.62:0/2330613231 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb41000cd80 con 0x55a71448f6e0 2023-07-14T16:30:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb4324edb80 10 client.19581 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb4324edb80 1 -- 172.21.15.62:0/2330613231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a71420c250 con 0x55a71448f6e0 2023-07-14T16:30:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb3f7fff700 1 -- 172.21.15.62:0/2330613231 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb40c01e040 con 0x55a71448f6e0 2023-07-14T16:30:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb3f7fff700 1 -- 172.21.15.62:0/2330613231 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb4100a1740 con 0x55a71448f6e0 2023-07-14T16:30:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb4324edb80 10 client.19581 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb4324edb80 10 client.19581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb4324edb80 20 client.19581 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb4324edb80 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a7144f7dd0 0x55a7144fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb4324edb80 4 client.19581 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:30:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.616+0000 7fb4324edb80 1 -- 172.21.15.62:0/2330613231 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a71439aff0 con 0x55a7144f7dd0 2023-07-14T16:30:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.617+0000 7fb41c99b700 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a7144f7dd0 0x55a7144fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.617+0000 7fb3f7fff700 10 client.19581 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.617+0000 7fb41c99b700 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a7144f7dd0 0x55a7144fa1e0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2622344a86cf30e server_cookie=2448e27148f1f02e in_seq=0 out_seq=0 2023-07-14T16:30:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.627+0000 7fb3f7fff700 1 -- 172.21.15.62:0/2330613231 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a71439aff0 con 0x55a7144f7dd0 2023-07-14T16:30:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.627+0000 7fb3f7fff700 10 client.19581 handle_command_reply: tid=0 2023-07-14T16:30:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.627+0000 7fb4324edb80 1 client.19581 shutdown 2023-07-14T16:30:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 -- 172.21.15.62:0/2330613231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a71448f6e0 msgr2=0x55a71449ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a71448f6e0 0x55a71449ded0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fb4100c9e70 tx=0x7fb4100b9390 comp rx=0 tx=0).stop 2023-07-14T16:30:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 -- 172.21.15.62:0/2330613231 shutdown_connections 2023-07-14T16:30:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a71448f6e0 0x55a71449ded0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55a7144f7dd0 0x55a7144fa1e0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a7142ea6b0 0x55a71449a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 --2- 172.21.15.62:0/2330613231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a71448e840 0x55a71449aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb3f7fff700 0 client.19581 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 -- 172.21.15.62:0/2330613231 >> 172.21.15.62:0/2330613231 conn(0x55a714483be0 msgr2=0x55a7144925d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 -- 172.21.15.62:0/2330613231 shutdown_connections 2023-07-14T16:30:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 1 -- 172.21.15.62:0/2330613231 wait complete. 2023-07-14T16:30:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.628+0000 7fb4324edb80 20 client.19581 trim_cache size 0 max 16384 2023-07-14T16:30:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:30:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:30:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:30:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:30:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 -- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4080a11d0 msgr2=0x7fb4080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4080a11d0 0x7fb4080a3680 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0098e0 tx=0x7fb40c00b260 comp rx=0 tx=0).stop 2023-07-14T16:30:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 -- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb418101680 msgr2=0x7fb418198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb418101680 0x7fb418198e80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb41000ef30 tx=0x7fb41000c3a0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 -- 172.21.15.62:0/4140622429 shutdown_connections 2023-07-14T16:30:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb418101680 0x7fb418198e80 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180ffb00 0x7fb418197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb4080a11d0 0x7fb4080a3680 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 --2- 172.21.15.62:0/4140622429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb418100cf0 0x7fb418198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 -- 172.21.15.62:0/4140622429 >> 172.21.15.62:0/4140622429 conn(0x7fb4180fb0e0 msgr2=0x7fb4180fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 -- 172.21.15.62:0/4140622429 shutdown_connections 2023-07-14T16:30:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:09.629+0000 7fb41ebff700 1 -- 172.21.15.62:0/4140622429 wait complete. 2023-07-14T16:30:11.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:10 smithi062 ceph-mon[95543]: pgmap v793: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:11.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:10 smithi081 ceph-mon[127795]: pgmap v793: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:11.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:10 smithi164 ceph-mon[97427]: pgmap v793: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:13.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:12 smithi062 ceph-mon[95543]: pgmap v794: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:30:13.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:12 smithi081 ceph-mon[127795]: pgmap v794: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:30:13.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:12 smithi164 ceph-mon[97427]: pgmap v794: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:30:15.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:14 smithi062 ceph-mon[95543]: pgmap v795: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:30:15.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:14 smithi081 ceph-mon[127795]: pgmap v795: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:30:15.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:14 smithi164 ceph-mon[97427]: pgmap v795: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:30:17.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:16 smithi062 ceph-mon[95543]: pgmap v796: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:30:17.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:16 smithi081 ceph-mon[127795]: pgmap v796: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:30:17.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:16 smithi164 ceph-mon[97427]: pgmap v796: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:30:19.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:18 smithi062 ceph-mon[95543]: pgmap v797: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:19.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:18 smithi081 ceph-mon[127795]: pgmap v797: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:19.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:18 smithi164 ceph-mon[97427]: pgmap v797: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:19.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:30:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.035+0000 7f8116f9d700 1 -- 172.21.15.62:0/1227576147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110102e30 msgr2=0x7f8110105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.035+0000 7f8116f9d700 1 --2- 172.21.15.62:0/1227576147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110102e30 0x7f8110105210 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8100009990 tx=0x7f8100009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.036+0000 7f8116f9d700 1 -- 172.21.15.62:0/1227576147 shutdown_connections 2023-07-14T16:30:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.036+0000 7f8116f9d700 1 --2- 172.21.15.62:0/1227576147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8110105750 0x7f8110107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.036+0000 7f8116f9d700 1 --2- 172.21.15.62:0/1227576147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8110100510 0x7f81101028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.036+0000 7f8116f9d700 1 --2- 172.21.15.62:0/1227576147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110102e30 0x7f8110105210 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.036+0000 7f8116f9d700 1 -- 172.21.15.62:0/1227576147 >> 172.21.15.62:0/1227576147 conn(0x7f81100f9ed0 msgr2=0x7f81100fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.036+0000 7f8116f9d700 1 -- 172.21.15.62:0/1227576147 shutdown_connections 2023-07-14T16:30:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.036+0000 7f8116f9d700 1 -- 172.21.15.62:0/1227576147 wait complete. 2023-07-14T16:30:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8116f9d700 1 Processor -- start 2023-07-14T16:30:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8116f9d700 1 -- start start 2023-07-14T16:30:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8116f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110100510 0x7f8110197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8116f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8110102e30 0x7f8110197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8116f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8110105750 0x7f811019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8116f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811019f1d0 con 0x7f8110100510 2023-07-14T16:30:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8116f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811019f310 con 0x7f8110105750 2023-07-14T16:30:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8116f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811019f4a0 con 0x7f8110102e30 2023-07-14T16:30:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.037+0000 7f8115f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110100510 0x7f8110197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f8115f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110100510 0x7f8110197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40128/0 (socket says 172.21.15.62:40128) 2023-07-14T16:30:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f8115f9b700 1 -- 172.21.15.62:0/3257344541 learned_addr learned my addr 172.21.15.62:0/3257344541 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f811579a700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8110102e30 0x7f8110197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f811679c700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8110105750 0x7f811019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f8115f9b700 1 -- 172.21.15.62:0/3257344541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8110102e30 msgr2=0x7f8110197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f8115f9b700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8110102e30 0x7f8110197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f8115f9b700 1 -- 172.21.15.62:0/3257344541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8110105750 msgr2=0x7f811019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f8115f9b700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8110105750 0x7f811019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f8115f9b700 1 -- 172.21.15.62:0/3257344541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81000095f0 con 0x7f8110100510 2023-07-14T16:30:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.038+0000 7f8115f9b700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110100510 0x7f8110197650 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f810c00d980 tx=0x7f810c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.039+0000 7f8106ffd700 1 -- 172.21.15.62:0/3257344541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c004220 con 0x7f8110100510 2023-07-14T16:30:20.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.039+0000 7f8106ffd700 1 -- 172.21.15.62:0/3257344541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f810c007640 con 0x7f8110100510 2023-07-14T16:30:20.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.039+0000 7f8116f9d700 1 -- 172.21.15.62:0/3257344541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811019f780 con 0x7f8110100510 2023-07-14T16:30:20.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.039+0000 7f8106ffd700 1 -- 172.21.15.62:0/3257344541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c005660 con 0x7f8110100510 2023-07-14T16:30:20.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.039+0000 7f8116f9d700 1 -- 172.21.15.62:0/3257344541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811019fc90 con 0x7f8110100510 2023-07-14T16:30:20.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.042+0000 7f8106ffd700 1 -- 172.21.15.62:0/3257344541 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f810c004750 con 0x7f8110100510 2023-07-14T16:30:20.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f812bbfcb80 1 Processor -- start 2023-07-14T16:30:20.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f812bbfcb80 1 -- start start 2023-07-14T16:30:20.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a6720 0x5598736a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7070 0x5598736a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 0x5598736a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598735abcb0 con 0x5598736a6720 2023-07-14T16:30:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598735fea40 con 0x5598736a7e30 2023-07-14T16:30:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598735f7ed0 con 0x5598736a7070 2023-07-14T16:30:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f8106ffd700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80fc0a11f0 0x7f80fc0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.044+0000 7f8106ffd700 1 -- 172.21.15.62:0/3257344541 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f810c0cc700 con 0x7f8110100510 2023-07-14T16:30:20.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f811679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a6720 0x5598736a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f811679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a6720 0x5598736a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40138/0 (socket says 172.21.15.62:40138) 2023-07-14T16:30:20.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f8115f9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7070 0x5598736a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f811679c700 1 -- 172.21.15.62:0/3870367895 learned_addr learned my addr 172.21.15.62:0/3870367895 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f811679c700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80fc0a11f0 0x7f80fc0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f811579a700 1 --2- 172.21.15.62:0/3870367895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 0x5598736a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f8115f9b700 1 -- 172.21.15.62:0/3870367895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 msgr2=0x5598736a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f8115f9b700 1 --2- 172.21.15.62:0/3870367895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 0x5598736a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f8115f9b700 1 -- 172.21.15.62:0/3870367895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a6720 msgr2=0x5598736a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f8115f9b700 1 --2- 172.21.15.62:0/3870367895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a6720 0x5598736a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f8115f9b700 1 -- 172.21.15.62:0/3870367895 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559873425de0 con 0x5598736a7070 2023-07-14T16:30:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.045+0000 7f8115f9b700 1 --2- 172.21.15.62:0/3870367895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7070 0x5598736a78a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f810c0997c0 tx=0x7f810c0999c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09f95061765d7af server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.046+0000 7f811679c700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80fc0a11f0 0x7f80fc0a36a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8108009fd0 tx=0x7f810800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.046+0000 7f80fbfff700 1 -- 172.21.15.62:0/3870367895 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c099ac0 con 0x5598736a7070 2023-07-14T16:30:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.046+0000 7f80fbfff700 1 -- 172.21.15.62:0/3870367895 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f810c09daf0 con 0x5598736a7070 2023-07-14T16:30:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.046+0000 7f80fbfff700 1 -- 172.21.15.62:0/3870367895 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c098fb0 con 0x5598736a7070 2023-07-14T16:30:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.046+0000 7f812bbfcb80 1 -- 172.21.15.62:0/3870367895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7070 msgr2=0x5598736a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.046+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/3870367895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7070 0x5598736a78a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f810c0997c0 tx=0x7f810c0999c0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 -- 172.21.15.62:0/3870367895 shutdown_connections 2023-07-14T16:30:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/3870367895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 0x5598736a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/3870367895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a6720 0x5598736a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/3870367895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7070 0x5598736a78a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 -- 172.21.15.62:0/3870367895 >> 172.21.15.62:0/3870367895 conn(0x55987369bff0 msgr2=0x5598736a1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 -- 172.21.15.62:0/3870367895 shutdown_connections 2023-07-14T16:30:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 -- 172.21.15.62:0/3870367895 wait complete. 2023-07-14T16:30:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 -- start start 2023-07-14T16:30:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.047+0000 7f812bbfcb80 1 Processor -- start 2023-07-14T16:30:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7070 0x5598736b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7e30 0x5598736b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x559873597ac0 con 0x5598736a7e30 2023-07-14T16:30:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598735977c0 con 0x5598736a7070 2023-07-14T16:30:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55987347b920 con 0x5598736a6720 2023-07-14T16:30:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f812bbfcb80 10 client.0 _ll_register_callbacks cb 0x559873595760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f811679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7e30 0x5598736b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f811679c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7e30 0x5598736b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40152/0 (socket says 172.21.15.62:40152) 2023-07-14T16:30:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f811679c700 1 -- 172.21.15.62:0/103018163 learned_addr learned my addr 172.21.15.62:0/103018163 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f811579a700 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7070 0x5598736b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.048+0000 7f8115f9b700 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f811679c700 1 -- 172.21.15.62:0/103018163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 msgr2=0x5598736b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f811679c700 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f811679c700 1 -- 172.21.15.62:0/103018163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7070 msgr2=0x5598736b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f811679c700 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7070 0x5598736b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f811679c700 1 -- 172.21.15.62:0/103018163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811019fc90 con 0x5598736a7e30 2023-07-14T16:30:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f8115f9b700 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f811679c700 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7e30 0x5598736b77f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f810801b000 tx=0x7f8108004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df64f3313bc5ae7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f80f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f80f97fa700 1 -- 172.21.15.62:0/103018163 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108021070 con 0x5598736a7e30 2023-07-14T16:30:20.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f812bbfcb80 1 -- 172.21.15.62:0/103018163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598735514f0 con 0x5598736a7e30 2023-07-14T16:30:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.049+0000 7f80f97fa700 1 -- 172.21.15.62:0/103018163 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f810801cc00 con 0x5598736a7e30 2023-07-14T16:30:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f80f97fa700 1 -- 172.21.15.62:0/103018163 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f810800e070 con 0x5598736a7e30 2023-07-14T16:30:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f812bbfcb80 10 client.19593 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f812bbfcb80 1 -- 172.21.15.62:0/103018163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55987340ec60 con 0x5598736a7e30 2023-07-14T16:30:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f80f97fa700 1 -- 172.21.15.62:0/103018163 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5598735514f0 con 0x5598736a7e30 2023-07-14T16:30:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f80f97fa700 1 -- 172.21.15.62:0/103018163 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8108004cb0 con 0x5598736a7e30 2023-07-14T16:30:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f812bbfcb80 10 client.19593 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f812bbfcb80 10 client.19593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f812bbfcb80 20 client.19593 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5598736bc000 0x5598736be410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f812bbfcb80 4 client.19593 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.050+0000 7f812bbfcb80 1 -- 172.21.15.62:0/103018163 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598735f0eb0 con 0x5598736bc000 2023-07-14T16:30:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.051+0000 7f8115f9b700 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5598736bc000 0x5598736be410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.051+0000 7f8115f9b700 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5598736bc000 0x5598736be410 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ec3a48fa1fef01 server_cookie=b4d545a5684c9905 in_seq=0 out_seq=0 2023-07-14T16:30:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.052+0000 7f80f97fa700 10 client.19593 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.052+0000 7f80f97fa700 1 -- 172.21.15.62:0/103018163 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5598735f0eb0 con 0x5598736bc000 2023-07-14T16:30:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.052+0000 7f80f97fa700 10 client.19593 handle_command_reply: tid=0 2023-07-14T16:30:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.052+0000 7f812bbfcb80 1 client.19593 shutdown 2023-07-14T16:30:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 -- 172.21.15.62:0/103018163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7e30 msgr2=0x5598736b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7e30 0x5598736b77f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f810801b000 tx=0x7f8108004ad0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 -- 172.21.15.62:0/103018163 shutdown_connections 2023-07-14T16:30:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7e30 0x5598736b77f0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5598736bc000 0x5598736be410 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/103018163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7070 0x5598736b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 -- 172.21.15.62:0/103018163 >> 172.21.15.62:0/103018163 conn(0x55987369bff0 msgr2=0x5598736a0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f80f97fa700 0 client.19593 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 -- 172.21.15.62:0/103018163 shutdown_connections 2023-07-14T16:30:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 1 -- 172.21.15.62:0/103018163 wait complete. 2023-07-14T16:30:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.053+0000 7f812bbfcb80 20 client.19593 trim_cache size 0 max 16384 2023-07-14T16:30:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.073+0000 7f812bbfcb80 1 Processor -- start 2023-07-14T16:30:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.073+0000 7f812bbfcb80 1 -- start start 2023-07-14T16:30:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a6720 0x5598736b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7e30 0x5598736ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598736af720 con 0x5598736a7070 2023-07-14T16:30:20.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598736afde0 con 0x5598736a6720 2023-07-14T16:30:20.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598736a2f30 con 0x5598736a7e30 2023-07-14T16:30:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f811579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f811579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40164/0 (socket says 172.21.15.62:40164) 2023-07-14T16:30:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f811579a700 1 -- 172.21.15.62:0/2531480484 learned_addr learned my addr 172.21.15.62:0/2531480484 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f811679c700 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7e30 0x5598736ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.074+0000 7f8115f9b700 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a6720 0x5598736b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f811579a700 1 -- 172.21.15.62:0/2531480484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7e30 msgr2=0x5598736ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f811579a700 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7e30 0x5598736ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f811579a700 1 -- 172.21.15.62:0/2531480484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a6720 msgr2=0x5598736b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f811579a700 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a6720 0x5598736b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f811579a700 1 -- 172.21.15.62:0/2531480484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55987344c250 con 0x5598736a7070 2023-07-14T16:30:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f811679c700 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7e30 0x5598736ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f811579a700 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b7710 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f810000b2e0 tx=0x7f8100005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fd66b894d27b19 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f8104ff9700 1 -- 172.21.15.62:0/2531480484 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810001d030 con 0x5598736a7070 2023-07-14T16:30:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f8104ff9700 1 -- 172.21.15.62:0/2531480484 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f810001e040 con 0x5598736a7070 2023-07-14T16:30:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.075+0000 7f8104ff9700 1 -- 172.21.15.62:0/2531480484 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810000ba00 con 0x5598736a7070 2023-07-14T16:30:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 -- 172.21.15.62:0/2531480484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 msgr2=0x5598736b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b7710 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f810000b2e0 tx=0x7f8100005960 comp rx=0 tx=0).stop 2023-07-14T16:30:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 -- 172.21.15.62:0/2531480484 shutdown_connections 2023-07-14T16:30:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a7e30 0x5598736ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a6720 0x5598736b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/2531480484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b7710 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 -- 172.21.15.62:0/2531480484 >> 172.21.15.62:0/2531480484 conn(0x55987369bff0 msgr2=0x5598736aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 -- 172.21.15.62:0/2531480484 shutdown_connections 2023-07-14T16:30:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.076+0000 7f812bbfcb80 1 -- 172.21.15.62:0/2531480484 wait complete. 2023-07-14T16:30:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.077+0000 7f812bbfcb80 1 -- start start 2023-07-14T16:30:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.077+0000 7f812bbfcb80 1 Processor -- start 2023-07-14T16:30:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.077+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.077+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.077+0000 7f812bbfcb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 0x5598736b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.077+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598736b4a30 con 0x5598736a7070 2023-07-14T16:30:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.077+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598736b78b0 con 0x5598736a7e30 2023-07-14T16:30:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.077+0000 7f812bbfcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598736b7a70 con 0x5598736a6720 2023-07-14T16:30:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f812bbfcb80 10 client.0 _ll_register_callbacks cb 0x7f8108009350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811579a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40166/0 (socket says 172.21.15.62:40166) 2023-07-14T16:30:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811579a700 1 -- 172.21.15.62:0/498319312 learned_addr learned my addr 172.21.15.62:0/498319312 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f8115f9b700 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811679c700 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 0x5598736b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811579a700 1 -- 172.21.15.62:0/498319312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 msgr2=0x5598736b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811579a700 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811579a700 1 -- 172.21.15.62:0/498319312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 msgr2=0x5598736b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811579a700 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 0x5598736b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.078+0000 7f811579a700 1 -- 172.21.15.62:0/498319312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55987340ec60 con 0x5598736a7070 2023-07-14T16:30:20.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.079+0000 7f811579a700 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b3db0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8100005930 tx=0x7f81000038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79b8d050c241227 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.079+0000 7f80f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.079+0000 7f80f97fa700 1 -- 172.21.15.62:0/498319312 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810001d070 con 0x5598736a7070 2023-07-14T16:30:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.079+0000 7f812bbfcb80 1 -- 172.21.15.62:0/498319312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f810c09daf0 con 0x5598736a7070 2023-07-14T16:30:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.079+0000 7f80f97fa700 1 -- 172.21.15.62:0/498319312 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f810000b980 con 0x5598736a7070 2023-07-14T16:30:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.079+0000 7f80f97fa700 1 -- 172.21.15.62:0/498319312 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f810000bb80 con 0x5598736a7070 2023-07-14T16:30:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.079+0000 7f812bbfcb80 10 client.19599 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.079+0000 7f812bbfcb80 1 -- 172.21.15.62:0/498319312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f8110108700 con 0x5598736a7070 2023-07-14T16:30:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f80f97fa700 1 -- 172.21.15.62:0/498319312 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f810c09daf0 con 0x5598736a7070 2023-07-14T16:30:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f80f97fa700 1 -- 172.21.15.62:0/498319312 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8100023040 con 0x5598736a7070 2023-07-14T16:30:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f812bbfcb80 10 client.19599 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f812bbfcb80 10 client.19599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f812bbfcb80 20 client.19599 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559873711190 0x559873713570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f812bbfcb80 4 client.19599 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:30:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f812bbfcb80 1 -- 172.21.15.62:0/498319312 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598735b3050 con 0x559873711190 2023-07-14T16:30:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.080+0000 7f8115f9b700 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559873711190 0x559873713570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.081+0000 7f80f97fa700 10 client.19599 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.081+0000 7f8115f9b700 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559873711190 0x559873713570 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bb3a022acf6cd0 server_cookie=84e29476db79e19a in_seq=0 out_seq=0 2023-07-14T16:30:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.081+0000 7f80f97fa700 1 -- 172.21.15.62:0/498319312 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5598735b3050 con 0x559873711190 2023-07-14T16:30:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.081+0000 7f80f97fa700 10 client.19599 handle_command_reply: tid=0 2023-07-14T16:30:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.081+0000 7f812bbfcb80 1 client.19599 shutdown 2023-07-14T16:30:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.081+0000 7f812bbfcb80 1 -- 172.21.15.62:0/498319312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 msgr2=0x5598736b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.081+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b3db0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8100005930 tx=0x7f81000038b0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 1 -- 172.21.15.62:0/498319312 shutdown_connections 2023-07-14T16:30:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x559873711190 0x559873713570 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598736a7e30 0x5598736b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598736a6720 0x5598736b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 1 --2- 172.21.15.62:0/498319312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598736a7070 0x5598736b3db0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 1 -- 172.21.15.62:0/498319312 >> 172.21.15.62:0/498319312 conn(0x55987369bff0 msgr2=0x5598736b80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f80f97fa700 0 client.19599 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 1 -- 172.21.15.62:0/498319312 shutdown_connections 2023-07-14T16:30:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 1 -- 172.21.15.62:0/498319312 wait complete. 2023-07-14T16:30:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.082+0000 7f812bbfcb80 20 client.19599 trim_cache size 0 max 16384 2023-07-14T16:30:20.068 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:30:20.068 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:30:20.068 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:30:20.068 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:30:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 -- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80fc0a11f0 msgr2=0x7f80fc0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80fc0a11f0 0x7f80fc0a36a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8108009fd0 tx=0x7f810800b040 comp rx=0 tx=0).stop 2023-07-14T16:30:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 -- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110100510 msgr2=0x7f8110197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110100510 0x7f8110197650 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f810c00d980 tx=0x7f810c00dc90 comp rx=0 tx=0).stop 2023-07-14T16:30:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 -- 172.21.15.62:0/3257344541 shutdown_connections 2023-07-14T16:30:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f80fc0a11f0 0x7f80fc0a36a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8110105750 0x7f811019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8110100510 0x7f8110197650 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 --2- 172.21.15.62:0/3257344541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8110102e30 0x7f8110197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 -- 172.21.15.62:0/3257344541 >> 172.21.15.62:0/3257344541 conn(0x7f81100f9ed0 msgr2=0x7f81100feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 -- 172.21.15.62:0/3257344541 shutdown_connections 2023-07-14T16:30:20.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.083+0000 7f8116f9d700 1 -- 172.21.15.62:0/3257344541 wait complete. 2023-07-14T16:30:20.074 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:30:20.074 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:30:20.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2170426909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 msgr2=0x7ff138105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2170426909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 0x7ff138105220 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7ff120009990 tx=0x7ff120009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2170426909 shutdown_connections 2023-07-14T16:30:20.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2170426909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff138105760 0x7ff138107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2170426909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff138100520 0x7ff138102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2170426909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 0x7ff138105220 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2170426909 >> 172.21.15.62:0/2170426909 conn(0x7ff1380f9ee0 msgr2=0x7ff1380fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2170426909 shutdown_connections 2023-07-14T16:30:20.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.320+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2170426909 wait complete. 2023-07-14T16:30:20.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.321+0000 7ff13cb9d700 1 Processor -- start 2023-07-14T16:30:20.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.321+0000 7ff13cb9d700 1 -- start start 2023-07-14T16:30:20.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff13cb9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff138100520 0x7ff138195460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff13cb9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 0x7ff1381959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff13cb9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff138105760 0x7ff13819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff13cb9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13819cfe0 con 0x7ff138102e40 2023-07-14T16:30:20.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff13cb9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13819d120 con 0x7ff138105760 2023-07-14T16:30:20.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff13cb9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13819d2b0 con 0x7ff138100520 2023-07-14T16:30:20.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 0x7ff1381959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 0x7ff1381959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40190/0 (socket says 172.21.15.62:40190) 2023-07-14T16:30:20.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff12ffff700 1 -- 172.21.15.62:0/2833952226 learned_addr learned my addr 172.21.15.62:0/2833952226 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.322+0000 7ff1377fe700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff138100520 0x7ff138195460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff137fff700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff138105760 0x7ff13819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff12ffff700 1 -- 172.21.15.62:0/2833952226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff138100520 msgr2=0x7ff138195460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff12ffff700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff138100520 0x7ff138195460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff12ffff700 1 -- 172.21.15.62:0/2833952226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff138105760 msgr2=0x7ff13819ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff12ffff700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff138105760 0x7ff13819ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff12ffff700 1 -- 172.21.15.62:0/2833952226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200095f0 con 0x7ff138102e40 2023-07-14T16:30:20.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff12ffff700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 0x7ff1381959a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff1200098e0 tx=0x7ff120004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff1357fa700 1 -- 172.21.15.62:0/2833952226 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12001d070 con 0x7ff138102e40 2023-07-14T16:30:20.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13819d530 con 0x7ff138102e40 2023-07-14T16:30:20.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff1357fa700 1 -- 172.21.15.62:0/2833952226 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff12000be30 con 0x7ff138102e40 2023-07-14T16:30:20.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.323+0000 7ff1357fa700 1 -- 172.21.15.62:0/2833952226 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120022e00 con 0x7ff138102e40 2023-07-14T16:30:20.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.324+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13819d9b0 con 0x7ff138102e40 2023-07-14T16:30:20.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.325+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1380fba50 con 0x7ff138102e40 2023-07-14T16:30:20.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.327+0000 7ff1357fa700 1 -- 172.21.15.62:0/2833952226 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff12000b9a0 con 0x7ff138102e40 2023-07-14T16:30:20.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.328+0000 7ff1357fa700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1240a12c0 0x7ff1240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.329+0000 7ff1357fa700 1 -- 172.21.15.62:0/2833952226 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff1200cde30 con 0x7ff138102e40 2023-07-14T16:30:20.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.329+0000 7ff1377fe700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1240a12c0 0x7ff1240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.330+0000 7ff1377fe700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1240a12c0 0x7ff1240a3770 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff128009c60 tx=0x7ff128009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.333+0000 7ff1357fa700 1 -- 172.21.15.62:0/2833952226 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff1200975c0 con 0x7ff138102e40 2023-07-14T16:30:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.639+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff13802c850 con 0x7ff138102e40 2023-07-14T16:30:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.640+0000 7ff1357fa700 1 -- 172.21.15.62:0/2833952226 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10828 (secure 0 0 0) 0x7ff12009abe0 con 0x7ff138102e40 2023-07-14T16:30:20.614 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:30:20.614 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:29:03.831862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:30:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1240a12c0 msgr2=0x7ff1240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1240a12c0 0x7ff1240a3770 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff128009c60 tx=0x7ff128009260 comp rx=0 tx=0).stop 2023-07-14T16:30:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 msgr2=0x7ff1381959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 0x7ff1381959a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff1200098e0 tx=0x7ff120004730 comp rx=0 tx=0).stop 2023-07-14T16:30:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 shutdown_connections 2023-07-14T16:30:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff1240a12c0 0x7ff1240a3770 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff138105760 0x7ff13819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff138100520 0x7ff138195460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 --2- 172.21.15.62:0/2833952226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138102e40 0x7ff1381959a0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 >> 172.21.15.62:0/2833952226 conn(0x7ff1380f9ee0 msgr2=0x7ff1381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 shutdown_connections 2023-07-14T16:30:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.643+0000 7ff13cb9d700 1 -- 172.21.15.62:0/2833952226 wait complete. 2023-07-14T16:30:20.618 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2023-07-14T16:30:20.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:30:20.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/1851164584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f78102ef0 msgr2=0x7f4f781052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/1851164584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f78102ef0 0x7f4f781052d0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009990 tx=0x7f4f6c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/1851164584 shutdown_connections 2023-07-14T16:30:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/1851164584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f78105810 0x7f4f78107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/1851164584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f781005d0 0x7f4f781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/1851164584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f78102ef0 0x7f4f781052d0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/1851164584 >> 172.21.15.62:0/1851164584 conn(0x7f4f780fa070 msgr2=0x7f4f780fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/1851164584 shutdown_connections 2023-07-14T16:30:20.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.903+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/1851164584 wait complete. 2023-07-14T16:30:20.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.904+0000 7f4f7d5ff700 1 Processor -- start 2023-07-14T16:30:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f7d5ff700 1 -- start start 2023-07-14T16:30:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f7d5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f781005d0 0x7f4f78197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f7d5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f78102ef0 0x7f4f78197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f7d5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f78105810 0x7f4f7819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f7d5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7819f4c0 con 0x7f4f781005d0 2023-07-14T16:30:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f7d5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7819f600 con 0x7f4f78105810 2023-07-14T16:30:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f7d5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7819f790 con 0x7f4f78102ef0 2023-07-14T16:30:20.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f781005d0 0x7f4f78197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f781005d0 0x7f4f78197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40208/0 (socket says 172.21.15.62:40208) 2023-07-14T16:30:20.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.905+0000 7f4f76ffd700 1 -- 172.21.15.62:0/849808067 learned_addr learned my addr 172.21.15.62:0/849808067 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f767fc700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f78102ef0 0x7f4f78197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f777fe700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f78105810 0x7f4f7819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f76ffd700 1 -- 172.21.15.62:0/849808067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f78102ef0 msgr2=0x7f4f78197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f78102ef0 0x7f4f78197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f76ffd700 1 -- 172.21.15.62:0/849808067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f78105810 msgr2=0x7f4f7819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f78105810 0x7f4f7819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f76ffd700 1 -- 172.21.15.62:0/849808067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0095f0 con 0x7f4f781005d0 2023-07-14T16:30:20.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f781005d0 0x7f4f78197940 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d980 tx=0x7f4f6800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f5ffff700 1 -- 172.21.15.62:0/849808067 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800bb60 con 0x7f4f781005d0 2023-07-14T16:30:20.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/849808067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7819fa70 con 0x7f4f781005d0 2023-07-14T16:30:20.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f5ffff700 1 -- 172.21.15.62:0/849808067 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f68007640 con 0x7f4f781005d0 2023-07-14T16:30:20.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.906+0000 7f4f5ffff700 1 -- 172.21.15.62:0/849808067 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68004d60 con 0x7f4f781005d0 2023-07-14T16:30:20.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.907+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/849808067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7819ff80 con 0x7f4f781005d0 2023-07-14T16:30:20.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.910+0000 7f4f5ffff700 1 -- 172.21.15.62:0/849808067 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4f68004020 con 0x7f4f781005d0 2023-07-14T16:30:20.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.912+0000 7f4f90e66b80 1 Processor -- start 2023-07-14T16:30:20.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.912+0000 7f4f5ffff700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f600a1150 0x7f4f600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.912+0000 7f4f90e66b80 1 -- start start 2023-07-14T16:30:20.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.912+0000 7f4f5ffff700 1 -- 172.21.15.62:0/849808067 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f680cc860 con 0x7f4f781005d0 2023-07-14T16:30:20.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.912+0000 7f4f777fe700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f600a1150 0x7f4f600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.912+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.912+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 0x558b8a2b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a1bccb0 con 0x558b8a2b74d0 2023-07-14T16:30:20.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a20fa40 con 0x558b8a2b7e20 2023-07-14T16:30:20.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a208ed0 con 0x558b8a2b8c70 2023-07-14T16:30:20.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40210/0 (socket says 172.21.15.62:40210) 2023-07-14T16:30:20.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2020010929 learned_addr learned my addr 172.21.15.62:0/2020010929 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f767fc700 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 0x558b8a2b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f777fe700 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2020010929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 msgr2=0x558b8a2b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2020010929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 msgr2=0x558b8a2b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 0x558b8a2b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.913+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2020010929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b8a036de0 con 0x558b8a2b74d0 2023-07-14T16:30:20.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.914+0000 7f4f777fe700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f600a1150 0x7f4f600a3600 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009550 tx=0x7f4f700079a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.914+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2b78e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f4f68099bb0 tx=0x7f4f68099580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f5788a7fa579cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.914+0000 7f4f5d7fa700 1 -- 172.21.15.62:0/2020010929 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800de60 con 0x558b8a2b74d0 2023-07-14T16:30:20.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.914+0000 7f4f5d7fa700 1 -- 172.21.15.62:0/2020010929 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f68099c20 con 0x558b8a2b74d0 2023-07-14T16:30:20.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.914+0000 7f4f5d7fa700 1 -- 172.21.15.62:0/2020010929 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68099110 con 0x558b8a2b74d0 2023-07-14T16:30:20.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2020010929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 msgr2=0x558b8a2b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2b78e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f4f68099bb0 tx=0x7f4f68099580 comp rx=0 tx=0).stop 2023-07-14T16:30:20.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2020010929 shutdown_connections 2023-07-14T16:30:20.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2b78e0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2020010929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 0x558b8a2b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2020010929 >> 172.21.15.62:0/2020010929 conn(0x558b8a2acda0 msgr2=0x558b8a2b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2020010929 shutdown_connections 2023-07-14T16:30:20.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.915+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2020010929 wait complete. 2023-07-14T16:30:20.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.916+0000 7f4f90e66b80 1 -- start start 2023-07-14T16:30:20.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.916+0000 7f4f90e66b80 1 Processor -- start 2023-07-14T16:30:20.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.916+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.916+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 0x558b8a2c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a1a8ac0 con 0x558b8a2b74d0 2023-07-14T16:30:20.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a1a87c0 con 0x558b8a2b7e20 2023-07-14T16:30:20.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a08c920 con 0x558b8a2b8c70 2023-07-14T16:30:20.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f90e66b80 10 client.0 _ll_register_callbacks cb 0x558b8a1a6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:20.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40220/0 (socket says 172.21.15.62:40220) 2023-07-14T16:30:20.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2718840743 learned_addr learned my addr 172.21.15.62:0/2718840743 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f767fc700 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 0x558b8a2c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f777fe700 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2718840743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 msgr2=0x558b8a2c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2718840743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 msgr2=0x558b8a2c1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 0x558b8a2c1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2718840743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7819ff80 con 0x558b8a2b74d0 2023-07-14T16:30:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.917+0000 7f4f777fe700 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c1140 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f4f68098210 tx=0x7f4f6800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523c1afc07301bcb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f5aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f5aff5700 1 -- 172.21.15.62:0/2718840743 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680b2420 con 0x558b8a2b74d0 2023-07-14T16:30:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f5aff5700 1 -- 172.21.15.62:0/2718840743 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f680a2070 con 0x558b8a2b74d0 2023-07-14T16:30:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2718840743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b8a1624f0 con 0x558b8a2b74d0 2023-07-14T16:30:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f5aff5700 1 -- 172.21.15.62:0/2718840743 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f68099110 con 0x558b8a2b74d0 2023-07-14T16:30:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f90e66b80 10 client.19617 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2718840743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558b8a01fc60 con 0x558b8a2b74d0 2023-07-14T16:30:20.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.918+0000 7f4f5aff5700 1 -- 172.21.15.62:0/2718840743 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558b8a1624f0 con 0x558b8a2b74d0 2023-07-14T16:30:20.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.919+0000 7f4f5aff5700 1 -- 172.21.15.62:0/2718840743 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4f68095a10 con 0x558b8a2b74d0 2023-07-14T16:30:20.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.919+0000 7f4f90e66b80 10 client.19617 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:20.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.919+0000 7f4f90e66b80 10 client.19617 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:30:20.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.919+0000 7f4f90e66b80 20 client.19617 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:20.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.919+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558b8a2cca00 0x558b8a2cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.919+0000 7f4f90e66b80 4 client.19617 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:20.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.919+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2718840743 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b8a201eb0 con 0x558b8a2cca00 2023-07-14T16:30:20.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.919+0000 7f4f767fc700 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558b8a2cca00 0x558b8a2cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.920+0000 7f4f767fc700 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558b8a2cca00 0x558b8a2cee10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40b9920aab60df86 server_cookie=c46497f9bedbf48f in_seq=0 out_seq=0 2023-07-14T16:30:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.920+0000 7f4f5aff5700 10 client.19617 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:30:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.920+0000 7f4f5aff5700 1 -- 172.21.15.62:0/2718840743 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558b8a201eb0 con 0x558b8a2cca00 2023-07-14T16:30:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f5aff5700 10 client.19617 handle_command_reply: tid=0 2023-07-14T16:30:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 client.19617 shutdown 2023-07-14T16:30:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2718840743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 msgr2=0x558b8a2c1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c1140 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f4f68098210 tx=0x7f4f6800de60 comp rx=0 tx=0).stop 2023-07-14T16:30:20.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2718840743 shutdown_connections 2023-07-14T16:30:20.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558b8a2cca00 0x558b8a2cee10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c1140 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2718840743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b7e20 0x558b8a2c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f5aff5700 0 client.19617 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:30:20.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2718840743 >> 172.21.15.62:0/2718840743 conn(0x558b8a2acda0 msgr2=0x558b8a2b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2718840743 shutdown_connections 2023-07-14T16:30:20.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2718840743 wait complete. 2023-07-14T16:30:20.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.921+0000 7f4f90e66b80 20 client.19617 trim_cache size 0 max 16384 2023-07-14T16:30:20.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.937+0000 7f4f90e66b80 1 Processor -- start 2023-07-14T16:30:20.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.937+0000 7f4f90e66b80 1 -- start start 2023-07-14T16:30:20.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.937+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.937+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b7e20 0x558b8a2c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.937+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b8c70 0x558b8a2d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.937+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a2b30a0 con 0x558b8a2b74d0 2023-07-14T16:30:20.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.937+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a2b3610 con 0x558b8a2b8c70 2023-07-14T16:30:20.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.937+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a2d3c70 con 0x558b8a2b7e20 2023-07-14T16:30:20.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40236/0 (socket says 172.21.15.62:40236) 2023-07-14T16:30:20.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2839534479 learned_addr learned my addr 172.21.15.62:0/2839534479 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f767fc700 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b7e20 0x558b8a2c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2839534479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b7e20 msgr2=0x558b8a2c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f777fe700 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b8c70 0x558b8a2d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b7e20 0x558b8a2c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2839534479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b8c70 msgr2=0x558b8a2d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b8c70 0x558b8a2d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 -- 172.21.15.62:0/2839534479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b8a05d250 con 0x558b8a2b74d0 2023-07-14T16:30:20.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f777fe700 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b8c70 0x558b8a2d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:30:20.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.938+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c07f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d950 tx=0x7f4f6800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e6efb6c96c5c88 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.939+0000 7f4f5dffb700 1 -- 172.21.15.62:0/2839534479 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680b2420 con 0x558b8a2b74d0 2023-07-14T16:30:20.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.939+0000 7f4f5dffb700 1 -- 172.21.15.62:0/2839534479 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f68099580 con 0x558b8a2b74d0 2023-07-14T16:30:20.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.939+0000 7f4f5dffb700 1 -- 172.21.15.62:0/2839534479 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68098cd0 con 0x558b8a2b74d0 2023-07-14T16:30:20.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.939+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2839534479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 msgr2=0x558b8a2c07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.939+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c07f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d950 tx=0x7f4f6800be60 comp rx=0 tx=0).stop 2023-07-14T16:30:20.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2839534479 shutdown_connections 2023-07-14T16:30:20.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b8c70 0x558b8a2d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b74d0 0x558b8a2c07f0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/2839534479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b7e20 0x558b8a2c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2839534479 >> 172.21.15.62:0/2839534479 conn(0x558b8a2acda0 msgr2=0x558b8a2b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2839534479 shutdown_connections 2023-07-14T16:30:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 -- 172.21.15.62:0/2839534479 wait complete. 2023-07-14T16:30:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 -- start start 2023-07-14T16:30:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.940+0000 7f4f90e66b80 1 Processor -- start 2023-07-14T16:30:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b74d0 0x558b8a2c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b7e20 0x558b8a2c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f90e66b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a2c7e90 con 0x558b8a2b7e20 2023-07-14T16:30:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a2c8000 con 0x558b8a2b74d0 2023-07-14T16:30:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f90e66b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8a2c81c0 con 0x558b8a2b8c70 2023-07-14T16:30:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f90e66b80 10 client.0 _ll_register_callbacks cb 0x7f4f6c009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f767fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b7e20 0x558b8a2c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f767fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b7e20 0x558b8a2c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40240/0 (socket says 172.21.15.62:40240) 2023-07-14T16:30:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f767fc700 1 -- 172.21.15.62:0/3123927103 learned_addr learned my addr 172.21.15.62:0/3123927103 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:20.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.941+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b74d0 0x558b8a2c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f767fc700 1 -- 172.21.15.62:0/3123927103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 msgr2=0x558b8a2c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f767fc700 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f767fc700 1 -- 172.21.15.62:0/3123927103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b74d0 msgr2=0x558b8a2c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:20.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f767fc700 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b74d0 0x558b8a2c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f767fc700 1 -- 172.21.15.62:0/3123927103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b8a01fc60 con 0x558b8a2b7e20 2023-07-14T16:30:20.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f767fc700 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b7e20 0x558b8a2c4930 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00c010 tx=0x7f4f6c029b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcfce03a8811fa2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:20.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f5aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:20.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f5aff5700 1 -- 172.21.15.62:0/3123927103 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c00bb70 con 0x558b8a2b7e20 2023-07-14T16:30:20.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f90e66b80 1 -- 172.21.15.62:0/3123927103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f68099c20 con 0x558b8a2b7e20 2023-07-14T16:30:20.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.942+0000 7f4f5aff5700 1 -- 172.21.15.62:0/3123927103 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f6c00bcd0 con 0x558b8a2b7e20 2023-07-14T16:30:20.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f5aff5700 1 -- 172.21.15.62:0/3123927103 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f6c0290b0 con 0x558b8a2b7e20 2023-07-14T16:30:20.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f90e66b80 10 client.19623 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:20.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f90e66b80 1 -- 172.21.15.62:0/3123927103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558b8a0352b0 con 0x558b8a2b7e20 2023-07-14T16:30:20.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f5aff5700 1 -- 172.21.15.62:0/3123927103 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f6c038020 con 0x558b8a2b7e20 2023-07-14T16:30:20.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f5aff5700 1 -- 172.21.15.62:0/3123927103 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4f6c0099c0 con 0x558b8a2b7e20 2023-07-14T16:30:20.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f90e66b80 10 client.19623 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:20.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f90e66b80 10 client.19623 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:30:20.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f90e66b80 20 client.19623 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:20.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558b8a321840 0x558b8a323c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:20.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f90e66b80 4 client.19623 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:30:20.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.943+0000 7f4f90e66b80 1 -- 172.21.15.62:0/3123927103 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558b8a1c4050 con 0x558b8a321840 2023-07-14T16:30:20.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.944+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558b8a321840 0x558b8a323c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:20.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.944+0000 7f4f76ffd700 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558b8a321840 0x558b8a323c50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e074d3b6aa32cbf server_cookie=fe145586d9a8a4c8 in_seq=0 out_seq=0 2023-07-14T16:30:20.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.944+0000 7f4f5aff5700 10 client.19623 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:30:20.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.944+0000 7f4f5aff5700 1 -- 172.21.15.62:0/3123927103 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558b8a1c4050 con 0x558b8a321840 2023-07-14T16:30:20.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.944+0000 7f4f5aff5700 10 client.19623 handle_command_reply: tid=0 2023-07-14T16:30:20.924 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.944+0000 7f4f90e66b80 1 client.19623 shutdown 2023-07-14T16:30:20.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.944+0000 7f4f90e66b80 1 -- 172.21.15.62:0/3123927103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b7e20 msgr2=0x558b8a2c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b7e20 0x558b8a2c4930 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00c010 tx=0x7f4f6c029b40 comp rx=0 tx=0).stop 2023-07-14T16:30:20.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 -- 172.21.15.62:0/3123927103 shutdown_connections 2023-07-14T16:30:20.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x558b8a321840 0x558b8a323c50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558b8a2b8c70 0x558b8a2c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f5aff5700 0 client.19623 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:30:20.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b8a2b74d0 0x558b8a2c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 --2- 172.21.15.62:0/3123927103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558b8a2b7e20 0x558b8a2c4930 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 -- 172.21.15.62:0/3123927103 >> 172.21.15.62:0/3123927103 conn(0x558b8a2acda0 msgr2=0x558b8a2bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 -- 172.21.15.62:0/3123927103 shutdown_connections 2023-07-14T16:30:20.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 1 -- 172.21.15.62:0/3123927103 wait complete. 2023-07-14T16:30:20.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.945+0000 7f4f90e66b80 20 client.19623 trim_cache size 0 max 16384 2023-07-14T16:30:20.926 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:30:20.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f600a1150 msgr2=0x7f4f600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f600a1150 0x7f4f600a3600 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009550 tx=0x7f4f700079a0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f781005d0 msgr2=0x7f4f78197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:20.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f781005d0 0x7f4f78197940 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d980 tx=0x7f4f6800dd40 comp rx=0 tx=0).stop 2023-07-14T16:30:20.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/849808067 shutdown_connections 2023-07-14T16:30:20.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4f600a1150 0x7f4f600a3600 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f78105810 0x7f4f7819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f781005d0 0x7f4f78197940 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 --2- 172.21.15.62:0/849808067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f78102ef0 0x7f4f78197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:20.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/849808067 >> 172.21.15.62:0/849808067 conn(0x7f4f780fa070 msgr2=0x7f4f78101230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:20.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/849808067 shutdown_connections 2023-07-14T16:30:20.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:20.946+0000 7f4f7d5ff700 1 -- 172.21.15.62:0/849808067 wait complete. 2023-07-14T16:30:20.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:20 smithi062 ceph-mon[95543]: pgmap v798: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 17 MiB/s wr, 16 op/s 2023-07-14T16:30:20.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:20 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2833952226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:30:20.938 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:30:21.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:20 smithi081 ceph-mon[127795]: pgmap v798: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 17 MiB/s wr, 16 op/s 2023-07-14T16:30:21.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:20 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2833952226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:30:21.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 -- 172.21.15.62:0/1263297769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 msgr2=0x7f2f341051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 --2- 172.21.15.62:0/1263297769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 0x7f2f341051f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009990 tx=0x7f2f28009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 -- 172.21.15.62:0/1263297769 shutdown_connections 2023-07-14T16:30:21.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 --2- 172.21.15.62:0/1263297769 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f34105730 0x7f2f34107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 --2- 172.21.15.62:0/1263297769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f341004f0 0x7f2f341028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 --2- 172.21.15.62:0/1263297769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 0x7f2f341051f0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 -- 172.21.15.62:0/1263297769 >> 172.21.15.62:0/1263297769 conn(0x7f2f340f9ed0 msgr2=0x7f2f340fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 -- 172.21.15.62:0/1263297769 shutdown_connections 2023-07-14T16:30:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.206+0000 7f2f33fff700 1 -- 172.21.15.62:0/1263297769 wait complete. 2023-07-14T16:30:21.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.207+0000 7f2f33fff700 1 Processor -- start 2023-07-14T16:30:21.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.207+0000 7f2f33fff700 1 -- start start 2023-07-14T16:30:21.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.207+0000 7f2f33fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f341004f0 0x7f2f341976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.207+0000 7f2f33fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 0x7f2f34197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.207+0000 7f2f33fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f34105730 0x7f2f3419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.207+0000 7f2f33fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419f270 con 0x7f2f34102e10 2023-07-14T16:30:21.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f33fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419f3b0 con 0x7f2f341004f0 2023-07-14T16:30:21.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f33fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419f540 con 0x7f2f34105730 2023-07-14T16:30:21.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 0x7f2f34197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 0x7f2f34197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40266/0 (socket says 172.21.15.62:40266) 2023-07-14T16:30:21.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f327fc700 1 -- 172.21.15.62:0/466988446 learned_addr learned my addr 172.21.15.62:0/466988446 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f337fe700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f34105730 0x7f2f3419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f341004f0 0x7f2f341976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f327fc700 1 -- 172.21.15.62:0/466988446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f34105730 msgr2=0x7f2f3419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f327fc700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f34105730 0x7f2f3419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f327fc700 1 -- 172.21.15.62:0/466988446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f341004f0 msgr2=0x7f2f341976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f327fc700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f341004f0 0x7f2f341976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.208+0000 7f2f327fc700 1 -- 172.21.15.62:0/466988446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f280095f0 con 0x7f2f34102e10 2023-07-14T16:30:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.209+0000 7f2f327fc700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 0x7f2f34197c30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2f28000c00 tx=0x7f2f28003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.209+0000 7f2f1bfff700 1 -- 172.21.15.62:0/466988446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2801d070 con 0x7f2f34102e10 2023-07-14T16:30:21.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.209+0000 7f2f1bfff700 1 -- 172.21.15.62:0/466988446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f2800bd70 con 0x7f2f34102e10 2023-07-14T16:30:21.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.209+0000 7f2f33fff700 1 -- 172.21.15.62:0/466988446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3419f7c0 con 0x7f2f34102e10 2023-07-14T16:30:21.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.209+0000 7f2f1bfff700 1 -- 172.21.15.62:0/466988446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28022c80 con 0x7f2f34102e10 2023-07-14T16:30:21.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.209+0000 7f2f33fff700 1 -- 172.21.15.62:0/466988446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3419fbe0 con 0x7f2f34102e10 2023-07-14T16:30:21.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.213+0000 7f2f1bfff700 1 -- 172.21.15.62:0/466988446 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2f28022470 con 0x7f2f34102e10 2023-07-14T16:30:21.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.214+0000 7f2f4cf09b80 1 Processor -- start 2023-07-14T16:30:21.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.214+0000 7f2f4cf09b80 1 -- start start 2023-07-14T16:30:21.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 0x55bce40b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce3fb3cb0 con 0x55bce40b78a0 2023-07-14T16:30:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce4006a40 con 0x55bce40b6f30 2023-07-14T16:30:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce3fffed0 con 0x55bce40b8670 2023-07-14T16:30:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f1bfff700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f1c0a11f0 0x7f2f1c0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 0x55bce40b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40b8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40274/0 (socket says 172.21.15.62:40274) 2023-07-14T16:30:21.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f32ffd700 1 -- 172.21.15.62:0/3117503079 learned_addr learned my addr 172.21.15.62:0/3117503079 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f337fe700 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f1bfff700 1 -- 172.21.15.62:0/466988446 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f280cdf80 con 0x7f2f34102e10 2023-07-14T16:30:21.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.215+0000 7f2f337fe700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f1c0a11f0 0x7f2f1c0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f32ffd700 1 -- 172.21.15.62:0/3117503079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 msgr2=0x55bce40b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 0x55bce40b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f32ffd700 1 -- 172.21.15.62:0/3117503079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 msgr2=0x55bce40b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f32ffd700 1 -- 172.21.15.62:0/3117503079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce3e2dde0 con 0x55bce40b78a0 2023-07-14T16:30:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f337fe700 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40b8090 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400bd70 tx=0x7f2f2400f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce25f25702179af server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f337fe700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f1c0a11f0 0x7f2f1c0a36a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55bce40b6cd0 tx=0x7f2f2c009100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f197fa700 1 -- 172.21.15.62:0/3117503079 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400fda0 con 0x55bce40b78a0 2023-07-14T16:30:21.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f197fa700 1 -- 172.21.15.62:0/3117503079 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f2401a8f0 con 0x55bce40b78a0 2023-07-14T16:30:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.216+0000 7f2f197fa700 1 -- 172.21.15.62:0/3117503079 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24011730 con 0x55bce40b78a0 2023-07-14T16:30:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/3117503079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 msgr2=0x55bce40b8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40b8090 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400bd70 tx=0x7f2f2400f640 comp rx=0 tx=0).stop 2023-07-14T16:30:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/3117503079 shutdown_connections 2023-07-14T16:30:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 0x55bce40b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/3117503079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40b8090 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/3117503079 >> 172.21.15.62:0/3117503079 conn(0x55bce40a3ff0 msgr2=0x55bce40a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/3117503079 shutdown_connections 2023-07-14T16:30:21.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.217+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/3117503079 wait complete. 2023-07-14T16:30:21.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.218+0000 7f2f4cf09b80 1 -- start start 2023-07-14T16:30:21.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.218+0000 7f2f4cf09b80 1 Processor -- start 2023-07-14T16:30:21.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b6f30 0x55bce40c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b8670 0x55bce40c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce3f9fac0 con 0x55bce40b78a0 2023-07-14T16:30:21.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce3f9f7c0 con 0x55bce40b8670 2023-07-14T16:30:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce3e83920 con 0x55bce40b6f30 2023-07-14T16:30:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f4cf09b80 10 client.0 _ll_register_callbacks cb 0x55bce3f9d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40c10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40290/0 (socket says 172.21.15.62:40290) 2023-07-14T16:30:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f327fc700 1 -- 172.21.15.62:0/2777535757 learned_addr learned my addr 172.21.15.62:0/2777535757 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.219+0000 7f2f337fe700 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b8670 0x55bce40c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b6f30 0x55bce40c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f327fc700 1 -- 172.21.15.62:0/2777535757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b6f30 msgr2=0x55bce40c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f327fc700 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b6f30 0x55bce40c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f327fc700 1 -- 172.21.15.62:0/2777535757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b8670 msgr2=0x55bce40c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f327fc700 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b8670 0x55bce40c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f327fc700 1 -- 172.21.15.62:0/2777535757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2400b9a0 con 0x55bce40b78a0 2023-07-14T16:30:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f327fc700 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40c10a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2f28097930 tx=0x7f2f2806d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9df8f61a11524 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f16ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:21.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f16ff5700 1 -- 172.21.15.62:0/2777535757 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2809b200 con 0x55bce40b78a0 2023-07-14T16:30:21.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f16ff5700 1 -- 172.21.15.62:0/2777535757 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f28003f40 con 0x55bce40b78a0 2023-07-14T16:30:21.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.220+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/2777535757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bce3f594f0 con 0x55bce40b78a0 2023-07-14T16:30:21.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f16ff5700 1 -- 172.21.15.62:0/2777535757 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f2809af70 con 0x55bce40b78a0 2023-07-14T16:30:21.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f4cf09b80 10 client.19635 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:21.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/2777535757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bce3e16c60 con 0x55bce40b78a0 2023-07-14T16:30:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f16ff5700 1 -- 172.21.15.62:0/2777535757 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bce3f594f0 con 0x55bce40b78a0 2023-07-14T16:30:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f16ff5700 1 -- 172.21.15.62:0/2777535757 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2f280d0110 con 0x55bce40b78a0 2023-07-14T16:30:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f4cf09b80 10 client.19635 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f4cf09b80 10 client.19635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:30:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f4cf09b80 20 client.19635 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bce40cc480 0x55bce40ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f4cf09b80 4 client.19635 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.221+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/2777535757 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bce3ff8eb0 con 0x55bce40cc480 2023-07-14T16:30:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.222+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bce40cc480 0x55bce40ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.222+0000 7f2f16ff5700 10 client.19635 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:30:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.222+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bce40cc480 0x55bce40ce890 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=182605d26ffb515 server_cookie=afbf9739d7c82ce in_seq=0 out_seq=0 2023-07-14T16:30:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.223+0000 7f2f16ff5700 1 -- 172.21.15.62:0/2777535757 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bce3ff8eb0 con 0x55bce40cc480 2023-07-14T16:30:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.223+0000 7f2f16ff5700 10 client.19635 handle_command_reply: tid=0 2023-07-14T16:30:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.223+0000 7f2f4cf09b80 1 client.19635 shutdown 2023-07-14T16:30:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/2777535757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 msgr2=0x55bce40c10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40c10a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2f28097930 tx=0x7f2f2806d4f0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/2777535757 shutdown_connections 2023-07-14T16:30:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bce40cc480 0x55bce40ce890 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b8670 0x55bce40c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b6f30 0x55bce40c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/2777535757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b78a0 0x55bce40c10a0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/2777535757 >> 172.21.15.62:0/2777535757 conn(0x55bce40a3ff0 msgr2=0x55bce40a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f16ff5700 0 client.19635 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:30:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/2777535757 shutdown_connections 2023-07-14T16:30:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/2777535757 wait complete. 2023-07-14T16:30:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.224+0000 7f2f4cf09b80 20 client.19635 trim_cache size 0 max 16384 2023-07-14T16:30:21.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.239+0000 7f2f4cf09b80 1 Processor -- start 2023-07-14T16:30:21.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.239+0000 7f2f4cf09b80 1 -- start start 2023-07-14T16:30:21.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.239+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40aa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.239+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b78a0 0x55bce40c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.239+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b8670 0x55bce40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.239+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce40d3730 con 0x55bce40b8670 2023-07-14T16:30:21.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.239+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce40bf560 con 0x55bce40b6f30 2023-07-14T16:30:21.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.239+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce40aae70 con 0x55bce40b78a0 2023-07-14T16:30:21.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b78a0 0x55bce40c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b8670 0x55bce40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b78a0 0x55bce40c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35546/0 (socket says 172.21.15.62:35546) 2023-07-14T16:30:21.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 -- 172.21.15.62:0/664419743 learned_addr learned my addr 172.21.15.62:0/664419743 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 -- 172.21.15.62:0/664419743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 msgr2=0x55bce40aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 -- 172.21.15.62:0/664419743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b8670 msgr2=0x55bce40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b8670 0x55bce40d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 -- 172.21.15.62:0/664419743 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce3e54250 con 0x55bce40b78a0 2023-07-14T16:30:21.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f337fe700 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b8670 0x55bce40d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:21.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.240+0000 7f2f327fc700 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b78a0 0x55bce40c80a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f28099780 tx=0x7f2f28009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7ae3a33e03eda2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.241+0000 7f2f19ffb700 1 -- 172.21.15.62:0/664419743 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2809b200 con 0x55bce40b78a0 2023-07-14T16:30:21.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.241+0000 7f2f19ffb700 1 -- 172.21.15.62:0/664419743 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f2809af70 con 0x55bce40b78a0 2023-07-14T16:30:21.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.241+0000 7f2f19ffb700 1 -- 172.21.15.62:0/664419743 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2809b430 con 0x55bce40b78a0 2023-07-14T16:30:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.241+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/664419743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b78a0 msgr2=0x55bce40c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.241+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b78a0 0x55bce40c80a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f28099780 tx=0x7f2f28009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/664419743 shutdown_connections 2023-07-14T16:30:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b8670 0x55bce40d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b6f30 0x55bce40aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/664419743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b78a0 0x55bce40c80a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/664419743 >> 172.21.15.62:0/664419743 conn(0x55bce40a3ff0 msgr2=0x55bce40a9430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/664419743 shutdown_connections 2023-07-14T16:30:21.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/664419743 wait complete. 2023-07-14T16:30:21.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 -- start start 2023-07-14T16:30:21.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.242+0000 7f2f4cf09b80 1 Processor -- start 2023-07-14T16:30:21.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b6f30 0x55bce40c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b78a0 0x55bce40c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f4cf09b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 0x55bce40c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.220 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce40c77e0 con 0x55bce40b6f30 2023-07-14T16:30:21.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce40c7950 con 0x55bce40b78a0 2023-07-14T16:30:21.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f4cf09b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce40c7b10 con 0x55bce40b8670 2023-07-14T16:30:21.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f4cf09b80 10 client.0 _ll_register_callbacks cb 0x55bce3f9d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:21.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b6f30 0x55bce40c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b6f30 0x55bce40c3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40304/0 (socket says 172.21.15.62:40304) 2023-07-14T16:30:21.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.243+0000 7f2f32ffd700 1 -- 172.21.15.62:0/1488246803 learned_addr learned my addr 172.21.15.62:0/1488246803 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f337fe700 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 0x55bce40c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f327fc700 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b78a0 0x55bce40c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f32ffd700 1 -- 172.21.15.62:0/1488246803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 msgr2=0x55bce40c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 0x55bce40c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f32ffd700 1 -- 172.21.15.62:0/1488246803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b78a0 msgr2=0x55bce40c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b78a0 0x55bce40c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f32ffd700 1 -- 172.21.15.62:0/1488246803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bce3e16c60 con 0x55bce40b6f30 2023-07-14T16:30:21.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f32ffd700 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b6f30 0x55bce40c3d40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400f990 tx=0x7f2f24005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbde8953ced911a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.244+0000 7f2f16ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:21.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f16ff5700 1 -- 172.21.15.62:0/1488246803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400fe60 con 0x55bce40b6f30 2023-07-14T16:30:21.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f16ff5700 1 -- 172.21.15.62:0/1488246803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f2401aa60 con 0x55bce40b6f30 2023-07-14T16:30:21.223 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/1488246803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f2401a8f0 con 0x55bce40b6f30 2023-07-14T16:30:21.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f16ff5700 1 -- 172.21.15.62:0/1488246803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f2401ac40 con 0x55bce40b6f30 2023-07-14T16:30:21.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 10 client.19641 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:21.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/1488246803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bce3e2c2b0 con 0x55bce40b6f30 2023-07-14T16:30:21.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f16ff5700 1 -- 172.21.15.62:0/1488246803 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f24038020 con 0x55bce40b6f30 2023-07-14T16:30:21.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f16ff5700 1 -- 172.21.15.62:0/1488246803 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2f24004390 con 0x55bce40b6f30 2023-07-14T16:30:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 10 client.19641 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 10 client.19641 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:30:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 20 client.19641 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bce4121170 0x55bce4123580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 4 client.19641 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:30:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.245+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/1488246803 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bce3fbb050 con 0x55bce4121170 2023-07-14T16:30:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.246+0000 7f2f327fc700 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bce4121170 0x55bce4123580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f16ff5700 10 client.19641 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:30:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f327fc700 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bce4121170 0x55bce4123580 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed7e793e5e9dcd50 server_cookie=854b9a2437bd906 in_seq=0 out_seq=0 2023-07-14T16:30:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f16ff5700 1 -- 172.21.15.62:0/1488246803 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bce3fbb050 con 0x55bce4121170 2023-07-14T16:30:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f16ff5700 10 client.19641 handle_command_reply: tid=0 2023-07-14T16:30:21.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 client.19641 shutdown 2023-07-14T16:30:21.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/1488246803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b6f30 msgr2=0x55bce40c3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b6f30 0x55bce40c3d40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400f990 tx=0x7f2f24005d40 comp rx=0 tx=0).stop 2023-07-14T16:30:21.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/1488246803 shutdown_connections 2023-07-14T16:30:21.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55bce4121170 0x55bce4123580 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bce40b8670 0x55bce40c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bce40b6f30 0x55bce40c3d40 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 --2- 172.21.15.62:0/1488246803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55bce40b78a0 0x55bce40c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/1488246803 >> 172.21.15.62:0/1488246803 conn(0x55bce40a3ff0 msgr2=0x55bce40a93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f16ff5700 0 client.19641 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:30:21.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/1488246803 shutdown_connections 2023-07-14T16:30:21.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.247+0000 7f2f4cf09b80 1 -- 172.21.15.62:0/1488246803 wait complete. 2023-07-14T16:30:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.248+0000 7f2f4cf09b80 20 client.19641 trim_cache size 0 max 16384 2023-07-14T16:30:21.232 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:30:21.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.248+0000 7f2f33fff700 1 -- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f1c0a11f0 msgr2=0x7f2f1c0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.248+0000 7f2f33fff700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f1c0a11f0 0x7f2f1c0a36a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55bce40b6cd0 tx=0x7f2f2c009100 comp rx=0 tx=0).stop 2023-07-14T16:30:21.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.248+0000 7f2f33fff700 1 -- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 msgr2=0x7f2f34197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.248+0000 7f2f33fff700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 0x7f2f34197c30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2f28000c00 tx=0x7f2f28003a50 comp rx=0 tx=0).stop 2023-07-14T16:30:21.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.249+0000 7f2f33fff700 1 -- 172.21.15.62:0/466988446 shutdown_connections 2023-07-14T16:30:21.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.249+0000 7f2f33fff700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f34105730 0x7f2f3419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.235 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.249+0000 7f2f33fff700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2f1c0a11f0 0x7f2f1c0a36a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.249+0000 7f2f33fff700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f341004f0 0x7f2f341976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.249+0000 7f2f33fff700 1 --2- 172.21.15.62:0/466988446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f34102e10 0x7f2f34197c30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.249+0000 7f2f33fff700 1 -- 172.21.15.62:0/466988446 >> 172.21.15.62:0/466988446 conn(0x7f2f340f9ed0 msgr2=0x7f2f34101930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.249+0000 7f2f33fff700 1 -- 172.21.15.62:0/466988446 shutdown_connections 2023-07-14T16:30:21.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.249+0000 7f2f33fff700 1 -- 172.21.15.62:0/466988446 wait complete. 2023-07-14T16:30:21.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:30:21.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:20 smithi164 ceph-mon[97427]: pgmap v798: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 17 MiB/s wr, 16 op/s 2023-07-14T16:30:21.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:20 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2833952226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:30:21.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/785147962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 msgr2=0x7f6d9410a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/785147962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 0x7f6d9410a150 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00b0a0 tx=0x7f6d8c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/785147962 shutdown_connections 2023-07-14T16:30:21.469 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/785147962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 0x7f6d9410a150 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/785147962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d94101d10 0x7f6d94102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/785147962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d94102f00 0x7f6d94103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/785147962 >> 172.21.15.62:0/785147962 conn(0x7f6d940fd350 msgr2=0x7f6d940ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/785147962 shutdown_connections 2023-07-14T16:30:21.471 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.495+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/785147962 wait complete. 2023-07-14T16:30:21.474 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.496+0000 7f6d9a7ff700 1 Processor -- start 2023-07-14T16:30:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.496+0000 7f6d9a7ff700 1 -- start start 2023-07-14T16:30:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.496+0000 7f6d9a7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d94101d10 0x7f6d94197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.496+0000 7f6d9a7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d94102f00 0x7f6d94197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.496+0000 7f6d9a7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 0x7f6d9419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.496+0000 7f6d9a7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9419f5e0 con 0x7f6d94103890 2023-07-14T16:30:21.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.496+0000 7f6d9a7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9419f720 con 0x7f6d94101d10 2023-07-14T16:30:21.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.496+0000 7f6d9a7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9419f8b0 con 0x7f6d94102f00 2023-07-14T16:30:21.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 0x7f6d9419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d937fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d94102f00 0x7f6d94197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 0x7f6d9419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40320/0 (socket says 172.21.15.62:40320) 2023-07-14T16:30:21.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d98d9c700 1 -- 172.21.15.62:0/463413008 learned_addr learned my addr 172.21.15.62:0/463413008 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d93fff700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d94101d10 0x7f6d94197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d98d9c700 1 -- 172.21.15.62:0/463413008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d94102f00 msgr2=0x7f6d94197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d94102f00 0x7f6d94197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d98d9c700 1 -- 172.21.15.62:0/463413008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d94101d10 msgr2=0x7f6d94197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d94101d10 0x7f6d94197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.497+0000 7f6d98d9c700 1 -- 172.21.15.62:0/463413008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c009d00 con 0x7f6d94103890 2023-07-14T16:30:21.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.498+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 0x7f6d9419f010 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00b070 tx=0x7f6d8c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.498+0000 7f6d917fa700 1 -- 172.21.15.62:0/463413008 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00e040 con 0x7f6d94103890 2023-07-14T16:30:21.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.498+0000 7f6d917fa700 1 -- 172.21.15.62:0/463413008 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d8c003e90 con 0x7f6d94103890 2023-07-14T16:30:21.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.498+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/463413008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9419fb30 con 0x7f6d94103890 2023-07-14T16:30:21.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.498+0000 7f6d917fa700 1 -- 172.21.15.62:0/463413008 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c01b450 con 0x7f6d94103890 2023-07-14T16:30:21.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.498+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/463413008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9419ffb0 con 0x7f6d94103890 2023-07-14T16:30:21.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.502+0000 7f6d917fa700 1 -- 172.21.15.62:0/463413008 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6d8c004020 con 0x7f6d94103890 2023-07-14T16:30:21.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6dae1a7b80 1 Processor -- start 2023-07-14T16:30:21.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6dae1a7b80 1 -- start start 2023-07-14T16:30:21.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6d917fa700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d7c0a11d0 0x7f6d7c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50bee94d0 0x55e50bee98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50beea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50beeac70 0x55e50beeb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bdeecb0 con 0x55e50bee94d0 2023-07-14T16:30:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50be41a40 con 0x55e50bee9e20 2023-07-14T16:30:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50be3aed0 con 0x55e50beeac70 2023-07-14T16:30:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.504+0000 7f6d917fa700 1 -- 172.21.15.62:0/463413008 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d8c0ce130 con 0x7f6d94103890 2023-07-14T16:30:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d937fe700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d7c0a11d0 0x7f6d7c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50beeac70 0x55e50beeb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50beeac70 0x55e50beeb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35598/0 (socket says 172.21.15.62:35598) 2023-07-14T16:30:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d98d9c700 1 -- 172.21.15.62:0/770902634 learned_addr learned my addr 172.21.15.62:0/770902634 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d93fff700 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50bee94d0 0x55e50bee98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d937fe700 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50beea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d93fff700 1 -- 172.21.15.62:0/770902634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50beeac70 msgr2=0x55e50beeb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d93fff700 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50beeac70 0x55e50beeb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d93fff700 1 -- 172.21.15.62:0/770902634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 msgr2=0x55e50beea6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d93fff700 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50beea6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d93fff700 1 -- 172.21.15.62:0/770902634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e50bc68de0 con 0x55e50bee94d0 2023-07-14T16:30:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50beeac70 0x55e50beeb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.505+0000 7f6d937fe700 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50beea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.506+0000 7f6d937fe700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d7c0a11d0 0x7f6d7c0a3680 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6d880076a0 tx=0x7f6d88012780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.506+0000 7f6d93fff700 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50bee94d0 0x55e50bee98e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6d84007c30 tx=0x7f6d84007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2695e13dd9ce683 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.506+0000 7f6d7a7fc700 1 -- 172.21.15.62:0/770902634 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84010ca0 con 0x55e50bee94d0 2023-07-14T16:30:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.506+0000 7f6d7a7fc700 1 -- 172.21.15.62:0/770902634 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d840044e0 con 0x55e50bee94d0 2023-07-14T16:30:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.506+0000 7f6d7a7fc700 1 -- 172.21.15.62:0/770902634 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84005020 con 0x55e50bee94d0 2023-07-14T16:30:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/770902634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50bee94d0 msgr2=0x55e50bee98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50bee94d0 0x55e50bee98e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6d84007c30 tx=0x7f6d84007f40 comp rx=0 tx=0).stop 2023-07-14T16:30:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/770902634 shutdown_connections 2023-07-14T16:30:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50beeac70 0x55e50beeb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50bee94d0 0x55e50bee98e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/770902634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50beea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/770902634 >> 172.21.15.62:0/770902634 conn(0x55e50bededa0 msgr2=0x55e50bee41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/770902634 shutdown_connections 2023-07-14T16:30:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.507+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/770902634 wait complete. 2023-07-14T16:30:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 1 -- start start 2023-07-14T16:30:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 1 Processor -- start 2023-07-14T16:30:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50bef3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50befa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bddaac0 con 0x55e50beeac70 2023-07-14T16:30:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bdda7c0 con 0x55e50bee9e20 2023-07-14T16:30:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bcbe920 con 0x55e50bee94d0 2023-07-14T16:30:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.508+0000 7f6dae1a7b80 10 client.0 _ll_register_callbacks cb 0x55e50bdd8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50befa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d937fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50bef3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50befa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40344/0 (socket says 172.21.15.62:40344) 2023-07-14T16:30:21.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 -- 172.21.15.62:0/3363163989 learned_addr learned my addr 172.21.15.62:0/3363163989 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d93fff700 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 -- 172.21.15.62:0/3363163989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 msgr2=0x55e50bef3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 -- 172.21.15.62:0/3363163989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 msgr2=0x55e50bef3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50bef3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 -- 172.21.15.62:0/3363163989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9419ffb0 con 0x55e50beeac70 2023-07-14T16:30:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d937fe700 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50bef3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50befa690 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c098050 tx=0x7f6d8c0bb120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36fe563c0bd9238 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.509+0000 7f6d737fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6d737fe700 1 -- 172.21.15.62:0/3363163989 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c09af40 con 0x55e50beeac70 2023-07-14T16:30:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3363163989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e50bd944f0 con 0x55e50beeac70 2023-07-14T16:30:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6d737fe700 1 -- 172.21.15.62:0/3363163989 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d8c009440 con 0x55e50beeac70 2023-07-14T16:30:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6d737fe700 1 -- 172.21.15.62:0/3363163989 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8c014b00 con 0x55e50beeac70 2023-07-14T16:30:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 10 client.19653 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3363163989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f6d84007890 con 0x55e50beeac70 2023-07-14T16:30:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6d737fe700 1 -- 172.21.15.62:0/3363163989 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e50bd944f0 con 0x55e50beeac70 2023-07-14T16:30:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6d737fe700 1 -- 172.21.15.62:0/3363163989 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d8c086070 con 0x55e50beeac70 2023-07-14T16:30:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 10 client.19653 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 10 client.19653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 20 client.19653 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e50befec60 0x55e50bf01070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 4 client.19653 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.510+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3363163989 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e50be33eb0 con 0x55e50befec60 2023-07-14T16:30:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.511+0000 7f6d93fff700 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e50befec60 0x55e50bf01070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.511+0000 7f6d737fe700 10 client.19653 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.511+0000 7f6d93fff700 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e50befec60 0x55e50bf01070 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9bfa03ea0a2875 server_cookie=d589814b16ec2c4 in_seq=0 out_seq=0 2023-07-14T16:30:21.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.512+0000 7f6d737fe700 1 -- 172.21.15.62:0/3363163989 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e50be33eb0 con 0x55e50befec60 2023-07-14T16:30:21.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.512+0000 7f6d737fe700 10 client.19653 handle_command_reply: tid=0 2023-07-14T16:30:21.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.512+0000 7f6dae1a7b80 1 client.19653 shutdown 2023-07-14T16:30:21.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3363163989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 msgr2=0x55e50befa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50befa690 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c098050 tx=0x7f6d8c0bb120 comp rx=0 tx=0).stop 2023-07-14T16:30:21.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3363163989 shutdown_connections 2023-07-14T16:30:21.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50befa690 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e50befec60 0x55e50bf01070 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3363163989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50bef3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6d737fe700 0 client.19653 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:21.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3363163989 >> 172.21.15.62:0/3363163989 conn(0x55e50bededa0 msgr2=0x55e50bee38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3363163989 shutdown_connections 2023-07-14T16:30:21.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3363163989 wait complete. 2023-07-14T16:30:21.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.513+0000 7f6dae1a7b80 20 client.19653 trim_cache size 0 max 16384 2023-07-14T16:30:21.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.529+0000 7f6dae1a7b80 1 Processor -- start 2023-07-14T16:30:21.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6dae1a7b80 1 -- start start 2023-07-14T16:30:21.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee94d0 0x55e50bf059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee9e20 0x55e50befa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bee38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bef0d90 con 0x55e50beeac70 2023-07-14T16:30:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50beee6a0 con 0x55e50bee94d0 2023-07-14T16:30:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bf05f10 con 0x55e50bee9e20 2023-07-14T16:30:21.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bee38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bee38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40346/0 (socket says 172.21.15.62:40346) 2023-07-14T16:30:21.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6d98d9c700 1 -- 172.21.15.62:0/3035238424 learned_addr learned my addr 172.21.15.62:0/3035238424 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6d937fe700 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee9e20 0x55e50befa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.530+0000 7f6d93fff700 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee94d0 0x55e50bf059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.531+0000 7f6d937fe700 1 -- 172.21.15.62:0/3035238424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee94d0 msgr2=0x55e50bf059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.531+0000 7f6d937fe700 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee94d0 0x55e50bf059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.531+0000 7f6d937fe700 1 -- 172.21.15.62:0/3035238424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 msgr2=0x55e50bee38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.531+0000 7f6d937fe700 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bee38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.531+0000 7f6d937fe700 1 -- 172.21.15.62:0/3035238424 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e50bc51c60 con 0x55e50bee9e20 2023-07-14T16:30:21.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.531+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bee38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:21.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.531+0000 7f6d937fe700 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee9e20 0x55e50befa650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005fd0 tx=0x7f6d8801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45c181642f646cb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.532+0000 7f6d7affd700 1 -- 172.21.15.62:0/3035238424 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88004a30 con 0x55e50bee9e20 2023-07-14T16:30:21.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.532+0000 7f6d7affd700 1 -- 172.21.15.62:0/3035238424 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d88024070 con 0x55e50bee9e20 2023-07-14T16:30:21.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.532+0000 7f6d7affd700 1 -- 172.21.15.62:0/3035238424 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d880296e0 con 0x55e50bee9e20 2023-07-14T16:30:21.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3035238424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee9e20 msgr2=0x55e50befa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee9e20 0x55e50befa650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005fd0 tx=0x7f6d8801d000 comp rx=0 tx=0).stop 2023-07-14T16:30:21.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3035238424 shutdown_connections 2023-07-14T16:30:21.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bee38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee94d0 0x55e50bf059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/3035238424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee9e20 0x55e50befa650 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3035238424 >> 172.21.15.62:0/3035238424 conn(0x55e50bededa0 msgr2=0x55e50bee41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3035238424 shutdown_connections 2023-07-14T16:30:21.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/3035238424 wait complete. 2023-07-14T16:30:21.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 -- start start 2023-07-14T16:30:21.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.533+0000 7f6dae1a7b80 1 Processor -- start 2023-07-14T16:30:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50befa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6dae1a7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bef6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bef7060 con 0x55e50beeac70 2023-07-14T16:30:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bef71d0 con 0x55e50bee9e20 2023-07-14T16:30:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6dae1a7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e50bef7390 con 0x55e50bee94d0 2023-07-14T16:30:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6dae1a7b80 10 client.0 _ll_register_callbacks cb 0x7f6d8c097260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bef6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6d93fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6d98d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bef6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40348/0 (socket says 172.21.15.62:40348) 2023-07-14T16:30:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6d98d9c700 1 -- 172.21.15.62:0/2181902708 learned_addr learned my addr 172.21.15.62:0/2181902708 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.534+0000 7f6d937fe700 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50befa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d98d9c700 1 -- 172.21.15.62:0/2181902708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 msgr2=0x55e50bef9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d98d9c700 1 -- 172.21.15.62:0/2181902708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 msgr2=0x55e50befa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50befa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d98d9c700 1 -- 172.21.15.62:0/2181902708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d84007890 con 0x55e50beeac70 2023-07-14T16:30:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d93fff700 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d98d9c700 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bef6a40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009fd0 tx=0x7f6d8c09b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d14981531c7b8ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d737fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d737fe700 1 -- 172.21.15.62:0/2181902708 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c09af40 con 0x55e50beeac70 2023-07-14T16:30:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/2181902708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d840044e0 con 0x55e50beeac70 2023-07-14T16:30:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d737fe700 1 -- 172.21.15.62:0/2181902708 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d8c009440 con 0x55e50beeac70 2023-07-14T16:30:21.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6d737fe700 1 -- 172.21.15.62:0/2181902708 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8c080030 con 0x55e50beeac70 2023-07-14T16:30:21.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6dae1a7b80 10 client.19659 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:21.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.535+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/2181902708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f6d880103e0 con 0x55e50beeac70 2023-07-14T16:30:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6d737fe700 1 -- 172.21.15.62:0/2181902708 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d840044e0 con 0x55e50beeac70 2023-07-14T16:30:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6d737fe700 1 -- 172.21.15.62:0/2181902708 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d8c0a3510 con 0x55e50beeac70 2023-07-14T16:30:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6dae1a7b80 10 client.19659 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6dae1a7b80 10 client.19659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6dae1a7b80 20 client.19659 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e50bf53d00 0x55e50bf56110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6dae1a7b80 4 client.19659 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:30:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/2181902708 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e50bdf6050 con 0x55e50bf53d00 2023-07-14T16:30:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.536+0000 7f6d93fff700 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e50bf53d00 0x55e50bf56110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.537+0000 7f6d93fff700 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e50bf53d00 0x55e50bf56110 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85ad14a6359b3ea5 server_cookie=775f9e886fc56fa6 in_seq=0 out_seq=0 2023-07-14T16:30:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.537+0000 7f6d737fe700 10 client.19659 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.537+0000 7f6d737fe700 1 -- 172.21.15.62:0/2181902708 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e50bdf6050 con 0x55e50bf53d00 2023-07-14T16:30:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.537+0000 7f6d737fe700 10 client.19659 handle_command_reply: tid=0 2023-07-14T16:30:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.537+0000 7f6dae1a7b80 1 client.19659 shutdown 2023-07-14T16:30:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.537+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/2181902708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 msgr2=0x55e50bef6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.537+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bef6a40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009fd0 tx=0x7f6d8c09b1e0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/2181902708 shutdown_connections 2023-07-14T16:30:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e50beeac70 0x55e50bef6a40 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e50bee94d0 0x55e50bef9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e50bf53d00 0x55e50bf56110 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 1 --2- 172.21.15.62:0/2181902708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e50bee9e20 0x55e50befa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/2181902708 >> 172.21.15.62:0/2181902708 conn(0x55e50bededa0 msgr2=0x55e50beedb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6d737fe700 0 client.19659 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/2181902708 shutdown_connections 2023-07-14T16:30:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 1 -- 172.21.15.62:0/2181902708 wait complete. 2023-07-14T16:30:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.538+0000 7f6dae1a7b80 20 client.19659 trim_cache size 0 max 16384 2023-07-14T16:30:21.530 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:30:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d7c0a11d0 msgr2=0x7f6d7c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d7c0a11d0 0x7f6d7c0a3680 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6d880076a0 tx=0x7f6d88012780 comp rx=0 tx=0).stop 2023-07-14T16:30:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 msgr2=0x7f6d9419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 0x7f6d9419f010 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00b070 tx=0x7f6d8c004c80 comp rx=0 tx=0).stop 2023-07-14T16:30:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/463413008 shutdown_connections 2023-07-14T16:30:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d94103890 0x7f6d9419f010 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6d7c0a11d0 0x7f6d7c0a3680 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d94101d10 0x7f6d94197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 --2- 172.21.15.62:0/463413008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d94102f00 0x7f6d94197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/463413008 >> 172.21.15.62:0/463413008 conn(0x7f6d940fd350 msgr2=0x7f6d941063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/463413008 shutdown_connections 2023-07-14T16:30:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:21.539+0000 7f6d9a7ff700 1 -- 172.21.15.62:0/463413008 wait complete. 2023-07-14T16:30:22.534 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:30:22.534 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:30:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.799+0000 7f21a69ff700 1 -- 172.21.15.62:0/374006108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 msgr2=0x7f21a0103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.799+0000 7f21a69ff700 1 --2- 172.21.15.62:0/374006108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 0x7f21a0103350 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2194009990 tx=0x7f2194009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.799+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1041235868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8068640 msgr2=0x7fc0e81048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.799+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1041235868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8068640 0x7fc0e81048d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009990 tx=0x7fc0d8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.799+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1041235868 shutdown_connections 2023-07-14T16:30:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.799+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1041235868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0e8107730 0x7fc0e8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.799+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1041235868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8068640 0x7fc0e81048d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.799+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1041235868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8104e10 0x7fc0e81071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.799+0000 7f21a69ff700 1 -- 172.21.15.62:0/374006108 shutdown_connections 2023-07-14T16:30:22.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.799+0000 7f21a69ff700 1 --2- 172.21.15.62:0/374006108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0103890 0x7f21a010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.799+0000 7f21a69ff700 1 --2- 172.21.15.62:0/374006108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0101d10 0x7f21a0102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.799+0000 7f21a69ff700 1 --2- 172.21.15.62:0/374006108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 0x7f21a0103350 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.775 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.799+0000 7f21a69ff700 1 -- 172.21.15.62:0/374006108 >> 172.21.15.62:0/374006108 conn(0x7f21a00fd350 msgr2=0x7f21a00ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.799+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1041235868 >> 172.21.15.62:0/1041235868 conn(0x7fc0e80f9e40 msgr2=0x7fc0e80fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.799+0000 7f21a69ff700 1 -- 172.21.15.62:0/374006108 shutdown_connections 2023-07-14T16:30:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.800+0000 7f21a69ff700 1 -- 172.21.15.62:0/374006108 wait complete. 2023-07-14T16:30:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.800+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1041235868 shutdown_connections 2023-07-14T16:30:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.800+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1041235868 wait complete. 2023-07-14T16:30:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.801+0000 7fc0ecf9d700 1 Processor -- start 2023-07-14T16:30:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.801+0000 7f21a69ff700 1 Processor -- start 2023-07-14T16:30:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.801+0000 7fc0ecf9d700 1 -- start start 2023-07-14T16:30:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.801+0000 7f21a69ff700 1 -- start start 2023-07-14T16:30:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.801+0000 7fc0ecf9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0e8068640 0x7fc0e8197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.801+0000 7f21a69ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0101d10 0x7f21a01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.801+0000 7f21a69ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 0x7f21a0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.801+0000 7fc0ecf9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8104e10 0x7fc0e8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.801+0000 7fc0ecf9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8107730 0x7fc0e819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.801+0000 7fc0ecf9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e819f390 con 0x7fc0e8104e10 2023-07-14T16:30:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.801+0000 7fc0ecf9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e819f4d0 con 0x7fc0e8068640 2023-07-14T16:30:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.801+0000 7fc0ecf9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e819f660 con 0x7fc0e8107730 2023-07-14T16:30:22.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.801+0000 7f21a69ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0103890 0x7f21a019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.801+0000 7f21a69ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019f540 con 0x7f21a0102f00 2023-07-14T16:30:22.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.801+0000 7f21a69ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019f680 con 0x7f21a0103890 2023-07-14T16:30:22.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.801+0000 7f21a69ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019f810 con 0x7f21a0101d10 2023-07-14T16:30:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0e8068640 0x7fc0e8197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0e8068640 0x7fc0e8197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57190/0 (socket says 172.21.15.62:57190) 2023-07-14T16:30:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e77fe700 1 -- 172.21.15.62:0/1547728389 learned_addr learned my addr 172.21.15.62:0/1547728389 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e7fff700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8107730 0x7fc0e819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0101d10 0x7f21a01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0101d10 0x7f21a01979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:43106/0 (socket says 172.21.15.62:43106) 2023-07-14T16:30:22.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219ffff700 1 -- 172.21.15.62:0/439414813 learned_addr learned my addr 172.21.15.62:0/439414813 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0103890 0x7f21a019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8104e10 0x7fc0e8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219f7fe700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 0x7f21a0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e7fff700 1 -- 172.21.15.62:0/1547728389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0e8068640 msgr2=0x7fc0e8197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e7fff700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0e8068640 0x7fc0e8197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e7fff700 1 -- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8104e10 msgr2=0x7fc0e8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e7fff700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8104e10 0x7fc0e8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.802+0000 7fc0e7fff700 1 -- 172.21.15.62:0/1547728389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d80095f0 con 0x7fc0e8107730 2023-07-14T16:30:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219ffff700 1 -- 172.21.15.62:0/439414813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0103890 msgr2=0x7f21a019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219ffff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0103890 0x7f21a019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219ffff700 1 -- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 msgr2=0x7f21a0197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219ffff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 0x7f21a0197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.802+0000 7f219ffff700 1 -- 172.21.15.62:0/439414813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940095f0 con 0x7f21a0101d10 2023-07-14T16:30:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.803+0000 7f219f7fe700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 0x7f21a0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.803+0000 7f219ffff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0101d10 0x7f21a01979c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f219000e970 tx=0x7f219000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.803+0000 7fc0e7fff700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8107730 0x7fc0e819edc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0007f10 tx=0x7fc0e000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.803+0000 7f219d7fa700 1 -- 172.21.15.62:0/439414813 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190009790 con 0x7f21a0101d10 2023-07-14T16:30:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.803+0000 7f219d7fa700 1 -- 172.21.15.62:0/439414813 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f219000eec0 con 0x7f21a0101d10 2023-07-14T16:30:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.803+0000 7f219d7fa700 1 -- 172.21.15.62:0/439414813 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190003c90 con 0x7f21a0101d10 2023-07-14T16:30:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.803+0000 7f21a69ff700 1 -- 172.21.15.62:0/439414813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a019faf0 con 0x7f21a0101d10 2023-07-14T16:30:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.803+0000 7fc0e4ff9700 1 -- 172.21.15.62:0/1547728389 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0004db0 con 0x7fc0e8107730 2023-07-14T16:30:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.803+0000 7fc0e4ff9700 1 -- 172.21.15.62:0/1547728389 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0e000f040 con 0x7fc0e8107730 2023-07-14T16:30:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.803+0000 7fc0e4ff9700 1 -- 172.21.15.62:0/1547728389 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0005960 con 0x7fc0e8107730 2023-07-14T16:30:22.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.803+0000 7f21a69ff700 1 -- 172.21.15.62:0/439414813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a01a0000 con 0x7f21a0101d10 2023-07-14T16:30:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.803+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1547728389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e819f940 con 0x7fc0e8107730 2023-07-14T16:30:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.804+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1547728389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e819fe00 con 0x7fc0e8107730 2023-07-14T16:30:22.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.809+0000 7f219d7fa700 1 -- 172.21.15.62:0/439414813 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2190003df0 con 0x7f21a0101d10 2023-07-14T16:30:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.809+0000 7fc0e4ff9700 1 -- 172.21.15.62:0/1547728389 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc0e000db20 con 0x7fc0e8107730 2023-07-14T16:30:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.809+0000 7fc101a99b80 1 Processor -- start 2023-07-14T16:30:22.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.809+0000 7f21ba2f4b80 1 Processor -- start 2023-07-14T16:30:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.810+0000 7fc101a99b80 1 -- start start 2023-07-14T16:30:22.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.810+0000 7f21ba2f4b80 1 -- start start 2023-07-14T16:30:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.810+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141aa1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.810+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb693960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.810+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb694760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.810+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb695160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.810+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa1a90 0x55c141aa22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.810+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa27e0 0x55c141aa2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.810+0000 7fc101a99b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1419a6c50 con 0x55c141aa27e0 2023-07-14T16:30:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.810+0000 7fc101a99b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1419f99e0 con 0x55c141aa1a90 2023-07-14T16:30:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.810+0000 7fc101a99b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1419f2e70 con 0x55c1418fd6b0 2023-07-14T16:30:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.810+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb598d30 con 0x5612cb693550 2023-07-14T16:30:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.810+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb5ebac0 con 0x5612cb694cf0 2023-07-14T16:30:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.810+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb5e4f50 con 0x5612cb693ea0 2023-07-14T16:30:22.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.811+0000 7f219f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb695160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.811+0000 7f219f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb695160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57194/0 (socket says 172.21.15.62:57194) 2023-07-14T16:30:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.811+0000 7f21a4f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb693960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.811+0000 7f219f7fe700 1 -- 172.21.15.62:0/1500398531 learned_addr learned my addr 172.21.15.62:0/1500398531 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa27e0 0x55c141aa2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa27e0 0x55c141aa2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50612/0 (socket says 172.21.15.62:50612) 2023-07-14T16:30:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/1096037517 learned_addr learned my addr 172.21.15.62:0/1096037517 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141aa1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa1a90 0x55c141aa22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.811+0000 7f219d7fa700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21880a11d0 0x7f21880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.811+0000 7f219f7fe700 1 -- 172.21.15.62:0/1500398531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 msgr2=0x5612cb694760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.811+0000 7f219ffff700 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb694760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/1096037517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa1a90 msgr2=0x55c141aa22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa1a90 0x55c141aa22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/1096037517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa27e0 msgr2=0x55c141aa2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa27e0 0x55c141aa2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.811+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/1096037517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c141820d80 con 0x55c1418fd6b0 2023-07-14T16:30:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.812+0000 7f219f7fe700 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb694760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.812+0000 7f219f7fe700 1 -- 172.21.15.62:0/1500398531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 msgr2=0x5612cb693960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.812+0000 7f219f7fe700 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb693960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.812+0000 7f219f7fe700 1 -- 172.21.15.62:0/1500398531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612cb412e60 con 0x5612cb694cf0 2023-07-14T16:30:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.812+0000 7f219d7fa700 1 -- 172.21.15.62:0/439414813 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2190014070 con 0x7f21a0101d10 2023-07-14T16:30:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.812+0000 7f219ffff700 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb694760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:30:22.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.812+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21880a11d0 0x7f21880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.812+0000 7fc0e4ff9700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc0d00a12a0 0x7fc0d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.812+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa1a90 0x55c141aa22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.812+0000 7fc0e4ff9700 1 -- 172.21.15.62:0/1547728389 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0e00cc0c0 con 0x7fc0e8107730 2023-07-14T16:30:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.812+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141aa1550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc012880 tx=0x7fc0dc00b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=897e63fa6bb46e7d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.812+0000 7f219f7fe700 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb695160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2194006010 tx=0x7f2194005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afac99ab1a89add1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.812+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc0d00a12a0 0x7fc0d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.813+0000 7fc0cdffb700 1 -- 172.21.15.62:0/1096037517 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc004030 con 0x55c1418fd6b0 2023-07-14T16:30:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.813+0000 7fc0cdffb700 1 -- 172.21.15.62:0/1096037517 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0dc003680 con 0x55c1418fd6b0 2023-07-14T16:30:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.813+0000 7fc0cdffb700 1 -- 172.21.15.62:0/1096037517 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00f650 con 0x55c1418fd6b0 2023-07-14T16:30:22.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.813+0000 7f21867fc700 1 -- 172.21.15.62:0/1500398531 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940047e0 con 0x5612cb694cf0 2023-07-14T16:30:22.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.813+0000 7f21867fc700 1 -- 172.21.15.62:0/1500398531 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f219401e040 con 0x5612cb694cf0 2023-07-14T16:30:22.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.813+0000 7f21867fc700 1 -- 172.21.15.62:0/1500398531 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194003680 con 0x5612cb694cf0 2023-07-14T16:30:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 -- 172.21.15.62:0/1096037517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 msgr2=0x55c141aa1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc0d00a12a0 0x7fc0d00a3750 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55c141a9ccb0 tx=0x7fc0d800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141aa1550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc012880 tx=0x7fc0dc00b410 comp rx=0 tx=0).stop 2023-07-14T16:30:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 -- 172.21.15.62:0/1096037517 shutdown_connections 2023-07-14T16:30:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa27e0 0x55c141aa2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141aa1550 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 --2- 172.21.15.62:0/1096037517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa1a90 0x55c141aa22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 -- 172.21.15.62:0/1096037517 >> 172.21.15.62:0/1096037517 conn(0x55c141a96d40 msgr2=0x55c141a9be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1500398531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 msgr2=0x5612cb695160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb695160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2194006010 tx=0x7f2194005c80 comp rx=0 tx=0).stop 2023-07-14T16:30:22.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21880a11d0 0x7f21880a3680 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f21980075c0 tx=0x7f2198009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 -- 172.21.15.62:0/1096037517 shutdown_connections 2023-07-14T16:30:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.814+0000 7fc101a99b80 1 -- 172.21.15.62:0/1096037517 wait complete. 2023-07-14T16:30:22.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1500398531 shutdown_connections 2023-07-14T16:30:22.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb695160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb693960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1500398531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb694760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1500398531 >> 172.21.15.62:0/1500398531 conn(0x5612cb688e20 msgr2=0x5612cb68e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1500398531 shutdown_connections 2023-07-14T16:30:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.814+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1500398531 wait complete. 2023-07-14T16:30:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.815+0000 7fc101a99b80 1 -- start start 2023-07-14T16:30:22.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.815+0000 7f21ba2f4b80 1 -- start start 2023-07-14T16:30:22.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.815+0000 7f21ba2f4b80 1 Processor -- start 2023-07-14T16:30:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.815+0000 7fc101a99b80 1 Processor -- start 2023-07-14T16:30:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.815+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141ab32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.815+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.815+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa27e0 0x55c141aba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.815+0000 7fc101a99b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c141992a60 con 0x55c141aa1a90 2023-07-14T16:30:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.815+0000 7fc101a99b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c141992760 con 0x55c141aa27e0 2023-07-14T16:30:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.815+0000 7fc101a99b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1418768c0 con 0x55c1418fd6b0 2023-07-14T16:30:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.816+0000 7fc101a99b80 10 client.0 _ll_register_callbacks cb 0x55c141990700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:22.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb69d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb69d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb6a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb584b40 con 0x5612cb693550 2023-07-14T16:30:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb584840 con 0x5612cb694cf0 2023-07-14T16:30:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb4689a0 con 0x5612cb693ea0 2023-07-14T16:30:22.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f21ba2f4b80 10 client.0 _ll_register_callbacks cb 0x5612cb5827e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.816+0000 7fc0e77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141ab32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.816+0000 7fc0e77fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141ab32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:43142/0 (socket says 172.21.15.62:43142) 2023-07-14T16:30:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.816+0000 7fc0e77fe700 1 -- 172.21.15.62:0/3518166780 learned_addr learned my addr 172.21.15.62:0/3518166780 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.816+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.816+0000 7fc0e7fff700 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa27e0 0x55c141aba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f219f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb69d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f219f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb69d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:43146/0 (socket says 172.21.15.62:43146) 2023-07-14T16:30:22.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f219f7fe700 1 -- 172.21.15.62:0/1940418170 learned_addr learned my addr 172.21.15.62:0/1940418170 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb6a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.816+0000 7f219ffff700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb69d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.817+0000 7fc0e77fe700 1 -- 172.21.15.62:0/3518166780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa27e0 msgr2=0x55c141aba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.817+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa27e0 0x55c141aba870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.817+0000 7fc0e77fe700 1 -- 172.21.15.62:0/3518166780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 msgr2=0x55c141ab3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.817+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.817+0000 7fc0e77fe700 1 -- 172.21.15.62:0/3518166780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc01a040 con 0x55c1418fd6b0 2023-07-14T16:30:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f21a4f9c700 1 -- 172.21.15.62:0/1940418170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 msgr2=0x5612cb69d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb69d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f21a4f9c700 1 -- 172.21.15.62:0/1940418170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 msgr2=0x5612cb69d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb69d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f21a4f9c700 1 -- 172.21.15.62:0/1940418170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a010aa80 con 0x5612cb694cf0 2023-07-14T16:30:22.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f219f7fe700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb69d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:22.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f219ffff700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb69d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.817+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141ab32f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8005490 tx=0x7fc0d8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=497e7676f52e7562 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.817+0000 7fc0c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:30:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb6a4720 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2198007d10 tx=0x7f2198004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbc8e42154d2881e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.817+0000 7f217f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:30:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.818+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/3518166780 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d802c070 con 0x55c1418fd6b0 2023-07-14T16:30:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.818+0000 7fc101a99b80 1 -- 172.21.15.62:0/3518166780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c14194c490 con 0x55c1418fd6b0 2023-07-14T16:30:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.818+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/3518166780 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0d8004aa0 con 0x55c1418fd6b0 2023-07-14T16:30:22.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.818+0000 7f217f7fe700 1 -- 172.21.15.62:0/1940418170 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800e040 con 0x5612cb694cf0 2023-07-14T16:30:22.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.818+0000 7f217f7fe700 1 -- 172.21.15.62:0/1940418170 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f219801f050 con 0x5612cb694cf0 2023-07-14T16:30:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.818+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/3518166780 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0d801d040 con 0x55c1418fd6b0 2023-07-14T16:30:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.818+0000 7fc101a99b80 10 client.29095 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.818+0000 7fc101a99b80 1 -- 172.21.15.62:0/3518166780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c141809c00 con 0x55c1418fd6b0 2023-07-14T16:30:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.818+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1940418170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612cb53e570 con 0x5612cb694cf0 2023-07-14T16:30:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.818+0000 7f217f7fe700 1 -- 172.21.15.62:0/1940418170 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2198003e50 con 0x5612cb694cf0 2023-07-14T16:30:22.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.818+0000 7f21ba2f4b80 10 client.29237 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:22.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.818+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1940418170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5612cb3fbce0 con 0x5612cb694cf0 2023-07-14T16:30:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/3518166780 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c14194c490 con 0x55c1418fd6b0 2023-07-14T16:30:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/3518166780 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0d8025480 con 0x55c1418fd6b0 2023-07-14T16:30:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc101a99b80 10 client.29095 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc101a99b80 10 client.29095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc101a99b80 20 client.29095 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc101a99b80 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c141abf0f0 0x55c141ac1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc101a99b80 4 client.29095 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc101a99b80 1 -- 172.21.15.62:0/3518166780 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1419ebe50 con 0x55c141abf0f0 2023-07-14T16:30:22.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.819+0000 7f217f7fe700 1 -- 172.21.15.62:0/1940418170 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5612cb53e570 con 0x5612cb694cf0 2023-07-14T16:30:22.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.819+0000 7f217f7fe700 1 -- 172.21.15.62:0/1940418170 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21980143e0 con 0x5612cb694cf0 2023-07-14T16:30:22.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.819+0000 7f21ba2f4b80 10 client.29237 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.819+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c141abf0f0 0x55c141ac1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.819+0000 7f21ba2f4b80 10 client.29237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:22.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.819+0000 7f21ba2f4b80 20 client.29237 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:22.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.819+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5612cb6a8f20 0x5612cb6ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.819+0000 7f21ba2f4b80 4 client.29237 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:22.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.819+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1940418170 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612cb5ddf30 con 0x5612cb6a8f20 2023-07-14T16:30:22.819 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.820+0000 7f219ffff700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5612cb6a8f20 0x5612cb6ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.820+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c141abf0f0 0x55c141ac1500 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f36d3bf14499cc server_cookie=7e22ca9fc3c03be1 in_seq=0 out_seq=0 2023-07-14T16:30:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.820+0000 7fc0c6ffd700 10 client.29095 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:22.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.820+0000 7f219ffff700 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5612cb6a8f20 0x5612cb6ab330 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a57a188ccfddb50a server_cookie=e34cd49f05068e40 in_seq=0 out_seq=0 2023-07-14T16:30:22.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.820+0000 7f217f7fe700 10 client.29237 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.821+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/3518166780 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c1419ebe50 con 0x55c141abf0f0 2023-07-14T16:30:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.821+0000 7fc0c6ffd700 10 client.29095 handle_command_reply: tid=0 2023-07-14T16:30:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.821+0000 7fc101a99b80 1 client.29095 shutdown 2023-07-14T16:30:22.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.821+0000 7f217f7fe700 1 -- 172.21.15.62:0/1940418170 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5612cb5ddf30 con 0x5612cb6a8f20 2023-07-14T16:30:22.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.821+0000 7f217f7fe700 10 client.29237 handle_command_reply: tid=0 2023-07-14T16:30:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.821+0000 7fc101a99b80 1 -- 172.21.15.62:0/3518166780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 msgr2=0x55c141ab32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.821+0000 7fc101a99b80 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141ab32f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8005490 tx=0x7fc0d8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.821+0000 7f21ba2f4b80 1 client.29237 shutdown 2023-07-14T16:30:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 1 -- 172.21.15.62:0/3518166780 shutdown_connections 2023-07-14T16:30:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c141abf0f0 0x55c141ac1500 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c141aa27e0 0x55c141aba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c1418fd6b0 0x55c141ab32f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 1 --2- 172.21.15.62:0/3518166780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 1 -- 172.21.15.62:0/3518166780 >> 172.21.15.62:0/3518166780 conn(0x55c141a96d40 msgr2=0x55c141a9b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc0c6ffd700 0 client.29095 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 1 -- 172.21.15.62:0/3518166780 shutdown_connections 2023-07-14T16:30:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 msgr2=0x5612cb6a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb6a4720 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2198007d10 tx=0x7f2198004ad0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 1 -- 172.21.15.62:0/3518166780 wait complete. 2023-07-14T16:30:22.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1940418170 shutdown_connections 2023-07-14T16:30:22.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5612cb6a8f20 0x5612cb6ab330 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb694cf0 0x5612cb6a4720 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb69d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/1940418170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb69d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f217f7fe700 0 client.29237 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:22.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1940418170 >> 172.21.15.62:0/1940418170 conn(0x5612cb688e20 msgr2=0x5612cb68d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.822+0000 7fc101a99b80 20 client.29095 trim_cache size 0 max 16384 2023-07-14T16:30:22.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1940418170 shutdown_connections 2023-07-14T16:30:22.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/1940418170 wait complete. 2023-07-14T16:30:22.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.822+0000 7f21ba2f4b80 20 client.29237 trim_cache size 0 max 16384 2023-07-14T16:30:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.843+0000 7fc101a99b80 1 Processor -- start 2023-07-14T16:30:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.843+0000 7fc101a99b80 1 -- start start 2023-07-14T16:30:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.843+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141aa8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.843+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141aba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.843+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 0x55c141aa9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.843+0000 7fc101a99b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c141aa95c0 con 0x55c141aa1a90 2023-07-14T16:30:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.843+0000 7fc101a99b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c141aa9f90 con 0x55c1418fd6b0 2023-07-14T16:30:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.843+0000 7fc101a99b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c141a9d6d0 con 0x55c141aa27e0 2023-07-14T16:30:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141aba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141aba830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50626/0 (socket says 172.21.15.62:50626) 2023-07-14T16:30:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/2017053010 learned_addr learned my addr 172.21.15.62:0/2017053010 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/2017053010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141aa8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/2017053010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 msgr2=0x55c141aa9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/2017053010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 0x55c141aa9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/2017053010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 msgr2=0x55c141aa8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/2017053010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141aa8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.844+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/2017053010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1418471f0 con 0x55c141aa1a90 2023-07-14T16:30:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.845+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/2017053010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141aba830 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc01c040 tx=0x7fc0dc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d95a2127f4f664 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.845+0000 7fc0ce7fc700 1 -- 172.21.15.62:0/2017053010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0272c0 con 0x55c141aa1a90 2023-07-14T16:30:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.845+0000 7fc0ce7fc700 1 -- 172.21.15.62:0/2017053010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0dc003d80 con 0x55c141aa1a90 2023-07-14T16:30:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.845+0000 7fc0ce7fc700 1 -- 172.21.15.62:0/2017053010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc027df0 con 0x55c141aa1a90 2023-07-14T16:30:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 -- 172.21.15.62:0/2017053010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 msgr2=0x55c141aba830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2017053010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141aba830 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc01c040 tx=0x7fc0dc00b860 comp rx=0 tx=0).stop 2023-07-14T16:30:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 -- 172.21.15.62:0/2017053010 shutdown_connections 2023-07-14T16:30:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2017053010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 0x55c141aa9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2017053010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141aa8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2017053010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141aba830 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 -- 172.21.15.62:0/2017053010 >> 172.21.15.62:0/2017053010 conn(0x55c141a96d40 msgr2=0x55c141aa5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 -- 172.21.15.62:0/2017053010 shutdown_connections 2023-07-14T16:30:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 -- 172.21.15.62:0/2017053010 wait complete. 2023-07-14T16:30:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.846+0000 7fc101a99b80 1 -- start start 2023-07-14T16:30:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.847+0000 7fc101a99b80 1 Processor -- start 2023-07-14T16:30:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.847+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141ab6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.847+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.847+0000 7fc101a99b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 0x55c141ab9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.847+0000 7fc101a99b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c141aba2f0 con 0x55c141aa1a90 2023-07-14T16:30:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.847+0000 7fc101a99b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c141aba460 con 0x55c1418fd6b0 2023-07-14T16:30:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.847+0000 7fc101a99b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c141aba620 con 0x55c141aa27e0 2023-07-14T16:30:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.847+0000 7fc101a99b80 10 client.0 _ll_register_callbacks cb 0x7fc0d8005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141ab6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50636/0 (socket says 172.21.15.62:50636) 2023-07-14T16:30:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e6ffd700 1 -- 172.21.15.62:0/2991142397 learned_addr learned my addr 172.21.15.62:0/2991142397 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.848+0000 7f21ba2f4b80 1 Processor -- start 2023-07-14T16:30:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e7fff700 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 0x55c141ab9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.848+0000 7f21ba2f4b80 1 -- start start 2023-07-14T16:30:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e77fe700 1 -- 172.21.15.62:0/2991142397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 msgr2=0x55c141ab9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 0x55c141ab9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e77fe700 1 -- 172.21.15.62:0/2991142397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 msgr2=0x55c141ab6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e77fe700 1 -- 172.21.15.62:0/2991142397 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c141820d80 con 0x55c1418fd6b0 2023-07-14T16:30:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:22.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.848+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693550 0x5612cb697c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.848+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb6a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.848+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb694cf0 0x5612cb698190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.848+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb6a6380 con 0x5612cb694cf0 2023-07-14T16:30:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.848+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb69cd10 con 0x5612cb693550 2023-07-14T16:30:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.848+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb694270 con 0x5612cb693ea0 2023-07-14T16:30:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0e77fe700 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141ab6a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80258b0 tx=0x7fc0d800b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368476cf8b1741b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.848+0000 7fc0c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:30:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f21a4f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb694cf0 0x5612cb698190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f219ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693550 0x5612cb697c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f21a4f9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb694cf0 0x5612cb698190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50650/0 (socket says 172.21.15.62:50650) 2023-07-14T16:30:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f21a4f9c700 1 -- 172.21.15.62:0/2580989581 learned_addr learned my addr 172.21.15.62:0/2580989581 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.849+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/2991142397 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d802c040 con 0x55c1418fd6b0 2023-07-14T16:30:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f219f7fe700 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb6a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.849+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/2991142397 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0d8003820 con 0x55c1418fd6b0 2023-07-14T16:30:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.849+0000 7fc101a99b80 1 -- 172.21.15.62:0/2991142397 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0dc003680 con 0x55c1418fd6b0 2023-07-14T16:30:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f21a4f9c700 1 -- 172.21.15.62:0/2580989581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 msgr2=0x5612cb6a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb6a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f21a4f9c700 1 -- 172.21.15.62:0/2580989581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693550 msgr2=0x5612cb697c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693550 0x5612cb697c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f21a4f9c700 1 -- 172.21.15.62:0/2580989581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612cb4392d0 con 0x5612cb694cf0 2023-07-14T16:30:22.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.849+0000 7f219ffff700 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693550 0x5612cb697c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.849+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/2991142397 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0d801d040 con 0x55c1418fd6b0 2023-07-14T16:30:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.849+0000 7fc101a99b80 10 client.29243 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc101a99b80 1 -- 172.21.15.62:0/2991142397 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c14181f250 con 0x55c1418fd6b0 2023-07-14T16:30:22.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.850+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb694cf0 0x5612cb698190 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2198014890 tx=0x7f21980148c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4641be0e7343aa5b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/2991142397 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0dc003680 con 0x55c1418fd6b0 2023-07-14T16:30:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/2991142397 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0d800f4f0 con 0x55c1418fd6b0 2023-07-14T16:30:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc101a99b80 10 client.29243 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc101a99b80 10 client.29243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc101a99b80 20 client.29243 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c141b13cc0 0x55c141b160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc101a99b80 4 client.29243 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:30:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.850+0000 7fc101a99b80 1 -- 172.21.15.62:0/2991142397 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c1419adff0 con 0x55c141b13cc0 2023-07-14T16:30:22.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.850+0000 7f2186ffd700 1 -- 172.21.15.62:0/2580989581 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800e070 con 0x5612cb694cf0 2023-07-14T16:30:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.850+0000 7f2186ffd700 1 -- 172.21.15.62:0/2580989581 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f219800f040 con 0x5612cb694cf0 2023-07-14T16:30:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.850+0000 7f2186ffd700 1 -- 172.21.15.62:0/2580989581 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198014e80 con 0x5612cb694cf0 2023-07-14T16:30:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.851+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c141b13cc0 0x55c141b160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2580989581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb694cf0 msgr2=0x5612cb698190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb694cf0 0x5612cb698190 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2198014890 tx=0x7f21980148c0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.851+0000 7fc0e6ffd700 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c141b13cc0 0x55c141b160a0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67df189430128992 server_cookie=a2d309ce0f07f3c1 in_seq=0 out_seq=0 2023-07-14T16:30:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2580989581 shutdown_connections 2023-07-14T16:30:22.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb694cf0 0x5612cb698190 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693550 0x5612cb697c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2580989581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb693ea0 0x5612cb6a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2580989581 >> 172.21.15.62:0/2580989581 conn(0x5612cb688e20 msgr2=0x5612cb69be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.851+0000 7fc0c6ffd700 10 client.29243 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2580989581 shutdown_connections 2023-07-14T16:30:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.851+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2580989581 wait complete. 2023-07-14T16:30:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.852+0000 7f21ba2f4b80 1 -- start start 2023-07-14T16:30:22.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.852+0000 7f21ba2f4b80 1 Processor -- start 2023-07-14T16:30:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb6a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693ea0 0x5612cb6a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f21ba2f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb694cf0 0x5612cb6a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb6a4ae0 con 0x5612cb693550 2023-07-14T16:30:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb6a1810 con 0x5612cb693ea0 2023-07-14T16:30:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f21ba2f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb6a19d0 con 0x5612cb694cf0 2023-07-14T16:30:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f21ba2f4b80 10 client.0 _ll_register_callbacks cb 0x7f2198003da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:22.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb6a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693ea0 0x5612cb6a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693ea0 0x5612cb6a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57266/0 (socket says 172.21.15.62:57266) 2023-07-14T16:30:22.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219f7fe700 1 -- 172.21.15.62:0/2909956962 learned_addr learned my addr 172.21.15.62:0/2909956962 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:22.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f21a4f9c700 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb694cf0 0x5612cb6a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219ffff700 1 -- 172.21.15.62:0/2909956962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb694cf0 msgr2=0x5612cb6a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219ffff700 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb694cf0 0x5612cb6a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219ffff700 1 -- 172.21.15.62:0/2909956962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693ea0 msgr2=0x5612cb6a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219ffff700 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693ea0 0x5612cb6a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.853+0000 7f219ffff700 1 -- 172.21.15.62:0/2909956962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a01a0000 con 0x5612cb693550 2023-07-14T16:30:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f219f7fe700 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693ea0 0x5612cb6a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f219ffff700 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb6a42c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f21900ca630 tx=0x7f219001d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7f870ddcf99602 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f217f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f217f7fe700 1 -- 172.21.15.62:0/2909956962 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190099ef0 con 0x5612cb693550 2023-07-14T16:30:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f217f7fe700 1 -- 172.21.15.62:0/2909956962 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f219009df20 con 0x5612cb693550 2023-07-14T16:30:22.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2909956962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f219401e040 con 0x5612cb693550 2023-07-14T16:30:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f217f7fe700 1 -- 172.21.15.62:0/2909956962 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21900a2340 con 0x5612cb693550 2023-07-14T16:30:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f21ba2f4b80 10 client.19695 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.854+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2909956962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5612cb3fbce0 con 0x5612cb693550 2023-07-14T16:30:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f217f7fe700 1 -- 172.21.15.62:0/2909956962 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f219401e040 con 0x5612cb693550 2023-07-14T16:30:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f217f7fe700 1 -- 172.21.15.62:0/2909956962 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f219009a150 con 0x5612cb693550 2023-07-14T16:30:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f21ba2f4b80 10 client.19695 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f21ba2f4b80 10 client.19695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:22.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f21ba2f4b80 20 client.19695 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:22.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5612cb6fe730 0x5612cb700b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:22.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f21ba2f4b80 4 client.19695 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:30:22.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2909956962 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5612cb5a00d0 con 0x5612cb6fe730 2023-07-14T16:30:22.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.855+0000 7f219f7fe700 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5612cb6fe730 0x5612cb700b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:22.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.856+0000 7f219f7fe700 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5612cb6fe730 0x5612cb700b40 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4719d8b847601cb7 server_cookie=28560fb4b2ea9459 in_seq=0 out_seq=0 2023-07-14T16:30:22.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.856+0000 7f217f7fe700 10 client.19695 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc0c6ffd700 1 -- 172.21.15.62:0/2991142397 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c1419adff0 con 0x55c141b13cc0 2023-07-14T16:30:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc0c6ffd700 10 client.29243 handle_command_reply: tid=0 2023-07-14T16:30:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 client.29243 shutdown 2023-07-14T16:30:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 -- 172.21.15.62:0/2991142397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 msgr2=0x55c141ab6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141ab6a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80258b0 tx=0x7fc0d800b9c0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 -- 172.21.15.62:0/2991142397 shutdown_connections 2023-07-14T16:30:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c141aa27e0 0x55c141ab9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c141b13cc0 0x55c141b160a0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c1418fd6b0 0x55c141ab6a60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 --2- 172.21.15.62:0/2991142397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c141aa1a90 0x55c141ab6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc0c6ffd700 0 client.29243 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.862+0000 7fc101a99b80 1 -- 172.21.15.62:0/2991142397 >> 172.21.15.62:0/2991142397 conn(0x55c141a96d40 msgr2=0x55c141aa5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.863+0000 7fc101a99b80 1 -- 172.21.15.62:0/2991142397 shutdown_connections 2023-07-14T16:30:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.863+0000 7fc101a99b80 1 -- 172.21.15.62:0/2991142397 wait complete. 2023-07-14T16:30:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.863+0000 7fc101a99b80 20 client.29243 trim_cache size 0 max 16384 2023-07-14T16:30:22.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f217f7fe700 1 -- 172.21.15.62:0/2909956962 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5612cb5a00d0 con 0x5612cb6fe730 2023-07-14T16:30:22.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f217f7fe700 10 client.19695 handle_command_reply: tid=0 2023-07-14T16:30:22.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 client.19695 shutdown 2023-07-14T16:30:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:30:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:30:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:30:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:30:22.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2909956962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 msgr2=0x5612cb6a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb6a42c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f21900ca630 tx=0x7f219001d450 comp rx=0 tx=0).stop 2023-07-14T16:30:22.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2909956962 shutdown_connections 2023-07-14T16:30:22.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612cb694cf0 0x5612cb6a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5612cb693550 0x5612cb6a42c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5612cb6fe730 0x5612cb700b40 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 --2- 172.21.15.62:0/2909956962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612cb693ea0 0x5612cb6a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2909956962 >> 172.21.15.62:0/2909956962 conn(0x5612cb688e20 msgr2=0x5612cb697c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f217f7fe700 0 client.19695 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.863+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc0d00a12a0 msgr2=0x7fc0d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.863+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc0d00a12a0 0x7fc0d00a3750 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55c141a9ccb0 tx=0x7fc0d800b540 comp rx=0 tx=0).stop 2023-07-14T16:30:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.863+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1547728389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8107730 msgr2=0x7fc0e819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.863+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8107730 0x7fc0e819edc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0007f10 tx=0x7fc0e000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.863+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2909956962 shutdown_connections 2023-07-14T16:30:22.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.864+0000 7f21ba2f4b80 1 -- 172.21.15.62:0/2909956962 wait complete. 2023-07-14T16:30:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.864+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1547728389 shutdown_connections 2023-07-14T16:30:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.864+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0e8107730 0x7fc0e819edc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.864+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc0d00a12a0 0x7fc0d00a3750 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.864+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0e8068640 0x7fc0e8197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.864+0000 7fc0ecf9d700 1 --2- 172.21.15.62:0/1547728389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e8104e10 0x7fc0e8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.864+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1547728389 >> 172.21.15.62:0/1547728389 conn(0x7fc0e80f9e40 msgr2=0x7fc0e8106250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.864+0000 7f21ba2f4b80 20 client.19695 trim_cache size 0 max 16384 2023-07-14T16:30:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.864+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1547728389 shutdown_connections 2023-07-14T16:30:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:22.864+0000 7fc0ecf9d700 1 -- 172.21.15.62:0/1547728389 wait complete. 2023-07-14T16:30:22.864 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:30:22.864 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:30:22.864 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "1f048c89-a126-4f78-aa23-72da4fb6d56c", 2023-07-14T16:30:22.864 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:30:22.864 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:30:22.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 -- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21880a11d0 msgr2=0x7f21880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21880a11d0 0x7f21880a3680 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f21980075c0 tx=0x7f2198009040 comp rx=0 tx=0).stop 2023-07-14T16:30:22.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 -- 172.21.15.62:0/439414813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0101d10 msgr2=0x7f21a01979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:22.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0101d10 0x7f21a01979c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f219000e970 tx=0x7f219000ec80 comp rx=0 tx=0).stop 2023-07-14T16:30:22.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 -- 172.21.15.62:0/439414813 shutdown_connections 2023-07-14T16:30:22.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f21880a11d0 0x7f21880a3680 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21a0103890 0x7f21a019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21a0101d10 0x7f21a01979c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 --2- 172.21.15.62:0/439414813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21a0102f00 0x7f21a0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:22.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 -- 172.21.15.62:0/439414813 >> 172.21.15.62:0/439414813 conn(0x7f21a00fd350 msgr2=0x7f21a00ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:22.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 -- 172.21.15.62:0/439414813 shutdown_connections 2023-07-14T16:30:22.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:22.865+0000 7f21a69ff700 1 -- 172.21.15.62:0/439414813 wait complete. 2023-07-14T16:30:22.867 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:22 smithi062 ceph-mon[95543]: pgmap v799: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:22.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:30:23.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 -- 172.21.15.62:0/2191704428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 msgr2=0x7f7e300ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2191704428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 0x7f7e300ffef0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009990 tx=0x7f7e20009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 -- 172.21.15.62:0/2191704428 shutdown_connections 2023-07-14T16:30:23.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2191704428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e30101660 0x7f7e30107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2191704428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 0x7f7e300ffef0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2191704428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e30100cd0 0x7f7e30101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 -- 172.21.15.62:0/2191704428 >> 172.21.15.62:0/2191704428 conn(0x7f7e300fb0e0 msgr2=0x7f7e300fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:23.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 -- 172.21.15.62:0/2191704428 shutdown_connections 2023-07-14T16:30:23.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.138+0000 7f7e349ff700 1 -- 172.21.15.62:0/2191704428 wait complete. 2023-07-14T16:30:23.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.139+0000 7f7e349ff700 1 Processor -- start 2023-07-14T16:30:23.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e349ff700 1 -- start start 2023-07-14T16:30:23.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e349ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 0x7f7e30197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e349ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e30100cd0 0x7f7e30197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e349ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e30101660 0x7f7e3019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e349ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3019f4c0 con 0x7f7e300ffae0 2023-07-14T16:30:23.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e349ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3019f600 con 0x7f7e30100cd0 2023-07-14T16:30:23.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e349ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3019f790 con 0x7f7e30101660 2023-07-14T16:30:23.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e2e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 0x7f7e30197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e2e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 0x7f7e30197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50678/0 (socket says 172.21.15.62:50678) 2023-07-14T16:30:23.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.140+0000 7f7e2e59c700 1 -- 172.21.15.62:0/2498731356 learned_addr learned my addr 172.21.15.62:0/2498731356 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:23.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e30100cd0 0x7f7e30197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e30101660 0x7f7e3019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e2e59c700 1 -- 172.21.15.62:0/2498731356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e30101660 msgr2=0x7f7e3019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e30101660 0x7f7e3019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e2e59c700 1 -- 172.21.15.62:0/2498731356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e30100cd0 msgr2=0x7f7e30197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e30100cd0 0x7f7e30197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e2e59c700 1 -- 172.21.15.62:0/2498731356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e200095f0 con 0x7f7e300ffae0 2023-07-14T16:30:23.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 0x7f7e30197940 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7e20005f50 tx=0x7f7e20004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:23.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e1f7fe700 1 -- 172.21.15.62:0/2498731356 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2001d070 con 0x7f7e300ffae0 2023-07-14T16:30:23.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e1f7fe700 1 -- 172.21.15.62:0/2498731356 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e200041e0 con 0x7f7e300ffae0 2023-07-14T16:30:23.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e349ff700 1 -- 172.21.15.62:0/2498731356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3019fa10 con 0x7f7e300ffae0 2023-07-14T16:30:23.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.141+0000 7f7e1f7fe700 1 -- 172.21.15.62:0/2498731356 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000f460 con 0x7f7e300ffae0 2023-07-14T16:30:23.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.142+0000 7f7e349ff700 1 -- 172.21.15.62:0/2498731356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3019fe30 con 0x7f7e300ffae0 2023-07-14T16:30:23.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.145+0000 7f7e1f7fe700 1 -- 172.21.15.62:0/2498731356 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7e20022470 con 0x7f7e300ffae0 2023-07-14T16:30:23.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.146+0000 7f7e1f7fe700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7e180a1150 0x7f7e180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.146+0000 7f7e482a8b80 1 Processor -- start 2023-07-14T16:30:23.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.146+0000 7f7e1f7fe700 1 -- 172.21.15.62:0/2498731356 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e200cddf0 con 0x7f7e300ffae0 2023-07-14T16:30:23.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.147+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7e180a1150 0x7f7e180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.147+0000 7f7e482a8b80 1 -- start start 2023-07-14T16:30:23.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.147+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 0x55ca0ff948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.147+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 0x55ca0ff956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.147+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ff960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.147+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0fe99cb0 con 0x55ca0ff95c70 2023-07-14T16:30:23.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.147+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0feeca40 con 0x55ca0ff94e20 2023-07-14T16:30:23.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.147+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0fee5ed0 con 0x55ca0ff944d0 2023-07-14T16:30:23.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ff960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ff960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50680/0 (socket says 172.21.15.62:50680) 2023-07-14T16:30:23.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2ed9d700 1 -- 172.21.15.62:0/794791710 learned_addr learned my addr 172.21.15.62:0/794791710 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:23.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 0x55ca0ff956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 0x55ca0ff948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7e180a1150 0x7f7e180a3600 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009bf0 tx=0x7f7e24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:23.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2ed9d700 1 -- 172.21.15.62:0/794791710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 msgr2=0x55ca0ff948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:23.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 0x55ca0ff948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2ed9d700 1 -- 172.21.15.62:0/794791710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 msgr2=0x55ca0ff956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:23.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 0x55ca0ff956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.148+0000 7f7e2ed9d700 1 -- 172.21.15.62:0/794791710 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0fd13de0 con 0x55ca0ff95c70 2023-07-14T16:30:23.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.149+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ff960e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009dc0 tx=0x7f7e280077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4ad6e08599ba43 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:23.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.149+0000 7f7e1cff9700 1 -- 172.21.15.62:0/794791710 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28014610 con 0x55ca0ff95c70 2023-07-14T16:30:23.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.149+0000 7f7e1cff9700 1 -- 172.21.15.62:0/794791710 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e2800f050 con 0x55ca0ff95c70 2023-07-14T16:30:23.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.149+0000 7f7e1cff9700 1 -- 172.21.15.62:0/794791710 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280137f0 con 0x55ca0ff95c70 2023-07-14T16:30:23.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 -- 172.21.15.62:0/794791710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 msgr2=0x55ca0ff960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ff960e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009dc0 tx=0x7f7e280077e0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 -- 172.21.15.62:0/794791710 shutdown_connections 2023-07-14T16:30:23.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ff960e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 0x55ca0ff948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/794791710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 0x55ca0ff956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 -- 172.21.15.62:0/794791710 >> 172.21.15.62:0/794791710 conn(0x55ca0ff89da0 msgr2=0x55ca0ff8f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:23.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 -- 172.21.15.62:0/794791710 shutdown_connections 2023-07-14T16:30:23.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.150+0000 7f7e482a8b80 1 -- 172.21.15.62:0/794791710 wait complete. 2023-07-14T16:30:23.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.151+0000 7f7e482a8b80 1 -- start start 2023-07-14T16:30:23.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.151+0000 7f7e482a8b80 1 Processor -- start 2023-07-14T16:30:23.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff94e20 0x55ca0ff9e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0fe85ac0 con 0x55ca0ff94e20 2023-07-14T16:30:23.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0fe857c0 con 0x55ca0ff944d0 2023-07-14T16:30:23.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0fd69920 con 0x55ca0ff95c70 2023-07-14T16:30:23.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e482a8b80 10 client.0 _ll_register_callbacks cb 0x55ca0fe83760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:23.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e2dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff94e20 0x55ca0ff9e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e2dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff94e20 0x55ca0ff9e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50686/0 (socket says 172.21.15.62:50686) 2023-07-14T16:30:23.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e2dd9b700 1 -- 172.21.15.62:0/3438724266 learned_addr learned my addr 172.21.15.62:0/3438724266 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:23.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.152+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e2dd9b700 1 -- 172.21.15.62:0/3438724266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff95c70 msgr2=0x55ca0ffa56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:23.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e2dd9b700 1 -- 172.21.15.62:0/3438724266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff944d0 msgr2=0x55ca0ff9e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:23.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e2dd9b700 1 -- 172.21.15.62:0/3438724266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e28009a20 con 0x55ca0ff94e20 2023-07-14T16:30:23.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff94e20 0x55ca0ff9e660 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f7e240128e0 tx=0x7f7e240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af06f53f278d9a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:23.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:23.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3438724266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24029430 con 0x55ca0ff94e20 2023-07-14T16:30:23.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3438724266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e24018070 con 0x55ca0ff94e20 2023-07-14T16:30:23.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.153+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3438724266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca0fe3f4f0 con 0x55ca0ff94e20 2023-07-14T16:30:23.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3438724266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e24029590 con 0x55ca0ff94e20 2023-07-14T16:30:23.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e482a8b80 10 client.19707 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:23.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3438724266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ca0fcfcc60 con 0x55ca0ff94e20 2023-07-14T16:30:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3438724266 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ca0fe3f4f0 con 0x55ca0ff94e20 2023-07-14T16:30:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3438724266 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e240208b0 con 0x55ca0ff94e20 2023-07-14T16:30:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e482a8b80 10 client.19707 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e482a8b80 10 client.19707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e482a8b80 20 client.19707 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ca0ffa9f20 0x55ca0ffac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e482a8b80 4 client.19707 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.154+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3438724266 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca0fedeeb0 con 0x55ca0ffa9f20 2023-07-14T16:30:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.155+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ca0ffa9f20 0x55ca0ffac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.156+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ca0ffa9f20 0x55ca0ffac330 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228527ce9ddcd3e6 server_cookie=fa257f797f165bb in_seq=0 out_seq=0 2023-07-14T16:30:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.156+0000 7f7e15ffb700 10 client.19707 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.156+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3438724266 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca0fedeeb0 con 0x55ca0ffa9f20 2023-07-14T16:30:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.156+0000 7f7e15ffb700 10 client.19707 handle_command_reply: tid=0 2023-07-14T16:30:23.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.156+0000 7f7e482a8b80 1 client.19707 shutdown 2023-07-14T16:30:23.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3438724266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff94e20 msgr2=0x55ca0ff9e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff94e20 0x55ca0ff9e660 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f7e240128e0 tx=0x7f7e240047c0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3438724266 shutdown_connections 2023-07-14T16:30:23.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ca0ffa9f20 0x55ca0ffac330 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3438724266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff94e20 0x55ca0ff9e660 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e15ffb700 0 client.19707 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:23.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3438724266 >> 172.21.15.62:0/3438724266 conn(0x55ca0ff89da0 msgr2=0x55ca0ff8e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:23.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3438724266 shutdown_connections 2023-07-14T16:30:23.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3438724266 wait complete. 2023-07-14T16:30:23.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.157+0000 7f7e482a8b80 20 client.19707 trim_cache size 0 max 16384 2023-07-14T16:30:23.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:22 smithi081 ceph-mon[127795]: pgmap v799: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:23.148 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:30:22 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:30:22.874+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v269>,<0x10000000003/client.0 v270>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v282>,<0x10000000001/sv_0 v296>,<0x10000000000/_nogroup v290>,<0x1/volumes v280>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:30:23.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.179+0000 7f7e482a8b80 1 Processor -- start 2023-07-14T16:30:23.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e482a8b80 1 -- start start 2023-07-14T16:30:23.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff94e20 0x55ca0ffa5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff95c70 0x55ca0ff8e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ff9f040 con 0x55ca0ff944d0 2023-07-14T16:30:23.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ff9bdd0 con 0x55ca0ff95c70 2023-07-14T16:30:23.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ffa9c20 con 0x55ca0ff94e20 2023-07-14T16:30:23.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e2e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e2e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50696/0 (socket says 172.21.15.62:50696) 2023-07-14T16:30:23.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e2e59c700 1 -- 172.21.15.62:0/3616347819 learned_addr learned my addr 172.21.15.62:0/3616347819 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:23.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff94e20 0x55ca0ffa5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.180+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff95c70 0x55ca0ff8e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e2e59c700 1 -- 172.21.15.62:0/3616347819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff94e20 msgr2=0x55ca0ffa5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:23.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff94e20 0x55ca0ffa5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e2e59c700 1 -- 172.21.15.62:0/3616347819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff95c70 msgr2=0x55ca0ff8e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:23.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff95c70 0x55ca0ff8e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e2e59c700 1 -- 172.21.15.62:0/3616347819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0fd3a250 con 0x55ca0ff944d0 2023-07-14T16:30:23.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9ce50 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f7e20003f60 tx=0x7f7e200cb770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459c70fb2d0bf18 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:23.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e1d7fa700 1 -- 172.21.15.62:0/3616347819 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2009f270 con 0x55ca0ff944d0 2023-07-14T16:30:23.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e1d7fa700 1 -- 172.21.15.62:0/3616347819 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e200a3600 con 0x55ca0ff944d0 2023-07-14T16:30:23.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.181+0000 7f7e1d7fa700 1 -- 172.21.15.62:0/3616347819 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2009a730 con 0x55ca0ff944d0 2023-07-14T16:30:23.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3616347819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff944d0 msgr2=0x55ca0ff9ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9ce50 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f7e20003f60 tx=0x7f7e200cb770 comp rx=0 tx=0).stop 2023-07-14T16:30:23.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3616347819 shutdown_connections 2023-07-14T16:30:23.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff95c70 0x55ca0ff8e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff944d0 0x55ca0ff9ce50 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3616347819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff94e20 0x55ca0ffa5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3616347819 >> 172.21.15.62:0/3616347819 conn(0x55ca0ff89da0 msgr2=0x55ca0ffb0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:23.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3616347819 shutdown_connections 2023-07-14T16:30:23.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.182+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3616347819 wait complete. 2023-07-14T16:30:23.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.183+0000 7f7e482a8b80 1 -- start start 2023-07-14T16:30:23.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.183+0000 7f7e482a8b80 1 Processor -- start 2023-07-14T16:30:23.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 0x55ca0ffa1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 0x55ca0ffa1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e482a8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ffa53e0 con 0x55ca0ff95c70 2023-07-14T16:30:23.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ffa5550 con 0x55ca0ff94e20 2023-07-14T16:30:23.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e482a8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0ffa5710 con 0x55ca0ff944d0 2023-07-14T16:30:23.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e482a8b80 10 client.0 _ll_register_callbacks cb 0x55ca0fe83760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:23.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50702/0 (socket says 172.21.15.62:50702) 2023-07-14T16:30:23.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2ed9d700 1 -- 172.21.15.62:0/3507146083 learned_addr learned my addr 172.21.15.62:0/3507146083 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:23.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 0x55ca0ffa1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2dd9b700 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 0x55ca0ffa1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2ed9d700 1 -- 172.21.15.62:0/3507146083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 msgr2=0x55ca0ffa1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:23.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 0x55ca0ffa1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2ed9d700 1 -- 172.21.15.62:0/3507146083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 msgr2=0x55ca0ffa1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:23.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 0x55ca0ffa1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.184+0000 7f7e2ed9d700 1 -- 172.21.15.62:0/3507146083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3019fe30 con 0x55ca0ff95c70 2023-07-14T16:30:23.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.185+0000 7f7e2ed9d700 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa4ea0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009550 tx=0x7f7e28014620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d58c77009b8a830 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:23.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.185+0000 7f7e15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:23.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.185+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3507146083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28022040 con 0x55ca0ff95c70 2023-07-14T16:30:23.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.185+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3507146083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e28014ba0 con 0x55ca0ff95c70 2023-07-14T16:30:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.185+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3507146083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e2800f050 con 0x55ca0ff95c70 2023-07-14T16:30:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.185+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3507146083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e280098c0 con 0x55ca0ff95c70 2023-07-14T16:30:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.185+0000 7f7e482a8b80 10 client.19713 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.185+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3507146083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ca0fd122b0 con 0x55ca0ff95c70 2023-07-14T16:30:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3507146083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e2800f050 con 0x55ca0ff95c70 2023-07-14T16:30:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3507146083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e2801d3f0 con 0x55ca0ff95c70 2023-07-14T16:30:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e482a8b80 10 client.19713 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e482a8b80 10 client.19713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e482a8b80 20 client.19713 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ca0ffff1e0 0x55ca100015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e482a8b80 4 client.19713 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:30:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3507146083 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ca0fea1050 con 0x55ca0ffff1e0 2023-07-14T16:30:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.186+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ca0ffff1e0 0x55ca100015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.187+0000 7f7e2e59c700 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ca0ffff1e0 0x55ca100015f0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a795bf7a1258f08 server_cookie=1cf6268254ded92 in_seq=0 out_seq=0 2023-07-14T16:30:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.187+0000 7f7e15ffb700 10 client.19713 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.187+0000 7f7e15ffb700 1 -- 172.21.15.62:0/3507146083 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ca0fea1050 con 0x55ca0ffff1e0 2023-07-14T16:30:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.187+0000 7f7e15ffb700 10 client.19713 handle_command_reply: tid=0 2023-07-14T16:30:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.187+0000 7f7e482a8b80 1 client.19713 shutdown 2023-07-14T16:30:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.187+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3507146083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 msgr2=0x55ca0ffa4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.187+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa4ea0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009550 tx=0x7f7e28014620 comp rx=0 tx=0).stop 2023-07-14T16:30:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3507146083 shutdown_connections 2023-07-14T16:30:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca0ff95c70 0x55ca0ffa4ea0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ca0ff944d0 0x55ca0ffa1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ca0ffff1e0 0x55ca100015f0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 1 --2- 172.21.15.62:0/3507146083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ca0ff94e20 0x55ca0ffa1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e15ffb700 0 client.19713 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:23.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3507146083 >> 172.21.15.62:0/3507146083 conn(0x55ca0ff89da0 msgr2=0x55ca0ff9ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:23.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3507146083 shutdown_connections 2023-07-14T16:30:23.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 1 -- 172.21.15.62:0/3507146083 wait complete. 2023-07-14T16:30:23.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e482a8b80 20 client.19713 trim_cache size 0 max 16384 2023-07-14T16:30:23.168 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:30:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:30:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:30:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "1f048c89-a126-4f78-aa23-72da4fb6d56c": { 2023-07-14T16:30:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:30:23.169 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "1f048c89-a126-4f78-aa23-72da4fb6d56c", 2023-07-14T16:30:23.169 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:30:23.169 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:30:23.169 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:30:23.169 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:30:23.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e349ff700 1 -- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7e180a1150 msgr2=0x7f7e180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7e180a1150 0x7f7e180a3600 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009bf0 tx=0x7f7e24009260 comp rx=0 tx=0).stop 2023-07-14T16:30:23.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e349ff700 1 -- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 msgr2=0x7f7e30197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:23.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.188+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 0x7f7e30197940 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7e20005f50 tx=0x7f7e20004000 comp rx=0 tx=0).stop 2023-07-14T16:30:23.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.189+0000 7f7e349ff700 1 -- 172.21.15.62:0/2498731356 shutdown_connections 2023-07-14T16:30:23.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.189+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e30101660 0x7f7e3019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.189+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e300ffae0 0x7f7e30197940 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.189+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7e180a1150 0x7f7e180a3600 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.189+0000 7f7e349ff700 1 --2- 172.21.15.62:0/2498731356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e30100cd0 0x7f7e30197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:23.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.189+0000 7f7e349ff700 1 -- 172.21.15.62:0/2498731356 >> 172.21.15.62:0/2498731356 conn(0x7f7e300fb0e0 msgr2=0x7f7e300fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:23.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.189+0000 7f7e349ff700 1 -- 172.21.15.62:0/2498731356 shutdown_connections 2023-07-14T16:30:23.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:23.189+0000 7f7e349ff700 1 -- 172.21.15.62:0/2498731356 wait complete. 2023-07-14T16:30:23.181 INFO:tasks.cephfs.filesystem:scrub status for tag:1f048c89-a126-4f78-aa23-72da4fb6d56c - {'path': '/', 'tag': '1f048c89-a126-4f78-aa23-72da4fb6d56c', 'options': 'recursive,force'} 2023-07-14T16:30:23.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:22 smithi164 ceph-mon[97427]: pgmap v799: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:24.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:30:24.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:30:24.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:30:24.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:30:24.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:30:24.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:30:24.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:23 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:30:24.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:30:24.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:30:24.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:30:24.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:30:24.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:30:24.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:23 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:30:24.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:30:24.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:30:24.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:30:24.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:30:24.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:30:24.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:24.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:23 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:25.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:24 smithi062 ceph-mon[95543]: pgmap v800: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 23 op/s 2023-07-14T16:30:25.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:24 smithi081 ceph-mon[127795]: pgmap v800: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 23 op/s 2023-07-14T16:30:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:24 smithi164 ceph-mon[97427]: pgmap v800: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 23 op/s 2023-07-14T16:30:27.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:26 smithi062 ceph-mon[95543]: pgmap v801: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 25 op/s 2023-07-14T16:30:27.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:26 smithi081 ceph-mon[127795]: pgmap v801: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 25 op/s 2023-07-14T16:30:27.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:26 smithi164 ceph-mon[97427]: pgmap v801: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 25 op/s 2023-07-14T16:30:29.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:28 smithi062 ceph-mon[95543]: pgmap v802: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:30:29.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:28 smithi081 ceph-mon[127795]: pgmap v802: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:30:29.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:28 smithi164 ceph-mon[97427]: pgmap v802: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:30:31.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:30 smithi062 ceph-mon[95543]: pgmap v803: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:31.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:30 smithi081 ceph-mon[127795]: pgmap v803: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:31.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:30 smithi164 ceph-mon[97427]: pgmap v803: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:33.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:32 smithi062 ceph-mon[95543]: pgmap v804: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:30:33.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:32 smithi081 ceph-mon[127795]: pgmap v804: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:30:33.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:32 smithi164 ceph-mon[97427]: pgmap v804: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 36 op/s 2023-07-14T16:30:34.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:33 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:30:34.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:33 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:30:34.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:33 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:30:34.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:33 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:30:34.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:33 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:30:34.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:33 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:30:34.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:33 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:30:34.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:33 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:30:34.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:33 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:30:35.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:34 smithi062 ceph-mon[95543]: pgmap v805: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:35.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:34 smithi081 ceph-mon[127795]: pgmap v805: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:35.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:34 smithi164 ceph-mon[97427]: pgmap v805: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 38 op/s 2023-07-14T16:30:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.142+0000 7fea649fe700 1 -- 172.21.15.62:0/1700482488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea600ff940 msgr2=0x7fea600ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.142+0000 7fea649fe700 1 --2- 172.21.15.62:0/1700482488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea600ff940 0x7fea600ffd50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fea50009990 tx=0x7fea50009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.143+0000 7fea649fe700 1 -- 172.21.15.62:0/1700482488 shutdown_connections 2023-07-14T16:30:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.143+0000 7fea649fe700 1 --2- 172.21.15.62:0/1700482488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea601014c0 0x7fea60107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.143+0000 7fea649fe700 1 --2- 172.21.15.62:0/1700482488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea600ff940 0x7fea600ffd50 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.143+0000 7fea649fe700 1 --2- 172.21.15.62:0/1700482488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100b30 0x7fea60100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.143+0000 7fea649fe700 1 -- 172.21.15.62:0/1700482488 >> 172.21.15.62:0/1700482488 conn(0x7fea600faf40 msgr2=0x7fea600fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.143+0000 7fea649fe700 1 -- 172.21.15.62:0/1700482488 shutdown_connections 2023-07-14T16:30:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.143+0000 7fea649fe700 1 -- 172.21.15.62:0/1700482488 wait complete. 2023-07-14T16:30:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.144+0000 7fea649fe700 1 Processor -- start 2023-07-14T16:30:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.144+0000 7fea649fe700 1 -- start start 2023-07-14T16:30:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea649fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea600ff940 0x7fea601955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea649fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100b30 0x7fea60195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea649fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea601014c0 0x7fea6019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea649fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6019d120 con 0x7fea601014c0 2023-07-14T16:30:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea649fe700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6019d260 con 0x7fea600ff940 2023-07-14T16:30:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea649fe700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6019d3f0 con 0x7fea60100b30 2023-07-14T16:30:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea5f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea601014c0 0x7fea6019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea5ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea600ff940 0x7fea601955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea5ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea600ff940 0x7fea601955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34322/0 (socket says 172.21.15.62:34322) 2023-07-14T16:30:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea5ed9d700 1 -- 172.21.15.62:0/1403975880 learned_addr learned my addr 172.21.15.62:0/1403975880 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.145+0000 7fea5e59c700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100b30 0x7fea60195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.146+0000 7fea5ed9d700 1 -- 172.21.15.62:0/1403975880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100b30 msgr2=0x7fea60195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.146+0000 7fea5ed9d700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100b30 0x7fea60195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.146+0000 7fea5ed9d700 1 -- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea601014c0 msgr2=0x7fea6019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.146+0000 7fea5ed9d700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea601014c0 0x7fea6019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.146+0000 7fea5ed9d700 1 -- 172.21.15.62:0/1403975880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500095f0 con 0x7fea600ff940 2023-07-14T16:30:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.146+0000 7fea5f59e700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea601014c0 0x7fea6019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.146+0000 7fea5ed9d700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea600ff940 0x7fea601955a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fea5000b5c0 tx=0x7fea50005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.146+0000 7fea47fff700 1 -- 172.21.15.62:0/1403975880 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5001d070 con 0x7fea600ff940 2023-07-14T16:30:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.147+0000 7fea649fe700 1 -- 172.21.15.62:0/1403975880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6019d670 con 0x7fea600ff940 2023-07-14T16:30:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.147+0000 7fea47fff700 1 -- 172.21.15.62:0/1403975880 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea50003f80 con 0x7fea600ff940 2023-07-14T16:30:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.147+0000 7fea47fff700 1 -- 172.21.15.62:0/1403975880 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50022aa0 con 0x7fea600ff940 2023-07-14T16:30:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.147+0000 7fea649fe700 1 -- 172.21.15.62:0/1403975880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6019daf0 con 0x7fea600ff940 2023-07-14T16:30:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.151+0000 7fea47fff700 1 -- 172.21.15.62:0/1403975880 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fea50003af0 con 0x7fea600ff940 2023-07-14T16:30:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.151+0000 7fea78bffb80 1 Processor -- start 2023-07-14T16:30:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.151+0000 7fea78bffb80 1 -- start start 2023-07-14T16:30:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327614550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327615c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea78bffb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557327519c50 con 0x5573276157e0 2023-07-14T16:30:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea78bffb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732756c9e0 con 0x557327614a90 2023-07-14T16:30:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea78bffb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557327565e70 con 0x5573274706b0 2023-07-14T16:30:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea5e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327615c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea5f59e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327614550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea5e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327615c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45280/0 (socket says 172.21.15.62:45280) 2023-07-14T16:30:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea5f59e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327614550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:48058/0 (socket says 172.21.15.62:48058) 2023-07-14T16:30:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea5e59c700 1 -- 172.21.15.62:0/3244285565 learned_addr learned my addr 172.21.15.62:0/3244285565 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.152+0000 7fea5ed9d700 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5f59e700 1 -- 172.21.15.62:0/3244285565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 msgr2=0x5573276152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5f59e700 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5f59e700 1 -- 172.21.15.62:0/3244285565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 msgr2=0x557327615c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5f59e700 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327615c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5f59e700 1 -- 172.21.15.62:0/3244285565 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557327393d80 con 0x5573274706b0 2023-07-14T16:30:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5e59c700 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327615c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5ed9d700 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea47fff700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fea480a1150 0x7fea480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5f59e700 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327614550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fea580076b0 tx=0x7fea580079c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7389f4ed7e3813e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea47fff700 1 -- 172.21.15.62:0/1403975880 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fea500cdfa0 con 0x7fea600ff940 2023-07-14T16:30:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea5ed9d700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fea480a1150 0x7fea480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.153+0000 7fea457fa700 1 -- 172.21.15.62:0/3244285565 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58004bb0 con 0x5573274706b0 2023-07-14T16:30:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea457fa700 1 -- 172.21.15.62:0/3244285565 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea5800f040 con 0x5573274706b0 2023-07-14T16:30:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea457fa700 1 -- 172.21.15.62:0/3244285565 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58005390 con 0x5573274706b0 2023-07-14T16:30:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea78bffb80 1 -- 172.21.15.62:0/3244285565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 msgr2=0x557327614550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea5ed9d700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fea480a1150 0x7fea480a3600 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea500cbe20 tx=0x7fea50009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327614550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fea580076b0 tx=0x7fea580079c0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea78bffb80 1 -- 172.21.15.62:0/3244285565 shutdown_connections 2023-07-14T16:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327615c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327614550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3244285565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.154+0000 7fea78bffb80 1 -- 172.21.15.62:0/3244285565 >> 172.21.15.62:0/3244285565 conn(0x557327609d40 msgr2=0x55732760ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.155+0000 7fea78bffb80 1 -- 172.21.15.62:0/3244285565 shutdown_connections 2023-07-14T16:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.155+0000 7fea78bffb80 1 -- 172.21.15.62:0/3244285565 wait complete. 2023-07-14T16:30:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.155+0000 7fea78bffb80 1 -- start start 2023-07-14T16:30:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.156+0000 7fea78bffb80 1 Processor -- start 2023-07-14T16:30:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.156+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573274706b0 0x55732761da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.156+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327614a90 0x55732761dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.156+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327625010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.156+0000 7fea78bffb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557327505a60 con 0x5573276157e0 2023-07-14T16:30:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.156+0000 7fea78bffb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557327505760 con 0x5573274706b0 2023-07-14T16:30:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.156+0000 7fea78bffb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573273e98c0 con 0x557327614a90 2023-07-14T16:30:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.156+0000 7fea78bffb80 10 client.0 _ll_register_callbacks cb 0x557327503700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327625010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327625010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45282/0 (socket says 172.21.15.62:45282) 2023-07-14T16:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 -- 172.21.15.62:0/3250337165 learned_addr learned my addr 172.21.15.62:0/3250337165 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573274706b0 0x55732761da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5e59c700 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327614a90 0x55732761dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 -- 172.21.15.62:0/3250337165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327614a90 msgr2=0x55732761dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327614a90 0x55732761dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 -- 172.21.15.62:0/3250337165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573274706b0 msgr2=0x55732761da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573274706b0 0x55732761da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.157+0000 7fea5f59e700 1 -- 172.21.15.62:0/3250337165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5400b9a0 con 0x5573276157e0 2023-07-14T16:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea5f59e700 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327625010 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fea58015940 tx=0x7fea5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5f5df01ea3fd53 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea42ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea42ff5700 1 -- 172.21.15.62:0/3250337165 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58005510 con 0x5573276157e0 2023-07-14T16:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea78bffb80 1 -- 172.21.15.62:0/3250337165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573274bf490 con 0x5573276157e0 2023-07-14T16:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea42ff5700 1 -- 172.21.15.62:0/3250337165 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fea58009440 con 0x5573276157e0 2023-07-14T16:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea42ff5700 1 -- 172.21.15.62:0/3250337165 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea58005670 con 0x5573276157e0 2023-07-14T16:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea78bffb80 10 client.19725 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea78bffb80 1 -- 172.21.15.62:0/3250337165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fea58009d80 con 0x5573276157e0 2023-07-14T16:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.158+0000 7fea42ff5700 1 -- 172.21.15.62:0/3250337165 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5573274bf490 con 0x5573276157e0 2023-07-14T16:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.159+0000 7fea42ff5700 1 -- 172.21.15.62:0/3250337165 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea5801dad0 con 0x5573276157e0 2023-07-14T16:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.159+0000 7fea78bffb80 10 client.19725 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.159+0000 7fea78bffb80 10 client.19725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.159+0000 7fea78bffb80 20 client.19725 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.159+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557327629840 0x55732762bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.159+0000 7fea78bffb80 4 client.19725 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.159+0000 7fea78bffb80 1 -- 172.21.15.62:0/3250337165 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55732755ee50 con 0x557327629840 2023-07-14T16:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.159+0000 7fea5e59c700 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557327629840 0x55732762bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.160+0000 7fea5e59c700 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557327629840 0x55732762bc50 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acba480340c0e41 server_cookie=3a00cd3f2d476a7 in_seq=0 out_seq=0 2023-07-14T16:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.160+0000 7fea42ff5700 10 client.19725 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea42ff5700 1 -- 172.21.15.62:0/3250337165 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55732755ee50 con 0x557327629840 2023-07-14T16:30:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea42ff5700 10 client.19725 handle_command_reply: tid=0 2023-07-14T16:30:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 client.19725 shutdown 2023-07-14T16:30:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 -- 172.21.15.62:0/3250337165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 msgr2=0x557327625010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327625010 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fea58015940 tx=0x7fea5800b040 comp rx=0 tx=0).stop 2023-07-14T16:30:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 -- 172.21.15.62:0/3250337165 shutdown_connections 2023-07-14T16:30:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327625010 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557327629840 0x55732762bc50 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573274706b0 0x55732761da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 --2- 172.21.15.62:0/3250337165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557327614a90 0x55732761dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 -- 172.21.15.62:0/3250337165 >> 172.21.15.62:0/3250337165 conn(0x557327609d40 msgr2=0x55732760e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea42ff5700 0 client.19725 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 -- 172.21.15.62:0/3250337165 shutdown_connections 2023-07-14T16:30:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.161+0000 7fea78bffb80 1 -- 172.21.15.62:0/3250337165 wait complete. 2023-07-14T16:30:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.162+0000 7fea78bffb80 20 client.19725 trim_cache size 0 max 16384 2023-07-14T16:30:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.178+0000 7fea78bffb80 1 Processor -- start 2023-07-14T16:30:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.178+0000 7fea78bffb80 1 -- start start 2023-07-14T16:30:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.178+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327618630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.178+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557327614a90 0x557327624fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.178+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573276157e0 0x557327618b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.178+0000 7fea78bffb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557327626d00 con 0x557327614a90 2023-07-14T16:30:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.178+0000 7fea78bffb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732761ec80 con 0x5573276157e0 2023-07-14T16:30:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.178+0000 7fea78bffb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732761cf90 con 0x5573274706b0 2023-07-14T16:30:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557327614a90 0x557327624fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557327614a90 0x557327624fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45292/0 (socket says 172.21.15.62:45292) 2023-07-14T16:30:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 -- 172.21.15.62:0/1578512186 learned_addr learned my addr 172.21.15.62:0/1578512186 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5f59e700 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327618630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5f59e700 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573276157e0 0x557327618b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 -- 172.21.15.62:0/1578512186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 msgr2=0x557327618630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327618630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 -- 172.21.15.62:0/1578512186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573276157e0 msgr2=0x557327618b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573276157e0 0x557327618b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 -- 172.21.15.62:0/1578512186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55732737cc00 con 0x557327614a90 2023-07-14T16:30:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5f59e700 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573276157e0 0x557327618b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.179+0000 7fea5e59c700 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557327614a90 0x557327624fd0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b7e0 tx=0x7fea5400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb177e2b7a5e7e5b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.180+0000 7fea45ffb700 1 -- 172.21.15.62:0/1578512186 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54016030 con 0x557327614a90 2023-07-14T16:30:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.180+0000 7fea45ffb700 1 -- 172.21.15.62:0/1578512186 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea54017040 con 0x557327614a90 2023-07-14T16:30:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.180+0000 7fea45ffb700 1 -- 172.21.15.62:0/1578512186 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54005280 con 0x557327614a90 2023-07-14T16:30:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 -- 172.21.15.62:0/1578512186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557327614a90 msgr2=0x557327624fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557327614a90 0x557327624fd0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b7e0 tx=0x7fea5400b410 comp rx=0 tx=0).stop 2023-07-14T16:30:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 -- 172.21.15.62:0/1578512186 shutdown_connections 2023-07-14T16:30:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573276157e0 0x557327618b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327618630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 --2- 172.21.15.62:0/1578512186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557327614a90 0x557327624fd0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 -- 172.21.15.62:0/1578512186 >> 172.21.15.62:0/1578512186 conn(0x557327609d40 msgr2=0x5573276305b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 -- 172.21.15.62:0/1578512186 shutdown_connections 2023-07-14T16:30:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 -- 172.21.15.62:0/1578512186 wait complete. 2023-07-14T16:30:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.181+0000 7fea78bffb80 1 -- start start 2023-07-14T16:30:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.182+0000 7fea78bffb80 1 Processor -- start 2023-07-14T16:30:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.182+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327620da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.182+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.182+0000 7fea78bffb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327624300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.182+0000 7fea78bffb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557327624840 con 0x5573276157e0 2023-07-14T16:30:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.182+0000 7fea78bffb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573276249b0 con 0x557327614a90 2023-07-14T16:30:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.182+0000 7fea78bffb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557327624b70 con 0x5573274706b0 2023-07-14T16:30:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.182+0000 7fea78bffb80 10 client.0 _ll_register_callbacks cb 0x7fea58004de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327624300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5f59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327624300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45304/0 (socket says 172.21.15.62:45304) 2023-07-14T16:30:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5f59e700 1 -- 172.21.15.62:0/2819306099 learned_addr learned my addr 172.21.15.62:0/2819306099 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5e59c700 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276212e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:34354/0 (socket says 172.21.15.62:34354) 2023-07-14T16:30:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5f59e700 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327620da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5e59c700 1 -- 172.21.15.62:0/2819306099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 msgr2=0x557327620da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5e59c700 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327620da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5e59c700 1 -- 172.21.15.62:0/2819306099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 msgr2=0x557327624300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5e59c700 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327624300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5e59c700 1 -- 172.21.15.62:0/2819306099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5400bd90 con 0x557327614a90 2023-07-14T16:30:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5f59e700 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327624300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.183+0000 7fea5e59c700 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276212e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5573276157b0 tx=0x7fea54020f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70700accd5ebc1d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.184+0000 7fea42ff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:30:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.184+0000 7fea42ff5700 1 -- 172.21.15.62:0/2819306099 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5402c030 con 0x557327614a90 2023-07-14T16:30:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.184+0000 7fea78bffb80 1 -- 172.21.15.62:0/2819306099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea5800f040 con 0x557327614a90 2023-07-14T16:30:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.184+0000 7fea42ff5700 1 -- 172.21.15.62:0/2819306099 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fea5402d070 con 0x557327614a90 2023-07-14T16:30:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.184+0000 7fea42ff5700 1 -- 172.21.15.62:0/2819306099 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea54016070 con 0x557327614a90 2023-07-14T16:30:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.184+0000 7fea78bffb80 10 client.29285 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.184+0000 7fea78bffb80 1 -- 172.21.15.62:0/2819306099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5573273ba1f0 con 0x557327614a90 2023-07-14T16:30:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.184+0000 7fea42ff5700 1 -- 172.21.15.62:0/2819306099 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea54037020 con 0x557327614a90 2023-07-14T16:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.185+0000 7fea42ff5700 1 -- 172.21.15.62:0/2819306099 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea54003680 con 0x557327614a90 2023-07-14T16:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.185+0000 7fea78bffb80 10 client.29285 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.185+0000 7fea78bffb80 10 client.29285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.185+0000 7fea78bffb80 20 client.29285 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.185+0000 7fea78bffb80 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55732767e520 0x557327680930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.185+0000 7fea78bffb80 4 client.29285 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.185+0000 7fea78bffb80 1 -- 172.21.15.62:0/2819306099 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557327520ff0 con 0x55732767e520 2023-07-14T16:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.185+0000 7fea5f59e700 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55732767e520 0x557327680930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.186+0000 7fea42ff5700 10 client.29285 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.186+0000 7fea5f59e700 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55732767e520 0x557327680930 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899379ddbbffeb30 server_cookie=f9ee636fbbcee0e1 in_seq=0 out_seq=0 2023-07-14T16:30:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea42ff5700 1 -- 172.21.15.62:0/2819306099 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557327520ff0 con 0x55732767e520 2023-07-14T16:30:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea42ff5700 10 client.29285 handle_command_reply: tid=0 2023-07-14T16:30:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 client.29285 shutdown 2023-07-14T16:30:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 -- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 msgr2=0x5573276212e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276212e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5573276157b0 tx=0x7fea54020f90 comp rx=0 tx=0).stop 2023-07-14T16:30:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 -- 172.21.15.62:0/2819306099 shutdown_connections 2023-07-14T16:30:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573276157e0 0x557327624300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573274706b0 0x557327620da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55732767e520 0x557327680930 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 --2- 172.21.15.62:0/2819306099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557327614a90 0x5573276212e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea78bffb80 1 -- 172.21.15.62:0/2819306099 >> 172.21.15.62:0/2819306099 conn(0x557327609d40 msgr2=0x557327618630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.196+0000 7fea42ff5700 0 client.29285 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.197+0000 7fea78bffb80 1 -- 172.21.15.62:0/2819306099 shutdown_connections 2023-07-14T16:30:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.197+0000 7fea78bffb80 1 -- 172.21.15.62:0/2819306099 wait complete. 2023-07-14T16:30:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.197+0000 7fea78bffb80 20 client.29285 trim_cache size 0 max 16384 2023-07-14T16:30:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:30:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:30:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:30:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:30:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 -- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fea480a1150 msgr2=0x7fea480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fea480a1150 0x7fea480a3600 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea500cbe20 tx=0x7fea50009310 comp rx=0 tx=0).stop 2023-07-14T16:30:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 -- 172.21.15.62:0/1403975880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea600ff940 msgr2=0x7fea601955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea600ff940 0x7fea601955a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fea5000b5c0 tx=0x7fea50005740 comp rx=0 tx=0).stop 2023-07-14T16:30:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 -- 172.21.15.62:0/1403975880 shutdown_connections 2023-07-14T16:30:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea601014c0 0x7fea6019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fea480a1150 0x7fea480a3600 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea600ff940 0x7fea601955a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 --2- 172.21.15.62:0/1403975880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea60100b30 0x7fea60195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 -- 172.21.15.62:0/1403975880 >> 172.21.15.62:0/1403975880 conn(0x7fea600faf40 msgr2=0x7fea60104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 -- 172.21.15.62:0/1403975880 shutdown_connections 2023-07-14T16:30:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:36.198+0000 7fea649fe700 1 -- 172.21.15.62:0/1403975880 wait complete. 2023-07-14T16:30:37.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:36 smithi062 ceph-mon[95543]: pgmap v806: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:30:37.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:36 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:30:37.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:36 smithi081 ceph-mon[127795]: pgmap v806: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:30:37.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:36 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:30:37.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:36 smithi164 ceph-mon[97427]: pgmap v806: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-14T16:30:37.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:36 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:30:38.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:30:38.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:30:38.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:30:38.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:30:38.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:30:38.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:30:38.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:30:38.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:30:38.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:30:39.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:38 smithi062 ceph-mon[95543]: pgmap v807: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 22 op/s 2023-07-14T16:30:39.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:38 smithi081 ceph-mon[127795]: pgmap v807: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 22 op/s 2023-07-14T16:30:39.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:38 smithi164 ceph-mon[97427]: pgmap v807: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 22 op/s 2023-07-14T16:30:41.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:40 smithi062 ceph-mon[95543]: pgmap v808: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:30:41.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:40 smithi081 ceph-mon[127795]: pgmap v808: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:30:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:40 smithi164 ceph-mon[97427]: pgmap v808: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:30:43.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:42 smithi062 ceph-mon[95543]: pgmap v809: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:30:43.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:42 smithi081 ceph-mon[127795]: pgmap v809: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:30:43.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:42 smithi164 ceph-mon[97427]: pgmap v809: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:30:45.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:44 smithi062 ceph-mon[95543]: pgmap v810: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:45.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:44 smithi081 ceph-mon[127795]: pgmap v810: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:45.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:44 smithi164 ceph-mon[97427]: pgmap v810: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 MiB/s wr, 15 op/s 2023-07-14T16:30:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:46 smithi062 ceph-mon[95543]: pgmap v811: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:30:47.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:46 smithi081 ceph-mon[127795]: pgmap v811: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:30:47.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:46 smithi164 ceph-mon[97427]: pgmap v811: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 MiB/s wr, 13 op/s 2023-07-14T16:30:49.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:48 smithi062 ceph-mon[95543]: pgmap v812: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:49.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:48 smithi081 ceph-mon[127795]: pgmap v812: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:49.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:48 smithi164 ceph-mon[97427]: pgmap v812: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 15 MiB/s wr, 13 op/s 2023-07-14T16:30:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.404+0000 7f130759e700 1 -- 172.21.15.62:0/2634657908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13080ffad0 msgr2=0x7f13080fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.404+0000 7f130759e700 1 --2- 172.21.15.62:0/2634657908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13080ffad0 0x7f13080fff40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f12f00098f0 tx=0x7f12f0009c00 comp rx=0 tx=0).stop 2023-07-14T16:30:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.404+0000 7f130759e700 1 -- 172.21.15.62:0/2634657908 shutdown_connections 2023-07-14T16:30:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.405+0000 7f130759e700 1 --2- 172.21.15.62:0/2634657908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1308100480 0x7f1308106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.405+0000 7f130759e700 1 --2- 172.21.15.62:0/2634657908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13080fe840 0x7f13080fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.405+0000 7f130759e700 1 --2- 172.21.15.62:0/2634657908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13080ffad0 0x7f13080fff40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.405+0000 7f130759e700 1 -- 172.21.15.62:0/2634657908 >> 172.21.15.62:0/2634657908 conn(0x7f13080f9e40 msgr2=0x7f13080fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.405+0000 7f130759e700 1 -- 172.21.15.62:0/2634657908 shutdown_connections 2023-07-14T16:30:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.405+0000 7f130759e700 1 -- 172.21.15.62:0/2634657908 wait complete. 2023-07-14T16:30:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.406+0000 7f130759e700 1 Processor -- start 2023-07-14T16:30:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.406+0000 7f130759e700 1 -- start start 2023-07-14T16:30:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.406+0000 7f130759e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13080fe840 0x7f1308102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.406+0000 7f130759e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13080ffad0 0x7f1308100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.406+0000 7f130759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308100480 0x7f13081013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.406+0000 7f130759e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1308101b90 con 0x7f1308100480 2023-07-14T16:30:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f130759e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1308101d00 con 0x7f13080ffad0 2023-07-14T16:30:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f130759e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130819f620 con 0x7f13080fe840 2023-07-14T16:30:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f1306d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308100480 0x7f13081013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f12fffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13080ffad0 0x7f1308100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f130659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13080fe840 0x7f1308102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f1306d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308100480 0x7f13081013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60452/0 (socket says 172.21.15.62:60452) 2023-07-14T16:30:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f1306d9d700 1 -- 172.21.15.62:0/4111143292 learned_addr learned my addr 172.21.15.62:0/4111143292 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f1306d9d700 1 -- 172.21.15.62:0/4111143292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13080fe840 msgr2=0x7f1308102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f1306d9d700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13080fe840 0x7f1308102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f1306d9d700 1 -- 172.21.15.62:0/4111143292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13080ffad0 msgr2=0x7f1308100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f1306d9d700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13080ffad0 0x7f1308100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.407+0000 7f1306d9d700 1 -- 172.21.15.62:0/4111143292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f0009520 con 0x7f1308100480 2023-07-14T16:30:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.408+0000 7f1306d9d700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308100480 0x7f13081013f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f1300007ee0 tx=0x7f130000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.408+0000 7f12ff7fe700 1 -- 172.21.15.62:0/4111143292 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300017070 con 0x7f1308100480 2023-07-14T16:30:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.408+0000 7f12ff7fe700 1 -- 172.21.15.62:0/4111143292 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f130000f040 con 0x7f1308100480 2023-07-14T16:30:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.408+0000 7f12ff7fe700 1 -- 172.21.15.62:0/4111143292 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300005710 con 0x7f1308100480 2023-07-14T16:30:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.408+0000 7f130759e700 1 -- 172.21.15.62:0/4111143292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130819f900 con 0x7f1308100480 2023-07-14T16:30:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.408+0000 7f130759e700 1 -- 172.21.15.62:0/4111143292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130819fdc0 con 0x7f1308100480 2023-07-14T16:30:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.412+0000 7f12ff7fe700 1 -- 172.21.15.62:0/4111143292 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1300005870 con 0x7f1308100480 2023-07-14T16:30:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f12ff7fe700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12f40a1150 0x7f12f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f12ff7fe700 1 -- 172.21.15.62:0/4111143292 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f13000cbff0 con 0x7f1308100480 2023-07-14T16:30:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f13202b8b80 1 Processor -- start 2023-07-14T16:30:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f130659c700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12f40a1150 0x7f12f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f13202b8b80 1 -- start start 2023-07-14T16:30:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca85550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca862a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca86c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f13202b8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656c98ac50 con 0x55656ca85a90 2023-07-14T16:30:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f13202b8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656c9dd9e0 con 0x55656c8e16b0 2023-07-14T16:30:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.414+0000 7f13202b8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656c9d6e70 con 0x55656ca867e0 2023-07-14T16:30:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f12fffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca862a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f12fffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca862a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60456/0 (socket says 172.21.15.62:60456) 2023-07-14T16:30:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f12fffff700 1 -- 172.21.15.62:0/3796366380 learned_addr learned my addr 172.21.15.62:0/3796366380 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f130659c700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12f40a1150 0x7f12f40a3600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f12f8005fd0 tx=0x7f12f8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f130659c700 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca85550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f1306d9d700 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca86c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f12fffff700 1 -- 172.21.15.62:0/3796366380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 msgr2=0x55656ca86c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f12fffff700 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca86c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f12fffff700 1 -- 172.21.15.62:0/3796366380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 msgr2=0x55656ca85550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f12fffff700 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca85550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.415+0000 7f12fffff700 1 -- 172.21.15.62:0/3796366380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55656c804d80 con 0x55656ca85a90 2023-07-14T16:30:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.416+0000 7f12fffff700 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca862a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009810 tx=0x7f12f000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ca51460c14a0e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.416+0000 7f12fcff9700 1 -- 172.21.15.62:0/3796366380 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f000fda0 con 0x55656ca85a90 2023-07-14T16:30:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.416+0000 7f12fcff9700 1 -- 172.21.15.62:0/3796366380 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12f001d040 con 0x55656ca85a90 2023-07-14T16:30:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.416+0000 7f12fcff9700 1 -- 172.21.15.62:0/3796366380 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0005040 con 0x55656ca85a90 2023-07-14T16:30:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 -- 172.21.15.62:0/3796366380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 msgr2=0x55656ca862a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca862a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009810 tx=0x7f12f000bec0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 -- 172.21.15.62:0/3796366380 shutdown_connections 2023-07-14T16:30:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca86c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca85550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3796366380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca862a0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 -- 172.21.15.62:0/3796366380 >> 172.21.15.62:0/3796366380 conn(0x55656ca7ad40 msgr2=0x55656ca7fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 -- 172.21.15.62:0/3796366380 shutdown_connections 2023-07-14T16:30:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.417+0000 7f13202b8b80 1 -- 172.21.15.62:0/3796366380 wait complete. 2023-07-14T16:30:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.418+0000 7f13202b8b80 1 -- start start 2023-07-14T16:30:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.418+0000 7f13202b8b80 1 Processor -- start 2023-07-14T16:30:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.418+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca8ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.418+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca8ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.418+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca95f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.418+0000 7f13202b8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656c976a60 con 0x55656ca85a90 2023-07-14T16:30:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.418+0000 7f13202b8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656c976760 con 0x55656c8e16b0 2023-07-14T16:30:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.418+0000 7f13202b8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656c85a8c0 con 0x55656ca867e0 2023-07-14T16:30:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f13202b8b80 10 client.0 _ll_register_callbacks cb 0x55656c974700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f12fffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca8ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f130659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca8ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f12fffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca8ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60468/0 (socket says 172.21.15.62:60468) 2023-07-14T16:30:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f12fffff700 1 -- 172.21.15.62:0/3571325614 learned_addr learned my addr 172.21.15.62:0/3571325614 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f1306d9d700 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca95f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f12fffff700 1 -- 172.21.15.62:0/3571325614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 msgr2=0x55656ca95f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f12fffff700 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca95f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f12fffff700 1 -- 172.21.15.62:0/3571325614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 msgr2=0x55656ca8ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f12fffff700 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca8ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f12fffff700 1 -- 172.21.15.62:0/3571325614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1308107590 con 0x55656ca85a90 2023-07-14T16:30:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f1306d9d700 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca95f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.419+0000 7f130659c700 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca8ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.420+0000 7f12fffff700 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca8ef40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006010 tx=0x7f12f000fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6c4d81acfabd12 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.420+0000 7f12edffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.420+0000 7f12edffb700 1 -- 172.21.15.62:0/3571325614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f000f6f0 con 0x55656ca85a90 2023-07-14T16:30:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.420+0000 7f12edffb700 1 -- 172.21.15.62:0/3571325614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12f0005050 con 0x55656ca85a90 2023-07-14T16:30:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.420+0000 7f13202b8b80 1 -- 172.21.15.62:0/3571325614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55656c930490 con 0x55656ca85a90 2023-07-14T16:30:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.420+0000 7f12edffb700 1 -- 172.21.15.62:0/3571325614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f0005210 con 0x55656ca85a90 2023-07-14T16:30:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.420+0000 7f13202b8b80 10 client.19743 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.420+0000 7f13202b8b80 1 -- 172.21.15.62:0/3571325614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55656c7edc00 con 0x55656ca85a90 2023-07-14T16:30:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f12edffb700 1 -- 172.21.15.62:0/3571325614 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55656c930490 con 0x55656ca85a90 2023-07-14T16:30:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f12edffb700 1 -- 172.21.15.62:0/3571325614 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12f003c070 con 0x55656ca85a90 2023-07-14T16:30:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f13202b8b80 10 client.19743 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f13202b8b80 10 client.19743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f13202b8b80 20 client.19743 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55656ca9a2d0 0x55656ca9c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f13202b8b80 4 client.19743 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f13202b8b80 1 -- 172.21.15.62:0/3571325614 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55656c9cfe50 con 0x55656ca9a2d0 2023-07-14T16:30:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.421+0000 7f130659c700 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55656ca9a2d0 0x55656ca9c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.422+0000 7f12edffb700 10 client.19743 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.422+0000 7f130659c700 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55656ca9a2d0 0x55656ca9c6e0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb5c8b0d03f4a67 server_cookie=ebbd2f7df400f85e in_seq=0 out_seq=0 2023-07-14T16:30:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f12edffb700 1 -- 172.21.15.62:0/3571325614 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55656c9cfe50 con 0x55656ca9a2d0 2023-07-14T16:30:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f12edffb700 10 client.19743 handle_command_reply: tid=0 2023-07-14T16:30:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 client.19743 shutdown 2023-07-14T16:30:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 -- 172.21.15.62:0/3571325614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 msgr2=0x55656ca8ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca8ef40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006010 tx=0x7f12f000fe50 comp rx=0 tx=0).stop 2023-07-14T16:30:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 -- 172.21.15.62:0/3571325614 shutdown_connections 2023-07-14T16:30:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca867e0 0x55656ca95f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55656ca9a2d0 0x55656ca9c6e0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656c8e16b0 0x55656ca8ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 --2- 172.21.15.62:0/3571325614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca8ef40 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f12edffb700 0 client.19743 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 -- 172.21.15.62:0/3571325614 >> 172.21.15.62:0/3571325614 conn(0x55656ca7ad40 msgr2=0x55656ca7f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 -- 172.21.15.62:0/3571325614 shutdown_connections 2023-07-14T16:30:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.423+0000 7f13202b8b80 1 -- 172.21.15.62:0/3571325614 wait complete. 2023-07-14T16:30:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.424+0000 7f13202b8b80 20 client.19743 trim_cache size 0 max 16384 2023-07-14T16:30:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.440+0000 7f13202b8b80 1 Processor -- start 2023-07-14T16:30:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.440+0000 7f13202b8b80 1 -- start start 2023-07-14T16:30:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.440+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656c8e16b0 0x55656ca8d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.440+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca95f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656caa1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f13202b8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656ca8deb0 con 0x55656ca85a90 2023-07-14T16:30:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f13202b8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656ca80bb0 con 0x55656ca867e0 2023-07-14T16:30:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f13202b8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656ca8d4f0 con 0x55656c8e16b0 2023-07-14T16:30:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f12fffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca95f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f12fffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca95f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60480/0 (socket says 172.21.15.62:60480) 2023-07-14T16:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f12fffff700 1 -- 172.21.15.62:0/2019939468 learned_addr learned my addr 172.21.15.62:0/2019939468 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f130659c700 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656c8e16b0 0x55656ca8d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f1306d9d700 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656caa1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f12fffff700 1 -- 172.21.15.62:0/2019939468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656c8e16b0 msgr2=0x55656ca8d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f12fffff700 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656c8e16b0 0x55656ca8d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f12fffff700 1 -- 172.21.15.62:0/2019939468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 msgr2=0x55656caa1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f12fffff700 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656caa1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f12fffff700 1 -- 172.21.15.62:0/2019939468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55656c82b1f0 con 0x55656ca85a90 2023-07-14T16:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.441+0000 7f1306d9d700 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656caa1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.442+0000 7f12fffff700 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca95f40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005c50 tx=0x7f12f000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93e8276c8d80aba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.442+0000 7f12fd7fa700 1 -- 172.21.15.62:0/2019939468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00323a0 con 0x55656ca85a90 2023-07-14T16:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.442+0000 7f12fd7fa700 1 -- 172.21.15.62:0/2019939468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12f0039070 con 0x55656ca85a90 2023-07-14T16:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.442+0000 7f12fd7fa700 1 -- 172.21.15.62:0/2019939468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00050e0 con 0x55656ca85a90 2023-07-14T16:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 -- 172.21.15.62:0/2019939468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 msgr2=0x55656ca95f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca95f40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005c50 tx=0x7f12f000bec0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 -- 172.21.15.62:0/2019939468 shutdown_connections 2023-07-14T16:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656caa1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656c8e16b0 0x55656ca8d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 --2- 172.21.15.62:0/2019939468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656ca85a90 0x55656ca95f40 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 -- 172.21.15.62:0/2019939468 >> 172.21.15.62:0/2019939468 conn(0x55656ca7ad40 msgr2=0x55656ca7fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 -- 172.21.15.62:0/2019939468 shutdown_connections 2023-07-14T16:30:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 -- 172.21.15.62:0/2019939468 wait complete. 2023-07-14T16:30:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.443+0000 7f13202b8b80 1 -- start start 2023-07-14T16:30:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f13202b8b80 1 Processor -- start 2023-07-14T16:30:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656c8e16b0 0x55656ca91850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca85a90 0x55656ca91d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f13202b8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656ca94db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f13202b8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656ca952f0 con 0x55656c8e16b0 2023-07-14T16:30:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f13202b8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656ca95460 con 0x55656ca867e0 2023-07-14T16:30:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f13202b8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656ca95620 con 0x55656ca85a90 2023-07-14T16:30:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f13202b8b80 10 client.0 _ll_register_callbacks cb 0x7f12f8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.444+0000 7f12fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca85a90 0x55656ca91d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f1306d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656ca94db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca85a90 0x55656ca91d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46614/0 (socket says 172.21.15.62:46614) 2023-07-14T16:30:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f1306d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656ca94db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:54150/0 (socket says 172.21.15.62:54150) 2023-07-14T16:30:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12fffff700 1 -- 172.21.15.62:0/45941911 learned_addr learned my addr 172.21.15.62:0/45941911 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f130659c700 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656c8e16b0 0x55656ca91850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12fffff700 1 -- 172.21.15.62:0/45941911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 msgr2=0x55656ca94db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12fffff700 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656ca94db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12fffff700 1 -- 172.21.15.62:0/45941911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656c8e16b0 msgr2=0x55656ca91850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12fffff700 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656c8e16b0 0x55656ca91850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12fffff700 1 -- 172.21.15.62:0/45941911 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55656c7edc00 con 0x55656ca85a90 2023-07-14T16:30:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f130659c700 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656c8e16b0 0x55656ca91850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f1306d9d700 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656ca94db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12fffff700 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca85a90 0x55656ca91d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12f000a010 tx=0x7f12f000bfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826cd7a765d31c99 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.445+0000 7f12edffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:30:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.446+0000 7f12edffb700 1 -- 172.21.15.62:0/45941911 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f002a500 con 0x55656ca85a90 2023-07-14T16:30:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.446+0000 7f12edffb700 1 -- 172.21.15.62:0/45941911 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12f0000e40 con 0x55656ca85a90 2023-07-14T16:30:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.446+0000 7f13202b8b80 1 -- 172.21.15.62:0/45941911 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12f001d040 con 0x55656ca85a90 2023-07-14T16:30:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.446+0000 7f12edffb700 1 -- 172.21.15.62:0/45941911 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f00050b0 con 0x55656ca85a90 2023-07-14T16:30:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.446+0000 7f13202b8b80 10 client.29161 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.446+0000 7f13202b8b80 1 -- 172.21.15.62:0/45941911 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55656c803250 con 0x55656ca85a90 2023-07-14T16:30:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.446+0000 7f12edffb700 1 -- 172.21.15.62:0/45941911 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12f001d040 con 0x55656ca85a90 2023-07-14T16:30:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.447+0000 7f12edffb700 1 -- 172.21.15.62:0/45941911 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12f0028040 con 0x55656ca85a90 2023-07-14T16:30:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.447+0000 7f13202b8b80 10 client.29161 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.447+0000 7f13202b8b80 10 client.29161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.447+0000 7f13202b8b80 20 client.29161 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.447+0000 7f13202b8b80 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55656caee820 0x55656caf0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.447+0000 7f13202b8b80 4 client.29161 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.447+0000 7f13202b8b80 1 -- 172.21.15.62:0/45941911 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55656c991ff0 con 0x55656caee820 2023-07-14T16:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.447+0000 7f130659c700 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55656caee820 0x55656caf0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.448+0000 7f12edffb700 10 client.29161 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.448+0000 7f130659c700 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55656caee820 0x55656caf0c00 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b0dc9c300757dc server_cookie=686a4ae813101269 in_seq=0 out_seq=0 2023-07-14T16:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f12edffb700 1 -- 172.21.15.62:0/45941911 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55656c991ff0 con 0x55656caee820 2023-07-14T16:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f12edffb700 10 client.29161 handle_command_reply: tid=0 2023-07-14T16:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 client.29161 shutdown 2023-07-14T16:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 -- 172.21.15.62:0/45941911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca85a90 msgr2=0x55656ca91d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca85a90 0x55656ca91d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12f000a010 tx=0x7f12f000bfb0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 -- 172.21.15.62:0/45941911 shutdown_connections 2023-07-14T16:30:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55656caee820 0x55656caf0c00 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55656ca867e0 0x55656ca94db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55656c8e16b0 0x55656ca91850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f12edffb700 0 client.29161 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 --2- 172.21.15.62:0/45941911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55656ca85a90 0x55656ca91d90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 -- 172.21.15.62:0/45941911 >> 172.21.15.62:0/45941911 conn(0x55656ca7ad40 msgr2=0x55656caa1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.458+0000 7f13202b8b80 1 -- 172.21.15.62:0/45941911 shutdown_connections 2023-07-14T16:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f13202b8b80 1 -- 172.21.15.62:0/45941911 wait complete. 2023-07-14T16:30:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f13202b8b80 20 client.29161 trim_cache size 0 max 16384 2023-07-14T16:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:30:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 -- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12f40a1150 msgr2=0x7f12f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12f40a1150 0x7f12f40a3600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f12f8005fd0 tx=0x7f12f8009310 comp rx=0 tx=0).stop 2023-07-14T16:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 -- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308100480 msgr2=0x7f13081013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308100480 0x7f13081013f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f1300007ee0 tx=0x7f130000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 -- 172.21.15.62:0/4111143292 shutdown_connections 2023-07-14T16:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1308100480 0x7f13081013f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13080fe840 0x7f1308102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f12f40a1150 0x7f12f40a3600 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.459+0000 7f130759e700 1 --2- 172.21.15.62:0/4111143292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13080ffad0 0x7f1308100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.460+0000 7f130759e700 1 -- 172.21.15.62:0/4111143292 >> 172.21.15.62:0/4111143292 conn(0x7f13080f9e40 msgr2=0x7f13080679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.460+0000 7f130759e700 1 -- 172.21.15.62:0/4111143292 shutdown_connections 2023-07-14T16:30:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:30:49.460+0000 7f130759e700 1 -- 172.21.15.62:0/4111143292 wait complete. 2023-07-14T16:30:51.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:50 smithi062 ceph-mon[95543]: pgmap v813: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:30:51.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:50 smithi081 ceph-mon[127795]: pgmap v813: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:30:51.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:50 smithi164 ceph-mon[97427]: pgmap v813: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-14T16:30:53.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:52 smithi062 ceph-mon[95543]: pgmap v814: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:30:53.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:52 smithi081 ceph-mon[127795]: pgmap v814: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:30:53.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:30:53.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:52 smithi164 ceph-mon[97427]: pgmap v814: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:30:53.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 -- 172.21.15.62:0/1759861004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 msgr2=0x7f17d4102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 --2- 172.21.15.62:0/1759861004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 0x7f17d4102010 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009990 tx=0x7f17c4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 -- 172.21.15.62:0/1759861004 shutdown_connections 2023-07-14T16:30:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 --2- 172.21.15.62:0/1759861004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d4103780 0x7f17d410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 --2- 172.21.15.62:0/1759861004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 0x7f17d4102010 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 --2- 172.21.15.62:0/1759861004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d4102df0 0x7f17d4103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 -- 172.21.15.62:0/1759861004 >> 172.21.15.62:0/1759861004 conn(0x7f17d40fd220 msgr2=0x7f17d40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 -- 172.21.15.62:0/1759861004 shutdown_connections 2023-07-14T16:30:53.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.528+0000 7f17d3fff700 1 -- 172.21.15.62:0/1759861004 wait complete. 2023-07-14T16:30:53.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.529+0000 7f17d3fff700 1 Processor -- start 2023-07-14T16:30:53.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.529+0000 7f17d3fff700 1 -- start start 2023-07-14T16:30:53.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 0x7f17d41976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d4102df0 0x7f17d4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d4103780 0x7f17d419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d3fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d419f270 con 0x7f17d4101c00 2023-07-14T16:30:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d3fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d419f3b0 con 0x7f17d4102df0 2023-07-14T16:30:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d3fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d419f540 con 0x7f17d4103780 2023-07-14T16:30:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 0x7f17d41976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 0x7f17d41976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44074/0 (socket says 172.21.15.62:44074) 2023-07-14T16:30:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.530+0000 7f17d2ffd700 1 -- 172.21.15.62:0/3156901628 learned_addr learned my addr 172.21.15.62:0/3156901628 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d27fc700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d4102df0 0x7f17d4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d37fe700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d4103780 0x7f17d419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d2ffd700 1 -- 172.21.15.62:0/3156901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d4103780 msgr2=0x7f17d419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d4103780 0x7f17d419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d2ffd700 1 -- 172.21.15.62:0/3156901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d4102df0 msgr2=0x7f17d4197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d4102df0 0x7f17d4197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d2ffd700 1 -- 172.21.15.62:0/3156901628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c40095f0 con 0x7f17d4101c00 2023-07-14T16:30:53.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d37fe700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d4103780 0x7f17d419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:30:53.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 0x7f17d41976f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f17c4000c00 tx=0x7f17c4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:53.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d89fd700 1 -- 172.21.15.62:0/3156901628 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c401d070 con 0x7f17d4101c00 2023-07-14T16:30:53.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d89fd700 1 -- 172.21.15.62:0/3156901628 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17c4003fc0 con 0x7f17d4101c00 2023-07-14T16:30:53.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d3fff700 1 -- 172.21.15.62:0/3156901628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d419f7c0 con 0x7f17d4101c00 2023-07-14T16:30:53.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.531+0000 7f17d89fd700 1 -- 172.21.15.62:0/3156901628 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4022c20 con 0x7f17d4101c00 2023-07-14T16:30:53.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.532+0000 7f17d3fff700 1 -- 172.21.15.62:0/3156901628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d419fbe0 con 0x7f17d4101c00 2023-07-14T16:30:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.536+0000 7f17d89fd700 1 -- 172.21.15.62:0/3156901628 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f17c4003b30 con 0x7f17d4101c00 2023-07-14T16:30:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.537+0000 7f17ed371b80 1 Processor -- start 2023-07-14T16:30:53.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.537+0000 7f17ed371b80 1 -- start start 2023-07-14T16:30:53.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.537+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x5573098f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.537+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x5573098f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.537+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 0x5573098fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.537+0000 7f17ed371b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573097fdcb0 con 0x5573098f9070 2023-07-14T16:30:53.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.537+0000 7f17ed371b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557309850a40 con 0x5573098f8720 2023-07-14T16:30:53.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.537+0000 7f17ed371b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557309849ed0 con 0x5573098f9e30 2023-07-14T16:30:53.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x5573098f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x5573098f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x5573098f98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44076/0 (socket says 172.21.15.62:44076) 2023-07-14T16:30:53.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d2ffd700 1 -- 172.21.15.62:0/2652637362 learned_addr learned my addr 172.21.15.62:0/2652637362 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:53.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d27fc700 1 --2- 172.21.15.62:0/2652637362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 0x5573098fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d89fd700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17bc0a1270 0x7f17bc0a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d89fd700 1 -- 172.21.15.62:0/3156901628 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17c40cdfc0 con 0x7f17d4101c00 2023-07-14T16:30:53.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d37fe700 1 -- 172.21.15.62:0/2652637362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 msgr2=0x5573098fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:53.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d37fe700 1 --2- 172.21.15.62:0/2652637362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 0x5573098fa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d37fe700 1 -- 172.21.15.62:0/2652637362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 msgr2=0x5573098f98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:53.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d37fe700 1 --2- 172.21.15.62:0/2652637362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x5573098f98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d37fe700 1 -- 172.21.15.62:0/2652637362 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557309677de0 con 0x5573098f8720 2023-07-14T16:30:53.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.538+0000 7f17d37fe700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17bc0a1270 0x7f17bc0a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.539+0000 7f17d37fe700 1 --2- 172.21.15.62:0/2652637362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x5573098f8b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009dd0 tx=0x7f17cc0077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923ae338e74e3b04 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:53.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.539+0000 7f17b9ffb700 1 -- 172.21.15.62:0/2652637362 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc007ea0 con 0x5573098f8720 2023-07-14T16:30:53.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.539+0000 7f17b9ffb700 1 -- 172.21.15.62:0/2652637362 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17cc017ca0 con 0x5573098f8720 2023-07-14T16:30:53.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.539+0000 7f17b9ffb700 1 -- 172.21.15.62:0/2652637362 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0213a0 con 0x5573098f8720 2023-07-14T16:30:53.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.539+0000 7f17d37fe700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17bc0a1270 0x7f17bc0a3720 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f17cc01f000 tx=0x7f17cc01c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:53.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 -- 172.21.15.62:0/2652637362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 msgr2=0x5573098f8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 --2- 172.21.15.62:0/2652637362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x5573098f8b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009dd0 tx=0x7f17cc0077e0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 -- 172.21.15.62:0/2652637362 shutdown_connections 2023-07-14T16:30:53.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 --2- 172.21.15.62:0/2652637362 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 0x5573098fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 --2- 172.21.15.62:0/2652637362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x5573098f8b30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 --2- 172.21.15.62:0/2652637362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x5573098f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 -- 172.21.15.62:0/2652637362 >> 172.21.15.62:0/2652637362 conn(0x5573098edff0 msgr2=0x5573098f3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:53.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 -- 172.21.15.62:0/2652637362 shutdown_connections 2023-07-14T16:30:53.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.540+0000 7f17ed371b80 1 -- 172.21.15.62:0/2652637362 wait complete. 2023-07-14T16:30:53.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.541+0000 7f17ed371b80 1 -- start start 2023-07-14T16:30:53.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.541+0000 7f17ed371b80 1 Processor -- start 2023-07-14T16:30:53.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309902340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x557309902880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 0x5573099098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17ed371b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573097e9ac0 con 0x5573098f9070 2023-07-14T16:30:53.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17ed371b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573097e97c0 con 0x5573098f8720 2023-07-14T16:30:53.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17ed371b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573096cd920 con 0x5573098f9e30 2023-07-14T16:30:53.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17ed371b80 10 client.0 _ll_register_callbacks cb 0x5573097e7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:53.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x557309902880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x557309902880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44088/0 (socket says 172.21.15.62:44088) 2023-07-14T16:30:53.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d27fc700 1 -- 172.21.15.62:0/52698759 learned_addr learned my addr 172.21.15.62:0/52698759 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:53.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309902340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d37fe700 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 0x5573099098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d27fc700 1 -- 172.21.15.62:0/52698759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 msgr2=0x5573099098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:53.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d27fc700 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 0x5573099098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d27fc700 1 -- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 msgr2=0x557309902340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:53.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d27fc700 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309902340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.542+0000 7f17d27fc700 1 -- 172.21.15.62:0/52698759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc009890 con 0x5573098f9070 2023-07-14T16:30:53.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309902340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:53.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17d27fc700 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x557309902880 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f17c80124f0 tx=0x7f17c8012800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498c92f031d7af4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:53.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:53.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17b2ffd700 1 -- 172.21.15.62:0/52698759 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8004220 con 0x5573098f9070 2023-07-14T16:30:53.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17ed371b80 1 -- 172.21.15.62:0/52698759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573097a34f0 con 0x5573098f9070 2023-07-14T16:30:53.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17b2ffd700 1 -- 172.21.15.62:0/52698759 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17c8004380 con 0x5573098f9070 2023-07-14T16:30:53.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17b2ffd700 1 -- 172.21.15.62:0/52698759 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c8004540 con 0x5573098f9070 2023-07-14T16:30:53.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17ed371b80 10 client.19761 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:53.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.543+0000 7f17ed371b80 1 -- 172.21.15.62:0/52698759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557309660c60 con 0x5573098f9070 2023-07-14T16:30:53.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.544+0000 7f17b2ffd700 1 -- 172.21.15.62:0/52698759 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5573097a34f0 con 0x5573098f9070 2023-07-14T16:30:53.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.544+0000 7f17b2ffd700 1 -- 172.21.15.62:0/52698759 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f17c800fa50 con 0x5573098f9070 2023-07-14T16:30:53.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.544+0000 7f17ed371b80 10 client.19761 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:53.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.544+0000 7f17ed371b80 10 client.19761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:53.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.544+0000 7f17ed371b80 20 client.19761 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:53.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.544+0000 7f17ed371b80 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55730990e120 0x557309910530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.544+0000 7f17ed371b80 4 client.19761 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:53.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.544+0000 7f17ed371b80 1 -- 172.21.15.62:0/52698759 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557309842eb0 con 0x55730990e120 2023-07-14T16:30:53.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.545+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55730990e120 0x557309910530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.545+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55730990e120 0x557309910530 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cecbf0e4233bb284 server_cookie=9a0b32bfc237550e in_seq=0 out_seq=0 2023-07-14T16:30:53.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.545+0000 7f17b2ffd700 10 client.19761 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:53.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17b2ffd700 1 -- 172.21.15.62:0/52698759 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557309842eb0 con 0x55730990e120 2023-07-14T16:30:53.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17b2ffd700 10 client.19761 handle_command_reply: tid=0 2023-07-14T16:30:53.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 client.19761 shutdown 2023-07-14T16:30:53.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 -- 172.21.15.62:0/52698759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 msgr2=0x557309902880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x557309902880 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f17c80124f0 tx=0x7f17c8012800 comp rx=0 tx=0).stop 2023-07-14T16:30:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 -- 172.21.15.62:0/52698759 shutdown_connections 2023-07-14T16:30:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9e30 0x5573099098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55730990e120 0x557309910530 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309902340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 --2- 172.21.15.62:0/52698759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9070 0x557309902880 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 -- 172.21.15.62:0/52698759 >> 172.21.15.62:0/52698759 conn(0x5573098edff0 msgr2=0x5573098f2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17b2ffd700 0 client.19761 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 -- 172.21.15.62:0/52698759 shutdown_connections 2023-07-14T16:30:53.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.546+0000 7f17ed371b80 1 -- 172.21.15.62:0/52698759 wait complete. 2023-07-14T16:30:53.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.547+0000 7f17ed371b80 20 client.19761 trim_cache size 0 max 16384 2023-07-14T16:30:53.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.567+0000 7f17ed371b80 1 Processor -- start 2023-07-14T16:30:53.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.567+0000 7f17ed371b80 1 -- start start 2023-07-14T16:30:53.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.567+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309900270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.567+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 0x557309909880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.567+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9e30 0x557309910130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17ed371b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730990b5b0 con 0x5573098f9e30 2023-07-14T16:30:53.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17ed371b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557309900020 con 0x5573098f8720 2023-07-14T16:30:53.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17ed371b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557309903430 con 0x5573098f9070 2023-07-14T16:30:53.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9e30 0x557309910130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309900270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9e30 0x557309910130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44096/0 (socket says 172.21.15.62:44096) 2023-07-14T16:30:53.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d37fe700 1 -- 172.21.15.62:0/466906298 learned_addr learned my addr 172.21.15.62:0/466906298 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:53.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d27fc700 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 0x557309909880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d2ffd700 1 -- 172.21.15.62:0/466906298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 msgr2=0x557309909880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:53.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 0x557309909880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d2ffd700 1 -- 172.21.15.62:0/466906298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9e30 msgr2=0x557309910130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:53.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9e30 0x557309910130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d2ffd700 1 -- 172.21.15.62:0/466906298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573098fa4d0 con 0x5573098f8720 2023-07-14T16:30:53.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.568+0000 7f17d37fe700 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9e30 0x557309910130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:53.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.569+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309900270 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f17c4097970 tx=0x7f17c40a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca14709edfa5e90 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:53.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.569+0000 7f17ba7fc700 1 -- 172.21.15.62:0/466906298 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c409add0 con 0x5573098f8720 2023-07-14T16:30:53.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.569+0000 7f17ba7fc700 1 -- 172.21.15.62:0/466906298 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17c4022470 con 0x5573098f8720 2023-07-14T16:30:53.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.569+0000 7f17ba7fc700 1 -- 172.21.15.62:0/466906298 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40a3830 con 0x5573098f8720 2023-07-14T16:30:53.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 -- 172.21.15.62:0/466906298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 msgr2=0x557309900270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309900270 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f17c4097970 tx=0x7f17c40a1040 comp rx=0 tx=0).stop 2023-07-14T16:30:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 -- 172.21.15.62:0/466906298 shutdown_connections 2023-07-14T16:30:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f9e30 0x557309910130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f8720 0x557309900270 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 --2- 172.21.15.62:0/466906298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 0x557309909880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 -- 172.21.15.62:0/466906298 >> 172.21.15.62:0/466906298 conn(0x5573098edff0 msgr2=0x557309901010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 -- 172.21.15.62:0/466906298 shutdown_connections 2023-07-14T16:30:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 -- 172.21.15.62:0/466906298 wait complete. 2023-07-14T16:30:53.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.570+0000 7f17ed371b80 1 -- start start 2023-07-14T16:30:53.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17ed371b80 1 Processor -- start 2023-07-14T16:30:53.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f8720 0x557309904da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 0x5573099052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17ed371b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f9e30 0x557309908300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17ed371b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557309908840 con 0x5573098f8720 2023-07-14T16:30:53.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17ed371b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573099089b0 con 0x5573098f9e30 2023-07-14T16:30:53.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17ed371b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557309908b70 con 0x5573098f9070 2023-07-14T16:30:53.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17ed371b80 10 client.0 _ll_register_callbacks cb 0x7f17c800db60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:53.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f8720 0x557309904da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f8720 0x557309904da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44106/0 (socket says 172.21.15.62:44106) 2023-07-14T16:30:53.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17d2ffd700 1 -- 172.21.15.62:0/432821604 learned_addr learned my addr 172.21.15.62:0/432821604 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:53.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.571+0000 7f17d27fc700 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 0x5573099052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17d37fe700 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f9e30 0x557309908300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17d2ffd700 1 -- 172.21.15.62:0/432821604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 msgr2=0x5573099052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:53.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 0x5573099052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17d2ffd700 1 -- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f9e30 msgr2=0x557309908300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:53.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f9e30 0x557309908300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17d2ffd700 1 -- 172.21.15.62:0/432821604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557309677de0 con 0x5573098f8720 2023-07-14T16:30:53.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17d37fe700 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f9e30 0x557309908300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:30:53.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17d2ffd700 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f8720 0x557309904da0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f17c40bb040 tx=0x7f17c4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54943f87cf5e52ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:53.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:53.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17b2ffd700 1 -- 172.21.15.62:0/432821604 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40b1060 con 0x5573098f8720 2023-07-14T16:30:53.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17b2ffd700 1 -- 172.21.15.62:0/432821604 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17c409b470 con 0x5573098f8720 2023-07-14T16:30:53.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.572+0000 7f17ed371b80 1 -- 172.21.15.62:0/432821604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17cc017ca0 con 0x5573098f8720 2023-07-14T16:30:53.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17b2ffd700 1 -- 172.21.15.62:0/432821604 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c409add0 con 0x5573098f8720 2023-07-14T16:30:53.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17ed371b80 10 client.19767 fetch_fsmap learned FSMap version 27 2023-07-14T16:30:53.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17ed371b80 1 -- 172.21.15.62:0/432821604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f17d419fbe0 con 0x5573098f8720 2023-07-14T16:30:53.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17b2ffd700 1 -- 172.21.15.62:0/432821604 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17cc017ca0 con 0x5573098f8720 2023-07-14T16:30:53.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17b2ffd700 1 -- 172.21.15.62:0/432821604 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f17c409f120 con 0x5573098f8720 2023-07-14T16:30:53.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17ed371b80 10 client.19767 fetch_fsmap finished waiting for FSMap version 27 2023-07-14T16:30:53.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17ed371b80 10 client.19767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:53.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17ed371b80 20 client.19767 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17ed371b80 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557309962250 0x557309964630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17ed371b80 4 client.19767 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:30:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.573+0000 7f17ed371b80 1 -- 172.21.15.62:0/432821604 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557309805050 con 0x557309962250 2023-07-14T16:30:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.574+0000 7f17d27fc700 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557309962250 0x557309964630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.574+0000 7f17d27fc700 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557309962250 0x557309964630 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12242d53c1d551a server_cookie=27bf53d0653297d7 in_seq=0 out_seq=0 2023-07-14T16:30:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.574+0000 7f17b2ffd700 10 client.19767 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.574+0000 7f17b2ffd700 1 -- 172.21.15.62:0/432821604 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557309805050 con 0x557309962250 2023-07-14T16:30:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.574+0000 7f17b2ffd700 10 client.19767 handle_command_reply: tid=0 2023-07-14T16:30:53.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.574+0000 7f17ed371b80 1 client.19767 shutdown 2023-07-14T16:30:53.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 -- 172.21.15.62:0/432821604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f8720 msgr2=0x557309904da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f8720 0x557309904da0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f17c40bb040 tx=0x7f17c4009310 comp rx=0 tx=0).stop 2023-07-14T16:30:53.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 -- 172.21.15.62:0/432821604 shutdown_connections 2023-07-14T16:30:53.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557309962250 0x557309964630 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5573098f9e30 0x557309908300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573098f8720 0x557309904da0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 --2- 172.21.15.62:0/432821604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5573098f9070 0x5573099052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17b2ffd700 0 client.19767 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:53.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 -- 172.21.15.62:0/432821604 >> 172.21.15.62:0/432821604 conn(0x5573098edff0 msgr2=0x557309900270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:53.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 -- 172.21.15.62:0/432821604 shutdown_connections 2023-07-14T16:30:53.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 1 -- 172.21.15.62:0/432821604 wait complete. 2023-07-14T16:30:53.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.575+0000 7f17ed371b80 20 client.19767 trim_cache size 0 max 16384 2023-07-14T16:30:53.557 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:30:53.558 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:30:53.558 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:30:53.558 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:30:53.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 -- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17bc0a1270 msgr2=0x7f17bc0a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17bc0a1270 0x7f17bc0a3720 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f17cc01f000 tx=0x7f17cc01c000 comp rx=0 tx=0).stop 2023-07-14T16:30:53.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 -- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 msgr2=0x7f17d41976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 0x7f17d41976f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f17c4000c00 tx=0x7f17c4005ef0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 -- 172.21.15.62:0/3156901628 shutdown_connections 2023-07-14T16:30:53.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17d4103780 0x7f17d419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17d4101c00 0x7f17d41976f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17bc0a1270 0x7f17bc0a3720 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3156901628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17d4102df0 0x7f17d4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 -- 172.21.15.62:0/3156901628 >> 172.21.15.62:0/3156901628 conn(0x7f17d40fd220 msgr2=0x7f17d41062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:53.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 -- 172.21.15.62:0/3156901628 shutdown_connections 2023-07-14T16:30:53.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.576+0000 7f17d3fff700 1 -- 172.21.15.62:0/3156901628 wait complete. 2023-07-14T16:30:53.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:30:53.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:30:53.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.816+0000 7f07f559d700 1 -- 172.21.15.62:0/69057146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07f0101570 msgr2=0x7f07f0107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.816+0000 7f07f559d700 1 --2- 172.21.15.62:0/69057146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07f0101570 0x7f07f0107e30 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f07e800b0a0 tx=0x7f07e800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.816+0000 7f07f559d700 1 -- 172.21.15.62:0/69057146 shutdown_connections 2023-07-14T16:30:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.816+0000 7f07f559d700 1 --2- 172.21.15.62:0/69057146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07f0101570 0x7f07f0107e30 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.816+0000 7f07f559d700 1 --2- 172.21.15.62:0/69057146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f00ff9f0 0x7f07f00ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.816+0000 7f07f559d700 1 --2- 172.21.15.62:0/69057146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07f0100be0 0x7f07f0101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.816+0000 7f07f559d700 1 -- 172.21.15.62:0/69057146 >> 172.21.15.62:0/69057146 conn(0x7f07f00fb010 msgr2=0x7f07f00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.817+0000 7f07f559d700 1 -- 172.21.15.62:0/69057146 shutdown_connections 2023-07-14T16:30:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.817+0000 7f07f559d700 1 -- 172.21.15.62:0/69057146 wait complete. 2023-07-14T16:30:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.818+0000 7f07f559d700 1 Processor -- start 2023-07-14T16:30:53.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.819+0000 7f07f559d700 1 -- start start 2023-07-14T16:30:53.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.819+0000 7f07f559d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07f00ff9f0 0x7f07f0197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.819+0000 7f07f559d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07f0100be0 0x7f07f0197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.819+0000 7f07f559d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f0101570 0x7f07f019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.819+0000 7f07f559d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f019f300 con 0x7f07f00ff9f0 2023-07-14T16:30:53.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.819+0000 7f07f559d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f019f440 con 0x7f07f0100be0 2023-07-14T16:30:53.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.819+0000 7f07f559d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f019f5d0 con 0x7f07f0101570 2023-07-14T16:30:53.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.819+0000 7f07effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07f00ff9f0 0x7f07f0197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07f0100be0 0x7f07f0197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07f0100be0 0x7f07f0197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:52802/0 (socket says 172.21.15.62:52802) 2023-07-14T16:30:53.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 -- 172.21.15.62:0/2642114127 learned_addr learned my addr 172.21.15.62:0/2642114127 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:53.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07f4d9c700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f0101570 0x7f07f019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 -- 172.21.15.62:0/2642114127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f0101570 msgr2=0x7f07f019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f0101570 0x7f07f019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 -- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07f00ff9f0 msgr2=0x7f07f0197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:53.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07f00ff9f0 0x7f07f0197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:53.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 -- 172.21.15.62:0/2642114127 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8009d00 con 0x7f07f0100be0 2023-07-14T16:30:53.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.820+0000 7f07ef7fe700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07f0100be0 0x7f07f0197cc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f07e400d980 tx=0x7f07e400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:53.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.821+0000 7f07ed7fa700 1 -- 172.21.15.62:0/2642114127 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4004220 con 0x7f07f0100be0 2023-07-14T16:30:53.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.821+0000 7f07ed7fa700 1 -- 172.21.15.62:0/2642114127 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07e4007640 con 0x7f07f0100be0 2023-07-14T16:30:53.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.821+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f019f8b0 con 0x7f07f0100be0 2023-07-14T16:30:53.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.821+0000 7f07ed7fa700 1 -- 172.21.15.62:0/2642114127 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4005660 con 0x7f07f0100be0 2023-07-14T16:30:53.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.821+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f019fe10 con 0x7f07f0100be0 2023-07-14T16:30:53.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.822+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f010b0d0 con 0x7f07f0100be0 2023-07-14T16:30:53.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.826+0000 7f07ed7fa700 1 -- 172.21.15.62:0/2642114127 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f07e4004750 con 0x7f07f0100be0 2023-07-14T16:30:53.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.828+0000 7f07ed7fa700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07d80a12c0 0x7f07d80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:53.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.828+0000 7f07ed7fa700 1 -- 172.21.15.62:0/2642114127 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f07e40cc5c0 con 0x7f07f0100be0 2023-07-14T16:30:53.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.828+0000 7f07effff700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07d80a12c0 0x7f07d80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:53.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.829+0000 7f07effff700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07d80a12c0 0x7f07d80a3770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f07f0100a40 tx=0x7f07e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:53.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:53.831+0000 7f07ed7fa700 1 -- 172.21.15.62:0/2642114127 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f07e4095d20 con 0x7f07f0100be0 2023-07-14T16:30:54.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.138+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07f010b2c0 con 0x7f07f0100be0 2023-07-14T16:30:54.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.140+0000 7f07ed7fa700 1 -- 172.21.15.62:0/2642114127 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10826 (secure 0 0 0) 0x7f07e4016090 con 0x7f07f0100be0 2023-07-14T16:30:54.114 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:30:54.114 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:30:53.891154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:30:54.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07d80a12c0 msgr2=0x7f07d80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07d80a12c0 0x7f07d80a3770 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f07f0100a40 tx=0x7f07e000b410 comp rx=0 tx=0).stop 2023-07-14T16:30:54.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07f0100be0 msgr2=0x7f07f0197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07f0100be0 0x7f07f0197cc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f07e400d980 tx=0x7f07e400dc90 comp rx=0 tx=0).stop 2023-07-14T16:30:54.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 shutdown_connections 2023-07-14T16:30:54.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f0101570 0x7f07f019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07f00ff9f0 0x7f07f0197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07d80a12c0 0x7f07d80a3770 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.142+0000 7f07f559d700 1 --2- 172.21.15.62:0/2642114127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07f0100be0 0x7f07f0197cc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.143+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 >> 172.21.15.62:0/2642114127 conn(0x7f07f00fb010 msgr2=0x7f07f01063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.143+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 shutdown_connections 2023-07-14T16:30:54.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.143+0000 7f07f559d700 1 -- 172.21.15.62:0/2642114127 wait complete. 2023-07-14T16:30:54.119 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2023-07-14T16:30:54.136 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:30:54.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 -- 172.21.15.62:0/2598871716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8800100cd0 msgr2=0x7f8800101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 --2- 172.21.15.62:0/2598871716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8800100cd0 0x7f8800101120 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009990 tx=0x7f87f4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 -- 172.21.15.62:0/2598871716 shutdown_connections 2023-07-14T16:30:54.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 --2- 172.21.15.62:0/2598871716 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8800101660 0x7f8800107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 --2- 172.21.15.62:0/2598871716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88000ffae0 0x7f88000ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 --2- 172.21.15.62:0/2598871716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8800100cd0 0x7f8800101120 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 -- 172.21.15.62:0/2598871716 >> 172.21.15.62:0/2598871716 conn(0x7f88000fb0e0 msgr2=0x7f88000fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 -- 172.21.15.62:0/2598871716 shutdown_connections 2023-07-14T16:30:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.381+0000 7f8805fff700 1 -- 172.21.15.62:0/2598871716 wait complete. 2023-07-14T16:30:54.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f8805fff700 1 Processor -- start 2023-07-14T16:30:54.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f8805fff700 1 -- start start 2023-07-14T16:30:54.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f8805fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88000ffae0 0x7f88001957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f8805fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8800100cd0 0x7f8800195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f8805fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8800101660 0x7f880019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f8805fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019d330 con 0x7f88000ffae0 2023-07-14T16:30:54.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f8805fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019d470 con 0x7f8800100cd0 2023-07-14T16:30:54.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f8805fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019d600 con 0x7f8800101660 2023-07-14T16:30:54.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.383+0000 7f87ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88000ffae0 0x7f88001957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88000ffae0 0x7f88001957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44140/0 (socket says 172.21.15.62:44140) 2023-07-14T16:30:54.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87ff7fe700 1 -- 172.21.15.62:0/4246990492 learned_addr learned my addr 172.21.15.62:0/4246990492 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87feffd700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8800100cd0 0x7f8800195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87fffff700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8800101660 0x7f880019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87ff7fe700 1 -- 172.21.15.62:0/4246990492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8800101660 msgr2=0x7f880019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8800101660 0x7f880019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87ff7fe700 1 -- 172.21.15.62:0/4246990492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8800100cd0 msgr2=0x7f8800195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8800100cd0 0x7f8800195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87ff7fe700 1 -- 172.21.15.62:0/4246990492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40095f0 con 0x7f88000ffae0 2023-07-14T16:30:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.384+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88000ffae0 0x7f88001957b0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e970 tx=0x7f87f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.385+0000 7f87fcff9700 1 -- 172.21.15.62:0/4246990492 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0009790 con 0x7f88000ffae0 2023-07-14T16:30:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.385+0000 7f87fcff9700 1 -- 172.21.15.62:0/4246990492 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87f000eec0 con 0x7f88000ffae0 2023-07-14T16:30:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.385+0000 7f8805fff700 1 -- 172.21.15.62:0/4246990492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880019d8e0 con 0x7f88000ffae0 2023-07-14T16:30:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.385+0000 7f87fcff9700 1 -- 172.21.15.62:0/4246990492 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0003c10 con 0x7f88000ffae0 2023-07-14T16:30:54.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.385+0000 7f8805fff700 1 -- 172.21.15.62:0/4246990492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880019de40 con 0x7f88000ffae0 2023-07-14T16:30:54.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.389+0000 7f87fcff9700 1 -- 172.21.15.62:0/4246990492 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f87f0003db0 con 0x7f88000ffae0 2023-07-14T16:30:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.390+0000 7f87fcff9700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87e80a11d0 0x7f87e80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.390+0000 7f87fcff9700 1 -- 172.21.15.62:0/4246990492 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87f0014070 con 0x7f88000ffae0 2023-07-14T16:30:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f8819a12b80 1 Processor -- start 2023-07-14T16:30:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f87feffd700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87e80a11d0 0x7f87e80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f8819a12b80 1 -- start start 2023-07-14T16:30:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbe480 0x55893afbe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afbf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbfc70 0x55893afc00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f8819a12b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893aec3cb0 con 0x55893afbedd0 2023-07-14T16:30:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f8819a12b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893af16a40 con 0x55893afbe480 2023-07-14T16:30:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f8819a12b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893af0fed0 con 0x55893afbfc70 2023-07-14T16:30:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.391+0000 7f87feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afbf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87e80a11d0 0x7f87e80a3680 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f87f4006010 tx=0x7f87f400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afbf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44146/0 (socket says 172.21.15.62:44146) 2023-07-14T16:30:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 -- 172.21.15.62:0/2414520534 learned_addr learned my addr 172.21.15.62:0/2414520534 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87fffff700 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbfc70 0x55893afc00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbe480 0x55893afbe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 -- 172.21.15.62:0/2414520534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbfc70 msgr2=0x55893afc00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbfc70 0x55893afc00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 -- 172.21.15.62:0/2414520534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbe480 msgr2=0x55893afbe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbe480 0x55893afbe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 -- 172.21.15.62:0/2414520534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55893ad3dde0 con 0x55893afbedd0 2023-07-14T16:30:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.392+0000 7f87feffd700 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afbf6e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f87f4015000 tx=0x7f87f4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5492644b5bddaa0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.393+0000 7f87e5ffb700 1 -- 172.21.15.62:0/2414520534 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4003c90 con 0x55893afbedd0 2023-07-14T16:30:54.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.393+0000 7f87e5ffb700 1 -- 172.21.15.62:0/2414520534 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87f401e040 con 0x55893afbedd0 2023-07-14T16:30:54.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.393+0000 7f87e5ffb700 1 -- 172.21.15.62:0/2414520534 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004960 con 0x55893afbedd0 2023-07-14T16:30:54.371 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.393+0000 7f8819a12b80 1 -- 172.21.15.62:0/2414520534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 msgr2=0x55893afbf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.393+0000 7f8819a12b80 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afbf6e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f87f4015000 tx=0x7f87f4020000 comp rx=0 tx=0).stop 2023-07-14T16:30:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.394+0000 7f8819a12b80 1 -- 172.21.15.62:0/2414520534 shutdown_connections 2023-07-14T16:30:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.394+0000 7f8819a12b80 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbfc70 0x55893afc00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.394+0000 7f8819a12b80 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbe480 0x55893afbe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.394+0000 7f8819a12b80 1 --2- 172.21.15.62:0/2414520534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afbf6e0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.394+0000 7f8819a12b80 1 -- 172.21.15.62:0/2414520534 >> 172.21.15.62:0/2414520534 conn(0x55893afb3da0 msgr2=0x55893afb91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.394+0000 7f8819a12b80 1 -- 172.21.15.62:0/2414520534 shutdown_connections 2023-07-14T16:30:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.394+0000 7f8819a12b80 1 -- 172.21.15.62:0/2414520534 wait complete. 2023-07-14T16:30:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.394+0000 7f8819a12b80 1 -- start start 2023-07-14T16:30:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.395+0000 7f8819a12b80 1 Processor -- start 2023-07-14T16:30:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.395+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afd0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.395+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afd0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.395+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.395+0000 7f8819a12b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893aeafac0 con 0x55893afbedd0 2023-07-14T16:30:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.395+0000 7f8819a12b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893aeaf7c0 con 0x55893afbfc70 2023-07-14T16:30:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.395+0000 7f8819a12b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893ad93920 con 0x55893afbe480 2023-07-14T16:30:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.395+0000 7f8819a12b80 10 client.0 _ll_register_callbacks cb 0x55893aead760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afd0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afd0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44154/0 (socket says 172.21.15.62:44154) 2023-07-14T16:30:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87feffd700 1 -- 172.21.15.62:0/4187396674 learned_addr learned my addr 172.21.15.62:0/4187396674 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afd0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87fffff700 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87ff7fe700 1 -- 172.21.15.62:0/4187396674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 msgr2=0x55893afd7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87ff7fe700 1 -- 172.21.15.62:0/4187396674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 msgr2=0x55893afd0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afd0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.396+0000 7f87ff7fe700 1 -- 172.21.15.62:0/4187396674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8800108850 con 0x55893afbe480 2023-07-14T16:30:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afd0970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87f0099c20 tx=0x7f87f000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e62fc500fc5574 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f87deffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:30:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f87deffd700 1 -- 172.21.15.62:0/4187396674 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0099610 con 0x55893afbe480 2023-07-14T16:30:54.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f87deffd700 1 -- 172.21.15.62:0/4187396674 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87f009df50 con 0x55893afbe480 2023-07-14T16:30:54.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f8819a12b80 1 -- 172.21.15.62:0/4187396674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55893ae694f0 con 0x55893afbe480 2023-07-14T16:30:54.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f87deffd700 1 -- 172.21.15.62:0/4187396674 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f00a2370 con 0x55893afbe480 2023-07-14T16:30:54.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f8819a12b80 10 client.29194 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:54.376 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f8819a12b80 1 -- 172.21.15.62:0/4187396674 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55893ad26c60 con 0x55893afbe480 2023-07-14T16:30:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.397+0000 7f87deffd700 1 -- 172.21.15.62:0/4187396674 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55893ae694f0 con 0x55893afbe480 2023-07-14T16:30:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.398+0000 7f87deffd700 1 -- 172.21.15.62:0/4187396674 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f87f0095d10 con 0x55893afbe480 2023-07-14T16:30:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.398+0000 7f8819a12b80 10 client.29194 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.398+0000 7f8819a12b80 10 client.29194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:30:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.398+0000 7f8819a12b80 20 client.29194 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.398+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55893afdc680 0x55893afdea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.398+0000 7f8819a12b80 4 client.29194 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.398+0000 7f8819a12b80 1 -- 172.21.15.62:0/4187396674 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55893af08eb0 con 0x55893afdc680 2023-07-14T16:30:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.398+0000 7f87feffd700 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55893afdc680 0x55893afdea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.399+0000 7f87feffd700 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55893afdc680 0x55893afdea90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbb1357e2f9ca3e8 server_cookie=c8bcde522c5f9819 in_seq=0 out_seq=0 2023-07-14T16:30:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.399+0000 7f87deffd700 10 client.29194 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:30:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f87deffd700 1 -- 172.21.15.62:0/4187396674 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55893af08eb0 con 0x55893afdc680 2023-07-14T16:30:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f87deffd700 10 client.29194 handle_command_reply: tid=0 2023-07-14T16:30:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 client.29194 shutdown 2023-07-14T16:30:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 -- 172.21.15.62:0/4187396674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 msgr2=0x55893afd0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afd0970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87f0099c20 tx=0x7f87f000cd50 comp rx=0 tx=0).stop 2023-07-14T16:30:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 -- 172.21.15.62:0/4187396674 shutdown_connections 2023-07-14T16:30:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55893afdc680 0x55893afdea90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afd0970 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f87deffd700 0 client.29194 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:30:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4187396674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afd0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 -- 172.21.15.62:0/4187396674 >> 172.21.15.62:0/4187396674 conn(0x55893afb3da0 msgr2=0x55893afb88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 -- 172.21.15.62:0/4187396674 shutdown_connections 2023-07-14T16:30:54.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 1 -- 172.21.15.62:0/4187396674 wait complete. 2023-07-14T16:30:54.381 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.400+0000 7f8819a12b80 20 client.29194 trim_cache size 0 max 16384 2023-07-14T16:30:54.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.419+0000 7f8819a12b80 1 Processor -- start 2023-07-14T16:30:54.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.419+0000 7f8819a12b80 1 -- start start 2023-07-14T16:30:54.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afcb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afcbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f8819a12b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893afb3970 con 0x55893afbedd0 2023-07-14T16:30:54.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f8819a12b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893afd1af0 con 0x55893afbfc70 2023-07-14T16:30:54.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f8819a12b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893afdc450 con 0x55893afbe480 2023-07-14T16:30:54.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f87feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afcbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f87feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afcbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44166/0 (socket says 172.21.15.62:44166) 2023-07-14T16:30:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f87feffd700 1 -- 172.21.15.62:0/4281565346 learned_addr learned my addr 172.21.15.62:0/4281565346 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afcb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.420+0000 7f87fffff700 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87feffd700 1 -- 172.21.15.62:0/4281565346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 msgr2=0x55893afcb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87feffd700 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afcb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87feffd700 1 -- 172.21.15.62:0/4281565346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 msgr2=0x55893afd7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87feffd700 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87feffd700 1 -- 172.21.15.62:0/4281565346 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55893ad64250 con 0x55893afbedd0 2023-07-14T16:30:54.397 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afcb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:54.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87feffd700 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afcbd80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f87f4003960 tx=0x7f87f400b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d33513bb3699c23 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87e67fc700 1 -- 172.21.15.62:0/4281565346 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4034040 con 0x55893afbedd0 2023-07-14T16:30:54.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87e67fc700 1 -- 172.21.15.62:0/4281565346 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87f4035040 con 0x55893afbedd0 2023-07-14T16:30:54.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.421+0000 7f87e67fc700 1 -- 172.21.15.62:0/4281565346 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f400b6b0 con 0x55893afbedd0 2023-07-14T16:30:54.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.422+0000 7f8819a12b80 1 -- 172.21.15.62:0/4281565346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 msgr2=0x55893afcbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.422+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afcbd80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f87f4003960 tx=0x7f87f400b2d0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.422+0000 7f8819a12b80 1 -- 172.21.15.62:0/4281565346 shutdown_connections 2023-07-14T16:30:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.422+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.422+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbe480 0x55893afcb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.422+0000 7f8819a12b80 1 --2- 172.21.15.62:0/4281565346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbedd0 0x55893afcbd80 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.422+0000 7f8819a12b80 1 -- 172.21.15.62:0/4281565346 >> 172.21.15.62:0/4281565346 conn(0x55893afb3da0 msgr2=0x55893afe33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.422+0000 7f8819a12b80 1 -- 172.21.15.62:0/4281565346 shutdown_connections 2023-07-14T16:30:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.423+0000 7f8819a12b80 1 -- 172.21.15.62:0/4281565346 wait complete. 2023-07-14T16:30:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.423+0000 7f8819a12b80 1 -- start start 2023-07-14T16:30:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.423+0000 7f8819a12b80 1 Processor -- start 2023-07-14T16:30:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbe480 0x55893afd3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbedd0 0x55893afd41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f8819a12b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f8819a12b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893afd7720 con 0x55893afbe480 2023-07-14T16:30:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f8819a12b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893afd7890 con 0x55893afbfc70 2023-07-14T16:30:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f8819a12b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893afd7a50 con 0x55893afbedd0 2023-07-14T16:30:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f8819a12b80 10 client.0 _ll_register_callbacks cb 0x55893aead760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbe480 0x55893afd3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbe480 0x55893afd3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44168/0 (socket says 172.21.15.62:44168) 2023-07-14T16:30:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87ff7fe700 1 -- 172.21.15.62:0/1707016732 learned_addr learned my addr 172.21.15.62:0/1707016732 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87feffd700 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbedd0 0x55893afd41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87fffff700 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87ff7fe700 1 -- 172.21.15.62:0/1707016732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbedd0 msgr2=0x55893afd41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbedd0 0x55893afd41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87ff7fe700 1 -- 172.21.15.62:0/1707016732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 msgr2=0x55893afd71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.424+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f87ff7fe700 1 -- 172.21.15.62:0/1707016732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55893ad26c60 con 0x55893afbe480 2023-07-14T16:30:54.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f87feffd700 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbedd0 0x55893afd41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f87ff7fe700 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbe480 0x55893afd3c80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f87f0096420 tx=0x7f87f0096390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8677ae729cdf5a12 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f87deffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f87deffd700 1 -- 172.21.15.62:0/1707016732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f00a20e0 con 0x55893afbe480 2023-07-14T16:30:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f87deffd700 1 -- 172.21.15.62:0/1707016732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87f009df50 con 0x55893afbe480 2023-07-14T16:30:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f8819a12b80 1 -- 172.21.15.62:0/1707016732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87f401e040 con 0x55893afbe480 2023-07-14T16:30:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f87deffd700 1 -- 172.21.15.62:0/1707016732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f00a2370 con 0x55893afbe480 2023-07-14T16:30:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f8819a12b80 10 client.19791 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.425+0000 7f8819a12b80 1 -- 172.21.15.62:0/1707016732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55893ad3dde0 con 0x55893afbe480 2023-07-14T16:30:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.426+0000 7f87deffd700 1 -- 172.21.15.62:0/1707016732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87f007e020 con 0x55893afbe480 2023-07-14T16:30:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.426+0000 7f87deffd700 1 -- 172.21.15.62:0/1707016732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f87f00ca2d0 con 0x55893afbe480 2023-07-14T16:30:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.426+0000 7f8819a12b80 10 client.19791 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.426+0000 7f8819a12b80 10 client.19791 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:30:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.426+0000 7f8819a12b80 20 client.19791 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.426+0000 7f8819a12b80 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55893b030f40 0x55893b033350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.426+0000 7f8819a12b80 4 client.19791 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:30:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.426+0000 7f8819a12b80 1 -- 172.21.15.62:0/1707016732 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55893aecb050 con 0x55893b030f40 2023-07-14T16:30:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.427+0000 7f87feffd700 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55893b030f40 0x55893b033350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.427+0000 7f87deffd700 10 client.19791 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:30:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.427+0000 7f87feffd700 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55893b030f40 0x55893b033350 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f3031796d403dbc server_cookie=f5599f91c893a402 in_seq=0 out_seq=0 2023-07-14T16:30:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.427+0000 7f87deffd700 1 -- 172.21.15.62:0/1707016732 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55893aecb050 con 0x55893b030f40 2023-07-14T16:30:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.427+0000 7f87deffd700 10 client.19791 handle_command_reply: tid=0 2023-07-14T16:30:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.427+0000 7f8819a12b80 1 client.19791 shutdown 2023-07-14T16:30:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 -- 172.21.15.62:0/1707016732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbe480 msgr2=0x55893afd3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbe480 0x55893afd3c80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f87f0096420 tx=0x7f87f0096390 comp rx=0 tx=0).stop 2023-07-14T16:30:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 -- 172.21.15.62:0/1707016732 shutdown_connections 2023-07-14T16:30:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55893b030f40 0x55893b033350 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55893afbfc70 0x55893afd71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55893afbe480 0x55893afd3c80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 --2- 172.21.15.62:0/1707016732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55893afbedd0 0x55893afd41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 -- 172.21.15.62:0/1707016732 >> 172.21.15.62:0/1707016732 conn(0x55893afb3da0 msgr2=0x55893afcb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f87deffd700 0 client.19791 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:30:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 -- 172.21.15.62:0/1707016732 shutdown_connections 2023-07-14T16:30:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 1 -- 172.21.15.62:0/1707016732 wait complete. 2023-07-14T16:30:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.428+0000 7f8819a12b80 20 client.19791 trim_cache size 0 max 16384 2023-07-14T16:30:54.410 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:30:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 -- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87e80a11d0 msgr2=0x7f87e80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87e80a11d0 0x7f87e80a3680 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f87f4006010 tx=0x7f87f400b260 comp rx=0 tx=0).stop 2023-07-14T16:30:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 -- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88000ffae0 msgr2=0x7f88001957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88000ffae0 0x7f88001957b0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e970 tx=0x7f87f000ec80 comp rx=0 tx=0).stop 2023-07-14T16:30:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 -- 172.21.15.62:0/4246990492 shutdown_connections 2023-07-14T16:30:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8800101660 0x7f880019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88000ffae0 0x7f88001957b0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f87e80a11d0 0x7f87e80a3680 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 --2- 172.21.15.62:0/4246990492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8800100cd0 0x7f8800195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 -- 172.21.15.62:0/4246990492 >> 172.21.15.62:0/4246990492 conn(0x7f88000fb0e0 msgr2=0x7f88000fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 -- 172.21.15.62:0/4246990492 shutdown_connections 2023-07-14T16:30:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.429+0000 7f8805fff700 1 -- 172.21.15.62:0/4246990492 wait complete. 2023-07-14T16:30:54.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:30:54.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 -- 172.21.15.62:0/3190624580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c0ff8c0 msgr2=0x7f7c3c0ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/3190624580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c0ff8c0 0x7f7c3c0ffcd0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009990 tx=0x7f7c30009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 -- 172.21.15.62:0/3190624580 shutdown_connections 2023-07-14T16:30:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/3190624580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c3c101440 0x7f7c3c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/3190624580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c0ff8c0 0x7f7c3c0ffcd0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/3190624580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c100ab0 0x7f7c3c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 -- 172.21.15.62:0/3190624580 >> 172.21.15.62:0/3190624580 conn(0x7f7c3c0faea0 msgr2=0x7f7c3c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 -- 172.21.15.62:0/3190624580 shutdown_connections 2023-07-14T16:30:54.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.697+0000 7f7c41d9d700 1 -- 172.21.15.62:0/3190624580 wait complete. 2023-07-14T16:30:54.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.698+0000 7f7c41d9d700 1 Processor -- start 2023-07-14T16:30:54.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c41d9d700 1 -- start start 2023-07-14T16:30:54.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c41d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0ff8c0 0x7f7c3c195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c41d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c100ab0 0x7f7c3c195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c41d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c3c101440 0x7f7c3c196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c41d9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c196fa0 con 0x7f7c3c100ab0 2023-07-14T16:30:54.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c41d9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c197110 con 0x7f7c3c101440 2023-07-14T16:30:54.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c41d9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c1972d0 con 0x7f7c3c0ff8c0 2023-07-14T16:30:54.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c100ab0 0x7f7c3c195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c100ab0 0x7f7c3c195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44186/0 (socket says 172.21.15.62:44186) 2023-07-14T16:30:54.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c3bfff700 1 -- 172.21.15.62:0/1089797719 learned_addr learned my addr 172.21.15.62:0/1089797719 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.699+0000 7f7c4159c700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c3c101440 0x7f7c3c196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.700+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0ff8c0 0x7f7c3c195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.700+0000 7f7c3bfff700 1 -- 172.21.15.62:0/1089797719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0ff8c0 msgr2=0x7f7c3c195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.700+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0ff8c0 0x7f7c3c195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.700+0000 7f7c3bfff700 1 -- 172.21.15.62:0/1089797719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c3c101440 msgr2=0x7f7c3c196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.700+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c3c101440 0x7f7c3c196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.700+0000 7f7c3bfff700 1 -- 172.21.15.62:0/1089797719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c300095f0 con 0x7f7c3c100ab0 2023-07-14T16:30:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.700+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c100ab0 0x7f7c3c195ac0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c9b0 tx=0x7f7c2800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.700+0000 7f7c39ffb700 1 -- 172.21.15.62:0/1089797719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28007b00 con 0x7f7c3c100ab0 2023-07-14T16:30:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.701+0000 7f7c41d9d700 1 -- 172.21.15.62:0/1089797719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c1975b0 con 0x7f7c3c100ab0 2023-07-14T16:30:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.701+0000 7f7c39ffb700 1 -- 172.21.15.62:0/1089797719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c2800ce50 con 0x7f7c3c100ab0 2023-07-14T16:30:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.701+0000 7f7c39ffb700 1 -- 172.21.15.62:0/1089797719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28018400 con 0x7f7c3c100ab0 2023-07-14T16:30:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.701+0000 7f7c41d9d700 1 -- 172.21.15.62:0/1089797719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c19dd70 con 0x7f7c3c100ab0 2023-07-14T16:30:54.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.704+0000 7f7c39ffb700 1 -- 172.21.15.62:0/1089797719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7c2800f450 con 0x7f7c3c100ab0 2023-07-14T16:30:54.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.705+0000 7f7c569abb80 1 Processor -- start 2023-07-14T16:30:54.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c569abb80 1 -- start start 2023-07-14T16:30:54.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c39ffb700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7c2c0a1150 0x7f7c2c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf249f720 0x561bf249fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c39ffb700 1 -- 172.21.15.62:0/1089797719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7c280cc890 con 0x7f7c3c100ab0 2023-07-14T16:30:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 0x561bf24a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c569abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf23a4cb0 con 0x561bf24a0070 2023-07-14T16:30:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c569abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf23f7a40 con 0x561bf24a0e30 2023-07-14T16:30:54.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c569abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf23f0ed0 con 0x561bf249f720 2023-07-14T16:30:54.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c4159c700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7c2c0a1150 0x7f7c2c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.706+0000 7f7c40d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf249f720 0x561bf249fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44196/0 (socket says 172.21.15.62:44196) 2023-07-14T16:30:54.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c3bfff700 1 -- 172.21.15.62:0/3551540109 learned_addr learned my addr 172.21.15.62:0/3551540109 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c4159c700 1 --2- 172.21.15.62:0/3551540109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 0x561bf24a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c40d9b700 1 -- 172.21.15.62:0/3551540109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 msgr2=0x561bf24a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/3551540109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 0x561bf24a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c40d9b700 1 -- 172.21.15.62:0/3551540109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 msgr2=0x561bf24a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/3551540109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c40d9b700 1 -- 172.21.15.62:0/3551540109 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bf221ede0 con 0x561bf249f720 2023-07-14T16:30:54.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c4159c700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7c2c0a1150 0x7f7c2c0a3600 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c34007740 tx=0x7f7c34014390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.707+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/3551540109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf249f720 0x561bf249fb30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c30000c00 tx=0x7f7c30005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b00fb4c4ee33427 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.708+0000 7f7c26ffd700 1 -- 172.21.15.62:0/3551540109 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3001d070 con 0x561bf249f720 2023-07-14T16:30:54.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.708+0000 7f7c26ffd700 1 -- 172.21.15.62:0/3551540109 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c3001e040 con 0x561bf249f720 2023-07-14T16:30:54.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.708+0000 7f7c26ffd700 1 -- 172.21.15.62:0/3551540109 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30004760 con 0x561bf249f720 2023-07-14T16:30:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 -- 172.21.15.62:0/3551540109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf249f720 msgr2=0x561bf249fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3551540109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf249f720 0x561bf249fb30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c30000c00 tx=0x7f7c30005bc0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 -- 172.21.15.62:0/3551540109 shutdown_connections 2023-07-14T16:30:54.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3551540109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 0x561bf24a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3551540109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf249f720 0x561bf249fb30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3551540109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 -- 172.21.15.62:0/3551540109 >> 172.21.15.62:0/3551540109 conn(0x561bf2494ff0 msgr2=0x561bf249a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 -- 172.21.15.62:0/3551540109 shutdown_connections 2023-07-14T16:30:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 -- 172.21.15.62:0/3551540109 wait complete. 2023-07-14T16:30:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.709+0000 7f7c569abb80 1 -- start start 2023-07-14T16:30:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c569abb80 1 Processor -- start 2023-07-14T16:30:54.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 0x561bf24ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 0x561bf24adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0e30 0x561bf24b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c569abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf2390ac0 con 0x561bf24a0e30 2023-07-14T16:30:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c569abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf23907c0 con 0x561bf249f720 2023-07-14T16:30:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c569abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf2274920 con 0x561bf24a0070 2023-07-14T16:30:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c569abb80 10 client.0 _ll_register_callbacks cb 0x561bf238e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:54.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c4159c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0e30 0x561bf24b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c4159c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0e30 0x561bf24b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44202/0 (socket says 172.21.15.62:44202) 2023-07-14T16:30:54.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c4159c700 1 -- 172.21.15.62:0/3899608036 learned_addr learned my addr 172.21.15.62:0/3899608036 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.710+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 0x561bf24ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 0x561bf24adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c4159c700 1 -- 172.21.15.62:0/3899608036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 msgr2=0x561bf24adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c4159c700 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 0x561bf24adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c4159c700 1 -- 172.21.15.62:0/3899608036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 msgr2=0x561bf24ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c4159c700 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 0x561bf24ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c4159c700 1 -- 172.21.15.62:0/3899608036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c108630 con 0x561bf24a0e30 2023-07-14T16:30:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c4159c700 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0e30 0x561bf24b4c10 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7c34007d70 tx=0x7f7c34007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81cea9b646d23ad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c1bfff700 1 -- 172.21.15.62:0/3899608036 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34014ea0 con 0x561bf24a0e30 2023-07-14T16:30:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c569abb80 1 -- 172.21.15.62:0/3899608036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bf234a4f0 con 0x561bf24a0e30 2023-07-14T16:30:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.711+0000 7f7c1bfff700 1 -- 172.21.15.62:0/3899608036 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7c34019e20 con 0x561bf24a0e30 2023-07-14T16:30:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c1bfff700 1 -- 172.21.15.62:0/3899608036 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c34024460 con 0x561bf24a0e30 2023-07-14T16:30:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c569abb80 10 client.19803 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c569abb80 1 -- 172.21.15.62:0/3899608036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561bf2207c60 con 0x561bf24a0e30 2023-07-14T16:30:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c1bfff700 1 -- 172.21.15.62:0/3899608036 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c34033020 con 0x561bf24a0e30 2023-07-14T16:30:54.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c1bfff700 1 -- 172.21.15.62:0/3899608036 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7c3402d610 con 0x561bf24a0e30 2023-07-14T16:30:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c569abb80 10 client.19803 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c569abb80 10 client.19803 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:30:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c569abb80 20 client.19803 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:54.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561bf24b9430 0x561bf24bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c569abb80 4 client.19803 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.712+0000 7f7c569abb80 1 -- 172.21.15.62:0/3899608036 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bf23e9eb0 con 0x561bf24b9430 2023-07-14T16:30:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.713+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561bf24b9430 0x561bf24bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.713+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561bf24b9430 0x561bf24bb840 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7496fa71f5ce7aca server_cookie=a5ff8854bea35d63 in_seq=0 out_seq=0 2023-07-14T16:30:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.713+0000 7f7c1bfff700 10 client.19803 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:30:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.714+0000 7f7c1bfff700 1 -- 172.21.15.62:0/3899608036 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561bf23e9eb0 con 0x561bf24b9430 2023-07-14T16:30:54.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.714+0000 7f7c1bfff700 10 client.19803 handle_command_reply: tid=0 2023-07-14T16:30:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.714+0000 7f7c569abb80 1 client.19803 shutdown 2023-07-14T16:30:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.714+0000 7f7c569abb80 1 -- 172.21.15.62:0/3899608036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0e30 msgr2=0x561bf24b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.714+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0e30 0x561bf24b4c10 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7c34007d70 tx=0x7f7c34007950 comp rx=0 tx=0).stop 2023-07-14T16:30:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 1 -- 172.21.15.62:0/3899608036 shutdown_connections 2023-07-14T16:30:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561bf24b9430 0x561bf24bb840 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0e30 0x561bf24b4c10 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 0x561bf24ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c1bfff700 0 client.19803 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:30:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 1 --2- 172.21.15.62:0/3899608036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 0x561bf24adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 1 -- 172.21.15.62:0/3899608036 >> 172.21.15.62:0/3899608036 conn(0x561bf2494ff0 msgr2=0x561bf2499b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 1 -- 172.21.15.62:0/3899608036 shutdown_connections 2023-07-14T16:30:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 1 -- 172.21.15.62:0/3899608036 wait complete. 2023-07-14T16:30:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.715+0000 7f7c569abb80 20 client.19803 trim_cache size 0 max 16384 2023-07-14T16:30:54.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.731+0000 7f7c569abb80 1 Processor -- start 2023-07-14T16:30:54.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.731+0000 7f7c569abb80 1 -- start start 2023-07-14T16:30:54.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 0x561bf249faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0e30 0x561bf24b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c569abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf24ab800 con 0x561bf24a0070 2023-07-14T16:30:54.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c569abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf24b9180 con 0x561bf249f720 2023-07-14T16:30:54.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c569abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf24a15c0 con 0x561bf24a0e30 2023-07-14T16:30:54.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44216/0 (socket says 172.21.15.62:44216) 2023-07-14T16:30:54.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c3bfff700 1 -- 172.21.15.62:0/2500663512 learned_addr learned my addr 172.21.15.62:0/2500663512 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c4159c700 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0e30 0x561bf24b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 0x561bf249faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c3bfff700 1 -- 172.21.15.62:0/2500663512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0e30 msgr2=0x561bf24b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0e30 0x561bf24b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c3bfff700 1 -- 172.21.15.62:0/2500663512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 msgr2=0x561bf249faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.732+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 0x561bf249faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.733+0000 7f7c3bfff700 1 -- 172.21.15.62:0/2500663512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bf2245250 con 0x561bf24a0070 2023-07-14T16:30:54.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.733+0000 7f7c4159c700 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0e30 0x561bf24b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:54.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.733+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24b4bd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7c280074a0 tx=0x7f7c28099600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f5bb1445c4fb7b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.733+0000 7f7c277fe700 1 -- 172.21.15.62:0/2500663512 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280b6070 con 0x561bf24a0070 2023-07-14T16:30:54.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.733+0000 7f7c277fe700 1 -- 172.21.15.62:0/2500663512 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c28099c50 con 0x561bf24a0070 2023-07-14T16:30:54.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.733+0000 7f7c277fe700 1 -- 172.21.15.62:0/2500663512 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280b2250 con 0x561bf24a0070 2023-07-14T16:30:54.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 -- 172.21.15.62:0/2500663512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 msgr2=0x561bf24b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24b4bd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7c280074a0 tx=0x7f7c28099600 comp rx=0 tx=0).stop 2023-07-14T16:30:54.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 -- 172.21.15.62:0/2500663512 shutdown_connections 2023-07-14T16:30:54.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0e30 0x561bf24b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf249f720 0x561bf249faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 --2- 172.21.15.62:0/2500663512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf24a0070 0x561bf24b4bd0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 -- 172.21.15.62:0/2500663512 >> 172.21.15.62:0/2500663512 conn(0x561bf2494ff0 msgr2=0x561bf24c01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 -- 172.21.15.62:0/2500663512 shutdown_connections 2023-07-14T16:30:54.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 -- 172.21.15.62:0/2500663512 wait complete. 2023-07-14T16:30:54.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 -- start start 2023-07-14T16:30:54.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.734+0000 7f7c569abb80 1 Processor -- start 2023-07-14T16:30:54.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf249f720 0x561bf24b4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 0x561bf24b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c569abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 0x561bf24b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c569abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf24b4fa0 con 0x561bf249f720 2023-07-14T16:30:54.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c569abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf24b1ca0 con 0x561bf24a0e30 2023-07-14T16:30:54.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c569abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf24b1e60 con 0x561bf24a0070 2023-07-14T16:30:54.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c569abb80 10 client.0 _ll_register_callbacks cb 0x561bf238e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:54.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c40d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf249f720 0x561bf24b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c40d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf249f720 0x561bf24b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44220/0 (socket says 172.21.15.62:44220) 2023-07-14T16:30:54.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c40d9b700 1 -- 172.21.15.62:0/327472809 learned_addr learned my addr 172.21.15.62:0/327472809 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.735+0000 7f7c4159c700 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 0x561bf24b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 0x561bf24b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c40d9b700 1 -- 172.21.15.62:0/327472809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 msgr2=0x561bf24b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 0x561bf24b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c40d9b700 1 -- 172.21.15.62:0/327472809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 msgr2=0x561bf24b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 0x561bf24b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c40d9b700 1 -- 172.21.15.62:0/327472809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bf2207c60 con 0x561bf249f720 2023-07-14T16:30:54.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c40d9b700 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf249f720 0x561bf24b4780 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b5c0 tx=0x7f7c3000b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6176ae3feb6ce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:54.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c1bfff700 1 -- 172.21.15.62:0/327472809 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30032070 con 0x561bf249f720 2023-07-14T16:30:54.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c569abb80 1 -- 172.21.15.62:0/327472809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c3001e040 con 0x561bf249f720 2023-07-14T16:30:54.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.736+0000 7f7c1bfff700 1 -- 172.21.15.62:0/327472809 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7c30034520 con 0x561bf249f720 2023-07-14T16:30:54.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c1bfff700 1 -- 172.21.15.62:0/327472809 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c3001d040 con 0x561bf249f720 2023-07-14T16:30:54.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c569abb80 10 client.19809 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:54.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c569abb80 1 -- 172.21.15.62:0/327472809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f7c3c19dd70 con 0x561bf249f720 2023-07-14T16:30:54.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c1bfff700 1 -- 172.21.15.62:0/327472809 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c3001e040 con 0x561bf249f720 2023-07-14T16:30:54.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c1bfff700 1 -- 172.21.15.62:0/327472809 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7c30034860 con 0x561bf249f720 2023-07-14T16:30:54.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c569abb80 10 client.19809 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c569abb80 10 client.19809 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:30:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c569abb80 20 client.19809 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c569abb80 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561bf250e720 0x561bf2510b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c569abb80 4 client.19809 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:30:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.737+0000 7f7c569abb80 1 -- 172.21.15.62:0/327472809 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561bf23ac050 con 0x561bf250e720 2023-07-14T16:30:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.738+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561bf250e720 0x561bf2510b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.738+0000 7f7c3bfff700 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561bf250e720 0x561bf2510b00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4057a2a1f18eefc7 server_cookie=9e762c5446da7c1 in_seq=0 out_seq=0 2023-07-14T16:30:54.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.738+0000 7f7c1bfff700 10 client.19809 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:30:54.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c1bfff700 1 -- 172.21.15.62:0/327472809 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561bf23ac050 con 0x561bf250e720 2023-07-14T16:30:54.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c1bfff700 10 client.19809 handle_command_reply: tid=0 2023-07-14T16:30:54.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 client.19809 shutdown 2023-07-14T16:30:54.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 -- 172.21.15.62:0/327472809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf249f720 msgr2=0x561bf24b4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf249f720 0x561bf24b4780 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b5c0 tx=0x7f7c3000b6f0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 -- 172.21.15.62:0/327472809 shutdown_connections 2023-07-14T16:30:54.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561bf250e720 0x561bf2510b00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bf24a0e30 0x561bf24b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bf249f720 0x561bf24b4780 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c1bfff700 0 client.19809 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:30:54.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 --2- 172.21.15.62:0/327472809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561bf24a0070 0x561bf24b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 -- 172.21.15.62:0/327472809 >> 172.21.15.62:0/327472809 conn(0x561bf2494ff0 msgr2=0x561bf24ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 -- 172.21.15.62:0/327472809 shutdown_connections 2023-07-14T16:30:54.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 1 -- 172.21.15.62:0/327472809 wait complete. 2023-07-14T16:30:54.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.739+0000 7f7c569abb80 20 client.19809 trim_cache size 0 max 16384 2023-07-14T16:30:54.721 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:30:54.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 -- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7c2c0a1150 msgr2=0x7f7c2c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7c2c0a1150 0x7f7c2c0a3600 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c34007740 tx=0x7f7c34014390 comp rx=0 tx=0).stop 2023-07-14T16:30:54.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 -- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c100ab0 msgr2=0x7f7c3c195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c100ab0 0x7f7c3c195ac0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c9b0 tx=0x7f7c2800ccc0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 -- 172.21.15.62:0/1089797719 shutdown_connections 2023-07-14T16:30:54.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7c2c0a1150 0x7f7c2c0a3600 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c3c101440 0x7f7c3c196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0ff8c0 0x7f7c3c195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 --2- 172.21.15.62:0/1089797719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c3c100ab0 0x7f7c3c195ac0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 -- 172.21.15.62:0/1089797719 >> 172.21.15.62:0/1089797719 conn(0x7f7c3c0faea0 msgr2=0x7f7c3c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 -- 172.21.15.62:0/1089797719 shutdown_connections 2023-07-14T16:30:54.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:54.740+0000 7f7c41d9d700 1 -- 172.21.15.62:0/1089797719 wait complete. 2023-07-14T16:30:54.734 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:30:54.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.007+0000 7f7498b9d700 1 -- 172.21.15.62:0/4235556180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 msgr2=0x7f7494102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.007+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4235556180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 0x7f7494102830 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f7484009990 tx=0x7f7484009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.007+0000 7f7498b9d700 1 -- 172.21.15.62:0/4235556180 shutdown_connections 2023-07-14T16:30:54.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.007+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4235556180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7494105690 0x7f7494107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.007+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4235556180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 0x7f7494102830 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.007+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4235556180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494102d70 0x7f7494105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.007+0000 7f7498b9d700 1 -- 172.21.15.62:0/4235556180 >> 172.21.15.62:0/4235556180 conn(0x7f74940f9ed0 msgr2=0x7f74940fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.008+0000 7f7498b9d700 1 -- 172.21.15.62:0/4235556180 shutdown_connections 2023-07-14T16:30:54.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.008+0000 7f7498b9d700 1 -- 172.21.15.62:0/4235556180 wait complete. 2023-07-14T16:30:54.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.009+0000 7f7498b9d700 1 Processor -- start 2023-07-14T16:30:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.009+0000 7f7498b9d700 1 -- start start 2023-07-14T16:30:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.009+0000 7f7498b9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 0x7f74941976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f7498b9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494102d70 0x7f7494197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f7498b9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7494105690 0x7f749419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f7498b9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749419f230 con 0x7f7494100450 2023-07-14T16:30:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f7498b9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749419f370 con 0x7f7494102d70 2023-07-14T16:30:54.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f7498b9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749419f500 con 0x7f7494105690 2023-07-14T16:30:54.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f74937fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 0x7f74941976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f74937fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 0x7f74941976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44234/0 (socket says 172.21.15.62:44234) 2023-07-14T16:30:54.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f74937fe700 1 -- 172.21.15.62:0/4239946622 learned_addr learned my addr 172.21.15.62:0/4239946622 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f7492ffd700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494102d70 0x7f7494197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f7493fff700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7494105690 0x7f749419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f74937fe700 1 -- 172.21.15.62:0/4239946622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7494105690 msgr2=0x7f749419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f74937fe700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7494105690 0x7f749419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f74937fe700 1 -- 172.21.15.62:0/4239946622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494102d70 msgr2=0x7f7494197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f74937fe700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494102d70 0x7f7494197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.010+0000 7f74937fe700 1 -- 172.21.15.62:0/4239946622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840095f0 con 0x7f7494100450 2023-07-14T16:30:54.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.011+0000 7f74937fe700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 0x7f74941976b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7484005910 tx=0x7f7484005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.011+0000 7f7490ff9700 1 -- 172.21.15.62:0/4239946622 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748401d070 con 0x7f7494100450 2023-07-14T16:30:54.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.011+0000 7f7490ff9700 1 -- 172.21.15.62:0/4239946622 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7484022470 con 0x7f7494100450 2023-07-14T16:30:54.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.011+0000 7f7490ff9700 1 -- 172.21.15.62:0/4239946622 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400f6c0 con 0x7f7494100450 2023-07-14T16:30:54.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.011+0000 7f7498b9d700 1 -- 172.21.15.62:0/4239946622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749419f780 con 0x7f7494100450 2023-07-14T16:30:54.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.011+0000 7f7498b9d700 1 -- 172.21.15.62:0/4239946622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749419fb50 con 0x7f7494100450 2023-07-14T16:30:54.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.015+0000 7f7490ff9700 1 -- 172.21.15.62:0/4239946622 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f74840225e0 con 0x7f7494100450 2023-07-14T16:30:54.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.016+0000 7f74ad6f8b80 1 Processor -- start 2023-07-14T16:30:54.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f74ad6f8b80 1 -- start start 2023-07-14T16:30:54.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x565552848b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x5655528498a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x56555284a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56555274dcb0 con 0x565552848720 2023-07-14T16:30:54.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655527a0a40 con 0x565552849e30 2023-07-14T16:30:54.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x565552799ed0 con 0x565552849070 2023-07-14T16:30:54.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f7490ff9700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f747c0a1270 0x7f747c0a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:54.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f7493fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x565552848b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f7493fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x565552848b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44238/0 (socket says 172.21.15.62:44238) 2023-07-14T16:30:54.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.017+0000 7f7493fff700 1 -- 172.21.15.62:0/3560544722 learned_addr learned my addr 172.21.15.62:0/3560544722 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:54.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7492ffd700 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x56555284a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7490ff9700 1 -- 172.21.15.62:0/4239946622 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f74840cddb0 con 0x7f7494100450 2023-07-14T16:30:54.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f74937fe700 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x5655528498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7492ffd700 1 -- 172.21.15.62:0/3560544722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 msgr2=0x5655528498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7492ffd700 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x5655528498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7492ffd700 1 -- 172.21.15.62:0/3560544722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 msgr2=0x565552848b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:54.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7492ffd700 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x565552848b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7492ffd700 1 -- 172.21.15.62:0/3560544722 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5655525c7de0 con 0x565552849e30 2023-07-14T16:30:54.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7493fff700 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x565552848b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:54.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7493fff700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f747c0a1270 0x7f747c0a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:54.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.018+0000 7f7492ffd700 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x56555284a2a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7488009cb0 tx=0x7f7488011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c30fe17982fbb232 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.019+0000 7f7479ffb700 1 -- 172.21.15.62:0/3560544722 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488011d20 con 0x565552849e30 2023-07-14T16:30:54.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.019+0000 7f7479ffb700 1 -- 172.21.15.62:0/3560544722 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7488011e80 con 0x565552849e30 2023-07-14T16:30:54.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.019+0000 7f7479ffb700 1 -- 172.21.15.62:0/3560544722 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488005280 con 0x565552849e30 2023-07-14T16:30:54.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3560544722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 msgr2=0x56555284a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:54.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f7493fff700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f747c0a1270 0x7f747c0a3720 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f748c007b20 tx=0x7f748c004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:54.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x56555284a2a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7488009cb0 tx=0x7f7488011440 comp rx=0 tx=0).stop 2023-07-14T16:30:54.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3560544722 shutdown_connections 2023-07-14T16:30:54.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x56555284a2a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x565552848b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3560544722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x5655528498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:54.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3560544722 >> 172.21.15.62:0/3560544722 conn(0x56555283dff0 msgr2=0x565552843460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:54.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3560544722 shutdown_connections 2023-07-14T16:30:54.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3560544722 wait complete. 2023-07-14T16:30:55.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.020+0000 7f74ad6f8b80 1 -- start start 2023-07-14T16:30:55.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.021+0000 7f74ad6f8b80 1 Processor -- start 2023-07-14T16:30:55.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.021+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x565552852310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.021+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552852850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.021+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 0x565552859890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.021+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x565552739ac0 con 0x565552849070 2023-07-14T16:30:55.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.021+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655527397c0 con 0x565552848720 2023-07-14T16:30:55.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.021+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56555261d920 con 0x565552849e30 2023-07-14T16:30:55.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.021+0000 7f74ad6f8b80 10 client.0 _ll_register_callbacks cb 0x565552737760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:55.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7492ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552852850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f74937fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x565552852310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7493fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 0x565552859890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7492ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552852850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44254/0 (socket says 172.21.15.62:44254) 2023-07-14T16:30:55.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7492ffd700 1 -- 172.21.15.62:0/3890794632 learned_addr learned my addr 172.21.15.62:0/3890794632 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:55.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7492ffd700 1 -- 172.21.15.62:0/3890794632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 msgr2=0x565552859890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:55.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7492ffd700 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 0x565552859890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7492ffd700 1 -- 172.21.15.62:0/3890794632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 msgr2=0x565552852310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7492ffd700 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x565552852310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f7492ffd700 1 -- 172.21.15.62:0/3890794632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c009b30 con 0x565552849070 2023-07-14T16:30:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.022+0000 7f74937fe700 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x565552852310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.023+0000 7f7492ffd700 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552852850 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7488005c50 tx=0x7f7488005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12436327c33875f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.023+0000 7f7472ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.023+0000 7f7472ffd700 1 -- 172.21.15.62:0/3890794632 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488004260 con 0x565552849070 2023-07-14T16:30:55.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.023+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3890794632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565552792eb0 con 0x565552849070 2023-07-14T16:30:55.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.023+0000 7f7472ffd700 1 -- 172.21.15.62:0/3890794632 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f74880043c0 con 0x565552849070 2023-07-14T16:30:55.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.023+0000 7f7472ffd700 1 -- 172.21.15.62:0/3890794632 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7488003680 con 0x565552849070 2023-07-14T16:30:55.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.023+0000 7f74ad6f8b80 10 client.19821 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:55.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.023+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3890794632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f74880098e0 con 0x565552849070 2023-07-14T16:30:55.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.024+0000 7f7472ffd700 1 -- 172.21.15.62:0/3890794632 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565552792eb0 con 0x565552849070 2023-07-14T16:30:55.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.024+0000 7f7472ffd700 1 -- 172.21.15.62:0/3890794632 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7488005150 con 0x565552849070 2023-07-14T16:30:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.024+0000 7f74ad6f8b80 10 client.19821 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.024+0000 7f74ad6f8b80 10 client.19821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.024+0000 7f74ad6f8b80 20 client.19821 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.024+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56555285de70 0x565552860280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.024+0000 7f74ad6f8b80 4 client.19821 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.024+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3890794632 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565552755050 con 0x56555285de70 2023-07-14T16:30:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.025+0000 7f74937fe700 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56555285de70 0x565552860280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.025+0000 7f7472ffd700 10 client.19821 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:55.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.025+0000 7f74937fe700 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56555285de70 0x565552860280 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bea698485c2b2ae server_cookie=d649d6a23a309ec5 in_seq=0 out_seq=0 2023-07-14T16:30:55.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f7472ffd700 1 -- 172.21.15.62:0/3890794632 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565552755050 con 0x56555285de70 2023-07-14T16:30:55.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f7472ffd700 10 client.19821 handle_command_reply: tid=0 2023-07-14T16:30:55.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 client.19821 shutdown 2023-07-14T16:30:55.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3890794632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 msgr2=0x565552852850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:55.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552852850 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7488005c50 tx=0x7f7488005d60 comp rx=0 tx=0).stop 2023-07-14T16:30:55.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3890794632 shutdown_connections 2023-07-14T16:30:55.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 0x565552859890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56555285de70 0x565552860280 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x565552852310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/3890794632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552852850 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f7472ffd700 0 client.19821 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:55.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3890794632 >> 172.21.15.62:0/3890794632 conn(0x56555283dff0 msgr2=0x565552842b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:55.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.026+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3890794632 shutdown_connections 2023-07-14T16:30:55.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.027+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/3890794632 wait complete. 2023-07-14T16:30:55.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.027+0000 7f74ad6f8b80 20 client.19821 trim_cache size 0 max 16384 2023-07-14T16:30:55.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.045+0000 7f74ad6f8b80 1 Processor -- start 2023-07-14T16:30:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.045+0000 7f74ad6f8b80 1 -- start start 2023-07-14T16:30:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x56555285a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x565552859820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x5655528522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56555284ff70 con 0x565552848720 2023-07-14T16:30:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x565552849690 con 0x565552849e30 2023-07-14T16:30:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56555285db90 con 0x565552849070 2023-07-14T16:30:55.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74937fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x56555285a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74937fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x56555285a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44260/0 (socket says 172.21.15.62:44260) 2023-07-14T16:30:55.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74937fe700 1 -- 172.21.15.62:0/992616691 learned_addr learned my addr 172.21.15.62:0/992616691 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:55.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f7492ffd700 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x565552859820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f7493fff700 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x5655528522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.046+0000 7f74937fe700 1 -- 172.21.15.62:0/992616691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 msgr2=0x565552859820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:55.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f74937fe700 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x565552859820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f74937fe700 1 -- 172.21.15.62:0/992616691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 msgr2=0x5655528522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:55.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f74937fe700 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x5655528522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f74937fe700 1 -- 172.21.15.62:0/992616691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5655525b0c60 con 0x565552848720 2023-07-14T16:30:55.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f7492ffd700 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x565552859820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:55.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f74937fe700 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x56555285a180 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f74840cb910 tx=0x7f74840a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0462d3bdd718959 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:55.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f747a7fc700 1 -- 172.21.15.62:0/992616691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748409b260 con 0x565552848720 2023-07-14T16:30:55.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f747a7fc700 1 -- 172.21.15.62:0/992616691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f748409f290 con 0x565552848720 2023-07-14T16:30:55.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.047+0000 7f747a7fc700 1 -- 172.21.15.62:0/992616691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484099430 con 0x565552848720 2023-07-14T16:30:55.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/992616691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 msgr2=0x56555285a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:55.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x56555285a180 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f74840cb910 tx=0x7f74840a1040 comp rx=0 tx=0).stop 2023-07-14T16:30:55.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/992616691 shutdown_connections 2023-07-14T16:30:55.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552849e30 0x5655528522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552848720 0x56555285a180 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/992616691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849070 0x565552859820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/992616691 >> 172.21.15.62:0/992616691 conn(0x56555283dff0 msgr2=0x565552851010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:55.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/992616691 shutdown_connections 2023-07-14T16:30:55.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.048+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/992616691 wait complete. 2023-07-14T16:30:55.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.049+0000 7f74ad6f8b80 1 -- start start 2023-07-14T16:30:55.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.049+0000 7f74ad6f8b80 1 Processor -- start 2023-07-14T16:30:55.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x5655528558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552855e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f74ad6f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 0x565552858e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x565552859360 con 0x565552849070 2023-07-14T16:30:55.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655528594d0 con 0x565552848720 2023-07-14T16:30:55.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f74ad6f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x565552859690 con 0x565552849e30 2023-07-14T16:30:55.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f74ad6f8b80 10 client.0 _ll_register_callbacks cb 0x7f748c009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:55.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f7492ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552855e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f7492ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552855e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44270/0 (socket says 172.21.15.62:44270) 2023-07-14T16:30:55.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f7492ffd700 1 -- 172.21.15.62:0/2234989803 learned_addr learned my addr 172.21.15.62:0/2234989803 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:55.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f74937fe700 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x5655528558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f7492ffd700 1 -- 172.21.15.62:0/2234989803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 msgr2=0x565552858e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:55.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f7492ffd700 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 0x565552858e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f7492ffd700 1 -- 172.21.15.62:0/2234989803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 msgr2=0x5655528558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:55.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f7492ffd700 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x5655528558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.050+0000 7f7492ffd700 1 -- 172.21.15.62:0/2234989803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7484095e70 con 0x565552849070 2023-07-14T16:30:55.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.051+0000 7f7492ffd700 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552855e00 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7488011e90 tx=0x7f748802b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f94343fb7cabae server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:55.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.051+0000 7f7472ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:55.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.051+0000 7f7472ffd700 1 -- 172.21.15.62:0/2234989803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488004ab0 con 0x565552849070 2023-07-14T16:30:55.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.051+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/2234989803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5655526f34f0 con 0x565552849070 2023-07-14T16:30:55.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.051+0000 7f7472ffd700 1 -- 172.21.15.62:0/2234989803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7488000e40 con 0x565552849070 2023-07-14T16:30:55.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.051+0000 7f7472ffd700 1 -- 172.21.15.62:0/2234989803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7488004470 con 0x565552849070 2023-07-14T16:30:55.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.051+0000 7f74ad6f8b80 10 client.19827 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:55.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.051+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/2234989803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5655525ee250 con 0x565552849070 2023-07-14T16:30:55.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f7472ffd700 1 -- 172.21.15.62:0/2234989803 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5655526f34f0 con 0x565552849070 2023-07-14T16:30:55.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f7472ffd700 1 -- 172.21.15.62:0/2234989803 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f748802bb40 con 0x565552849070 2023-07-14T16:30:55.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f74ad6f8b80 10 client.19827 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:55.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f74ad6f8b80 10 client.19827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:55.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f74ad6f8b80 20 client.19827 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:55.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5655528b2ca0 0x5655528b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f74ad6f8b80 4 client.19827 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:30:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/2234989803 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5655525e8f70 con 0x5655528b2ca0 2023-07-14T16:30:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.052+0000 7f74937fe700 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5655528b2ca0 0x5655528b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f7472ffd700 10 client.19827 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74937fe700 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5655528b2ca0 0x5655528b50b0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f22b73523ca6c17 server_cookie=6bb84a667159abde in_seq=0 out_seq=0 2023-07-14T16:30:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f7472ffd700 1 -- 172.21.15.62:0/2234989803 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5655525e8f70 con 0x5655528b2ca0 2023-07-14T16:30:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f7472ffd700 10 client.19827 handle_command_reply: tid=0 2023-07-14T16:30:55.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 client.19827 shutdown 2023-07-14T16:30:55.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/2234989803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 msgr2=0x565552855e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:55.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552855e00 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7488011e90 tx=0x7f748802b780 comp rx=0 tx=0).stop 2023-07-14T16:30:55.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/2234989803 shutdown_connections 2023-07-14T16:30:55.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565552849e30 0x565552858e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5655528b2ca0 0x5655528b50b0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x565552848720 0x5655528558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 --2- 172.21.15.62:0/2234989803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565552849070 0x565552855e00 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f7472ffd700 0 client.19827 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:55.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/2234989803 >> 172.21.15.62:0/2234989803 conn(0x56555283dff0 msgr2=0x56555285a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:55.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.053+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/2234989803 shutdown_connections 2023-07-14T16:30:55.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f74ad6f8b80 1 -- 172.21.15.62:0/2234989803 wait complete. 2023-07-14T16:30:55.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f74ad6f8b80 20 client.19827 trim_cache size 0 max 16384 2023-07-14T16:30:55.035 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:30:55.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 -- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f747c0a1270 msgr2=0x7f747c0a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:55.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f747c0a1270 0x7f747c0a3720 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f748c007b20 tx=0x7f748c004b40 comp rx=0 tx=0).stop 2023-07-14T16:30:55.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 -- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 msgr2=0x7f74941976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:55.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 0x7f74941976b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7484005910 tx=0x7f7484005720 comp rx=0 tx=0).stop 2023-07-14T16:30:55.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 -- 172.21.15.62:0/4239946622 shutdown_connections 2023-07-14T16:30:55.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7494105690 0x7f749419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7494100450 0x7f74941976b0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f747c0a1270 0x7f747c0a3720 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 --2- 172.21.15.62:0/4239946622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494102d70 0x7f7494197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:55.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.054+0000 7f7498b9d700 1 -- 172.21.15.62:0/4239946622 >> 172.21.15.62:0/4239946622 conn(0x7f74940f9ed0 msgr2=0x7f74940fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:55.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.055+0000 7f7498b9d700 1 -- 172.21.15.62:0/4239946622 shutdown_connections 2023-07-14T16:30:55.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:55.055+0000 7f7498b9d700 1 -- 172.21.15.62:0/4239946622 wait complete. 2023-07-14T16:30:55.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:54 smithi062 ceph-mon[95543]: pgmap v815: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:30:55.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:54 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:30:55.047 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:54 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2642114127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:30:55.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:54 smithi081 ceph-mon[127795]: pgmap v815: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:30:55.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:54 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:30:55.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:54 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2642114127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:30:55.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:54 smithi164 ceph-mon[97427]: pgmap v815: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:30:55.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:54 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:30:55.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:54 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2642114127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:30:56.049 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:30:56.050 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:30:56.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 -- 172.21.15.62:0/1982786568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80ffb50 msgr2=0x7f10c80fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.276 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 --2- 172.21.15.62:0/1982786568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80ffb50 0x7f10c80fff60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009990 tx=0x7f10c4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 -- 172.21.15.62:0/1982786568 shutdown_connections 2023-07-14T16:30:56.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 --2- 172.21.15.62:0/1982786568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c81016d0 0x7f10c8107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 --2- 172.21.15.62:0/1982786568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c80ffb50 0x7f10c80fff60 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 --2- 172.21.15.62:0/1982786568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c8100d40 0x7f10c8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 -- 172.21.15.62:0/1982786568 >> 172.21.15.62:0/1982786568 conn(0x7f10c80fb170 msgr2=0x7f10c80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 -- 172.21.15.62:0/1982786568 shutdown_connections 2023-07-14T16:30:56.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.302+0000 7f10d01ff700 1 -- 172.21.15.62:0/1982786568 wait complete. 2023-07-14T16:30:56.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.303+0000 7f10d01ff700 1 Processor -- start 2023-07-14T16:30:56.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.304+0000 7f10d01ff700 1 -- start start 2023-07-14T16:30:56.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.304+0000 7f10d01ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c80ffb50 0x7f10c8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.304+0000 7f10d01ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c8100d40 0x7f10c8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.304+0000 7f10d01ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c81016d0 0x7f10c819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.304+0000 7f10d01ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c819f4e0 con 0x7f10c81016d0 2023-07-14T16:30:56.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.304+0000 7f10d01ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c819f620 con 0x7f10c8100d40 2023-07-14T16:30:56.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.304+0000 7f10d01ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c819f7b0 con 0x7f10c80ffb50 2023-07-14T16:30:56.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.304+0000 7f10ce79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c81016d0 0x7f10c819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10cdf9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c80ffb50 0x7f10c8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10cd79a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c8100d40 0x7f10c8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10ce79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c81016d0 0x7f10c819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44276/0 (socket says 172.21.15.62:44276) 2023-07-14T16:30:56.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10ce79c700 1 -- 172.21.15.62:0/528207137 learned_addr learned my addr 172.21.15.62:0/528207137 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10cdf9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c80ffb50 0x7f10c8197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60054/0 (socket says 172.21.15.62:60054) 2023-07-14T16:30:56.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10ce79c700 1 -- 172.21.15.62:0/528207137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c80ffb50 msgr2=0x7f10c8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10ce79c700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c80ffb50 0x7f10c8197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10ce79c700 1 -- 172.21.15.62:0/528207137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c8100d40 msgr2=0x7f10c8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10ce79c700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c8100d40 0x7f10c8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10ce79c700 1 -- 172.21.15.62:0/528207137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40095f0 con 0x7f10c81016d0 2023-07-14T16:30:56.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.305+0000 7f10ce79c700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c81016d0 0x7f10c819ef10 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f10c0007ee0 tx=0x7f10c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.306+0000 7f10beffd700 1 -- 172.21.15.62:0/528207137 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0017070 con 0x7f10c81016d0 2023-07-14T16:30:56.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.306+0000 7f10beffd700 1 -- 172.21.15.62:0/528207137 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10c000f040 con 0x7f10c81016d0 2023-07-14T16:30:56.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.306+0000 7f10d01ff700 1 -- 172.21.15.62:0/528207137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c819fa90 con 0x7f10c81016d0 2023-07-14T16:30:56.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.306+0000 7f10beffd700 1 -- 172.21.15.62:0/528207137 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0005570 con 0x7f10c81016d0 2023-07-14T16:30:56.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.306+0000 7f10d01ff700 1 -- 172.21.15.62:0/528207137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c819fff0 con 0x7f10c81016d0 2023-07-14T16:30:56.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.309+0000 7f10beffd700 1 -- 172.21.15.62:0/528207137 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f10c000db20 con 0x7f10c81016d0 2023-07-14T16:30:56.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.310+0000 7f10e3b67b80 1 Processor -- start 2023-07-14T16:30:56.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10beffd700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b40a11d0 0x7f10b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10e3b67b80 1 -- start start 2023-07-14T16:30:56.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10beffd700 1 -- 172.21.15.62:0/528207137 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10c00cbff0 con 0x7f10c81016d0 2023-07-14T16:30:56.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10cd79a700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b40a11d0 0x7f10b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3ced7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3ced8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced8cf0 0x55ae3ced9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3cddcd30 con 0x55ae3ced7550 2023-07-14T16:30:56.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3ce2fac0 con 0x55ae3ced8cf0 2023-07-14T16:30:56.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.311+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3ce28f50 con 0x55ae3ced7ea0 2023-07-14T16:30:56.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cdf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3ced7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cdf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3ced7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44292/0 (socket says 172.21.15.62:44292) 2023-07-14T16:30:56.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cdf9b700 1 -- 172.21.15.62:0/2292199301 learned_addr learned my addr 172.21.15.62:0/2292199301 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cd79a700 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3ced8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10ce79c700 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced8cf0 0x55ae3ced9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cd79a700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b40a11d0 0x7f10b40a3680 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f10b8005950 tx=0x7f10b80106b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cd79a700 1 -- 172.21.15.62:0/2292199301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced8cf0 msgr2=0x55ae3ced9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cd79a700 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced8cf0 0x55ae3ced9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cd79a700 1 -- 172.21.15.62:0/2292199301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 msgr2=0x55ae3ced7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cd79a700 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3ced7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cd79a700 1 -- 172.21.15.62:0/2292199301 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae3cc56e60 con 0x55ae3ced7ea0 2023-07-14T16:30:56.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.312+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3ced7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:56.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.313+0000 7f10cd79a700 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3ced8760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f10b8010f90 tx=0x7f10b801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79d1c6dd8f6899ba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.292 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.313+0000 7f10b3fff700 1 -- 172.21.15.62:0/2292199301 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b80132d0 con 0x55ae3ced7ea0 2023-07-14T16:30:56.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.313+0000 7f10b3fff700 1 -- 172.21.15.62:0/2292199301 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10b8004cc0 con 0x55ae3ced7ea0 2023-07-14T16:30:56.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.313+0000 7f10b3fff700 1 -- 172.21.15.62:0/2292199301 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b80054c0 con 0x55ae3ced7ea0 2023-07-14T16:30:56.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2292199301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 msgr2=0x55ae3ced8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3ced8760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f10b8010f90 tx=0x7f10b801b000 comp rx=0 tx=0).stop 2023-07-14T16:30:56.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2292199301 shutdown_connections 2023-07-14T16:30:56.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced8cf0 0x55ae3ced9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3ced7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2292199301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3ced8760 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2292199301 >> 172.21.15.62:0/2292199301 conn(0x55ae3cecce20 msgr2=0x55ae3ced2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2292199301 shutdown_connections 2023-07-14T16:30:56.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.314+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2292199301 wait complete. 2023-07-14T16:30:56.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.315+0000 7f10e3b67b80 1 -- start start 2023-07-14T16:30:56.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.315+0000 7f10e3b67b80 1 Processor -- start 2023-07-14T16:30:56.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.315+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3cee1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.315+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced7ea0 0x55ae3cee1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.315+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3cdc8b40 con 0x55ae3ced7550 2023-07-14T16:30:56.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3cdc8840 con 0x55ae3ced7ea0 2023-07-14T16:30:56.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3ccac9a0 con 0x55ae3ced8cf0 2023-07-14T16:30:56.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10e3b67b80 10 client.0 _ll_register_callbacks cb 0x55ae3cdc67e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:56.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cdf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3cee1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cdf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3cee1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44302/0 (socket says 172.21.15.62:44302) 2023-07-14T16:30:56.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cdf9b700 1 -- 172.21.15.62:0/2743732053 learned_addr learned my addr 172.21.15.62:0/2743732053 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cd79a700 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced7ea0 0x55ae3cee1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10ce79c700 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cdf9b700 1 -- 172.21.15.62:0/2743732053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 msgr2=0x55ae3cee87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cdf9b700 1 -- 172.21.15.62:0/2743732053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced7ea0 msgr2=0x55ae3cee1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced7ea0 0x55ae3cee1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10cdf9b700 1 -- 172.21.15.62:0/2743732053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c81088c0 con 0x55ae3ced7550 2023-07-14T16:30:56.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.316+0000 7f10ce79c700 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:30:56.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.317+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3cee1250 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f10c4000c00 tx=0x7f10c4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacbfe77223225f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.317+0000 7f10b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:56.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.317+0000 7f10b17fa700 1 -- 172.21.15.62:0/2743732053 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c401d070 con 0x55ae3ced7550 2023-07-14T16:30:56.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.317+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2743732053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae3cd82570 con 0x55ae3ced7550 2023-07-14T16:30:56.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.317+0000 7f10b17fa700 1 -- 172.21.15.62:0/2743732053 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10c40056a0 con 0x55ae3ced7550 2023-07-14T16:30:56.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.317+0000 7f10b17fa700 1 -- 172.21.15.62:0/2743732053 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10c4009ee0 con 0x55ae3ced7550 2023-07-14T16:30:56.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.317+0000 7f10e3b67b80 10 client.19839 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:56.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.317+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2743732053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f10b8010cc0 con 0x55ae3ced7550 2023-07-14T16:30:56.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.318+0000 7f10b17fa700 1 -- 172.21.15.62:0/2743732053 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ae3cd82570 con 0x55ae3ced7550 2023-07-14T16:30:56.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.318+0000 7f10b17fa700 1 -- 172.21.15.62:0/2743732053 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f10c40039e0 con 0x55ae3ced7550 2023-07-14T16:30:56.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.318+0000 7f10e3b67b80 10 client.19839 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:56.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.318+0000 7f10e3b67b80 10 client.19839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:56.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.318+0000 7f10e3b67b80 20 client.19839 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:56.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.318+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ae3ceed020 0x55ae3ceef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.318+0000 7f10e3b67b80 4 client.19839 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:56.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.318+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2743732053 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae3ce21f30 con 0x55ae3ceed020 2023-07-14T16:30:56.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.319+0000 7f10cd79a700 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ae3ceed020 0x55ae3ceef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.319+0000 7f10b17fa700 10 client.19839 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:56.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.319+0000 7f10cd79a700 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ae3ceed020 0x55ae3ceef430 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=932a88894e2c36bf server_cookie=cc4396cf21b8e6d9 in_seq=0 out_seq=0 2023-07-14T16:30:56.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.320+0000 7f10b17fa700 1 -- 172.21.15.62:0/2743732053 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ae3ce21f30 con 0x55ae3ceed020 2023-07-14T16:30:56.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.320+0000 7f10b17fa700 10 client.19839 handle_command_reply: tid=0 2023-07-14T16:30:56.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.320+0000 7f10e3b67b80 1 client.19839 shutdown 2023-07-14T16:30:56.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.320+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2743732053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 msgr2=0x55ae3cee1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.320+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3cee1250 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f10c4000c00 tx=0x7f10c4005270 comp rx=0 tx=0).stop 2023-07-14T16:30:56.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2743732053 shutdown_connections 2023-07-14T16:30:56.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7550 0x55ae3cee1250 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ae3ceed020 0x55ae3ceef430 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/2743732053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3ced7ea0 0x55ae3cee1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2743732053 >> 172.21.15.62:0/2743732053 conn(0x55ae3cecce20 msgr2=0x55ae3ced1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10b17fa700 0 client.19839 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:56.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2743732053 shutdown_connections 2023-07-14T16:30:56.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 1 -- 172.21.15.62:0/2743732053 wait complete. 2023-07-14T16:30:56.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.321+0000 7f10e3b67b80 20 client.19839 trim_cache size 0 max 16384 2023-07-14T16:30:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10e3b67b80 1 Processor -- start 2023-07-14T16:30:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10e3b67b80 1 -- start start 2023-07-14T16:30:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7ea0 0x55ae3cedbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cedc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3cedf7d0 con 0x55ae3ced7ea0 2023-07-14T16:30:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3ced3d00 con 0x55ae3cee86f0 2023-07-14T16:30:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3ced2f50 con 0x55ae3ced8cf0 2023-07-14T16:30:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10cdf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7ea0 0x55ae3cedbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10cdf9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7ea0 0x55ae3cedbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44306/0 (socket says 172.21.15.62:44306) 2023-07-14T16:30:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.345+0000 7f10cdf9b700 1 -- 172.21.15.62:0/741806292 learned_addr learned my addr 172.21.15.62:0/741806292 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10cd79a700 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cedc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10cdf9b700 1 -- 172.21.15.62:0/741806292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 msgr2=0x55ae3cedc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cedc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10cdf9b700 1 -- 172.21.15.62:0/741806292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 msgr2=0x55ae3cee1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10ce79c700 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10cdf9b700 1 -- 172.21.15.62:0/741806292 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae3cc3fce0 con 0x55ae3ced7ea0 2023-07-14T16:30:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10ce79c700 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:30:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7ea0 0x55ae3cedbbf0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f10c4003e10 tx=0x7f10c4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae82d71b661fa66 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10bcff9700 1 -- 172.21.15.62:0/741806292 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c401d040 con 0x55ae3ced7ea0 2023-07-14T16:30:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.346+0000 7f10bcff9700 1 -- 172.21.15.62:0/741806292 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10c401e040 con 0x55ae3ced7ea0 2023-07-14T16:30:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10bcff9700 1 -- 172.21.15.62:0/741806292 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4004900 con 0x55ae3ced7ea0 2023-07-14T16:30:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10e3b67b80 1 -- 172.21.15.62:0/741806292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7ea0 msgr2=0x55ae3cedbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7ea0 0x55ae3cedbbf0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f10c4003e10 tx=0x7f10c4009310 comp rx=0 tx=0).stop 2023-07-14T16:30:56.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10e3b67b80 1 -- 172.21.15.62:0/741806292 shutdown_connections 2023-07-14T16:30:56.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced7ea0 0x55ae3cedbbf0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/741806292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced8cf0 0x55ae3cedc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10e3b67b80 1 -- 172.21.15.62:0/741806292 >> 172.21.15.62:0/741806292 conn(0x55ae3cecce20 msgr2=0x55ae3cedf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.347+0000 7f10e3b67b80 1 -- 172.21.15.62:0/741806292 shutdown_connections 2023-07-14T16:30:56.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.348+0000 7f10e3b67b80 1 -- 172.21.15.62:0/741806292 wait complete. 2023-07-14T16:30:56.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.348+0000 7f10e3b67b80 1 -- start start 2023-07-14T16:30:56.326 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.348+0000 7f10e3b67b80 1 Processor -- start 2023-07-14T16:30:56.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3cee53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10e3b67b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3cee6e60 con 0x55ae3ced8cf0 2023-07-14T16:30:56.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3cee6fd0 con 0x55ae3cee86f0 2023-07-14T16:30:56.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10e3b67b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae3cee7190 con 0x55ae3ced7ea0 2023-07-14T16:30:56.327 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10e3b67b80 10 client.0 _ll_register_callbacks cb 0x55ae3cdc67e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:56.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cd79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cd79a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44310/0 (socket says 172.21.15.62:44310) 2023-07-14T16:30:56.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cd79a700 1 -- 172.21.15.62:0/3783334168 learned_addr learned my addr 172.21.15.62:0/3783334168 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3cee53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10ce79c700 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cd79a700 1 -- 172.21.15.62:0/3783334168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 msgr2=0x55ae3cee53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cd79a700 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3cee53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.328 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cd79a700 1 -- 172.21.15.62:0/3783334168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 msgr2=0x55ae3cee6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cd79a700 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.349+0000 7f10cd79a700 1 -- 172.21.15.62:0/3783334168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b8010cc0 con 0x55ae3ced8cf0 2023-07-14T16:30:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.350+0000 7f10cd79a700 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee5910 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f10b8016010 tx=0x7f10b800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b095a39b9e03090 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.350+0000 7f10b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:30:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.350+0000 7f10b17fa700 1 -- 172.21.15.62:0/3783334168 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b802c040 con 0x55ae3ced8cf0 2023-07-14T16:30:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.350+0000 7f10b17fa700 1 -- 172.21.15.62:0/3783334168 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10b8019050 con 0x55ae3ced8cf0 2023-07-14T16:30:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.350+0000 7f10e3b67b80 1 -- 172.21.15.62:0/3783334168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10b8004cc0 con 0x55ae3ced8cf0 2023-07-14T16:30:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.350+0000 7f10b17fa700 1 -- 172.21.15.62:0/3783334168 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10b8041030 con 0x55ae3ced8cf0 2023-07-14T16:30:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.350+0000 7f10e3b67b80 10 client.19845 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.350+0000 7f10e3b67b80 1 -- 172.21.15.62:0/3783334168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ae3cc7d2d0 con 0x55ae3ced8cf0 2023-07-14T16:30:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10b17fa700 1 -- 172.21.15.62:0/3783334168 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10b8004cc0 con 0x55ae3ced8cf0 2023-07-14T16:30:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10b17fa700 1 -- 172.21.15.62:0/3783334168 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f10b8024030 con 0x55ae3ced8cf0 2023-07-14T16:30:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10e3b67b80 10 client.19845 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10e3b67b80 10 client.19845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10e3b67b80 20 client.19845 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ae3cf432a0 0x55ae3cf456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10e3b67b80 4 client.19845 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:30:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10e3b67b80 1 -- 172.21.15.62:0/3783334168 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ae3cde40d0 con 0x55ae3cf432a0 2023-07-14T16:30:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.351+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ae3cf432a0 0x55ae3cf456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.352+0000 7f10b17fa700 10 client.19845 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.352+0000 7f10cdf9b700 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ae3cf432a0 0x55ae3cf456b0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4454394cd9ac024c server_cookie=3a85174985641712 in_seq=0 out_seq=0 2023-07-14T16:30:56.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10b17fa700 1 -- 172.21.15.62:0/3783334168 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ae3cde40d0 con 0x55ae3cf432a0 2023-07-14T16:30:56.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10b17fa700 10 client.19845 handle_command_reply: tid=0 2023-07-14T16:30:56.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 client.19845 shutdown 2023-07-14T16:30:56.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 -- 172.21.15.62:0/3783334168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced8cf0 msgr2=0x55ae3cee5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee5910 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f10b8016010 tx=0x7f10b800b540 comp rx=0 tx=0).stop 2023-07-14T16:30:56.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 -- 172.21.15.62:0/3783334168 shutdown_connections 2023-07-14T16:30:56.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ae3cf432a0 0x55ae3cf456b0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ae3cee86f0 0x55ae3cee6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ae3ced7ea0 0x55ae3cee53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 --2- 172.21.15.62:0/3783334168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ae3ced8cf0 0x55ae3cee5910 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10b17fa700 0 client.19845 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:56.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.353+0000 7f10e3b67b80 1 -- 172.21.15.62:0/3783334168 >> 172.21.15.62:0/3783334168 conn(0x55ae3cecce20 msgr2=0x55ae3cedbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.354+0000 7f10e3b67b80 1 -- 172.21.15.62:0/3783334168 shutdown_connections 2023-07-14T16:30:56.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.354+0000 7f10e3b67b80 1 -- 172.21.15.62:0/3783334168 wait complete. 2023-07-14T16:30:56.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.354+0000 7f10e3b67b80 20 client.19845 trim_cache size 0 max 16384 2023-07-14T16:30:56.334 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:30:56.334 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:30:56.335 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "81367246-12e5-4eef-805a-b624998a3f7f", 2023-07-14T16:30:56.335 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:30:56.335 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:30:56.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.354+0000 7f10d01ff700 1 -- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b40a11d0 msgr2=0x7f10b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.354+0000 7f10d01ff700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b40a11d0 0x7f10b40a3680 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f10b8005950 tx=0x7f10b80106b0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.354+0000 7f10d01ff700 1 -- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c81016d0 msgr2=0x7f10c819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.354+0000 7f10d01ff700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c81016d0 0x7f10c819ef10 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f10c0007ee0 tx=0x7f10c000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.355+0000 7f10d01ff700 1 -- 172.21.15.62:0/528207137 shutdown_connections 2023-07-14T16:30:56.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.355+0000 7f10d01ff700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10c81016d0 0x7f10c819ef10 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.355+0000 7f10d01ff700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10c80ffb50 0x7f10c8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.355+0000 7f10d01ff700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10b40a11d0 0x7f10b40a3680 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.355+0000 7f10d01ff700 1 --2- 172.21.15.62:0/528207137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c8100d40 0x7f10c8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.355+0000 7f10d01ff700 1 -- 172.21.15.62:0/528207137 >> 172.21.15.62:0/528207137 conn(0x7f10c80fb170 msgr2=0x7f10c8106720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.355+0000 7f10d01ff700 1 -- 172.21.15.62:0/528207137 shutdown_connections 2023-07-14T16:30:56.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.355+0000 7f10d01ff700 1 -- 172.21.15.62:0/528207137 wait complete. 2023-07-14T16:30:56.352 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:30:56.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 -- 172.21.15.62:0/3866095670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141004b0 msgr2=0x7f7614102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 --2- 172.21.15.62:0/3866095670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141004b0 0x7f7614102890 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76100098f0 tx=0x7f7610009c00 comp rx=0 tx=0).stop 2023-07-14T16:30:56.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 -- 172.21.15.62:0/3866095670 shutdown_connections 2023-07-14T16:30:56.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 --2- 172.21.15.62:0/3866095670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76141056f0 0x7f7614107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 --2- 172.21.15.62:0/3866095670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141004b0 0x7f7614102890 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 --2- 172.21.15.62:0/3866095670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614102dd0 0x7f76141051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 -- 172.21.15.62:0/3866095670 >> 172.21.15.62:0/3866095670 conn(0x7f76140f9ed0 msgr2=0x7f76140fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 -- 172.21.15.62:0/3866095670 shutdown_connections 2023-07-14T16:30:56.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.600+0000 7f761a59d700 1 -- 172.21.15.62:0/3866095670 wait complete. 2023-07-14T16:30:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.601+0000 7f761a59d700 1 Processor -- start 2023-07-14T16:30:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.601+0000 7f761a59d700 1 -- start start 2023-07-14T16:30:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f761a59d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76141004b0 0x7f76140ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f761a59d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614102dd0 0x7f76140fe460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f761a59d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141056f0 0x7f76140fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f761a59d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76140ff140 con 0x7f7614102dd0 2023-07-14T16:30:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f761a59d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76140ff2b0 con 0x7f76141056f0 2023-07-14T16:30:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f761a59d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761419f590 con 0x7f76141004b0 2023-07-14T16:30:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f7618d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614102dd0 0x7f76140fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f7618d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614102dd0 0x7f76140fe460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44328/0 (socket says 172.21.15.62:44328) 2023-07-14T16:30:56.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f7618d9a700 1 -- 172.21.15.62:0/215514790 learned_addr learned my addr 172.21.15.62:0/215514790 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f761959b700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76141004b0 0x7f76140ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.602+0000 7f7619d9c700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141056f0 0x7f76140fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f7618d9a700 1 -- 172.21.15.62:0/215514790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76141004b0 msgr2=0x7f76140ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f7618d9a700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76141004b0 0x7f76140ffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f7618d9a700 1 -- 172.21.15.62:0/215514790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141056f0 msgr2=0x7f76140fe9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f7618d9a700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141056f0 0x7f76140fe9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f7618d9a700 1 -- 172.21.15.62:0/215514790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7610009520 con 0x7f7614102dd0 2023-07-14T16:30:56.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f7619d9c700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141056f0 0x7f76140fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:30:56.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f7618d9a700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614102dd0 0x7f76140fe460 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f760400e9c0 tx=0x7f760400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f760a7fc700 1 -- 172.21.15.62:0/215514790 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760400cbd0 con 0x7f7614102dd0 2023-07-14T16:30:56.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f760a7fc700 1 -- 172.21.15.62:0/215514790 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f760400eeb0 con 0x7f7614102dd0 2023-07-14T16:30:56.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f761a59d700 1 -- 172.21.15.62:0/215514790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761419f870 con 0x7f7614102dd0 2023-07-14T16:30:56.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.603+0000 7f760a7fc700 1 -- 172.21.15.62:0/215514790 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604005410 con 0x7f7614102dd0 2023-07-14T16:30:56.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.604+0000 7f761a59d700 1 -- 172.21.15.62:0/215514790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761419fdd0 con 0x7f7614102dd0 2023-07-14T16:30:56.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.607+0000 7f760a7fc700 1 -- 172.21.15.62:0/215514790 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f76040055b0 con 0x7f7614102dd0 2023-07-14T16:30:56.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.608+0000 7f762f17eb80 1 Processor -- start 2023-07-14T16:30:56.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.608+0000 7f760a7fc700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76000a1270 0x7f76000a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f762f17eb80 1 -- start start 2023-07-14T16:30:56.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f760a7fc700 1 -- 172.21.15.62:0/215514790 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7604014070 con 0x7f7614102dd0 2023-07-14T16:30:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2070 0x556fb84f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f7619d9c700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76000a1270 0x7f76000a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2e30 0x556fb84f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f762f17eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb83f6cb0 con 0x556fb84f2e30 2023-07-14T16:30:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f762f17eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb8449a40 con 0x556fb84f2070 2023-07-14T16:30:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.609+0000 7f762f17eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb8442ed0 con 0x556fb84f1720 2023-07-14T16:30:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f7619d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2e30 0x556fb84f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f761959b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f7619d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2e30 0x556fb84f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44330/0 (socket says 172.21.15.62:44330) 2023-07-14T16:30:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f7619d9c700 1 -- 172.21.15.62:0/31368602 learned_addr learned my addr 172.21.15.62:0/31368602 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f7618d9a700 1 --2- 172.21.15.62:0/31368602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2070 0x556fb84f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f761959b700 1 -- 172.21.15.62:0/31368602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2070 msgr2=0x556fb84f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f761959b700 1 --2- 172.21.15.62:0/31368602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2070 0x556fb84f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f761959b700 1 -- 172.21.15.62:0/31368602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2e30 msgr2=0x556fb84f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f761959b700 1 --2- 172.21.15.62:0/31368602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2e30 0x556fb84f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f761959b700 1 -- 172.21.15.62:0/31368602 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fb8270de0 con 0x556fb84f1720 2023-07-14T16:30:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f7619d9c700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76000a1270 0x7f76000a3720 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f760c0077d0 tx=0x7f760c007b90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.610+0000 7f761959b700 1 --2- 172.21.15.62:0/31368602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f1b30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7610009230 tx=0x7f7610005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3226de3c631e2c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.611+0000 7f75ff7fe700 1 -- 172.21.15.62:0/31368602 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761000fe60 con 0x556fb84f1720 2023-07-14T16:30:56.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.611+0000 7f75ff7fe700 1 -- 172.21.15.62:0/31368602 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f761001d040 con 0x556fb84f1720 2023-07-14T16:30:56.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.611+0000 7f75ff7fe700 1 -- 172.21.15.62:0/31368602 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76100052a0 con 0x556fb84f1720 2023-07-14T16:30:56.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 -- 172.21.15.62:0/31368602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 msgr2=0x556fb84f1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 --2- 172.21.15.62:0/31368602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f1b30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7610009230 tx=0x7f7610005dc0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 -- 172.21.15.62:0/31368602 shutdown_connections 2023-07-14T16:30:56.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 --2- 172.21.15.62:0/31368602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2e30 0x556fb84f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 --2- 172.21.15.62:0/31368602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f1b30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 --2- 172.21.15.62:0/31368602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2070 0x556fb84f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 -- 172.21.15.62:0/31368602 >> 172.21.15.62:0/31368602 conn(0x556fb84e6ff0 msgr2=0x556fb84ec460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 -- 172.21.15.62:0/31368602 shutdown_connections 2023-07-14T16:30:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 -- 172.21.15.62:0/31368602 wait complete. 2023-07-14T16:30:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.612+0000 7f762f17eb80 1 -- start start 2023-07-14T16:30:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.613+0000 7f762f17eb80 1 Processor -- start 2023-07-14T16:30:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f1720 0x556fb84fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f2070 0x556fb84fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb8502840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f762f17eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb83e2ac0 con 0x556fb84f1720 2023-07-14T16:30:56.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f762f17eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb83e27c0 con 0x556fb84f2e30 2023-07-14T16:30:56.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f762f17eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb82c6920 con 0x556fb84f2070 2023-07-14T16:30:56.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f762f17eb80 10 client.0 _ll_register_callbacks cb 0x556fb83e0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:56.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f761959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f1720 0x556fb84fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f7618d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f2070 0x556fb84fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f761959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f1720 0x556fb84fb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44338/0 (socket says 172.21.15.62:44338) 2023-07-14T16:30:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f761959b700 1 -- 172.21.15.62:0/1714011016 learned_addr learned my addr 172.21.15.62:0/1714011016 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f7619d9c700 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb8502840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f7618d9a700 1 -- 172.21.15.62:0/1714011016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 msgr2=0x556fb8502840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f7618d9a700 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb8502840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f7618d9a700 1 -- 172.21.15.62:0/1714011016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f1720 msgr2=0x556fb84fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.614+0000 7f7618d9a700 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f1720 0x556fb84fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f7618d9a700 1 -- 172.21.15.62:0/1714011016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76141086e0 con 0x556fb84f2070 2023-07-14T16:30:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f7618d9a700 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f2070 0x556fb84fb800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f760400c5b0 tx=0x7f7604099990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a96ad180a5d0b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f75fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:30:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f75fcff9700 1 -- 172.21.15.62:0/1714011016 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604099610 con 0x556fb84f2070 2023-07-14T16:30:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f75fcff9700 1 -- 172.21.15.62:0/1714011016 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f76040a2370 con 0x556fb84f2070 2023-07-14T16:30:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f762f17eb80 1 -- 172.21.15.62:0/1714011016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fb839c4f0 con 0x556fb84f2070 2023-07-14T16:30:56.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f75fcff9700 1 -- 172.21.15.62:0/1714011016 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76040b8250 con 0x556fb84f2070 2023-07-14T16:30:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f762f17eb80 10 client.29257 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.615+0000 7f762f17eb80 1 -- 172.21.15.62:0/1714011016 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556fb8259c60 con 0x556fb84f2070 2023-07-14T16:30:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f75fcff9700 1 -- 172.21.15.62:0/1714011016 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556fb839c4f0 con 0x556fb84f2070 2023-07-14T16:30:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f75fcff9700 1 -- 172.21.15.62:0/1714011016 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7604095d10 con 0x556fb84f2070 2023-07-14T16:30:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f762f17eb80 10 client.29257 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f762f17eb80 10 client.29257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f762f17eb80 20 client.29257 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556fb8506fd0 0x556fb85093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f762f17eb80 4 client.29257 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:30:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f762f17eb80 1 -- 172.21.15.62:0/1714011016 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fb843beb0 con 0x556fb8506fd0 2023-07-14T16:30:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.616+0000 7f761959b700 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556fb8506fd0 0x556fb85093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.617+0000 7f761959b700 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556fb8506fd0 0x556fb85093b0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da24456f9e19c61 server_cookie=c8f016e571216d71 in_seq=0 out_seq=0 2023-07-14T16:30:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.617+0000 7f75fcff9700 10 client.29257 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f75fcff9700 1 -- 172.21.15.62:0/1714011016 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556fb843beb0 con 0x556fb8506fd0 2023-07-14T16:30:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f75fcff9700 10 client.29257 handle_command_reply: tid=0 2023-07-14T16:30:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 client.29257 shutdown 2023-07-14T16:30:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 -- 172.21.15.62:0/1714011016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f2070 msgr2=0x556fb84fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f2070 0x556fb84fb800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f760400c5b0 tx=0x7f7604099990 comp rx=0 tx=0).stop 2023-07-14T16:30:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 -- 172.21.15.62:0/1714011016 shutdown_connections 2023-07-14T16:30:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556fb8506fd0 0x556fb85093b0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb8502840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f75fcff9700 0 client.29257 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f1720 0x556fb84fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1714011016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f2070 0x556fb84fb800 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 -- 172.21.15.62:0/1714011016 >> 172.21.15.62:0/1714011016 conn(0x556fb84e6ff0 msgr2=0x556fb84ebb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 -- 172.21.15.62:0/1714011016 shutdown_connections 2023-07-14T16:30:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 1 -- 172.21.15.62:0/1714011016 wait complete. 2023-07-14T16:30:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.618+0000 7f762f17eb80 20 client.29257 trim_cache size 0 max 16384 2023-07-14T16:30:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.636+0000 7f762f17eb80 1 Processor -- start 2023-07-14T16:30:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.636+0000 7f762f17eb80 1 -- start start 2023-07-14T16:30:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.636+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.636+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb8502800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.636+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb84f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.636+0000 7f762f17eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb84fa770 con 0x556fb84f2070 2023-07-14T16:30:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.636+0000 7f762f17eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb84fae30 con 0x556fb84f2e30 2023-07-14T16:30:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.636+0000 7f762f17eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb8506d60 con 0x556fb84f1720 2023-07-14T16:30:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb8502800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb8502800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44342/0 (socket says 172.21.15.62:44342) 2023-07-14T16:30:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 -- 172.21.15.62:0/3701618186 learned_addr learned my addr 172.21.15.62:0/3701618186 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7619d9c700 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb84f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f761959b700 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 -- 172.21.15.62:0/3701618186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 msgr2=0x556fb84f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 -- 172.21.15.62:0/3701618186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 msgr2=0x556fb84f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb84f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 -- 172.21.15.62:0/3701618186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fb8297250 con 0x556fb84f2070 2023-07-14T16:30:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.637+0000 7f7618d9a700 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb8502800 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7604096420 tx=0x7f76040984d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d727b8b105016359 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f75fffff700 1 -- 172.21.15.62:0/3701618186 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604096090 con 0x556fb84f2070 2023-07-14T16:30:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f75fffff700 1 -- 172.21.15.62:0/3701618186 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76040a2370 con 0x556fb84f2070 2023-07-14T16:30:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f75fffff700 1 -- 172.21.15.62:0/3701618186 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604099670 con 0x556fb84f2070 2023-07-14T16:30:56.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f762f17eb80 1 -- 172.21.15.62:0/3701618186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 msgr2=0x556fb8502800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f762f17eb80 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb8502800 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7604096420 tx=0x7f76040984d0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f762f17eb80 1 -- 172.21.15.62:0/3701618186 shutdown_connections 2023-07-14T16:30:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f762f17eb80 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb84f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f762f17eb80 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb84f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f762f17eb80 1 --2- 172.21.15.62:0/3701618186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb8502800 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.638+0000 7f762f17eb80 1 -- 172.21.15.62:0/3701618186 >> 172.21.15.62:0/3701618186 conn(0x556fb84e6ff0 msgr2=0x556fb84f9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.639+0000 7f762f17eb80 1 -- 172.21.15.62:0/3701618186 shutdown_connections 2023-07-14T16:30:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.639+0000 7f762f17eb80 1 -- 172.21.15.62:0/3701618186 wait complete. 2023-07-14T16:30:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.639+0000 7f762f17eb80 1 -- start start 2023-07-14T16:30:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.639+0000 7f762f17eb80 1 Processor -- start 2023-07-14T16:30:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb8501fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb85024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f762f17eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb84fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f762f17eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb84fef90 con 0x556fb84f2070 2023-07-14T16:30:56.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f762f17eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb84ff0d0 con 0x556fb84f2e30 2023-07-14T16:30:56.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f762f17eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb84ff290 con 0x556fb84f1720 2023-07-14T16:30:56.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f762f17eb80 10 client.0 _ll_register_callbacks cb 0x7f760400cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:30:56.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f7618d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb85024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.640+0000 7f761959b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb8501fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f761959b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb8501fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60134/0 (socket says 172.21.15.62:60134) 2023-07-14T16:30:56.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f7618d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb85024f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44350/0 (socket says 172.21.15.62:44350) 2023-07-14T16:30:56.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f761959b700 1 -- 172.21.15.62:0/1640389519 learned_addr learned my addr 172.21.15.62:0/1640389519 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:30:56.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f7619d9c700 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb84fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f761959b700 1 -- 172.21.15.62:0/1640389519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 msgr2=0x556fb84fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f761959b700 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb84fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f761959b700 1 -- 172.21.15.62:0/1640389519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 msgr2=0x556fb85024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:30:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f761959b700 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb85024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f761959b700 1 -- 172.21.15.62:0/1640389519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fb8259c60 con 0x556fb84f1720 2023-07-14T16:30:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f7618d9a700 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb85024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:30:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f761959b700 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb8501fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f761000bd20 tx=0x7f7610009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c0ed6c4f432ab71 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:30:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f75fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:30:56.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.641+0000 7f75fcff9700 1 -- 172.21.15.62:0/1640389519 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610031070 con 0x556fb84f1720 2023-07-14T16:30:56.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.642+0000 7f75fcff9700 1 -- 172.21.15.62:0/1640389519 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f76100050b0 con 0x556fb84f1720 2023-07-14T16:30:56.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.642+0000 7f762f17eb80 1 -- 172.21.15.62:0/1640389519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f761001d040 con 0x556fb84f1720 2023-07-14T16:30:56.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.642+0000 7f75fcff9700 1 -- 172.21.15.62:0/1640389519 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7610005920 con 0x556fb84f1720 2023-07-14T16:30:56.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.642+0000 7f762f17eb80 10 client.29263 fetch_fsmap learned FSMap version 28 2023-07-14T16:30:56.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.642+0000 7f762f17eb80 1 -- 172.21.15.62:0/1640389519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556fb826f2b0 con 0x556fb84f1720 2023-07-14T16:30:56.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.642+0000 7f75fcff9700 1 -- 172.21.15.62:0/1640389519 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f761001d040 con 0x556fb84f1720 2023-07-14T16:30:56.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.643+0000 7f75fcff9700 1 -- 172.21.15.62:0/1640389519 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7610015020 con 0x556fb84f1720 2023-07-14T16:30:56.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.643+0000 7f762f17eb80 10 client.29263 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:30:56.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.643+0000 7f762f17eb80 10 client.29263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:30:56.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.643+0000 7f762f17eb80 20 client.29263 populate_metadata read hostname 'smithi062' 2023-07-14T16:30:56.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.643+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556fb8564630 0x556fb8566a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:30:56.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.643+0000 7f762f17eb80 4 client.29263 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:30:56.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.643+0000 7f762f17eb80 1 -- 172.21.15.62:0/1640389519 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556fb83fe050 con 0x556fb8564630 2023-07-14T16:30:56.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.643+0000 7f7618d9a700 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556fb8564630 0x556fb8566a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:30:56.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.644+0000 7f75fcff9700 10 client.29263 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:56.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.644+0000 7f7618d9a700 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556fb8564630 0x556fb8566a40 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b38034207e02f1 server_cookie=c23d5cb80c324ceb in_seq=0 out_seq=0 2023-07-14T16:30:56.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f75fcff9700 1 -- 172.21.15.62:0/1640389519 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556fb83fe050 con 0x556fb8564630 2023-07-14T16:30:56.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f75fcff9700 10 client.29263 handle_command_reply: tid=0 2023-07-14T16:30:56.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 client.29263 shutdown 2023-07-14T16:30:56.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 -- 172.21.15.62:0/1640389519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 msgr2=0x556fb8501fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb8501fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f761000bd20 tx=0x7f7610009400 comp rx=0 tx=0).stop 2023-07-14T16:30:56.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 -- 172.21.15.62:0/1640389519 shutdown_connections 2023-07-14T16:30:56.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556fb8564630 0x556fb8566a40 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556fb84f2e30 0x556fb84fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556fb84f1720 0x556fb8501fb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f75fcff9700 0 client.29263 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:30:56.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 --2- 172.21.15.62:0/1640389519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fb84f2070 0x556fb85024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 -- 172.21.15.62:0/1640389519 >> 172.21.15.62:0/1640389519 conn(0x556fb84e6ff0 msgr2=0x556fb84f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 -- 172.21.15.62:0/1640389519 shutdown_connections 2023-07-14T16:30:56.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 1 -- 172.21.15.62:0/1640389519 wait complete. 2023-07-14T16:30:56.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.645+0000 7f762f17eb80 20 client.29263 trim_cache size 0 max 16384 2023-07-14T16:30:56.626 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:30:56.626 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:30:56.626 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:30:56.626 INFO:teuthology.orchestra.run.smithi062.stdout: "81367246-12e5-4eef-805a-b624998a3f7f": { 2023-07-14T16:30:56.626 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:30:56.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "81367246-12e5-4eef-805a-b624998a3f7f", 2023-07-14T16:30:56.627 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:30:56.627 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:30:56.627 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:30:56.627 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:30:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 -- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76000a1270 msgr2=0x7f76000a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76000a1270 0x7f76000a3720 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f760c0077d0 tx=0x7f760c007b90 comp rx=0 tx=0).stop 2023-07-14T16:30:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 -- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614102dd0 msgr2=0x7f76140fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:30:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614102dd0 0x7f76140fe460 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f760400e9c0 tx=0x7f760400ecd0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 -- 172.21.15.62:0/215514790 shutdown_connections 2023-07-14T16:30:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76000a1270 0x7f76000a3720 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76141056f0 0x7f76140fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76141004b0 0x7f76140ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 --2- 172.21.15.62:0/215514790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7614102dd0 0x7f76140fe460 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:30:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.646+0000 7f761a59d700 1 -- 172.21.15.62:0/215514790 >> 172.21.15.62:0/215514790 conn(0x7f76140f9ed0 msgr2=0x7f7614101110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:30:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.647+0000 7f761a59d700 1 -- 172.21.15.62:0/215514790 shutdown_connections 2023-07-14T16:30:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:30:56.647+0000 7f761a59d700 1 -- 172.21.15.62:0/215514790 wait complete. 2023-07-14T16:30:56.639 INFO:tasks.cephfs.filesystem:scrub status for tag:81367246-12e5-4eef-805a-b624998a3f7f - {'path': '/', 'tag': '81367246-12e5-4eef-805a-b624998a3f7f', 'options': 'recursive,force'} 2023-07-14T16:30:56.648 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:30:56 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:30:56.366+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v277>,<0x10000000003/client.0 v278>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v290>,<0x10000000001/sv_0 v304>,<0x10000000000/_nogroup v298>,<0x1/volumes v288>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:30:57.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:56 smithi062 ceph-mon[95543]: pgmap v816: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:30:57.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:56 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:30:57.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:56 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:57.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:56 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:30:57.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:56 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:30:57.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:56 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:57.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:56 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:57.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:56 smithi081 ceph-mon[127795]: pgmap v816: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:30:57.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:56 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:30:57.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:56 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:57.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:56 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:30:57.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:56 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:30:57.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:56 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:57.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:56 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:57.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:56 smithi164 ceph-mon[97427]: pgmap v816: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 13 op/s 2023-07-14T16:30:57.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:56 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:30:57.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:56 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:57.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:56 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:30:57.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:56 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:30:57.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:56 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:57.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:56 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:58.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:57 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:30:58.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:57 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:58.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:57 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:30:58.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:57 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:30:58.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:57 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:58.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:57 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:30:58.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:57 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:58.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:57 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:30:58.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:57 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:30:58.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:57 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:58.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:57 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:30:58.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:57 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:58.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:57 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:30:58.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:57 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:30:58.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:57 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:30:59.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:30:58 smithi062 ceph-mon[95543]: pgmap v817: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:30:59.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:30:58 smithi081 ceph-mon[127795]: pgmap v817: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:30:59.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:30:58 smithi164 ceph-mon[97427]: pgmap v817: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 18 op/s 2023-07-14T16:31:01.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:00 smithi062 ceph-mon[95543]: pgmap v818: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:31:01.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:00 smithi081 ceph-mon[127795]: pgmap v818: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:31:01.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:00 smithi164 ceph-mon[97427]: pgmap v818: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:31:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.650+0000 7fef773ff700 1 -- 172.21.15.62:0/3402277610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef701079d0 msgr2=0x7fef70109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.650+0000 7fef773ff700 1 --2- 172.21.15.62:0/3402277610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef701079d0 0x7fef70109e50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fef6800b0a0 tx=0x7fef6800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.651+0000 7fef773ff700 1 -- 172.21.15.62:0/3402277610 shutdown_connections 2023-07-14T16:31:02.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.651+0000 7fef773ff700 1 --2- 172.21.15.62:0/3402277610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef701079d0 0x7fef70109e50 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.651+0000 7fef773ff700 1 --2- 172.21.15.62:0/3402277610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef700744f0 0x7fef70072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.651+0000 7fef773ff700 1 --2- 172.21.15.62:0/3402277610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef70072f10 0x7fef70073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.651+0000 7fef773ff700 1 -- 172.21.15.62:0/3402277610 >> 172.21.15.62:0/3402277610 conn(0x7fef700fb160 msgr2=0x7fef700fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.651+0000 7fef773ff700 1 -- 172.21.15.62:0/3402277610 shutdown_connections 2023-07-14T16:31:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.651+0000 7fef773ff700 1 -- 172.21.15.62:0/3402277610 wait complete. 2023-07-14T16:31:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.652+0000 7fef773ff700 1 Processor -- start 2023-07-14T16:31:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.652+0000 7fef773ff700 1 -- start start 2023-07-14T16:31:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef773ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70072f10 0x7fef70197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef773ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef700744f0 0x7fef70198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef773ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef701079d0 0x7fef70198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef773ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70199540 con 0x7fef70072f10 2023-07-14T16:31:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef773ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701996b0 con 0x7fef701079d0 2023-07-14T16:31:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef773ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70199870 con 0x7fef700744f0 2023-07-14T16:31:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70072f10 0x7fef70197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70072f10 0x7fef70197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44378/0 (socket says 172.21.15.62:44378) 2023-07-14T16:31:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7519b700 1 -- 172.21.15.62:0/3604715190 learned_addr learned my addr 172.21.15.62:0/3604715190 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7599c700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef701079d0 0x7fef70198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7499a700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef700744f0 0x7fef70198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7519b700 1 -- 172.21.15.62:0/3604715190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef700744f0 msgr2=0x7fef70198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7519b700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef700744f0 0x7fef70198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7519b700 1 -- 172.21.15.62:0/3604715190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef701079d0 msgr2=0x7fef70198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7519b700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef701079d0 0x7fef70198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7519b700 1 -- 172.21.15.62:0/3604715190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68009d00 con 0x7fef70072f10 2023-07-14T16:31:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.653+0000 7fef7499a700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef700744f0 0x7fef70198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:31:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.654+0000 7fef7519b700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70072f10 0x7fef70197b20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00e9a0 tx=0x7fef6c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.654+0000 7fef667fc700 1 -- 172.21.15.62:0/3604715190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c009790 con 0x7fef70072f10 2023-07-14T16:31:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.654+0000 7fef773ff700 1 -- 172.21.15.62:0/3604715190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef70199b50 con 0x7fef70072f10 2023-07-14T16:31:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.654+0000 7fef667fc700 1 -- 172.21.15.62:0/3604715190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef6c00cbd0 con 0x7fef70072f10 2023-07-14T16:31:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.654+0000 7fef667fc700 1 -- 172.21.15.62:0/3604715190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c018680 con 0x7fef70072f10 2023-07-14T16:31:02.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.654+0000 7fef773ff700 1 -- 172.21.15.62:0/3604715190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef701a0310 con 0x7fef70072f10 2023-07-14T16:31:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.658+0000 7fef667fc700 1 -- 172.21.15.62:0/3604715190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fef6c00cd40 con 0x7fef70072f10 2023-07-14T16:31:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.659+0000 7fef8ad51b80 1 Processor -- start 2023-07-14T16:31:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.659+0000 7fef8ad51b80 1 -- start start 2023-07-14T16:31:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d54f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d54f840 0x55d10d5501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d5506e0 0x55d10d550b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d454c50 con 0x55d10d3ab6b0 2023-07-14T16:31:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d4a79e0 con 0x55d10d5506e0 2023-07-14T16:31:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d4a0e70 con 0x55d10d54f840 2023-07-14T16:31:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef7599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d54f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef7599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d54f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44386/0 (socket says 172.21.15.62:44386) 2023-07-14T16:31:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef7599c700 1 -- 172.21.15.62:0/2388700651 learned_addr learned my addr 172.21.15.62:0/2388700651 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef667fc700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fef5c0a1150 0x7fef5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.660+0000 7fef7519b700 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d54f840 0x55d10d5501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7499a700 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d5506e0 0x55d10d550b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef667fc700 1 -- 172.21.15.62:0/3604715190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef6c014070 con 0x7fef70072f10 2023-07-14T16:31:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7599c700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fef5c0a1150 0x7fef5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7519b700 1 -- 172.21.15.62:0/2388700651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d5506e0 msgr2=0x55d10d550b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7519b700 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d5506e0 0x55d10d550b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7519b700 1 -- 172.21.15.62:0/2388700651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 msgr2=0x55d10d54f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7519b700 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d54f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7519b700 1 -- 172.21.15.62:0/2388700651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d10d2ced80 con 0x55d10d54f840 2023-07-14T16:31:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7599c700 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d54f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7519b700 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d54f840 0x55d10d5501a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0ca6f0 tx=0x7fef6c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d93f9d105d2847 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.661+0000 7fef7599c700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fef5c0a1150 0x7fef5c0a3600 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55d10d54f060 tx=0x7fef6800b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef5b7fe700 1 -- 172.21.15.62:0/2388700651 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c09a060 con 0x55d10d54f840 2023-07-14T16:31:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef5b7fe700 1 -- 172.21.15.62:0/2388700651 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef6c0cce90 con 0x55d10d54f840 2023-07-14T16:31:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef5b7fe700 1 -- 172.21.15.62:0/2388700651 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c09a2c0 con 0x55d10d54f840 2023-07-14T16:31:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef8ad51b80 1 -- 172.21.15.62:0/2388700651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d54f840 msgr2=0x55d10d5501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d54f840 0x55d10d5501a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0ca6f0 tx=0x7fef6c01f000 comp rx=0 tx=0).stop 2023-07-14T16:31:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef8ad51b80 1 -- 172.21.15.62:0/2388700651 shutdown_connections 2023-07-14T16:31:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d5506e0 0x55d10d550b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d54f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/2388700651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d54f840 0x55d10d5501a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef8ad51b80 1 -- 172.21.15.62:0/2388700651 >> 172.21.15.62:0/2388700651 conn(0x55d10d544be0 msgr2=0x55d10d549cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.662+0000 7fef8ad51b80 1 -- 172.21.15.62:0/2388700651 shutdown_connections 2023-07-14T16:31:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.663+0000 7fef8ad51b80 1 -- 172.21.15.62:0/2388700651 wait complete. 2023-07-14T16:31:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.663+0000 7fef8ad51b80 1 -- start start 2023-07-14T16:31:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.663+0000 7fef8ad51b80 1 Processor -- start 2023-07-14T16:31:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d558860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d558da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d5506e0 0x55d10d55fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d440a60 con 0x55d10d3ab6b0 2023-07-14T16:31:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d440760 con 0x55d10d54f840 2023-07-14T16:31:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d3248c0 con 0x55d10d5506e0 2023-07-14T16:31:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef8ad51b80 10 client.0 _ll_register_callbacks cb 0x55d10d43e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef7519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d558860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef7519b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d558860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44396/0 (socket says 172.21.15.62:44396) 2023-07-14T16:31:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef7519b700 1 -- 172.21.15.62:0/3253716846 learned_addr learned my addr 172.21.15.62:0/3253716846 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef7499a700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d558da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.664+0000 7fef7599c700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d5506e0 0x55d10d55fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef7519b700 1 -- 172.21.15.62:0/3253716846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d5506e0 msgr2=0x55d10d55fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef7519b700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d5506e0 0x55d10d55fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef7519b700 1 -- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 msgr2=0x55d10d558da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef7519b700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d558da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef7519b700 1 -- 172.21.15.62:0/3253716846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7010aa70 con 0x55d10d3ab6b0 2023-07-14T16:31:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef7599c700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d5506e0 0x55d10d55fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef7499a700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d558da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef7519b700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d558860 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fef6c099ff0 tx=0x7fef6c0a24b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f8bf97cdf337b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef58ff9700 1 -- 172.21.15.62:0/3253716846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c09a4d0 con 0x55d10d3ab6b0 2023-07-14T16:31:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef58ff9700 1 -- 172.21.15.62:0/3253716846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef6c0a2020 con 0x55d10d3ab6b0 2023-07-14T16:31:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.665+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3253716846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d10d3fa490 con 0x55d10d3ab6b0 2023-07-14T16:31:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef58ff9700 1 -- 172.21.15.62:0/3253716846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef6c0a21e0 con 0x55d10d3ab6b0 2023-07-14T16:31:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef8ad51b80 10 client.19875 fetch_fsmap learned FSMap version 28 2023-07-14T16:31:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3253716846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fef701a0310 con 0x55d10d3ab6b0 2023-07-14T16:31:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef58ff9700 1 -- 172.21.15.62:0/3253716846 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d10d3fa490 con 0x55d10d3ab6b0 2023-07-14T16:31:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef58ff9700 1 -- 172.21.15.62:0/3253716846 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fef6c09ae40 con 0x55d10d3ab6b0 2023-07-14T16:31:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef8ad51b80 10 client.19875 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:31:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef8ad51b80 10 client.19875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef8ad51b80 20 client.19875 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d10d564270 0x55d10d566680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef8ad51b80 4 client.19875 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.666+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3253716846 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d10d499e50 con 0x55d10d564270 2023-07-14T16:31:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.667+0000 7fef7499a700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d10d564270 0x55d10d566680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.667+0000 7fef7499a700 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d10d564270 0x55d10d566680 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da2643a4a3b9a85 server_cookie=de3257c37d79dd6b in_seq=0 out_seq=0 2023-07-14T16:31:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.667+0000 7fef58ff9700 10 client.19875 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef58ff9700 1 -- 172.21.15.62:0/3253716846 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d10d499e50 con 0x55d10d564270 2023-07-14T16:31:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef58ff9700 10 client.19875 handle_command_reply: tid=0 2023-07-14T16:31:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 client.19875 shutdown 2023-07-14T16:31:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3253716846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 msgr2=0x55d10d558860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d558860 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fef6c099ff0 tx=0x7fef6c0a24b0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3253716846 shutdown_connections 2023-07-14T16:31:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d5506e0 0x55d10d55fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d3ab6b0 0x55d10d558860 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d10d564270 0x55d10d566680 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3253716846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d558da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef58ff9700 0 client.19875 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.668+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3253716846 >> 172.21.15.62:0/3253716846 conn(0x55d10d544be0 msgr2=0x55d10d5494e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.669+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3253716846 shutdown_connections 2023-07-14T16:31:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.669+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3253716846 wait complete. 2023-07-14T16:31:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.669+0000 7fef8ad51b80 20 client.19875 trim_cache size 0 max 16384 2023-07-14T16:31:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.688+0000 7fef8ad51b80 1 Processor -- start 2023-07-14T16:31:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.688+0000 7fef8ad51b80 1 -- start start 2023-07-14T16:31:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 0x55d10d557f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d55fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d557560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d546d40 con 0x55d10d5506e0 2023-07-14T16:31:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d54b7a0 con 0x55d10d54f840 2023-07-14T16:31:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d556850 con 0x55d10d3ab6b0 2023-07-14T16:31:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d557560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d557560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44402/0 (socket says 172.21.15.62:44402) 2023-07-14T16:31:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7599c700 1 -- 172.21.15.62:0/3192065540 learned_addr learned my addr 172.21.15.62:0/3192065540 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7519b700 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 0x55d10d557f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7499a700 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d55fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7599c700 1 -- 172.21.15.62:0/3192065540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 msgr2=0x55d10d557f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7599c700 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 0x55d10d557f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7599c700 1 -- 172.21.15.62:0/3192065540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 msgr2=0x55d10d55fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7599c700 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d55fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.689+0000 7fef7599c700 1 -- 172.21.15.62:0/3192065540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d10d2b7c00 con 0x55d10d5506e0 2023-07-14T16:31:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.690+0000 7fef7599c700 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d557560 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fef6800b0a0 tx=0x7fef680097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36a1a47e821da29 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.690+0000 7fef5bfff700 1 -- 172.21.15.62:0/3192065540 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6800e040 con 0x55d10d5506e0 2023-07-14T16:31:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.690+0000 7fef5bfff700 1 -- 172.21.15.62:0/3192065540 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef68004c50 con 0x55d10d5506e0 2023-07-14T16:31:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.690+0000 7fef5bfff700 1 -- 172.21.15.62:0/3192065540 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68005040 con 0x55d10d5506e0 2023-07-14T16:31:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3192065540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 msgr2=0x55d10d557560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d557560 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fef6800b0a0 tx=0x7fef680097f0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3192065540 shutdown_connections 2023-07-14T16:31:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d557560 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 0x55d10d557f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/3192065540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d55fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3192065540 >> 172.21.15.62:0/3192065540 conn(0x55d10d544be0 msgr2=0x55d10d56afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3192065540 shutdown_connections 2023-07-14T16:31:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 -- 172.21.15.62:0/3192065540 wait complete. 2023-07-14T16:31:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.691+0000 7fef8ad51b80 1 -- start start 2023-07-14T16:31:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.692+0000 7fef8ad51b80 1 Processor -- start 2023-07-14T16:31:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.692+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 0x55d10d55bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.692+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d55c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.692+0000 7fef8ad51b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d55f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.692+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d55f760 con 0x55d10d5506e0 2023-07-14T16:31:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.692+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d55f8d0 con 0x55d10d54f840 2023-07-14T16:31:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.692+0000 7fef8ad51b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10d55fa90 con 0x55d10d3ab6b0 2023-07-14T16:31:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.692+0000 7fef8ad51b80 10 client.0 _ll_register_callbacks cb 0x55d10d43e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d55f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7519b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 0x55d10d55bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d55f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44404/0 (socket says 172.21.15.62:44404) 2023-07-14T16:31:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 -- 172.21.15.62:0/1042137085 learned_addr learned my addr 172.21.15.62:0/1042137085 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7499a700 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d55c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 -- 172.21.15.62:0/1042137085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 msgr2=0x55d10d55bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 0x55d10d55bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 -- 172.21.15.62:0/1042137085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 msgr2=0x55d10d55c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d55c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 -- 172.21.15.62:0/1042137085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef68007b90 con 0x55d10d5506e0 2023-07-14T16:31:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef7599c700 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d55f220 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fef68009a20 tx=0x7fef68021ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729963639e34630e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.693+0000 7fef58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef58ff9700 1 -- 172.21.15.62:0/1042137085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6800e030 con 0x55d10d5506e0 2023-07-14T16:31:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef8ad51b80 1 -- 172.21.15.62:0/1042137085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef6c0cce90 con 0x55d10d5506e0 2023-07-14T16:31:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef58ff9700 1 -- 172.21.15.62:0/1042137085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef68020660 con 0x55d10d5506e0 2023-07-14T16:31:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef58ff9700 1 -- 172.21.15.62:0/1042137085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef68020820 con 0x55d10d5506e0 2023-07-14T16:31:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef8ad51b80 10 client.19881 fetch_fsmap learned FSMap version 28 2023-07-14T16:31:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef8ad51b80 1 -- 172.21.15.62:0/1042137085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d10d2f51f0 con 0x55d10d5506e0 2023-07-14T16:31:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef58ff9700 1 -- 172.21.15.62:0/1042137085 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef6c0cce90 con 0x55d10d5506e0 2023-07-14T16:31:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef58ff9700 1 -- 172.21.15.62:0/1042137085 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fef68046070 con 0x55d10d5506e0 2023-07-14T16:31:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef8ad51b80 10 client.19881 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T16:31:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef8ad51b80 10 client.19881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.694+0000 7fef8ad51b80 20 client.19881 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.695+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d10d5b8d10 0x55d10d5bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.695+0000 7fef8ad51b80 4 client.19881 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:31:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.695+0000 7fef8ad51b80 1 -- 172.21.15.62:0/1042137085 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d10d45bff0 con 0x55d10d5b8d10 2023-07-14T16:31:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.695+0000 7fef7519b700 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d10d5b8d10 0x55d10d5bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.695+0000 7fef58ff9700 10 client.19881 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.695+0000 7fef7519b700 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d10d5b8d10 0x55d10d5bb0f0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23eac9c18747d1e server_cookie=83df38cdd899e6b6 in_seq=0 out_seq=0 2023-07-14T16:31:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef58ff9700 1 -- 172.21.15.62:0/1042137085 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d10d45bff0 con 0x55d10d5b8d10 2023-07-14T16:31:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef58ff9700 10 client.19881 handle_command_reply: tid=0 2023-07-14T16:31:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 client.19881 shutdown 2023-07-14T16:31:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 -- 172.21.15.62:0/1042137085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 msgr2=0x55d10d55f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d55f220 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fef68009a20 tx=0x7fef68021ea0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 -- 172.21.15.62:0/1042137085 shutdown_connections 2023-07-14T16:31:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d10d5506e0 0x55d10d55f220 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d10d3ab6b0 0x55d10d55bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d10d5b8d10 0x55d10d5bb0f0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 --2- 172.21.15.62:0/1042137085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d10d54f840 0x55d10d55c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef8ad51b80 1 -- 172.21.15.62:0/1042137085 >> 172.21.15.62:0/1042137085 conn(0x55d10d544be0 msgr2=0x55d10d557560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.705+0000 7fef58ff9700 0 client.19881 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.706+0000 7fef8ad51b80 1 -- 172.21.15.62:0/1042137085 shutdown_connections 2023-07-14T16:31:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.706+0000 7fef8ad51b80 1 -- 172.21.15.62:0/1042137085 wait complete. 2023-07-14T16:31:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.706+0000 7fef8ad51b80 20 client.19881 trim_cache size 0 max 16384 2023-07-14T16:31:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:31:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:31:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:31:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:31:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 -- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fef5c0a1150 msgr2=0x7fef5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fef5c0a1150 0x7fef5c0a3600 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55d10d54f060 tx=0x7fef6800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 -- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70072f10 msgr2=0x7fef70197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70072f10 0x7fef70197b20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00e9a0 tx=0x7fef6c00ed60 comp rx=0 tx=0).stop 2023-07-14T16:31:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 -- 172.21.15.62:0/3604715190 shutdown_connections 2023-07-14T16:31:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fef5c0a1150 0x7fef5c0a3600 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef701079d0 0x7fef70198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70072f10 0x7fef70197b20 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 --2- 172.21.15.62:0/3604715190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef700744f0 0x7fef70198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 -- 172.21.15.62:0/3604715190 >> 172.21.15.62:0/3604715190 conn(0x7fef700fb160 msgr2=0x7fef700fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 -- 172.21.15.62:0/3604715190 shutdown_connections 2023-07-14T16:31:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:02.707+0000 7fef773ff700 1 -- 172.21.15.62:0/3604715190 wait complete. 2023-07-14T16:31:03.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:02 smithi062 ceph-mon[95543]: pgmap v819: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:31:03.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:02 smithi081 ceph-mon[127795]: pgmap v819: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:31:03.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:02 smithi164 ceph-mon[97427]: pgmap v819: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 35 op/s 2023-07-14T16:31:05.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:04 smithi062 ceph-mon[95543]: pgmap v820: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:31:05.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:04 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:05.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:04 smithi081 ceph-mon[127795]: pgmap v820: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:31:05.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:04 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:05.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:04 smithi164 ceph-mon[97427]: pgmap v820: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 38 op/s 2023-07-14T16:31:05.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:04 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:07.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:06 smithi062 ceph-mon[95543]: pgmap v821: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:31:07.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:06 smithi081 ceph-mon[127795]: pgmap v821: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:31:07.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:06 smithi164 ceph-mon[97427]: pgmap v821: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 35 op/s 2023-07-14T16:31:09.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:08 smithi062 ceph-mon[95543]: pgmap v822: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:31:09.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:08 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:31:09.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:08 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:31:09.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:08 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:31:09.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:08 smithi081 ceph-mon[127795]: pgmap v822: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:31:09.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:08 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:31:09.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:08 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:31:09.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:08 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:31:09.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:08 smithi164 ceph-mon[97427]: pgmap v822: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 35 op/s 2023-07-14T16:31:09.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:08 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:31:09.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:08 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:31:09.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:08 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:31:11.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:10 smithi062 ceph-mon[95543]: pgmap v823: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:31:11.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:10 smithi081 ceph-mon[127795]: pgmap v823: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:31:11.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:10 smithi164 ceph-mon[97427]: pgmap v823: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 34 op/s 2023-07-14T16:31:13.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:12 smithi062 ceph-mon[95543]: pgmap v824: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 21 op/s 2023-07-14T16:31:13.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:12 smithi081 ceph-mon[127795]: pgmap v824: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 21 op/s 2023-07-14T16:31:13.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:12 smithi164 ceph-mon[97427]: pgmap v824: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 21 op/s 2023-07-14T16:31:15.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:14 smithi062 ceph-mon[95543]: pgmap v825: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:15.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:14 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:15.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:14 smithi081 ceph-mon[127795]: pgmap v825: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:15.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:14 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:15.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:14 smithi164 ceph-mon[97427]: pgmap v825: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:15.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:14 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:17.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:16 smithi062 ceph-mon[95543]: pgmap v826: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:31:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 -- 172.21.15.62:0/351462373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db0100d00 msgr2=0x7f9db0101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 --2- 172.21.15.62:0/351462373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db0100d00 0x7f9db0101150 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009990 tx=0x7f9da4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 -- 172.21.15.62:0/351462373 shutdown_connections 2023-07-14T16:31:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 --2- 172.21.15.62:0/351462373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db0101690 0x7f9db0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 --2- 172.21.15.62:0/351462373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db00ffb10 0x7f9db00fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 --2- 172.21.15.62:0/351462373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db0100d00 0x7f9db0101150 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 -- 172.21.15.62:0/351462373 >> 172.21.15.62:0/351462373 conn(0x7f9db00fb130 msgr2=0x7f9db00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 -- 172.21.15.62:0/351462373 shutdown_connections 2023-07-14T16:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.165+0000 7f9db61ff700 1 -- 172.21.15.62:0/351462373 wait complete. 2023-07-14T16:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.167+0000 7f9db61ff700 1 Processor -- start 2023-07-14T16:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.167+0000 7f9db61ff700 1 -- start start 2023-07-14T16:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9db61ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db00ffb10 0x7f9db0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9db61ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db0100d00 0x7f9db0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9db61ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db0101690 0x7f9db0198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9db61ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0199490 con 0x7f9db00ffb10 2023-07-14T16:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9db61ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0199600 con 0x7f9db0100d00 2023-07-14T16:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9db61ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db01997c0 con 0x7f9db0101690 2023-07-14T16:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db00ffb10 0x7f9db0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db00ffb10 0x7f9db0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42494/0 (socket says 172.21.15.62:42494) 2023-07-14T16:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daf7fe700 1 -- 172.21.15.62:0/3891389277 learned_addr learned my addr 172.21.15.62:0/3891389277 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daffff700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db0101690 0x7f9db0198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db0100d00 0x7f9db0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daf7fe700 1 -- 172.21.15.62:0/3891389277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db0101690 msgr2=0x7f9db0198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db0101690 0x7f9db0198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daf7fe700 1 -- 172.21.15.62:0/3891389277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db0100d00 msgr2=0x7f9db0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db0100d00 0x7f9db0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daf7fe700 1 -- 172.21.15.62:0/3891389277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40095f0 con 0x7f9db00ffb10 2023-07-14T16:31:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.168+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db0100d00 0x7f9db0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:31:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.169+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db00ffb10 0x7f9db0197a70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9da000d980 tx=0x7f9da000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.169+0000 7f9dacff9700 1 -- 172.21.15.62:0/3891389277 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da000bb60 con 0x7f9db00ffb10 2023-07-14T16:31:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.169+0000 7f9db61ff700 1 -- 172.21.15.62:0/3891389277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db0199aa0 con 0x7f9db00ffb10 2023-07-14T16:31:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.169+0000 7f9dacff9700 1 -- 172.21.15.62:0/3891389277 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9da0007620 con 0x7f9db00ffb10 2023-07-14T16:31:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.169+0000 7f9dacff9700 1 -- 172.21.15.62:0/3891389277 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da000f3c0 con 0x7f9db00ffb10 2023-07-14T16:31:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.169+0000 7f9db61ff700 1 -- 172.21.15.62:0/3891389277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db01a0260 con 0x7f9db00ffb10 2023-07-14T16:31:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.173+0000 7f9dacff9700 1 -- 172.21.15.62:0/3891389277 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9da0010460 con 0x7f9db00ffb10 2023-07-14T16:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.175+0000 7f9dacff9700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d980a11d0 0x7f9d980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.175+0000 7f9dc9bbab80 1 Processor -- start 2023-07-14T16:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.175+0000 7f9dacff9700 1 -- 172.21.15.62:0/3891389277 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9da00ccba0 con 0x7f9db00ffb10 2023-07-14T16:31:17.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:16 smithi081 ceph-mon[127795]: pgmap v826: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.175+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d980a11d0 0x7f9d980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.175+0000 7f9dc9bbab80 1 -- start start 2023-07-14T16:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x561724856bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x561724857970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724858320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561724753c50 con 0x561724857eb0 2023-07-14T16:31:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617247a69e0 con 0x5617248570f0 2023-07-14T16:31:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172479fe70 con 0x5617246aa6b0 2023-07-14T16:31:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9daffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724858320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9daffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724858320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42506/0 (socket says 172.21.15.62:42506) 2023-07-14T16:31:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9daffff700 1 -- 172.21.15.62:0/3162945567 learned_addr learned my addr 172.21.15.62:0/3162945567 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x561724856bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.176+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d980a11d0 0x7f9d980a3680 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9da40098e0 tx=0x7f9da4004740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x561724857970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daffff700 1 -- 172.21.15.62:0/3162945567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 msgr2=0x561724856bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daffff700 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x561724856bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daffff700 1 -- 172.21.15.62:0/3162945567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 msgr2=0x561724857970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daffff700 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x561724857970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daffff700 1 -- 172.21.15.62:0/3162945567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617245cdd80 con 0x561724857eb0 2023-07-14T16:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x561724857970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x561724856bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9daffff700 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724858320 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009440 tx=0x7f9da8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5e499728eb309d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9d95ffb700 1 -- 172.21.15.62:0/3162945567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da800e070 con 0x561724857eb0 2023-07-14T16:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9d95ffb700 1 -- 172.21.15.62:0/3162945567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9da800f040 con 0x561724857eb0 2023-07-14T16:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.177+0000 7f9d95ffb700 1 -- 172.21.15.62:0/3162945567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8015ab0 con 0x561724857eb0 2023-07-14T16:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.178+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3162945567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 msgr2=0x561724858320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.178+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724858320 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009440 tx=0x7f9da8009920 comp rx=0 tx=0).stop 2023-07-14T16:31:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.178+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3162945567 shutdown_connections 2023-07-14T16:31:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.178+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724858320 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.178+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x561724856bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.178+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3162945567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x561724857970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.178+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3162945567 >> 172.21.15.62:0/3162945567 conn(0x561724843be0 msgr2=0x561724848cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.179+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3162945567 shutdown_connections 2023-07-14T16:31:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.179+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3162945567 wait complete. 2023-07-14T16:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.179+0000 7f9dc9bbab80 1 -- start start 2023-07-14T16:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.179+0000 7f9dc9bbab80 1 Processor -- start 2023-07-14T16:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617246aa6b0 0x5617248600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617248570f0 0x561724860630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724867670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172473fa60 con 0x561724857eb0 2023-07-14T16:31:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172473f760 con 0x5617246aa6b0 2023-07-14T16:31:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617246238c0 con 0x5617248570f0 2023-07-14T16:31:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9dc9bbab80 10 client.0 _ll_register_callbacks cb 0x56172473d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9daf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617246aa6b0 0x5617248600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9daf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617246aa6b0 0x5617248600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:43012/0 (socket says 172.21.15.62:43012) 2023-07-14T16:31:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.180+0000 7f9daf7fe700 1 -- 172.21.15.62:0/4233916688 learned_addr learned my addr 172.21.15.62:0/4233916688 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9daffff700 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724867670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9daeffd700 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617248570f0 0x561724860630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9daf7fe700 1 -- 172.21.15.62:0/4233916688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617248570f0 msgr2=0x561724860630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617248570f0 0x561724860630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9daf7fe700 1 -- 172.21.15.62:0/4233916688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 msgr2=0x561724867670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724867670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9daf7fe700 1 -- 172.21.15.62:0/4233916688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da80090f0 con 0x5617246aa6b0 2023-07-14T16:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617246aa6b0 0x5617248600f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9da0099cc0 tx=0x7f9da000be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a74a59e969ff6d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9d8effd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.181+0000 7f9d8effd700 1 -- 172.21.15.62:0/4233916688 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da00996b0 con 0x5617246aa6b0 2023-07-14T16:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9d8effd700 1 -- 172.21.15.62:0/4233916688 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9da009dff0 con 0x5617246aa6b0 2023-07-14T16:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/4233916688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617246f9490 con 0x5617246aa6b0 2023-07-14T16:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9d8effd700 1 -- 172.21.15.62:0/4233916688 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9da00a2410 con 0x5617246aa6b0 2023-07-14T16:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9dc9bbab80 10 client.29432 fetch_fsmap learned FSMap version 30 2023-07-14T16:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/4233916688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5617245b6c00 con 0x5617246aa6b0 2023-07-14T16:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9d8effd700 1 -- 172.21.15.62:0/4233916688 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9da00b8030 con 0x5617246aa6b0 2023-07-14T16:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9d8effd700 1 -- 172.21.15.62:0/4233916688 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9da00a1f80 con 0x5617246aa6b0 2023-07-14T16:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9dc9bbab80 10 client.29432 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T16:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9dc9bbab80 10 client.29432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9dc9bbab80 20 client.29432 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.182+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56172486be00 0x56172486e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.183+0000 7f9dc9bbab80 4 client.29432 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.183+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/4233916688 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561724798e50 con 0x56172486be00 2023-07-14T16:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.183+0000 7f9daeffd700 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56172486be00 0x56172486e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.183+0000 7f9daeffd700 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56172486be00 0x56172486e210 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41244bef675a8980 server_cookie=636f3afa2956a881 in_seq=0 out_seq=0 2023-07-14T16:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.183+0000 7f9d8effd700 10 client.29432 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.184+0000 7f9d8effd700 1 -- 172.21.15.62:0/4233916688 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561724798e50 con 0x56172486be00 2023-07-14T16:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.184+0000 7f9d8effd700 10 client.29432 handle_command_reply: tid=0 2023-07-14T16:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.184+0000 7f9dc9bbab80 1 client.29432 shutdown 2023-07-14T16:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/4233916688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617246aa6b0 msgr2=0x5617248600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617246aa6b0 0x5617248600f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9da0099cc0 tx=0x7f9da000be40 comp rx=0 tx=0).stop 2023-07-14T16:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/4233916688 shutdown_connections 2023-07-14T16:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561724857eb0 0x561724867670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56172486be00 0x56172486e210 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617246aa6b0 0x5617248600f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/4233916688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617248570f0 0x561724860630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9d8effd700 0 client.29432 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/4233916688 >> 172.21.15.62:0/4233916688 conn(0x561724843be0 msgr2=0x5617248484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/4233916688 shutdown_connections 2023-07-14T16:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/4233916688 wait complete. 2023-07-14T16:31:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.185+0000 7f9dc9bbab80 20 client.29432 trim_cache size 0 max 16384 2023-07-14T16:31:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.204+0000 7f9dc9bbab80 1 Processor -- start 2023-07-14T16:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.204+0000 7f9dc9bbab80 1 -- start start 2023-07-14T16:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617246aa6b0 0x56172485acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x56172485b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561724857eb0 0x561724867630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561724798e80 con 0x5617246aa6b0 2023-07-14T16:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561724861310 con 0x5617248570f0 2023-07-14T16:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172485f5a0 con 0x561724857eb0 2023-07-14T16:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617246aa6b0 0x56172485acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617246aa6b0 0x56172485acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42512/0 (socket says 172.21.15.62:42512) 2023-07-14T16:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daf7fe700 1 -- 172.21.15.62:0/3811279679 learned_addr learned my addr 172.21.15.62:0/3811279679 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daffff700 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561724857eb0 0x561724867630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x56172485b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daf7fe700 1 -- 172.21.15.62:0/3811279679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561724857eb0 msgr2=0x561724867630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561724857eb0 0x561724867630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daf7fe700 1 -- 172.21.15.62:0/3811279679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 msgr2=0x56172485b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x56172485b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daf7fe700 1 -- 172.21.15.62:0/3811279679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617245f41f0 con 0x5617246aa6b0 2023-07-14T16:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.205+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x56172485b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.206+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617246aa6b0 0x56172485acc0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9da00964c0 tx=0x7f9da0099920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0c13db69a9cd9e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.206+0000 7f9d967fc700 1 -- 172.21.15.62:0/3811279679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da009dff0 con 0x5617246aa6b0 2023-07-14T16:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.206+0000 7f9d967fc700 1 -- 172.21.15.62:0/3811279679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9da00b8040 con 0x5617246aa6b0 2023-07-14T16:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.206+0000 7f9d967fc700 1 -- 172.21.15.62:0/3811279679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da00b8410 con 0x5617246aa6b0 2023-07-14T16:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.206+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3811279679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617246aa6b0 msgr2=0x56172485acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617246aa6b0 0x56172485acc0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9da00964c0 tx=0x7f9da0099920 comp rx=0 tx=0).stop 2023-07-14T16:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3811279679 shutdown_connections 2023-07-14T16:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561724857eb0 0x561724867630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617246aa6b0 0x56172485acc0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3811279679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617248570f0 0x56172485b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3811279679 >> 172.21.15.62:0/3811279679 conn(0x561724843be0 msgr2=0x561724848c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3811279679 shutdown_connections 2023-07-14T16:31:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3811279679 wait complete. 2023-07-14T16:31:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 -- start start 2023-07-14T16:31:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.207+0000 7f9dc9bbab80 1 Processor -- start 2023-07-14T16:31:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x5617248637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617248570f0 0x561724863d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9dc9bbab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561724857eb0 0x561724866b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561724867100 con 0x5617248570f0 2023-07-14T16:31:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561724867270 con 0x561724857eb0 2023-07-14T16:31:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9dc9bbab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561724867430 con 0x5617246aa6b0 2023-07-14T16:31:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9dc9bbab80 10 client.0 _ll_register_callbacks cb 0x7f9da0010710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9daeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617248570f0 0x561724863d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9daeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617248570f0 0x561724863d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42516/0 (socket says 172.21.15.62:42516) 2023-07-14T16:31:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9daeffd700 1 -- 172.21.15.62:0/3825388296 learned_addr learned my addr 172.21.15.62:0/3825388296 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x5617248637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9daeffd700 1 -- 172.21.15.62:0/3825388296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 msgr2=0x5617248637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.208+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x5617248637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9daeffd700 1 -- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561724857eb0 msgr2=0x561724866b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9daffff700 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561724857eb0 0x561724866b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561724857eb0 0x561724866b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9daeffd700 1 -- 172.21.15.62:0/3825388296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617245b6c00 con 0x5617248570f0 2023-07-14T16:31:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9daffff700 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561724857eb0 0x561724866b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:31:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9daeffd700 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617248570f0 0x561724863d00 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000c00 tx=0x7f9da4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d361d2c2a923dd09 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9d8effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9d8effd700 1 -- 172.21.15.62:0/3825388296 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da402b030 con 0x5617248570f0 2023-07-14T16:31:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3825388296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9da800f040 con 0x5617248570f0 2023-07-14T16:31:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9d8effd700 1 -- 172.21.15.62:0/3825388296 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9da400bac0 con 0x5617248570f0 2023-07-14T16:31:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.209+0000 7f9d8effd700 1 -- 172.21.15.62:0/3825388296 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9da401e070 con 0x5617248570f0 2023-07-14T16:31:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9dc9bbab80 10 client.19899 fetch_fsmap learned FSMap version 30 2023-07-14T16:31:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3825388296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5617245cc250 con 0x5617248570f0 2023-07-14T16:31:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9d8effd700 1 -- 172.21.15.62:0/3825388296 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9da800f040 con 0x5617248570f0 2023-07-14T16:31:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9d8effd700 1 -- 172.21.15.62:0/3825388296 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9da4033380 con 0x5617248570f0 2023-07-14T16:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9dc9bbab80 10 client.19899 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T16:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9dc9bbab80 10 client.19899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9dc9bbab80 20 client.19899 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5617248c0db0 0x5617248c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9dc9bbab80 4 client.19899 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.210+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3825388296 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56172475aff0 con 0x5617248c0db0 2023-07-14T16:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.211+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5617248c0db0 0x5617248c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.211+0000 7f9daf7fe700 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5617248c0db0 0x5617248c31c0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144b4970645bb9a server_cookie=92f4aa2dadd0dbef in_seq=0 out_seq=0 2023-07-14T16:31:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.211+0000 7f9d8effd700 10 client.19899 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.221+0000 7f9d8effd700 1 -- 172.21.15.62:0/3825388296 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56172475aff0 con 0x5617248c0db0 2023-07-14T16:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.221+0000 7f9d8effd700 10 client.19899 handle_command_reply: tid=0 2023-07-14T16:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.221+0000 7f9dc9bbab80 1 client.19899 shutdown 2023-07-14T16:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3825388296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617248570f0 msgr2=0x561724863d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617248570f0 0x561724863d00 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000c00 tx=0x7f9da4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3825388296 shutdown_connections 2023-07-14T16:31:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5617248c0db0 0x5617248c31c0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561724857eb0 0x561724866b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5617246aa6b0 0x5617248637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9d8effd700 0 client.19899 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 --2- 172.21.15.62:0/3825388296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617248570f0 0x561724863d00 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3825388296 >> 172.21.15.62:0/3825388296 conn(0x561724843be0 msgr2=0x56172485fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3825388296 shutdown_connections 2023-07-14T16:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 1 -- 172.21.15.62:0/3825388296 wait complete. 2023-07-14T16:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.222+0000 7f9dc9bbab80 20 client.19899 trim_cache size 0 max 16384 2023-07-14T16:31:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:31:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:31:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 -- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d980a11d0 msgr2=0x7f9d980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d980a11d0 0x7f9d980a3680 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9da40098e0 tx=0x7f9da4004740 comp rx=0 tx=0).stop 2023-07-14T16:31:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 -- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db00ffb10 msgr2=0x7f9db0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db00ffb10 0x7f9db0197a70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f9da000d980 tx=0x7f9da000dd40 comp rx=0 tx=0).stop 2023-07-14T16:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 -- 172.21.15.62:0/3891389277 shutdown_connections 2023-07-14T16:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9db0101690 0x7f9db0198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9db00ffb10 0x7f9db0197a70 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9d980a11d0 0x7f9d980a3680 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 --2- 172.21.15.62:0/3891389277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9db0100d00 0x7f9db0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 -- 172.21.15.62:0/3891389277 >> 172.21.15.62:0/3891389277 conn(0x7f9db00fb130 msgr2=0x7f9db00fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 -- 172.21.15.62:0/3891389277 shutdown_connections 2023-07-14T16:31:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:17.223+0000 7f9db61ff700 1 -- 172.21.15.62:0/3891389277 wait complete. 2023-07-14T16:31:17.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:16 smithi164 ceph-mon[97427]: pgmap v826: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 MiB/s wr, 12 op/s 2023-07-14T16:31:19.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:18 smithi062 ceph-mon[95543]: pgmap v827: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:31:19.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:18 smithi081 ceph-mon[127795]: pgmap v827: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:31:19.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:18 smithi164 ceph-mon[97427]: pgmap v827: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 15 MiB/s wr, 12 op/s 2023-07-14T16:31:20.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:19 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:20.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:19 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:20.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:19 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:31:21.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:20 smithi062 ceph-mon[95543]: pgmap v828: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:21.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:20 smithi081 ceph-mon[127795]: pgmap v828: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:21.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:20 smithi164 ceph-mon[97427]: pgmap v828: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:23.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:22 smithi062 ceph-mon[95543]: pgmap v829: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:31:23.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:22 smithi081 ceph-mon[127795]: pgmap v829: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:31:23.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:22 smithi164 ceph-mon[97427]: pgmap v829: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-14T16:31:25.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:24 smithi062 ceph-mon[95543]: pgmap v830: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:25.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:24 smithi081 ceph-mon[127795]: pgmap v830: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:25.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:24 smithi164 ceph-mon[97427]: pgmap v830: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:26.640 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:31:26.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.903+0000 7f08cd1ff700 1 -- 172.21.15.62:0/3869334624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c80744f0 msgr2=0x7f08c8072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.903+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/3869334624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c80744f0 0x7f08c8072940 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b0a0 tx=0x7f08b800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.903+0000 7f08cd1ff700 1 -- 172.21.15.62:0/3869334624 shutdown_connections 2023-07-14T16:31:26.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.903+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/3869334624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08c8107980 0x7f08c8109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.903+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/3869334624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c80744f0 0x7f08c8072940 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.903+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/3869334624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08c8072f10 0x7f08c8073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.903+0000 7f08cd1ff700 1 -- 172.21.15.62:0/3869334624 >> 172.21.15.62:0/3869334624 conn(0x7f08c80fb130 msgr2=0x7f08c80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:26.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.904+0000 7f08cd1ff700 1 -- 172.21.15.62:0/3869334624 shutdown_connections 2023-07-14T16:31:26.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.904+0000 7f08cd1ff700 1 -- 172.21.15.62:0/3869334624 wait complete. 2023-07-14T16:31:26.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.905+0000 7f08cd1ff700 1 Processor -- start 2023-07-14T16:31:26.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.905+0000 7f08cd1ff700 1 -- start start 2023-07-14T16:31:26.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.905+0000 7f08cd1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08c8072f10 0x7f08c8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.905+0000 7f08cd1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08c80744f0 0x7f08c8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.905+0000 7f08cd1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c8107980 0x7f08c819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.905+0000 7f08cd1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c819f670 con 0x7f08c8107980 2023-07-14T16:31:26.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.905+0000 7f08cd1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c819f7b0 con 0x7f08c8072f10 2023-07-14T16:31:26.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.905+0000 7f08cd1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c819f940 con 0x7f08c80744f0 2023-07-14T16:31:26.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c8107980 0x7f08c819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08c8072f10 0x7f08c8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08c8072f10 0x7f08c8197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36244/0 (socket says 172.21.15.62:36244) 2023-07-14T16:31:26.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c6d9d700 1 -- 172.21.15.62:0/2010601337 learned_addr learned my addr 172.21.15.62:0/2010601337 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:26.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c8107980 0x7f08c819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36726/0 (socket says 172.21.15.62:36726) 2023-07-14T16:31:26.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c659c700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08c80744f0 0x7f08c8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c759e700 1 -- 172.21.15.62:0/2010601337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08c80744f0 msgr2=0x7f08c8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c759e700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08c80744f0 0x7f08c8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c759e700 1 -- 172.21.15.62:0/2010601337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08c8072f10 msgr2=0x7f08c8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c759e700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08c8072f10 0x7f08c8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c759e700 1 -- 172.21.15.62:0/2010601337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009d00 con 0x7f08c8107980 2023-07-14T16:31:26.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08c8072f10 0x7f08c8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:26.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.906+0000 7f08c759e700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c8107980 0x7f08c819f0a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f08c000d720 tx=0x7f08c000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:26.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.907+0000 7f08affff700 1 -- 172.21.15.62:0/2010601337 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0004220 con 0x7f08c8107980 2023-07-14T16:31:26.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.907+0000 7f08cd1ff700 1 -- 172.21.15.62:0/2010601337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c819fc20 con 0x7f08c8107980 2023-07-14T16:31:26.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.907+0000 7f08affff700 1 -- 172.21.15.62:0/2010601337 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c0004380 con 0x7f08c8107980 2023-07-14T16:31:26.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.907+0000 7f08affff700 1 -- 172.21.15.62:0/2010601337 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0003aa0 con 0x7f08c8107980 2023-07-14T16:31:26.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.907+0000 7f08cd1ff700 1 -- 172.21.15.62:0/2010601337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c81a0180 con 0x7f08c8107980 2023-07-14T16:31:26.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.910+0000 7f08affff700 1 -- 172.21.15.62:0/2010601337 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f08c0003c00 con 0x7f08c8107980 2023-07-14T16:31:26.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.912+0000 7f08e0a34b80 1 Processor -- start 2023-07-14T16:31:26.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.912+0000 7f08e0a34b80 1 -- start start 2023-07-14T16:31:26.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd2a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08affff700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08b00a11d0 0x7f08b00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd2b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddc2fcb0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddc82a40 con 0x55d2ddd2ae20 2023-07-14T16:31:26.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddc7bed0 con 0x55d2ddd2bc70 2023-07-14T16:31:26.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08affff700 1 -- 172.21.15.62:0/2010601337 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08c00cc1e0 con 0x7f08c8107980 2023-07-14T16:31:26.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd2a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08b00a11d0 0x7f08b00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd2a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36736/0 (socket says 172.21.15.62:36736) 2023-07-14T16:31:26.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08c659c700 1 -- 172.21.15.62:0/152289726 learned_addr learned my addr 172.21.15.62:0/152289726 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:26.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08c759e700 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.913+0000 7f08c659c700 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd2b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08c659c700 1 -- 172.21.15.62:0/152289726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 msgr2=0x55d2ddd2c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:26.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08c659c700 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd2c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08c659c700 1 -- 172.21.15.62:0/152289726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 msgr2=0x55d2ddd2b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:26.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08c659c700 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd2b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08c659c700 1 -- 172.21.15.62:0/152289726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2ddaa9de0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08c659c700 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd2a8e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00bf40 tx=0x7f08bc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5ba81a0d853c7d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:26.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08b00a11d0 0x7f08b00a3680 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b830 tx=0x7f08b8006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:26.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08ad7fa700 1 -- 172.21.15.62:0/152289726 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc017070 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08ad7fa700 1 -- 172.21.15.62:0/152289726 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08bc0044c0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.914+0000 7f08ad7fa700 1 -- 172.21.15.62:0/152289726 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc003c60 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 -- 172.21.15.62:0/152289726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 msgr2=0x55d2ddd2a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd2a8e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00bf40 tx=0x7f08bc00b810 comp rx=0 tx=0).stop 2023-07-14T16:31:26.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 -- 172.21.15.62:0/152289726 shutdown_connections 2023-07-14T16:31:26.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd2c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd2a8e0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/152289726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd2b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 -- 172.21.15.62:0/152289726 >> 172.21.15.62:0/152289726 conn(0x55d2ddd1fda0 msgr2=0x55d2ddd251d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 -- 172.21.15.62:0/152289726 shutdown_connections 2023-07-14T16:31:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.915+0000 7f08e0a34b80 1 -- 172.21.15.62:0/152289726 wait complete. 2023-07-14T16:31:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.916+0000 7f08e0a34b80 1 -- start start 2023-07-14T16:31:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.916+0000 7f08e0a34b80 1 Processor -- start 2023-07-14T16:31:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd3b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddc1bac0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddc1b7c0 con 0x55d2ddd2ae20 2023-07-14T16:31:26.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddaff920 con 0x55d2ddd2bc70 2023-07-14T16:31:26.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08e0a34b80 10 client.0 _ll_register_callbacks cb 0x55d2ddc19760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:26.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08c6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08c6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd341a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36740/0 (socket says 172.21.15.62:36740) 2023-07-14T16:31:26.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08c6d9d700 1 -- 172.21.15.62:0/19272578 learned_addr learned my addr 172.21.15.62:0/19272578 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:26.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08c659c700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.917+0000 7f08c759e700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd3b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08c6d9d700 1 -- 172.21.15.62:0/19272578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 msgr2=0x55d2ddd3b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:26.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd3b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08c6d9d700 1 -- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 msgr2=0x55d2ddd346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:26.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08c6d9d700 1 -- 172.21.15.62:0/19272578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc0123f0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08c759e700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd3b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:26.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08c659c700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:26.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd341a0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b070 tx=0x7f08b80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa045918fedd5ed1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:26.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:26.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.918+0000 7f08aaff5700 1 -- 172.21.15.62:0/19272578 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800e040 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08e0a34b80 1 -- 172.21.15.62:0/19272578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2ddbd54f0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08aaff5700 1 -- 172.21.15.62:0/19272578 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08b8005450 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08aaff5700 1 -- 172.21.15.62:0/19272578 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08b8003e50 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08e0a34b80 10 client.19911 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:26.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08e0a34b80 1 -- 172.21.15.62:0/19272578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d2dda92c60 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08aaff5700 1 -- 172.21.15.62:0/19272578 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d2ddbd54f0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08aaff5700 1 -- 172.21.15.62:0/19272578 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08b8028ac0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08e0a34b80 10 client.19911 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:26.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08e0a34b80 10 client.19911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:26.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08e0a34b80 20 client.19911 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:26.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2ddd3fe00 0x55d2ddd42210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.919+0000 7f08e0a34b80 4 client.19911 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:26.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.920+0000 7f08e0a34b80 1 -- 172.21.15.62:0/19272578 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2ddc74eb0 con 0x55d2ddd3fe00 2023-07-14T16:31:26.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.920+0000 7f08c659c700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2ddd3fe00 0x55d2ddd42210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.921+0000 7f08aaff5700 10 client.19911 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.920+0000 7f08c659c700 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2ddd3fe00 0x55d2ddd42210 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56936cddf7547193 server_cookie=ebce8f645d19a491 in_seq=0 out_seq=0 2023-07-14T16:31:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.921+0000 7f08aaff5700 1 -- 172.21.15.62:0/19272578 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d2ddc74eb0 con 0x55d2ddd3fe00 2023-07-14T16:31:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.921+0000 7f08aaff5700 10 client.19911 handle_command_reply: tid=0 2023-07-14T16:31:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.921+0000 7f08e0a34b80 1 client.19911 shutdown 2023-07-14T16:31:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 -- 172.21.15.62:0/19272578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 msgr2=0x55d2ddd341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd341a0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b070 tx=0x7f08b80099e0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 -- 172.21.15.62:0/19272578 shutdown_connections 2023-07-14T16:31:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd3b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd341a0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2ddd3fe00 0x55d2ddd42210 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/19272578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08aaff5700 0 client.19911 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 -- 172.21.15.62:0/19272578 >> 172.21.15.62:0/19272578 conn(0x55d2ddd1fda0 msgr2=0x55d2ddd248b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 -- 172.21.15.62:0/19272578 shutdown_connections 2023-07-14T16:31:26.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 1 -- 172.21.15.62:0/19272578 wait complete. 2023-07-14T16:31:26.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.922+0000 7f08e0a34b80 20 client.19911 trim_cache size 0 max 16384 2023-07-14T16:31:26.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.944+0000 7f08e0a34b80 1 Processor -- start 2023-07-14T16:31:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.944+0000 7f08e0a34b80 1 -- start start 2023-07-14T16:31:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd32020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddd31ea0 con 0x55d2ddd2ae20 2023-07-14T16:31:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddd3fb80 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddd26270 con 0x55d2ddd2bc70 2023-07-14T16:31:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36754/0 (socket says 172.21.15.62:36754) 2023-07-14T16:31:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c659c700 1 -- 172.21.15.62:0/1473924807 learned_addr learned my addr 172.21.15.62:0/1473924807 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c759e700 1 --2- 172.21.15.62:0/1473924807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd32020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c659c700 1 -- 172.21.15.62:0/1473924807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 msgr2=0x55d2ddd32020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c659c700 1 --2- 172.21.15.62:0/1473924807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd32020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c659c700 1 -- 172.21.15.62:0/1473924807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 msgr2=0x55d2ddd3c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c659c700 1 --2- 172.21.15.62:0/1473924807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.945+0000 7f08c659c700 1 -- 172.21.15.62:0/1473924807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2ddad0250 con 0x55d2ddd2ae20 2023-07-14T16:31:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.946+0000 7f08c659c700 1 --2- 172.21.15.62:0/1473924807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b6e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0036c0 tx=0x7f08bc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48773e627765c0e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.946+0000 7f08adffb700 1 -- 172.21.15.62:0/1473924807 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc024040 con 0x55d2ddd2ae20 2023-07-14T16:31:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.946+0000 7f08adffb700 1 -- 172.21.15.62:0/1473924807 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08bc0120a0 con 0x55d2ddd2ae20 2023-07-14T16:31:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.946+0000 7f08adffb700 1 -- 172.21.15.62:0/1473924807 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc017040 con 0x55d2ddd2ae20 2023-07-14T16:31:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 -- 172.21.15.62:0/1473924807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2ae20 msgr2=0x55d2ddd3b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/1473924807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b6e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0036c0 tx=0x7f08bc014040 comp rx=0 tx=0).stop 2023-07-14T16:31:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 -- 172.21.15.62:0/1473924807 shutdown_connections 2023-07-14T16:31:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/1473924807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd32020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/1473924807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/1473924807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b6e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 -- 172.21.15.62:0/1473924807 >> 172.21.15.62:0/1473924807 conn(0x55d2ddd1fda0 msgr2=0x55d2ddd46b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:26.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 -- 172.21.15.62:0/1473924807 shutdown_connections 2023-07-14T16:31:26.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 -- 172.21.15.62:0/1473924807 wait complete. 2023-07-14T16:31:26.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.947+0000 7f08e0a34b80 1 -- start start 2023-07-14T16:31:26.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.948+0000 7f08e0a34b80 1 Processor -- start 2023-07-14T16:31:26.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.948+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.948+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.948+0000 7f08e0a34b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd37a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.948+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddd38010 con 0x55d2ddd2bc70 2023-07-14T16:31:26.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.948+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddd38150 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.948+0000 7f08e0a34b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ddd38310 con 0x55d2ddd2ae20 2023-07-14T16:31:26.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.948+0000 7f08e0a34b80 10 client.0 _ll_register_callbacks cb 0x7f08b8009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:26.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd37a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36280/0 (socket says 172.21.15.62:36280) 2023-07-14T16:31:26.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd37a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36770/0 (socket says 172.21.15.62:36770) 2023-07-14T16:31:26.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 -- 172.21.15.62:0/2471442795 learned_addr learned my addr 172.21.15.62:0/2471442795 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:26.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c659c700 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 -- 172.21.15.62:0/2471442795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2ae20 msgr2=0x55d2ddd3b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:26.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 -- 172.21.15.62:0/2471442795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2bc70 msgr2=0x55d2ddd37a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:26.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd37a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 -- 172.21.15.62:0/2471442795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2ddaa9de0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c759e700 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd37a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:26.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c659c700 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:26.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08c6d9d700 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3afa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f08b800a010 tx=0x7f08b8007520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597ac6dcb141f544 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:26.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.949+0000 7f08aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:31:26.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.950+0000 7f08aaff5700 1 -- 172.21.15.62:0/2471442795 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800e070 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.929 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.950+0000 7f08aaff5700 1 -- 172.21.15.62:0/2471442795 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08b801bc60 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.950+0000 7f08e0a34b80 1 -- 172.21.15.62:0/2471442795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08bc0044c0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.950+0000 7f08aaff5700 1 -- 172.21.15.62:0/2471442795 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08b801be20 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.950+0000 7f08e0a34b80 10 client.29450 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:26.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.950+0000 7f08e0a34b80 1 -- 172.21.15.62:0/2471442795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d2ddaa82b0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08aaff5700 1 -- 172.21.15.62:0/2471442795 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08bc0044c0 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08aaff5700 1 -- 172.21.15.62:0/2471442795 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08b8005410 con 0x55d2ddd2a4d0 2023-07-14T16:31:26.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08e0a34b80 10 client.29450 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:26.930 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08e0a34b80 10 client.29450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:26.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08e0a34b80 20 client.29450 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:26.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2ddd94d90 0x55d2ddd971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:26.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08e0a34b80 4 client.29450 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:31:26.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08e0a34b80 1 -- 172.21.15.62:0/2471442795 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d2ddc37050 con 0x55d2ddd94d90 2023-07-14T16:31:26.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.951+0000 7f08c659c700 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2ddd94d90 0x55d2ddd971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:26.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08c659c700 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2ddd94d90 0x55d2ddd971a0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c750d729fb81b16 server_cookie=e166142263d2f12a in_seq=0 out_seq=0 2023-07-14T16:31:26.931 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08aaff5700 10 client.29450 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:26.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08aaff5700 1 -- 172.21.15.62:0/2471442795 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d2ddc37050 con 0x55d2ddd94d90 2023-07-14T16:31:26.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08aaff5700 10 client.29450 handle_command_reply: tid=0 2023-07-14T16:31:26.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 client.29450 shutdown 2023-07-14T16:31:26.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 -- 172.21.15.62:0/2471442795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 msgr2=0x55d2ddd3afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3afa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f08b800a010 tx=0x7f08b8007520 comp rx=0 tx=0).stop 2023-07-14T16:31:26.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 -- 172.21.15.62:0/2471442795 shutdown_connections 2023-07-14T16:31:26.932 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d2ddd2bc70 0x55d2ddd37a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d2ddd94d90 0x55d2ddd971a0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d2ddd2a4d0 0x55d2ddd3afa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 --2- 172.21.15.62:0/2471442795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d2ddd2ae20 0x55d2ddd3b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08aaff5700 0 client.29450 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:26.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.952+0000 7f08e0a34b80 1 -- 172.21.15.62:0/2471442795 >> 172.21.15.62:0/2471442795 conn(0x55d2ddd1fda0 msgr2=0x55d2ddd32020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:26.933 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08e0a34b80 1 -- 172.21.15.62:0/2471442795 shutdown_connections 2023-07-14T16:31:26.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08e0a34b80 1 -- 172.21.15.62:0/2471442795 wait complete. 2023-07-14T16:31:26.934 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08e0a34b80 20 client.29450 trim_cache size 0 max 16384 2023-07-14T16:31:26.934 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:31:26.935 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:31:26.935 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:31:26.935 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:31:26.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08cd1ff700 1 -- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08b00a11d0 msgr2=0x7f08b00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08b00a11d0 0x7f08b00a3680 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b830 tx=0x7f08b8006210 comp rx=0 tx=0).stop 2023-07-14T16:31:26.936 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08cd1ff700 1 -- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c8107980 msgr2=0x7f08c819f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:26.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c8107980 0x7f08c819f0a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f08c000d720 tx=0x7f08c000da30 comp rx=0 tx=0).stop 2023-07-14T16:31:26.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08cd1ff700 1 -- 172.21.15.62:0/2010601337 shutdown_connections 2023-07-14T16:31:26.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c8107980 0x7f08c819f0a0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f08b00a11d0 0x7f08b00a3680 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.953+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08c8072f10 0x7f08c8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.954+0000 7f08cd1ff700 1 --2- 172.21.15.62:0/2010601337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08c80744f0 0x7f08c8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:26.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.954+0000 7f08cd1ff700 1 -- 172.21.15.62:0/2010601337 >> 172.21.15.62:0/2010601337 conn(0x7f08c80fb130 msgr2=0x7f08c80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:26.937 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.954+0000 7f08cd1ff700 1 -- 172.21.15.62:0/2010601337 shutdown_connections 2023-07-14T16:31:26.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:26.954+0000 7f08cd1ff700 1 -- 172.21.15.62:0/2010601337 wait complete. 2023-07-14T16:31:26.946 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:31:26.946 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:31:27.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:26 smithi062 ceph-mon[95543]: pgmap v831: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:31:27.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:26 smithi081 ceph-mon[127795]: pgmap v831: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:31:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.208+0000 7f010b59e700 1 -- 172.21.15.62:0/4165789081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c1056a0 msgr2=0x7f010c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.208+0000 7f010b59e700 1 --2- 172.21.15.62:0/4165789081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c1056a0 0x7f010c107b00 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f010400b0a0 tx=0x7f010400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.209+0000 7f010b59e700 1 -- 172.21.15.62:0/4165789081 shutdown_connections 2023-07-14T16:31:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.209+0000 7f010b59e700 1 --2- 172.21.15.62:0/4165789081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c1056a0 0x7f010c107b00 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.209+0000 7f010b59e700 1 --2- 172.21.15.62:0/4165789081 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010c100460 0x7f010c102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.209+0000 7f010b59e700 1 --2- 172.21.15.62:0/4165789081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f010c102d80 0x7f010c105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.209+0000 7f010b59e700 1 -- 172.21.15.62:0/4165789081 >> 172.21.15.62:0/4165789081 conn(0x7f010c0f9ee0 msgr2=0x7f010c0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:27.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.209+0000 7f010b59e700 1 -- 172.21.15.62:0/4165789081 shutdown_connections 2023-07-14T16:31:27.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.209+0000 7f010b59e700 1 -- 172.21.15.62:0/4165789081 wait complete. 2023-07-14T16:31:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.210+0000 7f010b59e700 1 Processor -- start 2023-07-14T16:31:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010b59e700 1 -- start start 2023-07-14T16:31:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010b59e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c100460 0x7f010c1954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010b59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f010c102d80 0x7f010c195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010b59e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010c1056a0 0x7f010c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010b59e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19d070 con 0x7f010c100460 2023-07-14T16:31:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010b59e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19d1b0 con 0x7f010c102d80 2023-07-14T16:31:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010b59e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19d340 con 0x7f010c1056a0 2023-07-14T16:31:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c100460 0x7f010c1954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.211+0000 7f010a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c100460 0x7f010c1954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36780/0 (socket says 172.21.15.62:36780) 2023-07-14T16:31:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f010a59c700 1 -- 172.21.15.62:0/3175570845 learned_addr learned my addr 172.21.15.62:0/3175570845 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:27.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f010ad9d700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010c1056a0 0x7f010c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f0109d9b700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f010c102d80 0x7f010c195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f010a59c700 1 -- 172.21.15.62:0/3175570845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010c1056a0 msgr2=0x7f010c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f010a59c700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010c1056a0 0x7f010c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f010a59c700 1 -- 172.21.15.62:0/3175570845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f010c102d80 msgr2=0x7f010c195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f010a59c700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f010c102d80 0x7f010c195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f010a59c700 1 -- 172.21.15.62:0/3175570845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104009d00 con 0x7f010c100460 2023-07-14T16:31:27.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.212+0000 7f010a59c700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c100460 0x7f010c1954f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f00fc012db0 tx=0x7f00fc00c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:27.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.213+0000 7f00fb7fe700 1 -- 172.21.15.62:0/3175570845 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc009790 con 0x7f010c100460 2023-07-14T16:31:27.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.213+0000 7f00fb7fe700 1 -- 172.21.15.62:0/3175570845 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00fc01a8c0 con 0x7f010c100460 2023-07-14T16:31:27.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.213+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c0fe2b0 con 0x7f010c100460 2023-07-14T16:31:27.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.213+0000 7f00fb7fe700 1 -- 172.21.15.62:0/3175570845 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc018860 con 0x7f010c100460 2023-07-14T16:31:27.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.213+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c0fe810 con 0x7f010c100460 2023-07-14T16:31:27.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.214+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c0fba50 con 0x7f010c100460 2023-07-14T16:31:27.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.217+0000 7f00fb7fe700 1 -- 172.21.15.62:0/3175570845 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f00fc01a430 con 0x7f010c100460 2023-07-14T16:31:27.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.219+0000 7f00fb7fe700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f00f40a12c0 0x7f00f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.219+0000 7f00fb7fe700 1 -- 172.21.15.62:0/3175570845 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f00fc0d0400 con 0x7f010c100460 2023-07-14T16:31:27.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.219+0000 7f0109d9b700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f00f40a12c0 0x7f00f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.220+0000 7f0109d9b700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f00f40a12c0 0x7f00f40a3770 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0100009c60 tx=0x7f0100009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:27.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.225+0000 7f00fb7fe700 1 -- 172.21.15.62:0/3175570845 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f00fc053790 con 0x7f010c100460 2023-07-14T16:31:27.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:26 smithi164 ceph-mon[97427]: pgmap v831: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 MiB/s wr, 15 op/s 2023-07-14T16:31:27.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.593+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f010c02cab0 con 0x7f010c100460 2023-07-14T16:31:27.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.593+0000 7f00fb7fe700 1 -- 172.21.15.62:0/3175570845 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10828 (secure 0 0 0) 0x7f00fc099b00 con 0x7f010c100460 2023-07-14T16:31:27.567 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:31:27.567 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:31:18.877417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:31:27.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f00f40a12c0 msgr2=0x7f00f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f00f40a12c0 0x7f00f40a3770 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0100009c60 tx=0x7f0100009260 comp rx=0 tx=0).stop 2023-07-14T16:31:27.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c100460 msgr2=0x7f010c1954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c100460 0x7f010c1954f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f00fc012db0 tx=0x7f00fc00c5b0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 shutdown_connections 2023-07-14T16:31:27.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010c1056a0 0x7f010c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f010c100460 0x7f010c1954f0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f00f40a12c0 0x7f00f40a3770 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 --2- 172.21.15.62:0/3175570845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f010c102d80 0x7f010c195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 >> 172.21.15.62:0/3175570845 conn(0x7f010c0f9ee0 msgr2=0x7f010c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:27.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.596+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 shutdown_connections 2023-07-14T16:31:27.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.597+0000 7f010b59e700 1 -- 172.21.15.62:0/3175570845 wait complete. 2023-07-14T16:31:27.571 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2023-07-14T16:31:27.589 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:31:27.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 -- 172.21.15.62:0/1654279853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 msgr2=0x7f11e8102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 --2- 172.21.15.62:0/1654279853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 0x7f11e8102940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f11e40098f0 tx=0x7f11e4009c00 comp rx=0 tx=0).stop 2023-07-14T16:31:27.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 -- 172.21.15.62:0/1654279853 shutdown_connections 2023-07-14T16:31:27.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 --2- 172.21.15.62:0/1654279853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e81057a0 0x7f11e8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 --2- 172.21.15.62:0/1654279853 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 0x7f11e8102940 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 --2- 172.21.15.62:0/1654279853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11e8102e80 0x7f11e8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 -- 172.21.15.62:0/1654279853 >> 172.21.15.62:0/1654279853 conn(0x7f11e80fa000 msgr2=0x7f11e80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:27.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 -- 172.21.15.62:0/1654279853 shutdown_connections 2023-07-14T16:31:27.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.868+0000 7f11efdff700 1 -- 172.21.15.62:0/1654279853 wait complete. 2023-07-14T16:31:27.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.869+0000 7f11efdff700 1 Processor -- start 2023-07-14T16:31:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11efdff700 1 -- start start 2023-07-14T16:31:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11efdff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 0x7f11e8195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11efdff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e8102e80 0x7f11e8195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11efdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11e81057a0 0x7f11e819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11efdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819d310 con 0x7f11e81057a0 2023-07-14T16:31:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11efdff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819d450 con 0x7f11e8102e80 2023-07-14T16:31:27.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11efdff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819d5e0 con 0x7f11e8100560 2023-07-14T16:31:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11edb9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 0x7f11e8195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11ee39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11e81057a0 0x7f11e819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11edb9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 0x7f11e8195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:59778/0 (socket says 172.21.15.62:59778) 2023-07-14T16:31:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.870+0000 7f11edb9b700 1 -- 172.21.15.62:0/1751501753 learned_addr learned my addr 172.21.15.62:0/1751501753 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11ed39a700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e8102e80 0x7f11e8195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11edb9b700 1 -- 172.21.15.62:0/1751501753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e8102e80 msgr2=0x7f11e8195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11edb9b700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e8102e80 0x7f11e8195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.846 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11edb9b700 1 -- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11e81057a0 msgr2=0x7f11e819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11edb9b700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11e81057a0 0x7f11e819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11edb9b700 1 -- 172.21.15.62:0/1751501753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d80095f0 con 0x7f11e8100560 2023-07-14T16:31:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11ed39a700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e8102e80 0x7f11e8195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11ee39c700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11e81057a0 0x7f11e819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11edb9b700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 0x7f11e8195790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f11e400b790 tx=0x7f11e400b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.871+0000 7f11deffd700 1 -- 172.21.15.62:0/1751501753 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e401c070 con 0x7f11e8100560 2023-07-14T16:31:27.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.872+0000 7f11deffd700 1 -- 172.21.15.62:0/1751501753 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11e400f7b0 con 0x7f11e8100560 2023-07-14T16:31:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.872+0000 7f11efdff700 1 -- 172.21.15.62:0/1751501753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e4009520 con 0x7f11e8100560 2023-07-14T16:31:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.872+0000 7f11deffd700 1 -- 172.21.15.62:0/1751501753 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4004a50 con 0x7f11e8100560 2023-07-14T16:31:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.872+0000 7f11efdff700 1 -- 172.21.15.62:0/1751501753 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e819daf0 con 0x7f11e8100560 2023-07-14T16:31:27.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.875+0000 7f11deffd700 1 -- 172.21.15.62:0/1751501753 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f11e4020070 con 0x7f11e8100560 2023-07-14T16:31:27.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f12037e3b80 1 Processor -- start 2023-07-14T16:31:27.851 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f12037e3b80 1 -- start start 2023-07-14T16:31:27.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f11deffd700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f11d40a1150 0x7f11d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f11deffd700 1 -- 172.21.15.62:0/1751501753 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f11e40cd750 con 0x7f11e8100560 2023-07-14T16:31:27.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 0x55e92b124cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b125230 0x55e92b125af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b1264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f12037e3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b025cb0 con 0x55e92b126080 2023-07-14T16:31:27.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f12037e3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b078a40 con 0x55e92b1248e0 2023-07-14T16:31:27.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.877+0000 7f12037e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b071ed0 con 0x55e92b125230 2023-07-14T16:31:27.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f11d40a1150 0x7f11d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b1264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11edb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 0x55e92b124cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b1264f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36810/0 (socket says 172.21.15.62:36810) 2023-07-14T16:31:27.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 -- 172.21.15.62:0/1811750212 learned_addr learned my addr 172.21.15.62:0/1811750212 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:27.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ed39a700 1 --2- 172.21.15.62:0/1811750212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b125230 0x55e92b125af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 -- 172.21.15.62:0/1811750212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b125230 msgr2=0x55e92b125af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:27.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 --2- 172.21.15.62:0/1811750212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b125230 0x55e92b125af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 -- 172.21.15.62:0/1811750212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 msgr2=0x55e92b124cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:27.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 --2- 172.21.15.62:0/1811750212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 0x55e92b124cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.878+0000 7f11ee39c700 1 -- 172.21.15.62:0/1811750212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e92ae9fde0 con 0x55e92b126080 2023-07-14T16:31:27.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.879+0000 7f11ee39c700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f11d40a1150 0x7f11d40a3600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f11e0008cc0 tx=0x7f11e00093b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:27.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.879+0000 7f11ee39c700 1 --2- 172.21.15.62:0/1811750212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b1264f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009ce0 tx=0x7f11e001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3a2732a0dfd1eb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:27.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.879+0000 7f11d3fff700 1 -- 172.21.15.62:0/1811750212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0008260 con 0x55e92b126080 2023-07-14T16:31:27.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.879+0000 7f11d3fff700 1 -- 172.21.15.62:0/1811750212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11e000f040 con 0x55e92b126080 2023-07-14T16:31:27.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.879+0000 7f11d3fff700 1 -- 172.21.15.62:0/1811750212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0007760 con 0x55e92b126080 2023-07-14T16:31:27.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 -- 172.21.15.62:0/1811750212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 msgr2=0x55e92b1264f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 --2- 172.21.15.62:0/1811750212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b1264f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009ce0 tx=0x7f11e001d000 comp rx=0 tx=0).stop 2023-07-14T16:31:27.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 -- 172.21.15.62:0/1811750212 shutdown_connections 2023-07-14T16:31:27.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 --2- 172.21.15.62:0/1811750212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b1264f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 --2- 172.21.15.62:0/1811750212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 0x55e92b124cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 --2- 172.21.15.62:0/1811750212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b125230 0x55e92b125af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 -- 172.21.15.62:0/1811750212 >> 172.21.15.62:0/1811750212 conn(0x55e92b115da0 msgr2=0x55e92b11b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:27.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 -- 172.21.15.62:0/1811750212 shutdown_connections 2023-07-14T16:31:27.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.880+0000 7f12037e3b80 1 -- 172.21.15.62:0/1811750212 wait complete. 2023-07-14T16:31:27.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.881+0000 7f12037e3b80 1 -- start start 2023-07-14T16:31:27.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.881+0000 7f12037e3b80 1 Processor -- start 2023-07-14T16:31:27.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.881+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 0x55e92b12e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.881+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b12ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.881+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b126080 0x55e92b135ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f12037e3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b011ac0 con 0x55e92b125230 2023-07-14T16:31:27.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f12037e3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b0117c0 con 0x55e92b1248e0 2023-07-14T16:31:27.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f12037e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92aef5920 con 0x55e92b126080 2023-07-14T16:31:27.860 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f12037e3b80 10 client.0 _ll_register_callbacks cb 0x55e92b00f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:27.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ed39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b12ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ed39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b12ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36820/0 (socket says 172.21.15.62:36820) 2023-07-14T16:31:27.861 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ed39a700 1 -- 172.21.15.62:0/2451455293 learned_addr learned my addr 172.21.15.62:0/2451455293 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ee39c700 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b126080 0x55e92b135ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11edb9b700 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 0x55e92b12e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ed39a700 1 -- 172.21.15.62:0/2451455293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b126080 msgr2=0x55e92b135ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b126080 0x55e92b135ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ed39a700 1 -- 172.21.15.62:0/2451455293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 msgr2=0x55e92b12e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 0x55e92b12e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.882+0000 7f11ed39a700 1 -- 172.21.15.62:0/2451455293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e0008e00 con 0x55e92b125230 2023-07-14T16:31:27.862 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.883+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b12ea90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f11d8013960 tx=0x7f11d8013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8ac4be56cbc224 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:27.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.883+0000 7f11d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:27.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.883+0000 7f11d17fa700 1 -- 172.21.15.62:0/2451455293 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8009a10 con 0x55e92b125230 2023-07-14T16:31:27.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.883+0000 7f12037e3b80 1 -- 172.21.15.62:0/2451455293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e92afcb4f0 con 0x55e92b125230 2023-07-14T16:31:27.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.883+0000 7f11d17fa700 1 -- 172.21.15.62:0/2451455293 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f11d8011b20 con 0x55e92b125230 2023-07-14T16:31:27.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.883+0000 7f11d17fa700 1 -- 172.21.15.62:0/2451455293 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d8011ce0 con 0x55e92b125230 2023-07-14T16:31:27.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.883+0000 7f12037e3b80 10 client.19935 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:27.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.883+0000 7f12037e3b80 1 -- 172.21.15.62:0/2451455293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e92ae88c60 con 0x55e92b125230 2023-07-14T16:31:27.863 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.884+0000 7f11d17fa700 1 -- 172.21.15.62:0/2451455293 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e92afcb4f0 con 0x55e92b125230 2023-07-14T16:31:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.884+0000 7f11d17fa700 1 -- 172.21.15.62:0/2451455293 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11d801d3f0 con 0x55e92b125230 2023-07-14T16:31:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.884+0000 7f12037e3b80 10 client.19935 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.884+0000 7f12037e3b80 10 client.19935 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:31:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.884+0000 7f12037e3b80 20 client.19935 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.884+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e92b13a390 0x55e92b13c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.884+0000 7f12037e3b80 4 client.19935 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.884+0000 7f12037e3b80 1 -- 172.21.15.62:0/2451455293 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e92b06aeb0 con 0x55e92b13a390 2023-07-14T16:31:27.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.885+0000 7f11edb9b700 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e92b13a390 0x55e92b13c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.885+0000 7f11edb9b700 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e92b13a390 0x55e92b13c7a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58971bb8eedf0ff2 server_cookie=ee1da9060217030 in_seq=0 out_seq=0 2023-07-14T16:31:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.885+0000 7f11d17fa700 10 client.19935 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:31:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f11d17fa700 1 -- 172.21.15.62:0/2451455293 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e92b06aeb0 con 0x55e92b13a390 2023-07-14T16:31:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f11d17fa700 10 client.19935 handle_command_reply: tid=0 2023-07-14T16:31:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 client.19935 shutdown 2023-07-14T16:31:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 -- 172.21.15.62:0/2451455293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 msgr2=0x55e92b12ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b12ea90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f11d8013960 tx=0x7f11d8013c70 comp rx=0 tx=0).stop 2023-07-14T16:31:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 -- 172.21.15.62:0/2451455293 shutdown_connections 2023-07-14T16:31:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e92b13a390 0x55e92b13c7a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b126080 0x55e92b135ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b1248e0 0x55e92b12e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f11d17fa700 0 client.19935 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:31:27.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2451455293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b12ea90 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 -- 172.21.15.62:0/2451455293 >> 172.21.15.62:0/2451455293 conn(0x55e92b115da0 msgr2=0x55e92b11a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:27.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 -- 172.21.15.62:0/2451455293 shutdown_connections 2023-07-14T16:31:27.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.886+0000 7f12037e3b80 1 -- 172.21.15.62:0/2451455293 wait complete. 2023-07-14T16:31:27.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.887+0000 7f12037e3b80 20 client.19935 trim_cache size 0 max 16384 2023-07-14T16:31:27.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:27 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3175570845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:31:27.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.903+0000 7f12037e3b80 1 Processor -- start 2023-07-14T16:31:27.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.903+0000 7f12037e3b80 1 -- start start 2023-07-14T16:31:27.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.903+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 0x55e92b12d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.903+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b125230 0x55e92b135a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b12c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f12037e3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b126880 con 0x55e92b126080 2023-07-14T16:31:27.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f12037e3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b12c250 con 0x55e92b125230 2023-07-14T16:31:27.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f12037e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b13a090 con 0x55e92b1248e0 2023-07-14T16:31:27.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ed39a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b125230 0x55e92b135a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ed39a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b125230 0x55e92b135a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36346/0 (socket says 172.21.15.62:36346) 2023-07-14T16:31:27.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ed39a700 1 -- 172.21.15.62:0/2008992584 learned_addr learned my addr 172.21.15.62:0/2008992584 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:27.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ee39c700 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b12c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11edb9b700 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 0x55e92b12d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ed39a700 1 -- 172.21.15.62:0/2008992584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 msgr2=0x55e92b12d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:27.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 0x55e92b12d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ed39a700 1 -- 172.21.15.62:0/2008992584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 msgr2=0x55e92b12c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:27.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b12c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ed39a700 1 -- 172.21.15.62:0/2008992584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e92aec6250 con 0x55e92b125230 2023-07-14T16:31:27.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.904+0000 7f11ee39c700 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b12c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:27.883 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.905+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b125230 0x55e92b135a90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f11d8005a60 tx=0x7f11d8009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99fae35164dad38c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:27.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.905+0000 7f11dcbff700 1 -- 172.21.15.62:0/2008992584 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d801dbc0 con 0x55e92b125230 2023-07-14T16:31:27.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.905+0000 7f11dcbff700 1 -- 172.21.15.62:0/2008992584 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11d801d4f0 con 0x55e92b125230 2023-07-14T16:31:27.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.905+0000 7f11dcbff700 1 -- 172.21.15.62:0/2008992584 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8028760 con 0x55e92b125230 2023-07-14T16:31:27.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.906+0000 7f12037e3b80 1 -- 172.21.15.62:0/2008992584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b125230 msgr2=0x55e92b135a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.906+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b125230 0x55e92b135a90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f11d8005a60 tx=0x7f11d8009a90 comp rx=0 tx=0).stop 2023-07-14T16:31:27.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.906+0000 7f12037e3b80 1 -- 172.21.15.62:0/2008992584 shutdown_connections 2023-07-14T16:31:27.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.906+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b126080 0x55e92b12c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.906+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 0x55e92b12d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.906+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2008992584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b125230 0x55e92b135a90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.906+0000 7f12037e3b80 1 -- 172.21.15.62:0/2008992584 >> 172.21.15.62:0/2008992584 conn(0x55e92b115da0 msgr2=0x55e92b141100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:27.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.906+0000 7f12037e3b80 1 -- 172.21.15.62:0/2008992584 shutdown_connections 2023-07-14T16:31:27.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.907+0000 7f12037e3b80 1 -- 172.21.15.62:0/2008992584 wait complete. 2023-07-14T16:31:27.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.907+0000 7f12037e3b80 1 -- start start 2023-07-14T16:31:27.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.908+0000 7f12037e3b80 1 Processor -- start 2023-07-14T16:31:27.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.909+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 0x55e92b131c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.909+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b132160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.909+0000 7f12037e3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b126080 0x55e92b135180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.909+0000 7f12037e3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b1356c0 con 0x55e92b125230 2023-07-14T16:31:27.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.909+0000 7f12037e3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b135830 con 0x55e92b126080 2023-07-14T16:31:27.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.909+0000 7f12037e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92b1359f0 con 0x55e92b1248e0 2023-07-14T16:31:27.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.909+0000 7f12037e3b80 10 client.0 _ll_register_callbacks cb 0x7f11d8009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:27.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.909+0000 7f11ed39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b132160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ed39a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b132160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36836/0 (socket says 172.21.15.62:36836) 2023-07-14T16:31:27.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ed39a700 1 -- 172.21.15.62:0/2597471055 learned_addr learned my addr 172.21.15.62:0/2597471055 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:27.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11edb9b700 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 0x55e92b131c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ee39c700 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b126080 0x55e92b135180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ed39a700 1 -- 172.21.15.62:0/2597471055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 msgr2=0x55e92b131c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:27.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 0x55e92b131c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ed39a700 1 -- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b126080 msgr2=0x55e92b135180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:27.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b126080 0x55e92b135180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ed39a700 1 -- 172.21.15.62:0/2597471055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e92ae88c60 con 0x55e92b125230 2023-07-14T16:31:27.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.910+0000 7f11ee39c700 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b126080 0x55e92b135180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:31:27.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.911+0000 7f11ed39a700 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b132160 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f11d801d7f0 tx=0x7f11d8013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7ca2f28f69e07a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:27.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.911+0000 7f11d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:27.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.911+0000 7f11d17fa700 1 -- 172.21.15.62:0/2597471055 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d80040c0 con 0x55e92b125230 2023-07-14T16:31:27.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.911+0000 7f11d17fa700 1 -- 172.21.15.62:0/2597471055 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f11d8004220 con 0x55e92b125230 2023-07-14T16:31:27.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.911+0000 7f12037e3b80 1 -- 172.21.15.62:0/2597471055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11e000f040 con 0x55e92b125230 2023-07-14T16:31:27.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.911+0000 7f11d17fa700 1 -- 172.21.15.62:0/2597471055 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d80043e0 con 0x55e92b125230 2023-07-14T16:31:27.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.911+0000 7f12037e3b80 10 client.19941 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:27.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.911+0000 7f12037e3b80 1 -- 172.21.15.62:0/2597471055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e92ae9e2b0 con 0x55e92b125230 2023-07-14T16:31:27.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.912+0000 7f11d17fa700 1 -- 172.21.15.62:0/2597471055 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11e000f040 con 0x55e92b125230 2023-07-14T16:31:27.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.912+0000 7f11d17fa700 1 -- 172.21.15.62:0/2597471055 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11d802e030 con 0x55e92b125230 2023-07-14T16:31:27.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.912+0000 7f12037e3b80 10 client.19941 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:27.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.912+0000 7f12037e3b80 10 client.19941 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:31:27.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.912+0000 7f12037e3b80 20 client.19941 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:27.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.912+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e92b18f290 0x55e92b1916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:27.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.912+0000 7f12037e3b80 4 client.19941 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:31:27.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.912+0000 7f12037e3b80 1 -- 172.21.15.62:0/2597471055 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e92b02d050 con 0x55e92b18f290 2023-07-14T16:31:27.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.913+0000 7f11edb9b700 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e92b18f290 0x55e92b1916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:27.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.914+0000 7f11edb9b700 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e92b18f290 0x55e92b1916a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d172b4a33d9b4918 server_cookie=e39d6883d7ff3c1f in_seq=0 out_seq=0 2023-07-14T16:31:27.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.914+0000 7f11d17fa700 10 client.19941 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:31:27.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.914+0000 7f11d17fa700 1 -- 172.21.15.62:0/2597471055 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e92b02d050 con 0x55e92b18f290 2023-07-14T16:31:27.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.914+0000 7f11d17fa700 10 client.19941 handle_command_reply: tid=0 2023-07-14T16:31:27.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.914+0000 7f12037e3b80 1 client.19941 shutdown 2023-07-14T16:31:27.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.914+0000 7f12037e3b80 1 -- 172.21.15.62:0/2597471055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 msgr2=0x55e92b132160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.914+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b132160 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f11d801d7f0 tx=0x7f11d8013c70 comp rx=0 tx=0).stop 2023-07-14T16:31:27.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 1 -- 172.21.15.62:0/2597471055 shutdown_connections 2023-07-14T16:31:27.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55e92b18f290 0x55e92b1916a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e92b126080 0x55e92b135180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e92b1248e0 0x55e92b131c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 1 --2- 172.21.15.62:0/2597471055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e92b125230 0x55e92b132160 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 1 -- 172.21.15.62:0/2597471055 >> 172.21.15.62:0/2597471055 conn(0x55e92b115da0 msgr2=0x55e92b12d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:27.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f11d17fa700 0 client.19941 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:31:27.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 1 -- 172.21.15.62:0/2597471055 shutdown_connections 2023-07-14T16:31:27.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 1 -- 172.21.15.62:0/2597471055 wait complete. 2023-07-14T16:31:27.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.915+0000 7f12037e3b80 20 client.19941 trim_cache size 0 max 16384 2023-07-14T16:31:27.895 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:31:27.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 -- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f11d40a1150 msgr2=0x7f11d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f11d40a1150 0x7f11d40a3600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f11e0008cc0 tx=0x7f11e00093b0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 -- 172.21.15.62:0/1751501753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 msgr2=0x7f11e8195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:27.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 0x7f11e8195790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f11e400b790 tx=0x7f11e400b7c0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 -- 172.21.15.62:0/1751501753 shutdown_connections 2023-07-14T16:31:27.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11e81057a0 0x7f11e819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11e8100560 0x7f11e8195790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f11d40a1150 0x7f11d40a3600 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 --2- 172.21.15.62:0/1751501753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e8102e80 0x7f11e8195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:27.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 -- 172.21.15.62:0/1751501753 >> 172.21.15.62:0/1751501753 conn(0x7f11e80fa000 msgr2=0x7f11e81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:27.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 -- 172.21.15.62:0/1751501753 shutdown_connections 2023-07-14T16:31:27.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:27.916+0000 7f11efdff700 1 -- 172.21.15.62:0/1751501753 wait complete. 2023-07-14T16:31:27.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:31:28.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 -- 172.21.15.62:0/383645180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 msgr2=0x7f2510105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 --2- 172.21.15.62:0/383645180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 0x7f2510105300 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2500009990 tx=0x7f2500009ca0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 -- 172.21.15.62:0/383645180 shutdown_connections 2023-07-14T16:31:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 --2- 172.21.15.62:0/383645180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2510105840 0x7f2510107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 --2- 172.21.15.62:0/383645180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2510100600 0x7f25101029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 --2- 172.21.15.62:0/383645180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 0x7f2510105300 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 -- 172.21.15.62:0/383645180 >> 172.21.15.62:0/383645180 conn(0x7f25100fa000 msgr2=0x7f25100fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 -- 172.21.15.62:0/383645180 shutdown_connections 2023-07-14T16:31:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.159+0000 7f25177ff700 1 -- 172.21.15.62:0/383645180 wait complete. 2023-07-14T16:31:28.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.160+0000 7f25177ff700 1 Processor -- start 2023-07-14T16:31:28.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.160+0000 7f25177ff700 1 -- start start 2023-07-14T16:31:28.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f25177ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2510100600 0x7f25100fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f25177ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 0x7f25100fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f25177ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2510105840 0x7f25100feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f25177ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2510199630 con 0x7f2510102f20 2023-07-14T16:31:28.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f25177ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100ff030 con 0x7f2510105840 2023-07-14T16:31:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f25177ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100ff1f0 con 0x7f2510100600 2023-07-14T16:31:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2514d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 0x7f25100fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2514d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 0x7f25100fe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36858/0 (socket says 172.21.15.62:36858) 2023-07-14T16:31:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2514d9a700 1 -- 172.21.15.62:0/1933141713 learned_addr learned my addr 172.21.15.62:0/1933141713 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f251559b700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2510100600 0x7f25100fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2515d9c700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2510105840 0x7f25100feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2514d9a700 1 -- 172.21.15.62:0/1933141713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2510100600 msgr2=0x7f25100fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2514d9a700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2510100600 0x7f25100fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2514d9a700 1 -- 172.21.15.62:0/1933141713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2510105840 msgr2=0x7f25100feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2514d9a700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2510105840 0x7f25100feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.161+0000 7f2514d9a700 1 -- 172.21.15.62:0/1933141713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000095f0 con 0x7f2510102f20 2023-07-14T16:31:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.162+0000 7f2514d9a700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 0x7f25100fe5b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2500005910 tx=0x7f2500005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.162+0000 7f25067fc700 1 -- 172.21.15.62:0/1933141713 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250001d070 con 0x7f2510102f20 2023-07-14T16:31:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.162+0000 7f25067fc700 1 -- 172.21.15.62:0/1933141713 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f250000be70 con 0x7f2510102f20 2023-07-14T16:31:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.162+0000 7f25177ff700 1 -- 172.21.15.62:0/1933141713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25101a3fe0 con 0x7f2510102f20 2023-07-14T16:31:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.162+0000 7f25067fc700 1 -- 172.21.15.62:0/1933141713 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500022ea0 con 0x7f2510102f20 2023-07-14T16:31:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.162+0000 7f25177ff700 1 -- 172.21.15.62:0/1933141713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25101a4350 con 0x7f2510102f20 2023-07-14T16:31:28.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.165+0000 7f25067fc700 1 -- 172.21.15.62:0/1933141713 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2500022670 con 0x7f2510102f20 2023-07-14T16:31:28.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.167+0000 7f252b08db80 1 Processor -- start 2023-07-14T16:31:28.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.167+0000 7f252b08db80 1 -- start start 2023-07-14T16:31:28.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f252b08db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a884d0 0x555d18a888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f252b08db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f252b08db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a89c70 0x555d18a8a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f252b08db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1898dcb0 con 0x555d18a884d0 2023-07-14T16:31:28.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f252b08db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d189e0a40 con 0x555d18a88e20 2023-07-14T16:31:28.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f252b08db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d189d9ed0 con 0x555d18a89c70 2023-07-14T16:31:28.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f25067fc700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24fc0a11d0 0x7f24fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f25067fc700 1 -- 172.21.15.62:0/1933141713 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f25000cdca0 con 0x7f2510102f20 2023-07-14T16:31:28.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f2515d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a884d0 0x555d18a888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f2514d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a89c70 0x555d18a8a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f251559b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f2514d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a89c70 0x555d18a8a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:59828/0 (socket says 172.21.15.62:59828) 2023-07-14T16:31:28.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f2515d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a884d0 0x555d18a888e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36860/0 (socket says 172.21.15.62:36860) 2023-07-14T16:31:28.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f2514d9a700 1 -- 172.21.15.62:0/3139393610 learned_addr learned my addr 172.21.15.62:0/3139393610 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.168+0000 7f2515d9c700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24fc0a11d0 0x7f24fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f2514d9a700 1 -- 172.21.15.62:0/3139393610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 msgr2=0x555d18a896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f2514d9a700 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a896e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f2514d9a700 1 -- 172.21.15.62:0/3139393610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a884d0 msgr2=0x555d18a888e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f2514d9a700 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a884d0 0x555d18a888e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f2514d9a700 1 -- 172.21.15.62:0/3139393610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d18807de0 con 0x555d18a89c70 2023-07-14T16:31:28.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f251559b700 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f2515d9c700 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a884d0 0x555d18a888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f2514d9a700 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a89c70 0x555d18a8a0e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f25000cb750 tx=0x7f2500009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db92cdbb97ca045 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f2515d9c700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24fc0a11d0 0x7f24fc0a3680 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2508009fd0 tx=0x7f250800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f24fb7fe700 1 -- 172.21.15.62:0/3139393610 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000b2070 con 0x555d18a89c70 2023-07-14T16:31:28.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f24fb7fe700 1 -- 172.21.15.62:0/3139393610 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f250009b0f0 con 0x555d18a89c70 2023-07-14T16:31:28.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.169+0000 7f24fb7fe700 1 -- 172.21.15.62:0/3139393610 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500099570 con 0x555d18a89c70 2023-07-14T16:31:28.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:27 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3175570845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:31:28.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 -- 172.21.15.62:0/3139393610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a89c70 msgr2=0x555d18a8a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a89c70 0x555d18a8a0e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f25000cb750 tx=0x7f2500009510 comp rx=0 tx=0).stop 2023-07-14T16:31:28.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 -- 172.21.15.62:0/3139393610 shutdown_connections 2023-07-14T16:31:28.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a89c70 0x555d18a8a0e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a884d0 0x555d18a888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 --2- 172.21.15.62:0/3139393610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 -- 172.21.15.62:0/3139393610 >> 172.21.15.62:0/3139393610 conn(0x555d18a7dda0 msgr2=0x555d18a831d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 -- 172.21.15.62:0/3139393610 shutdown_connections 2023-07-14T16:31:28.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.170+0000 7f252b08db80 1 -- 172.21.15.62:0/3139393610 wait complete. 2023-07-14T16:31:28.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.171+0000 7f252b08db80 1 -- start start 2023-07-14T16:31:28.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.171+0000 7f252b08db80 1 Processor -- start 2023-07-14T16:31:28.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f252b08db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a91dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f252b08db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a92300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f252b08db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f252b08db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d18979ac0 con 0x555d18a89c70 2023-07-14T16:31:28.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f252b08db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d189797c0 con 0x555d18a88e20 2023-07-14T16:31:28.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f252b08db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1885d920 con 0x555d18a884d0 2023-07-14T16:31:28.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f252b08db80 10 client.0 _ll_register_callbacks cb 0x555d18977760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:28.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f251559b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a91dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f2515d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f251559b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a91dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:59834/0 (socket says 172.21.15.62:59834) 2023-07-14T16:31:28.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f251559b700 1 -- 172.21.15.62:0/3586013901 learned_addr learned my addr 172.21.15.62:0/3586013901 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.172+0000 7f2514d9a700 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a92300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f251559b700 1 -- 172.21.15.62:0/3586013901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 msgr2=0x555d18a92300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f251559b700 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a92300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f251559b700 1 -- 172.21.15.62:0/3586013901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 msgr2=0x555d18a99340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f251559b700 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f251559b700 1 -- 172.21.15.62:0/3586013901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2508012bf0 con 0x555d18a884d0 2023-07-14T16:31:28.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f2515d9c700 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f2514d9a700 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a92300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f251559b700 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a91dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f250c012cf0 tx=0x7f250c009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db67d6560eb56867 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.173+0000 7f24f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:31:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.174+0000 7f24f8ff9700 1 -- 172.21.15.62:0/3586013901 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c017070 con 0x555d18a884d0 2023-07-14T16:31:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.174+0000 7f24f8ff9700 1 -- 172.21.15.62:0/3586013901 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f250c004020 con 0x555d18a884d0 2023-07-14T16:31:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.174+0000 7f252b08db80 1 -- 172.21.15.62:0/3586013901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d189334f0 con 0x555d18a884d0 2023-07-14T16:31:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.174+0000 7f24f8ff9700 1 -- 172.21.15.62:0/3586013901 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250c022030 con 0x555d18a884d0 2023-07-14T16:31:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.174+0000 7f252b08db80 10 client.29350 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.174+0000 7f252b08db80 1 -- 172.21.15.62:0/3586013901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f250c0098e0 con 0x555d18a884d0 2023-07-14T16:31:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.174+0000 7f24f8ff9700 1 -- 172.21.15.62:0/3586013901 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555d189334f0 con 0x555d18a884d0 2023-07-14T16:31:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.175+0000 7f24f8ff9700 1 -- 172.21.15.62:0/3586013901 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f250c003680 con 0x555d18a884d0 2023-07-14T16:31:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.175+0000 7f252b08db80 10 client.29350 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.175+0000 7f252b08db80 10 client.29350 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:31:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.175+0000 7f252b08db80 20 client.29350 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.175+0000 7f252b08db80 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x555d18a9dc00 0x555d18aa0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.175+0000 7f252b08db80 4 client.29350 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:28.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.175+0000 7f252b08db80 1 -- 172.21.15.62:0/3586013901 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d189d2eb0 con 0x555d18a9dc00 2023-07-14T16:31:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.175+0000 7f2514d9a700 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x555d18a9dc00 0x555d18aa0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.176+0000 7f24f8ff9700 10 client.29350 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:31:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.176+0000 7f2514d9a700 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x555d18a9dc00 0x555d18aa0010 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c115575d3be32e6d server_cookie=ce29aba5eae4f3f2 in_seq=0 out_seq=0 2023-07-14T16:31:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.176+0000 7f24f8ff9700 1 -- 172.21.15.62:0/3586013901 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d189d2eb0 con 0x555d18a9dc00 2023-07-14T16:31:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f24f8ff9700 10 client.29350 handle_command_reply: tid=0 2023-07-14T16:31:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 client.29350 shutdown 2023-07-14T16:31:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 -- 172.21.15.62:0/3586013901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 msgr2=0x555d18a91dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a91dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f250c012cf0 tx=0x7f250c009d00 comp rx=0 tx=0).stop 2023-07-14T16:31:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 -- 172.21.15.62:0/3586013901 shutdown_connections 2023-07-14T16:31:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x555d18a9dc00 0x555d18aa0010 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f24f8ff9700 0 client.29350 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:31:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a91dc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 --2- 172.21.15.62:0/3586013901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a92300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 -- 172.21.15.62:0/3586013901 >> 172.21.15.62:0/3586013901 conn(0x555d18a7dda0 msgr2=0x555d18a828b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 -- 172.21.15.62:0/3586013901 shutdown_connections 2023-07-14T16:31:28.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 1 -- 172.21.15.62:0/3586013901 wait complete. 2023-07-14T16:31:28.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.177+0000 7f252b08db80 20 client.29350 trim_cache size 0 max 16384 2023-07-14T16:31:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.196+0000 7f252b08db80 1 Processor -- start 2023-07-14T16:31:28.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.196+0000 7f252b08db80 1 -- start start 2023-07-14T16:31:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f252b08db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a884d0 0x555d18a8cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f252b08db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a88e20 0x555d18a8d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f252b08db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f252b08db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d18a7b020 con 0x555d18a89c70 2023-07-14T16:31:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f252b08db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d18a912a0 con 0x555d18a884d0 2023-07-14T16:31:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f252b08db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d18a997a0 con 0x555d18a88e20 2023-07-14T16:31:28.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f2515d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f2515d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36876/0 (socket says 172.21.15.62:36876) 2023-07-14T16:31:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f2515d9c700 1 -- 172.21.15.62:0/4233308395 learned_addr learned my addr 172.21.15.62:0/4233308395 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f2514d9a700 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a88e20 0x555d18a8d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.197+0000 7f251559b700 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a884d0 0x555d18a8cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f2514d9a700 1 -- 172.21.15.62:0/4233308395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a884d0 msgr2=0x555d18a8cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f2514d9a700 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a884d0 0x555d18a8cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f2514d9a700 1 -- 172.21.15.62:0/4233308395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 msgr2=0x555d18a99260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f2514d9a700 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f2514d9a700 1 -- 172.21.15.62:0/4233308395 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25101a4350 con 0x555d18a88e20 2023-07-14T16:31:28.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f251559b700 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a884d0 0x555d18a8cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:28.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f2515d9c700 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f2514d9a700 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a88e20 0x555d18a8d010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f250009adf0 tx=0x7f250001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cfcfb3275ac258b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f24fbfff700 1 -- 172.21.15.62:0/4233308395 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000b2040 con 0x555d18a88e20 2023-07-14T16:31:28.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f24fbfff700 1 -- 172.21.15.62:0/4233308395 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25000b7070 con 0x555d18a88e20 2023-07-14T16:31:28.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.198+0000 7f24fbfff700 1 -- 172.21.15.62:0/4233308395 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250009a5e0 con 0x555d18a88e20 2023-07-14T16:31:28.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 -- 172.21.15.62:0/4233308395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a88e20 msgr2=0x555d18a8d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a88e20 0x555d18a8d010 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f250009adf0 tx=0x7f250001a040 comp rx=0 tx=0).stop 2023-07-14T16:31:28.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 -- 172.21.15.62:0/4233308395 shutdown_connections 2023-07-14T16:31:28.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a99260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a884d0 0x555d18a8cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 --2- 172.21.15.62:0/4233308395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a88e20 0x555d18a8d010 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 -- 172.21.15.62:0/4233308395 >> 172.21.15.62:0/4233308395 conn(0x555d18a7dda0 msgr2=0x555d18a99c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 -- 172.21.15.62:0/4233308395 shutdown_connections 2023-07-14T16:31:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.199+0000 7f252b08db80 1 -- 172.21.15.62:0/4233308395 wait complete. 2023-07-14T16:31:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.200+0000 7f252b08db80 1 -- start start 2023-07-14T16:31:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.200+0000 7f252b08db80 1 Processor -- start 2023-07-14T16:31:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.200+0000 7f252b08db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.200+0000 7f252b08db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.200+0000 7f252b08db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a98910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.200+0000 7f252b08db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d18a98e50 con 0x555d18a89c70 2023-07-14T16:31:28.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.200+0000 7f252b08db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d18a98fc0 con 0x555d18a88e20 2023-07-14T16:31:28.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.200+0000 7f252b08db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d18a99180 con 0x555d18a884d0 2023-07-14T16:31:28.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f252b08db80 10 client.0 _ll_register_callbacks cb 0x7f250c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:28.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f251559b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f251559b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:59864/0 (socket says 172.21.15.62:59864) 2023-07-14T16:31:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f251559b700 1 -- 172.21.15.62:0/903734477 learned_addr learned my addr 172.21.15.62:0/903734477 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f2515d9c700 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a98910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f2514d9a700 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f251559b700 1 -- 172.21.15.62:0/903734477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 msgr2=0x555d18a958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f251559b700 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f251559b700 1 -- 172.21.15.62:0/903734477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 msgr2=0x555d18a98910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f251559b700 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a98910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f251559b700 1 -- 172.21.15.62:0/903734477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d18807de0 con 0x555d18a884d0 2023-07-14T16:31:28.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.201+0000 7f2515d9c700 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a98910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:28.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.202+0000 7f251559b700 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a953b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f250c003a60 tx=0x7f250c009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fd56bda29e3c808 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.202+0000 7f24f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:31:28.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.202+0000 7f24f8ff9700 1 -- 172.21.15.62:0/903734477 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c022030 con 0x555d18a884d0 2023-07-14T16:31:28.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.202+0000 7f24f8ff9700 1 -- 172.21.15.62:0/903734477 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f250c023070 con 0x555d18a884d0 2023-07-14T16:31:28.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.202+0000 7f252b08db80 1 -- 172.21.15.62:0/903734477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f250009b0f0 con 0x555d18a884d0 2023-07-14T16:31:28.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.202+0000 7f24f8ff9700 1 -- 172.21.15.62:0/903734477 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250c017040 con 0x555d18a884d0 2023-07-14T16:31:28.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.202+0000 7f252b08db80 10 client.29356 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:28.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f252b08db80 1 -- 172.21.15.62:0/903734477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555d187f0c60 con 0x555d18a884d0 2023-07-14T16:31:28.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f24f8ff9700 1 -- 172.21.15.62:0/903734477 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f250c036020 con 0x555d18a884d0 2023-07-14T16:31:28.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f24f8ff9700 1 -- 172.21.15.62:0/903734477 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f250c0055b0 con 0x555d18a884d0 2023-07-14T16:31:28.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f252b08db80 10 client.29356 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:28.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f252b08db80 10 client.29356 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:31:28.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f252b08db80 20 client.29356 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:28.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f252b08db80 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x555d18af2e20 0x555d18af5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f252b08db80 4 client.29356 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:31:28.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.203+0000 7f252b08db80 1 -- 172.21.15.62:0/903734477 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555d18995050 con 0x555d18af2e20 2023-07-14T16:31:28.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.204+0000 7f2514d9a700 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x555d18af2e20 0x555d18af5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.204+0000 7f2514d9a700 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x555d18af2e20 0x555d18af5230 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f065378b48d7aa4 server_cookie=73ad7ea9b7f95a0b in_seq=0 out_seq=0 2023-07-14T16:31:28.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.204+0000 7f24f8ff9700 10 client.29356 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:31:28.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f24f8ff9700 1 -- 172.21.15.62:0/903734477 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555d18995050 con 0x555d18af2e20 2023-07-14T16:31:28.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f24f8ff9700 10 client.29356 handle_command_reply: tid=0 2023-07-14T16:31:28.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 client.29356 shutdown 2023-07-14T16:31:28.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 -- 172.21.15.62:0/903734477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 msgr2=0x555d18a953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a953b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f250c003a60 tx=0x7f250c009e90 comp rx=0 tx=0).stop 2023-07-14T16:31:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 -- 172.21.15.62:0/903734477 shutdown_connections 2023-07-14T16:31:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x555d18af2e20 0x555d18af5230 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555d18a89c70 0x555d18a98910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555d18a884d0 0x555d18a953b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f24f8ff9700 0 client.29356 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:31:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 --2- 172.21.15.62:0/903734477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d18a88e20 0x555d18a958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 -- 172.21.15.62:0/903734477 >> 172.21.15.62:0/903734477 conn(0x555d18a7dda0 msgr2=0x555d18a8cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 -- 172.21.15.62:0/903734477 shutdown_connections 2023-07-14T16:31:28.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 1 -- 172.21.15.62:0/903734477 wait complete. 2023-07-14T16:31:28.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.205+0000 7f252b08db80 20 client.29356 trim_cache size 0 max 16384 2023-07-14T16:31:28.186 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:31:28.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 -- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24fc0a11d0 msgr2=0x7f24fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24fc0a11d0 0x7f24fc0a3680 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2508009fd0 tx=0x7f250800b040 comp rx=0 tx=0).stop 2023-07-14T16:31:28.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 -- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 msgr2=0x7f25100fe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 0x7f25100fe5b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f2500005910 tx=0x7f2500005740 comp rx=0 tx=0).stop 2023-07-14T16:31:28.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 -- 172.21.15.62:0/1933141713 shutdown_connections 2023-07-14T16:31:28.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f24fc0a11d0 0x7f24fc0a3680 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2510105840 0x7f25100feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2510100600 0x7f25100fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 --2- 172.21.15.62:0/1933141713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2510102f20 0x7f25100fe5b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 -- 172.21.15.62:0/1933141713 >> 172.21.15.62:0/1933141713 conn(0x7f25100fa000 msgr2=0x7f2510101260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.206+0000 7f25177ff700 1 -- 172.21.15.62:0/1933141713 shutdown_connections 2023-07-14T16:31:28.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.207+0000 7f25177ff700 1 -- 172.21.15.62:0/1933141713 wait complete. 2023-07-14T16:31:28.202 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:31:28.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:27 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3175570845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:31:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.443+0000 7f2687fff700 1 -- 172.21.15.62:0/486827223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688100450 msgr2=0x7f2688102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.443+0000 7f2687fff700 1 --2- 172.21.15.62:0/486827223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688100450 0x7f2688102830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f26780098f0 tx=0x7f2678009c00 comp rx=0 tx=0).stop 2023-07-14T16:31:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.443+0000 7f2687fff700 1 -- 172.21.15.62:0/486827223 shutdown_connections 2023-07-14T16:31:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.443+0000 7f2687fff700 1 --2- 172.21.15.62:0/486827223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2688105690 0x7f2688107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.443+0000 7f2687fff700 1 --2- 172.21.15.62:0/486827223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688100450 0x7f2688102830 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.443+0000 7f2687fff700 1 --2- 172.21.15.62:0/486827223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688102d70 0x7f2688105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.443+0000 7f2687fff700 1 -- 172.21.15.62:0/486827223 >> 172.21.15.62:0/486827223 conn(0x7f26880f9ed0 msgr2=0x7f26880fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.444+0000 7f2687fff700 1 -- 172.21.15.62:0/486827223 shutdown_connections 2023-07-14T16:31:28.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.444+0000 7f2687fff700 1 -- 172.21.15.62:0/486827223 wait complete. 2023-07-14T16:31:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2687fff700 1 Processor -- start 2023-07-14T16:31:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2687fff700 1 -- start start 2023-07-14T16:31:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2687fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688100450 0x7f2688197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2687fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688102d70 0x7f2688197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2687fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2688105690 0x7f268819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2687fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268819f2b0 con 0x7f2688100450 2023-07-14T16:31:28.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2687fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268819f3f0 con 0x7f2688105690 2023-07-14T16:31:28.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2687fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268819f580 con 0x7f2688102d70 2023-07-14T16:31:28.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.445+0000 7f2686ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688100450 0x7f2688197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f2686ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688100450 0x7f2688197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36906/0 (socket says 172.21.15.62:36906) 2023-07-14T16:31:28.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f2686ffd700 1 -- 172.21.15.62:0/4150276228 learned_addr learned my addr 172.21.15.62:0/4150276228 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f26867fc700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688102d70 0x7f2688197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f26877fe700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2688105690 0x7f268819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f2686ffd700 1 -- 172.21.15.62:0/4150276228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688102d70 msgr2=0x7f2688197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f2686ffd700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688102d70 0x7f2688197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f2686ffd700 1 -- 172.21.15.62:0/4150276228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2688105690 msgr2=0x7f268819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f2686ffd700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2688105690 0x7f268819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f2686ffd700 1 -- 172.21.15.62:0/4150276228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678009520 con 0x7f2688100450 2023-07-14T16:31:28.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f26867fc700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688102d70 0x7f2688197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f26877fe700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2688105690 0x7f268819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.446+0000 7f2686ffd700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688100450 0x7f2688197730 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2678009fd0 tx=0x7f267800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.447+0000 7f266ffff700 1 -- 172.21.15.62:0/4150276228 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267801c070 con 0x7f2688100450 2023-07-14T16:31:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.447+0000 7f266ffff700 1 -- 172.21.15.62:0/4150276228 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f267800f7b0 con 0x7f2688100450 2023-07-14T16:31:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.447+0000 7f2687fff700 1 -- 172.21.15.62:0/4150276228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268819f800 con 0x7f2688100450 2023-07-14T16:31:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.447+0000 7f266ffff700 1 -- 172.21.15.62:0/4150276228 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678004910 con 0x7f2688100450 2023-07-14T16:31:28.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.447+0000 7f2687fff700 1 -- 172.21.15.62:0/4150276228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268819fc80 con 0x7f2688100450 2023-07-14T16:31:28.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.452+0000 7f266ffff700 1 -- 172.21.15.62:0/4150276228 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2678020070 con 0x7f2688100450 2023-07-14T16:31:28.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.453+0000 7f26a1162b80 1 Processor -- start 2023-07-14T16:31:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.453+0000 7f26a1162b80 1 -- start start 2023-07-14T16:31:28.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.453+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.453+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8070 0x5622592e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f26a1162b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622591eccb0 con 0x5622592e8e30 2023-07-14T16:31:28.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f26a1162b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225923fa40 con 0x5622592e8070 2023-07-14T16:31:28.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f26a1162b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562259238ed0 con 0x5622592e7720 2023-07-14T16:31:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f2686ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8070 0x5622592e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f26867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f2686ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8070 0x5622592e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36408/0 (socket says 172.21.15.62:36408) 2023-07-14T16:31:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f2686ffd700 1 -- 172.21.15.62:0/3709210648 learned_addr learned my addr 172.21.15.62:0/3709210648 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f26867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592e92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36912/0 (socket says 172.21.15.62:36912) 2023-07-14T16:31:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f266ffff700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f26700a1270 0x7f26700a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f266ffff700 1 -- 172.21.15.62:0/4150276228 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f26780cd5b0 con 0x7f2688100450 2023-07-14T16:31:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f26877fe700 1 --2- 172.21.15.62:0/3709210648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f2686ffd700 1 -- 172.21.15.62:0/3709210648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 msgr2=0x5622592e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.454+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3709210648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.455+0000 7f2686ffd700 1 -- 172.21.15.62:0/3709210648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 msgr2=0x5622592e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.455+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3709210648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.455+0000 7f2686ffd700 1 -- 172.21.15.62:0/3709210648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562259066de0 con 0x5622592e8070 2023-07-14T16:31:28.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.455+0000 7f26877fe700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f26700a1270 0x7f26700a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.455+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3709210648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8070 0x5622592e88a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f267809a700 tx=0x7f2678009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd154b8a4dd328cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.455+0000 7f266d7fa700 1 -- 172.21.15.62:0/3709210648 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267809a360 con 0x5622592e8070 2023-07-14T16:31:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.455+0000 7f266d7fa700 1 -- 172.21.15.62:0/3709210648 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26780d0bc0 con 0x5622592e8070 2023-07-14T16:31:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.455+0000 7f266d7fa700 1 -- 172.21.15.62:0/3709210648 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678098da0 con 0x5622592e8070 2023-07-14T16:31:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 -- 172.21.15.62:0/3709210648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8070 msgr2=0x5622592e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3709210648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8070 0x5622592e88a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f267809a700 tx=0x7f2678009260 comp rx=0 tx=0).stop 2023-07-14T16:31:28.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26877fe700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f26700a1270 0x7f26700a3720 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2680009fd0 tx=0x7f268000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 -- 172.21.15.62:0/3709210648 shutdown_connections 2023-07-14T16:31:28.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3709210648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3709210648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3709210648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8070 0x5622592e88a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 -- 172.21.15.62:0/3709210648 >> 172.21.15.62:0/3709210648 conn(0x5622592dcff0 msgr2=0x5622592e2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 -- 172.21.15.62:0/3709210648 shutdown_connections 2023-07-14T16:31:28.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.456+0000 7f26a1162b80 1 -- 172.21.15.62:0/3709210648 wait complete. 2023-07-14T16:31:28.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.457+0000 7f26a1162b80 1 -- start start 2023-07-14T16:31:28.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.457+0000 7f26a1162b80 1 Processor -- start 2023-07-14T16:31:28.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8070 0x5622592f1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26a1162b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622591d8ac0 con 0x5622592e8e30 2023-07-14T16:31:28.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26a1162b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622591d87c0 con 0x5622592e7720 2023-07-14T16:31:28.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26a1162b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622590bc920 con 0x5622592e8070 2023-07-14T16:31:28.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26a1162b80 10 client.0 _ll_register_callbacks cb 0x5622591d6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:28.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26867fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8070 0x5622592f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36920/0 (socket says 172.21.15.62:36920) 2023-07-14T16:31:28.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f26877fe700 1 -- 172.21.15.62:0/3659363923 learned_addr learned my addr 172.21.15.62:0/3659363923 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.458+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f26867fc700 1 -- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 msgr2=0x5622592f0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f26867fc700 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592f0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f26867fc700 1 -- 172.21.15.62:0/3659363923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 msgr2=0x5622592f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f26867fc700 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f26867fc700 1 -- 172.21.15.62:0/3659363923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268819fc80 con 0x5622592e8070 2023-07-14T16:31:28.458 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f26877fe700 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:28.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f26867fc700 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8070 0x5622592f1490 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f267c012cd0 tx=0x7f267c009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e630a92670a356e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f266aff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:31:28.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f266aff5700 1 -- 172.21.15.62:0/3659363923 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c017070 con 0x5622592e8070 2023-07-14T16:31:28.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f26a1162b80 1 -- 172.21.15.62:0/3659363923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622591924f0 con 0x5622592e8070 2023-07-14T16:31:28.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.459+0000 7f266aff5700 1 -- 172.21.15.62:0/3659363923 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f267c004110 con 0x5622592e8070 2023-07-14T16:31:28.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f266aff5700 1 -- 172.21.15.62:0/3659363923 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267c022030 con 0x5622592e8070 2023-07-14T16:31:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f26a1162b80 10 client.29368 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f26a1162b80 1 -- 172.21.15.62:0/3659363923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56225904fc60 con 0x5622592e8070 2023-07-14T16:31:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f266aff5700 1 -- 172.21.15.62:0/3659363923 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5622591924f0 con 0x5622592e8070 2023-07-14T16:31:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f266aff5700 1 -- 172.21.15.62:0/3659363923 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f267c0036f0 con 0x5622592e8070 2023-07-14T16:31:28.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f26a1162b80 10 client.29368 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:28.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f26a1162b80 10 client.29368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:28.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f26a1162b80 20 client.29368 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:28.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.460+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622592fcd80 0x5622592ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.461+0000 7f26a1162b80 4 client.29368 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.461+0000 7f26a1162b80 1 -- 172.21.15.62:0/3659363923 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562259231eb0 con 0x5622592fcd80 2023-07-14T16:31:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.461+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622592fcd80 0x5622592ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.461+0000 7f266aff5700 10 client.29368 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.461+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622592fcd80 0x5622592ff190 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa90f1f699c4234 server_cookie=79e1eea1d86f71b in_seq=0 out_seq=0 2023-07-14T16:31:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.462+0000 7f266aff5700 1 -- 172.21.15.62:0/3659363923 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562259231eb0 con 0x5622592fcd80 2023-07-14T16:31:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.462+0000 7f266aff5700 10 client.29368 handle_command_reply: tid=0 2023-07-14T16:31:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.462+0000 7f26a1162b80 1 client.29368 shutdown 2023-07-14T16:31:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 -- 172.21.15.62:0/3659363923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8070 msgr2=0x5622592f1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8070 0x5622592f1490 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f267c012cd0 tx=0x7f267c009e60 comp rx=0 tx=0).stop 2023-07-14T16:31:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 -- 172.21.15.62:0/3659363923 shutdown_connections 2023-07-14T16:31:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8e30 0x5622592f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622592fcd80 0x5622592ff190 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3659363923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8070 0x5622592f1490 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f266aff5700 0 client.29368 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 -- 172.21.15.62:0/3659363923 >> 172.21.15.62:0/3659363923 conn(0x5622592dcff0 msgr2=0x5622592e1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 -- 172.21.15.62:0/3659363923 shutdown_connections 2023-07-14T16:31:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 1 -- 172.21.15.62:0/3659363923 wait complete. 2023-07-14T16:31:28.468 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.463+0000 7f26a1162b80 20 client.29368 trim_cache size 0 max 16384 2023-07-14T16:31:28.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.480+0000 7f26a1162b80 1 Processor -- start 2023-07-14T16:31:28.475 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.480+0000 7f26a1162b80 1 -- start start 2023-07-14T16:31:28.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.480+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.480+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.480+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8e30 0x5622592f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.476 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.480+0000 7f26a1162b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622592e8440 con 0x5622592e8070 2023-07-14T16:31:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.480+0000 7f26a1162b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562259231eb0 con 0x5622592e7720 2023-07-14T16:31:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.480+0000 7f26a1162b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622591924f0 con 0x5622592e8e30 2023-07-14T16:31:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592ec1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36924/0 (socket says 172.21.15.62:36924) 2023-07-14T16:31:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 -- 172.21.15.62:0/3818351581 learned_addr learned my addr 172.21.15.62:0/3818351581 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26877fe700 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8e30 0x5622592f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 -- 172.21.15.62:0/3818351581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8e30 msgr2=0x5622592f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.479 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8e30 0x5622592f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 -- 172.21.15.62:0/3818351581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 msgr2=0x5622592ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 -- 172.21.15.62:0/3818351581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56225908d250 con 0x5622592e8070 2023-07-14T16:31:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f2686ffd700 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.481+0000 7f26867fc700 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592ec1d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f267c004020 tx=0x7f267c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dafc57026ed1bc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.482+0000 7f266dffb700 1 -- 172.21.15.62:0/3818351581 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c017070 con 0x5622592e8070 2023-07-14T16:31:28.480 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.482+0000 7f266dffb700 1 -- 172.21.15.62:0/3818351581 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f267c018040 con 0x5622592e8070 2023-07-14T16:31:28.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.482+0000 7f266dffb700 1 -- 172.21.15.62:0/3818351581 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c022040 con 0x5622592e8070 2023-07-14T16:31:28.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 -- 172.21.15.62:0/3818351581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 msgr2=0x5622592ec1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592ec1d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f267c004020 tx=0x7f267c005900 comp rx=0 tx=0).stop 2023-07-14T16:31:28.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 -- 172.21.15.62:0/3818351581 shutdown_connections 2023-07-14T16:31:28.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e8e30 0x5622592f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e7720 0x5622592ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 --2- 172.21.15.62:0/3818351581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592ec1d0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 -- 172.21.15.62:0/3818351581 >> 172.21.15.62:0/3818351581 conn(0x5622592dcff0 msgr2=0x5622592ef150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 -- 172.21.15.62:0/3818351581 shutdown_connections 2023-07-14T16:31:28.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 -- 172.21.15.62:0/3818351581 wait complete. 2023-07-14T16:31:28.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.483+0000 7f26a1162b80 1 -- start start 2023-07-14T16:31:28.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26a1162b80 1 Processor -- start 2023-07-14T16:31:28.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592f4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592f4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26a1162b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8e30 0x5622592f7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26a1162b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622592f7cb0 con 0x5622592e8070 2023-07-14T16:31:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26a1162b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622592f7e20 con 0x5622592e8e30 2023-07-14T16:31:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26a1162b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622592f7fe0 con 0x5622592e7720 2023-07-14T16:31:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26a1162b80 10 client.0 _ll_register_callbacks cb 0x7f267c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:28.483 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592f4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36930/0 (socket says 172.21.15.62:36930) 2023-07-14T16:31:28.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.484+0000 7f26867fc700 1 -- 172.21.15.62:0/1129005946 learned_addr learned my addr 172.21.15.62:0/1129005946 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:28.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f26877fe700 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8e30 0x5622592f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f2686ffd700 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f26867fc700 1 -- 172.21.15.62:0/1129005946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 msgr2=0x5622592f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f26867fc700 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f26867fc700 1 -- 172.21.15.62:0/1129005946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8e30 msgr2=0x5622592f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:28.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f26867fc700 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8e30 0x5622592f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f26867fc700 1 -- 172.21.15.62:0/1129005946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56225904fc60 con 0x5622592e8070 2023-07-14T16:31:28.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f2686ffd700 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:28.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f26867fc700 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592f4750 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f267c003a50 tx=0x7f267c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c201e5130cc847fc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:28.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f266aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:28.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f266aff5700 1 -- 172.21.15.62:0/1129005946 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c022070 con 0x5622592e8070 2023-07-14T16:31:28.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f26a1162b80 1 -- 172.21.15.62:0/1129005946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26780d0bc0 con 0x5622592e8070 2023-07-14T16:31:28.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.485+0000 7f266aff5700 1 -- 172.21.15.62:0/1129005946 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f267c02b070 con 0x5622592e8070 2023-07-14T16:31:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f266aff5700 1 -- 172.21.15.62:0/1129005946 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267c017070 con 0x5622592e8070 2023-07-14T16:31:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f26a1162b80 10 client.19977 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f26a1162b80 1 -- 172.21.15.62:0/1129005946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5622590652b0 con 0x5622592e8070 2023-07-14T16:31:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f266aff5700 1 -- 172.21.15.62:0/1129005946 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26780d0bc0 con 0x5622592e8070 2023-07-14T16:31:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f266aff5700 1 -- 172.21.15.62:0/1129005946 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f267c043070 con 0x5622592e8070 2023-07-14T16:31:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f26a1162b80 10 client.19977 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f26a1162b80 10 client.19977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f26a1162b80 20 client.19977 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:28.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f26a1162b80 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562259351780 0x562259353b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:28.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f26a1162b80 4 client.19977 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:31:28.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.486+0000 7f26a1162b80 1 -- 172.21.15.62:0/1129005946 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5622591f4050 con 0x562259351780 2023-07-14T16:31:28.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.487+0000 7f2686ffd700 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562259351780 0x562259353b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:28.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.487+0000 7f2686ffd700 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562259351780 0x562259353b90 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fccd4d8822091868 server_cookie=df56ca6e78b08505 in_seq=0 out_seq=0 2023-07-14T16:31:28.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.487+0000 7f266aff5700 10 client.19977 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:28.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f266aff5700 1 -- 172.21.15.62:0/1129005946 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622591f4050 con 0x562259351780 2023-07-14T16:31:28.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f266aff5700 10 client.19977 handle_command_reply: tid=0 2023-07-14T16:31:28.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 client.19977 shutdown 2023-07-14T16:31:28.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 -- 172.21.15.62:0/1129005946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 msgr2=0x5622592f4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592f4750 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f267c003a50 tx=0x7f267c009310 comp rx=0 tx=0).stop 2023-07-14T16:31:28.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 -- 172.21.15.62:0/1129005946 shutdown_connections 2023-07-14T16:31:28.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562259351780 0x562259353b90 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622592e8e30 0x5622592f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622592e7720 0x5622592f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f266aff5700 0 client.19977 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:28.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 --2- 172.21.15.62:0/1129005946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622592e8070 0x5622592f4750 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 -- 172.21.15.62:0/1129005946 >> 172.21.15.62:0/1129005946 conn(0x5622592dcff0 msgr2=0x5622592ebc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 -- 172.21.15.62:0/1129005946 shutdown_connections 2023-07-14T16:31:28.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 1 -- 172.21.15.62:0/1129005946 wait complete. 2023-07-14T16:31:28.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.488+0000 7f26a1162b80 20 client.19977 trim_cache size 0 max 16384 2023-07-14T16:31:28.490 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:31:28.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 -- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f26700a1270 msgr2=0x7f26700a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f26700a1270 0x7f26700a3720 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2680009fd0 tx=0x7f268000b040 comp rx=0 tx=0).stop 2023-07-14T16:31:28.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 -- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688100450 msgr2=0x7f2688197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:28.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688100450 0x7f2688197730 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2678009fd0 tx=0x7f267800b790 comp rx=0 tx=0).stop 2023-07-14T16:31:28.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 -- 172.21.15.62:0/4150276228 shutdown_connections 2023-07-14T16:31:28.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f26700a1270 0x7f26700a3720 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2688105690 0x7f268819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688100450 0x7f2688197730 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 --2- 172.21.15.62:0/4150276228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2688102d70 0x7f2688197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:28.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 -- 172.21.15.62:0/4150276228 >> 172.21.15.62:0/4150276228 conn(0x7f26880f9ed0 msgr2=0x7f26880fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:28.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 -- 172.21.15.62:0/4150276228 shutdown_connections 2023-07-14T16:31:28.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:28.489+0000 7f2687fff700 1 -- 172.21.15.62:0/4150276228 wait complete. 2023-07-14T16:31:29.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:28 smithi062 ceph-mon[95543]: pgmap v832: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:31:29.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:28 smithi081 ceph-mon[127795]: pgmap v832: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:31:29.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:28 smithi164 ceph-mon[97427]: pgmap v832: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-14T16:31:29.493 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:31:29.493 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:31:30.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.063+0000 7f4205f9d700 1 -- 172.21.15.62:0/1170766137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200103770 msgr2=0x7f420010a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.063+0000 7f4205f9d700 1 --2- 172.21.15.62:0/1170766137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200103770 0x7f420010a030 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f41f800b0a0 tx=0x7f41f800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.064+0000 7f4205f9d700 1 -- 172.21.15.62:0/1170766137 shutdown_connections 2023-07-14T16:31:30.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.064+0000 7f4205f9d700 1 --2- 172.21.15.62:0/1170766137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200103770 0x7f420010a030 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.064+0000 7f4205f9d700 1 --2- 172.21.15.62:0/1170766137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200101bf0 0x7f4200102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.064+0000 7f4205f9d700 1 --2- 172.21.15.62:0/1170766137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200102de0 0x7f4200103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.064+0000 7f4205f9d700 1 -- 172.21.15.62:0/1170766137 >> 172.21.15.62:0/1170766137 conn(0x7f42000fd1f0 msgr2=0x7f42000ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.064+0000 7f4205f9d700 1 -- 172.21.15.62:0/1170766137 shutdown_connections 2023-07-14T16:31:30.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.064+0000 7f4205f9d700 1 -- 172.21.15.62:0/1170766137 wait complete. 2023-07-14T16:31:30.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4205f9d700 1 Processor -- start 2023-07-14T16:31:30.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4205f9d700 1 -- start start 2023-07-14T16:31:30.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4205f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200101bf0 0x7f42001977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4205f9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200102de0 0x7f4200197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4205f9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200103770 0x7f420019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4205f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420019f350 con 0x7f4200101bf0 2023-07-14T16:31:30.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4205f9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420019f490 con 0x7f4200103770 2023-07-14T16:31:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4205f9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420019f620 con 0x7f4200102de0 2023-07-14T16:31:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.065+0000 7f4204f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200101bf0 0x7f42001977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f4204f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200101bf0 0x7f42001977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36952/0 (socket says 172.21.15.62:36952) 2023-07-14T16:31:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f4204f9b700 1 -- 172.21.15.62:0/826551003 learned_addr learned my addr 172.21.15.62:0/826551003 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f41f7fff700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200102de0 0x7f4200197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f420579c700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200103770 0x7f420019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f4204f9b700 1 -- 172.21.15.62:0/826551003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200102de0 msgr2=0x7f4200197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f4204f9b700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200102de0 0x7f4200197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f4204f9b700 1 -- 172.21.15.62:0/826551003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200103770 msgr2=0x7f420019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f4204f9b700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200103770 0x7f420019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f4204f9b700 1 -- 172.21.15.62:0/826551003 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0009520 con 0x7f4200101bf0 2023-07-14T16:31:30.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.066+0000 7f4204f9b700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200101bf0 0x7f42001977d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00d980 tx=0x7f41fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.067+0000 7f41f5ffb700 1 -- 172.21.15.62:0/826551003 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc004020 con 0x7f4200101bf0 2023-07-14T16:31:30.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.067+0000 7f41f5ffb700 1 -- 172.21.15.62:0/826551003 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41fc00f690 con 0x7f4200101bf0 2023-07-14T16:31:30.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.067+0000 7f4205f9d700 1 -- 172.21.15.62:0/826551003 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f8009d00 con 0x7f4200101bf0 2023-07-14T16:31:30.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.067+0000 7f41f5ffb700 1 -- 172.21.15.62:0/826551003 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0039e0 con 0x7f4200101bf0 2023-07-14T16:31:30.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.067+0000 7f4205f9d700 1 -- 172.21.15.62:0/826551003 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420019fcd0 con 0x7f4200101bf0 2023-07-14T16:31:30.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.070+0000 7f41f5ffb700 1 -- 172.21.15.62:0/826551003 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f41fc00f800 con 0x7f4200101bf0 2023-07-14T16:31:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.071+0000 7f421aa51b80 1 Processor -- start 2023-07-14T16:31:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.072+0000 7f421aa51b80 1 -- start start 2023-07-14T16:31:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.072+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604805797a0 0x560480579bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.072+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057a0f0 0x56048057a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.072+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x56048057b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.072+0000 7f421aa51b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048047ed30 con 0x56048057aeb0 2023-07-14T16:31:30.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.072+0000 7f421aa51b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604804d1ac0 con 0x5604805797a0 2023-07-14T16:31:30.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.072+0000 7f421aa51b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604804caf50 con 0x56048057a0f0 2023-07-14T16:31:30.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604805797a0 0x560480579bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604805797a0 0x560480579bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36460/0 (socket says 172.21.15.62:36460) 2023-07-14T16:31:30.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f41f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x56048057b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 -- 172.21.15.62:0/3350775550 learned_addr learned my addr 172.21.15.62:0/3350775550 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f4204f9b700 1 --2- 172.21.15.62:0/3350775550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057a0f0 0x56048057a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f41f5ffb700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1270 0x7f41e80a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f41f5ffb700 1 -- 172.21.15.62:0/826551003 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41fc0cc4e0 con 0x7f4200101bf0 2023-07-14T16:31:30.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 -- 172.21.15.62:0/3350775550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057a0f0 msgr2=0x56048057a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 --2- 172.21.15.62:0/3350775550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057a0f0 0x56048057a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 -- 172.21.15.62:0/3350775550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 msgr2=0x56048057b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 --2- 172.21.15.62:0/3350775550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x56048057b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 -- 172.21.15.62:0/3350775550 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604802f8e60 con 0x5604805797a0 2023-07-14T16:31:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.073+0000 7f420579c700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1270 0x7f41e80a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.074+0000 7f420579c700 1 --2- 172.21.15.62:0/3350775550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604805797a0 0x560480579bb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41f800b0a0 tx=0x7f41f8007690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f04ea636b5cdb7e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.074+0000 7f41e6ffd700 1 -- 172.21.15.62:0/3350775550 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800e040 con 0x5604805797a0 2023-07-14T16:31:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.074+0000 7f41e6ffd700 1 -- 172.21.15.62:0/3350775550 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41f8003e90 con 0x5604805797a0 2023-07-14T16:31:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.074+0000 7f41e6ffd700 1 -- 172.21.15.62:0/3350775550 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f801c830 con 0x5604805797a0 2023-07-14T16:31:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.074+0000 7f420579c700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1270 0x7f41e80a3720 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f41f802a000 tx=0x7f41f8027000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 -- 172.21.15.62:0/3350775550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604805797a0 msgr2=0x560480579bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 --2- 172.21.15.62:0/3350775550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604805797a0 0x560480579bb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f41f800b0a0 tx=0x7f41f8007690 comp rx=0 tx=0).stop 2023-07-14T16:31:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 -- 172.21.15.62:0/3350775550 shutdown_connections 2023-07-14T16:31:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 --2- 172.21.15.62:0/3350775550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x56048057b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 --2- 172.21.15.62:0/3350775550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5604805797a0 0x560480579bb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 --2- 172.21.15.62:0/3350775550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057a0f0 0x56048057a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 -- 172.21.15.62:0/3350775550 >> 172.21.15.62:0/3350775550 conn(0x56048056f070 msgr2=0x5604805744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 -- 172.21.15.62:0/3350775550 shutdown_connections 2023-07-14T16:31:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.075+0000 7f421aa51b80 1 -- 172.21.15.62:0/3350775550 wait complete. 2023-07-14T16:31:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.076+0000 7f421aa51b80 1 -- start start 2023-07-14T16:31:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.076+0000 7f421aa51b80 1 Processor -- start 2023-07-14T16:31:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604805797a0 0x5604805833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480583920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057aeb0 0x56048058a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f421aa51b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048046ab40 con 0x5604805797a0 2023-07-14T16:31:30.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f421aa51b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048046a840 con 0x56048057a0f0 2023-07-14T16:31:30.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f421aa51b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048034e9a0 con 0x56048057aeb0 2023-07-14T16:31:30.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f421aa51b80 10 client.0 _ll_register_callbacks cb 0x5604804687e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:30.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f41f7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480583920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f4204f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604805797a0 0x5604805833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f41f7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480583920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:36462/0 (socket says 172.21.15.62:36462) 2023-07-14T16:31:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f41f7fff700 1 -- 172.21.15.62:0/1084342777 learned_addr learned my addr 172.21.15.62:0/1084342777 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f420579c700 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057aeb0 0x56048058a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f4204f9b700 1 -- 172.21.15.62:0/1084342777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057aeb0 msgr2=0x56048058a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f4204f9b700 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057aeb0 0x56048058a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f4204f9b700 1 -- 172.21.15.62:0/1084342777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 msgr2=0x560480583920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.077+0000 7f4204f9b700 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480583920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f4204f9b700 1 -- 172.21.15.62:0/1084342777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420010a960 con 0x5604805797a0 2023-07-14T16:31:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f420579c700 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057aeb0 0x56048058a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f4204f9b700 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604805797a0 0x5604805833e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f41fc096310 tx=0x7f41fc099670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4d485e6e6206cc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f41dffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f41dffff700 1 -- 172.21.15.62:0/1084342777 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0cc2d0 con 0x5604805797a0 2023-07-14T16:31:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f421aa51b80 1 -- 172.21.15.62:0/1084342777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560480424570 con 0x5604805797a0 2023-07-14T16:31:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f41dffff700 1 -- 172.21.15.62:0/1084342777 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41fc00fb00 con 0x5604805797a0 2023-07-14T16:31:30.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f41dffff700 1 -- 172.21.15.62:0/1084342777 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41fc0998a0 con 0x5604805797a0 2023-07-14T16:31:30.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f421aa51b80 10 client.19989 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:30.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.078+0000 7f421aa51b80 1 -- 172.21.15.62:0/1084342777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5604802e1ce0 con 0x5604805797a0 2023-07-14T16:31:30.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f41dffff700 1 -- 172.21.15.62:0/1084342777 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560480424570 con 0x5604805797a0 2023-07-14T16:31:30.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f41dffff700 1 -- 172.21.15.62:0/1084342777 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41fc098d90 con 0x5604805797a0 2023-07-14T16:31:30.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f421aa51b80 10 client.19989 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:30.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f421aa51b80 10 client.19989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:30.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f421aa51b80 20 client.19989 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:30.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56048058f040 0x560480591450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f421aa51b80 4 client.19989 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:30.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f421aa51b80 1 -- 172.21.15.62:0/1084342777 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604804c3f30 con 0x56048058f040 2023-07-14T16:31:30.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.079+0000 7f41f7fff700 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56048058f040 0x560480591450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.080+0000 7f41f7fff700 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56048058f040 0x560480591450 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464166a6c6702e4a server_cookie=d9e3ba1904120637 in_seq=0 out_seq=0 2023-07-14T16:31:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.080+0000 7f41dffff700 10 client.19989 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f41dffff700 1 -- 172.21.15.62:0/1084342777 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5604804c3f30 con 0x56048058f040 2023-07-14T16:31:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f41dffff700 10 client.19989 handle_command_reply: tid=0 2023-07-14T16:31:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 client.19989 shutdown 2023-07-14T16:31:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 -- 172.21.15.62:0/1084342777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604805797a0 msgr2=0x5604805833e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604805797a0 0x5604805833e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f41fc096310 tx=0x7f41fc099670 comp rx=0 tx=0).stop 2023-07-14T16:31:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 -- 172.21.15.62:0/1084342777 shutdown_connections 2023-07-14T16:31:30.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56048057aeb0 0x56048058a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5604805797a0 0x5604805833e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56048058f040 0x560480591450 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1084342777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480583920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 -- 172.21.15.62:0/1084342777 >> 172.21.15.62:0/1084342777 conn(0x56048056f070 msgr2=0x560480573b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f41dffff700 0 client.19989 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 -- 172.21.15.62:0/1084342777 shutdown_connections 2023-07-14T16:31:30.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.081+0000 7f421aa51b80 1 -- 172.21.15.62:0/1084342777 wait complete. 2023-07-14T16:31:30.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.082+0000 7f421aa51b80 20 client.19989 trim_cache size 0 max 16384 2023-07-14T16:31:30.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.104+0000 7f421aa51b80 1 Processor -- start 2023-07-14T16:31:30.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f421aa51b80 1 -- start start 2023-07-14T16:31:30.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480573b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057a0f0 0x56048058a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057aeb0 0x560480591050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f421aa51b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048058ee20 con 0x56048057a0f0 2023-07-14T16:31:30.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f421aa51b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048057a710 con 0x56048057aeb0 2023-07-14T16:31:30.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f421aa51b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604805740c0 con 0x5604805797a0 2023-07-14T16:31:30.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f41f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057a0f0 0x56048058a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f41f7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057a0f0 0x56048058a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36992/0 (socket says 172.21.15.62:36992) 2023-07-14T16:31:30.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f41f7fff700 1 -- 172.21.15.62:0/1731875888 learned_addr learned my addr 172.21.15.62:0/1731875888 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f4204f9b700 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480573b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.105+0000 7f41f7fff700 1 -- 172.21.15.62:0/1731875888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 msgr2=0x560480573b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f41f7fff700 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480573b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f41f7fff700 1 -- 172.21.15.62:0/1731875888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057aeb0 msgr2=0x560480591050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f41f7fff700 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057aeb0 0x560480591050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f41f7fff700 1 -- 172.21.15.62:0/1731875888 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56048031f2d0 con 0x56048057a0f0 2023-07-14T16:31:30.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f4204f9b700 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480573b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:31:30.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f41f7fff700 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057a0f0 0x56048058a920 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009c10 tx=0x7f41f001fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ebef253f93b4c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f41e77fe700 1 -- 172.21.15.62:0/1731875888 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0012310 con 0x56048057a0f0 2023-07-14T16:31:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f41e77fe700 1 -- 172.21.15.62:0/1731875888 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41f0023020 con 0x56048057a0f0 2023-07-14T16:31:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.106+0000 7f41e77fe700 1 -- 172.21.15.62:0/1731875888 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f001f070 con 0x56048057a0f0 2023-07-14T16:31:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 -- 172.21.15.62:0/1731875888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057a0f0 msgr2=0x56048058a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057a0f0 0x56048058a920 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009c10 tx=0x7f41f001fd20 comp rx=0 tx=0).stop 2023-07-14T16:31:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 -- 172.21.15.62:0/1731875888 shutdown_connections 2023-07-14T16:31:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057aeb0 0x560480591050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480573b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 --2- 172.21.15.62:0/1731875888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057a0f0 0x56048058a920 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 -- 172.21.15.62:0/1731875888 >> 172.21.15.62:0/1731875888 conn(0x56048056f070 msgr2=0x560480575230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 -- 172.21.15.62:0/1731875888 shutdown_connections 2023-07-14T16:31:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 -- 172.21.15.62:0/1731875888 wait complete. 2023-07-14T16:31:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.107+0000 7f421aa51b80 1 -- start start 2023-07-14T16:31:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f421aa51b80 1 Processor -- start 2023-07-14T16:31:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480587ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480587ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f421aa51b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x560480589000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f421aa51b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560480589540 con 0x56048057aeb0 2023-07-14T16:31:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f421aa51b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604805896b0 con 0x56048057a0f0 2023-07-14T16:31:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f421aa51b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560480589870 con 0x5604805797a0 2023-07-14T16:31:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f421aa51b80 10 client.0 _ll_register_callbacks cb 0x7f41fc0b20f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.108+0000 7f420579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x560480589000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f41f7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480587ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f420579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x560480589000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36996/0 (socket says 172.21.15.62:36996) 2023-07-14T16:31:30.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f420579c700 1 -- 172.21.15.62:0/187332122 learned_addr learned my addr 172.21.15.62:0/187332122 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f4204f9b700 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480587ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f41f7fff700 1 -- 172.21.15.62:0/187332122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 msgr2=0x560480587ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f41f7fff700 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480587ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f41f7fff700 1 -- 172.21.15.62:0/187332122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 msgr2=0x560480589000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f41f7fff700 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x560480589000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f41f7fff700 1 -- 172.21.15.62:0/187332122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f0005ce0 con 0x56048057a0f0 2023-07-14T16:31:30.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f420579c700 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x560480589000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:30.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f41f7fff700 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480587ff0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005e90 tx=0x7f41f00046a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c129151bdbc6d28 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.109+0000 7f41dffff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:31:30.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.110+0000 7f41dffff700 1 -- 172.21.15.62:0/187332122 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0012520 con 0x56048057a0f0 2023-07-14T16:31:30.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.110+0000 7f41dffff700 1 -- 172.21.15.62:0/187332122 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41f001fe40 con 0x56048057a0f0 2023-07-14T16:31:30.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.110+0000 7f421aa51b80 1 -- 172.21.15.62:0/187332122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41f8003e90 con 0x56048057a0f0 2023-07-14T16:31:30.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.110+0000 7f41dffff700 1 -- 172.21.15.62:0/187332122 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41f0037030 con 0x56048057a0f0 2023-07-14T16:31:30.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.110+0000 7f421aa51b80 10 client.29522 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:30.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.110+0000 7f421aa51b80 1 -- 172.21.15.62:0/187332122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5604802f7330 con 0x56048057a0f0 2023-07-14T16:31:30.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f41dffff700 1 -- 172.21.15.62:0/187332122 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41f8003e90 con 0x56048057a0f0 2023-07-14T16:31:30.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f41dffff700 1 -- 172.21.15.62:0/187332122 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41f0010660 con 0x56048057a0f0 2023-07-14T16:31:30.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f421aa51b80 10 client.29522 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:30.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f421aa51b80 10 client.29522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:30.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f421aa51b80 20 client.29522 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:30.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f421aa51b80 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604805e50a0 0x5604805e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f421aa51b80 4 client.29522 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:31:30.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f421aa51b80 1 -- 172.21.15.62:0/187332122 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5604804860d0 con 0x5604805e50a0 2023-07-14T16:31:30.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.111+0000 7f4204f9b700 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604805e50a0 0x5604805e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.112+0000 7f4204f9b700 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604805e50a0 0x5604805e7480 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7dab50770020c8 server_cookie=232a60d459cd20a6 in_seq=0 out_seq=0 2023-07-14T16:31:30.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.112+0000 7f41dffff700 10 client.29522 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.113+0000 7f41dffff700 1 -- 172.21.15.62:0/187332122 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5604804860d0 con 0x5604805e50a0 2023-07-14T16:31:30.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.113+0000 7f41dffff700 10 client.29522 handle_command_reply: tid=0 2023-07-14T16:31:30.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.113+0000 7f421aa51b80 1 client.29522 shutdown 2023-07-14T16:31:30.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.113+0000 7f421aa51b80 1 -- 172.21.15.62:0/187332122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 msgr2=0x560480587ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.113+0000 7f421aa51b80 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480587ff0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005e90 tx=0x7f41f00046a0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 1 -- 172.21.15.62:0/187332122 shutdown_connections 2023-07-14T16:31:30.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56048057aeb0 0x560480589000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604805797a0 0x560480587ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5604805e50a0 0x5604805e7480 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 1 --2- 172.21.15.62:0/187332122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56048057a0f0 0x560480587ff0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f41dffff700 0 client.29522 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 1 -- 172.21.15.62:0/187332122 >> 172.21.15.62:0/187332122 conn(0x56048056f070 msgr2=0x560480581260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 1 -- 172.21.15.62:0/187332122 shutdown_connections 2023-07-14T16:31:30.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 1 -- 172.21.15.62:0/187332122 wait complete. 2023-07-14T16:31:30.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.114+0000 7f421aa51b80 20 client.29522 trim_cache size 0 max 16384 2023-07-14T16:31:30.093 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:31:30.093 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:31:30.094 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "79e669fc-b9b6-46a1-b8be-2b37b182f431", 2023-07-14T16:31:30.094 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:31:30.094 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:31:30.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 -- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1270 msgr2=0x7f41e80a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1270 0x7f41e80a3720 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f41f802a000 tx=0x7f41f8027000 comp rx=0 tx=0).stop 2023-07-14T16:31:30.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 -- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200101bf0 msgr2=0x7f42001977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200101bf0 0x7f42001977d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00d980 tx=0x7f41fc00dc90 comp rx=0 tx=0).stop 2023-07-14T16:31:30.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 -- 172.21.15.62:0/826551003 shutdown_connections 2023-07-14T16:31:30.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f41e80a1270 0x7f41e80a3720 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4200103770 0x7f420019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4200101bf0 0x7f42001977d0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 --2- 172.21.15.62:0/826551003 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4200102de0 0x7f4200197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 -- 172.21.15.62:0/826551003 >> 172.21.15.62:0/826551003 conn(0x7f42000fd1f0 msgr2=0x7f42000ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 -- 172.21.15.62:0/826551003 shutdown_connections 2023-07-14T16:31:30.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.115+0000 7f4205f9d700 1 -- 172.21.15.62:0/826551003 wait complete. 2023-07-14T16:31:30.110 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:31:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 -- 172.21.15.62:0/3432782402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4100560 msgr2=0x7fd8d4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/3432782402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4100560 0x7fd8d4102940 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009990 tx=0x7fd8c4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 -- 172.21.15.62:0/3432782402 shutdown_connections 2023-07-14T16:31:30.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/3432782402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8d41057a0 0x7fd8d4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/3432782402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4100560 0x7fd8d4102940 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/3432782402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d4102e80 0x7fd8d4105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 -- 172.21.15.62:0/3432782402 >> 172.21.15.62:0/3432782402 conn(0x7fd8d40fa000 msgr2=0x7fd8d40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 -- 172.21.15.62:0/3432782402 shutdown_connections 2023-07-14T16:31:30.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.363+0000 7fd8d9dff700 1 -- 172.21.15.62:0/3432782402 wait complete. 2023-07-14T16:31:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.365+0000 7fd8d9dff700 1 Processor -- start 2023-07-14T16:31:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.365+0000 7fd8d9dff700 1 -- start start 2023-07-14T16:31:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.365+0000 7fd8d9dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8d4100560 0x7fd8d40ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.365+0000 7fd8d9dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d4102e80 0x7fd8d40fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d9dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d41057a0 0x7fd8d40fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d9dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40ff1f0 con 0x7fd8d41057a0 2023-07-14T16:31:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d9dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40ff360 con 0x7fd8d4102e80 2023-07-14T16:31:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d9dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d419f7f0 con 0x7fd8d4100560 2023-07-14T16:31:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d41057a0 0x7fd8d40fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d41057a0 0x7fd8d40fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37016/0 (socket says 172.21.15.62:37016) 2023-07-14T16:31:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d3fff700 1 -- 172.21.15.62:0/1335378591 learned_addr learned my addr 172.21.15.62:0/1335378591 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d4102e80 0x7fd8d40fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8d4100560 0x7fd8d40ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d3fff700 1 -- 172.21.15.62:0/1335378591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8d4100560 msgr2=0x7fd8d40ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d3fff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8d4100560 0x7fd8d40ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d3fff700 1 -- 172.21.15.62:0/1335378591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d4102e80 msgr2=0x7fd8d40fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d3fff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d4102e80 0x7fd8d40fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.366+0000 7fd8d3fff700 1 -- 172.21.15.62:0/1335378591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40095f0 con 0x7fd8d41057a0 2023-07-14T16:31:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.367+0000 7fd8d3fff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d41057a0 0x7fd8d40fea50 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00ef30 tx=0x7fd8cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.367+0000 7fd8d0ff9700 1 -- 172.21.15.62:0/1335378591 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc018070 con 0x7fd8d41057a0 2023-07-14T16:31:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.367+0000 7fd8d0ff9700 1 -- 172.21.15.62:0/1335378591 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8cc00f040 con 0x7fd8d41057a0 2023-07-14T16:31:30.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.367+0000 7fd8d9dff700 1 -- 172.21.15.62:0/1335378591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d419fa70 con 0x7fd8d41057a0 2023-07-14T16:31:30.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.367+0000 7fd8d0ff9700 1 -- 172.21.15.62:0/1335378591 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc00cbc0 con 0x7fd8d41057a0 2023-07-14T16:31:30.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.367+0000 7fd8d9dff700 1 -- 172.21.15.62:0/1335378591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d419ff80 con 0x7fd8d41057a0 2023-07-14T16:31:30.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.371+0000 7fd8d0ff9700 1 -- 172.21.15.62:0/1335378591 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd8cc00cd80 con 0x7fd8d41057a0 2023-07-14T16:31:30.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.372+0000 7fd8ed6ecb80 1 Processor -- start 2023-07-14T16:31:30.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.372+0000 7fd8ed6ecb80 1 -- start start 2023-07-14T16:31:30.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e8954d0 0x56125e8958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e895e20 0x56125e8966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e896c70 0x56125e8970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e79acb0 con 0x56125e8954d0 2023-07-14T16:31:30.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e7eda40 con 0x56125e896c70 2023-07-14T16:31:30.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e7e6ed0 con 0x56125e895e20 2023-07-14T16:31:30.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8d0ff9700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8bc0a1150 0x7fd8bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e8954d0 0x56125e8958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.373+0000 7fd8d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e8954d0 0x56125e8958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37026/0 (socket says 172.21.15.62:37026) 2023-07-14T16:31:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/2093101901 learned_addr learned my addr 172.21.15.62:0/2093101901 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e895e20 0x56125e8966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d0ff9700 1 -- 172.21.15.62:0/1335378591 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8cc0cc3e0 con 0x7fd8d41057a0 2023-07-14T16:31:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d3fff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8bc0a1150 0x7fd8bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e896c70 0x56125e8970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d37fe700 1 -- 172.21.15.62:0/2093101901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e896c70 msgr2=0x56125e8970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e896c70 0x56125e8970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d37fe700 1 -- 172.21.15.62:0/2093101901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e8954d0 msgr2=0x56125e8958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e8954d0 0x56125e8958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d37fe700 1 -- 172.21.15.62:0/2093101901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56125e614de0 con 0x56125e895e20 2023-07-14T16:31:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e8954d0 0x56125e8958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.374+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e895e20 0x56125e8966e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40098e0 tx=0x7fd8c4005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e752e00446af522 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.375+0000 7fd8d3fff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8bc0a1150 0x7fd8bc0a3600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc04f060 tx=0x7fd8cc009ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.375+0000 7fd8b9ffb700 1 -- 172.21.15.62:0/2093101901 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c401d070 con 0x56125e895e20 2023-07-14T16:31:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.375+0000 7fd8b9ffb700 1 -- 172.21.15.62:0/2093101901 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8c401e040 con 0x56125e895e20 2023-07-14T16:31:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.375+0000 7fd8b9ffb700 1 -- 172.21.15.62:0/2093101901 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400b710 con 0x56125e895e20 2023-07-14T16:31:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2093101901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e895e20 msgr2=0x56125e8966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e895e20 0x56125e8966e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40098e0 tx=0x7fd8c4005f00 comp rx=0 tx=0).stop 2023-07-14T16:31:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2093101901 shutdown_connections 2023-07-14T16:31:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e896c70 0x56125e8970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e8954d0 0x56125e8958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/2093101901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e895e20 0x56125e8966e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2093101901 >> 172.21.15.62:0/2093101901 conn(0x56125e88ada0 msgr2=0x56125e8901d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2093101901 shutdown_connections 2023-07-14T16:31:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.376+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2093101901 wait complete. 2023-07-14T16:31:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.377+0000 7fd8ed6ecb80 1 -- start start 2023-07-14T16:31:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.377+0000 7fd8ed6ecb80 1 Processor -- start 2023-07-14T16:31:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.377+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e89f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.377+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e89f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.377+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.377+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e786ac0 con 0x56125e896c70 2023-07-14T16:31:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.377+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e7867c0 con 0x56125e895e20 2023-07-14T16:31:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.377+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e66a920 con 0x56125e8954d0 2023-07-14T16:31:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8ed6ecb80 10 client.0 _ll_register_callbacks cb 0x56125e784760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e89f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e89f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60002/0 (socket says 172.21.15.62:60002) 2023-07-14T16:31:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/3063216112 learned_addr learned my addr 172.21.15.62:0/3063216112 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e89f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/3063216112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 msgr2=0x56125e89f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e89f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/3063216112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 msgr2=0x56125e8a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.378+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/3063216112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d41087f0 con 0x56125e8954d0 2023-07-14T16:31:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.379+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e89f120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8010ed0 tx=0x7fd8c8009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cb5808f6d426025 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.379+0000 7fd8b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:31:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.379+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/3063216112 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800f9b0 con 0x56125e8954d0 2023-07-14T16:31:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.379+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/3063216112 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8c800fb10 con 0x56125e8954d0 2023-07-14T16:31:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.379+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/3063216112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56125e7404f0 con 0x56125e8954d0 2023-07-14T16:31:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.379+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/3063216112 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c800fcd0 con 0x56125e8954d0 2023-07-14T16:31:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8ed6ecb80 10 client.29401 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/3063216112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56125e5fdc60 con 0x56125e8954d0 2023-07-14T16:31:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/3063216112 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56125e7404f0 con 0x56125e8954d0 2023-07-14T16:31:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/3063216112 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd8c8005220 con 0x56125e8954d0 2023-07-14T16:31:30.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8ed6ecb80 10 client.29401 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:30.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8ed6ecb80 10 client.29401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:30.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8ed6ecb80 20 client.29401 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56125e8aaeb0 0x56125e8ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8ed6ecb80 4 client.29401 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.380+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/3063216112 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56125e7dfeb0 con 0x56125e8aaeb0 2023-07-14T16:31:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.381+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56125e8aaeb0 0x56125e8ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.382+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56125e8aaeb0 0x56125e8ad2c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8377c3f1c9e58ab4 server_cookie=163e9b3df789f264 in_seq=0 out_seq=0 2023-07-14T16:31:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.382+0000 7fd8b2ffd700 10 client.29401 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.382+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/3063216112 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56125e7dfeb0 con 0x56125e8aaeb0 2023-07-14T16:31:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.382+0000 7fd8b2ffd700 10 client.29401 handle_command_reply: tid=0 2023-07-14T16:31:30.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.382+0000 7fd8ed6ecb80 1 client.29401 shutdown 2023-07-14T16:31:30.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/3063216112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 msgr2=0x56125e89f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e89f120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8010ed0 tx=0x7fd8c8009830 comp rx=0 tx=0).stop 2023-07-14T16:31:30.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/3063216112 shutdown_connections 2023-07-14T16:31:30.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e89f120 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.368 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56125e8aaeb0 0x56125e8ad2c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/3063216112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e89f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/3063216112 >> 172.21.15.62:0/3063216112 conn(0x56125e88ada0 msgr2=0x56125e88f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8b2ffd700 0 client.29401 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/3063216112 shutdown_connections 2023-07-14T16:31:30.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/3063216112 wait complete. 2023-07-14T16:31:30.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.383+0000 7fd8ed6ecb80 20 client.29401 trim_cache size 0 max 16384 2023-07-14T16:31:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 -- 172.21.15.62:0/1265379167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8102bf0 msgr2=0x7fdcf8105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1265379167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8102bf0 0x7fdcf8105050 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009990 tx=0x7fdce4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 -- 172.21.15.62:0/1265379167 shutdown_connections 2023-07-14T16:31:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1265379167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf81056e0 0x7fdcf8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1265379167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf8068620 0x7fdcf81026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1265379167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8102bf0 0x7fdcf8105050 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 -- 172.21.15.62:0/1265379167 >> 172.21.15.62:0/1265379167 conn(0x7fdcf80fa000 msgr2=0x7fdcf80fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 -- 172.21.15.62:0/1265379167 shutdown_connections 2023-07-14T16:31:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.400+0000 7fdcfedff700 1 -- 172.21.15.62:0/1265379167 wait complete. 2023-07-14T16:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.401+0000 7fdcfedff700 1 Processor -- start 2023-07-14T16:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfedff700 1 -- start start 2023-07-14T16:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfedff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8068620 0x7fdcf8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfedff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf8102bf0 0x7fdcf8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfedff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf81056e0 0x7fdcf8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfedff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8199500 con 0x7fdcf8068620 2023-07-14T16:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfedff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8199670 con 0x7fdcf8102bf0 2023-07-14T16:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfedff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8199830 con 0x7fdcf81056e0 2023-07-14T16:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfcb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8068620 0x7fdcf8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfcb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8068620 0x7fdcf8197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37046/0 (socket says 172.21.15.62:37046) 2023-07-14T16:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfcb9b700 1 -- 172.21.15.62:0/1037664558 learned_addr learned my addr 172.21.15.62:0/1037664558 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf81056e0 0x7fdcf8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.402+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf8102bf0 0x7fdcf8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.403+0000 7fdcfcb9b700 1 -- 172.21.15.62:0/1037664558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf81056e0 msgr2=0x7fdcf8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.403+0000 7fdcfcb9b700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf81056e0 0x7fdcf8198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.403+0000 7fdcfcb9b700 1 -- 172.21.15.62:0/1037664558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf8102bf0 msgr2=0x7fdcf8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.403+0000 7fdcfcb9b700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf8102bf0 0x7fdcf8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.403+0000 7fdcfcb9b700 1 -- 172.21.15.62:0/1037664558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40095f0 con 0x7fdcf8068620 2023-07-14T16:31:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.403+0000 7fdcfcb9b700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8068620 0x7fdcf8197ae0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e970 tx=0x7fdcec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.403+0000 7fdcf5ffb700 1 -- 172.21.15.62:0/1037664558 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec009790 con 0x7fdcf8068620 2023-07-14T16:31:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.404+0000 7fdcfedff700 1 -- 172.21.15.62:0/1037664558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf8199b10 con 0x7fdcf8068620 2023-07-14T16:31:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.404+0000 7fdcf5ffb700 1 -- 172.21.15.62:0/1037664558 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcec00c7e0 con 0x7fdcf8068620 2023-07-14T16:31:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.404+0000 7fdcf5ffb700 1 -- 172.21.15.62:0/1037664558 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec005440 con 0x7fdcf8068620 2023-07-14T16:31:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.404+0000 7fdcfedff700 1 -- 172.21.15.62:0/1037664558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf81a02d0 con 0x7fdcf8068620 2023-07-14T16:31:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.407+0000 7fd8ed6ecb80 1 Processor -- start 2023-07-14T16:31:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.407+0000 7fd8ed6ecb80 1 -- start start 2023-07-14T16:31:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.407+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e8954d0 0x56125e89d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e895e20 0x56125e8a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e896c70 0x56125e899c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e8a8390 con 0x56125e895e20 2023-07-14T16:31:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e89ceb0 con 0x56125e8954d0 2023-07-14T16:31:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e8964c0 con 0x56125e896c70 2023-07-14T16:31:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e895e20 0x56125e8a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e895e20 0x56125e8a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37050/0 (socket says 172.21.15.62:37050) 2023-07-14T16:31:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d37fe700 1 -- 172.21.15.62:0/2611199269 learned_addr learned my addr 172.21.15.62:0/2611199269 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e8954d0 0x56125e89d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e896c70 0x56125e899c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d37fe700 1 -- 172.21.15.62:0/2611199269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e896c70 msgr2=0x56125e899c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e896c70 0x56125e899c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d37fe700 1 -- 172.21.15.62:0/2611199269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e8954d0 msgr2=0x56125e89d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e8954d0 0x56125e89d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.408+0000 7fd8d37fe700 1 -- 172.21.15.62:0/2611199269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56125e63b250 con 0x56125e895e20 2023-07-14T16:31:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.409+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e895e20 0x56125e8a6630 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4000c00 tx=0x7fd8c401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eb92baf02f74a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.409+0000 7fdd127aeb80 1 Processor -- start 2023-07-14T16:31:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.409+0000 7fdcf5ffb700 1 -- 172.21.15.62:0/1037664558 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdcec00c950 con 0x7fdcf8068620 2023-07-14T16:31:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.409+0000 7fdd127aeb80 1 -- start start 2023-07-14T16:31:30.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.409+0000 7fd8ba7fc700 1 -- 172.21.15.62:0/2611199269 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c401d040 con 0x56125e895e20 2023-07-14T16:31:30.388 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.409+0000 7fd8ba7fc700 1 -- 172.21.15.62:0/2611199269 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8c4035580 con 0x56125e895e20 2023-07-14T16:31:30.389 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.409+0000 7fd8ba7fc700 1 -- 172.21.15.62:0/2611199269 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40052a0 con 0x56125e895e20 2023-07-14T16:31:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd352626b0 0x55cd35406300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd354071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35407b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3530bc50 con 0x55cd354076e0 2023-07-14T16:31:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3535e9e0 con 0x55cd352626b0 2023-07-14T16:31:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd35357e70 con 0x55cd35406840 2023-07-14T16:31:30.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2611199269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e895e20 msgr2=0x56125e8a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.390 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e895e20 0x56125e8a6630 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4000c00 tx=0x7fd8c401a040 comp rx=0 tx=0).stop 2023-07-14T16:31:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdcf7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35407b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdcf7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35407b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37056/0 (socket says 172.21.15.62:37056) 2023-07-14T16:31:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdcf7fff700 1 -- 172.21.15.62:0/3691699689 learned_addr learned my addr 172.21.15.62:0/3691699689 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdcfcb9b700 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd354071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd352626b0 0x55cd35406300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2611199269 shutdown_connections 2023-07-14T16:31:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e896c70 0x56125e899c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e8954d0 0x56125e89d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/2611199269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e895e20 0x56125e8a6630 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.410+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2611199269 >> 172.21.15.62:0/2611199269 conn(0x56125e88ada0 msgr2=0x56125e89ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2611199269 shutdown_connections 2023-07-14T16:31:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fdcfd39c700 1 -- 172.21.15.62:0/3691699689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 msgr2=0x55cd354071a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.394 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/2611199269 wait complete. 2023-07-14T16:31:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd354071a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fdcfd39c700 1 -- 172.21.15.62:0/3691699689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 msgr2=0x55cd35407b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35407b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fdcfd39c700 1 -- 172.21.15.62:0/3691699689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd35185d80 con 0x55cd352626b0 2023-07-14T16:31:30.395 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fd8ed6ecb80 1 -- start start 2023-07-14T16:31:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd352626b0 0x55cd35406300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009c40 tx=0x7fdcf0009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b071e5f149b1a7d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.396 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.411+0000 7fd8ed6ecb80 1 Processor -- start 2023-07-14T16:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fdce2ffd700 1 -- 172.21.15.62:0/3691699689 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0019070 con 0x55cd352626b0 2023-07-14T16:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fdcf5ffb700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdce80a1250 0x7fdce80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fdce2ffd700 1 -- 172.21.15.62:0/3691699689 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcf000f040 con 0x55cd352626b0 2023-07-14T16:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fdce2ffd700 1 -- 172.21.15.62:0/3691699689 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0014640 con 0x55cd352626b0 2023-07-14T16:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fdcf5ffb700 1 -- 172.21.15.62:0/1037664558 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdcec014070 con 0x7fdcf8068620 2023-07-14T16:31:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e8a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e8a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8ed6ecb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e8a60a0 con 0x56125e896c70 2023-07-14T16:31:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e8a6210 con 0x56125e895e20 2023-07-14T16:31:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8ed6ecb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125e8a63d0 con 0x56125e8954d0 2023-07-14T16:31:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8ed6ecb80 10 client.0 _ll_register_callbacks cb 0x7fd8c8010bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37068/0 (socket says 172.21.15.62:37068) 2023-07-14T16:31:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/1458018963 learned_addr learned my addr 172.21.15.62:0/1458018963 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.412+0000 7fdcfcb9b700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdce80a1250 0x7fdce80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 -- 172.21.15.62:0/3691699689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd352626b0 msgr2=0x55cd35406300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd352626b0 0x55cd35406300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009c40 tx=0x7fdcf0009f50 comp rx=0 tx=0).stop 2023-07-14T16:31:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e8a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e8a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 -- 172.21.15.62:0/3691699689 shutdown_connections 2023-07-14T16:31:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35407b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd352626b0 0x55cd35406300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/3691699689 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd354071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 -- 172.21.15.62:0/3691699689 >> 172.21.15.62:0/3691699689 conn(0x55cd353fbbe0 msgr2=0x55cd35400cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/1458018963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 msgr2=0x56125e8a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e8a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 msgr2=0x56125e8a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e8a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d2ffd700 1 -- 172.21.15.62:0/1458018963 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56125e614de0 con 0x56125e896c70 2023-07-14T16:31:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e8a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdcfcb9b700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdce80a1250 0x7fdce80a3700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdcec005950 tx=0x7fdcec0d2530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 -- 172.21.15.62:0/3691699689 shutdown_connections 2023-07-14T16:31:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fdd127aeb80 1 -- 172.21.15.62:0/3691699689 wait complete. 2023-07-14T16:31:30.404 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:31:30.127+0000 7fdd0678a700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003c5 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ auth{1=1} v3745 ap=2 f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0) n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXs/-@178} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5565df402000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003c5:[<0x10000000007/.deps v2379>,<0x10000000006/ffsb v2032>,<0x10000000005/tmp v43>,<0x10000000003/client.0 v41>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v52>,<0x10000000001/sv_0 v75>,<0x10000000000/_nogroup v60>,<0x1/volumes v56>]//[]","memoryvalue":"(2)0x100000003c5:[<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v285>,<0x10000000003/client.0 v286>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v298>,<0x10000000001/sv_0 v312>,<0x10000000000/_nogroup v306>,<0x1/volumes v296>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.384037+0000 19=19+0)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","memory_value.dirstat":"f(v36 m2023-07-14T16:17:25.419289+0000 18=18+0)","memory_value.rstat":"n(v1 rc2023-07-14T16:17:25.419289+0000 b116303 19=18+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-14T16:31:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8d2ffd700 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a5b60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8001eb0 tx=0x7fd8c80105c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27202246c0d4a74f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.413+0000 7fd8b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.414+0000 7fdd127aeb80 1 -- start start 2023-07-14T16:31:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.414+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/1458018963 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800fe70 con 0x56125e896c70 2023-07-14T16:31:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.414+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/1458018963 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8c401e040 con 0x56125e896c70 2023-07-14T16:31:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.414+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/1458018963 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8c8000e40 con 0x56125e896c70 2023-07-14T16:31:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.414+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/1458018963 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c8005820 con 0x56125e896c70 2023-07-14T16:31:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.414+0000 7fdd127aeb80 1 Processor -- start 2023-07-14T16:31:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.414+0000 7fd8ed6ecb80 10 client.20022 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.414+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/1458018963 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56125e6132b0 con 0x56125e896c70 2023-07-14T16:31:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/1458018963 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8c401e040 con 0x56125e896c70 2023-07-14T16:31:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd3540fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd354076e0 0x55cd35416e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd352f7a60 con 0x55cd352626b0 2023-07-14T16:31:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd352f7760 con 0x55cd354076e0 2023-07-14T16:31:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd351db8c0 con 0x55cd35406840 2023-07-14T16:31:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdd127aeb80 10 client.0 _ll_register_callbacks cb 0x55cd352f5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:30.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/1458018963 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd8c8021620 con 0x56125e896c70 2023-07-14T16:31:30.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fd8ed6ecb80 10 client.20022 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:30.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fd8ed6ecb80 10 client.20022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:30.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fd8ed6ecb80 20 client.20022 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:30.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56125e9086a0 0x56125e90aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.410 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fd8ed6ecb80 4 client.20022 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:31:30.411 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/1458018963 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56125e7a2050 con 0x56125e9086a0 2023-07-14T16:31:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdcfd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdcfd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37070/0 (socket says 172.21.15.62:37070) 2023-07-14T16:31:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdcfd39c700 1 -- 172.21.15.62:0/1331588318 learned_addr learned my addr 172.21.15.62:0/1331588318 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.415+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd3540fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd354076e0 0x55cd35416e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.412 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56125e9086a0 0x56125e90aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcfd39c700 1 -- 172.21.15.62:0/1331588318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 msgr2=0x55cd3540fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd3540fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcfd39c700 1 -- 172.21.15.62:0/1331588318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd354076e0 msgr2=0x55cd35416e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd354076e0 0x55cd35416e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcfd39c700 1 -- 172.21.15.62:0/1331588318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf0005e70 con 0x55cd352626b0 2023-07-14T16:31:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd3540fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540f890 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009420 tx=0x7fdcf00093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5270d1138fc3082 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fd8d37fe700 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56125e9086a0 0x56125e90aab0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60862a518e719b50 server_cookie=65dbf5f200719eae in_seq=0 out_seq=0 2023-07-14T16:31:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fd8b2ffd700 10 client.20022 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcd7fff700 1 -- 172.21.15.62:0/1331588318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0019040 con 0x55cd352626b0 2023-07-14T16:31:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdcd7fff700 1 -- 172.21.15.62:0/1331588318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdcf00149a0 con 0x55cd352626b0 2023-07-14T16:31:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.416+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1331588318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd352b1490 con 0x55cd352626b0 2023-07-14T16:31:30.416 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8b2ffd700 1 -- 172.21.15.62:0/1458018963 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56125e7a2050 con 0x56125e9086a0 2023-07-14T16:31:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8b2ffd700 10 client.20022 handle_command_reply: tid=0 2023-07-14T16:31:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 client.20022 shutdown 2023-07-14T16:31:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fdcd7fff700 1 -- 172.21.15.62:0/1331588318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcf0014b60 con 0x55cd352626b0 2023-07-14T16:31:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fdd127aeb80 10 client.20025 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1331588318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55cd3516ec00 con 0x55cd352626b0 2023-07-14T16:31:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/1458018963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 msgr2=0x56125e8a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a5b60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8001eb0 tx=0x7fd8c80105c0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/1458018963 shutdown_connections 2023-07-14T16:31:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56125e896c70 0x56125e8a5b60 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56125e8954d0 0x56125e8a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x56125e9086a0 0x56125e90aab0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 --2- 172.21.15.62:0/1458018963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56125e895e20 0x56125e8a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/1458018963 >> 172.21.15.62:0/1458018963 conn(0x56125e88ada0 msgr2=0x56125e89ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8b2ffd700 0 client.20022 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fdcd7fff700 1 -- 172.21.15.62:0/1331588318 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cd352b1490 con 0x55cd352626b0 2023-07-14T16:31:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/1458018963 shutdown_connections 2023-07-14T16:31:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fd8ed6ecb80 1 -- 172.21.15.62:0/1458018963 wait complete. 2023-07-14T16:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.417+0000 7fdcd7fff700 1 -- 172.21.15.62:0/1331588318 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdcf0022cf0 con 0x55cd352626b0 2023-07-14T16:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fdd127aeb80 10 client.20025 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fdd127aeb80 10 client.20025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fdd127aeb80 20 client.20025 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd3541b5f0 0x55cd3541da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8ed6ecb80 20 client.20022 trim_cache size 0 max 16384 2023-07-14T16:31:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fdd127aeb80 4 client.20025 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1331588318 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd35350e50 con 0x55cd3541b5f0 2023-07-14T16:31:30.423 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:31:30.423 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:31:30.423 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:31:30.424 INFO:teuthology.orchestra.run.smithi062.stdout: "79e669fc-b9b6-46a1-b8be-2b37b182f431": { 2023-07-14T16:31:30.424 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:31:30.424 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "79e669fc-b9b6-46a1-b8be-2b37b182f431", 2023-07-14T16:31:30.424 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:31:30.424 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:31:30.424 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:31:30.425 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:31:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd3541b5f0 0x55cd3541da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 -- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8bc0a1150 msgr2=0x7fd8bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8bc0a1150 0x7fd8bc0a3600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc04f060 tx=0x7fd8cc009ed0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 -- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d41057a0 msgr2=0x7fd8d40fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d41057a0 0x7fd8d40fea50 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00ef30 tx=0x7fd8cc00c3a0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 -- 172.21.15.62:0/1335378591 shutdown_connections 2023-07-14T16:31:30.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d41057a0 0x7fd8d40fea50 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8d4100560 0x7fd8d40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd8bc0a1150 0x7fd8bc0a3600 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 --2- 172.21.15.62:0/1335378591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8d4102e80 0x7fd8d40fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.418+0000 7fd8d9dff700 1 -- 172.21.15.62:0/1335378591 >> 172.21.15.62:0/1335378591 conn(0x7fd8d40fa000 msgr2=0x7fd8d41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.419+0000 7fd8d9dff700 1 -- 172.21.15.62:0/1335378591 shutdown_connections 2023-07-14T16:31:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:31:30.419+0000 7fd8d9dff700 1 -- 172.21.15.62:0/1335378591 wait complete. 2023-07-14T16:31:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.419+0000 7fdcd7fff700 10 client.20025 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.419+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd3541b5f0 0x55cd3541da00 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa4248804247902 server_cookie=ef7264b7409e89a5 in_seq=0 out_seq=0 2023-07-14T16:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.419+0000 7fdcd7fff700 1 -- 172.21.15.62:0/1331588318 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd35350e50 con 0x55cd3541b5f0 2023-07-14T16:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.419+0000 7fdcd7fff700 10 client.20025 handle_command_reply: tid=0 2023-07-14T16:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 client.20025 shutdown 2023-07-14T16:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1331588318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 msgr2=0x55cd3540f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540f890 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009420 tx=0x7fdcf00093b0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1331588318 shutdown_connections 2023-07-14T16:31:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd3541b5f0 0x55cd3541da00 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd354076e0 0x55cd35416e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540f890 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1331588318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd35406840 0x55cd3540fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1331588318 >> 172.21.15.62:0/1331588318 conn(0x55cd353fbbe0 msgr2=0x55cd354004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdcd7fff700 0 client.20025 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1331588318 shutdown_connections 2023-07-14T16:31:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1331588318 wait complete. 2023-07-14T16:31:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.420+0000 7fdd127aeb80 20 client.20025 trim_cache size 0 max 16384 2023-07-14T16:31:30.432 INFO:tasks.cephfs.filesystem:scrub status for tag:79e669fc-b9b6-46a1-b8be-2b37b182f431 - {'path': '/', 'tag': '79e669fc-b9b6-46a1-b8be-2b37b182f431', 'options': 'recursive,force'} 2023-07-14T16:31:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.438+0000 7fdd127aeb80 1 Processor -- start 2023-07-14T16:31:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdd127aeb80 1 -- start start 2023-07-14T16:31:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35416dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd354076e0 0x55cd3540a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd35350e50 con 0x55cd352626b0 2023-07-14T16:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd352b1490 con 0x55cd35406840 2023-07-14T16:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd353cc060 con 0x55cd354076e0 2023-07-14T16:31:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdcfd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdcfd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37074/0 (socket says 172.21.15.62:37074) 2023-07-14T16:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.439+0000 7fdcfd39c700 1 -- 172.21.15.62:0/1689460607 learned_addr learned my addr 172.21.15.62:0/1689460607 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35416dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd354076e0 0x55cd3540a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdcfd39c700 1 -- 172.21.15.62:0/1689460607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd354076e0 msgr2=0x55cd3540a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd354076e0 0x55cd3540a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdcfd39c700 1 -- 172.21.15.62:0/1689460607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 msgr2=0x55cd35416dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35416dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdcfd39c700 1 -- 172.21.15.62:0/1689460607 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd351ac1f0 con 0x55cd352626b0 2023-07-14T16:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540d9e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009c40 tx=0x7fdcf0009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1b7961630ead47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdce37fe700 1 -- 172.21.15.62:0/1689460607 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0015c30 con 0x55cd352626b0 2023-07-14T16:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdce37fe700 1 -- 172.21.15.62:0/1689460607 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcf001f360 con 0x55cd352626b0 2023-07-14T16:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.440+0000 7fdce37fe700 1 -- 172.21.15.62:0/1689460607 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0021b10 con 0x55cd352626b0 2023-07-14T16:31:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1689460607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 msgr2=0x55cd3540d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540d9e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009c40 tx=0x7fdcf0009130 comp rx=0 tx=0).stop 2023-07-14T16:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1689460607 shutdown_connections 2023-07-14T16:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd354076e0 0x55cd3540a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd352626b0 0x55cd3540d9e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/1689460607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35416dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1689460607 >> 172.21.15.62:0/1689460607 conn(0x55cd353fbbe0 msgr2=0x55cd35422320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1689460607 shutdown_connections 2023-07-14T16:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.441+0000 7fdd127aeb80 1 -- 172.21.15.62:0/1689460607 wait complete. 2023-07-14T16:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 1 -- start start 2023-07-14T16:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 1 Processor -- start 2023-07-14T16:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd352626b0 0x55cd354120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35412620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35415640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd35415b80 con 0x55cd354076e0 2023-07-14T16:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd35415cf0 con 0x55cd35406840 2023-07-14T16:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd35415eb0 con 0x55cd352626b0 2023-07-14T16:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.442+0000 7fdd127aeb80 10 client.0 _ll_register_callbacks cb 0x7fdcec0d2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcfd39c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd352626b0 0x55cd354120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcfd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35415640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcfd39c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35415640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37088/0 (socket says 172.21.15.62:37088) 2023-07-14T16:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcf7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35412620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcfd39c700 1 -- 172.21.15.62:0/961292956 learned_addr learned my addr 172.21.15.62:0/961292956 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcf7fff700 1 -- 172.21.15.62:0/961292956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd352626b0 msgr2=0x55cd354120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd352626b0 0x55cd354120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcf7fff700 1 -- 172.21.15.62:0/961292956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 msgr2=0x55cd35415640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35415640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.443+0000 7fdcf7fff700 1 -- 172.21.15.62:0/961292956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd3516ec00 con 0x55cd35406840 2023-07-14T16:31:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.444+0000 7fdcf7fff700 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35412620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005910 tx=0x7fdce40055c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddddd155a440f8ee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.444+0000 7fdcd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:31:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.444+0000 7fdcd7fff700 1 -- 172.21.15.62:0/961292956 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce401d040 con 0x55cd35406840 2023-07-14T16:31:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.444+0000 7fdcd7fff700 1 -- 172.21.15.62:0/961292956 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdce40056b0 con 0x55cd35406840 2023-07-14T16:31:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.444+0000 7fdd127aeb80 1 -- 172.21.15.62:0/961292956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcf000f040 con 0x55cd35406840 2023-07-14T16:31:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.444+0000 7fdcd7fff700 1 -- 172.21.15.62:0/961292956 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdce40290c0 con 0x55cd35406840 2023-07-14T16:31:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.444+0000 7fdd127aeb80 10 client.29558 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.444+0000 7fdd127aeb80 1 -- 172.21.15.62:0/961292956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55cd35184250 con 0x55cd35406840 2023-07-14T16:31:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdcd7fff700 1 -- 172.21.15.62:0/961292956 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcf000f040 con 0x55cd35406840 2023-07-14T16:31:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdcd7fff700 1 -- 172.21.15.62:0/961292956 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdce403b070 con 0x55cd35406840 2023-07-14T16:31:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdd127aeb80 10 client.29558 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdd127aeb80 10 client.29558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdd127aeb80 20 client.29558 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd3546f940 0x55cd35471d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdd127aeb80 4 client.29558 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:31:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdd127aeb80 1 -- 172.21.15.62:0/961292956 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cd35312ff0 con 0x55cd3546f940 2023-07-14T16:31:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.445+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd3546f940 0x55cd35471d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.446+0000 7fdcd7fff700 10 client.29558 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.446+0000 7fdcfd39c700 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd3546f940 0x55cd35471d50 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b6903da89b465 server_cookie=fb52ee3315e10952 in_seq=0 out_seq=0 2023-07-14T16:31:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.455+0000 7fdcd7fff700 1 -- 172.21.15.62:0/961292956 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cd35312ff0 con 0x55cd3546f940 2023-07-14T16:31:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.455+0000 7fdcd7fff700 10 client.29558 handle_command_reply: tid=0 2023-07-14T16:31:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.455+0000 7fdd127aeb80 1 client.29558 shutdown 2023-07-14T16:31:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 -- 172.21.15.62:0/961292956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 msgr2=0x55cd35412620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35412620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005910 tx=0x7fdce40055c0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 -- 172.21.15.62:0/961292956 shutdown_connections 2023-07-14T16:31:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd354076e0 0x55cd35415640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd352626b0 0x55cd354120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55cd3546f940 0x55cd35471d50 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 --2- 172.21.15.62:0/961292956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd35406840 0x55cd35412620 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdcd7fff700 0 client.29558 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 -- 172.21.15.62:0/961292956 >> 172.21.15.62:0/961292956 conn(0x55cd353fbbe0 msgr2=0x55cd3540e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 -- 172.21.15.62:0/961292956 shutdown_connections 2023-07-14T16:31:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 1 -- 172.21.15.62:0/961292956 wait complete. 2023-07-14T16:31:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.456+0000 7fdd127aeb80 20 client.29558 trim_cache size 0 max 16384 2023-07-14T16:31:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:31:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:31:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:31:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:31:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 -- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdce80a1250 msgr2=0x7fdce80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdce80a1250 0x7fdce80a3700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdcec005950 tx=0x7fdcec0d2530 comp rx=0 tx=0).stop 2023-07-14T16:31:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 -- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8068620 msgr2=0x7fdcf8197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8068620 0x7fdcf8197ae0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e970 tx=0x7fdcec00ec80 comp rx=0 tx=0).stop 2023-07-14T16:31:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 -- 172.21.15.62:0/1037664558 shutdown_connections 2023-07-14T16:31:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf81056e0 0x7fdcf8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf8068620 0x7fdcf8197ae0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fdce80a1250 0x7fdce80a3700 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 --2- 172.21.15.62:0/1037664558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf8102bf0 0x7fdcf8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 -- 172.21.15.62:0/1037664558 >> 172.21.15.62:0/1037664558 conn(0x7fdcf80fa000 msgr2=0x7fdcf80fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 -- 172.21.15.62:0/1037664558 shutdown_connections 2023-07-14T16:31:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:30.457+0000 7fdcfedff700 1 -- 172.21.15.62:0/1037664558 wait complete. 2023-07-14T16:31:31.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: pgmap v833: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:31:31.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:31:31.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:31:31.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:31:31.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:31:31.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:31:31.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:31:31.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:30 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: pgmap v833: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:31:31.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:31:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:31:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:31:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:31:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:31:31.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:31:31.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:30 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: pgmap v833: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:31:31.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:31:31.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:31:31.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:31:31.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:31:31.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:31:31.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000003c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps) see mds.b log and `damage ls` output for details 2023-07-14T16:31:31.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:31.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:30 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:31:33.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:32 smithi062 ceph-mon[95543]: pgmap v834: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:31:33.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:32 smithi081 ceph-mon[127795]: pgmap v834: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:31:33.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:32 smithi164 ceph-mon[97427]: pgmap v834: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 22 op/s 2023-07-14T16:31:35.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:34 smithi062 ceph-mon[95543]: pgmap v835: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:31:35.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:34 smithi081 ceph-mon[127795]: pgmap v835: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:31:35.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:34 smithi164 ceph-mon[97427]: pgmap v835: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-14T16:31:37.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:36 smithi062 ceph-mon[95543]: pgmap v836: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:31:37.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:36 smithi081 ceph-mon[127795]: pgmap v836: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:31:37.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:36 smithi164 ceph-mon[97427]: pgmap v836: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 37 op/s 2023-07-14T16:31:38.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:37 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:31:38.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:37 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:31:38.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:37 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:31:39.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:38 smithi062 ceph-mon[95543]: pgmap v837: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:31:39.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:31:39.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:31:39.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:38 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:31:39.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:38 smithi081 ceph-mon[127795]: pgmap v837: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:31:39.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:31:39.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:31:39.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:38 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:31:39.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:38 smithi164 ceph-mon[97427]: pgmap v837: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 36 op/s 2023-07-14T16:31:39.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:31:39.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:31:39.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:38 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:31:41.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:40 smithi062 ceph-mon[95543]: pgmap v838: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:31:41.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:40 smithi081 ceph-mon[127795]: pgmap v838: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:31:41.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:40 smithi164 ceph-mon[97427]: pgmap v838: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 39 op/s 2023-07-14T16:31:43.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:42 smithi062 ceph-mon[95543]: pgmap v839: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:31:43.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:42 smithi081 ceph-mon[127795]: pgmap v839: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:31:43.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:42 smithi164 ceph-mon[97427]: pgmap v839: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 36 op/s 2023-07-14T16:31:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 -- 172.21.15.62:0/43382670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f01079c0 msgr2=0x7fe7f0109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/43382670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f01079c0 0x7fe7f0109e20 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800b0a0 tx=0x7fe7e800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 -- 172.21.15.62:0/43382670 shutdown_connections 2023-07-14T16:31:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/43382670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f01079c0 0x7fe7f0109e20 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/43382670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f00744d0 0x7fe7f0072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/43382670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f0072ef0 0x7fe7f0073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 -- 172.21.15.62:0/43382670 >> 172.21.15.62:0/43382670 conn(0x7fe7f00fb130 msgr2=0x7fe7f00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 -- 172.21.15.62:0/43382670 shutdown_connections 2023-07-14T16:31:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.643+0000 7fe7f65ff700 1 -- 172.21.15.62:0/43382670 wait complete. 2023-07-14T16:31:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.644+0000 7fe7f65ff700 1 Processor -- start 2023-07-14T16:31:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.644+0000 7fe7f65ff700 1 -- start start 2023-07-14T16:31:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7f65ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f0072ef0 0x7fe7f0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7f65ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f00744d0 0x7fe7f01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7f65ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f01079c0 0x7fe7f0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7f65ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0199550 con 0x7fe7f0072ef0 2023-07-14T16:31:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7f65ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f01996c0 con 0x7fe7f01079c0 2023-07-14T16:31:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7f65ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0199880 con 0x7fe7f00744d0 2023-07-14T16:31:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f0072ef0 0x7fe7f0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f0072ef0 0x7fe7f0197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58272/0 (socket says 172.21.15.62:58272) 2023-07-14T16:31:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7effff700 1 -- 172.21.15.62:0/1738170841 learned_addr learned my addr 172.21.15.62:0/1738170841 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f01079c0 0x7fe7f0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f00744d0 0x7fe7f01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7effff700 1 -- 172.21.15.62:0/1738170841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f00744d0 msgr2=0x7fe7f01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.645+0000 7fe7effff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f00744d0 0x7fe7f01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7effff700 1 -- 172.21.15.62:0/1738170841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f01079c0 msgr2=0x7fe7f0198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7effff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f01079c0 0x7fe7f0198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7effff700 1 -- 172.21.15.62:0/1738170841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8009d00 con 0x7fe7f0072ef0 2023-07-14T16:31:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f00744d0 0x7fe7f01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7effff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f0072ef0 0x7fe7f0197b80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e9a0 tx=0x7fe7e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7ed7fa700 1 -- 172.21.15.62:0/1738170841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0009790 con 0x7fe7f0072ef0 2023-07-14T16:31:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7ed7fa700 1 -- 172.21.15.62:0/1738170841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7e000cbd0 con 0x7fe7f0072ef0 2023-07-14T16:31:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7f65ff700 1 -- 172.21.15.62:0/1738170841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f0199b60 con 0x7fe7f0072ef0 2023-07-14T16:31:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7ed7fa700 1 -- 172.21.15.62:0/1738170841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0018680 con 0x7fe7f0072ef0 2023-07-14T16:31:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.646+0000 7fe7f65ff700 1 -- 172.21.15.62:0/1738170841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f01a0320 con 0x7fe7f0072ef0 2023-07-14T16:31:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.650+0000 7fe7ed7fa700 1 -- 172.21.15.62:0/1738170841 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe7e000cd40 con 0x7fe7f0072ef0 2023-07-14T16:31:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.651+0000 7fe7ed7fa700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7d80a1150 0x7fe7d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.651+0000 7fe7ed7fa700 1 -- 172.21.15.62:0/1738170841 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe7e0014070 con 0x7fe7f0072ef0 2023-07-14T16:31:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.652+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7d80a1150 0x7fe7d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.652+0000 7fe809fc0b80 1 Processor -- start 2023-07-14T16:31:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.652+0000 7fe809fc0b80 1 -- start start 2023-07-14T16:31:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.652+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7d80a1150 0x7fe7d80a3600 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009cb0 tx=0x7fe7e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 0x5567892e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892e06e0 0x5567892e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567891e4c50 con 0x55678913b6b0 2023-07-14T16:31:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567892379e0 con 0x5567892e06e0 2023-07-14T16:31:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556789230e70 con 0x5567892df840 2023-07-14T16:31:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892df300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58288/0 (socket says 172.21.15.62:58288) 2023-07-14T16:31:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7effff700 1 -- 172.21.15.62:0/2614367755 learned_addr learned my addr 172.21.15.62:0/2614367755 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 0x5567892e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7effff700 1 -- 172.21.15.62:0/2614367755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 msgr2=0x5567892e01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7effff700 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 0x5567892e01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892e06e0 0x5567892e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7effff700 1 -- 172.21.15.62:0/2614367755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892e06e0 msgr2=0x5567892e0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7effff700 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892e06e0 0x5567892e0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7effff700 1 -- 172.21.15.62:0/2614367755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55678905ed80 con 0x55678913b6b0 2023-07-14T16:31:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.653+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892e06e0 0x5567892e0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:31:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.654+0000 7fe7effff700 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892df300 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0099f50 tx=0x7fe7e0099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c174a3589b73f9e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.654+0000 7fe7d67fc700 1 -- 172.21.15.62:0/2614367755 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e009dff0 con 0x55678913b6b0 2023-07-14T16:31:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.654+0000 7fe7d67fc700 1 -- 172.21.15.62:0/2614367755 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7e00a2410 con 0x55678913b6b0 2023-07-14T16:31:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.654+0000 7fe7d67fc700 1 -- 172.21.15.62:0/2614367755 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00a21e0 con 0x55678913b6b0 2023-07-14T16:31:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2614367755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 msgr2=0x5567892df300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892df300 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0099f50 tx=0x7fe7e0099bd0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2614367755 shutdown_connections 2023-07-14T16:31:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892e06e0 0x5567892e0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892df300 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2614367755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 0x5567892e01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2614367755 >> 172.21.15.62:0/2614367755 conn(0x5567892d4be0 msgr2=0x5567892d9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2614367755 shutdown_connections 2023-07-14T16:31:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2614367755 wait complete. 2023-07-14T16:31:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.655+0000 7fe809fc0b80 1 -- start start 2023-07-14T16:31:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.656+0000 7fe809fc0b80 1 Processor -- start 2023-07-14T16:31:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.656+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.656+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.656+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.656+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567891d0a60 con 0x55678913b6b0 2023-07-14T16:31:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.656+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567891d0760 con 0x5567892df840 2023-07-14T16:31:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.656+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567890b48c0 con 0x5567892e06e0 2023-07-14T16:31:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.656+0000 7fe809fc0b80 10 client.0 _ll_register_callbacks cb 0x5567891ce700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58294/0 (socket says 172.21.15.62:58294) 2023-07-14T16:31:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 -- 172.21.15.62:0/1114545867 learned_addr learned my addr 172.21.15.62:0/1114545867 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 -- 172.21.15.62:0/1114545867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 msgr2=0x5567892efdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892efdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 -- 172.21.15.62:0/1114545867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 msgr2=0x5567892e8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892e8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 -- 172.21.15.62:0/1114545867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f01a0320 con 0x55678913b6b0 2023-07-14T16:31:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7effff700 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e8850 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0023040 tx=0x7fe7e0094f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e9faa4d2e64ec1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.657+0000 7fe7cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.658+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/1114545867 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e009dff0 con 0x55678913b6b0 2023-07-14T16:31:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.658+0000 7fe809fc0b80 1 -- 172.21.15.62:0/1114545867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55678918a490 con 0x55678913b6b0 2023-07-14T16:31:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.658+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/1114545867 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe7e00a1f80 con 0x55678913b6b0 2023-07-14T16:31:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.658+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/1114545867 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e00a2140 con 0x55678913b6b0 2023-07-14T16:31:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.658+0000 7fe809fc0b80 10 client.20043 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.658+0000 7fe809fc0b80 1 -- 172.21.15.62:0/1114545867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556789047c00 con 0x55678913b6b0 2023-07-14T16:31:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.658+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/1114545867 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7e0063020 con 0x55678913b6b0 2023-07-14T16:31:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.658+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/1114545867 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe7e0010430 con 0x55678913b6b0 2023-07-14T16:31:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.659+0000 7fe809fc0b80 10 client.20043 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.659+0000 7fe809fc0b80 10 client.20043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.659+0000 7fe809fc0b80 20 client.20043 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.659+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5567892f42d0 0x5567892f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.659+0000 7fe809fc0b80 4 client.20043 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.659+0000 7fe809fc0b80 1 -- 172.21.15.62:0/1114545867 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556789229e50 con 0x5567892f42d0 2023-07-14T16:31:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.659+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5567892f42d0 0x5567892f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.660+0000 7fe7cf7fe700 10 client.20043 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.660+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5567892f42d0 0x5567892f66e0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190f0ca6f1d378a1 server_cookie=eb366488d32eee02 in_seq=0 out_seq=0 2023-07-14T16:31:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/1114545867 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556789229e50 con 0x5567892f42d0 2023-07-14T16:31:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe7cf7fe700 10 client.20043 handle_command_reply: tid=0 2023-07-14T16:31:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 client.20043 shutdown 2023-07-14T16:31:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 -- 172.21.15.62:0/1114545867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 msgr2=0x5567892e8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e8850 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0023040 tx=0x7fe7e0094f90 comp rx=0 tx=0).stop 2023-07-14T16:31:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 -- 172.21.15.62:0/1114545867 shutdown_connections 2023-07-14T16:31:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e8850 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5567892f42d0 0x5567892f66e0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/1114545867 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892e8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 -- 172.21.15.62:0/1114545867 >> 172.21.15.62:0/1114545867 conn(0x5567892d4be0 msgr2=0x5567892d94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe7cf7fe700 0 client.20043 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 -- 172.21.15.62:0/1114545867 shutdown_connections 2023-07-14T16:31:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 1 -- 172.21.15.62:0/1114545867 wait complete. 2023-07-14T16:31:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.661+0000 7fe809fc0b80 20 client.20043 trim_cache size 0 max 16384 2023-07-14T16:31:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.684+0000 7fe809fc0b80 1 Processor -- start 2023-07-14T16:31:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.684+0000 7fe809fc0b80 1 -- start start 2023-07-14T16:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567892dbb90 con 0x55678913b6b0 2023-07-14T16:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556789229e50 con 0x5567892df840 2023-07-14T16:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567892a5060 con 0x5567892e06e0 2023-07-14T16:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58298/0 (socket says 172.21.15.62:58298) 2023-07-14T16:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7effff700 1 -- 172.21.15.62:0/242254334 learned_addr learned my addr 172.21.15.62:0/242254334 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7effff700 1 -- 172.21.15.62:0/242254334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 msgr2=0x5567892e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7effff700 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7effff700 1 -- 172.21.15.62:0/242254334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 msgr2=0x5567892efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7effff700 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7effff700 1 -- 172.21.15.62:0/242254334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567890851f0 con 0x55678913b6b0 2023-07-14T16:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.685+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.686+0000 7fe7effff700 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e35d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e970 tx=0x7fe7e0099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd08353a57ab70a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.686+0000 7fe7d6ffd700 1 -- 172.21.15.62:0/242254334 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0083810 con 0x55678913b6b0 2023-07-14T16:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.686+0000 7fe7d6ffd700 1 -- 172.21.15.62:0/242254334 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7e0063070 con 0x55678913b6b0 2023-07-14T16:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.686+0000 7fe7d6ffd700 1 -- 172.21.15.62:0/242254334 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00994b0 con 0x55678913b6b0 2023-07-14T16:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 -- 172.21.15.62:0/242254334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 msgr2=0x5567892e35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e35d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e970 tx=0x7fe7e0099bd0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 -- 172.21.15.62:0/242254334 shutdown_connections 2023-07-14T16:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892e06e0 0x5567892e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55678913b6b0 0x5567892e35d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/242254334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5567892df840 0x5567892efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 -- 172.21.15.62:0/242254334 >> 172.21.15.62:0/242254334 conn(0x5567892d4be0 msgr2=0x5567892e74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 -- 172.21.15.62:0/242254334 shutdown_connections 2023-07-14T16:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 -- 172.21.15.62:0/242254334 wait complete. 2023-07-14T16:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.687+0000 7fe809fc0b80 1 -- start start 2023-07-14T16:31:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe809fc0b80 1 Processor -- start 2023-07-14T16:31:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55678913b6b0 0x5567892ebc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 0x5567892ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe809fc0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567892e06e0 0x5567892ef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567892ef6d0 con 0x5567892e06e0 2023-07-14T16:31:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567892ef840 con 0x55678913b6b0 2023-07-14T16:31:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe809fc0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567892efa00 con 0x5567892df840 2023-07-14T16:31:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe809fc0b80 10 client.0 _ll_register_callbacks cb 0x7fe7e0099e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe7f4b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567892e06e0 0x5567892ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe7f4b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567892e06e0 0x5567892ef190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58302/0 (socket says 172.21.15.62:58302) 2023-07-14T16:31:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.688+0000 7fe7f4b9c700 1 -- 172.21.15.62:0/2828636660 learned_addr learned my addr 172.21.15.62:0/2828636660 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7ef7fe700 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 0x5567892ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7effff700 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55678913b6b0 0x5567892ebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7f4b9c700 1 -- 172.21.15.62:0/2828636660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 msgr2=0x5567892ec170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 0x5567892ec170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7f4b9c700 1 -- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55678913b6b0 msgr2=0x5567892ebc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55678913b6b0 0x5567892ebc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7f4b9c700 1 -- 172.21.15.62:0/2828636660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556789047c00 con 0x5567892e06e0 2023-07-14T16:31:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7effff700 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55678913b6b0 0x5567892ebc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7f4b9c700 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567892e06e0 0x5567892ef190 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80079c0 tx=0x7fe7e8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e654e0a0037fd4f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.689+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/2828636660 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e800e040 con 0x5567892e06e0 2023-07-14T16:31:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2828636660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7e00a2410 con 0x5567892e06e0 2023-07-14T16:31:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/2828636660 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe7e8023070 con 0x5567892e06e0 2023-07-14T16:31:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/2828636660 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e8003a00 con 0x5567892e06e0 2023-07-14T16:31:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe809fc0b80 10 client.20049 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2828636660 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55678905d250 con 0x5567892e06e0 2023-07-14T16:31:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/2828636660 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7e00a2410 con 0x5567892e06e0 2023-07-14T16:31:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/2828636660 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe7e8030070 con 0x5567892e06e0 2023-07-14T16:31:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe809fc0b80 10 client.20049 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe809fc0b80 10 client.20049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe809fc0b80 20 client.20049 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.690+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5567893492b0 0x55678934b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.691+0000 7fe809fc0b80 4 client.20049 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:31:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.691+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2828636660 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5567891ebff0 con 0x5567893492b0 2023-07-14T16:31:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.691+0000 7fe7effff700 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5567893492b0 0x55678934b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.692+0000 7fe7cf7fe700 10 client.20049 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.692+0000 7fe7effff700 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5567893492b0 0x55678934b6c0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f763ce6998fd77 server_cookie=7042cb97217eda0b in_seq=0 out_seq=0 2023-07-14T16:31:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.701+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/2828636660 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5567891ebff0 con 0x5567893492b0 2023-07-14T16:31:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.701+0000 7fe7cf7fe700 10 client.20049 handle_command_reply: tid=0 2023-07-14T16:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.701+0000 7fe809fc0b80 1 client.20049 shutdown 2023-07-14T16:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2828636660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567892e06e0 msgr2=0x5567892ef190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567892e06e0 0x5567892ef190 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80079c0 tx=0x7fe7e8007dc0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2828636660 shutdown_connections 2023-07-14T16:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5567892e06e0 0x5567892ef190 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5567893492b0 0x55678934b6c0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55678913b6b0 0x5567892ebc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 --2- 172.21.15.62:0/2828636660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5567892df840 0x5567892ec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe7cf7fe700 0 client.20049 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2828636660 >> 172.21.15.62:0/2828636660 conn(0x5567892d4be0 msgr2=0x5567892e35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2828636660 shutdown_connections 2023-07-14T16:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 1 -- 172.21.15.62:0/2828636660 wait complete. 2023-07-14T16:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.702+0000 7fe809fc0b80 20 client.20049 trim_cache size 0 max 16384 2023-07-14T16:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 -- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7d80a1150 msgr2=0x7fe7d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7d80a1150 0x7fe7d80a3600 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009cb0 tx=0x7fe7e4009260 comp rx=0 tx=0).stop 2023-07-14T16:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 -- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f0072ef0 msgr2=0x7fe7f0197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f0072ef0 0x7fe7f0197b80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e9a0 tx=0x7fe7e000ecb0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 -- 172.21.15.62:0/1738170841 shutdown_connections 2023-07-14T16:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7d80a1150 0x7fe7d80a3600 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7f01079c0 0x7fe7f0198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f0072ef0 0x7fe7f0197b80 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 --2- 172.21.15.62:0/1738170841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7f00744d0 0x7fe7f01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 -- 172.21.15.62:0/1738170841 >> 172.21.15.62:0/1738170841 conn(0x7fe7f00fb130 msgr2=0x7fe7f00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 -- 172.21.15.62:0/1738170841 shutdown_connections 2023-07-14T16:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:43.703+0000 7fe7f65ff700 1 -- 172.21.15.62:0/1738170841 wait complete. 2023-07-14T16:31:44.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:43 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:31:44.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:43 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:31:44.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:43 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:31:44.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:43 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:31:44.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:43 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:31:44.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:43 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:31:44.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:43 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:31:44.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:43 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:31:44.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:43 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:31:45.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:44 smithi062 ceph-mon[95543]: pgmap v840: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 30 op/s 2023-07-14T16:31:45.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:44 smithi081 ceph-mon[127795]: pgmap v840: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 30 op/s 2023-07-14T16:31:45.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:44 smithi164 ceph-mon[97427]: pgmap v840: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 30 op/s 2023-07-14T16:31:47.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:46 smithi062 ceph-mon[95543]: pgmap v841: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:31:47.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:46 smithi081 ceph-mon[127795]: pgmap v841: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:31:47.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:46 smithi164 ceph-mon[97427]: pgmap v841: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 19 op/s 2023-07-14T16:31:49.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:48 smithi062 ceph-mon[95543]: pgmap v842: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:31:49.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:48 smithi081 ceph-mon[127795]: pgmap v842: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:31:49.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:48 smithi164 ceph-mon[97427]: pgmap v842: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-14T16:31:51.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:50 smithi062 ceph-mon[95543]: pgmap v843: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:31:51.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:50 smithi081 ceph-mon[127795]: pgmap v843: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:31:51.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:50 smithi164 ceph-mon[97427]: pgmap v843: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 17 op/s 2023-07-14T16:31:53.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:52 smithi062 ceph-mon[95543]: pgmap v844: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 14 op/s 2023-07-14T16:31:53.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:52 smithi081 ceph-mon[127795]: pgmap v844: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 14 op/s 2023-07-14T16:31:53.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:52 smithi164 ceph-mon[97427]: pgmap v844: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 14 op/s 2023-07-14T16:31:55.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:54 smithi062 ceph-mon[95543]: pgmap v845: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:55.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:54 smithi081 ceph-mon[127795]: pgmap v845: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:55.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:54 smithi164 ceph-mon[97427]: pgmap v845: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 16 op/s 2023-07-14T16:31:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 -- 172.21.15.62:0/663234359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 msgr2=0x7fc34c105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 --2- 172.21.15.62:0/663234359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 0x7fc34c105160 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009990 tx=0x7fc33c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 -- 172.21.15.62:0/663234359 shutdown_connections 2023-07-14T16:31:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 --2- 172.21.15.62:0/663234359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc34c1056a0 0x7fc34c107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 --2- 172.21.15.62:0/663234359 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c100460 0x7fc34c102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 --2- 172.21.15.62:0/663234359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 0x7fc34c105160 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 -- 172.21.15.62:0/663234359 >> 172.21.15.62:0/663234359 conn(0x7fc34c0f9e40 msgr2=0x7fc34c0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 -- 172.21.15.62:0/663234359 shutdown_connections 2023-07-14T16:31:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.921+0000 7fc35259d700 1 -- 172.21.15.62:0/663234359 wait complete. 2023-07-14T16:31:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.922+0000 7fc35259d700 1 Processor -- start 2023-07-14T16:31:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.922+0000 7fc35259d700 1 -- start start 2023-07-14T16:31:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc35259d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c100460 0x7fc34c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc35259d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 0x7fc34c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc35259d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc34c1056a0 0x7fc34c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc35259d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19d1a0 con 0x7fc34c102d80 2023-07-14T16:31:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc35259d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19d2e0 con 0x7fc34c1056a0 2023-07-14T16:31:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc35259d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c19d470 con 0x7fc34c100460 2023-07-14T16:31:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc350d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 0x7fc34c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc350d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 0x7fc34c195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58744/0 (socket says 172.21.15.62:58744) 2023-07-14T16:31:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc350d9a700 1 -- 172.21.15.62:0/4224010394 learned_addr learned my addr 172.21.15.62:0/4224010394 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc35159b700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c100460 0x7fc34c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.923+0000 7fc351d9c700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc34c1056a0 0x7fc34c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc350d9a700 1 -- 172.21.15.62:0/4224010394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c100460 msgr2=0x7fc34c195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc350d9a700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c100460 0x7fc34c195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc350d9a700 1 -- 172.21.15.62:0/4224010394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc34c1056a0 msgr2=0x7fc34c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc350d9a700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc34c1056a0 0x7fc34c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc350d9a700 1 -- 172.21.15.62:0/4224010394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c0095f0 con 0x7fc34c102d80 2023-07-14T16:31:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc351d9c700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc34c1056a0 0x7fc34c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc350d9a700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 0x7fc34c195b60 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0098e0 tx=0x7fc33c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc3427fc700 1 -- 172.21.15.62:0/4224010394 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c01d070 con 0x7fc34c102d80 2023-07-14T16:31:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc35259d700 1 -- 172.21.15.62:0/4224010394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c19d6f0 con 0x7fc34c102d80 2023-07-14T16:31:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc3427fc700 1 -- 172.21.15.62:0/4224010394 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc33c00be30 con 0x7fc34c102d80 2023-07-14T16:31:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc3427fc700 1 -- 172.21.15.62:0/4224010394 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c022e00 con 0x7fc34c102d80 2023-07-14T16:31:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.924+0000 7fc35259d700 1 -- 172.21.15.62:0/4224010394 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c19db20 con 0x7fc34c102d80 2023-07-14T16:31:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.929+0000 7fc3427fc700 1 -- 172.21.15.62:0/4224010394 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc33c00b9a0 con 0x7fc34c102d80 2023-07-14T16:31:56.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.929+0000 7fc3671e3b80 1 Processor -- start 2023-07-14T16:31:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.929+0000 7fc3671e3b80 1 -- start start 2023-07-14T16:31:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be238550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be2392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be2397e0 0x5653be239c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be13dc50 con 0x5653be2397e0 2023-07-14T16:31:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be1909e0 con 0x5653be238a90 2023-07-14T16:31:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be189e70 con 0x5653be0946b0 2023-07-14T16:31:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc351d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be238550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc351d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be238550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38748/0 (socket says 172.21.15.62:38748) 2023-07-14T16:31:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc350d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be2397e0 0x5653be239c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc351d9c700 1 -- 172.21.15.62:0/3483499243 learned_addr learned my addr 172.21.15.62:0/3483499243 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.930+0000 7fc35159b700 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be2392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc351d9c700 1 -- 172.21.15.62:0/3483499243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 msgr2=0x5653be2392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc351d9c700 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be2392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc351d9c700 1 -- 172.21.15.62:0/3483499243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be2397e0 msgr2=0x5653be239c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc351d9c700 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be2397e0 0x5653be239c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc351d9c700 1 -- 172.21.15.62:0/3483499243 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653bdfb7d80 con 0x5653be0946b0 2023-07-14T16:31:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc35159b700 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be2392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:31:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc3427fc700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3380a1150 0x7fc3380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc351d9c700 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be238550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc344009640 tx=0x7fc344008f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95ffe8294fc8bde7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc3427fc700 1 -- 172.21.15.62:0/4224010394 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc33c0cdef0 con 0x7fc34c102d80 2023-07-14T16:31:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc3377fe700 1 -- 172.21.15.62:0/3483499243 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344003c00 con 0x5653be0946b0 2023-07-14T16:31:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc35159b700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3380a1150 0x7fc3380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc3377fe700 1 -- 172.21.15.62:0/3483499243 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc344011040 con 0x5653be0946b0 2023-07-14T16:31:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.931+0000 7fc3377fe700 1 -- 172.21.15.62:0/3483499243 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344013760 con 0x5653be0946b0 2023-07-14T16:31:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 -- 172.21.15.62:0/3483499243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 msgr2=0x5653be238550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc35159b700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3380a1150 0x7fc3380a3600 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x5653be233cb0 tx=0x7fc348009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be238550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc344009640 tx=0x7fc344008f90 comp rx=0 tx=0).stop 2023-07-14T16:31:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 -- 172.21.15.62:0/3483499243 shutdown_connections 2023-07-14T16:31:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be2397e0 0x5653be239c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be238550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/3483499243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be2392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 -- 172.21.15.62:0/3483499243 >> 172.21.15.62:0/3483499243 conn(0x5653be22dd40 msgr2=0x5653be232e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 -- 172.21.15.62:0/3483499243 shutdown_connections 2023-07-14T16:31:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.932+0000 7fc3671e3b80 1 -- 172.21.15.62:0/3483499243 wait complete. 2023-07-14T16:31:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.933+0000 7fc3671e3b80 1 -- start start 2023-07-14T16:31:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.933+0000 7fc3671e3b80 1 Processor -- start 2023-07-14T16:31:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be0946b0 0x5653be241b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be242080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 0x5653be2490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be129a60 con 0x5653be238a90 2023-07-14T16:31:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be129760 con 0x5653be0946b0 2023-07-14T16:31:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be00d8c0 con 0x5653be2397e0 2023-07-14T16:31:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc3671e3b80 10 client.0 _ll_register_callbacks cb 0x5653be127700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc350d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be242080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc350d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be242080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58758/0 (socket says 172.21.15.62:58758) 2023-07-14T16:31:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc350d9a700 1 -- 172.21.15.62:0/2924784053 learned_addr learned my addr 172.21.15.62:0/2924784053 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc35159b700 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be0946b0 0x5653be241b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.934+0000 7fc351d9c700 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 0x5653be2490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.935+0000 7fc350d9a700 1 -- 172.21.15.62:0/2924784053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 msgr2=0x5653be2490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.935+0000 7fc350d9a700 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 0x5653be2490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.935+0000 7fc350d9a700 1 -- 172.21.15.62:0/2924784053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be0946b0 msgr2=0x5653be241b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.935+0000 7fc350d9a700 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be0946b0 0x5653be241b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.935+0000 7fc350d9a700 1 -- 172.21.15.62:0/2924784053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344008e00 con 0x5653be238a90 2023-07-14T16:31:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.935+0000 7fc350d9a700 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be242080 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc33c097840 tx=0x7fc33c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b75431d713fea08 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.935+0000 7fc334ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.935+0000 7fc334ff9700 1 -- 172.21.15.62:0/2924784053 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c097680 con 0x5653be238a90 2023-07-14T16:31:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc334ff9700 1 -- 172.21.15.62:0/2924784053 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc33c09b340 con 0x5653be238a90 2023-07-14T16:31:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2924784053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653be0e3490 con 0x5653be238a90 2023-07-14T16:31:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc334ff9700 1 -- 172.21.15.62:0/2924784053 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc33c09b110 con 0x5653be238a90 2023-07-14T16:31:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 10 client.20061 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2924784053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5653bdfa0c00 con 0x5653be238a90 2023-07-14T16:31:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc334ff9700 1 -- 172.21.15.62:0/2924784053 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5653be0e3490 con 0x5653be238a90 2023-07-14T16:31:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc334ff9700 1 -- 172.21.15.62:0/2924784053 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc33c0a3270 con 0x5653be238a90 2023-07-14T16:31:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 10 client.20061 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 10 client.20061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 20 client.20061 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5653be24d4e0 0x5653be24f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 4 client.20061 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:31:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.936+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2924784053 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653be182e50 con 0x5653be24d4e0 2023-07-14T16:31:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.937+0000 7fc35159b700 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5653be24d4e0 0x5653be24f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.938+0000 7fc35159b700 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5653be24d4e0 0x5653be24f8f0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf69a5c8647bba3 server_cookie=f8f766e1112cb18d in_seq=0 out_seq=0 2023-07-14T16:31:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.938+0000 7fc334ff9700 10 client.20061 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.938+0000 7fc334ff9700 1 -- 172.21.15.62:0/2924784053 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5653be182e50 con 0x5653be24d4e0 2023-07-14T16:31:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.938+0000 7fc334ff9700 10 client.20061 handle_command_reply: tid=0 2023-07-14T16:31:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 client.20061 shutdown 2023-07-14T16:31:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2924784053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 msgr2=0x5653be242080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be242080 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc33c097840 tx=0x7fc33c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2924784053 shutdown_connections 2023-07-14T16:31:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 0x5653be2490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5653be24d4e0 0x5653be24f8f0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be0946b0 0x5653be241b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2924784053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be242080 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2924784053 >> 172.21.15.62:0/2924784053 conn(0x5653be22dd40 msgr2=0x5653be232640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc334ff9700 0 client.20061 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2924784053 shutdown_connections 2023-07-14T16:31:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2924784053 wait complete. 2023-07-14T16:31:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.939+0000 7fc3671e3b80 20 client.20061 trim_cache size 0 max 16384 2023-07-14T16:31:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.957+0000 7fc3671e3b80 1 Processor -- start 2023-07-14T16:31:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.957+0000 7fc3671e3b80 1 -- start start 2023-07-14T16:31:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be0946b0 0x5653be23c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be249080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 0x5653be23cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be24a0c0 con 0x5653be0946b0 2023-07-14T16:31:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be182e50 con 0x5653be238a90 2023-07-14T16:31:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be0e3490 con 0x5653be2397e0 2023-07-14T16:31:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc35159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be0946b0 0x5653be23c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc35159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be0946b0 0x5653be23c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58772/0 (socket says 172.21.15.62:58772) 2023-07-14T16:31:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc35159b700 1 -- 172.21.15.62:0/824376693 learned_addr learned my addr 172.21.15.62:0/824376693 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc351d9c700 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 0x5653be23cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.958+0000 7fc350d9a700 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be249080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.959+0000 7fc35159b700 1 -- 172.21.15.62:0/824376693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 msgr2=0x5653be23cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.959+0000 7fc35159b700 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 0x5653be23cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.959+0000 7fc35159b700 1 -- 172.21.15.62:0/824376693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 msgr2=0x5653be249080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.959+0000 7fc35159b700 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be249080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.959+0000 7fc35159b700 1 -- 172.21.15.62:0/824376693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653bdfde1f0 con 0x5653be0946b0 2023-07-14T16:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.959+0000 7fc35159b700 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be0946b0 0x5653be23c690 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0683c0 tx=0x7fc34801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357c156c74d57106 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.959+0000 7fc337fff700 1 -- 172.21.15.62:0/824376693 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348022040 con 0x5653be0946b0 2023-07-14T16:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc337fff700 1 -- 172.21.15.62:0/824376693 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc348023040 con 0x5653be0946b0 2023-07-14T16:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc337fff700 1 -- 172.21.15.62:0/824376693 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348017070 con 0x5653be0946b0 2023-07-14T16:31:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc3671e3b80 1 -- 172.21.15.62:0/824376693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be0946b0 msgr2=0x5653be23c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be0946b0 0x5653be23c690 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0683c0 tx=0x7fc34801b000 comp rx=0 tx=0).stop 2023-07-14T16:31:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc3671e3b80 1 -- 172.21.15.62:0/824376693 shutdown_connections 2023-07-14T16:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be2397e0 0x5653be23cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be0946b0 0x5653be23c690 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/824376693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be238a90 0x5653be249080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.960+0000 7fc3671e3b80 1 -- 172.21.15.62:0/824376693 >> 172.21.15.62:0/824376693 conn(0x5653be22dd40 msgr2=0x5653be23fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.961+0000 7fc3671e3b80 1 -- 172.21.15.62:0/824376693 shutdown_connections 2023-07-14T16:31:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.961+0000 7fc3671e3b80 1 -- 172.21.15.62:0/824376693 wait complete. 2023-07-14T16:31:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.961+0000 7fc3671e3b80 1 -- start start 2023-07-14T16:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.961+0000 7fc3671e3b80 1 Processor -- start 2023-07-14T16:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be244a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be244f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc3671e3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be2397e0 0x5653be247f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be2484d0 con 0x5653be238a90 2023-07-14T16:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be248640 con 0x5653be2397e0 2023-07-14T16:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc3671e3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be248800 con 0x5653be0946b0 2023-07-14T16:31:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc3671e3b80 10 client.0 _ll_register_callbacks cb 0x7fc33c022470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc350d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be244f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc350d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be244f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58788/0 (socket says 172.21.15.62:58788) 2023-07-14T16:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc350d9a700 1 -- 172.21.15.62:0/2460729534 learned_addr learned my addr 172.21.15.62:0/2460729534 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc351d9c700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be2397e0 0x5653be247f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.962+0000 7fc35159b700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be244a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc350d9a700 1 -- 172.21.15.62:0/2460729534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 msgr2=0x5653be244a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc350d9a700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be244a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc350d9a700 1 -- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be2397e0 msgr2=0x5653be247f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc350d9a700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be2397e0 0x5653be247f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc350d9a700 1 -- 172.21.15.62:0/2460729534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc348012d20 con 0x5653be238a90 2023-07-14T16:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc35159b700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be244a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc351d9c700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be2397e0 0x5653be247f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc350d9a700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be244f70 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc33c097d60 tx=0x7fc33c0227a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eefc8c4e50d9bcd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc334ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:31:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc334ff9700 1 -- 172.21.15.62:0/2460729534 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c084d90 con 0x5653be238a90 2023-07-14T16:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.963+0000 7fc334ff9700 1 -- 172.21.15.62:0/2460729534 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc33c09b340 con 0x5653be238a90 2023-07-14T16:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2460729534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc344011040 con 0x5653be238a90 2023-07-14T16:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc334ff9700 1 -- 172.21.15.62:0/2460729534 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc33c0d13c0 con 0x5653be238a90 2023-07-14T16:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 10 client.20067 fetch_fsmap learned FSMap version 31 2023-07-14T16:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2460729534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5653bdfb6250 con 0x5653be238a90 2023-07-14T16:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc334ff9700 1 -- 172.21.15.62:0/2460729534 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc33c0b7020 con 0x5653be238a90 2023-07-14T16:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc334ff9700 1 -- 172.21.15.62:0/2460729534 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc33c09a2a0 con 0x5653be238a90 2023-07-14T16:31:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 10 client.20067 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 10 client.20067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 20 client.20067 populate_metadata read hostname 'smithi062' 2023-07-14T16:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5653be2a1a00 0x5653be2a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 4 client.20067 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.964+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2460729534 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5653be144ff0 con 0x5653be2a1a00 2023-07-14T16:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.965+0000 7fc35159b700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5653be2a1a00 0x5653be2a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:31:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.966+0000 7fc334ff9700 10 client.20067 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.966+0000 7fc35159b700 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5653be2a1a00 0x5653be2a3de0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194c0ca23a732a13 server_cookie=b8f3a8d6a5d79e0e in_seq=0 out_seq=0 2023-07-14T16:31:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.975+0000 7fc334ff9700 1 -- 172.21.15.62:0/2460729534 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5653be144ff0 con 0x5653be2a1a00 2023-07-14T16:31:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.975+0000 7fc334ff9700 10 client.20067 handle_command_reply: tid=0 2023-07-14T16:31:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.975+0000 7fc3671e3b80 1 client.20067 shutdown 2023-07-14T16:31:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2460729534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 msgr2=0x5653be244f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be244f70 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc33c097d60 tx=0x7fc33c0227a0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2460729534 shutdown_connections 2023-07-14T16:31:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5653be2a1a00 0x5653be2a3de0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5653be2397e0 0x5653be247f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5653be0946b0 0x5653be244a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 --2- 172.21.15.62:0/2460729534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653be238a90 0x5653be244f70 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc334ff9700 0 client.20067 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:31:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2460729534 >> 172.21.15.62:0/2460729534 conn(0x5653be22dd40 msgr2=0x5653be23c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2460729534 shutdown_connections 2023-07-14T16:31:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 1 -- 172.21.15.62:0/2460729534 wait complete. 2023-07-14T16:31:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.976+0000 7fc3671e3b80 20 client.20067 trim_cache size 0 max 16384 2023-07-14T16:31:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:31:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:31:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:31:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:31:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 -- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3380a1150 msgr2=0x7fc3380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3380a1150 0x7fc3380a3600 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x5653be233cb0 tx=0x7fc348009260 comp rx=0 tx=0).stop 2023-07-14T16:31:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 -- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 msgr2=0x7fc34c195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:31:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 0x7fc34c195b60 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0098e0 tx=0x7fc33c004730 comp rx=0 tx=0).stop 2023-07-14T16:31:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 -- 172.21.15.62:0/4224010394 shutdown_connections 2023-07-14T16:31:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc3380a1150 0x7fc3380a3600 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc34c1056a0 0x7fc34c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc34c100460 0x7fc34c195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 --2- 172.21.15.62:0/4224010394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc34c102d80 0x7fc34c195b60 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:31:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 -- 172.21.15.62:0/4224010394 >> 172.21.15.62:0/4224010394 conn(0x7fc34c0f9e40 msgr2=0x7fc34c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:31:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 -- 172.21.15.62:0/4224010394 shutdown_connections 2023-07-14T16:31:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:31:56.977+0000 7fc35259d700 1 -- 172.21.15.62:0/4224010394 wait complete. 2023-07-14T16:31:57.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:56 smithi062 ceph-mon[95543]: pgmap v846: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 14 op/s 2023-07-14T16:31:57.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:56 smithi081 ceph-mon[127795]: pgmap v846: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 14 op/s 2023-07-14T16:31:57.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:56 smithi164 ceph-mon[97427]: pgmap v846: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 14 op/s 2023-07-14T16:31:59.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:58 smithi062 ceph-mon[95543]: pgmap v847: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:31:59.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:58 smithi081 ceph-mon[127795]: pgmap v847: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:31:59.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:58 smithi164 ceph-mon[97427]: pgmap v847: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 16 MiB/s wr, 13 op/s 2023-07-14T16:32:00.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:31:59 smithi164 ceph-mon[97427]: pgmap v848: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 18 op/s 2023-07-14T16:32:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:31:59 smithi062 ceph-mon[95543]: pgmap v848: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 18 op/s 2023-07-14T16:32:00.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:31:59 smithi081 ceph-mon[127795]: pgmap v848: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 18 op/s 2023-07-14T16:32:00.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:32:00.446 INFO:tasks.workunit.client.0.smithi062.stdout:fs setup took 874 secs 2023-07-14T16:32:00.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 -- 172.21.15.62:0/2405912109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0ffae0 msgr2=0x7fcebc0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2405912109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0ffae0 0x7fcebc0ffef0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009990 tx=0x7fceb8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 -- 172.21.15.62:0/2405912109 shutdown_connections 2023-07-14T16:32:00.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2405912109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcebc101660 0x7fcebc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2405912109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0ffae0 0x7fcebc0ffef0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2405912109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcebc100cd0 0x7fcebc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 -- 172.21.15.62:0/2405912109 >> 172.21.15.62:0/2405912109 conn(0x7fcebc0fb0e0 msgr2=0x7fcebc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:00.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 -- 172.21.15.62:0/2405912109 shutdown_connections 2023-07-14T16:32:00.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.669+0000 7fcec33ff700 1 -- 172.21.15.62:0/2405912109 wait complete. 2023-07-14T16:32:00.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.670+0000 7fcec33ff700 1 Processor -- start 2023-07-14T16:32:00.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec33ff700 1 -- start start 2023-07-14T16:32:00.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec33ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0ffae0 0x7fcebc1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec33ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcebc100cd0 0x7fcebc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec33ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcebc101660 0x7fcebc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec33ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc19f560 con 0x7fcebc100cd0 2023-07-14T16:32:00.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec33ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc19f6a0 con 0x7fcebc0ffae0 2023-07-14T16:32:00.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec33ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc19f830 con 0x7fcebc101660 2023-07-14T16:32:00.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcebc100cd0 0x7fcebc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcebc100cd0 0x7fcebc197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58800/0 (socket says 172.21.15.62:58800) 2023-07-14T16:32:00.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec099a700 1 -- 172.21.15.62:0/2231332346 learned_addr learned my addr 172.21.15.62:0/2231332346 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec199c700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcebc101660 0x7fcebc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.671+0000 7fcec119b700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0ffae0 0x7fcebc1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fcec099a700 1 -- 172.21.15.62:0/2231332346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcebc101660 msgr2=0x7fcebc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fcec099a700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcebc101660 0x7fcebc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fcec099a700 1 -- 172.21.15.62:0/2231332346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0ffae0 msgr2=0x7fcebc1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fcec099a700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0ffae0 0x7fcebc1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fcec099a700 1 -- 172.21.15.62:0/2231332346 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb80095f0 con 0x7fcebc100cd0 2023-07-14T16:32:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fcec099a700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcebc100cd0 0x7fcebc197f20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fceac00e9a0 tx=0x7fceac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fceb27fc700 1 -- 172.21.15.62:0/2231332346 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac009790 con 0x7fcebc100cd0 2023-07-14T16:32:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fcec33ff700 1 -- 172.21.15.62:0/2231332346 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcebc19fb10 con 0x7fcebc100cd0 2023-07-14T16:32:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fceb27fc700 1 -- 172.21.15.62:0/2231332346 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fceac00cbd0 con 0x7fcebc100cd0 2023-07-14T16:32:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fceb27fc700 1 -- 172.21.15.62:0/2231332346 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac018680 con 0x7fcebc100cd0 2023-07-14T16:32:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.672+0000 7fcec33ff700 1 -- 172.21.15.62:0/2231332346 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc1a0070 con 0x7fcebc100cd0 2023-07-14T16:32:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.676+0000 7fceb27fc700 1 -- 172.21.15.62:0/2231332346 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fceac00cd40 con 0x7fcebc100cd0 2023-07-14T16:32:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fced6d42b80 1 Processor -- start 2023-07-14T16:32:00.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fced6d42b80 1 -- start start 2023-07-14T16:32:00.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fceb27fc700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcea80a11d0 0x7fcea80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fceb27fc700 1 -- 172.21.15.62:0/2231332346 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fceac014070 con 0x7fcebc100cd0 2023-07-14T16:32:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc5a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc5ac70 0x55e20bc5b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fced6d42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bb5ecb0 con 0x55e20bc5ac70 2023-07-14T16:32:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fced6d42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bbb1a40 con 0x55e20bc59e20 2023-07-14T16:32:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fced6d42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bbaaed0 con 0x55e20bc594d0 2023-07-14T16:32:00.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.677+0000 7fcec199c700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcea80a11d0 0x7fcea80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec119b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec119b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38820/0 (socket says 172.21.15.62:38820) 2023-07-14T16:32:00.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec119b700 1 -- 172.21.15.62:0/223612789 learned_addr learned my addr 172.21.15.62:0/223612789 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:00.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec099a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc5a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec199c700 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc5ac70 0x55e20bc5b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec119b700 1 -- 172.21.15.62:0/223612789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 msgr2=0x55e20bc5a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:00.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec119b700 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc5a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec119b700 1 -- 172.21.15.62:0/223612789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc5ac70 msgr2=0x55e20bc5b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:00.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec119b700 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc5ac70 0x55e20bc5b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec119b700 1 -- 172.21.15.62:0/223612789 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e20b9d8de0 con 0x55e20bc594d0 2023-07-14T16:32:00.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec099a700 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc5a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:32:00.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.678+0000 7fcec199c700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcea80a11d0 0x7fcea80a3680 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fceb400f870 tx=0x7fceb400fc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:00.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.679+0000 7fcec119b700 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc598e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fceb8006010 tx=0x7fceb8005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f6f1ffa137de6d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:00.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.679+0000 7fcea77fe700 1 -- 172.21.15.62:0/223612789 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb8005300 con 0x55e20bc594d0 2023-07-14T16:32:00.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.679+0000 7fcea77fe700 1 -- 172.21.15.62:0/223612789 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fceb801e040 con 0x55e20bc594d0 2023-07-14T16:32:00.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.679+0000 7fcea77fe700 1 -- 172.21.15.62:0/223612789 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb800b8b0 con 0x55e20bc594d0 2023-07-14T16:32:00.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 -- 172.21.15.62:0/223612789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 msgr2=0x55e20bc598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc598e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fceb8006010 tx=0x7fceb8005f00 comp rx=0 tx=0).stop 2023-07-14T16:32:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 -- 172.21.15.62:0/223612789 shutdown_connections 2023-07-14T16:32:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc5ac70 0x55e20bc5b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc598e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 --2- 172.21.15.62:0/223612789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc5a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 -- 172.21.15.62:0/223612789 >> 172.21.15.62:0/223612789 conn(0x55e20bc4eda0 msgr2=0x55e20bc541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 -- 172.21.15.62:0/223612789 shutdown_connections 2023-07-14T16:32:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 -- 172.21.15.62:0/223612789 wait complete. 2023-07-14T16:32:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.680+0000 7fced6d42b80 1 -- start start 2023-07-14T16:32:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fced6d42b80 1 Processor -- start 2023-07-14T16:32:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc594d0 0x55e20bc630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 0x55e20bc6a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fced6d42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bb4aac0 con 0x55e20bc594d0 2023-07-14T16:32:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fced6d42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bb4a7c0 con 0x55e20bc59e20 2023-07-14T16:32:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fced6d42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20ba2e920 con 0x55e20bc5ac70 2023-07-14T16:32:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fced6d42b80 10 client.0 _ll_register_callbacks cb 0x55e20bb48760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fcec119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc594d0 0x55e20bc630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fcec119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc594d0 0x55e20bc630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58808/0 (socket says 172.21.15.62:58808) 2023-07-14T16:32:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fcec119b700 1 -- 172.21.15.62:0/2030752986 learned_addr learned my addr 172.21.15.62:0/2030752986 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.681+0000 7fcec199c700 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 0x55e20bc6a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.682+0000 7fcec099a700 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.682+0000 7fcec119b700 1 -- 172.21.15.62:0/2030752986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 msgr2=0x55e20bc6a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:00.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.682+0000 7fcec119b700 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 0x55e20bc6a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.682+0000 7fcec119b700 1 -- 172.21.15.62:0/2030752986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 msgr2=0x55e20bc635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.682+0000 7fcec119b700 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.682+0000 7fcec119b700 1 -- 172.21.15.62:0/2030752986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc1a0070 con 0x55e20bc594d0 2023-07-14T16:32:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.682+0000 7fcec119b700 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc594d0 0x55e20bc630b0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fceb80098e0 tx=0x7fceb8003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65f48e959de0cca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.682+0000 7fcea4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fcea4ff9700 1 -- 172.21.15.62:0/2030752986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb800b6b0 con 0x55e20bc594d0 2023-07-14T16:32:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fcea4ff9700 1 -- 172.21.15.62:0/2030752986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fceb800b810 con 0x55e20bc594d0 2023-07-14T16:32:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fced6d42b80 1 -- 172.21.15.62:0/2030752986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e20bb044f0 con 0x55e20bc594d0 2023-07-14T16:32:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fcea4ff9700 1 -- 172.21.15.62:0/2030752986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceb800b9d0 con 0x55e20bc594d0 2023-07-14T16:32:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fced6d42b80 10 client.20079 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fced6d42b80 1 -- 172.21.15.62:0/2030752986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fcebc108850 con 0x55e20bc594d0 2023-07-14T16:32:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fcea4ff9700 1 -- 172.21.15.62:0/2030752986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e20bb044f0 con 0x55e20bc594d0 2023-07-14T16:32:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fcea4ff9700 1 -- 172.21.15.62:0/2030752986 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fceb8004d20 con 0x55e20bc594d0 2023-07-14T16:32:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fced6d42b80 10 client.20079 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fced6d42b80 10 client.20079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.683+0000 7fced6d42b80 20 client.20079 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.684+0000 7fced6d42b80 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e20bc6e910 0x55e20bc70d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.684+0000 7fced6d42b80 4 client.20079 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.684+0000 7fced6d42b80 1 -- 172.21.15.62:0/2030752986 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e20bba3eb0 con 0x55e20bc6e910 2023-07-14T16:32:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.684+0000 7fcec099a700 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e20bc6e910 0x55e20bc70d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.685+0000 7fcec099a700 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e20bc6e910 0x55e20bc70d20 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8516f3b9378f7aa server_cookie=1c4a6a63c1f44b94 in_seq=0 out_seq=0 2023-07-14T16:32:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.685+0000 7fcea4ff9700 10 client.20079 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:00.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.685+0000 7fcea4ff9700 1 -- 172.21.15.62:0/2030752986 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e20bba3eb0 con 0x55e20bc6e910 2023-07-14T16:32:00.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.685+0000 7fcea4ff9700 10 client.20079 handle_command_reply: tid=0 2023-07-14T16:32:00.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 client.20079 shutdown 2023-07-14T16:32:00.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 -- 172.21.15.62:0/2030752986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc594d0 msgr2=0x55e20bc630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc594d0 0x55e20bc630b0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fceb80098e0 tx=0x7fceb8003920 comp rx=0 tx=0).stop 2023-07-14T16:32:00.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 -- 172.21.15.62:0/2030752986 shutdown_connections 2023-07-14T16:32:00.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 0x55e20bc6a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc594d0 0x55e20bc630b0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e20bc6e910 0x55e20bc70d20 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 --2- 172.21.15.62:0/2030752986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc59e20 0x55e20bc635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fcea4ff9700 0 client.20079 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:00.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 -- 172.21.15.62:0/2030752986 >> 172.21.15.62:0/2030752986 conn(0x55e20bc4eda0 msgr2=0x55e20bc538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:00.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 -- 172.21.15.62:0/2030752986 shutdown_connections 2023-07-14T16:32:00.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 1 -- 172.21.15.62:0/2030752986 wait complete. 2023-07-14T16:32:00.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.686+0000 7fced6d42b80 20 client.20079 trim_cache size 0 max 16384 2023-07-14T16:32:00.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fced6d42b80 1 Processor -- start 2023-07-14T16:32:00.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fced6d42b80 1 -- start start 2023-07-14T16:32:00.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc75640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc6a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc5ac70 0x55e20bc6af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fced6d42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bc75c10 con 0x55e20bc59e20 2023-07-14T16:32:00.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fced6d42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bc55270 con 0x55e20bc5ac70 2023-07-14T16:32:00.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fced6d42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bc598a0 con 0x55e20bc594d0 2023-07-14T16:32:00.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fcec099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc6a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.704+0000 7fcec099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc6a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58822/0 (socket says 172.21.15.62:58822) 2023-07-14T16:32:00.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec099a700 1 -- 172.21.15.62:0/39169747 learned_addr learned my addr 172.21.15.62:0/39169747 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:00.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec199c700 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc5ac70 0x55e20bc6af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec119b700 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc75640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec099a700 1 -- 172.21.15.62:0/39169747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 msgr2=0x55e20bc75640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:00.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec099a700 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc75640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec099a700 1 -- 172.21.15.62:0/39169747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc5ac70 msgr2=0x55e20bc6af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:00.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec099a700 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc5ac70 0x55e20bc6af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec099a700 1 -- 172.21.15.62:0/39169747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e20b9c1c60 con 0x55e20bc59e20 2023-07-14T16:32:00.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcec099a700 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc6a5f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fceac096980 tx=0x7fceac0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbb61ee1b47cb01 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:00.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcea7fff700 1 -- 172.21.15.62:0/39169747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac09df40 con 0x55e20bc59e20 2023-07-14T16:32:00.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcea7fff700 1 -- 172.21.15.62:0/39169747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fceac0a2360 con 0x55e20bc59e20 2023-07-14T16:32:00.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.705+0000 7fcea7fff700 1 -- 172.21.15.62:0/39169747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac0a1ed0 con 0x55e20bc59e20 2023-07-14T16:32:00.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 -- 172.21.15.62:0/39169747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 msgr2=0x55e20bc6a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc6a5f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fceac096980 tx=0x7fceac0cc040 comp rx=0 tx=0).stop 2023-07-14T16:32:00.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 -- 172.21.15.62:0/39169747 shutdown_connections 2023-07-14T16:32:00.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc5ac70 0x55e20bc6af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc594d0 0x55e20bc75640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 --2- 172.21.15.62:0/39169747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc6a5f0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 -- 172.21.15.62:0/39169747 >> 172.21.15.62:0/39169747 conn(0x55e20bc4eda0 msgr2=0x55e20bc61e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:00.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 -- 172.21.15.62:0/39169747 shutdown_connections 2023-07-14T16:32:00.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.706+0000 7fced6d42b80 1 -- 172.21.15.62:0/39169747 wait complete. 2023-07-14T16:32:00.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 1 -- start start 2023-07-14T16:32:00.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 1 Processor -- start 2023-07-14T16:32:00.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc594d0 0x55e20bc6a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc66e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 0x55e20bc67350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bc67920 con 0x55e20bc59e20 2023-07-14T16:32:00.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bc67a60 con 0x55e20bc594d0 2023-07-14T16:32:00.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20bcc27d0 con 0x55e20bc5ac70 2023-07-14T16:32:00.686 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.707+0000 7fced6d42b80 10 client.0 _ll_register_callbacks cb 0x7fceac00ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:00.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc66e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc66e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58834/0 (socket says 172.21.15.62:58834) 2023-07-14T16:32:00.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 -- 172.21.15.62:0/1698707450 learned_addr learned my addr 172.21.15.62:0/1698707450 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:00.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec119b700 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc594d0 0x55e20bc6a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 -- 172.21.15.62:0/1698707450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 msgr2=0x55e20bc67350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:00.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 0x55e20bc67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 -- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc594d0 msgr2=0x55e20bc6a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:00.687 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc594d0 0x55e20bc6a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 -- 172.21.15.62:0/1698707450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc19fb10 con 0x55e20bc59e20 2023-07-14T16:32:00.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec119b700 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc594d0 0x55e20bc6a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:00.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcec099a700 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc66e10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fceac099e50 tx=0x7fceac0982b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a334cbc931962fc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:00.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcea4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:00.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcea4ff9700 1 -- 172.21.15.62:0/1698707450 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac09df40 con 0x55e20bc59e20 2023-07-14T16:32:00.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fcea4ff9700 1 -- 172.21.15.62:0/1698707450 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fceac0a1ed0 con 0x55e20bc59e20 2023-07-14T16:32:00.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.708+0000 7fced6d42b80 1 -- 172.21.15.62:0/1698707450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fceb801e040 con 0x55e20bc59e20 2023-07-14T16:32:00.688 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fcea4ff9700 1 -- 172.21.15.62:0/1698707450 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceac0a2090 con 0x55e20bc59e20 2023-07-14T16:32:00.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fced6d42b80 10 client.20085 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:00.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fced6d42b80 1 -- 172.21.15.62:0/1698707450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e20b9ff250 con 0x55e20bc59e20 2023-07-14T16:32:00.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fcea4ff9700 1 -- 172.21.15.62:0/1698707450 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fceac07e020 con 0x55e20bc59e20 2023-07-14T16:32:00.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fcea4ff9700 1 -- 172.21.15.62:0/1698707450 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fceac095f00 con 0x55e20bc59e20 2023-07-14T16:32:00.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fced6d42b80 10 client.20085 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:00.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fced6d42b80 10 client.20085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:00.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fced6d42b80 20 client.20085 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:00.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fced6d42b80 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e20bcc3f60 0x55e20bcc6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fced6d42b80 4 client.20085 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:32:00.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fced6d42b80 1 -- 172.21.15.62:0/1698707450 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e20bb66050 con 0x55e20bcc3f60 2023-07-14T16:32:00.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.709+0000 7fcec119b700 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e20bcc3f60 0x55e20bcc6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fcec119b700 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e20bcc3f60 0x55e20bcc6370 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d77c97eb0c3361 server_cookie=ee26e4133df76ce9 in_seq=0 out_seq=0 2023-07-14T16:32:00.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fcea4ff9700 10 client.20085 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:00.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fcea4ff9700 1 -- 172.21.15.62:0/1698707450 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e20bb66050 con 0x55e20bcc3f60 2023-07-14T16:32:00.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fcea4ff9700 10 client.20085 handle_command_reply: tid=0 2023-07-14T16:32:00.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 client.20085 shutdown 2023-07-14T16:32:00.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 -- 172.21.15.62:0/1698707450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 msgr2=0x55e20bc66e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc66e10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fceac099e50 tx=0x7fceac0982b0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 -- 172.21.15.62:0/1698707450 shutdown_connections 2023-07-14T16:32:00.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e20bc5ac70 0x55e20bc67350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e20bcc3f60 0x55e20bcc6370 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e20bc594d0 0x55e20bc6a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 --2- 172.21.15.62:0/1698707450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e20bc59e20 0x55e20bc66e10 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fcea4ff9700 0 client.20085 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:00.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.710+0000 7fced6d42b80 1 -- 172.21.15.62:0/1698707450 >> 172.21.15.62:0/1698707450 conn(0x55e20bc4eda0 msgr2=0x55e20bc5da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:00.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fced6d42b80 1 -- 172.21.15.62:0/1698707450 shutdown_connections 2023-07-14T16:32:00.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fced6d42b80 1 -- 172.21.15.62:0/1698707450 wait complete. 2023-07-14T16:32:00.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fced6d42b80 20 client.20085 trim_cache size 0 max 16384 2023-07-14T16:32:00.692 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:32:00.693 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:32:00.693 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:32:00.693 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:32:00.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 -- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcea80a11d0 msgr2=0x7fcea80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcea80a11d0 0x7fcea80a3680 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fceb400f870 tx=0x7fceb400fc30 comp rx=0 tx=0).stop 2023-07-14T16:32:00.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 -- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcebc100cd0 msgr2=0x7fcebc197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcebc100cd0 0x7fcebc197f20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fceac00e9a0 tx=0x7fceac00ecb0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 -- 172.21.15.62:0/2231332346 shutdown_connections 2023-07-14T16:32:00.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcebc101660 0x7fcebc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fcea80a11d0 0x7fcea80a3680 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0ffae0 0x7fcebc1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 --2- 172.21.15.62:0/2231332346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcebc100cd0 0x7fcebc197f20 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.711+0000 7fcec33ff700 1 -- 172.21.15.62:0/2231332346 >> 172.21.15.62:0/2231332346 conn(0x7fcebc0fb0e0 msgr2=0x7fcebc0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:00.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.712+0000 7fcec33ff700 1 -- 172.21.15.62:0/2231332346 shutdown_connections 2023-07-14T16:32:00.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:00.712+0000 7fcec33ff700 1 -- 172.21.15.62:0/2231332346 wait complete. 2023-07-14T16:32:00.702 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:32:00.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:32:00.865 INFO:tasks.workunit.client.0.smithi062.stdout:Syncing()...0 sec 2023-07-14T16:32:00.867 INFO:tasks.workunit.client.0.smithi062.stdout:Starting Actual Benchmark At: Fri Jul 14 16:32:00 2023 2023-07-14T16:32:00.867 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:32:00.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 -- 172.21.15.62:0/2990163792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64300ff640 msgr2=0x7f64300ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 --2- 172.21.15.62:0/2990163792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64300ff640 0x7f64300ffab0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f642800b0a0 tx=0x7f642800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 -- 172.21.15.62:0/2990163792 shutdown_connections 2023-07-14T16:32:00.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 --2- 172.21.15.62:0/2990163792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64300ff640 0x7f64300ffab0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 --2- 172.21.15.62:0/2990163792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64300fe340 0x7f64300fe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 --2- 172.21.15.62:0/2990163792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64300fec90 0x7f64300ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 -- 172.21.15.62:0/2990163792 >> 172.21.15.62:0/2990163792 conn(0x7f64300f9fd0 msgr2=0x7f64300fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:00.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 -- 172.21.15.62:0/2990163792 shutdown_connections 2023-07-14T16:32:00.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.003+0000 7f64387ff700 1 -- 172.21.15.62:0/2990163792 wait complete. 2023-07-14T16:32:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.005+0000 7f64387ff700 1 Processor -- start 2023-07-14T16:32:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.005+0000 7f64387ff700 1 -- start start 2023-07-14T16:32:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.005+0000 7f64387ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64300fe340 0x7f6430197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.005+0000 7f64387ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64300fec90 0x7f6430197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.005+0000 7f64387ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64300ff640 0x7f643019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.005+0000 7f64387ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643019f500 con 0x7f64300ff640 2023-07-14T16:32:00.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.005+0000 7f64387ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643019f640 con 0x7f64300fe340 2023-07-14T16:32:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.005+0000 7f64387ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643019f7d0 con 0x7f64300fec90 2023-07-14T16:32:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.006+0000 7f643659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64300fe340 0x7f6430197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.006+0000 7f643659b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64300fe340 0x7f6430197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53746/0 (socket says 172.21.15.62:53746) 2023-07-14T16:32:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.006+0000 7f643659b700 1 -- 172.21.15.62:0/1172663115 learned_addr learned my addr 172.21.15.62:0/1172663115 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.006+0000 7f6435d9a700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64300fec90 0x7f6430197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.006+0000 7f6436d9c700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64300ff640 0x7f643019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.007+0000 7f643659b700 1 -- 172.21.15.62:0/1172663115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64300fec90 msgr2=0x7f6430197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.007+0000 7f643659b700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64300fec90 0x7f6430197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.007+0000 7f643659b700 1 -- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64300ff640 msgr2=0x7f643019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:00.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.007+0000 7f643659b700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64300ff640 0x7f643019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:00.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.007+0000 7f643659b700 1 -- 172.21.15.62:0/1172663115 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428009d00 con 0x7f64300fe340 2023-07-14T16:32:00.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.007+0000 7f643659b700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64300fe340 0x7f6430197980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f642c00d950 tx=0x7f642c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:00.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.007+0000 7f64277fe700 1 -- 172.21.15.62:0/1172663115 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c00f890 con 0x7f64300fe340 2023-07-14T16:32:00.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.008+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643019fab0 con 0x7f64300fe340 2023-07-14T16:32:00.987 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.008+0000 7f64277fe700 1 -- 172.21.15.62:0/1172663115 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f642c00fe80 con 0x7f64300fe340 2023-07-14T16:32:00.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.008+0000 7f64277fe700 1 -- 172.21.15.62:0/1172663115 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c00e3b0 con 0x7f64300fe340 2023-07-14T16:32:00.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.008+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64301a0010 con 0x7f64300fe340 2023-07-14T16:32:00.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.009+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6430109120 con 0x7f64300fe340 2023-07-14T16:32:00.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.012+0000 7f64277fe700 1 -- 172.21.15.62:0/1172663115 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f642c010460 con 0x7f64300fe340 2023-07-14T16:32:00.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.013+0000 7f64277fe700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f641c0a1220 0x7f641c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.013+0000 7f64277fe700 1 -- 172.21.15.62:0/1172663115 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f642c0cc7a0 con 0x7f64300fe340 2023-07-14T16:32:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.014+0000 7f6435d9a700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f641c0a1220 0x7f641c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.015+0000 7f6435d9a700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f641c0a1220 0x7f641c0a36d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f642000ac50 tx=0x7f642000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.015+0000 7f64277fe700 1 -- 172.21.15.62:0/1172663115 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f642c095ea0 con 0x7f64300fe340 2023-07-14T16:32:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.472+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f643002c850 con 0x7f64300fe340 2023-07-14T16:32:01.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.473+0000 7f64277fe700 1 -- 172.21.15.62:0/1172663115 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10828 (secure 0 0 0) 0x7f642c020090 con 0x7f64300fe340 2023-07-14T16:32:01.447 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:32:01.448 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:31:18.877417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:32:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f641c0a1220 msgr2=0x7f641c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f641c0a1220 0x7f641c0a36d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f642000ac50 tx=0x7f642000a380 comp rx=0 tx=0).stop 2023-07-14T16:32:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64300fe340 msgr2=0x7f6430197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64300fe340 0x7f6430197980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f642c00d950 tx=0x7f642c00dc60 comp rx=0 tx=0).stop 2023-07-14T16:32:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 shutdown_connections 2023-07-14T16:32:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64300ff640 0x7f643019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f641c0a1220 0x7f641c0a36d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64300fe340 0x7f6430197980 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 --2- 172.21.15.62:0/1172663115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64300fec90 0x7f6430197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.478+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 >> 172.21.15.62:0/1172663115 conn(0x7f64300f9fd0 msgr2=0x7f64300fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.479+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 shutdown_connections 2023-07-14T16:32:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.479+0000 7f64387ff700 1 -- 172.21.15.62:0/1172663115 wait complete. 2023-07-14T16:32:01.454 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2023-07-14T16:32:01.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:32:01.769 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:01 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1172663115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:32:01.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.862+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/1807560773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d08107780 msgr2=0x7f6d08107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.862+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/1807560773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d08107780 0x7f6d08107b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009990 tx=0x7f6cf8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.863+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/1807560773 shutdown_connections 2023-07-14T16:32:01.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.863+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/1807560773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d080716b0 0x7f6d08071b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.863+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/1807560773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d08107780 0x7f6d08107b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.863+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/1807560773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d081080d0 0x7f6d080710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.863+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/1807560773 >> 172.21.15.62:0/1807560773 conn(0x7f6d0806cd80 msgr2=0x7f6d0806f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:01.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.863+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/1807560773 shutdown_connections 2023-07-14T16:32:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.863+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/1807560773 wait complete. 2023-07-14T16:32:01.838 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.864+0000 7f6d0c9ff700 1 Processor -- start 2023-07-14T16:32:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.864+0000 7f6d0c9ff700 1 -- start start 2023-07-14T16:32:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0c9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d080716b0 0x7f6d0810e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0c9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d08107780 0x7f6d0810e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0c9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d081080d0 0x7f6d0810ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.839 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0c9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0810b4d0 con 0x7f6d081080d0 2023-07-14T16:32:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0c9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0810b640 con 0x7f6d080716b0 2023-07-14T16:32:01.840 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0c9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0810b800 con 0x7f6d08107780 2023-07-14T16:32:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d080716b0 0x7f6d0810e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.841 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d080716b0 0x7f6d0810e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53766/0 (socket says 172.21.15.62:53766) 2023-07-14T16:32:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d0659c700 1 -- 172.21.15.62:0/4053404184 learned_addr learned my addr 172.21.15.62:0/4053404184 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d081080d0 0x7f6d0810ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.865+0000 7f6d05d9b700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d08107780 0x7f6d0810e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.866+0000 7f6d0659c700 1 -- 172.21.15.62:0/4053404184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d08107780 msgr2=0x7f6d0810e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.866+0000 7f6d0659c700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d08107780 0x7f6d0810e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.866+0000 7f6d0659c700 1 -- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d081080d0 msgr2=0x7f6d0810ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.866+0000 7f6d0659c700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d081080d0 0x7f6d0810ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.866+0000 7f6d0659c700 1 -- 172.21.15.62:0/4053404184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80095f0 con 0x7f6d080716b0 2023-07-14T16:32:01.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.866+0000 7f6d0659c700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d080716b0 0x7f6d0810e1a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005910 tx=0x7f6cf8005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:01.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.866+0000 7f6cf77fe700 1 -- 172.21.15.62:0/4053404184 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf801d070 con 0x7f6d080716b0 2023-07-14T16:32:01.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.867+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/4053404184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d081ac6d0 con 0x7f6d080716b0 2023-07-14T16:32:01.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.867+0000 7f6cf77fe700 1 -- 172.21.15.62:0/4053404184 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6cf8022470 con 0x7f6d080716b0 2023-07-14T16:32:01.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.867+0000 7f6cf77fe700 1 -- 172.21.15.62:0/4053404184 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800f6c0 con 0x7f6d080716b0 2023-07-14T16:32:01.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.867+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/4053404184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d081acbd0 con 0x7f6d080716b0 2023-07-14T16:32:01.847 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.873+0000 7f6cf77fe700 1 -- 172.21.15.62:0/4053404184 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6cf800ba60 con 0x7f6d080716b0 2023-07-14T16:32:01.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.874+0000 7f6d20321b80 1 Processor -- start 2023-07-14T16:32:01.848 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.875+0000 7f6d20321b80 1 -- start start 2023-07-14T16:32:01.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.875+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd98c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.849 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.875+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98ce20 0x55eecd98d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.875+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98dc70 0x55eecd98e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.875+0000 7f6d20321b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd891cb0 con 0x55eecd98ce20 2023-07-14T16:32:01.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.875+0000 7f6d20321b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd8e4a40 con 0x55eecd98dc70 2023-07-14T16:32:01.850 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.875+0000 7f6d20321b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd8dded0 con 0x55eecd98c4d0 2023-07-14T16:32:01.852 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d06d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd98c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98ce20 0x55eecd98d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6cf77fe700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6cf00a11d0 0x7f6cf00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.853 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d06d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd98c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38886/0 (socket says 172.21.15.62:38886) 2023-07-14T16:32:01.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d06d9d700 1 -- 172.21.15.62:0/3244018221 learned_addr learned my addr 172.21.15.62:0/3244018221 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:01.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d05d9b700 1 --2- 172.21.15.62:0/3244018221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98dc70 0x55eecd98e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6cf77fe700 1 -- 172.21.15.62:0/4053404184 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cf80cdd60 con 0x7f6d080716b0 2023-07-14T16:32:01.854 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d06d9d700 1 -- 172.21.15.62:0/3244018221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98dc70 msgr2=0x55eecd98e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:01.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/3244018221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98dc70 0x55eecd98e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d06d9d700 1 -- 172.21.15.62:0/3244018221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98ce20 msgr2=0x55eecd98d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:01.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/3244018221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98ce20 0x55eecd98d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.855 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.876+0000 7f6d06d9d700 1 -- 172.21.15.62:0/3244018221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eecd70bde0 con 0x55eecd98c4d0 2023-07-14T16:32:01.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.877+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6cf00a11d0 0x7f6cf00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.877+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/3244018221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd98c8e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000a500 tx=0x7f6d0000a810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef36f20843e7121 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:01.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.877+0000 7f6cf4ff9700 1 -- 172.21.15.62:0/3244018221 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00009bb0 con 0x55eecd98c4d0 2023-07-14T16:32:01.856 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.877+0000 7f6cf4ff9700 1 -- 172.21.15.62:0/3244018221 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d000238d0 con 0x55eecd98c4d0 2023-07-14T16:32:01.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.877+0000 7f6cf4ff9700 1 -- 172.21.15.62:0/3244018221 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00017720 con 0x55eecd98c4d0 2023-07-14T16:32:01.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.878+0000 7f6d20321b80 1 -- 172.21.15.62:0/3244018221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 msgr2=0x55eecd98c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.878+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6cf00a11d0 0x7f6cf00a3680 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6d00020000 tx=0x7f6d00021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:01.857 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.878+0000 7f6d20321b80 1 --2- 172.21.15.62:0/3244018221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd98c8e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000a500 tx=0x7f6d0000a810 comp rx=0 tx=0).stop 2023-07-14T16:32:01.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.878+0000 7f6d20321b80 1 -- 172.21.15.62:0/3244018221 shutdown_connections 2023-07-14T16:32:01.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.878+0000 7f6d20321b80 1 --2- 172.21.15.62:0/3244018221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98dc70 0x55eecd98e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.878+0000 7f6d20321b80 1 --2- 172.21.15.62:0/3244018221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd98c8e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.858 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.878+0000 7f6d20321b80 1 --2- 172.21.15.62:0/3244018221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98ce20 0x55eecd98d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.878+0000 7f6d20321b80 1 -- 172.21.15.62:0/3244018221 >> 172.21.15.62:0/3244018221 conn(0x55eecd981da0 msgr2=0x55eecd9871d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:01.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.879+0000 7f6d20321b80 1 -- 172.21.15.62:0/3244018221 shutdown_connections 2023-07-14T16:32:01.859 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.879+0000 7f6d20321b80 1 -- 172.21.15.62:0/3244018221 wait complete. 2023-07-14T16:32:01.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.879+0000 7f6d20321b80 1 -- start start 2023-07-14T16:32:01.864 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.880+0000 7f6d20321b80 1 Processor -- start 2023-07-14T16:32:01.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.881+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd996110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.881+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 0x55eecd996650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.865 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.881+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd99d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.881+0000 7f6d20321b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd87dac0 con 0x55eecd98dc70 2023-07-14T16:32:01.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.881+0000 7f6d20321b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd87d7c0 con 0x55eecd98c4d0 2023-07-14T16:32:01.866 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.881+0000 7f6d20321b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd761920 con 0x55eecd98ce20 2023-07-14T16:32:01.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.881+0000 7f6d20321b80 10 client.0 _ll_register_callbacks cb 0x55eecd87b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:01.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.882+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd996110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.882+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd996110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53782/0 (socket says 172.21.15.62:53782) 2023-07-14T16:32:01.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.882+0000 7f6d0659c700 1 -- 172.21.15.62:0/1661304004 learned_addr learned my addr 172.21.15.62:0/1661304004 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:01.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.882+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd99d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.867 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.883+0000 7f6d0659c700 1 -- 172.21.15.62:0/1661304004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 msgr2=0x55eecd996650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:32:01.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.883+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 0x55eecd996650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.883+0000 7f6d0659c700 1 -- 172.21.15.62:0/1661304004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 msgr2=0x55eecd99d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:01.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.883+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd99d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.883+0000 7f6d0659c700 1 -- 172.21.15.62:0/1661304004 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d00009d10 con 0x55eecd98c4d0 2023-07-14T16:32:01.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.883+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd996110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6cf809aeb0 tx=0x7f6cf809acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafa2c19d0e57e9f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:01.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.883+0000 7f6cedffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:01.868 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.883+0000 7f6cedffb700 1 -- 172.21.15.62:0/1661304004 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8022800 con 0x55eecd98c4d0 2023-07-14T16:32:01.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.884+0000 7f6cedffb700 1 -- 172.21.15.62:0/1661304004 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cf809b1b0 con 0x55eecd98c4d0 2023-07-14T16:32:01.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.884+0000 7f6d20321b80 1 -- 172.21.15.62:0/1661304004 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eecd8374f0 con 0x55eecd98c4d0 2023-07-14T16:32:01.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.884+0000 7f6cedffb700 1 -- 172.21.15.62:0/1661304004 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cf809f1e0 con 0x55eecd98c4d0 2023-07-14T16:32:01.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.885+0000 7f6d20321b80 10 client.29618 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:01.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.885+0000 7f6d20321b80 1 -- 172.21.15.62:0/1661304004 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55eecd6f4c60 con 0x55eecd98c4d0 2023-07-14T16:32:01.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.885+0000 7f6cedffb700 1 -- 172.21.15.62:0/1661304004 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eecd8374f0 con 0x55eecd98c4d0 2023-07-14T16:32:01.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.886+0000 7f6cedffb700 1 -- 172.21.15.62:0/1661304004 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6cf80a30e0 con 0x55eecd98c4d0 2023-07-14T16:32:01.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.886+0000 7f6d20321b80 10 client.29618 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:01.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.886+0000 7f6d20321b80 10 client.29618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:32:01.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.886+0000 7f6d20321b80 20 client.29618 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:01.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.886+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55eecd9a1ab0 0x55eecd9a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.886+0000 7f6d20321b80 4 client.29618 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:01.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.886+0000 7f6d20321b80 1 -- 172.21.15.62:0/1661304004 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eecd8d6eb0 con 0x55eecd9a1ab0 2023-07-14T16:32:01.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.886+0000 7f6d05d9b700 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55eecd9a1ab0 0x55eecd9a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.888+0000 7f6cedffb700 10 client.29618 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:32:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.888+0000 7f6d05d9b700 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55eecd9a1ab0 0x55eecd9a3e90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66b5e4bdc4a2c8e0 server_cookie=8dea94ab8fe51930 in_seq=0 out_seq=0 2023-07-14T16:32:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.888+0000 7f6cedffb700 1 -- 172.21.15.62:0/1661304004 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eecd8d6eb0 con 0x55eecd9a1ab0 2023-07-14T16:32:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.888+0000 7f6cedffb700 10 client.29618 handle_command_reply: tid=0 2023-07-14T16:32:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.888+0000 7f6d20321b80 1 client.29618 shutdown 2023-07-14T16:32:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 -- 172.21.15.62:0/1661304004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 msgr2=0x55eecd996110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd996110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6cf809aeb0 tx=0x7f6cf809acf0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 -- 172.21.15.62:0/1661304004 shutdown_connections 2023-07-14T16:32:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55eecd9a1ab0 0x55eecd9a3e90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd99d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6cedffb700 0 client.29618 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:32:01.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd996110 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1661304004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 0x55eecd996650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 -- 172.21.15.62:0/1661304004 >> 172.21.15.62:0/1661304004 conn(0x55eecd981da0 msgr2=0x55eecd9868b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:01.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 -- 172.21.15.62:0/1661304004 shutdown_connections 2023-07-14T16:32:01.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 1 -- 172.21.15.62:0/1661304004 wait complete. 2023-07-14T16:32:01.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.889+0000 7f6d20321b80 20 client.29618 trim_cache size 0 max 16384 2023-07-14T16:32:01.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:01 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1172663115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:32:01.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.916+0000 7f6d20321b80 1 Processor -- start 2023-07-14T16:32:01.890 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.916+0000 7f6d20321b80 1 -- start start 2023-07-14T16:32:01.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.916+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd994eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.916+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 0x55eecd99d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.916+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd993fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.916+0000 7f6d20321b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd9953f0 con 0x55eecd98dc70 2023-07-14T16:32:01.891 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.916+0000 7f6d20321b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd988a00 con 0x55eecd98c4d0 2023-07-14T16:32:01.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.916+0000 7f6d20321b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd993d50 con 0x55eecd98ce20 2023-07-14T16:32:01.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.917+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd994eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.917+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd994eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53796/0 (socket says 172.21.15.62:53796) 2023-07-14T16:32:01.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.917+0000 7f6d0659c700 1 -- 172.21.15.62:0/1492733337 learned_addr learned my addr 172.21.15.62:0/1492733337 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:01.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.918+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd993fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.893 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.918+0000 7f6d05d9b700 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 0x55eecd99d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.918+0000 7f6d0659c700 1 -- 172.21.15.62:0/1492733337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 msgr2=0x55eecd99d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:01.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.918+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 0x55eecd99d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.918+0000 7f6d0659c700 1 -- 172.21.15.62:0/1492733337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 msgr2=0x55eecd993fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:01.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.918+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd993fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.918+0000 7f6d0659c700 1 -- 172.21.15.62:0/1492733337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eecd732250 con 0x55eecd98c4d0 2023-07-14T16:32:01.894 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.918+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd993fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:32:01.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.919+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd994eb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80a3040 tx=0x7f6cf809f110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b151d9ea280946 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:01.895 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.919+0000 7f6cf57fa700 1 -- 172.21.15.62:0/1492733337 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf809a110 con 0x55eecd98c4d0 2023-07-14T16:32:01.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.920+0000 7f6cf57fa700 1 -- 172.21.15.62:0/1492733337 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6cf8022800 con 0x55eecd98c4d0 2023-07-14T16:32:01.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.920+0000 7f6cf57fa700 1 -- 172.21.15.62:0/1492733337 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80a30e0 con 0x55eecd98c4d0 2023-07-14T16:32:01.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.921+0000 7f6d20321b80 1 -- 172.21.15.62:0/1492733337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 msgr2=0x55eecd994eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.921+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd994eb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80a3040 tx=0x7f6cf809f110 comp rx=0 tx=0).stop 2023-07-14T16:32:01.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.921+0000 7f6d20321b80 1 -- 172.21.15.62:0/1492733337 shutdown_connections 2023-07-14T16:32:01.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.921+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd993fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.921+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98c4d0 0x55eecd994eb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.921+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1492733337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98ce20 0x55eecd99d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.921+0000 7f6d20321b80 1 -- 172.21.15.62:0/1492733337 >> 172.21.15.62:0/1492733337 conn(0x55eecd981da0 msgr2=0x55eecd9a87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:01.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.922+0000 7f6d20321b80 1 -- 172.21.15.62:0/1492733337 shutdown_connections 2023-07-14T16:32:01.897 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:01 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1172663115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:32:01.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.922+0000 7f6d20321b80 1 -- 172.21.15.62:0/1492733337 wait complete. 2023-07-14T16:32:01.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.922+0000 7f6d20321b80 1 -- start start 2023-07-14T16:32:01.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.923+0000 7f6d20321b80 1 Processor -- start 2023-07-14T16:32:01.898 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.923+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd99d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.923+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98ce20 0x55eecd99d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.923+0000 7f6d20321b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd999c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.923+0000 7f6d20321b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd99a300 con 0x55eecd98dc70 2023-07-14T16:32:01.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.923+0000 7f6d20321b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd99a440 con 0x55eecd98ce20 2023-07-14T16:32:01.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.924+0000 7f6d20321b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd99a600 con 0x55eecd98c4d0 2023-07-14T16:32:01.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.924+0000 7f6d20321b80 10 client.0 _ll_register_callbacks cb 0x55eecd87b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:01.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.924+0000 7f6d06d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd999c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.924+0000 7f6d06d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd999c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58912/0 (socket says 172.21.15.62:58912) 2023-07-14T16:32:01.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.924+0000 7f6d06d9d700 1 -- 172.21.15.62:0/1119956063 learned_addr learned my addr 172.21.15.62:0/1119956063 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:01.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd99d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6d05d9b700 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98ce20 0x55eecd99d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6d06d9d700 1 -- 172.21.15.62:0/1119956063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 msgr2=0x55eecd99d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:01.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd99d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.901 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6d06d9d700 1 -- 172.21.15.62:0/1119956063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98ce20 msgr2=0x55eecd99d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98ce20 0x55eecd99d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6d06d9d700 1 -- 172.21.15.62:0/1119956063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eecd6f4c60 con 0x55eecd98dc70 2023-07-14T16:32:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6d06d9d700 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd999c20 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6d00006000 tx=0x7f6d0000a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367caceb0c86f81e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.925+0000 7f6cedffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.926+0000 7f6cedffb700 1 -- 172.21.15.62:0/1119956063 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00023e50 con 0x55eecd98dc70 2023-07-14T16:32:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.926+0000 7f6cedffb700 1 -- 172.21.15.62:0/1119956063 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d00006e40 con 0x55eecd98dc70 2023-07-14T16:32:01.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.926+0000 7f6d20321b80 1 -- 172.21.15.62:0/1119956063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d000238d0 con 0x55eecd98dc70 2023-07-14T16:32:01.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.926+0000 7f6cedffb700 1 -- 172.21.15.62:0/1119956063 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d00009b50 con 0x55eecd98dc70 2023-07-14T16:32:01.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.926+0000 7f6d20321b80 10 client.20106 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:01.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.926+0000 7f6d20321b80 1 -- 172.21.15.62:0/1119956063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55eecd70a2b0 con 0x55eecd98dc70 2023-07-14T16:32:01.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.927+0000 7f6cedffb700 1 -- 172.21.15.62:0/1119956063 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d000238d0 con 0x55eecd98dc70 2023-07-14T16:32:01.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.927+0000 7f6cedffb700 1 -- 172.21.15.62:0/1119956063 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d0002d040 con 0x55eecd98dc70 2023-07-14T16:32:01.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.927+0000 7f6d20321b80 10 client.20106 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:01.903 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.927+0000 7f6d20321b80 10 client.20106 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:32:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.927+0000 7f6d20321b80 20 client.20106 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.927+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55eecd9f6fd0 0x55eecd9f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.927+0000 7f6d20321b80 4 client.20106 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:32:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.927+0000 7f6d20321b80 1 -- 172.21.15.62:0/1119956063 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eecd899050 con 0x55eecd9f6fd0 2023-07-14T16:32:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.928+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55eecd9f6fd0 0x55eecd9f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.928+0000 7f6d0659c700 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55eecd9f6fd0 0x55eecd9f93e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1dc05a8c87f07248 server_cookie=3c1230ddac447b17 in_seq=0 out_seq=0 2023-07-14T16:32:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.928+0000 7f6cedffb700 10 client.20106 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:32:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6cedffb700 1 -- 172.21.15.62:0/1119956063 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eecd899050 con 0x55eecd9f6fd0 2023-07-14T16:32:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6cedffb700 10 client.20106 handle_command_reply: tid=0 2023-07-14T16:32:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 client.20106 shutdown 2023-07-14T16:32:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 -- 172.21.15.62:0/1119956063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 msgr2=0x55eecd999c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd999c20 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6d00006000 tx=0x7f6d0000a460 comp rx=0 tx=0).stop 2023-07-14T16:32:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 -- 172.21.15.62:0/1119956063 shutdown_connections 2023-07-14T16:32:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55eecd9f6fd0 0x55eecd9f93e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eecd98dc70 0x55eecd999c20 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eecd98c4d0 0x55eecd99d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 --2- 172.21.15.62:0/1119956063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55eecd98ce20 0x55eecd99d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6cedffb700 0 client.20106 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:32:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 -- 172.21.15.62:0/1119956063 >> 172.21.15.62:0/1119956063 conn(0x55eecd981da0 msgr2=0x55eecd993fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.929+0000 7f6d20321b80 1 -- 172.21.15.62:0/1119956063 shutdown_connections 2023-07-14T16:32:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.930+0000 7f6d20321b80 1 -- 172.21.15.62:0/1119956063 wait complete. 2023-07-14T16:32:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.930+0000 7f6d20321b80 20 client.20106 trim_cache size 0 max 16384 2023-07-14T16:32:01.909 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:32:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6cf00a11d0 msgr2=0x7f6cf00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6cf00a11d0 0x7f6cf00a3680 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6d00020000 tx=0x7f6d00021000 comp rx=0 tx=0).stop 2023-07-14T16:32:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/4053404184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d080716b0 msgr2=0x7f6d0810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d080716b0 0x7f6d0810e1a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005910 tx=0x7f6cf8005720 comp rx=0 tx=0).stop 2023-07-14T16:32:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/4053404184 shutdown_connections 2023-07-14T16:32:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d081080d0 0x7f6d0810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f6cf00a11d0 0x7f6cf00a3680 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d080716b0 0x7f6d0810e1a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 --2- 172.21.15.62:0/4053404184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d08107780 0x7f6d0810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.931+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/4053404184 >> 172.21.15.62:0/4053404184 conn(0x7f6d0806cd80 msgr2=0x7f6d0806f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.932+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/4053404184 shutdown_connections 2023-07-14T16:32:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:01.932+0000 7f6d0c9ff700 1 -- 172.21.15.62:0/4053404184 wait complete. 2023-07-14T16:32:01.935 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:32:02.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.306+0000 7f650db9d700 1 -- 172.21.15.62:0/1754817075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508101c00 msgr2=0x7f6508102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.306+0000 7f650db9d700 1 --2- 172.21.15.62:0/1754817075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508101c00 0x7f6508102010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f65040098f0 tx=0x7f6504009c00 comp rx=0 tx=0).stop 2023-07-14T16:32:02.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.307+0000 7f650db9d700 1 -- 172.21.15.62:0/1754817075 shutdown_connections 2023-07-14T16:32:02.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.307+0000 7f650db9d700 1 --2- 172.21.15.62:0/1754817075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508103780 0x7f650810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.307+0000 7f650db9d700 1 --2- 172.21.15.62:0/1754817075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508101c00 0x7f6508102010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.307+0000 7f650db9d700 1 --2- 172.21.15.62:0/1754817075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6508102df0 0x7f6508103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.307+0000 7f650db9d700 1 -- 172.21.15.62:0/1754817075 >> 172.21.15.62:0/1754817075 conn(0x7f65080fd220 msgr2=0x7f65080ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.307+0000 7f650db9d700 1 -- 172.21.15.62:0/1754817075 shutdown_connections 2023-07-14T16:32:02.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.307+0000 7f650db9d700 1 -- 172.21.15.62:0/1754817075 wait complete. 2023-07-14T16:32:02.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.308+0000 7f650db9d700 1 Processor -- start 2023-07-14T16:32:02.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.309+0000 7f650db9d700 1 -- start start 2023-07-14T16:32:02.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.309+0000 7f650db9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6508101c00 0x7f6508197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.309+0000 7f650db9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508102df0 0x7f6508197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6508101c00 0x7f6508197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6508101c00 0x7f6508197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53820/0 (socket says 172.21.15.62:53820) 2023-07-14T16:32:02.284 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f64fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508102df0 0x7f6508197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f64fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508102df0 0x7f6508197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38918/0 (socket says 172.21.15.62:38918) 2023-07-14T16:32:02.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f650db9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508103780 0x7f650819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f650db9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819f2c0 con 0x7f6508103780 2023-07-14T16:32:02.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f650db9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819f400 con 0x7f6508101c00 2023-07-14T16:32:02.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f650db9d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819f590 con 0x7f6508102df0 2023-07-14T16:32:02.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.310+0000 7f650cb9b700 1 -- 172.21.15.62:0/3897873639 learned_addr learned my addr 172.21.15.62:0/3897873639 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.311+0000 7f650d39c700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508103780 0x7f650819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.311+0000 7f650cb9b700 1 -- 172.21.15.62:0/3897873639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508102df0 msgr2=0x7f6508197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.287 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.311+0000 7f650cb9b700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508102df0 0x7f6508197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.311+0000 7f650cb9b700 1 -- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508103780 msgr2=0x7f650819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.311+0000 7f650cb9b700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508103780 0x7f650819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.311+0000 7f650cb9b700 1 -- 172.21.15.62:0/3897873639 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f8009c40 con 0x7f6508101c00 2023-07-14T16:32:02.288 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.311+0000 7f64fffff700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508102df0 0x7f6508197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:32:02.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.311+0000 7f650cb9b700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6508101c00 0x7f6508197740 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6504009fd0 tx=0x7f650400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.312+0000 7f64fdffb700 1 -- 172.21.15.62:0/3897873639 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650401c070 con 0x7f6508101c00 2023-07-14T16:32:02.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.312+0000 7f64fdffb700 1 -- 172.21.15.62:0/3897873639 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f650400f7b0 con 0x7f6508101c00 2023-07-14T16:32:02.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.312+0000 7f650db9d700 1 -- 172.21.15.62:0/3897873639 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6504009520 con 0x7f6508101c00 2023-07-14T16:32:02.289 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.312+0000 7f64fdffb700 1 -- 172.21.15.62:0/3897873639 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504003a00 con 0x7f6508101c00 2023-07-14T16:32:02.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.312+0000 7f650db9d700 1 -- 172.21.15.62:0/3897873639 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650819fb00 con 0x7f6508101c00 2023-07-14T16:32:02.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.319+0000 7f64fdffb700 1 -- 172.21.15.62:0/3897873639 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6504020030 con 0x7f6508101c00 2023-07-14T16:32:02.294 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.320+0000 7f65226a2b80 1 Processor -- start 2023-07-14T16:32:02.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.322+0000 7f65226a2b80 1 -- start start 2023-07-14T16:32:02.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.322+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f5720 0x5589840f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.322+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.322+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589840f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.322+0000 7f65226a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558983ffacb0 con 0x5589840f5720 2023-07-14T16:32:02.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.322+0000 7f65226a2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898404da40 con 0x5589840f6070 2023-07-14T16:32:02.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.322+0000 7f65226a2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558984046ed0 con 0x5589840f6e30 2023-07-14T16:32:02.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.323+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.323+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53836/0 (socket says 172.21.15.62:53836) 2023-07-14T16:32:02.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.323+0000 7f650cb9b700 1 -- 172.21.15.62:0/2995253883 learned_addr learned my addr 172.21.15.62:0/2995253883 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.323+0000 7f64fffff700 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589840f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.323+0000 7f650d39c700 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f5720 0x5589840f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.324+0000 7f650cb9b700 1 -- 172.21.15.62:0/2995253883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 msgr2=0x5589840f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.324+0000 7f650cb9b700 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589840f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.324+0000 7f650cb9b700 1 -- 172.21.15.62:0/2995253883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f5720 msgr2=0x5589840f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.324+0000 7f650cb9b700 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f5720 0x5589840f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.324+0000 7f650cb9b700 1 -- 172.21.15.62:0/2995253883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558983e74de0 con 0x5589840f6070 2023-07-14T16:32:02.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.324+0000 7f64fdffb700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64f00a9a80 0x7f64f00abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.324+0000 7f650d39c700 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f5720 0x5589840f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:02.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.324+0000 7f64fdffb700 1 -- 172.21.15.62:0/3897873639 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65040cd990 con 0x7f6508101c00 2023-07-14T16:32:02.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.325+0000 7f650cb9b700 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840f68a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65040d3a90 tx=0x7f65040d3ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90d597972a5276c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.325+0000 7f650d39c700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64f00a9a80 0x7f64f00abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.325+0000 7f64eeffd700 1 -- 172.21.15.62:0/2995253883 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400ba30 con 0x5589840f6070 2023-07-14T16:32:02.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.325+0000 7f64eeffd700 1 -- 172.21.15.62:0/2995253883 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65040d36d0 con 0x5589840f6070 2023-07-14T16:32:02.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.325+0000 7f64eeffd700 1 -- 172.21.15.62:0/2995253883 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650409a920 con 0x5589840f6070 2023-07-14T16:32:02.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f65226a2b80 1 -- 172.21.15.62:0/2995253883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 msgr2=0x5589840f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840f68a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65040d3a90 tx=0x7f65040d3ac0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f650d39c700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64f00a9a80 0x7f64f00abf30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6500005fd0 tx=0x7f6500009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f65226a2b80 1 -- 172.21.15.62:0/2995253883 shutdown_connections 2023-07-14T16:32:02.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589840f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f5720 0x5589840f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2995253883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840f68a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f65226a2b80 1 -- 172.21.15.62:0/2995253883 >> 172.21.15.62:0/2995253883 conn(0x5589840eaff0 msgr2=0x5589840f0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.326+0000 7f65226a2b80 1 -- 172.21.15.62:0/2995253883 shutdown_connections 2023-07-14T16:32:02.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.327+0000 7f65226a2b80 1 -- 172.21.15.62:0/2995253883 wait complete. 2023-07-14T16:32:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.327+0000 7f65226a2b80 1 -- start start 2023-07-14T16:32:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.327+0000 7f65226a2b80 1 Processor -- start 2023-07-14T16:32:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.328+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f5720 0x5589840fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.328+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.328+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 0x5589841064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.328+0000 7f65226a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558983fe6ac0 con 0x5589840f6e30 2023-07-14T16:32:02.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.328+0000 7f65226a2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558983fe67c0 con 0x5589840f6070 2023-07-14T16:32:02.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.328+0000 7f65226a2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558983eca920 con 0x5589840f5720 2023-07-14T16:32:02.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.328+0000 7f65226a2b80 10 client.0 _ll_register_callbacks cb 0x558983fe4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:02.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.329+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f5720 0x5589840fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.329+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f5720 0x5589840fef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38926/0 (socket says 172.21.15.62:38926) 2023-07-14T16:32:02.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.329+0000 7f650cb9b700 1 -- 172.21.15.62:0/66361771 learned_addr learned my addr 172.21.15.62:0/66361771 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.329+0000 7f64fffff700 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.330+0000 7f650cb9b700 1 -- 172.21.15.62:0/66361771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 msgr2=0x5589840ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.330+0000 7f650cb9b700 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.330+0000 7f650cb9b700 1 -- 172.21.15.62:0/66361771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 msgr2=0x5589841064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.330+0000 7f650cb9b700 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 0x5589841064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.330+0000 7f650cb9b700 1 -- 172.21.15.62:0/66361771 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650819fb00 con 0x5589840f5720 2023-07-14T16:32:02.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.330+0000 7f650cb9b700 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f5720 0x5589840fef60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6504097290 tx=0x7f6504005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e772cf04a737684 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.330+0000 7f64e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:32:02.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.331+0000 7f64e7fff700 1 -- 172.21.15.62:0/66361771 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400ba30 con 0x5589840f5720 2023-07-14T16:32:02.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.331+0000 7f64e7fff700 1 -- 172.21.15.62:0/66361771 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65040d0e40 con 0x5589840f5720 2023-07-14T16:32:02.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.331+0000 7f65226a2b80 1 -- 172.21.15.62:0/66361771 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55898403feb0 con 0x5589840f5720 2023-07-14T16:32:02.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.331+0000 7f64e7fff700 1 -- 172.21.15.62:0/66361771 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65040d36d0 con 0x5589840f5720 2023-07-14T16:32:02.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.331+0000 7f65226a2b80 10 client.29506 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:02.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.331+0000 7f65226a2b80 1 -- 172.21.15.62:0/66361771 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558983e5dc60 con 0x5589840f5720 2023-07-14T16:32:02.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.332+0000 7f64e7fff700 1 -- 172.21.15.62:0/66361771 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55898403feb0 con 0x5589840f5720 2023-07-14T16:32:02.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.332+0000 7f64e7fff700 1 -- 172.21.15.62:0/66361771 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6504003c80 con 0x5589840f5720 2023-07-14T16:32:02.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.332+0000 7f65226a2b80 10 client.29506 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:02.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.332+0000 7f65226a2b80 10 client.29506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:32:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.332+0000 7f65226a2b80 20 client.29506 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.332+0000 7f65226a2b80 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55898411b820 0x55898411dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.332+0000 7f65226a2b80 4 client.29506 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:02.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.332+0000 7f65226a2b80 1 -- 172.21.15.62:0/66361771 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558984002050 con 0x55898411b820 2023-07-14T16:32:02.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.333+0000 7f64fffff700 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55898411b820 0x55898411dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.334+0000 7f64fffff700 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55898411b820 0x55898411dc30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cab0ffbc2a0a80be server_cookie=f3e3e3739d47a631 in_seq=0 out_seq=0 2023-07-14T16:32:02.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.334+0000 7f64e7fff700 10 client.29506 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:32:02.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f64e7fff700 1 -- 172.21.15.62:0/66361771 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558984002050 con 0x55898411b820 2023-07-14T16:32:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f64e7fff700 10 client.29506 handle_command_reply: tid=0 2023-07-14T16:32:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 client.29506 shutdown 2023-07-14T16:32:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 -- 172.21.15.62:0/66361771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f5720 msgr2=0x5589840fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f5720 0x5589840fef60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6504097290 tx=0x7f6504005bc0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 -- 172.21.15.62:0/66361771 shutdown_connections 2023-07-14T16:32:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55898411b820 0x55898411dc30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 0x5589841064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f5720 0x5589840fef60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 --2- 172.21.15.62:0/66361771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f6070 0x5589840ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f64e7fff700 0 client.29506 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:32:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.335+0000 7f65226a2b80 1 -- 172.21.15.62:0/66361771 >> 172.21.15.62:0/66361771 conn(0x5589840eaff0 msgr2=0x5589840efb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.336+0000 7f65226a2b80 1 -- 172.21.15.62:0/66361771 shutdown_connections 2023-07-14T16:32:02.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.336+0000 7f65226a2b80 1 -- 172.21.15.62:0/66361771 wait complete. 2023-07-14T16:32:02.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.336+0000 7f65226a2b80 20 client.29506 trim_cache size 0 max 16384 2023-07-14T16:32:02.331 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.358+0000 7f65226a2b80 1 Processor -- start 2023-07-14T16:32:02.332 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.358+0000 7f65226a2b80 1 -- start start 2023-07-14T16:32:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.358+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x5589840efb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.358+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6070 0x5589841064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.358+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 0x5589840fd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.358+0000 7f65226a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898403fee0 con 0x5589840f6e30 2023-07-14T16:32:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.358+0000 7f65226a2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558984100180 con 0x5589840f5720 2023-07-14T16:32:02.333 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.358+0000 7f65226a2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589840fcf30 con 0x5589840f6070 2023-07-14T16:32:02.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.359+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x5589840efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.359+0000 7f650cb9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x5589840efb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53862/0 (socket says 172.21.15.62:53862) 2023-07-14T16:32:02.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.359+0000 7f650cb9b700 1 -- 172.21.15.62:0/3177148200 learned_addr learned my addr 172.21.15.62:0/3177148200 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.361+0000 7f64fffff700 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6070 0x5589841064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.361+0000 7f650d39c700 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 0x5589840fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f650cb9b700 1 -- 172.21.15.62:0/3177148200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6070 msgr2=0x5589841064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f650cb9b700 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6070 0x5589841064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f650cb9b700 1 -- 172.21.15.62:0/3177148200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 msgr2=0x5589840fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f650cb9b700 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 0x5589840fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f650cb9b700 1 -- 172.21.15.62:0/3177148200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558983e9b250 con 0x5589840f5720 2023-07-14T16:32:02.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f64fffff700 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6070 0x5589841064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:32:02.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f650d39c700 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 0x5589840fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:02.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f650cb9b700 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x5589840efb00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f65040972e0 tx=0x7f65040cdc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1e5ab8d536f8e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f64ef7fe700 1 -- 172.21.15.62:0/3177148200 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400ba30 con 0x5589840f5720 2023-07-14T16:32:02.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f64ef7fe700 1 -- 172.21.15.62:0/3177148200 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6504097120 con 0x5589840f5720 2023-07-14T16:32:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.362+0000 7f64ef7fe700 1 -- 172.21.15.62:0/3177148200 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650409a740 con 0x5589840f5720 2023-07-14T16:32:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.363+0000 7f65226a2b80 1 -- 172.21.15.62:0/3177148200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 msgr2=0x5589840efb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.363+0000 7f65226a2b80 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x5589840efb00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f65040972e0 tx=0x7f65040cdc80 comp rx=0 tx=0).stop 2023-07-14T16:32:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.364+0000 7f65226a2b80 1 -- 172.21.15.62:0/3177148200 shutdown_connections 2023-07-14T16:32:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.364+0000 7f65226a2b80 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6e30 0x5589840fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.364+0000 7f65226a2b80 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x5589840efb00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.338 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.364+0000 7f65226a2b80 1 --2- 172.21.15.62:0/3177148200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6070 0x5589841064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.364+0000 7f65226a2b80 1 -- 172.21.15.62:0/3177148200 >> 172.21.15.62:0/3177148200 conn(0x5589840eaff0 msgr2=0x558984122590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.339 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.364+0000 7f65226a2b80 1 -- 172.21.15.62:0/3177148200 shutdown_connections 2023-07-14T16:32:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.364+0000 7f65226a2b80 1 -- 172.21.15.62:0/3177148200 wait complete. 2023-07-14T16:32:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.365+0000 7f65226a2b80 1 -- start start 2023-07-14T16:32:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.365+0000 7f65226a2b80 1 Processor -- start 2023-07-14T16:32:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x558984105d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6070 0x558984106250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f65226a2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589841027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.340 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f65226a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558984102d80 con 0x5589840f6070 2023-07-14T16:32:02.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f65226a2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558984102ef0 con 0x5589840f5720 2023-07-14T16:32:02.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f65226a2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589841030b0 con 0x5589840f6e30 2023-07-14T16:32:02.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f65226a2b80 10 client.0 _ll_register_callbacks cb 0x7f65040d38e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f650d39c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589841027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f650d39c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589841027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38944/0 (socket says 172.21.15.62:38944) 2023-07-14T16:32:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.366+0000 7f650d39c700 1 -- 172.21.15.62:0/2266784742 learned_addr learned my addr 172.21.15.62:0/2266784742 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f650cb9b700 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x558984105d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f64fffff700 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6070 0x558984106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f650cb9b700 1 -- 172.21.15.62:0/2266784742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 msgr2=0x5589841027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f650cb9b700 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589841027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f650cb9b700 1 -- 172.21.15.62:0/2266784742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6070 msgr2=0x558984106250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f650cb9b700 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6070 0x558984106250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f650cb9b700 1 -- 172.21.15.62:0/2266784742 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558983e5dc60 con 0x5589840f5720 2023-07-14T16:32:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f650cb9b700 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x558984105d10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6504097290 tx=0x7f650409ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c3b93f665f5479 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.367+0000 7f64e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:02.343 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.368+0000 7f64e7fff700 1 -- 172.21.15.62:0/2266784742 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040cb3f0 con 0x5589840f5720 2023-07-14T16:32:02.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.368+0000 7f64e7fff700 1 -- 172.21.15.62:0/2266784742 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65040d0e40 con 0x5589840f5720 2023-07-14T16:32:02.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.368+0000 7f65226a2b80 1 -- 172.21.15.62:0/2266784742 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558983fa04f0 con 0x5589840f5720 2023-07-14T16:32:02.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.368+0000 7f64e7fff700 1 -- 172.21.15.62:0/2266784742 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650400ba30 con 0x5589840f5720 2023-07-14T16:32:02.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.368+0000 7f65226a2b80 10 client.29642 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:02.344 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.368+0000 7f65226a2b80 1 -- 172.21.15.62:0/2266784742 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558983e732b0 con 0x5589840f5720 2023-07-14T16:32:02.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f64e7fff700 1 -- 172.21.15.62:0/2266784742 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558983fa04f0 con 0x5589840f5720 2023-07-14T16:32:02.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f64e7fff700 1 -- 172.21.15.62:0/2266784742 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f650409a2d0 con 0x5589840f5720 2023-07-14T16:32:02.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f65226a2b80 10 client.29642 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:02.345 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f65226a2b80 10 client.29642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:32:02.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f65226a2b80 20 client.29642 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:02.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5589841703d0 0x5589841727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.346 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f65226a2b80 4 client.29642 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:32:02.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f65226a2b80 1 -- 172.21.15.62:0/2266784742 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558983e95f70 con 0x5589841703d0 2023-07-14T16:32:02.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.369+0000 7f64fffff700 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5589841703d0 0x5589841727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.371+0000 7f64fffff700 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5589841703d0 0x5589841727b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5af9bb75971ef607 server_cookie=4cc901518dc86b6b in_seq=0 out_seq=0 2023-07-14T16:32:02.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.371+0000 7f64e7fff700 10 client.29642 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:32:02.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.371+0000 7f64e7fff700 1 -- 172.21.15.62:0/2266784742 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558983e95f70 con 0x5589841703d0 2023-07-14T16:32:02.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.371+0000 7f64e7fff700 10 client.29642 handle_command_reply: tid=0 2023-07-14T16:32:02.347 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.371+0000 7f65226a2b80 1 client.29642 shutdown 2023-07-14T16:32:02.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f65226a2b80 1 -- 172.21.15.62:0/2266784742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 msgr2=0x558984105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x558984105d10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6504097290 tx=0x7f650409ace0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.348 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f65226a2b80 1 -- 172.21.15.62:0/2266784742 shutdown_connections 2023-07-14T16:32:02.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5589841703d0 0x5589841727b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589840f6e30 0x5589841027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5589840f5720 0x558984105d10 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6504097290 tx=0x7f650409ace0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f65226a2b80 1 --2- 172.21.15.62:0/2266784742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5589840f6070 0x558984106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f65226a2b80 1 -- 172.21.15.62:0/2266784742 >> 172.21.15.62:0/2266784742 conn(0x5589840eaff0 msgr2=0x5589840efb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.372+0000 7f64e7fff700 0 client.29642 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:32:02.349 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.373+0000 7f65226a2b80 1 -- 172.21.15.62:0/2266784742 shutdown_connections 2023-07-14T16:32:02.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.373+0000 7f65226a2b80 1 -- 172.21.15.62:0/2266784742 wait complete. 2023-07-14T16:32:02.350 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.373+0000 7f65226a2b80 20 client.29642 trim_cache size 0 max 16384 2023-07-14T16:32:02.350 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:32:02.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 -- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64f00a9a80 msgr2=0x7f64f00abf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64f00a9a80 0x7f64f00abf30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6500005fd0 tx=0x7f6500009310 comp rx=0 tx=0).stop 2023-07-14T16:32:02.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 -- 172.21.15.62:0/3897873639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6508101c00 msgr2=0x7f6508197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6508101c00 0x7f6508197740 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6504009fd0 tx=0x7f650400b790 comp rx=0 tx=0).stop 2023-07-14T16:32:02.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 -- 172.21.15.62:0/3897873639 shutdown_connections 2023-07-14T16:32:02.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6508103780 0x7f650819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.351 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f64f00a9a80 0x7f64f00abf30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6508101c00 0x7f6508197740 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 --2- 172.21.15.62:0/3897873639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6508102df0 0x7f6508197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.374+0000 7f650db9d700 1 -- 172.21.15.62:0/3897873639 >> 172.21.15.62:0/3897873639 conn(0x7f65080fd220 msgr2=0x7f65080ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.375+0000 7f650db9d700 1 -- 172.21.15.62:0/3897873639 shutdown_connections 2023-07-14T16:32:02.352 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.375+0000 7f650db9d700 1 -- 172.21.15.62:0/3897873639 wait complete. 2023-07-14T16:32:02.377 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:32:02.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.743+0000 7fe7e4fff700 1 -- 172.21.15.62:0/2891961454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7e0071830 msgr2=0x7fe7e010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.743+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/2891961454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7e0071830 0x7fe7e010b7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00098f0 tx=0x7fe7d0009c00 comp rx=0 tx=0).stop 2023-07-14T16:32:02.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.745+0000 7fe7e4fff700 1 -- 172.21.15.62:0/2891961454 shutdown_connections 2023-07-14T16:32:02.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.745+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/2891961454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e010bd60 0x7fe7e010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.745+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/2891961454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7e0070ee0 0x7fe7e00712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.745+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/2891961454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7e0071830 0x7fe7e010b7d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.745+0000 7fe7e4fff700 1 -- 172.21.15.62:0/2891961454 >> 172.21.15.62:0/2891961454 conn(0x7fe7e006ca90 msgr2=0x7fe7e006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.745+0000 7fe7e4fff700 1 -- 172.21.15.62:0/2891961454 shutdown_connections 2023-07-14T16:32:02.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.745+0000 7fe7e4fff700 1 -- 172.21.15.62:0/2891961454 wait complete. 2023-07-14T16:32:02.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.747+0000 7fe7e4fff700 1 Processor -- start 2023-07-14T16:32:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.747+0000 7fe7e4fff700 1 -- start start 2023-07-14T16:32:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.747+0000 7fe7e4fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0070ee0 0x7fe7e0113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.747+0000 7fe7e4fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7e0071830 0x7fe7e01140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.748+0000 7fe7e4fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7e010bd60 0x7fe7e01b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.748+0000 7fe7e4fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0114680 con 0x7fe7e0070ee0 2023-07-14T16:32:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.748+0000 7fe7e4fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01147f0 con 0x7fe7e0071830 2023-07-14T16:32:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.748+0000 7fe7e4fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01b4c90 con 0x7fe7e010bd60 2023-07-14T16:32:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.748+0000 7fe7ddd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7e0071830 0x7fe7e01140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.748+0000 7fe7ddd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7e0071830 0x7fe7e01140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53890/0 (socket says 172.21.15.62:53890) 2023-07-14T16:32:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.748+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/4157634661 learned_addr learned my addr 172.21.15.62:0/4157634661 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.748+0000 7fe7de59c700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0070ee0 0x7fe7e0113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.749+0000 7fe7ded9d700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7e010bd60 0x7fe7e01b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.749+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/4157634661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7e010bd60 msgr2=0x7fe7e01b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.749+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7e010bd60 0x7fe7e01b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.749+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0070ee0 msgr2=0x7fe7e0113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.749+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0070ee0 0x7fe7e0113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.749+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/4157634661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d0009520 con 0x7fe7e0071830 2023-07-14T16:32:02.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.749+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7e0071830 0x7fe7e01140b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000fa00 tx=0x7fe7d000b6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.750+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/4157634661 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d001c070 con 0x7fe7e0071830 2023-07-14T16:32:02.725 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.750+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/4157634661 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7d000bbb0 con 0x7fe7e0071830 2023-07-14T16:32:02.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.750+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/4157634661 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0004e20 con 0x7fe7e0071830 2023-07-14T16:32:02.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.751+0000 7fe7e4fff700 1 -- 172.21.15.62:0/4157634661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e01b4ee0 con 0x7fe7e0071830 2023-07-14T16:32:02.726 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.751+0000 7fe7e4fff700 1 -- 172.21.15.62:0/4157634661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e01b5250 con 0x7fe7e0071830 2023-07-14T16:32:02.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.757+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/4157634661 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe7d0005030 con 0x7fe7e0071830 2023-07-14T16:32:02.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.758+0000 7fe7f8829b80 1 Processor -- start 2023-07-14T16:32:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.761+0000 7fe7cf7fe700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7c80a12a0 0x7fe7c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.761+0000 7fe7cf7fe700 1 -- 172.21.15.62:0/4157634661 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe7d00cdc30 con 0x7fe7e0071830 2023-07-14T16:32:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.762+0000 7fe7f8829b80 1 -- start start 2023-07-14T16:32:02.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.762+0000 7fe7ded9d700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7c80a12a0 0x7fe7c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.762+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x5636147fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.762+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fce20 0x5636147fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.762+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fdc70 0x5636147fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.762+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563614701cb0 con 0x5636147fdc70 2023-07-14T16:32:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.762+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563614754a40 con 0x5636147fc4d0 2023-07-14T16:32:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.762+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361474ded0 con 0x5636147fce20 2023-07-14T16:32:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.763+0000 7fe7ddd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fce20 0x5636147fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.763+0000 7fe7ddd9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fce20 0x5636147fd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38968/0 (socket says 172.21.15.62:38968) 2023-07-14T16:32:02.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.763+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/2527612354 learned_addr learned my addr 172.21.15.62:0/2527612354 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.763+0000 7fe7de59c700 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x5636147fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.763+0000 7fe7ded9d700 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fdc70 0x5636147fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.766+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/2527612354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 msgr2=0x5636147fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.766+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x5636147fc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.766+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/2527612354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fdc70 msgr2=0x5636147fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.766+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fdc70 0x5636147fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.766+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/2527612354 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361457bde0 con 0x5636147fce20 2023-07-14T16:32:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.766+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fce20 0x5636147fd6e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00a2f70 tx=0x7fe7d0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a705bd9a37e8ba server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.767+0000 7fe7ccff9700 1 -- 172.21.15.62:0/2527612354 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00a34a0 con 0x5636147fce20 2023-07-14T16:32:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.767+0000 7fe7ccff9700 1 -- 172.21.15.62:0/2527612354 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7d00046d0 con 0x5636147fce20 2023-07-14T16:32:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.767+0000 7fe7ccff9700 1 -- 172.21.15.62:0/2527612354 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d009a5d0 con 0x5636147fce20 2023-07-14T16:32:02.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.768+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2527612354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fce20 msgr2=0x5636147fd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.768+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fce20 0x5636147fd6e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00a2f70 tx=0x7fe7d0009c00 comp rx=0 tx=0).stop 2023-07-14T16:32:02.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.768+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2527612354 shutdown_connections 2023-07-14T16:32:02.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.768+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fdc70 0x5636147fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.768+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x5636147fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.768+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/2527612354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fce20 0x5636147fd6e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.768+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2527612354 >> 172.21.15.62:0/2527612354 conn(0x5636147f1da0 msgr2=0x5636147f71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.769+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2527612354 shutdown_connections 2023-07-14T16:32:02.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.769+0000 7fe7ded9d700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7c80a12a0 0x7fe7c80a3750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005fd0 tx=0x7fe7d40099c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.769+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2527612354 wait complete. 2023-07-14T16:32:02.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.770+0000 7fe7f8829b80 1 -- start start 2023-07-14T16:32:02.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.770+0000 7fe7f8829b80 1 Processor -- start 2023-07-14T16:32:02.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 0x563614805db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x5636148062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636146edac0 con 0x5636147fc4d0 2023-07-14T16:32:02.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636146ed7c0 con 0x5636147fce20 2023-07-14T16:32:02.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636145d1920 con 0x5636147fdc70 2023-07-14T16:32:02.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7f8829b80 10 client.0 _ll_register_callbacks cb 0x5636146eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:02.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7ddd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x5636148062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7ddd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x5636148062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57672/0 (socket says 172.21.15.62:57672) 2023-07-14T16:32:02.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/1277075133 learned_addr learned my addr 172.21.15.62:0/1277075133 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.771+0000 7fe7de59c700 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 0x563614805db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.772+0000 7fe7ded9d700 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.772+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/1277075133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 msgr2=0x56361480d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.772+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.772+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/1277075133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 msgr2=0x563614805db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.772+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 0x563614805db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.772+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/1277075133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e01b5250 con 0x5636147fce20 2023-07-14T16:32:02.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.774+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x5636148062f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000fa30 tx=0x7fe7d00977f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51111f2cb1e8221d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.776+0000 7fe7c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.776+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1277075133 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d009a7d0 con 0x5636147fce20 2023-07-14T16:32:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.776+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1277075133 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe7d00a3010 con 0x5636147fce20 2023-07-14T16:32:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.776+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1277075133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636146a74f0 con 0x5636147fce20 2023-07-14T16:32:02.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.776+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1277075133 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d0099540 con 0x5636147fce20 2023-07-14T16:32:02.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.776+0000 7fe7f8829b80 10 client.29654 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:02.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.776+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1277075133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563614564c60 con 0x5636147fce20 2023-07-14T16:32:02.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.777+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1277075133 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7d00c1020 con 0x5636147fce20 2023-07-14T16:32:02.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.777+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1277075133 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe7d00052e0 con 0x5636147fce20 2023-07-14T16:32:02.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.777+0000 7fe7f8829b80 10 client.29654 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:02.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.777+0000 7fe7f8829b80 10 client.29654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:02.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.777+0000 7fe7f8829b80 20 client.29654 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:02.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.777+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563614811610 0x563614813a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.777+0000 7fe7f8829b80 4 client.29654 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:02.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.777+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1277075133 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563614746eb0 con 0x563614811610 2023-07-14T16:32:02.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.778+0000 7fe7de59c700 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563614811610 0x563614813a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.779+0000 7fe7de59c700 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563614811610 0x563614813a20 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e80faeb31ee70a server_cookie=71ae3707028639e0 in_seq=0 out_seq=0 2023-07-14T16:32:02.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.780+0000 7fe7c5ffb700 10 client.29654 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:02.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.780+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1277075133 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563614746eb0 con 0x563614811610 2023-07-14T16:32:02.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.780+0000 7fe7c5ffb700 10 client.29654 handle_command_reply: tid=0 2023-07-14T16:32:02.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.780+0000 7fe7f8829b80 1 client.29654 shutdown 2023-07-14T16:32:02.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1277075133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 msgr2=0x5636148062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x5636148062f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000fa30 tx=0x7fe7d00977f0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1277075133 shutdown_connections 2023-07-14T16:32:02.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 0x563614805db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563614811610 0x563614813a20 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1277075133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x5636148062f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7c5ffb700 0 client.29654 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:02.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1277075133 >> 172.21.15.62:0/1277075133 conn(0x5636147f1da0 msgr2=0x5636147f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1277075133 shutdown_connections 2023-07-14T16:32:02.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.781+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1277075133 wait complete. 2023-07-14T16:32:02.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.782+0000 7fe7f8829b80 20 client.29654 trim_cache size 0 max 16384 2023-07-14T16:32:02.770 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:02 smithi164 ceph-mon[97427]: pgmap v849: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:32:02.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.809+0000 7fe7f8829b80 1 Processor -- start 2023-07-14T16:32:02.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.810+0000 7fe7f8829b80 1 -- start start 2023-07-14T16:32:02.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.810+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x563614804b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.810+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fce20 0x56361480d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.810+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.810+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563614805920 con 0x5636147fce20 2023-07-14T16:32:02.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.810+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636147f8070 con 0x5636147fc4d0 2023-07-14T16:32:02.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.810+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636147f8cc0 con 0x5636147fdc70 2023-07-14T16:32:02.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7de59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x563614804b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7de59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x563614804b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57676/0 (socket says 172.21.15.62:57676) 2023-07-14T16:32:02.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7de59c700 1 -- 172.21.15.62:0/2099996071 learned_addr learned my addr 172.21.15.62:0/2099996071 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fce20 0x56361480d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7ded9d700 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7de59c700 1 -- 172.21.15.62:0/2099996071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 msgr2=0x56361480dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7de59c700 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7de59c700 1 -- 172.21.15.62:0/2099996071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fce20 msgr2=0x56361480d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7de59c700 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fce20 0x56361480d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7de59c700 1 -- 172.21.15.62:0/2099996071 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636145a2250 con 0x5636147fc4d0 2023-07-14T16:32:02.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.811+0000 7fe7ded9d700 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:32:02.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.812+0000 7fe7de59c700 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x563614804b80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009610 tx=0x7fe7d80078a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256711c38de2d6ad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.812+0000 7fe7cd7fa700 1 -- 172.21.15.62:0/2099996071 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d801c070 con 0x5636147fc4d0 2023-07-14T16:32:02.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.812+0000 7fe7cd7fa700 1 -- 172.21.15.62:0/2099996071 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7d8009e90 con 0x5636147fc4d0 2023-07-14T16:32:02.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.812+0000 7fe7cd7fa700 1 -- 172.21.15.62:0/2099996071 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8004720 con 0x5636147fc4d0 2023-07-14T16:32:02.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2099996071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 msgr2=0x563614804b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x563614804b80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009610 tx=0x7fe7d80078a0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2099996071 shutdown_connections 2023-07-14T16:32:02.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fc4d0 0x563614804b80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/2099996071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fce20 0x56361480d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2099996071 >> 172.21.15.62:0/2099996071 conn(0x5636147f1da0 msgr2=0x563614818380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2099996071 shutdown_connections 2023-07-14T16:32:02.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.813+0000 7fe7f8829b80 1 -- 172.21.15.62:0/2099996071 wait complete. 2023-07-14T16:32:02.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.814+0000 7fe7f8829b80 1 -- start start 2023-07-14T16:32:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.814+0000 7fe7f8829b80 1 Processor -- start 2023-07-14T16:32:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.815+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 0x56361480d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.815+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x563614809c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.815+0000 7fe7f8829b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.815+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361480a8d0 con 0x5636147fc4d0 2023-07-14T16:32:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.815+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636148654f0 con 0x5636147fce20 2023-07-14T16:32:02.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.815+0000 7fe7f8829b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636148656b0 con 0x5636147fdc70 2023-07-14T16:32:02.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.815+0000 7fe7f8829b80 10 client.0 _ll_register_callbacks cb 0x7fe7d009ad10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:02.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ddd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x563614809c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ddd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x563614809c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57690/0 (socket says 172.21.15.62:57690) 2023-07-14T16:32:02.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/1174719988 learned_addr learned my addr 172.21.15.62:0/1174719988 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:02.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ded9d700 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/1174719988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 msgr2=0x56361480a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/1174719988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 msgr2=0x56361480d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:02.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 0x56361480d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.816+0000 7fe7ddd9b700 1 -- 172.21.15.62:0/1174719988 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d401b070 con 0x5636147fce20 2023-07-14T16:32:02.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.818+0000 7fe7ddd9b700 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x563614809c60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00cbab0 tx=0x7fe7d009aa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43fd21de2f921b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:02.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.818+0000 7fe7c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:02.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.818+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1174719988 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00a3520 con 0x5636147fce20 2023-07-14T16:32:02.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.818+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1174719988 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe7d00c2080 con 0x5636147fce20 2023-07-14T16:32:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.819+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1174719988 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7d00046d0 con 0x5636147fce20 2023-07-14T16:32:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.819+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1174719988 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d009b5c0 con 0x5636147fce20 2023-07-14T16:32:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.819+0000 7fe7f8829b80 10 client.29660 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.819+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1174719988 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56361457a2b0 con 0x5636147fce20 2023-07-14T16:32:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.819+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1174719988 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7d00046d0 con 0x5636147fce20 2023-07-14T16:32:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.819+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1174719988 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe7d009a5d0 con 0x5636147fce20 2023-07-14T16:32:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.820+0000 7fe7f8829b80 10 client.29660 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:02.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.820+0000 7fe7f8829b80 10 client.29660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.820+0000 7fe7f8829b80 20 client.29660 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.820+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563614866ba0 0x563614868fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.820+0000 7fe7f8829b80 4 client.29660 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:32:02.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.820+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1174719988 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563614709050 con 0x563614866ba0 2023-07-14T16:32:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.820+0000 7fe7de59c700 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563614866ba0 0x563614868fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.821+0000 7fe7de59c700 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563614866ba0 0x563614868fb0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989445c8fa946f27 server_cookie=a75752e58eac1fae in_seq=0 out_seq=0 2023-07-14T16:32:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.821+0000 7fe7c5ffb700 10 client.29660 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:02.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.822+0000 7fe7c5ffb700 1 -- 172.21.15.62:0/1174719988 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563614709050 con 0x563614866ba0 2023-07-14T16:32:02.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.822+0000 7fe7c5ffb700 10 client.29660 handle_command_reply: tid=0 2023-07-14T16:32:02.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.822+0000 7fe7f8829b80 1 client.29660 shutdown 2023-07-14T16:32:02.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1174719988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 msgr2=0x563614809c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x563614809c60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00cbab0 tx=0x7fe7d009aa40 comp rx=0 tx=0).stop 2023-07-14T16:32:02.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1174719988 shutdown_connections 2023-07-14T16:32:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636147fdc70 0x56361480a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636147fc4d0 0x56361480d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563614866ba0 0x563614868fb0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7f8829b80 1 --2- 172.21.15.62:0/1174719988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636147fce20 0x563614809c60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1174719988 >> 172.21.15.62:0/1174719988 conn(0x5636147f1da0 msgr2=0x563614804b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.824+0000 7fe7c5ffb700 0 client.29660 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.825+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1174719988 shutdown_connections 2023-07-14T16:32:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.825+0000 7fe7f8829b80 1 -- 172.21.15.62:0/1174719988 wait complete. 2023-07-14T16:32:02.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.825+0000 7fe7f8829b80 20 client.29660 trim_cache size 0 max 16384 2023-07-14T16:32:02.804 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:32:02.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.826+0000 7fe7e4fff700 1 -- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7c80a12a0 msgr2=0x7fe7c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.826+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7c80a12a0 0x7fe7c80a3750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005fd0 tx=0x7fe7d40099c0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.826+0000 7fe7e4fff700 1 -- 172.21.15.62:0/4157634661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7e0071830 msgr2=0x7fe7e01140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:02.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.826+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7e0071830 0x7fe7e01140b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000fa00 tx=0x7fe7d000b6c0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.827+0000 7fe7e4fff700 1 -- 172.21.15.62:0/4157634661 shutdown_connections 2023-07-14T16:32:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.827+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7e010bd60 0x7fe7e01b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.827+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0070ee0 0x7fe7e0113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.827+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe7c80a12a0 0x7fe7c80a3750 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.827+0000 7fe7e4fff700 1 --2- 172.21.15.62:0/4157634661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7e0071830 0x7fe7e01140b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.827+0000 7fe7e4fff700 1 -- 172.21.15.62:0/4157634661 >> 172.21.15.62:0/4157634661 conn(0x7fe7e006ca90 msgr2=0x7fe7e006de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.827+0000 7fe7e4fff700 1 -- 172.21.15.62:0/4157634661 shutdown_connections 2023-07-14T16:32:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:02.827+0000 7fe7e4fff700 1 -- 172.21.15.62:0/4157634661 wait complete. 2023-07-14T16:32:02.825 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:02 smithi062 ceph-mon[95543]: pgmap v849: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:32:02.898 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:02 smithi081 ceph-mon[127795]: pgmap v849: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 15 MiB/s wr, 31 op/s 2023-07-14T16:32:03.829 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:32:03.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:32:04.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.200+0000 7f19fa1ff700 1 -- 172.21.15.62:0/2428663530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f40ffae0 msgr2=0x7f19f40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.200+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/2428663530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f40ffae0 0x7f19f40ffef0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009990 tx=0x7f19e4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.201+0000 7f19fa1ff700 1 -- 172.21.15.62:0/2428663530 shutdown_connections 2023-07-14T16:32:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.201+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/2428663530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4101660 0x7f19f4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.201+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/2428663530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f40ffae0 0x7f19f40ffef0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.201+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/2428663530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f4100cd0 0x7f19f4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.201+0000 7f19fa1ff700 1 -- 172.21.15.62:0/2428663530 >> 172.21.15.62:0/2428663530 conn(0x7f19f40fb0e0 msgr2=0x7f19f40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.201+0000 7f19fa1ff700 1 -- 172.21.15.62:0/2428663530 shutdown_connections 2023-07-14T16:32:04.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.201+0000 7f19fa1ff700 1 -- 172.21.15.62:0/2428663530 wait complete. 2023-07-14T16:32:04.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.203+0000 7f19fa1ff700 1 Processor -- start 2023-07-14T16:32:04.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.203+0000 7f19fa1ff700 1 -- start start 2023-07-14T16:32:04.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19fa1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f40ffae0 0x7f19f406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19fa1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f4100cd0 0x7f19f406f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19fa1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4101660 0x7f19f41abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19fa1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4072370 con 0x7f19f40ffae0 2023-07-14T16:32:04.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19fa1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40724e0 con 0x7f19f4100cd0 2023-07-14T16:32:04.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19fa1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f41ac510 con 0x7f19f4101660 2023-07-14T16:32:04.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19f2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f4100cd0 0x7f19f406f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19f2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f4100cd0 0x7f19f406f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57710/0 (socket says 172.21.15.62:57710) 2023-07-14T16:32:04.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.204+0000 7f19f2ffd700 1 -- 172.21.15.62:0/1717427485 learned_addr learned my addr 172.21.15.62:0/1717427485 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.205+0000 7f19f37fe700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f40ffae0 0x7f19f406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.206+0000 7f19f3fff700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4101660 0x7f19f41abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.206+0000 7f19f2ffd700 1 -- 172.21.15.62:0/1717427485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4101660 msgr2=0x7f19f41abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.206+0000 7f19f2ffd700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4101660 0x7f19f41abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.206+0000 7f19f2ffd700 1 -- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f40ffae0 msgr2=0x7f19f406ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.206+0000 7f19f2ffd700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f40ffae0 0x7f19f406ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.206+0000 7f19f2ffd700 1 -- 172.21.15.62:0/1717427485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e40095f0 con 0x7f19f4100cd0 2023-07-14T16:32:04.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.207+0000 7f19f2ffd700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f4100cd0 0x7f19f406f290 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f19e800e9a0 tx=0x7f19e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.208+0000 7f19f0ff9700 1 -- 172.21.15.62:0/1717427485 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8009790 con 0x7f19f4100cd0 2023-07-14T16:32:04.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.208+0000 7f19f0ff9700 1 -- 172.21.15.62:0/1717427485 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19e8010430 con 0x7f19f4100cd0 2023-07-14T16:32:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.208+0000 7f19fa1ff700 1 -- 172.21.15.62:0/1717427485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f41ac710 con 0x7f19f4100cd0 2023-07-14T16:32:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.208+0000 7f19f0ff9700 1 -- 172.21.15.62:0/1717427485 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8018680 con 0x7f19f4100cd0 2023-07-14T16:32:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.208+0000 7f19fa1ff700 1 -- 172.21.15.62:0/1717427485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f41acc20 con 0x7f19f4100cd0 2023-07-14T16:32:04.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.215+0000 7f19f0ff9700 1 -- 172.21.15.62:0/1717427485 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f19e800cb70 con 0x7f19f4100cd0 2023-07-14T16:32:04.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.216+0000 7f1a0dc15b80 1 Processor -- start 2023-07-14T16:32:04.190 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.216+0000 7f1a0dc15b80 1 -- start start 2023-07-14T16:32:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c35805960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805ea0 0x555c35806760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35806cf0 0x555c35807160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3570ad30 con 0x555c35806cf0 2023-07-14T16:32:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3575dac0 con 0x555c35805ea0 2023-07-14T16:32:04.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c35756f50 con 0x555c35805550 2023-07-14T16:32:04.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c35805960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c35805960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60126/0 (socket says 172.21.15.62:60126) 2023-07-14T16:32:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.217+0000 7f19f3fff700 1 -- 172.21.15.62:0/2242991144 learned_addr learned my addr 172.21.15.62:0/2242991144 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.218+0000 7f19f3fff700 1 -- 172.21.15.62:0/2242991144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805ea0 msgr2=0x555c35806760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.218+0000 7f19f3fff700 1 --2- 172.21.15.62:0/2242991144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805ea0 0x555c35806760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.218+0000 7f19f3fff700 1 -- 172.21.15.62:0/2242991144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35806cf0 msgr2=0x555c35807160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.218+0000 7f19f3fff700 1 --2- 172.21.15.62:0/2242991144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35806cf0 0x555c35807160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.218+0000 7f19f3fff700 1 -- 172.21.15.62:0/2242991144 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c35584e60 con 0x555c35805550 2023-07-14T16:32:04.195 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.218+0000 7f19f0ff9700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19dc0a12d0 0x7f19dc0a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.218+0000 7f19f0ff9700 1 -- 172.21.15.62:0/1717427485 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19e8014070 con 0x7f19f4100cd0 2023-07-14T16:32:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.218+0000 7f19f3fff700 1 --2- 172.21.15.62:0/2242991144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c35805960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009440 tx=0x7f19ec009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=989ff8c77a8e6844 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.219+0000 7f19d9ffb700 1 -- 172.21.15.62:0/2242991144 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00e070 con 0x555c35805550 2023-07-14T16:32:04.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.219+0000 7f19f37fe700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19dc0a12d0 0x7f19dc0a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.219+0000 7f19d9ffb700 1 -- 172.21.15.62:0/2242991144 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19ec00f040 con 0x555c35805550 2023-07-14T16:32:04.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.219+0000 7f19d9ffb700 1 -- 172.21.15.62:0/2242991144 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec015a80 con 0x555c35805550 2023-07-14T16:32:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.220+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2242991144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 msgr2=0x555c35805960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.220+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/2242991144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c35805960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009440 tx=0x7f19ec009920 comp rx=0 tx=0).stop 2023-07-14T16:32:04.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.221+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2242991144 shutdown_connections 2023-07-14T16:32:04.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.221+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/2242991144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35806cf0 0x555c35807160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.221+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/2242991144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c35805960 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.221+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/2242991144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805ea0 0x555c35806760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.221+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2242991144 >> 172.21.15.62:0/2242991144 conn(0x555c357fae20 msgr2=0x555c35800250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.221+0000 7f19f37fe700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19dc0a12d0 0x7f19dc0a3780 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f19e4005910 tx=0x7f19e4004000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.222+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2242991144 shutdown_connections 2023-07-14T16:32:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.222+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2242991144 wait complete. 2023-07-14T16:32:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.223+0000 7f1a0dc15b80 1 -- start start 2023-07-14T16:32:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.223+0000 7f1a0dc15b80 1 Processor -- start 2023-07-14T16:32:04.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.224+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c3580f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.224+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c3580f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.224+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35806cf0 0x555c358167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.224+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c356f6b40 con 0x555c35805ea0 2023-07-14T16:32:04.203 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.224+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c356f6840 con 0x555c35806cf0 2023-07-14T16:32:04.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.224+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c355da9a0 con 0x555c35805550 2023-07-14T16:32:04.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.224+0000 7f1a0dc15b80 10 client.0 _ll_register_callbacks cb 0x555c356f47e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:04.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35806cf0 0x555c358167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35806cf0 0x555c358167a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57726/0 (socket says 172.21.15.62:57726) 2023-07-14T16:32:04.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f3fff700 1 -- 172.21.15.62:0/3765588779 learned_addr learned my addr 172.21.15.62:0/3765588779 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f37fe700 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c3580f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f3fff700 1 -- 172.21.15.62:0/3765588779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 msgr2=0x555c3580f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f3fff700 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c3580f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f3fff700 1 -- 172.21.15.62:0/3765588779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 msgr2=0x555c3580f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f3fff700 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c3580f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.225+0000 7f19f3fff700 1 -- 172.21.15.62:0/3765588779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec0090f0 con 0x555c35806cf0 2023-07-14T16:32:04.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.226+0000 7f19f3fff700 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35806cf0 0x555c358167a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b010 tx=0x7f19ec016a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ab91a8cf2563fc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.226+0000 7f19d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:04.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.226+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3765588779 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00e040 con 0x555c35806cf0 2023-07-14T16:32:04.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.226+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3765588779 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19ec013750 con 0x555c35806cf0 2023-07-14T16:32:04.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.226+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3765588779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c356b0570 con 0x555c35806cf0 2023-07-14T16:32:04.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.227+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3765588779 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19ec013950 con 0x555c35806cf0 2023-07-14T16:32:04.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.227+0000 7f1a0dc15b80 10 client.29669 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:04.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.227+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3765588779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555c3556dce0 con 0x555c35806cf0 2023-07-14T16:32:04.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3765588779 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555c356b0570 con 0x555c35806cf0 2023-07-14T16:32:04.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3765588779 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19ec01c050 con 0x555c35806cf0 2023-07-14T16:32:04.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f1a0dc15b80 10 client.29669 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:04.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f1a0dc15b80 10 client.29669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:04.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f1a0dc15b80 20 client.29669 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:04.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555c3581ae60 0x555c3581d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f1a0dc15b80 4 client.29669 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:04.213 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3765588779 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c3574ff30 con 0x555c3581ae60 2023-07-14T16:32:04.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.228+0000 7f19f37fe700 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555c3581ae60 0x555c3581d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.229+0000 7f19f37fe700 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555c3581ae60 0x555c3581d270 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e42972b52d508b server_cookie=581648ec5dc37ad1 in_seq=0 out_seq=0 2023-07-14T16:32:04.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.229+0000 7f19d2ffd700 10 client.29669 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:04.214 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.230+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3765588779 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555c3574ff30 con 0x555c3581ae60 2023-07-14T16:32:04.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.230+0000 7f19d2ffd700 10 client.29669 handle_command_reply: tid=0 2023-07-14T16:32:04.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.230+0000 7f1a0dc15b80 1 client.29669 shutdown 2023-07-14T16:32:04.215 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.230+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3765588779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35806cf0 msgr2=0x555c358167a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.230+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35806cf0 0x555c358167a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b010 tx=0x7f19ec016a40 comp rx=0 tx=0).stop 2023-07-14T16:32:04.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3765588779 shutdown_connections 2023-07-14T16:32:04.216 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555c3581ae60 0x555c3581d270 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35806cf0 0x555c358167a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35805550 0x555c3580f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3765588779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c3580f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3765588779 >> 172.21.15.62:0/3765588779 conn(0x555c357fae20 msgr2=0x555c357ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f19d2ffd700 0 client.29669 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:04.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3765588779 shutdown_connections 2023-07-14T16:32:04.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.231+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3765588779 wait complete. 2023-07-14T16:32:04.218 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.232+0000 7f1a0dc15b80 20 client.29669 trim_cache size 0 max 16384 2023-07-14T16:32:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.263+0000 7f1a0dc15b80 1 Processor -- start 2023-07-14T16:32:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.263+0000 7f1a0dc15b80 1 -- start start 2023-07-14T16:32:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.263+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c3580d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.263+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c3580d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.263+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 0x555c358166c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.263+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3580ed90 con 0x555c35805ea0 2023-07-14T16:32:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.263+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3580dba0 con 0x555c35805550 2023-07-14T16:32:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.263+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c357fa9f0 con 0x555c35806cf0 2023-07-14T16:32:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.264+0000 7f19f37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c3580d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.264+0000 7f19f37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c3580d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57738/0 (socket says 172.21.15.62:57738) 2023-07-14T16:32:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.264+0000 7f19f37fe700 1 -- 172.21.15.62:0/2788804356 learned_addr learned my addr 172.21.15.62:0/2788804356 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.264+0000 7f19f3fff700 1 --2- 172.21.15.62:0/2788804356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 0x555c358166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.265+0000 7f19f37fe700 1 -- 172.21.15.62:0/2788804356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 msgr2=0x555c358166c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.265+0000 7f19f37fe700 1 --2- 172.21.15.62:0/2788804356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 0x555c358166c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.265+0000 7f19f37fe700 1 -- 172.21.15.62:0/2788804356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 msgr2=0x555c3580d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.265+0000 7f19f37fe700 1 --2- 172.21.15.62:0/2788804356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c3580d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.265+0000 7f19f37fe700 1 -- 172.21.15.62:0/2788804356 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c355ab2d0 con 0x555c35805550 2023-07-14T16:32:04.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.265+0000 7f19f37fe700 1 --2- 172.21.15.62:0/2788804356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c3580d080 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f19f4100b30 tx=0x7f19e4020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0b6ce6f207d3b8a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.266+0000 7f19da7fc700 1 -- 172.21.15.62:0/2788804356 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4005420 con 0x555c35805550 2023-07-14T16:32:04.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.266+0000 7f19da7fc700 1 -- 172.21.15.62:0/2788804356 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19e401e040 con 0x555c35805550 2023-07-14T16:32:04.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.266+0000 7f19da7fc700 1 -- 172.21.15.62:0/2788804356 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4003d60 con 0x555c35805550 2023-07-14T16:32:04.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2788804356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 msgr2=0x555c3580d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/2788804356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c3580d080 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f19f4100b30 tx=0x7f19e4020000 comp rx=0 tx=0).stop 2023-07-14T16:32:04.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2788804356 shutdown_connections 2023-07-14T16:32:04.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/2788804356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 0x555c358166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/2788804356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c3580d080 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/2788804356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c3580d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2788804356 >> 172.21.15.62:0/2788804356 conn(0x555c357fae20 msgr2=0x555c3581ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.246 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2788804356 shutdown_connections 2023-07-14T16:32:04.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.267+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/2788804356 wait complete. 2023-07-14T16:32:04.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.268+0000 7f1a0dc15b80 1 -- start start 2023-07-14T16:32:04.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.268+0000 7f1a0dc15b80 1 Processor -- start 2023-07-14T16:32:04.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.270+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c35813b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c35814090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f1a0dc15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 0x555c358150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c358155e0 con 0x555c35805ea0 2023-07-14T16:32:04.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c35815750 con 0x555c35805550 2023-07-14T16:32:04.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f1a0dc15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c358158c0 con 0x555c35806cf0 2023-07-14T16:32:04.251 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f1a0dc15b80 10 client.0 _ll_register_callbacks cb 0x7f19ec009310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:04.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f19f37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c35813b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f19f37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c35813b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57752/0 (socket says 172.21.15.62:57752) 2023-07-14T16:32:04.252 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.271+0000 7f19f37fe700 1 -- 172.21.15.62:0/3931703214 learned_addr learned my addr 172.21.15.62:0/3931703214 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.272+0000 7f19f3fff700 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 0x555c358150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.272+0000 7f19f37fe700 1 -- 172.21.15.62:0/3931703214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 msgr2=0x555c358150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.272+0000 7f19f37fe700 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 0x555c358150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.272+0000 7f19f37fe700 1 -- 172.21.15.62:0/3931703214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 msgr2=0x555c35814090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.272+0000 7f19f37fe700 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c35814090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.272+0000 7f19f37fe700 1 -- 172.21.15.62:0/3931703214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f4108850 con 0x555c35805550 2023-07-14T16:32:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.272+0000 7f19f37fe700 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c35813b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f19e4038040 tx=0x7f19e4003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c02ab9bd697c3a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.272+0000 7f19d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:04.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.273+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3931703214 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4035160 con 0x555c35805550 2023-07-14T16:32:04.255 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.273+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3931703214 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19e40352c0 con 0x555c35805550 2023-07-14T16:32:04.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.273+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3931703214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19ec00f040 con 0x555c35805550 2023-07-14T16:32:04.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3931703214 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e4045030 con 0x555c35805550 2023-07-14T16:32:04.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f1a0dc15b80 10 client.29675 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:04.256 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3931703214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555c35583330 con 0x555c35805550 2023-07-14T16:32:04.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3931703214 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19ec00f040 con 0x555c35805550 2023-07-14T16:32:04.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3931703214 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19e40045b0 con 0x555c35805550 2023-07-14T16:32:04.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f1a0dc15b80 10 client.29675 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f1a0dc15b80 10 client.29675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f1a0dc15b80 20 client.29675 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.274+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555c35881d80 0x555c35884160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.275+0000 7f1a0dc15b80 4 client.29675 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:32:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.275+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3931703214 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555c357120d0 con 0x555c35881d80 2023-07-14T16:32:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.276+0000 7f19f2ffd700 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555c35881d80 0x555c35884160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.278+0000 7f19f2ffd700 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555c35881d80 0x555c35884160 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ddcf70c144cea87 server_cookie=b8f0ada79e71f575 in_seq=0 out_seq=0 2023-07-14T16:32:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.278+0000 7f19d2ffd700 10 client.29675 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.279+0000 7f19d2ffd700 1 -- 172.21.15.62:0/3931703214 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555c357120d0 con 0x555c35881d80 2023-07-14T16:32:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.279+0000 7f19d2ffd700 10 client.29675 handle_command_reply: tid=0 2023-07-14T16:32:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.279+0000 7f1a0dc15b80 1 client.29675 shutdown 2023-07-14T16:32:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.279+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3931703214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 msgr2=0x555c35813b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.279+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c35813b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f19e4038040 tx=0x7f19e4003730 comp rx=0 tx=0).stop 2023-07-14T16:32:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3931703214 shutdown_connections 2023-07-14T16:32:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555c35806cf0 0x555c358150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555c35881d80 0x555c35884160 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555c35805550 0x555c35813b50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f1a0dc15b80 1 --2- 172.21.15.62:0/3931703214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c35805ea0 0x555c35814090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3931703214 >> 172.21.15.62:0/3931703214 conn(0x555c357fae20 msgr2=0x555c3580d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f19d2ffd700 0 client.29675 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3931703214 shutdown_connections 2023-07-14T16:32:04.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.282+0000 7f1a0dc15b80 1 -- 172.21.15.62:0/3931703214 wait complete. 2023-07-14T16:32:04.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.283+0000 7f1a0dc15b80 20 client.29675 trim_cache size 0 max 16384 2023-07-14T16:32:04.265 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:32:04.265 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:32:04.266 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "d6e9a627-76be-4f70-ba3d-f6f3d297f6c8", 2023-07-14T16:32:04.266 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:32:04.266 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:32:04.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 -- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19dc0a12d0 msgr2=0x7f19dc0a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19dc0a12d0 0x7f19dc0a3780 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f19e4005910 tx=0x7f19e4004000 comp rx=0 tx=0).stop 2023-07-14T16:32:04.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 -- 172.21.15.62:0/1717427485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f4100cd0 msgr2=0x7f19f406f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f4100cd0 0x7f19f406f290 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f19e800e9a0 tx=0x7f19e800ecb0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 -- 172.21.15.62:0/1717427485 shutdown_connections 2023-07-14T16:32:04.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19f4101660 0x7f19f41abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19f40ffae0 0x7f19f406ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f19dc0a12d0 0x7f19dc0a3780 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 --2- 172.21.15.62:0/1717427485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19f4100cd0 0x7f19f406f290 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.284+0000 7f19fa1ff700 1 -- 172.21.15.62:0/1717427485 >> 172.21.15.62:0/1717427485 conn(0x7f19f40fb0e0 msgr2=0x7f19f41041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.270 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.285+0000 7f19fa1ff700 1 -- 172.21.15.62:0/1717427485 shutdown_connections 2023-07-14T16:32:04.271 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.288+0000 7f19fa1ff700 1 -- 172.21.15.62:0/1717427485 wait complete. 2023-07-14T16:32:04.297 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:32:04.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.670+0000 7f4e109ff700 1 -- 172.21.15.62:0/2897859922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c070c70 msgr2=0x7f4e0c0710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.670+0000 7f4e109ff700 1 --2- 172.21.15.62:0/2897859922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c070c70 0x7f4e0c0710e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009990 tx=0x7f4df4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.670+0000 7f4e109ff700 1 -- 172.21.15.62:0/2897859922 shutdown_connections 2023-07-14T16:32:04.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.670+0000 7f4e109ff700 1 --2- 172.21.15.62:0/2897859922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e0c071620 0x7f4e0c110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.670+0000 7f4e109ff700 1 --2- 172.21.15.62:0/2897859922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e0c107a20 0x7f4e0c107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.670+0000 7f4e109ff700 1 --2- 172.21.15.62:0/2897859922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c070c70 0x7f4e0c0710e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.670+0000 7f4e109ff700 1 -- 172.21.15.62:0/2897859922 >> 172.21.15.62:0/2897859922 conn(0x7f4e0c06ca90 msgr2=0x7f4e0c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.670+0000 7f4e109ff700 1 -- 172.21.15.62:0/2897859922 shutdown_connections 2023-07-14T16:32:04.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.671+0000 7f4e109ff700 1 -- 172.21.15.62:0/2897859922 wait complete. 2023-07-14T16:32:04.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.672+0000 7f4e109ff700 1 Processor -- start 2023-07-14T16:32:04.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.672+0000 7f4e109ff700 1 -- start start 2023-07-14T16:32:04.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.673+0000 7f4e109ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e0c070c70 0x7f4e0c117050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.673+0000 7f4e109ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c071620 0x7f4e0c117590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.673+0000 7f4e109ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e0c107a20 0x7f4e0c11c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.673+0000 7f4e109ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c11cbc0 con 0x7f4e0c070c70 2023-07-14T16:32:04.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.673+0000 7f4e109ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c11cd30 con 0x7f4e0c071620 2023-07-14T16:32:04.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.673+0000 7f4e109ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c11cef0 con 0x7f4e0c107a20 2023-07-14T16:32:04.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.674+0000 7f4e09d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c071620 0x7f4e0c117590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.674+0000 7f4e09d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c071620 0x7f4e0c117590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57764/0 (socket says 172.21.15.62:57764) 2023-07-14T16:32:04.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.674+0000 7f4e09d9b700 1 -- 172.21.15.62:0/1036870899 learned_addr learned my addr 172.21.15.62:0/1036870899 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.674+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e0c107a20 0x7f4e0c11c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.674+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e0c070c70 0x7f4e0c117050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.675+0000 7f4e09d9b700 1 -- 172.21.15.62:0/1036870899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e0c107a20 msgr2=0x7f4e0c11c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.675+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e0c107a20 0x7f4e0c11c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.675+0000 7f4e09d9b700 1 -- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e0c070c70 msgr2=0x7f4e0c117050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.675+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e0c070c70 0x7f4e0c117050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.675+0000 7f4e09d9b700 1 -- 172.21.15.62:0/1036870899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40095f0 con 0x7f4e0c071620 2023-07-14T16:32:04.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.675+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c071620 0x7f4e0c117590 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000c00 tx=0x7f4df4003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.676+0000 7f4e037fe700 1 -- 172.21.15.62:0/1036870899 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df401d070 con 0x7f4e0c071620 2023-07-14T16:32:04.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.676+0000 7f4e037fe700 1 -- 172.21.15.62:0/1036870899 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4df400bd70 con 0x7f4e0c071620 2023-07-14T16:32:04.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.676+0000 7f4e109ff700 1 -- 172.21.15.62:0/1036870899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0c11d110 con 0x7f4e0c071620 2023-07-14T16:32:04.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.676+0000 7f4e037fe700 1 -- 172.21.15.62:0/1036870899 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4022c20 con 0x7f4e0c071620 2023-07-14T16:32:04.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.676+0000 7f4e109ff700 1 -- 172.21.15.62:0/1036870899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c11d570 con 0x7f4e0c071620 2023-07-14T16:32:04.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.682+0000 7f4e242f2b80 1 Processor -- start 2023-07-14T16:32:04.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.683+0000 7f4e242f2b80 1 -- start start 2023-07-14T16:32:04.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.683+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.683+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.683+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 0x5609f5a590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.683+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f595ccb0 con 0x5609f5a57e20 2023-07-14T16:32:04.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.683+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f59afa40 con 0x5609f5a58c70 2023-07-14T16:32:04.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.683+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f59a8ed0 con 0x5609f5a574d0 2023-07-14T16:32:04.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56378/0 (socket says 172.21.15.62:56378) 2023-07-14T16:32:04.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0a59c700 1 -- 172.21.15.62:0/2528614401 learned_addr learned my addr 172.21.15.62:0/2528614401 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/2528614401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0ad9d700 1 -- 172.21.15.62:0/2528614401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 msgr2=0x5609f5a590e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:32:04.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/2528614401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 0x5609f5a590e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0ad9d700 1 -- 172.21.15.62:0/2528614401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 msgr2=0x5609f5a586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/2528614401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.684+0000 7f4e0ad9d700 1 -- 172.21.15.62:0/2528614401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609f57d6de0 con 0x5609f5a574d0 2023-07-14T16:32:04.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.685+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/2528614401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a578e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00ab40 tx=0x7f4dfc00ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8964950baf5a504c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.685+0000 7f4e00ff9700 1 -- 172.21.15.62:0/2528614401 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00fcc0 con 0x5609f5a574d0 2023-07-14T16:32:04.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.685+0000 7f4e00ff9700 1 -- 172.21.15.62:0/2528614401 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dfc00fe20 con 0x5609f5a574d0 2023-07-14T16:32:04.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.685+0000 7f4e00ff9700 1 -- 172.21.15.62:0/2528614401 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc018440 con 0x5609f5a574d0 2023-07-14T16:32:04.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.685+0000 7f4e037fe700 1 -- 172.21.15.62:0/1036870899 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4df4003e90 con 0x7f4e0c071620 2023-07-14T16:32:04.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.686+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2528614401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 msgr2=0x5609f5a578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.686+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/2528614401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a578e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00ab40 tx=0x7f4dfc00ae50 comp rx=0 tx=0).stop 2023-07-14T16:32:04.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.686+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2528614401 shutdown_connections 2023-07-14T16:32:04.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.686+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/2528614401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 0x5609f5a590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.686+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/2528614401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a578e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.686+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/2528614401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.686+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2528614401 >> 172.21.15.62:0/2528614401 conn(0x5609f5a4cda0 msgr2=0x5609f5a521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.688+0000 7f4e037fe700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4df80a1150 0x7f4df80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.688+0000 7f4e037fe700 1 -- 172.21.15.62:0/1036870899 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4df40cdf40 con 0x7f4e0c071620 2023-07-14T16:32:04.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.690+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2528614401 shutdown_connections 2023-07-14T16:32:04.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.690+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2528614401 wait complete. 2023-07-14T16:32:04.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.691+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4df80a1150 0x7f4df80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.691+0000 7f4e242f2b80 1 -- start start 2023-07-14T16:32:04.665 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e242f2b80 1 Processor -- start 2023-07-14T16:32:04.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4df80a1150 0x7f4df80a3600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009550 tx=0x7f4e040091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a61170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 0x5609f5a686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f5948ac0 con 0x5609f5a57e20 2023-07-14T16:32:04.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f59487c0 con 0x5609f5a58c70 2023-07-14T16:32:04.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f582c920 con 0x5609f5a574d0 2023-07-14T16:32:04.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.692+0000 7f4e242f2b80 10 client.0 _ll_register_callbacks cb 0x5609f5946760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:04.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.693+0000 7f4e0a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a61170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.693+0000 7f4e0ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 0x5609f5a686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.667 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.693+0000 7f4e0a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a61170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60202/0 (socket says 172.21.15.62:60202) 2023-07-14T16:32:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.693+0000 7f4e0a59c700 1 -- 172.21.15.62:0/87955466 learned_addr learned my addr 172.21.15.62:0/87955466 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.693+0000 7f4e0ad9d700 1 -- 172.21.15.62:0/87955466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 msgr2=0x5609f5a61170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.693+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.693+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a61170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.693+0000 7f4e0ad9d700 1 -- 172.21.15.62:0/87955466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 msgr2=0x5609f5a616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.694+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.694+0000 7f4e0ad9d700 1 -- 172.21.15.62:0/87955466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc00a800 con 0x5609f5a58c70 2023-07-14T16:32:04.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.694+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:04.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.694+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 0x5609f5a686f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc01a040 tx=0x7f4dfc004a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b9d8b43f9d5cb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.694+0000 7f4df1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:04.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.694+0000 7f4df1ffb700 1 -- 172.21.15.62:0/87955466 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc003f30 con 0x5609f5a58c70 2023-07-14T16:32:04.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.694+0000 7f4e242f2b80 1 -- 172.21.15.62:0/87955466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609f59024f0 con 0x5609f5a58c70 2023-07-14T16:32:04.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.694+0000 7f4df1ffb700 1 -- 172.21.15.62:0/87955466 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4dfc0036f0 con 0x5609f5a58c70 2023-07-14T16:32:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.695+0000 7f4df1ffb700 1 -- 172.21.15.62:0/87955466 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dfc004090 con 0x5609f5a58c70 2023-07-14T16:32:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.695+0000 7f4e242f2b80 10 client.29684 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.695+0000 7f4e242f2b80 1 -- 172.21.15.62:0/87955466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5609f57bfc60 con 0x5609f5a58c70 2023-07-14T16:32:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.695+0000 7f4df1ffb700 1 -- 172.21.15.62:0/87955466 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5609f59024f0 con 0x5609f5a58c70 2023-07-14T16:32:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.695+0000 7f4df1ffb700 1 -- 172.21.15.62:0/87955466 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4dfc020040 con 0x5609f5a58c70 2023-07-14T16:32:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.695+0000 7f4e242f2b80 10 client.29684 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.695+0000 7f4e242f2b80 10 client.29684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:04.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.696+0000 7f4e242f2b80 20 client.29684 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:04.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.696+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5609f5a6cdb0 0x5609f5a6f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.696+0000 7f4e242f2b80 4 client.29684 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:04.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.696+0000 7f4e242f2b80 1 -- 172.21.15.62:0/87955466 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609f59a1eb0 con 0x5609f5a6cdb0 2023-07-14T16:32:04.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.696+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5609f5a6cdb0 0x5609f5a6f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.697+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5609f5a6cdb0 0x5609f5a6f1c0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301d4823e461fb0f server_cookie=ea64aadc448bba28 in_seq=0 out_seq=0 2023-07-14T16:32:04.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.697+0000 7f4df1ffb700 10 client.29684 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:04.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.698+0000 7f4df1ffb700 1 -- 172.21.15.62:0/87955466 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5609f59a1eb0 con 0x5609f5a6cdb0 2023-07-14T16:32:04.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.698+0000 7f4df1ffb700 10 client.29684 handle_command_reply: tid=0 2023-07-14T16:32:04.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.698+0000 7f4e242f2b80 1 client.29684 shutdown 2023-07-14T16:32:04.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 -- 172.21.15.62:0/87955466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 msgr2=0x5609f5a686f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 0x5609f5a686f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc01a040 tx=0x7f4dfc004a00 comp rx=0 tx=0).stop 2023-07-14T16:32:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 -- 172.21.15.62:0/87955466 shutdown_connections 2023-07-14T16:32:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5609f5a6cdb0 0x5609f5a6f1c0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a58c70 0x5609f5a686f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a574d0 0x5609f5a61170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/87955466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 -- 172.21.15.62:0/87955466 >> 172.21.15.62:0/87955466 conn(0x5609f5a4cda0 msgr2=0x5609f5a518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4df1ffb700 0 client.29684 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:04.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.699+0000 7f4e242f2b80 1 -- 172.21.15.62:0/87955466 shutdown_connections 2023-07-14T16:32:04.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.700+0000 7f4e242f2b80 1 -- 172.21.15.62:0/87955466 wait complete. 2023-07-14T16:32:04.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.701+0000 7f4e242f2b80 20 client.29684 trim_cache size 0 max 16384 2023-07-14T16:32:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.726+0000 7f4e242f2b80 1 Processor -- start 2023-07-14T16:32:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.726+0000 7f4e242f2b80 1 -- start start 2023-07-14T16:32:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a574d0 0x5609f5a68f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a57e20 0x5609f5a68680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 0x5609f5a6edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f5a534c0 con 0x5609f5a574d0 2023-07-14T16:32:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f5a60620 con 0x5609f5a57e20 2023-07-14T16:32:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f5a6cb50 con 0x5609f5a58c70 2023-07-14T16:32:04.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e0a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a574d0 0x5609f5a68f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e0a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a574d0 0x5609f5a68f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56390/0 (socket says 172.21.15.62:56390) 2023-07-14T16:32:04.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.727+0000 7f4e0a59c700 1 -- 172.21.15.62:0/2011133553 learned_addr learned my addr 172.21.15.62:0/2011133553 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.728+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a57e20 0x5609f5a68680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.728+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 0x5609f5a6edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.728+0000 7f4e0a59c700 1 -- 172.21.15.62:0/2011133553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 msgr2=0x5609f5a6edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.703 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.728+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 0x5609f5a6edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.728+0000 7f4e0a59c700 1 -- 172.21.15.62:0/2011133553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a57e20 msgr2=0x5609f5a68680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.728+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a57e20 0x5609f5a68680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.728+0000 7f4e0a59c700 1 -- 172.21.15.62:0/2011133553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609f57fd250 con 0x5609f5a574d0 2023-07-14T16:32:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.729+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a574d0 0x5609f5a68f20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004510 tx=0x7f4e0400b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db6da4808904db8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.729+0000 7f4e017fa700 1 -- 172.21.15.62:0/2011133553 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0402ae80 con 0x5609f5a574d0 2023-07-14T16:32:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.729+0000 7f4e017fa700 1 -- 172.21.15.62:0/2011133553 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e0400f040 con 0x5609f5a574d0 2023-07-14T16:32:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.729+0000 7f4e017fa700 1 -- 172.21.15.62:0/2011133553 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04007a20 con 0x5609f5a574d0 2023-07-14T16:32:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.730+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2011133553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a574d0 msgr2=0x5609f5a68f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.730+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a574d0 0x5609f5a68f20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004510 tx=0x7f4e0400b050 comp rx=0 tx=0).stop 2023-07-14T16:32:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.730+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2011133553 shutdown_connections 2023-07-14T16:32:04.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.730+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 0x5609f5a6edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.730+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a574d0 0x5609f5a68f20 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.730+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/2011133553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a57e20 0x5609f5a68680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.730+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2011133553 >> 172.21.15.62:0/2011133553 conn(0x5609f5a4cda0 msgr2=0x5609f5a5fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.731+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2011133553 shutdown_connections 2023-07-14T16:32:04.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.731+0000 7f4e242f2b80 1 -- 172.21.15.62:0/2011133553 wait complete. 2023-07-14T16:32:04.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.731+0000 7f4e242f2b80 1 -- start start 2023-07-14T16:32:04.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.731+0000 7f4e242f2b80 1 Processor -- start 2023-07-14T16:32:04.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.732+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a574d0 0x5609f5a68620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.732+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.732+0000 7f4e242f2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 0x5609f5a656b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.732+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f5ac0c60 con 0x5609f5a57e20 2023-07-14T16:32:04.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.732+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f5ac0da0 con 0x5609f5a574d0 2023-07-14T16:32:04.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.732+0000 7f4e242f2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f5ac0ee0 con 0x5609f5a58c70 2023-07-14T16:32:04.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.732+0000 7f4e242f2b80 10 client.0 _ll_register_callbacks cb 0x7f4e04003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:04.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.733+0000 7f4e09d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.733+0000 7f4e09d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a650f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56398/0 (socket says 172.21.15.62:56398) 2023-07-14T16:32:04.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.733+0000 7f4e09d9b700 1 -- 172.21.15.62:0/1580019074 learned_addr learned my addr 172.21.15.62:0/1580019074 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:04.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.733+0000 7f4e0ad9d700 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 0x5609f5a656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.733+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a574d0 0x5609f5a68620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4e09d9b700 1 -- 172.21.15.62:0/1580019074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 msgr2=0x5609f5a656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 0x5609f5a656b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4e09d9b700 1 -- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a574d0 msgr2=0x5609f5a68620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:04.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a574d0 0x5609f5a68620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4e09d9b700 1 -- 172.21.15.62:0/1580019074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e040042c0 con 0x5609f5a57e20 2023-07-14T16:32:04.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a574d0 0x5609f5a68620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:32:04.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4e09d9b700 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a650f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4df409b440 tx=0x7f4df409b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefef3230c20f867 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:04.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4df1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:04.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4df1ffb700 1 -- 172.21.15.62:0/1580019074 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4022470 con 0x5609f5a57e20 2023-07-14T16:32:04.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.734+0000 7f4df1ffb700 1 -- 172.21.15.62:0/1580019074 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4df409b4b0 con 0x5609f5a57e20 2023-07-14T16:32:04.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.735+0000 7f4e242f2b80 1 -- 172.21.15.62:0/1580019074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dfc00fe20 con 0x5609f5a57e20 2023-07-14T16:32:04.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.735+0000 7f4df1ffb700 1 -- 172.21.15.62:0/1580019074 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df409f4e0 con 0x5609f5a57e20 2023-07-14T16:32:04.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.735+0000 7f4e242f2b80 10 client.20151 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:04.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.735+0000 7f4e242f2b80 1 -- 172.21.15.62:0/1580019074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5609f57d52b0 con 0x5609f5a57e20 2023-07-14T16:32:04.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.735+0000 7f4df1ffb700 1 -- 172.21.15.62:0/1580019074 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dfc00fe20 con 0x5609f5a57e20 2023-07-14T16:32:04.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.736+0000 7f4df1ffb700 1 -- 172.21.15.62:0/1580019074 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4df40a33e0 con 0x5609f5a57e20 2023-07-14T16:32:04.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.736+0000 7f4e242f2b80 10 client.20151 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:04.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.736+0000 7f4e242f2b80 10 client.20151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:04.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.736+0000 7f4e242f2b80 20 client.20151 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:04.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.736+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5609f5ac24f0 0x5609f5ac4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:04.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.736+0000 7f4e242f2b80 4 client.20151 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:32:04.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.736+0000 7f4e242f2b80 1 -- 172.21.15.62:0/1580019074 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5609f5964050 con 0x5609f5ac24f0 2023-07-14T16:32:04.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.737+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5609f5ac24f0 0x5609f5ac4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:04.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.738+0000 7f4e0a59c700 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5609f5ac24f0 0x5609f5ac4900 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b6c27e845d44aa server_cookie=3a72b3bf1723b31c in_seq=0 out_seq=0 2023-07-14T16:32:04.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.738+0000 7f4df1ffb700 10 client.20151 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:04.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.738+0000 7f4df1ffb700 1 -- 172.21.15.62:0/1580019074 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5609f5964050 con 0x5609f5ac24f0 2023-07-14T16:32:04.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.738+0000 7f4df1ffb700 10 client.20151 handle_command_reply: tid=0 2023-07-14T16:32:04.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.738+0000 7f4e242f2b80 1 client.20151 shutdown 2023-07-14T16:32:04.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4e242f2b80 1 -- 172.21.15.62:0/1580019074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 msgr2=0x5609f5a650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a650f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4df409b440 tx=0x7f4df409b9f0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4e242f2b80 1 -- 172.21.15.62:0/1580019074 shutdown_connections 2023-07-14T16:32:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5609f5a58c70 0x5609f5a656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5609f5ac24f0 0x5609f5ac4900 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5609f5a574d0 0x5609f5a68620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4e242f2b80 1 --2- 172.21.15.62:0/1580019074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609f5a57e20 0x5609f5a650f0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4e242f2b80 1 -- 172.21.15.62:0/1580019074 >> 172.21.15.62:0/1580019074 conn(0x5609f5a4cda0 msgr2=0x5609f5a6edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.716 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.739+0000 7f4df1ffb700 0 client.20151 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:04.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.740+0000 7f4e242f2b80 1 -- 172.21.15.62:0/1580019074 shutdown_connections 2023-07-14T16:32:04.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.740+0000 7f4e242f2b80 1 -- 172.21.15.62:0/1580019074 wait complete. 2023-07-14T16:32:04.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.740+0000 7f4e242f2b80 20 client.20151 trim_cache size 0 max 16384 2023-07-14T16:32:04.718 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:32:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:32:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:32:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "d6e9a627-76be-4f70-ba3d-f6f3d297f6c8": { 2023-07-14T16:32:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:32:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "d6e9a627-76be-4f70-ba3d-f6f3d297f6c8", 2023-07-14T16:32:04.718 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:32:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:32:04.719 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:32:04.719 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:32:04.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 -- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4df80a1150 msgr2=0x7f4df80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4df80a1150 0x7f4df80a3600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009550 tx=0x7f4e040091f0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 -- 172.21.15.62:0/1036870899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c071620 msgr2=0x7f4e0c117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c071620 0x7f4e0c117590 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000c00 tx=0x7f4df4003a50 comp rx=0 tx=0).stop 2023-07-14T16:32:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 -- 172.21.15.62:0/1036870899 shutdown_connections 2023-07-14T16:32:04.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e0c107a20 0x7f4e0c11c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e0c070c70 0x7f4e0c117050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f4df80a1150 0x7f4df80a3600 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 --2- 172.21.15.62:0/1036870899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e0c071620 0x7f4e0c117590 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:04.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 -- 172.21.15.62:0/1036870899 >> 172.21.15.62:0/1036870899 conn(0x7f4e0c06ca90 msgr2=0x7f4e0c06fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:04.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 -- 172.21.15.62:0/1036870899 shutdown_connections 2023-07-14T16:32:04.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:04.741+0000 7f4e109ff700 1 -- 172.21.15.62:0/1036870899 wait complete. 2023-07-14T16:32:04.744 INFO:tasks.cephfs.filesystem:scrub status for tag:d6e9a627-76be-4f70-ba3d-f6f3d297f6c8 - {'path': '/', 'tag': 'd6e9a627-76be-4f70-ba3d-f6f3d297f6c8', 'options': 'recursive,force'} 2023-07-14T16:32:05.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:04 smithi062 ceph-mon[95543]: pgmap v850: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 366 op/s 2023-07-14T16:32:05.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:04 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:32:05.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:04 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:32:05.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:04 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:05.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:04 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:05.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:04 smithi081 ceph-mon[127795]: pgmap v850: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 366 op/s 2023-07-14T16:32:05.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:04 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:32:05.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:04 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:32:05.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:04 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:05.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:04 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:05.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:04 smithi164 ceph-mon[97427]: pgmap v850: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 366 op/s 2023-07-14T16:32:05.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:04 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:32:05.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:04 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:32:05.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:04 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:05.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:04 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:06.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:05 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:32:06.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:05 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:06.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:05 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:32:06.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:05 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:06.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:05 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:32:06.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:05 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:06.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:05 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:32:06.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:05 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:06.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:05 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:32:06.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:05 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:06.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:05 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:32:06.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:05 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:07.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:06 smithi062 ceph-mon[95543]: pgmap v851: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 589 op/s 2023-07-14T16:32:07.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:06 smithi081 ceph-mon[127795]: pgmap v851: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 589 op/s 2023-07-14T16:32:07.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:06 smithi164 ceph-mon[97427]: pgmap v851: 97 pgs: 97 active+clean; 13 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 589 op/s 2023-07-14T16:32:09.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:08 smithi062 ceph-mon[95543]: pgmap v852: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 1.57k op/s 2023-07-14T16:32:09.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:08 smithi081 ceph-mon[127795]: pgmap v852: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 1.57k op/s 2023-07-14T16:32:09.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:08 smithi164 ceph-mon[97427]: pgmap v852: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 16 MiB/s wr, 1.57k op/s 2023-07-14T16:32:11.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:10 smithi062 ceph-mon[95543]: pgmap v853: 97 pgs: 97 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 2.59k op/s 2023-07-14T16:32:11.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:10 smithi081 ceph-mon[127795]: pgmap v853: 97 pgs: 97 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 2.59k op/s 2023-07-14T16:32:11.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:10 smithi164 ceph-mon[97427]: pgmap v853: 97 pgs: 97 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 2.59k op/s 2023-07-14T16:32:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.676+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3779561390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a68100640 msgr2=0x7f0a68106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.676+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3779561390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a68100640 0x7f0a68106ee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009b00 tx=0x7f0a50009e10 comp rx=0 tx=0).stop 2023-07-14T16:32:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.676+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3779561390 shutdown_connections 2023-07-14T16:32:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.676+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3779561390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a68100640 0x7f0a68106ee0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.676+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3779561390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a680fea00 0x7f0a680fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.676+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3779561390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a680ffc90 0x7f0a68100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.676+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3779561390 >> 172.21.15.62:0/3779561390 conn(0x7f0a680fa000 msgr2=0x7f0a680fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.680+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3779561390 shutdown_connections 2023-07-14T16:32:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.680+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3779561390 wait complete. 2023-07-14T16:32:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.683+0000 7f0a6cdff700 1 Processor -- start 2023-07-14T16:32:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.683+0000 7f0a6cdff700 1 -- start start 2023-07-14T16:32:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.683+0000 7f0a6cdff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a680fea00 0x7f0a6806aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.683+0000 7f0a6cdff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a680ffc90 0x7f0a6806afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.683+0000 7f0a6cdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a68100640 0x7f0a6806be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.683+0000 7f0a6cdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6806c4d0 con 0x7f0a68100640 2023-07-14T16:32:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.683+0000 7f0a6cdff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6806c640 con 0x7f0a680ffc90 2023-07-14T16:32:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.683+0000 7f0a6cdff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6806c800 con 0x7f0a680fea00 2023-07-14T16:32:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.684+0000 7f0a6659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a680fea00 0x7f0a6806aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.684+0000 7f0a6659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a680fea00 0x7f0a6806aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60234/0 (socket says 172.21.15.62:60234) 2023-07-14T16:32:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.684+0000 7f0a6659c700 1 -- 172.21.15.62:0/3917947222 learned_addr learned my addr 172.21.15.62:0/3917947222 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.684+0000 7f0a65d9b700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a680ffc90 0x7f0a6806afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.686+0000 7f0a6659c700 1 -- 172.21.15.62:0/3917947222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a680ffc90 msgr2=0x7f0a6806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.686+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a680ffc90 0x7f0a6806afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.686+0000 7f0a6659c700 1 -- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a68100640 msgr2=0x7f0a6806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.686+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a68100640 0x7f0a6806be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.686+0000 7f0a6659c700 1 -- 172.21.15.62:0/3917947222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a50009730 con 0x7f0a680fea00 2023-07-14T16:32:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.687+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a680fea00 0x7f0a6806aa60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000e9c0 tx=0x7f0a6000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.687+0000 7f0a5f7fe700 1 -- 172.21.15.62:0/3917947222 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000cbd0 con 0x7f0a680fea00 2023-07-14T16:32:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.687+0000 7f0a5f7fe700 1 -- 172.21.15.62:0/3917947222 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a6000eeb0 con 0x7f0a680fea00 2023-07-14T16:32:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.687+0000 7f0a5f7fe700 1 -- 172.21.15.62:0/3917947222 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60005420 con 0x7f0a680fea00 2023-07-14T16:32:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.689+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3917947222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6806cae0 con 0x7f0a680fea00 2023-07-14T16:32:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.689+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3917947222 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a680712b0 con 0x7f0a680fea00 2023-07-14T16:32:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.695+0000 7f0a5f7fe700 1 -- 172.21.15.62:0/3917947222 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0a60005580 con 0x7f0a680fea00 2023-07-14T16:32:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.696+0000 7f0a807bdb80 1 Processor -- start 2023-07-14T16:32:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.697+0000 7f0a807bdb80 1 -- start start 2023-07-14T16:32:11.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.697+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c974bb6b0 0x563c9765f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.697+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c976601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.697+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c976606e0 0x563c97660b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.697+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97564c50 con 0x563c976606e0 2023-07-14T16:32:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.697+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c975b79e0 con 0x563c9765f840 2023-07-14T16:32:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.697+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c975b0e70 con 0x563c974bb6b0 2023-07-14T16:32:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.699+0000 7f0a65d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c976606e0 0x563c97660b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.699+0000 7f0a65d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c976606e0 0x563c97660b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56432/0 (socket says 172.21.15.62:56432) 2023-07-14T16:32:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.699+0000 7f0a65d9b700 1 -- 172.21.15.62:0/3616392838 learned_addr learned my addr 172.21.15.62:0/3616392838 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.699+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c976601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.699+0000 7f0a66d9d700 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c974bb6b0 0x563c9765f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.699+0000 7f0a5f7fe700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a540a12a0 0x7f0a540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.700+0000 7f0a6659c700 1 -- 172.21.15.62:0/3616392838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c974bb6b0 msgr2=0x563c9765f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.700+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c974bb6b0 0x563c9765f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.700+0000 7f0a6659c700 1 -- 172.21.15.62:0/3616392838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c976606e0 msgr2=0x563c97660b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.700+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c976606e0 0x563c97660b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.700+0000 7f0a6659c700 1 -- 172.21.15.62:0/3616392838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c973ded80 con 0x563c9765f840 2023-07-14T16:32:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.700+0000 7f0a66d9d700 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c974bb6b0 0x563c9765f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.700+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c976601a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a600ca630 tx=0x7f0a6001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e328915d6c328d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.700+0000 7f0a5f7fe700 1 -- 172.21.15.62:0/3917947222 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a60014070 con 0x7f0a680fea00 2023-07-14T16:32:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.701+0000 7f0a66d9d700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a540a12a0 0x7f0a540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.702+0000 7f0a5cff9700 1 -- 172.21.15.62:0/3616392838 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60099d70 con 0x563c9765f840 2023-07-14T16:32:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.702+0000 7f0a5cff9700 1 -- 172.21.15.62:0/3616392838 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a60099ae0 con 0x563c9765f840 2023-07-14T16:32:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.702+0000 7f0a5cff9700 1 -- 172.21.15.62:0/3616392838 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6009dfd0 con 0x563c9765f840 2023-07-14T16:32:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.702+0000 7f0a66d9d700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a540a12a0 0x7f0a540a3750 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000b5c0 tx=0x7f0a50005af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3616392838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 msgr2=0x563c976601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c976601a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a600ca630 tx=0x7f0a6001f000 comp rx=0 tx=0).stop 2023-07-14T16:32:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3616392838 shutdown_connections 2023-07-14T16:32:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c976606e0 0x563c97660b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c974bb6b0 0x563c9765f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3616392838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c976601a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3616392838 >> 172.21.15.62:0/3616392838 conn(0x563c97654be0 msgr2=0x563c97659cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3616392838 shutdown_connections 2023-07-14T16:32:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.704+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3616392838 wait complete. 2023-07-14T16:32:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.705+0000 7f0a807bdb80 1 -- start start 2023-07-14T16:32:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.706+0000 7f0a807bdb80 1 Processor -- start 2023-07-14T16:32:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.706+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c976687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.706+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c97668d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.706+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 0x563c9766fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.706+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97550a60 con 0x563c974bb6b0 2023-07-14T16:32:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.706+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97550760 con 0x563c976606e0 2023-07-14T16:32:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.706+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c974348c0 con 0x563c9765f840 2023-07-14T16:32:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.707+0000 7f0a807bdb80 10 client.0 _ll_register_callbacks cb 0x563c9754e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.707+0000 7f0a6659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c976687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.707+0000 7f0a65d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c97668d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.707+0000 7f0a65d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c97668d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60242/0 (socket says 172.21.15.62:60242) 2023-07-14T16:32:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.707+0000 7f0a6659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c976687f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56436/0 (socket says 172.21.15.62:56436) 2023-07-14T16:32:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.707+0000 7f0a65d9b700 1 -- 172.21.15.62:0/443929484 learned_addr learned my addr 172.21.15.62:0/443929484 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.708+0000 7f0a66d9d700 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 0x563c9766fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.708+0000 7f0a6659c700 1 -- 172.21.15.62:0/443929484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 msgr2=0x563c97668d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.708+0000 7f0a6659c700 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c97668d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.708+0000 7f0a6659c700 1 -- 172.21.15.62:0/443929484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 msgr2=0x563c9766fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.708+0000 7f0a6659c700 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 0x563c9766fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.708+0000 7f0a6659c700 1 -- 172.21.15.62:0/443929484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a680712b0 con 0x563c974bb6b0 2023-07-14T16:32:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.708+0000 7f0a6659c700 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c976687f0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0a600964a0 tx=0x7f0a600d24b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18467ed617488669 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.708+0000 7f0a4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.709+0000 7f0a4dffb700 1 -- 172.21.15.62:0/443929484 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a600d2180 con 0x563c974bb6b0 2023-07-14T16:32:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.709+0000 7f0a4dffb700 1 -- 172.21.15.62:0/443929484 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a600a23f0 con 0x563c974bb6b0 2023-07-14T16:32:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.709+0000 7f0a807bdb80 1 -- 172.21.15.62:0/443929484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c9750a490 con 0x563c974bb6b0 2023-07-14T16:32:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.709+0000 7f0a4dffb700 1 -- 172.21.15.62:0/443929484 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a60099d70 con 0x563c974bb6b0 2023-07-14T16:32:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.709+0000 7f0a807bdb80 10 client.20160 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.709+0000 7f0a807bdb80 1 -- 172.21.15.62:0/443929484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563c973c7c00 con 0x563c974bb6b0 2023-07-14T16:32:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.710+0000 7f0a4dffb700 1 -- 172.21.15.62:0/443929484 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c9750a490 con 0x563c974bb6b0 2023-07-14T16:32:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.711+0000 7f0a4dffb700 1 -- 172.21.15.62:0/443929484 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a60099490 con 0x563c974bb6b0 2023-07-14T16:32:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.711+0000 7f0a807bdb80 10 client.20160 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.711+0000 7f0a807bdb80 10 client.20160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.711+0000 7f0a807bdb80 20 client.20160 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.711+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c97674170 0x563c97676550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.711+0000 7f0a807bdb80 4 client.20160 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.711+0000 7f0a807bdb80 1 -- 172.21.15.62:0/443929484 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c975a9e50 con 0x563c97674170 2023-07-14T16:32:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.711+0000 7f0a65d9b700 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c97674170 0x563c97676550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.712+0000 7f0a4dffb700 10 client.20160 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.712+0000 7f0a65d9b700 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c97674170 0x563c97676550 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b4550287a0f9b7c server_cookie=76c519e3b13ae9fb in_seq=0 out_seq=0 2023-07-14T16:32:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a4dffb700 1 -- 172.21.15.62:0/443929484 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c975a9e50 con 0x563c97674170 2023-07-14T16:32:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a4dffb700 10 client.20160 handle_command_reply: tid=0 2023-07-14T16:32:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a807bdb80 1 client.20160 shutdown 2023-07-14T16:32:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a807bdb80 1 -- 172.21.15.62:0/443929484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 msgr2=0x563c976687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c976687f0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0a600964a0 tx=0x7f0a600d24b0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a807bdb80 1 -- 172.21.15.62:0/443929484 shutdown_connections 2023-07-14T16:32:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c97674170 0x563c97676550 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 0x563c9766fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c976687f0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.713+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/443929484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c97668d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.714+0000 7f0a807bdb80 1 -- 172.21.15.62:0/443929484 >> 172.21.15.62:0/443929484 conn(0x563c97654be0 msgr2=0x563c976594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.714+0000 7f0a4dffb700 0 client.20160 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.714+0000 7f0a807bdb80 1 -- 172.21.15.62:0/443929484 shutdown_connections 2023-07-14T16:32:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.714+0000 7f0a807bdb80 1 -- 172.21.15.62:0/443929484 wait complete. 2023-07-14T16:32:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.714+0000 7f0a807bdb80 20 client.20160 trim_cache size 0 max 16384 2023-07-14T16:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.736+0000 7f0a807bdb80 1 Processor -- start 2023-07-14T16:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a807bdb80 1 -- start start 2023-07-14T16:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9767aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c9766fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c976606e0 0x563c976669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c976683c0 con 0x563c974bb6b0 2023-07-14T16:32:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97666760 con 0x563c9765f840 2023-07-14T16:32:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97664050 con 0x563c976606e0 2023-07-14T16:32:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a6659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9767aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a6659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9767aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56446/0 (socket says 172.21.15.62:56446) 2023-07-14T16:32:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.737+0000 7f0a6659c700 1 -- 172.21.15.62:0/4223725955 learned_addr learned my addr 172.21.15.62:0/4223725955 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.738+0000 7f0a65d9b700 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c9766fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.738+0000 7f0a6659c700 1 -- 172.21.15.62:0/4223725955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c976606e0 msgr2=0x563c976669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.738+0000 7f0a6659c700 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c976606e0 0x563c976669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.738+0000 7f0a6659c700 1 -- 172.21.15.62:0/4223725955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 msgr2=0x563c9766fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.738+0000 7f0a6659c700 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c9766fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.738+0000 7f0a6659c700 1 -- 172.21.15.62:0/4223725955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c974051f0 con 0x563c974bb6b0 2023-07-14T16:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.738+0000 7f0a65d9b700 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c9766fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.738+0000 7f0a6659c700 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9767aeb0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0a600997c0 tx=0x7f0a6000c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336d791d629fd314 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.739+0000 7f0a5d7fa700 1 -- 172.21.15.62:0/4223725955 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a600ccc30 con 0x563c974bb6b0 2023-07-14T16:32:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.739+0000 7f0a5d7fa700 1 -- 172.21.15.62:0/4223725955 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a6009a4a0 con 0x563c974bb6b0 2023-07-14T16:32:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.739+0000 7f0a5d7fa700 1 -- 172.21.15.62:0/4223725955 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60095e10 con 0x563c974bb6b0 2023-07-14T16:32:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.740+0000 7f0a807bdb80 1 -- 172.21.15.62:0/4223725955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 msgr2=0x563c9767aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.740+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9767aeb0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0a600997c0 tx=0x7f0a6000c720 comp rx=0 tx=0).stop 2023-07-14T16:32:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.740+0000 7f0a807bdb80 1 -- 172.21.15.62:0/4223725955 shutdown_connections 2023-07-14T16:32:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.740+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c976606e0 0x563c976669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.740+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9767aeb0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.740+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/4223725955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c9765f840 0x563c9766fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.740+0000 7f0a807bdb80 1 -- 172.21.15.62:0/4223725955 >> 172.21.15.62:0/4223725955 conn(0x563c97654be0 msgr2=0x563c97667560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.741+0000 7f0a807bdb80 1 -- 172.21.15.62:0/4223725955 shutdown_connections 2023-07-14T16:32:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.741+0000 7f0a807bdb80 1 -- 172.21.15.62:0/4223725955 wait complete. 2023-07-14T16:32:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.741+0000 7f0a807bdb80 1 -- start start 2023-07-14T16:32:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.742+0000 7f0a807bdb80 1 Processor -- start 2023-07-14T16:32:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.742+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9766b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.742+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c9766b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.742+0000 7f0a807bdb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 0x563c9766e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.742+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9766ede0 con 0x563c974bb6b0 2023-07-14T16:32:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.742+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9766ef50 con 0x563c976606e0 2023-07-14T16:32:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.742+0000 7f0a807bdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9766f0c0 con 0x563c9765f840 2023-07-14T16:32:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.743+0000 7f0a807bdb80 10 client.0 _ll_register_callbacks cb 0x7f0a60096370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.743+0000 7f0a65d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c9766b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.743+0000 7f0a65d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c9766b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:60260/0 (socket says 172.21.15.62:60260) 2023-07-14T16:32:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.743+0000 7f0a65d9b700 1 -- 172.21.15.62:0/3773600316 learned_addr learned my addr 172.21.15.62:0/3773600316 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.743+0000 7f0a66d9d700 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 0x563c9766e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.743+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9766b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.744+0000 7f0a65d9b700 1 -- 172.21.15.62:0/3773600316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 msgr2=0x563c9766e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.744+0000 7f0a65d9b700 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 0x563c9766e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.744+0000 7f0a65d9b700 1 -- 172.21.15.62:0/3773600316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 msgr2=0x563c9766b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.744+0000 7f0a65d9b700 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9766b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.744+0000 7f0a65d9b700 1 -- 172.21.15.62:0/3773600316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c973c7c00 con 0x563c9765f840 2023-07-14T16:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.744+0000 7f0a65d9b700 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c9766b880 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a58000f80 tx=0x7f0a58009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea27ba96042e00 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.744+0000 7f0a4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.744+0000 7f0a4dffb700 1 -- 172.21.15.62:0/3773600316 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5801d070 con 0x563c9765f840 2023-07-14T16:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a4dffb700 1 -- 172.21.15.62:0/3773600316 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a5801d7f0 con 0x563c9765f840 2023-07-14T16:32:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3773600316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a60099ae0 con 0x563c9765f840 2023-07-14T16:32:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a4dffb700 1 -- 172.21.15.62:0/3773600316 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a5801d5e0 con 0x563c9765f840 2023-07-14T16:32:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a807bdb80 10 client.29569 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3773600316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563c973dd250 con 0x563c9765f840 2023-07-14T16:32:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a4dffb700 1 -- 172.21.15.62:0/3773600316 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a60099ae0 con 0x563c9765f840 2023-07-14T16:32:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a4dffb700 1 -- 172.21.15.62:0/3773600316 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a58008b40 con 0x563c9765f840 2023-07-14T16:32:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a807bdb80 10 client.29569 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a807bdb80 10 client.29569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a807bdb80 20 client.29569 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.745+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c976d98a0 0x563c976dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.746+0000 7f0a807bdb80 4 client.29569 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:32:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.746+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3773600316 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c9756bff0 con 0x563c976d98a0 2023-07-14T16:32:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.746+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c976d98a0 0x563c976dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.746+0000 7f0a6659c700 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c976d98a0 0x563c976dbcb0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d58fa2a63e0c94 server_cookie=67fd93fa5207672d in_seq=0 out_seq=0 2023-07-14T16:32:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.746+0000 7f0a4dffb700 10 client.29569 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.979+0000 7f0a4dffb700 1 -- 172.21.15.62:0/3773600316 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c9756bff0 con 0x563c976d98a0 2023-07-14T16:32:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.979+0000 7f0a4dffb700 10 client.29569 handle_command_reply: tid=0 2023-07-14T16:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.979+0000 7f0a807bdb80 1 client.29569 shutdown 2023-07-14T16:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3773600316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 msgr2=0x563c9766b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c9766b880 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a58000f80 tx=0x7f0a58009c00 comp rx=0 tx=0).stop 2023-07-14T16:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3773600316 shutdown_connections 2023-07-14T16:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c976d98a0 0x563c976dbcb0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c976606e0 0x563c9766e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c974bb6b0 0x563c9766b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 --2- 172.21.15.62:0/3773600316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c9765f840 0x563c9766b880 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3773600316 >> 172.21.15.62:0/3773600316 conn(0x563c97654be0 msgr2=0x563c9767aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a4dffb700 0 client.29569 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.980+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3773600316 shutdown_connections 2023-07-14T16:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.981+0000 7f0a807bdb80 1 -- 172.21.15.62:0/3773600316 wait complete. 2023-07-14T16:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.981+0000 7f0a807bdb80 20 client.29569 trim_cache size 0 max 16384 2023-07-14T16:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a540a12a0 msgr2=0x7f0a540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a540a12a0 0x7f0a540a3750 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000b5c0 tx=0x7f0a50005af0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3917947222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a680fea00 msgr2=0x7f0a6806aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a680fea00 0x7f0a6806aa60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000e9c0 tx=0x7f0a6000ecd0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3917947222 shutdown_connections 2023-07-14T16:32:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a68100640 0x7f0a6806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a680fea00 0x7f0a6806aa60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a540a12a0 0x7f0a540a3750 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.982+0000 7f0a6cdff700 1 --2- 172.21.15.62:0/3917947222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a680ffc90 0x7f0a6806afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.983+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3917947222 >> 172.21.15.62:0/3917947222 conn(0x7f0a680fa000 msgr2=0x7f0a680fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.983+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3917947222 shutdown_connections 2023-07-14T16:32:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:11.983+0000 7f0a6cdff700 1 -- 172.21.15.62:0/3917947222 wait complete. 2023-07-14T16:32:13.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:12 smithi062 ceph-mon[95543]: pgmap v854: 97 pgs: 97 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 3.36k op/s 2023-07-14T16:32:13.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:12 smithi081 ceph-mon[127795]: pgmap v854: 97 pgs: 97 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 3.36k op/s 2023-07-14T16:32:13.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:12 smithi164 ceph-mon[97427]: pgmap v854: 97 pgs: 97 active+clean; 13 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 3.36k op/s 2023-07-14T16:32:15.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:14 smithi062 ceph-mon[95543]: pgmap v855: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 18 MiB/s wr, 4.33k op/s 2023-07-14T16:32:15.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:14 smithi081 ceph-mon[127795]: pgmap v855: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 18 MiB/s wr, 4.33k op/s 2023-07-14T16:32:15.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:14 smithi164 ceph-mon[97427]: pgmap v855: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 18 MiB/s wr, 4.33k op/s 2023-07-14T16:32:17.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:16 smithi062 ceph-mon[95543]: pgmap v856: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:32:17.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:16 smithi081 ceph-mon[127795]: pgmap v856: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:32:17.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:16 smithi164 ceph-mon[97427]: pgmap v856: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:32:19.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:18 smithi062 ceph-mon[95543]: pgmap v857: 97 pgs: 97 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:32:19.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:18 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:32:19.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:18 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:32:19.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:18 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:32:19.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:18 smithi081 ceph-mon[127795]: pgmap v857: 97 pgs: 97 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:32:19.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:18 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:32:19.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:18 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:32:19.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:18 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:32:19.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:18 smithi164 ceph-mon[97427]: pgmap v857: 97 pgs: 97 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:32:19.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:18 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:32:19.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:18 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:32:19.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:18 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:32:21.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:20 smithi062 ceph-mon[95543]: pgmap v858: 97 pgs: 97 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 5.23k op/s 2023-07-14T16:32:21.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:20 smithi081 ceph-mon[127795]: pgmap v858: 97 pgs: 97 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 5.23k op/s 2023-07-14T16:32:21.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:20 smithi164 ceph-mon[97427]: pgmap v858: 97 pgs: 97 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 5.23k op/s 2023-07-14T16:32:23.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:22 smithi062 ceph-mon[95543]: pgmap v859: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2023-07-14T16:32:23.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:22 smithi081 ceph-mon[127795]: pgmap v859: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2023-07-14T16:32:23.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:22 smithi164 ceph-mon[97427]: pgmap v859: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 4.73k op/s 2023-07-14T16:32:25.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:24 smithi062 ceph-mon[95543]: pgmap v860: 97 pgs: 97 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2023-07-14T16:32:25.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:24 smithi081 ceph-mon[127795]: pgmap v860: 97 pgs: 97 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2023-07-14T16:32:25.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:24 smithi164 ceph-mon[97427]: pgmap v860: 97 pgs: 97 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 5.33k op/s 2023-07-14T16:32:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.315+0000 7f11877ff700 1 -- 172.21.15.62:0/788815302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 msgr2=0x7f11801026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.315+0000 7f11877ff700 1 --2- 172.21.15.62:0/788815302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 0x7f11801026d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f117c013ef0 tx=0x7f117c0097f0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.316+0000 7f11877ff700 1 -- 172.21.15.62:0/788815302 shutdown_connections 2023-07-14T16:32:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.316+0000 7f11877ff700 1 --2- 172.21.15.62:0/788815302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11801056e0 0x7f1180107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.316+0000 7f11877ff700 1 --2- 172.21.15.62:0/788815302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 0x7f11801026d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.316+0000 7f11877ff700 1 --2- 172.21.15.62:0/788815302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1180102c10 0x7f1180105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.316+0000 7f11877ff700 1 -- 172.21.15.62:0/788815302 >> 172.21.15.62:0/788815302 conn(0x7f11800fa000 msgr2=0x7f11800fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.316+0000 7f11877ff700 1 -- 172.21.15.62:0/788815302 shutdown_connections 2023-07-14T16:32:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.316+0000 7f11877ff700 1 -- 172.21.15.62:0/788815302 wait complete. 2023-07-14T16:32:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.318+0000 7f11877ff700 1 Processor -- start 2023-07-14T16:32:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.318+0000 7f11877ff700 1 -- start start 2023-07-14T16:32:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f11877ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 0x7f11801a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f11877ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1180102c10 0x7f11801a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f11877ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11801056e0 0x7f11801a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f11877ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11801a60e0 con 0x7f11801056e0 2023-07-14T16:32:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f11877ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11801a6250 con 0x7f1180068600 2023-07-14T16:32:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f11877ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11801a6410 con 0x7f1180102c10 2023-07-14T16:32:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f1184d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1180102c10 0x7f11801a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f1184d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1180102c10 0x7f11801a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50852/0 (socket says 172.21.15.62:50852) 2023-07-14T16:32:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f1184d9a700 1 -- 172.21.15.62:0/2380322189 learned_addr learned my addr 172.21.15.62:0/2380322189 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f118559b700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 0x7f11801a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.319+0000 7f1185d9c700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11801056e0 0x7f11801a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.320+0000 7f118559b700 1 -- 172.21.15.62:0/2380322189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1180102c10 msgr2=0x7f11801a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.320+0000 7f118559b700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1180102c10 0x7f11801a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.320+0000 7f118559b700 1 -- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11801056e0 msgr2=0x7f11801a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.320+0000 7f118559b700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11801056e0 0x7f11801a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.320+0000 7f118559b700 1 -- 172.21.15.62:0/2380322189 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c013b50 con 0x7f1180068600 2023-07-14T16:32:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.320+0000 7f118559b700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 0x7f11801a46c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f117c013e40 tx=0x7f117c013a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.321+0000 7f11727fc700 1 -- 172.21.15.62:0/2380322189 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0050d0 con 0x7f1180068600 2023-07-14T16:32:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.321+0000 7f11727fc700 1 -- 172.21.15.62:0/2380322189 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f117c0056c0 con 0x7f1180068600 2023-07-14T16:32:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.321+0000 7f11877ff700 1 -- 172.21.15.62:0/2380322189 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11801a6690 con 0x7f1180068600 2023-07-14T16:32:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.321+0000 7f11727fc700 1 -- 172.21.15.62:0/2380322189 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00f450 con 0x7f1180068600 2023-07-14T16:32:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.321+0000 7f11877ff700 1 -- 172.21.15.62:0/2380322189 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11801aceb0 con 0x7f1180068600 2023-07-14T16:32:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.329+0000 7f11727fc700 1 -- 172.21.15.62:0/2380322189 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f117c01b6a0 con 0x7f1180068600 2023-07-14T16:32:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.330+0000 7f119b196b80 1 Processor -- start 2023-07-14T16:32:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.331+0000 7f119b196b80 1 -- start start 2023-07-14T16:32:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.332+0000 7f119b196b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e411c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.332+0000 7f119b196b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411c840 0x5557e411d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.332+0000 7f119b196b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e411db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.332+0000 7f119b196b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e4021c50 con 0x5557e3f786b0 2023-07-14T16:32:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.332+0000 7f119b196b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e40749e0 con 0x5557e411d6e0 2023-07-14T16:32:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.332+0000 7f119b196b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e406de70 con 0x5557e411c840 2023-07-14T16:32:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e411db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e411db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53956/0 (socket says 172.21.15.62:53956) 2023-07-14T16:32:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 -- 172.21.15.62:0/4281386831 learned_addr learned my addr 172.21.15.62:0/4281386831 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f118559b700 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411c840 0x5557e411d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1185d9c700 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e411c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 -- 172.21.15.62:0/4281386831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411c840 msgr2=0x5557e411d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411c840 0x5557e411d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 -- 172.21.15.62:0/4281386831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 msgr2=0x5557e411c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e411c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 -- 172.21.15.62:0/4281386831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557e3e9bd80 con 0x5557e411d6e0 2023-07-14T16:32:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.333+0000 7f1184d9a700 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e411db80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f117400ae40 tx=0x7f117400a4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12939a68e6aeb20f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.334+0000 7f11727fc700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f116c0a1350 0x7f116c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.334+0000 7f116b7fe700 1 -- 172.21.15.62:0/4281386831 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174012040 con 0x5557e411d6e0 2023-07-14T16:32:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.334+0000 7f116b7fe700 1 -- 172.21.15.62:0/4281386831 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11740044c0 con 0x5557e411d6e0 2023-07-14T16:32:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.334+0000 7f116b7fe700 1 -- 172.21.15.62:0/4281386831 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174016420 con 0x5557e411d6e0 2023-07-14T16:32:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.334+0000 7f11727fc700 1 -- 172.21.15.62:0/2380322189 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f117c0cdc90 con 0x7f1180068600 2023-07-14T16:32:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.335+0000 7f1185d9c700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f116c0a1350 0x7f116c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.335+0000 7f119b196b80 1 -- 172.21.15.62:0/4281386831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 msgr2=0x5557e411db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.335+0000 7f119b196b80 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e411db80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f117400ae40 tx=0x7f117400a4f0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.336+0000 7f1185d9c700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f116c0a1350 0x7f116c0a3800 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5557e411c060 tx=0x7f117800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.336+0000 7f119b196b80 1 -- 172.21.15.62:0/4281386831 shutdown_connections 2023-07-14T16:32:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.336+0000 7f119b196b80 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e411db80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.336+0000 7f119b196b80 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e411c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.336+0000 7f119b196b80 1 --2- 172.21.15.62:0/4281386831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411c840 0x5557e411d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.336+0000 7f119b196b80 1 -- 172.21.15.62:0/4281386831 >> 172.21.15.62:0/4281386831 conn(0x5557e4111be0 msgr2=0x5557e4116cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.336+0000 7f119b196b80 1 -- 172.21.15.62:0/4281386831 shutdown_connections 2023-07-14T16:32:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.336+0000 7f119b196b80 1 -- 172.21.15.62:0/4281386831 wait complete. 2023-07-14T16:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.337+0000 7f119b196b80 1 -- start start 2023-07-14T16:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.338+0000 7f119b196b80 1 Processor -- start 2023-07-14T16:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.338+0000 7f119b196b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e41368c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.338+0000 7f119b196b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e4136e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.338+0000 7f119b196b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e413de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.338+0000 7f119b196b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e400da60 con 0x5557e3f786b0 2023-07-14T16:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.338+0000 7f119b196b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e400d760 con 0x5557e411c840 2023-07-14T16:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.338+0000 7f119b196b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e3ef18c0 con 0x5557e411d6e0 2023-07-14T16:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.338+0000 7f119b196b80 10 client.0 _ll_register_callbacks cb 0x5557e400b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.339+0000 7f1185d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e413de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.339+0000 7f1185d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e413de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50856/0 (socket says 172.21.15.62:50856) 2023-07-14T16:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.339+0000 7f1185d9c700 1 -- 172.21.15.62:0/3987675245 learned_addr learned my addr 172.21.15.62:0/3987675245 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.340+0000 7f1184d9a700 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e4136e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.340+0000 7f1185d9c700 1 -- 172.21.15.62:0/3987675245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 msgr2=0x5557e4136e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.340+0000 7f1185d9c700 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e4136e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.340+0000 7f1185d9c700 1 -- 172.21.15.62:0/3987675245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 msgr2=0x5557e41368c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.340+0000 7f1185d9c700 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e41368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.340+0000 7f1185d9c700 1 -- 172.21.15.62:0/3987675245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117400aa70 con 0x5557e411d6e0 2023-07-14T16:32:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.340+0000 7f1185d9c700 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e413de40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1178016990 tx=0x7f1178004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98cb55b037a27ecb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.340+0000 7f1168ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:32:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.341+0000 7f1168ff9700 1 -- 172.21.15.62:0/3987675245 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117800e040 con 0x5557e411d6e0 2023-07-14T16:32:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.341+0000 7f1168ff9700 1 -- 172.21.15.62:0/3987675245 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1178015820 con 0x5557e411d6e0 2023-07-14T16:32:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.342+0000 7f119b196b80 1 -- 172.21.15.62:0/3987675245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557e3fc7490 con 0x5557e411d6e0 2023-07-14T16:32:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.342+0000 7f1168ff9700 1 -- 172.21.15.62:0/3987675245 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11780159e0 con 0x5557e411d6e0 2023-07-14T16:32:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.342+0000 7f119b196b80 10 client.29581 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.342+0000 7f119b196b80 1 -- 172.21.15.62:0/3987675245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5557e3e84c00 con 0x5557e411d6e0 2023-07-14T16:32:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.342+0000 7f1168ff9700 1 -- 172.21.15.62:0/3987675245 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5557e3fc7490 con 0x5557e411d6e0 2023-07-14T16:32:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.343+0000 7f1168ff9700 1 -- 172.21.15.62:0/3987675245 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1178023bd0 con 0x5557e411d6e0 2023-07-14T16:32:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.343+0000 7f119b196b80 10 client.29581 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.343+0000 7f119b196b80 10 client.29581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.343+0000 7f119b196b80 20 client.29581 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.343+0000 7f119b196b80 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5557e41427f0 0x5557e4144c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.343+0000 7f119b196b80 4 client.29581 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.343+0000 7f119b196b80 1 -- 172.21.15.62:0/3987675245 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557e4066e50 con 0x5557e41427f0 2023-07-14T16:32:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.344+0000 7f118559b700 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5557e41427f0 0x5557e4144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.344+0000 7f118559b700 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5557e41427f0 0x5557e4144c00 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee2cc2dcc0aaefae server_cookie=8d72fef8bbb939f7 in_seq=0 out_seq=0 2023-07-14T16:32:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.344+0000 7f1168ff9700 10 client.29581 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.345+0000 7f1168ff9700 1 -- 172.21.15.62:0/3987675245 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5557e4066e50 con 0x5557e41427f0 2023-07-14T16:32:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.345+0000 7f1168ff9700 10 client.29581 handle_command_reply: tid=0 2023-07-14T16:32:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.346+0000 7f119b196b80 1 client.29581 shutdown 2023-07-14T16:32:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 -- 172.21.15.62:0/3987675245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 msgr2=0x5557e413de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e413de40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1178016990 tx=0x7f1178004d50 comp rx=0 tx=0).stop 2023-07-14T16:32:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 -- 172.21.15.62:0/3987675245 shutdown_connections 2023-07-14T16:32:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e413de40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e41368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5557e41427f0 0x5557e4144c00 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 --2- 172.21.15.62:0/3987675245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e4136e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 -- 172.21.15.62:0/3987675245 >> 172.21.15.62:0/3987675245 conn(0x5557e4111be0 msgr2=0x5557e41164e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f1168ff9700 0 client.29581 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.347+0000 7f119b196b80 1 -- 172.21.15.62:0/3987675245 shutdown_connections 2023-07-14T16:32:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.350+0000 7f119b196b80 1 -- 172.21.15.62:0/3987675245 wait complete. 2023-07-14T16:32:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.351+0000 7f119b196b80 20 client.29581 trim_cache size 0 max 16384 2023-07-14T16:32:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.382+0000 7f119b196b80 1 Processor -- start 2023-07-14T16:32:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.382+0000 7f119b196b80 1 -- start start 2023-07-14T16:32:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.382+0000 7f119b196b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e4123a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.382+0000 7f119b196b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e413de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.382+0000 7f119b196b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e4120590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.382+0000 7f119b196b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e4113d00 con 0x5557e3f786b0 2023-07-14T16:32:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.382+0000 7f119b196b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e41184e0 con 0x5557e411c840 2023-07-14T16:32:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.382+0000 7f119b196b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e410ebb0 con 0x5557e411d6e0 2023-07-14T16:32:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1184d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e413de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1184d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e413de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:53974/0 (socket says 172.21.15.62:53974) 2023-07-14T16:32:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1184d9a700 1 -- 172.21.15.62:0/4285785781 learned_addr learned my addr 172.21.15.62:0/4285785781 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1185d9c700 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e4120590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f118559b700 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e4123a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1184d9a700 1 -- 172.21.15.62:0/4285785781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 msgr2=0x5557e4120590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1184d9a700 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e4120590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1184d9a700 1 -- 172.21.15.62:0/4285785781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 msgr2=0x5557e4123a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1184d9a700 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e4123a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.383+0000 7f1184d9a700 1 -- 172.21.15.62:0/4285785781 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557e3ec21f0 con 0x5557e411c840 2023-07-14T16:32:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.384+0000 7f1184d9a700 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e413de00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1174019000 tx=0x7f1174005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6158c1db8edd080c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.384+0000 7f116bfff700 1 -- 172.21.15.62:0/4285785781 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174012030 con 0x5557e411c840 2023-07-14T16:32:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.385+0000 7f116bfff700 1 -- 172.21.15.62:0/4285785781 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1174016ab0 con 0x5557e411c840 2023-07-14T16:32:25.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.385+0000 7f116bfff700 1 -- 172.21.15.62:0/4285785781 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174021a60 con 0x5557e411c840 2023-07-14T16:32:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.385+0000 7f119b196b80 1 -- 172.21.15.62:0/4285785781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 msgr2=0x5557e413de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.385+0000 7f119b196b80 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e413de00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1174019000 tx=0x7f1174005910 comp rx=0 tx=0).stop 2023-07-14T16:32:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 -- 172.21.15.62:0/4285785781 shutdown_connections 2023-07-14T16:32:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e411d6e0 0x5557e4120590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e3f786b0 0x5557e4123a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 --2- 172.21.15.62:0/4285785781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411c840 0x5557e413de00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.360 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 -- 172.21.15.62:0/4285785781 >> 172.21.15.62:0/4285785781 conn(0x5557e4111be0 msgr2=0x5557e4149560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 -- 172.21.15.62:0/4285785781 shutdown_connections 2023-07-14T16:32:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 -- 172.21.15.62:0/4285785781 wait complete. 2023-07-14T16:32:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 -- start start 2023-07-14T16:32:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.386+0000 7f119b196b80 1 Processor -- start 2023-07-14T16:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.387+0000 7f119b196b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e3f786b0 0x5557e4139f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.387+0000 7f119b196b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e411c840 0x5557e413a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.387+0000 7f119b196b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e413d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.387+0000 7f119b196b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e413d8f0 con 0x5557e411c840 2023-07-14T16:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.387+0000 7f119b196b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e413da60 con 0x5557e411d6e0 2023-07-14T16:32:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.387+0000 7f119b196b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e413dc20 con 0x5557e3f786b0 2023-07-14T16:32:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.387+0000 7f119b196b80 10 client.0 _ll_register_callbacks cb 0x5557e400b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.387+0000 7f118559b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e3f786b0 0x5557e4139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1184d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e411c840 0x5557e413a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1184d9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e411c840 0x5557e413a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44430/0 (socket says 172.21.15.62:44430) 2023-07-14T16:32:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1184d9a700 1 -- 172.21.15.62:0/1239779429 learned_addr learned my addr 172.21.15.62:0/1239779429 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1185d9c700 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e413d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1184d9a700 1 -- 172.21.15.62:0/1239779429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e3f786b0 msgr2=0x5557e4139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1184d9a700 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e3f786b0 0x5557e4139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1184d9a700 1 -- 172.21.15.62:0/1239779429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 msgr2=0x5557e413d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1184d9a700 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e413d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.388+0000 7f1184d9a700 1 -- 172.21.15.62:0/1239779429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557e3e9bd80 con 0x5557e411c840 2023-07-14T16:32:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.389+0000 7f1184d9a700 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e411c840 0x5557e413a4d0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f117400ad60 tx=0x7f1174018f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68cfae469263bfe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.389+0000 7f1168ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.390+0000 7f1168ff9700 1 -- 172.21.15.62:0/1239779429 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174012050 con 0x5557e411c840 2023-07-14T16:32:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.390+0000 7f1168ff9700 1 -- 172.21.15.62:0/1239779429 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1174016830 con 0x5557e411c840 2023-07-14T16:32:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.390+0000 7f119b196b80 1 -- 172.21.15.62:0/1239779429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11740044c0 con 0x5557e411c840 2023-07-14T16:32:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.391+0000 7f1168ff9700 1 -- 172.21.15.62:0/1239779429 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11740217e0 con 0x5557e411c840 2023-07-14T16:32:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.391+0000 7f119b196b80 10 client.20181 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.391+0000 7f119b196b80 1 -- 172.21.15.62:0/1239779429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5557e3e9a250 con 0x5557e411c840 2023-07-14T16:32:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f1168ff9700 1 -- 172.21.15.62:0/1239779429 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11740044c0 con 0x5557e411c840 2023-07-14T16:32:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f1168ff9700 1 -- 172.21.15.62:0/1239779429 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f117400dd20 con 0x5557e411c840 2023-07-14T16:32:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f119b196b80 10 client.20181 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f119b196b80 10 client.20181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f119b196b80 20 client.20181 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f119b196b80 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5557e41974d0 0x5557e41998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f119b196b80 4 client.20181 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:32:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f119b196b80 1 -- 172.21.15.62:0/1239779429 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5557e4028ff0 con 0x5557e41974d0 2023-07-14T16:32:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.392+0000 7f118559b700 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5557e41974d0 0x5557e41998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:25.367 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.393+0000 7f118559b700 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5557e41974d0 0x5557e41998e0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5f4062257b7531 server_cookie=dad0ac3e0878f4dc in_seq=0 out_seq=0 2023-07-14T16:32:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.393+0000 7f1168ff9700 10 client.20181 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.408+0000 7f1168ff9700 1 -- 172.21.15.62:0/1239779429 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5557e4028ff0 con 0x5557e41974d0 2023-07-14T16:32:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.408+0000 7f1168ff9700 10 client.20181 handle_command_reply: tid=0 2023-07-14T16:32:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.408+0000 7f119b196b80 1 client.20181 shutdown 2023-07-14T16:32:25.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.410+0000 7f119b196b80 1 -- 172.21.15.62:0/1239779429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e411c840 msgr2=0x5557e413a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.410+0000 7f119b196b80 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e411c840 0x5557e413a4d0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f117400ad60 tx=0x7f1174018f20 comp rx=0 tx=0).stop 2023-07-14T16:32:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.410+0000 7f119b196b80 1 -- 172.21.15.62:0/1239779429 shutdown_connections 2023-07-14T16:32:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.410+0000 7f119b196b80 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5557e41974d0 0x5557e41998e0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.410+0000 7f119b196b80 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e411d6e0 0x5557e413d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.410+0000 7f119b196b80 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557e3f786b0 0x5557e4139f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.410+0000 7f119b196b80 1 --2- 172.21.15.62:0/1239779429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557e411c840 0x5557e413a4d0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.410+0000 7f119b196b80 1 -- 172.21.15.62:0/1239779429 >> 172.21.15.62:0/1239779429 conn(0x5557e4111be0 msgr2=0x5557e413e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.411+0000 7f1168ff9700 0 client.20181 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.411+0000 7f119b196b80 1 -- 172.21.15.62:0/1239779429 shutdown_connections 2023-07-14T16:32:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.411+0000 7f119b196b80 1 -- 172.21.15.62:0/1239779429 wait complete. 2023-07-14T16:32:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.412+0000 7f119b196b80 20 client.20181 trim_cache size 0 max 16384 2023-07-14T16:32:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:32:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:32:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:32:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:32:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.412+0000 7f11877ff700 1 -- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f116c0a1350 msgr2=0x7f116c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.412+0000 7f11877ff700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f116c0a1350 0x7f116c0a3800 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5557e411c060 tx=0x7f117800b040 comp rx=0 tx=0).stop 2023-07-14T16:32:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 -- 172.21.15.62:0/2380322189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 msgr2=0x7f11801a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 0x7f11801a46c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f117c013e40 tx=0x7f117c013a90 comp rx=0 tx=0).stop 2023-07-14T16:32:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 -- 172.21.15.62:0/2380322189 shutdown_connections 2023-07-14T16:32:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11801056e0 0x7f11801a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f116c0a1350 0x7f116c0a3800 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1180068600 0x7f11801a46c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 --2- 172.21.15.62:0/2380322189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1180102c10 0x7f11801a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 -- 172.21.15.62:0/2380322189 >> 172.21.15.62:0/2380322189 conn(0x7f11800fa000 msgr2=0x7f11801064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 -- 172.21.15.62:0/2380322189 shutdown_connections 2023-07-14T16:32:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:25.413+0000 7f11877ff700 1 -- 172.21.15.62:0/2380322189 wait complete. 2023-07-14T16:32:27.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:26 smithi062 ceph-mon[95543]: pgmap v861: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:32:27.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:26 smithi081 ceph-mon[127795]: pgmap v861: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:32:27.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:26 smithi164 ceph-mon[97427]: pgmap v861: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:32:29.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:28 smithi062 ceph-mon[95543]: pgmap v862: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 5.40k op/s 2023-07-14T16:32:29.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:28 smithi081 ceph-mon[127795]: pgmap v862: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 5.40k op/s 2023-07-14T16:32:29.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:28 smithi164 ceph-mon[97427]: pgmap v862: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 MiB/s wr, 5.40k op/s 2023-07-14T16:32:31.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:30 smithi081 ceph-mon[127795]: pgmap v863: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 5.41k op/s 2023-07-14T16:32:31.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:30 smithi164 ceph-mon[97427]: pgmap v863: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 5.41k op/s 2023-07-14T16:32:31.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:30 smithi062 ceph-mon[95543]: pgmap v863: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 5.41k op/s 2023-07-14T16:32:33.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:32 smithi081 ceph-mon[127795]: pgmap v864: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.84k op/s 2023-07-14T16:32:33.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:32 smithi164 ceph-mon[97427]: pgmap v864: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.84k op/s 2023-07-14T16:32:33.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:32 smithi062 ceph-mon[95543]: pgmap v864: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.84k op/s 2023-07-14T16:32:34.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:32:35.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:34 smithi062 ceph-mon[95543]: pgmap v865: 97 pgs: 97 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 5.48k op/s 2023-07-14T16:32:35.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:34 smithi081 ceph-mon[127795]: pgmap v865: 97 pgs: 97 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 5.48k op/s 2023-07-14T16:32:35.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:34 smithi164 ceph-mon[97427]: pgmap v865: 97 pgs: 97 active+clean; 13 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 5.48k op/s 2023-07-14T16:32:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.448+0000 7f8a533ff700 1 -- 172.21.15.62:0/1017962749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 msgr2=0x7f8a4c10e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.448+0000 7f8a533ff700 1 --2- 172.21.15.62:0/1017962749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 0x7f8a4c10e4f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009f90 tx=0x7f8a4400b330 comp rx=0 tx=0).stop 2023-07-14T16:32:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.448+0000 7f8a533ff700 1 -- 172.21.15.62:0/1017962749 shutdown_connections 2023-07-14T16:32:35.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.448+0000 7f8a533ff700 1 --2- 172.21.15.62:0/1017962749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 0x7f8a4c10e4f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.448+0000 7f8a533ff700 1 --2- 172.21.15.62:0/1017962749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4c074550 0x7f8a4c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.448+0000 7f8a533ff700 1 --2- 172.21.15.62:0/1017962749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c072f70 0x7f8a4c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.448+0000 7f8a533ff700 1 -- 172.21.15.62:0/1017962749 >> 172.21.15.62:0/1017962749 conn(0x7f8a4c0fb160 msgr2=0x7f8a4c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:35.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.449+0000 7f8a533ff700 1 -- 172.21.15.62:0/1017962749 shutdown_connections 2023-07-14T16:32:35.422 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.449+0000 7f8a533ff700 1 -- 172.21.15.62:0/1017962749 wait complete. 2023-07-14T16:32:35.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.451+0000 7f8a533ff700 1 Processor -- start 2023-07-14T16:32:35.423 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.451+0000 7f8a533ff700 1 -- start start 2023-07-14T16:32:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a533ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4c072f70 0x7f8a4c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a533ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c074550 0x7f8a4c06f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a533ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 0x7f8a4c1abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a533ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c06faa0 con 0x7f8a4c074550 2023-07-14T16:32:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a533ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c1ac460 con 0x7f8a4c10c110 2023-07-14T16:32:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a533ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c1ac5a0 con 0x7f8a4c072f70 2023-07-14T16:32:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a5199c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 0x7f8a4c1abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a5199c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 0x7f8a4c1abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40450/0 (socket says 172.21.15.62:40450) 2023-07-14T16:32:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.452+0000 7f8a5199c700 1 -- 172.21.15.62:0/3450162067 learned_addr learned my addr 172.21.15.62:0/3450162067 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.453+0000 7f8a5099a700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c074550 0x7f8a4c06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.453+0000 7f8a5199c700 1 -- 172.21.15.62:0/3450162067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4c072f70 msgr2=0x7f8a4c06ef00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T16:32:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.453+0000 7f8a5199c700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4c072f70 0x7f8a4c06ef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.453+0000 7f8a5199c700 1 -- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c074550 msgr2=0x7f8a4c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.453+0000 7f8a5199c700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c074550 0x7f8a4c06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.453+0000 7f8a5199c700 1 -- 172.21.15.62:0/3450162067 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44009c40 con 0x7f8a4c10c110 2023-07-14T16:32:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.453+0000 7f8a5199c700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 0x7f8a4c1abf20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a44008d10 tx=0x7f8a44008df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.454+0000 7f8a427fc700 1 -- 172.21.15.62:0/3450162067 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400e040 con 0x7f8a4c10c110 2023-07-14T16:32:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.454+0000 7f8a427fc700 1 -- 172.21.15.62:0/3450162067 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a4400be00 con 0x7f8a4c10c110 2023-07-14T16:32:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.454+0000 7f8a427fc700 1 -- 172.21.15.62:0/3450162067 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401b7e0 con 0x7f8a4c10c110 2023-07-14T16:32:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.454+0000 7f8a533ff700 1 -- 172.21.15.62:0/3450162067 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c1ac7a0 con 0x7f8a4c10c110 2023-07-14T16:32:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.454+0000 7f8a533ff700 1 -- 172.21.15.62:0/3450162067 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c1acb70 con 0x7f8a4c10c110 2023-07-14T16:32:35.432 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.459+0000 7f8a427fc700 1 -- 172.21.15.62:0/3450162067 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8a4401d470 con 0x7f8a4c10c110 2023-07-14T16:32:35.433 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.461+0000 7f8a66da3b80 1 Processor -- start 2023-07-14T16:32:35.434 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.461+0000 7f8a66da3b80 1 -- start start 2023-07-14T16:32:35.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.461+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557774ffb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x557774ffc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffcc70 0x557774ffd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774f00cb0 con 0x557774ffb4d0 2023-07-14T16:32:35.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774f53a40 con 0x557774ffcc70 2023-07-14T16:32:35.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774f4ced0 con 0x557774ffbe20 2023-07-14T16:32:35.435 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5119b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x557774ffc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5119b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x557774ffc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53028/0 (socket says 172.21.15.62:53028) 2023-07-14T16:32:35.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5119b700 1 -- 172.21.15.62:0/3849826916 learned_addr learned my addr 172.21.15.62:0/3849826916 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:35.436 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5099a700 1 --2- 172.21.15.62:0/3849826916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffcc70 0x557774ffd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5119b700 1 -- 172.21.15.62:0/3849826916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffcc70 msgr2=0x557774ffd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5119b700 1 --2- 172.21.15.62:0/3849826916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffcc70 0x557774ffd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5119b700 1 -- 172.21.15.62:0/3849826916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 msgr2=0x557774ffb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5119b700 1 --2- 172.21.15.62:0/3849826916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557774ffb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.462+0000 7f8a5119b700 1 -- 172.21.15.62:0/3849826916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557774d7ade0 con 0x557774ffbe20 2023-07-14T16:32:35.437 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.463+0000 7f8a5119b700 1 --2- 172.21.15.62:0/3849826916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x557774ffc6e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800bae0 tx=0x7f8a4800bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64dd04cdc15a3b9b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:35.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.464+0000 7f8a427fc700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8a380a12d0 0x7f8a380a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.465+0000 7f8a377fe700 1 -- 172.21.15.62:0/3849826916 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800da90 con 0x557774ffbe20 2023-07-14T16:32:35.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.465+0000 7f8a427fc700 1 -- 172.21.15.62:0/3450162067 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8a440cda60 con 0x7f8a4c10c110 2023-07-14T16:32:35.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.465+0000 7f8a377fe700 1 -- 172.21.15.62:0/3849826916 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a48004cc0 con 0x557774ffbe20 2023-07-14T16:32:35.438 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.465+0000 7f8a377fe700 1 -- 172.21.15.62:0/3849826916 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48005520 con 0x557774ffbe20 2023-07-14T16:32:35.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.466+0000 7f8a66da3b80 1 -- 172.21.15.62:0/3849826916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 msgr2=0x557774ffc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.466+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/3849826916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x557774ffc6e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800bae0 tx=0x7f8a4800bdf0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.439 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.467+0000 7f8a5099a700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8a380a12d0 0x7f8a380a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.440 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.468+0000 7f8a5099a700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8a380a12d0 0x7f8a380a3780 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005950 tx=0x7f8a3c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:35.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.468+0000 7f8a66da3b80 1 -- 172.21.15.62:0/3849826916 shutdown_connections 2023-07-14T16:32:35.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.468+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/3849826916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffcc70 0x557774ffd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.468+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/3849826916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557774ffb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.468+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/3849826916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x557774ffc6e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.441 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.468+0000 7f8a66da3b80 1 -- 172.21.15.62:0/3849826916 >> 172.21.15.62:0/3849826916 conn(0x557774ff0da0 msgr2=0x557774ff61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:35.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.468+0000 7f8a66da3b80 1 -- 172.21.15.62:0/3849826916 shutdown_connections 2023-07-14T16:32:35.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.468+0000 7f8a66da3b80 1 -- 172.21.15.62:0/3849826916 wait complete. 2023-07-14T16:32:35.442 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.469+0000 7f8a66da3b80 1 -- start start 2023-07-14T16:32:35.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.469+0000 7f8a66da3b80 1 Processor -- start 2023-07-14T16:32:35.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffb4d0 0x557775005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x5577750056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.443 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774eecac0 con 0x557774ffcc70 2023-07-14T16:32:35.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774eec7c0 con 0x557774ffb4d0 2023-07-14T16:32:35.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774dd0920 con 0x557774ffbe20 2023-07-14T16:32:35.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a66da3b80 10 client.0 _ll_register_callbacks cb 0x557774eea760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:35.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a5199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.444 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.470+0000 7f8a5199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41552/0 (socket says 172.21.15.62:41552) 2023-07-14T16:32:35.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.471+0000 7f8a5199c700 1 -- 172.21.15.62:0/4134359759 learned_addr learned my addr 172.21.15.62:0/4134359759 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:35.445 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.472+0000 7f8a5099a700 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x5577750056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.472+0000 7f8a5199c700 1 -- 172.21.15.62:0/4134359759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 msgr2=0x5577750056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:35.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.472+0000 7f8a5199c700 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x5577750056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.472+0000 7f8a5199c700 1 -- 172.21.15.62:0/4134359759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffb4d0 msgr2=0x557775005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:35.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.472+0000 7f8a5199c700 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffb4d0 0x557775005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.446 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.472+0000 7f8a5199c700 1 -- 172.21.15.62:0/4134359759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4800b740 con 0x557774ffcc70 2023-07-14T16:32:35.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.472+0000 7f8a5199c700 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500c710 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8a440973b0 tx=0x7f8a4409a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fdaf1a83f18e74 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:35.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.472+0000 7f8a34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:35.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.473+0000 7f8a34ff9700 1 -- 172.21.15.62:0/4134359759 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4409a5a0 con 0x557774ffcc70 2023-07-14T16:32:35.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.473+0000 7f8a34ff9700 1 -- 172.21.15.62:0/4134359759 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8a440a3270 con 0x557774ffcc70 2023-07-14T16:32:35.447 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.473+0000 7f8a66da3b80 1 -- 172.21.15.62:0/4134359759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557774ea64f0 con 0x557774ffcc70 2023-07-14T16:32:35.448 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.473+0000 7f8a34ff9700 1 -- 172.21.15.62:0/4134359759 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a440a2de0 con 0x557774ffcc70 2023-07-14T16:32:35.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.474+0000 7f8a66da3b80 10 client.20190 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:35.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.474+0000 7f8a66da3b80 1 -- 172.21.15.62:0/4134359759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557774d63c60 con 0x557774ffcc70 2023-07-14T16:32:35.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.474+0000 7f8a34ff9700 1 -- 172.21.15.62:0/4134359759 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557774ea64f0 con 0x557774ffcc70 2023-07-14T16:32:35.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.474+0000 7f8a34ff9700 1 -- 172.21.15.62:0/4134359759 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a44096e30 con 0x557774ffcc70 2023-07-14T16:32:35.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.474+0000 7f8a66da3b80 10 client.20190 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:35.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.474+0000 7f8a66da3b80 10 client.20190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:35.449 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.474+0000 7f8a66da3b80 20 client.20190 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:35.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.475+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557775021b10 0x557775023f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.475+0000 7f8a66da3b80 4 client.20190 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:35.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.475+0000 7f8a66da3b80 1 -- 172.21.15.62:0/4134359759 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557774f45eb0 con 0x557775021b10 2023-07-14T16:32:35.450 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.475+0000 7f8a5119b700 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557775021b10 0x557775023f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.479+0000 7f8a5119b700 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557775021b10 0x557775023f20 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7b5417cdb16470 server_cookie=57da87cd38aaca9e in_seq=0 out_seq=0 2023-07-14T16:32:35.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.479+0000 7f8a34ff9700 10 client.20190 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:35.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.480+0000 7f8a34ff9700 1 -- 172.21.15.62:0/4134359759 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557774f45eb0 con 0x557775021b10 2023-07-14T16:32:35.452 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.480+0000 7f8a34ff9700 10 client.20190 handle_command_reply: tid=0 2023-07-14T16:32:35.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.480+0000 7f8a66da3b80 1 client.20190 shutdown 2023-07-14T16:32:35.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a66da3b80 1 -- 172.21.15.62:0/4134359759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 msgr2=0x55777500c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.453 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500c710 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8a440973b0 tx=0x7f8a4409a810 comp rx=0 tx=0).stop 2023-07-14T16:32:35.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a66da3b80 1 -- 172.21.15.62:0/4134359759 shutdown_connections 2023-07-14T16:32:35.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500c710 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557775021b10 0x557775023f20 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffb4d0 0x557775005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/4134359759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffbe20 0x5577750056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.454 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a66da3b80 1 -- 172.21.15.62:0/4134359759 >> 172.21.15.62:0/4134359759 conn(0x557774ff0da0 msgr2=0x557774ff58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.481+0000 7f8a34ff9700 0 client.20190 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.482+0000 7f8a66da3b80 1 -- 172.21.15.62:0/4134359759 shutdown_connections 2023-07-14T16:32:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.482+0000 7f8a66da3b80 1 -- 172.21.15.62:0/4134359759 wait complete. 2023-07-14T16:32:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.482+0000 7f8a66da3b80 20 client.20190 trim_cache size 0 max 16384 2023-07-14T16:32:35.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.520+0000 7f8a66da3b80 1 Processor -- start 2023-07-14T16:32:35.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.520+0000 7f8a66da3b80 1 -- start start 2023-07-14T16:32:35.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.520+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557775003090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.520+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 0x55777500c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.520+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffcc70 0x557774fffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.520+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557775004d60 con 0x557774ffb4d0 2023-07-14T16:32:35.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.520+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557775004640 con 0x557774ffbe20 2023-07-14T16:32:35.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.520+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557774ffc1f0 con 0x557774ffcc70 2023-07-14T16:32:35.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557775003090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5119b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557775003090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41556/0 (socket says 172.21.15.62:41556) 2023-07-14T16:32:35.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5119b700 1 -- 172.21.15.62:0/1604070732 learned_addr learned my addr 172.21.15.62:0/1604070732 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:35.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5199c700 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffcc70 0x557774fffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5099a700 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 0x55777500c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5199c700 1 -- 172.21.15.62:0/1604070732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 msgr2=0x55777500c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:35.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5199c700 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 0x55777500c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5199c700 1 -- 172.21.15.62:0/1604070732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 msgr2=0x557775003090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:35.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5199c700 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557775003090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.521+0000 7f8a5199c700 1 -- 172.21.15.62:0/1604070732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557774da1250 con 0x557774ffcc70 2023-07-14T16:32:35.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.522+0000 7f8a5119b700 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557775003090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:32:35.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.523+0000 7f8a5199c700 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffcc70 0x557774fffbc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8a440978d0 tx=0x7f8a44099460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a2add13df98284 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:35.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.523+0000 7f8a37fff700 1 -- 172.21.15.62:0/1604070732 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401dbd0 con 0x557774ffcc70 2023-07-14T16:32:35.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.523+0000 7f8a37fff700 1 -- 172.21.15.62:0/1604070732 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a440a0040 con 0x557774ffcc70 2023-07-14T16:32:35.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.523+0000 7f8a37fff700 1 -- 172.21.15.62:0/1604070732 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a440a3040 con 0x557774ffcc70 2023-07-14T16:32:35.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.524+0000 7f8a66da3b80 1 -- 172.21.15.62:0/1604070732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffcc70 msgr2=0x557774fffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.524+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffcc70 0x557774fffbc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8a440978d0 tx=0x7f8a44099460 comp rx=0 tx=0).stop 2023-07-14T16:32:35.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.524+0000 7f8a66da3b80 1 -- 172.21.15.62:0/1604070732 shutdown_connections 2023-07-14T16:32:35.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.524+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffcc70 0x557774fffbc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.524+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffb4d0 0x557775003090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.524+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/1604070732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 0x55777500c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.524+0000 7f8a66da3b80 1 -- 172.21.15.62:0/1604070732 >> 172.21.15.62:0/1604070732 conn(0x557774ff0da0 msgr2=0x557775003e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:35.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.525+0000 7f8a66da3b80 1 -- 172.21.15.62:0/1604070732 shutdown_connections 2023-07-14T16:32:35.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.525+0000 7f8a66da3b80 1 -- 172.21.15.62:0/1604070732 wait complete. 2023-07-14T16:32:35.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.525+0000 7f8a66da3b80 1 -- start start 2023-07-14T16:32:35.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.526+0000 7f8a66da3b80 1 Processor -- start 2023-07-14T16:32:35.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.526+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffb4d0 0x557775008400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.526+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 0x557775008940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.526+0000 7f8a66da3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.526+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777500bea0 con 0x557774ffcc70 2023-07-14T16:32:35.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.526+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777500c010 con 0x557774ffbe20 2023-07-14T16:32:35.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.526+0000 7f8a66da3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777500c1d0 con 0x557774ffb4d0 2023-07-14T16:32:35.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.526+0000 7f8a66da3b80 10 client.0 _ll_register_callbacks cb 0x7f8a4409ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:35.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.527+0000 7f8a5199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.527+0000 7f8a5199c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41560/0 (socket says 172.21.15.62:41560) 2023-07-14T16:32:35.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.527+0000 7f8a5199c700 1 -- 172.21.15.62:0/2023794965 learned_addr learned my addr 172.21.15.62:0/2023794965 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:35.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.527+0000 7f8a5119b700 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffb4d0 0x557775008400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.527+0000 7f8a5099a700 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 0x557775008940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a5199c700 1 -- 172.21.15.62:0/2023794965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffb4d0 msgr2=0x557775008400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:35.502 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a5199c700 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffb4d0 0x557775008400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a5199c700 1 -- 172.21.15.62:0/2023794965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 msgr2=0x557775008940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:35.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a5199c700 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 0x557775008940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a5199c700 1 -- 172.21.15.62:0/2023794965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a440b2030 con 0x557774ffcc70 2023-07-14T16:32:35.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a5119b700 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffb4d0 0x557775008400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:32:35.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a5199c700 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500b960 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8a4409ae40 tx=0x7f8a4409a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf6f49f489324a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:35.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:35.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.528+0000 7f8a34ff9700 1 -- 172.21.15.62:0/2023794965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401dbd0 con 0x557774ffcc70 2023-07-14T16:32:35.503 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.529+0000 7f8a34ff9700 1 -- 172.21.15.62:0/2023794965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8a4401d890 con 0x557774ffcc70 2023-07-14T16:32:35.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.529+0000 7f8a66da3b80 1 -- 172.21.15.62:0/2023794965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a48004cc0 con 0x557774ffcc70 2023-07-14T16:32:35.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.529+0000 7f8a34ff9700 1 -- 172.21.15.62:0/2023794965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a4409a5b0 con 0x557774ffcc70 2023-07-14T16:32:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.529+0000 7f8a66da3b80 10 client.20196 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.529+0000 7f8a66da3b80 1 -- 172.21.15.62:0/2023794965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557774d792b0 con 0x557774ffcc70 2023-07-14T16:32:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.530+0000 7f8a34ff9700 1 -- 172.21.15.62:0/2023794965 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a48004cc0 con 0x557774ffcc70 2023-07-14T16:32:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.530+0000 7f8a34ff9700 1 -- 172.21.15.62:0/2023794965 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a440c5210 con 0x557774ffcc70 2023-07-14T16:32:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.530+0000 7f8a66da3b80 10 client.20196 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.530+0000 7f8a66da3b80 10 client.20196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.530+0000 7f8a66da3b80 20 client.20196 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.530+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577750762e0 0x5577750786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.530+0000 7f8a66da3b80 4 client.20196 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:32:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.530+0000 7f8a66da3b80 1 -- 172.21.15.62:0/2023794965 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557774f08050 con 0x5577750762e0 2023-07-14T16:32:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.531+0000 7f8a5119b700 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577750762e0 0x5577750786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.533+0000 7f8a5119b700 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577750762e0 0x5577750786f0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75baa24c8901d09e server_cookie=3332ffcee5a596a7 in_seq=0 out_seq=0 2023-07-14T16:32:35.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.533+0000 7f8a34ff9700 10 client.20196 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:35.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.533+0000 7f8a34ff9700 1 -- 172.21.15.62:0/2023794965 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557774f08050 con 0x5577750762e0 2023-07-14T16:32:35.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.533+0000 7f8a34ff9700 10 client.20196 handle_command_reply: tid=0 2023-07-14T16:32:35.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.533+0000 7f8a66da3b80 1 client.20196 shutdown 2023-07-14T16:32:35.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.533+0000 7f8a66da3b80 1 -- 172.21.15.62:0/2023794965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 msgr2=0x55777500b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.533+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500b960 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8a4409ae40 tx=0x7f8a4409a810 comp rx=0 tx=0).stop 2023-07-14T16:32:35.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.534+0000 7f8a66da3b80 1 -- 172.21.15.62:0/2023794965 shutdown_connections 2023-07-14T16:32:35.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.534+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557774ffcc70 0x55777500b960 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.534+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557774ffb4d0 0x557775008400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.534+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5577750762e0 0x5577750786f0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.534+0000 7f8a66da3b80 1 --2- 172.21.15.62:0/2023794965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557774ffbe20 0x557775008940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.534+0000 7f8a66da3b80 1 -- 172.21.15.62:0/2023794965 >> 172.21.15.62:0/2023794965 conn(0x557774ff0da0 msgr2=0x557775003090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:35.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.534+0000 7f8a34ff9700 0 client.20196 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:35.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.535+0000 7f8a66da3b80 1 -- 172.21.15.62:0/2023794965 shutdown_connections 2023-07-14T16:32:35.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.535+0000 7f8a66da3b80 1 -- 172.21.15.62:0/2023794965 wait complete. 2023-07-14T16:32:35.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.536+0000 7f8a66da3b80 20 client.20196 trim_cache size 0 max 16384 2023-07-14T16:32:35.509 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:32:35.510 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:32:35.510 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:32:35.510 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:32:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 -- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8a380a12d0 msgr2=0x7f8a380a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8a380a12d0 0x7f8a380a3780 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005950 tx=0x7f8a3c0058e0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 -- 172.21.15.62:0/3450162067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 msgr2=0x7f8a4c1abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 0x7f8a4c1abf20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a44008d10 tx=0x7f8a44008df0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 -- 172.21.15.62:0/3450162067 shutdown_connections 2023-07-14T16:32:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f8a380a12d0 0x7f8a380a3780 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a4c10c110 0x7f8a4c1abf20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a4c072f70 0x7f8a4c06ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 --2- 172.21.15.62:0/3450162067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a4c074550 0x7f8a4c06f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 -- 172.21.15.62:0/3450162067 >> 172.21.15.62:0/3450162067 conn(0x7f8a4c0fb160 msgr2=0x7f8a4c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 -- 172.21.15.62:0/3450162067 shutdown_connections 2023-07-14T16:32:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.537+0000 7f8a533ff700 1 -- 172.21.15.62:0/3450162067 wait complete. 2023-07-14T16:32:35.542 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:32:35.542 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:32:35.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.899+0000 7f04217ff700 1 -- 172.21.15.62:0/2248047993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c067280 msgr2=0x7f041c0fe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.899+0000 7f04217ff700 1 --2- 172.21.15.62:0/2248047993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c067280 0x7f041c0fe6b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f0410009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.899+0000 7f04217ff700 1 -- 172.21.15.62:0/2248047993 shutdown_connections 2023-07-14T16:32:35.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.899+0000 7f04217ff700 1 --2- 172.21.15.62:0/2248047993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c0ff340 0x7f041c0ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.899+0000 7f04217ff700 1 --2- 172.21.15.62:0/2248047993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c066de0 0x7f041c0fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.899+0000 7f04217ff700 1 --2- 172.21.15.62:0/2248047993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c067280 0x7f041c0fe6b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.899+0000 7f04217ff700 1 -- 172.21.15.62:0/2248047993 >> 172.21.15.62:0/2248047993 conn(0x7f041c0f9770 msgr2=0x7f041c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:35.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.901+0000 7f04217ff700 1 -- 172.21.15.62:0/2248047993 shutdown_connections 2023-07-14T16:32:35.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.901+0000 7f04217ff700 1 -- 172.21.15.62:0/2248047993 wait complete. 2023-07-14T16:32:35.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.902+0000 7f04217ff700 1 Processor -- start 2023-07-14T16:32:35.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.902+0000 7f04217ff700 1 -- start start 2023-07-14T16:32:35.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.903+0000 7f04217ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c066de0 0x7f041c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.903+0000 7f04217ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c067280 0x7f041c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.903+0000 7f04217ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c0ff340 0x7f041c1a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.903+0000 7f04217ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1a3960 con 0x7f041c0ff340 2023-07-14T16:32:35.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.903+0000 7f04217ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1a3aa0 con 0x7f041c066de0 2023-07-14T16:32:35.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.903+0000 7f04217ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1a3c30 con 0x7f041c067280 2023-07-14T16:32:35.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c067280 0x7f041c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c067280 0x7f041c19c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53076/0 (socket says 172.21.15.62:53076) 2023-07-14T16:32:35.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041a7fc700 1 -- 172.21.15.62:0/1656623747 learned_addr learned my addr 172.21.15.62:0/1656623747 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:35.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041affd700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c066de0 0x7f041c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041b7fe700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c0ff340 0x7f041c1a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041a7fc700 1 -- 172.21.15.62:0/1656623747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c066de0 msgr2=0x7f041c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041a7fc700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c066de0 0x7f041c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041a7fc700 1 -- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c0ff340 msgr2=0x7f041c1a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:35.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041a7fc700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c0ff340 0x7f041c1a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:35.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.904+0000 7f041a7fc700 1 -- 172.21.15.62:0/1656623747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04100095f0 con 0x7f041c067280 2023-07-14T16:32:35.878 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.905+0000 7f041a7fc700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c067280 0x7f041c19c320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04100036f0 tx=0x7f041000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:35.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.905+0000 7f0403fff700 1 -- 172.21.15.62:0/1656623747 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100043b0 con 0x7f041c067280 2023-07-14T16:32:35.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.905+0000 7f0403fff700 1 -- 172.21.15.62:0/1656623747 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0410004510 con 0x7f041c067280 2023-07-14T16:32:35.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.905+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c1a3eb0 con 0x7f041c067280 2023-07-14T16:32:35.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.905+0000 7f0403fff700 1 -- 172.21.15.62:0/1656623747 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100156b0 con 0x7f041c067280 2023-07-14T16:32:35.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.905+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c1a42f0 con 0x7f041c067280 2023-07-14T16:32:35.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.907+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c108860 con 0x7f041c067280 2023-07-14T16:32:35.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.910+0000 7f0403fff700 1 -- 172.21.15.62:0/1656623747 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f041000d980 con 0x7f041c067280 2023-07-14T16:32:35.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.912+0000 7f0403fff700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04040a12d0 0x7f04040a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:35.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.912+0000 7f0403fff700 1 -- 172.21.15.62:0/1656623747 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f04100cdbc0 con 0x7f041c067280 2023-07-14T16:32:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.916+0000 7f041affd700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04040a12d0 0x7f04040a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.916+0000 7f0403fff700 1 -- 172.21.15.62:0/1656623747 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0410097350 con 0x7f041c067280 2023-07-14T16:32:35.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:35.917+0000 7f041affd700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04040a12d0 0x7f04040a3780 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f041c0ff1a0 tx=0x7f040c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:36.349 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:32:36.349 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:31:18.877417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:32:36.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.375+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f041c108a50 con 0x7f041c067280 2023-07-14T16:32:36.357 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.376+0000 7f0403fff700 1 -- 172.21.15.62:0/1656623747 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10828 (secure 0 0 0) 0x7f041001a090 con 0x7f041c067280 2023-07-14T16:32:36.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04040a12d0 msgr2=0x7f04040a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04040a12d0 0x7f04040a3780 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f041c0ff1a0 tx=0x7f040c00b410 comp rx=0 tx=0).stop 2023-07-14T16:32:36.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c067280 msgr2=0x7f041c19c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c067280 0x7f041c19c320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04100036f0 tx=0x7f041000ec90 comp rx=0 tx=0).stop 2023-07-14T16:32:36.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 shutdown_connections 2023-07-14T16:32:36.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f041c0ff340 0x7f041c1a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f04040a12d0 0x7f04040a3780 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f041c066de0 0x7f041c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 --2- 172.21.15.62:0/1656623747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041c067280 0x7f041c19c320 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 >> 172.21.15.62:0/1656623747 conn(0x7f041c0f9770 msgr2=0x7f041c0fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:36.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 shutdown_connections 2023-07-14T16:32:36.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.381+0000 7f04217ff700 1 -- 172.21.15.62:0/1656623747 wait complete. 2023-07-14T16:32:36.362 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2023-07-14T16:32:36.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:32:36.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.830+0000 7f1adc9ff700 1 -- 172.21.15.62:0/4016519666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad80fea20 msgr2=0x7f1ad80fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.830+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/4016519666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad80fea20 0x7f1ad80fee30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009990 tx=0x7f1ac8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.831+0000 7f1adc9ff700 1 -- 172.21.15.62:0/4016519666 shutdown_connections 2023-07-14T16:32:36.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.831+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/4016519666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad81005a0 0x7f1ad8068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.831+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/4016519666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad80fea20 0x7f1ad80fee30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.831+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/4016519666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad80ffc10 0x7f1ad8100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.831+0000 7f1adc9ff700 1 -- 172.21.15.62:0/4016519666 >> 172.21.15.62:0/4016519666 conn(0x7f1ad80fa020 msgr2=0x7f1ad80fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.831+0000 7f1adc9ff700 1 -- 172.21.15.62:0/4016519666 shutdown_connections 2023-07-14T16:32:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.833+0000 7f1adc9ff700 1 -- 172.21.15.62:0/4016519666 wait complete. 2023-07-14T16:32:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.835+0000 7f1adc9ff700 1 Processor -- start 2023-07-14T16:32:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.835+0000 7f1adc9ff700 1 -- start start 2023-07-14T16:32:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.836+0000 7f1adc9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad80fea20 0x7f1ad810e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.836+0000 7f1adc9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad80ffc10 0x7f1ad810ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.836+0000 7f1adc9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81005a0 0x7f1ad8115e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.836+0000 7f1adc9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8116450 con 0x7f1ad81005a0 2023-07-14T16:32:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.836+0000 7f1adc9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8116590 con 0x7f1ad80fea20 2023-07-14T16:32:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.836+0000 7f1adc9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8116720 con 0x7f1ad80ffc10 2023-07-14T16:32:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.836+0000 7f1ad5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad80ffc10 0x7f1ad810ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.837+0000 7f1ad5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad80ffc10 0x7f1ad810ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53090/0 (socket says 172.21.15.62:53090) 2023-07-14T16:32:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.837+0000 7f1ad5d9b700 1 -- 172.21.15.62:0/2284495854 learned_addr learned my addr 172.21.15.62:0/2284495854 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.837+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81005a0 0x7f1ad8115e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.838+0000 7f1ad5d9b700 1 -- 172.21.15.62:0/2284495854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad80fea20 msgr2=0x7f1ad810e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.838+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad80fea20 0x7f1ad810e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.838+0000 7f1ad5d9b700 1 -- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81005a0 msgr2=0x7f1ad8115e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.838+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81005a0 0x7f1ad8115e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.838+0000 7f1ad5d9b700 1 -- 172.21.15.62:0/2284495854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80095f0 con 0x7f1ad80ffc10 2023-07-14T16:32:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.838+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad80ffc10 0x7f1ad810ee10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00e9a0 tx=0x7f1acc00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.839+0000 7f1ac77fe700 1 -- 172.21.15.62:0/2284495854 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc009790 con 0x7f1ad80ffc10 2023-07-14T16:32:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.839+0000 7f1ac77fe700 1 -- 172.21.15.62:0/2284495854 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1acc00cbd0 con 0x7f1ad80ffc10 2023-07-14T16:32:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.839+0000 7f1ac77fe700 1 -- 172.21.15.62:0/2284495854 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc018680 con 0x7f1ad80ffc10 2023-07-14T16:32:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.839+0000 7f1adc9ff700 1 -- 172.21.15.62:0/2284495854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad8069f30 con 0x7f1ad80ffc10 2023-07-14T16:32:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.840+0000 7f1adc9ff700 1 -- 172.21.15.62:0/2284495854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad806a3f0 con 0x7f1ad80ffc10 2023-07-14T16:32:36.820 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.847+0000 7f1af02a0b80 1 Processor -- start 2023-07-14T16:32:36.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.848+0000 7f1af02a0b80 1 -- start start 2023-07-14T16:32:36.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.848+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.848+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.821 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.848+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.848+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5dd48cb0 con 0x563c5de44c70 2023-07-14T16:32:36.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.848+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5dd9ba40 con 0x563c5de434d0 2023-07-14T16:32:36.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.848+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5dd94ed0 con 0x563c5de43e20 2023-07-14T16:32:36.822 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.849+0000 7f1ad659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.849+0000 7f1ad6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.849+0000 7f1ad659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53104/0 (socket says 172.21.15.62:53104) 2023-07-14T16:32:36.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.849+0000 7f1ad659c700 1 -- 172.21.15.62:0/3352831664 learned_addr learned my addr 172.21.15.62:0/3352831664 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:36.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.850+0000 7f1ad659c700 1 -- 172.21.15.62:0/3352831664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 msgr2=0x563c5de438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:36.823 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.850+0000 7f1ad659c700 1 --2- 172.21.15.62:0/3352831664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.850+0000 7f1ad659c700 1 -- 172.21.15.62:0/3352831664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 msgr2=0x563c5de450e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:32:36.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.850+0000 7f1ad659c700 1 --2- 172.21.15.62:0/3352831664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de450e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.850+0000 7f1ad659c700 1 -- 172.21.15.62:0/3352831664 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c5dbc2de0 con 0x563c5de43e20 2023-07-14T16:32:36.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.850+0000 7f1ac77fe700 1 -- 172.21.15.62:0/2284495854 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1acc00cd40 con 0x7f1ad80ffc10 2023-07-14T16:32:36.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.850+0000 7f1ad659c700 1 --2- 172.21.15.62:0/3352831664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de446e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005910 tx=0x7f1ac8009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75bb367554fd9f79 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:36.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.851+0000 7f1ac4ff9700 1 -- 172.21.15.62:0/3352831664 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac801d070 con 0x563c5de43e20 2023-07-14T16:32:36.824 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.851+0000 7f1ac4ff9700 1 -- 172.21.15.62:0/3352831664 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ac8004000 con 0x563c5de43e20 2023-07-14T16:32:36.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.851+0000 7f1ac4ff9700 1 -- 172.21.15.62:0/3352831664 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800bbd0 con 0x563c5de43e20 2023-07-14T16:32:36.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1ac77fe700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac00a12d0 0x7f1ac00a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3352831664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 msgr2=0x563c5de446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3352831664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de446e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005910 tx=0x7f1ac8009550 comp rx=0 tx=0).stop 2023-07-14T16:32:36.825 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1ac77fe700 1 -- 172.21.15.62:0/2284495854 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1acc014070 con 0x7f1ad80ffc10 2023-07-14T16:32:36.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3352831664 shutdown_connections 2023-07-14T16:32:36.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3352831664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3352831664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3352831664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de446e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3352831664 >> 172.21.15.62:0/3352831664 conn(0x563c5de38da0 msgr2=0x563c5de3e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:36.826 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.852+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac00a12d0 0x7f1ac00a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.853+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3352831664 shutdown_connections 2023-07-14T16:32:36.827 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.853+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3352831664 wait complete. 2023-07-14T16:32:36.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.854+0000 7f1af02a0b80 1 -- start start 2023-07-14T16:32:36.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.854+0000 7f1af02a0b80 1 Processor -- start 2023-07-14T16:32:36.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.855+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de434d0 0x563c5de4d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.855+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de4d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.855+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de44c70 0x563c5de546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.855+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5dd34ac0 con 0x563c5de434d0 2023-07-14T16:32:36.828 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.855+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5dd347c0 con 0x563c5de44c70 2023-07-14T16:32:36.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.855+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5dc18920 con 0x563c5de43e20 2023-07-14T16:32:36.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.855+0000 7f1af02a0b80 10 client.0 _ll_register_callbacks cb 0x563c5dd32760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:36.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.856+0000 7f1ad6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de44c70 0x563c5de546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.829 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.856+0000 7f1ad6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de44c70 0x563c5de546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40540/0 (socket says 172.21.15.62:40540) 2023-07-14T16:32:36.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.856+0000 7f1ad6d9d700 1 -- 172.21.15.62:0/3745468191 learned_addr learned my addr 172.21.15.62:0/3745468191 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:36.830 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.856+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de4d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.856+0000 7f1ad659c700 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de434d0 0x563c5de4d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.856+0000 7f1ad6d9d700 1 -- 172.21.15.62:0/3745468191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 msgr2=0x563c5de4d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:36.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.856+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de4d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.857+0000 7f1ad6d9d700 1 -- 172.21.15.62:0/3745468191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de434d0 msgr2=0x563c5de4d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:36.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.857+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de434d0 0x563c5de4d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.857+0000 7f1ad6d9d700 1 -- 172.21.15.62:0/3745468191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad8069030 con 0x563c5de44c70 2023-07-14T16:32:36.833 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.857+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de44c70 0x563c5de546d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0018ad0 tx=0x7f1ad0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1cc0e015068ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:36.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.857+0000 7f1abdffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:36.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.857+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac00a12d0 0x7f1ac00a3780 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00093c0 tx=0x7f1ad0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:36.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.858+0000 7f1abdffb700 1 -- 172.21.15.62:0/3745468191 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001d070 con 0x563c5de44c70 2023-07-14T16:32:36.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.858+0000 7f1abdffb700 1 -- 172.21.15.62:0/3745468191 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ad0016650 con 0x563c5de44c70 2023-07-14T16:32:36.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.858+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3745468191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c5dcee4f0 con 0x563c5de44c70 2023-07-14T16:32:36.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.859+0000 7f1abdffb700 1 -- 172.21.15.62:0/3745468191 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ad0016890 con 0x563c5de44c70 2023-07-14T16:32:36.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.859+0000 7f1af02a0b80 10 client.29750 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:36.834 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.859+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3745468191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563c5dbabc60 con 0x563c5de44c70 2023-07-14T16:32:36.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.860+0000 7f1abdffb700 1 -- 172.21.15.62:0/3745468191 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c5dcee4f0 con 0x563c5de44c70 2023-07-14T16:32:36.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.861+0000 7f1abdffb700 1 -- 172.21.15.62:0/3745468191 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ad00219f0 con 0x563c5de44c70 2023-07-14T16:32:36.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.861+0000 7f1af02a0b80 10 client.29750 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:36.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.861+0000 7f1af02a0b80 10 client.29750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:32:36.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.861+0000 7f1af02a0b80 20 client.29750 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:36.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.861+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563c5de59010 0x563c5de5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.862+0000 7f1af02a0b80 4 client.29750 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:36.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.862+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3745468191 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c5dd8deb0 con 0x563c5de59010 2023-07-14T16:32:36.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.862+0000 7f1ad659c700 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563c5de59010 0x563c5de5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.864+0000 7f1ad659c700 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563c5de59010 0x563c5de5b420 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d0725328c1b3fc4 server_cookie=b8dc3944a0f9a7c2 in_seq=0 out_seq=0 2023-07-14T16:32:36.837 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.864+0000 7f1abdffb700 10 client.29750 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:32:36.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:36 smithi062 ceph-mon[95543]: pgmap v866: 97 pgs: 97 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:32:36.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:36 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1656623747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:32:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.865+0000 7f1abdffb700 1 -- 172.21.15.62:0/3745468191 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c5dd8deb0 con 0x563c5de59010 2023-07-14T16:32:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.865+0000 7f1abdffb700 10 client.29750 handle_command_reply: tid=0 2023-07-14T16:32:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.866+0000 7f1af02a0b80 1 client.29750 shutdown 2023-07-14T16:32:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.869+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3745468191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de44c70 msgr2=0x563c5de546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.869+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de44c70 0x563c5de546d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0018ad0 tx=0x7f1ad0009920 comp rx=0 tx=0).stop 2023-07-14T16:32:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.869+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3745468191 shutdown_connections 2023-07-14T16:32:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.869+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563c5de59010 0x563c5de5b420 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.869+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de44c70 0x563c5de546d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.869+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de434d0 0x563c5de4d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.869+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/3745468191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de4d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.870+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3745468191 >> 172.21.15.62:0/3745468191 conn(0x563c5de38da0 msgr2=0x563c5de3d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.870+0000 7f1abdffb700 0 client.29750 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:32:36.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.870+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3745468191 shutdown_connections 2023-07-14T16:32:36.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.870+0000 7f1af02a0b80 1 -- 172.21.15.62:0/3745468191 wait complete. 2023-07-14T16:32:36.845 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.870+0000 7f1af02a0b80 20 client.29750 trim_cache size 0 max 16384 2023-07-14T16:32:36.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.896+0000 7f1af02a0b80 1 Processor -- start 2023-07-14T16:32:36.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.896+0000 7f1af02a0b80 1 -- start start 2023-07-14T16:32:36.869 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.897+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de434d0 0x563c5de4be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.897+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de43e20 0x563c5de54690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.897+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de54f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.897+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5de4ccc0 con 0x563c5de44c70 2023-07-14T16:32:36.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.897+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5de38970 con 0x563c5de43e20 2023-07-14T16:32:36.870 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.897+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5de441f0 con 0x563c5de434d0 2023-07-14T16:32:36.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de434d0 0x563c5de4be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de434d0 0x563c5de4be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53114/0 (socket says 172.21.15.62:53114) 2023-07-14T16:32:36.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad659c700 1 -- 172.21.15.62:0/305196291 learned_addr learned my addr 172.21.15.62:0/305196291 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:36.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de43e20 0x563c5de54690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.871 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de54f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad659c700 1 -- 172.21.15.62:0/305196291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de43e20 msgr2=0x563c5de54690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:36.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad659c700 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de43e20 0x563c5de54690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad659c700 1 -- 172.21.15.62:0/305196291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 msgr2=0x563c5de54f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:36.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad659c700 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de54f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.872 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.898+0000 7f1ad659c700 1 -- 172.21.15.62:0/305196291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c5dbe9250 con 0x563c5de434d0 2023-07-14T16:32:36.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.899+0000 7f1ad659c700 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de434d0 0x563c5de4be20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8024040 tx=0x7f1ac8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f60a9a828fdb1a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:36.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.899+0000 7f1ac57fa700 1 -- 172.21.15.62:0/305196291 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac801d040 con 0x563c5de434d0 2023-07-14T16:32:36.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.899+0000 7f1ac57fa700 1 -- 172.21.15.62:0/305196291 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ac8003c90 con 0x563c5de434d0 2023-07-14T16:32:36.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.899+0000 7f1ac57fa700 1 -- 172.21.15.62:0/305196291 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800b820 con 0x563c5de434d0 2023-07-14T16:32:36.873 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.900+0000 7f1af02a0b80 1 -- 172.21.15.62:0/305196291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de434d0 msgr2=0x563c5de4be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.900+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de434d0 0x563c5de4be20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8024040 tx=0x7f1ac8005fd0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.900+0000 7f1af02a0b80 1 -- 172.21.15.62:0/305196291 shutdown_connections 2023-07-14T16:32:36.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.900+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de54f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.900+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de434d0 0x563c5de4be20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.900+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/305196291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de43e20 0x563c5de54690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.901+0000 7f1af02a0b80 1 -- 172.21.15.62:0/305196291 >> 172.21.15.62:0/305196291 conn(0x563c5de38da0 msgr2=0x563c5de5fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:36.874 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.901+0000 7f1af02a0b80 1 -- 172.21.15.62:0/305196291 shutdown_connections 2023-07-14T16:32:36.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.901+0000 7f1af02a0b80 1 -- 172.21.15.62:0/305196291 wait complete. 2023-07-14T16:32:36.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.902+0000 7f1af02a0b80 1 -- start start 2023-07-14T16:32:36.875 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.902+0000 7f1af02a0b80 1 Processor -- start 2023-07-14T16:32:36.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de50c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1af02a0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de53c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.876 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5de54190 con 0x563c5de44c70 2023-07-14T16:32:36.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5de54300 con 0x563c5de434d0 2023-07-14T16:32:36.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1af02a0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5de544c0 con 0x563c5de43e20 2023-07-14T16:32:36.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1af02a0b80 10 client.0 _ll_register_callbacks cb 0x7f1ad00184a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:36.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40558/0 (socket says 172.21.15.62:40558) 2023-07-14T16:32:36.877 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad659c700 1 -- 172.21.15.62:0/345045437 learned_addr learned my addr 172.21.15.62:0/345045437 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:36.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de50c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de53c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad659c700 1 -- 172.21.15.62:0/345045437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 msgr2=0x563c5de50c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:36.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad659c700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de50c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad659c700 1 -- 172.21.15.62:0/345045437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 msgr2=0x563c5de53c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:36.879 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad659c700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de53c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.903+0000 7f1ad659c700 1 -- 172.21.15.62:0/345045437 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c5dbc2de0 con 0x563c5de434d0 2023-07-14T16:32:36.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.904+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de50c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:36.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.904+0000 7f1ad6d9d700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de53c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:36.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.904+0000 7f1ad659c700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de506f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009350 tx=0x7f1ac8005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b686c557bc19ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:36.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.904+0000 7f1abdffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:36.880 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.904+0000 7f1abdffb700 1 -- 172.21.15.62:0/345045437 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac801d040 con 0x563c5de434d0 2023-07-14T16:32:36.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.904+0000 7f1abdffb700 1 -- 172.21.15.62:0/345045437 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ac802ce40 con 0x563c5de434d0 2023-07-14T16:32:36.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.905+0000 7f1af02a0b80 1 -- 172.21.15.62:0/345045437 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c5dd50050 con 0x563c5de434d0 2023-07-14T16:32:36.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.905+0000 7f1abdffb700 1 -- 172.21.15.62:0/345045437 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac8003800 con 0x563c5de434d0 2023-07-14T16:32:36.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.905+0000 7f1af02a0b80 10 client.29756 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:36.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.905+0000 7f1af02a0b80 1 -- 172.21.15.62:0/345045437 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563c5dbc12b0 con 0x563c5de434d0 2023-07-14T16:32:36.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.906+0000 7f1abdffb700 1 -- 172.21.15.62:0/345045437 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c5dd50050 con 0x563c5de434d0 2023-07-14T16:32:36.881 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.906+0000 7f1abdffb700 1 -- 172.21.15.62:0/345045437 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ac8035070 con 0x563c5de434d0 2023-07-14T16:32:36.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.906+0000 7f1af02a0b80 10 client.29756 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:36.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.906+0000 7f1af02a0b80 10 client.29756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:32:36.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.906+0000 7f1af02a0b80 20 client.29756 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:36.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.906+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563c5deadce0 0x563c5deb00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:36.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.906+0000 7f1af02a0b80 4 client.29756 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:32:36.882 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.906+0000 7f1af02a0b80 1 -- 172.21.15.62:0/345045437 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c5dbe3f70 con 0x563c5deadce0 2023-07-14T16:32:36.884 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.907+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563c5deadce0 0x563c5deb00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:36.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.908+0000 7f1abdffb700 10 client.29756 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:32:36.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.908+0000 7f1ad5d9b700 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563c5deadce0 0x563c5deb00f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5064aada40888bb9 server_cookie=a3e11d5b911b45f in_seq=0 out_seq=0 2023-07-14T16:32:36.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.908+0000 7f1abdffb700 1 -- 172.21.15.62:0/345045437 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c5dbe3f70 con 0x563c5deadce0 2023-07-14T16:32:36.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.908+0000 7f1abdffb700 10 client.29756 handle_command_reply: tid=0 2023-07-14T16:32:36.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.908+0000 7f1af02a0b80 1 client.29756 shutdown 2023-07-14T16:32:36.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.909+0000 7f1af02a0b80 1 -- 172.21.15.62:0/345045437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 msgr2=0x563c5de506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.909+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de506f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009350 tx=0x7f1ac8005c80 comp rx=0 tx=0).stop 2023-07-14T16:32:36.885 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.910+0000 7f1af02a0b80 1 -- 172.21.15.62:0/345045437 shutdown_connections 2023-07-14T16:32:36.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.910+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x563c5deadce0 0x563c5deb00f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.910+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c5de44c70 0x563c5de53c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.910+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c5de434d0 0x563c5de506f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.910+0000 7f1af02a0b80 1 --2- 172.21.15.62:0/345045437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c5de43e20 0x563c5de50c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.910+0000 7f1abdffb700 0 client.29756 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:32:36.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.910+0000 7f1af02a0b80 1 -- 172.21.15.62:0/345045437 >> 172.21.15.62:0/345045437 conn(0x563c5de38da0 msgr2=0x563c5de4be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:36.886 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.911+0000 7f1af02a0b80 1 -- 172.21.15.62:0/345045437 shutdown_connections 2023-07-14T16:32:36.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.911+0000 7f1af02a0b80 1 -- 172.21.15.62:0/345045437 wait complete. 2023-07-14T16:32:36.887 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.911+0000 7f1af02a0b80 20 client.29756 trim_cache size 0 max 16384 2023-07-14T16:32:36.887 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:32:36.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.912+0000 7f1adc9ff700 1 -- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac00a12d0 msgr2=0x7f1ac00a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.912+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac00a12d0 0x7f1ac00a3780 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00093c0 tx=0x7f1ad0009040 comp rx=0 tx=0).stop 2023-07-14T16:32:36.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.912+0000 7f1adc9ff700 1 -- 172.21.15.62:0/2284495854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad80ffc10 msgr2=0x7f1ad810ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:36.888 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.912+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad80ffc10 0x7f1ad810ee10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00e9a0 tx=0x7f1acc00ed60 comp rx=0 tx=0).stop 2023-07-14T16:32:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.913+0000 7f1adc9ff700 1 -- 172.21.15.62:0/2284495854 shutdown_connections 2023-07-14T16:32:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.913+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ad81005a0 0x7f1ad8115e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.913+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1ac00a12d0 0x7f1ac00a3780 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.913+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ad80fea20 0x7f1ad810e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.913+0000 7f1adc9ff700 1 --2- 172.21.15.62:0/2284495854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ad80ffc10 0x7f1ad810ee10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.913+0000 7f1adc9ff700 1 -- 172.21.15.62:0/2284495854 >> 172.21.15.62:0/2284495854 conn(0x7f1ad80fa020 msgr2=0x7f1ad8108550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.913+0000 7f1adc9ff700 1 -- 172.21.15.62:0/2284495854 shutdown_connections 2023-07-14T16:32:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:36.914+0000 7f1adc9ff700 1 -- 172.21.15.62:0/2284495854 wait complete. 2023-07-14T16:32:36.915 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:32:37.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:36 smithi081 ceph-mon[127795]: pgmap v866: 97 pgs: 97 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:32:37.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:36 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1656623747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:32:37.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:36 smithi164 ceph-mon[97427]: pgmap v866: 97 pgs: 97 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:32:37.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:36 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1656623747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:32:37.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.693+0000 7fb345dff700 1 -- 172.21.15.62:0/869495220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb34006a220 msgr2=0x7fb34006a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:37.666 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.693+0000 7fb345dff700 1 --2- 172.21.15.62:0/869495220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb34006a220 0x7fb34006a690 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb334009990 tx=0x7fb334009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.696+0000 7fb345dff700 1 -- 172.21.15.62:0/869495220 shutdown_connections 2023-07-14T16:32:37.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.696+0000 7fb345dff700 1 --2- 172.21.15.62:0/869495220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3401148c0 0x7fb340116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.696+0000 7fb345dff700 1 --2- 172.21.15.62:0/869495220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb34006b800 0x7fb340069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.669 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.696+0000 7fb345dff700 1 --2- 172.21.15.62:0/869495220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb34006a220 0x7fb34006a690 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.696+0000 7fb345dff700 1 -- 172.21.15.62:0/869495220 >> 172.21.15.62:0/869495220 conn(0x7fb34006eb30 msgr2=0x7fb340070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:37.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.696+0000 7fb345dff700 1 -- 172.21.15.62:0/869495220 shutdown_connections 2023-07-14T16:32:37.670 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.696+0000 7fb345dff700 1 -- 172.21.15.62:0/869495220 wait complete. 2023-07-14T16:32:37.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.698+0000 7fb345dff700 1 Processor -- start 2023-07-14T16:32:37.671 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.698+0000 7fb345dff700 1 -- start start 2023-07-14T16:32:37.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.698+0000 7fb345dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb34006a220 0x7fb340110fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.698+0000 7fb345dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb34006b800 0x7fb3401114e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.698+0000 7fb345dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3401148c0 0x7fb3401b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.698+0000 7fb345dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3401145c0 con 0x7fb34006a220 2023-07-14T16:32:37.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.698+0000 7fb345dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb340114730 con 0x7fb34006b800 2023-07-14T16:32:37.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.698+0000 7fb345dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3401b4c80 con 0x7fb3401148c0 2023-07-14T16:32:37.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.699+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb34006a220 0x7fb340110fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.699+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb34006a220 0x7fb340110fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41656/0 (socket says 172.21.15.62:41656) 2023-07-14T16:32:37.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.699+0000 7fb33f7fe700 1 -- 172.21.15.62:0/3410902684 learned_addr learned my addr 172.21.15.62:0/3410902684 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:37.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.699+0000 7fb33ffff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3401148c0 0x7fb3401b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.700+0000 7fb33f7fe700 1 -- 172.21.15.62:0/3410902684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3401148c0 msgr2=0x7fb3401b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:37.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.700+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3401148c0 0x7fb3401b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.700+0000 7fb33f7fe700 1 -- 172.21.15.62:0/3410902684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb34006b800 msgr2=0x7fb3401114e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T16:32:37.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.700+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb34006b800 0x7fb3401114e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.700+0000 7fb33f7fe700 1 -- 172.21.15.62:0/3410902684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340095f0 con 0x7fb34006a220 2023-07-14T16:32:37.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.701+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb34006a220 0x7fb340110fa0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fb330007f10 tx=0x7fb33000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:37.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.701+0000 7fb33cff9700 1 -- 172.21.15.62:0/3410902684 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000ddd0 con 0x7fb34006a220 2023-07-14T16:32:37.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.701+0000 7fb33cff9700 1 -- 172.21.15.62:0/3410902684 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb33000f040 con 0x7fb34006a220 2023-07-14T16:32:37.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.701+0000 7fb345dff700 1 -- 172.21.15.62:0/3410902684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3401b4f30 con 0x7fb34006a220 2023-07-14T16:32:37.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.701+0000 7fb33cff9700 1 -- 172.21.15.62:0/3410902684 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3300125b0 con 0x7fb34006a220 2023-07-14T16:32:37.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.702+0000 7fb345dff700 1 -- 172.21.15.62:0/3410902684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3401b5400 con 0x7fb34006a220 2023-07-14T16:32:37.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.707+0000 7fb3597efb80 1 Processor -- start 2023-07-14T16:32:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.707+0000 7fb3597efb80 1 -- start start 2023-07-14T16:32:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.708+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee8648a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.708+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee8648b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.708+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee8648c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.708+0000 7fb3597efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8638fcb0 con 0x55ee8648ae20 2023-07-14T16:32:37.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.708+0000 7fb3597efb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee863e2a40 con 0x55ee8648a4d0 2023-07-14T16:32:37.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.708+0000 7fb3597efb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee863dbed0 con 0x55ee8648bc70 2023-07-14T16:32:37.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.717+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee8648b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.717+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee8648b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41658/0 (socket says 172.21.15.62:41658) 2023-07-14T16:32:37.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.717+0000 7fb33f7fe700 1 -- 172.21.15.62:0/1630009441 learned_addr learned my addr 172.21.15.62:0/1630009441 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:37.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.719+0000 7fb33ffff700 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee8648a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.723+0000 7fb33effd700 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee8648c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.726+0000 7fb33cff9700 1 -- 172.21.15.62:0/3410902684 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb3300133f0 con 0x7fb34006a220 2023-07-14T16:32:37.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.727+0000 7fb33f7fe700 1 -- 172.21.15.62:0/1630009441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 msgr2=0x55ee8648c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:37.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.727+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee8648c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.727+0000 7fb33f7fe700 1 -- 172.21.15.62:0/1630009441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 msgr2=0x55ee8648a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:37.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.727+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee8648a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.727+0000 7fb33f7fe700 1 -- 172.21.15.62:0/1630009441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee86209de0 con 0x55ee8648ae20 2023-07-14T16:32:37.701 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.727+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee8648b6e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb3300cf880 tx=0x7fb3300d4750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f9eb27c995e54e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:37.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.729+0000 7fb33cff9700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3280a1250 0x7fb3280a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.729+0000 7fb33cff9700 1 -- 172.21.15.62:0/3410902684 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb3300d1e30 con 0x7fb34006a220 2023-07-14T16:32:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.729+0000 7fb33ffff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3280a1250 0x7fb3280a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.729+0000 7fb325ffb700 1 -- 172.21.15.62:0/1630009441 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33009b4a0 con 0x55ee8648ae20 2023-07-14T16:32:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.730+0000 7fb325ffb700 1 -- 172.21.15.62:0/1630009441 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3300d9070 con 0x55ee8648ae20 2023-07-14T16:32:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.730+0000 7fb325ffb700 1 -- 172.21.15.62:0/1630009441 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330009410 con 0x55ee8648ae20 2023-07-14T16:32:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 -- 172.21.15.62:0/1630009441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 msgr2=0x55ee8648b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee8648b6e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb3300cf880 tx=0x7fb3300d4750 comp rx=0 tx=0).stop 2023-07-14T16:32:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb33ffff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3280a1250 0x7fb3280a3700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb338005950 tx=0x7fb33800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 -- 172.21.15.62:0/1630009441 shutdown_connections 2023-07-14T16:32:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee8648c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee8648a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 --2- 172.21.15.62:0/1630009441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee8648b6e0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 -- 172.21.15.62:0/1630009441 >> 172.21.15.62:0/1630009441 conn(0x55ee8647fda0 msgr2=0x55ee864851d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:37.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 -- 172.21.15.62:0/1630009441 shutdown_connections 2023-07-14T16:32:37.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.731+0000 7fb3597efb80 1 -- 172.21.15.62:0/1630009441 wait complete. 2023-07-14T16:32:37.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.732+0000 7fb3597efb80 1 -- start start 2023-07-14T16:32:37.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.732+0000 7fb3597efb80 1 Processor -- start 2023-07-14T16:32:37.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee864941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648ae20 0x55ee864946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648bc70 0x55ee8649b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb3597efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8637bac0 con 0x55ee8648a4d0 2023-07-14T16:32:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb3597efb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8637b7c0 con 0x55ee8648bc70 2023-07-14T16:32:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb3597efb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8625f920 con 0x55ee8648ae20 2023-07-14T16:32:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb3597efb80 10 client.0 _ll_register_callbacks cb 0x55ee86379760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee864941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee864941b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41662/0 (socket says 172.21.15.62:41662) 2023-07-14T16:32:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.733+0000 7fb33f7fe700 1 -- 172.21.15.62:0/2725750528 learned_addr learned my addr 172.21.15.62:0/2725750528 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.734+0000 7fb33effd700 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648ae20 0x55ee864946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.734+0000 7fb33f7fe700 1 -- 172.21.15.62:0/2725750528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648ae20 msgr2=0x55ee864946f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.734+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648ae20 0x55ee864946f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.734+0000 7fb33f7fe700 1 -- 172.21.15.62:0/2725750528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648bc70 msgr2=0x55ee8649b730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:32:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.734+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648bc70 0x55ee8649b730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.734+0000 7fb33f7fe700 1 -- 172.21.15.62:0/2725750528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3401b5400 con 0x55ee8648a4d0 2023-07-14T16:32:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.734+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee864941b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fb33009ecc0 tx=0x7fb330013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566f3867e604e552 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.734+0000 7fb31effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.735+0000 7fb31effd700 1 -- 172.21.15.62:0/2725750528 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330009410 con 0x55ee8648a4d0 2023-07-14T16:32:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.735+0000 7fb31effd700 1 -- 172.21.15.62:0/2725750528 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb3300d4e50 con 0x55ee8648a4d0 2023-07-14T16:32:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.735+0000 7fb3597efb80 1 -- 172.21.15.62:0/2725750528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee863354f0 con 0x55ee8648a4d0 2023-07-14T16:32:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.736+0000 7fb31effd700 1 -- 172.21.15.62:0/2725750528 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33009f1f0 con 0x55ee8648a4d0 2023-07-14T16:32:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.736+0000 7fb3597efb80 10 client.20223 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.736+0000 7fb3597efb80 1 -- 172.21.15.62:0/2725750528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ee861f2c60 con 0x55ee8648a4d0 2023-07-14T16:32:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.737+0000 7fb31effd700 1 -- 172.21.15.62:0/2725750528 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ee863354f0 con 0x55ee8648a4d0 2023-07-14T16:32:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.737+0000 7fb31effd700 1 -- 172.21.15.62:0/2725750528 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb3300dd030 con 0x55ee8648a4d0 2023-07-14T16:32:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.737+0000 7fb3597efb80 10 client.20223 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.737+0000 7fb3597efb80 10 client.20223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:32:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.737+0000 7fb3597efb80 20 client.20223 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.737+0000 7fb3597efb80 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55ee8649fb50 0x55ee864a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.737+0000 7fb3597efb80 4 client.20223 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.737+0000 7fb3597efb80 1 -- 172.21.15.62:0/2725750528 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee863d4eb0 con 0x55ee8649fb50 2023-07-14T16:32:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.738+0000 7fb33effd700 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55ee8649fb50 0x55ee864a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.739+0000 7fb33effd700 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55ee8649fb50 0x55ee864a1f60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db87a4b4de81acfe server_cookie=dffba9cf86a7fc5e in_seq=0 out_seq=0 2023-07-14T16:32:37.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.741+0000 7fb31effd700 10 client.20223 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:32:37.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.741+0000 7fb31effd700 1 -- 172.21.15.62:0/2725750528 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ee863d4eb0 con 0x55ee8649fb50 2023-07-14T16:32:37.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.741+0000 7fb31effd700 10 client.20223 handle_command_reply: tid=0 2023-07-14T16:32:37.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.741+0000 7fb3597efb80 1 client.20223 shutdown 2023-07-14T16:32:37.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.741+0000 7fb3597efb80 1 -- 172.21.15.62:0/2725750528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 msgr2=0x55ee864941b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:37.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.741+0000 7fb3597efb80 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee864941b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fb33009ecc0 tx=0x7fb330013c50 comp rx=0 tx=0).stop 2023-07-14T16:32:37.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.742+0000 7fb3597efb80 1 -- 172.21.15.62:0/2725750528 shutdown_connections 2023-07-14T16:32:37.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.742+0000 7fb3597efb80 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55ee8649fb50 0x55ee864a1f60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.742+0000 7fb3597efb80 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648bc70 0x55ee8649b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.742+0000 7fb3597efb80 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee864941b0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.742+0000 7fb3597efb80 1 --2- 172.21.15.62:0/2725750528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648ae20 0x55ee864946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.742+0000 7fb3597efb80 1 -- 172.21.15.62:0/2725750528 >> 172.21.15.62:0/2725750528 conn(0x55ee8647fda0 msgr2=0x55ee864848b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:37.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.749+0000 7fb31effd700 0 client.20223 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:32:37.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.750+0000 7fb3597efb80 1 -- 172.21.15.62:0/2725750528 shutdown_connections 2023-07-14T16:32:37.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.750+0000 7fb3597efb80 1 -- 172.21.15.62:0/2725750528 wait complete. 2023-07-14T16:32:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.751+0000 7fb3597efb80 20 client.20223 trim_cache size 0 max 16384 2023-07-14T16:32:37.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.767+0000 7fb3597efb80 1 Processor -- start 2023-07-14T16:32:37.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb3597efb80 1 -- start start 2023-07-14T16:32:37.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee86493860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648ae20 0x55ee8649b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee864a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb3597efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8647cef0 con 0x55ee8648a4d0 2023-07-14T16:32:37.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb3597efb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee86486d00 con 0x55ee8648ae20 2023-07-14T16:32:37.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb3597efb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee86491e00 con 0x55ee8648bc70 2023-07-14T16:32:37.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee86493860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee86493860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41668/0 (socket says 172.21.15.62:41668) 2023-07-14T16:32:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb33f7fe700 1 -- 172.21.15.62:0/1810407708 learned_addr learned my addr 172.21.15.62:0/1810407708 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb33ffff700 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee864a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.768+0000 7fb33effd700 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648ae20 0x55ee8649b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb33f7fe700 1 -- 172.21.15.62:0/1810407708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 msgr2=0x55ee864a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee864a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb33f7fe700 1 -- 172.21.15.62:0/1810407708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648ae20 msgr2=0x55ee8649b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648ae20 0x55ee8649b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb33f7fe700 1 -- 172.21.15.62:0/1810407708 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee86230250 con 0x55ee8648a4d0 2023-07-14T16:32:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee86493860 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb33009d310 tx=0x7fb3300d4750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a574c19092d895e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb3267fc700 1 -- 172.21.15.62:0/1810407708 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33009d3d0 con 0x55ee8648a4d0 2023-07-14T16:32:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb3267fc700 1 -- 172.21.15.62:0/1810407708 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb33009efc0 con 0x55ee8648a4d0 2023-07-14T16:32:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.769+0000 7fb3267fc700 1 -- 172.21.15.62:0/1810407708 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3300a0210 con 0x55ee8648a4d0 2023-07-14T16:32:37.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 -- 172.21.15.62:0/1810407708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 msgr2=0x55ee86493860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:37.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee86493860 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb33009d310 tx=0x7fb3300d4750 comp rx=0 tx=0).stop 2023-07-14T16:32:37.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 -- 172.21.15.62:0/1810407708 shutdown_connections 2023-07-14T16:32:37.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee864a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648a4d0 0x55ee86493860 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 --2- 172.21.15.62:0/1810407708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648ae20 0x55ee8649b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 -- 172.21.15.62:0/1810407708 >> 172.21.15.62:0/1810407708 conn(0x55ee8647fda0 msgr2=0x55ee86492e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:37.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 -- 172.21.15.62:0/1810407708 shutdown_connections 2023-07-14T16:32:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.770+0000 7fb3597efb80 1 -- 172.21.15.62:0/1810407708 wait complete. 2023-07-14T16:32:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 1 -- start start 2023-07-14T16:32:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 1 Processor -- start 2023-07-14T16:32:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee86497920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee86497e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee8649a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.755 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8649ac70 con 0x55ee8648ae20 2023-07-14T16:32:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8649ade0 con 0x55ee8648a4d0 2023-07-14T16:32:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8649af50 con 0x55ee8648bc70 2023-07-14T16:32:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.771+0000 7fb3597efb80 10 client.0 _ll_register_callbacks cb 0x55ee86379760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee86497e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee86497920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee86497e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41674/0 (socket says 172.21.15.62:41674) 2023-07-14T16:32:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33effd700 1 -- 172.21.15.62:0/517319512 learned_addr learned my addr 172.21.15.62:0/517319512 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33ffff700 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee8649a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33f7fe700 1 -- 172.21.15.62:0/517319512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 msgr2=0x55ee8649a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:37.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee8649a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33f7fe700 1 -- 172.21.15.62:0/517319512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 msgr2=0x55ee86497e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:37.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee86497e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33f7fe700 1 -- 172.21.15.62:0/517319512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee861f2c60 con 0x55ee8648a4d0 2023-07-14T16:32:37.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33effd700 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee86497e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:32:37.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb33f7fe700 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee86497920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3300d5010 tx=0x7fb3300a7460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a2365a4cb7d4c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:37.759 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.772+0000 7fb31effd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb31effd700 1 -- 172.21.15.62:0/517319512 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33009d3d0 con 0x55ee8648a4d0 2023-07-14T16:32:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb31effd700 1 -- 172.21.15.62:0/517319512 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb330009410 con 0x55ee8648a4d0 2023-07-14T16:32:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 1 -- 172.21.15.62:0/517319512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3300d9070 con 0x55ee8648a4d0 2023-07-14T16:32:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb31effd700 1 -- 172.21.15.62:0/517319512 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33009f1f0 con 0x55ee8648a4d0 2023-07-14T16:32:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 10 client.29765 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 1 -- 172.21.15.62:0/517319512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ee862082b0 con 0x55ee8648a4d0 2023-07-14T16:32:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb31effd700 1 -- 172.21.15.62:0/517319512 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3300d9070 con 0x55ee8648a4d0 2023-07-14T16:32:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb31effd700 1 -- 172.21.15.62:0/517319512 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb3300dd070 con 0x55ee8648a4d0 2023-07-14T16:32:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 10 client.29765 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 10 client.29765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:32:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 20 client.29765 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55ee864f4080 0x55ee864f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 4 client.29765 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:32:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.773+0000 7fb3597efb80 1 -- 172.21.15.62:0/517319512 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee86397050 con 0x55ee864f4080 2023-07-14T16:32:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.774+0000 7fb33effd700 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55ee864f4080 0x55ee864f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.774+0000 7fb33effd700 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55ee864f4080 0x55ee864f6460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96c79ed3bbfd0d3 server_cookie=22ef392f836a7d5d in_seq=0 out_seq=0 2023-07-14T16:32:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.774+0000 7fb31effd700 10 client.29765 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:32:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb31effd700 1 -- 172.21.15.62:0/517319512 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee86397050 con 0x55ee864f4080 2023-07-14T16:32:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb31effd700 10 client.29765 handle_command_reply: tid=0 2023-07-14T16:32:37.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 client.29765 shutdown 2023-07-14T16:32:37.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 -- 172.21.15.62:0/517319512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 msgr2=0x55ee86497920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:37.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee86497920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3300d5010 tx=0x7fb3300a7460 comp rx=0 tx=0).stop 2023-07-14T16:32:37.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 -- 172.21.15.62:0/517319512 shutdown_connections 2023-07-14T16:32:37.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55ee864f4080 0x55ee864f6460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ee8648bc70 0x55ee8649a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ee8648a4d0 0x55ee86497920 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.763 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 --2- 172.21.15.62:0/517319512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee8648ae20 0x55ee86497e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb31effd700 0 client.29765 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:32:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 -- 172.21.15.62:0/517319512 >> 172.21.15.62:0/517319512 conn(0x55ee8647fda0 msgr2=0x55ee864a1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 -- 172.21.15.62:0/517319512 shutdown_connections 2023-07-14T16:32:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 1 -- 172.21.15.62:0/517319512 wait complete. 2023-07-14T16:32:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.775+0000 7fb3597efb80 20 client.29765 trim_cache size 0 max 16384 2023-07-14T16:32:37.765 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:32:37.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 -- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3280a1250 msgr2=0x7fb3280a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:37.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3280a1250 0x7fb3280a3700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb338005950 tx=0x7fb33800b390 comp rx=0 tx=0).stop 2023-07-14T16:32:37.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 -- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb34006a220 msgr2=0x7fb340110fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:37.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb34006a220 0x7fb340110fa0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fb330007f10 tx=0x7fb33000d3b0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 -- 172.21.15.62:0/3410902684 shutdown_connections 2023-07-14T16:32:37.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3401148c0 0x7fb3401b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb34006a220 0x7fb340110fa0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb3280a1250 0x7fb3280a3700 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 --2- 172.21.15.62:0/3410902684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb34006b800 0x7fb3401114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:37.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 -- 172.21.15.62:0/3410902684 >> 172.21.15.62:0/3410902684 conn(0x7fb34006eb30 msgr2=0x7fb340070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:37.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 -- 172.21.15.62:0/3410902684 shutdown_connections 2023-07-14T16:32:37.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:37.776+0000 7fb345dff700 1 -- 172.21.15.62:0/3410902684 wait complete. 2023-07-14T16:32:37.770 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:32:38.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.164+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2444391412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c071830 msgr2=0x7ff39c10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.164+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2444391412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c071830 0x7ff39c10b7d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009990 tx=0x7ff38c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.164+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2444391412 shutdown_connections 2023-07-14T16:32:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.164+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2444391412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c10bd60 0x7ff39c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.164+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2444391412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c070ee0 0x7ff39c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.164+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2444391412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c071830 0x7ff39c10b7d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.164+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2444391412 >> 172.21.15.62:0/2444391412 conn(0x7ff39c06ca90 msgr2=0x7ff39c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.169+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2444391412 shutdown_connections 2023-07-14T16:32:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.169+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2444391412 wait complete. 2023-07-14T16:32:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.171+0000 7ff3a39ff700 1 Processor -- start 2023-07-14T16:32:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.171+0000 7ff3a39ff700 1 -- start start 2023-07-14T16:32:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.171+0000 7ff3a39ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c071830 0x7ff39c113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.171+0000 7ff3a39ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c10bd60 0x7ff39c114220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.171+0000 7ff3a39ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c1b2800 0x7ff39c1b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.171+0000 7ff3a39ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c114920 con 0x7ff39c1b2800 2023-07-14T16:32:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.171+0000 7ff3a39ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c1b5120 con 0x7ff39c10bd60 2023-07-14T16:32:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.171+0000 7ff3a39ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c1b52b0 con 0x7ff39c071830 2023-07-14T16:32:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.172+0000 7ff3a179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c071830 0x7ff39c113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.172+0000 7ff3a179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c071830 0x7ff39c113ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53198/0 (socket says 172.21.15.62:53198) 2023-07-14T16:32:38.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.172+0000 7ff3a179b700 1 -- 172.21.15.62:0/2508725293 learned_addr learned my addr 172.21.15.62:0/2508725293 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.173+0000 7ff3a0f9a700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c10bd60 0x7ff39c114220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.173+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c1b2800 0x7ff39c1b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.174+0000 7ff3a179b700 1 -- 172.21.15.62:0/2508725293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c10bd60 msgr2=0x7ff39c114220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.174+0000 7ff3a179b700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c10bd60 0x7ff39c114220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.150 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.174+0000 7ff3a179b700 1 -- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c1b2800 msgr2=0x7ff39c1b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.174+0000 7ff3a179b700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c1b2800 0x7ff39c1b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.174+0000 7ff3a179b700 1 -- 172.21.15.62:0/2508725293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0095f0 con 0x7ff39c071830 2023-07-14T16:32:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.174+0000 7ff3a179b700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c071830 0x7ff39c113ce0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b9e0 tx=0x7ff39400bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.175+0000 7ff3927fc700 1 -- 172.21.15.62:0/2508725293 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39402ea40 con 0x7ff39c071830 2023-07-14T16:32:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.175+0000 7ff3927fc700 1 -- 172.21.15.62:0/2508725293 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3940098f0 con 0x7ff39c071830 2023-07-14T16:32:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.175+0000 7ff3927fc700 1 -- 172.21.15.62:0/2508725293 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940345a0 con 0x7ff39c071830 2023-07-14T16:32:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.175+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2508725293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39c0727a0 con 0x7ff39c071830 2023-07-14T16:32:38.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.176+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2508725293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39c072ce0 con 0x7ff39c071830 2023-07-14T16:32:38.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.186+0000 7ff3b72d1b80 1 Processor -- start 2023-07-14T16:32:38.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.187+0000 7ff3927fc700 1 -- 172.21.15.62:0/2508725293 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff394002020 con 0x7ff39c071830 2023-07-14T16:32:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.187+0000 7ff3b72d1b80 1 -- start start 2023-07-14T16:32:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab134d0 0x5587fab138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab13e20 0x5587fab146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 0x5587fab150e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587faa18cb0 con 0x5587fab14c70 2023-07-14T16:32:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587faa6ba40 con 0x5587fab134d0 2023-07-14T16:32:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587faa64ed0 con 0x5587fab13e20 2023-07-14T16:32:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3a179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab13e20 0x5587fab146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3a179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab13e20 0x5587fab146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53214/0 (socket says 172.21.15.62:53214) 2023-07-14T16:32:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.188+0000 7ff3a179b700 1 -- 172.21.15.62:0/2450177885 learned_addr learned my addr 172.21.15.62:0/2450177885 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.189+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/2450177885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab134d0 0x5587fab138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.189+0000 7ff3a179b700 1 -- 172.21.15.62:0/2450177885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab134d0 msgr2=0x5587fab138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.189+0000 7ff3a179b700 1 --2- 172.21.15.62:0/2450177885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab134d0 0x5587fab138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.189+0000 7ff3a179b700 1 -- 172.21.15.62:0/2450177885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 msgr2=0x5587fab150e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.189+0000 7ff3a179b700 1 --2- 172.21.15.62:0/2450177885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 0x5587fab150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.189+0000 7ff3a179b700 1 -- 172.21.15.62:0/2450177885 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587fa892de0 con 0x5587fab13e20 2023-07-14T16:32:38.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.190+0000 7ff3927fc700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3880a1220 0x7ff3880a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.191+0000 7ff3a179b700 1 --2- 172.21.15.62:0/2450177885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab13e20 0x5587fab146e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3940f1eb0 tx=0x7ff3940f63c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d31d987fab9abc65 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.191+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3880a1220 0x7ff3880a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.191+0000 7ff3877fe700 1 -- 172.21.15.62:0/2450177885 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940b76d0 con 0x5587fab13e20 2023-07-14T16:32:38.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.191+0000 7ff3927fc700 1 -- 172.21.15.62:0/2508725293 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3940edf70 con 0x7ff39c071830 2023-07-14T16:32:38.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.191+0000 7ff3877fe700 1 -- 172.21.15.62:0/2450177885 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3940f1520 con 0x5587fab13e20 2023-07-14T16:32:38.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.191+0000 7ff3877fe700 1 -- 172.21.15.62:0/2450177885 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940baed0 con 0x5587fab13e20 2023-07-14T16:32:38.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.192+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3880a1220 0x7ff3880a36d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff398006fd0 tx=0x7ff398008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.192+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/2450177885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab13e20 msgr2=0x5587fab146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.192+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/2450177885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab13e20 0x5587fab146e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3940f1eb0 tx=0x7ff3940f63c0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.193+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/2450177885 shutdown_connections 2023-07-14T16:32:38.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.193+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/2450177885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 0x5587fab150e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.193+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/2450177885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab134d0 0x5587fab138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.193+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/2450177885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab13e20 0x5587fab146e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.193+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/2450177885 >> 172.21.15.62:0/2450177885 conn(0x5587fab08da0 msgr2=0x5587fab0e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.195+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/2450177885 shutdown_connections 2023-07-14T16:32:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.195+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/2450177885 wait complete. 2023-07-14T16:32:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.195+0000 7ff3b72d1b80 1 -- start start 2023-07-14T16:32:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.196+0000 7ff3b72d1b80 1 Processor -- start 2023-07-14T16:32:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.196+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 0x5587fab1d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.196+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab13e20 0x5587fab1d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.196+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab14c70 0x5587fab24700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.196+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587faa04ac0 con 0x5587fab134d0 2023-07-14T16:32:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.196+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587faa047c0 con 0x5587fab13e20 2023-07-14T16:32:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.196+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fa8e8920 con 0x5587fab14c70 2023-07-14T16:32:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.197+0000 7ff3b72d1b80 10 client.0 _ll_register_callbacks cb 0x5587faa02760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.197+0000 7ff3a0f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab13e20 0x5587fab1d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.197+0000 7ff3a0f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab13e20 0x5587fab1d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40634/0 (socket says 172.21.15.62:40634) 2023-07-14T16:32:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.197+0000 7ff3a0f9a700 1 -- 172.21.15.62:0/1925198130 learned_addr learned my addr 172.21.15.62:0/1925198130 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.197+0000 7ff3a179b700 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 0x5587fab1d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.197+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab14c70 0x5587fab24700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.198+0000 7ff3a0f9a700 1 -- 172.21.15.62:0/1925198130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab14c70 msgr2=0x5587fab24700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.198+0000 7ff3a0f9a700 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab14c70 0x5587fab24700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.198+0000 7ff3a0f9a700 1 -- 172.21.15.62:0/1925198130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 msgr2=0x5587fab1d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.198+0000 7ff3a0f9a700 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 0x5587fab1d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.198+0000 7ff3a0f9a700 1 -- 172.21.15.62:0/1925198130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39c072ce0 con 0x5587fab13e20 2023-07-14T16:32:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.200+0000 7ff3a0f9a700 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab13e20 0x5587fab1d6c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff38c015000 tx=0x7ff38c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849d3c834b6cda70 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.200+0000 7ff384ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:38.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.202+0000 7ff384ff9700 1 -- 172.21.15.62:0/1925198130 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00bbe0 con 0x5587fab13e20 2023-07-14T16:32:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.202+0000 7ff384ff9700 1 -- 172.21.15.62:0/1925198130 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff38c00bd40 con 0x5587fab13e20 2023-07-14T16:32:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.202+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1925198130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587faa5deb0 con 0x5587fab13e20 2023-07-14T16:32:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.203+0000 7ff384ff9700 1 -- 172.21.15.62:0/1925198130 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff38c0038c0 con 0x5587fab13e20 2023-07-14T16:32:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.203+0000 7ff3b72d1b80 10 client.29777 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.203+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1925198130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5587fa87bc60 con 0x5587fab13e20 2023-07-14T16:32:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.204+0000 7ff384ff9700 1 -- 172.21.15.62:0/1925198130 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5587faa5deb0 con 0x5587fab13e20 2023-07-14T16:32:38.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.204+0000 7ff384ff9700 1 -- 172.21.15.62:0/1925198130 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff38c02a070 con 0x5587fab13e20 2023-07-14T16:32:38.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.205+0000 7ff3b72d1b80 10 client.29777 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:38.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.205+0000 7ff3b72d1b80 10 client.29777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:38.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.205+0000 7ff3b72d1b80 20 client.29777 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:38.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.205+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fab28ee0 0x5587fab2b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.205+0000 7ff3b72d1b80 4 client.29777 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:38.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.205+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1925198130 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587faa20050 con 0x5587fab28ee0 2023-07-14T16:32:38.179 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.207+0000 7ff3a179b700 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fab28ee0 0x5587fab2b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.209+0000 7ff3a179b700 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fab28ee0 0x5587fab2b2f0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e254c415ccdb6902 server_cookie=ae7a4f3405e54131 in_seq=0 out_seq=0 2023-07-14T16:32:38.181 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.209+0000 7ff384ff9700 10 client.29777 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:38.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.210+0000 7ff384ff9700 1 -- 172.21.15.62:0/1925198130 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5587faa20050 con 0x5587fab28ee0 2023-07-14T16:32:38.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.210+0000 7ff384ff9700 10 client.29777 handle_command_reply: tid=0 2023-07-14T16:32:38.182 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.210+0000 7ff3b72d1b80 1 client.29777 shutdown 2023-07-14T16:32:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.211+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1925198130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab13e20 msgr2=0x5587fab1d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.211+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab13e20 0x5587fab1d6c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff38c015000 tx=0x7ff38c005750 comp rx=0 tx=0).stop 2023-07-14T16:32:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.211+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1925198130 shutdown_connections 2023-07-14T16:32:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.211+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab14c70 0x5587fab24700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.211+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 0x5587fab1d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.211+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fab28ee0 0x5587fab2b2f0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.211+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1925198130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab13e20 0x5587fab1d6c0 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff38c015000 tx=0x7ff38c005750 comp rx=0 tx=0).stop 2023-07-14T16:32:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.211+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1925198130 >> 172.21.15.62:0/1925198130 conn(0x5587fab08da0 msgr2=0x5587fab0d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.214+0000 7ff384ff9700 0 client.29777 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:38.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.214+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1925198130 shutdown_connections 2023-07-14T16:32:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.215+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1925198130 wait complete. 2023-07-14T16:32:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.215+0000 7ff3b72d1b80 20 client.29777 trim_cache size 0 max 16384 2023-07-14T16:32:38.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.248+0000 7ff3b72d1b80 1 Processor -- start 2023-07-14T16:32:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.251+0000 7ff3b72d1b80 1 -- start start 2023-07-14T16:32:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.251+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab134d0 0x5587fab0f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.251+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 0x5587fab246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.251+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab24f30 0x5587fab2aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.251+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fab1de00 con 0x5587fab14c70 2023-07-14T16:32:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.251+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fab0f070 con 0x5587fab24f30 2023-07-14T16:32:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.251+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fab1e170 con 0x5587fab134d0 2023-07-14T16:32:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab134d0 0x5587fab0f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a179b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab134d0 0x5587fab0f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53232/0 (socket says 172.21.15.62:53232) 2023-07-14T16:32:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a179b700 1 -- 172.21.15.62:0/1662782710 learned_addr learned my addr 172.21.15.62:0/1662782710 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a0f9a700 1 --2- 172.21.15.62:0/1662782710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 0x5587fab246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a179b700 1 -- 172.21.15.62:0/1662782710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab24f30 msgr2=0x5587fab2aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a179b700 1 --2- 172.21.15.62:0/1662782710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab24f30 0x5587fab2aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a179b700 1 -- 172.21.15.62:0/1662782710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 msgr2=0x5587fab246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a179b700 1 --2- 172.21.15.62:0/1662782710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 0x5587fab246c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.252+0000 7ff3a179b700 1 -- 172.21.15.62:0/1662782710 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587fa8b9250 con 0x5587fab134d0 2023-07-14T16:32:38.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.253+0000 7ff3a179b700 1 --2- 172.21.15.62:0/1662782710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab134d0 0x5587fab0f270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff3940b7e00 tx=0x7ff3940f1a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8683aa28424d468 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.254+0000 7ff387fff700 1 -- 172.21.15.62:0/1662782710 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940ee230 con 0x5587fab134d0 2023-07-14T16:32:38.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.254+0000 7ff387fff700 1 -- 172.21.15.62:0/1662782710 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3940f1520 con 0x5587fab134d0 2023-07-14T16:32:38.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.254+0000 7ff387fff700 1 -- 172.21.15.62:0/1662782710 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940baed0 con 0x5587fab134d0 2023-07-14T16:32:38.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1662782710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab134d0 msgr2=0x5587fab0f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1662782710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab134d0 0x5587fab0f270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff3940b7e00 tx=0x7ff3940f1a10 comp rx=0 tx=0).stop 2023-07-14T16:32:38.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1662782710 shutdown_connections 2023-07-14T16:32:38.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1662782710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab24f30 0x5587fab2aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1662782710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab134d0 0x5587fab0f270 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1662782710 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab14c70 0x5587fab246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1662782710 >> 172.21.15.62:0/1662782710 conn(0x5587fab08da0 msgr2=0x5587fab1be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1662782710 shutdown_connections 2023-07-14T16:32:38.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.255+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1662782710 wait complete. 2023-07-14T16:32:38.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.256+0000 7ff3b72d1b80 1 -- start start 2023-07-14T16:32:38.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.256+0000 7ff3b72d1b80 1 Processor -- start 2023-07-14T16:32:38.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 0x5587fab20850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab14c70 0x5587fab20d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3b72d1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab24f30 0x5587fab23660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fab23ba0 con 0x5587fab134d0 2023-07-14T16:32:38.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fab23d10 con 0x5587fab14c70 2023-07-14T16:32:38.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3b72d1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587fab23ed0 con 0x5587fab24f30 2023-07-14T16:32:38.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3b72d1b80 10 client.0 _ll_register_callbacks cb 0x7ff38c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:38.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3a0f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab14c70 0x5587fab20d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3a0f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab14c70 0x5587fab20d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40656/0 (socket says 172.21.15.62:40656) 2023-07-14T16:32:38.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.258+0000 7ff3a0f9a700 1 -- 172.21.15.62:0/1613632076 learned_addr learned my addr 172.21.15.62:0/1613632076 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.259+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab24f30 0x5587fab23660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.259+0000 7ff3a1f9c700 1 -- 172.21.15.62:0/1613632076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab14c70 msgr2=0x5587fab20d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.259+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab14c70 0x5587fab20d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.259+0000 7ff3a1f9c700 1 -- 172.21.15.62:0/1613632076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 msgr2=0x5587fab20850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:32:38.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.259+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 0x5587fab20850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.234 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.259+0000 7ff3a1f9c700 1 -- 172.21.15.62:0/1613632076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587fa892de0 con 0x5587fab24f30 2023-07-14T16:32:38.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.264+0000 7ff3a1f9c700 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab24f30 0x5587fab23660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff39800f4c0 tx=0x7ff398014740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41e726bfd5a67c0d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.264+0000 7ff384ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:32:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.264+0000 7ff384ff9700 1 -- 172.21.15.62:0/1613632076 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398015610 con 0x5587fab24f30 2023-07-14T16:32:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.265+0000 7ff384ff9700 1 -- 172.21.15.62:0/1613632076 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff398015770 con 0x5587fab24f30 2023-07-14T16:32:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.265+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1613632076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587fa8b3f70 con 0x5587fab24f30 2023-07-14T16:32:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.265+0000 7ff384ff9700 1 -- 172.21.15.62:0/1613632076 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff398015930 con 0x5587fab24f30 2023-07-14T16:32:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.266+0000 7ff3b72d1b80 10 client.29653 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.266+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1613632076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5587fa8912b0 con 0x5587fab24f30 2023-07-14T16:32:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.266+0000 7ff384ff9700 1 -- 172.21.15.62:0/1613632076 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5587fa8b3f70 con 0x5587fab24f30 2023-07-14T16:32:38.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.267+0000 7ff384ff9700 1 -- 172.21.15.62:0/1613632076 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff39802aac0 con 0x5587fab24f30 2023-07-14T16:32:38.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.267+0000 7ff3b72d1b80 10 client.29653 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:38.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.267+0000 7ff3b72d1b80 10 client.29653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:38.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.267+0000 7ff3b72d1b80 20 client.29653 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:38.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.267+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fab7dae0 0x5587fab7fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.267+0000 7ff3b72d1b80 4 client.29653 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:32:38.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.267+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1613632076 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587fa8e8f60 con 0x5587fab7dae0 2023-07-14T16:32:38.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.268+0000 7ff3a179b700 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fab7dae0 0x5587fab7fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.268+0000 7ff3a179b700 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fab7dae0 0x5587fab7fef0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e406e9867383106a server_cookie=37c4e05addb827cf in_seq=0 out_seq=0 2023-07-14T16:32:38.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.268+0000 7ff384ff9700 10 client.29653 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff384ff9700 1 -- 172.21.15.62:0/1613632076 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587fa8e8f60 con 0x5587fab7dae0 2023-07-14T16:32:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff384ff9700 10 client.29653 handle_command_reply: tid=0 2023-07-14T16:32:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 client.29653 shutdown 2023-07-14T16:32:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1613632076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab24f30 msgr2=0x5587fab23660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab24f30 0x5587fab23660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff39800f4c0 tx=0x7ff398014740 comp rx=0 tx=0).stop 2023-07-14T16:32:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1613632076 shutdown_connections 2023-07-14T16:32:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5587fab24f30 0x5587fab23660 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5587fab134d0 0x5587fab20850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5587fab7dae0 0x5587fab7fef0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 --2- 172.21.15.62:0/1613632076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587fab14c70 0x5587fab20d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1613632076 >> 172.21.15.62:0/1613632076 conn(0x5587fab08da0 msgr2=0x5587fab2aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.269+0000 7ff384ff9700 0 client.29653 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.270+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1613632076 shutdown_connections 2023-07-14T16:32:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.270+0000 7ff3b72d1b80 1 -- 172.21.15.62:0/1613632076 wait complete. 2023-07-14T16:32:38.245 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.273+0000 7ff3b72d1b80 20 client.29653 trim_cache size 0 max 16384 2023-07-14T16:32:38.246 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:32:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.274+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3880a1220 msgr2=0x7ff3880a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.274+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3880a1220 0x7ff3880a36d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff398006fd0 tx=0x7ff398008040 comp rx=0 tx=0).stop 2023-07-14T16:32:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.274+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2508725293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c071830 msgr2=0x7ff39c113ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.274+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c071830 0x7ff39c113ce0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b9e0 tx=0x7ff39400bda0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.275+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2508725293 shutdown_connections 2023-07-14T16:32:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.275+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff39c1b2800 0x7ff39c1b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.275+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff39c071830 0x7ff39c113ce0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.275+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7ff3880a1220 0x7ff3880a36d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.275+0000 7ff3a39ff700 1 --2- 172.21.15.62:0/2508725293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff39c10bd60 0x7ff39c114220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.249 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.275+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2508725293 >> 172.21.15.62:0/2508725293 conn(0x7ff39c06ca90 msgr2=0x7ff39c10ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.275+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2508725293 shutdown_connections 2023-07-14T16:32:38.250 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:38.275+0000 7ff3a39ff700 1 -- 172.21.15.62:0/2508725293 wait complete. 2023-07-14T16:32:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.666+0000 7f149359d700 1 -- 172.21.15.62:0/2358737321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 msgr2=0x7f148c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.666+0000 7f149359d700 1 --2- 172.21.15.62:0/2358737321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 0x7f148c107bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f14880098f0 tx=0x7f1488009c00 comp rx=0 tx=0).stop 2023-07-14T16:32:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.667+0000 7f149359d700 1 -- 172.21.15.62:0/2358737321 shutdown_connections 2023-07-14T16:32:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.667+0000 7f149359d700 1 --2- 172.21.15.62:0/2358737321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c071070 0x7f148c0714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.667+0000 7f149359d700 1 --2- 172.21.15.62:0/2358737321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 0x7f148c107bc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.667+0000 7f149359d700 1 --2- 172.21.15.62:0/2358737321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f148c108100 0x7f148c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.667+0000 7f149359d700 1 -- 172.21.15.62:0/2358737321 >> 172.21.15.62:0/2358737321 conn(0x7f148c06c910 msgr2=0x7f148c06ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.667+0000 7f149359d700 1 -- 172.21.15.62:0/2358737321 shutdown_connections 2023-07-14T16:32:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.667+0000 7f149359d700 1 -- 172.21.15.62:0/2358737321 wait complete. 2023-07-14T16:32:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.669+0000 7f149359d700 1 Processor -- start 2023-07-14T16:32:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.669+0000 7f149359d700 1 -- start start 2023-07-14T16:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.669+0000 7f149359d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c071070 0x7f148c116f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.670+0000 7f149359d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 0x7f148c117440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.670+0000 7f149359d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f148c108100 0x7f148c118480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.670+0000 7f149359d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c118a50 con 0x7f148c071070 2023-07-14T16:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.670+0000 7f149359d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c118bc0 con 0x7f148c108100 2023-07-14T16:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.670+0000 7f149359d700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c118d80 con 0x7f148c1077b0 2023-07-14T16:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.670+0000 7f1491d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 0x7f148c117440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.670+0000 7f1491d9a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 0x7f148c117440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53258/0 (socket says 172.21.15.62:53258) 2023-07-14T16:32:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.670+0000 7f1491d9a700 1 -- 172.21.15.62:0/1876949201 learned_addr learned my addr 172.21.15.62:0/1876949201 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.671+0000 7f1491d9a700 1 -- 172.21.15.62:0/1876949201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f148c108100 msgr2=0x7f148c118480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:32:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.671+0000 7f1491d9a700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f148c108100 0x7f148c118480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.671+0000 7f1491d9a700 1 -- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c071070 msgr2=0x7f148c116f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:32:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.671+0000 7f1491d9a700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c071070 0x7f148c116f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.671+0000 7f1491d9a700 1 -- 172.21.15.62:0/1876949201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1488009520 con 0x7f148c1077b0 2023-07-14T16:32:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.672+0000 7f1491d9a700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 0x7f148c117440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f147c00d950 tx=0x7f147c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.673+0000 7f14837fe700 1 -- 172.21.15.62:0/1876949201 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c009930 con 0x7f148c1077b0 2023-07-14T16:32:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.673+0000 7f149359d700 1 -- 172.21.15.62:0/1876949201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c119000 con 0x7f148c1077b0 2023-07-14T16:32:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.673+0000 7f149359d700 1 -- 172.21.15.62:0/1876949201 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c119560 con 0x7f148c1077b0 2023-07-14T16:32:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.673+0000 7f14837fe700 1 -- 172.21.15.62:0/1876949201 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f147c010460 con 0x7f148c1077b0 2023-07-14T16:32:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.673+0000 7f14837fe700 1 -- 172.21.15.62:0/1876949201 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c00f620 con 0x7f148c1077b0 2023-07-14T16:32:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.681+0000 7f14a81e4b80 1 Processor -- start 2023-07-14T16:32:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.681+0000 7f14a81e4b80 1 -- start start 2023-07-14T16:32:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38ab36b0 0x55fb38c57550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c57a90 0x55fb38c582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c587e0 0x55fb38c58c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38b5cc50 con 0x55fb38c57a90 2023-07-14T16:32:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38baf9e0 con 0x55fb38c587e0 2023-07-14T16:32:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38ba8e70 con 0x55fb38ab36b0 2023-07-14T16:32:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f1492d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38ab36b0 0x55fb38c57550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f1492d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38ab36b0 0x55fb38c57550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53272/0 (socket says 172.21.15.62:53272) 2023-07-14T16:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f1492d9c700 1 -- 172.21.15.62:0/2044497384 learned_addr learned my addr 172.21.15.62:0/2044497384 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.682+0000 7f149259b700 1 --2- 172.21.15.62:0/2044497384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c57a90 0x55fb38c582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.683+0000 7f1492d9c700 1 -- 172.21.15.62:0/2044497384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c587e0 msgr2=0x55fb38c58c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:32:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.683+0000 7f1492d9c700 1 --2- 172.21.15.62:0/2044497384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c587e0 0x55fb38c58c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.683+0000 7f1492d9c700 1 -- 172.21.15.62:0/2044497384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c57a90 msgr2=0x55fb38c582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.683+0000 7f1492d9c700 1 --2- 172.21.15.62:0/2044497384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c57a90 0x55fb38c582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.683+0000 7f1492d9c700 1 -- 172.21.15.62:0/2044497384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb389d6d80 con 0x55fb38ab36b0 2023-07-14T16:32:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.684+0000 7f1492d9c700 1 --2- 172.21.15.62:0/2044497384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38ab36b0 0x55fb38c57550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f148400e4e0 tx=0x7f14840099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca0b8b6e50037e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.684+0000 7f14837fe700 1 -- 172.21.15.62:0/1876949201 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f147c00f780 con 0x7f148c1077b0 2023-07-14T16:32:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.686+0000 7f14837fe700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14780a1290 0x7f14780a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.686+0000 7f14837fe700 1 -- 172.21.15.62:0/1876949201 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f147c0ccad0 con 0x7f148c1077b0 2023-07-14T16:32:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.688+0000 7f1480ff9700 1 -- 172.21.15.62:0/2044497384 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484017070 con 0x55fb38ab36b0 2023-07-14T16:32:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.688+0000 7f1480ff9700 1 -- 172.21.15.62:0/2044497384 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f148400e050 con 0x55fb38ab36b0 2023-07-14T16:32:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.688+0000 7f1480ff9700 1 -- 172.21.15.62:0/2044497384 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484007cb0 con 0x55fb38ab36b0 2023-07-14T16:32:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.688+0000 7f149259b700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14780a1290 0x7f14780a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2044497384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38ab36b0 msgr2=0x55fb38c57550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/2044497384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38ab36b0 0x55fb38c57550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f148400e4e0 tx=0x7f14840099e0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f149259b700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14780a1290 0x7f14780a3740 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55fb38c52cb0 tx=0x7f148800fb80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2044497384 shutdown_connections 2023-07-14T16:32:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/2044497384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c587e0 0x55fb38c58c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/2044497384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38ab36b0 0x55fb38c57550 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/2044497384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c57a90 0x55fb38c582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2044497384 >> 172.21.15.62:0/2044497384 conn(0x55fb38c4cd40 msgr2=0x55fb38c51e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2044497384 shutdown_connections 2023-07-14T16:32:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.689+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2044497384 wait complete. 2023-07-14T16:32:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.690+0000 7f14a81e4b80 1 -- start start 2023-07-14T16:32:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.690+0000 7f14a81e4b80 1 Processor -- start 2023-07-14T16:32:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c57a90 0x55fb38c61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c587e0 0x55fb38c68060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38b48a60 con 0x55fb38c587e0 2023-07-14T16:32:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38b48760 con 0x55fb38ab36b0 2023-07-14T16:32:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38a2c8c0 con 0x55fb38c57a90 2023-07-14T16:32:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f14a81e4b80 10 client.0 _ll_register_callbacks cb 0x55fb38b46700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f1492d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c587e0 0x55fb38c68060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f1492d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c587e0 0x55fb38c68060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41786/0 (socket says 172.21.15.62:41786) 2023-07-14T16:32:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.691+0000 7f1492d9c700 1 -- 172.21.15.62:0/494227777 learned_addr learned my addr 172.21.15.62:0/494227777 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.692+0000 7f149259b700 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.692+0000 7f1491d9a700 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c57a90 0x55fb38c61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.692+0000 7f1491d9a700 1 -- 172.21.15.62:0/494227777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38ab36b0 msgr2=0x55fb38c60ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.692+0000 7f1491d9a700 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.692+0000 7f1491d9a700 1 -- 172.21.15.62:0/494227777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c587e0 msgr2=0x55fb38c68060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.692+0000 7f1491d9a700 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c587e0 0x55fb38c68060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.692+0000 7f1491d9a700 1 -- 172.21.15.62:0/494227777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148400e1c0 con 0x55fb38c57a90 2023-07-14T16:32:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.694+0000 7f1491d9a700 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c57a90 0x55fb38c61020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f147c000c00 tx=0x7f147c099f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f201835c3aac189d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.694+0000 7f1475ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:32:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.695+0000 7f1475ffb700 1 -- 172.21.15.62:0/494227777 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c09a2f0 con 0x55fb38c57a90 2023-07-14T16:32:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.695+0000 7f1475ffb700 1 -- 172.21.15.62:0/494227777 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f147c099380 con 0x55fb38c57a90 2023-07-14T16:32:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.695+0000 7f14a81e4b80 1 -- 172.21.15.62:0/494227777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb38b02490 con 0x55fb38c57a90 2023-07-14T16:32:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.695+0000 7f1475ffb700 1 -- 172.21.15.62:0/494227777 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147c099540 con 0x55fb38c57a90 2023-07-14T16:32:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.697+0000 7f14a81e4b80 10 client.29665 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.697+0000 7f14a81e4b80 1 -- 172.21.15.62:0/494227777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55fb389bfc00 con 0x55fb38c57a90 2023-07-14T16:32:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.698+0000 7f1475ffb700 1 -- 172.21.15.62:0/494227777 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fb38b02490 con 0x55fb38c57a90 2023-07-14T16:32:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.698+0000 7f1475ffb700 1 -- 172.21.15.62:0/494227777 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f147c0adb90 con 0x55fb38c57a90 2023-07-14T16:32:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.698+0000 7f14a81e4b80 10 client.29665 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.698+0000 7f14a81e4b80 10 client.29665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.698+0000 7f14a81e4b80 20 client.29665 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.698+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fb38c6c910 0x55fb38c6ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.698+0000 7f14a81e4b80 4 client.29665 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.698+0000 7f14a81e4b80 1 -- 172.21.15.62:0/494227777 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb38ba1e50 con 0x55fb38c6c910 2023-07-14T16:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.699+0000 7f149259b700 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fb38c6c910 0x55fb38c6ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.700+0000 7f149259b700 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fb38c6c910 0x55fb38c6ed20 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=611a5f40105ef459 server_cookie=d887ad0284c3b8d1 in_seq=0 out_seq=0 2023-07-14T16:32:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.700+0000 7f1475ffb700 10 client.29665 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.700+0000 7f1475ffb700 1 -- 172.21.15.62:0/494227777 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fb38ba1e50 con 0x55fb38c6c910 2023-07-14T16:32:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.700+0000 7f1475ffb700 10 client.29665 handle_command_reply: tid=0 2023-07-14T16:32:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.700+0000 7f14a81e4b80 1 client.29665 shutdown 2023-07-14T16:32:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 -- 172.21.15.62:0/494227777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c57a90 msgr2=0x55fb38c61020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c57a90 0x55fb38c61020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f147c000c00 tx=0x7f147c099f80 comp rx=0 tx=0).stop 2023-07-14T16:32:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 -- 172.21.15.62:0/494227777 shutdown_connections 2023-07-14T16:32:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38c587e0 0x55fb38c68060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fb38c6c910 0x55fb38c6ed20 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/494227777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c57a90 0x55fb38c61020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 -- 172.21.15.62:0/494227777 >> 172.21.15.62:0/494227777 conn(0x55fb38c4cd40 msgr2=0x55fb38c51640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f1475ffb700 0 client.29665 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.701+0000 7f14a81e4b80 1 -- 172.21.15.62:0/494227777 shutdown_connections 2023-07-14T16:32:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.702+0000 7f14a81e4b80 1 -- 172.21.15.62:0/494227777 wait complete. 2023-07-14T16:32:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.702+0000 7f14a81e4b80 20 client.29665 trim_cache size 0 max 16384 2023-07-14T16:32:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.726+0000 7f14a81e4b80 1 Processor -- start 2023-07-14T16:32:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.726+0000 7f14a81e4b80 1 -- start start 2023-07-14T16:32:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.726+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.726+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c68020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.726+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.726+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38c4ee60 con 0x55fb38ab36b0 2023-07-14T16:32:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.726+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38c1d0c0 con 0x55fb38c57a90 2023-07-14T16:32:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.726+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38b02490 con 0x55fb38c587e0 2023-07-14T16:32:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.728+0000 7f1492d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.728+0000 7f1492d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c61020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53288/0 (socket says 172.21.15.62:53288) 2023-07-14T16:32:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.728+0000 7f1492d9c700 1 -- 172.21.15.62:0/2933809462 learned_addr learned my addr 172.21.15.62:0/2933809462 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.728+0000 7f1491d9a700 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c68020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.728+0000 7f149259b700 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.729+0000 7f149259b700 1 -- 172.21.15.62:0/2933809462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 msgr2=0x55fb38c61020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.729+0000 7f149259b700 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c61020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.729+0000 7f149259b700 1 -- 172.21.15.62:0/2933809462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 msgr2=0x55fb38c68020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.729+0000 7f149259b700 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c68020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.729+0000 7f149259b700 1 -- 172.21.15.62:0/2933809462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb389fd1f0 con 0x55fb38ab36b0 2023-07-14T16:32:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.730+0000 7f149259b700 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1488000c00 tx=0x7f148801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2948a29d1a360607 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.730+0000 7f14817fa700 1 -- 172.21.15.62:0/2933809462 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148802b040 con 0x55fb38ab36b0 2023-07-14T16:32:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.730+0000 7f14817fa700 1 -- 172.21.15.62:0/2933809462 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f148802fe70 con 0x55fb38ab36b0 2023-07-14T16:32:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.730+0000 7f14817fa700 1 -- 172.21.15.62:0/2933809462 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148802f780 con 0x55fb38ab36b0 2023-07-14T16:32:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.731+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2933809462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 msgr2=0x55fb38c60ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.731+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1488000c00 tx=0x7f148801d000 comp rx=0 tx=0).stop 2023-07-14T16:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.734+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2933809462 shutdown_connections 2023-07-14T16:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.734+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c61020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.734+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c60ae0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.734+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/2933809462 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c68020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.734+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2933809462 >> 172.21.15.62:0/2933809462 conn(0x55fb38c4cd40 msgr2=0x55fb38c5f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.735+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2933809462 shutdown_connections 2023-07-14T16:32:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.735+0000 7f14a81e4b80 1 -- 172.21.15.62:0/2933809462 wait complete. 2023-07-14T16:32:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.735+0000 7f14a81e4b80 1 -- start start 2023-07-14T16:32:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.736+0000 7f14a81e4b80 1 Processor -- start 2023-07-14T16:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.736+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c63ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.736+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.736+0000 7f14a81e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c67200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.736+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38c67740 con 0x55fb38ab36b0 2023-07-14T16:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.736+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38c678b0 con 0x55fb38c57a90 2023-07-14T16:32:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.736+0000 7f14a81e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb38c67a70 con 0x55fb38c587e0 2023-07-14T16:32:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.736+0000 7f14a81e4b80 10 client.0 _ll_register_callbacks cb 0x7f147c099ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.737+0000 7f1492d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c67200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.737+0000 7f1491d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.737+0000 7f1491d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40700/0 (socket says 172.21.15.62:40700) 2023-07-14T16:32:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.737+0000 7f1491d9a700 1 -- 172.21.15.62:0/3230051282 learned_addr learned my addr 172.21.15.62:0/3230051282 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.737+0000 7f149259b700 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c63ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.738+0000 7f1491d9a700 1 -- 172.21.15.62:0/3230051282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 msgr2=0x55fb38c67200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.738+0000 7f1491d9a700 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c67200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.738+0000 7f1491d9a700 1 -- 172.21.15.62:0/3230051282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 msgr2=0x55fb38c63ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.738+0000 7f1491d9a700 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c63ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.738+0000 7f1491d9a700 1 -- 172.21.15.62:0/3230051282 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c110ef0 con 0x55fb38c57a90 2023-07-14T16:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.739+0000 7f1491d9a700 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c641e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f147c0adf10 tx=0x7f147c0adda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0237f39b46088b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.739+0000 7f1475ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.739+0000 7f1475ffb700 1 -- 172.21.15.62:0/3230051282 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c014070 con 0x55fb38c57a90 2023-07-14T16:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.739+0000 7f1475ffb700 1 -- 172.21.15.62:0/3230051282 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f147c099580 con 0x55fb38c57a90 2023-07-14T16:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.739+0000 7f14a81e4b80 1 -- 172.21.15.62:0/3230051282 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f148400e050 con 0x55fb38c57a90 2023-07-14T16:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.740+0000 7f1475ffb700 1 -- 172.21.15.62:0/3230051282 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147c099c60 con 0x55fb38c57a90 2023-07-14T16:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.741+0000 7f14a81e4b80 10 client.29792 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.741+0000 7f14a81e4b80 1 -- 172.21.15.62:0/3230051282 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55fb389d5250 con 0x55fb38c57a90 2023-07-14T16:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.742+0000 7f1475ffb700 1 -- 172.21.15.62:0/3230051282 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f148400e050 con 0x55fb38c57a90 2023-07-14T16:32:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.743+0000 7f1475ffb700 1 -- 172.21.15.62:0/3230051282 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f147c0ca140 con 0x55fb38c57a90 2023-07-14T16:32:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.743+0000 7f14a81e4b80 10 client.29792 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.743+0000 7f14a81e4b80 10 client.29792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.743+0000 7f14a81e4b80 20 client.29792 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.743+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fb38cc1140 0x55fb38cc3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.743+0000 7f14a81e4b80 4 client.29792 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:32:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.743+0000 7f14a81e4b80 1 -- 172.21.15.62:0/3230051282 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb38b63ff0 con 0x55fb38cc1140 2023-07-14T16:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.744+0000 7f149259b700 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fb38cc1140 0x55fb38cc3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.744+0000 7f149259b700 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fb38cc1140 0x55fb38cc3550 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf7b0f6952953ce server_cookie=573bad7e41094225 in_seq=0 out_seq=0 2023-07-14T16:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.745+0000 7f1475ffb700 10 client.29792 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.759+0000 7f1475ffb700 1 -- 172.21.15.62:0/3230051282 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb38b63ff0 con 0x55fb38cc1140 2023-07-14T16:32:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.759+0000 7f1475ffb700 10 client.29792 handle_command_reply: tid=0 2023-07-14T16:32:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.759+0000 7f14a81e4b80 1 client.29792 shutdown 2023-07-14T16:32:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f14a81e4b80 1 -- 172.21.15.62:0/3230051282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 msgr2=0x55fb38c641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c641e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f147c0adf10 tx=0x7f147c0adda0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f14a81e4b80 1 -- 172.21.15.62:0/3230051282 shutdown_connections 2023-07-14T16:32:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fb38c587e0 0x55fb38c67200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb38ab36b0 0x55fb38c63ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fb38cc1140 0x55fb38cc3550 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f14a81e4b80 1 --2- 172.21.15.62:0/3230051282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb38c57a90 0x55fb38c641e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f14a81e4b80 1 -- 172.21.15.62:0/3230051282 >> 172.21.15.62:0/3230051282 conn(0x55fb38c4cd40 msgr2=0x55fb38c60a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.760+0000 7f1475ffb700 0 client.29792 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.761+0000 7f14a81e4b80 1 -- 172.21.15.62:0/3230051282 shutdown_connections 2023-07-14T16:32:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.761+0000 7f14a81e4b80 1 -- 172.21.15.62:0/3230051282 wait complete. 2023-07-14T16:32:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.762+0000 7f14a81e4b80 20 client.29792 trim_cache size 0 max 16384 2023-07-14T16:32:38.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:32:38.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:32:38.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:32:38.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:32:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.763+0000 7f149359d700 1 -- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14780a1290 msgr2=0x7f14780a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.763+0000 7f149359d700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14780a1290 0x7f14780a3740 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55fb38c52cb0 tx=0x7f148800fb80 comp rx=0 tx=0).stop 2023-07-14T16:32:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.763+0000 7f149359d700 1 -- 172.21.15.62:0/1876949201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 msgr2=0x7f148c117440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.763+0000 7f149359d700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 0x7f148c117440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f147c00d950 tx=0x7f147c00dd10 comp rx=0 tx=0).stop 2023-07-14T16:32:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.763+0000 7f149359d700 1 -- 172.21.15.62:0/1876949201 shutdown_connections 2023-07-14T16:32:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.763+0000 7f149359d700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f14780a1290 0x7f14780a3740 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.763+0000 7f149359d700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f148c108100 0x7f148c118480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.763+0000 7f149359d700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c071070 0x7f148c116f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.764+0000 7f149359d700 1 --2- 172.21.15.62:0/1876949201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f148c1077b0 0x7f148c117440 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.764+0000 7f149359d700 1 -- 172.21.15.62:0/1876949201 >> 172.21.15.62:0/1876949201 conn(0x7f148c06c910 msgr2=0x7f148c06f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.764+0000 7f149359d700 1 -- 172.21.15.62:0/1876949201 shutdown_connections 2023-07-14T16:32:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:38.764+0000 7f149359d700 1 -- 172.21.15.62:0/1876949201 wait complete. 2023-07-14T16:32:39.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:38 smithi062 ceph-mon[95543]: pgmap v867: 97 pgs: 97 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 5.12k op/s 2023-07-14T16:32:39.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:38 smithi081 ceph-mon[127795]: pgmap v867: 97 pgs: 97 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 5.12k op/s 2023-07-14T16:32:39.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:38 smithi164 ceph-mon[97427]: pgmap v867: 97 pgs: 97 active+clean; 13 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 5.12k op/s 2023-07-14T16:32:39.280 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:32:39.280 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:32:39.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.731+0000 7fa0579ff700 1 -- 172.21.15.62:0/1702153860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0501061a0 msgr2=0x7fa0501065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.704 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.731+0000 7fa0579ff700 1 --2- 172.21.15.62:0/1702153860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0501061a0 0x7fa0501065b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009990 tx=0x7fa04c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.732+0000 7fa0579ff700 1 -- 172.21.15.62:0/1702153860 shutdown_connections 2023-07-14T16:32:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.732+0000 7fa0579ff700 1 --2- 172.21.15.62:0/1702153860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa050107d20 0x7fa05010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.732+0000 7fa0579ff700 1 --2- 172.21.15.62:0/1702153860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0501061a0 0x7fa0501065b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.732+0000 7fa0579ff700 1 --2- 172.21.15.62:0/1702153860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa050107390 0x7fa0501077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.732+0000 7fa0579ff700 1 -- 172.21.15.62:0/1702153860 >> 172.21.15.62:0/1702153860 conn(0x7fa0500751c0 msgr2=0x7fa050077610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.733+0000 7fa0579ff700 1 -- 172.21.15.62:0/1702153860 shutdown_connections 2023-07-14T16:32:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.733+0000 7fa0579ff700 1 -- 172.21.15.62:0/1702153860 wait complete. 2023-07-14T16:32:39.708 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.736+0000 7fa0579ff700 1 Processor -- start 2023-07-14T16:32:39.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.736+0000 7fa0579ff700 1 -- start start 2023-07-14T16:32:39.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.736+0000 7fa0579ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0501061a0 0x7fa05006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.736+0000 7fa0579ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa050107390 0x7fa05006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.736+0000 7fa0579ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa050107d20 0x7fa0501abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.736+0000 7fa0579ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05006f950 con 0x7fa050107390 2023-07-14T16:32:39.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.737+0000 7fa0579ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05006fac0 con 0x7fa050107d20 2023-07-14T16:32:39.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.737+0000 7fa0579ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0501ac3d0 con 0x7fa0501061a0 2023-07-14T16:32:39.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa054f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa050107390 0x7fa05006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa055f9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa050107d20 0x7fa0501abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa055f9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa050107d20 0x7fa0501abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40708/0 (socket says 172.21.15.62:40708) 2023-07-14T16:32:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa055f9c700 1 -- 172.21.15.62:0/3219423598 learned_addr learned my addr 172.21.15.62:0/3219423598 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa05579b700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0501061a0 0x7fa05006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa055f9c700 1 -- 172.21.15.62:0/3219423598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0501061a0 msgr2=0x7fa05006ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa055f9c700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0501061a0 0x7fa05006ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa055f9c700 1 -- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa050107390 msgr2=0x7fa05006f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa055f9c700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa050107390 0x7fa05006f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.738+0000 7fa055f9c700 1 -- 172.21.15.62:0/3219423598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c0095f0 con 0x7fa050107d20 2023-07-14T16:32:39.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.739+0000 7fa054f9a700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa050107390 0x7fa05006f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:32:39.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.739+0000 7fa055f9c700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa050107d20 0x7fa0501abe90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa04800ef30 tx=0x7fa04800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.744+0000 7fa0467fc700 1 -- 172.21.15.62:0/3219423598 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048018070 con 0x7fa050107d20 2023-07-14T16:32:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.745+0000 7fa0467fc700 1 -- 172.21.15.62:0/3219423598 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa04800f040 con 0x7fa050107d20 2023-07-14T16:32:39.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.745+0000 7fa0467fc700 1 -- 172.21.15.62:0/3219423598 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800cbc0 con 0x7fa050107d20 2023-07-14T16:32:39.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.746+0000 7fa0579ff700 1 -- 172.21.15.62:0/3219423598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0501ac650 con 0x7fa050107d20 2023-07-14T16:32:39.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.746+0000 7fa0579ff700 1 -- 172.21.15.62:0/3219423598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0501aca80 con 0x7fa050107d20 2023-07-14T16:32:39.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.755+0000 7fa06b273b80 1 Processor -- start 2023-07-14T16:32:39.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.756+0000 7fa0467fc700 1 -- 172.21.15.62:0/3219423598 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa04800cd20 con 0x7fa050107d20 2023-07-14T16:32:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.756+0000 7fa06b273b80 1 -- start start 2023-07-14T16:32:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.757+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93550 0x564181f93960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.757+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181f93ea0 0x564181f94760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.757+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f94cf0 0x564181f95160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.757+0000 7fa06b273b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181e98d30 con 0x564181f94cf0 2023-07-14T16:32:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.757+0000 7fa06b273b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181eebac0 con 0x564181f93550 2023-07-14T16:32:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.757+0000 7fa06b273b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181ee4f50 con 0x564181f93ea0 2023-07-14T16:32:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.758+0000 7fa054f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f94cf0 0x564181f95160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.758+0000 7fa054f9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f94cf0 0x564181f95160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41846/0 (socket says 172.21.15.62:41846) 2023-07-14T16:32:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.758+0000 7fa054f9a700 1 -- 172.21.15.62:0/2184293567 learned_addr learned my addr 172.21.15.62:0/2184293567 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.759+0000 7fa054f9a700 1 -- 172.21.15.62:0/2184293567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181f93ea0 msgr2=0x564181f94760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:32:39.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.759+0000 7fa054f9a700 1 --2- 172.21.15.62:0/2184293567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181f93ea0 0x564181f94760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.759+0000 7fa054f9a700 1 -- 172.21.15.62:0/2184293567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93550 msgr2=0x564181f93960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:39.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.759+0000 7fa054f9a700 1 --2- 172.21.15.62:0/2184293567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93550 0x564181f93960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.759+0000 7fa054f9a700 1 -- 172.21.15.62:0/2184293567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564181d12e60 con 0x564181f94cf0 2023-07-14T16:32:39.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.759+0000 7fa054f9a700 1 --2- 172.21.15.62:0/2184293567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f94cf0 0x564181f95160 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fa040007ec0 tx=0x7fa0400129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a97edd40e9ff0dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.760+0000 7fa03b7fe700 1 -- 172.21.15.62:0/2184293567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040010630 con 0x564181f94cf0 2023-07-14T16:32:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.760+0000 7fa03b7fe700 1 -- 172.21.15.62:0/2184293567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa040010c20 con 0x564181f94cf0 2023-07-14T16:32:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.760+0000 7fa03b7fe700 1 -- 172.21.15.62:0/2184293567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04001acd0 con 0x564181f94cf0 2023-07-14T16:32:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.762+0000 7fa0467fc700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa03c0a12a0 0x7fa03c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.762+0000 7fa0467fc700 1 -- 172.21.15.62:0/3219423598 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa0480cba70 con 0x7fa050107d20 2023-07-14T16:32:39.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.765+0000 7fa05579b700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa03c0a12a0 0x7fa03c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.767+0000 7fa06b273b80 1 -- 172.21.15.62:0/2184293567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f94cf0 msgr2=0x564181f95160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.767+0000 7fa06b273b80 1 --2- 172.21.15.62:0/2184293567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f94cf0 0x564181f95160 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fa040007ec0 tx=0x7fa0400129f0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.767+0000 7fa06b273b80 1 -- 172.21.15.62:0/2184293567 shutdown_connections 2023-07-14T16:32:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.767+0000 7fa06b273b80 1 --2- 172.21.15.62:0/2184293567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f94cf0 0x564181f95160 secure :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fa040007ec0 tx=0x7fa0400129f0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.767+0000 7fa06b273b80 1 --2- 172.21.15.62:0/2184293567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93550 0x564181f93960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.767+0000 7fa06b273b80 1 --2- 172.21.15.62:0/2184293567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181f93ea0 0x564181f94760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.767+0000 7fa06b273b80 1 -- 172.21.15.62:0/2184293567 >> 172.21.15.62:0/2184293567 conn(0x564181f88e20 msgr2=0x564181f8e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:39.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.769+0000 7fa06b273b80 1 -- 172.21.15.62:0/2184293567 shutdown_connections 2023-07-14T16:32:39.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.770+0000 7fa06b273b80 1 -- 172.21.15.62:0/2184293567 wait complete. 2023-07-14T16:32:39.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.770+0000 7fa06b273b80 1 -- start start 2023-07-14T16:32:39.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.771+0000 7fa06b273b80 1 Processor -- start 2023-07-14T16:32:39.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181f9d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f9d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa06b273b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181e84b40 con 0x564181f93550 2023-07-14T16:32:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa06b273b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181e84840 con 0x564181f93ea0 2023-07-14T16:32:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa06b273b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181d689a0 con 0x564181fa29d0 2023-07-14T16:32:39.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa06b273b80 10 client.0 _ll_register_callbacks cb 0x564181e827e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa054f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f9d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa054f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f9d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40730/0 (socket says 172.21.15.62:40730) 2023-07-14T16:32:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.772+0000 7fa054f9a700 1 -- 172.21.15.62:0/4092073424 learned_addr learned my addr 172.21.15.62:0/4092073424 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.773+0000 7fa05579b700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa03c0a12a0 0x7fa03c0a3750 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa04c005910 tx=0x7fa04c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:39.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.773+0000 7fa054f9a700 1 -- 172.21.15.62:0/4092073424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 msgr2=0x564181fa4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:39.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.773+0000 7fa05579b700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181f9d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.774+0000 7fa055f9c700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.774+0000 7fa054f9a700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.774+0000 7fa054f9a700 1 -- 172.21.15.62:0/4092073424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 msgr2=0x564181f9d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.774+0000 7fa054f9a700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181f9d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.774+0000 7fa054f9a700 1 -- 172.21.15.62:0/4092073424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0400123f0 con 0x564181f93ea0 2023-07-14T16:32:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.774+0000 7fa05579b700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181f9d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.775+0000 7fa054f9a700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f9d9a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa040006010 tx=0x7fa04001a4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f257d539ff841b7c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.775+0000 7fa055f9c700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:39.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.775+0000 7fa038ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:39.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.775+0000 7fa038ff9700 1 -- 172.21.15.62:0/4092073424 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040004240 con 0x564181f93ea0 2023-07-14T16:32:39.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.776+0000 7fa038ff9700 1 -- 172.21.15.62:0/4092073424 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa0400043a0 con 0x564181f93ea0 2023-07-14T16:32:39.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.776+0000 7fa06b273b80 1 -- 172.21.15.62:0/4092073424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564181e3e570 con 0x564181f93ea0 2023-07-14T16:32:39.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.776+0000 7fa038ff9700 1 -- 172.21.15.62:0/4092073424 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa04001ae90 con 0x564181f93ea0 2023-07-14T16:32:39.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.776+0000 7fa06b273b80 10 client.29801 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:39.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.776+0000 7fa06b273b80 1 -- 172.21.15.62:0/4092073424 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564181cfbce0 con 0x564181f93ea0 2023-07-14T16:32:39.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.777+0000 7fa038ff9700 1 -- 172.21.15.62:0/4092073424 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564181e3e570 con 0x564181f93ea0 2023-07-14T16:32:39.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.777+0000 7fa038ff9700 1 -- 172.21.15.62:0/4092073424 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa040003880 con 0x564181f93ea0 2023-07-14T16:32:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.777+0000 7fa06b273b80 10 client.29801 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.777+0000 7fa06b273b80 10 client.29801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.777+0000 7fa06b273b80 20 client.29801 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.777+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564181fa9350 0x564181fab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.752 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.777+0000 7fa06b273b80 4 client.29801 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:39.753 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.777+0000 7fa06b273b80 1 -- 172.21.15.62:0/4092073424 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564181eddf30 con 0x564181fa9350 2023-07-14T16:32:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.780+0000 7fa05579b700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564181fa9350 0x564181fab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.781+0000 7fa038ff9700 10 client.29801 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.781+0000 7fa05579b700 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564181fa9350 0x564181fab760 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f07dd27de863a9 server_cookie=2edd72b0ddb7816 in_seq=0 out_seq=0 2023-07-14T16:32:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.782+0000 7fa038ff9700 1 -- 172.21.15.62:0/4092073424 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564181eddf30 con 0x564181fa9350 2023-07-14T16:32:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.782+0000 7fa038ff9700 10 client.29801 handle_command_reply: tid=0 2023-07-14T16:32:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.782+0000 7fa06b273b80 1 client.29801 shutdown 2023-07-14T16:32:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa06b273b80 1 -- 172.21.15.62:0/4092073424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 msgr2=0x564181f9d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f9d9a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa040006010 tx=0x7fa04001a4c0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.756 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa06b273b80 1 -- 172.21.15.62:0/4092073424 shutdown_connections 2023-07-14T16:32:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181f9d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564181fa9350 0x564181fab760 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4092073424 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f9d9a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa06b273b80 1 -- 172.21.15.62:0/4092073424 >> 172.21.15.62:0/4092073424 conn(0x564181f88e20 msgr2=0x564181f8d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.783+0000 7fa038ff9700 0 client.29801 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.784+0000 7fa06b273b80 1 -- 172.21.15.62:0/4092073424 shutdown_connections 2023-07-14T16:32:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.784+0000 7fa06b273b80 1 -- 172.21.15.62:0/4092073424 wait complete. 2023-07-14T16:32:39.758 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.784+0000 7fa06b273b80 20 client.29801 trim_cache size 0 max 16384 2023-07-14T16:32:39.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.819+0000 7fa06b273b80 1 Processor -- start 2023-07-14T16:32:39.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.819+0000 7fa06b273b80 1 -- start start 2023-07-14T16:32:39.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.819+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181fb8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.819+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f97260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.819+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.819+0000 7fa06b273b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181e3e570 con 0x564181f93550 2023-07-14T16:32:39.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.819+0000 7fa06b273b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181f8f700 con 0x564181f93ea0 2023-07-14T16:32:39.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.819+0000 7fa06b273b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181fa9170 con 0x564181fa29d0 2023-07-14T16:32:39.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53352/0 (socket says 172.21.15.62:53352) 2023-07-14T16:32:39.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 -- 172.21.15.62:0/4171294226 learned_addr learned my addr 172.21.15.62:0/4171294226 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:39.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa05579b700 1 --2- 172.21.15.62:0/4171294226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181fb8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 -- 172.21.15.62:0/4171294226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 msgr2=0x564181f97260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:32:39.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 --2- 172.21.15.62:0/4171294226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f97260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 -- 172.21.15.62:0/4171294226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 msgr2=0x564181fb8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:39.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 --2- 172.21.15.62:0/4171294226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181fb8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 -- 172.21.15.62:0/4171294226 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564181f8dc00 con 0x564181fa29d0 2023-07-14T16:32:39.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.820+0000 7fa055f9c700 1 --2- 172.21.15.62:0/4171294226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa56d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa048009c10 tx=0x7fa048098870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd519363b8043a6a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:39.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa03bfff700 1 -- 172.21.15.62:0/4171294226 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04809cec0 con 0x564181fa29d0 2023-07-14T16:32:39.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa03bfff700 1 -- 172.21.15.62:0/4171294226 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa0480a12e0 con 0x564181fa29d0 2023-07-14T16:32:39.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa03bfff700 1 -- 172.21.15.62:0/4171294226 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0480a0e50 con 0x564181fa29d0 2023-07-14T16:32:39.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa06b273b80 1 -- 172.21.15.62:0/4171294226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 msgr2=0x564181fa56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4171294226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa56d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa048009c10 tx=0x7fa048098870 comp rx=0 tx=0).stop 2023-07-14T16:32:39.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa06b273b80 1 -- 172.21.15.62:0/4171294226 shutdown_connections 2023-07-14T16:32:39.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4171294226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181fa29d0 0x564181fa56d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4171294226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181f93550 0x564181fb8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4171294226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181f97260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa06b273b80 1 -- 172.21.15.62:0/4171294226 >> 172.21.15.62:0/4171294226 conn(0x564181f88e20 msgr2=0x564181fb00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:39.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.821+0000 7fa06b273b80 1 -- 172.21.15.62:0/4171294226 shutdown_connections 2023-07-14T16:32:39.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.822+0000 7fa06b273b80 1 -- 172.21.15.62:0/4171294226 wait complete. 2023-07-14T16:32:39.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.822+0000 7fa06b273b80 1 -- start start 2023-07-14T16:32:39.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.822+0000 7fa06b273b80 1 Processor -- start 2023-07-14T16:32:39.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.822+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181f93550 0x564181fb8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.822+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181fb75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa06b273b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181fa29d0 0x564181fb7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa06b273b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181fb80e0 con 0x564181fa29d0 2023-07-14T16:32:39.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa06b273b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181fb8250 con 0x564181f93ea0 2023-07-14T16:32:39.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa06b273b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564181fb8410 con 0x564181f93550 2023-07-14T16:32:39.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa06b273b80 10 client.0 _ll_register_callbacks cb 0x7fa040012610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:39.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181fb75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181fb75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40748/0 (socket says 172.21.15.62:40748) 2023-07-14T16:32:39.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 -- 172.21.15.62:0/4203086687 learned_addr learned my addr 172.21.15.62:0/4203086687 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:39.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa055f9c700 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181fa29d0 0x564181fb7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 -- 172.21.15.62:0/4203086687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181f93550 msgr2=0x564181fb8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:39.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181f93550 0x564181fb8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 -- 172.21.15.62:0/4203086687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181fa29d0 msgr2=0x564181fb7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:39.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181fa29d0 0x564181fb7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 -- 172.21.15.62:0/4203086687 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0501aca80 con 0x564181f93ea0 2023-07-14T16:32:39.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa054f9a700 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181fb75d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa040007410 tx=0x7fa0400038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c3dd982f031e87 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:39.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.823+0000 7fa038ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:32:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.824+0000 7fa038ff9700 1 -- 172.21.15.62:0/4203086687 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040005520 con 0x564181f93ea0 2023-07-14T16:32:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.824+0000 7fa06b273b80 1 -- 172.21.15.62:0/4203086687 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa040010c20 con 0x564181f93ea0 2023-07-14T16:32:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.824+0000 7fa038ff9700 1 -- 172.21.15.62:0/4203086687 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa040000e40 con 0x564181f93ea0 2023-07-14T16:32:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.824+0000 7fa038ff9700 1 -- 172.21.15.62:0/4203086687 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa040012df0 con 0x564181f93ea0 2023-07-14T16:32:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.824+0000 7fa06b273b80 10 client.29804 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.824+0000 7fa06b273b80 1 -- 172.21.15.62:0/4203086687 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564181f8a660 con 0x564181f93ea0 2023-07-14T16:32:39.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.824+0000 7fa038ff9700 1 -- 172.21.15.62:0/4203086687 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa040010c20 con 0x564181f93ea0 2023-07-14T16:32:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.824+0000 7fa038ff9700 1 -- 172.21.15.62:0/4203086687 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa04002c4c0 con 0x564181f93ea0 2023-07-14T16:32:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa06b273b80 10 client.29804 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa06b273b80 10 client.29804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa06b273b80 20 client.29804 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564181ffee50 0x564182001260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:39.804 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa06b273b80 4 client.29804 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:32:39.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa06b273b80 1 -- 172.21.15.62:0/4203086687 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564181ea00d0 con 0x564181ffee50 2023-07-14T16:32:39.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa05579b700 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564181ffee50 0x564182001260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:39.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa038ff9700 10 client.29804 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:39.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.825+0000 7fa05579b700 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564181ffee50 0x564182001260 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b643a6734de435b server_cookie=807a5a213f7f3fde in_seq=0 out_seq=0 2023-07-14T16:32:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.826+0000 7fa038ff9700 1 -- 172.21.15.62:0/4203086687 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564181ea00d0 con 0x564181ffee50 2023-07-14T16:32:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.826+0000 7fa038ff9700 10 client.29804 handle_command_reply: tid=0 2023-07-14T16:32:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 client.29804 shutdown 2023-07-14T16:32:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 -- 172.21.15.62:0/4203086687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 msgr2=0x564181fb75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181fb75d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa040007410 tx=0x7fa0400038b0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 -- 172.21.15.62:0/4203086687 shutdown_connections 2023-07-14T16:32:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564181fa29d0 0x564181fb7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564181f93550 0x564181fb8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564181ffee50 0x564182001260 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 --2- 172.21.15.62:0/4203086687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564181f93ea0 0x564181fb75d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa038ff9700 0 client.29804 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:39.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 -- 172.21.15.62:0/4203086687 >> 172.21.15.62:0/4203086687 conn(0x564181f88e20 msgr2=0x564181f97260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:39.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 -- 172.21.15.62:0/4203086687 shutdown_connections 2023-07-14T16:32:39.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 1 -- 172.21.15.62:0/4203086687 wait complete. 2023-07-14T16:32:39.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.827+0000 7fa06b273b80 20 client.29804 trim_cache size 0 max 16384 2023-07-14T16:32:39.808 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:32:39.808 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:32:39.809 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "351df358-e789-4b1b-848f-5434d9c4820e", 2023-07-14T16:32:39.809 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:32:39.809 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:32:39.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 -- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa03c0a12a0 msgr2=0x7fa03c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa03c0a12a0 0x7fa03c0a3750 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa04c005910 tx=0x7fa04c00b260 comp rx=0 tx=0).stop 2023-07-14T16:32:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 -- 172.21.15.62:0/3219423598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa050107d20 msgr2=0x7fa0501abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa050107d20 0x7fa0501abe90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa04800ef30 tx=0x7fa04800c3a0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 -- 172.21.15.62:0/3219423598 shutdown_connections 2023-07-14T16:32:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa03c0a12a0 0x7fa03c0a3750 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa050107d20 0x7fa0501abe90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa0501061a0 0x7fa05006ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 --2- 172.21.15.62:0/3219423598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa050107390 0x7fa05006f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:39.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 -- 172.21.15.62:0/3219423598 >> 172.21.15.62:0/3219423598 conn(0x7fa0500751c0 msgr2=0x7fa050077530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:39.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 -- 172.21.15.62:0/3219423598 shutdown_connections 2023-07-14T16:32:39.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:39.828+0000 7fa0579ff700 1 -- 172.21.15.62:0/3219423598 wait complete. 2023-07-14T16:32:39.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:32:40.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:39 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:32:40.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:39 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:32:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.231+0000 7f48e2bff700 1 -- 172.21.15.62:0/3712165965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 msgr2=0x7f48dc10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.231+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3712165965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 0x7f48dc10b7d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f48d00098f0 tx=0x7f48d0009c00 comp rx=0 tx=0).stop 2023-07-14T16:32:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.231+0000 7f48e2bff700 1 -- 172.21.15.62:0/3712165965 shutdown_connections 2023-07-14T16:32:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.231+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3712165965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48dc10bd60 0x7f48dc10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.231+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3712165965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48dc070ee0 0x7f48dc0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.231+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3712165965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 0x7f48dc10b7d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.231+0000 7f48e2bff700 1 -- 172.21.15.62:0/3712165965 >> 172.21.15.62:0/3712165965 conn(0x7f48dc06ca90 msgr2=0x7f48dc06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:40.205 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.233+0000 7f48e2bff700 1 -- 172.21.15.62:0/3712165965 shutdown_connections 2023-07-14T16:32:40.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.233+0000 7f48e2bff700 1 -- 172.21.15.62:0/3712165965 wait complete. 2023-07-14T16:32:40.207 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.234+0000 7f48e2bff700 1 Processor -- start 2023-07-14T16:32:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.235+0000 7f48e2bff700 1 -- start start 2023-07-14T16:32:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.235+0000 7f48e2bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48dc070ee0 0x7f48dc113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.235+0000 7f48e2bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 0x7f48dc1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.235+0000 7f48e2bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48dc10bd60 0x7f48dc1b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.235+0000 7f48e2bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc1146a0 con 0x7f48dc10bd60 2023-07-14T16:32:40.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.235+0000 7f48e2bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc114810 con 0x7f48dc071830 2023-07-14T16:32:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.235+0000 7f48e2bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc1b4c90 con 0x7f48dc070ee0 2023-07-14T16:32:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.236+0000 7f48e099b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48dc070ee0 0x7f48dc113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.236+0000 7f48dbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 0x7f48dc1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.209 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.236+0000 7f48e099b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48dc070ee0 0x7f48dc113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53378/0 (socket says 172.21.15.62:53378) 2023-07-14T16:32:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.236+0000 7f48e099b700 1 -- 172.21.15.62:0/3106951283 learned_addr learned my addr 172.21.15.62:0/3106951283 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.237+0000 7f48dbfff700 1 -- 172.21.15.62:0/3106951283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48dc070ee0 msgr2=0x7f48dc113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.237+0000 7f48dbfff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48dc070ee0 0x7f48dc113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.237+0000 7f48dbfff700 1 -- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48dc10bd60 msgr2=0x7f48dc1b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.237+0000 7f48dbfff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48dc10bd60 0x7f48dc1b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.210 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.237+0000 7f48dbfff700 1 -- 172.21.15.62:0/3106951283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4009c40 con 0x7f48dc071830 2023-07-14T16:32:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.237+0000 7f48dbfff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 0x7f48dc1140d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f48d0000c00 tx=0x7f48d000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.238+0000 7f48d9ffb700 1 -- 172.21.15.62:0/3106951283 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d001c070 con 0x7f48dc071830 2023-07-14T16:32:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.238+0000 7f48d9ffb700 1 -- 172.21.15.62:0/3106951283 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48d000f7b0 con 0x7f48dc071830 2023-07-14T16:32:40.211 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.238+0000 7f48e2bff700 1 -- 172.21.15.62:0/3106951283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d0009520 con 0x7f48dc071830 2023-07-14T16:32:40.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.238+0000 7f48e2bff700 1 -- 172.21.15.62:0/3106951283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc1b51b0 con 0x7f48dc071830 2023-07-14T16:32:40.212 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.238+0000 7f48d9ffb700 1 -- 172.21.15.62:0/3106951283 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0003970 con 0x7f48dc071830 2023-07-14T16:32:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.245+0000 7f48f64eab80 1 Processor -- start 2023-07-14T16:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.248+0000 7f48f64eab80 1 -- start start 2023-07-14T16:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.248+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298a968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.248+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a96e20 0x55c298a976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.248+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298a97c70 0x55c298a980e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.248+0000 7f48f64eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29899bcb0 con 0x55c298a96e20 2023-07-14T16:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.248+0000 7f48f64eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2989eea40 con 0x55c298a964d0 2023-07-14T16:32:40.222 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.248+0000 7f48f64eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2989e7ed0 con 0x55c298a97c70 2023-07-14T16:32:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.249+0000 7f48e119c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298a968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.249+0000 7f48e119c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298a968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40786/0 (socket says 172.21.15.62:40786) 2023-07-14T16:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.249+0000 7f48e119c700 1 -- 172.21.15.62:0/368874812 learned_addr learned my addr 172.21.15.62:0/368874812 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.249+0000 7f48dbfff700 1 --2- 172.21.15.62:0/368874812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298a97c70 0x55c298a980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48e119c700 1 -- 172.21.15.62:0/368874812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298a97c70 msgr2=0x55c298a980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48e119c700 1 --2- 172.21.15.62:0/368874812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298a97c70 0x55c298a980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48e119c700 1 -- 172.21.15.62:0/368874812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a96e20 msgr2=0x55c298a976e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48e119c700 1 --2- 172.21.15.62:0/368874812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a96e20 0x55c298a976e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48e119c700 1 -- 172.21.15.62:0/368874812 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c298815de0 con 0x55c298a964d0 2023-07-14T16:32:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48e119c700 1 --2- 172.21.15.62:0/368874812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298a968e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e590 tx=0x7f48d400e8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9044ebf2f8d8cd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48c6ffd700 1 -- 172.21.15.62:0/368874812 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017070 con 0x55c298a964d0 2023-07-14T16:32:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48c6ffd700 1 -- 172.21.15.62:0/368874812 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48d400f040 con 0x55c298a964d0 2023-07-14T16:32:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.250+0000 7f48c6ffd700 1 -- 172.21.15.62:0/368874812 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400e050 con 0x55c298a964d0 2023-07-14T16:32:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.251+0000 7f48d9ffb700 1 -- 172.21.15.62:0/3106951283 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f48d0020070 con 0x7f48dc071830 2023-07-14T16:32:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.251+0000 7f48f64eab80 1 -- 172.21.15.62:0/368874812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 msgr2=0x55c298a968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.251+0000 7f48f64eab80 1 --2- 172.21.15.62:0/368874812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298a968e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e590 tx=0x7f48d400e8a0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.252+0000 7f48f64eab80 1 -- 172.21.15.62:0/368874812 shutdown_connections 2023-07-14T16:32:40.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.252+0000 7f48f64eab80 1 --2- 172.21.15.62:0/368874812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298a97c70 0x55c298a980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.252+0000 7f48f64eab80 1 --2- 172.21.15.62:0/368874812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298a968e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.252+0000 7f48f64eab80 1 --2- 172.21.15.62:0/368874812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a96e20 0x55c298a976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.227 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.252+0000 7f48f64eab80 1 -- 172.21.15.62:0/368874812 >> 172.21.15.62:0/368874812 conn(0x55c298a8bda0 msgr2=0x55c298a911d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:40.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.254+0000 7f48d9ffb700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48cc0a1250 0x7f48cc0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.254+0000 7f48d9ffb700 1 -- 172.21.15.62:0/3106951283 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f48d00cd8d0 con 0x7f48dc071830 2023-07-14T16:32:40.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.254+0000 7f48f64eab80 1 -- 172.21.15.62:0/368874812 shutdown_connections 2023-07-14T16:32:40.228 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.255+0000 7f48e099b700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48cc0a1250 0x7f48cc0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.255+0000 7f48f64eab80 1 -- 172.21.15.62:0/368874812 wait complete. 2023-07-14T16:32:40.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.255+0000 7f48f64eab80 1 -- start start 2023-07-14T16:32:40.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.256+0000 7f48e099b700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48cc0a1250 0x7f48cc0a3700 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55c298a97210 tx=0x7f48c8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:40.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.256+0000 7f48f64eab80 1 Processor -- start 2023-07-14T16:32:40.229 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.257+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a964d0 0x55c298aa01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.257+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a97c70 0x55c298aa0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.257+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298aa7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.257+0000 7f48f64eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c298987ac0 con 0x55c298a964d0 2023-07-14T16:32:40.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.257+0000 7f48f64eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2989877c0 con 0x55c298a97c70 2023-07-14T16:32:40.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.257+0000 7f48f64eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29886b920 con 0x55c298aa5740 2023-07-14T16:32:40.230 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.257+0000 7f48f64eab80 10 client.0 _ll_register_callbacks cb 0x55c298985760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:40.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.258+0000 7f48e119c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298aa7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.258+0000 7f48e119c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298aa7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53382/0 (socket says 172.21.15.62:53382) 2023-07-14T16:32:40.231 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.258+0000 7f48e119c700 1 -- 172.21.15.62:0/891822023 learned_addr learned my addr 172.21.15.62:0/891822023 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:40.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.260+0000 7f48e119c700 1 -- 172.21.15.62:0/891822023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a97c70 msgr2=0x55c298aa0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:40.232 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.260+0000 7f48e119c700 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a97c70 0x55c298aa0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.260+0000 7f48e119c700 1 -- 172.21.15.62:0/891822023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a964d0 msgr2=0x55c298aa01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.260+0000 7f48e119c700 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a964d0 0x55c298aa01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.260+0000 7f48e119c700 1 -- 172.21.15.62:0/891822023 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d400e240 con 0x55c298aa5740 2023-07-14T16:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.261+0000 7f48e119c700 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298aa7b20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48d4004b10 tx=0x7f48d4003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d987dd606791cf04 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.261+0000 7f48bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.264+0000 7f48bbfff700 1 -- 172.21.15.62:0/891822023 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017040 con 0x55c298aa5740 2023-07-14T16:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.264+0000 7f48bbfff700 1 -- 172.21.15.62:0/891822023 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f48d40047c0 con 0x55c298aa5740 2023-07-14T16:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.264+0000 7f48f64eab80 1 -- 172.21.15.62:0/891822023 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2989414f0 con 0x55c298aa5740 2023-07-14T16:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.264+0000 7f48bbfff700 1 -- 172.21.15.62:0/891822023 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d4007500 con 0x55c298aa5740 2023-07-14T16:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.264+0000 7f48f64eab80 10 client.29692 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.264+0000 7f48f64eab80 1 -- 172.21.15.62:0/891822023 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c2987fec60 con 0x55c298aa5740 2023-07-14T16:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.265+0000 7f48bbfff700 1 -- 172.21.15.62:0/891822023 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c2989414f0 con 0x55c298aa5740 2023-07-14T16:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.266+0000 7f48bbfff700 1 -- 172.21.15.62:0/891822023 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48d401c4d0 con 0x55c298aa5740 2023-07-14T16:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.266+0000 7f48f64eab80 10 client.29692 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.266+0000 7f48f64eab80 10 client.29692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.266+0000 7f48f64eab80 20 client.29692 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.266+0000 7f48f64eab80 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c298aac250 0x55c298aae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.266+0000 7f48f64eab80 4 client.29692 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.266+0000 7f48f64eab80 1 -- 172.21.15.62:0/891822023 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2989e0eb0 con 0x55c298aac250 2023-07-14T16:32:40.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.266+0000 7f48e099b700 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c298aac250 0x55c298aae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.240 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.267+0000 7f48e099b700 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c298aac250 0x55c298aae660 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10beb7b7ef249604 server_cookie=9d728ef2a246184a in_seq=0 out_seq=0 2023-07-14T16:32:40.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.267+0000 7f48bbfff700 10 client.29692 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:40.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.267+0000 7f48bbfff700 1 -- 172.21.15.62:0/891822023 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c2989e0eb0 con 0x55c298aac250 2023-07-14T16:32:40.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.268+0000 7f48bbfff700 10 client.29692 handle_command_reply: tid=0 2023-07-14T16:32:40.241 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.268+0000 7f48f64eab80 1 client.29692 shutdown 2023-07-14T16:32:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 -- 172.21.15.62:0/891822023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 msgr2=0x55c298aa7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298aa7b20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48d4004b10 tx=0x7f48d4003f00 comp rx=0 tx=0).stop 2023-07-14T16:32:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 -- 172.21.15.62:0/891822023 shutdown_connections 2023-07-14T16:32:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298aa7b20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.242 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a964d0 0x55c298aa01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c298aac250 0x55c298aae660 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 --2- 172.21.15.62:0/891822023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a97c70 0x55c298aa0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48bbfff700 0 client.29692 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 -- 172.21.15.62:0/891822023 >> 172.21.15.62:0/891822023 conn(0x55c298a8bda0 msgr2=0x55c298a91140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 -- 172.21.15.62:0/891822023 shutdown_connections 2023-07-14T16:32:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.269+0000 7f48f64eab80 1 -- 172.21.15.62:0/891822023 wait complete. 2023-07-14T16:32:40.243 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.270+0000 7f48f64eab80 20 client.29692 trim_cache size 0 max 16384 2023-07-14T16:32:40.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:39 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:32:40.277 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.305+0000 7f48f64eab80 1 Processor -- start 2023-07-14T16:32:40.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.305+0000 7f48f64eab80 1 -- start start 2023-07-14T16:32:40.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.305+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298abbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.278 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.305+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a97c70 0x55c298a9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.305+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298a97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.305+0000 7f48f64eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c298aaecc0 con 0x55c298a97c70 2023-07-14T16:32:40.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.305+0000 7f48f64eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c298a8b970 con 0x55c298a964d0 2023-07-14T16:32:40.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.305+0000 7f48f64eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c298a92c60 con 0x55c298aa5740 2023-07-14T16:32:40.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.306+0000 7f48e099b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298abbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.307+0000 7f48e099b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298abbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40804/0 (socket says 172.21.15.62:40804) 2023-07-14T16:32:40.280 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.307+0000 7f48e099b700 1 -- 172.21.15.62:0/3164291625 learned_addr learned my addr 172.21.15.62:0/3164291625 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:40.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.307+0000 7f48e119c700 1 --2- 172.21.15.62:0/3164291625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298a97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.308+0000 7f48e119c700 1 -- 172.21.15.62:0/3164291625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 msgr2=0x55c298abbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:40.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.308+0000 7f48e119c700 1 --2- 172.21.15.62:0/3164291625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298abbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.308+0000 7f48e119c700 1 -- 172.21.15.62:0/3164291625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a97c70 msgr2=0x55c298a9ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:40.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.308+0000 7f48e119c700 1 --2- 172.21.15.62:0/3164291625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a97c70 0x55c298a9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.281 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.308+0000 7f48e119c700 1 -- 172.21.15.62:0/3164291625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c29883c250 con 0x55c298aa5740 2023-07-14T16:32:40.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.308+0000 7f48e119c700 1 --2- 172.21.15.62:0/3164291625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298a97270 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48d401c860 tx=0x7f48d400e8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eafc4e35091c294 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:40.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.309+0000 7f48c77fe700 1 -- 172.21.15.62:0/3164291625 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017040 con 0x55c298aa5740 2023-07-14T16:32:40.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.309+0000 7f48c77fe700 1 -- 172.21.15.62:0/3164291625 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48d401cad0 con 0x55c298aa5740 2023-07-14T16:32:40.282 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.309+0000 7f48c77fe700 1 -- 172.21.15.62:0/3164291625 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4004bb0 con 0x55c298aa5740 2023-07-14T16:32:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.311+0000 7f48f64eab80 1 -- 172.21.15.62:0/3164291625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 msgr2=0x55c298a97270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.311+0000 7f48f64eab80 1 --2- 172.21.15.62:0/3164291625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298a97270 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48d401c860 tx=0x7f48d400e8a0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.313+0000 7f48f64eab80 1 -- 172.21.15.62:0/3164291625 shutdown_connections 2023-07-14T16:32:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.313+0000 7f48f64eab80 1 --2- 172.21.15.62:0/3164291625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298aa5740 0x55c298a97270 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.313+0000 7f48f64eab80 1 --2- 172.21.15.62:0/3164291625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298a964d0 0x55c298abbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.313+0000 7f48f64eab80 1 --2- 172.21.15.62:0/3164291625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a97c70 0x55c298a9ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.313+0000 7f48f64eab80 1 -- 172.21.15.62:0/3164291625 >> 172.21.15.62:0/3164291625 conn(0x55c298a8bda0 msgr2=0x55c298ab2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.321+0000 7f48f64eab80 1 -- 172.21.15.62:0/3164291625 shutdown_connections 2023-07-14T16:32:40.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.322+0000 7f48f64eab80 1 -- 172.21.15.62:0/3164291625 wait complete. 2023-07-14T16:32:40.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.322+0000 7f48f64eab80 1 -- start start 2023-07-14T16:32:40.295 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.323+0000 7f48f64eab80 1 Processor -- start 2023-07-14T16:32:40.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.323+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a97c70 0x55c298aba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.323+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298aa5740 0x55c298aba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.323+0000 7f48f64eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298abb250 0x55c298abb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.323+0000 7f48f64eab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c298abbc00 con 0x55c298a97c70 2023-07-14T16:32:40.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.323+0000 7f48f64eab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c298abbd70 con 0x55c298aa5740 2023-07-14T16:32:40.297 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.323+0000 7f48f64eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c298abbf30 con 0x55c298abb250 2023-07-14T16:32:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.323+0000 7f48f64eab80 10 client.0 _ll_register_callbacks cb 0x7f48d00d3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.324+0000 7f48e119c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298abb250 0x55c298abb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.324+0000 7f48e119c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298abb250 0x55c298abb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:53402/0 (socket says 172.21.15.62:53402) 2023-07-14T16:32:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.324+0000 7f48e119c700 1 -- 172.21.15.62:0/2136205971 learned_addr learned my addr 172.21.15.62:0/2136205971 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.324+0000 7f48dbfff700 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298aa5740 0x55c298aba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.326+0000 7f48e119c700 1 -- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298aa5740 msgr2=0x55c298aba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.326+0000 7f48e119c700 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298aa5740 0x55c298aba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.326+0000 7f48e119c700 1 -- 172.21.15.62:0/2136205971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a97c70 msgr2=0x55c298aba120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.326+0000 7f48e119c700 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a97c70 0x55c298aba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.326+0000 7f48e119c700 1 -- 172.21.15.62:0/2136205971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2987fec60 con 0x55c298abb250 2023-07-14T16:32:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.326+0000 7f48dbfff700 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298aa5740 0x55c298aba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:32:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.327+0000 7f48e119c700 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298abb250 0x55c298abb6c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009790 tx=0x7f48d4003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab277cfe2e62d82c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.327+0000 7f48bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:32:40.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.327+0000 7f48bbfff700 1 -- 172.21.15.62:0/2136205971 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017070 con 0x55c298abb250 2023-07-14T16:32:40.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.327+0000 7f48bbfff700 1 -- 172.21.15.62:0/2136205971 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f48d40047c0 con 0x55c298abb250 2023-07-14T16:32:40.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.328+0000 7f48f64eab80 1 -- 172.21.15.62:0/2136205971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48d400f040 con 0x55c298abb250 2023-07-14T16:32:40.301 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.328+0000 7f48bbfff700 1 -- 172.21.15.62:0/2136205971 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d4004980 con 0x55c298abb250 2023-07-14T16:32:40.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.329+0000 7f48f64eab80 10 client.29698 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:40.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.329+0000 7f48f64eab80 1 -- 172.21.15.62:0/2136205971 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c2988142b0 con 0x55c298abb250 2023-07-14T16:32:40.302 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.329+0000 7f48bbfff700 1 -- 172.21.15.62:0/2136205971 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48d400f040 con 0x55c298abb250 2023-07-14T16:32:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.329+0000 7f48bbfff700 1 -- 172.21.15.62:0/2136205971 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48d4038030 con 0x55c298abb250 2023-07-14T16:32:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.330+0000 7f48f64eab80 10 client.29698 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.330+0000 7f48f64eab80 10 client.29698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.330+0000 7f48f64eab80 20 client.29698 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.330+0000 7f48f64eab80 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c298b015f0 0x55c298b039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.330+0000 7f48f64eab80 4 client.29698 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:32:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.330+0000 7f48f64eab80 1 -- 172.21.15.62:0/2136205971 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2989a3050 con 0x55c298b015f0 2023-07-14T16:32:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.330+0000 7f48e099b700 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c298b015f0 0x55c298b039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.331+0000 7f48e099b700 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c298b015f0 0x55c298b039d0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a44273397dbfc5 server_cookie=c756268b34a3a223 in_seq=0 out_seq=0 2023-07-14T16:32:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.331+0000 7f48bbfff700 10 client.29698 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:40.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.331+0000 7f48bbfff700 1 -- 172.21.15.62:0/2136205971 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c2989a3050 con 0x55c298b015f0 2023-07-14T16:32:40.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.331+0000 7f48bbfff700 10 client.29698 handle_command_reply: tid=0 2023-07-14T16:32:40.305 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.331+0000 7f48f64eab80 1 client.29698 shutdown 2023-07-14T16:32:40.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.332+0000 7f48f64eab80 1 -- 172.21.15.62:0/2136205971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298abb250 msgr2=0x55c298abb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.332+0000 7f48f64eab80 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298abb250 0x55c298abb6c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009790 tx=0x7f48d4003ee0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.306 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48f64eab80 1 -- 172.21.15.62:0/2136205971 shutdown_connections 2023-07-14T16:32:40.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48f64eab80 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c298abb250 0x55c298abb6c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48f64eab80 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c298a97c70 0x55c298aba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48f64eab80 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55c298b015f0 0x55c298b039d0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48f64eab80 1 --2- 172.21.15.62:0/2136205971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c298aa5740 0x55c298aba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48f64eab80 1 -- 172.21.15.62:0/2136205971 >> 172.21.15.62:0/2136205971 conn(0x55c298a8bda0 msgr2=0x55c298a9e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:40.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48bbfff700 0 client.29698 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:40.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48f64eab80 1 -- 172.21.15.62:0/2136205971 shutdown_connections 2023-07-14T16:32:40.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.333+0000 7f48f64eab80 1 -- 172.21.15.62:0/2136205971 wait complete. 2023-07-14T16:32:40.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.336+0000 7f48f64eab80 20 client.29698 trim_cache size 0 max 16384 2023-07-14T16:32:40.309 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:32:40.309 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:32:40.309 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:32:40.309 INFO:teuthology.orchestra.run.smithi062.stdout: "351df358-e789-4b1b-848f-5434d9c4820e": { 2023-07-14T16:32:40.310 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:32:40.310 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "351df358-e789-4b1b-848f-5434d9c4820e", 2023-07-14T16:32:40.310 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:32:40.310 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:32:40.310 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:32:40.310 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:32:40.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 -- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48cc0a1250 msgr2=0x7f48cc0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48cc0a1250 0x7f48cc0a3700 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55c298a97210 tx=0x7f48c8009310 comp rx=0 tx=0).stop 2023-07-14T16:32:40.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 -- 172.21.15.62:0/3106951283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 msgr2=0x7f48dc1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:40.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 0x7f48dc1140d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f48d0000c00 tx=0x7f48d000b790 comp rx=0 tx=0).stop 2023-07-14T16:32:40.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 -- 172.21.15.62:0/3106951283 shutdown_connections 2023-07-14T16:32:40.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48dc10bd60 0x7f48dc1b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48dc070ee0 0x7f48dc113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f48cc0a1250 0x7f48cc0a3700 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 --2- 172.21.15.62:0/3106951283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48dc071830 0x7f48dc1140d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:40.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.337+0000 7f48e2bff700 1 -- 172.21.15.62:0/3106951283 >> 172.21.15.62:0/3106951283 conn(0x7f48dc06ca90 msgr2=0x7f48dc06de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:40.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.338+0000 7f48e2bff700 1 -- 172.21.15.62:0/3106951283 shutdown_connections 2023-07-14T16:32:40.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:32:40.338+0000 7f48e2bff700 1 -- 172.21.15.62:0/3106951283 wait complete. 2023-07-14T16:32:40.342 INFO:tasks.cephfs.filesystem:scrub status for tag:351df358-e789-4b1b-848f-5434d9c4820e - {'path': '/', 'tag': '351df358-e789-4b1b-848f-5434d9c4820e', 'options': 'recursive,force'} 2023-07-14T16:32:41.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: pgmap v868: 97 pgs: 97 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 4.95k op/s 2023-07-14T16:32:41.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:32:41.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:32:41.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:32:41.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:32:41.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:32:41.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:32:41.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:40 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:32:41.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: pgmap v868: 97 pgs: 97 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 4.95k op/s 2023-07-14T16:32:41.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:32:41.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:32:41.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:32:41.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:32:41.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:32:41.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:32:41.151 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:40 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:32:41.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: pgmap v868: 97 pgs: 97 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 4.95k op/s 2023-07-14T16:32:41.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:32:41.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:32:41.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:32:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:32:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:32:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:32:41.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:32:41.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:32:41.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:40 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:32:42.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:41 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi062 to 220.9M 2023-07-14T16:32:42.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:41 smithi062 ceph-mon[95543]: Unable to set osd_memory_target on smithi062 to 231684812: error parsing value: Value '231684812' is below minimum 939524096 2023-07-14T16:32:42.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:41 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi062 to 220.9M 2023-07-14T16:32:42.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:41 smithi081 ceph-mon[127795]: Unable to set osd_memory_target on smithi062 to 231684812: error parsing value: Value '231684812' is below minimum 939524096 2023-07-14T16:32:42.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:41 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi062 to 220.9M 2023-07-14T16:32:42.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:41 smithi164 ceph-mon[97427]: Unable to set osd_memory_target on smithi062 to 231684812: error parsing value: Value '231684812' is below minimum 939524096 2023-07-14T16:32:43.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:42 smithi062 ceph-mon[95543]: pgmap v869: 97 pgs: 97 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:32:43.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:42 smithi081 ceph-mon[127795]: pgmap v869: 97 pgs: 97 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:32:43.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:42 smithi164 ceph-mon[97427]: pgmap v869: 97 pgs: 97 active+clean; 13 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:32:45.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:44 smithi081 ceph-mon[127795]: pgmap v870: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2023-07-14T16:32:45.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:44 smithi164 ceph-mon[97427]: pgmap v870: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2023-07-14T16:32:45.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:44 smithi062 ceph-mon[95543]: pgmap v870: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2023-07-14T16:32:47.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:46 smithi081 ceph-mon[127795]: pgmap v871: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:32:47.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:46 smithi164 ceph-mon[97427]: pgmap v871: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:32:47.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:46 smithi062 ceph-mon[95543]: pgmap v871: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:32:49.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:48 smithi081 ceph-mon[127795]: pgmap v872: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 19 MiB/s wr, 4.86k op/s 2023-07-14T16:32:49.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:48 smithi164 ceph-mon[97427]: pgmap v872: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 19 MiB/s wr, 4.86k op/s 2023-07-14T16:32:49.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:48 smithi062 ceph-mon[95543]: pgmap v872: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 19 MiB/s wr, 4.86k op/s 2023-07-14T16:32:50.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:49 smithi081 ceph-mon[127795]: pgmap v873: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:32:50.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:49 smithi164 ceph-mon[97427]: pgmap v873: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:32:50.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:49 smithi062 ceph-mon[95543]: pgmap v873: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:32:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.041+0000 7fb9037ff700 1 -- 172.21.15.62:0/3302616488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8fc100d70 msgr2=0x7fb8fc1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.041+0000 7fb9037ff700 1 --2- 172.21.15.62:0/3302616488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8fc100d70 0x7fb8fc1031f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009990 tx=0x7fb8ec009ca0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.041+0000 7fb9037ff700 1 -- 172.21.15.62:0/3302616488 shutdown_connections 2023-07-14T16:32:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.041+0000 7fb9037ff700 1 --2- 172.21.15.62:0/3302616488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8fc103730 0x7fb8fc103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.041+0000 7fb9037ff700 1 --2- 172.21.15.62:0/3302616488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8fc0fe410 0x7fb8fc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.041+0000 7fb9037ff700 1 --2- 172.21.15.62:0/3302616488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8fc100d70 0x7fb8fc1031f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.042+0000 7fb9037ff700 1 -- 172.21.15.62:0/3302616488 >> 172.21.15.62:0/3302616488 conn(0x7fb8fc0fa000 msgr2=0x7fb8fc0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.042+0000 7fb9037ff700 1 -- 172.21.15.62:0/3302616488 shutdown_connections 2023-07-14T16:32:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.042+0000 7fb9037ff700 1 -- 172.21.15.62:0/3302616488 wait complete. 2023-07-14T16:32:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.043+0000 7fb9037ff700 1 Processor -- start 2023-07-14T16:32:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.044+0000 7fb9037ff700 1 -- start start 2023-07-14T16:32:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.044+0000 7fb9037ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8fc0fe410 0x7fb8fc19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.044+0000 7fb9037ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8fc100d70 0x7fb8fc19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb9037ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8fc103730 0x7fb8fc19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb9037ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc19d8e0 con 0x7fb8fc100d70 2023-07-14T16:32:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb9037ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc19da50 con 0x7fb8fc0fe410 2023-07-14T16:32:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb9037ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc19dc10 con 0x7fb8fc103730 2023-07-14T16:32:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb901d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8fc103730 0x7fb8fc19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb901d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8fc103730 0x7fb8fc19d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:36706/0 (socket says 172.21.15.62:36706) 2023-07-14T16:32:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb901d9c700 1 -- 172.21.15.62:0/456623393 learned_addr learned my addr 172.21.15.62:0/456623393 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb90159b700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8fc0fe410 0x7fb8fc19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.045+0000 7fb900d9a700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8fc100d70 0x7fb8fc19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.046+0000 7fb900d9a700 1 -- 172.21.15.62:0/456623393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8fc103730 msgr2=0x7fb8fc19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.046+0000 7fb900d9a700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8fc103730 0x7fb8fc19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.046+0000 7fb900d9a700 1 -- 172.21.15.62:0/456623393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8fc0fe410 msgr2=0x7fb8fc19bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.046+0000 7fb900d9a700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8fc0fe410 0x7fb8fc19bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.046+0000 7fb900d9a700 1 -- 172.21.15.62:0/456623393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0095f0 con 0x7fb8fc100d70 2023-07-14T16:32:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.047+0000 7fb900d9a700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8fc100d70 0x7fb8fc19c430 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0098e0 tx=0x7fb8ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.047+0000 7fb8f27fc700 1 -- 172.21.15.62:0/456623393 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec01d070 con 0x7fb8fc100d70 2023-07-14T16:32:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.047+0000 7fb8f27fc700 1 -- 172.21.15.62:0/456623393 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8ec00f460 con 0x7fb8fc100d70 2023-07-14T16:32:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.047+0000 7fb8f27fc700 1 -- 172.21.15.62:0/456623393 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec021670 con 0x7fb8fc100d70 2023-07-14T16:32:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.049+0000 7fb9037ff700 1 -- 172.21.15.62:0/456623393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc19de90 con 0x7fb8fc100d70 2023-07-14T16:32:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.049+0000 7fb9037ff700 1 -- 172.21.15.62:0/456623393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc1a4600 con 0x7fb8fc100d70 2023-07-14T16:32:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.056+0000 7fb917042b80 1 Processor -- start 2023-07-14T16:32:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.056+0000 7fb8f27fc700 1 -- 172.21.15.62:0/456623393 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb8ec00f860 con 0x7fb8fc100d70 2023-07-14T16:32:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.056+0000 7fb917042b80 1 -- start start 2023-07-14T16:32:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.056+0000 7fb917042b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf45996b0 0x55caf473d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.056+0000 7fb917042b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473d840 0x55caf473e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.056+0000 7fb917042b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf473eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.056+0000 7fb917042b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf4642c50 con 0x55caf473d840 2023-07-14T16:32:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.057+0000 7fb917042b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf46959e0 con 0x55caf45996b0 2023-07-14T16:32:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.057+0000 7fb917042b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf468ee70 con 0x55caf473e6e0 2023-07-14T16:32:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.058+0000 7fb90159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473d840 0x55caf473e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.058+0000 7fb90159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473d840 0x55caf473e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59232/0 (socket says 172.21.15.62:59232) 2023-07-14T16:32:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.058+0000 7fb90159b700 1 -- 172.21.15.62:0/3185258936 learned_addr learned my addr 172.21.15.62:0/3185258936 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.059+0000 7fb90159b700 1 -- 172.21.15.62:0/3185258936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 msgr2=0x55caf473eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.059+0000 7fb901d9c700 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf45996b0 0x55caf473d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.059+0000 7fb900d9a700 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf473eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.059+0000 7fb90159b700 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf473eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.059+0000 7fb90159b700 1 -- 172.21.15.62:0/3185258936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf45996b0 msgr2=0x55caf473d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.059+0000 7fb90159b700 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf45996b0 0x55caf473d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.059+0000 7fb90159b700 1 -- 172.21.15.62:0/3185258936 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caf44bcd80 con 0x55caf473d840 2023-07-14T16:32:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.060+0000 7fb901d9c700 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf45996b0 0x55caf473d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.060+0000 7fb900d9a700 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf473eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:32:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.060+0000 7fb8f27fc700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8e80a12a0 0x7fb8e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.060+0000 7fb8f27fc700 1 -- 172.21.15.62:0/456623393 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8ec0cd780 con 0x7fb8fc100d70 2023-07-14T16:32:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.061+0000 7fb901d9c700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8e80a12a0 0x7fb8e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.061+0000 7fb90159b700 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473d840 0x55caf473e1a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800bd10 tx=0x7fb8f80106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15fd82431529fb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.061+0000 7fb8e77fe700 1 -- 172.21.15.62:0/3185258936 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8010ec0 con 0x55caf473d840 2023-07-14T16:32:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.061+0000 7fb8e77fe700 1 -- 172.21.15.62:0/3185258936 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8f8015070 con 0x55caf473d840 2023-07-14T16:32:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.061+0000 7fb8e77fe700 1 -- 172.21.15.62:0/3185258936 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8003db0 con 0x55caf473d840 2023-07-14T16:32:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.062+0000 7fb901d9c700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8e80a12a0 0x7fb8e80a3750 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009fd0 tx=0x7fb8f400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.062+0000 7fb917042b80 1 -- 172.21.15.62:0/3185258936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473d840 msgr2=0x55caf473e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.062+0000 7fb917042b80 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473d840 0x55caf473e1a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800bd10 tx=0x7fb8f80106b0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.065+0000 7fb917042b80 1 -- 172.21.15.62:0/3185258936 shutdown_connections 2023-07-14T16:32:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.065+0000 7fb917042b80 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf473eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.065+0000 7fb917042b80 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf45996b0 0x55caf473d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.065+0000 7fb917042b80 1 --2- 172.21.15.62:0/3185258936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473d840 0x55caf473e1a0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.065+0000 7fb917042b80 1 -- 172.21.15.62:0/3185258936 >> 172.21.15.62:0/3185258936 conn(0x55caf4732be0 msgr2=0x55caf4737cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.066+0000 7fb917042b80 1 -- 172.21.15.62:0/3185258936 shutdown_connections 2023-07-14T16:32:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.066+0000 7fb917042b80 1 -- 172.21.15.62:0/3185258936 wait complete. 2023-07-14T16:32:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.067+0000 7fb917042b80 1 -- start start 2023-07-14T16:32:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.068+0000 7fb917042b80 1 Processor -- start 2023-07-14T16:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.069+0000 7fb917042b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf45996b0 0x55caf4757850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.069+0000 7fb917042b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf4757d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.071+0000 7fb917042b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf475edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.071+0000 7fb917042b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf462ea60 con 0x55caf45996b0 2023-07-14T16:32:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.071+0000 7fb917042b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf462e760 con 0x55caf473d840 2023-07-14T16:32:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.071+0000 7fb917042b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf45128c0 con 0x55caf473e6e0 2023-07-14T16:32:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.071+0000 7fb917042b80 10 client.0 _ll_register_callbacks cb 0x55caf462c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.072+0000 7fb90159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf45996b0 0x55caf4757850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.072+0000 7fb90159b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf45996b0 0x55caf4757850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59234/0 (socket says 172.21.15.62:59234) 2023-07-14T16:32:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.072+0000 7fb90159b700 1 -- 172.21.15.62:0/2217092645 learned_addr learned my addr 172.21.15.62:0/2217092645 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.072+0000 7fb900d9a700 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf4757d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.072+0000 7fb901d9c700 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf475edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.073+0000 7fb901d9c700 1 -- 172.21.15.62:0/2217092645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 msgr2=0x55caf4757d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.073+0000 7fb901d9c700 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf4757d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.073+0000 7fb901d9c700 1 -- 172.21.15.62:0/2217092645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf45996b0 msgr2=0x55caf4757850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.073+0000 7fb901d9c700 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf45996b0 0x55caf4757850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.073+0000 7fb901d9c700 1 -- 172.21.15.62:0/2217092645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f800b940 con 0x55caf473e6e0 2023-07-14T16:32:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.073+0000 7fb901d9c700 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf475edd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009c10 tx=0x7fb8f4004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9215502e223318af server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.073+0000 7fb8e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:32:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.074+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/2217092645 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4027070 con 0x55caf473e6e0 2023-07-14T16:32:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.074+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/2217092645 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8f4007e40 con 0x55caf473e6e0 2023-07-14T16:32:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.074+0000 7fb917042b80 1 -- 172.21.15.62:0/2217092645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55caf45e8490 con 0x55caf473e6e0 2023-07-14T16:32:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.075+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/2217092645 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8f400e040 con 0x55caf473e6e0 2023-07-14T16:32:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.075+0000 7fb917042b80 10 client.29707 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.075+0000 7fb917042b80 1 -- 172.21.15.62:0/2217092645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55caf44a5c00 con 0x55caf473e6e0 2023-07-14T16:32:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.076+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/2217092645 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55caf45e8490 con 0x55caf473e6e0 2023-07-14T16:32:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.076+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/2217092645 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb8f402a020 con 0x55caf473e6e0 2023-07-14T16:32:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.077+0000 7fb917042b80 10 client.29707 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.077+0000 7fb917042b80 10 client.29707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.078+0000 7fb917042b80 20 client.29707 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.078+0000 7fb917042b80 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55caf47635e0 0x55caf47659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.078+0000 7fb917042b80 4 client.29707 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:32:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.078+0000 7fb917042b80 1 -- 172.21.15.62:0/2217092645 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caf4687e50 con 0x55caf47635e0 2023-07-14T16:32:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.079+0000 7fb90159b700 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55caf47635e0 0x55caf47659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.083+0000 7fb90159b700 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55caf47635e0 0x55caf47659f0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d46406729455d2a server_cookie=17e1755f463c265 in_seq=0 out_seq=0 2023-07-14T16:32:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.083+0000 7fb8e4ff9700 10 client.29707 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/2217092645 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55caf4687e50 con 0x55caf47635e0 2023-07-14T16:32:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb8e4ff9700 10 client.29707 handle_command_reply: tid=0 2023-07-14T16:32:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 client.29707 shutdown 2023-07-14T16:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 -- 172.21.15.62:0/2217092645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 msgr2=0x55caf475edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf475edd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009c10 tx=0x7fb8f4004d50 comp rx=0 tx=0).stop 2023-07-14T16:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 -- 172.21.15.62:0/2217092645 shutdown_connections 2023-07-14T16:32:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf473e6e0 0x55caf475edd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf45996b0 0x55caf4757850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55caf47635e0 0x55caf47659f0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 --2- 172.21.15.62:0/2217092645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf4757d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.085+0000 7fb917042b80 1 -- 172.21.15.62:0/2217092645 >> 172.21.15.62:0/2217092645 conn(0x55caf4732be0 msgr2=0x55caf47374e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.086+0000 7fb8e4ff9700 0 client.29707 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.086+0000 7fb917042b80 1 -- 172.21.15.62:0/2217092645 shutdown_connections 2023-07-14T16:32:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.086+0000 7fb917042b80 1 -- 172.21.15.62:0/2217092645 wait complete. 2023-07-14T16:32:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.086+0000 7fb917042b80 20 client.29707 trim_cache size 0 max 16384 2023-07-14T16:32:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.126+0000 7fb917042b80 1 Processor -- start 2023-07-14T16:32:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.126+0000 7fb917042b80 1 -- start start 2023-07-14T16:32:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.127+0000 7fb917042b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 0x55caf4741550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.127+0000 7fb917042b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf475ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.127+0000 7fb917042b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf4741a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.127+0000 7fb917042b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf4687e50 con 0x55caf473e6e0 2023-07-14T16:32:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.127+0000 7fb917042b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf45e8490 con 0x55caf473d840 2023-07-14T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.127+0000 7fb917042b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf4703060 con 0x55caf45996b0 2023-07-14T16:32:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.128+0000 7fb900d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf475ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.128+0000 7fb901d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf4741a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.128+0000 7fb900d9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf475ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:38886/0 (socket says 172.21.15.62:38886) 2023-07-14T16:32:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.128+0000 7fb900d9a700 1 -- 172.21.15.62:0/2653896454 learned_addr learned my addr 172.21.15.62:0/2653896454 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.128+0000 7fb90159b700 1 --2- 172.21.15.62:0/2653896454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 0x55caf4741550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.129+0000 7fb901d9c700 1 -- 172.21.15.62:0/2653896454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 msgr2=0x55caf4741550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.129+0000 7fb901d9c700 1 --2- 172.21.15.62:0/2653896454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 0x55caf4741550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.129+0000 7fb901d9c700 1 -- 172.21.15.62:0/2653896454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 msgr2=0x55caf475ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:32:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.129+0000 7fb901d9c700 1 --2- 172.21.15.62:0/2653896454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf475ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.129+0000 7fb901d9c700 1 -- 172.21.15.62:0/2653896454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caf44e31f0 con 0x55caf473e6e0 2023-07-14T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.129+0000 7fb901d9c700 1 --2- 172.21.15.62:0/2653896454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf4741a90 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4023c60 tx=0x7fb8f4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716a86044b60c762 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.129+0000 7fb8e7fff700 1 -- 172.21.15.62:0/2653896454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400e070 con 0x55caf473e6e0 2023-07-14T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.129+0000 7fb8e7fff700 1 -- 172.21.15.62:0/2653896454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8f401c070 con 0x55caf473e6e0 2023-07-14T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.130+0000 7fb8e7fff700 1 -- 172.21.15.62:0/2653896454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4027070 con 0x55caf473e6e0 2023-07-14T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.130+0000 7fb917042b80 1 -- 172.21.15.62:0/2653896454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 msgr2=0x55caf4741a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.130+0000 7fb917042b80 1 --2- 172.21.15.62:0/2653896454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf4741a90 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4023c60 tx=0x7fb8f4009e50 comp rx=0 tx=0).stop 2023-07-14T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.131+0000 7fb917042b80 1 -- 172.21.15.62:0/2653896454 shutdown_connections 2023-07-14T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.131+0000 7fb917042b80 1 --2- 172.21.15.62:0/2653896454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf4741a90 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.131+0000 7fb917042b80 1 --2- 172.21.15.62:0/2653896454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 0x55caf4741550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.131+0000 7fb917042b80 1 --2- 172.21.15.62:0/2653896454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf475ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.131+0000 7fb917042b80 1 -- 172.21.15.62:0/2653896454 >> 172.21.15.62:0/2653896454 conn(0x55caf4732be0 msgr2=0x55caf4744a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.131+0000 7fb917042b80 1 -- 172.21.15.62:0/2653896454 shutdown_connections 2023-07-14T16:32:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.131+0000 7fb917042b80 1 -- 172.21.15.62:0/2653896454 wait complete. 2023-07-14T16:32:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.131+0000 7fb917042b80 1 -- start start 2023-07-14T16:32:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.132+0000 7fb917042b80 1 Processor -- start 2023-07-14T16:32:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.132+0000 7fb917042b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 0x55caf475a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.132+0000 7fb917042b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf475aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.132+0000 7fb917042b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf475def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.132+0000 7fb917042b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf475e430 con 0x55caf473e6e0 2023-07-14T16:32:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.132+0000 7fb917042b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf475e5a0 con 0x55caf473d840 2023-07-14T16:32:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.132+0000 7fb917042b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf475e760 con 0x55caf45996b0 2023-07-14T16:32:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.132+0000 7fb917042b80 10 client.0 _ll_register_callbacks cb 0x7fb8f4018810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:32:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.136+0000 7fb901d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf475def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.136+0000 7fb901d9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf475def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59250/0 (socket says 172.21.15.62:59250) 2023-07-14T16:32:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.136+0000 7fb901d9c700 1 -- 172.21.15.62:0/1997396471 learned_addr learned my addr 172.21.15.62:0/1997396471 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:32:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.137+0000 7fb901d9c700 1 -- 172.21.15.62:0/1997396471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 msgr2=0x55caf475a990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.137+0000 7fb901d9c700 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 0x55caf475a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.137+0000 7fb901d9c700 1 -- 172.21.15.62:0/1997396471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 msgr2=0x55caf475aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.137+0000 7fb901d9c700 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf475aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.137+0000 7fb901d9c700 1 -- 172.21.15.62:0/1997396471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caf44a5c00 con 0x55caf473e6e0 2023-07-14T16:32:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.144+0000 7fb901d9c700 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf475def0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b010 tx=0x7fb8f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a8bd6af3b5dc4a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.145+0000 7fb8e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.145+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/1997396471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4027030 con 0x55caf473e6e0 2023-07-14T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.145+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/1997396471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8f4028070 con 0x55caf473e6e0 2023-07-14T16:32:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.146+0000 7fb917042b80 1 -- 172.21.15.62:0/1997396471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8f8015070 con 0x55caf473e6e0 2023-07-14T16:32:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.147+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/1997396471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8f400e040 con 0x55caf473e6e0 2023-07-14T16:32:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.147+0000 7fb917042b80 10 client.20289 fetch_fsmap learned FSMap version 31 2023-07-14T16:32:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.147+0000 7fb917042b80 1 -- 172.21.15.62:0/1997396471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55caf44bb250 con 0x55caf473e6e0 2023-07-14T16:32:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.149+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/1997396471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8f8015070 con 0x55caf473e6e0 2023-07-14T16:32:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.149+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/1997396471 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb8f4045070 con 0x55caf473e6e0 2023-07-14T16:32:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.149+0000 7fb917042b80 10 client.20289 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:32:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.149+0000 7fb917042b80 10 client.20289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:32:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.149+0000 7fb917042b80 20 client.20289 populate_metadata read hostname 'smithi062' 2023-07-14T16:32:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.149+0000 7fb917042b80 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55caf47b7f40 0x55caf47ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:32:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.149+0000 7fb917042b80 4 client.20289 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:32:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.149+0000 7fb917042b80 1 -- 172.21.15.62:0/1997396471 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55caf4649ff0 con 0x55caf47b7f40 2023-07-14T16:32:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.150+0000 7fb90159b700 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55caf47b7f40 0x55caf47ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:32:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.151+0000 7fb90159b700 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55caf47b7f40 0x55caf47ba350 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a6aa4ebf5518c1 server_cookie=ab2a45aa9be11f8c in_seq=0 out_seq=0 2023-07-14T16:32:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.152+0000 7fb8e4ff9700 10 client.20289 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.170+0000 7fb8e4ff9700 1 -- 172.21.15.62:0/1997396471 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55caf4649ff0 con 0x55caf47b7f40 2023-07-14T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.170+0000 7fb8e4ff9700 10 client.20289 handle_command_reply: tid=0 2023-07-14T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 client.20289 shutdown 2023-07-14T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 -- 172.21.15.62:0/1997396471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 msgr2=0x55caf475def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf475def0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b010 tx=0x7fb8f4009920 comp rx=0 tx=0).stop 2023-07-14T16:32:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 -- 172.21.15.62:0/1997396471 shutdown_connections 2023-07-14T16:32:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55caf473e6e0 0x55caf475def0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55caf45996b0 0x55caf475a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55caf47b7f40 0x55caf47ba350 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 --2- 172.21.15.62:0/1997396471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55caf473d840 0x55caf475aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 -- 172.21.15.62:0/1997396471 >> 172.21.15.62:0/1997396471 conn(0x55caf4732be0 msgr2=0x55caf4741550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb8e4ff9700 0 client.20289 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:32:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 -- 172.21.15.62:0/1997396471 shutdown_connections 2023-07-14T16:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.171+0000 7fb917042b80 1 -- 172.21.15.62:0/1997396471 wait complete. 2023-07-14T16:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.172+0000 7fb917042b80 20 client.20289 trim_cache size 0 max 16384 2023-07-14T16:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:32:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.172+0000 7fb9037ff700 1 -- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8e80a12a0 msgr2=0x7fb8e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.172+0000 7fb9037ff700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8e80a12a0 0x7fb8e80a3750 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009fd0 tx=0x7fb8f400b040 comp rx=0 tx=0).stop 2023-07-14T16:32:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.172+0000 7fb9037ff700 1 -- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8fc100d70 msgr2=0x7fb8fc19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:32:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.172+0000 7fb9037ff700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8fc100d70 0x7fb8fc19c430 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0098e0 tx=0x7fb8ec005740 comp rx=0 tx=0).stop 2023-07-14T16:32:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.173+0000 7fb9037ff700 1 -- 172.21.15.62:0/456623393 shutdown_connections 2023-07-14T16:32:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.173+0000 7fb9037ff700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8fc103730 0x7fb8fc19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.173+0000 7fb9037ff700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8e80a12a0 0x7fb8e80a3750 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.173+0000 7fb9037ff700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8fc0fe410 0x7fb8fc19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.173+0000 7fb9037ff700 1 --2- 172.21.15.62:0/456623393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8fc100d70 0x7fb8fc19c430 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:32:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.173+0000 7fb9037ff700 1 -- 172.21.15.62:0/456623393 >> 172.21.15.62:0/456623393 conn(0x7fb8fc0fa000 msgr2=0x7fb8fc0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:32:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.173+0000 7fb9037ff700 1 -- 172.21.15.62:0/456623393 shutdown_connections 2023-07-14T16:32:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:32:52.173+0000 7fb9037ff700 1 -- 172.21.15.62:0/456623393 wait complete. 2023-07-14T16:32:53.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:52 smithi062 ceph-mon[95543]: pgmap v874: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 4.52k op/s 2023-07-14T16:32:53.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:52 smithi081 ceph-mon[127795]: pgmap v874: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 4.52k op/s 2023-07-14T16:32:53.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:52 smithi164 ceph-mon[97427]: pgmap v874: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 4.52k op/s 2023-07-14T16:32:54.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:53 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:32:54.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:53 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:32:54.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:53 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:32:54.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:53 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:32:54.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:53 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:32:54.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:53 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:32:54.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:53 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:32:54.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:53 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:32:54.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:53 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:32:55.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:54 smithi062 ceph-mon[95543]: pgmap v875: 97 pgs: 97 active+clean; 13 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2023-07-14T16:32:55.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:54 smithi081 ceph-mon[127795]: pgmap v875: 97 pgs: 97 active+clean; 13 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2023-07-14T16:32:55.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:54 smithi164 ceph-mon[97427]: pgmap v875: 97 pgs: 97 active+clean; 13 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 5.29k op/s 2023-07-14T16:32:57.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:56 smithi062 ceph-mon[95543]: pgmap v876: 97 pgs: 97 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 4.52k op/s 2023-07-14T16:32:57.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:56 smithi081 ceph-mon[127795]: pgmap v876: 97 pgs: 97 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 4.52k op/s 2023-07-14T16:32:57.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:56 smithi164 ceph-mon[97427]: pgmap v876: 97 pgs: 97 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 4.52k op/s 2023-07-14T16:32:59.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:32:58 smithi062 ceph-mon[95543]: pgmap v877: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 5.10k op/s 2023-07-14T16:32:59.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:32:58 smithi081 ceph-mon[127795]: pgmap v877: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 5.10k op/s 2023-07-14T16:32:59.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:32:58 smithi164 ceph-mon[97427]: pgmap v877: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 MiB/s wr, 5.10k op/s 2023-07-14T16:33:01.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:00 smithi062 ceph-mon[95543]: pgmap v878: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2023-07-14T16:33:01.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:00 smithi081 ceph-mon[127795]: pgmap v878: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2023-07-14T16:33:01.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:00 smithi164 ceph-mon[97427]: pgmap v878: 97 pgs: 97 active+clean; 13 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 5.00k op/s 2023-07-14T16:33:03.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:02 smithi062 ceph-mon[95543]: pgmap v879: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:33:03.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:02 smithi081 ceph-mon[127795]: pgmap v879: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:33:03.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:02 smithi164 ceph-mon[97427]: pgmap v879: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:33:05.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:04 smithi062 ceph-mon[95543]: pgmap v880: 97 pgs: 97 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2023-07-14T16:33:05.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:04 smithi081 ceph-mon[127795]: pgmap v880: 97 pgs: 97 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2023-07-14T16:33:05.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:04 smithi164 ceph-mon[97427]: pgmap v880: 97 pgs: 97 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2023-07-14T16:33:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 -- 172.21.15.62:0/2086052300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f680fe1c0 msgr2=0x7f0f681051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 --2- 172.21.15.62:0/2086052300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f680fe1c0 0x7f0f681051d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009990 tx=0x7f0f58009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 -- 172.21.15.62:0/2086052300 shutdown_connections 2023-07-14T16:33:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 --2- 172.21.15.62:0/2086052300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f68105760 0x7f0f68107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 --2- 172.21.15.62:0/2086052300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f680fd870 0x7f0f680fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 --2- 172.21.15.62:0/2086052300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f680fe1c0 0x7f0f681051d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 -- 172.21.15.62:0/2086052300 >> 172.21.15.62:0/2086052300 conn(0x7f0f680f9330 msgr2=0x7f0f680fb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 -- 172.21.15.62:0/2086052300 shutdown_connections 2023-07-14T16:33:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.477+0000 7f0f703ff700 1 -- 172.21.15.62:0/2086052300 wait complete. 2023-07-14T16:33:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.478+0000 7f0f703ff700 1 Processor -- start 2023-07-14T16:33:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.479+0000 7f0f703ff700 1 -- start start 2023-07-14T16:33:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.479+0000 7f0f703ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f680fd870 0x7f0f68193690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.479+0000 7f0f703ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f680fe1c0 0x7f0f68193bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.479+0000 7f0f703ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f68105760 0x7f0f68194a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.479+0000 7f0f703ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f681950b0 con 0x7f0f680fe1c0 2023-07-14T16:33:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.479+0000 7f0f703ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f68195220 con 0x7f0f680fd870 2023-07-14T16:33:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.479+0000 7f0f703ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f681953e0 con 0x7f0f68105760 2023-07-14T16:33:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.480+0000 7f0f6e99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f68105760 0x7f0f68194a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.480+0000 7f0f6e99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f68105760 0x7f0f68194a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46978/0 (socket says 172.21.15.62:46978) 2023-07-14T16:33:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.480+0000 7f0f6e99c700 1 -- 172.21.15.62:0/1505680015 learned_addr learned my addr 172.21.15.62:0/1505680015 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.480+0000 7f0f6e99c700 1 -- 172.21.15.62:0/1505680015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f680fd870 msgr2=0x7f0f68193690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T16:33:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.480+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f680fd870 0x7f0f68193690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.480+0000 7f0f6e99c700 1 -- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f680fe1c0 msgr2=0x7f0f68193bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.480+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f680fe1c0 0x7f0f68193bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.480+0000 7f0f6e99c700 1 -- 172.21.15.62:0/1505680015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580095f0 con 0x7f0f68105760 2023-07-14T16:33:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.481+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f68105760 0x7f0f68194a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000ef30 tx=0x7f0f6000c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.482+0000 7f0f5f7fe700 1 -- 172.21.15.62:0/1505680015 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60018070 con 0x7f0f68105760 2023-07-14T16:33:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.482+0000 7f0f5f7fe700 1 -- 172.21.15.62:0/1505680015 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f6000f040 con 0x7f0f68105760 2023-07-14T16:33:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.482+0000 7f0f5f7fe700 1 -- 172.21.15.62:0/1505680015 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60003d30 con 0x7f0f68105760 2023-07-14T16:33:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.484+0000 7f0f703ff700 1 -- 172.21.15.62:0/1505680015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f68195660 con 0x7f0f68105760 2023-07-14T16:33:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.484+0000 7f0f703ff700 1 -- 172.21.15.62:0/1505680015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6819be80 con 0x7f0f68105760 2023-07-14T16:33:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.496+0000 7f0f83c90b80 1 Processor -- start 2023-07-14T16:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.498+0000 7f0f83c90b80 1 -- start start 2023-07-14T16:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.498+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 0x561daca7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.498+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7b0f0 0x561daca7b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.498+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 0x561daca7c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.498+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dac977c50 con 0x561dac8ce6b0 2023-07-14T16:33:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.498+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dac9ca9e0 con 0x561daca7b0f0 2023-07-14T16:33:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.498+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dac9c3e70 con 0x561daca7beb0 2023-07-14T16:33:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6e19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7b0f0 0x561daca7b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6e19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7b0f0 0x561daca7b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51110/0 (socket says 172.21.15.62:51110) 2023-07-14T16:33:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6e19b700 1 -- 172.21.15.62:0/345701879 learned_addr learned my addr 172.21.15.62:0/345701879 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6d99a700 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 0x561daca7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6e19b700 1 -- 172.21.15.62:0/345701879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 msgr2=0x561daca7c320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:33:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 0x561daca7c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6e19b700 1 -- 172.21.15.62:0/345701879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 msgr2=0x561daca7abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 0x561daca7abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f6e19b700 1 -- 172.21.15.62:0/345701879 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dac7f1d80 con 0x561daca7b0f0 2023-07-14T16:33:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.499+0000 7f0f5f7fe700 1 -- 172.21.15.62:0/1505680015 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0f60004ad0 con 0x7f0f68105760 2023-07-14T16:33:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.500+0000 7f0f6d99a700 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 0x561daca7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:33:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.500+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7b0f0 0x561daca7b970 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f64007b50 tx=0x7f0f64007e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b8666acbb7be8f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.500+0000 7f0f5cbfe700 1 -- 172.21.15.62:0/345701879 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64004020 con 0x561daca7b0f0 2023-07-14T16:33:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.501+0000 7f0f5cbfe700 1 -- 172.21.15.62:0/345701879 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f64004830 con 0x561daca7b0f0 2023-07-14T16:33:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.501+0000 7f0f5cbfe700 1 -- 172.21.15.62:0/345701879 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400dc90 con 0x561daca7b0f0 2023-07-14T16:33:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.502+0000 7f0f83c90b80 1 -- 172.21.15.62:0/345701879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7b0f0 msgr2=0x561daca7b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.502+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7b0f0 0x561daca7b970 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f64007b50 tx=0x7f0f64007e60 comp rx=0 tx=0).stop 2023-07-14T16:33:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.502+0000 7f0f83c90b80 1 -- 172.21.15.62:0/345701879 shutdown_connections 2023-07-14T16:33:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.502+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 0x561daca7c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.502+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 0x561daca7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.502+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/345701879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7b0f0 0x561daca7b970 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.503+0000 7f0f83c90b80 1 -- 172.21.15.62:0/345701879 >> 172.21.15.62:0/345701879 conn(0x561daca67be0 msgr2=0x561daca6ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.503+0000 7f0f5f7fe700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f540a1320 0x7f0f540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.503+0000 7f0f83c90b80 1 -- 172.21.15.62:0/345701879 shutdown_connections 2023-07-14T16:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.503+0000 7f0f5f7fe700 1 -- 172.21.15.62:0/1505680015 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0f600cbd20 con 0x7f0f68105760 2023-07-14T16:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.503+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f540a1320 0x7f0f540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.503+0000 7f0f83c90b80 1 -- 172.21.15.62:0/345701879 wait complete. 2023-07-14T16:33:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.504+0000 7f0f83c90b80 1 -- start start 2023-07-14T16:33:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.505+0000 7f0f83c90b80 1 Processor -- start 2023-07-14T16:33:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.505+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f540a1320 0x7f0f540a37d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400d5c0 tx=0x7f0f64000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.506+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561daca73490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.506+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7b0f0 0x561daca739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.506+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 0x561daca78a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.506+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dac963a60 con 0x561daca7b0f0 2023-07-14T16:33:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.506+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dac963760 con 0x561dac8ce6b0 2023-07-14T16:33:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.506+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dac8478c0 con 0x561daca7beb0 2023-07-14T16:33:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.506+0000 7f0f83c90b80 10 client.0 _ll_register_callbacks cb 0x561dac961700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.507+0000 7f0f6e19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561daca73490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.507+0000 7f0f6e19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561daca73490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51126/0 (socket says 172.21.15.62:51126) 2023-07-14T16:33:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.507+0000 7f0f6e19b700 1 -- 172.21.15.62:0/234441159 learned_addr learned my addr 172.21.15.62:0/234441159 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.507+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 0x561daca78a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.508+0000 7f0f6e99c700 1 -- 172.21.15.62:0/234441159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 msgr2=0x561daca73490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.508+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561daca73490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.508+0000 7f0f6e99c700 1 -- 172.21.15.62:0/234441159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7b0f0 msgr2=0x561daca739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.508+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7b0f0 0x561daca739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.508+0000 7f0f6e99c700 1 -- 172.21.15.62:0/234441159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f64007780 con 0x561daca7beb0 2023-07-14T16:33:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.509+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 0x561daca78a00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0f600c97d0 tx=0x7f0f60099620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f497094dfd6f6792 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.509+0000 7f0f51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.509+0000 7f0f51ffb700 1 -- 172.21.15.62:0/234441159 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f600b0070 con 0x561daca7beb0 2023-07-14T16:33:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.509+0000 7f0f51ffb700 1 -- 172.21.15.62:0/234441159 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0f600c98e0 con 0x561daca7beb0 2023-07-14T16:33:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.510+0000 7f0f83c90b80 1 -- 172.21.15.62:0/234441159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561dac91d490 con 0x561daca7beb0 2023-07-14T16:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.510+0000 7f0f51ffb700 1 -- 172.21.15.62:0/234441159 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f600a10a0 con 0x561daca7beb0 2023-07-14T16:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.510+0000 7f0f83c90b80 10 client.29719 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.510+0000 7f0f83c90b80 1 -- 172.21.15.62:0/234441159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561dac7dac00 con 0x561daca7beb0 2023-07-14T16:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.510+0000 7f0f51ffb700 1 -- 172.21.15.62:0/234441159 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561dac91d490 con 0x561daca7beb0 2023-07-14T16:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.511+0000 7f0f51ffb700 1 -- 172.21.15.62:0/234441159 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f6000caa0 con 0x561daca7beb0 2023-07-14T16:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.511+0000 7f0f83c90b80 10 client.29719 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.511+0000 7f0f83c90b80 10 client.29719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.511+0000 7f0f83c90b80 20 client.29719 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.511+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561daca985a0 0x561daca9aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.511+0000 7f0f83c90b80 4 client.29719 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.511+0000 7f0f83c90b80 1 -- 172.21.15.62:0/234441159 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561dac9bce50 con 0x561daca985a0 2023-07-14T16:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.511+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561daca985a0 0x561daca9aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.512+0000 7f0f51ffb700 10 client.29719 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.512+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561daca985a0 0x561daca9aa10 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af8e4dc6910c39f server_cookie=eede90adf062a374 in_seq=0 out_seq=0 2023-07-14T16:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.513+0000 7f0f51ffb700 1 -- 172.21.15.62:0/234441159 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561dac9bce50 con 0x561daca985a0 2023-07-14T16:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.513+0000 7f0f51ffb700 10 client.29719 handle_command_reply: tid=0 2023-07-14T16:33:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.513+0000 7f0f83c90b80 1 client.29719 shutdown 2023-07-14T16:33:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.514+0000 7f0f83c90b80 1 -- 172.21.15.62:0/234441159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 msgr2=0x561daca78a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.514+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 0x561daca78a00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0f600c97d0 tx=0x7f0f60099620 comp rx=0 tx=0).stop 2023-07-14T16:33:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.515+0000 7f0f83c90b80 1 -- 172.21.15.62:0/234441159 shutdown_connections 2023-07-14T16:33:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.515+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7beb0 0x561daca78a00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.515+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561daca985a0 0x561daca9aa10 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.515+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561daca73490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.515+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/234441159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7b0f0 0x561daca739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.515+0000 7f0f51ffb700 0 client.29719 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.515+0000 7f0f83c90b80 1 -- 172.21.15.62:0/234441159 >> 172.21.15.62:0/234441159 conn(0x561daca67be0 msgr2=0x561daca6c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.515+0000 7f0f83c90b80 1 -- 172.21.15.62:0/234441159 shutdown_connections 2023-07-14T16:33:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.524+0000 7f0f83c90b80 1 -- 172.21.15.62:0/234441159 wait complete. 2023-07-14T16:33:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.525+0000 7f0f83c90b80 20 client.29719 trim_cache size 0 max 16384 2023-07-14T16:33:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.561+0000 7f0f83c90b80 1 Processor -- start 2023-07-14T16:33:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.561+0000 7f0f83c90b80 1 -- start start 2023-07-14T16:33:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 0x561daca722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561daca69150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7beb0 0x561dacaa82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daca74680 con 0x561dac8ce6b0 2023-07-14T16:33:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daca64c00 con 0x561daca7beb0 2023-07-14T16:33:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561daca72940 con 0x561daca7b0f0 2023-07-14T16:33:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f6d99a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561daca69150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f6d99a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561daca69150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47002/0 (socket says 172.21.15.62:47002) 2023-07-14T16:33:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.562+0000 7f0f6d99a700 1 -- 172.21.15.62:0/4184028477 learned_addr learned my addr 172.21.15.62:0/4184028477 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.563+0000 7f0f6d99a700 1 -- 172.21.15.62:0/4184028477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7beb0 msgr2=0x561dacaa82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.563+0000 7f0f6d99a700 1 --2- 172.21.15.62:0/4184028477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7beb0 0x561dacaa82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.563+0000 7f0f6d99a700 1 -- 172.21.15.62:0/4184028477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 msgr2=0x561daca722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.563+0000 7f0f6d99a700 1 --2- 172.21.15.62:0/4184028477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 0x561daca722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.563+0000 7f0f6d99a700 1 -- 172.21.15.62:0/4184028477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dac8181f0 con 0x561daca7b0f0 2023-07-14T16:33:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.563+0000 7f0f6d99a700 1 --2- 172.21.15.62:0/4184028477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561daca69150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f580095c0 tx=0x7f0f58005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4397fe045124293 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.564+0000 7f0f5d3ff700 1 -- 172.21.15.62:0/4184028477 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5801e070 con 0x561daca7b0f0 2023-07-14T16:33:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.564+0000 7f0f5d3ff700 1 -- 172.21.15.62:0/4184028477 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f58005590 con 0x561daca7b0f0 2023-07-14T16:33:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.564+0000 7f0f5d3ff700 1 -- 172.21.15.62:0/4184028477 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800b710 con 0x561daca7b0f0 2023-07-14T16:33:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.565+0000 7f0f83c90b80 1 -- 172.21.15.62:0/4184028477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 msgr2=0x561daca69150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.565+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/4184028477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561daca69150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f580095c0 tx=0x7f0f58005e80 comp rx=0 tx=0).stop 2023-07-14T16:33:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.565+0000 7f0f83c90b80 1 -- 172.21.15.62:0/4184028477 shutdown_connections 2023-07-14T16:33:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.565+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/4184028477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561daca7beb0 0x561dacaa82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.565+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/4184028477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561dac8ce6b0 0x561daca722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.565+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/4184028477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561daca69150 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.565+0000 7f0f83c90b80 1 -- 172.21.15.62:0/4184028477 >> 172.21.15.62:0/4184028477 conn(0x561daca67be0 msgr2=0x561daca9f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.566+0000 7f0f83c90b80 1 -- 172.21.15.62:0/4184028477 shutdown_connections 2023-07-14T16:33:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.566+0000 7f0f83c90b80 1 -- 172.21.15.62:0/4184028477 wait complete. 2023-07-14T16:33:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.566+0000 7f0f83c90b80 1 -- start start 2023-07-14T16:33:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.566+0000 7f0f83c90b80 1 Processor -- start 2023-07-14T16:33:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561dacaa6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561dacaa6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f83c90b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7beb0 0x561dacaa7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dacaa78d0 con 0x561daca7beb0 2023-07-14T16:33:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dacaa7a40 con 0x561dac8ce6b0 2023-07-14T16:33:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f83c90b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dacaa7c00 con 0x561daca7b0f0 2023-07-14T16:33:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f83c90b80 10 client.0 _ll_register_callbacks cb 0x7f0f60098af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f6e19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561dacaa6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f6e19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561dacaa6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51136/0 (socket says 172.21.15.62:51136) 2023-07-14T16:33:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f6d99a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561dacaa6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.567+0000 7f0f6e19b700 1 -- 172.21.15.62:0/2368028801 learned_addr learned my addr 172.21.15.62:0/2368028801 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7beb0 0x561dacaa7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f6e19b700 1 -- 172.21.15.62:0/2368028801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 msgr2=0x561dacaa6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561dacaa6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f6e19b700 1 -- 172.21.15.62:0/2368028801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7beb0 msgr2=0x561dacaa7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7beb0 0x561dacaa7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f6e19b700 1 -- 172.21.15.62:0/2368028801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f68108830 con 0x561dac8ce6b0 2023-07-14T16:33:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f6e99c700 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7beb0 0x561dacaa7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:33:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f6e19b700 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561dacaa6510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f64005fd0 tx=0x7f0f64005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf530e9cce8a271 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.568+0000 7f0f51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.569+0000 7f0f51ffb700 1 -- 172.21.15.62:0/2368028801 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6402b040 con 0x561dac8ce6b0 2023-07-14T16:33:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.569+0000 7f0f51ffb700 1 -- 172.21.15.62:0/2368028801 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0f640323c0 con 0x561dac8ce6b0 2023-07-14T16:33:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.569+0000 7f0f83c90b80 1 -- 172.21.15.62:0/2368028801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f64004830 con 0x561dac8ce6b0 2023-07-14T16:33:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.569+0000 7f0f51ffb700 1 -- 172.21.15.62:0/2368028801 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f6401f070 con 0x561dac8ce6b0 2023-07-14T16:33:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.569+0000 7f0f83c90b80 10 client.29843 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.569+0000 7f0f83c90b80 1 -- 172.21.15.62:0/2368028801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561dac7f0250 con 0x561dac8ce6b0 2023-07-14T16:33:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.570+0000 7f0f51ffb700 1 -- 172.21.15.62:0/2368028801 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f64004830 con 0x561dac8ce6b0 2023-07-14T16:33:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.570+0000 7f0f51ffb700 1 -- 172.21.15.62:0/2368028801 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f64010460 con 0x561dac8ce6b0 2023-07-14T16:33:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.570+0000 7f0f83c90b80 10 client.29843 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.570+0000 7f0f83c90b80 10 client.29843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.570+0000 7f0f83c90b80 20 client.29843 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.570+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561dacafe7d0 0x561dacb00be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.570+0000 7f0f83c90b80 4 client.29843 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:33:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.570+0000 7f0f83c90b80 1 -- 172.21.15.62:0/2368028801 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561dac97eff0 con 0x561dacafe7d0 2023-07-14T16:33:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.571+0000 7f0f6d99a700 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561dacafe7d0 0x561dacb00be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.571+0000 7f0f6d99a700 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561dacafe7d0 0x561dacb00be0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae4bb591201099 server_cookie=b8af736caacaef24 in_seq=0 out_seq=0 2023-07-14T16:33:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.571+0000 7f0f51ffb700 10 client.29843 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.587+0000 7f0f51ffb700 1 -- 172.21.15.62:0/2368028801 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561dac97eff0 con 0x561dacafe7d0 2023-07-14T16:33:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.587+0000 7f0f51ffb700 10 client.29843 handle_command_reply: tid=0 2023-07-14T16:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.587+0000 7f0f83c90b80 1 client.29843 shutdown 2023-07-14T16:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.587+0000 7f0f83c90b80 1 -- 172.21.15.62:0/2368028801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 msgr2=0x561dacaa6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.587+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561dacaa6510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f64005fd0 tx=0x7f0f64005d20 comp rx=0 tx=0).stop 2023-07-14T16:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 1 -- 172.21.15.62:0/2368028801 shutdown_connections 2023-07-14T16:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561daca7beb0 0x561dacaa7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x561dacafe7d0 0x561dacb00be0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dac8ce6b0 0x561dacaa6510 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 1 --2- 172.21.15.62:0/2368028801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561daca7b0f0 0x561dacaa6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 1 -- 172.21.15.62:0/2368028801 >> 172.21.15.62:0/2368028801 conn(0x561daca67be0 msgr2=0x561daca7ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f51ffb700 0 client.29843 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 1 -- 172.21.15.62:0/2368028801 shutdown_connections 2023-07-14T16:33:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 1 -- 172.21.15.62:0/2368028801 wait complete. 2023-07-14T16:33:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:33:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:33:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:33:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:33:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.588+0000 7f0f83c90b80 20 client.29843 trim_cache size 0 max 16384 2023-07-14T16:33:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 -- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f540a1320 msgr2=0x7f0f540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f540a1320 0x7f0f540a37d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400d5c0 tx=0x7f0f64000bc0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 -- 172.21.15.62:0/1505680015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f68105760 msgr2=0x7f0f68194a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f68105760 0x7f0f68194a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000ef30 tx=0x7f0f6000c450 comp rx=0 tx=0).stop 2023-07-14T16:33:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 -- 172.21.15.62:0/1505680015 shutdown_connections 2023-07-14T16:33:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f68105760 0x7f0f68194a50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0f540a1320 0x7f0f540a37d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f680fd870 0x7f0f68193690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 --2- 172.21.15.62:0/1505680015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f680fe1c0 0x7f0f68193bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.592+0000 7f0f703ff700 1 -- 172.21.15.62:0/1505680015 >> 172.21.15.62:0/1505680015 conn(0x7f0f680f9330 msgr2=0x7f0f680f9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.593+0000 7f0f703ff700 1 -- 172.21.15.62:0/1505680015 shutdown_connections 2023-07-14T16:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:05.593+0000 7f0f703ff700 1 -- 172.21.15.62:0/1505680015 wait complete. 2023-07-14T16:33:07.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:06 smithi062 ceph-mon[95543]: pgmap v881: 97 pgs: 97 active+clean; 13 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-07-14T16:33:07.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:06 smithi081 ceph-mon[127795]: pgmap v881: 97 pgs: 97 active+clean; 13 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-07-14T16:33:07.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:06 smithi164 ceph-mon[97427]: pgmap v881: 97 pgs: 97 active+clean; 13 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-07-14T16:33:09.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:08 smithi081 ceph-mon[127795]: pgmap v882: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2023-07-14T16:33:09.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:08 smithi164 ceph-mon[97427]: pgmap v882: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2023-07-14T16:33:09.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:08 smithi062 ceph-mon[95543]: pgmap v882: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2023-07-14T16:33:10.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:33:10.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.738+0000 7f222e5ff700 1 -- 172.21.15.62:0/4233380705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22281036e0 msgr2=0x7f2228103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.738+0000 7f222e5ff700 1 --2- 172.21.15.62:0/4233380705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22281036e0 0x7f2228103b50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2220009f90 tx=0x7f222000b330 comp rx=0 tx=0).stop 2023-07-14T16:33:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.738+0000 7f222e5ff700 1 -- 172.21.15.62:0/4233380705 shutdown_connections 2023-07-14T16:33:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.738+0000 7f222e5ff700 1 --2- 172.21.15.62:0/4233380705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22281036e0 0x7f2228103b50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.738+0000 7f222e5ff700 1 --2- 172.21.15.62:0/4233380705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22280fe3c0 0x7f22281007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.738+0000 7f222e5ff700 1 --2- 172.21.15.62:0/4233380705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2228100d20 0x7f22281031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.738+0000 7f222e5ff700 1 -- 172.21.15.62:0/4233380705 >> 172.21.15.62:0/4233380705 conn(0x7f22280fa070 msgr2=0x7f22280fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.741+0000 7f222e5ff700 1 -- 172.21.15.62:0/4233380705 shutdown_connections 2023-07-14T16:33:10.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.741+0000 7f222e5ff700 1 -- 172.21.15.62:0/4233380705 wait complete. 2023-07-14T16:33:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.743+0000 7f222e5ff700 1 Processor -- start 2023-07-14T16:33:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.743+0000 7f222e5ff700 1 -- start start 2023-07-14T16:33:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.744+0000 7f222e5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22280fe3c0 0x7f222819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.744+0000 7f222e5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2228100d20 0x7f222819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.744+0000 7f222e5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22281036e0 0x7f22281a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.744+0000 7f222e5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22281049e0 con 0x7f2228100d20 2023-07-14T16:33:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.744+0000 7f222e5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2228104b50 con 0x7f22280fe3c0 2023-07-14T16:33:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.744+0000 7f222e5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2228104d10 con 0x7f22281036e0 2023-07-14T16:33:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f2227fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22280fe3c0 0x7f222819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f2227fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22280fe3c0 0x7f222819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51148/0 (socket says 172.21.15.62:51148) 2023-07-14T16:33:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f2227fff700 1 -- 172.21.15.62:0/663456369 learned_addr learned my addr 172.21.15.62:0/663456369 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f22277fe700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2228100d20 0x7f222819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f222cb9c700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22281036e0 0x7f22281a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f22277fe700 1 -- 172.21.15.62:0/663456369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22281036e0 msgr2=0x7f22281a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.721 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f22277fe700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22281036e0 0x7f22281a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f22277fe700 1 -- 172.21.15.62:0/663456369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22280fe3c0 msgr2=0x7f222819be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f22277fe700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22280fe3c0 0x7f222819be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f22277fe700 1 -- 172.21.15.62:0/663456369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2220009c40 con 0x7f2228100d20 2023-07-14T16:33:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.745+0000 7f222cb9c700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22281036e0 0x7f22281a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:33:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.746+0000 7f22277fe700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2228100d20 0x7f222819c3a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f221c00be10 tx=0x7f221c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.747+0000 7f22257fa700 1 -- 172.21.15.62:0/663456369 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c009b40 con 0x7f2228100d20 2023-07-14T16:33:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.747+0000 7f22257fa700 1 -- 172.21.15.62:0/663456369 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221c00de10 con 0x7f2228100d20 2023-07-14T16:33:10.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.747+0000 7f22257fa700 1 -- 172.21.15.62:0/663456369 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0313d0 con 0x7f2228100d20 2023-07-14T16:33:10.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.748+0000 7f222e5ff700 1 -- 172.21.15.62:0/663456369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2228104f90 con 0x7f2228100d20 2023-07-14T16:33:10.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.748+0000 7f222e5ff700 1 -- 172.21.15.62:0/663456369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2228105450 con 0x7f2228100d20 2023-07-14T16:33:10.727 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.755+0000 7f2241ea0b80 1 Processor -- start 2023-07-14T16:33:10.728 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.756+0000 7f22257fa700 1 -- 172.21.15.62:0/663456369 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f221c032470 con 0x7f2228100d20 2023-07-14T16:33:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.757+0000 7f2241ea0b80 1 -- start start 2023-07-14T16:33:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b1b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 0x555901b1c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b1d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901a20cb0 con 0x555901b1cc70 2023-07-14T16:33:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901a73a40 con 0x555901b1be20 2023-07-14T16:33:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901a6ced0 con 0x555901b1b4d0 2023-07-14T16:33:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f222cb9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b1b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f222cb9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b1b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47030/0 (socket says 172.21.15.62:47030) 2023-07-14T16:33:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f222cb9c700 1 -- 172.21.15.62:0/2888954873 learned_addr learned my addr 172.21.15.62:0/2888954873 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:10.731 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.758+0000 7f22277fe700 1 --2- 172.21.15.62:0/2888954873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b1d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.759+0000 7f222cb9c700 1 -- 172.21.15.62:0/2888954873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 msgr2=0x555901b1c6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:33:10.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.759+0000 7f222cb9c700 1 --2- 172.21.15.62:0/2888954873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 0x555901b1c6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.759+0000 7f222cb9c700 1 -- 172.21.15.62:0/2888954873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 msgr2=0x555901b1d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:10.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.759+0000 7f222cb9c700 1 --2- 172.21.15.62:0/2888954873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b1d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.759+0000 7f22257fa700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22100a1350 0x7f22100a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.759+0000 7f222cb9c700 1 -- 172.21.15.62:0/2888954873 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55590189ade0 con 0x555901b1b4d0 2023-07-14T16:33:10.732 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.759+0000 7f22257fa700 1 -- 172.21.15.62:0/663456369 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f221c0ee2b0 con 0x7f2228100d20 2023-07-14T16:33:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.760+0000 7f222cb9c700 1 --2- 172.21.15.62:0/2888954873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b1b8e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2220009390 tx=0x7f22200093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ff7943ece99018 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.760+0000 7f220e7fc700 1 -- 172.21.15.62:0/2888954873 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220021070 con 0x555901b1b4d0 2023-07-14T16:33:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.761+0000 7f220e7fc700 1 -- 172.21.15.62:0/2888954873 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22200074e0 con 0x555901b1b4d0 2023-07-14T16:33:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.761+0000 7f220e7fc700 1 -- 172.21.15.62:0/2888954873 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222000e040 con 0x555901b1b4d0 2023-07-14T16:33:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.762+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2888954873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 msgr2=0x555901b1b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.762+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2888954873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b1b8e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2220009390 tx=0x7f22200093c0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.762+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2888954873 shutdown_connections 2023-07-14T16:33:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.762+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2888954873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b1d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.762+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2888954873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b1b8e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.762+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2888954873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 0x555901b1c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.762+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2888954873 >> 172.21.15.62:0/2888954873 conn(0x555901b10da0 msgr2=0x555901b161d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.763+0000 7f2227fff700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22100a1350 0x7f22100a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.763+0000 7f2227fff700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22100a1350 0x7f22100a3800 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2218005950 tx=0x7f2218009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:10.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.765+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2888954873 shutdown_connections 2023-07-14T16:33:10.738 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.766+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2888954873 wait complete. 2023-07-14T16:33:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.766+0000 7f2241ea0b80 1 -- start start 2023-07-14T16:33:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.767+0000 7f2241ea0b80 1 Processor -- start 2023-07-14T16:33:10.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.767+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b25170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.767+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 0x555901b256b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.767+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b2c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.767+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901a0cac0 con 0x555901b1cc70 2023-07-14T16:33:10.740 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.767+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901a0c7c0 con 0x555901b1be20 2023-07-14T16:33:10.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.767+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559018f0920 con 0x555901b1b4d0 2023-07-14T16:33:10.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.767+0000 7f2241ea0b80 10 client.0 _ll_register_callbacks cb 0x555901a0a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:10.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.768+0000 7f222cb9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b2c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.768+0000 7f222cb9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b2c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50706/0 (socket says 172.21.15.62:50706) 2023-07-14T16:33:10.741 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.768+0000 7f222cb9c700 1 -- 172.21.15.62:0/89339159 learned_addr learned my addr 172.21.15.62:0/89339159 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:10.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.769+0000 7f22277fe700 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 0x555901b256b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.769+0000 7f222cb9c700 1 -- 172.21.15.62:0/89339159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 msgr2=0x555901b25170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:33:10.742 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.769+0000 7f222cb9c700 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b25170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.769+0000 7f222cb9c700 1 -- 172.21.15.62:0/89339159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 msgr2=0x555901b256b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.769+0000 7f222cb9c700 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 0x555901b256b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.769+0000 7f222cb9c700 1 -- 172.21.15.62:0/89339159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2228104680 con 0x555901b1cc70 2023-07-14T16:33:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.770+0000 7f222cb9c700 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b2c6f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f222000b900 tx=0x7f222000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50aec8c3c343ae04 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.770+0000 7f22077fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.770+0000 7f22077fe700 1 -- 172.21.15.62:0/89339159 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220021030 con 0x555901b1cc70 2023-07-14T16:33:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.770+0000 7f22077fe700 1 -- 172.21.15.62:0/89339159 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2220022070 con 0x555901b1cc70 2023-07-14T16:33:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.771+0000 7f2241ea0b80 1 -- 172.21.15.62:0/89339159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559019c64f0 con 0x555901b1cc70 2023-07-14T16:33:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.771+0000 7f22077fe700 1 -- 172.21.15.62:0/89339159 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f222000e070 con 0x555901b1cc70 2023-07-14T16:33:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.771+0000 7f2241ea0b80 10 client.20304 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.771+0000 7f2241ea0b80 1 -- 172.21.15.62:0/89339159 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555901883c60 con 0x555901b1cc70 2023-07-14T16:33:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.771+0000 7f22077fe700 1 -- 172.21.15.62:0/89339159 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5559019c64f0 con 0x555901b1cc70 2023-07-14T16:33:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.771+0000 7f22077fe700 1 -- 172.21.15.62:0/89339159 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2220019040 con 0x555901b1cc70 2023-07-14T16:33:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.772+0000 7f2241ea0b80 10 client.20304 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.772+0000 7f2241ea0b80 10 client.20304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.772+0000 7f2241ea0b80 20 client.20304 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.772+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555901b30ac0 0x555901b32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.772+0000 7f2241ea0b80 4 client.20304 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.772+0000 7f2241ea0b80 1 -- 172.21.15.62:0/89339159 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555901a65eb0 con 0x555901b30ac0 2023-07-14T16:33:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.772+0000 7f2227fff700 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555901b30ac0 0x555901b32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.773+0000 7f2227fff700 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555901b30ac0 0x555901b32ed0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae57c4264ec126a7 server_cookie=1f9c6199a9266698 in_seq=0 out_seq=0 2023-07-14T16:33:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.773+0000 7f22077fe700 10 client.20304 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.774+0000 7f22077fe700 1 -- 172.21.15.62:0/89339159 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555901a65eb0 con 0x555901b30ac0 2023-07-14T16:33:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.774+0000 7f22077fe700 10 client.20304 handle_command_reply: tid=0 2023-07-14T16:33:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.774+0000 7f2241ea0b80 1 client.20304 shutdown 2023-07-14T16:33:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.774+0000 7f2241ea0b80 1 -- 172.21.15.62:0/89339159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 msgr2=0x555901b2c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.774+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b2c6f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f222000b900 tx=0x7f222000bcb0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.774+0000 7f2241ea0b80 1 -- 172.21.15.62:0/89339159 shutdown_connections 2023-07-14T16:33:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.774+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b2c6f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.775+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1b4d0 0x555901b25170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.775+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555901b30ac0 0x555901b32ed0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.775+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/89339159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1be20 0x555901b256b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.775+0000 7f2241ea0b80 1 -- 172.21.15.62:0/89339159 >> 172.21.15.62:0/89339159 conn(0x555901b10da0 msgr2=0x555901b15fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.775+0000 7f22077fe700 0 client.20304 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:10.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.775+0000 7f2241ea0b80 1 -- 172.21.15.62:0/89339159 shutdown_connections 2023-07-14T16:33:10.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.775+0000 7f2241ea0b80 1 -- 172.21.15.62:0/89339159 wait complete. 2023-07-14T16:33:10.750 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.775+0000 7f2241ea0b80 20 client.20304 trim_cache size 0 max 16384 2023-07-14T16:33:10.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.806+0000 7f2241ea0b80 1 Processor -- start 2023-07-14T16:33:10.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.806+0000 7f2241ea0b80 1 -- start start 2023-07-14T16:33:10.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.806+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1b4d0 0x555901b2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.806+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b2c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.806+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1cc70 0x555901b23070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.806+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901b109a0 con 0x555901b1b4d0 2023-07-14T16:33:10.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.806+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901b1b8a0 con 0x555901b1cc70 2023-07-14T16:33:10.780 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.806+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901b1ba30 con 0x555901b1be20 2023-07-14T16:33:10.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.807+0000 7f22277fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b2c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.807+0000 7f22277fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b2c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47054/0 (socket says 172.21.15.62:47054) 2023-07-14T16:33:10.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.807+0000 7f22277fe700 1 -- 172.21.15.62:0/569562023 learned_addr learned my addr 172.21.15.62:0/569562023 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:10.781 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.807+0000 7f2227fff700 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1b4d0 0x555901b2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.807+0000 7f2227fff700 1 -- 172.21.15.62:0/569562023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 msgr2=0x555901b2c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.807+0000 7f2227fff700 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b2c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.807+0000 7f2227fff700 1 -- 172.21.15.62:0/569562023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1cc70 msgr2=0x555901b23070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:33:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.807+0000 7f2227fff700 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1cc70 0x555901b23070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.808+0000 7f2227fff700 1 -- 172.21.15.62:0/569562023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559018c1250 con 0x555901b1b4d0 2023-07-14T16:33:10.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.808+0000 7f22277fe700 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b2c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:10.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.808+0000 7f2227fff700 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1b4d0 0x555901b2cfe0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x555901b30e90 tx=0x7f2218014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88598f153e1d7279 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:10.783 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.808+0000 7f220effd700 1 -- 172.21.15.62:0/569562023 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218025070 con 0x555901b1b4d0 2023-07-14T16:33:10.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.808+0000 7f220effd700 1 -- 172.21.15.62:0/569562023 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2218016070 con 0x555901b1b4d0 2023-07-14T16:33:10.784 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.808+0000 7f220effd700 1 -- 172.21.15.62:0/569562023 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218012880 con 0x555901b1b4d0 2023-07-14T16:33:10.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.810+0000 7f2241ea0b80 1 -- 172.21.15.62:0/569562023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1b4d0 msgr2=0x555901b2cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.785 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.810+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1b4d0 0x555901b2cfe0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x555901b30e90 tx=0x7f2218014000 comp rx=0 tx=0).stop 2023-07-14T16:33:10.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.810+0000 7f2241ea0b80 1 -- 172.21.15.62:0/569562023 shutdown_connections 2023-07-14T16:33:10.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.810+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b1cc70 0x555901b23070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.786 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.810+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1b4d0 0x555901b2cfe0 secure :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x555901b30e90 tx=0x7f2218014000 comp rx=0 tx=0).stop 2023-07-14T16:33:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.810+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/569562023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b2c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.810+0000 7f2241ea0b80 1 -- 172.21.15.62:0/569562023 >> 172.21.15.62:0/569562023 conn(0x555901b10da0 msgr2=0x555901b23df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.812+0000 7f2241ea0b80 1 -- 172.21.15.62:0/569562023 shutdown_connections 2023-07-14T16:33:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.812+0000 7f2241ea0b80 1 -- 172.21.15.62:0/569562023 wait complete. 2023-07-14T16:33:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.812+0000 7f2241ea0b80 1 -- start start 2023-07-14T16:33:10.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.813+0000 7f2241ea0b80 1 Processor -- start 2023-07-14T16:33:10.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.814+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b29370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.814+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.814+0000 7f2241ea0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b82960 0x555901b84d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.814+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901b29df0 con 0x555901b1cc70 2023-07-14T16:33:10.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.814+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901b29f30 con 0x555901b82960 2023-07-14T16:33:10.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.814+0000 7f2241ea0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555901b96290 con 0x555901b1be20 2023-07-14T16:33:10.789 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.814+0000 7f2241ea0b80 10 client.0 _ll_register_callbacks cb 0x7f221c0f1b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:10.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.815+0000 7f2227fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b29370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.815+0000 7f222cb9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b82960 0x555901b84d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.815+0000 7f222cb9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b82960 0x555901b84d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51164/0 (socket says 172.21.15.62:51164) 2023-07-14T16:33:10.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.815+0000 7f222cb9c700 1 -- 172.21.15.62:0/2883891675 learned_addr learned my addr 172.21.15.62:0/2883891675 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:10.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.815+0000 7f22277fe700 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.816+0000 7f2227fff700 1 -- 172.21.15.62:0/2883891675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b82960 msgr2=0x555901b84d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:10.790 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.816+0000 7f2227fff700 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b82960 0x555901b84d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.816+0000 7f2227fff700 1 -- 172.21.15.62:0/2883891675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 msgr2=0x555901b298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:10.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.816+0000 7f2227fff700 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.791 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.816+0000 7f2227fff700 1 -- 172.21.15.62:0/2883891675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2228105450 con 0x555901b1be20 2023-07-14T16:33:10.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.816+0000 7f2227fff700 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b29370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x555901b17e20 tx=0x7f2218011da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0408caa7aa7b15c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:10.792 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.816+0000 7f22077fe700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:10.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.817+0000 7f22077fe700 1 -- 172.21.15.62:0/2883891675 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218025040 con 0x555901b1be20 2023-07-14T16:33:10.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.817+0000 7f22077fe700 1 -- 172.21.15.62:0/2883891675 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2218018070 con 0x555901b1be20 2023-07-14T16:33:10.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.817+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2883891675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22200074e0 con 0x555901b1be20 2023-07-14T16:33:10.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.817+0000 7f22077fe700 1 -- 172.21.15.62:0/2883891675 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22180251a0 con 0x555901b1be20 2023-07-14T16:33:10.793 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.817+0000 7f2241ea0b80 10 client.29737 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:10.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.817+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2883891675 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f2218011e30 con 0x555901b1be20 2023-07-14T16:33:10.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.818+0000 7f22077fe700 1 -- 172.21.15.62:0/2883891675 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221803d020 con 0x555901b1be20 2023-07-14T16:33:10.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.818+0000 7f22077fe700 1 -- 172.21.15.62:0/2883891675 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f221802c070 con 0x555901b1be20 2023-07-14T16:33:10.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.818+0000 7f2241ea0b80 10 client.29737 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:10.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.818+0000 7f2241ea0b80 10 client.29737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:10.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.818+0000 7f2241ea0b80 20 client.29737 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:10.794 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.818+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555901b97a30 0x555901b99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:10.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.818+0000 7f2241ea0b80 4 client.29737 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:33:10.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.818+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2883891675 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555901a28050 con 0x555901b97a30 2023-07-14T16:33:10.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.819+0000 7f22277fe700 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555901b97a30 0x555901b99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:10.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.819+0000 7f22277fe700 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555901b97a30 0x555901b99e40 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15169b702a1395a8 server_cookie=6098dcfb2545fce2 in_seq=0 out_seq=0 2023-07-14T16:33:10.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.819+0000 7f22077fe700 10 client.29737 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:10.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.819+0000 7f22077fe700 1 -- 172.21.15.62:0/2883891675 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555901a28050 con 0x555901b97a30 2023-07-14T16:33:10.795 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.819+0000 7f22077fe700 10 client.29737 handle_command_reply: tid=0 2023-07-14T16:33:10.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 client.29737 shutdown 2023-07-14T16:33:10.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2883891675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 msgr2=0x555901b29370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.796 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b29370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x555901b17e20 tx=0x7f2218011da0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2883891675 shutdown_connections 2023-07-14T16:33:10.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555901b97a30 0x555901b99e40 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555901b82960 0x555901b84d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555901b1be20 0x555901b29370 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f22077fe700 0 client.29737 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:10.797 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 --2- 172.21.15.62:0/2883891675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555901b1cc70 0x555901b298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2883891675 >> 172.21.15.62:0/2883891675 conn(0x555901b10da0 msgr2=0x555901b23070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:10.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.820+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2883891675 shutdown_connections 2023-07-14T16:33:10.798 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.821+0000 7f2241ea0b80 1 -- 172.21.15.62:0/2883891675 wait complete. 2023-07-14T16:33:10.798 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:33:10.798 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:33:10.799 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:33:10.799 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:33:10.799 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.821+0000 7f2241ea0b80 20 client.29737 trim_cache size 0 max 16384 2023-07-14T16:33:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 -- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22100a1350 msgr2=0x7f22100a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22100a1350 0x7f22100a3800 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2218005950 tx=0x7f2218009310 comp rx=0 tx=0).stop 2023-07-14T16:33:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 -- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2228100d20 msgr2=0x7f222819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2228100d20 0x7f222819c3a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f221c00be10 tx=0x7f221c00dc30 comp rx=0 tx=0).stop 2023-07-14T16:33:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 -- 172.21.15.62:0/663456369 shutdown_connections 2023-07-14T16:33:10.800 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22281036e0 0x7f22281a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f22100a1350 0x7f22100a3800 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22280fe3c0 0x7f222819be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 --2- 172.21.15.62:0/663456369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2228100d20 0x7f222819c3a0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:10.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 -- 172.21.15.62:0/663456369 >> 172.21.15.62:0/663456369 conn(0x7f22280fa070 msgr2=0x7f22280ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:10.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 -- 172.21.15.62:0/663456369 shutdown_connections 2023-07-14T16:33:10.801 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:10.822+0000 7f222e5ff700 1 -- 172.21.15.62:0/663456369 wait complete. 2023-07-14T16:33:10.827 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:33:10.828 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:33:11.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:10 smithi062 ceph-mon[95543]: pgmap v883: 97 pgs: 97 active+clean; 13 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.05k op/s 2023-07-14T16:33:11.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:10 smithi081 ceph-mon[127795]: pgmap v883: 97 pgs: 97 active+clean; 13 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.05k op/s 2023-07-14T16:33:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.215+0000 7f027d5ff700 1 -- 172.21.15.62:0/3570992262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 msgr2=0x7f0278101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.215+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3570992262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 0x7f0278101160 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0268009990 tx=0x7f0268009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.191 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.216+0000 7f027d5ff700 1 -- 172.21.15.62:0/3570992262 shutdown_connections 2023-07-14T16:33:11.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.216+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3570992262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02781016a0 0x7f0278107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.216+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3570992262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02780ffb20 0x7f02780fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.216+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3570992262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 0x7f0278101160 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.216+0000 7f027d5ff700 1 -- 172.21.15.62:0/3570992262 >> 172.21.15.62:0/3570992262 conn(0x7f02780fb160 msgr2=0x7f02780fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:11.192 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.216+0000 7f027d5ff700 1 -- 172.21.15.62:0/3570992262 shutdown_connections 2023-07-14T16:33:11.193 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.217+0000 7f027d5ff700 1 -- 172.21.15.62:0/3570992262 wait complete. 2023-07-14T16:33:11.194 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.220+0000 7f027d5ff700 1 Processor -- start 2023-07-14T16:33:11.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.222+0000 7f027d5ff700 1 -- start start 2023-07-14T16:33:11.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.222+0000 7f027d5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02780ffb20 0x7f027806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:11.196 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.222+0000 7f027d5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 0x7f027806f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:11.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.222+0000 7f027d5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02781016a0 0x7f02781b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:11.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.222+0000 7f027d5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780745f0 con 0x7f0278100d10 2023-07-14T16:33:11.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.222+0000 7f027d5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278074760 con 0x7f02780ffb20 2023-07-14T16:33:11.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.222+0000 7f027d5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02781b4e50 con 0x7f02781016a0 2023-07-14T16:33:11.197 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.223+0000 7f02767fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 0x7f027806f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.223+0000 7f02767fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 0x7f027806f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50738/0 (socket says 172.21.15.62:50738) 2023-07-14T16:33:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.223+0000 7f02767fc700 1 -- 172.21.15.62:0/3044411433 learned_addr learned my addr 172.21.15.62:0/3044411433 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.223+0000 7f02777fe700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02781016a0 0x7f02781b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:11.198 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.223+0000 7f0276ffd700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02780ffb20 0x7f027806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.224+0000 7f02777fe700 1 -- 172.21.15.62:0/3044411433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02780ffb20 msgr2=0x7f027806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.224+0000 7f02777fe700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02780ffb20 0x7f027806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.224+0000 7f02777fe700 1 -- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 msgr2=0x7f027806f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.224+0000 7f02777fe700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 0x7f027806f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.224+0000 7f02777fe700 1 -- 172.21.15.62:0/3044411433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680095f0 con 0x7f02781016a0 2023-07-14T16:33:11.199 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.225+0000 7f02777fe700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02781016a0 0x7f02781b4910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f027000efc0 tx=0x7f027000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.225+0000 7f025ffff700 1 -- 172.21.15.62:0/3044411433 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270018070 con 0x7f02781016a0 2023-07-14T16:33:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.225+0000 7f025ffff700 1 -- 172.21.15.62:0/3044411433 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f027000f040 con 0x7f02781016a0 2023-07-14T16:33:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.225+0000 7f025ffff700 1 -- 172.21.15.62:0/3044411433 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000cc50 con 0x7f02781016a0 2023-07-14T16:33:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.225+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02781b5050 con 0x7f02781016a0 2023-07-14T16:33:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.225+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02781b5500 con 0x7f02781016a0 2023-07-14T16:33:11.200 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.226+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0278106e00 con 0x7f02781016a0 2023-07-14T16:33:11.201 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.228+0000 7f025ffff700 1 -- 172.21.15.62:0/3044411433 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0270003a00 con 0x7f02781016a0 2023-07-14T16:33:11.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.230+0000 7f025ffff700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f02600a1350 0x7f02600a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:11.202 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.230+0000 7f025ffff700 1 -- 172.21.15.62:0/3044411433 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f02700cc380 con 0x7f02781016a0 2023-07-14T16:33:11.204 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.232+0000 7f0276ffd700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f02600a1350 0x7f02600a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:11.206 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.233+0000 7f0276ffd700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f02600a1350 0x7f02600a3800 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0278100b70 tx=0x7f026c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:11.208 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.236+0000 7f025ffff700 1 -- 172.21.15.62:0/3044411433 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0270095b10 con 0x7f02781016a0 2023-07-14T16:33:11.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:10 smithi164 ceph-mon[97427]: pgmap v883: 97 pgs: 97 active+clean; 13 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.05k op/s 2023-07-14T16:33:11.684 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.712+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f027802c850 con 0x7f02781016a0 2023-07-14T16:33:11.685 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.713+0000 7f025ffff700 1 -- 172.21.15.62:0/3044411433 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10828 (secure 0 0 0) 0x7f0270099130 con 0x7f02781016a0 2023-07-14T16:33:11.688 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:33:11.688 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:31:18.877417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:33:11.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.720+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f02600a1350 msgr2=0x7f02600a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:11.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.720+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f02600a1350 0x7f02600a3800 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0278100b70 tx=0x7f026c009190 comp rx=0 tx=0).stop 2023-07-14T16:33:11.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.720+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02781016a0 msgr2=0x7f02781b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:11.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.720+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02781016a0 0x7f02781b4910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f027000efc0 tx=0x7f027000c3a0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.722+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 shutdown_connections 2023-07-14T16:33:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.722+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02781016a0 0x7f02781b4910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.722+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f02600a1350 0x7f02600a3800 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.722+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02780ffb20 0x7f027806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.722+0000 7f027d5ff700 1 --2- 172.21.15.62:0/3044411433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0278100d10 0x7f027806f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:11.695 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.722+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 >> 172.21.15.62:0/3044411433 conn(0x7f02780fb160 msgr2=0x7f0278103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.722+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 shutdown_connections 2023-07-14T16:33:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:11.722+0000 7f027d5ff700 1 -- 172.21.15.62:0/3044411433 wait complete. 2023-07-14T16:33:11.696 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2023-07-14T16:33:11.728 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:33:12.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.131+0000 7f3e08dff700 1 -- 172.21.15.62:0/3085810917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e041061a0 msgr2=0x7f3e041065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.131+0000 7f3e08dff700 1 --2- 172.21.15.62:0/3085810917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e041061a0 0x7f3e041065b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009f90 tx=0x7f3df400b330 comp rx=0 tx=0).stop 2023-07-14T16:33:12.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.131+0000 7f3e08dff700 1 -- 172.21.15.62:0/3085810917 shutdown_connections 2023-07-14T16:33:12.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.131+0000 7f3e08dff700 1 --2- 172.21.15.62:0/3085810917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e04107d20 0x7f3e0410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.131+0000 7f3e08dff700 1 --2- 172.21.15.62:0/3085810917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e041061a0 0x7f3e041065b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.131+0000 7f3e08dff700 1 --2- 172.21.15.62:0/3085810917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e04107390 0x7f3e041077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.131+0000 7f3e08dff700 1 -- 172.21.15.62:0/3085810917 >> 172.21.15.62:0/3085810917 conn(0x7f3e040751c0 msgr2=0x7f3e04077610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.132+0000 7f3e08dff700 1 -- 172.21.15.62:0/3085810917 shutdown_connections 2023-07-14T16:33:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.132+0000 7f3e08dff700 1 -- 172.21.15.62:0/3085810917 wait complete. 2023-07-14T16:33:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.133+0000 7f3e08dff700 1 Processor -- start 2023-07-14T16:33:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.133+0000 7f3e08dff700 1 -- start start 2023-07-14T16:33:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3e08dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e041061a0 0x7f3e04195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3e08dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e04107390 0x7f3e041983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3e08dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e04107d20 0x7f3e0419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3e08dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419b100 con 0x7f3e041061a0 2023-07-14T16:33:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3e08dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419b270 con 0x7f3e04107390 2023-07-14T16:33:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3e08dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419b430 con 0x7f3e04107d20 2023-07-14T16:33:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3df9d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e04107390 0x7f3e041983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3df9d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e04107390 0x7f3e041983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51188/0 (socket says 172.21.15.62:51188) 2023-07-14T16:33:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3df9d9b700 1 -- 172.21.15.62:0/4044782617 learned_addr learned my addr 172.21.15.62:0/4044782617 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3e02d9d700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e04107d20 0x7f3e0419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.134+0000 7f3e0259c700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e041061a0 0x7f3e04195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.135+0000 7f3df9d9b700 1 -- 172.21.15.62:0/4044782617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e04107d20 msgr2=0x7f3e0419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.135+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e04107d20 0x7f3e0419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.135+0000 7f3df9d9b700 1 -- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e041061a0 msgr2=0x7f3e04195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.135+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e041061a0 0x7f3e04195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.135+0000 7f3df9d9b700 1 -- 172.21.15.62:0/4044782617 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec0095f0 con 0x7f3e04107390 2023-07-14T16:33:12.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.135+0000 7f3e0259c700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e041061a0 0x7f3e04195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:12.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.136+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e04107390 0x7f3e041983d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00ec10 tx=0x7f3dec00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.137+0000 7f3dfbfff700 1 -- 172.21.15.62:0/4044782617 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec00cd40 con 0x7f3e04107390 2023-07-14T16:33:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.137+0000 7f3dfbfff700 1 -- 172.21.15.62:0/4044782617 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3dec0044b0 con 0x7f3e04107390 2023-07-14T16:33:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.137+0000 7f3dfbfff700 1 -- 172.21.15.62:0/4044782617 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec010430 con 0x7f3e04107390 2023-07-14T16:33:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.137+0000 7f3e08dff700 1 -- 172.21.15.62:0/4044782617 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df4009c40 con 0x7f3e04107390 2023-07-14T16:33:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.138+0000 7f3e08dff700 1 -- 172.21.15.62:0/4044782617 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0419ba40 con 0x7f3e04107390 2023-07-14T16:33:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.143+0000 7f3dfbfff700 1 -- 172.21.15.62:0/4044782617 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3dec003fd0 con 0x7f3e04107390 2023-07-14T16:33:12.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.144+0000 7f3e1c761b80 1 Processor -- start 2023-07-14T16:33:12.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.144+0000 7f3e1c761b80 1 -- start start 2023-07-14T16:33:12.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c1678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c167e20 0x55769c1686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c168c70 0x55769c1690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c06ccb0 con 0x55769c168c70 2023-07-14T16:33:12.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c0bfa40 con 0x55769c167e20 2023-07-14T16:33:12.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c0b8ed0 con 0x55769c1674d0 2023-07-14T16:33:12.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:11 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3044411433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:33:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3df9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c168c70 0x55769c1690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3df9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c168c70 0x55769c1690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50758/0 (socket says 172.21.15.62:50758) 2023-07-14T16:33:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.145+0000 7f3df9d9b700 1 -- 172.21.15.62:0/3519357155 learned_addr learned my addr 172.21.15.62:0/3519357155 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.146+0000 7f3e0259c700 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c167e20 0x55769c1686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.146+0000 7f3e02d9d700 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c1678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.146+0000 7f3e0259c700 1 -- 172.21.15.62:0/3519357155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 msgr2=0x55769c1678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.146+0000 7f3e0259c700 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c1678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.146+0000 7f3e0259c700 1 -- 172.21.15.62:0/3519357155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c168c70 msgr2=0x55769c1690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.146+0000 7f3e0259c700 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c168c70 0x55769c1690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.146+0000 7f3e0259c700 1 -- 172.21.15.62:0/3519357155 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55769bee6de0 con 0x55769c167e20 2023-07-14T16:33:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.147+0000 7f3dfbfff700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3df00a11d0 0x7f3df00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.147+0000 7f3dfbfff700 1 -- 172.21.15.62:0/4044782617 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3dec014070 con 0x7f3e04107390 2023-07-14T16:33:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.147+0000 7f3e0259c700 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c167e20 0x55769c1686e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b6c0 tx=0x7f3df4007520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce236033b07d734 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.148+0000 7f3df8d99700 1 -- 172.21.15.62:0/3519357155 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df40092d0 con 0x55769c167e20 2023-07-14T16:33:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.148+0000 7f3df8d99700 1 -- 172.21.15.62:0/3519357155 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3df4031070 con 0x55769c167e20 2023-07-14T16:33:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.148+0000 7f3df8d99700 1 -- 172.21.15.62:0/3519357155 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df400bbf0 con 0x55769c167e20 2023-07-14T16:33:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.149+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3519357155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c167e20 msgr2=0x55769c1686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.149+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c167e20 0x55769c1686e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b6c0 tx=0x7f3df4007520 comp rx=0 tx=0).stop 2023-07-14T16:33:12.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.151+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3519357155 shutdown_connections 2023-07-14T16:33:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.151+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c168c70 0x55769c1690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.151+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c1678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.151+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/3519357155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c167e20 0x55769c1686e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.151+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3519357155 >> 172.21.15.62:0/3519357155 conn(0x55769c15cda0 msgr2=0x55769c1621d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.152+0000 7f3e02d9d700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3df00a11d0 0x7f3df00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.153+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3519357155 shutdown_connections 2023-07-14T16:33:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.153+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3519357155 wait complete. 2023-07-14T16:33:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.153+0000 7f3e02d9d700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3df00a11d0 0x7f3df00a3680 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3e0410eaf0 tx=0x7f3dfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.154+0000 7f3e1c761b80 1 -- start start 2023-07-14T16:33:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.154+0000 7f3e1c761b80 1 Processor -- start 2023-07-14T16:33:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.155+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c1674d0 0x55769c171110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.155+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c171650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.155+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c168c70 0x55769c178690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c058ac0 con 0x55769c167e20 2023-07-14T16:33:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c0587c0 con 0x55769c1674d0 2023-07-14T16:33:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769bf3c920 con 0x55769c168c70 2023-07-14T16:33:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3e1c761b80 10 client.0 _ll_register_callbacks cb 0x55769c056760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:12.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3df9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c171650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3e0259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c1674d0 0x55769c171110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3e02d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c168c70 0x55769c178690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3e0259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c1674d0 0x55769c171110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51214/0 (socket says 172.21.15.62:51214) 2023-07-14T16:33:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.156+0000 7f3e0259c700 1 -- 172.21.15.62:0/725419380 learned_addr learned my addr 172.21.15.62:0/725419380 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.157+0000 7f3e0259c700 1 -- 172.21.15.62:0/725419380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c168c70 msgr2=0x55769c178690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.157+0000 7f3e0259c700 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c168c70 0x55769c178690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.157+0000 7f3e0259c700 1 -- 172.21.15.62:0/725419380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 msgr2=0x55769c171650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.157+0000 7f3e0259c700 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c171650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.157+0000 7f3e0259c700 1 -- 172.21.15.62:0/725419380 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0410ef10 con 0x55769c1674d0 2023-07-14T16:33:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.157+0000 7f3e0259c700 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c1674d0 0x55769c171110 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3df4004ed0 tx=0x7f3df4004f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4640877f54020d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.157+0000 7f3de9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.158+0000 7f3de9ffb700 1 -- 172.21.15.62:0/725419380 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4007c60 con 0x55769c1674d0 2023-07-14T16:33:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.158+0000 7f3de9ffb700 1 -- 172.21.15.62:0/725419380 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3df4007dc0 con 0x55769c1674d0 2023-07-14T16:33:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.158+0000 7f3e1c761b80 1 -- 172.21.15.62:0/725419380 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55769c0124f0 con 0x55769c1674d0 2023-07-14T16:33:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.158+0000 7f3de9ffb700 1 -- 172.21.15.62:0/725419380 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df4003a00 con 0x55769c1674d0 2023-07-14T16:33:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.158+0000 7f3e1c761b80 10 client.29873 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.158+0000 7f3e1c761b80 1 -- 172.21.15.62:0/725419380 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55769becfc60 con 0x55769c1674d0 2023-07-14T16:33:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.159+0000 7f3de9ffb700 1 -- 172.21.15.62:0/725419380 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55769c0124f0 con 0x55769c1674d0 2023-07-14T16:33:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.159+0000 7f3de9ffb700 1 -- 172.21.15.62:0/725419380 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3df400b050 con 0x55769c1674d0 2023-07-14T16:33:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.159+0000 7f3e1c761b80 10 client.29873 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.159+0000 7f3e1c761b80 10 client.29873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:33:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.159+0000 7f3e1c761b80 20 client.29873 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.159+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55769c17c9c0 0x55769c17edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.159+0000 7f3e1c761b80 4 client.29873 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.159+0000 7f3e1c761b80 1 -- 172.21.15.62:0/725419380 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55769c0b1eb0 con 0x55769c17c9c0 2023-07-14T16:33:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.160+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55769c17c9c0 0x55769c17edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.161+0000 7f3de9ffb700 10 client.29873 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:33:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.161+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55769c17c9c0 0x55769c17edd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b076462f081d18c8 server_cookie=efed1b39af7c135c in_seq=0 out_seq=0 2023-07-14T16:33:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.162+0000 7f3de9ffb700 1 -- 172.21.15.62:0/725419380 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55769c0b1eb0 con 0x55769c17c9c0 2023-07-14T16:33:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.162+0000 7f3de9ffb700 10 client.29873 handle_command_reply: tid=0 2023-07-14T16:33:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.162+0000 7f3e1c761b80 1 client.29873 shutdown 2023-07-14T16:33:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 -- 172.21.15.62:0/725419380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c1674d0 msgr2=0x55769c171110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c1674d0 0x55769c171110 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3df4004ed0 tx=0x7f3df4004f00 comp rx=0 tx=0).stop 2023-07-14T16:33:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 -- 172.21.15.62:0/725419380 shutdown_connections 2023-07-14T16:33:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55769c17c9c0 0x55769c17edd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c168c70 0x55769c178690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c1674d0 0x55769c171110 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/725419380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c171650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 -- 172.21.15.62:0/725419380 >> 172.21.15.62:0/725419380 conn(0x55769c15cda0 msgr2=0x55769c1618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3de9ffb700 0 client.29873 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:33:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.163+0000 7f3e1c761b80 1 -- 172.21.15.62:0/725419380 shutdown_connections 2023-07-14T16:33:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.164+0000 7f3e1c761b80 1 -- 172.21.15.62:0/725419380 wait complete. 2023-07-14T16:33:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.164+0000 7f3e1c761b80 20 client.29873 trim_cache size 0 max 16384 2023-07-14T16:33:12.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:11 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3044411433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:33:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.189+0000 7f3e1c761b80 1 Processor -- start 2023-07-14T16:33:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.189+0000 7f3e1c761b80 1 -- start start 2023-07-14T16:33:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.189+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c16f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c178650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 0x55769c16bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c168570 con 0x55769c167e20 2023-07-14T16:33:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c15c990 con 0x55769c168c70 2023-07-14T16:33:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c1705e0 con 0x55769c1674d0 2023-07-14T16:33:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3df9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c178650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3df9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c178650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50782/0 (socket says 172.21.15.62:50782) 2023-07-14T16:33:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3df9d9b700 1 -- 172.21.15.62:0/3330689905 learned_addr learned my addr 172.21.15.62:0/3330689905 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.190+0000 7f3e0259c700 1 --2- 172.21.15.62:0/3330689905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c16f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.191+0000 7f3e0259c700 1 -- 172.21.15.62:0/3330689905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 msgr2=0x55769c16bbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:33:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.191+0000 7f3e0259c700 1 --2- 172.21.15.62:0/3330689905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 0x55769c16bbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.191+0000 7f3e0259c700 1 -- 172.21.15.62:0/3330689905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 msgr2=0x55769c178650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.191+0000 7f3e0259c700 1 --2- 172.21.15.62:0/3330689905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c178650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.191+0000 7f3e0259c700 1 -- 172.21.15.62:0/3330689905 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55769c169310 con 0x55769c1674d0 2023-07-14T16:33:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.191+0000 7f3e0259c700 1 --2- 172.21.15.62:0/3330689905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c16f060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3df4023ae0 tx=0x7f3df4036480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf26592a60b9570c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.192+0000 7f3df959a700 1 -- 172.21.15.62:0/3330689905 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df40367f0 con 0x55769c1674d0 2023-07-14T16:33:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.192+0000 7f3df959a700 1 -- 172.21.15.62:0/3330689905 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3df403b040 con 0x55769c1674d0 2023-07-14T16:33:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.192+0000 7f3df959a700 1 -- 172.21.15.62:0/3330689905 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4003a00 con 0x55769c1674d0 2023-07-14T16:33:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.193+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3330689905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 msgr2=0x55769c16f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.193+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/3330689905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c16f060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3df4023ae0 tx=0x7f3df4036480 comp rx=0 tx=0).stop 2023-07-14T16:33:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.193+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3330689905 shutdown_connections 2023-07-14T16:33:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.193+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/3330689905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 0x55769c16bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.193+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/3330689905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c1674d0 0x55769c16f060 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.193+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/3330689905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c167e20 0x55769c178650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.193+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3330689905 >> 172.21.15.62:0/3330689905 conn(0x55769c15cda0 msgr2=0x55769c178f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.193+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3330689905 shutdown_connections 2023-07-14T16:33:12.167 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.194+0000 7f3e1c761b80 1 -- 172.21.15.62:0/3330689905 wait complete. 2023-07-14T16:33:12.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.194+0000 7f3e1c761b80 1 -- start start 2023-07-14T16:33:12.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.194+0000 7f3e1c761b80 1 Processor -- start 2023-07-14T16:33:12.168 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.195+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c1674d0 0x55769c1747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.195+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c167e20 0x55769c174d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.195+0000 7f3e1c761b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 0x55769c177ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.195+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c178220 con 0x55769c1674d0 2023-07-14T16:33:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.195+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c178390 con 0x55769c168c70 2023-07-14T16:33:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.195+0000 7f3e1c761b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55769c178500 con 0x55769c167e20 2023-07-14T16:33:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.195+0000 7f3e1c761b80 10 client.0 _ll_register_callbacks cb 0x7f3dec0d0830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e0259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c1674d0 0x55769c1747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e0259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c1674d0 0x55769c1747e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50790/0 (socket says 172.21.15.62:50790) 2023-07-14T16:33:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e0259c700 1 -- 172.21.15.62:0/474153876 learned_addr learned my addr 172.21.15.62:0/474153876 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c167e20 0x55769c174d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e02d9d700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 0x55769c177ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e0259c700 1 -- 172.21.15.62:0/474153876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c167e20 msgr2=0x55769c174d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e0259c700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c167e20 0x55769c174d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e0259c700 1 -- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 msgr2=0x55769c177ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.171 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e0259c700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 0x55769c177ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3e0259c700 1 -- 172.21.15.62:0/474153876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55769becfc60 con 0x55769c1674d0 2023-07-14T16:33:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.196+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c167e20 0x55769c174d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.197+0000 7f3e02d9d700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 0x55769c177ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.198+0000 7f3e0259c700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c1674d0 0x55769c1747e0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b700 tx=0x7f3df4024e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d8e558040fb111 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.172 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.198+0000 7f3de9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:12.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.198+0000 7f3de9ffb700 1 -- 172.21.15.62:0/474153876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df40367f0 con 0x55769c1674d0 2023-07-14T16:33:12.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.198+0000 7f3de9ffb700 1 -- 172.21.15.62:0/474153876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3df4007c70 con 0x55769c1674d0 2023-07-14T16:33:12.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.198+0000 7f3e1c761b80 1 -- 172.21.15.62:0/474153876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3df4031070 con 0x55769c1674d0 2023-07-14T16:33:12.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.201+0000 7f3de9ffb700 1 -- 172.21.15.62:0/474153876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df4007e30 con 0x55769c1674d0 2023-07-14T16:33:12.173 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.201+0000 7f3e1c761b80 10 client.20334 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:12.174 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.201+0000 7f3e1c761b80 1 -- 172.21.15.62:0/474153876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55769bf0d250 con 0x55769c1674d0 2023-07-14T16:33:12.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.203+0000 7f3de9ffb700 1 -- 172.21.15.62:0/474153876 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3df4031070 con 0x55769c1674d0 2023-07-14T16:33:12.175 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.203+0000 7f3de9ffb700 1 -- 172.21.15.62:0/474153876 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3df400b990 con 0x55769c1674d0 2023-07-14T16:33:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.204+0000 7f3e1c761b80 10 client.20334 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.204+0000 7f3e1c761b80 10 client.20334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:33:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.204+0000 7f3e1c761b80 20 client.20334 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.204+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55769c1e26d0 0x55769c1e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.204+0000 7f3e1c761b80 4 client.20334 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:33:12.176 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.204+0000 7f3e1c761b80 1 -- 172.21.15.62:0/474153876 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55769c074050 con 0x55769c1e26d0 2023-07-14T16:33:12.177 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.204+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55769c1e26d0 0x55769c1e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.206+0000 7f3df9d9b700 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55769c1e26d0 0x55769c1e4ae0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a21f75307612235c server_cookie=fd9d89ca6d104a51 in_seq=0 out_seq=0 2023-07-14T16:33:12.178 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.206+0000 7f3de9ffb700 10 client.20334 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:33:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.207+0000 7f3de9ffb700 1 -- 172.21.15.62:0/474153876 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55769c074050 con 0x55769c1e26d0 2023-07-14T16:33:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.207+0000 7f3de9ffb700 10 client.20334 handle_command_reply: tid=0 2023-07-14T16:33:12.180 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.208+0000 7f3e1c761b80 1 client.20334 shutdown 2023-07-14T16:33:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.212+0000 7f3e1c761b80 1 -- 172.21.15.62:0/474153876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c1674d0 msgr2=0x55769c1747e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.212+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c1674d0 0x55769c1747e0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b700 tx=0x7f3df4024e00 comp rx=0 tx=0).stop 2023-07-14T16:33:12.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3e1c761b80 1 -- 172.21.15.62:0/474153876 shutdown_connections 2023-07-14T16:33:12.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55769c1e26d0 0x55769c1e4ae0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55769c168c70 0x55769c177ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.185 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55769c1674d0 0x55769c1747e0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3e1c761b80 1 --2- 172.21.15.62:0/474153876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55769c167e20 0x55769c174d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3de9ffb700 0 client.20334 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:33:12.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3e1c761b80 1 -- 172.21.15.62:0/474153876 >> 172.21.15.62:0/474153876 conn(0x55769c15cda0 msgr2=0x55769c16f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.186 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3e1c761b80 1 -- 172.21.15.62:0/474153876 shutdown_connections 2023-07-14T16:33:12.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.213+0000 7f3e1c761b80 1 -- 172.21.15.62:0/474153876 wait complete. 2023-07-14T16:33:12.187 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.214+0000 7f3e1c761b80 20 client.20334 trim_cache size 0 max 16384 2023-07-14T16:33:12.187 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:33:12.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 -- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3df00a11d0 msgr2=0x7f3df00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3df00a11d0 0x7f3df00a3680 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3e0410eaf0 tx=0x7f3dfc009260 comp rx=0 tx=0).stop 2023-07-14T16:33:12.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 -- 172.21.15.62:0/4044782617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e04107390 msgr2=0x7f3e041983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e04107390 0x7f3e041983d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00ec10 tx=0x7f3dec00ef20 comp rx=0 tx=0).stop 2023-07-14T16:33:12.188 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 -- 172.21.15.62:0/4044782617 shutdown_connections 2023-07-14T16:33:12.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e04107d20 0x7f3e0419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e041061a0 0x7f3e04195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3df00a11d0 0x7f3df00a3680 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 --2- 172.21.15.62:0/4044782617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e04107390 0x7f3e041983d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 -- 172.21.15.62:0/4044782617 >> 172.21.15.62:0/4044782617 conn(0x7f3e040751c0 msgr2=0x7f3e04077480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.215+0000 7f3e08dff700 1 -- 172.21.15.62:0/4044782617 shutdown_connections 2023-07-14T16:33:12.189 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.216+0000 7f3e08dff700 1 -- 172.21.15.62:0/4044782617 wait complete. 2023-07-14T16:33:12.216 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:33:12.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:11 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3044411433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:33:12.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.590+0000 7f36e95ff700 1 -- 172.21.15.62:0/1058161892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e40fe9e0 msgr2=0x7f36e40fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.590+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1058161892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e40fe9e0 0x7f36e40fedf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009990 tx=0x7f36d4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.590+0000 7f36e95ff700 1 -- 172.21.15.62:0/1058161892 shutdown_connections 2023-07-14T16:33:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.590+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1058161892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36e4100560 0x7f36e4106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.590+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1058161892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e40fe9e0 0x7f36e40fedf0 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009990 tx=0x7f36d4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.590+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1058161892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36e40ffbd0 0x7f36e4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.590+0000 7f36e95ff700 1 -- 172.21.15.62:0/1058161892 >> 172.21.15.62:0/1058161892 conn(0x7f36e40fa000 msgr2=0x7f36e40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.591+0000 7f36e95ff700 1 -- 172.21.15.62:0/1058161892 shutdown_connections 2023-07-14T16:33:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.591+0000 7f36e95ff700 1 -- 172.21.15.62:0/1058161892 wait complete. 2023-07-14T16:33:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.593+0000 7f36e95ff700 1 Processor -- start 2023-07-14T16:33:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.593+0000 7f36e95ff700 1 -- start start 2023-07-14T16:33:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.593+0000 7f36e95ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36e40ffbd0 0x7f36e410caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.593+0000 7f36e95ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36e4100560 0x7f36e410d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.593+0000 7f36e95ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e41a9f20 0x7f36e41ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.593+0000 7f36e95ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e41ac840 con 0x7f36e4100560 2023-07-14T16:33:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.593+0000 7f36e95ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4069f30 con 0x7f36e40ffbd0 2023-07-14T16:33:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.593+0000 7f36e95ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e406a0f0 con 0x7f36e41a9f20 2023-07-14T16:33:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.594+0000 7f36e37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e41a9f20 0x7f36e41ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.594+0000 7f36e37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e41a9f20 0x7f36e41ac300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47138/0 (socket says 172.21.15.62:47138) 2023-07-14T16:33:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.594+0000 7f36e37fe700 1 -- 172.21.15.62:0/1633572443 learned_addr learned my addr 172.21.15.62:0/1633572443 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.594+0000 7f36e37fe700 1 -- 172.21.15.62:0/1633572443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36e40ffbd0 msgr2=0x7f36e410caf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T16:33:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.594+0000 7f36e37fe700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36e40ffbd0 0x7f36e410caf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.594+0000 7f36e37fe700 1 -- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36e4100560 msgr2=0x7f36e410d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.594+0000 7f36e37fe700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36e4100560 0x7f36e410d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.594+0000 7f36e37fe700 1 -- 172.21.15.62:0/1633572443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40095f0 con 0x7f36e41a9f20 2023-07-14T16:33:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.595+0000 7f36e37fe700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e41a9f20 0x7f36e41ac300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00b970 tx=0x7f36dc00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.595+0000 7f36cbfff700 1 -- 172.21.15.62:0/1633572443 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00c9b0 con 0x7f36e41a9f20 2023-07-14T16:33:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.595+0000 7f36cbfff700 1 -- 172.21.15.62:0/1633572443 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36dc014440 con 0x7f36e41a9f20 2023-07-14T16:33:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.595+0000 7f36cbfff700 1 -- 172.21.15.62:0/1633572443 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0075d0 con 0x7f36e41a9f20 2023-07-14T16:33:12.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.595+0000 7f36e95ff700 1 -- 172.21.15.62:0/1633572443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e406a3d0 con 0x7f36e41a9f20 2023-07-14T16:33:12.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.595+0000 7f36e95ff700 1 -- 172.21.15.62:0/1633572443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e406a910 con 0x7f36e41a9f20 2023-07-14T16:33:12.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.602+0000 7f36fcf91b80 1 Processor -- start 2023-07-14T16:33:12.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.604+0000 7f36cbfff700 1 -- 172.21.15.62:0/1633572443 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f36dc0145b0 con 0x7f36e41a9f20 2023-07-14T16:33:12.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.604+0000 7f36fcf91b80 1 -- start start 2023-07-14T16:33:12.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.606+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb2f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.606+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.606+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 0x55a13bb310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.606+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13ba34cb0 con 0x55a13bb30c70 2023-07-14T16:33:12.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.606+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13ba87a40 con 0x55a13bb2f4d0 2023-07-14T16:33:12.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.606+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13ba80ed0 con 0x55a13bb2fe20 2023-07-14T16:33:12.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.607+0000 7f36e27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb2f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.607+0000 7f36cbfff700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36cc0a1320 0x7f36cc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.607+0000 7f36e27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb2f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51256/0 (socket says 172.21.15.62:51256) 2023-07-14T16:33:12.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.607+0000 7f36e27fc700 1 -- 172.21.15.62:0/1558239465 learned_addr learned my addr 172.21.15.62:0/1558239465 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.607+0000 7f36cbfff700 1 -- 172.21.15.62:0/1633572443 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f36dc006e40 con 0x7f36e41a9f20 2023-07-14T16:33:12.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.607+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.608+0000 7f36e2ffd700 1 -- 172.21.15.62:0/1558239465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 msgr2=0x55a13bb2f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.608+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb2f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.608+0000 7f36e2ffd700 1 -- 172.21.15.62:0/1558239465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 msgr2=0x55a13bb310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.608+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 0x55a13bb310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.608+0000 7f36e2ffd700 1 -- 172.21.15.62:0/1558239465 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a13b8aede0 con 0x55a13bb2fe20 2023-07-14T16:33:12.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.608+0000 7f36e27fc700 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb2f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:12.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.608+0000 7f36e37fe700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36cc0a1320 0x7f36cc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.608+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb306e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f36e40fe520 tx=0x7f36d4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63df2f5e13ee53c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.609+0000 7f36c97fa700 1 -- 172.21.15.62:0/1558239465 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d401d070 con 0x55a13bb2fe20 2023-07-14T16:33:12.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.609+0000 7f36c97fa700 1 -- 172.21.15.62:0/1558239465 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36d400bc60 con 0x55a13bb2fe20 2023-07-14T16:33:12.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.609+0000 7f36c97fa700 1 -- 172.21.15.62:0/1558239465 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d40056b0 con 0x55a13bb2fe20 2023-07-14T16:33:12.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.610+0000 7f36fcf91b80 1 -- 172.21.15.62:0/1558239465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 msgr2=0x55a13bb306e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.610+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb306e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f36e40fe520 tx=0x7f36d4005c00 comp rx=0 tx=0).stop 2023-07-14T16:33:12.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.612+0000 7f36e37fe700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36cc0a1320 0x7f36cc0a37d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0956f0 tx=0x7f36dc004be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.612+0000 7f36fcf91b80 1 -- 172.21.15.62:0/1558239465 shutdown_connections 2023-07-14T16:33:12.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.612+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 0x55a13bb310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.612+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb2f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.612+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/1558239465 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb306e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.612+0000 7f36fcf91b80 1 -- 172.21.15.62:0/1558239465 >> 172.21.15.62:0/1558239465 conn(0x55a13bb24da0 msgr2=0x55a13bb2a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.613+0000 7f36fcf91b80 1 -- 172.21.15.62:0/1558239465 shutdown_connections 2023-07-14T16:33:12.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.613+0000 7f36fcf91b80 1 -- 172.21.15.62:0/1558239465 wait complete. 2023-07-14T16:33:12.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.614+0000 7f36fcf91b80 1 -- start start 2023-07-14T16:33:12.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.614+0000 7f36fcf91b80 1 Processor -- start 2023-07-14T16:33:12.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.615+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.615+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2fe20 0x55a13bb39620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.615+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 0x55a13bb40660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.640 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.615+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13ba20ac0 con 0x55a13bb2fe20 2023-07-14T16:33:12.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.615+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13ba207c0 con 0x55a13bb2f4d0 2023-07-14T16:33:12.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.615+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13b904920 con 0x55a13bb30c70 2023-07-14T16:33:12.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.615+0000 7f36fcf91b80 10 client.0 _ll_register_callbacks cb 0x55a13ba1e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:12.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.616+0000 7f36e2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2fe20 0x55a13bb39620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.616+0000 7f36e2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2fe20 0x55a13bb39620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50816/0 (socket says 172.21.15.62:50816) 2023-07-14T16:33:12.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.616+0000 7f36e2ffd700 1 -- 172.21.15.62:0/4142779602 learned_addr learned my addr 172.21.15.62:0/4142779602 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.617+0000 7f36e2ffd700 1 -- 172.21.15.62:0/4142779602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 msgr2=0x55a13bb40660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:33:12.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.617+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 0x55a13bb40660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.617+0000 7f36e2ffd700 1 -- 172.21.15.62:0/4142779602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 msgr2=0x55a13bb390e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:33:12.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.617+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb390e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.617+0000 7f36e2ffd700 1 -- 172.21.15.62:0/4142779602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e4107750 con 0x55a13bb2fe20 2023-07-14T16:33:12.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.618+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2fe20 0x55a13bb39620 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009ce0 tx=0x7f36d400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931206d7bcb8e4d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.618+0000 7f36c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:12.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.618+0000 7f36c6ff5700 1 -- 172.21.15.62:0/4142779602 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d401d030 con 0x55a13bb2fe20 2023-07-14T16:33:12.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.618+0000 7f36c6ff5700 1 -- 172.21.15.62:0/4142779602 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f36d4003a30 con 0x55a13bb2fe20 2023-07-14T16:33:12.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.618+0000 7f36fcf91b80 1 -- 172.21.15.62:0/4142779602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a13b9da4f0 con 0x55a13bb2fe20 2023-07-14T16:33:12.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.619+0000 7f36c6ff5700 1 -- 172.21.15.62:0/4142779602 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d4003bf0 con 0x55a13bb2fe20 2023-07-14T16:33:12.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.619+0000 7f36fcf91b80 10 client.20340 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:12.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.619+0000 7f36fcf91b80 1 -- 172.21.15.62:0/4142779602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a13b897c60 con 0x55a13bb2fe20 2023-07-14T16:33:12.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.623+0000 7f36c6ff5700 1 -- 172.21.15.62:0/4142779602 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a13b9da4f0 con 0x55a13bb2fe20 2023-07-14T16:33:12.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.623+0000 7f36c6ff5700 1 -- 172.21.15.62:0/4142779602 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f36d400fc10 con 0x55a13bb2fe20 2023-07-14T16:33:12.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.624+0000 7f36fcf91b80 10 client.20340 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:12.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.624+0000 7f36fcf91b80 10 client.20340 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:33:12.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.624+0000 7f36fcf91b80 20 client.20340 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:12.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.624+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55a13bb44a10 0x55a13bb46e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.624+0000 7f36fcf91b80 4 client.20340 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:12.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.624+0000 7f36fcf91b80 1 -- 172.21.15.62:0/4142779602 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a13ba79eb0 con 0x55a13bb44a10 2023-07-14T16:33:12.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.624+0000 7f36e27fc700 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55a13bb44a10 0x55a13bb46e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.629+0000 7f36e27fc700 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55a13bb44a10 0x55a13bb46e20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9d58207d945704e server_cookie=c3451c2b4f677e37 in_seq=0 out_seq=0 2023-07-14T16:33:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.629+0000 7f36c6ff5700 10 client.20340 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:33:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.630+0000 7f36c6ff5700 1 -- 172.21.15.62:0/4142779602 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a13ba79eb0 con 0x55a13bb44a10 2023-07-14T16:33:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.630+0000 7f36c6ff5700 10 client.20340 handle_command_reply: tid=0 2023-07-14T16:33:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.630+0000 7f36fcf91b80 1 client.20340 shutdown 2023-07-14T16:33:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.630+0000 7f36fcf91b80 1 -- 172.21.15.62:0/4142779602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2fe20 msgr2=0x55a13bb39620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.630+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2fe20 0x55a13bb39620 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009ce0 tx=0x7f36d400bef0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.631+0000 7f36fcf91b80 1 -- 172.21.15.62:0/4142779602 shutdown_connections 2023-07-14T16:33:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.631+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55a13bb44a10 0x55a13bb46e20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.631+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 0x55a13bb40660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.631+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb390e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.631+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/4142779602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2fe20 0x55a13bb39620 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.631+0000 7f36fcf91b80 1 -- 172.21.15.62:0/4142779602 >> 172.21.15.62:0/4142779602 conn(0x55a13bb24da0 msgr2=0x55a13bb298b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.633+0000 7f36c6ff5700 0 client.20340 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:33:12.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.633+0000 7f36fcf91b80 1 -- 172.21.15.62:0/4142779602 shutdown_connections 2023-07-14T16:33:12.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.633+0000 7f36fcf91b80 1 -- 172.21.15.62:0/4142779602 wait complete. 2023-07-14T16:33:12.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.634+0000 7f36fcf91b80 20 client.20340 trim_cache size 0 max 16384 2023-07-14T16:33:12.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.660+0000 7f36fcf91b80 1 Processor -- start 2023-07-14T16:33:12.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36fcf91b80 1 -- start start 2023-07-14T16:33:12.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb46a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb40620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 0x55a13bb33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13bb37700 con 0x55a13bb30c70 2023-07-14T16:33:12.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13bb2b050 con 0x55a13bb2f4d0 2023-07-14T16:33:12.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13bb36da0 con 0x55a13bb2fe20 2023-07-14T16:33:12.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36e27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 0x55a13bb33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.661+0000 7f36e27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 0x55a13bb33b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50828/0 (socket says 172.21.15.62:50828) 2023-07-14T16:33:12.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.662+0000 7f36e27fc700 1 -- 172.21.15.62:0/3999766399 learned_addr learned my addr 172.21.15.62:0/3999766399 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.662+0000 7f36e27fc700 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb46a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.662+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb40620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.662+0000 7f36e27fc700 1 -- 172.21.15.62:0/3999766399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 msgr2=0x55a13bb40620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.662+0000 7f36e27fc700 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb40620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.662+0000 7f36e27fc700 1 -- 172.21.15.62:0/3999766399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 msgr2=0x55a13bb33b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.662+0000 7f36e27fc700 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 0x55a13bb33b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.662+0000 7f36e27fc700 1 -- 172.21.15.62:0/3999766399 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a13b8d5250 con 0x55a13bb2f4d0 2023-07-14T16:33:12.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.663+0000 7f36e27fc700 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb46a20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f36d8018500 tx=0x7f36d80058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b9163846046f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.664+0000 7f36c9ffb700 1 -- 172.21.15.62:0/3999766399 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8018530 con 0x55a13bb2f4d0 2023-07-14T16:33:12.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.664+0000 7f36c9ffb700 1 -- 172.21.15.62:0/3999766399 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36d8020020 con 0x55a13bb2f4d0 2023-07-14T16:33:12.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.664+0000 7f36c9ffb700 1 -- 172.21.15.62:0/3999766399 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d801a590 con 0x55a13bb2f4d0 2023-07-14T16:33:12.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.666+0000 7f36fcf91b80 1 -- 172.21.15.62:0/3999766399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 msgr2=0x55a13bb46a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.666+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb46a20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f36d8018500 tx=0x7f36d80058e0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.666+0000 7f36fcf91b80 1 -- 172.21.15.62:0/3999766399 shutdown_connections 2023-07-14T16:33:12.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.666+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb30c70 0x55a13bb33b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.666+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2f4d0 0x55a13bb46a20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.666+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/3999766399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb2fe20 0x55a13bb40620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.666+0000 7f36fcf91b80 1 -- 172.21.15.62:0/3999766399 >> 172.21.15.62:0/3999766399 conn(0x55a13bb24da0 msgr2=0x55a13bb36ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.667+0000 7f36fcf91b80 1 -- 172.21.15.62:0/3999766399 shutdown_connections 2023-07-14T16:33:12.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.667+0000 7f36fcf91b80 1 -- 172.21.15.62:0/3999766399 wait complete. 2023-07-14T16:33:12.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.667+0000 7f36fcf91b80 1 -- start start 2023-07-14T16:33:12.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.668+0000 7f36fcf91b80 1 Processor -- start 2023-07-14T16:33:12.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.668+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2f4d0 0x55a13bb3feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.668+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2fe20 0x55a13bb403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.654 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.668+0000 7f36fcf91b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 0x55a13bb3c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.668+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13bb3cf70 con 0x55a13bb2f4d0 2023-07-14T16:33:12.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.668+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13bb3d0b0 con 0x55a13bb2fe20 2023-07-14T16:33:12.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.668+0000 7f36fcf91b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a13bb3d270 con 0x55a13bb30c70 2023-07-14T16:33:12.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.668+0000 7f36fcf91b80 10 client.0 _ll_register_callbacks cb 0x7f36d4009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:12.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2f4d0 0x55a13bb3feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2f4d0 0x55a13bb3feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50832/0 (socket says 172.21.15.62:50832) 2023-07-14T16:33:12.655 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 -- 172.21.15.62:0/616720797 learned_addr learned my addr 172.21.15.62:0/616720797 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:12.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2fe20 0x55a13bb403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 0x55a13bb3c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 -- 172.21.15.62:0/616720797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 msgr2=0x55a13bb3c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 0x55a13bb3c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 -- 172.21.15.62:0/616720797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2fe20 msgr2=0x55a13bb403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:12.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2fe20 0x55a13bb403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e27fc700 1 -- 172.21.15.62:0/616720797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d80181e0 con 0x55a13bb2f4d0 2023-07-14T16:33:12.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.669+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2fe20 0x55a13bb403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:12.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.670+0000 7f36e27fc700 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2f4d0 0x55a13bb3feb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f36d80185d0 tx=0x7f36d8018f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdd74fba8e3c82 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:12.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.670+0000 7f36c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:12.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.670+0000 7f36c6ff5700 1 -- 172.21.15.62:0/616720797 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d801a510 con 0x55a13bb2f4d0 2023-07-14T16:33:12.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.670+0000 7f36c6ff5700 1 -- 172.21.15.62:0/616720797 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f36d8021070 con 0x55a13bb2f4d0 2023-07-14T16:33:12.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.670+0000 7f36fcf91b80 1 -- 172.21.15.62:0/616720797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36d400bc60 con 0x55a13bb2f4d0 2023-07-14T16:33:12.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.671+0000 7f36c6ff5700 1 -- 172.21.15.62:0/616720797 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d800aa10 con 0x55a13bb2f4d0 2023-07-14T16:33:12.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.671+0000 7f36fcf91b80 10 client.20346 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:12.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.671+0000 7f36fcf91b80 1 -- 172.21.15.62:0/616720797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a13b8ad2b0 con 0x55a13bb2f4d0 2023-07-14T16:33:12.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.671+0000 7f36c6ff5700 1 -- 172.21.15.62:0/616720797 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36d400bc60 con 0x55a13bb2f4d0 2023-07-14T16:33:12.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.671+0000 7f36c6ff5700 1 -- 172.21.15.62:0/616720797 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f36d8005130 con 0x55a13bb2f4d0 2023-07-14T16:33:12.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.672+0000 7f36fcf91b80 10 client.20346 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:12.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.672+0000 7f36fcf91b80 10 client.20346 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:33:12.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.672+0000 7f36fcf91b80 20 client.20346 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:12.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.672+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55a13bb99d10 0x55a13bb9c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:12.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.672+0000 7f36fcf91b80 4 client.20346 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:33:12.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.672+0000 7f36fcf91b80 1 -- 172.21.15.62:0/616720797 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a13ba3c050 con 0x55a13bb99d10 2023-07-14T16:33:12.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.672+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55a13bb99d10 0x55a13bb9c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:12.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.676+0000 7f36e2ffd700 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55a13bb99d10 0x55a13bb9c120 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cec5990ee28cee3 server_cookie=ae612e4e22cc0c9b in_seq=0 out_seq=0 2023-07-14T16:33:12.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.676+0000 7f36c6ff5700 10 client.20346 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:33:12.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.676+0000 7f36c6ff5700 1 -- 172.21.15.62:0/616720797 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a13ba3c050 con 0x55a13bb99d10 2023-07-14T16:33:12.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.676+0000 7f36c6ff5700 10 client.20346 handle_command_reply: tid=0 2023-07-14T16:33:12.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.676+0000 7f36fcf91b80 1 client.20346 shutdown 2023-07-14T16:33:12.659 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36fcf91b80 1 -- 172.21.15.62:0/616720797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2f4d0 msgr2=0x55a13bb3feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2f4d0 0x55a13bb3feb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f36d80185d0 tx=0x7f36d8018f10 comp rx=0 tx=0).stop 2023-07-14T16:33:12.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36fcf91b80 1 -- 172.21.15.62:0/616720797 shutdown_connections 2023-07-14T16:33:12.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x55a13bb99d10 0x55a13bb9c120 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a13bb30c70 0x55a13bb3c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a13bb2f4d0 0x55a13bb3feb0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36fcf91b80 1 --2- 172.21.15.62:0/616720797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a13bb2fe20 0x55a13bb403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.660 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36fcf91b80 1 -- 172.21.15.62:0/616720797 >> 172.21.15.62:0/616720797 conn(0x55a13bb24da0 msgr2=0x55a13bb46a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.661 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.677+0000 7f36c6ff5700 0 client.20346 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:33:12.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.678+0000 7f36fcf91b80 1 -- 172.21.15.62:0/616720797 shutdown_connections 2023-07-14T16:33:12.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.678+0000 7f36fcf91b80 1 -- 172.21.15.62:0/616720797 wait complete. 2023-07-14T16:33:12.662 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.678+0000 7f36fcf91b80 20 client.20346 trim_cache size 0 max 16384 2023-07-14T16:33:12.662 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:33:12.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 -- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36cc0a1320 msgr2=0x7f36cc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36cc0a1320 0x7f36cc0a37d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0956f0 tx=0x7f36dc004be0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 -- 172.21.15.62:0/1633572443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e41a9f20 msgr2=0x7f36e41ac300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:12.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e41a9f20 0x7f36e41ac300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00b970 tx=0x7f36dc00bd30 comp rx=0 tx=0).stop 2023-07-14T16:33:12.663 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 -- 172.21.15.62:0/1633572443 shutdown_connections 2023-07-14T16:33:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f36e41a9f20 0x7f36e41ac300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f36cc0a1320 0x7f36cc0a37d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36e40ffbd0 0x7f36e410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 --2- 172.21.15.62:0/1633572443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36e4100560 0x7f36e410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.679+0000 7f36e95ff700 1 -- 172.21.15.62:0/1633572443 >> 172.21.15.62:0/1633572443 conn(0x7f36e40fa000 msgr2=0x7f36e4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.680+0000 7f36e95ff700 1 -- 172.21.15.62:0/1633572443 shutdown_connections 2023-07-14T16:33:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:12.680+0000 7f36e95ff700 1 -- 172.21.15.62:0/1633572443 wait complete. 2023-07-14T16:33:12.680 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:33:13.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.074+0000 7f863cdff700 1 -- 172.21.15.62:0/3441944730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 msgr2=0x7f8638071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.074+0000 7f863cdff700 1 --2- 172.21.15.62:0/3441944730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 0x7f8638071b00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f86280074e0 tx=0x7f862800af90 comp rx=0 tx=0).stop 2023-07-14T16:33:13.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.075+0000 7f863cdff700 1 -- 172.21.15.62:0/3441944730 shutdown_connections 2023-07-14T16:33:13.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.075+0000 7f863cdff700 1 --2- 172.21.15.62:0/3441944730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86381080d0 0x7f863810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.075+0000 7f863cdff700 1 --2- 172.21.15.62:0/3441944730 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 0x7f8638071b00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.075+0000 7f863cdff700 1 --2- 172.21.15.62:0/3441944730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8638107720 0x7f8638107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.075+0000 7f863cdff700 1 -- 172.21.15.62:0/3441944730 >> 172.21.15.62:0/3441944730 conn(0x7f863806cd80 msgr2=0x7f863806f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:13.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.076+0000 7f863cdff700 1 -- 172.21.15.62:0/3441944730 shutdown_connections 2023-07-14T16:33:13.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.076+0000 7f863cdff700 1 -- 172.21.15.62:0/3441944730 wait complete. 2023-07-14T16:33:13.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.078+0000 7f863cdff700 1 Processor -- start 2023-07-14T16:33:13.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.078+0000 7f863cdff700 1 -- start start 2023-07-14T16:33:13.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.078+0000 7f863cdff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 0x7f86381170e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.078+0000 7f863cdff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8638107720 0x7f8638117620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.078+0000 7f863cdff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86381080d0 0x7f863811e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.078+0000 7f863cdff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863811ec60 con 0x7f8638107720 2023-07-14T16:33:13.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.078+0000 7f863cdff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863811edd0 con 0x7f86380716f0 2023-07-14T16:33:13.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.078+0000 7f863cdff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863811ef90 con 0x7f86381080d0 2023-07-14T16:33:13.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.079+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8638107720 0x7f8638117620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.079+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8638107720 0x7f8638117620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39368/0 (socket says 172.21.15.62:39368) 2023-07-14T16:33:13.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.079+0000 7f8635d9b700 1 -- 172.21.15.62:0/1858296611 learned_addr learned my addr 172.21.15.62:0/1858296611 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.079+0000 7f863659c700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 0x7f86381170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.079+0000 7f8636d9d700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86381080d0 0x7f863811e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.080+0000 7f8635d9b700 1 -- 172.21.15.62:0/1858296611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86381080d0 msgr2=0x7f863811e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.080+0000 7f8635d9b700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86381080d0 0x7f863811e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.080+0000 7f8635d9b700 1 -- 172.21.15.62:0/1858296611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 msgr2=0x7f86381170e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.080+0000 7f8635d9b700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 0x7f86381170e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.080+0000 7f8635d9b700 1 -- 172.21.15.62:0/1858296611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280071c0 con 0x7f8638107720 2023-07-14T16:33:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.080+0000 7f863659c700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 0x7f86381170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.080+0000 7f8635d9b700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8638107720 0x7f8638117620 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f863000b840 tx=0x7f863000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.081+0000 7f86277fe700 1 -- 172.21.15.62:0/1858296611 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000f840 con 0x7f8638107720 2023-07-14T16:33:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.081+0000 7f86277fe700 1 -- 172.21.15.62:0/1858296611 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f863000fe30 con 0x7f8638107720 2023-07-14T16:33:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.081+0000 7f86277fe700 1 -- 172.21.15.62:0/1858296611 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000d3d0 con 0x7f8638107720 2023-07-14T16:33:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.082+0000 7f863cdff700 1 -- 172.21.15.62:0/1858296611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863811f210 con 0x7f8638107720 2023-07-14T16:33:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.083+0000 7f863cdff700 1 -- 172.21.15.62:0/1858296611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86381b55d0 con 0x7f8638107720 2023-07-14T16:33:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.089+0000 7f8650739b80 1 Processor -- start 2023-07-14T16:33:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.089+0000 7f86277fe700 1 -- 172.21.15.62:0/1858296611 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f863000f9a0 con 0x7f8638107720 2023-07-14T16:33:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.090+0000 7f8650739b80 1 -- start start 2023-07-14T16:33:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8650739b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0214d0 0x55b8fe0218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8650739b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe021e20 0x55b8fe0226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8650739b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe022c70 0x55b8fe0230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8650739b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fdf26cb0 con 0x55b8fe022c70 2023-07-14T16:33:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8650739b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fdf79a40 con 0x55b8fe0214d0 2023-07-14T16:33:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8650739b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fdf72ed0 con 0x55b8fe021e20 2023-07-14T16:33:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8636d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0214d0 0x55b8fe0218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8636d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0214d0 0x55b8fe0218e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35052/0 (socket says 172.21.15.62:35052) 2023-07-14T16:33:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.091+0000 7f8636d9d700 1 -- 172.21.15.62:0/2275621838 learned_addr learned my addr 172.21.15.62:0/2275621838 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.093+0000 7f863659c700 1 --2- 172.21.15.62:0/2275621838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe021e20 0x55b8fe0226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.094+0000 7f863659c700 1 -- 172.21.15.62:0/2275621838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0214d0 msgr2=0x55b8fe0218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.094+0000 7f863659c700 1 --2- 172.21.15.62:0/2275621838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0214d0 0x55b8fe0218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.094+0000 7f863659c700 1 -- 172.21.15.62:0/2275621838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe022c70 msgr2=0x55b8fe0230e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.094+0000 7f863659c700 1 --2- 172.21.15.62:0/2275621838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe022c70 0x55b8fe0230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.094+0000 7f863659c700 1 -- 172.21.15.62:0/2275621838 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8fdda0de0 con 0x55b8fe021e20 2023-07-14T16:33:13.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.095+0000 7f86277fe700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86200a1350 0x7f86200a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.095+0000 7f86277fe700 1 -- 172.21.15.62:0/1858296611 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86300cc760 con 0x7f8638107720 2023-07-14T16:33:13.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.097+0000 7f863659c700 1 --2- 172.21.15.62:0/2275621838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe021e20 0x55b8fe0226e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8628016040 tx=0x7f8628003700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbcc9f677ebad83e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:13.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.099+0000 7f8636d9d700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86200a1350 0x7f86200a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.100+0000 7f8636d9d700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86200a1350 0x7f86200a3800 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f862c005fd0 tx=0x7f862c00a310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:13.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.103+0000 7f8624ff9700 1 -- 172.21.15.62:0/2275621838 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800f070 con 0x55b8fe021e20 2023-07-14T16:33:13.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.103+0000 7f8624ff9700 1 -- 172.21.15.62:0/2275621838 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86280056d0 con 0x55b8fe021e20 2023-07-14T16:33:13.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.103+0000 7f8624ff9700 1 -- 172.21.15.62:0/2275621838 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628004630 con 0x55b8fe021e20 2023-07-14T16:33:13.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.105+0000 7f8650739b80 1 -- 172.21.15.62:0/2275621838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe021e20 msgr2=0x55b8fe0226e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.105+0000 7f8650739b80 1 --2- 172.21.15.62:0/2275621838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe021e20 0x55b8fe0226e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8628016040 tx=0x7f8628003700 comp rx=0 tx=0).stop 2023-07-14T16:33:13.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.105+0000 7f8650739b80 1 -- 172.21.15.62:0/2275621838 shutdown_connections 2023-07-14T16:33:13.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.105+0000 7f8650739b80 1 --2- 172.21.15.62:0/2275621838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe022c70 0x55b8fe0230e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.105+0000 7f8650739b80 1 --2- 172.21.15.62:0/2275621838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0214d0 0x55b8fe0218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.105+0000 7f8650739b80 1 --2- 172.21.15.62:0/2275621838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe021e20 0x55b8fe0226e0 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8628016040 tx=0x7f8628003700 comp rx=0 tx=0).stop 2023-07-14T16:33:13.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.105+0000 7f8650739b80 1 -- 172.21.15.62:0/2275621838 >> 172.21.15.62:0/2275621838 conn(0x55b8fe016da0 msgr2=0x55b8fe01c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:13.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.106+0000 7f8650739b80 1 -- 172.21.15.62:0/2275621838 shutdown_connections 2023-07-14T16:33:13.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.106+0000 7f8650739b80 1 -- 172.21.15.62:0/2275621838 wait complete. 2023-07-14T16:33:13.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.107+0000 7f8650739b80 1 -- start start 2023-07-14T16:33:13.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.107+0000 7f8650739b80 1 Processor -- start 2023-07-14T16:33:13.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8650739b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe02b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8650739b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe02b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8650739b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 0x55b8fe032cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8650739b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fdf12ac0 con 0x55b8fe0214d0 2023-07-14T16:33:13.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8650739b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fdf127c0 con 0x55b8fe0308e0 2023-07-14T16:33:13.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8650739b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fddf6920 con 0x55b8fe022c70 2023-07-14T16:33:13.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8650739b80 10 client.0 _ll_register_callbacks cb 0x55b8fdf10760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:13.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe02b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe02b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56290/0 (socket says 172.21.15.62:56290) 2023-07-14T16:33:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f8635d9b700 1 -- 172.21.15.62:0/1159620749 learned_addr learned my addr 172.21.15.62:0/1159620749 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.108+0000 7f863659c700 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe02b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f8636d9d700 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 0x55b8fe032cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f863659c700 1 -- 172.21.15.62:0/1159620749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 msgr2=0x55b8fe02b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f863659c700 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe02b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f863659c700 1 -- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 msgr2=0x55b8fe032cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f863659c700 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 0x55b8fe032cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f863659c700 1 -- 172.21.15.62:0/1159620749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863810f260 con 0x55b8fe0214d0 2023-07-14T16:33:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f8636d9d700 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 0x55b8fe032cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:33:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f863659c700 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe02b370 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f8628014000 tx=0x7f862800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be47f9fed5be9115 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.109+0000 7f861dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:13.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.110+0000 7f861dffb700 1 -- 172.21.15.62:0/1159620749 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800f040 con 0x55b8fe0214d0 2023-07-14T16:33:13.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.110+0000 7f861dffb700 1 -- 172.21.15.62:0/1159620749 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8628005450 con 0x55b8fe0214d0 2023-07-14T16:33:13.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.110+0000 7f8650739b80 1 -- 172.21.15.62:0/1159620749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8fdecc4f0 con 0x55b8fe0214d0 2023-07-14T16:33:13.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.110+0000 7f861dffb700 1 -- 172.21.15.62:0/1159620749 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8628004630 con 0x55b8fe0214d0 2023-07-14T16:33:13.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.111+0000 7f8650739b80 10 client.20352 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.111+0000 7f8650739b80 1 -- 172.21.15.62:0/1159620749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b8fdd89c60 con 0x55b8fe0214d0 2023-07-14T16:33:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.111+0000 7f861dffb700 1 -- 172.21.15.62:0/1159620749 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b8fdecc4f0 con 0x55b8fe0214d0 2023-07-14T16:33:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.112+0000 7f861dffb700 1 -- 172.21.15.62:0/1159620749 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8628023070 con 0x55b8fe0214d0 2023-07-14T16:33:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.112+0000 7f8650739b80 10 client.20352 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.112+0000 7f8650739b80 10 client.20352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.112+0000 7f8650739b80 20 client.20352 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.112+0000 7f8650739b80 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b8fe036f40 0x55b8fe039320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.112+0000 7f8650739b80 4 client.20352 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.112+0000 7f8650739b80 1 -- 172.21.15.62:0/1159620749 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8fdf6beb0 con 0x55b8fe036f40 2023-07-14T16:33:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.112+0000 7f8635d9b700 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b8fe036f40 0x55b8fe039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.113+0000 7f8635d9b700 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b8fe036f40 0x55b8fe039320 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0c20d6a4514cb0 server_cookie=e84167961aed73bb in_seq=0 out_seq=0 2023-07-14T16:33:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.113+0000 7f861dffb700 10 client.20352 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:13.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.114+0000 7f861dffb700 1 -- 172.21.15.62:0/1159620749 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b8fdf6beb0 con 0x55b8fe036f40 2023-07-14T16:33:13.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.114+0000 7f861dffb700 10 client.20352 handle_command_reply: tid=0 2023-07-14T16:33:13.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.114+0000 7f8650739b80 1 client.20352 shutdown 2023-07-14T16:33:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.114+0000 7f8650739b80 1 -- 172.21.15.62:0/1159620749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 msgr2=0x55b8fe02b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.114+0000 7f8650739b80 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe02b370 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f8628014000 tx=0x7f862800cad0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f8650739b80 1 -- 172.21.15.62:0/1159620749 shutdown_connections 2023-07-14T16:33:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f8650739b80 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b8fe036f40 0x55b8fe039320 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f8650739b80 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 0x55b8fe032cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f8650739b80 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe02b370 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f8650739b80 1 --2- 172.21.15.62:0/1159620749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe02b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f8650739b80 1 -- 172.21.15.62:0/1159620749 >> 172.21.15.62:0/1159620749 conn(0x55b8fe016da0 msgr2=0x55b8fe01c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:13.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f861dffb700 0 client.20352 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:13.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f8650739b80 1 -- 172.21.15.62:0/1159620749 shutdown_connections 2023-07-14T16:33:13.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.115+0000 7f8650739b80 1 -- 172.21.15.62:0/1159620749 wait complete. 2023-07-14T16:33:13.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.116+0000 7f8650739b80 20 client.20352 trim_cache size 0 max 16384 2023-07-14T16:33:13.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.140+0000 7f8650739b80 1 Processor -- start 2023-07-14T16:33:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.140+0000 7f8650739b80 1 -- start start 2023-07-14T16:33:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.140+0000 7f8650739b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe046b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.140+0000 7f8650739b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe022c70 0x55b8fe022460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.140+0000 7f8650739b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe0308e0 0x55b8fe0335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.140+0000 7f8650739b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fe0169e0 con 0x55b8fe0214d0 2023-07-14T16:33:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.140+0000 7f8650739b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fdf6beb0 con 0x55b8fe022c70 2023-07-14T16:33:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.140+0000 7f8650739b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fdecc4f0 con 0x55b8fe0308e0 2023-07-14T16:33:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.141+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe022c70 0x55b8fe022460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.141+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe022c70 0x55b8fe022460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35080/0 (socket says 172.21.15.62:35080) 2023-07-14T16:33:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.141+0000 7f8635d9b700 1 -- 172.21.15.62:0/3971387829 learned_addr learned my addr 172.21.15.62:0/3971387829 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.141+0000 7f8636d9d700 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe0308e0 0x55b8fe0335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.141+0000 7f863659c700 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe046b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.142+0000 7f8636d9d700 1 -- 172.21.15.62:0/3971387829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe022c70 msgr2=0x55b8fe022460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.142+0000 7f8636d9d700 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe022c70 0x55b8fe022460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.142+0000 7f8636d9d700 1 -- 172.21.15.62:0/3971387829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 msgr2=0x55b8fe046b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.142+0000 7f8636d9d700 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe046b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.142+0000 7f8636d9d700 1 -- 172.21.15.62:0/3971387829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8fddc7250 con 0x55b8fe0308e0 2023-07-14T16:33:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.142+0000 7f8636d9d700 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe0308e0 0x55b8fe0335e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55b8fe02c420 tx=0x7f862c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=733f2e1f24def06b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.143+0000 7f86257fa700 1 -- 172.21.15.62:0/3971387829 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c011d80 con 0x55b8fe0308e0 2023-07-14T16:33:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.143+0000 7f86257fa700 1 -- 172.21.15.62:0/3971387829 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f862c015070 con 0x55b8fe0308e0 2023-07-14T16:33:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.143+0000 7f86257fa700 1 -- 172.21.15.62:0/3971387829 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c01b670 con 0x55b8fe0308e0 2023-07-14T16:33:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 -- 172.21.15.62:0/3971387829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe0308e0 msgr2=0x55b8fe0335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe0308e0 0x55b8fe0335e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55b8fe02c420 tx=0x7f862c013000 comp rx=0 tx=0).stop 2023-07-14T16:33:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 -- 172.21.15.62:0/3971387829 shutdown_connections 2023-07-14T16:33:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe0308e0 0x55b8fe0335e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe046b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 --2- 172.21.15.62:0/3971387829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe022c70 0x55b8fe022460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 -- 172.21.15.62:0/3971387829 >> 172.21.15.62:0/3971387829 conn(0x55b8fe016da0 msgr2=0x55b8fe0292e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 -- 172.21.15.62:0/3971387829 shutdown_connections 2023-07-14T16:33:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.144+0000 7f8650739b80 1 -- 172.21.15.62:0/3971387829 wait complete. 2023-07-14T16:33:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.145+0000 7f8650739b80 1 -- start start 2023-07-14T16:33:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.145+0000 7f8650739b80 1 Processor -- start 2023-07-14T16:33:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.145+0000 7f8650739b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe0445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8650739b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe044af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8650739b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 0x55b8fe045ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8650739b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fe045ff0 con 0x55b8fe0214d0 2023-07-14T16:33:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8650739b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fe046160 con 0x55b8fe0308e0 2023-07-14T16:33:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8650739b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fe046320 con 0x55b8fe022c70 2023-07-14T16:33:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8650739b80 10 client.0 _ll_register_callbacks cb 0x7f8628016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe044af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8635d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe044af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56312/0 (socket says 172.21.15.62:56312) 2023-07-14T16:33:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.146+0000 7f8635d9b700 1 -- 172.21.15.62:0/3873504877 learned_addr learned my addr 172.21.15.62:0/3873504877 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.147+0000 7f863659c700 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe0445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.147+0000 7f863659c700 1 -- 172.21.15.62:0/3873504877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 msgr2=0x55b8fe044af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.147+0000 7f863659c700 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe044af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.147+0000 7f863659c700 1 -- 172.21.15.62:0/3873504877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 msgr2=0x55b8fe045ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:33:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.147+0000 7f863659c700 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 0x55b8fe045ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.147+0000 7f863659c700 1 -- 172.21.15.62:0/3873504877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c010c50 con 0x55b8fe0214d0 2023-07-14T16:33:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.147+0000 7f863659c700 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe0445b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f8628003900 tx=0x7f862800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e25e68afbef1bf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.148+0000 7f861dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.148+0000 7f861dffb700 1 -- 172.21.15.62:0/3873504877 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800f070 con 0x55b8fe0214d0 2023-07-14T16:33:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.148+0000 7f861dffb700 1 -- 172.21.15.62:0/3873504877 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8628025430 con 0x55b8fe0214d0 2023-07-14T16:33:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.149+0000 7f8650739b80 1 -- 172.21.15.62:0/3873504877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86280056d0 con 0x55b8fe0214d0 2023-07-14T16:33:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.149+0000 7f861dffb700 1 -- 172.21.15.62:0/3873504877 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8628004180 con 0x55b8fe0214d0 2023-07-14T16:33:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.149+0000 7f8650739b80 10 client.20358 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.149+0000 7f8650739b80 1 -- 172.21.15.62:0/3873504877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b8fdd9f2b0 con 0x55b8fe0214d0 2023-07-14T16:33:13.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:12 smithi062 ceph-mon[95543]: pgmap v884: 97 pgs: 97 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.57k op/s 2023-07-14T16:33:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.150+0000 7f861dffb700 1 -- 172.21.15.62:0/3873504877 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86280056d0 con 0x55b8fe0214d0 2023-07-14T16:33:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.150+0000 7f861dffb700 1 -- 172.21.15.62:0/3873504877 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8628007510 con 0x55b8fe0214d0 2023-07-14T16:33:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.150+0000 7f8650739b80 10 client.20358 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.150+0000 7f8650739b80 10 client.20358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.150+0000 7f8650739b80 20 client.20358 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.150+0000 7f8650739b80 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b8fe08b960 0x55b8fe08dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.150+0000 7f8650739b80 4 client.20358 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:33:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.150+0000 7f8650739b80 1 -- 172.21.15.62:0/3873504877 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b8fdf2e050 con 0x55b8fe08b960 2023-07-14T16:33:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.151+0000 7f8635d9b700 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b8fe08b960 0x55b8fe08dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.153+0000 7f8635d9b700 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b8fe08b960 0x55b8fe08dd70 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48a70fba7d873b server_cookie=e0b421a2214b5774 in_seq=0 out_seq=0 2023-07-14T16:33:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.153+0000 7f861dffb700 10 client.20358 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.153+0000 7f861dffb700 1 -- 172.21.15.62:0/3873504877 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b8fdf2e050 con 0x55b8fe08b960 2023-07-14T16:33:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.153+0000 7f861dffb700 10 client.20358 handle_command_reply: tid=0 2023-07-14T16:33:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.153+0000 7f8650739b80 1 client.20358 shutdown 2023-07-14T16:33:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f8650739b80 1 -- 172.21.15.62:0/3873504877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 msgr2=0x55b8fe0445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f8650739b80 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe0445b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f8628003900 tx=0x7f862800ca40 comp rx=0 tx=0).stop 2023-07-14T16:33:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f8650739b80 1 -- 172.21.15.62:0/3873504877 shutdown_connections 2023-07-14T16:33:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f8650739b80 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b8fe08b960 0x55b8fe08dd70 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f8650739b80 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8fe0308e0 0x55b8fe045ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f8650739b80 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8fe0214d0 0x55b8fe0445b0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f8650739b80 1 --2- 172.21.15.62:0/3873504877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b8fe022c70 0x55b8fe044af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f8650739b80 1 -- 172.21.15.62:0/3873504877 >> 172.21.15.62:0/3873504877 conn(0x55b8fe016da0 msgr2=0x55b8fe022460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.154+0000 7f861dffb700 0 client.20358 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.155+0000 7f8650739b80 1 -- 172.21.15.62:0/3873504877 shutdown_connections 2023-07-14T16:33:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.155+0000 7f8650739b80 1 -- 172.21.15.62:0/3873504877 wait complete. 2023-07-14T16:33:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.155+0000 7f8650739b80 20 client.20358 trim_cache size 0 max 16384 2023-07-14T16:33:13.129 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:33:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.156+0000 7f863cdff700 1 -- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86200a1350 msgr2=0x7f86200a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.156+0000 7f863cdff700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86200a1350 0x7f86200a3800 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f862c005fd0 tx=0x7f862c00a310 comp rx=0 tx=0).stop 2023-07-14T16:33:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.156+0000 7f863cdff700 1 -- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8638107720 msgr2=0x7f8638117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.156+0000 7f863cdff700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8638107720 0x7f8638117620 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f863000b840 tx=0x7f863000bb50 comp rx=0 tx=0).stop 2023-07-14T16:33:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.157+0000 7f863cdff700 1 -- 172.21.15.62:0/1858296611 shutdown_connections 2023-07-14T16:33:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.157+0000 7f863cdff700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86381080d0 0x7f863811e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.157+0000 7f863cdff700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f86200a1350 0x7f86200a3800 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.157+0000 7f863cdff700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86380716f0 0x7f86381170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.157+0000 7f863cdff700 1 --2- 172.21.15.62:0/1858296611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8638107720 0x7f8638117620 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.157+0000 7f863cdff700 1 -- 172.21.15.62:0/1858296611 >> 172.21.15.62:0/1858296611 conn(0x7f863806cd80 msgr2=0x7f863806e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.158+0000 7f863cdff700 1 -- 172.21.15.62:0/1858296611 shutdown_connections 2023-07-14T16:33:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:13.158+0000 7f863cdff700 1 -- 172.21.15.62:0/1858296611 wait complete. 2023-07-14T16:33:13.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:12 smithi164 ceph-mon[97427]: pgmap v884: 97 pgs: 97 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.57k op/s 2023-07-14T16:33:13.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:12 smithi081 ceph-mon[127795]: pgmap v884: 97 pgs: 97 active+clean; 13 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.57k op/s 2023-07-14T16:33:14.158 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:33:14.158 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:33:14.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 -- 172.21.15.62:0/1769861481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a4074550 msgr2=0x7f50a40729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/1769861481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a4074550 0x7f50a40729a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f5094009990 tx=0x7f5094009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 -- 172.21.15.62:0/1769861481 shutdown_connections 2023-07-14T16:33:14.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/1769861481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a410c0b0 0x7f50a410e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/1769861481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a4074550 0x7f50a40729a0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/1769861481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a4072f70 0x7f50a40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 -- 172.21.15.62:0/1769861481 >> 172.21.15.62:0/1769861481 conn(0x7f50a40fb0e0 msgr2=0x7f50a40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:14.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 -- 172.21.15.62:0/1769861481 shutdown_connections 2023-07-14T16:33:14.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.540+0000 7f50aa5ff700 1 -- 172.21.15.62:0/1769861481 wait complete. 2023-07-14T16:33:14.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.542+0000 7f50aa5ff700 1 Processor -- start 2023-07-14T16:33:14.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.542+0000 7f50aa5ff700 1 -- start start 2023-07-14T16:33:14.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.542+0000 7f50aa5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a4072f70 0x7f50a419be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.542+0000 7f50aa5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a4074550 0x7f50a419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.542+0000 7f50aa5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a410c0b0 0x7f50a41a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.542+0000 7f50aa5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41a3980 con 0x7f50a4074550 2023-07-14T16:33:14.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.542+0000 7f50aa5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41a3ac0 con 0x7f50a4072f70 2023-07-14T16:33:14.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.542+0000 7f50aa5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41a3c50 con 0x7f50a410c0b0 2023-07-14T16:33:14.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.543+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a4072f70 0x7f50a419be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.543+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a4072f70 0x7f50a419be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35100/0 (socket says 172.21.15.62:35100) 2023-07-14T16:33:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.543+0000 7f50a3fff700 1 -- 172.21.15.62:0/2223784736 learned_addr learned my addr 172.21.15.62:0/2223784736 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.543+0000 7f50a37fe700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a4074550 0x7f50a419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.543+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a410c0b0 0x7f50a41a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.544+0000 7f50a3fff700 1 -- 172.21.15.62:0/2223784736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a410c0b0 msgr2=0x7f50a41a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.544+0000 7f50a3fff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a410c0b0 0x7f50a41a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.544+0000 7f50a3fff700 1 -- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a4074550 msgr2=0x7f50a419c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.544+0000 7f50a3fff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a4074550 0x7f50a419c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.544+0000 7f50a3fff700 1 -- 172.21.15.62:0/2223784736 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50940095f0 con 0x7f50a4072f70 2023-07-14T16:33:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.544+0000 7f50a3fff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a4072f70 0x7f50a419be00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f50940098e0 tx=0x7f5094005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.546+0000 7f50a17fa700 1 -- 172.21.15.62:0/2223784736 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509401d070 con 0x7f50a4072f70 2023-07-14T16:33:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.546+0000 7f50a17fa700 1 -- 172.21.15.62:0/2223784736 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f509400f460 con 0x7f50a4072f70 2023-07-14T16:33:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.546+0000 7f50a17fa700 1 -- 172.21.15.62:0/2223784736 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094017650 con 0x7f50a4072f70 2023-07-14T16:33:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.546+0000 7f50aa5ff700 1 -- 172.21.15.62:0/2223784736 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a41a3ed0 con 0x7f50a4072f70 2023-07-14T16:33:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.546+0000 7f50aa5ff700 1 -- 172.21.15.62:0/2223784736 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a41a4240 con 0x7f50a4072f70 2023-07-14T16:33:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.555+0000 7f50bdf15b80 1 Processor -- start 2023-07-14T16:33:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.556+0000 7f50bdf15b80 1 -- start start 2023-07-14T16:33:14.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.556+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd424960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.556+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd425760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.556+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd425cf0 0x55e8bd426160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.556+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd329d30 con 0x55e8bd424550 2023-07-14T16:33:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.556+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd37cac0 con 0x55e8bd424ea0 2023-07-14T16:33:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.556+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd375f50 con 0x55e8bd425cf0 2023-07-14T16:33:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.557+0000 7f50a17fa700 1 -- 172.21.15.62:0/2223784736 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f50940177b0 con 0x7f50a4072f70 2023-07-14T16:33:14.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.557+0000 7f50a37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd425cf0 0x55e8bd426160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.557+0000 7f50a37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd425cf0 0x55e8bd426160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56338/0 (socket says 172.21.15.62:56338) 2023-07-14T16:33:14.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.557+0000 7f50a37fe700 1 -- 172.21.15.62:0/3331680869 learned_addr learned my addr 172.21.15.62:0/3331680869 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.557+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/3331680869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd424960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.558+0000 7f50a37fe700 1 -- 172.21.15.62:0/3331680869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 msgr2=0x55e8bd425760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:33:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.558+0000 7f50a37fe700 1 --2- 172.21.15.62:0/3331680869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd425760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.558+0000 7f50a37fe700 1 -- 172.21.15.62:0/3331680869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 msgr2=0x55e8bd424960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.558+0000 7f50a37fe700 1 --2- 172.21.15.62:0/3331680869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd424960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.558+0000 7f50a37fe700 1 -- 172.21.15.62:0/3331680869 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bd1a3e60 con 0x55e8bd425cf0 2023-07-14T16:33:14.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.559+0000 7f50a37fe700 1 --2- 172.21.15.62:0/3331680869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd425cf0 0x55e8bd426160 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5090009d30 tx=0x7f5090011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=822b2a2ab71040d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:14.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.559+0000 7f50a17fa700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f50880a1350 0x7f50880a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.560+0000 7f50a17fa700 1 -- 172.21.15.62:0/2223784736 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f50940cdee0 con 0x7f50a4072f70 2023-07-14T16:33:14.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.560+0000 7f509a7fc700 1 -- 172.21.15.62:0/3331680869 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090004220 con 0x55e8bd425cf0 2023-07-14T16:33:14.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.561+0000 7f509a7fc700 1 -- 172.21.15.62:0/3331680869 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5090004380 con 0x55e8bd425cf0 2023-07-14T16:33:14.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.561+0000 7f509a7fc700 1 -- 172.21.15.62:0/3331680869 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090019830 con 0x55e8bd425cf0 2023-07-14T16:33:14.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.561+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f50880a1350 0x7f50880a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3331680869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd425cf0 msgr2=0x55e8bd426160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3331680869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd425cf0 0x55e8bd426160 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5090009d30 tx=0x7f5090011720 comp rx=0 tx=0).stop 2023-07-14T16:33:14.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3331680869 shutdown_connections 2023-07-14T16:33:14.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3331680869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd425cf0 0x55e8bd426160 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3331680869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd424960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3331680869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd425760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3331680869 >> 172.21.15.62:0/3331680869 conn(0x55e8bd419e20 msgr2=0x55e8bd41f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:14.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f50880a1350 0x7f50880a3800 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f509c0099d0 tx=0x7f509c009100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:14.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3331680869 shutdown_connections 2023-07-14T16:33:14.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.562+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3331680869 wait complete. 2023-07-14T16:33:14.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.563+0000 7f50bdf15b80 1 -- start start 2023-07-14T16:33:14.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.563+0000 7f50bdf15b80 1 Processor -- start 2023-07-14T16:33:14.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.564+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424550 0x55e8bd42e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.564+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd42e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.564+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 0x55e8bd4357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.564+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd315b40 con 0x55e8bd425cf0 2023-07-14T16:33:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.564+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd315840 con 0x55e8bd424ea0 2023-07-14T16:33:14.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.564+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd1f99a0 con 0x55e8bd424550 2023-07-14T16:33:14.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.564+0000 7f50bdf15b80 10 client.0 _ll_register_callbacks cb 0x55e8bd3137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:14.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.565+0000 7f50a37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd42e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.565+0000 7f50a37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd42e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35126/0 (socket says 172.21.15.62:35126) 2023-07-14T16:33:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.565+0000 7f50a37fe700 1 -- 172.21.15.62:0/440142510 learned_addr learned my addr 172.21.15.62:0/440142510 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.566+0000 7f50a37fe700 1 -- 172.21.15.62:0/440142510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424550 msgr2=0x55e8bd42e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.566+0000 7f50a37fe700 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424550 0x55e8bd42e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.566+0000 7f50a37fe700 1 -- 172.21.15.62:0/440142510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 msgr2=0x55e8bd4357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:14.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.566+0000 7f50a37fe700 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 0x55e8bd4357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.566+0000 7f50a37fe700 1 -- 172.21.15.62:0/440142510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5090009960 con 0x55e8bd424ea0 2023-07-14T16:33:14.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.566+0000 7f50a37fe700 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd42e790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5090006010 tx=0x7f5090009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=153889ac90cb68c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:14.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.567+0000 7f50877fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:14.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.567+0000 7f50877fe700 1 -- 172.21.15.62:0/440142510 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090005240 con 0x55e8bd424ea0 2023-07-14T16:33:14.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.567+0000 7f50bdf15b80 1 -- 172.21.15.62:0/440142510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8bd2cf570 con 0x55e8bd424ea0 2023-07-14T16:33:14.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.567+0000 7f50877fe700 1 -- 172.21.15.62:0/440142510 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f50900053a0 con 0x55e8bd424ea0 2023-07-14T16:33:14.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.567+0000 7f50877fe700 1 -- 172.21.15.62:0/440142510 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5090003a40 con 0x55e8bd424ea0 2023-07-14T16:33:14.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.568+0000 7f50bdf15b80 10 client.29906 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:14.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.568+0000 7f50bdf15b80 1 -- 172.21.15.62:0/440142510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e8bd18cce0 con 0x55e8bd424ea0 2023-07-14T16:33:14.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.568+0000 7f50877fe700 1 -- 172.21.15.62:0/440142510 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e8bd2cf570 con 0x55e8bd424ea0 2023-07-14T16:33:14.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.568+0000 7f50877fe700 1 -- 172.21.15.62:0/440142510 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5090021040 con 0x55e8bd424ea0 2023-07-14T16:33:14.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.569+0000 7f50bdf15b80 10 client.29906 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.569+0000 7f50bdf15b80 10 client.29906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.569+0000 7f50bdf15b80 20 client.29906 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.569+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8bd44afc0 0x55e8bd44d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.569+0000 7f50bdf15b80 4 client.29906 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.569+0000 7f50bdf15b80 1 -- 172.21.15.62:0/440142510 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8bd36ef30 con 0x55e8bd44afc0 2023-07-14T16:33:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.569+0000 7f50a3fff700 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8bd44afc0 0x55e8bd44d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.570+0000 7f50a3fff700 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8bd44afc0 0x55e8bd44d3d0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd5ae6db406ce3f7 server_cookie=47d31b2f49f038db in_seq=0 out_seq=0 2023-07-14T16:33:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.570+0000 7f50877fe700 10 client.29906 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.571+0000 7f50877fe700 1 -- 172.21.15.62:0/440142510 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e8bd36ef30 con 0x55e8bd44afc0 2023-07-14T16:33:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.571+0000 7f50877fe700 10 client.29906 handle_command_reply: tid=0 2023-07-14T16:33:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.571+0000 7f50bdf15b80 1 client.29906 shutdown 2023-07-14T16:33:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50bdf15b80 1 -- 172.21.15.62:0/440142510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 msgr2=0x55e8bd42e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd42e790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5090006010 tx=0x7f5090009780 comp rx=0 tx=0).stop 2023-07-14T16:33:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50bdf15b80 1 -- 172.21.15.62:0/440142510 shutdown_connections 2023-07-14T16:33:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 0x55e8bd4357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424550 0x55e8bd42e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8bd44afc0 0x55e8bd44d3d0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/440142510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424ea0 0x55e8bd42e790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50bdf15b80 1 -- 172.21.15.62:0/440142510 >> 172.21.15.62:0/440142510 conn(0x55e8bd419e20 msgr2=0x55e8bd41e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.572+0000 7f50877fe700 0 client.29906 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.573+0000 7f50bdf15b80 1 -- 172.21.15.62:0/440142510 shutdown_connections 2023-07-14T16:33:14.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.573+0000 7f50bdf15b80 1 -- 172.21.15.62:0/440142510 wait complete. 2023-07-14T16:33:14.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.573+0000 7f50bdf15b80 20 client.29906 trim_cache size 0 max 16384 2023-07-14T16:33:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.606+0000 7f50bdf15b80 1 Processor -- start 2023-07-14T16:33:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50bdf15b80 1 -- start start 2023-07-14T16:33:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd42c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd435790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd425cf0 0x55e8bd42e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd425330 con 0x55e8bd424550 2023-07-14T16:33:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd42be40 con 0x55e8bd425cf0 2023-07-14T16:33:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd429720 con 0x55e8bd424ea0 2023-07-14T16:33:14.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd42c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd42c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39436/0 (socket says 172.21.15.62:39436) 2023-07-14T16:33:14.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.607+0000 7f50a3fff700 1 -- 172.21.15.62:0/1125769806 learned_addr learned my addr 172.21.15.62:0/1125769806 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:14.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd425cf0 0x55e8bd42e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a37fe700 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd435790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a3fff700 1 -- 172.21.15.62:0/1125769806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 msgr2=0x55e8bd435790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:14.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a3fff700 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd435790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a3fff700 1 -- 172.21.15.62:0/1125769806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd425cf0 msgr2=0x55e8bd42e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:14.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a3fff700 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd425cf0 0x55e8bd42e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a3fff700 1 -- 172.21.15.62:0/1125769806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8bd42f1a0 con 0x55e8bd424550 2023-07-14T16:33:14.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a37fe700 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd435790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:14.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.608+0000 7f50a3fff700 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd42c060 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f509409b2c0 tx=0x7f5094009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945f6d9238aa0796 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.609+0000 7f509affd700 1 -- 172.21.15.62:0/1125769806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509409ae70 con 0x55e8bd424550 2023-07-14T16:33:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.609+0000 7f509affd700 1 -- 172.21.15.62:0/1125769806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f509409ac90 con 0x55e8bd424550 2023-07-14T16:33:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.609+0000 7f509affd700 1 -- 172.21.15.62:0/1125769806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940cbaa0 con 0x55e8bd424550 2023-07-14T16:33:14.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.610+0000 7f50bdf15b80 1 -- 172.21.15.62:0/1125769806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 msgr2=0x55e8bd42c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.610+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd42c060 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f509409b2c0 tx=0x7f5094009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.610+0000 7f50bdf15b80 1 -- 172.21.15.62:0/1125769806 shutdown_connections 2023-07-14T16:33:14.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.610+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd425cf0 0x55e8bd42e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.610+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd424550 0x55e8bd42c060 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.610+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/1125769806 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd435790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.610+0000 7f50bdf15b80 1 -- 172.21.15.62:0/1125769806 >> 172.21.15.62:0/1125769806 conn(0x55e8bd419e20 msgr2=0x55e8bd4470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:14.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.611+0000 7f50bdf15b80 1 -- 172.21.15.62:0/1125769806 shutdown_connections 2023-07-14T16:33:14.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.611+0000 7f50bdf15b80 1 -- 172.21.15.62:0/1125769806 wait complete. 2023-07-14T16:33:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.611+0000 7f50bdf15b80 1 -- start start 2023-07-14T16:33:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.612+0000 7f50bdf15b80 1 Processor -- start 2023-07-14T16:33:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.612+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424550 0x55e8bd4324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.612+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd432a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.612+0000 7f50bdf15b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 0x55e8bd49ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.612+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd432f90 con 0x55e8bd425cf0 2023-07-14T16:33:14.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.612+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd433100 con 0x55e8bd424550 2023-07-14T16:33:14.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.612+0000 7f50bdf15b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8bd49f3e0 con 0x55e8bd424ea0 2023-07-14T16:33:14.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.612+0000 7f50bdf15b80 10 client.0 _ll_register_callbacks cb 0x55e8bd3137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:14.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.613+0000 7f50a37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd432a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.613+0000 7f50a37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd432a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56368/0 (socket says 172.21.15.62:56368) 2023-07-14T16:33:14.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.613+0000 7f50a37fe700 1 -- 172.21.15.62:0/3582886438 learned_addr learned my addr 172.21.15.62:0/3582886438 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.613+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 0x55e8bd49ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.613+0000 7f50a3fff700 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424550 0x55e8bd4324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.614+0000 7f50a8b9c700 1 -- 172.21.15.62:0/3582886438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 msgr2=0x55e8bd432a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.614+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd432a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.614+0000 7f50a8b9c700 1 -- 172.21.15.62:0/3582886438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424550 msgr2=0x55e8bd4324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.614+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424550 0x55e8bd4324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.614+0000 7f50a8b9c700 1 -- 172.21.15.62:0/3582886438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a41a4240 con 0x55e8bd425cf0 2023-07-14T16:33:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.615+0000 7f50a8b9c700 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 0x55e8bd49ee70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f509c018040 tx=0x7f509c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d362285e552556a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.615+0000 7f50877fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.616+0000 7f50877fe700 1 -- 172.21.15.62:0/3582886438 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0048e0 con 0x55e8bd425cf0 2023-07-14T16:33:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.616+0000 7f50877fe700 1 -- 172.21.15.62:0/3582886438 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f509c016640 con 0x55e8bd425cf0 2023-07-14T16:33:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.616+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3582886438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5090004380 con 0x55e8bd425cf0 2023-07-14T16:33:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.617+0000 7f50877fe700 1 -- 172.21.15.62:0/3582886438 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f509c016800 con 0x55e8bd425cf0 2023-07-14T16:33:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.617+0000 7f50bdf15b80 10 client.20373 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.617+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3582886438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e8bd1ca2d0 con 0x55e8bd425cf0 2023-07-14T16:33:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50877fe700 1 -- 172.21.15.62:0/3582886438 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5090004380 con 0x55e8bd425cf0 2023-07-14T16:33:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50877fe700 1 -- 172.21.15.62:0/3582886438 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f509c036ac0 con 0x55e8bd425cf0 2023-07-14T16:33:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50bdf15b80 10 client.20373 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50bdf15b80 10 client.20373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50bdf15b80 20 client.20373 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8bd4a0f20 0x55e8bd4a3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50bdf15b80 4 client.20373 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:33:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3582886438 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e8bd3310d0 con 0x55e8bd4a0f20 2023-07-14T16:33:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.618+0000 7f50a3fff700 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8bd4a0f20 0x55e8bd4a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.619+0000 7f50a3fff700 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8bd4a0f20 0x55e8bd4a3330 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8db271c745f9e22 server_cookie=3e835dbacaf08c7a in_seq=0 out_seq=0 2023-07-14T16:33:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.619+0000 7f50877fe700 10 client.20373 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.621+0000 7f50877fe700 1 -- 172.21.15.62:0/3582886438 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e8bd3310d0 con 0x55e8bd4a0f20 2023-07-14T16:33:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.621+0000 7f50877fe700 10 client.20373 handle_command_reply: tid=0 2023-07-14T16:33:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.621+0000 7f50bdf15b80 1 client.20373 shutdown 2023-07-14T16:33:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3582886438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 msgr2=0x55e8bd49ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 0x55e8bd49ee70 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f509c018040 tx=0x7f509c004b40 comp rx=0 tx=0).stop 2023-07-14T16:33:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3582886438 shutdown_connections 2023-07-14T16:33:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e8bd425cf0 0x55e8bd49ee70 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e8bd4a0f20 0x55e8bd4a3330 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e8bd424550 0x55e8bd4324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 --2- 172.21.15.62:0/3582886438 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e8bd424ea0 0x55e8bd432a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3582886438 >> 172.21.15.62:0/3582886438 conn(0x55e8bd419e20 msgr2=0x55e8bd42c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50877fe700 0 client.20373 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.622+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3582886438 shutdown_connections 2023-07-14T16:33:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.623+0000 7f50bdf15b80 1 -- 172.21.15.62:0/3582886438 wait complete. 2023-07-14T16:33:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.623+0000 7f50bdf15b80 20 client.20373 trim_cache size 0 max 16384 2023-07-14T16:33:14.599 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:33:14.600 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:33:14.600 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "a737103c-a50d-4e9f-832d-82113c553e0e", 2023-07-14T16:33:14.600 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:33:14.601 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:33:14.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 -- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f50880a1350 msgr2=0x7f50880a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f50880a1350 0x7f50880a3800 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f509c0099d0 tx=0x7f509c009100 comp rx=0 tx=0).stop 2023-07-14T16:33:14.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 -- 172.21.15.62:0/2223784736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a4072f70 msgr2=0x7f50a419be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a4072f70 0x7f50a419be00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f50940098e0 tx=0x7f5094005740 comp rx=0 tx=0).stop 2023-07-14T16:33:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 -- 172.21.15.62:0/2223784736 shutdown_connections 2023-07-14T16:33:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50a410c0b0 0x7f50a41a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f50880a1350 0x7f50880a3800 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a4072f70 0x7f50a419be00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 --2- 172.21.15.62:0/2223784736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a4074550 0x7f50a419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 -- 172.21.15.62:0/2223784736 >> 172.21.15.62:0/2223784736 conn(0x7f50a40fb0e0 msgr2=0x7f50a40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 -- 172.21.15.62:0/2223784736 shutdown_connections 2023-07-14T16:33:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:14.624+0000 7f50aa5ff700 1 -- 172.21.15.62:0/2223784736 wait complete. 2023-07-14T16:33:14.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:33:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.012+0000 7f357c5ff700 1 -- 172.21.15.62:0/3350965743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3574071b60 msgr2=0x7f35741081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.012+0000 7f357c5ff700 1 --2- 172.21.15.62:0/3350965743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3574071b60 0x7f35741081a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f356c009f90 tx=0x7f356c00b330 comp rx=0 tx=0).stop 2023-07-14T16:33:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.012+0000 7f357c5ff700 1 -- 172.21.15.62:0/3350965743 shutdown_connections 2023-07-14T16:33:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.012+0000 7f357c5ff700 1 --2- 172.21.15.62:0/3350965743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3574071b60 0x7f35741081a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.012+0000 7f357c5ff700 1 --2- 172.21.15.62:0/3350965743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3574071f80 0x7f3574107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.012+0000 7f357c5ff700 1 --2- 172.21.15.62:0/3350965743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35740716f0 0x7f3574107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.012+0000 7f357c5ff700 1 -- 172.21.15.62:0/3350965743 >> 172.21.15.62:0/3350965743 conn(0x7f357406cd80 msgr2=0x7f357406f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.013+0000 7f357c5ff700 1 -- 172.21.15.62:0/3350965743 shutdown_connections 2023-07-14T16:33:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.013+0000 7f357c5ff700 1 -- 172.21.15.62:0/3350965743 wait complete. 2023-07-14T16:33:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.014+0000 7f357c5ff700 1 Processor -- start 2023-07-14T16:33:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.015+0000 7f357c5ff700 1 -- start start 2023-07-14T16:33:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.015+0000 7f357c5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35740716f0 0x7f3574117150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.015+0000 7f357c5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3574071b60 0x7f3574117690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.015+0000 7f357c5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357411c6f0 0x7f357411cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.015+0000 7f357c5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357411d0a0 con 0x7f35740716f0 2023-07-14T16:33:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.015+0000 7f357c5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357411d210 con 0x7f357411c6f0 2023-07-14T16:33:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.015+0000 7f357c5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357411d3d0 con 0x7f3574071b60 2023-07-14T16:33:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.015+0000 7f357a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35740716f0 0x7f3574117150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f357a39b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35740716f0 0x7f3574117150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39454/0 (socket says 172.21.15.62:39454) 2023-07-14T16:33:14.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f357a39b700 1 -- 172.21.15.62:0/4040732339 learned_addr learned my addr 172.21.15.62:0/4040732339 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:14.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f357ab9c700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357411c6f0 0x7f357411cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f3579b9a700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3574071b60 0x7f3574117690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f357a39b700 1 -- 172.21.15.62:0/4040732339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3574071b60 msgr2=0x7f3574117690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f357a39b700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3574071b60 0x7f3574117690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f357a39b700 1 -- 172.21.15.62:0/4040732339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357411c6f0 msgr2=0x7f357411cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f357a39b700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357411c6f0 0x7f357411cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.016+0000 7f357a39b700 1 -- 172.21.15.62:0/4040732339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c009c40 con 0x7f35740716f0 2023-07-14T16:33:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.017+0000 7f357a39b700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35740716f0 0x7f3574117150 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f357000e9c0 tx=0x7f357000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:14.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.017+0000 7f356b7fe700 1 -- 172.21.15.62:0/4040732339 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570004a30 con 0x7f35740716f0 2023-07-14T16:33:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.017+0000 7f356b7fe700 1 -- 172.21.15.62:0/4040732339 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f357000cc50 con 0x7f35740716f0 2023-07-14T16:33:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.017+0000 7f356b7fe700 1 -- 172.21.15.62:0/4040732339 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570005410 con 0x7f35740716f0 2023-07-14T16:33:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.018+0000 7f357c5ff700 1 -- 172.21.15.62:0/4040732339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357411d6b0 con 0x7f35740716f0 2023-07-14T16:33:14.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.018+0000 7f357c5ff700 1 -- 172.21.15.62:0/4040732339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357411db40 con 0x7f35740716f0 2023-07-14T16:33:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.024+0000 7f356b7fe700 1 -- 172.21.15.62:0/4040732339 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3570005570 con 0x7f35740716f0 2023-07-14T16:33:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.025+0000 7f358fe7fb80 1 Processor -- start 2023-07-14T16:33:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.025+0000 7f358fe7fb80 1 -- start start 2023-07-14T16:33:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.025+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e024d0 0x557368e028e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.025+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e02e20 0x557368e036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.025+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.025+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368d07cb0 con 0x557368e024d0 2023-07-14T16:33:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.025+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368d5aa40 con 0x557368e03c70 2023-07-14T16:33:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.025+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368d53ed0 con 0x557368e02e20 2023-07-14T16:33:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.026+0000 7f357ab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e024d0 0x557368e028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.026+0000 7f357ab9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e024d0 0x557368e028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39462/0 (socket says 172.21.15.62:39462) 2023-07-14T16:33:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.026+0000 7f357ab9c700 1 -- 172.21.15.62:0/6728838 learned_addr learned my addr 172.21.15.62:0/6728838 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:15.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.026+0000 7f3579b9a700 1 --2- 172.21.15.62:0/6728838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.027+0000 7f357ab9c700 1 -- 172.21.15.62:0/6728838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e02e20 msgr2=0x557368e036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:15.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.027+0000 7f357ab9c700 1 --2- 172.21.15.62:0/6728838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e02e20 0x557368e036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.027+0000 7f357ab9c700 1 -- 172.21.15.62:0/6728838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 msgr2=0x557368e040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.027+0000 7f357ab9c700 1 --2- 172.21.15.62:0/6728838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e040e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.027+0000 7f357ab9c700 1 -- 172.21.15.62:0/6728838 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557368b81de0 con 0x557368e024d0 2023-07-14T16:33:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.027+0000 7f356b7fe700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f35600a1320 0x7f35600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.027+0000 7f356b7fe700 1 -- 172.21.15.62:0/4040732339 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3570014070 con 0x7f35740716f0 2023-07-14T16:33:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.027+0000 7f357ab9c700 1 --2- 172.21.15.62:0/6728838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e024d0 0x557368e028e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f356c009c10 tx=0x7f356c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0612d19f666a52 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.028+0000 7f3568bfe700 1 -- 172.21.15.62:0/6728838 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00e040 con 0x557368e024d0 2023-07-14T16:33:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.028+0000 7f3568bfe700 1 -- 172.21.15.62:0/6728838 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f356c003e90 con 0x557368e024d0 2023-07-14T16:33:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.028+0000 7f3568bfe700 1 -- 172.21.15.62:0/6728838 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c005040 con 0x557368e024d0 2023-07-14T16:33:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.029+0000 7f357a39b700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f35600a1320 0x7f35600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.029+0000 7f358fe7fb80 1 -- 172.21.15.62:0/6728838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e024d0 msgr2=0x557368e028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:15.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.029+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/6728838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e024d0 0x557368e028e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f356c009c10 tx=0x7f356c0093e0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.030+0000 7f358fe7fb80 1 -- 172.21.15.62:0/6728838 shutdown_connections 2023-07-14T16:33:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.030+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/6728838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.030+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/6728838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e024d0 0x557368e028e0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.030+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/6728838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e02e20 0x557368e036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.030+0000 7f358fe7fb80 1 -- 172.21.15.62:0/6728838 >> 172.21.15.62:0/6728838 conn(0x557368df7da0 msgr2=0x557368dfd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:15.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.031+0000 7f357a39b700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f35600a1320 0x7f35600a37d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3570099d30 tx=0x7f3570096170 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.033+0000 7f358fe7fb80 1 -- 172.21.15.62:0/6728838 shutdown_connections 2023-07-14T16:33:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.033+0000 7f358fe7fb80 1 -- 172.21.15.62:0/6728838 wait complete. 2023-07-14T16:33:15.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.034+0000 7f358fe7fb80 1 -- start start 2023-07-14T16:33:15.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.035+0000 7f358fe7fb80 1 Processor -- start 2023-07-14T16:33:15.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.036+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e0c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.036+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e0c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.036+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e13710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.036+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368cf3ac0 con 0x557368e02e20 2023-07-14T16:33:15.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.036+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368cf37c0 con 0x557368e03c70 2023-07-14T16:33:15.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.036+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368bd7920 con 0x557368e024d0 2023-07-14T16:33:15.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.036+0000 7f358fe7fb80 10 client.0 _ll_register_callbacks cb 0x557368cf1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:15.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.036+0000 7f357ab9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e13710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f357ab9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e13710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35174/0 (socket says 172.21.15.62:35174) 2023-07-14T16:33:15.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f357ab9c700 1 -- 172.21.15.62:0/3797648398 learned_addr learned my addr 172.21.15.62:0/3797648398 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:15.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f357ab9c700 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e0c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f3579b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e0c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f357ab9c700 1 -- 172.21.15.62:0/3797648398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 msgr2=0x557368e0c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f357ab9c700 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e0c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f357ab9c700 1 -- 172.21.15.62:0/3797648398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 msgr2=0x557368e0c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f357ab9c700 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e0c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.037+0000 7f357ab9c700 1 -- 172.21.15.62:0/3797648398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357410ae40 con 0x557368e03c70 2023-07-14T16:33:15.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.038+0000 7f357ab9c700 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e13710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f356c003a00 tx=0x7f356c006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6323ccf77a3d93fb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:15.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.040+0000 7f355dffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:15.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.040+0000 7f355dffb700 1 -- 172.21.15.62:0/3797648398 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00e070 con 0x557368e03c70 2023-07-14T16:33:15.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.040+0000 7f355dffb700 1 -- 172.21.15.62:0/3797648398 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f356c005160 con 0x557368e03c70 2023-07-14T16:33:15.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.040+0000 7f358fe7fb80 1 -- 172.21.15.62:0/3797648398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557368cad4f0 con 0x557368e03c70 2023-07-14T16:33:15.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.041+0000 7f355dffb700 1 -- 172.21.15.62:0/3797648398 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f356c003d20 con 0x557368e03c70 2023-07-14T16:33:15.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.041+0000 7f358fe7fb80 10 client.29921 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:15.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.041+0000 7f358fe7fb80 1 -- 172.21.15.62:0/3797648398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557368b6ac60 con 0x557368e03c70 2023-07-14T16:33:15.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.041+0000 7f355dffb700 1 -- 172.21.15.62:0/3797648398 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557368cad4f0 con 0x557368e03c70 2023-07-14T16:33:15.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.042+0000 7f355dffb700 1 -- 172.21.15.62:0/3797648398 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f356c022020 con 0x557368e03c70 2023-07-14T16:33:15.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.042+0000 7f358fe7fb80 10 client.29921 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:15.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.042+0000 7f358fe7fb80 10 client.29921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:15.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.042+0000 7f358fe7fb80 20 client.29921 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:15.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.042+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557368e17a20 0x557368e19e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.042+0000 7f358fe7fb80 4 client.29921 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:15.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.042+0000 7f358fe7fb80 1 -- 172.21.15.62:0/3797648398 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557368d4ceb0 con 0x557368e17a20 2023-07-14T16:33:15.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.043+0000 7f3579b9a700 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557368e17a20 0x557368e19e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.043+0000 7f3579b9a700 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557368e17a20 0x557368e19e00 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec431ff4a4fa759 server_cookie=30f4fee285404a7d in_seq=0 out_seq=0 2023-07-14T16:33:15.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.044+0000 7f355dffb700 10 client.29921 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:15.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.044+0000 7f355dffb700 1 -- 172.21.15.62:0/3797648398 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557368d4ceb0 con 0x557368e17a20 2023-07-14T16:33:15.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.044+0000 7f355dffb700 10 client.29921 handle_command_reply: tid=0 2023-07-14T16:33:15.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.044+0000 7f358fe7fb80 1 client.29921 shutdown 2023-07-14T16:33:15.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f358fe7fb80 1 -- 172.21.15.62:0/3797648398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 msgr2=0x557368e13710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:15.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e13710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f356c003a00 tx=0x7f356c006160 comp rx=0 tx=0).stop 2023-07-14T16:33:15.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f358fe7fb80 1 -- 172.21.15.62:0/3797648398 shutdown_connections 2023-07-14T16:33:15.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557368e17a20 0x557368e19e00 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e03c70 0x557368e13710 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e0c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/3797648398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e0c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f358fe7fb80 1 -- 172.21.15.62:0/3797648398 >> 172.21.15.62:0/3797648398 conn(0x557368df7da0 msgr2=0x557368dfc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:15.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.045+0000 7f355dffb700 0 client.29921 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:15.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.046+0000 7f358fe7fb80 1 -- 172.21.15.62:0/3797648398 shutdown_connections 2023-07-14T16:33:15.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.046+0000 7f358fe7fb80 1 -- 172.21.15.62:0/3797648398 wait complete. 2023-07-14T16:33:15.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.046+0000 7f358fe7fb80 20 client.29921 trim_cache size 0 max 16384 2023-07-14T16:33:15.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.083+0000 7f358fe7fb80 1 Processor -- start 2023-07-14T16:33:15.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.083+0000 7f358fe7fb80 1 -- start start 2023-07-14T16:33:15.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.084+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e19a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.084+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e02e20 0x557368e136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.084+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e03c70 0x557368e06c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.084+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368e0bd60 con 0x557368e03c70 2023-07-14T16:33:15.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.084+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368dfec50 con 0x557368e02e20 2023-07-14T16:33:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.084+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368df79c0 con 0x557368e024d0 2023-07-14T16:33:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e19a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e19a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56420/0 (socket says 172.21.15.62:56420) 2023-07-14T16:33:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 -- 172.21.15.62:0/2811913375 learned_addr learned my addr 172.21.15.62:0/2811913375 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 --2- 172.21.15.62:0/2811913375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e03c70 0x557368e06c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 -- 172.21.15.62:0/2811913375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e02e20 msgr2=0x557368e136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 --2- 172.21.15.62:0/2811913375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e02e20 0x557368e136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 -- 172.21.15.62:0/2811913375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e03c70 msgr2=0x557368e06c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:15.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 --2- 172.21.15.62:0/2811913375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e03c70 0x557368e06c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.085+0000 7f357ab9c700 1 -- 172.21.15.62:0/2811913375 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557368e04310 con 0x557368e024d0 2023-07-14T16:33:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.086+0000 7f357ab9c700 1 --2- 172.21.15.62:0/2811913375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e19a30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f356c002230 tx=0x7f356c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c77593e0d2b473e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.086+0000 7f35693ff700 1 -- 172.21.15.62:0/2811913375 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00e070 con 0x557368e024d0 2023-07-14T16:33:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.086+0000 7f35693ff700 1 -- 172.21.15.62:0/2811913375 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f356c029030 con 0x557368e024d0 2023-07-14T16:33:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.087+0000 7f35693ff700 1 -- 172.21.15.62:0/2811913375 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c0324e0 con 0x557368e024d0 2023-07-14T16:33:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.087+0000 7f358fe7fb80 1 -- 172.21.15.62:0/2811913375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 msgr2=0x557368e19a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.087+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/2811913375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e19a30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f356c002230 tx=0x7f356c0096c0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.088+0000 7f358fe7fb80 1 -- 172.21.15.62:0/2811913375 shutdown_connections 2023-07-14T16:33:15.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.088+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/2811913375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e03c70 0x557368e06c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.088+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/2811913375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e024d0 0x557368e19a30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.088+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/2811913375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e02e20 0x557368e136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.088+0000 7f358fe7fb80 1 -- 172.21.15.62:0/2811913375 >> 172.21.15.62:0/2811913375 conn(0x557368df7da0 msgr2=0x557368e0adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:15.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.088+0000 7f358fe7fb80 1 -- 172.21.15.62:0/2811913375 shutdown_connections 2023-07-14T16:33:15.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.088+0000 7f358fe7fb80 1 -- 172.21.15.62:0/2811913375 wait complete. 2023-07-14T16:33:15.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.088+0000 7f358fe7fb80 1 -- start start 2023-07-14T16:33:15.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.089+0000 7f358fe7fb80 1 Processor -- start 2023-07-14T16:33:15.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.089+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e024d0 0x557368e110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.089+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e11600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.089+0000 7f358fe7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e03c70 0x557368e12610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.089+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368e12b50 con 0x557368e02e20 2023-07-14T16:33:15.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.089+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368e12cc0 con 0x557368e024d0 2023-07-14T16:33:15.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.089+0000 7f358fe7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557368e12e80 con 0x557368e03c70 2023-07-14T16:33:15.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.089+0000 7f358fe7fb80 10 client.0 _ll_register_callbacks cb 0x7f356c007d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:15.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.090+0000 7f3579b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.090+0000 7f3579b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e11600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39504/0 (socket says 172.21.15.62:39504) 2023-07-14T16:33:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.090+0000 7f3579b9a700 1 -- 172.21.15.62:0/933258443 learned_addr learned my addr 172.21.15.62:0/933258443 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.090+0000 7f357ab9c700 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e03c70 0x557368e12610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.090+0000 7f357ab9c700 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e024d0 0x557368e110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.091+0000 7f3579b9a700 1 -- 172.21.15.62:0/933258443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e03c70 msgr2=0x557368e12610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.091+0000 7f3579b9a700 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e03c70 0x557368e12610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.091+0000 7f3579b9a700 1 -- 172.21.15.62:0/933258443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e024d0 msgr2=0x557368e110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.091+0000 7f3579b9a700 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e024d0 0x557368e110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.091+0000 7f3579b9a700 1 -- 172.21.15.62:0/933258443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557368b6ac60 con 0x557368e02e20 2023-07-14T16:33:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.092+0000 7f3579b9a700 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e11600 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f356401db70 tx=0x7f356401de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c0c16476b11d33 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.094+0000 7f355dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.094+0000 7f355dffb700 1 -- 172.21.15.62:0/933258443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564017040 con 0x557368e02e20 2023-07-14T16:33:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.094+0000 7f355dffb700 1 -- 172.21.15.62:0/933258443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f356400bd80 con 0x557368e02e20 2023-07-14T16:33:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.094+0000 7f358fe7fb80 1 -- 172.21.15.62:0/933258443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f356c003e90 con 0x557368e02e20 2023-07-14T16:33:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.096+0000 7f355dffb700 1 -- 172.21.15.62:0/933258443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3564003680 con 0x557368e02e20 2023-07-14T16:33:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.096+0000 7f358fe7fb80 10 client.20391 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.096+0000 7f358fe7fb80 1 -- 172.21.15.62:0/933258443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557368ba8250 con 0x557368e02e20 2023-07-14T16:33:15.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.097+0000 7f355dffb700 1 -- 172.21.15.62:0/933258443 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f356c003e90 con 0x557368e02e20 2023-07-14T16:33:15.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.097+0000 7f355dffb700 1 -- 172.21.15.62:0/933258443 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3564012690 con 0x557368e02e20 2023-07-14T16:33:15.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.097+0000 7f358fe7fb80 10 client.20391 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:15.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.097+0000 7f358fe7fb80 10 client.20391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:15.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.097+0000 7f358fe7fb80 20 client.20391 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:15.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.097+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557368e6c490 0x557368e6e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:15.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.097+0000 7f358fe7fb80 4 client.20391 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:33:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.097+0000 7f358fe7fb80 1 -- 172.21.15.62:0/933258443 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557368d0f050 con 0x557368e6c490 2023-07-14T16:33:15.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.099+0000 7f357ab9c700 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557368e6c490 0x557368e6e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:15.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.100+0000 7f357ab9c700 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557368e6c490 0x557368e6e8a0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ad09d1180d84a6 server_cookie=c48b685ca6b6da55 in_seq=0 out_seq=0 2023-07-14T16:33:15.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.100+0000 7f355dffb700 10 client.20391 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.101+0000 7f355dffb700 1 -- 172.21.15.62:0/933258443 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557368d0f050 con 0x557368e6c490 2023-07-14T16:33:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.101+0000 7f355dffb700 10 client.20391 handle_command_reply: tid=0 2023-07-14T16:33:15.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.103+0000 7f358fe7fb80 1 client.20391 shutdown 2023-07-14T16:33:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.105+0000 7f358fe7fb80 1 -- 172.21.15.62:0/933258443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 msgr2=0x557368e11600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:15.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.105+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e11600 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f356401db70 tx=0x7f356401de80 comp rx=0 tx=0).stop 2023-07-14T16:33:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f358fe7fb80 1 -- 172.21.15.62:0/933258443 shutdown_connections 2023-07-14T16:33:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557368e03c70 0x557368e12610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557368e6c490 0x557368e6e8a0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557368e024d0 0x557368e110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f358fe7fb80 1 --2- 172.21.15.62:0/933258443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557368e02e20 0x557368e11600 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f358fe7fb80 1 -- 172.21.15.62:0/933258443 >> 172.21.15.62:0/933258443 conn(0x557368df7da0 msgr2=0x557368e0adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f355dffb700 0 client.20391 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f358fe7fb80 1 -- 172.21.15.62:0/933258443 shutdown_connections 2023-07-14T16:33:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.107+0000 7f358fe7fb80 1 -- 172.21.15.62:0/933258443 wait complete. 2023-07-14T16:33:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.108+0000 7f358fe7fb80 20 client.20391 trim_cache size 0 max 16384 2023-07-14T16:33:15.082 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:33:15.082 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:33:15.082 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:33:15.082 INFO:teuthology.orchestra.run.smithi062.stdout: "a737103c-a50d-4e9f-832d-82113c553e0e": { 2023-07-14T16:33:15.082 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:33:15.082 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "a737103c-a50d-4e9f-832d-82113c553e0e", 2023-07-14T16:33:15.082 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:33:15.083 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:33:15.083 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:33:15.083 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:33:15.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.111+0000 7f357c5ff700 1 -- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f35600a1320 msgr2=0x7f35600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:15.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.111+0000 7f357c5ff700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f35600a1320 0x7f35600a37d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3570099d30 tx=0x7f3570096170 comp rx=0 tx=0).stop 2023-07-14T16:33:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.111+0000 7f357c5ff700 1 -- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35740716f0 msgr2=0x7f3574117150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.111+0000 7f357c5ff700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35740716f0 0x7f3574117150 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f357000e9c0 tx=0x7f357000ecd0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.112+0000 7f357c5ff700 1 -- 172.21.15.62:0/4040732339 shutdown_connections 2023-07-14T16:33:15.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.112+0000 7f357c5ff700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f35600a1320 0x7f35600a37d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.112+0000 7f357c5ff700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f357411c6f0 0x7f357411cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.112+0000 7f357c5ff700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35740716f0 0x7f3574117150 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.112+0000 7f357c5ff700 1 --2- 172.21.15.62:0/4040732339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3574071b60 0x7f3574117690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:15.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.112+0000 7f357c5ff700 1 -- 172.21.15.62:0/4040732339 >> 172.21.15.62:0/4040732339 conn(0x7f357406cd80 msgr2=0x7f357406f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:15.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.113+0000 7f357c5ff700 1 -- 172.21.15.62:0/4040732339 shutdown_connections 2023-07-14T16:33:15.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:15.113+0000 7f357c5ff700 1 -- 172.21.15.62:0/4040732339 wait complete. 2023-07-14T16:33:15.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:14 smithi062 ceph-mon[95543]: pgmap v885: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:33:15.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:14 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:33:15.112 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:14 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:15.114 INFO:tasks.cephfs.filesystem:scrub status for tag:a737103c-a50d-4e9f-832d-82113c553e0e - {'path': '/', 'tag': 'a737103c-a50d-4e9f-832d-82113c553e0e', 'options': 'recursive,force'} 2023-07-14T16:33:15.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:14 smithi081 ceph-mon[127795]: pgmap v885: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:33:15.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:14 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:33:15.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:14 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:15.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:14 smithi164 ceph-mon[97427]: pgmap v885: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:33:15.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:14 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:33:15.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:14 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:15 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:33:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:15 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:15 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:33:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:15 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:33:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:15 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:15 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:15 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:33:16.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:15 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:15 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:33:16.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:15 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:33:16.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:15 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:15 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:15 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:33:16.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:15 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:15 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:33:16.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:15 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:33:16.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:15 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:16.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:15 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:17.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:16 smithi062 ceph-mon[95543]: pgmap v886: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-07-14T16:33:17.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:16 smithi081 ceph-mon[127795]: pgmap v886: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-07-14T16:33:17.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:16 smithi164 ceph-mon[97427]: pgmap v886: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-07-14T16:33:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.839+0000 7f9dec3ff700 1 -- 172.21.15.62:0/428546356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 msgr2=0x7f9de4107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.839+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/428546356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 0x7f9de4107720 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009990 tx=0x7f9de0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.840+0000 7f9dec3ff700 1 -- 172.21.15.62:0/428546356 shutdown_connections 2023-07-14T16:33:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.840+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/428546356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9de4107c60 0x7f9de41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.840+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/428546356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de4071760 0x7f9de4071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.840+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/428546356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 0x7f9de4107720 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.840+0000 7f9dec3ff700 1 -- 172.21.15.62:0/428546356 >> 172.21.15.62:0/428546356 conn(0x7f9de406cd80 msgr2=0x7f9de406f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.840+0000 7f9dec3ff700 1 -- 172.21.15.62:0/428546356 shutdown_connections 2023-07-14T16:33:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.840+0000 7f9dec3ff700 1 -- 172.21.15.62:0/428546356 wait complete. 2023-07-14T16:33:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.841+0000 7f9dec3ff700 1 Processor -- start 2023-07-14T16:33:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.841+0000 7f9dec3ff700 1 -- start start 2023-07-14T16:33:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.842+0000 7f9dec3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 0x7f9de41171b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.842+0000 7f9dec3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9de4071760 0x7f9de41176f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.842+0000 7f9dec3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de4107c60 0x7f9de41185c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.842+0000 7f9dec3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4118c20 con 0x7f9de4107c60 2023-07-14T16:33:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.842+0000 7f9dec3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4118d90 con 0x7f9de4069270 2023-07-14T16:33:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.842+0000 7f9dec3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4118f50 con 0x7f9de4071760 2023-07-14T16:33:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.844+0000 7f9dea19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 0x7f9de41171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.844+0000 7f9dea19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 0x7f9de41171b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35214/0 (socket says 172.21.15.62:35214) 2023-07-14T16:33:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.844+0000 7f9dea19b700 1 -- 172.21.15.62:0/1910566310 learned_addr learned my addr 172.21.15.62:0/1910566310 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.844+0000 7f9dea99c700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de4107c60 0x7f9de41185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.845+0000 7f9de999a700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9de4071760 0x7f9de41176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.845+0000 7f9dea19b700 1 -- 172.21.15.62:0/1910566310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9de4071760 msgr2=0x7f9de41176f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.845+0000 7f9dea19b700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9de4071760 0x7f9de41176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.845+0000 7f9dea19b700 1 -- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de4107c60 msgr2=0x7f9de41185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.845+0000 7f9dea19b700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de4107c60 0x7f9de41185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.845+0000 7f9dea19b700 1 -- 172.21.15.62:0/1910566310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00095f0 con 0x7f9de4069270 2023-07-14T16:33:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.845+0000 7f9dea19b700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 0x7f9de41171b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400b500 tx=0x7f9dd400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.846+0000 7f9ddb7fe700 1 -- 172.21.15.62:0/1910566310 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4011830 con 0x7f9de4069270 2023-07-14T16:33:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.846+0000 7f9ddb7fe700 1 -- 172.21.15.62:0/1910566310 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dd4011e20 con 0x7f9de4069270 2023-07-14T16:33:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.846+0000 7f9ddb7fe700 1 -- 172.21.15.62:0/1910566310 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4010dd0 con 0x7f9de4069270 2023-07-14T16:33:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.846+0000 7f9dec3ff700 1 -- 172.21.15.62:0/1910566310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de4119230 con 0x7f9de4069270 2023-07-14T16:33:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.846+0000 7f9dec3ff700 1 -- 172.21.15.62:0/1910566310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de411da80 con 0x7f9de4069270 2023-07-14T16:33:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.854+0000 7f9dffca3b80 1 Processor -- start 2023-07-14T16:33:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.861+0000 7f9dffca3b80 1 -- start start 2023-07-14T16:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.861+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af065300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.861+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af0661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.861+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 0x55f3af066b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.861+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3aef6ac50 con 0x55f3aeec16b0 2023-07-14T16:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.861+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3aefbd9e0 con 0x55f3af065840 2023-07-14T16:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.861+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3aefb6e70 con 0x55f3af0666e0 2023-07-14T16:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.863+0000 7f9ddb7fe700 1 -- 172.21.15.62:0/1910566310 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9dd4011990 con 0x7f9de4069270 2023-07-14T16:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.864+0000 7f9dea99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af065300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.864+0000 7f9dea99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af065300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39524/0 (socket says 172.21.15.62:39524) 2023-07-14T16:33:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.864+0000 7f9dea99c700 1 -- 172.21.15.62:0/1125534489 learned_addr learned my addr 172.21.15.62:0/1125534489 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.865+0000 7f9de999a700 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 0x55f3af066b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.865+0000 7f9dea19b700 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af0661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.865+0000 7f9dea99c700 1 -- 172.21.15.62:0/1125534489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 msgr2=0x55f3af066b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.865+0000 7f9dea99c700 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 0x55f3af066b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.865+0000 7f9dea99c700 1 -- 172.21.15.62:0/1125534489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 msgr2=0x55f3af0661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.866+0000 7f9ddb7fe700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9dd00a1320 0x7f9dd00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.866+0000 7f9ddb7fe700 1 -- 172.21.15.62:0/1910566310 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9dd40cc0c0 con 0x7f9de4069270 2023-07-14T16:33:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.866+0000 7f9dea99c700 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af0661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.866+0000 7f9dea99c700 1 -- 172.21.15.62:0/1125534489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3aede4d80 con 0x55f3aeec16b0 2023-07-14T16:33:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.868+0000 7f9de999a700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9dd00a1320 0x7f9dd00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.868+0000 7f9dea99c700 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af065300 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009440 tx=0x7f9ddc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf84b06942a3cca3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.869+0000 7f9dd8bfe700 1 -- 172.21.15.62:0/1125534489 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00e070 con 0x55f3aeec16b0 2023-07-14T16:33:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.869+0000 7f9dd8bfe700 1 -- 172.21.15.62:0/1125534489 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ddc00f040 con 0x55f3aeec16b0 2023-07-14T16:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.869+0000 7f9dd8bfe700 1 -- 172.21.15.62:0/1125534489 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc015af0 con 0x55f3aeec16b0 2023-07-14T16:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.869+0000 7f9de999a700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9dd00a1320 0x7f9dd00a37d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000c00 tx=0x7f9de000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.870+0000 7f9dffca3b80 1 -- 172.21.15.62:0/1125534489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 msgr2=0x55f3af065300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.870+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af065300 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009440 tx=0x7f9ddc009920 comp rx=0 tx=0).stop 2023-07-14T16:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.871+0000 7f9dffca3b80 1 -- 172.21.15.62:0/1125534489 shutdown_connections 2023-07-14T16:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.871+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 0x55f3af066b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.871+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af065300 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.871+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/1125534489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af0661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.871+0000 7f9dffca3b80 1 -- 172.21.15.62:0/1125534489 >> 172.21.15.62:0/1125534489 conn(0x55f3af05abe0 msgr2=0x55f3af05fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.871+0000 7f9dffca3b80 1 -- 172.21.15.62:0/1125534489 shutdown_connections 2023-07-14T16:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.871+0000 7f9dffca3b80 1 -- 172.21.15.62:0/1125534489 wait complete. 2023-07-14T16:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.872+0000 7f9dffca3b80 1 -- start start 2023-07-14T16:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.872+0000 7f9dffca3b80 1 Processor -- start 2023-07-14T16:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.873+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.873+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af06ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.873+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 0x55f3af075e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.873+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3aef56a60 con 0x55f3aeec16b0 2023-07-14T16:33:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.873+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3aef56760 con 0x55f3af065840 2023-07-14T16:33:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.873+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3aee3a8c0 con 0x55f3af0666e0 2023-07-14T16:33:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.873+0000 7f9dffca3b80 10 client.0 _ll_register_callbacks cb 0x55f3aef54700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.874+0000 7f9dea19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.874+0000 7f9dea19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af06e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39538/0 (socket says 172.21.15.62:39538) 2023-07-14T16:33:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.874+0000 7f9dea19b700 1 -- 172.21.15.62:0/3631945554 learned_addr learned my addr 172.21.15.62:0/3631945554 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.874+0000 7f9de999a700 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af06ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.875+0000 7f9de999a700 1 -- 172.21.15.62:0/3631945554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 msgr2=0x55f3af075e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.875+0000 7f9de999a700 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 0x55f3af075e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.875+0000 7f9de999a700 1 -- 172.21.15.62:0/3631945554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 msgr2=0x55f3af06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.875+0000 7f9de999a700 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af06e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.875+0000 7f9de999a700 1 -- 172.21.15.62:0/3631945554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc0090f0 con 0x55f3af065840 2023-07-14T16:33:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.878+0000 7f9de999a700 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af06ee10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9de0003f60 tx=0x7f9de0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3362dc3af1469210 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.879+0000 7f9dcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.879+0000 7f9dcdffb700 1 -- 172.21.15.62:0/3631945554 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de000be80 con 0x55f3af065840 2023-07-14T16:33:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.879+0000 7f9dcdffb700 1 -- 172.21.15.62:0/3631945554 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9de0004500 con 0x55f3af065840 2023-07-14T16:33:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.879+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3631945554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3aef10490 con 0x55f3af065840 2023-07-14T16:33:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.880+0000 7f9dcdffb700 1 -- 172.21.15.62:0/3631945554 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9de0003d40 con 0x55f3af065840 2023-07-14T16:33:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.880+0000 7f9dffca3b80 10 client.29936 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.880+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3631945554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f3aedcdc00 con 0x55f3af065840 2023-07-14T16:33:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.880+0000 7f9dcdffb700 1 -- 172.21.15.62:0/3631945554 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f3aef10490 con 0x55f3af065840 2023-07-14T16:33:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.880+0000 7f9dcdffb700 1 -- 172.21.15.62:0/3631945554 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9de000f740 con 0x55f3af065840 2023-07-14T16:33:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.881+0000 7f9dffca3b80 10 client.29936 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.881+0000 7f9dffca3b80 10 client.29936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.881+0000 7f9dffca3b80 20 client.29936 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.881+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3af07a6a0 0x55f3af07cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.881+0000 7f9dffca3b80 4 client.29936 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.881+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3631945554 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3aefafe50 con 0x55f3af07a6a0 2023-07-14T16:33:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.882+0000 7f9dea19b700 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3af07a6a0 0x55f3af07cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.882+0000 7f9dcdffb700 10 client.29936 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.883+0000 7f9dea19b700 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3af07a6a0 0x55f3af07cab0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fdf53bade69442d server_cookie=468213a4f1a86b9e in_seq=0 out_seq=0 2023-07-14T16:33:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dcdffb700 1 -- 172.21.15.62:0/3631945554 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f3aefafe50 con 0x55f3af07a6a0 2023-07-14T16:33:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dcdffb700 10 client.29936 handle_command_reply: tid=0 2023-07-14T16:33:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 client.29936 shutdown 2023-07-14T16:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 msgr2=0x55f3af06ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af06ee10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9de0003f60 tx=0x7f9de0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3631945554 shutdown_connections 2023-07-14T16:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af0666e0 0x55f3af075e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3aeec16b0 0x55f3af06e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3af07a6a0 0x55f3af07cab0 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3631945554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af065840 0x55f3af06ee10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3631945554 >> 172.21.15.62:0/3631945554 conn(0x55f3af05abe0 msgr2=0x55f3af05fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.884+0000 7f9dcdffb700 0 client.29936 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.885+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3631945554 shutdown_connections 2023-07-14T16:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.885+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3631945554 wait complete. 2023-07-14T16:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.885+0000 7f9dffca3b80 20 client.29936 trim_cache size 0 max 16384 2023-07-14T16:33:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.913+0000 7f9dffca3b80 1 Processor -- start 2023-07-14T16:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.914+0000 7f9dffca3b80 1 -- start start 2023-07-14T16:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.914+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3aeec16b0 0x55f3af06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.914+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af0666e0 0x55f3af075e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.914+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af06d060 0x55f3af06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.914+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3aef10490 con 0x55f3af0666e0 2023-07-14T16:33:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.914+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3af06fac0 con 0x55f3aeec16b0 2023-07-14T16:33:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.914+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3af02b060 con 0x55f3af06d060 2023-07-14T16:33:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.915+0000 7f9dea99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af06d060 0x55f3af06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.915+0000 7f9dea99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af06d060 0x55f3af06e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56458/0 (socket says 172.21.15.62:56458) 2023-07-14T16:33:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.915+0000 7f9dea99c700 1 -- 172.21.15.62:0/3130439709 learned_addr learned my addr 172.21.15.62:0/3130439709 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.915+0000 7f9dea19b700 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3aeec16b0 0x55f3af06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.915+0000 7f9de999a700 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af0666e0 0x55f3af075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.916+0000 7f9de999a700 1 -- 172.21.15.62:0/3130439709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af06d060 msgr2=0x55f3af06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.916+0000 7f9de999a700 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af06d060 0x55f3af06e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.916+0000 7f9de999a700 1 -- 172.21.15.62:0/3130439709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3aeec16b0 msgr2=0x55f3af06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.916+0000 7f9de999a700 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3aeec16b0 0x55f3af06cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.916+0000 7f9de999a700 1 -- 172.21.15.62:0/3130439709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3af066d80 con 0x55f3af0666e0 2023-07-14T16:33:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.916+0000 7f9de999a700 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af0666e0 0x55f3af075e10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f9de0025000 tx=0x7f9de0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c262fe5b58c96f2f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.917+0000 7f9dd93ff700 1 -- 172.21.15.62:0/3130439709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de000fdc0 con 0x55f3af0666e0 2023-07-14T16:33:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.917+0000 7f9dd93ff700 1 -- 172.21.15.62:0/3130439709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9de000f8f0 con 0x55f3af0666e0 2023-07-14T16:33:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.917+0000 7f9dd93ff700 1 -- 172.21.15.62:0/3130439709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0004a90 con 0x55f3af0666e0 2023-07-14T16:33:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.918+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3130439709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af0666e0 msgr2=0x55f3af075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.918+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af0666e0 0x55f3af075e10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f9de0025000 tx=0x7f9de0003d40 comp rx=0 tx=0).stop 2023-07-14T16:33:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.918+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3130439709 shutdown_connections 2023-07-14T16:33:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.918+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3af06d060 0x55f3af06e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.918+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3aeec16b0 0x55f3af06cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.918+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/3130439709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af0666e0 0x55f3af075e10 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.918+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3130439709 >> 172.21.15.62:0/3130439709 conn(0x55f3af05abe0 msgr2=0x55f3af06d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.919+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3130439709 shutdown_connections 2023-07-14T16:33:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.919+0000 7f9dffca3b80 1 -- 172.21.15.62:0/3130439709 wait complete. 2023-07-14T16:33:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.919+0000 7f9dffca3b80 1 -- start start 2023-07-14T16:33:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.920+0000 7f9dffca3b80 1 Processor -- start 2023-07-14T16:33:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.920+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3aeec16b0 0x55f3af071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.920+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af0666e0 0x55f3af071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.920+0000 7f9dffca3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af06d060 0x55f3af074b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.920+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3af0750a0 con 0x55f3af06d060 2023-07-14T16:33:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.920+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3af075210 con 0x55f3af0666e0 2023-07-14T16:33:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.920+0000 7f9dffca3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3af0753d0 con 0x55f3aeec16b0 2023-07-14T16:33:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.920+0000 7f9dffca3b80 10 client.0 _ll_register_callbacks cb 0x7f9de00057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9dea19b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3aeec16b0 0x55f3af071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9de999a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af0666e0 0x55f3af071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9dea19b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3aeec16b0 0x55f3af071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:56474/0 (socket says 172.21.15.62:56474) 2023-07-14T16:33:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9dea19b700 1 -- 172.21.15.62:0/2256969271 learned_addr learned my addr 172.21.15.62:0/2256969271 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9de999a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af0666e0 0x55f3af071b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35254/0 (socket says 172.21.15.62:35254) 2023-07-14T16:33:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9dea99c700 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af06d060 0x55f3af074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9de999a700 1 -- 172.21.15.62:0/2256969271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3aeec16b0 msgr2=0x55f3af071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9de999a700 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3aeec16b0 0x55f3af071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9de999a700 1 -- 172.21.15.62:0/2256969271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af06d060 msgr2=0x55f3af074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9de999a700 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af06d060 0x55f3af074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.922+0000 7f9de999a700 1 -- 172.21.15.62:0/2256969271 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3aedcdc00 con 0x55f3af0666e0 2023-07-14T16:33:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.923+0000 7f9de999a700 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af0666e0 0x55f3af071b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55f3af065f70 tx=0x7f9de0004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac48b873037bd50 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.924+0000 7f9dcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.924+0000 7f9dcdffb700 1 -- 172.21.15.62:0/2256969271 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0005a10 con 0x55f3af0666e0 2023-07-14T16:33:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.924+0000 7f9dcdffb700 1 -- 172.21.15.62:0/2256969271 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9de003d030 con 0x55f3af0666e0 2023-07-14T16:33:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.924+0000 7f9dffca3b80 1 -- 172.21.15.62:0/2256969271 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ddc00f040 con 0x55f3af0666e0 2023-07-14T16:33:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.924+0000 7f9dcdffb700 1 -- 172.21.15.62:0/2256969271 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9de000be80 con 0x55f3af0666e0 2023-07-14T16:33:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.924+0000 7f9dffca3b80 10 client.29942 fetch_fsmap learned FSMap version 31 2023-07-14T16:33:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.924+0000 7f9dffca3b80 1 -- 172.21.15.62:0/2256969271 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f3aee0b1f0 con 0x55f3af0666e0 2023-07-14T16:33:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dcdffb700 1 -- 172.21.15.62:0/2256969271 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ddc00f040 con 0x55f3af0666e0 2023-07-14T16:33:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dcdffb700 1 -- 172.21.15.62:0/2256969271 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9de004c700 con 0x55f3af0666e0 2023-07-14T16:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dffca3b80 10 client.29942 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T16:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dffca3b80 10 client.29942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dffca3b80 20 client.29942 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3af0f0960 0x55f3af0f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dffca3b80 4 client.29942 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dffca3b80 1 -- 172.21.15.62:0/2256969271 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f3aef71ff0 con 0x55f3af0f0960 2023-07-14T16:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.928+0000 7f9dea19b700 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3af0f0960 0x55f3af0f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.929+0000 7f9dea19b700 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3af0f0960 0x55f3af0f2d40 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc134814bba43d1 server_cookie=ea1d91f8ce43244 in_seq=0 out_seq=0 2023-07-14T16:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.929+0000 7f9dcdffb700 10 client.29942 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.955+0000 7f9dcdffb700 1 -- 172.21.15.62:0/2256969271 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f3aef71ff0 con 0x55f3af0f0960 2023-07-14T16:33:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.955+0000 7f9dcdffb700 10 client.29942 handle_command_reply: tid=0 2023-07-14T16:33:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.956+0000 7f9dffca3b80 1 client.29942 shutdown 2023-07-14T16:33:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 -- 172.21.15.62:0/2256969271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af0666e0 msgr2=0x55f3af071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af0666e0 0x55f3af071b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55f3af065f70 tx=0x7f9de0004870 comp rx=0 tx=0).stop 2023-07-14T16:33:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 -- 172.21.15.62:0/2256969271 shutdown_connections 2023-07-14T16:33:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f3af06d060 0x55f3af074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f3aeec16b0 0x55f3af071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55f3af0f0960 0x55f3af0f2d40 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 --2- 172.21.15.62:0/2256969271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f3af0666e0 0x55f3af071b40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 -- 172.21.15.62:0/2256969271 >> 172.21.15.62:0/2256969271 conn(0x55f3af05abe0 msgr2=0x55f3af069630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dcdffb700 0 client.29942 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 -- 172.21.15.62:0/2256969271 shutdown_connections 2023-07-14T16:33:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.957+0000 7f9dffca3b80 1 -- 172.21.15.62:0/2256969271 wait complete. 2023-07-14T16:33:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.958+0000 7f9dffca3b80 20 client.29942 trim_cache size 0 max 16384 2023-07-14T16:33:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:33:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:33:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:33:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:33:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 -- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9dd00a1320 msgr2=0x7f9dd00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9dd00a1320 0x7f9dd00a37d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000c00 tx=0x7f9de000b260 comp rx=0 tx=0).stop 2023-07-14T16:33:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 -- 172.21.15.62:0/1910566310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 msgr2=0x7f9de41171b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 0x7f9de41171b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400b500 tx=0x7f9dd400b8c0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 -- 172.21.15.62:0/1910566310 shutdown_connections 2023-07-14T16:33:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9de4107c60 0x7f9de41185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f9dd00a1320 0x7f9dd00a37d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9de4069270 0x7f9de41171b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 --2- 172.21.15.62:0/1910566310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9de4071760 0x7f9de41176f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 -- 172.21.15.62:0/1910566310 >> 172.21.15.62:0/1910566310 conn(0x7f9de406cd80 msgr2=0x7f9de406f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 -- 172.21.15.62:0/1910566310 shutdown_connections 2023-07-14T16:33:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:18.959+0000 7f9dec3ff700 1 -- 172.21.15.62:0/1910566310 wait complete. 2023-07-14T16:33:19.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:18 smithi062 ceph-mon[95543]: pgmap v887: 97 pgs: 97 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:33:19.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:18 smithi081 ceph-mon[127795]: pgmap v887: 97 pgs: 97 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:33:19.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:18 smithi164 ceph-mon[97427]: pgmap v887: 97 pgs: 97 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:33:21.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:20 smithi062 ceph-mon[95543]: pgmap v888: 97 pgs: 97 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 4.83k op/s 2023-07-14T16:33:21.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:20 smithi081 ceph-mon[127795]: pgmap v888: 97 pgs: 97 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 4.83k op/s 2023-07-14T16:33:21.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:20 smithi164 ceph-mon[97427]: pgmap v888: 97 pgs: 97 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 4.83k op/s 2023-07-14T16:33:23.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:22 smithi062 ceph-mon[95543]: pgmap v889: 97 pgs: 97 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2023-07-14T16:33:23.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:22 smithi081 ceph-mon[127795]: pgmap v889: 97 pgs: 97 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2023-07-14T16:33:23.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:22 smithi164 ceph-mon[97427]: pgmap v889: 97 pgs: 97 active+clean; 13 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2023-07-14T16:33:25.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:24 smithi081 ceph-mon[127795]: pgmap v890: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:33:25.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:24 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:33:25.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:24 smithi164 ceph-mon[97427]: pgmap v890: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:33:25.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:24 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:33:25.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:24 smithi062 ceph-mon[95543]: pgmap v890: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 20 MiB/s wr, 5.16k op/s 2023-07-14T16:33:25.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:24 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:33:27.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:26 smithi081 ceph-mon[127795]: pgmap v891: 97 pgs: 97 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.46k op/s 2023-07-14T16:33:27.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:26 smithi164 ceph-mon[97427]: pgmap v891: 97 pgs: 97 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.46k op/s 2023-07-14T16:33:27.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:26 smithi062 ceph-mon[95543]: pgmap v891: 97 pgs: 97 active+clean; 13 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.46k op/s 2023-07-14T16:33:29.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:28 smithi164 ceph-mon[97427]: pgmap v892: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2023-07-14T16:33:29.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:33:29.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:33:29.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:28 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:33:29.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:28 smithi062 ceph-mon[95543]: pgmap v892: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2023-07-14T16:33:29.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:33:29.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:33:29.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:28 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:33:29.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:28 smithi081 ceph-mon[127795]: pgmap v892: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 5.07k op/s 2023-07-14T16:33:29.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:33:29.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:33:29.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:28 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:33:30.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:29 smithi164 ceph-mon[97427]: pgmap v893: 97 pgs: 97 active+clean; 13 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2023-07-14T16:33:30.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:29 smithi062 ceph-mon[95543]: pgmap v893: 97 pgs: 97 active+clean; 13 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2023-07-14T16:33:30.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:29 smithi081 ceph-mon[127795]: pgmap v893: 97 pgs: 97 active+clean; 13 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 5.17k op/s 2023-07-14T16:33:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.195+0000 7f85665ff700 1 -- 172.21.15.62:0/212752751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560074550 msgr2=0x7f85600729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.195+0000 7f85665ff700 1 --2- 172.21.15.62:0/212752751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560074550 0x7f85600729a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8558009990 tx=0x7f8558009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.197+0000 7f85665ff700 1 -- 172.21.15.62:0/212752751 shutdown_connections 2023-07-14T16:33:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.197+0000 7f85665ff700 1 --2- 172.21.15.62:0/212752751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f856010c0d0 0x7f856010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.197+0000 7f85665ff700 1 --2- 172.21.15.62:0/212752751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560074550 0x7f85600729a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.197+0000 7f85665ff700 1 --2- 172.21.15.62:0/212752751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8560072f70 0x7f85600733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.197+0000 7f85665ff700 1 -- 172.21.15.62:0/212752751 >> 172.21.15.62:0/212752751 conn(0x7f85600fb0e0 msgr2=0x7f85600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.198+0000 7f85665ff700 1 -- 172.21.15.62:0/212752751 shutdown_connections 2023-07-14T16:33:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.198+0000 7f85665ff700 1 -- 172.21.15.62:0/212752751 wait complete. 2023-07-14T16:33:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.199+0000 7f85665ff700 1 Processor -- start 2023-07-14T16:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.200+0000 7f85665ff700 1 -- start start 2023-07-14T16:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.200+0000 7f85665ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560072f70 0x7f8560108830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.200+0000 7f85665ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8560074550 0x7f8560108d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.200+0000 7f85665ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f856010c0d0 0x7f8560109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.200+0000 7f85665ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856010a2a0 con 0x7f8560074550 2023-07-14T16:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.200+0000 7f85665ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856010a410 con 0x7f856010c0d0 2023-07-14T16:33:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.200+0000 7f85665ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856010a5d0 con 0x7f8560072f70 2023-07-14T16:33:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560072f70 0x7f8560108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8560074550 0x7f8560108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8560074550 0x7f8560108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55002/0 (socket says 172.21.15.62:55002) 2023-07-14T16:33:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855f7fe700 1 -- 172.21.15.62:0/979304168 learned_addr learned my addr 172.21.15.62:0/979304168 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f8564b9c700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f856010c0d0 0x7f8560109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855ffff700 1 -- 172.21.15.62:0/979304168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f856010c0d0 msgr2=0x7f8560109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855ffff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f856010c0d0 0x7f8560109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855ffff700 1 -- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8560074550 msgr2=0x7f8560108d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855ffff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8560074550 0x7f8560108d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.201+0000 7f855ffff700 1 -- 172.21.15.62:0/979304168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85580095f0 con 0x7f8560072f70 2023-07-14T16:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.202+0000 7f855ffff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560072f70 0x7f8560108830 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85580098e0 tx=0x7f8558003a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.204+0000 7f855d7fa700 1 -- 172.21.15.62:0/979304168 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855801d070 con 0x7f8560072f70 2023-07-14T16:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.204+0000 7f855d7fa700 1 -- 172.21.15.62:0/979304168 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f855800bd70 con 0x7f8560072f70 2023-07-14T16:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.204+0000 7f855d7fa700 1 -- 172.21.15.62:0/979304168 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855800fdd0 con 0x7f8560072f70 2023-07-14T16:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.204+0000 7f85665ff700 1 -- 172.21.15.62:0/979304168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856010a850 con 0x7f8560072f70 2023-07-14T16:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.204+0000 7f85665ff700 1 -- 172.21.15.62:0/979304168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856010b0e0 con 0x7f8560072f70 2023-07-14T16:33:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.210+0000 7f855d7fa700 1 -- 172.21.15.62:0/979304168 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f855800f560 con 0x7f8560072f70 2023-07-14T16:33:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.211+0000 7f8579ea1b80 1 Processor -- start 2023-07-14T16:33:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.213+0000 7f8579ea1b80 1 -- start start 2023-07-14T16:33:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.214+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55623117e6b0 0x556231322300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.214+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556231322840 0x5562313231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.214+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 0x556231323b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.214+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556231227c50 con 0x556231322840 2023-07-14T16:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.214+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623127a9e0 con 0x5562313236e0 2023-07-14T16:33:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.214+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556231273e70 con 0x55623117e6b0 2023-07-14T16:33:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.215+0000 7f8564b9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55623117e6b0 0x556231322300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.215+0000 7f8564b9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55623117e6b0 0x556231322300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:35348/0 (socket says 172.21.15.62:35348) 2023-07-14T16:33:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.215+0000 7f8564b9c700 1 -- 172.21.15.62:0/3991334435 learned_addr learned my addr 172.21.15.62:0/3991334435 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.215+0000 7f855f7fe700 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 0x556231323b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.215+0000 7f855ffff700 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556231322840 0x5562313231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.218+0000 7f8564b9c700 1 -- 172.21.15.62:0/3991334435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 msgr2=0x556231323b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.218+0000 7f8564b9c700 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 0x556231323b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.218+0000 7f8564b9c700 1 -- 172.21.15.62:0/3991334435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556231322840 msgr2=0x5562313231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.219+0000 7f855d7fa700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f854c0a11d0 0x7f854c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.220+0000 7f8564b9c700 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556231322840 0x5562313231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.220+0000 7f8564b9c700 1 -- 172.21.15.62:0/3991334435 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562310a1d80 con 0x55623117e6b0 2023-07-14T16:33:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.220+0000 7f855d7fa700 1 -- 172.21.15.62:0/979304168 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f85580cd9f0 con 0x7f8560072f70 2023-07-14T16:33:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.221+0000 7f8564b9c700 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55623117e6b0 0x556231322300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f855000a430 tx=0x7f8550009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bafb76553a40c234 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.222+0000 7f85567fc700 1 -- 172.21.15.62:0/3991334435 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550010040 con 0x55623117e6b0 2023-07-14T16:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.222+0000 7f85567fc700 1 -- 172.21.15.62:0/3991334435 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8550013a20 con 0x55623117e6b0 2023-07-14T16:33:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.222+0000 7f85567fc700 1 -- 172.21.15.62:0/3991334435 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550014ad0 con 0x55623117e6b0 2023-07-14T16:33:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.223+0000 7f855f7fe700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f854c0a11d0 0x7f854c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.224+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3991334435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55623117e6b0 msgr2=0x556231322300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.224+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55623117e6b0 0x556231322300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f855000a430 tx=0x7f8550009650 comp rx=0 tx=0).stop 2023-07-14T16:33:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.224+0000 7f855f7fe700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f854c0a11d0 0x7f854c0a3680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8548006fd0 tx=0x7f8548008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.225+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3991334435 shutdown_connections 2023-07-14T16:33:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.225+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 0x556231323b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.225+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55623117e6b0 0x556231322300 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.225+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3991334435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556231322840 0x5562313231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.225+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3991334435 >> 172.21.15.62:0/3991334435 conn(0x556231317be0 msgr2=0x55623131ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.225+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3991334435 shutdown_connections 2023-07-14T16:33:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.225+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3991334435 wait complete. 2023-07-14T16:33:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.226+0000 7f8579ea1b80 1 -- start start 2023-07-14T16:33:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.226+0000 7f8579ea1b80 1 Processor -- start 2023-07-14T16:33:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55623117e6b0 0x55623132b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 0x55623132be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562313236e0 0x556231332ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556231213a60 con 0x5562313236e0 2023-07-14T16:33:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556231213760 con 0x55623117e6b0 2023-07-14T16:33:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562310f78c0 con 0x556231322840 2023-07-14T16:33:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8579ea1b80 10 client.0 _ll_register_callbacks cb 0x556231211700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8564b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562313236e0 0x556231332ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8564b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562313236e0 0x556231332ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55030/0 (socket says 172.21.15.62:55030) 2023-07-14T16:33:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f8564b9c700 1 -- 172.21.15.62:0/3129031114 learned_addr learned my addr 172.21.15.62:0/3129031114 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f855f7fe700 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 0x55623132be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.227+0000 7f855ffff700 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55623117e6b0 0x55623132b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f855ffff700 1 -- 172.21.15.62:0/3129031114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 msgr2=0x55623132be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f855ffff700 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 0x55623132be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f855ffff700 1 -- 172.21.15.62:0/3129031114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562313236e0 msgr2=0x556231332ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f855ffff700 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562313236e0 0x556231332ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f855ffff700 1 -- 172.21.15.62:0/3129031114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8550009c60 con 0x55623117e6b0 2023-07-14T16:33:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f855ffff700 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55623117e6b0 0x55623132b950 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f85580cb550 tx=0x7f855809b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b26f708b9ad0af0b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f853f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f853f7fe700 1 -- 172.21.15.62:0/3129031114 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85580a0070 con 0x55623117e6b0 2023-07-14T16:33:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f853f7fe700 1 -- 172.21.15.62:0/3129031114 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f855809aea0 con 0x55623117e6b0 2023-07-14T16:33:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.228+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3129031114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562311cd490 con 0x55623117e6b0 2023-07-14T16:33:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f853f7fe700 1 -- 172.21.15.62:0/3129031114 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f855809ac70 con 0x55623117e6b0 2023-07-14T16:33:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f8579ea1b80 10 client.29951 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3129031114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55623108ac00 con 0x55623117e6b0 2023-07-14T16:33:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f853f7fe700 1 -- 172.21.15.62:0/3129031114 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85580b2030 con 0x55623117e6b0 2023-07-14T16:33:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f853f7fe700 1 -- 172.21.15.62:0/3129031114 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8558096c90 con 0x55623117e6b0 2023-07-14T16:33:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f8579ea1b80 10 client.29951 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f8579ea1b80 10 client.29951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f8579ea1b80 20 client.29951 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.229+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5562313372c0 0x5562313396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.230+0000 7f8579ea1b80 4 client.29951 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.230+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3129031114 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55623126ce50 con 0x5562313372c0 2023-07-14T16:33:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.230+0000 7f855f7fe700 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5562313372c0 0x5562313396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.230+0000 7f855f7fe700 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5562313372c0 0x5562313396d0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef1334ebefc5e5b server_cookie=3751cc86d94ef011 in_seq=0 out_seq=0 2023-07-14T16:33:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.230+0000 7f853f7fe700 10 client.29951 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.231+0000 7f853f7fe700 1 -- 172.21.15.62:0/3129031114 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55623126ce50 con 0x5562313372c0 2023-07-14T16:33:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.231+0000 7f853f7fe700 10 client.29951 handle_command_reply: tid=0 2023-07-14T16:33:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.231+0000 7f8579ea1b80 1 client.29951 shutdown 2023-07-14T16:33:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55623117e6b0 msgr2=0x55623132b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55623117e6b0 0x55623132b950 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f85580cb550 tx=0x7f855809b3e0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3129031114 shutdown_connections 2023-07-14T16:33:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562313236e0 0x556231332ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5562313372c0 0x5562313396d0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55623117e6b0 0x55623132b950 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3129031114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 0x55623132be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3129031114 >> 172.21.15.62:0/3129031114 conn(0x556231317be0 msgr2=0x55623131c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f853f7fe700 0 client.29951 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3129031114 shutdown_connections 2023-07-14T16:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3129031114 wait complete. 2023-07-14T16:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.232+0000 7f8579ea1b80 20 client.29951 trim_cache size 0 max 16384 2023-07-14T16:33:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.259+0000 7f8579ea1b80 1 Processor -- start 2023-07-14T16:33:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.261+0000 7f8579ea1b80 1 -- start start 2023-07-14T16:33:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.261+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623117e6b0 0x556231326630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.261+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556231322840 0x556231326b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.261+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562313236e0 0x556231332e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.261+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556231322f00 con 0x55623117e6b0 2023-07-14T16:33:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.261+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623132a3b0 con 0x556231322840 2023-07-14T16:33:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.261+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623126ce50 con 0x5562313236e0 2023-07-14T16:33:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623117e6b0 0x556231326630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623117e6b0 0x556231326630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55042/0 (socket says 172.21.15.62:55042) 2023-07-14T16:33:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855ffff700 1 -- 172.21.15.62:0/3689750391 learned_addr learned my addr 172.21.15.62:0/3689750391 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855f7fe700 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556231322840 0x556231326b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f8564b9c700 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562313236e0 0x556231332e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855ffff700 1 -- 172.21.15.62:0/3689750391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562313236e0 msgr2=0x556231332e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855ffff700 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562313236e0 0x556231332e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855ffff700 1 -- 172.21.15.62:0/3689750391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556231322840 msgr2=0x556231326b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855ffff700 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556231322840 0x556231326b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.263+0000 7f855ffff700 1 -- 172.21.15.62:0/3689750391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562310c81f0 con 0x55623117e6b0 2023-07-14T16:33:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.264+0000 7f855ffff700 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623117e6b0 0x556231326630 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f855809aba0 tx=0x7f85580cdcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495911ef47edcdeb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.264+0000 7f8556ffd700 1 -- 172.21.15.62:0/3689750391 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85580a0050 con 0x55623117e6b0 2023-07-14T16:33:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.265+0000 7f8556ffd700 1 -- 172.21.15.62:0/3689750391 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85580b2030 con 0x55623117e6b0 2023-07-14T16:33:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.265+0000 7f8556ffd700 1 -- 172.21.15.62:0/3689750391 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855809eed0 con 0x55623117e6b0 2023-07-14T16:33:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.265+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3689750391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623117e6b0 msgr2=0x556231326630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.265+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623117e6b0 0x556231326630 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f855809aba0 tx=0x7f85580cdcb0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.266+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3689750391 shutdown_connections 2023-07-14T16:33:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.266+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5562313236e0 0x556231332e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.266+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623117e6b0 0x556231326630 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.266+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/3689750391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556231322840 0x556231326b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.266+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3689750391 >> 172.21.15.62:0/3689750391 conn(0x556231317be0 msgr2=0x55623132a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.266+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3689750391 shutdown_connections 2023-07-14T16:33:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.266+0000 7f8579ea1b80 1 -- 172.21.15.62:0/3689750391 wait complete. 2023-07-14T16:33:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.266+0000 7f8579ea1b80 1 -- start start 2023-07-14T16:33:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.267+0000 7f8579ea1b80 1 Processor -- start 2023-07-14T16:33:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 0x5562313327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 0x556231332cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8579ea1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623132f250 0x55623132f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623132fc00 con 0x55623132f250 2023-07-14T16:33:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623132fd70 con 0x5562313236e0 2023-07-14T16:33:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8579ea1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623138b170 con 0x556231322840 2023-07-14T16:33:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8579ea1b80 10 client.0 _ll_register_callbacks cb 0x556231211700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8564b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623132f250 0x55623132f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8564b9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623132f250 0x55623132f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55052/0 (socket says 172.21.15.62:55052) 2023-07-14T16:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f8564b9c700 1 -- 172.21.15.62:0/928422289 learned_addr learned my addr 172.21.15.62:0/928422289 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f855f7fe700 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 0x556231332cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.268+0000 7f855ffff700 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 0x5562313327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.269+0000 7f8564b9c700 1 -- 172.21.15.62:0/928422289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 msgr2=0x5562313327b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.269+0000 7f8564b9c700 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 0x5562313327b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.269+0000 7f8564b9c700 1 -- 172.21.15.62:0/928422289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 msgr2=0x556231332cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.269+0000 7f8564b9c700 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 0x556231332cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.269+0000 7f8564b9c700 1 -- 172.21.15.62:0/928422289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55623108ac00 con 0x55623132f250 2023-07-14T16:33:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.269+0000 7f8564b9c700 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623132f250 0x55623132f6c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8550004c20 tx=0x7f85500138e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb46a747ea3c5a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.269+0000 7f853f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.270+0000 7f853f7fe700 1 -- 172.21.15.62:0/928422289 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550010040 con 0x55623132f250 2023-07-14T16:33:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.270+0000 7f853f7fe700 1 -- 172.21.15.62:0/928422289 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8550006e40 con 0x55623132f250 2023-07-14T16:33:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.270+0000 7f8579ea1b80 1 -- 172.21.15.62:0/928422289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8550013a20 con 0x55623132f250 2023-07-14T16:33:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.270+0000 7f853f7fe700 1 -- 172.21.15.62:0/928422289 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8550013e20 con 0x55623132f250 2023-07-14T16:33:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.270+0000 7f8579ea1b80 10 client.20421 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.270+0000 7f8579ea1b80 1 -- 172.21.15.62:0/928422289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5562310a0250 con 0x55623132f250 2023-07-14T16:33:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.271+0000 7f853f7fe700 1 -- 172.21.15.62:0/928422289 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8550013a20 con 0x55623132f250 2023-07-14T16:33:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.271+0000 7f853f7fe700 1 -- 172.21.15.62:0/928422289 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8550008030 con 0x55623132f250 2023-07-14T16:33:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.271+0000 7f8579ea1b80 10 client.20421 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.271+0000 7f8579ea1b80 10 client.20421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.271+0000 7f8579ea1b80 20 client.20421 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.271+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55623138c950 0x55623138ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.271+0000 7f8579ea1b80 4 client.20421 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:33:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.271+0000 7f8579ea1b80 1 -- 172.21.15.62:0/928422289 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55623122eff0 con 0x55623138c950 2023-07-14T16:33:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.272+0000 7f855ffff700 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55623138c950 0x55623138ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.275+0000 7f855ffff700 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55623138c950 0x55623138ed60 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e92a7be223bbc0d server_cookie=b8c67a4ef9f8928f in_seq=0 out_seq=0 2023-07-14T16:33:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.275+0000 7f853f7fe700 10 client.20421 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.290+0000 7f853f7fe700 1 -- 172.21.15.62:0/928422289 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55623122eff0 con 0x55623138c950 2023-07-14T16:33:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.290+0000 7f853f7fe700 10 client.20421 handle_command_reply: tid=0 2023-07-14T16:33:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.290+0000 7f8579ea1b80 1 client.20421 shutdown 2023-07-14T16:33:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 -- 172.21.15.62:0/928422289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623132f250 msgr2=0x55623132f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623132f250 0x55623132f6c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8550004c20 tx=0x7f85500138e0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 -- 172.21.15.62:0/928422289 shutdown_connections 2023-07-14T16:33:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55623132f250 0x55623132f6c0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556231322840 0x5562313327b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55623138c950 0x55623138ed60 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 --2- 172.21.15.62:0/928422289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5562313236e0 0x556231332cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 -- 172.21.15.62:0/928422289 >> 172.21.15.62:0/928422289 conn(0x556231317be0 msgr2=0x556231326630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f853f7fe700 0 client.20421 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.291+0000 7f8579ea1b80 1 -- 172.21.15.62:0/928422289 shutdown_connections 2023-07-14T16:33:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.292+0000 7f8579ea1b80 1 -- 172.21.15.62:0/928422289 wait complete. 2023-07-14T16:33:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.292+0000 7f8579ea1b80 20 client.20421 trim_cache size 0 max 16384 2023-07-14T16:33:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:33:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:33:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:33:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:33:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 -- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f854c0a11d0 msgr2=0x7f854c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f854c0a11d0 0x7f854c0a3680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8548006fd0 tx=0x7f8548008040 comp rx=0 tx=0).stop 2023-07-14T16:33:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 -- 172.21.15.62:0/979304168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560072f70 msgr2=0x7f8560108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560072f70 0x7f8560108830 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85580098e0 tx=0x7f8558003a80 comp rx=0 tx=0).stop 2023-07-14T16:33:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 -- 172.21.15.62:0/979304168 shutdown_connections 2023-07-14T16:33:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f854c0a11d0 0x7f854c0a3680 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f856010c0d0 0x7f8560109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8560072f70 0x7f8560108830 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 --2- 172.21.15.62:0/979304168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8560074550 0x7f8560108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 -- 172.21.15.62:0/979304168 >> 172.21.15.62:0/979304168 conn(0x7f85600fb0e0 msgr2=0x7f85600fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 -- 172.21.15.62:0/979304168 shutdown_connections 2023-07-14T16:33:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:32.293+0000 7f85665ff700 1 -- 172.21.15.62:0/979304168 wait complete. 2023-07-14T16:33:33.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:32 smithi062 ceph-mon[95543]: pgmap v894: 97 pgs: 97 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:33:33.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:32 smithi081 ceph-mon[127795]: pgmap v894: 97 pgs: 97 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:33:33.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:32 smithi164 ceph-mon[97427]: pgmap v894: 97 pgs: 97 active+clean; 13 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:33:35.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:34 smithi062 ceph-mon[95543]: pgmap v895: 97 pgs: 97 active+clean; 13 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2023-07-14T16:33:35.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:34 smithi081 ceph-mon[127795]: pgmap v895: 97 pgs: 97 active+clean; 13 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2023-07-14T16:33:35.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:34 smithi164 ceph-mon[97427]: pgmap v895: 97 pgs: 97 active+clean; 13 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 5.39k op/s 2023-07-14T16:33:37.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:36 smithi062 ceph-mon[95543]: pgmap v896: 97 pgs: 97 active+clean; 13 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:33:37.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:36 smithi081 ceph-mon[127795]: pgmap v896: 97 pgs: 97 active+clean; 13 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:33:37.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:36 smithi164 ceph-mon[97427]: pgmap v896: 97 pgs: 97 active+clean; 13 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:33:39.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:38 smithi062 ceph-mon[95543]: pgmap v897: 97 pgs: 97 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 4.98k op/s 2023-07-14T16:33:39.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:38 smithi081 ceph-mon[127795]: pgmap v897: 97 pgs: 97 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 4.98k op/s 2023-07-14T16:33:39.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:38 smithi164 ceph-mon[97427]: pgmap v897: 97 pgs: 97 active+clean; 13 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 4.98k op/s 2023-07-14T16:33:41.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:40 smithi081 ceph-mon[127795]: pgmap v898: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.93k op/s 2023-07-14T16:33:41.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:40 smithi164 ceph-mon[97427]: pgmap v898: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.93k op/s 2023-07-14T16:33:41.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:40 smithi062 ceph-mon[95543]: pgmap v898: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.93k op/s 2023-07-14T16:33:42.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:41 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:33:42.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:41 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:33:42.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:41 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:33:43.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: pgmap v899: 97 pgs: 97 active+clean; 13 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:33:43.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.149 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:33:43.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:33:43.150 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:42 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:33:43.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: pgmap v899: 97 pgs: 97 active+clean; 13 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:33:43.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:33:43.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:33:43.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:42 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:33:43.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: pgmap v899: 97 pgs: 97 active+clean; 13 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:33:43.520 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T16:33:43.522 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:33:43.522 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:33:43.522 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:42 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:33:44.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:43 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi081 to 220.9M 2023-07-14T16:33:44.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:43 smithi081 ceph-mon[127795]: Adjusting osd_memory_target on smithi164 to 220.9M 2023-07-14T16:33:44.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:43 smithi081 ceph-mon[127795]: Unable to set osd_memory_target on smithi081 to 231674777: error parsing value: Value '231674777' is below minimum 939524096 2023-07-14T16:33:44.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:43 smithi081 ceph-mon[127795]: Unable to set osd_memory_target on smithi164 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T16:33:44.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:43 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi081 to 220.9M 2023-07-14T16:33:44.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:43 smithi164 ceph-mon[97427]: Adjusting osd_memory_target on smithi164 to 220.9M 2023-07-14T16:33:44.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:43 smithi164 ceph-mon[97427]: Unable to set osd_memory_target on smithi081 to 231674777: error parsing value: Value '231674777' is below minimum 939524096 2023-07-14T16:33:44.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:43 smithi164 ceph-mon[97427]: Unable to set osd_memory_target on smithi164 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T16:33:44.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:43 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi081 to 220.9M 2023-07-14T16:33:44.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:43 smithi062 ceph-mon[95543]: Adjusting osd_memory_target on smithi164 to 220.9M 2023-07-14T16:33:44.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:43 smithi062 ceph-mon[95543]: Unable to set osd_memory_target on smithi081 to 231674777: error parsing value: Value '231674777' is below minimum 939524096 2023-07-14T16:33:44.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:43 smithi062 ceph-mon[95543]: Unable to set osd_memory_target on smithi164 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-14T16:33:45.116 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:33:45.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:44 smithi062 ceph-mon[95543]: pgmap v900: 97 pgs: 97 active+clean; 13 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 4.96k op/s 2023-07-14T16:33:45.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:44 smithi164 ceph-mon[97427]: pgmap v900: 97 pgs: 97 active+clean; 13 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 4.96k op/s 2023-07-14T16:33:45.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:44 smithi081 ceph-mon[127795]: pgmap v900: 97 pgs: 97 active+clean; 13 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 4.96k op/s 2023-07-14T16:33:45.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.614+0000 7f6740dff700 1 -- 172.21.15.62:0/2094904375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f673c10bd60 msgr2=0x7f673c10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.614+0000 7f6740dff700 1 --2- 172.21.15.62:0/2094904375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f673c10bd60 0x7f673c10e1f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f672c009990 tx=0x7f672c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.617+0000 7f6740dff700 1 -- 172.21.15.62:0/2094904375 shutdown_connections 2023-07-14T16:33:45.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.617+0000 7f6740dff700 1 --2- 172.21.15.62:0/2094904375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f673c10bd60 0x7f673c10e1f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.617+0000 7f6740dff700 1 --2- 172.21.15.62:0/2094904375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f673c070ee0 0x7f673c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.617+0000 7f6740dff700 1 --2- 172.21.15.62:0/2094904375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f673c071830 0x7f673c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.617+0000 7f6740dff700 1 -- 172.21.15.62:0/2094904375 >> 172.21.15.62:0/2094904375 conn(0x7f673c06ca90 msgr2=0x7f673c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.619+0000 7f6740dff700 1 -- 172.21.15.62:0/2094904375 shutdown_connections 2023-07-14T16:33:45.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.619+0000 7f6740dff700 1 -- 172.21.15.62:0/2094904375 wait complete. 2023-07-14T16:33:45.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.621+0000 7f6740dff700 1 Processor -- start 2023-07-14T16:33:45.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.621+0000 7f6740dff700 1 -- start start 2023-07-14T16:33:45.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.622+0000 7f6740dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f673c070ee0 0x7f673c114a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.622+0000 7f6740dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f673c071830 0x7f673c114f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.622+0000 7f6740dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f673c10bd60 0x7f673c115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.622+0000 7f6740dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c116590 con 0x7f673c070ee0 2023-07-14T16:33:45.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.622+0000 7f6740dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c116700 con 0x7f673c10bd60 2023-07-14T16:33:45.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.622+0000 7f6740dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c1168c0 con 0x7f673c071830 2023-07-14T16:33:45.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.622+0000 7f6739d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f673c071830 0x7f673c114f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.622+0000 7f673ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f673c10bd60 0x7f673c115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.623+0000 7f673ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f673c10bd60 0x7f673c115fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47770/0 (socket says 172.21.15.62:47770) 2023-07-14T16:33:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.623+0000 7f673ad9d700 1 -- 172.21.15.62:0/1564332463 learned_addr learned my addr 172.21.15.62:0/1564332463 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.623+0000 7f673ad9d700 1 -- 172.21.15.62:0/1564332463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f673c071830 msgr2=0x7f673c114f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.623+0000 7f673ad9d700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f673c071830 0x7f673c114f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.623+0000 7f673ad9d700 1 -- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f673c070ee0 msgr2=0x7f673c114a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.623+0000 7f673ad9d700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f673c070ee0 0x7f673c114a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.623+0000 7f673ad9d700 1 -- 172.21.15.62:0/1564332463 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c0095f0 con 0x7f673c10bd60 2023-07-14T16:33:45.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.623+0000 7f673ad9d700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f673c10bd60 0x7f673c115fc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f672c005e50 tx=0x7f672c003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.620+0000 7fbf47dff700 1 -- 172.21.15.62:0/541108625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf40102f30 msgr2=0x7fbf40103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.620+0000 7fbf47dff700 1 --2- 172.21.15.62:0/541108625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf40102f30 0x7fbf40103380 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0131e0 tx=0x7fbf2c011f90 comp rx=0 tx=0).stop 2023-07-14T16:33:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.621+0000 7fbf47dff700 1 -- 172.21.15.62:0/541108625 shutdown_connections 2023-07-14T16:33:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.621+0000 7fbf47dff700 1 --2- 172.21.15.62:0/541108625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf401038c0 0x7fbf4010a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.621+0000 7fbf47dff700 1 --2- 172.21.15.62:0/541108625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf40101d40 0x7fbf40102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.621+0000 7fbf47dff700 1 --2- 172.21.15.62:0/541108625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf40102f30 0x7fbf40103380 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.621+0000 7fbf47dff700 1 -- 172.21.15.62:0/541108625 >> 172.21.15.62:0/541108625 conn(0x7fbf400fd360 msgr2=0x7fbf400ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.621+0000 7fbf47dff700 1 -- 172.21.15.62:0/541108625 shutdown_connections 2023-07-14T16:33:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.621+0000 7fbf47dff700 1 -- 172.21.15.62:0/541108625 wait complete. 2023-07-14T16:33:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.623+0000 7fbf47dff700 1 Processor -- start 2023-07-14T16:33:45.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.624+0000 7f672b7fe700 1 -- 172.21.15.62:0/1564332463 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c01d070 con 0x7f673c10bd60 2023-07-14T16:33:45.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.624+0000 7f672b7fe700 1 -- 172.21.15.62:0/1564332463 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f672c00bdb0 con 0x7f673c10bd60 2023-07-14T16:33:45.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.624+0000 7f672b7fe700 1 -- 172.21.15.62:0/1564332463 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00fdf0 con 0x7f673c10bd60 2023-07-14T16:33:45.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.624+0000 7f6740dff700 1 -- 172.21.15.62:0/1564332463 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c116ae0 con 0x7f673c10bd60 2023-07-14T16:33:45.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.624+0000 7f6740dff700 1 -- 172.21.15.62:0/1564332463 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c116f40 con 0x7f673c10bd60 2023-07-14T16:33:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.626+0000 7fbf47dff700 1 -- start start 2023-07-14T16:33:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.626+0000 7fbf47dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf40101d40 0x7fbf401b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.626+0000 7fbf47dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf40102f30 0x7fbf401b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.627+0000 7fbf47dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf401038c0 0x7fbf401b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.627+0000 7fbf47dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf401b70e0 con 0x7fbf40101d40 2023-07-14T16:33:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.627+0000 7fbf47dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf401b7250 con 0x7fbf401038c0 2023-07-14T16:33:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.627+0000 7fbf47dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf401b7410 con 0x7fbf40102f30 2023-07-14T16:33:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.627+0000 7fbf4639c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf401038c0 0x7fbf401b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.627+0000 7fbf4639c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf401038c0 0x7fbf401b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47776/0 (socket says 172.21.15.62:47776) 2023-07-14T16:33:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.627+0000 7fbf4639c700 1 -- 172.21.15.62:0/338464480 learned_addr learned my addr 172.21.15.62:0/338464480 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.628+0000 7fbf4639c700 1 -- 172.21.15.62:0/338464480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf40102f30 msgr2=0x7fbf401b5c00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T16:33:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.628+0000 7fbf4639c700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf40102f30 0x7fbf401b5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.628+0000 7fbf4639c700 1 -- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf40101d40 msgr2=0x7fbf401b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.628+0000 7fbf4639c700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf40101d40 0x7fbf401b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.628+0000 7fbf4639c700 1 -- 172.21.15.62:0/338464480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c011e00 con 0x7fbf401038c0 2023-07-14T16:33:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.629+0000 7fbf4639c700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf401038c0 0x7fbf401b6a80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800d980 tx=0x7fbf3800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.631+0000 7f675472db80 1 Processor -- start 2023-07-14T16:33:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.632+0000 7fbf3effd700 1 -- 172.21.15.62:0/338464480 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38009940 con 0x7fbf401038c0 2023-07-14T16:33:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.632+0000 7fbf3effd700 1 -- 172.21.15.62:0/338464480 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf38004830 con 0x7fbf401038c0 2023-07-14T16:33:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.632+0000 7fbf3effd700 1 -- 172.21.15.62:0/338464480 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38005410 con 0x7fbf401038c0 2023-07-14T16:33:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.632+0000 7fbf47dff700 1 -- 172.21.15.62:0/338464480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf401b76f0 con 0x7fbf401038c0 2023-07-14T16:33:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.633+0000 7fbf47dff700 1 -- 172.21.15.62:0/338464480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4006a110 con 0x7fbf401038c0 2023-07-14T16:33:45.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.633+0000 7f672b7fe700 1 -- 172.21.15.62:0/1564332463 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f672c00b920 con 0x7f673c10bd60 2023-07-14T16:33:45.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.634+0000 7f675472db80 1 -- start start 2023-07-14T16:33:45.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.634+0000 7f675472db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb74d0 0x55b067eb78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.634+0000 7f675472db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb7e20 0x55b067eb86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.634+0000 7f675472db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb8c70 0x55b067eb90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.635+0000 7f675472db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067dbccb0 con 0x55b067eb7e20 2023-07-14T16:33:45.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.635+0000 7f675472db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067e0fa40 con 0x55b067eb8c70 2023-07-14T16:33:45.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.635+0000 7f675472db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067e08ed0 con 0x55b067eb74d0 2023-07-14T16:33:45.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.635+0000 7f6739d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb74d0 0x55b067eb78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.635+0000 7f6739d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb74d0 0x55b067eb78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33344/0 (socket says 172.21.15.62:33344) 2023-07-14T16:33:45.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.635+0000 7f6739d9b700 1 -- 172.21.15.62:0/3735283826 learned_addr learned my addr 172.21.15.62:0/3735283826 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.635+0000 7f6739d9b700 1 --2- 172.21.15.62:0/3735283826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb8c70 0x55b067eb90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.636+0000 7f6739d9b700 1 -- 172.21.15.62:0/3735283826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb8c70 msgr2=0x55b067eb90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.636+0000 7f6739d9b700 1 --2- 172.21.15.62:0/3735283826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb8c70 0x55b067eb90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.636+0000 7f6739d9b700 1 -- 172.21.15.62:0/3735283826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb7e20 msgr2=0x55b067eb86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.636+0000 7f6739d9b700 1 --2- 172.21.15.62:0/3735283826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb7e20 0x55b067eb86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.636+0000 7f6739d9b700 1 -- 172.21.15.62:0/3735283826 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b067c36de0 con 0x55b067eb74d0 2023-07-14T16:33:45.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.636+0000 7f6739d9b700 1 --2- 172.21.15.62:0/3735283826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb74d0 0x55b067eb78e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6734009540 tx=0x7f6734009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=843cfd656f4e8307 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.637+0000 7f672b7fe700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67240a12a0 0x7f67240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.637+0000 7f672b7fe700 1 -- 172.21.15.62:0/1564332463 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f672c0cd940 con 0x7f673c10bd60 2023-07-14T16:33:45.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.639+0000 7f673a59c700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67240a12a0 0x7f67240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.640+0000 7fbf5b649b80 1 Processor -- start 2023-07-14T16:33:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.642+0000 7fbf5b649b80 1 -- start start 2023-07-14T16:33:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.642+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8afa6b0 0x558ef8c9e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.642+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9e840 0x558ef8c9f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.642+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9f6e0 0x558ef8c9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.642+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8ba3c50 con 0x558ef8c9f6e0 2023-07-14T16:33:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.642+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8bf69e0 con 0x558ef8c9e840 2023-07-14T16:33:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.642+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8befe70 con 0x558ef8afa6b0 2023-07-14T16:33:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.643+0000 7fbf45b9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9e840 0x558ef8c9f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.643+0000 7fbf45b9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9e840 0x558ef8c9f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47784/0 (socket says 172.21.15.62:47784) 2023-07-14T16:33:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.643+0000 7fbf45b9b700 1 -- 172.21.15.62:0/1516849691 learned_addr learned my addr 172.21.15.62:0/1516849691 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.643+0000 7fbf4539a700 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8afa6b0 0x558ef8c9e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.643+0000 7fbf4539a700 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9f6e0 0x558ef8c9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf45b9b700 1 -- 172.21.15.62:0/1516849691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8afa6b0 msgr2=0x558ef8c9e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8afa6b0 0x558ef8c9e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf45b9b700 1 -- 172.21.15.62:0/1516849691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9f6e0 msgr2=0x558ef8c9fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9f6e0 0x558ef8c9fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf45b9b700 1 -- 172.21.15.62:0/1516849691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ef8a1dd80 con 0x558ef8c9e840 2023-07-14T16:33:45.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.644+0000 7f6728ff9700 1 -- 172.21.15.62:0/3735283826 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400e070 con 0x55b067eb74d0 2023-07-14T16:33:45.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.644+0000 7f6728ff9700 1 -- 172.21.15.62:0/3735283826 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f673400f040 con 0x55b067eb74d0 2023-07-14T16:33:45.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.644+0000 7f6728ff9700 1 -- 172.21.15.62:0/3735283826 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673401a810 con 0x55b067eb74d0 2023-07-14T16:33:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9e840 0x558ef8c9f1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009d00 tx=0x7fbf340077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94244d69665086f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.644+0000 7f673a59c700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67240a12a0 0x7f67240a3750 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f67300094f0 tx=0x7f6730009440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf3effd700 1 -- 172.21.15.62:0/338464480 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbf38005570 con 0x7fbf401038c0 2023-07-14T16:33:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf2bfff700 1 -- 172.21.15.62:0/1516849691 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf340135f0 con 0x558ef8c9e840 2023-07-14T16:33:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf2bfff700 1 -- 172.21.15.62:0/1516849691 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf3400f040 con 0x558ef8c9e840 2023-07-14T16:33:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.644+0000 7fbf2bfff700 1 -- 172.21.15.62:0/1516849691 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34012710 con 0x558ef8c9e840 2023-07-14T16:33:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.645+0000 7fbf5b649b80 1 -- 172.21.15.62:0/1516849691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9e840 msgr2=0x558ef8c9f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.645+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9e840 0x558ef8c9f1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009d00 tx=0x7fbf340077e0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.645+0000 7f675472db80 1 -- 172.21.15.62:0/3735283826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb74d0 msgr2=0x55b067eb78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.645+0000 7f675472db80 1 --2- 172.21.15.62:0/3735283826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb74d0 0x55b067eb78e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6734009540 tx=0x7f6734009920 comp rx=0 tx=0).stop 2023-07-14T16:33:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.647+0000 7fbf5b649b80 1 -- 172.21.15.62:0/1516849691 shutdown_connections 2023-07-14T16:33:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.647+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9f6e0 0x558ef8c9fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.647+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8afa6b0 0x558ef8c9e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.647+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/1516849691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9e840 0x558ef8c9f1a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.647+0000 7fbf5b649b80 1 -- 172.21.15.62:0/1516849691 >> 172.21.15.62:0/1516849691 conn(0x558ef8c93be0 msgr2=0x558ef8c98cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.646+0000 7f675472db80 1 -- 172.21.15.62:0/3735283826 shutdown_connections 2023-07-14T16:33:45.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.646+0000 7f675472db80 1 --2- 172.21.15.62:0/3735283826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb8c70 0x55b067eb90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.646+0000 7f675472db80 1 --2- 172.21.15.62:0/3735283826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb74d0 0x55b067eb78e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.646+0000 7f675472db80 1 --2- 172.21.15.62:0/3735283826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb7e20 0x55b067eb86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.646+0000 7f675472db80 1 -- 172.21.15.62:0/3735283826 >> 172.21.15.62:0/3735283826 conn(0x55b067eacda0 msgr2=0x55b067eb21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.647+0000 7f675472db80 1 -- 172.21.15.62:0/3735283826 shutdown_connections 2023-07-14T16:33:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.647+0000 7fbf5b649b80 1 -- 172.21.15.62:0/1516849691 shutdown_connections 2023-07-14T16:33:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.647+0000 7fbf3effd700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf300a1270 0x7fbf300a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.647+0000 7fbf3effd700 1 -- 172.21.15.62:0/338464480 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf380cc780 con 0x7fbf401038c0 2023-07-14T16:33:45.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.647+0000 7f675472db80 1 -- 172.21.15.62:0/3735283826 wait complete. 2023-07-14T16:33:45.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.647+0000 7f675472db80 1 -- start start 2023-07-14T16:33:45.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.648+0000 7f675472db80 1 Processor -- start 2023-07-14T16:33:45.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.648+0000 7f675472db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb74d0 0x55b067ec1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.648+0000 7f675472db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb7e20 0x55b067ec16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.648+0000 7f675472db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb8c70 0x55b067ec8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.648+0000 7f675472db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067da8ac0 con 0x55b067eb8c70 2023-07-14T16:33:45.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.648+0000 7f675472db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067da87c0 con 0x55b067eb74d0 2023-07-14T16:33:45.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.648+0000 7f675472db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067c8c920 con 0x55b067eb7e20 2023-07-14T16:33:45.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.648+0000 7f675472db80 10 client.0 _ll_register_callbacks cb 0x55b067da6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:45.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f6739d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb7e20 0x55b067ec16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f6739d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb7e20 0x55b067ec16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33352/0 (socket says 172.21.15.62:33352) 2023-07-14T16:33:45.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f6739d9b700 1 -- 172.21.15.62:0/1074915644 learned_addr learned my addr 172.21.15.62:0/1074915644 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f673ad9d700 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb8c70 0x55b067ec8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f6739d9b700 1 -- 172.21.15.62:0/1074915644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb74d0 msgr2=0x55b067ec1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f6739d9b700 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb74d0 0x55b067ec1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.648+0000 7fbf5b649b80 1 -- 172.21.15.62:0/1516849691 wait complete. 2023-07-14T16:33:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.649+0000 7fbf5b649b80 1 -- start start 2023-07-14T16:33:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.649+0000 7fbf5b649b80 1 Processor -- start 2023-07-14T16:33:45.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f6739d9b700 1 -- 172.21.15.62:0/1074915644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb8c70 msgr2=0x55b067ec8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f6739d9b700 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb8c70 0x55b067ec8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f6739d9b700 1 -- 172.21.15.62:0/1074915644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6734008510 con 0x55b067eb7e20 2023-07-14T16:33:45.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.649+0000 7f673ad9d700 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb8c70 0x55b067ec8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:45.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.650+0000 7f6739d9b700 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb7e20 0x55b067ec16d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6734013880 tx=0x7f6734013960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8dbd0530b280f83 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.650+0000 7f6721ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:45.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.650+0000 7f6721ffb700 1 -- 172.21.15.62:0/1074915644 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400e070 con 0x55b067eb7e20 2023-07-14T16:33:45.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.650+0000 7f675472db80 1 -- 172.21.15.62:0/1074915644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b067d624f0 con 0x55b067eb7e20 2023-07-14T16:33:45.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.650+0000 7f6721ffb700 1 -- 172.21.15.62:0/1074915644 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67340095e0 con 0x55b067eb7e20 2023-07-14T16:33:45.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.650+0000 7f6721ffb700 1 -- 172.21.15.62:0/1074915644 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6734003c00 con 0x55b067eb7e20 2023-07-14T16:33:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 0x558ef8ca78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8b8fa60 con 0x558ef8c9e840 2023-07-14T16:33:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8b8f760 con 0x558ef8afa6b0 2023-07-14T16:33:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8a738c0 con 0x558ef8c9f6e0 2023-07-14T16:33:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf5b649b80 10 client.0 _ll_register_callbacks cb 0x558ef8b8d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf4639c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf4639c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33354/0 (socket says 172.21.15.62:33354) 2023-07-14T16:33:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.651+0000 7fbf4639c700 1 -- 172.21.15.62:0/2481227866 learned_addr learned my addr 172.21.15.62:0/2481227866 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f675472db80 10 client.29863 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f675472db80 1 -- 172.21.15.62:0/1074915644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b067c1fc60 con 0x55b067eb7e20 2023-07-14T16:33:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f6721ffb700 1 -- 172.21.15.62:0/1074915644 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b067d624f0 con 0x55b067eb7e20 2023-07-14T16:33:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f6721ffb700 1 -- 172.21.15.62:0/1074915644 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f673401a550 con 0x55b067eb7e20 2023-07-14T16:33:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f675472db80 10 client.29863 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f675472db80 10 client.29863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f675472db80 20 client.29863 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f675472db80 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b067ecce90 0x55b067ecf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f675472db80 4 client.29863 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:45.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.651+0000 7f675472db80 1 -- 172.21.15.62:0/1074915644 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b067e01eb0 con 0x55b067ecce90 2023-07-14T16:33:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf4539a700 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf4639c700 1 -- 172.21.15.62:0/2481227866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 msgr2=0x558ef8ca78b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:33:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf4639c700 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 0x558ef8ca78b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf4639c700 1 -- 172.21.15.62:0/2481227866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 msgr2=0x558ef8ca7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf4639c700 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf4639c700 1 -- 172.21.15.62:0/2481227866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf34009960 con 0x558ef8c9f6e0 2023-07-14T16:33:45.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.652+0000 7f673a59c700 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b067ecce90 0x55b067ecf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf4639c700 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caee30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbf38096040 tx=0x7fbf380a20c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e5df3f61b01729f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf297fa700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.652+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf300a1270 0x7fbf300a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.653+0000 7fbf297fa700 1 -- 172.21.15.62:0/2481227866 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38014020 con 0x558ef8c9f6e0 2023-07-14T16:33:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.653+0000 7fbf297fa700 1 -- 172.21.15.62:0/2481227866 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf38099230 con 0x558ef8c9f6e0 2023-07-14T16:33:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.653+0000 7fbf5b649b80 1 -- 172.21.15.62:0/2481227866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ef8b49490 con 0x558ef8c9f6e0 2023-07-14T16:33:45.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.653+0000 7f673a59c700 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b067ecce90 0x55b067ecf2a0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf15992349d5e5b server_cookie=88b4a7eaf62822b3 in_seq=0 out_seq=0 2023-07-14T16:33:45.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.653+0000 7f6721ffb700 10 client.29863 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.653+0000 7fbf297fa700 1 -- 172.21.15.62:0/2481227866 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf380b2200 con 0x558ef8c9f6e0 2023-07-14T16:33:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.653+0000 7fbf5b649b80 10 client.29866 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.653+0000 7fbf5b649b80 1 -- 172.21.15.62:0/2481227866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558ef8a06c00 con 0x558ef8c9f6e0 2023-07-14T16:33:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.653+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf300a1270 0x7fbf300a3720 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbf34004b10 tx=0x7fbf3400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf297fa700 1 -- 172.21.15.62:0/2481227866 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558ef8b49490 con 0x558ef8c9f6e0 2023-07-14T16:33:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf297fa700 1 -- 172.21.15.62:0/2481227866 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbf38098aa0 con 0x558ef8c9f6e0 2023-07-14T16:33:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf5b649b80 10 client.29866 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf5b649b80 10 client.29866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf5b649b80 20 client.29866 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558ef8cb3540 0x558ef8cb5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf5b649b80 4 client.29866 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf5b649b80 1 -- 172.21.15.62:0/2481227866 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ef8be8e50 con 0x558ef8cb3540 2023-07-14T16:33:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.654+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558ef8cb3540 0x558ef8cb5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.654+0000 7f6721ffb700 1 -- 172.21.15.62:0/1074915644 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b067e01eb0 con 0x55b067ecce90 2023-07-14T16:33:45.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.654+0000 7f6721ffb700 10 client.29863 handle_command_reply: tid=0 2023-07-14T16:33:45.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.654+0000 7f675472db80 1 client.29863 shutdown 2023-07-14T16:33:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.655+0000 7fbf297fa700 10 client.29866 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.655+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558ef8cb3540 0x558ef8cb5950 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806ad835cb882f3b server_cookie=56c5373e00481186 in_seq=0 out_seq=0 2023-07-14T16:33:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 -- 172.21.15.62:0/1074915644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb7e20 msgr2=0x55b067ec16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb7e20 0x55b067ec16d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6734013880 tx=0x7f6734013960 comp rx=0 tx=0).stop 2023-07-14T16:33:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 -- 172.21.15.62:0/1074915644 shutdown_connections 2023-07-14T16:33:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb8c70 0x55b067ec8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b067ecce90 0x55b067ecf2a0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb74d0 0x55b067ec1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 --2- 172.21.15.62:0/1074915644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb7e20 0x55b067ec16d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 -- 172.21.15.62:0/1074915644 >> 172.21.15.62:0/1074915644 conn(0x55b067eacda0 msgr2=0x55b067eb18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f6721ffb700 0 client.29863 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 -- 172.21.15.62:0/1074915644 shutdown_connections 2023-07-14T16:33:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.655+0000 7f675472db80 1 -- 172.21.15.62:0/1074915644 wait complete. 2023-07-14T16:33:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.656+0000 7fbf297fa700 1 -- 172.21.15.62:0/2481227866 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558ef8be8e50 con 0x558ef8cb3540 2023-07-14T16:33:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.656+0000 7fbf297fa700 10 client.29866 handle_command_reply: tid=0 2023-07-14T16:33:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.656+0000 7fbf5b649b80 1 client.29866 shutdown 2023-07-14T16:33:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.656+0000 7fbf5b649b80 1 -- 172.21.15.62:0/2481227866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 msgr2=0x558ef8caee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.656+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caee30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbf38096040 tx=0x7fbf380a20c0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.657+0000 7f675472db80 20 client.29863 trim_cache size 0 max 16384 2023-07-14T16:33:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.657+0000 7fbf5b649b80 1 -- 172.21.15.62:0/2481227866 shutdown_connections 2023-07-14T16:33:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.657+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caee30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.657+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558ef8cb3540 0x558ef8cb5950 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.657+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 0x558ef8ca78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.657+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/2481227866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.657+0000 7fbf297fa700 0 client.29866 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.657+0000 7fbf5b649b80 1 -- 172.21.15.62:0/2481227866 >> 172.21.15.62:0/2481227866 conn(0x558ef8c93be0 msgr2=0x558ef8c984e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.659+0000 7fbf5b649b80 1 -- 172.21.15.62:0/2481227866 shutdown_connections 2023-07-14T16:33:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.659+0000 7fbf5b649b80 1 -- 172.21.15.62:0/2481227866 wait complete. 2023-07-14T16:33:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.659+0000 7fbf5b649b80 20 client.29866 trim_cache size 0 max 16384 2023-07-14T16:33:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.686+0000 7fbf5b649b80 1 Processor -- start 2023-07-14T16:33:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.686+0000 7fbf5b649b80 1 -- start start 2023-07-14T16:33:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 0x558ef8ca78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8c93aa0 con 0x558ef8c9e840 2023-07-14T16:33:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8c9ef00 con 0x558ef8afa6b0 2023-07-14T16:33:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8be8e50 con 0x558ef8c9f6e0 2023-07-14T16:33:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf4539a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf4539a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32820/0 (socket says 172.21.15.62:32820) 2023-07-14T16:33:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.687+0000 7fbf4539a700 1 -- 172.21.15.62:0/3949195535 learned_addr learned my addr 172.21.15.62:0/3949195535 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.688+0000 7fbf4639c700 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.688+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 0x558ef8ca78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.688+0000 7fbf4539a700 1 -- 172.21.15.62:0/3949195535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 msgr2=0x558ef8caed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.688+0000 7fbf4539a700 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.688+0000 7fbf4539a700 1 -- 172.21.15.62:0/3949195535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 msgr2=0x558ef8ca78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.688+0000 7fbf4539a700 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 0x558ef8ca78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.688+0000 7fbf4539a700 1 -- 172.21.15.62:0/3949195535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ef8a441f0 con 0x558ef8c9e840 2023-07-14T16:33:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.689+0000 7fbf4539a700 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005fd0 tx=0x7fbf2c011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210658b24b395ea0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.689+0000 7fbf3cbff700 1 -- 172.21.15.62:0/3949195535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c016040 con 0x558ef8c9e840 2023-07-14T16:33:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.689+0000 7fbf3cbff700 1 -- 172.21.15.62:0/3949195535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf2c0117f0 con 0x558ef8c9e840 2023-07-14T16:33:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.689+0000 7fbf3cbff700 1 -- 172.21.15.62:0/3949195535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c00fae0 con 0x558ef8c9e840 2023-07-14T16:33:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.690+0000 7fbf5b649b80 1 -- 172.21.15.62:0/3949195535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 msgr2=0x558ef8ca7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.690+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005fd0 tx=0x7fbf2c011cc0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.690+0000 7fbf5b649b80 1 -- 172.21.15.62:0/3949195535 shutdown_connections 2023-07-14T16:33:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.690+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9f6e0 0x558ef8caed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.690+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 0x558ef8ca78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.690+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8c9e840 0x558ef8ca7df0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.690+0000 7fbf5b649b80 1 -- 172.21.15.62:0/3949195535 >> 172.21.15.62:0/3949195535 conn(0x558ef8c93be0 msgr2=0x558ef8ca5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.692+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/3949195535 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8afa6b0 0x558ef8ca78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:33:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.697+0000 7fbf5b649b80 1 -- 172.21.15.62:0/3949195535 shutdown_connections 2023-07-14T16:33:45.672 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.700+0000 7f675472db80 1 Processor -- start 2023-07-14T16:33:45.673 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.700+0000 7f675472db80 1 -- start start 2023-07-14T16:33:45.674 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.700+0000 7f675472db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb74d0 0x55b067eceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.700+0000 7f675472db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 0x55b067ec86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.700+0000 7f675472db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ebf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.700+0000 7f675472db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067ec0d60 con 0x55b067eb74d0 2023-07-14T16:33:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.700+0000 7f675472db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067eb3d00 con 0x55b067eb7e20 2023-07-14T16:33:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.700+0000 7f675472db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067eb3070 con 0x55b067eb8c70 2023-07-14T16:33:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f6739d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 0x55b067ec86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f6739d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 0x55b067ec86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47806/0 (socket says 172.21.15.62:47806) 2023-07-14T16:33:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f6739d9b700 1 -- 172.21.15.62:0/2063835762 learned_addr learned my addr 172.21.15.62:0/2063835762 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f673ad9d700 1 --2- 172.21.15.62:0/2063835762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ebf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f673ad9d700 1 -- 172.21.15.62:0/2063835762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 msgr2=0x55b067ec86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f673ad9d700 1 --2- 172.21.15.62:0/2063835762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 0x55b067ec86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f673ad9d700 1 -- 172.21.15.62:0/2063835762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb74d0 msgr2=0x55b067eceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f673ad9d700 1 --2- 172.21.15.62:0/2063835762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb74d0 0x55b067eceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f673ad9d700 1 -- 172.21.15.62:0/2063835762 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b067c5d250 con 0x55b067eb8c70 2023-07-14T16:33:45.677 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.701+0000 7f673ad9d700 1 --2- 172.21.15.62:0/2063835762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ebf060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f672c0cb3f0 tx=0x7f672c09ab60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d3033957a8c1d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.702+0000 7fbf5b649b80 1 -- 172.21.15.62:0/3949195535 wait complete. 2023-07-14T16:33:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.702+0000 7fbf5b649b80 1 -- start start 2023-07-14T16:33:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.702+0000 7f67297fa700 1 -- 172.21.15.62:0/2063835762 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c09a6f0 con 0x55b067eb8c70 2023-07-14T16:33:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.702+0000 7f67297fa700 1 -- 172.21.15.62:0/2063835762 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f672c00bbd0 con 0x55b067eb8c70 2023-07-14T16:33:45.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.702+0000 7f67297fa700 1 -- 172.21.15.62:0/2063835762 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00f560 con 0x55b067eb8c70 2023-07-14T16:33:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.703+0000 7fbf5b649b80 1 Processor -- start 2023-07-14T16:33:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.703+0000 7f675472db80 1 -- 172.21.15.62:0/2063835762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 msgr2=0x55b067ebf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.703+0000 7f675472db80 1 --2- 172.21.15.62:0/2063835762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ebf060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f672c0cb3f0 tx=0x7f672c09ab60 comp rx=0 tx=0).stop 2023-07-14T16:33:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.703+0000 7f675472db80 1 -- 172.21.15.62:0/2063835762 shutdown_connections 2023-07-14T16:33:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.703+0000 7f675472db80 1 --2- 172.21.15.62:0/2063835762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ebf060 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.703+0000 7f675472db80 1 --2- 172.21.15.62:0/2063835762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb74d0 0x55b067eceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.703+0000 7f675472db80 1 --2- 172.21.15.62:0/2063835762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 0x55b067ec86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.680 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.703+0000 7f675472db80 1 -- 172.21.15.62:0/2063835762 >> 172.21.15.62:0/2063835762 conn(0x55b067eacda0 msgr2=0x55b067ebfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.706+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9e840 0x558ef8caaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.706+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9f6e0 0x558ef8cab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.707+0000 7fbf5b649b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8cae430 0x558ef8cae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.707+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8caede0 con 0x558ef8cae430 2023-07-14T16:33:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.707+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8caef50 con 0x558ef8c9f6e0 2023-07-14T16:33:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.707+0000 7fbf5b649b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ef8caf110 con 0x558ef8c9e840 2023-07-14T16:33:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.707+0000 7fbf5b649b80 10 client.0 _ll_register_callbacks cb 0x7fbf38099b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.707+0000 7fbf45b9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9e840 0x558ef8caaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.707+0000 7fbf45b9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9e840 0x558ef8caaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33384/0 (socket says 172.21.15.62:33384) 2023-07-14T16:33:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.707+0000 7fbf45b9b700 1 -- 172.21.15.62:0/217389110 learned_addr learned my addr 172.21.15.62:0/217389110 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.708+0000 7fbf45b9b700 1 -- 172.21.15.62:0/217389110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9f6e0 msgr2=0x558ef8cab410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:33:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.708+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9f6e0 0x558ef8cab410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.708+0000 7fbf45b9b700 1 -- 172.21.15.62:0/217389110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8cae430 msgr2=0x558ef8cae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.708+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8cae430 0x558ef8cae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.708+0000 7fbf45b9b700 1 -- 172.21.15.62:0/217389110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4010aab0 con 0x558ef8c9e840 2023-07-14T16:33:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.709+0000 7fbf45b9b700 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9e840 0x558ef8caaed0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009930 tx=0x7fbf34009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e76e26ea43eaf2d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.709+0000 7fbf297fa700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.709+0000 7fbf297fa700 1 -- 172.21.15.62:0/217389110 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3402e040 con 0x558ef8c9e840 2023-07-14T16:33:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.710+0000 7fbf297fa700 1 -- 172.21.15.62:0/217389110 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf340133f0 con 0x558ef8c9e840 2023-07-14T16:33:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.710+0000 7fbf5b649b80 1 -- 172.21.15.62:0/217389110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf3400f040 con 0x558ef8c9e840 2023-07-14T16:33:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.710+0000 7fbf297fa700 1 -- 172.21.15.62:0/217389110 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf34013d80 con 0x558ef8c9e840 2023-07-14T16:33:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.710+0000 7fbf5b649b80 10 client.29875 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.710+0000 7fbf5b649b80 1 -- 172.21.15.62:0/217389110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558ef8a1c250 con 0x558ef8c9e840 2023-07-14T16:33:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.707+0000 7f675472db80 1 -- 172.21.15.62:0/2063835762 shutdown_connections 2023-07-14T16:33:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.710+0000 7f675472db80 1 -- 172.21.15.62:0/2063835762 wait complete. 2023-07-14T16:33:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.711+0000 7f675472db80 1 -- start start 2023-07-14T16:33:45.689 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.711+0000 7f675472db80 1 Processor -- start 2023-07-14T16:33:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.713+0000 7f675472db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb74d0 0x55b067ec4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.713+0000 7f675472db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 0x55b067ec4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.713+0000 7f675472db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ec7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.713+0000 7f675472db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067ec7ba0 con 0x55b067eb74d0 2023-07-14T16:33:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.713+0000 7f675472db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067ec7d10 con 0x55b067eb7e20 2023-07-14T16:33:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.713+0000 7f675472db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067ec7ed0 con 0x55b067eb8c70 2023-07-14T16:33:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.713+0000 7f675472db80 10 client.0 _ll_register_callbacks cb 0x7f67340087c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ec7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ec7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33388/0 (socket says 172.21.15.62:33388) 2023-07-14T16:33:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 -- 172.21.15.62:0/3418758258 learned_addr learned my addr 172.21.15.62:0/3418758258 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 -- 172.21.15.62:0/3418758258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 msgr2=0x55b067ec4640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:33:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 0x55b067ec4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.691 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 -- 172.21.15.62:0/3418758258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb74d0 msgr2=0x55b067ec4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb74d0 0x55b067ec4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 -- 172.21.15.62:0/3418758258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c116f40 con 0x55b067eb8c70 2023-07-14T16:33:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.714+0000 7f673ad9d700 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ec7660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f672c0a3310 tx=0x7f672c003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cce195a6a8d54eb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.719+0000 7f6721ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.719+0000 7f6721ffb700 1 -- 172.21.15.62:0/3418758258 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c09edc0 con 0x55b067eb8c70 2023-07-14T16:33:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.719+0000 7f6721ffb700 1 -- 172.21.15.62:0/3418758258 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f672c0a2cc0 con 0x55b067eb8c70 2023-07-14T16:33:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.720+0000 7f675472db80 1 -- 172.21.15.62:0/3418758258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f673400f040 con 0x55b067eb8c70 2023-07-14T16:33:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.720+0000 7f6721ffb700 1 -- 172.21.15.62:0/3418758258 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f672c0a2e80 con 0x55b067eb8c70 2023-07-14T16:33:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.720+0000 7f675472db80 10 client.29878 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.720+0000 7f675472db80 1 -- 172.21.15.62:0/3418758258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b067c352b0 con 0x55b067eb8c70 2023-07-14T16:33:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.716+0000 7fbf297fa700 1 -- 172.21.15.62:0/217389110 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf3400f040 con 0x558ef8c9e840 2023-07-14T16:33:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.716+0000 7fbf297fa700 1 -- 172.21.15.62:0/217389110 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbf34041070 con 0x558ef8c9e840 2023-07-14T16:33:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.721+0000 7fbf5b649b80 10 client.29875 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.721+0000 7fbf5b649b80 10 client.29875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.721+0000 7fbf5b649b80 20 client.29875 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.721+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558ef8d089a0 0x558ef8d0adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.721+0000 7fbf5b649b80 4 client.29875 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:33:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.721+0000 7fbf5b649b80 1 -- 172.21.15.62:0/217389110 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ef8baaff0 con 0x558ef8d089a0 2023-07-14T16:33:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.720+0000 7f6721ffb700 1 -- 172.21.15.62:0/3418758258 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f673400f040 con 0x55b067eb8c70 2023-07-14T16:33:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.720+0000 7f6721ffb700 1 -- 172.21.15.62:0/3418758258 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f672c099cf0 con 0x55b067eb8c70 2023-07-14T16:33:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.721+0000 7f675472db80 10 client.29878 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.721+0000 7f675472db80 10 client.29878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.721+0000 7fbf4539a700 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558ef8d089a0 0x558ef8d0adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.721+0000 7f675472db80 20 client.29878 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:45.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.721+0000 7f675472db80 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b067f213c0 0x55b067f237a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:45.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.721+0000 7f675472db80 4 client.29878 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:33:45.699 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.721+0000 7f675472db80 1 -- 172.21.15.62:0/3418758258 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b067dc4050 con 0x55b067f213c0 2023-07-14T16:33:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.721+0000 7f673a59c700 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b067f213c0 0x55b067f237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.722+0000 7f673a59c700 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b067f213c0 0x55b067f237a0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355ad04764d58532 server_cookie=4e9b41cf2b8f4a10 in_seq=0 out_seq=0 2023-07-14T16:33:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.722+0000 7f6721ffb700 10 client.29878 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.722+0000 7fbf4539a700 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558ef8d089a0 0x558ef8d0adb0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e299e78a44a7b0 server_cookie=d74456f2b242e88b in_seq=0 out_seq=0 2023-07-14T16:33:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.722+0000 7fbf297fa700 10 client.29875 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:45.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f6721ffb700 1 -- 172.21.15.62:0/3418758258 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b067dc4050 con 0x55b067f213c0 2023-07-14T16:33:45.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f6721ffb700 10 client.29878 handle_command_reply: tid=0 2023-07-14T16:33:45.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 client.29878 shutdown 2023-07-14T16:33:45.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 -- 172.21.15.62:0/3418758258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 msgr2=0x55b067ec7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ec7660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f672c0a3310 tx=0x7f672c003e10 comp rx=0 tx=0).stop 2023-07-14T16:33:45.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 -- 172.21.15.62:0/3418758258 shutdown_connections 2023-07-14T16:33:45.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b067eb8c70 0x55b067ec7660 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b067eb74d0 0x55b067ec4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b067f213c0 0x55b067f237a0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 --2- 172.21.15.62:0/3418758258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b067eb7e20 0x55b067ec4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f6721ffb700 0 client.29878 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:45.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.737+0000 7f675472db80 1 -- 172.21.15.62:0/3418758258 >> 172.21.15.62:0/3418758258 conn(0x55b067eacda0 msgr2=0x55b067eceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.738+0000 7f675472db80 1 -- 172.21.15.62:0/3418758258 shutdown_connections 2023-07-14T16:33:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.736+0000 7fbf297fa700 1 -- 172.21.15.62:0/217389110 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ef8baaff0 con 0x558ef8d089a0 2023-07-14T16:33:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.736+0000 7fbf297fa700 10 client.29875 handle_command_reply: tid=0 2023-07-14T16:33:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 client.29875 shutdown 2023-07-14T16:33:45.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.738+0000 7f675472db80 1 -- 172.21.15.62:0/3418758258 wait complete. 2023-07-14T16:33:45.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.739+0000 7f675472db80 20 client.29878 trim_cache size 0 max 16384 2023-07-14T16:33:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 -- 172.21.15.62:0/217389110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9e840 msgr2=0x558ef8caaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9e840 0x558ef8caaed0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009930 tx=0x7fbf34009250 comp rx=0 tx=0).stop 2023-07-14T16:33:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 -- 172.21.15.62:0/217389110 shutdown_connections 2023-07-14T16:33:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558ef8cae430 0x558ef8cae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.714 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:33:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:33:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:33:45.715 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:33:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558ef8c9e840 0x558ef8caaed0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009930 tx=0x7fbf34009250 comp rx=0 tx=0).stop 2023-07-14T16:33:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x558ef8d089a0 0x558ef8d0adb0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 --2- 172.21.15.62:0/217389110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558ef8c9f6e0 0x558ef8cab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf5b649b80 1 -- 172.21.15.62:0/217389110 >> 172.21.15.62:0/217389110 conn(0x558ef8c93be0 msgr2=0x558ef8ca2590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.739+0000 7fbf297fa700 0 client.29875 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 -- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67240a12a0 msgr2=0x7f67240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67240a12a0 0x7f67240a3750 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f67300094f0 tx=0x7f6730009440 comp rx=0 tx=0).stop 2023-07-14T16:33:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 -- 172.21.15.62:0/1564332463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f673c10bd60 msgr2=0x7f673c115fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f673c10bd60 0x7f673c115fc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f672c005e50 tx=0x7f672c003920 comp rx=0 tx=0).stop 2023-07-14T16:33:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 -- 172.21.15.62:0/1564332463 shutdown_connections 2023-07-14T16:33:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f67240a12a0 0x7f67240a3750 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f673c10bd60 0x7f673c115fc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f673c070ee0 0x7f673c114a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 --2- 172.21.15.62:0/1564332463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f673c071830 0x7f673c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.718 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.740+0000 7f6740dff700 1 -- 172.21.15.62:0/1564332463 >> 172.21.15.62:0/1564332463 conn(0x7f673c06ca90 msgr2=0x7f673c06ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.742+0000 7fbf5b649b80 1 -- 172.21.15.62:0/217389110 shutdown_connections 2023-07-14T16:33:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.742+0000 7fbf5b649b80 1 -- 172.21.15.62:0/217389110 wait complete. 2023-07-14T16:33:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.742+0000 7fbf5b649b80 20 client.29875 trim_cache size 0 max 16384 2023-07-14T16:33:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.741+0000 7f6740dff700 1 -- 172.21.15.62:0/1564332463 shutdown_connections 2023-07-14T16:33:45.719 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:45.741+0000 7f6740dff700 1 -- 172.21.15.62:0/1564332463 wait complete. 2023-07-14T16:33:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:33:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:33:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:33:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:33:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.743+0000 7fbf47dff700 1 -- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf300a1270 msgr2=0x7fbf300a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.743+0000 7fbf47dff700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf300a1270 0x7fbf300a3720 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbf34004b10 tx=0x7fbf3400b040 comp rx=0 tx=0).stop 2023-07-14T16:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.743+0000 7fbf47dff700 1 -- 172.21.15.62:0/338464480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf401038c0 msgr2=0x7fbf401b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.743+0000 7fbf47dff700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf401038c0 0x7fbf401b6a80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800d980 tx=0x7fbf3800dd40 comp rx=0 tx=0).stop 2023-07-14T16:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.744+0000 7fbf47dff700 1 -- 172.21.15.62:0/338464480 shutdown_connections 2023-07-14T16:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.744+0000 7fbf47dff700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fbf300a1270 0x7fbf300a3720 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.744+0000 7fbf47dff700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf401038c0 0x7fbf401b6a80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.744+0000 7fbf47dff700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf40101d40 0x7fbf401b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.744+0000 7fbf47dff700 1 --2- 172.21.15.62:0/338464480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf40102f30 0x7fbf401b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.744+0000 7fbf47dff700 1 -- 172.21.15.62:0/338464480 >> 172.21.15.62:0/338464480 conn(0x7fbf400fd360 msgr2=0x7fbf400fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.744+0000 7fbf47dff700 1 -- 172.21.15.62:0/338464480 shutdown_connections 2023-07-14T16:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:45.744+0000 7fbf47dff700 1 -- 172.21.15.62:0/338464480 wait complete. 2023-07-14T16:33:45.749 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:33:45.749 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:33:46.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.160+0000 7fdaa29ff700 1 -- 172.21.15.62:0/634700949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 msgr2=0x7fda9c103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:46.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.160+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/634700949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 0x7fda9c103350 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fda90009990 tx=0x7fda90009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.160+0000 7fdaa29ff700 1 -- 172.21.15.62:0/634700949 shutdown_connections 2023-07-14T16:33:46.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.160+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/634700949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c103890 0x7fda9c10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.160+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/634700949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda9c101d10 0x7fda9c102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.160+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/634700949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 0x7fda9c103350 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.160+0000 7fdaa29ff700 1 -- 172.21.15.62:0/634700949 >> 172.21.15.62:0/634700949 conn(0x7fda9c0fd350 msgr2=0x7fda9c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:46.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.161+0000 7fdaa29ff700 1 -- 172.21.15.62:0/634700949 shutdown_connections 2023-07-14T16:33:46.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.161+0000 7fdaa29ff700 1 -- 172.21.15.62:0/634700949 wait complete. 2023-07-14T16:33:46.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.162+0000 7fdaa29ff700 1 Processor -- start 2023-07-14T16:33:46.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.162+0000 7fdaa29ff700 1 -- start start 2023-07-14T16:33:46.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.163+0000 7fdaa29ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c101d10 0x7fda9c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:46.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.163+0000 7fdaa29ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 0x7fda9c06c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:46.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.163+0000 7fdaa29ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda9c103890 0x7fda9c071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:46.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.163+0000 7fdaa29ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c071860 con 0x7fda9c101d10 2023-07-14T16:33:46.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.163+0000 7fdaa29ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0719a0 con 0x7fda9c102f00 2023-07-14T16:33:46.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.163+0000 7fdaa29ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c071b30 con 0x7fda9c103890 2023-07-14T16:33:46.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fdaa0f9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda9c103890 0x7fda9c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:46.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fdaa0f9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda9c103890 0x7fda9c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33400/0 (socket says 172.21.15.62:33400) 2023-07-14T16:33:46.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fdaa0f9c700 1 -- 172.21.15.62:0/2670973950 learned_addr learned my addr 172.21.15.62:0/2670973950 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:46.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fda9b7fe700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 0x7fda9c06c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:46.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fda9bfff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c101d10 0x7fda9c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:46.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fda9b7fe700 1 -- 172.21.15.62:0/2670973950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda9c103890 msgr2=0x7fda9c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:46.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fda9b7fe700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda9c103890 0x7fda9c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.141 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fda9b7fe700 1 -- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c101d10 msgr2=0x7fda9c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:46.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fda9b7fe700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c101d10 0x7fda9c06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.164+0000 7fda9b7fe700 1 -- 172.21.15.62:0/2670973950 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900095f0 con 0x7fda9c102f00 2023-07-14T16:33:46.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.165+0000 7fda9b7fe700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 0x7fda9c06c230 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda900039f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:46.142 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.165+0000 7fda997fa700 1 -- 172.21.15.62:0/2670973950 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9001d070 con 0x7fda9c102f00 2023-07-14T16:33:46.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.165+0000 7fda997fa700 1 -- 172.21.15.62:0/2670973950 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda9000bdf0 con 0x7fda9c102f00 2023-07-14T16:33:46.143 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.165+0000 7fda997fa700 1 -- 172.21.15.62:0/2670973950 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90022e40 con 0x7fda9c102f00 2023-07-14T16:33:46.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.165+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c071db0 con 0x7fda9c102f00 2023-07-14T16:33:46.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.165+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c072290 con 0x7fda9c102f00 2023-07-14T16:33:46.144 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.167+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c066a70 con 0x7fda9c102f00 2023-07-14T16:33:46.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.170+0000 7fda997fa700 1 -- 172.21.15.62:0/2670973950 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fda900225e0 con 0x7fda9c102f00 2023-07-14T16:33:46.145 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.172+0000 7fda997fa700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda840a12d0 0x7fda840a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:46.146 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.172+0000 7fda997fa700 1 -- 172.21.15.62:0/2670973950 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fda900cdcb0 con 0x7fda9c102f00 2023-07-14T16:33:46.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.174+0000 7fda997fa700 1 -- 172.21.15.62:0/2670973950 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fda90097440 con 0x7fda9c102f00 2023-07-14T16:33:46.147 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.175+0000 7fda9bfff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda840a12d0 0x7fda840a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:46.149 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.176+0000 7fda9bfff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda840a12d0 0x7fda840a3780 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fda9c102d60 tx=0x7fda9400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:46.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.654+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda9c10caf0 con 0x7fda9c102f00 2023-07-14T16:33:46.634 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:33:46.634 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:33:23.894782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:33:46.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.659+0000 7fda997fa700 1 -- 172.21.15.62:0/2670973950 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10826 (secure 0 0 0) 0x7fda90027090 con 0x7fda9c102f00 2023-07-14T16:33:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda840a12d0 msgr2=0x7fda840a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda840a12d0 0x7fda840a3780 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fda9c102d60 tx=0x7fda9400d040 comp rx=0 tx=0).stop 2023-07-14T16:33:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 msgr2=0x7fda9c06c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 0x7fda9c06c230 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda900039f0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 shutdown_connections 2023-07-14T16:33:46.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda9c103890 0x7fda9c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda9c101d10 0x7fda9c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fda840a12d0 0x7fda840a3780 secure :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fda9c102d60 tx=0x7fda9400d040 comp rx=0 tx=0).stop 2023-07-14T16:33:46.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 --2- 172.21.15.62:0/2670973950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda9c102f00 0x7fda9c06c230 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda900039f0 comp rx=0 tx=0).stop 2023-07-14T16:33:46.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.668+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 >> 172.21.15.62:0/2670973950 conn(0x7fda9c0fd350 msgr2=0x7fda9c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:46.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.669+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 shutdown_connections 2023-07-14T16:33:46.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:46.669+0000 7fdaa29ff700 1 -- 172.21.15.62:0/2670973950 wait complete. 2023-07-14T16:33:46.644 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 32 2023-07-14T16:33:46.686 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:33:47.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.076+0000 7f11061ff700 1 -- 172.21.15.62:0/1420022157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11001057c0 msgr2=0x7f1100107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.076+0000 7f11061ff700 1 --2- 172.21.15.62:0/1420022157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11001057c0 0x7f1100107c00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b0a0 tx=0x7f10f800b3b0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.076+0000 7f11061ff700 1 -- 172.21.15.62:0/1420022157 shutdown_connections 2023-07-14T16:33:47.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.076+0000 7f11061ff700 1 --2- 172.21.15.62:0/1420022157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11001057c0 0x7f1100107c00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.076+0000 7f11061ff700 1 --2- 172.21.15.62:0/1420022157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11000fd740 0x7f11000fdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.076+0000 7f11061ff700 1 --2- 172.21.15.62:0/1420022157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11000fe120 0x7f1100105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.076+0000 7f11061ff700 1 -- 172.21.15.62:0/1420022157 >> 172.21.15.62:0/1420022157 conn(0x7f11000f9340 msgr2=0x7f11000fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.077+0000 7f11061ff700 1 -- 172.21.15.62:0/1420022157 shutdown_connections 2023-07-14T16:33:47.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.077+0000 7f11061ff700 1 -- 172.21.15.62:0/1420022157 wait complete. 2023-07-14T16:33:47.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.078+0000 7f11061ff700 1 Processor -- start 2023-07-14T16:33:47.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.078+0000 7f11061ff700 1 -- start start 2023-07-14T16:33:47.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.079+0000 7f11061ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11000fd740 0x7f11001935b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.079+0000 7f11061ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11000fe120 0x7f1100198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.079+0000 7f11061ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11001057c0 0x7f1100193b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.079+0000 7f11061ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11001ac170 con 0x7f11000fd740 2023-07-14T16:33:47.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.079+0000 7f11061ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11001ac2b0 con 0x7f11000fe120 2023-07-14T16:33:47.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.079+0000 7f11061ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11001ac420 con 0x7f11001057c0 2023-07-14T16:33:47.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11001057c0 0x7f1100193b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10fffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11001057c0 0x7f1100193b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33412/0 (socket says 172.21.15.62:33412) 2023-07-14T16:33:47.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10fffff700 1 -- 172.21.15.62:0/3668902232 learned_addr learned my addr 172.21.15.62:0/3668902232 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11000fd740 0x7f11001935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10feffd700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11000fe120 0x7f1100198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10fffff700 1 -- 172.21.15.62:0/3668902232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11000fe120 msgr2=0x7f1100198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10fffff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11000fe120 0x7f1100198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10fffff700 1 -- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11000fd740 msgr2=0x7f11001935b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10fffff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11000fd740 0x7f11001935b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.080+0000 7f10fffff700 1 -- 172.21.15.62:0/3668902232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8009d00 con 0x7f11001057c0 2023-07-14T16:33:47.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.081+0000 7f10fffff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11001057c0 0x7f1100193b70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b070 tx=0x7f10f80126d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.081+0000 7f10fcff9700 1 -- 172.21.15.62:0/3668902232 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800e040 con 0x7f11001057c0 2023-07-14T16:33:47.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.081+0000 7f10fcff9700 1 -- 172.21.15.62:0/3668902232 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10f8012e70 con 0x7f11001057c0 2023-07-14T16:33:47.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.081+0000 7f10fcff9700 1 -- 172.21.15.62:0/3668902232 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8005690 con 0x7f11001057c0 2023-07-14T16:33:47.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.081+0000 7f11061ff700 1 -- 172.21.15.62:0/3668902232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11001ac6a0 con 0x7f11001057c0 2023-07-14T16:33:47.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.081+0000 7f11061ff700 1 -- 172.21.15.62:0/3668902232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11001acaf0 con 0x7f11001057c0 2023-07-14T16:33:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.088+0000 7f1119ae9b80 1 Processor -- start 2023-07-14T16:33:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.088+0000 7f10fcff9700 1 -- 172.21.15.62:0/3668902232 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f10f80057f0 con 0x7f11001057c0 2023-07-14T16:33:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.088+0000 7f1119ae9b80 1 -- start start 2023-07-14T16:33:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.089+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f5248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.089+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f5256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.089+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f5260e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.089+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f429cb0 con 0x55718f524e20 2023-07-14T16:33:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.089+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f47ca40 con 0x55718f5244d0 2023-07-14T16:33:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.089+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f475ed0 con 0x55718f525c70 2023-07-14T16:33:47.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.090+0000 7f10feffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f5260e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.091+0000 7f10feffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f5260e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33426/0 (socket says 172.21.15.62:33426) 2023-07-14T16:33:47.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.091+0000 7f10feffd700 1 -- 172.21.15.62:0/609137 learned_addr learned my addr 172.21.15.62:0/609137 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.091+0000 7f10feffd700 1 --2- 172.21.15.62:0/609137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f5248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.092+0000 7f10fcff9700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10e40a12a0 0x7f10e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.092+0000 7f10fcff9700 1 -- 172.21.15.62:0/3668902232 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10f80cdef0 con 0x7f11001057c0 2023-07-14T16:33:47.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.093+0000 7f10feffd700 1 -- 172.21.15.62:0/609137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 msgr2=0x55718f5248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.093+0000 7f10feffd700 1 --2- 172.21.15.62:0/609137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f5248e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.093+0000 7f10feffd700 1 -- 172.21.15.62:0/609137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 msgr2=0x55718f5256e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.093+0000 7f10feffd700 1 --2- 172.21.15.62:0/609137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f5256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.093+0000 7f10feffd700 1 -- 172.21.15.62:0/609137 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718f2a3de0 con 0x55718f525c70 2023-07-14T16:33:47.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.093+0000 7f10feffd700 1 --2- 172.21.15.62:0/609137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f5260e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f10ec014710 tx=0x7f10ec00c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee852f4199b1f43b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.094+0000 7f10f5ffb700 1 -- 172.21.15.62:0/609137 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0142f0 con 0x55718f525c70 2023-07-14T16:33:47.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.094+0000 7f10f5ffb700 1 -- 172.21.15.62:0/609137 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10ec0128e0 con 0x55718f525c70 2023-07-14T16:33:47.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.094+0000 7f10f5ffb700 1 -- 172.21.15.62:0/609137 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00f6c0 con 0x55718f525c70 2023-07-14T16:33:47.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.096+0000 7f10fffff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10e40a12a0 0x7f10e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.097+0000 7f1119ae9b80 1 -- 172.21.15.62:0/609137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 msgr2=0x55718f5260e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.097+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/609137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f5260e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f10ec014710 tx=0x7f10ec00c510 comp rx=0 tx=0).stop 2023-07-14T16:33:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.098+0000 7f10fffff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10e40a12a0 0x7f10e40a3750 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f10f809b040 tx=0x7f10f8097040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.098+0000 7f1119ae9b80 1 -- 172.21.15.62:0/609137 shutdown_connections 2023-07-14T16:33:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.098+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/609137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f5260e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.098+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/609137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f5248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.098+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/609137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f5256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.098+0000 7f1119ae9b80 1 -- 172.21.15.62:0/609137 >> 172.21.15.62:0/609137 conn(0x55718f519da0 msgr2=0x55718f51f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.098+0000 7f1119ae9b80 1 -- 172.21.15.62:0/609137 shutdown_connections 2023-07-14T16:33:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.098+0000 7f1119ae9b80 1 -- 172.21.15.62:0/609137 wait complete. 2023-07-14T16:33:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.099+0000 7f1119ae9b80 1 -- start start 2023-07-14T16:33:47.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.099+0000 7f1119ae9b80 1 Processor -- start 2023-07-14T16:33:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.100+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.100+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f524e20 0x55718f52e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.100+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f525c70 0x55718f535760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.100+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f415ac0 con 0x55718f525c70 2023-07-14T16:33:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.100+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f4157c0 con 0x55718f524e20 2023-07-14T16:33:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.100+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f2f9920 con 0x55718f5244d0 2023-07-14T16:33:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.100+0000 7f1119ae9b80 10 client.0 _ll_register_callbacks cb 0x55718f413760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.101+0000 7f10ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f524e20 0x55718f52e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.101+0000 7f10ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f524e20 0x55718f52e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47876/0 (socket says 172.21.15.62:47876) 2023-07-14T16:33:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.101+0000 7f10ff7fe700 1 -- 172.21.15.62:0/3014080481 learned_addr learned my addr 172.21.15.62:0/3014080481 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.101+0000 7f10feffd700 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.102+0000 7f10feffd700 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f525c70 0x55718f535760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.102+0000 7f10feffd700 1 -- 172.21.15.62:0/3014080481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f524e20 msgr2=0x55718f52e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.102+0000 7f10feffd700 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f524e20 0x55718f52e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.102+0000 7f10feffd700 1 -- 172.21.15.62:0/3014080481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f525c70 msgr2=0x55718f535760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.102+0000 7f10feffd700 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f525c70 0x55718f535760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.102+0000 7f10feffd700 1 -- 172.21.15.62:0/3014080481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec014470 con 0x55718f5244d0 2023-07-14T16:33:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.103+0000 7f10feffd700 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52e1e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c3a0 tx=0x7f10ec016040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6727934d11d2ad50 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.104+0000 7f10e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.104+0000 7f10e2ffd700 1 -- 172.21.15.62:0/3014080481 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec012a50 con 0x55718f5244d0 2023-07-14T16:33:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.104+0000 7f10e2ffd700 1 -- 172.21.15.62:0/3014080481 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10ec012bb0 con 0x55718f5244d0 2023-07-14T16:33:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.104+0000 7f1119ae9b80 1 -- 172.21.15.62:0/3014080481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55718f3cf4f0 con 0x55718f5244d0 2023-07-14T16:33:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.104+0000 7f10e2ffd700 1 -- 172.21.15.62:0/3014080481 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10ec012d70 con 0x55718f5244d0 2023-07-14T16:33:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.104+0000 7f1119ae9b80 10 client.29890 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.104+0000 7f1119ae9b80 1 -- 172.21.15.62:0/3014080481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55718f28cc60 con 0x55718f5244d0 2023-07-14T16:33:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.105+0000 7f10e2ffd700 1 -- 172.21.15.62:0/3014080481 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55718f3cf4f0 con 0x55718f5244d0 2023-07-14T16:33:47.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.105+0000 7f10e2ffd700 1 -- 172.21.15.62:0/3014080481 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f10ec036070 con 0x55718f5244d0 2023-07-14T16:33:47.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.105+0000 7f1119ae9b80 10 client.29890 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:47.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.105+0000 7f1119ae9b80 10 client.29890 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:33:47.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.106+0000 7f1119ae9b80 20 client.29890 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:47.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.106+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55718f539da0 0x55718f53c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.106+0000 7f1119ae9b80 4 client.29890 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.106+0000 7f1119ae9b80 1 -- 172.21.15.62:0/3014080481 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55718f46eeb0 con 0x55718f539da0 2023-07-14T16:33:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.106+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55718f539da0 0x55718f53c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.107+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55718f539da0 0x55718f53c1b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa34ff2151e25e51 server_cookie=1673a6612b92555e in_seq=0 out_seq=0 2023-07-14T16:33:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.107+0000 7f10e2ffd700 10 client.29890 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:33:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.108+0000 7f10e2ffd700 1 -- 172.21.15.62:0/3014080481 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55718f46eeb0 con 0x55718f539da0 2023-07-14T16:33:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.108+0000 7f10e2ffd700 10 client.29890 handle_command_reply: tid=0 2023-07-14T16:33:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.108+0000 7f1119ae9b80 1 client.29890 shutdown 2023-07-14T16:33:47.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.109+0000 7f1119ae9b80 1 -- 172.21.15.62:0/3014080481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 msgr2=0x55718f52e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.109+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52e1e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c3a0 tx=0x7f10ec016040 comp rx=0 tx=0).stop 2023-07-14T16:33:47.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.109+0000 7f1119ae9b80 1 -- 172.21.15.62:0/3014080481 shutdown_connections 2023-07-14T16:33:47.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.109+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55718f539da0 0x55718f53c1b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.109+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f525c70 0x55718f535760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.109+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52e1e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.109+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/3014080481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f524e20 0x55718f52e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.109+0000 7f1119ae9b80 1 -- 172.21.15.62:0/3014080481 >> 172.21.15.62:0/3014080481 conn(0x55718f519da0 msgr2=0x55718f51e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.110+0000 7f10e2ffd700 0 client.29890 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:33:47.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.110+0000 7f1119ae9b80 1 -- 172.21.15.62:0/3014080481 shutdown_connections 2023-07-14T16:33:47.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.110+0000 7f1119ae9b80 1 -- 172.21.15.62:0/3014080481 wait complete. 2023-07-14T16:33:47.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.110+0000 7f1119ae9b80 20 client.29890 trim_cache size 0 max 16384 2023-07-14T16:33:47.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.134+0000 7f1119ae9b80 1 Processor -- start 2023-07-14T16:33:47.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f1119ae9b80 1 -- start start 2023-07-14T16:33:47.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f535720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f525c70 0x55718f536050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f52ddb0 con 0x55718f524e20 2023-07-14T16:33:47.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f520d00 con 0x55718f525c70 2023-07-14T16:33:47.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f52f1f0 con 0x55718f5244d0 2023-07-14T16:33:47.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f10feffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f10feffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33442/0 (socket says 172.21.15.62:33442) 2023-07-14T16:33:47.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.135+0000 7f10feffd700 1 -- 172.21.15.62:0/4129446026 learned_addr learned my addr 172.21.15.62:0/4129446026 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.136+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f535720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.136+0000 7f10feffd700 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f525c70 0x55718f536050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.110 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.136+0000 7f10ff7fe700 1 -- 172.21.15.62:0/4129446026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 msgr2=0x55718f52ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.136+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.136+0000 7f10ff7fe700 1 -- 172.21.15.62:0/4129446026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f525c70 msgr2=0x55718f536050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.136+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f525c70 0x55718f536050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.136+0000 7f10ff7fe700 1 -- 172.21.15.62:0/4129446026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718f2ca250 con 0x55718f524e20 2023-07-14T16:33:47.111 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.137+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f535720 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x55718f53ca20 tx=0x7f10f0018e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cf8c1e2cb329e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.138+0000 7f10f67fc700 1 -- 172.21.15.62:0/4129446026 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0018ea0 con 0x55718f524e20 2023-07-14T16:33:47.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.138+0000 7f10f67fc700 1 -- 172.21.15.62:0/4129446026 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10f000b690 con 0x55718f524e20 2023-07-14T16:33:47.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.138+0000 7f10f67fc700 1 -- 172.21.15.62:0/4129446026 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0015ba0 con 0x55718f524e20 2023-07-14T16:33:47.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.139+0000 7f1119ae9b80 1 -- 172.21.15.62:0/4129446026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 msgr2=0x55718f535720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.139+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f535720 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x55718f53ca20 tx=0x7f10f0018e30 comp rx=0 tx=0).stop 2023-07-14T16:33:47.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.139+0000 7f1119ae9b80 1 -- 172.21.15.62:0/4129446026 shutdown_connections 2023-07-14T16:33:47.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.139+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f525c70 0x55718f536050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.139+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f5244d0 0x55718f52ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.139+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/4129446026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f535720 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.139+0000 7f1119ae9b80 1 -- 172.21.15.62:0/4129446026 >> 172.21.15.62:0/4129446026 conn(0x55718f519da0 msgr2=0x55718f540ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.140+0000 7f1119ae9b80 1 -- 172.21.15.62:0/4129446026 shutdown_connections 2023-07-14T16:33:47.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.140+0000 7f1119ae9b80 1 -- 172.21.15.62:0/4129446026 wait complete. 2023-07-14T16:33:47.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.141+0000 7f1119ae9b80 1 -- start start 2023-07-14T16:33:47.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.141+0000 7f1119ae9b80 1 Processor -- start 2023-07-14T16:33:47.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f531710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f531c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f1119ae9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f534c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f5351b0 con 0x55718f524e20 2023-07-14T16:33:47.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f535320 con 0x55718f5244d0 2023-07-14T16:33:47.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f1119ae9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718f5354e0 con 0x55718f525c70 2023-07-14T16:33:47.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f1119ae9b80 10 client.0 _ll_register_callbacks cb 0x7f10ec014370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:47.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f10feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f531710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f10feffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f534c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f10feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f531710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47886/0 (socket says 172.21.15.62:47886) 2023-07-14T16:33:47.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.142+0000 7f10feffd700 1 -- 172.21.15.62:0/1978850062 learned_addr learned my addr 172.21.15.62:0/1978850062 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.143+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f531c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.143+0000 7f10feffd700 1 -- 172.21.15.62:0/1978850062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 msgr2=0x55718f534c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.143+0000 7f10feffd700 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f534c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.143+0000 7f10feffd700 1 -- 172.21.15.62:0/1978850062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 msgr2=0x55718f531c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.143+0000 7f10feffd700 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f531c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.143+0000 7f10feffd700 1 -- 172.21.15.62:0/1978850062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f000e040 con 0x55718f5244d0 2023-07-14T16:33:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.144+0000 7f10feffd700 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f531710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c3a0 tx=0x7f10ec016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a89991eb55e53f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.144+0000 7f10e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.144+0000 7f10e2ffd700 1 -- 172.21.15.62:0/1978850062 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00c8d0 con 0x55718f5244d0 2023-07-14T16:33:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.144+0000 7f10e2ffd700 1 -- 172.21.15.62:0/1978850062 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f10ec000e40 con 0x55718f5244d0 2023-07-14T16:33:47.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.144+0000 7f1119ae9b80 1 -- 172.21.15.62:0/1978850062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10ec0128e0 con 0x55718f5244d0 2023-07-14T16:33:47.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.145+0000 7f10e2ffd700 1 -- 172.21.15.62:0/1978850062 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10ec011910 con 0x55718f5244d0 2023-07-14T16:33:47.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.145+0000 7f1119ae9b80 10 client.30002 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:47.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.145+0000 7f1119ae9b80 1 -- 172.21.15.62:0/1978850062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55718f2a22b0 con 0x55718f5244d0 2023-07-14T16:33:47.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.146+0000 7f10e2ffd700 1 -- 172.21.15.62:0/1978850062 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10ec0128e0 con 0x55718f5244d0 2023-07-14T16:33:47.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.146+0000 7f10e2ffd700 1 -- 172.21.15.62:0/1978850062 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f10ec014ad0 con 0x55718f5244d0 2023-07-14T16:33:47.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:46 smithi062 ceph-mon[95543]: pgmap v901: 97 pgs: 97 active+clean; 13 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2023-07-14T16:33:47.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:46 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2670973950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:33:47.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.147+0000 7f1119ae9b80 10 client.30002 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:47.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.147+0000 7f1119ae9b80 10 client.30002 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:33:47.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.147+0000 7f1119ae9b80 20 client.30002 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:47.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.147+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55718f58e8a0 0x55718f590cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.147+0000 7f1119ae9b80 4 client.30002 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:33:47.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.147+0000 7f1119ae9b80 1 -- 172.21.15.62:0/1978850062 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55718f431050 con 0x55718f58e8a0 2023-07-14T16:33:47.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.147+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55718f58e8a0 0x55718f590cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.148+0000 7f10ff7fe700 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55718f58e8a0 0x55718f590cb0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98748c8cb59f2faf server_cookie=7cd6d581855cc43b in_seq=0 out_seq=0 2023-07-14T16:33:47.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.148+0000 7f10e2ffd700 10 client.30002 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:33:47.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f10e2ffd700 1 -- 172.21.15.62:0/1978850062 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55718f431050 con 0x55718f58e8a0 2023-07-14T16:33:47.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f10e2ffd700 10 client.30002 handle_command_reply: tid=0 2023-07-14T16:33:47.125 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 client.30002 shutdown 2023-07-14T16:33:47.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 -- 172.21.15.62:0/1978850062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 msgr2=0x55718f531710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f531710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c3a0 tx=0x7f10ec016040 comp rx=0 tx=0).stop 2023-07-14T16:33:47.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 -- 172.21.15.62:0/1978850062 shutdown_connections 2023-07-14T16:33:47.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55718f58e8a0 0x55718f590cb0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55718f525c70 0x55718f534c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55718f5244d0 0x55718f531710 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 --2- 172.21.15.62:0/1978850062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55718f524e20 0x55718f531c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.152+0000 7f1119ae9b80 1 -- 172.21.15.62:0/1978850062 >> 172.21.15.62:0/1978850062 conn(0x55718f519da0 msgr2=0x55718f536050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.153+0000 7f10e2ffd700 0 client.30002 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:33:47.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.153+0000 7f1119ae9b80 1 -- 172.21.15.62:0/1978850062 shutdown_connections 2023-07-14T16:33:47.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.153+0000 7f1119ae9b80 1 -- 172.21.15.62:0/1978850062 wait complete. 2023-07-14T16:33:47.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.155+0000 7f1119ae9b80 20 client.30002 trim_cache size 0 max 16384 2023-07-14T16:33:47.129 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:33:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 -- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10e40a12a0 msgr2=0x7f10e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10e40a12a0 0x7f10e40a3750 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f10f809b040 tx=0x7f10f8097040 comp rx=0 tx=0).stop 2023-07-14T16:33:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 -- 172.21.15.62:0/3668902232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11001057c0 msgr2=0x7f1100193b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11001057c0 0x7f1100193b70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b070 tx=0x7f10f80126d0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 -- 172.21.15.62:0/3668902232 shutdown_connections 2023-07-14T16:33:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11001057c0 0x7f1100193b70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11000fd740 0x7f11001935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f10e40a12a0 0x7f10e40a3750 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 --2- 172.21.15.62:0/3668902232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11000fe120 0x7f1100198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.157+0000 7f11061ff700 1 -- 172.21.15.62:0/3668902232 >> 172.21.15.62:0/3668902232 conn(0x7f11000f9340 msgr2=0x7f11000f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.158+0000 7f11061ff700 1 -- 172.21.15.62:0/3668902232 shutdown_connections 2023-07-14T16:33:47.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.158+0000 7f11061ff700 1 -- 172.21.15.62:0/3668902232 wait complete. 2023-07-14T16:33:47.162 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:33:47.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:46 smithi164 ceph-mon[97427]: pgmap v901: 97 pgs: 97 active+clean; 13 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2023-07-14T16:33:47.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:46 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2670973950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:33:47.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:46 smithi081 ceph-mon[127795]: pgmap v901: 97 pgs: 97 active+clean; 13 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2023-07-14T16:33:47.405 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:46 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2670973950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:33:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.570+0000 7f441e9ff700 1 -- 172.21.15.62:0/2652526909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fdd20 msgr2=0x7f44180fe130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.570+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2652526909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fdd20 0x7f44180fe130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f440800c310 tx=0x7f440800c620 comp rx=0 tx=0).stop 2023-07-14T16:33:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.570+0000 7f441e9ff700 1 -- 172.21.15.62:0/2652526909 shutdown_connections 2023-07-14T16:33:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.570+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2652526909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44180ff8a0 0x7f4418106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.570+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2652526909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fdd20 0x7f44180fe130 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.570+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2652526909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44180fef10 0x7f44180ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.571+0000 7f441e9ff700 1 -- 172.21.15.62:0/2652526909 >> 172.21.15.62:0/2652526909 conn(0x7f44180f9340 msgr2=0x7f44180fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.571+0000 7f441e9ff700 1 -- 172.21.15.62:0/2652526909 shutdown_connections 2023-07-14T16:33:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.571+0000 7f441e9ff700 1 -- 172.21.15.62:0/2652526909 wait complete. 2023-07-14T16:33:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.573+0000 7f441e9ff700 1 Processor -- start 2023-07-14T16:33:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.573+0000 7f441e9ff700 1 -- start start 2023-07-14T16:33:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.574+0000 7f441e9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44180fdd20 0x7f4418071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.574+0000 7f441e9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fef10 0x7f44180715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.574+0000 7f441e9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44180ff8a0 0x7f44181ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.574+0000 7f441e9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4418071ba0 con 0x7f44180ff8a0 2023-07-14T16:33:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.574+0000 7f441e9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4418071d10 con 0x7f44180fdd20 2023-07-14T16:33:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.575+0000 7f440f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fef10 0x7f44180715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.575+0000 7f440f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fef10 0x7f44180715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33454/0 (socket says 172.21.15.62:33454) 2023-07-14T16:33:47.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.575+0000 7f440f7fe700 1 -- 172.21.15.62:0/2451777816 learned_addr learned my addr 172.21.15.62:0/2451777816 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.575+0000 7f441cf9c700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44180ff8a0 0x7f44181ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.575+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44181a37f0 con 0x7f44180fef10 2023-07-14T16:33:47.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.575+0000 7f4417fff700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44180fdd20 0x7f4418071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.576+0000 7f440f7fe700 1 -- 172.21.15.62:0/2451777816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44180fdd20 msgr2=0x7f4418071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.576+0000 7f440f7fe700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44180fdd20 0x7f4418071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.576+0000 7f440f7fe700 1 -- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44180ff8a0 msgr2=0x7f44181ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.576+0000 7f440f7fe700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44180ff8a0 0x7f44181ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.576+0000 7f440f7fe700 1 -- 172.21.15.62:0/2451777816 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440800c050 con 0x7f44180fef10 2023-07-14T16:33:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.576+0000 7f440f7fe700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fef10 0x7f44180715d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f440000b550 tx=0x7f440000b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.577+0000 7f4415ffb700 1 -- 172.21.15.62:0/2451777816 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400011830 con 0x7f44180fef10 2023-07-14T16:33:47.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.577+0000 7f4415ffb700 1 -- 172.21.15.62:0/2451777816 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4400011e20 con 0x7f44180fef10 2023-07-14T16:33:47.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.577+0000 7f4415ffb700 1 -- 172.21.15.62:0/2451777816 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400010dc0 con 0x7f44180fef10 2023-07-14T16:33:47.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.577+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44181a3ad0 con 0x7f44180fef10 2023-07-14T16:33:47.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.577+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44181a4010 con 0x7f44180fef10 2023-07-14T16:33:47.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.583+0000 7f443229db80 1 Processor -- start 2023-07-14T16:33:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.585+0000 7f4415ffb700 1 -- 172.21.15.62:0/2451777816 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4400011990 con 0x7f44180fef10 2023-07-14T16:33:47.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.586+0000 7f443229db80 1 -- start start 2023-07-14T16:33:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.586+0000 7f443229db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb3044d0 0x5598fb3048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.586+0000 7f443229db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb304e20 0x5598fb3056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.586+0000 7f443229db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 0x5598fb3060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.586+0000 7f443229db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb209cb0 con 0x5598fb3044d0 2023-07-14T16:33:47.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.586+0000 7f443229db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb25ca40 con 0x5598fb304e20 2023-07-14T16:33:47.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.586+0000 7f443229db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb255ed0 con 0x5598fb305c70 2023-07-14T16:33:47.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.587+0000 7f4417fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb304e20 0x5598fb3056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.587+0000 7f4417fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb304e20 0x5598fb3056e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47904/0 (socket says 172.21.15.62:47904) 2023-07-14T16:33:47.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.587+0000 7f4417fff700 1 -- 172.21.15.62:0/611826486 learned_addr learned my addr 172.21.15.62:0/611826486 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.587+0000 7f440f7fe700 1 --2- 172.21.15.62:0/611826486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 0x5598fb3060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.588+0000 7f4415ffb700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44040a1220 0x7f44040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.588+0000 7f4417fff700 1 -- 172.21.15.62:0/611826486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 msgr2=0x5598fb3060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.588+0000 7f4417fff700 1 --2- 172.21.15.62:0/611826486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 0x5598fb3060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.588+0000 7f4417fff700 1 -- 172.21.15.62:0/611826486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb3044d0 msgr2=0x5598fb3048e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.588+0000 7f4417fff700 1 --2- 172.21.15.62:0/611826486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb3044d0 0x5598fb3048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.588+0000 7f4417fff700 1 -- 172.21.15.62:0/611826486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598fb083de0 con 0x5598fb304e20 2023-07-14T16:33:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.588+0000 7f4415ffb700 1 -- 172.21.15.62:0/2451777816 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44000cc2e0 con 0x7f44180fef10 2023-07-14T16:33:47.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.588+0000 7f4417fff700 1 --2- 172.21.15.62:0/611826486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb304e20 0x5598fb3056e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4408013520 tx=0x7f4408013550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad6a202d6f31d85 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.589+0000 7f440e7fc700 1 -- 172.21.15.62:0/611826486 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440800f040 con 0x5598fb304e20 2023-07-14T16:33:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.589+0000 7f440e7fc700 1 -- 172.21.15.62:0/611826486 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4408013e80 con 0x5598fb304e20 2023-07-14T16:33:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.589+0000 7f440e7fc700 1 -- 172.21.15.62:0/611826486 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408012a20 con 0x5598fb304e20 2023-07-14T16:33:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.590+0000 7f443229db80 1 -- 172.21.15.62:0/611826486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb304e20 msgr2=0x5598fb3056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.590+0000 7f443229db80 1 --2- 172.21.15.62:0/611826486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb304e20 0x5598fb3056e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4408013520 tx=0x7f4408013550 comp rx=0 tx=0).stop 2023-07-14T16:33:47.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.592+0000 7f441cf9c700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44040a1220 0x7f44040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.592+0000 7f443229db80 1 -- 172.21.15.62:0/611826486 shutdown_connections 2023-07-14T16:33:47.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.592+0000 7f443229db80 1 --2- 172.21.15.62:0/611826486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 0x5598fb3060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.592+0000 7f443229db80 1 --2- 172.21.15.62:0/611826486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb3044d0 0x5598fb3048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.592+0000 7f443229db80 1 --2- 172.21.15.62:0/611826486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb304e20 0x5598fb3056e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.592+0000 7f443229db80 1 -- 172.21.15.62:0/611826486 >> 172.21.15.62:0/611826486 conn(0x5598fb2f9da0 msgr2=0x5598fb2ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.593+0000 7f441cf9c700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44040a1220 0x7f44040a36d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4418072710 tx=0x7f4410008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.595+0000 7f443229db80 1 -- 172.21.15.62:0/611826486 shutdown_connections 2023-07-14T16:33:47.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.595+0000 7f443229db80 1 -- 172.21.15.62:0/611826486 wait complete. 2023-07-14T16:33:47.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.596+0000 7f443229db80 1 -- start start 2023-07-14T16:33:47.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.596+0000 7f443229db80 1 Processor -- start 2023-07-14T16:33:47.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.597+0000 7f443229db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb3044d0 0x5598fb30e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.597+0000 7f443229db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb304e20 0x5598fb30e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.597+0000 7f443229db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb3156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.597+0000 7f443229db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb1f5ac0 con 0x5598fb3044d0 2023-07-14T16:33:47.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.597+0000 7f443229db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb1f57c0 con 0x5598fb305c70 2023-07-14T16:33:47.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.597+0000 7f443229db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb0d9920 con 0x5598fb304e20 2023-07-14T16:33:47.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.597+0000 7f443229db80 10 client.0 _ll_register_callbacks cb 0x5598fb1f3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:47.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.598+0000 7f441cf9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb3156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.598+0000 7f441cf9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb3156c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47920/0 (socket says 172.21.15.62:47920) 2023-07-14T16:33:47.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.598+0000 7f441cf9c700 1 -- 172.21.15.62:0/1516632187 learned_addr learned my addr 172.21.15.62:0/1516632187 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.598+0000 7f441cf9c700 1 -- 172.21.15.62:0/1516632187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb304e20 msgr2=0x5598fb30e680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:33:47.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.598+0000 7f441cf9c700 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb304e20 0x5598fb30e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.598+0000 7f441cf9c700 1 -- 172.21.15.62:0/1516632187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb3044d0 msgr2=0x5598fb30e140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:33:47.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.598+0000 7f441cf9c700 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb3044d0 0x5598fb30e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.598+0000 7f441cf9c700 1 -- 172.21.15.62:0/1516632187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4418106a90 con 0x5598fb305c70 2023-07-14T16:33:47.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.599+0000 7f441cf9c700 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb3156c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4410014d00 tx=0x7f4410004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3125d2fe8fe7eec7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.599+0000 7f43f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:47.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.599+0000 7f43f77fe700 1 -- 172.21.15.62:0/1516632187 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410013de0 con 0x5598fb305c70 2023-07-14T16:33:47.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.599+0000 7f43f77fe700 1 -- 172.21.15.62:0/1516632187 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f441001c040 con 0x5598fb305c70 2023-07-14T16:33:47.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.599+0000 7f443229db80 1 -- 172.21.15.62:0/1516632187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598fb24eeb0 con 0x5598fb305c70 2023-07-14T16:33:47.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.600+0000 7f43f77fe700 1 -- 172.21.15.62:0/1516632187 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4410020410 con 0x5598fb305c70 2023-07-14T16:33:47.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.600+0000 7f443229db80 10 client.30011 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:47.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.600+0000 7f443229db80 1 -- 172.21.15.62:0/1516632187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5598fb06cc60 con 0x5598fb305c70 2023-07-14T16:33:47.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.600+0000 7f43f77fe700 1 -- 172.21.15.62:0/1516632187 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5598fb24eeb0 con 0x5598fb305c70 2023-07-14T16:33:47.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.601+0000 7f43f77fe700 1 -- 172.21.15.62:0/1516632187 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f44100298e0 con 0x5598fb305c70 2023-07-14T16:33:47.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.601+0000 7f443229db80 10 client.30011 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:47.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.601+0000 7f443229db80 10 client.30011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:33:47.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.601+0000 7f443229db80 20 client.30011 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:47.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.601+0000 7f443229db80 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5598fb31a050 0x5598fb31c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.601+0000 7f443229db80 4 client.30011 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:47.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.601+0000 7f443229db80 1 -- 172.21.15.62:0/1516632187 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598fb211050 con 0x5598fb31a050 2023-07-14T16:33:47.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.601+0000 7f4417fff700 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5598fb31a050 0x5598fb31c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.602+0000 7f4417fff700 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5598fb31a050 0x5598fb31c460 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec029e89901fea server_cookie=bbe078a52ca25e3f in_seq=0 out_seq=0 2023-07-14T16:33:47.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.602+0000 7f43f77fe700 10 client.30011 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:33:47.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.603+0000 7f43f77fe700 1 -- 172.21.15.62:0/1516632187 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5598fb211050 con 0x5598fb31a050 2023-07-14T16:33:47.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.603+0000 7f43f77fe700 10 client.30011 handle_command_reply: tid=0 2023-07-14T16:33:47.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.603+0000 7f443229db80 1 client.30011 shutdown 2023-07-14T16:33:47.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.606+0000 7f443229db80 1 -- 172.21.15.62:0/1516632187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 msgr2=0x5598fb3156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.606+0000 7f443229db80 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb3156c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4410014d00 tx=0x7f4410004750 comp rx=0 tx=0).stop 2023-07-14T16:33:47.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.607+0000 7f443229db80 1 -- 172.21.15.62:0/1516632187 shutdown_connections 2023-07-14T16:33:47.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.607+0000 7f443229db80 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5598fb31a050 0x5598fb31c460 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.607+0000 7f443229db80 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb3156c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.607+0000 7f443229db80 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb3044d0 0x5598fb30e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.607+0000 7f443229db80 1 --2- 172.21.15.62:0/1516632187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb304e20 0x5598fb30e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.607+0000 7f443229db80 1 -- 172.21.15.62:0/1516632187 >> 172.21.15.62:0/1516632187 conn(0x5598fb2f9da0 msgr2=0x5598fb2fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.607+0000 7f43f77fe700 0 client.30011 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:33:47.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.611+0000 7f443229db80 1 -- 172.21.15.62:0/1516632187 shutdown_connections 2023-07-14T16:33:47.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.611+0000 7f443229db80 1 -- 172.21.15.62:0/1516632187 wait complete. 2023-07-14T16:33:47.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.612+0000 7f443229db80 20 client.30011 trim_cache size 0 max 16384 2023-07-14T16:33:47.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.637+0000 7f443229db80 1 Processor -- start 2023-07-14T16:33:47.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.637+0000 7f443229db80 1 -- start start 2023-07-14T16:33:47.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.638+0000 7f443229db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb3044d0 0x5598fb315ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.638+0000 7f443229db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb315680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.638+0000 7f443229db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb31c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.638+0000 7f443229db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb30dd10 con 0x5598fb304e20 2023-07-14T16:33:47.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.638+0000 7f443229db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb319cd0 con 0x5598fb305c70 2023-07-14T16:33:47.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.638+0000 7f443229db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb300c60 con 0x5598fb3044d0 2023-07-14T16:33:47.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.638+0000 7f4417fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb3044d0 0x5598fb315ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.638+0000 7f4417fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb3044d0 0x5598fb315ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33476/0 (socket says 172.21.15.62:33476) 2023-07-14T16:33:47.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.639+0000 7f4417fff700 1 -- 172.21.15.62:0/3717333653 learned_addr learned my addr 172.21.15.62:0/3717333653 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.639+0000 7f440f7fe700 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb315680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.640+0000 7f441cf9c700 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb31c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.640+0000 7f4417fff700 1 -- 172.21.15.62:0/3717333653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 msgr2=0x5598fb31c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.640+0000 7f4417fff700 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb31c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.640+0000 7f4417fff700 1 -- 172.21.15.62:0/3717333653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 msgr2=0x5598fb315680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.640+0000 7f4417fff700 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb315680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.640+0000 7f4417fff700 1 -- 172.21.15.62:0/3717333653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598fb0aa250 con 0x5598fb3044d0 2023-07-14T16:33:47.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.640+0000 7f440f7fe700 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb315680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:47.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.640+0000 7f4417fff700 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb3044d0 0x5598fb315ef0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f440800c310 tx=0x7f440800c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6278b933bdac90a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.641+0000 7f440effd700 1 -- 172.21.15.62:0/3717333653 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440800f040 con 0x5598fb3044d0 2023-07-14T16:33:47.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.641+0000 7f440effd700 1 -- 172.21.15.62:0/3717333653 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4408013e80 con 0x5598fb3044d0 2023-07-14T16:33:47.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.641+0000 7f440effd700 1 -- 172.21.15.62:0/3717333653 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408012a30 con 0x5598fb3044d0 2023-07-14T16:33:47.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.642+0000 7f443229db80 1 -- 172.21.15.62:0/3717333653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb3044d0 msgr2=0x5598fb315ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.642+0000 7f443229db80 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb3044d0 0x5598fb315ef0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f440800c310 tx=0x7f440800c620 comp rx=0 tx=0).stop 2023-07-14T16:33:47.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.642+0000 7f443229db80 1 -- 172.21.15.62:0/3717333653 shutdown_connections 2023-07-14T16:33:47.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.642+0000 7f443229db80 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb305c70 0x5598fb31c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.642+0000 7f443229db80 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb3044d0 0x5598fb315ef0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.642+0000 7f443229db80 1 --2- 172.21.15.62:0/3717333653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb315680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.642+0000 7f443229db80 1 -- 172.21.15.62:0/3717333653 >> 172.21.15.62:0/3717333653 conn(0x5598fb2f9da0 msgr2=0x5598fb320da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.642+0000 7f443229db80 1 -- 172.21.15.62:0/3717333653 shutdown_connections 2023-07-14T16:33:47.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.643+0000 7f443229db80 1 -- 172.21.15.62:0/3717333653 wait complete. 2023-07-14T16:33:47.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.643+0000 7f443229db80 1 -- start start 2023-07-14T16:33:47.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.643+0000 7f443229db80 1 Processor -- start 2023-07-14T16:33:47.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.644+0000 7f443229db80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb3044d0 0x5598fb310f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.644+0000 7f443229db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb311490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.618 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.644+0000 7f443229db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 0x5598fb3143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.644+0000 7f443229db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb314930 con 0x5598fb304e20 2023-07-14T16:33:47.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.644+0000 7f443229db80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb314aa0 con 0x5598fb3044d0 2023-07-14T16:33:47.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.644+0000 7f443229db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fb314c60 con 0x5598fb305c70 2023-07-14T16:33:47.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.644+0000 7f443229db80 10 client.0 _ll_register_callbacks cb 0x7f44100148e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:47.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.645+0000 7f4417fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb3044d0 0x5598fb310f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.645+0000 7f4417fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb3044d0 0x5598fb310f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:47934/0 (socket says 172.21.15.62:47934) 2023-07-14T16:33:47.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.645+0000 7f4417fff700 1 -- 172.21.15.62:0/3348106448 learned_addr learned my addr 172.21.15.62:0/3348106448 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:47.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.645+0000 7f440f7fe700 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb311490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.645+0000 7f441cf9c700 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 0x5598fb3143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.646+0000 7f440f7fe700 1 -- 172.21.15.62:0/3348106448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 msgr2=0x5598fb3143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.646+0000 7f440f7fe700 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 0x5598fb3143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.646+0000 7f440f7fe700 1 -- 172.21.15.62:0/3348106448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb3044d0 msgr2=0x5598fb310f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:47.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.646+0000 7f440f7fe700 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb3044d0 0x5598fb310f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.646+0000 7f440f7fe700 1 -- 172.21.15.62:0/3348106448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598fb083de0 con 0x5598fb304e20 2023-07-14T16:33:47.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.646+0000 7f440f7fe700 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb311490 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f44000c9e40 tx=0x7f44000cf8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bbe5abfad91644 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:47.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.647+0000 7f43f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:47.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.648+0000 7f43f77fe700 1 -- 172.21.15.62:0/3348106448 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400099240 con 0x5598fb304e20 2023-07-14T16:33:47.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.648+0000 7f43f77fe700 1 -- 172.21.15.62:0/3348106448 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4400010480 con 0x5598fb304e20 2023-07-14T16:33:47.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.648+0000 7f443229db80 1 -- 172.21.15.62:0/3348106448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598fb0a4f70 con 0x5598fb304e20 2023-07-14T16:33:47.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.648+0000 7f43f77fe700 1 -- 172.21.15.62:0/3348106448 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4400099060 con 0x5598fb304e20 2023-07-14T16:33:47.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.650+0000 7f443229db80 10 client.20469 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:47.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.650+0000 7f443229db80 1 -- 172.21.15.62:0/3348106448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5598fb0822b0 con 0x5598fb304e20 2023-07-14T16:33:47.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.651+0000 7f43f77fe700 1 -- 172.21.15.62:0/3348106448 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44000b1080 con 0x5598fb304e20 2023-07-14T16:33:47.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.651+0000 7f43f77fe700 1 -- 172.21.15.62:0/3348106448 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f44000cb030 con 0x5598fb304e20 2023-07-14T16:33:47.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.651+0000 7f443229db80 10 client.20469 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:47.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.651+0000 7f443229db80 10 client.20469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:33:47.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.651+0000 7f443229db80 20 client.20469 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:47.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.651+0000 7f443229db80 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5598fb36e950 0x5598fb370d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:47.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.651+0000 7f443229db80 4 client.20469 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:33:47.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.651+0000 7f443229db80 1 -- 172.21.15.62:0/3348106448 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598fb0d9f60 con 0x5598fb36e950 2023-07-14T16:33:47.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.652+0000 7f4417fff700 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5598fb36e950 0x5598fb370d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:47.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.654+0000 7f4417fff700 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5598fb36e950 0x5598fb370d60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50dc5ca9a31edf09 server_cookie=31789ccd051af04d in_seq=0 out_seq=0 2023-07-14T16:33:47.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.654+0000 7f43f77fe700 10 client.20469 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:33:47.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f43f77fe700 1 -- 172.21.15.62:0/3348106448 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598fb0d9f60 con 0x5598fb36e950 2023-07-14T16:33:47.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f43f77fe700 10 client.20469 handle_command_reply: tid=0 2023-07-14T16:33:47.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 client.20469 shutdown 2023-07-14T16:33:47.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 -- 172.21.15.62:0/3348106448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 msgr2=0x5598fb311490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb311490 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f44000c9e40 tx=0x7f44000cf8d0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 -- 172.21.15.62:0/3348106448 shutdown_connections 2023-07-14T16:33:47.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x5598fb36e950 0x5598fb370d60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5598fb305c70 0x5598fb3143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5598fb3044d0 0x5598fb310f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 --2- 172.21.15.62:0/3348106448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598fb304e20 0x5598fb311490 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.655+0000 7f443229db80 1 -- 172.21.15.62:0/3348106448 >> 172.21.15.62:0/3348106448 conn(0x5598fb2f9da0 msgr2=0x5598fb31c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.656+0000 7f43f77fe700 0 client.20469 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:33:47.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.656+0000 7f443229db80 1 -- 172.21.15.62:0/3348106448 shutdown_connections 2023-07-14T16:33:47.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.656+0000 7f443229db80 1 -- 172.21.15.62:0/3348106448 wait complete. 2023-07-14T16:33:47.631 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.656+0000 7f443229db80 20 client.20469 trim_cache size 0 max 16384 2023-07-14T16:33:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:33:47.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44040a1220 msgr2=0x7f44040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44040a1220 0x7f44040a36d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4418072710 tx=0x7f4410008040 comp rx=0 tx=0).stop 2023-07-14T16:33:47.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fef10 msgr2=0x7f44180715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:47.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fef10 0x7f44180715d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f440000b550 tx=0x7f440000b860 comp rx=0 tx=0).stop 2023-07-14T16:33:47.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 shutdown_connections 2023-07-14T16:33:47.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44180ff8a0 0x7f44181ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f44040a1220 0x7f44040a36d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44180fdd20 0x7f4418071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 --2- 172.21.15.62:0/2451777816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44180fef10 0x7f44180715d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:47.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.659+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 >> 172.21.15.62:0/2451777816 conn(0x7f44180f9340 msgr2=0x7f44180fb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:47.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.660+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 shutdown_connections 2023-07-14T16:33:47.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:47.660+0000 7f441e9ff700 1 -- 172.21.15.62:0/2451777816 wait complete. 2023-07-14T16:33:47.667 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:33:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 -- 172.21.15.62:0/69441804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc144103670 msgr2=0x7fc144103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 --2- 172.21.15.62:0/69441804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc144103670 0x7fc144103ae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009990 tx=0x7fc13c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 -- 172.21.15.62:0/69441804 shutdown_connections 2023-07-14T16:33:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 --2- 172.21.15.62:0/69441804 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc144103670 0x7fc144103ae0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 --2- 172.21.15.62:0/69441804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1440fe350 0x7fc144100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 --2- 172.21.15.62:0/69441804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc144100cb0 0x7fc144103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 -- 172.21.15.62:0/69441804 >> 172.21.15.62:0/69441804 conn(0x7fc1440fa000 msgr2=0x7fc1440fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 -- 172.21.15.62:0/69441804 shutdown_connections 2023-07-14T16:33:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.059+0000 7fc148bff700 1 -- 172.21.15.62:0/69441804 wait complete. 2023-07-14T16:33:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.060+0000 7fc148bff700 1 Processor -- start 2023-07-14T16:33:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.061+0000 7fc148bff700 1 -- start start 2023-07-14T16:33:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.061+0000 7fc148bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1440fe350 0x7fc14406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.061+0000 7fc148bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc144100cb0 0x7fc14406f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.061+0000 7fc148bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc144103670 0x7fc1441abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.061+0000 7fc148bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc144072400 con 0x7fc144103670 2023-07-14T16:33:48.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.061+0000 7fc148bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc144072540 con 0x7fc144100cb0 2023-07-14T16:33:48.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.061+0000 7fc148bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc144104970 con 0x7fc1440fe350 2023-07-14T16:33:48.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.062+0000 7fc142d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc144103670 0x7fc1441abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.062+0000 7fc142d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc144103670 0x7fc1441abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32998/0 (socket says 172.21.15.62:32998) 2023-07-14T16:33:48.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.062+0000 7fc142d9d700 1 -- 172.21.15.62:0/325542235 learned_addr learned my addr 172.21.15.62:0/325542235 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:48.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.062+0000 7fc14259c700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1440fe350 0x7fc14406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.063+0000 7fc139d9b700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc144100cb0 0x7fc14406f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.063+0000 7fc139d9b700 1 -- 172.21.15.62:0/325542235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1440fe350 msgr2=0x7fc14406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.063+0000 7fc139d9b700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1440fe350 0x7fc14406ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.063+0000 7fc139d9b700 1 -- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc144103670 msgr2=0x7fc1441abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.063+0000 7fc139d9b700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc144103670 0x7fc1441abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.063+0000 7fc139d9b700 1 -- 172.21.15.62:0/325542235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c0095f0 con 0x7fc144100cb0 2023-07-14T16:33:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.063+0000 7fc139d9b700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc144100cb0 0x7fc14406f320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00d950 tx=0x7fc12c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:48.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.064+0000 7fc13bfff700 1 -- 172.21.15.62:0/325542235 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c00f890 con 0x7fc144100cb0 2023-07-14T16:33:48.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.064+0000 7fc13bfff700 1 -- 172.21.15.62:0/325542235 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc12c00fe80 con 0x7fc144100cb0 2023-07-14T16:33:48.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.064+0000 7fc13bfff700 1 -- 172.21.15.62:0/325542235 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c00e3b0 con 0x7fc144100cb0 2023-07-14T16:33:48.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.065+0000 7fc148bff700 1 -- 172.21.15.62:0/325542235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc144104c50 con 0x7fc144100cb0 2023-07-14T16:33:48.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.065+0000 7fc148bff700 1 -- 172.21.15.62:0/325542235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc144105110 con 0x7fc144100cb0 2023-07-14T16:33:48.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.071+0000 7fc13bfff700 1 -- 172.21.15.62:0/325542235 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc12c00f9f0 con 0x7fc144100cb0 2023-07-14T16:33:48.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.072+0000 7fc15c5abb80 1 Processor -- start 2023-07-14T16:33:48.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.072+0000 7fc15c5abb80 1 -- start start 2023-07-14T16:33:48.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ace20 0x55ce2b9ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9adc70 0x55ce2b9ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b8b1cb0 con 0x55ce2b9adc70 2023-07-14T16:33:48.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b904a40 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b8fded0 con 0x55ce2b9ace20 2023-07-14T16:33:48.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc14259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ace20 0x55ce2b9ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc14259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ace20 0x55ce2b9ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33518/0 (socket says 172.21.15.62:33518) 2023-07-14T16:33:48.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.073+0000 7fc14259c700 1 -- 172.21.15.62:0/1199522909 learned_addr learned my addr 172.21.15.62:0/1199522909 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:48.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc142d9d700 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc139d9b700 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9adc70 0x55ce2b9ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc13bfff700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1300a12d0 0x7fc1300a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc142d9d700 1 -- 172.21.15.62:0/1199522909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ace20 msgr2=0x55ce2b9ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc13bfff700 1 -- 172.21.15.62:0/325542235 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc12c0cc190 con 0x7fc144100cb0 2023-07-14T16:33:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc142d9d700 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ace20 0x55ce2b9ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc142d9d700 1 -- 172.21.15.62:0/1199522909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9adc70 msgr2=0x55ce2b9ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc142d9d700 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9adc70 0x55ce2b9ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc142d9d700 1 -- 172.21.15.62:0/1199522909 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce2b72bde0 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.074+0000 7fc142d9d700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1300a12d0 0x7fc1300a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.075+0000 7fc142d9d700 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9ac8e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0098e0 tx=0x7fc13c0040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5195164ce31190 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:48.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.075+0000 7fc138d99700 1 -- 172.21.15.62:0/1199522909 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c01d070 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.075+0000 7fc138d99700 1 -- 172.21.15.62:0/1199522909 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc13c01e040 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.076+0000 7fc138d99700 1 -- 172.21.15.62:0/1199522909 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00bdd0 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.076+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1199522909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ac4d0 msgr2=0x55ce2b9ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.076+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9ac8e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0098e0 tx=0x7fc13c0040d0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.076+0000 7fc142d9d700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1300a12d0 0x7fc1300a3780 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc13c02e000 tx=0x7fc13c02b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:48.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.077+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1199522909 shutdown_connections 2023-07-14T16:33:48.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.077+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9adc70 0x55ce2b9ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.077+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9ac8e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.077+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/1199522909 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ace20 0x55ce2b9ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.077+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1199522909 >> 172.21.15.62:0/1199522909 conn(0x55ce2b9a1da0 msgr2=0x55ce2b9a71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:48.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.077+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1199522909 shutdown_connections 2023-07-14T16:33:48.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.077+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1199522909 wait complete. 2023-07-14T16:33:48.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.077+0000 7fc15c5abb80 1 -- start start 2023-07-14T16:33:48.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.078+0000 7fc15c5abb80 1 Processor -- start 2023-07-14T16:33:48.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.078+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.078+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ace20 0x55ce2b9b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.078+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9adc70 0x55ce2b9bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.079+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b89dac0 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.079+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b89d7c0 con 0x55ce2b9ace20 2023-07-14T16:33:48.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.079+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b781920 con 0x55ce2b9adc70 2023-07-14T16:33:48.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.079+0000 7fc15c5abb80 10 client.0 _ll_register_callbacks cb 0x55ce2b89b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:48.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.079+0000 7fc142d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9adc70 0x55ce2b9bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.079+0000 7fc142d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9adc70 0x55ce2b9bd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33520/0 (socket says 172.21.15.62:33520) 2023-07-14T16:33:48.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.079+0000 7fc142d9d700 1 -- 172.21.15.62:0/3487637741 learned_addr learned my addr 172.21.15.62:0/3487637741 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:48.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.079+0000 7fc139d9b700 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ace20 0x55ce2b9b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.080+0000 7fc142d9d700 1 -- 172.21.15.62:0/3487637741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ace20 msgr2=0x55ce2b9b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:48.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.080+0000 7fc142d9d700 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ace20 0x55ce2b9b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.080+0000 7fc142d9d700 1 -- 172.21.15.62:0/3487637741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ac4d0 msgr2=0x55ce2b9b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:48.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.080+0000 7fc142d9d700 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.080+0000 7fc142d9d700 1 -- 172.21.15.62:0/3487637741 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc144104610 con 0x55ce2b9adc70 2023-07-14T16:33:48.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.080+0000 7fc142d9d700 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9adc70 0x55ce2b9bd740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00b920 tx=0x7fc13c005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2240b6a0042950af server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:48.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.080+0000 7fc129ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:48.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.080+0000 7fc129ffb700 1 -- 172.21.15.62:0/3487637741 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c01d030 con 0x55ce2b9adc70 2023-07-14T16:33:48.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.081+0000 7fc129ffb700 1 -- 172.21.15.62:0/3487637741 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc13c003a60 con 0x55ce2b9adc70 2023-07-14T16:33:48.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.081+0000 7fc15c5abb80 1 -- 172.21.15.62:0/3487637741 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce2b8574f0 con 0x55ce2b9adc70 2023-07-14T16:33:48.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.081+0000 7fc129ffb700 1 -- 172.21.15.62:0/3487637741 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc13c00bdd0 con 0x55ce2b9adc70 2023-07-14T16:33:48.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.081+0000 7fc15c5abb80 10 client.29923 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:48.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.081+0000 7fc15c5abb80 1 -- 172.21.15.62:0/3487637741 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ce2b714c60 con 0x55ce2b9adc70 2023-07-14T16:33:48.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.081+0000 7fc129ffb700 1 -- 172.21.15.62:0/3487637741 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ce2b8574f0 con 0x55ce2b9adc70 2023-07-14T16:33:48.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.083+0000 7fc129ffb700 1 -- 172.21.15.62:0/3487637741 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc13c004500 con 0x55ce2b9adc70 2023-07-14T16:33:48.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.083+0000 7fc15c5abb80 10 client.29923 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:48.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.083+0000 7fc15c5abb80 10 client.29923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:48.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.083+0000 7fc15c5abb80 20 client.29923 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:48.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.083+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ce2b9c1a70 0x55ce2b9c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.083+0000 7fc15c5abb80 4 client.29923 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:48.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.083+0000 7fc15c5abb80 1 -- 172.21.15.62:0/3487637741 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce2b8f6eb0 con 0x55ce2b9c1a70 2023-07-14T16:33:48.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.083+0000 7fc14259c700 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ce2b9c1a70 0x55ce2b9c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.085+0000 7fc14259c700 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ce2b9c1a70 0x55ce2b9c3e50 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947d6b691a3a2b35 server_cookie=f14c7fd0e588af10 in_seq=0 out_seq=0 2023-07-14T16:33:48.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.085+0000 7fc129ffb700 10 client.29923 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:48.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.086+0000 7fc129ffb700 1 -- 172.21.15.62:0/3487637741 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ce2b8f6eb0 con 0x55ce2b9c1a70 2023-07-14T16:33:48.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.086+0000 7fc129ffb700 10 client.29923 handle_command_reply: tid=0 2023-07-14T16:33:48.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.086+0000 7fc15c5abb80 1 client.29923 shutdown 2023-07-14T16:33:48.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.088+0000 7fc15c5abb80 1 -- 172.21.15.62:0/3487637741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9adc70 msgr2=0x55ce2b9bd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.088+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9adc70 0x55ce2b9bd740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00b920 tx=0x7fc13c005fb0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc15c5abb80 1 -- 172.21.15.62:0/3487637741 shutdown_connections 2023-07-14T16:33:48.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9adc70 0x55ce2b9bd740 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ce2b9c1a70 0x55ce2b9c3e50 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/3487637741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9ace20 0x55ce2b9b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc15c5abb80 1 -- 172.21.15.62:0/3487637741 >> 172.21.15.62:0/3487637741 conn(0x55ce2b9a1da0 msgr2=0x55ce2b9a68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc129ffb700 0 client.29923 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc15c5abb80 1 -- 172.21.15.62:0/3487637741 shutdown_connections 2023-07-14T16:33:48.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.089+0000 7fc15c5abb80 1 -- 172.21.15.62:0/3487637741 wait complete. 2023-07-14T16:33:48.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.090+0000 7fc15c5abb80 20 client.29923 trim_cache size 0 max 16384 2023-07-14T16:33:48.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.115+0000 7fc15c5abb80 1 Processor -- start 2023-07-14T16:33:48.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.115+0000 7fc15c5abb80 1 -- start start 2023-07-14T16:33:48.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b9a7f90 con 0x55ce2b9ace20 2023-07-14T16:33:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b9b54c0 con 0x55ce2b9adc70 2023-07-14T16:33:48.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b9a8640 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc14259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc14259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33524/0 (socket says 172.21.15.62:33524) 2023-07-14T16:33:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc14259c700 1 -- 172.21.15.62:0/1500287041 learned_addr learned my addr 172.21.15.62:0/1500287041 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc142d9d700 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.116+0000 7fc139d9b700 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.117+0000 7fc14259c700 1 -- 172.21.15.62:0/1500287041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 msgr2=0x55ce2b9b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.117+0000 7fc14259c700 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.117+0000 7fc14259c700 1 -- 172.21.15.62:0/1500287041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 msgr2=0x55ce2b9bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.117+0000 7fc14259c700 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.117+0000 7fc14259c700 1 -- 172.21.15.62:0/1500287041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce2b9ae560 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.118+0000 7fc14259c700 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b40a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc134004a30 tx=0x7fc134004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1284b4f1c2f4789a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:48.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.118+0000 7fc13959a700 1 -- 172.21.15.62:0/1500287041 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13401b070 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.118+0000 7fc13959a700 1 -- 172.21.15.62:0/1500287041 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc13400f040 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.118+0000 7fc13959a700 1 -- 172.21.15.62:0/1500287041 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13400e040 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1500287041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 msgr2=0x55ce2b9b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b40a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc134004a30 tx=0x7fc134004ad0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1500287041 shutdown_connections 2023-07-14T16:33:48.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9b40a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/1500287041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1500287041 >> 172.21.15.62:0/1500287041 conn(0x55ce2b9a1da0 msgr2=0x55ce2b9c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:48.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1500287041 shutdown_connections 2023-07-14T16:33:48.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.119+0000 7fc15c5abb80 1 -- 172.21.15.62:0/1500287041 wait complete. 2023-07-14T16:33:48.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.120+0000 7fc15c5abb80 1 -- start start 2023-07-14T16:33:48.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.120+0000 7fc15c5abb80 1 Processor -- start 2023-07-14T16:33:48.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.121+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.121+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.121+0000 7fc15c5abb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.121+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b9ba070 con 0x55ce2b9ace20 2023-07-14T16:33:48.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.121+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b9ba1e0 con 0x55ce2b9adc70 2023-07-14T16:33:48.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.121+0000 7fc15c5abb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2b9ba3a0 con 0x55ce2b9ac4d0 2023-07-14T16:33:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.121+0000 7fc15c5abb80 10 client.0 _ll_register_callbacks cb 0x7fc12c0956c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc139d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc139d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33034/0 (socket says 172.21.15.62:33034) 2023-07-14T16:33:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc139d9b700 1 -- 172.21.15.62:0/953020613 learned_addr learned my addr 172.21.15.62:0/953020613 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc142d9d700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc14259c700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9bcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc139d9b700 1 -- 172.21.15.62:0/953020613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 msgr2=0x55ce2b9bcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:48.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc139d9b700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9bcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc139d9b700 1 -- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 msgr2=0x55ce2b9b9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:48.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc139d9b700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.099 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.122+0000 7fc139d9b700 1 -- 172.21.15.62:0/953020613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc134007460 con 0x55ce2b9ace20 2023-07-14T16:33:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc139d9b700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd4f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc12c095fc0 tx=0x7fc12c0b9130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e4ac55704c54e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc129ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc14259c700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9bcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc129ffb700 1 -- 172.21.15.62:0/953020613 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c098eb0 con 0x55ce2b9ace20 2023-07-14T16:33:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc142d9d700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc129ffb700 1 -- 172.21.15.62:0/953020613 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc12c0106b0 con 0x55ce2b9ace20 2023-07-14T16:33:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc15c5abb80 1 -- 172.21.15.62:0/953020613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc13c01e040 con 0x55ce2b9ace20 2023-07-14T16:33:48.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc129ffb700 1 -- 172.21.15.62:0/953020613 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc12c00fca0 con 0x55ce2b9ace20 2023-07-14T16:33:48.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc15c5abb80 10 client.20484 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:48.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.123+0000 7fc15c5abb80 1 -- 172.21.15.62:0/953020613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ce2b752250 con 0x55ce2b9ace20 2023-07-14T16:33:48.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.124+0000 7fc129ffb700 1 -- 172.21.15.62:0/953020613 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc12c0b5070 con 0x55ce2b9ace20 2023-07-14T16:33:48.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.124+0000 7fc129ffb700 1 -- 172.21.15.62:0/953020613 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc12c09d160 con 0x55ce2b9ace20 2023-07-14T16:33:48.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.124+0000 7fc15c5abb80 10 client.20484 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:48.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.124+0000 7fc15c5abb80 10 client.20484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:48.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.124+0000 7fc15c5abb80 20 client.20484 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:48.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.124+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ce2ba16f10 0x55ce2ba19320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:48.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.124+0000 7fc15c5abb80 4 client.20484 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:33:48.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.124+0000 7fc15c5abb80 1 -- 172.21.15.62:0/953020613 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ce2b8b9050 con 0x55ce2ba16f10 2023-07-14T16:33:48.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.125+0000 7fc14259c700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ce2ba16f10 0x55ce2ba19320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:48.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.125+0000 7fc14259c700 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ce2ba16f10 0x55ce2ba19320 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f367e9f988c81a6e server_cookie=b8659be5cc288cd2 in_seq=0 out_seq=0 2023-07-14T16:33:48.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.125+0000 7fc129ffb700 10 client.20484 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:48.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc129ffb700 1 -- 172.21.15.62:0/953020613 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ce2b8b9050 con 0x55ce2ba16f10 2023-07-14T16:33:48.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc129ffb700 10 client.20484 handle_command_reply: tid=0 2023-07-14T16:33:48.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 client.20484 shutdown 2023-07-14T16:33:48.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 -- 172.21.15.62:0/953020613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 msgr2=0x55ce2b9bd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd4f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc12c095fc0 tx=0x7fc12c0b9130 comp rx=0 tx=0).stop 2023-07-14T16:33:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 -- 172.21.15.62:0/953020613 shutdown_connections 2023-07-14T16:33:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55ce2ba16f10 0x55ce2ba19320 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce2b9adc70 0x55ce2b9b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ce2b9ac4d0 0x55ce2b9bcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 --2- 172.21.15.62:0/953020613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce2b9ace20 0x55ce2b9bd4f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc15c5abb80 1 -- 172.21.15.62:0/953020613 >> 172.21.15.62:0/953020613 conn(0x55ce2b9a1da0 msgr2=0x55ce2b9b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.126+0000 7fc129ffb700 0 client.20484 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:48.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.127+0000 7fc15c5abb80 1 -- 172.21.15.62:0/953020613 shutdown_connections 2023-07-14T16:33:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.127+0000 7fc15c5abb80 1 -- 172.21.15.62:0/953020613 wait complete. 2023-07-14T16:33:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.127+0000 7fc15c5abb80 20 client.20484 trim_cache size 0 max 16384 2023-07-14T16:33:48.106 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:33:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 -- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1300a12d0 msgr2=0x7fc1300a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1300a12d0 0x7fc1300a3780 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc13c02e000 tx=0x7fc13c02b000 comp rx=0 tx=0).stop 2023-07-14T16:33:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 -- 172.21.15.62:0/325542235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc144100cb0 msgr2=0x7fc14406f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc144100cb0 0x7fc14406f320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00d950 tx=0x7fc12c00dc60 comp rx=0 tx=0).stop 2023-07-14T16:33:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 -- 172.21.15.62:0/325542235 shutdown_connections 2023-07-14T16:33:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc144103670 0x7fc1441abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1440fe350 0x7fc14406ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc1300a12d0 0x7fc1300a3780 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 --2- 172.21.15.62:0/325542235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc144100cb0 0x7fc14406f320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.128+0000 7fc148bff700 1 -- 172.21.15.62:0/325542235 >> 172.21.15.62:0/325542235 conn(0x7fc1440fa000 msgr2=0x7fc1440feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.129+0000 7fc148bff700 1 -- 172.21.15.62:0/325542235 shutdown_connections 2023-07-14T16:33:48.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:48.129+0000 7fc148bff700 1 -- 172.21.15.62:0/325542235 wait complete. 2023-07-14T16:33:49.137 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:33:49.137 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:33:49.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:48 smithi062 ceph-mon[95543]: pgmap v902: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 4.63k op/s 2023-07-14T16:33:49.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:48 smithi164 ceph-mon[97427]: pgmap v902: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 4.63k op/s 2023-07-14T16:33:49.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:48 smithi081 ceph-mon[127795]: pgmap v902: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 4.63k op/s 2023-07-14T16:33:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.531+0000 7f54393ff700 1 -- 172.21.15.62:0/1119913602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434074ed0 msgr2=0x7f54340752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.531+0000 7f54393ff700 1 --2- 172.21.15.62:0/1119913602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434074ed0 0x7f54340752e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5424009990 tx=0x7f5424009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.531+0000 7f54393ff700 1 -- 172.21.15.62:0/1119913602 shutdown_connections 2023-07-14T16:33:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.531+0000 7f54393ff700 1 --2- 172.21.15.62:0/1119913602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5434076ac0 0x7f5434077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.531+0000 7f54393ff700 1 --2- 172.21.15.62:0/1119913602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434074ed0 0x7f54340752e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.531+0000 7f54393ff700 1 --2- 172.21.15.62:0/1119913602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434076110 0x7f5434076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.532+0000 7f54393ff700 1 -- 172.21.15.62:0/1119913602 >> 172.21.15.62:0/1119913602 conn(0x7f54340fd350 msgr2=0x7f54340ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:49.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.533+0000 7f54393ff700 1 -- 172.21.15.62:0/1119913602 shutdown_connections 2023-07-14T16:33:49.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.534+0000 7f54393ff700 1 -- 172.21.15.62:0/1119913602 wait complete. 2023-07-14T16:33:49.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.535+0000 7f54393ff700 1 Processor -- start 2023-07-14T16:33:49.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.535+0000 7f54393ff700 1 -- start start 2023-07-14T16:33:49.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54393ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434074ed0 0x7f543410c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54393ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5434076110 0x7f543410cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54393ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434076ac0 0x7f54341abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54393ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543410fb90 con 0x7f5434076110 2023-07-14T16:33:49.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54393ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543410fd00 con 0x7f5434074ed0 2023-07-14T16:33:49.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54393ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54341ac4a0 con 0x7f5434076ac0 2023-07-14T16:33:49.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54327fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5434076110 0x7f543410cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54327fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5434076110 0x7f543410cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33062/0 (socket says 172.21.15.62:33062) 2023-07-14T16:33:49.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54327fc700 1 -- 172.21.15.62:0/530301815 learned_addr learned my addr 172.21.15.62:0/530301815 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:49.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f5432ffd700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434074ed0 0x7f543410c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.536+0000 7f54337fe700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434076ac0 0x7f54341abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.537+0000 7f54337fe700 1 -- 172.21.15.62:0/530301815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434074ed0 msgr2=0x7f543410c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.537+0000 7f54337fe700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434074ed0 0x7f543410c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.537+0000 7f54337fe700 1 -- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5434076110 msgr2=0x7f543410cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.537+0000 7f54337fe700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5434076110 0x7f543410cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.537+0000 7f54337fe700 1 -- 172.21.15.62:0/530301815 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240095f0 con 0x7f5434076ac0 2023-07-14T16:33:49.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.537+0000 7f5432ffd700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434074ed0 0x7f543410c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:49.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.539+0000 7f54337fe700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434076ac0 0x7f54341abf60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f542c007f10 tx=0x7f542c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:49.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.540+0000 7f541bfff700 1 -- 172.21.15.62:0/530301815 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c009ea0 con 0x7f5434076ac0 2023-07-14T16:33:49.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.540+0000 7f541bfff700 1 -- 172.21.15.62:0/530301815 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f542c00f040 con 0x7f5434076ac0 2023-07-14T16:33:49.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.540+0000 7f541bfff700 1 -- 172.21.15.62:0/530301815 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0045e0 con 0x7f5434076ac0 2023-07-14T16:33:49.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.540+0000 7f54393ff700 1 -- 172.21.15.62:0/530301815 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54341ac640 con 0x7f5434076ac0 2023-07-14T16:33:49.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.540+0000 7f54393ff700 1 -- 172.21.15.62:0/530301815 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54341aca70 con 0x7f5434076ac0 2023-07-14T16:33:49.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.549+0000 7f544cd03b80 1 Processor -- start 2023-07-14T16:33:49.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.551+0000 7f541bfff700 1 -- 172.21.15.62:0/530301815 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f542c004740 con 0x7f5434076ac0 2023-07-14T16:33:49.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.551+0000 7f544cd03b80 1 -- start start 2023-07-14T16:33:49.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.552+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316550 0x55925d316960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.552+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d317760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.552+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d318160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.552+0000 7f544cd03b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d21bd30 con 0x55925d316ea0 2023-07-14T16:33:49.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.552+0000 7f544cd03b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d26eac0 con 0x55925d317cf0 2023-07-14T16:33:49.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.552+0000 7f544cd03b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d267f50 con 0x55925d316550 2023-07-14T16:33:49.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.553+0000 7f54327fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d318160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.553+0000 7f54327fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d318160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48038/0 (socket says 172.21.15.62:48038) 2023-07-14T16:33:49.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.553+0000 7f54327fc700 1 -- 172.21.15.62:0/2026419831 learned_addr learned my addr 172.21.15.62:0/2026419831 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:49.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.553+0000 7f54327fc700 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316550 0x55925d316960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.553+0000 7f5432ffd700 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d317760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.554+0000 7f54327fc700 1 -- 172.21.15.62:0/2026419831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316550 msgr2=0x55925d316960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:49.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.554+0000 7f54327fc700 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316550 0x55925d316960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.554+0000 7f54327fc700 1 -- 172.21.15.62:0/2026419831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 msgr2=0x55925d317760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:49.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.554+0000 7f54327fc700 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d317760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.554+0000 7f54327fc700 1 -- 172.21.15.62:0/2026419831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55925d095e60 con 0x55925d317cf0 2023-07-14T16:33:49.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.554+0000 7f541bfff700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f541c0a1220 0x7f541c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.554+0000 7f54327fc700 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d318160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5428011930 tx=0x7f542800a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caad7e6f5ddcfb1e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:49.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.554+0000 7f541bfff700 1 -- 172.21.15.62:0/530301815 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f542c0cc020 con 0x7f5434076ac0 2023-07-14T16:33:49.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.555+0000 7f5432ffd700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f541c0a1220 0x7f541c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.556+0000 7f54197fa700 1 -- 172.21.15.62:0/2026419831 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428004830 con 0x55925d317cf0 2023-07-14T16:33:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.556+0000 7f54197fa700 1 -- 172.21.15.62:0/2026419831 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5428004e20 con 0x55925d317cf0 2023-07-14T16:33:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.556+0000 7f54197fa700 1 -- 172.21.15.62:0/2026419831 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800f640 con 0x55925d317cf0 2023-07-14T16:33:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.557+0000 7f544cd03b80 1 -- 172.21.15.62:0/2026419831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 msgr2=0x55925d318160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.557+0000 7f5432ffd700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f541c0a1220 0x7f541c0a36d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5424006010 tx=0x7f5424005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.557+0000 7f544cd03b80 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d318160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5428011930 tx=0x7f542800a8c0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.557+0000 7f544cd03b80 1 -- 172.21.15.62:0/2026419831 shutdown_connections 2023-07-14T16:33:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.558+0000 7f544cd03b80 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d318160 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.558+0000 7f544cd03b80 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316550 0x55925d316960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.558+0000 7f544cd03b80 1 --2- 172.21.15.62:0/2026419831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d317760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.558+0000 7f544cd03b80 1 -- 172.21.15.62:0/2026419831 >> 172.21.15.62:0/2026419831 conn(0x55925d30be20 msgr2=0x55925d311250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.558+0000 7f544cd03b80 1 -- 172.21.15.62:0/2026419831 shutdown_connections 2023-07-14T16:33:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.558+0000 7f544cd03b80 1 -- 172.21.15.62:0/2026419831 wait complete. 2023-07-14T16:33:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.558+0000 7f544cd03b80 1 -- start start 2023-07-14T16:33:49.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.559+0000 7f544cd03b80 1 Processor -- start 2023-07-14T16:33:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.559+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 0x55925d320260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.559+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d3207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.559+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.559+0000 7f544cd03b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d207b40 con 0x55925d316550 2023-07-14T16:33:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.559+0000 7f544cd03b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d207840 con 0x55925d317cf0 2023-07-14T16:33:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.559+0000 7f544cd03b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d0eb9a0 con 0x55925d316ea0 2023-07-14T16:33:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.559+0000 7f544cd03b80 10 client.0 _ll_register_callbacks cb 0x55925d2057e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:49.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54337fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54337fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3277e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48042/0 (socket says 172.21.15.62:48042) 2023-07-14T16:33:49.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54337fe700 1 -- 172.21.15.62:0/1656843448 learned_addr learned my addr 172.21.15.62:0/1656843448 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:49.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54327fc700 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d3207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54327fc700 1 -- 172.21.15.62:0/1656843448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 msgr2=0x55925d3277e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:49.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54327fc700 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3277e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54327fc700 1 -- 172.21.15.62:0/1656843448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 msgr2=0x55925d320260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:49.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54327fc700 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 0x55925d320260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.562+0000 7f54327fc700 1 -- 172.21.15.62:0/1656843448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5428019040 con 0x55925d316ea0 2023-07-14T16:33:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.563+0000 7f54327fc700 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d3207a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f542800b010 tx=0x7f5428005360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93bbdbb0d5bb773f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.563+0000 7f5416ff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.563+0000 7f5416ff5700 1 -- 172.21.15.62:0/1656843448 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428004a20 con 0x55925d316ea0 2023-07-14T16:33:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.563+0000 7f5416ff5700 1 -- 172.21.15.62:0/1656843448 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5428000e40 con 0x55925d316ea0 2023-07-14T16:33:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.563+0000 7f544cd03b80 1 -- 172.21.15.62:0/1656843448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55925d1c1570 con 0x55925d316ea0 2023-07-14T16:33:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.564+0000 7f5416ff5700 1 -- 172.21.15.62:0/1656843448 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5428004b80 con 0x55925d316ea0 2023-07-14T16:33:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.564+0000 7f544cd03b80 10 client.29941 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.564+0000 7f544cd03b80 1 -- 172.21.15.62:0/1656843448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55925d07ece0 con 0x55925d316ea0 2023-07-14T16:33:49.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.567+0000 7f5416ff5700 1 -- 172.21.15.62:0/1656843448 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55925d1c1570 con 0x55925d316ea0 2023-07-14T16:33:49.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.567+0000 7f5416ff5700 1 -- 172.21.15.62:0/1656843448 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f542801e5a0 con 0x55925d316ea0 2023-07-14T16:33:49.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.567+0000 7f544cd03b80 10 client.29941 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:49.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.567+0000 7f544cd03b80 10 client.29941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:49.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.567+0000 7f544cd03b80 20 client.29941 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:49.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.567+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55925d33ce60 0x55925d33f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.567+0000 7f544cd03b80 4 client.29941 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:49.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.567+0000 7f544cd03b80 1 -- 172.21.15.62:0/1656843448 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55925d260f30 con 0x55925d33ce60 2023-07-14T16:33:49.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.572+0000 7f5432ffd700 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55925d33ce60 0x55925d33f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.573+0000 7f5432ffd700 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55925d33ce60 0x55925d33f270 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842e34f75a780786 server_cookie=d27d3089e80a0e4a in_seq=0 out_seq=0 2023-07-14T16:33:49.546 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.573+0000 7f5416ff5700 10 client.29941 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:49.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.574+0000 7f5416ff5700 1 -- 172.21.15.62:0/1656843448 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55925d260f30 con 0x55925d33ce60 2023-07-14T16:33:49.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.574+0000 7f5416ff5700 10 client.29941 handle_command_reply: tid=0 2023-07-14T16:33:49.547 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.574+0000 7f544cd03b80 1 client.29941 shutdown 2023-07-14T16:33:49.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 -- 172.21.15.62:0/1656843448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 msgr2=0x55925d3207a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d3207a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f542800b010 tx=0x7f5428005360 comp rx=0 tx=0).stop 2023-07-14T16:33:49.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 -- 172.21.15.62:0/1656843448 shutdown_connections 2023-07-14T16:33:49.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55925d33ce60 0x55925d33f270 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 0x55925d320260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1656843448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d3207a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 -- 172.21.15.62:0/1656843448 >> 172.21.15.62:0/1656843448 conn(0x55925d30be20 msgr2=0x55925d310930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:49.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f5416ff5700 0 client.29941 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:49.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.575+0000 7f544cd03b80 1 -- 172.21.15.62:0/1656843448 shutdown_connections 2023-07-14T16:33:49.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.576+0000 7f544cd03b80 1 -- 172.21.15.62:0/1656843448 wait complete. 2023-07-14T16:33:49.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.576+0000 7f544cd03b80 20 client.29941 trim_cache size 0 max 16384 2023-07-14T16:33:49.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.613+0000 7f544cd03b80 1 Processor -- start 2023-07-14T16:33:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.615+0000 7f544cd03b80 1 -- start start 2023-07-14T16:33:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.615+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d316550 0x55925d310930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.615+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d3277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.615+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d317cf0 0x55925d33ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.615+0000 7f544cd03b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d31f710 con 0x55925d316ea0 2023-07-14T16:33:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.615+0000 7f544cd03b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d312540 con 0x55925d316550 2023-07-14T16:33:49.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.615+0000 7f544cd03b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d33cc40 con 0x55925d317cf0 2023-07-14T16:33:49.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f5432ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d316550 0x55925d310930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f5432ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d316550 0x55925d310930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48056/0 (socket says 172.21.15.62:48056) 2023-07-14T16:33:49.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f5432ffd700 1 -- 172.21.15.62:0/1621293124 learned_addr learned my addr 172.21.15.62:0/1621293124 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:49.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f54337fe700 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d317cf0 0x55925d33ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f54327fc700 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d3277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f54337fe700 1 -- 172.21.15.62:0/1621293124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d316550 msgr2=0x55925d310930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f54337fe700 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d316550 0x55925d310930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f54337fe700 1 -- 172.21.15.62:0/1621293124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 msgr2=0x55925d3277a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f54337fe700 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d3277a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f54337fe700 1 -- 172.21.15.62:0/1621293124 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55925d321430 con 0x55925d317cf0 2023-07-14T16:33:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.616+0000 7f5432ffd700 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d316550 0x55925d310930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.617+0000 7f54327fc700 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d3277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.617+0000 7f54337fe700 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d317cf0 0x55925d33ee70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f542c0a1360 tx=0x7f542c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ad9a874446a64 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.617+0000 7f5419ffb700 1 -- 172.21.15.62:0/1621293124 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c003d00 con 0x55925d317cf0 2023-07-14T16:33:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.617+0000 7f5419ffb700 1 -- 172.21.15.62:0/1621293124 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f542c0ae070 con 0x55925d317cf0 2023-07-14T16:33:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.617+0000 7f5419ffb700 1 -- 172.21.15.62:0/1621293124 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c09d170 con 0x55925d317cf0 2023-07-14T16:33:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.618+0000 7f544cd03b80 1 -- 172.21.15.62:0/1621293124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d317cf0 msgr2=0x55925d33ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.618+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d317cf0 0x55925d33ee70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f542c0a1360 tx=0x7f542c004ad0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.619+0000 7f544cd03b80 1 -- 172.21.15.62:0/1621293124 shutdown_connections 2023-07-14T16:33:49.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.619+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d317cf0 0x55925d33ee70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.619+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d316550 0x55925d310930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.619+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1621293124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316ea0 0x55925d3277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.619+0000 7f544cd03b80 1 -- 172.21.15.62:0/1621293124 >> 172.21.15.62:0/1621293124 conn(0x55925d30be20 msgr2=0x55925d31eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.620+0000 7f544cd03b80 1 -- 172.21.15.62:0/1621293124 shutdown_connections 2023-07-14T16:33:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.623+0000 7f544cd03b80 1 -- 172.21.15.62:0/1621293124 wait complete. 2023-07-14T16:33:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.623+0000 7f544cd03b80 1 -- start start 2023-07-14T16:33:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.624+0000 7f544cd03b80 1 Processor -- start 2023-07-14T16:33:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.624+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 0x55925d3275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.624+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d324080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.624+0000 7f544cd03b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.624+0000 7f544cd03b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d324ca0 con 0x55925d316550 2023-07-14T16:33:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.624+0000 7f544cd03b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d390d10 con 0x55925d317cf0 2023-07-14T16:33:49.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.625+0000 7f544cd03b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925d390e50 con 0x55925d316ea0 2023-07-14T16:33:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.625+0000 7f544cd03b80 10 client.0 _ll_register_callbacks cb 0x7f5424009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.625+0000 7f54327fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d324080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.625+0000 7f54327fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d324080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33592/0 (socket says 172.21.15.62:33592) 2023-07-14T16:33:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.625+0000 7f54327fc700 1 -- 172.21.15.62:0/1619497842 learned_addr learned my addr 172.21.15.62:0/1619497842 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:49.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.626+0000 7f54337fe700 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.626+0000 7f5432ffd700 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 0x55925d3275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.626+0000 7f54337fe700 1 -- 172.21.15.62:0/1619497842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 msgr2=0x55925d324080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:49.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.626+0000 7f54337fe700 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d324080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.626+0000 7f54337fe700 1 -- 172.21.15.62:0/1619497842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 msgr2=0x55925d3275e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.626+0000 7f54337fe700 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 0x55925d3275e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.626+0000 7f54337fe700 1 -- 172.21.15.62:0/1619497842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5434077c40 con 0x55925d317cf0 2023-07-14T16:33:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.627+0000 7f54337fe700 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3245c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f542c09d630 tx=0x7f542c003d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c1922dcbc4e70a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.627+0000 7f5416ff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.628+0000 7f5416ff5700 1 -- 172.21.15.62:0/1619497842 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0c7070 con 0x55925d317cf0 2023-07-14T16:33:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.628+0000 7f5416ff5700 1 -- 172.21.15.62:0/1619497842 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f542c0a1890 con 0x55925d317cf0 2023-07-14T16:33:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.628+0000 7f544cd03b80 1 -- 172.21.15.62:0/1619497842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5428004e20 con 0x55925d317cf0 2023-07-14T16:33:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.629+0000 7f5416ff5700 1 -- 172.21.15.62:0/1619497842 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f542c0977e0 con 0x55925d317cf0 2023-07-14T16:33:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.629+0000 7f544cd03b80 10 client.30047 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:49.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.629+0000 7f544cd03b80 1 -- 172.21.15.62:0/1619497842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55925d07ece0 con 0x55925d317cf0 2023-07-14T16:33:49.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.630+0000 7f5416ff5700 1 -- 172.21.15.62:0/1619497842 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5428004e20 con 0x55925d317cf0 2023-07-14T16:33:49.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.630+0000 7f5416ff5700 1 -- 172.21.15.62:0/1619497842 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f542c098930 con 0x55925d317cf0 2023-07-14T16:33:49.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.630+0000 7f544cd03b80 10 client.30047 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.630+0000 7f544cd03b80 10 client.30047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.630+0000 7f544cd03b80 20 client.30047 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.630+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55925d3924e0 0x55925d3948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.630+0000 7f544cd03b80 4 client.30047 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:33:49.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.630+0000 7f544cd03b80 1 -- 172.21.15.62:0/1619497842 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55925d2230d0 con 0x55925d3924e0 2023-07-14T16:33:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.631+0000 7f5432ffd700 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55925d3924e0 0x55925d3948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.632+0000 7f5432ffd700 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55925d3924e0 0x55925d3948f0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e06e45db4dc8ad0 server_cookie=fe032cd40cdf899e in_seq=0 out_seq=0 2023-07-14T16:33:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.633+0000 7f5416ff5700 10 client.30047 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.634+0000 7f5416ff5700 1 -- 172.21.15.62:0/1619497842 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55925d2230d0 con 0x55925d3924e0 2023-07-14T16:33:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.634+0000 7f5416ff5700 10 client.30047 handle_command_reply: tid=0 2023-07-14T16:33:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.634+0000 7f544cd03b80 1 client.30047 shutdown 2023-07-14T16:33:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f544cd03b80 1 -- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 msgr2=0x55925d3245c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3245c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f542c09d630 tx=0x7f542c003d80 comp rx=0 tx=0).stop 2023-07-14T16:33:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f544cd03b80 1 -- 172.21.15.62:0/1619497842 shutdown_connections 2023-07-14T16:33:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55925d3924e0 0x55925d3948f0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55925d317cf0 0x55925d3245c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55925d316550 0x55925d3275e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f544cd03b80 1 --2- 172.21.15.62:0/1619497842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55925d316ea0 0x55925d324080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f5416ff5700 0 client.30047 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.635+0000 7f544cd03b80 1 -- 172.21.15.62:0/1619497842 >> 172.21.15.62:0/1619497842 conn(0x55925d30be20 msgr2=0x55925d310930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.636+0000 7f544cd03b80 1 -- 172.21.15.62:0/1619497842 shutdown_connections 2023-07-14T16:33:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.636+0000 7f544cd03b80 1 -- 172.21.15.62:0/1619497842 wait complete. 2023-07-14T16:33:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.636+0000 7f544cd03b80 20 client.30047 trim_cache size 0 max 16384 2023-07-14T16:33:49.611 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:33:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:33:49.612 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "90a3a4dd-8583-4356-a548-a770b9371f7d", 2023-07-14T16:33:49.613 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:33:49.613 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:33:49.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 -- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f541c0a1220 msgr2=0x7f541c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f541c0a1220 0x7f541c0a36d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5424006010 tx=0x7f5424005ab0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 -- 172.21.15.62:0/530301815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434076ac0 msgr2=0x7f54341abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434076ac0 0x7f54341abf60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f542c007f10 tx=0x7f542c00d3b0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 -- 172.21.15.62:0/530301815 shutdown_connections 2023-07-14T16:33:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5434076ac0 0x7f54341abf60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f541c0a1220 0x7f541c0a36d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5434074ed0 0x7f543410c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 --2- 172.21.15.62:0/530301815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5434076110 0x7f543410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.637+0000 7f54393ff700 1 -- 172.21.15.62:0/530301815 >> 172.21.15.62:0/530301815 conn(0x7f54340fd350 msgr2=0x7f54340feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.638+0000 7f54393ff700 1 -- 172.21.15.62:0/530301815 shutdown_connections 2023-07-14T16:33:49.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:49.640+0000 7f54393ff700 1 -- 172.21.15.62:0/530301815 wait complete. 2023-07-14T16:33:49.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:33:49.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:49 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:33:49.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:49 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.050+0000 7fde4a1ff700 1 -- 172.21.15.62:0/3925288433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 msgr2=0x7fde44100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.050+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/3925288433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 0x7fde44100770 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fde3401c0d0 tx=0x7fde3401c3e0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.051+0000 7fde4a1ff700 1 -- 172.21.15.62:0/3925288433 shutdown_connections 2023-07-14T16:33:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.051+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/3925288433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde44103670 0x7fde44103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.051+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/3925288433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 0x7fde44100770 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.051+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/3925288433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde44100cb0 0x7fde44103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.051+0000 7fde4a1ff700 1 -- 172.21.15.62:0/3925288433 >> 172.21.15.62:0/3925288433 conn(0x7fde440fa000 msgr2=0x7fde440fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:50.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.051+0000 7fde4a1ff700 1 -- 172.21.15.62:0/3925288433 shutdown_connections 2023-07-14T16:33:50.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.051+0000 7fde4a1ff700 1 -- 172.21.15.62:0/3925288433 wait complete. 2023-07-14T16:33:50.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.052+0000 7fde4a1ff700 1 Processor -- start 2023-07-14T16:33:50.027 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.054+0000 7fde4a1ff700 1 -- start start 2023-07-14T16:33:50.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.054+0000 7fde4a1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 0x7fde4406bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.054+0000 7fde4a1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde44100cb0 0x7fde4406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.054+0000 7fde4a1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde44103670 0x7fde440709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.054+0000 7fde4a1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44071180 con 0x7fde44103670 2023-07-14T16:33:50.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.054+0000 7fde4a1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440712f0 con 0x7fde440fe350 2023-07-14T16:33:50.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.054+0000 7fde4a1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440714b0 con 0x7fde44100cb0 2023-07-14T16:33:50.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde437fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 0x7fde4406bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde44100cb0 0x7fde4406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde437fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 0x7fde4406bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48084/0 (socket says 172.21.15.62:48084) 2023-07-14T16:33:50.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde437fe700 1 -- 172.21.15.62:0/1174518057 learned_addr learned my addr 172.21.15.62:0/1174518057 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:50.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde43fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde44103670 0x7fde440709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde437fe700 1 -- 172.21.15.62:0/1174518057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde44100cb0 msgr2=0x7fde4406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde437fe700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde44100cb0 0x7fde4406e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde437fe700 1 -- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde44103670 msgr2=0x7fde440709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde437fe700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde44103670 0x7fde440709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.056+0000 7fde437fe700 1 -- 172.21.15.62:0/1174518057 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3401ad10 con 0x7fde440fe350 2023-07-14T16:33:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.057+0000 7fde437fe700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 0x7fde4406bf40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde34003c30 tx=0x7fde34003c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.058+0000 7fde417fa700 1 -- 172.21.15.62:0/1174518057 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3401f050 con 0x7fde440fe350 2023-07-14T16:33:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.058+0000 7fde417fa700 1 -- 172.21.15.62:0/1174518057 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde3401de60 con 0x7fde440fe350 2023-07-14T16:33:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.058+0000 7fde417fa700 1 -- 172.21.15.62:0/1174518057 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3402ca30 con 0x7fde440fe350 2023-07-14T16:33:50.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.058+0000 7fde4a1ff700 1 -- 172.21.15.62:0/1174518057 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde44071730 con 0x7fde440fe350 2023-07-14T16:33:50.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.058+0000 7fde4a1ff700 1 -- 172.21.15.62:0/1174518057 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde44071c70 con 0x7fde440fe350 2023-07-14T16:33:50.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.067+0000 7fde5dae6b80 1 Processor -- start 2023-07-14T16:33:50.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.067+0000 7fde5dae6b80 1 -- start start 2023-07-14T16:33:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.067+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8faa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.067+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faae20 0x5652e8fab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.067+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 0x5652e8fac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.067+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8eafcb0 con 0x5652e8fabc70 2023-07-14T16:33:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.067+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8f02a40 con 0x5652e8faae20 2023-07-14T16:33:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.067+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8efbed0 con 0x5652e8faa4d0 2023-07-14T16:33:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.068+0000 7fde437fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faae20 0x5652e8fab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.068+0000 7fde43fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8faa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.068+0000 7fde43fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8faa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33620/0 (socket says 172.21.15.62:33620) 2023-07-14T16:33:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.068+0000 7fde43fff700 1 -- 172.21.15.62:0/2238333078 learned_addr learned my addr 172.21.15.62:0/2238333078 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.069+0000 7fde43fff700 1 -- 172.21.15.62:0/2238333078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faae20 msgr2=0x5652e8fab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.069+0000 7fde43fff700 1 --2- 172.21.15.62:0/2238333078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faae20 0x5652e8fab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.069+0000 7fde43fff700 1 -- 172.21.15.62:0/2238333078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 msgr2=0x5652e8fac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.069+0000 7fde43fff700 1 --2- 172.21.15.62:0/2238333078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 0x5652e8fac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.069+0000 7fde43fff700 1 -- 172.21.15.62:0/2238333078 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e8d29de0 con 0x5652e8faa4d0 2023-07-14T16:33:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.069+0000 7fde43fff700 1 --2- 172.21.15.62:0/2238333078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8faa8e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b800 tx=0x7fde3c00bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=610c99c980583c8a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:50.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.069+0000 7fde39ffb700 1 -- 172.21.15.62:0/2238333078 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c012630 con 0x5652e8faa4d0 2023-07-14T16:33:50.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.070+0000 7fde39ffb700 1 -- 172.21.15.62:0/2238333078 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde3c012c20 con 0x5652e8faa4d0 2023-07-14T16:33:50.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.070+0000 7fde39ffb700 1 -- 172.21.15.62:0/2238333078 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c014c90 con 0x5652e8faa4d0 2023-07-14T16:33:50.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2238333078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 msgr2=0x5652e8faa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2238333078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8faa8e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b800 tx=0x7fde3c00bb10 comp rx=0 tx=0).stop 2023-07-14T16:33:50.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2238333078 shutdown_connections 2023-07-14T16:33:50.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2238333078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 0x5652e8fac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2238333078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8faa8e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2238333078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faae20 0x5652e8fab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2238333078 >> 172.21.15.62:0/2238333078 conn(0x5652e8f9fda0 msgr2=0x5652e8fa51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:50.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde417fa700 1 -- 172.21.15.62:0/1174518057 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fde3402cb90 con 0x7fde440fe350 2023-07-14T16:33:50.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2238333078 shutdown_connections 2023-07-14T16:33:50.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.071+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2238333078 wait complete. 2023-07-14T16:33:50.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.072+0000 7fde5dae6b80 1 -- start start 2023-07-14T16:33:50.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.072+0000 7fde5dae6b80 1 Processor -- start 2023-07-14T16:33:50.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.073+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fb41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.073+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fb4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.073+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 0x5652e8fbb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.073+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8e9bac0 con 0x5652e8faae20 2023-07-14T16:33:50.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.073+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8e9b7c0 con 0x5652e8fabc70 2023-07-14T16:33:50.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.073+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8d7f920 con 0x5652e8faa4d0 2023-07-14T16:33:50.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.073+0000 7fde5dae6b80 10 client.0 _ll_register_callbacks cb 0x5652e8e99760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:50.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde43fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fb41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde43fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fb41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33636/0 (socket says 172.21.15.62:33636) 2023-07-14T16:33:50.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde43fff700 1 -- 172.21.15.62:0/2969602507 learned_addr learned my addr 172.21.15.62:0/2969602507 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:50.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde437fe700 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fb4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde43fff700 1 -- 172.21.15.62:0/2969602507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 msgr2=0x5652e8fbb740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:33:50.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde43fff700 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 0x5652e8fbb740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde43fff700 1 -- 172.21.15.62:0/2969602507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 msgr2=0x5652e8fb4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:50.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde43fff700 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fb4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.074+0000 7fde43fff700 1 -- 172.21.15.62:0/2969602507 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c00e040 con 0x5652e8faa4d0 2023-07-14T16:33:50.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.075+0000 7fde437fe700 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fb4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:50.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.075+0000 7fde417fa700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fde300a1260 0x7fde300a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.075+0000 7fde43fff700 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fb41c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde3c008010 tx=0x7fde3c006ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cd7d378a79c2658 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:50.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.075+0000 7fde417fa700 1 -- 172.21.15.62:0/1174518057 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fde340df470 con 0x7fde440fe350 2023-07-14T16:33:50.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:33:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde437fe700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fde300a1260 0x7fde300a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde22ffd700 1 -- 172.21.15.62:0/2969602507 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c014d20 con 0x5652e8faa4d0 2023-07-14T16:33:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde22ffd700 1 -- 172.21.15.62:0/2969602507 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fde3c012630 con 0x5652e8faa4d0 2023-07-14T16:33:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2969602507 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652e8e554f0 con 0x5652e8faa4d0 2023-07-14T16:33:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde22ffd700 1 -- 172.21.15.62:0/2969602507 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde3c014e80 con 0x5652e8faa4d0 2023-07-14T16:33:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde437fe700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fde300a1260 0x7fde300a3710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fde340a8dc0 tx=0x7fde340e1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde5dae6b80 10 client.29959 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.076+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2969602507 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5652e8d12c60 con 0x5652e8faa4d0 2023-07-14T16:33:50.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.077+0000 7fde22ffd700 1 -- 172.21.15.62:0/2969602507 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5652e8e554f0 con 0x5652e8faa4d0 2023-07-14T16:33:50.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.077+0000 7fde22ffd700 1 -- 172.21.15.62:0/2969602507 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fde3c01dc00 con 0x5652e8faa4d0 2023-07-14T16:33:50.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.077+0000 7fde5dae6b80 10 client.29959 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:50.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.077+0000 7fde5dae6b80 10 client.29959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:50.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.077+0000 7fde5dae6b80 20 client.29959 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:50.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.077+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5652e8fbfe10 0x5652e8fc2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.077+0000 7fde5dae6b80 4 client.29959 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:50.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.077+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2969602507 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652e8ef4eb0 con 0x5652e8fbfe10 2023-07-14T16:33:50.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.079+0000 7fde3bfff700 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5652e8fbfe10 0x5652e8fc2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.080+0000 7fde3bfff700 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5652e8fbfe10 0x5652e8fc2220 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20fe91554df526b6 server_cookie=42985df9b8232187 in_seq=0 out_seq=0 2023-07-14T16:33:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.080+0000 7fde22ffd700 10 client.29959 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.081+0000 7fde22ffd700 1 -- 172.21.15.62:0/2969602507 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5652e8ef4eb0 con 0x5652e8fbfe10 2023-07-14T16:33:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.081+0000 7fde22ffd700 10 client.29959 handle_command_reply: tid=0 2023-07-14T16:33:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.081+0000 7fde5dae6b80 1 client.29959 shutdown 2023-07-14T16:33:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.081+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2969602507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 msgr2=0x5652e8fb41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.081+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fb41c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde3c008010 tx=0x7fde3c006ea0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2969602507 shutdown_connections 2023-07-14T16:33:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5652e8fbfe10 0x5652e8fc2220 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 0x5652e8fbb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fb41c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde22ffd700 0 client.29959 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:50.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/2969602507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fb4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2969602507 >> 172.21.15.62:0/2969602507 conn(0x5652e8f9fda0 msgr2=0x5652e8fa48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:50.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2969602507 shutdown_connections 2023-07-14T16:33:50.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.082+0000 7fde5dae6b80 1 -- 172.21.15.62:0/2969602507 wait complete. 2023-07-14T16:33:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.083+0000 7fde5dae6b80 20 client.29959 trim_cache size 0 max 16384 2023-07-14T16:33:50.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.111+0000 7fde5dae6b80 1 Processor -- start 2023-07-14T16:33:50.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.111+0000 7fde5dae6b80 1 -- start start 2023-07-14T16:33:50.084 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.111+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.111+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fbb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.111+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 0x5652e8fc6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.111+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8fc70c0 con 0x5652e8faae20 2023-07-14T16:33:50.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.111+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8fa6000 con 0x5652e8fabc70 2023-07-14T16:33:50.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.111+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8fab1f0 con 0x5652e8faa4d0 2023-07-14T16:33:50.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.112+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fbb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.112+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fbb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33146/0 (socket says 172.21.15.62:33146) 2023-07-14T16:33:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.112+0000 7fde3bfff700 1 -- 172.21.15.62:0/1579630906 learned_addr learned my addr 172.21.15.62:0/1579630906 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.113+0000 7fde43fff700 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.113+0000 7fde43fff700 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 0x5652e8fc6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.113+0000 7fde43fff700 1 -- 172.21.15.62:0/1579630906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 msgr2=0x5652e8fc6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.113+0000 7fde43fff700 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 0x5652e8fc6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.113+0000 7fde43fff700 1 -- 172.21.15.62:0/1579630906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 msgr2=0x5652e8fbb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.113+0000 7fde43fff700 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fbb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.113+0000 7fde43fff700 1 -- 172.21.15.62:0/1579630906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e8d50250 con 0x5652e8faa4d0 2023-07-14T16:33:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.114+0000 7fde43fff700 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fac040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fde3c004730 tx=0x7fde3c009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b0b54833d88339 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.115+0000 7fde3a7fc700 1 -- 172.21.15.62:0/1579630906 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0325d0 con 0x5652e8faa4d0 2023-07-14T16:33:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.115+0000 7fde3a7fc700 1 -- 172.21.15.62:0/1579630906 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde3c0148e0 con 0x5652e8faa4d0 2023-07-14T16:33:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.115+0000 7fde3a7fc700 1 -- 172.21.15.62:0/1579630906 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c032c70 con 0x5652e8faa4d0 2023-07-14T16:33:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.116+0000 7fde5dae6b80 1 -- 172.21.15.62:0/1579630906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 msgr2=0x5652e8fac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.116+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fac040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fde3c004730 tx=0x7fde3c009040 comp rx=0 tx=0).stop 2023-07-14T16:33:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.117+0000 7fde5dae6b80 1 -- 172.21.15.62:0/1579630906 shutdown_connections 2023-07-14T16:33:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.117+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8fabc70 0x5652e8fc6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.117+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faa4d0 0x5652e8fac040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.117+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/1579630906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8faae20 0x5652e8fbb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.117+0000 7fde5dae6b80 1 -- 172.21.15.62:0/1579630906 >> 172.21.15.62:0/1579630906 conn(0x5652e8f9fda0 msgr2=0x5652e8fa51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.117+0000 7fde5dae6b80 1 -- 172.21.15.62:0/1579630906 shutdown_connections 2023-07-14T16:33:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.117+0000 7fde5dae6b80 1 -- 172.21.15.62:0/1579630906 wait complete. 2023-07-14T16:33:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.118+0000 7fde5dae6b80 1 -- start start 2023-07-14T16:33:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.118+0000 7fde5dae6b80 1 Processor -- start 2023-07-14T16:33:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faa4d0 0x5652e8fb7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faae20 0x5652e8fb7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde5dae6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 0x5652e8fbabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8fbb120 con 0x5652e8fabc70 2023-07-14T16:33:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8fbb290 con 0x5652e8faa4d0 2023-07-14T16:33:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde5dae6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e8fbb450 con 0x5652e8faae20 2023-07-14T16:33:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde5dae6b80 10 client.0 _ll_register_callbacks cb 0x7fde340e2d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faae20 0x5652e8fb7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faae20 0x5652e8fb7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:33660/0 (socket says 172.21.15.62:33660) 2023-07-14T16:33:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde3bfff700 1 -- 172.21.15.62:0/881641426 learned_addr learned my addr 172.21.15.62:0/881641426 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.119+0000 7fde43fff700 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faa4d0 0x5652e8fb7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.120+0000 7fde43fff700 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 0x5652e8fbabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.121+0000 7fde43fff700 1 -- 172.21.15.62:0/881641426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faae20 msgr2=0x5652e8fb7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.121+0000 7fde43fff700 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faae20 0x5652e8fb7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.121+0000 7fde43fff700 1 -- 172.21.15.62:0/881641426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 msgr2=0x5652e8fbabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.121+0000 7fde43fff700 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 0x5652e8fbabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.121+0000 7fde43fff700 1 -- 172.21.15.62:0/881641426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652e8d12c60 con 0x5652e8faa4d0 2023-07-14T16:33:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.122+0000 7fde43fff700 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faa4d0 0x5652e8fb7680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0148b0 tx=0x7fde3c01f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b72a2903877c7a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.122+0000 7fde22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:33:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.122+0000 7fde22ffd700 1 -- 172.21.15.62:0/881641426 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c038070 con 0x5652e8faa4d0 2023-07-14T16:33:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.122+0000 7fde22ffd700 1 -- 172.21.15.62:0/881641426 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fde3c000e40 con 0x5652e8faa4d0 2023-07-14T16:33:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.122+0000 7fde5dae6b80 1 -- 172.21.15.62:0/881641426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde3c012c20 con 0x5652e8faa4d0 2023-07-14T16:33:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.122+0000 7fde22ffd700 1 -- 172.21.15.62:0/881641426 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde3c032bf0 con 0x5652e8faa4d0 2023-07-14T16:33:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.123+0000 7fde5dae6b80 10 client.30062 fetch_fsmap learned FSMap version 32 2023-07-14T16:33:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.123+0000 7fde5dae6b80 1 -- 172.21.15.62:0/881641426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5652e8d282b0 con 0x5652e8faa4d0 2023-07-14T16:33:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.123+0000 7fde22ffd700 1 -- 172.21.15.62:0/881641426 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde3c012c20 con 0x5652e8faa4d0 2023-07-14T16:33:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.123+0000 7fde22ffd700 1 -- 172.21.15.62:0/881641426 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fde3c014d20 con 0x5652e8faa4d0 2023-07-14T16:33:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.123+0000 7fde5dae6b80 10 client.30062 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T16:33:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.123+0000 7fde5dae6b80 10 client.30062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.123+0000 7fde5dae6b80 20 client.30062 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.123+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5652e90147d0 0x5652e9016bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.124+0000 7fde5dae6b80 4 client.30062 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:33:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.124+0000 7fde5dae6b80 1 -- 172.21.15.62:0/881641426 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652e8eb7050 con 0x5652e90147d0 2023-07-14T16:33:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.124+0000 7fde3bfff700 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5652e90147d0 0x5652e9016bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.128+0000 7fde3bfff700 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5652e90147d0 0x5652e9016bb0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed73f62cfaacc90 server_cookie=366fa702debf984 in_seq=0 out_seq=0 2023-07-14T16:33:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.128+0000 7fde22ffd700 10 client.30062 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.128+0000 7fde22ffd700 1 -- 172.21.15.62:0/881641426 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5652e8eb7050 con 0x5652e90147d0 2023-07-14T16:33:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.128+0000 7fde22ffd700 10 client.30062 handle_command_reply: tid=0 2023-07-14T16:33:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 client.30062 shutdown 2023-07-14T16:33:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 -- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faa4d0 msgr2=0x5652e8fb7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faa4d0 0x5652e8fb7680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0148b0 tx=0x7fde3c01f040 comp rx=0 tx=0).stop 2023-07-14T16:33:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 -- 172.21.15.62:0/881641426 shutdown_connections 2023-07-14T16:33:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652e8fabc70 0x5652e8fbabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5652e90147d0 0x5652e9016bb0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652e8faa4d0 0x5652e8fb7680 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 --2- 172.21.15.62:0/881641426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5652e8faae20 0x5652e8fb7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde5dae6b80 1 -- 172.21.15.62:0/881641426 >> 172.21.15.62:0/881641426 conn(0x5652e8f9fda0 msgr2=0x5652e8fb2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.129+0000 7fde22ffd700 0 client.30062 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.130+0000 7fde5dae6b80 1 -- 172.21.15.62:0/881641426 shutdown_connections 2023-07-14T16:33:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.130+0000 7fde5dae6b80 1 -- 172.21.15.62:0/881641426 wait complete. 2023-07-14T16:33:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.131+0000 7fde5dae6b80 20 client.30062 trim_cache size 0 max 16384 2023-07-14T16:33:50.105 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:33:50.105 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:33:50.105 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:33:50.105 INFO:teuthology.orchestra.run.smithi062.stdout: "90a3a4dd-8583-4356-a548-a770b9371f7d": { 2023-07-14T16:33:50.105 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:33:50.105 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "90a3a4dd-8583-4356-a548-a770b9371f7d", 2023-07-14T16:33:50.105 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:33:50.106 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:33:50.106 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:33:50.106 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:33:50.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.133+0000 7fde4a1ff700 1 -- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fde300a1260 msgr2=0x7fde300a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.133+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fde300a1260 0x7fde300a3710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fde340a8dc0 tx=0x7fde340e1040 comp rx=0 tx=0).stop 2023-07-14T16:33:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.133+0000 7fde4a1ff700 1 -- 172.21.15.62:0/1174518057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 msgr2=0x7fde4406bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.133+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 0x7fde4406bf40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde34003c30 tx=0x7fde34003c60 comp rx=0 tx=0).stop 2023-07-14T16:33:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.134+0000 7fde4a1ff700 1 -- 172.21.15.62:0/1174518057 shutdown_connections 2023-07-14T16:33:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.134+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde44103670 0x7fde440709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.134+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fde300a1260 0x7fde300a3710 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.134+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde440fe350 0x7fde4406bf40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.134+0000 7fde4a1ff700 1 --2- 172.21.15.62:0/1174518057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde44100cb0 0x7fde4406e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.134+0000 7fde4a1ff700 1 -- 172.21.15.62:0/1174518057 >> 172.21.15.62:0/1174518057 conn(0x7fde440fa000 msgr2=0x7fde440fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.135+0000 7fde4a1ff700 1 -- 172.21.15.62:0/1174518057 shutdown_connections 2023-07-14T16:33:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:33:50.136+0000 7fde4a1ff700 1 -- 172.21.15.62:0/1174518057 wait complete. 2023-07-14T16:33:50.143 INFO:tasks.cephfs.filesystem:scrub status for tag:90a3a4dd-8583-4356-a548-a770b9371f7d - {'path': '/', 'tag': '90a3a4dd-8583-4356-a548-a770b9371f7d', 'options': 'recursive,force'} 2023-07-14T16:33:50.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:49 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:33:50.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:49 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:50.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:49 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:33:50.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:49 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:50 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:33:51.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:50 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:50 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:33:51.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:50 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:33:51.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:50 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:50 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:50 smithi164 ceph-mon[97427]: pgmap v903: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.65k op/s 2023-07-14T16:33:51.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:50 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:33:51.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:50 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:50 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:33:51.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:50 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:33:51.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:50 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:50 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:50 smithi062 ceph-mon[95543]: pgmap v903: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.65k op/s 2023-07-14T16:33:51.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:50 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:33:51.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:50 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:50 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:33:51.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:50 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:33:51.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:50 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:50 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:33:51.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:50 smithi081 ceph-mon[127795]: pgmap v903: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.65k op/s 2023-07-14T16:33:52.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:51 smithi164 ceph-mon[97427]: pgmap v904: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:33:52.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:51 smithi062 ceph-mon[95543]: pgmap v904: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:33:52.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:51 smithi081 ceph-mon[127795]: pgmap v904: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:33:54.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:54 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:33:54.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:54 smithi062 ceph-mon[95543]: pgmap v905: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 4.92k op/s 2023-07-14T16:33:54.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:54 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:33:54.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:54 smithi081 ceph-mon[127795]: pgmap v905: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 4.92k op/s 2023-07-14T16:33:54.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:54 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:33:54.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:54 smithi164 ceph-mon[97427]: pgmap v905: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 19 MiB/s wr, 4.92k op/s 2023-07-14T16:33:57.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:56 smithi062 ceph-mon[95543]: pgmap v906: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2023-07-14T16:33:57.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:56 smithi081 ceph-mon[127795]: pgmap v906: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2023-07-14T16:33:57.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:56 smithi164 ceph-mon[97427]: pgmap v906: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2023-07-14T16:33:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 -- 172.21.15.62:0/1026235768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 msgr2=0x7f8490100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 --2- 172.21.15.62:0/1026235768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 0x7f8490100830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8488009990 tx=0x7f8488009ca0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 -- 172.21.15.62:0/1026235768 shutdown_connections 2023-07-14T16:33:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 --2- 172.21.15.62:0/1026235768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490103730 0x7f8490103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 --2- 172.21.15.62:0/1026235768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 0x7f8490100830 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 --2- 172.21.15.62:0/1026235768 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8490100d70 0x7f84901031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 -- 172.21.15.62:0/1026235768 >> 172.21.15.62:0/1026235768 conn(0x7f84900fa000 msgr2=0x7f84900fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 -- 172.21.15.62:0/1026235768 shutdown_connections 2023-07-14T16:33:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.996+0000 7f84973ff700 1 -- 172.21.15.62:0/1026235768 wait complete. 2023-07-14T16:33:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.997+0000 7f84973ff700 1 Processor -- start 2023-07-14T16:33:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.998+0000 7f84973ff700 1 -- start start 2023-07-14T16:33:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.998+0000 7f84973ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 0x7f849010bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f84973ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8490100d70 0x7f849010c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f84973ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490103730 0x7f849010cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f84973ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849010d650 con 0x7f8490103730 2023-07-14T16:33:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f84973ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849010d7c0 con 0x7f8490100d70 2023-07-14T16:33:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f84973ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849010d980 con 0x7f84900fe410 2023-07-14T16:33:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f849519b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 0x7f849010bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f849519b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 0x7f849010bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37000/0 (socket says 172.21.15.62:37000) 2023-07-14T16:33:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f849519b700 1 -- 172.21.15.62:0/1752451519 learned_addr learned my addr 172.21.15.62:0/1752451519 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:58.999+0000 7f849599c700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490103730 0x7f849010cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.000+0000 7f849499a700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8490100d70 0x7f849010c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.000+0000 7f849519b700 1 -- 172.21.15.62:0/1752451519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8490100d70 msgr2=0x7f849010c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.000+0000 7f849519b700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8490100d70 0x7f849010c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.000+0000 7f849519b700 1 -- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490103730 msgr2=0x7f849010cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.000+0000 7f849519b700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490103730 0x7f849010cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.000+0000 7f849519b700 1 -- 172.21.15.62:0/1752451519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84880095f0 con 0x7f84900fe410 2023-07-14T16:33:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.000+0000 7f849599c700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490103730 0x7f849010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:33:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.001+0000 7f849519b700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 0x7f849010bbe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8488005b40 tx=0x7f8488003aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.001+0000 7f848e7fc700 1 -- 172.21.15.62:0/1752451519 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848801d070 con 0x7f84900fe410 2023-07-14T16:33:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.001+0000 7f848e7fc700 1 -- 172.21.15.62:0/1752451519 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f848800f460 con 0x7f84900fe410 2023-07-14T16:33:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.001+0000 7f84973ff700 1 -- 172.21.15.62:0/1752451519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849010dc00 con 0x7f84900fe410 2023-07-14T16:33:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.001+0000 7f848e7fc700 1 -- 172.21.15.62:0/1752451519 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488021410 con 0x7f84900fe410 2023-07-14T16:33:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.001+0000 7f84973ff700 1 -- 172.21.15.62:0/1752451519 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849010e410 con 0x7f84900fe410 2023-07-14T16:33:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.008+0000 7f84aae0bb80 1 Processor -- start 2023-07-14T16:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.013+0000 7f84aae0bb80 1 -- start start 2023-07-14T16:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.014+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 0x55e409776300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.014+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e409776840 0x55e4097771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.014+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409777b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.014+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40967bc50 con 0x55e4095d26b0 2023-07-14T16:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.014+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4096ce9e0 con 0x55e409776840 2023-07-14T16:33:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.014+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4096c7e70 con 0x55e4097776e0 2023-07-14T16:33:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.014+0000 7f848e7fc700 1 -- 172.21.15.62:0/1752451519 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f848800f7d0 con 0x7f84900fe410 2023-07-14T16:33:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.015+0000 7f849499a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409777b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.015+0000 7f849499a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409777b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37006/0 (socket says 172.21.15.62:37006) 2023-07-14T16:33:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.015+0000 7f849499a700 1 -- 172.21.15.62:0/4069112645 learned_addr learned my addr 172.21.15.62:0/4069112645 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.015+0000 7f849499a700 1 -- 172.21.15.62:0/4069112645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e409776840 msgr2=0x55e4097771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.015+0000 7f849499a700 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e409776840 0x55e4097771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.015+0000 7f849599c700 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 0x55e409776300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.015+0000 7f849499a700 1 -- 172.21.15.62:0/4069112645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 msgr2=0x55e409776300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.015+0000 7f849499a700 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 0x55e409776300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.016+0000 7f849499a700 1 -- 172.21.15.62:0/4069112645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4094f5d80 con 0x55e4097776e0 2023-07-14T16:33:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.016+0000 7f849599c700 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 0x55e409776300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:33:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.016+0000 7f849499a700 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409777b80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8484009860 tx=0x7f8484009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85dd69f5ff3bd7f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.017+0000 7f84777fe700 1 -- 172.21.15.62:0/4069112645 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400d610 con 0x55e4097776e0 2023-07-14T16:33:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.017+0000 7f84777fe700 1 -- 172.21.15.62:0/4069112645 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f848400dc00 con 0x55e4097776e0 2023-07-14T16:33:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.017+0000 7f84777fe700 1 -- 172.21.15.62:0/4069112645 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484014c50 con 0x55e4097776e0 2023-07-14T16:33:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4069112645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 msgr2=0x55e409777b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409777b80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8484009860 tx=0x7f8484009b70 comp rx=0 tx=0).stop 2023-07-14T16:33:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f848e7fc700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f84780a11d0 0x7f84780a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4069112645 shutdown_connections 2023-07-14T16:33:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409777b80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 0x55e409776300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4069112645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e409776840 0x55e4097771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4069112645 >> 172.21.15.62:0/4069112645 conn(0x55e40976bbe0 msgr2=0x55e409770cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.018+0000 7f848e7fc700 1 -- 172.21.15.62:0/1752451519 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84880cdd50 con 0x7f84900fe410 2023-07-14T16:33:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.019+0000 7f849599c700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f84780a11d0 0x7f84780a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.019+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4069112645 shutdown_connections 2023-07-14T16:33:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.019+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4069112645 wait complete. 2023-07-14T16:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.019+0000 7f84aae0bb80 1 -- start start 2023-07-14T16:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.019+0000 7f849599c700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f84780a11d0 0x7f84780a3680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55e409776060 tx=0x7f8480009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.020+0000 7f84aae0bb80 1 Processor -- start 2023-07-14T16:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.020+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e40977f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.020+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e40977fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.020+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4097776e0 0x55e409786ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.020+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409667a60 con 0x55e4097776e0 2023-07-14T16:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.020+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409667760 con 0x55e4095d26b0 2023-07-14T16:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.020+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40954b8c0 con 0x55e409776840 2023-07-14T16:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.021+0000 7f84aae0bb80 10 client.0 _ll_register_callbacks cb 0x55e409665700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.021+0000 7f849599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4097776e0 0x55e409786ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.021+0000 7f849599c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4097776e0 0x55e409786ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38464/0 (socket says 172.21.15.62:38464) 2023-07-14T16:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.021+0000 7f849599c700 1 -- 172.21.15.62:0/4093732824 learned_addr learned my addr 172.21.15.62:0/4093732824 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.021+0000 7f849499a700 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e40977fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.021+0000 7f849519b700 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e40977f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.022+0000 7f849599c700 1 -- 172.21.15.62:0/4093732824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 msgr2=0x55e40977fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.022+0000 7f849599c700 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e40977fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.022+0000 7f849599c700 1 -- 172.21.15.62:0/4093732824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 msgr2=0x55e40977f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.022+0000 7f849599c700 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e40977f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.022+0000 7f849599c700 1 -- 172.21.15.62:0/4093732824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848400e040 con 0x55e4097776e0 2023-07-14T16:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.022+0000 7f849499a700 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e40977fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.024+0000 7f849599c700 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4097776e0 0x55e409786ee0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f8480016fb0 tx=0x7f848000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cef60e13bdabbd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.024+0000 7f8474ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.024+0000 7f8474ff9700 1 -- 172.21.15.62:0/4093732824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848001e070 con 0x55e4097776e0 2023-07-14T16:33:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.024+0000 7f8474ff9700 1 -- 172.21.15.62:0/4093732824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84800198e0 con 0x55e4097776e0 2023-07-14T16:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.031+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4093732824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e409621490 con 0x55e4097776e0 2023-07-14T16:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.033+0000 7f8474ff9700 1 -- 172.21.15.62:0/4093732824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8480019aa0 con 0x55e4097776e0 2023-07-14T16:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.033+0000 7f84aae0bb80 10 client.20520 fetch_fsmap learned FSMap version 34 2023-07-14T16:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.033+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4093732824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e4094dec00 con 0x55e4097776e0 2023-07-14T16:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.034+0000 7f8474ff9700 1 -- 172.21.15.62:0/4093732824 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e409621490 con 0x55e4097776e0 2023-07-14T16:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.034+0000 7f8474ff9700 1 -- 172.21.15.62:0/4093732824 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f848002a070 con 0x55e4097776e0 2023-07-14T16:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.034+0000 7f84aae0bb80 10 client.20520 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.034+0000 7f84aae0bb80 10 client.20520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.034+0000 7f84aae0bb80 20 client.20520 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.034+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e40978b6e0 0x55e40978daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.034+0000 7f84aae0bb80 4 client.20520 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:33:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.034+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4093732824 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4096c0e50 con 0x55e40978b6e0 2023-07-14T16:33:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.035+0000 7f849519b700 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e40978b6e0 0x55e40978daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.035+0000 7f849519b700 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e40978b6e0 0x55e40978daf0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86d1e2525fe073 server_cookie=cb8b7cfcb74ab7c3 in_seq=0 out_seq=0 2023-07-14T16:33:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.035+0000 7f8474ff9700 10 client.20520 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.036+0000 7f8474ff9700 1 -- 172.21.15.62:0/4093732824 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e4096c0e50 con 0x55e40978b6e0 2023-07-14T16:33:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.036+0000 7f8474ff9700 10 client.20520 handle_command_reply: tid=0 2023-07-14T16:33:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.036+0000 7f84aae0bb80 1 client.20520 shutdown 2023-07-14T16:33:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4093732824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4097776e0 msgr2=0x55e409786ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4097776e0 0x55e409786ee0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f8480016fb0 tx=0x7f848000b040 comp rx=0 tx=0).stop 2023-07-14T16:33:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4093732824 shutdown_connections 2023-07-14T16:33:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4097776e0 0x55e409786ee0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e40978b6e0 0x55e40978daf0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e40977f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/4093732824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e40977fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4093732824 >> 172.21.15.62:0/4093732824 conn(0x55e40976bbe0 msgr2=0x55e4097704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f8474ff9700 0 client.20520 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4093732824 shutdown_connections 2023-07-14T16:33:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.037+0000 7f84aae0bb80 1 -- 172.21.15.62:0/4093732824 wait complete. 2023-07-14T16:33:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.038+0000 7f84aae0bb80 20 client.20520 trim_cache size 0 max 16384 2023-07-14T16:33:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.062+0000 7f84aae0bb80 1 Processor -- start 2023-07-14T16:33:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.062+0000 7f84aae0bb80 1 -- start start 2023-07-14T16:33:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.062+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 0x55e409777ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.062+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e409786ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.062+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4097776e0 0x55e40977e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.062+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409772b90 con 0x55e4095d26b0 2023-07-14T16:33:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.062+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4096c0e50 con 0x55e4097776e0 2023-07-14T16:33:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.062+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409621490 con 0x55e409776840 2023-07-14T16:33:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.064+0000 7f849499a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e409786ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.064+0000 7f849499a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e409786ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37026/0 (socket says 172.21.15.62:37026) 2023-07-14T16:33:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.064+0000 7f849499a700 1 -- 172.21.15.62:0/2511838607 learned_addr learned my addr 172.21.15.62:0/2511838607 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.064+0000 7f849599c700 1 --2- 172.21.15.62:0/2511838607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4097776e0 0x55e40977e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.065+0000 7f849599c700 1 -- 172.21.15.62:0/2511838607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 msgr2=0x55e409786ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.065+0000 7f849599c700 1 --2- 172.21.15.62:0/2511838607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e409786ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.065+0000 7f849599c700 1 -- 172.21.15.62:0/2511838607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 msgr2=0x55e409777ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:33:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.065+0000 7f849599c700 1 --2- 172.21.15.62:0/2511838607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 0x55e409777ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.065+0000 7f849599c700 1 -- 172.21.15.62:0/2511838607 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e40951c1f0 con 0x55e4097776e0 2023-07-14T16:33:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.065+0000 7f849599c700 1 --2- 172.21.15.62:0/2511838607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4097776e0 0x55e40977e600 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8480009010 tx=0x7f8480009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc98aa5f635390df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.065+0000 7f8477fff700 1 -- 172.21.15.62:0/2511838607 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848001e030 con 0x55e4097776e0 2023-07-14T16:33:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.066+0000 7f8477fff700 1 -- 172.21.15.62:0/2511838607 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f848000f050 con 0x55e4097776e0 2023-07-14T16:33:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.066+0000 7f8477fff700 1 -- 172.21.15.62:0/2511838607 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800226e0 con 0x55e4097776e0 2023-07-14T16:33:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.066+0000 7f84aae0bb80 1 -- 172.21.15.62:0/2511838607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4097776e0 msgr2=0x55e40977e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.066+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/2511838607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4097776e0 0x55e40977e600 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8480009010 tx=0x7f8480009680 comp rx=0 tx=0).stop 2023-07-14T16:33:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.070+0000 7f84aae0bb80 1 -- 172.21.15.62:0/2511838607 shutdown_connections 2023-07-14T16:33:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.070+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/2511838607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4097776e0 0x55e40977e600 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.070+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/2511838607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4095d26b0 0x55e409777ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.070+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/2511838607 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409776840 0x55e409786ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.070+0000 7f84aae0bb80 1 -- 172.21.15.62:0/2511838607 >> 172.21.15.62:0/2511838607 conn(0x55e40976bbe0 msgr2=0x55e409792450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.075+0000 7f84aae0bb80 1 -- 172.21.15.62:0/2511838607 shutdown_connections 2023-07-14T16:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.075+0000 7f84aae0bb80 1 -- 172.21.15.62:0/2511838607 wait complete. 2023-07-14T16:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.076+0000 7f84aae0bb80 1 -- start start 2023-07-14T16:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.076+0000 7f84aae0bb80 1 Processor -- start 2023-07-14T16:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.077+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e409782860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.077+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e409776840 0x55e409782da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.077+0000 7f84aae0bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409785dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.077+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409786300 con 0x55e409776840 2023-07-14T16:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.077+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409786470 con 0x55e4095d26b0 2023-07-14T16:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.077+0000 7f84aae0bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e409786630 con 0x55e4097776e0 2023-07-14T16:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.077+0000 7f84aae0bb80 10 client.0 _ll_register_callbacks cb 0x7f8480016980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.077+0000 7f849599c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409785dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.078+0000 7f849599c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409785dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37034/0 (socket says 172.21.15.62:37034) 2023-07-14T16:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.078+0000 7f849599c700 1 -- 172.21.15.62:0/3553916205 learned_addr learned my addr 172.21.15.62:0/3553916205 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.078+0000 7f849519b700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e409782860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.078+0000 7f849499a700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e409776840 0x55e409782da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f849499a700 1 -- 172.21.15.62:0/3553916205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 msgr2=0x55e409785dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f849499a700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409785dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f849499a700 1 -- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 msgr2=0x55e409782860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:33:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f849499a700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e409782860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f849499a700 1 -- 172.21.15.62:0/3553916205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4094dec00 con 0x55e409776840 2023-07-14T16:33:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f849599c700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409785dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f849519b700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e409782860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:33:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f849499a700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e409776840 0x55e409782da0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8484009f70 tx=0x7f8484004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcc21375e05aa6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:33:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.079+0000 7f8474ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.081+0000 7f8474ff9700 1 -- 172.21.15.62:0/3553916205 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484004480 con 0x55e409776840 2023-07-14T16:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.081+0000 7f8474ff9700 1 -- 172.21.15.62:0/3553916205 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84840045e0 con 0x55e409776840 2023-07-14T16:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.081+0000 7f84aae0bb80 1 -- 172.21.15.62:0/3553916205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f848400dc00 con 0x55e409776840 2023-07-14T16:33:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.085+0000 7f8474ff9700 1 -- 172.21.15.62:0/3553916205 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8484003720 con 0x55e409776840 2023-07-14T16:33:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.087+0000 7f84aae0bb80 10 client.20523 fetch_fsmap learned FSMap version 34 2023-07-14T16:33:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.087+0000 7f84aae0bb80 1 -- 172.21.15.62:0/3553916205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e4094f4250 con 0x55e409776840 2023-07-14T16:33:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f8474ff9700 1 -- 172.21.15.62:0/3553916205 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f848400dc00 con 0x55e409776840 2023-07-14T16:33:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f8474ff9700 1 -- 172.21.15.62:0/3553916205 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f848401b050 con 0x55e409776840 2023-07-14T16:33:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f84aae0bb80 10 client.20523 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:33:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f84aae0bb80 10 client.20523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:33:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f84aae0bb80 20 client.20523 populate_metadata read hostname 'smithi062' 2023-07-14T16:33:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4097dbc60 0x55e4097de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:33:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f84aae0bb80 4 client.20523 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:33:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f84aae0bb80 1 -- 172.21.15.62:0/3553916205 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e409682ff0 con 0x55e4097dbc60 2023-07-14T16:33:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.089+0000 7f849519b700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4097dbc60 0x55e4097de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:33:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.090+0000 7f849519b700 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4097dbc60 0x55e4097de070 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7076f50b6206ff1 server_cookie=c7192a4a6dedb5c0 in_seq=0 out_seq=0 2023-07-14T16:33:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.090+0000 7f8474ff9700 10 client.20523 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.102+0000 7f8474ff9700 1 -- 172.21.15.62:0/3553916205 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e409682ff0 con 0x55e4097dbc60 2023-07-14T16:33:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.102+0000 7f8474ff9700 10 client.20523 handle_command_reply: tid=0 2023-07-14T16:33:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 client.20523 shutdown 2023-07-14T16:33:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 -- 172.21.15.62:0/3553916205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e409776840 msgr2=0x55e409782da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e409776840 0x55e409782da0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8484009f70 tx=0x7f8484004040 comp rx=0 tx=0).stop 2023-07-14T16:33:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 -- 172.21.15.62:0/3553916205 shutdown_connections 2023-07-14T16:33:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4097776e0 0x55e409785dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4097dbc60 0x55e4097de070 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4095d26b0 0x55e409782860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 --2- 172.21.15.62:0/3553916205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e409776840 0x55e409782da0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 -- 172.21.15.62:0/3553916205 >> 172.21.15.62:0/3553916205 conn(0x55e40976bbe0 msgr2=0x55e40977e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f8474ff9700 0 client.20523 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:33:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 -- 172.21.15.62:0/3553916205 shutdown_connections 2023-07-14T16:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.103+0000 7f84aae0bb80 1 -- 172.21.15.62:0/3553916205 wait complete. 2023-07-14T16:33:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.104+0000 7f84aae0bb80 20 client.20523 trim_cache size 0 max 16384 2023-07-14T16:33:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 -- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f84780a11d0 msgr2=0x7f84780a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f84780a11d0 0x7f84780a3680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55e409776060 tx=0x7f8480009040 comp rx=0 tx=0).stop 2023-07-14T16:33:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 -- 172.21.15.62:0/1752451519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 msgr2=0x7f849010bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:33:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 0x7f849010bbe0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8488005b40 tx=0x7f8488003aa0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 -- 172.21.15.62:0/1752451519 shutdown_connections 2023-07-14T16:33:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490103730 0x7f849010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84900fe410 0x7f849010bbe0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f84780a11d0 0x7f84780a3680 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 --2- 172.21.15.62:0/1752451519 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8490100d70 0x7f849010c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:33:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.112+0000 7f84973ff700 1 -- 172.21.15.62:0/1752451519 >> 172.21.15.62:0/1752451519 conn(0x7f84900fa000 msgr2=0x7f84900fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:33:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.114+0000 7f84973ff700 1 -- 172.21.15.62:0/1752451519 shutdown_connections 2023-07-14T16:33:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:33:59.115+0000 7f84973ff700 1 -- 172.21.15.62:0/1752451519 wait complete. 2023-07-14T16:33:59.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:58 smithi062 ceph-mon[95543]: pgmap v907: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 4.78k op/s 2023-07-14T16:33:59.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:58 smithi081 ceph-mon[127795]: pgmap v907: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 4.78k op/s 2023-07-14T16:33:59.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:58 smithi164 ceph-mon[97427]: pgmap v907: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 4.78k op/s 2023-07-14T16:34:00.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:33:59 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:00.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:33:59 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:00.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:33:59 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:01.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:00 smithi081 ceph-mon[127795]: pgmap v908: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:01.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:00 smithi164 ceph-mon[97427]: pgmap v908: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:01.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:00 smithi062 ceph-mon[95543]: pgmap v908: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:03.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:02 smithi081 ceph-mon[127795]: pgmap v909: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:34:03.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:02 smithi164 ceph-mon[97427]: pgmap v909: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:34:03.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:02 smithi062 ceph-mon[95543]: pgmap v909: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:34:04.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:34:04.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:34:04.148 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:03 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:34:04.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:34:04.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:34:04.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:03 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:34:04.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:34:04.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:34:04.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:03 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:34:05.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:04 smithi164 ceph-mon[97427]: pgmap v910: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 5.15k op/s 2023-07-14T16:34:05.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:04 smithi062 ceph-mon[95543]: pgmap v910: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 5.15k op/s 2023-07-14T16:34:05.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:04 smithi081 ceph-mon[127795]: pgmap v910: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 MiB/s wr, 5.15k op/s 2023-07-14T16:34:07.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:06 smithi164 ceph-mon[97427]: pgmap v911: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-07-14T16:34:07.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:06 smithi062 ceph-mon[95543]: pgmap v911: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-07-14T16:34:07.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:06 smithi081 ceph-mon[127795]: pgmap v911: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-07-14T16:34:09.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:08 smithi164 ceph-mon[97427]: pgmap v912: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:09.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:08 smithi062 ceph-mon[95543]: pgmap v912: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:09.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:08 smithi081 ceph-mon[127795]: pgmap v912: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:11.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:10 smithi164 ceph-mon[97427]: pgmap v913: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2023-07-14T16:34:11.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:10 smithi062 ceph-mon[95543]: pgmap v913: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2023-07-14T16:34:11.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:10 smithi081 ceph-mon[127795]: pgmap v913: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2023-07-14T16:34:12.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:12 smithi164 ceph-mon[97427]: pgmap v914: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.379+0000 7fec193ff700 1 -- 172.21.15.62:0/2876878125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107340 msgr2=0x7fec14107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.379+0000 7fec193ff700 1 --2- 172.21.15.62:0/2876878125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107340 0x7fec14107790 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fec080098f0 tx=0x7fec08009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.380+0000 7fec193ff700 1 -- 172.21.15.62:0/2876878125 shutdown_connections 2023-07-14T16:34:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.380+0000 7fec193ff700 1 --2- 172.21.15.62:0/2876878125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec14107cd0 0x7fec1410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.380+0000 7fec193ff700 1 --2- 172.21.15.62:0/2876878125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec14106150 0x7fec14106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.380+0000 7fec193ff700 1 --2- 172.21.15.62:0/2876878125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107340 0x7fec14107790 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.380+0000 7fec193ff700 1 -- 172.21.15.62:0/2876878125 >> 172.21.15.62:0/2876878125 conn(0x7fec14075190 msgr2=0x7fec140775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.386+0000 7fec193ff700 1 -- 172.21.15.62:0/2876878125 shutdown_connections 2023-07-14T16:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.386+0000 7fec193ff700 1 -- 172.21.15.62:0/2876878125 wait complete. 2023-07-14T16:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.387+0000 7fec193ff700 1 Processor -- start 2023-07-14T16:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.389+0000 7fec193ff700 1 -- start start 2023-07-14T16:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.389+0000 7fec193ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec14106150 0x7fec14080a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.389+0000 7fec193ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec14107340 0x7fec14080fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.389+0000 7fec193ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107cd0 0x7fec1407efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.389+0000 7fec193ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1407f790 con 0x7fec14107cd0 2023-07-14T16:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.389+0000 7fec193ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1407f900 con 0x7fec14106150 2023-07-14T16:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.389+0000 7fec193ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1407fac0 con 0x7fec14107340 2023-07-14T16:34:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.390+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec14106150 0x7fec14080a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.390+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec14106150 0x7fec14080a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:42958/0 (socket says 172.21.15.62:42958) 2023-07-14T16:34:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.390+0000 7fec12ffd700 1 -- 172.21.15.62:0/3294340399 learned_addr learned my addr 172.21.15.62:0/3294340399 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.390+0000 7fec137fe700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107cd0 0x7fec1407efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.391+0000 7fec127fc700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec14107340 0x7fec14080fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.391+0000 7fec137fe700 1 -- 172.21.15.62:0/3294340399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec14107340 msgr2=0x7fec14080fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.391+0000 7fec137fe700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec14107340 0x7fec14080fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.391+0000 7fec137fe700 1 -- 172.21.15.62:0/3294340399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec14106150 msgr2=0x7fec14080a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.391+0000 7fec137fe700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec14106150 0x7fec14080a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.391+0000 7fec137fe700 1 -- 172.21.15.62:0/3294340399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec08009520 con 0x7fec14107cd0 2023-07-14T16:34:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.392+0000 7fec137fe700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107cd0 0x7fec1407efc0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00e970 tx=0x7fec0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.393+0000 7febfbfff700 1 -- 172.21.15.62:0/3294340399 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00cbd0 con 0x7fec14107cd0 2023-07-14T16:34:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.393+0000 7febfbfff700 1 -- 172.21.15.62:0/3294340399 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec0c00ee60 con 0x7fec14107cd0 2023-07-14T16:34:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.393+0000 7febfbfff700 1 -- 172.21.15.62:0/3294340399 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c004dc0 con 0x7fec14107cd0 2023-07-14T16:34:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.395+0000 7fec193ff700 1 -- 172.21.15.62:0/3294340399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec14069f30 con 0x7fec14107cd0 2023-07-14T16:34:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.395+0000 7fec193ff700 1 -- 172.21.15.62:0/3294340399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1406a3c0 con 0x7fec14107cd0 2023-07-14T16:34:12.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:12 smithi062 ceph-mon[95543]: pgmap v914: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:34:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.401+0000 7fec2cceab80 1 Processor -- start 2023-07-14T16:34:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.404+0000 7febfbfff700 1 -- 172.21.15.62:0/3294340399 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fec0c005020 con 0x7fec14107cd0 2023-07-14T16:34:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.404+0000 7fec2cceab80 1 -- start start 2023-07-14T16:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e6839300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 0x55e4e683a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e683a6e0 0x55e4e683ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec2cceab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e673ec50 con 0x55e4e683a6e0 2023-07-14T16:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec2cceab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e67919e0 con 0x55e4e6839840 2023-07-14T16:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec2cceab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e678ae70 con 0x55e4e66956b0 2023-07-14T16:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec127fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e6839300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec127fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e6839300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47914/0 (socket says 172.21.15.62:47914) 2023-07-14T16:34:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.405+0000 7fec127fc700 1 -- 172.21.15.62:0/1351278876 learned_addr learned my addr 172.21.15.62:0/1351278876 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.408+0000 7febfbfff700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febfc0a12a0 0x7febfc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.408+0000 7fec127fc700 1 -- 172.21.15.62:0/1351278876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 msgr2=0x55e4e683a1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.408+0000 7fec127fc700 1 --2- 172.21.15.62:0/1351278876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 0x55e4e683a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.408+0000 7fec127fc700 1 -- 172.21.15.62:0/1351278876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e683a6e0 msgr2=0x55e4e683ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.408+0000 7fec127fc700 1 --2- 172.21.15.62:0/1351278876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e683a6e0 0x55e4e683ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.408+0000 7fec127fc700 1 -- 172.21.15.62:0/1351278876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4e65b8d80 con 0x55e4e66956b0 2023-07-14T16:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.408+0000 7febfbfff700 1 -- 172.21.15.62:0/3294340399 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fec0c014070 con 0x7fec14107cd0 2023-07-14T16:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.408+0000 7fec127fc700 1 --2- 172.21.15.62:0/1351278876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e6839300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fec080098f0 tx=0x7fec0800be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9c2f0c1538f161c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.409+0000 7febf97fa700 1 -- 172.21.15.62:0/1351278876 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0800f4e0 con 0x55e4e66956b0 2023-07-14T16:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.409+0000 7febf97fa700 1 -- 172.21.15.62:0/1351278876 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec0800f640 con 0x55e4e66956b0 2023-07-14T16:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.409+0000 7febf97fa700 1 -- 172.21.15.62:0/1351278876 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0800b760 con 0x55e4e66956b0 2023-07-14T16:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.410+0000 7fec2cceab80 1 -- 172.21.15.62:0/1351278876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 msgr2=0x55e4e6839300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.410+0000 7fec2cceab80 1 --2- 172.21.15.62:0/1351278876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e6839300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fec080098f0 tx=0x7fec0800be60 comp rx=0 tx=0).stop 2023-07-14T16:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.411+0000 7fec137fe700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febfc0a12a0 0x7febfc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.411+0000 7fec2cceab80 1 -- 172.21.15.62:0/1351278876 shutdown_connections 2023-07-14T16:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.411+0000 7fec2cceab80 1 --2- 172.21.15.62:0/1351278876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e683a6e0 0x55e4e683ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.411+0000 7fec2cceab80 1 --2- 172.21.15.62:0/1351278876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e6839300 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.411+0000 7fec2cceab80 1 --2- 172.21.15.62:0/1351278876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 0x55e4e683a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.411+0000 7fec2cceab80 1 -- 172.21.15.62:0/1351278876 >> 172.21.15.62:0/1351278876 conn(0x55e4e682ebe0 msgr2=0x55e4e6833cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.411+0000 7fec2cceab80 1 -- 172.21.15.62:0/1351278876 shutdown_connections 2023-07-14T16:34:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.411+0000 7fec2cceab80 1 -- 172.21.15.62:0/1351278876 wait complete. 2023-07-14T16:34:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.412+0000 7fec2cceab80 1 -- start start 2023-07-14T16:34:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.412+0000 7fec2cceab80 1 Processor -- start 2023-07-14T16:34:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.413+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e68428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.413+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6839840 0x55e4e6842df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.413+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 0x55e4e6849e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.413+0000 7fec2cceab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e672aa60 con 0x55e4e6839840 2023-07-14T16:34:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.413+0000 7fec2cceab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e672a760 con 0x55e4e683a6e0 2023-07-14T16:34:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.413+0000 7fec2cceab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e660e8c0 con 0x55e4e66956b0 2023-07-14T16:34:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.413+0000 7fec2cceab80 10 client.0 _ll_register_callbacks cb 0x55e4e6728700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.414+0000 7fec127fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e68428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.414+0000 7fec127fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 0x55e4e6849e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.414+0000 7fec127fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e68428b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47926/0 (socket says 172.21.15.62:47926) 2023-07-14T16:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.414+0000 7fec127fc700 1 -- 172.21.15.62:0/4217897784 learned_addr learned my addr 172.21.15.62:0/4217897784 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.415+0000 7fec12ffd700 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6839840 0x55e4e6842df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.415+0000 7fec127fc700 1 -- 172.21.15.62:0/4217897784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 msgr2=0x55e4e6849e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.415+0000 7fec127fc700 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 0x55e4e6849e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.415+0000 7fec127fc700 1 -- 172.21.15.62:0/4217897784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6839840 msgr2=0x55e4e6842df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.415+0000 7fec127fc700 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6839840 0x55e4e6842df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.416+0000 7fec137fe700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febfc0a12a0 0x7febfc0a3750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0a1ff0 tx=0x7fec0c004f20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.416+0000 7fec127fc700 1 -- 172.21.15.62:0/4217897784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0801f040 con 0x55e4e66956b0 2023-07-14T16:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.416+0000 7fec127fc700 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e68428b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fec08009ff0 tx=0x7fec08009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2489af168465c8c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.416+0000 7febf6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.416+0000 7febf6ff5700 1 -- 172.21.15.62:0/4217897784 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0800fb00 con 0x55e4e66956b0 2023-07-14T16:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.416+0000 7febf6ff5700 1 -- 172.21.15.62:0/4217897784 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fec0800fc60 con 0x55e4e66956b0 2023-07-14T16:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.416+0000 7fec2cceab80 1 -- 172.21.15.62:0/4217897784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4e66e4490 con 0x55e4e66956b0 2023-07-14T16:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.417+0000 7febf6ff5700 1 -- 172.21.15.62:0/4217897784 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec0800fe20 con 0x55e4e66956b0 2023-07-14T16:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.417+0000 7fec2cceab80 10 client.29995 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.417+0000 7fec2cceab80 1 -- 172.21.15.62:0/4217897784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e4e65a1c00 con 0x55e4e66956b0 2023-07-14T16:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.417+0000 7febf6ff5700 1 -- 172.21.15.62:0/4217897784 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e4e66e4490 con 0x55e4e66956b0 2023-07-14T16:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.417+0000 7febf6ff5700 1 -- 172.21.15.62:0/4217897784 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fec08015020 con 0x55e4e66956b0 2023-07-14T16:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.418+0000 7fec2cceab80 10 client.29995 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.418+0000 7fec2cceab80 10 client.29995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.418+0000 7fec2cceab80 20 client.29995 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.418+0000 7fec2cceab80 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4e684e110 0x55e4e6850520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.418+0000 7fec2cceab80 4 client.29995 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.418+0000 7fec2cceab80 1 -- 172.21.15.62:0/4217897784 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4e6783e50 con 0x55e4e684e110 2023-07-14T16:34:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.418+0000 7fec12ffd700 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4e684e110 0x55e4e6850520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.422+0000 7fec12ffd700 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4e684e110 0x55e4e6850520 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc14e0ee0d259ce9 server_cookie=520759f37928f5a0 in_seq=0 out_seq=0 2023-07-14T16:34:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.422+0000 7febf6ff5700 10 client.29995 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.424+0000 7febf6ff5700 1 -- 172.21.15.62:0/4217897784 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e4e6783e50 con 0x55e4e684e110 2023-07-14T16:34:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.424+0000 7febf6ff5700 10 client.29995 handle_command_reply: tid=0 2023-07-14T16:34:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.424+0000 7fec2cceab80 1 client.29995 shutdown 2023-07-14T16:34:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.424+0000 7fec2cceab80 1 -- 172.21.15.62:0/4217897784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 msgr2=0x55e4e68428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.424+0000 7fec2cceab80 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e68428b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fec08009ff0 tx=0x7fec08009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7fec2cceab80 1 -- 172.21.15.62:0/4217897784 shutdown_connections 2023-07-14T16:34:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7fec2cceab80 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4e684e110 0x55e4e6850520 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7fec2cceab80 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 0x55e4e6849e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7fec2cceab80 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e66956b0 0x55e4e68428b0 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fec08009ff0 tx=0x7fec08009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7fec2cceab80 1 --2- 172.21.15.62:0/4217897784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6839840 0x55e4e6842df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7fec2cceab80 1 -- 172.21.15.62:0/4217897784 >> 172.21.15.62:0/4217897784 conn(0x55e4e682ebe0 msgr2=0x55e4e68334e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7febf6ff5700 0 client.29995 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:12.399 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:12 smithi081 ceph-mon[127795]: pgmap v914: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:34:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7fec2cceab80 1 -- 172.21.15.62:0/4217897784 shutdown_connections 2023-07-14T16:34:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.425+0000 7fec2cceab80 1 -- 172.21.15.62:0/4217897784 wait complete. 2023-07-14T16:34:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.426+0000 7fec2cceab80 20 client.29995 trim_cache size 0 max 16384 2023-07-14T16:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.458+0000 7fec2cceab80 1 Processor -- start 2023-07-14T16:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.458+0000 7fec2cceab80 1 -- start start 2023-07-14T16:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.458+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e6839840 0x55e4e683d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.458+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 0x55e4e683db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 0x55e4e684fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec2cceab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e6843aa0 con 0x55e4e6849df0 2023-07-14T16:34:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec2cceab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e67ff060 con 0x55e4e683a6e0 2023-07-14T16:34:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec2cceab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e6839c10 con 0x55e4e6839840 2023-07-14T16:34:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec127fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e6839840 0x55e4e683d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec127fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e6839840 0x55e4e683d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47934/0 (socket says 172.21.15.62:47934) 2023-07-14T16:34:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec127fc700 1 -- 172.21.15.62:0/3765350845 learned_addr learned my addr 172.21.15.62:0/3765350845 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec127fc700 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 0x55e4e684fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.459+0000 7fec12ffd700 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 0x55e4e683db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.460+0000 7fec127fc700 1 -- 172.21.15.62:0/3765350845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 msgr2=0x55e4e683db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.460+0000 7fec127fc700 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 0x55e4e683db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.460+0000 7fec127fc700 1 -- 172.21.15.62:0/3765350845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 msgr2=0x55e4e684fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.460+0000 7fec127fc700 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 0x55e4e684fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.460+0000 7fec127fc700 1 -- 172.21.15.62:0/3765350845 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4e683ad80 con 0x55e4e6839840 2023-07-14T16:34:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.460+0000 7fec127fc700 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e6839840 0x55e4e683d630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec080058d0 tx=0x7fec0800bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b3e122f8341ea9a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.463+0000 7febf9ffb700 1 -- 172.21.15.62:0/3765350845 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08005070 con 0x55e4e6839840 2023-07-14T16:34:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.463+0000 7febf9ffb700 1 -- 172.21.15.62:0/3765350845 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec08024800 con 0x55e4e6839840 2023-07-14T16:34:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.463+0000 7febf9ffb700 1 -- 172.21.15.62:0/3765350845 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08003680 con 0x55e4e6839840 2023-07-14T16:34:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.466+0000 7fec2cceab80 1 -- 172.21.15.62:0/3765350845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e6839840 msgr2=0x55e4e683d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.466+0000 7fec2cceab80 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e6839840 0x55e4e683d630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec080058d0 tx=0x7fec0800bc20 comp rx=0 tx=0).stop 2023-07-14T16:34:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.468+0000 7fec2cceab80 1 -- 172.21.15.62:0/3765350845 shutdown_connections 2023-07-14T16:34:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.468+0000 7fec2cceab80 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 0x55e4e684fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.468+0000 7fec2cceab80 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e6839840 0x55e4e683d630 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.468+0000 7fec2cceab80 1 --2- 172.21.15.62:0/3765350845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e683a6e0 0x55e4e683db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.468+0000 7fec2cceab80 1 -- 172.21.15.62:0/3765350845 >> 172.21.15.62:0/3765350845 conn(0x55e4e682ebe0 msgr2=0x55e4e68415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.473+0000 7fec2cceab80 1 -- 172.21.15.62:0/3765350845 shutdown_connections 2023-07-14T16:34:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.476+0000 7fec2cceab80 1 -- 172.21.15.62:0/3765350845 wait complete. 2023-07-14T16:34:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.477+0000 7fec2cceab80 1 -- start start 2023-07-14T16:34:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.477+0000 7fec2cceab80 1 Processor -- start 2023-07-14T16:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 0x55e4e6847de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e683a6e0 0x55e4e6848320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec2cceab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 0x55e4e6848860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec2cceab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e6848f30 con 0x55e4e6849df0 2023-07-14T16:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec2cceab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e68490a0 con 0x55e4e6839840 2023-07-14T16:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec2cceab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4e6849260 con 0x55e4e683a6e0 2023-07-14T16:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec2cceab80 10 client.0 _ll_register_callbacks cb 0x7fec08005210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e683a6e0 0x55e4e6848320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e683a6e0 0x55e4e6848320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:47950/0 (socket says 172.21.15.62:47950) 2023-07-14T16:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.478+0000 7fec12ffd700 1 -- 172.21.15.62:0/839976054 learned_addr learned my addr 172.21.15.62:0/839976054 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.479+0000 7fec127fc700 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 0x55e4e6847de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.479+0000 7fec127fc700 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 0x55e4e6848860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.479+0000 7fec127fc700 1 -- 172.21.15.62:0/839976054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e683a6e0 msgr2=0x55e4e6848320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.479+0000 7fec127fc700 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e683a6e0 0x55e4e6848320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.479+0000 7fec127fc700 1 -- 172.21.15.62:0/839976054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 msgr2=0x55e4e6848860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.479+0000 7fec127fc700 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 0x55e4e6848860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.479+0000 7fec127fc700 1 -- 172.21.15.62:0/839976054 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4e65a1c00 con 0x55e4e6839840 2023-07-14T16:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.480+0000 7fec127fc700 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 0x55e4e6847de0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fec0800a010 tx=0x7fec08009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d955fcf7d374daa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.480+0000 7febf6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.480+0000 7febf6ff5700 1 -- 172.21.15.62:0/839976054 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08005390 con 0x55e4e6839840 2023-07-14T16:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.480+0000 7febf6ff5700 1 -- 172.21.15.62:0/839976054 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fec08000e40 con 0x55e4e6839840 2023-07-14T16:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.480+0000 7fec2cceab80 1 -- 172.21.15.62:0/839976054 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec0800f640 con 0x55e4e6839840 2023-07-14T16:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.481+0000 7febf6ff5700 1 -- 172.21.15.62:0/839976054 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec080054f0 con 0x55e4e6839840 2023-07-14T16:34:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.483+0000 7fec2cceab80 10 client.30089 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.483+0000 7fec2cceab80 1 -- 172.21.15.62:0/839976054 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e4e65df1f0 con 0x55e4e6839840 2023-07-14T16:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.484+0000 7febf6ff5700 1 -- 172.21.15.62:0/839976054 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec0800f640 con 0x55e4e6839840 2023-07-14T16:34:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.484+0000 7febf6ff5700 1 -- 172.21.15.62:0/839976054 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fec08024d20 con 0x55e4e6839840 2023-07-14T16:34:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.484+0000 7fec2cceab80 10 client.30089 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.484+0000 7fec2cceab80 10 client.30089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.484+0000 7fec2cceab80 20 client.30089 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.484+0000 7fec2cceab80 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4e68a2950 0x55e4e68a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.484+0000 7fec2cceab80 4 client.30089 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.484+0000 7fec2cceab80 1 -- 172.21.15.62:0/839976054 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e4e6745ff0 con 0x55e4e68a2950 2023-07-14T16:34:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.485+0000 7fec12ffd700 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4e68a2950 0x55e4e68a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.485+0000 7febf6ff5700 10 client.30089 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.486+0000 7fec12ffd700 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4e68a2950 0x55e4e68a4d60 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a03ca989dc4d0fcc server_cookie=c2bb7159b532e89b in_seq=0 out_seq=0 2023-07-14T16:34:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.501+0000 7febf6ff5700 1 -- 172.21.15.62:0/839976054 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e4e6745ff0 con 0x55e4e68a2950 2023-07-14T16:34:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.501+0000 7febf6ff5700 10 client.30089 handle_command_reply: tid=0 2023-07-14T16:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.501+0000 7fec2cceab80 1 client.30089 shutdown 2023-07-14T16:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7fec2cceab80 1 -- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 msgr2=0x55e4e6847de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7fec2cceab80 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 0x55e4e6847de0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fec0800a010 tx=0x7fec08009f50 comp rx=0 tx=0).stop 2023-07-14T16:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7fec2cceab80 1 -- 172.21.15.62:0/839976054 shutdown_connections 2023-07-14T16:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7fec2cceab80 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4e6849df0 0x55e4e6848860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7fec2cceab80 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55e4e68a2950 0x55e4e68a4d60 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7fec2cceab80 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4e6839840 0x55e4e6847de0 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fec0800a010 tx=0x7fec08009f50 comp rx=0 tx=0).stop 2023-07-14T16:34:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7fec2cceab80 1 --2- 172.21.15.62:0/839976054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4e683a6e0 0x55e4e6848320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7fec2cceab80 1 -- 172.21.15.62:0/839976054 >> 172.21.15.62:0/839976054 conn(0x55e4e682ebe0 msgr2=0x55e4e683d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.502+0000 7febf6ff5700 0 client.30089 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.503+0000 7fec2cceab80 1 -- 172.21.15.62:0/839976054 shutdown_connections 2023-07-14T16:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.503+0000 7fec2cceab80 1 -- 172.21.15.62:0/839976054 wait complete. 2023-07-14T16:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.504+0000 7fec2cceab80 20 client.30089 trim_cache size 0 max 16384 2023-07-14T16:34:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:34:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:34:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:34:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:34:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.504+0000 7fec193ff700 1 -- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febfc0a12a0 msgr2=0x7febfc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.504+0000 7fec193ff700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febfc0a12a0 0x7febfc0a3750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0a1ff0 tx=0x7fec0c004f20 comp rx=0 tx=0).stop 2023-07-14T16:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.504+0000 7fec193ff700 1 -- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107cd0 msgr2=0x7fec1407efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.504+0000 7fec193ff700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107cd0 0x7fec1407efc0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00e970 tx=0x7fec0c00ec80 comp rx=0 tx=0).stop 2023-07-14T16:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.505+0000 7fec193ff700 1 -- 172.21.15.62:0/3294340399 shutdown_connections 2023-07-14T16:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.505+0000 7fec193ff700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec14107cd0 0x7fec1407efc0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.505+0000 7fec193ff700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7febfc0a12a0 0x7febfc0a3750 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.505+0000 7fec193ff700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec14106150 0x7fec14080a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.505+0000 7fec193ff700 1 --2- 172.21.15.62:0/3294340399 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec14107340 0x7fec14080fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.505+0000 7fec193ff700 1 -- 172.21.15.62:0/3294340399 >> 172.21.15.62:0/3294340399 conn(0x7fec14075190 msgr2=0x7fec1410a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.508+0000 7fec193ff700 1 -- 172.21.15.62:0/3294340399 shutdown_connections 2023-07-14T16:34:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:12.508+0000 7fec193ff700 1 -- 172.21.15.62:0/3294340399 wait complete. 2023-07-14T16:34:15.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:14 smithi081 ceph-mon[127795]: pgmap v915: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2023-07-14T16:34:15.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:14 smithi164 ceph-mon[97427]: pgmap v915: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2023-07-14T16:34:15.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:14 smithi062 ceph-mon[95543]: pgmap v915: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 4.89k op/s 2023-07-14T16:34:17.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:16 smithi081 ceph-mon[127795]: pgmap v916: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2023-07-14T16:34:17.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:16 smithi164 ceph-mon[97427]: pgmap v916: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2023-07-14T16:34:17.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:16 smithi062 ceph-mon[95543]: pgmap v916: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.02k op/s 2023-07-14T16:34:19.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:18 smithi081 ceph-mon[127795]: pgmap v917: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2023-07-14T16:34:19.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:18 smithi164 ceph-mon[97427]: pgmap v917: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2023-07-14T16:34:19.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:18 smithi062 ceph-mon[95543]: pgmap v917: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2023-07-14T16:34:20.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:34:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.539+0000 7f98f89ff700 1 -- 172.21.15.62:0/604116184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f4102e80 msgr2=0x7f98f4105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.539+0000 7f98f89ff700 1 --2- 172.21.15.62:0/604116184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f4102e80 0x7f98f4105260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098f0 tx=0x7f98e8009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.541+0000 7f98f89ff700 1 -- 172.21.15.62:0/604116184 shutdown_connections 2023-07-14T16:34:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.541+0000 7f98f89ff700 1 --2- 172.21.15.62:0/604116184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98f41057a0 0x7f98f4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.541+0000 7f98f89ff700 1 --2- 172.21.15.62:0/604116184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f4100560 0x7f98f4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.541+0000 7f98f89ff700 1 --2- 172.21.15.62:0/604116184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f4102e80 0x7f98f4105260 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.541+0000 7f98f89ff700 1 -- 172.21.15.62:0/604116184 >> 172.21.15.62:0/604116184 conn(0x7f98f40fa000 msgr2=0x7f98f40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.541+0000 7f98f89ff700 1 -- 172.21.15.62:0/604116184 shutdown_connections 2023-07-14T16:34:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.541+0000 7f98f89ff700 1 -- 172.21.15.62:0/604116184 wait complete. 2023-07-14T16:34:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.543+0000 7f98f89ff700 1 Processor -- start 2023-07-14T16:34:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.543+0000 7f98f89ff700 1 -- start start 2023-07-14T16:34:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.544+0000 7f98f89ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98f4100560 0x7f98f41933a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.544+0000 7f98f89ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f4102e80 0x7f98f41938e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.544+0000 7f98f89ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f41057a0 0x7f98f41a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.544+0000 7f98f89ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4194240 con 0x7f98f4100560 2023-07-14T16:34:20.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.544+0000 7f98f89ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41a3aa0 con 0x7f98f4102e80 2023-07-14T16:34:20.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.544+0000 7f98f89ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41a3c60 con 0x7f98f41057a0 2023-07-14T16:34:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.545+0000 7f98f259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98f4100560 0x7f98f41933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.545+0000 7f98f259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98f4100560 0x7f98f41933a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47790/0 (socket says 172.21.15.62:47790) 2023-07-14T16:34:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.545+0000 7f98f259c700 1 -- 172.21.15.62:0/68042578 learned_addr learned my addr 172.21.15.62:0/68042578 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.545+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f4102e80 0x7f98f41938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.545+0000 7f98f2d9d700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f41057a0 0x7f98f41a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.546+0000 7f98f2d9d700 1 -- 172.21.15.62:0/68042578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f4102e80 msgr2=0x7f98f41938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.546+0000 7f98f2d9d700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f4102e80 0x7f98f41938e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.546+0000 7f98f2d9d700 1 -- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98f4100560 msgr2=0x7f98f41933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.546+0000 7f98f2d9d700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98f4100560 0x7f98f41933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.546+0000 7f98f2d9d700 1 -- 172.21.15.62:0/68042578 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8009520 con 0x7f98f41057a0 2023-07-14T16:34:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.546+0000 7f98f2d9d700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f41057a0 0x7f98f41a3560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00d720 tx=0x7f98ec00da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.547+0000 7f98e37fe700 1 -- 172.21.15.62:0/68042578 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec004220 con 0x7f98f41057a0 2023-07-14T16:34:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.547+0000 7f98e37fe700 1 -- 172.21.15.62:0/68042578 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98ec004380 con 0x7f98f41057a0 2023-07-14T16:34:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.547+0000 7f98e37fe700 1 -- 172.21.15.62:0/68042578 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec003aa0 con 0x7f98f41057a0 2023-07-14T16:34:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.547+0000 7f98f89ff700 1 -- 172.21.15.62:0/68042578 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f41a3f40 con 0x7f98f41057a0 2023-07-14T16:34:20.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.547+0000 7f98f89ff700 1 -- 172.21.15.62:0/68042578 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f41a4370 con 0x7f98f41057a0 2023-07-14T16:34:20.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.555+0000 7f990c237b80 1 Processor -- start 2023-07-14T16:34:20.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.556+0000 7f98e37fe700 1 -- 172.21.15.62:0/68042578 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f98ec003c00 con 0x7f98f41057a0 2023-07-14T16:34:20.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.556+0000 7f990c237b80 1 -- start start 2023-07-14T16:34:20.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.557+0000 7f990c237b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4fef88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.557+0000 7f990c237b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4fef96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.557+0000 7f990c237b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef9c80 0x562c4fefa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.557+0000 7f990c237b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4fdeccb0 con 0x562c4fef9c80 2023-07-14T16:34:20.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.557+0000 7f990c237b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4fe3fa40 con 0x562c4fef84e0 2023-07-14T16:34:20.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.557+0000 7f990c237b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4fe38ed0 con 0x562c4fef8e30 2023-07-14T16:34:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.560+0000 7f98f1d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4fef88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.560+0000 7f98f1d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4fef88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41384/0 (socket says 172.21.15.62:41384) 2023-07-14T16:34:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.560+0000 7f98f1d9b700 1 -- 172.21.15.62:0/3134525063 learned_addr learned my addr 172.21.15.62:0/3134525063 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.561+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/3134525063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef9c80 0x562c4fefa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.561+0000 7f98f1d9b700 1 -- 172.21.15.62:0/3134525063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 msgr2=0x562c4fef96f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.561+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/3134525063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4fef96f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.561+0000 7f98f1d9b700 1 -- 172.21.15.62:0/3134525063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef9c80 msgr2=0x562c4fefa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.561+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/3134525063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef9c80 0x562c4fefa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.561+0000 7f98f1d9b700 1 -- 172.21.15.62:0/3134525063 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c4fc66de0 con 0x562c4fef84e0 2023-07-14T16:34:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.562+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/3134525063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4fef88f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098f0 tx=0x7f98e8019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f5ee781a216446 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.562+0000 7f98e37fe700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f98dc0a1220 0x7f98dc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.562+0000 7f98e37fe700 1 -- 172.21.15.62:0/68042578 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98ec0cc180 con 0x7f98f41057a0 2023-07-14T16:34:20.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.564+0000 7f98e0ff9700 1 -- 172.21.15.62:0/3134525063 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e800fb80 con 0x562c4fef84e0 2023-07-14T16:34:20.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.564+0000 7f98e0ff9700 1 -- 172.21.15.62:0/3134525063 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98e800fce0 con 0x562c4fef84e0 2023-07-14T16:34:20.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.564+0000 7f98e0ff9700 1 -- 172.21.15.62:0/3134525063 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e800b560 con 0x562c4fef84e0 2023-07-14T16:34:20.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.565+0000 7f98f259c700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f98dc0a1220 0x7f98dc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.566+0000 7f990c237b80 1 -- 172.21.15.62:0/3134525063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 msgr2=0x562c4fef88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.566+0000 7f990c237b80 1 --2- 172.21.15.62:0/3134525063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4fef88f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098f0 tx=0x7f98e8019040 comp rx=0 tx=0).stop 2023-07-14T16:34:20.548 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.575+0000 7f990c237b80 1 -- 172.21.15.62:0/3134525063 shutdown_connections 2023-07-14T16:34:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.575+0000 7f990c237b80 1 --2- 172.21.15.62:0/3134525063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef9c80 0x562c4fefa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.575+0000 7f990c237b80 1 --2- 172.21.15.62:0/3134525063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4fef88f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.575+0000 7f990c237b80 1 --2- 172.21.15.62:0/3134525063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4fef96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.549 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.575+0000 7f990c237b80 1 -- 172.21.15.62:0/3134525063 >> 172.21.15.62:0/3134525063 conn(0x562c4fedcda0 msgr2=0x562c4fee21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.575+0000 7f990c237b80 1 -- 172.21.15.62:0/3134525063 shutdown_connections 2023-07-14T16:34:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.575+0000 7f990c237b80 1 -- 172.21.15.62:0/3134525063 wait complete. 2023-07-14T16:34:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.576+0000 7f990c237b80 1 -- start start 2023-07-14T16:34:20.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.576+0000 7f990c237b80 1 Processor -- start 2023-07-14T16:34:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.577+0000 7f990c237b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff13100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.577+0000 7f990c237b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4ff13640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.577+0000 7f990c237b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 0x562c4ff1a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.577+0000 7f990c237b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4fdd8ac0 con 0x562c4fef84e0 2023-07-14T16:34:20.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.577+0000 7f990c237b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4fdd87c0 con 0x562c4fef9c80 2023-07-14T16:34:20.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.577+0000 7f990c237b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4fcbc920 con 0x562c4fef8e30 2023-07-14T16:34:20.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.577+0000 7f990c237b80 10 client.0 _ll_register_callbacks cb 0x562c4fdd6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:20.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.577+0000 7f98f259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff13100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.578+0000 7f98f259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff13100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47802/0 (socket says 172.21.15.62:47802) 2023-07-14T16:34:20.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.578+0000 7f98f259c700 1 -- 172.21.15.62:0/1289496512 learned_addr learned my addr 172.21.15.62:0/1289496512 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.578+0000 7f98f259c700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f98dc0a1220 0x7f98dc0a36d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f98f4194ad0 tx=0x7f98e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.579+0000 7f98f259c700 1 -- 172.21.15.62:0/1289496512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 msgr2=0x562c4ff13640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.579+0000 7f98f259c700 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4ff13640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.579+0000 7f98f259c700 1 -- 172.21.15.62:0/1289496512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 msgr2=0x562c4ff1a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.579+0000 7f98f259c700 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 0x562c4ff1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.579+0000 7f98f259c700 1 -- 172.21.15.62:0/1289496512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e801f040 con 0x562c4fef84e0 2023-07-14T16:34:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.579+0000 7f98f259c700 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff13100 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f98e4008b30 tx=0x7f98e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83e8c03a3bcfd47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.580+0000 7f98d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:34:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.580+0000 7f98d9ffb700 1 -- 172.21.15.62:0/1289496512 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4026070 con 0x562c4fef84e0 2023-07-14T16:34:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.580+0000 7f98d9ffb700 1 -- 172.21.15.62:0/1289496512 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98e4009480 con 0x562c4fef84e0 2023-07-14T16:34:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.580+0000 7f990c237b80 1 -- 172.21.15.62:0/1289496512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c4fd924f0 con 0x562c4fef84e0 2023-07-14T16:34:20.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.580+0000 7f98d9ffb700 1 -- 172.21.15.62:0/1289496512 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98e40048f0 con 0x562c4fef84e0 2023-07-14T16:34:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.581+0000 7f990c237b80 10 client.20550 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.581+0000 7f990c237b80 1 -- 172.21.15.62:0/1289496512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562c4fc4fc60 con 0x562c4fef84e0 2023-07-14T16:34:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.582+0000 7f98d9ffb700 1 -- 172.21.15.62:0/1289496512 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562c4fd924f0 con 0x562c4fef84e0 2023-07-14T16:34:20.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.582+0000 7f98d9ffb700 1 -- 172.21.15.62:0/1289496512 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f98e4029030 con 0x562c4fef84e0 2023-07-14T16:34:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.582+0000 7f990c237b80 10 client.20550 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.582+0000 7f990c237b80 10 client.20550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.582+0000 7f990c237b80 20 client.20550 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:20.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.582+0000 7f990c237b80 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562c4ff1f090 0x562c4ff214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.582+0000 7f990c237b80 4 client.20550 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:20.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.582+0000 7f990c237b80 1 -- 172.21.15.62:0/1289496512 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c4fe31eb0 con 0x562c4ff1f090 2023-07-14T16:34:20.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.589+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562c4ff1f090 0x562c4ff214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.590+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562c4ff1f090 0x562c4ff214a0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1535991f0ed9775e server_cookie=4ec01b1211636094 in_seq=0 out_seq=0 2023-07-14T16:34:20.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.590+0000 7f98d9ffb700 10 client.20550 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:20.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.591+0000 7f98d9ffb700 1 -- 172.21.15.62:0/1289496512 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562c4fe31eb0 con 0x562c4ff1f090 2023-07-14T16:34:20.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.591+0000 7f98d9ffb700 10 client.20550 handle_command_reply: tid=0 2023-07-14T16:34:20.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.591+0000 7f990c237b80 1 client.20550 shutdown 2023-07-14T16:34:20.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.592+0000 7f990c237b80 1 -- 172.21.15.62:0/1289496512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 msgr2=0x562c4ff13100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.592+0000 7f990c237b80 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff13100 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f98e4008b30 tx=0x7f98e4004ad0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f990c237b80 1 -- 172.21.15.62:0/1289496512 shutdown_connections 2023-07-14T16:34:20.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f990c237b80 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562c4ff1f090 0x562c4ff214a0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f990c237b80 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 0x562c4ff1a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f990c237b80 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff13100 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f990c237b80 1 --2- 172.21.15.62:0/1289496512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4ff13640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f990c237b80 1 -- 172.21.15.62:0/1289496512 >> 172.21.15.62:0/1289496512 conn(0x562c4fedcda0 msgr2=0x562c4fee18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:20.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f98d9ffb700 0 client.20550 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:20.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f990c237b80 1 -- 172.21.15.62:0/1289496512 shutdown_connections 2023-07-14T16:34:20.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.593+0000 7f990c237b80 1 -- 172.21.15.62:0/1289496512 wait complete. 2023-07-14T16:34:20.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.594+0000 7f990c237b80 20 client.20550 trim_cache size 0 max 16384 2023-07-14T16:34:20.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.630+0000 7f990c237b80 1 Processor -- start 2023-07-14T16:34:20.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.631+0000 7f990c237b80 1 -- start start 2023-07-14T16:34:20.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.631+0000 7f990c237b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff00010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.631+0000 7f990c237b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4ff1a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.631+0000 7f990c237b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 0x562c4ff00550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.631+0000 7f990c237b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4fee3070 con 0x562c4fef84e0 2023-07-14T16:34:20.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.631+0000 7f990c237b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4fedc970 con 0x562c4fef9c80 2023-07-14T16:34:20.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.631+0000 7f990c237b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4ff1c310 con 0x562c4fef8e30 2023-07-14T16:34:20.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.632+0000 7f98f259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff00010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.632+0000 7f98f259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff00010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47812/0 (socket says 172.21.15.62:47812) 2023-07-14T16:34:20.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.632+0000 7f98f259c700 1 -- 172.21.15.62:0/3482139282 learned_addr learned my addr 172.21.15.62:0/3482139282 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.632+0000 7f98f2d9d700 1 --2- 172.21.15.62:0/3482139282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 0x562c4ff00550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.632+0000 7f98f2d9d700 1 -- 172.21.15.62:0/3482139282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 msgr2=0x562c4ff1a640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:20.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.632+0000 7f98f2d9d700 1 --2- 172.21.15.62:0/3482139282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4ff1a640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.632+0000 7f98f2d9d700 1 -- 172.21.15.62:0/3482139282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 msgr2=0x562c4ff00010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:20.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.632+0000 7f98f2d9d700 1 --2- 172.21.15.62:0/3482139282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff00010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.633+0000 7f98f2d9d700 1 -- 172.21.15.62:0/3482139282 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c4fefa570 con 0x562c4fef9c80 2023-07-14T16:34:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.633+0000 7f98f2d9d700 1 --2- 172.21.15.62:0/3482139282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 0x562c4ff00550 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0ca000 tx=0x7f98ec003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26462cbbf92dcce7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.633+0000 7f98e17fa700 1 -- 172.21.15.62:0/3482139282 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec098f00 con 0x562c4fef9c80 2023-07-14T16:34:20.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.634+0000 7f98e17fa700 1 -- 172.21.15.62:0/3482139282 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98ec0044f0 con 0x562c4fef9c80 2023-07-14T16:34:20.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.634+0000 7f98e17fa700 1 -- 172.21.15.62:0/3482139282 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec097940 con 0x562c4fef9c80 2023-07-14T16:34:20.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.635+0000 7f990c237b80 1 -- 172.21.15.62:0/3482139282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 msgr2=0x562c4ff00550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.611 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.635+0000 7f990c237b80 1 --2- 172.21.15.62:0/3482139282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 0x562c4ff00550 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0ca000 tx=0x7f98ec003f80 comp rx=0 tx=0).stop 2023-07-14T16:34:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.635+0000 7f990c237b80 1 -- 172.21.15.62:0/3482139282 shutdown_connections 2023-07-14T16:34:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.635+0000 7f990c237b80 1 --2- 172.21.15.62:0/3482139282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef9c80 0x562c4ff00550 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.635+0000 7f990c237b80 1 --2- 172.21.15.62:0/3482139282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef84e0 0x562c4ff00010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.635+0000 7f990c237b80 1 --2- 172.21.15.62:0/3482139282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef8e30 0x562c4ff1a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.635+0000 7f990c237b80 1 -- 172.21.15.62:0/3482139282 >> 172.21.15.62:0/3482139282 conn(0x562c4fedcda0 msgr2=0x562c4ff210a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.635+0000 7f990c237b80 1 -- 172.21.15.62:0/3482139282 shutdown_connections 2023-07-14T16:34:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.636+0000 7f990c237b80 1 -- 172.21.15.62:0/3482139282 wait complete. 2023-07-14T16:34:20.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.636+0000 7f990c237b80 1 -- start start 2023-07-14T16:34:20.614 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.636+0000 7f990c237b80 1 Processor -- start 2023-07-14T16:34:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.637+0000 7f990c237b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4ff16650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.637+0000 7f990c237b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef8e30 0x562c4ff16b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.637+0000 7f990c237b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef9c80 0x562c4ff19bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.637+0000 7f990c237b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4ff1a0f0 con 0x562c4fef8e30 2023-07-14T16:34:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.637+0000 7f990c237b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4ff1a260 con 0x562c4fef84e0 2023-07-14T16:34:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.637+0000 7f990c237b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4ff1a420 con 0x562c4fef9c80 2023-07-14T16:34:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.637+0000 7f990c237b80 10 client.0 _ll_register_callbacks cb 0x7f98e40093d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.638+0000 7f98f259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4ff16650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.638+0000 7f98f259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4ff16650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41408/0 (socket says 172.21.15.62:41408) 2023-07-14T16:34:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.638+0000 7f98f259c700 1 -- 172.21.15.62:0/3799997732 learned_addr learned my addr 172.21.15.62:0/3799997732 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.638+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef8e30 0x562c4ff16b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.639+0000 7f98f1d9b700 1 -- 172.21.15.62:0/3799997732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef9c80 msgr2=0x562c4ff19bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.639+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef9c80 0x562c4ff19bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.639+0000 7f98f1d9b700 1 -- 172.21.15.62:0/3799997732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 msgr2=0x562c4ff16650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.639+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4ff16650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.639+0000 7f98f1d9b700 1 -- 172.21.15.62:0/3799997732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f41a4370 con 0x562c4fef8e30 2023-07-14T16:34:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.640+0000 7f98f1d9b700 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef8e30 0x562c4ff16b90 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098c0 tx=0x7f98e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a64fdc28f1f595 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.640+0000 7f98d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:34:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.641+0000 7f98d9ffb700 1 -- 172.21.15.62:0/3799997732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e802d040 con 0x562c4fef8e30 2023-07-14T16:34:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.641+0000 7f98d9ffb700 1 -- 172.21.15.62:0/3799997732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98e802f2b0 con 0x562c4fef8e30 2023-07-14T16:34:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.641+0000 7f990c237b80 1 -- 172.21.15.62:0/3799997732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98e800fce0 con 0x562c4fef8e30 2023-07-14T16:34:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.642+0000 7f98d9ffb700 1 -- 172.21.15.62:0/3799997732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98e800f7a0 con 0x562c4fef8e30 2023-07-14T16:34:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.642+0000 7f990c237b80 10 client.20556 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.642+0000 7f990c237b80 1 -- 172.21.15.62:0/3799997732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562c4fc8d250 con 0x562c4fef8e30 2023-07-14T16:34:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.643+0000 7f98d9ffb700 1 -- 172.21.15.62:0/3799997732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98e800fce0 con 0x562c4fef8e30 2023-07-14T16:34:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.643+0000 7f98d9ffb700 1 -- 172.21.15.62:0/3799997732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f98e8015020 con 0x562c4fef8e30 2023-07-14T16:34:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.644+0000 7f990c237b80 10 client.20556 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.644+0000 7f990c237b80 10 client.20556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:20.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.644+0000 7f990c237b80 20 client.20556 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:20.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.644+0000 7f990c237b80 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562c4ff73d40 0x562c4ff76150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:20.624 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.644+0000 7f990c237b80 4 client.20556 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:34:20.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.644+0000 7f990c237b80 1 -- 172.21.15.62:0/3799997732 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c4fdf4050 con 0x562c4ff73d40 2023-07-14T16:34:20.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.644+0000 7f98f259c700 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562c4ff73d40 0x562c4ff76150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:20.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.646+0000 7f98f259c700 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562c4ff73d40 0x562c4ff76150 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331f79fc6286303a server_cookie=69d5793ed4c1bc5e in_seq=0 out_seq=0 2023-07-14T16:34:20.625 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.646+0000 7f98d9ffb700 10 client.20556 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:20.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.647+0000 7f98d9ffb700 1 -- 172.21.15.62:0/3799997732 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562c4fdf4050 con 0x562c4ff73d40 2023-07-14T16:34:20.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.647+0000 7f98d9ffb700 10 client.20556 handle_command_reply: tid=0 2023-07-14T16:34:20.626 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.647+0000 7f990c237b80 1 client.20556 shutdown 2023-07-14T16:34:20.627 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.649+0000 7f990c237b80 1 -- 172.21.15.62:0/3799997732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef8e30 msgr2=0x562c4ff16b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.649+0000 7f990c237b80 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef8e30 0x562c4ff16b90 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098c0 tx=0x7f98e8009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:20.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.652+0000 7f990c237b80 1 -- 172.21.15.62:0/3799997732 shutdown_connections 2023-07-14T16:34:20.628 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.652+0000 7f990c237b80 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562c4fef9c80 0x562c4ff19bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.652+0000 7f990c237b80 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x562c4ff73d40 0x562c4ff76150 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.652+0000 7f990c237b80 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562c4fef84e0 0x562c4ff16650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.652+0000 7f990c237b80 1 --2- 172.21.15.62:0/3799997732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562c4fef8e30 0x562c4ff16b90 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.629 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.652+0000 7f990c237b80 1 -- 172.21.15.62:0/3799997732 >> 172.21.15.62:0/3799997732 conn(0x562c4fedcda0 msgr2=0x562c4ff00010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:20.630 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.652+0000 7f98d9ffb700 0 client.20556 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:20.630 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:34:20.631 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:34:20.631 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:34:20.631 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:34:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.653+0000 7f990c237b80 1 -- 172.21.15.62:0/3799997732 shutdown_connections 2023-07-14T16:34:20.632 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.653+0000 7f990c237b80 1 -- 172.21.15.62:0/3799997732 wait complete. 2023-07-14T16:34:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.654+0000 7f990c237b80 20 client.20556 trim_cache size 0 max 16384 2023-07-14T16:34:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.655+0000 7f98f89ff700 1 -- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f98dc0a1220 msgr2=0x7f98dc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.655+0000 7f98f89ff700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f98dc0a1220 0x7f98dc0a36d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f98f4194ad0 tx=0x7f98e400b040 comp rx=0 tx=0).stop 2023-07-14T16:34:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.655+0000 7f98f89ff700 1 -- 172.21.15.62:0/68042578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f41057a0 msgr2=0x7f98f41a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:20.633 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.655+0000 7f98f89ff700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f41057a0 0x7f98f41a3560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00d720 tx=0x7f98ec00da30 comp rx=0 tx=0).stop 2023-07-14T16:34:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.656+0000 7f98f89ff700 1 -- 172.21.15.62:0/68042578 shutdown_connections 2023-07-14T16:34:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.656+0000 7f98f89ff700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98f41057a0 0x7f98f41a3560 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.656+0000 7f98f89ff700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98f4100560 0x7f98f41933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.656+0000 7f98f89ff700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f98dc0a1220 0x7f98dc0a36d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.656+0000 7f98f89ff700 1 --2- 172.21.15.62:0/68042578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f4102e80 0x7f98f41938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.656+0000 7f98f89ff700 1 -- 172.21.15.62:0/68042578 >> 172.21.15.62:0/68042578 conn(0x7f98f40fa000 msgr2=0x7f98f41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.656+0000 7f98f89ff700 1 -- 172.21.15.62:0/68042578 shutdown_connections 2023-07-14T16:34:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:20.657+0000 7f98f89ff700 1 -- 172.21.15.62:0/68042578 wait complete. 2023-07-14T16:34:20.659 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:34:20.660 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:34:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.059+0000 7f81221ff700 1 -- 172.21.15.62:0/3888664453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f811c070e80 msgr2=0x7f811c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.059+0000 7f81221ff700 1 --2- 172.21.15.62:0/3888664453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f811c070e80 0x7f811c071290 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8114009990 tx=0x7f8114009ca0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.059+0000 7f81221ff700 1 -- 172.21.15.62:0/3888664453 shutdown_connections 2023-07-14T16:34:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.059+0000 7f81221ff700 1 --2- 172.21.15.62:0/3888664453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f811c10bdb0 0x7f811c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.059+0000 7f81221ff700 1 --2- 172.21.15.62:0/3888664453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f811c070e80 0x7f811c071290 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.059+0000 7f81221ff700 1 --2- 172.21.15.62:0/3888664453 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c0717d0 0x7f811c10b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.060+0000 7f81221ff700 1 -- 172.21.15.62:0/3888664453 >> 172.21.15.62:0/3888664453 conn(0x7f811c06ca40 msgr2=0x7f811c06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.060+0000 7f81221ff700 1 -- 172.21.15.62:0/3888664453 shutdown_connections 2023-07-14T16:34:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.060+0000 7f81221ff700 1 -- 172.21.15.62:0/3888664453 wait complete. 2023-07-14T16:34:21.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.061+0000 7f81221ff700 1 Processor -- start 2023-07-14T16:34:21.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.063+0000 7f81221ff700 1 -- start start 2023-07-14T16:34:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.063+0000 7f81221ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c070e80 0x7f811c1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.063+0000 7f81221ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f811c0717d0 0x7f811c1a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.063+0000 7f81221ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f811c10bdb0 0x7f811c1abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.063+0000 7f81221ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1ac160 con 0x7f811c10bdb0 2023-07-14T16:34:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.063+0000 7f81221ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1ac2a0 con 0x7f811c0717d0 2023-07-14T16:34:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.063+0000 7f81221ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1ac430 con 0x7f811c070e80 2023-07-14T16:34:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.064+0000 7f811affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f811c0717d0 0x7f811c1a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.064+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c070e80 0x7f811c1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.064+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c070e80 0x7f811c1a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44458/0 (socket says 172.21.15.62:44458) 2023-07-14T16:34:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.064+0000 7f811b7fe700 1 -- 172.21.15.62:0/1252751370 learned_addr learned my addr 172.21.15.62:0/1252751370 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.064+0000 7f811bfff700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f811c10bdb0 0x7f811c1abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.065+0000 7f811b7fe700 1 -- 172.21.15.62:0/1252751370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f811c0717d0 msgr2=0x7f811c1a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.065+0000 7f811b7fe700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f811c0717d0 0x7f811c1a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.065+0000 7f811b7fe700 1 -- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f811c10bdb0 msgr2=0x7f811c1abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.065+0000 7f811b7fe700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f811c10bdb0 0x7f811c1abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.065+0000 7f811b7fe700 1 -- 172.21.15.62:0/1252751370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81140095f0 con 0x7f811c070e80 2023-07-14T16:34:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.065+0000 7f811b7fe700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c070e80 0x7f811c1a45e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8114005950 tx=0x7f811400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.066+0000 7f8118ff9700 1 -- 172.21.15.62:0/1252751370 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811401d070 con 0x7f811c070e80 2023-07-14T16:34:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.066+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c1ac6b0 con 0x7f811c070e80 2023-07-14T16:34:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.066+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c1acb50 con 0x7f811c070e80 2023-07-14T16:34:21.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.066+0000 7f8118ff9700 1 -- 172.21.15.62:0/1252751370 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8114003ca0 con 0x7f811c070e80 2023-07-14T16:34:21.039 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.066+0000 7f8118ff9700 1 -- 172.21.15.62:0/1252751370 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114004a20 con 0x7f811c070e80 2023-07-14T16:34:21.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.071+0000 7f8118ff9700 1 -- 172.21.15.62:0/1252751370 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8114004b80 con 0x7f811c070e80 2023-07-14T16:34:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.072+0000 7f8118ff9700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81080a12d0 0x7f81080a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.073+0000 7f8118ff9700 1 -- 172.21.15.62:0/1252751370 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f81140cd980 con 0x7f811c070e80 2023-07-14T16:34:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.073+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c11a070 con 0x7f811c070e80 2023-07-14T16:34:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.073+0000 7f811affd700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81080a12d0 0x7f81080a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.077+0000 7f811affd700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81080a12d0 0x7f81080a3780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f810c009300 tx=0x7f810c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.080+0000 7f8118ff9700 1 -- 172.21.15.62:0/1252751370 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f811405a790 con 0x7f811c070e80 2023-07-14T16:34:21.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:20 smithi062 ceph-mon[95543]: pgmap v918: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.62k op/s 2023-07-14T16:34:21.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:20 smithi164 ceph-mon[97427]: pgmap v918: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.62k op/s 2023-07-14T16:34:21.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:20 smithi081 ceph-mon[127795]: pgmap v918: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.62k op/s 2023-07-14T16:34:21.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.580+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f811c1ad3b0 con 0x7f811c070e80 2023-07-14T16:34:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.582+0000 7f8118ff9700 1 -- 172.21.15.62:0/1252751370 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f8114097110 con 0x7f811c070e80 2023-07-14T16:34:21.554 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:34:21.554 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:33:58.847557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:34:21.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.589+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81080a12d0 msgr2=0x7f81080a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:21.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.589+0000 7f81221ff700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81080a12d0 0x7f81080a3780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f810c009300 tx=0x7f810c00b040 comp rx=0 tx=0).stop 2023-07-14T16:34:21.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.589+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c070e80 msgr2=0x7f811c1a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:21.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.589+0000 7f81221ff700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c070e80 0x7f811c1a45e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8114005950 tx=0x7f811400bd60 comp rx=0 tx=0).stop 2023-07-14T16:34:21.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.595+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 shutdown_connections 2023-07-14T16:34:21.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.595+0000 7f81221ff700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f811c10bdb0 0x7f811c1abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.595+0000 7f81221ff700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f811c070e80 0x7f811c1a45e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.595+0000 7f81221ff700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f81080a12d0 0x7f81080a3780 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.595+0000 7f81221ff700 1 --2- 172.21.15.62:0/1252751370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f811c0717d0 0x7f811c1a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.595+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 >> 172.21.15.62:0/1252751370 conn(0x7f811c06ca40 msgr2=0x7f811c06dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:21.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.597+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 shutdown_connections 2023-07-14T16:34:21.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.597+0000 7f81221ff700 1 -- 172.21.15.62:0/1252751370 wait complete. 2023-07-14T16:34:21.571 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 34 2023-07-14T16:34:21.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:34:21.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.995+0000 7f94583ff700 1 -- 172.21.15.62:0/2282827594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9450100d00 msgr2=0x7f9450101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:21.968 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.995+0000 7f94583ff700 1 --2- 172.21.15.62:0/2282827594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9450100d00 0x7f9450101150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f94400098f0 tx=0x7f9440009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.999+0000 7f94583ff700 1 -- 172.21.15.62:0/2282827594 shutdown_connections 2023-07-14T16:34:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.999+0000 7f94583ff700 1 --2- 172.21.15.62:0/2282827594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9450101690 0x7f9450107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.999+0000 7f94583ff700 1 --2- 172.21.15.62:0/2282827594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94500ffb10 0x7f94500fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.999+0000 7f94583ff700 1 --2- 172.21.15.62:0/2282827594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9450100d00 0x7f9450101150 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:21.999+0000 7f94583ff700 1 -- 172.21.15.62:0/2282827594 >> 172.21.15.62:0/2282827594 conn(0x7f94500fb130 msgr2=0x7f94500fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:21.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.001+0000 7f94583ff700 1 -- 172.21.15.62:0/2282827594 shutdown_connections 2023-07-14T16:34:21.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.001+0000 7f94583ff700 1 -- 172.21.15.62:0/2282827594 wait complete. 2023-07-14T16:34:21.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.003+0000 7f94583ff700 1 Processor -- start 2023-07-14T16:34:21.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.004+0000 7f94583ff700 1 -- start start 2023-07-14T16:34:21.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.004+0000 7f94583ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94500ffb10 0x7f945019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.004+0000 7f94583ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9450100d00 0x7f945019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.004+0000 7f94583ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450101690 0x7f94501a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.004+0000 7f94583ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94501a3a30 con 0x7f9450100d00 2023-07-14T16:34:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.004+0000 7f94583ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94501932c0 con 0x7f94500ffb10 2023-07-14T16:34:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.004+0000 7f94583ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9450193450 con 0x7f9450101690 2023-07-14T16:34:21.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.005+0000 7f945699c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450101690 0x7f94501a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.005+0000 7f945699c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450101690 0x7f94501a3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44478/0 (socket says 172.21.15.62:44478) 2023-07-14T16:34:21.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.005+0000 7f945699c700 1 -- 172.21.15.62:0/2986817431 learned_addr learned my addr 172.21.15.62:0/2986817431 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.006+0000 7f945599a700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9450100d00 0x7f945019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.007+0000 7f945699c700 1 -- 172.21.15.62:0/2986817431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94500ffb10 msgr2=0x7f945019beb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:34:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.007+0000 7f945699c700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94500ffb10 0x7f945019beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.007+0000 7f945699c700 1 -- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9450100d00 msgr2=0x7f945019c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.007+0000 7f945699c700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9450100d00 0x7f945019c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.007+0000 7f945699c700 1 -- 172.21.15.62:0/2986817431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440009520 con 0x7f9450101690 2023-07-14T16:34:21.981 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.008+0000 7f945699c700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450101690 0x7f94501a3460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f944800b970 tx=0x7f944800bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:21.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.009+0000 7f94477fe700 1 -- 172.21.15.62:0/2986817431 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800c9b0 con 0x7f9450101690 2023-07-14T16:34:21.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.009+0000 7f94477fe700 1 -- 172.21.15.62:0/2986817431 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9448014440 con 0x7f9450101690 2023-07-14T16:34:21.985 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.009+0000 7f94477fe700 1 -- 172.21.15.62:0/2986817431 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480075d0 con 0x7f9450101690 2023-07-14T16:34:21.988 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.009+0000 7f94583ff700 1 -- 172.21.15.62:0/2986817431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9450193730 con 0x7f9450101690 2023-07-14T16:34:21.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.009+0000 7f94583ff700 1 -- 172.21.15.62:0/2986817431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9450193bf0 con 0x7f9450101690 2023-07-14T16:34:21.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.015+0000 7f946bc8ab80 1 Processor -- start 2023-07-14T16:34:21.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.017+0000 7f946bc8ab80 1 -- start start 2023-07-14T16:34:21.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.017+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f1018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.017+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f1026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.017+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f102c70 0x556c4f1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:21.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.017+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f006cb0 con 0x556c4f102c70 2023-07-14T16:34:21.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.017+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f059a40 con 0x556c4f101e20 2023-07-14T16:34:21.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.017+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f052ed0 con 0x556c4f1014d0 2023-07-14T16:34:21.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.022+0000 7f945619b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f1026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.022+0000 7f945599a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f1018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.022+0000 7f945599a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f1018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44494/0 (socket says 172.21.15.62:44494) 2023-07-14T16:34:21.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.022+0000 7f945599a700 1 -- 172.21.15.62:0/2024985824 learned_addr learned my addr 172.21.15.62:0/2024985824 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:21.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.022+0000 7f945599a700 1 --2- 172.21.15.62:0/2024985824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f102c70 0x556c4f1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:21.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.023+0000 7f945599a700 1 -- 172.21.15.62:0/2024985824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 msgr2=0x556c4f1026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:21.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.023+0000 7f945599a700 1 --2- 172.21.15.62:0/2024985824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f1026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.023+0000 7f945599a700 1 -- 172.21.15.62:0/2024985824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f102c70 msgr2=0x556c4f1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:21.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.023+0000 7f945599a700 1 --2- 172.21.15.62:0/2024985824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f102c70 0x556c4f1030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:21.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.023+0000 7f945599a700 1 -- 172.21.15.62:0/2024985824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c4ee80de0 con 0x556c4f1014d0 2023-07-14T16:34:21.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.024+0000 7f945599a700 1 --2- 172.21.15.62:0/2024985824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f1018e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f944000c010 tx=0x7f944000be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d99b1197411d92 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:21.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.026+0000 7f9444bfe700 1 -- 172.21.15.62:0/2024985824 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000b700 con 0x556c4f1014d0 2023-07-14T16:34:21.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.026+0000 7f9444bfe700 1 -- 172.21.15.62:0/2024985824 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9440005040 con 0x556c4f1014d0 2023-07-14T16:34:21.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.026+0000 7f9444bfe700 1 -- 172.21.15.62:0/2024985824 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400038e0 con 0x556c4f1014d0 2023-07-14T16:34:22.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.029+0000 7f946bc8ab80 1 -- 172.21.15.62:0/2024985824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 msgr2=0x556c4f1018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.029+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/2024985824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f1018e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f944000c010 tx=0x7f944000be60 comp rx=0 tx=0).stop 2023-07-14T16:34:22.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.030+0000 7f94477fe700 1 -- 172.21.15.62:0/2986817431 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f94480145b0 con 0x7f9450101690 2023-07-14T16:34:22.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.030+0000 7f946bc8ab80 1 -- 172.21.15.62:0/2024985824 shutdown_connections 2023-07-14T16:34:22.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.030+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/2024985824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f102c70 0x556c4f1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.030+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/2024985824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f1018e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.030+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/2024985824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.030+0000 7f946bc8ab80 1 -- 172.21.15.62:0/2024985824 >> 172.21.15.62:0/2024985824 conn(0x556c4f0f6da0 msgr2=0x556c4f0fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.031+0000 7f946bc8ab80 1 -- 172.21.15.62:0/2024985824 shutdown_connections 2023-07-14T16:34:22.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.031+0000 7f946bc8ab80 1 -- 172.21.15.62:0/2024985824 wait complete. 2023-07-14T16:34:22.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.032+0000 7f946bc8ab80 1 -- start start 2023-07-14T16:34:22.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.033+0000 7f946bc8ab80 1 Processor -- start 2023-07-14T16:34:22.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.034+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f10b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.034+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f101e20 0x556c4f10b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.034+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f102c70 0x556c4f1126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.034+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4eff2ac0 con 0x556c4f101e20 2023-07-14T16:34:22.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.034+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4eff27c0 con 0x556c4f102c70 2023-07-14T16:34:22.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.034+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4eed6920 con 0x556c4f1014d0 2023-07-14T16:34:22.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.034+0000 7f946bc8ab80 10 client.0 _ll_register_callbacks cb 0x556c4eff0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:22.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.035+0000 7f94477fe700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f943c0a11e0 0x7f943c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.035+0000 7f94477fe700 1 -- 172.21.15.62:0/2986817431 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9448006e40 con 0x7f9450101690 2023-07-14T16:34:22.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.035+0000 7f945599a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f102c70 0x556c4f1126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.035+0000 7f945599a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f10b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.036+0000 7f945599a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f10b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44510/0 (socket says 172.21.15.62:44510) 2023-07-14T16:34:22.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.036+0000 7f945599a700 1 -- 172.21.15.62:0/1015081600 learned_addr learned my addr 172.21.15.62:0/1015081600 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.036+0000 7f945599a700 1 -- 172.21.15.62:0/1015081600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f102c70 msgr2=0x556c4f1126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.036+0000 7f945599a700 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f102c70 0x556c4f1126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.036+0000 7f945599a700 1 -- 172.21.15.62:0/1015081600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f101e20 msgr2=0x556c4f10b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.037+0000 7f945599a700 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f101e20 0x556c4f10b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.037+0000 7f945599a700 1 -- 172.21.15.62:0/1015081600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9450108880 con 0x556c4f1014d0 2023-07-14T16:34:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.037+0000 7f945699c700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f943c0a11e0 0x7f943c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.037+0000 7f945599a700 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f10b130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f944000f930 tx=0x7f944000f4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0fdce4f89f41ad0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.037+0000 7f9439ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:34:22.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.037+0000 7f9439ffb700 1 -- 172.21.15.62:0/1015081600 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000ba80 con 0x556c4f1014d0 2023-07-14T16:34:22.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.037+0000 7f9439ffb700 1 -- 172.21.15.62:0/1015081600 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9440003680 con 0x556c4f1014d0 2023-07-14T16:34:22.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.038+0000 7f946bc8ab80 1 -- 172.21.15.62:0/1015081600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c4efac4f0 con 0x556c4f1014d0 2023-07-14T16:34:22.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.038+0000 7f9439ffb700 1 -- 172.21.15.62:0/1015081600 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9440003840 con 0x556c4f1014d0 2023-07-14T16:34:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.038+0000 7f946bc8ab80 10 client.30025 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.038+0000 7f946bc8ab80 1 -- 172.21.15.62:0/1015081600 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x556c4ee69c60 con 0x556c4f1014d0 2023-07-14T16:34:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.038+0000 7f945699c700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f943c0a11e0 0x7f943c0a3690 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9448098f00 tx=0x7f9448004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.039+0000 7f9439ffb700 1 -- 172.21.15.62:0/1015081600 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556c4efac4f0 con 0x556c4f1014d0 2023-07-14T16:34:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.039+0000 7f9439ffb700 1 -- 172.21.15.62:0/1015081600 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9440015020 con 0x556c4f1014d0 2023-07-14T16:34:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.039+0000 7f946bc8ab80 10 client.30025 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:22.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.039+0000 7f946bc8ab80 10 client.30025 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:34:22.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.039+0000 7f946bc8ab80 20 client.30025 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:22.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.039+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556c4f116a60 0x556c4f118e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.039+0000 7f946bc8ab80 4 client.30025 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:22.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.039+0000 7f946bc8ab80 1 -- 172.21.15.62:0/1015081600 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c4f04beb0 con 0x556c4f116a60 2023-07-14T16:34:22.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.040+0000 7f945619b700 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556c4f116a60 0x556c4f118e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.044+0000 7f945619b700 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556c4f116a60 0x556c4f118e70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54402b0630922066 server_cookie=a88067a8ea2c2be2 in_seq=0 out_seq=0 2023-07-14T16:34:22.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.044+0000 7f9439ffb700 10 client.30025 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:34:22.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.045+0000 7f9439ffb700 1 -- 172.21.15.62:0/1015081600 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556c4f04beb0 con 0x556c4f116a60 2023-07-14T16:34:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.045+0000 7f9439ffb700 10 client.30025 handle_command_reply: tid=0 2023-07-14T16:34:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.045+0000 7f946bc8ab80 1 client.30025 shutdown 2023-07-14T16:34:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.045+0000 7f946bc8ab80 1 -- 172.21.15.62:0/1015081600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 msgr2=0x556c4f10b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.045+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f10b130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f944000f930 tx=0x7f944000f4d0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.046+0000 7f946bc8ab80 1 -- 172.21.15.62:0/1015081600 shutdown_connections 2023-07-14T16:34:22.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.046+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556c4f116a60 0x556c4f118e70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.046+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f102c70 0x556c4f1126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.046+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f1014d0 0x556c4f10b130 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.046+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/1015081600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f101e20 0x556c4f10b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.046+0000 7f946bc8ab80 1 -- 172.21.15.62:0/1015081600 >> 172.21.15.62:0/1015081600 conn(0x556c4f0f6da0 msgr2=0x556c4f0fb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.046+0000 7f9439ffb700 0 client.30025 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:34:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.046+0000 7f946bc8ab80 1 -- 172.21.15.62:0/1015081600 shutdown_connections 2023-07-14T16:34:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.047+0000 7f946bc8ab80 1 -- 172.21.15.62:0/1015081600 wait complete. 2023-07-14T16:34:22.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.052+0000 7f946bc8ab80 20 client.30025 trim_cache size 0 max 16384 2023-07-14T16:34:22.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.078+0000 7f946bc8ab80 1 Processor -- start 2023-07-14T16:34:22.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.078+0000 7f946bc8ab80 1 -- start start 2023-07-14T16:34:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.078+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f109e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.078+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f112670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.078+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.078+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f0fd4c0 con 0x556c4f1014d0 2023-07-14T16:34:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.078+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f10a5e0 con 0x556c4f101e20 2023-07-14T16:34:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.078+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f10c080 con 0x556c4f102c70 2023-07-14T16:34:22.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.079+0000 7f945599a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.079+0000 7f945599a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f109e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47870/0 (socket says 172.21.15.62:47870) 2023-07-14T16:34:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.079+0000 7f945599a700 1 -- 172.21.15.62:0/4103925437 learned_addr learned my addr 172.21.15.62:0/4103925437 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.079+0000 7f945619b700 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.079+0000 7f945599a700 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.080+0000 7f945599a700 1 -- 172.21.15.62:0/4103925437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 msgr2=0x556c4f112ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.080+0000 7f945599a700 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f112ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.080+0000 7f945599a700 1 -- 172.21.15.62:0/4103925437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 msgr2=0x556c4f112670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.080+0000 7f945599a700 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f112670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.080+0000 7f945599a700 1 -- 172.21.15.62:0/4103925437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c4eea7250 con 0x556c4f1014d0 2023-07-14T16:34:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.080+0000 7f945619b700 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:34:22.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.087+0000 7f945599a700 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f109e20 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9440009340 tx=0x7f9440009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec00153e4312f22 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.088+0000 7f94453ff700 1 -- 172.21.15.62:0/4103925437 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440026c00 con 0x556c4f1014d0 2023-07-14T16:34:22.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.088+0000 7f94453ff700 1 -- 172.21.15.62:0/4103925437 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9440034d60 con 0x556c4f1014d0 2023-07-14T16:34:22.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.088+0000 7f94453ff700 1 -- 172.21.15.62:0/4103925437 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440034600 con 0x556c4f1014d0 2023-07-14T16:34:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.089+0000 7f946bc8ab80 1 -- 172.21.15.62:0/4103925437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 msgr2=0x556c4f109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.089+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f109e20 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9440009340 tx=0x7f9440009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.090+0000 7f946bc8ab80 1 -- 172.21.15.62:0/4103925437 shutdown_connections 2023-07-14T16:34:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.090+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.090+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f109e20 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.090+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/4103925437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.090+0000 7f946bc8ab80 1 -- 172.21.15.62:0/4103925437 >> 172.21.15.62:0/4103925437 conn(0x556c4f0f6da0 msgr2=0x556c4f11d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.091+0000 7f946bc8ab80 1 -- 172.21.15.62:0/4103925437 shutdown_connections 2023-07-14T16:34:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.093+0000 7f946bc8ab80 1 -- 172.21.15.62:0/4103925437 wait complete. 2023-07-14T16:34:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.093+0000 7f946bc8ab80 1 -- start start 2023-07-14T16:34:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.094+0000 7f946bc8ab80 1 Processor -- start 2023-07-14T16:34:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.094+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f10e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.094+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f10eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.094+0000 7f946bc8ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f111b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.094+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f1120c0 con 0x556c4f1014d0 2023-07-14T16:34:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.094+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f112230 con 0x556c4f101e20 2023-07-14T16:34:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.094+0000 7f946bc8ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4f1123f0 con 0x556c4f102c70 2023-07-14T16:34:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.094+0000 7f946bc8ab80 10 client.0 _ll_register_callbacks cb 0x7f9440005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.097+0000 7f945599a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.097+0000 7f945599a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f10e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.097+0000 7f945599a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f10e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47874/0 (socket says 172.21.15.62:47874) 2023-07-14T16:34:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.097+0000 7f945599a700 1 -- 172.21.15.62:0/3488610295 learned_addr learned my addr 172.21.15.62:0/3488610295 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.098+0000 7f945599a700 1 -- 172.21.15.62:0/3488610295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 msgr2=0x556c4f10eb60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.098+0000 7f945599a700 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f10eb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.098+0000 7f945599a700 1 -- 172.21.15.62:0/3488610295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 msgr2=0x556c4f10e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.098+0000 7f945599a700 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f10e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.098+0000 7f945599a700 1 -- 172.21.15.62:0/3488610295 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c009c10 con 0x556c4f102c70 2023-07-14T16:34:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.098+0000 7f945599a700 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f111b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9440034d30 tx=0x7f9440009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ae9e09a75c2c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.099+0000 7f9439ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:34:22.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.099+0000 7f9439ffb700 1 -- 172.21.15.62:0/3488610295 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440026650 con 0x556c4f102c70 2023-07-14T16:34:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.099+0000 7f9439ffb700 1 -- 172.21.15.62:0/3488610295 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9440000e40 con 0x556c4f102c70 2023-07-14T16:34:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.099+0000 7f946bc8ab80 1 -- 172.21.15.62:0/3488610295 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9440005040 con 0x556c4f102c70 2023-07-14T16:34:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.099+0000 7f9439ffb700 1 -- 172.21.15.62:0/3488610295 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94400267b0 con 0x556c4f102c70 2023-07-14T16:34:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.099+0000 7f946bc8ab80 10 client.30031 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.099+0000 7f946bc8ab80 1 -- 172.21.15.62:0/3488610295 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x556c4ee69c60 con 0x556c4f102c70 2023-07-14T16:34:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.100+0000 7f9439ffb700 1 -- 172.21.15.62:0/3488610295 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9440005040 con 0x556c4f102c70 2023-07-14T16:34:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.100+0000 7f9439ffb700 1 -- 172.21.15.62:0/3488610295 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9440015040 con 0x556c4f102c70 2023-07-14T16:34:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.100+0000 7f946bc8ab80 10 client.30031 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.100+0000 7f946bc8ab80 10 client.30031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:34:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.100+0000 7f946bc8ab80 20 client.30031 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.100+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556c4f16b5d0 0x556c4f16d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.100+0000 7f946bc8ab80 4 client.30031 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:34:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.100+0000 7f946bc8ab80 1 -- 172.21.15.62:0/3488610295 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c4f00e050 con 0x556c4f16b5d0 2023-07-14T16:34:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.102+0000 7f945619b700 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556c4f16b5d0 0x556c4f16d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.103+0000 7f945619b700 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556c4f16b5d0 0x556c4f16d9e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=796c680d9296c132 server_cookie=167d368593186bcf in_seq=0 out_seq=0 2023-07-14T16:34:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.103+0000 7f9439ffb700 10 client.30031 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:34:22.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.103+0000 7f9439ffb700 1 -- 172.21.15.62:0/3488610295 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c4f00e050 con 0x556c4f16b5d0 2023-07-14T16:34:22.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.103+0000 7f9439ffb700 10 client.30031 handle_command_reply: tid=0 2023-07-14T16:34:22.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.103+0000 7f946bc8ab80 1 client.30031 shutdown 2023-07-14T16:34:22.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 -- 172.21.15.62:0/3488610295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 msgr2=0x556c4f111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f111b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9440034d30 tx=0x7f9440009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:22.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 -- 172.21.15.62:0/3488610295 shutdown_connections 2023-07-14T16:34:22.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x556c4f16b5d0 0x556c4f16d9e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.077 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556c4f102c70 0x556c4f111b80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556c4f1014d0 0x556c4f10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 --2- 172.21.15.62:0/3488610295 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c4f101e20 0x556c4f10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 -- 172.21.15.62:0/3488610295 >> 172.21.15.62:0/3488610295 conn(0x556c4f0f6da0 msgr2=0x556c4f109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f9439ffb700 0 client.30031 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:34:22.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 -- 172.21.15.62:0/3488610295 shutdown_connections 2023-07-14T16:34:22.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.104+0000 7f946bc8ab80 1 -- 172.21.15.62:0/3488610295 wait complete. 2023-07-14T16:34:22.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.105+0000 7f946bc8ab80 20 client.30031 trim_cache size 0 max 16384 2023-07-14T16:34:22.084 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:34:22.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 -- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f943c0a11e0 msgr2=0x7f943c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f943c0a11e0 0x7f943c0a3690 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9448098f00 tx=0x7f9448004ad0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.085 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 -- 172.21.15.62:0/2986817431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450101690 msgr2=0x7f94501a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450101690 0x7f94501a3460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f944800b970 tx=0x7f944800bd30 comp rx=0 tx=0).stop 2023-07-14T16:34:22.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 -- 172.21.15.62:0/2986817431 shutdown_connections 2023-07-14T16:34:22.086 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450101690 0x7f94501a3460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f943c0a11e0 0x7f943c0a3690 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94500ffb10 0x7f945019beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 --2- 172.21.15.62:0/2986817431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9450100d00 0x7f945019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 -- 172.21.15.62:0/2986817431 >> 172.21.15.62:0/2986817431 conn(0x7f94500fb130 msgr2=0x7f9450104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.106+0000 7f94583ff700 1 -- 172.21.15.62:0/2986817431 shutdown_connections 2023-07-14T16:34:22.088 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.107+0000 7f94583ff700 1 -- 172.21.15.62:0/2986817431 wait complete. 2023-07-14T16:34:22.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:34:22.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:21 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1252751370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:34:22.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:21 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1252751370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:34:22.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:21 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1252751370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:34:22.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.515+0000 7f71619ff700 1 -- 172.21.15.62:0/3455227739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f715c0feac0 msgr2=0x7f715c0fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.515+0000 7f71619ff700 1 --2- 172.21.15.62:0/3455227739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f715c0feac0 0x7f715c0fef30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7150009990 tx=0x7f7150009ca0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.515+0000 7f71619ff700 1 -- 172.21.15.62:0/3455227739 shutdown_connections 2023-07-14T16:34:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.515+0000 7f71619ff700 1 --2- 172.21.15.62:0/3455227739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f715c0ff470 0x7f715c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.515+0000 7f71619ff700 1 --2- 172.21.15.62:0/3455227739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715c0fd990 0x7f715c0fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.515+0000 7f71619ff700 1 --2- 172.21.15.62:0/3455227739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f715c0feac0 0x7f715c0fef30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.515+0000 7f71619ff700 1 -- 172.21.15.62:0/3455227739 >> 172.21.15.62:0/3455227739 conn(0x7f715c0f9340 msgr2=0x7f715c0fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.517+0000 7f71619ff700 1 -- 172.21.15.62:0/3455227739 shutdown_connections 2023-07-14T16:34:22.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.517+0000 7f71619ff700 1 -- 172.21.15.62:0/3455227739 wait complete. 2023-07-14T16:34:22.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.518+0000 7f71619ff700 1 Processor -- start 2023-07-14T16:34:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.519+0000 7f71619ff700 1 -- start start 2023-07-14T16:34:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.519+0000 7f71619ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715c0fd990 0x7f715c10a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.519+0000 7f71619ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f715c0feac0 0x7f715c10a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.519+0000 7f71619ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f715c0ff470 0x7f715c1abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.519+0000 7f71619ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c10dad0 con 0x7f715c0fd990 2023-07-14T16:34:22.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.519+0000 7f71619ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c10dc10 con 0x7f715c0feac0 2023-07-14T16:34:22.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.519+0000 7f71619ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c069f30 con 0x7f715c0ff470 2023-07-14T16:34:22.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.520+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f715c0ff470 0x7f715c1abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.520+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f715c0ff470 0x7f715c1abf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44560/0 (socket says 172.21.15.62:44560) 2023-07-14T16:34:22.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.520+0000 7f715b7fe700 1 -- 172.21.15.62:0/1015975666 learned_addr learned my addr 172.21.15.62:0/1015975666 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.521+0000 7f715affd700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715c0fd990 0x7f715c10a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.521+0000 7f715affd700 1 -- 172.21.15.62:0/1015975666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f715c0ff470 msgr2=0x7f715c1abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.521+0000 7f715affd700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f715c0ff470 0x7f715c1abf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.521+0000 7f715affd700 1 -- 172.21.15.62:0/1015975666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f715c0feac0 msgr2=0x7f715c10a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.521+0000 7f715affd700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f715c0feac0 0x7f715c10a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.521+0000 7f715affd700 1 -- 172.21.15.62:0/1015975666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500095f0 con 0x7f715c0fd990 2023-07-14T16:34:22.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.522+0000 7f715affd700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715c0fd990 0x7f715c10a4b0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9f0 tx=0x7f714c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.522+0000 7f71609fd700 1 -- 172.21.15.62:0/1015975666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c00eee0 con 0x7f715c0fd990 2023-07-14T16:34:22.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.522+0000 7f71609fd700 1 -- 172.21.15.62:0/1015975666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f714c00cbd0 con 0x7f715c0fd990 2023-07-14T16:34:22.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.522+0000 7f71609fd700 1 -- 172.21.15.62:0/1015975666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c018680 con 0x7f715c0fd990 2023-07-14T16:34:22.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.523+0000 7f71619ff700 1 -- 172.21.15.62:0/1015975666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c06a210 con 0x7f715c0fd990 2023-07-14T16:34:22.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.523+0000 7f71619ff700 1 -- 172.21.15.62:0/1015975666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c06a6d0 con 0x7f715c0fd990 2023-07-14T16:34:22.501 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.529+0000 7f71609fd700 1 -- 172.21.15.62:0/1015975666 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f714c00cd40 con 0x7f715c0fd990 2023-07-14T16:34:22.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.532+0000 7f71752cfb80 1 Processor -- start 2023-07-14T16:34:22.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.532+0000 7f71752cfb80 1 -- start start 2023-07-14T16:34:22.504 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.533+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.533+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a03e20 0x561f31a046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.533+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a04c70 0x561f31a050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.533+0000 7f71752cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f31908cb0 con 0x561f31a034d0 2023-07-14T16:34:22.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.533+0000 7f71752cfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f3195ba40 con 0x561f31a04c70 2023-07-14T16:34:22.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.533+0000 7f71752cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f31954ed0 con 0x561f31a03e20 2023-07-14T16:34:22.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.534+0000 7f715a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a04c70 0x561f31a050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.534+0000 7f715a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a04c70 0x561f31a050e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41504/0 (socket says 172.21.15.62:41504) 2023-07-14T16:34:22.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.534+0000 7f715a7fc700 1 -- 172.21.15.62:0/1309603625 learned_addr learned my addr 172.21.15.62:0/1309603625 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.534+0000 7f715a7fc700 1 -- 172.21.15.62:0/1309603625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a03e20 msgr2=0x561f31a046e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.534+0000 7f715b7fe700 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.534+0000 7f71609fd700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71440a1250 0x7f71440a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.534+0000 7f715affd700 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a03e20 0x561f31a046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.535+0000 7f715a7fc700 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a03e20 0x561f31a046e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.535+0000 7f715a7fc700 1 -- 172.21.15.62:0/1309603625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 msgr2=0x561f31a038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.535+0000 7f715a7fc700 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.535+0000 7f715a7fc700 1 -- 172.21.15.62:0/1309603625 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f31782de0 con 0x561f31a04c70 2023-07-14T16:34:22.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.535+0000 7f715b7fe700 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:34:22.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.535+0000 7f71609fd700 1 -- 172.21.15.62:0/1015975666 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f714c014070 con 0x7f715c0fd990 2023-07-14T16:34:22.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.535+0000 7f715a7fc700 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a04c70 0x561f31a050e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7150009530 tx=0x7f7150009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d6cf63a037f9d12 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.535+0000 7f715b7fe700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71440a1250 0x7f71440a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.537+0000 7f715b7fe700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71440a1250 0x7f71440a3700 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f715400f350 tx=0x7f7154004be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.538+0000 7f7141ffb700 1 -- 172.21.15.62:0/1309603625 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715001d070 con 0x561f31a04c70 2023-07-14T16:34:22.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.538+0000 7f7141ffb700 1 -- 172.21.15.62:0/1309603625 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7150004080 con 0x561f31a04c70 2023-07-14T16:34:22.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.538+0000 7f7141ffb700 1 -- 172.21.15.62:0/1309603625 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500056b0 con 0x561f31a04c70 2023-07-14T16:34:22.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.540+0000 7f71752cfb80 1 -- 172.21.15.62:0/1309603625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a04c70 msgr2=0x561f31a050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.540+0000 7f71752cfb80 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a04c70 0x561f31a050e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7150009530 tx=0x7f7150009560 comp rx=0 tx=0).stop 2023-07-14T16:34:22.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.540+0000 7f71752cfb80 1 -- 172.21.15.62:0/1309603625 shutdown_connections 2023-07-14T16:34:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.540+0000 7f71752cfb80 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a04c70 0x561f31a050e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.540+0000 7f71752cfb80 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.540+0000 7f71752cfb80 1 --2- 172.21.15.62:0/1309603625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a03e20 0x561f31a046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.540+0000 7f71752cfb80 1 -- 172.21.15.62:0/1309603625 >> 172.21.15.62:0/1309603625 conn(0x561f319f8da0 msgr2=0x561f319fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.541+0000 7f71752cfb80 1 -- 172.21.15.62:0/1309603625 shutdown_connections 2023-07-14T16:34:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.541+0000 7f71752cfb80 1 -- 172.21.15.62:0/1309603625 wait complete. 2023-07-14T16:34:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.542+0000 7f71752cfb80 1 -- start start 2023-07-14T16:34:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.542+0000 7f71752cfb80 1 Processor -- start 2023-07-14T16:34:22.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.543+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a034d0 0x561f31a0d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.543+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a0d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.543+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a04c70 0x561f31a146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.543+0000 7f71752cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f318f4ac0 con 0x561f31a04c70 2023-07-14T16:34:22.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.543+0000 7f71752cfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f318f47c0 con 0x561f31a03e20 2023-07-14T16:34:22.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.543+0000 7f71752cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f317d8920 con 0x561f31a034d0 2023-07-14T16:34:22.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.543+0000 7f71752cfb80 10 client.0 _ll_register_callbacks cb 0x561f318f2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:22.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.544+0000 7f715a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a0d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.544+0000 7f715a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a0d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41510/0 (socket says 172.21.15.62:41510) 2023-07-14T16:34:22.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.544+0000 7f715a7fc700 1 -- 172.21.15.62:0/3611699506 learned_addr learned my addr 172.21.15.62:0/3611699506 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.544+0000 7f715a7fc700 1 -- 172.21.15.62:0/3611699506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a034d0 msgr2=0x561f31a0d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.544+0000 7f715a7fc700 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a034d0 0x561f31a0d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.544+0000 7f715a7fc700 1 -- 172.21.15.62:0/3611699506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a04c70 msgr2=0x561f31a146d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.544+0000 7f715a7fc700 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a04c70 0x561f31a146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.544+0000 7f715a7fc700 1 -- 172.21.15.62:0/3611699506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c104610 con 0x561f31a03e20 2023-07-14T16:34:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.545+0000 7f715a7fc700 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a0d690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7150005910 tx=0x7f7150004ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3bc0dba8f940b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.545+0000 7f713affd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.545+0000 7f713affd700 1 -- 172.21.15.62:0/3611699506 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715001d050 con 0x561f31a03e20 2023-07-14T16:34:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.545+0000 7f713affd700 1 -- 172.21.15.62:0/3611699506 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7150003880 con 0x561f31a03e20 2023-07-14T16:34:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.545+0000 7f71752cfb80 1 -- 172.21.15.62:0/3611699506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f318ae4f0 con 0x561f31a03e20 2023-07-14T16:34:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.546+0000 7f713affd700 1 -- 172.21.15.62:0/3611699506 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71500041f0 con 0x561f31a03e20 2023-07-14T16:34:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.546+0000 7f71752cfb80 10 client.30128 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.546+0000 7f71752cfb80 1 -- 172.21.15.62:0/3611699506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561f3176bc60 con 0x561f31a03e20 2023-07-14T16:34:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.547+0000 7f713affd700 1 -- 172.21.15.62:0/3611699506 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561f318ae4f0 con 0x561f31a03e20 2023-07-14T16:34:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.547+0000 7f713affd700 1 -- 172.21.15.62:0/3611699506 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7150022470 con 0x561f31a03e20 2023-07-14T16:34:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.547+0000 7f71752cfb80 10 client.30128 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.547+0000 7f71752cfb80 10 client.30128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:34:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.547+0000 7f71752cfb80 20 client.30128 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.547+0000 7f71752cfb80 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561f31a18900 0x561f31a1ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.547+0000 7f71752cfb80 4 client.30128 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.547+0000 7f71752cfb80 1 -- 172.21.15.62:0/3611699506 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f3194deb0 con 0x561f31a18900 2023-07-14T16:34:22.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.548+0000 7f715affd700 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561f31a18900 0x561f31a1ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.548+0000 7f713affd700 10 client.30128 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:34:22.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.549+0000 7f715affd700 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561f31a18900 0x561f31a1ace0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=229716034ff08ca5 server_cookie=b53ac5717608ee65 in_seq=0 out_seq=0 2023-07-14T16:34:22.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f713affd700 1 -- 172.21.15.62:0/3611699506 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561f3194deb0 con 0x561f31a18900 2023-07-14T16:34:22.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f713affd700 10 client.30128 handle_command_reply: tid=0 2023-07-14T16:34:22.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 client.30128 shutdown 2023-07-14T16:34:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 -- 172.21.15.62:0/3611699506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 msgr2=0x561f31a0d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a0d690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7150005910 tx=0x7f7150004ac0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 -- 172.21.15.62:0/3611699506 shutdown_connections 2023-07-14T16:34:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561f31a18900 0x561f31a1ace0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a04c70 0x561f31a146d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a034d0 0x561f31a0d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 --2- 172.21.15.62:0/3611699506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a0d690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f71752cfb80 1 -- 172.21.15.62:0/3611699506 >> 172.21.15.62:0/3611699506 conn(0x561f319f8da0 msgr2=0x561f319fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.550+0000 7f713affd700 0 client.30128 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:34:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.551+0000 7f71752cfb80 1 -- 172.21.15.62:0/3611699506 shutdown_connections 2023-07-14T16:34:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.551+0000 7f71752cfb80 1 -- 172.21.15.62:0/3611699506 wait complete. 2023-07-14T16:34:22.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.551+0000 7f71752cfb80 20 client.30128 trim_cache size 0 max 16384 2023-07-14T16:34:22.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.581+0000 7f71752cfb80 1 Processor -- start 2023-07-14T16:34:22.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.581+0000 7f71752cfb80 1 -- start start 2023-07-14T16:34:22.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.581+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a0c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.581+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.581+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a14fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.581+0000 7f71752cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f31a186f0 con 0x561f31a034d0 2023-07-14T16:34:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.581+0000 7f71752cfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f319ff070 con 0x561f31a03e20 2023-07-14T16:34:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.581+0000 7f71752cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f319ffc50 con 0x561f31a04c70 2023-07-14T16:34:22.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.582+0000 7f715a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.582+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a14fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.582+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a14fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44590/0 (socket says 172.21.15.62:44590) 2023-07-14T16:34:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.582+0000 7f715b7fe700 1 -- 172.21.15.62:0/1842390150 learned_addr learned my addr 172.21.15.62:0/1842390150 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.582+0000 7f715affd700 1 --2- 172.21.15.62:0/1842390150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a0c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.583+0000 7f715a7fc700 1 -- 172.21.15.62:0/1842390150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 msgr2=0x561f31a14fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.583+0000 7f715a7fc700 1 --2- 172.21.15.62:0/1842390150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a14fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.583+0000 7f715a7fc700 1 -- 172.21.15.62:0/1842390150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 msgr2=0x561f31a0c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.583+0000 7f715a7fc700 1 --2- 172.21.15.62:0/1842390150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a0c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.583+0000 7f715a7fc700 1 -- 172.21.15.62:0/1842390150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f317a9250 con 0x561f31a03e20 2023-07-14T16:34:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.583+0000 7f715a7fc700 1 --2- 172.21.15.62:0/1842390150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a145f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7150003f60 tx=0x7f7150005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd97e38c07772dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.583+0000 7f71427fc700 1 -- 172.21.15.62:0/1842390150 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715001d070 con 0x561f31a03e20 2023-07-14T16:34:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.584+0000 7f71427fc700 1 -- 172.21.15.62:0/1842390150 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f715001e040 con 0x561f31a03e20 2023-07-14T16:34:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.584+0000 7f71427fc700 1 -- 172.21.15.62:0/1842390150 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150003680 con 0x561f31a03e20 2023-07-14T16:34:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.584+0000 7f71752cfb80 1 -- 172.21.15.62:0/1842390150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 msgr2=0x561f31a145f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.584+0000 7f71752cfb80 1 --2- 172.21.15.62:0/1842390150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a145f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7150003f60 tx=0x7f7150005a20 comp rx=0 tx=0).stop 2023-07-14T16:34:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.585+0000 7f71752cfb80 1 -- 172.21.15.62:0/1842390150 shutdown_connections 2023-07-14T16:34:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.585+0000 7f71752cfb80 1 --2- 172.21.15.62:0/1842390150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a14fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.585+0000 7f71752cfb80 1 --2- 172.21.15.62:0/1842390150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a0c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.585+0000 7f71752cfb80 1 --2- 172.21.15.62:0/1842390150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a145f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.585+0000 7f71752cfb80 1 -- 172.21.15.62:0/1842390150 >> 172.21.15.62:0/1842390150 conn(0x561f319f8da0 msgr2=0x561f31a1f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.585+0000 7f71752cfb80 1 -- 172.21.15.62:0/1842390150 shutdown_connections 2023-07-14T16:34:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.585+0000 7f71752cfb80 1 -- 172.21.15.62:0/1842390150 wait complete. 2023-07-14T16:34:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.585+0000 7f71752cfb80 1 -- start start 2023-07-14T16:34:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.586+0000 7f71752cfb80 1 Processor -- start 2023-07-14T16:34:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.586+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.586+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a12a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.586+0000 7f71752cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.586+0000 7f71752cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f31a13f50 con 0x561f31a034d0 2023-07-14T16:34:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.586+0000 7f71752cfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f31a140c0 con 0x561f31a03e20 2023-07-14T16:34:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.586+0000 7f71752cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f31a14280 con 0x561f31a04c70 2023-07-14T16:34:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.587+0000 7f71752cfb80 10 client.0 _ll_register_callbacks cb 0x7f714c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.587+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.587+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a13a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:44596/0 (socket says 172.21.15.62:44596) 2023-07-14T16:34:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.587+0000 7f715b7fe700 1 -- 172.21.15.62:0/2885471504 learned_addr learned my addr 172.21.15.62:0/2885471504 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.587+0000 7f715affd700 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.587+0000 7f715a7fc700 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a12a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.588+0000 7f715b7fe700 1 -- 172.21.15.62:0/2885471504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 msgr2=0x561f31a12a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.588+0000 7f715b7fe700 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a12a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.588+0000 7f715b7fe700 1 -- 172.21.15.62:0/2885471504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 msgr2=0x561f31a124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.588+0000 7f715b7fe700 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.588+0000 7f715b7fe700 1 -- 172.21.15.62:0/2885471504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f3176bc60 con 0x561f31a04c70 2023-07-14T16:34:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.588+0000 7f713affd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:34:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.588+0000 7f715b7fe700 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a13a10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f715400faa0 tx=0x7f7154009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61e45c12275bb61 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.589+0000 7f713affd700 1 -- 172.21.15.62:0/2885471504 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400e040 con 0x561f31a04c70 2023-07-14T16:34:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.589+0000 7f713affd700 1 -- 172.21.15.62:0/2885471504 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7154006e40 con 0x561f31a04c70 2023-07-14T16:34:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.589+0000 7f71752cfb80 1 -- 172.21.15.62:0/2885471504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7150004080 con 0x561f31a04c70 2023-07-14T16:34:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.590+0000 7f713affd700 1 -- 172.21.15.62:0/2885471504 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7154013c70 con 0x561f31a04c70 2023-07-14T16:34:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.590+0000 7f71752cfb80 10 client.30040 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.590+0000 7f71752cfb80 1 -- 172.21.15.62:0/2885471504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561f317812b0 con 0x561f31a04c70 2023-07-14T16:34:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.590+0000 7f713affd700 1 -- 172.21.15.62:0/2885471504 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7150004080 con 0x561f31a04c70 2023-07-14T16:34:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.590+0000 7f713affd700 1 -- 172.21.15.62:0/2885471504 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7154020420 con 0x561f31a04c70 2023-07-14T16:34:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.591+0000 7f71752cfb80 10 client.30040 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.591+0000 7f71752cfb80 10 client.30040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:34:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.591+0000 7f71752cfb80 20 client.30040 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.591+0000 7f71752cfb80 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561f31a6d880 0x561f31a6fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.591+0000 7f71752cfb80 4 client.30040 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:34:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.591+0000 7f71752cfb80 1 -- 172.21.15.62:0/2885471504 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561f31910050 con 0x561f31a6d880 2023-07-14T16:34:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.591+0000 7f715affd700 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561f31a6d880 0x561f31a6fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.596+0000 7f713affd700 10 client.30040 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:34:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.596+0000 7f715affd700 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561f31a6d880 0x561f31a6fc90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bd63c334cbe3d4 server_cookie=db6af56c2884e86c in_seq=0 out_seq=0 2023-07-14T16:34:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.596+0000 7f713affd700 1 -- 172.21.15.62:0/2885471504 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561f31910050 con 0x561f31a6d880 2023-07-14T16:34:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.596+0000 7f713affd700 10 client.30040 handle_command_reply: tid=0 2023-07-14T16:34:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.596+0000 7f71752cfb80 1 client.30040 shutdown 2023-07-14T16:34:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f71752cfb80 1 -- 172.21.15.62:0/2885471504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 msgr2=0x561f31a13a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f71752cfb80 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a13a10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f715400faa0 tx=0x7f7154009710 comp rx=0 tx=0).stop 2023-07-14T16:34:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f71752cfb80 1 -- 172.21.15.62:0/2885471504 shutdown_connections 2023-07-14T16:34:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f71752cfb80 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x561f31a6d880 0x561f31a6fc90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f71752cfb80 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561f31a04c70 0x561f31a13a10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f71752cfb80 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561f31a034d0 0x561f31a124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f71752cfb80 1 --2- 172.21.15.62:0/2885471504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561f31a03e20 0x561f31a12a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f71752cfb80 1 -- 172.21.15.62:0/2885471504 >> 172.21.15.62:0/2885471504 conn(0x561f319f8da0 msgr2=0x561f31a0b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.599+0000 7f713affd700 0 client.30040 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:34:22.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.600+0000 7f71752cfb80 1 -- 172.21.15.62:0/2885471504 shutdown_connections 2023-07-14T16:34:22.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.600+0000 7f71752cfb80 1 -- 172.21.15.62:0/2885471504 wait complete. 2023-07-14T16:34:22.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.600+0000 7f71752cfb80 20 client.30040 trim_cache size 0 max 16384 2023-07-14T16:34:22.573 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:34:22.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 -- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71440a1250 msgr2=0x7f71440a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71440a1250 0x7f71440a3700 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f715400f350 tx=0x7f7154004be0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 -- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715c0fd990 msgr2=0x7f715c10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715c0fd990 0x7f715c10a4b0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e9f0 tx=0x7f714c00ed00 comp rx=0 tx=0).stop 2023-07-14T16:34:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 -- 172.21.15.62:0/1015975666 shutdown_connections 2023-07-14T16:34:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f715c0ff470 0x7f715c1abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715c0fd990 0x7f715c10a4b0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f71440a1250 0x7f71440a3700 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 --2- 172.21.15.62:0/1015975666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f715c0feac0 0x7f715c10a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.601+0000 7f71619ff700 1 -- 172.21.15.62:0/1015975666 >> 172.21.15.62:0/1015975666 conn(0x7f715c0f9340 msgr2=0x7f715c0f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.602+0000 7f71619ff700 1 -- 172.21.15.62:0/1015975666 shutdown_connections 2023-07-14T16:34:22.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:22.602+0000 7f71619ff700 1 -- 172.21.15.62:0/1015975666 wait complete. 2023-07-14T16:34:22.601 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:34:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 -- 172.21.15.62:0/1137429491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3470102f00 msgr2=0x7f3470103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 --2- 172.21.15.62:0/1137429491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3470102f00 0x7f3470103350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3458009990 tx=0x7f3458009ca0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.989 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 -- 172.21.15.62:0/1137429491 shutdown_connections 2023-07-14T16:34:22.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 --2- 172.21.15.62:0/1137429491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3470103890 0x7f347010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 --2- 172.21.15.62:0/1137429491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3470101d10 0x7f3470102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 --2- 172.21.15.62:0/1137429491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3470102f00 0x7f3470103350 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 -- 172.21.15.62:0/1137429491 >> 172.21.15.62:0/1137429491 conn(0x7f34700fd350 msgr2=0x7f34700ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:22.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 -- 172.21.15.62:0/1137429491 shutdown_connections 2023-07-14T16:34:22.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.017+0000 7f34755ff700 1 -- 172.21.15.62:0/1137429491 wait complete. 2023-07-14T16:34:22.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.018+0000 7f34755ff700 1 Processor -- start 2023-07-14T16:34:22.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f34755ff700 1 -- start start 2023-07-14T16:34:22.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f34755ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3470101d10 0x7f347006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f34755ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3470102f00 0x7f347006f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f34755ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3470103890 0x7f34701a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:22.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f34755ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700723f0 con 0x7f3470103890 2023-07-14T16:34:22.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f34755ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3470072560 con 0x7f3470102f00 2023-07-14T16:34:22.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f34755ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34701a8020 con 0x7f3470101d10 2023-07-14T16:34:22.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3470103890 0x7f34701a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3470103890 0x7f34701a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49292/0 (socket says 172.21.15.62:49292) 2023-07-14T16:34:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f34667fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3470102f00 0x7f347006f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f346f7fe700 1 -- 172.21.15.62:0/1095167274 learned_addr learned my addr 172.21.15.62:0/1095167274 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.019+0000 7f346effd700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3470101d10 0x7f347006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:22.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.020+0000 7f346effd700 1 -- 172.21.15.62:0/1095167274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3470102f00 msgr2=0x7f347006f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.020+0000 7f346effd700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3470102f00 0x7f347006f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.020+0000 7f346effd700 1 -- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3470103890 msgr2=0x7f34701a7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.020+0000 7f346effd700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3470103890 0x7f34701a7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.020+0000 7f346effd700 1 -- 172.21.15.62:0/1095167274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34580095f0 con 0x7f3470101d10 2023-07-14T16:34:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.020+0000 7f346effd700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3470101d10 0x7f347006edd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f346000e9c0 tx=0x7f346000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.021+0000 7f346cff9700 1 -- 172.21.15.62:0/1095167274 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460004220 con 0x7f3470101d10 2023-07-14T16:34:22.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.021+0000 7f34755ff700 1 -- 172.21.15.62:0/1095167274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3470074950 con 0x7f3470101d10 2023-07-14T16:34:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.021+0000 7f34755ff700 1 -- 172.21.15.62:0/1095167274 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3470074eb0 con 0x7f3470101d10 2023-07-14T16:34:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.021+0000 7f346cff9700 1 -- 172.21.15.62:0/1095167274 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3460004380 con 0x7f3470101d10 2023-07-14T16:34:22.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.021+0000 7f346cff9700 1 -- 172.21.15.62:0/1095167274 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460003b20 con 0x7f3470101d10 2023-07-14T16:34:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.028+0000 7f3488eefb80 1 Processor -- start 2023-07-14T16:34:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.029+0000 7f3488eefb80 1 -- start start 2023-07-14T16:34:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.029+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cc4d0 0x563c665cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.029+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cce20 0x563c665cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.029+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.029+0000 7f3488eefb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c664d1cb0 con 0x563c665cdc70 2023-07-14T16:34:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.029+0000 7f3488eefb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c66524a40 con 0x563c665cce20 2023-07-14T16:34:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.029+0000 7f3488eefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6651ded0 con 0x563c665cc4d0 2023-07-14T16:34:23.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.030+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cc4d0 0x563c665cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.030+0000 7f346cff9700 1 -- 172.21.15.62:0/1095167274 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3460003d00 con 0x7f3470101d10 2023-07-14T16:34:23.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.031+0000 7f34667fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.031+0000 7f34667fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49308/0 (socket says 172.21.15.62:49308) 2023-07-14T16:34:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.031+0000 7f34667fc700 1 -- 172.21.15.62:0/714393991 learned_addr learned my addr 172.21.15.62:0/714393991 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.031+0000 7f34667fc700 1 -- 172.21.15.62:0/714393991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cc4d0 msgr2=0x563c665cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.031+0000 7f34667fc700 1 --2- 172.21.15.62:0/714393991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cc4d0 0x563c665cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.031+0000 7f34667fc700 1 -- 172.21.15.62:0/714393991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cce20 msgr2=0x563c665cd6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.031+0000 7f34667fc700 1 --2- 172.21.15.62:0/714393991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cce20 0x563c665cd6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.031+0000 7f34667fc700 1 -- 172.21.15.62:0/714393991 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c6634bde0 con 0x563c665cdc70 2023-07-14T16:34:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.032+0000 7f34667fc700 1 --2- 172.21.15.62:0/714393991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665ce0e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f3458009990 tx=0x7f3458005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf048c962b44ca32 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.032+0000 7f346cff9700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f345c0a1250 0x7f345c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.032+0000 7f34657fa700 1 -- 172.21.15.62:0/714393991 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345801d070 con 0x563c665cdc70 2023-07-14T16:34:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.033+0000 7f346cff9700 1 -- 172.21.15.62:0/1095167274 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3460014070 con 0x7f3470101d10 2023-07-14T16:34:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.033+0000 7f34657fa700 1 -- 172.21.15.62:0/714393991 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f345801e040 con 0x563c665cdc70 2023-07-14T16:34:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.033+0000 7f34657fa700 1 -- 172.21.15.62:0/714393991 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800b910 con 0x563c665cdc70 2023-07-14T16:34:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.033+0000 7f346f7fe700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f345c0a1250 0x7f345c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.033+0000 7f3488eefb80 1 -- 172.21.15.62:0/714393991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 msgr2=0x563c665ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.033+0000 7f3488eefb80 1 --2- 172.21.15.62:0/714393991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665ce0e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f3458009990 tx=0x7f3458005c90 comp rx=0 tx=0).stop 2023-07-14T16:34:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.034+0000 7f3488eefb80 1 -- 172.21.15.62:0/714393991 shutdown_connections 2023-07-14T16:34:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.034+0000 7f3488eefb80 1 --2- 172.21.15.62:0/714393991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665ce0e0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.034+0000 7f3488eefb80 1 --2- 172.21.15.62:0/714393991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cc4d0 0x563c665cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.034+0000 7f3488eefb80 1 --2- 172.21.15.62:0/714393991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cce20 0x563c665cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.034+0000 7f3488eefb80 1 -- 172.21.15.62:0/714393991 >> 172.21.15.62:0/714393991 conn(0x563c665c1da0 msgr2=0x563c665c71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.034+0000 7f346f7fe700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f345c0a1250 0x7f345c0a3700 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3468009510 tx=0x7f3468009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.034+0000 7f3488eefb80 1 -- 172.21.15.62:0/714393991 shutdown_connections 2023-07-14T16:34:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.034+0000 7f3488eefb80 1 -- 172.21.15.62:0/714393991 wait complete. 2023-07-14T16:34:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.035+0000 7f3488eefb80 1 -- start start 2023-07-14T16:34:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.035+0000 7f3488eefb80 1 Processor -- start 2023-07-14T16:34:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.036+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.036+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 0x563c665d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.036+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.036+0000 7f3488eefb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c664bdac0 con 0x563c665cdc70 2023-07-14T16:34:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.036+0000 7f3488eefb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c664bd7c0 con 0x563c665cc4d0 2023-07-14T16:34:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.036+0000 7f3488eefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c663a1920 con 0x563c665cce20 2023-07-14T16:34:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.036+0000 7f3488eefb80 10 client.0 _ll_register_callbacks cb 0x563c664bb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49312/0 (socket says 172.21.15.62:49312) 2023-07-14T16:34:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346f7fe700 1 -- 172.21.15.62:0/2025913763 learned_addr learned my addr 172.21.15.62:0/2025913763 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f34667fc700 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 0x563c665d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346effd700 1 -- 172.21.15.62:0/2025913763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 msgr2=0x563c665d6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346effd700 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 0x563c665d6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346effd700 1 -- 172.21.15.62:0/2025913763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 msgr2=0x563c665dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346effd700 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.038+0000 7f346effd700 1 -- 172.21.15.62:0/2025913763 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347010aa80 con 0x563c665cc4d0 2023-07-14T16:34:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.039+0000 7f346effd700 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d61f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3460099b00 tx=0x7f3460099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de80fd01d2c49d4d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.039+0000 7f34567fc700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.039+0000 7f34567fc700 1 -- 172.21.15.62:0/2025913763 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346009de30 con 0x563c665cc4d0 2023-07-14T16:34:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.039+0000 7f34567fc700 1 -- 172.21.15.62:0/2025913763 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34600a2250 con 0x563c665cc4d0 2023-07-14T16:34:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.040+0000 7f3488eefb80 1 -- 172.21.15.62:0/2025913763 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c664774f0 con 0x563c665cc4d0 2023-07-14T16:34:23.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.040+0000 7f34567fc700 1 -- 172.21.15.62:0/2025913763 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34600992f0 con 0x563c665cc4d0 2023-07-14T16:34:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.040+0000 7f3488eefb80 10 client.30143 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.040+0000 7f3488eefb80 1 -- 172.21.15.62:0/2025913763 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563c66334c60 con 0x563c665cc4d0 2023-07-14T16:34:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.041+0000 7f34567fc700 1 -- 172.21.15.62:0/2025913763 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c664774f0 con 0x563c665cc4d0 2023-07-14T16:34:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.044+0000 7f34567fc700 1 -- 172.21.15.62:0/2025913763 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3460016c00 con 0x563c665cc4d0 2023-07-14T16:34:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.044+0000 7f3488eefb80 10 client.30143 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.044+0000 7f3488eefb80 10 client.30143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.044+0000 7f3488eefb80 20 client.30143 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.044+0000 7f3488eefb80 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c665f3070 0x563c665f5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.044+0000 7f3488eefb80 4 client.30143 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.044+0000 7f3488eefb80 1 -- 172.21.15.62:0/2025913763 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c66516eb0 con 0x563c665f3070 2023-07-14T16:34:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.045+0000 7f34667fc700 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c665f3070 0x563c665f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.046+0000 7f34667fc700 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c665f3070 0x563c665f5480 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c806ee7d5fbc10 server_cookie=9e88ddf5dd2acec in_seq=0 out_seq=0 2023-07-14T16:34:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.046+0000 7f34567fc700 10 client.30143 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.047+0000 7f34567fc700 1 -- 172.21.15.62:0/2025913763 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c66516eb0 con 0x563c665f3070 2023-07-14T16:34:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.047+0000 7f34567fc700 10 client.30143 handle_command_reply: tid=0 2023-07-14T16:34:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.048+0000 7f3488eefb80 1 client.30143 shutdown 2023-07-14T16:34:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 -- 172.21.15.62:0/2025913763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 msgr2=0x563c665d61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d61f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3460099b00 tx=0x7f3460099a30 comp rx=0 tx=0).stop 2023-07-14T16:34:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 -- 172.21.15.62:0/2025913763 shutdown_connections 2023-07-14T16:34:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c665f3070 0x563c665f5480 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d61f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 --2- 172.21.15.62:0/2025913763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 0x563c665d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 -- 172.21.15.62:0/2025913763 >> 172.21.15.62:0/2025913763 conn(0x563c665c1da0 msgr2=0x563c665c68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f34567fc700 0 client.30143 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 -- 172.21.15.62:0/2025913763 shutdown_connections 2023-07-14T16:34:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.049+0000 7f3488eefb80 1 -- 172.21.15.62:0/2025913763 wait complete. 2023-07-14T16:34:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.050+0000 7f3488eefb80 20 client.30143 trim_cache size 0 max 16384 2023-07-14T16:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.072+0000 7f3488eefb80 1 Processor -- start 2023-07-14T16:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.073+0000 7f3488eefb80 1 -- start start 2023-07-14T16:34:23.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.073+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.073+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cce20 0x563c665dd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.073+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cdc70 0x563c665d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.073+0000 7f3488eefb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c665d6ff0 con 0x563c665cce20 2023-07-14T16:34:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.073+0000 7f3488eefb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c665f0410 con 0x563c665cc4d0 2023-07-14T16:34:23.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.073+0000 7f3488eefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c665cc8a0 con 0x563c665cdc70 2023-07-14T16:34:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57854/0 (socket says 172.21.15.62:57854) 2023-07-14T16:34:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346effd700 1 -- 172.21.15.62:0/4281082188 learned_addr learned my addr 172.21.15.62:0/4281082188 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346f7fe700 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cdc70 0x563c665d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f34667fc700 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cce20 0x563c665dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346effd700 1 -- 172.21.15.62:0/4281082188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cdc70 msgr2=0x563c665d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346effd700 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cdc70 0x563c665d4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346effd700 1 -- 172.21.15.62:0/4281082188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cce20 msgr2=0x563c665dd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:23.050 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346effd700 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cce20 0x563c665dd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.074+0000 7f346effd700 1 -- 172.21.15.62:0/4281082188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c66372250 con 0x563c665cc4d0 2023-07-14T16:34:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.075+0000 7f346effd700 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665ce040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f34600a1d20 tx=0x7f3460095b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5301a103d416570 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.075+0000 7f3465ffb700 1 -- 172.21.15.62:0/4281082188 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34600a2250 con 0x563c665cc4d0 2023-07-14T16:34:23.051 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.075+0000 7f3465ffb700 1 -- 172.21.15.62:0/4281082188 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f346009de30 con 0x563c665cc4d0 2023-07-14T16:34:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.075+0000 7f3465ffb700 1 -- 172.21.15.62:0/4281082188 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34600992f0 con 0x563c665cc4d0 2023-07-14T16:34:23.052 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.076+0000 7f3488eefb80 1 -- 172.21.15.62:0/4281082188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 msgr2=0x563c665ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.076+0000 7f3488eefb80 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665ce040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f34600a1d20 tx=0x7f3460095b20 comp rx=0 tx=0).stop 2023-07-14T16:34:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.078+0000 7f3488eefb80 1 -- 172.21.15.62:0/4281082188 shutdown_connections 2023-07-14T16:34:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.078+0000 7f3488eefb80 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cdc70 0x563c665d4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.078+0000 7f3488eefb80 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665ce040 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.078+0000 7f3488eefb80 1 --2- 172.21.15.62:0/4281082188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cce20 0x563c665dd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.078+0000 7f3488eefb80 1 -- 172.21.15.62:0/4281082188 >> 172.21.15.62:0/4281082188 conn(0x563c665c1da0 msgr2=0x563c665f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.079+0000 7f3488eefb80 1 -- 172.21.15.62:0/4281082188 shutdown_connections 2023-07-14T16:34:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.079+0000 7f3488eefb80 1 -- 172.21.15.62:0/4281082188 wait complete. 2023-07-14T16:34:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.080+0000 7f3488eefb80 1 -- start start 2023-07-14T16:34:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.080+0000 7f3488eefb80 1 Processor -- start 2023-07-14T16:34:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.081+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.081+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 0x563c665d9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.081+0000 7f3488eefb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.081+0000 7f3488eefb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c665dced0 con 0x563c665cdc70 2023-07-14T16:34:23.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.081+0000 7f3488eefb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c665dd040 con 0x563c665cc4d0 2023-07-14T16:34:23.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.081+0000 7f3488eefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c665dd200 con 0x563c665cce20 2023-07-14T16:34:23.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.081+0000 7f3488eefb80 10 client.0 _ll_register_callbacks cb 0x7f3460099bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:23.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.082+0000 7f34667fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 0x563c665d9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.082+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.082+0000 7f346f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49332/0 (socket says 172.21.15.62:49332) 2023-07-14T16:34:23.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.082+0000 7f346f7fe700 1 -- 172.21.15.62:0/1541722659 learned_addr learned my addr 172.21.15.62:0/1541722659 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:23.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.082+0000 7f346effd700 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.082+0000 7f346f7fe700 1 -- 172.21.15.62:0/1541722659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 msgr2=0x563c665d9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:23.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.082+0000 7f346f7fe700 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 0x563c665d9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.083+0000 7f346f7fe700 1 -- 172.21.15.62:0/1541722659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 msgr2=0x563c665d9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:23.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.083+0000 7f346f7fe700 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.083+0000 7f346f7fe700 1 -- 172.21.15.62:0/1541722659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c66334c60 con 0x563c665cdc70 2023-07-14T16:34:23.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.083+0000 7f346f7fe700 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dc990 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f3468007c30 tx=0x7f3468004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5354c8a27968b8c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:23.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.083+0000 7f34567fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:34:23.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.083+0000 7f34567fc700 1 -- 172.21.15.62:0/1541722659 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468018050 con 0x563c665cdc70 2023-07-14T16:34:23.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.083+0000 7f34567fc700 1 -- 172.21.15.62:0/1541722659 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f346801fb90 con 0x563c665cdc70 2023-07-14T16:34:23.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.083+0000 7f3488eefb80 1 -- 172.21.15.62:0/1541722659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f345801e040 con 0x563c665cdc70 2023-07-14T16:34:23.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.084+0000 7f34567fc700 1 -- 172.21.15.62:0/1541722659 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f346801fd50 con 0x563c665cdc70 2023-07-14T16:34:23.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.084+0000 7f3488eefb80 10 client.20601 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:23.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.084+0000 7f3488eefb80 1 -- 172.21.15.62:0/1541722659 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563c6634a2b0 con 0x563c665cdc70 2023-07-14T16:34:23.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.084+0000 7f34567fc700 1 -- 172.21.15.62:0/1541722659 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f345801e040 con 0x563c665cdc70 2023-07-14T16:34:23.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.085+0000 7f34567fc700 1 -- 172.21.15.62:0/1541722659 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f34680294b0 con 0x563c665cdc70 2023-07-14T16:34:23.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.085+0000 7f3488eefb80 10 client.20601 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:23.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.085+0000 7f3488eefb80 10 client.20601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:23.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.085+0000 7f3488eefb80 20 client.20601 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:23.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.085+0000 7f3488eefb80 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c66647fc0 0x563c6664a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:23.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.085+0000 7f3488eefb80 4 client.20601 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:34:23.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.085+0000 7f3488eefb80 1 -- 172.21.15.62:0/1541722659 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c664d9050 con 0x563c66647fc0 2023-07-14T16:34:23.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.086+0000 7f346effd700 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c66647fc0 0x563c6664a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:23.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.086+0000 7f346effd700 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c66647fc0 0x563c6664a3d0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc10c6d632c0d83 server_cookie=980f651f5534c120 in_seq=0 out_seq=0 2023-07-14T16:34:23.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.086+0000 7f34567fc700 10 client.20601 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:23.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.086+0000 7f34567fc700 1 -- 172.21.15.62:0/1541722659 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c664d9050 con 0x563c66647fc0 2023-07-14T16:34:23.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.086+0000 7f34567fc700 10 client.20601 handle_command_reply: tid=0 2023-07-14T16:34:23.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.086+0000 7f3488eefb80 1 client.20601 shutdown 2023-07-14T16:34:23.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f3488eefb80 1 -- 172.21.15.62:0/1541722659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 msgr2=0x563c665dc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:23.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f3488eefb80 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dc990 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f3468007c30 tx=0x7f3468004b40 comp rx=0 tx=0).stop 2023-07-14T16:34:23.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f3488eefb80 1 -- 172.21.15.62:0/1541722659 shutdown_connections 2023-07-14T16:34:23.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f3488eefb80 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563c665cdc70 0x563c665dc990 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f3488eefb80 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x563c66647fc0 0x563c6664a3d0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f3488eefb80 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c665cc4d0 0x563c665d9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f3488eefb80 1 --2- 172.21.15.62:0/1541722659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c665cce20 0x563c665d9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f3488eefb80 1 -- 172.21.15.62:0/1541722659 >> 172.21.15.62:0/1541722659 conn(0x563c665c1da0 msgr2=0x563c665d4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:23.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.093+0000 7f34567fc700 0 client.20601 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:23.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.094+0000 7f3488eefb80 1 -- 172.21.15.62:0/1541722659 shutdown_connections 2023-07-14T16:34:23.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.094+0000 7f3488eefb80 1 -- 172.21.15.62:0/1541722659 wait complete. 2023-07-14T16:34:23.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.095+0000 7f3488eefb80 20 client.20601 trim_cache size 0 max 16384 2023-07-14T16:34:23.068 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:34:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 -- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f345c0a1250 msgr2=0x7f345c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f345c0a1250 0x7f345c0a3700 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3468009510 tx=0x7f3468009040 comp rx=0 tx=0).stop 2023-07-14T16:34:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 -- 172.21.15.62:0/1095167274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3470101d10 msgr2=0x7f347006edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3470101d10 0x7f347006edd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f346000e9c0 tx=0x7f346000ecd0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 -- 172.21.15.62:0/1095167274 shutdown_connections 2023-07-14T16:34:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3470103890 0x7f34701a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3470101d10 0x7f347006edd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f345c0a1250 0x7f345c0a3700 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 --2- 172.21.15.62:0/1095167274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3470102f00 0x7f347006f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 -- 172.21.15.62:0/1095167274 >> 172.21.15.62:0/1095167274 conn(0x7f34700fd350 msgr2=0x7f34700ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 -- 172.21.15.62:0/1095167274 shutdown_connections 2023-07-14T16:34:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:23.096+0000 7f34755ff700 1 -- 172.21.15.62:0/1095167274 wait complete. 2023-07-14T16:34:23.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:22 smithi164 ceph-mon[97427]: pgmap v919: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2023-07-14T16:34:23.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:22 smithi062 ceph-mon[95543]: pgmap v919: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2023-07-14T16:34:23.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:22 smithi081 ceph-mon[127795]: pgmap v919: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 4.14k op/s 2023-07-14T16:34:24.099 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T16:34:24.100 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T16:34:24.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.520+0000 7fc027bff700 1 -- 172.21.15.62:0/2830255480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020107a20 msgr2=0x7fc020107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.520+0000 7fc027bff700 1 --2- 172.21.15.62:0/2830255480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020107a20 0x7fc020107e30 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009990 tx=0x7fc01c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.520+0000 7fc027bff700 1 -- 172.21.15.62:0/2830255480 shutdown_connections 2023-07-14T16:34:24.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.520+0000 7fc027bff700 1 --2- 172.21.15.62:0/2830255480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc020071620 0x7fc020110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.520+0000 7fc027bff700 1 --2- 172.21.15.62:0/2830255480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020107a20 0x7fc020107e30 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.520+0000 7fc027bff700 1 --2- 172.21.15.62:0/2830255480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc020070c70 0x7fc0200710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.520+0000 7fc027bff700 1 -- 172.21.15.62:0/2830255480 >> 172.21.15.62:0/2830255480 conn(0x7fc02006ca90 msgr2=0x7fc02006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:24.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.521+0000 7fc027bff700 1 -- 172.21.15.62:0/2830255480 shutdown_connections 2023-07-14T16:34:24.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.521+0000 7fc027bff700 1 -- 172.21.15.62:0/2830255480 wait complete. 2023-07-14T16:34:24.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.522+0000 7fc027bff700 1 Processor -- start 2023-07-14T16:34:24.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.522+0000 7fc027bff700 1 -- start start 2023-07-14T16:34:24.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc027bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020070c70 0x7fc0201a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc027bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc020071620 0x7fc0201a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc027bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc020107a20 0x7fc0201abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc027bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201ac110 con 0x7fc020070c70 2023-07-14T16:34:24.495 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc027bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc020116510 con 0x7fc020071620 2023-07-14T16:34:24.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc027bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201166a0 con 0x7fc020107a20 2023-07-14T16:34:24.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc02519a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc020071620 0x7fc0201a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc02519a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc020071620 0x7fc0201a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57878/0 (socket says 172.21.15.62:57878) 2023-07-14T16:34:24.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc02519a700 1 -- 172.21.15.62:0/2799470860 learned_addr learned my addr 172.21.15.62:0/2799470860 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:24.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.523+0000 7fc02599b700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020070c70 0x7fc0201a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02519a700 1 -- 172.21.15.62:0/2799470860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc020107a20 msgr2=0x7fc0201abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02619c700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc020107a20 0x7fc0201abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02519a700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc020107a20 0x7fc0201abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02519a700 1 -- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020070c70 msgr2=0x7fc0201a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02519a700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020070c70 0x7fc0201a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02519a700 1 -- 172.21.15.62:0/2799470860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c0095f0 con 0x7fc020071620 2023-07-14T16:34:24.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02619c700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc020107a20 0x7fc0201abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:34:24.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02599b700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020070c70 0x7fc0201a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:34:24.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc02519a700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc020071620 0x7fc0201a4ad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc01000e9a0 tx=0x7fc01000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:24.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.524+0000 7fc016ffd700 1 -- 172.21.15.62:0/2799470860 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010009790 con 0x7fc020071620 2023-07-14T16:34:24.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.525+0000 7fc016ffd700 1 -- 172.21.15.62:0/2799470860 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc01000cbd0 con 0x7fc020071620 2023-07-14T16:34:24.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.525+0000 7fc016ffd700 1 -- 172.21.15.62:0/2799470860 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010018680 con 0x7fc020071620 2023-07-14T16:34:24.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.525+0000 7fc027bff700 1 -- 172.21.15.62:0/2799470860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc020116980 con 0x7fc020071620 2023-07-14T16:34:24.499 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.525+0000 7fc027bff700 1 -- 172.21.15.62:0/2799470860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc020116e40 con 0x7fc020071620 2023-07-14T16:34:24.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.537+0000 7fc016ffd700 1 -- 172.21.15.62:0/2799470860 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc0100187e0 con 0x7fc020071620 2023-07-14T16:34:24.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.537+0000 7fc03b518b80 1 Processor -- start 2023-07-14T16:34:24.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.537+0000 7fc03b518b80 1 -- start start 2023-07-14T16:34:24.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.538+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938550 0x5572ba938960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.538+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938ea0 0x5572ba939760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.538+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba93a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.538+0000 7fc03b518b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba83dd30 con 0x5572ba939cf0 2023-07-14T16:34:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.538+0000 7fc03b518b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba890ac0 con 0x5572ba938550 2023-07-14T16:34:24.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.538+0000 7fc03b518b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba889f50 con 0x5572ba938ea0 2023-07-14T16:34:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.539+0000 7fc02599b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938ea0 0x5572ba939760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.539+0000 7fc02599b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938ea0 0x5572ba939760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39640/0 (socket says 172.21.15.62:39640) 2023-07-14T16:34:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.539+0000 7fc02599b700 1 -- 172.21.15.62:0/2579759241 learned_addr learned my addr 172.21.15.62:0/2579759241 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.539+0000 7fc02619c700 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938550 0x5572ba938960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.539+0000 7fc02519a700 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba93a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.539+0000 7fc02599b700 1 -- 172.21.15.62:0/2579759241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938550 msgr2=0x5572ba938960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:24.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.539+0000 7fc02599b700 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938550 0x5572ba938960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.539+0000 7fc02599b700 1 -- 172.21.15.62:0/2579759241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 msgr2=0x5572ba93a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:24.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.540+0000 7fc02599b700 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba93a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.540+0000 7fc02599b700 1 -- 172.21.15.62:0/2579759241 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572ba6b7e60 con 0x5572ba938ea0 2023-07-14T16:34:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.540+0000 7fc02619c700 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938550 0x5572ba938960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:34:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.540+0000 7fc02599b700 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938ea0 0x5572ba939760 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00b2e0 tx=0x7fc01c005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8893049ab5dcbbcb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.541+0000 7fc016ffd700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc00c0a11a0 0x7fc00c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.541+0000 7fc00bfff700 1 -- 172.21.15.62:0/2579759241 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c005230 con 0x5572ba938ea0 2023-07-14T16:34:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.541+0000 7fc00bfff700 1 -- 172.21.15.62:0/2579759241 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc01c01e040 con 0x5572ba938ea0 2023-07-14T16:34:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.541+0000 7fc00bfff700 1 -- 172.21.15.62:0/2579759241 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00b970 con 0x5572ba938ea0 2023-07-14T16:34:24.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.541+0000 7fc016ffd700 1 -- 172.21.15.62:0/2799470860 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc010014070 con 0x7fc020071620 2023-07-14T16:34:24.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.541+0000 7fc02519a700 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba93a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:34:24.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.542+0000 7fc03b518b80 1 -- 172.21.15.62:0/2579759241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938ea0 msgr2=0x5572ba939760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.542+0000 7fc03b518b80 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938ea0 0x5572ba939760 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00b2e0 tx=0x7fc01c005c60 comp rx=0 tx=0).stop 2023-07-14T16:34:24.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.542+0000 7fc02619c700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc00c0a11a0 0x7fc00c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.544+0000 7fc03b518b80 1 -- 172.21.15.62:0/2579759241 shutdown_connections 2023-07-14T16:34:24.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.544+0000 7fc03b518b80 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba93a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.544+0000 7fc03b518b80 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938550 0x5572ba938960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.544+0000 7fc03b518b80 1 --2- 172.21.15.62:0/2579759241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938ea0 0x5572ba939760 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.544+0000 7fc03b518b80 1 -- 172.21.15.62:0/2579759241 >> 172.21.15.62:0/2579759241 conn(0x5572ba92de20 msgr2=0x5572ba933250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:24.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.545+0000 7fc03b518b80 1 -- 172.21.15.62:0/2579759241 shutdown_connections 2023-07-14T16:34:24.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.545+0000 7fc02619c700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc00c0a11a0 0x7fc00c0a3650 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc018009550 tx=0x7fc01800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:24.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.545+0000 7fc03b518b80 1 -- 172.21.15.62:0/2579759241 wait complete. 2023-07-14T16:34:24.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.546+0000 7fc03b518b80 1 -- start start 2023-07-14T16:34:24.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.546+0000 7fc03b518b80 1 Processor -- start 2023-07-14T16:34:24.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.547+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938550 0x5572ba942180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.547+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba9426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.547+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba949700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.547+0000 7fc03b518b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba829b40 con 0x5572ba939cf0 2023-07-14T16:34:24.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.547+0000 7fc03b518b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba829840 con 0x5572ba938ea0 2023-07-14T16:34:24.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.547+0000 7fc03b518b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba70d9a0 con 0x5572ba938550 2023-07-14T16:34:24.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.547+0000 7fc03b518b80 10 client.0 _ll_register_callbacks cb 0x5572ba8277e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:24.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.548+0000 7fc02619c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba949700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.548+0000 7fc02619c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba949700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49380/0 (socket says 172.21.15.62:49380) 2023-07-14T16:34:24.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.548+0000 7fc02619c700 1 -- 172.21.15.62:0/364558603 learned_addr learned my addr 172.21.15.62:0/364558603 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:24.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.548+0000 7fc02519a700 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba9426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.549+0000 7fc02619c700 1 -- 172.21.15.62:0/364558603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938550 msgr2=0x5572ba942180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.549+0000 7fc02619c700 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938550 0x5572ba942180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.549+0000 7fc02619c700 1 -- 172.21.15.62:0/364558603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 msgr2=0x5572ba9426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.549+0000 7fc02619c700 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba9426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.549+0000 7fc02619c700 1 -- 172.21.15.62:0/364558603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc020111050 con 0x5572ba939cf0 2023-07-14T16:34:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.549+0000 7fc02619c700 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba949700 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc0180179e0 tx=0x7fc018004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5da8154415a1598 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.549+0000 7fc0097fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:34:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.562+0000 7fc0097fa700 1 -- 172.21.15.62:0/364558603 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018007e80 con 0x5572ba939cf0 2023-07-14T16:34:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.562+0000 7fc0097fa700 1 -- 172.21.15.62:0/364558603 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc018015a80 con 0x5572ba939cf0 2023-07-14T16:34:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.562+0000 7fc03b518b80 1 -- 172.21.15.62:0/364558603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572ba7e3570 con 0x5572ba939cf0 2023-07-14T16:34:24.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.563+0000 7fc0097fa700 1 -- 172.21.15.62:0/364558603 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc018017e50 con 0x5572ba939cf0 2023-07-14T16:34:24.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.563+0000 7fc03b518b80 10 client.20610 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:24.535 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.563+0000 7fc03b518b80 1 -- 172.21.15.62:0/364558603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5572ba6a0ce0 con 0x5572ba939cf0 2023-07-14T16:34:24.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.564+0000 7fc0097fa700 1 -- 172.21.15.62:0/364558603 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5572ba7e3570 con 0x5572ba939cf0 2023-07-14T16:34:24.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.564+0000 7fc0097fa700 1 -- 172.21.15.62:0/364558603 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc01802a070 con 0x5572ba939cf0 2023-07-14T16:34:24.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.564+0000 7fc03b518b80 10 client.20610 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:24.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.564+0000 7fc03b518b80 10 client.20610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:24.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.564+0000 7fc03b518b80 20 client.20610 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:24.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.564+0000 7fc03b518b80 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5572ba94df00 0x5572ba950310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.564+0000 7fc03b518b80 4 client.20610 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:24.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.564+0000 7fc03b518b80 1 -- 172.21.15.62:0/364558603 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572ba882f30 con 0x5572ba94df00 2023-07-14T16:34:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.566+0000 7fc02599b700 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5572ba94df00 0x5572ba950310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.567+0000 7fc0097fa700 10 client.20610 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:24.539 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.567+0000 7fc02599b700 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5572ba94df00 0x5572ba950310 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffdd4ffb35eaa9ca server_cookie=dbeafdd851d69c27 in_seq=0 out_seq=0 2023-07-14T16:34:24.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.568+0000 7fc0097fa700 1 -- 172.21.15.62:0/364558603 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5572ba882f30 con 0x5572ba94df00 2023-07-14T16:34:24.540 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.568+0000 7fc0097fa700 10 client.20610 handle_command_reply: tid=0 2023-07-14T16:34:24.541 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.568+0000 7fc03b518b80 1 client.20610 shutdown 2023-07-14T16:34:24.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc03b518b80 1 -- 172.21.15.62:0/364558603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 msgr2=0x5572ba949700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc03b518b80 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba949700 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc0180179e0 tx=0x7fc018004b40 comp rx=0 tx=0).stop 2023-07-14T16:34:24.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc03b518b80 1 -- 172.21.15.62:0/364558603 shutdown_connections 2023-07-14T16:34:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc03b518b80 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba939cf0 0x5572ba949700 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc03b518b80 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba938550 0x5572ba942180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc03b518b80 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5572ba94df00 0x5572ba950310 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc03b518b80 1 --2- 172.21.15.62:0/364558603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba9426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc03b518b80 1 -- 172.21.15.62:0/364558603 >> 172.21.15.62:0/364558603 conn(0x5572ba92de20 msgr2=0x5572ba932930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.569+0000 7fc0097fa700 0 client.20610 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:24.544 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.570+0000 7fc03b518b80 1 -- 172.21.15.62:0/364558603 shutdown_connections 2023-07-14T16:34:24.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.570+0000 7fc03b518b80 1 -- 172.21.15.62:0/364558603 wait complete. 2023-07-14T16:34:24.545 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.570+0000 7fc03b518b80 20 client.20610 trim_cache size 0 max 16384 2023-07-14T16:34:24.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.610+0000 7fc03b518b80 1 Processor -- start 2023-07-14T16:34:24.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.610+0000 7fc03b518b80 1 -- start start 2023-07-14T16:34:24.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.610+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba940040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.610+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba9496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.582 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.610+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 0x5572ba942180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.610+0000 7fc03b518b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba9395d0 con 0x5572ba938550 2023-07-14T16:34:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.610+0000 7fc03b518b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba94b2e0 con 0x5572ba938ea0 2023-07-14T16:34:24.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.610+0000 7fc03b518b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba943080 con 0x5572ba939cf0 2023-07-14T16:34:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02599b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba940040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02599b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba940040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49388/0 (socket says 172.21.15.62:49388) 2023-07-14T16:34:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02599b700 1 -- 172.21.15.62:0/4129999508 learned_addr learned my addr 172.21.15.62:0/4129999508 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02519a700 1 --2- 172.21.15.62:0/4129999508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba9496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02599b700 1 -- 172.21.15.62:0/4129999508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 msgr2=0x5572ba942180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:24.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02599b700 1 --2- 172.21.15.62:0/4129999508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 0x5572ba942180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02599b700 1 -- 172.21.15.62:0/4129999508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 msgr2=0x5572ba9496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02599b700 1 --2- 172.21.15.62:0/4129999508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba9496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.611+0000 7fc02599b700 1 -- 172.21.15.62:0/4129999508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572ba6de2d0 con 0x5572ba938550 2023-07-14T16:34:24.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.612+0000 7fc02599b700 1 --2- 172.21.15.62:0/4129999508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba940040 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fc01c003f60 tx=0x7fc01c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fcf1a8b2c814a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.613+0000 7fc014bff700 1 -- 172.21.15.62:0/4129999508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c003900 con 0x5572ba938550 2023-07-14T16:34:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.613+0000 7fc014bff700 1 -- 172.21.15.62:0/4129999508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc01c02de80 con 0x5572ba938550 2023-07-14T16:34:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.613+0000 7fc014bff700 1 -- 172.21.15.62:0/4129999508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00b710 con 0x5572ba938550 2023-07-14T16:34:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.615+0000 7fc03b518b80 1 -- 172.21.15.62:0/4129999508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 msgr2=0x5572ba940040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.615+0000 7fc03b518b80 1 --2- 172.21.15.62:0/4129999508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba940040 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fc01c003f60 tx=0x7fc01c0092d0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.616+0000 7fc03b518b80 1 -- 172.21.15.62:0/4129999508 shutdown_connections 2023-07-14T16:34:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.616+0000 7fc03b518b80 1 --2- 172.21.15.62:0/4129999508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 0x5572ba942180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.616+0000 7fc03b518b80 1 --2- 172.21.15.62:0/4129999508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba940040 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.616+0000 7fc03b518b80 1 --2- 172.21.15.62:0/4129999508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba9496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.616+0000 7fc03b518b80 1 -- 172.21.15.62:0/4129999508 >> 172.21.15.62:0/4129999508 conn(0x5572ba92de20 msgr2=0x5572ba932930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.616+0000 7fc03b518b80 1 -- 172.21.15.62:0/4129999508 shutdown_connections 2023-07-14T16:34:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.617+0000 7fc03b518b80 1 -- 172.21.15.62:0/4129999508 wait complete. 2023-07-14T16:34:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.617+0000 7fc03b518b80 1 -- start start 2023-07-14T16:34:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.618+0000 7fc03b518b80 1 Processor -- start 2023-07-14T16:34:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.618+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba946440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.618+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba946980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.618+0000 7fc03b518b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 0x5572ba9a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.618+0000 7fc03b518b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba946f50 con 0x5572ba938550 2023-07-14T16:34:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.618+0000 7fc03b518b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba9470c0 con 0x5572ba938ea0 2023-07-14T16:34:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.618+0000 7fc03b518b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ba9a2370 con 0x5572ba939cf0 2023-07-14T16:34:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.618+0000 7fc03b518b80 10 client.0 _ll_register_callbacks cb 0x7fc018007dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.619+0000 7fc02619c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 0x5572ba9a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.619+0000 7fc02619c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 0x5572ba9a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39668/0 (socket says 172.21.15.62:39668) 2023-07-14T16:34:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.619+0000 7fc02619c700 1 -- 172.21.15.62:0/1852905009 learned_addr learned my addr 172.21.15.62:0/1852905009 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.619+0000 7fc02519a700 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba946980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.619+0000 7fc02599b700 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba946440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.620+0000 7fc02519a700 1 -- 172.21.15.62:0/1852905009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 msgr2=0x5572ba9a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.620+0000 7fc02519a700 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 0x5572ba9a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.620+0000 7fc02519a700 1 -- 172.21.15.62:0/1852905009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 msgr2=0x5572ba946440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.620+0000 7fc02519a700 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba946440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.620+0000 7fc02519a700 1 -- 172.21.15.62:0/1852905009 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c027040 con 0x5572ba938ea0 2023-07-14T16:34:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.620+0000 7fc02519a700 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba946980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc0100ca6c0 tx=0x7fc01009a4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf470e3e070befd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.620+0000 7fc0097fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.620+0000 7fc0097fa700 1 -- 172.21.15.62:0/1852905009 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0100a23d0 con 0x5572ba938ea0 2023-07-14T16:34:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.621+0000 7fc0097fa700 1 -- 172.21.15.62:0/1852905009 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0100a1f40 con 0x5572ba938ea0 2023-07-14T16:34:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.621+0000 7fc03b518b80 1 -- 172.21.15.62:0/1852905009 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572ba8450d0 con 0x5572ba938ea0 2023-07-14T16:34:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.621+0000 7fc0097fa700 1 -- 172.21.15.62:0/1852905009 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0100a2100 con 0x5572ba938ea0 2023-07-14T16:34:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.621+0000 7fc03b518b80 10 client.30167 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.621+0000 7fc03b518b80 1 -- 172.21.15.62:0/1852905009 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5572ba6b6330 con 0x5572ba938ea0 2023-07-14T16:34:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.623+0000 7fc0097fa700 1 -- 172.21.15.62:0/1852905009 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0100b1030 con 0x5572ba938ea0 2023-07-14T16:34:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.624+0000 7fc0097fa700 1 -- 172.21.15.62:0/1852905009 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc010095e00 con 0x5572ba938ea0 2023-07-14T16:34:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.624+0000 7fc03b518b80 10 client.30167 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.624+0000 7fc03b518b80 10 client.30167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.624+0000 7fc03b518b80 20 client.30167 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.624+0000 7fc03b518b80 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5572ba9a3dc0 0x5572ba9a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.624+0000 7fc03b518b80 4 client.30167 mds_command: new command op to 24602 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T16:34:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.624+0000 7fc03b518b80 1 -- 172.21.15.62:0/1852905009 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5572ba6d8ff0 con 0x5572ba9a3dc0 2023-07-14T16:34:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.626+0000 7fc02599b700 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5572ba9a3dc0 0x5572ba9a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.627+0000 7fc02599b700 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5572ba9a3dc0 0x5572ba9a61d0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=461dfc471cc4900b server_cookie=fc445410e2c8d4d0 in_seq=0 out_seq=0 2023-07-14T16:34:24.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc0097fa700 10 client.30167 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc0097fa700 1 -- 172.21.15.62:0/1852905009 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5572ba6d8ff0 con 0x5572ba9a3dc0 2023-07-14T16:34:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc0097fa700 10 client.30167 handle_command_reply: tid=0 2023-07-14T16:34:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc03b518b80 1 client.30167 shutdown 2023-07-14T16:34:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc03b518b80 1 -- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 msgr2=0x5572ba946980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc03b518b80 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba946980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc0100ca6c0 tx=0x7fc01009a4c0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc03b518b80 1 -- 172.21.15.62:0/1852905009 shutdown_connections 2023-07-14T16:34:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc03b518b80 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572ba939cf0 0x5572ba9a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc03b518b80 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572ba938550 0x5572ba946440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.628+0000 7fc03b518b80 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5572ba9a3dc0 0x5572ba9a61d0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.629+0000 7fc03b518b80 1 --2- 172.21.15.62:0/1852905009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5572ba938ea0 0x5572ba946980 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.629+0000 7fc03b518b80 1 -- 172.21.15.62:0/1852905009 >> 172.21.15.62:0/1852905009 conn(0x5572ba92de20 msgr2=0x5572ba940040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.629+0000 7fc0097fa700 0 client.30167 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.629+0000 7fc03b518b80 1 -- 172.21.15.62:0/1852905009 shutdown_connections 2023-07-14T16:34:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.629+0000 7fc03b518b80 1 -- 172.21.15.62:0/1852905009 wait complete. 2023-07-14T16:34:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.629+0000 7fc03b518b80 20 client.30167 trim_cache size 0 max 16384 2023-07-14T16:34:24.603 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:34:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2023-07-14T16:34:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "34613b6b-2bfd-4e57-a479-708b9df79e2c", 2023-07-14T16:34:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2023-07-14T16:34:24.603 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:34:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.632+0000 7fc027bff700 1 -- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc00c0a11a0 msgr2=0x7fc00c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.632+0000 7fc027bff700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc00c0a11a0 0x7fc00c0a3650 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc018009550 tx=0x7fc01800b040 comp rx=0 tx=0).stop 2023-07-14T16:34:24.606 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.632+0000 7fc027bff700 1 -- 172.21.15.62:0/2799470860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc020071620 msgr2=0x7fc0201a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.632+0000 7fc027bff700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc020071620 0x7fc0201a4ad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc01000e9a0 tx=0x7fc01000ed60 comp rx=0 tx=0).stop 2023-07-14T16:34:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.633+0000 7fc027bff700 1 -- 172.21.15.62:0/2799470860 shutdown_connections 2023-07-14T16:34:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.633+0000 7fc027bff700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc020107a20 0x7fc0201abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.633+0000 7fc027bff700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc020070c70 0x7fc0201a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.633+0000 7fc027bff700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fc00c0a11a0 0x7fc00c0a3650 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.633+0000 7fc027bff700 1 --2- 172.21.15.62:0/2799470860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc020071620 0x7fc0201a4ad0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:24.607 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.633+0000 7fc027bff700 1 -- 172.21.15.62:0/2799470860 >> 172.21.15.62:0/2799470860 conn(0x7fc02006ca90 msgr2=0x7fc02006fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.637+0000 7fc027bff700 1 -- 172.21.15.62:0/2799470860 shutdown_connections 2023-07-14T16:34:24.609 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:24.638+0000 7fc027bff700 1 -- 172.21.15.62:0/2799470860 wait complete. 2023-07-14T16:34:24.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:34:24.897 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.639+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002bd [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in~ auth v3749 ap=1 s=3524 n(v0 rc2023-07-14T16:17:21.050144+0000 b3524 1=1+0) (iversion lock) caps={25145=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c1dae53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000002bd:[<0x10000000007/config.h.in~ v3490>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000002bd:[<0x10000000007/config.h.in~ v3749>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v330>,<0x10000000003/client.0 v331>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v344>,<0x10000000001/sv_0 v352>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.897 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.639+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c3 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rwlock.o auth v3729 ap=1 s=14016 n(v0 rc2023-07-14T16:17:22.604137+0000 b14016 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc0f6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c3:[<0x10000000007/rwlock.o v3729>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c3:[<0x10000000007/rwlock.o v3729>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v330>,<0x10000000003/client.0 v331>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v344>,<0x10000000001/sv_0 v352>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.897 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.641+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c4 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/cirlist.o auth v3727 ap=1 s=14328 n(v0 rc2023-07-14T16:17:22.736807+0000 b14328 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc0b2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c4:[<0x10000000007/cirlist.o v3727>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c4:[<0x10000000007/cirlist.o v3727>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v330>,<0x10000000003/client.0 v331>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v344>,<0x10000000001/sv_0 v352>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.898 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.643+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004bb [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h auth v3707 ap=1 s=4082 n(v0 rc2023-07-14T16:17:21.212851+0000 b4082 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc018680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004bb:[<0x10000000007/config.h v3534>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000004bb:[<0x10000000007/config.h v3707>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v330>,<0x10000000003/client.0 v331>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v344>,<0x10000000001/sv_0 v352>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.898 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.645+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c7 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fs.o auth v3721 ap=1 s=65480 n(v0 rc2023-07-14T16:17:23.443570+0000 b65480 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc159180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c7:[<0x10000000007/ffsb_fs.o v3721>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c7:[<0x10000000007/ffsb_fs.o v3721>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v330>,<0x10000000003/client.0 v331>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v344>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.898 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.647+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004cf [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb auth v3689 ap=1 s=423704 n(v0 rc2023-07-14T16:17:25.614776+0000 b423704 1=1+0) (ifile excl) (iversion lock) cr={25145=0-4194304@1} caps={25145=pAsLsXsFsxcrwb/pFscr@1},l=25145 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc156b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004cf:[<0x10000000007/ffsb v3689>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004cf:[<0x10000000007/ffsb v3689>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v330>,<0x10000000003/client.0 v331>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v344>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.898 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.647+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004d0 [...2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/tmp/ auth{0=1,1=1} v3537 ap=2 f(v0 m2023-07-14T16:17:25.662917+0000 2=0+2) n(v121 rc2023-07-14T16:34:21.295742+0000 b13421772800 131=128+3) (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25145=pAsLsXsFs/-@1} | dirtyscattered=0 lock=3 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55c1dae54100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000004d0:[<0x10000000006/tmp v3290>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(2)0x100000004d0:[<0x10000000006/tmp v3535>,<0x10000000005/tmp v330>,<0x10000000003/client.0 v331>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v344>,<0x10000000001/sv_0 v352>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T16:17:25.662917+0000 2=0+2)","ondisk_value.rstat":"n(v0 rc2023-07-14T16:34:17.709499+0000 b13421772800 131=128+3)","memory_value.dirstat":"f(v0 m2023-07-14T16:17:25.662917+0000 2=0+2)","memory_value.rstat":"n(v121 rc2023-07-14T16:34:21.295742+0000 b13421772800 131=128+3)","error_str":""},"return_code":0} 2023-07-14T16:34:24.898 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.648+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004cb [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/parser.o auth v3743 ap=1 s=178656 n(v0 rc2023-07-14T16:17:24.980270+0000 b178656 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc159700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004cb:[<0x10000000007/parser.o v3743>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004cb:[<0x10000000007/parser.o v3743>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.898 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.648+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004b4 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in auth v3528 ap=1 s=3824 n(v0 rc2023-07-14T16:17:21.070120+0000 b3824 1=1+0) (iversion lock) caps={25145=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dbf88580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004b4:[<0x10000000007/config.h.in v3528>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000004b4:[<0x10000000007/config.h.in v3528>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.899 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.649+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c6 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_tg.o auth v3723 ap=1 s=53576 n(v0 rc2023-07-14T16:17:23.194952+0000 b53576 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dbde6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c6:[<0x10000000007/ffsb_tg.o v3723>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c6:[<0x10000000007/ffsb_tg.o v3723>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.899 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.652+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c0 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fh.o auth v3735 ap=1 s=55416 n(v0 rc2023-07-14T16:17:22.089148+0000 b55416 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc181700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c0:[<0x10000000007/fh.o v3562>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000004c0:[<0x10000000007/fh.o v3735>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.899 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.652+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c9 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_op.o auth v3717 ap=1 s=34184 n(v0 rc2023-07-14T16:17:23.777132+0000 b34184 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc17fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c9:[<0x10000000007/ffsb_op.o v3717>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c9:[<0x10000000007/ffsb_op.o v3717>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.899 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.652+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004b2 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/Makefile auth v3701 ap=1 s=19637 n(v0 rc2023-07-14T16:17:20.572262+0000 b19637 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc181180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004b2:[<0x10000000007/Makefile v3454>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000004b2:[<0x10000000007/Makefile v3701>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.899 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.652+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004bc [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/stamp-h1 auth v3695 ap=1 s=23 n(v0 rc2023-07-14T16:17:21.220521+0000 b23 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc184580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004bc:[<0x10000000007/stamp-h1 v3538>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000004bc:[<0x10000000007/stamp-h1 v3695>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.899 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.652+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004be [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rand.o auth v3737 ap=1 s=20416 n(v0 rc2023-07-14T16:17:21.693475+0000 b20416 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc184b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004be:[<0x10000000007/rand.o v3550>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000004be:[<0x10000000007/rand.o v3737>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.899 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.655+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004cd [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_stats.o auth v3711 ap=1 s=25560 n(v0 rc2023-07-14T16:17:25.307117+0000 b25560 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc18cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004cd:[<0x10000000007/ffsb_stats.o v3711>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004cd:[<0x10000000007/ffsb_stats.o v3711>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.900 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.655+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004bd [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fileops.o auth v3739 ap=1 s=84832 n(v0 rc2023-07-14T16:17:21.539484+0000 b84832 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc18b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004bd:[<0x10000000007/fileops.o v3544>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000004bd:[<0x10000000007/fileops.o v3739>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.900 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.655+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c8 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_thread.o auth v3719 ap=1 s=35704 n(v0 rc2023-07-14T16:17:23.599030+0000 b35704 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc18d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c8:[<0x10000000007/ffsb_thread.o v3719>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c8:[<0x10000000007/ffsb_thread.o v3719>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.900 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.656+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004ce [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/list.o auth v3709 ap=1 s=6384 n(v0 rc2023-07-14T16:17:25.404469+0000 b6384 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc18d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004ce:[<0x10000000007/list.o v3709>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004ce:[<0x10000000007/list.o v3709>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.900 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.656+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004bf [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/main.o auth v3741 ap=1 s=44504 n(v0 rc2023-07-14T16:17:21.890556+0000 b44504 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc262b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004bf:[<0x10000000007/main.o v3556>,<0x10000000006/ffsb v3103>,<0x10000000005/tmp v62>,<0x10000000003/client.0 v61>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v69>,<0x10000000001/sv_0 v90>,<0x10000000000/_nogroup v76>,<0x1/volumes v73>]//[]","memoryvalue":"(3)0x100000004bf:[<0x10000000007/main.o v3741>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.900 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.656+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004ca [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/util.o auth v3715 ap=1 s=49920 n(v0 rc2023-07-14T16:17:24.110269+0000 b49920 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc264100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004ca:[<0x10000000007/util.o v3715>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004ca:[<0x10000000007/util.o v3715>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.900 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.657+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c5 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rbt.o auth v3725 ap=1 s=55192 n(v0 rc2023-07-14T16:17:22.983963+0000 b55192 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc268580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c5:[<0x10000000007/rbt.o v3725>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c5:[<0x10000000007/rbt.o v3725>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.901 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.660+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004cc [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fc.o auth v3713 ap=1 s=28216 n(v0 rc2023-07-14T16:17:25.131747+0000 b28216 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc269b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004cc:[<0x10000000007/ffsb_fc.o v3713>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004cc:[<0x10000000007/ffsb_fc.o v3713>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.901 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.661+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c2 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/metaops.o auth v3731 ap=1 s=30464 n(v0 rc2023-07-14T16:17:22.491605+0000 b30464 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc26a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c2:[<0x10000000007/metaops.o v3731>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c2:[<0x10000000007/metaops.o v3731>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.902 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.661+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004c1 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/filelist.o auth v3733 ap=1 s=51344 n(v0 rc2023-07-14T16:17:22.333127+0000 b51344 1=1+0) (iversion lock) caps={25145=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc26b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000004c1:[<0x10000000007/filelist.o v3733>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v83>,<0x10000000003/client.0 v81>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v89>,<0x10000000001/sv_0 v106>,<0x10000000000/_nogroup v93>,<0x1/volumes v88>]//[]","memoryvalue":"(3)0x100000004c1:[<0x10000000007/filelist.o v3733>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.902 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.670+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000200 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_op.Po auth v362 ap=1 s=5980 n(v0 rc2023-07-14T16:17:23.833533+0000 b5980 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc271600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000200:[<0x100000003c5/ffsb_op.Po v299>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x20000000200:[<0x100000003c5/ffsb_op.Po v362>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.902 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.671+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f6 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/main.Po auth v363 ap=1 s=7474 n(v0 rc2023-07-14T16:17:21.907261+0000 b7474 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc271b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001f6:[<0x100000003c5/main.Po v189>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001f6:[<0x100000003c5/main.Po v363>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.902 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.671+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fa [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rwlock.Po auth v361 ap=1 s=4304 n(v0 rc2023-07-14T16:17:22.619464+0000 b4304 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc271080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001fa:[<0x100000003c5/rwlock.Po v233>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001fa:[<0x100000003c5/rwlock.Po v361>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.902 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.672+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000203 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fc.Po auth v364 ap=1 s=6027 n(v0 rc2023-07-14T16:17:25.146181+0000 b6027 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc272100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000203:[<0x100000003c5/ffsb_fc.Po v332>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x20000000203:[<0x100000003c5/ffsb_fc.Po v364>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.902 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.672+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000202 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/parser.Po auth v365 ap=1 s=6011 n(v0 rc2023-07-14T16:17:24.998327+0000 b6011 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc272680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000202:[<0x100000003c5/parser.Po v321>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x20000000202:[<0x100000003c5/parser.Po v365>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.902 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.672+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f8 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/filelist.Po auth v366 ap=1 s=8208 n(v0 rc2023-07-14T16:17:22.361578+0000 b8208 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc272c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001f8:[<0x100000003c5/filelist.Po v211>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001f8:[<0x100000003c5/filelist.Po v366>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.672+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000201 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/util.Po auth v368 ap=1 s=9703 n(v0 rc2023-07-14T16:17:24.601768+0000 b9703 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc273700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000201:[<0x100000003c5/util.Po v310>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x20000000201:[<0x100000003c5/util.Po v368>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.672+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000205 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/list.Po auth v369 ap=1 s=2713 n(v0 rc2023-07-14T16:17:25.419289+0000 b2713 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000205:[<0x100000003c5/list.Po v354>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x20000000205:[<0x100000003c5/list.Po v369>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.673+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000000 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fileops.Po auth v370 ap=1 s=7628 n(v0 rc2023-07-14T16:17:21.553309+0000 b7628 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000000:[<0x100000003c5/fileops.Po v167>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x20000000000:[<0x100000003c5/fileops.Po v370>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.673+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fd [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_tg.Po auth v371 ap=1 s=5936 n(v0 rc2023-07-14T16:17:23.211131+0000 b5936 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001fd:[<0x100000003c5/ffsb_tg.Po v266>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001fd:[<0x100000003c5/ffsb_tg.Po v371>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.673+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fb [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/cirlist.Po auth v367 ap=1 s=4909 n(v0 rc2023-07-14T16:17:22.751650+0000 b4909 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc273180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001fb:[<0x100000003c5/cirlist.Po v244>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001fb:[<0x100000003c5/cirlist.Po v367>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.674+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fe [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fs.Po auth v374 ap=1 s=7772 n(v0 rc2023-07-14T16:17:23.457114+0000 b7772 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001fe:[<0x100000003c5/ffsb_fs.Po v277>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001fe:[<0x100000003c5/ffsb_fs.Po v374>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.903 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.674+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001ff [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_thread.Po auth v372 ap=1 s=5941 n(v0 rc2023-07-14T16:17:23.614822+0000 b5941 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001ff:[<0x100000003c5/ffsb_thread.Po v288>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001ff:[<0x100000003c5/ffsb_thread.Po v372>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.904 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.674+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000001 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rand.Po auth v377 ap=1 s=7886 n(v0 rc2023-07-14T16:17:21.710429+0000 b7886 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc018c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000001:[<0x100000003c5/rand.Po v178>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x20000000001:[<0x100000003c5/rand.Po v377>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.904 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.675+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001fc [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rbt.Po auth v373 ap=1 s=4759 n(v0 rc2023-07-14T16:17:22.999809+0000 b4759 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001fc:[<0x100000003c5/rbt.Po v255>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001fc:[<0x100000003c5/rbt.Po v373>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.904 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.675+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f7 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fh.Po auth v376 ap=1 s=7473 n(v0 rc2023-07-14T16:17:22.104221+0000 b7473 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001f7:[<0x100000003c5/fh.Po v200>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001f7:[<0x100000003c5/fh.Po v376>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.904 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.675+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x200000001f9 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/metaops.Po auth v378 ap=1 s=7204 n(v0 rc2023-07-14T16:17:22.506529+0000 b7204 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000001f9:[<0x100000003c5/metaops.Po v222>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x200000001f9:[<0x100000003c5/metaops.Po v378>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.904 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:34:24.675+0000 7f4b9c9b7700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000204 [2,head] /volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_stats.Po auth v375 ap=1 s=6375 n(v0 rc2023-07-14T16:17:25.323343+0000 b6375 1=1+0) (ifile excl) (iversion lock) caps={25145=pAsLsXsFsxcrwb/-@1},l=25145(-1) | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55c1dc27c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000000204:[<0x100000003c5/ffsb_stats.Po v343>,<0x10000000007/.deps v3745>,<0x10000000006/ffsb v3292>,<0x10000000005/tmp v295>,<0x10000000003/client.0 v296>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v308>,<0x10000000001/sv_0 v322>,<0x10000000000/_nogroup v316>,<0x1/volumes v306>]//[]","memoryvalue":"(3)0x20000000204:[<0x100000003c5/ffsb_stats.Po v375>,<0x10000000007/.deps v3747>,<0x10000000006/ffsb v3531>,<0x10000000005/tmp v332>,<0x10000000003/client.0 v333>,<0x10000000002/41d87a1f-2ab4-4b30-af2a-b32c58d851e2 v346>,<0x10000000001/sv_0 v354>,<0x10000000000/_nogroup v62>,<0x1/volumes v108>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T16:34:24.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: pgmap v920: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:24.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:34:24.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000002bd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in~) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.946 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c3 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rwlock.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.946 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c4 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/cirlist.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.946 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004bb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.946 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c7 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fs.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.946 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004cf (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004d0 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/tmp) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004cb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/parser.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004b4 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c6 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_tg.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c0 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fh.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c9 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_op.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004b2 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/Makefile) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004bc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/stamp-h1) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004be (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rand.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004cd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_stats.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004bd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fileops.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c8 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_thread.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004ce (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/list.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004bf (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/main.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004ca (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/util.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.948 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rbt.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004cc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fc.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c2 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/metaops.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x100000004c1 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/filelist.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x20000000200 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_op.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001f6 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/main.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001fa (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rwlock.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x20000000203 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fc.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x20000000202 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/parser.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001f8 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/filelist.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x20000000201 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/util.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x20000000205 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/list.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x20000000000 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fileops.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001fd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_tg.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001fb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/cirlist.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001fe (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fs.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.951 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001ff (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_thread.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.951 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x20000000001 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rand.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.951 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001fc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rbt.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.951 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001f7 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fh.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.951 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x200000001f9 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/metaops.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:24.951 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:24 smithi062 ceph-mon[95543]: Scrub error on inode 0x20000000204 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_stats.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.058+0000 7f5ae53ff700 1 -- 172.21.15.62:0/119092348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0103670 msgr2=0x7f5ae0103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.058+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/119092348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0103670 0x7f5ae0103ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009990 tx=0x7f5ad8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.062+0000 7f5ae53ff700 1 -- 172.21.15.62:0/119092348 shutdown_connections 2023-07-14T16:34:25.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.062+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/119092348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0103670 0x7f5ae0103ae0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.062+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/119092348 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae00fe350 0x7f5ae0100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.062+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/119092348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae0100cb0 0x7f5ae0103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.062+0000 7f5ae53ff700 1 -- 172.21.15.62:0/119092348 >> 172.21.15.62:0/119092348 conn(0x7f5ae00fa000 msgr2=0x7f5ae00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.040 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.069+0000 7f5ae53ff700 1 -- 172.21.15.62:0/119092348 shutdown_connections 2023-07-14T16:34:25.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.069+0000 7f5ae53ff700 1 -- 172.21.15.62:0/119092348 wait complete. 2023-07-14T16:34:25.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.071+0000 7f5ae53ff700 1 Processor -- start 2023-07-14T16:34:25.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.071+0000 7f5ae53ff700 1 -- start start 2023-07-14T16:34:25.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.071+0000 7f5ae53ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae00fe350 0x7f5ae006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.071+0000 7f5ae53ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0100cb0 0x7f5ae006f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.071+0000 7f5ae53ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae0103670 0x7f5ae01abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.071+0000 7f5ae53ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0072400 con 0x7f5ae0103670 2023-07-14T16:34:25.044 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.072+0000 7f5ae53ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0072540 con 0x7f5ae00fe350 2023-07-14T16:34:25.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.072+0000 7f5ae53ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0104970 con 0x7f5ae0100cb0 2023-07-14T16:34:25.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.072+0000 7f5adeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae00fe350 0x7f5ae006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.072+0000 7f5adeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae00fe350 0x7f5ae006ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57938/0 (socket says 172.21.15.62:57938) 2023-07-14T16:34:25.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.072+0000 7f5adeffd700 1 -- 172.21.15.62:0/1009945201 learned_addr learned my addr 172.21.15.62:0/1009945201 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.072+0000 7f5adf7fe700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae0103670 0x7f5ae01abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adeffd700 1 -- 172.21.15.62:0/1009945201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0100cb0 msgr2=0x7f5ae006f320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:34:25.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adeffd700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0100cb0 0x7f5ae006f320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adeffd700 1 -- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae0103670 msgr2=0x7f5ae01abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adeffd700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae0103670 0x7f5ae01abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adeffd700 1 -- 172.21.15.62:0/1009945201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad80095f0 con 0x7f5ae00fe350 2023-07-14T16:34:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adeffd700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae00fe350 0x7f5ae006ede0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000ef10 tx=0x7f5ad000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adcff9700 1 -- 172.21.15.62:0/1009945201 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000ca70 con 0x7f5ae00fe350 2023-07-14T16:34:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5ae53ff700 1 -- 172.21.15.62:0/1009945201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae0104bf0 con 0x7f5ae00fe350 2023-07-14T16:34:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adcff9700 1 -- 172.21.15.62:0/1009945201 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ad000f040 con 0x7f5ae00fe350 2023-07-14T16:34:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.073+0000 7f5adcff9700 1 -- 172.21.15.62:0/1009945201 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0014480 con 0x7f5ae00fe350 2023-07-14T16:34:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.074+0000 7f5ae53ff700 1 -- 172.21.15.62:0/1009945201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae0105150 con 0x7f5ae00fe350 2023-07-14T16:34:25.053 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.081+0000 7f5af8d29b80 1 Processor -- start 2023-07-14T16:34:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.081+0000 7f5af8d29b80 1 -- start start 2023-07-14T16:34:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aec4d0 0x55b579aec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aece20 0x55b579aed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aedc70 0x55b579aee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5adcff9700 1 -- 172.21.15.62:0/1009945201 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5ad0007500 con 0x7f5ae00fe350 2023-07-14T16:34:25.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5799f1cb0 con 0x55b579aece20 2023-07-14T16:34:25.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b579a44a40 con 0x55b579aec4d0 2023-07-14T16:34:25.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b579a3ded0 con 0x55b579aedc70 2023-07-14T16:34:25.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5adf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aec4d0 0x55b579aec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5adf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aec4d0 0x55b579aec8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57950/0 (socket says 172.21.15.62:57950) 2023-07-14T16:34:25.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.082+0000 7f5adf7fe700 1 -- 172.21.15.62:0/2331409260 learned_addr learned my addr 172.21.15.62:0/2331409260 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.083+0000 7f5adeffd700 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aece20 0x55b579aed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.083+0000 7f5ad67fc700 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aedc70 0x55b579aee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.083+0000 7f5ad67fc700 1 -- 172.21.15.62:0/2331409260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aec4d0 msgr2=0x55b579aec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.083+0000 7f5ad67fc700 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aec4d0 0x55b579aec8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.083+0000 7f5ad67fc700 1 -- 172.21.15.62:0/2331409260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aece20 msgr2=0x55b579aed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.083+0000 7f5ad67fc700 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aece20 0x55b579aed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.083+0000 7f5ad67fc700 1 -- 172.21.15.62:0/2331409260 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57986bde0 con 0x55b579aedc70 2023-07-14T16:34:25.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.084+0000 7f5ad67fc700 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aedc70 0x55b579aee0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009b60 tx=0x7f5ac8009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35cac4adcf429ec6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.085+0000 7f5ad57fa700 1 -- 172.21.15.62:0/2331409260 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8004230 con 0x55b579aedc70 2023-07-14T16:34:25.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.085+0000 7f5ad57fa700 1 -- 172.21.15.62:0/2331409260 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ac8004cc0 con 0x55b579aedc70 2023-07-14T16:34:25.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.085+0000 7f5ad57fa700 1 -- 172.21.15.62:0/2331409260 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac800eec0 con 0x55b579aedc70 2023-07-14T16:34:25.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.085+0000 7f5adcff9700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5acc0a12a0 0x7f5acc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.085+0000 7f5adcff9700 1 -- 172.21.15.62:0/1009945201 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5ad00ccbb0 con 0x7f5ae00fe350 2023-07-14T16:34:25.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.086+0000 7f5af8d29b80 1 -- 172.21.15.62:0/2331409260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aedc70 msgr2=0x55b579aee0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.086+0000 7f5adf7fe700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5acc0a12a0 0x7f5acc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.086+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aedc70 0x55b579aee0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009b60 tx=0x7f5ac8009e70 comp rx=0 tx=0).stop 2023-07-14T16:34:25.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.086+0000 7f5af8d29b80 1 -- 172.21.15.62:0/2331409260 shutdown_connections 2023-07-14T16:34:25.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.086+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aedc70 0x55b579aee0e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.086+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aec4d0 0x55b579aec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.086+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/2331409260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aece20 0x55b579aed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.086+0000 7f5af8d29b80 1 -- 172.21.15.62:0/2331409260 >> 172.21.15.62:0/2331409260 conn(0x55b579ae1da0 msgr2=0x55b579ae71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.087+0000 7f5adf7fe700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5acc0a12a0 0x7f5acc0a3750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009990 tx=0x7f5ad80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.087+0000 7f5af8d29b80 1 -- 172.21.15.62:0/2331409260 shutdown_connections 2023-07-14T16:34:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.087+0000 7f5af8d29b80 1 -- 172.21.15.62:0/2331409260 wait complete. 2023-07-14T16:34:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.088+0000 7f5af8d29b80 1 -- start start 2023-07-14T16:34:25.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.088+0000 7f5af8d29b80 1 Processor -- start 2023-07-14T16:34:25.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.089+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.089+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aece20 0x55b579af66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.089+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aedc70 0x55b579afd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.089+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5799ddac0 con 0x55b579aedc70 2023-07-14T16:34:25.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.089+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5799dd7c0 con 0x55b579aece20 2023-07-14T16:34:25.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.089+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5798c1920 con 0x55b579aec4d0 2023-07-14T16:34:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.089+0000 7f5af8d29b80 10 client.0 _ll_register_callbacks cb 0x55b5799db760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.090+0000 7f5ad67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aece20 0x55b579af66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.090+0000 7f5adeffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.090+0000 7f5ad67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aece20 0x55b579af66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:57966/0 (socket says 172.21.15.62:57966) 2023-07-14T16:34:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.090+0000 7f5ad67fc700 1 -- 172.21.15.62:0/3349158270 learned_addr learned my addr 172.21.15.62:0/3349158270 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.091+0000 7f5adf7fe700 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aedc70 0x55b579afd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.091+0000 7f5ad67fc700 1 -- 172.21.15.62:0/3349158270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 msgr2=0x55b579af6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.091+0000 7f5ad67fc700 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.091+0000 7f5ad67fc700 1 -- 172.21.15.62:0/3349158270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aedc70 msgr2=0x55b579afd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.091+0000 7f5ad67fc700 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aedc70 0x55b579afd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.091+0000 7f5ad67fc700 1 -- 172.21.15.62:0/3349158270 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac8009810 con 0x55b579aece20 2023-07-14T16:34:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.092+0000 7f5ad67fc700 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aece20 0x55b579af66d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800e880 tx=0x7f5ac8004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f28c9bbe106ec8d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.092+0000 7f5ac67fc700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.092+0000 7f5ac67fc700 1 -- 172.21.15.62:0/3349158270 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8004e30 con 0x55b579aece20 2023-07-14T16:34:25.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.092+0000 7f5ac67fc700 1 -- 172.21.15.62:0/3349158270 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5ac8014070 con 0x55b579aece20 2023-07-14T16:34:25.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.092+0000 7f5af8d29b80 1 -- 172.21.15.62:0/3349158270 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5799974f0 con 0x55b579aece20 2023-07-14T16:34:25.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.094+0000 7f5ac67fc700 1 -- 172.21.15.62:0/3349158270 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ac8005230 con 0x55b579aece20 2023-07-14T16:34:25.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.094+0000 7f5af8d29b80 10 client.30179 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:25.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.094+0000 7f5af8d29b80 1 -- 172.21.15.62:0/3349158270 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b579854c60 con 0x55b579aece20 2023-07-14T16:34:25.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5ac67fc700 1 -- 172.21.15.62:0/3349158270 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b5799974f0 con 0x55b579aece20 2023-07-14T16:34:25.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5ac67fc700 1 -- 172.21.15.62:0/3349158270 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5ac8018bb0 con 0x55b579aece20 2023-07-14T16:34:25.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5af8d29b80 10 client.30179 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:25.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5af8d29b80 10 client.30179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:25.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5af8d29b80 20 client.30179 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b579b01d60 0x55b579b04170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5af8d29b80 4 client.30179 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5af8d29b80 1 -- 172.21.15.62:0/3349158270 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b579a36eb0 con 0x55b579b01d60 2023-07-14T16:34:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.095+0000 7f5adeffd700 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b579b01d60 0x55b579b04170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.097+0000 7f5adeffd700 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b579b01d60 0x55b579b04170 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9cd5e23aca3308 server_cookie=91d227c0b11b6d8 in_seq=0 out_seq=0 2023-07-14T16:34:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.097+0000 7f5ac67fc700 10 client.30179 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.099+0000 7f5ac67fc700 1 -- 172.21.15.62:0/3349158270 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b579a36eb0 con 0x55b579b01d60 2023-07-14T16:34:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.099+0000 7f5ac67fc700 10 client.30179 handle_command_reply: tid=0 2023-07-14T16:34:25.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.099+0000 7f5af8d29b80 1 client.30179 shutdown 2023-07-14T16:34:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.100+0000 7f5af8d29b80 1 -- 172.21.15.62:0/3349158270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aece20 msgr2=0x55b579af66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.100+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aece20 0x55b579af66d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800e880 tx=0x7f5ac8004980 comp rx=0 tx=0).stop 2023-07-14T16:34:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.101+0000 7f5af8d29b80 1 -- 172.21.15.62:0/3349158270 shutdown_connections 2023-07-14T16:34:25.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.101+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579aedc70 0x55b579afd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.101+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.101+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b579b01d60 0x55b579b04170 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.101+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/3349158270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aece20 0x55b579af66d0 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800e880 tx=0x7f5ac8004980 comp rx=0 tx=0).stop 2023-07-14T16:34:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.101+0000 7f5af8d29b80 1 -- 172.21.15.62:0/3349158270 >> 172.21.15.62:0/3349158270 conn(0x55b579ae1da0 msgr2=0x55b579ae68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.101+0000 7f5ac67fc700 0 client.30179 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.102+0000 7f5af8d29b80 1 -- 172.21.15.62:0/3349158270 shutdown_connections 2023-07-14T16:34:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.102+0000 7f5af8d29b80 1 -- 172.21.15.62:0/3349158270 wait complete. 2023-07-14T16:34:25.076 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.102+0000 7f5af8d29b80 20 client.30179 trim_cache size 0 max 16384 2023-07-14T16:34:25.105 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.133+0000 7f5af8d29b80 1 Processor -- start 2023-07-14T16:34:25.106 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.133+0000 7f5af8d29b80 1 -- start start 2023-07-14T16:34:25.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.134+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.134+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 0x55b579af10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.107 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.134+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.134+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b579af5d00 con 0x55b579afd6d0 2023-07-14T16:34:25.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.134+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b579aed1f0 con 0x55b579aedc70 2023-07-14T16:34:25.109 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.134+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b579ae80a0 con 0x55b579aec4d0 2023-07-14T16:34:25.112 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.135+0000 7f5adf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.113 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.135+0000 7f5adf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49446/0 (socket says 172.21.15.62:49446) 2023-07-14T16:34:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.135+0000 7f5adf7fe700 1 -- 172.21.15.62:0/278006508 learned_addr learned my addr 172.21.15.62:0/278006508 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.136+0000 7f5adeffd700 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.137+0000 7f5ad67fc700 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 0x55b579af10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.137+0000 7f5adeffd700 1 -- 172.21.15.62:0/278006508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 msgr2=0x55b579af10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.137+0000 7f5adeffd700 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 0x55b579af10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.137+0000 7f5adeffd700 1 -- 172.21.15.62:0/278006508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 msgr2=0x55b579af5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.137+0000 7f5adeffd700 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.137+0000 7f5adeffd700 1 -- 172.21.15.62:0/278006508 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b579af5110 con 0x55b579aec4d0 2023-07-14T16:34:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.138+0000 7f5adeffd700 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af0b70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000eee0 tx=0x7f5ad00ca530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923eeab3784c13f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.138+0000 7f5ad5ffb700 1 -- 172.21.15.62:0/278006508 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00998d0 con 0x55b579aec4d0 2023-07-14T16:34:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.138+0000 7f5ad5ffb700 1 -- 172.21.15.62:0/278006508 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ad00112c0 con 0x55b579aec4d0 2023-07-14T16:34:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.138+0000 7f5ad5ffb700 1 -- 172.21.15.62:0/278006508 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000cd60 con 0x55b579aec4d0 2023-07-14T16:34:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.139+0000 7f5af8d29b80 1 -- 172.21.15.62:0/278006508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 msgr2=0x55b579af0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.139+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af0b70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000eee0 tx=0x7f5ad00ca530 comp rx=0 tx=0).stop 2023-07-14T16:34:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.140+0000 7f5af8d29b80 1 -- 172.21.15.62:0/278006508 shutdown_connections 2023-07-14T16:34:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.140+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.140+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579af0b70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.140+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/278006508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 0x55b579af10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.140+0000 7f5af8d29b80 1 -- 172.21.15.62:0/278006508 >> 172.21.15.62:0/278006508 conn(0x55b579ae1da0 msgr2=0x55b579b03d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.140+0000 7f5af8d29b80 1 -- 172.21.15.62:0/278006508 shutdown_connections 2023-07-14T16:34:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.140+0000 7f5af8d29b80 1 -- 172.21.15.62:0/278006508 wait complete. 2023-07-14T16:34:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.141+0000 7f5af8d29b80 1 -- start start 2023-07-14T16:34:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.141+0000 7f5af8d29b80 1 Processor -- start 2023-07-14T16:34:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579afcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 0x55b579af9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5af8d29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b579afa5d0 con 0x55b579afd6d0 2023-07-14T16:34:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b579b55c30 con 0x55b579aedc70 2023-07-14T16:34:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5af8d29b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b579b55d70 con 0x55b579aec4d0 2023-07-14T16:34:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5af8d29b80 10 client.0 _ll_register_callbacks cb 0x55b5799db760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5adf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5adf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49452/0 (socket says 172.21.15.62:49452) 2023-07-14T16:34:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.142+0000 7f5adf7fe700 1 -- 172.21.15.62:0/1060004077 learned_addr learned my addr 172.21.15.62:0/1060004077 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.143+0000 7f5adf7fe700 1 -- 172.21.15.62:0/1060004077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 msgr2=0x55b579afcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.143+0000 7f5adf7fe700 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579afcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.143+0000 7f5adf7fe700 1 -- 172.21.15.62:0/1060004077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 msgr2=0x55b579af9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.143+0000 7f5adf7fe700 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 0x55b579af9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.143+0000 7f5adf7fe700 1 -- 172.21.15.62:0/1060004077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae0105150 con 0x55b579afd6d0 2023-07-14T16:34:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.144+0000 7f5adf7fe700 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af9f80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005390 tx=0x7f5ad8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c412267ffa4dfa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.144+0000 7f5ac67fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:34:25.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.144+0000 7f5ac67fc700 1 -- 172.21.15.62:0/1060004077 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800bde0 con 0x55b579afd6d0 2023-07-14T16:34:25.120 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.144+0000 7f5ac67fc700 1 -- 172.21.15.62:0/1060004077 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5ad8004500 con 0x55b579afd6d0 2023-07-14T16:34:25.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.151+0000 7f5af8d29b80 1 -- 172.21.15.62:0/1060004077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ac8004cc0 con 0x55b579afd6d0 2023-07-14T16:34:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.152+0000 7f5ac67fc700 1 -- 172.21.15.62:0/1060004077 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad80046c0 con 0x55b579afd6d0 2023-07-14T16:34:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.152+0000 7f5af8d29b80 10 client.20634 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.152+0000 7f5af8d29b80 1 -- 172.21.15.62:0/1060004077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b579ae35e0 con 0x55b579afd6d0 2023-07-14T16:34:25.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.154+0000 7f5ac67fc700 1 -- 172.21.15.62:0/1060004077 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ac8004cc0 con 0x55b579afd6d0 2023-07-14T16:34:25.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.154+0000 7f5ac67fc700 1 -- 172.21.15.62:0/1060004077 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5ad803c070 con 0x55b579afd6d0 2023-07-14T16:34:25.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.154+0000 7f5af8d29b80 10 client.20634 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:25.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.154+0000 7f5af8d29b80 10 client.20634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:25.126 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.154+0000 7f5af8d29b80 20 client.20634 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:25.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.154+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b579b577e0 0x55b579b59bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.154+0000 7f5af8d29b80 4 client.20634 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:34:25.127 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.154+0000 7f5af8d29b80 1 -- 172.21.15.62:0/1060004077 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b5799f9050 con 0x55b579b577e0 2023-07-14T16:34:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.156+0000 7f5adeffd700 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b579b577e0 0x55b579b59bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.158+0000 7f5adeffd700 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b579b577e0 0x55b579b59bf0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d459f4926a2b1428 server_cookie=2ac04f5ac432eecf in_seq=0 out_seq=0 2023-07-14T16:34:25.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.158+0000 7f5ac67fc700 10 client.20634 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:25.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.159+0000 7f5ac67fc700 1 -- 172.21.15.62:0/1060004077 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b5799f9050 con 0x55b579b577e0 2023-07-14T16:34:25.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.159+0000 7f5ac67fc700 10 client.20634 handle_command_reply: tid=0 2023-07-14T16:34:25.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.161+0000 7f5af8d29b80 1 client.20634 shutdown 2023-07-14T16:34:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.161+0000 7f5af8d29b80 1 -- 172.21.15.62:0/1060004077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 msgr2=0x55b579af9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.161+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af9f80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005390 tx=0x7f5ad8009310 comp rx=0 tx=0).stop 2023-07-14T16:34:25.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.162+0000 7f5af8d29b80 1 -- 172.21.15.62:0/1060004077 shutdown_connections 2023-07-14T16:34:25.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.162+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b579afd6d0 0x55b579af9f80 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.162+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b579aec4d0 0x55b579afcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.162+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b579b577e0 0x55b579b59bf0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.162+0000 7f5af8d29b80 1 --2- 172.21.15.62:0/1060004077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b579aedc70 0x55b579af9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.162+0000 7f5af8d29b80 1 -- 172.21.15.62:0/1060004077 >> 172.21.15.62:0/1060004077 conn(0x55b579ae1da0 msgr2=0x55b579af0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.163+0000 7f5ac67fc700 0 client.20634 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:25.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.163+0000 7f5af8d29b80 1 -- 172.21.15.62:0/1060004077 shutdown_connections 2023-07-14T16:34:25.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.163+0000 7f5af8d29b80 1 -- 172.21.15.62:0/1060004077 wait complete. 2023-07-14T16:34:25.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.163+0000 7f5af8d29b80 20 client.20634 trim_cache size 0 max 16384 2023-07-14T16:34:25.136 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:34:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T16:34:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2023-07-14T16:34:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "34613b6b-2bfd-4e57-a479-708b9df79e2c": { 2023-07-14T16:34:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2023-07-14T16:34:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "34613b6b-2bfd-4e57-a479-708b9df79e2c", 2023-07-14T16:34:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2023-07-14T16:34:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:34:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:34:25.138 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:34:25.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 -- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5acc0a12a0 msgr2=0x7f5acc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5acc0a12a0 0x7f5acc0a3750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009990 tx=0x7f5ad80058e0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 -- 172.21.15.62:0/1009945201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae00fe350 msgr2=0x7f5ae006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae00fe350 0x7f5ae006ede0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000ef10 tx=0x7f5ad000c3a0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 -- 172.21.15.62:0/1009945201 shutdown_connections 2023-07-14T16:34:25.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae0103670 0x7f5ae01abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5acc0a12a0 0x7f5acc0a3750 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ae00fe350 0x7f5ae006ede0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 --2- 172.21.15.62:0/1009945201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ae0100cb0 0x7f5ae006f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.164+0000 7f5ae53ff700 1 -- 172.21.15.62:0/1009945201 >> 172.21.15.62:0/1009945201 conn(0x7f5ae00fa000 msgr2=0x7f5ae00feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.139 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.165+0000 7f5ae53ff700 1 -- 172.21.15.62:0/1009945201 shutdown_connections 2023-07-14T16:34:25.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:25.169+0000 7f5ae53ff700 1 -- 172.21.15.62:0/1009945201 wait complete. 2023-07-14T16:34:25.168 INFO:tasks.cephfs.filesystem:scrub status for tag:34613b6b-2bfd-4e57-a479-708b9df79e2c - {'path': '/', 'tag': '34613b6b-2bfd-4e57-a479-708b9df79e2c', 'options': 'recursive,force'} 2023-07-14T16:34:25.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: pgmap v920: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:25.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:34:25.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000002bd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in~) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c3 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rwlock.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c4 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/cirlist.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004bb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c7 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fs.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004cf (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004d0 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/tmp) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004cb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/parser.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004b4 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c6 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_tg.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c0 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fh.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c9 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_op.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004b2 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/Makefile) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004bc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/stamp-h1) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004be (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rand.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004cd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_stats.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004bd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fileops.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c8 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_thread.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004ce (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/list.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004bf (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/main.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004ca (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/util.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rbt.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.273 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004cc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fc.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c2 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/metaops.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x100000004c1 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/filelist.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x20000000200 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_op.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001f6 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/main.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001fa (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rwlock.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x20000000203 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fc.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.274 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x20000000202 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/parser.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001f8 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/filelist.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x20000000201 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/util.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x20000000205 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/list.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x20000000000 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fileops.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001fd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_tg.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001fb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/cirlist.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.275 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001fe (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fs.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001ff (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_thread.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x20000000001 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rand.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001fc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rbt.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001f7 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fh.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x200000001f9 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/metaops.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.276 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:24 smithi164 ceph-mon[97427]: Scrub error on inode 0x20000000204 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_stats.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: pgmap v920: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 4.88k op/s 2023-07-14T16:34:25.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:34:25.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000002bd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in~) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c3 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rwlock.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c4 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/cirlist.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004bb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c7 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fs.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004cf (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.398 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004d0 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/tmp) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.399 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004cb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/parser.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.399 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004b4 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.399 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c6 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_tg.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.399 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c0 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fh.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c9 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_op.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004b2 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/Makefile) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004bc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/stamp-h1) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004be (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rand.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004cd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_stats.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.400 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004bd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fileops.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c8 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_thread.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004ce (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/list.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004bf (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/main.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004ca (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/util.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c5 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rbt.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004cc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fc.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c2 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/metaops.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x100000004c1 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/filelist.o) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x20000000200 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_op.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001f6 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/main.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001fa (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rwlock.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x20000000203 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fc.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x20000000202 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/parser.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001f8 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/filelist.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.402 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x20000000201 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/util.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x20000000205 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/list.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x20000000000 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fileops.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001fd (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_tg.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001fb (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/cirlist.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001fe (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fs.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001ff (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_thread.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x20000000001 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rand.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.403 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001fc (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rbt.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001f7 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fh.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x200000001f9 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/metaops.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.404 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:24 smithi081 ceph-mon[127795]: Scrub error on inode 0x20000000204 (/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_stats.Po) see mds.e log and `damage ls` output for details 2023-07-14T16:34:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.770+0000 7f1c019ff700 1 -- 172.21.15.62:0/3061678042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc0711a0 msgr2=0x7f1bfc0715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.770+0000 7f1c019ff700 1 --2- 172.21.15.62:0/3061678042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc0711a0 0x7f1bfc0715b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0098f0 tx=0x7f1bec009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.770+0000 7f1c019ff700 1 -- 172.21.15.62:0/3061678042 shutdown_connections 2023-07-14T16:34:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.770+0000 7f1c019ff700 1 --2- 172.21.15.62:0/3061678042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bfc077240 0x7f1bfc0796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.770+0000 7f1c019ff700 1 --2- 172.21.15.62:0/3061678042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc0711a0 0x7f1bfc0715b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.770+0000 7f1c019ff700 1 --2- 172.21.15.62:0/3061678042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bfc074880 0x7f1bfc076d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.770+0000 7f1c019ff700 1 -- 172.21.15.62:0/3061678042 >> 172.21.15.62:0/3061678042 conn(0x7f1bfc06ca50 msgr2=0x7f1bfc06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.771+0000 7f1c019ff700 1 -- 172.21.15.62:0/3061678042 shutdown_connections 2023-07-14T16:34:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.771+0000 7f1c019ff700 1 -- 172.21.15.62:0/3061678042 wait complete. 2023-07-14T16:34:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.772+0000 7f1c019ff700 1 Processor -- start 2023-07-14T16:34:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.773+0000 7f1c019ff700 1 -- start start 2023-07-14T16:34:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.773+0000 7f1c019ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bfc0711a0 0x7f1bfc082410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.773+0000 7f1c019ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bfc074880 0x7f1bfc082950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1c019ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc077240 0x7f1bfc1b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1c019ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc1b7190 con 0x7f1bfc0711a0 2023-07-14T16:34:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1c019ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc1b7300 con 0x7f1bfc077240 2023-07-14T16:34:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1c019ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc1b74c0 con 0x7f1bfc074880 2023-07-14T16:34:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bfc0711a0 0x7f1bfc082410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bfc0711a0 0x7f1bfc082410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49482/0 (socket says 172.21.15.62:49482) 2023-07-14T16:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfaffd700 1 -- 172.21.15.62:0/2348555159 learned_addr learned my addr 172.21.15.62:0/2348555159 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bfc074880 0x7f1bfc082950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfa7fc700 1 -- 172.21.15.62:0/2348555159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc077240 msgr2=0x7f1bfc1b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc077240 0x7f1bfc1b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc077240 0x7f1bfc1b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfa7fc700 1 -- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bfc0711a0 msgr2=0x7f1bfc082410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bfc0711a0 0x7f1bfc082410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfa7fc700 1 -- 172.21.15.62:0/2348555159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec009520 con 0x7f1bfc074880 2023-07-14T16:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.774+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc077240 0x7f1bfc1b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.775+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bfc0711a0 0x7f1bfc082410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.775+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bfc074880 0x7f1bfc082950 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000b450 tx=0x7f1bf000b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.775+0000 7f1c009fd700 1 -- 172.21.15.62:0/2348555159 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0011830 con 0x7f1bfc074880 2023-07-14T16:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.775+0000 7f1c019ff700 1 -- 172.21.15.62:0/2348555159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc1b77a0 con 0x7f1bfc074880 2023-07-14T16:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.775+0000 7f1c009fd700 1 -- 172.21.15.62:0/2348555159 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bf0011e20 con 0x7f1bfc074880 2023-07-14T16:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.775+0000 7f1c009fd700 1 -- 172.21.15.62:0/2348555159 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0010d10 con 0x7f1bfc074880 2023-07-14T16:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.775+0000 7f1c019ff700 1 -- 172.21.15.62:0/2348555159 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc1bdf80 con 0x7f1bfc074880 2023-07-14T16:34:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.781+0000 7f1c009fd700 1 -- 172.21.15.62:0/2348555159 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1bf0011990 con 0x7f1bfc074880 2023-07-14T16:34:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.783+0000 7f1c153bfb80 1 Processor -- start 2023-07-14T16:34:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.783+0000 7f1c153bfb80 1 -- start start 2023-07-14T16:34:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a8e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8e840 0x55d5a3a8f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a3993c50 con 0x55d5a3a8e840 2023-07-14T16:34:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a39e69e0 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a39dfe70 con 0x55d5a38ea6b0 2023-07-14T16:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1c009fd700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1be40a1220 0x7f1be40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1bfa7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1bfa7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a8fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:58004/0 (socket says 172.21.15.62:58004) 2023-07-14T16:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.784+0000 7f1bfa7fc700 1 -- 172.21.15.62:0/3112037858 learned_addr learned my addr 172.21.15.62:0/3112037858 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.785+0000 7f1c009fd700 1 -- 172.21.15.62:0/2348555159 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1bf00cc300 con 0x7f1bfc074880 2023-07-14T16:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.785+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8e840 0x55d5a3a8f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.785+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a8e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.785+0000 7f1bfa7fc700 1 -- 172.21.15.62:0/3112037858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a38ea6b0 msgr2=0x55d5a3a8e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.785+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a8e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.785+0000 7f1bfa7fc700 1 -- 172.21.15.62:0/3112037858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8e840 msgr2=0x55d5a3a8f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.785+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8e840 0x55d5a3a8f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.785+0000 7f1bfa7fc700 1 -- 172.21.15.62:0/3112037858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5a380dd80 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.786+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a8e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.786+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a8fb80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0004890 tx=0x7f1bf0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89ff0b50e6861182 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.786+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1be40a1220 0x7f1be40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.786+0000 7f1be1ffb700 1 -- 172.21.15.62:0/3112037858 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0099200 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.786+0000 7f1be1ffb700 1 -- 172.21.15.62:0/3112037858 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bf0099020 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.786+0000 7f1be1ffb700 1 -- 172.21.15.62:0/3112037858 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf009d6f0 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.787+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3112037858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 msgr2=0x55d5a3a8fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.787+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a8fb80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0004890 tx=0x7f1bf0004750 comp rx=0 tx=0).stop 2023-07-14T16:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.788+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1be40a1220 0x7f1be40a36d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009750 tx=0x7f1bf400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.790+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3112037858 shutdown_connections 2023-07-14T16:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.790+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a8fb80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.790+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a8e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.790+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3112037858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8e840 0x55d5a3a8f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.790+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3112037858 >> 172.21.15.62:0/3112037858 conn(0x55d5a3a83be0 msgr2=0x55d5a3a88cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.791+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3112037858 shutdown_connections 2023-07-14T16:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.791+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3112037858 wait complete. 2023-07-14T16:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.792+0000 7f1c153bfb80 1 -- start start 2023-07-14T16:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.792+0000 7f1c153bfb80 1 Processor -- start 2023-07-14T16:34:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.793+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.793+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 0x55d5a3a97e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.793+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.793+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a397fa60 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.793+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a397f760 con 0x55d5a38ea6b0 2023-07-14T16:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.793+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a38638c0 con 0x55d5a3a8e840 2023-07-14T16:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.793+0000 7f1c153bfb80 10 client.0 _ll_register_callbacks cb 0x55d5a397d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.794+0000 7f1bfaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.794+0000 7f1bfaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:58016/0 (socket says 172.21.15.62:58016) 2023-07-14T16:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.794+0000 7f1bfaffd700 1 -- 172.21.15.62:0/2384598267 learned_addr learned my addr 172.21.15.62:0/2384598267 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.794+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.794+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 0x55d5a3a97e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.795+0000 7f1bfaffd700 1 -- 172.21.15.62:0/2384598267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 msgr2=0x55d5a3a97e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.795+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 0x55d5a3a97e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.795+0000 7f1bfaffd700 1 -- 172.21.15.62:0/2384598267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8f6e0 msgr2=0x55d5a3a9ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.795+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.795+0000 7f1bfaffd700 1 -- 172.21.15.62:0/2384598267 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc1bdf80 con 0x55d5a38ea6b0 2023-07-14T16:34:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.795+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:34:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.795+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a978e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00b980 tx=0x7f1bec00ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c4f56cfe8870c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.795+0000 7f1bdaffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.798+0000 7f1bdaffd700 1 -- 172.21.15.62:0/2384598267 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec01c040 con 0x55d5a38ea6b0 2023-07-14T16:34:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.798+0000 7f1bdaffd700 1 -- 172.21.15.62:0/2384598267 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1bec005450 con 0x55d5a38ea6b0 2023-07-14T16:34:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.798+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2384598267 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5a3939490 con 0x55d5a38ea6b0 2023-07-14T16:34:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.798+0000 7f1bdaffd700 1 -- 172.21.15.62:0/2384598267 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bec005610 con 0x55d5a38ea6b0 2023-07-14T16:34:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.798+0000 7f1c153bfb80 10 client.30191 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.798+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2384598267 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d5a37f6c00 con 0x55d5a38ea6b0 2023-07-14T16:34:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.799+0000 7f1bdaffd700 1 -- 172.21.15.62:0/2384598267 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d5a3939490 con 0x55d5a38ea6b0 2023-07-14T16:34:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.799+0000 7f1bdaffd700 1 -- 172.21.15.62:0/2384598267 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1bec015020 con 0x55d5a38ea6b0 2023-07-14T16:34:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.799+0000 7f1c153bfb80 10 client.30191 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.799+0000 7f1c153bfb80 10 client.30191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.799+0000 7f1c153bfb80 20 client.30191 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.799+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d5a3aa3780 0x55d5a3aa5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.799+0000 7f1c153bfb80 4 client.30191 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.799+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2384598267 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5a39d8e50 con 0x55d5a3aa3780 2023-07-14T16:34:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.800+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d5a3aa3780 0x55d5a3aa5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.801+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d5a3aa3780 0x55d5a3aa5b90 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a2e1b64e96d19 server_cookie=9d0ad2bbeef98af9 in_seq=0 out_seq=0 2023-07-14T16:34:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.801+0000 7f1bdaffd700 10 client.30191 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.802+0000 7f1bdaffd700 1 -- 172.21.15.62:0/2384598267 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d5a39d8e50 con 0x55d5a3aa3780 2023-07-14T16:34:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.802+0000 7f1bdaffd700 10 client.30191 handle_command_reply: tid=0 2023-07-14T16:34:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.802+0000 7f1c153bfb80 1 client.30191 shutdown 2023-07-14T16:34:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.804+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2384598267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a38ea6b0 msgr2=0x55d5a3a978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.804+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a978e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00b980 tx=0x7f1bec00ba60 comp rx=0 tx=0).stop 2023-07-14T16:34:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.804+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2384598267 shutdown_connections 2023-07-14T16:34:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.804+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.804+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d5a3aa3780 0x55d5a3aa5b90 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.804+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a978e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.804+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2384598267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 0x55d5a3a97e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.804+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2384598267 >> 172.21.15.62:0/2384598267 conn(0x55d5a3a83be0 msgr2=0x55d5a3a884e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.805+0000 7f1bdaffd700 0 client.30191 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.805+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2384598267 shutdown_connections 2023-07-14T16:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.805+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2384598267 wait complete. 2023-07-14T16:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.805+0000 7f1c153bfb80 20 client.30191 trim_cache size 0 max 16384 2023-07-14T16:34:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.833+0000 7f1c153bfb80 1 Processor -- start 2023-07-14T16:34:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.833+0000 7f1c153bfb80 1 -- start start 2023-07-14T16:34:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a92550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8e840 0x55d5a3a92a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a3a96d90 con 0x55d5a38ea6b0 2023-07-14T16:34:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a3a98910 con 0x55d5a3a8e840 2023-07-14T16:34:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a3a85d30 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1bfa7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8e840 0x55d5a3a92a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1bfa7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8e840 0x55d5a3a92a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:58028/0 (socket says 172.21.15.62:58028) 2023-07-14T16:34:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.834+0000 7f1bfa7fc700 1 -- 172.21.15.62:0/2830524393 learned_addr learned my addr 172.21.15.62:0/2830524393 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.835+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a92550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.835+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.835+0000 7f1bfaffd700 1 -- 172.21.15.62:0/2830524393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8f6e0 msgr2=0x55d5a3a9ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.835+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.835+0000 7f1bfaffd700 1 -- 172.21.15.62:0/2830524393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8e840 msgr2=0x55d5a3a92a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.835+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8e840 0x55d5a3a92a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.835+0000 7f1bfaffd700 1 -- 172.21.15.62:0/2830524393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5a38341f0 con 0x55d5a38ea6b0 2023-07-14T16:34:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.836+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a92550 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f1bec022550 tx=0x7f1bec009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440bfe3e4e928451 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.837+0000 7f1be27fc700 1 -- 172.21.15.62:0/2830524393 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec01c070 con 0x55d5a38ea6b0 2023-07-14T16:34:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.837+0000 7f1be27fc700 1 -- 172.21.15.62:0/2830524393 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bec022c00 con 0x55d5a38ea6b0 2023-07-14T16:34:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.837+0000 7f1be27fc700 1 -- 172.21.15.62:0/2830524393 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec00e3e0 con 0x55d5a38ea6b0 2023-07-14T16:34:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.838+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2830524393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 msgr2=0x55d5a3a92550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.838+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a92550 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f1bec022550 tx=0x7f1bec009420 comp rx=0 tx=0).stop 2023-07-14T16:34:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.838+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2830524393 shutdown_connections 2023-07-14T16:34:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.838+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.838+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a92550 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.838+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/2830524393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8e840 0x55d5a3a92a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.838+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2830524393 >> 172.21.15.62:0/2830524393 conn(0x55d5a3a83be0 msgr2=0x55d5a3a95a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.842+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2830524393 shutdown_connections 2023-07-14T16:34:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.842+0000 7f1c153bfb80 1 -- 172.21.15.62:0/2830524393 wait complete. 2023-07-14T16:34:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.842+0000 7f1c153bfb80 1 -- start start 2023-07-14T16:34:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.842+0000 7f1c153bfb80 1 Processor -- start 2023-07-14T16:34:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.843+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.843+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 0x55d5a3a9d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.843+0000 7f1c153bfb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.843+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a3a9e090 con 0x55d5a38ea6b0 2023-07-14T16:34:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.843+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a3a9e200 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.843+0000 7f1c153bfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a3a9e3c0 con 0x55d5a3a8e840 2023-07-14T16:34:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.843+0000 7f1c153bfb80 10 client.0 _ll_register_callbacks cb 0x7f1bec00f850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.844+0000 7f1bfb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.844+0000 7f1bfb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:58036/0 (socket says 172.21.15.62:58036) 2023-07-14T16:34:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.844+0000 7f1bfb7fe700 1 -- 172.21.15.62:0/3478034508 learned_addr learned my addr 172.21.15.62:0/3478034508 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.844+0000 7f1bfa7fc700 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 0x55d5a3a9d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.844+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.845+0000 7f1bfb7fe700 1 -- 172.21.15.62:0/3478034508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 msgr2=0x55d5a3a9d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.845+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 0x55d5a3a9d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.845+0000 7f1bfb7fe700 1 -- 172.21.15.62:0/3478034508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 msgr2=0x55d5a3a9ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.845+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a9ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.845+0000 7f1bfb7fe700 1 -- 172.21.15.62:0/3478034508 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5a37f6c00 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.845+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a9ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:34:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.845+0000 7f1bfb7fe700 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9d950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4016980 tx=0x7f1bf4006040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce32fcbfb2183d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.845+0000 7f1bdaffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.846+0000 7f1bdaffd700 1 -- 172.21.15.62:0/3478034508 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf401f040 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.846+0000 7f1bdaffd700 1 -- 172.21.15.62:0/3478034508 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1bf4015a00 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.846+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3478034508 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bf0099020 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.848+0000 7f1bdaffd700 1 -- 172.21.15.62:0/3478034508 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf4015bc0 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.848+0000 7f1c153bfb80 10 client.30197 fetch_fsmap learned FSMap version 34 2023-07-14T16:34:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.848+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3478034508 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d5a380c250 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.849+0000 7f1bdaffd700 1 -- 172.21.15.62:0/3478034508 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bf0099020 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.849+0000 7f1bdaffd700 1 -- 172.21.15.62:0/3478034508 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1bf402b070 con 0x55d5a3a8f6e0 2023-07-14T16:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.849+0000 7f1c153bfb80 10 client.30197 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T16:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.849+0000 7f1c153bfb80 10 client.30197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.849+0000 7f1c153bfb80 20 client.30197 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.849+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d5a3b09180 0x55d5a3b0b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.849+0000 7f1c153bfb80 4 client.30197 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:34:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.849+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3478034508 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5a399aff0 con 0x55d5a3b09180 2023-07-14T16:34:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.850+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d5a3b09180 0x55d5a3b0b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.851+0000 7f1bfaffd700 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d5a3b09180 0x55d5a3b0b590 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702827f5f8102a66 server_cookie=115345ed9edb1293 in_seq=0 out_seq=0 2023-07-14T16:34:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.851+0000 7f1bdaffd700 10 client.30197 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.865+0000 7f1bdaffd700 1 -- 172.21.15.62:0/3478034508 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5a399aff0 con 0x55d5a3b09180 2023-07-14T16:34:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.865+0000 7f1bdaffd700 10 client.30197 handle_command_reply: tid=0 2023-07-14T16:34:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.866+0000 7f1c153bfb80 1 client.30197 shutdown 2023-07-14T16:34:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.866+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3478034508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 msgr2=0x55d5a3a9d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.867+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9d950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4016980 tx=0x7f1bf4006040 comp rx=0 tx=0).stop 2023-07-14T16:34:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3478034508 shutdown_connections 2023-07-14T16:34:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d5a3b09180 0x55d5a3b0b590 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d5a3a8f6e0 0x55d5a3a9d950 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d5a38ea6b0 0x55d5a3a9ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1c153bfb80 1 --2- 172.21.15.62:0/3478034508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5a3a8e840 0x55d5a3a9d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3478034508 >> 172.21.15.62:0/3478034508 conn(0x55d5a3a83be0 msgr2=0x55d5a3a95a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1bdaffd700 0 client.30197 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3478034508 shutdown_connections 2023-07-14T16:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.869+0000 7f1c153bfb80 1 -- 172.21.15.62:0/3478034508 wait complete. 2023-07-14T16:34:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.870+0000 7f1c153bfb80 20 client.30197 trim_cache size 0 max 16384 2023-07-14T16:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:34:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:34:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.871+0000 7f1c019ff700 1 -- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1be40a1220 msgr2=0x7f1be40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.871+0000 7f1c019ff700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1be40a1220 0x7f1be40a36d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009750 tx=0x7f1bf400b040 comp rx=0 tx=0).stop 2023-07-14T16:34:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.871+0000 7f1c019ff700 1 -- 172.21.15.62:0/2348555159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bfc074880 msgr2=0x7f1bfc082950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.871+0000 7f1c019ff700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bfc074880 0x7f1bfc082950 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000b450 tx=0x7f1bf000b760 comp rx=0 tx=0).stop 2023-07-14T16:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.872+0000 7f1c019ff700 1 -- 172.21.15.62:0/2348555159 shutdown_connections 2023-07-14T16:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.872+0000 7f1c019ff700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f1be40a1220 0x7f1be40a36d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.872+0000 7f1c019ff700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1bfc077240 0x7f1bfc1b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.872+0000 7f1c019ff700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bfc0711a0 0x7f1bfc082410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.872+0000 7f1c019ff700 1 --2- 172.21.15.62:0/2348555159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bfc074880 0x7f1bfc082950 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000b450 tx=0x7f1bf000b760 comp rx=0 tx=0).stop 2023-07-14T16:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.872+0000 7f1c019ff700 1 -- 172.21.15.62:0/2348555159 >> 172.21.15.62:0/2348555159 conn(0x7f1bfc06ca50 msgr2=0x7f1bfc078ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.872+0000 7f1c019ff700 1 -- 172.21.15.62:0/2348555159 shutdown_connections 2023-07-14T16:34:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:25.872+0000 7f1c019ff700 1 -- 172.21.15.62:0/2348555159 wait complete. 2023-07-14T16:34:26.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:25 smithi164 ceph-mon[97427]: scrub queued for path: / 2023-07-14T16:34:26.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:25 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:25 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:34:26.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:25 smithi164 ceph-mon[97427]: scrub summary: active paths [/] 2023-07-14T16:34:26.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:25 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:25 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:25 smithi164 ceph-mon[97427]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:25 smithi062 ceph-mon[95543]: scrub queued for path: / 2023-07-14T16:34:26.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:25 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:25 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:34:26.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:25 smithi062 ceph-mon[95543]: scrub summary: active paths [/] 2023-07-14T16:34:26.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:25 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:25 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:25 smithi062 ceph-mon[95543]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:25 smithi081 ceph-mon[127795]: scrub queued for path: / 2023-07-14T16:34:26.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:25 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:25 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:34:26.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:25 smithi081 ceph-mon[127795]: scrub summary: active paths [/] 2023-07-14T16:34:26.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:25 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:25 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:26.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:25 smithi081 ceph-mon[127795]: scrub summary: idle+waiting paths [/] 2023-07-14T16:34:27.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:26 smithi164 ceph-mon[97427]: pgmap v921: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.12k op/s 2023-07-14T16:34:27.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:26 smithi062 ceph-mon[95543]: pgmap v921: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.12k op/s 2023-07-14T16:34:27.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:26 smithi081 ceph-mon[127795]: pgmap v921: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 4.12k op/s 2023-07-14T16:34:28.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:27 smithi164 ceph-mon[97427]: pgmap v922: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 4.69k op/s 2023-07-14T16:34:28.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:27 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-a[95519]: 2023-07-14T16:34:27.986+0000 7f9c3e1e2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-14T16:34:28.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:27 smithi062 ceph-mon[95543]: pgmap v922: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 4.69k op/s 2023-07-14T16:34:28.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:27 smithi081 ceph-mon[127795]: pgmap v922: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 4.69k op/s 2023-07-14T16:34:29.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:28 smithi164 ceph-mon[97427]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-14T16:34:29.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:28 smithi164 ceph-mon[97427]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:active 2023-07-14T16:34:29.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:28 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:29.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:28 smithi062 ceph-mon[95543]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-14T16:34:29.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:28 smithi062 ceph-mon[95543]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:active 2023-07-14T16:34:29.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:28 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:29.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:28 smithi081 ceph-mon[127795]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-14T16:34:29.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:28 smithi081 ceph-mon[127795]: mds.? [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] up:active 2023-07-14T16:34:29.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:28 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:30.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:30 smithi164 ceph-mon[97427]: pgmap v923: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 4.75k op/s 2023-07-14T16:34:30.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:30 smithi062 ceph-mon[95543]: pgmap v923: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 4.75k op/s 2023-07-14T16:34:30.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:30 smithi081 ceph-mon[127795]: pgmap v923: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 MiB/s wr, 4.75k op/s 2023-07-14T16:34:33.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:32 smithi062 ceph-mon[95543]: pgmap v924: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2023-07-14T16:34:33.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:32 smithi081 ceph-mon[127795]: pgmap v924: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2023-07-14T16:34:33.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:32 smithi164 ceph-mon[97427]: pgmap v924: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 4.28k op/s 2023-07-14T16:34:35.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:34 smithi081 ceph-mon[127795]: pgmap v925: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2023-07-14T16:34:35.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:34 smithi164 ceph-mon[97427]: pgmap v925: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2023-07-14T16:34:35.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:34 smithi062 ceph-mon[95543]: pgmap v925: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 5.03k op/s 2023-07-14T16:34:37.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:36 smithi081 ceph-mon[127795]: pgmap v926: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:34:37.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:36 smithi164 ceph-mon[97427]: pgmap v926: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:34:37.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:36 smithi062 ceph-mon[95543]: pgmap v926: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 4.20k op/s 2023-07-14T16:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 -- 172.21.15.62:0/3215520817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640ff4d0 msgr2=0x7f1464103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 --2- 172.21.15.62:0/3215520817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640ff4d0 0x7f1464103d80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f145400b0a0 tx=0x7f145400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 -- 172.21.15.62:0/3215520817 shutdown_connections 2023-07-14T16:34:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 --2- 172.21.15.62:0/3215520817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640ff4d0 0x7f1464103d80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 --2- 172.21.15.62:0/3215520817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14640fddc0 0x7f14640fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 --2- 172.21.15.62:0/3215520817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14640feb20 0x7f14640fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 -- 172.21.15.62:0/3215520817 >> 172.21.15.62:0/3215520817 conn(0x7f14640f9780 msgr2=0x7f14640fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 -- 172.21.15.62:0/3215520817 shutdown_connections 2023-07-14T16:34:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.130+0000 7f146c3ff700 1 -- 172.21.15.62:0/3215520817 wait complete. 2023-07-14T16:34:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.132+0000 7f146c3ff700 1 Processor -- start 2023-07-14T16:34:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.132+0000 7f146c3ff700 1 -- start start 2023-07-14T16:34:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146c3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14640fddc0 0x7f146410a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146c3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640feb20 0x7f146410a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146c3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14640ff4d0 0x7f146410b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146c3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146410bad0 con 0x7f14640ff4d0 2023-07-14T16:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146c3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146410bc40 con 0x7f14640feb20 2023-07-14T16:34:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146c3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146410be00 con 0x7f14640fddc0 2023-07-14T16:34:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146a19b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14640fddc0 0x7f146410a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146a19b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14640fddc0 0x7f146410a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:50888/0 (socket says 172.21.15.62:50888) 2023-07-14T16:34:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.133+0000 7f146a19b700 1 -- 172.21.15.62:0/854845076 learned_addr learned my addr 172.21.15.62:0/854845076 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.134+0000 7f146a99c700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14640ff4d0 0x7f146410b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.135+0000 7f146a99c700 1 -- 172.21.15.62:0/854845076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14640fddc0 msgr2=0x7f146410a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.135+0000 7f146a99c700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14640fddc0 0x7f146410a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.135+0000 7f146a99c700 1 -- 172.21.15.62:0/854845076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640feb20 msgr2=0x7f146410a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.135+0000 7f146a99c700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640feb20 0x7f146410a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.135+0000 7f146a99c700 1 -- 172.21.15.62:0/854845076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454009d00 con 0x7f14640ff4d0 2023-07-14T16:34:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.135+0000 7f146a99c700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14640ff4d0 0x7f146410b470 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f1454009fd0 tx=0x7f145400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.136+0000 7f14637fe700 1 -- 172.21.15.62:0/854845076 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454021070 con 0x7f14640ff4d0 2023-07-14T16:34:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.136+0000 7f14637fe700 1 -- 172.21.15.62:0/854845076 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1454003e90 con 0x7f14640ff4d0 2023-07-14T16:34:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.136+0000 7f14637fe700 1 -- 172.21.15.62:0/854845076 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400e040 con 0x7f14640ff4d0 2023-07-14T16:34:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.137+0000 7f146c3ff700 1 -- 172.21.15.62:0/854845076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146410c080 con 0x7f14640ff4d0 2023-07-14T16:34:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.137+0000 7f146c3ff700 1 -- 172.21.15.62:0/854845076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146410c890 con 0x7f14640ff4d0 2023-07-14T16:34:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.141+0000 7f14637fe700 1 -- 172.21.15.62:0/854845076 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1454004cf0 con 0x7f14640ff4d0 2023-07-14T16:34:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.145+0000 7f147fdd9b80 1 Processor -- start 2023-07-14T16:34:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.145+0000 7f147fdd9b80 1 -- start start 2023-07-14T16:34:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.146+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c2f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.146+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c301a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.146+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c30b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.146+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648b34c50 con 0x55b648c306e0 2023-07-14T16:34:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.146+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648b879e0 con 0x55b648a8b6b0 2023-07-14T16:34:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.146+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648b80e70 con 0x55b648c2f840 2023-07-14T16:34:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.146+0000 7f146999a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c2f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.147+0000 7f146999a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c2f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50454/0 (socket says 172.21.15.62:50454) 2023-07-14T16:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.147+0000 7f146999a700 1 -- 172.21.15.62:0/2384365383 learned_addr learned my addr 172.21.15.62:0/2384365383 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.147+0000 7f146999a700 1 --2- 172.21.15.62:0/2384365383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c30b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.148+0000 7f146999a700 1 -- 172.21.15.62:0/2384365383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 msgr2=0x55b648c301a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.148+0000 7f146999a700 1 --2- 172.21.15.62:0/2384365383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c301a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.148+0000 7f146999a700 1 -- 172.21.15.62:0/2384365383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 msgr2=0x55b648c30b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.148+0000 7f146999a700 1 --2- 172.21.15.62:0/2384365383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c30b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.148+0000 7f146999a700 1 -- 172.21.15.62:0/2384365383 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6489aed80 con 0x55b648a8b6b0 2023-07-14T16:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.148+0000 7f146999a700 1 --2- 172.21.15.62:0/2384365383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c2f300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1458014520 tx=0x7f145800d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9333d581933f5e78 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.150+0000 7f1460bfe700 1 -- 172.21.15.62:0/2384365383 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458019070 con 0x55b648a8b6b0 2023-07-14T16:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.150+0000 7f1460bfe700 1 -- 172.21.15.62:0/2384365383 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14580044b0 con 0x55b648a8b6b0 2023-07-14T16:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.150+0000 7f14637fe700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f144c0b22e0 0x7f144c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.151+0000 7f14637fe700 1 -- 172.21.15.62:0/854845076 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f14540cd1b0 con 0x7f14640ff4d0 2023-07-14T16:34:39.123 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.151+0000 7f1460bfe700 1 -- 172.21.15.62:0/2384365383 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458003bd0 con 0x55b648a8b6b0 2023-07-14T16:34:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.152+0000 7f147fdd9b80 1 -- 172.21.15.62:0/2384365383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 msgr2=0x55b648c2f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.124 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.152+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/2384365383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c2f300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1458014520 tx=0x7f145800d520 comp rx=0 tx=0).stop 2023-07-14T16:34:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.153+0000 7f147fdd9b80 1 -- 172.21.15.62:0/2384365383 shutdown_connections 2023-07-14T16:34:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.153+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/2384365383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c30b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.153+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/2384365383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c2f300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.153+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/2384365383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c301a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.153+0000 7f147fdd9b80 1 -- 172.21.15.62:0/2384365383 >> 172.21.15.62:0/2384365383 conn(0x55b648c24be0 msgr2=0x55b648c29cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.154+0000 7f146a19b700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f144c0b22e0 0x7f144c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.155+0000 7f147fdd9b80 1 -- 172.21.15.62:0/2384365383 shutdown_connections 2023-07-14T16:34:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.155+0000 7f147fdd9b80 1 -- 172.21.15.62:0/2384365383 wait complete. 2023-07-14T16:34:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.156+0000 7f147fdd9b80 1 -- start start 2023-07-14T16:34:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.156+0000 7f147fdd9b80 1 Processor -- start 2023-07-14T16:34:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.157+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.157+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c38df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.157+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.157+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648b20a60 con 0x55b648c306e0 2023-07-14T16:34:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.157+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648b20760 con 0x55b648a8b6b0 2023-07-14T16:34:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.157+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648a048c0 con 0x55b648c2f840 2023-07-14T16:34:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.157+0000 7f147fdd9b80 10 client.0 _ll_register_callbacks cb 0x55b648b1e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.157+0000 7f146a19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.158+0000 7f146a19b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c388b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50456/0 (socket says 172.21.15.62:50456) 2023-07-14T16:34:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.158+0000 7f146a19b700 1 -- 172.21.15.62:0/194583995 learned_addr learned my addr 172.21.15.62:0/194583995 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.158+0000 7f146999a700 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c38df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.159+0000 7f146999a700 1 -- 172.21.15.62:0/194583995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 msgr2=0x55b648c388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.159+0000 7f146999a700 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.159+0000 7f146999a700 1 -- 172.21.15.62:0/194583995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 msgr2=0x55b648c3fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.159+0000 7f146999a700 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.159+0000 7f146999a700 1 -- 172.21.15.62:0/194583995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145800dc10 con 0x55b648c2f840 2023-07-14T16:34:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.159+0000 7f146999a700 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c38df0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f14580036e0 tx=0x7f1458005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51ab4aa2c2b6567d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.160+0000 7f1449ffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:34:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.160+0000 7f1449ffb700 1 -- 172.21.15.62:0/194583995 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458019040 con 0x55b648c2f840 2023-07-14T16:34:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.160+0000 7f1449ffb700 1 -- 172.21.15.62:0/194583995 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1458004a30 con 0x55b648c2f840 2023-07-14T16:34:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.160+0000 7f147fdd9b80 1 -- 172.21.15.62:0/194583995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b648ada490 con 0x55b648c2f840 2023-07-14T16:34:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.160+0000 7f1449ffb700 1 -- 172.21.15.62:0/194583995 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1458004bf0 con 0x55b648c2f840 2023-07-14T16:34:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.160+0000 7f147fdd9b80 10 client.30097 fetch_fsmap learned FSMap version 35 2023-07-14T16:34:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.160+0000 7f147fdd9b80 1 -- 172.21.15.62:0/194583995 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b648997c00 con 0x55b648c2f840 2023-07-14T16:34:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.161+0000 7f1449ffb700 1 -- 172.21.15.62:0/194583995 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b648ada490 con 0x55b648c2f840 2023-07-14T16:34:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.162+0000 7f1449ffb700 1 -- 172.21.15.62:0/194583995 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1458003e10 con 0x55b648c2f840 2023-07-14T16:34:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.162+0000 7f147fdd9b80 10 client.30097 fetch_fsmap finished waiting for FSMap version 35 2023-07-14T16:34:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.162+0000 7f147fdd9b80 10 client.30097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.162+0000 7f147fdd9b80 20 client.30097 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.162+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b648c44610 0x55b648c46a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.162+0000 7f147fdd9b80 4 client.30097 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.162+0000 7f147fdd9b80 1 -- 172.21.15.62:0/194583995 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b648b79e50 con 0x55b648c44610 2023-07-14T16:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.162+0000 7f146a19b700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f144c0b22e0 0x7f144c0b4790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1464109dd0 tx=0x7f145c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.163+0000 7f146a19b700 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b648c44610 0x55b648c46a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.164+0000 7f146a19b700 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b648c44610 0x55b648c46a80 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1653c4095187dfcd server_cookie=2ccb7fa29523bdb8 in_seq=0 out_seq=0 2023-07-14T16:34:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.164+0000 7f1449ffb700 10 client.30097 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.165+0000 7f1449ffb700 1 -- 172.21.15.62:0/194583995 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b648b79e50 con 0x55b648c44610 2023-07-14T16:34:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.165+0000 7f1449ffb700 10 client.30097 handle_command_reply: tid=0 2023-07-14T16:34:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.165+0000 7f147fdd9b80 1 client.30097 shutdown 2023-07-14T16:34:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f147fdd9b80 1 -- 172.21.15.62:0/194583995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 msgr2=0x55b648c38df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c38df0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f14580036e0 tx=0x7f1458005fd0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f147fdd9b80 1 -- 172.21.15.62:0/194583995 shutdown_connections 2023-07-14T16:34:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b648c44610 0x55b648c46a80 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/194583995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c38df0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f147fdd9b80 1 -- 172.21.15.62:0/194583995 >> 172.21.15.62:0/194583995 conn(0x55b648c24be0 msgr2=0x55b648c294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.166+0000 7f1449ffb700 0 client.30097 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.167+0000 7f147fdd9b80 1 -- 172.21.15.62:0/194583995 shutdown_connections 2023-07-14T16:34:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.167+0000 7f147fdd9b80 1 -- 172.21.15.62:0/194583995 wait complete. 2023-07-14T16:34:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.169+0000 7f147fdd9b80 20 client.30097 trim_cache size 0 max 16384 2023-07-14T16:34:39.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:38 smithi081 ceph-mon[127795]: pgmap v927: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 4.87k op/s 2023-07-14T16:34:39.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:38 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:34:39.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:38 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:34:39.147 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:38 smithi081 ceph-mon[127795]: scrub summary: idle 2023-07-14T16:34:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.202+0000 7f147fdd9b80 1 Processor -- start 2023-07-14T16:34:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.202+0000 7f147fdd9b80 1 -- start start 2023-07-14T16:34:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.202+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.202+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c33b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.202+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.202+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648c39aa0 con 0x55b648c306e0 2023-07-14T16:34:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.202+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648c21c00 con 0x55b648a8b6b0 2023-07-14T16:34:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.202+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648bf5060 con 0x55b648c2f840 2023-07-14T16:34:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.204+0000 7f146a99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.204+0000 7f146a99c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53506/0 (socket says 172.21.15.62:53506) 2023-07-14T16:34:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.204+0000 7f146a99c700 1 -- 172.21.15.62:0/3130913327 learned_addr learned my addr 172.21.15.62:0/3130913327 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.204+0000 7f146a99c700 1 -- 172.21.15.62:0/3130913327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 msgr2=0x55b648c33b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.204+0000 7f146a99c700 1 --2- 172.21.15.62:0/3130913327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c33b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.204+0000 7f146a99c700 1 -- 172.21.15.62:0/3130913327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 msgr2=0x55b648c335d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:34:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.204+0000 7f146a99c700 1 --2- 172.21.15.62:0/3130913327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c335d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.204+0000 7f146a99c700 1 -- 172.21.15.62:0/3130913327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6489d51f0 con 0x55b648c306e0 2023-07-14T16:34:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.205+0000 7f146a99c700 1 --2- 172.21.15.62:0/3130913327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fdf0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f145409a590 tx=0x7f145409a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae11aa775b592cc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.206+0000 7f14613ff700 1 -- 172.21.15.62:0/3130913327 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145409a600 con 0x55b648c306e0 2023-07-14T16:34:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.206+0000 7f14613ff700 1 -- 172.21.15.62:0/3130913327 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f145409e630 con 0x55b648c306e0 2023-07-14T16:34:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.206+0000 7f14613ff700 1 -- 172.21.15.62:0/3130913327 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454099af0 con 0x55b648c306e0 2023-07-14T16:34:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.207+0000 7f147fdd9b80 1 -- 172.21.15.62:0/3130913327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 msgr2=0x55b648c3fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.207+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/3130913327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fdf0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f145409a590 tx=0x7f145409a3d0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.208+0000 7f147fdd9b80 1 -- 172.21.15.62:0/3130913327 shutdown_connections 2023-07-14T16:34:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.208+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/3130913327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3fdf0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.208+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/3130913327 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648a8b6b0 0x55b648c335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.208+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/3130913327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648c2f840 0x55b648c33b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.208+0000 7f147fdd9b80 1 -- 172.21.15.62:0/3130913327 >> 172.21.15.62:0/3130913327 conn(0x55b648c24be0 msgr2=0x55b648c36ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.211+0000 7f147fdd9b80 1 -- 172.21.15.62:0/3130913327 shutdown_connections 2023-07-14T16:34:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.211+0000 7f147fdd9b80 1 -- 172.21.15.62:0/3130913327 wait complete. 2023-07-14T16:34:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.211+0000 7f147fdd9b80 1 -- start start 2023-07-14T16:34:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.212+0000 7f147fdd9b80 1 Processor -- start 2023-07-14T16:34:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.212+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648a8b6b0 0x55b648c3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.212+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648c2f840 0x55b648c3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.212+0000 7f147fdd9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.212+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648c3f040 con 0x55b648c306e0 2023-07-14T16:34:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.212+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648c3f1b0 con 0x55b648c2f840 2023-07-14T16:34:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.212+0000 7f147fdd9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b648c3f370 con 0x55b648a8b6b0 2023-07-14T16:34:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f147fdd9b80 10 client.0 _ll_register_callbacks cb 0x7f145800dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146999a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648c2f840 0x55b648c3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146a19b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648a8b6b0 0x55b648c3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146999a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648c2f840 0x55b648c3bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50462/0 (socket says 172.21.15.62:50462) 2023-07-14T16:34:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146999a700 1 -- 172.21.15.62:0/263868338 learned_addr learned my addr 172.21.15.62:0/263868338 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146a99c700 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146999a700 1 -- 172.21.15.62:0/263868338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648a8b6b0 msgr2=0x55b648c3b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146999a700 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648a8b6b0 0x55b648c3b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146999a700 1 -- 172.21.15.62:0/263868338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 msgr2=0x55b648c3eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.213+0000 7f146999a700 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.214+0000 7f146999a700 1 -- 172.21.15.62:0/263868338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146410c890 con 0x55b648c2f840 2023-07-14T16:34:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.214+0000 7f146999a700 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648c2f840 0x55b648c3bae0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f145800d390 tx=0x7f1458005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389e15af42cbd09b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.214+0000 7f1449ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.215+0000 7f1449ffb700 1 -- 172.21.15.62:0/263868338 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458019030 con 0x55b648c2f840 2023-07-14T16:34:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.215+0000 7f1449ffb700 1 -- 172.21.15.62:0/263868338 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1458004bc0 con 0x55b648c2f840 2023-07-14T16:34:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.215+0000 7f147fdd9b80 1 -- 172.21.15.62:0/263868338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14580044b0 con 0x55b648c2f840 2023-07-14T16:34:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.215+0000 7f1449ffb700 1 -- 172.21.15.62:0/263868338 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1458005590 con 0x55b648c2f840 2023-07-14T16:34:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.216+0000 7f147fdd9b80 10 client.30209 fetch_fsmap learned FSMap version 35 2023-07-14T16:34:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.216+0000 7f147fdd9b80 1 -- 172.21.15.62:0/263868338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b6489ad250 con 0x55b648c2f840 2023-07-14T16:34:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.217+0000 7f1449ffb700 1 -- 172.21.15.62:0/263868338 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14580044b0 con 0x55b648c2f840 2023-07-14T16:34:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.217+0000 7f1449ffb700 1 -- 172.21.15.62:0/263868338 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 35) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f145802f070 con 0x55b648c2f840 2023-07-14T16:34:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.217+0000 7f147fdd9b80 10 client.30209 fetch_fsmap finished waiting for FSMap version 35 2023-07-14T16:34:39.189 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.217+0000 7f147fdd9b80 10 client.30209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.217+0000 7f147fdd9b80 20 client.30209 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.217+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b648c98cd0 0x55b648c9b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.217+0000 7f147fdd9b80 4 client.30209 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:34:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.217+0000 7f147fdd9b80 1 -- 172.21.15.62:0/263868338 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b648b3bff0 con 0x55b648c98cd0 2023-07-14T16:34:39.190 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.218+0000 7f146a19b700 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b648c98cd0 0x55b648c9b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.219+0000 7f1449ffb700 10 client.30209 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.219+0000 7f146a19b700 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b648c98cd0 0x55b648c9b0e0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8130c14f4ef08d server_cookie=2bbdb37437aaebb2 in_seq=0 out_seq=0 2023-07-14T16:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.237+0000 7f1449ffb700 1 -- 172.21.15.62:0/263868338 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b648b3bff0 con 0x55b648c98cd0 2023-07-14T16:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.237+0000 7f1449ffb700 10 client.30209 handle_command_reply: tid=0 2023-07-14T16:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.237+0000 7f147fdd9b80 1 client.30209 shutdown 2023-07-14T16:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f147fdd9b80 1 -- 172.21.15.62:0/263868338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648c2f840 msgr2=0x55b648c3bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648c2f840 0x55b648c3bae0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f145800d390 tx=0x7f1458005f60 comp rx=0 tx=0).stop 2023-07-14T16:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f147fdd9b80 1 -- 172.21.15.62:0/263868338 shutdown_connections 2023-07-14T16:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b648c306e0 0x55b648c3eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b648a8b6b0 0x55b648c3b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55b648c98cd0 0x55b648c9b0e0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f147fdd9b80 1 --2- 172.21.15.62:0/263868338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b648c2f840 0x55b648c3bae0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f147fdd9b80 1 -- 172.21.15.62:0/263868338 >> 172.21.15.62:0/263868338 conn(0x55b648c24be0 msgr2=0x55b648c335d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.239+0000 7f1449ffb700 0 client.30209 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.240+0000 7f147fdd9b80 1 -- 172.21.15.62:0/263868338 shutdown_connections 2023-07-14T16:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.240+0000 7f147fdd9b80 1 -- 172.21.15.62:0/263868338 wait complete. 2023-07-14T16:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.241+0000 7f147fdd9b80 20 client.30209 trim_cache size 0 max 16384 2023-07-14T16:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.242+0000 7f146c3ff700 1 -- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f144c0b22e0 msgr2=0x7f144c0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.242+0000 7f146c3ff700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f144c0b22e0 0x7f144c0b4790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1464109dd0 tx=0x7f145c009260 comp rx=0 tx=0).stop 2023-07-14T16:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.242+0000 7f146c3ff700 1 -- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14640ff4d0 msgr2=0x7f146410b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.242+0000 7f146c3ff700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14640ff4d0 0x7f146410b470 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f1454009fd0 tx=0x7f145400bce0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.243+0000 7f146c3ff700 1 -- 172.21.15.62:0/854845076 shutdown_connections 2023-07-14T16:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.243+0000 7f146c3ff700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14640ff4d0 0x7f146410b470 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.243+0000 7f146c3ff700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14640fddc0 0x7f146410a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.243+0000 7f146c3ff700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f144c0b22e0 0x7f144c0b4790 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.243+0000 7f146c3ff700 1 --2- 172.21.15.62:0/854845076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640feb20 0x7f146410a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.243+0000 7f146c3ff700 1 -- 172.21.15.62:0/854845076 >> 172.21.15.62:0/854845076 conn(0x7f14640f9780 msgr2=0x7f14640fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.244+0000 7f146c3ff700 1 -- 172.21.15.62:0/854845076 shutdown_connections 2023-07-14T16:34:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:39.244+0000 7f146c3ff700 1 -- 172.21.15.62:0/854845076 wait complete. 2023-07-14T16:34:39.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:38 smithi164 ceph-mon[97427]: pgmap v927: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 4.87k op/s 2023-07-14T16:34:39.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:38 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:34:39.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:38 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:34:39.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:38 smithi164 ceph-mon[97427]: scrub summary: idle 2023-07-14T16:34:39.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:38 smithi062 ceph-mon[95543]: pgmap v927: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 4.87k op/s 2023-07-14T16:34:39.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:38 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:34:39.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:38 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:34:39.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:38 smithi062 ceph-mon[95543]: scrub summary: idle 2023-07-14T16:34:41.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:40 smithi164 ceph-mon[97427]: pgmap v928: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 4.92k op/s 2023-07-14T16:34:41.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:40 smithi062 ceph-mon[95543]: pgmap v928: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 4.92k op/s 2023-07-14T16:34:41.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:40 smithi081 ceph-mon[127795]: pgmap v928: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 4.92k op/s 2023-07-14T16:34:43.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:42 smithi164 ceph-mon[97427]: pgmap v929: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:34:43.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:42 smithi062 ceph-mon[95543]: pgmap v929: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:34:43.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:42 smithi081 ceph-mon[127795]: pgmap v929: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:34:44.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:43 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:34:44.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:43 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:34:44.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:43 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:34:45.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:44 smithi164 ceph-mon[97427]: pgmap v930: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 5.11k op/s 2023-07-14T16:34:45.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:44 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:45.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:44 smithi062 ceph-mon[95543]: pgmap v930: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 5.11k op/s 2023-07-14T16:34:45.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:44 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:45.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:44 smithi081 ceph-mon[127795]: pgmap v930: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 5.11k op/s 2023-07-14T16:34:45.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:44 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:34:47.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:46 smithi164 ceph-mon[97427]: pgmap v931: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.21k op/s 2023-07-14T16:34:47.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:34:47.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:34:47.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:34:47.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:46 smithi062 ceph-mon[95543]: pgmap v931: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.21k op/s 2023-07-14T16:34:47.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:34:47.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:34:47.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:34:47.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:46 smithi081 ceph-mon[127795]: pgmap v931: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.21k op/s 2023-07-14T16:34:47.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:34:47.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:34:47.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:34:49.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:48 smithi164 ceph-mon[97427]: pgmap v932: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:34:49.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:48 smithi062 ceph-mon[95543]: pgmap v932: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:34:49.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:48 smithi081 ceph-mon[127795]: pgmap v932: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:34:51.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:50 smithi164 ceph-mon[97427]: pgmap v933: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2023-07-14T16:34:51.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:50 smithi062 ceph-mon[95543]: pgmap v933: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2023-07-14T16:34:51.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:50 smithi081 ceph-mon[127795]: pgmap v933: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 4.59k op/s 2023-07-14T16:34:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2874127697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0100d70 msgr2=0x7fb9e01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2874127697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0100d70 0x7fb9e01031f0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009990 tx=0x7fb9d4009ca0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2874127697 shutdown_connections 2023-07-14T16:34:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2874127697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e0103730 0x7fb9e0103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2874127697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e00fe410 0x7fb9e0100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2874127697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0100d70 0x7fb9e01031f0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2874127697 >> 172.21.15.62:0/2874127697 conn(0x7fb9e00fa000 msgr2=0x7fb9e00fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2874127697 shutdown_connections 2023-07-14T16:34:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.508+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2874127697 wait complete. 2023-07-14T16:34:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.509+0000 7fb9e61ff700 1 Processor -- start 2023-07-14T16:34:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9e61ff700 1 -- start start 2023-07-14T16:34:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9e61ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e00fe410 0x7fb9e019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9e61ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e0100d70 0x7fb9e019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9e61ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0103730 0x7fb9e019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9e61ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e019da70 con 0x7fb9e0103730 2023-07-14T16:34:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9e61ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e019dbe0 con 0x7fb9e0100d70 2023-07-14T16:34:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9e61ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e019dda0 con 0x7fb9e00fe410 2023-07-14T16:34:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9df7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e00fe410 0x7fb9e019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9df7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e00fe410 0x7fb9e019c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:51226/0 (socket says 172.21.15.62:51226) 2023-07-14T16:34:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9df7fe700 1 -- 172.21.15.62:0/2549651504 learned_addr learned my addr 172.21.15.62:0/2549651504 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9deffd700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e0100d70 0x7fb9e019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.510+0000 7fb9dffff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0103730 0x7fb9e019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9dffff700 1 -- 172.21.15.62:0/2549651504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e00fe410 msgr2=0x7fb9e019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9dffff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e00fe410 0x7fb9e019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9dffff700 1 -- 172.21.15.62:0/2549651504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e0100d70 msgr2=0x7fb9e019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9dffff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e0100d70 0x7fb9e019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9dffff700 1 -- 172.21.15.62:0/2549651504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d40095f0 con 0x7fb9e0103730 2023-07-14T16:34:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e00fe410 0x7fb9e019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:34:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9deffd700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e0100d70 0x7fb9e019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:34:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9dffff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0103730 0x7fb9e019d410 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800be20 tx=0x7fb9d8009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.511+0000 7fb9dcff9700 1 -- 172.21.15.62:0/2549651504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800cbb0 con 0x7fb9e0103730 2023-07-14T16:34:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.512+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2549651504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e019e080 con 0x7fb9e0103730 2023-07-14T16:34:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.512+0000 7fb9dcff9700 1 -- 172.21.15.62:0/2549651504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9d800cd10 con 0x7fb9e0103730 2023-07-14T16:34:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.512+0000 7fb9dcff9700 1 -- 172.21.15.62:0/2549651504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80125e0 con 0x7fb9e0103730 2023-07-14T16:34:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.512+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2549651504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e01a48c0 con 0x7fb9e0103730 2023-07-14T16:34:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.518+0000 7fb9dcff9700 1 -- 172.21.15.62:0/2549651504 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb9d80149d0 con 0x7fb9e0103730 2023-07-14T16:34:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.518+0000 7fb9f9af6b80 1 Processor -- start 2023-07-14T16:34:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9f9af6b80 1 -- start start 2023-07-14T16:34:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945d7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945d81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945d8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555944dcc50 con 0x5555945d86e0 2023-07-14T16:34:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559452f9e0 con 0x5555945d7840 2023-07-14T16:34:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x555594528e70 con 0x5555944336b0 2023-07-14T16:34:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9deffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945d7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9deffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945d7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:51238/0 (socket says 172.21.15.62:51238) 2023-07-14T16:34:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.519+0000 7fb9deffd700 1 -- 172.21.15.62:0/3430192083 learned_addr learned my addr 172.21.15.62:0/3430192083 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.520+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945d81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.520+0000 7fb9deffd700 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945d8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.520+0000 7fb9df7fe700 1 -- 172.21.15.62:0/3430192083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 msgr2=0x5555945d7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.520+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945d7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.520+0000 7fb9df7fe700 1 -- 172.21.15.62:0/3430192083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 msgr2=0x5555945d8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.520+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945d8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.520+0000 7fb9df7fe700 1 -- 172.21.15.62:0/3430192083 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555594356d80 con 0x5555945d7840 2023-07-14T16:34:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.521+0000 7fb9dcff9700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9c80a1220 0x7fb9c80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.521+0000 7fb9dcff9700 1 -- 172.21.15.62:0/2549651504 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9d80cc170 con 0x7fb9e0103730 2023-07-14T16:34:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.521+0000 7fb9deffd700 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945d7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:34:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.521+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945d81a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009e70 tx=0x7fb9d000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8086944204f9754 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.522+0000 7fb9deffd700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9c80a1220 0x7fb9c80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.522+0000 7fb9c5ffb700 1 -- 172.21.15.62:0/3430192083 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000ec20 con 0x5555945d7840 2023-07-14T16:34:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.522+0000 7fb9c5ffb700 1 -- 172.21.15.62:0/3430192083 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9d000ed80 con 0x5555945d7840 2023-07-14T16:34:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.522+0000 7fb9c5ffb700 1 -- 172.21.15.62:0/3430192083 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0012630 con 0x5555945d7840 2023-07-14T16:34:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3430192083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 msgr2=0x5555945d81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945d81a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009e70 tx=0x7fb9d000e4b0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9deffd700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9c80a1220 0x7fb9c80a36d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x5555945d7060 tx=0x7fb9d400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3430192083 shutdown_connections 2023-07-14T16:34:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945d8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945d7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3430192083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945d81a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3430192083 >> 172.21.15.62:0/3430192083 conn(0x5555945ccbe0 msgr2=0x5555945d1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3430192083 shutdown_connections 2023-07-14T16:34:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.523+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3430192083 wait complete. 2023-07-14T16:34:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.524+0000 7fb9f9af6b80 1 -- start start 2023-07-14T16:34:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.525+0000 7fb9f9af6b80 1 Processor -- start 2023-07-14T16:34:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.525+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555944336b0 0x5555945e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.525+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555945d7840 0x5555945e0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.526+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d86e0 0x5555945e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.526+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555944c8a60 con 0x5555944336b0 2023-07-14T16:34:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.526+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555944c8760 con 0x5555945d86e0 2023-07-14T16:34:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.526+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555943ac8c0 con 0x5555945d7840 2023-07-14T16:34:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.526+0000 7fb9f9af6b80 10 client.0 _ll_register_callbacks cb 0x5555944c6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.526+0000 7fb9dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d86e0 0x5555945e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.526+0000 7fb9dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d86e0 0x5555945e7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35952/0 (socket says 172.21.15.62:35952) 2023-07-14T16:34:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.526+0000 7fb9dffff700 1 -- 172.21.15.62:0/1668229527 learned_addr learned my addr 172.21.15.62:0/1668229527 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.527+0000 7fb9dffff700 1 -- 172.21.15.62:0/1668229527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555945d7840 msgr2=0x5555945e0da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:34:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.527+0000 7fb9dffff700 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555945d7840 0x5555945e0da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.527+0000 7fb9dffff700 1 -- 172.21.15.62:0/1668229527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555944336b0 msgr2=0x5555945e0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.527+0000 7fb9dffff700 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555944336b0 0x5555945e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.527+0000 7fb9dffff700 1 -- 172.21.15.62:0/1668229527 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d4020040 con 0x5555945d86e0 2023-07-14T16:34:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.530+0000 7fb9dffff700 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d86e0 0x5555945e7de0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8095fa0 tx=0x7fb9d8098e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e24059d257ff7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.531+0000 7fb9beffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.531+0000 7fb9beffd700 1 -- 172.21.15.62:0/1668229527 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80a1980 con 0x5555945d86e0 2023-07-14T16:34:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.531+0000 7fb9beffd700 1 -- 172.21.15.62:0/1668229527 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9d809d120 con 0x5555945d86e0 2023-07-14T16:34:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.531+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/1668229527 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555594482490 con 0x5555945d86e0 2023-07-14T16:34:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.532+0000 7fb9beffd700 1 -- 172.21.15.62:0/1668229527 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d809d2e0 con 0x5555945d86e0 2023-07-14T16:34:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.532+0000 7fb9f9af6b80 10 client.30218 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.532+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/1668229527 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fb9d0009aa0 con 0x5555945d86e0 2023-07-14T16:34:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.532+0000 7fb9beffd700 1 -- 172.21.15.62:0/1668229527 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555594482490 con 0x5555945d86e0 2023-07-14T16:34:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.534+0000 7fb9beffd700 1 -- 172.21.15.62:0/1668229527 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb9d80a14f0 con 0x5555945d86e0 2023-07-14T16:34:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.534+0000 7fb9f9af6b80 10 client.30218 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.534+0000 7fb9f9af6b80 10 client.30218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.534+0000 7fb9f9af6b80 20 client.30218 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.534+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5555945fd750 0x5555945ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.534+0000 7fb9f9af6b80 4 client.30218 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.534+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/1668229527 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555594521e50 con 0x5555945fd750 2023-07-14T16:34:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.536+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5555945fd750 0x5555945ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.537+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5555945fd750 0x5555945ffb60 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b032a3f8588e5142 server_cookie=120fc50bca646da6 in_seq=0 out_seq=0 2023-07-14T16:34:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.537+0000 7fb9beffd700 10 client.30218 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.538+0000 7fb9beffd700 1 -- 172.21.15.62:0/1668229527 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555594521e50 con 0x5555945fd750 2023-07-14T16:34:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.538+0000 7fb9beffd700 10 client.30218 handle_command_reply: tid=0 2023-07-14T16:34:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.538+0000 7fb9f9af6b80 1 client.30218 shutdown 2023-07-14T16:34:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.539+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/1668229527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d86e0 msgr2=0x5555945e7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.539+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d86e0 0x5555945e7de0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8095fa0 tx=0x7fb9d8098e90 comp rx=0 tx=0).stop 2023-07-14T16:34:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.540+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/1668229527 shutdown_connections 2023-07-14T16:34:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.540+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5555945fd750 0x5555945ffb60 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.540+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d86e0 0x5555945e7de0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.540+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555944336b0 0x5555945e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.540+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/1668229527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555945d7840 0x5555945e0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.540+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/1668229527 >> 172.21.15.62:0/1668229527 conn(0x5555945ccbe0 msgr2=0x5555945d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.540+0000 7fb9beffd700 0 client.30218 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.540+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/1668229527 shutdown_connections 2023-07-14T16:34:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.541+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/1668229527 wait complete. 2023-07-14T16:34:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.541+0000 7fb9f9af6b80 20 client.30218 trim_cache size 0 max 16384 2023-07-14T16:34:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.566+0000 7fb9f9af6b80 1 Processor -- start 2023-07-14T16:34:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.566+0000 7fb9f9af6b80 1 -- start start 2023-07-14T16:34:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555944824c0 con 0x5555945d86e0 2023-07-14T16:34:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555945c9c30 con 0x5555945d7840 2023-07-14T16:34:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555945ced00 con 0x5555944336b0 2023-07-14T16:34:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9df7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9df7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945db5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:51252/0 (socket says 172.21.15.62:51252) 2023-07-14T16:34:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9df7fe700 1 -- 172.21.15.62:0/4101743279 learned_addr learned my addr 172.21.15.62:0/4101743279 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.567+0000 7fb9deffd700 1 --2- 172.21.15.62:0/4101743279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.568+0000 7fb9df7fe700 1 -- 172.21.15.62:0/4101743279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 msgr2=0x5555945e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.568+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/4101743279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.568+0000 7fb9df7fe700 1 -- 172.21.15.62:0/4101743279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 msgr2=0x5555945dbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.568+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/4101743279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.568+0000 7fb9df7fe700 1 -- 172.21.15.62:0/4101743279 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559433fc00 con 0x5555944336b0 2023-07-14T16:34:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.568+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/4101743279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945db5f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0010010 tx=0x7fb9d0006a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23b4a5abc2ff4dff server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.568+0000 7fb9c67fc700 1 -- 172.21.15.62:0/4101743279 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0025520 con 0x5555944336b0 2023-07-14T16:34:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.568+0000 7fb9c67fc700 1 -- 172.21.15.62:0/4101743279 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9d0023070 con 0x5555944336b0 2023-07-14T16:34:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.569+0000 7fb9c67fc700 1 -- 172.21.15.62:0/4101743279 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0025de0 con 0x5555944336b0 2023-07-14T16:34:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.569+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/4101743279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 msgr2=0x5555945db5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.569+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/4101743279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945db5f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0010010 tx=0x7fb9d0006a90 comp rx=0 tx=0).stop 2023-07-14T16:34:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.570+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/4101743279 shutdown_connections 2023-07-14T16:34:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.570+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/4101743279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945dbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.570+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/4101743279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945db5f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.570+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/4101743279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.570+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/4101743279 >> 172.21.15.62:0/4101743279 conn(0x5555945ccbe0 msgr2=0x5555945deaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.570+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/4101743279 shutdown_connections 2023-07-14T16:34:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.570+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/4101743279 wait complete. 2023-07-14T16:34:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.571+0000 7fb9f9af6b80 1 -- start start 2023-07-14T16:34:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.571+0000 7fb9f9af6b80 1 Processor -- start 2023-07-14T16:34:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9f9af6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555945e71d0 con 0x5555945d86e0 2023-07-14T16:34:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555945e7340 con 0x5555945d7840 2023-07-14T16:34:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9f9af6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555945e7500 con 0x5555944336b0 2023-07-14T16:34:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9f9af6b80 10 client.0 _ll_register_callbacks cb 0x7fb9d8099300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9deffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.572+0000 7fb9deffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:35968/0 (socket says 172.21.15.62:35968) 2023-07-14T16:34:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.573+0000 7fb9deffd700 1 -- 172.21.15.62:0/3341543050 learned_addr learned my addr 172.21.15.62:0/3341543050 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.573+0000 7fb9dffff700 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.573+0000 7fb9deffd700 1 -- 172.21.15.62:0/3341543050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 msgr2=0x5555945e5fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:34:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.573+0000 7fb9deffd700 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945e5fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.573+0000 7fb9deffd700 1 -- 172.21.15.62:0/3341543050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 msgr2=0x5555945e6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.573+0000 7fb9deffd700 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945e6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.573+0000 7fb9deffd700 1 -- 172.21.15.62:0/3341543050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555594356d80 con 0x5555945d7840 2023-07-14T16:34:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.574+0000 7fb9deffd700 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e6520 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4005850 tx=0x7fb9d401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2bf34e8c39459b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.574+0000 7fb9beffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.574+0000 7fb9beffd700 1 -- 172.21.15.62:0/3341543050 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4004190 con 0x5555945d7840 2023-07-14T16:34:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.574+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3341543050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9d000ed80 con 0x5555945d7840 2023-07-14T16:34:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.574+0000 7fb9beffd700 1 -- 172.21.15.62:0/3341543050 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9d400f670 con 0x5555945d7840 2023-07-14T16:34:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.574+0000 7fb9beffd700 1 -- 172.21.15.62:0/3341543050 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d400f830 con 0x5555945d7840 2023-07-14T16:34:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.575+0000 7fb9f9af6b80 10 client.30224 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.575+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3341543050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55559437d1f0 con 0x5555945d7840 2023-07-14T16:34:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.575+0000 7fb9beffd700 1 -- 172.21.15.62:0/3341543050 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9d000ed80 con 0x5555945d7840 2023-07-14T16:34:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.576+0000 7fb9beffd700 1 -- 172.21.15.62:0/3341543050 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb9d40189b0 con 0x5555945d7840 2023-07-14T16:34:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.576+0000 7fb9f9af6b80 10 client.30224 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.576+0000 7fb9f9af6b80 10 client.30224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.576+0000 7fb9f9af6b80 20 client.30224 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.576+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555594652250 0x555594654660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.576+0000 7fb9f9af6b80 4 client.30224 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:34:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.576+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3341543050 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555944e3ff0 con 0x555594652250 2023-07-14T16:34:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.577+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555594652250 0x555594654660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.580+0000 7fb9df7fe700 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555594652250 0x555594654660 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cba8df34492089a server_cookie=4157ac4f40858575 in_seq=0 out_seq=0 2023-07-14T16:34:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.580+0000 7fb9beffd700 10 client.30224 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.601+0000 7fb9beffd700 1 -- 172.21.15.62:0/3341543050 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555944e3ff0 con 0x555594652250 2023-07-14T16:34:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.601+0000 7fb9beffd700 10 client.30224 handle_command_reply: tid=0 2023-07-14T16:34:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.601+0000 7fb9f9af6b80 1 client.30224 shutdown 2023-07-14T16:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.606+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3341543050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 msgr2=0x5555945e6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.606+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e6520 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4005850 tx=0x7fb9d401a040 comp rx=0 tx=0).stop 2023-07-14T16:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3341543050 shutdown_connections 2023-07-14T16:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555945d86e0 0x5555945e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5555944336b0 0x5555945e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x555594652250 0x555594654660 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9f9af6b80 1 --2- 172.21.15.62:0/3341543050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5555945d7840 0x5555945e6520 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3341543050 >> 172.21.15.62:0/3341543050 conn(0x5555945ccbe0 msgr2=0x5555945db5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9beffd700 0 client.30224 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3341543050 shutdown_connections 2023-07-14T16:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.608+0000 7fb9f9af6b80 1 -- 172.21.15.62:0/3341543050 wait complete. 2023-07-14T16:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.609+0000 7fb9f9af6b80 20 client.30224 trim_cache size 0 max 16384 2023-07-14T16:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:34:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.611+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9c80a1220 msgr2=0x7fb9c80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.611+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9c80a1220 0x7fb9c80a36d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x5555945d7060 tx=0x7fb9d400b540 comp rx=0 tx=0).stop 2023-07-14T16:34:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.611+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0103730 msgr2=0x7fb9e019d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.611+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0103730 0x7fb9e019d410 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800be20 tx=0x7fb9d8009ef0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.612+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2549651504 shutdown_connections 2023-07-14T16:34:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.612+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9e0103730 0x7fb9e019d410 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.612+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9e00fe410 0x7fb9e019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.612+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb9c80a1220 0x7fb9c80a36d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.612+0000 7fb9e61ff700 1 --2- 172.21.15.62:0/2549651504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb9e0100d70 0x7fb9e019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.612+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2549651504 >> 172.21.15.62:0/2549651504 conn(0x7fb9e00fa000 msgr2=0x7fb9e00ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.612+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2549651504 shutdown_connections 2023-07-14T16:34:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:34:52.613+0000 7fb9e61ff700 1 -- 172.21.15.62:0/2549651504 wait complete. 2023-07-14T16:34:53.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:52 smithi164 ceph-mon[97427]: pgmap v934: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.23k op/s 2023-07-14T16:34:53.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:52 smithi062 ceph-mon[95543]: pgmap v934: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.23k op/s 2023-07-14T16:34:53.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:52 smithi081 ceph-mon[127795]: pgmap v934: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.23k op/s 2023-07-14T16:34:55.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T16:34:55.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:54 smithi164 ceph-mon[97427]: pgmap v935: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.55k op/s 2023-07-14T16:34:55.319 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:54 smithi062 ceph-mon[95543]: pgmap v935: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.55k op/s 2023-07-14T16:34:55.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:54 smithi081 ceph-mon[127795]: pgmap v935: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.55k op/s 2023-07-14T16:34:55.572 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.600+0000 7f350d5ff700 1 -- 172.21.15.62:0/3670048199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3508071830 msgr2=0x7f350810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.573 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.600+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3670048199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3508071830 0x7f350810b7d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009990 tx=0x7f34f0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.601+0000 7f350d5ff700 1 -- 172.21.15.62:0/3670048199 shutdown_connections 2023-07-14T16:34:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.601+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3670048199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f350810bd60 0x7f350810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.601+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3670048199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3508070ee0 0x7f35080712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.601+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3670048199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3508071830 0x7f350810b7d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.601+0000 7f350d5ff700 1 -- 172.21.15.62:0/3670048199 >> 172.21.15.62:0/3670048199 conn(0x7f350806ca90 msgr2=0x7f350806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:55.574 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.602+0000 7f350d5ff700 1 -- 172.21.15.62:0/3670048199 shutdown_connections 2023-07-14T16:34:55.575 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.602+0000 7f350d5ff700 1 -- 172.21.15.62:0/3670048199 wait complete. 2023-07-14T16:34:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.604+0000 7f350d5ff700 1 Processor -- start 2023-07-14T16:34:55.576 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.604+0000 7f350d5ff700 1 -- start start 2023-07-14T16:34:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.604+0000 7f350d5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3508070ee0 0x7f35081a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.604+0000 7f350d5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508071830 0x7f35081a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.604+0000 7f350d5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f350810bd60 0x7f35081abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.604+0000 7f350d5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35081ac1a0 con 0x7f3508070ee0 2023-07-14T16:34:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.604+0000 7f350d5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35081ac2e0 con 0x7f3508071830 2023-07-14T16:34:55.577 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.604+0000 7f350d5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35081ac470 con 0x7f350810bd60 2023-07-14T16:34:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f3506ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3508070ee0 0x7f35081a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f3506ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3508070ee0 0x7f35081a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49464/0 (socket says 172.21.15.62:49464) 2023-07-14T16:34:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f3506ffd700 1 -- 172.21.15.62:0/3409042604 learned_addr learned my addr 172.21.15.62:0/3409042604 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f35067fc700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508071830 0x7f35081a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f35077fe700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f350810bd60 0x7f35081abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f35067fc700 1 -- 172.21.15.62:0/3409042604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f350810bd60 msgr2=0x7f35081abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f35067fc700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f350810bd60 0x7f35081abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f35067fc700 1 -- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3508070ee0 msgr2=0x7f35081a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f35067fc700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3508070ee0 0x7f35081a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.605+0000 7f35067fc700 1 -- 172.21.15.62:0/3409042604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00095f0 con 0x7f3508071830 2023-07-14T16:34:55.579 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.606+0000 7f35067fc700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508071830 0x7f35081a4b60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009990 tx=0x7f34f0003a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.606+0000 7f34fffff700 1 -- 172.21.15.62:0/3409042604 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f001d070 con 0x7f3508071830 2023-07-14T16:34:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.606+0000 7f34fffff700 1 -- 172.21.15.62:0/3409042604 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34f00041f0 con 0x7f3508071830 2023-07-14T16:34:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.606+0000 7f350d5ff700 1 -- 172.21.15.62:0/3409042604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35081ac6f0 con 0x7f3508071830 2023-07-14T16:34:55.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.606+0000 7f34fffff700 1 -- 172.21.15.62:0/3409042604 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000fc70 con 0x7f3508071830 2023-07-14T16:34:55.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.607+0000 7f350d5ff700 1 -- 172.21.15.62:0/3409042604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35081acbf0 con 0x7f3508071830 2023-07-14T16:34:55.587 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.614+0000 7f3520e82b80 1 Processor -- start 2023-07-14T16:34:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.614+0000 7f34fffff700 1 -- 172.21.15.62:0/3409042604 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f34f000f460 con 0x7f3508071830 2023-07-14T16:34:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.615+0000 7f3520e82b80 1 -- start start 2023-07-14T16:34:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.615+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b464d0 0x55d874b468e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.615+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.588 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.615+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.615+0000 7f3520e82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874a4bcb0 con 0x55d874b464d0 2023-07-14T16:34:55.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.615+0000 7f3520e82b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874a9ea40 con 0x55d874b47c70 2023-07-14T16:34:55.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.615+0000 7f3520e82b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874a97ed0 con 0x55d874b46e20 2023-07-14T16:34:55.589 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f3506ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f3506ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41826/0 (socket says 172.21.15.62:41826) 2023-07-14T16:34:55.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f3506ffd700 1 -- 172.21.15.62:0/683486771 learned_addr learned my addr 172.21.15.62:0/683486771 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:55.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f35067fc700 1 --2- 172.21.15.62:0/683486771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f3506ffd700 1 -- 172.21.15.62:0/683486771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 msgr2=0x55d874b480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:55.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f3506ffd700 1 --2- 172.21.15.62:0/683486771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f3506ffd700 1 -- 172.21.15.62:0/683486771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b464d0 msgr2=0x55d874b468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:55.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f3506ffd700 1 --2- 172.21.15.62:0/683486771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b464d0 0x55d874b468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.590 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.616+0000 7f3506ffd700 1 -- 172.21.15.62:0/683486771 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8748c5de0 con 0x55d874b46e20 2023-07-14T16:34:55.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.617+0000 7f3506ffd700 1 --2- 172.21.15.62:0/683486771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b476e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009bb0 tx=0x7f34f8009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6294ea983319454d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:55.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.618+0000 7f34fd7fa700 1 -- 172.21.15.62:0/683486771 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80096b0 con 0x55d874b46e20 2023-07-14T16:34:55.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.618+0000 7f34fd7fa700 1 -- 172.21.15.62:0/683486771 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34f800ee80 con 0x55d874b46e20 2023-07-14T16:34:55.591 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.618+0000 7f34fd7fa700 1 -- 172.21.15.62:0/683486771 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80123d0 con 0x55d874b46e20 2023-07-14T16:34:55.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.618+0000 7f34fffff700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f34f40a12a0 0x7f34f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.619+0000 7f34fffff700 1 -- 172.21.15.62:0/3409042604 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34f00cdb30 con 0x7f3508071830 2023-07-14T16:34:55.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.619+0000 7f3520e82b80 1 -- 172.21.15.62:0/683486771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 msgr2=0x55d874b476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.592 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.619+0000 7f3520e82b80 1 --2- 172.21.15.62:0/683486771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b476e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009bb0 tx=0x7f34f8009ec0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.619+0000 7f3520e82b80 1 -- 172.21.15.62:0/683486771 shutdown_connections 2023-07-14T16:34:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.619+0000 7f3520e82b80 1 --2- 172.21.15.62:0/683486771 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.619+0000 7f3520e82b80 1 --2- 172.21.15.62:0/683486771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b464d0 0x55d874b468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.619+0000 7f3520e82b80 1 --2- 172.21.15.62:0/683486771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b476e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.619+0000 7f3520e82b80 1 -- 172.21.15.62:0/683486771 >> 172.21.15.62:0/683486771 conn(0x55d874b3bda0 msgr2=0x55d874b411d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.620+0000 7f35077fe700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f34f40a12a0 0x7f34f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.593 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.620+0000 7f3520e82b80 1 -- 172.21.15.62:0/683486771 shutdown_connections 2023-07-14T16:34:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.620+0000 7f3520e82b80 1 -- 172.21.15.62:0/683486771 wait complete. 2023-07-14T16:34:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.621+0000 7f3520e82b80 1 -- start start 2023-07-14T16:34:55.594 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.621+0000 7f3520e82b80 1 Processor -- start 2023-07-14T16:34:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.622+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.622+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b46e20 0x55d874b50730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.622+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b57770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.622+0000 7f3520e82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874a37ac0 con 0x55d874b46e20 2023-07-14T16:34:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.622+0000 7f3520e82b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874a377c0 con 0x55d874b47c70 2023-07-14T16:34:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.622+0000 7f3520e82b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87491b920 con 0x55d874b464d0 2023-07-14T16:34:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.622+0000 7f3520e82b80 10 client.0 _ll_register_callbacks cb 0x55d874a35760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:55.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.623+0000 7f35077fe700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f34f40a12a0 0x7f34f40a3750 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3500004b10 tx=0x7f350000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:55.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.623+0000 7f35077fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b57770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.623+0000 7f35077fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b57770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48632/0 (socket says 172.21.15.62:48632) 2023-07-14T16:34:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.623+0000 7f35077fe700 1 -- 172.21.15.62:0/609597508 learned_addr learned my addr 172.21.15.62:0/609597508 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.623+0000 7f3506ffd700 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.624+0000 7f35077fe700 1 -- 172.21.15.62:0/609597508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 msgr2=0x55d874b501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.624+0000 7f35077fe700 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.624+0000 7f35077fe700 1 -- 172.21.15.62:0/609597508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b46e20 msgr2=0x55d874b50730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:55.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.624+0000 7f35077fe700 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b46e20 0x55d874b50730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.624+0000 7f35077fe700 1 -- 172.21.15.62:0/609597508 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f8009810 con 0x55d874b47c70 2023-07-14T16:34:55.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.624+0000 7f35077fe700 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b57770 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3500016df0 tx=0x7f3500004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c843862873db4f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:55.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.624+0000 7f34ee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:55.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.624+0000 7f34ee7fc700 1 -- 172.21.15.62:0/609597508 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500014de0 con 0x55d874b47c70 2023-07-14T16:34:55.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.625+0000 7f34ee7fc700 1 -- 172.21.15.62:0/609597508 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3500016e20 con 0x55d874b47c70 2023-07-14T16:34:55.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.625+0000 7f3520e82b80 1 -- 172.21.15.62:0/609597508 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8749f14f0 con 0x55d874b47c70 2023-07-14T16:34:55.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.625+0000 7f34ee7fc700 1 -- 172.21.15.62:0/609597508 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f350000ae90 con 0x55d874b47c70 2023-07-14T16:34:55.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.626+0000 7f3520e82b80 10 client.30233 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:55.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.626+0000 7f3520e82b80 1 -- 172.21.15.62:0/609597508 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d8748aec60 con 0x55d874b47c70 2023-07-14T16:34:55.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.626+0000 7f34ee7fc700 1 -- 172.21.15.62:0/609597508 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d8749f14f0 con 0x55d874b47c70 2023-07-14T16:34:55.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.626+0000 7f34ee7fc700 1 -- 172.21.15.62:0/609597508 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f35000204a0 con 0x55d874b47c70 2023-07-14T16:34:55.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.626+0000 7f3520e82b80 10 client.30233 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:55.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.626+0000 7f3520e82b80 10 client.30233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:55.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.626+0000 7f3520e82b80 20 client.30233 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:55.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.627+0000 7f3520e82b80 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d874b6d090 0x55d874b6f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.627+0000 7f3520e82b80 4 client.30233 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:55.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.627+0000 7f3520e82b80 1 -- 172.21.15.62:0/609597508 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d874a90eb0 con 0x55d874b6d090 2023-07-14T16:34:55.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.627+0000 7f3506ffd700 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d874b6d090 0x55d874b6f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.628+0000 7f34ee7fc700 10 client.30233 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:55.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.628+0000 7f3506ffd700 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d874b6d090 0x55d874b6f4a0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf53b479c38ffe6c server_cookie=a5e017a99ee6932 in_seq=0 out_seq=0 2023-07-14T16:34:55.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f34ee7fc700 1 -- 172.21.15.62:0/609597508 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d874a90eb0 con 0x55d874b6d090 2023-07-14T16:34:55.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f34ee7fc700 10 client.30233 handle_command_reply: tid=0 2023-07-14T16:34:55.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 client.30233 shutdown 2023-07-14T16:34:55.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 -- 172.21.15.62:0/609597508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 msgr2=0x55d874b57770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b57770 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3500016df0 tx=0x7f3500004b40 comp rx=0 tx=0).stop 2023-07-14T16:34:55.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 -- 172.21.15.62:0/609597508 shutdown_connections 2023-07-14T16:34:55.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d874b6d090 0x55d874b6f4a0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b47c70 0x55d874b57770 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 --2- 172.21.15.62:0/609597508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b46e20 0x55d874b50730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.630+0000 7f3520e82b80 1 -- 172.21.15.62:0/609597508 >> 172.21.15.62:0/609597508 conn(0x55d874b3bda0 msgr2=0x55d874b408b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:55.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.631+0000 7f34ee7fc700 0 client.30233 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:55.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.631+0000 7f3520e82b80 1 -- 172.21.15.62:0/609597508 shutdown_connections 2023-07-14T16:34:55.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.631+0000 7f3520e82b80 1 -- 172.21.15.62:0/609597508 wait complete. 2023-07-14T16:34:55.605 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.631+0000 7f3520e82b80 20 client.30233 trim_cache size 0 max 16384 2023-07-14T16:34:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.662+0000 7f3520e82b80 1 Processor -- start 2023-07-14T16:34:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.662+0000 7f3520e82b80 1 -- start start 2023-07-14T16:34:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.662+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b464d0 0x55d874b6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.662+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b57730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.662+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b4e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.635 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.662+0000 7f3520e82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874b51370 con 0x55d874b47c70 2023-07-14T16:34:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.662+0000 7f3520e82b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874b4f6a0 con 0x55d874b464d0 2023-07-14T16:34:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.662+0000 7f3520e82b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874b4fd60 con 0x55d874b46e20 2023-07-14T16:34:55.636 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.663+0000 7f35067fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b57730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.663+0000 7f35067fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b57730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41836/0 (socket says 172.21.15.62:41836) 2023-07-14T16:34:55.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.663+0000 7f35067fc700 1 -- 172.21.15.62:0/1998058948 learned_addr learned my addr 172.21.15.62:0/1998058948 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:55.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.663+0000 7f3506ffd700 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b464d0 0x55d874b6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.663+0000 7f35077fe700 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b4e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.664+0000 7f3506ffd700 1 -- 172.21.15.62:0/1998058948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 msgr2=0x55d874b57730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:55.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.664+0000 7f3506ffd700 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b57730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.637 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.664+0000 7f3506ffd700 1 -- 172.21.15.62:0/1998058948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 msgr2=0x55d874b4e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.664+0000 7f3506ffd700 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b4e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.664+0000 7f3506ffd700 1 -- 172.21.15.62:0/1998058948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d874b48310 con 0x55d874b464d0 2023-07-14T16:34:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.664+0000 7f35077fe700 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b4e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:34:55.638 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.664+0000 7f3506ffd700 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b464d0 0x55d874b6f0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000c00 tx=0x7f34f8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d7892fc9813088d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:55.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.667+0000 7f34fdffb700 1 -- 172.21.15.62:0/1998058948 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8021040 con 0x55d874b464d0 2023-07-14T16:34:55.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.667+0000 7f34fdffb700 1 -- 172.21.15.62:0/1998058948 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34f800ec70 con 0x55d874b464d0 2023-07-14T16:34:55.639 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.667+0000 7f34fdffb700 1 -- 172.21.15.62:0/1998058948 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8029340 con 0x55d874b464d0 2023-07-14T16:34:55.641 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.669+0000 7f3520e82b80 1 -- 172.21.15.62:0/1998058948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b464d0 msgr2=0x55d874b6f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.669+0000 7f3520e82b80 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b464d0 0x55d874b6f0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000c00 tx=0x7f34f8008040 comp rx=0 tx=0).stop 2023-07-14T16:34:55.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.669+0000 7f3520e82b80 1 -- 172.21.15.62:0/1998058948 shutdown_connections 2023-07-14T16:34:55.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.670+0000 7f3520e82b80 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b4e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.642 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.670+0000 7f3520e82b80 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b464d0 0x55d874b6f0a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.670+0000 7f3520e82b80 1 --2- 172.21.15.62:0/1998058948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b46e20 0x55d874b57730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.670+0000 7f3520e82b80 1 -- 172.21.15.62:0/1998058948 >> 172.21.15.62:0/1998058948 conn(0x55d874b3bda0 msgr2=0x55d874b4ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:55.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.670+0000 7f3520e82b80 1 -- 172.21.15.62:0/1998058948 shutdown_connections 2023-07-14T16:34:55.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.670+0000 7f3520e82b80 1 -- 172.21.15.62:0/1998058948 wait complete. 2023-07-14T16:34:55.643 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.670+0000 7f3520e82b80 1 -- start start 2023-07-14T16:34:55.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.670+0000 7f3520e82b80 1 Processor -- start 2023-07-14T16:34:55.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.671+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b55150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.671+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b46e20 0x55d874b55690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.644 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.671+0000 7f3520e82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.671+0000 7f3520e82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874b56be0 con 0x55d874b47c70 2023-07-14T16:34:55.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.671+0000 7f3520e82b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874b56d50 con 0x55d874b46e20 2023-07-14T16:34:55.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.671+0000 7f3520e82b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d874b56f10 con 0x55d874b464d0 2023-07-14T16:34:55.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.671+0000 7f3520e82b80 10 client.0 _ll_register_callbacks cb 0x7f3500016790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:55.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f3506ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b55150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f35067fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b46e20 0x55d874b55690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f3506ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b55150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41840/0 (socket says 172.21.15.62:41840) 2023-07-14T16:34:55.645 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f3506ffd700 1 -- 172.21.15.62:0/675004287 learned_addr learned my addr 172.21.15.62:0/675004287 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:55.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f35067fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b46e20 0x55d874b55690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48652/0 (socket says 172.21.15.62:48652) 2023-07-14T16:34:55.646 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f35077fe700 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f35067fc700 1 -- 172.21.15.62:0/675004287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 msgr2=0x55d874b55150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:55.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f35067fc700 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b55150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f35067fc700 1 -- 172.21.15.62:0/675004287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 msgr2=0x55d874b566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:55.647 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f35067fc700 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.672+0000 7f35067fc700 1 -- 172.21.15.62:0/675004287 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8748c5de0 con 0x55d874b46e20 2023-07-14T16:34:55.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.673+0000 7f35067fc700 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b46e20 0x55d874b55690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f34f00a2e10 tx=0x7f34f0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d06aa684d20d97b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:55.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.673+0000 7f34ee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:55.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.673+0000 7f34ee7fc700 1 -- 172.21.15.62:0/675004287 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00cdea0 con 0x55d874b46e20 2023-07-14T16:34:55.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.673+0000 7f34ee7fc700 1 -- 172.21.15.62:0/675004287 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f34f009efb0 con 0x55d874b46e20 2023-07-14T16:34:55.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.673+0000 7f3520e82b80 1 -- 172.21.15.62:0/675004287 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34f800ee80 con 0x55d874b46e20 2023-07-14T16:34:55.648 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.673+0000 7f34ee7fc700 1 -- 172.21.15.62:0/675004287 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34f00a3340 con 0x55d874b46e20 2023-07-14T16:34:55.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.673+0000 7f3520e82b80 10 client.30239 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:55.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.673+0000 7f3520e82b80 1 -- 172.21.15.62:0/675004287 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d8748ec250 con 0x55d874b46e20 2023-07-14T16:34:55.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f34ee7fc700 1 -- 172.21.15.62:0/675004287 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34f800ee80 con 0x55d874b46e20 2023-07-14T16:34:55.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f34ee7fc700 1 -- 172.21.15.62:0/675004287 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34f0096e20 con 0x55d874b46e20 2023-07-14T16:34:55.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f3520e82b80 10 client.30239 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:55.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f3520e82b80 10 client.30239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:55.649 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f3520e82b80 20 client.30239 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:55.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f3520e82b80 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d874bc18c0 0x55d874bc3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:55.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f3520e82b80 4 client.30239 mds_command: new command op to 24602 tid=0[{"prefix": "scrub status"}] 2023-07-14T16:34:55.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f3520e82b80 1 -- 172.21.15.62:0/675004287 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d874a53050 con 0x55d874bc18c0 2023-07-14T16:34:55.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.674+0000 7f3506ffd700 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d874bc18c0 0x55d874bc3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:55.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.675+0000 7f3506ffd700 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d874bc18c0 0x55d874bc3cd0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4d16068091136e server_cookie=30dca517884374b6 in_seq=0 out_seq=0 2023-07-14T16:34:55.650 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.675+0000 7f34ee7fc700 10 client.30239 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:55.651 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.675+0000 7f34ee7fc700 1 -- 172.21.15.62:0/675004287 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d874a53050 con 0x55d874bc18c0 2023-07-14T16:34:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.675+0000 7f34ee7fc700 10 client.30239 handle_command_reply: tid=0 2023-07-14T16:34:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.675+0000 7f3520e82b80 1 client.30239 shutdown 2023-07-14T16:34:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.675+0000 7f3520e82b80 1 -- 172.21.15.62:0/675004287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b46e20 msgr2=0x55d874b55690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.675+0000 7f3520e82b80 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b46e20 0x55d874b55690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f34f00a2e10 tx=0x7f34f0000bc0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 1 -- 172.21.15.62:0/675004287 shutdown_connections 2023-07-14T16:34:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d874b47c70 0x55d874b566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d874b464d0 0x55d874b55150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55d874bc18c0 0x55d874bc3cd0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 1 --2- 172.21.15.62:0/675004287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d874b46e20 0x55d874b55690 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 1 -- 172.21.15.62:0/675004287 >> 172.21.15.62:0/675004287 conn(0x55d874b3bda0 msgr2=0x55d874b6f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f34ee7fc700 0 client.30239 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 1 -- 172.21.15.62:0/675004287 shutdown_connections 2023-07-14T16:34:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 1 -- 172.21.15.62:0/675004287 wait complete. 2023-07-14T16:34:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.676+0000 7f3520e82b80 20 client.30239 trim_cache size 0 max 16384 2023-07-14T16:34:55.654 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:34:55.654 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2023-07-14T16:34:55.654 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2023-07-14T16:34:55.654 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:34:55.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.683+0000 7f350d5ff700 1 -- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f34f40a12a0 msgr2=0x7f34f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.683+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f34f40a12a0 0x7f34f40a3750 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3500004b10 tx=0x7f350000c040 comp rx=0 tx=0).stop 2023-07-14T16:34:55.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.683+0000 7f350d5ff700 1 -- 172.21.15.62:0/3409042604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508071830 msgr2=0x7f35081a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:55.656 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.683+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508071830 0x7f35081a4b60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009990 tx=0x7f34f0003a60 comp rx=0 tx=0).stop 2023-07-14T16:34:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.684+0000 7f350d5ff700 1 -- 172.21.15.62:0/3409042604 shutdown_connections 2023-07-14T16:34:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.684+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f350810bd60 0x7f35081abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.684+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3508070ee0 0x7f35081a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.684+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f34f40a12a0 0x7f34f40a3750 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.684+0000 7f350d5ff700 1 --2- 172.21.15.62:0/3409042604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3508071830 0x7f35081a4b60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:55.657 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.684+0000 7f350d5ff700 1 -- 172.21.15.62:0/3409042604 >> 172.21.15.62:0/3409042604 conn(0x7f350806ca90 msgr2=0x7f350806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:55.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.684+0000 7f350d5ff700 1 -- 172.21.15.62:0/3409042604 shutdown_connections 2023-07-14T16:34:55.658 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:55.685+0000 7f350d5ff700 1 -- 172.21.15.62:0/3409042604 wait complete. 2023-07-14T16:34:55.686 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T16:34:55.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T16:34:56.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.090+0000 7f82acbff700 1 -- 172.21.15.62:0/1255855055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82a80fe9d0 msgr2=0x7f82a80fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:56.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.090+0000 7f82acbff700 1 --2- 172.21.15.62:0/1255855055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82a80fe9d0 0x7f82a80fede0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f82a000dd20 tx=0x7f82a000f330 comp rx=0 tx=0).stop 2023-07-14T16:34:56.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.090+0000 7f82acbff700 1 -- 172.21.15.62:0/1255855055 shutdown_connections 2023-07-14T16:34:56.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.090+0000 7f82acbff700 1 --2- 172.21.15.62:0/1255855055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a8100550 0x7f82a8106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.090+0000 7f82acbff700 1 --2- 172.21.15.62:0/1255855055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82a80fe9d0 0x7f82a80fede0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.090+0000 7f82acbff700 1 --2- 172.21.15.62:0/1255855055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82a80ffbc0 0x7f82a8100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.090+0000 7f82acbff700 1 -- 172.21.15.62:0/1255855055 >> 172.21.15.62:0/1255855055 conn(0x7f82a80f9fd0 msgr2=0x7f82a80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:56.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.091+0000 7f82acbff700 1 -- 172.21.15.62:0/1255855055 shutdown_connections 2023-07-14T16:34:56.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.091+0000 7f82acbff700 1 -- 172.21.15.62:0/1255855055 wait complete. 2023-07-14T16:34:56.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.093+0000 7f82acbff700 1 Processor -- start 2023-07-14T16:34:56.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.093+0000 7f82acbff700 1 -- start start 2023-07-14T16:34:56.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.093+0000 7f82acbff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82a80fe9d0 0x7f82a81935a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:56.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.093+0000 7f82acbff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a80ffbc0 0x7f82a8193ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:56.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.093+0000 7f82acbff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82a8100550 0x7f82a819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:56.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.093+0000 7f82acbff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a8069f30 con 0x7f82a80fe9d0 2023-07-14T16:34:56.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.093+0000 7f82acbff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a806a0a0 con 0x7f82a8100550 2023-07-14T16:34:56.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.093+0000 7f82acbff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a806a260 con 0x7f82a80ffbc0 2023-07-14T16:34:56.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.094+0000 7f82a6d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82a8100550 0x7f82a819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:56.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.094+0000 7f82a5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a80ffbc0 0x7f82a8193ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:56.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.094+0000 7f82a5d9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a80ffbc0 0x7f82a8193ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41848/0 (socket says 172.21.15.62:41848) 2023-07-14T16:34:56.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.094+0000 7f82a5d9b700 1 -- 172.21.15.62:0/422453439 learned_addr learned my addr 172.21.15.62:0/422453439 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:56.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.095+0000 7f82a659c700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82a80fe9d0 0x7f82a81935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:56.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.095+0000 7f82a6d9d700 1 -- 172.21.15.62:0/422453439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a80ffbc0 msgr2=0x7f82a8193ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:56.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.095+0000 7f82a6d9d700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a80ffbc0 0x7f82a8193ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.095+0000 7f82a6d9d700 1 -- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82a80fe9d0 msgr2=0x7f82a81935a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:56.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.095+0000 7f82a6d9d700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82a80fe9d0 0x7f82a81935a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.095+0000 7f82a6d9d700 1 -- 172.21.15.62:0/422453439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a000da00 con 0x7f82a8100550 2023-07-14T16:34:56.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.096+0000 7f82a6d9d700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82a8100550 0x7f82a819ab50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f829000e970 tx=0x7f829000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:56.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.097+0000 7f829f7fe700 1 -- 172.21.15.62:0/422453439 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290004220 con 0x7f82a8100550 2023-07-14T16:34:56.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.097+0000 7f829f7fe700 1 -- 172.21.15.62:0/422453439 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8290004cc0 con 0x7f82a8100550 2023-07-14T16:34:56.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.097+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a806a4e0 con 0x7f82a8100550 2023-07-14T16:34:56.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.097+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a806a9f0 con 0x7f82a8100550 2023-07-14T16:34:56.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.098+0000 7f829f7fe700 1 -- 172.21.15.62:0/422453439 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290005590 con 0x7f82a8100550 2023-07-14T16:34:56.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.099+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8288005320 con 0x7f82a8100550 2023-07-14T16:34:56.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.101+0000 7f829f7fe700 1 -- 172.21.15.62:0/422453439 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8290004380 con 0x7f82a8100550 2023-07-14T16:34:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.104+0000 7f829f7fe700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82940a1350 0x7f82940a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.104+0000 7f829f7fe700 1 -- 172.21.15.62:0/422453439 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8290014070 con 0x7f82a8100550 2023-07-14T16:34:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.105+0000 7f82a659c700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82940a1350 0x7f82940a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.106+0000 7f829f7fe700 1 -- 172.21.15.62:0/422453439 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8290096260 con 0x7f82a8100550 2023-07-14T16:34:56.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.106+0000 7f82a659c700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82940a1350 0x7f82940a3800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f82a0006010 tx=0x7f82a000f680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:56.562 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:34:56.563 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14613,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3850024319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3850024319},{"type":"v1","addr":"172.21.15.62:6835","nonce":3850024319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/1601453648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":1601453648},{"type":"v1","addr":"172.21.15.62:6837","nonce":1601453648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/194603776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":194603776},{"type":"v1","addr":"172.21.15.164:6837","nonce":194603776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14709,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3506657450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3506657450},{"type":"v1","addr":"172.21.15.62:6839","nonce":3506657450}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/33062382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":33062382},{"type":"v1","addr":"172.21.15.62:6841","nonce":33062382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/2570105256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":2570105256},{"type":"v1","addr":"172.21.15.81:6833","nonce":2570105256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T16:13:58.805235+0000","modified":"2023-07-14T16:34:43.934380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24602,"mds_1":24565,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/903698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":903698978},{"type":"v1","addr":"172.21.15.164:6833","nonce":903698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":316,"addr":"172.21.15.81:6835/337855571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":337855571},{"type":"v1","addr":"172.21.15.81:6835","nonce":337855571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6835/1558258971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":1558258971},{"type":"v1","addr":"172.21.15.164:6835","nonce":1558258971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24551":{"gid":24551,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6837/3627777030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3627777030},{"type":"v1","addr":"172.21.15.81:6837","nonce":3627777030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.164:6839/794647512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":794647512},{"type":"v1","addr":"172.21.15.164:6839","nonce":794647512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24602":{"gid":24602,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6839/2113600738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":2113600738},{"type":"v1","addr":"172.21.15.81:6839","nonce":2113600738}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T16:34:56.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.579+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8288005190 con 0x7f82a8100550 2023-07-14T16:34:56.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.584+0000 7f829f7fe700 1 -- 172.21.15.62:0/422453439 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f829001c020 con 0x7f82a8100550 2023-07-14T16:34:56.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.591+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82940a1350 msgr2=0x7f82940a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:56.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.591+0000 7f82acbff700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82940a1350 0x7f82940a3800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f82a0006010 tx=0x7f82a000f680 comp rx=0 tx=0).stop 2023-07-14T16:34:56.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.591+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82a8100550 msgr2=0x7f82a819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:56.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.591+0000 7f82acbff700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82a8100550 0x7f82a819ab50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f829000e970 tx=0x7f829000ec80 comp rx=0 tx=0).stop 2023-07-14T16:34:56.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.592+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 shutdown_connections 2023-07-14T16:34:56.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.592+0000 7f82acbff700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82940a1350 0x7f82940a3800 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.592+0000 7f82acbff700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82a8100550 0x7f82a819ab50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.592+0000 7f82acbff700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82a80fe9d0 0x7f82a81935a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.592+0000 7f82acbff700 1 --2- 172.21.15.62:0/422453439 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82a80ffbc0 0x7f82a8193ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.592+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 >> 172.21.15.62:0/422453439 conn(0x7f82a80f9fd0 msgr2=0x7f82a8103120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:56.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.592+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 shutdown_connections 2023-07-14T16:34:56.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:56.592+0000 7f82acbff700 1 -- 172.21.15.62:0/422453439 wait complete. 2023-07-14T16:34:56.568 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 36 2023-07-14T16:34:56.598 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T16:34:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.015+0000 7f016d9ff700 1 -- 172.21.15.62:0/3925358377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168071830 msgr2=0x7f016810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.015+0000 7f016d9ff700 1 --2- 172.21.15.62:0/3925358377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168071830 0x7f016810b7d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f015c009990 tx=0x7f015c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.018+0000 7f016d9ff700 1 -- 172.21.15.62:0/3925358377 shutdown_connections 2023-07-14T16:34:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.018+0000 7f016d9ff700 1 --2- 172.21.15.62:0/3925358377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f016810bd60 0x7f016810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.018+0000 7f016d9ff700 1 --2- 172.21.15.62:0/3925358377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0168070ee0 0x7f01680712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.018+0000 7f016d9ff700 1 --2- 172.21.15.62:0/3925358377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168071830 0x7f016810b7d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.018+0000 7f016d9ff700 1 -- 172.21.15.62:0/3925358377 >> 172.21.15.62:0/3925358377 conn(0x7f016806ca90 msgr2=0x7f016806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.018+0000 7f016d9ff700 1 -- 172.21.15.62:0/3925358377 shutdown_connections 2023-07-14T16:34:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.018+0000 7f016d9ff700 1 -- 172.21.15.62:0/3925358377 wait complete. 2023-07-14T16:34:56.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.021+0000 7f016d9ff700 1 Processor -- start 2023-07-14T16:34:56.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.021+0000 7f016d9ff700 1 -- start start 2023-07-14T16:34:56.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.021+0000 7f016d9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168070ee0 0x7f01681a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:56.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.021+0000 7f016d9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0168071830 0x7f01681a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:56.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.021+0000 7f016d9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f016810bd60 0x7f01681abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:56.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.021+0000 7f016d9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01681ac1f0 con 0x7f016810bd60 2023-07-14T16:34:56.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.021+0000 7f016d9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01681ac330 con 0x7f0168070ee0 2023-07-14T16:34:56.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.021+0000 7f016d9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01681ac4c0 con 0x7f0168071830 2023-07-14T16:34:56.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.022+0000 7f01667fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0168071830 0x7f01681a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:56.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.022+0000 7f01667fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0168071830 0x7f01681a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41858/0 (socket says 172.21.15.62:41858) 2023-07-14T16:34:56.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.022+0000 7f01667fc700 1 -- 172.21.15.62:0/806375152 learned_addr learned my addr 172.21.15.62:0/806375152 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:56.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.022+0000 7f01677fe700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f016810bd60 0x7f01681abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:56.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.023+0000 7f0166ffd700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168070ee0 0x7f01681a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:56.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.023+0000 7f01667fc700 1 -- 172.21.15.62:0/806375152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168070ee0 msgr2=0x7f01681a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:56.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.023+0000 7f01667fc700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168070ee0 0x7f01681a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.023+0000 7f01667fc700 1 -- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f016810bd60 msgr2=0x7f01681abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:56.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.023+0000 7f01667fc700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f016810bd60 0x7f01681abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:56.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.023+0000 7f01667fc700 1 -- 172.21.15.62:0/806375152 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0095f0 con 0x7f0168071830 2023-07-14T16:34:56.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.023+0000 7f0166ffd700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168070ee0 0x7f01681a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:34:56.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.025+0000 7f01667fc700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0168071830 0x7f01681a4bb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f015c003680 tx=0x7f015c003a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:56.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.025+0000 7f016c9fd700 1 -- 172.21.15.62:0/806375152 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c01d070 con 0x7f0168071830 2023-07-14T16:34:56.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.025+0000 7f016c9fd700 1 -- 172.21.15.62:0/806375152 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015c00bd50 con 0x7f0168071830 2023-07-14T16:34:56.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.025+0000 7f016c9fd700 1 -- 172.21.15.62:0/806375152 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c00fe80 con 0x7f0168071830 2023-07-14T16:34:56.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.025+0000 7f016d9ff700 1 -- 172.21.15.62:0/806375152 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01681ac740 con 0x7f0168071830 2023-07-14T16:34:56.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.025+0000 7f016d9ff700 1 -- 172.21.15.62:0/806375152 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01681acb90 con 0x7f0168071830 2023-07-14T16:34:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.032+0000 7f018127bb80 1 Processor -- start 2023-07-14T16:34:57.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.034+0000 7f016c9fd700 1 -- 172.21.15.62:0/806375152 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f015c00f460 con 0x7f0168071830 2023-07-14T16:34:57.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.034+0000 7f018127bb80 1 -- start start 2023-07-14T16:34:57.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f018127bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570914684d0 0x5570914688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f018127bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x5570914696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f018127bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091469c70 0x55709146a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f018127bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709136dcb0 con 0x5570914684d0 2023-07-14T16:34:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f018127bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570913c0a40 con 0x557091468e20 2023-07-14T16:34:57.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f018127bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570913b9ed0 con 0x557091469c70 2023-07-14T16:34:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f0166ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x5570914696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f0166ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x5570914696e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48686/0 (socket says 172.21.15.62:48686) 2023-07-14T16:34:57.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.035+0000 7f0166ffd700 1 -- 172.21.15.62:0/3760224980 learned_addr learned my addr 172.21.15.62:0/3760224980 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.037+0000 7f0166ffd700 1 -- 172.21.15.62:0/3760224980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091469c70 msgr2=0x55709146a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.037+0000 7f0166ffd700 1 --2- 172.21.15.62:0/3760224980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091469c70 0x55709146a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.037+0000 7f0166ffd700 1 -- 172.21.15.62:0/3760224980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570914684d0 msgr2=0x5570914688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.037+0000 7f0166ffd700 1 --2- 172.21.15.62:0/3760224980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570914684d0 0x5570914688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.037+0000 7f0166ffd700 1 -- 172.21.15.62:0/3760224980 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570911e7de0 con 0x557091468e20 2023-07-14T16:34:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.037+0000 7f0166ffd700 1 --2- 172.21.15.62:0/3760224980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x5570914696e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f01580078c0 tx=0x7f0158007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28df9fb83f600069 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.037+0000 7f016c9fd700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f01500a1250 0x7f01500a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.038+0000 7f016c9fd700 1 -- 172.21.15.62:0/806375152 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f015c0cd9c0 con 0x7f0168071830 2023-07-14T16:34:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.038+0000 7f014dffb700 1 -- 172.21.15.62:0/3760224980 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015800d670 con 0x557091468e20 2023-07-14T16:34:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.038+0000 7f014dffb700 1 -- 172.21.15.62:0/3760224980 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015800dc60 con 0x557091468e20 2023-07-14T16:34:57.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.038+0000 7f014dffb700 1 -- 172.21.15.62:0/3760224980 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158013ce0 con 0x557091468e20 2023-07-14T16:34:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.040+0000 7f01677fe700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f01500a1250 0x7f01500a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.040+0000 7f018127bb80 1 -- 172.21.15.62:0/3760224980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 msgr2=0x5570914696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.040+0000 7f018127bb80 1 --2- 172.21.15.62:0/3760224980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x5570914696e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f01580078c0 tx=0x7f0158007bd0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.040+0000 7f018127bb80 1 -- 172.21.15.62:0/3760224980 shutdown_connections 2023-07-14T16:34:57.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.040+0000 7f018127bb80 1 --2- 172.21.15.62:0/3760224980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091469c70 0x55709146a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.040+0000 7f018127bb80 1 --2- 172.21.15.62:0/3760224980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570914684d0 0x5570914688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.040+0000 7f018127bb80 1 --2- 172.21.15.62:0/3760224980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x5570914696e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.040+0000 7f018127bb80 1 -- 172.21.15.62:0/3760224980 >> 172.21.15.62:0/3760224980 conn(0x55709145dda0 msgr2=0x5570914631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.041+0000 7f018127bb80 1 -- 172.21.15.62:0/3760224980 shutdown_connections 2023-07-14T16:34:57.016 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.041+0000 7f01677fe700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f01500a1250 0x7f01500a3700 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0160009fd0 tx=0x7f016000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.041+0000 7f018127bb80 1 -- 172.21.15.62:0/3760224980 wait complete. 2023-07-14T16:34:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.042+0000 7f018127bb80 1 -- start start 2023-07-14T16:34:57.017 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.042+0000 7f018127bb80 1 Processor -- start 2023-07-14T16:34:57.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f018127bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091472230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f018127bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091468e20 0x557091472770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f018127bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 0x5570914797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.018 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f018127bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557091359ac0 con 0x557091468e20 2023-07-14T16:34:57.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f018127bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570913597c0 con 0x557091469c70 2023-07-14T16:34:57.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f018127bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709123d920 con 0x5570914684d0 2023-07-14T16:34:57.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f018127bb80 10 client.0 _ll_register_callbacks cb 0x557091357760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:57.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f0166ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091472230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f0166ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091472230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41862/0 (socket says 172.21.15.62:41862) 2023-07-14T16:34:57.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f0166ffd700 1 -- 172.21.15.62:0/1060577315 learned_addr learned my addr 172.21.15.62:0/1060577315 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.019 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.043+0000 7f01667fc700 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091468e20 0x557091472770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.044+0000 7f01677fe700 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 0x5570914797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.044+0000 7f0166ffd700 1 -- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 msgr2=0x5570914797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.044+0000 7f0166ffd700 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 0x5570914797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.044+0000 7f0166ffd700 1 -- 172.21.15.62:0/1060577315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091468e20 msgr2=0x557091472770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.044+0000 7f0166ffd700 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091468e20 0x557091472770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.044+0000 7f0166ffd700 1 -- 172.21.15.62:0/1060577315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015800e040 con 0x5570914684d0 2023-07-14T16:34:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.044+0000 7f01677fe700 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 0x5570914797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:34:57.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.044+0000 7f0166ffd700 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091472230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0158007140 tx=0x7f0158013480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=817a350376762c38 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.045+0000 7f0146ffd700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:34:57.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.045+0000 7f0146ffd700 1 -- 172.21.15.62:0/1060577315 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0158013c70 con 0x5570914684d0 2023-07-14T16:34:57.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.045+0000 7f0146ffd700 1 -- 172.21.15.62:0/1060577315 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0158013dd0 con 0x5570914684d0 2023-07-14T16:34:57.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.045+0000 7f018127bb80 1 -- 172.21.15.62:0/1060577315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570913134f0 con 0x5570914684d0 2023-07-14T16:34:57.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f0146ffd700 1 -- 172.21.15.62:0/1060577315 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f015800d670 con 0x5570914684d0 2023-07-14T16:34:57.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f018127bb80 10 client.30139 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:57.021 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f018127bb80 1 -- 172.21.15.62:0/1060577315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5570911d0c60 con 0x5570914684d0 2023-07-14T16:34:57.022 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f0146ffd700 1 -- 172.21.15.62:0/1060577315 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5570913134f0 con 0x5570914684d0 2023-07-14T16:34:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f0146ffd700 1 -- 172.21.15.62:0/1060577315 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0158031b50 con 0x5570914684d0 2023-07-14T16:34:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f018127bb80 10 client.30139 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f018127bb80 10 client.30139 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:34:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f018127bb80 20 client.30139 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f018127bb80 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55709147de10 0x557091480220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f018127bb80 4 client.30139 mds_command: new command op to 24512 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.046+0000 7f018127bb80 1 -- 172.21.15.62:0/1060577315 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570913b2eb0 con 0x55709147de10 2023-07-14T16:34:57.023 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.047+0000 7f01667fc700 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55709147de10 0x557091480220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.048+0000 7f0146ffd700 10 client.30139 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:34:57.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.048+0000 7f01667fc700 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55709147de10 0x557091480220 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf12d17dfca76d51 server_cookie=3b8d6ddb1619d456 in_seq=0 out_seq=0 2023-07-14T16:34:57.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.048+0000 7f0146ffd700 1 -- 172.21.15.62:0/1060577315 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5570913b2eb0 con 0x55709147de10 2023-07-14T16:34:57.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f0146ffd700 10 client.30139 handle_command_reply: tid=0 2023-07-14T16:34:57.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 client.30139 shutdown 2023-07-14T16:34:57.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 -- 172.21.15.62:0/1060577315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 msgr2=0x557091472230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091472230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0158007140 tx=0x7f0158013480 comp rx=0 tx=0).stop 2023-07-14T16:34:57.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 -- 172.21.15.62:0/1060577315 shutdown_connections 2023-07-14T16:34:57.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x55709147de10 0x557091480220 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 0x5570914797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091472230 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 --2- 172.21.15.62:0/1060577315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091468e20 0x557091472770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f018127bb80 1 -- 172.21.15.62:0/1060577315 >> 172.21.15.62:0/1060577315 conn(0x55709145dda0 msgr2=0x5570914628b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.025 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.049+0000 7f0146ffd700 0 client.30139 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:34:57.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.052+0000 7f018127bb80 1 -- 172.21.15.62:0/1060577315 shutdown_connections 2023-07-14T16:34:57.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.052+0000 7f018127bb80 1 -- 172.21.15.62:0/1060577315 wait complete. 2023-07-14T16:34:57.028 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.056+0000 7f018127bb80 20 client.30139 trim_cache size 0 max 16384 2023-07-14T16:34:57.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.085+0000 7f018127bb80 1 Processor -- start 2023-07-14T16:34:57.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.089+0000 7f018127bb80 1 -- start start 2023-07-14T16:34:57.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.089+0000 7f018127bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570914684d0 0x557091470130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.089+0000 7f018127bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091468e20 0x557091479770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.089+0000 7f018127bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 0x55709146ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.089+0000 7f018127bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570914640d0 con 0x5570914684d0 2023-07-14T16:34:57.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.089+0000 7f018127bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557091473420 con 0x557091469c70 2023-07-14T16:34:57.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.089+0000 7f018127bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557091463ed0 con 0x557091468e20 2023-07-14T16:34:57.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.090+0000 7f01667fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091468e20 0x557091479770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.090+0000 7f01667fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091468e20 0x557091479770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41866/0 (socket says 172.21.15.62:41866) 2023-07-14T16:34:57.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.090+0000 7f01667fc700 1 -- 172.21.15.62:0/2748105446 learned_addr learned my addr 172.21.15.62:0/2748105446 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.091+0000 7f01667fc700 1 -- 172.21.15.62:0/2748105446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 msgr2=0x55709146ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.091+0000 7f01667fc700 1 --2- 172.21.15.62:0/2748105446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 0x55709146ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.091+0000 7f01667fc700 1 -- 172.21.15.62:0/2748105446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570914684d0 msgr2=0x557091470130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.091+0000 7f01667fc700 1 --2- 172.21.15.62:0/2748105446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570914684d0 0x557091470130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.091+0000 7f01667fc700 1 -- 172.21.15.62:0/2748105446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55709120e250 con 0x557091468e20 2023-07-14T16:34:57.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.091+0000 7f01667fc700 1 --2- 172.21.15.62:0/2748105446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091468e20 0x557091479770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f015c00bec0 tx=0x7f015c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78670f6d8aedc6f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.092+0000 7f014e7fc700 1 -- 172.21.15.62:0/2748105446 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c0b9070 con 0x557091468e20 2023-07-14T16:34:57.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.092+0000 7f014e7fc700 1 -- 172.21.15.62:0/2748105446 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015c0b2070 con 0x557091468e20 2023-07-14T16:34:57.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.092+0000 7f014e7fc700 1 -- 172.21.15.62:0/2748105446 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c0a3260 con 0x557091468e20 2023-07-14T16:34:57.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.093+0000 7f018127bb80 1 -- 172.21.15.62:0/2748105446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091468e20 msgr2=0x557091479770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.093+0000 7f018127bb80 1 --2- 172.21.15.62:0/2748105446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091468e20 0x557091479770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f015c00bec0 tx=0x7f015c000bc0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.093+0000 7f018127bb80 1 -- 172.21.15.62:0/2748105446 shutdown_connections 2023-07-14T16:34:57.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.093+0000 7f018127bb80 1 --2- 172.21.15.62:0/2748105446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091469c70 0x55709146ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.093+0000 7f018127bb80 1 --2- 172.21.15.62:0/2748105446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5570914684d0 0x557091470130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.093+0000 7f018127bb80 1 --2- 172.21.15.62:0/2748105446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557091468e20 0x557091479770 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.093+0000 7f018127bb80 1 -- 172.21.15.62:0/2748105446 >> 172.21.15.62:0/2748105446 conn(0x55709145dda0 msgr2=0x557091470e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.093+0000 7f018127bb80 1 -- 172.21.15.62:0/2748105446 shutdown_connections 2023-07-14T16:34:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.094+0000 7f018127bb80 1 -- 172.21.15.62:0/2748105446 wait complete. 2023-07-14T16:34:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.094+0000 7f018127bb80 1 -- start start 2023-07-14T16:34:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.095+0000 7f018127bb80 1 Processor -- start 2023-07-14T16:34:57.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.095+0000 7f018127bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091479260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.095+0000 7f018127bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x557091477870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.095+0000 7f018127bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091469c70 0x557091477db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f018127bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557091479a80 con 0x557091469c70 2023-07-14T16:34:57.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f018127bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570914782f0 con 0x557091468e20 2023-07-14T16:34:57.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f018127bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557091478430 con 0x5570914684d0 2023-07-14T16:34:57.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f018127bb80 10 client.0 _ll_register_callbacks cb 0x7f0158007710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:57.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f0166ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091479260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f0166ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091479260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41880/0 (socket says 172.21.15.62:41880) 2023-07-14T16:34:57.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f0166ffd700 1 -- 172.21.15.62:0/3471784530 learned_addr learned my addr 172.21.15.62:0/3471784530 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f01677fe700 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091469c70 0x557091477db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.096+0000 7f01667fc700 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x557091477870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.097+0000 7f01667fc700 1 -- 172.21.15.62:0/3471784530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 msgr2=0x557091479260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.097+0000 7f01667fc700 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091479260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.097+0000 7f01667fc700 1 -- 172.21.15.62:0/3471784530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091469c70 msgr2=0x557091477db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.097+0000 7f01667fc700 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091469c70 0x557091477db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.097+0000 7f01667fc700 1 -- 172.21.15.62:0/3471784530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01681acb90 con 0x557091468e20 2023-07-14T16:34:57.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.100+0000 7f01667fc700 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x557091477870 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f015c09ab10 tx=0x7f015c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=175cf45bd1ea1036 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.100+0000 7f0146ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:57.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.100+0000 7f0146ffd700 1 -- 172.21.15.62:0/3471784530 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c0b9040 con 0x557091468e20 2023-07-14T16:34:57.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.101+0000 7f0146ffd700 1 -- 172.21.15.62:0/3471784530 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f015c00f710 con 0x557091468e20 2023-07-14T16:34:57.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.101+0000 7f018127bb80 1 -- 172.21.15.62:0/3471784530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f015800dc60 con 0x557091468e20 2023-07-14T16:34:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.101+0000 7f0146ffd700 1 -- 172.21.15.62:0/3471784530 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f015c0cb1a0 con 0x557091468e20 2023-07-14T16:34:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.101+0000 7f018127bb80 10 client.30260 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.102+0000 7f018127bb80 1 -- 172.21.15.62:0/3471784530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5570911e62b0 con 0x557091468e20 2023-07-14T16:34:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.102+0000 7f0146ffd700 1 -- 172.21.15.62:0/3471784530 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015800dc60 con 0x557091468e20 2023-07-14T16:34:57.075 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.102+0000 7f0146ffd700 1 -- 172.21.15.62:0/3471784530 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f015c0a2db0 con 0x557091468e20 2023-07-14T16:34:57.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.104+0000 7f018127bb80 10 client.30260 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:57.078 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.104+0000 7f018127bb80 10 client.30260 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2023-07-14T16:34:57.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.104+0000 7f018127bb80 20 client.30260 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:57.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.104+0000 7f018127bb80 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5570914e3d40 0x5570914e6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.104+0000 7f018127bb80 4 client.30260 mds_command: new command op to 24512 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:34:57.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.104+0000 7f018127bb80 1 -- 172.21.15.62:0/3471784530 --> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557091375050 con 0x5570914e3d40 2023-07-14T16:34:57.079 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.105+0000 7f0166ffd700 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5570914e3d40 0x5570914e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.106+0000 7f0146ffd700 10 client.30260 ms_handle_connect on v2:172.21.15.81:6834/337855571 2023-07-14T16:34:57.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.106+0000 7f0166ffd700 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5570914e3d40 0x5570914e6150 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=921a0b658f5e1a0 server_cookie=26514ceb2ddd2788 in_seq=0 out_seq=0 2023-07-14T16:34:57.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.107+0000 7f0146ffd700 1 -- 172.21.15.62:0/3471784530 <== mds.2 v2:172.21.15.81:6834/337855571 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+9145 (crc 0 0 0) 0x557091375050 con 0x5570914e3d40 2023-07-14T16:34:57.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.107+0000 7f0146ffd700 10 client.30260 handle_command_reply: tid=0 2023-07-14T16:34:57.080 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.107+0000 7f018127bb80 1 client.30260 shutdown 2023-07-14T16:34:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.108+0000 7f018127bb80 1 -- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 msgr2=0x557091477870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.108+0000 7f018127bb80 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x557091477870 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f015c09ab10 tx=0x7f015c005c90 comp rx=0 tx=0).stop 2023-07-14T16:34:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.109+0000 7f018127bb80 1 -- 172.21.15.62:0/3471784530 shutdown_connections 2023-07-14T16:34:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.109+0000 7f018127bb80 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:6834/337855571,v1:172.21.15.81:6835/337855571] conn(0x5570914e3d40 0x5570914e6150 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.109+0000 7f018127bb80 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557091469c70 0x557091477db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.109+0000 7f018127bb80 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5570914684d0 0x557091479260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.109+0000 7f018127bb80 1 --2- 172.21.15.62:0/3471784530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557091468e20 0x557091477870 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.109+0000 7f018127bb80 1 -- 172.21.15.62:0/3471784530 >> 172.21.15.62:0/3471784530 conn(0x55709145dda0 msgr2=0x557091470130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.082 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.109+0000 7f0146ffd700 0 client.30260 ms_handle_reset on v2:172.21.15.81:6834/337855571 2023-07-14T16:34:57.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.109+0000 7f018127bb80 1 -- 172.21.15.62:0/3471784530 shutdown_connections 2023-07-14T16:34:57.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.110+0000 7f018127bb80 1 -- 172.21.15.62:0/3471784530 wait complete. 2023-07-14T16:34:57.083 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.111+0000 7f018127bb80 20 client.30260 trim_cache size 0 max 16384 2023-07-14T16:34:57.084 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2023-07-14T16:34:57.084 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.084 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.085 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 29144257, 2023-07-14T16:34:57.085 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256067, 2023-07-14T16:34:57.085 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fc.Po" 2023-07-14T16:34:57.085 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.085 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.086 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.086 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 268903644, 2023-07-14T16:34:57.086 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628995, 2023-07-14T16:34:57.087 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rwlock.o" 2023-07-14T16:34:57.087 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.087 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.087 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.087 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 294017427, 2023-07-14T16:34:57.087 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256064, 2023-07-14T16:34:57.087 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_op.Po" 2023-07-14T16:34:57.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.088 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.088 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.088 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 332135477, 2023-07-14T16:34:57.088 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256054, 2023-07-14T16:34:57.088 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/main.Po" 2023-07-14T16:34:57.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.089 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.089 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.089 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 467240244, 2023-07-14T16:34:57.089 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629000, 2023-07-14T16:34:57.089 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_thread.o" 2023-07-14T16:34:57.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.089 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.089 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.090 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 503356804, 2023-07-14T16:34:57.090 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256066, 2023-07-14T16:34:57.090 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/parser.Po" 2023-07-14T16:34:57.090 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.090 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.090 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.090 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 518823357, 2023-07-14T16:34:57.091 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256062, 2023-07-14T16:34:57.091 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fs.Po" 2023-07-14T16:34:57.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.091 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.091 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.091 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 545345685, 2023-07-14T16:34:57.091 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628987, 2023-07-14T16:34:57.091 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h" 2023-07-14T16:34:57.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.092 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.092 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.092 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 547529498, 2023-07-14T16:34:57.092 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256059, 2023-07-14T16:34:57.092 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/cirlist.Po" 2023-07-14T16:34:57.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.093 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.093 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.093 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 662921452, 2023-07-14T16:34:57.093 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628998, 2023-07-14T16:34:57.093 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_tg.o" 2023-07-14T16:34:57.093 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.093 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.093 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.094 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 665414258, 2023-07-14T16:34:57.094 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628999, 2023-07-14T16:34:57.094 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fs.o" 2023-07-14T16:34:57.094 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.094 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.094 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.094 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 677933795, 2023-07-14T16:34:57.095 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256060, 2023-07-14T16:34:57.095 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rbt.Po" 2023-07-14T16:34:57.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.095 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.095 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.095 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1024359789, 2023-07-14T16:34:57.095 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256068, 2023-07-14T16:34:57.095 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_stats.Po" 2023-07-14T16:34:57.096 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.096 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.096 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.096 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1355940457, 2023-07-14T16:34:57.096 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628991, 2023-07-14T16:34:57.096 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/main.o" 2023-07-14T16:34:57.096 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.096 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.097 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.097 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1374977809, 2023-07-14T16:34:57.097 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628992, 2023-07-14T16:34:57.097 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fh.o" 2023-07-14T16:34:57.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.097 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.097 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.097 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1501417065, 2023-07-14T16:34:57.098 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629002, 2023-07-14T16:34:57.098 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/util.o" 2023-07-14T16:34:57.098 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.098 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.098 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.098 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1875471956, 2023-07-14T16:34:57.098 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629001, 2023-07-14T16:34:57.098 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_op.o" 2023-07-14T16:34:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2036491788, 2023-07-14T16:34:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628978, 2023-07-14T16:34:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/Makefile" 2023-07-14T16:34:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.099 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.100 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.100 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2167197996, 2023-07-14T16:34:57.100 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628477, 2023-07-14T16:34:57.100 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in~" 2023-07-14T16:34:57.100 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.100 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.100 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.100 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2326319758, 2023-07-14T16:34:57.101 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629005, 2023-07-14T16:34:57.101 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_stats.o" 2023-07-14T16:34:57.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.102 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.102 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.102 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2384713424, 2023-07-14T16:34:57.102 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629004, 2023-07-14T16:34:57.102 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fc.o" 2023-07-14T16:34:57.102 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.103 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.103 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.103 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2448116865, 2023-07-14T16:34:57.103 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256058, 2023-07-14T16:34:57.103 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rwlock.Po" 2023-07-14T16:34:57.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.103 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.103 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.104 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2913004853, 2023-07-14T16:34:57.104 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256057, 2023-07-14T16:34:57.104 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/metaops.Po" 2023-07-14T16:34:57.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.104 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.104 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.105 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3030855656, 2023-07-14T16:34:57.105 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628994, 2023-07-14T16:34:57.105 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/metaops.o" 2023-07-14T16:34:57.105 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.105 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.105 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.105 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3039845192, 2023-07-14T16:34:57.106 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256061, 2023-07-14T16:34:57.106 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_tg.Po" 2023-07-14T16:34:57.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.106 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.106 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.106 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3080609784, 2023-07-14T16:34:57.106 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628993, 2023-07-14T16:34:57.106 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/filelist.o" 2023-07-14T16:34:57.107 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.107 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.107 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.107 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3125677993, 2023-07-14T16:34:57.107 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256065, 2023-07-14T16:34:57.107 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/util.Po" 2023-07-14T16:34:57.107 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.107 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.108 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.108 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3214309731, 2023-07-14T16:34:57.108 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628988, 2023-07-14T16:34:57.108 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/stamp-h1" 2023-07-14T16:34:57.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.108 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.108 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.108 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3240159515, 2023-07-14T16:34:57.109 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023255552, 2023-07-14T16:34:57.109 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fileops.Po" 2023-07-14T16:34:57.109 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.109 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.109 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.109 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3298915533, 2023-07-14T16:34:57.109 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629003, 2023-07-14T16:34:57.109 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/parser.o" 2023-07-14T16:34:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3418078762, 2023-07-14T16:34:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023255553, 2023-07-14T16:34:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rand.Po" 2023-07-14T16:34:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.110 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.111 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.111 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3595090028, 2023-07-14T16:34:57.111 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629007, 2023-07-14T16:34:57.111 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb" 2023-07-14T16:34:57.111 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.112 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.112 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.112 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3620380938, 2023-07-14T16:34:57.112 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628990, 2023-07-14T16:34:57.112 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rand.o" 2023-07-14T16:34:57.112 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.112 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.112 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.113 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3622665720, 2023-07-14T16:34:57.113 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628996, 2023-07-14T16:34:57.113 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/cirlist.o" 2023-07-14T16:34:57.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.113 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.113 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.113 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3697428185, 2023-07-14T16:34:57.113 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629008, 2023-07-14T16:34:57.114 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/tmp" 2023-07-14T16:34:57.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.114 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.114 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.114 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3745515000, 2023-07-14T16:34:57.114 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628989, 2023-07-14T16:34:57.114 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fileops.o" 2023-07-14T16:34:57.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.115 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.115 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.115 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3747423216, 2023-07-14T16:34:57.115 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628980, 2023-07-14T16:34:57.115 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in" 2023-07-14T16:34:57.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.115 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.116 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.116 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3773069627, 2023-07-14T16:34:57.116 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256069, 2023-07-14T16:34:57.116 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/list.Po" 2023-07-14T16:34:57.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.116 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.116 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.116 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 3815028410, 2023-07-14T16:34:57.117 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256056, 2023-07-14T16:34:57.117 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/filelist.Po" 2023-07-14T16:34:57.117 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.117 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.117 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.117 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4004922964, 2023-07-14T16:34:57.117 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511629006, 2023-07-14T16:34:57.117 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/list.o" 2023-07-14T16:34:57.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.118 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.118 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.118 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4170503966, 2023-07-14T16:34:57.118 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511628997, 2023-07-14T16:34:57.118 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rbt.o" 2023-07-14T16:34:57.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.118 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.119 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.119 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4184007015, 2023-07-14T16:34:57.119 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256063, 2023-07-14T16:34:57.119 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_thread.Po" 2023-07-14T16:34:57.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:34:57.119 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:34:57.119 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2023-07-14T16:34:57.120 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4232185085, 2023-07-14T16:34:57.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2199023256055, 2023-07-14T16:34:57.120 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fh.Po" 2023-07-14T16:34:57.120 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:34:57.120 INFO:teuthology.orchestra.run.smithi062.stdout:] 2023-07-14T16:34:57.121 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 -- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f01500a1250 msgr2=0x7f01500a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f01500a1250 0x7f01500a3700 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0160009fd0 tx=0x7f016000d040 comp rx=0 tx=0).stop 2023-07-14T16:34:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 -- 172.21.15.62:0/806375152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0168071830 msgr2=0x7f01681a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0168071830 0x7f01681a4bb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f015c003680 tx=0x7f015c003a80 comp rx=0 tx=0).stop 2023-07-14T16:34:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 -- 172.21.15.62:0/806375152 shutdown_connections 2023-07-14T16:34:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f016810bd60 0x7f01681abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f01500a1250 0x7f01500a3700 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0168070ee0 0x7f01681a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.122 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 --2- 172.21.15.62:0/806375152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0168071830 0x7f01681a4bb0 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f015c003680 tx=0x7f015c003a80 comp rx=0 tx=0).stop 2023-07-14T16:34:57.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 -- 172.21.15.62:0/806375152 >> 172.21.15.62:0/806375152 conn(0x7f016806ca90 msgr2=0x7f016806dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.112+0000 7f016d9ff700 1 -- 172.21.15.62:0/806375152 shutdown_connections 2023-07-14T16:34:57.123 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.113+0000 7f016d9ff700 1 -- 172.21.15.62:0/806375152 wait complete. 2023-07-14T16:34:57.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T16:34:57.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:57 smithi062 ceph-mon[95543]: pgmap v936: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2023-07-14T16:34:57.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:57 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/422453439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:34:57.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:57 smithi164 ceph-mon[97427]: pgmap v936: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2023-07-14T16:34:57.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:57 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/422453439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:34:57.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:57 smithi081 ceph-mon[127795]: pgmap v936: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2023-07-14T16:34:57.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:57 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/422453439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T16:34:57.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.510+0000 7f4328bff700 1 -- 172.21.15.62:0/3242878322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43240feac0 msgr2=0x7f43240fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.510+0000 7f4328bff700 1 --2- 172.21.15.62:0/3242878322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43240feac0 0x7f43240fef30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4314009f90 tx=0x7f431400b330 comp rx=0 tx=0).stop 2023-07-14T16:34:57.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.511+0000 7f4328bff700 1 -- 172.21.15.62:0/3242878322 shutdown_connections 2023-07-14T16:34:57.484 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.511+0000 7f4328bff700 1 --2- 172.21.15.62:0/3242878322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43240ff470 0x7f4324103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.511+0000 7f4328bff700 1 --2- 172.21.15.62:0/3242878322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43240fd990 0x7f43240fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.511+0000 7f4328bff700 1 --2- 172.21.15.62:0/3242878322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43240feac0 0x7f43240fef30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.511+0000 7f4328bff700 1 -- 172.21.15.62:0/3242878322 >> 172.21.15.62:0/3242878322 conn(0x7f43240f9340 msgr2=0x7f43240fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.511+0000 7f4328bff700 1 -- 172.21.15.62:0/3242878322 shutdown_connections 2023-07-14T16:34:57.485 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.511+0000 7f4328bff700 1 -- 172.21.15.62:0/3242878322 wait complete. 2023-07-14T16:34:57.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.513+0000 7f4328bff700 1 Processor -- start 2023-07-14T16:34:57.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.513+0000 7f4328bff700 1 -- start start 2023-07-14T16:34:57.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.513+0000 7f4328bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43240fd990 0x7f432410a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.486 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.513+0000 7f4328bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43240feac0 0x7f432410ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.513+0000 7f4328bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43240ff470 0x7f43241abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.513+0000 7f4328bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432410b290 con 0x7f43240fd990 2023-07-14T16:34:57.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.514+0000 7f4328bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324069f30 con 0x7f43240ff470 2023-07-14T16:34:57.487 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.514+0000 7f4328bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432406a0f0 con 0x7f43240feac0 2023-07-14T16:34:57.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.514+0000 7f4322d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43240ff470 0x7f43241abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.514+0000 7f4322d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43240ff470 0x7f43241abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48732/0 (socket says 172.21.15.62:48732) 2023-07-14T16:34:57.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.514+0000 7f4322d9d700 1 -- 172.21.15.62:0/1767138664 learned_addr learned my addr 172.21.15.62:0/1767138664 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.488 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.514+0000 7f4319d9b700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43240feac0 0x7f432410ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.515+0000 7f432259c700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43240fd990 0x7f432410a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.515+0000 7f4322d9d700 1 -- 172.21.15.62:0/1767138664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43240feac0 msgr2=0x7f432410ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.515+0000 7f4322d9d700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43240feac0 0x7f432410ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.515+0000 7f4322d9d700 1 -- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43240fd990 msgr2=0x7f432410a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.515+0000 7f4322d9d700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43240fd990 0x7f432410a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.515+0000 7f4322d9d700 1 -- 172.21.15.62:0/1767138664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009c40 con 0x7f43240ff470 2023-07-14T16:34:57.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.515+0000 7f432259c700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43240fd990 0x7f432410a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:34:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.515+0000 7f4322d9d700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43240ff470 0x7f43241abfb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f431c00b550 tx=0x7f431c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.517+0000 7f431bfff700 1 -- 172.21.15.62:0/1767138664 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c011830 con 0x7f43240ff470 2023-07-14T16:34:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.517+0000 7f431bfff700 1 -- 172.21.15.62:0/1767138664 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f431c011e20 con 0x7f43240ff470 2023-07-14T16:34:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.517+0000 7f431bfff700 1 -- 172.21.15.62:0/1767138664 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c010da0 con 0x7f43240ff470 2023-07-14T16:34:57.491 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.518+0000 7f4328bff700 1 -- 172.21.15.62:0/1767138664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432406a3d0 con 0x7f43240ff470 2023-07-14T16:34:57.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.518+0000 7f4328bff700 1 -- 172.21.15.62:0/1767138664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432406a890 con 0x7f43240ff470 2023-07-14T16:34:57.500 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.526+0000 7f431bfff700 1 -- 172.21.15.62:0/1767138664 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f431c011990 con 0x7f43240ff470 2023-07-14T16:34:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.532+0000 7f433c5a2b80 1 Processor -- start 2023-07-14T16:34:57.505 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.532+0000 7f431bfff700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43100a1350 0x7f43100a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.532+0000 7f431bfff700 1 -- 172.21.15.62:0/1767138664 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f431c0cbfb0 con 0x7f43240ff470 2023-07-14T16:34:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f433c5a2b80 1 -- start start 2023-07-14T16:34:57.506 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa93e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 0x563faa93f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa9400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa843cb0 con 0x563faa93e4d0 2023-07-14T16:34:57.507 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa896a40 con 0x563faa93ee20 2023-07-14T16:34:57.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa88fed0 con 0x563faa93fc70 2023-07-14T16:34:57.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f4319d9b700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43100a1350 0x7f43100a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f432259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa93e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f432259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa93e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49598/0 (socket says 172.21.15.62:49598) 2023-07-14T16:34:57.508 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.533+0000 7f432259c700 1 -- 172.21.15.62:0/2971332650 learned_addr learned my addr 172.21.15.62:0/2971332650 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.534+0000 7f4322d9d700 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa9400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.534+0000 7f432259c700 1 -- 172.21.15.62:0/2971332650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 msgr2=0x563faa9400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.534+0000 7f432259c700 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa9400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.534+0000 7f432259c700 1 -- 172.21.15.62:0/2971332650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 msgr2=0x563faa93f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.509 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.534+0000 7f432259c700 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 0x563faa93f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.534+0000 7f432259c700 1 -- 172.21.15.62:0/2971332650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563faa6bdde0 con 0x563faa93e4d0 2023-07-14T16:34:57.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.534+0000 7f4322d9d700 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa9400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:34:57.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.534+0000 7f432259c700 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa93e8e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f430c009cc0 tx=0x7f430c010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4c4366408a62c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.539+0000 7f4319d9b700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43100a1350 0x7f43100a3800 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f431400b6c0 tx=0x7f43140093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.542+0000 7f4318d99700 1 -- 172.21.15.62:0/2971332650 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c010c70 con 0x563faa93e4d0 2023-07-14T16:34:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.542+0000 7f4318d99700 1 -- 172.21.15.62:0/2971332650 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f430c0044c0 con 0x563faa93e4d0 2023-07-14T16:34:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.542+0000 7f4318d99700 1 -- 172.21.15.62:0/2971332650 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c003e50 con 0x563faa93e4d0 2023-07-14T16:34:57.515 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.543+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2971332650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 msgr2=0x563faa93e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.543+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa93e8e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f430c009cc0 tx=0x7f430c010370 comp rx=0 tx=0).stop 2023-07-14T16:34:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.544+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2971332650 shutdown_connections 2023-07-14T16:34:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.544+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa9400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.544+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa93e8e0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.518 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.544+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2971332650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 0x563faa93f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.544+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2971332650 >> 172.21.15.62:0/2971332650 conn(0x563faa933da0 msgr2=0x563faa9391d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.544+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2971332650 shutdown_connections 2023-07-14T16:34:57.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.544+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2971332650 wait complete. 2023-07-14T16:34:57.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.545+0000 7f433c5a2b80 1 -- start start 2023-07-14T16:34:57.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.545+0000 7f433c5a2b80 1 Processor -- start 2023-07-14T16:34:57.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.546+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93e4d0 0x563faa9481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.546+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93ee20 0x563faa9486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.546+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93fc70 0x563faa94f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.546+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa82fac0 con 0x563faa93fc70 2023-07-14T16:34:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.546+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa82f7c0 con 0x563faa93e4d0 2023-07-14T16:34:57.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.546+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa713920 con 0x563faa93ee20 2023-07-14T16:34:57.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.546+0000 7f433c5a2b80 10 client.0 _ll_register_callbacks cb 0x563faa82d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:57.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.548+0000 7f4322d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93fc70 0x563faa94f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.548+0000 7f4322d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93fc70 0x563faa94f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49614/0 (socket says 172.21.15.62:49614) 2023-07-14T16:34:57.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.548+0000 7f4322d9d700 1 -- 172.21.15.62:0/3952789948 learned_addr learned my addr 172.21.15.62:0/3952789948 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.525 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.548+0000 7f4319d9b700 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93ee20 0x563faa9486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.548+0000 7f432259c700 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93e4d0 0x563faa9481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.549+0000 7f4322d9d700 1 -- 172.21.15.62:0/3952789948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93ee20 msgr2=0x563faa9486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.549+0000 7f4322d9d700 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93ee20 0x563faa9486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.549+0000 7f4322d9d700 1 -- 172.21.15.62:0/3952789948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93e4d0 msgr2=0x563faa9481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.549+0000 7f4322d9d700 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93e4d0 0x563faa9481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.549+0000 7f4322d9d700 1 -- 172.21.15.62:0/3952789948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c0098f0 con 0x563faa93fc70 2023-07-14T16:34:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.550+0000 7f4322d9d700 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93fc70 0x563faa94f720 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f431c099300 tx=0x7f431c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbc6e41a4e76b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.550+0000 7f4309ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:34:57.527 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.550+0000 7f4309ffb700 1 -- 172.21.15.62:0/3952789948 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c099370 con 0x563faa93fc70 2023-07-14T16:34:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.550+0000 7f4309ffb700 1 -- 172.21.15.62:0/3952789948 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f431c078090 con 0x563faa93fc70 2023-07-14T16:34:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.550+0000 7f433c5a2b80 1 -- 172.21.15.62:0/3952789948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563faa7e94f0 con 0x563faa93fc70 2023-07-14T16:34:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.551+0000 7f4309ffb700 1 -- 172.21.15.62:0/3952789948 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f431c09d3a0 con 0x563faa93fc70 2023-07-14T16:34:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.551+0000 7f433c5a2b80 10 client.20703 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.551+0000 7f433c5a2b80 1 -- 172.21.15.62:0/3952789948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563faa6a6c60 con 0x563faa93fc70 2023-07-14T16:34:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.552+0000 7f4309ffb700 1 -- 172.21.15.62:0/3952789948 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563faa7e94f0 con 0x563faa93fc70 2023-07-14T16:34:57.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.552+0000 7f4309ffb700 1 -- 172.21.15.62:0/3952789948 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f431c098910 con 0x563faa93fc70 2023-07-14T16:34:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.552+0000 7f433c5a2b80 10 client.20703 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.552+0000 7f433c5a2b80 10 client.20703 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:34:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.552+0000 7f433c5a2b80 20 client.20703 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.552+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563faa953e60 0x563faa956270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.552+0000 7f433c5a2b80 4 client.20703 mds_command: new command op to 24565 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.552+0000 7f433c5a2b80 1 -- 172.21.15.62:0/3952789948 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563faa888eb0 con 0x563faa953e60 2023-07-14T16:34:57.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.553+0000 7f432259c700 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563faa953e60 0x563faa956270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.554+0000 7f432259c700 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563faa953e60 0x563faa956270 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96b506a132baf551 server_cookie=c47fe27c3de045e4 in_seq=0 out_seq=0 2023-07-14T16:34:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.554+0000 7f4309ffb700 10 client.20703 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:34:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.554+0000 7f4309ffb700 1 -- 172.21.15.62:0/3952789948 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563faa888eb0 con 0x563faa953e60 2023-07-14T16:34:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.554+0000 7f4309ffb700 10 client.20703 handle_command_reply: tid=0 2023-07-14T16:34:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 client.20703 shutdown 2023-07-14T16:34:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 -- 172.21.15.62:0/3952789948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93fc70 msgr2=0x563faa94f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93fc70 0x563faa94f720 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f431c099300 tx=0x7f431c004750 comp rx=0 tx=0).stop 2023-07-14T16:34:57.531 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 -- 172.21.15.62:0/3952789948 shutdown_connections 2023-07-14T16:34:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563faa953e60 0x563faa956270 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93fc70 0x563faa94f720 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93e4d0 0x563faa9481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/3952789948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93ee20 0x563faa9486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f4309ffb700 0 client.20703 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:34:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.555+0000 7f433c5a2b80 1 -- 172.21.15.62:0/3952789948 >> 172.21.15.62:0/3952789948 conn(0x563faa933da0 msgr2=0x563faa9388b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.556+0000 7f433c5a2b80 1 -- 172.21.15.62:0/3952789948 shutdown_connections 2023-07-14T16:34:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.556+0000 7f433c5a2b80 1 -- 172.21.15.62:0/3952789948 wait complete. 2023-07-14T16:34:57.533 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.556+0000 7f433c5a2b80 20 client.20703 trim_cache size 0 max 16384 2023-07-14T16:34:57.550 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.578+0000 7f433c5a2b80 1 Processor -- start 2023-07-14T16:34:57.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.578+0000 7f433c5a2b80 1 -- start start 2023-07-14T16:34:57.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.578+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93e4d0 0x563faa950010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.578+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93ee20 0x563faa94f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.578+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93fc70 0x563faa942b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.578+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa947650 con 0x563faa93ee20 2023-07-14T16:34:57.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.578+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa953c80 con 0x563faa93fc70 2023-07-14T16:34:57.552 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.578+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa947d10 con 0x563faa93e4d0 2023-07-14T16:34:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.579+0000 7f432259c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93e4d0 0x563faa950010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.579+0000 7f4322d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93fc70 0x563faa942b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.579+0000 7f4322d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93fc70 0x563faa942b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48754/0 (socket says 172.21.15.62:48754) 2023-07-14T16:34:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.579+0000 7f4322d9d700 1 -- 172.21.15.62:0/1967460305 learned_addr learned my addr 172.21.15.62:0/1967460305 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.579+0000 7f4319d9b700 1 --2- 172.21.15.62:0/1967460305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93ee20 0x563faa94f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.580+0000 7f4322d9d700 1 -- 172.21.15.62:0/1967460305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93e4d0 msgr2=0x563faa950010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.553 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.580+0000 7f4322d9d700 1 --2- 172.21.15.62:0/1967460305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93e4d0 0x563faa950010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.580+0000 7f4322d9d700 1 -- 172.21.15.62:0/1967460305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93ee20 msgr2=0x563faa94f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.580+0000 7f4322d9d700 1 --2- 172.21.15.62:0/1967460305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93ee20 0x563faa94f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.580+0000 7f4322d9d700 1 -- 172.21.15.62:0/1967460305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563faa6e4250 con 0x563faa93fc70 2023-07-14T16:34:57.554 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.581+0000 7f4322d9d700 1 --2- 172.21.15.62:0/1967460305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93fc70 0x563faa942b70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f431c00bd10 tx=0x7f431c094f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cccd9fdbec412c29 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.581+0000 7f431959a700 1 -- 172.21.15.62:0/1967460305 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c010f00 con 0x563faa93fc70 2023-07-14T16:34:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.581+0000 7f431959a700 1 -- 172.21.15.62:0/1967460305 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f431c00d040 con 0x563faa93fc70 2023-07-14T16:34:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.581+0000 7f431959a700 1 -- 172.21.15.62:0/1967460305 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0982d0 con 0x563faa93fc70 2023-07-14T16:34:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.582+0000 7f433c5a2b80 1 -- 172.21.15.62:0/1967460305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93fc70 msgr2=0x563faa942b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.582+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/1967460305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93fc70 0x563faa942b70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f431c00bd10 tx=0x7f431c094f90 comp rx=0 tx=0).stop 2023-07-14T16:34:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.582+0000 7f433c5a2b80 1 -- 172.21.15.62:0/1967460305 shutdown_connections 2023-07-14T16:34:57.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.582+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/1967460305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93fc70 0x563faa942b70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.582+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/1967460305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93e4d0 0x563faa950010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.582+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/1967460305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93ee20 0x563faa94f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.582+0000 7f433c5a2b80 1 -- 172.21.15.62:0/1967460305 >> 172.21.15.62:0/1967460305 conn(0x563faa933da0 msgr2=0x563faa946e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.583+0000 7f433c5a2b80 1 -- 172.21.15.62:0/1967460305 shutdown_connections 2023-07-14T16:34:57.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.583+0000 7f433c5a2b80 1 -- 172.21.15.62:0/1967460305 wait complete. 2023-07-14T16:34:57.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.583+0000 7f433c5a2b80 1 -- start start 2023-07-14T16:34:57.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.584+0000 7f433c5a2b80 1 Processor -- start 2023-07-14T16:34:57.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.584+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa94b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.557 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.584+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 0x563faa94bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.584+0000 7f433c5a2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa94ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.584+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa94f340 con 0x563faa93e4d0 2023-07-14T16:34:57.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.584+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa94f4b0 con 0x563faa93ee20 2023-07-14T16:34:57.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.584+0000 7f433c5a2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x563faa94f670 con 0x563faa93fc70 2023-07-14T16:34:57.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.584+0000 7f433c5a2b80 10 client.0 _ll_register_callbacks cb 0x563faa82d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:57.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.585+0000 7f432259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa94b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.558 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.585+0000 7f432259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa94b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49638/0 (socket says 172.21.15.62:49638) 2023-07-14T16:34:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.585+0000 7f432259c700 1 -- 172.21.15.62:0/2859124319 learned_addr learned my addr 172.21.15.62:0/2859124319 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.586+0000 7f4319d9b700 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 0x563faa94bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.586+0000 7f4322d9d700 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa94ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.586+0000 7f432259c700 1 -- 172.21.15.62:0/2859124319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 msgr2=0x563faa94ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.586+0000 7f432259c700 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa94ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.586+0000 7f432259c700 1 -- 172.21.15.62:0/2859124319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 msgr2=0x563faa94bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.586+0000 7f432259c700 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 0x563faa94bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.586+0000 7f432259c700 1 -- 172.21.15.62:0/2859124319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563faa6a6c60 con 0x563faa93e4d0 2023-07-14T16:34:57.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.586+0000 7f432259c700 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa94b8a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f430c0093f0 tx=0x7f430c028630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d097fbf011de09d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.560 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.587+0000 7f4309ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:34:57.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.587+0000 7f4309ffb700 1 -- 172.21.15.62:0/2859124319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c010e40 con 0x563faa93e4d0 2023-07-14T16:34:57.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.587+0000 7f4309ffb700 1 -- 172.21.15.62:0/2859124319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f430c016070 con 0x563faa93e4d0 2023-07-14T16:34:57.561 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.587+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2859124319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563faa84b050 con 0x563faa93e4d0 2023-07-14T16:34:57.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.590+0000 7f4309ffb700 1 -- 172.21.15.62:0/2859124319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f430c028a60 con 0x563faa93e4d0 2023-07-14T16:34:57.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.591+0000 7f433c5a2b80 10 client.20709 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:57.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.591+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2859124319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563faa6bc2b0 con 0x563faa93e4d0 2023-07-14T16:34:57.563 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.591+0000 7f4309ffb700 1 -- 172.21.15.62:0/2859124319 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563faa84b050 con 0x563faa93e4d0 2023-07-14T16:34:57.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.591+0000 7f4309ffb700 1 -- 172.21.15.62:0/2859124319 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f430c010490 con 0x563faa93e4d0 2023-07-14T16:34:57.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.592+0000 7f433c5a2b80 10 client.20709 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:57.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.592+0000 7f433c5a2b80 10 client.20709 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-14T16:34:57.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.592+0000 7f433c5a2b80 20 client.20709 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:57.564 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.592+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563faa9a9060 0x563faa9ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.592+0000 7f433c5a2b80 4 client.20709 mds_command: new command op to 24565 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:34:57.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.592+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2859124319 --> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563faa6def70 con 0x563faa9a9060 2023-07-14T16:34:57.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.592+0000 7f4319d9b700 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563faa9a9060 0x563faa9ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.593+0000 7f4309ffb700 10 client.20709 ms_handle_connect on v2:172.21.15.164:6838/794647512 2023-07-14T16:34:57.565 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.593+0000 7f4319d9b700 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563faa9a9060 0x563faa9ab470 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bf1e311db1e3785 server_cookie=374e31bd000ed3a9 in_seq=0 out_seq=0 2023-07-14T16:34:57.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.593+0000 7f4309ffb700 1 -- 172.21.15.62:0/2859124319 <== mds.1 v2:172.21.15.164:6838/794647512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563faa6def70 con 0x563faa9a9060 2023-07-14T16:34:57.566 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.593+0000 7f4309ffb700 10 client.20709 handle_command_reply: tid=0 2023-07-14T16:34:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.593+0000 7f433c5a2b80 1 client.20709 shutdown 2023-07-14T16:34:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2859124319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 msgr2=0x563faa94b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa94b8a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f430c0093f0 tx=0x7f430c028630 comp rx=0 tx=0).stop 2023-07-14T16:34:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2859124319 shutdown_connections 2023-07-14T16:34:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.164:6838/794647512,v1:172.21.15.164:6839/794647512] conn(0x563faa9a9060 0x563faa9ab470 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563faa93fc70 0x563faa94ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.567 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563faa93e4d0 0x563faa94b8a0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 --2- 172.21.15.62:0/2859124319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563faa93ee20 0x563faa94bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2859124319 >> 172.21.15.62:0/2859124319 conn(0x563faa933da0 msgr2=0x563faa950010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f4309ffb700 0 client.20709 ms_handle_reset on v2:172.21.15.164:6838/794647512 2023-07-14T16:34:57.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2859124319 shutdown_connections 2023-07-14T16:34:57.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.594+0000 7f433c5a2b80 1 -- 172.21.15.62:0/2859124319 wait complete. 2023-07-14T16:34:57.568 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.595+0000 7f433c5a2b80 20 client.20709 trim_cache size 0 max 16384 2023-07-14T16:34:57.569 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:34:57.569 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.595+0000 7f4328bff700 1 -- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43100a1350 msgr2=0x7f43100a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.595+0000 7f4328bff700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43100a1350 0x7f43100a3800 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f431400b6c0 tx=0x7f43140093f0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.595+0000 7f4328bff700 1 -- 172.21.15.62:0/1767138664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43240ff470 msgr2=0x7f43241abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.595+0000 7f4328bff700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43240ff470 0x7f43241abfb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f431c00b550 tx=0x7f431c00b910 comp rx=0 tx=0).stop 2023-07-14T16:34:57.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.596+0000 7f4328bff700 1 -- 172.21.15.62:0/1767138664 shutdown_connections 2023-07-14T16:34:57.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.596+0000 7f4328bff700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f43100a1350 0x7f43100a3800 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.596+0000 7f4328bff700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43240ff470 0x7f43241abfb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.570 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.596+0000 7f4328bff700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43240fd990 0x7f432410a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.596+0000 7f4328bff700 1 --2- 172.21.15.62:0/1767138664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43240feac0 0x7f432410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.596+0000 7f4328bff700 1 -- 172.21.15.62:0/1767138664 >> 172.21.15.62:0/1767138664 conn(0x7f43240f9340 msgr2=0x7f43240f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.596+0000 7f4328bff700 1 -- 172.21.15.62:0/1767138664 shutdown_connections 2023-07-14T16:34:57.571 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:57.596+0000 7f4328bff700 1 -- 172.21.15.62:0/1767138664 wait complete. 2023-07-14T16:34:57.601 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T16:34:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.000+0000 7f17e09ff700 1 -- 172.21.15.62:0/2371556151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc100cb0 msgr2=0x7f17dc103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.000+0000 7f17e09ff700 1 --2- 172.21.15.62:0/2371556151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc100cb0 0x7f17dc103130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0098f0 tx=0x7f17cc009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.001+0000 7f17e09ff700 1 -- 172.21.15.62:0/2371556151 shutdown_connections 2023-07-14T16:34:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.001+0000 7f17e09ff700 1 --2- 172.21.15.62:0/2371556151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17dc103670 0x7f17dc103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.001+0000 7f17e09ff700 1 --2- 172.21.15.62:0/2371556151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc0fe350 0x7f17dc100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.001+0000 7f17e09ff700 1 --2- 172.21.15.62:0/2371556151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc100cb0 0x7f17dc103130 secure :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0098f0 tx=0x7f17cc009c00 comp rx=0 tx=0).stop 2023-07-14T16:34:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.001+0000 7f17e09ff700 1 -- 172.21.15.62:0/2371556151 >> 172.21.15.62:0/2371556151 conn(0x7f17dc0fa000 msgr2=0x7f17dc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.001+0000 7f17e09ff700 1 -- 172.21.15.62:0/2371556151 shutdown_connections 2023-07-14T16:34:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.001+0000 7f17e09ff700 1 -- 172.21.15.62:0/2371556151 wait complete. 2023-07-14T16:34:57.975 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.003+0000 7f17e09ff700 1 Processor -- start 2023-07-14T16:34:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.003+0000 7f17e09ff700 1 -- start start 2023-07-14T16:34:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17e09ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17dc0fe350 0x7f17dc19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17e09ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc103670 0x7f17dc19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17e09ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc1a15f0 0x7f17dc1a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17e09ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc1a3fa0 con 0x7f17dc0fe350 2023-07-14T16:34:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17e09ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc069f30 con 0x7f17dc103670 2023-07-14T16:34:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17e09ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc06a0c0 con 0x7f17dc1a15f0 2023-07-14T16:34:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc103670 0x7f17dc19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc103670 0x7f17dc19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48784/0 (socket says 172.21.15.62:48784) 2023-07-14T16:34:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17d3fff700 1 -- 172.21.15.62:0/528040557 learned_addr learned my addr 172.21.15.62:0/528040557 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.004+0000 7f17da59c700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17dc0fe350 0x7f17dc19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.005+0000 7f17dad9d700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc1a15f0 0x7f17dc1a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.005+0000 7f17da59c700 1 -- 172.21.15.62:0/528040557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc1a15f0 msgr2=0x7f17dc1a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.005+0000 7f17da59c700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc1a15f0 0x7f17dc1a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.005+0000 7f17da59c700 1 -- 172.21.15.62:0/528040557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc103670 msgr2=0x7f17dc19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.005+0000 7f17da59c700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc103670 0x7f17dc19c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.979 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.005+0000 7f17da59c700 1 -- 172.21.15.62:0/528040557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc009520 con 0x7f17dc0fe350 2023-07-14T16:34:57.980 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.005+0000 7f17da59c700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17dc0fe350 0x7f17dc19c050 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f17c400e9c0 tx=0x7f17c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.010+0000 7f17d37fe700 1 -- 172.21.15.62:0/528040557 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c400c9c0 con 0x7f17dc0fe350 2023-07-14T16:34:57.983 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.010+0000 7f17d37fe700 1 -- 172.21.15.62:0/528040557 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17c4010430 con 0x7f17dc0fe350 2023-07-14T16:34:57.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.010+0000 7f17d37fe700 1 -- 172.21.15.62:0/528040557 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4018680 con 0x7f17dc0fe350 2023-07-14T16:34:57.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.010+0000 7f17e09ff700 1 -- 172.21.15.62:0/528040557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc06a3a0 con 0x7f17dc0fe350 2023-07-14T16:34:57.984 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.010+0000 7f17e09ff700 1 -- 172.21.15.62:0/528040557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc06a860 con 0x7f17dc0fe350 2023-07-14T16:34:57.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.017+0000 7f17d37fe700 1 -- 172.21.15.62:0/528040557 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f17c400cb20 con 0x7f17dc0fe350 2023-07-14T16:34:57.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.017+0000 7f17f43b4b80 1 Processor -- start 2023-07-14T16:34:57.990 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.017+0000 7f17f43b4b80 1 -- start start 2023-07-14T16:34:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 0x564a01ac68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac6e20 0x564a01ac76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac7c70 0x564a01ac80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a019cbcb0 con 0x564a01ac64d0 2023-07-14T16:34:57.991 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01a1ea40 con 0x564a01ac6e20 2023-07-14T16:34:57.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01a17ed0 con 0x564a01ac7c70 2023-07-14T16:34:57.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17da59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac6e20 0x564a01ac76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17da59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac6e20 0x564a01ac76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48788/0 (socket says 172.21.15.62:48788) 2023-07-14T16:34:57.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17da59c700 1 -- 172.21.15.62:0/3275727899 learned_addr learned my addr 172.21.15.62:0/3275727899 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:57.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.018+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac7c70 0x564a01ac80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.019+0000 7f17dad9d700 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 0x564a01ac68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.019+0000 7f17da59c700 1 -- 172.21.15.62:0/3275727899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac7c70 msgr2=0x564a01ac80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.019+0000 7f17da59c700 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac7c70 0x564a01ac80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.019+0000 7f17da59c700 1 -- 172.21.15.62:0/3275727899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 msgr2=0x564a01ac68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.019+0000 7f17da59c700 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 0x564a01ac68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.019+0000 7f17da59c700 1 -- 172.21.15.62:0/3275727899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a01845de0 con 0x564a01ac6e20 2023-07-14T16:34:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.019+0000 7f17da59c700 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac6e20 0x564a01ac76e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f17c40d2a80 tx=0x7f17c40d2ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919c80f8ad3ed23c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.020+0000 7f17d0ff9700 1 -- 172.21.15.62:0/3275727899 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40d48d0 con 0x564a01ac6e20 2023-07-14T16:34:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.020+0000 7f17d0ff9700 1 -- 172.21.15.62:0/3275727899 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17c40d1030 con 0x564a01ac6e20 2023-07-14T16:34:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.020+0000 7f17d0ff9700 1 -- 172.21.15.62:0/3275727899 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40d7ea0 con 0x564a01ac6e20 2023-07-14T16:34:57.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.021+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3275727899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac6e20 msgr2=0x564a01ac76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.021+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac6e20 0x564a01ac76e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f17c40d2a80 tx=0x7f17c40d2ab0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.023+0000 7f17d37fe700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c80a12d0 0x7f17c80a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.023+0000 7f17d37fe700 1 -- 172.21.15.62:0/528040557 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17c4014070 con 0x7f17dc0fe350 2023-07-14T16:34:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.024+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3275727899 shutdown_connections 2023-07-14T16:34:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.024+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac7c70 0x564a01ac80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.024+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 0x564a01ac68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.024+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3275727899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac6e20 0x564a01ac76e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:57.997 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.024+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3275727899 >> 172.21.15.62:0/3275727899 conn(0x564a01abbda0 msgr2=0x564a01ac11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.025+0000 7f17dad9d700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c80a12d0 0x7f17c80a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.025+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3275727899 shutdown_connections 2023-07-14T16:34:57.998 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.026+0000 7f17dad9d700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c80a12d0 0x7f17c80a3780 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009540 tx=0x7f17d400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:58.003 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.029+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3275727899 wait complete. 2023-07-14T16:34:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.030+0000 7f17f43b4b80 1 -- start start 2023-07-14T16:34:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.030+0000 7f17f43b4b80 1 Processor -- start 2023-07-14T16:34:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.031+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 0x564a01ad0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.004 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a019b7ac0 con 0x564a01ac64d0 2023-07-14T16:34:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a019b77c0 con 0x564a01ac7c70 2023-07-14T16:34:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0189b920 con 0x564a01ac6e20 2023-07-14T16:34:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17f43b4b80 10 client.0 _ll_register_callbacks cb 0x564a019b5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:58.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17dad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17dad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48804/0 (socket says 172.21.15.62:48804) 2023-07-14T16:34:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.032+0000 7f17dad9d700 1 -- 172.21.15.62:0/2701356206 learned_addr learned my addr 172.21.15.62:0/2701356206 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.034+0000 7f17dad9d700 1 -- 172.21.15.62:0/2701356206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 msgr2=0x564a01ad0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:58.006 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.034+0000 7f17dad9d700 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.034+0000 7f17dad9d700 1 -- 172.21.15.62:0/2701356206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 msgr2=0x564a01ad0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.034+0000 7f17dad9d700 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 0x564a01ad0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.034+0000 7f17dad9d700 1 -- 172.21.15.62:0/2701356206 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc06a860 con 0x564a01ac7c70 2023-07-14T16:34:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.034+0000 7f17dad9d700 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad75b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17d40169e0 tx=0x7f17d4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f0540d3a7bf322 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:58.007 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.034+0000 7f17c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:58.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.035+0000 7f17c1ffb700 1 -- 172.21.15.62:0/2701356206 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400e040 con 0x564a01ac7c70 2023-07-14T16:34:58.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.035+0000 7f17c1ffb700 1 -- 172.21.15.62:0/2701356206 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17d4015a20 con 0x564a01ac7c70 2023-07-14T16:34:58.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.035+0000 7f17f43b4b80 1 -- 172.21.15.62:0/2701356206 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a019714f0 con 0x564a01ac7c70 2023-07-14T16:34:58.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.035+0000 7f17c1ffb700 1 -- 172.21.15.62:0/2701356206 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17d4016e50 con 0x564a01ac7c70 2023-07-14T16:34:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.035+0000 7f17f43b4b80 10 client.30287 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.035+0000 7f17f43b4b80 1 -- 172.21.15.62:0/2701356206 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x564a0182ec60 con 0x564a01ac7c70 2023-07-14T16:34:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.036+0000 7f17c1ffb700 1 -- 172.21.15.62:0/2701356206 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564a019714f0 con 0x564a01ac7c70 2023-07-14T16:34:58.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.036+0000 7f17c1ffb700 1 -- 172.21.15.62:0/2701356206 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f17d4023c00 con 0x564a01ac7c70 2023-07-14T16:34:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.036+0000 7f17f43b4b80 10 client.30287 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.036+0000 7f17f43b4b80 10 client.30287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.036+0000 7f17f43b4b80 20 client.30287 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.036+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a01adbf10 0x564a01ade320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.036+0000 7f17f43b4b80 4 client.30287 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:34:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.036+0000 7f17f43b4b80 1 -- 172.21.15.62:0/2701356206 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a01a10eb0 con 0x564a01adbf10 2023-07-14T16:34:58.010 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.037+0000 7f17da59c700 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a01adbf10 0x564a01ade320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:58.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.038+0000 7f17da59c700 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a01adbf10 0x564a01ade320 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67cf8b3b367cf01 server_cookie=4363810465c9a2e0 in_seq=0 out_seq=0 2023-07-14T16:34:58.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.038+0000 7f17c1ffb700 10 client.30287 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:58.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.039+0000 7f17c1ffb700 1 -- 172.21.15.62:0/2701356206 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a01a10eb0 con 0x564a01adbf10 2023-07-14T16:34:58.011 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.039+0000 7f17c1ffb700 10 client.30287 handle_command_reply: tid=0 2023-07-14T16:34:58.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.039+0000 7f17f43b4b80 1 client.30287 shutdown 2023-07-14T16:34:58.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17f43b4b80 1 -- 172.21.15.62:0/2701356206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 msgr2=0x564a01ad75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:58.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad75b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f17d40169e0 tx=0x7f17d4004b40 comp rx=0 tx=0).stop 2023-07-14T16:34:58.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17f43b4b80 1 -- 172.21.15.62:0/2701356206 shutdown_connections 2023-07-14T16:34:58.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a01adbf10 0x564a01ade320 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad75b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac64d0 0x564a01ad0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/2701356206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.013 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17f43b4b80 1 -- 172.21.15.62:0/2701356206 >> 172.21.15.62:0/2701356206 conn(0x564a01abbda0 msgr2=0x564a01ac08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:58.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.040+0000 7f17c1ffb700 0 client.30287 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:58.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.042+0000 7f17f43b4b80 1 -- 172.21.15.62:0/2701356206 shutdown_connections 2023-07-14T16:34:58.014 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.042+0000 7f17f43b4b80 1 -- 172.21.15.62:0/2701356206 wait complete. 2023-07-14T16:34:58.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.043+0000 7f17f43b4b80 20 client.30287 trim_cache size 0 max 16384 2023-07-14T16:34:58.043 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.071+0000 7f17f43b4b80 1 Processor -- start 2023-07-14T16:34:58.045 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.071+0000 7f17f43b4b80 1 -- start start 2023-07-14T16:34:58.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.071+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac64d0 0x564a01ac80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.071+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.071+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac7c70 0x564a01aced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.071+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01ad9240 con 0x564a01ac7c70 2023-07-14T16:34:58.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.071+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01acddf0 con 0x564a01ac64d0 2023-07-14T16:34:58.046 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.071+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01ac2ca0 con 0x564a01ac6e20 2023-07-14T16:34:58.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.072+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:58.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.072+0000 7f17da59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac64d0 0x564a01ac80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:58.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.073+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:41982/0 (socket says 172.21.15.62:41982) 2023-07-14T16:34:58.047 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.073+0000 7f17d3fff700 1 -- 172.21.15.62:0/3624141998 learned_addr learned my addr 172.21.15.62:0/3624141998 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:58.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.074+0000 7f17d3fff700 1 -- 172.21.15.62:0/3624141998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac64d0 msgr2=0x564a01ac80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:58.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.074+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3624141998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac64d0 0x564a01ac80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.074+0000 7f17d3fff700 1 -- 172.21.15.62:0/3624141998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac7c70 msgr2=0x564a01aced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:58.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.074+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3624141998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac7c70 0x564a01aced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.048 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.074+0000 7f17d3fff700 1 -- 172.21.15.62:0/3624141998 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a0186c250 con 0x564a01ac6e20 2023-07-14T16:34:58.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.074+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3624141998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad7570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009810 tx=0x7f17cc00bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84805493f862c3f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:58.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.075+0000 7f17d17fa700 1 -- 172.21.15.62:0/3624141998 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc01c050 con 0x564a01ac6e20 2023-07-14T16:34:58.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.076+0000 7f17d17fa700 1 -- 172.21.15.62:0/3624141998 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17cc01d040 con 0x564a01ac6e20 2023-07-14T16:34:58.049 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.076+0000 7f17d17fa700 1 -- 172.21.15.62:0/3624141998 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc005040 con 0x564a01ac6e20 2023-07-14T16:34:58.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.081+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3624141998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 msgr2=0x564a01ad7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:58.054 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.081+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3624141998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad7570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009810 tx=0x7f17cc00bf30 comp rx=0 tx=0).stop 2023-07-14T16:34:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.082+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3624141998 shutdown_connections 2023-07-14T16:34:58.055 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.082+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3624141998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac7c70 0x564a01aced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.082+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3624141998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac64d0 0x564a01ac80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.082+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3624141998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac6e20 0x564a01ad7570 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.082+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3624141998 >> 172.21.15.62:0/3624141998 conn(0x564a01abbda0 msgr2=0x564a01ae2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:58.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.083+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3624141998 shutdown_connections 2023-07-14T16:34:58.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.083+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3624141998 wait complete. 2023-07-14T16:34:58.058 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.084+0000 7f17f43b4b80 1 -- start start 2023-07-14T16:34:58.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.084+0000 7f17f43b4b80 1 Processor -- start 2023-07-14T16:34:58.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.085+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac64d0 0x564a01ad1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.059 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.085+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac6e20 0x564a01ad1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.085+0000 7f17f43b4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.085+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01ad6f70 con 0x564a01ac6e20 2023-07-14T16:34:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.085+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01ad70e0 con 0x564a01ac7c70 2023-07-14T16:34:58.060 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.085+0000 7f17f43b4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01ad72a0 con 0x564a01ac64d0 2023-07-14T16:34:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.085+0000 7f17f43b4b80 10 client.0 _ll_register_callbacks cb 0x7f17d4007e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:34:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17da59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac64d0 0x564a01ad1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:58.061 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17dad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17dad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:48818/0 (socket says 172.21.15.62:48818) 2023-07-14T16:34:58.062 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17dad9d700 1 -- 172.21.15.62:0/3319385956 learned_addr learned my addr 172.21.15.62:0/3319385956 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:34:58.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17d3fff700 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac6e20 0x564a01ad1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:58.064 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17dad9d700 1 -- 172.21.15.62:0/3319385956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac64d0 msgr2=0x564a01ad1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17dad9d700 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac64d0 0x564a01ad1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17dad9d700 1 -- 172.21.15.62:0/3319385956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac6e20 msgr2=0x564a01ad1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:34:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17dad9d700 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac6e20 0x564a01ad1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.086+0000 7f17dad9d700 1 -- 172.21.15.62:0/3319385956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc104610 con 0x564a01ac7c70 2023-07-14T16:34:58.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.087+0000 7f17dad9d700 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad6a30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f17d4025000 tx=0x7f17d402cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65cb102c55fba66 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:34:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.087+0000 7f17c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:34:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.087+0000 7f17c1ffb700 1 -- 172.21.15.62:0/3319385956 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400e040 con 0x564a01ac7c70 2023-07-14T16:34:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.087+0000 7f17c1ffb700 1 -- 172.21.15.62:0/3319385956 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17d4015a80 con 0x564a01ac7c70 2023-07-14T16:34:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.088+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3319385956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17c40d1030 con 0x564a01ac7c70 2023-07-14T16:34:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.088+0000 7f17c1ffb700 1 -- 172.21.15.62:0/3319385956 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17d4016e50 con 0x564a01ac7c70 2023-07-14T16:34:58.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.088+0000 7f17f43b4b80 10 client.30293 fetch_fsmap learned FSMap version 36 2023-07-14T16:34:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.088+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3319385956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x564a018442b0 con 0x564a01ac7c70 2023-07-14T16:34:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.089+0000 7f17c1ffb700 1 -- 172.21.15.62:0/3319385956 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17c40d1030 con 0x564a01ac7c70 2023-07-14T16:34:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.089+0000 7f17c1ffb700 1 -- 172.21.15.62:0/3319385956 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f17d403d030 con 0x564a01ac7c70 2023-07-14T16:34:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.089+0000 7f17f43b4b80 10 client.30293 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:34:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.089+0000 7f17f43b4b80 10 client.30293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T16:34:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.089+0000 7f17f43b4b80 20 client.30293 populate_metadata read hostname 'smithi062' 2023-07-14T16:34:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.089+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a01b41fc0 0x564a01b443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:34:58.067 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.089+0000 7f17f43b4b80 4 client.30293 mds_command: new command op to 24602 tid=0[{"prefix": "damage ls"}] 2023-07-14T16:34:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.089+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3319385956 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a019d3050 con 0x564a01b41fc0 2023-07-14T16:34:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.090+0000 7f17da59c700 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a01b41fc0 0x564a01b443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:34:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.091+0000 7f17da59c700 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a01b41fc0 0x564a01b443d0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dda2cad25eec776 server_cookie=c7cfd8451dc33064 in_seq=0 out_seq=0 2023-07-14T16:34:58.068 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.091+0000 7f17c1ffb700 10 client.30293 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:58.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17c1ffb700 1 -- 172.21.15.62:0/3319385956 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a019d3050 con 0x564a01b41fc0 2023-07-14T16:34:58.069 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17c1ffb700 10 client.30293 handle_command_reply: tid=0 2023-07-14T16:34:58.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 client.30293 shutdown 2023-07-14T16:34:58.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3319385956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 msgr2=0x564a01ad6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:58.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad6a30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f17d4025000 tx=0x7f17d402cbf0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.070 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3319385956 shutdown_connections 2023-07-14T16:34:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x564a01b41fc0 0x564a01b443d0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564a01ac7c70 0x564a01ad6a30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564a01ac64d0 0x564a01ad1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17c1ffb700 0 client.30293 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:34:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 --2- 172.21.15.62:0/3319385956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a01ac6e20 0x564a01ad1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.094+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3319385956 >> 172.21.15.62:0/3319385956 conn(0x564a01abbda0 msgr2=0x564a01aced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:58.071 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.095+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3319385956 shutdown_connections 2023-07-14T16:34:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.095+0000 7f17f43b4b80 1 -- 172.21.15.62:0/3319385956 wait complete. 2023-07-14T16:34:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.095+0000 7f17f43b4b80 20 client.30293 trim_cache size 0 max 16384 2023-07-14T16:34:58.072 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2023-07-14T16:34:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 -- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c80a12d0 msgr2=0x7f17c80a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c80a12d0 0x7f17c80a3780 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009540 tx=0x7f17d400b040 comp rx=0 tx=0).stop 2023-07-14T16:34:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 -- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17dc0fe350 msgr2=0x7f17dc19c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:34:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17dc0fe350 0x7f17dc19c050 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f17c400e9c0 tx=0x7f17c400ecd0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 -- 172.21.15.62:0/528040557 shutdown_connections 2023-07-14T16:34:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17dc1a15f0 0x7f17dc1a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17dc0fe350 0x7f17dc19c050 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f17c80a12d0 0x7f17c80a3780 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 --2- 172.21.15.62:0/528040557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc103670 0x7f17dc19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:34:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 -- 172.21.15.62:0/528040557 >> 172.21.15.62:0/528040557 conn(0x7f17dc0fa000 msgr2=0x7f17dc0feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:34:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.096+0000 7f17e09ff700 1 -- 172.21.15.62:0/528040557 shutdown_connections 2023-07-14T16:34:58.074 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:34:58.097+0000 7f17e09ff700 1 -- 172.21.15.62:0/528040557 wait complete. 2023-07-14T16:34:58.099 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 29144257, 'ino': 2199023256067, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fc.Po'}, {'damage_type': 'backtrace', 'id': 268903644, 'ino': 1099511628995, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rwlock.o'}, {'damage_type': 'backtrace', 'id': 294017427, 'ino': 2199023256064, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_op.Po'}, {'damage_type': 'backtrace', 'id': 332135477, 'ino': 2199023256054, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/main.Po'}, {'damage_type': 'backtrace', 'id': 467240244, 'ino': 1099511629000, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_thread.o'}, {'damage_type': 'backtrace', 'id': 503356804, 'ino': 2199023256066, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/parser.Po'}, {'damage_type': 'backtrace', 'id': 518823357, 'ino': 2199023256062, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_fs.Po'}, {'damage_type': 'backtrace', 'id': 545345685, 'ino': 1099511628987, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h'}, {'damage_type': 'backtrace', 'id': 547529498, 'ino': 2199023256059, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/cirlist.Po'}, {'damage_type': 'backtrace', 'id': 662921452, 'ino': 1099511628998, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_tg.o'}, {'damage_type': 'backtrace', 'id': 665414258, 'ino': 1099511628999, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fs.o'}, {'damage_type': 'backtrace', 'id': 677933795, 'ino': 2199023256060, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rbt.Po'}, {'damage_type': 'backtrace', 'id': 1024359789, 'ino': 2199023256068, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_stats.Po'}, {'damage_type': 'backtrace', 'id': 1355940457, 'ino': 1099511628991, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/main.o'}, {'damage_type': 'backtrace', 'id': 1374977809, 'ino': 1099511628992, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fh.o'}, {'damage_type': 'backtrace', 'id': 1501417065, 'ino': 1099511629002, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/util.o'}, {'damage_type': 'backtrace', 'id': 1875471956, 'ino': 1099511629001, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_op.o'}, {'damage_type': 'backtrace', 'id': 2036491788, 'ino': 1099511628978, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/Makefile'}, {'damage_type': 'backtrace', 'id': 2167197996, 'ino': 1099511628477, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in~'}, {'damage_type': 'backtrace', 'id': 2326319758, 'ino': 1099511629005, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_stats.o'}, {'damage_type': 'backtrace', 'id': 2384713424, 'ino': 1099511629004, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb_fc.o'}, {'damage_type': 'backtrace', 'id': 2448116865, 'ino': 2199023256058, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rwlock.Po'}, {'damage_type': 'backtrace', 'id': 2913004853, 'ino': 2199023256057, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/metaops.Po'}, {'damage_type': 'backtrace', 'id': 3030855656, 'ino': 1099511628994, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/metaops.o'}, {'damage_type': 'backtrace', 'id': 3039845192, 'ino': 2199023256061, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_tg.Po'}, {'damage_type': 'backtrace', 'id': 3080609784, 'ino': 1099511628993, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/filelist.o'}, {'damage_type': 'backtrace', 'id': 3125677993, 'ino': 2199023256065, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/util.Po'}, {'damage_type': 'backtrace', 'id': 3214309731, 'ino': 1099511628988, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/stamp-h1'}, {'damage_type': 'backtrace', 'id': 3240159515, 'ino': 2199023255552, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fileops.Po'}, {'damage_type': 'backtrace', 'id': 3298915533, 'ino': 1099511629003, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/parser.o'}, {'damage_type': 'backtrace', 'id': 3418078762, 'ino': 2199023255553, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/rand.Po'}, {'damage_type': 'backtrace', 'id': 3595090028, 'ino': 1099511629007, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/ffsb'}, {'damage_type': 'backtrace', 'id': 3620380938, 'ino': 1099511628990, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rand.o'}, {'damage_type': 'backtrace', 'id': 3622665720, 'ino': 1099511628996, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/cirlist.o'}, {'damage_type': 'backtrace', 'id': 3697428185, 'ino': 1099511629008, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/tmp'}, {'damage_type': 'backtrace', 'id': 3745515000, 'ino': 1099511628989, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/fileops.o'}, {'damage_type': 'backtrace', 'id': 3747423216, 'ino': 1099511628980, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/config.h.in'}, {'damage_type': 'backtrace', 'id': 3773069627, 'ino': 2199023256069, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/list.Po'}, {'damage_type': 'backtrace', 'id': 3815028410, 'ino': 2199023256056, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/filelist.Po'}, {'damage_type': 'backtrace', 'id': 4004922964, 'ino': 1099511629006, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/list.o'}, {'damage_type': 'backtrace', 'id': 4170503966, 'ino': 1099511628997, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/rbt.o'}, {'damage_type': 'backtrace', 'id': 4184007015, 'ino': 2199023256063, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/ffsb_thread.Po'}, {'damage_type': 'backtrace', 'id': 4232185085, 'ino': 2199023256055, 'path': '/volumes/_nogroup/sv_0/41d87a1f-2ab4-4b30-af2a-b32c58d851e2/client.0/tmp/ffsb/.deps/fh.Po'}] 2023-07-14T16:34:58.099 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-07-14T16:34:58.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:34:58 smithi164 ceph-mon[97427]: pgmap v937: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2023-07-14T16:34:58.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:34:58 smithi062 ceph-mon[95543]: pgmap v937: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2023-07-14T16:34:58.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:34:58 smithi081 ceph-mon[127795]: pgmap v937: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2023-07-14T16:35:01.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:00 smithi164 ceph-mon[97427]: pgmap v938: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.38k op/s 2023-07-14T16:35:01.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:00 smithi062 ceph-mon[95543]: pgmap v938: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.38k op/s 2023-07-14T16:35:01.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:00 smithi081 ceph-mon[127795]: pgmap v938: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 4.38k op/s 2023-07-14T16:35:03.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:02 smithi164 ceph-mon[97427]: pgmap v939: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:35:03.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:02 smithi062 ceph-mon[95543]: pgmap v939: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:35:03.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:02 smithi081 ceph-mon[127795]: pgmap v939: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:35:05.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:04 smithi164 ceph-mon[97427]: pgmap v940: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.32k op/s 2023-07-14T16:35:05.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:04 smithi062 ceph-mon[95543]: pgmap v940: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.32k op/s 2023-07-14T16:35:05.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:04 smithi081 ceph-mon[127795]: pgmap v940: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.32k op/s 2023-07-14T16:35:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.868+0000 7f7e197ff700 1 -- 172.21.15.62:0/1583465923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e1410bd60 msgr2=0x7f7e1410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.868+0000 7f7e197ff700 1 --2- 172.21.15.62:0/1583465923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e1410bd60 0x7f7e1410e210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009ec0 tx=0x7f7e0c00b330 comp rx=0 tx=0).stop 2023-07-14T16:35:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.868+0000 7f7e197ff700 1 -- 172.21.15.62:0/1583465923 shutdown_connections 2023-07-14T16:35:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.868+0000 7f7e197ff700 1 --2- 172.21.15.62:0/1583465923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e1410bd60 0x7f7e1410e210 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.868+0000 7f7e197ff700 1 --2- 172.21.15.62:0/1583465923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e14070f00 0x7f7e14071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.868+0000 7f7e197ff700 1 --2- 172.21.15.62:0/1583465923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14071850 0x7f7e1410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.868+0000 7f7e197ff700 1 -- 172.21.15.62:0/1583465923 >> 172.21.15.62:0/1583465923 conn(0x7f7e1406ca50 msgr2=0x7f7e1406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.869+0000 7f7e197ff700 1 -- 172.21.15.62:0/1583465923 shutdown_connections 2023-07-14T16:35:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.869+0000 7f7e197ff700 1 -- 172.21.15.62:0/1583465923 wait complete. 2023-07-14T16:35:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.870+0000 7f7e197ff700 1 Processor -- start 2023-07-14T16:35:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.871+0000 7f7e197ff700 1 -- start start 2023-07-14T16:35:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.871+0000 7f7e197ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14070f00 0x7f7e141a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.871+0000 7f7e197ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e14071850 0x7f7e141a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.871+0000 7f7e197ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e1410bd60 0x7f7e141a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.871+0000 7f7e197ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e141a61c0 con 0x7f7e14070f00 2023-07-14T16:35:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.871+0000 7f7e197ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e141a6330 con 0x7f7e1410bd60 2023-07-14T16:35:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.871+0000 7f7e197ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e141a64f0 con 0x7f7e14071850 2023-07-14T16:35:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.872+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14070f00 0x7f7e141a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.872+0000 7f7e127fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e14071850 0x7f7e141a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.872+0000 7f7e12ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14070f00 0x7f7e141a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49130/0 (socket says 172.21.15.62:49130) 2023-07-14T16:35:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.872+0000 7f7e12ffd700 1 -- 172.21.15.62:0/285234751 learned_addr learned my addr 172.21.15.62:0/285234751 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.872+0000 7f7e137fe700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e1410bd60 0x7f7e141a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.873+0000 7f7e12ffd700 1 -- 172.21.15.62:0/285234751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e14071850 msgr2=0x7f7e141a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.873+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e14071850 0x7f7e141a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.873+0000 7f7e12ffd700 1 -- 172.21.15.62:0/285234751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e1410bd60 msgr2=0x7f7e141a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.873+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e1410bd60 0x7f7e141a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.873+0000 7f7e12ffd700 1 -- 172.21.15.62:0/285234751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c005e70 con 0x7f7e14070f00 2023-07-14T16:35:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.873+0000 7f7e127fc700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e14071850 0x7f7e141a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:35:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.873+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14070f00 0x7f7e141a47a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400d980 tx=0x7f7e0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.874+0000 7f7dfbfff700 1 -- 172.21.15.62:0/285234751 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04004220 con 0x7f7e14070f00 2023-07-14T16:35:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.874+0000 7f7dfbfff700 1 -- 172.21.15.62:0/285234751 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e04007640 con 0x7f7e14070f00 2023-07-14T16:35:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.874+0000 7f7dfbfff700 1 -- 172.21.15.62:0/285234751 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04005660 con 0x7f7e14070f00 2023-07-14T16:35:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.874+0000 7f7e197ff700 1 -- 172.21.15.62:0/285234751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e141a67d0 con 0x7f7e14070f00 2023-07-14T16:35:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.874+0000 7f7e197ff700 1 -- 172.21.15.62:0/285234751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1410ee20 con 0x7f7e14070f00 2023-07-14T16:35:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.884+0000 7f7e2d1b3b80 1 Processor -- start 2023-07-14T16:35:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.885+0000 7f7e2d1b3b80 1 -- start start 2023-07-14T16:35:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.885+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca1b66b0 0x55fbca35a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.886+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca35a840 0x55fbca35b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.886+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca35bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.886+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca25fc50 con 0x55fbca35b6e0 2023-07-14T16:35:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.886+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca2b29e0 con 0x55fbca1b66b0 2023-07-14T16:35:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.886+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca2abe70 con 0x55fbca35a840 2023-07-14T16:35:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.886+0000 7f7dfbfff700 1 -- 172.21.15.62:0/285234751 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7e04004750 con 0x7f7e14070f00 2023-07-14T16:35:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca1b66b0 0x55fbca35a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca1b66b0 0x55fbca35a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51640/0 (socket says 172.21.15.62:51640) 2023-07-14T16:35:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 -- 172.21.15.62:0/417414780 learned_addr learned my addr 172.21.15.62:0/417414780 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e127fc700 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca35bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca35a840 0x55fbca35b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 -- 172.21.15.62:0/417414780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca35a840 msgr2=0x55fbca35b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca35a840 0x55fbca35b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 -- 172.21.15.62:0/417414780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 msgr2=0x55fbca35bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca35bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 -- 172.21.15.62:0/417414780 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbca0d9d80 con 0x55fbca1b66b0 2023-07-14T16:35:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.887+0000 7f7e137fe700 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca1b66b0 0x55fbca35a300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009c80 tx=0x7f7e0c0074e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f89396d3e0dbe665 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.888+0000 7f7df97fa700 1 -- 172.21.15.62:0/417414780 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00e040 con 0x55fbca1b66b0 2023-07-14T16:35:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.888+0000 7f7df97fa700 1 -- 172.21.15.62:0/417414780 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e0c00b9d0 con 0x55fbca1b66b0 2023-07-14T16:35:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.888+0000 7f7df97fa700 1 -- 172.21.15.62:0/417414780 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c01c6a0 con 0x55fbca1b66b0 2023-07-14T16:35:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.889+0000 7f7dfbfff700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1220 0x7f7dfc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.889+0000 7f7dfbfff700 1 -- 172.21.15.62:0/285234751 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e040ccbd0 con 0x7f7e14070f00 2023-07-14T16:35:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.890+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/417414780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca1b66b0 msgr2=0x55fbca35a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.890+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca1b66b0 0x55fbca35a300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009c80 tx=0x7f7e0c0074e0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.891+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/417414780 shutdown_connections 2023-07-14T16:35:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.891+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca35bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.891+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca1b66b0 0x55fbca35a300 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.891+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/417414780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca35a840 0x55fbca35b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.891+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/417414780 >> 172.21.15.62:0/417414780 conn(0x55fbca34fbe0 msgr2=0x55fbca354cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.891+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/417414780 shutdown_connections 2023-07-14T16:35:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.892+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1220 0x7f7dfc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.892+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/417414780 wait complete. 2023-07-14T16:35:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.892+0000 7f7e2d1b3b80 1 -- start start 2023-07-14T16:35:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.893+0000 7f7e2d1b3b80 1 Processor -- start 2023-07-14T16:35:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca3638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35a840 0x55fbca363df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35b6e0 0x55fbca36ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca24ba60 con 0x55fbca35a840 2023-07-14T16:35:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca24b760 con 0x55fbca35b6e0 2023-07-14T16:35:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca12f8c0 con 0x55fbca1b66b0 2023-07-14T16:35:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e2d1b3b80 10 client.0 _ll_register_callbacks cb 0x55fbca249700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35b6e0 0x55fbca36ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35b6e0 0x55fbca36ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51644/0 (socket says 172.21.15.62:51644) 2023-07-14T16:35:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.894+0000 7f7e137fe700 1 -- 172.21.15.62:0/1663904029 learned_addr learned my addr 172.21.15.62:0/1663904029 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e127fc700 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35a840 0x55fbca363df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e137fe700 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca3638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e12ffd700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1220 0x7f7dfc0a36d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7e040c9460 tx=0x7f7e04000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e127fc700 1 -- 172.21.15.62:0/1663904029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 msgr2=0x55fbca3638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e127fc700 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca3638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e127fc700 1 -- 172.21.15.62:0/1663904029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35b6e0 msgr2=0x55fbca36ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e127fc700 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35b6e0 0x55fbca36ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e127fc700 1 -- 172.21.15.62:0/1663904029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e14117640 con 0x55fbca35a840 2023-07-14T16:35:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.895+0000 7f7e137fe700 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35b6e0 0x55fbca36ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:35:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.896+0000 7f7e127fc700 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35a840 0x55fbca363df0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7e08011ce0 tx=0x7f7e08009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8618095b10834b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.896+0000 7f7df6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:35:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.896+0000 7f7df6ff5700 1 -- 172.21.15.62:0/1663904029 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08016070 con 0x55fbca35a840 2023-07-14T16:35:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.896+0000 7f7df6ff5700 1 -- 172.21.15.62:0/1663904029 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e08004240 con 0x55fbca35a840 2023-07-14T16:35:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.896+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/1663904029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbca205490 con 0x55fbca35a840 2023-07-14T16:35:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.897+0000 7f7df6ff5700 1 -- 172.21.15.62:0/1663904029 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e08004400 con 0x55fbca35a840 2023-07-14T16:35:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.897+0000 7f7e2d1b3b80 10 client.20733 fetch_fsmap learned FSMap version 36 2023-07-14T16:35:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.897+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/1663904029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fbca0c2c00 con 0x55fbca35a840 2023-07-14T16:35:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.897+0000 7f7df6ff5700 1 -- 172.21.15.62:0/1663904029 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fbca205490 con 0x55fbca35a840 2023-07-14T16:35:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.897+0000 7f7df6ff5700 1 -- 172.21.15.62:0/1663904029 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e08003a30 con 0x55fbca35a840 2023-07-14T16:35:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.898+0000 7f7e2d1b3b80 10 client.20733 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:35:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.898+0000 7f7e2d1b3b80 10 client.20733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.898+0000 7f7e2d1b3b80 20 client.20733 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.898+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fbca36f770 0x55fbca371b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.898+0000 7f7e2d1b3b80 4 client.20733 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:35:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.898+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/1663904029 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbca2a4e50 con 0x55fbca36f770 2023-07-14T16:35:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.899+0000 7f7e137fe700 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fbca36f770 0x55fbca371b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.906+0000 7f7e137fe700 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fbca36f770 0x55fbca371b80 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be4421e1dd1790d server_cookie=43c6c280ccfd46c3 in_seq=0 out_seq=0 2023-07-14T16:35:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.906+0000 7f7df6ff5700 10 client.20733 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.907+0000 7f7df6ff5700 1 -- 172.21.15.62:0/1663904029 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fbca2a4e50 con 0x55fbca36f770 2023-07-14T16:35:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.907+0000 7f7df6ff5700 10 client.20733 handle_command_reply: tid=0 2023-07-14T16:35:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.907+0000 7f7e2d1b3b80 1 client.20733 shutdown 2023-07-14T16:35:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.907+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/1663904029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35a840 msgr2=0x55fbca363df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.907+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35a840 0x55fbca363df0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7e08011ce0 tx=0x7f7e08009d70 comp rx=0 tx=0).stop 2023-07-14T16:35:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/1663904029 shutdown_connections 2023-07-14T16:35:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fbca36f770 0x55fbca371b80 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35b6e0 0x55fbca36ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca3638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/1663904029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35a840 0x55fbca363df0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/1663904029 >> 172.21.15.62:0/1663904029 conn(0x55fbca34fbe0 msgr2=0x55fbca3544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7df6ff5700 0 client.20733 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/1663904029 shutdown_connections 2023-07-14T16:35:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.908+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/1663904029 wait complete. 2023-07-14T16:35:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.909+0000 7f7e2d1b3b80 20 client.20733 trim_cache size 0 max 16384 2023-07-14T16:35:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.938+0000 7f7e2d1b3b80 1 Processor -- start 2023-07-14T16:35:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.939+0000 7f7e2d1b3b80 1 -- start start 2023-07-14T16:35:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.939+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca354c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.939+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca36adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.939+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca361b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.939+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca34cbb0 con 0x55fbca35b6e0 2023-07-14T16:35:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.939+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca36caa0 con 0x55fbca35a840 2023-07-14T16:35:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.939+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca3618b0 con 0x55fbca1b66b0 2023-07-14T16:35:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.940+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca354c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.943+0000 7f7e137fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca354c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:43748/0 (socket says 172.21.15.62:43748) 2023-07-14T16:35:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.943+0000 7f7e137fe700 1 -- 172.21.15.62:0/860125140 learned_addr learned my addr 172.21.15.62:0/860125140 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.943+0000 7f7e137fe700 1 --2- 172.21.15.62:0/860125140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca361b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.944+0000 7f7e137fe700 1 -- 172.21.15.62:0/860125140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 msgr2=0x55fbca36adf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:35:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.944+0000 7f7e137fe700 1 --2- 172.21.15.62:0/860125140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca36adf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.944+0000 7f7e137fe700 1 -- 172.21.15.62:0/860125140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 msgr2=0x55fbca361b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.944+0000 7f7e137fe700 1 --2- 172.21.15.62:0/860125140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca361b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.944+0000 7f7e137fe700 1 -- 172.21.15.62:0/860125140 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbca1001f0 con 0x55fbca1b66b0 2023-07-14T16:35:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.944+0000 7f7e137fe700 1 --2- 172.21.15.62:0/860125140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca354c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c000f80 tx=0x7f7e0c0074e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6a2be32d2628065 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.945+0000 7f7df9ffb700 1 -- 172.21.15.62:0/860125140 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c033460 con 0x55fbca1b66b0 2023-07-14T16:35:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.945+0000 7f7df9ffb700 1 -- 172.21.15.62:0/860125140 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e0c0335c0 con 0x55fbca1b66b0 2023-07-14T16:35:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.945+0000 7f7df9ffb700 1 -- 172.21.15.62:0/860125140 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c01ce90 con 0x55fbca1b66b0 2023-07-14T16:35:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.946+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/860125140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 msgr2=0x55fbca354c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.946+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/860125140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca354c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c000f80 tx=0x7f7e0c0074e0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.946+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/860125140 shutdown_connections 2023-07-14T16:35:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.946+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/860125140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca361b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.946+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/860125140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca354c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.947+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/860125140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca36adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.947+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/860125140 >> 172.21.15.62:0/860125140 conn(0x55fbca34fbe0 msgr2=0x55fbca3625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.948+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/860125140 shutdown_connections 2023-07-14T16:35:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.948+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/860125140 wait complete. 2023-07-14T16:35:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.948+0000 7f7e2d1b3b80 1 -- start start 2023-07-14T16:35:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.949+0000 7f7e2d1b3b80 1 Processor -- start 2023-07-14T16:35:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.949+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca366330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.949+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca366870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.949+0000 7f7e2d1b3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca369890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.949+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca369dd0 con 0x55fbca35b6e0 2023-07-14T16:35:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.949+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca369f40 con 0x55fbca35a840 2023-07-14T16:35:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.949+0000 7f7e2d1b3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbca36a100 con 0x55fbca1b66b0 2023-07-14T16:35:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.949+0000 7f7e2d1b3b80 10 client.0 _ll_register_callbacks cb 0x7f7e080117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e127fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca366870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e127fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca366870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51656/0 (socket says 172.21.15.62:51656) 2023-07-14T16:35:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e127fc700 1 -- 172.21.15.62:0/2156322969 learned_addr learned my addr 172.21.15.62:0/2156322969 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca369890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca366330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e127fc700 1 -- 172.21.15.62:0/2156322969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 msgr2=0x55fbca366330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e127fc700 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca366330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e127fc700 1 -- 172.21.15.62:0/2156322969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 msgr2=0x55fbca369890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e127fc700 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca369890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e127fc700 1 -- 172.21.15.62:0/2156322969 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c02c040 con 0x55fbca35a840 2023-07-14T16:35:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.950+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca369890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:35:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.951+0000 7f7e127fc700 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca366870 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7e080118f0 tx=0x7f7e0800f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=185af4e9959ea2a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.951+0000 7f7df6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:35:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.951+0000 7f7df6ff5700 1 -- 172.21.15.62:0/2156322969 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08016040 con 0x55fbca35a840 2023-07-14T16:35:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.951+0000 7f7df6ff5700 1 -- 172.21.15.62:0/2156322969 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e08000e40 con 0x55fbca35a840 2023-07-14T16:35:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.951+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/2156322969 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e0c00b9d0 con 0x55fbca35a840 2023-07-14T16:35:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.952+0000 7f7df6ff5700 1 -- 172.21.15.62:0/2156322969 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e08035030 con 0x55fbca35a840 2023-07-14T16:35:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.952+0000 7f7e2d1b3b80 10 client.30308 fetch_fsmap learned FSMap version 36 2023-07-14T16:35:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.952+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/2156322969 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fbca0d8250 con 0x55fbca35a840 2023-07-14T16:35:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.956+0000 7f7df6ff5700 1 -- 172.21.15.62:0/2156322969 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e0c00b9d0 con 0x55fbca35a840 2023-07-14T16:35:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.956+0000 7f7df6ff5700 1 -- 172.21.15.62:0/2156322969 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e0800f830 con 0x55fbca35a840 2023-07-14T16:35:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.957+0000 7f7e2d1b3b80 10 client.30308 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T16:35:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.957+0000 7f7e2d1b3b80 10 client.30308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.957+0000 7f7e2d1b3b80 20 client.30308 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.957+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fbca3c3cd0 0x55fbca3c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.957+0000 7f7e2d1b3b80 4 client.30308 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:35:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.957+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/2156322969 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fbca266ff0 con 0x55fbca3c3cd0 2023-07-14T16:35:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.957+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fbca3c3cd0 0x55fbca3c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.958+0000 7f7e137fe700 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fbca3c3cd0 0x55fbca3c60e0 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f6eaa2bf72a941 server_cookie=acaa622b997f6160 in_seq=0 out_seq=0 2023-07-14T16:35:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.958+0000 7f7df6ff5700 10 client.30308 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.973+0000 7f7df6ff5700 1 -- 172.21.15.62:0/2156322969 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fbca266ff0 con 0x55fbca3c3cd0 2023-07-14T16:35:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.974+0000 7f7df6ff5700 10 client.30308 handle_command_reply: tid=0 2023-07-14T16:35:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.974+0000 7f7e2d1b3b80 1 client.30308 shutdown 2023-07-14T16:35:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.974+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/2156322969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 msgr2=0x55fbca366870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.974+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca366870 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7e080118f0 tx=0x7f7e0800f6a0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.980+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/2156322969 shutdown_connections 2023-07-14T16:35:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.980+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fbca35b6e0 0x55fbca369890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.980+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55fbca1b66b0 0x55fbca366330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.980+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55fbca3c3cd0 0x55fbca3c60e0 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.980+0000 7f7e2d1b3b80 1 --2- 172.21.15.62:0/2156322969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbca35a840 0x55fbca366870 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.980+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/2156322969 >> 172.21.15.62:0/2156322969 conn(0x55fbca34fbe0 msgr2=0x55fbca361b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.982+0000 7f7df6ff5700 0 client.30308 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.990+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/2156322969 shutdown_connections 2023-07-14T16:35:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.991+0000 7f7e2d1b3b80 1 -- 172.21.15.62:0/2156322969 wait complete. 2023-07-14T16:35:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:35:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:35:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:35:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:35:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.995+0000 7f7e2d1b3b80 20 client.30308 trim_cache size 0 max 16384 2023-07-14T16:35:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.997+0000 7f7e197ff700 1 -- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1220 msgr2=0x7f7dfc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.997+0000 7f7e197ff700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1220 0x7f7dfc0a36d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7e040c9460 tx=0x7f7e04000bc0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.997+0000 7f7e197ff700 1 -- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14070f00 msgr2=0x7f7e141a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:05.997+0000 7f7e197ff700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14070f00 0x7f7e141a47a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400d980 tx=0x7f7e0400dc90 comp rx=0 tx=0).stop 2023-07-14T16:35:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:06.000+0000 7f7e197ff700 1 -- 172.21.15.62:0/285234751 shutdown_connections 2023-07-14T16:35:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:06.000+0000 7f7e197ff700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f7dfc0a1220 0x7f7dfc0a36d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:06.000+0000 7f7e197ff700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e1410bd60 0x7f7e141a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:06.000+0000 7f7e197ff700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e14070f00 0x7f7e141a47a0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:06.000+0000 7f7e197ff700 1 --2- 172.21.15.62:0/285234751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e14071850 0x7f7e141a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:06.000+0000 7f7e197ff700 1 -- 172.21.15.62:0/285234751 >> 172.21.15.62:0/285234751 conn(0x7f7e1406ca50 msgr2=0x7f7e1406e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:06.002+0000 7f7e197ff700 1 -- 172.21.15.62:0/285234751 shutdown_connections 2023-07-14T16:35:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:06.002+0000 7f7e197ff700 1 -- 172.21.15.62:0/285234751 wait complete. 2023-07-14T16:35:07.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:06 smithi164 ceph-mon[97427]: pgmap v941: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2023-07-14T16:35:07.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:06 smithi062 ceph-mon[95543]: pgmap v941: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2023-07-14T16:35:07.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:06 smithi081 ceph-mon[127795]: pgmap v941: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2023-07-14T16:35:09.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:08 smithi164 ceph-mon[97427]: pgmap v942: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:35:09.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:08 smithi062 ceph-mon[95543]: pgmap v942: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:35:09.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:08 smithi081 ceph-mon[127795]: pgmap v942: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:35:11.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:10 smithi164 ceph-mon[97427]: pgmap v943: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.42k op/s 2023-07-14T16:35:11.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:10 smithi062 ceph-mon[95543]: pgmap v943: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.42k op/s 2023-07-14T16:35:11.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:10 smithi081 ceph-mon[127795]: pgmap v943: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.42k op/s 2023-07-14T16:35:13.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:12 smithi164 ceph-mon[97427]: pgmap v944: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:35:13.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:12 smithi062 ceph-mon[95543]: pgmap v944: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:35:13.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:12 smithi081 ceph-mon[127795]: pgmap v944: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:35:15.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:15 smithi164 ceph-mon[97427]: pgmap v945: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:35:15.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:15 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:35:15.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:15 smithi062 ceph-mon[95543]: pgmap v945: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:35:15.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:15 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:35:15.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:15 smithi081 ceph-mon[127795]: pgmap v945: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:35:15.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:15 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:35:16.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:16 smithi062 ceph-mon[95543]: pgmap v946: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2023-07-14T16:35:16.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:16 smithi081 ceph-mon[127795]: pgmap v946: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2023-07-14T16:35:16.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:16 smithi164 ceph-mon[97427]: pgmap v946: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2023-07-14T16:35:19.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:18 smithi062 ceph-mon[95543]: pgmap v947: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.61k op/s 2023-07-14T16:35:19.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:18 smithi081 ceph-mon[127795]: pgmap v947: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.61k op/s 2023-07-14T16:35:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.291+0000 7fe4901ff700 1 -- 172.21.15.62:0/2093073952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488071850 msgr2=0x7fe48810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.291+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2093073952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488071850 0x7fe48810b7d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0098f0 tx=0x7fe47c009c00 comp rx=0 tx=0).stop 2023-07-14T16:35:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.292+0000 7fe4901ff700 1 -- 172.21.15.62:0/2093073952 shutdown_connections 2023-07-14T16:35:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.292+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2093073952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe48810bd60 0x7fe48810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.292+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2093073952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe488070f00 0x7fe488071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.292+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2093073952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488071850 0x7fe48810b7d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.292+0000 7fe4901ff700 1 -- 172.21.15.62:0/2093073952 >> 172.21.15.62:0/2093073952 conn(0x7fe48806ca50 msgr2=0x7fe48806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.292+0000 7fe4901ff700 1 -- 172.21.15.62:0/2093073952 shutdown_connections 2023-07-14T16:35:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.292+0000 7fe4901ff700 1 -- 172.21.15.62:0/2093073952 wait complete. 2023-07-14T16:35:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.294+0000 7fe4901ff700 1 Processor -- start 2023-07-14T16:35:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.294+0000 7fe4901ff700 1 -- start start 2023-07-14T16:35:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe4901ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488070f00 0x7fe488113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe4901ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe488071850 0x7fe488114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe4901ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe48810bd60 0x7fe488114f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe4901ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488115580 con 0x7fe488071850 2023-07-14T16:35:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe4901ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881156f0 con 0x7fe488070f00 2023-07-14T16:35:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe4901ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881158b0 con 0x7fe48810bd60 2023-07-14T16:35:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe48e79c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe48810bd60 0x7fe488114f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe48e79c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe48810bd60 0x7fe488114f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37868/0 (socket says 172.21.15.62:37868) 2023-07-14T16:35:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.295+0000 7fe48e79c700 1 -- 172.21.15.62:0/2792839000 learned_addr learned my addr 172.21.15.62:0/2792839000 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.296+0000 7fe48df9b700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488070f00 0x7fe488113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.271 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:18 smithi164 ceph-mon[97427]: pgmap v947: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 MiB/s wr, 4.61k op/s 2023-07-14T16:35:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.296+0000 7fe48d79a700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe488071850 0x7fe488114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.297+0000 7fe48e79c700 1 -- 172.21.15.62:0/2792839000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488070f00 msgr2=0x7fe488113b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.297+0000 7fe48e79c700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488070f00 0x7fe488113b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.297+0000 7fe48e79c700 1 -- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe488071850 msgr2=0x7fe488114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.297+0000 7fe48e79c700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe488071850 0x7fe488114050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.297+0000 7fe48e79c700 1 -- 172.21.15.62:0/2792839000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c009520 con 0x7fe48810bd60 2023-07-14T16:35:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.297+0000 7fe48e79c700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe48810bd60 0x7fe488114f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe48000e990 tx=0x7fe48000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.298+0000 7fe486ffd700 1 -- 172.21.15.62:0/2792839000 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480004c40 con 0x7fe48810bd60 2023-07-14T16:35:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.298+0000 7fe486ffd700 1 -- 172.21.15.62:0/2792839000 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe480004da0 con 0x7fe48810bd60 2023-07-14T16:35:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.298+0000 7fe486ffd700 1 -- 172.21.15.62:0/2792839000 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480005730 con 0x7fe48810bd60 2023-07-14T16:35:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.299+0000 7fe4901ff700 1 -- 172.21.15.62:0/2792839000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe488115b90 con 0x7fe48810bd60 2023-07-14T16:35:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.299+0000 7fe4901ff700 1 -- 172.21.15.62:0/2792839000 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe488116340 con 0x7fe48810bd60 2023-07-14T16:35:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.304+0000 7fe486ffd700 1 -- 172.21.15.62:0/2792839000 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe48000c910 con 0x7fe48810bd60 2023-07-14T16:35:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.306+0000 7fe4a3b42b80 1 Processor -- start 2023-07-14T16:35:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.308+0000 7fe4a3b42b80 1 -- start start 2023-07-14T16:35:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.308+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9db6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.308+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db6840 0x5581a9db71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.308+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 0x5581a9db7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.308+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9cbbc50 con 0x5581a9db6840 2023-07-14T16:35:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.308+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9d0e9e0 con 0x5581a9db76e0 2023-07-14T16:35:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.308+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9d07e70 con 0x5581a9c126b0 2023-07-14T16:35:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db6840 0x5581a9db71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db6840 0x5581a9db71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53716/0 (socket says 172.21.15.62:53716) 2023-07-14T16:35:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48df9b700 1 -- 172.21.15.62:0/548042165 learned_addr learned my addr 172.21.15.62:0/548042165 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48d79a700 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9db6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48d79a700 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 0x5581a9db7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48df9b700 1 -- 172.21.15.62:0/548042165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 msgr2=0x5581a9db6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48df9b700 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9db6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48df9b700 1 -- 172.21.15.62:0/548042165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 msgr2=0x5581a9db7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48df9b700 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 0x5581a9db7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.309+0000 7fe48df9b700 1 -- 172.21.15.62:0/548042165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a9b35d80 con 0x5581a9db6840 2023-07-14T16:35:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.310+0000 7fe48df9b700 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db6840 0x5581a9db71a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe4780115e0 tx=0x7fe4780099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a8538c8f6c6860 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.310+0000 7fe486ffd700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4700a1350 0x7fe4700a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.310+0000 7fe46ffff700 1 -- 172.21.15.62:0/548042165 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800e040 con 0x5581a9db6840 2023-07-14T16:35:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.310+0000 7fe46ffff700 1 -- 172.21.15.62:0/548042165 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe47800f040 con 0x5581a9db6840 2023-07-14T16:35:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.310+0000 7fe46ffff700 1 -- 172.21.15.62:0/548042165 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478015870 con 0x5581a9db6840 2023-07-14T16:35:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.311+0000 7fe486ffd700 1 -- 172.21.15.62:0/2792839000 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe4800cc7f0 con 0x7fe48810bd60 2023-07-14T16:35:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.311+0000 7fe48d79a700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4700a1350 0x7fe4700a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.311+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/548042165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db6840 msgr2=0x5581a9db71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.311+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db6840 0x5581a9db71a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe4780115e0 tx=0x7fe4780099e0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.312+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/548042165 shutdown_connections 2023-07-14T16:35:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.312+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 0x5581a9db7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.312+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9db6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.312+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/548042165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db6840 0x5581a9db71a0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.312+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/548042165 >> 172.21.15.62:0/548042165 conn(0x5581a9dabbe0 msgr2=0x5581a9db0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.312+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/548042165 shutdown_connections 2023-07-14T16:35:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.312+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/548042165 wait complete. 2023-07-14T16:35:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.313+0000 7fe48d79a700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4700a1350 0x7fe4700a3800 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe488114b60 tx=0x7fe47c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.313+0000 7fe4a3b42b80 1 -- start start 2023-07-14T16:35:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.313+0000 7fe4a3b42b80 1 Processor -- start 2023-07-14T16:35:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.314+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9c126b0 0x5581a9dbf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.314+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9db6840 0x5581a9dbfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.314+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 0x5581a9dc6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.314+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9ca7a60 con 0x5581a9c126b0 2023-07-14T16:35:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.314+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9ca7760 con 0x5581a9db76e0 2023-07-14T16:35:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.314+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9b8b8c0 con 0x5581a9db6840 2023-07-14T16:35:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.314+0000 7fe4a3b42b80 10 client.0 _ll_register_callbacks cb 0x5581a9ca5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.314+0000 7fe48df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9c126b0 0x5581a9dbf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.315+0000 7fe48df9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9c126b0 0x5581a9dbf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53730/0 (socket says 172.21.15.62:53730) 2023-07-14T16:35:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.315+0000 7fe48df9b700 1 -- 172.21.15.62:0/3947975545 learned_addr learned my addr 172.21.15.62:0/3947975545 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.315+0000 7fe48df9b700 1 -- 172.21.15.62:0/3947975545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9db6840 msgr2=0x5581a9dbfda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:35:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.315+0000 7fe48df9b700 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9db6840 0x5581a9dbfda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.315+0000 7fe48df9b700 1 -- 172.21.15.62:0/3947975545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 msgr2=0x5581a9dc6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.315+0000 7fe48df9b700 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 0x5581a9dc6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.315+0000 7fe48df9b700 1 -- 172.21.15.62:0/3947975545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe478011290 con 0x5581a9c126b0 2023-07-14T16:35:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.316+0000 7fe48df9b700 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9c126b0 0x5581a9dbf860 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe4780062a0 tx=0x7fe478007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef718a8050cd2a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.316+0000 7fe46d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:35:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.316+0000 7fe46d7fa700 1 -- 172.21.15.62:0/3947975545 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800e050 con 0x5581a9c126b0 2023-07-14T16:35:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.316+0000 7fe46d7fa700 1 -- 172.21.15.62:0/3947975545 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe478024070 con 0x5581a9c126b0 2023-07-14T16:35:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.316+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3947975545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581a9c61490 con 0x5581a9c126b0 2023-07-14T16:35:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.317+0000 7fe46d7fa700 1 -- 172.21.15.62:0/3947975545 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4780157c0 con 0x5581a9c126b0 2023-07-14T16:35:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.317+0000 7fe4a3b42b80 10 client.20751 fetch_fsmap learned FSMap version 37 2023-07-14T16:35:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.317+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3947975545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5581a9b1ec00 con 0x5581a9c126b0 2023-07-14T16:35:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.317+0000 7fe46d7fa700 1 -- 172.21.15.62:0/3947975545 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5581a9c61490 con 0x5581a9c126b0 2023-07-14T16:35:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.317+0000 7fe46d7fa700 1 -- 172.21.15.62:0/3947975545 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe478029b50 con 0x5581a9c126b0 2023-07-14T16:35:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.318+0000 7fe4a3b42b80 10 client.20751 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T16:35:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.318+0000 7fe4a3b42b80 10 client.20751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.318+0000 7fe4a3b42b80 20 client.20751 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.318+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5581a9dcb580 0x5581a9dcd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.318+0000 7fe4a3b42b80 4 client.20751 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:35:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.318+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3947975545 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581a9d00e50 con 0x5581a9dcb580 2023-07-14T16:35:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.319+0000 7fe48d79a700 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5581a9dcb580 0x5581a9dcd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.321+0000 7fe48d79a700 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5581a9dcb580 0x5581a9dcd990 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1778a78f869f37b5 server_cookie=18d132a93965bc9b in_seq=0 out_seq=0 2023-07-14T16:35:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.321+0000 7fe46d7fa700 10 client.20751 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.322+0000 7fe46d7fa700 1 -- 172.21.15.62:0/3947975545 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5581a9d00e50 con 0x5581a9dcb580 2023-07-14T16:35:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.322+0000 7fe46d7fa700 10 client.20751 handle_command_reply: tid=0 2023-07-14T16:35:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.322+0000 7fe4a3b42b80 1 client.20751 shutdown 2023-07-14T16:35:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.322+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3947975545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9c126b0 msgr2=0x5581a9dbf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.322+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9c126b0 0x5581a9dbf860 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe4780062a0 tx=0x7fe478007dc0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3947975545 shutdown_connections 2023-07-14T16:35:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5581a9dcb580 0x5581a9dcd990 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db76e0 0x5581a9dc6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9c126b0 0x5581a9dbf860 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3947975545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9db6840 0x5581a9dbfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3947975545 >> 172.21.15.62:0/3947975545 conn(0x5581a9dabbe0 msgr2=0x5581a9db04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe46d7fa700 0 client.20751 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3947975545 shutdown_connections 2023-07-14T16:35:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3947975545 wait complete. 2023-07-14T16:35:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.323+0000 7fe4a3b42b80 20 client.20751 trim_cache size 0 max 16384 2023-07-14T16:35:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.355+0000 7fe4a3b42b80 1 Processor -- start 2023-07-14T16:35:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.361+0000 7fe4a3b42b80 1 -- start start 2023-07-14T16:35:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.361+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dd22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.361+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 0x5581a9dc6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.361+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9db0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.361+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9c61490 con 0x5581a9db76e0 2023-07-14T16:35:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.361+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9dbed10 con 0x5581a9db6840 2023-07-14T16:35:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.361+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9dadd00 con 0x5581a9c126b0 2023-07-14T16:35:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9db0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9db0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53736/0 (socket says 172.21.15.62:53736) 2023-07-14T16:35:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48e79c700 1 -- 172.21.15.62:0/3597927646 learned_addr learned my addr 172.21.15.62:0/3597927646 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48df9b700 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dd22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48d79a700 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 0x5581a9dc6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48df9b700 1 -- 172.21.15.62:0/3597927646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 msgr2=0x5581a9dc6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48df9b700 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 0x5581a9dc6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48df9b700 1 -- 172.21.15.62:0/3597927646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 msgr2=0x5581a9db0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48df9b700 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9db0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.362+0000 7fe48df9b700 1 -- 172.21.15.62:0/3597927646 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a9b5c1f0 con 0x5581a9c126b0 2023-07-14T16:35:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.363+0000 7fe48df9b700 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dd22b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe478028610 tx=0x7fe478005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebe18af2995005b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.363+0000 7fe484ff9700 1 -- 172.21.15.62:0/3597927646 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800e030 con 0x5581a9c126b0 2023-07-14T16:35:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.363+0000 7fe484ff9700 1 -- 172.21.15.62:0/3597927646 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe478022070 con 0x5581a9c126b0 2023-07-14T16:35:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.363+0000 7fe484ff9700 1 -- 172.21.15.62:0/3597927646 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478015a20 con 0x5581a9c126b0 2023-07-14T16:35:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.364+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3597927646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 msgr2=0x5581a9dd22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.364+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dd22b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe478028610 tx=0x7fe478005b90 comp rx=0 tx=0).stop 2023-07-14T16:35:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.366+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3597927646 shutdown_connections 2023-07-14T16:35:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.366+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9db0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.366+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dd22b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.366+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/3597927646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 0x5581a9dc6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.366+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3597927646 >> 172.21.15.62:0/3597927646 conn(0x5581a9dabbe0 msgr2=0x5581a9dbe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.366+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3597927646 shutdown_connections 2023-07-14T16:35:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.366+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/3597927646 wait complete. 2023-07-14T16:35:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.367+0000 7fe4a3b42b80 1 -- start start 2023-07-14T16:35:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.367+0000 7fe4a3b42b80 1 Processor -- start 2023-07-14T16:35:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dc2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 0x5581a9dc2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe4a3b42b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9dc5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9dc64e0 con 0x5581a9db76e0 2023-07-14T16:35:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9dc6650 con 0x5581a9db6840 2023-07-14T16:35:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe4a3b42b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9dc6810 con 0x5581a9c126b0 2023-07-14T16:35:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe4a3b42b80 10 client.0 _ll_register_callbacks cb 0x7fe4780114b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe48df9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dc2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe48df9b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dc2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37918/0 (socket says 172.21.15.62:37918) 2023-07-14T16:35:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe48df9b700 1 -- 172.21.15.62:0/215955890 learned_addr learned my addr 172.21.15.62:0/215955890 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:19.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.368+0000 7fe48e79c700 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9dc5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.342 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.369+0000 7fe48d79a700 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 0x5581a9dc2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.369+0000 7fe48df9b700 1 -- 172.21.15.62:0/215955890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 msgr2=0x5581a9dc2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.369+0000 7fe48df9b700 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 0x5581a9dc2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.369+0000 7fe48df9b700 1 -- 172.21.15.62:0/215955890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 msgr2=0x5581a9dc5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.369+0000 7fe48df9b700 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9dc5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.369+0000 7fe48df9b700 1 -- 172.21.15.62:0/215955890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581a9b1ec00 con 0x5581a9c126b0 2023-07-14T16:35:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.370+0000 7fe48df9b700 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dc2a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe478000f80 tx=0x7fe478009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37c67ec7b241ba53 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.370+0000 7fe46d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:35:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.370+0000 7fe46d7fa700 1 -- 172.21.15.62:0/215955890 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800e050 con 0x5581a9c126b0 2023-07-14T16:35:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.370+0000 7fe46d7fa700 1 -- 172.21.15.62:0/215955890 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe478024040 con 0x5581a9c126b0 2023-07-14T16:35:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.370+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/215955890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe47800f040 con 0x5581a9c126b0 2023-07-14T16:35:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.371+0000 7fe46d7fa700 1 -- 172.21.15.62:0/215955890 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe478033a00 con 0x5581a9c126b0 2023-07-14T16:35:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.371+0000 7fe4a3b42b80 10 client.30211 fetch_fsmap learned FSMap version 37 2023-07-14T16:35:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.371+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/215955890 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5581a9b34250 con 0x5581a9c126b0 2023-07-14T16:35:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.372+0000 7fe46d7fa700 1 -- 172.21.15.62:0/215955890 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe47800f040 con 0x5581a9c126b0 2023-07-14T16:35:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.372+0000 7fe46d7fa700 1 -- 172.21.15.62:0/215955890 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe478028640 con 0x5581a9c126b0 2023-07-14T16:35:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.373+0000 7fe4a3b42b80 10 client.30211 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T16:35:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.373+0000 7fe4a3b42b80 10 client.30211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.373+0000 7fe4a3b42b80 20 client.30211 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.373+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5581a9e1fe20 0x5581a9e22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.373+0000 7fe4a3b42b80 4 client.30211 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:35:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.373+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/215955890 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5581a9cc2ff0 con 0x5581a9e1fe20 2023-07-14T16:35:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.373+0000 7fe48d79a700 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5581a9e1fe20 0x5581a9e22230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.374+0000 7fe48d79a700 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5581a9e1fe20 0x5581a9e22230 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f86f4ed0c586516 server_cookie=d3606e77726c49f in_seq=0 out_seq=0 2023-07-14T16:35:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.374+0000 7fe46d7fa700 10 client.30211 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.404+0000 7fe46d7fa700 1 -- 172.21.15.62:0/215955890 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5581a9cc2ff0 con 0x5581a9e1fe20 2023-07-14T16:35:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.404+0000 7fe46d7fa700 10 client.30211 handle_command_reply: tid=0 2023-07-14T16:35:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.404+0000 7fe4a3b42b80 1 client.30211 shutdown 2023-07-14T16:35:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.404+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/215955890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 msgr2=0x5581a9dc2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.404+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dc2a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe478000f80 tx=0x7fe478009ad0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/215955890 shutdown_connections 2023-07-14T16:35:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5581a9db76e0 0x5581a9dc5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5581a9c126b0 0x5581a9dc2a40 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe478000f80 tx=0x7fe478009ad0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5581a9e1fe20 0x5581a9e22230 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe4a3b42b80 1 --2- 172.21.15.62:0/215955890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581a9db6840 0x5581a9dc2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/215955890 >> 172.21.15.62:0/215955890 conn(0x5581a9dabbe0 msgr2=0x5581a9dd22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe46d7fa700 0 client.30211 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/215955890 shutdown_connections 2023-07-14T16:35:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.405+0000 7fe4a3b42b80 1 -- 172.21.15.62:0/215955890 wait complete. 2023-07-14T16:35:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.406+0000 7fe4a3b42b80 20 client.30211 trim_cache size 0 max 16384 2023-07-14T16:35:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:35:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:35:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:35:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:35:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 -- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4700a1350 msgr2=0x7fe4700a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4700a1350 0x7fe4700a3800 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe488114b60 tx=0x7fe47c0058e0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 -- 172.21.15.62:0/2792839000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe48810bd60 msgr2=0x7fe488114f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe48810bd60 0x7fe488114f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe48000e990 tx=0x7fe48000eca0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 -- 172.21.15.62:0/2792839000 shutdown_connections 2023-07-14T16:35:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe48810bd60 0x7fe488114f20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe4700a1350 0x7fe4700a3800 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe488070f00 0x7fe488113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 --2- 172.21.15.62:0/2792839000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe488071850 0x7fe488114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.407+0000 7fe4901ff700 1 -- 172.21.15.62:0/2792839000 >> 172.21.15.62:0/2792839000 conn(0x7fe48806ca50 msgr2=0x7fe48806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.415+0000 7fe4901ff700 1 -- 172.21.15.62:0/2792839000 shutdown_connections 2023-07-14T16:35:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:19.416+0000 7fe4901ff700 1 -- 172.21.15.62:0/2792839000 wait complete. 2023-07-14T16:35:21.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:20 smithi164 ceph-mon[97427]: pgmap v948: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 4.68k op/s 2023-07-14T16:35:21.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:20 smithi062 ceph-mon[95543]: pgmap v948: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 4.68k op/s 2023-07-14T16:35:21.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:20 smithi081 ceph-mon[127795]: pgmap v948: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 4.68k op/s 2023-07-14T16:35:23.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:22 smithi164 ceph-mon[97427]: pgmap v949: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.19k op/s 2023-07-14T16:35:23.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:22 smithi062 ceph-mon[95543]: pgmap v949: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.19k op/s 2023-07-14T16:35:23.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:22 smithi081 ceph-mon[127795]: pgmap v949: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.19k op/s 2023-07-14T16:35:25.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:24 smithi164 ceph-mon[97427]: pgmap v950: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2023-07-14T16:35:25.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:24 smithi062 ceph-mon[95543]: pgmap v950: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2023-07-14T16:35:25.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:24 smithi081 ceph-mon[127795]: pgmap v950: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2023-07-14T16:35:27.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:26 smithi164 ceph-mon[97427]: pgmap v951: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2023-07-14T16:35:27.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:26 smithi062 ceph-mon[95543]: pgmap v951: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2023-07-14T16:35:27.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:26 smithi081 ceph-mon[127795]: pgmap v951: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2023-07-14T16:35:29.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:28 smithi164 ceph-mon[97427]: pgmap v952: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.46k op/s 2023-07-14T16:35:29.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:28 smithi062 ceph-mon[95543]: pgmap v952: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.46k op/s 2023-07-14T16:35:29.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:28 smithi081 ceph-mon[127795]: pgmap v952: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.46k op/s 2023-07-14T16:35:31.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:31 smithi164 ceph-mon[97427]: pgmap v953: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2023-07-14T16:35:31.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:31 smithi062 ceph-mon[95543]: pgmap v953: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2023-07-14T16:35:31.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:31 smithi081 ceph-mon[127795]: pgmap v953: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 MiB/s wr, 4.40k op/s 2023-07-14T16:35:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.674+0000 7fb7055ff700 1 -- 172.21.15.62:0/892678620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb700071850 msgr2=0x7fb70010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.674+0000 7fb7055ff700 1 --2- 172.21.15.62:0/892678620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb700071850 0x7fb70010b7d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009990 tx=0x7fb6e8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.674+0000 7fb7055ff700 1 -- 172.21.15.62:0/892678620 shutdown_connections 2023-07-14T16:35:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.674+0000 7fb7055ff700 1 --2- 172.21.15.62:0/892678620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb70010bd60 0x7fb70010e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.674+0000 7fb7055ff700 1 --2- 172.21.15.62:0/892678620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb700070f00 0x7fb700071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.674+0000 7fb7055ff700 1 --2- 172.21.15.62:0/892678620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb700071850 0x7fb70010b7d0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.674+0000 7fb7055ff700 1 -- 172.21.15.62:0/892678620 >> 172.21.15.62:0/892678620 conn(0x7fb70006ca50 msgr2=0x7fb70006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.675+0000 7fb7055ff700 1 -- 172.21.15.62:0/892678620 shutdown_connections 2023-07-14T16:35:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.675+0000 7fb7055ff700 1 -- 172.21.15.62:0/892678620 wait complete. 2023-07-14T16:35:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.676+0000 7fb7055ff700 1 Processor -- start 2023-07-14T16:35:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.679+0000 7fb7055ff700 1 -- start start 2023-07-14T16:35:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.679+0000 7fb7055ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb700070f00 0x7fb700113c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.679+0000 7fb7055ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb700071850 0x7fb7001141a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.679+0000 7fb7055ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70010bd60 0x7fb700115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.679+0000 7fb7055ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7001156d0 con 0x7fb700070f00 2023-07-14T16:35:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.679+0000 7fb7055ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb700115840 con 0x7fb70010bd60 2023-07-14T16:35:32.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.679+0000 7fb7055ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb700115a00 con 0x7fb700071850 2023-07-14T16:35:32.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.680+0000 7fb6f67fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb700071850 0x7fb7001141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.680+0000 7fb6feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb700070f00 0x7fb700113c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.652 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.680+0000 7fb6f67fc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb700071850 0x7fb7001141a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:58106/0 (socket says 172.21.15.62:58106) 2023-07-14T16:35:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.680+0000 7fb6f67fc700 1 -- 172.21.15.62:0/1033518887 learned_addr learned my addr 172.21.15.62:0/1033518887 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.681+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70010bd60 0x7fb700115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.681+0000 7fb6f67fc700 1 -- 172.21.15.62:0/1033518887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70010bd60 msgr2=0x7fb700115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.681+0000 7fb6f67fc700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70010bd60 0x7fb700115070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.681+0000 7fb6f67fc700 1 -- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb700070f00 msgr2=0x7fb700113c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.681+0000 7fb6f67fc700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb700070f00 0x7fb700113c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.653 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.681+0000 7fb6f67fc700 1 -- 172.21.15.62:0/1033518887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80095f0 con 0x7fb700071850 2023-07-14T16:35:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.681+0000 7fb6f67fc700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb700071850 0x7fb7001141a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8006010 tx=0x7fb6e8003a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.682+0000 7fb6fcff9700 1 -- 172.21.15.62:0/1033518887 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e801d070 con 0x7fb700071850 2023-07-14T16:35:32.654 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.682+0000 7fb6fcff9700 1 -- 172.21.15.62:0/1033518887 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6e8004220 con 0x7fb700071850 2023-07-14T16:35:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.682+0000 7fb6fcff9700 1 -- 172.21.15.62:0/1033518887 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800fe50 con 0x7fb700071850 2023-07-14T16:35:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.682+0000 7fb7055ff700 1 -- 172.21.15.62:0/1033518887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb700115c80 con 0x7fb700071850 2023-07-14T16:35:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.682+0000 7fb7055ff700 1 -- 172.21.15.62:0/1033518887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb700116490 con 0x7fb700071850 2023-07-14T16:35:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.687+0000 7fb718f82b80 1 Processor -- start 2023-07-14T16:35:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.688+0000 7fb718f82b80 1 -- start start 2023-07-14T16:35:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.688+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606139b6b0 0x55606153f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.688+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x5560615401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.688+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x556061540b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.688+0000 7fb718f82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556061444c50 con 0x5560615406e0 2023-07-14T16:35:32.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.688+0000 7fb718f82b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560614979e0 con 0x55606153f840 2023-07-14T16:35:32.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.688+0000 7fb718f82b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556061490e70 con 0x55606139b6b0 2023-07-14T16:35:32.660 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x5560615401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x5560615401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50778/0 (socket says 172.21.15.62:50778) 2023-07-14T16:35:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6feffd700 1 -- 172.21.15.62:0/2923997163 learned_addr learned my addr 172.21.15.62:0/2923997163 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/2923997163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606139b6b0 0x55606153f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6fcff9700 1 -- 172.21.15.62:0/1033518887 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb6e800f5f0 con 0x7fb700071850 2023-07-14T16:35:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6feffd700 1 -- 172.21.15.62:0/2923997163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606139b6b0 msgr2=0x55606153f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6feffd700 1 --2- 172.21.15.62:0/2923997163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606139b6b0 0x55606153f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6feffd700 1 -- 172.21.15.62:0/2923997163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 msgr2=0x556061540b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6feffd700 1 --2- 172.21.15.62:0/2923997163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x556061540b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.689+0000 7fb6feffd700 1 -- 172.21.15.62:0/2923997163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560612bed80 con 0x55606153f840 2023-07-14T16:35:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.690+0000 7fb6feffd700 1 --2- 172.21.15.62:0/2923997163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x5560615401a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b500 tx=0x7fb6f00096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb79c740023ede6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.691+0000 7fb6fcff9700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6ec0a1350 0x7fb6ec0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.691+0000 7fb6fcff9700 1 -- 172.21.15.62:0/1033518887 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6e80cdc80 con 0x7fb700071850 2023-07-14T16:35:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.692+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6ec0a1350 0x7fb6ec0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.693+0000 7fb6f57fa700 1 -- 172.21.15.62:0/2923997163 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0010040 con 0x55606153f840 2023-07-14T16:35:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.693+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6ec0a1350 0x7fb6ec0a3800 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x55606153f060 tx=0x7fb6f800a4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.693+0000 7fb6f57fa700 1 -- 172.21.15.62:0/2923997163 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6f000ac80 con 0x55606153f840 2023-07-14T16:35:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.693+0000 7fb6f57fa700 1 -- 172.21.15.62:0/2923997163 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0014ba0 con 0x55606153f840 2023-07-14T16:35:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.694+0000 7fb718f82b80 1 -- 172.21.15.62:0/2923997163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 msgr2=0x5560615401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.694+0000 7fb718f82b80 1 --2- 172.21.15.62:0/2923997163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x5560615401a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b500 tx=0x7fb6f00096b0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.694+0000 7fb718f82b80 1 -- 172.21.15.62:0/2923997163 shutdown_connections 2023-07-14T16:35:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.694+0000 7fb718f82b80 1 --2- 172.21.15.62:0/2923997163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x556061540b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.694+0000 7fb718f82b80 1 --2- 172.21.15.62:0/2923997163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606139b6b0 0x55606153f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.694+0000 7fb718f82b80 1 --2- 172.21.15.62:0/2923997163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x5560615401a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.694+0000 7fb718f82b80 1 -- 172.21.15.62:0/2923997163 >> 172.21.15.62:0/2923997163 conn(0x556061534be0 msgr2=0x556061539cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.694+0000 7fb718f82b80 1 -- 172.21.15.62:0/2923997163 shutdown_connections 2023-07-14T16:35:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.695+0000 7fb718f82b80 1 -- 172.21.15.62:0/2923997163 wait complete. 2023-07-14T16:35:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.695+0000 7fb718f82b80 1 -- start start 2023-07-14T16:35:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.696+0000 7fb718f82b80 1 Processor -- start 2023-07-14T16:35:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.696+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 0x556061548910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.696+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606153f840 0x556061548e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.696+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 0x55606154fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.696+0000 7fb718f82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556061430a60 con 0x55606153f840 2023-07-14T16:35:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.696+0000 7fb718f82b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556061430760 con 0x55606139b6b0 2023-07-14T16:35:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.696+0000 7fb718f82b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560613148c0 con 0x5560615406e0 2023-07-14T16:35:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.696+0000 7fb718f82b80 10 client.0 _ll_register_callbacks cb 0x55606142e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.698+0000 7fb6feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 0x556061548910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.698+0000 7fb6feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 0x556061548910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50788/0 (socket says 172.21.15.62:50788) 2023-07-14T16:35:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.698+0000 7fb6feffd700 1 -- 172.21.15.62:0/909581308 learned_addr learned my addr 172.21.15.62:0/909581308 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.700+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 0x55606154fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.700+0000 7fb6f67fc700 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606153f840 0x556061548e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.701+0000 7fb6ff7fe700 1 -- 172.21.15.62:0/909581308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 msgr2=0x556061548910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.701+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 0x556061548910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.701+0000 7fb6ff7fe700 1 -- 172.21.15.62:0/909581308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606153f840 msgr2=0x556061548e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.701+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606153f840 0x556061548e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.701+0000 7fb6ff7fe700 1 -- 172.21.15.62:0/909581308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f0009ce0 con 0x5560615406e0 2023-07-14T16:35:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.701+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 0x55606154fe90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80197b0 tx=0x7fb6f8004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7121002916a0c323 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.701+0000 7fb6e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:35:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.702+0000 7fb6e67fc700 1 -- 172.21.15.62:0/909581308 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f801e070 con 0x5560615406e0 2023-07-14T16:35:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.702+0000 7fb6e67fc700 1 -- 172.21.15.62:0/909581308 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6f8017420 con 0x5560615406e0 2023-07-14T16:35:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.702+0000 7fb718f82b80 1 -- 172.21.15.62:0/909581308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560613ea490 con 0x5560615406e0 2023-07-14T16:35:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.703+0000 7fb6e67fc700 1 -- 172.21.15.62:0/909581308 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f80192e0 con 0x5560615406e0 2023-07-14T16:35:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.703+0000 7fb718f82b80 10 client.30217 fetch_fsmap learned FSMap version 37 2023-07-14T16:35:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.703+0000 7fb718f82b80 1 -- 172.21.15.62:0/909581308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5560612a7c00 con 0x5560615406e0 2023-07-14T16:35:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.704+0000 7fb6e67fc700 1 -- 172.21.15.62:0/909581308 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5560613ea490 con 0x5560615406e0 2023-07-14T16:35:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.704+0000 7fb6e67fc700 1 -- 172.21.15.62:0/909581308 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb6f8022610 con 0x5560615406e0 2023-07-14T16:35:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.704+0000 7fb718f82b80 10 client.30217 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T16:35:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.704+0000 7fb718f82b80 10 client.30217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.704+0000 7fb718f82b80 20 client.30217 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.704+0000 7fb718f82b80 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556061554770 0x556061556b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.704+0000 7fb718f82b80 4 client.30217 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:35:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.704+0000 7fb718f82b80 1 -- 172.21.15.62:0/909581308 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556061489e50 con 0x556061554770 2023-07-14T16:35:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.705+0000 7fb6feffd700 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556061554770 0x556061556b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.706+0000 7fb6e67fc700 10 client.30217 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.706+0000 7fb6feffd700 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556061554770 0x556061556b80 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e7274047ea7503 server_cookie=cdd12f1409923b63 in_seq=0 out_seq=0 2023-07-14T16:35:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb6e67fc700 1 -- 172.21.15.62:0/909581308 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556061489e50 con 0x556061554770 2023-07-14T16:35:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb6e67fc700 10 client.30217 handle_command_reply: tid=0 2023-07-14T16:35:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb718f82b80 1 client.30217 shutdown 2023-07-14T16:35:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb718f82b80 1 -- 172.21.15.62:0/909581308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 msgr2=0x55606154fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb718f82b80 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 0x55606154fe90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80197b0 tx=0x7fb6f8004960 comp rx=0 tx=0).stop 2023-07-14T16:35:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb718f82b80 1 -- 172.21.15.62:0/909581308 shutdown_connections 2023-07-14T16:35:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb718f82b80 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 0x55606154fe90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb718f82b80 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556061554770 0x556061556b80 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.707+0000 7fb718f82b80 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 0x556061548910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.708+0000 7fb6e67fc700 0 client.30217 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.708+0000 7fb718f82b80 1 --2- 172.21.15.62:0/909581308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606153f840 0x556061548e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.708+0000 7fb718f82b80 1 -- 172.21.15.62:0/909581308 >> 172.21.15.62:0/909581308 conn(0x556061534be0 msgr2=0x5560615394e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.710+0000 7fb718f82b80 1 -- 172.21.15.62:0/909581308 shutdown_connections 2023-07-14T16:35:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.713+0000 7fb718f82b80 1 -- 172.21.15.62:0/909581308 wait complete. 2023-07-14T16:35:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.713+0000 7fb718f82b80 20 client.30217 trim_cache size 0 max 16384 2023-07-14T16:35:32.710 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.739+0000 7fb718f82b80 1 Processor -- start 2023-07-14T16:35:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.739+0000 7fb718f82b80 1 -- start start 2023-07-14T16:35:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.740+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606139b6b0 0x55606153f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.740+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x55606154fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.740+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 0x556061543630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.740+0000 7fb718f82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556061505090 con 0x55606139b6b0 2023-07-14T16:35:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.740+0000 7fb718f82b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556061547df0 con 0x55606153f840 2023-07-14T16:35:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.740+0000 7fb718f82b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x556061551b00 con 0x5560615406e0 2023-07-14T16:35:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.740+0000 7fb6f67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x55606154fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.741+0000 7fb6f67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x55606154fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:50796/0 (socket says 172.21.15.62:50796) 2023-07-14T16:35:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.741+0000 7fb6f67fc700 1 -- 172.21.15.62:0/1632705667 learned_addr learned my addr 172.21.15.62:0/1632705667 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.741+0000 7fb6f67fc700 1 -- 172.21.15.62:0/1632705667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 msgr2=0x556061543630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:35:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.741+0000 7fb6f67fc700 1 --2- 172.21.15.62:0/1632705667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 0x556061543630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.741+0000 7fb6f67fc700 1 -- 172.21.15.62:0/1632705667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606139b6b0 msgr2=0x55606153f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.741+0000 7fb6f67fc700 1 --2- 172.21.15.62:0/1632705667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606139b6b0 0x55606153f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.741+0000 7fb6f67fc700 1 -- 172.21.15.62:0/1632705667 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560612e51f0 con 0x55606153f840 2023-07-14T16:35:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.742+0000 7fb6f67fc700 1 --2- 172.21.15.62:0/1632705667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x55606154fe50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80cdf20 tx=0x7fb6e80cde90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dddc0871b9f5df5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.742+0000 7fb6f5ffb700 1 -- 172.21.15.62:0/1632705667 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8020370 con 0x55606153f840 2023-07-14T16:35:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.742+0000 7fb6f5ffb700 1 -- 172.21.15.62:0/1632705667 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6e809b0d0 con 0x55606153f840 2023-07-14T16:35:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.742+0000 7fb6f5ffb700 1 -- 172.21.15.62:0/1632705667 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e809b590 con 0x55606153f840 2023-07-14T16:35:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.743+0000 7fb718f82b80 1 -- 172.21.15.62:0/1632705667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 msgr2=0x55606154fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.743+0000 7fb718f82b80 1 --2- 172.21.15.62:0/1632705667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x55606154fe50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80cdf20 tx=0x7fb6e80cde90 comp rx=0 tx=0).stop 2023-07-14T16:35:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.744+0000 7fb718f82b80 1 -- 172.21.15.62:0/1632705667 shutdown_connections 2023-07-14T16:35:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.744+0000 7fb718f82b80 1 --2- 172.21.15.62:0/1632705667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560615406e0 0x556061543630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.744+0000 7fb718f82b80 1 --2- 172.21.15.62:0/1632705667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55606139b6b0 0x55606153f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.744+0000 7fb718f82b80 1 --2- 172.21.15.62:0/1632705667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606153f840 0x55606154fe50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.744+0000 7fb718f82b80 1 -- 172.21.15.62:0/1632705667 >> 172.21.15.62:0/1632705667 conn(0x556061534be0 msgr2=0x5560615475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.744+0000 7fb718f82b80 1 -- 172.21.15.62:0/1632705667 shutdown_connections 2023-07-14T16:35:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.744+0000 7fb718f82b80 1 -- 172.21.15.62:0/1632705667 wait complete. 2023-07-14T16:35:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.744+0000 7fb718f82b80 1 -- start start 2023-07-14T16:35:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.745+0000 7fb718f82b80 1 Processor -- start 2023-07-14T16:35:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.745+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 0x55606154bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.746+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606153f840 0x55606154c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.746+0000 7fb718f82b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x55606154f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.746+0000 7fb718f82b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606154f6d0 con 0x5560615406e0 2023-07-14T16:35:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.746+0000 7fb718f82b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606154f840 con 0x55606139b6b0 2023-07-14T16:35:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.746+0000 7fb718f82b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55606154fa00 con 0x55606153f840 2023-07-14T16:35:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.746+0000 7fb718f82b80 10 client.0 _ll_register_callbacks cb 0x7fb6f8009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.747+0000 7fb6ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x55606154f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.747+0000 7fb6ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x55606154f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40618/0 (socket says 172.21.15.62:40618) 2023-07-14T16:35:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.747+0000 7fb6ff7fe700 1 -- 172.21.15.62:0/558364021 learned_addr learned my addr 172.21.15.62:0/558364021 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.747+0000 7fb6f67fc700 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606153f840 0x55606154c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.747+0000 7fb6ff7fe700 1 -- 172.21.15.62:0/558364021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606153f840 msgr2=0x55606154c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.747+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606153f840 0x55606154c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.747+0000 7fb6ff7fe700 1 -- 172.21.15.62:0/558364021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 msgr2=0x55606154bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.747+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 0x55606154bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.748+0000 7fb6ff7fe700 1 -- 172.21.15.62:0/558364021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb700116490 con 0x5560615406e0 2023-07-14T16:35:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.748+0000 7fb6ff7fe700 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x55606154f190 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8000c00 tx=0x7fb6f80226f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b773e9bb5aa6cf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.748+0000 7fb6e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:35:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.749+0000 7fb6e67fc700 1 -- 172.21.15.62:0/558364021 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f801e070 con 0x5560615406e0 2023-07-14T16:35:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.749+0000 7fb6e67fc700 1 -- 172.21.15.62:0/558364021 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb6f8022c90 con 0x5560615406e0 2023-07-14T16:35:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.749+0000 7fb718f82b80 1 -- 172.21.15.62:0/558364021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6f000ac80 con 0x5560615406e0 2023-07-14T16:35:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.750+0000 7fb6e67fc700 1 -- 172.21.15.62:0/558364021 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f80192e0 con 0x5560615406e0 2023-07-14T16:35:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.750+0000 7fb718f82b80 10 client.20769 fetch_fsmap learned FSMap version 37 2023-07-14T16:35:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.750+0000 7fb718f82b80 1 -- 172.21.15.62:0/558364021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5560612bd250 con 0x5560615406e0 2023-07-14T16:35:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.750+0000 7fb6e67fc700 1 -- 172.21.15.62:0/558364021 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6f000ac80 con 0x5560615406e0 2023-07-14T16:35:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.750+0000 7fb6e67fc700 1 -- 172.21.15.62:0/558364021 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb6f802c600 con 0x5560615406e0 2023-07-14T16:35:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.751+0000 7fb718f82b80 10 client.20769 fetch_fsmap finished waiting for FSMap version 37 2023-07-14T16:35:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.751+0000 7fb718f82b80 10 client.20769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.751+0000 7fb718f82b80 20 client.20769 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.751+0000 7fb718f82b80 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5560615a9660 0x5560615aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.751+0000 7fb718f82b80 4 client.20769 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:35:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.751+0000 7fb718f82b80 1 -- 172.21.15.62:0/558364021 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55606144bff0 con 0x5560615a9660 2023-07-14T16:35:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.751+0000 7fb6feffd700 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5560615a9660 0x5560615aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.752+0000 7fb6feffd700 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5560615a9660 0x5560615aba70 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f06dfeadcbc9c0 server_cookie=e858f27d2dcfcf99 in_seq=0 out_seq=0 2023-07-14T16:35:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.752+0000 7fb6e67fc700 10 client.20769 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.770+0000 7fb6e67fc700 1 -- 172.21.15.62:0/558364021 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55606144bff0 con 0x5560615a9660 2023-07-14T16:35:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.770+0000 7fb6e67fc700 10 client.20769 handle_command_reply: tid=0 2023-07-14T16:35:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.771+0000 7fb718f82b80 1 client.20769 shutdown 2023-07-14T16:35:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.775+0000 7fb718f82b80 1 -- 172.21.15.62:0/558364021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 msgr2=0x55606154f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.775+0000 7fb718f82b80 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x55606154f190 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8000c00 tx=0x7fb6f80226f0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.776+0000 7fb718f82b80 1 -- 172.21.15.62:0/558364021 shutdown_connections 2023-07-14T16:35:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.776+0000 7fb718f82b80 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560615406e0 0x55606154f190 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.776+0000 7fb718f82b80 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5560615a9660 0x5560615aba70 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.776+0000 7fb718f82b80 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55606139b6b0 0x55606154bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.776+0000 7fb718f82b80 1 --2- 172.21.15.62:0/558364021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55606153f840 0x55606154c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.776+0000 7fb718f82b80 1 -- 172.21.15.62:0/558364021 >> 172.21.15.62:0/558364021 conn(0x556061534be0 msgr2=0x556061543630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.776+0000 7fb6e67fc700 0 client.20769 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.776+0000 7fb718f82b80 1 -- 172.21.15.62:0/558364021 shutdown_connections 2023-07-14T16:35:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.777+0000 7fb718f82b80 1 -- 172.21.15.62:0/558364021 wait complete. 2023-07-14T16:35:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.777+0000 7fb718f82b80 20 client.20769 trim_cache size 0 max 16384 2023-07-14T16:35:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:35:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:35:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:35:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:35:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 -- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6ec0a1350 msgr2=0x7fb6ec0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6ec0a1350 0x7fb6ec0a3800 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x55606153f060 tx=0x7fb6f800a4a0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 -- 172.21.15.62:0/1033518887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb700071850 msgr2=0x7fb7001141a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb700071850 0x7fb7001141a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8006010 tx=0x7fb6e8003a60 comp rx=0 tx=0).stop 2023-07-14T16:35:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 -- 172.21.15.62:0/1033518887 shutdown_connections 2023-07-14T16:35:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb6ec0a1350 0x7fb6ec0a3800 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb70010bd60 0x7fb700115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb700070f00 0x7fb700113c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 --2- 172.21.15.62:0/1033518887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb700071850 0x7fb7001141a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 -- 172.21.15.62:0/1033518887 >> 172.21.15.62:0/1033518887 conn(0x7fb70006ca50 msgr2=0x7fb70006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 -- 172.21.15.62:0/1033518887 shutdown_connections 2023-07-14T16:35:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:32.779+0000 7fb7055ff700 1 -- 172.21.15.62:0/1033518887 wait complete. 2023-07-14T16:35:33.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:33 smithi164 ceph-mon[97427]: pgmap v954: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2023-07-14T16:35:33.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:33 smithi062 ceph-mon[95543]: pgmap v954: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2023-07-14T16:35:33.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:33 smithi081 ceph-mon[127795]: pgmap v954: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2023-07-14T16:35:35.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:35 smithi164 ceph-mon[97427]: pgmap v955: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:35:35.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:35 smithi062 ceph-mon[95543]: pgmap v955: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:35:35.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:35 smithi081 ceph-mon[127795]: pgmap v955: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.48k op/s 2023-07-14T16:35:37.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:37 smithi164 ceph-mon[97427]: pgmap v956: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2023-07-14T16:35:37.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:37 smithi062 ceph-mon[95543]: pgmap v956: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2023-07-14T16:35:37.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:37 smithi081 ceph-mon[127795]: pgmap v956: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2023-07-14T16:35:39.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:39 smithi164 ceph-mon[97427]: pgmap v957: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:35:39.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:39 smithi062 ceph-mon[95543]: pgmap v957: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:35:39.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:39 smithi081 ceph-mon[127795]: pgmap v957: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.30k op/s 2023-07-14T16:35:41.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:41 smithi062 ceph-mon[95543]: pgmap v958: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:35:41.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:41 smithi081 ceph-mon[127795]: pgmap v958: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:35:41.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:41 smithi164 ceph-mon[97427]: pgmap v958: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:35:42.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:42 smithi062 ceph-mon[95543]: pgmap v959: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2023-07-14T16:35:42.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:42 smithi081 ceph-mon[127795]: pgmap v959: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2023-07-14T16:35:42.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:42 smithi164 ceph-mon[97427]: pgmap v959: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.03k op/s 2023-07-14T16:35:44.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:44 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:35:44.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:44 smithi062 ceph-mon[95543]: pgmap v960: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:35:44.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:44 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:35:44.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:44 smithi081 ceph-mon[127795]: pgmap v960: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:35:44.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:44 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:35:44.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:44 smithi164 ceph-mon[97427]: pgmap v960: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 MiB/s wr, 4.72k op/s 2023-07-14T16:35:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.061+0000 7fed7e3ff700 1 -- 172.21.15.62:0/2182921995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 msgr2=0x7fed7810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.061+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/2182921995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 0x7fed7810b7d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0098f0 tx=0x7fed6c009c00 comp rx=0 tx=0).stop 2023-07-14T16:35:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.061+0000 7fed7e3ff700 1 -- 172.21.15.62:0/2182921995 shutdown_connections 2023-07-14T16:35:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.061+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/2182921995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed7810bd60 0x7fed7810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.061+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/2182921995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed78070f00 0x7fed78071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.061+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/2182921995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 0x7fed7810b7d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.061+0000 7fed7e3ff700 1 -- 172.21.15.62:0/2182921995 >> 172.21.15.62:0/2182921995 conn(0x7fed7806ca50 msgr2=0x7fed7806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.062+0000 7fed7e3ff700 1 -- 172.21.15.62:0/2182921995 shutdown_connections 2023-07-14T16:35:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.062+0000 7fed7e3ff700 1 -- 172.21.15.62:0/2182921995 wait complete. 2023-07-14T16:35:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.064+0000 7fed7e3ff700 1 Processor -- start 2023-07-14T16:35:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.064+0000 7fed7e3ff700 1 -- start start 2023-07-14T16:35:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed7e3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed78070f00 0x7fed781a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed7e3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 0x7fed781a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed7e3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed7810bd60 0x7fed781a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed7e3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a6170 con 0x7fed7810bd60 2023-07-14T16:35:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed7e3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a62e0 con 0x7fed78070f00 2023-07-14T16:35:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed7e3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a64a0 con 0x7fed78071850 2023-07-14T16:35:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed777fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 0x7fed781a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed777fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 0x7fed781a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37082/0 (socket says 172.21.15.62:37082) 2023-07-14T16:35:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed777fe700 1 -- 172.21.15.62:0/1771604823 learned_addr learned my addr 172.21.15.62:0/1771604823 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.065+0000 7fed7c99c700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed7810bd60 0x7fed781a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.066+0000 7fed77fff700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed78070f00 0x7fed781a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.066+0000 7fed777fe700 1 -- 172.21.15.62:0/1771604823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed78070f00 msgr2=0x7fed781a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.066+0000 7fed777fe700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed78070f00 0x7fed781a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.066+0000 7fed777fe700 1 -- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed7810bd60 msgr2=0x7fed781a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.066+0000 7fed777fe700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed7810bd60 0x7fed781a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.066+0000 7fed777fe700 1 -- 172.21.15.62:0/1771604823 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c009520 con 0x7fed78071850 2023-07-14T16:35:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.066+0000 7fed7c99c700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed7810bd60 0x7fed781a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:35:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.069+0000 7fed777fe700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 0x7fed781a4c90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00faa0 tx=0x7fed6c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.070+0000 7fed757fa700 1 -- 172.21.15.62:0/1771604823 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c01c070 con 0x7fed78071850 2023-07-14T16:35:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.070+0000 7fed757fa700 1 -- 172.21.15.62:0/1771604823 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed6c005500 con 0x7fed78071850 2023-07-14T16:35:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.070+0000 7fed757fa700 1 -- 172.21.15.62:0/1771604823 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c003e80 con 0x7fed78071850 2023-07-14T16:35:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.070+0000 7fed7e3ff700 1 -- 172.21.15.62:0/1771604823 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed781a6720 con 0x7fed78071850 2023-07-14T16:35:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.071+0000 7fed7e3ff700 1 -- 172.21.15.62:0/1771604823 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7810eea0 con 0x7fed78071850 2023-07-14T16:35:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.077+0000 7fed757fa700 1 -- 172.21.15.62:0/1771604823 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fed6c003680 con 0x7fed78071850 2023-07-14T16:35:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.077+0000 7fed91d10b80 1 Processor -- start 2023-07-14T16:35:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.078+0000 7fed91d10b80 1 -- start start 2023-07-14T16:35:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.079+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ec776b0 0x55686ee1b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.079+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1b840 0x55686ee1c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.079+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1c6e0 0x55686ee1cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.079+0000 7fed91d10b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ed20c50 con 0x55686ee1c6e0 2023-07-14T16:35:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.079+0000 7fed91d10b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ed739e0 con 0x55686ee1b840 2023-07-14T16:35:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.079+0000 7fed91d10b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ed6ce70 con 0x55686ec776b0 2023-07-14T16:35:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed77fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1b840 0x55686ee1c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed77fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1b840 0x55686ee1c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40558/0 (socket says 172.21.15.62:40558) 2023-07-14T16:35:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed77fff700 1 -- 172.21.15.62:0/3226458581 learned_addr learned my addr 172.21.15.62:0/3226458581 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed757fa700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed600a12d0 0x7fed600a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed777fe700 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1c6e0 0x55686ee1cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed77fff700 1 -- 172.21.15.62:0/3226458581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ec776b0 msgr2=0x55686ee1b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed77fff700 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ec776b0 0x55686ee1b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed77fff700 1 -- 172.21.15.62:0/3226458581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1c6e0 msgr2=0x55686ee1cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed77fff700 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1c6e0 0x55686ee1cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed77fff700 1 -- 172.21.15.62:0/3226458581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55686eb9ad80 con 0x55686ee1b840 2023-07-14T16:35:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.080+0000 7fed757fa700 1 -- 172.21.15.62:0/1771604823 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed6c0cdd60 con 0x7fed78071850 2023-07-14T16:35:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.081+0000 7fed777fe700 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1c6e0 0x55686ee1cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:35:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.081+0000 7fed77fff700 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1b840 0x55686ee1c1a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fed6800a9a0 tx=0x7fed6800acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a9bff4344f07e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.081+0000 7fed7c99c700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed600a12d0 0x7fed600a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.081+0000 7fed5e7fc700 1 -- 172.21.15.62:0/3226458581 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800ca90 con 0x55686ee1b840 2023-07-14T16:35:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.081+0000 7fed5e7fc700 1 -- 172.21.15.62:0/3226458581 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed6800cbf0 con 0x55686ee1b840 2023-07-14T16:35:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.081+0000 7fed5e7fc700 1 -- 172.21.15.62:0/3226458581 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68018360 con 0x55686ee1b840 2023-07-14T16:35:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.082+0000 7fed91d10b80 1 -- 172.21.15.62:0/3226458581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1b840 msgr2=0x55686ee1c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.082+0000 7fed91d10b80 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1b840 0x55686ee1c1a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fed6800a9a0 tx=0x7fed6800acb0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.083+0000 7fed7c99c700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed600a12d0 0x7fed600a3780 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fed70009610 tx=0x7fed7000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.083+0000 7fed91d10b80 1 -- 172.21.15.62:0/3226458581 shutdown_connections 2023-07-14T16:35:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.083+0000 7fed91d10b80 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1c6e0 0x55686ee1cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.083+0000 7fed91d10b80 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ec776b0 0x55686ee1b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.083+0000 7fed91d10b80 1 --2- 172.21.15.62:0/3226458581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1b840 0x55686ee1c1a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.083+0000 7fed91d10b80 1 -- 172.21.15.62:0/3226458581 >> 172.21.15.62:0/3226458581 conn(0x55686ee10be0 msgr2=0x55686ee15cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.083+0000 7fed91d10b80 1 -- 172.21.15.62:0/3226458581 shutdown_connections 2023-07-14T16:35:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.083+0000 7fed91d10b80 1 -- 172.21.15.62:0/3226458581 wait complete. 2023-07-14T16:35:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.084+0000 7fed91d10b80 1 -- start start 2023-07-14T16:35:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.085+0000 7fed91d10b80 1 Processor -- start 2023-07-14T16:35:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.086+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ec776b0 0x55686ee24950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.086+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1b840 0x55686ee24e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.086+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee2bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.086+0000 7fed91d10b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ed0ca60 con 0x55686ec776b0 2023-07-14T16:35:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.086+0000 7fed91d10b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ed0c760 con 0x55686ee1c6e0 2023-07-14T16:35:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.086+0000 7fed91d10b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ebf08c0 con 0x55686ee1b840 2023-07-14T16:35:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.086+0000 7fed91d10b80 10 client.0 _ll_register_callbacks cb 0x55686ed0a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.087+0000 7fed7c99c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee2bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.087+0000 7fed7c99c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee2bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40574/0 (socket says 172.21.15.62:40574) 2023-07-14T16:35:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.087+0000 7fed7c99c700 1 -- 172.21.15.62:0/2864262510 learned_addr learned my addr 172.21.15.62:0/2864262510 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed77fff700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ec776b0 0x55686ee24950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed777fe700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1b840 0x55686ee24e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed7c99c700 1 -- 172.21.15.62:0/2864262510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1b840 msgr2=0x55686ee24e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed7c99c700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1b840 0x55686ee24e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed7c99c700 1 -- 172.21.15.62:0/2864262510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ec776b0 msgr2=0x55686ee24950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed7c99c700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ec776b0 0x55686ee24950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed7c99c700 1 -- 172.21.15.62:0/2864262510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6800a5d0 con 0x55686ee1c6e0 2023-07-14T16:35:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed777fe700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1b840 0x55686ee24e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:35:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed77fff700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ec776b0 0x55686ee24950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:35:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed7c99c700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee2bed0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fed700166d0 tx=0x7fed70004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cecde9bf2627c01 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.088+0000 7fed5bff7700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:35:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.089+0000 7fed5bff7700 1 -- 172.21.15.62:0/2864262510 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70021600 con 0x55686ee1c6e0 2023-07-14T16:35:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.089+0000 7fed5bff7700 1 -- 172.21.15.62:0/2864262510 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed70021760 con 0x55686ee1c6e0 2023-07-14T16:35:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.089+0000 7fed91d10b80 1 -- 172.21.15.62:0/2864262510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55686ecc6490 con 0x55686ee1c6e0 2023-07-14T16:35:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.089+0000 7fed5bff7700 1 -- 172.21.15.62:0/2864262510 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed70007e60 con 0x55686ee1c6e0 2023-07-14T16:35:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.089+0000 7fed91d10b80 10 client.30344 fetch_fsmap learned FSMap version 38 2023-07-14T16:35:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.089+0000 7fed91d10b80 1 -- 172.21.15.62:0/2864262510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55686eb83c00 con 0x55686ee1c6e0 2023-07-14T16:35:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.090+0000 7fed5bff7700 1 -- 172.21.15.62:0/2864262510 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55686ecc6490 con 0x55686ee1c6e0 2023-07-14T16:35:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.090+0000 7fed5bff7700 1 -- 172.21.15.62:0/2864262510 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed70029070 con 0x55686ee1c6e0 2023-07-14T16:35:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.090+0000 7fed91d10b80 10 client.30344 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T16:35:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.090+0000 7fed91d10b80 10 client.30344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.090+0000 7fed91d10b80 20 client.30344 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.090+0000 7fed91d10b80 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55686ee30630 0x55686ee32a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.090+0000 7fed91d10b80 4 client.30344 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:35:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.090+0000 7fed91d10b80 1 -- 172.21.15.62:0/2864262510 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55686ed65e50 con 0x55686ee30630 2023-07-14T16:35:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.091+0000 7fed77fff700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55686ee30630 0x55686ee32a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.092+0000 7fed5bff7700 10 client.30344 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.092+0000 7fed77fff700 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55686ee30630 0x55686ee32a40 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca2ea45dcf9abb67 server_cookie=b12edd68b2db3ca4 in_seq=0 out_seq=0 2023-07-14T16:35:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.093+0000 7fed5bff7700 1 -- 172.21.15.62:0/2864262510 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55686ed65e50 con 0x55686ee30630 2023-07-14T16:35:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.093+0000 7fed5bff7700 10 client.30344 handle_command_reply: tid=0 2023-07-14T16:35:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.093+0000 7fed91d10b80 1 client.30344 shutdown 2023-07-14T16:35:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.095+0000 7fed91d10b80 1 -- 172.21.15.62:0/2864262510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 msgr2=0x55686ee2bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.095+0000 7fed91d10b80 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee2bed0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fed700166d0 tx=0x7fed70004d50 comp rx=0 tx=0).stop 2023-07-14T16:35:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed91d10b80 1 -- 172.21.15.62:0/2864262510 shutdown_connections 2023-07-14T16:35:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed91d10b80 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55686ee30630 0x55686ee32a40 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed91d10b80 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee2bed0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed91d10b80 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ec776b0 0x55686ee24950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed91d10b80 1 --2- 172.21.15.62:0/2864262510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1b840 0x55686ee24e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed5bff7700 0 client.30344 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed91d10b80 1 -- 172.21.15.62:0/2864262510 >> 172.21.15.62:0/2864262510 conn(0x55686ee10be0 msgr2=0x55686ee154e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed91d10b80 1 -- 172.21.15.62:0/2864262510 shutdown_connections 2023-07-14T16:35:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.096+0000 7fed91d10b80 1 -- 172.21.15.62:0/2864262510 wait complete. 2023-07-14T16:35:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.097+0000 7fed91d10b80 20 client.30344 trim_cache size 0 max 16384 2023-07-14T16:35:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.126+0000 7fed91d10b80 1 Processor -- start 2023-07-14T16:35:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.130+0000 7fed91d10b80 1 -- start start 2023-07-14T16:35:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.130+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ec776b0 0x55686ee22a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.130+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 0x55686ee22fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.130+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1c6e0 0x55686ee2be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.130+0000 7fed91d10b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ede1060 con 0x55686ee1b840 2023-07-14T16:35:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.130+0000 7fed91d10b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ee23e00 con 0x55686ec776b0 2023-07-14T16:35:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.130+0000 7fed91d10b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ee244c0 con 0x55686ee1c6e0 2023-07-14T16:35:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed77fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ec776b0 0x55686ee22a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed77fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ec776b0 0x55686ee22a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:40578/0 (socket says 172.21.15.62:40578) 2023-07-14T16:35:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed77fff700 1 -- 172.21.15.62:0/122322017 learned_addr learned my addr 172.21.15.62:0/122322017 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed7c99c700 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1c6e0 0x55686ee2be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed777fe700 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 0x55686ee22fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed77fff700 1 -- 172.21.15.62:0/122322017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1c6e0 msgr2=0x55686ee2be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed77fff700 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1c6e0 0x55686ee2be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed77fff700 1 -- 172.21.15.62:0/122322017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 msgr2=0x55686ee22fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed77fff700 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 0x55686ee22fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.131+0000 7fed77fff700 1 -- 172.21.15.62:0/122322017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55686ebc11f0 con 0x55686ec776b0 2023-07-14T16:35:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.132+0000 7fed77fff700 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ec776b0 0x55686ee22a90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fed6800afd0 tx=0x7fed6801c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=790055a925f043dc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.132+0000 7fed777fe700 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 0x55686ee22fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:35:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.132+0000 7fed5effd700 1 -- 172.21.15.62:0/122322017 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6801f040 con 0x55686ec776b0 2023-07-14T16:35:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.132+0000 7fed5effd700 1 -- 172.21.15.62:0/122322017 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed6802c020 con 0x55686ec776b0 2023-07-14T16:35:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.132+0000 7fed5effd700 1 -- 172.21.15.62:0/122322017 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68028330 con 0x55686ec776b0 2023-07-14T16:35:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.134+0000 7fed91d10b80 1 -- 172.21.15.62:0/122322017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ec776b0 msgr2=0x55686ee22a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.134+0000 7fed91d10b80 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ec776b0 0x55686ee22a90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fed6800afd0 tx=0x7fed6801c040 comp rx=0 tx=0).stop 2023-07-14T16:35:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.135+0000 7fed91d10b80 1 -- 172.21.15.62:0/122322017 shutdown_connections 2023-07-14T16:35:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.135+0000 7fed91d10b80 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee1c6e0 0x55686ee2be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.135+0000 7fed91d10b80 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ec776b0 0x55686ee22a90 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fed6800afd0 tx=0x7fed6801c040 comp rx=0 tx=0).stop 2023-07-14T16:35:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.135+0000 7fed91d10b80 1 --2- 172.21.15.62:0/122322017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 0x55686ee22fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.135+0000 7fed91d10b80 1 -- 172.21.15.62:0/122322017 >> 172.21.15.62:0/122322017 conn(0x55686ee10be0 msgr2=0x55686ee23670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.136+0000 7fed91d10b80 1 -- 172.21.15.62:0/122322017 shutdown_connections 2023-07-14T16:35:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.136+0000 7fed91d10b80 1 -- 172.21.15.62:0/122322017 wait complete. 2023-07-14T16:35:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.137+0000 7fed91d10b80 1 -- start start 2023-07-14T16:35:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.137+0000 7fed91d10b80 1 Processor -- start 2023-07-14T16:35:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 0x55686ee27dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee28300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed91d10b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee2b320 0x55686ee2b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed91d10b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ee2bcd0 con 0x55686ee1b840 2023-07-14T16:35:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed91d10b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ee2be40 con 0x55686ee1c6e0 2023-07-14T16:35:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed91d10b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55686ee2c000 con 0x55686ee2b320 2023-07-14T16:35:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed91d10b80 10 client.0 _ll_register_callbacks cb 0x7fed70008ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed7c99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee2b320 0x55686ee2b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed7c99c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee2b320 0x55686ee2b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:37116/0 (socket says 172.21.15.62:37116) 2023-07-14T16:35:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.138+0000 7fed7c99c700 1 -- 172.21.15.62:0/1193186342 learned_addr learned my addr 172.21.15.62:0/1193186342 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.139+0000 7fed7c99c700 1 -- 172.21.15.62:0/1193186342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 msgr2=0x55686ee28300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.139+0000 7fed7c99c700 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee28300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.139+0000 7fed7c99c700 1 -- 172.21.15.62:0/1193186342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 msgr2=0x55686ee27dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:35:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.139+0000 7fed7c99c700 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 0x55686ee27dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.139+0000 7fed7c99c700 1 -- 172.21.15.62:0/1193186342 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55686eb9ad80 con 0x55686ee2b320 2023-07-14T16:35:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.140+0000 7fed7c99c700 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee2b320 0x55686ee2b790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fed70000f80 tx=0x7fed70029600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fc8cce790e665bb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.140+0000 7fed5bff7700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:35:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.140+0000 7fed5bff7700 1 -- 172.21.15.62:0/1193186342 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70029960 con 0x55686ee2b320 2023-07-14T16:35:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.141+0000 7fed5bff7700 1 -- 172.21.15.62:0/1193186342 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed70029ac0 con 0x55686ee2b320 2023-07-14T16:35:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.141+0000 7fed91d10b80 1 -- 172.21.15.62:0/1193186342 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed6800cbf0 con 0x55686ee2b320 2023-07-14T16:35:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.141+0000 7fed5bff7700 1 -- 172.21.15.62:0/1193186342 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed70008b10 con 0x55686ee2b320 2023-07-14T16:35:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.142+0000 7fed91d10b80 10 client.30232 fetch_fsmap learned FSMap version 38 2023-07-14T16:35:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.142+0000 7fed91d10b80 1 -- 172.21.15.62:0/1193186342 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55686eb99250 con 0x55686ee2b320 2023-07-14T16:35:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.142+0000 7fed5bff7700 1 -- 172.21.15.62:0/1193186342 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed6800cbf0 con 0x55686ee2b320 2023-07-14T16:35:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.142+0000 7fed5bff7700 1 -- 172.21.15.62:0/1193186342 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed700167b0 con 0x55686ee2b320 2023-07-14T16:35:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.143+0000 7fed91d10b80 10 client.30232 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T16:35:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.143+0000 7fed91d10b80 10 client.30232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.143+0000 7fed91d10b80 20 client.30232 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.143+0000 7fed91d10b80 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55686ee859b0 0x55686ee87e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.143+0000 7fed91d10b80 4 client.30232 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:35:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.143+0000 7fed91d10b80 1 -- 172.21.15.62:0/1193186342 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55686ed27ff0 con 0x55686ee859b0 2023-07-14T16:35:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.143+0000 7fed77fff700 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55686ee859b0 0x55686ee87e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.144+0000 7fed77fff700 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55686ee859b0 0x55686ee87e40 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43a10e6af72925e server_cookie=abef7d49c059914c in_seq=0 out_seq=0 2023-07-14T16:35:46.118 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.144+0000 7fed5bff7700 10 client.30232 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.154+0000 7fed5bff7700 1 -- 172.21.15.62:0/1193186342 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55686ed27ff0 con 0x55686ee859b0 2023-07-14T16:35:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.154+0000 7fed5bff7700 10 client.30232 handle_command_reply: tid=0 2023-07-14T16:35:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.154+0000 7fed91d10b80 1 client.30232 shutdown 2023-07-14T16:35:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.166+0000 7fed91d10b80 1 -- 172.21.15.62:0/1193186342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee2b320 msgr2=0x55686ee2b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.166+0000 7fed91d10b80 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee2b320 0x55686ee2b790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fed70000f80 tx=0x7fed70029600 comp rx=0 tx=0).stop 2023-07-14T16:35:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.167+0000 7fed91d10b80 1 -- 172.21.15.62:0/1193186342 shutdown_connections 2023-07-14T16:35:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.167+0000 7fed91d10b80 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55686ee2b320 0x55686ee2b790 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.167+0000 7fed91d10b80 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55686ee1b840 0x55686ee27dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.167+0000 7fed91d10b80 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55686ee859b0 0x55686ee87e40 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.167+0000 7fed91d10b80 1 --2- 172.21.15.62:0/1193186342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55686ee1c6e0 0x55686ee28300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.167+0000 7fed91d10b80 1 -- 172.21.15.62:0/1193186342 >> 172.21.15.62:0/1193186342 conn(0x55686ee10be0 msgr2=0x55686ee1f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.168+0000 7fed5bff7700 0 client.30232 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.169+0000 7fed91d10b80 1 -- 172.21.15.62:0/1193186342 shutdown_connections 2023-07-14T16:35:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.169+0000 7fed91d10b80 1 -- 172.21.15.62:0/1193186342 wait complete. 2023-07-14T16:35:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.172+0000 7fed91d10b80 20 client.30232 trim_cache size 0 max 16384 2023-07-14T16:35:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:35:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:35:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:35:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:35:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 -- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed600a12d0 msgr2=0x7fed600a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed600a12d0 0x7fed600a3780 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fed70009610 tx=0x7fed7000b040 comp rx=0 tx=0).stop 2023-07-14T16:35:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 -- 172.21.15.62:0/1771604823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 msgr2=0x7fed781a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 0x7fed781a4c90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00faa0 tx=0x7fed6c00b8f0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 -- 172.21.15.62:0/1771604823 shutdown_connections 2023-07-14T16:35:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed7810bd60 0x7fed781a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fed600a12d0 0x7fed600a3780 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed78070f00 0x7fed781a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 --2- 172.21.15.62:0/1771604823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed78071850 0x7fed781a4c90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.173+0000 7fed7e3ff700 1 -- 172.21.15.62:0/1771604823 >> 172.21.15.62:0/1771604823 conn(0x7fed7806ca50 msgr2=0x7fed7806e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.174+0000 7fed7e3ff700 1 -- 172.21.15.62:0/1771604823 shutdown_connections 2023-07-14T16:35:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:46.174+0000 7fed7e3ff700 1 -- 172.21.15.62:0/1771604823 wait complete. 2023-07-14T16:35:47.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:46 smithi081 ceph-mon[127795]: pgmap v961: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:35:47.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:46 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:35:47.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:46 smithi164 ceph-mon[97427]: pgmap v961: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:35:47.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:46 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:35:47.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:46 smithi062 ceph-mon[95543]: pgmap v961: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:35:47.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:46 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:35:49.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:48 smithi081 ceph-mon[127795]: pgmap v962: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 4.55k op/s 2023-07-14T16:35:49.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:35:49.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:35:49.146 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:35:49.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:48 smithi164 ceph-mon[97427]: pgmap v962: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 4.55k op/s 2023-07-14T16:35:49.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:35:49.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:35:49.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:35:49.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:48 smithi062 ceph-mon[95543]: pgmap v962: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 4.55k op/s 2023-07-14T16:35:49.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:35:49.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:35:49.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:35:51.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:50 smithi164 ceph-mon[97427]: pgmap v963: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:35:51.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:50 smithi062 ceph-mon[95543]: pgmap v963: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:35:51.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:50 smithi081 ceph-mon[127795]: pgmap v963: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:35:53.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:52 smithi164 ceph-mon[97427]: pgmap v964: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2023-07-14T16:35:53.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:52 smithi062 ceph-mon[95543]: pgmap v964: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2023-07-14T16:35:53.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:52 smithi081 ceph-mon[127795]: pgmap v964: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2023-07-14T16:35:55.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:54 smithi164 ceph-mon[97427]: pgmap v965: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 4.63k op/s 2023-07-14T16:35:55.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:54 smithi062 ceph-mon[95543]: pgmap v965: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 4.63k op/s 2023-07-14T16:35:55.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:54 smithi081 ceph-mon[127795]: pgmap v965: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 4.63k op/s 2023-07-14T16:35:57.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:56 smithi164 ceph-mon[97427]: pgmap v966: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.87k op/s 2023-07-14T16:35:57.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:56 smithi062 ceph-mon[95543]: pgmap v966: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.87k op/s 2023-07-14T16:35:57.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:56 smithi081 ceph-mon[127795]: pgmap v966: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.87k op/s 2023-07-14T16:35:59.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:35:59 smithi164 ceph-mon[97427]: pgmap v967: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 4.50k op/s 2023-07-14T16:35:59.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:35:58 smithi062 ceph-mon[95543]: pgmap v967: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 4.50k op/s 2023-07-14T16:35:59.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:35:59 smithi081 ceph-mon[127795]: pgmap v967: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 4.50k op/s 2023-07-14T16:35:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.739+0000 7f40fa1ff700 1 -- 172.21.15.62:0/2119614001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107340 msgr2=0x7f40f4107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.739+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/2119614001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107340 0x7f40f4107790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f40e40098f0 tx=0x7f40e4009c00 comp rx=0 tx=0).stop 2023-07-14T16:35:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.740+0000 7f40fa1ff700 1 -- 172.21.15.62:0/2119614001 shutdown_connections 2023-07-14T16:35:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.740+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/2119614001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40f4107cd0 0x7f40f410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.740+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/2119614001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f4106150 0x7f40f4106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.740+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/2119614001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107340 0x7f40f4107790 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.740+0000 7f40fa1ff700 1 -- 172.21.15.62:0/2119614001 >> 172.21.15.62:0/2119614001 conn(0x7f40f4075190 msgr2=0x7f40f40775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.743+0000 7f40fa1ff700 1 -- 172.21.15.62:0/2119614001 shutdown_connections 2023-07-14T16:35:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.743+0000 7f40fa1ff700 1 -- 172.21.15.62:0/2119614001 wait complete. 2023-07-14T16:35:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.744+0000 7f40fa1ff700 1 Processor -- start 2023-07-14T16:35:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.745+0000 7f40fa1ff700 1 -- start start 2023-07-14T16:35:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.745+0000 7f40fa1ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f4106150 0x7f40f407dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.745+0000 7f40fa1ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40f4107340 0x7f40f407e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.745+0000 7f40fa1ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107cd0 0x7f40f407f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.745+0000 7f40fa1ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f407f720 con 0x7f40f4106150 2023-07-14T16:35:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.745+0000 7f40fa1ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f407f890 con 0x7f40f4107340 2023-07-14T16:35:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.745+0000 7f40fa1ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f407fa50 con 0x7f40f4107cd0 2023-07-14T16:35:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.746+0000 7f40f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f4106150 0x7f40f407dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.746+0000 7f40f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f4106150 0x7f40f407dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50192/0 (socket says 172.21.15.62:50192) 2023-07-14T16:35:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.746+0000 7f40f37fe700 1 -- 172.21.15.62:0/3653763749 learned_addr learned my addr 172.21.15.62:0/3653763749 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.746+0000 7f40f3fff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107cd0 0x7f40f407f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.746+0000 7f40f2ffd700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40f4107340 0x7f40f407e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.747+0000 7f40f3fff700 1 -- 172.21.15.62:0/3653763749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40f4107340 msgr2=0x7f40f407e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.747+0000 7f40f3fff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40f4107340 0x7f40f407e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.747+0000 7f40f3fff700 1 -- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f4106150 msgr2=0x7f40f407dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.747+0000 7f40f3fff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f4106150 0x7f40f407dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.747+0000 7f40f3fff700 1 -- 172.21.15.62:0/3653763749 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e4009520 con 0x7f40f4107cd0 2023-07-14T16:35:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.747+0000 7f40f2ffd700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40f4107340 0x7f40f407e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:35:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.747+0000 7f40f37fe700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f4106150 0x7f40f407dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:35:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.748+0000 7f40f3fff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107cd0 0x7f40f407f0c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00efc0 tx=0x7f40ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.748+0000 7f40f0ff9700 1 -- 172.21.15.62:0/3653763749 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec018070 con 0x7f40f4107cd0 2023-07-14T16:35:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.748+0000 7f40f0ff9700 1 -- 172.21.15.62:0/3653763749 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40ec00f040 con 0x7f40f4107cd0 2023-07-14T16:35:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.748+0000 7f40f0ff9700 1 -- 172.21.15.62:0/3653763749 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00cba0 con 0x7f40f4107cd0 2023-07-14T16:35:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.749+0000 7f40fa1ff700 1 -- 172.21.15.62:0/3653763749 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f407fcd0 con 0x7f40f4107cd0 2023-07-14T16:35:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.749+0000 7f40fa1ff700 1 -- 172.21.15.62:0/3653763749 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f40804e0 con 0x7f40f4107cd0 2023-07-14T16:35:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.757+0000 7f410dc1cb80 1 Processor -- start 2023-07-14T16:35:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.758+0000 7f40f0ff9700 1 -- 172.21.15.62:0/3653763749 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f40ec003a00 con 0x7f40f4107cd0 2023-07-14T16:35:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.758+0000 7f410dc1cb80 1 -- start start 2023-07-14T16:35:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.758+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e66c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.758+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 0x557f0e66d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.758+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 0x557f0e66db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.758+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e571c50 con 0x557f0e4c86b0 2023-07-14T16:35:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.758+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e5c49e0 con 0x557f0e66c840 2023-07-14T16:35:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.758+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e5bde70 con 0x557f0e66d6e0 2023-07-14T16:35:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.759+0000 7f40f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e66c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.759+0000 7f40f2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e66c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50200/0 (socket says 172.21.15.62:50200) 2023-07-14T16:35:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.759+0000 7f40f2ffd700 1 -- 172.21.15.62:0/1640589485 learned_addr learned my addr 172.21.15.62:0/1640589485 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.759+0000 7f40f37fe700 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 0x557f0e66d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.759+0000 7f40f2ffd700 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 0x557f0e66db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.760+0000 7f40f2ffd700 1 -- 172.21.15.62:0/1640589485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 msgr2=0x557f0e66db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.760+0000 7f40f2ffd700 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 0x557f0e66db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.760+0000 7f40f2ffd700 1 -- 172.21.15.62:0/1640589485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 msgr2=0x557f0e66d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.760+0000 7f40f2ffd700 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 0x557f0e66d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.760+0000 7f40f2ffd700 1 -- 172.21.15.62:0/1640589485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f0e3ebd80 con 0x557f0e4c86b0 2023-07-14T16:35:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.760+0000 7f40f0ff9700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40dc0a12a0 0x7f40dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.760+0000 7f40f0ff9700 1 -- 172.21.15.62:0/3653763749 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40ec0cbaf0 con 0x7f40f4107cd0 2023-07-14T16:35:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.761+0000 7f40f2ffd700 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e66c300 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009810 tx=0x7f40e4019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6df50fc6306677 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.761+0000 7f40d9ffb700 1 -- 172.21.15.62:0/1640589485 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e400f7b0 con 0x557f0e4c86b0 2023-07-14T16:35:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.762+0000 7f40d9ffb700 1 -- 172.21.15.62:0/1640589485 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40e400b9f0 con 0x557f0e4c86b0 2023-07-14T16:35:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.762+0000 7f40d9ffb700 1 -- 172.21.15.62:0/1640589485 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4003680 con 0x557f0e4c86b0 2023-07-14T16:35:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.763+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1640589485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 msgr2=0x557f0e66c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.763+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e66c300 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009810 tx=0x7f40e4019040 comp rx=0 tx=0).stop 2023-07-14T16:35:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.764+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1640589485 shutdown_connections 2023-07-14T16:35:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.764+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 0x557f0e66db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.764+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e66c300 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.764+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1640589485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 0x557f0e66d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.764+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1640589485 >> 172.21.15.62:0/1640589485 conn(0x557f0e661be0 msgr2=0x557f0e666cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.764+0000 7f40f37fe700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40dc0a12a0 0x7f40dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.765+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1640589485 shutdown_connections 2023-07-14T16:35:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.765+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1640589485 wait complete. 2023-07-14T16:35:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.766+0000 7f410dc1cb80 1 -- start start 2023-07-14T16:35:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.766+0000 7f410dc1cb80 1 Processor -- start 2023-07-14T16:35:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.767+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e6758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.767+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66c840 0x557f0e675e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.767+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66d6e0 0x557f0e67ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.767+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e55da60 con 0x557f0e4c86b0 2023-07-14T16:35:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.767+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e55d760 con 0x557f0e66d6e0 2023-07-14T16:35:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.767+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e4418c0 con 0x557f0e66c840 2023-07-14T16:35:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.767+0000 7f410dc1cb80 10 client.0 _ll_register_callbacks cb 0x557f0e55b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.768+0000 7f40f37fe700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40dc0a12a0 0x7f40dc0a3750 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009fd0 tx=0x7f40e8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.768+0000 7f40f2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66c840 0x557f0e675e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.768+0000 7f40f2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66c840 0x557f0e675e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46940/0 (socket says 172.21.15.62:46940) 2023-07-14T16:35:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.768+0000 7f40f2ffd700 1 -- 172.21.15.62:0/4267194435 learned_addr learned my addr 172.21.15.62:0/4267194435 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.769+0000 7f40f3fff700 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66d6e0 0x557f0e67ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.769+0000 7f40f37fe700 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e6758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.769+0000 7f40f3fff700 1 -- 172.21.15.62:0/4267194435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66c840 msgr2=0x557f0e675e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.769+0000 7f40f3fff700 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66c840 0x557f0e675e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.769+0000 7f40f3fff700 1 -- 172.21.15.62:0/4267194435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 msgr2=0x557f0e6758f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.769+0000 7f40f3fff700 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e6758f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.769+0000 7f40f3fff700 1 -- 172.21.15.62:0/4267194435 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e401f040 con 0x557f0e66d6e0 2023-07-14T16:35:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.770+0000 7f40f3fff700 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66d6e0 0x557f0e67ce70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0c95a0 tx=0x7f40ec098890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c35f29cc2c0a29 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.770+0000 7f40d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:35:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.770+0000 7f40d2ffd700 1 -- 172.21.15.62:0/4267194435 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec098eb0 con 0x557f0e66d6e0 2023-07-14T16:35:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.770+0000 7f40d2ffd700 1 -- 172.21.15.62:0/4267194435 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40ec09cee0 con 0x557f0e66d6e0 2023-07-14T16:35:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.770+0000 7f410dc1cb80 1 -- 172.21.15.62:0/4267194435 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f0e517490 con 0x557f0e66d6e0 2023-07-14T16:35:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.771+0000 7f40d2ffd700 1 -- 172.21.15.62:0/4267194435 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40ec0a1300 con 0x557f0e66d6e0 2023-07-14T16:35:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.771+0000 7f410dc1cb80 10 client.30359 fetch_fsmap learned FSMap version 38 2023-07-14T16:35:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.771+0000 7f410dc1cb80 1 -- 172.21.15.62:0/4267194435 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557f0e3d4c00 con 0x557f0e66d6e0 2023-07-14T16:35:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.773+0000 7f40d2ffd700 1 -- 172.21.15.62:0/4267194435 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40ec0c7080 con 0x557f0e66d6e0 2023-07-14T16:35:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.773+0000 7f40d2ffd700 1 -- 172.21.15.62:0/4267194435 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40ec0a0e70 con 0x557f0e66d6e0 2023-07-14T16:35:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.774+0000 7f410dc1cb80 10 client.30359 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T16:35:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.774+0000 7f410dc1cb80 10 client.30359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.774+0000 7f410dc1cb80 20 client.30359 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.774+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557f0e6816a0 0x557f0e683ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.774+0000 7f410dc1cb80 4 client.30359 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:35:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.774+0000 7f410dc1cb80 1 -- 172.21.15.62:0/4267194435 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f0e5b6e50 con 0x557f0e6816a0 2023-07-14T16:35:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.775+0000 7f40f37fe700 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557f0e6816a0 0x557f0e683ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.776+0000 7f40f37fe700 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557f0e6816a0 0x557f0e683ab0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1caf361bf394a7c server_cookie=a821f3e9f9a4fff in_seq=0 out_seq=0 2023-07-14T16:35:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.776+0000 7f40d2ffd700 10 client.30359 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.778+0000 7f40d2ffd700 1 -- 172.21.15.62:0/4267194435 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f0e5b6e50 con 0x557f0e6816a0 2023-07-14T16:35:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.778+0000 7f40d2ffd700 10 client.30359 handle_command_reply: tid=0 2023-07-14T16:35:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.778+0000 7f410dc1cb80 1 client.30359 shutdown 2023-07-14T16:35:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.779+0000 7f410dc1cb80 1 -- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66d6e0 msgr2=0x557f0e67ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.779+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66d6e0 0x557f0e67ce70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0c95a0 tx=0x7f40ec098890 comp rx=0 tx=0).stop 2023-07-14T16:35:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f410dc1cb80 1 -- 172.21.15.62:0/4267194435 shutdown_connections 2023-07-14T16:35:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557f0e6816a0 0x557f0e683ab0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66d6e0 0x557f0e67ce70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e4c86b0 0x557f0e6758f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/4267194435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66c840 0x557f0e675e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f410dc1cb80 1 -- 172.21.15.62:0/4267194435 >> 172.21.15.62:0/4267194435 conn(0x557f0e661be0 msgr2=0x557f0e6664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f40d2ffd700 0 client.30359 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f410dc1cb80 1 -- 172.21.15.62:0/4267194435 shutdown_connections 2023-07-14T16:35:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.781+0000 7f410dc1cb80 1 -- 172.21.15.62:0/4267194435 wait complete. 2023-07-14T16:35:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.782+0000 7f410dc1cb80 20 client.30359 trim_cache size 0 max 16384 2023-07-14T16:35:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.811+0000 7f410dc1cb80 1 Processor -- start 2023-07-14T16:35:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.811+0000 7f410dc1cb80 1 -- start start 2023-07-14T16:35:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.811+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e4c86b0 0x557f0e673b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.811+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66c840 0x557f0e67ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.811+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 0x557f0e670680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.811+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e6677a0 con 0x557f0e66c840 2023-07-14T16:35:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.811+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e674da0 con 0x557f0e4c86b0 2023-07-14T16:35:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.811+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e675460 con 0x557f0e66d6e0 2023-07-14T16:35:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.812+0000 7f40f37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e4c86b0 0x557f0e673b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.812+0000 7f40f37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e4c86b0 0x557f0e673b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:38916/0 (socket says 172.21.15.62:38916) 2023-07-14T16:35:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.812+0000 7f40f37fe700 1 -- 172.21.15.62:0/3945663234 learned_addr learned my addr 172.21.15.62:0/3945663234 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.812+0000 7f40f2ffd700 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66c840 0x557f0e67ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.812+0000 7f40f3fff700 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 0x557f0e670680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.813+0000 7f40f37fe700 1 -- 172.21.15.62:0/3945663234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 msgr2=0x557f0e670680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.813+0000 7f40f37fe700 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 0x557f0e670680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.813+0000 7f40f37fe700 1 -- 172.21.15.62:0/3945663234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66c840 msgr2=0x557f0e67ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:35:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.813+0000 7f40f37fe700 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66c840 0x557f0e67ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.813+0000 7f40f37fe700 1 -- 172.21.15.62:0/3945663234 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f0e4121f0 con 0x557f0e4c86b0 2023-07-14T16:35:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.813+0000 7f40f2ffd700 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66c840 0x557f0e67ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:35:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.813+0000 7f40f37fe700 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e4c86b0 0x557f0e673b30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f40e80041f0 tx=0x7f40e801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f532a3de4a820777 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.815+0000 7f40da7fc700 1 -- 172.21.15.62:0/3945663234 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8025040 con 0x557f0e4c86b0 2023-07-14T16:35:59.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.815+0000 7f40da7fc700 1 -- 172.21.15.62:0/3945663234 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40e802a3a0 con 0x557f0e4c86b0 2023-07-14T16:35:59.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.815+0000 7f40da7fc700 1 -- 172.21.15.62:0/3945663234 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8017070 con 0x557f0e4c86b0 2023-07-14T16:35:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.820+0000 7f410dc1cb80 1 -- 172.21.15.62:0/3945663234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e4c86b0 msgr2=0x557f0e673b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.820+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e4c86b0 0x557f0e673b30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f40e80041f0 tx=0x7f40e801a000 comp rx=0 tx=0).stop 2023-07-14T16:35:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.826+0000 7f410dc1cb80 1 -- 172.21.15.62:0/3945663234 shutdown_connections 2023-07-14T16:35:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.826+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e66d6e0 0x557f0e670680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.826+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e4c86b0 0x557f0e673b30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.826+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/3945663234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66c840 0x557f0e67ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.826+0000 7f410dc1cb80 1 -- 172.21.15.62:0/3945663234 >> 172.21.15.62:0/3945663234 conn(0x557f0e661be0 msgr2=0x557f0e674560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.834+0000 7f410dc1cb80 1 -- 172.21.15.62:0/3945663234 shutdown_connections 2023-07-14T16:35:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.834+0000 7f410dc1cb80 1 -- 172.21.15.62:0/3945663234 wait complete. 2023-07-14T16:35:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.835+0000 7f410dc1cb80 1 -- start start 2023-07-14T16:35:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.835+0000 7f410dc1cb80 1 Processor -- start 2023-07-14T16:35:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.836+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e4c86b0 0x557f0e678fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.836+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 0x557f0e679500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.836+0000 7f410dc1cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66d6e0 0x557f0e67c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.836+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e67c5b0 con 0x557f0e66d6e0 2023-07-14T16:35:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.836+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e67c720 con 0x557f0e66c840 2023-07-14T16:35:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.836+0000 7f410dc1cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f0e67c8e0 con 0x557f0e4c86b0 2023-07-14T16:35:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.836+0000 7f410dc1cb80 10 client.0 _ll_register_callbacks cb 0x7f40ec098c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:35:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.837+0000 7f40f3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66d6e0 0x557f0e67c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.837+0000 7f40f3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66d6e0 0x557f0e67c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50232/0 (socket says 172.21.15.62:50232) 2023-07-14T16:35:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.837+0000 7f40f3fff700 1 -- 172.21.15.62:0/1906983171 learned_addr learned my addr 172.21.15.62:0/1906983171 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:35:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.838+0000 7f40f3fff700 1 -- 172.21.15.62:0/1906983171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e4c86b0 msgr2=0x557f0e678fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:35:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.838+0000 7f40f3fff700 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e4c86b0 0x557f0e678fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.838+0000 7f40f3fff700 1 -- 172.21.15.62:0/1906983171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 msgr2=0x557f0e679500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:35:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.838+0000 7f40f3fff700 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 0x557f0e679500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.838+0000 7f40f3fff700 1 -- 172.21.15.62:0/1906983171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e80122c0 con 0x557f0e66d6e0 2023-07-14T16:35:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.839+0000 7f40f3fff700 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66d6e0 0x557f0e67c070 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0953b0 tx=0x7f40ec0acc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641a6e6c76271fca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:35:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.841+0000 7f40d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:35:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.841+0000 7f40d2ffd700 1 -- 172.21.15.62:0/1906983171 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec09cf60 con 0x557f0e66d6e0 2023-07-14T16:35:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.841+0000 7f40d2ffd700 1 -- 172.21.15.62:0/1906983171 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40ec0b7020 con 0x557f0e66d6e0 2023-07-14T16:35:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.841+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1906983171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40e400b9f0 con 0x557f0e66d6e0 2023-07-14T16:35:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.843+0000 7f40d2ffd700 1 -- 172.21.15.62:0/1906983171 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40ec098eb0 con 0x557f0e66d6e0 2023-07-14T16:35:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.843+0000 7f410dc1cb80 10 client.20796 fetch_fsmap learned FSMap version 38 2023-07-14T16:35:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.843+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1906983171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557f0e3ea250 con 0x557f0e66d6e0 2023-07-14T16:35:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.844+0000 7f40d2ffd700 1 -- 172.21.15.62:0/1906983171 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40e400b9f0 con 0x557f0e66d6e0 2023-07-14T16:35:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.844+0000 7f40d2ffd700 1 -- 172.21.15.62:0/1906983171 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f40ec0a0e70 con 0x557f0e66d6e0 2023-07-14T16:35:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.847+0000 7f410dc1cb80 10 client.20796 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T16:35:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.847+0000 7f410dc1cb80 10 client.20796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:35:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.847+0000 7f410dc1cb80 20 client.20796 populate_metadata read hostname 'smithi062' 2023-07-14T16:35:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.847+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557f0e6d6070 0x557f0e6d8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:35:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.847+0000 7f410dc1cb80 4 client.20796 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:35:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.847+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1906983171 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f0e578ff0 con 0x557f0e6d6070 2023-07-14T16:35:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.848+0000 7f40f37fe700 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557f0e6d6070 0x557f0e6d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:35:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.849+0000 7f40f37fe700 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557f0e6d6070 0x557f0e6d8480 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4bf0aaa81eea70 server_cookie=d3447c091d47decc in_seq=0 out_seq=0 2023-07-14T16:35:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.849+0000 7f40d2ffd700 10 client.20796 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.864+0000 7f40d2ffd700 1 -- 172.21.15.62:0/1906983171 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f0e578ff0 con 0x557f0e6d6070 2023-07-14T16:35:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.864+0000 7f40d2ffd700 10 client.20796 handle_command_reply: tid=0 2023-07-14T16:35:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.868+0000 7f410dc1cb80 1 client.20796 shutdown 2023-07-14T16:35:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.870+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1906983171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66d6e0 msgr2=0x557f0e67c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.871+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66d6e0 0x557f0e67c070 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0953b0 tx=0x7f40ec0acc00 comp rx=0 tx=0).stop 2023-07-14T16:35:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.871+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1906983171 shutdown_connections 2023-07-14T16:35:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.871+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f0e66d6e0 0x557f0e67c070 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.871+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f0e4c86b0 0x557f0e678fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.871+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x557f0e6d6070 0x557f0e6d8480 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.871+0000 7f410dc1cb80 1 --2- 172.21.15.62:0/1906983171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557f0e66c840 0x557f0e679500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.844 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.871+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1906983171 >> 172.21.15.62:0/1906983171 conn(0x557f0e661be0 msgr2=0x557f0e673b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.873+0000 7f40d2ffd700 0 client.20796 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:35:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.873+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1906983171 shutdown_connections 2023-07-14T16:35:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.874+0000 7f410dc1cb80 1 -- 172.21.15.62:0/1906983171 wait complete. 2023-07-14T16:35:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.874+0000 7f410dc1cb80 20 client.20796 trim_cache size 0 max 16384 2023-07-14T16:35:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:35:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:35:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:35:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:35:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.876+0000 7f40fa1ff700 1 -- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40dc0a12a0 msgr2=0x7f40dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.876+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40dc0a12a0 0x7f40dc0a3750 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009fd0 tx=0x7f40e8009310 comp rx=0 tx=0).stop 2023-07-14T16:35:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.876+0000 7f40fa1ff700 1 -- 172.21.15.62:0/3653763749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107cd0 msgr2=0x7f40f407f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:35:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.876+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107cd0 0x7f40f407f0c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00efc0 tx=0x7f40ec00c3a0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.878+0000 7f40fa1ff700 1 -- 172.21.15.62:0/3653763749 shutdown_connections 2023-07-14T16:35:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.878+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40f4107cd0 0x7f40f407f0c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.878+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f4106150 0x7f40f407dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.878+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f40dc0a12a0 0x7f40dc0a3750 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.878+0000 7f40fa1ff700 1 --2- 172.21.15.62:0/3653763749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40f4107340 0x7f40f407e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:35:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.878+0000 7f40fa1ff700 1 -- 172.21.15.62:0/3653763749 >> 172.21.15.62:0/3653763749 conn(0x7f40f4075190 msgr2=0x7f40f4077450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:35:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.879+0000 7f40fa1ff700 1 -- 172.21.15.62:0/3653763749 shutdown_connections 2023-07-14T16:35:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:35:59.879+0000 7f40fa1ff700 1 -- 172.21.15.62:0/3653763749 wait complete. 2023-07-14T16:36:01.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:01 smithi164 ceph-mon[97427]: pgmap v968: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:36:01.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:01 smithi062 ceph-mon[95543]: pgmap v968: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:36:01.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:01 smithi081 ceph-mon[127795]: pgmap v968: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 4.53k op/s 2023-07-14T16:36:03.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:03 smithi164 ceph-mon[97427]: pgmap v969: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2023-07-14T16:36:03.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:03 smithi062 ceph-mon[95543]: pgmap v969: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2023-07-14T16:36:03.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:03 smithi081 ceph-mon[127795]: pgmap v969: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.10k op/s 2023-07-14T16:36:05.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:05 smithi164 ceph-mon[97427]: pgmap v970: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2023-07-14T16:36:05.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:05 smithi062 ceph-mon[95543]: pgmap v970: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2023-07-14T16:36:05.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:05 smithi081 ceph-mon[127795]: pgmap v970: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 4.71k op/s 2023-07-14T16:36:07.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:07 smithi164 ceph-mon[97427]: pgmap v971: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2023-07-14T16:36:07.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:07 smithi062 ceph-mon[95543]: pgmap v971: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2023-07-14T16:36:07.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:07 smithi081 ceph-mon[127795]: pgmap v971: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.92k op/s 2023-07-14T16:36:09.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:09 smithi164 ceph-mon[97427]: pgmap v972: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 4.44k op/s 2023-07-14T16:36:09.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:09 smithi062 ceph-mon[95543]: pgmap v972: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 4.44k op/s 2023-07-14T16:36:09.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:09 smithi081 ceph-mon[127795]: pgmap v972: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 4.44k op/s 2023-07-14T16:36:11.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:11 smithi062 ceph-mon[95543]: pgmap v973: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2023-07-14T16:36:11.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:11 smithi081 ceph-mon[127795]: pgmap v973: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2023-07-14T16:36:11.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:11 smithi164 ceph-mon[97427]: pgmap v973: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2023-07-14T16:36:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.164+0000 7f5b43bff700 1 -- 172.21.15.62:0/3995862287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 msgr2=0x7f5b3c071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.164+0000 7f5b43bff700 1 --2- 172.21.15.62:0/3995862287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 0x7f5b3c071100 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009990 tx=0x7f5b30009ca0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.165+0000 7f5b43bff700 1 -- 172.21.15.62:0/3995862287 shutdown_connections 2023-07-14T16:36:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.165+0000 7f5b43bff700 1 --2- 172.21.15.62:0/3995862287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b3c071640 0x7f5b3c110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.165+0000 7f5b43bff700 1 --2- 172.21.15.62:0/3995862287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b3c107a20 0x7f5b3c107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.165+0000 7f5b43bff700 1 --2- 172.21.15.62:0/3995862287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 0x7f5b3c071100 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.165+0000 7f5b43bff700 1 -- 172.21.15.62:0/3995862287 >> 172.21.15.62:0/3995862287 conn(0x7f5b3c06ca50 msgr2=0x7f5b3c06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.166+0000 7f5b43bff700 1 -- 172.21.15.62:0/3995862287 shutdown_connections 2023-07-14T16:36:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.166+0000 7f5b43bff700 1 -- 172.21.15.62:0/3995862287 wait complete. 2023-07-14T16:36:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.168+0000 7f5b43bff700 1 Processor -- start 2023-07-14T16:36:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.168+0000 7f5b43bff700 1 -- start start 2023-07-14T16:36:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.168+0000 7f5b43bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 0x7f5b3c1a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.168+0000 7f5b43bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b3c071640 0x7f5b3c1a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.168+0000 7f5b43bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b3c107a20 0x7f5b3c1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.168+0000 7f5b43bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1a6090 con 0x7f5b3c071640 2023-07-14T16:36:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.168+0000 7f5b43bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1a6200 con 0x7f5b3c070c90 2023-07-14T16:36:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.168+0000 7f5b43bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1a63c0 con 0x7f5b3c107a20 2023-07-14T16:36:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.169+0000 7f5b4119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b3c071640 0x7f5b3c1a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.169+0000 7f5b4119a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b3c071640 0x7f5b3c1a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35076/0 (socket says 172.21.15.62:35076) 2023-07-14T16:36:13.144 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.169+0000 7f5b4119a700 1 -- 172.21.15.62:0/4284427026 learned_addr learned my addr 172.21.15.62:0/4284427026 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.170+0000 7f5b4199b700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 0x7f5b3c1a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.170+0000 7f5b4219c700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b3c107a20 0x7f5b3c1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.170+0000 7f5b4199b700 1 -- 172.21.15.62:0/4284427026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b3c107a20 msgr2=0x7f5b3c1a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.170+0000 7f5b4199b700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b3c107a20 0x7f5b3c1a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.170+0000 7f5b4199b700 1 -- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b3c071640 msgr2=0x7f5b3c1a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.170+0000 7f5b4199b700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b3c071640 0x7f5b3c1a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.170+0000 7f5b4199b700 1 -- 172.21.15.62:0/4284427026 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b300095f0 con 0x7f5b3c070c90 2023-07-14T16:36:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.171+0000 7f5b4199b700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 0x7f5b3c1a4670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400ef10 tx=0x7f5b3400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.172+0000 7f5b3affd700 1 -- 172.21.15.62:0/4284427026 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400ca70 con 0x7f5b3c070c90 2023-07-14T16:36:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.172+0000 7f5b3affd700 1 -- 172.21.15.62:0/4284427026 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b3400f040 con 0x7f5b3c070c90 2023-07-14T16:36:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.172+0000 7f5b3affd700 1 -- 172.21.15.62:0/4284427026 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34014480 con 0x7f5b3c070c90 2023-07-14T16:36:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.172+0000 7f5b43bff700 1 -- 172.21.15.62:0/4284427026 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c1a6640 con 0x7f5b3c070c90 2023-07-14T16:36:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.173+0000 7f5b43bff700 1 -- 172.21.15.62:0/4284427026 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c1ace60 con 0x7f5b3c070c90 2023-07-14T16:36:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.184+0000 7f5b3affd700 1 -- 172.21.15.62:0/4284427026 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b34007500 con 0x7f5b3c070c90 2023-07-14T16:36:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.184+0000 7f5b574bab80 1 Processor -- start 2023-07-14T16:36:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.185+0000 7f5b574bab80 1 -- start start 2023-07-14T16:36:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.185+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea575300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.185+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea5761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.185+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 0x560eea576b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.185+0000 7f5b574bab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea47ac50 con 0x560eea5766e0 2023-07-14T16:36:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.185+0000 7f5b574bab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea4cd9e0 con 0x560eea3d16b0 2023-07-14T16:36:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.185+0000 7f5b574bab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea4c6e70 con 0x560eea575840 2023-07-14T16:36:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.186+0000 7f5b4219c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea575300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.186+0000 7f5b4219c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea575300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:39172/0 (socket says 172.21.15.62:39172) 2023-07-14T16:36:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.186+0000 7f5b4219c700 1 -- 172.21.15.62:0/432868888 learned_addr learned my addr 172.21.15.62:0/432868888 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.187+0000 7f5b3affd700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b240a12d0 0x7f5b240a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.187+0000 7f5b4199b700 1 --2- 172.21.15.62:0/432868888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea5761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.187+0000 7f5b4219c700 1 -- 172.21.15.62:0/432868888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 msgr2=0x560eea5761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.187+0000 7f5b4219c700 1 --2- 172.21.15.62:0/432868888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea5761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.187+0000 7f5b4219c700 1 -- 172.21.15.62:0/432868888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 msgr2=0x560eea576b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.187+0000 7f5b4219c700 1 --2- 172.21.15.62:0/432868888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 0x560eea576b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.187+0000 7f5b4219c700 1 -- 172.21.15.62:0/432868888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560eea2f4d80 con 0x560eea3d16b0 2023-07-14T16:36:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.188+0000 7f5b3affd700 1 -- 172.21.15.62:0/4284427026 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b340ccbd0 con 0x7f5b3c070c90 2023-07-14T16:36:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.189+0000 7f5b4219c700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b240a12d0 0x7f5b240a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.190+0000 7f5b4219c700 1 --2- 172.21.15.62:0/432868888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea575300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009fa0 tx=0x7f5b2c010650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475bf7fab07c2db4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.190+0000 7f5b4219c700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b240a12d0 0x7f5b240a3780 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c018000 tx=0x7f5b2c01a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.191+0000 7f5b23fff700 1 -- 172.21.15.62:0/432868888 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c01c440 con 0x560eea3d16b0 2023-07-14T16:36:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.191+0000 7f5b23fff700 1 -- 172.21.15.62:0/432868888 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b2c020030 con 0x560eea3d16b0 2023-07-14T16:36:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.191+0000 7f5b23fff700 1 -- 172.21.15.62:0/432868888 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c011660 con 0x560eea3d16b0 2023-07-14T16:36:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.193+0000 7f5b574bab80 1 -- 172.21.15.62:0/432868888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 msgr2=0x560eea575300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.193+0000 7f5b574bab80 1 --2- 172.21.15.62:0/432868888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea575300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009fa0 tx=0x7f5b2c010650 comp rx=0 tx=0).stop 2023-07-14T16:36:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.193+0000 7f5b574bab80 1 -- 172.21.15.62:0/432868888 shutdown_connections 2023-07-14T16:36:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.193+0000 7f5b574bab80 1 --2- 172.21.15.62:0/432868888 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 0x560eea576b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.193+0000 7f5b574bab80 1 --2- 172.21.15.62:0/432868888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea575300 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.193+0000 7f5b574bab80 1 --2- 172.21.15.62:0/432868888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea5761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.193+0000 7f5b574bab80 1 -- 172.21.15.62:0/432868888 >> 172.21.15.62:0/432868888 conn(0x560eea56abe0 msgr2=0x560eea56fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.194+0000 7f5b574bab80 1 -- 172.21.15.62:0/432868888 shutdown_connections 2023-07-14T16:36:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.194+0000 7f5b574bab80 1 -- 172.21.15.62:0/432868888 wait complete. 2023-07-14T16:36:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.195+0000 7f5b574bab80 1 -- start start 2023-07-14T16:36:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.195+0000 7f5b574bab80 1 Processor -- start 2023-07-14T16:36:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea3d16b0 0x560eea57e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea575840 0x560eea57ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 0x560eea585ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b574bab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea466a60 con 0x560eea5766e0 2023-07-14T16:36:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b574bab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea466760 con 0x560eea575840 2023-07-14T16:36:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b574bab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea34a8c0 con 0x560eea3d16b0 2023-07-14T16:36:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b574bab80 10 client.0 _ll_register_callbacks cb 0x560eea464700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:36:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b4199b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea3d16b0 0x560eea57e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b4199b700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea3d16b0 0x560eea57e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:57252/0 (socket says 172.21.15.62:57252) 2023-07-14T16:36:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b4199b700 1 -- 172.21.15.62:0/539804202 learned_addr learned my addr 172.21.15.62:0/539804202 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.196+0000 7f5b4119a700 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea575840 0x560eea57ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b4219c700 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 0x560eea585ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b4119a700 1 -- 172.21.15.62:0/539804202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea3d16b0 msgr2=0x560eea57e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b4119a700 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea3d16b0 0x560eea57e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b4119a700 1 -- 172.21.15.62:0/539804202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 msgr2=0x560eea585ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b4119a700 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 0x560eea585ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b4119a700 1 -- 172.21.15.62:0/539804202 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c009ad0 con 0x560eea575840 2023-07-14T16:36:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b4219c700 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 0x560eea585ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:36:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b4119a700 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea575840 0x560eea57ee80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006010 tx=0x7f5b30003a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d1710793633fa9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.197+0000 7f5b217fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:36:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.198+0000 7f5b217fa700 1 -- 172.21.15.62:0/539804202 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3001d070 con 0x560eea575840 2023-07-14T16:36:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.198+0000 7f5b217fa700 1 -- 172.21.15.62:0/539804202 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b30005550 con 0x560eea575840 2023-07-14T16:36:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.198+0000 7f5b574bab80 1 -- 172.21.15.62:0/539804202 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560eea420490 con 0x560eea575840 2023-07-14T16:36:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.198+0000 7f5b217fa700 1 -- 172.21.15.62:0/539804202 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b30005710 con 0x560eea575840 2023-07-14T16:36:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.198+0000 7f5b574bab80 10 client.30374 fetch_fsmap learned FSMap version 38 2023-07-14T16:36:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.198+0000 7f5b574bab80 1 -- 172.21.15.62:0/539804202 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560eea2ddc00 con 0x560eea575840 2023-07-14T16:36:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.200+0000 7f5b217fa700 1 -- 172.21.15.62:0/539804202 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560eea420490 con 0x560eea575840 2023-07-14T16:36:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.200+0000 7f5b217fa700 1 -- 172.21.15.62:0/539804202 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b300046e0 con 0x560eea575840 2023-07-14T16:36:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.200+0000 7f5b574bab80 10 client.30374 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T16:36:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.200+0000 7f5b574bab80 10 client.30374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:36:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.200+0000 7f5b574bab80 20 client.30374 populate_metadata read hostname 'smithi062' 2023-07-14T16:36:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.200+0000 7f5b574bab80 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560eea58a840 0x560eea58cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.200+0000 7f5b574bab80 4 client.30374 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:36:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.200+0000 7f5b574bab80 1 -- 172.21.15.62:0/539804202 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560eea4bfe50 con 0x560eea58a840 2023-07-14T16:36:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.201+0000 7f5b4199b700 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560eea58a840 0x560eea58cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.202+0000 7f5b4199b700 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560eea58a840 0x560eea58cc50 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33affad1fe38af05 server_cookie=660b161a2b85626e in_seq=0 out_seq=0 2023-07-14T16:36:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.203+0000 7f5b217fa700 10 client.30374 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.203+0000 7f5b217fa700 1 -- 172.21.15.62:0/539804202 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560eea4bfe50 con 0x560eea58a840 2023-07-14T16:36:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.203+0000 7f5b217fa700 10 client.30374 handle_command_reply: tid=0 2023-07-14T16:36:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 client.30374 shutdown 2023-07-14T16:36:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 -- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea575840 msgr2=0x560eea57ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea575840 0x560eea57ee80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006010 tx=0x7f5b30003a80 comp rx=0 tx=0).stop 2023-07-14T16:36:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 -- 172.21.15.62:0/539804202 shutdown_connections 2023-07-14T16:36:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea5766e0 0x560eea585ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea3d16b0 0x560eea57e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560eea58a840 0x560eea58cc50 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 --2- 172.21.15.62:0/539804202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea575840 0x560eea57ee80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 -- 172.21.15.62:0/539804202 >> 172.21.15.62:0/539804202 conn(0x560eea56abe0 msgr2=0x560eea56f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b217fa700 0 client.30374 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.204+0000 7f5b574bab80 1 -- 172.21.15.62:0/539804202 shutdown_connections 2023-07-14T16:36:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.205+0000 7f5b574bab80 1 -- 172.21.15.62:0/539804202 wait complete. 2023-07-14T16:36:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.205+0000 7f5b574bab80 20 client.30374 trim_cache size 0 max 16384 2023-07-14T16:36:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.235+0000 7f5b574bab80 1 Processor -- start 2023-07-14T16:36:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.235+0000 7f5b574bab80 1 -- start start 2023-07-14T16:36:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.235+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea3d16b0 0x560eea57cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.235+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea585e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.235+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea5766e0 0x560eea57e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.235+0000 7f5b574bab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea53b060 con 0x560eea3d16b0 2023-07-14T16:36:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.235+0000 7f5b574bab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea57ddf0 con 0x560eea5766e0 2023-07-14T16:36:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.236+0000 7f5b574bab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea57e4b0 con 0x560eea575840 2023-07-14T16:36:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.236+0000 7f5b4119a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea585e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.236+0000 7f5b4119a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea585e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:57268/0 (socket says 172.21.15.62:57268) 2023-07-14T16:36:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.236+0000 7f5b4119a700 1 -- 172.21.15.62:0/1277332238 learned_addr learned my addr 172.21.15.62:0/1277332238 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.236+0000 7f5b4219c700 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea5766e0 0x560eea57e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.237+0000 7f5b4199b700 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea3d16b0 0x560eea57cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.237+0000 7f5b4199b700 1 -- 172.21.15.62:0/1277332238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 msgr2=0x560eea585e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.237+0000 7f5b4199b700 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea585e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.237+0000 7f5b4199b700 1 -- 172.21.15.62:0/1277332238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea5766e0 msgr2=0x560eea57e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.237+0000 7f5b4199b700 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea5766e0 0x560eea57e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.237+0000 7f5b4199b700 1 -- 172.21.15.62:0/1277332238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560eea31b1f0 con 0x560eea3d16b0 2023-07-14T16:36:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.237+0000 7f5b4219c700 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea5766e0 0x560eea57e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:36:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.238+0000 7f5b4199b700 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea3d16b0 0x560eea57cb20 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5b34099c90 tx=0x7f5b340ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4201bcf1f99cdd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.238+0000 7f5b38bff700 1 -- 172.21.15.62:0/1277332238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34099d60 con 0x560eea3d16b0 2023-07-14T16:36:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.238+0000 7f5b38bff700 1 -- 172.21.15.62:0/1277332238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b34099f90 con 0x560eea3d16b0 2023-07-14T16:36:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.238+0000 7f5b38bff700 1 -- 172.21.15.62:0/1277332238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400cd60 con 0x560eea3d16b0 2023-07-14T16:36:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.239+0000 7f5b574bab80 1 -- 172.21.15.62:0/1277332238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea3d16b0 msgr2=0x560eea57cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.239+0000 7f5b574bab80 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea3d16b0 0x560eea57cb20 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5b34099c90 tx=0x7f5b340ce040 comp rx=0 tx=0).stop 2023-07-14T16:36:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.240+0000 7f5b574bab80 1 -- 172.21.15.62:0/1277332238 shutdown_connections 2023-07-14T16:36:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.240+0000 7f5b574bab80 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea5766e0 0x560eea57e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.240+0000 7f5b574bab80 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea3d16b0 0x560eea57cb20 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.240+0000 7f5b574bab80 1 --2- 172.21.15.62:0/1277332238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea575840 0x560eea585e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.240+0000 7f5b574bab80 1 -- 172.21.15.62:0/1277332238 >> 172.21.15.62:0/1277332238 conn(0x560eea56abe0 msgr2=0x560eea57d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.240+0000 7f5b574bab80 1 -- 172.21.15.62:0/1277332238 shutdown_connections 2023-07-14T16:36:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.240+0000 7f5b574bab80 1 -- 172.21.15.62:0/1277332238 wait complete. 2023-07-14T16:36:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.241+0000 7f5b574bab80 1 -- start start 2023-07-14T16:36:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.243+0000 7f5b574bab80 1 Processor -- start 2023-07-14T16:36:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.243+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea582080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.243+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea575840 0x560eea5825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.243+0000 7f5b574bab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea5766e0 0x560eea584ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.243+0000 7f5b574bab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea585420 con 0x560eea575840 2023-07-14T16:36:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.243+0000 7f5b574bab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea585590 con 0x560eea3d16b0 2023-07-14T16:36:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.243+0000 7f5b574bab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eea585700 con 0x560eea5766e0 2023-07-14T16:36:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.243+0000 7f5b574bab80 10 client.0 _ll_register_callbacks cb 0x560eea464700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:36:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.244+0000 7f5b4219c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea5766e0 0x560eea584ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.244+0000 7f5b4219c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea5766e0 0x560eea584ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:57270/0 (socket says 172.21.15.62:57270) 2023-07-14T16:36:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.244+0000 7f5b4219c700 1 -- 172.21.15.62:0/3673186893 learned_addr learned my addr 172.21.15.62:0/3673186893 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.244+0000 7f5b4199b700 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea582080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.245+0000 7f5b4199b700 1 -- 172.21.15.62:0/3673186893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea5766e0 msgr2=0x560eea584ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.245+0000 7f5b4199b700 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea5766e0 0x560eea584ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.245+0000 7f5b4199b700 1 -- 172.21.15.62:0/3673186893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea575840 msgr2=0x560eea5825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.245+0000 7f5b4199b700 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea575840 0x560eea5825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.245+0000 7f5b4199b700 1 -- 172.21.15.62:0/3673186893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560eea2f4d80 con 0x560eea3d16b0 2023-07-14T16:36:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.245+0000 7f5b4199b700 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea582080 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5b340cafd0 tx=0x7f5b34099d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbb646248897803 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.245+0000 7f5b217fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:36:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.246+0000 7f5b217fa700 1 -- 172.21.15.62:0/3673186893 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3409dfc0 con 0x560eea3d16b0 2023-07-14T16:36:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.246+0000 7f5b217fa700 1 -- 172.21.15.62:0/3673186893 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b340d4040 con 0x560eea3d16b0 2023-07-14T16:36:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.246+0000 7f5b574bab80 1 -- 172.21.15.62:0/3673186893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b2c020030 con 0x560eea3d16b0 2023-07-14T16:36:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.246+0000 7f5b217fa700 1 -- 172.21.15.62:0/3673186893 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b340a23e0 con 0x560eea3d16b0 2023-07-14T16:36:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.246+0000 7f5b574bab80 10 client.30380 fetch_fsmap learned FSMap version 38 2023-07-14T16:36:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.246+0000 7f5b574bab80 1 -- 172.21.15.62:0/3673186893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f5b3c1ace60 con 0x560eea3d16b0 2023-07-14T16:36:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.247+0000 7f5b217fa700 1 -- 172.21.15.62:0/3673186893 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b2c020030 con 0x560eea3d16b0 2023-07-14T16:36:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.248+0000 7f5b217fa700 1 -- 172.21.15.62:0/3673186893 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b34099510 con 0x560eea3d16b0 2023-07-14T16:36:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.248+0000 7f5b574bab80 10 client.30380 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T16:36:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.248+0000 7f5b574bab80 10 client.30380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:36:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.248+0000 7f5b574bab80 20 client.30380 populate_metadata read hostname 'smithi062' 2023-07-14T16:36:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.248+0000 7f5b574bab80 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560eea5df030 0x560eea5e1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.248+0000 7f5b574bab80 4 client.30380 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:36:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.248+0000 7f5b574bab80 1 -- 172.21.15.62:0/3673186893 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560eea481ff0 con 0x560eea5df030 2023-07-14T16:36:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.251+0000 7f5b4119a700 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560eea5df030 0x560eea5e1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.257+0000 7f5b217fa700 10 client.30380 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.257+0000 7f5b4119a700 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560eea5df030 0x560eea5e1440 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0abce563c46a48 server_cookie=bdbb225e1ab11aa0 in_seq=0 out_seq=0 2023-07-14T16:36:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.275+0000 7f5b217fa700 1 -- 172.21.15.62:0/3673186893 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560eea481ff0 con 0x560eea5df030 2023-07-14T16:36:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.275+0000 7f5b217fa700 10 client.30380 handle_command_reply: tid=0 2023-07-14T16:36:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.275+0000 7f5b574bab80 1 client.30380 shutdown 2023-07-14T16:36:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.277+0000 7f5b574bab80 1 -- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 msgr2=0x560eea582080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.277+0000 7f5b574bab80 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea582080 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5b340cafd0 tx=0x7f5b34099d60 comp rx=0 tx=0).stop 2023-07-14T16:36:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b574bab80 1 -- 172.21.15.62:0/3673186893 shutdown_connections 2023-07-14T16:36:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b574bab80 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560eea5766e0 0x560eea584ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b574bab80 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x560eea5df030 0x560eea5e1440 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b574bab80 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560eea3d16b0 0x560eea582080 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b574bab80 1 --2- 172.21.15.62:0/3673186893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560eea575840 0x560eea5825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b574bab80 1 -- 172.21.15.62:0/3673186893 >> 172.21.15.62:0/3673186893 conn(0x560eea56abe0 msgr2=0x560eea57cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b217fa700 0 client.30380 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b574bab80 1 -- 172.21.15.62:0/3673186893 shutdown_connections 2023-07-14T16:36:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.278+0000 7f5b574bab80 1 -- 172.21.15.62:0/3673186893 wait complete. 2023-07-14T16:36:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.279+0000 7f5b574bab80 20 client.30380 trim_cache size 0 max 16384 2023-07-14T16:36:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:36:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:36:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:36:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:36:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 -- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b240a12d0 msgr2=0x7f5b240a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b240a12d0 0x7f5b240a3780 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c018000 tx=0x7f5b2c01a000 comp rx=0 tx=0).stop 2023-07-14T16:36:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 -- 172.21.15.62:0/4284427026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 msgr2=0x7f5b3c1a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 0x7f5b3c1a4670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400ef10 tx=0x7f5b3400c3a0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 -- 172.21.15.62:0/4284427026 shutdown_connections 2023-07-14T16:36:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b3c107a20 0x7f5b3c1a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b240a12d0 0x7f5b240a3780 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b3c070c90 0x7f5b3c1a4670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 --2- 172.21.15.62:0/4284427026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b3c071640 0x7f5b3c1a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.280+0000 7f5b43bff700 1 -- 172.21.15.62:0/4284427026 >> 172.21.15.62:0/4284427026 conn(0x7f5b3c06ca50 msgr2=0x7f5b3c10a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.281+0000 7f5b43bff700 1 -- 172.21.15.62:0/4284427026 shutdown_connections 2023-07-14T16:36:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:13.281+0000 7f5b43bff700 1 -- 172.21.15.62:0/4284427026 wait complete. 2023-07-14T16:36:13.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:13 smithi062 ceph-mon[95543]: pgmap v974: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2023-07-14T16:36:13.401 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:13 smithi081 ceph-mon[127795]: pgmap v974: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2023-07-14T16:36:13.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:13 smithi164 ceph-mon[97427]: pgmap v974: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2023-07-14T16:36:15.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:15 smithi062 ceph-mon[95543]: pgmap v975: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:36:15.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:15 smithi081 ceph-mon[127795]: pgmap v975: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:36:15.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:15 smithi164 ceph-mon[97427]: pgmap v975: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2023-07-14T16:36:16.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:16 smithi062 ceph-mon[95543]: pgmap v976: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2023-07-14T16:36:16.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:16 smithi081 ceph-mon[127795]: pgmap v976: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2023-07-14T16:36:16.517 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:16 smithi164 ceph-mon[97427]: pgmap v976: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2023-07-14T16:36:19.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:18 smithi081 ceph-mon[127795]: pgmap v977: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-07-14T16:36:19.267 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:18 smithi164 ceph-mon[97427]: pgmap v977: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-07-14T16:36:19.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:18 smithi062 ceph-mon[95543]: pgmap v977: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 4.31k op/s 2023-07-14T16:36:21.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:20 smithi164 ceph-mon[97427]: pgmap v978: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-07-14T16:36:21.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:20 smithi062 ceph-mon[95543]: pgmap v978: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-07-14T16:36:21.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:20 smithi081 ceph-mon[127795]: pgmap v978: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2023-07-14T16:36:23.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:22 smithi164 ceph-mon[97427]: pgmap v979: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:36:23.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:22 smithi062 ceph-mon[95543]: pgmap v979: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:36:23.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:22 smithi081 ceph-mon[127795]: pgmap v979: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:36:25.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:24 smithi164 ceph-mon[97427]: pgmap v980: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:36:25.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:24 smithi062 ceph-mon[95543]: pgmap v980: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:36:25.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:24 smithi081 ceph-mon[127795]: pgmap v980: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.66k op/s 2023-07-14T16:36:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.592+0000 7fe53b3ff700 1 -- 172.21.15.62:0/3023777667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534100ca0 msgr2=0x7fe5341010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.592+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/3023777667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534100ca0 0x7fe5341010f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe524009990 tx=0x7fe524009ca0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.592+0000 7fe53b3ff700 1 -- 172.21.15.62:0/3023777667 shutdown_connections 2023-07-14T16:36:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.592+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/3023777667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe534101630 0x7fe534107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.592+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/3023777667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5340ffab0 0x7fe5340ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.592+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/3023777667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534100ca0 0x7fe5341010f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.592+0000 7fe53b3ff700 1 -- 172.21.15.62:0/3023777667 >> 172.21.15.62:0/3023777667 conn(0x7fe5340fb0b0 msgr2=0x7fe5340fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.592+0000 7fe53b3ff700 1 -- 172.21.15.62:0/3023777667 shutdown_connections 2023-07-14T16:36:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.593+0000 7fe53b3ff700 1 -- 172.21.15.62:0/3023777667 wait complete. 2023-07-14T16:36:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.594+0000 7fe53b3ff700 1 Processor -- start 2023-07-14T16:36:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.595+0000 7fe53b3ff700 1 -- start start 2023-07-14T16:36:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.595+0000 7fe53b3ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5340ffab0 0x7fe5341a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.595+0000 7fe53b3ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe534100ca0 0x7fe5341a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.595+0000 7fe53b3ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534101630 0x7fe5341a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.595+0000 7fe53b3ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5341a2540 con 0x7fe534100ca0 2023-07-14T16:36:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.595+0000 7fe53b3ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5341a26b0 con 0x7fe5340ffab0 2023-07-14T16:36:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.595+0000 7fe53b3ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5341a2870 con 0x7fe534101630 2023-07-14T16:36:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53999c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534101630 0x7fe5341a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53999c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534101630 0x7fe5341a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42066/0 (socket says 172.21.15.62:42066) 2023-07-14T16:36:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53999c700 1 -- 172.21.15.62:0/2146626247 learned_addr learned my addr 172.21.15.62:0/2146626247 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53919b700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5340ffab0 0x7fe5341a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53899a700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe534100ca0 0x7fe5341a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53899a700 1 -- 172.21.15.62:0/2146626247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534101630 msgr2=0x7fe5341a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53899a700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534101630 0x7fe5341a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53899a700 1 -- 172.21.15.62:0/2146626247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5340ffab0 msgr2=0x7fe5341a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53899a700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5340ffab0 0x7fe5341a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.596+0000 7fe53899a700 1 -- 172.21.15.62:0/2146626247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c009c40 con 0x7fe534100ca0 2023-07-14T16:36:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.597+0000 7fe53899a700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe534100ca0 0x7fe5341a5750 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe5240098e0 tx=0x7fe524005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.598+0000 7fe52a7fc700 1 -- 172.21.15.62:0/2146626247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52401d070 con 0x7fe534100ca0 2023-07-14T16:36:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.598+0000 7fe52a7fc700 1 -- 172.21.15.62:0/2146626247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe52400f460 con 0x7fe534100ca0 2023-07-14T16:36:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.598+0000 7fe52a7fc700 1 -- 172.21.15.62:0/2146626247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524021670 con 0x7fe534100ca0 2023-07-14T16:36:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.598+0000 7fe53b3ff700 1 -- 172.21.15.62:0/2146626247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5240095f0 con 0x7fe534100ca0 2023-07-14T16:36:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.598+0000 7fe53b3ff700 1 -- 172.21.15.62:0/2146626247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe534072d50 con 0x7fe534100ca0 2023-07-14T16:36:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.608+0000 7fe54eca5b80 1 Processor -- start 2023-07-14T16:36:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.608+0000 7fe52a7fc700 1 -- 172.21.15.62:0/2146626247 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe52400f860 con 0x7fe534100ca0 2023-07-14T16:36:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.608+0000 7fe54eca5b80 1 -- start start 2023-07-14T16:36:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.609+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab83300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.609+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 0x55933ab841a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.609+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab84b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.609+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933aa88c50 con 0x55933ab846e0 2023-07-14T16:36:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.609+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933aadb9e0 con 0x55933ab83840 2023-07-14T16:36:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.609+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933aad4e70 con 0x55933a9df6b0 2023-07-14T16:36:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.610+0000 7fe53919b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 0x55933ab841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.610+0000 7fe53999c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab83300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.610+0000 7fe53999c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab83300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42072/0 (socket says 172.21.15.62:42072) 2023-07-14T16:36:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.610+0000 7fe53999c700 1 -- 172.21.15.62:0/1206412406 learned_addr learned my addr 172.21.15.62:0/1206412406 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.610+0000 7fe53919b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 0x55933ab841a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:45356/0 (socket says 172.21.15.62:45356) 2023-07-14T16:36:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.611+0000 7fe53999c700 1 -- 172.21.15.62:0/1206412406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 msgr2=0x55933ab841a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.611+0000 7fe53999c700 1 --2- 172.21.15.62:0/1206412406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 0x55933ab841a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.611+0000 7fe53999c700 1 -- 172.21.15.62:0/1206412406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 msgr2=0x55933ab84b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.611+0000 7fe53999c700 1 --2- 172.21.15.62:0/1206412406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab84b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.611+0000 7fe53999c700 1 -- 172.21.15.62:0/1206412406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55933a902d80 con 0x55933a9df6b0 2023-07-14T16:36:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.612+0000 7fe53999c700 1 --2- 172.21.15.62:0/1206412406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab83300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe52c007610 tx=0x7fe52c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=406e37ae865baca1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.612+0000 7fe51f7fe700 1 -- 172.21.15.62:0/1206412406 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00e030 con 0x55933a9df6b0 2023-07-14T16:36:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.612+0000 7fe51f7fe700 1 -- 172.21.15.62:0/1206412406 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe52c00f040 con 0x55933a9df6b0 2023-07-14T16:36:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.612+0000 7fe51f7fe700 1 -- 172.21.15.62:0/1206412406 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c014b90 con 0x55933a9df6b0 2023-07-14T16:36:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.612+0000 7fe52a7fc700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5200a1350 0x7fe5200a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.613+0000 7fe52a7fc700 1 -- 172.21.15.62:0/2146626247 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe5240cdb10 con 0x7fe534100ca0 2023-07-14T16:36:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.614+0000 7fe54eca5b80 1 -- 172.21.15.62:0/1206412406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 msgr2=0x55933ab83300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.614+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/1206412406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab83300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe52c007610 tx=0x7fe52c009920 comp rx=0 tx=0).stop 2023-07-14T16:36:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.614+0000 7fe53919b700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5200a1350 0x7fe5200a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.614+0000 7fe54eca5b80 1 -- 172.21.15.62:0/1206412406 shutdown_connections 2023-07-14T16:36:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.614+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/1206412406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab84b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.614+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/1206412406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab83300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.614+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/1206412406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 0x55933ab841a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.614+0000 7fe54eca5b80 1 -- 172.21.15.62:0/1206412406 >> 172.21.15.62:0/1206412406 conn(0x55933ab78be0 msgr2=0x55933ab7dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.615+0000 7fe54eca5b80 1 -- 172.21.15.62:0/1206412406 shutdown_connections 2023-07-14T16:36:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.615+0000 7fe53919b700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5200a1350 0x7fe5200a3800 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe534100b00 tx=0x7fe530009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.615+0000 7fe54eca5b80 1 -- 172.21.15.62:0/1206412406 wait complete. 2023-07-14T16:36:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.615+0000 7fe54eca5b80 1 -- start start 2023-07-14T16:36:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.616+0000 7fe54eca5b80 1 Processor -- start 2023-07-14T16:36:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.617+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933a9df6b0 0x55933ab8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.617+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab83840 0x55933ab8cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.617+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab846e0 0x55933ab93ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.617+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933aa74a60 con 0x55933ab83840 2023-07-14T16:36:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.617+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933aa74760 con 0x55933a9df6b0 2023-07-14T16:36:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.617+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933a9588c0 con 0x55933ab846e0 2023-07-14T16:36:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.617+0000 7fe54eca5b80 10 client.0 _ll_register_callbacks cb 0x55933aa72700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:36:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53999c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab846e0 0x55933ab93ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53999c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab846e0 0x55933ab93ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:42078/0 (socket says 172.21.15.62:42078) 2023-07-14T16:36:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53999c700 1 -- 172.21.15.62:0/2629603594 learned_addr learned my addr 172.21.15.62:0/2629603594 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53899a700 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab83840 0x55933ab8cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53999c700 1 -- 172.21.15.62:0/2629603594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933a9df6b0 msgr2=0x55933ab8c960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T16:36:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53999c700 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933a9df6b0 0x55933ab8c960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53999c700 1 -- 172.21.15.62:0/2629603594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab83840 msgr2=0x55933ab8cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53999c700 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab83840 0x55933ab8cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.618+0000 7fe53999c700 1 -- 172.21.15.62:0/2629603594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c009220 con 0x55933ab846e0 2023-07-14T16:36:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.619+0000 7fe53999c700 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab846e0 0x55933ab93ee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000f80 tx=0x7fe52c014500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced08937fe82b6d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.619+0000 7fe51cff9700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:36:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.619+0000 7fe51cff9700 1 -- 172.21.15.62:0/2629603594 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c022070 con 0x55933ab846e0 2023-07-14T16:36:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.619+0000 7fe51cff9700 1 -- 172.21.15.62:0/2629603594 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe52c003c10 con 0x55933ab846e0 2023-07-14T16:36:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.620+0000 7fe54eca5b80 1 -- 172.21.15.62:0/2629603594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55933aa2e490 con 0x55933ab846e0 2023-07-14T16:36:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.620+0000 7fe51cff9700 1 -- 172.21.15.62:0/2629603594 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe52c00e040 con 0x55933ab846e0 2023-07-14T16:36:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.620+0000 7fe54eca5b80 10 client.30277 fetch_fsmap learned FSMap version 38 2023-07-14T16:36:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.620+0000 7fe54eca5b80 1 -- 172.21.15.62:0/2629603594 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55933a8ebc00 con 0x55933ab846e0 2023-07-14T16:36:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.621+0000 7fe51cff9700 1 -- 172.21.15.62:0/2629603594 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55933aa2e490 con 0x55933ab846e0 2023-07-14T16:36:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.621+0000 7fe51cff9700 1 -- 172.21.15.62:0/2629603594 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe52c004030 con 0x55933ab846e0 2023-07-14T16:36:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.621+0000 7fe54eca5b80 10 client.30277 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T16:36:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.621+0000 7fe54eca5b80 10 client.30277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:36:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.621+0000 7fe54eca5b80 20 client.30277 populate_metadata read hostname 'smithi062' 2023-07-14T16:36:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.621+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55933ab985d0 0x55933ab9a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.621+0000 7fe54eca5b80 4 client.30277 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:36:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.621+0000 7fe54eca5b80 1 -- 172.21.15.62:0/2629603594 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55933aacde50 con 0x55933ab985d0 2023-07-14T16:36:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.623+0000 7fe53919b700 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55933ab985d0 0x55933ab9a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.624+0000 7fe53919b700 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55933ab985d0 0x55933ab9a9e0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11cee63f05a11f21 server_cookie=22806f08b1f218a5 in_seq=0 out_seq=0 2023-07-14T16:36:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.624+0000 7fe51cff9700 10 client.30277 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.624+0000 7fe51cff9700 1 -- 172.21.15.62:0/2629603594 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55933aacde50 con 0x55933ab985d0 2023-07-14T16:36:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.624+0000 7fe51cff9700 10 client.30277 handle_command_reply: tid=0 2023-07-14T16:36:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.625+0000 7fe54eca5b80 1 client.30277 shutdown 2023-07-14T16:36:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.627+0000 7fe54eca5b80 1 -- 172.21.15.62:0/2629603594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab846e0 msgr2=0x55933ab93ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.627+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab846e0 0x55933ab93ee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000f80 tx=0x7fe52c014500 comp rx=0 tx=0).stop 2023-07-14T16:36:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.628+0000 7fe54eca5b80 1 -- 172.21.15.62:0/2629603594 shutdown_connections 2023-07-14T16:36:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.628+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab846e0 0x55933ab93ee0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.628+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55933ab985d0 0x55933ab9a9e0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.628+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933a9df6b0 0x55933ab8c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.628+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/2629603594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab83840 0x55933ab8cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.628+0000 7fe54eca5b80 1 -- 172.21.15.62:0/2629603594 >> 172.21.15.62:0/2629603594 conn(0x55933ab78be0 msgr2=0x55933ab7d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.628+0000 7fe51cff9700 0 client.30277 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.629+0000 7fe54eca5b80 1 -- 172.21.15.62:0/2629603594 shutdown_connections 2023-07-14T16:36:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.629+0000 7fe54eca5b80 1 -- 172.21.15.62:0/2629603594 wait complete. 2023-07-14T16:36:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.630+0000 7fe54eca5b80 20 client.30277 trim_cache size 0 max 16384 2023-07-14T16:36:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.659+0000 7fe54eca5b80 1 Processor -- start 2023-07-14T16:36:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.661+0000 7fe54eca5b80 1 -- start start 2023-07-14T16:36:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.661+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933a9df6b0 0x55933ab8b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.661+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab83840 0x55933ab93ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.661+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab846e0 0x55933ab8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.661+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933aacde50 con 0x55933a9df6b0 2023-07-14T16:36:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.661+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933aa2e490 con 0x55933ab846e0 2023-07-14T16:36:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.661+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ab8be10 con 0x55933ab83840 2023-07-14T16:36:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.662+0000 7fe53919b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933a9df6b0 0x55933ab8b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.662+0000 7fe53919b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933a9df6b0 0x55933ab8b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49520/0 (socket says 172.21.15.62:49520) 2023-07-14T16:36:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.662+0000 7fe53919b700 1 -- 172.21.15.62:0/764408435 learned_addr learned my addr 172.21.15.62:0/764408435 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.662+0000 7fe53999c700 1 --2- 172.21.15.62:0/764408435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab846e0 0x55933ab8c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.663+0000 7fe53919b700 1 -- 172.21.15.62:0/764408435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab83840 msgr2=0x55933ab93ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.663+0000 7fe53919b700 1 --2- 172.21.15.62:0/764408435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab83840 0x55933ab93ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.663+0000 7fe53919b700 1 -- 172.21.15.62:0/764408435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab846e0 msgr2=0x55933ab8c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.663+0000 7fe53919b700 1 --2- 172.21.15.62:0/764408435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab846e0 0x55933ab8c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.663+0000 7fe53919b700 1 -- 172.21.15.62:0/764408435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55933a9291f0 con 0x55933a9df6b0 2023-07-14T16:36:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.663+0000 7fe53919b700 1 --2- 172.21.15.62:0/764408435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933a9df6b0 0x55933ab8b5d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fe530015010 tx=0x7fe530016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5192b153a564243 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.664+0000 7fe51ffff700 1 -- 172.21.15.62:0/764408435 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530005ce0 con 0x55933a9df6b0 2023-07-14T16:36:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.664+0000 7fe51ffff700 1 -- 172.21.15.62:0/764408435 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe530004cc0 con 0x55933a9df6b0 2023-07-14T16:36:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.664+0000 7fe51ffff700 1 -- 172.21.15.62:0/764408435 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530005280 con 0x55933a9df6b0 2023-07-14T16:36:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.665+0000 7fe54eca5b80 1 -- 172.21.15.62:0/764408435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933a9df6b0 msgr2=0x55933ab8b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.665+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/764408435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933a9df6b0 0x55933ab8b5d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fe530015010 tx=0x7fe530016000 comp rx=0 tx=0).stop 2023-07-14T16:36:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.665+0000 7fe54eca5b80 1 -- 172.21.15.62:0/764408435 shutdown_connections 2023-07-14T16:36:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.665+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/764408435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab846e0 0x55933ab8c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.665+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/764408435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933a9df6b0 0x55933ab8b5d0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.665+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/764408435 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933ab83840 0x55933ab93ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.665+0000 7fe54eca5b80 1 -- 172.21.15.62:0/764408435 >> 172.21.15.62:0/764408435 conn(0x55933ab78be0 msgr2=0x55933ab9f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.666+0000 7fe54eca5b80 1 -- 172.21.15.62:0/764408435 shutdown_connections 2023-07-14T16:36:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.666+0000 7fe54eca5b80 1 -- 172.21.15.62:0/764408435 wait complete. 2023-07-14T16:36:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.666+0000 7fe54eca5b80 1 -- start start 2023-07-14T16:36:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.667+0000 7fe54eca5b80 1 Processor -- start 2023-07-14T16:36:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.667+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab90010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 0x55933ab90550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe54eca5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab93570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ab93ab0 con 0x55933ab846e0 2023-07-14T16:36:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ab93c20 con 0x55933ab83840 2023-07-14T16:36:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe54eca5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ab93de0 con 0x55933a9df6b0 2023-07-14T16:36:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe54eca5b80 10 client.0 _ll_register_callbacks cb 0x7fe52c009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:36:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe53999c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab93570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe53999c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab93570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49522/0 (socket says 172.21.15.62:49522) 2023-07-14T16:36:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.668+0000 7fe53999c700 1 -- 172.21.15.62:0/4281882532 learned_addr learned my addr 172.21.15.62:0/4281882532 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.669+0000 7fe53999c700 1 -- 172.21.15.62:0/4281882532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 msgr2=0x55933ab90010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:36:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.669+0000 7fe53999c700 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab90010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.669+0000 7fe53999c700 1 -- 172.21.15.62:0/4281882532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 msgr2=0x55933ab90550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:36:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.669+0000 7fe53999c700 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 0x55933ab90550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.669+0000 7fe53999c700 1 -- 172.21.15.62:0/4281882532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5300095e0 con 0x55933ab846e0 2023-07-14T16:36:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.670+0000 7fe53999c700 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab93570 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fe52c007cb0 tx=0x7fe52c007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb56a7881efd2cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.670+0000 7fe51cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:36:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.670+0000 7fe51cff9700 1 -- 172.21.15.62:0/4281882532 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c022070 con 0x55933ab846e0 2023-07-14T16:36:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.670+0000 7fe51cff9700 1 -- 172.21.15.62:0/4281882532 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe52c003c10 con 0x55933ab846e0 2023-07-14T16:36:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.670+0000 7fe54eca5b80 1 -- 172.21.15.62:0/4281882532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe52c00f040 con 0x55933ab846e0 2023-07-14T16:36:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.671+0000 7fe51cff9700 1 -- 172.21.15.62:0/4281882532 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe52c00e040 con 0x55933ab846e0 2023-07-14T16:36:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.671+0000 7fe54eca5b80 10 client.20820 fetch_fsmap learned FSMap version 38 2023-07-14T16:36:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.671+0000 7fe54eca5b80 1 -- 172.21.15.62:0/4281882532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55933a901250 con 0x55933ab846e0 2023-07-14T16:36:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.672+0000 7fe51cff9700 1 -- 172.21.15.62:0/4281882532 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe52c03e030 con 0x55933ab846e0 2023-07-14T16:36:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.672+0000 7fe51cff9700 1 -- 172.21.15.62:0/4281882532 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe52c015b80 con 0x55933ab846e0 2023-07-14T16:36:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.672+0000 7fe54eca5b80 10 client.20820 fetch_fsmap finished waiting for FSMap version 38 2023-07-14T16:36:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.672+0000 7fe54eca5b80 10 client.20820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:36:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.672+0000 7fe54eca5b80 20 client.20820 populate_metadata read hostname 'smithi062' 2023-07-14T16:36:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.672+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55933abed370 0x55933abef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.672+0000 7fe54eca5b80 4 client.20820 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:36:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.672+0000 7fe54eca5b80 1 -- 172.21.15.62:0/4281882532 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55933aa8fff0 con 0x55933abed370 2023-07-14T16:36:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.673+0000 7fe53919b700 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55933abed370 0x55933abef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.673+0000 7fe53919b700 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55933abed370 0x55933abef780 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cb432d0621c8fd server_cookie=1354d68a004ef75f in_seq=0 out_seq=0 2023-07-14T16:36:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.673+0000 7fe51cff9700 10 client.20820 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.694+0000 7fe51cff9700 1 -- 172.21.15.62:0/4281882532 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55933aa8fff0 con 0x55933abed370 2023-07-14T16:36:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.694+0000 7fe51cff9700 10 client.20820 handle_command_reply: tid=0 2023-07-14T16:36:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.694+0000 7fe54eca5b80 1 client.20820 shutdown 2023-07-14T16:36:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.694+0000 7fe54eca5b80 1 -- 172.21.15.62:0/4281882532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 msgr2=0x55933ab93570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.694+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab93570 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fe52c007cb0 tx=0x7fe52c007500 comp rx=0 tx=0).stop 2023-07-14T16:36:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.695+0000 7fe54eca5b80 1 -- 172.21.15.62:0/4281882532 shutdown_connections 2023-07-14T16:36:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.695+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55933ab846e0 0x55933ab93570 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.695+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55933a9df6b0 0x55933ab90010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.695+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55933abed370 0x55933abef780 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.695+0000 7fe54eca5b80 1 --2- 172.21.15.62:0/4281882532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55933ab83840 0x55933ab90550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.695+0000 7fe54eca5b80 1 -- 172.21.15.62:0/4281882532 >> 172.21.15.62:0/4281882532 conn(0x55933ab78be0 msgr2=0x55933ab8b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.696+0000 7fe51cff9700 0 client.20820 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.696+0000 7fe54eca5b80 1 -- 172.21.15.62:0/4281882532 shutdown_connections 2023-07-14T16:36:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.697+0000 7fe54eca5b80 1 -- 172.21.15.62:0/4281882532 wait complete. 2023-07-14T16:36:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.698+0000 7fe54eca5b80 20 client.20820 trim_cache size 0 max 16384 2023-07-14T16:36:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:36:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:36:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:36:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:36:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.699+0000 7fe53b3ff700 1 -- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5200a1350 msgr2=0x7fe5200a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.699+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5200a1350 0x7fe5200a3800 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe534100b00 tx=0x7fe530009260 comp rx=0 tx=0).stop 2023-07-14T16:36:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.699+0000 7fe53b3ff700 1 -- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe534100ca0 msgr2=0x7fe5341a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.699+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe534100ca0 0x7fe5341a5750 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe5240098e0 tx=0x7fe524005740 comp rx=0 tx=0).stop 2023-07-14T16:36:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.700+0000 7fe53b3ff700 1 -- 172.21.15.62:0/2146626247 shutdown_connections 2023-07-14T16:36:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.700+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe534101630 0x7fe5341a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.700+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fe5200a1350 0x7fe5200a3800 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.700+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5340ffab0 0x7fe5341a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.700+0000 7fe53b3ff700 1 --2- 172.21.15.62:0/2146626247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe534100ca0 0x7fe5341a5750 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.700+0000 7fe53b3ff700 1 -- 172.21.15.62:0/2146626247 >> 172.21.15.62:0/2146626247 conn(0x7fe5340fb0b0 msgr2=0x7fe5340fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.700+0000 7fe53b3ff700 1 -- 172.21.15.62:0/2146626247 shutdown_connections 2023-07-14T16:36:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:26.700+0000 7fe53b3ff700 1 -- 172.21.15.62:0/2146626247 wait complete. 2023-07-14T16:36:27.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:26 smithi164 ceph-mon[97427]: pgmap v981: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:36:27.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:26 smithi062 ceph-mon[95543]: pgmap v981: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:36:27.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:26 smithi081 ceph-mon[127795]: pgmap v981: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 3.94k op/s 2023-07-14T16:36:29.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:29 smithi164 ceph-mon[97427]: pgmap v982: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2023-07-14T16:36:29.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:29 smithi062 ceph-mon[95543]: pgmap v982: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2023-07-14T16:36:29.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:29 smithi081 ceph-mon[127795]: pgmap v982: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 4.37k op/s 2023-07-14T16:36:30.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:30 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:36:30.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:30 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:36:30.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:30 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:36:31.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:31 smithi164 ceph-mon[97427]: pgmap v983: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:36:31.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:31 smithi062 ceph-mon[95543]: pgmap v983: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:36:31.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:31 smithi081 ceph-mon[127795]: pgmap v983: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:36:33.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:33 smithi062 ceph-mon[95543]: pgmap v984: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2023-07-14T16:36:33.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:33 smithi081 ceph-mon[127795]: pgmap v984: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2023-07-14T16:36:33.517 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:33 smithi164 ceph-mon[97427]: pgmap v984: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.83k op/s 2023-07-14T16:36:35.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:35 smithi062 ceph-mon[95543]: pgmap v985: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.32k op/s 2023-07-14T16:36:35.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:35 smithi081 ceph-mon[127795]: pgmap v985: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.32k op/s 2023-07-14T16:36:35.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:35 smithi164 ceph-mon[97427]: pgmap v985: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 4.32k op/s 2023-07-14T16:36:37.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:37 smithi062 ceph-mon[95543]: pgmap v986: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2023-07-14T16:36:37.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:37 smithi081 ceph-mon[127795]: pgmap v986: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2023-07-14T16:36:37.517 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:37 smithi164 ceph-mon[97427]: pgmap v986: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2023-07-14T16:36:39.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:39 smithi062 ceph-mon[95543]: pgmap v987: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2023-07-14T16:36:39.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:39 smithi081 ceph-mon[127795]: pgmap v987: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2023-07-14T16:36:39.517 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:39 smithi164 ceph-mon[97427]: pgmap v987: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 4.06k op/s 2023-07-14T16:36:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.011+0000 7fb8461ff700 1 -- 172.21.15.62:0/92954622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 msgr2=0x7fb840107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.011+0000 7fb8461ff700 1 --2- 172.21.15.62:0/92954622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 0x7fb840107b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb834009bc0 tx=0x7fb834009ed0 comp rx=0 tx=0).stop 2023-07-14T16:36:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.014+0000 7fb8461ff700 1 -- 172.21.15.62:0/92954622 shutdown_connections 2023-07-14T16:36:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.014+0000 7fb8461ff700 1 --2- 172.21.15.62:0/92954622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 0x7fb840107b90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.014+0000 7fb8461ff700 1 --2- 172.21.15.62:0/92954622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840068620 0x7fb8401026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.014+0000 7fb8461ff700 1 --2- 172.21.15.62:0/92954622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840102bf0 0x7fb840105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.014+0000 7fb8461ff700 1 -- 172.21.15.62:0/92954622 >> 172.21.15.62:0/92954622 conn(0x7fb8400fa000 msgr2=0x7fb8400fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.015+0000 7fb8461ff700 1 -- 172.21.15.62:0/92954622 shutdown_connections 2023-07-14T16:36:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.015+0000 7fb8461ff700 1 -- 172.21.15.62:0/92954622 wait complete. 2023-07-14T16:36:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.016+0000 7fb8461ff700 1 Processor -- start 2023-07-14T16:36:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.017+0000 7fb8461ff700 1 -- start start 2023-07-14T16:36:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.017+0000 7fb8461ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840068620 0x7fb84006aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.017+0000 7fb8461ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840102bf0 0x7fb84006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.017+0000 7fb8461ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 0x7fb84006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:39.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.017+0000 7fb8461ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84006c520 con 0x7fb840068620 2023-07-14T16:36:39.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.017+0000 7fb8461ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84006c690 con 0x7fb840102bf0 2023-07-14T16:36:39.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.017+0000 7fb8461ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84006c850 con 0x7fb8401056e0 2023-07-14T16:36:39.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.018+0000 7fb83f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840068620 0x7fb84006aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:39.990 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.018+0000 7fb83f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840068620 0x7fb84006aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56334/0 (socket says 172.21.15.62:56334) 2023-07-14T16:36:39.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.018+0000 7fb83f7fe700 1 -- 172.21.15.62:0/2803281403 learned_addr learned my addr 172.21.15.62:0/2803281403 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:39.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.018+0000 7fb83ffff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 0x7fb84006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:39.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.018+0000 7fb83effd700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840102bf0 0x7fb84006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:39.991 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.019+0000 7fb83f7fe700 1 -- 172.21.15.62:0/2803281403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 msgr2=0x7fb84006bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.019+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 0x7fb84006bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.019+0000 7fb83f7fe700 1 -- 172.21.15.62:0/2803281403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840102bf0 msgr2=0x7fb84006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.019+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840102bf0 0x7fb84006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.019+0000 7fb83f7fe700 1 -- 172.21.15.62:0/2803281403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb834009820 con 0x7fb840068620 2023-07-14T16:36:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.019+0000 7fb83ffff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 0x7fb84006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:36:39.992 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.019+0000 7fb83effd700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840102bf0 0x7fb84006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:36:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.019+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840068620 0x7fb84006aab0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb83800efc0 tx=0x7fb83800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.020+0000 7fb83cff9700 1 -- 172.21.15.62:0/2803281403 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800cad0 con 0x7fb840068620 2023-07-14T16:36:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.020+0000 7fb83cff9700 1 -- 172.21.15.62:0/2803281403 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb83800f040 con 0x7fb840068620 2023-07-14T16:36:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.020+0000 7fb83cff9700 1 -- 172.21.15.62:0/2803281403 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838014480 con 0x7fb840068620 2023-07-14T16:36:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.020+0000 7fb8461ff700 1 -- 172.21.15.62:0/2803281403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84006cad0 con 0x7fb840068620 2023-07-14T16:36:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.021+0000 7fb8461ff700 1 -- 172.21.15.62:0/2803281403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb840071300 con 0x7fb840068620 2023-07-14T16:36:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.029+0000 7fb859a34b80 1 Processor -- start 2023-07-14T16:36:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.030+0000 7fb83cff9700 1 -- 172.21.15.62:0/2803281403 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb83800cc30 con 0x7fb840068620 2023-07-14T16:36:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.030+0000 7fb859a34b80 1 -- start start 2023-07-14T16:36:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.031+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.031+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cc840 0x5622e26cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.031+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.031+0000 7fb859a34b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e25d1c50 con 0x5622e26cd6e0 2023-07-14T16:36:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.031+0000 7fb859a34b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e26249e0 con 0x5622e26cc840 2023-07-14T16:36:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.031+0000 7fb859a34b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e261de70 con 0x5622e25286b0 2023-07-14T16:36:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.032+0000 7fb83f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cc840 0x5622e26cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.032+0000 7fb83f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cc840 0x5622e26cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41540/0 (socket says 172.21.15.62:41540) 2023-07-14T16:36:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.032+0000 7fb83f7fe700 1 -- 172.21.15.62:0/1998733455 learned_addr learned my addr 172.21.15.62:0/1998733455 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.032+0000 7fb83ffff700 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83f7fe700 1 -- 172.21.15.62:0/1998733455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 msgr2=0x5622e26cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83f7fe700 1 -- 172.21.15.62:0/1998733455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 msgr2=0x5622e26cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83effd700 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83f7fe700 1 -- 172.21.15.62:0/1998733455 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622e244bd80 con 0x5622e26cc840 2023-07-14T16:36:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83ffff700 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:36:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83effd700 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:36:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.033+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cc840 0x5622e26cd1a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb8380d0a00 tx=0x7fb8380d0d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442b1c75c407e8b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.034+0000 7fb83cff9700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1320 0x7fb8280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.034+0000 7fb83cff9700 1 -- 172.21.15.62:0/2803281403 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8380ccb90 con 0x7fb840068620 2023-07-14T16:36:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.034+0000 7fb825ffb700 1 -- 172.21.15.62:0/1998733455 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380d3070 con 0x5622e26cc840 2023-07-14T16:36:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.034+0000 7fb825ffb700 1 -- 172.21.15.62:0/1998733455 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8380d0640 con 0x5622e26cc840 2023-07-14T16:36:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.034+0000 7fb825ffb700 1 -- 172.21.15.62:0/1998733455 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380dbe70 con 0x5622e26cc840 2023-07-14T16:36:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.035+0000 7fb83ffff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1320 0x7fb8280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.036+0000 7fb859a34b80 1 -- 172.21.15.62:0/1998733455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cc840 msgr2=0x5622e26cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.036+0000 7fb83ffff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1320 0x7fb8280a37d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb83400b5c0 tx=0x7fb834005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.036+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cc840 0x5622e26cd1a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb8380d0a00 tx=0x7fb8380d0d10 comp rx=0 tx=0).stop 2023-07-14T16:36:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.037+0000 7fb859a34b80 1 -- 172.21.15.62:0/1998733455 shutdown_connections 2023-07-14T16:36:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.037+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.037+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.037+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1998733455 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cc840 0x5622e26cd1a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.037+0000 7fb859a34b80 1 -- 172.21.15.62:0/1998733455 >> 172.21.15.62:0/1998733455 conn(0x5622e26c1be0 msgr2=0x5622e26c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.037+0000 7fb859a34b80 1 -- 172.21.15.62:0/1998733455 shutdown_connections 2023-07-14T16:36:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.039+0000 7fb859a34b80 1 -- 172.21.15.62:0/1998733455 wait complete. 2023-07-14T16:36:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.039+0000 7fb859a34b80 1 -- start start 2023-07-14T16:36:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.040+0000 7fb859a34b80 1 Processor -- start 2023-07-14T16:36:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.041+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e25286b0 0x5622e26d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.041+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cc840 0x5622e26d5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.041+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26cd6e0 0x5622e26dce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.041+0000 7fb859a34b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e25bda60 con 0x5622e26cc840 2023-07-14T16:36:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.041+0000 7fb859a34b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e25bd760 con 0x5622e25286b0 2023-07-14T16:36:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.041+0000 7fb859a34b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e24a18c0 con 0x5622e26cd6e0 2023-07-14T16:36:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.041+0000 7fb859a34b80 10 client.0 _ll_register_callbacks cb 0x5622e25bb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:36:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.042+0000 7fb83effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cc840 0x5622e26d5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.042+0000 7fb83effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cc840 0x5622e26d5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56348/0 (socket says 172.21.15.62:56348) 2023-07-14T16:36:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.042+0000 7fb83effd700 1 -- 172.21.15.62:0/1532519522 learned_addr learned my addr 172.21.15.62:0/1532519522 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.043+0000 7fb83ffff700 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26cd6e0 0x5622e26dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.043+0000 7fb83effd700 1 -- 172.21.15.62:0/1532519522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26cd6e0 msgr2=0x5622e26dce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.043+0000 7fb83effd700 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26cd6e0 0x5622e26dce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.043+0000 7fb83effd700 1 -- 172.21.15.62:0/1532519522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e25286b0 msgr2=0x5622e26d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.043+0000 7fb83effd700 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e25286b0 0x5622e26d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.043+0000 7fb83effd700 1 -- 172.21.15.62:0/1532519522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb840071300 con 0x5622e26cc840 2023-07-14T16:36:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.045+0000 7fb83effd700 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cc840 0x5622e26d5e40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb83000e470 tx=0x7fb83000e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa575de7125a1fd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.045+0000 7fb81effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:36:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.045+0000 7fb81effd700 1 -- 172.21.15.62:0/1532519522 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000c070 con 0x5622e26cc840 2023-07-14T16:36:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.046+0000 7fb81effd700 1 -- 172.21.15.62:0/1532519522 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb830013410 con 0x5622e26cc840 2023-07-14T16:36:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.046+0000 7fb859a34b80 1 -- 172.21.15.62:0/1532519522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622e2616e50 con 0x5622e26cc840 2023-07-14T16:36:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.046+0000 7fb81effd700 1 -- 172.21.15.62:0/1532519522 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8300135d0 con 0x5622e26cc840 2023-07-14T16:36:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.046+0000 7fb859a34b80 10 client.20826 fetch_fsmap learned FSMap version 39 2023-07-14T16:36:40.017 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.046+0000 7fb859a34b80 1 -- 172.21.15.62:0/1532519522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5622e2434c00 con 0x5622e26cc840 2023-07-14T16:36:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.047+0000 7fb81effd700 1 -- 172.21.15.62:0/1532519522 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5622e2616e50 con 0x5622e26cc840 2023-07-14T16:36:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.047+0000 7fb81effd700 1 -- 172.21.15.62:0/1532519522 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb83001c7b0 con 0x5622e26cc840 2023-07-14T16:36:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.047+0000 7fb859a34b80 10 client.20826 fetch_fsmap finished waiting for FSMap version 39 2023-07-14T16:36:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.047+0000 7fb859a34b80 10 client.20826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:36:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.047+0000 7fb859a34b80 20 client.20826 populate_metadata read hostname 'smithi062' 2023-07-14T16:36:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.047+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622e26e1750 0x5622e26e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.047+0000 7fb859a34b80 4 client.20826 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:36:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.047+0000 7fb859a34b80 1 -- 172.21.15.62:0/1532519522 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622e25d8ff0 con 0x5622e26e1750 2023-07-14T16:36:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.048+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622e26e1750 0x5622e26e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.048+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622e26e1750 0x5622e26e3b60 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfb78ffcf9ec90a server_cookie=feabc7ac84d8d4ba in_seq=0 out_seq=0 2023-07-14T16:36:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.049+0000 7fb81effd700 10 client.20826 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.053+0000 7fb81effd700 1 -- 172.21.15.62:0/1532519522 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5622e25d8ff0 con 0x5622e26e1750 2023-07-14T16:36:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.054+0000 7fb81effd700 10 client.20826 handle_command_reply: tid=0 2023-07-14T16:36:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.054+0000 7fb859a34b80 1 client.20826 shutdown 2023-07-14T16:36:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb859a34b80 1 -- 172.21.15.62:0/1532519522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cc840 msgr2=0x5622e26d5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cc840 0x5622e26d5e40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb83000e470 tx=0x7fb83000e830 comp rx=0 tx=0).stop 2023-07-14T16:36:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb859a34b80 1 -- 172.21.15.62:0/1532519522 shutdown_connections 2023-07-14T16:36:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26cd6e0 0x5622e26dce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622e26e1750 0x5622e26e3b60 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e25286b0 0x5622e26d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1532519522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cc840 0x5622e26d5e40 secure :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb83000e470 tx=0x7fb83000e830 comp rx=0 tx=0).stop 2023-07-14T16:36:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb859a34b80 1 -- 172.21.15.62:0/1532519522 >> 172.21.15.62:0/1532519522 conn(0x5622e26c1be0 msgr2=0x5622e26c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.055+0000 7fb81effd700 0 client.20826 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.056+0000 7fb859a34b80 1 -- 172.21.15.62:0/1532519522 shutdown_connections 2023-07-14T16:36:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.057+0000 7fb859a34b80 1 -- 172.21.15.62:0/1532519522 wait complete. 2023-07-14T16:36:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.060+0000 7fb859a34b80 20 client.20826 trim_cache size 0 max 16384 2023-07-14T16:36:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.089+0000 7fb859a34b80 1 Processor -- start 2023-07-14T16:36:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.091+0000 7fb859a34b80 1 -- start start 2023-07-14T16:36:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.092+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e25286b0 0x5622e26d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.092+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.092+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26d4060 0x5622e26d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.092+0000 7fb859a34b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e26d54d0 con 0x5622e26cd6e0 2023-07-14T16:36:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.092+0000 7fb859a34b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e2616e80 con 0x5622e25286b0 2023-07-14T16:36:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.092+0000 7fb859a34b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e26d6580 con 0x5622e26d4060 2023-07-14T16:36:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.092+0000 7fb83effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.093+0000 7fb83effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26dcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56362/0 (socket says 172.21.15.62:56362) 2023-07-14T16:36:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.093+0000 7fb83effd700 1 -- 172.21.15.62:0/1459811431 learned_addr learned my addr 172.21.15.62:0/1459811431 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.093+0000 7fb83effd700 1 -- 172.21.15.62:0/1459811431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26d4060 msgr2=0x5622e26d0630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:36:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.093+0000 7fb83effd700 1 --2- 172.21.15.62:0/1459811431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26d4060 0x5622e26d0630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.093+0000 7fb83effd700 1 -- 172.21.15.62:0/1459811431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e25286b0 msgr2=0x5622e26d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.093+0000 7fb83effd700 1 --2- 172.21.15.62:0/1459811431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e25286b0 0x5622e26d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.093+0000 7fb83effd700 1 -- 172.21.15.62:0/1459811431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622e24721f0 con 0x5622e26cd6e0 2023-07-14T16:36:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.094+0000 7fb83effd700 1 --2- 172.21.15.62:0/1459811431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26dcda0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb83001cb10 tx=0x7fb83000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f50289591f0df22 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.095+0000 7fb8267fc700 1 -- 172.21.15.62:0/1459811431 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000c070 con 0x5622e26cd6e0 2023-07-14T16:36:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.096+0000 7fb8267fc700 1 -- 172.21.15.62:0/1459811431 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb83001c7b0 con 0x5622e26cd6e0 2023-07-14T16:36:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.096+0000 7fb8267fc700 1 -- 172.21.15.62:0/1459811431 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83002ed80 con 0x5622e26cd6e0 2023-07-14T16:36:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.097+0000 7fb859a34b80 1 -- 172.21.15.62:0/1459811431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 msgr2=0x5622e26dcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.097+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1459811431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26dcda0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb83001cb10 tx=0x7fb83000ece0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.097+0000 7fb859a34b80 1 -- 172.21.15.62:0/1459811431 shutdown_connections 2023-07-14T16:36:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.097+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1459811431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e26d4060 0x5622e26d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.097+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1459811431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e25286b0 0x5622e26d3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.097+0000 7fb859a34b80 1 --2- 172.21.15.62:0/1459811431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26cd6e0 0x5622e26dcda0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.097+0000 7fb859a34b80 1 -- 172.21.15.62:0/1459811431 >> 172.21.15.62:0/1459811431 conn(0x5622e26c1be0 msgr2=0x5622e26e84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.098+0000 7fb859a34b80 1 -- 172.21.15.62:0/1459811431 shutdown_connections 2023-07-14T16:36:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.098+0000 7fb859a34b80 1 -- 172.21.15.62:0/1459811431 wait complete. 2023-07-14T16:36:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.098+0000 7fb859a34b80 1 -- start start 2023-07-14T16:36:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.099+0000 7fb859a34b80 1 Processor -- start 2023-07-14T16:36:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.099+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26d8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.099+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cd6e0 0x5622e26d9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.099+0000 7fb859a34b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26d4060 0x5622e26dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.099+0000 7fb859a34b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e26dc880 con 0x5622e26d4060 2023-07-14T16:36:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.099+0000 7fb859a34b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e26dc9f0 con 0x5622e26cd6e0 2023-07-14T16:36:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.099+0000 7fb859a34b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e26dcbb0 con 0x5622e25286b0 2023-07-14T16:36:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.099+0000 7fb859a34b80 10 client.0 _ll_register_callbacks cb 0x7fb830009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:36:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.101+0000 7fb83effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cd6e0 0x5622e26d9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.101+0000 7fb83ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26d4060 0x5622e26dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.101+0000 7fb83f7fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26d8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.101+0000 7fb83ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26d4060 0x5622e26dc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56376/0 (socket says 172.21.15.62:56376) 2023-07-14T16:36:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.101+0000 7fb83ffff700 1 -- 172.21.15.62:0/2429052225 learned_addr learned my addr 172.21.15.62:0/2429052225 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.102+0000 7fb83effd700 1 -- 172.21.15.62:0/2429052225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 msgr2=0x5622e26d8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.102+0000 7fb83effd700 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26d8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.102+0000 7fb83effd700 1 -- 172.21.15.62:0/2429052225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26d4060 msgr2=0x5622e26dc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.102+0000 7fb83effd700 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26d4060 0x5622e26dc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.102+0000 7fb83effd700 1 -- 172.21.15.62:0/2429052225 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622e2434c00 con 0x5622e26cd6e0 2023-07-14T16:36:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.102+0000 7fb83effd700 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cd6e0 0x5622e26d9320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb83001cc10 tx=0x7fb83000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eed34be12908502 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.102+0000 7fb81effd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:36:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.103+0000 7fb81effd700 1 -- 172.21.15.62:0/2429052225 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000c040 con 0x5622e26cd6e0 2023-07-14T16:36:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.103+0000 7fb81effd700 1 -- 172.21.15.62:0/2429052225 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb830000e40 con 0x5622e26cd6e0 2023-07-14T16:36:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.103+0000 7fb859a34b80 1 -- 172.21.15.62:0/2429052225 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622e2577490 con 0x5622e26cd6e0 2023-07-14T16:36:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.103+0000 7fb81effd700 1 -- 172.21.15.62:0/2429052225 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb83003d030 con 0x5622e26cd6e0 2023-07-14T16:36:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.104+0000 7fb859a34b80 10 client.30404 fetch_fsmap learned FSMap version 39 2023-07-14T16:36:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.104+0000 7fb859a34b80 1 -- 172.21.15.62:0/2429052225 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5622e244a250 con 0x5622e26cd6e0 2023-07-14T16:36:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.104+0000 7fb81effd700 1 -- 172.21.15.62:0/2429052225 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5622e2577490 con 0x5622e26cd6e0 2023-07-14T16:36:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.104+0000 7fb81effd700 1 -- 172.21.15.62:0/2429052225 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 39) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb830037be0 con 0x5622e26cd6e0 2023-07-14T16:36:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.105+0000 7fb859a34b80 10 client.30404 fetch_fsmap finished waiting for FSMap version 39 2023-07-14T16:36:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.105+0000 7fb859a34b80 10 client.30404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:36:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.105+0000 7fb859a34b80 20 client.30404 populate_metadata read hostname 'smithi062' 2023-07-14T16:36:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.105+0000 7fb859a34b80 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622e2758460 0x5622e275a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.105+0000 7fb859a34b80 4 client.30404 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:36:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.105+0000 7fb859a34b80 1 -- 172.21.15.62:0/2429052225 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5622e246cf10 con 0x5622e2758460 2023-07-14T16:36:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.106+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622e2758460 0x5622e275a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.107+0000 7fb83f7fe700 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622e2758460 0x5622e275a870 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8fd1460fa9851fb server_cookie=8a7425227495bd09 in_seq=0 out_seq=0 2023-07-14T16:36:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.107+0000 7fb81effd700 10 client.30404 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.132+0000 7fb81effd700 1 -- 172.21.15.62:0/2429052225 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5622e246cf10 con 0x5622e2758460 2023-07-14T16:36:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.132+0000 7fb81effd700 10 client.30404 handle_command_reply: tid=0 2023-07-14T16:36:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.132+0000 7fb859a34b80 1 client.30404 shutdown 2023-07-14T16:36:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.132+0000 7fb859a34b80 1 -- 172.21.15.62:0/2429052225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cd6e0 msgr2=0x5622e26d9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.132+0000 7fb859a34b80 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cd6e0 0x5622e26d9320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb83001cc10 tx=0x7fb83000ec70 comp rx=0 tx=0).stop 2023-07-14T16:36:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.133+0000 7fb859a34b80 1 -- 172.21.15.62:0/2429052225 shutdown_connections 2023-07-14T16:36:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.133+0000 7fb859a34b80 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622e26d4060 0x5622e26dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.133+0000 7fb859a34b80 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5622e25286b0 0x5622e26d8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.133+0000 7fb859a34b80 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5622e2758460 0x5622e275a870 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.133+0000 7fb859a34b80 1 --2- 172.21.15.62:0/2429052225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5622e26cd6e0 0x5622e26d9320 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.133+0000 7fb859a34b80 1 -- 172.21.15.62:0/2429052225 >> 172.21.15.62:0/2429052225 conn(0x5622e26c1be0 msgr2=0x5622e26d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.133+0000 7fb81effd700 0 client.30404 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.134+0000 7fb859a34b80 1 -- 172.21.15.62:0/2429052225 shutdown_connections 2023-07-14T16:36:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.134+0000 7fb859a34b80 1 -- 172.21.15.62:0/2429052225 wait complete. 2023-07-14T16:36:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.134+0000 7fb859a34b80 20 client.30404 trim_cache size 0 max 16384 2023-07-14T16:36:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:36:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:36:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:36:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:36:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 -- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1320 msgr2=0x7fb8280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1320 0x7fb8280a37d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb83400b5c0 tx=0x7fb834005fb0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 -- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840068620 msgr2=0x7fb84006aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840068620 0x7fb84006aab0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb83800efc0 tx=0x7fb83800c3a0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 -- 172.21.15.62:0/2803281403 shutdown_connections 2023-07-14T16:36:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8401056e0 0x7fb84006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb840068620 0x7fb84006aab0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb8280a1320 0x7fb8280a37d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 --2- 172.21.15.62:0/2803281403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb840102bf0 0x7fb84006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.137+0000 7fb8461ff700 1 -- 172.21.15.62:0/2803281403 >> 172.21.15.62:0/2803281403 conn(0x7fb8400fa000 msgr2=0x7fb840100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.138+0000 7fb8461ff700 1 -- 172.21.15.62:0/2803281403 shutdown_connections 2023-07-14T16:36:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:40.138+0000 7fb8461ff700 1 -- 172.21.15.62:0/2803281403 wait complete. 2023-07-14T16:36:41.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:41 smithi062 ceph-mon[95543]: pgmap v988: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2023-07-14T16:36:41.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:41 smithi081 ceph-mon[127795]: pgmap v988: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2023-07-14T16:36:41.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:41 smithi164 ceph-mon[97427]: pgmap v988: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.13k op/s 2023-07-14T16:36:43.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:43 smithi062 ceph-mon[95543]: pgmap v989: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2023-07-14T16:36:43.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:43 smithi081 ceph-mon[127795]: pgmap v989: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2023-07-14T16:36:43.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:43 smithi164 ceph-mon[97427]: pgmap v989: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2023-07-14T16:36:45.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:45 smithi062 ceph-mon[95543]: pgmap v990: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:36:45.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:45 smithi081 ceph-mon[127795]: pgmap v990: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:36:45.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:45 smithi164 ceph-mon[97427]: pgmap v990: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2023-07-14T16:36:46.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:46 smithi081 ceph-mon[127795]: pgmap v991: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2023-07-14T16:36:46.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:46 smithi164 ceph-mon[97427]: pgmap v991: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2023-07-14T16:36:46.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:46 smithi062 ceph-mon[95543]: pgmap v991: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2023-07-14T16:36:49.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:48 smithi164 ceph-mon[97427]: pgmap v992: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.27k op/s 2023-07-14T16:36:49.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:48 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:36:49.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:48 smithi062 ceph-mon[95543]: pgmap v992: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.27k op/s 2023-07-14T16:36:49.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:48 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:36:49.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:48 smithi081 ceph-mon[127795]: pgmap v992: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.27k op/s 2023-07-14T16:36:49.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:48 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:36:50.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:49 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:36:50.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:49 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:50.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:49 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:50.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:49 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:36:50.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:49 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:50.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:49 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:50.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:49 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:36:50.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:49 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:50.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:49 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:51.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:50 smithi164 ceph-mon[97427]: pgmap v993: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-07-14T16:36:51.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:50 smithi062 ceph-mon[95543]: pgmap v993: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-07-14T16:36:51.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:51 smithi081 ceph-mon[127795]: pgmap v993: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.33k op/s 2023-07-14T16:36:53.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:53 smithi164 ceph-mon[97427]: pgmap v994: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-07-14T16:36:53.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:53 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:53 smithi062 ceph-mon[95543]: pgmap v994: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-07-14T16:36:53.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:53 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:53 smithi081 ceph-mon[127795]: pgmap v994: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.82k op/s 2023-07-14T16:36:53.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:53 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.449+0000 7fd7d9dff700 1 -- 172.21.15.62:0/519105127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d410c0d0 msgr2=0x7fd7d410e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.449+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/519105127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d410c0d0 0x7fd7d410e4b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01c0d0 tx=0x7fd7cc01c3e0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.451+0000 7fd7d9dff700 1 -- 172.21.15.62:0/519105127 shutdown_connections 2023-07-14T16:36:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.451+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/519105127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d410c0d0 0x7fd7d410e4b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.451+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/519105127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7d4074550 0x7fd7d40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.451+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/519105127 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7d4072f70 0x7fd7d40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.451+0000 7fd7d9dff700 1 -- 172.21.15.62:0/519105127 >> 172.21.15.62:0/519105127 conn(0x7fd7d40fb0e0 msgr2=0x7fd7d40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.452+0000 7fd7d9dff700 1 -- 172.21.15.62:0/519105127 shutdown_connections 2023-07-14T16:36:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.452+0000 7fd7d9dff700 1 -- 172.21.15.62:0/519105127 wait complete. 2023-07-14T16:36:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.453+0000 7fd7d9dff700 1 Processor -- start 2023-07-14T16:36:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.454+0000 7fd7d9dff700 1 -- start start 2023-07-14T16:36:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.454+0000 7fd7d9dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d4072f70 0x7fd7d4193fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.454+0000 7fd7d9dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7d4074550 0x7fd7d4194520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.454+0000 7fd7d9dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7d410c0d0 0x7fd7d4194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.454+0000 7fd7d9dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4195220 con 0x7fd7d410c0d0 2023-07-14T16:36:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.454+0000 7fd7d9dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4195390 con 0x7fd7d4072f70 2023-07-14T16:36:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.454+0000 7fd7d9dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4195550 con 0x7fd7d4074550 2023-07-14T16:36:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.461+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7d4074550 0x7fd7d4194520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.461+0000 7fd7d37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d4072f70 0x7fd7d4193fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.461+0000 7fd7d37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d4072f70 0x7fd7d4193fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:51040/0 (socket says 172.21.15.62:51040) 2023-07-14T16:36:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.461+0000 7fd7d37fe700 1 -- 172.21.15.62:0/443018811 learned_addr learned my addr 172.21.15.62:0/443018811 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.467+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7d410c0d0 0x7fd7d4194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.469+0000 7fd7d3fff700 1 -- 172.21.15.62:0/443018811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7d4074550 msgr2=0x7fd7d4194520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.469+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7d4074550 0x7fd7d4194520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.469+0000 7fd7d3fff700 1 -- 172.21.15.62:0/443018811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d4072f70 msgr2=0x7fd7d4193fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.469+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d4072f70 0x7fd7d4193fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.469+0000 7fd7d3fff700 1 -- 172.21.15.62:0/443018811 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc01ad10 con 0x7fd7d410c0d0 2023-07-14T16:36:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.469+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7d410c0d0 0x7fd7d4194a80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc026000 tx=0x7fd7cc01dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.476+0000 7fd7d0ff9700 1 -- 172.21.15.62:0/443018811 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc01f050 con 0x7fd7d410c0d0 2023-07-14T16:36:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.476+0000 7fd7d0ff9700 1 -- 172.21.15.62:0/443018811 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7cc01dec0 con 0x7fd7d410c0d0 2023-07-14T16:36:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.476+0000 7fd7d0ff9700 1 -- 172.21.15.62:0/443018811 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc02c6b0 con 0x7fd7d410c0d0 2023-07-14T16:36:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.477+0000 7fd7d9dff700 1 -- 172.21.15.62:0/443018811 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d41957d0 con 0x7fd7d410c0d0 2023-07-14T16:36:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.477+0000 7fd7d9dff700 1 -- 172.21.15.62:0/443018811 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d4069f30 con 0x7fd7d410c0d0 2023-07-14T16:36:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.484+0000 7fd7ed7f4b80 1 Processor -- start 2023-07-14T16:36:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.489+0000 7fd7ed7f4b80 1 -- start start 2023-07-14T16:36:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.489+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a3f5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.489+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.489+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a3f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.489+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a2fac50 con 0x55915a3f5840 2023-07-14T16:36:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.489+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a34d9e0 con 0x55915a2516b0 2023-07-14T16:36:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.489+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a346e70 con 0x55915a3f66e0 2023-07-14T16:36:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.491+0000 7fd7d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.491+0000 7fd7d37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34976/0 (socket says 172.21.15.62:34976) 2023-07-14T16:36:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.491+0000 7fd7d37fe700 1 -- 172.21.15.62:0/93665880 learned_addr learned my addr 172.21.15.62:0/93665880 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.492+0000 7fd7d37fe700 1 -- 172.21.15.62:0/93665880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 msgr2=0x55915a3f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.492+0000 7fd7d37fe700 1 --2- 172.21.15.62:0/93665880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a3f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.492+0000 7fd7d37fe700 1 -- 172.21.15.62:0/93665880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 msgr2=0x55915a3f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.492+0000 7fd7d37fe700 1 --2- 172.21.15.62:0/93665880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a3f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.492+0000 7fd7d37fe700 1 -- 172.21.15.62:0/93665880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55915a174d80 con 0x55915a3f5840 2023-07-14T16:36:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.492+0000 7fd7d37fe700 1 --2- 172.21.15.62:0/93665880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3f61a0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009b80 tx=0x7fd7c4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386e05c09cc33dd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.497+0000 7fd7b9ffb700 1 -- 172.21.15.62:0/93665880 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400e9f0 con 0x55915a3f5840 2023-07-14T16:36:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.497+0000 7fd7b9ffb700 1 -- 172.21.15.62:0/93665880 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7c4013470 con 0x55915a3f5840 2023-07-14T16:36:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.497+0000 7fd7b9ffb700 1 -- 172.21.15.62:0/93665880 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4012630 con 0x55915a3f5840 2023-07-14T16:36:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.502+0000 7fd7d0ff9700 1 -- 172.21.15.62:0/443018811 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd7cc005040 con 0x7fd7d410c0d0 2023-07-14T16:36:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.503+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/93665880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 msgr2=0x55915a3f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.503+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/93665880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3f61a0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009b80 tx=0x7fd7c4009e90 comp rx=0 tx=0).stop 2023-07-14T16:36:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.505+0000 7fd7d0ff9700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7bc0a1350 0x7fd7bc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.505+0000 7fd7d0ff9700 1 -- 172.21.15.62:0/443018811 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7cc0de650 con 0x7fd7d410c0d0 2023-07-14T16:36:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.505+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/93665880 shutdown_connections 2023-07-14T16:36:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.505+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/93665880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a3f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.505+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/93665880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a3f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.505+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/93665880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3f61a0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.505+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/93665880 >> 172.21.15.62:0/93665880 conn(0x55915a3eabe0 msgr2=0x55915a3efcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.509+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/93665880 shutdown_connections 2023-07-14T16:36:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.511+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7bc0a1350 0x7fd7bc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.512+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/93665880 wait complete. 2023-07-14T16:36:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.512+0000 7fd7ed7f4b80 1 -- start start 2023-07-14T16:36:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.513+0000 7fd7ed7f4b80 1 Processor -- start 2023-07-14T16:36:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.514+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a3fe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.514+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3fee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.514+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a405e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.514+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a2e6a60 con 0x55915a3f5840 2023-07-14T16:36:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.514+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a2e6760 con 0x55915a2516b0 2023-07-14T16:36:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.514+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a1ca8c0 con 0x55915a3f66e0 2023-07-14T16:36:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.514+0000 7fd7ed7f4b80 10 client.0 _ll_register_callbacks cb 0x55915a2e4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:36:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3fee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34980/0 (socket says 172.21.15.62:34980) 2023-07-14T16:36:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/3989180246 learned_addr learned my addr 172.21.15.62:0/3989180246 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7bc0a1350 0x7fd7bc0a3800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55915a3f5060 tx=0x7fd7c80047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a405e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/3989180246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 msgr2=0x55915a405e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a405e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/3989180246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 msgr2=0x55915a3fe8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T16:36:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a3fe8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.515+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/3989180246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c40097e0 con 0x55915a3f5840 2023-07-14T16:36:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.516+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3fee00 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800e630 tx=0x7fd7c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8fd26da64335d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.516+0000 7fd7b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:36:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.516+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/3989180246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8026630 con 0x55915a3f5840 2023-07-14T16:36:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.517+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/3989180246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c801d410 con 0x55915a3f5840 2023-07-14T16:36:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.517+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/3989180246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55915a2a0490 con 0x55915a3f5840 2023-07-14T16:36:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.517+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/3989180246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c800ee80 con 0x55915a3f5840 2023-07-14T16:36:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.517+0000 7fd7ed7f4b80 10 client.20841 fetch_fsmap learned FSMap version 40 2023-07-14T16:36:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.517+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/3989180246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55915a15dc00 con 0x55915a3f5840 2023-07-14T16:36:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.518+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/3989180246 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55915a2a0490 con 0x55915a3f5840 2023-07-14T16:36:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.518+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/3989180246 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7c801d5d0 con 0x55915a3f5840 2023-07-14T16:36:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.518+0000 7fd7ed7f4b80 10 client.20841 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T16:36:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.518+0000 7fd7ed7f4b80 10 client.20841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:36:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.518+0000 7fd7ed7f4b80 20 client.20841 populate_metadata read hostname 'smithi062' 2023-07-14T16:36:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.518+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55915a40a7d0 0x55915a40cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.518+0000 7fd7ed7f4b80 4 client.20841 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:36:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.518+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/3989180246 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55915a33fe50 con 0x55915a40a7d0 2023-07-14T16:36:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.519+0000 7fd7d37fe700 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55915a40a7d0 0x55915a40cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.520+0000 7fd7d37fe700 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55915a40a7d0 0x55915a40cbe0 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb085cd5f7f9774 server_cookie=dc30ce101f91749f in_seq=0 out_seq=0 2023-07-14T16:36:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.520+0000 7fd7b2ffd700 10 client.20841 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/3989180246 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55915a33fe50 con 0x55915a40a7d0 2023-07-14T16:36:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7b2ffd700 10 client.20841 handle_command_reply: tid=0 2023-07-14T16:36:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 client.20841 shutdown 2023-07-14T16:36:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/3989180246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 msgr2=0x55915a3fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3fee00 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800e630 tx=0x7fd7c8004750 comp rx=0 tx=0).stop 2023-07-14T16:36:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/3989180246 shutdown_connections 2023-07-14T16:36:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a405e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55915a40a7d0 0x55915a40cbe0 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a3fe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/3989180246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a3fee00 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/3989180246 >> 172.21.15.62:0/3989180246 conn(0x55915a3eabe0 msgr2=0x55915a3ef4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.521+0000 7fd7b2ffd700 0 client.20841 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.522+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/3989180246 shutdown_connections 2023-07-14T16:36:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.522+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/3989180246 wait complete. 2023-07-14T16:36:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.522+0000 7fd7ed7f4b80 20 client.20841 trim_cache size 0 max 16384 2023-07-14T16:36:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.553+0000 7fd7ed7f4b80 1 Processor -- start 2023-07-14T16:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.554+0000 7fd7ed7f4b80 1 -- start start 2023-07-14T16:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.554+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a2516b0 0x55915a411520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.554+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a3f5840 0x55915a405e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.554+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f66e0 0x55915a3fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.554+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a3fe490 con 0x55915a3f66e0 2023-07-14T16:36:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.554+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a3f5c10 con 0x55915a3f5840 2023-07-14T16:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.554+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a3ecd30 con 0x55915a2516b0 2023-07-14T16:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.555+0000 7fd7d37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a2516b0 0x55915a411520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.555+0000 7fd7d37fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a2516b0 0x55915a411520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:46492/0 (socket says 172.21.15.62:46492) 2023-07-14T16:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.555+0000 7fd7d37fe700 1 -- 172.21.15.62:0/1293463562 learned_addr learned my addr 172.21.15.62:0/1293463562 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.555+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a3f5840 0x55915a405e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.555+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f66e0 0x55915a3fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.556+0000 7fd7d3fff700 1 -- 172.21.15.62:0/1293463562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a2516b0 msgr2=0x55915a411520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.556+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a2516b0 0x55915a411520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.556+0000 7fd7d3fff700 1 -- 172.21.15.62:0/1293463562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a3f5840 msgr2=0x55915a405e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.556+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a3f5840 0x55915a405e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.556+0000 7fd7d3fff700 1 -- 172.21.15.62:0/1293463562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55915a19b1f0 con 0x55915a3f66e0 2023-07-14T16:36:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.557+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f66e0 0x55915a3fd590 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc018960 tx=0x7fd7cc0dbfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be83f18f16c31298 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.557+0000 7fd7ba7fc700 1 -- 172.21.15.62:0/1293463562 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc005a90 con 0x55915a3f66e0 2023-07-14T16:36:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.557+0000 7fd7ba7fc700 1 -- 172.21.15.62:0/1293463562 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7cc02b3b0 con 0x55915a3f66e0 2023-07-14T16:36:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.557+0000 7fd7ba7fc700 1 -- 172.21.15.62:0/1293463562 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b3ef0 con 0x55915a3f66e0 2023-07-14T16:36:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.558+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1293463562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f66e0 msgr2=0x55915a3fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.559+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f66e0 0x55915a3fd590 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc018960 tx=0x7fd7cc0dbfd0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.559+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1293463562 shutdown_connections 2023-07-14T16:36:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.559+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f66e0 0x55915a3fd590 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.559+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a2516b0 0x55915a411520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.559+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1293463562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a3f5840 0x55915a405e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.559+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1293463562 >> 172.21.15.62:0/1293463562 conn(0x55915a3eabe0 msgr2=0x55915a3efc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.560+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1293463562 shutdown_connections 2023-07-14T16:36:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.561+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1293463562 wait complete. 2023-07-14T16:36:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.561+0000 7fd7ed7f4b80 1 -- start start 2023-07-14T16:36:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.562+0000 7fd7ed7f4b80 1 Processor -- start 2023-07-14T16:36:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.562+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a403d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.562+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a45a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.562+0000 7fd7ed7f4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a45cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.562+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a45d3c0 con 0x55915a3f5840 2023-07-14T16:36:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.562+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a45d530 con 0x55915a2516b0 2023-07-14T16:36:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.562+0000 7fd7ed7f4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55915a45d6f0 con 0x55915a3f66e0 2023-07-14T16:36:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.562+0000 7fd7ed7f4b80 10 client.0 _ll_register_callbacks cb 0x7fd7c8011e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:36:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.563+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a45a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.563+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a45a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34992/0 (socket says 172.21.15.62:34992) 2023-07-14T16:36:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.563+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1631238151 learned_addr learned my addr 172.21.15.62:0/1631238151 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:36:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.564+0000 7fd7d3fff700 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a45cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.564+0000 7fd7d37fe700 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a403d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.564+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1631238151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 msgr2=0x55915a45cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.564+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a45cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.564+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1631238151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 msgr2=0x55915a403d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:36:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.564+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a403d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.564+0000 7fd7d2ffd700 1 -- 172.21.15.62:0/1631238151 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d4069f30 con 0x55915a3f5840 2023-07-14T16:36:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.565+0000 7fd7d2ffd700 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a45a640 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8005a80 tx=0x7fd7c800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228d83e36765ef5b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:36:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.565+0000 7fd7b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2023-07-14T16:36:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.566+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/1631238151 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8004240 con 0x55915a3f5840 2023-07-14T16:36:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.566+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/1631238151 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd7c80043a0 con 0x55915a3f5840 2023-07-14T16:36:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.566+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1631238151 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7c4013470 con 0x55915a3f5840 2023-07-14T16:36:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.567+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/1631238151 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c8004560 con 0x55915a3f5840 2023-07-14T16:36:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.567+0000 7fd7ed7f4b80 10 client.20847 fetch_fsmap learned FSMap version 40 2023-07-14T16:36:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.567+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1631238151 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55915a173250 con 0x55915a3f5840 2023-07-14T16:36:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.572+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/1631238151 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7c803e030 con 0x55915a3f5840 2023-07-14T16:36:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.572+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/1631238151 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7c8035020 con 0x55915a3f5840 2023-07-14T16:36:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.572+0000 7fd7ed7f4b80 10 client.20847 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T16:36:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.572+0000 7fd7ed7f4b80 10 client.20847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:36:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.572+0000 7fd7ed7f4b80 20 client.20847 populate_metadata read hostname 'smithi062' 2023-07-14T16:36:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.572+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55915a45f0a0 0x55915a4614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:36:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.572+0000 7fd7ed7f4b80 4 client.20847 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:36:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.572+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1631238151 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55915a301ff0 con 0x55915a45f0a0 2023-07-14T16:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.573+0000 7fd7d37fe700 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55915a45f0a0 0x55915a4614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.573+0000 7fd7d37fe700 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55915a45f0a0 0x55915a4614b0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41fb3f98a42cb5dc server_cookie=779433e4e766f219 in_seq=0 out_seq=0 2023-07-14T16:36:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.574+0000 7fd7b2ffd700 10 client.20847 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7b2ffd700 1 -- 172.21.15.62:0/1631238151 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55915a301ff0 con 0x55915a45f0a0 2023-07-14T16:36:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7b2ffd700 10 client.20847 handle_command_reply: tid=0 2023-07-14T16:36:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 client.20847 shutdown 2023-07-14T16:36:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1631238151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 msgr2=0x55915a45a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a45a640 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8005a80 tx=0x7fd7c800ef10 comp rx=0 tx=0).stop 2023-07-14T16:36:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1631238151 shutdown_connections 2023-07-14T16:36:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55915a3f66e0 0x55915a45cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x55915a45f0a0 0x55915a4614b0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55915a2516b0 0x55915a403d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 --2- 172.21.15.62:0/1631238151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55915a3f5840 0x55915a45a640 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.600+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1631238151 >> 172.21.15.62:0/1631238151 conn(0x55915a3eabe0 msgr2=0x55915a3fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.601+0000 7fd7b2ffd700 0 client.20847 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:36:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.601+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1631238151 shutdown_connections 2023-07-14T16:36:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.601+0000 7fd7ed7f4b80 1 -- 172.21.15.62:0/1631238151 wait complete. 2023-07-14T16:36:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.602+0000 7fd7ed7f4b80 20 client.20847 trim_cache size 0 max 16384 2023-07-14T16:36:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:36:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:36:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:36:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:36:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 -- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7bc0a1350 msgr2=0x7fd7bc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7bc0a1350 0x7fd7bc0a3800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55915a3f5060 tx=0x7fd7c80047c0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 -- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7d410c0d0 msgr2=0x7fd7d4194a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:36:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7d410c0d0 0x7fd7d4194a80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc026000 tx=0x7fd7cc01dd20 comp rx=0 tx=0).stop 2023-07-14T16:36:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 -- 172.21.15.62:0/443018811 shutdown_connections 2023-07-14T16:36:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7d410c0d0 0x7fd7d4194a80 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fd7bc0a1350 0x7fd7bc0a3800 secure :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55915a3f5060 tx=0x7fd7c80047c0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7d4072f70 0x7fd7d4193fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 --2- 172.21.15.62:0/443018811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7d4074550 0x7fd7d4194520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:36:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.603+0000 7fd7d9dff700 1 -- 172.21.15.62:0/443018811 >> 172.21.15.62:0/443018811 conn(0x7fd7d40fb0e0 msgr2=0x7fd7d40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:36:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.612+0000 7fd7d9dff700 1 -- 172.21.15.62:0/443018811 shutdown_connections 2023-07-14T16:36:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:36:53.612+0000 7fd7d9dff700 1 -- 172.21.15.62:0/443018811 wait complete. 2023-07-14T16:36:55.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:55 smithi164 ceph-mon[97427]: pgmap v995: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:36:55.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:55 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:55 smithi062 ceph-mon[95543]: pgmap v995: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:36:55.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:55 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:55 smithi081 ceph-mon[127795]: pgmap v995: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.39k op/s 2023-07-14T16:36:55.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:55.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:55 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:56.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:56.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:56 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:56.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:56 smithi062 ceph-mon[95543]: pgmap v996: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.66k op/s 2023-07-14T16:36:56.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:56.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:56 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:56.895 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:56 smithi081 ceph-mon[127795]: pgmap v996: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.66k op/s 2023-07-14T16:36:57.019 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:57.019 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:56 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:36:57.019 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:56 smithi164 ceph-mon[97427]: pgmap v996: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.66k op/s 2023-07-14T16:36:59.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:36:58 smithi164 ceph-mon[97427]: pgmap v997: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2023-07-14T16:36:59.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:36:58 smithi062 ceph-mon[95543]: pgmap v997: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2023-07-14T16:36:59.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:36:58 smithi081 ceph-mon[127795]: pgmap v997: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2023-07-14T16:37:01.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:01 smithi164 ceph-mon[97427]: pgmap v998: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2023-07-14T16:37:01.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:01.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:01.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:37:01.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:37:01.270 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:01 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:01.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:01 smithi062 ceph-mon[95543]: pgmap v998: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2023-07-14T16:37:01.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:01.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:01.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:37:01.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:37:01.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:01 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:01.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:01 smithi081 ceph-mon[127795]: pgmap v998: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 4.05k op/s 2023-07-14T16:37:01.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:01.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:01.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:37:01.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:37:01.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:01 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:37:03.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:03 smithi062 ceph-mon[95543]: pgmap v999: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2023-07-14T16:37:03.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:03 smithi081 ceph-mon[127795]: pgmap v999: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2023-07-14T16:37:03.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:03 smithi164 ceph-mon[97427]: pgmap v999: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.57k op/s 2023-07-14T16:37:05.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:05 smithi062 ceph-mon[95543]: pgmap v1000: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-07-14T16:37:05.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:05 smithi081 ceph-mon[127795]: pgmap v1000: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-07-14T16:37:05.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:05 smithi164 ceph-mon[97427]: pgmap v1000: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.22k op/s 2023-07-14T16:37:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 -- 172.21.15.62:0/1849893458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f607c102f40 msgr2=0x7f607c105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 --2- 172.21.15.62:0/1849893458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f607c102f40 0x7f607c105320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f606c0098f0 tx=0x7f606c009c00 comp rx=0 tx=0).stop 2023-07-14T16:37:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 -- 172.21.15.62:0/1849893458 shutdown_connections 2023-07-14T16:37:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 --2- 172.21.15.62:0/1849893458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c105860 0x7f607c107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 --2- 172.21.15.62:0/1849893458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f607c100620 0x7f607c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 --2- 172.21.15.62:0/1849893458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f607c102f40 0x7f607c105320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 -- 172.21.15.62:0/1849893458 >> 172.21.15.62:0/1849893458 conn(0x7f607c0fa000 msgr2=0x7f607c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 -- 172.21.15.62:0/1849893458 shutdown_connections 2023-07-14T16:37:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.789+0000 7f6083bff700 1 -- 172.21.15.62:0/1849893458 wait complete. 2023-07-14T16:37:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.790+0000 7f6083bff700 1 Processor -- start 2023-07-14T16:37:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.790+0000 7f6083bff700 1 -- start start 2023-07-14T16:37:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f6083bff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c100620 0x7f607c0fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f6083bff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f607c102f40 0x7f607c0feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f6083bff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f607c105860 0x7f607c0ff050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f6083bff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c19f8b0 con 0x7f607c100620 2023-07-14T16:37:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f6083bff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c19f9f0 con 0x7f607c102f40 2023-07-14T16:37:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f6083bff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c19fb30 con 0x7f607c105860 2023-07-14T16:37:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c100620 0x7f607c0fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c100620 0x7f607c0fe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53996/0 (socket says 172.21.15.62:53996) 2023-07-14T16:37:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608199b700 1 -- 172.21.15.62:0/1771549968 learned_addr learned my addr 172.21.15.62:0/1771549968 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608219c700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f607c105860 0x7f607c0ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608119a700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f607c102f40 0x7f607c0feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608199b700 1 -- 172.21.15.62:0/1771549968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f607c105860 msgr2=0x7f607c0ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608199b700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f607c105860 0x7f607c0ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608199b700 1 -- 172.21.15.62:0/1771549968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f607c102f40 msgr2=0x7f607c0feb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608199b700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f607c102f40 0x7f607c0feb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.791+0000 7f608199b700 1 -- 172.21.15.62:0/1771549968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c009520 con 0x7f607c100620 2023-07-14T16:37:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.792+0000 7f608199b700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c100620 0x7f607c0fe5d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f607800e9f0 tx=0x7f607800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.792+0000 7f6072ffd700 1 -- 172.21.15.62:0/1771549968 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078009690 con 0x7f607c100620 2023-07-14T16:37:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.792+0000 7f6083bff700 1 -- 172.21.15.62:0/1771549968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c19fdc0 con 0x7f607c100620 2023-07-14T16:37:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.792+0000 7f6072ffd700 1 -- 172.21.15.62:0/1771549968 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f607800ce50 con 0x7f607c100620 2023-07-14T16:37:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.792+0000 7f6072ffd700 1 -- 172.21.15.62:0/1771549968 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078018420 con 0x7f607c100620 2023-07-14T16:37:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.792+0000 7f6083bff700 1 -- 172.21.15.62:0/1771549968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c1a0190 con 0x7f607c100620 2023-07-14T16:37:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.797+0000 7f6072ffd700 1 -- 172.21.15.62:0/1771549968 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f607800c9c0 con 0x7f607c100620 2023-07-14T16:37:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.798+0000 7f60975eeb80 1 Processor -- start 2023-07-14T16:37:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.798+0000 7f60975eeb80 1 -- start start 2023-07-14T16:37:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f6072ffd700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f60680a1150 0x7f60680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe27b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f6072ffd700 1 -- 172.21.15.62:0/1771549968 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6078014070 con 0x7f607c100620 2023-07-14T16:37:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f60975eeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbd2bc50 con 0x5559bbc826b0 2023-07-14T16:37:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f60975eeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbd7e9e0 con 0x5559bbe26840 2023-07-14T16:37:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f60975eeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbd77e70 con 0x5559bbe276e0 2023-07-14T16:37:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f608119a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f608119a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe271a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:38760/0 (socket says 172.21.15.62:38760) 2023-07-14T16:37:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f608119a700 1 -- 172.21.15.62:0/1523599788 learned_addr learned my addr 172.21.15.62:0/1523599788 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f608219c700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f60680a1150 0x7f60680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f608199b700 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.799+0000 7f608219c700 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe27b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608119a700 1 -- 172.21.15.62:0/1523599788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 msgr2=0x5559bbe27b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608119a700 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe27b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608119a700 1 -- 172.21.15.62:0/1523599788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 msgr2=0x5559bbe26300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608119a700 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe26300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608119a700 1 -- 172.21.15.62:0/1523599788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559bbba5d80 con 0x5559bbe26840 2023-07-14T16:37:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608219c700 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe27b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:37:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608199b700 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:37:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608119a700 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe271a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f606c0098c0 tx=0x7f606c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81a15a0309e7dfe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f608219c700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f60680a1150 0x7f60680a3600 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6074009eb0 tx=0x7f6074009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f6067fff700 1 -- 172.21.15.62:0/1523599788 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00f460 con 0x5559bbe26840 2023-07-14T16:37:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f6067fff700 1 -- 172.21.15.62:0/1523599788 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f606c01d040 con 0x5559bbe26840 2023-07-14T16:37:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.800+0000 7f6067fff700 1 -- 172.21.15.62:0/1523599788 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0052a0 con 0x5559bbe26840 2023-07-14T16:37:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 -- 172.21.15.62:0/1523599788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 msgr2=0x5559bbe271a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe271a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f606c0098c0 tx=0x7f606c005dc0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 -- 172.21.15.62:0/1523599788 shutdown_connections 2023-07-14T16:37:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe27b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 --2- 172.21.15.62:0/1523599788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe271a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 -- 172.21.15.62:0/1523599788 >> 172.21.15.62:0/1523599788 conn(0x5559bbe1bbe0 msgr2=0x5559bbe20cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 -- 172.21.15.62:0/1523599788 shutdown_connections 2023-07-14T16:37:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.801+0000 7f60975eeb80 1 -- 172.21.15.62:0/1523599788 wait complete. 2023-07-14T16:37:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.802+0000 7f60975eeb80 1 -- start start 2023-07-14T16:37:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.802+0000 7f60975eeb80 1 Processor -- start 2023-07-14T16:37:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe2f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe36d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f60975eeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbd17a60 con 0x5559bbc826b0 2023-07-14T16:37:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f60975eeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbd17760 con 0x5559bbe26840 2023-07-14T16:37:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f60975eeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbbfb8c0 con 0x5559bbe276e0 2023-07-14T16:37:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f60975eeb80 10 client.0 _ll_register_callbacks cb 0x5559bbd15700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:37:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f608119a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f608199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe2f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f608119a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe2fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:38776/0 (socket says 172.21.15.62:38776) 2023-07-14T16:37:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f608119a700 1 -- 172.21.15.62:0/3818061152 learned_addr learned my addr 172.21.15.62:0/3818061152 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.803+0000 7f608219c700 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe36d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f608119a700 1 -- 172.21.15.62:0/3818061152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 msgr2=0x5559bbe36d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f608119a700 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe36d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f608119a700 1 -- 172.21.15.62:0/3818061152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 msgr2=0x5559bbe2f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f608119a700 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe2f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f608119a700 1 -- 172.21.15.62:0/3818061152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c108860 con 0x5559bbe26840 2023-07-14T16:37:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f608119a700 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe2fd10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f606c005e90 tx=0x7f606c00bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=847dd6de3156099e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f60657fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:37:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f60657fa700 1 -- 172.21.15.62:0/3818061152 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00f800 con 0x5559bbe26840 2023-07-14T16:37:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f60975eeb80 1 -- 172.21.15.62:0/3818061152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559bbcd1490 con 0x5559bbe26840 2023-07-14T16:37:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.804+0000 7f60657fa700 1 -- 172.21.15.62:0/3818061152 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f606c005450 con 0x5559bbe26840 2023-07-14T16:37:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60657fa700 1 -- 172.21.15.62:0/3818061152 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606c005610 con 0x5559bbe26840 2023-07-14T16:37:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60975eeb80 10 client.30425 fetch_fsmap learned FSMap version 40 2023-07-14T16:37:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60975eeb80 1 -- 172.21.15.62:0/3818061152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5559bbb8ec00 con 0x5559bbe26840 2023-07-14T16:37:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60657fa700 1 -- 172.21.15.62:0/3818061152 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f606c03a020 con 0x5559bbe26840 2023-07-14T16:37:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60657fa700 1 -- 172.21.15.62:0/3818061152 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f606c015020 con 0x5559bbe26840 2023-07-14T16:37:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60975eeb80 10 client.30425 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T16:37:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60975eeb80 10 client.30425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:37:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60975eeb80 20 client.30425 populate_metadata read hostname 'smithi062' 2023-07-14T16:37:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5559bbe3b050 0x5559bbe3d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60975eeb80 4 client.30425 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:37:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.805+0000 7f60975eeb80 1 -- 172.21.15.62:0/3818061152 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559bbd70e50 con 0x5559bbe3b050 2023-07-14T16:37:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.806+0000 7f608199b700 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5559bbe3b050 0x5559bbe3d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.806+0000 7f608199b700 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5559bbe3b050 0x5559bbe3d460 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad888d670a0b210 server_cookie=7456c782892f32cd in_seq=0 out_seq=0 2023-07-14T16:37:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.806+0000 7f60657fa700 10 client.30425 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:37:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60657fa700 1 -- 172.21.15.62:0/3818061152 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5559bbd70e50 con 0x5559bbe3b050 2023-07-14T16:37:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60657fa700 10 client.30425 handle_command_reply: tid=0 2023-07-14T16:37:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 client.30425 shutdown 2023-07-14T16:37:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 -- 172.21.15.62:0/3818061152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 msgr2=0x5559bbe2fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe2fd10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f606c005e90 tx=0x7f606c00bc30 comp rx=0 tx=0).stop 2023-07-14T16:37:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 -- 172.21.15.62:0/3818061152 shutdown_connections 2023-07-14T16:37:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe36d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe2f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5559bbe3b050 0x5559bbe3d460 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3818061152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe2fd10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 -- 172.21.15.62:0/3818061152 >> 172.21.15.62:0/3818061152 conn(0x5559bbe1bbe0 msgr2=0x5559bbe204e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60657fa700 0 client.30425 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:37:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 -- 172.21.15.62:0/3818061152 shutdown_connections 2023-07-14T16:37:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.807+0000 7f60975eeb80 1 -- 172.21.15.62:0/3818061152 wait complete. 2023-07-14T16:37:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.808+0000 7f60975eeb80 20 client.30425 trim_cache size 0 max 16384 2023-07-14T16:37:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.824+0000 7f60975eeb80 1 Processor -- start 2023-07-14T16:37:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.824+0000 7f60975eeb80 1 -- start start 2023-07-14T16:37:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe27ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe36d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe2a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f60975eeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbe1dd00 con 0x5559bbc826b0 2023-07-14T16:37:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f60975eeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbe22120 con 0x5559bbe26840 2023-07-14T16:37:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f60975eeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbe27ff0 con 0x5559bbe276e0 2023-07-14T16:37:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe27ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe27ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54022/0 (socket says 172.21.15.62:54022) 2023-07-14T16:37:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608199b700 1 -- 172.21.15.62:0/3496237555 learned_addr learned my addr 172.21.15.62:0/3496237555 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608219c700 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe2a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608119a700 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe36d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608199b700 1 -- 172.21.15.62:0/3496237555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 msgr2=0x5559bbe2a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608199b700 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe2a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608199b700 1 -- 172.21.15.62:0/3496237555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 msgr2=0x5559bbe36d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608199b700 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe36d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.825+0000 7f608199b700 1 -- 172.21.15.62:0/3496237555 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559bbbcc1f0 con 0x5559bbc826b0 2023-07-14T16:37:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.826+0000 7f608199b700 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe27ab0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6078096260 tx=0x7f6078099f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b57473430d25eb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.826+0000 7f6070bff700 1 -- 172.21.15.62:0/3496237555 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60780a2200 con 0x5559bbc826b0 2023-07-14T16:37:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.826+0000 7f6070bff700 1 -- 172.21.15.62:0/3496237555 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6078010430 con 0x5559bbc826b0 2023-07-14T16:37:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.826+0000 7f6070bff700 1 -- 172.21.15.62:0/3496237555 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078098240 con 0x5559bbc826b0 2023-07-14T16:37:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 -- 172.21.15.62:0/3496237555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 msgr2=0x5559bbe27ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe27ab0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6078096260 tx=0x7f6078099f30 comp rx=0 tx=0).stop 2023-07-14T16:37:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 -- 172.21.15.62:0/3496237555 shutdown_connections 2023-07-14T16:37:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe2a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe27ab0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3496237555 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe36d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 -- 172.21.15.62:0/3496237555 >> 172.21.15.62:0/3496237555 conn(0x5559bbe1bbe0 msgr2=0x5559bbe41da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 -- 172.21.15.62:0/3496237555 shutdown_connections 2023-07-14T16:37:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 -- 172.21.15.62:0/3496237555 wait complete. 2023-07-14T16:37:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.827+0000 7f60975eeb80 1 -- start start 2023-07-14T16:37:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f60975eeb80 1 Processor -- start 2023-07-14T16:37:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe32c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe33150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f60975eeb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe36170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f60975eeb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbe366b0 con 0x5559bbc826b0 2023-07-14T16:37:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f60975eeb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbe36820 con 0x5559bbe26840 2023-07-14T16:37:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f60975eeb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559bbe369e0 con 0x5559bbe276e0 2023-07-14T16:37:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f60975eeb80 10 client.0 _ll_register_callbacks cb 0x7f606c009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:37:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.828+0000 7f608199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe32c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608119a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe33150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608199b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe32c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54032/0 (socket says 172.21.15.62:54032) 2023-07-14T16:37:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608199b700 1 -- 172.21.15.62:0/3565031128 learned_addr learned my addr 172.21.15.62:0/3565031128 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608219c700 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe36170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608119a700 1 -- 172.21.15.62:0/3565031128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 msgr2=0x5559bbe36170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608119a700 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe36170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608119a700 1 -- 172.21.15.62:0/3565031128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 msgr2=0x5559bbe32c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608119a700 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe32c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608119a700 1 -- 172.21.15.62:0/3565031128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c1a0190 con 0x5559bbe26840 2023-07-14T16:37:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f608119a700 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe33150 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f606c017a90 tx=0x7f606c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35fe09b91a693528 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f60657fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-07-14T16:37:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.829+0000 7f60657fa700 1 -- 172.21.15.62:0/3565031128 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c005920 con 0x5559bbe26840 2023-07-14T16:37:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60657fa700 1 -- 172.21.15.62:0/3565031128 <== mon.2 v2:172.21.15.81:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f606c000e40 con 0x5559bbe26840 2023-07-14T16:37:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 1 -- 172.21.15.62:0/3565031128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f606c01d040 con 0x5559bbe26840 2023-07-14T16:37:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60657fa700 1 -- 172.21.15.62:0/3565031128 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606c036910 con 0x5559bbe26840 2023-07-14T16:37:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 10 client.30431 fetch_fsmap learned FSMap version 40 2023-07-14T16:37:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 1 -- 172.21.15.62:0/3565031128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5559bbba4250 con 0x5559bbe26840 2023-07-14T16:37:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60657fa700 1 -- 172.21.15.62:0/3565031128 <== mon.2 v2:172.21.15.81:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f606c01d040 con 0x5559bbe26840 2023-07-14T16:37:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60657fa700 1 -- 172.21.15.62:0/3565031128 <== mon.2 v2:172.21.15.81:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f606c009920 con 0x5559bbe26840 2023-07-14T16:37:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 10 client.30431 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T16:37:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 10 client.30431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:37:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 20 client.30431 populate_metadata read hostname 'smithi062' 2023-07-14T16:37:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5559bbe8fb50 0x5559bbe91f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 4 client.30431 mds_command: new command op to 24602 tid=0[{"prefix": "flush journal"}] 2023-07-14T16:37:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.830+0000 7f60975eeb80 1 -- 172.21.15.62:0/3565031128 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559bbd32ff0 con 0x5559bbe8fb50 2023-07-14T16:37:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.831+0000 7f608199b700 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5559bbe8fb50 0x5559bbe91f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.831+0000 7f60657fa700 10 client.30431 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:37:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.831+0000 7f608199b700 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5559bbe8fb50 0x5559bbe91f60 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d9dfaf8674ce90 server_cookie=4397828a1199a5d2 in_seq=0 out_seq=0 2023-07-14T16:37:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.846+0000 7f60657fa700 1 -- 172.21.15.62:0/3565031128 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559bbd32ff0 con 0x5559bbe8fb50 2023-07-14T16:37:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.846+0000 7f60657fa700 10 client.30431 handle_command_reply: tid=0 2023-07-14T16:37:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.846+0000 7f60975eeb80 1 client.30431 shutdown 2023-07-14T16:37:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.846+0000 7f60975eeb80 1 -- 172.21.15.62:0/3565031128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 msgr2=0x5559bbe33150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.846+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe33150 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f606c017a90 tx=0x7f606c000bc0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 1 -- 172.21.15.62:0/3565031128 shutdown_connections 2023-07-14T16:37:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559bbe276e0 0x5559bbe36170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5559bbc826b0 0x5559bbe32c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x5559bbe8fb50 0x5559bbe91f60 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 1 --2- 172.21.15.62:0/3565031128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5559bbe26840 0x5559bbe33150 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60657fa700 0 client.30431 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:37:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 1 -- 172.21.15.62:0/3565031128 >> 172.21.15.62:0/3565031128 conn(0x5559bbe1bbe0 msgr2=0x5559bbe2a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 1 -- 172.21.15.62:0/3565031128 shutdown_connections 2023-07-14T16:37:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 1 -- 172.21.15.62:0/3565031128 wait complete. 2023-07-14T16:37:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.847+0000 7f60975eeb80 20 client.30431 trim_cache size 0 max 16384 2023-07-14T16:37:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:{ 2023-07-14T16:37:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "message": "", 2023-07-14T16:37:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout: "return_code": 0 2023-07-14T16:37:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stdout:} 2023-07-14T16:37:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 -- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f60680a1150 msgr2=0x7f60680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f60680a1150 0x7f60680a3600 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6074009eb0 tx=0x7f6074009920 comp rx=0 tx=0).stop 2023-07-14T16:37:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 -- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c100620 msgr2=0x7f607c0fe5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c100620 0x7f607c0fe5d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f607800e9f0 tx=0x7f607800ed00 comp rx=0 tx=0).stop 2023-07-14T16:37:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 -- 172.21.15.62:0/1771549968 shutdown_connections 2023-07-14T16:37:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f607c105860 0x7f607c0ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c100620 0x7f607c0fe5d0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f60680a1150 0x7f60680a3600 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 --2- 172.21.15.62:0/1771549968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f607c102f40 0x7f607c0feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 -- 172.21.15.62:0/1771549968 >> 172.21.15.62:0/1771549968 conn(0x7f607c0fa000 msgr2=0x7f607c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 -- 172.21.15.62:0/1771549968 shutdown_connections 2023-07-14T16:37:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:06.848+0000 7f6083bff700 1 -- 172.21.15.62:0/1771549968 wait complete. 2023-07-14T16:37:07.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:07 smithi062 ceph-mon[95543]: pgmap v1001: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2023-07-14T16:37:07.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:07 smithi081 ceph-mon[127795]: pgmap v1001: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2023-07-14T16:37:07.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:07 smithi164 ceph-mon[97427]: pgmap v1001: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2023-07-14T16:37:09.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:09 smithi062 ceph-mon[95543]: pgmap v1002: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:37:09.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:09 smithi081 ceph-mon[127795]: pgmap v1002: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:37:09.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:09 smithi164 ceph-mon[97427]: pgmap v1002: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 4.00k op/s 2023-07-14T16:37:11.358 INFO:tasks.workunit.client.0.smithi062.stdout:Syncing()...1 sec 2023-07-14T16:37:11.358 INFO:tasks.workunit.client.0.smithi062.stdout:FFSB benchmark finished at: Fri Jul 14 16:37:11 2023 2023-07-14T16:37:11.358 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.358 INFO:tasks.workunit.client.0.smithi062.stdout:Results: 2023-07-14T16:37:11.358 INFO:tasks.workunit.client.0.smithi062.stdout:Benchmark took 310.49 sec 2023-07-14T16:37:11.359 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.359 INFO:tasks.workunit.client.0.smithi062.stdout:Total Results 2023-07-14T16:37:11.359 INFO:tasks.workunit.client.0.smithi062.stdout:=============== 2023-07-14T16:37:11.359 INFO:tasks.workunit.client.0.smithi062.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2023-07-14T16:37:11.359 INFO:tasks.workunit.client.0.smithi062.stdout: ======= ============ ========= ======= =========== ========== 2023-07-14T16:37:11.359 INFO:tasks.workunit.client.0.smithi062.stdout: write : 1377280 4435.76 100.000% 100.000% 17.3MB/sec 2023-07-14T16:37:11.359 INFO:tasks.workunit.client.0.smithi062.stdout:- 2023-07-14T16:37:11.360 INFO:tasks.workunit.client.0.smithi062.stdout:4435.76 Transactions per Second 2023-07-14T16:37:11.360 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.360 INFO:tasks.workunit.client.0.smithi062.stdout:Throughput Results 2023-07-14T16:37:11.360 INFO:tasks.workunit.client.0.smithi062.stdout:=================== 2023-07-14T16:37:11.360 INFO:tasks.workunit.client.0.smithi062.stdout:Write Throughput: 17.3MB/sec 2023-07-14T16:37:11.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:11 smithi062 ceph-mon[95543]: pgmap v1003: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2023-07-14T16:37:11.373 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.373 INFO:tasks.workunit.client.0.smithi062.stdout:System Call Latency statistics in millisecs 2023-07-14T16:37:11.373 INFO:tasks.workunit.client.0.smithi062.stdout:===== 2023-07-14T16:37:11.373 INFO:tasks.workunit.client.0.smithi062.stdout: Min Avg Max Total Calls 2023-07-14T16:37:11.373 INFO:tasks.workunit.client.0.smithi062.stdout: ======== ======== ======== ============ 2023-07-14T16:37:11.374 INFO:tasks.workunit.client.0.smithi062.stdout:[ open] 1.070000 24.159508 940.356995 1076 2023-07-14T16:37:11.374 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2023-07-14T16:37:11.374 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2023-07-14T16:37:11.374 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2023-07-14T16:37:11.374 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[3] 500.000000 - 1000.000000 : 1 2023-07-14T16:37:11.374 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2023-07-14T16:37:11.374 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[5] 100.000000 - 200.000000 : 4 2023-07-14T16:37:11.374 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[6] 50.000000 - 100.000000 : 36 2023-07-14T16:37:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[7] 20.000000 - 50.000000 : 545 2023-07-14T16:37:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[8] 10.000000 - 20.000000 : 379 2023-07-14T16:37:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[9] 5.000000 - 10.000000 : 74 2023-07-14T16:37:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[10] 2.000000 - 5.000000 : 31 2023-07-14T16:37:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[11] 1.000000 - 2.000000 : 6 2023-07-14T16:37:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[12] 0.500000 - 1.000000 : 0 2023-07-14T16:37:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2023-07-14T16:37:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2023-07-14T16:37:11.376 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2023-07-14T16:37:11.376 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2023-07-14T16:37:11.376 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2023-07-14T16:37:11.376 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2023-07-14T16:37:11.376 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.376 INFO:tasks.workunit.client.0.smithi062.stdout:[ write] 0.184000 7.027285 517.684998 1377280 2023-07-14T16:37:11.376 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2023-07-14T16:37:11.377 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2023-07-14T16:37:11.377 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2023-07-14T16:37:11.377 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[3] 500.000000 - 1000.000000 : 1 2023-07-14T16:37:11.377 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[4] 200.000000 - 500.000000 : 496 2023-07-14T16:37:11.377 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[5] 100.000000 - 200.000000 : 148 2023-07-14T16:37:11.377 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[6] 50.000000 - 100.000000 : 2710 2023-07-14T16:37:11.377 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[7] 20.000000 - 50.000000 : 78779 2023-07-14T16:37:11.377 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[8] 10.000000 - 20.000000 : 211438 2023-07-14T16:37:11.378 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[9] 5.000000 - 10.000000 : 377537 2023-07-14T16:37:11.378 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[10] 2.000000 - 5.000000 : 381861 2023-07-14T16:37:11.378 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[11] 1.000000 - 2.000000 : 74414 2023-07-14T16:37:11.378 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[12] 0.500000 - 1.000000 : 100953 2023-07-14T16:37:11.378 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[13] 0.200000 - 0.500000 : 148842 2023-07-14T16:37:11.378 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[14] 0.100000 - 0.200000 : 101 2023-07-14T16:37:11.378 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2023-07-14T16:37:11.378 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2023-07-14T16:37:11.379 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2023-07-14T16:37:11.379 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2023-07-14T16:37:11.379 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.379 INFO:tasks.workunit.client.0.smithi062.stdout:[ lseek] 0.000000 0.001686 0.761000 1377280 2023-07-14T16:37:11.379 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2023-07-14T16:37:11.379 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2023-07-14T16:37:11.379 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2023-07-14T16:37:11.379 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2023-07-14T16:37:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2023-07-14T16:37:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2023-07-14T16:37:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2023-07-14T16:37:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2023-07-14T16:37:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2023-07-14T16:37:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2023-07-14T16:37:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[10] 2.000000 - 5.000000 : 0 2023-07-14T16:37:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[11] 1.000000 - 2.000000 : 0 2023-07-14T16:37:11.381 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[12] 0.500000 - 1.000000 : 1 2023-07-14T16:37:11.381 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[13] 0.200000 - 0.500000 : 3 2023-07-14T16:37:11.381 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[14] 0.100000 - 0.200000 : 7 2023-07-14T16:37:11.381 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[15] 0.050000 - 0.100000 : 148 2023-07-14T16:37:11.381 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[16] 0.020000 - 0.050000 : 1344 2023-07-14T16:37:11.381 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[17] 0.010000 - 0.020000 : 1924 2023-07-14T16:37:11.381 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[18] 0.000000 - 0.010000 : 1373853 2023-07-14T16:37:11.381 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.382 INFO:tasks.workunit.client.0.smithi062.stdout:[ close] 0.041000 3.503262 88.596001 1076 2023-07-14T16:37:11.382 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2023-07-14T16:37:11.382 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2023-07-14T16:37:11.382 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2023-07-14T16:37:11.382 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2023-07-14T16:37:11.382 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2023-07-14T16:37:11.382 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2023-07-14T16:37:11.382 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[6] 50.000000 - 100.000000 : 2 2023-07-14T16:37:11.383 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[7] 20.000000 - 50.000000 : 40 2023-07-14T16:37:11.383 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[8] 10.000000 - 20.000000 : 64 2023-07-14T16:37:11.383 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[9] 5.000000 - 10.000000 : 95 2023-07-14T16:37:11.383 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[10] 2.000000 - 5.000000 : 220 2023-07-14T16:37:11.383 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[11] 1.000000 - 2.000000 : 109 2023-07-14T16:37:11.383 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[12] 0.500000 - 1.000000 : 26 2023-07-14T16:37:11.383 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[13] 0.200000 - 0.500000 : 99 2023-07-14T16:37:11.383 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[14] 0.100000 - 0.200000 : 148 2023-07-14T16:37:11.384 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[15] 0.050000 - 0.100000 : 262 2023-07-14T16:37:11.384 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[16] 0.020000 - 0.050000 : 11 2023-07-14T16:37:11.384 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2023-07-14T16:37:11.384 INFO:tasks.workunit.client.0.smithi062.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2023-07-14T16:37:11.384 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.384 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.384 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.384 INFO:tasks.workunit.client.0.smithi062.stdout: 2023-07-14T16:37:11.385 INFO:tasks.workunit.client.0.smithi062.stdout:1.0% User Time 2023-07-14T16:37:11.385 INFO:tasks.workunit.client.0.smithi062.stdout:44.9% System Time 2023-07-14T16:37:11.385 INFO:tasks.workunit.client.0.smithi062.stdout:45.9% CPU Utilization 2023-07-14T16:37:11.385 INFO:tasks.workunit.client.0.smithi062.stderr:+ cd .. 2023-07-14T16:37:11.388 INFO:tasks.workunit.client.0.smithi062.stderr:+ rm -r tmp ffsb 2023-07-14T16:37:11.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:11 smithi081 ceph-mon[127795]: pgmap v1003: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2023-07-14T16:37:11.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:11 smithi164 ceph-mon[97427]: pgmap v1003: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.63k op/s 2023-07-14T16:37:13.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:13 smithi062 ceph-mon[95543]: pgmap v1004: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2023-07-14T16:37:13.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:13 smithi081 ceph-mon[127795]: pgmap v1004: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2023-07-14T16:37:13.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:13 smithi164 ceph-mon[97427]: pgmap v1004: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2023-07-14T16:37:14.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:14 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:14.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:14 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:14.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:14 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:15.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:15 smithi062 ceph-mon[95543]: pgmap v1005: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2023-07-14T16:37:15.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:15 smithi081 ceph-mon[127795]: pgmap v1005: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2023-07-14T16:37:15.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:15 smithi164 ceph-mon[97427]: pgmap v1005: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2023-07-14T16:37:17.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:17 smithi062 ceph-mon[95543]: pgmap v1006: 97 pgs: 97 active+clean; 12 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.4 MiB/s wr, 1.61k op/s 2023-07-14T16:37:17.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:17 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:17.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:17 smithi081 ceph-mon[127795]: pgmap v1006: 97 pgs: 97 active+clean; 12 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.4 MiB/s wr, 1.61k op/s 2023-07-14T16:37:17.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:17 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:17.520 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:17 smithi164 ceph-mon[97427]: pgmap v1006: 97 pgs: 97 active+clean; 12 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.4 MiB/s wr, 1.61k op/s 2023-07-14T16:37:17.520 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:17 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:18.487 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-07-14T16:37:18.488 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-07-14T16:37:18.520 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:18 smithi164 ceph-mon[97427]: pgmap v1007: 97 pgs: 97 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.2 MiB/s wr, 1.34k op/s 2023-07-14T16:37:18.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:18 smithi062 ceph-mon[95543]: pgmap v1007: 97 pgs: 97 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.2 MiB/s wr, 1.34k op/s 2023-07-14T16:37:18.645 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:18 smithi081 ceph-mon[127795]: pgmap v1007: 97 pgs: 97 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.2 MiB/s wr, 1.34k op/s 2023-07-14T16:37:18.713 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2023-07-14T16:37:18.714 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-14T16:37:19.896 DEBUG:teuthology.parallel:result is None 2023-07-14T16:37:19.897 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-07-14T16:37:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.085+0000 7f76bcfff700 1 -- 172.21.15.62:0/4075701389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76b8070f00 msgr2=0x7f76b8071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.085+0000 7f76bcfff700 1 --2- 172.21.15.62:0/4075701389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76b8070f00 0x7f76b8071310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009990 tx=0x7f76a0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.086+0000 7f76bcfff700 1 -- 172.21.15.62:0/4075701389 shutdown_connections 2023-07-14T16:37:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.086+0000 7f76bcfff700 1 --2- 172.21.15.62:0/4075701389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b810bd60 0x7f76b810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.086+0000 7f76bcfff700 1 --2- 172.21.15.62:0/4075701389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76b8070f00 0x7f76b8071310 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009990 tx=0x7f76a0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.086+0000 7f76bcfff700 1 --2- 172.21.15.62:0/4075701389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76b8071850 0x7f76b810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.086+0000 7f76bcfff700 1 -- 172.21.15.62:0/4075701389 >> 172.21.15.62:0/4075701389 conn(0x7f76b806ca50 msgr2=0x7f76b806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.086+0000 7f76bcfff700 1 -- 172.21.15.62:0/4075701389 shutdown_connections 2023-07-14T16:37:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.087+0000 7f76bcfff700 1 -- 172.21.15.62:0/4075701389 wait complete. 2023-07-14T16:37:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.089+0000 7f76bcfff700 1 Processor -- start 2023-07-14T16:37:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.089+0000 7f76bcfff700 1 -- start start 2023-07-14T16:37:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.090+0000 7f76bcfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b8071850 0x7f76b81a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.090+0000 7f76bcfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76b810bd60 0x7f76b81a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.090+0000 7f76bcfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76b81a5cf0 0x7f76b81a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.090+0000 7f76bcfff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81a6730 con 0x7f76b8071850 2023-07-14T16:37:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.090+0000 7f76bcfff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81a68a0 con 0x7f76b81a5cf0 2023-07-14T16:37:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.090+0000 7f76bcfff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81a6a60 con 0x7f76b810bd60 2023-07-14T16:37:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.091+0000 7f76b659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b8071850 0x7f76b81a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.091+0000 7f76b659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b8071850 0x7f76b81a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36516/0 (socket says 172.21.15.62:36516) 2023-07-14T16:37:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.091+0000 7f76b659c700 1 -- 172.21.15.62:0/3639724539 learned_addr learned my addr 172.21.15.62:0/3639724539 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.091+0000 7f76b6d9d700 1 --2- 172.21.15.62:0/3639724539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76b81a5cf0 0x7f76b81a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.092+0000 7f76b6d9d700 1 -- 172.21.15.62:0/3639724539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76b810bd60 msgr2=0x7f76b81a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.092+0000 7f76b6d9d700 1 --2- 172.21.15.62:0/3639724539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f76b810bd60 0x7f76b81a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.092+0000 7f76b6d9d700 1 -- 172.21.15.62:0/3639724539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b8071850 msgr2=0x7f76b81a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.092+0000 7f76b6d9d700 1 --2- 172.21.15.62:0/3639724539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76b8071850 0x7f76b81a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.092+0000 7f76b6d9d700 1 -- 172.21.15.62:0/3639724539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00095f0 con 0x7f76b81a5cf0 2023-07-14T16:37:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.092+0000 7f76b6d9d700 1 --2- 172.21.15.62:0/3639724539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76b81a5cf0 0x7f76b81a6160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f76b000cb70 tx=0x7f76b000cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:20.069 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.099+0000 7f76af7fe700 1 -- 172.21.15.62:0/3639724539 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b000a8c0 con 0x7f76b81a5cf0 2023-07-14T16:37:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.099+0000 7f76af7fe700 1 -- 172.21.15.62:0/3639724539 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76b0011400 con 0x7f76b81a5cf0 2023-07-14T16:37:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.099+0000 7f76af7fe700 1 -- 172.21.15.62:0/3639724539 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00185c0 con 0x7f76b81a5cf0 2023-07-14T16:37:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.099+0000 7f76bcfff700 1 -- 172.21.15.62:0/3639724539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b81acfa0 con 0x7f76b81a5cf0 2023-07-14T16:37:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.100+0000 7f76bcfff700 1 -- 172.21.15.62:0/3639724539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b81ad450 con 0x7f76b81a5cf0 2023-07-14T16:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.107+0000 7f76d0a05b80 1 Processor -- start 2023-07-14T16:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.109+0000 7f76d0a05b80 1 -- start start 2023-07-14T16:37:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.109+0000 7f76d0a05b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a3a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.109+0000 7f76d0a05b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3a840 0x556007a3b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.109+0000 7f76d0a05b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3b6e0 0x556007a3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.109+0000 7f76d0a05b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600793fc50 con 0x556007a3b6e0 2023-07-14T16:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.109+0000 7f76d0a05b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560079929e0 con 0x556007a3a840 2023-07-14T16:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.109+0000 7f76d0a05b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600798be70 con 0x5560078966b0 2023-07-14T16:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.110+0000 7f76affff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3b6e0 0x556007a3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.110+0000 7f76affff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3b6e0 0x556007a3bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36532/0 (socket says 172.21.15.62:36532) 2023-07-14T16:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.110+0000 7f76affff700 1 -- 172.21.15.62:0/3434338963 learned_addr learned my addr 172.21.15.62:0/3434338963 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.112+0000 7f76b659c700 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3a840 0x556007a3b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.112+0000 7f76affff700 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a3a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.116+0000 7f76affff700 1 -- 172.21.15.62:0/3434338963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 msgr2=0x556007a3a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.116+0000 7f76affff700 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a3a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.116+0000 7f76affff700 1 -- 172.21.15.62:0/3434338963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3a840 msgr2=0x556007a3b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.116+0000 7f76affff700 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3a840 0x556007a3b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.116+0000 7f76affff700 1 -- 172.21.15.62:0/3434338963 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560077b9d80 con 0x556007a3b6e0 2023-07-14T16:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.117+0000 7f76affff700 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3b6e0 0x556007a3bb80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f76a8034800 tx=0x7f76a800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b540d0a20f81fbf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.118+0000 7f76af7fe700 1 -- 172.21.15.62:0/3639724539 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f76b0011570 con 0x7f76b81a5cf0 2023-07-14T16:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.118+0000 7f76acff9700 1 -- 172.21.15.62:0/3434338963 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004a30 con 0x556007a3b6e0 2023-07-14T16:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.118+0000 7f76acff9700 1 -- 172.21.15.62:0/3434338963 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76a803a070 con 0x556007a3b6e0 2023-07-14T16:37:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.118+0000 7f76acff9700 1 -- 172.21.15.62:0/3434338963 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a802f3d0 con 0x556007a3b6e0 2023-07-14T16:37:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.120+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3434338963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3b6e0 msgr2=0x556007a3bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.120+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3b6e0 0x556007a3bb80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f76a8034800 tx=0x7f76a800af90 comp rx=0 tx=0).stop 2023-07-14T16:37:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.120+0000 7f76af7fe700 1 --2- 172.21.15.62:0/3639724539 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76a40a1350 0x7f76a40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.121+0000 7f76af7fe700 1 -- 172.21.15.62:0/3639724539 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f76b00cc5a0 con 0x7f76b81a5cf0 2023-07-14T16:37:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.121+0000 7f76affff700 1 --2- 172.21.15.62:0/3639724539 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76a40a1350 0x7f76a40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.121+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3434338963 shutdown_connections 2023-07-14T16:37:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.121+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3b6e0 0x556007a3bb80 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.121+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a3a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.121+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3434338963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3a840 0x556007a3b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.121+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3434338963 >> 172.21.15.62:0/3434338963 conn(0x556007a2fbe0 msgr2=0x556007a34cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.122+0000 7f76affff700 1 --2- 172.21.15.62:0/3639724539 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f76a40a1350 0x7f76a40a3800 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f76a8000ea0 tx=0x7f76a8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.124+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3434338963 shutdown_connections 2023-07-14T16:37:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.124+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3434338963 wait complete. 2023-07-14T16:37:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.125+0000 7f76d0a05b80 1 -- start start 2023-07-14T16:37:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.125+0000 7f76d0a05b80 1 Processor -- start 2023-07-14T16:37:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.126+0000 7f76d0a05b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a43880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.126+0000 7f76d0a05b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3a840 0x556007a43dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.126+0000 7f76d0a05b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3b6e0 0x556007a4ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.126+0000 7f76d0a05b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600792ba60 con 0x556007a3a840 2023-07-14T16:37:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.126+0000 7f76d0a05b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600792b760 con 0x556007a3b6e0 2023-07-14T16:37:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.126+0000 7f76d0a05b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600780f8c0 con 0x5560078966b0 2023-07-14T16:37:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.126+0000 7f76d0a05b80 10 client.0 _ll_register_callbacks cb 0x556007929700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T16:37:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.127+0000 7f76b659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a43880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.127+0000 7f76b659c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a43880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:38266/0 (socket says 172.21.15.62:38266) 2023-07-14T16:37:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.127+0000 7f76b659c700 1 -- 172.21.15.62:0/3382042248 learned_addr learned my addr 172.21.15.62:0/3382042248 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.128+0000 7f76b659c700 1 -- 172.21.15.62:0/3382042248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3b6e0 msgr2=0x556007a4ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.128+0000 7f76b659c700 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3b6e0 0x556007a4ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.128+0000 7f76b659c700 1 -- 172.21.15.62:0/3382042248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3a840 msgr2=0x556007a43dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T16:37:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.128+0000 7f76b659c700 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3a840 0x556007a43dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.128+0000 7f76b659c700 1 -- 172.21.15.62:0/3382042248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a80344e0 con 0x5560078966b0 2023-07-14T16:37:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.129+0000 7f76b659c700 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a43880 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005270 tx=0x7f76a0005350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25da8c5fc72df62 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.130+0000 7f769dffb700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2023-07-14T16:37:20.101 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-07-14T16:37:20.101 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-07-14T16:37:20.111 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi062.front.sepia.ceph.com 2023-07-14T16:37:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.130+0000 7f769dffb700 1 -- 172.21.15.62:0/3382042248 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0009ee0 con 0x5560078966b0 2023-07-14T16:37:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.130+0000 7f769dffb700 1 -- 172.21.15.62:0/3382042248 <== mon.1 v2:172.21.15.164:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f76a0004500 con 0x5560078966b0 2023-07-14T16:37:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.131+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3382042248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560078e5490 con 0x5560078966b0 2023-07-14T16:37:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.131+0000 7f769dffb700 1 -- 172.21.15.62:0/3382042248 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76a00046c0 con 0x5560078966b0 2023-07-14T16:37:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f76d0a05b80 10 client.30328 fetch_fsmap learned FSMap version 43 2023-07-14T16:37:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3382042248 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5560077a2c00 con 0x5560078966b0 2023-07-14T16:37:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f769dffb700 1 -- 172.21.15.62:0/3382042248 <== mon.1 v2:172.21.15.164:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5560078e5490 con 0x5560078966b0 2023-07-14T16:37:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f769dffb700 1 -- 172.21.15.62:0/3382042248 <== mon.1 v2:172.21.15.164:3300/0 5 ==== fsmap(e 43) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f76a0003bd0 con 0x5560078966b0 2023-07-14T16:37:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f76d0a05b80 10 client.30328 fetch_fsmap finished waiting for FSMap version 43 2023-07-14T16:37:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f76d0a05b80 10 client.30328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T16:37:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f76d0a05b80 20 client.30328 populate_metadata read hostname 'smithi062' 2023-07-14T16:37:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556007a4f810 0x556007a51c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f76d0a05b80 4 client.30328 mds_command: new command op to 24602 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T16:37:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.132+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3382042248 --> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556007984e50 con 0x556007a4f810 2023-07-14T16:37:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.134+0000 7f76affff700 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556007a4f810 0x556007a51c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.134+0000 7f76affff700 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556007a4f810 0x556007a51c20 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f6fd99ff2bc3ae server_cookie=9c4a587338c317bf in_seq=0 out_seq=0 2023-07-14T16:37:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.134+0000 7f769dffb700 10 client.30328 ms_handle_connect on v2:172.21.15.81:6838/2113600738 2023-07-14T16:37:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.135+0000 7f769dffb700 1 -- 172.21.15.62:0/3382042248 <== mds.0 v2:172.21.15.81:6838/2113600738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556007984e50 con 0x556007a4f810 2023-07-14T16:37:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.135+0000 7f769dffb700 10 client.30328 handle_command_reply: tid=0 2023-07-14T16:37:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.135+0000 7f76d0a05b80 1 client.30328 shutdown 2023-07-14T16:37:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.136+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3382042248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 msgr2=0x556007a43880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.136+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a43880 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005270 tx=0x7f76a0005350 comp rx=0 tx=0).stop 2023-07-14T16:37:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.136+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3382042248 shutdown_connections 2023-07-14T16:37:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.136+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.81:6838/2113600738,v1:172.21.15.81:6839/2113600738] conn(0x556007a4f810 0x556007a51c20 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.136+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556007a3b6e0 0x556007a4ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.136+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560078966b0 0x556007a43880 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.137+0000 7f76d0a05b80 1 --2- 172.21.15.62:0/3382042248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556007a3a840 0x556007a43dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.137+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3382042248 >> 172.21.15.62:0/3382042248 conn(0x556007a2fbe0 msgr2=0x556007a344e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.137+0000 7f769dffb700 0 client.30328 ms_handle_reset on v2:172.21.15.81:6838/2113600738 2023-07-14T16:37:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.137+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3382042248 shutdown_connections 2023-07-14T16:37:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.138+0000 7f76d0a05b80 1 -- 172.21.15.62:0/3382042248 wait complete. 2023-07-14T16:37:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi062.front.sepia.ceph.com.smithi062.stderr:2023-07-14T16:37:20.139+0000 7f76d0a05b80 20 client.30328 trim_cache size 0 max 16384 2023-07-14T16:37:20.134 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-14T16:37:20.145 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-14T16:37:20.146 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-14T16:37:20.146 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-14T16:37:20.158 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:37:20.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:20 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:20.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:20 smithi062 ceph-mon[95543]: pgmap v1008: 97 pgs: 97 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.2 MiB/s wr, 639 op/s 2023-07-14T16:37:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.539+0000 7f9105dff700 1 -- 172.21.15.62:0/4212981258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 msgr2=0x7f91000712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.539+0000 7f9105dff700 1 --2- 172.21.15.62:0/4212981258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 0x7f91000712e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009990 tx=0x7f90f0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.540+0000 7f9105dff700 1 -- 172.21.15.62:0/4212981258 shutdown_connections 2023-07-14T16:37:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.540+0000 7f9105dff700 1 --2- 172.21.15.62:0/4212981258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f910010bd60 0x7f910010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.540+0000 7f9105dff700 1 --2- 172.21.15.62:0/4212981258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 0x7f91000712e0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.540+0000 7f9105dff700 1 --2- 172.21.15.62:0/4212981258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100071820 0x7f910010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.540+0000 7f9105dff700 1 -- 172.21.15.62:0/4212981258 >> 172.21.15.62:0/4212981258 conn(0x7f910006ca40 msgr2=0x7f910006ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.542+0000 7f9105dff700 1 -- 172.21.15.62:0/4212981258 shutdown_connections 2023-07-14T16:37:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.542+0000 7f9105dff700 1 -- 172.21.15.62:0/4212981258 wait complete. 2023-07-14T16:37:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.544+0000 7f9105dff700 1 Processor -- start 2023-07-14T16:37:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.546+0000 7f9105dff700 1 -- start start 2023-07-14T16:37:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.546+0000 7f9105dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 0x7f9100113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.546+0000 7f9105dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100071820 0x7f9100113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.546+0000 7f9105dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f910010bd60 0x7f91001b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.546+0000 7f9105dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91001170b0 con 0x7f9100070ed0 2023-07-14T16:37:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.546+0000 7f9105dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100117220 con 0x7f9100071820 2023-07-14T16:37:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.546+0000 7f9105dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91001b4c70 con 0x7f910010bd60 2023-07-14T16:37:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.548+0000 7f90ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 0x7f9100113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.548+0000 7f90ff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 0x7f9100113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36560/0 (socket says 172.21.15.62:36560) 2023-07-14T16:37:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.548+0000 7f90ff7fe700 1 -- 172.21.15.62:0/1210083740 learned_addr learned my addr 172.21.15.62:0/1210083740 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.548+0000 7f90ff7fe700 1 -- 172.21.15.62:0/1210083740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f910010bd60 msgr2=0x7f91001b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.548+0000 7f90ff7fe700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f910010bd60 0x7f91001b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.521 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.548+0000 7f90ff7fe700 1 -- 172.21.15.62:0/1210083740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100071820 msgr2=0x7f9100113fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.548+0000 7f90ff7fe700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100071820 0x7f9100113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.548+0000 7f90ff7fe700 1 -- 172.21.15.62:0/1210083740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00095f0 con 0x7f9100070ed0 2023-07-14T16:37:20.522 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.549+0000 7f90ff7fe700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 0x7f9100113a90 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000c00 tx=0x7f90f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:20.522 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:20 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:20.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:20 smithi164 ceph-mon[97427]: pgmap v1008: 97 pgs: 97 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.2 MiB/s wr, 639 op/s 2023-07-14T16:37:20.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.550+0000 7f90fcff9700 1 -- 172.21.15.62:0/1210083740 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f001d070 con 0x7f9100070ed0 2023-07-14T16:37:20.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.550+0000 7f90fcff9700 1 -- 172.21.15.62:0/1210083740 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90f000f460 con 0x7f9100070ed0 2023-07-14T16:37:20.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.550+0000 7f90fcff9700 1 -- 172.21.15.62:0/1210083740 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0021670 con 0x7f9100070ed0 2023-07-14T16:37:20.523 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.550+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91001b4e70 con 0x7f9100070ed0 2023-07-14T16:37:20.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.550+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91001b51e0 con 0x7f9100070ed0 2023-07-14T16:37:20.524 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.554+0000 7f90fcff9700 1 -- 172.21.15.62:0/1210083740 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f90f000ba60 con 0x7f9100070ed0 2023-07-14T16:37:20.526 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.556+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e4005320 con 0x7f9100070ed0 2023-07-14T16:37:20.528 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.557+0000 7f90fcff9700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90e00a1340 0x7f90e00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:20.529 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.558+0000 7f90fcff9700 1 -- 172.21.15.62:0/1210083740 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f90f00cdd30 con 0x7f9100070ed0 2023-07-14T16:37:20.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.565+0000 7f90feffd700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90e00a1340 0x7f90e00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.567+0000 7f90fcff9700 1 -- 172.21.15.62:0/1210083740 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f90f00974c0 con 0x7f9100070ed0 2023-07-14T16:37:20.542 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.572+0000 7f90feffd700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90e00a1340 0x7f90e00a37f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9100072530 tx=0x7f90f8009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:20.645 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:20 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:20.645 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:20 smithi081 ceph-mon[127795]: pgmap v1008: 97 pgs: 97 active+clean; 9.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.2 MiB/s wr, 639 op/s 2023-07-14T16:37:20.889 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.918+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e4005190 con 0x7f9100070ed0 2023-07-14T16:37:20.892 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.921+0000 7f90fcff9700 1 -- 172.21.15.62:0/1210083740 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f90f009aae0 con 0x7f9100070ed0 2023-07-14T16:37:20.893 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:37:20.894 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:37:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90e00a1340 msgr2=0x7f90e00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90e00a1340 0x7f90e00a37f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9100072530 tx=0x7f90f8009470 comp rx=0 tx=0).stop 2023-07-14T16:37:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 msgr2=0x7f9100113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:20.897 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 0x7f9100113a90 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000c00 tx=0x7f90f00057f0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 shutdown_connections 2023-07-14T16:37:20.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f910010bd60 0x7f91001b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9100070ed0 0x7f9100113a90 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f90e00a1340 0x7f90e00a37f0 secure :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9100072530 tx=0x7f90f8009470 comp rx=0 tx=0).stop 2023-07-14T16:37:20.899 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 --2- 172.21.15.62:0/1210083740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100071820 0x7f9100113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.926+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 >> 172.21.15.62:0/1210083740 conn(0x7f910006ca40 msgr2=0x7f910006e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.928+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 shutdown_connections 2023-07-14T16:37:20.900 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:20.929+0000 7f9105dff700 1 -- 172.21.15.62:0/1210083740 wait complete. 2023-07-14T16:37:20.936 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:37:20.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:37:21.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.336+0000 7f7959dff700 1 -- 172.21.15.62:0/3498120848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954100cb0 msgr2=0x7f7954103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:21.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.336+0000 7f7959dff700 1 --2- 172.21.15.62:0/3498120848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954100cb0 0x7f7954103130 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7944009990 tx=0x7f7944009ca0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.336+0000 7f7959dff700 1 -- 172.21.15.62:0/3498120848 shutdown_connections 2023-07-14T16:37:21.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.336+0000 7f7959dff700 1 --2- 172.21.15.62:0/3498120848 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7954103670 0x7f7954103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.308 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.336+0000 7f7959dff700 1 --2- 172.21.15.62:0/3498120848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f79540fe350 0x7f7954100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.336+0000 7f7959dff700 1 --2- 172.21.15.62:0/3498120848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954100cb0 0x7f7954103130 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.336+0000 7f7959dff700 1 -- 172.21.15.62:0/3498120848 >> 172.21.15.62:0/3498120848 conn(0x7f79540fa000 msgr2=0x7f79540fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:21.309 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.337+0000 7f7959dff700 1 -- 172.21.15.62:0/3498120848 shutdown_connections 2023-07-14T16:37:21.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.338+0000 7f7959dff700 1 -- 172.21.15.62:0/3498120848 wait complete. 2023-07-14T16:37:21.310 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.339+0000 7f7959dff700 1 Processor -- start 2023-07-14T16:37:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.340+0000 7f7959dff700 1 -- start start 2023-07-14T16:37:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.340+0000 7f7959dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540fe350 0x7f795419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.340+0000 7f7959dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7954100cb0 0x7f795419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.340+0000 7f7959dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954103670 0x7f79541a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.340+0000 7f7959dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79541a38f0 con 0x7f7954103670 2023-07-14T16:37:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.340+0000 7f7959dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79541049a0 con 0x7f79540fe350 2023-07-14T16:37:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.340+0000 7f7959dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7954104b60 con 0x7f7954100cb0 2023-07-14T16:37:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.341+0000 7f7953fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954103670 0x7f79541a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.341+0000 7f7953fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954103670 0x7f79541a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36566/0 (socket says 172.21.15.62:36566) 2023-07-14T16:37:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.341+0000 7f7953fff700 1 -- 172.21.15.62:0/875704258 learned_addr learned my addr 172.21.15.62:0/875704258 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.341+0000 7f7952ffd700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7954100cb0 0x7f795419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.342+0000 7f7953fff700 1 -- 172.21.15.62:0/875704258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7954100cb0 msgr2=0x7f795419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.342+0000 7f7953fff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7954100cb0 0x7f795419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.342+0000 7f7953fff700 1 -- 172.21.15.62:0/875704258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540fe350 msgr2=0x7f795419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.342+0000 7f7953fff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540fe350 0x7f795419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.342+0000 7f7953fff700 1 -- 172.21.15.62:0/875704258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440095f0 con 0x7f7954103670 2023-07-14T16:37:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.343+0000 7f7953fff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954103670 0x7f79541a3320 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f794c00ef30 tx=0x7f794c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.344+0000 7f7950ff9700 1 -- 172.21.15.62:0/875704258 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c018070 con 0x7f7954103670 2023-07-14T16:37:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.344+0000 7f7950ff9700 1 -- 172.21.15.62:0/875704258 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f794c00f040 con 0x7f7954103670 2023-07-14T16:37:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.344+0000 7f7950ff9700 1 -- 172.21.15.62:0/875704258 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c003d00 con 0x7f7954103670 2023-07-14T16:37:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.344+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7954104de0 con 0x7f7954103670 2023-07-14T16:37:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.344+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79541052a0 con 0x7f7954103670 2023-07-14T16:37:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.346+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79540fbb70 con 0x7f7954103670 2023-07-14T16:37:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.348+0000 7f7950ff9700 1 -- 172.21.15.62:0/875704258 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f794c00caa0 con 0x7f7954103670 2023-07-14T16:37:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.351+0000 7f7950ff9700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f793c0a12d0 0x7f793c0a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.351+0000 7f7950ff9700 1 -- 172.21.15.62:0/875704258 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f794c0cc300 con 0x7f7954103670 2023-07-14T16:37:21.324 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.354+0000 7f79537fe700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f793c0a12d0 0x7f793c0a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:21.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.358+0000 7f7950ff9700 1 -- 172.21.15.62:0/875704258 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f794c095a90 con 0x7f7954103670 2023-07-14T16:37:21.329 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.359+0000 7f79537fe700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f793c0a12d0 0x7f793c0a3780 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f795419d280 tx=0x7f7948005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:21.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:21 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1210083740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:37:21.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:21 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1210083740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:37:21.645 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:21 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1210083740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:37:21.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.744+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7954061120 con 0x7f7954103670 2023-07-14T16:37:21.717 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.746+0000 7f7950ff9700 1 -- 172.21.15.62:0/875704258 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f794c0990b0 con 0x7f7954103670 2023-07-14T16:37:21.717 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:37:21.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.751+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f793c0a12d0 msgr2=0x7f793c0a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:21.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.751+0000 7f7959dff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f793c0a12d0 0x7f793c0a3780 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f795419d280 tx=0x7f7948005c80 comp rx=0 tx=0).stop 2023-07-14T16:37:21.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.751+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954103670 msgr2=0x7f79541a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:21.722 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.751+0000 7f7959dff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954103670 0x7f79541a3320 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f794c00ef30 tx=0x7f794c00c450 comp rx=0 tx=0).stop 2023-07-14T16:37:21.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.752+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 shutdown_connections 2023-07-14T16:37:21.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.752+0000 7f7959dff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7954103670 0x7f79541a3320 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.752+0000 7f7959dff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f793c0a12d0 0x7f793c0a3780 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.752+0000 7f7959dff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540fe350 0x7f795419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.752+0000 7f7959dff700 1 --2- 172.21.15.62:0/875704258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7954100cb0 0x7f795419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:21.723 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.752+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 >> 172.21.15.62:0/875704258 conn(0x7f79540fa000 msgr2=0x7f79540feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:21.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.752+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 shutdown_connections 2023-07-14T16:37:21.724 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:21.752+0000 7f7959dff700 1 -- 172.21.15.62:0/875704258 wait complete. 2023-07-14T16:37:21.756 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:37:22.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.157+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/1877212523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8071830 msgr2=0x7fa3c810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:22.128 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.157+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/1877212523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8071830 0x7fa3c810b7d0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009990 tx=0x7fa3bc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.129 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.159+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/1877212523 shutdown_connections 2023-07-14T16:37:22.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.159+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/1877212523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c810bd60 0x7fa3c810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.159+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/1877212523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3c8070ee0 0x7fa3c80712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.159+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/1877212523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8071830 0x7fa3c810b7d0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.159+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/1877212523 >> 172.21.15.62:0/1877212523 conn(0x7fa3c806ca90 msgr2=0x7fa3c806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:22.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.160+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/1877212523 shutdown_connections 2023-07-14T16:37:22.132 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.161+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/1877212523 wait complete. 2023-07-14T16:37:22.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.162+0000 7fa3cf7ff700 1 Processor -- start 2023-07-14T16:37:22.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.163+0000 7fa3cf7ff700 1 -- start start 2023-07-14T16:37:22.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.163+0000 7fa3cf7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8070ee0 0x7fa3c8113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:22.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.163+0000 7fa3cf7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c8071830 0x7fa3c81140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:22.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.163+0000 7fa3cf7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3c810bd60 0x7fa3c81b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:22.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.163+0000 7fa3cf7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81146b0 con 0x7fa3c8070ee0 2023-07-14T16:37:22.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.163+0000 7fa3cf7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8114820 con 0x7fa3c810bd60 2023-07-14T16:37:22.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.163+0000 7fa3cf7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81b4da0 con 0x7fa3c8071830 2023-07-14T16:37:22.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.164+0000 7fa3cd59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8070ee0 0x7fa3c8113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:22.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.164+0000 7fa3cd59b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8070ee0 0x7fa3c8113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36586/0 (socket says 172.21.15.62:36586) 2023-07-14T16:37:22.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.164+0000 7fa3cd59b700 1 -- 172.21.15.62:0/3728749495 learned_addr learned my addr 172.21.15.62:0/3728749495 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.165+0000 7fa3cd59b700 1 -- 172.21.15.62:0/3728749495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c8071830 msgr2=0x7fa3c81140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.165+0000 7fa3cd59b700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c8071830 0x7fa3c81140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.165+0000 7fa3cd59b700 1 -- 172.21.15.62:0/3728749495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3c810bd60 msgr2=0x7fa3c81b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.165+0000 7fa3cd59b700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3c810bd60 0x7fa3c81b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.136 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.165+0000 7fa3cd59b700 1 -- 172.21.15.62:0/3728749495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0095f0 con 0x7fa3c8070ee0 2023-07-14T16:37:22.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.166+0000 7fa3cd59b700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8070ee0 0x7fa3c8113ba0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000b550 tx=0x7fa3c000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:22.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.167+0000 7fa3c67fc700 1 -- 172.21.15.62:0/3728749495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0011830 con 0x7fa3c8070ee0 2023-07-14T16:37:22.137 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.167+0000 7fa3c67fc700 1 -- 172.21.15.62:0/3728749495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3c0011e20 con 0x7fa3c8070ee0 2023-07-14T16:37:22.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.167+0000 7fa3c67fc700 1 -- 172.21.15.62:0/3728749495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0010e10 con 0x7fa3c8070ee0 2023-07-14T16:37:22.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.167+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c81b5080 con 0x7fa3c8070ee0 2023-07-14T16:37:22.138 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.167+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c81b55c0 con 0x7fa3c8070ee0 2023-07-14T16:37:22.140 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.169+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c810af60 con 0x7fa3c8070ee0 2023-07-14T16:37:22.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.179+0000 7fa3c67fc700 1 -- 172.21.15.62:0/3728749495 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa3c0011990 con 0x7fa3c8070ee0 2023-07-14T16:37:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.182+0000 7fa3c67fc700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3b00a1320 0x7fa3b00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.182+0000 7fa3c67fc700 1 -- 172.21.15.62:0/3728749495 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa3c00cc830 con 0x7fa3c8070ee0 2023-07-14T16:37:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.183+0000 7fa3c67fc700 1 -- 172.21.15.62:0/3728749495 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa3c0095fb0 con 0x7fa3c8070ee0 2023-07-14T16:37:22.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.185+0000 7fa3ccd9a700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3b00a1320 0x7fa3b00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:22.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.189+0000 7fa3ccd9a700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3b00a1320 0x7fa3b00a37d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b5c0 tx=0x7fa3bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:22 smithi164 ceph-mon[97427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:22.526 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:22 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/875704258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:37:22.527 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:22 smithi164 ceph-mon[97427]: pgmap v1009: 97 pgs: 97 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 725 KiB/s wr, 280 op/s 2023-07-14T16:37:22.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.580+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3c802c9e0 con 0x7fa3c8070ee0 2023-07-14T16:37:22.551 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.581+0000 7fa3c67fc700 1 -- 172.21.15.62:0/3728749495 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3c00150a0 con 0x7fa3c8070ee0 2023-07-14T16:37:22.552 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:37:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3b00a1320 msgr2=0x7fa3b00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3b00a1320 0x7fa3b00a37d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b5c0 tx=0x7fa3bc005fd0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8070ee0 msgr2=0x7fa3c8113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8070ee0 0x7fa3c8113ba0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000b550 tx=0x7fa3c000b910 comp rx=0 tx=0).stop 2023-07-14T16:37:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 shutdown_connections 2023-07-14T16:37:22.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fa3b00a1320 0x7fa3b00a37d0 secure :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00b5c0 tx=0x7fa3bc005fd0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3c810bd60 0x7fa3c81b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3c8070ee0 0x7fa3c8113ba0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 --2- 172.21.15.62:0/3728749495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c8071830 0x7fa3c81140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.556 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.584+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 >> 172.21.15.62:0/3728749495 conn(0x7fa3c806ca90 msgr2=0x7fa3c806ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.589+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 shutdown_connections 2023-07-14T16:37:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.589+0000 7fa3cf7ff700 1 -- 172.21.15.62:0/3728749495 wait complete. 2023-07-14T16:37:22.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:22 smithi062 ceph-mon[95543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:22.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:22 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/875704258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:37:22.590 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:22 smithi062 ceph-mon[95543]: pgmap v1009: 97 pgs: 97 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 725 KiB/s wr, 280 op/s 2023-07-14T16:37:22.593 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:37:22.646 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:22 smithi081 ceph-mon[127795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T16:37:22.647 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:22 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/875704258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:37:22.647 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:22 smithi081 ceph-mon[127795]: pgmap v1009: 97 pgs: 97 active+clean; 7.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 725 KiB/s wr, 280 op/s 2023-07-14T16:37:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.977+0000 7f2cbedff700 1 -- 172.21.15.62:0/477977411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 msgr2=0x7f2cb81029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.977+0000 7f2cbedff700 1 --2- 172.21.15.62:0/477977411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 0x7f2cb81029e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009990 tx=0x7f2ca8009ca0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.977+0000 7f2cbedff700 1 -- 172.21.15.62:0/477977411 shutdown_connections 2023-07-14T16:37:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.977+0000 7f2cbedff700 1 --2- 172.21.15.62:0/477977411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cb8105840 0x7f2cb8107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.977+0000 7f2cbedff700 1 --2- 172.21.15.62:0/477977411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 0x7f2cb81029e0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.977+0000 7f2cbedff700 1 --2- 172.21.15.62:0/477977411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cb8102f20 0x7f2cb8105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.977+0000 7f2cbedff700 1 -- 172.21.15.62:0/477977411 >> 172.21.15.62:0/477977411 conn(0x7f2cb80fa000 msgr2=0x7f2cb80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:22.949 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.978+0000 7f2cbedff700 1 -- 172.21.15.62:0/477977411 shutdown_connections 2023-07-14T16:37:22.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.978+0000 7f2cbedff700 1 -- 172.21.15.62:0/477977411 wait complete. 2023-07-14T16:37:22.950 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.980+0000 7f2cbedff700 1 Processor -- start 2023-07-14T16:37:22.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.980+0000 7f2cbedff700 1 -- start start 2023-07-14T16:37:22.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.980+0000 7f2cbedff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 0x7f2cb8193660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:22.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.981+0000 7f2cbedff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cb8102f20 0x7f2cb8193ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:22.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.981+0000 7f2cbedff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cb8105840 0x7f2cb819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:22.961 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.981+0000 7f2cbedff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819b1e0 con 0x7f2cb8100600 2023-07-14T16:37:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.981+0000 7f2cbedff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81ac3c0 con 0x7f2cb8102f20 2023-07-14T16:37:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.981+0000 7f2cbedff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81ac500 con 0x7f2cb8105840 2023-07-14T16:37:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.981+0000 7f2cbcb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 0x7f2cb8193660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.981+0000 7f2cbcb9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 0x7f2cb8193660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44366/0 (socket says 172.21.15.62:44366) 2023-07-14T16:37:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.981+0000 7f2cbcb9b700 1 -- 172.21.15.62:0/2425351228 learned_addr learned my addr 172.21.15.62:0/2425351228 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.982+0000 7f2cbd39c700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cb8105840 0x7f2cb819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:22.962 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.982+0000 7f2cbcb9b700 1 -- 172.21.15.62:0/2425351228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cb8105840 msgr2=0x7f2cb819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.982+0000 7f2cbcb9b700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cb8105840 0x7f2cb819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.983+0000 7f2cbcb9b700 1 -- 172.21.15.62:0/2425351228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cb8102f20 msgr2=0x7f2cb8193ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T16:37:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.983+0000 7f2cbcb9b700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cb8102f20 0x7f2cb8193ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.983+0000 7f2cbcb9b700 1 -- 172.21.15.62:0/2425351228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80095f0 con 0x7f2cb8100600 2023-07-14T16:37:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.983+0000 7f2cbcb9b700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 0x7f2cb8193660 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8004730 tx=0x7f2ca8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.985+0000 7f2cb5ffb700 1 -- 172.21.15.62:0/2425351228 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca801d070 con 0x7f2cb8100600 2023-07-14T16:37:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.985+0000 7f2cb5ffb700 1 -- 172.21.15.62:0/2425351228 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ca8004190 con 0x7f2cb8100600 2023-07-14T16:37:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.985+0000 7f2cb5ffb700 1 -- 172.21.15.62:0/2425351228 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8005290 con 0x7f2cb8100600 2023-07-14T16:37:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.987+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb819b3b0 con 0x7f2cb8100600 2023-07-14T16:37:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.987+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb819b8c0 con 0x7f2cb8100600 2023-07-14T16:37:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.988+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb80fbb70 con 0x7f2cb8100600 2023-07-14T16:37:22.969 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:22.999+0000 7f2cb5ffb700 1 -- 172.21.15.62:0/2425351228 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2ca80053f0 con 0x7f2cb8100600 2023-07-14T16:37:22.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.002+0000 7f2cb5ffb700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ca40c32e0 0x7f2ca40c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:22.972 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.002+0000 7f2cb5ffb700 1 -- 172.21.15.62:0/2425351228 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2ca80cdd80 con 0x7f2cb8100600 2023-07-14T16:37:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.002+0000 7f2cb5ffb700 1 -- 172.21.15.62:0/2425351228 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2ca809abb0 con 0x7f2cb8100600 2023-07-14T16:37:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.004+0000 7f2cb7fff700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ca40c32e0 0x7f2ca40c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:22.978 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.007+0000 7f2cb7fff700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ca40c32e0 0x7f2ca40c5790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000ab90 tx=0x7f2cb0009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:23.321 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.351+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2cb802cac0 con 0x7f2cb8100600 2023-07-14T16:37:23.322 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:37:23.322 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.351+0000 7f2cb5ffb700 1 -- 172.21.15.62:0/2425351228 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ca8026020 con 0x7f2cb8100600 2023-07-14T16:37:23.334 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.362+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ca40c32e0 msgr2=0x7f2ca40c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:23.335 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.362+0000 7f2cbedff700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ca40c32e0 0x7f2ca40c5790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000ab90 tx=0x7f2cb0009230 comp rx=0 tx=0).stop 2023-07-14T16:37:23.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.362+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 msgr2=0x7f2cb8193660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:23.336 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.362+0000 7f2cbedff700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 0x7f2cb8193660 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8004730 tx=0x7f2ca8009ee0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.365+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 shutdown_connections 2023-07-14T16:37:23.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.365+0000 7f2cbedff700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cb8105840 0x7f2cb819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.365+0000 7f2cbedff700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cb8100600 0x7f2cb8193660 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.365+0000 7f2cbedff700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2ca40c32e0 0x7f2ca40c5790 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.365+0000 7f2cbedff700 1 --2- 172.21.15.62:0/2425351228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cb8102f20 0x7f2cb8193ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.337 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.365+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 >> 172.21.15.62:0/2425351228 conn(0x7f2cb80fa000 msgr2=0x7f2cb8101260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:23.341 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.368+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 shutdown_connections 2023-07-14T16:37:23.342 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.370+0000 7f2cbedff700 1 -- 172.21.15.62:0/2425351228 wait complete. 2023-07-14T16:37:23.375 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-14T16:37:23.507 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:23 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3728749495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:37:23.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:23 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3728749495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:37:23.645 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:23 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3728749495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:37:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.793+0000 7f388c5ff700 1 -- 172.21.15.62:0/3135359726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3884070ed0 msgr2=0x7f38840712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.793+0000 7f388c5ff700 1 --2- 172.21.15.62:0/3135359726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3884070ed0 0x7f38840712e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f3874009990 tx=0x7f3874009ca0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.794+0000 7f388c5ff700 1 -- 172.21.15.62:0/3135359726 shutdown_connections 2023-07-14T16:37:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.794+0000 7f388c5ff700 1 --2- 172.21.15.62:0/3135359726 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f388410bd60 0x7f388410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.794+0000 7f388c5ff700 1 --2- 172.21.15.62:0/3135359726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3884070ed0 0x7f38840712e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.794+0000 7f388c5ff700 1 --2- 172.21.15.62:0/3135359726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3884071820 0x7f388410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.794+0000 7f388c5ff700 1 -- 172.21.15.62:0/3135359726 >> 172.21.15.62:0/3135359726 conn(0x7f388406ca40 msgr2=0x7f388406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.794+0000 7f388c5ff700 1 -- 172.21.15.62:0/3135359726 shutdown_connections 2023-07-14T16:37:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.795+0000 7f388c5ff700 1 -- 172.21.15.62:0/3135359726 wait complete. 2023-07-14T16:37:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.796+0000 7f388c5ff700 1 Processor -- start 2023-07-14T16:37:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.796+0000 7f388c5ff700 1 -- start start 2023-07-14T16:37:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.797+0000 7f388c5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3884070ed0 0x7f3884113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.797+0000 7f388c5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3884071820 0x7f3884113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.797+0000 7f388c5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f388410bd60 0x7f38841a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.797+0000 7f388c5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38841170e0 con 0x7f3884070ed0 2023-07-14T16:37:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.797+0000 7f388c5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3884117250 con 0x7f3884071820 2023-07-14T16:37:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.797+0000 7f388c5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38841a3c60 con 0x7f388410bd60 2023-07-14T16:37:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.798+0000 7f3889b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3884071820 0x7f3884113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.798+0000 7f3889b9a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3884071820 0x7f3884113f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:41386/0 (socket says 172.21.15.62:41386) 2023-07-14T16:37:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.798+0000 7f3889b9a700 1 -- 172.21.15.62:0/1443804460 learned_addr learned my addr 172.21.15.62:0/1443804460 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:37:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.798+0000 7f388a39b700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3884070ed0 0x7f3884113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.798+0000 7f388ab9c700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f388410bd60 0x7f38841a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.799+0000 7f388ab9c700 1 -- 172.21.15.62:0/1443804460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3884071820 msgr2=0x7f3884113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.799+0000 7f388ab9c700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3884071820 0x7f3884113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.799+0000 7f388ab9c700 1 -- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3884070ed0 msgr2=0x7f3884113a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.799+0000 7f388ab9c700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3884070ed0 0x7f3884113a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.799+0000 7f388ab9c700 1 -- 172.21.15.62:0/1443804460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740095f0 con 0x7f388410bd60 2023-07-14T16:37:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.800+0000 7f388ab9c700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f388410bd60 0x7f38841a3720 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f387c007f10 tx=0x7f387c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.801+0000 7f387b7fe700 1 -- 172.21.15.62:0/1443804460 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c004db0 con 0x7f388410bd60 2023-07-14T16:37:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.801+0000 7f387b7fe700 1 -- 172.21.15.62:0/1443804460 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f387c00f040 con 0x7f388410bd60 2023-07-14T16:37:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.801+0000 7f387b7fe700 1 -- 172.21.15.62:0/1443804460 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0058a0 con 0x7f388410bd60 2023-07-14T16:37:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.801+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38841a3f40 con 0x7f388410bd60 2023-07-14T16:37:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.801+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38841a43d0 con 0x7f388410bd60 2023-07-14T16:37:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.804+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388410af60 con 0x7f388410bd60 2023-07-14T16:37:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.806+0000 7f387b7fe700 1 -- 172.21.15.62:0/1443804460 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f387c00db20 con 0x7f388410bd60 2023-07-14T16:37:23.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.808+0000 7f387b7fe700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38700a1250 0x7f38700a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:37:23.779 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.809+0000 7f387b7fe700 1 -- 172.21.15.62:0/1443804460 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f387c0cc180 con 0x7f388410bd60 2023-07-14T16:37:23.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.810+0000 7f388a39b700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38700a1250 0x7f38700a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:37:23.782 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.811+0000 7f388a39b700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38700a1250 0x7f38700a3700 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3874006010 tx=0x7f387400b970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:37:23.788 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:23.818+0000 7f387b7fe700 1 -- 172.21.15.62:0/1443804460 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f387c04e790 con 0x7f388410bd60 2023-07-14T16:37:24.290 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.319+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f3884061120 con 0x7f388410bd60 2023-07-14T16:37:24.307 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.333+0000 7f387b7fe700 1 -- 172.21.15.62:0/1443804460 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733243 (secure 0 0 0) 0x7f387c095880 con 0x7f388410bd60 2023-07-14T16:37:24.311 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:37:24.312 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:37:24.312 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 18, 2023-07-14T16:37:24.313 INFO:teuthology.orchestra.run.smithi062.stdout: "active_gid": 14152, 2023-07-14T16:37:24.313 INFO:teuthology.orchestra.run.smithi062.stdout: "active_name": "x", 2023-07-14T16:37:24.313 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addrs": { 2023-07-14T16:37:24.314 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2023-07-14T16:37:24.314 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.314 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2023-07-14T16:37:24.314 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6800", 2023-07-14T16:37:24.315 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 4048727837 2023-07-14T16:37:24.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.315 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2023-07-14T16:37:24.316 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6801", 2023-07-14T16:37:24.316 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 4048727837 2023-07-14T16:37:24.316 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.317 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:24.317 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.317 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addr": "172.21.15.62:6801/4048727837", 2023-07-14T16:37:24.317 INFO:teuthology.orchestra.run.smithi062.stdout: "active_change": "2023-07-14T16:06:19.370143+0000", 2023-07-14T16:37:24.317 INFO:teuthology.orchestra.run.smithi062.stdout: "active_mgr_features": 4540138322906710015, 2023-07-14T16:37:24.318 INFO:teuthology.orchestra.run.smithi062.stdout: "available": true, 2023-07-14T16:37:24.318 INFO:teuthology.orchestra.run.smithi062.stdout: "standbys": [ 2023-07-14T16:37:24.318 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.319 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24112, 2023-07-14T16:37:24.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "z", 2023-07-14T16:37:24.319 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540138322906710015, 2023-07-14T16:37:24.319 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2023-07-14T16:37:24.319 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2023-07-14T16:37:24.320 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.320 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.320 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.320 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:24.320 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:24.320 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.320 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.321 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:24.321 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.321 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.321 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.321 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T16:37:24.321 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.322 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.322 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.322 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.322 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.322 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.322 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.322 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.322 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.323 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.323 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.323 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.323 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.323 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.323 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.323 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.323 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.324 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.324 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.324 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.324 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.325 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.325 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.325 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.325 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.325 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.325 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.325 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.326 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.326 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.326 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.326 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.326 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.327 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.327 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.327 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.327 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.327 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.327 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.328 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.328 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.328 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.328 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.328 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.328 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.328 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.329 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.329 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.329 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.329 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.329 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.329 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.330 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2023-07-14T16:37:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2023-07-14T16:37:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.331 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2023-07-14T16:37:24.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.333 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.333 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.333 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2023-07-14T16:37:24.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2023-07-14T16:37:24.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.333 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.334 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.334 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2023-07-14T16:37:24.334 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.334 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2023-07-14T16:37:24.335 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.335 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.335 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.335 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2023-07-14T16:37:24.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2023-07-14T16:37:24.336 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.336 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.336 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.336 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.337 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2023-07-14T16:37:24.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.337 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2023-07-14T16:37:24.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2023-07-14T16:37:24.338 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.338 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2023-07-14T16:37:24.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.339 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2023-07-14T16:37:24.339 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2023-07-14T16:37:24.339 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.340 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.340 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2023-07-14T16:37:24.340 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.340 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2023-07-14T16:37:24.340 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.341 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.341 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.341 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2023-07-14T16:37:24.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2023-07-14T16:37:24.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.342 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.342 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.342 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.342 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.342 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.342 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.343 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2023-07-14T16:37:24.343 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.343 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.343 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.344 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2023-07-14T16:37:24.344 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2023-07-14T16:37:24.344 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.344 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.344 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.344 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.345 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.345 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.345 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.345 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T16:37:24.345 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.345 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.346 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2023-07-14T16:37:24.346 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2023-07-14T16:37:24.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.346 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.346 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.347 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.347 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2023-07-14T16:37:24.347 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.347 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.348 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.348 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.348 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.348 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.349 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.349 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2023-07-14T16:37:24.349 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.349 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.350 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.350 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2023-07-14T16:37:24.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2023-07-14T16:37:24.351 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.351 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.351 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.351 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.352 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.352 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T16:37:24.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.354 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.354 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2023-07-14T16:37:24.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2023-07-14T16:37:24.355 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.355 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.355 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.355 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2023-07-14T16:37:24.356 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.356 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T16:37:24.357 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T16:37:24.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.357 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.357 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2023-07-14T16:37:24.358 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2023-07-14T16:37:24.358 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:24.358 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.359 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.359 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:24.359 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2023-07-14T16:37:24.359 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2023-07-14T16:37:24.360 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.360 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T16:37:24.360 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T16:37:24.360 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.360 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.360 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2023-07-14T16:37:24.361 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2023-07-14T16:37:24.361 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:24.361 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.361 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.361 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2023-07-14T16:37:24.361 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:24.361 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2023-07-14T16:37:24.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.362 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T16:37:24.362 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.362 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2023-07-14T16:37:24.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2023-07-14T16:37:24.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T16:37:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T16:37:24.363 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T16:37:24.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.364 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2023-07-14T16:37:24.364 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2023-07-14T16:37:24.364 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:24.364 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2023-07-14T16:37:24.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2023-07-14T16:37:24.365 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2023-07-14T16:37:24.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.365 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2023-07-14T16:37:24.365 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T16:37:24.365 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.366 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.366 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.366 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2023-07-14T16:37:24.366 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2023-07-14T16:37:24.366 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.366 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.366 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.366 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2023-07-14T16:37:24.367 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.367 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.367 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.367 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T16:37:24.367 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T16:37:24.367 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.368 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.368 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2023-07-14T16:37:24.368 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2023-07-14T16:37:24.368 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:24.368 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.368 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.368 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:24.369 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2023-07-14T16:37:24.369 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2023-07-14T16:37:24.369 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.369 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T16:37:24.369 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T16:37:24.369 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.370 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.370 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.370 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.370 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.371 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.371 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.371 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.371 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.371 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.372 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.372 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.372 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.372 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.372 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.372 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.372 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.373 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.373 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.373 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.373 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.373 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.373 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.374 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.374 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.374 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.374 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.375 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.375 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.376 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.376 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.376 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.376 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.376 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.376 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.377 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.377 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.377 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.377 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.377 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.377 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.377 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.377 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.378 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.378 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.378 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.378 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.378 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.378 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.378 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.379 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.379 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.379 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.379 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2023-07-14T16:37:24.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2023-07-14T16:37:24.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:24.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:24.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.381 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.381 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T16:37:24.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.382 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.382 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.383 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2023-07-14T16:37:24.383 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2023-07-14T16:37:24.383 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.384 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2023-07-14T16:37:24.384 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.385 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.385 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.385 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2023-07-14T16:37:24.385 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2023-07-14T16:37:24.385 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap" 2023-07-14T16:37:24.386 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2023-07-14T16:37:24.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.387 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.387 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.387 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2023-07-14T16:37:24.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2023-07-14T16:37:24.388 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.388 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.388 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.388 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.388 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.389 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.389 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T16:37:24.389 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.389 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.389 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.389 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:24.390 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:24.390 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.390 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.390 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.390 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:24.390 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.390 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.390 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.391 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T16:37:24.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.391 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.391 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2023-07-14T16:37:24.391 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2023-07-14T16:37:24.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.392 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.392 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:24.392 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:24.392 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.392 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.392 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T16:37:24.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T16:37:24.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.393 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.393 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2023-07-14T16:37:24.393 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2023-07-14T16:37:24.393 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:24.393 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.394 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:24.394 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.394 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.394 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.394 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T16:37:24.394 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.395 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.395 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2023-07-14T16:37:24.395 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.396 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.396 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.396 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2023-07-14T16:37:24.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2023-07-14T16:37:24.396 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:24.396 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.396 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.396 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2023-07-14T16:37:24.397 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.397 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.397 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.397 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T16:37:24.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.397 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.397 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.398 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2023-07-14T16:37:24.398 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2023-07-14T16:37:24.398 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.398 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.398 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2023-07-14T16:37:24.398 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.399 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.399 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T16:37:24.399 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.399 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.399 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.399 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.399 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2023-07-14T16:37:24.399 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2023-07-14T16:37:24.400 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.400 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.400 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.400 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2023-07-14T16:37:24.400 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T16:37:24.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.401 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.401 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.401 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.401 INFO:teuthology.orchestra.run.smithi062.stdout: "alertmanager_web_password": { 2023-07-14T16:37:24.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alertmanager_web_password", 2023-07-14T16:37:24.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.402 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:24.402 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.402 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.402 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.402 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Alertmanager web password", 2023-07-14T16:37:24.403 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.403 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.403 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.403 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.403 INFO:teuthology.orchestra.run.smithi062.stdout: "alertmanager_web_user": { 2023-07-14T16:37:24.403 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alertmanager_web_user", 2023-07-14T16:37:24.403 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.403 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.404 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.404 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:24.404 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.404 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.404 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.405 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Alertmanager web user", 2023-07-14T16:37:24.405 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.405 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.405 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.405 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.405 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2023-07-14T16:37:24.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2023-07-14T16:37:24.406 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.406 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.406 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.406 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.406 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.406 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T16:37:24.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T16:37:24.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.407 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2023-07-14T16:37:24.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2023-07-14T16:37:24.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:24.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.408 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.409 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T16:37:24.409 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.409 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.409 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.409 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.409 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2023-07-14T16:37:24.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T16:37:24.410 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:24.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2023-07-14T16:37:24.410 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.410 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.410 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T16:37:24.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.411 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.411 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2023-07-14T16:37:24.411 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2023-07-14T16:37:24.411 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.411 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.412 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.412 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.412 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.412 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.412 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.412 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T16:37:24.413 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.413 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.413 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.413 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.414 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2023-07-14T16:37:24.414 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2023-07-14T16:37:24.414 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.414 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.414 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.414 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.415 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.415 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T16:37:24.415 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.415 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.415 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.415 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.415 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2023-07-14T16:37:24.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2023-07-14T16:37:24.416 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.416 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.416 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.416 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.416 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.416 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.416 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T16:37:24.417 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.417 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.417 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.417 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.417 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2023-07-14T16:37:24.417 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2023-07-14T16:37:24.417 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.418 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.418 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.418 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T16:37:24.418 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.418 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.418 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.418 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:24.418 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.419 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.419 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.419 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.419 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2023-07-14T16:37:24.419 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2023-07-14T16:37:24.419 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.420 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.420 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.420 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T16:37:24.420 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.420 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.420 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.420 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2023-07-14T16:37:24.420 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.421 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.421 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2023-07-14T16:37:24.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2023-07-14T16:37:24.421 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.421 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.421 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T16:37:24.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.422 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2023-07-14T16:37:24.422 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.422 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.422 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.423 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2023-07-14T16:37:24.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2023-07-14T16:37:24.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.423 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.423 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.423 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T16:37:24.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.424 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:24.424 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.424 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.424 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.424 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.425 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2023-07-14T16:37:24.425 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2023-07-14T16:37:24.425 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.425 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.425 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.425 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T16:37:24.425 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2023-07-14T16:37:24.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.426 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2023-07-14T16:37:24.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2023-07-14T16:37:24.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T16:37:24.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.427 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.428 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2023-07-14T16:37:24.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.428 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.428 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2023-07-14T16:37:24.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2023-07-14T16:37:24.429 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.429 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.429 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T16:37:24.429 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.429 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2023-07-14T16:37:24.430 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.430 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.430 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.430 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.430 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2023-07-14T16:37:24.430 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2023-07-14T16:37:24.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.431 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.431 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T16:37:24.431 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.431 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.431 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2023-07-14T16:37:24.432 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.432 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.432 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2023-07-14T16:37:24.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2023-07-14T16:37:24.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.433 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.433 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T16:37:24.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.433 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.433 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2023-07-14T16:37:24.433 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2023-07-14T16:37:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2023-07-14T16:37:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.434 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T16:37:24.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2023-07-14T16:37:24.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.435 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.435 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.436 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2023-07-14T16:37:24.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2023-07-14T16:37:24.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.436 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.436 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T16:37:24.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:24.437 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.437 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.437 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.437 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.438 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2023-07-14T16:37:24.438 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2023-07-14T16:37:24.438 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T16:37:24.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.438 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2023-07-14T16:37:24.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2023-07-14T16:37:24.440 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.440 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.440 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.440 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T16:37:24.440 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.440 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.441 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2023-07-14T16:37:24.441 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.441 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.441 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.441 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2023-07-14T16:37:24.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2023-07-14T16:37:24.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T16:37:24.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.442 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.442 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2023-07-14T16:37:24.443 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.443 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.443 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2023-07-14T16:37:24.443 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2023-07-14T16:37:24.443 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.443 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.444 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.444 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.444 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.444 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.444 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.444 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T16:37:24.444 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.445 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.445 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.445 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.445 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2023-07-14T16:37:24.445 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2023-07-14T16:37:24.445 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.445 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.446 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.446 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:24.446 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.446 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.446 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.446 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T16:37:24.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.447 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.447 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.447 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2023-07-14T16:37:24.447 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T16:37:24.447 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.448 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2023-07-14T16:37:24.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.448 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T16:37:24.448 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.450 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2023-07-14T16:37:24.450 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2023-07-14T16:37:24.450 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.450 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.451 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.451 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2023-07-14T16:37:24.451 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.451 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.452 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T16:37:24.452 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.453 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.453 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.454 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2023-07-14T16:37:24.454 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2023-07-14T16:37:24.454 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.454 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.455 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.455 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2023-07-14T16:37:24.455 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.456 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.456 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2023-07-14T16:37:24.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.457 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.457 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.457 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2023-07-14T16:37:24.457 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2023-07-14T16:37:24.458 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.458 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.458 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.458 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.459 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T16:37:24.460 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.460 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.460 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2023-07-14T16:37:24.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2023-07-14T16:37:24.461 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.461 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.461 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.461 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:24.461 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.461 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.461 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2023-07-14T16:37:24.462 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.462 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.462 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.462 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.462 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2023-07-14T16:37:24.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2023-07-14T16:37:24.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.463 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.463 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:24.463 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.463 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.463 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.463 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2023-07-14T16:37:24.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.463 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.464 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.464 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.464 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.465 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.465 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.465 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.465 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.466 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.466 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.466 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.466 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.466 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.466 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.466 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.466 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.467 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2023-07-14T16:37:24.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2023-07-14T16:37:24.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.467 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.467 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T16:37:24.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.468 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.469 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.469 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.469 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.469 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.469 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.469 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.469 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.470 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.470 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T16:37:24.470 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.470 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.470 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.470 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.471 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.471 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.472 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.472 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.472 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.472 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.472 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.472 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.472 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.473 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.473 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.473 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.473 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.473 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.473 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.474 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.474 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.474 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.474 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.474 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.474 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.475 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.475 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.475 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T16:37:24.475 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T16:37:24.475 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.475 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.476 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.476 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.476 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.476 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.476 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.476 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T16:37:24.476 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.477 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.477 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T16:37:24.477 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T16:37:24.477 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.477 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2023-07-14T16:37:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T16:37:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.478 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.479 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.479 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2023-07-14T16:37:24.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2023-07-14T16:37:24.479 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.479 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.480 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.480 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:24.480 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.480 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.480 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2023-07-14T16:37:24.481 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.481 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.481 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2023-07-14T16:37:24.481 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2023-07-14T16:37:24.481 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.481 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.482 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.482 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:24.482 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.482 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.482 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.482 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T16:37:24.482 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.482 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.483 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.483 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2023-07-14T16:37:24.483 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2023-07-14T16:37:24.483 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.483 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.483 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.484 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.484 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.484 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.484 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2023-07-14T16:37:24.484 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.484 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.484 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.485 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.485 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2023-07-14T16:37:24.485 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2023-07-14T16:37:24.485 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.485 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.485 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2023-07-14T16:37:24.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.486 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.486 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.486 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2023-07-14T16:37:24.486 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2023-07-14T16:37:24.486 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.486 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T16:37:24.486 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.487 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.487 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.487 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2023-07-14T16:37:24.487 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2023-07-14T16:37:24.487 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.487 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.487 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T16:37:24.488 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.488 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.488 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.488 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T16:37:24.488 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.488 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.488 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.489 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_web_password": { 2023-07-14T16:37:24.489 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_web_password", 2023-07-14T16:37:24.489 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.489 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.489 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.489 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:24.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.490 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.490 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus web password", 2023-07-14T16:37:24.490 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.490 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.490 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_web_user": { 2023-07-14T16:37:24.490 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_web_user", 2023-07-14T16:37:24.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.491 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.491 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.491 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:24.491 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.491 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.491 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.491 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus web user", 2023-07-14T16:37:24.492 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.492 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.492 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.492 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.492 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2023-07-14T16:37:24.492 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2023-07-14T16:37:24.492 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.493 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.493 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.493 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.493 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.493 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T16:37:24.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.493 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.494 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.494 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2023-07-14T16:37:24.494 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2023-07-14T16:37:24.494 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.494 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.494 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.494 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.495 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.495 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T16:37:24.495 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.495 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.495 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.496 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2023-07-14T16:37:24.496 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2023-07-14T16:37:24.496 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.496 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.496 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.497 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.497 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T16:37:24.497 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.497 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.497 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2023-07-14T16:37:24.498 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2023-07-14T16:37:24.498 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.498 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.498 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.498 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.498 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.498 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.499 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T16:37:24.499 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.499 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.499 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.499 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2023-07-14T16:37:24.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2023-07-14T16:37:24.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.500 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.500 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.500 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.500 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.500 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.500 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.500 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T16:37:24.501 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.501 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.501 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.501 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.501 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2023-07-14T16:37:24.501 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2023-07-14T16:37:24.501 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.501 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.502 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.502 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2023-07-14T16:37:24.502 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.502 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.502 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.502 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2023-07-14T16:37:24.502 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.502 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.503 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.503 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2023-07-14T16:37:24.503 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2023-07-14T16:37:24.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.503 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.503 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.504 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.504 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.504 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.504 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T16:37:24.504 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.504 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.504 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.505 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2023-07-14T16:37:24.505 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2023-07-14T16:37:24.505 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.505 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.505 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.505 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.505 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.505 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.506 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.506 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T16:37:24.506 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.506 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.506 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.506 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.506 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2023-07-14T16:37:24.507 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2023-07-14T16:37:24.507 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.507 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.507 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.507 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.507 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.507 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.507 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.508 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T16:37:24.508 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.508 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.508 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.508 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.508 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2023-07-14T16:37:24.508 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2023-07-14T16:37:24.508 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.509 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.509 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.509 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.509 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.509 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.509 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.509 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T16:37:24.509 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.510 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.510 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.510 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2023-07-14T16:37:24.510 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2023-07-14T16:37:24.510 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.510 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.510 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.511 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.511 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.511 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.511 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T16:37:24.511 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.511 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.511 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.512 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.512 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2023-07-14T16:37:24.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2023-07-14T16:37:24.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.512 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.512 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.512 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.512 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.513 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.513 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.513 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T16:37:24.513 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.513 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.513 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.513 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.513 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.514 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2023-07-14T16:37:24.514 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.514 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.514 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.514 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.515 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.515 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.515 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.515 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.515 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.516 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.516 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.516 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.516 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.516 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.516 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.516 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.517 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.517 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.517 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.517 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.517 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.517 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.517 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.517 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.518 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.518 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.518 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.518 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.518 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.518 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.518 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.518 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.519 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.519 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.519 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.519 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.519 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.520 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.520 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.520 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.520 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.520 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.520 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.520 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.520 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.521 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.521 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.521 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.521 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.521 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.521 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.521 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.522 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.522 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.522 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.522 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.522 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.522 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.522 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.522 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.523 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.523 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.523 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2023-07-14T16:37:24.523 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2023-07-14T16:37:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2023-07-14T16:37:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T16:37:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2023-07-14T16:37:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2023-07-14T16:37:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.526 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.526 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2023-07-14T16:37:24.526 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.526 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.526 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.526 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T16:37:24.526 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.526 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.527 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.527 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.527 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.527 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.527 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2023-07-14T16:37:24.527 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.527 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.528 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.528 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T16:37:24.528 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T16:37:24.528 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.528 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.528 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.528 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:24.528 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.529 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.529 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.529 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.529 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.529 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.529 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.529 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T16:37:24.530 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T16:37:24.530 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.530 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.530 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.530 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.530 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.530 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.530 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.531 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.531 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.531 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.531 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.531 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T16:37:24.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T16:37:24.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.532 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.532 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.532 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.533 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.533 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.533 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.533 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.533 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2023-07-14T16:37:24.533 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T16:37:24.533 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.534 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.534 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.534 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.534 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.534 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.534 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.535 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.535 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.535 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.535 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T16:37:24.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T16:37:24.535 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.535 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.535 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.536 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.536 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.536 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.536 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.537 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.537 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.537 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.537 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.537 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T16:37:24.537 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T16:37:24.538 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.538 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.538 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.538 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.538 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.538 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.539 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.539 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.539 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.539 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T16:37:24.539 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T16:37:24.539 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.539 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.540 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.540 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.540 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.540 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.540 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.540 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.540 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.541 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.541 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T16:37:24.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T16:37:24.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.541 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.541 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.542 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.542 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.542 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.542 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.542 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T16:37:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T16:37:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.543 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.544 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.544 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.544 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.544 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.544 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T16:37:24.545 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T16:37:24.545 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.545 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.545 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.545 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.545 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.545 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.545 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.546 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.546 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.546 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.546 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T16:37:24.546 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T16:37:24.547 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.547 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.547 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.547 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.547 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.547 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.547 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.548 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.548 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T16:37:24.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T16:37:24.548 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.548 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.549 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.549 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.550 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.550 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T16:37:24.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T16:37:24.550 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.550 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.550 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.551 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.551 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.551 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.551 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.551 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.552 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T16:37:24.552 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T16:37:24.552 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.552 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.552 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.552 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.552 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.553 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.553 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.553 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.553 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.553 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.553 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.553 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.553 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T16:37:24.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T16:37:24.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.554 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.554 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.554 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:24.554 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.554 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.555 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.555 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.555 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.555 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.555 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.555 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T16:37:24.555 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T16:37:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2023-07-14T16:37:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2023-07-14T16:37:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.558 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.558 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.558 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.558 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.558 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.559 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T16:37:24.559 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T16:37:24.559 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.559 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:24.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.560 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.560 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.560 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.560 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.560 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.560 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.561 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T16:37:24.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T16:37:24.561 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.561 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.561 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.561 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.561 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.562 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.562 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.562 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.562 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.562 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.562 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.562 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T16:37:24.563 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T16:37:24.563 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.563 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.563 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.563 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.563 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.564 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.564 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.564 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.564 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.564 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.564 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.565 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T16:37:24.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T16:37:24.565 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.565 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.565 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.565 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.565 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.565 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.566 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.566 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.566 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.566 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.566 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.566 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T16:37:24.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T16:37:24.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.567 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.567 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.567 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.567 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.567 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.567 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.567 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.568 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.568 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T16:37:24.568 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T16:37:24.568 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.569 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.569 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.569 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.569 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.569 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.569 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.570 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.570 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.570 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T16:37:24.570 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T16:37:24.570 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.571 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.571 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.571 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.571 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.571 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.571 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.571 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.572 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.572 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.572 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T16:37:24.572 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T16:37:24.572 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.573 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.573 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.573 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.573 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.573 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.573 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.573 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.573 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.574 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.574 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.574 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T16:37:24.574 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T16:37:24.574 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.574 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.574 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.576 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T16:37:24.576 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T16:37:24.576 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.576 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.577 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.577 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.577 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.577 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.577 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.577 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.578 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T16:37:24.578 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T16:37:24.578 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.578 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.578 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T16:37:24.579 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T16:37:24.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.580 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.580 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.580 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.581 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.581 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.581 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.581 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.581 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T16:37:24.581 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T16:37:24.581 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.581 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.582 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.582 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.582 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.582 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.582 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.582 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.582 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.582 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.583 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.583 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.583 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T16:37:24.583 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T16:37:24.583 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.583 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.583 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.583 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.584 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.584 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.584 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.584 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.584 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.584 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.585 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T16:37:24.585 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T16:37:24.585 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.585 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.585 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.585 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T16:37:24.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T16:37:24.587 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.587 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.587 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.587 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T16:37:24.587 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.587 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.587 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.588 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.588 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.588 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.588 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.588 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T16:37:24.588 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T16:37:24.588 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.589 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.589 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.589 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:24.589 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.589 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.589 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.589 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.590 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.590 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.590 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.590 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T16:37:24.590 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T16:37:24.590 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.590 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.591 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2023-07-14T16:37:24.591 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.591 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.591 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.591 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.591 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.591 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.592 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T16:37:24.592 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T16:37:24.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.592 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2023-07-14T16:37:24.592 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.593 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.593 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.593 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.593 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.593 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.593 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.593 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.593 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T16:37:24.594 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T16:37:24.594 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.594 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.594 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.594 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.594 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.594 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.594 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.595 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.595 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.595 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.595 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.595 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.595 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T16:37:24.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T16:37:24.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.596 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.596 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.596 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:24.596 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.596 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.596 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.596 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.597 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.597 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.597 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.597 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.597 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T16:37:24.597 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T16:37:24.597 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.597 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.598 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.598 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.598 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.598 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.598 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.598 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.598 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.599 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.599 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.599 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T16:37:24.599 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T16:37:24.599 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.599 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.599 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.599 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.600 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.600 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.600 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.600 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.600 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.600 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.600 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.601 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T16:37:24.601 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T16:37:24.601 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.601 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.601 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.601 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:24.601 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.602 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.602 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.602 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.602 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.602 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.602 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.602 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T16:37:24.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T16:37:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.603 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.604 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.604 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.604 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.604 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.604 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T16:37:24.604 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T16:37:24.604 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.605 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.605 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:24.605 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.605 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.605 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.605 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.606 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.606 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2023-07-14T16:37:24.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2023-07-14T16:37:24.606 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.606 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.606 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.607 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.607 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.607 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.607 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.607 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.607 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.607 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.608 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2023-07-14T16:37:24.608 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2023-07-14T16:37:24.608 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.608 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.608 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.608 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.608 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.608 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.609 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.609 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.609 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.609 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.609 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.609 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2023-07-14T16:37:24.609 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2023-07-14T16:37:24.610 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.610 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.610 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.610 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.610 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.610 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.610 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2023-07-14T16:37:24.611 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.611 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.611 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.611 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2023-07-14T16:37:24.611 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2023-07-14T16:37:24.611 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.612 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.612 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.612 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2023-07-14T16:37:24.612 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.612 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.612 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.612 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.612 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.613 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.613 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.613 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2023-07-14T16:37:24.613 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2023-07-14T16:37:24.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.613 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.613 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.614 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.614 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.614 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.614 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.614 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.614 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.614 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.615 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.615 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.615 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.615 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.615 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.615 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.615 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.615 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.616 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.616 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.616 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.616 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.616 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.616 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.616 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.616 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.617 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.617 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.617 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.617 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.617 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.617 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.618 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.618 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.618 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.618 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.618 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.618 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.618 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.618 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.619 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.619 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.619 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.620 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.620 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.621 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.621 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.621 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.621 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.621 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.621 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.622 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.622 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.622 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.622 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.622 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.622 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.623 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.623 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.623 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.623 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.623 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.624 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2023-07-14T16:37:24.624 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2023-07-14T16:37:24.624 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.624 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.624 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.624 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.625 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.625 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2023-07-14T16:37:24.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.625 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2023-07-14T16:37:24.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T16:37:24.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.626 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.626 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.627 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.627 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:24.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:24.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.628 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2023-07-14T16:37:24.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.628 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.628 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.628 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.629 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.629 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:24.629 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:24.629 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.629 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.629 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.629 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2023-07-14T16:37:24.630 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.630 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.630 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.630 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.630 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.631 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2023-07-14T16:37:24.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2023-07-14T16:37:24.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.631 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.631 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.631 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.632 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.632 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.632 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.632 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2023-07-14T16:37:24.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2023-07-14T16:37:24.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.633 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2023-07-14T16:37:24.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.633 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.633 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.634 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.634 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.634 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2023-07-14T16:37:24.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2023-07-14T16:37:24.634 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.634 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.635 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.635 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2023-07-14T16:37:24.635 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.635 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.635 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.635 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.635 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2023-07-14T16:37:24.635 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.636 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.636 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.636 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.636 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.636 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2023-07-14T16:37:24.636 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2023-07-14T16:37:24.637 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.637 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.637 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.637 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2023-07-14T16:37:24.637 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2023-07-14T16:37:24.637 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2023-07-14T16:37:24.637 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.637 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.638 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.639 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.640 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.640 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.640 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2023-07-14T16:37:24.640 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.640 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.641 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2023-07-14T16:37:24.641 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2023-07-14T16:37:24.641 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.641 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.641 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.642 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.642 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.642 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2023-07-14T16:37:24.642 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.642 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.642 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.642 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.643 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.643 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.643 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.643 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.643 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.644 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.644 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.644 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.644 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.644 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.644 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.644 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.645 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.646 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.647 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.647 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.647 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.647 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.648 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.648 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.648 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.648 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.649 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.649 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.649 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.649 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.649 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.650 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.650 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.650 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.650 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.650 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.650 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.650 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.650 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.651 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.651 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.651 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.651 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2023-07-14T16:37:24.651 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2023-07-14T16:37:24.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.652 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.652 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2023-07-14T16:37:24.652 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.652 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.652 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.652 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T16:37:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2023-07-14T16:37:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2023-07-14T16:37:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.653 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.654 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.654 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2023-07-14T16:37:24.654 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.654 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.654 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.654 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T16:37:24.654 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.654 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.655 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.655 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2023-07-14T16:37:24.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2023-07-14T16:37:24.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.655 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2023-07-14T16:37:24.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2023-07-14T16:37:24.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.657 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2023-07-14T16:37:24.657 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2023-07-14T16:37:24.657 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.657 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.657 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.657 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2023-07-14T16:37:24.657 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.658 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.658 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T16:37:24.658 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.658 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.658 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.658 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2023-07-14T16:37:24.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2023-07-14T16:37:24.659 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.659 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.659 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.659 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.659 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.659 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.659 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.659 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T16:37:24.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.660 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.660 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.660 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:24.660 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:24.660 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.660 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.661 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:24.661 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.661 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T16:37:24.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.662 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.662 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.662 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2023-07-14T16:37:24.662 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2023-07-14T16:37:24.662 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.662 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.662 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2023-07-14T16:37:24.663 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.663 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.663 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.663 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T16:37:24.663 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.663 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.663 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.663 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2023-07-14T16:37:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.664 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.665 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.666 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.666 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.666 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.666 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.666 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.666 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.666 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.667 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.667 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.667 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.667 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.667 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.667 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.667 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.668 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.668 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.668 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.668 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.668 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.668 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.668 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.668 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.669 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.669 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.669 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.669 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.669 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.669 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.670 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.670 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.670 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.670 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.670 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.670 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.670 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.671 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.671 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.671 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.671 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.671 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.671 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.671 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.671 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.672 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.672 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.672 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.672 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.672 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.672 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.672 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.672 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.673 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.673 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2023-07-14T16:37:24.673 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2023-07-14T16:37:24.674 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.674 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2023-07-14T16:37:24.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.674 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.675 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.675 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.675 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.675 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2023-07-14T16:37:24.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2023-07-14T16:37:24.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.676 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.676 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2023-07-14T16:37:24.676 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.676 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.676 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.677 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.677 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.677 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:24.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:24.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:24.678 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.678 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.678 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.678 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.678 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.678 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.678 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.679 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.679 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2023-07-14T16:37:24.679 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2023-07-14T16:37:24.679 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2023-07-14T16:37:24.679 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.680 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2023-07-14T16:37:24.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2023-07-14T16:37:24.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.680 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2023-07-14T16:37:24.680 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.681 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.681 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T16:37:24.681 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.681 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.681 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2023-07-14T16:37:24.681 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2023-07-14T16:37:24.682 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.682 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2023-07-14T16:37:24.682 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.682 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.682 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.682 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T16:37:24.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.683 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.683 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.683 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.683 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2023-07-14T16:37:24.683 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2023-07-14T16:37:24.683 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.683 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2023-07-14T16:37:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.684 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.685 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.685 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:24.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:24.685 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.685 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.685 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.685 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2023-07-14T16:37:24.686 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2023-07-14T16:37:24.686 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.686 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T16:37:24.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.686 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.686 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.687 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.687 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.687 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.687 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.688 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.688 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.688 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.688 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.688 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.688 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.688 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.691 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.692 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.693 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.694 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.695 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2023-07-14T16:37:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2023-07-14T16:37:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.696 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2023-07-14T16:37:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2023-07-14T16:37:24.697 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2023-07-14T16:37:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2023-07-14T16:37:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.698 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2023-07-14T16:37:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2023-07-14T16:37:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2023-07-14T16:37:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.699 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2023-07-14T16:37:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T16:37:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.700 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.701 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.701 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2023-07-14T16:37:24.701 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2023-07-14T16:37:24.701 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.701 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.701 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.701 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:24.702 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:24.702 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2023-07-14T16:37:24.702 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T16:37:24.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.702 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2023-07-14T16:37:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2023-07-14T16:37:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.703 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2023-07-14T16:37:24.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.704 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.704 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2023-07-14T16:37:24.704 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2023-07-14T16:37:24.705 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.705 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.705 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2023-07-14T16:37:24.705 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.705 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.705 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T16:37:24.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.706 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.706 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.706 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2023-07-14T16:37:24.707 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.707 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.707 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.707 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.707 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.707 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.708 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.708 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.708 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.708 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.708 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.708 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.709 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.709 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.709 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.709 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.709 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.709 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.710 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.710 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.710 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.710 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.710 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.711 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.711 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.711 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.711 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.711 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.711 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.712 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.712 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.712 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.712 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.712 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.713 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.713 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.713 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.713 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.713 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.713 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.713 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.715 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.715 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.715 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.715 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.716 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.716 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.716 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.716 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.716 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2023-07-14T16:37:24.716 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.717 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.717 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.717 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.717 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.717 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.717 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.717 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.717 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.718 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.718 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.718 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.718 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.718 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.718 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.718 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.718 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.719 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.719 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.719 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.719 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.719 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.720 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.720 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.720 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.720 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.720 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.720 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.720 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.721 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.722 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.722 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.722 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.723 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.723 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.723 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.723 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.723 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.723 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.723 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.723 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.724 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.724 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.724 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.724 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.724 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.725 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.725 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.725 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.725 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.725 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.725 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.726 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.726 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.726 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2023-07-14T16:37:24.726 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.726 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.726 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.726 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2023-07-14T16:37:24.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2023-07-14T16:37:24.727 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.727 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.727 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.727 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2023-07-14T16:37:24.727 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T16:37:24.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.728 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.728 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.728 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2023-07-14T16:37:24.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2023-07-14T16:37:24.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.729 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:24.729 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2023-07-14T16:37:24.729 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.729 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T16:37:24.729 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.730 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.730 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.730 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.730 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.730 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.731 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.731 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.731 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.731 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.731 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.731 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.731 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.732 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.732 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.732 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.732 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.732 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.732 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.733 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.733 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.733 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.733 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.733 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.734 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.734 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.734 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.734 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.735 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.736 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.736 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.736 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.736 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.736 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.736 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.737 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.737 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.737 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.737 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.738 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.738 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.738 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.738 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.738 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.738 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.738 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.739 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.739 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.739 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.739 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2023-07-14T16:37:24.739 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.739 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.740 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.740 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2023-07-14T16:37:24.740 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2023-07-14T16:37:24.740 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.740 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2023-07-14T16:37:24.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.741 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.741 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.741 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2023-07-14T16:37:24.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T16:37:24.741 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.741 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.743 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.743 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.743 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.743 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.743 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.743 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.743 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.743 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.744 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.744 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.744 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.744 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.745 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.745 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.745 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.745 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.745 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.745 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.746 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.746 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.746 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.746 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.746 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.746 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.747 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.747 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.747 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.747 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.747 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.747 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.747 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.747 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.749 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.749 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.749 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.749 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.749 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.749 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.749 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.749 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.750 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.750 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.750 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.750 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2023-07-14T16:37:24.750 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2023-07-14T16:37:24.751 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.751 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.751 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.751 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.751 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.751 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.751 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.751 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2023-07-14T16:37:24.752 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T16:37:24.752 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.752 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.752 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2023-07-14T16:37:24.752 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2023-07-14T16:37:24.752 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.752 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.753 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.753 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2023-07-14T16:37:24.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.753 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.753 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2023-07-14T16:37:24.753 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.754 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.754 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.754 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2023-07-14T16:37:24.754 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2023-07-14T16:37:24.754 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.754 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.754 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.754 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2023-07-14T16:37:24.755 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.755 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.755 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T16:37:24.755 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.755 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.755 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.756 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2023-07-14T16:37:24.756 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2023-07-14T16:37:24.756 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.756 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.756 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.756 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.756 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2023-07-14T16:37:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.757 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2023-07-14T16:37:24.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2023-07-14T16:37:24.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.758 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.758 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2023-07-14T16:37:24.758 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.758 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.758 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.759 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T16:37:24.759 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T16:37:24.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.759 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.759 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.759 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.759 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2023-07-14T16:37:24.760 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.760 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.760 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.760 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.760 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.761 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.761 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.761 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.761 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.761 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.762 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.762 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.762 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.762 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.762 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.762 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.762 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.762 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.763 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.763 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.763 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.763 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.763 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.763 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.764 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.764 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.764 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.764 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.764 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.764 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.765 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.765 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.765 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.765 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.766 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.766 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.766 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.766 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.766 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.766 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.766 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.766 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.767 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.767 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.767 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.767 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.767 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.767 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.767 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.768 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.768 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.768 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.768 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.768 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.768 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.768 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.769 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.769 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.769 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.769 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.769 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.769 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2023-07-14T16:37:24.769 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.770 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.770 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.770 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.770 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.770 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.770 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.770 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.770 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.771 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.771 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.771 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.771 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.771 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.772 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.772 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.772 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.772 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.772 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.773 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.773 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.773 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.773 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.773 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.773 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.773 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.773 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.774 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.774 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.774 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.774 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.774 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.774 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.774 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.775 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.775 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.775 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.775 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.775 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.775 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.775 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.775 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.776 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.776 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.776 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.776 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.776 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.776 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.776 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.778 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.778 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.778 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.778 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.779 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.779 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2023-07-14T16:37:24.779 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.779 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.779 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.780 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.780 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.780 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.780 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.780 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.780 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.781 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.781 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.781 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.781 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.781 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.781 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.781 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.781 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.782 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.782 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.782 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.783 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.783 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.783 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.783 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.784 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.784 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.784 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.784 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.784 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.785 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.785 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.785 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.785 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.785 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.785 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.785 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.787 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.787 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.787 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.787 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.787 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.787 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.787 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.787 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.788 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.788 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.789 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2023-07-14T16:37:24.789 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.789 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.789 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.789 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2023-07-14T16:37:24.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2023-07-14T16:37:24.790 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.790 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.790 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.790 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.790 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.790 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T16:37:24.791 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.791 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.791 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.791 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.792 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.792 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.792 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.792 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.792 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.792 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.792 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.793 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.793 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.793 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.793 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.793 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.793 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.793 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.793 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.794 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.794 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.794 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.794 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.794 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.794 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.794 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.795 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.795 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.795 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.795 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.795 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.796 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.796 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.797 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.797 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.797 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.797 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.797 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.797 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.797 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.798 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.798 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.798 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.798 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.799 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.799 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.800 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.800 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2023-07-14T16:37:24.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2023-07-14T16:37:24.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.800 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.800 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.800 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.801 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.801 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2023-07-14T16:37:24.801 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2023-07-14T16:37:24.801 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2023-07-14T16:37:24.801 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.801 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2023-07-14T16:37:24.802 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.802 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.802 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.802 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.802 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.802 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2023-07-14T16:37:24.803 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.803 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.803 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.803 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.804 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.804 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.804 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.804 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.804 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.804 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.804 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.804 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.805 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.805 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.805 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.805 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.805 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.805 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.805 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.806 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.806 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.806 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.806 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.806 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.807 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.807 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.807 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.807 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.807 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.808 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.808 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.808 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.808 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.808 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.808 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.809 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.810 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.810 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.810 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.810 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.810 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.810 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.810 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.811 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.811 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.811 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.811 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.811 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.811 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.811 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.811 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.812 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.812 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.812 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.812 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.812 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.812 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2023-07-14T16:37:24.812 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.812 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.813 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.813 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.813 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.813 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.813 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.814 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.814 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.814 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.814 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.814 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.814 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.815 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.815 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.815 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.815 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.818 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.818 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.818 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.818 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.818 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.818 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.819 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.819 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.819 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.819 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.819 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.819 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.820 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.820 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.820 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.821 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.821 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.821 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.821 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.822 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.822 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2023-07-14T16:37:24.822 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.822 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.822 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.822 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.823 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.823 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.823 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.823 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.823 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.824 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.824 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.824 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.824 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.824 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.824 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.824 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.824 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.825 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.825 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.825 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.825 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.825 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.825 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.825 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.825 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.826 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.826 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.826 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.826 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.826 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.826 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.827 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.827 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.827 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.827 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.827 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.827 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.827 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.828 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.828 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.828 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.828 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.828 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.828 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.828 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.828 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.829 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.829 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.829 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.829 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.829 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.829 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.829 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.830 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.830 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.830 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.830 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.830 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.830 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.831 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.831 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.831 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.831 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.831 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.831 INFO:teuthology.orchestra.run.smithi062.stdout: "noautoscale": { 2023-07-14T16:37:24.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "noautoscale", 2023-07-14T16:37:24.831 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.832 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.832 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.832 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.832 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "global autoscale flag", 2023-07-14T16:37:24.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T16:37:24.833 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.833 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.833 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.833 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:24.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:24.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.834 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:24.834 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.834 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.835 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.835 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2023-07-14T16:37:24.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2023-07-14T16:37:24.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:24.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.835 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2023-07-14T16:37:24.835 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2023-07-14T16:37:24.836 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2023-07-14T16:37:24.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T16:37:24.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.836 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.837 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.837 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.837 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.837 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2023-07-14T16:37:24.837 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.837 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.837 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.837 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2023-07-14T16:37:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2023-07-14T16:37:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.838 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.839 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T16:37:24.839 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.839 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.839 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2023-07-14T16:37:24.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2023-07-14T16:37:24.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.840 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.840 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.840 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.840 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.840 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.841 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.841 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.841 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.841 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.841 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.841 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.841 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.842 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.842 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.842 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.842 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.842 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.843 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.843 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.843 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.843 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.843 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.845 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.845 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.845 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.845 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.845 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.845 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.845 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.846 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.846 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.846 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.846 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.846 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.846 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.846 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.846 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.847 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.847 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.847 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.847 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.847 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.847 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.847 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.847 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.848 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.848 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.848 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.849 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.849 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.849 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.850 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.850 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2023-07-14T16:37:24.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2023-07-14T16:37:24.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.850 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.850 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2023-07-14T16:37:24.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.851 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.851 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2023-07-14T16:37:24.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.852 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:24.852 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:24.852 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:24.852 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.852 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.852 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:24.852 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.852 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2023-07-14T16:37:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.854 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.854 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2023-07-14T16:37:24.854 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.854 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.854 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.854 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2023-07-14T16:37:24.854 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2023-07-14T16:37:24.854 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.855 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.856 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.856 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.856 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2023-07-14T16:37:24.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2023-07-14T16:37:24.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.857 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.857 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.857 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.857 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.857 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T16:37:24.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T16:37:24.857 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.858 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.858 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.858 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.858 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.858 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.858 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.858 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.859 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.859 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.859 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.859 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.859 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.859 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.859 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.859 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.860 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.860 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.860 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.860 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.860 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.860 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.861 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.861 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.861 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.861 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.861 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.861 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.861 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.861 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.862 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.862 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.862 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.862 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.862 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.862 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.863 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.863 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.863 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.863 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.864 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.864 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.864 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.864 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.864 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.864 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.864 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.865 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.865 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.865 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.865 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.865 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.866 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.866 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.866 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.866 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.866 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.866 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.867 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.867 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2023-07-14T16:37:24.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2023-07-14T16:37:24.867 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.867 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.867 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.868 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.868 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.868 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.868 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.868 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.869 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T16:37:24.869 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T16:37:24.869 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.869 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.869 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.869 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2023-07-14T16:37:24.869 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.869 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.870 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.870 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.870 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.870 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.870 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2023-07-14T16:37:24.870 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2023-07-14T16:37:24.871 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:24.871 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.871 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.871 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2023-07-14T16:37:24.871 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.871 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.871 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.871 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.872 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.872 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:24.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:24.872 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.872 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2023-07-14T16:37:24.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.873 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T16:37:24.873 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.873 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.874 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.874 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:24.874 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:24.874 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.874 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.874 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.874 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2023-07-14T16:37:24.875 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T16:37:24.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.875 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.875 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.875 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.876 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2023-07-14T16:37:24.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2023-07-14T16:37:24.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.876 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.876 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.876 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2023-07-14T16:37:24.876 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.876 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.877 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.877 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.877 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.877 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2023-07-14T16:37:24.877 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2023-07-14T16:37:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2023-07-14T16:37:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2023-07-14T16:37:24.879 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2023-07-14T16:37:24.879 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.879 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.879 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.879 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.879 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2023-07-14T16:37:24.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2023-07-14T16:37:24.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.880 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.880 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.880 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2023-07-14T16:37:24.880 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2023-07-14T16:37:24.880 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2023-07-14T16:37:24.880 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.881 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.881 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.882 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.882 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2023-07-14T16:37:24.882 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.882 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.882 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.882 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.882 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.882 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.883 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.884 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.884 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.884 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.884 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.884 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.884 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.884 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.885 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.885 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.885 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.885 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.887 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.887 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.887 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.887 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.887 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.887 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.888 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.888 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.888 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.888 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.888 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.888 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.888 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.888 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.889 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.889 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.889 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.889 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.889 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.889 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.890 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.890 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.890 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.890 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.891 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.891 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.891 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2023-07-14T16:37:24.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2023-07-14T16:37:24.891 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.891 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.892 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.892 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:24.892 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.892 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.892 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.893 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.893 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.893 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2023-07-14T16:37:24.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T16:37:24.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.893 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.894 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.894 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.894 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.894 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.894 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.895 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2023-07-14T16:37:24.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2023-07-14T16:37:24.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.895 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.895 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.895 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.896 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.896 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.896 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.896 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.897 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2023-07-14T16:37:24.897 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.897 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.897 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.897 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2023-07-14T16:37:24.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2023-07-14T16:37:24.898 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.898 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.899 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.899 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.899 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.899 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2023-07-14T16:37:24.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2023-07-14T16:37:24.899 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.899 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.900 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.900 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.900 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.901 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.901 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.901 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.902 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.902 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.902 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.902 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.902 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.903 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.903 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.903 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.903 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.904 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.904 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.904 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.904 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.904 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.905 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.905 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.905 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.905 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.905 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.905 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.905 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.906 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.906 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.906 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.906 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.906 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.907 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.907 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.907 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.907 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.907 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.907 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.907 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.908 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.908 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.908 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.908 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.909 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.910 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.910 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:24.910 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:24.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.910 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.911 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.911 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.912 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.912 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.912 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:24.912 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:24.912 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.912 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.912 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.912 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.913 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.913 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.913 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.913 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.913 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.913 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.913 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.913 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.914 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.914 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.914 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.914 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2023-07-14T16:37:24.914 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.914 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.914 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.914 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.915 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.915 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.915 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.915 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.915 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.916 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.916 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.916 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.916 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.916 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.916 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.916 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.916 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.917 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.917 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.918 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.918 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.919 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.919 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.919 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.919 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.919 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.920 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.920 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.920 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.920 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.920 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.920 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.921 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.921 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.921 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.921 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.921 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.921 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.922 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.922 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.922 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.922 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.922 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.922 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.922 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.923 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.923 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.923 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.923 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.923 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.923 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.923 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.924 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2023-07-14T16:37:24.924 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.924 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.924 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.924 INFO:teuthology.orchestra.run.smithi062.stdout: "drive_group_interval": { 2023-07-14T16:37:24.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "drive_group_interval", 2023-07-14T16:37:24.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:24.925 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.925 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.925 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300.0", 2023-07-14T16:37:24.925 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.925 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T16:37:24.925 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.926 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.926 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.926 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.926 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.926 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.927 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.927 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.927 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.927 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.929 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.929 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.929 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.929 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.929 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.929 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.930 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.930 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.930 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.930 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.930 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.930 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.930 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.930 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.931 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.931 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.931 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.932 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.932 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.932 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.932 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.932 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.932 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.932 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.932 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.933 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.933 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.933 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.933 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.933 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.934 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.934 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.934 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.934 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.934 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.934 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.935 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2023-07-14T16:37:24.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2023-07-14T16:37:24.935 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.935 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.935 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.936 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2023-07-14T16:37:24.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.936 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.936 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T16:37:24.936 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.936 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.937 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.937 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.937 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.937 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.937 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.937 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2023-07-14T16:37:24.937 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.937 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.938 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.938 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.938 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.938 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.938 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.938 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.938 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.938 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.939 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.939 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.939 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.939 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.939 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.939 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.939 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.939 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.940 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.940 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.940 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.940 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.940 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.940 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.940 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.942 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.942 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.942 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.942 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.942 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.942 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.943 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.943 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.943 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.943 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.944 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.944 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.944 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.944 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.944 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.944 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.944 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.944 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.945 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.945 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.945 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.945 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.945 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.945 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.945 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.946 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.946 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.946 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.946 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.946 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.946 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.946 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2023-07-14T16:37:24.947 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2023-07-14T16:37:24.947 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.947 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.947 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.947 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.947 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.947 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.947 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.948 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.948 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.948 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.948 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.948 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.948 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2023-07-14T16:37:24.948 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2023-07-14T16:37:24.948 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.949 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.949 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.949 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2023-07-14T16:37:24.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.949 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.949 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.949 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.949 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.950 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.950 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.950 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.950 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2023-07-14T16:37:24.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2023-07-14T16:37:24.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.950 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.951 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.951 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.951 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.951 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.951 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.951 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.951 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2023-07-14T16:37:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2023-07-14T16:37:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.952 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.953 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.953 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.953 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.953 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.953 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.953 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.953 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.953 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2023-07-14T16:37:24.954 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2023-07-14T16:37:24.954 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:24.954 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.954 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.954 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.954 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.954 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.955 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.955 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.955 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.955 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2023-07-14T16:37:24.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2023-07-14T16:37:24.955 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.956 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.956 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.956 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.956 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.956 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.957 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.957 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.957 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.957 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2023-07-14T16:37:24.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2023-07-14T16:37:24.957 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.957 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.958 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.958 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.958 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.958 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.958 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.958 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.958 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.959 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.959 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.959 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2023-07-14T16:37:24.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2023-07-14T16:37:24.959 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.959 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.959 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.959 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:24.960 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.960 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.960 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.960 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.960 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.960 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.960 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.960 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2023-07-14T16:37:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2023-07-14T16:37:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:24.961 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2023-07-14T16:37:24.962 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.962 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.962 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.962 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.962 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.962 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.962 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2023-07-14T16:37:24.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2023-07-14T16:37:24.963 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.963 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.963 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.963 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.963 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.963 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.963 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.963 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.964 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.964 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.964 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.964 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.964 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2023-07-14T16:37:24.964 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2023-07-14T16:37:24.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.964 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.965 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.965 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.965 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.965 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.965 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.965 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.965 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.965 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2023-07-14T16:37:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2023-07-14T16:37:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:24.966 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2023-07-14T16:37:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2023-07-14T16:37:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2023-07-14T16:37:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.969 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T16:37:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2023-07-14T16:37:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2023-07-14T16:37:24.970 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.971 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.971 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.971 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.971 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.971 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.971 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.971 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2023-07-14T16:37:24.971 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.972 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.972 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.972 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.972 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.972 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.972 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.972 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.973 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.973 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.973 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.973 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.973 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.973 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.973 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.973 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.974 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.974 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.974 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.974 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.974 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.974 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.974 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.975 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.975 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.975 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.975 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.975 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.975 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.975 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.975 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.976 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.976 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.976 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.976 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.976 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.976 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.976 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.977 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.977 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.977 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.977 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.977 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.977 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.977 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.977 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.978 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.978 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.978 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.978 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.978 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.978 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.978 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.978 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.979 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.979 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.979 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.979 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.979 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.979 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.979 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.979 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.980 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.980 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.980 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.980 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.980 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.980 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.981 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.981 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.981 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.981 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.981 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.981 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2023-07-14T16:37:24.981 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.982 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.982 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.982 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.982 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.982 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.982 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.982 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.982 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.983 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.983 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.983 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.983 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.983 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.983 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.983 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.983 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.984 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.984 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.984 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.984 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.984 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.984 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.984 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.984 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.985 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.985 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.985 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.985 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.985 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.985 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.985 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.985 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.986 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.986 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.986 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.986 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.986 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.986 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.987 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.987 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.987 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.987 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.987 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.987 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.987 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.987 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.988 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.988 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.988 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.988 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.988 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.988 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.988 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.989 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.989 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.989 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.989 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.989 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.989 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.989 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.989 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.990 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.990 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.990 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.990 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.990 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.990 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.990 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.990 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.991 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:24.991 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.991 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:24.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2023-07-14T16:37:24.991 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:24.991 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:24.991 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:24.991 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:24.992 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:24.992 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.992 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.992 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.992 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:24.992 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.992 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.993 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.993 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.993 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.993 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.993 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.993 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.993 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.993 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:24.994 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.995 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.995 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.995 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.995 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.995 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.995 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:24.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.995 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.996 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.996 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.996 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.996 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:24.996 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:24.996 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:24.996 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.996 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.997 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:24.997 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.997 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:24.997 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:24.997 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:24.997 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:24.997 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:24.997 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:24.998 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:24.998 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:24.998 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:24.998 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:24.998 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:24.998 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:24.998 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:24.998 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:24.999 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:24.999 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:24.999 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:24.999 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:24.999 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:24.999 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:24.999 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:24.999 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.000 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.000 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.000 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.000 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.000 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.000 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.000 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.000 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.001 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.001 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2023-07-14T16:37:25.001 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.001 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.001 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.001 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2023-07-14T16:37:25.001 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2023-07-14T16:37:25.001 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.002 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.002 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.002 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T16:37:25.002 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.002 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.002 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.002 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.002 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.003 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.003 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.003 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.003 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.003 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.003 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.003 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.004 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2023-07-14T16:37:25.004 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.004 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.004 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.004 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.004 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.004 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.004 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.005 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.005 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.005 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.005 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.005 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.005 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.005 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.006 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.006 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.006 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.006 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.006 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.006 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.006 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.006 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.007 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.007 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.007 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.007 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.007 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.007 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.008 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.008 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.008 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.008 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.008 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.010 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.012 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.013 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2023-07-14T16:37:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.014 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2023-07-14T16:37:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2023-07-14T16:37:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.015 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T16:37:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2023-07-14T16:37:25.016 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2023-07-14T16:37:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.017 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T16:37:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2023-07-14T16:37:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2023-07-14T16:37:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.018 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T16:37:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.019 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2023-07-14T16:37:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2023-07-14T16:37:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.020 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T16:37:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2023-07-14T16:37:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2023-07-14T16:37:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.022 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T16:37:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.023 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2023-07-14T16:37:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2023-07-14T16:37:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.024 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2023-07-14T16:37:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2023-07-14T16:37:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.026 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2023-07-14T16:37:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2023-07-14T16:37:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.027 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T16:37:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2023-07-14T16:37:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2023-07-14T16:37:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.029 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2023-07-14T16:37:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2023-07-14T16:37:25.031 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2023-07-14T16:37:25.032 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2023-07-14T16:37:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2023-07-14T16:37:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.033 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2023-07-14T16:37:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2023-07-14T16:37:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.034 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.035 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2023-07-14T16:37:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2023-07-14T16:37:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.036 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.038 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.039 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.040 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.041 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.042 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.043 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.044 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.045 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2023-07-14T16:37:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2023-07-14T16:37:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.047 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2023-07-14T16:37:25.048 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2023-07-14T16:37:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2023-07-14T16:37:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2023-07-14T16:37:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.050 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T16:37:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.051 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2023-07-14T16:37:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.052 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.053 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.054 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.055 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.056 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.056 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.056 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.056 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.056 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.056 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.056 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.056 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.057 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.057 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.057 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.057 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.057 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.057 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.057 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.057 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.058 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.058 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.058 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.058 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.058 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.058 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.058 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.059 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.059 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.059 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.059 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.059 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.059 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.059 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.059 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.060 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.060 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.060 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.060 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.060 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.060 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.060 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.060 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.061 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.061 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.061 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.061 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.061 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.061 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.061 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.061 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.062 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.062 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.062 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.062 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.062 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2023-07-14T16:37:25.062 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.062 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.062 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.063 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.063 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.063 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.063 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.063 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.063 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.063 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.063 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.064 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.064 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.064 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.064 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.064 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.064 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.064 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.064 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.065 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.065 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.065 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.065 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.065 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.065 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.065 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.066 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.066 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.066 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.066 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.066 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.066 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.066 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.066 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.067 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.067 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.067 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.067 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.067 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.067 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.067 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.067 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.069 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.069 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.069 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.069 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.069 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.069 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.069 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.069 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.070 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.070 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.070 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.070 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.070 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.070 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.070 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.071 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.071 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.071 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.071 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.071 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2023-07-14T16:37:25.071 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2023-07-14T16:37:25.072 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.072 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2023-07-14T16:37:25.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.072 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.072 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.073 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T16:37:25.073 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.073 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.073 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.073 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2023-07-14T16:37:25.073 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2023-07-14T16:37:25.073 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T16:37:25.074 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.075 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.075 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.075 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.075 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.075 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.075 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2023-07-14T16:37:25.075 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.076 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.076 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.076 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2023-07-14T16:37:25.076 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2023-07-14T16:37:25.076 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.076 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.076 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.076 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2023-07-14T16:37:25.077 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.077 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.077 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.077 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.077 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.077 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.078 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2023-07-14T16:37:25.078 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2023-07-14T16:37:25.078 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.078 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.078 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.078 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.078 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.078 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.079 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.079 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.079 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.079 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.079 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.079 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.080 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.080 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.080 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.080 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.081 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.081 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.081 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.081 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.081 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.081 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.081 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.082 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.083 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.083 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.083 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.083 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.083 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.083 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.083 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.083 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.084 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.084 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.084 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.084 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.084 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.084 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.085 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.085 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.085 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.085 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.085 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.085 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.086 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.086 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.086 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.086 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.086 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.086 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.086 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.086 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.087 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.087 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.087 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.087 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.087 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.087 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.087 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.087 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.088 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.088 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.088 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.088 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.088 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.088 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.089 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.090 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.090 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.090 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2023-07-14T16:37:25.090 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2023-07-14T16:37:25.090 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.090 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.090 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.091 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.092 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2023-07-14T16:37:25.092 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2023-07-14T16:37:25.092 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.092 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.092 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.092 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2023-07-14T16:37:25.092 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.093 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.093 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.093 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.093 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.093 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.093 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.093 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.093 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2023-07-14T16:37:25.094 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2023-07-14T16:37:25.094 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.094 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.094 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.094 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T16:37:25.094 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.094 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.095 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.095 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.095 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.095 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.095 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.095 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.095 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.095 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.096 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:25.096 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.096 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.096 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24116, 2023-07-14T16:37:25.096 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "y", 2023-07-14T16:37:25.096 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540138322906710015, 2023-07-14T16:37:25.096 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2023-07-14T16:37:25.096 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2023-07-14T16:37:25.097 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.097 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.097 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.097 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.097 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T16:37:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.099 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.099 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.099 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.099 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.099 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.099 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.099 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.100 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.100 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.100 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.100 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.100 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.100 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.100 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.100 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.101 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.101 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.101 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.101 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.101 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.101 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.101 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.102 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.102 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.102 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.103 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.103 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.103 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.103 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.103 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.104 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.104 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.104 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.104 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.104 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.104 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.104 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.104 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.105 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.105 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.105 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.105 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.105 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.105 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.105 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.105 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.106 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.106 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.106 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.106 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.106 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.106 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.106 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.107 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.107 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.107 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.107 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.107 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.107 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.107 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.107 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.108 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2023-07-14T16:37:25.108 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2023-07-14T16:37:25.108 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.108 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.108 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.108 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.108 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.109 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.109 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.109 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2023-07-14T16:37:25.109 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.109 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.109 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.109 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.109 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2023-07-14T16:37:25.110 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2023-07-14T16:37:25.110 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.110 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.110 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.110 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2023-07-14T16:37:25.110 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.110 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.111 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2023-07-14T16:37:25.111 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.111 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.111 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.111 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.111 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2023-07-14T16:37:25.111 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2023-07-14T16:37:25.111 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.112 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.112 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.112 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.112 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.112 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.112 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.112 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2023-07-14T16:37:25.113 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.113 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.113 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.113 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2023-07-14T16:37:25.113 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2023-07-14T16:37:25.113 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.113 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.114 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.114 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.114 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.114 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.114 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.114 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2023-07-14T16:37:25.114 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.114 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.115 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.115 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2023-07-14T16:37:25.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2023-07-14T16:37:25.115 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.115 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.115 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.115 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2023-07-14T16:37:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2023-07-14T16:37:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2023-07-14T16:37:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2023-07-14T16:37:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.118 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.118 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2023-07-14T16:37:25.118 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.118 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.118 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.118 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2023-07-14T16:37:25.118 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2023-07-14T16:37:25.119 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.119 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.119 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.119 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.119 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.119 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.119 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.119 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T16:37:25.120 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.120 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.120 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.120 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2023-07-14T16:37:25.120 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2023-07-14T16:37:25.120 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.120 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.121 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.121 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.121 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.121 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.121 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2023-07-14T16:37:25.121 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.121 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.122 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.122 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.122 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.122 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.122 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2023-07-14T16:37:25.122 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.122 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2023-07-14T16:37:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2023-07-14T16:37:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.123 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.124 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.124 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.124 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T16:37:25.124 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.124 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.124 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.124 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2023-07-14T16:37:25.125 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2023-07-14T16:37:25.125 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.125 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.125 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.125 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2023-07-14T16:37:25.125 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.125 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.125 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.126 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T16:37:25.126 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T16:37:25.126 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.126 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2023-07-14T16:37:25.126 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2023-07-14T16:37:25.126 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.127 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.127 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2023-07-14T16:37:25.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2023-07-14T16:37:25.127 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.127 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T16:37:25.128 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T16:37:25.128 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.128 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.128 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2023-07-14T16:37:25.128 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2023-07-14T16:37:25.128 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.128 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.129 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.129 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2023-07-14T16:37:25.129 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:25.129 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2023-07-14T16:37:25.129 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.129 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T16:37:25.129 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.129 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.130 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.130 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2023-07-14T16:37:25.130 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2023-07-14T16:37:25.130 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.130 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.130 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.131 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T16:37:25.131 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.131 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.131 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.131 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T16:37:25.131 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T16:37:25.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.131 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.132 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2023-07-14T16:37:25.132 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2023-07-14T16:37:25.132 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2023-07-14T16:37:25.132 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2023-07-14T16:37:25.133 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2023-07-14T16:37:25.133 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.133 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2023-07-14T16:37:25.133 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T16:37:25.133 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.133 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.133 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2023-07-14T16:37:25.134 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2023-07-14T16:37:25.134 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.134 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.134 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2023-07-14T16:37:25.134 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.134 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.134 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.135 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T16:37:25.135 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T16:37:25.135 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.135 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.135 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2023-07-14T16:37:25.135 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2023-07-14T16:37:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2023-07-14T16:37:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2023-07-14T16:37:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.136 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T16:37:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T16:37:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.138 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.138 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.138 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.138 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.138 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.138 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.138 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.138 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.139 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.139 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.139 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.139 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.139 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.139 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.139 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.139 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.140 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.140 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.140 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.140 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.140 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.140 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.140 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.141 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.141 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.141 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.141 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.141 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.141 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.141 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.141 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.142 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.142 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.142 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.142 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.142 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.142 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.142 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.142 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.143 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.143 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.143 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.143 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.143 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.143 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.143 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.143 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.144 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.144 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.144 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.144 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.144 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.145 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.146 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.146 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.146 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.146 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2023-07-14T16:37:25.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2023-07-14T16:37:25.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.146 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.146 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.147 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.147 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.147 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.147 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.147 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T16:37:25.147 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.147 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.147 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2023-07-14T16:37:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2023-07-14T16:37:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2023-07-14T16:37:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.149 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.149 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.149 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2023-07-14T16:37:25.149 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2023-07-14T16:37:25.149 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap" 2023-07-14T16:37:25.149 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.149 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2023-07-14T16:37:25.150 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.150 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.150 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.150 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2023-07-14T16:37:25.150 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2023-07-14T16:37:25.150 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.150 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.151 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.151 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.151 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.151 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T16:37:25.151 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.151 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.152 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.152 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.152 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:25.152 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:25.152 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.152 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.152 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.152 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.153 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.153 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.153 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.153 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T16:37:25.153 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.153 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.153 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.154 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2023-07-14T16:37:25.154 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2023-07-14T16:37:25.154 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.154 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.154 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.154 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:25.154 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:25.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.155 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.155 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T16:37:25.155 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T16:37:25.155 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.155 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.155 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2023-07-14T16:37:25.155 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2023-07-14T16:37:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.156 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T16:37:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.157 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2023-07-14T16:37:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2023-07-14T16:37:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2023-07-14T16:37:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2023-07-14T16:37:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T16:37:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.159 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2023-07-14T16:37:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2023-07-14T16:37:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.160 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2023-07-14T16:37:25.161 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.161 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.161 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T16:37:25.161 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.161 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.161 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.161 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.162 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2023-07-14T16:37:25.162 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2023-07-14T16:37:25.162 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.162 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.162 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.162 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2023-07-14T16:37:25.162 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.162 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.163 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T16:37:25.163 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.163 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.163 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.163 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.163 INFO:teuthology.orchestra.run.smithi062.stdout: "alertmanager_web_password": { 2023-07-14T16:37:25.164 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alertmanager_web_password", 2023-07-14T16:37:25.164 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.164 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.164 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.164 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.164 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.164 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.164 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.165 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Alertmanager web password", 2023-07-14T16:37:25.165 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.165 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.165 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.165 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.165 INFO:teuthology.orchestra.run.smithi062.stdout: "alertmanager_web_user": { 2023-07-14T16:37:25.165 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alertmanager_web_user", 2023-07-14T16:37:25.165 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.166 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.166 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.166 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.166 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.166 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.166 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.166 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Alertmanager web user", 2023-07-14T16:37:25.166 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.167 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.167 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.167 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.167 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2023-07-14T16:37:25.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2023-07-14T16:37:25.167 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.167 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.167 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.168 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.168 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.168 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.168 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.168 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T16:37:25.168 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T16:37:25.168 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.168 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.169 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.169 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2023-07-14T16:37:25.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2023-07-14T16:37:25.169 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.169 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.169 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.169 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:25.169 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.170 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.170 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.170 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T16:37:25.170 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.170 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.170 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.170 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.170 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2023-07-14T16:37:25.171 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T16:37:25.171 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.171 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.171 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.171 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2023-07-14T16:37:25.171 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.171 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.172 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.172 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T16:37:25.172 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.172 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.172 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.172 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.172 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2023-07-14T16:37:25.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2023-07-14T16:37:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T16:37:25.174 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.174 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.174 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.174 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.174 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2023-07-14T16:37:25.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2023-07-14T16:37:25.175 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.175 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.175 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.175 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.175 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.175 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.176 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.176 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T16:37:25.176 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.176 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.176 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.176 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.176 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2023-07-14T16:37:25.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2023-07-14T16:37:25.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.177 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.177 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.177 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.177 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.177 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.177 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T16:37:25.178 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.178 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.178 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.178 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.178 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2023-07-14T16:37:25.178 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2023-07-14T16:37:25.178 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.179 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.179 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T16:37:25.179 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.179 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.179 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.179 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:25.179 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.180 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.180 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.180 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2023-07-14T16:37:25.180 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2023-07-14T16:37:25.180 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.180 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T16:37:25.181 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.181 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.181 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.181 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2023-07-14T16:37:25.181 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.181 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2023-07-14T16:37:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2023-07-14T16:37:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.182 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T16:37:25.183 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.183 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.183 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.183 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2023-07-14T16:37:25.183 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.183 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.183 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.184 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2023-07-14T16:37:25.184 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2023-07-14T16:37:25.184 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.184 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T16:37:25.184 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.184 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.185 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:25.185 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.185 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.185 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.185 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.185 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2023-07-14T16:37:25.185 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2023-07-14T16:37:25.186 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.186 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.186 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.186 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T16:37:25.186 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.186 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.186 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.186 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2023-07-14T16:37:25.187 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.187 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.187 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.187 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.187 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2023-07-14T16:37:25.187 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2023-07-14T16:37:25.187 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.187 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.188 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T16:37:25.188 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.188 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.188 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.188 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2023-07-14T16:37:25.188 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.188 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.189 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.189 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.189 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2023-07-14T16:37:25.189 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2023-07-14T16:37:25.189 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.189 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.189 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.189 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T16:37:25.190 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.190 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.190 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.190 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2023-07-14T16:37:25.190 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.190 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.190 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.190 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.191 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2023-07-14T16:37:25.191 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2023-07-14T16:37:25.191 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.191 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.191 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.191 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T16:37:25.191 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2023-07-14T16:37:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.192 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2023-07-14T16:37:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2023-07-14T16:37:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T16:37:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.193 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.194 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2023-07-14T16:37:25.194 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.194 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.194 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.194 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2023-07-14T16:37:25.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2023-07-14T16:37:25.194 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.195 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.195 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.195 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T16:37:25.195 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.195 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.195 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.195 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2023-07-14T16:37:25.195 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.196 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.196 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.196 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.196 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2023-07-14T16:37:25.196 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2023-07-14T16:37:25.196 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.196 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.197 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.197 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T16:37:25.197 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.197 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.197 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.197 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:25.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.198 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.198 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.198 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2023-07-14T16:37:25.198 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2023-07-14T16:37:25.198 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.198 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.198 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.198 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T16:37:25.199 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.199 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.199 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.199 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:25.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.199 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.199 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.200 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2023-07-14T16:37:25.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2023-07-14T16:37:25.200 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.200 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.200 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.200 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T16:37:25.200 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.200 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.201 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2023-07-14T16:37:25.201 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.201 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.201 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.201 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.201 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2023-07-14T16:37:25.201 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2023-07-14T16:37:25.202 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.202 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.202 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.202 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T16:37:25.202 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.202 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.202 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.202 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2023-07-14T16:37:25.203 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.203 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.203 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.203 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2023-07-14T16:37:25.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2023-07-14T16:37:25.203 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.204 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.204 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.204 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.204 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.204 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.204 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.204 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T16:37:25.204 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2023-07-14T16:37:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2023-07-14T16:37:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T16:37:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.207 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2023-07-14T16:37:25.207 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T16:37:25.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2023-07-14T16:37:25.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.208 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.208 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.208 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T16:37:25.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.208 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.208 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2023-07-14T16:37:25.209 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2023-07-14T16:37:25.209 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.209 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2023-07-14T16:37:25.209 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.209 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.210 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.210 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T16:37:25.210 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.210 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.210 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.210 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2023-07-14T16:37:25.210 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2023-07-14T16:37:25.211 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.211 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.211 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.211 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2023-07-14T16:37:25.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.211 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.211 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.211 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2023-07-14T16:37:25.212 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.212 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.212 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.212 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.212 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2023-07-14T16:37:25.212 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2023-07-14T16:37:25.212 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.212 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.213 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.213 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.213 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.213 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.213 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.213 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T16:37:25.213 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.214 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.214 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2023-07-14T16:37:25.214 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2023-07-14T16:37:25.214 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.214 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.214 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.214 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.215 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.215 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.215 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.215 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2023-07-14T16:37:25.215 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.215 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.215 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.216 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.216 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2023-07-14T16:37:25.216 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2023-07-14T16:37:25.216 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.216 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.216 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.216 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:25.217 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.217 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.217 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.217 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2023-07-14T16:37:25.217 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.217 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.217 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.218 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.218 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.218 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.218 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.218 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.218 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.218 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.218 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.219 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.219 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.219 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.219 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.219 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.219 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.219 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.219 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.220 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.220 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.220 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.220 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.220 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.220 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2023-07-14T16:37:25.220 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2023-07-14T16:37:25.221 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.221 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.221 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.221 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.221 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.221 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.221 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.221 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T16:37:25.222 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.222 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.222 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.222 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.222 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.222 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.222 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.222 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.223 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.223 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.223 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.223 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.223 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.223 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T16:37:25.223 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.224 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.224 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.224 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.224 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.224 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.224 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.225 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.225 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.225 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.225 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.225 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.225 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.225 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.226 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.226 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.226 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.226 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.226 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.226 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.226 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.227 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.227 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.227 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.227 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.227 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.227 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.227 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.227 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.228 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.228 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.228 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.228 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.228 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.228 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T16:37:25.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T16:37:25.229 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.229 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.229 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.229 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.229 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.229 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.229 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.230 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T16:37:25.230 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.230 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.230 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.230 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T16:37:25.230 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T16:37:25.230 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.231 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.231 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2023-07-14T16:37:25.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.231 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T16:37:25.231 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.232 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.232 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2023-07-14T16:37:25.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2023-07-14T16:37:25.232 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.232 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.232 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.233 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.233 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.233 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.233 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2023-07-14T16:37:25.233 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.233 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.233 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.234 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2023-07-14T16:37:25.234 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2023-07-14T16:37:25.234 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.234 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.234 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.234 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.234 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.235 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.235 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.235 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T16:37:25.235 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.235 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.235 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2023-07-14T16:37:25.236 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2023-07-14T16:37:25.236 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.236 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.236 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.236 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.236 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.236 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2023-07-14T16:37:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2023-07-14T16:37:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2023-07-14T16:37:25.238 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.238 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.238 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.238 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2023-07-14T16:37:25.238 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.238 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.238 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.238 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2023-07-14T16:37:25.239 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2023-07-14T16:37:25.239 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.239 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T16:37:25.239 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.239 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2023-07-14T16:37:25.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2023-07-14T16:37:25.240 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.240 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.240 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.240 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T16:37:25.240 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.240 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.241 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.241 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T16:37:25.241 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.241 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.241 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.241 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.241 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_web_password": { 2023-07-14T16:37:25.241 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_web_password", 2023-07-14T16:37:25.242 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.242 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.242 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.242 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.242 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.242 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.242 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.242 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus web password", 2023-07-14T16:37:25.243 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.243 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.243 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.243 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_web_user": { 2023-07-14T16:37:25.243 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_web_user", 2023-07-14T16:37:25.243 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.243 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.244 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.244 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.244 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus web user", 2023-07-14T16:37:25.244 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.244 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2023-07-14T16:37:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2023-07-14T16:37:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.246 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.246 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.246 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.246 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T16:37:25.246 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.246 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.246 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.246 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.247 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2023-07-14T16:37:25.247 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2023-07-14T16:37:25.247 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.247 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.247 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.247 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.247 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.247 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.248 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.248 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T16:37:25.248 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.248 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2023-07-14T16:37:25.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2023-07-14T16:37:25.249 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.249 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.249 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.249 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.249 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.249 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.249 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.249 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T16:37:25.250 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.250 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.250 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.250 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.250 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2023-07-14T16:37:25.250 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2023-07-14T16:37:25.250 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.250 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.251 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.251 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.251 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.251 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.251 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.251 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T16:37:25.251 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.251 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.252 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2023-07-14T16:37:25.252 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2023-07-14T16:37:25.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.252 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.252 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2023-07-14T16:37:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2023-07-14T16:37:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2023-07-14T16:37:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.255 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2023-07-14T16:37:25.255 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.255 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.255 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.255 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.255 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2023-07-14T16:37:25.255 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2023-07-14T16:37:25.256 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.256 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.256 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.256 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.256 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.256 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.256 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.257 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T16:37:25.257 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.257 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2023-07-14T16:37:25.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2023-07-14T16:37:25.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.258 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.258 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.258 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.258 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.258 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.258 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.258 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T16:37:25.258 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.259 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.259 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.259 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2023-07-14T16:37:25.259 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2023-07-14T16:37:25.259 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.259 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.259 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.260 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.260 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.260 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.260 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T16:37:25.260 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.260 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.260 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.261 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2023-07-14T16:37:25.261 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2023-07-14T16:37:25.261 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.261 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.261 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.261 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.262 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.262 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T16:37:25.262 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.262 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.262 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.263 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T16:37:25.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.264 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2023-07-14T16:37:25.264 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2023-07-14T16:37:25.265 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.265 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.265 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.265 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.265 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.265 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.265 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.265 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T16:37:25.266 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.266 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.266 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.266 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.266 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.266 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.266 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2023-07-14T16:37:25.267 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.267 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.267 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.267 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.268 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.268 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.268 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.268 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.268 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.268 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.268 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.268 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.270 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.270 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.270 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.270 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.270 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.270 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.270 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.272 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:24 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2425351228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:37:25.272 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:24 smithi081 ceph-mon[127795]: pgmap v1010: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 242 KiB/s wr, 230 op/s 2023-07-14T16:37:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:24 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2425351228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:37:25.272 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:24 smithi164 ceph-mon[97427]: pgmap v1010: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 242 KiB/s wr, 230 op/s 2023-07-14T16:37:25.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.275 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.276 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.276 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.276 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.277 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.277 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.277 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.277 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.277 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.277 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.277 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.278 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.278 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.278 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.278 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.278 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.278 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.278 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.279 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.279 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.279 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.279 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.279 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.279 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.279 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.280 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.280 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.280 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.280 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.280 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.280 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2023-07-14T16:37:25.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2023-07-14T16:37:25.281 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.281 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.281 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.281 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2023-07-14T16:37:25.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.281 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.281 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.281 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T16:37:25.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.282 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.282 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.282 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2023-07-14T16:37:25.282 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2023-07-14T16:37:25.282 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.282 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2023-07-14T16:37:25.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.283 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.283 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T16:37:25.283 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.283 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.284 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.284 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.284 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2023-07-14T16:37:25.284 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T16:37:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T16:37:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.286 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.286 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.286 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.286 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.286 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.286 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.287 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T16:37:25.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T16:37:25.287 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.287 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.287 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.287 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.287 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.287 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.288 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.288 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.288 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.288 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.288 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.288 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T16:37:25.288 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T16:37:25.289 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.289 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.289 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.289 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.289 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.290 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.290 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.290 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2023-07-14T16:37:25.290 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T16:37:25.290 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.290 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.291 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.291 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.291 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.291 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.291 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.291 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.291 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.292 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.292 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.292 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T16:37:25.292 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T16:37:25.292 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.292 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.293 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.293 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.293 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.293 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.293 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.294 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.294 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T16:37:25.294 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T16:37:25.294 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.294 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.294 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.294 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.295 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.295 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.295 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.295 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.296 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T16:37:25.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T16:37:25.296 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.296 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.296 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.297 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.297 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.297 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.297 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.298 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.298 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T16:37:25.298 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T16:37:25.298 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.298 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.298 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.298 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.299 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.299 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.299 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.299 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.299 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.299 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.300 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T16:37:25.300 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T16:37:25.300 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.300 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.300 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.301 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.301 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.301 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.301 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.301 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.301 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.301 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T16:37:25.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T16:37:25.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.302 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.302 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.302 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.302 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.302 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.303 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.303 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.303 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.303 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T16:37:25.303 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T16:37:25.304 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.304 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.304 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.304 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.304 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.304 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.304 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.304 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.305 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.305 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.305 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.305 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.305 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T16:37:25.305 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T16:37:25.305 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.305 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.306 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.306 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.306 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.306 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.306 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.306 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.306 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.306 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.307 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.307 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.307 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T16:37:25.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T16:37:25.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.307 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.307 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.307 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.308 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.308 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.308 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.308 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.308 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.308 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.309 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.309 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T16:37:25.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T16:37:25.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.309 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.309 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.309 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.309 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.310 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.310 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.310 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.310 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.310 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.310 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T16:37:25.311 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T16:37:25.311 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.311 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.311 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.311 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.311 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.311 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.311 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.312 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.312 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.312 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.312 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.312 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.312 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T16:37:25.312 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T16:37:25.312 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.313 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.313 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.313 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.313 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.313 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.313 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.313 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.313 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.314 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.314 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.314 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2023-07-14T16:37:25.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2023-07-14T16:37:25.314 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.314 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.315 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.315 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.315 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.315 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.315 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.315 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.316 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T16:37:25.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T16:37:25.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.316 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.316 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.316 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.317 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.317 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.317 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.317 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.317 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.317 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.317 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.317 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T16:37:25.318 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T16:37:25.318 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.318 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.318 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.318 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.318 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.318 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.318 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.319 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.319 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.319 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.319 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.319 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.319 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T16:37:25.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T16:37:25.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.320 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.320 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.320 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.320 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.320 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.320 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.320 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.321 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.321 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.321 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T16:37:25.321 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T16:37:25.321 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.322 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.322 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.322 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.322 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.322 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.322 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.322 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.323 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.323 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T16:37:25.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T16:37:25.323 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.323 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.323 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.323 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.324 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.325 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T16:37:25.325 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T16:37:25.325 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.325 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.325 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.325 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.325 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.326 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.326 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.326 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.326 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.326 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T16:37:25.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T16:37:25.327 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.327 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.327 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.327 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.327 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.327 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.327 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.328 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.328 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.328 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.328 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.328 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T16:37:25.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T16:37:25.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.329 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.329 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.329 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.329 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.329 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.329 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.330 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.330 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.330 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.330 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T16:37:25.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T16:37:25.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.330 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.331 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.331 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.331 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.331 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.332 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T16:37:25.332 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T16:37:25.332 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.332 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.332 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.332 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.332 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.333 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.333 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.333 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.333 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.333 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.333 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.333 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T16:37:25.334 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T16:37:25.334 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.334 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.334 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.334 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.334 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.335 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.335 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.335 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.335 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.335 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T16:37:25.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T16:37:25.335 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.336 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.336 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.336 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.336 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.337 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T16:37:25.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T16:37:25.337 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.337 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.337 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.338 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.338 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.338 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.338 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.339 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T16:37:25.339 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T16:37:25.339 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.339 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.339 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.339 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.339 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.340 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T16:37:25.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T16:37:25.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.341 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.341 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.341 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.341 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.341 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.341 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T16:37:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T16:37:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.343 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T16:37:25.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.343 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.343 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.343 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.344 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.344 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.344 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T16:37:25.344 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T16:37:25.344 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.344 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.344 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.345 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.345 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.345 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.345 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.345 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.345 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.346 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T16:37:25.346 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T16:37:25.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.346 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.346 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.346 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2023-07-14T16:37:25.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.347 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.347 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.347 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.347 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.347 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.348 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T16:37:25.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T16:37:25.348 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.348 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.348 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.348 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2023-07-14T16:37:25.348 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.348 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.349 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.349 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.349 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.349 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T16:37:25.349 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T16:37:25.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.350 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.350 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.350 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.350 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.350 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.351 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.351 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T16:37:25.351 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T16:37:25.351 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.351 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.352 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.352 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.352 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.352 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.352 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.353 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.353 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T16:37:25.353 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T16:37:25.353 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.353 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.353 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.353 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.354 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.354 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.354 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.354 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.354 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.354 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.355 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.355 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T16:37:25.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T16:37:25.355 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.355 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.355 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.355 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.355 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.356 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.356 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.356 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.356 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.356 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T16:37:25.357 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T16:37:25.357 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.357 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.357 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.357 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.357 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.357 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.357 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.358 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.358 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.358 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.358 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.358 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.358 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T16:37:25.358 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T16:37:25.358 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.359 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.359 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.359 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.359 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.359 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.359 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.359 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.359 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.360 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.360 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.360 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T16:37:25.360 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T16:37:25.360 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.360 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.361 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.361 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:25.361 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.361 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.361 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.361 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.361 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.362 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2023-07-14T16:37:25.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2023-07-14T16:37:25.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.362 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.362 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.362 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.363 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.363 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.363 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.363 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.363 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.363 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.363 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.363 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.364 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2023-07-14T16:37:25.364 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2023-07-14T16:37:25.364 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.364 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.364 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.364 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.364 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.364 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.365 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.365 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.365 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.365 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.365 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.365 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2023-07-14T16:37:25.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2023-07-14T16:37:25.366 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.366 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.366 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.366 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.366 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.366 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.366 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.366 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2023-07-14T16:37:25.367 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.367 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.367 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2023-07-14T16:37:25.367 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2023-07-14T16:37:25.367 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.368 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.368 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.368 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2023-07-14T16:37:25.368 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.368 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.368 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.368 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.368 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.369 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.369 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2023-07-14T16:37:25.369 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2023-07-14T16:37:25.369 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.369 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.369 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.370 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.370 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.370 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.370 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.370 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.370 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.370 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.371 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.371 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.371 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.371 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.371 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.371 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.371 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.372 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.372 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.372 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.372 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.372 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.372 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.372 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.372 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.373 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.373 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.373 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.373 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.373 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.373 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.373 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.374 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.374 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.374 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.374 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.374 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.374 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.374 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.374 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.376 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.376 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.376 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.376 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.376 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.376 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.376 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.376 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.377 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.377 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.377 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.377 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.377 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.377 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.377 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.377 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.378 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.378 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.378 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.378 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.378 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.378 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.378 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.379 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.379 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.379 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.379 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.379 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.380 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.380 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2023-07-14T16:37:25.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2023-07-14T16:37:25.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.381 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.381 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2023-07-14T16:37:25.381 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.381 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.381 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2023-07-14T16:37:25.382 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T16:37:25.382 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.382 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.382 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.382 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.382 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.382 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.382 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.383 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.383 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.383 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.383 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.383 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:25.383 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:25.383 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.384 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2023-07-14T16:37:25.384 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.384 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.384 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.384 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.384 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.385 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.385 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.385 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:25.385 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:25.385 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.385 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.385 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.386 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2023-07-14T16:37:25.386 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.386 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.386 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.386 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2023-07-14T16:37:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2023-07-14T16:37:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2023-07-14T16:37:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2023-07-14T16:37:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2023-07-14T16:37:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2023-07-14T16:37:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2023-07-14T16:37:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2023-07-14T16:37:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2023-07-14T16:37:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2023-07-14T16:37:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2023-07-14T16:37:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2023-07-14T16:37:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2023-07-14T16:37:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2023-07-14T16:37:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2023-07-14T16:37:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2023-07-14T16:37:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2023-07-14T16:37:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2023-07-14T16:37:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2023-07-14T16:37:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2023-07-14T16:37:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.400 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.400 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.400 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.400 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.400 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.400 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.401 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.401 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.401 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.401 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.401 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.402 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.402 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.402 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.402 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.402 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.403 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.403 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.403 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.403 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.403 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.403 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.403 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.403 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.404 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.404 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.404 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.404 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.404 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.404 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.404 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.404 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.405 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.405 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.405 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.405 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.405 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.405 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.405 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.405 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.406 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.406 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.406 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.406 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.406 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.406 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.407 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.407 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.408 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2023-07-14T16:37:25.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2023-07-14T16:37:25.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2023-07-14T16:37:25.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.408 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.409 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T16:37:25.409 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.409 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.409 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.409 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.409 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2023-07-14T16:37:25.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2023-07-14T16:37:25.410 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2023-07-14T16:37:25.410 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.410 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.410 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T16:37:25.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.411 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.411 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2023-07-14T16:37:25.411 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2023-07-14T16:37:25.411 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.411 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.412 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.412 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2023-07-14T16:37:25.412 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.412 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.412 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.412 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2023-07-14T16:37:25.412 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.413 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.413 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.413 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.413 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2023-07-14T16:37:25.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2023-07-14T16:37:25.413 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.413 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.413 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.414 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2023-07-14T16:37:25.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.414 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.414 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T16:37:25.414 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.414 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.415 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.416 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.416 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T16:37:25.416 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.416 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.418 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T16:37:25.418 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.418 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.418 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.418 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.418 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2023-07-14T16:37:25.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2023-07-14T16:37:25.419 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.419 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.419 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.419 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2023-07-14T16:37:25.419 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.419 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.419 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T16:37:25.420 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.420 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.420 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.420 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.420 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.420 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.420 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.420 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2023-07-14T16:37:25.421 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.421 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.421 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.421 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.421 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.421 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.421 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.422 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.422 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.422 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.422 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.423 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.423 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.423 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.423 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.423 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.423 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.424 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.424 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.424 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.424 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.424 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.425 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.425 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.425 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.425 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.431 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.431 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.432 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.432 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.432 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.432 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.432 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.432 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.434 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.434 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.434 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.435 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.435 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.436 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.436 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2023-07-14T16:37:25.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2023-07-14T16:37:25.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.436 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.436 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2023-07-14T16:37:25.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.437 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.437 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.437 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.437 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.438 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2023-07-14T16:37:25.438 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2023-07-14T16:37:25.438 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2023-07-14T16:37:25.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.438 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.439 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.439 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.439 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:25.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:25.440 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.440 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.440 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.440 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:25.440 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.440 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.441 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.441 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.441 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.441 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.441 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.441 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.442 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2023-07-14T16:37:25.442 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2023-07-14T16:37:25.442 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2023-07-14T16:37:25.442 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.442 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2023-07-14T16:37:25.442 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2023-07-14T16:37:25.442 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.443 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.443 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2023-07-14T16:37:25.443 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.443 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.443 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T16:37:25.443 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.443 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.444 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.444 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2023-07-14T16:37:25.444 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2023-07-14T16:37:25.444 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.444 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.444 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.444 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2023-07-14T16:37:25.445 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.445 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.445 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T16:37:25.445 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.445 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.445 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.445 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.446 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2023-07-14T16:37:25.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2023-07-14T16:37:25.446 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.446 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.446 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.446 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.446 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.446 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.447 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.447 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2023-07-14T16:37:25.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.447 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.447 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.447 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.448 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.448 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.448 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2023-07-14T16:37:25.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2023-07-14T16:37:25.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.449 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T16:37:25.449 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.449 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.449 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.451 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.451 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.451 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.451 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.451 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.451 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.451 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.451 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.453 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.453 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.453 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.453 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.453 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.453 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.453 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.453 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.455 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.455 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.455 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.455 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.455 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.455 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.455 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.455 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.456 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.456 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.456 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.456 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.456 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.456 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.457 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.457 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.457 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.457 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.457 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.457 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.457 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.457 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.458 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.458 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.458 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.458 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.458 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2023-07-14T16:37:25.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2023-07-14T16:37:25.458 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.458 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.459 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2023-07-14T16:37:25.459 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.459 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.460 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2023-07-14T16:37:25.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2023-07-14T16:37:25.460 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.460 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.460 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.460 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2023-07-14T16:37:25.461 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.461 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.461 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2023-07-14T16:37:25.461 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.461 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.461 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.462 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2023-07-14T16:37:25.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2023-07-14T16:37:25.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.462 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.462 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2023-07-14T16:37:25.462 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.462 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.463 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.463 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T16:37:25.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.463 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.463 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.463 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2023-07-14T16:37:25.463 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2023-07-14T16:37:25.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.464 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.464 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:25.464 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:25.464 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2023-07-14T16:37:25.464 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.464 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T16:37:25.465 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.465 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.465 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.465 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.465 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2023-07-14T16:37:25.465 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2023-07-14T16:37:25.465 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.465 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.466 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.466 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.466 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.466 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.466 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.466 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2023-07-14T16:37:25.466 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.466 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2023-07-14T16:37:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2023-07-14T16:37:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2023-07-14T16:37:25.468 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.468 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T16:37:25.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.469 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.469 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.469 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.469 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.469 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2023-07-14T16:37:25.469 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.469 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.469 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.470 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.470 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.470 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.470 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.470 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.470 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.470 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.471 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.471 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.471 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.471 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.471 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.471 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.471 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.472 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.472 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.472 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.472 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.472 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.473 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.473 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.473 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.474 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.474 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.474 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.474 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.474 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.475 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.475 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.475 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.475 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.475 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.476 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.476 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.476 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.476 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.476 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.476 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.476 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.477 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.477 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.477 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.477 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.477 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.477 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.478 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.478 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.478 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.478 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.478 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.478 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.478 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.479 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.479 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.479 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2023-07-14T16:37:25.479 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.479 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.479 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.480 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.480 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.480 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.480 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.480 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.480 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.480 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.481 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.481 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.481 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.481 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.481 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.481 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.481 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.481 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.482 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.482 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.482 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.482 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.482 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.482 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.484 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.484 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.484 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.484 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.484 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.485 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.485 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.485 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.485 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.485 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.485 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.487 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.487 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.487 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.487 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.487 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.488 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.488 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.488 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.488 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.488 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.488 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.488 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.488 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2023-07-14T16:37:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2023-07-14T16:37:25.489 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2023-07-14T16:37:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2023-07-14T16:37:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.490 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T16:37:25.491 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.491 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.491 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.491 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2023-07-14T16:37:25.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2023-07-14T16:37:25.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.491 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.492 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.492 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.492 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2023-07-14T16:37:25.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.492 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.492 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T16:37:25.492 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.492 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.493 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.493 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.493 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.493 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.493 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.493 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.493 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.494 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.494 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.494 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.494 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.494 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.494 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.494 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.494 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.495 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.495 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.495 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.495 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.495 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.495 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.495 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.496 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.496 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.496 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.496 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.497 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.497 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.497 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.497 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.498 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.498 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.498 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.498 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.498 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.498 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.498 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.499 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.499 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.499 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.499 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.499 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.499 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.499 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.499 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.500 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.500 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.500 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.500 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.500 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.501 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.501 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.501 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.501 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.501 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.501 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.501 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.502 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.502 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.502 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2023-07-14T16:37:25.502 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.502 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.502 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.503 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2023-07-14T16:37:25.503 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2023-07-14T16:37:25.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.503 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.503 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2023-07-14T16:37:25.503 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.503 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.504 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.504 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2023-07-14T16:37:25.504 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T16:37:25.504 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.504 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.504 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.505 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.505 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.505 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.505 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.505 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.505 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.505 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.505 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.506 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.506 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.506 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.506 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.506 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.506 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.506 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.506 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.507 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.507 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.507 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.507 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.507 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.507 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.507 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.507 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.508 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.508 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.508 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.508 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.508 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.508 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.508 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.508 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.509 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.509 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.509 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.509 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.509 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.509 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.509 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.509 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.510 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.510 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.510 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.510 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.510 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.510 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.510 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.510 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.511 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.511 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.511 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.511 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.511 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.511 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.511 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.512 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.512 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.512 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.512 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.512 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.512 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.513 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.513 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.513 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.513 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.513 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.513 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2023-07-14T16:37:25.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2023-07-14T16:37:25.514 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.514 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.514 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.514 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.514 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.515 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.515 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2023-07-14T16:37:25.515 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T16:37:25.515 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.515 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.516 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2023-07-14T16:37:25.516 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2023-07-14T16:37:25.516 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.516 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.516 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.516 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2023-07-14T16:37:25.516 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.517 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.517 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.517 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2023-07-14T16:37:25.517 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.517 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.517 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.517 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.517 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2023-07-14T16:37:25.518 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2023-07-14T16:37:25.518 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.518 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.518 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.518 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2023-07-14T16:37:25.518 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.518 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.518 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.519 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T16:37:25.519 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.519 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.519 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.519 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2023-07-14T16:37:25.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2023-07-14T16:37:25.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2023-07-14T16:37:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.521 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.521 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.521 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.521 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2023-07-14T16:37:25.521 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2023-07-14T16:37:25.521 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.521 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.521 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.522 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2023-07-14T16:37:25.522 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.522 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.522 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.522 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T16:37:25.522 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T16:37:25.522 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.522 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.523 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.523 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.523 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.523 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2023-07-14T16:37:25.523 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.523 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.524 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.525 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.525 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.525 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.525 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.525 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.525 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.526 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.526 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.526 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.526 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.526 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.526 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.527 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.527 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.527 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.527 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.527 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.527 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.527 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.527 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.528 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.528 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.528 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.528 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.528 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.528 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.528 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.529 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.529 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.529 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.529 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.529 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.529 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.529 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.529 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.530 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.530 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.530 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.530 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.530 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.530 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.530 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.530 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.531 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.531 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.531 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.531 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.531 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.532 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.532 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.532 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.532 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.532 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.533 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.533 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.533 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2023-07-14T16:37:25.533 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.533 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.533 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.533 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.534 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.534 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.534 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.534 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.534 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.534 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.535 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.535 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.535 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.535 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.535 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.535 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.535 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.535 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.536 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.536 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.536 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.536 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.536 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.536 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.536 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.536 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.537 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.537 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.537 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.537 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.537 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.537 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.537 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.539 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.539 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.539 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.539 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.539 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.539 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.539 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.539 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.540 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.540 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.540 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.540 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.540 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.540 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.540 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.541 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.541 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.541 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.541 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.541 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.542 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.542 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.542 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.542 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.542 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.542 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.543 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2023-07-14T16:37:25.543 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.543 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.543 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.543 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.544 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.544 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.544 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.544 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.544 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.544 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.544 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.545 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.545 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.545 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.545 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.545 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.545 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.546 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.546 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.546 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.546 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.547 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.547 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.547 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.547 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.547 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.547 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.547 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.548 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.548 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.548 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.548 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.550 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.550 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.550 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.550 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.550 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.550 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.550 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.551 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.551 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.551 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.551 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.551 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.551 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.552 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.552 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.552 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.552 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.552 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.552 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.553 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.553 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.553 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.553 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2023-07-14T16:37:25.553 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.553 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.553 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.553 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2023-07-14T16:37:25.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2023-07-14T16:37:25.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.554 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.554 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.554 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.554 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.554 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.554 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.555 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T16:37:25.555 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.555 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.555 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.555 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.555 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.555 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.556 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.556 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.556 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.556 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.556 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.556 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.556 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.556 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.557 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.557 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.557 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.557 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.557 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.557 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.557 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.557 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.558 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.558 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.558 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.558 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.558 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.558 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.558 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.559 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.559 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.559 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.559 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.559 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.559 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.560 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.560 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.560 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.560 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.560 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.560 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.561 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.561 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.561 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.561 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.561 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.561 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.561 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.561 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.562 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.562 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.562 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.562 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.562 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.562 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.563 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.564 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.564 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.564 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.564 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2023-07-14T16:37:25.564 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2023-07-14T16:37:25.564 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.564 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.565 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.565 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.565 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.565 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.565 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.565 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2023-07-14T16:37:25.565 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2023-07-14T16:37:25.565 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2023-07-14T16:37:25.566 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.566 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2023-07-14T16:37:25.566 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.566 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.566 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.566 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.566 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.567 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2023-07-14T16:37:25.567 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.567 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.567 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.567 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.568 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.568 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.568 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.568 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.568 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.568 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.569 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.569 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.569 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.569 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.569 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.569 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.569 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.569 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.570 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.570 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.570 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.570 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.570 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.570 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.571 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.571 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.571 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.571 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.571 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.571 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.571 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.571 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.572 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.572 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.572 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.572 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.572 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.572 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.572 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.573 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.573 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.573 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.573 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.573 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.573 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.573 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.573 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.574 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.574 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.574 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.574 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.574 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.574 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.574 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.575 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.575 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.575 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.575 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.575 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.575 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.575 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.575 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.576 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.576 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.576 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.576 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.576 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.576 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.576 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2023-07-14T16:37:25.577 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.577 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.577 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.577 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.577 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.577 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.578 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.578 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.578 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.578 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.579 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.579 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.579 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.579 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.579 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.579 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.579 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.579 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.580 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.580 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.580 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.581 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.581 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.581 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.581 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.581 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.581 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.581 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.581 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.582 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.582 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.582 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.582 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.582 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.582 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.582 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.583 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.583 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.583 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.583 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.583 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.583 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.583 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.584 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.584 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.584 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.584 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.584 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.584 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.584 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.585 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.585 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.585 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.585 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.585 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.585 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.585 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.586 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.586 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.586 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.586 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.586 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2023-07-14T16:37:25.587 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.587 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.587 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.587 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.587 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.587 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.587 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.587 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.588 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.588 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.588 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.588 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.588 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.588 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.588 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.588 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.589 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.589 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.589 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.589 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.589 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.589 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.589 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.589 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.590 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.590 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.590 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.590 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.591 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.591 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.591 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.591 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.591 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.591 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.591 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.592 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.592 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.592 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.592 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.593 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.594 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.594 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.594 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.594 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.594 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.594 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.594 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.594 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.595 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.595 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.595 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.595 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.595 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.596 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.596 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.596 INFO:teuthology.orchestra.run.smithi062.stdout: "noautoscale": { 2023-07-14T16:37:25.596 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "noautoscale", 2023-07-14T16:37:25.596 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.596 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.596 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.597 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.597 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.597 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.597 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.597 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "global autoscale flag", 2023-07-14T16:37:25.597 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T16:37:25.597 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.597 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.598 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:25.598 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:25.598 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.598 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.598 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.598 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.599 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.599 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.599 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.599 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.599 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.599 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2023-07-14T16:37:25.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2023-07-14T16:37:25.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.600 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.600 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2023-07-14T16:37:25.600 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2023-07-14T16:37:25.600 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.600 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2023-07-14T16:37:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T16:37:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.601 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2023-07-14T16:37:25.602 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.602 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.602 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.602 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2023-07-14T16:37:25.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2023-07-14T16:37:25.602 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.603 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.603 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.603 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.603 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.603 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.603 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.603 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T16:37:25.603 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.604 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.604 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.604 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.604 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2023-07-14T16:37:25.604 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2023-07-14T16:37:25.604 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.605 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.605 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.605 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.605 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.605 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.606 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.606 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.606 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.606 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.606 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.607 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.607 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.607 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.607 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.607 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.607 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.607 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.607 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.608 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.608 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.608 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.608 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.608 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.608 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.608 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.608 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.609 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.609 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.609 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.609 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.609 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.609 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.609 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.609 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.610 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.610 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.610 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.610 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.610 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.610 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.610 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.611 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.611 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.611 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.611 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.611 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.611 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.611 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.611 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.612 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.612 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.612 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.612 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.612 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.612 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.612 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.613 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.613 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.613 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.613 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.613 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.613 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.614 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2023-07-14T16:37:25.615 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2023-07-14T16:37:25.615 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.615 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.615 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.615 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2023-07-14T16:37:25.615 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.615 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.615 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.616 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2023-07-14T16:37:25.616 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.616 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.616 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.616 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:25.616 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:25.617 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.617 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.617 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.617 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:25.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.617 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.617 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.617 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2023-07-14T16:37:25.618 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.618 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.618 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.618 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.618 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.618 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2023-07-14T16:37:25.619 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.619 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.619 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.619 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2023-07-14T16:37:25.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2023-07-14T16:37:25.619 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.619 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.619 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.620 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.620 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.620 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.621 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2023-07-14T16:37:25.621 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2023-07-14T16:37:25.621 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.621 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.621 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.621 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.622 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.622 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.622 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.622 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T16:37:25.622 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T16:37:25.622 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.622 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.623 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.623 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.623 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.623 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.624 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.624 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.624 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.624 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.624 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.624 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.624 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.626 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.626 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.627 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.627 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.628 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.628 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.629 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.629 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.629 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.629 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.629 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.629 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.629 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.629 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.630 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.630 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.630 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.630 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.630 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.631 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.631 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.631 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.631 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.631 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.631 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.632 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2023-07-14T16:37:25.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2023-07-14T16:37:25.632 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.632 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.632 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.632 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.633 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.633 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.633 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.633 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.633 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T16:37:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T16:37:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2023-07-14T16:37:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.634 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2023-07-14T16:37:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2023-07-14T16:37:25.636 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.636 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.636 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.636 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2023-07-14T16:37:25.636 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.636 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.636 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.636 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.638 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.638 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2023-07-14T16:37:25.638 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.638 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.638 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.638 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T16:37:25.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.639 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.639 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:25.639 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:25.639 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.639 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.639 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2023-07-14T16:37:25.640 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.640 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.640 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.640 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T16:37:25.640 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.640 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.641 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2023-07-14T16:37:25.641 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2023-07-14T16:37:25.641 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.641 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.641 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2023-07-14T16:37:25.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2023-07-14T16:37:25.643 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2023-07-14T16:37:25.643 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.643 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.643 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.643 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2023-07-14T16:37:25.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2023-07-14T16:37:25.644 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2023-07-14T16:37:25.644 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.644 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.644 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.644 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.644 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.645 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2023-07-14T16:37:25.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2023-07-14T16:37:25.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.645 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.645 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2023-07-14T16:37:25.645 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2023-07-14T16:37:25.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2023-07-14T16:37:25.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.646 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.646 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.646 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.646 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.646 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.647 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2023-07-14T16:37:25.647 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.647 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.647 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.647 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.648 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.649 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.649 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.649 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.649 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.649 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.649 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.649 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.650 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.650 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.650 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.650 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.650 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.650 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.651 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.651 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.651 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.652 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.652 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.652 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.652 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.653 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.653 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.653 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.653 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.653 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.654 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.654 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.654 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.654 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.654 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.654 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.654 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.655 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.655 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.655 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.655 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2023-07-14T16:37:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2023-07-14T16:37:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2023-07-14T16:37:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T16:37:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.659 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.659 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.659 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.659 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.659 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.659 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.659 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.659 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.660 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2023-07-14T16:37:25.660 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2023-07-14T16:37:25.660 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.660 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.660 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.660 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.661 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.661 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.661 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.661 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2023-07-14T16:37:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2023-07-14T16:37:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2023-07-14T16:37:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.664 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.664 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.664 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2023-07-14T16:37:25.664 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2023-07-14T16:37:25.664 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.665 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.665 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.665 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.666 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.666 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.666 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.666 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.666 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.667 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.667 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.667 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.667 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.667 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.667 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.668 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.668 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.668 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.668 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.668 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.669 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.669 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.669 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.669 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.669 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.669 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.670 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.670 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.670 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.670 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.670 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.670 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.671 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.671 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.671 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.671 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.672 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.672 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.672 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.672 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.672 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.672 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.672 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.673 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.673 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.674 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.674 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.674 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.675 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.675 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.675 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.675 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:25.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:25.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.676 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.676 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.676 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.676 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.676 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.677 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.677 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.677 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:25.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:25.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.678 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.678 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.678 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.678 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.678 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.678 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.678 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.679 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.679 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2023-07-14T16:37:25.679 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.679 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.679 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.681 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.681 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.681 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.681 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.681 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.681 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.681 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.681 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.682 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.682 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.682 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.682 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.682 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.682 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.682 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.682 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.683 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.683 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.683 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.683 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.683 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.683 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.683 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.684 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.684 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.684 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.684 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.684 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.684 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.684 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.685 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.685 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.685 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.685 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.685 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.685 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.685 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.685 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.686 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.686 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.686 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.686 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.686 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.687 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.687 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.687 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.687 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.688 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.688 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.688 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.688 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.688 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.688 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.688 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.689 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2023-07-14T16:37:25.689 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.689 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.689 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.689 INFO:teuthology.orchestra.run.smithi062.stdout: "drive_group_interval": { 2023-07-14T16:37:25.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "drive_group_interval", 2023-07-14T16:37:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300.0", 2023-07-14T16:37:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T16:37:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.691 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.692 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.692 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.692 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.692 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.692 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.692 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.692 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.692 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.694 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.694 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.694 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.695 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.696 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.696 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.696 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.697 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.697 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.697 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.697 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.698 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.698 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.698 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.698 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.698 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.698 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.698 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.698 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.699 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.699 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.699 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.699 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.699 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.699 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.699 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.699 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.700 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.700 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.700 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.700 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.700 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.701 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2023-07-14T16:37:25.701 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2023-07-14T16:37:25.701 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.701 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.701 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.701 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2023-07-14T16:37:25.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.701 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.702 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T16:37:25.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.702 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.702 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.702 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.703 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2023-07-14T16:37:25.703 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.703 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.703 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.703 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.704 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.704 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.704 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.704 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.704 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.704 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.705 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.705 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.705 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.705 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.705 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.705 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.705 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.706 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.706 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.706 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.706 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.707 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.707 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.707 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.707 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.707 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.708 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.708 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.708 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.708 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.709 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.709 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.709 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.709 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.709 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.710 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.710 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.710 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.710 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.710 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.711 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.711 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.711 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.711 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.711 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.711 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.711 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.712 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.712 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.712 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.712 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.712 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2023-07-14T16:37:25.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2023-07-14T16:37:25.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.713 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.713 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.713 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.714 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2023-07-14T16:37:25.714 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2023-07-14T16:37:25.714 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.714 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.714 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.715 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2023-07-14T16:37:25.715 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.716 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2023-07-14T16:37:25.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2023-07-14T16:37:25.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.716 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.716 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.716 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.716 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.717 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.717 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.717 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.717 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2023-07-14T16:37:25.718 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2023-07-14T16:37:25.718 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.718 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.718 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.718 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.718 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.718 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.719 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.719 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.719 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.719 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2023-07-14T16:37:25.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2023-07-14T16:37:25.720 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.720 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.720 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.720 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.720 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.720 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.720 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.721 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2023-07-14T16:37:25.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2023-07-14T16:37:25.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.721 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.722 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.722 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.722 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.722 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.723 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.723 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2023-07-14T16:37:25.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2023-07-14T16:37:25.723 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.723 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.723 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.723 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.724 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.724 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.724 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.725 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2023-07-14T16:37:25.725 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2023-07-14T16:37:25.725 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.725 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.725 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.725 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.726 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.726 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.726 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.726 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.726 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.726 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.726 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2023-07-14T16:37:25.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2023-07-14T16:37:25.727 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.727 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.727 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.727 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.727 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:25.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2023-07-14T16:37:25.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.728 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.728 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.728 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2023-07-14T16:37:25.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2023-07-14T16:37:25.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.729 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.729 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.729 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.729 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.729 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.730 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2023-07-14T16:37:25.730 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2023-07-14T16:37:25.730 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.730 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.730 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.731 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.731 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.731 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.731 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.731 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.731 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.731 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.732 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2023-07-14T16:37:25.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2023-07-14T16:37:25.732 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.732 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.732 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.732 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.732 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.733 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.734 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.734 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2023-07-14T16:37:25.734 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.734 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.734 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.734 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2023-07-14T16:37:25.734 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2023-07-14T16:37:25.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.735 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T16:37:25.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.736 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2023-07-14T16:37:25.736 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2023-07-14T16:37:25.736 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.737 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2023-07-14T16:37:25.737 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.738 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.738 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.738 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.738 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.738 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.739 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.739 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.739 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.739 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.739 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.739 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.740 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.740 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.740 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.740 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.740 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.740 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.740 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.740 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.741 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.741 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.741 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.741 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.741 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.741 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.742 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.742 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.742 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.742 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.742 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.743 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.743 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.743 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.743 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.743 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.744 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.745 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.745 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.745 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.745 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.745 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.745 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.745 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.746 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.746 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.746 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.746 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.746 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.746 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.746 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.747 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.747 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.747 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.747 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.747 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.747 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2023-07-14T16:37:25.748 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.748 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.748 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.748 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.748 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.748 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.749 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.749 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.749 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.749 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.749 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.749 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.749 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.750 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.750 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.750 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.750 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.750 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.750 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.750 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.751 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.751 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.751 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.751 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.751 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.751 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.752 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.752 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.752 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.752 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.752 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.752 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.752 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.753 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.753 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.753 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.753 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.753 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.753 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.754 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.754 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.754 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.754 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.754 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.754 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.754 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.754 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.755 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.755 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.755 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.755 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.755 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.755 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.756 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.756 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.756 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.756 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.756 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.756 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.756 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.757 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.757 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.757 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.757 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.757 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.757 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2023-07-14T16:37:25.758 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.758 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.758 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.758 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.758 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.759 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.759 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.759 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.759 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.759 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.759 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.759 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.759 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.760 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.760 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.760 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.760 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.760 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.760 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.760 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.761 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.761 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.761 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.761 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.761 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.761 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.762 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.762 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.762 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.762 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.762 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.762 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.763 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.763 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.763 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.763 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.763 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.763 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.764 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.765 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.765 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.765 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.765 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.766 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.767 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.767 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.767 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.767 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.767 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2023-07-14T16:37:25.768 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.768 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.768 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.768 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2023-07-14T16:37:25.768 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2023-07-14T16:37:25.768 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.769 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.769 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.769 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T16:37:25.769 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.769 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.769 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.769 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.770 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.770 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.770 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.770 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.770 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.771 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.771 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.771 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.771 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2023-07-14T16:37:25.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.772 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.772 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.772 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.772 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.773 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.773 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.773 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.773 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.773 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.773 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.774 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.774 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.774 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.774 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.774 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.774 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.774 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.775 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.775 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.775 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.775 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.775 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.776 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.776 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.776 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.776 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.776 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.776 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.776 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.777 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.777 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.777 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.777 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.777 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.778 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.778 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.778 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.778 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.778 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.778 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.778 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.779 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.779 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.779 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.779 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.779 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.779 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.779 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.780 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.780 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.780 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.780 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.780 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.781 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.782 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.782 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.782 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2023-07-14T16:37:25.782 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.783 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.783 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.783 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2023-07-14T16:37:25.783 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2023-07-14T16:37:25.783 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.783 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.784 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.784 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.784 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.784 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T16:37:25.784 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.784 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.784 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.785 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2023-07-14T16:37:25.785 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2023-07-14T16:37:25.785 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.785 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.785 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.785 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.785 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.786 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.786 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T16:37:25.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.786 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.787 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2023-07-14T16:37:25.787 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2023-07-14T16:37:25.787 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.787 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.787 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.787 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.787 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.787 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T16:37:25.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.788 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2023-07-14T16:37:25.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2023-07-14T16:37:25.789 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.789 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.789 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.789 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.789 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.789 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.789 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T16:37:25.790 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.790 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.790 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.790 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2023-07-14T16:37:25.790 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2023-07-14T16:37:25.790 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.790 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.791 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.791 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.791 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.791 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T16:37:25.791 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.792 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.792 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2023-07-14T16:37:25.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2023-07-14T16:37:25.792 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.792 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.792 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.792 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.793 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.793 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.793 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.793 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.793 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.793 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.793 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.793 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.794 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2023-07-14T16:37:25.794 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2023-07-14T16:37:25.794 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.794 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.794 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.794 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.794 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.795 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.795 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.795 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.795 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.795 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2023-07-14T16:37:25.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2023-07-14T16:37:25.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.796 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T16:37:25.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.796 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.797 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.797 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.797 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2023-07-14T16:37:25.797 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2023-07-14T16:37:25.797 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.798 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.798 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.798 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.798 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.799 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.799 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.799 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.799 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.799 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.800 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.800 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2023-07-14T16:37:25.800 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2023-07-14T16:37:25.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.800 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.801 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.801 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2023-07-14T16:37:25.801 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2023-07-14T16:37:25.801 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.801 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.801 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.801 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2023-07-14T16:37:25.802 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.802 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.802 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.802 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.802 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.802 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.802 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2023-07-14T16:37:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2023-07-14T16:37:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2023-07-14T16:37:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2023-07-14T16:37:25.805 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.805 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.805 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.805 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.806 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.806 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.806 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.806 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.807 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.807 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.807 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.807 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.807 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.807 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.807 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.808 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.808 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.808 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.808 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.808 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.808 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.808 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.808 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.809 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.809 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.809 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.809 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.809 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.809 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.810 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.810 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.810 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.810 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.810 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.810 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.811 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.811 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.811 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.811 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.811 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.812 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.812 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.812 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.812 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.812 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.812 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.812 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.813 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.813 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.813 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.813 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.814 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.814 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.814 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.814 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.814 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.815 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2023-07-14T16:37:25.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2023-07-14T16:37:25.815 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.815 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.815 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.815 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.816 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.816 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.816 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.816 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.816 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.817 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2023-07-14T16:37:25.817 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2023-07-14T16:37:25.817 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.817 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.817 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.817 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.818 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2023-07-14T16:37:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2023-07-14T16:37:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T16:37:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.821 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.821 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.821 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.821 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2023-07-14T16:37:25.821 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.821 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.821 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.822 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.822 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.822 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.823 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.823 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.823 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.823 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.823 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.823 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.824 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.824 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.824 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.824 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.824 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.824 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.824 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.825 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.825 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.825 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.826 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.826 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.827 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.827 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.827 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.827 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.828 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.828 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.828 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.828 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.828 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.828 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.828 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.828 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.829 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.829 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.829 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.829 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.829 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.829 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.830 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.830 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.830 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.830 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.830 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.831 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.831 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.831 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.831 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2023-07-14T16:37:25.831 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.831 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.831 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.833 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.833 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.833 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.833 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.833 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.833 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.833 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.833 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.835 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.836 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.836 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.836 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.836 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.836 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.837 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.838 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.838 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.838 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.838 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.838 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.838 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.838 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.838 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.839 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.839 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.839 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.839 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.839 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.840 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.840 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.840 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2023-07-14T16:37:25.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2023-07-14T16:37:25.841 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.841 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.841 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.841 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2023-07-14T16:37:25.841 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.841 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.842 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T16:37:25.842 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.842 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.842 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.842 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.842 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2023-07-14T16:37:25.842 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2023-07-14T16:37:25.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.843 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.843 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.843 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T16:37:25.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.845 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.845 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.845 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.845 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2023-07-14T16:37:25.845 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.845 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.845 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.846 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2023-07-14T16:37:25.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2023-07-14T16:37:25.846 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.846 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.846 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.846 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2023-07-14T16:37:25.846 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.846 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.847 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.847 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.847 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.847 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.847 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.847 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2023-07-14T16:37:25.847 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2023-07-14T16:37:25.848 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.848 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.848 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.848 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.848 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.848 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.848 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.849 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.849 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.849 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.849 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.849 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.849 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.850 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.850 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.850 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.850 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.850 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.850 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.850 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.851 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.851 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.851 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.851 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.852 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.852 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.852 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.852 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.852 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.852 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.852 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.854 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.854 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.854 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.854 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.854 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.854 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.854 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.854 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.855 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.855 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.855 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.855 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.855 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.855 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.855 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.856 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.856 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.856 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.856 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.856 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.856 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.856 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.857 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.857 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.857 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.857 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.857 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.857 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.858 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.858 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.858 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.858 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.858 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.858 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.859 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.859 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.859 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.859 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.859 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.859 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.860 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2023-07-14T16:37:25.860 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2023-07-14T16:37:25.860 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.860 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.860 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.860 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.860 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.861 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.861 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.861 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.861 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.861 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.861 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.861 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.861 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2023-07-14T16:37:25.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2023-07-14T16:37:25.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.862 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.862 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.862 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2023-07-14T16:37:25.862 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.862 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.862 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.863 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.863 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.863 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.863 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.863 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2023-07-14T16:37:25.863 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2023-07-14T16:37:25.864 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.864 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.864 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.864 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T16:37:25.864 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.864 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.864 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.864 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.865 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.865 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.865 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.865 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.865 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.865 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.865 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:25.865 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.866 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.866 INFO:teuthology.orchestra.run.smithi062.stdout: "modules": [ 2023-07-14T16:37:25.866 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2023-07-14T16:37:25.866 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard", 2023-07-14T16:37:25.866 INFO:teuthology.orchestra.run.smithi062.stdout: "iostat", 2023-07-14T16:37:25.866 INFO:teuthology.orchestra.run.smithi062.stdout: "nfs", 2023-07-14T16:37:25.866 INFO:teuthology.orchestra.run.smithi062.stdout: "restful" 2023-07-14T16:37:25.866 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.867 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2023-07-14T16:37:25.867 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2023-07-14T16:37:25.867 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.867 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.867 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.867 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:25.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:25.868 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.868 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.868 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.868 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.868 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.869 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T16:37:25.869 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.869 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.869 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.869 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.869 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.869 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.869 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.870 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.870 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.870 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.870 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.870 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.870 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.870 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.870 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.871 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.871 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.871 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.871 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.871 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.871 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.871 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.871 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.872 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.872 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.872 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.872 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.872 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.873 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.873 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.873 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.873 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.874 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.874 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.874 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.874 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.874 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.874 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.874 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.875 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.875 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.875 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.875 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.875 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.875 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.876 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.876 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.876 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.876 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.877 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.877 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.878 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.878 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.878 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2023-07-14T16:37:25.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2023-07-14T16:37:25.878 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.878 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.879 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.879 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.879 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.879 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2023-07-14T16:37:25.879 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.879 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.880 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.880 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.880 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2023-07-14T16:37:25.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2023-07-14T16:37:25.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.880 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.880 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.882 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2023-07-14T16:37:25.882 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2023-07-14T16:37:25.882 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.882 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.882 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.883 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.883 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2023-07-14T16:37:25.883 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.883 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.883 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.883 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2023-07-14T16:37:25.884 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2023-07-14T16:37:25.884 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.884 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.884 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.884 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.884 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.884 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2023-07-14T16:37:25.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.885 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2023-07-14T16:37:25.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2023-07-14T16:37:25.886 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.886 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.886 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.886 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2023-07-14T16:37:25.886 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.886 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.886 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.886 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2023-07-14T16:37:25.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.887 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.887 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2023-07-14T16:37:25.887 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2023-07-14T16:37:25.887 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.887 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.888 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.888 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.888 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.888 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.888 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.888 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2023-07-14T16:37:25.888 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.888 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.889 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.889 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2023-07-14T16:37:25.889 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2023-07-14T16:37:25.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.889 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.889 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.889 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.890 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.890 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T16:37:25.890 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.890 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.890 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.891 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2023-07-14T16:37:25.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2023-07-14T16:37:25.891 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.891 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.891 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.891 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.891 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.891 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.892 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2023-07-14T16:37:25.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.892 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.893 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2023-07-14T16:37:25.893 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.893 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.893 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.893 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2023-07-14T16:37:25.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2023-07-14T16:37:25.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.894 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.894 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.894 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.894 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.894 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T16:37:25.894 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.895 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.895 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.895 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2023-07-14T16:37:25.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2023-07-14T16:37:25.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.895 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2023-07-14T16:37:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T16:37:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T16:37:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.896 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.897 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2023-07-14T16:37:25.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2023-07-14T16:37:25.897 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.897 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.897 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.897 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.897 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2023-07-14T16:37:25.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2023-07-14T16:37:25.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T16:37:25.898 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T16:37:25.898 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.898 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.898 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2023-07-14T16:37:25.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2023-07-14T16:37:25.899 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.899 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.899 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.899 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2023-07-14T16:37:25.899 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:25.899 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2023-07-14T16:37:25.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T16:37:25.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.900 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.900 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.900 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.900 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2023-07-14T16:37:25.900 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2023-07-14T16:37:25.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.901 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.901 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T16:37:25.901 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.901 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.901 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.901 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T16:37:25.902 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T16:37:25.902 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.902 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.902 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.902 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2023-07-14T16:37:25.902 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2023-07-14T16:37:25.902 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.902 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.903 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.903 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2023-07-14T16:37:25.903 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2023-07-14T16:37:25.903 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2023-07-14T16:37:25.903 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.903 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2023-07-14T16:37:25.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T16:37:25.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.904 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.904 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.904 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2023-07-14T16:37:25.904 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2023-07-14T16:37:25.904 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.904 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.904 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2023-07-14T16:37:25.905 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.905 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.905 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.905 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T16:37:25.905 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T16:37:25.905 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.905 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.905 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.906 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2023-07-14T16:37:25.906 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2023-07-14T16:37:25.906 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.906 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.906 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.906 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2023-07-14T16:37:25.906 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2023-07-14T16:37:25.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.907 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T16:37:25.907 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T16:37:25.907 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.907 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.907 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.907 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.908 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.908 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.908 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.908 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.908 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.908 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.909 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.909 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.909 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.909 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.909 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.909 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.909 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.909 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.910 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.910 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.910 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.910 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.911 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.911 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.912 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.912 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.912 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.912 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.912 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.912 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.912 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.913 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.913 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.913 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.913 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.913 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.913 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.913 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.913 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.914 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.914 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.914 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.914 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.914 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.914 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.914 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.914 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.915 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.915 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.915 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.915 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.915 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.915 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.916 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.916 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.916 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.916 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.916 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2023-07-14T16:37:25.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2023-07-14T16:37:25.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.917 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.917 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.917 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:25.917 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.917 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.917 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T16:37:25.918 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.918 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.918 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.918 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2023-07-14T16:37:25.918 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2023-07-14T16:37:25.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.919 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.919 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2023-07-14T16:37:25.919 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.919 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.919 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.919 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2023-07-14T16:37:25.919 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2023-07-14T16:37:25.919 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap" 2023-07-14T16:37:25.920 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.920 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2023-07-14T16:37:25.920 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.920 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.920 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2023-07-14T16:37:25.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2023-07-14T16:37:25.921 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.921 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.921 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.921 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.921 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T16:37:25.922 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.922 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:25.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:25.922 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.922 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.923 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T16:37:25.923 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.923 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.924 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.924 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.924 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2023-07-14T16:37:25.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2023-07-14T16:37:25.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.924 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:25.925 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:25.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.925 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T16:37:25.925 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T16:37:25.925 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.925 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.926 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2023-07-14T16:37:25.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2023-07-14T16:37:25.926 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:25.926 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.926 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.926 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:25.926 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.926 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.927 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T16:37:25.927 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.927 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.927 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.927 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.927 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:25.927 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.928 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:25.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2023-07-14T16:37:25.928 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:25.928 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:25.928 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:25.928 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2023-07-14T16:37:25.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2023-07-14T16:37:25.928 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.929 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.929 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2023-07-14T16:37:25.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.929 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T16:37:25.929 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.930 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.930 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.930 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.930 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2023-07-14T16:37:25.930 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2023-07-14T16:37:25.930 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.930 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2023-07-14T16:37:25.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.931 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T16:37:25.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.931 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.932 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.932 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2023-07-14T16:37:25.932 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2023-07-14T16:37:25.932 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:25.932 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.932 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.933 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2023-07-14T16:37:25.933 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.933 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.933 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.933 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T16:37:25.933 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.933 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.934 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.934 INFO:teuthology.orchestra.run.smithi062.stdout: "alertmanager_web_password": { 2023-07-14T16:37:25.934 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alertmanager_web_password", 2023-07-14T16:37:25.934 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.934 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.934 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.934 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.935 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.935 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.935 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Alertmanager web password", 2023-07-14T16:37:25.935 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.935 INFO:teuthology.orchestra.run.smithi062.stdout: "alertmanager_web_user": { 2023-07-14T16:37:25.936 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alertmanager_web_user", 2023-07-14T16:37:25.936 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.936 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.936 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.936 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:25.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.936 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.937 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Alertmanager web user", 2023-07-14T16:37:25.937 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.937 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.937 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.937 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.937 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2023-07-14T16:37:25.937 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2023-07-14T16:37:25.937 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.938 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.938 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.938 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.938 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.938 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.938 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.938 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T16:37:25.938 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T16:37:25.939 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.939 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.939 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.939 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2023-07-14T16:37:25.939 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2023-07-14T16:37:25.939 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.939 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.940 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:25.940 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.940 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.940 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.940 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T16:37:25.940 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.940 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.941 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.941 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.941 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2023-07-14T16:37:25.941 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T16:37:25.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:25.941 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.941 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.941 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2023-07-14T16:37:25.942 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.942 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.942 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.942 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T16:37:25.942 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.942 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.942 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.943 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2023-07-14T16:37:25.943 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2023-07-14T16:37:25.943 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.943 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T16:37:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2023-07-14T16:37:25.944 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2023-07-14T16:37:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.945 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T16:37:25.946 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.946 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.946 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.946 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2023-07-14T16:37:25.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2023-07-14T16:37:25.946 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.946 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.947 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.947 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.947 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.947 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.947 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.947 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T16:37:25.947 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.947 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.948 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.948 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.948 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2023-07-14T16:37:25.948 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2023-07-14T16:37:25.948 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.948 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.948 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.948 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T16:37:25.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.949 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.949 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.949 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:25.949 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.949 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.949 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.949 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.950 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2023-07-14T16:37:25.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2023-07-14T16:37:25.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.950 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.950 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T16:37:25.950 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.951 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.951 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2023-07-14T16:37:25.951 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.951 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.951 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.951 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.951 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2023-07-14T16:37:25.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2023-07-14T16:37:25.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.952 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.952 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T16:37:25.952 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.952 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.952 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.953 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2023-07-14T16:37:25.953 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.953 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.953 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.953 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.953 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2023-07-14T16:37:25.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2023-07-14T16:37:25.953 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.954 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.954 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.954 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T16:37:25.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.954 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.954 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.954 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:25.954 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.955 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.955 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2023-07-14T16:37:25.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2023-07-14T16:37:25.955 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.955 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.955 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T16:37:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2023-07-14T16:37:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2023-07-14T16:37:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2023-07-14T16:37:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T16:37:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2023-07-14T16:37:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2023-07-14T16:37:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2023-07-14T16:37:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T16:37:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2023-07-14T16:37:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2023-07-14T16:37:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2023-07-14T16:37:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T16:37:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2023-07-14T16:37:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2023-07-14T16:37:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2023-07-14T16:37:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T16:37:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2023-07-14T16:37:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2023-07-14T16:37:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2023-07-14T16:37:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.964 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T16:37:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2023-07-14T16:37:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.965 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.966 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2023-07-14T16:37:25.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2023-07-14T16:37:25.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.966 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.966 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.966 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T16:37:25.966 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.966 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.967 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.967 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:25.967 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.967 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.967 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.967 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2023-07-14T16:37:25.967 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2023-07-14T16:37:25.968 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.968 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.968 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.968 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T16:37:25.968 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.968 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.968 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.968 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2023-07-14T16:37:25.969 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.969 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.969 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.969 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.969 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2023-07-14T16:37:25.969 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2023-07-14T16:37:25.969 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.969 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.970 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.970 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T16:37:25.970 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.970 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.970 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.970 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2023-07-14T16:37:25.970 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.970 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.971 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.971 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2023-07-14T16:37:25.971 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2023-07-14T16:37:25.971 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.971 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.971 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.971 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T16:37:25.972 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.972 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.972 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.972 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2023-07-14T16:37:25.972 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.972 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.972 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2023-07-14T16:37:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2023-07-14T16:37:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.973 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.974 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.974 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T16:37:25.974 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.974 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.974 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.974 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2023-07-14T16:37:25.974 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2023-07-14T16:37:25.975 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.975 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.975 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.975 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:25.975 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.975 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.975 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.975 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T16:37:25.976 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.976 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.976 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.976 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2023-07-14T16:37:25.976 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T16:37:25.976 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.977 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.977 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.977 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2023-07-14T16:37:25.977 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.977 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.977 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.977 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T16:37:25.977 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.978 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.978 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.978 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.978 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2023-07-14T16:37:25.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2023-07-14T16:37:25.978 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.978 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.978 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.979 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2023-07-14T16:37:25.979 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.979 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.979 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.979 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T16:37:25.979 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.979 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.979 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.980 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.980 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2023-07-14T16:37:25.980 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2023-07-14T16:37:25.980 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.980 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.980 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.980 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2023-07-14T16:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2023-07-14T16:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.982 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.983 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T16:37:25.983 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.983 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.983 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.983 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.983 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2023-07-14T16:37:25.983 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2023-07-14T16:37:25.983 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.984 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.984 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.984 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:25.984 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.984 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.984 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.984 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2023-07-14T16:37:25.984 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2023-07-14T16:37:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2023-07-14T16:37:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.986 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:25.986 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.986 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.986 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.986 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2023-07-14T16:37:25.986 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.986 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.986 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.987 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:25.987 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:25.987 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.987 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.987 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.987 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:25.987 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.988 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.988 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.988 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.988 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.988 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.988 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.989 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.989 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.989 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.989 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.989 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.989 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.989 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.989 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2023-07-14T16:37:25.990 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2023-07-14T16:37:25.990 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.990 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.990 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.990 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.990 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.990 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.990 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.991 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T16:37:25.991 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.991 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.991 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.991 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.991 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:25.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:25.991 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.992 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.992 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.992 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:25.992 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.992 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.992 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.992 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T16:37:25.992 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.993 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.993 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.993 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.993 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:25.993 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:25.993 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.993 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.993 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.994 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:25.994 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.994 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.994 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:25.994 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:25.994 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:25.994 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:25.994 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:25.995 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:25.995 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:25.995 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:25.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.995 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.995 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.995 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.996 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:25.996 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:25.996 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.996 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.996 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:25.996 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.996 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.996 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.997 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.997 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:25.997 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.997 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.997 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.997 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T16:37:25.997 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T16:37:25.998 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:25.998 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:25.998 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:25.998 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:25.998 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:25.998 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:25.998 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:25.998 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T16:37:25.999 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:25.999 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:25.999 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:25.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:25.999 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T16:37:25.999 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T16:37:25.999 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:25.999 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.000 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.000 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2023-07-14T16:37:26.000 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.000 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.000 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.000 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T16:37:26.000 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.000 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2023-07-14T16:37:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2023-07-14T16:37:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2023-07-14T16:37:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2023-07-14T16:37:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2023-07-14T16:37:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T16:37:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2023-07-14T16:37:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2023-07-14T16:37:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2023-07-14T16:37:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2023-07-14T16:37:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2023-07-14T16:37:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2023-07-14T16:37:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2023-07-14T16:37:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2023-07-14T16:37:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T16:37:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2023-07-14T16:37:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2023-07-14T16:37:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T16:37:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T16:37:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_web_password": { 2023-07-14T16:37:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_web_password", 2023-07-14T16:37:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus web password", 2023-07-14T16:37:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_web_user": { 2023-07-14T16:37:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_web_user", 2023-07-14T16:37:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus web user", 2023-07-14T16:37:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2023-07-14T16:37:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2023-07-14T16:37:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T16:37:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2023-07-14T16:37:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2023-07-14T16:37:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T16:37:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2023-07-14T16:37:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2023-07-14T16:37:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T16:37:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2023-07-14T16:37:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2023-07-14T16:37:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T16:37:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.021 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2023-07-14T16:37:26.021 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2023-07-14T16:37:26.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.021 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.021 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.021 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.021 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.022 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.022 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.022 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T16:37:26.022 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.022 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.022 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.022 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2023-07-14T16:37:26.023 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2023-07-14T16:37:26.023 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.023 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.023 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.023 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2023-07-14T16:37:26.023 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.023 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.023 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.024 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2023-07-14T16:37:26.024 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.024 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.024 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.024 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2023-07-14T16:37:26.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2023-07-14T16:37:26.024 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.025 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.025 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.025 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.025 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.025 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.025 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.025 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.026 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.027 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.027 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.027 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.027 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.027 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T16:37:26.027 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.027 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.028 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2023-07-14T16:37:26.028 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2023-07-14T16:37:26.028 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.028 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.028 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.028 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.029 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.029 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.029 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.029 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T16:37:26.029 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.029 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.029 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.030 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2023-07-14T16:37:26.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2023-07-14T16:37:26.030 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.030 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.030 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.030 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.030 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.030 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.031 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T16:37:26.031 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.031 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.031 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.031 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2023-07-14T16:37:26.032 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2023-07-14T16:37:26.032 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.032 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.032 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.032 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.032 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.032 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.033 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T16:37:26.033 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.033 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.033 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.033 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2023-07-14T16:37:26.033 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2023-07-14T16:37:26.033 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.034 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.034 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.034 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.034 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.034 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.034 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.034 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T16:37:26.034 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.035 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.035 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.035 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.035 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.035 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.035 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2023-07-14T16:37:26.035 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.036 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.036 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.036 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.036 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.036 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.036 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.036 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.036 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.037 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.037 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.037 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.037 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.037 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.037 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.037 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.037 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.038 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.038 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.038 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.038 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.038 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.038 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.038 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.038 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.039 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.039 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.039 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.039 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.039 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.039 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.039 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.039 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.040 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.041 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.041 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.041 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.041 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.041 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.041 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.041 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.041 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.042 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.042 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.042 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.042 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.042 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.042 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.042 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.042 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.043 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.043 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.043 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.043 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.043 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.043 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.043 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.044 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.044 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.044 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.044 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.044 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.044 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.044 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.045 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2023-07-14T16:37:26.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2023-07-14T16:37:26.045 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.045 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.045 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.045 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2023-07-14T16:37:26.045 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.046 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.046 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.046 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T16:37:26.046 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.046 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.046 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.046 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2023-07-14T16:37:26.047 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2023-07-14T16:37:26.047 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.047 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.047 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.047 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2023-07-14T16:37:26.047 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.047 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.047 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.048 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T16:37:26.048 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.048 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.048 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.048 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.048 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.048 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.049 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2023-07-14T16:37:26.049 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.049 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.049 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.049 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T16:37:26.049 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T16:37:26.049 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.049 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.050 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.050 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:26.050 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.050 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.050 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.050 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.050 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.051 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.051 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.051 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T16:37:26.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T16:37:26.051 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.052 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.052 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.052 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.052 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.052 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.052 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.052 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.052 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.053 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T16:37:26.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T16:37:26.053 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.053 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.053 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.053 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.053 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.054 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.054 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.054 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.054 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.054 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.054 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2023-07-14T16:37:26.055 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T16:37:26.055 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.055 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.055 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.055 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.055 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.055 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.055 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.056 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.056 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.056 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.056 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.056 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.056 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T16:37:26.056 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T16:37:26.056 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.057 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.057 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.057 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.057 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.057 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.057 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.057 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.058 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.058 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.058 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.058 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T16:37:26.058 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T16:37:26.058 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.058 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.058 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.059 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.059 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.059 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.059 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.059 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.059 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.059 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.059 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.060 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.060 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T16:37:26.060 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T16:37:26.060 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.060 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.060 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.060 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.060 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.061 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.061 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.061 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.061 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.061 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.061 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.061 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.061 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T16:37:26.062 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T16:37:26.062 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.062 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.062 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.062 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.062 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.062 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.062 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.063 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.063 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.063 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.063 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.063 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T16:37:26.063 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T16:37:26.063 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.064 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.064 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.064 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.064 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.064 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.064 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.064 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.064 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T16:37:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T16:37:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.067 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.067 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T16:37:26.067 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T16:37:26.067 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.067 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.067 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.067 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.067 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.068 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.068 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.068 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.068 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.068 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.068 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.068 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.068 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T16:37:26.069 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T16:37:26.069 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.069 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.069 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.069 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.069 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.069 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.069 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.070 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.070 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.070 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.070 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.070 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T16:37:26.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T16:37:26.070 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.071 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.071 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.071 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.071 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.071 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.071 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.071 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.072 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.072 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.072 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.072 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T16:37:26.072 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T16:37:26.072 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.073 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.073 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.073 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.073 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.073 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.073 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.073 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.074 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.074 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T16:37:26.074 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T16:37:26.074 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.074 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.074 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.074 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:26.074 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.075 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.075 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.075 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.075 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.075 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.075 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.075 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T16:37:26.076 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T16:37:26.076 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.076 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.076 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.076 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.076 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.076 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.076 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2023-07-14T16:37:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2023-07-14T16:37:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.078 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.078 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.078 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.078 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.078 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.078 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.078 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.078 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.079 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.079 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.079 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T16:37:26.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T16:37:26.079 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.079 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.079 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:26.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.080 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.080 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.080 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.081 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.081 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.081 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T16:37:26.081 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T16:37:26.081 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.081 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.081 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.081 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.082 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.082 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.082 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.082 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.082 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.082 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T16:37:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T16:37:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T16:37:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T16:37:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.085 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.086 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.087 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.087 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.087 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.087 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.087 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.087 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.087 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.088 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.088 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T16:37:26.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T16:37:26.088 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.088 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.088 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.088 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T16:37:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T16:37:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.091 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.091 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.091 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.091 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T16:37:26.091 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T16:37:26.092 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.092 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.092 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.092 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.092 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.092 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.092 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.093 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.094 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.094 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.094 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.094 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.094 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.094 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.094 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.095 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.095 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.095 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T16:37:26.095 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T16:37:26.095 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.095 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.095 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.096 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.096 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.096 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.096 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.096 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.096 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.096 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.096 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.097 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T16:37:26.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T16:37:26.097 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.097 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.097 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.097 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T16:37:26.098 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T16:37:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.099 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.100 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.100 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.100 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.100 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.100 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T16:37:26.100 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T16:37:26.100 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.101 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.102 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.102 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.102 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.102 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T16:37:26.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T16:37:26.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.103 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.103 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.103 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.103 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.103 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.103 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.103 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.104 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T16:37:26.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T16:37:26.104 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.104 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.104 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.104 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.104 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.105 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.105 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.105 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.105 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.105 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.105 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.105 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.105 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T16:37:26.106 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T16:37:26.106 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.106 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.106 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.106 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T16:37:26.106 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.106 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.106 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.107 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.107 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.107 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.107 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.107 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.107 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T16:37:26.107 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T16:37:26.107 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.108 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.108 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.108 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:26.108 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.108 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.108 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.108 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.108 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.109 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.109 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.109 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.109 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T16:37:26.109 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T16:37:26.109 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.109 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.109 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.110 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2023-07-14T16:37:26.110 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.110 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.110 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.110 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.110 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.110 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.111 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.111 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T16:37:26.111 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T16:37:26.111 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.111 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.111 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.111 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2023-07-14T16:37:26.111 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.112 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.112 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.112 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.112 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.112 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.112 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.112 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.112 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T16:37:26.113 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T16:37:26.113 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.113 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.113 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.113 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.113 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.113 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.113 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.114 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.114 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.114 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.114 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.114 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T16:37:26.114 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T16:37:26.114 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.115 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.115 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.115 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2023-07-14T16:37:26.115 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.115 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.115 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.115 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.115 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.116 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.116 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.116 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T16:37:26.116 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T16:37:26.116 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.116 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.117 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.117 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.117 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.117 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.117 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.117 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.117 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.117 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.118 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.118 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T16:37:26.118 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T16:37:26.118 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.118 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.118 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.118 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.119 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.119 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.119 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.119 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.119 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.119 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.119 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.120 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T16:37:26.120 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T16:37:26.120 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.120 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.120 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.120 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:26.120 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.120 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T16:37:26.121 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T16:37:26.122 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.122 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.122 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.122 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:26.122 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.122 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.122 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.122 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.123 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.123 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.123 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.123 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T16:37:26.123 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T16:37:26.123 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.124 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.124 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:26.124 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.124 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.124 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.124 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.124 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.124 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.125 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.125 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.125 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2023-07-14T16:37:26.125 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2023-07-14T16:37:26.125 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.125 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.125 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.125 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.127 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2023-07-14T16:37:26.127 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2023-07-14T16:37:26.127 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.127 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.127 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.128 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.128 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.128 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.128 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.128 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.128 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2023-07-14T16:37:26.128 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2023-07-14T16:37:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2023-07-14T16:37:26.130 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.130 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.130 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.130 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2023-07-14T16:37:26.130 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2023-07-14T16:37:26.130 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.130 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.131 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.131 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2023-07-14T16:37:26.131 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.131 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.131 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.131 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.131 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.132 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.132 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2023-07-14T16:37:26.132 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2023-07-14T16:37:26.132 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.133 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.133 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.133 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.133 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.133 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.133 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.133 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.134 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.135 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.135 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.135 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.135 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.135 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.135 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.135 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.136 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.136 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.136 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.136 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.136 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.136 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.136 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.137 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.137 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.137 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.137 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.137 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.137 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.137 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.139 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.139 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.139 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.139 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.139 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.139 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.139 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.139 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.141 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.141 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.141 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.141 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.141 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.141 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.141 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.141 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.142 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.142 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.142 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.142 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.142 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.142 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.142 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.142 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.143 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2023-07-14T16:37:26.143 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2023-07-14T16:37:26.143 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.143 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.143 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.143 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.143 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.143 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2023-07-14T16:37:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2023-07-14T16:37:26.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T16:37:26.145 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.145 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.145 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.145 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.145 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:26.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.147 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.147 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.147 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2023-07-14T16:37:26.147 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.147 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.147 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.147 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.147 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.149 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2023-07-14T16:37:26.149 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.149 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.149 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.149 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.149 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.149 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.149 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.150 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2023-07-14T16:37:26.150 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2023-07-14T16:37:26.150 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.150 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.150 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.150 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.150 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.151 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.151 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.151 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.151 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.151 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.151 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.151 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.151 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2023-07-14T16:37:26.152 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2023-07-14T16:37:26.152 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.152 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.152 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.152 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2023-07-14T16:37:26.152 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.152 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.153 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.153 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.153 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.153 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.153 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2023-07-14T16:37:26.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2023-07-14T16:37:26.153 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.154 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.154 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.154 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2023-07-14T16:37:26.154 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.154 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.154 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.154 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2023-07-14T16:37:26.155 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.155 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.155 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.155 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.155 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.155 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2023-07-14T16:37:26.155 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2023-07-14T16:37:26.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.156 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.156 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.156 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2023-07-14T16:37:26.156 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2023-07-14T16:37:26.156 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2023-07-14T16:37:26.156 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.156 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.157 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.157 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.157 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.157 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.157 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2023-07-14T16:37:26.157 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2023-07-14T16:37:26.157 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.157 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.158 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.159 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.159 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.159 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.159 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.159 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.159 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2023-07-14T16:37:26.159 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.159 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.160 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.160 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2023-07-14T16:37:26.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2023-07-14T16:37:26.160 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.160 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.160 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.160 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.160 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.161 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.161 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2023-07-14T16:37:26.161 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.161 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.161 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.161 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.161 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.162 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.164 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.164 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.164 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.164 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.164 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.164 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.164 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.164 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.165 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.165 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.165 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.165 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.165 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.165 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.165 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.165 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.166 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.166 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.166 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.166 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.166 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.166 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.166 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.166 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.167 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.167 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.167 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.167 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.167 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.167 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.167 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.167 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.168 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.168 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.168 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.168 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.168 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.168 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.168 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.168 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.169 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.169 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.169 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.169 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.169 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.169 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.169 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2023-07-14T16:37:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2023-07-14T16:37:26.170 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.171 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.171 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.171 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2023-07-14T16:37:26.171 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.171 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.171 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.171 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T16:37:26.171 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2023-07-14T16:37:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2023-07-14T16:37:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.173 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2023-07-14T16:37:26.173 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.173 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T16:37:26.173 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.173 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.173 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.174 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.174 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2023-07-14T16:37:26.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2023-07-14T16:37:26.174 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.174 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.174 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.174 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2023-07-14T16:37:26.174 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.175 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.175 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.175 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2023-07-14T16:37:26.175 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.175 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.175 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.175 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2023-07-14T16:37:26.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2023-07-14T16:37:26.176 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.176 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.176 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.176 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2023-07-14T16:37:26.176 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.176 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.176 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.177 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T16:37:26.177 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.177 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.177 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.177 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2023-07-14T16:37:26.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2023-07-14T16:37:26.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.178 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.178 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.178 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.178 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.178 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.178 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T16:37:26.179 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.179 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.179 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.179 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.179 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:26.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:26.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.180 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.180 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:26.180 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.180 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T16:37:26.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.181 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.181 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.181 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2023-07-14T16:37:26.181 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2023-07-14T16:37:26.181 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.181 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2023-07-14T16:37:26.182 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.182 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.182 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.182 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T16:37:26.182 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.182 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.182 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.183 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.183 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2023-07-14T16:37:26.183 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.183 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.183 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.184 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.184 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.184 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.184 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.184 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.184 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.185 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.185 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.185 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.185 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.185 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.185 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.185 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.185 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.186 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.186 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.186 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.186 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.186 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.186 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.186 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.187 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.187 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.187 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.187 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.187 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.187 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.187 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.187 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.188 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.188 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.188 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.188 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.188 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.188 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.188 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.189 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.189 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.189 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.189 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.189 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.189 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.189 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.189 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.190 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.190 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.190 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.190 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.190 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.190 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.190 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.190 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.191 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.191 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.191 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.191 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.191 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.191 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.191 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.191 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.192 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.192 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.192 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.192 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.192 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.192 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.192 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2023-07-14T16:37:26.192 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2023-07-14T16:37:26.193 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.193 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.193 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.193 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2023-07-14T16:37:26.193 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.193 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.193 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.193 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.194 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.194 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.194 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.194 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2023-07-14T16:37:26.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2023-07-14T16:37:26.194 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.194 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.195 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.195 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2023-07-14T16:37:26.195 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.195 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.195 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.195 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.195 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.195 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.196 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.196 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.196 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:26.196 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:26.196 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.196 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.196 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.197 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2023-07-14T16:37:26.197 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.197 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.197 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.197 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.197 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.198 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.198 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.198 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.198 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.198 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2023-07-14T16:37:26.198 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2023-07-14T16:37:26.198 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2023-07-14T16:37:26.198 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.199 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2023-07-14T16:37:26.199 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2023-07-14T16:37:26.199 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.199 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.199 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.199 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2023-07-14T16:37:26.199 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.199 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.200 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.200 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T16:37:26.200 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.200 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.200 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.200 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.200 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2023-07-14T16:37:26.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2023-07-14T16:37:26.201 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.201 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.201 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.201 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2023-07-14T16:37:26.201 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.201 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.202 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T16:37:26.202 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.202 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.202 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.202 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.202 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2023-07-14T16:37:26.202 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2023-07-14T16:37:26.203 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.203 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.203 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.203 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.203 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.203 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.203 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.203 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2023-07-14T16:37:26.204 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.204 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.204 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.204 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.204 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:26.204 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:26.204 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.205 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.205 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.205 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2023-07-14T16:37:26.205 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2023-07-14T16:37:26.205 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.205 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.205 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T16:37:26.205 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.206 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.206 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.206 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.206 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.206 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.206 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.209 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.209 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.209 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.209 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.209 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.210 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.210 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.210 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.210 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.210 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.210 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.210 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.211 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.211 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.211 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.211 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.211 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.211 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.211 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.212 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.212 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.212 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.212 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.212 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.212 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.212 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.212 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.213 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.213 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.213 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.213 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.213 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.215 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.215 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2023-07-14T16:37:26.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2023-07-14T16:37:26.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.215 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.215 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.215 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.216 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.216 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.216 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.216 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2023-07-14T16:37:26.216 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.216 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.216 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.217 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2023-07-14T16:37:26.217 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2023-07-14T16:37:26.217 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.217 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.217 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.217 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2023-07-14T16:37:26.217 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.218 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.218 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.218 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2023-07-14T16:37:26.218 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.218 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.218 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2023-07-14T16:37:26.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2023-07-14T16:37:26.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.219 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.219 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.219 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2023-07-14T16:37:26.219 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.219 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.219 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.220 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T16:37:26.220 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.220 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.220 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.220 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.220 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2023-07-14T16:37:26.220 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2023-07-14T16:37:26.220 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.221 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.221 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.221 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:26.221 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:26.221 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2023-07-14T16:37:26.221 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.221 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T16:37:26.221 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2023-07-14T16:37:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2023-07-14T16:37:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.222 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.223 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.223 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.223 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.223 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.223 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2023-07-14T16:37:26.223 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.223 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.224 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.224 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2023-07-14T16:37:26.224 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2023-07-14T16:37:26.224 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.224 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2023-07-14T16:37:26.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.225 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.225 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.225 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T16:37:26.225 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.225 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.225 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.225 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.225 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.226 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2023-07-14T16:37:26.226 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.226 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.226 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.226 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.227 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.227 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.227 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.227 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.227 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.227 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.227 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.227 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.228 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.228 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.228 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.228 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.228 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.228 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.228 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.228 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.229 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.229 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.229 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.229 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.229 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.229 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.229 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.230 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.230 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.230 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.230 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.230 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.230 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.230 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.230 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.231 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.231 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.231 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.231 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.231 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.231 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.232 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.232 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.232 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.232 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.232 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.233 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.233 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.233 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.233 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.233 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.233 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.233 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.234 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.234 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.234 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.234 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.234 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.234 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.234 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.235 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.235 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.235 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.235 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.235 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.235 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.235 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.236 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.236 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.236 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2023-07-14T16:37:26.236 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.236 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.236 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.236 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.237 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.237 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.237 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.237 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.237 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.237 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.237 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.237 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.238 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.238 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.238 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.238 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.238 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.238 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.238 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.238 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.239 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.239 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.239 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.239 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.240 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.240 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.240 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.240 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.240 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.240 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.240 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.240 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.241 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.241 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.241 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.241 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.241 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.241 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.241 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.241 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.242 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.242 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.242 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.242 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.242 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.242 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.242 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.242 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.243 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.243 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.243 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.243 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.243 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.243 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.243 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.244 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.244 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.244 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.244 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.244 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.245 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.245 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.245 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.245 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.245 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.245 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.246 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.246 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2023-07-14T16:37:26.246 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.246 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.246 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.246 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2023-07-14T16:37:26.246 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2023-07-14T16:37:26.247 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.247 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.247 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.247 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2023-07-14T16:37:26.247 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.247 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.247 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T16:37:26.248 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.248 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2023-07-14T16:37:26.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2023-07-14T16:37:26.248 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.248 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.249 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.249 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:26.249 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2023-07-14T16:37:26.249 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.249 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.249 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T16:37:26.249 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.249 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.250 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.250 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.250 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.250 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.250 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.250 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.250 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.250 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.251 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.251 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.251 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.251 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.251 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.251 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.251 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.252 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.252 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.252 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.252 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.252 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.252 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.253 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.253 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.253 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.253 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.253 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.253 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.254 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.254 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.254 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.254 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.254 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.254 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.255 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.255 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.255 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.255 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.255 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.255 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.256 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.256 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.256 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.256 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.256 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.256 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.256 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.256 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.259 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.259 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.259 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.259 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.259 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2023-07-14T16:37:26.259 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.259 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.259 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.260 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2023-07-14T16:37:26.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2023-07-14T16:37:26.260 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.260 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.260 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2023-07-14T16:37:26.260 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.260 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2023-07-14T16:37:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T16:37:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.262 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.262 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.262 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.262 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.262 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.262 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.263 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.263 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.263 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.263 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.263 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.263 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.263 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.264 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.264 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.264 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.264 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.265 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.265 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.265 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.265 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.265 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.265 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.265 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.265 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.266 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.266 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.266 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.266 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.266 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.266 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.267 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.267 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.267 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.267 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.267 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.267 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.267 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.267 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.268 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.268 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.269 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.269 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.269 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.270 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.270 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.270 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.270 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.270 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.270 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2023-07-14T16:37:26.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2023-07-14T16:37:26.271 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.271 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.271 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.271 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2023-07-14T16:37:26.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T16:37:26.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.272 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.272 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.272 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2023-07-14T16:37:26.272 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2023-07-14T16:37:26.272 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.272 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.273 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.273 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2023-07-14T16:37:26.273 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.273 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.273 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.273 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2023-07-14T16:37:26.273 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.273 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.274 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.274 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2023-07-14T16:37:26.274 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2023-07-14T16:37:26.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.274 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.274 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.274 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2023-07-14T16:37:26.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T16:37:26.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.275 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2023-07-14T16:37:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2023-07-14T16:37:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.276 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.277 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2023-07-14T16:37:26.277 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.277 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.277 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.277 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2023-07-14T16:37:26.277 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2023-07-14T16:37:26.278 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.278 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.278 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.278 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2023-07-14T16:37:26.278 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.278 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.278 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.278 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T16:37:26.279 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T16:37:26.279 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.279 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.279 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.279 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.279 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.279 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.279 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2023-07-14T16:37:26.280 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.280 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.280 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.280 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.280 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.280 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.280 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.281 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.281 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.281 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.281 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.281 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.281 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.281 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.282 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.282 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.282 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.282 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.282 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.282 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.283 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.283 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.283 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.283 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.284 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.284 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.285 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.285 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.285 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.285 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.285 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.286 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.286 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.286 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.286 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.286 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.286 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.286 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.287 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.287 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.287 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.287 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.287 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.287 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.288 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.289 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.289 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.289 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.289 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.289 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2023-07-14T16:37:26.289 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.289 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.290 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.290 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.290 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.290 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.290 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.290 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.290 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.290 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.291 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.291 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.291 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.291 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.291 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.291 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.291 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.292 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.292 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.292 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.292 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.292 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.292 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.292 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.292 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.293 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.293 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.293 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.293 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.294 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.294 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.294 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.294 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.294 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.294 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.294 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.295 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.295 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.295 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.295 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.296 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.296 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.296 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.296 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.296 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.297 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.297 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.297 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.297 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.297 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.297 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.298 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2023-07-14T16:37:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.299 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.300 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.300 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.300 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.300 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.300 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.301 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.301 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.301 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.301 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.301 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.301 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.301 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.301 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.302 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.302 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.302 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.302 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.302 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.303 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.304 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.304 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.304 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.304 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.304 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.304 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.305 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.305 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.305 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.305 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.305 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.305 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.305 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.305 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.306 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.306 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.306 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.306 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.306 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.306 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.306 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.306 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.307 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.307 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.307 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.307 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.307 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.307 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.308 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.308 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.308 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.308 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.308 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.308 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.308 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.309 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2023-07-14T16:37:26.309 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.309 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.309 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.309 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2023-07-14T16:37:26.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2023-07-14T16:37:26.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T16:37:26.310 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.311 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.311 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.311 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.311 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.311 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.311 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.311 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.311 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.312 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.312 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.312 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.312 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.312 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.312 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.312 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.312 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.313 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.313 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.313 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.313 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.313 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.313 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.313 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.314 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.314 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.314 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.314 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.314 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.315 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.315 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.315 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.315 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.315 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.316 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.316 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.316 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.316 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.316 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.317 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.317 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.317 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.317 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.317 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.317 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.317 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.317 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.319 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.319 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.319 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.319 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.319 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.319 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.319 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.320 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.320 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2023-07-14T16:37:26.320 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2023-07-14T16:37:26.320 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.320 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.320 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.321 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.321 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.321 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.321 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.321 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2023-07-14T16:37:26.321 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2023-07-14T16:37:26.321 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2023-07-14T16:37:26.321 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.322 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2023-07-14T16:37:26.322 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.322 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.322 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.322 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.322 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.322 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.322 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2023-07-14T16:37:26.323 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.323 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.323 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.323 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.323 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.323 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.324 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.324 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.324 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.325 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.325 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.325 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.325 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.325 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.325 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.325 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.325 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.326 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.326 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.326 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.326 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.326 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.327 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.327 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.327 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.327 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.327 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.327 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.327 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.328 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.328 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.328 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.328 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.328 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.328 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.329 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.329 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.329 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.329 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.329 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.329 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.329 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.332 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.332 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.332 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2023-07-14T16:37:26.332 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.332 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.333 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.333 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.333 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.333 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.333 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.333 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.334 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.335 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.335 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.335 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.335 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.335 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.335 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.337 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.337 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.337 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.338 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.338 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.339 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.339 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.339 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.339 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.339 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.340 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.340 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.340 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.340 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.340 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.341 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.341 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.341 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.341 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.341 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.341 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.341 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.342 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.342 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.342 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.342 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2023-07-14T16:37:26.342 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.342 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.342 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.343 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.343 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.343 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.343 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.344 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.344 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.344 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.344 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.344 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.344 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.344 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.344 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.345 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.345 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.345 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.345 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.345 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.345 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.346 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.346 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.346 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.346 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.346 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.346 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.346 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.347 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.347 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.347 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.347 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.347 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.347 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.348 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.350 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.350 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.350 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.350 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.351 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.351 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.351 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.351 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.352 INFO:teuthology.orchestra.run.smithi062.stdout: "noautoscale": { 2023-07-14T16:37:26.352 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "noautoscale", 2023-07-14T16:37:26.352 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.352 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.352 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "global autoscale flag", 2023-07-14T16:37:26.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T16:37:26.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.353 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.353 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:26.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:26.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.354 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.354 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:26.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.354 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.354 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.355 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.355 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.355 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.355 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.355 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.355 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2023-07-14T16:37:26.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2023-07-14T16:37:26.355 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:26.356 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.356 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.356 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2023-07-14T16:37:26.356 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2023-07-14T16:37:26.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.356 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2023-07-14T16:37:26.357 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T16:37:26.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.357 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.357 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.357 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.357 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.357 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2023-07-14T16:37:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2023-07-14T16:37:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2023-07-14T16:37:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T16:37:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2023-07-14T16:37:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2023-07-14T16:37:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2023-07-14T16:37:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2023-07-14T16:37:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2023-07-14T16:37:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2023-07-14T16:37:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2023-07-14T16:37:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2023-07-14T16:37:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2023-07-14T16:37:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2023-07-14T16:37:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2023-07-14T16:37:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2023-07-14T16:37:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2023-07-14T16:37:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2023-07-14T16:37:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2023-07-14T16:37:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T16:37:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T16:37:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.380 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.380 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.380 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.380 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.380 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.380 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.380 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.380 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.381 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.381 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.381 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.381 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.381 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.381 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.382 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.382 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.382 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.382 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.382 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.382 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.350+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38700a1250 msgr2=0x7f38700a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:26.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.350+0000 7f388c5ff700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38700a1250 0x7f38700a3700 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3874006010 tx=0x7f387400b970 comp rx=0 tx=0).stop 2023-07-14T16:37:26.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.350+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f388410bd60 msgr2=0x7f38841a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:37:26.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.350+0000 7f388c5ff700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f388410bd60 0x7f38841a3720 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f387c007f10 tx=0x7f387c00d3b0 comp rx=0 tx=0).stop 2023-07-14T16:37:26.384 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.351+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 shutdown_connections 2023-07-14T16:37:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.351+0000 7f388c5ff700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f388410bd60 0x7f38841a3720 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.351+0000 7f388c5ff700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3884070ed0 0x7f3884113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.351+0000 7f388c5ff700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38700a1250 0x7f38700a3700 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.351+0000 7f388c5ff700 1 --2- 172.21.15.62:0/1443804460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3884071820 0x7f3884113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:37:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.351+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 >> 172.21.15.62:0/1443804460 conn(0x7f388406ca40 msgr2=0x7f388406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:37:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.352+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 shutdown_connections 2023-07-14T16:37:26.385 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:37:24.357+0000 7f388c5ff700 1 -- 172.21.15.62:0/1443804460 wait complete. 2023-07-14T16:37:26.386 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:25 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1443804460' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T16:37:26.387 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:25 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1443804460' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T16:37:26.388 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.388 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.388 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.388 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.388 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.388 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.389 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.389 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.389 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.389 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.389 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.389 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.390 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.390 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.390 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.390 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.390 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.391 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.391 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.391 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.392 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.392 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.392 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.392 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.392 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.393 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.393 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2023-07-14T16:37:26.393 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2023-07-14T16:37:26.393 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.393 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.394 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.394 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.394 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.394 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.394 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.394 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.394 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.395 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T16:37:26.395 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T16:37:26.395 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.395 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.396 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.396 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2023-07-14T16:37:26.396 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.396 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.397 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.397 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.397 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.398 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.398 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.398 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2023-07-14T16:37:26.398 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2023-07-14T16:37:26.399 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:26.399 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.399 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.399 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2023-07-14T16:37:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.401 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.401 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.402 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.402 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:26.402 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:26.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.403 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.403 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2023-07-14T16:37:26.403 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.403 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.403 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.403 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T16:37:26.403 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.404 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.404 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.404 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.404 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:26.404 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:26.404 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.404 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.404 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.405 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2023-07-14T16:37:26.405 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.405 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.405 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.405 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T16:37:26.405 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.405 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.405 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.406 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2023-07-14T16:37:26.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2023-07-14T16:37:26.406 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.406 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.406 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.406 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2023-07-14T16:37:26.407 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.407 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.408 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2023-07-14T16:37:26.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2023-07-14T16:37:26.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2023-07-14T16:37:26.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.409 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.409 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2023-07-14T16:37:26.409 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2023-07-14T16:37:26.409 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.409 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.409 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.409 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.410 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.410 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.410 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2023-07-14T16:37:26.410 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2023-07-14T16:37:26.410 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2023-07-14T16:37:26.411 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2023-07-14T16:37:26.411 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2023-07-14T16:37:26.411 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.411 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.412 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.412 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.412 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.412 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.412 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2023-07-14T16:37:26.413 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.413 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.413 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.413 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.413 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.414 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.414 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.414 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.414 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.414 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.414 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.415 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.415 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.415 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.415 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.415 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.415 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.415 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.416 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.416 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.416 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.417 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.417 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.417 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.418 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.418 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.418 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.418 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.418 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.418 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.419 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.419 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.419 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.419 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.419 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.419 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.419 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.420 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.420 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.420 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.420 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.420 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.420 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.420 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.421 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.421 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.421 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.421 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.421 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.422 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.422 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.422 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.423 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2023-07-14T16:37:26.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2023-07-14T16:37:26.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.423 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.423 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2023-07-14T16:37:26.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.424 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.424 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.424 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2023-07-14T16:37:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T16:37:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.427 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2023-07-14T16:37:26.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2023-07-14T16:37:26.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.428 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.428 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.428 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.429 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.429 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2023-07-14T16:37:26.429 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.429 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.430 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.430 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2023-07-14T16:37:26.430 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2023-07-14T16:37:26.430 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.430 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.430 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.430 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.431 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.431 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.431 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.431 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.431 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2023-07-14T16:37:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2023-07-14T16:37:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.432 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.433 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.434 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.434 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.434 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.435 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.435 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.435 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.436 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.437 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.438 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.438 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.438 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.439 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.439 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.439 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.439 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.440 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.440 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.440 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.441 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.443 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.443 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.443 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.443 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.444 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.444 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.444 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.444 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.444 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.445 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.445 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.445 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.445 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.446 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.446 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.446 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.446 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.446 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.446 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.446 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.447 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.447 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.447 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.447 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.447 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.447 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.448 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.448 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.449 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2023-07-14T16:37:26.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2023-07-14T16:37:26.449 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.449 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.449 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.450 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.450 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.450 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.450 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.450 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.450 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.450 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.450 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.451 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2023-07-14T16:37:26.451 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2023-07-14T16:37:26.451 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.451 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.451 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.451 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.451 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.452 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.452 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.452 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.452 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.452 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.452 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.453 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.453 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2023-07-14T16:37:26.453 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.453 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.453 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.453 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.454 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.454 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.454 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.454 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.454 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.454 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.454 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.454 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.455 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.455 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.455 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.455 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.455 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.456 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.456 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.456 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.456 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.457 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.457 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.457 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.457 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.457 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.457 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.457 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.458 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.458 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.458 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.458 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.458 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.458 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.458 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.458 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.459 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.459 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.459 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.460 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.460 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.460 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.460 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.460 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.460 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.460 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.461 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.461 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.461 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.461 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.461 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.461 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.462 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.462 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.462 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.462 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.462 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.462 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.463 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.463 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.463 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.463 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.463 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.463 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2023-07-14T16:37:26.464 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.464 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.464 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.464 INFO:teuthology.orchestra.run.smithi062.stdout: "drive_group_interval": { 2023-07-14T16:37:26.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "drive_group_interval", 2023-07-14T16:37:26.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:26.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.465 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300.0", 2023-07-14T16:37:26.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.465 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T16:37:26.465 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.465 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.466 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.466 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.466 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.466 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.467 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.468 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.468 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.468 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.468 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.468 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.469 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.469 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.469 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.469 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.469 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.469 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.469 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.470 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.470 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.470 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.470 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.470 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.471 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.471 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.471 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.471 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.472 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.472 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.472 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.473 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.474 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.474 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.474 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.474 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.474 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.474 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.474 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.475 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.475 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.475 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.476 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.476 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.476 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.477 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.477 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.477 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.478 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.478 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.478 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2023-07-14T16:37:26.478 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2023-07-14T16:37:26.478 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.478 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.479 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.479 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2023-07-14T16:37:26.479 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.479 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.479 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.479 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T16:37:26.479 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.479 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.480 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.480 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.480 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.480 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.480 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2023-07-14T16:37:26.480 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.481 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.481 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.481 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.481 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.481 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.481 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.482 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.482 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.482 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.482 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.482 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.482 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.482 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.482 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.483 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.483 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.483 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.483 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.484 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.484 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.484 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.484 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.484 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.484 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.485 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.485 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.485 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.485 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.485 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.486 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.486 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.486 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.486 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.487 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.487 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.488 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.488 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.488 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.488 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.488 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.488 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.488 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.489 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.489 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.489 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.489 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.489 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.489 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.489 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.489 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.490 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.490 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.490 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.490 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.490 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.490 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.490 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.491 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.491 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.491 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2023-07-14T16:37:26.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2023-07-14T16:37:26.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.491 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.491 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.492 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.492 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.492 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.492 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.492 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.492 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.492 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.493 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2023-07-14T16:37:26.493 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2023-07-14T16:37:26.493 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.493 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.493 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.493 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2023-07-14T16:37:26.493 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.494 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.494 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.494 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.494 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.494 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.494 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.494 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2023-07-14T16:37:26.495 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2023-07-14T16:37:26.495 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.495 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.495 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.495 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.495 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.496 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.496 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.496 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.496 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.496 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2023-07-14T16:37:26.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2023-07-14T16:37:26.497 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.497 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.497 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.497 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.497 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.497 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.498 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.498 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.498 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.498 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.499 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2023-07-14T16:37:26.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2023-07-14T16:37:26.499 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2023-07-14T16:37:26.499 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.499 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.499 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.499 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.500 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.500 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.500 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.500 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.500 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.500 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.500 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2023-07-14T16:37:26.501 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2023-07-14T16:37:26.501 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.501 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.501 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.501 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.501 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.501 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.502 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.502 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.502 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.502 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.502 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2023-07-14T16:37:26.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2023-07-14T16:37:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.503 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.504 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.504 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.504 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.504 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2023-07-14T16:37:26.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2023-07-14T16:37:26.504 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.505 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.506 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.506 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.506 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.506 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2023-07-14T16:37:26.506 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2023-07-14T16:37:26.506 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.507 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.507 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.507 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.507 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2023-07-14T16:37:26.507 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2023-07-14T16:37:26.507 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.507 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.507 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.508 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.508 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.508 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.508 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2023-07-14T16:37:26.508 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2023-07-14T16:37:26.509 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.509 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.509 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.509 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.509 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.509 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.509 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.510 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.510 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.510 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.510 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.510 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2023-07-14T16:37:26.510 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2023-07-14T16:37:26.510 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.511 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.511 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.511 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.511 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.511 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.511 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.511 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.512 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.512 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.512 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.512 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2023-07-14T16:37:26.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2023-07-14T16:37:26.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.513 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.513 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.513 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.513 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.513 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.513 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.513 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.514 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.514 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.514 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.514 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.514 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.514 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2023-07-14T16:37:26.515 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.515 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.515 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.515 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2023-07-14T16:37:26.515 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2023-07-14T16:37:26.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.515 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.516 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.516 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.516 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.516 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.516 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T16:37:26.517 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.517 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.517 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.517 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2023-07-14T16:37:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2023-07-14T16:37:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.519 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.519 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.519 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.519 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2023-07-14T16:37:26.519 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.519 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.520 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.521 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.521 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.521 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.521 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.521 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.521 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.521 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.522 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.522 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.522 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.522 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.522 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.522 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.522 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.522 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.523 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.523 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.523 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.523 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.523 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.523 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.523 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.523 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.524 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.524 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.524 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.524 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.524 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.524 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.525 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.526 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.526 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.526 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.526 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.526 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.526 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.526 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.526 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.528 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.528 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.528 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.528 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.528 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.528 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.528 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.529 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.529 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.529 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.529 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.529 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2023-07-14T16:37:26.530 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.530 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.530 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.530 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.530 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.530 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.530 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.531 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.531 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.531 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.531 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.531 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.531 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.531 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.531 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.532 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.532 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.532 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.532 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.532 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.532 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.532 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.533 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.533 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.533 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.533 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.533 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.533 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.533 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.534 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.534 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.534 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.534 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.534 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.534 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.534 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.534 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.535 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.535 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.535 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.535 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.535 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.535 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.536 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.536 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.536 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.536 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.536 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.536 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.536 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.536 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.537 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.537 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.537 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.537 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.537 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.537 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.538 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.538 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.538 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.538 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.538 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.538 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.539 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.539 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.539 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.539 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.539 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.539 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.539 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.540 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2023-07-14T16:37:26.540 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.540 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.540 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.540 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.540 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.541 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.541 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.541 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.541 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.541 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.541 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.542 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.542 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.542 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.542 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.542 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.542 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.542 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.543 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.543 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.543 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.544 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.544 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.544 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.544 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.545 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.545 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.545 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.545 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.545 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.545 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.545 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.545 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.546 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.546 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.546 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.546 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.546 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.546 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.546 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.546 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.547 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.547 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.547 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.547 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.547 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.547 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.548 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.548 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.548 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.548 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.548 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.548 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.548 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.549 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.549 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.549 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.549 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.549 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.549 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.550 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2023-07-14T16:37:26.550 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.550 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.550 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.550 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2023-07-14T16:37:26.551 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2023-07-14T16:37:26.551 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.551 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.551 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T16:37:26.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.552 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.552 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.552 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.552 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.552 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:26.553 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:26.553 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.553 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.553 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.553 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2023-07-14T16:37:26.553 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.553 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.553 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.554 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.554 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.554 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.554 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.554 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.554 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.555 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.555 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.555 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.555 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.555 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.555 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.555 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.556 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.556 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.556 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.556 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.556 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.556 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.557 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.557 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.557 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.557 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.557 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.558 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.558 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.558 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.558 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.558 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.561 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.561 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.561 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.561 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.561 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.562 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.562 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.562 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.562 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.562 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.562 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.562 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.563 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.564 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.564 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.564 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.564 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.564 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2023-07-14T16:37:26.565 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.565 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.565 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.565 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2023-07-14T16:37:26.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2023-07-14T16:37:26.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.566 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.566 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.566 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.566 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.567 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.567 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T16:37:26.567 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.567 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.567 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.567 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2023-07-14T16:37:26.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2023-07-14T16:37:26.568 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.568 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.568 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.568 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.568 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.569 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T16:37:26.569 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.569 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.569 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.569 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2023-07-14T16:37:26.569 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2023-07-14T16:37:26.569 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.570 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.570 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.570 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2023-07-14T16:37:26.570 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.570 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.570 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T16:37:26.570 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.571 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.571 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.571 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.571 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2023-07-14T16:37:26.571 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2023-07-14T16:37:26.571 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.572 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.572 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.572 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.572 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.572 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.572 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.573 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T16:37:26.573 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.573 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.573 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.573 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2023-07-14T16:37:26.573 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2023-07-14T16:37:26.574 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.574 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.574 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.574 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.574 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.574 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.574 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.574 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T16:37:26.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.575 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2023-07-14T16:37:26.575 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2023-07-14T16:37:26.575 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.575 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.576 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.576 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.576 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.576 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.576 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.577 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.577 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2023-07-14T16:37:26.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2023-07-14T16:37:26.577 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.577 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.577 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.578 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.578 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.578 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.578 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.578 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.579 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2023-07-14T16:37:26.579 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2023-07-14T16:37:26.579 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.579 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.579 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.579 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T16:37:26.579 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.580 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2023-07-14T16:37:26.581 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2023-07-14T16:37:26.581 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.581 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.581 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.581 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.581 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.582 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.582 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.582 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.582 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.582 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.582 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.583 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.583 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:26.583 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:26.583 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.583 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.583 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.583 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2023-07-14T16:37:26.584 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2023-07-14T16:37:26.584 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.584 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.584 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.584 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.585 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.585 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2023-07-14T16:37:26.585 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2023-07-14T16:37:26.585 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.586 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.586 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.586 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2023-07-14T16:37:26.586 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.587 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.587 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.587 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.587 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.588 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.588 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.589 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2023-07-14T16:37:26.589 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2023-07-14T16:37:26.589 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.589 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.591 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.591 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.591 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.591 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.592 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2023-07-14T16:37:26.592 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2023-07-14T16:37:26.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.593 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.593 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.593 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.593 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.593 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.594 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.594 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.594 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.594 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.594 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.594 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.594 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.594 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.595 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.595 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.595 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.596 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.596 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.596 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.596 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.596 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.596 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.597 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.597 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.597 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.597 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.597 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.597 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.598 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.598 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.598 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.598 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.598 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.598 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.599 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.599 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.599 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.600 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.600 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.601 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.601 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.601 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.601 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.602 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.602 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.602 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.602 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.602 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.602 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.602 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.602 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.603 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.603 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.603 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.603 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.604 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.604 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.604 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.604 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.604 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.605 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2023-07-14T16:37:26.605 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2023-07-14T16:37:26.605 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2023-07-14T16:37:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2023-07-14T16:37:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2023-07-14T16:37:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2023-07-14T16:37:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.610 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T16:37:26.610 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.610 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.610 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.610 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.610 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.611 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.611 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.611 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.611 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2023-07-14T16:37:26.611 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.611 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.611 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.612 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.612 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.612 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.612 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.612 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.612 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.612 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.612 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.613 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.613 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.613 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.613 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.613 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.613 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.613 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.613 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.614 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.614 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.614 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.615 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.615 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.615 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.615 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.615 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.615 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.615 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.615 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.616 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.616 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.616 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.616 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.616 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.616 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.617 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.617 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.617 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.617 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.617 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.618 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.618 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.618 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.618 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.618 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.618 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.618 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.618 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.619 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.619 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.619 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.620 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.621 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.621 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.621 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.621 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2023-07-14T16:37:26.622 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.622 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.622 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.622 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.622 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.622 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.623 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.623 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.623 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.623 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.624 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.624 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.624 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.624 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.624 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.624 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.625 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.625 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.625 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.625 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.626 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.626 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.626 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.626 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.627 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.628 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.628 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.628 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.628 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.629 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.629 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.629 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.629 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.629 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.629 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.629 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.630 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.630 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.630 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.630 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.630 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.630 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.630 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.630 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.631 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.631 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.631 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.631 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2023-07-14T16:37:26.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2023-07-14T16:37:26.632 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.632 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.632 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.632 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2023-07-14T16:37:26.632 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.632 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.633 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T16:37:26.633 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.633 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.633 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.633 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.633 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2023-07-14T16:37:26.633 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2023-07-14T16:37:26.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2023-07-14T16:37:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T16:37:26.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.635 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.635 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.635 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.636 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2023-07-14T16:37:26.636 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2023-07-14T16:37:26.636 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2023-07-14T16:37:26.636 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2023-07-14T16:37:26.636 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2023-07-14T16:37:26.636 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2023-07-14T16:37:26.636 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2023-07-14T16:37:26.637 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.637 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.637 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2023-07-14T16:37:26.637 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.637 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.637 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.637 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.637 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.638 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2023-07-14T16:37:26.638 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2023-07-14T16:37:26.638 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.638 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.639 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.639 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.640 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2023-07-14T16:37:26.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2023-07-14T16:37:26.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2023-07-14T16:37:26.640 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2023-07-14T16:37:26.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.642 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2023-07-14T16:37:26.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2023-07-14T16:37:26.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.642 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.643 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.643 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.643 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.643 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.643 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.644 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.644 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.644 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.644 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.644 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.644 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.644 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.644 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2023-07-14T16:37:26.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2023-07-14T16:37:26.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.645 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.645 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.645 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.646 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.646 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.646 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.646 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.647 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2023-07-14T16:37:26.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2023-07-14T16:37:26.647 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.647 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.647 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.647 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2023-07-14T16:37:26.647 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.647 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2023-07-14T16:37:26.648 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2023-07-14T16:37:26.648 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2023-07-14T16:37:26.648 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2023-07-14T16:37:26.648 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2023-07-14T16:37:26.648 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2023-07-14T16:37:26.648 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2023-07-14T16:37:26.648 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.649 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.649 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.649 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2023-07-14T16:37:26.649 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2023-07-14T16:37:26.649 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2023-07-14T16:37:26.650 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.650 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2023-07-14T16:37:26.650 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2023-07-14T16:37:26.650 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.650 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.650 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.650 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.650 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.651 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.651 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2023-07-14T16:37:26.651 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2023-07-14T16:37:26.651 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.651 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.652 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2023-07-14T16:37:26.652 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.652 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.652 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.652 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.652 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.652 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.653 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.653 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.653 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2023-07-14T16:37:26.653 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2023-07-14T16:37:26.653 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2023-07-14T16:37:26.653 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.653 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.653 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2023-07-14T16:37:26.654 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.654 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.654 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.654 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.654 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.654 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.654 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.655 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2023-07-14T16:37:26.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2023-07-14T16:37:26.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2023-07-14T16:37:26.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2023-07-14T16:37:26.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2023-07-14T16:37:26.655 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T16:37:26.655 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2023-07-14T16:37:26.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2023-07-14T16:37:26.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2023-07-14T16:37:26.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2023-07-14T16:37:26.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2023-07-14T16:37:26.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2023-07-14T16:37:26.657 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2023-07-14T16:37:26.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.657 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.657 INFO:teuthology.orchestra.run.smithi062.stdout: "services": { 2023-07-14T16:37:26.657 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard": "https://172.21.15.62:8443/" 2023-07-14T16:37:26.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.658 INFO:teuthology.orchestra.run.smithi062.stdout: "always_on_modules": { 2023-07-14T16:37:26.658 INFO:teuthology.orchestra.run.smithi062.stdout: "octopus": [ 2023-07-14T16:37:26.658 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2023-07-14T16:37:26.658 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2023-07-14T16:37:26.658 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2023-07-14T16:37:26.658 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2023-07-14T16:37:26.658 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2023-07-14T16:37:26.658 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2023-07-14T16:37:26.659 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2023-07-14T16:37:26.659 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2023-07-14T16:37:26.659 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2023-07-14T16:37:26.659 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2023-07-14T16:37:26.659 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.659 INFO:teuthology.orchestra.run.smithi062.stdout: "pacific": [ 2023-07-14T16:37:26.659 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2023-07-14T16:37:26.659 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2023-07-14T16:37:26.660 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2023-07-14T16:37:26.660 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2023-07-14T16:37:26.660 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2023-07-14T16:37:26.660 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2023-07-14T16:37:26.660 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2023-07-14T16:37:26.660 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2023-07-14T16:37:26.660 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2023-07-14T16:37:26.661 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2023-07-14T16:37:26.661 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.661 INFO:teuthology.orchestra.run.smithi062.stdout: "quincy": [ 2023-07-14T16:37:26.661 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2023-07-14T16:37:26.661 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2023-07-14T16:37:26.661 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2023-07-14T16:37:26.661 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2023-07-14T16:37:26.661 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2023-07-14T16:37:26.662 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2023-07-14T16:37:26.662 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2023-07-14T16:37:26.662 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2023-07-14T16:37:26.662 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2023-07-14T16:37:26.662 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2023-07-14T16:37:26.662 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2023-07-14T16:37:26.662 INFO:teuthology.orchestra.run.smithi062.stdout: "reef": [ 2023-07-14T16:37:26.662 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2023-07-14T16:37:26.663 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2023-07-14T16:37:26.663 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2023-07-14T16:37:26.663 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2023-07-14T16:37:26.663 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2023-07-14T16:37:26.663 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2023-07-14T16:37:26.663 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2023-07-14T16:37:26.663 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2023-07-14T16:37:26.663 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2023-07-14T16:37:26.664 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2023-07-14T16:37:26.664 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:26.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.664 INFO:teuthology.orchestra.run.smithi062.stdout: "last_failure_osd_epoch": 3, 2023-07-14T16:37:26.664 INFO:teuthology.orchestra.run.smithi062.stdout: "active_clients": [ 2023-07-14T16:37:26.664 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.664 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2023-07-14T16:37:26.664 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2023-07-14T16:37:26.665 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2023-07-14T16:37:26.665 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2023-07-14T16:37:26.665 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2265576573 2023-07-14T16:37:26.665 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.665 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:26.665 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.665 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "libcephsqlite", 2023-07-14T16:37:26.666 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2023-07-14T16:37:26.666 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.666 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2023-07-14T16:37:26.666 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2023-07-14T16:37:26.666 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 955325623 2023-07-14T16:37:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2023-07-14T16:37:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2023-07-14T16:37:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2023-07-14T16:37:26.668 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2023-07-14T16:37:26.668 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1924318640 2023-07-14T16:37:26.668 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.668 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:26.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:26.668 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2023-07-14T16:37:26.669 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2023-07-14T16:37:26.669 INFO:teuthology.orchestra.run.smithi062.stdout: { 2023-07-14T16:37:26.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2023-07-14T16:37:26.669 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2023-07-14T16:37:26.669 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2133524705 2023-07-14T16:37:26.669 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.669 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:26.669 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:26.670 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2023-07-14T16:37:26.670 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:37:26.677 DEBUG:tasks.check_counter:Getting stats from a 2023-07-14T16:37:26.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-14T16:37:26.678 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:24 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2425351228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:37:26.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:24 smithi062 ceph-mon[95543]: pgmap v1010: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 242 KiB/s wr, 230 op/s 2023-07-14T16:37:26.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:25 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1443804460' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T16:37:26.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:26 smithi062 ceph-mon[95543]: pgmap v1011: 97 pgs: 97 active+clean; 3.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 143 KiB/s wr, 228 op/s 2023-07-14T16:37:26.680 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:26 smithi081 ceph-mon[127795]: pgmap v1011: 97 pgs: 97 active+clean; 3.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 143 KiB/s wr, 228 op/s 2023-07-14T16:37:26.680 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:26 smithi164 ceph-mon[97427]: pgmap v1011: 97 pgs: 97 active+clean; 3.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 143 KiB/s wr, 228 op/s 2023-07-14T16:37:27.615 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:37:29.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:28 smithi164 ceph-mon[97427]: pgmap v1012: 97 pgs: 97 active+clean; 882 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 75 KiB/s wr, 273 op/s 2023-07-14T16:37:29.338 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:37:29.338 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:37:29.338 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2023-07-14T16:37:29.338 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2023-07-14T16:37:29.339 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2023-07-14T16:37:29.339 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2023-07-14T16:37:29.339 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2023-07-14T16:37:29.339 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2023-07-14T16:37:29.339 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.003725917, 2023-07-14T16:37:29.339 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T16:37:29.339 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000656897, 2023-07-14T16:37:29.339 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:37:29.340 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:29.340 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:29.340 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:29.340 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:29.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.340 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:29.340 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:29.341 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:29.341 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:29.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.341 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T16:37:29.341 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T16:37:29.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.341 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:37:29.341 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 382, 2023-07-14T16:37:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 377, 2023-07-14T16:37:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 519936, 2023-07-14T16:37:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 215968, 2023-07-14T16:37:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2023-07-14T16:37:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2023-07-14T16:37:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.061193361, 2023-07-14T16:37:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.028078871, 2023-07-14T16:37:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.022269241, 2023-07-14T16:37:29.343 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.008569379, 2023-07-14T16:37:29.343 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:29.343 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 377, 2023-07-14T16:37:29.343 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.030409664, 2023-07-14T16:37:29.343 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000080662 2023-07-14T16:37:29.343 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.343 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:29.343 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:29.344 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:29.344 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:29.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.344 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 519936, 2023-07-14T16:37:29.344 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 215968 2023-07-14T16:37:29.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.344 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:37:29.345 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2, 2023-07-14T16:37:29.345 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 288, 2023-07-14T16:37:29.345 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 320, 2023-07-14T16:37:29.345 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 202080, 2023-07-14T16:37:29.345 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2023-07-14T16:37:29.345 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2023-07-14T16:37:29.345 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.024907085, 2023-07-14T16:37:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.020007182, 2023-07-14T16:37:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001874607, 2023-07-14T16:37:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:37:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 286, 2023-07-14T16:37:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.015499337, 2023-07-14T16:37:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000054193 2023-07-14T16:37:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.347 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:29.347 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:29.347 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:29.347 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:29.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.347 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T16:37:29.347 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 202080 2023-07-14T16:37:29.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.348 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2023-07-14T16:37:29.348 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2023-07-14T16:37:29.348 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2023-07-14T16:37:29.348 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.348 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2023-07-14T16:37:29.348 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:37:29.348 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2023-07-14T16:37:29.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:37:29.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:37:29.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:37:29.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:37:29.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:37:29.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:37:29.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:37:29.349 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:37:29.350 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:37:29.350 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:37:29.350 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:37:29.350 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:37:29.350 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:37:29.350 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2023-07-14T16:37:29.350 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:37:29.350 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2023-07-14T16:37:29.351 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:37:29.351 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:37:29.351 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:37:29.351 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:37:29.351 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:37:29.351 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:37:29.351 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:37:29.351 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2023-07-14T16:37:29.352 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:37:29.352 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:37:29.352 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:37:29.352 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2023-07-14T16:37:29.352 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2023-07-14T16:37:29.352 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2023-07-14T16:37:29.352 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:37:29.352 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:37:29.353 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:37:29.353 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:37:29.353 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 149735, 2023-07-14T16:37:29.353 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2023-07-14T16:37:29.353 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:37:29.353 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2023-07-14T16:37:29.353 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2023-07-14T16:37:29.353 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2023-07-14T16:37:29.354 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:37:29.354 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2023-07-14T16:37:29.354 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:37:29.354 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2023-07-14T16:37:29.354 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2023-07-14T16:37:29.354 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2023-07-14T16:37:29.354 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:37:29.354 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:37:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2023-07-14T16:37:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2023-07-14T16:37:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2023-07-14T16:37:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2023-07-14T16:37:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2023-07-14T16:37:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2023-07-14T16:37:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2023-07-14T16:37:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2023-07-14T16:37:29.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:29.356 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:37:29.356 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-07-14T16:37:29.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-07-14T16:37:29.356 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-07-14T16:37:29.356 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 380, 2023-07-14T16:37:29.356 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 463552, 2023-07-14T16:37:29.356 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:29.357 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 380, 2023-07-14T16:37:29.357 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-07-14T16:37:29.357 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-07-14T16:37:29.357 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 380, 2023-07-14T16:37:29.357 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 463552, 2023-07-14T16:37:29.357 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-07-14T16:37:29.357 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:29.357 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:29.358 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:29.358 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:29.358 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:29.358 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:37:29.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:28 smithi062 ceph-mon[95543]: pgmap v1012: 97 pgs: 97 active+clean; 882 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 75 KiB/s wr, 273 op/s 2023-07-14T16:37:29.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:28 smithi081 ceph-mon[127795]: pgmap v1012: 97 pgs: 97 active+clean; 882 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 75 KiB/s wr, 273 op/s 2023-07-14T16:37:29.810 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-07-14T16:37:29.810 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-07-14T16:37:29.811 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-07-14T16:37:29.811 DEBUG:tasks.check_counter:Getting stats from d 2023-07-14T16:37:29.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-14T16:37:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:37:31.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:31 smithi062 ceph-mon[95543]: pgmap v1013: 97 pgs: 97 active+clean; 53 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 75 KiB/s wr, 271 op/s 2023-07-14T16:37:31.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:31 smithi164 ceph-mon[97427]: pgmap v1013: 97 pgs: 97 active+clean; 53 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 75 KiB/s wr, 271 op/s 2023-07-14T16:37:31.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:31 smithi081 ceph-mon[127795]: pgmap v1013: 97 pgs: 97 active+clean; 53 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 75 KiB/s wr, 271 op/s 2023-07-14T16:37:32.145 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:37:32.145 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:37:32.145 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2023-07-14T16:37:32.146 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2023-07-14T16:37:32.146 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2023-07-14T16:37:32.146 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2023-07-14T16:37:32.146 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2023-07-14T16:37:32.147 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2023-07-14T16:37:32.147 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.003366802, 2023-07-14T16:37:32.147 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T16:37:32.147 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000595040, 2023-07-14T16:37:32.148 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:37:32.148 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:32.149 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:32.149 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:32.149 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:32.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.149 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:32.150 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:32.150 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:32.150 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:32.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.150 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T16:37:32.150 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T16:37:32.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:37:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 386, 2023-07-14T16:37:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 383, 2023-07-14T16:37:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 519824, 2023-07-14T16:37:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 220000, 2023-07-14T16:37:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2023-07-14T16:37:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2023-07-14T16:37:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.063418413, 2023-07-14T16:37:32.152 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.029221581, 2023-07-14T16:37:32.152 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.024335397, 2023-07-14T16:37:32.152 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.008484244, 2023-07-14T16:37:32.152 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:32.152 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 382, 2023-07-14T16:37:32.152 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.021550491, 2023-07-14T16:37:32.152 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000056414 2023-07-14T16:37:32.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.153 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:32.153 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:32.153 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:32.153 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:32.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.153 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 519824, 2023-07-14T16:37:32.154 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 220000 2023-07-14T16:37:32.154 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.154 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:37:32.154 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 6, 2023-07-14T16:37:32.154 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 295, 2023-07-14T16:37:32.154 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3232, 2023-07-14T16:37:32.154 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 203408, 2023-07-14T16:37:32.154 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2023-07-14T16:37:32.155 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2023-07-14T16:37:32.155 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.026085086, 2023-07-14T16:37:32.155 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.020394622, 2023-07-14T16:37:32.155 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002682836, 2023-07-14T16:37:32.155 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:37:32.155 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:32.155 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 294, 2023-07-14T16:37:32.155 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.024129531, 2023-07-14T16:37:32.156 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000082073 2023-07-14T16:37:32.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.156 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:32.156 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:32.156 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:32.156 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:32.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.156 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3232, 2023-07-14T16:37:32.157 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 203408 2023-07-14T16:37:32.157 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.157 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2023-07-14T16:37:32.157 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2023-07-14T16:37:32.157 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2023-07-14T16:37:32.157 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.157 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2023-07-14T16:37:32.157 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:37:32.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2023-07-14T16:37:32.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:37:32.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:37:32.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:37:32.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:37:32.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:37:32.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:37:32.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:37:32.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:37:32.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:37:32.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:37:32.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:37:32.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:37:32.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:37:32.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2023-07-14T16:37:32.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:37:32.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2023-07-14T16:37:32.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:37:32.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:37:32.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:37:32.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:37:32.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:37:32.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:37:32.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:37:32.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2023-07-14T16:37:32.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:37:32.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:37:32.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:37:32.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2023-07-14T16:37:32.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2023-07-14T16:37:32.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2023-07-14T16:37:32.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:37:32.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:37:32.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:37:32.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:37:32.162 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 152712, 2023-07-14T16:37:32.162 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2023-07-14T16:37:32.163 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:37:32.163 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2023-07-14T16:37:32.163 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2023-07-14T16:37:32.163 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2023-07-14T16:37:32.163 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:37:32.163 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2023-07-14T16:37:32.163 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:37:32.164 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2023-07-14T16:37:32.164 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2023-07-14T16:37:32.164 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2023-07-14T16:37:32.164 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:37:32.164 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:37:32.164 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2023-07-14T16:37:32.164 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2023-07-14T16:37:32.165 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2023-07-14T16:37:32.165 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2023-07-14T16:37:32.165 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2023-07-14T16:37:32.165 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2023-07-14T16:37:32.165 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2023-07-14T16:37:32.165 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2023-07-14T16:37:32.165 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:32.165 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:37:32.166 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-07-14T16:37:32.166 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-07-14T16:37:32.166 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-07-14T16:37:32.166 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 388, 2023-07-14T16:37:32.166 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 465229, 2023-07-14T16:37:32.166 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:32.166 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 388, 2023-07-14T16:37:32.166 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 388, 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 465229, 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:32.167 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:32.168 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:37:33.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:33 smithi164 ceph-mon[97427]: pgmap v1014: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 65 KiB/s wr, 229 op/s 2023-07-14T16:37:33.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:33 smithi062 ceph-mon[95543]: pgmap v1014: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 65 KiB/s wr, 229 op/s 2023-07-14T16:37:33.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:33 smithi081 ceph-mon[127795]: pgmap v1014: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 65 KiB/s wr, 229 op/s 2023-07-14T16:37:35.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:35 smithi164 ceph-mon[97427]: pgmap v1015: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 183 op/s 2023-07-14T16:37:35.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:35 smithi062 ceph-mon[95543]: pgmap v1015: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 183 op/s 2023-07-14T16:37:35.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:35 smithi081 ceph-mon[127795]: pgmap v1015: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 KiB/s wr, 183 op/s 2023-07-14T16:37:37.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:37 smithi164 ceph-mon[97427]: pgmap v1016: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 114 op/s 2023-07-14T16:37:37.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:37 smithi062 ceph-mon[95543]: pgmap v1016: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 114 op/s 2023-07-14T16:37:37.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:37 smithi081 ceph-mon[127795]: pgmap v1016: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 114 op/s 2023-07-14T16:37:38.756 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-14T16:37:38.756 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-14T16:37:38.756 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-07-14T16:37:38.757 DEBUG:tasks.check_counter:Getting stats from g 2023-07-14T16:37:38.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-14T16:37:39.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:39 smithi164 ceph-mon[97427]: pgmap v1017: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 91 op/s 2023-07-14T16:37:39.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:39 smithi062 ceph-mon[95543]: pgmap v1017: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 91 op/s 2023-07-14T16:37:39.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:39 smithi081 ceph-mon[127795]: pgmap v1017: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.9 KiB/s wr, 91 op/s 2023-07-14T16:37:39.509 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:37:41.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:41 smithi081 ceph-mon[127795]: pgmap v1018: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 36 op/s 2023-07-14T16:37:41.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:41 smithi164 ceph-mon[97427]: pgmap v1018: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 36 op/s 2023-07-14T16:37:41.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:41 smithi062 ceph-mon[95543]: pgmap v1018: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 36 op/s 2023-07-14T16:37:43.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:43 smithi164 ceph-mon[97427]: pgmap v1019: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-07-14T16:37:43.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:43 smithi062 ceph-mon[95543]: pgmap v1019: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-07-14T16:37:43.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:43 smithi081 ceph-mon[127795]: pgmap v1019: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-07-14T16:37:45.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:45 smithi164 ceph-mon[97427]: pgmap v1020: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:37:45.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:45 smithi062 ceph-mon[95543]: pgmap v1020: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:37:45.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:45 smithi081 ceph-mon[127795]: pgmap v1020: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:37:47.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:47 smithi164 ceph-mon[97427]: pgmap v1021: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:37:47.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:47 smithi062 ceph-mon[95543]: pgmap v1021: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:37:47.377 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:37:47.377 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:37:47.378 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 4, 2023-07-14T16:37:47.378 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 4, 2023-07-14T16:37:47.378 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 2912, 2023-07-14T16:37:47.378 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 448, 2023-07-14T16:37:47.379 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2023-07-14T16:37:47.379 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2023-07-14T16:37:47.379 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.004211050, 2023-07-14T16:37:47.380 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000094090, 2023-07-14T16:37:47.380 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001178130, 2023-07-14T16:37:47.380 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:37:47.380 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:47.381 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3, 2023-07-14T16:37:47.381 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.001423628, 2023-07-14T16:37:47.381 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000474542 2023-07-14T16:37:47.382 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.382 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:47.382 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:47.382 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:47.383 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:47.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.383 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 2912, 2023-07-14T16:37:47.384 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T16:37:47.384 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.384 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:37:47.384 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 385, 2023-07-14T16:37:47.385 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 382, 2023-07-14T16:37:47.385 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 519552, 2023-07-14T16:37:47.385 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 219408, 2023-07-14T16:37:47.386 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2023-07-14T16:37:47.386 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2023-07-14T16:37:47.386 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.062137096, 2023-07-14T16:37:47.386 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.029352660, 2023-07-14T16:37:47.387 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.022395559, 2023-07-14T16:37:47.387 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.008517797, 2023-07-14T16:37:47.387 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:47.388 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 381, 2023-07-14T16:37:47.388 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.027972468, 2023-07-14T16:37:47.388 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000073418 2023-07-14T16:37:47.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.389 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:47.389 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:47.389 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:47.390 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:47.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.390 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 519552, 2023-07-14T16:37:47.390 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 219408 2023-07-14T16:37:47.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.391 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:37:47.391 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2, 2023-07-14T16:37:47.391 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 292, 2023-07-14T16:37:47.392 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 320, 2023-07-14T16:37:47.392 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 202784, 2023-07-14T16:37:47.392 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2023-07-14T16:37:47.392 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2023-07-14T16:37:47.393 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.026509202, 2023-07-14T16:37:47.393 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.021345957, 2023-07-14T16:37:47.393 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002029183, 2023-07-14T16:37:47.394 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:37:47.394 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:47.394 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 290, 2023-07-14T16:37:47.394 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.015124438, 2023-07-14T16:37:47.395 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000052153 2023-07-14T16:37:47.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.395 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:47.395 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:47.396 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:47.396 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:47.396 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.397 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T16:37:47.397 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 202784 2023-07-14T16:37:47.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.397 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2023-07-14T16:37:47.397 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2023-07-14T16:37:47.397 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2023-07-14T16:37:47.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.398 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2023-07-14T16:37:47.398 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:37:47.398 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2023-07-14T16:37:47.398 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:37:47.398 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:37:47.398 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:37:47.398 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:37:47.398 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:37:47.399 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:37:47.399 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:37:47.399 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:37:47.399 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:37:47.399 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:37:47.399 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:37:47.399 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:37:47.400 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:37:47.400 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2023-07-14T16:37:47.400 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:37:47.400 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2023-07-14T16:37:47.400 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:37:47.400 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:37:47.401 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:37:47.401 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:37:47.401 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:37:47.402 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:37:47.402 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:37:47.402 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2023-07-14T16:37:47.402 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:37:47.402 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:37:47.402 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:37:47.402 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2023-07-14T16:37:47.402 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2023-07-14T16:37:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2023-07-14T16:37:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:37:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:37:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:37:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:37:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148704, 2023-07-14T16:37:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 29, 2023-07-14T16:37:47.403 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:37:47.404 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2023-07-14T16:37:47.404 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2023-07-14T16:37:47.404 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2023-07-14T16:37:47.404 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:37:47.404 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2023-07-14T16:37:47.404 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:37:47.404 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2023-07-14T16:37:47.405 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2023-07-14T16:37:47.405 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2023-07-14T16:37:47.405 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:37:47.405 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:37:47.405 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2023-07-14T16:37:47.405 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2023-07-14T16:37:47.405 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2023-07-14T16:37:47.405 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2023-07-14T16:37:47.406 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2023-07-14T16:37:47.406 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2023-07-14T16:37:47.406 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2023-07-14T16:37:47.406 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2023-07-14T16:37:47.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:47.406 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:37:47.406 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-07-14T16:37:47.406 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-07-14T16:37:47.407 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-07-14T16:37:47.407 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 387, 2023-07-14T16:37:47.407 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 465099, 2023-07-14T16:37:47.407 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:47.407 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 387, 2023-07-14T16:37:47.407 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-07-14T16:37:47.407 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-07-14T16:37:47.407 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 387, 2023-07-14T16:37:47.408 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 465099, 2023-07-14T16:37:47.408 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-07-14T16:37:47.408 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:47.408 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:47.408 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:47.408 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:47.408 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:47.409 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:37:47.409 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:47 smithi081 ceph-mon[127795]: pgmap v1021: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-07-14T16:37:47.825 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-07-14T16:37:47.825 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-07-14T16:37:47.826 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-07-14T16:37:47.826 DEBUG:tasks.check_counter:Getting stats from j 2023-07-14T16:37:47.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-14T16:37:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T16:37:49.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:49 smithi062 ceph-mon[95543]: pgmap v1022: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-14T16:37:49.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:49 smithi164 ceph-mon[97427]: pgmap v1022: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-14T16:37:49.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:49 smithi081 ceph-mon[127795]: pgmap v1022: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-07-14T16:37:50.052 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-07-14T16:37:50.052 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:37:50.052 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2023-07-14T16:37:50.052 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2023-07-14T16:37:50.052 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2023-07-14T16:37:50.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2023-07-14T16:37:50.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2023-07-14T16:37:50.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2023-07-14T16:37:50.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.003450310, 2023-07-14T16:37:50.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T16:37:50.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000660645, 2023-07-14T16:37:50.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:37:50.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:50.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:50.054 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:50.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:50.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:50.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:50.054 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:50.055 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:50.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.055 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T16:37:50.055 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T16:37:50.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.055 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:37:50.055 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2, 2023-07-14T16:37:50.055 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 298, 2023-07-14T16:37:50.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 320, 2023-07-14T16:37:50.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 203840, 2023-07-14T16:37:50.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2023-07-14T16:37:50.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2023-07-14T16:37:50.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.026484554, 2023-07-14T16:37:50.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.021138364, 2023-07-14T16:37:50.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002179701, 2023-07-14T16:37:50.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:37:50.057 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:50.057 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 296, 2023-07-14T16:37:50.057 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.018517592, 2023-07-14T16:37:50.057 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000062559 2023-07-14T16:37:50.057 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.057 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:50.057 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:50.057 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:50.058 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:50.058 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.058 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T16:37:50.058 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 203840 2023-07-14T16:37:50.058 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.058 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:37:50.058 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 395, 2023-07-14T16:37:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 390, 2023-07-14T16:37:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 524096, 2023-07-14T16:37:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 223408, 2023-07-14T16:37:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2023-07-14T16:37:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2023-07-14T16:37:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.065882109, 2023-07-14T16:37:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.031119099, 2023-07-14T16:37:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.023969639, 2023-07-14T16:37:50.060 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.008796072, 2023-07-14T16:37:50.060 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:50.060 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 390, 2023-07-14T16:37:50.060 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.037055924, 2023-07-14T16:37:50.060 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000095015 2023-07-14T16:37:50.060 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.061 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:50.061 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:50.061 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:50.061 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:50.061 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.061 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 524096, 2023-07-14T16:37:50.062 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 223408 2023-07-14T16:37:50.062 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.062 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2023-07-14T16:37:50.062 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2023-07-14T16:37:50.062 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2023-07-14T16:37:50.062 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.062 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2023-07-14T16:37:50.062 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:37:50.063 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2023-07-14T16:37:50.063 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:37:50.063 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:37:50.063 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:37:50.063 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:37:50.063 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:37:50.063 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:37:50.064 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:37:50.064 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:37:50.064 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:37:50.064 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:37:50.064 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:37:50.064 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:37:50.064 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:37:50.064 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2023-07-14T16:37:50.065 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:37:50.065 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2023-07-14T16:37:50.065 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:37:50.065 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:37:50.065 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:37:50.065 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:37:50.065 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:37:50.065 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:37:50.066 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:37:50.066 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2023-07-14T16:37:50.066 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:37:50.066 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:37:50.066 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:37:50.066 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2023-07-14T16:37:50.066 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2023-07-14T16:37:50.066 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2023-07-14T16:37:50.067 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:37:50.067 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:37:50.067 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:37:50.067 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:37:50.067 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148721, 2023-07-14T16:37:50.067 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2023-07-14T16:37:50.067 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:37:50.067 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2023-07-14T16:37:50.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2023-07-14T16:37:50.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2023-07-14T16:37:50.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:37:50.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2023-07-14T16:37:50.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:37:50.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2023-07-14T16:37:50.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2023-07-14T16:37:50.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2023-07-14T16:37:50.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:37:50.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:37:50.069 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2023-07-14T16:37:50.069 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2023-07-14T16:37:50.069 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2023-07-14T16:37:50.069 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2023-07-14T16:37:50.069 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2023-07-14T16:37:50.069 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2023-07-14T16:37:50.070 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2023-07-14T16:37:50.070 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2023-07-14T16:37:50.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2023-07-14T16:37:50.070 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:37:50.070 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2023-07-14T16:37:50.070 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2023-07-14T16:37:50.070 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2023-07-14T16:37:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 393, 2023-07-14T16:37:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 465879, 2023-07-14T16:37:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 393, 2023-07-14T16:37:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2023-07-14T16:37:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2023-07-14T16:37:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 393, 2023-07-14T16:37:50.071 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 465879, 2023-07-14T16:37:50.072 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2023-07-14T16:37:50.072 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2023-07-14T16:37:50.072 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2023-07-14T16:37:50.072 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2023-07-14T16:37:50.072 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:50.072 INFO:teuthology.orchestra.run.smithi062.stdout: } 2023-07-14T16:37:50.072 INFO:teuthology.orchestra.run.smithi062.stdout:} 2023-07-14T16:37:50.550 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-14T16:37:50.550 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-14T16:37:50.550 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-07-14T16:37:50.550 DEBUG:tasks.check_counter:Getting stats from b 2023-07-14T16:37:50.551 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-14T16:37:51.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:51 smithi164 ceph-mon[97427]: pgmap v1023: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T16:37:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:37:51.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:51 smithi062 ceph-mon[95543]: pgmap v1023: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T16:37:51.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:51 smithi081 ceph-mon[127795]: pgmap v1023: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T16:37:52.925 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-07-14T16:37:52.926 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:37:52.926 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 6386, 2023-07-14T16:37:52.926 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 6545, 2023-07-14T16:37:52.926 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 2530508, 2023-07-14T16:37:52.926 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 15047533, 2023-07-14T16:37:52.927 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 476, 2023-07-14T16:37:52.927 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 3, 2023-07-14T16:37:52.927 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.868472089, 2023-07-14T16:37:52.927 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.243852347, 2023-07-14T16:37:52.927 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.428533546, 2023-07-14T16:37:52.927 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.110543534, 2023-07-14T16:37:52.927 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:52.928 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6547, 2023-07-14T16:37:52.928 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.900459612, 2023-07-14T16:37:52.928 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000137537 2023-07-14T16:37:52.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.928 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:52.928 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 5887, 2023-07-14T16:37:52.928 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001559771, 2023-07-14T16:37:52.929 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000264 2023-07-14T16:37:52.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.929 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T16:37:52.929 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T16:37:52.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.929 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:37:52.929 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 5539, 2023-07-14T16:37:52.929 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 5186, 2023-07-14T16:37:52.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 2769483, 2023-07-14T16:37:52.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 4148617, 2023-07-14T16:37:52.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 11, 2023-07-14T16:37:52.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2023-07-14T16:37:52.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.548981401, 2023-07-14T16:37:52.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.221268536, 2023-07-14T16:37:52.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.197247394, 2023-07-14T16:37:52.930 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.115860954, 2023-07-14T16:37:52.931 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:52.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 5188, 2023-07-14T16:37:52.931 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.158036620, 2023-07-14T16:37:52.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000223214 2023-07-14T16:37:52.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.931 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:52.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2674, 2023-07-14T16:37:52.931 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000604262, 2023-07-14T16:37:52.932 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000225 2023-07-14T16:37:52.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 649840, 2023-07-14T16:37:52.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 274480 2023-07-14T16:37:52.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.932 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:37:52.932 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 10162, 2023-07-14T16:37:52.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 11918, 2023-07-14T16:37:52.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 3739439, 2023-07-14T16:37:52.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 7689111, 2023-07-14T16:37:52.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 15, 2023-07-14T16:37:52.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 8, 2023-07-14T16:37:52.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 1.070624703, 2023-07-14T16:37:52.933 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.402422200, 2023-07-14T16:37:52.934 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.574073472, 2023-07-14T16:37:52.934 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.155939471, 2023-07-14T16:37:52.934 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:52.934 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 11910, 2023-07-14T16:37:52.934 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.307457249, 2023-07-14T16:37:52.934 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000109778 2023-07-14T16:37:52.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:52.935 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 15544, 2023-07-14T16:37:52.935 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.003326857, 2023-07-14T16:37:52.935 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000214 2023-07-14T16:37:52.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 50768, 2023-07-14T16:37:52.935 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 590496 2023-07-14T16:37:52.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.936 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2023-07-14T16:37:52.936 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2023-07-14T16:37:52.936 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2023-07-14T16:37:52.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.936 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2023-07-14T16:37:52.936 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2023-07-14T16:37:52.937 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2023-07-14T16:37:52.937 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6296, 2023-07-14T16:37:52.937 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2.859071869, 2023-07-14T16:37:52.937 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000454109 2023-07-14T16:37:52.937 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:52.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.938 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2023-07-14T16:37:52.938 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2023-07-14T16:37:52.938 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2023-07-14T16:37:52.938 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1890, 2023-07-14T16:37:52.938 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.089465125, 2023-07-14T16:37:52.938 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000047336 2023-07-14T16:37:52.938 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:52.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.939 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2023-07-14T16:37:52.939 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 3665, 2023-07-14T16:37:52.939 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 3146, 2023-07-14T16:37:52.939 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2023-07-14T16:37:52.939 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3146, 2023-07-14T16:37:52.940 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 47.367344509, 2023-07-14T16:37:52.940 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.015056371 2023-07-14T16:37:52.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.940 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2023-07-14T16:37:52.940 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 519, 2023-07-14T16:37:52.940 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 11, 2023-07-14T16:37:52.940 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 4, 2023-07-14T16:37:52.941 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 765, 2023-07-14T16:37:52.941 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2023-07-14T16:37:52.941 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2023-07-14T16:37:52.941 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 16, 2023-07-14T16:37:52.941 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 2, 2023-07-14T16:37:52.941 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 4, 2023-07-14T16:37:52.941 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 10, 2023-07-14T16:37:52.941 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 13, 2023-07-14T16:37:52.942 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 557, 2023-07-14T16:37:52.942 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 1, 2023-07-14T16:37:52.942 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 1, 2023-07-14T16:37:52.942 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 5, 2023-07-14T16:37:52.942 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 6316, 2023-07-14T16:37:52.942 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 5356, 2023-07-14T16:37:52.942 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 518, 2023-07-14T16:37:52.942 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 4, 2023-07-14T16:37:52.943 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2023-07-14T16:37:52.943 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2023-07-14T16:37:52.943 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 308, 2023-07-14T16:37:52.943 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 4, 2023-07-14T16:37:52.943 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2023-07-14T16:37:52.943 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 10, 2023-07-14T16:37:52.943 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 528, 2023-07-14T16:37:52.944 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 8, 2023-07-14T16:37:52.944 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 243, 2023-07-14T16:37:52.944 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2023-07-14T16:37:52.944 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 4200, 2023-07-14T16:37:52.944 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2023-07-14T16:37:52.944 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 4, 2023-07-14T16:37:52.945 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 246, 2023-07-14T16:37:52.945 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 0, 2023-07-14T16:37:52.945 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 4200, 2023-07-14T16:37:52.945 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2023-07-14T16:37:52.946 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T16:37:52.946 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2023-07-14T16:37:52.946 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 832, 2023-07-14T16:37:52.946 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 35, 2023-07-14T16:37:52.946 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 797, 2023-07-14T16:37:52.946 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 3368, 2023-07-14T16:37:52.947 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 8, 2023-07-14T16:37:52.947 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 1051, 2023-07-14T16:37:52.947 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 1502, 2023-07-14T16:37:52.947 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 5, 2023-07-14T16:37:52.947 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T16:37:52.947 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T16:37:52.948 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 2771, 2023-07-14T16:37:52.948 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 1041, 2023-07-14T16:37:52.948 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 26, 2023-07-14T16:37:52.948 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 1937 2023-07-14T16:37:52.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.948 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2023-07-14T16:37:52.948 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 2, 2023-07-14T16:37:52.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2023-07-14T16:37:52.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2023-07-14T16:37:52.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 4, 2023-07-14T16:37:52.949 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 15, 2023-07-14T16:37:52.949 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2023-07-14T16:37:52.949 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2023-07-14T16:37:52.949 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2023-07-14T16:37:52.949 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 555, 2023-07-14T16:37:52.950 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 494, 2023-07-14T16:37:52.950 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 240, 2023-07-14T16:37:52.950 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2023-07-14T16:37:52.950 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2023-07-14T16:37:52.950 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2023-07-14T16:37:52.950 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2023-07-14T16:37:52.950 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2023-07-14T16:37:52.950 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2023-07-14T16:37:52.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 35, 2023-07-14T16:37:52.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 11, 2023-07-14T16:37:52.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2023-07-14T16:37:52.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2023-07-14T16:37:52.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2023-07-14T16:37:52.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2023-07-14T16:37:52.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.951 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2023-07-14T16:37:52.952 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 4248, 2023-07-14T16:37:52.952 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 4133, 2023-07-14T16:37:52.952 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 4133, 2023-07-14T16:37:52.952 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 115, 2023-07-14T16:37:52.952 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2023-07-14T16:37:52.952 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 25, 2023-07-14T16:37:52.952 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 99, 2023-07-14T16:37:52.953 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 98, 2023-07-14T16:37:52.953 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 98, 2023-07-14T16:37:52.953 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2023-07-14T16:37:52.953 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2023-07-14T16:37:52.953 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 1, 2023-07-14T16:37:52.953 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 13857916, 2023-07-14T16:37:52.953 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 14044017, 2023-07-14T16:37:52.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2023-07-14T16:37:52.954 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2023-07-14T16:37:52.954 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1012, 2023-07-14T16:37:52.954 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 9.761984053, 2023-07-14T16:37:52.954 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.009646229 2023-07-14T16:37:52.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.955 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2023-07-14T16:37:52.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.955 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2023-07-14T16:37:52.955 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 21, 2023-07-14T16:37:52.956 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 1030, 2023-07-14T16:37:52.956 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 1009, 2023-07-14T16:37:52.956 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 20, 2023-07-14T16:37:52.956 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 211, 2023-07-14T16:37:52.956 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 191, 2023-07-14T16:37:52.956 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 16, 2023-07-14T16:37:52.956 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 1301, 2023-07-14T16:37:52.957 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 1285, 2023-07-14T16:37:52.957 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 1, 2023-07-14T16:37:52.957 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 969, 2023-07-14T16:37:52.957 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 968, 2023-07-14T16:37:52.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 60344, 2023-07-14T16:37:52.957 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 207156 2023-07-14T16:37:52.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.958 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2023-07-14T16:37:52.958 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 4859, 2023-07-14T16:37:52.958 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 46, 2023-07-14T16:37:52.958 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 3665, 2023-07-14T16:37:52.959 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 28, 2023-07-14T16:37:52.959 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 120, 2023-07-14T16:37:52.959 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2023-07-14T16:37:52.959 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 618, 2023-07-14T16:37:52.959 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.640229837, 2023-07-14T16:37:52.959 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001035970 2023-07-14T16:37:52.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.960 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2023-07-14T16:37:52.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 731, 2023-07-14T16:37:52.960 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.193812722, 2023-07-14T16:37:52.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000265133 2023-07-14T16:37:52.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.960 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2023-07-14T16:37:52.960 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.961 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.961 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.961 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2023-07-14T16:37:52.961 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 61, 2023-07-14T16:37:52.961 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.209361828, 2023-07-14T16:37:52.961 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.003432161 2023-07-14T16:37:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2023-07-14T16:37:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 340, 2023-07-14T16:37:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.377120563, 2023-07-14T16:37:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.004050354 2023-07-14T16:37:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2023-07-14T16:37:52.962 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.963 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.963 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2023-07-14T16:37:52.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.963 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.963 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.964 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2023-07-14T16:37:52.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.964 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.964 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2023-07-14T16:37:52.964 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.965 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.965 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.965 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2023-07-14T16:37:52.965 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.965 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.966 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.966 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2023-07-14T16:37:52.966 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.966 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.966 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.966 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2023-07-14T16:37:52.967 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 105, 2023-07-14T16:37:52.967 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.150561039, 2023-07-14T16:37:52.967 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001433914 2023-07-14T16:37:52.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.967 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2023-07-14T16:37:52.967 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.967 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.967 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.968 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2023-07-14T16:37:52.968 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.968 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.968 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.969 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2023-07-14T16:37:52.969 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 289, 2023-07-14T16:37:52.969 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.186136148, 2023-07-14T16:37:52.969 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000644069 2023-07-14T16:37:52.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.969 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2023-07-14T16:37:52.969 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 164, 2023-07-14T16:37:52.970 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.635184661, 2023-07-14T16:37:52.970 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.003873077 2023-07-14T16:37:52.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.970 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2023-07-14T16:37:52.970 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 274, 2023-07-14T16:37:52.970 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 43.133241225, 2023-07-14T16:37:52.970 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.157420588 2023-07-14T16:37:52.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.971 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2023-07-14T16:37:52.971 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.971 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.971 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.971 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2023-07-14T16:37:52.971 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.971 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.972 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.972 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2023-07-14T16:37:52.972 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 16, 2023-07-14T16:37:52.972 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.110604207, 2023-07-14T16:37:52.972 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.006912762 2023-07-14T16:37:52.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.972 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2023-07-14T16:37:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2023-07-14T16:37:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.974 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.974 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2023-07-14T16:37:52.974 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3, 2023-07-14T16:37:52.974 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.003699102, 2023-07-14T16:37:52.974 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001233034 2023-07-14T16:37:52.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.974 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2023-07-14T16:37:52.975 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.975 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.975 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.975 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2023-07-14T16:37:52.975 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2023-07-14T16:37:52.976 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000550866, 2023-07-14T16:37:52.976 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000275433 2023-07-14T16:37:52.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.976 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2023-07-14T16:37:52.976 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.976 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.976 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.977 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2023-07-14T16:37:52.977 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6, 2023-07-14T16:37:52.977 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.019888615, 2023-07-14T16:37:52.977 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.003314769 2023-07-14T16:37:52.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.977 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2023-07-14T16:37:52.977 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 13, 2023-07-14T16:37:52.978 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.012970030, 2023-07-14T16:37:52.978 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000997694 2023-07-14T16:37:52.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.978 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2023-07-14T16:37:52.978 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 524, 2023-07-14T16:37:52.978 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.693983666, 2023-07-14T16:37:52.978 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001324396 2023-07-14T16:37:52.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.979 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2023-07-14T16:37:52.979 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2023-07-14T16:37:52.979 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2023-07-14T16:37:52.979 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:52.979 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:52.979 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:52.979 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:52.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.980 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2023-07-14T16:37:52.980 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 1, 2023-07-14T16:37:52.980 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 2, 2023-07-14T16:37:52.980 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 1, 2023-07-14T16:37:52.980 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 1, 2023-07-14T16:37:52.980 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2023-07-14T16:37:52.981 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 173, 2023-07-14T16:37:52.981 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 173, 2023-07-14T16:37:52.981 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 1329 2023-07-14T16:37:52.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.981 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2023-07-14T16:37:52.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 240, 2023-07-14T16:37:52.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 240, 2023-07-14T16:37:52.981 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:37:52.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:37:52.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:37:52.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:37:52.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:37:52.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:37:52.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:37:52.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:37:52.982 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:37:52.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:37:52.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:37:52.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:37:52.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:37:52.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2023-07-14T16:37:52.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:37:52.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2023-07-14T16:37:52.983 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:37:52.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:37:52.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:37:52.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:37:52.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:37:52.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:37:52.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:37:52.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2023-07-14T16:37:52.984 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:37:52.985 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:37:52.985 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:37:52.985 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2023-07-14T16:37:52.985 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2023-07-14T16:37:52.985 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2023-07-14T16:37:52.985 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:37:52.985 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:37:52.986 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:37:52.986 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:37:52.986 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 283089, 2023-07-14T16:37:52.986 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 106, 2023-07-14T16:37:52.986 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:37:52.986 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2023-07-14T16:37:52.986 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2023-07-14T16:37:52.987 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2023-07-14T16:37:52.987 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:37:52.987 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2023-07-14T16:37:52.987 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:37:52.987 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2023-07-14T16:37:52.987 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 7528, 2023-07-14T16:37:52.987 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 158, 2023-07-14T16:37:52.987 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:37:52.988 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:37:52.988 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2023-07-14T16:37:52.988 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2023-07-14T16:37:52.988 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 85613, 2023-07-14T16:37:52.988 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 505, 2023-07-14T16:37:52.988 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2023-07-14T16:37:52.988 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2023-07-14T16:37:52.988 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2023-07-14T16:37:52.989 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2023-07-14T16:37:52.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.989 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2023-07-14T16:37:52.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2023-07-14T16:37:52.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2023-07-14T16:37:52.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 9176, 2023-07-14T16:37:52.989 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 11257862, 2023-07-14T16:37:52.990 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 31, 2023-07-14T16:37:52.990 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 9176, 2023-07-14T16:37:52.990 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2023-07-14T16:37:52.990 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 9176, 2023-07-14T16:37:52.990 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 135.407437075, 2023-07-14T16:37:52.990 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.014756695 2023-07-14T16:37:52.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.990 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2023-07-14T16:37:52.991 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2023-07-14T16:37:52.991 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 9176, 2023-07-14T16:37:52.991 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 11729 2023-07-14T16:37:52.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.991 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 9176, 2023-07-14T16:37:52.991 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 4328, 2023-07-14T16:37:52.991 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 4848, 2023-07-14T16:37:52.991 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2023-07-14T16:37:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2023-07-14T16:37:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 667, 2023-07-14T16:37:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 431, 2023-07-14T16:37:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 78, 2023-07-14T16:37:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 1055, 2023-07-14T16:37:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 325, 2023-07-14T16:37:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2023-07-14T16:37:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2023-07-14T16:37:52.993 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2023-07-14T16:37:52.993 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2023-07-14T16:37:52.993 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 2066, 2023-07-14T16:37:52.993 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2023-07-14T16:37:52.993 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2023-07-14T16:37:52.993 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2023-07-14T16:37:52.993 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 4250, 2023-07-14T16:37:52.993 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 866, 2023-07-14T16:37:52.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2023-07-14T16:37:52.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2023-07-14T16:37:52.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2023-07-14T16:37:52.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2023-07-14T16:37:52.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2023-07-14T16:37:52.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2023-07-14T16:37:52.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T16:37:52.994 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2023-07-14T16:37:52.995 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2023-07-14T16:37:52.995 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 5, 2023-07-14T16:37:52.995 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2023-07-14T16:37:52.995 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2023-07-14T16:37:52.995 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2023-07-14T16:37:52.995 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2023-07-14T16:37:52.995 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2023-07-14T16:37:52.995 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2023-07-14T16:37:52.996 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2023-07-14T16:37:52.996 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2023-07-14T16:37:52.996 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2023-07-14T16:37:52.996 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2023-07-14T16:37:52.996 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2023-07-14T16:37:52.996 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2023-07-14T16:37:52.996 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2023-07-14T16:37:52.997 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2023-07-14T16:37:52.997 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2023-07-14T16:37:52.997 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2023-07-14T16:37:52.997 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 88, 2023-07-14T16:37:52.997 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2023-07-14T16:37:52.997 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 3, 2023-07-14T16:37:52.997 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2023-07-14T16:37:52.997 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 12, 2023-07-14T16:37:52.998 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2023-07-14T16:37:52.998 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2023-07-14T16:37:52.998 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 1798, 2023-07-14T16:37:52.998 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 30, 2023-07-14T16:37:52.998 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 158 2023-07-14T16:37:52.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.998 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2023-07-14T16:37:52.998 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2023-07-14T16:37:52.999 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2023-07-14T16:37:52.999 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2023-07-14T16:37:52.999 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2023-07-14T16:37:52.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.999 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2023-07-14T16:37:52.999 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 3952 2023-07-14T16:37:52.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:52.999 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2023-07-14T16:37:53.000 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2023-07-14T16:37:53.000 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 25, 2023-07-14T16:37:53.000 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2023-07-14T16:37:53.000 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 10, 2023-07-14T16:37:53.000 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 494, 2023-07-14T16:37:53.000 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2023-07-14T16:37:53.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:53.000 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:37:53.001 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:53.001 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2023-07-14T16:37:53.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:53.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 22083, 2023-07-14T16:37:53.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 7017673, 2023-07-14T16:37:53.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:53.001 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 22083, 2023-07-14T16:37:53.001 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:53.002 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:53.002 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 22083, 2023-07-14T16:37:53.002 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 7017673, 2023-07-14T16:37:53.002 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:53.002 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:53.002 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:53.002 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:53.002 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:53.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:53.003 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2023-07-14T16:37:53.003 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:53.003 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2023-07-14T16:37:53.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:53.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2023-07-14T16:37:53.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2023-07-14T16:37:53.003 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:53.004 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2023-07-14T16:37:53.004 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 9176, 2023-07-14T16:37:53.004 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 11307564, 2023-07-14T16:37:53.004 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 9176, 2023-07-14T16:37:53.004 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 11307564, 2023-07-14T16:37:53.004 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:53.004 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:53.004 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:53.005 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:53.005 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:53.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:53.005 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2023-07-14T16:37:53.005 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:53.005 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2023-07-14T16:37:53.005 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:53.006 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2023-07-14T16:37:53.006 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2023-07-14T16:37:53.006 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:53.006 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2023-07-14T16:37:53.006 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 9176, 2023-07-14T16:37:53.006 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 9176, 2023-07-14T16:37:53.006 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 9176, 2023-07-14T16:37:53.006 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 9176, 2023-07-14T16:37:53.007 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:53.007 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:53.007 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:53.007 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:53.007 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:53.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:53.007 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2023-07-14T16:37:53.007 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:53.008 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2023-07-14T16:37:53.008 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:53.008 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 494, 2023-07-14T16:37:53.008 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 49894, 2023-07-14T16:37:53.008 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:53.008 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 494, 2023-07-14T16:37:53.008 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:53.008 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:53.009 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 43, 2023-07-14T16:37:53.009 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 49894, 2023-07-14T16:37:53.009 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:53.009 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:53.009 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:53.009 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:53.009 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:53.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:53.010 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x5565debd00a0": { 2023-07-14T16:37:53.010 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:53.010 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2023-07-14T16:37:53.010 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:53.010 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 4248, 2023-07-14T16:37:53.010 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 9849713, 2023-07-14T16:37:53.010 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:53.010 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 4248, 2023-07-14T16:37:53.011 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:53.011 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:53.011 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 1012, 2023-07-14T16:37:53.011 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 9849713, 2023-07-14T16:37:53.011 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:53.011 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:53.011 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:53.011 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:53.012 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:53.012 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:53.012 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-07-14T16:37:53.268 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:53 smithi164 ceph-mon[97427]: pgmap v1024: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T16:37:53.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:53 smithi062 ceph-mon[95543]: pgmap v1024: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T16:37:53.368 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:53 smithi081 ceph-mon[127795]: pgmap v1024: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T16:37:53.372 INFO:tasks.check_counter:Daemon mds.b mds.exported=10 2023-07-14T16:37:53.372 INFO:tasks.check_counter:Daemon mds.b mds.imported=8 2023-07-14T16:37:53.372 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2023-07-14T16:37:53.372 DEBUG:tasks.check_counter:Getting stats from e 2023-07-14T16:37:53.373 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-14T16:37:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:37:55.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:55 smithi164 ceph-mon[97427]: pgmap v1025: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T16:37:55.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:55 smithi062 ceph-mon[95543]: pgmap v1025: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T16:37:55.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:55 smithi081 ceph-mon[127795]: pgmap v1025: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-07-14T16:37:55.856 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-07-14T16:37:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:37:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 2959, 2023-07-14T16:37:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 4198, 2023-07-14T16:37:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 900261, 2023-07-14T16:37:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 4353944, 2023-07-14T16:37:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 28, 2023-07-14T16:37:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 7, 2023-07-14T16:37:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.343168942, 2023-07-14T16:37:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.129208974, 2023-07-14T16:37:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.182874182, 2023-07-14T16:37:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.036961800, 2023-07-14T16:37:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4196, 2023-07-14T16:37:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.924414459, 2023-07-14T16:37:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000220308 2023-07-14T16:37:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3926, 2023-07-14T16:37:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001102265, 2023-07-14T16:37:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000280 2023-07-14T16:37:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 912, 2023-07-14T16:37:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 688 2023-07-14T16:37:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:37:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1708, 2023-07-14T16:37:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 1671, 2023-07-14T16:37:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 1214855, 2023-07-14T16:37:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 716788, 2023-07-14T16:37:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 9, 2023-07-14T16:37:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2023-07-14T16:37:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.194184898, 2023-07-14T16:37:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.076008581, 2023-07-14T16:37:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.094569903, 2023-07-14T16:37:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.027675624, 2023-07-14T16:37:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1671, 2023-07-14T16:37:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.208795215, 2023-07-14T16:37:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000124952 2023-07-14T16:37:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1247, 2023-07-14T16:37:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000272317, 2023-07-14T16:37:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000218 2023-07-14T16:37:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 635280, 2023-07-14T16:37:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 253872 2023-07-14T16:37:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:37:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1965, 2023-07-14T16:37:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 2586, 2023-07-14T16:37:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 896219, 2023-07-14T16:37:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 2039125, 2023-07-14T16:37:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 70, 2023-07-14T16:37:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2023-07-14T16:37:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.313042953, 2023-07-14T16:37:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.146115321, 2023-07-14T16:37:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.129800923, 2023-07-14T16:37:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.027285911, 2023-07-14T16:37:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2584, 2023-07-14T16:37:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.473939007, 2023-07-14T16:37:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000183412 2023-07-14T16:37:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2638, 2023-07-14T16:37:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000662216, 2023-07-14T16:37:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000251 2023-07-14T16:37:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T16:37:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 608496 2023-07-14T16:37:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2023-07-14T16:37:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2023-07-14T16:37:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2023-07-14T16:37:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2023-07-14T16:37:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2023-07-14T16:37:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2023-07-14T16:37:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1415, 2023-07-14T16:37:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.756018381, 2023-07-14T16:37:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000534288 2023-07-14T16:37:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2023-07-14T16:37:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2023-07-14T16:37:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2023-07-14T16:37:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1547, 2023-07-14T16:37:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.069263938, 2023-07-14T16:37:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000044773 2023-07-14T16:37:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2023-07-14T16:37:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 1179, 2023-07-14T16:37:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 1178, 2023-07-14T16:37:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2023-07-14T16:37:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1178, 2023-07-14T16:37:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 3.361039029, 2023-07-14T16:37:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002853174 2023-07-14T16:37:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2023-07-14T16:37:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 1, 2023-07-14T16:37:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2023-07-14T16:37:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2023-07-14T16:37:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 11, 2023-07-14T16:37:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2023-07-14T16:37:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2023-07-14T16:37:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 18, 2023-07-14T16:37:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 1, 2023-07-14T16:37:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 3, 2023-07-14T16:37:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 14, 2023-07-14T16:37:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 16, 2023-07-14T16:37:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 16, 2023-07-14T16:37:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 1, 2023-07-14T16:37:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 1, 2023-07-14T16:37:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 4, 2023-07-14T16:37:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 1830, 2023-07-14T16:37:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 1801, 2023-07-14T16:37:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2023-07-14T16:37:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 4, 2023-07-14T16:37:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2023-07-14T16:37:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2023-07-14T16:37:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2023-07-14T16:37:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 10, 2023-07-14T16:37:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2023-07-14T16:37:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 3, 2023-07-14T16:37:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 13, 2023-07-14T16:37:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 5, 2023-07-14T16:37:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 363, 2023-07-14T16:37:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2023-07-14T16:37:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 562, 2023-07-14T16:37:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2023-07-14T16:37:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 324, 2023-07-14T16:37:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 13424350693, 2023-07-14T16:37:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 0, 2023-07-14T16:37:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 562, 2023-07-14T16:37:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2023-07-14T16:37:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T16:37:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2023-07-14T16:37:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 82, 2023-07-14T16:37:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T16:37:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 82, 2023-07-14T16:37:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 480, 2023-07-14T16:37:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2023-07-14T16:37:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 163, 2023-07-14T16:37:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 644, 2023-07-14T16:37:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T16:37:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T16:37:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T16:37:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 543, 2023-07-14T16:37:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 350, 2023-07-14T16:37:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 8, 2023-07-14T16:37:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 627 2023-07-14T16:37:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2023-07-14T16:37:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2023-07-14T16:37:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 3, 2023-07-14T16:37:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 2, 2023-07-14T16:37:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 7, 2023-07-14T16:37:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 4, 2023-07-14T16:37:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 1, 2023-07-14T16:37:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2023-07-14T16:37:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2023-07-14T16:37:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 313, 2023-07-14T16:37:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 312, 2023-07-14T16:37:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2023-07-14T16:37:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2023-07-14T16:37:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2023-07-14T16:37:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2023-07-14T16:37:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2023-07-14T16:37:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2023-07-14T16:37:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2023-07-14T16:37:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T16:37:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 3, 2023-07-14T16:37:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2023-07-14T16:37:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2023-07-14T16:37:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2023-07-14T16:37:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2023-07-14T16:37:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2023-07-14T16:37:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 1394, 2023-07-14T16:37:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2023-07-14T16:37:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2023-07-14T16:37:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 1394, 2023-07-14T16:37:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2023-07-14T16:37:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2023-07-14T16:37:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 2, 2023-07-14T16:37:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2023-07-14T16:37:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2023-07-14T16:37:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2023-07-14T16:37:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2023-07-14T16:37:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2023-07-14T16:37:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2023-07-14T16:37:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 6549773, 2023-07-14T16:37:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2023-07-14T16:37:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2023-07-14T16:37:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 451, 2023-07-14T16:37:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 3.614337133, 2023-07-14T16:37:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.008014051 2023-07-14T16:37:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2023-07-14T16:37:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2023-07-14T16:37:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 20, 2023-07-14T16:37:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 334, 2023-07-14T16:37:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 314, 2023-07-14T16:37:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 19, 2023-07-14T16:37:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 103, 2023-07-14T16:37:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 84, 2023-07-14T16:37:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 18, 2023-07-14T16:37:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 694, 2023-07-14T16:37:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 676, 2023-07-14T16:37:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 1, 2023-07-14T16:37:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 315, 2023-07-14T16:37:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 314, 2023-07-14T16:37:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 56116, 2023-07-14T16:37:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 198964 2023-07-14T16:37:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2023-07-14T16:37:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 1513, 2023-07-14T16:37:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 4, 2023-07-14T16:37:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 1179, 2023-07-14T16:37:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 20, 2023-07-14T16:37:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 18, 2023-07-14T16:37:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2023-07-14T16:37:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2, 2023-07-14T16:37:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.009809710, 2023-07-14T16:37:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.004904855 2023-07-14T16:37:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2023-07-14T16:37:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 461, 2023-07-14T16:37:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.166632829, 2023-07-14T16:37:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000361459 2023-07-14T16:37:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2023-07-14T16:37:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2023-07-14T16:37:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2023-07-14T16:37:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 167, 2023-07-14T16:37:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.123684096, 2023-07-14T16:37:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000740623 2023-07-14T16:37:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2023-07-14T16:37:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2023-07-14T16:37:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2023-07-14T16:37:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2023-07-14T16:37:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2023-07-14T16:37:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2023-07-14T16:37:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2023-07-14T16:37:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2023-07-14T16:37:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2023-07-14T16:37:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2023-07-14T16:37:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 223, 2023-07-14T16:37:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.159349551, 2023-07-14T16:37:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000714571 2023-07-14T16:37:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2023-07-14T16:37:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12, 2023-07-14T16:37:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.020253610, 2023-07-14T16:37:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001687800 2023-07-14T16:37:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2023-07-14T16:37:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2023-07-14T16:37:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2023-07-14T16:37:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2023-07-14T16:37:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 81, 2023-07-14T16:37:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2.509925605, 2023-07-14T16:37:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.030986735 2023-07-14T16:37:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2023-07-14T16:37:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2023-07-14T16:37:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2023-07-14T16:37:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2023-07-14T16:37:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2023-07-14T16:37:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2023-07-14T16:37:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.902 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.902 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.902 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2023-07-14T16:37:55.902 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.902 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.902 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.903 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2023-07-14T16:37:55.903 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.903 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.903 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.903 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2023-07-14T16:37:55.903 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 232, 2023-07-14T16:37:55.903 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.371383628, 2023-07-14T16:37:55.904 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001600791 2023-07-14T16:37:55.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.904 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2023-07-14T16:37:55.904 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2023-07-14T16:37:55.904 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2023-07-14T16:37:55.904 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.904 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.904 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.905 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.905 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2023-07-14T16:37:55.905 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 1, 2023-07-14T16:37:55.905 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 2, 2023-07-14T16:37:55.905 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 1, 2023-07-14T16:37:55.905 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 1, 2023-07-14T16:37:55.905 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2023-07-14T16:37:55.906 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 873, 2023-07-14T16:37:55.906 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 873, 2023-07-14T16:37:55.906 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 1336 2023-07-14T16:37:55.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.906 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2023-07-14T16:37:55.906 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 120, 2023-07-14T16:37:55.906 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 120, 2023-07-14T16:37:55.906 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:37:55.907 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:37:55.907 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:37:55.907 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:37:55.907 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:37:55.907 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:37:55.907 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:37:55.907 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:37:55.907 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:37:55.908 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:37:55.908 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:37:55.908 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:37:55.908 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:37:55.908 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2023-07-14T16:37:55.908 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:37:55.908 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2023-07-14T16:37:55.908 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:37:55.909 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:37:55.909 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:37:55.909 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:37:55.909 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:37:55.909 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:37:55.909 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:37:55.909 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2023-07-14T16:37:55.909 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:37:55.910 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:37:55.910 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:37:55.910 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2023-07-14T16:37:55.910 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2023-07-14T16:37:55.910 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2023-07-14T16:37:55.910 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:37:55.910 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:37:55.910 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:37:55.911 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:37:55.911 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 301136, 2023-07-14T16:37:55.911 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 123, 2023-07-14T16:37:55.911 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:37:55.911 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2023-07-14T16:37:55.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2023-07-14T16:37:55.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2023-07-14T16:37:55.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:37:55.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2023-07-14T16:37:55.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:37:55.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2023-07-14T16:37:55.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 7528, 2023-07-14T16:37:55.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 158, 2023-07-14T16:37:55.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:37:55.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:37:55.912 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2023-07-14T16:37:55.913 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2023-07-14T16:37:55.913 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 83078, 2023-07-14T16:37:55.913 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 519, 2023-07-14T16:37:55.913 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2023-07-14T16:37:55.913 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2023-07-14T16:37:55.913 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2023-07-14T16:37:55.913 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2023-07-14T16:37:55.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.914 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2023-07-14T16:37:55.914 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2023-07-14T16:37:55.914 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2023-07-14T16:37:55.914 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 2447, 2023-07-14T16:37:55.914 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 2433999, 2023-07-14T16:37:55.914 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 33, 2023-07-14T16:37:55.914 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 2447, 2023-07-14T16:37:55.914 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2023-07-14T16:37:55.915 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2447, 2023-07-14T16:37:55.915 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 96.993172510, 2023-07-14T16:37:55.915 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.039637585 2023-07-14T16:37:55.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.915 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2023-07-14T16:37:55.915 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2023-07-14T16:37:55.915 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2447, 2023-07-14T16:37:55.915 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2599 2023-07-14T16:37:55.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.916 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 2447, 2023-07-14T16:37:55.916 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 570, 2023-07-14T16:37:55.916 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 1877, 2023-07-14T16:37:55.916 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2023-07-14T16:37:55.916 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2023-07-14T16:37:55.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 0, 2023-07-14T16:37:55.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 10, 2023-07-14T16:37:55.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 8, 2023-07-14T16:37:55.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 459, 2023-07-14T16:37:55.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 38, 2023-07-14T16:37:55.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2023-07-14T16:37:55.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2023-07-14T16:37:55.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2023-07-14T16:37:55.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2023-07-14T16:37:55.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 1284, 2023-07-14T16:37:55.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2023-07-14T16:37:55.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2023-07-14T16:37:55.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2023-07-14T16:37:55.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 562, 2023-07-14T16:37:55.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 20, 2023-07-14T16:37:55.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2023-07-14T16:37:55.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2023-07-14T16:37:55.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2023-07-14T16:37:55.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2023-07-14T16:37:55.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2023-07-14T16:37:55.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2023-07-14T16:37:55.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T16:37:55.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2023-07-14T16:37:55.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2023-07-14T16:37:55.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2023-07-14T16:37:55.920 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2023-07-14T16:37:55.920 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2023-07-14T16:37:55.920 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2023-07-14T16:37:55.920 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2023-07-14T16:37:55.920 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2023-07-14T16:37:55.920 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2023-07-14T16:37:55.920 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2023-07-14T16:37:55.920 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2023-07-14T16:37:55.921 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2023-07-14T16:37:55.921 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2023-07-14T16:37:55.921 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2023-07-14T16:37:55.921 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2023-07-14T16:37:55.921 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2023-07-14T16:37:55.921 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2023-07-14T16:37:55.921 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2023-07-14T16:37:55.921 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2023-07-14T16:37:55.922 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 88, 2023-07-14T16:37:55.922 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2023-07-14T16:37:55.922 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 2, 2023-07-14T16:37:55.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2023-07-14T16:37:55.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 17, 2023-07-14T16:37:55.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 5, 2023-07-14T16:37:55.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2023-07-14T16:37:55.922 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 153, 2023-07-14T16:37:55.923 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 0, 2023-07-14T16:37:55.923 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 65 2023-07-14T16:37:55.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.923 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2023-07-14T16:37:55.923 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2023-07-14T16:37:55.923 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2023-07-14T16:37:55.923 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2023-07-14T16:37:55.923 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2023-07-14T16:37:55.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2023-07-14T16:37:55.924 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 1187 2023-07-14T16:37:55.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.924 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2023-07-14T16:37:55.924 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2023-07-14T16:37:55.924 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 25, 2023-07-14T16:37:55.924 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2023-07-14T16:37:55.925 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 10, 2023-07-14T16:37:55.925 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 312, 2023-07-14T16:37:55.925 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2023-07-14T16:37:55.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.925 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:37:55.925 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:55.925 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2023-07-14T16:37:55.925 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:55.926 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 6628, 2023-07-14T16:37:55.926 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 2387738, 2023-07-14T16:37:55.926 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:55.926 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 6628, 2023-07-14T16:37:55.926 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:55.926 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:55.926 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 6628, 2023-07-14T16:37:55.926 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 2387738, 2023-07-14T16:37:55.927 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:55.927 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.927 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.927 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.927 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.927 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2023-07-14T16:37:55.927 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:55.928 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2023-07-14T16:37:55.928 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:55.928 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2023-07-14T16:37:55.928 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2023-07-14T16:37:55.928 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:55.928 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2023-07-14T16:37:55.928 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 2447, 2023-07-14T16:37:55.928 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 2465511, 2023-07-14T16:37:55.929 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2447, 2023-07-14T16:37:55.929 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 2465511, 2023-07-14T16:37:55.929 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:55.929 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.929 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.929 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.929 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2023-07-14T16:37:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2023-07-14T16:37:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2023-07-14T16:37:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2023-07-14T16:37:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:55.930 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2023-07-14T16:37:55.931 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 2447, 2023-07-14T16:37:55.931 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 2447, 2023-07-14T16:37:55.931 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 2447, 2023-07-14T16:37:55.931 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 2447, 2023-07-14T16:37:55.931 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:55.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.931 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.931 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.932 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.932 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2023-07-14T16:37:55.932 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:55.932 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2023-07-14T16:37:55.932 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:55.932 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 312, 2023-07-14T16:37:55.933 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 31512, 2023-07-14T16:37:55.933 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:55.933 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 312, 2023-07-14T16:37:55.933 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:55.933 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:55.933 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 8, 2023-07-14T16:37:55.933 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 31512, 2023-07-14T16:37:55.933 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:55.934 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.934 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.934 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.934 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:55.934 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55c1dbd440a0": { 2023-07-14T16:37:55.934 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:55.934 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2023-07-14T16:37:55.935 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:55.935 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 1394, 2023-07-14T16:37:55.935 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 2355469, 2023-07-14T16:37:55.935 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:55.935 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 1394, 2023-07-14T16:37:55.935 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:55.935 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:55.935 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 451, 2023-07-14T16:37:55.936 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 2355469, 2023-07-14T16:37:55.936 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:55.936 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:55.936 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:55.936 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:55.936 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.936 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:55.936 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-07-14T16:37:56.402 INFO:tasks.check_counter:Daemon mds.e mds.exported=3 2023-07-14T16:37:56.402 INFO:tasks.check_counter:Daemon mds.e mds.imported=5 2023-07-14T16:37:56.402 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-07-14T16:37:56.402 DEBUG:tasks.check_counter:Getting stats from h 2023-07-14T16:37:56.403 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-14T16:37:57.123 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:37:57.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:57 smithi062 ceph-mon[95543]: pgmap v1026: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:37:57.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:57 smithi081 ceph-mon[127795]: pgmap v1026: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:37:57.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:57 smithi164 ceph-mon[97427]: pgmap v1026: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:37:58.816 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-07-14T16:37:58.816 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:37:58.816 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1653, 2023-07-14T16:37:58.816 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 1653, 2023-07-14T16:37:58.817 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 4735347, 2023-07-14T16:37:58.817 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 513297, 2023-07-14T16:37:58.817 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 6, 2023-07-14T16:37:58.817 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2023-07-14T16:37:58.817 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.224131492, 2023-07-14T16:37:58.817 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.095110423, 2023-07-14T16:37:58.817 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.049397949, 2023-07-14T16:37:58.818 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.052741497, 2023-07-14T16:37:58.818 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:58.818 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1652, 2023-07-14T16:37:58.818 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.116406988, 2023-07-14T16:37:58.818 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000070464 2023-07-14T16:37:58.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.818 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:58.818 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.819 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.819 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.819 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 2912, 2023-07-14T16:37:58.819 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T16:37:58.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.820 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:37:58.820 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1811, 2023-07-14T16:37:58.820 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 2105, 2023-07-14T16:37:58.820 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 1913101, 2023-07-14T16:37:58.820 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 1114928, 2023-07-14T16:37:58.820 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 6, 2023-07-14T16:37:58.820 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 4, 2023-07-14T16:37:58.821 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.202026302, 2023-07-14T16:37:58.821 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.059857746, 2023-07-14T16:37:58.821 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.052980515, 2023-07-14T16:37:58.821 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.070487377, 2023-07-14T16:37:58.821 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:58.821 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2103, 2023-07-14T16:37:58.821 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.117036817, 2023-07-14T16:37:58.821 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000055652 2023-07-14T16:37:58.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.822 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:58.822 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.822 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.822 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.822 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-14T16:37:58.822 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 551952 2023-07-14T16:37:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:37:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1658, 2023-07-14T16:37:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 1631, 2023-07-14T16:37:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 5139087, 2023-07-14T16:37:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 607677, 2023-07-14T16:37:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 3, 2023-07-14T16:37:58.823 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2023-07-14T16:37:58.824 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.223831389, 2023-07-14T16:37:58.824 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.072840230, 2023-07-14T16:37:58.824 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.077955373, 2023-07-14T16:37:58.824 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.054880821, 2023-07-14T16:37:58.824 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:37:58.824 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1630, 2023-07-14T16:37:58.824 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.080124717, 2023-07-14T16:37:58.824 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000049156 2023-07-14T16:37:58.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.825 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:37:58.825 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.825 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.825 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.825 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 626368, 2023-07-14T16:37:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 223280 2023-07-14T16:37:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2023-07-14T16:37:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2023-07-14T16:37:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2023-07-14T16:37:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2023-07-14T16:37:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2023-07-14T16:37:58.827 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2023-07-14T16:37:58.827 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 7450, 2023-07-14T16:37:58.827 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.251140013, 2023-07-14T16:37:58.827 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000033710 2023-07-14T16:37:58.827 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.827 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2023-07-14T16:37:58.827 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2023-07-14T16:37:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2023-07-14T16:37:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.828 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2023-07-14T16:37:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 0, 2023-07-14T16:37:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 0, 2023-07-14T16:37:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2023-07-14T16:37:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2023-07-14T16:37:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2023-07-14T16:37:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2023-07-14T16:37:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2023-07-14T16:37:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 0, 2023-07-14T16:37:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2023-07-14T16:37:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2023-07-14T16:37:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 566, 2023-07-14T16:37:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 359, 2023-07-14T16:37:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 204, 2023-07-14T16:37:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 3, 2023-07-14T16:37:58.831 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 52, 2023-07-14T16:37:58.832 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2023-07-14T16:37:58.832 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 0, 2023-07-14T16:37:58.832 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 0, 2023-07-14T16:37:58.832 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 3, 2023-07-14T16:37:58.832 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 0, 2023-07-14T16:37:58.832 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 0, 2023-07-14T16:37:58.832 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2023-07-14T16:37:58.833 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 0, 2023-07-14T16:37:58.833 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2023-07-14T16:37:58.833 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2023-07-14T16:37:58.833 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2023-07-14T16:37:58.833 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2023-07-14T16:37:58.833 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2023-07-14T16:37:58.833 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2023-07-14T16:37:58.833 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2023-07-14T16:37:58.834 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2023-07-14T16:37:58.834 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2023-07-14T16:37:58.834 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2023-07-14T16:37:58.834 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2023-07-14T16:37:58.834 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2023-07-14T16:37:58.834 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 4, 2023-07-14T16:37:58.834 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 246, 2023-07-14T16:37:58.834 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 0, 2023-07-14T16:37:58.835 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T16:37:58.835 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2023-07-14T16:37:58.835 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T16:37:58.835 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2023-07-14T16:37:58.835 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2023-07-14T16:37:58.835 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T16:37:58.835 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T16:37:58.835 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2023-07-14T16:37:58.836 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2023-07-14T16:37:58.836 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T16:37:58.836 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 0, 2023-07-14T16:37:58.836 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T16:37:58.836 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T16:37:58.836 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T16:37:58.836 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 0, 2023-07-14T16:37:58.836 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 0, 2023-07-14T16:37:58.837 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 0, 2023-07-14T16:37:58.837 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 0 2023-07-14T16:37:58.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.837 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2023-07-14T16:37:58.837 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2023-07-14T16:37:58.837 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2023-07-14T16:37:58.837 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2023-07-14T16:37:58.837 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 0, 2023-07-14T16:37:58.838 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 0, 2023-07-14T16:37:58.838 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2023-07-14T16:37:58.838 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2023-07-14T16:37:58.838 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2023-07-14T16:37:58.838 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 0, 2023-07-14T16:37:58.838 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 0, 2023-07-14T16:37:58.838 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2023-07-14T16:37:58.839 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2023-07-14T16:37:58.839 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2023-07-14T16:37:58.839 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2023-07-14T16:37:58.839 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2023-07-14T16:37:58.839 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2023-07-14T16:37:58.839 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2023-07-14T16:37:58.839 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T16:37:58.839 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2023-07-14T16:37:58.840 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2023-07-14T16:37:58.840 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2023-07-14T16:37:58.840 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2023-07-14T16:37:58.840 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2023-07-14T16:37:58.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.841 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2023-07-14T16:37:58.841 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 0, 2023-07-14T16:37:58.841 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2023-07-14T16:37:58.841 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2023-07-14T16:37:58.841 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 0, 2023-07-14T16:37:58.841 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2023-07-14T16:37:58.841 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2023-07-14T16:37:58.842 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 0, 2023-07-14T16:37:58.842 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2023-07-14T16:37:58.842 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2023-07-14T16:37:58.842 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2023-07-14T16:37:58.842 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2023-07-14T16:37:58.842 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2023-07-14T16:37:58.842 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 13857916, 2023-07-14T16:37:58.842 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 0, 2023-07-14T16:37:58.843 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 14040589, 2023-07-14T16:37:58.843 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2023-07-14T16:37:58.843 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.843 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.843 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 4248 2023-07-14T16:37:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2023-07-14T16:37:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 14, 2023-07-14T16:37:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 972, 2023-07-14T16:37:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 958, 2023-07-14T16:37:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 11, 2023-07-14T16:37:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 190, 2023-07-14T16:37:58.845 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 179, 2023-07-14T16:37:58.845 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 566, 2023-07-14T16:37:58.845 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 1259, 2023-07-14T16:37:58.845 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 693, 2023-07-14T16:37:58.845 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 0, 2023-07-14T16:37:58.845 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 0, 2023-07-14T16:37:58.845 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 0, 2023-07-14T16:37:58.845 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 50784, 2023-07-14T16:37:58.846 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 207156 2023-07-14T16:37:58.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.846 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2023-07-14T16:37:58.846 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 0, 2023-07-14T16:37:58.846 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 0, 2023-07-14T16:37:58.846 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 0, 2023-07-14T16:37:58.846 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 0, 2023-07-14T16:37:58.846 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 0, 2023-07-14T16:37:58.847 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2023-07-14T16:37:58.847 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.847 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.847 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.847 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2023-07-14T16:37:58.847 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.847 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.848 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.848 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2023-07-14T16:37:58.848 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.848 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.848 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.849 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2023-07-14T16:37:58.849 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.849 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.849 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.849 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2023-07-14T16:37:58.850 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.850 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.850 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.850 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2023-07-14T16:37:58.850 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.850 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.851 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.851 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2023-07-14T16:37:58.851 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.851 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.852 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.852 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2023-07-14T16:37:58.852 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.852 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.852 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.853 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2023-07-14T16:37:58.853 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.853 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.853 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.853 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2023-07-14T16:37:58.853 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.853 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.854 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.854 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2023-07-14T16:37:58.854 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.854 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.854 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.854 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2023-07-14T16:37:58.855 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.855 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.855 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.855 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2023-07-14T16:37:58.855 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.855 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.856 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.856 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2023-07-14T16:37:58.856 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.856 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.856 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.856 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2023-07-14T16:37:58.857 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.857 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.857 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.857 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2023-07-14T16:37:58.857 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.857 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.857 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.858 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2023-07-14T16:37:58.858 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.858 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.858 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2023-07-14T16:37:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2023-07-14T16:37:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.859 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.860 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.860 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2023-07-14T16:37:58.860 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.860 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.860 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.861 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2023-07-14T16:37:58.861 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.861 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.861 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.862 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2023-07-14T16:37:58.862 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.862 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.862 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.862 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2023-07-14T16:37:58.862 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.863 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.863 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.863 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2023-07-14T16:37:58.863 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.863 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.863 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.864 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2023-07-14T16:37:58.864 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.864 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.864 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.864 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2023-07-14T16:37:58.865 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.865 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.865 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.865 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2023-07-14T16:37:58.865 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.865 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.866 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.866 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2023-07-14T16:37:58.866 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.866 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.866 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.866 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2023-07-14T16:37:58.867 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.867 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.867 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.867 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2023-07-14T16:37:58.867 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2023-07-14T16:37:58.867 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2023-07-14T16:37:58.867 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.868 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.868 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.868 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2023-07-14T16:37:58.868 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 1, 2023-07-14T16:37:58.868 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 2, 2023-07-14T16:37:58.869 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 1, 2023-07-14T16:37:58.869 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 1, 2023-07-14T16:37:58.869 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2023-07-14T16:37:58.869 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 0, 2023-07-14T16:37:58.869 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 0, 2023-07-14T16:37:58.869 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 3224 2023-07-14T16:37:58.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2023-07-14T16:37:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:37:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2023-07-14T16:37:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:37:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:37:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:37:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:37:58.870 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:37:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:37:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:37:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:37:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:37:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:37:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:37:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:37:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:37:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2023-07-14T16:37:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:37:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2023-07-14T16:37:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:37:58.873 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:37:58.873 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:37:58.873 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:37:58.873 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:37:58.873 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:37:58.873 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:37:58.873 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2023-07-14T16:37:58.874 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:37:58.874 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:37:58.874 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:37:58.874 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2023-07-14T16:37:58.874 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2023-07-14T16:37:58.874 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2023-07-14T16:37:58.874 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:37:58.874 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:37:58.875 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:37:58.875 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:37:58.875 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 191198, 2023-07-14T16:37:58.875 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 53, 2023-07-14T16:37:58.875 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:37:58.875 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2023-07-14T16:37:58.875 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2023-07-14T16:37:58.875 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2023-07-14T16:37:58.876 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:37:58.876 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2023-07-14T16:37:58.876 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:37:58.876 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2023-07-14T16:37:58.876 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 7528, 2023-07-14T16:37:58.876 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 158, 2023-07-14T16:37:58.876 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:37:58.877 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:37:58.877 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2023-07-14T16:37:58.877 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2023-07-14T16:37:58.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 443248, 2023-07-14T16:37:58.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 2398, 2023-07-14T16:37:58.877 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2023-07-14T16:37:58.877 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2023-07-14T16:37:58.877 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2023-07-14T16:37:58.878 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2023-07-14T16:37:58.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.878 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2023-07-14T16:37:58.878 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2023-07-14T16:37:58.878 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2023-07-14T16:37:58.878 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 4695, 2023-07-14T16:37:58.878 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 26, 2023-07-14T16:37:58.878 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 4, 2023-07-14T16:37:58.879 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 4695, 2023-07-14T16:37:58.879 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2023-07-14T16:37:58.879 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4695, 2023-07-14T16:37:58.879 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 3.669930839, 2023-07-14T16:37:58.879 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000781667 2023-07-14T16:37:58.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.879 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2023-07-14T16:37:58.880 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2023-07-14T16:37:58.880 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4695, 2023-07-14T16:37:58.880 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 4696 2023-07-14T16:37:58.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.880 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 4695, 2023-07-14T16:37:58.880 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 4695, 2023-07-14T16:37:58.880 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 0, 2023-07-14T16:37:58.880 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2023-07-14T16:37:58.881 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2023-07-14T16:37:58.881 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 2844, 2023-07-14T16:37:58.881 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 0, 2023-07-14T16:37:58.881 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 1848, 2023-07-14T16:37:58.881 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 0, 2023-07-14T16:37:58.881 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 0, 2023-07-14T16:37:58.881 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2023-07-14T16:37:58.881 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2023-07-14T16:37:58.882 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2023-07-14T16:37:58.882 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2023-07-14T16:37:58.882 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 0, 2023-07-14T16:37:58.882 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2023-07-14T16:37:58.883 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2023-07-14T16:37:58.883 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2023-07-14T16:37:58.883 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 2, 2023-07-14T16:37:58.883 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 0, 2023-07-14T16:37:58.883 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2023-07-14T16:37:58.883 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2023-07-14T16:37:58.883 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2023-07-14T16:37:58.884 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2023-07-14T16:37:58.884 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2023-07-14T16:37:58.884 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2023-07-14T16:37:58.884 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T16:37:58.884 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2023-07-14T16:37:58.884 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2023-07-14T16:37:58.884 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2023-07-14T16:37:58.884 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2023-07-14T16:37:58.885 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2023-07-14T16:37:58.885 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2023-07-14T16:37:58.885 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2023-07-14T16:37:58.885 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2023-07-14T16:37:58.885 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2023-07-14T16:37:58.885 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2023-07-14T16:37:58.885 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2023-07-14T16:37:58.885 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2023-07-14T16:37:58.886 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2023-07-14T16:37:58.886 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2023-07-14T16:37:58.886 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2023-07-14T16:37:58.886 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2023-07-14T16:37:58.886 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2023-07-14T16:37:58.886 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2023-07-14T16:37:58.886 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2023-07-14T16:37:58.887 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 88, 2023-07-14T16:37:58.887 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2023-07-14T16:37:58.887 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 2, 2023-07-14T16:37:58.887 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 5, 2023-07-14T16:37:58.887 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 6, 2023-07-14T16:37:58.887 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 1, 2023-07-14T16:37:58.887 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2023-07-14T16:37:58.887 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 0, 2023-07-14T16:37:58.888 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 2, 2023-07-14T16:37:58.888 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 0 2023-07-14T16:37:58.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.888 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2023-07-14T16:37:58.888 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 0, 2023-07-14T16:37:58.888 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2023-07-14T16:37:58.888 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2023-07-14T16:37:58.889 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2023-07-14T16:37:58.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.889 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2023-07-14T16:37:58.889 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2023-07-14T16:37:58.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.889 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2023-07-14T16:37:58.890 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2023-07-14T16:37:58.890 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T16:37:58.890 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2023-07-14T16:37:58.890 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 0, 2023-07-14T16:37:58.890 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 0, 2023-07-14T16:37:58.890 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2023-07-14T16:37:58.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.891 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:37:58.891 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:58.891 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2023-07-14T16:37:58.891 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:58.891 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 5118, 2023-07-14T16:37:58.891 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 11302383, 2023-07-14T16:37:58.891 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:58.892 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 5118, 2023-07-14T16:37:58.892 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:58.892 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:58.892 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 5118, 2023-07-14T16:37:58.892 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 11302383, 2023-07-14T16:37:58.892 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:58.892 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.893 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.893 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.893 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.893 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2023-07-14T16:37:58.893 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:58.893 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2023-07-14T16:37:58.894 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:58.894 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2023-07-14T16:37:58.894 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2023-07-14T16:37:58.894 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:58.894 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2023-07-14T16:37:58.894 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 4695, 2023-07-14T16:37:58.894 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 9849723, 2023-07-14T16:37:58.894 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 4695, 2023-07-14T16:37:58.895 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 9849723, 2023-07-14T16:37:58.895 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:58.895 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.895 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.895 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.895 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.895 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2023-07-14T16:37:58.896 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:58.896 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2023-07-14T16:37:58.896 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:58.896 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2023-07-14T16:37:58.896 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2023-07-14T16:37:58.896 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:58.896 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2023-07-14T16:37:58.897 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 4695, 2023-07-14T16:37:58.897 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 4695, 2023-07-14T16:37:58.897 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 4695, 2023-07-14T16:37:58.897 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 4695, 2023-07-14T16:37:58.897 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:58.897 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.897 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.897 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.898 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.898 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2023-07-14T16:37:58.898 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:58.898 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2023-07-14T16:37:58.898 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:58.898 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2023-07-14T16:37:58.898 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2023-07-14T16:37:58.899 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:58.899 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2023-07-14T16:37:58.899 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:58.899 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:58.899 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2023-07-14T16:37:58.899 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2023-07-14T16:37:58.899 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:58.899 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.900 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.900 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.900 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:37:58.901 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x560e4442a500": { 2023-07-14T16:37:58.901 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:37:58.901 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2023-07-14T16:37:58.901 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:37:58.901 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2023-07-14T16:37:58.902 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2023-07-14T16:37:58.902 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:37:58.902 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2023-07-14T16:37:58.902 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:37:58.902 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:37:58.902 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2023-07-14T16:37:58.902 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2023-07-14T16:37:58.902 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:37:58.903 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:37:58.903 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:37:58.903 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:37:58.903 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.903 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:37:58.904 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-07-14T16:37:59.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:37:59 smithi081 ceph-mon[127795]: pgmap v1027: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:37:59.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:37:59 smithi062 ceph-mon[95543]: pgmap v1027: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:37:59.396 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-07-14T16:37:59.396 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-07-14T16:37:59.396 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2023-07-14T16:37:59.396 DEBUG:tasks.check_counter:Getting stats from k 2023-07-14T16:37:59.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-14T16:37:59.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:37:59 smithi164 ceph-mon[97427]: pgmap v1027: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:38:00.117 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T16:38:01.280 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:01 smithi062 ceph-mon[95543]: pgmap v1028: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:01.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:01 smithi081 ceph-mon[127795]: pgmap v1028: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:01.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:01 smithi164 ceph-mon[97427]: pgmap v1028: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:01.836 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-07-14T16:38:01.837 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:38:01.837 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 397, 2023-07-14T16:38:01.837 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 394, 2023-07-14T16:38:01.837 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 522944, 2023-07-14T16:38:01.837 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 225648, 2023-07-14T16:38:01.838 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 4, 2023-07-14T16:38:01.838 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 0, 2023-07-14T16:38:01.838 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.079113628, 2023-07-14T16:38:01.838 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.034574749, 2023-07-14T16:38:01.838 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.025531616, 2023-07-14T16:38:01.838 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.010772152, 2023-07-14T16:38:01.838 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 393, 2023-07-14T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.032338802, 2023-07-14T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000082287 2023-07-14T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:38:01.840 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:38:01.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.840 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 522944, 2023-07-14T16:38:01.840 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 225648 2023-07-14T16:38:01.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.840 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:38:01.840 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 2, 2023-07-14T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 301, 2023-07-14T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 320, 2023-07-14T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 144944, 2023-07-14T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 4, 2023-07-14T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2023-07-14T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.022533666, 2023-07-14T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.016031045, 2023-07-14T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.001450660, 2023-07-14T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 299, 2023-07-14T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.025393382, 2023-07-14T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000084927 2023-07-14T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 144944 2023-07-14T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 4, 2023-07-14T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 4, 2023-07-14T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 2912, 2023-07-14T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 448, 2023-07-14T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 2, 2023-07-14T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 1, 2023-07-14T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.001175666, 2023-07-14T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.000068926, 2023-07-14T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.001098910, 2023-07-14T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3, 2023-07-14T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.001133747, 2023-07-14T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000377915 2023-07-14T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 2912, 2023-07-14T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2023-07-14T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 0, 2023-07-14T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2023-07-14T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2023-07-14T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2023-07-14T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:38:01.849 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:38:01.849 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:38:01.849 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:38:01.849 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:38:01.849 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:38:01.849 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:38:01.849 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:38:01.850 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2023-07-14T16:38:01.850 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:38:01.850 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2023-07-14T16:38:01.850 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:38:01.850 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:38:01.850 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:38:01.850 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2023-07-14T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2023-07-14T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2023-07-14T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2023-07-14T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 148704, 2023-07-14T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 29, 2023-07-14T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2023-07-14T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2023-07-14T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2023-07-14T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2023-07-14T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2023-07-14T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 0, 2023-07-14T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 0, 2023-07-14T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2023-07-14T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2023-07-14T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 928, 2023-07-14T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 2, 2023-07-14T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2023-07-14T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2023-07-14T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2023-07-14T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2023-07-14T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-07-14T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2023-07-14T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2023-07-14T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2023-07-14T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 399, 2023-07-14T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 466787, 2023-07-14T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 399, 2023-07-14T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2023-07-14T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2023-07-14T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 399, 2023-07-14T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 466787, 2023-07-14T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2023-07-14T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2023-07-14T16:38:01.858 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2023-07-14T16:38:01.858 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2023-07-14T16:38:01.858 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:38:01.858 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-07-14T16:38:01.858 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-07-14T16:38:02.067 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:02 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:38:02.144 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:02 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:38:02.397 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-07-14T16:38:02.397 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-07-14T16:38:02.398 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-07-14T16:38:02.398 DEBUG:tasks.check_counter:Getting stats from c 2023-07-14T16:38:02.398 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-14T16:38:02.481 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:02 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T16:38:03.149 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:38:03.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:03 smithi062 ceph-mon[95543]: pgmap v1029: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:38:03.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:38:03.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:38:03.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:03 smithi062 ceph-mon[95543]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:03 smithi081 ceph-mon[127795]: pgmap v1029: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:38:03.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:38:03.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:38:03.396 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:03 smithi081 ceph-mon[127795]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.465 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:03 smithi164 ceph-mon[97427]: pgmap v1029: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:38:03.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:03.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T16:38:03.466 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T16:38:03.467 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:03 smithi164 ceph-mon[97427]: from='mgr.14152 172.21.15.62:0/3692596312' entity='mgr.x' 2023-07-14T16:38:04.695 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2023-07-14T16:38:04.695 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:38:04.695 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 0, 2023-07-14T16:38:04.695 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 2, 2023-07-14T16:38:04.696 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 0, 2023-07-14T16:38:04.696 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 0, 2023-07-14T16:38:04.696 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2023-07-14T16:38:04.696 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 0, 2023-07-14T16:38:04.696 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.003273247, 2023-07-14T16:38:04.696 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T16:38:04.697 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.000521035, 2023-07-14T16:38:04.697 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:38:04.697 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:04.697 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:04.697 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:04.697 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T16:38:04.699 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T16:38:04.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.699 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:38:04.699 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 396, 2023-07-14T16:38:04.699 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 391, 2023-07-14T16:38:04.699 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 524432, 2023-07-14T16:38:04.699 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 223568, 2023-07-14T16:38:04.700 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 2, 2023-07-14T16:38:04.700 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2023-07-14T16:38:04.700 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.069094852, 2023-07-14T16:38:04.700 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.024609314, 2023-07-14T16:38:04.700 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.026705320, 2023-07-14T16:38:04.700 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.011655144, 2023-07-14T16:38:04.700 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:04.701 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 391, 2023-07-14T16:38:04.701 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.031403893, 2023-07-14T16:38:04.701 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000080316 2023-07-14T16:38:04.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.701 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:04.701 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:04.701 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:04.702 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:04.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.702 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 524432, 2023-07-14T16:38:04.702 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 223568 2023-07-14T16:38:04.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.702 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:38:04.702 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2, 2023-07-14T16:38:04.702 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 298, 2023-07-14T16:38:04.703 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 320, 2023-07-14T16:38:04.703 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 144416, 2023-07-14T16:38:04.703 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2023-07-14T16:38:04.703 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2023-07-14T16:38:04.703 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.023567766, 2023-07-14T16:38:04.703 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.016441931, 2023-07-14T16:38:04.703 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.001930583, 2023-07-14T16:38:04.704 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T16:38:04.704 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:04.704 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 296, 2023-07-14T16:38:04.704 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.026311631, 2023-07-14T16:38:04.704 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000088890 2023-07-14T16:38:04.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.704 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:04.705 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:04.705 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:04.705 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:04.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.705 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T16:38:04.705 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 144416 2023-07-14T16:38:04.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.705 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2023-07-14T16:38:04.706 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 0, 2023-07-14T16:38:04.706 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2023-07-14T16:38:04.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.706 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2023-07-14T16:38:04.706 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:38:04.706 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2023-07-14T16:38:04.706 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:38:04.707 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:38:04.707 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:38:04.707 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:38:04.707 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:38:04.707 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:38:04.707 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:38:04.707 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:38:04.707 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:38:04.708 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:38:04.708 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:38:04.708 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:38:04.708 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:38:04.708 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2023-07-14T16:38:04.708 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:38:04.708 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2023-07-14T16:38:04.709 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:38:04.709 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:38:04.709 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:38:04.709 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:38:04.709 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:38:04.709 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:38:04.709 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:38:04.709 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2023-07-14T16:38:04.710 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:38:04.710 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:38:04.710 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:38:04.710 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2023-07-14T16:38:04.710 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2023-07-14T16:38:04.710 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2023-07-14T16:38:04.710 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:38:04.710 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:38:04.711 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:38:04.711 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:38:04.711 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 148704, 2023-07-14T16:38:04.711 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 29, 2023-07-14T16:38:04.711 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:38:04.711 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2023-07-14T16:38:04.711 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2023-07-14T16:38:04.711 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2023-07-14T16:38:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:38:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2023-07-14T16:38:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:38:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2023-07-14T16:38:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 0, 2023-07-14T16:38:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 0, 2023-07-14T16:38:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:38:04.712 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:38:04.713 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2023-07-14T16:38:04.713 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2023-07-14T16:38:04.713 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 928, 2023-07-14T16:38:04.713 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 2, 2023-07-14T16:38:04.713 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2023-07-14T16:38:04.713 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2023-07-14T16:38:04.713 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2023-07-14T16:38:04.713 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2023-07-14T16:38:04.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:04.714 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:38:04.714 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:04.714 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2023-07-14T16:38:04.714 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:04.714 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 394, 2023-07-14T16:38:04.714 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 466073, 2023-07-14T16:38:04.715 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:04.715 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 394, 2023-07-14T16:38:04.715 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:04.715 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:04.715 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 394, 2023-07-14T16:38:04.715 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 466073, 2023-07-14T16:38:04.715 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:04.715 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:04.716 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:04.716 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:04.716 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:04.716 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:04.716 INFO:teuthology.orchestra.run.smithi164.stdout:} 2023-07-14T16:38:05.129 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.c 2023-07-14T16:38:05.129 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.c 2023-07-14T16:38:05.129 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2023-07-14T16:38:05.129 DEBUG:tasks.check_counter:Getting stats from f 2023-07-14T16:38:05.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-14T16:38:05.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:05 smithi062 ceph-mon[95543]: pgmap v1030: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:05.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:05 smithi081 ceph-mon[127795]: pgmap v1030: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:05.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:05 smithi164 ceph-mon[97427]: pgmap v1030: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:05.865 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:38:07.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:07 smithi164 ceph-mon[97427]: pgmap v1031: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:38:07.312 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2023-07-14T16:38:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:38:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 5119, 2023-07-14T16:38:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 5121, 2023-07-14T16:38:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 6413660, 2023-07-14T16:38:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 1592016, 2023-07-14T16:38:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 6, 2023-07-14T16:38:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2023-07-14T16:38:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.513532309, 2023-07-14T16:38:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.180330607, 2023-07-14T16:38:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.131301237, 2023-07-14T16:38:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.154442726, 2023-07-14T16:38:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 5119, 2023-07-14T16:38:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.245391305, 2023-07-14T16:38:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000047937 2023-07-14T16:38:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T16:38:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T16:38:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:38:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 8, 2023-07-14T16:38:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 303, 2023-07-14T16:38:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 3788, 2023-07-14T16:38:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 611852, 2023-07-14T16:38:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2023-07-14T16:38:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2023-07-14T16:38:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.022758830, 2023-07-14T16:38:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.015390965, 2023-07-14T16:38:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.003231958, 2023-07-14T16:38:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000082793, 2023-07-14T16:38:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 302, 2023-07-14T16:38:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.030312083, 2023-07-14T16:38:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000100371 2023-07-14T16:38:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3232, 2023-07-14T16:38:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 611168 2023-07-14T16:38:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:38:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 426, 2023-07-14T16:38:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 399, 2023-07-14T16:38:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 627564, 2023-07-14T16:38:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 225799, 2023-07-14T16:38:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2023-07-14T16:38:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2023-07-14T16:38:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.075240401, 2023-07-14T16:38:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.032038245, 2023-07-14T16:38:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.029077021, 2023-07-14T16:38:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.009937902, 2023-07-14T16:38:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 398, 2023-07-14T16:38:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.030479217, 2023-07-14T16:38:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000076580 2023-07-14T16:38:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 627296, 2023-07-14T16:38:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 225488 2023-07-14T16:38:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2023-07-14T16:38:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2023-07-14T16:38:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2023-07-14T16:38:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2023-07-14T16:38:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2023-07-14T16:38:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2023-07-14T16:38:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 8841, 2023-07-14T16:38:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.264865823, 2023-07-14T16:38:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000029958 2023-07-14T16:38:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2023-07-14T16:38:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2023-07-14T16:38:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2023-07-14T16:38:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2023-07-14T16:38:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2023-07-14T16:38:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2023-07-14T16:38:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2023-07-14T16:38:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2023-07-14T16:38:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2023-07-14T16:38:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2023-07-14T16:38:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2023-07-14T16:38:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2023-07-14T16:38:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2023-07-14T16:38:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2023-07-14T16:38:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 123, 2023-07-14T16:38:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2023-07-14T16:38:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 0, 2023-07-14T16:38:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 123, 2023-07-14T16:38:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 123, 2023-07-14T16:38:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 14, 2023-07-14T16:38:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2023-07-14T16:38:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2023-07-14T16:38:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2023-07-14T16:38:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2023-07-14T16:38:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2023-07-14T16:38:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2023-07-14T16:38:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2023-07-14T16:38:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2023-07-14T16:38:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2023-07-14T16:38:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2023-07-14T16:38:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2023-07-14T16:38:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2023-07-14T16:38:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2023-07-14T16:38:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2023-07-14T16:38:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2023-07-14T16:38:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2023-07-14T16:38:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2023-07-14T16:38:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2023-07-14T16:38:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2023-07-14T16:38:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2023-07-14T16:38:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2023-07-14T16:38:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2023-07-14T16:38:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T16:38:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2023-07-14T16:38:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T16:38:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2023-07-14T16:38:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2023-07-14T16:38:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T16:38:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T16:38:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2023-07-14T16:38:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2023-07-14T16:38:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T16:38:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2023-07-14T16:38:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T16:38:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T16:38:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T16:38:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2023-07-14T16:38:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2023-07-14T16:38:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2023-07-14T16:38:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2023-07-14T16:38:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2023-07-14T16:38:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2023-07-14T16:38:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2023-07-14T16:38:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2023-07-14T16:38:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2023-07-14T16:38:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2023-07-14T16:38:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2023-07-14T16:38:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2023-07-14T16:38:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2023-07-14T16:38:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2023-07-14T16:38:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2023-07-14T16:38:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2023-07-14T16:38:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2023-07-14T16:38:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2023-07-14T16:38:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2023-07-14T16:38:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2023-07-14T16:38:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2023-07-14T16:38:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2023-07-14T16:38:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T16:38:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2023-07-14T16:38:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2023-07-14T16:38:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2023-07-14T16:38:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2023-07-14T16:38:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2023-07-14T16:38:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2023-07-14T16:38:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2023-07-14T16:38:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2023-07-14T16:38:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2023-07-14T16:38:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 0, 2023-07-14T16:38:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2023-07-14T16:38:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2023-07-14T16:38:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2023-07-14T16:38:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2023-07-14T16:38:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2023-07-14T16:38:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 4, 2023-07-14T16:38:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2023-07-14T16:38:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2023-07-14T16:38:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2023-07-14T16:38:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 0, 2023-07-14T16:38:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 9192372, 2023-07-14T16:38:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2023-07-14T16:38:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 2507 2023-07-14T16:38:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2023-07-14T16:38:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 6, 2023-07-14T16:38:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 349, 2023-07-14T16:38:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 343, 2023-07-14T16:38:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 11, 2023-07-14T16:38:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 103, 2023-07-14T16:38:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 92, 2023-07-14T16:38:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 123, 2023-07-14T16:38:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 441, 2023-07-14T16:38:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 318, 2023-07-14T16:38:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2023-07-14T16:38:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2023-07-14T16:38:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2023-07-14T16:38:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 50552, 2023-07-14T16:38:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 207156 2023-07-14T16:38:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2023-07-14T16:38:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2023-07-14T16:38:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2023-07-14T16:38:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2023-07-14T16:38:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2023-07-14T16:38:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2023-07-14T16:38:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2023-07-14T16:38:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2023-07-14T16:38:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2023-07-14T16:38:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2023-07-14T16:38:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2023-07-14T16:38:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2023-07-14T16:38:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2023-07-14T16:38:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2023-07-14T16:38:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2023-07-14T16:38:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2023-07-14T16:38:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2023-07-14T16:38:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.348 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.348 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.348 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2023-07-14T16:38:07.348 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.348 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.348 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2023-07-14T16:38:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2023-07-14T16:38:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2023-07-14T16:38:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2023-07-14T16:38:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2023-07-14T16:38:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2023-07-14T16:38:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2023-07-14T16:38:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2023-07-14T16:38:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2023-07-14T16:38:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2023-07-14T16:38:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2023-07-14T16:38:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2023-07-14T16:38:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2023-07-14T16:38:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2023-07-14T16:38:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2023-07-14T16:38:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2023-07-14T16:38:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2023-07-14T16:38:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2023-07-14T16:38:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2023-07-14T16:38:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2023-07-14T16:38:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2023-07-14T16:38:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2023-07-14T16:38:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2023-07-14T16:38:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2023-07-14T16:38:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2023-07-14T16:38:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2023-07-14T16:38:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2023-07-14T16:38:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2023-07-14T16:38:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 3164 2023-07-14T16:38:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2023-07-14T16:38:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:38:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2023-07-14T16:38:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:38:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:38:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:38:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:38:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:38:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:38:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:38:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:38:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:38:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:38:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:38:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:38:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:38:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2023-07-14T16:38:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:38:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2023-07-14T16:38:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:38:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:38:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:38:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:38:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:38:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:38:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:38:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2023-07-14T16:38:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:38:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:38:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:38:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2023-07-14T16:38:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2023-07-14T16:38:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2023-07-14T16:38:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:38:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:38:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:38:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:38:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 183118, 2023-07-14T16:38:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 49, 2023-07-14T16:38:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:38:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2023-07-14T16:38:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2023-07-14T16:38:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2023-07-14T16:38:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:38:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2023-07-14T16:38:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:38:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2023-07-14T16:38:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7528, 2023-07-14T16:38:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 158, 2023-07-14T16:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2023-07-14T16:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2023-07-14T16:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 115202, 2023-07-14T16:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 652, 2023-07-14T16:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2023-07-14T16:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2023-07-14T16:38:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2023-07-14T16:38:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2023-07-14T16:38:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2023-07-14T16:38:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2023-07-14T16:38:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2023-07-14T16:38:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 5122, 2023-07-14T16:38:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2023-07-14T16:38:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2023-07-14T16:38:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 5122, 2023-07-14T16:38:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2023-07-14T16:38:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 5122, 2023-07-14T16:38:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 5.060863803, 2023-07-14T16:38:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000988063 2023-07-14T16:38:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2023-07-14T16:38:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2023-07-14T16:38:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 5122, 2023-07-14T16:38:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 5123 2023-07-14T16:38:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 5122, 2023-07-14T16:38:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 5122, 2023-07-14T16:38:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2023-07-14T16:38:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2023-07-14T16:38:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2023-07-14T16:38:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 2782, 2023-07-14T16:38:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2023-07-14T16:38:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 2338, 2023-07-14T16:38:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2023-07-14T16:38:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2023-07-14T16:38:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2023-07-14T16:38:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2023-07-14T16:38:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2023-07-14T16:38:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2023-07-14T16:38:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2023-07-14T16:38:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2023-07-14T16:38:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2023-07-14T16:38:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2023-07-14T16:38:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2023-07-14T16:38:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2023-07-14T16:38:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2023-07-14T16:38:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2023-07-14T16:38:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2023-07-14T16:38:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2023-07-14T16:38:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2023-07-14T16:38:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2023-07-14T16:38:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T16:38:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2023-07-14T16:38:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2023-07-14T16:38:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2023-07-14T16:38:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2023-07-14T16:38:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2023-07-14T16:38:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2023-07-14T16:38:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2023-07-14T16:38:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2023-07-14T16:38:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2023-07-14T16:38:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2023-07-14T16:38:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2023-07-14T16:38:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2023-07-14T16:38:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2023-07-14T16:38:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2023-07-14T16:38:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2023-07-14T16:38:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2023-07-14T16:38:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2023-07-14T16:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2023-07-14T16:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2023-07-14T16:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 88, 2023-07-14T16:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2023-07-14T16:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 2, 2023-07-14T16:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 4, 2023-07-14T16:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 5, 2023-07-14T16:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 1, 2023-07-14T16:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2023-07-14T16:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2023-07-14T16:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2023-07-14T16:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2023-07-14T16:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2023-07-14T16:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2023-07-14T16:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2023-07-14T16:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T16:38:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2023-07-14T16:38:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2023-07-14T16:38:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2023-07-14T16:38:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2023-07-14T16:38:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:38:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2023-07-14T16:38:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 5549, 2023-07-14T16:38:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 6520865, 2023-07-14T16:38:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 5549, 2023-07-14T16:38:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 5549, 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 6520865, 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2023-07-14T16:38:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2023-07-14T16:38:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 5122, 2023-07-14T16:38:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 5001074, 2023-07-14T16:38:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 5122, 2023-07-14T16:38:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 5001074, 2023-07-14T16:38:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2023-07-14T16:38:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2023-07-14T16:38:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 5122, 2023-07-14T16:38:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 5122, 2023-07-14T16:38:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 5122, 2023-07-14T16:38:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 5122, 2023-07-14T16:38:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2023-07-14T16:38:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2023-07-14T16:38:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2023-07-14T16:38:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2023-07-14T16:38:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x564160c70500": { 2023-07-14T16:38:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2023-07-14T16:38:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2023-07-14T16:38:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2023-07-14T16:38:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:07.392 INFO:teuthology.orchestra.run.smithi164.stdout:} 2023-07-14T16:38:07.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:07 smithi062 ceph-mon[95543]: pgmap v1031: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:38:07.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:07 smithi081 ceph-mon[127795]: pgmap v1031: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:38:07.758 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-14T16:38:07.758 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-14T16:38:07.758 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-07-14T16:38:07.759 DEBUG:tasks.check_counter:Getting stats from i 2023-07-14T16:38:07.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-14T16:38:08.494 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:38:09.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:09 smithi062 ceph-mon[95543]: pgmap v1032: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:38:09.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:09 smithi081 ceph-mon[127795]: pgmap v1032: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:38:09.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:09 smithi164 ceph-mon[97427]: pgmap v1032: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:38:10.008 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2023-07-14T16:38:10.009 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:38:10.009 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 7398, 2023-07-14T16:38:10.009 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 7225, 2023-07-14T16:38:10.009 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 3107218, 2023-07-14T16:38:10.009 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 8378374, 2023-07-14T16:38:10.009 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 70, 2023-07-14T16:38:10.010 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2023-07-14T16:38:10.010 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.812410020, 2023-07-14T16:38:10.010 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.329117229, 2023-07-14T16:38:10.010 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.332555153, 2023-07-14T16:38:10.010 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.123448850, 2023-07-14T16:38:10.010 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:10.010 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7224, 2023-07-14T16:38:10.010 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.412008347, 2023-07-14T16:38:10.011 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000057033 2023-07-14T16:38:10.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.011 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:10.011 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 5515, 2023-07-14T16:38:10.011 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002062928, 2023-07-14T16:38:10.011 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000374 2023-07-14T16:38:10.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.012 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 642448, 2023-07-14T16:38:10.012 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 269536 2023-07-14T16:38:10.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.012 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:38:10.012 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 6011, 2023-07-14T16:38:10.013 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 8413, 2023-07-14T16:38:10.013 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 2325553, 2023-07-14T16:38:10.013 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 3731598, 2023-07-14T16:38:10.013 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 21, 2023-07-14T16:38:10.013 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 6, 2023-07-14T16:38:10.013 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.685333268, 2023-07-14T16:38:10.013 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.282267997, 2023-07-14T16:38:10.013 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.362478920, 2023-07-14T16:38:10.014 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.065983544, 2023-07-14T16:38:10.014 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:10.014 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 8410, 2023-07-14T16:38:10.014 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.382869858, 2023-07-14T16:38:10.014 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000045525 2023-07-14T16:38:10.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.014 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:10.014 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7135, 2023-07-14T16:38:10.015 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002120390, 2023-07-14T16:38:10.015 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000297 2023-07-14T16:38:10.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.015 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 3232, 2023-07-14T16:38:10.015 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 615664 2023-07-14T16:38:10.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.015 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:38:10.015 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 3438, 2023-07-14T16:38:10.016 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 3449, 2023-07-14T16:38:10.016 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1656633, 2023-07-14T16:38:10.016 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 1374364, 2023-07-14T16:38:10.016 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 10, 2023-07-14T16:38:10.016 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2023-07-14T16:38:10.016 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.279692609, 2023-07-14T16:38:10.016 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.102852465, 2023-07-14T16:38:10.017 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.107653668, 2023-07-14T16:38:10.017 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.058704265, 2023-07-14T16:38:10.017 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:10.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3449, 2023-07-14T16:38:10.017 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.224563178, 2023-07-14T16:38:10.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000065109 2023-07-14T16:38:10.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.018 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:10.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1295, 2023-07-14T16:38:10.018 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000329695, 2023-07-14T16:38:10.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000254 2023-07-14T16:38:10.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.018 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T16:38:10.018 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T16:38:10.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.019 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2023-07-14T16:38:10.019 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2023-07-14T16:38:10.019 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2023-07-14T16:38:10.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.019 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2023-07-14T16:38:10.019 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2023-07-14T16:38:10.019 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2023-07-14T16:38:10.019 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7423, 2023-07-14T16:38:10.020 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1.820417363, 2023-07-14T16:38:10.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000245240 2023-07-14T16:38:10.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.020 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2023-07-14T16:38:10.020 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2023-07-14T16:38:10.020 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2023-07-14T16:38:10.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1226, 2023-07-14T16:38:10.021 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.050449928, 2023-07-14T16:38:10.021 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000041150 2023-07-14T16:38:10.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.021 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2023-07-14T16:38:10.021 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 1196, 2023-07-14T16:38:10.021 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 1059, 2023-07-14T16:38:10.021 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2023-07-14T16:38:10.022 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1059, 2023-07-14T16:38:10.022 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1.132289582, 2023-07-14T16:38:10.022 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001069206 2023-07-14T16:38:10.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.022 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2023-07-14T16:38:10.023 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 137, 2023-07-14T16:38:10.023 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2023-07-14T16:38:10.023 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2023-07-14T16:38:10.023 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 20, 2023-07-14T16:38:10.023 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 1, 2023-07-14T16:38:10.023 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2023-07-14T16:38:10.023 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 44, 2023-07-14T16:38:10.023 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 1, 2023-07-14T16:38:10.024 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 29, 2023-07-14T16:38:10.024 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 14, 2023-07-14T16:38:10.024 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 15, 2023-07-14T16:38:10.024 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 303, 2023-07-14T16:38:10.024 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 4, 2023-07-14T16:38:10.024 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 4, 2023-07-14T16:38:10.025 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 5, 2023-07-14T16:38:10.025 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 1923, 2023-07-14T16:38:10.025 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 1137, 2023-07-14T16:38:10.025 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 124, 2023-07-14T16:38:10.025 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 8, 2023-07-14T16:38:10.025 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2023-07-14T16:38:10.025 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2023-07-14T16:38:10.025 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2023-07-14T16:38:10.026 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2023-07-14T16:38:10.026 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2023-07-14T16:38:10.026 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 11, 2023-07-14T16:38:10.026 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 282, 2023-07-14T16:38:10.026 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 11, 2023-07-14T16:38:10.026 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 217, 2023-07-14T16:38:10.027 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2023-07-14T16:38:10.027 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 6650, 2023-07-14T16:38:10.027 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2023-07-14T16:38:10.027 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 324, 2023-07-14T16:38:10.027 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 13424350693, 2023-07-14T16:38:10.027 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2023-07-14T16:38:10.027 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 6650, 2023-07-14T16:38:10.028 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2023-07-14T16:38:10.028 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T16:38:10.028 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2023-07-14T16:38:10.028 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 1985, 2023-07-14T16:38:10.028 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T16:38:10.029 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 1985, 2023-07-14T16:38:10.029 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 4665, 2023-07-14T16:38:10.029 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2023-07-14T16:38:10.029 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 52, 2023-07-14T16:38:10.029 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 1571, 2023-07-14T16:38:10.029 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 1, 2023-07-14T16:38:10.029 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T16:38:10.030 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T16:38:10.030 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 1754, 2023-07-14T16:38:10.030 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 1595, 2023-07-14T16:38:10.030 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 12, 2023-07-14T16:38:10.030 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 313 2023-07-14T16:38:10.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.030 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2023-07-14T16:38:10.030 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2023-07-14T16:38:10.031 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 3, 2023-07-14T16:38:10.031 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 2, 2023-07-14T16:38:10.031 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 10, 2023-07-14T16:38:10.031 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 2, 2023-07-14T16:38:10.031 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 1, 2023-07-14T16:38:10.031 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2023-07-14T16:38:10.031 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2023-07-14T16:38:10.031 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 64, 2023-07-14T16:38:10.032 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 63, 2023-07-14T16:38:10.032 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2023-07-14T16:38:10.032 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2023-07-14T16:38:10.032 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2023-07-14T16:38:10.102 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2023-07-14T16:38:10.102 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2023-07-14T16:38:10.102 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2023-07-14T16:38:10.102 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2023-07-14T16:38:10.102 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T16:38:10.103 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 11, 2023-07-14T16:38:10.103 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2023-07-14T16:38:10.103 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 1, 2023-07-14T16:38:10.103 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2023-07-14T16:38:10.103 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2023-07-14T16:38:10.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.103 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2023-07-14T16:38:10.104 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 2507, 2023-07-14T16:38:10.104 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2023-07-14T16:38:10.104 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2023-07-14T16:38:10.104 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 2507, 2023-07-14T16:38:10.104 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2023-07-14T16:38:10.105 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2023-07-14T16:38:10.105 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 4, 2023-07-14T16:38:10.105 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2023-07-14T16:38:10.105 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2023-07-14T16:38:10.105 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 4, 2023-07-14T16:38:10.105 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2023-07-14T16:38:10.105 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2023-07-14T16:38:10.105 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2023-07-14T16:38:10.106 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 9194595, 2023-07-14T16:38:10.106 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2023-07-14T16:38:10.106 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2023-07-14T16:38:10.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1553, 2023-07-14T16:38:10.106 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 5.359676168, 2023-07-14T16:38:10.106 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.003451175 2023-07-14T16:38:10.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.107 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2023-07-14T16:38:10.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.107 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2023-07-14T16:38:10.107 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 23, 2023-07-14T16:38:10.107 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 360, 2023-07-14T16:38:10.107 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 337, 2023-07-14T16:38:10.107 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 29, 2023-07-14T16:38:10.107 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 115, 2023-07-14T16:38:10.108 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 86, 2023-07-14T16:38:10.108 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 44, 2023-07-14T16:38:10.108 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 456, 2023-07-14T16:38:10.108 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 412, 2023-07-14T16:38:10.108 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 4, 2023-07-14T16:38:10.108 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 336, 2023-07-14T16:38:10.108 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 332, 2023-07-14T16:38:10.108 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 55600, 2023-07-14T16:38:10.109 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 198964 2023-07-14T16:38:10.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.109 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2023-07-14T16:38:10.109 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 1350, 2023-07-14T16:38:10.109 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 34, 2023-07-14T16:38:10.109 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 1196, 2023-07-14T16:38:10.109 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 25, 2023-07-14T16:38:10.110 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 118, 2023-07-14T16:38:10.110 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2023-07-14T16:38:10.110 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 143, 2023-07-14T16:38:10.110 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.196084231, 2023-07-14T16:38:10.110 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001371218 2023-07-14T16:38:10.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.110 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2023-07-14T16:38:10.110 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 9, 2023-07-14T16:38:10.111 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.003337304, 2023-07-14T16:38:10.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000370811 2023-07-14T16:38:10.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.111 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2023-07-14T16:38:10.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.111 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.111 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.112 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2023-07-14T16:38:10.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.112 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.112 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2023-07-14T16:38:10.112 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 503, 2023-07-14T16:38:10.112 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.266860606, 2023-07-14T16:38:10.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000530537 2023-07-14T16:38:10.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.113 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2023-07-14T16:38:10.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.113 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.113 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.114 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2023-07-14T16:38:10.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.114 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.114 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.115 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2023-07-14T16:38:10.115 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.115 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.115 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.116 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2023-07-14T16:38:10.116 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.116 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.116 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.116 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2023-07-14T16:38:10.117 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.117 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.117 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.117 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2023-07-14T16:38:10.117 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.117 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.118 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.118 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2023-07-14T16:38:10.118 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.118 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.118 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.119 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2023-07-14T16:38:10.119 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.119 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.119 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.120 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2023-07-14T16:38:10.120 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.120 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.120 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.120 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2023-07-14T16:38:10.120 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 330, 2023-07-14T16:38:10.121 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.274210361, 2023-07-14T16:38:10.121 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000830940 2023-07-14T16:38:10.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.121 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2023-07-14T16:38:10.121 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7, 2023-07-14T16:38:10.121 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.004878504, 2023-07-14T16:38:10.121 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000696929 2023-07-14T16:38:10.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.122 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2023-07-14T16:38:10.122 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 18, 2023-07-14T16:38:10.122 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.062860000, 2023-07-14T16:38:10.122 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.003492222 2023-07-14T16:38:10.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.122 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2023-07-14T16:38:10.122 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.123 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.123 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.123 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2023-07-14T16:38:10.123 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.123 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.123 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.124 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2023-07-14T16:38:10.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2023-07-14T16:38:10.124 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.016017199, 2023-07-14T16:38:10.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.008008599 2023-07-14T16:38:10.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.124 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2023-07-14T16:38:10.124 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.125 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.125 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.125 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2023-07-14T16:38:10.125 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.125 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.126 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.126 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2023-07-14T16:38:10.126 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2023-07-14T16:38:10.126 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001041423, 2023-07-14T16:38:10.126 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001041423 2023-07-14T16:38:10.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.126 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2023-07-14T16:38:10.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.127 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.127 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2023-07-14T16:38:10.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2023-07-14T16:38:10.127 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000807461, 2023-07-14T16:38:10.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000403730 2023-07-14T16:38:10.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.128 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2023-07-14T16:38:10.128 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.128 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.128 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.129 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2023-07-14T16:38:10.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.129 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.129 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2023-07-14T16:38:10.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.129 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.130 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2023-07-14T16:38:10.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 44, 2023-07-14T16:38:10.130 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.306192493, 2023-07-14T16:38:10.130 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.006958920 2023-07-14T16:38:10.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.130 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2023-07-14T16:38:10.131 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2023-07-14T16:38:10.131 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2023-07-14T16:38:10.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.131 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.131 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2023-07-14T16:38:10.132 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2023-07-14T16:38:10.132 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2023-07-14T16:38:10.132 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 0, 2023-07-14T16:38:10.132 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2023-07-14T16:38:10.132 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2023-07-14T16:38:10.132 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 196, 2023-07-14T16:38:10.132 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 196, 2023-07-14T16:38:10.133 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1349 2023-07-14T16:38:10.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.133 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2023-07-14T16:38:10.133 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:38:10.133 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2023-07-14T16:38:10.133 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:38:10.133 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:38:10.134 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:38:10.134 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:38:10.134 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:38:10.134 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:38:10.134 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:38:10.134 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:38:10.134 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:38:10.134 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:38:10.135 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:38:10.135 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:38:10.135 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:38:10.135 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2023-07-14T16:38:10.135 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:38:10.135 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2023-07-14T16:38:10.135 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:38:10.136 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:38:10.136 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:38:10.136 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:38:10.136 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:38:10.136 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:38:10.136 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:38:10.137 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2023-07-14T16:38:10.137 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:38:10.137 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:38:10.137 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:38:10.137 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2023-07-14T16:38:10.137 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2023-07-14T16:38:10.137 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2023-07-14T16:38:10.137 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:38:10.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:38:10.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:38:10.138 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:38:10.138 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 292047, 2023-07-14T16:38:10.138 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 115, 2023-07-14T16:38:10.138 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:38:10.138 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2023-07-14T16:38:10.138 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2023-07-14T16:38:10.139 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2023-07-14T16:38:10.139 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:38:10.139 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2023-07-14T16:38:10.139 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:38:10.139 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2023-07-14T16:38:10.139 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7528, 2023-07-14T16:38:10.139 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 158, 2023-07-14T16:38:10.139 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:38:10.140 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:38:10.140 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2023-07-14T16:38:10.140 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2023-07-14T16:38:10.140 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 122367, 2023-07-14T16:38:10.140 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 705, 2023-07-14T16:38:10.140 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2023-07-14T16:38:10.141 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2023-07-14T16:38:10.141 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2023-07-14T16:38:10.141 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2023-07-14T16:38:10.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.141 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2023-07-14T16:38:10.141 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2023-07-14T16:38:10.141 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2023-07-14T16:38:10.142 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 9709, 2023-07-14T16:38:10.142 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 5226797, 2023-07-14T16:38:10.142 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 27, 2023-07-14T16:38:10.142 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 9709, 2023-07-14T16:38:10.142 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2023-07-14T16:38:10.142 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 9709, 2023-07-14T16:38:10.142 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 102.890197919, 2023-07-14T16:38:10.142 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.010597404 2023-07-14T16:38:10.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.143 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2023-07-14T16:38:10.143 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2023-07-14T16:38:10.143 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 9709, 2023-07-14T16:38:10.143 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 10120 2023-07-14T16:38:10.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.144 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 9709, 2023-07-14T16:38:10.144 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 6661, 2023-07-14T16:38:10.145 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 3048, 2023-07-14T16:38:10.145 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2023-07-14T16:38:10.145 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2023-07-14T16:38:10.145 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2023-07-14T16:38:10.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 10, 2023-07-14T16:38:10.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 11, 2023-07-14T16:38:10.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 1564, 2023-07-14T16:38:10.146 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 92, 2023-07-14T16:38:10.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2023-07-14T16:38:10.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2023-07-14T16:38:10.147 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2023-07-14T16:38:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 1, 2023-07-14T16:38:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 1132, 2023-07-14T16:38:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2023-07-14T16:38:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2023-07-14T16:38:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2023-07-14T16:38:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 6650, 2023-07-14T16:38:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 20, 2023-07-14T16:38:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2023-07-14T16:38:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2023-07-14T16:38:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2023-07-14T16:38:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2023-07-14T16:38:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2023-07-14T16:38:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2023-07-14T16:38:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T16:38:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2023-07-14T16:38:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2023-07-14T16:38:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 1, 2023-07-14T16:38:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2023-07-14T16:38:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2023-07-14T16:38:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2023-07-14T16:38:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2023-07-14T16:38:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2023-07-14T16:38:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2023-07-14T16:38:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2023-07-14T16:38:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2023-07-14T16:38:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2023-07-14T16:38:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2023-07-14T16:38:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2023-07-14T16:38:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2023-07-14T16:38:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2023-07-14T16:38:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2023-07-14T16:38:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2023-07-14T16:38:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2023-07-14T16:38:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 88, 2023-07-14T16:38:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2023-07-14T16:38:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 2, 2023-07-14T16:38:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 12, 2023-07-14T16:38:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 12, 2023-07-14T16:38:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2023-07-14T16:38:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2023-07-14T16:38:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 450, 2023-07-14T16:38:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2023-07-14T16:38:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 189 2023-07-14T16:38:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2023-07-14T16:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2023-07-14T16:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2023-07-14T16:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2023-07-14T16:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2023-07-14T16:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2023-07-14T16:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 1252 2023-07-14T16:38:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2023-07-14T16:38:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2023-07-14T16:38:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 26, 2023-07-14T16:38:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2023-07-14T16:38:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 6, 2023-07-14T16:38:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 63, 2023-07-14T16:38:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2023-07-14T16:38:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:38:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2023-07-14T16:38:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 16843, 2023-07-14T16:38:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 5545110, 2023-07-14T16:38:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 16843, 2023-07-14T16:38:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 16843, 2023-07-14T16:38:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 5545110, 2023-07-14T16:38:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2023-07-14T16:38:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2023-07-14T16:38:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 9709, 2023-07-14T16:38:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 5233160, 2023-07-14T16:38:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 9709, 2023-07-14T16:38:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 5233160, 2023-07-14T16:38:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2023-07-14T16:38:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2023-07-14T16:38:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 9709, 2023-07-14T16:38:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 9709, 2023-07-14T16:38:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 9709, 2023-07-14T16:38:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 9709, 2023-07-14T16:38:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2023-07-14T16:38:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2023-07-14T16:38:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 63, 2023-07-14T16:38:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 6363, 2023-07-14T16:38:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 63, 2023-07-14T16:38:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 11, 2023-07-14T16:38:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 6363, 2023-07-14T16:38:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55a07669c0a0": { 2023-07-14T16:38:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2023-07-14T16:38:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 2507, 2023-07-14T16:38:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 5000291, 2023-07-14T16:38:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 2507, 2023-07-14T16:38:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1553, 2023-07-14T16:38:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 5000291, 2023-07-14T16:38:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:10.170 INFO:teuthology.orchestra.run.smithi164.stdout:} 2023-07-14T16:38:10.467 INFO:tasks.check_counter:Daemon mds.i mds.exported=11 2023-07-14T16:38:10.468 INFO:tasks.check_counter:Daemon mds.i mds.imported=11 2023-07-14T16:38:10.468 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2023-07-14T16:38:10.468 DEBUG:tasks.check_counter:Getting stats from l 2023-07-14T16:38:10.469 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 1356e48a-2260-11ee-9b33-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-14T16:38:11.193 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T16:38:11.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:11 smithi062 ceph-mon[95543]: pgmap v1033: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:11.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:11 smithi081 ceph-mon[127795]: pgmap v1033: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:11.454 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:11 smithi164 ceph-mon[97427]: pgmap v1033: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:12.585 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2023-07-14T16:38:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T16:38:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1688, 2023-07-14T16:38:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1690, 2023-07-14T16:38:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 2794492, 2023-07-14T16:38:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 525032, 2023-07-14T16:38:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 7, 2023-07-14T16:38:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 2, 2023-07-14T16:38:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.181066221, 2023-07-14T16:38:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.055385708, 2023-07-14T16:38:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.052340065, 2023-07-14T16:38:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.049635824, 2023-07-14T16:38:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1688, 2023-07-14T16:38:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.097363616, 2023-07-14T16:38:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000057679 2023-07-14T16:38:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T16:38:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T16:38:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T16:38:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1803, 2023-07-14T16:38:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1777, 2023-07-14T16:38:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 959778, 2023-07-14T16:38:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 655435, 2023-07-14T16:38:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2023-07-14T16:38:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2023-07-14T16:38:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.228657261, 2023-07-14T16:38:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.074671630, 2023-07-14T16:38:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.066259946, 2023-07-14T16:38:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.067921798, 2023-07-14T16:38:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1777, 2023-07-14T16:38:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.099012505, 2023-07-14T16:38:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000055718 2023-07-14T16:38:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 622736, 2023-07-14T16:38:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 229360 2023-07-14T16:38:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T16:38:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1444, 2023-07-14T16:38:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1745, 2023-07-14T16:38:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 485008, 2023-07-14T16:38:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 1005061, 2023-07-14T16:38:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2023-07-14T16:38:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2023-07-14T16:38:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.217645975, 2023-07-14T16:38:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.077677348, 2023-07-14T16:38:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.052572188, 2023-07-14T16:38:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.059085083, 2023-07-14T16:38:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T16:38:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1743, 2023-07-14T16:38:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.140357782, 2023-07-14T16:38:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000080526 2023-07-14T16:38:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2023-07-14T16:38:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-14T16:38:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 556288 2023-07-14T16:38:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2023-07-14T16:38:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2023-07-14T16:38:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2023-07-14T16:38:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2023-07-14T16:38:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2023-07-14T16:38:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2023-07-14T16:38:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7549, 2023-07-14T16:38:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.236401607, 2023-07-14T16:38:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000031315 2023-07-14T16:38:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2023-07-14T16:38:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2023-07-14T16:38:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2023-07-14T16:38:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2023-07-14T16:38:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2023-07-14T16:38:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2023-07-14T16:38:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2023-07-14T16:38:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2023-07-14T16:38:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2023-07-14T16:38:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2023-07-14T16:38:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2023-07-14T16:38:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2023-07-14T16:38:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2023-07-14T16:38:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2023-07-14T16:38:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 323, 2023-07-14T16:38:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2023-07-14T16:38:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 317, 2023-07-14T16:38:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 6, 2023-07-14T16:38:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 322, 2023-07-14T16:38:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2023-07-14T16:38:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2023-07-14T16:38:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2023-07-14T16:38:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 3, 2023-07-14T16:38:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2023-07-14T16:38:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2023-07-14T16:38:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2023-07-14T16:38:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2023-07-14T16:38:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2023-07-14T16:38:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2023-07-14T16:38:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2023-07-14T16:38:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2023-07-14T16:38:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2023-07-14T16:38:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2023-07-14T16:38:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2023-07-14T16:38:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2023-07-14T16:38:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2023-07-14T16:38:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2023-07-14T16:38:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2023-07-14T16:38:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2023-07-14T16:38:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2023-07-14T16:38:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2023-07-14T16:38:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2023-07-14T16:38:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T16:38:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2023-07-14T16:38:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T16:38:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2023-07-14T16:38:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2023-07-14T16:38:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T16:38:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T16:38:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2023-07-14T16:38:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2023-07-14T16:38:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T16:38:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2023-07-14T16:38:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T16:38:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T16:38:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T16:38:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2023-07-14T16:38:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2023-07-14T16:38:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2023-07-14T16:38:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2023-07-14T16:38:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2023-07-14T16:38:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2023-07-14T16:38:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2023-07-14T16:38:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2023-07-14T16:38:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2023-07-14T16:38:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2023-07-14T16:38:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2023-07-14T16:38:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2023-07-14T16:38:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2023-07-14T16:38:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2023-07-14T16:38:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2023-07-14T16:38:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2023-07-14T16:38:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2023-07-14T16:38:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2023-07-14T16:38:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2023-07-14T16:38:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2023-07-14T16:38:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2023-07-14T16:38:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2023-07-14T16:38:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T16:38:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2023-07-14T16:38:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2023-07-14T16:38:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2023-07-14T16:38:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2023-07-14T16:38:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2023-07-14T16:38:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2023-07-14T16:38:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2023-07-14T16:38:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2023-07-14T16:38:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2023-07-14T16:38:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 0, 2023-07-14T16:38:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2023-07-14T16:38:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2023-07-14T16:38:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2023-07-14T16:38:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2023-07-14T16:38:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2023-07-14T16:38:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2023-07-14T16:38:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2023-07-14T16:38:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2023-07-14T16:38:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2023-07-14T16:38:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 0, 2023-07-14T16:38:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 6548122, 2023-07-14T16:38:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2023-07-14T16:38:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 1394 2023-07-14T16:38:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2023-07-14T16:38:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 11, 2023-07-14T16:38:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 326, 2023-07-14T16:38:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 315, 2023-07-14T16:38:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 9, 2023-07-14T16:38:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 94, 2023-07-14T16:38:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 85, 2023-07-14T16:38:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 323, 2023-07-14T16:38:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 686, 2023-07-14T16:38:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 363, 2023-07-14T16:38:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2023-07-14T16:38:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2023-07-14T16:38:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2023-07-14T16:38:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 51592, 2023-07-14T16:38:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 207156 2023-07-14T16:38:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2023-07-14T16:38:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2023-07-14T16:38:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2023-07-14T16:38:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2023-07-14T16:38:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2023-07-14T16:38:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2023-07-14T16:38:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2023-07-14T16:38:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2023-07-14T16:38:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2023-07-14T16:38:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2023-07-14T16:38:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2023-07-14T16:38:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2023-07-14T16:38:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2023-07-14T16:38:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2023-07-14T16:38:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2023-07-14T16:38:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2023-07-14T16:38:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2023-07-14T16:38:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2023-07-14T16:38:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2023-07-14T16:38:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2023-07-14T16:38:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2023-07-14T16:38:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2023-07-14T16:38:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2023-07-14T16:38:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2023-07-14T16:38:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2023-07-14T16:38:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2023-07-14T16:38:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2023-07-14T16:38:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2023-07-14T16:38:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2023-07-14T16:38:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2023-07-14T16:38:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2023-07-14T16:38:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2023-07-14T16:38:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2023-07-14T16:38:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2023-07-14T16:38:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2023-07-14T16:38:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2023-07-14T16:38:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2023-07-14T16:38:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2023-07-14T16:38:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2023-07-14T16:38:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 1, 2023-07-14T16:38:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 2, 2023-07-14T16:38:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 1, 2023-07-14T16:38:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2023-07-14T16:38:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2023-07-14T16:38:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2023-07-14T16:38:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2023-07-14T16:38:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 3168 2023-07-14T16:38:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2023-07-14T16:38:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2023-07-14T16:38:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2023-07-14T16:38:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T16:38:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2023-07-14T16:38:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T16:38:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2023-07-14T16:38:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T16:38:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T16:38:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T16:38:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T16:38:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T16:38:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2023-07-14T16:38:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T16:38:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T16:38:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2023-07-14T16:38:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2023-07-14T16:38:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2023-07-14T16:38:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2023-07-14T16:38:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T16:38:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T16:38:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T16:38:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T16:38:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T16:38:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2023-07-14T16:38:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2023-07-14T16:38:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2023-07-14T16:38:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T16:38:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T16:38:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2023-07-14T16:38:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2023-07-14T16:38:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2023-07-14T16:38:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2023-07-14T16:38:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T16:38:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2023-07-14T16:38:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T16:38:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2023-07-14T16:38:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 190184, 2023-07-14T16:38:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 53, 2023-07-14T16:38:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2023-07-14T16:38:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2023-07-14T16:38:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2023-07-14T16:38:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2023-07-14T16:38:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2023-07-14T16:38:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2023-07-14T16:38:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2023-07-14T16:38:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2023-07-14T16:38:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 7528, 2023-07-14T16:38:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 158, 2023-07-14T16:38:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T16:38:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2023-07-14T16:38:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2023-07-14T16:38:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2023-07-14T16:38:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 263214, 2023-07-14T16:38:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 1487, 2023-07-14T16:38:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2023-07-14T16:38:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2023-07-14T16:38:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2023-07-14T16:38:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2023-07-14T16:38:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2023-07-14T16:38:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2023-07-14T16:38:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2023-07-14T16:38:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 4501, 2023-07-14T16:38:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2023-07-14T16:38:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2023-07-14T16:38:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 4501, 2023-07-14T16:38:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2023-07-14T16:38:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4501, 2023-07-14T16:38:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 5.378891355, 2023-07-14T16:38:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001195043 2023-07-14T16:38:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2023-07-14T16:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2023-07-14T16:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4501, 2023-07-14T16:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 4502 2023-07-14T16:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 4501, 2023-07-14T16:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 4501, 2023-07-14T16:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2023-07-14T16:38:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2023-07-14T16:38:12.645 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2023-07-14T16:38:12.645 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 2811, 2023-07-14T16:38:12.645 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2023-07-14T16:38:12.645 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 1688, 2023-07-14T16:38:12.645 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2023-07-14T16:38:12.645 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2023-07-14T16:38:12.645 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2023-07-14T16:38:12.645 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2023-07-14T16:38:12.646 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2023-07-14T16:38:12.646 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2023-07-14T16:38:12.646 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2023-07-14T16:38:12.646 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2023-07-14T16:38:12.646 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2023-07-14T16:38:12.646 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2023-07-14T16:38:12.646 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2023-07-14T16:38:12.646 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2023-07-14T16:38:12.647 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2023-07-14T16:38:12.647 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2023-07-14T16:38:12.647 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2023-07-14T16:38:12.647 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2023-07-14T16:38:12.647 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2023-07-14T16:38:12.647 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2023-07-14T16:38:12.647 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T16:38:12.647 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2023-07-14T16:38:12.648 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2023-07-14T16:38:12.648 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2023-07-14T16:38:12.648 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2023-07-14T16:38:12.648 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2023-07-14T16:38:12.648 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2023-07-14T16:38:12.648 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2023-07-14T16:38:12.648 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2023-07-14T16:38:12.648 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2023-07-14T16:38:12.649 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2023-07-14T16:38:12.649 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2023-07-14T16:38:12.649 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2023-07-14T16:38:12.649 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2023-07-14T16:38:12.649 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2023-07-14T16:38:12.649 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2023-07-14T16:38:12.649 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2023-07-14T16:38:12.649 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2023-07-14T16:38:12.650 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2023-07-14T16:38:12.650 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2023-07-14T16:38:12.650 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 88, 2023-07-14T16:38:12.650 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2023-07-14T16:38:12.650 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 2, 2023-07-14T16:38:12.650 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 5, 2023-07-14T16:38:12.650 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 6, 2023-07-14T16:38:12.650 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 1, 2023-07-14T16:38:12.651 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2023-07-14T16:38:12.651 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2023-07-14T16:38:12.651 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2023-07-14T16:38:12.651 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2023-07-14T16:38:12.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.651 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2023-07-14T16:38:12.651 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2023-07-14T16:38:12.651 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2023-07-14T16:38:12.652 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2023-07-14T16:38:12.652 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2023-07-14T16:38:12.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.652 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2023-07-14T16:38:12.652 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2023-07-14T16:38:12.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.652 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2023-07-14T16:38:12.652 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2023-07-14T16:38:12.653 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T16:38:12.653 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2023-07-14T16:38:12.653 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2023-07-14T16:38:12.653 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2023-07-14T16:38:12.653 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2023-07-14T16:38:12.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.653 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T16:38:12.653 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:12.654 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2023-07-14T16:38:12.654 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:12.654 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 4931, 2023-07-14T16:38:12.654 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 3770552, 2023-07-14T16:38:12.654 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:12.654 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 4931, 2023-07-14T16:38:12.654 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:12.654 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 4931, 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 3770552, 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.655 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2023-07-14T16:38:12.656 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:12.656 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2023-07-14T16:38:12.656 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:12.656 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:12.656 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:12.656 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:12.656 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:12.657 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 4501, 2023-07-14T16:38:12.657 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 2355474, 2023-07-14T16:38:12.657 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 4501, 2023-07-14T16:38:12.657 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2355474, 2023-07-14T16:38:12.657 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:12.657 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.657 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.657 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.658 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2023-07-14T16:38:12.658 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:12.658 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2023-07-14T16:38:12.658 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:12.658 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:12.658 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:12.659 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:12.659 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:12.659 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 4501, 2023-07-14T16:38:12.659 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 4501, 2023-07-14T16:38:12.659 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 4501, 2023-07-14T16:38:12.659 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 4501, 2023-07-14T16:38:12.659 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:12.659 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.660 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.660 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.660 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2023-07-14T16:38:12.660 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:12.660 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2023-07-14T16:38:12.661 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:12.661 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:12.661 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:12.661 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:12.661 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:12.661 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:12.661 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:12.661 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x5578c4728500": { 2023-07-14T16:38:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2023-07-14T16:38:12.663 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2023-07-14T16:38:12.663 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2023-07-14T16:38:12.663 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2023-07-14T16:38:12.663 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2023-07-14T16:38:12.663 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2023-07-14T16:38:12.663 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2023-07-14T16:38:12.663 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2023-07-14T16:38:12.663 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2023-07-14T16:38:12.664 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2023-07-14T16:38:12.664 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2023-07-14T16:38:12.664 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2023-07-14T16:38:12.664 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2023-07-14T16:38:12.664 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2023-07-14T16:38:12.664 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2023-07-14T16:38:12.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2023-07-14T16:38:12.665 INFO:teuthology.orchestra.run.smithi164.stdout:} 2023-07-14T16:38:13.069 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-07-14T16:38:13.070 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-07-14T16:38:13.070 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2023-07-14T16:38:13.071 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-07-14T16:38:13.082 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-07-14T16:38:13.083 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:38:13.084 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T16:38:13.156 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:38:13.157 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T16:38:13.205 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:38:13.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:13 smithi062 ceph-mon[95543]: pgmap v1034: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:38:13.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:13 smithi081 ceph-mon[127795]: pgmap v1034: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:38:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.484+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/3876703086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 msgr2=0x7f0a340fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.484+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/3876703086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 0x7f0a340fff10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009990 tx=0x7f0a30009ca0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.485+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/3876703086 shutdown_connections 2023-07-14T16:38:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.485+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/3876703086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a34101680 0x7f0a34107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.456 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.485+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/3876703086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 0x7f0a340fff10 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.485+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/3876703086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a34100cf0 0x7f0a34101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.485+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/3876703086 >> 172.21.15.62:0/3876703086 conn(0x7f0a340fb0e0 msgr2=0x7f0a340fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:13.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.485+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/3876703086 shutdown_connections 2023-07-14T16:38:13.457 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.485+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/3876703086 wait complete. 2023-07-14T16:38:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.486+0000 7f0a3b7ff700 1 Processor -- start 2023-07-14T16:38:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.486+0000 7f0a3b7ff700 1 -- start start 2023-07-14T16:38:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.486+0000 7f0a3b7ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 0x7f0a341957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.486+0000 7f0a3b7ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a34100cf0 0x7f0a34195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.486+0000 7f0a3b7ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a34101680 0x7f0a3419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.486+0000 7f0a3b7ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419d350 con 0x7f0a340ffb00 2023-07-14T16:38:13.459 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.486+0000 7f0a3b7ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419d490 con 0x7f0a34100cf0 2023-07-14T16:38:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.486+0000 7f0a3b7ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419d620 con 0x7f0a34101680 2023-07-14T16:38:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 0x7f0a341957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a39d9c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a34101680 0x7f0a3419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 0x7f0a341957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42644/0 (socket says 172.21.15.62:42644) 2023-07-14T16:38:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 -- 172.21.15.62:0/658562437 learned_addr learned my addr 172.21.15.62:0/658562437 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a38d9a700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a34100cf0 0x7f0a34195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:13.460 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 -- 172.21.15.62:0/658562437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a34101680 msgr2=0x7f0a3419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a34101680 0x7f0a3419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 -- 172.21.15.62:0/658562437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a34100cf0 msgr2=0x7f0a34195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a34100cf0 0x7f0a34195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 -- 172.21.15.62:0/658562437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a300095f0 con 0x7f0a340ffb00 2023-07-14T16:38:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.487+0000 7f0a3959b700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 0x7f0a341957d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0a30004730 tx=0x7f0a30009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.488+0000 7f0a2a7fc700 1 -- 172.21.15.62:0/658562437 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3001d070 con 0x7f0a340ffb00 2023-07-14T16:38:13.461 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.488+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3419d8a0 con 0x7f0a340ffb00 2023-07-14T16:38:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.488+0000 7f0a2a7fc700 1 -- 172.21.15.62:0/658562437 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a30004c60 con 0x7f0a340ffb00 2023-07-14T16:38:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.488+0000 7f0a2a7fc700 1 -- 172.21.15.62:0/658562437 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30003af0 con 0x7f0a340ffb00 2023-07-14T16:38:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.488+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3419dcc0 con 0x7f0a340ffb00 2023-07-14T16:38:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.489+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a341070e0 con 0x7f0a340ffb00 2023-07-14T16:38:13.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.491+0000 7f0a2a7fc700 1 -- 172.21.15.62:0/658562437 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0a300047d0 con 0x7f0a340ffb00 2023-07-14T16:38:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.492+0000 7f0a2a7fc700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a200a11a0 0x7f0a200a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.492+0000 7f0a2a7fc700 1 -- 172.21.15.62:0/658562437 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a300cd880 con 0x7f0a340ffb00 2023-07-14T16:38:13.463 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.492+0000 7f0a38d9a700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a200a11a0 0x7f0a200a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:13.464 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.493+0000 7f0a38d9a700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a200a11a0 0x7f0a200a3650 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0a24007be0 tx=0x7f0a240073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.496+0000 7f0a2a7fc700 1 -- 172.21.15.62:0/658562437 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0a30097010 con 0x7f0a340ffb00 2023-07-14T16:38:13.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:13 smithi164 ceph-mon[97427]: pgmap v1034: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T16:38:13.676 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.705+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a3402c850 con 0x7f0a340ffb00 2023-07-14T16:38:13.678 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.707+0000 7f0a2a7fc700 1 -- 172.21.15.62:0/658562437 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7f0a3009a630 con 0x7f0a340ffb00 2023-07-14T16:38:13.679 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:38:13.679 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:38:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a200a11a0 msgr2=0x7f0a200a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a200a11a0 0x7f0a200a3650 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0a24007be0 tx=0x7f0a240073d0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 msgr2=0x7f0a341957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 0x7f0a341957d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0a30004730 tx=0x7f0a30009580 comp rx=0 tx=0).stop 2023-07-14T16:38:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 shutdown_connections 2023-07-14T16:38:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a34101680 0x7f0a3419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a340ffb00 0x7f0a341957d0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f0a200a11a0 0x7f0a200a3650 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 --2- 172.21.15.62:0/658562437 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a34100cf0 0x7f0a34195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 >> 172.21.15.62:0/658562437 conn(0x7f0a340fb0e0 msgr2=0x7f0a34104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 shutdown_connections 2023-07-14T16:38:13.683 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.710+0000 7f0a3b7ff700 1 -- 172.21.15.62:0/658562437 wait complete. 2023-07-14T16:38:13.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:38:13.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:38:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/1709996844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74101670 msgr2=0x7f2d74107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/1709996844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74101670 0x7f2d74107f30 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00b0a0 tx=0x7f2d6c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/1709996844 shutdown_connections 2023-07-14T16:38:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/1709996844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74101670 0x7f2d74107f30 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/1709996844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d740ffaf0 0x7f2d740fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/1709996844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d74100ce0 0x7f2d74101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/1709996844 >> 172.21.15.62:0/1709996844 conn(0x7f2d740fb130 msgr2=0x7f2d740fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/1709996844 shutdown_connections 2023-07-14T16:38:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.938+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/1709996844 wait complete. 2023-07-14T16:38:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.939+0000 7f2d7b5ff700 1 Processor -- start 2023-07-14T16:38:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.939+0000 7f2d7b5ff700 1 -- start start 2023-07-14T16:38:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.939+0000 7f2d7b5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d740ffaf0 0x7f2d74195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:13.912 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.939+0000 7f2d7b5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74100ce0 0x7f2d74195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.939+0000 7f2d7b5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d74101670 0x7f2d7419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d7b5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419d400 con 0x7f2d74100ce0 2023-07-14T16:38:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d7b5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419d540 con 0x7f2d740ffaf0 2023-07-14T16:38:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d7b5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419d6d0 con 0x7f2d74101670 2023-07-14T16:38:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74100ce0 0x7f2d74195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74100ce0 0x7f2d74195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42658/0 (socket says 172.21.15.62:42658) 2023-07-14T16:38:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 -- 172.21.15.62:0/3237341597 learned_addr learned my addr 172.21.15.62:0/3237341597 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:13.913 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d7939b700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d740ffaf0 0x7f2d74195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d79b9c700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d74101670 0x7f2d7419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 -- 172.21.15.62:0/3237341597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d74101670 msgr2=0x7f2d7419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d74101670 0x7f2d7419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 -- 172.21.15.62:0/3237341597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d740ffaf0 msgr2=0x7f2d74195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d740ffaf0 0x7f2d74195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 -- 172.21.15.62:0/3237341597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c009d00 con 0x7f2d74100ce0 2023-07-14T16:38:13.914 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.940+0000 7f2d78b9a700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74100ce0 0x7f2d74195dc0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400d950 tx=0x7f2d6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.941+0000 7f2d6a7fc700 1 -- 172.21.15.62:0/3237341597 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400f890 con 0x7f2d74100ce0 2023-07-14T16:38:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.941+0000 7f2d6a7fc700 1 -- 172.21.15.62:0/3237341597 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d6400fe80 con 0x7f2d74100ce0 2023-07-14T16:38:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.941+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7419d9b0 con 0x7f2d74100ce0 2023-07-14T16:38:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.941+0000 7f2d6a7fc700 1 -- 172.21.15.62:0/3237341597 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400e3b0 con 0x7f2d74100ce0 2023-07-14T16:38:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.941+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7419df10 con 0x7f2d74100ce0 2023-07-14T16:38:13.915 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.942+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d74107020 con 0x7f2d74100ce0 2023-07-14T16:38:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.944+0000 7f2d6a7fc700 1 -- 172.21.15.62:0/3237341597 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2d64010460 con 0x7f2d74100ce0 2023-07-14T16:38:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.945+0000 7f2d6a7fc700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d600a11d0 0x7f2d600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.945+0000 7f2d6a7fc700 1 -- 172.21.15.62:0/3237341597 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d640cc6c0 con 0x7f2d74100ce0 2023-07-14T16:38:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.946+0000 7f2d7939b700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d600a11d0 0x7f2d600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.946+0000 7f2d7939b700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d600a11d0 0x7f2d600a3680 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2d74100b40 tx=0x7f2d70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:13.920 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:13.950+0000 7f2d6a7fc700 1 -- 172.21.15.62:0/3237341597 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2d64095dc0 con 0x7f2d74100ce0 2023-07-14T16:38:14.130 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.159+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d740611c0 con 0x7f2d74100ce0 2023-07-14T16:38:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.160+0000 7f2d6a7fc700 1 -- 172.21.15.62:0/3237341597 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d640993e0 con 0x7f2d74100ce0 2023-07-14T16:38:14.131 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:38:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.162+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d600a11d0 msgr2=0x7f2d600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.162+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d600a11d0 0x7f2d600a3680 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2d74100b40 tx=0x7f2d70009260 comp rx=0 tx=0).stop 2023-07-14T16:38:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.162+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74100ce0 msgr2=0x7f2d74195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.162+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74100ce0 0x7f2d74195dc0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400d950 tx=0x7f2d6400dc60 comp rx=0 tx=0).stop 2023-07-14T16:38:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.163+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 shutdown_connections 2023-07-14T16:38:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.163+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d74101670 0x7f2d7419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.163+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f2d600a11d0 0x7f2d600a3680 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.163+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d740ffaf0 0x7f2d74195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.163+0000 7f2d7b5ff700 1 --2- 172.21.15.62:0/3237341597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d74100ce0 0x7f2d74195dc0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.163+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 >> 172.21.15.62:0/3237341597 conn(0x7f2d740fb130 msgr2=0x7f2d74104160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.163+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 shutdown_connections 2023-07-14T16:38:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.163+0000 7f2d7b5ff700 1 -- 172.21.15.62:0/3237341597 wait complete. 2023-07-14T16:38:14.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:38:14.358 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 -- 172.21.15.62:0/1326775394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c0ffae0 msgr2=0x7f3c0c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1326775394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c0ffae0 0x7f3c0c0ffef0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009990 tx=0x7f3bfc009ca0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 -- 172.21.15.62:0/1326775394 shutdown_connections 2023-07-14T16:38:14.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1326775394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c0c101660 0x7f3c0c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1326775394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c0ffae0 0x7f3c0c0ffef0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1326775394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c0c100cd0 0x7f3c0c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 -- 172.21.15.62:0/1326775394 >> 172.21.15.62:0/1326775394 conn(0x7f3c0c0fb0e0 msgr2=0x7f3c0c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:14.359 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 -- 172.21.15.62:0/1326775394 shutdown_connections 2023-07-14T16:38:14.360 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.387+0000 7f3c10fff700 1 -- 172.21.15.62:0/1326775394 wait complete. 2023-07-14T16:38:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.388+0000 7f3c10fff700 1 Processor -- start 2023-07-14T16:38:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.388+0000 7f3c10fff700 1 -- start start 2023-07-14T16:38:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c10fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c0c0ffae0 0x7f3c0c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c10fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c0c100cd0 0x7f3c0c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c10fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c101660 0x7f3c0c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c10fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c19f520 con 0x7f3c0c101660 2023-07-14T16:38:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c10fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c19f660 con 0x7f3c0c100cd0 2023-07-14T16:38:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c10fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c19f7f0 con 0x7f3c0c0ffae0 2023-07-14T16:38:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c101660 0x7f3c0c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c101660 0x7f3c0c19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42672/0 (socket says 172.21.15.62:42672) 2023-07-14T16:38:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0ad9d700 1 -- 172.21.15.62:0/1866337320 learned_addr learned my addr 172.21.15.62:0/1866337320 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0a59c700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c0c0ffae0 0x7f3c0c1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c03fff700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c0c100cd0 0x7f3c0c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0ad9d700 1 -- 172.21.15.62:0/1866337320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c0c0ffae0 msgr2=0x7f3c0c1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0ad9d700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c0c0ffae0 0x7f3c0c1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0ad9d700 1 -- 172.21.15.62:0/1866337320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c0c100cd0 msgr2=0x7f3c0c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0ad9d700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c0c100cd0 0x7f3c0c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c0ad9d700 1 -- 172.21.15.62:0/1866337320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0095f0 con 0x7f3c0c101660 2023-07-14T16:38:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.389+0000 7f3c03fff700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c0c100cd0 0x7f3c0c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:38:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.390+0000 7f3c0ad9d700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c101660 0x7f3c0c19ef50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400ef30 tx=0x7f3c0400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.390+0000 7f3c037fe700 1 -- 172.21.15.62:0/1866337320 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04018070 con 0x7f3c0c101660 2023-07-14T16:38:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.390+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0c19fa70 con 0x7f3c0c101660 2023-07-14T16:38:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.390+0000 7f3c037fe700 1 -- 172.21.15.62:0/1866337320 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c0400f040 con 0x7f3c0c101660 2023-07-14T16:38:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.390+0000 7f3c037fe700 1 -- 172.21.15.62:0/1866337320 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400cc70 con 0x7f3c0c101660 2023-07-14T16:38:14.364 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.390+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c19ff30 con 0x7f3c0c101660 2023-07-14T16:38:14.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.391+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0c10b160 con 0x7f3c0c101660 2023-07-14T16:38:14.365 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.393+0000 7f3c037fe700 1 -- 172.21.15.62:0/1866337320 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3c04007500 con 0x7f3c0c101660 2023-07-14T16:38:14.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.394+0000 7f3c037fe700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3bf80a1150 0x7f3bf80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:14.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.396+0000 7f3c037fe700 1 -- 172.21.15.62:0/1866337320 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3c040cc4b0 con 0x7f3c0c101660 2023-07-14T16:38:14.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.396+0000 7f3c0a59c700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3bf80a1150 0x7f3bf80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:14.366 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.396+0000 7f3c0a59c700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3bf80a1150 0x7f3bf80a3600 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006010 tx=0x7f3bfc00b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:14.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:14 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/658562437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:38:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.399+0000 7f3c037fe700 1 -- 172.21.15.62:0/1866337320 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3c04095c40 con 0x7f3c0c101660 2023-07-14T16:38:14.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:14 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/658562437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:38:14.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:14 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/658562437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:38:14.580 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.610+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c0c1a0540 con 0x7f3c0c101660 2023-07-14T16:38:14.581 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.611+0000 7f3c037fe700 1 -- 172.21.15.62:0/1866337320 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c04099260 con 0x7f3c0c101660 2023-07-14T16:38:14.581 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:38:14.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3bf80a1150 msgr2=0x7f3bf80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3bf80a1150 0x7f3bf80a3600 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006010 tx=0x7f3bfc00b2a0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.583 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c101660 msgr2=0x7f3c0c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c101660 0x7f3c0c19ef50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400ef30 tx=0x7f3c0400c3a0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 shutdown_connections 2023-07-14T16:38:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c0c101660 0x7f3c0c19ef50 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3c0c0ffae0 0x7f3c0c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f3bf80a1150 0x7f3bf80a3600 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 --2- 172.21.15.62:0/1866337320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c0c100cd0 0x7f3c0c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.584 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 >> 172.21.15.62:0/1866337320 conn(0x7f3c0c0fb0e0 msgr2=0x7f3c0c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:14.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 shutdown_connections 2023-07-14T16:38:14.585 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.613+0000 7f3c10fff700 1 -- 172.21.15.62:0/1866337320 wait complete. 2023-07-14T16:38:14.601 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:38:14.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.838+0000 7f38057ff700 1 -- 172.21.15.62:0/738832968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 msgr2=0x7f38000ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.838+0000 7f38057ff700 1 --2- 172.21.15.62:0/738832968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 0x7f38000ffef0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009990 tx=0x7f37f0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.839+0000 7f38057ff700 1 -- 172.21.15.62:0/738832968 shutdown_connections 2023-07-14T16:38:14.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.839+0000 7f38057ff700 1 --2- 172.21.15.62:0/738832968 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3800101660 0x7f3800107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.839+0000 7f38057ff700 1 --2- 172.21.15.62:0/738832968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 0x7f38000ffef0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.839+0000 7f38057ff700 1 --2- 172.21.15.62:0/738832968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3800100cd0 0x7f3800101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.839+0000 7f38057ff700 1 -- 172.21.15.62:0/738832968 >> 172.21.15.62:0/738832968 conn(0x7f38000fb0e0 msgr2=0x7f38000fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:14.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.839+0000 7f38057ff700 1 -- 172.21.15.62:0/738832968 shutdown_connections 2023-07-14T16:38:14.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.839+0000 7f38057ff700 1 -- 172.21.15.62:0/738832968 wait complete. 2023-07-14T16:38:14.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.840+0000 7f38057ff700 1 Processor -- start 2023-07-14T16:38:14.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.840+0000 7f38057ff700 1 -- start start 2023-07-14T16:38:14.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f38057ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 0x7f38000742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:14.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f38057ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3800100cd0 0x7f3800072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:14.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f38057ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3800101660 0x7f3800072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f38057ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000735e0 con 0x7f38000ffae0 2023-07-14T16:38:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f38057ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3800073750 con 0x7f3800101660 2023-07-14T16:38:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f38057ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380019f6e0 con 0x7f3800100cd0 2023-07-14T16:38:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 0x7f38000742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 0x7f38000742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42684/0 (socket says 172.21.15.62:42684) 2023-07-14T16:38:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37feffd700 1 -- 172.21.15.62:0/2792417533 learned_addr learned my addr 172.21.15.62:0/2792417533 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37ff7fe700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3800101660 0x7f3800072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37fe7fc700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3800100cd0 0x7f3800072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37feffd700 1 -- 172.21.15.62:0/2792417533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3800100cd0 msgr2=0x7f3800072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37feffd700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3800100cd0 0x7f3800072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37feffd700 1 -- 172.21.15.62:0/2792417533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3800101660 msgr2=0x7f3800072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37feffd700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3800101660 0x7f3800072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37feffd700 1 -- 172.21.15.62:0/2792417533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00095f0 con 0x7f38000ffae0 2023-07-14T16:38:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.841+0000 7f37ff7fe700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3800101660 0x7f3800072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:38:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.842+0000 7f37feffd700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 0x7f38000742c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f37f00059d0 tx=0x7f37f0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.842+0000 7f37e7fff700 1 -- 172.21.15.62:0/2792417533 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f001d070 con 0x7f38000ffae0 2023-07-14T16:38:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.842+0000 7f37e7fff700 1 -- 172.21.15.62:0/2792417533 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37f0004140 con 0x7f38000ffae0 2023-07-14T16:38:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.842+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380019f960 con 0x7f38000ffae0 2023-07-14T16:38:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.842+0000 7f37e7fff700 1 -- 172.21.15.62:0/2792417533 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0022e00 con 0x7f38000ffae0 2023-07-14T16:38:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.842+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380019fd80 con 0x7f38000ffae0 2023-07-14T16:38:14.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.843+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3800106a10 con 0x7f38000ffae0 2023-07-14T16:38:14.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.845+0000 7f37e7fff700 1 -- 172.21.15.62:0/2792417533 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f37f0003cb0 con 0x7f38000ffae0 2023-07-14T16:38:14.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.847+0000 7f37e7fff700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f37e80a1150 0x7f37e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:14.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.847+0000 7f37e7fff700 1 -- 172.21.15.62:0/2792417533 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f37f00ce650 con 0x7f38000ffae0 2023-07-14T16:38:14.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.847+0000 7f37fe7fc700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f37e80a1150 0x7f37e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:14.817 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.848+0000 7f37fe7fc700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f37e80a1150 0x7f37e80a3600 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005ea0 tx=0x7f37f4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:14.818 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:14.849+0000 7f37e7fff700 1 -- 172.21.15.62:0/2792417533 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f37f0097de0 con 0x7f38000ffae0 2023-07-14T16:38:15.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.059+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f380002cac0 con 0x7f38000ffae0 2023-07-14T16:38:15.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.060+0000 7f37e7fff700 1 -- 172.21.15.62:0/2792417533 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37f0027020 con 0x7f38000ffae0 2023-07-14T16:38:15.030 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:38:15.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f37e80a1150 msgr2=0x7f37e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f37e80a1150 0x7f37e80a3600 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005ea0 tx=0x7f37f4005e30 comp rx=0 tx=0).stop 2023-07-14T16:38:15.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 msgr2=0x7f38000742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 0x7f38000742c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f37f00059d0 tx=0x7f37f0005740 comp rx=0 tx=0).stop 2023-07-14T16:38:15.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 shutdown_connections 2023-07-14T16:38:15.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f37e80a1150 0x7f37e80a3600 secure :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005ea0 tx=0x7f37f4005e30 comp rx=0 tx=0).stop 2023-07-14T16:38:15.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3800101660 0x7f3800072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38000ffae0 0x7f38000742c0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 --2- 172.21.15.62:0/2792417533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3800100cd0 0x7f3800072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 >> 172.21.15.62:0/2792417533 conn(0x7f38000fb0e0 msgr2=0x7f38000fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:15.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 shutdown_connections 2023-07-14T16:38:15.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.062+0000 7f38057ff700 1 -- 172.21.15.62:0/2792417533 wait complete. 2023-07-14T16:38:15.052 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T16:38:15.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.287+0000 7f5b7abff700 1 -- 172.21.15.62:0/64639753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 msgr2=0x7f5b74109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.287+0000 7f5b7abff700 1 --2- 172.21.15.62:0/64639753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 0x7f5b74109e20 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00b0a0 tx=0x7f5b6c00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.288+0000 7f5b7abff700 1 -- 172.21.15.62:0/64639753 shutdown_connections 2023-07-14T16:38:15.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.288+0000 7f5b7abff700 1 --2- 172.21.15.62:0/64639753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 0x7f5b74109e20 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.288+0000 7f5b7abff700 1 --2- 172.21.15.62:0/64639753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b74074550 0x7f5b740729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.288+0000 7f5b7abff700 1 --2- 172.21.15.62:0/64639753 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b74072f70 0x7f5b740733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.288+0000 7f5b7abff700 1 -- 172.21.15.62:0/64639753 >> 172.21.15.62:0/64639753 conn(0x7f5b740fb0e0 msgr2=0x7f5b740fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:15.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.288+0000 7f5b7abff700 1 -- 172.21.15.62:0/64639753 shutdown_connections 2023-07-14T16:38:15.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.288+0000 7f5b7abff700 1 -- 172.21.15.62:0/64639753 wait complete. 2023-07-14T16:38:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7abff700 1 Processor -- start 2023-07-14T16:38:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7abff700 1 -- start start 2023-07-14T16:38:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7abff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b74072f70 0x7f5b74197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7abff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b74074550 0x7f5b74197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:15.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7abff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 0x7f5b7419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7abff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7419f5e0 con 0x7f5b741079a0 2023-07-14T16:38:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7abff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7419f720 con 0x7f5b74072f70 2023-07-14T16:38:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7abff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7419f8b0 con 0x7f5b74074550 2023-07-14T16:38:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.289+0000 7f5b7919c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 0x7f5b7419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b74072f70 0x7f5b74197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b73fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b74074550 0x7f5b74197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:15.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b74072f70 0x7f5b74197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.62:33728/0 (socket says 172.21.15.62:33728) 2023-07-14T16:38:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 -- 172.21.15.62:0/1222569265 learned_addr learned my addr 172.21.15.62:0/1222569265 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b73fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b74074550 0x7f5b74197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39370/0 (socket says 172.21.15.62:39370) 2023-07-14T16:38:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 -- 172.21.15.62:0/1222569265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b74074550 msgr2=0x7f5b74197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b74074550 0x7f5b74197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 -- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 msgr2=0x7f5b7419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 0x7f5b7419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 -- 172.21.15.62:0/1222569265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c009d00 con 0x7f5b74072f70 2023-07-14T16:38:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7919c700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 0x7f5b7419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:38:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.290+0000 7f5b7899b700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b74072f70 0x7f5b74197a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800e970 tx=0x7f5b6800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.291+0000 7f5b71ffb700 1 -- 172.21.15.62:0/1222569265 <== mon.2 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68009790 con 0x7f5b74072f70 2023-07-14T16:38:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.291+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7419fb90 con 0x7f5b74072f70 2023-07-14T16:38:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.291+0000 7f5b71ffb700 1 -- 172.21.15.62:0/1222569265 <== mon.2 v2:172.21.15.81:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b6800c7e0 con 0x7f5b74072f70 2023-07-14T16:38:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.291+0000 7f5b71ffb700 1 -- 172.21.15.62:0/1222569265 <== mon.2 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68005440 con 0x7f5b74072f70 2023-07-14T16:38:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.291+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b741a00f0 con 0x7f5b74072f70 2023-07-14T16:38:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.292+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b74106050 con 0x7f5b74072f70 2023-07-14T16:38:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.294+0000 7f5b71ffb700 1 -- 172.21.15.62:0/1222569265 <== mon.2 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b6800c950 con 0x7f5b74072f70 2023-07-14T16:38:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.295+0000 7f5b71ffb700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b640a1150 0x7f5b640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.296+0000 7f5b71ffb700 1 -- 172.21.15.62:0/1222569265 <== mon.2 v2:172.21.15.81:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b68014070 con 0x7f5b74072f70 2023-07-14T16:38:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.296+0000 7f5b73fff700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b640a1150 0x7f5b640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.296+0000 7f5b73fff700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b640a1150 0x7f5b640a3600 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5b74074270 tx=0x7f5b6000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:15.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.298+0000 7f5b71ffb700 1 -- 172.21.15.62:0/1222569265 <== mon.2 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5b680968a0 con 0x7f5b74072f70 2023-07-14T16:38:15.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:15 smithi062 ceph-mon[95543]: pgmap v1035: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:15.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:15 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3237341597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:15 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1866337320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:15 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/2792417533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:15 smithi081 ceph-mon[127795]: pgmap v1035: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:15.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:15 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3237341597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:15 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1866337320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:15 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/2792417533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.493 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.523+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5b7402c850 con 0x7f5b74072f70 2023-07-14T16:38:15.494 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.524+0000 7f5b71ffb700 1 -- 172.21.15.62:0/1222569265 <== mon.2 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+587 (secure 0 0 0) 0x7f5b68096020 con 0x7f5b74072f70 2023-07-14T16:38:15.494 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/105803689 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:15.494 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1873685153 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:15.494 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/1177875314 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:15.494 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/105803689 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:15.495 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1315520653 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:15.495 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/1177875314 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:15.495 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/926428906 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:15.495 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/804633476 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:15.495 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3081660148 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:15.495 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/743698528 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:15.496 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b640a1150 msgr2=0x7f5b640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b640a1150 0x7f5b640a3600 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5b74074270 tx=0x7f5b6000b410 comp rx=0 tx=0).stop 2023-07-14T16:38:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b74072f70 msgr2=0x7f5b74197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b74072f70 0x7f5b74197a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800e970 tx=0x7f5b6800ec80 comp rx=0 tx=0).stop 2023-07-14T16:38:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 shutdown_connections 2023-07-14T16:38:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b741079a0 0x7f5b7419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f5b640a1150 0x7f5b640a3600 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.497 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b74072f70 0x7f5b74197a60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 --2- 172.21.15.62:0/1222569265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b74074550 0x7f5b74197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.526+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 >> 172.21.15.62:0/1222569265 conn(0x7f5b740fb0e0 msgr2=0x7f5b740fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.527+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 shutdown_connections 2023-07-14T16:38:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.527+0000 7f5b7abff700 1 -- 172.21.15.62:0/1222569265 wait complete. 2023-07-14T16:38:15.498 INFO:teuthology.orchestra.run.smithi062.stderr:listed 10 entries 2023-07-14T16:38:15.515 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-07-14T16:38:15.515 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T16:38:15.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:15 smithi164 ceph-mon[97427]: pgmap v1035: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-14T16:38:15.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:15 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3237341597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:15 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1866337320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:15 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/2792417533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:38:15.559 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T16:38:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3558728495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f41079c0 msgr2=0x7fb0f4109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3558728495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f41079c0 0x7fb0f4109e40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00b0a0 tx=0x7fb0ec00b3b0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3558728495 shutdown_connections 2023-07-14T16:38:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3558728495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f41079c0 0x7fb0f4109e40 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3558728495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f4074550 0x7fb0f40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3558728495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f4072f70 0x7fb0f40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3558728495 >> 172.21.15.62:0/3558728495 conn(0x7fb0f40fb0e0 msgr2=0x7fb0f40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3558728495 shutdown_connections 2023-07-14T16:38:15.803 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.832+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3558728495 wait complete. 2023-07-14T16:38:15.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.833+0000 7fb0fb5ff700 1 Processor -- start 2023-07-14T16:38:15.805 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0fb5ff700 1 -- start start 2023-07-14T16:38:15.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0fb5ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f4072f70 0x7fb0f4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:15.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0fb5ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f4074550 0x7fb0f4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:15.806 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0fb5ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f41079c0 0x7fb0f419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0fb5ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419f660 con 0x7fb0f4072f70 2023-07-14T16:38:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0fb5ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419f7a0 con 0x7fb0f41079c0 2023-07-14T16:38:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0fb5ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f419f930 con 0x7fb0f4074550 2023-07-14T16:38:15.807 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0f939b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f4072f70 0x7fb0f4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0f939b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f4072f70 0x7fb0f4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42708/0 (socket says 172.21.15.62:42708) 2023-07-14T16:38:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.834+0000 7fb0f939b700 1 -- 172.21.15.62:0/3822270588 learned_addr learned my addr 172.21.15.62:0/3822270588 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f8b9a700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f4074550 0x7fb0f4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:15.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f9b9c700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f41079c0 0x7fb0f419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:15.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f939b700 1 -- 172.21.15.62:0/3822270588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f4074550 msgr2=0x7fb0f4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.809 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f939b700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f4074550 0x7fb0f4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f939b700 1 -- 172.21.15.62:0/3822270588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f41079c0 msgr2=0x7fb0f419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f939b700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f41079c0 0x7fb0f419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f939b700 1 -- 172.21.15.62:0/3822270588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec009d00 con 0x7fb0f4072f70 2023-07-14T16:38:15.810 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f9b9c700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f41079c0 0x7fb0f419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T16:38:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0f939b700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f4072f70 0x7fb0f4197ae0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000d980 tx=0x7fb0f000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0ea7fc700 1 -- 172.21.15.62:0/3822270588 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0009b40 con 0x7fb0f4072f70 2023-07-14T16:38:15.811 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0ea7fc700 1 -- 172.21.15.62:0/3822270588 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0f000de60 con 0x7fb0f4072f70 2023-07-14T16:38:15.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f419fc10 con 0x7fb0f4072f70 2023-07-14T16:38:15.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0ea7fc700 1 -- 172.21.15.62:0/3822270588 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000f620 con 0x7fb0f4072f70 2023-07-14T16:38:15.812 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.835+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f41a0170 con 0x7fb0f4072f70 2023-07-14T16:38:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.837+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f4106070 con 0x7fb0f4072f70 2023-07-14T16:38:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.839+0000 7fb0ea7fc700 1 -- 172.21.15.62:0/3822270588 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb0f0010460 con 0x7fb0f4072f70 2023-07-14T16:38:15.814 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.840+0000 7fb0ea7fc700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0e00a1220 0x7fb0e00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.840+0000 7fb0ea7fc700 1 -- 172.21.15.62:0/3822270588 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0f00cc4b0 con 0x7fb0f4072f70 2023-07-14T16:38:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.840+0000 7fb0f8b9a700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0e00a1220 0x7fb0e00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:15.815 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.841+0000 7fb0f8b9a700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0e00a1220 0x7fb0e00a36d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4006fd0 tx=0x7fb0e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:15.816 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:15.844+0000 7fb0ea7fc700 1 -- 172.21.15.62:0/3822270588 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb0f0095bb0 con 0x7fb0f4072f70 2023-07-14T16:38:16.024 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.055+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0f402c850 con 0x7fb0f4072f70 2023-07-14T16:38:16.026 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.056+0000 7fb0ea7fc700 1 -- 172.21.15.62:0/3822270588 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22500 (secure 0 0 0) 0x7fb0f00991d0 con 0x7fb0f4072f70 2023-07-14T16:38:16.026 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-07-14T16:38:16.027 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"1356e48a-2260-11ee-9b33-001a4aab830c","created":"2023-07-14T16:05:22.767545+0000","modified":"2023-07-14T16:14:20.655639+0000","last_up_change":"2023-07-14T16:13:00.797215+0000","last_in_change":"2023-07-14T16:12:39.838429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T16:08:57.974260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T16:13:56.880628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T16:13:58.253604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ae1fdd9-0e93-4148-af93-87c47080d59d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6803","nonce":2960730798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6805","nonce":2960730798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6809","nonce":2960730798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2960730798},{"type":"v1","addr":"172.21.15.62:6807","nonce":2960730798}]},"public_addr":"172.21.15.62:6803/2960730798","cluster_addr":"172.21.15.62:6805/2960730798","heartbeat_back_addr":"172.21.15.62:6809/2960730798","heartbeat_front_addr":"172.21.15.62:6807/2960730798","state":["exists","up"]},{"osd":1,"uuid":"ebe4ff93-a6a2-4cc9-af7c-6beabca4b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6801","nonce":3123830761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3123830761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3123830761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3123830761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3123830761}]},"public_addr":"172.21.15.81:6801/3123830761","cluster_addr":"172.21.15.81:6803/3123830761","heartbeat_back_addr":"172.21.15.81:6807/3123830761","heartbeat_front_addr":"172.21.15.81:6805/3123830761","state":["exists","up"]},{"osd":2,"uuid":"831ec85f-142a-4e71-bd2c-9f2c1e8d8d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6801","nonce":3167529694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6803","nonce":3167529694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6807","nonce":3167529694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3167529694},{"type":"v1","addr":"172.21.15.164:6805","nonce":3167529694}]},"public_addr":"172.21.15.164:6801/3167529694","cluster_addr":"172.21.15.164:6803/3167529694","heartbeat_back_addr":"172.21.15.164:6807/3167529694","heartbeat_front_addr":"172.21.15.164:6805/3167529694","state":["exists","up"]},{"osd":3,"uuid":"9b2e9feb-b1c8-471d-8ded-98528b1b5890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6811","nonce":1003536176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6813","nonce":1003536176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6817","nonce":1003536176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1003536176},{"type":"v1","addr":"172.21.15.62:6815","nonce":1003536176}]},"public_addr":"172.21.15.62:6811/1003536176","cluster_addr":"172.21.15.62:6813/1003536176","heartbeat_back_addr":"172.21.15.62:6817/1003536176","heartbeat_front_addr":"172.21.15.62:6815/1003536176","state":["exists","up"]},{"osd":4,"uuid":"d6647685-0173-4443-b612-13675c942486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6809","nonce":550163990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6811","nonce":550163990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6815","nonce":550163990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":550163990},{"type":"v1","addr":"172.21.15.81:6813","nonce":550163990}]},"public_addr":"172.21.15.81:6809/550163990","cluster_addr":"172.21.15.81:6811/550163990","heartbeat_back_addr":"172.21.15.81:6815/550163990","heartbeat_front_addr":"172.21.15.81:6813/550163990","state":["exists","up"]},{"osd":5,"uuid":"ee43949b-be2c-40e7-8efa-b8768a6d0165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6809","nonce":1499523865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6811","nonce":1499523865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6815","nonce":1499523865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1499523865},{"type":"v1","addr":"172.21.15.164:6813","nonce":1499523865}]},"public_addr":"172.21.15.164:6809/1499523865","cluster_addr":"172.21.15.164:6811/1499523865","heartbeat_back_addr":"172.21.15.164:6815/1499523865","heartbeat_front_addr":"172.21.15.164:6813/1499523865","state":["exists","up"]},{"osd":6,"uuid":"47c5b778-8a5b-4a01-9976-c0ddf8635724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6819","nonce":348099216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6821","nonce":348099216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6825","nonce":348099216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":348099216},{"type":"v1","addr":"172.21.15.62:6823","nonce":348099216}]},"public_addr":"172.21.15.62:6819/348099216","cluster_addr":"172.21.15.62:6821/348099216","heartbeat_back_addr":"172.21.15.62:6825/348099216","heartbeat_front_addr":"172.21.15.62:6823/348099216","state":["exists","up"]},{"osd":7,"uuid":"02e215e4-f68b-44d5-8172-836fed42ce7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6817","nonce":1584123297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6819","nonce":1584123297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6823","nonce":1584123297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":1584123297},{"type":"v1","addr":"172.21.15.81:6821","nonce":1584123297}]},"public_addr":"172.21.15.81:6817/1584123297","cluster_addr":"172.21.15.81:6819/1584123297","heartbeat_back_addr":"172.21.15.81:6823/1584123297","heartbeat_front_addr":"172.21.15.81:6821/1584123297","state":["exists","up"]},{"osd":8,"uuid":"30aea523-6f39-4ad1-8852-ec62c90e4c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6817","nonce":1299247730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6819","nonce":1299247730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6823","nonce":1299247730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1299247730},{"type":"v1","addr":"172.21.15.164:6821","nonce":1299247730}]},"public_addr":"172.21.15.164:6817/1299247730","cluster_addr":"172.21.15.164:6819/1299247730","heartbeat_back_addr":"172.21.15.164:6823/1299247730","heartbeat_front_addr":"172.21.15.164:6821/1299247730","state":["exists","up"]},{"osd":9,"uuid":"52d65846-0e8d-4cef-b984-1e4a5e97f5b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6827","nonce":1938724436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6829","nonce":1938724436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6833","nonce":1938724436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1938724436},{"type":"v1","addr":"172.21.15.62:6831","nonce":1938724436}]},"public_addr":"172.21.15.62:6827/1938724436","cluster_addr":"172.21.15.62:6829/1938724436","heartbeat_back_addr":"172.21.15.62:6833/1938724436","heartbeat_front_addr":"172.21.15.62:6831/1938724436","state":["exists","up"]},{"osd":10,"uuid":"1a4dc80e-e9cd-4da4-b65b-6ef47b74aec1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6825","nonce":737826799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6827","nonce":737826799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6831","nonce":737826799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":737826799},{"type":"v1","addr":"172.21.15.81:6829","nonce":737826799}]},"public_addr":"172.21.15.81:6825/737826799","cluster_addr":"172.21.15.81:6827/737826799","heartbeat_back_addr":"172.21.15.81:6831/737826799","heartbeat_front_addr":"172.21.15.81:6829/737826799","state":["exists","up"]},{"osd":11,"uuid":"7fe4fc0d-eb60-4df7-869d-1925ffddd634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6825","nonce":1185509832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6827","nonce":1185509832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6831","nonce":1185509832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1185509832},{"type":"v1","addr":"172.21.15.164:6829","nonce":1185509832}]},"public_addr":"172.21.15.164:6825/1185509832","cluster_addr":"172.21.15.164:6827/1185509832","heartbeat_back_addr":"172.21.15.164:6831/1185509832","heartbeat_front_addr":"172.21.15.164:6829/1185509832","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:07:59.261067+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:26.391706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:08:54.172258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:21.699583+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:09:48.262749+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:15.452505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:10:42.429487+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:09.356847+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:11:37.455952+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:04.348969+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:31.574596+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T16:12:58.991528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1873685153":"2023-07-15T16:06:19.369987+0000","172.21.15.62:6801/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/105803689":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/1315520653":"2023-07-15T16:05:52.663390+0000","172.21.15.62:6800/1177875314":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/926428906":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/804633476":"2023-07-15T16:06:19.369987+0000","172.21.15.62:0/3081660148":"2023-07-15T16:05:52.663390+0000","172.21.15.62:0/743698528":"2023-07-15T16:06:19.369987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T16:38:16.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0e00a1220 msgr2=0x7fb0e00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0e00a1220 0x7fb0e00a36d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4006fd0 tx=0x7fb0e4008040 comp rx=0 tx=0).stop 2023-07-14T16:38:16.029 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f4072f70 msgr2=0x7fb0f4197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f4072f70 0x7fb0f4197ae0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000d980 tx=0x7fb0f000dd40 comp rx=0 tx=0).stop 2023-07-14T16:38:16.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 shutdown_connections 2023-07-14T16:38:16.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7fb0e00a1220 0x7fb0e00a36d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.030 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f41079c0 0x7fb0f419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0f4072f70 0x7fb0f4197ae0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 --2- 172.21.15.62:0/3822270588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0f4074550 0x7fb0f4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.058+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 >> 172.21.15.62:0/3822270588 conn(0x7fb0f40fb0e0 msgr2=0x7fb0f40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:16.031 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.059+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 shutdown_connections 2023-07-14T16:38:16.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.059+0000 7fb0fb5ff700 1 -- 172.21.15.62:0/3822270588 wait complete. 2023-07-14T16:38:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T16:08:57.974260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T16:13:56.880628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T16:13:58.253604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T16:38:16.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T16:38:16.257 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.286+0000 7f0740fff700 1 -- 172.21.15.62:0/2930447486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c101680 msgr2=0x7f073c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.286+0000 7f0740fff700 1 --2- 172.21.15.62:0/2930447486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c101680 0x7f073c107f40 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f073400b0a0 tx=0x7f073400b3b0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.286+0000 7f0740fff700 1 -- 172.21.15.62:0/2930447486 shutdown_connections 2023-07-14T16:38:16.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.287+0000 7f0740fff700 1 --2- 172.21.15.62:0/2930447486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c101680 0x7f073c107f40 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.258 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.287+0000 7f0740fff700 1 --2- 172.21.15.62:0/2930447486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f073c0ffb00 0x7f073c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.287+0000 7f0740fff700 1 --2- 172.21.15.62:0/2930447486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c100cf0 0x7f073c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.287+0000 7f0740fff700 1 -- 172.21.15.62:0/2930447486 >> 172.21.15.62:0/2930447486 conn(0x7f073c0fb120 msgr2=0x7f073c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.287+0000 7f0740fff700 1 -- 172.21.15.62:0/2930447486 shutdown_connections 2023-07-14T16:38:16.259 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.287+0000 7f0740fff700 1 -- 172.21.15.62:0/2930447486 wait complete. 2023-07-14T16:38:16.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f0740fff700 1 Processor -- start 2023-07-14T16:38:16.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f0740fff700 1 -- start start 2023-07-14T16:38:16.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f0740fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c0ffb00 0x7f073c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:16.261 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f0740fff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f073c100cf0 0x7f073c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f0740fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c101680 0x7f073c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f0740fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c0735e0 con 0x7f073c0ffb00 2023-07-14T16:38:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f0740fff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c073750 con 0x7f073c101680 2023-07-14T16:38:16.262 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f0740fff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c19f960 con 0x7f073c100cf0 2023-07-14T16:38:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f073a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c0ffb00 0x7f073c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f073a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c0ffb00 0x7f073c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42722/0 (socket says 172.21.15.62:42722) 2023-07-14T16:38:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.288+0000 7f073a59c700 1 -- 172.21.15.62:0/3222460898 learned_addr learned my addr 172.21.15.62:0/3222460898 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f0739d9b700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f073c100cf0 0x7f073c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f073ad9d700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c101680 0x7f073c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f073a59c700 1 -- 172.21.15.62:0/3222460898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f073c100cf0 msgr2=0x7f073c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f073a59c700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f073c100cf0 0x7f073c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.264 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f073a59c700 1 -- 172.21.15.62:0/3222460898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c101680 msgr2=0x7f073c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f073a59c700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c101680 0x7f073c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f073a59c700 1 -- 172.21.15.62:0/3222460898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734009d00 con 0x7f073c0ffb00 2023-07-14T16:38:16.265 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f0739d9b700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f073c100cf0 0x7f073c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T16:38:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f073a59c700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c0ffb00 0x7f073c0742c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f072c00d980 tx=0x7f072c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f072b7fe700 1 -- 172.21.15.62:0/3222460898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c004020 con 0x7f073c0ffb00 2023-07-14T16:38:16.266 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f072b7fe700 1 -- 172.21.15.62:0/3222460898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f072c009730 con 0x7f073c0ffb00 2023-07-14T16:38:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f072b7fe700 1 -- 172.21.15.62:0/3222460898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c003a00 con 0x7f073c0ffb00 2023-07-14T16:38:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c19fc40 con 0x7f073c0ffb00 2023-07-14T16:38:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.289+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c1a00d0 con 0x7f073c0ffb00 2023-07-14T16:38:16.267 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.291+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c107090 con 0x7f073c0ffb00 2023-07-14T16:38:16.268 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.292+0000 7f072b7fe700 1 -- 172.21.15.62:0/3222460898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f072c0098a0 con 0x7f073c0ffb00 2023-07-14T16:38:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.294+0000 7f072b7fe700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07240a1150 0x7f07240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.294+0000 7f072b7fe700 1 -- 172.21.15.62:0/3222460898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f072c0cc690 con 0x7f073c0ffb00 2023-07-14T16:38:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.294+0000 7f0739d9b700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07240a1150 0x7f07240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.295+0000 7f0739d9b700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07240a1150 0x7f07240a3600 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f073000bb90 tx=0x7f0730005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:16.269 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.298+0000 7f072b7fe700 1 -- 172.21.15.62:0/3222460898 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f072c095d90 con 0x7f073c0ffb00 2023-07-14T16:38:16.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:16 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/1222569265' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T16:38:16.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:16 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3822270588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:38:16.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:16 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/1222569265' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T16:38:16.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:16 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3822270588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:38:16.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.508+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f073c061230 con 0x7f073c0ffb00 2023-07-14T16:38:16.478 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.508+0000 7f072b7fe700 1 -- 172.21.15.62:0/3222460898 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f072c0993b0 con 0x7f073c0ffb00 2023-07-14T16:38:16.479 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2023-07-14T16:38:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07240a1150 msgr2=0x7f07240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07240a1150 0x7f07240a3600 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f073000bb90 tx=0x7f0730005d80 comp rx=0 tx=0).stop 2023-07-14T16:38:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c0ffb00 msgr2=0x7f073c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c0ffb00 0x7f073c0742c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f072c00d980 tx=0x7f072c00dc90 comp rx=0 tx=0).stop 2023-07-14T16:38:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 shutdown_connections 2023-07-14T16:38:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f07240a1150 0x7f07240a3600 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c101680 0x7f073c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f073c0ffb00 0x7f073c0742c0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 --2- 172.21.15.62:0/3222460898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f073c100cf0 0x7f073c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 >> 172.21.15.62:0/3222460898 conn(0x7f073c0fb120 msgr2=0x7f073c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 shutdown_connections 2023-07-14T16:38:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.511+0000 7f0740fff700 1 -- 172.21.15.62:0/3222460898 wait complete. 2023-07-14T16:38:16.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T16:38:16.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:16 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/1222569265' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T16:38:16.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:16 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3822270588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T16:38:16.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 -- 172.21.15.62:0/3625125237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 msgr2=0x7f8608101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.706 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3625125237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 0x7f8608101120 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f85f80098f0 tx=0x7f85f8009c00 comp rx=0 tx=0).stop 2023-07-14T16:38:16.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 -- 172.21.15.62:0/3625125237 shutdown_connections 2023-07-14T16:38:16.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3625125237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8608101660 0x7f8608107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3625125237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86080ffae0 0x7f86080ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3625125237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 0x7f8608101120 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 -- 172.21.15.62:0/3625125237 >> 172.21.15.62:0/3625125237 conn(0x7f86080fb0e0 msgr2=0x7f86080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:16.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 -- 172.21.15.62:0/3625125237 shutdown_connections 2023-07-14T16:38:16.707 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.736+0000 7f860f9ff700 1 -- 172.21.15.62:0/3625125237 wait complete. 2023-07-14T16:38:16.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.737+0000 7f860f9ff700 1 Processor -- start 2023-07-14T16:38:16.709 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.737+0000 7f860f9ff700 1 -- start start 2023-07-14T16:38:16.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860f9ff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86080ffae0 0x7f86080742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:16.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860f9ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 0x7f8608072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:16.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860f9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8608101660 0x7f8608072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:16.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860f9ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080735e0 con 0x7f8608101660 2023-07-14T16:38:16.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860f9ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608073750 con 0x7f8608100cd0 2023-07-14T16:38:16.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860f9ff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860819f8a0 con 0x7f86080ffae0 2023-07-14T16:38:16.710 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8608101660 0x7f8608072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:16.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860df9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8608101660 0x7f8608072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42742/0 (socket says 172.21.15.62:42742) 2023-07-14T16:38:16.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860df9c700 1 -- 172.21.15.62:0/3031160152 learned_addr learned my addr 172.21.15.62:0/3031160152 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:16.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860cf9a700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 0x7f8608072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:16.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.738+0000 7f860d79b700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86080ffae0 0x7f86080742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:16.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860df9c700 1 -- 172.21.15.62:0/3031160152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86080ffae0 msgr2=0x7f86080742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860df9c700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86080ffae0 0x7f86080742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.711 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860df9c700 1 -- 172.21.15.62:0/3031160152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 msgr2=0x7f8608072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860df9c700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 0x7f8608072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860df9c700 1 -- 172.21.15.62:0/3031160152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8600009d00 con 0x7f8608101660 2023-07-14T16:38:16.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860cf9a700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 0x7f8608072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:38:16.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860df9c700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8608101660 0x7f8608072e40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f860000e3c0 tx=0x7f860000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:16.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f85fe7fc700 1 -- 172.21.15.62:0/3031160152 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86000090d0 con 0x7f8608101660 2023-07-14T16:38:16.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f85fe7fc700 1 -- 172.21.15.62:0/3031160152 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f860000f040 con 0x7f8608101660 2023-07-14T16:38:16.712 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f8009520 con 0x7f8608101660 2023-07-14T16:38:16.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f85fe7fc700 1 -- 172.21.15.62:0/3031160152 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600007b10 con 0x7f8608101660 2023-07-14T16:38:16.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.739+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860819ff50 con 0x7f8608101660 2023-07-14T16:38:16.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.740+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86081071b0 con 0x7f8608101660 2023-07-14T16:38:16.713 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.743+0000 7f85fe7fc700 1 -- 172.21.15.62:0/3031160152 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8600007c70 con 0x7f8608101660 2023-07-14T16:38:16.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.744+0000 7f85fe7fc700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f85f40a11d0 0x7f85f40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:16.714 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.744+0000 7f85fe7fc700 1 -- 172.21.15.62:0/3031160152 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86000cc840 con 0x7f8608101660 2023-07-14T16:38:16.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.744+0000 7f860d79b700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f85f40a11d0 0x7f85f40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:16.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.745+0000 7f860d79b700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f85f40a11d0 0x7f85f40a3680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8608100b30 tx=0x7f8604009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:16.715 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.746+0000 7f85fe7fc700 1 -- 172.21.15.62:0/3031160152 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8600095fd0 con 0x7f8608101660 2023-07-14T16:38:16.922 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.952+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f860802c9e0 con 0x7f8608101660 2023-07-14T16:38:16.923 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.953+0000 7f85fe7fc700 1 -- 172.21.15.62:0/3031160152 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86000995f0 con 0x7f8608101660 2023-07-14T16:38:16.923 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2023-07-14T16:38:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f85f40a11d0 msgr2=0x7f85f40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f85f40a11d0 0x7f85f40a3680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8608100b30 tx=0x7f8604009260 comp rx=0 tx=0).stop 2023-07-14T16:38:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8608101660 msgr2=0x7f8608072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:16.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8608101660 0x7f8608072e40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f860000e3c0 tx=0x7f860000e6d0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 shutdown_connections 2023-07-14T16:38:16.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8608101660 0x7f8608072e40 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86080ffae0 0x7f86080742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f85f40a11d0 0x7f85f40a3680 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.927 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 --2- 172.21.15.62:0/3031160152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608100cd0 0x7f8608072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:16.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.955+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 >> 172.21.15.62:0/3031160152 conn(0x7f86080fb0e0 msgr2=0x7f86081042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:16.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.956+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 shutdown_connections 2023-07-14T16:38:16.928 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:16.956+0000 7f860f9ff700 1 -- 172.21.15.62:0/3031160152 wait complete. 2023-07-14T16:38:16.944 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T16:38:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 -- 172.21.15.62:0/3227917890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c0ffae0 msgr2=0x7f828c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 --2- 172.21.15.62:0/3227917890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c0ffae0 0x7f828c0ffef0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f827c009990 tx=0x7f827c009ca0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 -- 172.21.15.62:0/3227917890 shutdown_connections 2023-07-14T16:38:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 --2- 172.21.15.62:0/3227917890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f828c101660 0x7f828c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 --2- 172.21.15.62:0/3227917890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c0ffae0 0x7f828c0ffef0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 --2- 172.21.15.62:0/3227917890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f828c100cd0 0x7f828c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 -- 172.21.15.62:0/3227917890 >> 172.21.15.62:0/3227917890 conn(0x7f828c0fb0e0 msgr2=0x7f828c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 -- 172.21.15.62:0/3227917890 shutdown_connections 2023-07-14T16:38:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.182+0000 7f8290dff700 1 -- 172.21.15.62:0/3227917890 wait complete. 2023-07-14T16:38:17.154 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.184+0000 7f8290dff700 1 Processor -- start 2023-07-14T16:38:17.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.184+0000 7f8290dff700 1 -- start start 2023-07-14T16:38:17.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.184+0000 7f8290dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f828c0ffae0 0x7f828c195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:17.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.184+0000 7f8290dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c100cd0 0x7f828c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:17.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.184+0000 7f8290dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f828c101660 0x7f828c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:17.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.184+0000 7f8290dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c19d380 con 0x7f828c100cd0 2023-07-14T16:38:17.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.184+0000 7f8290dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c19d4c0 con 0x7f828c101660 2023-07-14T16:38:17.155 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.184+0000 7f8290dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c19d650 con 0x7f828c0ffae0 2023-07-14T16:38:17.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f8289d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c100cd0 0x7f828c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:17.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f828c0ffae0 0x7f828c195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:17.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828a59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f828c0ffae0 0x7f828c195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.62:39434/0 (socket says 172.21.15.62:39434) 2023-07-14T16:38:17.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f8289d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c100cd0 0x7f828c195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42770/0 (socket says 172.21.15.62:42770) 2023-07-14T16:38:17.156 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828a59c700 1 -- 172.21.15.62:0/7139468 learned_addr learned my addr 172.21.15.62:0/7139468 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:17.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828ad9d700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f828c101660 0x7f828c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:17.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828a59c700 1 -- 172.21.15.62:0/7139468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f828c101660 msgr2=0x7f828c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.157 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828a59c700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f828c101660 0x7f828c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828a59c700 1 -- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c100cd0 msgr2=0x7f828c195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828a59c700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c100cd0 0x7f828c195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f828a59c700 1 -- 172.21.15.62:0/7139468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0095f0 con 0x7f828c0ffae0 2023-07-14T16:38:17.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.185+0000 7f8289d9b700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c100cd0 0x7f828c195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T16:38:17.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.186+0000 7f828a59c700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f828c0ffae0 0x7f828c195800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f827c00b5c0 tx=0x7f827c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:17.158 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.186+0000 7f827b7fe700 1 -- 172.21.15.62:0/7139468 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c01d070 con 0x7f828c0ffae0 2023-07-14T16:38:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.186+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c19d8d0 con 0x7f828c0ffae0 2023-07-14T16:38:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.186+0000 7f827b7fe700 1 -- 172.21.15.62:0/7139468 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f827c004000 con 0x7f828c0ffae0 2023-07-14T16:38:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.186+0000 7f827b7fe700 1 -- 172.21.15.62:0/7139468 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c022d00 con 0x7f828c0ffae0 2023-07-14T16:38:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.186+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c19dd50 con 0x7f828c0ffae0 2023-07-14T16:38:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.187+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c1071d0 con 0x7f828c0ffae0 2023-07-14T16:38:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.189+0000 7f827b7fe700 1 -- 172.21.15.62:0/7139468 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f827c003b70 con 0x7f828c0ffae0 2023-07-14T16:38:17.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.190+0000 7f827b7fe700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82740a11d0 0x7f82740a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:17.160 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.190+0000 7f827b7fe700 1 -- 172.21.15.62:0/7139468 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f827c0ce340 con 0x7f828c0ffae0 2023-07-14T16:38:17.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.191+0000 7f8289d9b700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82740a11d0 0x7f82740a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:17.161 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.191+0000 7f8289d9b700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82740a11d0 0x7f82740a3680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8280005950 tx=0x7f828000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:17.162 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.193+0000 7f827b7fe700 1 -- 172.21.15.62:0/7139468 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f827c097ad0 con 0x7f828c0ffae0 2023-07-14T16:38:17.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:17 smithi062 ceph-mon[95543]: pgmap v1036: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:38:17.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:17 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3222460898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:38:17.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:17 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3031160152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:38:17.369 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.399+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f828c02c9e0 con 0x7f828c0ffae0 2023-07-14T16:38:17.370 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.400+0000 7f827b7fe700 1 -- 172.21.15.62:0/7139468 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f827c022e60 con 0x7f828c0ffae0 2023-07-14T16:38:17.370 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2023-07-14T16:38:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.402+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82740a11d0 msgr2=0x7f82740a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.402+0000 7f8290dff700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82740a11d0 0x7f82740a3680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8280005950 tx=0x7f828000b410 comp rx=0 tx=0).stop 2023-07-14T16:38:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.402+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f828c0ffae0 msgr2=0x7f828c195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.402+0000 7f8290dff700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f828c0ffae0 0x7f828c195800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f827c00b5c0 tx=0x7f827c005740 comp rx=0 tx=0).stop 2023-07-14T16:38:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.403+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 shutdown_connections 2023-07-14T16:38:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.403+0000 7f8290dff700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f82740a11d0 0x7f82740a3680 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.403+0000 7f8290dff700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f828c101660 0x7f828c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.403+0000 7f8290dff700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f828c0ffae0 0x7f828c195800 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.403+0000 7f8290dff700 1 --2- 172.21.15.62:0/7139468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c100cd0 0x7f828c195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.403+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 >> 172.21.15.62:0/7139468 conn(0x7f828c0fb0e0 msgr2=0x7f828c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:17.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.403+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 shutdown_connections 2023-07-14T16:38:17.374 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.403+0000 7f8290dff700 1 -- 172.21.15.62:0/7139468 wait complete. 2023-07-14T16:38:17.392 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T16:38:17.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:17 smithi081 ceph-mon[127795]: pgmap v1036: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:38:17.394 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:17 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3222460898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:38:17.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:17 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3031160152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:38:17.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:17 smithi164 ceph-mon[97427]: pgmap v1036: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-14T16:38:17.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:17 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3222460898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T16:38:17.519 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:17 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3031160152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T16:38:17.596 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 -- 172.21.15.62:0/2461109323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 msgr2=0x7f38f00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 --2- 172.21.15.62:0/2461109323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 0x7f38f00ffef0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009990 tx=0x7f38e0009ca0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 -- 172.21.15.62:0/2461109323 shutdown_connections 2023-07-14T16:38:17.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 --2- 172.21.15.62:0/2461109323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0101660 0x7f38f0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 --2- 172.21.15.62:0/2461109323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 0x7f38f00ffef0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 --2- 172.21.15.62:0/2461109323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f0100cd0 0x7f38f0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.597 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 -- 172.21.15.62:0/2461109323 >> 172.21.15.62:0/2461109323 conn(0x7f38f00fb0e0 msgr2=0x7f38f00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:17.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 -- 172.21.15.62:0/2461109323 shutdown_connections 2023-07-14T16:38:17.598 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.626+0000 7f38f4dff700 1 -- 172.21.15.62:0/2461109323 wait complete. 2023-07-14T16:38:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.627+0000 7f38f4dff700 1 Processor -- start 2023-07-14T16:38:17.599 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38f4dff700 1 -- start start 2023-07-14T16:38:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38f4dff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 0x7f38f00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38f4dff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100cd0 0x7f38f0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38f4dff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f0101660 0x7f38f0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38f4dff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00735e0 con 0x7f38f00ffae0 2023-07-14T16:38:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38f4dff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0073750 con 0x7f38f0101660 2023-07-14T16:38:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38f4dff700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f019f7d0 con 0x7f38f0100cd0 2023-07-14T16:38:17.600 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 0x7f38f00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38ee59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 0x7f38f00742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42784/0 (socket says 172.21.15.62:42784) 2023-07-14T16:38:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38ee59c700 1 -- 172.21.15.62:0/3116867349 learned_addr learned my addr 172.21.15.62:0/3116867349 (peer_addr_for_me v2:172.21.15.62:0/0) 2023-07-14T16:38:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38eed9d700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f0101660 0x7f38f0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.628+0000 7f38edd9b700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100cd0 0x7f38f0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38ee59c700 1 -- 172.21.15.62:0/3116867349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100cd0 msgr2=0x7f38f0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38ee59c700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100cd0 0x7f38f0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38ee59c700 1 -- 172.21.15.62:0/3116867349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f0101660 msgr2=0x7f38f0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38ee59c700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f0101660 0x7f38f0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38ee59c700 1 -- 172.21.15.62:0/3116867349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00095f0 con 0x7f38f00ffae0 2023-07-14T16:38:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38ee59c700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 0x7f38f00742c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b5c0 tx=0x7f38e0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38df7fe700 1 -- 172.21.15.62:0/3116867349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e001d070 con 0x7f38f00ffae0 2023-07-14T16:38:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f019fa50 con 0x7f38f00ffae0 2023-07-14T16:38:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38df7fe700 1 -- 172.21.15.62:0/3116867349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38e0004000 con 0x7f38f00ffae0 2023-07-14T16:38:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38df7fe700 1 -- 172.21.15.62:0/3116867349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0022d00 con 0x7f38f00ffae0 2023-07-14T16:38:17.602 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.629+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f019fe70 con 0x7f38f00ffae0 2023-07-14T16:38:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.631+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f010b160 con 0x7f38f00ffae0 2023-07-14T16:38:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.633+0000 7f38df7fe700 1 -- 172.21.15.62:0/3116867349 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f38e0003b70 con 0x7f38f00ffae0 2023-07-14T16:38:17.603 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.634+0000 7f38df7fe700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T16:38:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.634+0000 7f38df7fe700 1 -- 172.21.15.62:0/3116867349 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f38e00cdca0 con 0x7f38f00ffae0 2023-07-14T16:38:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.634+0000 7f38edd9b700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T16:38:17.604 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.635+0000 7f38edd9b700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f38e4005950 tx=0x7f38e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T16:38:17.608 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.638+0000 7f38df7fe700 1 -- 172.21.15.62:0/3116867349 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f38e0097490 con 0x7f38f00ffae0 2023-07-14T16:38:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.862+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38f0106a60 con 0x7f38f00ffae0 2023-07-14T16:38:17.832 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.862+0000 7f38df7fe700 1 -- 172.21.15.62:0/3116867349 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+587 (secure 0 0 0) 0x7f38e0027260 con 0x7f38f00ffae0 2023-07-14T16:38:17.833 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/105803689 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:17.833 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1873685153 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:17.833 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/1177875314 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:17.833 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/105803689 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:17.833 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1315520653 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:17.833 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/1177875314 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:17.833 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/926428906 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:17.834 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/804633476 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:17.834 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3081660148 2023-07-15T16:05:52.663390+0000 2023-07-14T16:38:17.834 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/743698528 2023-07-15T16:06:19.369987+0000 2023-07-14T16:38:17.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 msgr2=0x7f38d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f38e4005950 tx=0x7f38e40058e0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 msgr2=0x7f38f00742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T16:38:17.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 0x7f38f00742c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b5c0 tx=0x7f38e0005740 comp rx=0 tx=0).stop 2023-07-14T16:38:17.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 shutdown_connections 2023-07-14T16:38:17.835 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:6800/4048727837,v1:172.21.15.62:6801/4048727837] conn(0x7f38d80a1150 0x7f38d80a3600 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38f0101660 0x7f38f0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38f00ffae0 0x7f38f00742c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 --2- 172.21.15.62:0/3116867349 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38f0100cd0 0x7f38f0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T16:38:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 >> 172.21.15.62:0/3116867349 conn(0x7f38f00fb0e0 msgr2=0x7f38f00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T16:38:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 shutdown_connections 2023-07-14T16:38:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:2023-07-14T16:38:17.865+0000 7f38f4dff700 1 -- 172.21.15.62:0/3116867349 wait complete. 2023-07-14T16:38:17.837 INFO:teuthology.orchestra.run.smithi062.stderr:listed 10 entries 2023-07-14T16:38:17.853 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi062.front.sepia.ceph.com... 2023-07-14T16:38:17.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:38:17.854 DEBUG:teuthology.orchestra.run.smithi062:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-14T16:38:17.940 INFO:teuthology.orchestra.run:waiting for 300 2023-07-14T16:38:17.944 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi062.stderr:ceph-fuse[143835]: fuse finished with error 0 and tester_r 0 2023-07-14T16:38:18.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:18 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/7139468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:38:18.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:18 smithi062 ceph-mon[95543]: from='client.? 172.21.15.62:0/3116867349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T16:38:18.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:18 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/7139468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:38:18.397 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:18 smithi081 ceph-mon[127795]: from='client.? 172.21.15.62:0/3116867349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T16:38:18.523 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:18 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/7139468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T16:38:18.524 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:18 smithi164 ceph-mon[97427]: from='client.? 172.21.15.62:0/3116867349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T16:38:19.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:19 smithi062 ceph-mon[95543]: pgmap v1037: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:38:19.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:19 smithi081 ceph-mon[127795]: pgmap v1037: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:38:19.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:19 smithi164 ceph-mon[97427]: pgmap v1037: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T16:38:21.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:21 smithi062 ceph-mon[95543]: pgmap v1038: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2023-07-14T16:38:21.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:21 smithi081 ceph-mon[127795]: pgmap v1038: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2023-07-14T16:38:21.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:21 smithi164 ceph-mon[97427]: pgmap v1038: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2023-07-14T16:38:23.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:23 smithi062 ceph-mon[95543]: pgmap v1039: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-14T16:38:23.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:23 smithi081 ceph-mon[127795]: pgmap v1039: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-14T16:38:23.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:23 smithi164 ceph-mon[97427]: pgmap v1039: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-14T16:38:24.042 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2023-07-14T16:38:24.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:38:24.044 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T16:38:24.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:38:24.112 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2023-07-14T16:38:24.161 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T16:38:24.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:38:24.162 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T16:38:24.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T16:38:24.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2023-07-14T16:38:24.393 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-14T16:38:24.403 INFO:tasks.cephadm:Teardown begin 2023-07-14T16:38:24.404 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/cephadm.py", line 1677, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-14T16:38:24.404 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:38:24.478 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:38:24.554 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:38:24.637 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-14T16:38:24.638 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T16:38:24.683 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T16:38:24.728 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T16:38:24.774 INFO:tasks.cephadm:Stopping all daemons... 2023-07-14T16:38:24.775 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-14T16:38:24.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.a 2023-07-14T16:38:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:24 smithi062 systemd[1]: Stopping Ceph mon.a for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:25.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:25 smithi062 ceph-mon[95543]: pgmap v1040: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-14T16:38:25.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:25 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-a[95519]: 2023-07-14T16:38:25.328+0000 7f9c411e8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:38:25.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:25 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-a[95519]: 2023-07-14T16:38:25.328+0000 7f9c411e8700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-14T16:38:25.395 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:25 smithi081 ceph-mon[127795]: pgmap v1040: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-14T16:38:25.518 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:25 smithi164 ceph-mon[97427]: pgmap v1040: 97 pgs: 97 active+clean; 9.1 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2023-07-14T16:38:25.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jul 14 16:38:25 smithi062 bash[208433]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-a 2023-07-14T16:38:26.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.a.service' 2023-07-14T16:38:26.264 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:26.265 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-14T16:38:26.265 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-14T16:38:26.266 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.b 2023-07-14T16:38:26.645 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:26 smithi081 systemd[1]: Stopping Ceph mon.b for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:27.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:26 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-b[127771]: 2023-07-14T16:38:26.835+0000 7f05efc3f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:38:27.145 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:26 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-b[127771]: 2023-07-14T16:38:26.835+0000 7f05efc3f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-14T16:38:27.645 INFO:journalctl@ceph.mon.b.smithi081.stdout:Jul 14 16:38:27 smithi081 bash[160470]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-b 2023-07-14T16:38:27.755 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.b.service' 2023-07-14T16:38:27.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:27.867 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-14T16:38:27.868 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-14T16:38:27.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.c 2023-07-14T16:38:28.269 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:27 smithi164 systemd[1]: Stopping Ceph mon.c for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:28.768 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:28 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-c[97403]: 2023-07-14T16:38:28.501+0000 7ff4ed260700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:38:28.769 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:28 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-c[97403]: 2023-07-14T16:38:28.501+0000 7ff4ed260700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-14T16:38:29.104 INFO:journalctl@ceph.mon.c.smithi164.stdout:Jul 14 16:38:28 smithi164 bash[131760]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mon-c 2023-07-14T16:38:29.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mon.c.service' 2023-07-14T16:38:29.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:29.249 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-14T16:38:29.250 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-14T16:38:29.250 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.x 2023-07-14T16:38:29.617 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:38:29 smithi062 systemd[1]: Stopping Ceph mgr.x for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:30.048 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:38:29 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:38:29.803+0000 7fdbb6eb6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:38:30.048 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:38:29 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x[95790]: 2023-07-14T16:38:29.803+0000 7fdbb6eb6700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-14T16:38:30.302 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jul 14 16:38:30 smithi062 bash[208590]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-x 2023-07-14T16:38:30.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.x.service' 2023-07-14T16:38:30.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:30.694 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-14T16:38:30.694 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-14T16:38:30.695 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.y 2023-07-14T16:38:31.145 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:38:30 smithi081 systemd[1]: Stopping Ceph mgr.y for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:31.887 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:38:31 smithi081 bash[160627]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-y 2023-07-14T16:38:32.085 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.y.service' 2023-07-14T16:38:32.144 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:38:31 smithi081 systemd[1]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-14T16:38:32.145 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:38:32 smithi081 systemd[1]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.y.service: Failed with result 'exit-code'. 2023-07-14T16:38:32.145 INFO:journalctl@ceph.mgr.y.smithi081.stdout:Jul 14 16:38:32 smithi081 systemd[1]: Stopped Ceph mgr.y for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:38:32.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:32.171 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-14T16:38:32.172 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-14T16:38:32.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.z 2023-07-14T16:38:32.518 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:38:32 smithi164 systemd[1]: Stopping Ceph mgr.z for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:33.408 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:38:33 smithi164 bash[131910]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mgr-z 2023-07-14T16:38:33.667 INFO:journalctl@ceph.mgr.z.smithi164.stdout:Jul 14 16:38:33 smithi164 systemd[1]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-14T16:38:33.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mgr.z.service' 2023-07-14T16:38:33.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:33.766 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-14T16:38:33.766 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-14T16:38:33.767 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.0 2023-07-14T16:38:34.118 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:38:33 smithi062 systemd[1]: Stopping Ceph osd.0 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:34.539 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:38:34 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-0[108585]: 2023-07-14T16:38:34.318+0000 7efde3759700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:38:34.540 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:38:34 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-0[108585]: 2023-07-14T16:38:34.318+0000 7efde3759700 -1 osd.0 88 *** Got signal Terminated *** 2023-07-14T16:38:34.540 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:38:34 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-0[108585]: 2023-07-14T16:38:34.318+0000 7efde3759700 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:38:40.617 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jul 14 16:38:40 smithi062 bash[208746]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-0 2023-07-14T16:38:42.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.0.service' 2023-07-14T16:38:42.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:42.879 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-14T16:38:42.879 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-14T16:38:42.879 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.1 2023-07-14T16:38:43.394 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:38:43 smithi081 systemd[1]: Stopping Ceph osd.1 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:43.894 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:38:43 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-1[132879]: 2023-07-14T16:38:43.434+0000 7f61f6d3e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:38:43.895 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:38:43 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-1[132879]: 2023-07-14T16:38:43.434+0000 7f61f6d3e700 -1 osd.1 88 *** Got signal Terminated *** 2023-07-14T16:38:43.895 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:38:43 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-1[132879]: 2023-07-14T16:38:43.434+0000 7f61f6d3e700 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:38:49.645 INFO:journalctl@ceph.osd.1.smithi081.stdout:Jul 14 16:38:49 smithi081 bash[160782]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-1 2023-07-14T16:38:51.197 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.1.service' 2023-07-14T16:38:51.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:51.288 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-14T16:38:51.288 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-14T16:38:51.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.2 2023-07-14T16:38:51.768 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:38:51 smithi164 systemd[1]: Stopping Ceph osd.2 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:38:52.268 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:38:51 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-2[103230]: 2023-07-14T16:38:51.840+0000 7ff7d8e49700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:38:52.268 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:38:51 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-2[103230]: 2023-07-14T16:38:51.840+0000 7ff7d8e49700 -1 osd.2 88 *** Got signal Terminated *** 2023-07-14T16:38:52.268 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:38:51 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-2[103230]: 2023-07-14T16:38:51.840+0000 7ff7d8e49700 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:38:58.268 INFO:journalctl@ceph.osd.2.smithi164.stdout:Jul 14 16:38:58 smithi164 bash[132069]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-2 2023-07-14T16:38:59.830 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.2.service' 2023-07-14T16:38:59.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:38:59.919 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-14T16:38:59.920 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-14T16:38:59.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.3 2023-07-14T16:39:00.367 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jul 14 16:39:00 smithi062 systemd[1]: Stopping Ceph osd.3 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:39:00.867 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jul 14 16:39:00 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-3[114098]: 2023-07-14T16:39:00.472+0000 7f9e5f6ba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:39:00.868 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jul 14 16:39:00 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-3[114098]: 2023-07-14T16:39:00.472+0000 7f9e5f6ba700 -1 osd.3 88 *** Got signal Terminated *** 2023-07-14T16:39:00.868 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jul 14 16:39:00 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-3[114098]: 2023-07-14T16:39:00.472+0000 7f9e5f6ba700 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:39:06.617 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jul 14 16:39:06 smithi062 bash[209030]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-3 2023-07-14T16:39:08.375 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.3.service' 2023-07-14T16:39:08.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:39:08.466 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-14T16:39:08.467 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-14T16:39:08.467 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.4 2023-07-14T16:39:08.894 INFO:journalctl@ceph.osd.4.smithi081.stdout:Jul 14 16:39:08 smithi081 systemd[1]: Stopping Ceph osd.4 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:39:09.395 INFO:journalctl@ceph.osd.4.smithi081.stdout:Jul 14 16:39:09 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-4[138437]: 2023-07-14T16:39:09.021+0000 7fb95f90e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:39:09.395 INFO:journalctl@ceph.osd.4.smithi081.stdout:Jul 14 16:39:09 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-4[138437]: 2023-07-14T16:39:09.021+0000 7fb95f90e700 -1 osd.4 88 *** Got signal Terminated *** 2023-07-14T16:39:09.395 INFO:journalctl@ceph.osd.4.smithi081.stdout:Jul 14 16:39:09 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-4[138437]: 2023-07-14T16:39:09.021+0000 7fb95f90e700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:39:15.394 INFO:journalctl@ceph.osd.4.smithi081.stdout:Jul 14 16:39:15 smithi081 bash[161055]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-4 2023-07-14T16:39:16.938 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.4.service' 2023-07-14T16:39:17.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:39:17.028 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-14T16:39:17.029 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-14T16:39:17.029 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.5 2023-07-14T16:39:17.518 INFO:journalctl@ceph.osd.5.smithi164.stdout:Jul 14 16:39:17 smithi164 systemd[1]: Stopping Ceph osd.5 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:39:18.018 INFO:journalctl@ceph.osd.5.smithi164.stdout:Jul 14 16:39:17 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-5[108925]: 2023-07-14T16:39:17.567+0000 7f5393dbb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:39:18.018 INFO:journalctl@ceph.osd.5.smithi164.stdout:Jul 14 16:39:17 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-5[108925]: 2023-07-14T16:39:17.567+0000 7f5393dbb700 -1 osd.5 88 *** Got signal Terminated *** 2023-07-14T16:39:18.018 INFO:journalctl@ceph.osd.5.smithi164.stdout:Jul 14 16:39:17 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-5[108925]: 2023-07-14T16:39:17.567+0000 7f5393dbb700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:39:24.018 INFO:journalctl@ceph.osd.5.smithi164.stdout:Jul 14 16:39:23 smithi164 bash[132332]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-5 2023-07-14T16:39:25.467 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.5.service' 2023-07-14T16:39:25.557 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:39:25.558 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-14T16:39:25.558 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-14T16:39:25.559 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.6 2023-07-14T16:39:26.074 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jul 14 16:39:25 smithi062 systemd[1]: Stopping Ceph osd.6 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:39:26.367 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jul 14 16:39:26 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-6[119574]: 2023-07-14T16:39:26.103+0000 7ffa43f3b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:39:26.367 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jul 14 16:39:26 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-6[119574]: 2023-07-14T16:39:26.103+0000 7ffa43f3b700 -1 osd.6 88 *** Got signal Terminated *** 2023-07-14T16:39:26.367 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jul 14 16:39:26 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-6[119574]: 2023-07-14T16:39:26.103+0000 7ffa43f3b700 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:39:32.117 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jul 14 16:39:31 smithi062 bash[209305]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-6 2023-07-14T16:39:33.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.6.service' 2023-07-14T16:39:33.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:39:33.998 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-14T16:39:33.999 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-14T16:39:33.999 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.7 2023-07-14T16:39:34.395 INFO:journalctl@ceph.osd.7.smithi081.stdout:Jul 14 16:39:34 smithi081 systemd[1]: Stopping Ceph osd.7 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:39:34.894 INFO:journalctl@ceph.osd.7.smithi081.stdout:Jul 14 16:39:34 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-7[143993]: 2023-07-14T16:39:34.566+0000 7fbcfa692700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:39:34.895 INFO:journalctl@ceph.osd.7.smithi081.stdout:Jul 14 16:39:34 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-7[143993]: 2023-07-14T16:39:34.567+0000 7fbcfa692700 -1 osd.7 88 *** Got signal Terminated *** 2023-07-14T16:39:34.895 INFO:journalctl@ceph.osd.7.smithi081.stdout:Jul 14 16:39:34 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-7[143993]: 2023-07-14T16:39:34.567+0000 7fbcfa692700 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:39:40.894 INFO:journalctl@ceph.osd.7.smithi081.stdout:Jul 14 16:39:40 smithi081 bash[161336]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-7 2023-07-14T16:39:41.268 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:40 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:40.829+0000 7f88c9052700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:38.366355+0000 front 2023-07-14T16:38:38.366285+0000 (oldest deadline 2023-07-14T16:39:40.666274+0000) 2023-07-14T16:39:41.828 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:41 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:41.563+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:42.268 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:41 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:41.858+0000 7f88c9052700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:38.366355+0000 front 2023-07-14T16:38:38.366285+0000 (oldest deadline 2023-07-14T16:39:40.666274+0000) 2023-07-14T16:39:42.394 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:42 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:42.136+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:42.524 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.7.service' 2023-07-14T16:39:42.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:39:42.615 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-14T16:39:42.615 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-14T16:39:42.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.8 2023-07-14T16:39:42.767 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:42 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:42.535+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:42.768 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:42 smithi164 systemd[1]: Stopping Ceph osd.8 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:39:43.125 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:42 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:42.813+0000 7f88c9052700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:38.366355+0000 front 2023-07-14T16:38:38.366285+0000 (oldest deadline 2023-07-14T16:39:40.666274+0000) 2023-07-14T16:39:43.504 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:43 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:43.154+0000 7f88cca5c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:39:43.504 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:43 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:43.154+0000 7f88cca5c700 -1 osd.8 88 *** Got signal Terminated *** 2023-07-14T16:39:43.504 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:43 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:43.155+0000 7f88cca5c700 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:39:43.616 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:43 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:43.396+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:43.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:43 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:43.178+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:43.768 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:43 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:43.765+0000 7f88c9052700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:38.366355+0000 front 2023-07-14T16:38:38.366285+0000 (oldest deadline 2023-07-14T16:39:40.666274+0000) 2023-07-14T16:39:43.768 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:43 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:43.534+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:44.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:44 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:44.218+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:44.767 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:44 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:44.722+0000 7f88c9052700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:38.366355+0000 front 2023-07-14T16:38:38.366285+0000 (oldest deadline 2023-07-14T16:39:40.666274+0000) 2023-07-14T16:39:44.768 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:44 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:44.530+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:44.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:44 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:44.432+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:45.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:45 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:45.222+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:45.768 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:45 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:45.738+0000 7f88c9052700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:38.366355+0000 front 2023-07-14T16:38:38.366285+0000 (oldest deadline 2023-07-14T16:39:40.666274+0000) 2023-07-14T16:39:45.768 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:45 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:45.533+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:45.866 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:45 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:45.445+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:46.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:46 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:46.272+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:46.768 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:46 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:46.689+0000 7f88c9052700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:38.366355+0000 front 2023-07-14T16:38:38.366285+0000 (oldest deadline 2023-07-14T16:39:40.666274+0000) 2023-07-14T16:39:46.768 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:46 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:46.506+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:46.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:46 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:46.482+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:47.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:47 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:47.307+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:47.768 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:47 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8[114391]: 2023-07-14T16:39:47.673+0000 7f88c9052700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:38.366355+0000 front 2023-07-14T16:38:38.366285+0000 (oldest deadline 2023-07-14T16:39:40.666274+0000) 2023-07-14T16:39:47.768 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:47 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:47.515+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:47.866 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:47 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:47.443+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:48.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:48 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:48.339+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:48.745 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:48 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:48.501+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:48.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:48 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:48.466+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:49.510 INFO:journalctl@ceph.osd.8.smithi164.stdout:Jul 14 16:39:49 smithi164 bash[132601]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-8 2023-07-14T16:39:49.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:49 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:49.330+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:49.767 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:49 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:49.539+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:49.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:49 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:49.421+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:50.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:50 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:50.376+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:50.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:50 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:50.376+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:50.786 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:50 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:50.564+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:50.787 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:50 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:50.564+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:50.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:50 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:50.444+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:50.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:50 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:50.444+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:46.205735+0000 front 2023-07-14T16:38:46.205595+0000 (oldest deadline 2023-07-14T16:39:50.305882+0000) 2023-07-14T16:39:51.059 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.8.service' 2023-07-14T16:39:51.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:39:51.149 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-14T16:39:51.150 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-14T16:39:51.150 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.9 2023-07-14T16:39:51.617 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:51 smithi062 systemd[1]: Stopping Ceph osd.9 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:39:51.617 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:51 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:51.427+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:51.617 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:51 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:51.427+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:46.205735+0000 front 2023-07-14T16:38:46.205595+0000 (oldest deadline 2023-07-14T16:39:50.305882+0000) 2023-07-14T16:39:51.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:51 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:51.341+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:51.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:51 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:51.341+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:52.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:51 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:51.552+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:52.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:51 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:51.552+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:52.116 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:51 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:51.699+0000 7f61999a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:39:52.117 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:51 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:51.699+0000 7f61999a3700 -1 osd.9 88 *** Got signal Terminated *** 2023-07-14T16:39:52.117 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:51 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:51.699+0000 7f61999a3700 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:39:52.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:52 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:52.316+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:52.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:52 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:52.316+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:52.767 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:52 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:52.545+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:52.768 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:52 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:52.545+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:52.866 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:52 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:52.462+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:52.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:52 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:52.462+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:46.205735+0000 front 2023-07-14T16:38:46.205595+0000 (oldest deadline 2023-07-14T16:39:50.305882+0000) 2023-07-14T16:39:53.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:53 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:53.358+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:53.645 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:53 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:53.358+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:53.866 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:53 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:53.502+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:53.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:53 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:53.502+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:46.205735+0000 front 2023-07-14T16:38:46.205595+0000 (oldest deadline 2023-07-14T16:39:50.305882+0000) 2023-07-14T16:39:54.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:53 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:53.579+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:54.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:53 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:53.579+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:54.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:54 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:54.392+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:54.645 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:54 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:54.392+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:54.866 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:54 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:54.551+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:54.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:54 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:54.551+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:46.205735+0000 front 2023-07-14T16:38:46.205595+0000 (oldest deadline 2023-07-14T16:39:50.305882+0000) 2023-07-14T16:39:55.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:54 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:54.595+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:55.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:54 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:54.595+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:55.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:55 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:55.392+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:55.645 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:55 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:55.392+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:55.866 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:55 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:55.531+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:55.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:55 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:55.531+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:46.205735+0000 front 2023-07-14T16:38:46.205595+0000 (oldest deadline 2023-07-14T16:39:50.305882+0000) 2023-07-14T16:39:56.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:55 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:55.612+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:56.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:55 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:55.612+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:56.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:56 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:56.350+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:56.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:56 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:56.350+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:56.866 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:56 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:56.498+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.903297+0000 front 2023-07-14T16:38:36.903344+0000 (oldest deadline 2023-07-14T16:39:42.803492+0000) 2023-07-14T16:39:56.867 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:56 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9[125128]: 2023-07-14T16:39:56.498+0000 7f6193f95700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:46.205735+0000 front 2023-07-14T16:38:46.205595+0000 (oldest deadline 2023-07-14T16:39:50.305882+0000) 2023-07-14T16:39:57.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:56 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:56.651+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:57.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:56 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:56.651+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:57.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:57 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:57.380+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:57.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:57 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:57.380+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:57.866 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jul 14 16:39:57 smithi062 bash[209585]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-9 2023-07-14T16:39:58.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:57 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:57.650+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:58.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:57 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:57.650+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:58.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:58 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:58.365+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:58.645 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:58 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:58.365+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:39:59.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:58 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:58.669+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:39:59.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:58 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:58.669+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:39:59.497 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.9.service' 2023-07-14T16:39:59.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:39:59.605 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-14T16:39:59.606 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-14T16:39:59.606 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.10 2023-07-14T16:39:59.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:59 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:59.383+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:39:59.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:59 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:39:59.383+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:40:00.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:59 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:59.641+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:00.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:39:59 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:39:59.641+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:00.037 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:39:59 smithi081 systemd[1]: Stopping Ceph osd.10 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:00.377 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:00 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:00.163+0000 7f32b4372700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:00.377 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:00 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:00.163+0000 7f32b4372700 -1 osd.10 88 *** Got signal Terminated *** 2023-07-14T16:40:00.377 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:00 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:00.163+0000 7f32b4372700 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:40:00.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:00 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:00.408+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:40:00.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:00 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:00.408+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:40:01.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:00 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:00.629+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:01.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:00 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:00.629+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:01.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:01 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:01.365+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:40:01.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:01 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:01.365+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:40:02.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:01 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:01.617+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:02.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:01 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:01.617+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:02.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:01 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:01.617+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:02.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:02 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:02.355+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:40:02.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:02 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:02.355+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:40:02.645 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:02 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:02.355+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:56.396153+0000 front 2023-07-14T16:38:56.396023+0000 (oldest deadline 2023-07-14T16:40:02.295854+0000) 2023-07-14T16:40:03.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:02 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:02.607+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:03.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:02 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:02.607+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:03.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:02 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:02.607+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:03.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:03 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:03.358+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:40:03.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:03 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:03.358+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:40:03.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:03 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:03.358+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:56.396153+0000 front 2023-07-14T16:38:56.396023+0000 (oldest deadline 2023-07-14T16:40:02.295854+0000) 2023-07-14T16:40:04.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:03 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:03.613+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:04.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:03 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:03.613+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:04.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:03 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:03.613+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:04.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:04 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:04.320+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.190854+0000 front 2023-07-14T16:38:36.190780+0000 (oldest deadline 2023-07-14T16:39:41.490816+0000) 2023-07-14T16:40:04.644 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:04 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:04.320+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.492713+0000 front 2023-07-14T16:38:45.492677+0000 (oldest deadline 2023-07-14T16:39:50.192903+0000) 2023-07-14T16:40:04.645 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:04 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10[149440]: 2023-07-14T16:40:04.320+0000 7f32b0968700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:56.396153+0000 front 2023-07-14T16:38:56.396023+0000 (oldest deadline 2023-07-14T16:40:02.295854+0000) 2023-07-14T16:40:05.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:04 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:04.634+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:05.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:04 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:04.634+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:05.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:04 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:04.634+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:06.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:05 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:05.595+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:06.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:05 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:05.595+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:06.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:05 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:05.595+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:06.394 INFO:journalctl@ceph.osd.10.smithi081.stdout:Jul 14 16:40:06 smithi081 bash[161608]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-10 2023-07-14T16:40:07.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:06 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:06.623+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:07.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:06 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:06.623+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:07.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:06 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:06.623+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:07.955 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.10.service' 2023-07-14T16:40:08.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:07 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:07.662+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:08.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:07 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:07.662+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:08.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:07 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:07.662+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:08.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:07 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:07.662+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2023-07-14T16:39:04.452139+0000 front 2023-07-14T16:39:04.452162+0000 (oldest deadline 2023-07-14T16:40:07.352378+0000) 2023-07-14T16:40:08.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:08.084 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-14T16:40:08.084 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-14T16:40:08.085 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.11 2023-07-14T16:40:08.517 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:08 smithi164 systemd[1]: Stopping Ceph osd.11 for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:09.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:08 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:08.617+0000 7ff7e174a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:09.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:08 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:08.617+0000 7ff7e174a700 -1 osd.11 88 *** Got signal Terminated *** 2023-07-14T16:40:09.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:08 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:08.617+0000 7ff7e174a700 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T16:40:09.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:08 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:08.684+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:09.019 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:08 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:08.684+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:09.019 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:08 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:08.684+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:09.019 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:08 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:08.684+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2023-07-14T16:39:04.452139+0000 front 2023-07-14T16:39:04.452162+0000 (oldest deadline 2023-07-14T16:40:07.352378+0000) 2023-07-14T16:40:10.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:09 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:09.686+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:10.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:09 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:09.686+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:10.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:09 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:09.686+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:10.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:09 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:09.686+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2023-07-14T16:39:04.452139+0000 front 2023-07-14T16:39:04.452162+0000 (oldest deadline 2023-07-14T16:40:07.352378+0000) 2023-07-14T16:40:11.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:10 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:10.656+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:11.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:10 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:10.656+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:11.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:10 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:10.656+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:11.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:10 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:10.656+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2023-07-14T16:39:04.452139+0000 front 2023-07-14T16:39:04.452162+0000 (oldest deadline 2023-07-14T16:40:07.352378+0000) 2023-07-14T16:40:12.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:11 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:11.660+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:12.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:11 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:11.660+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:12.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:11 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:11.660+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:12.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:11 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:11.660+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2023-07-14T16:39:04.452139+0000 front 2023-07-14T16:39:04.452162+0000 (oldest deadline 2023-07-14T16:40:07.352378+0000) 2023-07-14T16:40:13.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:12 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:12.685+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2023-07-14T16:38:36.644927+0000 front 2023-07-14T16:38:36.644536+0000 (oldest deadline 2023-07-14T16:39:41.344694+0000) 2023-07-14T16:40:13.017 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:12 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:12.685+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.81:6804 osd.1 since back 2023-07-14T16:38:45.947175+0000 front 2023-07-14T16:38:45.947017+0000 (oldest deadline 2023-07-14T16:39:49.947836+0000) 2023-07-14T16:40:13.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:12 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:12.685+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.164:6804 osd.2 since back 2023-07-14T16:38:54.549731+0000 front 2023-07-14T16:38:54.549772+0000 (oldest deadline 2023-07-14T16:40:01.550749+0000) 2023-07-14T16:40:13.018 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:12 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11[119838]: 2023-07-14T16:40:12.685+0000 7ff7ddd40700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2023-07-14T16:39:04.452139+0000 front 2023-07-14T16:39:04.452162+0000 (oldest deadline 2023-07-14T16:40:07.352378+0000) 2023-07-14T16:40:14.767 INFO:journalctl@ceph.osd.11.smithi164.stdout:Jul 14 16:40:14 smithi164 bash[132864]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-osd-11 2023-07-14T16:40:16.264 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@osd.11.service' 2023-07-14T16:40:16.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:16.354 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-14T16:40:16.355 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-14T16:40:16.355 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.a 2023-07-14T16:40:16.867 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:40:16 smithi062 systemd[1]: Stopping Ceph mds.a for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:17.366 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:40:16 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-a[130708]: 2023-07-14T16:40:16.904+0000 7f1b83076700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:17.367 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:40:16 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-a[130708]: 2023-07-14T16:40:16.904+0000 7f1b83076700 -1 mds.a *** got signal Terminated *** 2023-07-14T16:40:21.616 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jul 14 16:40:21 smithi062 bash[209865]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-a 2023-07-14T16:40:21.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.a.service' 2023-07-14T16:40:21.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:21.743 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-14T16:40:21.743 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-14T16:40:21.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.d 2023-07-14T16:40:22.117 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:40:21 smithi062 systemd[1]: Stopping Ceph mds.d for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:22.616 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:40:22 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-d[129608]: 2023-07-14T16:40:22.270+0000 7f00004e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:22.616 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:40:22 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-d[129608]: 2023-07-14T16:40:22.270+0000 7f00004e2700 -1 mds.d *** got signal Terminated *** 2023-07-14T16:40:23.866 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jul 14 16:40:23 smithi062 bash[210004]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-d 2023-07-14T16:40:23.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.d.service' 2023-07-14T16:40:24.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:24.070 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-14T16:40:24.071 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-14T16:40:24.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.g 2023-07-14T16:40:24.366 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:40:24 smithi062 systemd[1]: Stopping Ceph mds.g for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:24.866 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:40:24 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-g[130302]: 2023-07-14T16:40:24.595+0000 7f13eeb2a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:24.867 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:40:24 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-g[130302]: 2023-07-14T16:40:24.595+0000 7f13eeb2a700 -1 mds.g *** got signal Terminated *** 2023-07-14T16:40:26.945 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jul 14 16:40:26 smithi062 bash[210148]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-g 2023-07-14T16:40:27.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.g.service' 2023-07-14T16:40:27.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:27.332 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-14T16:40:27.332 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-14T16:40:27.333 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.j 2023-07-14T16:40:27.616 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:40:27 smithi062 systemd[1]: Stopping Ceph mds.j for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:28.116 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:40:27 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-j[129221]: 2023-07-14T16:40:27.869+0000 7f6778843700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:28.116 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:40:27 smithi062 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-j[129221]: 2023-07-14T16:40:27.869+0000 7f6778843700 -1 mds.j *** got signal Terminated *** 2023-07-14T16:40:32.366 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jul 14 16:40:32 smithi062 bash[210288]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-j 2023-07-14T16:40:32.594 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.j.service' 2023-07-14T16:40:32.677 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:32.678 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-14T16:40:32.678 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-14T16:40:32.679 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.b 2023-07-14T16:40:33.144 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:40:32 smithi081 systemd[1]: Stopping Ceph mds.b for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:33.644 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:40:33 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:40:33.243+0000 7fdd07f8d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:33.644 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:40:33 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b[153226]: 2023-07-14T16:40:33.243+0000 7fdd07f8d700 -1 mds.b *** got signal Terminated *** 2023-07-14T16:40:41.394 INFO:journalctl@ceph.mds.b.smithi081.stdout:Jul 14 16:40:41 smithi081 bash[161887]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-b 2023-07-14T16:40:41.553 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.b.service' 2023-07-14T16:40:41.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:41.639 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-14T16:40:41.639 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-14T16:40:41.639 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.e 2023-07-14T16:40:41.894 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:40:41 smithi081 systemd[1]: Stopping Ceph mds.e for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:42.644 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:40:42 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:40:42.227+0000 7f4ba41c6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:42.644 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:40:42 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e[152539]: 2023-07-14T16:40:42.227+0000 7f4ba41c6700 -1 mds.e *** got signal Terminated *** 2023-07-14T16:40:45.394 INFO:journalctl@ceph.mds.e.smithi081.stdout:Jul 14 16:40:45 smithi081 bash[162033]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-e 2023-07-14T16:40:45.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.e.service' 2023-07-14T16:40:45.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:45.763 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-14T16:40:45.763 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-14T16:40:45.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.h 2023-07-14T16:40:46.144 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:40:45 smithi081 systemd[1]: Stopping Ceph mds.h for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:46.644 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:40:46 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-h[152882]: 2023-07-14T16:40:46.333+0000 7f6f13d75700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:46.644 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:40:46 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-h[152882]: 2023-07-14T16:40:46.333+0000 7f6f13d75700 -1 mds.h *** got signal Terminated *** 2023-07-14T16:40:49.144 INFO:journalctl@ceph.mds.h.smithi081.stdout:Jul 14 16:40:48 smithi081 bash[162179]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-h 2023-07-14T16:40:49.236 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.h.service' 2023-07-14T16:40:49.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:49.317 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-14T16:40:49.317 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-14T16:40:49.318 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.k 2023-07-14T16:40:49.644 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:40:49 smithi081 systemd[1]: Stopping Ceph mds.k for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:50.144 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:40:49 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-k[152185]: 2023-07-14T16:40:49.881+0000 7f046206a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:50.144 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:40:49 smithi081 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-k[152185]: 2023-07-14T16:40:49.881+0000 7f046206a700 -1 mds.k *** got signal Terminated *** 2023-07-14T16:40:53.644 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:40:53 smithi081 bash[162324]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-k 2023-07-14T16:40:53.887 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.k.service' 2023-07-14T16:40:53.948 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:40:53 smithi081 systemd[1]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.k.service: Succeeded. 2023-07-14T16:40:53.948 INFO:journalctl@ceph.mds.k.smithi081.stdout:Jul 14 16:40:53 smithi081 systemd[1]: Stopped Ceph mds.k for 1356e48a-2260-11ee-9b33-001a4aab830c. 2023-07-14T16:40:53.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:53.969 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-14T16:40:53.970 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-14T16:40:53.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.c 2023-07-14T16:40:54.518 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:40:54 smithi164 systemd[1]: Stopping Ceph mds.c for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:55.017 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:40:54 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-c[123556]: 2023-07-14T16:40:54.618+0000 7f48cc5ea700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:55.017 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:40:54 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-c[123556]: 2023-07-14T16:40:54.618+0000 7f48cc5ea700 -1 mds.c *** got signal Terminated *** 2023-07-14T16:40:57.767 INFO:journalctl@ceph.mds.c.smithi164.stdout:Jul 14 16:40:57 smithi164 bash[133144]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-c 2023-07-14T16:40:57.872 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.c.service' 2023-07-14T16:40:57.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:40:57.960 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-14T16:40:57.960 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-14T16:40:57.961 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.f 2023-07-14T16:40:58.267 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:40:58 smithi164 systemd[1]: Stopping Ceph mds.f for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:40:58.767 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:40:58 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-f[123211]: 2023-07-14T16:40:58.484+0000 7f6615de9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:40:58.767 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:40:58 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-f[123211]: 2023-07-14T16:40:58.484+0000 7f6615de9700 -1 mds.f *** got signal Terminated *** 2023-07-14T16:41:02.767 INFO:journalctl@ceph.mds.f.smithi164.stdout:Jul 14 16:41:02 smithi164 bash[133282]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-f 2023-07-14T16:41:02.810 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.f.service' 2023-07-14T16:41:02.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:41:02.893 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-14T16:41:02.893 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-14T16:41:02.894 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.i 2023-07-14T16:41:03.267 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:41:03 smithi164 systemd[1]: Stopping Ceph mds.i for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:41:03.767 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:41:03 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-i[123904]: 2023-07-14T16:41:03.458+0000 7f21aa5a8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:41:03.767 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:41:03 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-i[123904]: 2023-07-14T16:41:03.458+0000 7f21aa5a8700 -1 mds.i *** got signal Terminated *** 2023-07-14T16:41:08.514 INFO:journalctl@ceph.mds.i.smithi164.stdout:Jul 14 16:41:08 smithi164 bash[133425]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-i 2023-07-14T16:41:08.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.i.service' 2023-07-14T16:41:08.622 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:41:08.623 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-14T16:41:08.623 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-14T16:41:08.624 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.l 2023-07-14T16:41:09.017 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:41:08 smithi164 systemd[1]: Stopping Ceph mds.l for 1356e48a-2260-11ee-9b33-001a4aab830c... 2023-07-14T16:41:09.517 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:41:09 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-l[122861]: 2023-07-14T16:41:09.140+0000 7f956b3c7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T16:41:09.517 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:41:09 smithi164 ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-l[122861]: 2023-07-14T16:41:09.140+0000 7f956b3c7700 -1 mds.l *** got signal Terminated *** 2023-07-14T16:41:14.078 INFO:journalctl@ceph.mds.l.smithi164.stdout:Jul 14 16:41:13 smithi164 bash[133572]: ceph-1356e48a-2260-11ee-9b33-001a4aab830c-mds-l 2023-07-14T16:41:14.105 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1356e48a-2260-11ee-9b33-001a4aab830c@mds.l.service' 2023-07-14T16:41:14.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T16:41:14.188 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-14T16:41:14.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1356e48a-2260-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T16:41:17.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1356e48a-2260-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T16:41:21.615 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1356e48a-2260-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T16:41:25.269 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:41:25.350 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:41:25.428 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T16:41:25.501 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-14T16:41:25.503 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi062/crash 2023-07-14T16:41:25.504 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T16:41:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T16:41:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2023-07-14T16:41:25.605 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi081/crash 2023-07-14T16:41:25.606 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T16:41:25.696 INFO:teuthology.orchestra.run.smithi081.stderr:tar: /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T16:41:25.696 INFO:teuthology.orchestra.run.smithi081.stderr:tar: Error is not recoverable: exiting now 2023-07-14T16:41:25.699 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi164/crash 2023-07-14T16:41:25.700 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T16:41:25.829 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T16:41:25.829 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2023-07-14T16:41:25.831 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-14T16:41:25.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-14T16:41:25.918 INFO:teuthology.orchestra.run.smithi062.stderr:grep: /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph.log: No such file or directory 2023-07-14T16:41:25.923 INFO:tasks.cephadm:Compressing logs... 2023-07-14T16:41:25.923 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T16:41:25.928 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T16:41:25.932 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T16:41:26.000 INFO:teuthology.orchestra.run.smithi081.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2023-07-14T16:41:26.000 INFO:teuthology.orchestra.run.smithi081.stderr:-5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T16:41:26.000 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.10.log 2023-07-14T16:41:26.001 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T16:41:26.001 INFO:teuthology.orchestra.run.smithi164.stderr:: No such file or directory 2023-07-14T16:41:26.002 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.5.log 2023-07-14T16:41:26.002 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.h.log 2023-07-14T16:41:26.002 INFO:teuthology.orchestra.run.smithi081.stderr: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T16:41:26.003 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.e.log 2023-07-14T16:41:26.003 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.2.log 2023-07-14T16:41:26.004 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.5.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T16:41:26.004 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.11.log 2023-07-14T16:41:26.004 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.4.log 2023-07-14T16:41:26.005 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.1.log 2023-07-14T16:41:26.005 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.z.log 2023-07-14T16:41:26.007 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.8.log 2023-07-14T16:41:26.007 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.k.log 2023-07-14T16:41:26.008 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.f.log 2023-07-14T16:41:26.009 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T16:41:26.010 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.c.log 2023-07-14T16:41:26.010 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.7.log 2023-07-14T16:41:26.011 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T16:41:26.011 INFO:teuthology.orchestra.run.smithi062.stderr:: No such file or directory 2023-07-14T16:41:26.011 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.0.log 2023-07-14T16:41:26.012 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.i.log 2023-07-14T16:41:26.012 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.b.log 2023-07-14T16:41:26.013 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.g.log 2023-07-14T16:41:26.013 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.a.log 2023-07-14T16:41:26.014 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.6.log 2023-07-14T16:41:26.015 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.3.log 2023-07-14T16:41:26.017 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.d.log 2023-07-14T16:41:26.018 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.x.log 2023-07-14T16:41:26.019 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.9.log 2023-07-14T16:41:26.020 INFO:teuthology.orchestra.run.smithi062.stderr: 94.6% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.g.log.gz 2023-07-14T16:41:26.020 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.c.log: /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.i.log: 93.5% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.z.log.gz 2023-07-14T16:41:26.020 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.c.log 2023-07-14T16:41:26.021 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.c.log: 94.6% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.c.log.gz 2023-07-14T16:41:26.021 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.x.log: 94.5% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.a.log.gz 2023-07-14T16:41:26.021 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T16:41:26.022 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T16:41:26.022 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.b.log 2023-07-14T16:41:26.022 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.b.log: 94.5% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.k.log.gz 2023-07-14T16:41:26.022 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.7.log: 91.4% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T16:41:26.023 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.j.log 2023-07-14T16:41:26.023 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.a.log 2023-07-14T16:41:26.024 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2023-07-14T16:41:26.024 INFO:teuthology.orchestra.run.smithi062.stderr: 94.6% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.d.log.gz 2023-07-14T16:41:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.a.log: /var/log/ceph/ceph-client.0.log: 94.6% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.j.log.gz 2023-07-14T16:41:26.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.l.log 2023-07-14T16:41:26.032 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.y.log 2023-07-14T16:41:26.033 INFO:teuthology.orchestra.run.smithi164.stderr: 91.4% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T16:41:26.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.y.log: 93.4% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.y.log.gz 2023-07-14T16:41:26.232 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.l.log.gz 2023-07-14T16:41:26.290 INFO:teuthology.orchestra.run.smithi164.stderr: 94.4% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.f.log.gz 2023-07-14T16:41:26.447 INFO:teuthology.orchestra.run.smithi081.stderr: 94.4% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.h.log.gz 2023-07-14T16:41:26.517 INFO:teuthology.orchestra.run.smithi062.stderr: 90.1% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mgr.x.log.gz 2023-07-14T16:41:26.694 INFO:teuthology.orchestra.run.smithi164.stderr: 91.9% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.c.log.gz 2023-07-14T16:41:26.732 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.b.log.gz 2023-07-14T16:41:27.111 INFO:teuthology.orchestra.run.smithi062.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T16:41:27.261 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.e.log.gz 2023-07-14T16:41:28.125 INFO:teuthology.orchestra.run.smithi062.stderr: 91.5% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T16:41:28.220 INFO:teuthology.orchestra.run.smithi164.stderr: 93.5% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.i.log.gz 2023-07-14T16:41:30.123 INFO:teuthology.orchestra.run.smithi081.stderr: 93.5% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mds.b.log.gz 2023-07-14T16:41:30.594 INFO:teuthology.orchestra.run.smithi062.stderr: 90.3% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-mon.a.log.gz 2023-07-14T16:42:05.012 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.8.log.gz 2023-07-14T16:42:07.293 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.2.log.gz 2023-07-14T16:42:08.612 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.4.log.gz 2023-07-14T16:42:09.763 INFO:teuthology.orchestra.run.smithi164.stderr: 91.0% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.11.log.gz 2023-07-14T16:42:09.938 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.7.log.gz 2023-07-14T16:42:11.083 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.10.log.gz 2023-07-14T16:42:14.052 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.1.log.gz 2023-07-14T16:42:14.056 INFO:teuthology.orchestra.run.smithi081.stderr: 2023-07-14T16:42:14.057 INFO:teuthology.orchestra.run.smithi081.stderr:real 0m48.086s 2023-07-14T16:42:14.057 INFO:teuthology.orchestra.run.smithi081.stderr:user 2m52.185s 2023-07-14T16:42:14.057 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m13.609s 2023-07-14T16:42:14.602 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.5.log.gz 2023-07-14T16:42:14.605 INFO:teuthology.orchestra.run.smithi164.stderr: 2023-07-14T16:42:14.605 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m48.633s 2023-07-14T16:42:14.605 INFO:teuthology.orchestra.run.smithi164.stderr:user 2m42.435s 2023-07-14T16:42:14.606 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m13.007s 2023-07-14T16:42:24.645 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.9.log.gz 2023-07-14T16:42:33.193 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.6.log.gz 2023-07-14T16:42:54.719 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.0.log.gz 2023-07-14T16:42:55.746 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/1356e48a-2260-11ee-9b33-001a4aab830c/ceph-osd.3.log.gz 2023-07-14T16:45:17.191 INFO:teuthology.orchestra.run.smithi062.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2023-07-14T16:45:17.195 INFO:teuthology.orchestra.run.smithi062.stderr: 2023-07-14T16:45:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:real 3m51.216s 2023-07-14T16:45:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:user 5m31.518s 2023-07-14T16:45:17.195 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m40.445s 2023-07-14T16:45:17.196 INFO:tasks.cephadm:Archiving logs... 2023-07-14T16:45:17.197 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi062/log 2023-07-14T16:45:17.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T16:45:55.689 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi081/log 2023-07-14T16:45:55.690 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T16:46:19.686 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi164/log 2023-07-14T16:46:19.687 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T16:46:42.620 INFO:tasks.cephadm:Removing cluster... 2023-07-14T16:46:42.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1356e48a-2260-11ee-9b33-001a4aab830c --force 2023-07-14T16:46:44.652 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1356e48a-2260-11ee-9b33-001a4aab830c --force 2023-07-14T16:46:46.455 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1356e48a-2260-11ee-9b33-001a4aab830c --force 2023-07-14T16:46:48.243 INFO:tasks.cephadm:Removing cephadm ... 2023-07-14T16:46:48.244 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T16:46:48.294 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T16:46:48.343 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T16:46:48.387 INFO:tasks.cephadm:Teardown complete 2023-07-14T16:46:48.388 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-14T16:46:48.400 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-14T16:46:48.400 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-14T16:46:48.401 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T16:46:48.406 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T16:46:48.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T16:46:58.928 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-14T16:46:58.949 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-14T16:46:58.950 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T16:46:58.956 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T16:46:58.963 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T16:46:58.999 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-07-14T16:46:59.004 INFO:teuthology.orchestra.run.smithi081.stderr:bash: ntpq: command not found 2023-07-14T16:46:59.009 INFO:teuthology.orchestra.run.smithi164.stderr:bash: ntpq: command not found 2023-07-14T16:46:59.012 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T16:46:59.012 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2023-07-14T16:46:59.013 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 +172us[ +134us] +/- 82ms 2023-07-14T16:46:59.013 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 +2974us[+2935us] +/- 73ms 2023-07-14T16:46:59.013 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -2759us[-2798us] +/- 63ms 2023-07-14T16:46:59.013 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T16:46:59.017 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T16:46:59.018 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2023-07-14T16:46:59.018 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +174us[ +148us] +/- 82ms 2023-07-14T16:46:59.018 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 +3040us[+3015us] +/- 74ms 2023-07-14T16:46:59.018 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -2759us[-2785us] +/- 63ms 2023-07-14T16:46:59.018 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T16:46:59.038 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T16:46:59.039 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-07-14T16:46:59.039 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +162us[ +149us] +/- 82ms 2023-07-14T16:46:59.039 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 +2982us[+2969us] +/- 73ms 2023-07-14T16:46:59.039 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -2783us[-2796us] +/- 63ms 2023-07-14T16:46:59.039 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T16:46:59.040 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-14T16:46:59.054 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-14T16:46:59.055 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-14T16:46:59.075 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T16:46:59.340 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T16:46:59.515 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T16:46:59.692 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T16:46:59.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T16:46:59.772 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T16:46:59.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T16:46:59.848 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T16:46:59.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T16:46:59.925 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-14T16:46:59.937 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-14T16:46:59.949 INFO:teuthology.task.internal:Duration was 3616.229622 seconds 2023-07-14T16:46:59.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-14T16:46:59.973 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-14T16:46:59.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T16:46:59.983 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T16:46:59.987 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T16:47:00.100 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T16:47:00.106 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T16:47:00.133 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T16:47:00.488 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-14T16:47:00.489 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2023-07-14T16:47:00.490 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T16:47:00.550 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2023-07-14T16:47:00.551 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T16:47:00.610 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2023-07-14T16:47:00.611 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T16:47:00.670 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-14T16:47:00.670 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T16:47:00.675 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T16:47:00.680 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T16:47:00.792 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-14T16:47:00.793 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T16:47:00.799 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T16:47:00.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T16:47:00.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-14T16:47:00.981 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-14T16:47:00.981 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T16:47:01.052 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T16:47:01.121 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T16:47:01.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-14T16:47:01.203 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T16:47:01.208 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T16:47:01.212 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T16:47:01.275 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2023-07-14T16:47:01.280 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2023-07-14T16:47:01.281 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2023-07-14T16:47:01.329 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T16:47:01.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T16:47:01.372 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T16:47:01.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T16:47:01.415 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T16:47:01.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T16:47:01.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-14T16:47:01.473 INFO:teuthology.task.internal:Transferring archived files... 2023-07-14T16:47:01.474 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi062 2023-07-14T16:47:01.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T16:47:01.558 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi081 2023-07-14T16:47:01.558 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T16:47:01.636 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606/remote/smithi164 2023-07-14T16:47:01.637 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T16:47:01.715 INFO:teuthology.task.internal:Removing archive directory... 2023-07-14T16:47:01.716 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T16:47:01.720 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T16:47:01.725 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T16:47:01.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-14T16:47:01.781 INFO:teuthology.task.internal:Not uploading archives. 2023-07-14T16:47:01.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-14T16:47:01.792 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-14T16:47:01.793 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T16:47:01.804 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T16:47:01.809 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T16:47:01.843 INFO:teuthology.orchestra.run.smithi062.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 16:47 /home/ubuntu/cephtest 2023-07-14T16:47:01.847 INFO:teuthology.orchestra.run.smithi081.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 16:47 /home/ubuntu/cephtest 2023-07-14T16:47:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 16:47 /home/ubuntu/cephtest 2023-07-14T16:47:01.856 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-14T16:47:01.867 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-14T16:47:01.880 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-14T16:47:01.880 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T16:47:05.721 DEBUG:teuthology.nuke:shortname: smithi062 2023-07-14T16:47:05.722 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T16:47:05.744 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 15:31:12.468236', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T16:47:05.750 DEBUG:teuthology.nuke:shortname: smithi081 2023-07-14T16:47:05.750 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T16:47:05.774 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 15:31:12.469664', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T16:47:10.785 DEBUG:teuthology.nuke:shortname: smithi164 2023-07-14T16:47:10.786 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T16:47:10.881 INFO:teuthology.orchestra.console:Power off smithi062 2023-07-14T16:47:10.882 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T16:47:10.900 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/7337606', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 15:31:12.470699', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T16:47:10.911 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:10.935 INFO:teuthology.orchestra.console:Power off smithi081 2023-07-14T16:47:10.936 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T16:47:10.959 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:10.982 INFO:teuthology.orchestra.console:Power off smithi164 2023-07-14T16:47:10.982 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T16:47:14.949 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:14.995 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:20.062 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:20.108 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:25.176 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:25.221 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:25.388 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2023-07-14T16:47:26.019 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:30.333 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:30.547 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2023-07-14T16:47:31.033 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:36.148 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:41.264 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:46.376 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:51.492 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:47:56.606 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:48:01.722 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:48:06.834 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:48:11.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:48:12.159 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2023-07-14T16:48:12.265 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} duration: 3616.2296221256256 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default mon.a-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 mon.b-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 mon.c-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 owner: scheduled_rishabh@teuthology status: fail success: false 2023-07-14T16:48:12.266 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:48:17.351 INFO:teuthology.run:FAIL